Dynamic Optimization
Application of calculus of
variation
Lecture 33,34,35
Bolza problem
• Let the system (Plant) is described by
• 𝑋 𝑡 = 𝑓(𝑋 𝑡 , 𝑈 𝑡 , 𝑡)
nx1
nx1
nx1
---(1)
mx1
• And performance index or objective function is
• 𝐽 . = 𝑆 𝑋 𝑡 ,𝑡
Terminal Cost
𝑡=𝑡𝑓
+
𝑡𝑓
𝑉(𝑋
𝑡0
𝑡 , 𝑈 𝑡 , 𝑡) 𝑑𝑡
Integral Cost function
---(2)
“Bolza” problem…
• Objective: Find U(t) (control Law) that minimize performance index
given by equ (2) subjected to the dynamics (constraint) given by equ
(1). Such an optimal control problem is known as “Bolza” problem.
Solution using
Lagrangian function
Solution is Similar to two point boundary value problem
Necessary conditions:
• Using Lemma, necessary conditions are
•
•
𝜕𝐿 .
𝜕𝑋 𝑡
𝜕𝐿 .
𝜕𝑈 𝑡
−
∗
𝑑 𝜕𝐿 .
𝑑𝑡 𝜕𝑋 𝑡
= 0𝑛×1
---(3)
∗
= 0𝑚×1
---(4)
• Boundary condition
• 𝐿 . −
𝜕𝐿 .
𝜕𝑋 𝑡
𝑇
𝑋(𝑡)
∗
0, when 𝛿𝑡𝑓 is free
𝛿𝑡𝑓 +
𝑡=𝑡𝑓
𝜕𝐿 .
𝜕𝑋 𝑡
𝑇
∗ 𝑡=𝑡𝑓
0, when 𝛿𝑋𝑓 is free
𝛿𝑋𝑓 = 0
---(5)
• Where
𝜕𝑠 .
𝜕𝑋 𝑡
𝑇
𝜕𝑠 .
𝜕𝑡
• 𝐿 𝑋 𝑡 ,𝑈 𝑡 ,𝜆 𝑡 ,𝑡 = 𝐻 𝑋 𝑡 ,𝑈 𝑡 ,𝑡 +
𝑋 𝑡 +
−
𝜆𝑇 𝑋(𝑡)
• 𝐻 𝑋 𝑡 , 𝑈 𝑡 , 𝑡 = 𝑉 𝑋 𝑡 , 𝑈 𝑡 , 𝑡 + 𝜆𝑇 𝑡 𝑓 𝑋 𝑡 , 𝑈 𝑡 , 𝑡 known as
Hamiltonian function
• Equ (5) is the general boundary equation in terms of Lagrangian
function
• So equ (3), (4) & (5) need to be solved to obtain the optimal input
𝑈 ∗ (𝑡) and hence 𝑋 ∗ (𝑡)
Lecture 35
Hamiltonian Formulation
for solution of optimal
control problem
• Above Bolza problem can be solved by using Hamiltonian
formulation, which gives simple results as compared to Lagrangian
formulation.
Steps to solve Bolza problem using
Hamiltonian Formulation
• Write Hamiltonian function using
• 𝐻 𝑋 𝑡 , 𝑈 𝑡 , 𝑡 = 𝑉 𝑋 𝑡 , 𝑈 𝑡 , 𝑡 + 𝜆𝑇 𝑡 𝑓 𝑋 𝑡 , 𝑈 𝑡 , 𝑡
• Find
•
•
•
𝜕𝐻(.)
=
𝜕𝑋 𝑡 ∗
𝜕𝐻(.)
=
𝜕𝑈 𝑡 ∗
𝜕𝐻(.)
=
𝜕𝜆(𝑡) ∗
− 𝜆(𝑡)
0𝑚×1
𝑋(𝑡)
∗
---(6)
---(7)
∗
---(8)
Steps to solve Bolza problem using
Hamiltonian Formulation
• Use boundary conditions given by following equation
• 𝐻 . +
𝜕𝑠 .
𝑋(𝑡)
𝜕𝑡 ∗
𝑡=𝑡𝑓
+
𝜕𝑠(.)
𝜕𝑋(𝑡)
− 𝜆(𝑡)
𝑇
∗ 𝑡=𝑡𝑓
𝛿𝑋𝑓 = 0
---(9)
• Equ (9) is the general boundary equation in terms of Hamiltonian
function
• So equ (6), (7), (8) & (9) need to be solved to obtain the optimal input
𝑈 ∗ (𝑡) and hence 𝑋 ∗ (𝑡)
Solving boundary conditions
• For solving equ (5) & (9) we can have following situations
• Case 1: Fixed final time and fixed final state (𝛿𝑡𝑓 = 0 & 𝛿𝑋𝑓 = 0)
• => no boundary conditions, don’t consider equ (5) or equ (9)
• Case 2: Free final time and fixed final state (𝛿𝑡𝑓 ≠ 0 & 𝛿𝑋𝑓 = 0)
• Case 3: Fixed final time and free final state (𝛿𝑡𝑓 = 0 & 𝛿𝑋𝑓 ≠ 0)
• Case 4: Free final time and free final state (𝛿𝑡𝑓 ≠ 0 & 𝛿𝑋𝑓 ≠ 0)
Sufficient condition:
• In order to determine the nature of optimization (obtained value is
minimum or maximum) we need to check sufficient condition.
• 2nd variation of the functional
•
𝛻2𝐽
=
𝑡𝑓
𝑡0
𝛿𝑋(𝑡)
𝛿𝑈(𝑡)
𝜕2 𝐻 .
𝜕𝑋 2 𝑡
𝜕𝐻 .
𝜕𝑋 𝑡 𝜕𝑈 𝑡
𝜕𝐻 .
𝜕𝑋 𝑡 𝜕𝑈 𝑡
𝜕2 𝐻 .
𝜕𝑈 2 𝑡
Say P which is a (n+m)x(n+m) matrix
𝛿𝑋(𝑡)
𝑑𝑡
𝛿𝑈(𝑡)
∗
Sufficient condition…
• If 𝛻 2 𝐽 > 0 ⇒ 𝑃 > 0 (+𝑑𝑒𝑓𝑖𝑛𝑒𝑖𝑡𝑒)
• => functional value will be minimum
• =>J*(.) will be the minimum value along with the optimum trajectory x*(t)
• If 𝛻 2 𝐽 < 0 ⇒ 𝑃 < 0 (−𝑑𝑒𝑓𝑖𝑛𝑒𝑖𝑡𝑒)
• => functional value will be maximum
• =>J*(.) will be the maximum value along with the optimum trajectory x*(t)