12.3
679
Optimal Control Theory
which minimizes the functional, called the performance index,
J =
•
T
0
f (0x , u, t)
dt
(12.21)
where
ffix1ffi.
ffi ffi
,x
2
x = ffi . ffi
ffi . ffi/
ffi . ffi
- xn ffi
0
is called the state vector, t the time parameter, T the terminal time, and f
of x, u, and t. The state variables x
dxi
dt
or
= f i(x
m; t),
1,
x
i
2,
and the control variables u
...,x
n;
u
1,
u
2,
i
0
is a function
are related as
i=1 2 . .,n
.,,
...,u
x8 = f(x, u, t)
(12.22)
In many problems, the system is linear and Eq. (12.22) can be stated as
x8 = [A]x + [B]u
(12.23)
where [A] is an n × n matrix and [B] is an n × m matrix. Further, while finding the
control vector u, the state vector x is to be transferred from a known initial vector x0
at t = 0 to a terminal vector
at t = T , where some (or all or none) of the
xT
state
variables are specified.
12.3.1
Necessary Conditions for Optimal Control
To derive the necessary conditions for the optimal control, we consider the following
simple problem:
Find u which minimizes J =
•
T
0
f
0(x,
u, t) dt
(12.24)
subject to
x8 = f (x, u, t)
(12.25)
with the boundary condition x(0) = k 1. To solve this optimal control problem, we
introduce a Lagrange multiplier  and define an augmented functional J _ as
J_ =
•
T
0
{f
0(x,
u, t) + [f (x, u, t) Š x8]} dt
(12.26)
Since the integrand
F =f
0
+  (f Š 8x)
(12.27)
680
Optimal Control and Optimality Criteria Methods
is a function of the two variables x and u, we can write the Euler-Lagrange equations
[with u
1
= x, u
1_=
x/t = x8, u
F
x
F
•
F
x8
dt
•
d
Š
u
•
d
Š
u/t = u8
2_=
= u and u
2
F
•
u8
dt
in Eq. (12.10)] as
=0
(12.28)
=0
(12.29)
In view of relation (12.27), Eqs. (12.28) and (12.29) can be expressed as
f0
f
+
x
+ 8 = 0
x
f
+
u
f
u
(12.30)
(12.31)
=0
A new functional H , called the Hamiltonian, is defined as
H =f
+ f
(12.32)
= 8
(12.33)
=0
(12.34)
0
and Eqs. (12.30) and (12.31) can be rewritten as
Š
H
x
H
u
Equations (12.33) and (12.34) represent two first-order differential equations. The integration of these equations leads to two constants whose values can be found from the
known boundary conditions of the problem. If two boundary conditions are specified
as x(0) = k 1 and x(T ) = k 2, the two integration constants can be evaluated without
any difficulty. On the other hand, if only one boundary condition is specified as, say,
x(0) = k 1, the free-end condition is used as F/x8 = 0 or  = 0 at t = T .
Example 12.4 Find the optimal control u that makes the functional
J =
1
•
(x 2 + u 2) dt
(E 1)
0
stationary with
(E 2)
x8 = u
and x(0) = 1. Note that the value of x is not specified at t = 1.
SOLUTION
The Hamiltonian can be expressed as
H =f
0
+ u = x
2
2
+ u + u
(E )
3
and Eqs. (12.33) and (12.34) give
Š 2x = 8
2u +  = 0
(E 4)
(E 5)
12.3
Differentiation of Eq. (E
Equations (E
4)
5)
Optimal Control Theory
681
leads to
2u8 + 8 =
0
(E 6)
u8 = x
(E 7)
and (E 6) yield
Since x8 = u [Eq. (E 2)], we obtain
x¨ = u8
=x
that is,
x¨ Š x = 0
The solution of Eq. (E
8)
(E 8)
is given by
(E 9)
x(t) = c 1 sinh t + c 2 cosh t
where c 1 and c 2 are constants. By using the initial condition x(0) = 1, we obtain c 2 = 1.
Since x is not fixed at the terminal point t = T = 1, we use the condition  = 0 at
t = 1 in Eq. (E 5) and obtain u(t = 1) = 0. But
(E10)
u = x8 = c 1 cosh t + sinh t
Thus
u(1) = 0 = c 1 cosh 1 + sinh 1
or
c1 =
Š sinh 1
(E11)
cosh 1
and hence the optimal control is
u(t) =
=
Š sinh 1
cosh 1
• cosh t + sinh t
Š sinh 1 · cosh t + cosh 1 · sinh t
cosh 1
=
Š sinh (1 Š t)
cosh 1
(E 12)
The corresponding state trajectory is given by
x(t) = u8 =
12.3.2
cosh (1 Š t)
cosh 1
(E 13)
Necessary Conditions for a General Problem
We shall now consider the basic optimal control problem stated earlier:
Find the optimal control vector u that minimizes
J =
T
•
0
f (0x , u,
t) dt
(12.35)
subject to
xi8
= f i(x, u, t), i = 1, 2, . . . , n
(12.36)
682
Optimal Control and Optimality Criteria Methods
Now we introduce a Lagrange multiplier p i, also known as the adjoint variable, for
the ith constraint equation in (12.36) and form an augmented functional J _ as
J_ =
T
•
ff
n
f0 + 1
0
i=1
fi
p i(f
)
i Š xi8
dt
(12.37)
The Hamiltonian functional, H , is defined as
n
H =f 0+1
p ifi
(12.38)
i=1
such that
T
•
J_ =
2
0
H Š 1n p ixi8
3
(12.39)
dt
i=1
Since the integrand
F = H Š 1n p ixi8
(12.40)
i=1
depends on x, u, and t, there are n + m dependent variables (x and u) and hence the
Euler-Lagrange equations become
F
xi
F
uj
Š
Š
d
• F •
dt
xi8
d
dt
•
F
•
uj8
= 0,
i = 1, 2, . .
.,n
(12.41)
= 0,
j = 1, 2, . .
.,m
(12.42)
In view of relation (12.40), Eqs. (12.41) and (12.42) can be rewritten as
Š
H
xi
H
ui
= p ,i
= 0,
i = 1, 2, . .
j = 1, 2, . .
.,n
(12.43)
.,m
(12.44)
Equations (12.43) are knowns as adjoint equations.
The optimum solutions for x, u, and p can be obtained by solving Eqs. (12.36),
(12.43), and (12.44). There are totally 2n + m equations with nx i's, np i's, and mu j's
as unknowns. If we know the initial conditions x i(0), i = 1, 2, . . . , n, and the terminal
conditions x j(T ), j = 1, 2, . . . , l, with l < n, we will have the terminal values of the
remaining variables, namely x j(T ), j = l + 1, l + 2, . . . , n, free. Hence we will have
to use the free end conditions
p j(T ) = 0,
j = l + 1, l + 2, . . . , n
Equations (12.45) are called the transversality conditions.
(12.45)