Homework #2 Solution
ChE 391
1. For the system in slide 19 (Optimal Control file), determine the value of R that will keep |𝑢| ≤
1 for 𝑥1 (0) = 𝑥2 (0) = 1 and 𝑸 = 𝑰.
For this value of R, determine the time required to reach within some small tolerance (e.g., 1%)
of the origin. Compare with the minimum time trajectory on the phase plane.
Solution:
In this system,
𝐴=[
−1 0
1
1
], 𝐵 = [ ], 𝑄 = [
1 0
0
0
0
], 𝑅 is a scalar.
1
The Riccati Equation is
𝑷̇ + 𝑷𝑨 + 𝑨𝑇 𝑷 − 𝑷𝑩𝑹−1 𝑩𝑇 𝑷 + 𝑸 = 𝑶
As 𝑡𝑓 → ∞, 𝑷̇ = 0
𝑝11
Assume 𝑷 = [𝑝
12
𝑝12
𝑝22 ], then from the Riccati Equation, we have
2
−2𝑝11 + 2𝑝12 − 𝑅 −1 𝑝11
+1=0
−1
{ −𝑝12 + 𝑝22 − 𝑝11 𝑝12 𝑅 = 0
2
−𝑝21
+1=0
As 𝑷 is positive defined matrix, we can have the solution of 𝑝11 , 𝑝12 , 𝑝22 as
𝑝11 = √𝑅, 𝑝12 = √𝑅, 𝑝22 = 1 + √𝑅
The 𝑷 = [√𝑅
√𝑅
√𝑅 ], 𝐾 = 𝑅 −1 𝑩𝑇 𝑷 = [1⁄
√𝑅
1 + √𝑅
1⁄ ]
√𝑅
As 𝒙̇ = 𝑨𝒙 + 𝑩𝒖 = 𝑨𝒙 − 𝑩𝑲𝒙 = (𝑨 − 𝑩𝑲)𝒙
𝑨 − 𝑩𝑲 = [−1 − 1⁄√𝑅
1
− 1⁄√𝑅 ]
0
Then we can have the solution of 𝒙 as
𝑠 − √𝑅⁄𝑅
𝑋(𝑠) = [
𝑠(𝑠 + 1 + √𝑅⁄𝑅 ) + √𝑅⁄𝑅
As 𝒖 = −𝑲𝒙 =
−1
[1
√𝑅
1]𝒙
𝑠 + 2 + √𝑅⁄𝑅
𝑠(𝑠 + 1 + √𝑅⁄𝑅 ) + √𝑅⁄𝑅
𝑇
]
𝑈(𝑠) =
(−2⁄√𝑅 )(𝑠 + 1)
𝑠 2 + 𝑠(1 + √𝑅⁄𝑅 ) + √𝑅⁄𝑅
=
(−2⁄√𝑅 )(𝑠 + 1)
(𝑠 + 1)(𝑠 + 1⁄√𝑅 )
=
−2⁄√𝑅
𝑠 + 1⁄√𝑅
So 𝑢(𝑡) = (−2⁄√𝑅 ) (𝑒 −(1⁄√𝑅)𝑡 − 1) + 𝑢(0)
As 𝑢(0) =
−1
[1
√𝑅
1
1] [ ] = −2/√𝑅
1
𝑢(𝑡) = (−2⁄√𝑅 )𝑒 −(1⁄√𝑅)𝑡
For 𝑡 ≥ 0, −
2
√𝑅
≤ 𝑢(𝑡) < 0
In order to make sure that |𝑢| ≤ 1, we need to make
2
√𝑅
≤ 1 𝑅 ≥ 4
So 𝑅 = 4.
1
−2𝑡
−𝑡
When 𝑅 = 4, 𝑢(𝑡) = −𝑒 −(1⁄2)𝑡 , 𝑥 = [ 3𝑒 − 2𝑒 1 ]
−3𝑒 −𝑡 + 4𝑒 −2𝑡
then the time required to reach within 1% tolerance can be calculated or found from the plot.
𝑡 = 11.98 𝑠 ≈ 12 𝑠
Show this trajectory in phase plane.
Trajectory with optimal control
1.4
1.2
1
x2
0.8
0.6
0.4
0.2
0
-0.4
-0.2
0
0.2
0.4
x1
Minimum Time Trajectory
0.6
0.8
1
𝑡𝑓
min 𝑉 = ∫ 1𝑑𝑡
0
𝑥̇ 1 = −𝑥1 + 𝑢, 𝑥̇ 2 = 𝑥1
𝐻 = 𝐿 + 𝝀𝑇 𝒇 = 1 + 𝜆1 (−𝑥1 + 𝑢) + 𝜆2 (𝑥1 )
𝐻𝑢 = 𝜆1 (cannot solve for optimal 𝑢)
As |𝑢| ≤ 1, the minimum time trajectory can be carried out by switching 𝑢 between ±1.
As this is a 2nd order system, and the eigenvalues of 𝐴 are 0, -1 (not complex number), there’s
only one switch. So the minimum time trajectory may begin from 𝑢 = 1, and then switch to 𝑢 = −1, or
start from 𝑢 = −1 and switch to 𝑢 = 1.
𝑥̇ 1 = −𝑥1 + 𝑢
𝑥̇ 2 = 𝑥1
∴
𝑑𝑥2
𝑥1
=
𝑑𝑥1 −𝑥1 + 𝑢
If 𝑢 = 1,
𝑥1 = 1 + 𝑘1 𝑒 −𝑡 , 𝑥2 = 𝑡 − 𝑘1 𝑒 −𝑡 + 𝑘2
𝑥2 = −𝑥1 − ln(1 − 𝑥1 ) + 𝑐1 for 𝑥1 ≠ 1
If 𝑢 = −1,
𝑥1 = −1 + 𝑘3 𝑒 −𝑡 , 𝑥2 = −𝑡 − 𝑘3 𝑒 −𝑡 + 𝑘4
𝑥2 = −𝑥1 + ln(𝑥1 + 1) + 𝑐2 for 𝑥1 ≠ −1
By the phase analysis, we can have the minimum time trajectory as start from u=-1 and end with
u=1.
u=+1
1
0.5
0
-0.5
-1
-1
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
1
0.2
0.4
0.6
0.8
1
u=-1
1
0.5
0
-0.5
-1
-1
-0.8
-0.6
-0.4
-0.2
0
The switching time can be determined by the point of intersection of the two trajectories.
𝑢 = −1 should pass the initial point (1,1) at t=0
𝑥2 = −𝑥1 + ln(𝑥1 + 1) + 2 − ln 2
𝑥1 = −1 + 2𝑒 −𝑡 , 𝑥2 = −𝑡 − 2𝑒 −𝑡 + 3
𝑢 = 1 should pass the final point (0,0) 
𝑥2 = −𝑥1 − ln(1 − 𝑥1 )
The intersection point of u=1 and u=-1 in phase plane can be calculated as the solution of functions
𝑥 = −𝑥1 + ln(𝑥1 + 1) + 2 − ln 2
{ 2
𝑥2 = −𝑥1 − ln(1 − 𝑥1 )
There are two solutions:
Phase Plane
6
u=+1
u=-1
4
x
2
2
0
-2
-4
-6
-1
-0.8
-0.6
-0.4
-0.2
0
x1
0.2
0.4
0.6
0.8
1
As direction of the two trajectories has been shown in previous figure, the only solution is the
trajectory start from u=-1, then move forward to the intersection near x=-0.8, then switch to u=0
and approach to the end point.
The intersection point can be calculated by “fzero” function in MATLAB.
Minimum Time Trajectory
1.4
1.2
1
x2
0.8
0.6
0.4
0.2
0
-1
-0.8
-0.6
-0.4
-0.2
0
x1
0.2
0.4
0.6
0.8
1
Time required for the minimum time trajectory is 3.23s<12s.
2. For the same system in problem 1 (R=1), integrate the dynamic Riccati equation with MATLAB
and determine when the value of P (t) converges to within 1% of its steady state value.
Plot of P(t)
2.5
2
P(t)
1.5
1
0.5
0
0
1
2
3
4
5
t
6
7
8
9
Then it’s easy to see that after t=4.25s, the value of P (t) converges to within 1% of its steady
state value.
Appendix: (MATLAB file)
Problem1:
% Modern Control System, HW2, Problem 1
clear all;
close all;
clc
%Parameters of the system and optimization target function
A=[-1 0;1 0];
B=[1;0];
Q=eye(2);
R=4;
% Calculate the steady state solution of Riccati Equation
P=care(A,B,Q,R);
% system with optimal control
% xdot=(A-B*K)*x=(A-B*(R^-1)*B'*P)*x, u=-Kx;
K=(R^-1)*B'*P;
A_new=A-B*K;
C_new=-K;
sys=ss(A_new,[],C_new,[]);
%response to the initial condition
x0=[1 1]';
[Y,T,X]=initial(sys,x0);
10
figure;
plot(X(:,1),X(:,2));title('Trajectory with optimal control');
xlabel('x1');
ylabel('x2');
% Minimum time trajectory
% plot the trajectories of u=1 and u=-1
v1=[-1:0.1:1];
v2=v1;
[x1,x2] = meshgrid(v1,v2);
x1dotm=-x1+1; x2dotm=x1;%u=1
x1dotn=-x1-1; x2dotn=x1;%u=-1
figure;
subplot(2,1,1);quiver(x1,x2,x1dotm,x2dotm);hold on; axis tight;title('u=+1');
subplot(2,1,2);quiver(x1,x2,x1dotn,x2dotn);hold on; axis tight;title('u=-1');
% plot u=1 (pass final point) and u=-1 (pass initial point)
x1=-1:0.001:1;
for i=1:length(x1)
x21=-x1-log(1-x1);%u=1 pass final point;
x22=-x1+log(1+x1)+2-log(2);%u=-1 pass final point;
end;
plot(x1,x21,x1,x22);
title('Phase Plane');
xlabel('x_1');
ylabel('x_2');
legend('u=+1','u=-1');
% calculate the intersection point near -0.8
fun_1=@(x)log(1-x)+log(1+x)+2-log(2);
x1_in=fzero(fun_1,-0.8);
x2_in=-x1_in-log(1-x1_in);
%simulate the minimum time trajectory
t=0:0.0001:5;
k1=2;
k2=3;
t_in=-log((x1_in+1)/k1);
u=zeros(length(t),1);
x11=zeros(length(t),1);
x21=zeros(length(t),1);
k3=(x1_in-1)/exp(-t_in);
k4=x2_in-t_in+k3*exp(-t_in);
for i=1:length(t)
if t(i)<t_in
x11(i)=k1*exp(-t(i))-1;
x21(i)=k2-k1*exp(-t(i))-t(i);
u(i)=-1;
else
x11(i)=k3*exp(-t(i))+1;
x21(i)=k4-k3*exp(-t(i))+t(i);
u(i)=1;
if x11(i)>0
break;
else
end;
end;
end;
t(i)%time required for the minimum time trajectory
plot(x11(1:i,1),x21(1:i,1));title('Minimum Time Trajectory');
xlabel('x1');ylabel('x2');
Problem2:
function dXdt = mRiccati(t, X, A, B, Q)
X = reshape(X, size(A)); %Convert from "n^2"-by-1 to "n"-by-"n"
dXdt =( A.'*X + X*A - X*B*B.'*X + Q); %Determine derivative
dXdt = dXdt(:);
%ode function defined
A = [-1 0; 1 0];
B = [1; 0];
Q = [1 0; 0 1];
X0 = [0; 0; 0; 0];
[T X] = ode45(@mRiccati, [0 10], X0, [], A, B, Q);
plot(T,X(:,1),T,X(:,2),T,X(:,3),T,X(:,4));
title('Plot of P(t)');
xlabel('t');
ylabel('P(t)');