Spectral Properties of LinearQuadratic Regulators !
Robert Stengel!
Optimal Control and Estimation MAE 546 !
Princeton University, 2015"
!! Stability margins of single-input/singleoutput (SISO) systems"
!! Characterizations of frequency response"
!! Loop transfer function"
!! Return difference function"
!! Kalman inequality"
!! Stability margins of scalar linearquadratic regulators"
Copyright 2015 by Robert Stengel. All rights reserved. For educational use only.!
http://www.princeton.edu/~stengel/MAE546.html!
http://www.princeton.edu/~stengel/OptConEst.html
!
1!
Return Difference Function
and Closed-Loop Roots!
Single-Input/Single-Output
Control Systems!
2!
SISO Transfer Function and Return
Difference Function"
•! Unit feedback
control law"
•! Block diagram
algebra"
y ( s ) = A ( s ) "# yC ( s ) ! y ( s ) $%
"#1 + A ( s ) $% y ( s ) = A ( s ) yC ( s )
y(s)
A(s)
=
: Closed - Loop Transfer Function
yC ( s ) 1 + A ( s )
A ( s ) : Open - Loop Transfer Function
!"1 + A ( s ) #$ : Return Difference Function
3!
Return Difference
Function and
Root Locus"
A(s) =
1+ A ( s ) = 1+
kn ( s )
d (s)
kn ( s )
= 0 defines locus of roots
d (s)
d ( s ) + kn ( s ) = 0 defines locus of closed-loop roots
4!
Return Difference
Example"
A(s) =
kn ( s )
k (s ! z)
1.25 ( s + 40 )
= 2
=
d ( s ) s + 2"# n s + # n 2 s 2 + 2 ( 0.3) ( 7 ) s + ( 7 )2
1+ A ( s ) = 1+
k (s ! z)
1.25 ( s + 40 )
=
1+
2 = 0
2
s 2 + 2"# n s + # n 2
s + 2 ( 0.3) ( 7 ) s + ( 7 )
! s 2 + 2 ( 0.3) ( 7 ) s + ( 7 )2 # + 1.25 ( s + 40 ) = 0
"
$
5!
Closed-Loop Transfer
Function Example"
$
!
1.25 ( s + 40 )
# 2
2 &
A(s)
# s + 2 ( 0.3) ( 7 ) s + ( 7 ) &%
= "
1+ A ( s )
$
!
1.25 ( s + 40 )
1+ # 2
2 &
#" s + 2 ( 0.3) ( 7 ) s + ( 7 ) &%
A(s)
1.25 ( s + 40 )
= 2
1+ A ( s ) ! s + 2 ( 0.3) ( 7 ) s + ( 7 )2 # + 1.25 ( s + 40 )
"
$
1.25 ( s + 40 )
= 2
2
s + !" 2 ( 0.3) ( 7 ) + 1.25 #$ s + !"( 7 ) + 1.25 ( 40 ) #$
kn ( s )
=
d ( s ) + kn ( s )
6!
SISO Frequency-Response Plots!
7!
Open-Loop Frequency Response:
Bode Plot"
A ( j! ) =
( j! )
2
K ( j! " z )
+ 2#! n j! + ! n2
=
( j! )
2
1.25 ( j! + 40 )
+ 2 ( 0.3) ( 7 ) j! + ( 7 )
2
Two plots
• 20 log A ( j! ) vs. log !
• ! "# A ( j! ) $% vs. log !
•! Gain Margin"
–! Referenced to 0 dB line"
–! Evaluated where phase angle =
–180°"
•! Phase Margin"
–! Referenced to –180°"
–! Evaluated where amplitude ratio
= 0 dB!
8!
Open-Loop Frequency Response:
Nyquist Plot"
Re ( A ( j! )) vs. Imag ( A ( j! ))
•!
•!
•!
Single plot; input
frequency not shown
explicitly"
Gain and Phase Margins
referenced to (–1) point"
GM and PM represented
as length and angle"
Only positive frequencies need be considered!
9!
Open-Loop Frequency Response:
Nichols Chart"
20 log10 "# A ( j! ) $% vs. ! "# A ( j! ) $%
!! Single plot"
!! Gain and Phase Margins
shown directly"
10!
Algebraic Riccati Equation in
the Frequency Domain!
11!
Linear-Quadratic Control"
•! Quadratic cost function for infinite final time"
J=
(
(
$
"
$"
1 "
T
T
$ & Q 0 ' & !x(t) ' dt = 1 " !xT (t)Q!x(t) + !u(t)R!uT (t) $ dt
!x
(t)
!u
(t)
%
% & 0 R ' & !u(t) '
2 t)o #
2 t)o #
#
%#
%
•! Linear, time-invariant
dynamic system"
•! Constant-gain optimal
control law"
!!x(t) = F!x(t) + G!u(t)
!u ( t ) = "R "1GT P!x ( t )
= "C!x ( t )
Algebraic Riccati equation"
0 = !Q ! FT P ! PF + PGR !1GT P
Q = !FT P ! PF + CT RC
12!
Frequency Characteristics of the
Algebraic Riccati Equation"
Add and subtract sP such that"
(
)
P ( !F ) + !FT P + CT RC = Q
(
)
P ( sI n ! F ) + !sI n ! FT P + CT RC = Q
State !
Characteristic!
Matrix!
Adjoint!
Characteristic!
Matrix!
13!
Frequency Characteristics of the
Algebraic Riccati Equation"
(
)
P ( sI n ! F ) + !sI n ! FT P + CT RC = Q
Pre-multiply each term by"
(
GT !sI n ! FT
)
!1
Post-multiply each term by"
( sIn ! F )!1 G
GT ( !sI n ! FT ) PG + GT P ( sI n ! F ) G + GT ( !sI n ! FT ) CT RC ( sI n ! F ) G
!1
!1
!1
= GT ( !sI n ! FT ) Q ( sI n ! F ) G
!1
!1
!1
14!
Frequency Characteristics of the
Algebraic Riccati Equation"
Substitute with the control gain matrix"
C = R !1GT P
GT P = RC
GT ( !sI n ! FT ) CT R + RC ( sI n ! F ) G + GT ( !sI n ! FT ) CT RC ( sI n ! F ) G
!1
!1
!1
!1
= GT ( !sI n ! FT ) Q ( sI n ! F ) G
!1
!1
Add R to both sides"
R + GT ( !sI n ! FT ) CT R + RC ( sI n ! F ) G + GT ( !sI n ! FT ) CT RC ( sI n ! F ) G
!1
!1
!1
!1
= R + GT ( !sI n ! FT ) Q ( sI n ! F ) G
!1
!1
15!
Frequency Characteristics of the
Algebraic Riccati Equation"
R + GT ( !sI n ! FT ) CT R + RC ( sI n ! F ) G + GT ( !sI n ! FT ) CT RC ( sI n ! F ) G
!1
!1
!1
!1
= R + GT ( !sI n ! FT ) Q ( sI n ! F ) G
!1
!1
The left side can be factored as*"
" I + GT ( !sI ! FT )!1 CT $ R " I + C ( sI ! F )!1 G $
n
n
%
# m
% # m
= R + GT ( !sI n ! FT ) Q ( sI n ! F ) G
!1
!1
* Verify by multiplying the factored form!
16!
Modal Expression of
Algebraic Riccati Equation"
Define the loop transfer function matrix"
A ( s ) = C ( sI n ! F ) G
!1
17!
Modal Expression of
Algebraic Riccati Equation"
Recall the cost function transfer matrix"
Y ( s ) = H ( sI n ! F ) G, where Q = HT H
!1
Laplace transform of algebraic Riccati equation becomes"
T
"# I m + A ( !s ) $% R "# I m + A ( s ) $% = R + Y T ( !s ) Y ( s )
18!
Algebraic Riccati Equation"
T
"# I m + A ( !s ) $% R "# I m + A ( s ) $% = R + Y T ( !s ) Y ( s )
•! Cost function transfer matrix, Y(s)"
–! Reflects control-induced state
variations in the cost function"
–! Governs closed-loop modal properties
as R becomes small"
–! Does not depend on R or P"
Y ( s ) = H ( sI n ! F ) G
!1
!ui = "C ( si I n " F ) G!ui
•! Loop transfer function matrix, A(s)"
–! Defines the modal control vector
when s = si"
"1
= "A ( si ) !ui , i = 1, n
19!
LQ Regulator Portrayed as a
Unit-Feedback System"
A ( s ) = C ( sI n ! F ) G
!1
I m + A ( s ) = I m + C ( sI n ! F ) G : Return Difference Matrix
!1
20!
Determinant of Return
Difference Matrix Defines
Closed-Loop Eigenvalues"
I m + A ( s ) = I m + C ( sI n ! F ) G
!1
= Im +
CAdj ( sI n ! F ) G
CAdj ( sI n ! F ) G
= Im +
sI n ! F
" OL ( s )
Characteristic Equation!
! OL ( s ) I m + A ( s ) = ! OL ( s ) I m +
CAdj ( sI n " F ) G
! OL ( s )
= ! OL ( s ) I m + CAdj ( sI n " F ) G = ! CL ( s ) = 0
21!
Stability Margins and
Robustness of Scalar
LQ Regulators!
22!
Scalar Case"
Multivariable algebraic Riccati equation"
T
"# I m + A ( !s ) $% R "# I m + A ( s ) $% = R + Y T ( !s ) Y ( s )
Algebraic Riccati equation with scalar control"
"#1 + A ( !s ) $% r "#1 + A ( s ) $% = r + Y T ( !s )Y ( s )
where!
A ( s ) = C ( sI n ! F ) G
!1
dim ( C ) = (1 " n )
(1 " 1)
Y ( s ) = H ( sI n ! F ) G
!1
dim "#Y ( s ) $% = ( p & 1)
dim ( F ) = ( n " n )
dim ( H ) = ( p & n )
dim ( G ) = ( n " 1)
23!
Scalar Case"
Let s = j!
#$1 + A ( ! j" ) %& r #$1 + A ( j" ) %& = r + Y T ( ! j" )Y ( j" )
or!
Y T ( ! j" )Y ( j" )
#$1 + A ( ! j" ) %& #$1 + A ( j" ) %& = 1 +
r
A ( j! ) is a complex variable
{
}{#$1 + c (" )%& + jd (" )}
#$1 + A ( ! j" ) %& #$1 + A ( j" ) %& = #$1 + c (" ) %& ! jd (" )
{
2
}
= #$1 + c (" ) %& + d 2 (" ) = 1 + A ( j" )
2
(absolute value)
24!
Kalman Inequality"
In frequency domain, cost transfer function becomes"
Yi ( j! ) = "#li (! ) + jmi (! ) $% , i = 1, p
p
#$li2 (" ) + mi2 (" ) %&
Y T ( ! j" )Y ( j" )
1+
= 1+ '
r
r
i =1
Consequently, the return difference function
magnitude is greater than one"
1 + A ( j! ) " 1 Kalman Inequality
25!
Nyquist Plot Showing Consequences
of Kalman Inequality"
26!
Uncertain Gain and Phase
Modifications to the LQ
Feedback Loop"
How large an uncertainty in loop gain or phase angle can be
tolerated by the LQ regulator?"
27!
LQ Gain Margin
Revealed by
Kalman Inequality"
!! Stability is preserved if"
!! No encirclements of the –1 point,
or"
!! Number of counterclockwise
encirclements of the –1 point
equals the number of unstable
open-loop roots"
!! Loop gain change expands or
shrinks entire Nyquist plot"
28!
Gain Change Expands or
Shrinks Entire Plot"
kU ! Uncertain gain
Aoptimal ( j! ) = C LQ ( j! I n " F ) G
"1
Anon " optimal ( j! ) = kU Aoptimal ( j! )
= kU C LQ ( j! I n " F ) G
"1
Anon " optimal ( j! ) = kU Aoptimal ( j! )
!! Closed-Loop LQ system is
stable until –1 point is reached,
and # of encirclements changes"
29!
Scalar LQ Regulator Gain Margin"
Increased gain margin = Infinity"
Decreased gain margin = 50%"
30!
LQ Phase Margin
Revealed by Kalman
Inequality"
!! Stability is preserved if"
!! No encirclements of the –1 point"
!! Number of counterclockwise encirclements of the –1
point equals the number of unstable open-loop roots"
Return Difference Function, 1 + A ( j! ) , is excluded from a unit circle centered at ( –1, 0 )
A ( j! ) = 1 intersects a unit circle centered at the origin
(
Intersection of the unit circles occurs where the phase angle of A ( j! ) = –180 o ± 60 o
)
Therefore, Phase Margin of LQ regulator # 60°"
31!
LQ Regulator Preserves Stability with
Phase Uncertainties of At Least –60°"
!U ! Uncertain phase angle, deg
Aoptimal ( j" ) = C LQ ( j" I n # F ) G
#1
Anon # optimal ( j" ) = e j!U Aoptimal ( j" )
= e j!U C LQ ( j" I n # F ) G
#1
! non # optimal = ! optimal + !U
!! Phase-angle change rotates
entire Nyquist plot"
!! Closed-Loop LQ system is
stable until –1 point is reached"
32!
Reduced-Gain-/Phase-Margin
Tradeoff"
Reduced loop gain decreases allowable phase lag while
retaining closed-loop stability"
33!
Next Time:!
Singular Value Analysis of
LQ Systems!
34!
Supplemental Material
35!
Effect of Time Delay!
36!
Time Delay Example:
DC Motor Control"
Control command delayed by ! sec
37!
Effect of Time Delay on
Step Response"
With no delay!
Phase lag due to
time delay reduces
closed-loop stability"
38!
Bode Plot of
Pure Time Delay"
(
)
AR e! j"# = 1
$ ( e! j"# ) = !"#
As input frequency
increases, phase angle
eventually exceeds –180°!
39!
Effect of Pure Time Delay on LQ
Regulator Loop Transfer Function"
Laplace transform of time-delayed signal:
L #$u ( t ! " ) %& = e!" s L #$u ( t ) %& = e!" s u ( s )
" U ! Uncertain time delay, sec
Aoptimal ( j! ) = C LQ ( j! I n " F ) G
"1
Anon " optimal ( j! ) = e – #U j! Aoptimal ( j! )
= e – #U j! C LQ ( j! I n " F ) G
"1
40!
Effect of Pure Time
Delay on LQ Regulator
Loop Transfer Function"
Crossover frequency, ! cross , is frequency for which
Aoptimal ( j! cross ) = C LQ ( j! cross I n " F ) G = 1
"1
Time delay that produces 60° phase lag"
!U =
#
60° $ # '
, sec
=
&
)
" cross % 180° ( 3" cross
41!