Dynamic Systems Theory - State-space Linear
Systems
Kushal Prasad
December 6, 2012
Let w(t) be a sample of a Wiener process. The Ito stochastic integral
Rb
a
b(t) dw = limρ→0
Pn−1
i=1
b(ti )(w(ti+1 ) − w(ti ))
ρ = maxi |ti+1 − ti |
Properties :
Rb
1. E( a b(t) dw) = 0
Rb
Rb
Rb
2. E( a f (t) dw. a g(t) dw) = q a E(f (t)g(t)) dt
We write,
dx = f (x, t) dt + g(x, t) dw
(∗)
as a proxy for
ẋ = f (x, t) + g(x, t)ẇ
What (∗) really means is,
Rt
Rt
xt − xo = 0 f (x, τ ) dτ + 0 g(x, τ ) dw
(∗∗)
↑ This is obtained by our
non-anticipating limiting process
ITO’s Lemma : Let x(t) be the unique solution of the vector Ito stochastic
differential equation
dx = f (x, t) dt + g(x, t)], dw
(∗)
where x and f are n-vectors, g(· , ·) is an m × m matrix and w(t) is an mdimensional Wiener with
E(w(t) w(t)T ) = Q · t
1
Dynamic Systems Theory
2
Let Q(x, t) be a scalar real valued function continuously differentiable in t and
having continuous second partial derivatives in x. Then the stochastic differential dQ of Q is,
dQ =
∂φ
∂t
dt +
∂φT
∂x
where dx is defined by (∗), and
 ∂φ2
∂2φ
∂x2
∂φ2
∂x1 ∂xn
..
.
..
.
...
..
.
∂φ2
∂x1 ∂xn
∂φ2
∂x2 ∂xn
...
∂x21

=

dx + 12 tr(gQg
∂2φ
∂x2 ) dt,
∂φ2
∂x1 ∂xn

..
.



∂φ2
∂x2n
Example :
ẋ = Ax + B ẇ or more formally and correctly,
dx = Ax dt + B dw
Compute the second moment,
d(x xT ) = x dxT + dx xT + (??) dt
The i − j th element of x xT is xi xj = φ
The ITO correction term for this entry is,

0 0 ...
0 0 . . .

1
T 
1
2 tr(BQB 

1
0 ...

0
0

)dt


0
↑ zero’s everywhere except 1’s in i − j th
& j − ith positions
= i − j th element of BQB T dt
d(xxT ) = xxT AT dt + x dwT B T + AxxT dt + B dw xT + BQB T dt
P
Taking expectations and writing o (t) = E(x(t)x(t)T )
P
P
P
d o = o AT dt + A o dt + BQB T dt
⇓ this is an ordinary differential equation
P˙
P
P T
T
o =A
o+
o A + BQB
This may be solved by the matrix variation of constants formula,
Rt
P
P
T
At
AT t
+ 0 expA(t−σ) BQB T expA (t−σ) dσ
o (t) = exp
o (0) exp
Dynamic Systems Theory
3
Problem addressed by Kalman and Bucy was to find an optimal state estimate
of
ẋ = Ax + Bu + F ẇ
y = Cx + v̇
Path to the solution: Find the optimal m.s.-estimate among linear observers
x̂˙ = Ax̂ + BU + K(y − C x̂)
Let e = x − x̂
ė = Ax + BU + F ẇ − Ax̂ − Bu − K(y − C x̂)
= (A − KC)e + F ẇ − K v̇
d(eeT ) = de eT + e deT + L dt
where, L = variance of ξ = F ẇ − K v̇
E((F ẇ − K v̇)(F ẇ − K v̇)T )
= F QF T + KRK T
where Q = E(ẇẇT ), R = E(v̇ v̇ T )
and we assume ẇ, v̇ are uncorrelated.
d(eeT ) = (A−KC)eeT dt+eeT (A−KC)T dt+B dw eT +e dwT B T +F QF T dt+
KRK T dt
P
Taking expectations of both sides, writing ee (t) = E(eeT )
P
P
P
d
T
T
T
ee (t) = AK
ee (t) +
ee (t)AK + F QF + KRK
dt
where AK P
= A − KC. We wish to find the gain matrix K that minimizes the
m.s.-error ee (t) at each time t1 .
What it means for one symmetric matrix to be less than another:
P ≤ Q ⇔ xT P x ≤ xT Qx
Let
for all x.
Po
ee (t)
be the optimal error variance. Then writing
P
Po
P
P = ee (t) − ee (t) where ee (t) error for a nominal gain K(t).
The P satisfies
P Po
Ṗ = ˙ − ˙
= (A − KC)
Po
+
P
Po
(A − KC)T + F QF T + KRK T − (A − K o C)
−
Dynamic Systems Theory
4
P
T
(A − K o C)T − F QF T − KRK o
T
= (A−KC)P
+(K o −K)R(K o −K)T +(K o −K)(C
P +PT(A−KC)
o
( o C − K R)(K o − K)T
Po
T
−RK o )+
= (A − KC)P + P (A − KC)T + L
Po
T
o
o
where L = (K
− K)T + (K o − K)(C
−RK o )+
Po− K)R(K
(
C T − K o R)(K o − K)T
The solution to this equation is
P (t) = ΦAC (t, 0)P0 ΦAC (t, 0)T +
P
Po
We assume P0 = ee (0) − oe (0) = 0.
Rt
δ
ΦAC (t, s)LΦAC (t, s)T ds
P (t) will be positive semi-definite precisely when L is positive semi-definite. In
rder for L to be positive semi-definite for choices of nominal gain K, we must
have,
Po T
C − K oR = 0
Thus the optimal gain is
Ko =
Po
ee (t)C
T
R−1
Let’s verifyPthat this ’Kalman gain’ is optimal.
o
First note
(t) satisfies
P˙ o
Po Po T
T
= AK
+
AK + F QF TP+ K o RK o
o
where AK = A − K o C, K o =
C T R−1
P˙ o
Po T −1 Po Po T
Po
= (A −
C R C)
+ P
(A − C T R−1P
C
)+
o T −1
o
F QF T +
C R C
=A
Po
+
Po
AT + F QF T −
Po
C T R−1 C
Po
Using this equation
the optimal error we rewrite the equation for
P
Pfor
o
P = ee − ee
P
P
Po Po T
Ṗ = A) KC) P
KC)T + F QF T + KRK T − S
−
A
ee +
ee (A −P
o T −1
o
−F QF T +
C R C
T
= AP + P AT + KRKK
C
= AP + P AT + (K −
|
o
X
Po
−
Po
CT KT +
C T R−1 )R(K −
{z
L
X
Po
C T R−1)T
}
Dynamic Systems Theory
P (t) =
RT
0
5
ΦA (t, s)L(s)ΦTA (t, s) ds
This is positive semi definite and = 0 ⇔
Po T −1
K−
C R P= 0
Po
(t)
ee (t) ≥
with equality holding ⇔ K =
This shows
Po
C T R−1 .
SUMMARY : Given the finite dimensional linear system,
ẋ = Ax + Bu + F ẇ
y = Cx + v
ẇ is Gaussian white noise N (O, Q)
v̇ is Gaussian white noise N (O, R)
ẇ, v̇ uncorrelated
The mean-squared optimal observer is
Po
Po T −1
x̂˙ = (A −
(t)C T R−1 C)x̂ + Bu +
C R y
x̂(0) = E(x0 ) = x¯0
with
Po
(t) given as the solution of
P˙ o
Po Po T
Po T −1 Po
(t) = A
+
A + F QF T −
c R C
Po
(0) = E((x0 − x¯0 )(x0 − x¯0 )T )