2.160 System Identification, Estimation, and Learning
Lecture Notes No. 11
March 15, 2006
6.5 Times-Series Data Compression
b2
u(t)
b1
y(t)
FIR
b3
t
Finite Impulse Response Model
Consider a FIR Model
y (t ) = b1u (t − 1) + b2 u (t − 2) + ⋅ ⋅ ⋅ ⋅ ⋅ ⋅ + bm u (t − m)
The transfer functions of (11) are then: G (q ) = B ( q ), H (q ) = 1
One-Step-Ahead Predictor
[
]
yı(t θ ) = H −1 (q)G ( q)u(t ) + 1 − H −1 (q) y (t )
= G ( q )u ( t )
yı (t θ ) = ϕ T (t )θ : linear regression
Given input data {u(t ), 1 ≤ t ≤ N } , the least square estimate of the parameter vector
was obtained as
θıNLS = arg min
θ
1
N
2
N
1
( y(t ) − yı (t | θ )) = (R( N ))−1 f (N )
∑
t =1 2
(40)
where
1
R(N ) =
N
N
∑ϕ (t )ϕ (t )
T
= ΦΦ
T
t =1
1
and f ( N ) =
N
N
∑ ϕ (t ) y (t )
(41)
t =1
Pro’s and Con’s of FIR Modeling
Pros.
LSE gives a consistent estimate
limθı
N →∞
LS
N
= θ0
as long as the input sequence {u (t )} is
uncorrelated with the noise e(t), which may be colored.
Cons
Two failure scenarios
• The impulse response has a slow decaying mode
1
• The sampling rate is high
The number of parameters, m, is too large to estimate.
can hardly be satisfied.
The persistently exciting condition rankΦ = full rank
T
Check the eigenvalues of ΦΦ (or the singular values of Φ )
λ1 ≥ λ2 ≥ � ≥ λm
It is likely
λ1 ≥ λ2 ≥ � ≥ λn ≥ λn+1 ≅0 = � = λm
Often m becomes more than 50 and it is difficult to obtain such an input series having
50 non-zero singular values.
Time series data compression is an effective method for coping with this difficulty.
Before formulating the above least square estimate problem, data are processed so
that the information contained in the series of regressor may be represented in
compact, low-dimensional form.
ϕm
ϕ1
ϕn
.
. . .
. .
.
λ ≅0
ϕ1
ϕ2
Filter Design
Lk(q)
i
Rm
Rn
n<m
Coordinate Transformation (Data Compression)
L1(q)
u(t)
L2(q)
b1
b2
∑
y(t)
�
Ln(q)
bn
2
6.6 Continuous-Time Laguerre Series Expansion
Let us begin with continuous-time Laguerre expansion.
[Theorem 6.6]
Assume a transfer function G(s) is
•
Strictly proper
•
Analytic in Re(s) > 0
•
Continuous in Re(s) ≥ 0
lim G (
s ) = 0
G(∞) = 0
s →∞
Then, there exists a sequence
(45)
No pole on RHP
{g k }
such that
∞
G (s ) = ∑ g k
k =1
2a ⎛ s − a ⎞
⎜
⎟
s+a ⎝s+a⎠
k −1
(46)
where a > 0
Proof:
Consider the transformation given by
s+a
z =
s − a
sz − az = s + a
Im
s−a
s
s ( z − 1) = a ( z + 1)
∴ s = a
Bilinear transformation
z =
z-plane
(47)
s+a
s−a
z +1
z −1
(48)
Im
=1
for s = jω
(42)
1
∠z = ∠( s + a ) − ∠( s − a )
-1
Re
s−a
z =
Re
s
s+a
s−a
1
s + a The
imaginary axis is
mapped to the unit circle.
>1
(43)
The right half plane is
mapped to the outside of
the unit circle.
3
Im
Im
G(s) is analytic in Re( s ) > 0
1
-1
1
Re
Consider
Re
⎛
⎞
⎜ z +1⎟
G (z) = G⎜ a
⎟
z −�1 ⎟
⎜ ��
⎝ s ⎠
G (z ) is analytic outside
(44)
the unit circle.
Therefore, there exists a Laurent expansion for G (z ) :
∞
G ( z ) =
∑ g k z − k
z >1
(49)
k =1
s +
a
=
1 ∴
G ( z )
= 0 at z=1
s →∞ s − a
Considering that G (∞)
=
0
lim
G ( z ) =
∞
1
(1 −
z −1 )
∑ g k z −( k −1)
2a
k =1
(50)
Substituting (40) into (46)
s
− a ∞
1
⎛ s
+ a
⎞
⎛ s +
a
⎞
G⎜
(1
−
)∑
g k ⎜
⎟ = G ( s ) =
⎟
s
+ a k =1 ⎝
s −
a ⎠
2a
⎝ s − a ⎠
∞
∴
G ( s ) =
∑ g k
k =1
Lk (s ) =
2a ⎛ s − a ⎞
⎜
⎟
s +
a
⎝
s
+
a ⎠
2a ⎛ s − a ⎞
⎜
⎟
s +
a ⎝
s + a ⎠
Low Pass Filter
− (k −1)
(k −1)
(k −1)
(51)
s−a
s+a
jω − a
=1
jω + a
All-pass Filter
The Laplace transform of the Laguerre functions
The Main Point
The above Laguerre series expansion can be used for data compression if the
parameter a , called a Laguerre pole, is chosen such that slow poles, i.e. dominant
poles, of the original system are close to the Laguerre pole. Let pi be a slow real pole
of the original transfer function G(s). If the Laguerre pole a is chosen such that
4
a ≅pi , then a truncated Laguerre expansion:
n
Gn (s) = ∑ g k
k =1
2a ⎛ s − a ⎞
⎜
⎟
s+a ⎝s+a⎠
k −1
→ G( s )
(52)
converges to the original G(s) quickly.
- Recall -
For a continuous-time system, a pole close to the imaginary axis is slow to converge,
while a pole far from the imaginary axis converges quickly. Likewise, in discrete time, a pole close to the origin of a z-plane quickly converges, while the ones near the unit
circle are slow.
z-plane
Im
s-plane
Fast
Slow
Fast
Re
Slow
Choose a such that p1 ≅−a , where p1 is a slow, stable pole. Using the bilinear
transform, this slow pole in the s-plane can be transferred to a fast pole in the z-plane,
as shown below. Representing in the z-plane, the transfer function can be truncated;
just a few terms can approximate the impulse response since it converges quickly.
p1
Im
Re
a ≅p1
{g k }
Transformation
Im
z=
1
s+a
s−a
Truncated
-1
1
time
Re
Fast Convergence
-1
Fast pole z ≅0
When the original transfer function has many poles, the Laguerre pole is placed near
the dominant pole so that most of the slow poles may be approximated by the
Laguerre pole.
5
Im
Im
Dominant pole
1
-1
1
z
Re
Re
In the transformed plane, all
the poles are confined within
a circle of radius r<1.
−a
With the bilinear transform, these slow poles are transferred to the ones near the
origin of the z-plane. They are fast, hence the transfer function can be approximated
to a low order model.
6.7 Discrete-Time Laguerre Series Expansion
[Theorem 6.7]
Assume that a Z-transform G(z) is
•
Strictly proper
•
Analytic in z > 1
•
Continuous in z ≥ 1
G(∞) = 0
RHP
Then
∞
G (s ) = ∑ g k
k =1
where
K ⎛ 1 − az ⎞
⎜
⎟
z−a⎝ z−a ⎠
k −1
(53)
-1<a<1 and
K = (1 − a 2 )T
Proof
Consider the bilinear transformation:
z−a
w=
1 − az
w − azw = z − a
w + a = z (
aw + 1) therefore, z =
(54)
(55)
w+a
aw + 1
(56)
6
Transformation
z−a
w=
1 − az
z-plane
w-plane
⎛ w+a ⎞
G ( w) = G ⎜
⎟
⎝ aw + 1 ⎠
z >1
w >1
⎛ w
+
a ⎞
G (w) =
G
⎜
⎟ is analytic in w > 1, and is proper G(∞
) =
0
⎝
aw
+
1
⎠
lim
w
=−
z →∞
1
a
1
G
(
− ) = 0
a
(57)
Therefore
G ( w) =
∞
T
(a
+
w
−1 )∑ g k w −( k −1)
K
k =1
(58)
−(k −1)
1
−
az ∞
⎛ z −
a ⎞ T
⎛ z−a ⎞
G( z ) = G ⎜
)∑ g k ⎜
⎟ =
(a +
⎟
z −
a k =1 ⎝
1
−
az ⎠
⎝ 1
−
az ⎠
K
∞
∴ G ( s ) =
∑ g k
k =1
(59)
k −1
K ⎛ 1 −
az
⎞
⎜
⎟
z −
a
⎝
z −
a ⎠
(53)
Q.E.D.
Now we can write
y (t ) =
G ( q)u(t )
k −1
n
K ⎛
1
−
aq
⎞
=
∑
g k
⎜⎜
⎟⎟ u(t ) =
∑
g k Lk ( q)u(t )
q − a
⎝
q
−
a ⎠
k =1
k =1
n
where Lk ( q ), k =
1,�, n is a series of filters. Once the original input data are filtered
7
with Lk ( q ), k = 1,�, n ,
xk = Lk ( q )u(t ), k = 1,�, n
(60)
The output y (t ) is represented as a moving average of the transformed input xk ,
that is, a FIR model.
L1(q)
u(t)
L2(q)
Ln (q )
x1
x2
xn
g1
g2
y(t)
∑
gn
Furthermore, xk (t ) can be computed recursively.
x1 (t) = L1 ( q)u(t ) =
K
Kq −1
u( t ) =
u (t )
q−a
1 − aq −1
x1 (t) − ax1 (t − 1) = Ku(t − 1)
x1 (t) = ax1 (t − 1) + Ku(t − 1)
x2 (t) =
1 − aq
q −1 − a
x1 (t) =
x1 (t)
q−a
1 − aq −1
x2 (t) − ax 2 (t − 1) = x1 (t − 1) − ax1 (t )
x2 (t) = ax2 (t − 1) + x1 (t − 1) − ax1 (t )
x
k (t) = axk (t − 1) + xk −1 (t − 1) − axk −1 (t)
(61)
Recursive Filters
8