# ITS8 ```Time series, spring 2014
10
Slides for ITS, sections 8.1, 8.4, 8.5 (see
also SSPSE 8.6.1)
Exercises: ITS 8.1, 8.7, 8.9, 8.12
State space models
Let 𝒀𝒀𝑡𝑡 = 𝑌𝑌𝑡𝑡,1 , … , 𝑌𝑌𝑡𝑡,𝑤𝑤 ′ and 𝑿𝑿𝒕𝒕 = 𝑋𝑋𝑡𝑡,1 , … , 𝑋𝑋𝑡𝑡,𝑣𝑣 ′ be random
vectors, let 𝑾𝑾𝑡𝑡 ~WN(𝟎𝟎, 𝑅𝑅) and 𝑽𝑽𝑡𝑡 ~WN 𝟎𝟎, 𝑄𝑄 , and let 𝐺𝐺 be a
𝑤𝑤 &times; 𝑣𝑣 matrix and 𝐹𝐹 be a 𝑣𝑣 &times; 𝑣𝑣 matrix. A state space model
satisfies the equations
𝒀𝒀𝑡𝑡 = 𝐺𝐺𝑿𝑿𝑡𝑡 + 𝑾𝑾𝑡𝑡 ,
�
𝑿𝑿𝑡𝑡+1 = 𝐹𝐹𝑿𝑿𝑡𝑡 + 𝑽𝑽𝑡𝑡
the observation equation
the state equation,
for 𝑡𝑡 = 0, &plusmn;1, &plusmn;2, … The equation is stable if 𝐹𝐹 has all its
eigenvalues inside of the unit circle. Then
∞
𝑿𝑿𝑡𝑡 = � 𝐹𝐹𝑗𝑗 𝑽𝑽𝑡𝑡−𝑗𝑗−1
𝑗𝑗=0
𝑗𝑗 𝑽𝑽
and 𝒀𝒀𝑡𝑡 = 𝑊𝑊𝑡𝑡 + ∑∞
𝐺𝐺𝐹𝐹
𝑡𝑡−𝑗𝑗−1
𝑗𝑗=0
Kalman recursion
Estimation of 𝑿𝑿𝑡𝑡 from
• 𝒀𝒀0 , …, 𝒀𝒀𝑡𝑡−1 is the prediction problem
• 𝒀𝒀0 , …, 𝒀𝒀𝑡𝑡 is the filtering problem
• 𝒀𝒀0 , …, 𝒀𝒀𝑛𝑛 for 𝑛𝑛 &gt; 𝑡𝑡 is the smoothing problem
� 𝑡𝑡 = 𝑃𝑃𝑡𝑡−1 𝑿𝑿𝑡𝑡 of 𝑿𝑿𝑡𝑡 is given by the
The best linear predictor 𝑿𝑿
recursion
� 𝑡𝑡+1 = 𝐹𝐹 𝑿𝑿
� 𝑡𝑡 + ΘΔ−1
�
𝑿𝑿
𝑡𝑡 (𝒀𝒀𝑡𝑡 − 𝐺𝐺 𝑿𝑿𝑡𝑡 )
′
�
�
• Ω𝑡𝑡 = 𝐸𝐸[ 𝑿𝑿𝑡𝑡 − 𝑿𝑿𝑡𝑡 𝑿𝑿𝑡𝑡 − 𝑿𝑿𝑡𝑡 ]
• Δ𝑡𝑡 = 𝐺𝐺Ω′𝑡𝑡 𝐺𝐺 ′ + 𝑅𝑅 and Δ−1
𝑡𝑡 is (generalized) inverse of Δ𝑡𝑡
• Θ𝑡𝑡 = 𝐹𝐹Ω′𝑡𝑡 𝐺𝐺 ′
� 1 and Ω1 are obtained by direct
• The initial values 𝑿𝑿
computation (or by cheating and setting them equal to 0 and 𝐼𝐼,
respectively)
• Ω𝑡𝑡 , Δ𝑡𝑡 , and Θ𝑡𝑡 converge to limiting values, which may be used
to simplify the recursions for large 𝑡𝑡
� 𝑡𝑡 = 𝑃𝑃𝑡𝑡−1 𝑿𝑿𝑡𝑡 of 𝑿𝑿𝑡𝑡 for 𝑡𝑡 &gt; 1 is given
The best linear predictor 𝑿𝑿
by the recursions
� 𝑡𝑡+1 = 𝐹𝐹 𝑿𝑿
� 𝑡𝑡 + Θt Δ−1
�
𝑿𝑿
𝑡𝑡 𝒀𝒀𝑡𝑡 − 𝐺𝐺 𝑿𝑿𝑡𝑡
′
Ω𝑡𝑡 = 𝐹𝐹Ω′𝑡𝑡 𝐹𝐹 ′ + 𝑄𝑄 − Θt Δ−1
𝑡𝑡 Θt
Pf: Innovations are defined recursively by 𝑰𝑰0 = 𝒀𝒀0 and
� 𝑡𝑡 + 𝑾𝑾𝑡𝑡
𝑰𝑰𝑡𝑡 = 𝒀𝒀𝑡𝑡 − 𝑃𝑃𝑡𝑡−1 𝒀𝒀𝑡𝑡 = 𝐺𝐺 𝑿𝑿𝑡𝑡 − 𝑃𝑃𝑡𝑡−1 𝑿𝑿
Since
𝑃𝑃𝑡𝑡 ⋅ = 𝑃𝑃𝑡𝑡−1 ⋅ + 𝑃𝑃 ⋅ 𝑰𝑰𝑡𝑡
it follows that
�
𝑿𝑿𝑡𝑡+1 = 𝑃𝑃𝑡𝑡−1 𝑿𝑿𝑡𝑡+1 + 𝑃𝑃 𝑿𝑿𝑡𝑡+1 𝑰𝑰𝑡𝑡
= 𝑃𝑃𝑡𝑡−1 𝐹𝐹𝑿𝑿𝑡𝑡 + 𝑽𝑽𝑡𝑡 + 𝐸𝐸 𝑿𝑿𝑡𝑡+1 𝑰𝑰′𝑡𝑡 𝐸𝐸 𝑰𝑰𝑡𝑡 𝑰𝑰′𝑡𝑡 −1 𝑰𝑰𝑡𝑡
−1
� 𝑡𝑡 + Θ𝑡𝑡 Δ−1
�
�
= 𝐹𝐹 𝑿𝑿
𝑡𝑡 𝑰𝑰𝑡𝑡 = 𝐹𝐹 𝑿𝑿𝑡𝑡 + Θ𝑡𝑡 Δ𝑡𝑡 (𝒀𝒀𝑡𝑡 −𝐺𝐺 𝑿𝑿𝑡𝑡 )
Further
Ω𝑡𝑡+1 = 𝐸𝐸
� 𝑡𝑡+1 𝑿𝑿𝑡𝑡+1 − 𝑿𝑿
� 𝑡𝑡+1
𝑿𝑿𝑡𝑡+1 − 𝑿𝑿
′
� 𝑡𝑡+1 𝑿𝑿
� ′𝑡𝑡+1
= 𝐸𝐸 𝑿𝑿𝑡𝑡+1 𝑿𝑿𝑡𝑡+1
− 𝐸𝐸 𝑿𝑿
′
′
� 𝑡𝑡 𝑿𝑿
� ′𝑡𝑡 𝐹𝐹 ′ − Θ𝑡𝑡 Δ−1
= 𝐹𝐹𝐹𝐹 𝑿𝑿𝑡𝑡 𝑿𝑿′𝑡𝑡 𝐹𝐹 ′ + 𝑄𝑄 − 𝐹𝐹𝐹𝐹 𝑿𝑿
𝑡𝑡 Θ𝑡𝑡
′
= 𝐹𝐹Ω𝑡𝑡 𝐹𝐹 ′ + 𝑄𝑄 − Θ𝑡𝑡 Δ−1
Θ
𝑡𝑡
𝑡𝑡
� 𝑡𝑡+1
• ℎ-step predictor is �
𝑿𝑿𝑡𝑡+ℎ = 𝐹𝐹 ℎ−1 𝑿𝑿
• Similar computations solve filtering and smoothing problem
• Similar computations if 𝐺𝐺 and 𝐹𝐹 depend on 𝑡𝑡
Estimation
Let 𝜃𝜃 be a vector which contains all the parameters of the state
space model (𝐺𝐺, 𝐹𝐹, parameters of 𝑊𝑊 and 𝑉𝑉, … ). The conditional
likelihood given 𝒀𝒀0 is
𝑛𝑛
𝐿𝐿 𝜃𝜃; 𝒀𝒀1 , … , 𝒀𝒀𝑛𝑛 = � 𝑓𝑓( 𝒀𝒀𝑡𝑡 |𝒀𝒀1 , … , 𝒀𝒀𝒕𝒕−𝟏𝟏 )
𝑡𝑡=1
and if all variables are jointly normal, then
𝒇𝒇(𝒀𝒀𝑡𝑡 𝒀𝒀1 , … , 𝒀𝒀𝒕𝒕−1 = 2𝜋𝜋
𝑤𝑤
−2
det Δ𝑡𝑡
1
−2
1
2
exp(− 𝑰𝑰′𝑡𝑡 Δ−1 𝑰𝑰𝑡𝑡 ),
and estimates may be found by numerical maximization of the
log conditional likelihood function.
• Sometimes it is useful to continuously downweigh distant
observations to adapt to changes in the situation which is
model changes with time. This can be done recursively
```