Time series, spring 2014
10
Slides for ITS, sections 8.1, 8.4, 8.5 (see
also SSPSE 8.6.1)
Exercises: ITS 8.1, 8.7, 8.9, 8.12
State space models
Let πππ‘π‘ = πππ‘π‘,1 , … , πππ‘π‘,π€π€ ′ and πΏπΏππ = πππ‘π‘,1 , … , πππ‘π‘,π£π£ ′ be random
vectors, let πΎπΎπ‘π‘ ~WN(ππ, π
π
) and π½π½π‘π‘ ~WN ππ, ππ , and let πΊπΊ be a
π€π€ × π£π£ matrix and πΉπΉ be a π£π£ × π£π£ matrix. A state space model
satisfies the equations
πππ‘π‘ = πΊπΊπΏπΏπ‘π‘ + πΎπΎπ‘π‘ ,
οΏ½
πΏπΏπ‘π‘+1 = πΉπΉπΏπΏπ‘π‘ + π½π½π‘π‘
the observation equation
the state equation,
for π‘π‘ = 0, ±1, ±2, … The equation is stable if πΉπΉ has all its
eigenvalues inside of the unit circle. Then
∞
πΏπΏπ‘π‘ = οΏ½ πΉπΉππ π½π½π‘π‘−ππ−1
ππ=0
ππ π½π½
and πππ‘π‘ = πππ‘π‘ + ∑∞
πΊπΊπΉπΉ
π‘π‘−ππ−1
ππ=0
Kalman recursion
Estimation of πΏπΏπ‘π‘ from
• ππ0 , …, πππ‘π‘−1 is the prediction problem
• ππ0 , …, πππ‘π‘ is the filtering problem
• ππ0 , …, ππππ for ππ > π‘π‘ is the smoothing problem
οΏ½ π‘π‘ = πππ‘π‘−1 πΏπΏπ‘π‘ of πΏπΏπ‘π‘ is given by the
The best linear predictor πΏπΏ
recursion
οΏ½ π‘π‘+1 = πΉπΉ πΏπΏ
οΏ½ π‘π‘ + ΘΔ−1
οΏ½
πΏπΏ
π‘π‘ (πππ‘π‘ − πΊπΊ πΏπΏπ‘π‘ )
′
οΏ½
οΏ½
• Ωπ‘π‘ = πΈπΈ[ πΏπΏπ‘π‘ − πΏπΏπ‘π‘ πΏπΏπ‘π‘ − πΏπΏπ‘π‘ ]
• Δπ‘π‘ = πΊπΊΩ′π‘π‘ πΊπΊ ′ + π
π
and Δ−1
π‘π‘ is (generalized) inverse of Δπ‘π‘
• Θπ‘π‘ = πΉπΉΩ′π‘π‘ πΊπΊ ′
οΏ½ 1 and Ω1 are obtained by direct
• The initial values πΏπΏ
computation (or by cheating and setting them equal to 0 and πΌπΌ,
respectively)
• Ωπ‘π‘ , Δπ‘π‘ , and Θπ‘π‘ converge to limiting values, which may be used
to simplify the recursions for large π‘π‘
οΏ½ π‘π‘ = πππ‘π‘−1 πΏπΏπ‘π‘ of πΏπΏπ‘π‘ for π‘π‘ > 1 is given
The best linear predictor πΏπΏ
by the recursions
οΏ½ π‘π‘+1 = πΉπΉ πΏπΏ
οΏ½ π‘π‘ + Θt Δ−1
οΏ½
πΏπΏ
π‘π‘ πππ‘π‘ − πΊπΊ πΏπΏπ‘π‘
′
Ωπ‘π‘ = πΉπΉΩ′π‘π‘ πΉπΉ ′ + ππ − Θt Δ−1
π‘π‘ Θt
Pf: Innovations are defined recursively by π°π°0 = ππ0 and
οΏ½ π‘π‘ + πΎπΎπ‘π‘
π°π°π‘π‘ = πππ‘π‘ − πππ‘π‘−1 πππ‘π‘ = πΊπΊ πΏπΏπ‘π‘ − πππ‘π‘−1 πΏπΏ
Since
πππ‘π‘ ⋅ = πππ‘π‘−1 ⋅ + ππ ⋅ π°π°π‘π‘
it follows that
οΏ½
πΏπΏπ‘π‘+1 = πππ‘π‘−1 πΏπΏπ‘π‘+1 + ππ πΏπΏπ‘π‘+1 π°π°π‘π‘
= πππ‘π‘−1 πΉπΉπΏπΏπ‘π‘ + π½π½π‘π‘ + πΈπΈ πΏπΏπ‘π‘+1 π°π°′π‘π‘ πΈπΈ π°π°π‘π‘ π°π°′π‘π‘ −1 π°π°π‘π‘
−1
οΏ½ π‘π‘ + Θπ‘π‘ Δ−1
οΏ½
οΏ½
= πΉπΉ πΏπΏ
π‘π‘ π°π°π‘π‘ = πΉπΉ πΏπΏπ‘π‘ + Θπ‘π‘ Δπ‘π‘ (πππ‘π‘ −πΊπΊ πΏπΏπ‘π‘ )
Further
Ωπ‘π‘+1 = πΈπΈ
οΏ½ π‘π‘+1 πΏπΏπ‘π‘+1 − πΏπΏ
οΏ½ π‘π‘+1
πΏπΏπ‘π‘+1 − πΏπΏ
′
οΏ½ π‘π‘+1 πΏπΏ
οΏ½ ′π‘π‘+1
= πΈπΈ πΏπΏπ‘π‘+1 πΏπΏπ‘π‘+1
− πΈπΈ πΏπΏ
′
′
οΏ½ π‘π‘ πΏπΏ
οΏ½ ′π‘π‘ πΉπΉ ′ − Θπ‘π‘ Δ−1
= πΉπΉπΉπΉ πΏπΏπ‘π‘ πΏπΏ′π‘π‘ πΉπΉ ′ + ππ − πΉπΉπΉπΉ πΏπΏ
π‘π‘ Θπ‘π‘
′
= πΉπΉΩπ‘π‘ πΉπΉ ′ + ππ − Θπ‘π‘ Δ−1
Θ
π‘π‘
π‘π‘
οΏ½ π‘π‘+1
• β-step predictor is οΏ½
πΏπΏπ‘π‘+β = πΉπΉ β−1 πΏπΏ
• Similar computations solve filtering and smoothing problem
• Similar computations if πΊπΊ and πΉπΉ depend on π‘π‘
Estimation
Let ππ be a vector which contains all the parameters of the state
space model (πΊπΊ, πΉπΉ, parameters of ππ and ππ, … ). The conditional
likelihood given ππ0 is
ππ
πΏπΏ ππ; ππ1 , … , ππππ = οΏ½ ππ( πππ‘π‘ |ππ1 , … , ππππ−ππ )
π‘π‘=1
and if all variables are jointly normal, then
ππ(πππ‘π‘ ππ1 , … , ππππ−1 = 2ππ
π€π€
−2
det Δπ‘π‘
1
−2
1
2
exp(− π°π°′π‘π‘ Δ−1 π°π°π‘π‘ ),
and estimates may be found by numerical maximization of the
log conditional likelihood function.
• Sometimes it is useful to continuously downweigh distant
observations to adapt to changes in the situation which is
model changes with time. This can be done recursively
0
