Rob J Hyndman
State space models
3: ARIMA and RegARMA models, and dlm
Outline
1 ARIMA models in state space form
2 RegARMA models in state space form
3 The dlm package for R
4 MLE using the dlm package
5 Filtering, smoothing and forecasting using
the dlm package
6 Final remarks
State space models
3: ARIMA and RegARMA models, and dlm
2
Linear Gaussian SS models
Observation equation
State equation
yt = f 0 xt + εt
xt = Gxt−1 + wt
State vector xt of length p
G a p × p matrix, f a vector of length p
εt ∼ NID(0, σ 2), wt ∼ NID(0, W ).
State space models
3: ARIMA and RegARMA models, and dlm
3
Outline
1 ARIMA models in state space form
2 RegARMA models in state space form
3 The dlm package for R
4 MLE using the dlm package
5 Filtering, smoothing and forecasting using
the dlm package
6 Final remarks
State space models
3: ARIMA and RegARMA models, and dlm
4
ARMA models in state space form
AR(2) model
yt = φ1 yt−1 + φ2 yt−2 + et ,
et ∼ NID(0, σ 2 )
et
yt
.
and wt =
Let xt =
0
y t −1
Then
yt = [1 0]xt
xt =
φ1 φ2
1
0
xt−1 + wt
Now in state space form
We can use Kalman filter to compute likelihood
and forecasts.
State space models
3: ARIMA and RegARMA models, and dlm
5
ARMA models in state space form
AR(2) model
yt = φ1 yt−1 + φ2 yt−2 + et ,
et ∼ NID(0, σ 2 )
et
yt
.
and wt =
Let xt =
0
y t −1
Then
yt = [1 0]xt
xt =
φ1 φ2
1
0
xt−1 + wt
Now in state space form
We can use Kalman filter to compute likelihood
and forecasts.
State space models
3: ARIMA and RegARMA models, and dlm
5
ARMA models in state space form
AR(2) model
yt = φ1 yt−1 + φ2 yt−2 + et ,
et ∼ NID(0, σ 2 )
et
yt
.
and wt =
Let xt =
0
y t −1
Then
yt = [1 0]xt
xt =
φ1 φ2
1
0
xt−1 + wt
Now in state space form
We can use Kalman filter to compute likelihood
and forecasts.
State space models
3: ARIMA and RegARMA models, and dlm
5
ARMA models in state space form
AR(2) model
yt = φ1 yt−1 + φ2 yt−2 + et ,
et ∼ NID(0, σ 2 )
et
yt
.
and wt =
Let xt =
0
y t −1
Then
yt = [1 0]xt
xt =
φ1 φ2
1
0
xt−1 + wt
Now in state space form
We can use Kalman filter to compute likelihood
and forecasts.
State space models
3: ARIMA and RegARMA models, and dlm
5
ARMA models in state space form
AR(2) model
yt = φ1 yt−1 + φ2 yt−2 + et ,
et ∼ NID(0, σ 2 )
et
yt
.
and wt =
Let xt =
0
y t −1
Then
yt = [1 0]xt
xt =
φ1 φ2
1
0
xt−1 + wt
Now in state space form
We can use Kalman filter to compute likelihood
and forecasts.
State space models
3: ARIMA and RegARMA models, and dlm
5
ARMA models in state space form
AR(2) model
yt = φ1 yt−1 + φ2 yt−2 + et ,
et ∼ NID(0, σ 2 )
Alternative formulation
et
yt
.
and wt =
Let xt =
0
φ2 yt−1
yt = 1 0 xt
φ1 1
xt =
x
+ wt
φ 2 0 t −1
Alternative state space form
We can use Kalman filter to compute likelihood
and forecasts.
State space models
3: ARIMA and RegARMA models, and dlm
6
ARMA models in state space form
AR(2) model
yt = φ1 yt−1 + φ2 yt−2 + et ,
et ∼ NID(0, σ 2 )
Alternative formulation
et
yt
.
and wt =
Let xt =
0
φ2 yt−1
yt = 1 0 xt
φ1 1
xt =
x
+ wt
φ 2 0 t −1
Alternative state space form
We can use Kalman filter to compute likelihood
and forecasts.
State space models
3: ARIMA and RegARMA models, and dlm
6
ARMA models in state space form
AR(2) model
yt = φ1 yt−1 + φ2 yt−2 + et ,
et ∼ NID(0, σ 2 )
Alternative formulation
et
yt
.
and wt =
Let xt =
0
φ2 yt−1
yt = 1 0 xt
φ1 1
xt =
x
+ wt
φ 2 0 t −1
Alternative state space form
We can use Kalman filter to compute likelihood
and forecasts.
State space models
3: ARIMA and RegARMA models, and dlm
6
ARMA models in state space form
AR(2) model
yt = φ1 yt−1 + φ2 yt−2 + et ,
et ∼ NID(0, σ 2 )
Alternative formulation
et
yt
.
and wt =
Let xt =
0
φ2 yt−1
yt = 1 0 xt
φ1 1
xt =
x
+ wt
φ 2 0 t −1
Alternative state space form
We can use Kalman filter to compute likelihood
and forecasts.
State space models
3: ARIMA and RegARMA models, and dlm
6
ARMA models in state space form
AR(2) model
yt = φ1 yt−1 + φ2 yt−2 + et ,
et ∼ NID(0, σ 2 )
Alternative formulation
et
yt
.
and wt =
Let xt =
0
φ2 yt−1
yt = 1 0 xt
φ1 1
xt =
x
+ wt
φ 2 0 t −1
Alternative state space form
We can use Kalman filter to compute likelihood
and forecasts.
State space models
3: ARIMA and RegARMA models, and dlm
6
ARMA models in state space form
AR(p) model
et ∼ NID(0, σ 2 )
yt = φ1 yt−1 + · · · + φp yt−p + et ,



Let xt = 

yt
y t −1
..
.

 
et

0

 
and
w
=

 .. .
t

.
yt−p+1
0
yt = 1 0 0 . . .

φ1
1

xt =  .
 ..
φ2
0
0
...
State space models
0 xt

. . . φp−1 φp
...
0
0

..
..  xt−1 + wt
..
.
.
.
0
1
0
3: ARIMA and RegARMA models, and dlm
7
ARMA models in state space form
AR(p) model
et ∼ NID(0, σ 2 )
yt = φ1 yt−1 + · · · + φp yt−p + et ,



Let xt = 

yt
y t −1
..
.

 
et

0

 
and
w
=

 .. .
t

.
yt−p+1
0
yt = 1 0 0 . . .

φ1
1

xt =  .
 ..
φ2
0
0
...
State space models
0 xt

. . . φp−1 φp
...
0
0

..
..  xt−1 + wt
..
.
.
.
0
1
0
3: ARIMA and RegARMA models, and dlm
7
ARMA models in state space form
ARMA(1, 1) model
et ∼ NID(0, σ 2 )
yt = φyt−1 + θ et−1 + et ,
yt
et
Let xt =
and wt =
.
θ et
θ et
φ 1
y t = 1 0 xt
xt =
State space models
0 0
xt−1 + wt
3: ARIMA and RegARMA models, and dlm
8
ARMA models in state space form
ARMA(1, 1) model
et ∼ NID(0, σ 2 )
yt = φyt−1 + θ et−1 + et ,
yt
et
Let xt =
and wt =
.
θ et
θ et
φ 1
y t = 1 0 xt
xt =
State space models
0 0
xt−1 + wt
3: ARIMA and RegARMA models, and dlm
8
ARMA models in state space form
ARMA(p, q) model
yt = φ1 yt−1 + · · · + φp yt−p + θ1 et−1 + · · · + θq et−q + et
Let r = max(p, q + 1), θi = 0, q < i ≤ r, φj = 0, p < j ≤ r.
yt = 1 0 . . .

φ1
1
0 xt
0
...

0
.. 
.

1

..

.
 φ2 0 1
 θ1 
 .



.
.
.
xt =  ..
..
..
. . 0 xt−1 +  ..  et


 . 
φ
0 1
r −1 0 . . .
θr −1
φr 0 0 . . . 0
The arima function in R is implemented using this formulation.
State space models
3: ARIMA and RegARMA models, and dlm
9
ARMA models in state space form
ARMA(p, q) model
yt = φ1 yt−1 + · · · + φp yt−p + θ1 et−1 + · · · + θq et−q + et
Let r = max(p, q + 1), θi = 0, q < i ≤ r, φj = 0, p < j ≤ r.
yt = 1 0 . . .

φ1
1
0 xt
0
...

0
.. 
.

1

..

.
 φ2 0 1
 θ1 
 .



.
.
.
xt =  ..
..
..
. . 0 xt−1 +  ..  et


 . 
φ
0 1
r −1 0 . . .
θr −1
φr 0 0 . . . 0
The arima function in R is implemented using this formulation.
State space models
3: ARIMA and RegARMA models, and dlm
9
ARMA models in state space form
ARMA(p, q) model
yt = φ1 yt−1 + · · · + φp yt−p + θ1 et−1 + · · · + θq et−q + et
Let r = max(p, q + 1), θi = 0, q < i ≤ r, φj = 0, p < j ≤ r.
yt = 1 0 . . .

φ1
1
0 xt
0
...

0
.. 
.

1

..

.
 φ2 0 1
 θ1 
 .



.
.
.
xt =  ..
..
..
. . 0 xt−1 +  ..  et


 . 
φ
0 1
r −1 0 . . .
θr −1
φr 0 0 . . . 0
The arima function in R is implemented using this formulation.
State space models
3: ARIMA and RegARMA models, and dlm
9
ARMA models in state space form
ARMA(p, q) model
yt = φ1 yt−1 + · · · + φp yt−p + θ1 et−1 + · · · + θq et−q + et
Let r = max(p, q + 1), θi = 0, q < i ≤ r, φj = 0, p < j ≤ r.
yt = 1 0 . . .

φ1
1
0 xt
0
...

0
.. 
.

1

..

.
 φ2 0 1
 θ1 
 .



.
.
.
xt =  ..
..
..
. . 0 xt−1 +  ..  et


 . 
φ
0 1
r −1 0 . . .
θr −1
φr 0 0 . . . 0
The arima function in R is implemented using this formulation.
State space models
3: ARIMA and RegARMA models, and dlm
9
Outline
1 ARIMA models in state space form
2 RegARMA models in state space form
3 The dlm package for R
4 MLE using the dlm package
5 Filtering, smoothing and forecasting using
the dlm package
6 Final remarks
State space models
3: ARIMA and RegARMA models, and dlm
10
RegARMA models in state space form
Linear regression with AR(2) error
yt = α + β zt + nt
nt = φ1 nt−1 + φ2 nt−2 + et ,
et ∼ NID(0, σ 2 )
Regression model
yt = [1, zt ]xt + nt
xt = [α, β]0
xt = xt −1
AR(2) model
nt = [1, 0]ft
ft = [nt , φ2 nt−1 ]0
φ1 1
et
ft =
f t −1 +
φ2 0
0
State space models
3: ARIMA and RegARMA models, and dlm
11
RegARMA models in state space form
Linear regression with AR(2) error
yt = α + β zt + nt
nt = φ1 nt−1 + φ2 nt−2 + et ,
et ∼ NID(0, σ 2 )
Regression model
yt = [1, zt ]xt + nt
xt = [α, β]0
xt = xt −1
AR(2) model
nt = [1, 0]ft
ft = [nt , φ2 nt−1 ]0
φ1 1
et
ft =
f t −1 +
φ2 0
0
State space models
3: ARIMA and RegARMA models, and dlm
11
RegARMA models in state space form
Linear regression with AR(2) error
yt = α + β zt + nt
et ∼ NID(0, σ 2 )
nt = φ1 nt−1 + φ2 nt−2 + et ,
Combined state space model
xt = [α, β, nt , φ2 nt−1 ]0
yt = [1, zt , 1, 0]xt

1
0
xt = 
0
0
0 0
1 0
0 φ1
0 φ2

 
0
0
 
0
 ft−1 +  0 
et 
1
0
0
State space models
3: ARIMA and RegARMA models, and dlm
12
RegARMA models in state space form
Any two state models can be combined:
yt = f10 xt + f20 zt + εt
xt = G1 xt−1 + wt
zt = G2 zt−1 + ut
xt
0
yt = f10 f2
xt
G1 O
=
zt
O G2
+ εt
zt
x t −1
wt
+
zt−1
ut
So we can take a modular approach to defining
state space models.
This is implemented in the dlm package in R.
State space models
3: ARIMA and RegARMA models, and dlm
13
RegARMA models in state space form
Any two state models can be combined:
yt = f10 xt + f20 zt + εt
xt = G1 xt−1 + wt
zt = G2 zt−1 + ut
xt
0
yt = f10 f2
xt
G1 O
=
zt
O G2
+ εt
zt
x t −1
wt
+
zt−1
ut
So we can take a modular approach to defining
state space models.
This is implemented in the dlm package in R.
State space models
3: ARIMA and RegARMA models, and dlm
13
RegARMA models in state space form
Any two state models can be combined:
yt = f10 xt + f20 zt + εt
xt = G1 xt−1 + wt
zt = G2 zt−1 + ut
xt
0
yt = f10 f2
xt
G1 O
=
zt
O G2
+ εt
zt
x t −1
wt
+
zt−1
ut
So we can take a modular approach to defining
state space models.
This is implemented in the dlm package in R.
State space models
3: ARIMA and RegARMA models, and dlm
13
RegARMA models in state space form
Any two state models can be combined:
yt = f10 xt + f20 zt + εt
xt = G1 xt−1 + wt
zt = G2 zt−1 + ut
xt
0
yt = f10 f2
xt
G1 O
=
zt
O G2
+ εt
zt
x t −1
wt
+
zt−1
ut
So we can take a modular approach to defining
state space models.
This is implemented in the dlm package in R.
State space models
3: ARIMA and RegARMA models, and dlm
13
RegARMA models in state space form
Any two state models can be combined:
yt = f10 xt + f20 zt + εt
xt = G1 xt−1 + wt
zt = G2 zt−1 + ut
xt
0
yt = f10 f2
xt
G1 O
=
zt
O G2
+ εt
zt
x t −1
wt
+
zt−1
ut
So we can take a modular approach to defining
state space models.
This is implemented in the dlm package in R.
State space models
3: ARIMA and RegARMA models, and dlm
13
Outline
1 ARIMA models in state space form
2 RegARMA models in state space form
3 The dlm package for R
4 MLE using the dlm package
5 Filtering, smoothing and forecasting using
the dlm package
6 Final remarks
State space models
3: ARIMA and RegARMA models, and dlm
14
State space models in dlm
yt = ft0 xt + εt
εt ∼ NID(0, σ 2 )
wt ∼ NID(0, Wt )
xt = Gt xt−1 + wt
x0 ∼ NID(m0 , C0 )
Model Parameter List Name Time Varying Name
f
G
σ2
W
m0
C0
State space models
FF
GG
V
W
m0
C0
JFF
JGG
JV
JW
3: ARIMA and RegARMA models, and dlm
15
State space models in dlm
Functions to create dlm objects
Function
Model
dlm
dlmModARMA
dlmModPoly
dlmModReg
dlmModSeas
dlmModTrig
generic DLM
ARMA process
nth order polynomial DLM
Linear regression
Periodic — Seasonal factors
Periodic — Trigonometric form
State space models
3: ARIMA and RegARMA models, and dlm
16
Local level model
yt = ft0 xt + εt
xt = Gt xt−1 + wt
εt ∼ NID(0, σ 2 )
wt ∼ NID(0, Wt )
x0 ∼ NID(m0 , C0 )
xt = `t , ft = 1, Gt = 1.
Suppose σ 2 = 0.8, Wt = 0.1, m0 = 0, C0 = 107 .
dlm() function specification
dlm(FF=1, GG=1, V=0.8, W=0.1, m0=0, C0=1e7)
State space models
3: ARIMA and RegARMA models, and dlm
17
Local level model
yt = ft0 xt + εt
xt = Gt xt−1 + wt
εt ∼ NID(0, σ 2 )
wt ∼ NID(0, Wt )
x0 ∼ NID(m0 , C0 )
xt = `t , ft = 1, Gt = 1.
Suppose σ 2 = 0.8, Wt = 0.1, m0 = 0, C0 = 107 .
dlm() function specification
dlm(FF=1, GG=1, V=0.8, W=0.1, m0=0, C0=1e7)
State space models
3: ARIMA and RegARMA models, and dlm
17
Local level model
yt = ft0 xt + εt
xt = Gt xt−1 + wt
εt ∼ NID(0, σ 2 )
wt ∼ NID(0, Wt )
x0 ∼ NID(m0 , C0 )
xt = `t , ft = 1, Gt = 1.
Suppose σ 2 = 0.8, Wt = 0.1, m0 = 0, C0 = 107 .
dlm() function specification
dlm(FF=1, GG=1, V=0.8, W=0.1, m0=0, C0=1e7)
dlmModPoly() function specification
dlmModPoly(order=1, dV=0.8, dW=0.1)
State space models
3: ARIMA and RegARMA models, and dlm
18
Local level model
> mod <- dlmModPoly(order=1, dV=.8, dW=.1)
> names(mod)
[1] "m0" "C0" "FF" "V" "GG" "W" "JFF" "JV" "JGG" "JW"
> FF(mod)
[,1]
[1,]
1
> GG(mod)
[,1]
[1,]
1
> class(mod)
[1] "dlm"
State space models
3: ARIMA and RegARMA models, and dlm
19
Local trend model
yt = ft0 xt + εt
εt ∼ NID(0, σ 2 )
wt ∼ NID(0, Wt )
xt = Gt xt−1 + wt
x0 ∼ NID(m0 , C0 )
`t
1
xt =
, f t = 1 0 , Gt =
bt
0
0.2
Suppose σ 2 = 0.8, Wt =
0
1
.
1
7
10
0
0
0
, C0 =
, m0 =
0 107
0 .1
0
dlm() function specification
dlm(FF=matrix(c(1,0),nrow=1),
GG=matrix(c(1,0,1,1),ncol=2),
V=0.8, W=diag(c(0.2,0.1)),
m0=c(0,0), C0=diag(c(1e7,1e7)))
State space models
3: ARIMA and RegARMA models, and dlm
20
Local trend model
yt = ft0 xt + εt
εt ∼ NID(0, σ 2 )
wt ∼ NID(0, Wt )
xt = Gt xt−1 + wt
x0 ∼ NID(m0 , C0 )
`t
1
xt =
, f t = 1 0 , Gt =
bt
0
0.2
Suppose σ 2 = 0.8, Wt =
0
1
.
1
7
10
0
0
0
, C0 =
, m0 =
0 107
0 .1
0
dlm() function specification
dlm(FF=matrix(c(1,0),nrow=1),
GG=matrix(c(1,0,1,1),ncol=2),
V=0.8, W=diag(c(0.2,0.1)),
m0=c(0,0), C0=diag(c(1e7,1e7)))
State space models
3: ARIMA and RegARMA models, and dlm
20
Local trend model
yt = ft0 xt + εt
εt ∼ NID(0, σ 2 )
wt ∼ NID(0, Wt )
xt = Gt xt−1 + wt
x0 ∼ NID(m0 , C0 )
`t
1
xt =
, f t = 1 0 , Gt =
bt
0
0.2
Suppose σ 2 = 0.8, Wt =
0
1
.
1
7
10
0
0
0
, C0 =
, m0 =
0 107
0 .1
0
dlmModPoly() function specification
dlmModPoly(order=2, dV=0.8, dW=c(0.2,0.1))
State space models
3: ARIMA and RegARMA models, and dlm
21
Time varying regression model
yt = ft0 xt + εt
xt = Gt xt−1 + wt
εt ∼ NID(0, σ 2 )
wt ∼ NID(0, Wt )
x0 ∼ NID(m0 , C0 )
αt
1 0
.
xt =
, ft = 1 zt , Gt =
0 1
βt
7
1 0
0
10
2
Suppose σ = 15, Wt =
, m0 =
, C0 =
0 2
0
0
0
107
dlmModReg() function specification
dlmModReg(z, dV=15, dW=c(1,2))
State space models
3: ARIMA and RegARMA models, and dlm
22
Time varying regression model
yt = ft0 xt + εt
xt = Gt xt−1 + wt
εt ∼ NID(0, σ 2 )
wt ∼ NID(0, Wt )
x0 ∼ NID(m0 , C0 )
αt
1 0
.
xt =
, ft = 1 zt , Gt =
0 1
βt
7
1 0
0
10
2
Suppose σ = 15, Wt =
, m0 =
, C0 =
0 2
0
0
0
107
dlmModReg() function specification
dlmModReg(z, dV=15, dW=c(1,2))
State space models
3: ARIMA and RegARMA models, and dlm
22
Linear regression with AR(2) errors
yt = α + β zt + nt
nt = φ1 nt−1 + φ2 nt−2 + et ,
State space models
et ∼ NID(0, u)
3: ARIMA and RegARMA models, and dlm
23
Linear regression with AR(2) errors
yt = ft0 xt + εt
xt = Gt xt−1 + wt
εt ∼ NID(0, σ 2 )
wt ∼ NID(0, Wt )
x0 ∼ NID(m0 , C0 )

α
β



1 0 0 0


0 1 0 0



xt = 
 nt , ft = 1 zt 1 0 , Gt = 0 0 φ1 1.
φ2 nt−1
0 0 φ2 0


 
 7

0 0 0 0
0
10
0
0
0
7
0 0 0 0
 


, m0 = 0, C0 =  0 10 0 7 0 
Set σ 2 = 0, Wt = 
0 0 u 0
0
 0 0 10 0 
0 0 0 0
0
0
0
0 107
R (dlm) specification
dlmModReg(z, dV=0, dW=c(0,0)) +
dlmModARMA(ar=c(phi1,phi2), sigma=u)
State space models
3: ARIMA and RegARMA models, and dlm
24
Linear regression with AR(2) errors
yt = ft0 xt + εt
xt = Gt xt−1 + wt
εt ∼ NID(0, σ 2 )
wt ∼ NID(0, Wt )
x0 ∼ NID(m0 , C0 )

α
β



1 0 0 0


0 1 0 0



xt = 
 nt , ft = 1 zt 1 0 , Gt = 0 0 φ1 1.
φ2 nt−1
0 0 φ2 0


 
 7

0 0 0 0
0
10
0
0
0
7
0 0 0 0
 


, m0 = 0, C0 =  0 10 0 7 0 
Set σ 2 = 0, Wt = 
0 0 u 0
0
 0 0 10 0 
0 0 0 0
0
0
0
0 107
R (dlm) specification
dlmModReg(z, dV=0, dW=c(0,0)) +
dlmModARMA(ar=c(phi1,phi2), sigma=u)
State space models
3: ARIMA and RegARMA models, and dlm
24
Outline
1 ARIMA models in state space form
2 RegARMA models in state space form
3 The dlm package for R
4 MLE using the dlm package
5 Filtering, smoothing and forecasting using
the dlm package
6 Final remarks
State space models
3: ARIMA and RegARMA models, and dlm
25
Local level model estimation
Requirements
A “build” function that takes possible
parameter values and returns the model.
Initial values for the parameters.
Example
loclvl <- function(p) {
dlmModPoly(1, dV=exp(p[1]), dW=exp(p[2]))
}
fit <- dlmMLE(oil, parm=c(0,0), build=loclvl)
mod <- loclvl(fit$par)
State space models
3: ARIMA and RegARMA models, and dlm
26
Local level model estimation
Requirements
A “build” function that takes possible
parameter values and returns the model.
Initial values for the parameters.
Example
loclvl <- function(p) {
dlmModPoly(1, dV=exp(p[1]), dW=exp(p[2]))
}
fit <- dlmMLE(oil, parm=c(0,0), build=loclvl)
mod <- loclvl(fit$par)
State space models
3: ARIMA and RegARMA models, and dlm
26
Local level model estimation
Requirements
A “build” function that takes possible
parameter values and returns the model.
Initial values for the parameters.
Example
loclvl <- function(p) {
dlmModPoly(1, dV=exp(p[1]), dW=exp(p[2]))
}
fit <- dlmMLE(oil, parm=c(0,0), build=loclvl)
mod <- loclvl(fit$par)
State space models
3: ARIMA and RegARMA models, and dlm
26
Local level model estimation
Requirements
A “build” function that takes possible
parameter values and returns the model.
Initial values for the parameters.
Example
loclvl <- function(p) {
dlmModPoly(1, dV=exp(p[1]), dW=exp(p[2]))
}
fit <- dlmMLE(oil, parm=c(0,0), build=loclvl)
mod <- loclvl(fit$par)
State space models
3: ARIMA and RegARMA models, and dlm
26
Local level model estimation
>
+
+
>
>
>
loclvl <- function(p) {
dlmModPoly(1, dV=exp(p[1]), dW=exp(p[2]))
}
fit <- dlmMLE(oil, parm=c(0,0), build=loclvl)
mod <- loclvl(fit$par)
V(oil.fit)
[,1]
[1,] 0.0002563
> W(oil.fit)
[,1]
[1,] 2427
> StructTS(oil, type="level")
Variances:
level epsilon
2427
0
State space models
3: ARIMA and RegARMA models, and dlm
27
Local trend model estimation
>
+
+
>
>
>
loctrend <- function(p) {
dlmModPoly(2, dV=exp(p[1]), dW=exp(p[2:3]))
}
fit <- dlmMLE(ausair, parm=c(0,0,0), build=loctrend)
ausair.fit <- loctrend(fit$par)
V(ausair.fit)
[,1]
[1,] 1.6563e-06
> W(ausair.fit)
[,1]
[,2]
[1,] 2.327 0.000000
[2,] 0.000 0.021735
> StructTS(ausair, type="trend")
Variances:
level
slope epsilon
2.2827
0.0265
0.0000
State space models
3: ARIMA and RegARMA models, and dlm
28
Local trend model estimation
>
+
+
>
>
>
loctrend <- function(p) {
dlmModPoly(2, dV=exp(p[1]), dW=exp(p[2:3]))
}
fit <- dlmMLE(ausair, parm=c(0,0,0), build=loctrend)
ausair.fit <- loctrend(fit$par)
V(ausair.fit)
[,1]
Different initialization and
[1,] 1.6563e-06
optimization choices often
> W(ausair.fit)
[,1]
[,2]
given different parameter
[1,] 2.327 0.000000
estimates.
[2,] 0.000 0.021735
> StructTS(ausair, type="trend")
Variances:
level
slope epsilon
2.2827
0.0265
0.0000
State space models
3: ARIMA and RegARMA models, and dlm
28
Linear regression with AR(2) errors
R (dlm) specification
dlmModReg(z, dV=0, dW=c(0,0)) +
dlmModARMA(ar=c(phi1,phi2), sigma=u)
State space models
3: ARIMA and RegARMA models, and dlm
29
Linear regression with AR(2) errors
R (dlm) specification
dlmModReg(z, dV=0, dW=c(0,0)) +
dlmModARMA(ar=c(phi1,phi2), sigma=u)
MLE
regar2 <- function(p) {
dlmModReg(z, dV=.0001, dW=c(0,0)) +
dlmModARMA(ar=c(p[1],p[2]), sigma=exp(p[3]))
}
z <- usconsumption[,1]
fit <- dlmMLE(usconsumption[,2], parm=c(0,0,0),
build=regar2)
mod <- regar2(fit$par)
State space models
3: ARIMA and RegARMA models, and dlm
30
Linear regression with AR(2) errors
R (dlm) specification
dlmModReg(z, dV=0, dW=c(0,0)) +
dlmModARMA(ar=c(phi1,phi2), sigma=u)
V must be positive.
MLE
regar2 <- function(p) {
dlmModReg(z, dV=.0001, dW=c(0,0)) +
dlmModARMA(ar=c(p[1],p[2]), sigma=exp(p[3]))
}
z <- usconsumption[,1]
fit <- dlmMLE(usconsumption[,2], parm=c(0,0,0),
build=regar2)
mod <- regar2(fit$par)
State space models
3: ARIMA and RegARMA models, and dlm
30
BSM model
bsm <- function(p) {
mod <- dlmModPoly() + dlmModSeas(4)
V(mod) <- exp(p[1])
diag(W(mod))[1:3] <- exp(p[2:4])
return(mod)
}
fit <- dlmMLE(austourists, parm=c(0,0,0,0),
build=bsm)
ausbsm <- bsm(fit$par)
State space models
3: ARIMA and RegARMA models, and dlm
31
Outline
1 ARIMA models in state space form
2 RegARMA models in state space form
3 The dlm package for R
4 MLE using the dlm package
5 Filtering, smoothing and forecasting using
the dlm package
6 Final remarks
State space models
3: ARIMA and RegARMA models, and dlm
32
More dlm functions
dlmFilter: Kalman filter. Returns
filtered values of state vectors.
dlmSmooth: Kalman smoother. Returns
smoothed values of state vectors.
dlmForecast: Means and variances of
future observations and states.
State space models
3: ARIMA and RegARMA models, and dlm
33
More dlm functions
dlmFilter: Kalman filter. Returns
filtered values of state vectors.
dlmSmooth: Kalman smoother. Returns
smoothed values of state vectors.
dlmForecast: Means and variances of
future observations and states.
State space models
3: ARIMA and RegARMA models, and dlm
33
More dlm functions
dlmFilter: Kalman filter. Returns
filtered values of state vectors.
dlmSmooth: Kalman smoother. Returns
smoothed values of state vectors.
dlmForecast: Means and variances of
future observations and states.
State space models
3: ARIMA and RegARMA models, and dlm
33
BSM model
Decomposition by Kalman smoothing
ausSmooth <- dlmSmooth(austourists, mod = ausbsm)
x <- cbind(austourists, dropFirst(ausSmooth$s[,c(1,3)]))
colnames(x) <- c("Aus tourists", "Trend", "Seasonal")
plot(x, type = ’o’, main = "Australian tourist numbers")
State space models
3: ARIMA and RegARMA models, and dlm
34
BSM model
●
50
●
40
●
30
●
●
45 20
●
●
●
●
●
●
●
10 25
5
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
−10
●
2000
2002
●
●
●
●
●
●
●
●
●
●
●
●
●
●
2004
●
●
2006
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
0
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
Seasonal
●
●
35
Trend
Aus tourists
60
Australian tourist numbers
●
●
2008
●
●
2010
Time
State space models
3: ARIMA and RegARMA models, and dlm
35
BSM model
Forecasting by Kalman filter
Filt <- dlmFilter(austourists, mod = ausbsm)
Fore <- dlmForecast(Filt, nAhead = 20)
fsd <- sqrt(unlist(Fore$Q))
pl <- Fore$f + qnorm(0.05, sd = fsd)
pu <- Fore$f + qnorm(0.95, sd = fsd)
fc <- list(mean=Fore$f, lower=pl, upper=pu,
x=austourists, level=90)
plot.forecast(fc, main="Australian tourists")
State space models
3: ARIMA and RegARMA models, and dlm
36
BSM model
20
30
40
50
60
70
80
Australian tourists
2000
2005
State space models
2010
3: ARIMA and RegARMA models, and dlm
2015
37
Outline
1 ARIMA models in state space form
2 RegARMA models in state space form
3 The dlm package for R
4 MLE using the dlm package
5 Filtering, smoothing and forecasting using
the dlm package
6 Final remarks
State space models
3: ARIMA and RegARMA models, and dlm
38
Final remarks
State space models come in lots of flavours. We
have only touched the surface.
We haven’t even mentioned the Bayesian
flavours.
State space models are a flexible way of
handling lots of time series models and provide
a framework for handling missing values,
likelihood estimation, smoothing, forecasting,
etc.
State space models
3: ARIMA and RegARMA models, and dlm
39
Final remarks
State space models come in lots of flavours. We
have only touched the surface.
We haven’t even mentioned the Bayesian
flavours.
State space models are a flexible way of
handling lots of time series models and provide
a framework for handling missing values,
likelihood estimation, smoothing, forecasting,
etc.
State space models
3: ARIMA and RegARMA models, and dlm
39
Final remarks
State space models come in lots of flavours. We
have only touched the surface.
We haven’t even mentioned the Bayesian
flavours.
State space models are a flexible way of
handling lots of time series models and provide
a framework for handling missing values,
likelihood estimation, smoothing, forecasting,
etc.
State space models
3: ARIMA and RegARMA models, and dlm
39
Recommended References
1
2
3
4
RJ Hyndman, AB Koehler, J Keith Ord, and
RD Snyder (2008). Forecasting with
exponential smoothing: the state space
approach. Springer
AC Harvey (1989). Forecasting, structural
time series models and the Kalman filter.
Cambridge University Press
J Durbin and SJ Koopman (2001). Time series
analysis by state space methods. Oxford
University Press
G Petris, S Petrone, and P Campagnoli (2009).
Dynamic Linear Models with R. Springer
State space models
3: ARIMA and RegARMA models, and dlm
40
Contact details
www.robjhyndman.com
Rob.Hyndman@monash.edu
State space models
3: ARIMA and RegARMA models, and dlm
41