Time Series Analysis
Pros & cons
Jonas Mellin
HELICOPTER – Initial presentation of HS/IF
Jonas Mellin, 2013
1
Overview
• Linear state space model
• Trends & seasons
• Basic structural time series
– Combining parameters
•
•
•
•
•
Types of models
Usefulness
Parameter estimation
Pros & cons
R packages
HELICOPTER – Initial presentation of HS/IF
Jonas Mellin, 2013
2
Basic: Linear State Space
• State equation (first order AR eq.)
𝞪𝑡+1 = 𝑇𝑡 𝞪𝑡 + 𝑅𝑡 𝞰𝑡 ,
𝞰𝑡 ~ 𝑁 0, 𝑄𝑡
• Observation equation
𝑦𝑡 = 𝑍𝑡 𝞪𝑡 + 𝞮𝑡 ,
𝞮𝑡 ~ 𝑁 0, 𝐻𝑡
• Machine learning -> constants
• Extensible to multiple states, observations, lags
HELICOPTER – Initial presentation of HS/IF
Jonas Mellin, 2013
3
Trends
𝑦𝑡 = 𝞵𝑡 + 𝞮𝑡 ,
𝞮𝑡 ~ 𝑁 0, 𝞼2𝞮
𝞵𝑡+1 = 𝞵𝑡 + 𝞶𝑡 + 𝞷𝑡 ,
𝞷𝑡 ~ 𝑁 0, 𝞼2𝞷
𝞶𝑡+1 = 𝞶𝑡 + 𝞯𝑡 ,
𝞯𝑡 ~ 𝑁 0, 𝞼2𝞯
𝞵𝑡
𝞵𝑡+1
1
𝑦𝑡 = (1 0) 𝞶 +𝞮𝑡 , 𝞶
=
0
𝑡
𝑡+1
HELICOPTER – Initial presentation of HS/IF
Jonas Mellin, 2013
1
1
𝞵𝑡
𝞷𝑡
𝞶𝑡 + 𝞯𝑡
4
Seasons
γ𝑡+1 = −
λ𝑗 =
2π𝑗
𝑠
𝑠
2
[ ]
∗
( γ𝑗 cos λ𝑗 𝑡+γ𝑗 sin λ𝑗 𝑡)
𝑗=1
,𝑗=
𝑠
1, … , [ ]
2
HELICOPTER – Initial presentation of HS/IF
Jonas Mellin, 2013
5
Basic structural time series
• Any combination of
– error, trend, and season
• For example
– 𝑦𝑡 = μ𝑡 + γ𝑡 + ε𝑡
– α𝑡 = μ𝑡 υ𝑡 γ𝑡 γ𝑡−1 … γ𝑡−𝑠+2 ′
– 𝑍𝑡 = 𝑍 μ , 𝑍 γ , 𝑇𝑡 = 𝑑𝑖𝑎𝑔(𝑇 μ , 𝑇 γ )
– 𝑅𝑡 = 𝑑𝑖𝑎𝑔(𝑅 μ , 𝑅 γ ), 𝑄𝑡 = 𝑑𝑖𝑎𝑔(𝑄 μ , 𝑄 γ )
HELICOPTER – Initial presentation of HS/IF
Jonas Mellin, 2013
6
𝑍 μ = (1,0), 𝑍[γ] = (1,0, … , 0)
𝑇μ
1
=
0
1
, 𝑇γ
1
−1
1
= 0
0
𝑅 μ = 𝐼2 ,
𝑄𝜇 =
⋮
−1
0
1
0
⋯
⋱
⋯
−1
0
0
𝑅 γ = 1,0, … , 0
𝞼2𝞷
0
0
𝞼2𝞯
,
1
⋮
−1
0
0
0
′
𝑄 γ =𝞼2ω
HELICOPTER – Initial presentation of HS/IF
Jonas Mellin, 2013
7
Season (s=4)
• α𝑡 = μ𝑡 υ𝑡 γ𝑡 γ𝑡−1 γ𝑡−2 ′
• 𝑍𝑡 = 1 0 1 0 0
1 1 0
0
0
1 0
0 1 0
0
0
0 1
• 𝑇𝑡 = 0 0 −1 −1 −1 , 𝑅𝑡 = 0 0
0 0 1
0
0
0 0
0 0 0
1
0
0 0
2
𝞼𝞷 0
0
• 𝑄𝑡 = 0
𝞼2𝞯
0
0
0
𝞼2ω
HELICOPTER – Initial presentation of HS/IF
Jonas Mellin, 2013
0
0
1
0
0
8
Basic: Linear State Space: Recap
• State equation (first order AR eq.)
𝞪𝑡+1 = 𝑇𝑡 𝞪𝑡 + 𝑅𝑡 𝞰𝑡 ,
𝞰𝑡 ~ 𝑁 0, 𝑄𝑡
• Observation equation
𝑦𝑡 = 𝑍𝑡 𝞪𝑡 + 𝞮𝑡 ,
𝞮𝑡 ~ 𝑁 0, 𝐻𝑡
HELICOPTER – Initial presentation of HS/IF
Jonas Mellin, 2013
9
Types of model
•
•
•
•
•
Local model/structural time series
Linear/(non-linear) state space
Gaussian/(non-Gaussian)
Univariate/multivariate
Can model ARMA(p,q) and ARIMA(p,q)
– Box-Jenkins
HELICOPTER – Initial presentation of HS/IF
Jonas Mellin, 2013
10
Usefulness
•
•
•
•
•
•
•
Filtering
Smoothing
Estimating missing observations
Forecasting
Simulations
Compare and contrast models
Dynamic factor analysis
HELICOPTER – Initial presentation of HS/IF
Jonas Mellin, 2013
11
Parameter estimation
• Maximum likelihood estimation
– Loglikelihood
• log 𝐿
𝑛
= − log
2
2π −
1
2
𝑛
𝑡=1(log 𝐹𝑡
+
𝑣𝑡2
)
𝐹𝑡
– Maximize this
• 𝞼2ε and 𝞼2𝜂 converge, given 𝞪1 or P1, where P1 is the
initial variance of y1
HELICOPTER – Initial presentation of HS/IF
Jonas Mellin, 2013
12
Advantages & disadvantages
• Advantages
–
–
–
–
Mature
Generic
Models can be analyzed (why-perspective)
Multivariate analysis possible
• Disadvantages
– Cannot find optimal model itself,
• search-based optimization required
– More complex than ARMA, ARIMA
– Can be hard to specify relations
HELICOPTER – Initial presentation of HS/IF
Jonas Mellin, 2013
13
Examples of existing packages
• R language
– MARSS
• Multi-variate analysis
• Flexible
– KFAS
• Univariate analysis
• Less flexible
HELICOPTER – Initial presentation of HS/IF
Jonas Mellin, 2013
14
References
• Durbin, J 2012, Time series analysis by state space methods,
2nd ed., Oxford University Press, Oxford.
• Holmes, E, Ward, E & Wills, K 2013, MARSS: Multivariate
Autoregressive State-Space Modeling, viewed
<http://cran.r-project.org/web/packages/MARSS/>.
• Holmes, EE, Ward, EJ & Wills, K 2012, ‘MARSS: Multivariate
autoregressive state-space models for analyzing time-series
data’, The R Journal, vol. 4, no. 1, p. 30.
• http://cran.r-project.org/web/views/TimeSeries.html
• http://www.abs.gov.au/websitedbs/D3310114.nsf/home/Ti
me+Series+Analysis:+The+Basics
HELICOPTER – Initial presentation of HS/IF
Jonas Mellin, 2013
15