The classical approach:
Method
Time series regression
Decomposition
Pros
• Easy to implement
• Fairly easy to interpret
• Covariates may be added (normalization)
• Inference is possible
(though sometimes questionable)
• Easy to interpret
• Possible to have dynamic seasonal effects
• Cyclic components can be estimated
Cons
• Static
• Normal-based inference not generally reliable
• Cyclic component hard to estimate
• Descriptive (no inference per def)
• Static in trend

Explanation to the static behaviour:
The classical approach assumes all components except the irregular ones
(i.e.
 t and IR t
) to be deterministic , i.e. fixed functions or constants
To overcome this problem, all components should be allowed to be stochastic , i.e. be random variates.
A time series y t should from a statistical point of view be treated as a stochastic process .
We will interchangeably use the terms time series and process depending on the situation.

Stationary and non-stationary time series
20
10
0
Index 10 20 30 40 50 60 70 80 90 100
Characteristics for a stationary time series:
• Constant mean
• Constant variance
3000
2000
1000
0
Index
 A time series with trend is non-stationary!
100 200 300

A stationary times series can be modelled on basis of the serial correlations in it.
A non-stationary time series can be transformed into a stationary time series, modelled and back-transformed to original scale (e.g. for purposes of forecasting)
– models
This part has to do with the transformation
Auto Regressive,
Integrated,
Moving Average
These parts can be modelled on a stationary series

Different types of transformation
1. From a series with linear trend to a series with no trend:
First-order differences z t
= y t
– y t – 1
MTB > diff c1 c2

10
5
20
15
0
Note that the differences series varies around zero.
Variable linear trend no trend

2. From a series with quadratic trend to a series with no trend:
Second-order differences w t
= z t
– z t – 1
= ( y t
– y t – 1
) – ( y t – 1
– y t – 2
) = y t
– 2 y t – 1
+ y t – 2
MTB > diff 2 c3 c4

10
5
20
15
0
Variable quadratic trend no trend 2

3. From a series with non-constant variance (heteroscedastic) to a series with constant variance (homoscedastic):
Box-Cox transformations (per def 1964) g
  t



 y t
    ln

 y t

  
1 for
 
0 and y t for
 
0 and y t
  
0
  
0
Practically
 is chosen so that y t
+
 is always > 0
Simpler form: If we know that y t measurements) is always > 0 (as is the usual case for g
  t







1
4 ln
1 y t y t y y t t y t if modest heterosced asticity
" if pronounced heterosced asticity if heavy heterosced asticity if extreme heterosced asticity

The log transform (ln y t
) usually also makes the data ”more” normally distributed
Example: Application of root (
 y t
) and log (ln y t
) transforms
15
10
25
20
5
0
Variable original root log

AR
Consider the model y t
= δ +
 ·y t
–1
+ a t with { a t
} i.i.d with zero mean and constant variance = σ 2 and where δ (delta) and

(phi) are (unknown) parameters
Set
δ
= 0 by sake of simplicity

E ( y t
) = 0
Let R (k) = Cov ( y t
,y t-k
) = Cov ( y t
,y t+k
) = E ( y t
·y t-k
) = E ( y t
·y t+k
)

R (0) = Var( y t
) assumed to be constant

Now:
R (0) = E ( y t
·y t
) = E ( y t
·
(
 ·y t1
+ a t
) =
 · E
( y t
·y t1
) + E ( y t
·a t
) =
=
 ·R (1) + E ((
 ·y t1
+ a t
) ·a t
) =
 ·R (1) +
 · E ( y t1
·a t
) + E ( a t
·a t
)=
=
 ·R (1) + 0 + σ 2 (for a t is independent of y t1
)
R (1) = E ( y t
·y t+ 1
) = E ( y t
·
(
 ·y t
+ a t+ 1
) =
 · E
( y t
·y t
) + E ( y t
·a t+ 1
) =
=
 ·R
(0) + 0 (for a t+1 is independent of y t
)
R (2) = E ( y t
·y t+ 2
) = E ( y t
· (
 ·y t+ 1
+ a t+ 2
) =
 · E ( y t
·y t+ 1
) +
+ E ( y t
·a t+ 2
) =
 ·R (1) + 0 (for a t+ 1 is independent of y t
)


R (0) =
 ·R
(1) +
σ 2
R (1) =
 ·R
(0)
R (2) =
 ·R (1)
…
Yule-Walker equations

R ( k ) =
 ·R
( k
– 1) =…=  k ·R
(0)
R (0) =

2 ·R
(0) +
σ 2 
R ( 0 )


2
1
 
2

Note that for R (0) to become positive and finite (which we require from a variance) the following must hold:

2 
1
  
1
This in effect the condition for an AR (1)-process to be weakly stationary
Note now that
Corr ( y t
, y t
 k
)
  k

Cov (
Var ( y t
) y t
, y t

Var
 k
(
) y t
 k
)

  k

 k 
R
R
  k
R ( k )
R ( 0 )

R ( 0 )

R ( k )
R ( 0 )

ρ k is called the Autocorrelation function (ACF) of y t
”Auto” because it gives correlations within the same time series.
For pairs of different time series one can define the Cross correlation function which gives correlations at different lags between series.
By studying the ACF it might be possible to identify the approximate magnitude of


Examples:
ACF for AR(1), phi=0.1
1
0.8
0.6
0.4
0.2
0
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 k
ACF for AR(1), phi=0.3
1
0.8
0.6
0.4
0.2
0
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 k

ACF for AR(1), phi=0.5
1
0.8
0.6
0.4
0.2
0
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
ACF for AR(1), phi=0.8
1
0.8
0.6
0.4
0.2
0
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
ACF for AR(1), phi=0.99
1
0.8
0.6
0.4
0.2
0
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

ACF for AR(1), phi=-0.1
1
0.8
0.6
0.4
0.2
0
-0.2
-0.4
-0.6
-0.8
-1
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
ACF for AR(1), phi=-0.5
0.2
0
-0.2
-0.4
-0.6
-0.8
-1
1
0.8
0.6
0.4
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
ACF for AR(1), phi=-0.8
1
0.8
0.6
0.4
0.2
0
-0.2
-0.4
-0.6
-0.8
-1
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

The look of an ACF can be similar for different kinds of time series, e.g. the ACF for an AR(1) with

= 0.3 could be approximately the same as the ACF for an Auto-regressive time series of higher order than 1 (we will discuss higher order AR-models later)
To do a less ambiguous identification we need another statistic:
The Partial Autocorrelation function (PACF):
υ k
= Corr ( y t
,y t-k
| y t-k +1
, y t-k +2
,…, y t1
) i.e. the conditional correlation between y t and y t-k observations in-between.
given all
Note that –1
 υ k

1

A concept sometimes hard to interpret, but it can be shown that for AR(1)-models with
 positive the look of the PACF is
1.00
0.00
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 k and for AR(1)-models with
 negative the look of the PACF is
1
0
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
-1 k

Assume now that we have a sample y
1
, y
2
,…, y n assumed to follow an AR(1)-model.
from a time series
Example:
Monthly exchange rates
DKK/USD 1991-1998
10
8
6
4
2
0

The ACF and the PACF can be estimated from data by their sample counterparts:
Sample Autocorrelation function (SAC): r k
 t n

 k

1
( y t
 t n 

1
( y t y )( y t
 k
 y )
2
 y ) if n large, otherwise a scaling might be needed
Sample Partial Autocorrelation function (SPAC)
Complicated structure, so not shown here

The variance function of these two estimators can also be estimated

Opportunity to test
H
0
:
 k
= 0 vs.
H a
:
 k

0 or
H
0
:
 k
= 0 vs.
H a
:
 k

0 for a particular value of k.
Estimated sample functions are usually plotted together with critical limits based on estimated variances.

Example (cont) DKK/USD exchange:
SAC:
SPAC: Critical limits

Ignoring all bars within the red limits, we would identify the series as being an AR(1) with positive

.
The value of
 is approximately 0.9 (ordinate of first bar in SAC plot and in SPAC plot)

AR(2):
AR(3):
AR( p ):
y t
   
1 y t

1
 
2 y t

2
 a t y t
   
2 y t

2
 a t or y t2 must be present y t
   
1 y t

1
 
2 y t

2
 
3 y t

3 or other combinations with

3 y t3
 a t y t
   
1 y t

1

...
  p y t
 p
 a t i.e. different combinations with
 p y t-p

Stationarity conditions:
For p > 2, difficult to express on closed form.
For p = 2: y t
   
1 y t

1
 
2 y t

2
 a t
The values of

1 and

2 must lie within the blue triangle in the figure below:

Typical patterns of ACF and PACF functions for higher order stationary AR-models (AR( p )):
ACF: Similar pattern as for AR(1), i.e. (exponentially) decreasing bars, (most often) positive for

1 negative.
positive and alternating for

1
PACF: The first p values of k are non-zero with decreasing magnitude. The rest are all zero (cut-off point at p )
(Most often) all positive if

1 negative positive and alternating if

1

Examples:
AR(2),

1 positive:
1
ACF
1
PACF
0
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
0
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
AR(5),

1 negative:
ACF PACF
1 1
0
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
0
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
-1 -1