Model specification (identification)
We already know about the sample autocorrelation function (SAC):


n
rk
Y  Y  Y
 Y  Y 
t  k 1
t k
2
t
n
t 1
Y

t
Properties:
• Not unbiased (since a ratio between two random variables)
• Bias decreases with n
• Variance complicated, common to use general large-sample results
Large-sample results (asymptotics):
For large n the random vector
H
n  r1  1 ,
n  r2  2 ,,
n  rm  m 
Note : n  rj   j  
rj   j
1 n
A kind of st udent izing
;
has an approximate multivariate normal distribution with
zero mean vector and covariance matrix ( cij ) where
cij 
 

r  
2







2







2







2






r i
r j
r i
r j
i
r
r j
j
r
r i
i
j
r

This gives that
c 
c

 Var (rk)  0 as n 
rk is asymptotic
ally N   k , kk  (i.e. rk  kk )
n 
n

ckj
Covrk , rj  
does not diminish as n 
ckk  c jj
Hence, the distribution of rk will depend on the correlation structure of Yt and
accordingly on the model behind (i.e. if it is and AR(1), an ARMA(2,1) etc.)
For an AR(1), i.e. Yt =   Yt – 1 + et


1  1   2  1   2k
Var rk   ckk   
n 
1 2





 1 2




n
 2  k   2k   
2
1
1


  
 n 1   2
k 0
k large
i.e. not dependent on k for large lags
For an MA(q)
q

1 
2

Varrk   ckk   1  2    j  for k  q
n 
j 1

i.e. not dependent on k
after the qth lag
For white noise
1
1
Varrk  
since  j  
n
0

j 0
 ckk  
j0
r  
Only some terms 0
 2
1
n
i.e. whitenoise - based
95 % bounds
k

1 

 2
 1  2   rj2 
n 
j 1

i.e. bounds for rk based on an
MAk  - process with  j
estimated by rj
Partial autocorrelation function

kk   k  Corr Yt , Yt  k Yt 1 , Yt  2 ,, Yt k 1

Describes the “specific” part of the correlation between Yt and Yt – k that is not
due to successive serial correlations between the variables Yt – 1 , Yt – 2 , …, Yt – k .
Partial correlations are used for other types of data as well (for instance in linear
models of cross-sectional data.
Patterns
• For an AR(p)-process, k cuts off after lag p (i.e. the same type of behaviour
like k has for an MA(q)-process
• For an MA(q)-process k shows approximately the same pattern as does k
for an AR(p)-process
Estimation from data , Sample Partial Autocorrelation function (SPAC):
No explicit formula, estimation has to be made recursively
Properties of SPAC:
More involved, but for an AR(p)-process SPAC-values for lags greater than p are
approximately normally distributed with zero mean and variance 1/n
Extended Autocorrelation function (EACF)
One (of several) tool to improve the choice of orders of ARMA(p, q)-processes.
Very clear as a theoretical function, but noisy when estimated on series not too
long.
AR, MA or ARMA?
No pattern at all?
EACF table for Y
AR/MA
0 1 2 3 4 5 6 7 8 9 10 11 12 13
0ooooooooooo o o o
1ooooooooooo o o o
2xoooooooooo o o o
3ooxoooooooo o o o
4xoooooooooo o o o
5xxooooooooo o o o
6oxoooxooooo o o o
7oooooxooooo o o o
True process: Yt = 1.3 + 0.2Yt – 1 + et – 0.1et – 1
ARMA(0,0) or
ARMA(1,0)?
Model selection from more analytical tools
Dickey-Fuller Unit-Root test
H0: The process Yt is difference non-stationary (Yt is stationary)
Ha : The process Yt is stationary
Augmented Dickey-Fuller test statistic (ADF):
, X t an AR p  - process
Yt  Yt 1  X t

Yt  Yt  Yt 1    1Yt 1  X t    1Yt 1  1 X t 1     p X t  p  et
If  =1 (difference non-stationary) 
X t  Yt  Yt 1 
Yt  Yt 1  1  1  Yt 1  1  Yt 1  Yt  2      p  Yt  p  Yt  p 1   et


0
Fit the model
Yt  Yt 1    1  Yt 1  1  Yt 1  Yt 2      p  Yt  p  Yt  p 1   et 
 a  Yt 1  1  Yt 1  Yt 2      p  Yt  p  Yt  p 1   et
and test H0: a = 0 (difference non-stationary) vs. Ha : a < 0 (stationary)
using the test statistic
ADF 
aˆ
s.e.aˆ 
However, not t-distributed under H0. Another sampling distributions has been
derived and tables (programmed in R)
Akaike’s criteria
For an ARMA(p,q)-process, let k = p + q + 1
and find the values of the parameters p, q, 1 , …, p , 1 , …, q of the model
1  B 
1



p
q

B
Y



1



B





B
et
p
t
1
q
that minimizes
• –2log{max L(p, q, 1 , …, p , 1 , …, q )} + 2k
AIC [Akaike’s Information Criterion]. Works well when the true process have
(at least one) infinite order
• –2log{max L(p, q, 1 , …, p , 1 , …, q )} + klog(n)
BIC [(Schwarz) Bayesian Information Criterion]. Works well when we “know” that
the true process is a finite-order ARMA(p,q)