Autocorrelation in Time Series
KNNL – Chapter 12
Issues in Autocorrelated Data
• When error terms are correlated (not independent),
problems occur when using ordinary least squares
(OLS) estimates
– Regression Coefficients are Unbiased, but not Minimum
Variance
– MSE underestimates s2
– Standard errors of regression coefficients based on OLS
underestimate the true standard error
–  Inflated t and F statistics and artificially narrow
confidence intervals
Autocorrelated Errors (1st Order)   t   t 1  ut
where ut  uncorrelated disturbances (typically assumed to be normal)
First-Order Model - I
First-Order Autoregressive Model ( AR(1)) :
Simple Regression: Yt   0  1 X t   t
 t   t 1  ut
t  1,..., n
  autoregression parameter with   1
ut ~ N  0, s 2  and independent
Generalizes to Multiple Regression:
Yt   0  1 X t1  ...   p 1 X t , p 1   t
 t   t 1  ut
t  1,..., n
Properties of Errors (assumption regarding 1 for model consistency):

s2 
1 ~ N  0,
2 
 1  
 2  1  u2  E  2    E 1  E u2   0
 s2 
s2
2
s  2    s 1  s u2    
s 
2 
1


1  2


2
2
2
2
s 2
Covariance: s  2 , 1  s  1  u2 , 1  s  2   s u2 , 1  s  2   0 
1  2
2
Corrrelation:   2 , 1 
s  2 , 1

s  2 s 1
2
s 2
1  2
s
2
s
2
1  2 1  2

2
First-Order Model - II
In General:

 t   t 1  ut     t 2  ut 1   ut    t  2   ut 1  ut  ...    s ut  s
2
s 0
E  t   0

s2
 s
  2s 2
2
2s
s  t   s   ut  s     s ut  s   s   
1  2
s 0
 s 0
 s 0
2
2
 ss 2
Covariance: s  t ,  t  s  
1  2
Correlation:   t ,  t  s    s
s0
s0
1

2


1

2

s
 2
σ 2 ε 

1
2 
1  

  n 1  n  2  n 3

AR(2) :  t  1 t 1   2 t  2  ut
 n 1 

 n2 
 n 3 
1




Even Higher or models can be fit as well.
Test For Independence - Durbin-Watson Test
Yt   0  1 X t1  ...   p 1 X t , p 1   t
 t   t 1  ut ut ~ NID  0, s 2 
 1
H 0 :   0  Errors are uncorrelated over time
H A :   0  Positively correlated
1) Obtain Residuals from Regression
2) Compute Durbin-Watson Statistic (given below)
3) Obtain Critical Values from Table B.7, pp. 1330-1331 (R will provide a p-value)
If DW  d L  p  1, n   Reject H 0
n
Test Statistic: DW 
 e  e 
2
t 1
t
t 2
If DW  dU  p  1, n   Conclude H 0
n
e
t 1
2
t
s 2
E  t   0 E  t  t 1 
1  2

n
 e  e 
t 2
t
t 1
2
s 2
  e   e  2 et et 1  2 e  2n
1  2
t 2
t 2
t 2
t 1
n
n
2
t
n
2
t 1
 Under H 0 , expect DW  2
n
2
t
Otherwise Inconclusive
Autocorrelation - Remedial Measures
• Determine whether a missing predictor variable can
explain the autocorrelation in the errors
• Include a linear (trend) term if the residuals show a
consistent increasing or decreasing pattern
• Include seasonal dummy variables if data are quarterly
or monthly and residuals show cyclic behavior
• Use transformed Variables that remove the (estimated)
autocorrelation parameter (Cochrane-Orcutt and
Hildreth-Lu Procedures)
• Use First Differences
• Estimated Generalized Least Squares
Transformed Variables
Suppose  is known: Yt   0  1 X t   t
 t   t 1  ut
Let Yt '  Yt  Yt 1    0  1 X t   t      0  1 X t 1   t 1  
 0 1     1  X t   X t 1     t   t 1    0 1     1  X t   X t 1   ut
 Yt '   0'  1' X t'  ut
(Standard Simple linear regression with independent errors)
where:
Yt '  Yt  Yt 1
X t'  X t   X t 1
 0'   0 1   
1'  1
In Practice, we need to estimate  with a sample based value r
Yt '  Yt  rYt 1
X t'  X t  rX t 1
^
Fit: Y '  b0'  b1' X ' and if errors are uncorrelated, back transform to:
b0'
Y  b0  b1 X where: b0 
1 r
^
s b0  
s b0' 
1 r
b1  b1'
s b1  s b1' 
Cochrane-Orcutt Method
• Start by estimating  in Model: t = t-1 + ut by
regression through the origin for residuals (see below)
• Fit transformed regression model (previous slide)
• Check to see if new residuals are uncorrelated (DurbinWatson test), based on the transformed model
• If uncorrelated, stop and keep current model
• If correlated, repeat process with new estimate r based
on current regression residuals from the original (back
transformed) model
n
r
e
t 2
n
e
t 1 t
e
t 2
2
t 1
Hildreth-Lu and First Difference Methods
• Hildreth-Lu Method
– Find value of r (between 0 and 1) that minimizes the SSE for
the transformed model by grid search
– Apply the transformed analysis based on the estimated r
• First Differences Method
– Uses  = 1 in transformed model (Yt’ = Yt – Yt-1 Xt’ = Xt – Xt-1 )
– Set b0’ = 0 and fits regression through origin of Y’ on X’
– When back-transforming:
b0  Y  b X
'
1
b1  b
'
1
Forecasting with Autocorrelated Errors
Makes use of any of the 3 estimation techniques (C-O, H-L, First Differences):
Yt   0  1 X t   t
 t   t 1  ut
 Yt   0  1 X t   t 1  ut
  Yn 1   0  1 X n 1   n  un 1
3 Elements:
^
1. Expected Value:  0  1 X n 1 Estimated as Y n 1  b0  b1 X n 1
2. Multiple of period n Error Term:  n
Estimated as ren
3. Current disturbance un 1 ~ N  0, s 2 
Forecast for period n  1 (note the notation is "Forecast", not F -distribution :
^
Fn 1  Y n 1  ren
Standard Error of the Prediction (based on transformed model):


'

X n 1  X ' 
1
2

s pred  MSE ' 1 
 n
2
n

1

X i'  X ' 



i2
Approximate 95% PI: Fn 1  t 1   2  ; n  3 s pred




(First Differences has n - 2 df)