Regression with Autocorrelated Errors
U.S. Wine Consumption and Adult
Population – 1934-2002
Data Description
• Y=U.S. Annual Wine Consumption (Millions of
Gallons)
• X=U.S. Adult Population (Millions of People)
• Years – 1934-2002 (Post Prohibition)
• Model:
Yt   0  1 X t   t
 t   t  1 t 1  ...   q t q
 t ~ iid 0, 
2
     
Ordinary Least Squares Regression
Regression Statistics
Multiple R
0.965612383
R Square
0.932407274
Adjusted R Square
0.931398427
Standard Error
48.64438813
Observations
69
Intercept
apop_m
ANOVA
df
1
67
68
Regression
Residual
Total
Coefficients
-347.9736
4.3092
SS
2186985.91
158540.53
2345526.43
Standard Error
t Stat
21.9895
-15.8245
0.1417
30.4012
^
 e  e 
2
t 1
t
n
e
t 1
2
t

19008.8044
 0.11989871
158540.53
d L  p  1, n  70   1.58 dU  p  1, n  70   1.64
DW  dU  Autocorrel ation is present
Significance F
0.0000
Upper 95%
-304.0824
4.5921
 e  2366.28  48.64
n
t 2
Lower 95%
-391.8649
4.0263
F
924.23
^
W t  347.97  4.31Pt
Durbin - Watson Test : DW 
P-value
0.0000
0.0000
MS
2186985.91
2366.28
Residuals versus Time
150
100
Residuals
50
0
-50
-100
1
3
5
7
9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49 51 53 55 57 59 61 63 65 67 69
Covariance Structure (q=1)
Yt   0  1 X t   t
 t   t   t 1  t ~ iid 0,  2 
    
1    1
2
Assuming : E  1   0 V  1  
1  2
 2 2  2 1   2    2 2
2
 E  2   0 V  2   V  2    V  1    


1  2
1  2
1  2
2
2
 2
COV  1 ,  2   COV  1 , 2  1   V  1  
1  2
In General :
2
E  t   0 V  t  
1  2
 k 2
COV  t ,  t  k  
1  2
 
 1  
 1
   n 1 
 


1
  n2 
  2  
2  
V ε   V 


2
 

1




 
 
 n 1

n2
  



1


 n
  2  1   2 V  t   V  t   COV  t ,  t 1 


COV  t ,  t 1 
V  t 
Generalized Least Squares (q=1)
 1

2  
V  V Y   V ε  
1  2  
 n 1
 

1

 n2
 1  2

 
 0
1
Define : T  
 

 0
 0
  n 1 

  n2 

 


1 
0
0

0
1

0
1


0
0

 

0
0
0
0
 1
 
0

0
0



0
1
Consider t he Case where n  4 :
 1  2


T 1 V  
 0

 0
0
0
1
0

1
0

 1  2

2

 0
T 1 VT 1' 
1  2  0

 0
0

2
0 
2
0 1  

1
 1  2
1  2
1


 2
 3
 

2

1

2
1

0
 2 1  2
 1   2 
1  2
0
0
3

2
2

  1  2

1 
 1  2

 0
 0

 0
 3 1  2  1  2

 2 1   2   0
 1   2    0

2
1     0
 1  2
1  2
0
 2 1  2
 1   2 
1  2
0
0

0
1

0
1
0
0
 3 1  2 

 2 1   2 
 1   2  

1   2 
0 

0 

 

1 
1   2
0
0
0 


1  2
0
0 
2  0
  2 I  V T 1 Y  T 1 VT 1'   2 I  Transform Y*  T 1 Y  T 1 Xβ  T 1ε
2
2
1   0
0
1 
0 


0
0
1   2 
 0


Estimated GLS (q=1)
^
1 n 2
 (0)   et
n t 1
^
^ 1
T

^ 2
 
 ^
  ( 0)
^

 

 0



 0

 0
1 n
 (1)   et et 1
n t 2
^
^

0
0

0
0
1
0

0
0

1

0
0





^
 (1)
0


1
0
0

0


^
 ( 0)
^
^
^
  (0)    (1)


0


0

0


0

1
^
0
^ 2
^
1
^
β EG LS
^ 1 ^ 1




X'
T
'
T
X




1
^ 1
X' T ' T
^ 1 ^ 1





V  β EG LS    e 
X'
T
'
T
X






^
^
^ 2
^ ^
^


(
0
)



(
1
)



s2  
n  p '1
^ 1
^ 2
e
Y
^
1


SE   i  


^
ti 
 EGLS ,i
 ^

SE   EGLS ,i 


^
^
s 2 /  ( 0)
1
^
^
^

 ^


 Y  X β EG LS ' T ' T  Y  X β EG LS 




n  p '1
ti 
i
 ^ 
SE   i 


Estimated GLS (q=1) – Wine/Population Data
^
^
 (0)  2297.69  (1)  2147.89
^
  0.9348
^ 2
  289.82
0
0 
0
0
0
 0.3552
- 0.9348

1
0

0
0
0



1
^

0
- 0.9348 1 
0
0
0
T 











0
0
0  - 0.9348
1
0


0
0
0 
0
- 0.9348 1

^
^ 2
- 347.23
β EGLS  
 e  280.5479

 4.2540 
^ ^
4.2540
4.2540

 5482.225 - 31.515
V  β EGLS   
t


 9.358
1

0.2066 
0.2066 0.4546

  - 31.515
s 2  4.39
 
SE   i  
 
^
^
^
s 2 /  (0)  .04372
ti 
i
^ 
SE   i 
 

0.9348
 21.38
0.04372
Transformed residuals versus Year - EGLS (q=1)
60
40
Residuals
20
0
-20
-40
-60
1
3
5
7
9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49 51 53 55 57 59 61 63 65 67 69
DW  1.67 d L  p  1, n  70  1.58 dU  p  1, n  70  1.64
DW  dU  Do not concludeAu tocorrelat ion is present
Untransformed data and Fitted Equation from EGLS
700
600
500
Wine Sales
400
Y
Yhat_w
300
200
100
0
50
70
90
110
130
150
Population
170
190
210
230
250
Estimated GLS (General q) - I
1) Fit Ordinary Least Squares Regression and Obtain OLS Residuals :
β OLS  X' X  X' Y e OLS  Y  X β OLS
^
^
1
2) Estimate the autocovari ances to lag q and the following qxq matrix and qx1 vector :
^
^
 ^

^ 

(
0
)

(
1
)


(
q

1
)
 ^

 ^ (1) 


e
e
^
^

t
t

h
^
^
^

 (0)   (q  2) γ   (2) 
 (h)  t  h 1
h  0,1,..., q Γ q    (1)
q



n









^
^
^ (q  1) ^ (q  2) 



 ( 0) 

 (q )
3) Estimate the Coefficien ts of the lagged errors - qx1 vector and V  t    2 :
n
^
^ 1 ^
^ 2
^
^ ^
ρ  Γ q γ q    ( 0)  ρ ' γ q
^ 1
^ 1
^
^
4) Obtain the Cholesky Decomposit ion of Γ q : Γ q  P q ' P q
^ 1/2
^
5) Obtain the transform ation matrix : T  V
^ 2 ^
T11   P q
^
 q
0

T2   

0

 0

^
q

(q  q)
^
1
q  (n  q) 
T12  0
(assuming n  2(q  1)) :
T1  T11 T12 
0 
0
0
0
0
 1 1 
0
0
0
0




1
^


 
^
^
0
0
0
0  q

1
0
0
0 
^
0
q
 1
0
1
^

0
0


0

1
q  n 
(n  q)  n 
^
T 
T   1  ( n  n)
T2 
Estimated GLS (General q) - II
1
^ ^
 ^ ^ 
6) Obtain the estimated generalize d least squares estimate : β EGLS   X' T' T X  X' T' T Y


7) Obtain the Estimated error vari ance from the transform ed model :
^
^
^

 ^ ^

 Y  X β EGLS ' T' T Y  X β EGLS 
^ 2



e  
n  p ' q
^
 ^  ^ ^ 
8) Obtain the estimated variance - covariance matrix of β EGLS : V  β EGLS    e  X' T' T X 




^
2
^
1
^
9) Obtain estimated t - statistics for regression coefficien ts : ti 
 EGLS ,i
^

SE   EGLS ,i 


10) Obtain residual MS for estimates of autoregres sive parameters  and the standard errors :
^ ^
^


(
0
)

ρ
'
γ
q
^


^ 
s2  
SE   i   s 2  ii
n  p ' q
 
11) Obtain t - statistics for the  i :
^
^
ti 
i
^ 
SE   i 
 
^ 1
 ii  i diagonal element of Γ q
th
and compare with the t - distributi on with df  n  p 'q
SAS Proc Autoreg Output
The AUTOREG Procedure
Dependent Variable
wine
Ordinary Least Squares Estimates
SSE
158540.525
DFE
MSE
2366
Root MSE
SBC
738.318203
AIC
Regress R-Square
0.9324
Total R-Square
Durbin-Watson
0.1199
Variable
Intercept
adpop
DF
1
1
Estimate
-347.9736
4.3092
Standard
Error
21.9895
0.1417
t Value
-15.82
30.40
67
48.64439
733.84999
0.9324
Approx
Pr > |t|
<.0001
<.0001
Estimates of Autocorrelations
Lag
Covariance
Correlation
0
1
2297.7
2147.9
1.000000
0.934807
-1 9 8 7 6 5 4 3 2 1 0 1 2 3 4 5 6 7 8 9 1
|
|
|********************|
|******************* |
SAS Proc Autoreg Output
Preliminary MSE
289.8
Estimates of Autoregressive Parameters
Standard
Lag
Coefficient
Error
t Value
1
-0.934807
0.043717
-21.38
SSE
MSE
SBC
Regress R-Square
Durbin-Watson
Yule-Walker Estimates
18516.1612
DFE
280.54790
Root MSE
596.454422
AIC
0.5702
Total R-Square
1.6728
Variable
DF
Estimate
Standard
Error
Intercept
adpop
1
1
-347.2297
4.2540
74.0420
0.4546
66
16.74956
589.752103
0.9921
t Value
Approx
Pr > |t|
-4.69
9.36
<.0001
<.0001