Heteroscedastitciy and autocorrelation
Generalized least squares (GLS)
Generalized linear regression models
Ragnar Nymoen
Department of Economics, UiO
23 January 2009
ECON 4610: Lecture 3
Heteroscedastitciy and autocorrelation
Generalized least squares (GLS)
Overview
This is topic 3 in the lecture plan
Heteroscedasticity and autocorrelation
Testing for het and auto.
The regression model with het and auto, matrix notation
GLS
Main reference is G Ch 8;. B Ch 3.1-3-4; K: Ch 8
ECON 4610: Lecture 3
Heteroscedastitciy and autocorrelation
Generalized least squares (GLS)
Heteroscedasticity example
Assume the stochastic variables yij and xij , where j refer to
individuals, and i refers to for example socioeconomic class, or
region, or education level, or some other category/group. For yij
and xij we have the classical regression model:
yij
" ij
D
1
C
N.0,
2 x2ij
2
C "ij , i D 1, 2, ...n, j D 1, 2, ..., mi , i D 1, 2, ...n
/ (and independent)
Assume that we can only observe the group means:
yi D
1
C
2 x2i
C "i , i D 1, 2, ...n
where
yi D
mi
mi
mi
1 X
1 X
1 X
yij , xi D
x2ij , "i D
"ij
mi j D1
mi j D1
mi j D1
ECON 4610: Lecture 3
(1)
Heteroscedastitciy and autocorrelation
Generalized least squares (GLS)
In (1) the disturbances are heteroscedastic:
"
#
mi
1 X
1 2 2
mi
Var ["i ] D Var
" ij D
mi j D1
mi
2
2
D
mi
OLS estimators of (1) are unbiased and consistent, but the
estimated coe¢ cient variances no longer correspond to the
true variances. Hence OLS estimators are no longer BLUE.
The practical problem is that OLS based inference (t and F
tests) is no longer exact (because of the erroneous variance
estimation).
The solution is to re-install the classical assumptions by
switching from (1) to
yi D
1 x1i
C
2 x2i
C " i , i D 1, 2, ...n
where
yi D
p
mi yi ,x1i D
p
p
p
mi , x2i D mi xi , "i D mi xi
ECON 4610: Lecture 3
(2)
Heteroscedastitciy and autocorrelation
Generalized least squares (GLS)
OLS estimators of (2) are BLUE since
Var [" i ] D mi Var ["i ] D
2
and all other classical assumptions also hold.
In the example, the problem would not exist if we have
observations at the individual level— because homoscedasticity
at the individual level was assumed in (1).
This is not general–In fact heteroscedasticity is a much
focused problem when modelling cross-section and panel data,
because of the heterogeneity existing among individuals.
Care must be taken to avoid heteroscedasticity also in models
of time series— but for simplicity we follow Greene and say
that heterscedasticity is mainly a feature of cross-section and
panel models, while autocorrelation is a feature of time series
models.
ECON 4610: Lecture 3
Heteroscedastitciy and autocorrelation
Generalized least squares (GLS)
Autocorrelation
Consider
yt D
1
C
2 x2t
C " t , t D 1, 2, ..., T
(3)
subject to residual autocorrelation:
Cov ."t , "t j / 6D 0 for j > 0.
As long at x2t is exogenous in the econometric sense (cf slide set 1,
Greene Ch 2.3) the OLS estimators remain unbiased and
consistent, so the main problem is how to obtain correct inference,
since estimated coe¢ cient variances are over/underestimated.
ECON 4610: Lecture 3
Heteroscedastitciy and autocorrelation
Generalized least squares (GLS)
A speci…c form at autocorrelation that gives much insight is 1.
order autoregressive autocorrelation:
"t D "t
1
C
1<
t,
< 1, t D 1, 2, ..., T
where t have all the classical residual properties, in particular let
Var [ t ] D 2 (it is a constant, not dependent on t).
Repeated substitution gives
"t D
1
X
j
t j,
The variance of "t is
"
#
1
X
j
2
Var
.1 C . /2 C .
t j D
j D0
1<
<1
(4)
j D0
2 2
/ C.
2
3 2
/ C .../ D
1
2
2
(5)
ECON 4610: Lecture 3
.
Heteroscedastitciy and autocorrelation
Generalized least squares (GLS)
Autocovariance. First between " t and "t
Cov ." t , "t
1/
D E [" t " t
(4) shows that
t
1]
D E [. " t
1
1:
C
is uncorrelated with "t
Cov ." t , "t
1/
D Var [" t
t /" t 1 ]
1,
DE
"2t
1
C
t "t 1
so
D Var ["t ].
1]
because of (5)
Cov ."t , "t
2/
D E [. " t
D
2
1
C
t /" t 2 ]
2
"t
2
Var ["t ], and so on:
Cov ."t , "t j / D
ACFj D
DE .
j
Cov ."t , "t j /
D
Var [" t ]
Var [" t ]
j
,1<
ECON 4610: Lecture 3
<1
C
t
C
t /" t 2
Heteroscedastitciy and autocorrelation
Generalized least squares (GLS)
Matrix notation for heteroscedasticity and autocorrelation
Using the same notation as before:
y D X C"
we can write the model that we now consider as
E ["] D 0
Var ["] D
2
•D6
where the conditioning on X is implicit, and • is n n (or T
and positive de…nite (all leading determinants are positive).
ECON 4610: Lecture 3
T/
Heteroscedastitciy and autocorrelation
Generalized least squares (GLS)
In the two examples: Heteroscedasticity
0
1/m1
0
...
0
B
0
1/m2 ...
0
Var ["] D 2 • D 2 B
@
0
1/mn
and, autocorrelation
Var ["] D
2
•D
2
0
B
B
@
1
T 1
...
1 ...
T 1
T 2
1
1
C
C,
A
ECON 4610: Lecture 3
2
1
C
C
A
2
D
1
2
.
Heteroscedastitciy and autocorrelation
Generalized least squares (GLS)
Testing for absence of het and auto
Sometimes het and auto are implied by the economics theory.
But often we start out from the belief that the theory i.e., X
explain all systematic variation in y.
Good practice to test this assumption, and regard the absence
of het and auto as a “null hypothesis” to be tested.
Modern econometric programs makes this easy.
The rationale for testing is that the OLS residuals, given a
reasonable sample size, are informative measurements of the
disturbances (see B p 94)
ECON 4610: Lecture 3
Heteroscedastitciy and autocorrelation
Generalized least squares (GLS)
Some residual tests (you should know)
Autocorrelation
Plots of the estimated autocorrelation function (ACFj /. Gives
valuable informal “test”
Durbin-Watson test statistics. Formal test of 1. order
autoregressive autocorrelation.
LM-tests that are valid also with lagged endogenous variable.
Regress et on J lags, and the set of regressors, and test
signi…cance of the J lags.
Heteroscedasticity
White’s test. Regress ei2 on the set of regressors and their
squares and cross-products (if feasible).
Has power against the alternative that Var ["i ] depends on xi2
and xi xj .
but has power also against omitted variables and wrong
functional form— so is a general test of misspeci…cation
ECON 4610: Lecture 3
Heteroscedastitciy and autocorrelation
Generalized least squares (GLS)
In fact all the tests have power against other departures from
the classical assumptions.
So a signi…cant Durbin-Watson does not give reason to
conclude that “the model is true, we only need to adjust for
autocorrelation”.
The test does not indicate a correct model, they are
noncontructive as Greene says.
In practice one must interpret the evidence within a wider
reference framework, and often re-specify the systematic part
of the model (X ).
Still— here we only consider the other response: That of
estimating under het and auto.
ECON 4610: Lecture 3
Heteroscedastitciy and autocorrelation
Generalized least squares (GLS)
In the case with known •. De…ne P (n
n/ so that
1
P•P0 D I () •
D P0 P
(6)
(Greene p 154, B 79-79)).
Pre-multiply the equation with P to obtain the model in
transformed variables:
Py D PX C P"
(7)
The transformation of the variables re-instals homoscedasticity:
Var [P"] D E [.P"/.P"/0 ]
D E [.P""0 P0 ] D
2
P•P0 D
2
I.
OLS on (7) gives the GLS estimator:
bGLS D [.PX/0 .PX/]
1
.PX/0 .Py/
(8)
Clearly, this GLS estimator, inherits all the properties that OLS has
in the classical regression model.
ECON 4610: Lecture 3
Heteroscedastitciy and autocorrelation
Generalized least squares (GLS)
Feasible GLS
Use (6) to express (8) as
bGLS D .X0 • 1 X/
1
X0 • 1 y
(9)
which reminds us that bGLS depends on exact numerical knowledge
about •.
The feasible GLS estimator is bFGLS de…ned by replacing • in
(9) by a consistent estimator.
Asymptotically bFGLS has lower variance than b, but not
necessarily in small samples (hence, if in doubt about
seriousness of heteroscedasticity, then may report OLS as
well).
ECON 4610: Lecture 3
Heteroscedastitciy and autocorrelation
Generalized least squares (GLS)
How feasible is feasible GLS?
Feasible GLS hinges on knowing the form of het/auto, because
then it is usually possible to …nd a consistent estimator of •.
Sometimes this is practical
Example: With 1. order autocorrelation detected, can use OLS
residuals to estimate both 2 and consistently.
In other applications this may be di¢ cult, the exact form of
heteroscedasticity for example.
An important alternative is to correct the estimated variance
of the OLS estimator.
Greene, sec 8.4.4: White’s heteroscedastic consistent estimator
(of Var [b]).
Based on the model of het that was used as the alternative
hypothesis for White’s test.
ECON 4610: Lecture 3