Chapter 5: The Generalized Linear Regression Model
and Heteroscedasticity
Advanced Econometrics - HEC Lausanne
Christophe Hurlin
University of Orléans
December 15, 2013
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 15, 2013
1 / 153
Section 1
Introduction
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 15, 2013
2 / 153
1. Introduction
The outline of this chapter is the following:
Section 2. The generalized linear regression model
Section 3. Ine¢ ciency of the Ordinary Least Squares
Section 4. Generalized Least Squares (GLS)
Section 5. Heteroscedasticity
Section 6. Testing for heteroscedasticity
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 15, 2013
3 / 153
1. Introduction
References
Amemiya T. (1985), Advanced Econometrics. Harvard University Press.
Greene W. (2007), Econometric Analysis, sixth edition, Pearson - Prentice
Hil (recommended)
Pelgrin, F. (2010), Lecture notes Advanced Econometrics, HEC Lausanne (a
special thank)
Ruud P., (2000) An introduction to Classical Econometric Theory, Oxford
University Press.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 15, 2013
4 / 153
1. Introduction
Notations: In this chapter, I will (try to...) follow some conventions of
notation.
fY ( y )
probability density or mass function
FY ( y )
cumulative distribution function
Pr ()
probability
y
vector
Y
matrix
Be careful: in this chapter, I don’t distinguish between a random vector
(matrix) and a vector (matrix) of deterministic elements (except in section
2). For more appropriate notations, see:
Abadir and Magnus (2002), Notation in econometrics: a proposal for a
standard, Econometrics Journal.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 15, 2013
5 / 153
Section 2
The generalized linear regression model
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 15, 2013
6 / 153
2. The generalized linear regression model
Objectives
The objective of this section are the following:
1
De…ne the generalized linear regression model
2
De…ne the concept of heteroscedasticity
3
De…ne the concept of autocorrelation (or correlation) of disturbances
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 15, 2013
7 / 153
2. The generalized linear regression model
Consider the (population) multiple linear regression model:
y = Xβ + ε
where (cf. chapter 3):
y is a N
1 vector of observations yi for i = 1, .., N
X is a N K matrix of K explicative variables xik for k = 1, ., K and
i = 1, .., N
ε is a N
1 vector of error terms εi .
β = ( β1 ..βK )> is a K
Christophe Hurlin (University of Orléans)
1 vector of parameters
Advanced Econometrics - HEC Lausanne
December 15, 2013
8 / 153
2. The generalized linear regression model
In chapter 3 (linear regression model), we assume spherical disturbances
(assumption A4):
V ( ε j X ) = σ 2 IN
In this chapter, we will relax the assumption that the errors are
independent and/or identically distributed and we will study:
1
Heteroscedasticity
2
Autocorrelation or correlation.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 15, 2013
9 / 153
2. The generalized linear regression model
De…nition (Generalized linear regression model)
The generalized linear regression model is de…ned as to be:
y = Xβ + ε
where X is a matrix of …xed or random regressors, β 2 RK , and the error
term ε satis…es:
E ( εj X) = 0N 1
V ( ε j X) = Σ = σ 2 Ω
where Ω and Σ are symmetric positive de…nite matrices.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 15, 2013
10 / 153
2. The generalized linear regression model
Reminder
V ( εj X) = E εε> X
| {z }
|
{z
}
N N
0
N N
V ε21 X
Cov ( ε1 ε2 j X)
B E ( ε2 ε1 j X)
V ε22 X
=B
@
..
..
Cov ( εN ε1 j X)
..
Christophe Hurlin (University of Orléans)
1
.. Cov ( ε1 εN j X)
.. Cov ( ε2 εN j X) C
C
A
..
..
2
..
V εN X
Advanced Econometrics - HEC Lausanne
December 15, 2013
11 / 153
2. The generalized linear regression model
Remark
In the generalized linear regression model, we have
V ( ε j X) = Σ = σ 2 Ω
with
0
σ21 σ12
B σ21 σ2
2
Σ=B
@ ..
..
σN 1 ..
and ω ij = σij /σ2 .
Christophe Hurlin (University of Orléans)
1
0
.. σ1N
ω 11 ω 12
C
B
.. σ2N C
ω 21 ω 22
= σ2 B
@ ..
.. .. A
..
2
.. σN
ωN 1
..
Advanced Econometrics - HEC Lausanne
1
.. ω 1N
.. ω 2N C
C
..
.. A
.. ω NN
December 15, 2013
12 / 153
2. The generalized linear regression model
De…nition (Heteroscedasticity)
Disturbances are heteroscedastic when they have di¤erent (conditional)
variances:
V ( εi j X) 6= V ( εj j X) for i 6= j
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 15, 2013
13 / 153
2. The generalized linear regression model
Remarks
1
Heteroscedasticity often arises in volatile high-frequency time-series
data such as daily observations in …nancial markets.
2
Heteroscedasticity often arises in cross-section data where the scale
of the dependent variable and the explanatory power of the model
tend to vary across observations. Microeconomic data such as
expenditure surveys are typical
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 15, 2013
14 / 153
2. The generalized linear regression model
Example (Heteroscedasticity)
If the disturbances are heteroscedastic but they are still assumed
uncorrelated across observations, so Ω and Σ would be:
0 2
1
0
σ1 0 .. 0
ω 1 0 .. 0
B 0 σ2 .. 0 C
B
ω 2 .. 0
2
2B 0
2
C
Σ=B
@ .. .. .. .. A = σ Ω = σ @ ..
.. .. ..
0 .. .. σ2N
0
.. .. ω N
to be
with ω i = σ2i /σ2 for i = 1, .., N.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 15, 2013
1
C
C
A
15 / 153
2. The generalized linear regression model
De…nition (Autocorrelation)
Disturbances are autocorrelated (or correlated) when:
Cov ( εi , εj j X) 6= 0
Christophe Hurlin (University of Orléans)
for i 6= j
Advanced Econometrics - HEC Lausanne
December 15, 2013
16 / 153
2. The generalized linear regression model
Example (Autocorrelation)
For instance, time-series data are usually homoscedastic, but
autocorrelated, so Ω and Σ would be:
0 2
1
0
σ
σ12 .. σ1N
1
ω 12 .. ω 1N
B σ21 σ2 .. σ2N C
B
ω
1 .. ω 2N
C = σ2 Ω = σ2 B 21
Σ=B
@ ..
A
@
.. .. ..
..
.. ..
..
σN 1 .. .. σ2
ωN 1
.. ..
1
1
C
C
A
with ω ij = σij /σ2 for i = 1, .., N denotes the correlation (autocorrelation)
ω ij =
Christophe Hurlin (University of Orléans)
σij
= cor (εi , εj )
σ2
Advanced Econometrics - HEC Lausanne
December 15, 2013
17 / 153
2. The generalized linear regression model
Key Concepts
1
The generalized linear regression model
2
Heteroscedasticity
3
Autocorrelation (or correlation) of disturbances
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 15, 2013
18 / 153
Section 3
Ine¢ ciency of the Ordinary Least Squares
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 15, 2013
19 / 153
3. Ine¢ ciency of the Ordinary Least Squares
Objectives
The objective of this section are the following:
1
Study the properties of the OLS estimator in the generalized linear
regression model
2
Study the …nite sample properties of the OLS
3
Study the asymptotic properties of the OLS
4
Introduce the concept of robust / non-robust inference
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 15, 2013
20 / 153
3. Ine¢ ciency of the Ordinary Least Squares
Introduction
Assume that the data are generated by the generalized linear regression
model:
y = Xβ + ε
E ( εj X) = 0N
1
V ( ε j X) = σ 2 Ω = Σ
b
Now consider the OLS estimator, denoted β
OLS , of the parameters β:
>
b
β
OLS = X X
1
X> y
We will study its …nite sample and asymptotic properties.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 15, 2013
21 / 153
3. Ine¢ ciency of the Ordinary Least Squares
De…nition (Assumption 3: Strict exogeneity of the regressors)
The regressors are exogenous in the sense that:
E ( εj X) = 0N
Christophe Hurlin (University of Orléans)
1
Advanced Econometrics - HEC Lausanne
December 15, 2013
22 / 153
3. Ine¢ ciency of the Ordinary Least Squares
Finite sample properties of the OLS estimator
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 15, 2013
23 / 153
3. Ine¢ ciency of the Ordinary Least Squares
De…nition (Bias)
In the generalized linear regression model, under the assumption A3
(exogeneity), the OLS estimator is unbiased:
b
E β
OLS
= β0
where β0 denotes the true value of the parameters.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 15, 2013
24 / 153
3. Ine¢ ciency of the Ordinary Least Squares
Remark
Heteroscedasticity and/or autocorrelation don’t induce a bias for the
OLS estimator
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 15, 2013
25 / 153
3. Ine¢ ciency of the Ordinary Least Squares
Proof
>
b
β
OLS = X X
1
X> y = β 0 + X> X
1
X> ε
So we have:
>
b
E β
OLS X = β0 + X X
1
X> E ( ε j X )
Under assumption A3 (exogeneity), E ( εj X) = 0. Then, we get:
b
E β
OLS X = β0
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 15, 2013
26 / 153
3. Ine¢ ciency of the Ordinary Least Squares
Proof (cont’d)
b
E β
OLS X = β0
So, we have:
b
E β
OLS
b
= EX E β
OLS X
= EX ( β 0 ) = β 0
where EX denotes the expectation with respect to the distribution of X.
The OLS estimator is unbiased:
b
E β
OLS
Christophe Hurlin (University of Orléans)
= β0
Advanced Econometrics - HEC Lausanne
December 15, 2013
27 / 153
3. Ine¢ ciency of the Ordinary Least Squares
De…nition (Bias)
In the generalized linear regression model, under the assumption A3
(exogeneity), the OLS estimator has a conditional variance covariance
matrix given by
2
>
b
V β
OLS X = σ0 X X
1
X> ΩX X> X
1
and a variance covariance matrix given by:
b
V β
OLS
Christophe Hurlin (University of Orléans)
b
= EX V β
OLS X
Advanced Econometrics - HEC Lausanne
December 15, 2013
28 / 153
3. Ine¢ ciency of the Ordinary Least Squares
Proof
>
b
β
OLS = X X
1
X> y = β 0 + X> X
1
X> ε
So we have:
b
V β
OLS X
= E
=
X> X
= σ20 X> X
Christophe Hurlin (University of Orléans)
1
X> X
1
X> εε> X X> X
1
X> E εε> X X X> X
1
X> ΩX X> X
Advanced Econometrics - HEC Lausanne
X
1
1
December 15, 2013
29 / 153
3. Ine¢ ciency of the Ordinary Least Squares
De…nition (Variance estimator)
An estimator of the variance covariance matrix of the OLS estimator
b
β
OLS is given by
b
b β
V
OLS
b 2 X> X
=σ
1
b
X> ΩX
X> X
1
b is a consistent estimator of Σ = σ 2 Ω. This estimator holds
b2 Ω
where σ
whether X is stochastic or non-stochastic.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 15, 2013
30 / 153
3. Ine¢ ciency of the Ordinary Least Squares
De…nition (Normality assumption)
Under assumptions A3 (exogeneity) and A6 (normality), the OLS
estimator obtained in the generalized linear regression model has an
(exact) normal conditional distribution:
b
β
OLS X
N
Christophe Hurlin (University of Orléans)
β 0 , σ 2 X> X
1
X> ΩX X> X
Advanced Econometrics - HEC Lausanne
1
December 15, 2013
31 / 153
3. Ine¢ ciency of the Ordinary Least Squares
Asymptotic properties of the OLS estimator
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 15, 2013
32 / 153
3. Ine¢ ciency of the Ordinary Least Squares
Assumptions
plim
plim
1 >
X X=Q
N
1 >
X ΩX = Q
N
where:
1
Q is a K
2
Q is a K
K …nite (non null) de…nite positive matrix
K …nite (non null) de…nite positive matrix with
rank (Q) = K
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 15, 2013
33 / 153
3. Ine¢ ciency of the Ordinary Least Squares
De…nition (Consistency of the OLS estimator)
If plim N 1 X> ΩX and plim N 1 X> X are both …nite positive de…nite
b
matrices, then β
OLS is a consistent estimator of β:
p
b
β
OLS ! β0
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 15, 2013
34 / 153
3. Ine¢ ciency of the Ordinary Least Squares
Proof
>
b
β
OLS = β0 + X X
1
X> ε
We know that under assumption A3 (exogeneity):
1 >
X ε = 0K 1
N
1
plim X> X = Q
N
plim
So, we have
b
plim β
OLS = β0
b is consistent.
So, the estimator β
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 15, 2013
35 / 153
3. Ine¢ ciency of the Ordinary Least Squares
De…nition (Asymptotic distribution of the OLS)
If the regressors are su¢ ciently well behaved and the o¤-diagonal terms in
diminish su¢ ciently rapidly, then the least squares estimator is
asymptotically normally distributed with
p
where
b
N β
OLS
Q = plim
Christophe Hurlin (University of Orléans)
d
β0 ! N 0, σ2 Q
1 >
X X
N
Q = plim
Advanced Econometrics - HEC Lausanne
1
Q Q
1
1 >
X ΩX
N
December 15, 2013
36 / 153
3. Ine¢ ciency of the Ordinary Least Squares
Remark
1
Regularity conditions include the exogeneity conditions, but also (i)
the regressors are su¢ ciently well-behaved and (ii) the o¤-diagonal
terms of the variance-covariance matrix diminish su¢ ciently rapidly
(relative to the diagonal elements).
2
For a formal proof in a general case, see Amemiya (1985, p. 187).
Amemiya T. (1985), Advanced Econometrics. Harvard University Press.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 15, 2013
37 / 153
3. Ine¢ ciency of the Ordinary Least Squares
De…nition (Asymptotic variance)
Under suitable regularity conditions, the asymptotic variance covariance
b is given by:
matrix of the OLS estimator β
with
b
Vasy β
OLS
Q = plim
Christophe Hurlin (University of Orléans)
1 >
X X
N
=
σ2
Q
N
1
Q Q
Q = plim
Advanced Econometrics - HEC Lausanne
1
1 >
X ΩX
N
December 15, 2013
38 / 153
3. Ine¢ ciency of the Ordinary Least Squares
Fact (Non-robust inference)
Because the variance of the least squares estimator is not
σ 2 X> X
b 2 X> X
σ
1
1
statistical inference ( non-robust inference) based on
may be misleading. For instance the t-test-statistic:
t βk =
b
β
pk
b mkk
σ
where mkk is kth diagonal element of X> X do not have a Student
distribution.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 15, 2013
39 / 153
3. Ine¢ ciency of the Ordinary Least Squares
Robust / Non-robust inference
As a consequence, the familiar inference procedures based on the F
and t distributions will no longer be appropriate.
b
The question is to know how to estimate V β
OLS in the context
of the linear generalized regression model in order to make robust
inference.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 15, 2013
40 / 153
3. Ine¢ ciency of the Ordinary Least Squares
De…nition (Estimator of the asymptotic variance covariance matrix)
If Σ = σ2 Ω were known, the consistent estimator of the (asymptotic)
b
variance covariance of β
OLS would be:
b
b asy β
V
OLS
=
Christophe Hurlin (University of Orléans)
σ2
N
1 >
X X
N
1
1 >
X ΩX
N
Advanced Econometrics - HEC Lausanne
1 >
X X
N
1
December 15, 2013
41 / 153
3. Ine¢ ciency of the Ordinary Least Squares
Proof
By de…nition:
1 >
X X
N
1
Q = plim X> ΩX
N
Q = plim
So,
b
b asy β
plim V
OLS
= plim
=
σ2
N
σ2
Q
N
1
1 >
X X
N
Q Q
1
1 >
X ΩX
N
1 >
X X
N
1
1
Or equivalently
b
b asy β
V
OLS
Christophe Hurlin (University of Orléans)
p
b
! Vasy β
OLS
Advanced Econometrics - HEC Lausanne
December 15, 2013
42 / 153
3. Ine¢ ciency of the Ordinary Least Squares
Reminder
X> X =
N
∑ xi xi>
i =1
X> ΩX =
N
N
∑ ∑ ωij xi xi>
i =1 j =1
X> ΣX =
N
∑
N
∑ σij xi xi> = σ2
i =1 j =1
Christophe Hurlin (University of Orléans)
N
N
∑ ∑ ωij xi xi>
i =1 j =1
Advanced Econometrics - HEC Lausanne
December 15, 2013
43 / 153
3. Ine¢ ciency of the Ordinary Least Squares
Remark
The estimator
b
b asy β
V
OLS
σ2
=
N
1
1 >
X X
N
1 >
X ΩX
N
1
1 >
X X
N
can also be written as
b
b asy β
V
OLS
σ2
=
N
1
N
Christophe Hurlin (University of Orléans)
N
∑
i =1
xi xi>
!
1
1
N
N
N
∑∑
i =1 j =1
Advanced Econometrics - HEC Lausanne
ω ij xi xi>
!
1
N
N
∑
i =1
December 15, 2013
xi xi>
!
1
44 / 153
3. Ine¢ ciency of the Ordinary Least Squares
Remark
b
b asy β
In the next section, we will give a feasible estimator V
OLS
speci…c case of an heteroscedastic model.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
in the
December 15, 2013
45 / 153
3. Ine¢ ciency of the Ordinary Least Squares
Summary
In the GLR model, under some regularity conditions:
1
The OLS estimator is unbiased
2
The OLS estimator is (weakly) consistent
3
The OLS estimator is asymptotically normally distributed
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 15, 2013
46 / 153
3. Ine¢ ciency of the Ordinary Least Squares
Summary
But...
1
The inference based on the estimator σ2 X> X
2
The OLS is ine¢ cient.
b
V β
OLS
Christophe Hurlin (University of Orléans)
1
is misleading.
I N 1 ( β0 ) is a positive de…nite matrix
Advanced Econometrics - HEC Lausanne
December 15, 2013
47 / 153
3. Ine¢ ciency of the Ordinary Least Squares
Key Concepts
1
OLS estimator in the generalized regression model
2
Finite sample properties
3
Asymptotic variance covariance matrix of the OLS estimator
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 15, 2013
48 / 153
Section 4
Generalized Least Squares (GLS)
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 15, 2013
49 / 153
4. Generalized Least Squares (GLS)
Objectives
The objective of this section are the following:
1
De…ne the Generalized Least Squares (GLS)
2
De…ne the Feasible Generalized Least Squares (FGLS)
3
Study the statistical properties of the GLS and FGLS estimators
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 15, 2013
50 / 153
4. Generalized Least Squares (GLS)
Consider the generalized linear regression model with
V ( ε j X) = Σ = σ 2 Ω
We will distinguish two cases:
Case 1: the variance covariance matrix Σ is known (unrealistic case)
Case 2: the variance covariance matrix Σ is unknown
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 15, 2013
51 / 153
4. Generalized Least Squares (GLS)
Case 1:
Σ
is known
The Generalized Least Squares (GLS) estimator
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 15, 2013
52 / 153
4. Generalized Least Squares (GLS)
De…nition (Factorisation)
Since Ω is a positive de…nite matrix, it can factored as follows:
Ω = CΛC>
where the columns of C are the characteristics vectors of Ω, the
characteristic roots of Ω are arrayed in the diagonal matrix Λ, and
C> C = CC> = IN
where I denotes the identity matrix.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 15, 2013
53 / 153
4. Generalized Least Squares (GLS)
De…nition
We de…ne the matrix P such that
P> = CΛ
1/2
so that
Ω
Christophe Hurlin (University of Orléans)
1
= P> P
Advanced Econometrics - HEC Lausanne
December 15, 2013
54 / 153
4. Generalized Least Squares (GLS)
Proof
P> = CΛ
Since Λ is diagonal, Λ
1/2
Λ
P> P = CΛ
1/2
1/2
1/2
=Λ
1
Λ
C> = CΛ
1/2
, and we have:
1
C>
Consider the quantity P> PΩ:
P> PΩ = CΛ
1
C> CΛC>
= CΛ 1 ΛC>
= CC>
= IN
Since C satis…es CC> = IN . Then, P> P = Ω
Christophe Hurlin (University of Orléans)
1
Advanced Econometrics - HEC Lausanne
December 15, 2013
55 / 153
4. Generalized Least Squares (GLS)
GLS estimator
Premultiply the generalized linear regression model by P to obtain
Py = PXβ + Pε
or equivalently
y = X β+ε
The conditional variance of ε is
V ( ε j X) = E ε ε
>
X
= PE εε> X P>
= σ2 PΩP>
= σ2 Λ 1/2 C> CΛC> CΛ
= σ 2 IN
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
1/2
December 15, 2013
56 / 153
4. Generalized Least Squares (GLS)
GLS estimator (cont’d)
y = X β+ε
V ( ε j X ) = σ 2 IN
The classical regression model applies to this transformed model.
If Ω is assumed to be known, y = Py and X = PX are observed data.
So, we can apply the ordinary least squares to this transformed model:
b= X
β
Christophe Hurlin (University of Orléans)
>
X
1
X
>
y
Advanced Econometrics - HEC Lausanne
December 15, 2013
57 / 153
4. Generalized Least Squares (GLS)
GLS estimator (cont’d)
b =
β
X
>
1
X
=
X> P> PX
=
X> Ω
1
X
X
1
1
>
y
X> P> Py
X> Ω
1
y
This estimator is the generalized least squares (GLS) estimator of β.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 15, 2013
58 / 153
4. Generalized Least Squares (GLS)
De…nition (GLS estimator)
The Generalized Least Squares (GLS) estimator of β is de…ned as to be:
>
b
β
GLS = X Ω
Christophe Hurlin (University of Orléans)
1
X
1
X> Ω
Advanced Econometrics - HEC Lausanne
1
y
December 15, 2013
59 / 153
4. Generalized Least Squares (GLS)
De…nition (Bias)
b
Under the exogeneity assumption (A3), the estimator β
GLS is unbiased:
b
E β
GLS
= β0
where β0 denotes the true value of the parameters.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 15, 2013
60 / 153
4. Generalized Least Squares (GLS)
Proof
We have:
>
b
β
GLS = X Ω
1
1
X
X> Ω
1
y = β 0 + X> Ω
So,
>
b
E β
GLS X = β0 + X Ω
1
X
1
X> Ω
1
X
1
1
X> Ω
1
ε
E ( ε j X)
Under the exogeneity assumption A3, E ( εj X) = 0, so we have
and
b
E β
GLS
b
E β
GLS X = β0
b
= EX E β
GLS X
Christophe Hurlin (University of Orléans)
= EX ( β 0 ) = β 0
Advanced Econometrics - HEC Lausanne
December 15, 2013
61 / 153
4. Generalized Least Squares (GLS)
De…nition (Variance covariance matrix)
b
The conditional variance covariance matrix of the estimator β
GLS is
de…ned as to be:
2
b
V β
X> Ω
GLS X = σ
1
X
1
X
1
The variance covariance matrix is given by
b
V β
GLS
Christophe Hurlin (University of Orléans)
= σ2 EX
X> Ω
Advanced Econometrics - HEC Lausanne
1
December 15, 2013
62 / 153
4. Generalized Least Squares (GLS)
Proof
b
Consider the de…nition of β
GLS in the transformed model:
b
β
GLS = β0 + X
b
V β
GLS X = X
Since E ε ε
>
>
X
1
>
X
1
X
>
E ε ε
X
>
>
X X
ε
X
>
X
1
X = σ2 IN , we have
b
V β
GLS X
= σ2 X
>
X
= σ2 X
>
X
1
Christophe Hurlin (University of Orléans)
>
X
X
>
X
1
1
= σ2 X> P> PX
= σ 2 X> Ω
X
1
X
1
1
Advanced Econometrics - HEC Lausanne
December 15, 2013
63 / 153
4. Generalized Least Squares (GLS)
De…nition (Consistency)
b
Under the exogeneity assumption A3, the GLS estimator β
GLS is (weakly)
consistent:
p
b
β
GLS ! β0
as soon as
1 >
X X =Q
N
where Q is a …nite positive de…nite matrix.
plim
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 15, 2013
64 / 153
4. Generalized Least Squares (GLS)
Proof
>
b
β
GLS = β0 + X Ω
1
X
1
X> Ω
1
ε
Under the assumption A3 (exogeneity):
plim
1 >
X Ω
N
plim
1 >
X Ω
N
1
ε = 0K
1
1
X=Q
So, we have
b
plim β
GLS = β0
b
The estimator β
GLS is weakly consistent.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 15, 2013
65 / 153
4. Generalized Least Squares (GLS)
De…nition (Asymptotic distribution)
b
Under some regularity conditions, the GLS estimator β
GLS is
asymptotically normally distributed:
p
where
b
N β
GLS
Q = plim
Christophe Hurlin (University of Orléans)
1
X
N
d
β0 ! N 0, σ2 Q
>
X = plim
1 >
X Ω
N
Advanced Econometrics - HEC Lausanne
1
1
X
December 15, 2013
66 / 153
4. Generalized Least Squares (GLS)
De…nition (Asymptotic variance covariance matrix)
b
The asymptotic variance covariance matrix of the estimator β
GLS is:
b
Vasy β
GLS
=
σ2
Q
N
1
If Σ = σ2 Ω is known, a consistent estimator is given by:
b
b asy β
V
GLS
=
σ2
X> Ω
N
1
X
1
This estimator holds whether X is stochastic or non-stochastic.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 15, 2013
67 / 153
4. Generalized Least Squares (GLS)
Theorem (BLUE estimator)
b
The GLS estimator β
GLS is the minimum variance linear unbiased
estimator ( BLUE estimator) in the semi-parametric generalized linear
regression model. In particular, the matrix de…ned by:
b
Vasy β
OLS
is a positive semi de…nite matrix.
Christophe Hurlin (University of Orléans)
b
Vasy β
GLS
Advanced Econometrics - HEC Lausanne
December 15, 2013
68 / 153
4. Generalized Least Squares (GLS)
Theorem (E¢ ciency)
Under suitable regularity conditions, in a parametric generalized linear
b
regression model, the GLS estimator β
GLS is e¢ cient
b
V β
GLS
= I N 1 ( β0 )
where I N 1 ( β0 ) denotes the FDCR or Cramer-Rao bound.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 15, 2013
69 / 153
4. Generalized Least Squares (GLS)
Remark
In a Gaussian generalized linear regression model (under assumption A6),
the likelihood of the sample is given by:
LN (θ; y j x ) =
2πσ2
exp
N /2
jΩj
1
(y
2σ2
N /2
Xβ)> Ω
1
(y
Xβ)
The log-likelihood is de…ned as to be:
`N (θ; y j x ) =
Christophe Hurlin (University of Orléans)
N
N
ln 2πσ2
log (jΩj)
2
2
1
(y Xβ)> Ω 1 (y Xβ)
2σ2
Advanced Econometrics - HEC Lausanne
December 15, 2013
70 / 153
4. Generalized Least Squares (GLS)
Remark
For testing hypotheses, we can apply the full set of results in Chapter 4 to
the transformed model. For instance, for testing the p linear constraints
H0 : Rβ = q, the appropriate test-statistic is:
F=
1
b
Rβ
GLS
p
q
Christophe Hurlin (University of Orléans)
>
σ 2 R X> Ω
1
X
1
1
R>
Advanced Econometrics - HEC Lausanne
b
Rβ
GLS
q
December 15, 2013
71 / 153
4. Generalized Least Squares (GLS)
Fact
To summarize, all the results for the classical model, including the usual
inference procedures, apply to the transformed model.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 15, 2013
72 / 153
4. Generalized Least Squares (GLS)
Case 2:
Σ
is unknown
The Feasible Generalized Least Squares (FGLS) estimator
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 15, 2013
73 / 153
4. Generalized Least Squares (GLS)
Introduction
1
2
If Σ contains unknown parameters that must be estimated, then
generalized least squares is not feasible.
With an unrestricted matrix Σ = σ2 Ω, there are N (N + 1) /2
additional parameters (since Σ is symmetric) to estimate
3
This number is far too many to estimate with N observations.
4
Obviously, some structure must be imposed on the model if we are
to proceed.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 15, 2013
74 / 153
4. Generalized Least Squares (GLS)
De…nition (Structure of variance covariance matrix)
We assume that the conditional variance covariance matrix of the
disturbances can be expressed as a function of a small set of parameters α:
V ( ε j X) = σ 2 Ω ( α )
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 15, 2013
75 / 153
4. Generalized Least Squares (GLS)
Example (Time series)
For instance, a commonly used formula in time-series
0
1
ρ
ρ2
ρ3 ..
B ρ
1
ρ
ρ2 ..
B
B ρ2
ρ
1
ρ ..
Ω (ρ) = B
2
B ρ3
ρ
ρ
1 ..
B
@ ..
..
..
.. ..
ρN 1 ρN 2 ρN 3 .. ..
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
settings is
1
ρN 1
ρN 2 C
C
ρN 3 C
C
.. C
C
.. A
1
December 15, 2013
76 / 153
4. Generalized Least Squares (GLS)
Example (Heteroscedascticity)
If we consider a heteroscedastic model, where the variance of εi depends
on a variable zi , with
V ( εi j X) = σ2 ziθ
we have
0
B
B
Ω (θ ) = B
B
@
Christophe Hurlin (University of Orléans)
z1θ 0 0
0 z2θ 0
0 0 z3θ
.. .. ..
0 0 0
.. 0
.. 0
.. 0
.. ..
.. zNθ
Advanced Econometrics - HEC Lausanne
1
C
C
C
C
A
December 15, 2013
77 / 153
4. Generalized Least Squares (GLS)
De…nition (Feasible Generalized Least Squares (FGLS))
Consider a consistent estimator b
α of α, then the Feasible Least Generalized
Squares (FGLS) estimator of β is de…ned as to be:
>b
b
β
FGLS = X Ω
1
X
1
b
X> Ω
1
y
b = Ω (b
where Ω
α) is a consistent estimator of Ω (α) .
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 15, 2013
78 / 153
4. Generalized Least Squares (GLS)
Remark
If
plim
plim
1 >b
X Ω
N
1
1 >b
X Ω
N
X
1
y
1 >
X Ω
N
1
X
=0
1 >
X Ω
N
1
y
=0
Then the GLS and FGLS estimators are asymptotically equivalent
b
β
FGLS
Christophe Hurlin (University of Orléans)
p
b
β
GLS ! 0K
1
Advanced Econometrics - HEC Lausanne
December 15, 2013
79 / 153
4. Generalized Least Squares (GLS)
Theorem (E¢ ciency)
An asymptotically e¢ cient FGLS estimator does not require that we have
an e¢ cient estimator of α; only a consistent one is required to achieve full
e¢ ciency for the FGLS estimator.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 15, 2013
80 / 153
4. Generalized Least Squares (GLS)
Remark
If the estimator b
α is consistent
p
b
α!α
then the FGLS estimator has the same asymptotic properties (consistency,
e¢ ciency, asymptotic distribution etc.) than the GLS estimator.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 15, 2013
81 / 153
4. Generalized Least Squares (GLS)
Key Concepts
1
Factorisation of the variance covariance matrix
2
Generalized Least Squares (GLS) estimator
3
Feasible Generalized Least Squares (FGLS) estimator
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 15, 2013
82 / 153
Section 5
Heteroscedasticity
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 15, 2013
83 / 153
5. Heteroscedasticity
Objectives
The objective of this section are the following:
1
To determine the properties of the OLS in presence of
heteroscedasticity
2
To estimate the asymptotic variance covariance matrix of the OLS
estimator in presence of heteroscedasticity
3
To introduce the concept of robust inference (to heteroscedasticity)
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 15, 2013
84 / 153
5. Heteroscedasticity
Introduction
In the rest of this chapter, we will focus on the case of heteroscedastic
disturbances.
V ( εi j X) = σ2i for i = 1, .., N
Heteroscedasticity arises in numerous applications, in both cross-section
and time-series data.
For example, even after accounting for …rm sizes, we expect to observe
greater variation in the pro…ts of large …rms than in those of small ones.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 15, 2013
85 / 153
5. Heteroscedasticity
Assumption: We assume that the disturbances are pairwise
uncorrelated and heteroscedastic:
V ( ε j X) = Σ = σ 2 Ω
with
0
σ21 0
B 0 σ2
2
Σ=B
@ .. ..
0 ..
1
0
.. 0
ω1 0
B
.. 0 C
C = σ2 Ω = σ2 B 0 ω 2
@ ..
.. .. A
..
2
.. σN
0
..
with ω i = σ2i /σ2 for i = 1, .., N.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
1
.. 0
.. 0 C
C
.. .. A
.. ω N
December 15, 2013
86 / 153
5. Heteroscedasticity
De…nition (Scaling)
The fact to scale the variances as
σ2i = σ2 ω i for i = 1, .., N
allows us to use a normalisation on Ω
trace (Ω) =
N
∑ ωi = N
i =1
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 15, 2013
87 / 153
5. Heteroscedasticity
Introduction (cont’d)
We will consider three cases:
Case 1: the heteroscedasticity form (structure) is unknown: OLS
estimator and robust inference
Case 2: the variance covariance matrix Σ is known: GLS or Weighted
Least Square (WLS)
Case 3: the variance covariance matrix Σ is unknown but its form
(structure) is known: two-steps or iterated FGLS estimator
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 15, 2013
88 / 153
5. Heteroscedasticity
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 15, 2013
89 / 153
5. Heteroscedasticity
Case 1: Heteroscedasticity of unknown form
OLS and robust inference
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 15, 2013
90 / 153
5. Heteroscedasticity
Assumption: We assume that the variances σ2i are unknown for i = 1, ..N
and no particular form (structure) is imposed on Ω (or Σ).
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 15, 2013
91 / 153
5. Heteroscedasticity
Introduction
1
The GLS cannot be implemented since Σ is unknown.
2
The FGLS estimator requires to estimate (in a …rst step) N
parameters σ21 , .., σ2N . With N observations, the FGLS is not feasible.
3
The only solution to estimate β consists in using the OLS.
4
Under suitable regularity conditions, the OLS estimator is unbiased,
consistent, asymptotically normally distributed but... ine¢ cient.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 15, 2013
92 / 153
5. Heteroscedasticity
Introduction (cont’d)
Consider the OLS estimator:
>
b
β
OLS = X X
We know that
asy
b
β
OLS
with
N
b
Vasy β
OLS
Q = plim
Christophe Hurlin (University of Orléans)
1 >
X X
N
β0 ,
σ2
Q
N
=
σ2
Q
N
1
X> y
1
Q Q
1
Q Q
Q = plim
Advanced Econometrics - HEC Lausanne
1
1
1 >
X ΩX
N
December 15, 2013
93 / 153
5. Heteroscedasticity
Problem (Robust inference with OLS)
The conventionally estimated covariance matrix for the least squares
estimator σ2 X> X
1
1
is inappropriate; the appropriate matrix is
1
1
σ 2 X> X
X> ΩX
X> X
. It is unlikely that these two would
coincide, so the usual estimators of the standard errors are likely to be
erroneous. The inference (test-statistics) based σ2 X> X
misleading.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
1
is
December 15, 2013
94 / 153
5. Heteroscedasticity
Question
b
How to estimate Vasy β
OLS
and to make robust inference?
b
Vasy β
OLS
Q = plim
Christophe Hurlin (University of Orléans)
1 >
X X
N
=
σ2
Q
N
1
Q Q
Q = plim
Advanced Econometrics - HEC Lausanne
1
1 >
X ΩX
N
December 15, 2013
95 / 153
5. Heteroscedasticity
We seek an estimator for
Q = plim
1 >
1
X ΩX = plim
N
N
N
∑ ωi xi xi> = EX
ω i xi xi>
i =1
or equivalently of
Q
1
1
= plim X> ΣX = plim
N
N
N
∑ σ2i xi xi> = EX
σi xi xi>
i =1
with
Q
Christophe Hurlin (University of Orléans)
= σ2 Q
Advanced Econometrics - HEC Lausanne
December 15, 2013
96 / 153
5. Heteroscedasticity
Q
1
1
= plim X> ΣX = plim
N
N
N
∑ σ2i xi xi>
i =1
White (1980) shows that under very general condition, the estimator
S0 =
where bεi = yi
1
N
N
∑ bε2i xi xi>
i =1
b
xi> β
OLS , converges to Q
p
S0 ! Q
= σ2 Q
= σ2 Q
White, H. “A Heteroscedasticity-Consistent Covariance Matrix Estimator and
a Direct Test for Heteroscedasticity.” Econometrica, 48, 1980, pp. 817–838.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 15, 2013
97 / 153
5. Heteroscedasticity
We know that:
b
Vasy β
OLS
S0 =
1
1 >
X X
N
So,
1
N
1 >
X X
N
Christophe Hurlin (University of Orléans)
1
N
S0
σ2
Q
N
N
1
Q Q
1
p
∑ bε2i xi xi> ! σ2 Q
i =1
=
1
=
1
N
N
∑
xi xi>
i =1
1 >
X X
N
1
!
1
p
!Q
1
p
b
! Vasy β
OLS
Advanced Econometrics - HEC Lausanne
December 15, 2013
98 / 153
5. Heteroscedasticity
De…nition (White heteroscedasticity consistent estimator)
The White consistent estimator of the asymptotic variance-covariance
b
matrix of the ordinary least squares estimator β
OLS in the generalized
linear regression model is de…ned to be:
b
b asy β
V
OLS
with
= N X> X
b
b asy β
V
OLS
S0 =
Christophe Hurlin (University of Orléans)
1
N
1
S0 X> X
1
p
b
! Vasy β
OLS
N
∑ bε2i xi xi>
i =1
Advanced Econometrics - HEC Lausanne
December 15, 2013
99 / 153
5. Heteroscedasticity
Corollary (White heteroscedasticity consistent estimator)
The White consistent estimator can written as:
! 1
!
N
N
1
1
1
2
>
>
b
b asy β
bεi xi xi
V
xi xi
OLS =
N N i∑
N i∑
=1
=1
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
1
N
N
∑
xi xi>
i =1
December 15, 2013
!
1
100 / 153
5. Heteroscedasticity
Remarks
1
This result is extremely important and useful. It implies that without
actually specifying the type of heteroscedasticity, we can still make
appropriate inferences based on the results of least squares.
2
This implication is especially useful if we are unsure of the precise
nature of the heteroscedasticity (which is probably most of the time).
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 15, 2013
101 / 153
5. Heteroscedasticity
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 15, 2013
102 / 153
5. Heteroscedasticity
Remark
Given the normalisation trace(Ω) = N, we have:
σ2 =
Christophe Hurlin (University of Orléans)
1
N
N
∑ σ2i
i =1
Advanced Econometrics - HEC Lausanne
December 15, 2013
103 / 153
5. Heteroscedasticity
De…nition (SSR)
b2 de…ned by:
The least squares estimator σ
b2 =
σ
b
ε>b
ε
1
=
N K
N K
N
∑ bε2i
i =1
converges to the probability limit of the average variance of the
disturbances
1 N 2
p
b2 ! lim σ2 = lim
σ
∑ σi
N !∞
N !∞ N
i =1
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 15, 2013
104 / 153
5. Heteroscedasticity
Example (White robust estimator. Source: Greene (2012))
Consider the generalized linear regression model:
AVGEXPi = β1 + β2 AGEi + β3 Ownrenti + β4 Incomei + β5 Income2i + εi
where AVGEXP denotes the Avg. monthly credit card expenditure,
Ownrent denotes a binary variable (individual owns (1) or rents (0) home),
Age denotes the age in years, Income denotes the income divided by
10,000. The data are available in …le Chapter5_data.xls. Question:
write a Matlab code to (1) estimate the parameters by OLS, (2) compute
the standard errors and the robust standard errors and (3) compare your
results with Eviews.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 15, 2013
105 / 153
5. Heteroscedasticity
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 15, 2013
106 / 153
5. Heteroscedasticity
2000
OLS residuals
1500
1000
500
0
-500
1
2
3
4
5
6
Income
7
8
9
10
This graph is the sign of heteroscedasticity.. the variance of the residuals
seems to depend on the income.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 15, 2013
107 / 153
5. Heteroscedasticity
The values are the same.. perfect
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 15, 2013
108 / 153
5. Heteroscedasticity
The values are di¤erent... Why?
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 15, 2013
109 / 153
5. Heteroscedasticity
Remark
This di¤erence is due to the fact that Eviews uses a …nite sample
correction for S0 (Davidson and MacKinnon, 1993)
S0 =
N
1
N
K
∑ bε2i xi xi>
i =1
Davidson, R. and J. MacKinnon. Estimation and Inference in Econometrics.
New York: Oxford University Press, 1993.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 15, 2013
110 / 153
5. Heteroscedasticity
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 15, 2013
111 / 153
5. Heteroscedasticity
The values are now identical.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 15, 2013
112 / 153
5. Heteroscedasticity
Case 2: Heteroscedasticity with known
Σ
GLS and Weighted Least Squares
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 15, 2013
113 / 153
5. Heteroscedasticity
Assumption: We assume that the disturbances are heteroscedastic with
V ( ε j X) = Σ = σ 2 Ω
with
0
σ21 0
B 0 σ2
2
Σ=B
@ .. ..
0 ..
1
0
.. 0
ω1 0
B
.. 0 C
C = σ2 Ω = σ2 B 0 ω 2
@ ..
.. .. A
..
2
.. σN
0
..
where the parameters σ2i and ω i are known for i = 1, ..N.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
1
.. 0
.. 0 C
C
.. .. A
.. ω N
December 15, 2013
114 / 153
5. Heteroscedasticity
De…nition (GLS estimator)
In presence of heteroscedasticity, the Generalized Least Squares (GLS)
estimator of β is de…ned as to:
or equivalently by
b
β
GLS =
b
β
GLS =
Christophe Hurlin (University of Orléans)
N
xi xi>
∑
i =1 ω i
N
xi xi>
∑ 2
i =1 σ i
!
!
1
1
N
xi yi
∑ ωi
i =1
N
xi yi
∑ σ2
i
i =1
Advanced Econometrics - HEC Lausanne
!
!
December 15, 2013
115 / 153
5. Heteroscedasticity
Proof
In general, whatever the form of Σ = σ2 Ω, we have:
>
b
β
GLS = X Ω
1
1
X
X> Ω
1
y
Since Ω is diagonal:
X> Ω
X> Ω
1
N
X=
1
xi xi>
∑
i =1 ω i
N
y=
xi yi
i =1 ω i
∑
As a consequence:
b
β
GLS =
Christophe Hurlin (University of Orléans)
N
xi x>
∑ ωii
i =1
!
1
N
xi yi
∑ ωi
i =1
Advanced Econometrics - HEC Lausanne
!
December 15, 2013
116 / 153
5. Heteroscedasticity
Remark
b
β
GLS =
N
xi x>
∑ ωii
i =1
!
1
N
xi yi
∑ ωi
i =1
!
This formula is similar to that obtained for a Weighted Least Squares
(WLS).
! 1
!
N
N
>
b
β
∑ δi xi xi
∑ δi xi yi
WLS =
i =1
Christophe Hurlin (University of Orléans)
i =1
Advanced Econometrics - HEC Lausanne
December 15, 2013
117 / 153
5. Heteroscedasticity
Fact (GLS and WLS)
In presence of heteroscedasticity, the GLS estimator is a particular case of
the Weighted Least Squares (WLS) estimator.
b
β
WLS =
N
∑ δi xi xi>
i =1
!
1
N
∑ δi xi yi
i =1
!
b
b
where δi is an arbitrary weight. For δi = 1/ω i , we have β
WLS = βGLS .
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 15, 2013
118 / 153
5. Heteroscedasticity
Remark
1
The WLS estimator is consistent regardless of the weights used, as
long as the weights are uncorrelated with the disturbances.
2
In general, we consider a weight which is proportional to one
explicative variable (the income in the last example):
σ2i = σ2 xik2 () δi =
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
1
xik2
December 15, 2013
119 / 153
5. Heteroscedasticity
Case 3: Heteroscedasticity for a given structure
FGLS and two-step or iterated estimators
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 15, 2013
120 / 153
5. Heteroscedasticity
Assumption: We assume that the disturbances are heteroscedastic with
V ( ε j X) = Σ ( α ) = σ 2 Ω ( α )
where α denotes a set of parameters.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 15, 2013
121 / 153
5. Heteroscedasticity
Example (Restriction)
We assume that
V ( εi j X) = σ2i (α) = σ2 zi> α
2
where α = (α1 : .. : αH )> is a H 1 vector of parameters and zi is H
of explicative variables (not necessarily the same as in xi ).
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 15, 2013
1
122 / 153
5. Heteroscedasticity
Example (Harvey’s (1976) restriction)
Harvey (1976) considers a restriction of the form:
V ( εi j X) = σ2i (α) = exp xi> α
where α = (α1 : .. : αH )> is a H 1 vector of parameters and zi is H
of explicative variables (not necessarily the same as in xi ).
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 15, 2013
1
123 / 153
5. Heteroscedasticity
We know that the GLS estimator is de…ned by:
N
∑
b
β
GLS =
i =1
xi xi>
σ2i (α)
!
1
N
!
∑
xi yi
σ2i (α)
N
xi yi
σ2i (b
α)
i =1
S, the feasible GLS (FGLS) estimator is:
b
β
FGLS =
Christophe Hurlin (University of Orléans)
N
∑
i =1
xi xi>
σ2i (b
α)
!
1
∑
i =1
Advanced Econometrics - HEC Lausanne
!
December 15, 2013
124 / 153
5. Heteroscedasticity
If we assume for instance that
V ( εi j X) = σ2i (α) = exp zi> α
where zi is a vector of H variables, a way to estimate α consists in
considering the model:
ln bε2i = zi> α + vi
and to estimate α by OLS. The OLS is consistent even it is ine¢ cient (due
to the heteroscedasticity). Given b
α, we have a consistent estimator for σ2i :
p
b2i = exp zi> b
σ
α ! σ2i (α)
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 15, 2013
125 / 153
5. Heteroscedasticity
Problem
In order to estimate β by the GLS, we need b
α, and to estimate α, we need
b
...
the residuals bεi = yi xi> β
GLS
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 15, 2013
126 / 153
5. Heteroscedasticity
Two solutions
1
A two steps FGLS estimator
2
An iterative FGLS estimator
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 15, 2013
127 / 153
5. Heteroscedasticity
De…nition (Two-steps FGLS estimator)
First step: estimate the parameters β by OLS. Compute the residuals
b
bεi = yi xi> β
OLS and estimate the parameters α according to the
α)
appropriate model. Second step: compute the estimated variances σ2i (b
and compute the FGLS estimator:
b
β
FGLS =
Christophe Hurlin (University of Orléans)
N
∑
i =1
xi xi>
σ2i (b
α)
!
1
N
∑
i =1
xi yi
σ2i (b
α)
Advanced Econometrics - HEC Lausanne
!
December 15, 2013
128 / 153
5. Heteroscedasticity
De…nition (Iterated FGLS estimator)
Estimate the parameters β by OLS. Compute the residuals
b
bεi = yi xi> β
OLS and estimate the parameters α according to the
appropriate model. Compute the estimated variances σ2i (b
α) and compute
the FGLS estimator:
! 1
!
N
N
>
x
x
(1 )
x
y
i
i
i
b
β
∑ 2 αi )
∑ 2 α)
FGLS =
i =1 σ i ( b
i =1 σ i ( b
(1 )
b
xi> β
FGLS and estimate the parameters α
b (2 ) and so
according to the appropriate model. Compute the FGLS β
FGLS
on...The procedure stop when
Compute the residuals bεi = yi
sup
j =1,..,K
b (i )
β
j ,FGLS
Christophe Hurlin (University of Orléans)
b (i 1 ) < threshold (ex: 0.001)
β
j ,FGLS
Advanced Econometrics - HEC Lausanne
December 15, 2013
129 / 153
5. Heteroscedasticity
Example (Harvey’s (1976) multiplicative model of heteroscedasticity)
Consider the generalized linear regression model:
AVGEXPi = β1 + β2 AGEi + β3 Ownrenti + β4 Incomei + β5 Income2i + εi
where the heteroscedasticity satis…es the Harvey’s (1976) speci…cation
V ( εi j X) = σ2i = exp (α1 + α2 Incomei )
The data are available in …le Chapter5_data.xls. Question: write a
Matlab code to estimate the parameters by FGLS by using a two-step and
an iterative estimator.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 15, 2013
130 / 153
5. Heteroscedasticity
Remark
A way to get the estimates of the parameters α1 and α2 is to consider the
regression:
ln bε2i = α1 + α2 Incomei + vi
and to estimate the parameters by OLS.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 15, 2013
131 / 153
5. Heteroscedasticity
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 15, 2013
132 / 153
5. Heteroscedasticity
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 15, 2013
133 / 153
5. Heteroscedasticity
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 15, 2013
134 / 153
5. Heteroscedasticity
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 15, 2013
135 / 153
5. Heteroscedasticity
Key Concepts
1
OLS and robust inference
2
White heteroscedasticity consistent estimator
3
GLS and Weighted Least Squares (WLS)
4
FGLS: two-steps and iterated estimators
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 15, 2013
136 / 153
Section 6
Testing for Heteroscedasticity
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 15, 2013
137 / 153
6. Testing for heteroscedasticity
Objectives
The objective of this section are to introduce the following tests for
heteroscedasticity:
1
White general test
2
The Breusch-Pagan / Godfrey LM test
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 15, 2013
138 / 153
6. Testing for heteroscedasticity
De…nition (White test for heteroscedasticity)
The White test for heteroscedasticity is based on:
H0 : σ2i = σ2
H1 : σ2i 6= σ2j
Christophe Hurlin (University of Orléans)
for i = 1, .., N
for at least one pair (i, j )
Advanced Econometrics - HEC Lausanne
December 15, 2013
139 / 153
6. Testing for heteroscedasticity
The intuition of the test is based on the following idea:
1
If there is no heteroscedasticity (under the null H0 ):
b
Vasy β
OLS
2
= σ2 Q
1
b
b asy β
V
OLS
= σ 2 X> X
b
Vasy β
OLS
= σ2 Q
1
Under the alternative (heteroscedasticity):
b
b asy β
V
OLS
Christophe Hurlin (University of Orléans)
= σ 2 X> X
1
1
Q Q
1
X> ΩX X> X
Advanced Econometrics - HEC Lausanne
1
December 15, 2013
140 / 153
6. Testing for heteroscedasticity
White (1980) proposes the following procedure and test-statistic:
Step 1: Estimation of the model using the OLS estimator of β.
Step 2: Determine the residuals bεi = yi
b
xi> β
OLS .
Step 3: Regress bε2i on a constant and all unique columns vectors contained
in X and all the squares and cross-products of the column vectors in X.
Step 4: Determine the coe¢ cient of determination, R2 , of the previous
regression.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 15, 2013
141 / 153
6. Testing for heteroscedasticity
De…nition (White test for heteroscedasticity)
Under the null, the White test-statistic N R2 converges:
N
d
R2 ! χ2 (m
H0
1)
where m is the number of explanatory variables in the regression of bε2i .
The critical region of size α is
W= y :N
where χ21
α
R2 > χ21
α
denotes the 1-α critical value of the χ2 (m
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
1) distribution.
December 15, 2013
142 / 153
6. Testing for heteroscedasticity
Example (White’s (1980) test for heteroscedasticity)
Consider the generalized linear regression model:
AVGEXPi = β1 + β2 AGEi + β3 Ownrenti + β4 Incomei + β5 Income2i + εi
The data are available in …le Chapter5_data.xls. Question: write a
Matlab code to compute the White test-statistic for heteroscedasticity and
its p-value. What is you conclusion for a signi…cance level of 5%?
Compare your results with Eviews.
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 15, 2013
143 / 153
6. Testing for heteroscedasticity
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 15, 2013
144 / 153
6. Testing for heteroscedasticity
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 15, 2013
145 / 153
6. Testing for heteroscedasticity
De…nition (Breusch and Pagan test)
Breusch and Pagan (1979) have devised a Lagrange multiplier test of
the hypothesis that
σ2i = σ2 f α0 + zi> α
where zi = (zi 1 ..zip )> is a p 1 vector of independent variables. The test
is:
H0 : α = 0p 1 (homoscedasticity)
H1 : α 6= 0p
Christophe Hurlin (University of Orléans)
1
(heteroscedasticity)
Advanced Econometrics - HEC Lausanne
December 15, 2013
146 / 153
6. Testing for heteroscedasticity
The test can be carried out with a simple regression of
gi = N
bε2i
b
ε>b
ε
1=N
bε2i
2
εi
∑N
i =1 b
1
on the variables zik for k = 1, ., N and a constant term.
gi = α0 + α1 zi 1 + ... + αp zip + vi
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 15, 2013
147 / 153
6. Testing for heteroscedasticity
De…nition (Breusch and Pagan test-statistic)
De…ne Z the N (p + 1) matrix of observations on (1, zi ) and let g be
the N
1 vector of observations
gi = N
bε2i
b
ε>b
ε
1
Then, the Breusch and Pagan’s test-statistic is de…ned by:
LM =
1 >
g Z Z> Z
2
1
Z> g
Under the null, we have:
d
LM ! χ2 (p )
H0
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 15, 2013
148 / 153
6. Testing for heteroscedasticity
Example (Breusch and Pagan’s (1979) test for heteroscedasticity)
Consider the generalized linear regression model:
AVGEXPi = β1 + β2 AGEi + β3 Ownrenti + β4 Incomei + β5 Income2i + εi
The data are available in …le Chapter5_data.xls. Question: write a
Matlab code to compute the Breusch and Pagan test-statistic for
heteroscedasticity with zi = xi and its p-value. What is you conclusion for
a signi…cance level of 5%?
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 15, 2013
149 / 153
6. Testing for heteroscedasticity
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 15, 2013
150 / 153
6. Testing for heteroscedasticity
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 15, 2013
151 / 153
6. Testing for heteroscedasticity
Key Concepts
1
White test for heteroscedasticity
2
Breusch and Pagan test for heteroscedasticity
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 15, 2013
152 / 153
End of Chapter 5
Christophe Hurlin (University of Orléans)
Christophe Hurlin (University of Orléans)
Advanced Econometrics - HEC Lausanne
December 15, 2013
153 / 153