Linear Model Under General Variance Structure:
Heteroscedasticity
• A Definition of Heteroscedasticity
In this section, we consider a special case of the model
y = X β + e, or yt = xt β + et, t = 1, .., T.
We relax assumption A4, while maintaining assumption A3.
§ That is, we assume that
E(et et*) = V(et) = σ2 wt2 if t = t*,
= Cov(et, et*) = 0 if t ≠ t*
where wt > 0, t = 1, …, T. While maintaining a zero covariance
across et’s (as implied by assumption A3), this allows for the variance
of et, V(et) = σ2 wt2, to vary across observations (which relaxes
assumption A4). A changing variance of et across observations is
called heteroscedasticity.
§ Given V(e) = σ2 ψ, this corresponds to the following specification for
the (T×T) matrix ψ
w12 0

2
 0 w2
ψ=  .
.

 0 0
. 0

. 0
. . .

. wT2 .
This implies that the variance of et, V(et), is proportional to wt2 (or
equivalently, the standard deviation of et is proportional to wt). In the
special case where σ2 = 1, then V(et) = wt2, and wt is simply the
standard deviation of et, t = 1, 2, …, T.
§ The matrix ψ being non-singular, we have
1/(w12 )
0

2
 0 1/(w2 )
ψ-1 =  .
.

0
 0
0 

.
0 
.
. , → P =

. .1/(wT2 )
.
1/ w1 0
 0 1/ w
2

 .
.

0
 0
0 
. 0 
. . 

. .1/ wT 
.
where ψ-1 = P' P.
§ Then, model M* becomes
y* = X* β + e*, P y = P X β + P e, or
 y1 / w1   x1 / w1 
 y / w  x / w 
 2 2  2 2
 .  =  .  β+

 

yT / wT  xT / wT 
 e1 / w1 
e / w 
 2 2
 . .


eT / wT 
This implies that the t-th observation (yt, xt) is weighted by the inverse
of wt.
§ The generalized least squares (GLS) estimator of β is
βg = (X' ψ-1 X)-1 X' ψ-1 Y
= [ Σ Tt=1 xt' xt/wt2]-1 [ Σ Tt=1 xt' yt/wt2]
= [ Σ Tt=1 (xt/wt)' (xt/wt)]-1 [ Σ Tt=1 (xt/wt)' (yt/wt)].
It is also called a weighted least squares estimator, where the weights
are the inverse of wt.
§ This is as a nice intuitive interpretation. Observations that have a
larger variance (i.e. a larger wt) are less reliable and are weighted less.
Alternatively, observations that have a smaller variance (i.e. a smaller
wt) are more reliable and are weighted more.
• Heteroscedasticity Case 1: Two Unknown Variances
Assume that the sample is partitioned into two sub-samples: T = Ta + Tb.
§ The first sub-sample has Ta observations:
2
ya = Xa β + ea,
where Ya is (Ta×1), Xa is a (Ta×K) matrix of explanatory variables, β
is a (K×1) vector of parameters, and ea a (Ta×1) error term vector
where ea ~ (0, σa2 I T ).
a
§ The second sub-sample has Tb observations: Yb = Xb β + eb,
where yb is (Tb×1), Xb is a (Tb×K) matrix of explanatory variables, β
is a (K×1) vector of parameters, and eb is a (Tb×1) error term vector
where eb ~ (0, σb2 I T ).
b
§ In the case where σa2 ≠ σb2, we have
Y  X 
Y = X β + e, or  a  =  a  β +
 Yb   X b 
 ea 
e  , where
 b
2
 ea 
 ea  s a ITa
E(e) = E   = 0, and V(e) = V   = 
eb 
eb   0
0 
.
2
s b ITb 
Using our earlier notation (where V(e) = σ2 ψ), let σ2 = 1. It follows
that V(et) = wt2 = σa2 if t belongs to first sub-sample
= σb2 if t belongs to the second sub-sample,
and
σ a2 IT
a
ψ= 
 0


.
2
σ b ITb 
0
§ Then, the GLS estimator of β is βg = (X'ψ-1 X)-1 X' ψ-1 y
X 
= [(Xa' Xb') ψ-1  a  ]-1 (Xa' Xb') ψ-1
 Xb 
 Ya 
Y ,
 b
= [(Xa' Xa)/σa2 + (Xb' Xb)/σb2]-1 [(Xa' Ya)/σa2 + (Xb' Yb)/σb2]
= [(Xa/σa)'(Xa/σa)+(Xb/σb)'(Xb/σb)]-1 [(Xa/σa)'(Ya/σa)+(Xb/σb)'(Yb)/σb].
3
• Parameter Estimation Under Case 1
§ Obtain the least squares estimator of β, βs = (X'X)-1 X'Y which is a
consistent estimator of β. Estimate the associated error term
es = Y – X βs which is a consistent estimator of e.
§ Evaluate σiu2 = (yi – Xi βs)' (yi – Xi βs)/(Ti – K), which is an unbiased
and consistent estimator of σi2, i = a, b.
§ Evaluate feasible generalized least squares (FGLS) estimator of β,
βfg = [(Xa' Xa)/σua2 + (Xb' Xb)/σub2]-1 [(Xa' Ya)/σua2 + (Xb' Yb)/σub2).
§ The above FGLS estimator βfg is a consistent, and asymptotically
efficient estimator of β, satisfying
βfg ≈ N[β, (Xa' Xa/σau2 + Xb' Xb/σbu2)-1] as T → ∞.
• Hypothesis Testing Under Case 1
Consider the hypothesis
H0: σa2 = σb2
Null hypothesis:
Alternative hypothesis: H1: σa2 ≠ σb2.
Recall that the sum of squared standard normal variables is distributed as
?2. It follows that, under the normality of e,
[(Ti – K)/σi2] σiu2 = (Yi – Xi βs)' (Yi – Xi βs)/σi2
2
, or σiu2/σi2 ∼
= ΣTt=1 (yi – Xit βs)2/σi2 ∼ χ (T
-K)
i
χ (2Ti − K ) /(Ti
– K), i = a, b.
§ Since σau2 and σbu2 are independently distributed, this implies
2
σ au
/ σ a2
2
σ bu
/ σ b2
∼
χ (2T
/(Ta − K )
χ (2T
/(Tb − K )
a −K )
b −K )
= F(Ta -K,Tb -K) .
§ With null hypothesis H0: σa2 = σb2, we have σau2/σbu2 ∼ F(Ta -K,Tb -K) ,
implying that, under H0, the test statistic (σau2/σbu2) has an F
distribution with (Ta – K) and (Tb – K) degrees of freedom.
4
§ This suggests the following test procedure:
q
Choose the level of significance α = P(Type-1 error) =
P(reject H0| H0 is true)
q
Find c1 and c2 satisfying:
α/2 = P( F( T −K ,T − K ) < c1], and
a
b
α/2 = P( F( T −K ,T − K ) > c2].
a
b
q
Do not reject H0 if c1 ≤ (σau2/σbu2) ≤ c2,
q
Reject H0 if (σau2/σbu2) < c1, or (σau2/σbu2) > c2.
• Heteroscedasticity Case 2: Multiplicative Heteroscedasticity
Consider the model Y = X β + e,
where E(e) = 0, and
E(et et*) = Cov(et, et*) = 0 if t ≠ t*,
= V(et) = σt2 = exp(zt α) if t = t*, t = 1, …, T,
zt α = [1 z t1
α1 
α 
n
. z tn ]  2  = α1 + Σi=2
αi zti,
 . 
 
α n 
α is a (n×1) paramater vector and the zti’s are explanatory variables.
§ It follows that
n
n
V(et) = σt2 = exp(α1 + Σi=2
αi zti) = exp(α1)⋅ Π i=2
exp(αi zti).
This expression indicates that the variance of et depends in a
multiplicative fashion on the explanatory variables zt. This is thus a
case of multiplicative heteroscedasticity.
5
• Parameter Estimation Under Case 2
§ Obtain the least squares estimator of β, βs = (X'X)-1 X'Y which is a
consistent estimator of β. Also evaluate the error term vector:
es = y – X βs which is a consistent estimator of e.
 es1 
§ Given es =  M  , regress ln(est2) on zt = [1
esT 
z t1 . z tn ] ,
t = 1, …, T, to
obtain the least squares estimator
αs = ( ΣTt=1 zt' zt)-1[ ΣTt=1 zt' ln(est2)].
The estimator exp[zt αs] is asymptotically proportional to σt2.
§ Evaluate the FGLS estimator of β,
βfg = [ ΣTt=1 xt'xt/exp(zt αs)]-1 [ ΣTt=1 xt' yt/exp(zt αs)].
§ The above FGLS estimator βfg is a consistent, and asymptotically
efficient estimator of β, satisfying βfg ≈ N(β, V(βfg)] as T → ∞.
§ The variance of βfg, V(βfg), can be consistently estimated by
G ⋅ [ ΣTt=1 xt' xt/exp(zt αs)]-1
where the coefficient of proportionality G is
G = [ ΣTt=1 (yt - xt βfg)2/exp(zt αs)]/(T-K).
6