The Aitken Model
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
1 / 41
The Aitken Model (AM):
Suppose
y = Xβ + ε,
where
E(ε) = 0
and Var(ε) = σ 2 V
for some σ 2 > 0 and some known positive definite matrix V.
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
2 / 41
Because σ 2 V is a variance matrix, V is symmetric and positive definite,
∴ ∃ a symmetric and positive definite matrix V 1/2 3
V 1/2 V 1/2 = V
and V 1/2 is nonsingular with V −1/2 ≡ (V 1/2 )−1 .
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
3 / 41
It follows that under the AM,
V −1/2 y = V −1/2 Xβ + V −1/2 ε ⇐⇒ z = Uβ + δ,
where
z = V −1/2 y,
U = V −1/2 X,
and δ = V −1/2 ε
with
E(δ) = 0
and
Var(δ) = V −1/2 σ 2 VV −1/2
= σ 2 V −1/2 V 1/2 V 1/2 V −1/2
= σ 2 I.
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
4 / 41
Thus, the AM for y is equivalent to the GMM for z = V −1/2 y.
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
5 / 41
Estimability in the AM:
The AM is just a special case of the GLM.
Thus, as before, c0 β is estimable iff c ∈ C(X0 ).
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
6 / 41
Note that
C(X0 ) = C(X0 V −1/2 )
= C((V −1/2 X)0 )
= C(U0 ).
Thus, c ∈ C(X0 ) ⇐⇒ c ∈ C(U0 ).
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
7 / 41
Let Ly be the collection of all linear estimators that are linear in y.
Let Lz be the collection of all linear estimators in z = V −1/2 y. Show that
c
Copyright 2012
Dan Nettleton (Iowa State University)
Ly = Lz .
Statistics 611
8 / 41
Estimating E(y) under the Aitken Model:
Consider QGLS (b) = (y − Xb)0 V −1 (y − Xb).
Finding β̂ GLS that minimizes QGLS (b) over b ∈ Rp is a
Generalized Least Squares (GLS) problem.
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
11 / 41
If
QGLS (β̂ GLS ) ≤ QGLS (b) ∀ b ∈ Rp ,
β̂ GLS is a solution to the GLS problem.
Xβ̂ GLS is known as GLS estimator of E(y) if β̂ GLS is a solution to the
GLS problem.
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
12 / 41
Show that β̂ GLS minimizes QGLS (b) over b ∈ Rp iff β̂ GLS solves
X0 V −1 Xb = X0 V −1 y.
These equations are known as the Aitken Equations (AE).
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
13 / 41
Henceforth, we will use β̂ GLS to denote a solution to the AE.
We will use β̂ or β̂ OLS to denote a solution to the NE
X0 Xb = X0 y. (Ordinary Least Squares)
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
18 / 41
Because of the equivalence between the AE
X0 V −1 Xb = X0 V −1 y
and the NE
U0 Ub = U0 z,
we know a solution to AE is
(U0 U)− U0 z = (X0 V −1 X)− X0 V −1 y.
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
19 / 41
Theorem 4.2 (Aitken Theorem):
Suppose the Aitken Model holds. If c0 β is estimable, then c0 β̂ GLS is the
BLUE of c0 β.
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
20 / 41
Suppose c0 β is estimable.
Suppose the AM holds.
Find Var(c0 β̂ GLS ).
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
22 / 41
Estimation of σ 2 under the Aitken Model:
An unbiased estimator of σ 2 is
z0 (I−PU )z
n−r
based on our previous result for
the GMM.
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
25 / 41
Now, note that
z0 (I − PU )z = z0 (I − PU )0 (I − PU )z
= k(I − PU )zk2 = kz − PU zk2
= kz − U(U0 U)− U0 zk2
= kz − Uβ̂ GLS k2 = kV −1/2 y − V −1/2 Xβ̂ GLS k2
= kV −1/2 (y − Xβ̂ GLS )k2
= (y − Xβ̂ GLS )0 V −1 (y − Xβ̂ GLS ).
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
26 / 41
Thus,
2
σ̂GLS
≡
(y − Xβ̂ GLS )0 V −1 (y − Xβ̂ GLS )
n−r
is an unbiased estimator of σ 2 under the AM.
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
27 / 41
A Simple Example
Suppose for i = 1, . . . , n,
yi = βxi + εi ,
where ε1 , . . . , εn are uncorrelated, E(εi ) = 0 and Var(εi ) = σ 2 xi > 0.
Find the BLUE of β and an unbiased estimator of σ 2 .
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
28 / 41
Find Var(β̂ GLS ) for this example.
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
33 / 41
Find β̂ OLS for this example.
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
36 / 41
Find Var(β̂ OLS ) in this example.
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
38 / 41