Equivalent Models and Reparameterization
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
1 / 19
Two linear models
y = Wα + ε
and y = Xβ + ε
are equivalent, or reparameterizations of each other, iff
c
Copyright 2012
Dan Nettleton (Iowa State University)
C(X) = C(W).
Statistics 611
2 / 19
Result 2.8:
If C(X) = C(W), then PX = PW .
Proof of Result 2.8:
∀ y ∈ Rn , y = PX y + (I − PX )y = PW y + (I − PW )y.
We have written y as a sum of a vector in C(X) = C(W) and a vector in
C(X)⊥ = C(W)⊥ .
Such a decomposition is unique by Result A.4.
Therefore, PX y = PW y
∀ y ∈ Rn ⇒ P X = P W .
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
3 / 19
Corollary 2.4:
If C(X) = C(W), then
ŷ = PX y = PW y
and
ε̂ = (I − PX )y = (I − PW )y = y − ŷ.
The fitted values and residuals are the same for the models that are
reparameterizations of one another.
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
4 / 19
Recall C(X) = C(W) ⇐⇒
W = XT for some matrix T and
X = WS for some matrix S.
Result 2.9: If C(W) = C(X) and α̂ solves the NE W 0 Wa = W 0 y, then
β̂ = T α̂ solves the NE X0 Xb = X0 y, where T is defined by W = XT.
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
5 / 19
Example:
Suppose

y11
 y12 


 ..
 .
 
y1n 
y
1

= 1 .
y=

y2
 y21 
 y22 


 .
 ..

y2n2
Consider
1
X = n1
1n2
1n1
0n2
0n1
1n2
c
Copyright 2012
Dan Nettleton (Iowa State University)
and
1
W = n1
1n2
0n1
.
1n2
Statistics 611
7 / 19
Find T 3 W = XT.
Find β̂ a solution to X0 Xb = X0 y.
Find α̂ a solution to W 0 Wa = W 0 y.
Show that Xβ̂ = W α̂.
c
Copyright 2012
Dan Nettleton (Iowa State University)
Statistics 611
8 / 19