6. Reparameterization:
Models that may appear to be dierent at rst
sight, may actually be equivalent.
Example 6.1
Two-way classication in Example 3.3. Consider the \cell mean" model.
i = 1; 2
j = 1; 2
k = 1; 2
Yijk = ij + ijk
Matrix notation:
2
6
6
6
6
6
6
6
6
6
6
6
6
6
4
Y111
Y112
Y121
Y122
Y211
Y212
Y221
Y222
3 2
77 6
77 6
6
77 6
6
77 = 6
6
77 6
6
77 6
6
75 6
6
4
1
1
0
0
0
0
0
0
or
0
0
1
1
0
0
0
0
3
0
0
0
0
1
1
0
0
07
0 77
0 777
0 77
0 77
0 777
15
1
2
66
64
11
12
21
22
2
6
3 666
77 666
75 + 66
66
66
4
358
357
66
66
66
66
64
Y111
Y112
Y121
Y122
Y211
Y212
Y221
Y222
or
1
77
66 1
77
61
77 = 666 1
77
66 1
75
64 1
1
1
1
1
1
1
0
0
0
0
0
0
0
0
1
1
1
1
1
1
0
0
1
1
0
0
0
0
1
1
0
0
1
1
1
1
0
0
0
0
0
0
0
0
1
1
0
0
0
0
0
0
0
0
1
1
0
0
0
0
0
0
0
0
1
1
77 6
6
77 6
6
77 6
6
77 6
6
75 6
6
6
4
3
77
77
77
77
77
77
75
Y = W + where ijk NID(0; 2)
The \eects" model:
Yijk ; + i + j + ( )ij + ijk
where
i = 1; 2
ijk NID(0; 2) j = 1; 2
k = 1; 2
Matrix
notation:
2
3
2
32
111
112
121
122
211
212
221
222
The models are \equivalent" in the sense that
the space spanned by the columns of W is the
same as the space spanned by columns of X .
Consequently, you can nd matrices F and G
such that
1
2
1
2
11
12
21
22
Y = X + 3
77
77
77
77
77
75
2
6
6
6
6
6
6
6
W = X 666
6
6
6
6
6
4
0
0
0
0
0
0
1
0
0
0
0
0
0
0
0
0
1
0
3
07
0 77
0 77
0 777
0 77 = XF
0 77
0 777
05
1
and
2
6
X = W 664
359
0
0
0
0
0
1
0
0
0
1
1
1
1
1
1
0
0
0
0
1
1
1
0
1
0
0
1
0
1
1
0
0
0
0
1
0
0
0
0
1
0
0 37
0 77 = W G
05
1
360
Then,
(i) rank(X ) = rank (W )
Example 6.2 Regression model for the yield of
a chemical process in Example 3.1.
Yi = 0 + 1X1i + 2X2i + i
%
%
-
(ii) Estimated mean responses are the same:
Y^ = X (X T X );X T Y
= W (W T W );1W T Y
or
Y^ = PX Y = PW Y
yield
temperature
time
An \equivalent" model:
Yi = 0 + 1(X1i ; X1:) + 2(X2i ; X2:) + 1
(iii) Residual vectors are the same
e = Y ; Y^ = (I ; PX )Y
= (I ; PW )Y
361
For the rst model:
2
3
2
Y
1 X11
1
66
7
66
66 Y2 777
66 1 X12
66 Y3 77 = 66 1 X13
4 Y4 5
4 1 X14
Y5
1 X15
= X + 362
The space spanned by the columns of X is the
same as the space spanned by the columns of
W.
X21
X22
X23
X24
X25
3
77 2
77 6
77 4
5
0
1
2
2
3 6
6
75 + 6
6
6
6
4
1
2
3
4
5
3
7
7
7
7
7
7
5
2
X = W 64
Y1
Y2
Y3
Y4
Y5
3
21
77
61
75 = 664 1
1
X11 ; X1
X12 ; X1
X13 ; X1
X14 ; X1
1 X15 ; X1
= W + X21 ; X2
X22 ; X2
X23 ; X2
X24 ; X2
X25 ; X2
3
2
2
3
77 0
6
75 4 1 5 + 6
6
4
2
3
X1 X2
7
1 0
0 1
5 = WG
and
For the second model:
2
66
64
1
0
0
1
2
3
4
5
3
77
75
2
W = X 64
3
1 ;X1 ;X2 7
0 1 0 5 = XF
0 0 1
and
Y^ = PX Y = PW Y
e = Y ; Y^ = (I ; PX )Y = (I ; PW )Y
363
364
Defn 6.1: Consider two Gauss-Markov models:
Model 1: E (Y) = X and V ar(Y) = 2I
Equivalently, if the linear space spanned by the
columns of X is the same as the linear space
spanned by the columns of W .
The previous examples illustrate that if one
model is a reparameterization of the other,
then
(i) rank(X ) = rank(W )
Model 2: E (Y) = W and V ar(Y) = 2I
where X is an n k model matrix and W is an
n q model matrix.
(ii) Least squares estimates of response means
are the same, i.e., Y^ = PX Y = PW Y
We say that one model is a reparameterization
of the other if there is a k q matrix F and a
q k matrix G such that
W = XF and X = W G :
(iii) Residuals are the same, i.e.,
e = Y ; Y^ = (I ; PX )Y = (I ; Pw)Y
(iv) An unbiased estimator for 2 is provided by
MSE = SSE=(n-rank(X)) where,
SSE = eT e =
YT (I ; PX )Y
= YT (I ; PW )Y
365
366
Result 6.1. Suppose two Gauss-Markov
models:
Model 1: E (Y) = X V ar(Y) = 2I
Model 2: E (Y) = W V ar(Y) = 2I
Reasons fro reparameterizing models:
are reparameterizations of each other, and let
F be a matrix such that W = XF . Then
(i) Reduce the number of parameters
Obtain a full rank model
Avoid use of generalized inverse
(i) If CT is estimable under model 1, then
= F and CT F is estimable under
Model 2.
(ii) Make computations easier
In Examples 6.1 and 6.2,
W T W is a diagonal matrix and
(W T W );1 is easy to compute.
(iii) More meaningfull interpretation of parameters.
(ii) Let ^ = (X T X );X T Y and ^ =
(W T W );W T Y. If CT is estimable, then
CT ^ = CT F ^
(iii) if H0 : CT = d is testable under Model
1, then H0 : CT F = d is testable under
Model 2.
367
368
Proof:
(i) If CT is estimable for Model 1, then (by
Result 3.9 (i))
CT = aT X for some a :
Hence,
CT F = aT X F = aT W
which implies that CT F is estimable for
Model 2.
(ii) Since CT is estimable, the unique b.l.u.e.
is
CT ^ = CT (X T X );X T Y
= aT X T (X T X );X T Y
= aT PX Y for some a
369
By a similar argument
Then,
PW PX = PX
PW = P + W T
= (PX PX )T
= PWT PXT
= P W PX
= PX
371
Since CT F is also estimable, the unique
b.l.u.e. for CT F is
CT F (W T W );W T Y = aT XF (W T W );W T Y
= aT W (W T W );W T Y
= aT PW Y
for the same a.
Hence, the estimators are the same if PX =
PW . To show this, note that
PX W = PX XF = XF = W
which implies
PX PW = PX W (W T W );W T
= W (W T W );W T
= PW
370