Math 6020-1, Spring 2014; Partial Solutions to
Assignment 1
7.1. Our model is y = β0 + β1 z + ε. Therefore, the design matrix is


1 10
1 5 


1 7 

Z=
1 19 .


1 11
1 8
And the observed values are Y 0 = (15 , 9 , 3 , 25 , 7 , 13). One computes to
find that
6 60
1
−1/12
0
0
−1
ZZ=
and (Z Z) =
.
60 720
−1/12 1/12
Therefore, the least-squares estimate for β is
−2/3
−0.6667
βb = (Z 0 Z)−1 Z 0 Y = 19
≈
.
/15
1.2667
In addition,


12
 5.667 


 8.2 
b
b


Y = Zβ ≈ 

 23.4 
13.2667
9.4667


3
 3.3333 


 −5.2 
b

.
and εb = Y − Y ≈ 

 1.6 
−6.2667
3.5333
Finally,
SSres := εb0 εb ≈ 101.4667.
7.3. Here, Y = Zβ +ε, where E(ε) = 0 and Cov(ε) = σ 2 V where V is known,
positive definite, and full rank. The trick is to reparametrize the problem
into standard form. Namely, let
Y∗ := V −1/2 Y , Z∗ := V −1/2 Z, ε∗ := V −1/2 ε.
1
Then, Y∗ = Z∗ β + ε∗ ; that is, we have another linear model. But now the
noise satisfies
E(ε∗ ) = V −1/2 E(ε) = 0,
and because the transpose of V −1/2 is itself,
Cov(ε∗ ) = V −1/2 Cov(ε)V −1/2 = σ 2 I.
We know that, for the starred linear model, the [usual] least-squares estimate of β is
−1 0 −1
Z V Y,
βb∗ = (Z∗0 Z∗ )−1 Z∗0 Y∗ = Z 0 V −1 Z
and that this βb∗ is the least-squares estimate in the sense that it is the
minimizer of
0 2
min kY∗ − Z∗ bk = min V −1/2 [Y − Zb]
V −1/2 [Y − Zb]
b
b
0
= min [Y − Zb] V −1 [Y − Zb] .
b
In other words, βb∗ is the weighted least-squares estimate [in terms of the
original data Y ]. Your text writes βb∗ alternatively as βbW .
Next we recall that S∗2 := (n − r − 1)−1 εb0∗ εb∗ is an unbiased estimator of
σ 2 . To finish, it remains to compute S∗2 in terms of the original data Y
[and not the starred data]. But clearly,
0 εb0∗ εb∗ = Y∗ − Z∗ βb∗
Y∗ − Z∗ βb∗
h
i0 h
i
= V −1/2 Y − Z βb∗
V −1/2 Y − Z βb∗
= Y − Z βb∗ V −1 Y − Z βb∗ .
This establishes the statement about the unbiased estimator of σ 2 .
7.4. Consider the linear model y = βz + ε. Here, r = 0 so that β is a scalar,
and the design matrix is now the vector Z = (z1 , . . . , zn )0 .
(a) First we study the case that Cov(ε) = σ 2 I. In this case, the leastsquares estimate of β is
Pn
zi Yi
0
−1 0
b
β = (Z Z) Z Y = Pi=1
n
2 .
i=1 zi
(b) Next we study the case that Cov(ε) = σ 2 diag(z1 , . . . , zn ). In this
case, we apply 7.3 with V = diag(z1 , . . . , zn ). It is easy to see that
V −1 = diag(z1−1 , . . . , zn−1 ), and hence
Z 0 V −1 = (1 , . . . , 1) := 10 .
2
In particular,
Pn
Yi
0
−1 0
b
β∗ = (1 Z) 1 Y = Pi=1
.
n
i=1 zi
(E1 )
(c) Finally, we consider the case that Cov(ε) = σ 2 diag(z12 , . . . , zn2 ) :=
σ 2 V . Now, V −1 = diag(z1−2 , . . . , zn−2 ) and hence
Z 0 V −1 = z1−1 , . . . , zn−1 .
In particular,
n
1 X Yj
.
βb∗ =
n i=1 zj
(E2 )
Some final comments. We can translate the method of 7.3 to the present
setting as follows. For (b) Var(εj ) = σ 2 zj . The method of 7.3 is this, in
the present context: Rescale the linear model as
z
ε
Y
pj =β pj +pj .
|zj |
|zj |
|zj |
| {z }
| {z } | {z }
zj∗
Yj∗
ε∗
j
Pn
Pn
Pn
Pn
∗
∗ 2
So, by (a), βb∗ =
i=1 Yi /
i=1 (zi ) =
j=1 Yi /
i=1 zi as in (E1 ).
Similarly, for (c), we rescale the model as
εj
Yj
=β+
,
zj
zj
|{z}
|{z}
ε∗
j
Yj∗
Pn
Pn
with zj∗ := 1, and proceed to find that βb∗ =
Yi∗ / i=1 (zi∗ )2 =
i=1
Pn
n−1 j=1 (Yi /zi ) as in (E2 ).
3