Math 6010
Solutions to homework 2 1/2 (Week 5)
Pn
2, p. 41. Since Ŷ = HY , i=1 (Yi − Ŷi ) = 10 (I − H)Y . It suffice to show that
(I − H)1 = 0. But this is evident because I − H projects all vectors onto
the orthogonal subspace of C(X), whereas 1 ∈ C(X), since 1 is the first
column of X.
5, p. 41. What we are seeing is that the ith observation satisfies Ti = w sin θi +noise.
The least-squares estimator is found by solving
min f (w) := min
w>0
w>0
n
X
(Ti − w sin θi )2 .
i=1
Of course,
f 0 (w) = −2
n
X
(Ti − w sin θi ) sin θi ,
f 00 (w) = 2w
n
X
(sin θi )2 .
i=1
i=1
Recall that w is a positive unknown. Therefore, f 00 (w) > 0 for all w > 0,
whence we have the answer once we solve f 0 (w) = 0. That is,
Pn
n
n
X
X
Ti sin θi
2
Ti sin θi = w
(sin θi ) =⇒ ŵ = Pi=1
.
n
2
i=1 (sin θi )
i=1
i=1
Note that 0 < θi < π/2 and Ti > 0 [tension of the ith string], therefore
ŵ > 0, as it should be.
8, p. 42. Ŷ is a projection of Y onto C(X),1 whereas Y − Ŷ is a/the projection of Y
Pn
onto the [C(X)]⊥ . In particular, Ŷ ⊥ Y − Ŷ , whence i=1 Ŷi (Yi − Ŷi ) =
Ŷ 0 (Y − Ŷ ) = 0.
1, p. 44. Recall that Cov(β̂) = σ 2 (X 0 X)−1 , and
1 Pn
2
1
i=1 xi
n
(X 0 X)−1 = Pn
2
−x̄
i=1 (xi − x̄)
Pn−x̄
i=1
x2i
.
Therefore, Cov(β̂0 , β̂1 ) = −x̄, which is zero if and only if x̄ = 0.
1 It
is the projection when X is full rank.
1
4, p. 44. Here, p = 3 and the design matrix is

1 x1,1 − x̄1

..
X =  ...
.
1
x1,n − x̄1

x2,2 − x̄2

..
.
.
x2,n − x̄n
In particular,


n
0
0
P
P
n
n
2
(x1,i − x̄1 )(x2,i − x̄2 )
X 0X = 0 P
i=1 (xi,n − x̄1 )
i=1
P
n
n
2
0
i=1 (x1,i − x̄1 )(x2,i − x̄2 )
I=1 (x2,i − x̄2 )


1
0
0
= n 0 sx1 ,x1 sx1 ,x2  ,
0 sx1 ,x2 sx2 ,x2
where
n
sxi ,xj :=
1X
(xi,k − x̄i ) (xj,k − x̄j ) .
n
k=1
0
The inverse of X X is
(X 0 X)−1

1
1
0
=
n sx1 ,x1 sx2 ,x2 − s2x1 ,x2
0
0
sx2 ,x2
−sx1 ,x2
0

−sx1 ,x2  .
sx1 ,x1
Therefore,
Var β̂1 = σ 2 (X 0 X)−1 2,2 =
σ 2 sx2 ,x2
.
n sx1 ,x1 sx2 ,x2 − s2x1 ,x2
The rest is high-school algebra:
sx1 ,x1 sx2 ,x2 −
s2x1 ,x2
= sx1 ,x2 sx2 ,x2
s2x1 ,x2
1−
sx1 ,x1 sx2 ,x2
!
= sx1 ,x1 sx2 ,x2 (1 − r2 );
and hence
Var β̂1 =
σ2
σ2
.
= Pn
2
2
2
nsx1 ,x1 (1 − r )
i=1 (x1,i − x̄1 ) · (1 − r )
2