Math 6010 Solutions to homework 2 1/2 (Week 5) Pn 2, p. 41. Since Ŷ = HY , i=1 (Yi − Ŷi ) = 10 (I − H)Y . It suffice to show that (I − H)1 = 0. But this is evident because I − H projects all vectors onto the orthogonal subspace of C(X), whereas 1 ∈ C(X), since 1 is the first column of X. 5, p. 41. What we are seeing is that the ith observation satisfies Ti = w sin θi +noise. The least-squares estimator is found by solving min f (w) := min w>0 w>0 n X (Ti − w sin θi )2 . i=1 Of course, f 0 (w) = −2 n X (Ti − w sin θi ) sin θi , f 00 (w) = 2w n X (sin θi )2 . i=1 i=1 Recall that w is a positive unknown. Therefore, f 00 (w) > 0 for all w > 0, whence we have the answer once we solve f 0 (w) = 0. That is, Pn n n X X Ti sin θi 2 Ti sin θi = w (sin θi ) =⇒ ŵ = Pi=1 . n 2 i=1 (sin θi ) i=1 i=1 Note that 0 < θi < π/2 and Ti > 0 [tension of the ith string], therefore ŵ > 0, as it should be. 8, p. 42. Ŷ is a projection of Y onto C(X),1 whereas Y − Ŷ is a/the projection of Y Pn onto the [C(X)]⊥ . In particular, Ŷ ⊥ Y − Ŷ , whence i=1 Ŷi (Yi − Ŷi ) = Ŷ 0 (Y − Ŷ ) = 0. 1, p. 44. Recall that Cov(β̂) = σ 2 (X 0 X)−1 , and 1 Pn 2 1 i=1 xi n (X 0 X)−1 = Pn 2 −x̄ i=1 (xi − x̄) Pn−x̄ i=1 x2i . Therefore, Cov(β̂0 , β̂1 ) = −x̄, which is zero if and only if x̄ = 0. 1 It is the projection when X is full rank. 1 4, p. 44. Here, p = 3 and the design matrix is 1 x1,1 − x̄1 .. X = ... . 1 x1,n − x̄1 x2,2 − x̄2 .. . . x2,n − x̄n In particular, n 0 0 P P n n 2 (x1,i − x̄1 )(x2,i − x̄2 ) X 0X = 0 P i=1 (xi,n − x̄1 ) i=1 P n n 2 0 i=1 (x1,i − x̄1 )(x2,i − x̄2 ) I=1 (x2,i − x̄2 ) 1 0 0 = n 0 sx1 ,x1 sx1 ,x2 , 0 sx1 ,x2 sx2 ,x2 where n sxi ,xj := 1X (xi,k − x̄i ) (xj,k − x̄j ) . n k=1 0 The inverse of X X is (X 0 X)−1 1 1 0 = n sx1 ,x1 sx2 ,x2 − s2x1 ,x2 0 0 sx2 ,x2 −sx1 ,x2 0 −sx1 ,x2 . sx1 ,x1 Therefore, Var β̂1 = σ 2 (X 0 X)−1 2,2 = σ 2 sx2 ,x2 . n sx1 ,x1 sx2 ,x2 − s2x1 ,x2 The rest is high-school algebra: sx1 ,x1 sx2 ,x2 − s2x1 ,x2 = sx1 ,x2 sx2 ,x2 s2x1 ,x2 1− sx1 ,x1 sx2 ,x2 ! = sx1 ,x1 sx2 ,x2 (1 − r2 ); and hence Var β̂1 = σ2 σ2 . = Pn 2 2 2 nsx1 ,x1 (1 − r ) i=1 (x1,i − x̄1 ) · (1 − r ) 2