Math 6010
Partial solutions to homework 5
1. We have a linear model with p = 2, parameter vector β = (β1 , β2 )0 and
design matrix,


a 1 b1

..  .
X =  ...
.
an
Note that
Pn
a2
X X = Pni=1 i
i=1 ai bi
0
bn
Pn
ai bi
Pi=1
,
n
2
i=1 bi
so that
0
−1
(X X)
Pn 2
bi
Pi=1
= Pn
n
Pn 2
Pn
2
2
−
i=1 ai bi
( i=1 ai ) ( i=1 bi ) − ( i=1 ai bi )
1
Pn
−P i=1 ai bi
,
n
2
i=1 ai
provided that X has full rank.1 Since
β̂ ∼ N2 β , σ 2 (X 0 X)−1 ,
we can read off the following:
Pn
σ 2 i=1 ai bi
Cov β̂1 , β̂2 = − Pn
Pn
Pn
2.
( i=1 a2i ) ( i=1 b2i ) − ( i=1 ai bi )
Pn
This is zero if and only if i=1 ai bi = 0. Because β̂1 and β̂2 are jointly
normal, they are
Pnindependent if and only if they are uncorrelated; that is,
if and only if i=1 ai bi = 0.
2. See page 65 of Lecture 9 in my lecture notes.
5. Since Var(Ŷ ) = X Var(β̂)X 0 = σ 2 X(X 0 X)−1 X 0 , the variance of Ŷi is
the (i , i)th element of the preceding variance/covariance matrix; that is,
Var(Ŷi ) = σ 2
p X
p
X
Xi,j (XX)−1 j,k Xi,k .
j=1 k=1
P
Pn
P
1 In this case, this means that ( n
2
2
b2i ) 6= ( n
i=1
i=1 ai bi ) . According to the
Pn
Pnai )( 2 i=1 P
n
2
Cauchy–Schwarz inequality, ( i=1 ai )( i=1 bi ) ≥ ( i=1 ai bi )2 . Therefore, X has full rank
if and only if the Cauchy–Schwarz inequality is a strict inequality. This turns out to mean
that zi := ai bi is not a linear function of i.
1
Because
n
X
Xi,j Xi,k = [X 0 X]j,k ,
i=1
it follows that
n
X
i=1
Var(Ŷi ) = σ 2
p X
p
X
(XX)−1 j,k [X 0 X]j,k = σ 2 tr (X 0 X)−1 (X 0 X) .
j=1 k=1
Because (X 0 X)−1 (X 0 X) = Ip×p , its trace is p; this does the job.
2