STAT 510
Homework 1 Solutions
Spring 2016
1. We can express A and B in terms of vectors:

    
1 5
1
5
A = 4 −1 = 4 , −1 = a1 , a2 ,
3×2
0 1
0
1
0 b
3, 3, −2, 4
3 3 −2 4
B =
=
.
= (1)
5 −1 2 8
5, −1, 2, 8
b0(2)
2×4
Then AB is a sum of two matrices:
X
2
b0(1)
=
ai b0(i)
AB = a1 , a2
b0(2)
3×4
i=1
 
 
1 5 


= 4 3 3 −2 4 + −1 5 −1 2 8
0
1

 

3 3 −2 4
25 −5 10 40
= 12 12 −8 16 + −5 1 −2 −8
0 0 0 0
5 −1 2
8


28 −2 8 44
=  7 13 −10 8  .
5 −1 2
8
2. Suppose u ∼ tm (δ). Clearly there exist independent random variables y ∼ N (δ, 1) and
w ∼ χ2m . By slide 20 of set 1,
y
p
∼ tm (δ).
w/m
p
Since both u and the random variable defined by y/ w/m have tm (δ) distributions, they
must have the same cdf, and thus they are equal in distribution:
y
d
.
u= p
w/m
Squaring, we obtain
d
u2 =
y2
.
w/m
Since y ∼ N (δ, 1), we have y 2 ∼ χ21 (δ 2 /2) by slide 15 of set 1. Further, since y and w are
independent, y 2 and w are also independent (see Theorem 4.3.5 in Casella and Berger1 ). By
slide 22 of set 1, it follows that
d
u2 =
y2
y 2 /1
=
∼ F1,m (δ 2 /2).
w/m
w/m
Hence, u2 ∼ F1,m (δ 2 /2).
1
Casella, G. and Berger, R.L. (2002) Statistical Inference, 2nd edition, Duxubry Press.
1
Comments:
• Some of you said that since u ∼ tm (δ), there
p exist independent random variables y ∼
2
N (δ, 1) and w ∼ χm such that u = y/ w/m. This is a much stronger claim than
p
p
d
equality in distribution (note that u = y/ w/m =⇒ u = y/ w/m, but not the
converse), and is not needed here.
• Many of you forgot about the independence condition! This is a very important condition,
p
highlighted by this counterexample. Suppose y ∼ N (0, 1). Let w = y 2 and u = y/ w/1.
By slide 16 of set 1, w ∼ χ21 . Then we have
y
,
u= p
w/1
where y ∼ N (0, 1) and w ∼ χ21 .
(1)
Since w = y 2 ,
y
y
y
u= p
=p
=
=
|y|
w/1
y 2 /1
−1
1
with probability 0.5
with probability 0.5
Obviously u does not have a t1 distribution; in fact, u isn’t even a continuous random
variable. Thus, we’ve found that u 6∼ t1 despite having (1) hold.
3. Since x ∼ N (2, 1) independent of y ∼ N (0, 1), together they are multivariate normal:
x
2
1 0
∼N
,
,
y
0
0 1
where Cov(x, y) = Cov(y, x) = 0 since x ⊥ y.
Put µ = [2, 0]0 so that we can write [x, y]0 ∼ N (µ, I2×2 ). Using the results on non-central
chi-square distributions (slide 15 of set 1),
x
2
2
x + y = x, y
∼ χ22 (µ0 µ/2),
y
where the non-centrality parameter reduces to
4
1 2
2, 0
= = 2.
µ µ/2 =
0
2
2
0
This gives us x2 + y 2 ∼ χ22 (2), so that
p
P
x2 + y 2 > 3 = P x2 + y 2 > 9 = P χ22 (2) > 9 .
We can compute the above probability in R using the pchisq() function. The documentation
brought up by ?pchisq says
“The non-central chi-squared distribution with df = n degrees of freedom and noncentrality parameter ncp = λ [...] this is the distribution of the sum of squares of n
normals each with variance one, λ being the sum of squares of the normal means.”
2
This implies R parameterizes the non-centrality parameter using µ0 µ = µ21 +· · ·+µ2n (this is the
sum of squares of normal means) rather than µ0 µ/2. So, we need to double the non-centrality
parameter we obtained above when inputing the function arguments. The code
> pchisq(9, df = 2, ncp = 4, lower.tail = FALSE)
gives the result
p
2
2
P
x + y > 3 = 0.2144.
Comments: Some students did a Monte Carlo experiment to approximate this probability;
others did a change of variable transformation or used a moment generating function to
establish the distribution of x2 +y 2 . Noticing that we can write x2 +y 2 in terms of a multivariate
normal vector and applying the results in the slide set, as done above, is not an approximation
(as Monte Carlo methods are), and may be easier than a change of variable transformation
or using a moment generating function.
iid
4. We have z1 , z2 ∼ N (0, 1). By slide 12 of set 1,
z1
∼ N (02×2 , I2×2 ).
z2
(a) Let A =
h
√1 , − √1
2
2
i
. By slide 13 of set 1,
h
i z z1
1
1 z
1
1
1
√ (z1 − z2 ) = √ 1, −1
= √2 , − √2
= A 1 ∼ N (0, AA0 ).
z2
z2
z2
2
2
1
1
1
1
0
√
√
√
√
Since AA = 2 · 2 + − 2 · − 2 = 1, we have
1
√ (z1 − z2 ) ∼ N (0, 1).
2
Squaring, by slide 16 of set 1, we obtain a central chi-square random variable:
1
(z1 − z2 )2 ∼ χ21 .
2
(b) Notice that
√1 (z1 + z2 )
z1 + z2
=q 2
.
|z1 − z2 |
1
2 /1
(z
−
z
)
2
2 1
(2)
Similar to the first step in part (a), the numerator in (2) is standard normal:
1
√ (z1 + z2 ) ∼ N (0, 1).
2
From the result in part (a), the part of the denominator in (2) under the square root is
a central chi-square on one degree of freedom.
3
To have a t1 distribution, we need to show that the random variables in the numerator
and under the square root in the denominator in (2) are independent. We can do this by
showing that [z1 + z2 , z1 − z2 ]0 has bivariate normal distribution with zero covariances.
Let
1 1
B=
,
1 −1
so that
z1 + z2
1 1
z1
z
=
=B 1 .
z1 − z2
1 −1 z2
z2
Then
1 1
1 1
2 0
BB =
=
,
1 −1 1 −1
0 2
0
and we can use the result on slide 13 of set 1 to obtain
z1 + z2
0
2 0
∼N
,
.
z1 − z2
0
0 2
Since z1 + z2 and z1 − z2 are jointly normally distributed, Cov(z1 + z2 , z1 − z2 ) = 0 implies
that they are independent. Consequently, √12 (z1 + z2 ) is independent of 12 (z1 − z2 )2 .
By the result on slide 21 of set 25,
√1 (z1 + z2 )
z1 + z2
=q 2
∼ t1 .
|z1 − z2 |
1
2 /1
(z
−
z
)
2
2 1
Comments:
• Many students failed to check the independence condition needed to establish a t1 distribution in (b). See the comment section in question 1 for the potential perils of doing so.
Other students did not show that z1 + z2 and z1 − z2 were jointly normally distributed;
without this, zero covariance does not imply independence. A web search will provide
counterexamples where x and y are both normally distributed with Cov(x, y) = 0, but
x 6⊥ y because [x, y]0 is not bivariate normal.
• Some students did a change of variable transformation or used a moment generating
+z2
function, others showed that |zz11 −z
is a ratio of independent standard normal random
2|
variables and hence Cauchy(0,1), which is the same distribution as t1 .
4
iid
5. We are given y1 , . . . , yn ∼ N (µ, σ 2 ).
(a) Using properties of matrix transposition,
n
1 X
s =
(yi − ȳ )2
n − 1 i=1
2
1
(y − ȳ 1n×1 )0 (y − ȳ 1n×1 )
n−1
!0
!
n
n
1
1X
1X
=
y−
yi 1n×1
y−
yi 1n×1
n−1
n i=1
n i=1
!0
!
n
n
X
X
1
1
1
=
y − 1n×1
yi
y − 1n×1
yi
n−1
n
n
i=1
i=1
0 1
1
1
0
0
=
y − 1n×1 1n×1 y
y − 1n×1 1n×1 y
n−1
n
n
0 1
1
1
0
0
=
In×n − 1n×1 1n×1 y
In×n − 1n×1 1n×1 y
n−1
n
n
0 1
1
1
0
0
0
y In×n − 1n×1 1n×1
In×n − 1n×1 1n×1 y
=
n−1
n
n
0 1
1
1
0
0
0
=y
In×n − 1n×1 1n×1
In×n − 1n×1 1n×1 y
n−1
n
n
0
= y By,
=
where
1
1
0
0
0
In×n − 1n×1 1n×1 ) (In×n − 1n×1 1n×1 .
n
n
In×n − P1n×1 , because
1
B=
n−1
Notice that B =
1
n−1
1
1n×1 1n×1 = 1n×1 (n−1 )1n×1
n
= 1n×1 (10n×1 1n×1 )−1 1n×1
= P1n×1 .
Comments: As seen above, we do not need to assume any model for y (e.g., we do not
need the GMMNE on slide 12 of set 2) to show the existence of B such that s2 = y 0 By.
(b) From part (a), we have B =
1
(I
n−1 n×n
iid
− P1n×1 ). Since yi ∼ N (µ, σ 2 ), it follows that
[y1 , . . . , yn ]0 = y ∼ N (µ, Σ) ,
where µ = µ1n×1 and Σ = σ 2 In×n . Clearly Σ is positive definite, assuming that σ 2 > 0:
5
for any non-zero t = [t1 , . . . , tn ]0 ∈ Rn (i.e., t 6= 0n×1 ), we have
t0 Σt = t0 [σ 2 In×n ]t
= σ 2 t0 In×n t
= σ 2 t0 t
= σ 2 (t21 + · · · + t2n )
>0
since t 6= 0n×1 implies ti 6= 0 for at least one i ∈ { 1, . . . , n }.
Set A = n−1
B. By properties of rank and the results on slide 5 of set 2 that pertain to
σ2
the rank of an orthogonal projection matrix,
n−1
1
rank(A) = rank
(In×n − P1n×1 )
·
σ2
n−1
= rank(In×n − P1n×1 )
= n − rank(1n×1 )
= n − 1.
As an orthogonal projection matrix, P1n×1 is symmetric, and clearly In×n is symmetric.
Hence A is also symmetric. We also have that AΣ is idempotent:
1
n−1
1
n−1
2
2
·
(In×n − P1n×1 ) σ In×n
·
(In×n − P1n×1 ) σ In×n
AΣAΣ =
σ2
n−1
σ2
n−1
= (In×n − P1n×1 )(In×n − P1n×1 )
= (In×n − P1n×1 )
1
n−1
2
·
(In×n − P1n×1 ) σ In×n
=
σ2
n−1
= AΣ,
since In×n − P1n×1 is idempotent (this is easy to show using the fact that P1n×1 is idempotent).
We have now established all the ingredients that we need to apply the result on slide 18
of set 1. Hence,
n−1 2
s ∼ χ2n−1 (µ0 Aµ/2).
y 0 Ay =
σ2
6
The non-centrality parameter reduces to zero:
µ0 Aµ/2 = [µ1n×1 ]0
1
n−1 1
(In×n − P1n×1 ) [µ1n×1 ] ·
2
σ n−1
2
µ2 0
1 (In×n − P1n×1 )1n×1
2σ 2 n×1
µ2
= 2 (10n×1 In×n 1n×1 − 10n×1 P1n×1 1n×1 )
2σ
µ2
= 2 (10n×1 1n×1 − 10n×1 1n×1 (10n×1 1n×1 )−1 10n×1 1n×1 )
2σ
µ2 0
= 2 (1n×1 1n×1 − 10n×1 1n×1 )
2σ
µ2
= 2 (n−1 − n−1 )
2σ
= 0,
=
proving the desired result that
n−1 2
s ∼ χ2n−1 .
2
σ
Comments: Many students didn’t establish all the conditions listed on slide 18, such
as A is symmetric or that y is multivariate normal. A few students assumed the noncentrality parameter was zero without any mention of µ0 Aµ/2, and others only showed
minimal work in establishing that µ0 Aµ/2 = 0.
6. (a) Consider a matrix

Am×n
a11
 a21

=  ..
 .
a12
a22
..
.
···
···
..
.
am1 am2 · · ·

a1n
a2n 

..  ,
. 
amn
so that



a11 a21 · · · am1
a11 a12 · · · a1n
 a12 a22 · · · am2   a21 a22 · · · a2n 



0
A A =  ..
..
..   ..
..
.. 
.
.
.
.
 .
.
.
.
.  .
.
. 
a1n a2n · · · anm
am1 am2 · · · amn

 Pm
P
Pm
m
2
a
a
a
·
·
·
a
a
i1
i2
i1
in
i1
i=1
i=1
i=1
P
Pm
m
2
 Pm ai1 ai2

···
 i=1
i=1 ai2
i=1 ai2 ain 
=
.
..
..
..
...


.
.
.
Pm
Pm
Pm 2
i=1 ai1 ain
i=1 ai2 ain · · ·
i=1 ain
⇒: [“only if” part] Suppose that A = 0m×n . Clearly then A0n×m = 0n×m , which implies
A0 A = 0n×m 0m×n = 0n×n .
0
⇐: [“if” part] Suppose that A
A = 0n×n . This requires that the diagonal elements of
P
0
2
A A are only zeros, that is, m
i=1 aij = 0 for j = 1, . . . n. Consequently, aij = 0 for
7
j = 1, . . . , n, i = 1, . . . , m, which implies A = 0m×n .
Thus, A = 0 ⇐⇒ A0 A = 0.
Comments: This question, as well as the next one, requires a proof of an “if and only if”
statement. This means you need to consider both directions (both the “if” and “only
if” parts) to prove the desired conclusion. While it is trivial to many that A = 0 =⇒
A0 A = 0, if you don’t include this in your proof, you can’t claim to have proven the
desired result. Additionally, your proof needs to hold for a matrix A with any dimension,
not just 2 × 2.
(b) ⇐: [“if” part] Suppose XA = XB. This part of the proof is trivial: multiplying both
sides by X 0 gives X 0 XA = X 0 XB.
⇒: [“only if” part] Conversely, suppose X 0 XA = X 0 XB. By properties of matrix
algebra and transpose, as well as the result of part (a), we have
X 0 XA = X 0 XB =⇒ X 0 XA − X 0 XB = 0
=⇒ X 0 X(A − B) = 0
=⇒ (A − B)0 X 0 X(A − B) = 0
0
=⇒ X(A − B) X(A − B) = 0
=⇒ X(A − B) = 0
=⇒ XA − XB = 0
=⇒ XA = XB.
by part (a)
Therefore, X 0 XA = X 0 XB ⇐⇒ XA = XB.
Comments: As in part (a), a proof of an“if and only if” statement requires both directions
to be complete. Additionally, note that depending on the matrix dimensions, we may
have
0
X(A − B) X(A − B) = 0n×n 6= 0m×n = X(A − B),
where m, n ∈ { 1, 2, . . . }.
(c) Let (X 0 X)− be any generalized inverse of X 0 X, which by definition implies
X 0 X(X 0 X)− X 0 X = X 0 X,
where X has dimension m × n, say. Put A = (X 0 X)−1 X 0 X and B = In×n , so that
X 0 X (X 0 X)− X 0 X = X 0 X = X 0 X In×n =⇒ X 0 XA = X 0 XB.
|{z}
{z
}
|
B
A
By the result of part (b), it follows that XA = XB, and hence
X(X 0 X)− X 0 X = X.
8
(d) Let A be any symmetric matrix and G be any generalized inverse of A. By definition,
AGA = A.
Now, transpose both sides and use the fact that A0 = A by symmetry:
(AGA)0 = A0 =⇒ A0 G0 A0 = A0
=⇒ AG0 A = A.
Hence, G0 is a generalized inverse of A.
(e) Let G be any generalized inverse of X 0 X. Notice that X 0 X is symmetric, so by part (d),
G0 is also a generalized inverse of X 0 X. The result of part (c) holds for any generalized
inverse of X 0 X, and hence holds using G0 . Using the result of part (c) with G0 and then
taking transposes gives
XG0 X 0 X = X =⇒ (XG0 X 0 X)0 = X 0
=⇒ X 0 [X 0 ]0 [G0 ]0 X 0 = X 0
=⇒ X 0 XGX 0 = X 0 .
Because we chose G to be any generalized inverse of X 0 X,
X 0 X(X 0 X)− X 0 = X 0 .
Comments: We could have
(X 0 X)− 6= [(X 0 X)− ]0 ,
so it is important that (c) and (d) are used at the right steps in your proof so it is clear
that you aren’t trying to say (X 0 X)− = [(X 0 X)− ]0 . On a related note, we may also
have [(X 0 X)− ]0 6= [(X 0 X)0 ]− .
(f) This part requires two proofs that PX is idempotent for full credit.
1. By part (c),
PX PX = X(X 0 X)− X 0 X (X 0 X)− X 0
{z
}
|
X
0
= X(X X)− X 0
= PX .
2. By part (e),
PX PX = X(X 0 X)− X 0 X(X 0 X)− X 0
|
{z
}
X0
0
−
= X(X X) X
= PX .
9
0
(g) Let G1 and G2 be any generalized inverses of X 0 X. By parts (c) and (e), we have
XG1 X 0 = XG1 X 0 XG2 X 0
|
{z
}
part (e) holds for any generalized inverse of X 0 X
= XG1 X X G2 X 0
| {z }
part (c) holds for any generalized inverse of X 0 X
X0
0
X
= XG2 X 0 .
Comments: A few students stated that G1 6= G2 at the beginning of their proof, and
hence did not prove that the desired result holds for any two generalized inverses of
X 0 X, which was your task. Others tried to use the fact that PX is the same matrix
regardless of which generalized inverse of X 0 X is used, but this is what we are trying to
show.
(h) Let (X 0 X)− be any generalized inverse of X 0 X. We know that X 0 X is a symmetric
matrix, so the result of part (d) says that if (X 0 X)− is a generalized inverse of X 0 X,
then [(X 0 X)− ]0 is a generalized inverse of X 0 X. The result of part (g) then establishes
that X(X 0 X)− X 0 = X[(X 0 X)− ]0 X 0 . Hence, these results and properties of matrix
transpose give
0
0
PX
= X(X 0 X)− X 0
= [X 0 ]0 [(X 0 X)− ]0 X 0
= X[(X 0 X)− ]0 X 0
= X(X 0 X)− X 0
by parts (d,g) as explained above
= PX .
Comments: It is important to use parts (d) and (g) at the right steps in your proof so it is
clear that you aren’t trying to say (X 0 X)− = [(X 0 X)− ]0 .
7. Let X be an n × p matrix and y be an n × 1 vector. Suppose that z ∈ C(X) and z 6= PX y,
which implies (PX y − z) 6= 0n×1 . Observe that z ∈ C(X) implies that PX z = z. Using this
result and the fact that PX is symmetric and idempotent, it follows that
(y − PX y)0 (PX y − z) = (y 0 − [PX y]0 )(PX y − z)
0
)(PX y − z)
= (y 0 − y 0 PX
0
0
= (y − y PX )(PX y − z)
= y 0 PX y − y 0 z − y 0 PX PX y + y 0 PX z
= y 0 PX y − y 0 z − y 0 PX y + y 0 PX z
= −y 0 z + y 0 z
= 0.
Now that we have (y − PX y)0 (PX y − z) = 0 and (PX y − z) 6= 0, we can use the same
10
argument provided in the homework with a = y − PX y and b = PX y − z:
ky − zk2 = ky − PX y + PX y − zk2
= (y − PX y + PX y − z)0 (y − PX y + PX y − z)
= (y − PX y)0 + (PX y − z)0 y − PX y + PX y − z
= (y − PX y)0 (y − PX y) + 2(y − PX y)0 (PX y − z) + (PX y − z)0 (PX y − z)
= ky − PX yk2 + kPX y − zk2
> ky − PX yk2 .
Hence, ky − zk > ky − PX yk, which says that PX y is the unique point in C(X) that is
closest to y in Euclidean distance.
Comments: You can instead show that (y − PX y)0 (PX y − z) = 0 by orthogonality, but as
this is a proof, you need to provide sufficient reasoning or work to establish this.
11