STAT 501
Spring 2005
Assignment 1
Name ________________
Reading Assignment:
Johnson and Wichern, Chapter 1, Sections 2.5 and 2.6, Chapter 4, and Chapter 3. Review
matrix operations in Chapter 2 and Supplement 2A.
Written Assignment: Due Monday, January 24, in class.
1.
Consider a random vector X = (X1 X 2 X 3 )′ with mean vector µ = (3, 2, 1)′ and
covariance matrix Σ. The eigenvalues of Σ are λ1 = 12, λ2 = 6, λ3 = 2 and the
corresponding eigenvectors are
⎡ 1/ 3 ⎤
⎢
⎥
e 1 = ⎢ 1/ 3 ⎥
~
⎢
⎥
⎢⎣ 1/ 3 ⎥⎦
⎡ 2/ 6
⎢
e 2 = ⎢ − 1/ 6
~
⎢
⎢⎣ − 1/ 6
⎤
⎥
⎥
⎥
⎥⎦
⎡ 0
⎤
⎢
⎥
e 3 = ⎢ 1/ 2 ⎥
~
⎢ − 1/ 2 ⎥
⎣
⎦
Evaluate the following quantities. Parts (a), (b), (c), (d) and (g) should be done without
using a computer.
(a)
⎮Σ⎮ = _____
(d)
⎡
⎢
∑= ⎢
⎢
⎢
⎣
(f)
⎡
⎢
−1/ 2
∑
= ⎢
⎢
⎢
⎣
(b) trace(Σ) = _____
⎤
⎥
⎥
⎥
⎥
⎦
(e)
⎤
⎥
⎥
⎥
⎥
⎦
(c) V(e′ 1 X ) = _____
~ ~
⎡
⎢
∑ −1 = ⎢
⎢
⎢
⎣
⎤
⎥
⎥
⎥
⎥
⎦
2
(g)
Define Y1 = e1′ X , Y2 = e′2 X , and Y3 = e′3X. Find the mean vector and
covariance matrix for Y = (Y1 Y2 Y3 )′ = Γ′ X , where the i-th column of Γ is the i-th
eigenvector for Σ. Evaluate
⎡
⎢
E( Y) = ⎢
⎢
⎢
⎣
⎤
⎥
⎥
⎥
⎥
⎦
⎡
⎢
V(Y) = Γ′ ∑ Γ = ⎢
⎢
⎢
⎣
⎤
⎥
⎥
⎥
⎥
⎦
(h) The linear transformation, Y = Γ′ X , examined in part (g) is often called a rotation
because it corresponds to simply rotating the coordinate axes. Use the definition of
eigenvectors to show that lengths of vectors are not changed by this transformation,
i.e., show Y′ Y = X′ X when Y = Γ′ X .
(i)
Let Γ be a matrix for which the i-th column of Γ is the i-th eigenvector for Σ, and
let Y = Γ′ X , µ X = E(X) , Σ X = V(X) , µ Y = Γ′ µ X , and ∑ Y = V(Y) = Γ′ ∑X Γ .
Determine which of the following measures of variability or distance are unaffected
by rotations. Justify your answers by either giving a counter example or a proof using
the properties of eigenvalues, eigenvectors, and matrix operations given in chapter 2
of Johnson and Wichern,
(a) Is |ΣX| = |ΣY| ?
(b) Is trace (ΣX) = trace (ΣY) ?
(c) Is ΣX = ΣY ?
−1
(d) Is (X − µ X )′ ∑X
(X − µ X ) = (Y − µ Y )′ ∑ −Y1(Y − µ Y ) ?
(e) Is (X − µ X )′(X − µ X ) = (Y − µ Y )′(Y − µ Y ) ?
2.
Consider a bivariate normal population with mean vector µ
where
−3
⎡ 2 ⎤
⎡ 5
and
µ = ⎢
∑ = ⎢
⎥
9
⎣ 1 ⎦
⎣ −3
(a)
Write down a formula for the joint density function.
and covariance matrix Σ,
⎤
⎥
⎦
3
(b)
Find the eigenvalues and eigenvectors for Σ.
⎛
e1 = ⎜⎜
⎜
⎝
λ1 = ______ λ2 = _____
(c) Find values for ⎮Σ⎮ = _________
⎞
⎟
⎟
⎟
⎠
⎛
e2 = ⎜⎜
⎜
⎝
⎞
⎟
⎟
⎟
⎠
trace (Σ) = _________
(d)
Write down the formula for the boundary of the smallest region such that there is
probability 0.5 that a randomly selected observation will be inside the boundary.
(e)
Sketch the boundary of the region you identified in part (d).
X2
4
3
2
1
X1
-4
-3
-2
-1
1
2
3
4
5
6
-1
-2
3.
(f)
Determine the area of the region described in parts (d) and (e). ______________
(g)
Determine the area of the smallest region such that there is probability
0.95 that a randomly selected observation will be in that region. ______________
The joint density function for X1 and X 2 is
f (x1, x 2 ) = 3π −1 exp{−4.5x12 + 3x1x 2 − 2.5x 22 − 3x1 + 17x 2 − 32.5}
Find the mean vector and the covariance matrix for this bivariate distribution.
4
4.
Let X be a normally distributed random vector with
~
⎛ −3 ⎞
µ = ⎜⎜ 1 ⎟⎟
⎜ −2 ⎟
⎝
⎠
(a)
and
⎛ 4
∑ = ⎜⎜ 0
⎜ −1
⎝
Which of the following are pairs of independent random variables?
(i)
X1 and X2
(iv) (X1 , X3 ) and X2
(ii) X1 and X3
(v) X1 + X3 and X1 - 2X2
(iii) X1 and X1 + 3X2 – 2X3
(vi) X1 + X2+ X3 and 4X1 - 2X2 + 3X3
(b)
What is the distribution of Y = (X1 , X 2 )′ ?
(c)
What is the conditional distribution of Y = (X1 , X 2 )′ given X 3 = x 3 ?
(d)
Find the correlation between X1 and X2 and a formula for the partial correlation
between X1 and X2 given X3 = x3.
ρ12 = ____________
(e)
5.
0 −1 ⎞
5 0 ⎟⎟
0 2 ⎟⎠
ρ12•3 = _______________
What is the conditional distribution of X2 given X1 = x1 and X3 = x3 ?
⎛
⎞
Suppose X ~ N 4 ⎜ µ , ∑ ⎟ where
~
⎝~ ~⎠
⎛ −4 ⎞
⎜ 3 ⎟
⎟
µ = ⎜
⎜
⎟
0
~
⎜
⎟
⎝ 1 ⎠
⎛
⎜
∑ = ⎜
⎜
⎜
⎝
4
0
2
2
0 2 2
1 1 0
1 9 −2
0 −2 4
⎞
⎟
⎟
⎟
⎟
⎠
(a)
What is the distribution of Z1 = 3X1 − 5X2 + X4 ?
(b)
What is the joint distribution of Z1 in part (a) and Z2 = 2X1 − X3 + 3X4 .
(c)
⎛X ⎞
Find the conditional distribution of ⎜ 1 ⎟ given X3 = 1 .
⎝ X4 ⎠
(d)
Find the partial correlation between X1 and X4 given X2 = x2 and X3 = x3 .
5
6.
⎛
⎛
⎞
⎞
Let X be N ⎜⎜ µ1 , ∑1 ⎟⎟ and Y be N ⎜⎜ µ 2 , ∑ 2 ⎟⎟ where X and Y are independent
~
~
~
~
⎝ ~
⎝ ~
⎠
⎠
and
⎛ 8 −2 ⎞
∑1 = ⎜⎜
⎟⎟
⎝− 2 4 ⎠
⎛ 2⎞
µ1 = ⎜⎜ ⎟⎟
~
⎝ 6⎠
⎛ 3 ⎞
⎛ 4
∑ 2 = ⎜⎜
⎝ 2
µ 2 = ⎜⎜ ⎟⎟
~
⎝ 4 ⎠
2⎞
⎟
4 ⎟⎠
(a) Evaluate Cov ⎛⎜ X − Y , X + Y ⎞⎟
~
~
~⎠
⎝~
(b)
Are X − Y and X + Y independent random vectors? Explain.
~
~
~
~
(c) Show that the joint distribution for the four dimensional random vector
⎡ X − Y ⎤
~ ⎥
⎢ ~
is a multivariate normal distribution.
⎢ X + Y ⎥
~ ⎦
⎣ ~
7.
Let d ( p , q ) denote a measure of distance between p and q . Johnson and Wichern
~
~ ~
~
indicate that any distance measure should possess the following four properties:
d ( p , q ) = d ( q , p)
(i)
~ ~
~ ~
d (p , q) > 0
(ii)
if p ≠ q
~
~ ~
d (p , q) = 0
(iii)
~
if p = q
~
~ ~
~
d ( p , q ) ≤ d ( p , r ) + d ( r , q ) , where r = (r1 , r2 )' .
(iv)
~ ~
~ ~
~ ~
~
These properties are satisfied, for example, by Euclidean distance, i.e., .
d ( p , q ) = [ ( p − q ) ' ( p − q )]1 / 2
~ ~
~
~
~
~
(a) In this class we will often consider distance measures of the form
d ( p , q ) = [ ( p − q ) ' A ( p − q )]1 / 2 . Sometimes A will be the inverse of a covariance
~ ~
~
~
~
~
matrix which implies that A is symmetric and positive definite. Show that properties
(i) through (iv) are satisfied when A is symmetric and positive definite.
6
(b) Does A have to be both symmetric and positive definite for
d ( p , q ) = [ ( p − q ) ' A ( p − q )]1 / 2 to satisfy properties (i) through (iv)?
~ ~
~
~
~
~
Present your proofs, counter examples or explanations.
(c) For k-dimensional vectors it is obvious that the measure
d ( p , q ) = max{| p1 − q1 |, | p2 − q 2 |, ..., | p k − q k | }
~ ~
satisfies properties (i), (ii), (iii). Either prove or disprove that it satisfies
property (iv), the triangle inequality.
8.
(a) By multiplication of partitioned matrices, verify for yourself that
⎡ Iq×q B ⎤
⎢
⎥
⎢⎣ 0r ×q I r ×r ⎥⎦
−1
⎡ Iq×q
=⎢
⎢⎣ 0r ×q
−B ⎤
⎡ I r ×r 0r ×q ⎤
⎥ and ⎢
⎥
I r ×r ⎥⎦
⎢⎣ B Iq×q ⎥⎦
−1
⎡ I r ×r 0r ×q
=⎢
⎢⎣ − B Iq×q
⎤
⎥
⎥⎦
for any q × r matrix B, where 0a ×b denotes a matrix of zeros and Ia ×a denotes
an identity matrix.
(b) Show that
Iq×q
B
0r ×q I r ×r
=
I r ×r 0r ×q
= 1 for any q × r matrix B.
B Iq×q
⎡A A ⎤
(c) By multiplication of partitioned matrices verify that if A = ⎢ 11 12 ⎥ is a square
⎣ A 21 A 22 ⎦
matrix for which A11 is q × q matrix and A 22 is an r × r matrix and the inverse of A 22
exists, then
−1
−1 ⎤
⎡ A11 − A12 A 22
0q×r
A 21 0q×r ⎤ ⎡ Iq×q − A12 A 22
⎡ A A ⎤ ⎡ Iq×q
⎢
⎥=⎢
⎥ ⎢ 11 12 ⎥ ⎢
−1
0r ×q
A 22 ⎥⎦ ⎢⎣ 0r ×q
I r ×r ⎥⎦ ⎣ A 21 A 22 ⎦ ⎢⎣ − A 22
⎢⎣
A 21 I r ×r
(d) Use the results from parts (a) through (c) to show that
−1
A = A 22 A11 − A12 A 22
A 21 for any square matrix A with A 22 ≠ 0 .
⎤
⎥
⎥⎦
7
(e) Use the results from parts (a) through (d) to show that
⎡ Iq×q
A −1 = ⎢
−1
A 21
⎢⎣ − A 22
0q × r ⎤
⎥
I r ×r ⎥⎦
−1
⎢( A11 − A12 A 22 A 21 )
−1 ⎡
⎢
⎢⎣
0r ×q
−1
⎤
−1 ⎤ −1
0q×r ⎥ ⎡ Iq×q − A12 A 22
⎢
⎥
⎥
I
⎥
−1 ⎢ 0r ×q
r ×r ⎦
A 22 ⎥⎦ ⎣
for any symmetric matrix A with A 22 ≠ 0 .
9. Consider a multivariate normal distribution with positive definite covariance matrix
⎡ ∑11 ∑12 ⎤
−1
∑=⎢
⎥ . Note that from part (d) in problem 8, ∑ = ∑22 ∑11 − ∑12 ∑22 ∑ 21 ,
∑
∑
⎣ 21 22 ⎦
the product of determinants of covariance matrices for a marginal and a conditional
normal distribution. Use the result from part (e) in problem 8 to expand
⎛ x1 − µ1 ⎞′ ⎡ ∑11 ∑12 ⎤ −1 ⎛ x1 − µ1 ⎞
1
−
(x − µ)′ ∑ (x − µ) = ⎜ ⎟
⎟
⎜
⎜⎝ x 2 − µ2 ⎟⎠ ⎢⎣ ∑21 ∑22 ⎥⎦ ⎜⎝ x 2 − µ 2 ⎟⎠
in an appropriate way to show that the multivariate normal density function is a product of
the density functions for the marginal distribution of X 2 and a conditional distribution.
What happens when ∑12 = 0 , i.e. when X1 and X 2 are uncorrelated?
For additional practice you could do problems 3.6, 3.8, 3.9, 3.10, 3.14, 3.16 at the end of
Chapter 3 and problems 4.1, 4.3, 4.4, 4.5, 4.14, 4.15, 4.16 , 4.17 at the end of Chapter 4.
Problem 4.8 gives a trivial example of a non-normal bivariate distribution with normal marginal
distributions. We will consider a more substantial example on the next assignment. Do not hand
in these additional problems; answers will be given on the solution sheet for this assignment.