Solutions
Stat755
Homework 4
4.1 consider a bivariate normal distribution with μ1 = 1, μ2 = 3, σ 11 = 2, σ 22 = 1 and
ρ12 = −.8. (a) Write out the bivariate normal density. (b) Write out the squared statistical distance expression ( x − μ ) ' Σ −1 ( x − μ ) as a quadratic function of x1 and x2. Sln: (a) the density function of bivarate normal distribution is the following: f ( x1 , x2 ) =
=
⎧
⎡
exp ⎨− 2 1−1ρ 2 ⎢
(
2
12 )
⎣
⎩
2π σ 11σ 22 (1 − ρ12 )
1
5
6π 2
{
25 ⎡
exp − 18
⎢⎣
( )
x1 −1
2
2
( ) + ( ) − 2ρ ( )( )⎤⎥⎦ ⎭⎫⎬
x1 − μ1
+ ( x2 − 3) + 85
2
2
x2 − μ 2
σ11
2
σ 22
( )( x
x1 −1
2
2
12
x1 − μ1
x2 − μ 2
σ11
σ 22
}
− 3) ⎤
⎥⎦
(b) The quadratic funciton is the following: ( x − μ ) ' Σ −1 ( x − μ )
=
1
2
1− ρ12
(
⎡
⎢⎣
⎡
) ⎢⎣
( ) + ( ) − 2ρ ( )( )⎤⎥⎦
x1 − μ1
( )
x1 −1
2
σ11
2
x2 − μ2
2
σ 22
+ ( x2 − 3) + 85
x1 − μ1
x2 − μ2
σ11
σ 22
12
( )( x
x1 −1
− 3) ⎤
⎥⎦
=
25
9
=
475 20 2 25 x1 20 2 x1 25 x12 50 x2 20 2 x2 20
25 x22
+
−
−
+
−
−
+
2 x1 x2 +
18
3
9
3
18
3
9
9
9
2
2
2
2
4.2 consider a bivariate normal population with μ1 = 0, μ2 = 2, σ 11 = 2, σ 22 = 1 and
ρ12 = .5. (a) Write out the bivariate normal density (b) Write out the squared generalized distance expression ( x − μ ) ' Σ −1 ( x − μ ) as a function of x1 and x2. (c) Determin (and sketch) the constant‐density contour that contains 50% of the probability. Sln: (a) the density function of bivarate normal distribution is the following: 1 Solutions
Stat755
f ( x1 , x2 ) =
=
⎧
⎡
exp ⎨ − 2 1−1ρ 2 ⎢
⎩ ( 12 ) ⎣
2π σ 11σ 22 (1 − ρ122 )
1
1
2π
3
2
{
exp − 23 ⎡
⎣⎢
( )
x1
2
2
Homework 4
( ) ( ) − 2ρ ( )( )⎤⎥⎦ ⎭⎫⎬
2
x1 − μ1
σ11
+ ( x2 − 2 ) − 1⋅
2
( )( x
σ 22
x1
2
2
2
x2 − μ2
+
12
x1 − μ1
x2 − μ2
σ11
σ 22
}
− 2)⎤
⎦⎥
(b) ( x − μ ) ' Σ −1 ( x − μ )
( ) + ( ) − 2ρ ( )( )⎤⎥⎦
(
⎡
) ⎢⎣
= 23 ⎡
⎢⎣
( )
=
1
2
1− ρ12
x1
2
x1 − μ1
σ11
2
2
x2 − μ2
2
σ 22
12
+ ( x2 − 2 ) −
2
( )( x
x1
2
2
x1 − μ1
x2 − μ2
σ11
σ 22
− 2)⎤
⎥⎦
8 2 2 x1 x12 8 x2 1
2x2
= +
+ −
−
2 x1 x2 + 2
3
3
3
3 3
3
(c) The solid ellipsoid of (x1,x2) satisfy ( x − μ ) ' Σ −1 ( x − μ ) ≤ χ p2 (α ) = c 2 will has probability 50%, where p = 2. So, α = 50%. From distribution table, we have
χ 22 ( 0.5 ) =1.39, therefore, c = 1.18. σ 11
σ 21 ⎤ ⎡
⎡σ
The covariance matrix Σ = ⎢ 11
=⎢
⎥
⎣σ 21 σ 22 ⎦ ⎣⎢ ρ 21 σ 11σ 22
ρ12 σ 11σ 22 ⎤ ⎡ 2
σ 22
⎥=⎢1
⎦⎥ ⎣
⎤
. 1 ⎥⎦
1
2
2
⎛
⎛
⎡ 0.460 ⎤ ⎞
⎡ −.888⎤ ⎞
The eigen pairs are ⎜ 2.37, ⎢
⎟ and ⎜ .634, ⎢
⎥ ⎟ . Therefore, we have the axes ⎥
.460
.0.888
−
−
⎣
⎦⎠
⎣
⎦
⎝
⎝
⎠
as: c λ1 = 1.18 ⋅ 2.37 = 1.81 and c λ2 = 1.18 ⋅ 0.634 = 0.937. 2 Solutions
Stat755
Homework 4
⎡ 1 −2 0 ⎤
4.3 Let X be N 3 ( μ , Σ ) with μ ' = [ −3 1 4] and Σ = ⎢ −2 5 0 ⎥ . Which of the ⎢
⎥
⎢⎣ 0 0 2 ⎥⎦
following random variables are independent? Explain. (a) X1 and X2. (b) X2 and X3 . (c) (X1,X2) and X3. (d) X 2 − 52 X 1 − X 3 . X1 + X 2
and X3. (e) X2 and 2
Sln: (a) σ 12 = σ 21 = −2, X1 and X2 are dependent. (b) σ 13 = σ 31 = 0, X2 and X3 are independent. ⎡ X1 ⎤
⎡ Y1 ⎤ ⎡1 1 0 ⎤ ⎢ ⎥ ⎡ X 1 + X 2 ⎤
⎡1 1 0 ⎤
(c) Let A = ⎢
⎥ . Therefore,
⎥ , so ⎢Y ⎥ = ⎢0 0 1 ⎥ ⎢ X 2 ⎥ = ⎢ X
⎣0 0 1 ⎦
⎦⎢ ⎥ ⎣
3
⎣ 2⎦ ⎣
⎦
⎣ X3 ⎦
⎡ Y1 ⎤
T
⎢Y ⎥ ∼ N ( Aμ , AΣA ) . ⎣ 2⎦
⎡ 1 −2 0 ⎤ ⎡1 0 ⎤
⎡1 0 ⎤
⎡1 1 0 ⎤ ⎢
⎡ −1 3 0 ⎤ ⎢
⎥
⎢
⎥
⎥ = ⎡2 0⎤ . AΣA = ⎢
−
=
2
5
0
1
0
1
0
⎥
⎢
⎥
⎥
⎢
⎥⎢
⎥
⎢
⎥ ⎢
⎣0 0 1 ⎦ ⎢ 0 0 2 ⎥ ⎢0 1 ⎥ ⎣ 0 0 2 ⎦ ⎢0 1 ⎥ ⎣0 2 ⎦
⎣
⎦⎣
⎦
⎣
⎦
T
σ 12 = 0, Thus, (X1,X2) and X3 are independent. ⎡ X1 ⎤
⎡ 12 12 0 ⎤
⎡ Y1 ⎤ ⎡ 12 12 0 ⎤ ⎢ ⎥ ⎡ 12 X 1 + 12 X 2 ⎤
(d) Let A = ⎢
⎥ . Therefore,
⎥ , so ⎢Y ⎥ = ⎢0 0 1 ⎥ ⎢ X 2 ⎥ = ⎢
X3
⎣0 0 1 ⎦
⎣ 2⎦ ⎣
⎦⎢X ⎥ ⎣
⎦
⎣ 3⎦
⎡Y1 ⎤
⎢Y ⎥ ∼ N ( Aμ ,
⎣ 2⎦
AΣAT ) . ⎡ 1 −2 0 ⎤ ⎡ 12 0 ⎤
0
⎡ − 12
⎡
⎤⎢
⎢
⎥
⎥
T
1
AΣA = ⎢
⎥ ⎢ −2 5 0 ⎥ ⎢ 2 0 ⎥ = ⎢
⎣0 0 1 ⎦ ⎢ 0 0 2 ⎥ ⎢0 1 ⎥ ⎣ 0
⎣
⎦⎣
⎦
1
2
1
2
σ 12 = 0, Thus, X1 + X 2
and X3 are independent. 2
3 ⎡ 1 0⎤
0 ⎤ ⎢ 12
⎡ 12 0 ⎤
⎥
0⎥ = ⎢
. 0 2 ⎥⎦ ⎢ 2
0 2 ⎥⎦
⎣
⎢⎣0 1 ⎥⎦
3
2
Solutions
Stat755
Homework 4
⎡ X1 ⎤
X2
⎡ 0 1 0⎤
⎤
⎡ Y1 ⎤ ⎡ 0 1 0 ⎤ ⎢ ⎥ ⎡
, so ⎢ ⎥ = ⎢ 5
(e) Let A = ⎢ 5
X
=
2
⎥
⎥
⎢
⎥ . Therefore,
5
⎢ ⎥
⎣Y2 ⎦ ⎣ − 2 1 −1⎦ ⎢ X ⎥ ⎣ − 2 X 1 + X 2 − X 3 ⎦
⎣ − 2 1 −1⎦
⎣ 3⎦
⎡ Y1 ⎤
T
⎢Y ⎥ ∼ N ( Aμ , AΣA ) . ⎣ 2⎦
⎡0
AΣAT = ⎢ 5
⎣− 2
⎡0 − 52 ⎤
⎡ 1 −2 0 ⎤ ⎡0 − 52 ⎤
1 0 ⎤⎢
⎡ −2 5 0 ⎤ ⎢
⎡ 5 10 ⎤
1 1 ⎥⎥ = ⎢
. −2 5 0 ⎥⎥ ⎢⎢1 1 ⎥⎥ = ⎢ 9
⎥
⎥
⎢
⎢
1 −1⎦
10 23 14 ⎥⎦
− 2 10 −2 ⎦
⎣
⎣
⎢⎣0 −1 ⎥⎦
⎢⎣ 0 0 2 ⎥⎦ ⎢⎣0 −1 ⎥⎦
σ 12 = 10, Thus, X1 + X 2
and X3 are dependent. 2
⎡1 1 1 ⎤
4.4 Let X be N 3 ( μ , Σ ) with μ ' = [ 2 −3 1] and Σ = ⎢1 3 2 ⎥ .
⎢
⎥
⎢⎣1 2 2 ⎥⎦
(a) Find the distribution of 3X1 2X2+ X3. (b) Relabel the variables if necessary, and find a 2 by 1 vector a such that X2 and
⎡X ⎤
X 2 − a ' ⎢ 1 ⎥ are independent. ⎣ X3 ⎦
⎡ X1 ⎤
Sln: (a) Let a ' = [3 −2 1] , so [3 −2 1] ⎢ X 2 ⎥ = [3 X 1
⎢ ⎥
⎢⎣ X 3 ⎥⎦
[3 X 1
−2 X 2
−2 X 2
X 3 ] . Therefore,
⎡2⎤
X 3 ] . ∼ N ( a ' μ , a ' Σa ) . Where a ' μ = [3 −2 1] ⎢ −3⎥ = 13,
⎢ ⎥
⎣⎢ 1 ⎦⎥
⎡1 1 1 ⎤ ⎡ 3 ⎤
⎡3⎤
⎢
⎥
⎢
⎥
a ' Σa = [3 −2 1] ⎢1 3 2 ⎥ ⎢ −2 ⎥ = [ 2 −1 1] ⎢⎢ −2 ⎥⎥ = 9. The distribution is N (13, 9 ) . ⎢⎣1 2 2 ⎥⎦ ⎢⎣ 1 ⎥⎦
⎢⎣ 1 ⎥⎦
(b) Let a ' = [ a1
⎡ X1 ⎤
a2 ] , so Y = X 2 − a ' ⎢ ⎥ = X 2 − [ a1
⎣ X3 ⎦
⎡ 0 1 0 ⎤
⎡X ⎤
Therefore, A = ⎢
. Thus, ⎢ 2 ⎥ ∼ N ( Aμ ,
⎥
⎣Y ⎦
⎣ −a1 1 −a2 ⎦
4 ⎡X ⎤
a2 ] ⎢ 1 ⎥ = [ −a1 X 1
⎣ X3 ⎦
AΣAT ) . Where, X2
−a2 X 3 ] . Solutions
Stat755
Homework 4
⎡1 1 1 ⎤ ⎡0 − a1 ⎤
⎡ 0 1 0 ⎤⎢
AΣA = ⎢
1 3 2 ⎥⎥ ⎢⎢1 1 ⎥⎥
⎥
⎢
⎣ − a1 1 −a2 ⎦ ⎢1 2 2 ⎥ ⎢0 − a ⎥
2⎦
⎣
⎦⎣
T
⎡0 − a1 ⎤
⎤⎢
1 1 ⎥⎥ 3 − a1 − 2a2 2 − a1 − 2a2 ⎥⎦ ⎢
⎢⎣0 −a2 ⎥⎦
3
3 − a1 − 2a2
⎡
⎤
=⎢
⎥
2
2
⎣3 − a1 − 2a2 a1 − 2a1 − 4a2 + 2a1a2 + 2a2 + 3⎦
1
⎡
=⎢
⎣1 − a1 − a2
3
2
⎡X ⎤
Since 3 − a1 − 2a2 = 0 implies X2 and X 2 − a ' ⎢ 1 ⎥ are independent. So we have vector
⎣ X3 ⎦
⎡3⎤
⎡ −2 ⎤
a = ⎢ ⎥ + c ⎢ ⎥ ,c ∈ . ⎣0⎦
⎣1⎦
⎡ 4 0 −1⎤
4.6 Let X be N 3 ( μ , Σ ) with μ ' = [1 −1 2] and Σ = ⎢ 0 5 0 ⎥ . Which of the following ⎢
⎥
⎢⎣ −1 0 2 ⎥⎦
random variables are independent? Explain. (a) X1 and X2. (b) X1 and X3. (c) X2 and X3. (d) (X1, X3) and X2. (e) X1 and
X1 + 3 X 2 − 2 X 3. Sln: (a) σ 12 = σ 21 = 0, X1 and X2 are independent. (b) σ 13 = σ 31 = −1, X1 and X3 are dependent. (c) σ 23 = σ 32 = 0, X2 and X3 are independent. ⎡ X1 ⎤
⎡ Y1 ⎤ ⎡1 0 1 ⎤ ⎢ ⎥ ⎡ X 1 + X 3 ⎤
⎡1 0 1 ⎤
(d) Let A = ⎢
⎥ . Therefore,
⎥ , so ⎢Y ⎥ = ⎢0 1 0 ⎥ ⎢ X 2 ⎥ = ⎢ X
⎣0 1 0 ⎦
⎦ ⎢X ⎥ ⎣
⎣ 2⎦ ⎣
⎦
2
⎣ 3⎦
⎡ Y1 ⎤
T
⎢Y ⎥ ∼ N ( Aμ , AΣA ) . ⎣ 2⎦
⎡ 4 0 −1⎤ ⎡1 0 ⎤
⎡1 0 ⎤
⎡1 0 1 ⎤ ⎢
⎡3 0 1⎤ ⎢
⎥
⎢
⎥
⎥ = ⎡4 0⎤ . =
AΣA = ⎢
0
5
0
0
1
0
1
⎥
⎢
⎥
⎥
⎢
⎥⎢
⎥
⎢
⎥ ⎢
⎣0 1 0 ⎦ ⎢ −1 0 2 ⎥ ⎢1 0 ⎥ ⎣0 5 0 ⎦ ⎢1 0 ⎥ ⎣ 0 5 ⎦
⎣
⎦⎣
⎦
⎣
⎦
T
σ 12 = 0 Thus, (X1,X3) and X2 are independent. 5 Solutions
Stat755
Homework 4
⎡ X1 ⎤
X1
⎡ Y1 ⎤ ⎡1 0 0 ⎤ ⎢ ⎥ ⎡
⎤
⎡1 0 0 ⎤
(e) Let A = ⎢
, so ⎢ ⎥ = ⎢
,⎢X2 ⎥ = ⎢
⎥ . Therefore,
⎥
⎥
⎣1 3 −2 ⎦
⎣Y2 ⎦ ⎣1 3 −2 ⎦ ⎢ X ⎥ ⎣ X 1 + 2 X 3 − 2 X 2 ⎦
⎣ 3⎦
⎡ Y1 ⎤
T
⎢Y ⎥ ∼ N ( Aμ , AΣA ) . ⎣ 2⎦
⎡ 4 0 −1⎤ ⎡1 1 ⎤
⎡1 1 ⎤
⎡1 0 0 ⎤ ⎢
⎡ 4 0 −1⎤ ⎢
⎡4 6 ⎤
⎥
⎢
⎥
0 5 0 ⎥ ⎢0 3 ⎥ = ⎢
0 3 ⎥⎥ = ⎢
. AΣA = ⎢
⎥
⎥
⎢
⎢
6 15 −5⎦
6 61⎥⎦
⎣1 3 −2 ⎦ ⎢
⎣
⎣
⎢⎣0 −2 ⎥⎦
⎣ −1 0 2 ⎥⎦ ⎢⎣0 −2 ⎥⎦
T
σ 12 = 6, Thus, X1 and X 1 + 3 X 2 − 2 X 3 are dependent. 6