THE UNIVERSITY OF HONG KONG
DEPARTMENT OF STATISTICS AND ACTUARIAL SCIENCE
STAT1301
PROBABILITY AND STATISTICS I
EXAMPLE CLASS 6
Review
Joint and Marginal Distributions
1. Let X1 , . . . , Xn be random variables defined on the same sample space Ω. The joint distribution
function of (X1 , . . . , Xn ) is defined by
F (x1 , . . . , xn ) = P (X1 ≤ x1 , X2 ≤ x2 , . . . , Xn ≤ xn , )
The distribution function FXi of each Xi is called the marginal distribution function of Xi .
2. For discrete random variables X1 , . . . , Xn , the joint probability mass function of (X1 , . . . , Xn )
is
f (x1 , . . . , xn ) = P (X1 = x1 , X2 = x2 , . . . , Xn = xn , )
The mass function fXi = P (Xi = x) of each Xi is called the marginal probability mass function
of Xi .
fXi (x) =
X
u1
...
XX
...
X
f (u1 , . . . , ui−1 , x, ui+1 , . . . , un )
ui−1 ui+1
un
3. Random variables X1 , . . . , Xn are (jointly) continuous if their joint distribution function F
satisfies
Z
Z
x1
F (x1 , . . . , xn ) =
xn
...
−∞
f (u1 , . . . , un )dun . . . du1 ,
−∞
for some nonnegative function f : (−∞, ∞)n → [0, ∞). The function f is called the joint
probability density function of (X1 , . . . , Xn ). The pdf of Xi is called the marginal pdf of Xi .
Z ∞
Z ∞
fXi (x) =
...
f (u1 , . . . , ui−1 , x, ui+1 , . . . , un )dun . . . dui+1 dui−1 . . . du1 .
−∞
−∞
4. If a joint distribution function F possesses all partial derivatives at (X1 , . . . , Xn ), then the joint
pdf is
f (x1 , . . . , xn ) =
∂n
F (x1 , . . . , xn ).
∂x1 . . . ∂xn
1
Independence of random variables
Random variables X1 , . . . , Xn are independent if and only if their joint pmf(pdf) or cdf is equal to
the product of their marginal pmfs(pdfs) or cdfs, i.e.
f (x1 , . . . , xn ) = fX1 (x1 ) . . . fXn (xn ) or F (x1 , . . . , xn ) = FX1 (x1 ) . . . FXn (xn )
Proposition Random variables X and Y are independent if and only if
1. the supports of X and Y do not depend on each other
2. f (x, y) can be factorized as g(x)h(y)
This proposition applies to both discrete and continuous random variables and can be generalized to
multivariate cases.
Expectation of function of random variables
Definition For random variables X1 , . . . , Xn with joint pmf or pdf f (x1 , . . . , xn ), if u(X1 , . . . , Xn ) is
a function of these random variables, then the expectation of this function is defined as
E(u(X1 , . . . , Xn )) =
X
...
xn
x1
Z
Z
∞
X
u(x1 , . . . , xn )f (x1 , . . . , xn ) discrete
∞
u(x1 , . . . , xn )f (x1 , . . . , xn ) continuous
...
E(u(X1 , . . . , Xn )) =
−∞
−∞
Key Properties
If X and Y are independent, then E(XY ) = E(X)E(Y ) and MX+Y (t) = MX (t)MY (t)
Problems
Problem 1. multinomial distribution In a three-way election, candidate A, B and C has probability of p1 , p2 and p3 to receive one vote(p1 + p2 + p3 = 1). If there are n votes in total, find the
joint and marginal distribution of A, B and C’s votes.
Solution Let X1 , X2 and X3 denote the votes of A, B and C respectively. (X1 , X2 , X3 ) follows a
multinomial distribution. The joint pmf is:
¶
n
pn1 pn2 pn3
n1 , n 2 , n 3 1 2 3
µ
f (n1 , n2 , n3 ) = P (X1 = n1 , X2 = n2 , X3 = n3 ) =
2
Note that n1 +n2 +n3 = n. Therefore, there are only two random variables here since n3 = n−n1 −n2 .
The marginal pmf for X1 can be computed:
fX1 (n1 ) =
n−n
X1
n2 =1
=
n−n
X1
n2 =1
=
n−n
X1
n2 =1
=
f (n1 , n2 , n − n1 − n2 )
µ
¶
n
pn1 pn2 (1 − p1 − p2 )n−n1 −n2
n1 , n2 , n − n1 − n2 1 2
n!
pn1 pn2 (1 − p1 − p2 )n−n1 −n2
n1 !n2 !(n − n1 − n2 )! 1 2
n−n
X1
n!
(n − n1 )!
pn1 1
pn2 2 (1 − p1 − p2 )n−n1 −n2
(n − n1 )!n1 !
n
!(n
−
n
−
n
)!
1
2
n =1 2
2
n!
=
pn1 1 (1 − p1 )n−n1
n1 !(n − n1 )!
∼ Binomial(n, p1 )
Conclusion: Marginal distribution of a multinomial distribution is binomial. And E(Xi ) = npi ,
V ar(Xi ) = npi (1 − pi ).
Problem 2. Let X, Y be two discrete random variables with their joint pmf f (x, y) tabulated below:
x, y
1
2
3
4
1
0.05 0.01 0.10 0.04
2
0.15 0.20 0.05 0.10
3
0.10 0.05 0.10 0.05
1. Tabulate the joint cdf F (x, y) of (X, Y ) for x = 1, 2, 3 and y = 1, 2, 3, 4.
2. Calculate the marginal pmfs of X and Y respectively. Are X and Y independent?
3. Calculate the marginal cdfs of X and Y respectively.
Solution
1. Joint cdf F (x, y)
x, y
1
2
1
0.05
0.06 0.16 0.20
2
0.20
0.41 0.56 0.70
3
0.30
0.56 0.81 1.00
2. Marginal pmf of X and Y :
3
3
4
x, y
1
2
3
4
pX (x)
1
0.05 0.01 0.10 0.04
0.20
2
0.15 0.20 0.05 0.10
0.50
3
0.10 0.05 0.10 0.05
0.30
pY (y)
0.30 0.26 0.25 0.19
1.00
Since P (X = 1, Y = 1) = 0.05 6= pX (X = 1)pY (Y = 1) = 0.20 × 0.30 = 0.06, X and Y are not
independent.
3. Marginal cdf of X:


0





0.2
FX (x) = P (X ≤ x) =


0.7





1
Marginal cdf of Y :



0






0.3



FY (y) = P (Y ≤ y) =
for x < 1
for 1 ≤ x < 2
for 2 ≤ x < 3
for x ≥ 3
for y < 1
for 1 ≤ y < 2
0.56 for 2 ≤ y < 3





0.81 for 3 ≤ y < 4





1
for x ≥ 4
Problem 3. Suppose that the joint pdf of X and Y is specified as follows:

cx2 y for x2 ≤ y ≤ 1,
f (x, y) =
0
otherwise.
Determine the value of the constant c and then the value of P (X ≥ Y ).
Solution The set S of points (x, y) for which f (x, y) > 0 is sketched in Fig.1. Since f (x, y) = 0
outside S, it follows that:
Z
∞
Z
Z
∞
1
Z
1
f (x, y)dxdy =
−∞
Thus, c =
−∞
−1
x2
cx2 ydydx =
4
c=1
21
21
.
4
The subset S0 of S where x ≥ y is sketched in Fig.2. Hence,
Z 1Z x
21 2
3
P (X ≥ Y ) =
x ydydx =
20
0
x2 4
4
Problem 4. Prove the following proposition: The continuous(discrete) random variables X and Y
are independent if and only if their joint pdf(pmf) can be expressed as fX,Y (x, y) = g(x)h(y), where
−∞ < x, y < ∞.
Proof : ⇒ (necessity) Independence implies that the density is the product of the marginal densities
of X and Y . So fX,Y (x, y) can be factorized as fX (x)fY (y) which is one form of g(x)h(y).
⇐ (sufficiency) If fX,Y (x, y) = g(x)h(y), then
Z ∞Z ∞
1=
fX,Y (x, y)dxdy
−∞ −∞
Z ∞
Z ∞
=
g(x)dx
h(y)dy
−∞
−∞
= C1 C2
R∞
where C1 = −∞ g(x)dx and C2 = −∞ h(y)dy. Also,
Z ∞
Z
fX (x) =
fX,Y (x, y)dy =
R∞
Z
−∞
Z
∞
fY (y) =
∞
g(x)h(y)dy = C2 g(x)
−∞
∞
fX,Y (x, y)dx =
−∞
g(x)h(y)dx = C1 h(y)
−∞
Since C1 C2 = 1, we know that fX,Y (x, y) = g(x)h(y) =
fX (x)fY (y)
C1 C2
which implies the independence.
Thus the proof is complete.
Problem 5. Buffon’s needle problem A table is ruled with equidistant parallel lines a distance D
apart. A needle of length L, where L ≤ D, is randomly thrown on the table. What is the probability
that the needle will intersect one of the lines(the other probability being that the needle will be
completely contained in the strip between two lines)?
Solution Let us determine the position of the needle by specifying the distance X from the middle
point of the needle to the nearest parallel line, and the angle θ between the needle and the projected
line of length X. The needle will intersect a line if the hypotenuse of the right triangle is less than
L/2, that is, if
X
cosθ
< L/2 or X < L2 cosθ. As X varies between 0 and D/2 and θ between 0 and π/2,
it is reasonable to assume that they are independent, uniformly distributed random variables over
these respective ranges. Hence,
L
P (X < cos θ) =
2
Z Z
fX (x)fθ (y)dxdy
x<L/2 cos y
Z π/2 Z L/2 cos y
4
=
dxdy
πD 0
0
Z π/2
4
L
=
cos ydy
πD 0
2
2L
=
πD
5
Problem 6. Consider the following two cases.
1. If U and V are jointly continuous, show that P (U = V ) = 0.
2. Let X be uniformly distributed on (0, 1), and let Y = X. Then X and Y are continuous, and
P (X = Y ) = 1. Is there a contradiction here?
Solution
1. We have that
Z Z
P (U = V ) =
ZZ
u
fU,V (u, v)dudv =
fU,V (u, v)dudv = 0.
u
(u,v):u=v
2. There is no contradiction. The reason is that although X and Y are separately continuous,
they are not jointly continuous. By definition this means there exists no integrable function
f : [0, 1]2 → R such that
Z
x
Z
y
f (u, v)dudv,
P (X ≤ x, Y ≤ y) =
0 ≤ x, y ≤ 1.
0
0
The strict proof for jointly non-continuousness may need advanced knowledge in measure theory.
An easier way: Assume the integrable function exists, we try to find some contradiction(in fact,
this question serves as one contradiction).

x for x < y
F (x, y) = P (X ≤ x, Y ≤ y) = P (X ≤ x, X ≤ y) = P (X ≤ min(x, y)) =
y for y ≤ x
We try to find the f (x, y) =
∂ 2 F (x,y)
.
∂x∂y
since
∂F (x,y)
∂x
exists almost everywhere (except at x = y)
and equals to 1 if x < y and 0 if x > y. Thus, we have f (x, y) = 0 almost everywhere. It can’t
be a valid pdf since it is 0 almost everywhere. This is the contradiction.
6