Joint Distribution of two or More Random Variables
• Sometimes more than one measurement (r.v.) is taken on each member of
the sample space. In cases like this there will be a few random variables
defined on the same probability space and we would like to explore their
joint distribution.
• Joint behavior of 2 random variable (continuous or discrete), X and Y
determined by their joint cumulative distribution function
FX ,Y x, y   P X  x, Y  y .
• n – dimensional case
FX1 ,..., X n x1 ,..., x n   P X 1  x1 , , X n  x n .
week 7
1
Discrete case
• Suppose X, Y are discrete random variables defined on the same probability
space.
• The joint probability mass function of 2 discrete random variables X and Y
is the function pX,Y(x,y) defined for all pairs of real numbers x and y by
p X ,Y x, y   P X  x and Y  y 
• For a joint pmf pX,Y(x,y) we must have: pX,Y(x,y) ≥ 0 for all values of x,y and
 p x, y   1
X ,Y
x
y
week 7
2
Example for illustration
• Toss a coin 3 times. Define,
X: number of heads on 1st toss, Y: total number of heads.
• The sample space is Ω ={TTT, TTH, THT, HTT, THH, HTH, HHT, HHH}.
• We display the joint distribution of X and Y in the following table
• Can we recover the probability mass function for X and Y from the joint table?
• To find the probability mass function of X we sum the appropriate rows of the
table of the joint probability function.
• Similarly, to find the mass function for Y we sum the appropriate columns.
week 7
3
Marginal Probability Function
• The marginal probability mass function for X is
p X x    p X ,Y x, y 
y
• The marginal probability mass function for Y is
pY  y    p X ,Y x, y 
x
• Case of several discrete random variables is analogous.
• If X1,…,Xm are discrete random variables on the same sample space with
joint probability function
p X1 ,... X n x1 ,...xm   P X 1  x1 ,..., X m  xm 
The marginal probability function for X1 is
p X1 x1  
p
X1 ,... X n
x1,...xm 
x2 ,..., xm
• The 2-dimentional marginal probability function for X1 and X2 is
p X X x1 , x2    p X ,... X x1 , x2 , x3 ,..., xm 
1
2
1
x3 ,..., xm
n
week 7
4
Example
• Roll a die twice. Let X: number of 1’s and Y: total of the 2 die.
There is no available form of the joint mass function for X, Y. We display
the joint distribution of X and Y with the following table.
• The marginal probability mass function of X and Y are
• Find P(X ≤ 1 and Y ≤ 4)
week 7
5
Independence of random variables
• Recall the definition of random variable X: mapping from Ω to R such that
   : X    x F for x  R .
• By definition of F, this implies that (X > 1.4) is and event and for the
discrete case (X = 2) is an event.
• In general  X  A   : X    A is an event for any set A that is formed
by taking unions / complements / intersections of intervals from R.
• Definition
Random variables X and Y are independent if the events  X  A and Y  B 
are independent.
week 7
6
Theorem
• Two discrete random variables X and Y with joint pmf pX,Y(x,y) and
marginal mass function pX(x) and pY(y), are independent if and only if
p X ,Y  x, y   p X  x  pY  y 
• Proof:
• Question: Back to the rolling die 2 times example, are X and Y
independent?
week 7
7
Conditional Probability on a joint discrete distribution
• Given the joint pmf of X and Y, we want to find
P X  x and Y  y 
P X  x | Y  y  
PY  y 
and
PY  y | X  x  
P X  x and Y  y 
P X  x 
• Back to the example on slide 5,
PY  2 | X  1  0
2
1
P Y  3 | X  1  36 
10
5
36

PY  12 | X  1  0
• These 11 probabilities give the conditional pmf of Y given X = 1.
week 7
8
Definition
• For X, Y discrete random variables with joint pmf pX,Y(x,y) and marginal mass
function pX(x) and pY(y). If x is a number such that pX(x) > 0, then the
conditional pmf of Y given X = x is
pY | X  y | x   pY | X  y | X  x  
• Is this a valid pmf?
p X ,Y x, y 
p X x 
• Similarly, the conditional pmf of X given Y = y is
p X |Y x | y   p X |Y x | Y  y  
p X ,Y x, y 
pY  y 
• Note, from the above conditional pmf we get
p X ,Y x, y   p X |Y x | y  pY  y 
Summing both sides over all possible values of Y we get
p X x    p X ,Y x, y    p X |Y x | y  pY  y 
y
y
This is an extremely useful application of the law of total probability.
• Note: If X, Y are independent random variables then PX|Y(x|y) = PX(x).
week 7
9
Example
• Suppose we roll a fair die; whatever number comes up we toss a coin that
many times. What is the distribution of the number of heads?
Let X = number of heads, Y = number on die.
We know that
1
pY  y  
, y  1, 2, 3, 4, 5, 6
6
Want to find pX(x).
• The conditional probability function of X given Y = y is given by
 y  1 
p X |Y x | y     
 x  2 
2
for
• By the Law of Total Probability we have
x = 0, 1, …, y.
 y  1  1
p X x    p X |Y x | y  pY  y       
6
y
y  x  x  2 
6
y
Possible values of x: 0,1,2,…,6.
week 7
10
Example
• Interested in the 1st and 2nd success in repeated Bernoulli trails with probability of
success p on any trail. Let X = number of failures before 1st success and
Y = number of failures before 2nd success (including those before the 1st success).
• The marginal pmf of X and Y are
for
x = 0, 1, 2, ….
p X x   P X  x   pq x
• and
pY  y   PY  y    y  1 p 2 q y for
y = 0, 1, 2, ….
• The joint mass function, pX,Y(x,y) , where x, y are non-negative integers and x ≤ y is
the probability that the 1st success occurred after x failure and the 2nd success after
y – x more failures (after 1st success), e.g.
p X ,Y 2,5  PFFSFFFS  p 2 q 5
• The joint pmf is then given by
 p 2q y
p X ,Y x, y   
 0
0 x y
otherwise
• Find the marginal probability functions of X, Y. Are X, Y independent?
• What is the conditional probability function of X given Y = 5
week 7
11
Some Notes on Multiple Integration
• Recall: simple integral  f x dx cab be regarded as the limit as N  ∞ of
a
the Riemann sum of the form N
b
 f x x
i
i 1
i
where ∆xi is the length of the ith subinterval of some partition of [a, b].
• By analogy, a double integral
 f x, y dA
D
where D is a finite region in the xy-plane, can be regarded as the limit as
N  ∞ of the Riemann sum of the form
N
N
 f x , y A  f x , y x y
i 1
i
i
i
i 1
i
i
i
i
where Ai is the area of the ith rectangle in a decomposition of D into
rectangles.
week 7
12
Evaluation of Double Integrals
• Idea – evaluate as an iterated integral. Restrict D now to a region such that
any vertical line cuts its boundary in 2 points. So we can describe D by
yL(x) ≤ y ≤ yU(x) and xL ≤ x ≤ xU
• Theorem
Let D be the subset of R2 defined by yL(x) ≤ y ≤ yU(x) and xL ≤ x ≤ xU
If f(x,y) is continuous on D then

D
xU
yU
f x, y dA     f x, y dy  dx

xL 
 yL
• Comment: When evaluating a double integral, the shape of the region or
the form of f (x,y) may dictate that you interchange the role of x and y and
integrate first w.r.t x and then y.
week 7
13
Examples
1) Evaluate
 x  ydA where D is the triangle with vertices (0,0), (2,0) and (0,1).
D
2) Evaluate
2
2
xy
x

y
dA where D is the region in the 1st quadrate bounded

D
by y = 0, y = x and x2 + y2 = 1.
3) Integrate f x, y   xe x2 y over the set of points in the positive quadrate for
which y > 2x.
week 7
14
The Joint Distribution of two Continuous R.V’s
• Definition
Random variables X and Y are (jointly) continuous if there is a non-negative
function fX,Y(x,y) such that
P X , Y   A   f X ,Y x, y dxdy
A
for any “reasonable” 2-dimensional set A.
• fX,Y(x,y) is called a joint density function for (X, Y).
• In particular , if A = {(X, Y): X ≤ x, Y ≤ x}, the joint CDF of X,Y is
FX ,Y x, y   
x

y
 
f X ,Y u, v du, dv
• From Fundamental Theorem of Calculus we have
2
2
f X .Y x, y  
FX ,Y x, y  
FX ,Y x, y 
xy
yx
week 7
15
Properties of joint density function
•
f X ,Y  x, y   0 for all x, y  R
• It’s integral over R2 is

 

 
f X ,Y x, y dxdy  1
week 7
16
Example
• Consider the following bivariate density function

12 2
x  xy

f X ,Y  x, y    7

0

0  x 1, 0  y 1
otherwise
• Check if it’s a valid density function.
• Compute P(X > Y).
week 7
17
Properties of Joint Distribution Function
For random variables X, Y , FX,Y : R2  [0,1] given by FX,Y (x,y) = P(X ≤ x,Y ≤ x)

lim FX ,Y x, y   0
x  
y  
 lim FX ,Y x, y   1
x 
y 
• FX,Y (x,y) is non-decreasing in each variable i.e.
FX ,Y  x1 , y1   FX ,Y  x2 , y 2 
if x1 ≤ x2 and y1 ≤ y2 .
• lim FX ,Y x, y   FY  y 
x 
and
lim FX ,Y x, y   FX x 
y 
week 7
18
Marginal Densities and Distribution Functions
• The marginal (cumulative) distribution function of X is
FX x   P X  x   
x


 
f X ,Y u, y dydu
• The marginal density of X is then

f X x   F x    f X ,Y x, y dy
'
X

• Similarly the marginal density of Y is

fY  y    f X ,Y x, y dx

week 7
19
Example
• Consider the following bivariate density function
6 xy2
f X ,Y x, y   
0

0  x 1, 0  y 1
otherwise
• Check if it is a valid density function.
• Find the joint CDF of (X, Y) and compute P(X ≤ ½ ,Y ≤ ½ ).
• Compute P(X ≤ 2 ,Y ≤ ½ ).
• Find the marginal densities of X and Y.
week 7
20
Generalization to higher dimensions
Suppose X, Y, Z are jointly continuous random variables with density f(x,y,z), then
• Marginal density of X is given by:
f X x   


 f x, y, z dydz
 
• Marginal density of X, Y is given by :

f X ,Y x, y    f x, y, z dz

week 7
21
Example
Given the following joint CDF of X, Y
FX ,Y x, y   x 2 y  y 2 x  x 2 y 2
0  x 1, 0  y 1
• Find the joint density of X, Y.
• Find the marginal densities of X and Y and identify them.
week 7
22
Example
Consider the joint density
2 e  y
f X ,Y x, y   
 0
0 x y
otherwise
where λ is a positive parameter.
• Check if it is a valid density.
• Find the marginal densities of X and Y and identify them.
week 7
23