Chapter 5
Joint Probability Distributions
Joint, n. 1. a cheap, sordid place. 2. the
movable place where two bones join. 3. one
of the portions in which a carcass is divided
by a butcher. 4. adj., shared or common to
two or more
Chapter 5A
Discrete RV
This week in Prob/Stat
today’s
good
stuff
time
permitting
Joint Probability Distributions

It is often useful (necessary) to have more than one
RV defined in a random experiment. Examples:

Polyethylene Specs



Dimensions of a part



X = Melt Point
Y = Density
X = length
Y = width
If X & Y are two RV’s, the probability distribution that
defines their simultaneous behavior is a Joint
Probability Distribution
Two Discrete Random Variables


Let X = a discrete random variable, the number of
orders placed per day for a high cost item
Let Y = a discrete random variable, the number of
items in stock
X/Y
0
1
2
sum
0
0.2
0.1
0.08
0.38
1
0.15
0.17
0.09
0.41
2
0.03
0.05
0.08
0.16
3
0.01
0.01
0.03
0.05
sum
0.39
0.33
0.28
1
Joint Probability Mass Function, fxy(x,y)
5-1 Two Discrete Random Variables
5-1.1 Joint Probability Distributions
Two Discrete Random Variables


Let X = a discrete random variable, the number of orders placed
per day for a high cost item
Let Y = a discrete random variable, the number of items in stock
X/Y
0
0.2
0.1
0.08
0.38
0
1
2
sum
1
0.15
0.17
0.09
0.41
2
0.03
0.05
0.08
0.16
3
0.01
0.01
0.03
0.05
Pr{X=0, Y=1} = fxy(0,1) = .15
1 f XY ( x, y )  0
 2    f XY  x, y   1
x
 3
y
f XY  x, y   P ( X  x, Y  y )
sum
0.39
0.33
0.28
1
5-1 Two Discrete Random Variables
5-1.2 Marginal Probability Distributions
• The individual probability distribution of a random variable is
referred to as its marginal probability distribution.
•The marginal probability distribution of X can be determined
from the joint probability distribution of X and other random
variables.
•To determine P(X = x), we sum P(X = x, Y = y) over all points
in the range of (X, Y ) for which X = x.
•Subscripts on the probability mass functions distinguish
between the random variables.
5-1 Two Discrete Random Variables
Definition: Marginal Probability Mass Functions
Two Discrete Random Variables


Let X = a discrete random variable, the number of orders placed
per day for a high cost item
Let Y = a discrete random variable, the number of items in stock
X/Y
0
1
2
fy(y)
0
0.2
0.1
0.08
0.38
1
0.15
0.17
0.09
0.41
2
0.03
0.05
0.08
0.16
3
0.01
0.01
0.03
0.05
fx(x)
0.39
0.33
0.28
1
Pr{X = 1} = Pr{X=1, Y=0} + Pr{X=1, Y=1}
+ Pr{X=1, Y=2} + Pr{X=1, Y=3} = fx(1) =
.1 + .17 + .05 + .01 =.33
Pr{Y  2} = fy(2) + fy(3) = .16 + .05 = .21
Marginal Mean & Variance

If the marginal probability distribution of X has the probability
mass function fX(x), then


E ( X )   X   xf X ( x)   x   f XY ( x, y )    xf XY ( x, y )
x
x
 Rx
 x Rx
=  xf XY ( x, y )
R
V ( X )   X2    x   X  f X ( x)    x   x 
2
x
2
x
f
XY
( x, y )
Rx
=   x   x  f XY  x, y     x   x  f XY  x, y 
2
x


Rx
2
R
Rx denotes all points of (X,Y) for which X = x and
R denotes all points in the range of (X,Y)
Using the Marginal Distributions
X/Y
0
1
2
fy(y)
0
0.2
0.1
0.08
0.38
1
0.15
0.17
0.09
0.41
2
0.03
0.05
0.08
0.16
3
0.01
0.01
0.03
0.05
fx(x)
0.39
0.33
0.28
1
E[X] = x = 0 (.39) + 1 (.33) + 2 (.28) = .89
E[Y] = y = 0 (.38) + 1 (.41) + 2 (.16) + 3 (.05) = .88
Var[X] = x2 = 02 (.39) + 12 (.33) + 22 (.28) - .892 = .6579
Var[Y] = y2 = 02 (.38) + 12 (.41) + 22 (.16) + 32 (.05) - .882 = .7256
5-1.3 Conditional Probability Distributions
Conditional Distribution of Y
X/Y
0
1
2
fy(y)
0
0.2
0.1
0.08
0.38
1
0.15
0.17
0.09
0.41
2
0.03
0.05
0.08
0.16
3
0.01
0.01
0.03
0.05
fx(x)
0.39
0.33
0.28
1
fxy(x,y)
fY|x(y)
x=0
x=1
x=2
0
1
2
3
0.512821 0.384615 0.076923 0.025641
0.30303 0.515152 0.151515 0.030303
0.285714 0.321429 0.285714 0.107143
fY|x(y) = fxy(x,y) / fx(x)
fY|x = 1(2) = fxy(1,2) / fx(1) = .05 / .33 = .151515
Conditional Distribution of X
X/Y
0
1
2
fy(y)
0
0.2
0.1
0.08
0.38
1
0.15
0.17
0.09
0.41
2
0.03
0.05
0.08
0.16
3
0.01
0.01
0.03
0.05
fx(x)
0.39
0.33
0.28
1
fxy(x,y)
fX|y(x)
y=0
y=1
0 0.526316 0.365854
1 0.263158 0.414634
2 0.210526 0.219512
y=2
0.1875
0.3125
0.5
y=3
0.2
0.2
0.6
fX|y(x) = fxy(x,y) / fy(y)
fX|y = 2(1) = fxy(1,2) / fy(2) = .05 / .16 = .3125
Conditional Mean and Variance
Conditional Mean & Var of Y
fY|x(y)
x=0
x=1
x=2
0
1
2
3
E[Y|x]
0.512821 0.384615 0.076923 0.025641 0.6154
0.30303 0.515152 0.151515 0.030303 0.9091
0.285714 0.321429 0.285714 0.107143 1.2143
V[Y|x]
0.5444
0.5675
0.9541
fY|x(y) = fxy(x,y) / fx(x)
E[Y|x=1] = 0 (.30303) + 1 (.515152)
+ 2 (.151515) + 3 (.030303) = .9091
Var[Y|x=1] = 0 (.30303) + 1 (.515152)
+ 4(.151515) + 9 (.030303) - .90912 = .5675
Conditional Mean & Var of X
fX|y(x)
E[X|y]
V[X|y]
y=0
y=1
0 0.526316 0.365854
1 0.263158 0.414634
2 0.210526 0.219512
0.6842
0.8537
0.6371
0.5640
y=2
0.1875
0.3125
0.5
1.3125
0.5898
y=3
0.2
0.2
0.6
1.4000
0.6400
fX|y(x)
E[X|y =2] = 0 (.1875) + 1 (.3125) + 2 (.5) = 1.3125
Var[X|y =2] = 0 (.1875) + 1 (.3125) + 4 (.5) - 1.31252 = .5896
5-1.4 Independence
Are X and Y Independent?
X/Y
0
1
2
fy(y)
0
0.2
0.1
0.08
0.38
1
0.15
0.17
0.09
0.41
2
0.03
0.05
0.08
0.16
3
0.01
0.01
0.03
0.05
fx(x)
0.39
0.33
0.28
1
fx(1) fy(2) = (.33) (.16) = .0528  fxy(1,2) = .05
fx(2) fy(0) = (.28) (.38) = .1064  fxy(2,0) = .08
No Chuck, they are
not independent.
More on Independence


Many evaluations of independence are based
on knowledge of the physical situation.
If we are reasoning based on data, we will
need statistical tools to help us.

It is very, very unlikely that counts and estimated
probabilities will yield exact equalities as in the
conditions for establishing independence.
The Search for Independence

Let X = a discrete random variable, the number of
defects in a lot of size 3 where the probability of a
defect is a constant .1.
X

B(3,.1)
Let Y = a discrete random, the demands in a given
day for the number of units from the above lot.
fy(y)
0
0.3
1
0.2
2
0.4
3
0.1
The Search Continues
assuming independence: fxy(x,y) = fx(x) fy(y)
X/Y
0
1
2
3
fy(y)
0
0.2187
0.0729
0.0081
0.0003
0.3
1
0.1458
0.0486
0.0054
0.0002
0.2
2
0.2916
0.0972
0.0108
0.0004
0.4
3
0.0729
0.0243
0.0027
0.0001
0.1
fx(x)
0.729
0.243
0.027
0.001
X
B(3,.1)
fxy(1,2) = fx(1) fy(2) = (.243) (.4) = .0972
Remember: P(A  B) = P(A) P(B) if A and B are independent
Recap - Sample Problem

X
-1
-1
-0.5
-0.5
0.5
0.5
1
1
Assume X & Y are jointly distributed with the
following joint probability mass function:
Y
-2.0
-1.0
-1.0
0.0
0.0
1.0
1.0
2.0
PROBABILITY
1/8
1/8
1/16
1/16
3/16
1/4
1/16
1/8
Y
1/8
1/4
1/16
1/8
1/8
1/16
1/16
3/16
X
Sample Problem Cont’d
f X ( x)  P( X  x )

Determine the marginal probability distribution of X
 P(X = -1) = 1/8 + 1/8 = 1/4
4
 P(X = -0.5) = 1/16 + 1/16 = 1/8
Pxi
 P(X = 0.5) = 3/16 + 1/4 = 7/16
i 1
 P(X = 1) = 1/16 + 1/8 = 3/16

X
-1
-1
-0.5
-0.5
0.5
0.5
1
1
Y
-2.0
-1.0
-1.0
0.0
0.0
1.0
1.0
2.0
PROBABILITY
1/8
1/8
1/16
1/16
3/16
1/4
1/16
1/8
 1.0
Sample Problem Cont’d

Determine the conditional probability distribution of Y given
that X = 1.
fY x ( y)  f XY ( x, y) / f X ( x)
P(Y = 1 | X = 1) = P(X = 1, Y = 1)/P(X = 1)
= (1/16)/(3/16) = 1/3
X
-1
-1
-0.5
-0.5
0.5
0.5
1
1
Y
-2.0
-1.0
-1.0
0.0
0.0
1.0
1.0
2.0
PROBABILITY
1/8
1/8
1/16
1/16
3/16
1/4
1/16
1/8
P(Y = 2 | X = 1) = P(X = 1, Y = 2)/P(X = 1)
= (1/8)/(3/16) = 2/3
Sample Problem Cont’d

Determine the conditional probability distribution of
X given that Y = 1.
f X y ( x)  f XY ( x, y) / fY ( y)
P(X = 0.5 | Y = 1) = P(X = 0.5, Y = 1)/P(Y = 1)
= (1/4)/(5/16) = 4/5
X
-1
-1
-0.5
-0.5
0.5
0.5
1
1
Y
-2.0
-1.0
-1.0
0.0
0.0
1.0
1.0
2.0
PROBABILITY
1/8
1/8
1/16
1/16
3/16
1/4
1/16
1/8
P(X = 1 | Y = 1) = P(X = 1, Y = 1)/P(Y = 1)
= (1/16)/(5/16) = 1/5
5-1.5 Multiple Discrete Random Variables
Definition: Joint Probability Mass Function
5-1.5 Multiple Discrete Random Variables
Definition: Marginal Probability Mass Function
5-1.5 Multiple Discrete Random Variables
Mean and Variance from Joint Probability
5-1.6 Multinomial Probability Distribution
5-1.6 Multinomial Probability Distribution
The Necessary Example




Final inspection of products coming off of the assembly
line categorizes every item as either acceptable, needing
rework, or rejected.
Historically, 90 percent have been acceptable, 7 percent
needed rework, and 3 percent have been rejected.
For the next 10 items that are produced, what is the
probability that there will be 8 acceptable, 2 reworks, and
no rejects?
Let X1 = number acceptable, X2 = number reworks, X3 =
number rejects
10!
8
2
0
Pr{ X 1  8, X 2  2, X 3  0} 
.9  .07  .03  .194
8!2!0!
More of the Necessary Example



The production process is assumed to be out of control
(i.e. the probability of an acceptable item is less than .9) if
there are fewer than 8 acceptable items produced from a
lot size of 10?
What is the probability that the production process will be
assumed to be out of control when the probability of an
acceptable item remains .9?
Let X1 = number acceptable X1 B(10,.9); E[ X1 ]  9
10 
x
10  x
Pr{ X 1  8}     .9  .1
 .07
x 0  x 
7
This week in Prob/Stat
Wednesday’s
good
stuff