IME 254
Engineering Probability and Statistics I
Chapter 2: Probability
Section 2.6
Jointly Distributed Random Variables
Bivariate Probability Distribution for
Discrete Random Variables
Consider two random variables X and Y that
concurrently map simple events in the sample space
to real numbers
Sample
space
Random variable
X
−∞, ∞
Random variable
Y
−∞, ∞
Bivariate (joint) probability distribution, denoted by
p(x,y), is a table, graph or formula that specifies the
probability of intersection of any two numerical events
X=x and Y=y
Bivariate Probability Distribution for
Discrete Random Variables (cont.)
Recall from Section 2.3:
P(A ∩ B) = P(A)P(B|A) = P(B)P(A|B)
If we assign two numbers to each point in the sample
space– one corresponding to the value of discrete
random variable X, and the second to discrete
random variable Y– then specific values of X and Y
represent two numerical events. The probability of the
intersection of these two events is as follows:
P(X ∩ Y) = P(X = x, Y = y) = p(x,y)
= pX(x)pY(y|x)
= pY(y)pX(x|y)
Bivariate Probability Distribution for
Discrete Random Variables (cont..)
Bivariate probability distributions should satisfy two conditions
෍ 𝑝(𝑥, 𝑦) = 1
∀(𝑥,𝑦)
0 ≤ 𝑝 𝑥, 𝑦 ≤ 1
We can obtain the probability distribution of each individual
random variable from the bivariate distribution using
𝑝𝑋 𝑥 = ෍ 𝑝(𝑥, 𝑦)
𝑦
𝑝𝑌 𝑦 = ෍ 𝑝(𝑥, 𝑦)
𝑥
Which are so-called marginal probability distributions
Example 1: Bivariate Probability
Distribution
Let X represent the number of weekly credit card
purchases a person makes, and Y the number of
credit cards a person owns. Suppose the bivariate
table for the two variables looks as follows:
p(x,y)
Y: # of
cards
1
2
3
0
0.08
0.08
0.04
X: purchases/week
1
2
3
0.1
0.02
0.1
0.05
0.05
0.22
0.04
0.18
0.04
a) Find the marginal probability distribution of X
b) What is the probability that a randomly selected
person owns two credit cards and makes two or
fewer purchases per week?
Example 1: Solution
a) Marginal probability distributions of X and Y are
added as a row and column to the joint probability
distribution table respectively
p(x,y)
Y: # of
cards
P(X=x)
X: purchases/week
P(Y=y)
0
1
2
3
1
0.08
0.1
0.1
0.02
2
0.08
0.05
0.22
0.05
3
0.04
0.04
0.04
0.18
Example 1: Solution (part a)
a) Marginal probability distributions of X and Y are
added as a row and column to the joint probability
distribution table respectively
p(x,y)
Y: # of
cards
P(X=x)
X: purchases/week
P(Y=y)
0
1
2
3
1
0.08
0.1
0.1
0.02
0.3
2
0.08
0.05
0.22
0.05
0.4
3
0.04
0.04
0.04
0.18
0.3
0.2
0.19
0.36
0.25
1
Example 1: Bivariate Probability
Distribution (cont.)
p(x,y)
Y
(# of cards)
P(X=x)
X (purchases)
P(Y=y)
0
1
2
3
1
0.08
0.1
0.1
0.02
0.3
2
0.08
0.05
0.22
0.05
0.4
3
0.04
0.04
0.04
0.18
0.3
0.2
0.19
0.36
0.25
1
b) What is the probability that a randomly selected
person owns two credit cards and makes two or
fewer purchases per week?
Example 1: Solution (part b)
p(x,y)
Y
(# of cards)
P(X=x)
X (purchases)
P(Y=y)
0
1
2
3
1
0.08
0.1
0.1
0.02
0.3
2
0.08
0.05
0.22
0.05
0.4
3
0.04
0.04
0.04
0.18
0.3
0.2
0.19
0.36
0.25
1
b) What is the probability that a randomly selected
person owns two credit cards and makes two or
fewer purchases per week?
Example 1: Solution (part b cont.)
p(x,y)
Y
(# of cards)
P(X=x)
X (purchases)
P(Y=y)
0
1
2
3
1
0.08
0.1
0.1
0.02
0.3
2
0.08
0.05
0.22
0.05
0.4
3
0.04
0.04
0.04
0.18
0.3
0.2
0.19
0.36
0.25
1
b) What is the probability that a randomly selected
person owns two credit cards and makes two or
fewer purchases per week?
This is the probability of the intersection of two
events 𝑋 ≤ 2 and 𝑌 = 2, which can be calculated as
𝑝 𝑋 ≤ 2, 𝑌 = 2 = 𝑝 0,2 + 𝑝 1,2 + 𝑝 2,2 = 0.35
Conditional Probability Distribution
for Discrete Random Variables
The probability of numerical event X=x given that the
event Y=y has occurred is called the conditional
probability of X=x given Y=y.
Conditional probability distribution is a table, graph or
formula that gives conditional probabilities of X=x
given Y=y for all possible values of random variable X
Conditional probability distribution of random variable
X given Y=y is calculated as
𝑝 𝑥, 𝑦
𝑝 𝑥, 𝑦
𝑝 𝑥𝑦 =
=
𝑝𝑌(𝑦)
𝑝(𝑦)
Example 2: Conditional Probability
Distribution
p(x,y)
Y
(# of cards)
P(X=x)
X (purchases)
P(Y=y)
0
1
2
3
1
0.08
0.1
0.1
0.02
0.3
2
0.08
0.05
0.22
0.05
0.4
3
0.04
0.04
0.04
0.18
0.3
0.2
0.19
0.36
0.25
1
a. Using bivariate probability distribution, find the
conditional probability distribution of r.v. Y given that
X=1
b. What is the probability that a randomly selected
person makes no purchases per week given that
he/she owns two credit cards?
Example 3: Bivariate Probability
Distribution
Consider the experiment of tossing a pair of dice. Let X
be the outcome (i.e., the number of dots appearing
face up) on the first die, and let Y be the outcome on
the second die.
a. Find the joint probability distribution p(x,y).
b. Find the marginal probability distributions pX(x) and
pY(y).
c. Find the conditional probability distributions pX(x|y)
and pY(y|x).
Example 3: Solution
Conditional probability distributions pX(x|y) and pY(y|x):
Y: Die 2
1
2
3
4
5
6
P(X = x)
1
2
X: Die 1
3
4
5
6
P(Y = y)
X=1
P(X=x|Y=1)
1
X=2
X=3
X=4
X=5
X=6
Bivariate Probability Distribution for
Continuous Random Variables
The conditional density functions for
continuous random variables X and Y are
𝑓 𝑥,𝑦
f1(x|y) = 𝑓 (𝑦)
2
𝑓 𝑥,𝑦
f2(y|x) = 𝑓 (𝑥)
1
Expected Value of Functions of
Two Random Variables
Let h(X,Y) be a function of discrete random
variables X and Y. Then the expected value
(mean) of h(X,Y) is defined to be
𝜇ℎ(𝑋,𝑌) = ෍ ෍ ℎ 𝑥, 𝑦 𝑝(𝑥, 𝑦)
𝑦
Also,
𝑥
𝐸 𝑋𝑌 = σ𝑥 σ𝑦 𝑥𝑦𝑝(𝑥, 𝑦)
Expected Value of Functions of
Two Random Variables (cont.)
Let c be a constant. Expected value of c is
𝐸(𝑐) = 𝑐
Let c be a constant, and let g(X,Y) be a
function of the random variables X and Y.
The expected value of cg(X,Y) is
𝐸[𝑐𝑔 𝑋, 𝑌 ] = 𝑐𝐸[𝑔 𝑋, 𝑌 ]
Expected Value of Functions of
Two Random Variables (cont..)
Let gX(X,Y), gY(X,Y), … , gk(X,Y) be k functions of
the random variables X and Y. The expected
value of the sum of these functions is
𝐸 𝑔𝑋 𝑋, 𝑌 + 𝑔𝑌 𝑋, 𝑌 + ⋯ + 𝑔𝑘 𝑋, 𝑌 +
= 𝐸 𝑔𝑋 𝑋, 𝑌 + 𝐸 𝑔𝑌 𝑋, 𝑌 + ⋯ + 𝐸[𝑔𝑘 𝑋, 𝑌 ]
Independence
Let X and Y be discrete random variables with
joint probability distribution p(x,y) and marginal
probability distributions pX(x) and pY(y). Then X
and Y are said to be independent if and only if
𝑃 𝑥, 𝑦 = 𝑝𝑋 (𝑥)𝑝𝑌 (𝑦)
for all pairs of values of x and y
If X and Y are independent random variables,
then
𝐸 𝑋𝑌 = 𝐸 𝑋 𝐸(𝑌)
Correlation Between
Random Variables
Suppose X and Y are random variables, and we
collect a sample of n pairs (x,y) from them and plot
those n data points
A positive correlation is when Y increases as X increase
A negative correlation is when Y decreases as X
increases
Linear Correlation
When investigating whether X and Y are positively or
negatively correlated
• we tend to think of it in terms of linear relationships
• we do not consider any nonlinear relationship
In the two cases below there is not any linear
correlation between X and Y; however, unlike case
(c), case (d) shows a strong nonlinear relationship
Correlation
Against All Odds: “Correlation”
Directed by Graham Chedd, fl. 1973; presented by Pardis Sabeti,
1975-; produced by Maggie Villiger, fl. 1999 and Graham Chedd,
fl. 1973, Chedd-Angier Production Company, Inc., in Against all
odds: Inside Statistics, Unit 12 (District of Columbia: Annenberg
Learner, 2014), 11 mins
Measuring Linear Correlation
Using Sample Covariance
Sample covariance is the cross product of deviations
from the mean for each data point (x,y) in the sample
σ𝑛𝑖=1 𝑥𝑖 − 𝑋ത 𝑦𝑖 − 𝑌ത
𝐶𝑜𝑣 𝑋, 𝑌 =
𝑛−1
Measuring Linear Correlation
Using Sample Covariance (cont.)
Sample covariance is the cross product of deviations
from the mean for each data point (x,y) in the sample
σ𝑛𝑖=1 𝑥𝑖 − 𝑋ത 𝑦𝑖 − 𝑌ത
𝐶𝑜𝑣 𝑋, 𝑌 =
𝑛−1
A positive (negative) sample covariance implies a
positive (negative) correlation
Population Covariance
For two r.v. X and Y the covariance is defined as
𝐶𝑜𝑣 𝑋, 𝑌 = 𝐸 𝑋 − 𝜇𝑋 𝑌 − 𝜇𝑌 = 𝐸 𝑋𝑌 − 𝜇𝑋 𝜇𝑌
If
o 𝑪𝒐𝒗 𝑿, 𝒀 = 𝟎, then X and Y are linearly uncorrelated
o 𝑪𝒐𝒗 𝑿, 𝒀 > 𝟎, then X and Y are positively correlated
o 𝑪𝒐𝒗 𝑿, 𝒀 < 𝟎, then X and Y are negatively
correlated
If X and Y are independent random variables then
𝑪𝒐𝒗 𝑿, 𝒀 =0
Example 4: Covariance of X and Y
a) Find the expected value for X, and interpret the result
b) Find the expected value for Y, and interpret the result
c) Calculate E[XY]
d) Calculate Covariance
p(x,y)
Y
(# of cards)
P(X=x)
X (purchases)
P(Y=y)
0
1
2
3
1
0.08
0.1
0.1
0.02
0.3
2
0.08
0.05
0.22
0.05
0.4
3
0.04
0.04
0.04
0.18
0.3
0.2
0.19
0.36
0.25
1
Example 4: Covariance of X and Y - a
a) Find the expected value for X, and interpret the result
p(x,y)
Y
(# of cards)
P(X=x)
X (purchases)
P(Y=y)
0
1
2
3
1
0.08
0.1
0.1
0.02
0.3
2
0.08
0.05
0.22
0.05
0.4
3
0.04
0.04
0.04
0.18
0.3
0.2
0.19
0.36
0.25
1
𝐸 𝑋 = 𝜇𝑥 = ෍ 𝑥𝑝(𝑥)
𝑎𝑙𝑙 𝑥
Example 4: Covariance of X and Y - b
b) Find the expected value for Y, and interpret the result
p(x,y)
Y
(# of cards)
P(X=x)
X (purchases)
P(Y=y)
0
1
2
3
1
0.08
0.1
0.1
0.02
0.3
2
0.08
0.05
0.22
0.05
0.4
3
0.04
0.04
0.04
0.18
0.3
0.2
0.19
0.36
0.25
1
𝐸 𝑌 = 𝜇𝑦 = ෍ 𝑦𝑝(𝑦)
𝑎𝑙𝑙 𝑦
Example 4: Covariance of X and Y - c
c) Calculate E[XY]
p(x,y)
Y
(# of cards)
P(X=x)
X (purchases)
P(Y=y)
0
1
2
3
1
0.08
0.1
0.1
0.02
0.3
2
0.08
0.05
0.22
0.05
0.4
3
0.04
0.04
0.04
0.18
0.3
0.2
0.19
0.36
0.25
1
𝐸 𝑋𝑌 = ෍ ෍ 𝑥𝑦𝑝(𝑥, 𝑦)
𝑎𝑙𝑙 𝑥 𝑎𝑙𝑙 𝑦
Example 4: Covariance of X and Y - d
d) Calculate Covariance
p(x,y)
Y
(# of cards)
P(X=x)
X (purchases)
P(Y=y)
0
1
2
3
1
0.08
0.1
0.1
0.02
0.3
2
0.08
0.05
0.22
0.05
0.4
3
0.04
0.04
0.04
0.18
0.3
0.2
0.19
0.36
0.25
1
𝐶𝑜𝑣 𝑋, 𝑌 = 𝐸 𝑋𝑌 − 𝜇𝑋 𝜇𝑌
Coefficient of Correlation
• To adjust covariance for units of measurement of X
and Y we define coefficient of correlation between
X and Y as
𝐶𝑜𝑣 𝑋, 𝑌
𝜌(𝑋, 𝑌) =
𝜎𝑋 𝜎𝑌
where σX and σY are standard deviations
of r.v. X and Y
Coefficient of Correlation (cont.)
• To adjust covariance for units of measurement of X
and Y we define coefficient of correlation between
X and Y as
𝐶𝑜𝑣 𝑋, 𝑌
𝜌(𝑋, 𝑌) =
𝜎𝑋 𝜎𝑌
where σX and σY are standard deviations
of r.v. X and Y
• It can be shown that −1 < 𝜌 < 1
• 𝜌 is used to quantify the strength of linear correlation
between X and Y
Example: Linear Correlation
The local ice cream shop keeps track of how much
ice cream they sell versus the temperature on that
day. The table below shows the data for the last 12
days:
Temperature
°C
Ice Cream
Sales
14.2°
$215
16.4°
$325
11.9°
$185
15.2°
$332
18.5°
$406
22.1°
$522
19.4°
$412
25.1°
$614
23.4°
$544
18.1°
$421
22.6°
$445
17.2°
$408
• Sample Covariance: 484.1
• Correlation Coefficient: 0.96
(a strong positive correlation)
Example 5: Coefficient of Correlation
p(x,y)
X (purchases)
Y
(# of cards)
P(Y=y)
0
1
2
3
1
0.08
0.1
0.1
0.02
0.3
2
0.08
0.05
0.22
0.05
0.4
3
0.04
0.04
0.04
0.18
0.3
0.2
0.19
0.36
0.25
1
P(X=x)
Calculate the coefficient of correlation:
σ𝑛𝑖=1 𝑥𝑖 − 𝑋ത 𝑦𝑖 − 𝑌ത
𝐶𝑜𝑣 𝑋, 𝑌 =
𝑛−1
𝑪𝒐𝒗 𝑿, 𝒀
𝝆(𝑿, 𝒀) =
𝝈𝑿 𝝈𝒀
2
2
𝜎𝑥 = ෍ 𝑥 − 𝜇𝑥 𝑝(𝑥)
𝑎𝑙𝑙 𝑥
𝜇𝑥 = 1.66
𝜇𝑦 = 2
2
𝜎𝑦 2 = ෍ 𝑦 − 𝜇𝑦 𝑝(𝑦)
𝑎𝑙𝑙 𝑦
Example 5: Coefficient of Correlation Calculate 𝜎𝑥
p(x,y)
X (purchases)
Y
(# of cards)
P(X=x)
P(Y=y)
0
1
2
3
1
0.08
0.1
0.1
0.02
0.3
2
0.08
0.05
0.22
0.05
0.4
3
0.04
0.04
0.04
0.18
0.3
0.2
0.19
0.36
0.25
1
𝜎𝑥 2 = ෍ 𝑥 − 𝜇𝑥 2 𝑝(𝑥)
𝑎𝑙𝑙 𝑥
(𝜇𝑥 = 1.66)
Example 5: Coefficient of Correlation Calculate 𝜎𝑦
p(x,y)
Y
(# of cards)
P(X=x)
X (purchases)
P(Y=y)
0
1
2
3
1
0.08
0.1
0.1
0.02
0.3
2
0.08
0.05
0.22
0.05
0.4
3
0.04
0.04
0.04
0.18
0.3
0.2
0.19
0.36
0.25
1
2
𝜎𝑦 2 = ෍ 𝑦 − 𝜇𝑦 𝑝(𝑦)
𝑎𝑙𝑙 𝑦
(𝜇𝑦 = 2)
Example 5: Coefficient of Correlation –
Calculate 𝜌(𝑋, 𝑌)
p(x,y)
Y
(# of cards)
P(X=x)
X (purchases)
P(Y=y)
0
1
2
3
1
0.08
0.1
0.1
0.02
0.3
2
0.08
0.05
0.22
0.05
0.4
3
0.04
0.04
0.04
0.18
0.3
0.2
0.19
0.36
0.25
1
𝐶𝑜𝑣 𝑋, 𝑌
𝜌 𝑋, 𝑌 =
𝜎𝑋 𝜎𝑌