MAT2371 Introduction to Probability
§4 Bivariate Distributions
Zao-Li CHEN
University of Ottawa
Department of Mathematics and Statistics
Z.-L. Chen
MAT2371B Fall 2023
1 / 26
Motivation (1)
•
Two discrete random variables X and Y .
•
Couple them into a vector (X, Y ).
For example, at a Tim Hortons:
X = the number of small coffee sold today,
Y = the number of chocolate croissant sold today.
Z.-L. Chen
MAT2371B Fall 2023
2 / 26
Motivation (2)
•
Two continuous random variables X and Y .
•
Couple them into a vector (X, Y ).
For example, consider the weather data:
X = the total rainfall in Ottawa in 2050,
Y = the total rainfall in Gatineau in 2050.
Z.-L. Chen
MAT2371B Fall 2023
3 / 26
Joint PMF (1)
Definition 1
Let X and Y be two discrete r.v.’s defined on a same sample space
S. Let Se ⊂ R2 be the state space of (X, Y ).
n
o
Se =
X(s), Y (s) , s ∈ S .
(1)
The joint pmf of X and Y is the function
f (x, y) = P{X = x, Y = y}.
Z.-L. Chen
MAT2371B Fall 2023
(2)
4 / 26
Joint PMF(2)
The joint pmf of X and Y in Definition 1 satisfies:
0 ⩽ f (x, y) ⩽ 1.
P
•
e f (x, y) = 1.
(x,y)∈S
•
•
e
For any A ⊂ S,
P{(X, Y ) ∈ A} =
X
f (x, y).
(x,y)∈A
Z.-L. Chen
MAT2371B Fall 2023
5 / 26
Example
Example 2
•
Let X ∼ Bernoulli(1/2) and Y ∼ Bernoulli(1/3).
X and Y are independent.
e and the joint pmf f (x, y).
• Find S
•
Z.-L. Chen
MAT2371B Fall 2023
6 / 26
Marginal PMF
Let (X, Y ) have the joint pmf f (x, y) with state space Se
eX = {x : (x, y) ∈ Se for some y}.
• The state space of X, S
eY = {y : (x, y) ∈ Se for some x}.
• The state space of Y , S
•
Definition 3
The marginal pmf of X and Y are, respectively,
X
fX (x) =
f (x, y) = P{X = x}, x ∈ SeX ,
(3)
eY
y∈S
fY (y) =
X
f (x, y) = P{Y = y},
y ∈ SeY .
(4)
eX
x∈S
Z.-L. Chen
MAT2371B Fall 2023
7 / 26
Textbook example 4.1-4
Example 4
•
Let the joint pmf of X and Y be
f (x, y) =
•
xy 2
,
13
(x, y) = (1, 1), (1, 2), (2, 2).
Find SeX , fX (x), SeY , fY (y).
(
Z.-L. Chen
5/13,
8/13,
x=1
;
x=2
SeX = {1, 2},
fX (x) =
SeY = {1, 2},
(
1/13,
y=1
fX (x) =
.
12/13, y = 2
MAT2371B Fall 2023
8 / 26
Independence and Dependence
Definition 5
•
Let X and Y have joint pmf f (x, y).
•
X and Y are independent if
f (x, y) = fX (x)fY (y),
•
for all x ∈ SeX , y ∈ SeY .
(5)
Otherwise, X and Y are dependent.
Observation If X and Y are independent, then
n
o
Se = SeX × SeY = (x, y) : x ∈ SeX , y ∈ SeY ,
which is a “rectangle”.
Z.-L. Chen
MAT2371B Fall 2023
9 / 26
Textbook example 4.1-2
Example 6
•
Let the joint pmf of X and Y be
f (x, y) = (x + y)/21,
•
Find fX (3) and fY (2).
•
Are X and Y independent?
fX (3) = f (3, 1) + f (3, 2) =
x = 1, 2, 3,
y = 1, 2.
3
.
7
4
.
7
f (3, 2) ̸= fX (3)fY (2) ⇝ X and Y are dependent.
fY (2) = f (1, 2) + f (2, 2) + f (3, 2) =
Z.-L. Chen
MAT2371B Fall 2023
10 / 26
Textbook example 4.1-7 (1)
Example 7 (Textbook example 4.1-7)
•
200 balls: 40 black, 60 brown, 100 white.
•
Select a sample of size 25, without replacement.
•
Let X = number of black balls, Y = number of brown balls.
•
Find the joint pmf f (x, y), fX (x), and fY (y).
•
Are X and Y independent?
Z.-L. Chen
MAT2371B Fall 2023
11 / 26
Textbook example 4.1-7 (2)
e = {(x, y) ∈ N2 : x + y ⩽ 25} and joint pmf
• S
0
40
x
f (x, y) =
60
y
100
25−x−y
200
25
,
e
(x, y) ∈ S.
eX = SeY = {0, . . . , 25} and marginal pmf.
• S
40
x
160
25−x
200
25
fX (x) =
60
y
140
25−y
200
25
, x ∈ SeX ;
fY (y) =
, y ∈ SeY .
e=
• X and Y are not independent because S
̸ SeX × SeY .
Z.-L. Chen
MAT2371B Fall 2023
12 / 26
Expectation (1)
Definition 8
•
e
Let (X, Y ) have joint pmf f (x, y) and state space S.
•
u : R2 → R, a bivariate function.
•
The random variable u(X, Y ) has expectation
X
E(u(X, Y )) =
u(x, y)f (x, y).
e
(x,y)∈S
Z.-L. Chen
MAT2371B Fall 2023
13 / 26
Expectation (2)
The function u(x, y) may take some special forms.
•
If u(x, y) = x, then u(X, Y ) = X and
E(u(X, Y )) = E(X) = µX
.
•
If u(x, y) = (x − µX )2 , then
E(u(X, Y )) = E[(X − µX )2 ] = Var(X).
K What happens when u(x, y) = y and u(x, y) = µY = E(Y )?
Z.-L. Chen
MAT2371B Fall 2023
14 / 26
Textbook example 4.1-6
Example 9
•
X and Y have joint pmf
f (x, y) = (3 − x − y)/8,
•
x ∈ {0, 1},
y ∈ {0, 1}.
Find E(X + Y ).
Take u(x, y) = x + y, then
E(u(X, Y )) =
1 X
1
X
x=0 y=0
=0·
Z.-L. Chen
(x + y)
3−x−y
8
3
2
2
1
3
+1· +1· +2· = .
8
8
8
8
4
MAT2371B Fall 2023
15 / 26
Linearity
Theorem 10 (∗)
If X1 , . . . , Xn are random variables, which are all defined a same
sample space. And, if a1 , . . . , an are constants, then
n
n
X
X
ai E(Xi ).
E
ai Xi =
i=1
(6)
i=1
(6) is valid for all types of r.v.’s: discrete, continuous, and mixed.
Z.-L. Chen
MAT2371B Fall 2023
16 / 26
Covariance
•
Consider u(X, Y ) = (X − µX )(Y − µY ) and define
σXY = Cov(X, Y ) = E[(X − µX )(Y − µY )],
(7)
the covariance of X and Y .
•
By a direct expansion,
Cov(X, Y ) = E(XY − µX Y − µY X + µX µY )
Theorem 10
========= E(XY ) − µX E(Y ) − µY E(X) + µX µY
= E(XY ) − µX µY
Z.-L. Chen
MAT2371B Fall 2023
17 / 26
Example
Example 11 (Textbook example 4.2-3)
•
Let X and Y have the joint pmf
f (x, y) = 1/3,
(x, y) = (0, 1), (1, 0), (2, 1).
•
Are X and Y independent?
•
Find µX , µY , Cov(X, Y ).
e is not “rectangular”, X and Y must be
• The state space S
dependent.
• µX = 1, µY = 2/3, Cov(X, Y ) = E(XY ) − µX µY = 0.
Z.-L. Chen
MAT2371B Fall 2023
18 / 26
Correlation
•
Consider (X, Y ) such that σX > 0, σY > 0.
•
Define
ρ=
Cov(X, Y )
,
σX σY
(8)
the correlation coefficient of X and Y .
K E(XY ) = µX µY + ρσX σY .
KK∗ If ρ exists, then −1 ⩽ ρ ⩽ 1.
Theorem 12 (∗)
If r.v.’s X and Y are independent, then
Cov(X, Y ) = ρ = 0.
However, the converse is not true in general.
Z.-L. Chen
MAT2371B Fall 2023
19 / 26
Joint PDF (1)
Definition 13
Let X and Y be two continuous r.v.’s defined on a same sample
space S. Let Se ⊂ R2 be the state space of (X, Y ).
n
o
Se =
X(s), Y (s) , s ∈ S .
(9)
The The joint cdf is the function
F (x, y) = P(X ≤ x, Y ≤ y).
The joint pdf is the function f (·, ·) such that
Z x Z y
F (x, y) =
f (u, v)dudv.
−∞
Z.-L. Chen
(10)
(11)
−∞
MAT2371B Fall 2023
20 / 26
Joint PDF (2)
The joint pdf in Definition 13 satisfies:
f (x, y) ⩾ 0.
R∞ R∞
• −∞ −∞ f (u, v)dudv = 1.
•
•
e
For any A ⊂ S,
Z
P{(X, Y ) ∈ A} =
f (u, v)dudv.
(x,y)∈A
Z.-L. Chen
MAT2371B Fall 2023
21 / 26
Expectation
Definition 14
•
e
Let (X, Y ) have joint pdf f (x, y) and state space S.
•
u : R2 → R, a bivariate function.
•
The random variable u(X, Y ) has expectation
Z ∞Z ∞
E(u(X, Y )) =
u(x, y)f (x, y)dxdy
−∞
Z.-L. Chen
−∞
MAT2371B Fall 2023
22 / 26
Marginal PDF and Independence
Theorem 15
Let (X, Y ) have joint pdf f (·, ·), the the marginal densities for X
and Y are
Z ∞
f (x, y)dy,
fX (x) =
−∞
Z ∞
f (x, y)dx.
fY (y) =
−∞
Theorem 16
X and Y are independent if and only if, for all x and y,
f (x, y) = fX (x)fY (y).
Z.-L. Chen
MAT2371B Fall 2023
23 / 26
Example
Consider f (x, y) = 4(1 − xy)/3 for 0 < x, y < 1.
R1R1
• Verify that 0 0 f (x, y)dxdy = 1.
•
•
The marginals are
Z 1
fX (x) =
f (x, y)dy = 4(1 − x/2)/3,
0
Z 1
fY (y) =
f (x, y)dx = 4(1 − y/2)/3,
0 < x < 1;
0 < y < 1.
0
•
X and Y are not independent, but identically distributed.
Z.-L. Chen
MAT2371B Fall 2023
24 / 26
Bivariate Independence (1)
•
Random variables X and Y are independent if and only if
P(X ⩽ x, Y ⩽ y) = P(X ⩽ x)P(Y ⩽ y),
F (x, y) = FX (x)FY (y)
for all x, y ∈ R.
•
For discrete/continuous random variables, it is equivalent to
that
f (x, y) = fX (x)fY (y)
for all x, y ∈ R.
Z.-L. Chen
MAT2371B Fall 2023
25 / 26
Bivariate Independence (2)
In general, two discrete/continuous random variables X and Y are
independent if and only if
fX,Y (x, y) = g(x)h(y),
i.e. the joint pmf/pdf can be factorized as the product g(x)h(y) of
a function of x alone and a function of y alone.
Z.-L. Chen
MAT2371B Fall 2023
26 / 26