Pairs of Random Variables
Random Process
Introduction

In this lecture you will study:




Joint pmf, cdf, and pdf
Joint moments
The degree of “correlation” between two random variables
Conditional probabilities of a pair of random variables
Two Random Variables

The mapping is written as
outcome is S
to each
Example 1
Example 2
Two Random Variables

The events evolving a pair of random variables (X,Y) can
be represented by regions in the plane
Two Random Variables

To determine the probability that the pair
some region B in the plane, we have

Thus, the probability is

The joint pmf, cdf, and pdf provide approaches to
specifying the probability law that governs the behavior of
the pair (X,Y)
Firstly, we have to determine what we call product form

where Ak is one-dimensional event
is in
Two Random Variables

The probability of product-form events is

Some two-dimensional product-form events are shown
below
Pairs of Discrete Random Variables

Let the vector random variable
assume values
from some countable set
The joint pmf of X specifies the probabilities of event

The values of the pmf on the set SX,Y provide

Pairs of Discrete Random Variables
Pairs of Discrete Random Variables

The probability of any event B is the sum of the pmf over
the outcomes in B

When the event B is the entire sample space SX,Y, we have
Marginal Probability Mass Function


The joint pmf provides the information about the joint
behavior of X and Y
The marginal probability mass function shows the random
variables in isolation
similarly
Example 3
The Joint Cdf of X and Y

The joint cumulative distribution function of X and Y is
defined as the probability of the event

The properties are
The Joint Cdf of X and Y
Example 4
The Joint Pdf of Two Continuous Random
Variables



Generally, the probability of events in any shape can be
approximated by rctangles of infinitesimal width that leads
to integral operation
Random variables X and Y are jointly continuous if the
probability of events involving (X,Y) can be expressed as
an integral of probability density function
The joint probability density function is given by
The Joint Pdf of Two Continuous Random
Variables
The Joint Pdf of Two Continuous Random
Variables

The joint cdf can be obtained by using this equation

It follows

The probability of rectangular region is obtained by
letting
The Joint Pdf of Two Continuous Random
Variables

We can, then, prove that the probability of an infinitesimal
rectangle is

The marginal pdf’s can be obtained by
The Joint Pdf of Two Continuous Random
Variables
Example 5
Example 5
Example 6
Independence of Two Random Variables

X and Y are independent random variable if any event A1
defined in terms of X is independent of any event A2
defined in terms of Y

If X and Y are independent discrete random variables,
then the joint pmf is equal to the product of the marginal
pmf’s
Independence of Two Random Variables

If the joint pmf of X and Y equals the product of the
marginal pmf’s, then X and Y are independent

Discrete random variables X and Y are independent iff
the joint pmf is equal to the product of the marginal pmf’s
for all xj, yk
Independence of Two Random Variables

In general, the random variables X and Y are independent
iff their joint cdf is equal to the product of its marginal
cdf’s

In continuous case, X and Y are independent iff their joint
pdf’s is equal to the product of the marginal pdf’s
Joint Moments and Expected Values

The expected value of

Sum of random variable
is given by
Joint Moments and Expected Values

In general, the expected value of a sum of n random
variables is equal to the sum of the expected values

Suppose that
, we can get
Joint Moments and Expected Values

The jk-th joint moment of X and Y is given by

When j = 1 and k = 1, we can say that as correlation of X
and Y
And when E[XY] = 0, then we say that X and Y are
orthogonal

Conditional Probability
Case 1: X is a Discrete Random Variable
 For X and Y discrete random variables, the conditional
pmf of Y given X = x is given by

The probability of an event A given X = xk is found by
using

If X and Y are independent, we have
Conditional Probability

The joint pmf can be expressed as the product of a
conditional pmf and marginal pmf

The probability that Y is in A can be given by
Conditional Probability

Example:
Conditional Probability

Suppose Y is a continuous random variable, the
conditional cdf of Y given X = xk is

We, therefore, can get the conditional pdf of Y given X =
xk

If X and Y are independent, then
The probability of event A given X = xk is obtained by

Conditional Probability

Example: binary communications system
Conditional Probability
Case 2: X is a continuous random variable
 If X is a continuous random variable then P[X = x] = 0
 If X and Y have a joint pdf that is continuous and nonzero
over some region of the plane, we have conditional cdf of
Y given X = x
Conditional Probability

The conditional pdf of Y given X = x is

The probability of event A given X = x is obtained by

If X and Y are independent, then

The probability Y in A is
and
Conditional Probability

Example
Conditional Expectation

The conditional expectation of Y given X = x is given by

When X and Y are both discrete random variables
Conditional Expectation

In particular we have
where
Pairs of Jointly Gaussian Random Variables

The random variables X and Y are said to be jointly
Gaussian if their joint pdf has form
Lab assignment



In group of 2 (for international class: do it personally), refer to
Garcia’s book, example 5.49, page 285
Run the program in MATLAB and analyze the result
Your analysis should contain:






The purpose of the program
Line by line explanation of the program (do not copy from the book,
remember NO PLAGIARISM is allowed)
The explanation of Fig. 5.28 and 5.29
The relationship between the purpose of the program and the
content of chaper 5 (i.e. It answers the question: why do we study
Gaussian distribution in this chapter?)
Try using different parameter’s values, such as 100 observation,
10000 observation, etc and analyze it
Due date: next week
Regular Class:
NEXT WEEK: QUIZ 1
Material: Chapter 1 to 5, Garcia’s book
Duration: max 1 hour