Chapter 2: Random Variables
2.1. Concept of a Random Variable
2.2. Distribution Functions
2.3. Density Functions
2.4. Mean Values and Moments
2.5. The Gaussian Random Variable
2.6. Density Functions Related to Gaussian
2.7. Other Probability Density Functions
2.8. Conditional Probability Distribution and Density Functions
2.9. Examples and Applications
Random variable:
A real function whose domain is that of the outcomes of an experiment
(sample space, S) and whose actual value is unknown in advance of the
experiment.
From: http://en.wikipedia.org/wiki/Random_variable
A random variable can be thought of as the numeric result of operating a non-deterministic
mechanism or performing a non-deterministic experiment to generate a random result.
Unlike the common practice with other mathematical variables, a random variable cannot be
assigned a value; a random variable does not describe the actual outcome of a particular
experiment, but rather describes the possible, as-yet-undetermined outcomes in terms of real
numbers.
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Spring 2015
1 of 17
ECE 3800
Probability Distribution Function (PDF) also called the
Cumulative Distribution Function (CDF)
Probability Distribution Function:The probability of the event that the observed random variable
X is less than or equal to the allowed value x.
FX  x   Pr  X  x 
The defined function can be discrete or continuous along the x-axis. Constraints on the
probability distribution function are:
1. 0  FX  x   1, for    x  
2. FX     0 and FX    1
3. FX is non-decreasing as x increases
4. Pr  x1  X  x2   FX  x 2   FX  x1 
Note: This is also known as the cumulative distribution function or cdf in many references!
(this avoids a major problems with language that is coming soon …)
From: http://en.wikipedia.org/wiki/Cumulative_distribution_function
In probability theory, the cumulative distribution function (abbreviated cdf) completely describes
the probability distribution of a real-valued random variable, X. For every real number x, the cdf
is given by
FX  x   Pr  X  x 
where the right-hand side represents the probability that the variable X takes on a value less than
or equal to x. The probability that X lies in the interval (a, b) is therefore F(b) − F(a) if a ≤ b. It is
conventional to use a capital F for a cumulative distribution function, in contrast to the lowercase f used for probability density functions and probability mass functions.
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Spring 2015
2 of 17
ECE 3800
For discrete events, the probability density function, on the x-axis, consists of discrete steps
“climbing” towards 1 at the appropriate points.
For a six-sided die,
Pr X  aint eger ,1  a int eger  6  
1
6
the probability density function can be defined as:
1.0
FX  x 
x
0.0
1
2
3
4
5
6
For discrete events, Pr X  a int eger ,1  a int eger  6   0 or
Pr X  a int eger ,1  a int eger  6   FX a int eger     FX a int eger     0
Examples:
Pr  X  1  FX 1 
1
6
Pr  X  3  FX 3 
1
2
Pr  X  5  FX 5 
5
6
Pr  X  7   FX 7   1.0
Pr  X  4   1  FX 4   1 
4 2

6 6
Pr 2  X  5  FX 5  FX 2  
5 2 3
 
6 6 6
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Spring 2015
3 of 17
ECE 3800
For continuous events, the PDF consists of a continuous, non-decreasing curve. For example:
FX  x 
1.0
x
0.0
-10
10
Examples:
Pr  X  0   FX 0  
1
2
Pr  X  5  FX 5 
Pr  X  3  FX 3 
1
4
1
13
 3  10  
20
20
Pr  X  7   FX  7  
1
3
  7  10  
20
20
Pr  X  5  1  FX 5  1 
1
5 1
 5  10  

20
20 4
Pr  1  X  1  FX 1  FX  1 
2
1
1
 1  10  
  1  10  
20
20
20
What about Pr  X  0
Pr    X     FX    FX    
1
1
2
   10  
    10  
20
20
20
2   2  0

0
limPr    X     lim 
 0
 0 20 

 20
As there are an infinite number of points in any region of the x axis, the probability of any
specific point for a continuous distribution is zero.
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Spring 2015
4 of 17
ECE 3800
Probability Density Function (pdf)
The derivative of the probability distribution function
f X  x   lim
FX  x     FX  x 

 0

dFX x 
dx
An interpretation is
f X  x   dx  Pr x  X  x  dx   FX  x  dx   FX  x 
Properties of the pdf include
1.
f X x   0, for    x  

2.
 f x   dx  1
X

x
3. F X 
 f X u   du

4. Pr  x1  X  x 2  
x2
 f x   dx
X
x1
From: http://en.wikipedia.org/wiki/Probability_density_function
In mathematics, a probability density function (pdf) serves to represent a probability distribution
in terms of integrals. A probability density function is everywhere non-negative and its integral
from −∞ to +∞ is equal to 1. If a probability distribution has density f(x), then intuitively the
infinitesimal interval [x, x + dx] has probability f(x) dx.
Be careful of the possible confusion “little” pdf (density) vs. “big” PDF (distribution).
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Spring 2015
5 of 17
ECE 3800
Probability Mass Function (pmf)
The probability that a discrete random variable takes on an exact value is defined as the pmf.
f X  x   Pr  X  x 
f X x   FX x   FX x   
Note that for discrete random variables, the probability distribution function, PDF, is not
continuous at the discrete inputs of interest.
Properties of the pdf include
1.
f X x   0, for    x  

2.
 f X u   1
u  
3. F X x  
x
 f X u 
u  
4. Pr  x1  X  x 2  
x2
 f u 
u  x1
X
From: http://en.wikipedia.org/wiki/Probability_mass_function
In probability theory, a probability mass function (abbreviated pmf) gives the probability that a
discrete random variable is exactly equal to some value. A probability mass function differs from
a probability density function in that the values of the latter, defined only for continuous random
variables, are not probabilities; rather, its integral over a set of possible values of the random
variable is a probability.
Note: The textbook does not differentiate between the probability density function and
probability mass function. Notice that in the definitions, the pmf represents the actual probability
while the pdf is defined in terms of the derivative of the “distribution” function (PDF or cdf).
If you wish to pursue correct mathematical derivations, use pmf and pdf and CDF. If you just
intend to apply this concept to engineering problems, you can do it like the textbook. …
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Spring 2015
6 of 17
ECE 3800
Examples:
1.0
1.0
f X x 
FX  x 
1/6
x
x
0.0
0.0
1
2
3
4
5
6
1
Probability Distribution Function (PDF)
FX  x 
2
3
4
5
6
Probability Mass Function (pmf)
1.0
1.0
f X x 
1/20
x
0.0
-10
x
0.0
10
-10
Probability Distribution Function (PDF)
10
Probability Density Function (pdf)
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Spring 2015
7 of 17
ECE 3800
Examples:
FX 
Given a PDF of
1 
2
 x 
 1   tan 1   , for    x  
2  
 5 
Probability Distribution Function (PDF)
1
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
-100
-80
-60
-40
-20
0
20
40
60
80
100
Define the pdf …
The derivative of the PDF is the pdf. Therefore,
f X x  
5  1 

, for    x  
  x 2  52 
Probability Density Function (pdf)
0.07
0.06
0.05
0.04
0.03
0.02
0.01
0
-100
Math hint:
-80
-60
-40
-20
0
20
40
60
80
100
d
du
1

tan 1 u   2 2 
dx
u  1 dx
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Spring 2015
8 of 17
ECE 3800
Updating Previous Examples
Experiment: Flip two Coins and count the number of heads
S  0,1,2
S Pair  TT , TH , HT , HH 
for x  0
0,
1
 4,
FX  x   
3 4 ,

1,
For FX  x   Pr  X  x 
for 0  x  1
for 1  x  2
for 2  x
And the probability mass function, f X  x   Pr  X  x  , is then
1 ,
 4
 2 ,
f X x    4
1 ,
 4
0,
1.0
for x  0
for x  1
for x  2
else
1.0
FX  x 
f X x 
1/2
1/4
x
0.0
x
0.0
-1
0
1
2
3
4
-1
Probability Distribution Function (PDF)
0
1
2
3
4
Probability Mass Function (pmf)
Note: The pmf corresponds to Bernoulli trials of 0, 1, and 2 occurrences in 2 trials with a
probability of 50%.
 n
Pr  A occuring k times in n trials   p n k     p k  q n k
k 
 2
 2
 2
1
2
1
p 2 0     0.5 2 
p 2 1     0.5 2 
p 2 2     0.5 2 
4
4
4
0
1
 2
What is the pmf and PDF for a 4-coin set ?
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Spring 2015
9 of 17
ECE 3800
Exercise 2-2.1 A random experiment consists of flipping four coins and taking the random
variable to be the number of heads.
 4
pmf k   p 4 k      p k  q 4 k
k 
n  4

 
FN n       p k  q 4 k 
k  0  k 

(a) Sketch the pmf and PDF.
Using HW 1-10.7 for n=4 and p=0.5:
Probability mass and distribution functions
1
pmf
CDF
0.9
0.8
0.7
Probability
0.6
0.5
0.4
0.3
0.2
0.1
0
k=
k=
k=
k=
k=
0
0.5
1
1.5
2
2.5
Subscribers
3
3.5
4
0, Prod = 0.062500, Cum pmf = 0.062500
1, Prod = 0.250000, Cum pmf = 0.312500
2, Prod = 0.375000, Cum pmf = 0.687500
3, Prod = 0.250000, Cum pmf = 0.937500
4, Prod = 0.062500, Cum pmf = 1.000000
(b) What is the probability that the random variable is less than 3.5?
3  4

 
Pr n  3.5  FN 3.5      p k  q 4 k   p 4 0  p 4 1  p 4 2  p 4 3
k  0  k 

Pr n  3.5  FN 3.5  0.9375 
15
16
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Spring 2015
10 of 17
ECE 3800
(c) What is the probability that the random variable is greater than 2.5?
Pr 2.5  n   p 4 3  p 4 4
2  4

 
Pr 2.5  n   1  FN 2.5  1      p k  q 4 k   1  p 4 0   p 4 1  p 4 2
k 0  k 

Pr 2.5  n   0.25  0.0625  0.3125 
5
16
Pr 2.5  n   1  FN 2.5  1  0.6875  0.3125 
5
16
(d) What is the probability that the random variable is greater than 0.5 and less than or equal to
3.0?
Pr 0.5  n  3  p 4 1  p 4 2  p 4 3
3  4

 
Pr 0.5  n  3  FN 3  FN 0.5      p k  q 4k   p 4 1  p 4 2  p 4 3
k 1  k 

14 7

16 8
14 7

Pr 0.5  n  3  0.9375  0.0625  0.875 
16 8
Pr 0.5  n  3  0.250  0.375  0.250  0.875 
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Spring 2015
11 of 17
ECE 3800
First Look describing “named” random variables
Note: there are documents describing specific discrete and continuous pmf, pdf and PDF or CDF
on the password web site. The textbook also lists them in Appendix B, p. 425.
Uniform Random Variables
The uniform random variable arises in situations where all values in an interval of the real line
are equally likely to occur. The uniform random variable U in the interval [a,b] has pdf:
 1
, a xb

fU x   b  a
0,
x  0 and x  b
x0
0,
x  a

, a xb
FU  x  
b  a
xb
1,
FX x 
1.0
1.0
f X x 
1/(b-a)
x
a
b
x
a
b
Exponential Random Variables
The exponential random variable arises in the modeling of the time between occurrence of events
(e.g., the time between customer demands for call connections), and in the modeling of the
lifetime of devices and systems. The exponential random variable X with parameter l has pdf
x0
dFX x 0,

dz
  exp   x, x  0
x0
0,
FX  x  
1  exp   x, x  0
f X x 
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Spring 2015
12 of 17
ECE 3800
The Gaussian probability density function (pdf)
The Gaussian or Normal probability density function is defined as:
f X x  


 x X 2 
, for    x  
 exp


2
2  
 2 

1
X is the mean and  is the variance
where
The Gaussian Probability Distribution Function (PDF)


 v X 2 
  dv
FX  x  
 exp


2
2  
2 


v  
x

1
The PDF can not be represented in a closed form solution!
Gaussian PDF and pdf
1
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
-8
-6
-4
-2
0
2
4
6
8
Bernoulli Random Variable
Every Bernoulli trial, regardless of the event A, is equivalent to the tossing of a biased coin with
probability of heads p. In this sense, coin tossing can be viewed as representative of a
fundamental mechanism for generating randomness, and the Bernoulli random variable is the
model associated with it.
S X  0,1
p0  1  p  q and p1  p , for 0  p  1
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Spring 2015
13 of 17
ECE 3800
The Binomial Random Variable
The binomial random variable arises in applications where there are two types of objects (i.e.,
heads/tails, correct/erroneous bits, good/defective items, active/silent speakers), and we are
interested in the number of type 1 objects in a randomly selected batch of size n, where the type
of each object is independent of the types of the other objects in the batch.
S X  0,1,2,, n
n
nk
pk     p k  1  p  ,
k 
for k  0,1,2,, n
The Poisson Random Variable
In many applications, we are interested in counting the number of occurrences of an event in a
certain time period or in a certain region in space. The Poisson random variable arises in
situations where the events occur “completely at random” in time or space. For example, the
Poisson random variable arises in counts of emissions from radioactive substances, in counts of
demands for telephone connections, and in counts of defects in a semiconductor chip.
S X  0,1,2,
pk 
k
k!
 e  ,
for k  0,1,2,
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Spring 2015
14 of 17
ECE 3800
Functions of random variables
In engineering analysis, many times one random variable is a function of a second random
variable, for example,




random power derived from a random voltage Y  X 2
circular area derived from a random measurement of the diameter Y  X 2
DC voltage measurement in the presence of R.V. noise Y  a  X
linear relationships Y  m  X  b
Think of Y  g  X 
… what can be described for the probability density functions (pdf) of Y and X?
Since the PDF is the integral of the pdf, we should have:
f X  x   dx  f Y  y   dy
For Y  g  X  a monotonically increasing function of X, the new pdf should be related to the
previous pdf as something like:
f Y  y   f X x  
dx
dy
For Y  g  X  a monotonically decreasing function of X, the new pdf must be increasing. As a
result the new pdf should be related to the previous pdf as:
f Y  y   f X x  
dx
dy
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Spring 2015
15 of 17
ECE 3800
Example: Amplitude scaling (an amplifier) a random variable … Y  A  X
f Y  y   f X x  
dx
dy
dx 1

dy A
 y 1
fY  y  f X   
 A A
Therefore,
1

f X  x    20
0
Example:
for  10  x  10
else
Let an amplifier have a gain of A  5
Then
And
Or
Y  A  X and
dx
1

dy  5
1
 y 1 1 
fY  y  f X   
   20
 A  A 5 0

 1

f Y  y   100
0
for  10 
y
 10
5
else
for  50  y  50
else
A uniform distribution remains a uniform distribution when gain is applied.
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Spring 2015
16 of 17
ECE 3800
f X x   e  x  u x 
Text example:
Y  X3
With
f X  x   lim
What is the PDF of X
F X  x     FX  x 

 0
FX  x  
The definition

dFX  x 
dx
x
 f v   dv
X
v  
FX  x  
x
e
v
 u v   dv
v  
x
FX  x  
e
v
 dv
v 0
FX  x    e  v
x
v 0
 e  x   1  1  e  x ,
for 0  x  
Probability Distribution Function (PDF)
Probability Density Function (pdf)
1
1
0.9
0.9
0.8
0.8
0.7
0.7
0.6
0.6
0.5
0.5
0.4
0.4
0.3
0.3
0.2
0.2
0.1
0.1
0
0
1
2
3
4
5
6
7
8
9
0
10
Probability Distribution Function (PDF)
0
1
2
3
4
5
6
7
8
9
10
Probability Density Function (pdf)
f Y  y   f X x  
What is the new pdf for Y
dx
dy
d  y 3 
dx
  1   y 23 
 



dy
dy
3 
1
1
1
1
2
3
f Y  y   e  y  u  y 3     y 3 

 3 

fY  y 
1  y 13  2 3
e
 y  u y 
3
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Spring 2015
17 of 17
ECE 3800