Chapter 2: Random Variables
2.1. Concept of a Random Variable
2.2. Distribution Functions
2.3. Density Functions
Functions of random variables
2.4. Mean Values and Moments
Hypergeometric Distribution
2.5. The Gaussian Random Variable
Histograms
2.6. Density Functions Related to Gaussian
2.7. Other Probability Density Functions
2.8. Conditional Probability Distribution and Density Functions
2.9. Examples and Applications
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Spring 2015
1 of 18
ECE 3800
Homework Problem request/Discussion
2-7.2 (uniform density)
A continuous-valued random voltage ranging from -20 V to +20 V
is to be quantized so that it can be represented by a binary sequence.
a.) If the rms quantizing error is to be less than 1% of the maximum value of the voltage, find the
minimum number of quantizing levels that are required.
2

q 
2
The variance for a uniform distribution is
.
 
12
q
The rms voltage size is then

12
q
Rms quantization error 1% of max voltage implies  
 20  0.01
12
q  20  0.01  12  0.6928
2  20
The number of required levels is then
 57.73
q
Rounding up would require 58 levels.
Yes this is a strange way to state the problem and find a solutions!
b.) If the number of quantizing levels is to be a power of 2, find the minimum number of
quantizing levels that will still meet the requirements.
The next highest power of 2 is 64 or 2^6.
c.) How many binary digits are required to represent each quantizing level?
To represent the values, you need a 6-bit binary number.
Voltage step sizes
2  20
 0.625  q
26
[Finding an ADC] Quantization error is +/- ½ least-significant bit. For 1% accuracy (or better), I
would consider (in binary) 128 levels (27) or more. To adequately cover a +/- 20 V range, I
would make the ADC measure a +/- 25.6 V range and then have binary voltage steps of 0.2V.
The mean error would be zero, the maximum error would be +/- 0.1 V and the “rms” error would
be sqrt(0.22/12) = 0.058V.
Standard ADC number of bits: 8, 12, 16, 20, and 24 are typical. Important factor in selection,
what sample rate is required! The type and cost will vary in relationship to samples-per-second.
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Spring 2015
2 of 18
ECE 3800
2.6
Density Functions Related to Gaussian
Rayleigh Distribution
For a two dimensional problem (positions in x and y with two independent Gaussian random
variable noise or offset terms), the distance from a desired point is described as a radial or vector
magnitude, the radial error or offset is described by the Rayleigh Distribution.
R
For
X 2 Y 2
The probability density function (pdf) is
  r2 
,
 exp
 2  2 
2


 0,
f R r  
r
for 0  r
for r  0
The probability distribution function (PDF) can be derived as:
r
  v2 
  dv,
 exp
2
2


 2  
v 0
 0,
FR r  

v
  r2 
,
FR r   1  exp
 2  2 


 0,
for 0  r
for r  0
for 0  r
for r  0
Rayleigh Dist. and Density Fundtions
1
0.9
0.8
0.7
0.6
pdf
PDF
0.5
0.4
0.3
0.2
0.1
0
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Spring 2015
3 of 18
ECE 3800
The first moment
R  E R  


r
r 0
  r2 
  dr
 exp
2
2


 2  
r

x
Closed form solution (p. 421)
2


 exp  a 2  x 2  dx 
r 0
R  E R  
1

2


4


2 
3 

2
 

2

4  a3

The second moment

   r
2
R ER
2
2

r 0
  r2 
  dr
 exp
2
2


 2  
r
u  v 
u
u
 x   v   x  u  v   v   x
x
x
x

2
 r
  r 
  dr
R 2  E R 2   r 2   2  exp
2 


2



r 0

2
 r 
v
r

 2  exp
u  r 2 and
2 
x 
 2  
v
 u  x  x  
Math tables hint
 
 
  r2 
u

 2  r and v   exp
2 
x
 2  
and keep going until you get ….
 
R2  E R2  2  2
The second central moment, variance or standard deviation is

 
  

 2
2


E  r  R    R2  R 2  E R 2   2     2


2


E r  R




  R2  R 2  E R 2  2   2  

2

2

Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Spring 2015
4 of 18
ECE 3800
Example: Archery target shooting with “Gaussian”    X   Y  0.25 
1
4
Assume a 1 foot radius target with a 1 inch radius Bulls-eye
  r2 
r
, for 0  r
f R r  
 exp
2
2


 2  
 0,
for r  0
FR r 
  r2 
,
 1  exp
 2  2 


 0,

for 0  r
for r  0

f R r   16  r  exp  8  r 2 ,
 0,


FR r   1  exp  8  r 2 ,
 0,
R  E R  


r
r 0

for 0  r
for r  0
for 0  r
for r  0
  r2 
  dr    1    0.313
 exp
 2  2 
2 4
32
2


r
2

2
1 

  1


E  r  R    R2   2     2   2       
 0.0268


2
2   4
8 32




2
1 
E r  R    R 

 0.164


8 32
Probability of a Bulls-eye (1 inch radius)

1 2 
1
 8 
FR    1  exp  8 
 1  exp 
  0.0540


12
144
 12 




Probability of missing the target (1 foot radius)



1  FR 1  1  1  exp  8  12  exp 8  3.35 10 4
see RayleighGen.m
see Rayleigh.m (based on textbook example p. 90)
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Spring 2015
5 of 18
ECE 3800
Maxwell Distribution
For a three dimensional problem (the velocity of a particle in a perfect gas), the velocity vector
magnitude is described by the Maxwell Distribution.
V  V X 2  VY 2  VZ 2
For
The probability density function (pdf) is
  v2
2 v2
f V v  
 3  exp
2
 
 2 
 0,

,

for 0  v
for v  0
The probability distribution function (PDF) can be derived as:
v
  x2 
2 x2
  dx,
FV v   
 3  exp
2 
 
 2  
x 0
for 0  v
 0,
for v  0
The solution requires numerical integration!
The first moment
V  EV  


  x2
2 x2
 3  exp
2
 
 2 
x
x 0

8
  dx 



The second moment

  x
V2  EV2 
x 0
2

  x2
2 x2
 3  exp
2
 
 2 
 

  dx

V 2  E V 2  3  2
The second central moment, variance or standard deviation is
2

 8
2
E  v  V    V2  V 2  EV 2  3   2  
  



 
2
8

E  v  V    V2  V 2  E V 2   3     2  0.453   2








Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Spring 2015
6 of 18
ECE 3800
from http://en.wikipedia.org/wiki/Maxwell-Boltzmann_distribution
see MaxwellGen.m
Maxwell-Boltzmann Dist. and Density Fundtions
1
0.9
0.8
0.7
0.6
pdf
PDF
0.5
0.4
0.3
0.2
0.1
0
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
5
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Spring 2015
7 of 18
ECE 3800
Other Distributions
Exponential Distribution
For events that are relative to an exponential decay, such as the probability of a new event
happening in the near term.
The probability density function (pdf) is
1
  
 exp
f T   
,
M
M 
 0,
for 0  
for   0
The probability distribution function (PDF) can be derived as:
  
FT    1  exp
for 0  
,
M 
 0,
for   0
http://en.wikipedia.org/wiki/Exponential_distribution
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Spring 2015
8 of 18
ECE 3800
From http://en.wikipedia.org/wiki/Exponential_distribution
In real world scenarios, the assumption of a constant rate (or probability per unit time) is rarely
satisfied. For example, the rate of incoming phone calls differs according to the time of day. But
if we focus on a time interval during which the rate is roughly constant, such as from 2 to 4 PM
during work days, the exponential distribution can be used as a good approximate model for the
time until the next phone call arrives.
Similar caveats apply to the following examples which yield approximately exponentially
distributed variables:

the time until you have your next car accident;

the time until a radioactive particle decays, or the time between beeps of a geiger counter;

the number of dice rolls needed until you roll a six 11 times in a row;

the time until a large meteor strike causes a mass extinction event.
Exponential variables can also be used to model situations where certain events occur with a
constant probability per unit distance:

the distance between mutations on a DNA strand;

the distance between road kill on a given highway;
In queueing theory, the inter-arrival times (i.e. the times between customers entering the system)
are often modeled as exponentially distributed variables.
Reliability theory and reliability engineering also make extensive use of the exponential
distribution. Because of the memoryless property of this distribution, it is well-suited to model
the constant hazard rate portion of the bathtub curve used in reliability theory.
In physics, if you observe a gas at a fixed temperature and pressure in a uniform gravitational
field, the heights of the various molecules also follow an approximate exponential distribution.
This is a consequence of the entropy property.
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Spring 2015
9 of 18
ECE 3800
The first moment
T  E T  


x
x 0

From tables
1
 x
 exp
  dx
M
M 
x  expax   dx 
expax 
 ax  1
a2

 x
exp

1
 M    x  1   1

T  E T  


M  1 2  M
M

 
M 
0

1
 1 
 
M 
2
 1  M
The second moment

2
    x  M1  exp Mx   dx
T 2  E T 2   2  M 2
T  ET
2
2
x0
The second central moment, variance or standard deviation is
2
E    T    T2  T 2  ET 2  2  M 2  M 2  M 2




The memoryless property: Derive the PrT  t  h | T  h
Based on prior set theory
PrT  t  h | T  h 
We know that
PrT  t  h  PrT  h PrT  t  h

PrT  h
PrT  h


PrT  h  1  FT h  1  1  exp  h
Therefore
PrT  t  h | T  h 

exp  t  h
M

  exp h M 


M  exp  t
M
exp  h
M
But this is the same as PrT  t  , the probability is not dependent upon the “time history” and
thereby can be called memoryless. (Note, this is the only continuous random variable that
exhibits this property.)


Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Spring 2015
10 of 18
ECE 3800
Example: Component failure
Component failure in a spacecraft occurs independently and uniformly with the average time
between failures of 100 days. The spacecraft starts on a 200-day mission with all components
functioning. What is the probability that it will complete the mission without a component
failure?
2
T  M  100 and E    T    T2  M 2  10 4




The probability density function (pdf) is
1
  
 exp
f T   
,
100
 100 
 0,
for 0  
for   0
The probability distribution function (PDF) can be derived as:
  
FT    1  exp
for 0  
,
 100 
 0,
for   0
Probability of no failures:

   
NoFailure  PrT     1  FT    1  1  exp

 100 

  
NoFailure    1  FT    exp

 100 
  200 
NoFailure 200   exp
  0.1352
 100 
If the satellite has lasted 100 days, what is the probability that it will last another 200 days?
PrT    h | T  h
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Spring 2015
11 of 18
ECE 3800
Example p. 94: Component failure #2
A satellite has a traveling wave tube (TWT) that is used for transmitting a microwave signal back
to earth. The TWT has a mean-time-to-failure (MTTF) of four years. What is the probability that
the TWT lasts longer than 4 years? Lasts 1 year or less?
2
T  M  4 and E    T    T2  M 2  16




The probability density function (pdf) is
1
  
f T     exp
,
4
 4 
 0,
for 0  
for   0
The probability distribution function (PDF) can be derived as:
  
FT    1  exp
for 0  
,
 4 
 0,
for   0
(a) Probability of lasting 4 or more years:

  4 
4
Prt  4  1  FT 4  1  1  exp
  0.368
  exp
 4 
 4 

(b) Probability of lasting 1 or less years:
 1
Prt  1  FT 1  1  exp   0.221
 4 
(c) Probability that it will fail between years 4 and 6:

  4 
  6  
Pr4  t  6  FT 6  FT 4   1  exp
  exp 1  exp 1.5
  1  exp
 4 
 4  

Pr4  t  6  FT 6  FT 4  0.0821
Typically a satellite would have multiple, redundant TWTs. Then, if one failed another would be
enabled until all available TWTs failed. If, for example, we had 4 TWTs, we would be
interested in “the 4th failure event” and may not care about the first 3 failure events (the satellite
still operates).
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Spring 2015
12 of 18
ECE 3800
The Erlang Distribution: The random variable that described the time
between any event and the kth following event.
http://en.wikipedia.org/wiki/Erlang_distribution
The Erlang distribution was developed by A. K. Erlang to examine the number of telephone calls
which might be made at the same time to the operators of the switching stations. This work on
telephone traffic engineering has been expanded to consider waiting times in queueing systems
in general. The distribution is now used in the field of stochastic processes.
The probability density function (pdf) is
 k 1
  
 exp
,
M 
M k  k  1!
 0,
f KT  

The first moment
for 0   , k  0,1, 2, ...
for   0
KT  E KT   k  M
The variance
2
 KT
 k M 2
What is the probability that the four TWTs will last longer than 4 years if each successive TWT
is not used until the previous one fails?
f 4T  

3
  
 exp
,
M 
M 4  3!
for 0   , k  4
for   0
 0,
1  F4T   4  1 

t3
t
 M 4  3!  exp M   dt
t 0
From tables

x  expax   dx  expax  
m
m
  1r 
r 0
m! x m  r
m  r !a r 1
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Spring 2015
13 of 18
ECE 3800
Therefore
t
F4T   4  
 exp  
M 
M 4  3!
1
3
  1r 
r 0
3! x 3  r
3  r !a r 1

3
 1
3t2
6t
6  
t  t
 exp   



F4T   4  

 M    M 1 M  2  M  3 M  4  
 M 4  6
0
3


1
3  2
6 
6    1
6 
    
 exp




F4T    
    4    4  

4

1

2

3

4
 M   M
M
M
M    M  6  M  
 M  6
3

3  2
6 
     


 1
F4T    1  exp

 M   6  M 3 6  M 2 6  M

What is the probability that the four TWTs will last longer than 4 years if each successive TWT
is not used until the previous one fails?
3
3  4 2 6  4 
  4   4


 1
F4T 4  1  exp

 4   6  4 3 6  4 2 6  4 
16
1 3 6 
F4T 4   1  exp 1      1  1  exp 1   0.0190
6
6 6 6 
1  F4T 4  1  0.0190  0.981
and finally
Significantly better than the previous result … where we had

4
  4 
Prt  4  1  FT 4  1  1  exp
  0.368
  exp
 4 
 4 

For critical application, redundant devices are regularly used. When this happens, failures must
be detected and a means to replace the failed unit with a “spare” must be available.
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Spring 2015
14 of 18
ECE 3800
Discrete Density Functions
Binomial Distribution
The probability mass function (pmf) is
f B x  
n
 n k
   p  1  p n  k    x  k 
k
k 0 

The probability distribution function (PDF) can be derived as:
FB  x  
n
 n k
   p  1  p n  k  u  x  k 
k
k 0 

http://en.wikipedia.org/wiki/Binomial_distribution
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Spring 2015
15 of 18
ECE 3800
Poisson Distribution
Sometimes λ is taken to be the rate, i.e., the average number of occurrences per unit time. In that
case, if N t is the number of occurrences before time t then we have:
The probability mass function (pmf) is
t x
Pr  N t  x   f P  x, t   e t 
for x  0,1,2.
x!
or
The probability distribution function (PDF) can be derived as:
x
x
k 0
k 0
FP  x    f P  x, t    e
Alternately, from the textbook
f P k   e
a

k 0
Given the math equivalence

t  x

x!
k

a

for k  0,1,2. and a  0
k!
E k   a and  k2  a
Note that

 t

a k
k 0
k!
f P k    e  a 
ex  1 x 
a k
k 0
k!
 f k   e
a
P
 ea  1
Then


k 0
k 0
E k    k  f P k    k  e

E k   0   a  e
k 1
E k   a  e
a
a
1

x2
xk
  
2!
k  0 k!

k 0

 e a  
a
k

a

k!
k 1
k 1



a
a
a

 ae 
k  1!
k 1 k  1!
k

a

k  0 k !

 a  e a  e a  a
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Spring 2015
16 of 18
ECE 3800
And computing the second moment
E k 2    k 2  f P k    k 2  e  a 


k 0
a k
k!
k 0
a   a  e a  k  a k 1

k  1!
k  1!
k 1
k 1
k
k


a   a  e a  a   a  e a   k  a k
E k 2   a  e a   k  1 


k !
k !
k 0
k  0 k !
k 0
k 1

a   a  a 2  e a  e a
E k 2   a  e a  e a  a 2  e a  
k 1 k  1!
E k 2   0   a  k  e a 

k 1

E k 2   a  a 2
Then
 k2  E k 2   E k 2  a  a 2  a 2  a
http://en.wikipedia.org/wiki/Poisson_distribution
Examples of events that can be modelled as Poisson distributions include:

The number of unstable nuclei that decayed within a given period of time in a piece of
radioactive substance.

The number of cars that pass through a certain point on a road during a given period of
time.

The number of spelling mistakes a secretary makes while typing a single page.

The number of phone calls at a call center per minute.

The number of times a web server is accessed per minute.
 For instance, the number of edits per hour recorded on Wikipedia's Recent
Changes page follows an approximately Poisson distribution.

The number of roadkill found per unit length of road (this one too).

The number of mutations in a given stretch of DNA after a certain amount of radiation.

The number of pine trees per unit area of mixed forest.

The number of stars in a given volume of space.

The distribution of visual receptor cells in the retina of the human eye.
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Spring 2015
17 of 18
ECE 3800
Poisson Example
There are 500 typos in a book of 750 pages. Assume that the typos per page are Poisson
Distributed.
Let X equal the number of typos on a given page. Then  
500
errors per page
750
What is the probability that there is one typo on a page?
Let t=1 page, Nt=X, the number of typos on the page
0
2
3
2
 
3
    0.5134
0!
2
3
2
 
3
    0.3423
1!
2
3
2
 
3
    0.1141
2!
2
3
2
 
3
    0.0254
3!

Pr  N t  0  f P 0  e
1
Pr  N t  1  f P 1  e

2
Pr  N t  2  f P 2   e

3
Pr  N t  3  f P 3  e

See Matlab example for number of errors in multiple page groups t=1:5.
Error Distribution Function for k pages k=1:5
1
0.9
0.8
0.7
probability
0.6
0.5
k=1
k=2
k=3
k=4
k=5
0.4
0.3
0.2
0.1
0
0
1
2
3
4
5
6
number of errors
7
8
9
10
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Spring 2015
18 of 18
ECE 3800