Chapter 2: Random Variables
2.1. Concept of a Random Variable
2.2. Distribution Functions
2.3. Density Functions
Functions of random variables
2.4. Mean Values and Moments
Hypergeometric Distribution
2.5. The Gaussian Random Variable
Histograms
2.6. Density Functions Related to Gaussian
2.7. Other Probability Density Functions
2.8. Conditional Probability Distribution and Density Functions
2.9. Examples and Applications
Announcement: SWE Hosted Industry Dinner and Networking, 12 February!
For those of you who did not get the invitation on any of the Engineering
Facebook pages, the Society of Women Engineers is hosting an industry dinner
for all engineering students Thursday, February 12th, 2015 from 6:30-8:00.
Several companies (Eaton, Whirlpool, Comsumers Energy, etc.) are coming to
network with YOU over a nice, free dinner. There will also be a few key
speakers to talk about their experiences in the workforce.
Seats will be given on a first come first serve basis. If you would like to
register for this event please email your resume in pdf form to
swe.industrydinner.wmu@gmail.com.
I look forward to seeing you at the dinner!
Chelsea Russell
SWE Industry Dinner Coordinator
Western Michigan Universtiy
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Spring 2015
1 of 20
ECE 3800
2-5
Gaussian or Normal probability density function
The Gaussian or Normal probability density function is defined as:
f X x  
where


 x X 2 
, for    x  
 exp


2
2  
 2 

1
X is the mean and  is the variance.
The “normal density function” has a zero mean and unit variance. See Appendix D, p. 432.
0.4
0.35
0.3
0.25
0.2
0.15
0.1
0.05
0
-8
-6
-4
-2
0
2
4
6
8
Not yet proven reasons for importance:
1. It provides a good mathematical model for a great many different physically observed
random phenomena that can be justified theoretically in many ways.
2. It is one of the few density functions that can be extended to handle an arbitrarily large
number of random variables conveniently.
3. Linear combinations of Gaussian random variables lead to new random variables that are
also Gaussian. This is not true for most other density functions.
4. The random process from which Gaussian random variables are derived can be completely
specified, in a statistical sense, from a knowledge of the first and second moments. This is
not true for other processes. All higher level moments are sums, products and/or powers of
the mean and variance.
5. In system analysis, the Gaussian process is often the only one for which a complete statistical
analysis can be carried through in either the linear or nonlinear situation.
6. The function is infinitely differentiable (all the derivatives exist).
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Spring 2015
2 of 20
ECE 3800
Gaussian or Normal Distribution
http://en.wikipedia.org/wiki/Normal_distribution
http://en.wikipedia.org/wiki/Normal_distribution#Occurrence
To summarize, here is a list of situations where approximate normality is sometimes assumed.
For a fuller discussion, see below.

In counting problems (so the central limit theorem includes a discrete-to-continuum
approximation) where reproductive random variables are involved, such as
 Binomial random variables, associated to yes/no questions;
 Poisson random variables, associated to rare events;

In physiological measurements of biological specimens:
 The logarithm of measures of size of living tissue (length, height, skin area,
weight);
 The length of inert appendages (hair, claws, nails, teeth) of biological specimens,
in the direction of growth; presumably the thickness of tree bark also falls under
this category;
 Other physiological measures may be normally distributed, but there is no reason
to expect that a priori;

Measurement errors are assumed to be normally distributed, and any deviation from
normality must be explained;

Financial variables
 The logarithm of interest rates, exchange rates, and inflation; these variables
behave like compound interest, not like simple interest, and so are multiplicative;
 Stock-market indices are supposed to be multiplicative too, but some researchers
claim that they are Levy-distributed variables instead of lognormal;
 Other financial variables may be normally distributed, but there is no reason to
expect that a priori;

Light intensity
 The intensity of laser light is normally distributed;
 Thermal light has a Bose-Einstein distribution on very short time scales, and a
normal distribution on longer timescales due to the central limit theorem.
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Spring 2015
3 of 20
ECE 3800
The Gaussian Probability Distribution Function is


 v X 2 
  dv
 exp
FX  x  


2
2  
2 


v  
x

1
The PDF can not be represented in a closed form solution!
f X x  


 x X 2 
, for    x  
 exp


2
2  
 2 

1
Important notes on the curve:
1. There is only one maximum and it occurs at the mean value.
2. The density function is symmetric about the mean value.
3. The width of the density function is directly proportional to the standard deviation, . The
width of 2 occurs at the points where the height is 0.607 of the maximum value. These are
also the points of the maximum slope. Also note that:
Pr  X    X  X     0.683
Pr  X  2  X  X  2   0.955
4. The maximum value of the density function is inversely proportional to the standard
deviation, .
 
fX X 
1
2  
5. Since the density function has an area of unity, it can be used as a representation of the
impulse or delta function by letting  approach zero. That is

 1
 x X
 x  X  lim 
 exp
 0  2  
 2  2





2




Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Spring 2015
4 of 20
ECE 3800
Moments and Central Moments of Gaussian
X  X 
n


0,
n
 E X  X   
n
1  3    n  1   ,
 
Xn E Xn
X ,

2
 2  X ,

3
2
3    X  X ,

2
4
4
2
3    6    X  X ,
n odd
n even
n 1
n2
n3
n4
This is a partial list. As previously stated all moments are a function of the mean and
variance/standard deviation.
As a convenient property, the Gaussian is completely describe once the mean and variance are
known!
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Spring 2015
5 of 20
ECE 3800
Computing the Gaussian PDF
The PDF is tabulated in Appendix D for a zero mean, unit variance pdf.
For these values, it is often described as “normalized” and is defined as
x
 2 
1
x  
  exp    d
2   
 2 
The distribution function is then converted based on the relationship
x X
u

xX
FX  x   u   
 




When using Appendix D, the negative values of u are derived from the positive as
 u   1  u 
The normal function is perfectly symmetric about zero; therefore, this property must exist.

The “tail” of the function from –infinity to u must be equivalent to the “tail” of the
function from +infinity to –u.
Gaussian Normal Density and Distribution
1
pdf
PDF
X: 2
Y: 0.9772
0.9
X: 1
Y: 0.8413
0.8
0.7
0.6
X: 0
Y: 0.5
0.5
X: 0
Y: 0.3989
0.4
0.3
X: -1
Y: 0.242
X: 1
Y: 0.242
0.2
0.1
0
-8
-6
-4
-2
0
2
4
6
8
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Spring 2015
6 of 20
ECE 3800
Another defined function that is related to the Gaussian (and used) is the Q-function,
this is the probability “from the tail of the Gaussian” see Appendix E.:
Q x  

 u2 
  du
exp 


2
2


ux
1


The Q-function is the complement of the normal function, :
Q x   1    x 
Therefore note that:
Q x   1  Q x 
x X
FX  x   1  Q
 
x X

  
 







Gaussian Normal Density, Dist, and Q Function
1
0.9
0.8
0.7
0.6
pdf
Normal Function
Q Function
0.5
0.4
0.3
0.2
0.1
0
-8
-6
-4
-2
0
2
4
6
8
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Spring 2015
7 of 20
ECE 3800
An approximation for the Q function (provided in the text is) p. 70
 a2 

1 
1
,
 
Qa   1 
 exp 
 2 
 2  a 2  a  2


for x  a  3
The “tails of the Gaussian” and thereby the Q-function are used for events that happen very
rarely.
Text example: False detection in an IC trigger circuit for detecting “digital” signal levels.
Assume Gaussian noise of zero mean and 0.2V^2 variance.
Zero level: 0.5V plus noise
One level detection: 2.5 V.
Can or will and at what probability will the zero create a false one?
Zero-level equivalent: Gaussian with 0.5V mean and 0.2V^2 variance.
 2.5  0.5 
Prx  2.5  Q
  Q4.47   3.911  10 6  AppE 
0.2 

So, in a few msec of detection level for a gigahertz clocked system … say 100,000 clocks
what is the probability there has been a false trigger?

Pr FalseTrigg er   1  1  Pr x  2.5
100000
  0.321
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Spring 2015
8 of 20
ECE 3800
Another way to find values for the Gaussian
The error function, defined as
erf  x  
Q x  
FX x   1 
2

2
 exp u  du
x

u 0
1 
 x 
 1  erf 

2 
 2 
 x X
1 
 1  erf 
2 
 2 
 xX
 1 1
    erf 
 2 


 2 2




The error function (Y = ERF(X)) is built-in to MATLAB. Appendix G provides an overview of
the functions and how they can be used within MATLAB.
Note: There is a typo in the definition for the erf on p. 441. The integration should be as shown
above, not the integral from x to infinity.
From MATLAB:
ERF Error function.
Y = ERF(X) is the error function for each element of X. X must be
real. The error function is defined as:
erf(x) = 2/sqrt(pi) * integral from 0 to x of exp(-t^2) dt.
See also erfc, erfcx, erfinv.
Reference page in Help browser
doc erf
Central Limit Theorem
The normalized sum of a large number of independent variables, having the same probability
density function, has a probability density function that approaches a Gaussian density function.
 1 n

1 n

Y  n       X k   X  
  X k  X 
n
n

1
k
k 1






Y  0 and E Y  Y    x2
2
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Spring 2015
9 of 20
ECE 3800
Example: Intelligence Quotient (IQ)
For IQ Tests:  IQ  100 and  IQ  15
For an IQ test you’ve taken, the particular test has a raw score mean and standard deviation of
 raw  25 and  raw  4
If your score was x  28 , what upper percent are you a member of
(i.e., you would be in the top XX%)?
The “translation factor” for the normal distribution involves:
z
x

First find the normal distribution value describing your performance:
z
28  25
 0.75
4
From the table in appendix D
x X
FX  x   
 

  0.75  0.7734


To determine the upper percentile …
1  FX  x   1  0.7734  0.2266
What would be your equivalent score on a 100 point test?
z
x IQ   IQ
28  25
 0.75 
 IQ
4
x IQ  z   IQ   IQ  0.75  15  100  0.75  111.25
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Spring 2015
10 of 20
ECE 3800
Histograms
Providing a means of showing the results of repeated experiments match the expected probability
density function (or probability mass function) that was used (or expected).
x
 f pdf u   du
FX x  
The pdf

FX x  
The pmf
x
 f pmf u 
u  
To translate from a continuous distribution to a bin-based discrete distribution, take the area of
the density curve within the range of a bin that you wish to define and equate it to the probability
mass function at the center of the bin …
Let x be divided into bins so that bin i ranges from
bin
bin
 x i  i  bin 
i  bin 
2
2
From pdf

k  B  B


2
i

F X i  
f p u   du 


k   

B

 k  B  2


k B 
Let
 fBX u   du
f pmf k  
k B 
F X i  
Then
B
2
2
i
  f pmf k 
k  
What would a first order approximation for the pmf elements be?
The bin center times the bin width.
k B 
B
2
 fBX u   du  f X k  B   B  f pmf k 
k B 
2
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Spring 2015
11 of 20
ECE 3800
Turning a distribution into bins:
Assume that a random variables probability density function is triangular and can be described as
0,
1  x,

f pdf  x   
1  x,
0,
for    x  1
for  1  x  0
for 0  x  1
for 1  x  
Use 5 bins located.
The bin width is 2/5=0.4
The bin centers are at –0.8, -0.4, 0.0, 0.4, and 0.8.
Compute the pmf elements ….
f pmf 1 
f pmf 2 
 0. 6
0.6
1
1.0

u2
1  u  du  u 
2
0.2
0.2
 0.6
 0.6

u2
1  u  du  u 
2
f pmf 3 
0
0.2
 0.2
0
1  u  du  
0.36  
1

   0. 6 
    1    0.08
2  
2

0.04  
0.36 

   0.2 
    0.6 
  0.24
2  
2 

u2
1  u  du  u 
2
0
 0.2
u2
u
2
0.2
0
0.04  
0.04 

f pmf 3    0.2 
   0.2 
  0.36
2  
2 

f pmf 4  f pmf 2  0.24
f pmf 5  f pmf 1  0.08
An a-priori pmf derived from the pdf.
Now for an experiment … if I ran 1000 experiments, how many results would I expect to reside
in each bin?
1000 * f pmf i 
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Spring 2015
12 of 20
ECE 3800
Computer Examples
See
GaussianGen.m
GaussianRandn
TriGen.m
TriRand.m
ExponentialGen.m
If you know the PDF, you can use an inverse process to generate random values.
F x   fn x 
Then
x  fn 1 F  x 
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Spring 2015
13 of 20
ECE 3800
Example 4.11 Binary Communications
From: Alberto Leon-Garcia, “Probability, Statistics, and Random Processes For Electrical
Engineering, 3rd ed.”, Pearson Prentice Hall, 2008, ISBN: 013-147122-8.
A binary transmission system sends a “0” bit by transmitting a –v voltage signal, and a “1” bit by
transmitting a +v. The received signal is corrupted by Gaussian noise and given by:
Y  X N
where X is the transmitted signal, and N is a noise voltage with pdf f N x  . Assume that
P[“1”] = p = 1 – P[“0”]. Find the pdf of Y.
Let B0 be the event “0” is transmitted and B1 be the event “1” is transmitted, then form a
partition, and
FY x   FY  x | B0   PB0   FY  x | B1   PB1 
FY  x  PY  x | X  v  1  p  PY  x | X  v  p
Since Y  X  N the event Y  x | X  v is equivalent to v  N  x | and N  x  v | , and
the event Y  x | X  v is equivalent to N  x  v | . Therefore the conditional cdf’s are
FY  x | B0   PN  x  v |  FN x  v 
FY x | B1   PN  x  v |  FN x  v 
The cdf is then
FY x   FN  x  v   1  p   FN  x  v   p
The pdf is then
fY x 
f Y  x 
d
FY  x
dx
d
d
FN x  v  1  p  FN x  v  p
dx
dx
f Y  x   f N  x  v   1  p   f N  x  v   p
The Gaussian random variable has pdf:
f N x 
 x2 

 exp 
2 ,
2

2

2  


1
for    x  
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Spring 2015
14 of 20
ECE 3800
The conditional pdfs are
  x  v 2 

 exp 
f N x | B0   f N x  v  
2 
2   2
 2  
  x  v 2 
1

 exp 
f N x | B1   f N x  v 
2 
2

2

2  


1
The pdf of the received signal Y is then:
  x  v 2 
 x  v 2 
1


 p
 exp 
 1  p  
 exp 
f Y x 
2 
2 
2
2


2

2

2  
2  




1
A detection threshold, TH, can be established based on the pdf. Then, the probabilities of
correctly detecting the transmitted signals can be computed. as X>TH and X<TH.
2.6
Density Functions Related to Gaussian
Rayleigh Distribution – 2D Gaussian
Maxwell Distribution – 3D Gaussian
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Spring 2015
15 of 20
ECE 3800
Rayleigh Distribution
For a two dimensional problem (positions in x and y with two independent Gaussian random
variable noise or offset terms), the distance from a desired point is described as a radial or vector
magnitude, the radial error or offset is described by the Rayleigh Distribution.
R
For
X 2 Y 2
The probability density function (pdf) is
  r2 
,
 exp
2
2


 2  
 0,
f R r  
r
for 0  r
for r  0
The probability distribution function (PDF) can be derived as:
r
  v2 
  dv,
 exp
 2  2 
2


v 0
 0,
FR r  

v
for 0  r
for r  0
r
FR r 
  r2 
 ,
  exp
 2  2 

0
for 0  r
 0,
for r  0
  r2 
,
FR r   1  exp
 2  2 


 0,
for 0  r
for r  0
http://en.wikipedia.org/wiki/Rayleigh_distribution
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Spring 2015
16 of 20
ECE 3800
The first moment
R  E R  


r
r 0
  r2 
  dr
 exp
2
2


 2  
r

x
Closed form solution (p. 421)
2


 exp  a 2  x 2  dx 
r 0
R  E R  
1

2


4


2 
3 

2
 

2

4  a3

The second moment

   r
2
R ER
2
2

r 0
  r2 
  dr
 exp
2
2


 2  
r
u  v 
u
u
 x   v   x  u  v   v   x
x
x
x

2
 r
  r 
  dr
R 2  E R 2   r 2   2  exp
2 


2



r 0

2
 r 
v
r

 2  exp
u  r 2 and
2 
x 
 2  
v
 u  x  x  
Math tables hint
 
 
  r2 
u

 2  r and v   exp
2 
x
 2  
and keep going until you get ….
 
R2  E R2  2  2
The second central moment, variance or standard deviation is

 
  

 2
2


E  r  R    R2  R 2  E R 2   2     2


2


E r  R




  R2  R 2  E R 2  2   2  

2

2

Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Spring 2015
17 of 20
ECE 3800
Example: Archery target shooting with    X   Y  0.25 
1
4
Assume a 1 foot radius target with a 1 inch radius Bulls-eye
  r2 
r
, for 0  r
f R r  
 exp
2
2


 2  
for r  0
 0,
FR r 
  r2 
,
 1  exp
 2  2 


 0,

for 0  r
for r  0

f R r   16  r  exp  8  r 2 ,
 0,


FR r   1  exp  8  r 2 ,
 0,
R  E R  


r
r 0

for 0  r
for r  0
for 0  r
for r  0
  r2 
  dr    1    0.313
 exp
 2  2 
2 4
32
2


r

2
2
1 

  1


E  r  R    R2   2     2   2       
 0.0268


2
2   4
8 32




2
1 
E r  R    R 

 0.164


8 32
Probability of a Bulls-eye (1 inch radius)

1 2 
1
 8 
FR    1  exp  8 
 1  exp 
  0.0540


12
144
 12 




Probability of missing the target (1 foot radius)



1  FR 1  1  1  exp  8  12  exp 8  3.35 10 4
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Spring 2015
18 of 20
ECE 3800
Maxwell Distribution
For a three dimensional problem (the velocity of a particle in a perfect gas), the velocity vector
magnitude is described by the Maxwell Distribution.
V  V X 2  VY 2  VZ 2
For
The probability density function (pdf) is
  v2
2 v2
f V v  
 3  exp
2
 
 2 
 0,

,

The probability distribution function (PDF) can be derived as:
v
  x2 
2 x2
  dx,
FV v   
 3  exp
2 
 
 2  
x 0
for 0  v
for v  0
for 0  v
 0,
for v  0
The solution requires numerical integration!
The first moment
V  E V  


  x2 
2 x2
8
  dx 

 3  exp
2 
 

 2  
x
x 0
The second moment

  x
V2  EV2 
x 0
2

  x2 
2 x2
  dx
 3  exp
2 
 
 2  
 
V 2  E V 2  3  2
The second central moment, variance or standard deviation is
2
 8

2
  
E  v  V    V2  V 2  E V 2  3   2  


 

2
8

E  v  V    V2  V 2  E V 2   3     2  0.453   2








http://en.wikipedia.org/wiki/Maxwell-Boltzmann_distribution
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Spring 2015
19 of 20
ECE 3800
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Spring 2015
20 of 20
ECE 3800