Chapter 2: Random Variables
2.1. Concept of a Random Variable
2.2. Distribution Functions
2.3. Density Functions
Functions of random variables
2.4. Mean Values and Moments
2.5. The Gaussian Random Variable
2.6. Density Functions Related to Gaussian
2.7. Other Probability Density Functions
2.8. Conditional Probability Distribution and Density Functions
2.9. Examples and Applications
Hypergeometric Distribution
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Fall 2014
1 of 20
ECE 3800
Mean Values and Moments
Mean Value: the expected mean value of measurements of a process involving a random
variable.
This is commonly called the expectation operator or expected value of …
and is mathematically described as:
X  E X  

 x  f x   dx
X

X  EX  

 x  f X x  
x  

 x  Pr  X  x 
x  
For laboratory experiments, the expected value of a voltage measurement can be thought of as
the DC voltage.
General concept of an expected value
In general, the expected value of a function is:
E g  X  

 g  X   f x   dx
X

E g  X  

 g  X   f X x  
x  

 g  X   Pr  X  x 
x  
If we know the expected value, you have a simple estimate of future expected outcomes.
xˆ  X  E X 
Or for y  g  x 
yˆ  E y   Eg  X 
Moments
The moments of a random variable are defined as the expected value of the powers of the
measured output or …

   x
X EX
n
n
n
 f X  x   dx


   x
X EX
n
n
n
 Pr  X  x 
x  
Therefore, the mean or average is sometimes called the first moment.
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Fall 2014
2 of 20
ECE 3800
Expected Mean Squared Value or Second Moment
The mean square value or second moment is

   x
X EX
2
2
 f X  x   dx
2


   x
X EX
2
2
2
 Pr  X  x 
x  
Central Moments
The central moments are the moments of the difference between a random variable and its mean.

X  X 
n
    x  X 

E XX
n
 f X  x   dx
n


X  X 
n
E XX
    x  X 

n
n
 Pr  X  x 
x  
Notice that the first central moment (n=1) is 0 …
The second central moment is referred to as the variance of the random variable …


2
  XX
2

    x  X 

E XX
2
 f X  x   dx
2


2  X X

2

E XX
    x  X 
2

2
 Pr  X  x 
x  
Note that:

   EX  X  X  X 
  E X   2  X  E X   X
  E X   X  E X   E X 
2 E X X
2
2
2
2
2
2
2
2  X2 X
2
2
2
The variance is equal to the 2nd moment minus the square of the first moment …
The variance is also referred to as the standard deviation.
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Fall 2014
3 of 20
ECE 3800
Example: Text offset uniform density
  x  20
0
1

f X x   
 20
0
20  x  40
40  x  
Mean Value (average DC value)
X  E X  

x f
X
x   dx

X  EX  
40

20
40

1
1
 x  dx
x   dx 
20
20
20
1 x2
X  E X  

20 2
X  E X  
40
20
1  40 2 20 2  1  1600 400  1  1200 






  30

20  2
2  20  2
2  20  2 
Mean Square

   x
X EX
2
2
 

20
 
X EX
2
1 x3


20 3
 
X 2  E X 2 
40

20
 f X  x   dx

40
X2 E X2 
2
2
x2 
1
 dx
20
1  40 3 20 3  1  4 3 2 3  10 3


     1
20  3
3  2 3
3  10
1  64 8  10 3 56
 10 2  933.3
   1 
2  3 3  10
6
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Fall 2014
4 of 20
ECE 3800
Variance or Standard Deviation (average AC value squared)

  XX
2


2
    x  X 

E XX
2
2
 f X  x   dx




2
2  X X
E XX
    x  30
2
40
2

20

2  X X

2

2  X X
2

E X X
    x
1
 dx
20

40
2
2
 60  x  900 
20
   933.3  60  30  900 1
 X  X   E X  X    33.3

2

1
 dx
20
E XX
2
2
2
or
2  X2 X
2
 2  933 .3  30 2  933.3  900  33.3
The variance or stand deviation is then
  33.3  5.77
The bulk of the density function appears at in the range defined by 30+/-5.77.
Pr X    X  X    
X 
 f x   dx
X 
Pr X    X  X    
X 
1
 20  dx 
X
X     X     2  
20
20


10
 0.577
Pr X  2    X  X  2     1.0
The meaning of standard deviation:





It has the same units as the “mean” (e.g. if x is a voltage, the variance is proportional to
power, but the standard deviation is in voltage again).
If small, all values are close to the mean.
If large, the values are widely spread out.
It is typical in sciences to plot means with “error bars” related to the standard deviation.
Usually we are looking at +/- one standard deviation, but +/- 2 std dev may also be of
interest (higher probability the point lie within the range).
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Fall 2014
5 of 20
ECE 3800
Three Coins
Experiment: Flip two Coins and count the number of heads
S Pair  TTT , HTT , THT , TTH , HHT , HTH , THH , HHH 
S  0,1,2,3
 n
Pr  A occuring k times in n trials   p n k     p k  q n k
k 
And the probability mass function, f X  x   Pr  X  x  , is then
1 ,
 8
3 ,
 8

f X x    3 ,
8

 18 ,

0,

For FX  x   Pr  X  x 
for x  0
for x  1
for x  2
for x  3
else
for x  0
0,

 18 ,

FX  x    4 ,
8

7
 8,

1,
for 0  x  1
for 1  x  2
for 2  x  3
for 3  x
Mean Value
X  EX  

 x  Pr X  x
x  
3
X  E  X    x  Pr  X  x 
x 0
3
1 3  6  3 12
1
3
X  E X   0   1   2   3  

 1 .5
8
8
8
8
8
8
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Fall 2014
6 of 20
ECE 3800
Second Moment (mean squares)

X
2
   x
EX
2
2
 Pr  X  x 
x  
 
1
3
3
1 3  12  9  24
X 2  E X 2  0 2   12   2 2   3 2  

3
8
8
8
8
8
8
Variance and standard deviation
2  X2 X
2
 2  3  1.5 2  3  2.25  0.75
  0.866
The majority of the density function appears at 1.5+/-0.866.
Pr X    X  X    
X 
 Pr  X  x 
x  X 
Pr  X    X  X      Pr  X  x  
2
x 1
3 3 6
   0.75
8 8 8
2 standard deviations contains the entire density function …
Pr X  2    X  X  2     1.0
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Fall 2014
7 of 20
ECE 3800
Triangular density function
Assume that a random variables probability density function is triangular and can be described as
for    x  1
for  1  x  0
for 0  x  1
for 1  x  
0,
1  x,

f X x   
1  x,
0,
Find the probability distribution function.
FX  x  
The definition
x
 f v   dv
X
v  
For
FX  x   0
   x  1
For  1  x  0
FX x  
x
 1  v  dv
v  1
x

v 2 

FX x   v 

2 

1

1  x2
1
x 2  
 1  
x
FX x    x 

2  
2 2
2

For 0  x  1
1
FX x   
2
x
 1  v  dv
v0
v2 
1 
FX x     v  
2 
2 
FX x  
For 1  x  
x
0
1 
1
x 2 
x2

x
 x
2 
2 
2
2
FX  x   1
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Fall 2014
8 of 20
ECE 3800
Therefore,
for    x  1
0,
 2
x  x  1 ,

2
FX x    2
1
 x2
 2  x  2 ,

1,
for  1  x  0
for 0  x  1
for 1  x  
1.2
1
0.8
0.6
0.4
0.2
0
-0.2
-2
-1.5
-1
-0.5
0
0.5
1
1.5
2
Mean Value (average DC value)
X  E X  

x f
X
x   dx

X  EX  
0
1
1
0
 x  1  x  dx   x  1  x  dx
0
1
 x 2 x3 
 x 2 x3 




 
X  E X  
 2

 2
3
3 

 1 
0
 1 1  1 1
X  E X           0
 2 3  2 3
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Fall 2014
9 of 20
ECE 3800
Mean Square

X
2
   x
EX
2
2
 f X  x   dx

   x
0
X
2
EX
2
1
2

 1  x   dx  x 2  1  x   dx
1
0
0
 
1
4
 x3 x 4 
 3
  x  x 

X2 E X2 
 3
 3
4 
4 

1 
0
 
 1 1 1 1
1 1 1
X 2  E X 2           2     
 3 4 3 4
3 4 6
Variance or Standard Deviation (average AC value squared)

  XX
2

2

    x  X 

E XX
2
2
 f X  x   dx

2  X2 X
2
2 
1
1
 02 
6
6

1
 0.408
6
Probability of being within a standard deviation
Pr X    X  X    
X 
 f x   dx
X 
 1 1

2

  2  
Pr X    X  X     2   1  x   dx  2    
 6  12   0.650
2


0



Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Fall 2014
10 of 20
ECE 3800
Black-Jack Card Point Count
The “value” in black-jack of cards is the value of the card for two through 9, 10 for 10’s and face
cards, and 11 for aces. What is the mean value of the cards in a deck?
And the probability mass function, f X  x   Pr  X  x  , is then
 4 , for x  2,3,4,5,6,7,8,9
 52
16 , for x  10
f X  x    52
 4 , for x  11
 52
0,
else
Mean Value
X  EX  

 x  Pr X  x
x  
4
X  E X  

52
9
 k  52 10  52 11 
16
4
4  44  16  10  4  11  380  7.31
52
k 2
52
Second Moment (mean squares)
 
X2 E X2 

x
2
 Pr  X  x 
x  
 
X2 E X2 
4

52
9
 k 2  52 10 2  52 112 
16
4
4  284  16  100  4  121  3220  61.92
52
k 2
52
Variance
2  X2 X
2
 2  61.92  7.312  8.52
  2.92
So, if you want another card :
E X     7.31  2.92
Pr X    X  X    
X 
 Pr  X  x 
x  X 
Pr  X    X  X      Pr  X  x  
10
x 5
5  4  16 36

 0.69
52
52
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Fall 2014
11 of 20
ECE 3800
What if we originally assumed that an Ace was worth 1 instead of 11 ?
4 ,
 52
 4 ,
f X  x    52
16 ,
 52
0,
for x  1
for x  2,3,4,5,6,7,8,9
for x  10
else
Mean Value
X  EX  

 x  Pr X  x
x  
X  E X  
4  44  16  10  4  1  340  6.54
4
4 9
16
k 
 10 
1 
52
52
52
52 k  2
52
Second Moment (mean squares)

   x
X EX
2
2
2
 Pr  X  x 
x  
 
X2 E X2 
4 2 4  284  16  100  4  1 2740
4 9 2 16
k 
 10 2 
1 

 52.69
52
52
52
52 k  2
52
Variance
2  X2 X
2
 2  52.69  6.54 2  9.94
  3.15
(a larger standard deviation due to the high number of cards at one extreme value)
So, if you want another card :
E X     6.54  3.15
Pr X    X  X    
X 
 Pr  X  x 
x  X 
Pr  X    X  X      Pr  X  x  
9
x4
6  4 24

 0.46
52
52
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Fall 2014
12 of 20
ECE 3800
Three Coins Betting …. a function of probability!
Experiment: Flip two Coins and count the number of heads, g(x) is winnings or lose
1 ,
 8
3 ,
 8

f X x    3 ,
8

 18 ,

0,

for x  0
for x  1
for x  2
let
for x  3
else
10,
 4,

g  x   5,
 12,

0,
for x  0
for x  1
for x  2
for x  3
else
Expected value of winnings or loss each bet ….

E g  X  
 g x  Pr X  x
x  
E g  X  
3
 g x  Pr X  x
x0
1
3
3
1 10  12  15  12  1
X  E X   10   4   5   12  
  0.125
8
8
8
8
8
8
On average you win 0.125 every time we play the game ….
All gambling games can be computed this way. There are the “true odds” for the game and then
the payout for the game’s outcome.

Guess what, probability will always say that “the house” wins!

Gamblers are always “hoping to beat the odds”!
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Fall 2014
13 of 20
ECE 3800
Roulette Wheel: Red/Black Betting …. a function of probability!
Experiment: Bet Red on a roulette wheel …
18 ,
 38
18 ,

f X  x    38
2 ,
 38
0,
for x  Re d
for x  Black
let
for x  Green
else
for x  Re d
for x  Black
for x  Green
1,
 1,

g x   
 1,
0,
else
Expected value of winnings/loss each bet ….
E g  X  

 g x   Pr X  x 
x  
X  E X   1 
18
18
2
2
 1  1

 0.0526
38
38
38
38
Typically for roulette, payouts are based on 36 (or less) not the actual 38 possible outcomes.
If you bet on one number … payout is $35
1 ,
 38
 1 ,
f X  x    38
1 ,
 38
0,
for x  1 : 36
for x  0
let
for x  00
35,

g  x    1,
0,

for x  yournumber
for x  antythingelse
else
else
E g  X  

 g x   Pr X  x 
x  
X  E X   35 
1
37
2
 1

 0.0526
38
38
38
So, as a generality, “the house” receives more than a nickel for every dollar bet!
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Fall 2014
14 of 20
ECE 3800
Arbitrary offset, uniform density … use variables instead of values.
Offset uniform density
for x  a
0,
 1

, for a  x  b
f X x   
b  a
for b  x
0,
Mean Value (average DC value)

X  E X  
x f
X
x   dx

b

X  EX  
x
1
 dx
ba
a
b
 1
x 2 

X  EX   
ba 2 

a
X  EX  
1  b 2 a 2  b  a   b  a  b  a 




2 
2  b  a 
2
b  a  2
Mean Square

X
2
   x
2
 f X  x   dx
  x
2

EX
2

b
X2 E X2 
1
 dx
ba
a
 
1  x 3 

X2 E X2 
b  a  3 
 
X2 E X2 
3
b
a

1  b
a  1 b a
1
 
  b2  a  b  a2




ba  3
3  3 ba
3
3
3
3

variance on the next page ….
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Fall 2014
15 of 20
ECE 3800
Variance or Standard Deviation (average AC value squared)

  XX
2


2
    x  X 

E XX
2
2
 f X  x   dx

2  X2 X

2

2

1
 b  a  
  b2  a  b  a2  

3
 2 
1
3
1
4
1
3
1
2
2
1
3
1
4
 2  b2      a  b      a2    
2 
b 2  2  a  b  a 2 b  a 

12
12
2
Application
b
For a unit uniform variance pdf
1
1
and a  
2
2

0,


f X  x   1,


0,

X  E X  
X2 
for x  
for 
for
1
2
1
1
x
2
2
1
x
2
b  a   0.5  0.5  0

2
2

1
1
2
 0.5 2  0.5  0.5   0.5 
3
12
2 
b  a 2
12

0.5  0.52
12

1
12
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Fall 2014
16 of 20
ECE 3800
Hypergeometric Distribution
From: http://en.wikipedia.org/wiki/Hypergeometric_distribution
In probability theory and statistics, the hypergeometric distribution is a discrete probability
distribution (probability mass function) that describes the number of successes in a sequence of n
draws from a finite population without replacement.
A typical example is the following: There is a shipment of N objects in which D are defective.
The hypergeometric distribution describes the probability that in a sample of n distinctive objects
drawn from the shipment exactly x objects are defective.
 D  N  D

   
x   n  x 

Pr  x  X , N , D, n  
N
 
n
for
max0, D  n  N   x  minn, D 
The equation is derived based on a non-replacement Bernoulli Trials …
Where the denominator term defines the number of trial possibilities, the 1st numerator term
defines the number of ways to achieve the desired x, and the 2nd numerator term defines the
filling of the remainder of the set.
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Fall 2014
17 of 20
ECE 3800
Quality Control Example
A batch of 50 items contains 10 defective items. Suppose 10 items are selected at random and
tested. What is the probability that exactly 5 of the items tested are defective?
The number of ways of selecting 10 items out of a batch of 50 is the number of combinations of
size 10 from a set of 50 objects:
 50 
50!
C1050    
 10  10!40!
The number of ways of selecting 5 defective and 5 nondefective items from the batch of 50 is the
product N1 x N2 where N1 is the number of ways of selecting the 5 items from the set of 10
defective items, and N2 is the number of ways of selecting 5 items from the 40 nondefective
items.
 10!   40! 
C 510  C 540  


 5!5!   5!35! 
Thus the probability that exactly 5 tested items are defective is the desired ways the selection can
be made divided by the total number of ways selection can be made, or
C C
C1050
10
5
40
5
 10!   40! 



5
!
5
!
5
!

35
!






 50! 


 10!40! 

252  658008
 0.01614
10272278170
Another Use: From “The Minnesota State Lottery” – a better description than Michigan
There are N objects in which D are of interest. The hypergeometric distribution describes the
probability that in a sample of n distinctive objects drawn from the total set exactly x objects are
of interest.
Lotteries …
N= number of balls to be selected at random
D = the balls that you want selected
n = the number of balls drawn
x = the number of desired balls in the set that is drawn
https://www.mnlottery.com/games/figuring_the_odds/hypergeometric_distribution/
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Fall 2014
18 of 20
ECE 3800
Example: Michigan’s Classic Lotto 47
Prize Structure For Classic Lotto 47 (web site data)
Match
Prize
Odds of Winning
6 of 6
Jackpot
1 in 10,737,573
5 of 6
$2,500 (guaranteed)
1 in 43,649
4 of 6
$100 (guaranteed)
1 in 873
$5 (guaranteed)
1 in 50
3 of 6
Overall Odds: 1 in 47
Matlab Odds
Match
Odds of Winning 1 in
Percent Probability
6 of 6
10737573
<1x10-5%
5 of 6
43648.7
0.0023%
4 of 6
872.97
0.1146%
3 of 6
50.36
1.9856%
2 of 6
7.07
14.1471%
1 of 6
2.39
41.8753%
0 of 6
2
Chance of winning 2.1024%
ROI per dollar without jackpot ~ $0.2711
41.8753%
see Matlab simulation “MI_Lotto.m”
Matlab Note: binomial coefficient = nchoosek(n,k)
Keno anyone? Another Michigan gambling game 80 balls, 20 drawn, you need to match k of n
selected for n=2 to 20.
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Fall 2014
19 of 20
ECE 3800
MI Keno
A Keno ticket with the payouts is shown!
Another hypergeometric density function
N= number of balls to be selected at random (80)
D = the balls that you want selected (D)
n = the number of balls drawn (20)
x = the number of desired balls in the set that is drawn (0:D)
MI Keno ROI and Pr[win]
1
ROI per $
Pr[win]
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
1
2
3
4
5
6
7
8
9
10
In general, you get $0.65 back for every $1 played. I did not include a “kicker” bet.
The overall odds of a Kicker (1, 2, 3, 4, 5, 10) number being 2 or higher are 1:1.67.
see MI_Keno.m
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Fall 2014
20 of 20
ECE 3800