Section 6.2: Generating Discrete Random Variates
is the function
is the cdf of
The inverse distribution function (idf) of
:
for all
as
➤
is the smallest possible value of
for which
That is, if
,
is greater than
➤
Section 6.2: Generating Discrete Random Variates
1
+&
'
,-$ *
!
"%
may be
/
.
!
.
#"!
.$
()& '*
+&
'
,-$ *
!
!
#"!
.$
()& '*
where
and
Theorem 6.2.1 Let
be the cdf of
➤
Two common ways of plotting the same cdf with
"%
➤
Example 6.2.1
,
For any
if
,
else
for which
is the unique possible value of
where
12 ✦
0
✦
Section 6.2: Generating Discrete Random Variates
2
Algorithm 6.2.1
For
defines
➤
3
x = a;
while (F(x) <= )
x++;
return x; /*x is
, the following linear search algorithm
4 65
37
*/
Average case analysis:
Let
be the number of while loop passes
8
➤
Section 6.2: Generating Discrete Random Variates
0
<
0
;
9:
9: 0 ;
0
✦
9:
✦
8; 8
✦
3
Algorithm 6.2.2
Idea: start at a more likely point
For
defines
➤
➤
, a more efficient linear search algorithm
4 65
37
x = mode; /*initialize with the mode of X */
if (F(x) <= u)
while (F(x) <= u)
x++;
else if (F(a) <= u)
while (F(x-1) > u)
x--;
else
x = a;
return x; /* x is
*/
➤
, consider binary search
For large
Section 6.2: Generating Discrete Random Variates
4
Idf Examples
0=
:
0=
0=
1 Section 6.2: Generating Discrete Random Variates
iff
, then
and
=
is Bernoulli
If
➤
can be determined explicitly
In some cases
➤
5
Section 6.2: Generating Discrete Random Variates
@
A
,
0
0
0
1 0
>?
0 0
1
>?
1
0 0
is Equilikely
0
Therefore, for all
➤
For
12 ➤
0
For
➤
If
Example 6.2.3: Idf for Equilikely
,
,
6
For all
,
Section 6.2: Generating Discrete Random Variates
E
E =
0
1
>?
12 0=
>?
B
1
0=
0
C
1
D
0=
B
C
=
is Geometric
EF
GE =
0
H
➤
For
➤
For
➤
If
Example 6.2.4: Idf for Geometric
,
,
..
7
is a discrete random variable with idf
Random Variate Generation By Inversion
is Uniform
Theorem 6.2.2:
and
J
✦
I
Theorem 6.2.2 allows any discrete random variable (with known idf)
to be generated with one call to Random()
Algorithm 6.2.3: If is a discrete random variable with idf
random variate can be generated as
,a
➤
Proof: next slide
➤
are identically distributed
➤
J
is the discrete random variable defined by
➤
I
Continuous random variable
J
➤
➤
u = Random();
(u);
return
Section 6.2: Generating Discrete Random Variates
8
K
K
It follows that
so
From definition of , if
:
It follows that
so
K
✦
M
J
L
such that
, so
I
:
✦
Prove that
➤
Proof for Theorem 6.2.2
N
K
N
J
N such that
L
then
and
R
I
PO
J
0
N
R
N
N
N
0
0
J
Section 6.2: Generating Discrete Random Variates
I
N
PO
,
K
N N
S
PO
N
QPO
and
J
, from definition of
N
if
✦
K
✦
Let
theorem 6.2.1:
if
,
have the same pdf
and
J
Prove that
➤
12 ✦
MK
✦
9
Inversion Examples
with pdf
T T T
V U V U D
Example 6.2.5 Consider
R
➤
]
ga
^ `
Section 6.2: Generating Discrete Random Variates
[ g
ZY
\
WX
^ ]^ ]
] `
^ ]
Y
deb cf
[ g
ZY
\
WX
_^] _^]
] `
_^]
Y
deb cf
ga
]
is plotted using two formats
^ `
The cdf for
10
Algorithm for Example 6.2.5
u = Random();
if (u < 0.1)
return 2;
else if (u < 0.4)
return 3;
else
return 6;
returns 2 with probability 0.1, 3 with probability 0.3 and 6 with
probability 0.6 which corresponds to the pdf of
Section 6.2: Generating Discrete Random Variates
D
Problems may arise when
h
➤
h
U
This example can be made more efficient: check the ranges for
associated with
first (the mode), then
, then
V
➤
is large or infinite
11
More Inversion Examples
random variate
Example 6.2.6 Generate a Bernoulli
=
➤
u = Random();
if (u < 1-p)
return 0;
else
return 1;
Example 6.2.7 Generate a Equilikely
random variate
➤
u = Random();
return a + (long) (u * (b - a + 1));
random variate
Example 6.2.8 Generate a Geometric
=
➤
u = Random()
return (long) (log(1.0 - u) / log(p));
Section 6.2: Generating Discrete Random Variates
12
Example 6.2.9
random variate
D
i
o
n
0=
lkjm
=
i
is a Binomial
i
=
➤
=
i i
0
0
0
q
i
Incomplete beta function
p
➤
Inversion is not easily applied to generation of Binomial
random variates
Section 6.2: Generating Discrete Random Variates
i
=
➤
Except for special cases, an incomplete beta function cannot be
inverted to form a “closed form” expression for the idf
13
Algorithm Design Criteria
The design of a correct, exact and efficient algorithm to generate
corresponding random variates is often complex
✦
✦
✦
✦
✦
✦
✦
Portability - implementable in high-level languages
Exactness - histogram of variates should converge to pdf
Robustness - performance should be insensitive to small
changes in parameters and should work properly for all
reasonable parameter values
Efficiency - it should be time efficient (set-up time and marginal
execution time) and memory efficient
Clarity - it is easy to understand and implement
Synchronization - exactly one call to Random is required
Monotonicity - it is synchronized and the transformation from to
is monotone increasing (or decreasing)
➤
➤
Inversion satisfies some criteria, but not necessarily all
Section 6.2: Generating Discrete Random Variates
14
, the pdf is (to
precision)
Cm
|
{
y
x
v
u
C
t 
€m
mm
y
€m
mw
m
€m
Cu
C
€m
uC
x
€m
ux
C
€m
um
C
€m
CC
C
€m
mw
u
€m
mC
C
€m
mm
u
€m
mm
m
}~
t
m
w
z
T
s
To generate Binomial
T r
➤
ss
Example 6.2.10
Random variates can be generated by filling a 1000-element
with 6 0’s, 40 1’s, 121 2’s, etc.
integer-valued array
:;
➤
j = Equilikely(0,999);
return a[j];
VD
ƒ ‚
V…„
T ss
Set-up time and memory efficiency could be problematic:
precision, need 100 000-element array
for
ss
➤
The algorithm is not exact:
s
➤
This algorithm is portable, robust, clear, synchronized and
monotone, with small marginal execution time
R
➤
Section 6.2: Generating Discrete Random Variates
15
➤
Œ
‰Šˆ
‹Šˆ
‡†
Example 6.2.11: Exact Algorithm for Binomial
An exact algorithm is based on
filling an 11-element floating-point array with cdf values
then using Alg. 6.2.2 with
to initialize the search
by inversion
In general, to generate Binomial
i
=
compute a floating-point array of
cdf values
use Alg. 6.2.2 with
to initialize the search
A
=
➤
i
The library rvms can be used to compute the cdf array by calling
cdfBinomial(n,p,x) for
➤
✦
i@
✦
i
➤
✦
r
✦
Only drawback is some inefficiency (setup time and memory)
Section 6.2: Generating Discrete Random Variates
16
Example 6.2.12
➤
The cdf array from Example 6.2.11 can be eliminated
✦
✦
✦
cdf values computed as needed by Alg. 6.2.2
Reduces set-up time and memory
Increases marginal execution time
Function idfBinomial(n,p,u) in library rvms does this
➤
Binomial
random variates can be generated by inversion as
i
=
➤
u = Random();
return idfBinomial(n, p, u);
Inversion can be used for the six models:
Section 6.2: Generating Discrete Random Variates
=
=
<
✦
i =
Inversion is ideal for Equilikely
, Bernoulli and Geometric
For Binomial
, Pascal
and Poisson , time and
memory efficiency can be a problem if inversion is used
✦
i
=
➤
/* use the library rvms*/
17
Alternative Random Variate Generation Algorithms
Example 6.2.13 Binomial Random Variates
random variate can be generated by summing an
sequence
= i =
A Binomial
iid Bernoulli
➤
x = 0;
for (i = 0; i < n; i++)
x += Bernoulli(p);
return x;
The algorithm is: portable, exact, robust, clear
➤
The algorithm is not: synchronized or monotone
➤
Marginal execution:
complexity
i Ž
➤
Section 6.2: Generating Discrete Random Variates
18
Poisson Random Variates
random variable is the
random variable
The Poisson
cdf
➤
An incomplete gamma function cannot be inverted to form an idf
should not be used
is equal to an incomplete gamma function
0
D
‘
<
<
i
algorithm for Binomial
i
=
➤
➤
i Ž
The previous
when is large
/
i
limiting case of a
i
<i‚
Binomial
< 
For large , Poisson
i
➤
A Poisson
Binomial
i
<
<i‚ ➤
Inversion to generate a Poisson
Examples 6.2.11 and 6.2.12
Section 6.2: Generating Discrete Random Variates
requires searching the cdf as in
<
➤
19
Example 6.2.14
random variates:
<
An algorithm to generate Poisson
➤
’
a = 0.0;
x = 0;
while (a < ) {
a += Exponential(1.0);
x++;
}
return x - 1;
i
The algorithm does not rely on inversion or the “large ” version of
Binomial
i
=
➤
➤
The algorithm is not: synchronized or monotone; marginal
execution time can be inefficient for large
<
➤
The algorithm is: portable, exact, robust
➤
It is obscure. Clarity will be provided in Section 7.3
Section 6.2: Generating Discrete Random Variates
20
Pascal Random Variates
cdf is equal to an incomplete beta function:
n
n
C
u
C
sequence
where
random variates to
=
Example 6.2.15 Summing Geometric
random variate:
generate a Pascal
i
=
➤
=
is Pascal
iff
is an iid Geometric
i
=
➤
u
D
q
i
=
0
A Pascal
i
=
➤
x = 0;
for(i = 0; i < n; i++)
x += Geometric(p);
return x;
The algorithm is: portable, exact, robust, clear; not synchronized
or monotone; marginal execution complexity is
i Ž
➤
Section 6.2: Generating Discrete Random Variates
21
Library rvgs
Includes 6 discrete random variate generators (as below) and
7 continuous random variate generators
✦
✦
➤
➤
=
=
=
✦
=
✦
Bernoulli(double )
Binomial(long , double )
Equilikely(long , long )
Geometric(double )
Pascal(long , double )
Poisson(double )
<
✦
long
long
long
long
long
long
i
✦
i
➤
Functions Bernoulli, Equilikely, Geometric use inversion;
essentially ideal
Functions Binomial, Pascal, Poisson do not use inversion
Section 6.2: Generating Discrete Random Variates
22
Library rvms
➤
Provides accurate pdf, cdf, idf functions for many random variates
➤
Idfs can be used to generate random variates by inversion
➤
➤
➤
Functions idfBinomial, idfPascal, idfPoisson may have
high marginal execution times
Not recommended when many observations are needed due to
time inefficiency
Array of cdf values with inversion may be preferred
Section 6.2: Generating Discrete Random Variates
23