Section 7.2: Generating Continuous Random Variates
The inverse distribution function (idf) of
:
for all
as
is the function
is the unique possible value for
There is a one-to-one correspondence between possible values
and cdf values
➤
where
➤
✦
Assumes the cdf is strictly monotone increasing
True if
for all
✦
Section 7.2: Generating Continuous Random Variates
1
Continuous Random Variable idfs
%%%
%%%
%%%
%% %
%%
%%
%% %
%%
%%
%%
%%%
%%
%%
%%
%%
%%
%%%
%%
%%
%%
%%
%%
%%
%%
%%
%%
%%
%%
%%%
%%
%%
%%
%%
%%
%%
%%
%%
%%
%%
%%%
%%
%%
%%
%%
%%
%%
%%
%%
%%%
%%
%%%
%%%
%%
%%
%%
%%
%%
%%
%%
%%
%%%
%%
%%
%
$
! "
#
Unlike the a discrete random variable, the idf for a continuous
random variable is a true inverse
➤
Can sometimes determine the idf in “closed form” by solving
for
➤
Section 7.2: Generating Continuous Random Variates
2
Examples
,
&
&
,
(
)
&
) *
,
&
&
.
1
0/
)
)
for
+
,
,
,
,
(
+
6
and pdf
Section 7.2: Generating Continuous Random Variates
,
,
(
7
5
(
+
6
(
)
,
3 2
4
)
.
. '
)
,
-
(
(
for
(
is a continuous variable with possible value
, the cdf is
If
➤
+
is Exponential
If
➤
,
is Uniform
If
& '
➤
3
➤
Theorem 7.2.1:
✦
and
8
9
:
;
are identically distributed
Proof: exercise
Algorithm 7.2.1: If
is a continuous random variable with idf
, a continuous random variate can be generated as
8
➤
is the continuous random variable defined by
;
➤
is Uniform
Continuous random variable
;
➤
:
is a continuous random variable with idf
➤
Random Variate Generation By Inversion
u = Random();
(u);
return
Section 7.2: Generating Continuous Random Variates
4
(
Example 7.2.4 Generate a Uniform
➤
& '
Inversion Examples
random variate
➤
Example 7.2.5 Generate a Exponential
. '
u = Random();
return a + (b - a) * u;
random variate
is Uniform
:
iff
)
is Uniform
Note:
:
)
.
u = Random();
* log(1 - u);
return
, so can also use
)
.
u = Random();
* log(u);
return
Because this algorithm has reverse monotonicity, the algorithm in
Example 7.2.5 is preferred
Section 7.2: Generating Continuous Random Variates
5
Example 6.2.4 and 6.2.5
➤
➤
➤
Both are portable, exact, robust, efficient, clear, synchronized and
monotone
It is not always possible to solve for a continuous random variable
idf explicitly by algebraic techniques
Two other options may be available
Use a function that accurately approximates
Determine the idf by solving
numerically
Section 7.2: Generating Continuous Random Variates
8
✦
✦
➤
Algorithms in Example 7.2.4 and 7.2.5 are ideal
6
➤
, the cdf is the special function
<
89
The idf
8
➤
➤
is a Normal
<
If
;
➤
Approximate Inversion
cannot be evaluated in closed form
The idf can be approximated as the ratio of two fourth degree
polynomials (Odeh and Evans, 1974)
The approximation is efficient and essentially has negligible error
Section 7.2: Generating Continuous Random Variates
7
=
A
?
@
>
J
,
I
,
,
I
I
J
I
I
)
)
,
JK
E
,
I
,
5
3 2
5
)
3 2
JK
I
E
E
H +
'
)
F
E
D
and
I
F G
E
)
E
<
idf approximation is
E
H +
'
D
<
B C
<
, a Normal
where
For any
➤
Approximation of
E
ON&
-
-
8
E
(
N -
-
E
(
8
L
H
-
E
(
N
8
8
8
8
&
ML&
F
-
E
E
N
and
The ten coefficients can be chosen to produce an absolute error
for all
less than
,
Section 7.2: Generating Continuous Random Variates
I
,
I
P
➤
8
Inversion can be used to generate Normal
➤
Example 7.2.6
variates:
D
<
u = Random();
;
return
➤
➤
This algorithm is portable, essentially exact, robust, reasonably
efficient, synchronized and monotone
Clarity?
Section 7.2: Generating Continuous Random Variates
9
,
-
-
✦
✦
✦
I
I
,
Q
Section 7.2: Generating Continuous Random Variates
I
I
)
✦
R,
The mean is 0.0 and the standard deviation is 1.0
Possible values are
Justification is provided by the central limit theorem (Section 9.1)
This algorithm is: portable, robust, relatively efficient and clear
This algorithm is not: exact, synchronized or monotone
Q
✦
is approximately Normal
I
6)
6
-
Q
:
:
:
is an iid sequence of Uniform
;
6
:
I
:
I
I 6
If
➤
:
Alternative Method 1
10
:
RVs (Box and Muller, 1958)
➤
This algorithm is: portable, exact, robust and relatively efficient;
➤
This algorithm is not: clear or monotone
➤
The algorithm is synchronized only in pair-wise fashion
Section 7.2: Generating Continuous Random Variates
6
W 5
3 Y
V
:
5
3 2
4
;
)
6
will be independent Normal
RVs then
6
:
W 5
X
:
and
)
UV
5
3 2
;
S T
are independent Uniform
and
6
:
If
➤
:
Alternative Method 2
11
&
Z
Z
. '
Example 7.2.7 Generate a Normal
➤
➤
.
(
Random variates corresponding to Normal
and Lognormal
can be generated by using a Normal
random variate generator
Normal and Lognormal Random Variates
random variate
(
&
Example 7.2.8 Generate a Lognormal
➤
Z
.
R[
-
z = Normal(0.0, 1.0);
;
return
/* see Definition 7.1.7 */
random variate
➤
1
0/
z = Normal(0.0, 1.0);
(a + b * z);
return
/* see Definition 7.1.8 */
Both algorithms are essentially ideal
Section 7.2: Generating Continuous Random Variates
12
Numerical Inversion
\
\
8
8
8
E
6
E
, ignore
) ^
) E
^
E
\
5
]
) *
-
E
E
-
E
) -
E
E
\
6
is expanded in a Taylor’s series about the point
For small
for which
E
➤
Recall
, let be close to the value of
E
➤
If
➤
Given
89
➤
Newton’s method provides a good compromise between rate of
convergence and robustness
➤
Numerical inversion provides another way to generate continuous
can be solved for iteratively
random variates; that is,
➤
Section 7.2: Generating Continuous Random Variates
and higher order terms
13
rrr
rrr
rr r
rr r
rrr
r rr
rrr
r rr
rr r
rr r
r rr
r rr
r rr
rr r
r rr
rrr
r rr
r r r r rr r
r r r rrr
r rr rrr
r rr r rr
rr r r rr
rr r r rr
r rr rr
rrr rrr
rr r rrr
r r r rrr
rrr rrr
rr r rrr
rrr rrr
rrrrrr
r r rr rr
r rrr rr
rr rr r r u
r r rr r r
r r rr r r
r r rr r r
rrr rr
rrrr rrr
rrr rrr
rrr rrr
rrr r rr
rrr r r r
rrr r rr
rrr r r r r
rrr rr r
rrr r rr
rrr r r r
rrr r r r
rrr rr rr
rr r rrrr
rrr r r r r
rrr
r
r rr r r r r r
rrr
rrr
rrr
rrr
rrrr
rrr
rrrr
rrr
rrr
rrr
rrr
rrr
rrr
rrr
rrr
rrr
rrr
r rru
rrr
rrr
rrr
rrr
rrr
rrr
r rr
rrr
rrr
r rr
rrr
rrr
rrr
r rr
rrr
rrr
r rr
rrr
rr r
r rr
r rr
rrr
rr r
rrr
rrr
rrr
rr r
r
rr r
rrr
rr r
rrr
r rr
rr r
rrr
r rr
rr r
rrr
rrr
rrr
rrr
p
rrr
rrr
r rr
rrr
rrr
r rr
rr r
r rr
r rr
rr r
r rr
rr r
rrr
r rr
rrr
rrr
rrr
r rr
rrr
rrr
rr
rrr
Section 7.2: Generating Continuous Random Variates
i
q
_
E
_
-
E
E
`
8
8
8
5
_
) E
L
E
_ E
'
and iterate to solve for :
_b
a
as
`
E
B
-
E
) E
) B
-
E
E
E
and solve for
o
l
stm
j
Use initial guess
h
rr r
o
➤
Set
ge
d c
r rr
rr r
rr r
rrr
rr r
rrr
rrr
rrr
rrr
d c
n
m
➤
f
d c
e
i
j
k
i
l
Newton’s Method
to obtain
14
.
is an
Given a convergence parameter
Iterate until
v
,
_
)
_b
✦
E
E
^
✦
The test for convergence
v ➤
✦
The best choice for the initial value is the mode
For most continuous RVs described in text,
essentially equivalent choice
L
✦
E
L
The choice of an initial value
^
➤
E
Two Issues Relative to Newton’s Method
Section 7.2: Generating Continuous Random Variates
15
89
and a convergence
v 8
, the pdf
, the cdf
, this algorithm will solve for
Given
parameter
➤
Algorithm 7.2.2
If is small and is non-negative, a negative value of
early in the iterative process.
Section 7.2: Generating Continuous Random Variates
E
and
E
Negative will cause
E
➤
➤
G
v
.
.
x = ; /* is E[X]*/
do {
t = x;
x = t + (u - F(t)) / f(t);
} while (|x-t| > );
*/
return x; /* x is
may occur
to be undefined for positive RVs
16
Modified Algorithm 7.2.2
➤
The following modification can be used to avoid the problem
M
v
.
.
x = ; /* is E[X]*/
do {
t = x;
x = t + (u - F(t)) / f(t);
if (x <= 0.0)
x = 0.5 * t;
} while (|x-t| > );
*/
return x; /* x is
(
E.g., the Erlang
idf function in rvms is based on Alg.7.2.2 and
can be used with Algorithm 7.2.1
w '
➤
Algorithms 7.2.1 and 7.2.2 together provide a general purpose
inversion approach to continuous random variate generation
➤
u = Random();
return idfErlang(n, b, u);
Section 7.2: Generating Continuous Random Variates
17
Alternative Random Variate Generation Algorithms
Erlang Random Variates
(
random variate can be generated by summing
random variates
w
w
(
An Erlang
Exponential
➤
x = 0.0;
for (i = 0; i < n; i++)
x += Exponential(b);
return x;
➤
The algorithm is not efficient (it is
Section 7.2: Generating Continuous Random Variates
The algorithm is: portable, exact, robust, and clear
x
w'
➤
), synchronized or monotone
18
Modified Algorithms for Erlang Random Variates
➤
To increase computational efficiency, use
t = 1.0;
for (i = 0; i < n; i++)
t *= (1.0 - Random());
return -b * log(t);
This algorithm requires only one log() evaluation, rather than
➤
Can further improve efficiency by using
w
➤
The algorithm remains
, so is not efficient if
Section 7.2: Generating Continuous Random Variates
w
➤
x
w'
t *= Random();
is large
19
Chisquare Random Variates
5
5
random variate is
-
-
-
8
8

6

;
I
I
I 6
;
8
w
~
&}
;
6
;
H|
{ `
;
6
random variable iff
are iid Normal
random variables
6
w '
zy
;
is a
where
➤
+
w '
If is an even positive integer, an Erlang
random variable
equivalent to a Chisquare
w
➤
x = 0.0;
for (i = 0; i < n; i++)
z = Normal(0.0, 1.0);
x += (z * z);
return x;
➤
The algorithm is not: efficient(it is
Section 7.2: Generating Continuous Random Variates
x
w
The algorithm is: portable, exact, robust, clear
➤
), synchronized or monotone
20
;
✦
+
€
w
;+
where
is Normal
is Chisquare
and are independent
w '
✦
iff
€
;
€
✦
is Student
➤
w '
Student Random Variates
z = Normal(0.0, 1.0);
v = Chisquare(n);
return z / sqrt(v / n);
➤
The algorithm is: portable, exact, robust, clear
➤
The algorithm is not synchronized or monotone
➤
Efficiency depends on algs. used for Normal and Chisquare
Section 7.2: Generating Continuous Random Variates
21
Testing for Correctness using Histograms
A natural way to do this at the computational level is:
use the algorithm to generate a sample of random variates and
construct a -bin continuous-data histogram with bin width
is the histogram density and
is the pdf
a
‚
and
✦
✦
ƒ
w
In practice, using a large but finite value of and a small but nonand
will not
zero value of , perfect agreement between
be achieved
‚
➤
as
w
ƒ
✦
ƒ

‚
✦
w
➤
In the discrete case, it is due to natural sampling variability
In the continuous case, the quantization error associated with
binning the sample is an additional factor
Section 7.2: Generating Continuous Random Variates
22
Quantization Error
5
-
5
†
„
)
‰
†
†
†
†
†
†
†
8
8
)
]
ˆ
\
-
-
‚
‚
Ž
†
†
8
8
8

5

8
8
8
‹ Š
Œ
„
\
‰
5
]
)
)
8
\
\
\
\
\
-
-
\
6
†
at
The probability of falling within the bin is
be a small histogram bin
➤
‡
Use the Taylor expansion of
‚+
➤
‚+
Let
†
'
➤
Section 7.2: Generating Continuous Random Variates
23
Quantization Error (2)
For all
➤
Unless
, the histogram density is
\
\
6
-
†
†
B
‹ Š
„
‚
✦
➤
†
✦
\
\
†
, there is a positive or negative bias between
, the experimental density of the histogram bin and
, the theoretical pdf evaluated at the bin midpoint
ƒ
5

‚
ƒ
„
➤
This bias may be significant if the curvature of the pdf is large at the
bin midpoint
Section 7.2: Generating Continuous Random Variates
24
Example 7.2.9
is a continuous random variable with pdf
-
‰
5
➤
The cdf
➤
The idf is
is
,
)
7
)
,
L
-
6
)
E
E
Ž

➤
Note the pdf curvature is very large close to
; therefore, the
histogram will not match the pdf well for the bins close to
➤
Section 7.2: Generating Continuous Random Variates
25
Example 7.2.9 ctd.
8
8
I
I
I
I
I
8
J
’
†
5
J
5
J
5
J
’
a
J
I
J
5
J
J
5

5
I
”ˆ
ˆ
ˆ“
I
I
”
I
I
“’
”
I
“
I
I
ˆ
“”
ˆ“
I
I
I
‘
‘
†
ƒ
and
’
ˆ
J“
5
Q
”
I
Q
J
’
5
ˆ
’
J
5
”
“
I
5

†
‚
\
\
6
), the curvature bias is
5
For the first bin (
†
➤
I
I
‘
†
,
, as
w
Using histogram bin widths of
are (with d.dddd precision):
ƒ
Correctness of the inversion can be tested by constructing a
histogram
’
➤
can be generated using inversion
‚
➤
Random variates for
➤
Section 7.2: Generating Continuous Random Variates
26
For large samples (as
a
w
Section 7.2: Generating Continuous Random Variates
),
➤
Eliminates binning quantization error
➤
ƒ
Compare the empirical cdf (section 4.3) with the population cdf
➤
Testing for Correctness using the Empirical cdf
27
Library rvgs
Contains 7 continuous random variate generators
✦
✦
(
w
Section 7.2: Generating Continuous Random Variates
(
.
Z
(
✦
&
✦
.
✦
Chisquare(long )
Erlang(long , double )
Exponential(double )
Lognormal(double , double )
Normal(double , double )
Student(long )
Uniform(double , double )
&
✦
double
double
double
double
double
double
double
w
✦
w
➤
28