THE FUNDAMENTAL TRICK FOR COMPUTER SIMULATIONS
                     
This document could be called “the fundamental theorem of computer simulation,” but
no one has yet bestowed that high-power name on the result.
The single most-studied computer simulation task is that of generating random numbers
from the uniform (0, 1) distribution. There are quite a few tricks for doing this, and it is
now reasonable to accept the integrity of computer random uniform numbers.
Computer uniform random numbers can be generated with lightning speed, and these
have become the standard tool for simulations.
Recall that if U is uniform on (0, 1), then its density is fU(u) = I( 0 ≤ u ≤ 1).
There are situations in which, for mathematical mischief, the ≤ signs are
replaced with < .
The cumulative distribution function is
0

FU  u   u
1

for u  0
for 0  u  1
for u  1
Here’s the theorem:
Suppose that X is a continuous random variable with cumulative distribution function F.
Then the distribution of F(X) is uniform on (0, 1).
It’s important to understand what is being done here, and an example will help. Let’s
say that X has the exponential density with mean 4. In this example, the density is
1  4x
I  x  0
f(x) = e
4
The cumulative distribution function is
for x  0
0


F x  
x

1  e 4 for x  0
As a trivial observation, at each value of x* which is possible for the random variable X
(meaning f(x*) > 0) it will happen that F(x*) is increasing. Thus F is a monotone
increasing function, and it’s strictly increasing at every possible value for X. This means
that F has an inverse F -1 at every value which is possible for X.

X
By F(X) we mean the random variable computed as 1  e 4 . This is an upper-case X,
and we are thinking of this calculation as a random variable. There is now no concern
with F(x) for x < 0, as these are impossible values.
1

gs2011
THE FUNDAMENTAL TRICK FOR COMPUTER SIMULATIONS
                     
The proof of the theorem is very slick. Define U = F(X). We need to show that U has
the uniform (0, 1) distribution, and this can be accomplished if we show that
P[ U ≤ u ] = u for any u in the interval [0, 1].
For any value x* which is possible for X,
{ F(x*) ≤ u }  { x* ≤ F -1(u) }
Since F is strictly increasing at values of x* which are possible for X, the inverse
function is well-defined.
Then
P[ U ≤ u ] = P[ F(X) ≤ u ] = P[ X ≤ F -1(u) ] = F( F -1(u) ) = u
Done.
Here’s a side note on finding inverse functions. If y = g(x), the inverse g-1 is
found just by solving for x in the equation y = g(x) (if possible). The result will
1
give g-1. If y = x3 + 1 the solution is x = 3 y  1 =  y  1 3 . Both x and y are
handled as “dummy” symbols, so you’ll see statements like these:
1
The inverse function of x3 + 1 is  x  1 3 .
The inverse function of x3 is
3
x.
Care is needed for many-to-one functions. The function y = x2 maps both x = -2
and x = +2 to the same value of y, so the square function does not have an inverse.
If we can agree that we’re only dealing with positive values of x, then the inverse
of x 2 is the positive square root x .
If we are considering sin x, where x ranges over all real numbers, then sin x
does not have an inverse. If we can agree that the only x’s of concern are in the
inverval (-, +), then sin-1 can be defined.
It helps to see the theorem in the context of the example. For that example,
X
X




4
F(X) = 1  e . Let’s ask then P  1  e 4  u  .
X
X






X 

P  1  e 4  u  = P  1  u  e 4  = P  log 1  u   
4 

2

gs2011
THE FUNDAMENTAL TRICK FOR COMPUTER SIMULATIONS
                     
The “log” here is base e.
= P  4   log 1  u 

 X  = P  X  4   log 1  u   
Since u is a value in (0, 1), it must happen that 1 – u is
also in (0, 1). So log(1 – u) is negative.

= F 4   log 1  u 

= 1  e

1
 4   log1  u  
4
= 1 - (1 – u) = u
Here’s how this is used. Let’s suppose that you want to simulate values X1, X2, X3, …
from a continuous cumulative distribution function F. Create the random uniform
sample U1, U2, U3, ….. The values F -1(U1), F -1(U2), F -1(U3), … accomplish the job.
The Cauchy random variable has distribution function F(x) =
up the inverse, use u =
1
1

tan 1  x  and solve for x.
2

1
1

tan 1  x  .
2

To set
This will lead to
1 
 
x  tan    u    . Generate independent uniform random variables U1, U2, U3,…
2 
 
1 
1 
1 
 
 
 
and then use tan    U1    , tan    U 2    , tan    U3    ,...
2 
2 
2 
 
 
 
for the Cauchy sample.
The exponential random variable has distribution function F(x) = 1 – e – x for x ≥ 0. The
solution to u = F(x) is x = - log(1 – u). Generate independent uniform random
variables U1, U2, U3,… and then use -log(1 – U1), -log(1 – U2), -log(1 – U3), … as the
exponential sample.
In practice, this gets made even simpler. Just observe that U and 1 – U
have the same distribution. The exponential sample can be made
as -log(U1), -log(U2), -log(U3), … This is a small saving, of course; it
just saves one subtraction for each generated number.
3

gs2011
THE FUNDAMENTAL TRICK FOR COMPUTER SIMULATIONS
                     
Suppose that you want to simulate a sample from the density f  x  =
cumulative distribution function is F(x) = 1 
1
.
x
1
I  x  1 . The
x2
The solution to u = F(x) is
1
. Generate independent uniform random variables U1, U2, U3,… and then use
1 u
1
1
1
,
,
, ... for the desired sample. You’ll probably invoke the “even
1  U1 1  U 2 1  U 3
1 1 1
,
,
, ...
simpler” remark just above to create the sample as
U1 U 2 U 3
x=
This is a wonderful idea, but it doesn’t solve all your problems.
This does not help you with simulating discrete random variables.
Sometimes you will not be able to find F -1 in a closed from. The normal cumulative
distribution function, for example, cannot be inverted.
4

gs2011