Lecture 26
1. Moment generating functions
Let X be a continuous random variable with density f . Its moment generating function is defined as
! ∞
tX
M (t) = E[e ] =
etx f (x) dx,
(19)
provided that the integral exists.
−∞
Example 26.1 (Uniform(0 , 1)). If X = Uniform(0 , 1), then
! 1
et − 1
M (t) = E[etX ] =
etx dx =
.
t
0
Example 26.2 (Gamma). If X = Gamma(α , λ), then
! ∞
λα α−1 −λx
M (t) =
etx
x
e
dx
Γ(α)
0
! ∞
λα
=
xα−1 e−(λ−t)x dx.
Γ(α) 0
If t ≥ λ, then the integral is infinite. On the other hand, if t < λ, then
! ∞
z α−1
dz
λα
(z = (λ − t)x)
M (t) =
e−z
Γ(α) 0 (λ − t)α−1
λ−t
! ∞
λα
=
z α−1 e−z dz
Γ(α) × (λ − t)α 0
#$
%
"
Γ(α)
=
λα
(λ − t)α
.
91
92
Thus,
26
*α
)
λ

λ−t
M (t) =

∞
if t < λ,
otherwise.
Example 26.3 (N (0 , 1)). If X = N (0 , 1), then
! ∞
1
2
M (t) = √
etx e−x /2 dx
2π −∞
) 2
*
! ∞
x − 2tx
1
exp −
dx
=√
2
2π −∞
) 2
*
!
2
et /2 ∞
x − 2tx + t2
=√
exp −
dx
2
2π −∞
*
)
!
2
et /2 ∞
(x − t)2
=√
dx
exp −
2
2π −∞
!
2
et /2 ∞ −u2 /2
=√
e
du
(u = x − t)
2π −∞
2 /2
= et
.
2. Relation to moments
Suppose we know the function M (t) = E exp(tX). Then, we can compute
the moments of X from the function M by successive differentiation. For instance, suppose X is a continuous random variable with moment generating
function M and density function f , and note that
!
d ∞ tx
d + tX ,
E[e ] =
e f (x) dx.
M # (t) =
dt
dt −∞
Now, if the integral converges absolutely, then a general fact states that we
can take the derivative under the integral sign. That is,
! ∞
.
xetx f (x) dx = E XetX .
M # (t) =
−∞
The same end-result holds if X is discrete with mass function f , but this
time,
/
.
M # (t) =
xetx f (x) = E XetX .
Therefore, in any event:
x
M # (0) = E[X].
In general, this procedure yields,
.
M (n) (t) = E X n etX .
93
2. Relation to moments
Therefore,
M (n) (0) = E [X n ] .
Example 26.4 (Uniform). We saw earlier that if X is distributed uniformly
on (0 , 1), then for all real numbers t,
M (t) =
Therefore,
M # (t) =
whence
tet − et + 1
,
t2
et − 1
.
t
M ## (t) =
t2 et − 2tet + 2et − 2
,
t3
tet
1
tet − et + 1
=
lim
= ,
2
t$0 2t
t$0
t
2
EX = M # (0) = lim
by l’Hopital’s rule. Similarly,
- .
t2 et − 2tet + 2et − 2
t2 et
1
E X 2 = lim
=
lim
= .
t$0
t$0 3t2
t3
3
Alternatively, 0these can be checked by direct computation, using the fact
1
that E[X n ] = 0 xn dx = 1/(n + 1).
A discussion of physical CLT machines . . . .