Probability - Central Limit Theorem and Moment Generating
Function Proof
Kinglee Tham
December 2025
1
Sampling Distributions and Standardization
Let X1 , X2 , . . . , Xn be i.i.d. with mean µ and variance σ 2 . Define the sample mean:
n
1X
Xn =
Xi .
n i=1
By linearity,
σ2
.
n
As n increases, the variance shrinks. The distribution of the sample mean becomes more concentrated
around µ.
E[X n ] = µ,
Var(X n ) =
Standardized sample mean
Define the standardized version:
Zn =
Xn − µ
√ .
σ/ n
This rescales X n so that Zn has mean 0 and variance 1. The CLT states that Zn converges in distribution
to Norm(0, 1).
2
Moment Generating Functions (MGFs)
2.1
MGF of a standardized variable
Let MX (t) = E[etX ] denote the mgf of X.
For the standardized RV
Yi =
Xi − µ
,
σ
we have:
E[Yi ] = 0,
Var(Yi ) = 1.
The mgf is:
MY (t) = E[etYi ].
We will expand MY (t) using the Taylor series for etY around t = 0.
1
3
MGF of the Standardized Sample Mean
We want the mgf of the standardized mean:
√
Zn =
n
n(X n − µ)
1 X
=√
Yi .
σ
n i=1
Since Yi are i.i.d.,
MZn (t) =
4
MY
t
√
n
n
.
Taylor Expansion Without Big-O Notation
We expand using the Taylor series:
etY = 1 + tY +
t2 Y 2
t3 Y 3
t4 Y 4
+
+
+ ···
2
6
24
Take expectation term-by-term:
MY (t) = 1 + tE[Y ] +
t2
t3
t4
E[Y 2 ] + E[Y 3 ] + E[Y 4 ] + · · ·
2
6
24
Since Y is standardized:
E[Y 2 ] = 1,
E[Y ] = 0,
This becomes:
MY (t) = 1 +
√
Now plug in t/ n:
MY
t
√
n
=1+
t2
t3
t4
+ E[Y 3 ] + E[Y 4 ] + · · ·
2
6
24
t3
t4
t2
+ 3/2 E[Y 3 ] +
E[Y 4 ] + · · ·
2n 6n
24n2
2
t
Every term after 2n
contains a higher power of n1 . As n → ∞, these terms go to 0. Thus the only term
t2
.
that matters in the limit is 2n
So we write:
t2
t
+ (terms that vanish as n → ∞).
MY √
=1+
2n
n
Raise to the n-th power:
MZn (t) =
n
t2
1+
+ vanishing terms .
2n
Take logs (ignoring terms that vanish):
log MZn (t) = n ·
t2
t2
→ .
2n
2
Exponentiate:
2
MZn (t) → et /2 .
This is exactly the mgf of the standard normal distribution.
2
5
Central Limit Theorem (MGF Proof )
Theorem (Central Limit Theorem)
Let X1 , . . . , Xn be i.i.d. with mean µ and variance σ 2 < ∞. Then the standardized mean satisfies:
Xn − µ d
√
−
→ Norm(0, 1).
σ/ n
Summary
• Expand the exponential in the mgf.
√
• Higher-order terms go to zero after substituting t/ n.
2
• Only the t2 term survives.
2
• The mgf converges to et /2 .
d
Since mgfs determine distributions, Zn −
→ N (0, 1).
3