Chapter 8:
Limit Theorems
Katherine Davies
Chapter 8: Limit Theorems
1 / 12
Two important results
In probability, there are some important and useful results; three of
them are:
1
Markov’s Inequality
2
Chebyshev’s Inequality
3
The Central Limit Theorem
Katherine Davies
Chapter 8: Limit Theorems
2 / 12
Markov’s Inequality
If X is a random variable that takes only non-negative values, then for
any value a > 0,
E [X ]
.
P (X > a) ≤
a
Katherine Davies
Chapter 8: Limit Theorems
3 / 12
Chebyshev’s Inequality
If X is a random variable with finite mean µ and variance σ2 , then for
any value k > 0,
σ2
P (|X − µ| ≥ k ) ≤ 2 .
k
Katherine Davies
Chapter 8: Limit Theorems
4 / 12
Example
Suppose that the scores in an exam have a mean of 70 and a variance
of 6. What percentage of students have their scores away from the
mean by at least 10?
Katherine Davies
Chapter 8: Limit Theorems
5 / 12
Central Limit Theorem I
Theorem
Let X1 , X2 , . . . be a sequence of independent and identically distributed
random variables, each having finite mean µ and variance σ2 .
Then the distribution of
X1 + · · · + Xn − nµ
√
σ n
tends to the standard normal as n → ∞.
Katherine Davies
Chapter 8: Limit Theorems
6 / 12
Central Limit Theorem II
In other words, for −∞ < µ < ∞, we have that
Z a
2
X1 + · · · Xn − nµ
1
√
√
P
≤a →
e −x /2 dx as n → ∞.
2 π −∞
σ n
↑
PDF of N (0, 1)
Katherine Davies
Chapter 8: Limit Theorems
7 / 12
Example
If 10 fair dice are rolled, find the approximate probability that the sum
obtained is between 30 and 40, inclusive.
Katherine Davies
Chapter 8: Limit Theorems
8 / 12
Example
Let Xi , i = 1, . . . , 10 be independent random variables, each uniformly
distributed over (0,1). Calculate an approximation to P ∑10
X
>
6
.
i =1 i
Katherine Davies
Chapter 8: Limit Theorems
9 / 12
Galton board
In Chapter 5, we looked actually look at the Central Limit Theorem
(without knowing it) as it applied to the binomial distribution. The
Galton board, also known as a Quincunx or bean machine is a
physical model of the binomial distribution which demonstrates the
Central Limit Theorem.
We may ask: Why and how did the CLT apply to the binomial
distribution?
We now know that Sn is the sum of n i.i.d. Bernoulli(p) random
variables, and since the CLT applies to sums, it certainly applies here!
Katherine Davies
Chapter 8: Limit Theorems
10 / 12
Galton board
The board was invented by Sir Francis Galton and consists of a board
with evenly spaced pegs or nails, which make the board look like a
whole bunch of triangles.
The board is placed vertically, often with a funnel at the top, which is
filled with balls, but in such a way that only one ball at a time can fall
from the funnel and onto the board (and amongst the pegs).
When a ball is released, it hits a peg and can go left or right, with
probabilities p and 1 − p, respectively, say.
Once the ball reaches the bottom, the slot it ends up in is essentially a
sum of lefts (1=success) and rights (0=failure), i.e., the sum of n
independent Bernoulli random variables!
Katherine Davies
Chapter 8: Limit Theorems
11 / 12
Galton board
See here for a
demonstration.
Katherine Davies
Chapter 8: Limit Theorems
12 / 12