Lecture 11.
Central Limit Theorem
Juwon Seo
Juwon Seo
Lecture 11. Central Limit Theorem
1 / 24
Announcement
Today’s lecture is the last lecture of this course. Well done guys!
Juwon Seo
Lecture 11. Central Limit Theorem
2 / 24
Outline
Today we will learn
I Central limit theorem
I Interval estimation (confidence intervals)
I Hypothesis testing
Juwon Seo
Lecture 11. Central Limit Theorem
3 / 24
Estimation -continued
Suppose we have an iid sequence of random variables
{X1 , X2 , X3 , . . . Xn }. n is the sample size.
I We know a good (unbiased, efficient and consistent) estimator for
E(X) :
n
1X
X̄n =
Xi .
n i=1
I Although we know that X̄n →p E(X) and E(X̄n ) = E(X), we have
no clue on how X̄n is close to E(X). If you are unlucky, it is
possible for us to have X̄n far away from the true mean E(X).
I Although we can calculate X̄n from our observations, we
CANNOT say X̄n = E(X).
Juwon Seo
Lecture 11. Central Limit Theorem
4 / 24
Estimation -continued
Maybe instead, can we say something like...?
I The probability that E(X) is between a and b is 0.95 or,
I We are 95% confident that E(X) is between a and b.
We can find such an interval (a, b)! This is called an interval
estimator.
Juwon Seo
Lecture 11. Central Limit Theorem
5 / 24
Central Limit Theorem
Assuming we have iid random variables {X1 , X2 , X3 , . . . Xn } and each
of them follows N(µ, σ 2 ). Since we know that
1 X̄n has a normal distribution,
2 E(X̄n ) = µ and,
2
3 Var(X̄n ) = σn ,
the distribution of the random variable
X̄n − µ
σ
√
n
is N(0, 1).
Juwon Seo
Lecture 11. Central Limit Theorem
6 / 24
Central Limit Theorem
From the Standard Normal Table, we can find a constant c such that
X̄n − µ
√ ≤ c = 0.95
P −c ≤
σ/ n
Juwon Seo
Lecture 11. Central Limit Theorem
7 / 24
Central Limit Theorem
From the Standard Normal Table, we can find a constant c such that
X̄n − µ
√ ≤ c = 0.95
P −c ≤
σ/ n
c is 1.96:
X̄n − µ
P −1.96 ≤ √ ≤ 1.96 = 0.95
σ n
Juwon Seo
Lecture 11. Central Limit Theorem
8 / 24
Central Limit Theorem
X̄n − µ
√ ≤ 1.96 = P X̄n − 1.96n−1/2 σ ≤ µ ≤ X̄n + 1.96n−1/2 σ
P −1.96 ≤
σ/ n
If σ is known, we can say
I µ is between X̄n − 1.96n−1/2 σ and X̄n + 1.96n−1/2 σ with the probability 0.95
or,
I we are 95% sure that µ is between X̄n − 1.96n−1/2 σ and X̄n + 1.96n−1/2 σ.
(X̄n − 1.96n−1/2 σ, X̄n + 1.96n−1/2 σ) is called the 95% confidence interval.
Juwon Seo
Lecture 11. Central Limit Theorem
9 / 24
Central Limit Theorem
We may also define the 90% confidence interval and the 80%
confidence interval for µ:
I (X̄n − 1.645n−1/2 σ, X̄n + 1.645n−1/2 σ) is the 90% confidence
interval.
I (X̄n − 1.28n−1/2 σ, X̄n + 1.28n−1/2 σ) is the 80% confidence
interval.
Juwon Seo
Lecture 11. Central Limit Theorem
10 / 24
Central Limit Theorem
How do we assume that we know σ 2 ?
In practice, we do not know σ 2 but we can estimate it by the sample
variance! As the sample size gets larger, the sample variance σ̂ 2
converges in probability to σ 2 . Therefore
X̄n − µ
σ̂
√
n
also has the same limit distribution N(0, 1). If σ 2 is unknown,
we find the 95% confidence interval as
(X̄n − 1.96n−1/2 σ̂, X̄n + 1.96n−1/2 σ̂).
Juwon Seo
Lecture 11. Central Limit Theorem
11 / 24
Central Limit Theorem
Can we assume that each of {X1 , X2 , X3 , . . . Xn } follows a normal
distribution? Actually, we do not know that each of {Xi } has a normal
distribution. Sometimes it is not normal!
However, the Central Limit Theorem justifies that we can use the
same confidence interval
(X̄n − 1.96n−1/2 σ̂, X̄n + 1.96n−1/2 σ̂)
for any iid random variables {X1 , X2 , X3 , . . . Xn } (with any
distribution), when the sample size is large enough.
Juwon Seo
Lecture 11. Central Limit Theorem
12 / 24
Central Limit Theorem
Theorem
Suppose we have an iid sequence of random variables
{X1 , X2 , X3 , . . . Xn }. We denote the common expected value by
µ and the common variance by σ 2 . We define the sample mean
X̄n by:
n
1X
Xi .
X̄n =
n i=1
The Central Limit Theorem says
√
X̄n − µ
n(X̄n − µ)
√ =
→d N(0, 1),
σ
σ/ n
as n → ∞.
Juwon Seo
Lecture 11. Central Limit Theorem
13 / 24
Central Limit Theorem (Another version)
Theorem
Suppose we have an iid sequence of random variables
{X1 , X2 , X3 , . . . Xn }. We denote the common expected value by
µ and the common variance by σ 2 . We define the sample mean
X̄n by:
n
1X
Xi .
X̄n =
n i=1
The Central Limit Theorem says
n
1 X
√
n i=1
Xi − µ
σ
→d N(0, 1),
as n → ∞.
Juwon Seo
Lecture 11. Central Limit Theorem
14 / 24
Central Limit Theorem (Another version)
I Here I randomly drew the samples {Xi } from the Bernoulli( 13 )
distribution
the sample size n. As n increases, we can see
P fixing
−(1/3) that √1n ni=1 Xi√
approaches N(0, 1).
2/3
Juwon Seo
Lecture 11. Central Limit Theorem
15 / 24
Central Limit Theorem
A restaurant wants to examine the mean of a dinner check. A random
sample of 50 dinners showed an average bill of $36 and the sample
variance of $122 . We want to find the confidence interval for the true
mean.
You may assume that the sample size 50 is large enough to apply the
Central Limit Theorem.
Juwon Seo
Lecture 11. Central Limit Theorem
16 / 24
Central Limit Theorem
Find 95% confidence interval for the population mean of the dinner
check for all costumers.
Note that n−1/2 σ̂ =
√12
50
≈ 1.697, thus,
(X̄n −1.96n−1/2 σ̂, X̄n +1.96n−1/2 σ̂) = (36−1.96×1.697, 36+1.96×1.697)
Interpretation:µ is between 32.67 and 39.33 with probability 0.95.
Juwon Seo
Lecture 11. Central Limit Theorem
17 / 24
Central Limit Theorem
Find 90% confidence interval for the population mean of the dinner
check for all costumers.
(X̄n −1.645n−1/2 σ̂, X̄n +1.645n−1/2 σ̂) = (36−1.645×1.697, 36+1.645×1.697)
Interpretation:µ is between 33.21 and 38.79 with probability 0.9.
Juwon Seo
Lecture 11. Central Limit Theorem
18 / 24
Hypothesis testing
Suppose that you collected iid samples and calculated the sample
mean, which is 0.02. Based on what you observe, you want determine
if you can say µ = 0.
H0 : µ = 0.
I Is our sample mean close enough to 0 such that we can conclude
that µ = 0?
I Or is 0.02 far away from 0?
Juwon Seo
Lecture 11. Central Limit Theorem
19 / 24
Hypothesis testing
Statistical test:
I H0 : µ = 0 is called the null hypothesis.
I Idea: If µ̂ is close enough to 0, we accept the null hypothesis
(some textbook says, “we cannot reject the null hypothesis") and
conclude that µ = 0.
Otherwise, we reject the null hypothesis and conclude that µ 6= 0.
I How do we define being close enough?
If the confidence interval of µ (which has µ̂ as the center)
includes 0, we can conclude that µ̂ is close to 0.
Juwon Seo
Lecture 11. Central Limit Theorem
20 / 24
Hypothesis testing
In the previous example, let us test if
H0 : µ = 35.
I Since 95% confidence interval does include 35, we accept the null
hypothesis (or we cannot reject the null hypothesis) and say
(µ = 35) at the 95% confidence level.
I Since 90% confidence interval does include 35, we accept the null
hypothesis (or we cannot reject the null hypothesis) and say
(µ = 35) at the 90% confidence level.
Juwon Seo
Lecture 11. Central Limit Theorem
21 / 24
Hypothesis testing
What if we test for the null hypothesis
H0 : µ = 33?
I Since 95% confidence interval does include 33, we accept the null
hypothesis and say (µ = 33) at the 95% confidence level.
I Since 90% confidence interval does not include 33, we reject the
null hypothesis and say (µ 6= 33) at the 90% confidence level.
Juwon Seo
Lecture 11. Central Limit Theorem
22 / 24
Hypothesis testing
Assuming that the random sample has 150 diners now, and that the
average bill and sample variance remain unchanged: the sample mean
is 36 and the sample variance of 122 . Verify the following hypotheses
at the 95% confidence level.
I H0 : µ = 34
I H0 : µ = 35
Juwon Seo
Lecture 11. Central Limit Theorem
23 / 24
Hypothesis testing
I The 95% confidence interval is
(36 − 1.96 ∗ 150
− 1/2
∗ 12, 36 + 1.96 ∗ 150
− 1/2
∗ 12)
= (36 − 1.920, 36 + 1.920)
= (34.08, 37.92)
I Reject (H0 : µ = 34) at the 95% confidence level. We cannot
conclude that µ = 34 at the 95% confidence level.
I Accept (H0 : µ = 35) at the 95% confidence level. We can
conclude that µ = 35 at the 95% confidence level.
Juwon Seo
Lecture 11. Central Limit Theorem
24 / 24