Math 3680
Lecture #12
The Central Limit Theorem
for Sample Sums and
Sample Averages
In the previous lecture, we introduced the central limit
theorem when drawing from a box containing
exclusively “0”s and “1”s.
We now generalize this technique to other kinds of
populations: heights of college students, incomes, or
anything else which is not dichotomous.
Linear Combinations of
Random Variables
Theorem. For any random variables X and Y and
any constants a, b, and c, we have
E( a X + b Y + c ) = a E( X ) + b E ( Y ) + c.
Note: X and Y do not have to be independent.
Definition: Random variables X and Y are independent if for
all constants a and b,
P(X ≤ a and Y ≤ b) = P(X ≤ a) P(Y ≤ b)
For discrete random variables, this is the same as saying that
for all x and all y,
P(X=x and Y=y) = P(X =x) P(Y=y)
Theorem. If X and Y are independent, then
E( X Y ) = E( X ) E ( Y ).
Proof. We show the discrete case; the continuous case
is analogous (using integrals instead of summations).
E ( X Y )   x y P( X  x, Y  y )
x
y
  x y P( X  x) P(Y  y )
x
y


  x P( X  x)  y P(Y  y ) 
x
 y

  x P( X  x) E (Y )
x
 E ( X ) E (Y ).
Theorem. For any random variable X and any
constant a, we have
Var( a X ) = a2 Var( X ),
SD( a X ) = |a| SD( X ).
(Remember this from earlier?)
Theorem. For any independent random variables X
and Y, we have
Var( X + Y ) = Var( X ) + Var ( Y ).
Proof.
2
2
Var ( X Y )  E ([ X  Y ] )  [ E ( X  Y )]
 E ( X 2  2 XY  Y 2 )  [ E ( X )  E (Y )]2
 E ( X )  2 E ( XY )  E (Y )
2
2
 [ E ( X )]  2 E ( X ) E (Y )  [ E (Y )]
 Var ( X )  Var (Y ).
2
2
Note: The assumption of independence is critical in
the last theorem. For example, if X = Y, then
Var( X + X ) = Var( 2 X ) = 4 Var( X )
 Var( X ) + Var( X )
Example. Let X and Y be independent r.v.’s with
X ~ Binomial(8, 0.4) and Y ~ Binomial(8, 0.4).
Find E( X 2 ) and E( X Y ).
Example. Let S be the sum of 5 thrown dice.
Find E( S ) and SD( S ).
The Central Limit Theorem
(or the Law of Averages
for the Sample Sum)
The normal approximation may be used for other random
variables beside binomial ones.
Theorem. Suppose random variables X1, X2, …, Xn are drawn
with replacement from a large population with mean m and
standard deviation s. Let
SUM = X1 + X2 + …+ Xn.
Then E(SUM) = n m and SD(SUM) = s n .
(Why?)
Furthermore, if n is “large,” then we may accurately
approximate probabilities of SUM by converting to standard
units and using the normal curve.
The normal approximation may be used for other random
variables beside binomial ones.
Theorem. Suppose random variables X1, X2, …, Xn are drawn
without replacement from a population of size N with mean m
and standard deviation s. Let
SUM = X1 + X2 + …+ Xn.
N n
Then E(SUM) = n m and SD(SUM) = s n
. (Why?)
N 1
Furthermore, if n is “large,” then we may accurately
approximate probabilities of SUM by converting to standard
units and using the normal curve.
Question: How large is large “enough” for the normal
curve to be applicable? The answer is, It depends. If
the box itself follows the normal distribution exactly,
then so will SUM, no matter what the value of n is.
However, this trivial case rarely happens in practice.
For a more typical example, let’s look at
P(X = 1) = 1/3
P(X = 2) = 1/3
P(X = 3) = 1/3
Sum of 1 Random Variables
0.3
0.25
0.2
0.15
0.1
0.05
1
2
3
Sum of 2 Random Variables
0.3
0.25
0.2
0.15
0.1
0.05
1
2
3
4
5
6
Sum of 3 Random Variables
0.25
0.2
0.15
0.1
0.05
2
4
6
8
Sum of 4 Random Variables
0.2
0.15
0.1
0.05
2
4
6
8
10
12
Sum of 5 Random Variables
0.2
0.15
0.1
0.05
2.5
5
7.5
10
12.5
15
Sum of 10 Random Variables
0.14
0.12
0.1
0.08
0.06
0.04
0.02
10
15
20
25
30
Sum of 15 Random Variables
0.12
0.1
0.08
0.06
0.04
0.02
25
30
35
40
Sum of 20 Random Variables
0.1
0.08
0.06
0.04
0.02
30
35
40
45
50
55
Sum of 25 Random Variables
0.08
0.06
0.04
0.02
40
45
50
55
60
65
Sum of 30 Random Variables
0.08
0.06
0.04
0.02
50
55
60
65
70
75
Sum of 35 Random Variables
0.08
0.06
0.04
0.02
60
70
80
90
Sum of 40 Random Variables
0.06
0.04
0.02
70
80
90
100
Another example: suppose
P(X = 1) = 1/7
P(X = 2) = 1/7
P(X = 5) = 3/7
P(X = 9) = 1/7
P(X = 20) = 1/7
Sum of 1 Random Variables
0.4
0.3
0.2
0.1
5
10
15
20
Sum of 2 Random Variables
0.2
0.15
0.1
0.05
10
20
30
40
Sum of 3 Random Variables
0.12
0.1
0.08
0.06
0.04
0.02
10
20
30
40
50
60
Sum of 4 Random Variables
0.08
0.06
0.04
0.02
20
40
60
80
Sum of 5 Random Variables
0.06
0.05
0.04
0.03
0.02
0.01
20
40
60
80
100
Sum of 6 Random Variables
0.05
0.04
0.03
0.02
0.01
20
40
60
80
100
120
Sum of 7 Random Variables
0.05
0.04
0.03
0.02
0.01
20
40
60
80
100
120
140
Sum of 8 Random Variables
0.04
0.03
0.02
0.01
25
50
75
100
125
150
Sum of 9 Random Variables
0.035
0.03
0.025
0.02
0.015
0.01
0.005
25
50
75
100
125
150
175
Sum of 10 Random Variables
0.03
0.025
0.02
0.015
0.01
0.005
20
40
60
80
100
120
140
Sum of 15 Random Variables
0.02
0.015
0.01
0.005
25
50
75
100
125
150
175
Sum of 20 Random Variables
0.0175
0.015
0.0125
0.01
0.0075
0.005
0.0025
100
150
200
Sum of 25 Random Variables
0.014
0.012
0.01
0.008
0.006
0.004
0.002
100
150
200
250
Sum of 30 Random Variables
0.012
0.01
0.008
0.006
0.004
0.002
100
150
200
250
300
Sum of 35 Random Variables
0.01
0.008
0.006
0.004
0.002
100
150
200
250
300
350
Sum of 40 Random Variables
0.01
0.008
0.006
0.004
0.002
200
250
300
350
400
Sum of 45 Random Variables
0.01
0.008
0.006
0.004
0.002
200
250
300
350
400
450
Sum of 50 Random Variables
0.008
0.006
0.004
0.002
250
300
350
400
450
500
Example. Two hundred tickets are drawn at random
with replacement from the following box of tickets:
1
2
3
4
5
• What is the smallest possible sum? The biggest?
• What is the expected sum?
• Find the probability that the sum of the tickets is
more than 630.
Example: Thirty-six jets wait to take off from an
airport. The average taxi and take-off time for each
jet is 8.5 minutes, with an SD of 2.5 minutes. What
is the probability that the total taxi and take-off time
for the 36 jets is less than 320 minutes?
Example. A gambler makes 1000 column bets at
roulette. The chance of winning on any one play is
12/38. The gambler can either win $2 or lose $1 on
each play. Find the probability that, in total, the
gambler wins at least $0.
Example. A gambler makes 10,000 column bets at
roulette. The chance of winning on any one play is
12/38. The gambler can either win $2 or lose $1 on
each play. Find the probability that, in total, the
gambler wins at least $0.
The Central Limit Theorem
(or the Law of Averages)
for the Sample Mean
Theorem. Suppose random variables X1, X2, …, Xn are drawn
with replacement from a large population with mean m and
standard deviation s. Let
X1  X 2    X n
X
n
Then E( X ) = m and SD(
)=
s
n
.
(Why?)
Furthermore, if n is “large,” then we may accurately
approximate probabilities of
by converting to standard
units and using the normal curve. (As before, this is exact if
the original population follows the normal curve.)
Theorem. Suppose random variables X1, X2, …, Xn are drawn
without replacement from a population of size N with mean m
and standard deviation s. Let
X1  X 2    X n
X
n
Then E( X ) = m and SD(
)=
s
n
N n
.
N 1
(Why?)
Furthermore, if n is “large,” then we may accurately
approximate probabilities of
by converting to standard
units and using the normal curve. (As before, this is exact if
the original population follows the normal curve.)
Example: The cookie machine at Chips Ahoy adds
a random number of chips to each cookie. The
number of chips is a random number with average
28.5 and SD 5.3. Find the probability that, in a bag
of 50 cookies, the average number of chips per
cookie is at least 30.
Example: A computer generates 100 random
numbers between 0 and 1 (presumably evenly). Find
the probability that the average of these numbers is
between 0.48 and 0.49.
Review of Law of Averages
Population
Statistic
(Parameter)
Use the Law of
Averages for
Sample...
Dichotomous (p)
(0-1 box)
K
Count
p
Proportion
SUM
Sum
Quantitative (m)
Mean
COUNT (0/1 box)
E(K) = n p
SD(K) = np (1  p )
PROPORTION (0/1 box)
E(P) = p
SD(P) = p (1  p )
n
SUM
E(SUM) = n m
SD(SUM) = s n
AVERAGE
E( X ) = m
SD(X ) = s
n
For all four types of problems, multiply the
SD by the finite population correction factor
if drawing without replacement.
N n
N 1