Statistics 512 Notes 2
Confidence Intervals
Definition: For a sample X 1 , , X n from a model
{P , } , a (1   ) confidence interval for a parameter
g ( ) is an interval Cn  [a( X1 ,..., X n ), b( X1 ,..., X n )] such
that
P ( g ( )  Cn )  1   for all   .
In words, Cn is a function of the random sample that traps
the parameter g ( ) with probability at least (1   ) .
Commonly, people use 95% confidence intervals which
corresponds to choosing   0.05 .
Example: Suppose X1 , X 2 , X 3 , X 4 are iid N (  ,1) . The
interval C  [ X  1, X  1] is a 0.9544 confidence interval
for  :
P (   [ X  1, X  1])  P( X  1    X  1)
=P( 1  X    1)
X 
 2)
1/ 4
=P( 2  Z  2)
=0.9544
=P( 2 
Motivation for confidence intervals:
A confidence interval can be thought of as an estimate of
the parameter, i.e., we estimate  by [ X  1, X  1] rather
than the point estimate X .
What is gained by the using interval rather than the point
estimate since the interval is less precise?
We gain confidence. We have the assurance that in 95.44%
of repeated samples, the confidence interval will contain
.
In practice, confidence intervals are usually used along
with point estimates to give a sense of the accuracy of the
point estimate.
Interpretation of confidence intervals
A confidence interval is not a probability statement about
g ( ) since  is a fixed parameter, not a random variable.
Common textbook interpretation: If we repeat the
experiment over and over, a 95% confidence interval will
contain the parameter 95% of the time. This is correct but
not particularly useful since we rarely repeat the same
experiment over and over.
More useful interpretation (Wasserman, All of Statistics) :
On day 1, you collect data and construct a 95 percent
confidence interval for a parameter 1 . On day 2, you
collect new data and construct a 95 percent confidence
interval for an unrelated parameter  2 . On day 3, you
collect new data and construct a 95 percent confidence
interval for an unrelated parameter 3 . You continue this
way constructing 95 percent confidence intervals for a
sequence of unrelated parameters 1 , 2 ,
Then 95
percent of your intervals will trap the true parameter value.
Confidence interval is not a probability statement about  :
The fact that a confidence interval is not a probability
statement about  is confusing. Let  be a fixed, known
real number and let X1 , X 2 be iid random variables such
that P( X i  1)  P( X i  1)  1/ 2 . Now define
Yi    X i and suppose we only observe Y1 , Y2 . Define the
following “confidence interval” which actually contains
only one point:
{Y1  1} if Y1  Y2
C
{(Y1  Y2 ) / 2} if Y1  Y2
No matter what  is, we have P (  C )  3/ 4 so this is a
75 percent confidence interval. Suppose we now do the
experiment and we get Y1  15 and Y2  17 . Then our 75
percent confidence interval is {16}. However, we are
certain that  is 16.
Some common confidence intervals
1. CI for mean of normal distribution with known variance:
X 1 , , X n iid N (  ,  2 ) where  2 known.
X 
Then 
~ N (0,1)
n
1
Let z   ( ) where  is the CDF of a standard normal
random variable, e.g., z.975  1.96 . We have

X 
1    P   z  
z 
1
1

2
2

n










P   z 
 X    z 
X
1
n
2
 1 2 n



 
P  X  z 
 X z 

1
1
n
n
2
2


Thus, X  z1 
2

(1   ) CI for 
n is a
2. CI for mean of normal distribution with unknown
variance.
X 1 , , X n iid N (  ,  2 ) where  2 unknown.
X 
T
S
Key fact: The random variable
, where
n
1
n
2
S2 
(
X

X
)

i
, has a Student’s t-distribution
n  1 i 1
with n-1 degrees of freedom. (Section 3.6.3, page 186)
Let t ,n be the inverse of the CDF of the Student’s tdistribution with n degrees of freedomevaluated at  .
Note t  t1
Following the same steps as above, we have


X


1    P  t  
 t   
1 , n
2 
 1 2 ,n S
n




S
S
P  t 
 X    t 
X
1 , n
n
2
 1 2 ,n n


S
S 
P  X  t 
   X t 

1 , n
1 , n
n
n
2
2


S
X

t

(1   ) CI for 
Thus,
1 , n
n is a
2
Note: t1  ,n  z1  so we pay a price for not knowing the
2
2
variance but as n  , t1 ,n  z1 .
2
2
3. CI for mean of iid sample from unknown distribution:
Central Limit Theorem (Theorem 4.4.1): For an iid sample
from a distribution that has mean  and positive variance
X 
Yn  n
2
 , the random variable

converges in
n
distribution to a standard normal random variable.
Slutsky’s Theorem (Theorem 4.3.5):
D
P
P
D
X n  X , An  a, Bn  b, then An  Bn X n  a  bX .
4
From the weak law of large numbers, if E ( X )  
P
1
n
2
2
2
S 
(
X

X
)



i
.
n  1 i 1
Thus, combining Slutsky’s Theorem and the central limit
theorem,
Xn   D
 N (0,1)
S
n
S
X

z
An approximate (1   ) CI for  is n 1 / 2 n because


X


1    P   z  
 z   
1
S
2 
 1 2
n




S
S
P   z 
 X    z 
X
1
n
2
 1 2 n


S
S 
P  X  z 
 X z 

1
1
n
n
2
2


Application: A food-processing company is considering
marketing a new spice mix for Creole and Cajun cooking.
They interview 200 consumers and find that 37 would
purchase such a product. Find an approximate 95%
confidence interval for p, the true proportion of buyers.