Confidence Intervals
Lecture XXI
Interval Estimation
• As we discussed when we talked about
continuous distribution functions, the
probability of a specific number under a
continuous distribution is zero.
• Thus, if we conceptualize any estimator, either a
nonparametric estimate of the mean or a
parametric estimate of a function, the
probability of the true value equal to the
estimated value is obviously zero.
• Thus, usually talk about estimated values in
terms of confidence intervals. Specifically, as in
the case when we discussed the probability of a
continuous variable, we define some range of
outcomes.
Confidence Intervals
• Amemiya notes a difference between confidence
and probability. Most troubling is our classic
definition of probability as “a probabilistic
statement involving parameters.” This is
troublesome due to our inability without some
additional Bayesian structure to state anything
concrete about probabilities.
• Example 8.2.1. Let Xi be distributed as a
Bernoulli distribution, i=1,2,…,n. Then,
T  X ~ N  p, p(1  p) / n 
A
Therefore, we have
Z
Tp A
~ N 0,1
p1  p 
n
• Why? By the Central Limit Theory
• Given this distribution, we can ask questions
about the probability. Specifically, we know that
if Z is distributed N(0,1), then we can define
 k  P Z  k 
Table of Normal Probabilities
k
1.0000
1.5000
1.6449
1.7500
1.9600
2.0000
2.3263

0.1587
0.0668
0.0500
0.0401
0.0250
0.0228
0.0100

0.3173
0.1336
0.1000
0.0801
0.0500
0.0455
0.0200
• The values of k can be derived from the
standard normal table as


 Tp

P
 k   k
 p1  p 



n
• Assuming that the sample value of T is t, the
confidence is defined by


 t p

C
 k   k
 p1  p 



n

• Building on the first term


 Tp


p 1  p  


P
 k  P T  p  k

n
 p 1  p 





n

p1  p  

2
 P T  p   k 2
n 

 2

k2
k2 2
2
 P T  2Tp  p 
p
p  0
n
n


 2 k 2 
 P  p 1   
n
 


k2 
2
p 2T    T  0   k
n


• Using this probability, it is possible to define two
numbers h1(T) and h2(T) for which this
inequality holds.
Ph1 T   p  h2 T    k where
2
2



k
k
k  2
2T    2T    41  T
n
n
n


h1 T , h2 T  
 k2 
21  
n

2
2
Table 1. Sample Statistics for T for 3 Draws
Draw 1
Draw 2
Statistic
p=.4
p=.6
p=.4
p=.6
T
0.300
0.500
0.480
0.600
S
0.463
0.505
0.505
0.495
T(1-T)/50
0.030
0.035
0.035
0.034
T
0.320
0.500
0.430
0.590
S
0.469
0.503
0.498
0.494
T(1-T)/100
0.022
0.025
0.025
0.024
T
0.347
0.547
0.393
0.587
S
0.478
0.499
0.490
0.494
T(1-T)/150
0.018
0.020
0.019
0.020
T
0.360
0.570
0.400
0.605
S
0.481
0.496
0.491
0.490
T(1-T)/200
0.016
0.017
0.017
0.017
Draw 3
p=.4
p=.6
0.480
0.660
0.505
0.479
0.035
0.032
0.380
0.590
0.488
0.494
0.024
0.024
0.380
0.593
0.487
0.493
0.019
0.020
0.380
0.575
0.487
0.496
0.017
0.017
Table 2. Empirical Confidence Intervals for Samples
Draw 1
Draw 2
Draws
p=.4 p=.6
p=.4 p=.6
50 Lower Bound
0.191 0.366 0.348 0.462
Upper Bound
0.438 0.634 0.615 0.724
100 Lower Bound
0.237 0.404 0.337 0.492
Upper Bound
0.417 0.596 0.528 0.681
150 Lower Bound
0.275 0.467 0.319 0.507
Upper Bound
0.426 0.624 0.473 0.662
200 Lower Bound
0.297 0.501 0.335 0.536
Upper Bound
0.429 0.637 0.469 0.670
Draw 3
p=.4 p=.6
0.348 0.522
0.615 0.776
0.291 0.492
0.478 0.681
0.306 0.513
0.460 0.669
0.316 0.506
0.449 0.642
• Example 8.2.2. Let Xi ~ N(m,s2), i=1,2,…n where
m is unknown and s2 is known. We have

T  X ~ N m ,s
2
n


T m

P
 k   k
 s2



 n

• Example 8.2.3. Suppose that Xi~N(m,s2),
i=1,2,3,…n with both m and s2 unknown. Let
TX
be an estimator of m and
1 n
2
S  i 1  X i  X 
n
2
be the estimator of s2.
• Then the probability distribution
tn1  S
1
T 1
n 1
or the Student’s t distribution with n-1 degrees
of freedom is the probability distribution.
• Theorem 5.4.1: Let X1, X2,…Xn be a random
sample from a N(m,s2) distribution, and let
1 n
X  i 1 X i
n
1
n
2
X i  X 
S 

i 1
n 1
2
•
X and S2 are independent random variables
•
X ~ N m ,s

2
n
• (N-1)S2/s2 has a chi-squared distribution with
n-1 degrees of freedom.
• The proof of independence is based on the fact
that S2 is a function of the deviations from the
mean which, by definition, must be independent
of the mean.
• The chi-square is defined as:
f x  
1
p

2

 2
x
p
p 1  x
2
2
e
2
• In general, the gamma distribution function is
defined through the gamma function:

    t
0
 1 t
e dt
1
1

  0
 1  t
t
e
 1 t
t e dt  f t  
 
• Substituting X=bt gives the traditional two
parameter form of the distribution function
f x  , b  
1
 b

x
x

 1
b
e
• The expected value of the gamma distribution is
b and the variance is b2.
• Lemma 5.4.1. (Facts about chi-squared random
variables) We use the notation cp2 to denote a
chi-squared random variable with p degrees of
freedom.
▫ If Z is a N(0,1) random variable, the Z2~c12, that is,
the square of a standard normal random variable
is a chi-squared random variable.
▫ If X1, X2,…Xn are independent, and Xi~cpi2, then
X1+X2+…Xn~c2p1+p2+…pn, that is, independent chisquared variables add to a chi-square variable, and
the degrees of freedom also add.
• The first part of the Lemma follows from the
transformation of random variables for Y=X2
which yields:
fY  y  
1
2

f  y  f  y 
y
X
X
• Returning to the proof at hand, we want to show
that (N-1)S2/s2 has a chi-squared distribution
with n-1 degrees of freedom. To demonstrate
this, note that
 n  1 S
2
n
  n  2 S
2
n 1
2
 n 1 

  X n  X n 1 
 n 
1
n 
1 n

2
Sn 
X

X
  i n  j 1 j 
n  1 i 1 
2
2
1 n 1
1
n 1 
1 n 1
1


2
 n  1 Sn   X n   j 1 X j  X n    i 1  X i   j 1 X j  X n 
n
n
n
n




2
n  1
  n  1


n 1 
1 n 1  1 
2
n

1
S

X

X

X

X j   Xn 
  n 


n
n 1 
i
i 1 
j 1
n
n
 n 

 n

2
2
• If n=2, we get
1
2
S  X 2  X1 
2
2
2
• Given that (X2-X1)/√2 is distributed N(0,1),
S22~c12 and by extension for n=k, (k-1)Sk2~ck-12.
• Given these results for the chi-square, the
distribution of the Student’s t then follows.
X m

S
n
X  m 
S
s
n
2
s
2

• Note that this creates a standard normal random
variable in the numerator and
c
2
n 1
in the denominator
n 1
• The complete distribution found by multiplying
the standard normal time the chi-squared
distribution times the Jacobian of the
transformation yields:
 p2


1
2  1

fT t  
1
 p 1
p
   p  2 
2
2

 
t 
1


p
2
