Chapter 9
Bootstrap Confidence Intervals
William Q. Meeker and Luis A. Escobar
Iowa State University and Louisiana State University
Copyright 1998-2008 W. Q. Meeker and L. A. Escobar.
Based on the authors’ text Statistical Methods for Reliability
Data, John Wiley & Sons Inc. 1998.
January 13, 2014
3h 41min
9-1
Bootstrap Confidence Intervals
Chapter 9 Objectives
• Explain basic ideas behind the use of computer simulation
to obtain bootstrap confidence intervals.
• Explain different methods for generating bootstrap samples.
• Obtain and interpret simulation-based pointwise parametric
bootstrap confidence intervals.
9-2
Bootstrap Sampling and
Bootstrap Confidence Intervals
b − µ)/sc
• Instead of assuming Zµb = (µ
eµb ∼
˙ NOR(0, 1), use
Monte Carlo simulation to approximate the distribution of
Zµb.
b ∗ − µ)/
b sc
• Simulate B = 4000 values of Zµb∗ = (µ
eµb∗ .
• Some bootstrap approximations:
◮ Zµb ∼
˙ Zµb∗
◮ Zlog(σb) ∼
˙ Zlog(σb∗)
◮ Zlogit[Fb(t)] ∼
˙ Zlogit[Fb∗(t)]
when computing confidence intervals for µ, σ, and F .
9-3
A Simple Bootstrap Re-Sampling Method
Population or Process
Actual Sample Data From
Resample with Replacement from DATA
Population or Process
(Draw B Samples, each of Size n)
(Used to Estimate Model Parameters)
n units
DATA
F(t; θ)
^θ
DATA*1
^θ*
1
DATA*2
^θ*
2
..
.
DATA*B
^θ*
B
9-4
A Simple Parametric Bootstrap
Sampling Method
Population or Process
Actual Sample Data From
Population or Process
(Used to Estimate Model Parameters)
Simulated Censored Samples From
F(t; ^θ)
(Draw B samples, each of size n)
n units
F(t; θ)
DATA
DATA*1
^
F(t; θ)
^θ*
1
DATA*2
^θ*
2
..
.
DATA*B
^θ*
B
9-5
Scatterplot of 1,000 (Out of B =10,000) Bootstrap
b ∗ and σ
b ∗ for Shock Absorber
Estimates µ
•
0.8
•
0.6
σ
0.4
0.2
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•• • •• • •• •• • •
• •
• •
•
• ••• •
•
• • •••• •••••••••••••••••••• • • •••• • •• •
• • • • •
• • •
•
••• ••• • • • •••••••••••••••••••••••••••••• •••••• ••• • •••
•
• •• •
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
• •• •
•
•
•
••• ••• • •••••••••••••••••••••••••••••••••• •••••••• •••• •• ••• ••
• ••• • •••••••••••••••••••••••••••••••••••••••••••••••• ••••••• ••••••••••••••••••••••• • •• •
•• ••• •• •••••••••• ••••••••••••• ••• •• •• • •
•
• •••••• •••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••
••
•
• ••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••• • •• ••
•••• •• • •••••• •• • •
• • • • ••••••••••
• • • ••••••••••••••••••••••••••••••••••••••••• • •• •
•
• • • ••••••• •••• •••• •• ••
•
•
•
•
•
•
•
• • ••
•
•
• ••••• ••••••••••••• ••••
•
• ••
•
• ••
10.0
10.5
•
µ
11.0
11.5
9-6
b σ
b ) from the Original Sample
Weibull Plot of F (t; µ,
b ∗, σ
b ∗)
(dark line) and 50 (Out of B =10,000) F (t; µ
Computed from Bootstrap Samples for the
Shock Absorber
.7
.5
.3
.2
Proportion Failing
.1
.05
.03
.02
.01
.005
.003
.001
.0005
10000
15000
20000
25000
Kilometers
9-7
Bootstrap Confidence Interval for µ
• With complete data or Type II censoring,
Zµb∗ =
j
b∗
b
µ
j −µ
sceµb∗
j
has a distribution that does not depend on any unknown
parameters. Such a quantity is called a pivotal quantity.
• By the definition of quantiles, then
=1−α
< Zµb∗ ≤ zµb∗
Pr zµb∗
j
(1−α/2)
(α/2)
• Simple algebra shows that
[µ,
e
b − zµ
µ̃] = [µ
b∗
(1−α/2)
sceµb,
b − zµ
µ
b∗
(α/2)
sceµb]
provides an exact 95% confidence interval for µ. With other
kinds of censoring, the interval is, in general, only approximate.
9-8
b ∗ and Zµ
Bootstrap Distributions of Weibull µ
b∗ Based on
B=10,000 Bootstrap Samples for the Shock Absorber
Bootstrap Estimates
10.0
10.2
10.4
Bootstrap-t Untransformed
10.6
-3
muhat*
-2
-1
0
1
Z-muhat*
Bootstrap-t Untransformed
Bootstrap cdf
1
.5
0
-3
-2
-1
0
1
Z-muhat*
9-9
Bootstrap Confidence Interval for σ
• With complete data or Type II censoring,
b ∗) − log(σ
b)
log(σ
Zlog(σb∗) =
scelog(σb∗)
has a distribution that does not depend on any unknown
parameters. Such a quantity is called a pivotal quantity.
• By the definition of quantiles, then
=1−α
Pr zlog(σb∗)
< Zlog(σb∗) ≤ zlog(σb∗)
(1−α/2)
(α/2)
j
• Simple algebra shows that
[σ ,
e
b /w ,
σ̃] = [σ
e
b /w]
e
σ
provides an exact 95% confidence interval for σ, where w =
e
c
e = exp zlog(σ
exp zlog(σb∗)
scelog(σb) and w
b ∗)(α/2) selog(σ
b)
(1−α/2)
With other kinds of censoring, the interval is, in general,
only approximate.
9 - 10
b ∗ , Zσ
Bootstrap Distributions of σ
b ∗ , and Zlog(σ
b ∗) Based
on B=10,000 Bootstrap Samples
Bootstrap Estimates
0.2
0.3
0.4
Bootstrap-t Untransformed
0.5
-4
sigmahat*
-2
0
2
Z-sigmahat*
Bootstrap-t log-transform
Bootstrap-t log-transform
Bootstrap cdf
1
.5
0
-3
-2
-1
0
Z-log(sigmahat*)
1
2
-3
-2
-1
0
1
2
Z-log(sigmahat*)
9 - 11
Bootstrap Confidence Interval for F (te)
• With complete data or Type II censoring [using F = F (te)],
logit(Fb ∗) − logit(Fb )
Zlogit(Fb∗) =
scelogit(Fb∗)
has a distribution that does not depend on any unknown
parameters. Such a quantity is called a pivotal quantity.
• By the definition of quantiles, then
< Zlogit(Fb∗) ≤ zlogit(Fb∗)
Pr zlogit(Fb∗)
(1−α/2)
(α/2)
j
• Simple algebra shows that
[F ,
e


F̃ ] = 
Fb
Fb
,
!
=1−α



Fb + (1 − Fb ) × w
Fb + (1 − Fb ) × w
e
e
where provides an exact 95% confidence interval for F , where w =
e
i
i
h
h
b
b
e = exp zlogit(Fb∗) se
se
exp zlogit(Fb∗ )
b) With other
b) and w
logit(F
logit(F
(1−α/2)
(α/2)
kinds of censoring, the interval is, in general, only approximate.
9 - 12
Bootstrap Distributions of Fb (te)∗, ZFb(t )∗ , and
e
Zlogit[Fb(t )∗] for te=10,000 km Based on B=10,000
e
Bootstrap Samples
Bootstrap Estimates
0.0
0.02
0.06
Bootstrap-t Untransformed
0.10
-20
F(10000)hat*
-15
-10
-5
0
Z-F(10000)hat*
Bootstrap-t logit-transformed
Bootstrap-t logit-transformed
Bootstrap cdf
1
.5
0
-2
-1
0
1
Z-logit(F(10000)hat*)
2
3
-2
-1
0
1
2
Z-logit(F(10000)hat*)
9 - 13
Bootstrap Confidence Interval for tp
• With complete data or Type II censoring,
Zlog(bt∗) =
p
log(tb∗p) − log(tbp)]
scelog(bt∗)
p
has a distribution that does not depend on any unknown
parameters. Such a quantity is called a pivotal quantity.
• By the definition of quantiles, then
< Zlog(bt∗) ≤ zlog(bt∗)
Pr zlog(bt∗)
=1−α
p (1−α/2)
p j
p (α/2)
• Simple algebra shows that
[tp,
e
t˜p] = [tbp/w,
e
e
tbp/w]
provides an exact 95% confidence interval for tp, where w =
e
e = exp z
scelog(bt ) and w
exp zlog(bt∗)
scelog(bt )
∗)
b
log(
t
p
p
p (1−α/2)
p (α/2)
With other kinds of censoring, the interval is, in general,
only approximate.
9 - 14
Bootstrap Distributions of tb∗p, Zbt∗ , and Zlog[bt∗] for
p
p
te=10,000 km Based on B=10,000 Bootstrap Samples
Bootstrap Estimates
10000
14000
Bootstrap-t Untransformed
18000
-2
-1
t0.1hat*
0
1
2
3
4
Z-t0.1hat*
Bootstrap-t log-transform
Bootstrap-t log-transform
Bootstrap cdf
1
.5
0
-2
0
2
Z-log(t0.1hat*)
4
-2
-1
0
1
2
3
4
Z-log(t0.1hat*)
9 - 15