Chapter 9
Bootstrap Confidence Intervals
William Q. Meeker and Luis A. Escobar
Iowa State University and Louisiana State University
9-1
Copyright 1998-2008 W. Q. Meeker and L. A. Escobar.
Based on the authors’ text Statistical Methods for Reliability
Data, John Wiley & Sons Inc. 1998.
December 14, 2015
8h 9min
Bootstrap Sampling and
Bootstrap Confidence Intervals
9-3
b − µ)/sc
• Instead of assuming Zµb = (µ
eµb ∼
˙ NOR(0, 1), use
Monte Carlo simulation to approximate the distribution of
Zµb.
b ∗ − µ)/
b sc
• Simulate B = 4000 values of Zµb∗ = (µ
eµb∗ .
• Some bootstrap approximations:
◮ Zµb ∼
˙ Zµb∗
◮ Zlog(σb) ∼
˙ Zlog(σb∗)
◮ Zlogit[Fb(t)] ∼
˙ Zlogit[Fb∗(t)]
Simulated Censored Samples From
DATA*1
(Draw B samples, each of size n)
DATA
^*
θ
1
^)
F(t; θ
when computing confidence intervals for µ, σ, and F .
n units
^)
F(t; θ
Actual Sample Data From
Population or Process
(Used to Estimate Model Parameters)
A Simple Parametric Bootstrap
Sampling Method
Population or Process
F(t; θ)
DATA*2
^*
θ
2
.
.
.
DATA*B
^*
θ
B
9-5
Bootstrap Confidence Intervals
Chapter 9 Objectives
• Explain basic ideas behind the use of computer simulation
to obtain bootstrap confidence intervals.
• Explain different methods for generating bootstrap samples.
^*
θ
1
DATA*1
9-4
9-2
• Obtain and interpret simulation-based pointwise parametric
bootstrap confidence intervals.
n units
^
θ
^*
θ
B
DATA*B
.
..
^*
θ
2
DATA*2
DATA
Actual Sample Data From
Resample with Replacement from DATA
Population or Process
(Draw B Samples, each of Size n)
(Used to Estimate Model Parameters)
A Simple Bootstrap Re-Sampling Method
F(t; θ)
Population or Process
0.8
0.6
0.4
µ
•
•
11.0
11.5
•
Scatterplot of 1,000 (Out of B =10,000) Bootstrap
b ∗ and σ
b ∗ for Shock Absorber
Estimates µ
σ
0.2
10.5
•
•
•
•
•
•
•
•
•
•
•• •
•
•
•
• •• •
••
••• • • • • • • •• • •
• •
•
•
•
• • •••• ••••••••••••••••••• • • •••• • •• •
• • • •
• • •
•
••• ••• • • • ••••••••••••••••••••••••••••••••• •••••• ••• • •••
• •• •
• ••• • ••••••••• ••••• •••••••••••••• ••••••• •• ••••
•
• •
•
• •
••• ••• • •••••••••••••••••••••••••••••••• ••••••••• •••• •• ••• ••
• •• • •• •••••••••••••••••••••••••••••• ••••• ••••••••••• • •• • •• •
•• •••• •••••••••••••••••••••••••••••••••••••••••••• ••• ••• •••••••••••• •
•
• •••••• ••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••
••
• • • •••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••• • •• ••
• • •• • • •
• •• •••••••••••••••••••
•
•••• •••••••••••••••••••••••••••••• •••
•
•
• •• ••••••••••••••••••••••••••••••••••• ••• •
• • • •••• • •• •• •
•
• ••••• ••••••••••••• ••••
• ••
••
• ••
10.0
9-6
10000
15000
Kilometers
20000
25000
9-7
b σ
b ) from the Original Sample
Weibull Plot of F (t; µ,
b ∗, σ
b ∗)
(dark line) and 50 (Out of B =10,000) F (t; µ
Computed from Bootstrap Samples for the
Shock Absorber
.7
.5
.3
.2
.1
.05
.03
.02
.01
.005
.003
.001
10.0
1
.5
0
muhat*
10.4
10.6
Bootstrap Estimates
10.2
-2
Z-muhat*
-1
0
1
Bootstrap-t Untransformed
-3
-2
Z-muhat*
-1
0
1
Bootstrap-t Untransformed
-3
9-9
b ∗ and Zµ
Bootstrap Distributions of Weibull µ
b∗ Based on
B=10,000 Bootstrap Samples for the Shock Absorber
.0005
Proportion Failing
0.4
sigmahat*
0.3
0.5
Bootstrap Estimates
0.2
-1
0
Z-log(sigmahat*)
-2
1
2
Bootstrap-t log-transform
-3
Bootstrap cdf
0
.5
1
-3
-2
Z-sigmahat*
-1
0
0
Z-log(sigmahat*)
-2
1
2
9 - 11
2
Bootstrap-t log-transform
-4
Bootstrap-t Untransformed
b ∗ , Zσ
Bootstrap Distributions of σ
b ∗ , and Zlog(σ
b ∗) Based
on B=10,000 Bootstrap Samples
Bootstrap cdf
Bootstrap Confidence Interval for µ
j
sceµb∗
b j∗ − µ
b
µ
• With complete data or Type II censoring,
j
Zµb∗ =
(1−α/2)
sceµb]
=1−α
(α/2)
b − zµ
µ
b∗
has a distribution that does not depend on any unknown
parameters. Such a quantity is called a pivotal quantity.
j
sceµb,
< Zµb∗ ≤ zµb∗
• By the definition of quantiles, then
(α/2)
Pr zµb∗
(1−α/2)
b − zµ
µ̃] = [µ
b∗
• Simple algebra shows that
e
[µ,
9-8
provides an exact 95% confidence interval for µ. With other
kinds of censoring, the interval is, in general, only approximate.
Bootstrap Confidence Interval for σ
b ∗) − log(σ
b)
log(σ
scelog(σb∗)
• With complete data or Type II censoring,
Zlog(σb∗) =
j
b /w]
e
σ
< Zlog(σb∗) ≤ zlog(σb∗)
e
b /w ,
σ̃] = [σ
(1−α/2)
=1−α
has a distribution that does not depend on any unknown
parameters. Such a quantity is called a pivotal quantity.
(α/2)
• By the definition of quantiles, then
Pr zlog(σb∗)
e
[σ ,
• Simple algebra shows that
(1−α/2)
e = exp zlog(σ
scelog(σb) and w
b∗)
(α/2)
scelog(σb)
e
provides an exact 95% confidence interval for σ, where w =
exp zlog(σb∗)
9 - 10
With other kinds of censoring, the interval is, in general,
only approximate.
Bootstrap Confidence Interval for F (te)
logit(Fb ∗) − logit(Fb )
scelogit(Fb∗)
• With complete data or Type II censoring [using F = F (te)],
Zlogit(Fb∗) =
j
< Zlogit(Fb∗) ≤ zlogit(Fb∗)
(α/2)
(1−α/2)
!

=1−α
has a distribution that does not depend on any unknown
parameters. Such a quantity is called a pivotal quantity.
(α/2)
• By the definition of quantiles, then
Pr zlogit(Fb∗)

• Simple algebra shows that
∗
Fb
Fb


,
[F , F̃ ] = 

Fb + (1 − Fb ) × w
Fb + (1 − Fb ) × w
e
e
e
where provides an exact 95% confidence interval for F , where w =
e
i
i
h
h
∗
(1−α/2)
9 - 12
b
b
se
e = exp zlogit(Fb ) se
exp zlogit(Fb )
b) With other
b) and w
logit(F
logit(F
kinds of censoring, the interval is, in general, only approximate.
0.06
0.10
Bootstrap Estimates
0.02
F(10000)hat*
0
1
2
3
0
.5
1
-2
-15
-10
-5
Z-F(10000)hat*
-1
0
1
2
Z-logit(F(10000)hat*)
0
9 - 13
Bootstrap-t logit-transformed
-20
Bootstrap-t Untransformed
Bootstrap Distributions of Fb (te)∗, ZFb(t )∗ , and
e
Zlogit[Fb(t )∗] for te=10,000 km Based on B=10,000
e
Bootstrap Samples
0.0
-1
Bootstrap-t logit-transformed
-2
Z-logit(F(10000)hat*)
14000
18000
Bootstrap Estimates
10000
t0.1hat*
2
Z-log(t0.1hat*)
0
4
Bootstrap-t log-transform
-2
-2
-1
-1
0
1
Z-t0.1hat*
0
1
2
2
Z-log(t0.1hat*)
3
3
4
9 - 15
Bootstrap-t log-transform
-2
Bootstrap-t Untransformed
1
.5
0
4
te=10,000 km Based on B=10,000 Bootstrap Samples
Bootstrap Distributions of tbp∗, Zbt∗ , and Zlog[bt∗] for
p
p
Bootstrap cdf
Bootstrap cdf
Bootstrap Confidence Interval for tp
p
scelog(bt∗)
log(tbp∗) − log(tbp)]
• With complete data or Type II censoring,
p
Zlog(bt∗) =
e
tbp/w]
p (1−α/2)
=1−α
has a distribution that does not depend on any unknown
parameters. Such a quantity is called a pivotal quantity.
p j
t˜p] = [tbp/w,
e
< Zlog(bt∗) ≤ zlog(bt∗)
• By the definition of quantiles, then
p (α/2)
Pr zlog(bt∗)
e
[tp,
• Simple algebra shows that
p (1−α/2)
p
p (α/2)
e = exp z
scelog(bt ) and w
log(b
t∗ )
p
scelog(bt )
e
provides an exact 95% confidence interval for tp, where w =
exp zlog(bt∗)
9 - 14
With other kinds of censoring, the interval is, in general,
only approximate.