Chapter 12
Prediction of Future Random Quantities
William Q. Meeker and Luis A. Escobar
Iowa State University and Louisiana State University
12 - 1
Copyright 1998-2008 W. Q. Meeker and L. A. Escobar.
Based on the authors’ text Statistical Methods for Reliability
Data, John Wiley & Sons Inc. 1998.
December 14, 2015
8h 9min
Introduction
Motivation: Prediction problems are of interest to consumers, managers, engineers, and scientists.
• A consumer would like to bound the failure time of a product to be purchased.
• Managers want to predict future warranty costs.
• Engineers want to predict the number of failures in a future
life test.
12 - 3
• Engineers want to predict the number of failures during the
following time period (week, month, etc.) of an ongoing
life test experiment.
New-Sample Prediction
r Failures
tc
∞
Based on previous (possibly censored) life test data, one
could be interested in:
0
?
• Time to failure of a new item.
n Units
at Start
Future Unit(s)
0
• Time until k failures in a future sample of m units.
12 - 5
• Number of failures by time tw in a future sample of m
units.
Prediction of Future Random Quantities
Chapter 12 Objectives
• Describe problem background and motivation, and some
general prediction problem.
• Define probability prediction, naive statistical prediction,
and coverage probability.
• Discuss calibrating statistical prediction intervals and pivotal
methods.
12 - 2
• Illustrate prediction of the number of future field failures
◮ Single cohort
◮ Multiple cohorts
• Extensions.
Related Literature
• Surveys and methods: Hahn and Nelson (1973), Patel
(1989), Hahn and Meeker (1991).
• Analytical frequentist theory: Cox (1975), Atwood (1984).
• Simulation/bootstrap frequentist theory: Beran (1990),
Bai, Bickel, and Olshen (1990), Efron and Tibshirani (1993).
12 - 4
• Log-location-scale distributions with failure (Type II)
censored data—frequentist approach: Faulkenberry (1973),
Lawless (1973), Nelson and Schmee (1979), Engelhardt and
Bain (1979), Mee and Kushary (1994).
• Likelihood theory: Kalbfleisch (1971).
• Bayesian theory: Geisser (1993).
Within-Sample Prediction
Predict future events in a process based on early data from
the process. Followed n units until tc and observed r failures.
Data are first r of n failure times: t(1) < . . . < t(r).
Want to predict:
0
r Failures
tc
?
tw
∞
• Number of additional failures in interval [tc, tw ).
n Units
at Start
• Time of next failure.
• Time until k additional failures.
12 - 6
Needed for Prediction
In general to predict one needs:
• A statistical model to describe the population or process
of interest. This model usually depends on a set of parameters θ .
• Information on the values of the parameters θ . This information could come from
◮ laboratory test.
◮ field data.
• Nonparametric new-sample prediction also possible (e.g.,
Chapter 5 of Hahn and Meeker 1991).
12 - 7
Example 1: Probability Prediction for Failure Time of
a Single Future Unit Based on Known Parameters
• Assume cycles to failure follows a lognormal distribution
with known parameters µ = 4.160, σ = .5451
Te ] = [tα/2,
t1−α/2]
• A 90% probability prediction interval is
e
P I(α) = [T ,
157.1] .
= [exp(4.160 − 1.645 × .5451), exp(4.160 + 1.645 × .5451)
= [26.1,
e
• Then Pr(T ≤ T ≤ Te ) = Pr(26.1 ≤ T ≤ 157.1) = .90.
12 - 9
• With misspecified parameters, coverage probability may not
be .90.
Coverage Probabilities Concepts
• Conditional coverage probability for the interval:
b and [T ,
For fixed DATA (and thus fixed θ
Te ]):
e
b.
Te ] depends on θ
Te ] because F (t; θ ) depends on θ .
e
= F (Te ; θ ) − F (T ; θ )
e
b ; θ ] = Pr(T ≤ T ≤ Te | θ
b ; θ)
CP[P I(α) | θ
Unknown given [T ,
e
Random because [T ,
e
o
• Unconditional coverage probability for the procedure:
ne
CP[P I(α); θ ] = Pr(T ≤ T ≤ Te ; θ )
θ
b ; θ] .
= Eb CP[P I(α) | θ
In general CP[P I(α); θ ] 6= 1 − α.
12 - 11
• When CP[P I(α); θ ] does not depend on θ , P I(α) is an exact
procedure.
Probability Prediction Interval
(θ Known)
Te ] = [tα/2,
e
= 1 − α.
t1−α/2]
α/2
= Pr(tα/2 ≤ T ≤ t1−α/2)
1−α
t 1−α/2
12 - 8
• An exact 100(1 − α)% probability prediction interval is (ignoring any data)
e
P I(α) = [T ,
where tp = tp(θ ) is the pth quantile of T .
• Probability of coverage:
α/2
t α/2
Pr[T ∈ P I(α)] = Pr(T ≤ T ≤ Te )
0
Statistical Prediction Interval
(θ Unknown)
Objective: Want to predict the random quantity T based on
a learning sample information (DATA).
e
joint distribution that depends on a parameter θ .
e
b and predicThe random DATA leads parameter estimate θ
tion interval P I(α) = [T , Te ]. Thus [T , Te ] and T have a
Probability of coverage: P I(α) is an exact 100(1 − α)%
prediction interval procedure if
e
Pr[T ∈ P I(α)] = Pr(T ≤ T ≤ Te ) = 1 − α.
First we consider evaluation, then specification of P I(α).
12 - 10
One-Sided and Two-Sided Prediction Intervals
e
α/2
T
~
e
1−α
~
T
α/2
Pr(T ≤ T ≤ Te ) = 1 − α.
Pr(0 < T ≤ Te ) = 1 − α/2,
Pr(T ≤ T < ∞) = 1 − α/2
and
• Combining lower and upper 100(1−α/2)% prediction bounds
gives an equal-probability two-sided 100(1 − α)% prediction
interval.
• If
then
0
12 - 12
Naive Statistical Prediction Interval
Te ] = [tbα/2,
tb1−α/2]
• When θ is unknown, a naive prediction interval is
e
P I(α) = [T ,
b ) is the ML estimate of the p quantile of T .
where tbp = tp(θ
0.2
alpha_upper_cal= 0.9997
alpha_nom= 0.05
alpha_lower_cal= 0.02225
0.8
(number of bootstrap samples= 1000 )
0.6
alpha_cal
0.4
1.0
12 - 15
12 - 13
• Coverage probability may be far from nominal 1 − α, especially with small samples.
CP[PI(alpha_cal);thetahat]
ej
12 - 17
• To obtain a PI with a coverage probability of 100(1 − α)%,
b ] = 1 − α.
find αc such that CP[P I(αc); θ
b.
CP [P I(α0); θ ] at θ
ej
compare with the simulated Tj∗. The proportion of the B
trials having Tj∗ > T ∗ gives the Monte Carlo evaluation of
• Use α0 to compute T ∗ = tbα0 from simulated DATAj∗ and
b from simulated DATA∗.
• Compute ML estimates θ
j
j
∗
b to simulate
• Use the assumed model and ML estimates θ
the sampling and prediction process by computing DATAj∗
and Tj∗, j = 1, . . . , B for a large number B (e.g., B = 4000
or B = 10000). For each simulated sample/prediction:
To evaluate the coverage probability of P I(α0) for some
specified 0 < α0 < 1, do the following:
Simulation of the Sampling/Prediction Process
0.0
Prediction interval calibration curve
lognormal model
1.0
0.8
0.6
0.4
0.2
0.0
Asymptotic Approximation for CP[P I(α); θ ]
i
∞], we have
As suggested by Cox (1975) and Atwood (1984):
e
he
i
b ; θ)
= Pr(T ≤ T < ∞; θ ) = g(α, θ
• For the naive lower prediction bound:
b ),
P I(α) = [T , ∞] = [tbα, ∞] = [tα(θ
h
b; θ
CP P I(α) | θ
θ
b ; θ) .
CP [P I(α); θ ] = Eb g(α, θ
θ
Eb θ̂ − θ
i
= a(θ ) + o(1/n)
= B (θ ) + o(1/n).
i
θ
k
k
2
b
X
∂g(α,
θ
;
θ
)
∂
g(α,
θb ; θ ) 1 X
1
ai
bij
+
+o
n i=1
∂ θbi
∂ θbi ∂ θbj θ 2n i,j=1
• Under regularity conditions, using a Taylor expansion of
b ; θ ), it follows that
g(α, θ
CP[P I(α); θ ] = α+
h
12 - 14
where ai, bij are elements of vector a and matrix B defined by
θ
Eb (θ̂ − θ )(θ̂ − θ )′
These are, in general, difficult to compute.
Calibrating One-Sided Prediction Bounds
• To calibrate the naive one-sided prediction bound, find αc,
such that
e
b = 1 − α.
= Pr tbαc ≤ T ≤ ∞; θ
h
b ] = Pr T ≤ T ≤ ∞; θ
b
CP[P I(αc); θ
e
where T = tbαc is the ML estimator of the tαc quantile of T .
• Can do this analytically or by simulation.
12 - 16
• When for arbitrary α, CP[P I(α); θ ] does not depend on θ ,
the calibrated P I(αc) procedure is exact.
• For a two-sided interval, do separately for each tail.
The Effect of Calibration
Result: Beran (1990) showed that, under regularity conditions, with P I(αc) being a once-calibrated prediction,
CP[P I(αc); θ ] = 1 − α + O 1/n2
and that the order of the approximation can be improved
by iterating the calibration procedure.
12 - 18
•
•
•
••
••
••
•
•
•
100
200
500
12 - 19
Lognormal probability plot of bearing life test data
censored after 80 million cycles with lognormal ML
estimates and pointwise 95% confidence intervals
.98
.9
.95
.8
.7
.6
.5
.4
.3
•
50
Millions of Cycles
Upper
0.99
[26.1,
174.4]
157.1]
12 - 21
-5
0
Z.logT*
5
10
0.0
0.2
0.6
(Z.logT*)
0.4
0.8
1.0
Example 2: Lower Prediction Bound for a
Single Independent Future T
Based on Time-Censored (Type I) Data
e
T = tb.036 = exp[4.160 + (−1.802)(.5451)] = 24.0
where z.036 = −1.802.
n Units
at Start
0
r Failures
tc
?
tw
∞
12 - 24
• Prediction problem: Find an upper bound for the number
of future failures, K, in the interval (tc, tw ], tc < tw .
• The sample DATA are singly time-censored (Type I)
from F (t). Observe n units until time tc. Failure times
are recorded for the r > 0 units that fail in (0, tc]; n − r
unfailed at tc.
Within-Sample Prediction
Predict Number of Failures in Next Time Interval
12 - 22
• Thus the calibrated lower 95% lognormal prediction bound
is
b ] = .95
• From simulation CP[P I(1 − .964); θ
• Need to calibrate to account for sampling variability in the
parameter estimates.
• The naive one-sided lower 95% lognormal prediction bound
(assuming no sampling error) is:
tb.05 = exp[4.160 + (−1.645)(.5451)] = 26.1.
• Life test run for 80 million cycles; 15 of 23 ball bearings
b=
failed. ML estimates of the lognormal parameters are: µ
b = .5451.
4.160, σ
12 - 20
Simulation of the bearing life test censored after 80
million cycles (n = 23 and r = 15), lognormal model,
b ∗)/σ
b∗
histograms of pivotal–like Zlog(T ∗) = (log(T ∗) − µ
and Φ[Zlog(T ∗)]
-10
200
.2
•
20
Lower
0.95
0.97
number of simulated samples= 100000
1-alpha= 0.95
1-alpha_cal_upper= 0.967
1-alpha_cal_lower= 0.964
0.93
1 - alpha_cal
Naive
[24.0,
Lognormal
Calibrated
12 - 23
1000
800
600
0
400
4000
3000
1000
2000
0
.1
.05
.02
10
0.99
0.97
0.95
0.93
Comparison of Approximate 90% Prediction Intervals
for Bearing Life from a Life Test that was
Type I Censored at 80 Million Cycles
CP[PI(1-alpha_cal);thetahat]
Prediction interval calibration function for the bearing
life test data censored after 80 million cycles,
lognormal model
Proportion Failing
Distribution of K and Naive Prediction Bound
Pr(tc < T ≤ tw )
F (tw ; θ ) − F (tc; θ )
=
.
Pr(T > tc)
1 − F (tc; θ )
K ∼ BIN (n − r, ρ)
• Conditional on DATA, the number of failures K in (tc, tw ]
is distributed as
where
ρ=
b.
• Obtain ρ̂ by evaluating at θ
12 - 25
• The naive 100(1 − α)% upper prediction bound for K is
f
K(1
− α) = K̂1−α, the estimate of the 1 − α quantile of the
distribution of K. This is computed as the smallest integer
such that
BINCDF(K, n − r, ρ̂) > 1 − α.
h
i
Calibration of the Naive Upper-Prediction Bound
for the Number of Field Failures
0.90
B
1 X
Pj
B j=1
Upper
0.96
0.98
number of simulated samples= 100000
1-alpha= 0.95
1-alpha_cal_upper= 0.986
1-alpha_cal_lower= 0.981
Lower
0.94
1 - alpha_cal
0.92
1.00
Example 3: Prediction of the Number of Future
Failures
• n = 10, 000 units put into service; 80 failures in 48 months.
Want an upper prediction bound on the number of the
remaining
n − r = 10000 − 80 = 9920 units
that will fail between 48 and 60 months.
F̂ (60) − F̂ (48)
= .003233.
1 − F̂ (48)
• Weibull time to failure distribution assumed; ML estimates:
b = 1152, βb = 1.518 giving
α
ρb =
12 - 26
• Point prediction for the number failing between 48 and 60
months is
(n − r) × ρ̂ = 9920 × .003233 = 32.07.
0.0
Lower
0.2
0.4
0.6
1 - alpha_cal
Upper
0.8
Example 3–Computations
1.0
12 - 28
Example 3. Calibration functions for upper and lower
prediction bounds on the number of future field failures
0
^
K
32
Naive
^
K .95
42
Calibrated
^
K .986
45
Number Failing
12 - 30
• Thus the calibrated 95% upper prediction bound on K is
f = K̂
K
.9863 = 45, the smallest integer K such that
BINCDF(K, 9920, .003233) ≥ .9863.
b ] ≈ .95.
• From simulation CP[P I(.9863); θ
BINCDF(K, 9920, .003233) > .95.
• The naive 95% upper prediction bound on K is K̂.95 = 42,
the smallest integer K such that
CP[PI(alpha_cal);thetahat]
• Find αc such that
b ] = Pr K ≤ K(1
f
CP[P I(αc); θ
− αc ) = 1 − α
b] =
CP[P I(αc); θ
• A Monte Carlo evaluation of the unconditional coverage
probability is
0.88
12 - 29
1.0
0.8
0.6
0.4
0.2
0.0
where
e
Pj = BINCDF K (1 − αc)j∗; n − rj∗, ρb
12 - 27
is the conditional coverage probability for the jth simulated
b
interval evaluated at ρ.
• Similar for the lower prediction bound.
CP[PI(1-alpha_cal);thetahat]
Example 3. Calibration functions for upper and lower
prediction bounds on the number of future field failures
1.00
0.98
0.96
0.94
0.92
0.90
0.88
n1
0
r1
r2
r3
Failures Observed in the Field
Number of Failures:
0
rs
t cs
t c3
t c2
t c1
Future
K 1?
K 2?
K 3?
K s?
K?
t ws
t w3
t w2
t w1
Staggered Entry Prediction Problem
n2
0
n3
ns
0
Total Number of Future Failures:
∞
∞
∞
∞
12 - 31
Within-Sample Prediction With Staggered Entry
• The objective it to predict future events in a process based
on several sets of early data from the process.
• Units enter the field in groups over time. Need to predict
the total number of new failures (in all groups) when unfailed units are observed for an additional period of length
∆t.
• For group i, ni units are followed for a period of length tcj
and ri failures were observed, i = 1, . . . , s.
DATAi for set i (i = 1, . . . , s) are the first ri of ni failure
times, say t(i1) < · · · < t(iri).
12 - 33
Distribution of the Number of Future Failures
F (twi; θ ) − F (tcj ; θ )
Pr(tcj < T ≤ twi)
=
.
Pr(T > tcj )
1 − F (tcj ; θ )
• Conditional on DATAi, the number of additional failures Ki
in group i during interval (tcj , twi] (where twi = tcj + ∆t) is
distributed as Ki ∼ BIN (ni − ri, ρi) with
ρi =
b.
• Obtain ρbi by evaluating ρ = (ρi, . . . , ρs) at θ
P
s
• Let K = i=1
Ki be the total number of additional failures
over ∆t. Conditional on the DATA (and the fixed censoring
times) K ∼ SBINCDF(k; n − r , ρ) a sum of s independent
binomials; n − r = (n1 − r1, . . . , ns − rs) and ρ = (ρ1, . . . , ρs).
12 - 35
f
• A naive 100(1 − α)% upper prediction bound K(1
− α) is
b∗
computed as the smallest integer k such that SBINCDF(k, n − r ∗, ρ
1 − α.
Bearing-Cage Field-Failure Data
(from Abernethy et al. 1983)
• A total of 1703 units failed introduced into service over a
period of eight years (about 1600 in the past three years).
• Time measured in hours of service.
• Six out of 1703 units failed.
• Unexpected failures early in life mandated a design change.
12 - 32
• How many failures in the next year (point prediction and
upper prediction bound requested), assuming 300 hours of
service.
Hours in
Service
50
150
250
350
450
550
.
.
.
1650
1750
1850
1950
2050
ni
288
148
125
112
107
99
.
.
.
6
0
1
0
2
1703
Failed
ri
0
0
1
1
1
0
.
.
.
0
0
0
0
0
6
At Risk
ni − ri
288
148
124
111
106
99
.
.
.
6
0
1
0
2
ρbi
.000763
.001158
.001558
.001962
.002369
.002778
.
.
.
.007368
.007791
.008214
.008638
.009062
12 - 34
(ni − ri) × ρbi
.2196
.1714
.1932
.2178
.2511
.2750
.
.
.
.0442
.0000
.0082
.0000
.0181
5.057
Bearing Cage Data and Future-Failure Risk Analysis
Group
i
1
2
3
4
5
6
.
.
.
17
18
19
20
21
Total
h
i
Calibration of the Naive Upper Prediction Bound
for the Staggered Entry Number of Field Failures
• Find αc such that
b ] = Pr K ≤ K(1
f
CP[P I(αc); θ
− αc ) = 1 − α
b] =
CP[P I(αc); θ
B
1 X
Pj
B j=1
• A Monte Carlo evaluation of the unconditional coverage
probability is
where
e
b
Pj = SBINCDF K (1 − αc)j∗; n − r ∗, ρ
is the conditional coverage probability for the jth simulated
b.
interval evaluated at ρ
• Similar for the lower prediction bound.
12 - 36
Lower
Upper
0.2
0.4
0.6
0.8
1.0
5.057
^
K
9
^
K .95
Naive
12
^
K .991
Calibrated
12 - 37
12 - 39
Upper
0.94
0.96
0.98
number of simulated samples= 100000
1-alpha= 0.95
1-alpha_cal_upper= 0.991
1-alpha_cal_lower= 0.959
0.92
1.00
12 - 38
Example 4: Calibration functions for upper and lower
prediction bounds on the number of future field
failures with staggered entry
0.90
Lower
0.88
1 - alpha_cal
12 - 40
• Asymptotic theory promises good approximation when not
exact; use simulation to verify and compare with other approximate methods.
• Today, the computational price is small; general-purpose
software needed.
◮ Modeling of spatial and temporal variability in environmental factors like UV radiation, acid rain, temperature,
and humidity.
◮ Staggered entry with differences in remaining warranty
period.
◮ Staggered entry with differences among cohort distributions.
• Methodology can be extended to:
Concluding Remarks and Future Work
CP[PI(1-alpha_cal);thetahat]
Example 4: Calibration functions for upper and lower
prediction bounds on the number of future field
failures with staggered entry
0.0
1 - alpha_cal
Example 4–Computations
b ] ≈ .95.
• From simulation CP[P I(.9916); θ
b ) > .95.
SBINCDF(K, n − r , ρ
• The naive 95% upper prediction bound on K is K̂.95 = 9,
the smallest integer K such that
CP[PI(alpha_cal);thetahat]
• Thus the calibrated 95% upper prediction bound on K is
f = K̂
K
.9916 = 11, the smallest integer K such that
b ) > .9916.
SBINCDF(K, n − r , ρ
0
Number Failing
1.00
0.98
0.96
0.94
0.92
0.90
0.88
1.0
0.8
0.6
0.4
0.2
0.0