Chapter 3 Nonparametric Estimation William Q. Meeker and Luis A. Escobar Iowa State University and Louisiana State University Year 1 2 Year 2 π3 2 Year 3 3-1 Copyright 1998-2008 W. Q. Meeker and L. A. Escobar. Based on the authors’ text Statistical Methods for Reliability Data, John Wiley & Sons Inc. 1998. December 14, 2015 8h 8min 1 π2 Cracked tubes π1 i=1 πi = 1. 2.5 95 π4 3.0 3-3 L(π ) = C × [π1]1 × [π2]2 × [π3]2 × [π4]95 4 X Uncracked tubes Data for Plant 1 of the Heat Exchanger Tube Crack Data 100 tubes at start Plant 1 Unconditional Failure Probability 0.20 0.15 2.0 0.0 Chapter 3 Nonparametric Estimation Objectives • Show the use of the binomial distribution to estimate F (t) from interval and singly right censored data, without assumptions on F (t). This is called nonparametric estimation. • Explain and illustrate how to compute standard error for Fb (t) and approximate confidence intervals for F (t). • Show how to extend nonparametric estimation to allow for multiply right-censored data (Kaplan-Meier estimator). 3-2 • Describe and illustrate a generalization that provides a nonparametric estimator of F (t) with arbitrary censoring. A Nonparametric Estimator of F (ti) Based on Binomial Theory for Interval Singly-Censored Data We consider the nonparametric estimate of F (ti) for data situations as illustrate by Plant 1 of the Heat Exchanger Tube Crack: • The data are: n : sample size . n Pi j=1 dj di : # of failures (deaths) in the ith interval n v h i u b ub t F (ti) 1 − F (ti) # of failures up to time ti = n • Simple binomial theory gives Fb (ti) = sceFb = Fb (2) = 3/100, Fb (3) = 5/100. 3-4 • For Plant 1 (n = 100, d1 = 1, d2 = 2, d3 = 2), one gets: Fb (1) = 1/100, Comments on the Nonparametric Estimate of F (ti) 3-6 • Fb (t) is defined only at the upper ends of the intervals (ti−1, ti]. Fb (ti) − Fb (ti−1) = di/n. • Fb (ti) is the ML estimator of F (ti). 1.5 Years 0.10 1.0 • The increase in Fb at each value of ti is 0.5 3-5 0.05 0.0 Nonparametric Estimate for Plant 1 from the Heat Exchanger Tube Crack Data Likelihood: Proportion Failing Confidence Intervals A point estimate can be misleading. It is important to quantify uncertainty in point estimates. • Confidence intervals are very useful in quantifying uncertainty in point estimates due to sampling error arising from limited sample sizes. 3-7 • In general, confidence intervals do not quantify possible deviations arising from incorrectly specified model or model assumptions. Pointwise Binomial-Based Confidence Interval for F (ti) 1+ n − nFb (nFb + 1)F(1−α/2;2nFb+2,2n−2nFb) • A 100(1 − α)% conservative confidence interval for F (ti) based on binomial sampling (see Chapter 6 of Hahn and Meeker, 1991) is −1 (n − nFb + 1)F(1−α/2;2n−2nFb+2,2nFb) F (ti ) = 1+ nFb e −1 F̃ (ti ) = where Fb = Fb (ti) and F(1−α/2;ν1,ν2) is the 100(1 − α/2) quantile of the F distribution with (ν1, ν2) degrees of freedom. 3-9 • This confidence interval is conservative in the sense that the actual coverage probability is at least equal to 1 − α. 0.20 0.15 0.10 0.05 0.0 0.0 0.5 1.0 Years 1.5 2.0 2.5 3 - 11 3.0 Plant 1 Heat Exchanger Tube Crack Nonparametric Estimate with Conservative Pointwise 95% Confidence Intervals Based on Binomial Theory Proportion Failing Some Characteristic Features of Confidence Intervals • The level of confidence expresses one’s confidence (not probability) that a specific interval contains the quantity of interest. • The actual coverage probability is the probability that the procedure will result in an interval containing the quantity of interest. • A confidence interval is approximate if the specified level of confidence is not equal to the actual coverage probability. 3-8 • With censored data most confidence intervals are approximate. Better approximations generally require more computations. Pointwise Normal-Approximation Confidence Interval for F (ti) F̃ (ti)] = Fb (ti) ± z(1−α/2)sceFb . • For a specified value of ti, an approximate 100(1 − α)% confidence interval for F (ti) is e [F (ti), Fb (ti) − F (ti) ∼ ˙ NOR(0, 1). sceFb 3 - 10 where z is the 1−α/2 quantile of the standard normal (1−α/2) r h i Fb (ti) 1 − Fb (ti) /n is an estimate distribution and sceFb = of the standard error of Fb (ti). ZFb = • This confidence interval is based on Year ti 1 di 1 Fb (ti ) 0.01 b se .00995 b F 2 2 0.03 0.05 b F .01706 0.02179 [ .0164, .1128 ] [ .0073, .0927 ] [ .0062, .0852 ] [−.0034, .0634 ] [ .0003, .0545 ] [−.0095, .0295 ] Pointwise Confidence Interval F (ti ) F̃ (ti ) e Calculations of the Nonparametric Estimate of F (ti) for Plant 1 from the Heat Exchanger Tube Crack Data (0 − 1] 2 95% Confidence Intervals for F (1) Based on Binomial Theory Based on ZFb ∼ ˙ NOR(0, 1) (1 − 2] 3 95% Confidence Intervals for F (2) Based on Binomial Theory Based on Z ∼ ˙ NOR(0, 1) (2 − 3] 95% Confidence Intervals for F (3) Based on Binomial Theory Based on ZFb ∼ ˙ NOR(0, 1) 3 - 12 .10 2.50 10.00 48.00 168.00 .15 3.00 12.50 48.00 263.00 600 .60 4.00 20.00 54.00 593.00 800 .80 4.00 20.00 74.00 1000 1200 .80 6.00 43.00 84.00 3 - 15 4128 2 2 2 2 2 2 2 Count 3 - 13 Integrated Circuit (IC) Failure Times in Hours Data from Meeker (1987) .10 1.20 10.00 43.00 94.00 400 1400 When the test ended at 1370 hours, there were 28 observed failures and 4128 unfailed units. 200 Integrated Circuit Failure Data After 1370 Hours Event Plot Integrated Circuit Life Test Data Note: Ties in the data. Reason? Row 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 0 Hours Comments on the Nonparametric Estimate of F (t) • Fb (t) is defined for all t in the interval (0, tc] where tc is the singly censoring time. • Fb (t) is the ML estimator of F (t). • The estimate Fb (t) is a step up function with a step of size 1/n at each exact failure time (unless there are ties). Sometimes the step size is a multiple of 1/n because there are ties on the failure times. • When there is no censoring, Fb (t) is the well known empirical cdf. 3 - 17 Nonparametric Estimator of F (t) Based on Binomial Theory for Exact Failures and Singly Right Censored Data When the number of inspections increases the width of the intervals (ti−1, ti] approaches zero and the failure times are exact. • For the integrated circuit life test data, we have: n = 4156 with 28 exact failures in 1370 hours. n n v h i u b ub t F (te ) 1 − F (te ) . # of failures up to time te For any particular te, 0 < te ≤ 1370, simple binomial theory gives Fb (te) = sceFb = 3 - 14 • Methods to obtain confidence intervals for F (te) are the same as the methods described for the interval data. 200 400 800 Hours 600 1000 1200 3 - 16 1400 Nonparametric Estimate for the IC Data with Normal Approximation Pointwise 95% Confidence Intervals Based on Zlogit(Fb) 0.012 0.010 0 Plant 1 Plant 2 Plant 3 100 100 100 1 1 2 99 98 99 2 3 97 99 95 2 95 95 Uncracked tubes π4 95 97 2 197 π3 300 All Plants 5 95 π2 95 4 2 L( π __) = C [π 1] [π 2] [ π 3] [π4] [π 3+ π 4] [π 2+ π 3+ π 4] 5 π1 4 Failure Probability Likelihood: 99 3 - 18 Pooling of the Heat Exchanger Tube Crack Data 0.0 0.002 0.004 0.006 0.008 Proportion Failing A Nonparametric Estimator of F (ti) Based on Interval Data and Multiple Censoring The combined data from the heat exchanger tube crack are multiply censored and the simple binomial method to estimate F (ti) cannot be used. 1.0 1.5 Years i with 2.0 pbj = 2.5 dj nj 3 - 23 3 - 21 3.0 3 - 19 Here we describe a more general method to compute a nonparametric estimator of F (ti). b ) Fb (ti) = 1 − S(t i 1 − pbj b ) = S(t i i h Y where # of failures (deaths) in the ith interval sample size j=1 : i−1 X j=0 rj , the risk set at ti−1 : dj − n i−1 X # of right censored obs at ti j=0 di : ni = n − ri 0.5 Nonparametric Estimate for the Heat Exchanger Tube Crack Data 0.20 0.15 0.10 0.05 0.0 0.0 Estimating the Standard Error of Fb (ti) • An estimate of the standard error, seFb , is v u r u i i h pbj uX d Fb (t ) = S(t b )t sceFb = Var . i i b j=1 nj (1 − pj ) which is known as Greenwood’s formula. i i i h h X b p j d S(t d Fb (t ) = Var b ) =S b2(t ) Var i i i b j=1 nj (1 − pj ) • Using the variance formula, one gets Proportion Failing Calculations of the Nonparametric Estimate of F (ti) for the Poooled Heat Exchanger Tube Crack Data ti 197 300 ni 5 4 di 95 99 ri 5/197 4/300 95/97 192/197 296/300 .9418 .9616 .9867 .0582 .0384 .0133 Fb (ti) 1 2/97 b ) S(t i Year 2 95 1 − pbi (0 − 1] 2 pbi (1 − 2] 97 h i 3 - 20 3 Approximate Variance of Fb (ti) i h i Qi bj . j=1 1 − p (2 − 3] h b ) and S(t b )= • Recall, Fb (ti) = 1 − S(t i i b ) . • Then Var Fb (ti) = Var S(t i . i X ∂S qbj − qj j=1 ∂qj qj b ) is • A Taylor series first-order approximation of S(t i b ) ≈ S(t ) + S(t i i where qj = 1 − pj . • Then it follows that pj j=1 nj (1 − pj ) i h i X b ) ≈ S 2(t ) Var S(t i i 3 - 22 Pointwise Normal-Approximation Confidence Interval for F (ti)-Based on Logit Transformation logit[Fb (ti)] − logit[F (ti)] ∼ ˙ NOR(0, 1). scelogit(Fb) • Generally better confidence intervals can be obtained by using the logit transformation (logit(p) = log[p/(1 − p)]) and basing the confidence intervals on Zlogit(Fb) = # ˜ Fb ) logit( = logit(Fb ) ± z(1−α/2)sceFb /[Fb (1 − Fb )] = logit(Fb ) ± z(1−α/2)scelogit(Fb) • A pointwise normal-approximation 100(1 − α)% confidence interval for logit[F (ti)] is " logit(Fb ), e since scelogit(Fb) = sceFb /[Fb (1 − Fb )]. 3 - 24 Pointwise Normal-Approximation Confidence Interval for F (ti)-Based on Logit Transformation logit−1(v) = 1 1 + exp(−v) 1 F̃ (ti)] = logit−1 logit(Fb ) ± z(1−α/2)scelogit(Fb) = , Fb Fb + (1 − Fb )/w # 1 + exp −logit(Fb ) ∓ z(1−α/2)scelogit(Fb) " Fb Fb + (1 − Fb ) × w • The confidence interval for F (ti) is obtained from the interval for logit(F ) and using the inverse logit transformation • Then [F (ti), e = 3 - 25 • The endpoints F (ti) and F̃ (ti) will always lie between 0 and 1. where w = exp{z(1−α/2)sceFb /[Fb (1 − Fb )]}. e Normal-Approximation Pointwise Confidence Intervals for the Heat Exchanger Tube Crack Data √ .0000438 = .00662 Computation of approximate 95% confidence intervals: 1 bb = • For F (1) with Fb (t1) = .0133, se F (t ) bb∼ Based on: ZFb = [Fb (t1) − F (t1)]/se ˙ NOR(0, 1). F [F (t1), F̃ (t1)] = .0133 ± 1.96(.00662) = [.0003, .0263]. e 3 - 27 b Based on: Zlogit(Fb) = [logit[Fb (t1)] − logit[F (t1)]/se ˙ NOR(0, 1). b) ∼ logit(F .0133 .0133 = [.0050, .0350 [F (t1), F̃ (t1)] = , .0133 + (1 − .0133) × w .0133 + (1 − .0133)/w e .0001639 = .0128 w = exp{1.96(.00662)/[.0133(1 − .0133)]} = 2.687816. √ [F (t2), F̃ (t2)] = [.0198, .0730] . e [F (t2), F̃ (t2)] = [.0133, .0635]. e 2 bb = • For F (2) with Fb (t2) = .0384, se F (t ) Based on: Z , b F Based on: Zlogit(Fb) , 0.20 0.15 0.10 0.05 0.0 0.0 0.5 1.0 Years 1.5 2.0 2.5 3 - 29 3.0 Heat Exchanger Tube Crack Nonparametric Estimate with Pointwise 95% Confidence Intervals Based on Zlogit(Fb) Proportion Failing " # .0133 = .0000438 300(.9867) .0000438 = .00662 " .0001639 = .0128 3 - 26 .0254 .0133 + = .0001639 300(.9867) 197(.9746) # Normal-Approximation Pointwise Confidence Intervals for the Heat Exchanger Tube Crack Data • Computation of standard errors i √ √ = (.9616)2 = = (.9867)2 i i h X pbj d Fb (t ) = S b2(t ) Var i i b j=1 nj (1 − pj ) h d Fb (t ) Var 1 i F (t1) sce b h d Fb (t ) Var 2 sceFb(t ) = 2 Year 1 ti .0133 Fb (ti ) .00662 bb se F b F .0384 b F .0582 .0128 .0187 [.0050, .0350] [.0003, .0263] [.0198, .0730] [.0133, .0635] [.0307, .1076] [.0216, .0949] 3 - 28 Pointwise Confidence Intervals Results of Calculations for Nonparametric Pointwise Confidence Intervals for F (ti) for the Heat Exchanger Tube Crack Data (0 − 1] 2 95% Confidence Intervals for F (1) Based on Zlogit(Fb) ∼ ˙ NOR(0, 1) Based on Z ∼ ˙ NOR(0, 1) (1 − 2] 3 95% Confidence Intervals for F (2) Based on Zlogit(Fb) ∼ ˙ NOR(0, 1) Based on Z ∼ ˙ NOR(0, 1) (2 − 3] 95% Confidence Intervals for F (3) Based on Zlogit(Fb) ∼ ˙ NOR(0, 1) Based on ZFb ∼ ˙ NOR(0, 1) Shock Absorber Failure Data First reported in O’Connor (1985). • Failure times, in number of kilometers of use, of vehicle shock absorbers. • Two failure modes, denoted by M1 and M2. • One might be interested in the distribution of time to failure for mode M1, mode M2, or in the overall failure-time distribution of the part. Here we do not differentiate between modes M1 and M2. We will estimate the distribution of time to failure by either mode M1 or M2. 3 - 30 10000 15000 20000 25000 ShockAbsorber Data (Both Failure Modes) 5000 Kilometers 30000 Failure Pattern in the Shock Absorber Data Failure Mode Ignored (O’Connor 1985) Row 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 0 tj (km) 38 37 36 35 34 33 32 31 30 29 28 27 26 ... nj 1 0 0 0 1 0 0 0 0 0 0 0 1 ... dj 0 1 1 1 0 1 1 1 1 1 1 1 0 ... rj 5000 1 − pbj 25/26 ... 33/34 37/38 Conditional pbj 1/38 1/34 1/26 ... 10000 15000 Kilometers 3 - 31 3 - 33 0.09130 ... 0.05495 0.02632 Fb (tj ) Unconditional b S(t j) 0.97368 0.94505 0.90870 ... 20000 25000 Nonparametric Estimation of F (t) with Exact Failures (Kaplan-Meier) Estimator In the limit, as the number of inspections increases and the width of the inspection intervals approaches zero, we get the product-limit or Kaplan-Meier estimator: • Failures are concentrated in a small number of intervals of infinitesimal length. • Fb (t) will be constant over all intervals that have no failures. • Fb (t) is a step function with jumps at each reported failure time. 3 - 32 Note: The binomial estimator for exact failures and singly right censored data is a special case of the Kaplan-Meier estimate. 5000 10000 15000 Kilometers 20000 25000 3 - 34 Nonparametric Estimate for Shock Absorber Data with Pointwise 95% Confidence Intervals Based on Zlogit(Fb) 1.0 0.8 0.6 0.4 0.2 0.0 0 Need for Nonparametric Simultaneous Confidence Bands for F (t) 3 - 36 • Simultaneous confidence bands for F (t) are necessary to quantify the sampling uncertainty over a range of values of t. • Pointwise confidence intervals for F (t) are useful for making a statement about F (t) at one particular value of t. Proportion Failing Nonparametric Estimates for the Shock Absorber Data up to 12,220 km 6,700 6,950 7,820 8,790 9,120 9,660 9,820 11,310 11,690 11,850 11,880 12,140 12,200 ... 1.0 0.8 0.6 0.4 0.2 0.0 0 3 - 35 Nonparametric Estimate for Shock Absorber Data with Simultaneous 95% Confidence Bands Based on Zlogit(Fb) Proportion Failing Nonparametric Simultaneous Confidence Bands for F (t) Approximate 100(1 − α)% simultaneous confidence bands for F can be obtained from e F (t), F̃ (t) = Fb (t)±e(a,b,1−α/2)sceFb (t) for all t ∈ [tL(a), tU (b)] where [tL(a), tU (b)] is a complicated function of the censoring pattern in the data. Comments: • The approximate factors e(a,b,1−α/2) can be computed from a large-sample approximation given in Nair (1984). • e(a,b,1−α/2) is the same for all values of t. 3 - 37 • The factors e(a,b,1−α/2) are greater than the corresponding z(1−α/2). 0.0 0.5 1.0 1.5 2.0 2.5 3 - 39 3.0 Nonparametric Estimate Heat Exchanger Tube Crack Data with Simultaneous 95% Confidence Bands Based on Zmax logit(Fb) 0.20 0.15 0.10 0.05 0.0 Years Nonparametric Estimation of F (ti) with Arbitrary Censoring ◮ Arbitrary censoring (e.g., both left and right). ◮ Censoring with overlapping intervals. ◮ Truncated data. 3 - 41 • The nonparametric maximum likelihood generalizations provided by the Peto/Turnbull estimator can be used for • The methods described so far work only for some kinds of censoring patterns (multiple right censoring, interval censoring with intervals that do not overlap, and some other very special censoring patterns.) Proportion Failing Confidence Level .80 .90 .95 .99 2.92 3.17 3.41 3.88 2.90 3.15 3.39 3.87 2.84 3.10 3.34 3.82 2.92 3.17 3.41 3.88 2.86 3.12 3.36 3.85 2.84 3.10 3.34 3.83 2.76 3.03 3.28 3.77 2.90 3.15 3.39 3.87 2.84 3.10 3.34 3.83 2.81 3.07 3.31 3.81 2.73 3.00 3.25 3.75 2.84 3.10 3.34 3.82 2.76 3.03 3.28 3.77 2.73 3.00 3.25 3.75 2.62 2.91 3.16 3.68 2.80 3.07 3.31 3.80 2.72 3.00 3.25 3.75 2.68 2.96 3.21 3.72 2.56 2.85 3.11 3.64 Factors e(a,b,1−α/2) for Computing the EP Nonparametric Simultaneous Approximate Confidence Bands Limits a b .005 .999 .01 .999 .05 .999 .001 .995 .005 .995 .01 .995 .05 .995 .001 .99 .005 .99 .01 .99 .05 .99 .001 .95 .005 .95 .01 .95 .05 .95 .001 .9 .005 .9 .01 .9 .05 .9 3 - 38 Better Nonparametric Simultaneous Confidence Bands for F (t) • The approximate 100(1−α)% simultaneous confidence bands b b(t) for all t ∈ [tL(a), tU (b)] F (t), F̃ (t) = Fb (t) ± e(a,b,1−α/2) se F e Fb (t) − F (t) max . t ∈ [tL(a), tU (b)] sceFb(t) are based on the approximate distribution of ZmaxFb = | | | | 30 | | | 50 3 - 42 39 4 49 2 31 7 66 5 25 9 30 9 33 6 7 22 12 21 19 21 15 Count 3 - 40 • It is generally better to compute the simultaneous confidence bands based on the logit transformation of Fb . This gives # " Fb (t) Fb (t) , [F (t), F̃ (t)] = Fb (t) + [1 − Fb (t)] × w Fb (t) + [1 − Fb (t)]/w e b b/[Fb (1 − Fb )]}. where w = exp{e(a,b,1−α/2)se F | | ? ? ? Turbine Wheel Crack Initiation Data ? ? ? ? 20 | These are based on the approximate distribution of " # logit[Fb (t)] − logit[F (t)] . b se b(t)] logit[F ? ? ? 10 40 Event Plot Turbine Wheel Inspection Data max Zmax logit(Fb) = t ∈ [tL (a), tU (b)] Row 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 0 Hundreds of Hours 100-hours of Exposure ti 0 4 2 7 5 9 9 6 22 21 21 # Cracked Left Censored 39 49 31 66 25 30 33 7 12 19 15 # Not Cracked Right Censored 39 4 49 2 31 18 26 34 Proportion Cracked Crude Estimate of F (t) 0/39 = .000 4/53 = .075 2/33 = .060 7/73 = .096 5/30 = .167 9/39 = .231 9/42 = .214 6/13 = .462 22/34 = .647 21/40 = .525 21/36 = .583 21 15 42 46 π 2 π 3 π 4 π 5 π6 π 7 π 8 π 9 π10 π11 π12 10 10 20 30 Hundreds of Hours 40 50 3 - 47 10 30 Hundreds of Hours 20 40 50 Plot of Proportions Failing Versus Hours of Exposure for the Turbine Wheel Inspection Data 0.8 0.6 0.4 0.2 0.0 0 3 - 44 [π1 + π2]4 × [π3 + · · · + π12]49 × [π1 + · · · + π3]2 × [π4 + · · · + π12]31 × ... [π1 + · · · + π11]21 × [π12]15 • Uncertain censoring times. 3 - 46 3 - 48 • Maximum likelihood methods to compute nonparametric confidence intervals and confidence bands. Other Topics in Chapter 3 11 π . The values of π , . . . , π where π12 = 1 − i=1 1 11 that i b , the ML estimator of π . Then maximize L(π ) gives π Pi Fb (ti) = j=1 π̂j , i = 1, . . . , m. P L(π ) = L(π ; DATA) = C × [π1]0 × [π2 + · · · + π12]39 × • Illustration: the likelihood for the turbine wheel inspection data is • Basic idea: write the likelihood and maximize this likelib from which one gets Fb (ti) (Peto 1973). hood to obtain pb or π Nonparametric Estimation of F (t) with Arbitrary Censoring-General Approach Proportion Failing Turbine Wheel Inspection Data Summary 4 10 14 18 22 26 30 34 38 42 46 Data from Nelson (1982), page 409. • The analysts did not know the initiation time for any of the wheels. 3 - 43 • All they knew about each wheel was its exposure time and whether a crack had initiated or not. Units grouped by exposure time. 0 π1 4 Hundreds of Hours 3 - 45 Basic Parameters Used in Computing the Nonparametric ML Estimate of F (t) for the Turbine Wheel Data 0 0.8 0.6 0.4 0.2 0.0 0 Nonparametric ML Estimate for the Turbine Wheel Data with 95% Pointwise Confidence Intervals for F (ti) Based on Zlogit(Fb) Proportion Failing