Unit 17: Failure-Time Regression Analysis Ramón V. León Notes largely based on “Statistical Methods for Reliability Data” by W.Q. Meeker and L. A. Escobar, Wiley, 1998 and on their class notes. 11/14/2010 Stat 567: Unit 17 - Ramón León 1 Unit 17 Objectives • Describe applications of failure-time regression • Describe graphical methods for displaying censored regression data • Introduce time-scale transformation functions • Describe simple regression models to relate life to explanatory variables • Illustrate the use of likelihood methods for censored regression data • Explain the importance of model diagnostics • Describe and illustrate extensions to nonstandard multiple regression models 11/14/2010 Stat 567: Unit 17 - Ramón León 2 Computer Program Execution Time Versus System Load • Time to complete a computationally intensive task • Information from the Unix uptime command • Predictions needed for scheduling subsequent steps in a multi-step computational process 11/14/2010 Stat 567: Unit 17 - Ramón León 3 Scatter Plot of Computer Program Execution Time Versus System Load 11/14/2010 Stat 567: Unit 17 - Ramón León 4 Explanatory Variables for Failure Times Useful explanatory variables explain/predict why some units fail quickly and some survive a long time • Continuous variables like stress, temperature, voltage, and pressure • Discrete variables like number of hardening treatments or number of simultaneous users of a system • Categorical variables like manufacturer, design, and location 11/14/2010 Stat 567: Unit 17 - Ramón León 5 Failure-Time Regression Analysis 11/14/2010 Stat 567: Unit 17 - Ramón León 6 Nickel-Base Super-alloy Fatigue Data 26 Observations in Total, 4 Censored (Nelson 1984, 1990) • Thousands of cycles to failure as a function of pseudo-stress in ksi • 26 units tested; 4 units did not fail. Objective: Explore models that might be used to describe the relationship between life length and the amount of pseudo-stress applied to the tested specimens. 11/14/2010 Stat 567: Unit 17 - Ramón León 7 Nickel-Based Super-Alloy Fatigue Data (Nelson 1984, 1990) 11/14/2010 Stat 567: Unit 17 - Ramón León 8 Scale Accelerated Failure Time (SAFT) Models Illustration of Acceleration and Deceleration 11/14/2010 Stat 567: Unit 17 - Ramón León 9 Scale Accelerated Failure Time (SAFT) Model 11/14/2010 Stat 567: Unit 17 - Ramón León 10 The SAFT Transformation Models and Acceleration In a time transformation plot of T(x0) vs. T(x) • The SAFT models are straight lines through the origin • Accelerating SAFT models are straight lines below the diagonal • Decelerating SAFT models are straight lines above the diagonal 11/14/2010 Stat 567: Unit 17 - Ramón León 11 Some Properties of SAFT Models F t ; x P T ( x) t T ( x0 ) P t AF ( x) P T ( x0 ) t AF ( x) F AF ( x)t , x0 11/14/2010 Stat 567: Unit 17 - Ramón León 12 Some Properties of SAFT Models p F t p ( x), x F AF ( x)t p ( x), x0 and p F t p ( x0 ), x0 t p ( x0 ) AF ( x)t p ( x) 11/14/2010 Stat 567: Unit 17 - Ramón León 13 Some Properties of SAFT Models 11/14/2010 Stat 567: Unit 17 - Ramón León 14 Weibull Probability Plot of Two Members from an SAFT Model with a Baseline Lognormal Distribution p t p ( x) 11/14/2010 t p ( x0 ) Stat 567: Unit 17 - Ramón León 15 log t p ( x) ( x) since p nor 11/14/2010 Stat 567: Unit 17 - Ramón León 16 1 since log t p ( x) 0 1 x nor ( p) 1 x log t p (0) 11/14/2010 Stat 567: Unit 17 - Ramón León 17 Also log T ( x) where N(0,1) since P T t P log T log t P ( x) log t log t ( x) P log t ( x) nor 11/14/2010 Stat 567: Unit 17 - Ramón León 18 1 log tˆp x ˆ0 ˆ1 x nor p ˆ 11/14/2010 Stat 567: Unit 17 - Ramón León 19 Likelihood for Lognormal Distribution Simple Regression Model with Right Censored Data 11/14/2010 Stat 567: Unit 17 - Ramón León 20 Estimated Parameter VarianceCovariance Matrix 11/14/2010 Stat 567: Unit 17 - Ramón León 21 Standard Errors and Confidence Intervals for Parameters 11/14/2010 Stat 567: Unit 17 - Ramón León 22 11/14/2010 Stat 567: Unit 17 - Ramón León 23 11/14/2010 Stat 567: Unit 17 - Ramón León 24 11/14/2010 Stat 567: Unit 17 - Ramón León 25 11/14/2010 Stat 567: Unit 17 - Ramón León 26 SPLIDA Output 11/14/2010 Stat 567: Unit 17 - Ramón León 27 Recall cov(aX bY , Z ) a cov( X , Z ) b cov(Y , Z ) Var(aX bY ) a VarX b VarY 2ab cov( X ,Y ) 2 11/14/2010 2 Stat 567: Unit 17 - Ramón León 28 Standard Errors and Confidence Intervals for Quantities at Specific Explanatory Variable Conditions 11/14/2010 Stat 567: Unit 17 - Ramón León 29 11/14/2010 Stat 567: Unit 17 - Ramón León 30 11/14/2010 Stat 567: Unit 17 - Ramón León 31 Likelihood for Weibull Distribution Quadratic Regression Model with Right Censored Data 11/14/2010 Stat 567: Unit 17 - Ramón León 32 11/14/2010 Stat 567: Unit 17 - Ramón León 33 11/14/2010 Stat 567: Unit 17 - Ramón León 34 11/14/2010 Stat 567: Unit 17 - Ramón León 35 JMP Loss Function 11/14/2010 Stat 567: Unit 17 - Ramón León 36 Stanford Heart Transplant Data Quadratic Log-Mean Lognormal Regression Model 11/14/2010 Stat 567: Unit 17 - Ramón León 37 Stanford Heart Transplant Data Quadratic Log-Location Weibull Regression Model 11/14/2010 Stat 567: Unit 17 - Ramón León 38 Extrapolation and Empirical Models • Empirical models can be useful, providing a smooth curve to describe a population or a process • Should not be used to extrapolate outside of the range of one’s data • There are many different kinds of extrapolation – In an explanatory variable like stress – To the upper tail of a distribution – To the lowest tail of a distribution • Need to get the right curve to extrapolate: look toward physical or other process theory 11/14/2010 Stat 567: Unit 17 - Ramón León 39 Checking Model Assumptions • Graphical checks using generalizations of usual diagnostics (including residual analysis) – – – – Residuals versus fitted values Probability plot of residuals Other residual plots Influence analysis (or sensitivity) analysis. (See Escobar and Meeker 1992 Biometrics) • Most analytical test can be suitably generalized, at least approximately, for censored data (especially using likelihood ratio tests). 11/14/2010 Stat 567: Unit 17 - Ramón León 40 Definition of Residuals 11/14/2010 Stat 567: Unit 17 - Ramón León 41 Plot of Standardized Residuals Versus Fitted Values for the Log-Quadratic Weibull Regression Model Fit to the Super Alloy Data on Log-Log Axes 11/14/2010 Stat 567: Unit 17 - Ramón León 42 Probability Plot of the Standardized Residuals for the Log-Quadratic Weibull Regression Model Fit to the Super Alloy Data 11/14/2010 Stat 567: Unit 17 - Ramón León 43 Empirical Regression Models and Sensitivity Analysis Objectives • Describe a class of regression models that can be used to describe the relationship between failure time and explanatory variables • Describe and illustrate the use of empirical regression models • Illustrate the need for sensitivity analysis • Show how to conduct a sensitivity analysis 11/14/2010 Stat 567: Unit 17 - Ramón León 44 Picciotto Data (Picciotto 1970) • Cycles to failure on specimens of yarn of different lengths • Subset of data at particular level of stress and particular specimen lengths (30mm, 60mm, and 90mm) • Original data were uncensored. For the purposes of illustration, the data censored at 100 thousand cycles • Suppose that the goal is to estimate life for 10 mm units. 11/14/2010 Stat 567: Unit 17 - Ramón León 45 Piccioto Data Showing Fraction Failing at Each Length 11/14/2010 Stat 567: Unit 17 - Ramón León 46 Picciotto Data Multiple Weibull Probability Plot with Weibull ML Estimates 11/14/2010 Stat 567: Unit 17 - Ramón León 47 Picciotto Data Multiple Lognormal Probability Plot with Lognormal ML Estimates 11/14/2010 Stat 567: Unit 17 - Ramón León 48 Suggested Strategy for Fitting Empirical Regression Models and Sensitivity Analysis • Use data and previous experience to chose a base-line model: – Distribution at individual conditions – Relationship between explanatory variables and distributions at individual conditions • Fit models • Use diagnostics (e.g., residual analysis) to check models • Assess uncertainty – Confidence intervals quantify statistical uncertainty – Perturb and otherwise change the model and reanalyze (sensitivity analysis) to assess model uncertainty 11/14/2010 Stat 567: Unit 17 - Ramón León 49 Picciotto Data Multiple Lognormal Probability Plot with Lognormal ML Log-Linear Regression Model Estimates 11/14/2010 Stat 567: Unit 17 - Ramón León 50 Model: log T 11/14/2010 0 1 log Length , Stat 567: Unit 17 - Ramón León N(0,1) 51 Picciotto Data Log-Linear Model Residual vs. Fitted Values Plot 11/14/2010 Stat 567: Unit 17 - Ramón León 52 Picciotto Data Multiple Lognormal Probability Plot with Lognormal ML Square Root-Linear Regression Model Estimates 11/14/2010 Stat 567: Unit 17 - Ramón León 53 Model: log T 11/14/2010 0 1 log Length Z , Z~N(0,1) Stat 567: Unit 17 - Ramón León 54 Picciotto Data Square Root – Linear Model Residuals vs. Fitted Values Plot 11/14/2010 Stat 567: Unit 17 - Ramón León 55 Examples of Power Transformations 11/14/2010 Stat 567: Unit 17 - Ramón León 56 Box-Cox Transformation 11/14/2010 Stat 567: Unit 17 - Ramón León 57 Picciotto Data Relationship Sensitivity Analysis for the .1 Quantile at Length 30 mm. 11/14/2010 Stat 567: Unit 17 - Ramón León 58 Picciotto Data Relationship Sensitivity Analysis for the .1 Quantile at Length 10 mm. 11/14/2010 Stat 567: Unit 17 - Ramón León 59 Picciotto Data Profile Plot for Different Box-Cox Parameters for the Length Relationship 11/14/2010 Stat 567: Unit 17 - Ramón León 60 Picciotto Data Relationship Sensitivity Analysis .1, .5, .9 Quantiles at 10 mm Length 11/14/2010 Stat 567: Unit 17 - Ramón León 61 Picciotto Data Relationship/Distribution Sensitivity Analysis .1 Quantile at 10 mm Length 11/14/2010 Stat 567: Unit 17 - Ramón León 62 Glass Capacitor Failure Data • Experiment designed to determine the effect of voltage and temperature on capacitor life. • 2 x 4 factorial, 8 units at each combination. • Test at each combination run until 4 of 8 units failed (Type II Censoring). • Original Data from Zelen (1959) 11/14/2010 Stat 567: Unit 17 - Ramón León 63 Scatter Plot the Effect of Voltage and Temperature on Glass Capacitor Life (Zelen 1959) 11/14/2010 Stat 567: Unit 17 - Ramón León 64 Weibull Probability Plots of Glass Capacitor Life Test Results at Individual Temperature and Voltage Test Conditions 11/14/2010 Stat 567: Unit 17 - Ramón León 65 Likelihood Ratio Test Let L( H ) be the maximized loglikelihood under model H . To test H 0 versus H 1 where H 0 H 1 use 2 L( H 1 ) L( H o ) 2log L H 0 2log L H 1 = Loss H 0 Loss H 1 which has a Chi Square distribution under the null hypothesis H o with dim H 1 dim H 0 degrees of fredom. 11/14/2010 Stat 567: Unit 17 - Ramón León 66 Two-Variable Regression Models 11/14/2010 Stat 567: Unit 17 - Ramón León 67 -31.8 -31.7 -30.2 -30.3 -24.8 -28.4 -26.0 -28.4 -231.6 Sum of maximized loglikelihoods is -231.6 11/14/2010 Stat 567: Unit 17 - Ramón León 68 Test of Additive Model Versus Independent Models at each Experimental Condition Test statistic: 2 231.6 244.24 25.28 2 2 .95,16 4 .95,12 5.226 11/14/2010 Stat 567: Unit 17 - Ramón León 69 Weibull Probability Plots with Weibull Regression Model ML Estimates of F(t) at each Set of Conditions for the Glass Capacitor Data: Model With Interaction 11/14/2010 Stat 567: Unit 17 - Ramón León 70 Estimates of Weibull t.5 Plotted for each Combination of the Glass Capacitor Test Conditions 11/14/2010 Stat 567: Unit 17 - Ramón León 71 Superalloy JMP Loss Function Let’s think of some interesting tests? 11/14/2010 Stat 567: Unit 17 - Ramón León 72 Review of the Weibull Plot For the Weibull the p quantile is t p log(1 p ) 1 or log t p log log log(1 p ) 1 1 log t p log log log(1 p ) So a plot of log log 1 F (t ) log log S (t against t will plot as straight line if the distribution is Weibull Proportional Hazard Failure Time Model h(t; x ) ( x )h(t; x0 ) log 1 F (t; x ) H (t; x ) t t 0 0 h( s; x )ds = ( x ) h( s; x0 )ds ( x ) H (t , x0 ) ( x ) log 1 F (t , x0 ) log 1 F (t; x ) log 1 F (t , x0 ) (x) 1 F (t; x ) 1 F (t , x0 ) ( x) 11/14/2010 S (t; x ) S (t , x0 ) ( x ) Stat 567: Unit 17 - Ramón León 74 S (t , x ) S (t , x0 ) ( x ) log S (t , x ) ( x )log S (t , x0 ) log log S (t , x ) log ( x )log S (t , x0 ) log ( x ) log log S (t , x0 ) log log S (t , x ) log log S (t , x0 ) log ( x ) Thus in a Weibull probability scale F ( t ; x ) and F (t ; x0 ) are equidistant. In particular, Weibull plots of F (t ; x ) and F (t ; x0 ) are translations of each other along the probability axis. Weibull Probability Plot of Two Members from a PH Model with Baseline Lognormal Distribution 11/14/2010 Stat 567: Unit 17 - Ramón León 76 Weibull Probability Plot of Two Members from a PH/SAFT Model with Baseline Weibull Distribution 11/14/2010 Stat 567: Unit 17 - Ramón León 77 Contrasting SAFT and PH Models 11/14/2010 Stat 567: Unit 17 - Ramón León 78 Interpreting PH Models as Failure Time Transformation 11/14/2010 Stat 567: Unit 17 - Ramón León 79 Proportional Hazard Model (Lognormal Baseline) as a Time Transformation Amount of acceleration (deceleration) depends on the position in time. 11/14/2010 Stat 567: Unit 17 - Ramón León 80 General Failure Time Transformation Model 11/14/2010 Stat 567: Unit 17 - Ramón León 81 General Failure Time Transformation Graph 11/14/2010 Stat 567: Unit 17 - Ramón León 82 General Failure Time Transformation Model 11/14/2010 Stat 567: Unit 17 - Ramón León 83 Some Comments on the Proportional Hazard Failure Time Model 11/14/2010 Stat 567: Unit 17 - Ramón León 84 Cox’s Semiparametric Proportional Hazard rates Model 11/14/2010 Stat 567: Unit 17 - Ramón León 85 Simple Example h(t , x) h0 (t )e x (x is a scalar in this example) Consider the following data: 11/14/2010 Stat 567: Unit 17 - Ramón León 86 Motivation Using the partial likelihood to estimate is equivalent to estimating by the method of maximum likelihood knowing only -that the units under stress x = 100 were the 1st and 4th to fail -that the units under stress x = 200 were the 2nd and 3rd to fail -between what failures the censoring occurred Cox’s partial likelihood estimation method corresponds to the method of maximum likelihood when only the ranks of the failures times (and the location between the failure ranks of the censoring) is considered. 11/14/2010 Stat 567: Unit 17 - Ramón León 87 Partial Likelihood The partial likelihood can be used just as the likelihood to: - To estimate maximize the partial likelihood - To estimate the variance-covariance matrix of the estimates invert the matrix of negatives of second partial derivatives of the log partial likelihood -To obtain confidence intervals use the (partial) likelihood ratio method or the standard error method - Compare nested models using partial likelihood ratios 11/14/2010 Stat 567: Unit 17 - Ramón León 88 Risk Sets 11/14/2010 Stat 567: Unit 17 - Ramón León 89 Partial Likelihood Formula r r i exp x (i ) L( ) i 1 exp xl i 1 ci lR i where x(i ) is the vector of regressor variables associated with units observed to fail at time t(i ) , i 1, 2,..., r 11/14/2010 Stat 567: Unit 17 - Ramón León 90 Partial Likelihood Calculation 11/14/2010 Stat 567: Unit 17 - Ramón León 91 Partial Likelihood Calculation 11/14/2010 Stat 567: Unit 17 - Ramón León 92 A Game Three players throw three balls in a paper basket. We have the following probabilities: Player P(Player makes basket) 1 p1 2 p2 3 p3 Suppose exactly one ball landed in the basket, what is the probability that player i was the one that made the basket? pi p1 p2 p3 11/14/2010 Stat 567: Unit 17 - Ramón León 93 One Multiplicand of the Partial Likelihood h(t(3) , 200)t h(t(3) , 200)t h(t(3) ,100)t h(t(3) , 200)t 11/14/2010 Stat 567: Unit 17 - Ramón León 94 One Multiplicand of the Partial Likelihood h(t(3) , 200)t h(t(3) , 200)t h(t(3) ,100)t h(t(3) , 200)t h0 (t(3) )e 200 t h0 (t(3) )e 200 200 e 200 100 t h0 (t(3) )e t h0 (t(3) )e 200 t x( 3) e e xl 100 200 e e e lR3 11/14/2010 Stat 567: Unit 17 - Ramón León 95 Remark Cox’s partial likelihood estimation method ignores the actual times of the failures. This is reasonable since without knowing h0(t) the actual failure times provide little or no information on the 's 11/14/2010 Stat 567: Unit 17 - Ramón León 96 Comments on the Cox PH Model Likelihood 11/14/2010 Stat 567: Unit 17 - Ramón León 97 Picciotto Cox PH Model Survival Estimates (Linear Axes) 11/14/2010 Stat 567: Unit 17 - Ramón León 98 Picciotto Cox PH Model Survival Estimates (on Weibull Probability Scales) 11/14/2010 Stat 567: Unit 17 - Ramón León 99 Cox Proportional Hazards Model in JMP We analyze the failure stresses of a single carbon fiber as a function of length x. (Data available at the course home page.) 2 .01 .05 .1 .2 1 log(-log(Surv)) 0 .4 .6 -1 .8 -2 .9 -3 .95 -4 .99 -5 -6 This curves should be vertical displacements of each other not necessarily straight lines -7 1 2 3 4 5 6 7 Stress 11/14/2010 Stat 567: Unit 17 - Ramón León 100 JMP Analysis 11/14/2010 Stat 567: Unit 17 - Ramón León 101 Effect Likelihood Ratio Tests Source Nparm Length 1 11/14/2010 DF L-R ChiSquare Prob>ChiSq 1 140.962733 Stat 567: Unit 17 - Ramón León 0.0000 102 Parameter Estimates Term Estimate Lower CL Upper CL Length 0.04923859 0.0040304 0.0413392 0.05717 h(t , x) h0 (t )e Std Error ( x xu ) h0 (t )e 0.4923675( x 20.25) where xu 20.25 is the unweighted mean of the values of the regressors (lengths): (1+10+20+50)/4 =20.25 11/14/2010 Stat 567: Unit 17 - Ramón León 103 11/14/2010 Stat 567: Unit 17 - Ramón León 104 JMP Analysis S0 (t ) = Survival at xu 20.25 S (t , x) S0 (t ) ( x) exp ( x xu ) S0 (t ) exp 0.4923675( x 20.25) S0 (t ) Can be used to calculate Shoenfeld residuals. More about this latter. 11/14/2010 Stat 567: Unit 17 - Ramón León 105 Some Mathematics 4 h(t ,1) h(t ,10) h(t , 20) h(t ,50) 4 4 h(t , x ) i i 1 4 4 xu ( xi xu ) h ( t ) e h ( t ) e 0 0 i 4 4 h0 (t )e xu e 4 4 xi e i 1 4 xi i 1 h0 (t )e xu 4 e 4 xu h0 (t )e xu e xu h0 (t ) h0(t) is the geometric mean of the hazard rates at the four loads (lengths) 11/14/2010 Stat 567: Unit 17 - Ramón León 106 Risk Ratios Term Risk Ratio Lower CL Length 1.050471 Upper CL 1.042206 1.058836 For simplicity assume that x1 x2 ( x2 xu ) h(t , x2 ) h0 (t )e ( x1 xu ) h(t , x1 ) h0 (t )e x2 e x2 x1 x2 x1 0.049 x2 x1 e e 1.05 x1 e 11/14/2010 Stat 567: Unit 17 - Ramón León 107 Categorical JMP Analysis Whole Model Number of Events Number of Censorings Total Number Model Difference Full Reduced 250 0 250 -LogLikelihood ChiSquare 105.449 1028.625 1134.075 210.8989 DF Prob>Chisq 3 <.0001 Effect Likelihood Ratio Tests 11/14/2010 Source Nparm Length 3 DF L-R ChiSquare Prob>ChiSq 3 210.898899 Stat 567: Unit 17 - Ramón León 0.0000 108 Parameter Estimates Term Estimate Std Error Lower CL Upper CL -1.9230939 0.163213 -2.250564 -1.610452 Length[1] -0.1355 Length[10] -0.3619084 0.1178567 -0.598202 Length[20] 0.89469015 0.1231856 0.6508217 1.1345373 h(t , x) h0 (t ) exp 1.92 x1 0.36 x2 0.89 x3 1 x1 -1 0 1 length=50 x2 -1 0 o.w. length=1 1 length=50 x3 -1 0 o.w. length=10 length=20 length=50 o.w. h(t ,1) h0 (t )e 1.92 , h(t ,10) h0 (t )e 0.36 h(t , 20) h0 (t )e0.89 , h(t ,50) h0 (t )e1.39 11/14/2010 Stat 567: Unit 17 - Ramón León 109 Categorical JMP Analysis What mean if regressor is categorical? Baseline Survival at mean 1.0 0.9 Baseline Surv 0.8 0.7 h0 (t ) geometric 0.6 0.5 mean of the hazard 0.4 0.3 rates at the four 0.2 0.1 loads 0.0 1 2 3 4 5 6 Stress 11/14/2010 Stat 567: Unit 17 - Ramón León 110 Risk Ratios Term Risk Ratio Lower CL Length[1] Length[10] Length[20] 0.146154 0.696346 2.446578 h(t ,1) h0 (t )e 1.92 h(t ,10) h0 (t )e 0.146h0 (t ) 0.696h0 (t ) 0.89 2.447 h0 (t ) 1.39 4.015h0 (t ) h(t ,50) h0 (t )e 11/14/2010 0.10534 0.199797 0.5498 0.873279 1.917115 3.109734 0.36 h(t , 20) h0 (t )e Upper CL Stat 567: Unit 17 - Ramón León 111 Shoenfeld (Imputation) Residuals xiT ˆ H 0 (ti )e if t i is a failure time xiT ˆ H 0 (ti )e 1 if t i is a censoring time Remark: This are uncensored residuals unlike the usual residuals Based: -For any positive random variable T, H(T) is an exponential random variable with mean = 1 -For the an exponential random variable T we have E (T | T c) c Useful for plotting against predicted values. In practice, they don’t work too well though they are based on a very clever idea. 11/14/2010 Stat 567: Unit 17 - Ramón León 112 The Proportional Hazard Failure Time Model 11/14/2010 Stat 567: Unit 17 - Ramón León 113 Statistical Methods for Semiparametric (Cox) PH Model 11/14/2010 Stat 567: Unit 17 - Ramón León 114 Cox PH Model Likelihood (Probability of the Data) 11/14/2010 Stat 567: Unit 17 - Ramón León 115 Other Topics in Failure-Time Regression • Models with non-location-scale distributions • Nonlinear relationship regression model • Fatigue-limit regression model • Special topics in regression: random effects models, nonparametric SAFT regression models, parametric PH regression models 11/14/2010 Stat 567: Unit 17 - Ramón León 116