Time to event analytic methods for
health economic evaluation
Bohdan Nosyk, PhD
Associate Professor
St. Paul’s Hospital Canfar Chair in HIV/AIDS Research
Faculty of Health Sciences, Simon Fraser University
Research Scientist, Health Economics
Michael Smith Foundation for Health Research Scholar
BC Centre for Excellence in HIV/AIDS
Disclosure of conflicts of interest
• None to declare
Outline
• Context & key challenge
• State of the science for TTE analysis in CEA
• Three techniques to address methodological challenges
of TTE data
• Next steps
Context
•
Model-based health economic evaluation
– Sculpher et al., Health Economics. 2006; 15: 677-87.
•
Applications in chronic disease
– recurrent course
– Multiple comobidities/outcomes
•
Problem: How do we estimate accurate transition probabilities in the presence of:
– Competing risks
– Recurrent events
– Time-varying exposure, measured/unmeasured confounding
•
Cohort-based models: Allowance for time-dependence in transition probabilities
– Markov model:
• According to time in model
• Within initial health state
– Semi-Markov model:
• According to time in model
• Within initial health state
• Within subsequent health states
State of the science for TTE analysis in CEA
•
CEA ‘Best Practices’ guidelines:
– Generally not specific in describing methods to deal with TTE data
•
Latimer et al (2014) review for TTE analysis:
– Specific to extrapolation of (single episode, single outcome) RCT data
• “Analyst should demonstrate that all standard parametric models
(exponential, Weibull, Gompertz, log-logistic) have been considered and
compared”.
•
Briggs, Sculpher and Claxton (2006) text:
– Parametric (Weibull) regression model
• Allows for time-dependence in transition probability
• Regression-based approach can handle heterogeneity
• Underlying theoretical distribution can be sampled from in PSA
Data sources for CEA
• Clinical trial data:
– Limited duration, chronicity, recurrence likely to be inadequately
captured
• Published literature:
– Sparse, incompatible outputs for CEA, limited external validity
• Observational data:
– Prospective cohort studies
– Disease registries
– Retrospective studies, based on health administrative databases
(ie. treatment utilization)
• Comparison of effectiveness of competing treatment regimens may be
biased (endogeneity/selection bias)
TTE methods for different forms of data
• Standard: single, non-recurrent event
– Semi-parametric: Cox Proportional Hazards (CPH) model
– Parametric: Accelerated Failure Time models (exponential, Weibull,
Gompertz)
• Multiple outcomes
– Semi-parametric: CPH Competing risks models (Cause-specific
hazards, subdistribution hazards)
– Semi-parametric: Multi-state Markov Models
• Non-terminal, recurrent events
– Semi-parametric: CPH frailty models
• Time varying exposure, confounding
– Semi-parametric: CPH model with time-varying covariates
– Marginal structural models
An important distinction among competing risks models
cause-specific hazard:
outcome: {duration, censor}
subdistribution hazard:
outcome: {duration, multinomial}
csHR: (strong) assumption of independent competing risks required for valid inference
Lau et al, Am J Epidemiol., 2009; 170: 244-56
1. Capturing competing risks
Multi-state Markov modeling:
CD4>500
• Regression-based approach;
• Can handle heterogeneity
Off ART:
CD4: 350-499
CD4: 350-499
CD4: 200-349
Death
•
Designed for multiple
outcomes, continuous-time
data
• Outputs represent
subdistribution hazards
• Semi-parametric method: no direct means of handling time-dependence;
not useful for PSA
• High dimensionality – problems with model convergence
•
•
•
Nosyk et al, J Acquir Immune Defic Syndr. 2013;63(5):653-659.
Craig and Sendi, Health Econ. 2002; 11: 33-42.
R code: msm package: http://cran.r-project.org/web/packages/msm/index.html
2. Adjusting the hazard of TTE of successive
‘recurrent’ episodes
%
CPH frailty model form:
hij(t) = h0(t)vj exp(β’Zij)
where
vj ~ γ(1/θ, 1/θ)
3rd episode:
2nd episode:
Time in health state
t
1st episode:
𝑘
𝑡 𝑘
𝑏
𝑘−1
exp −𝒉𝟑
𝑘𝑡
𝑏
𝑘
𝑡 𝑘
𝑏
𝑘−1
exp −𝒉𝟐
𝑘𝑡
𝑘
𝑏𝑘
𝑡 𝑘−1 exp −
𝑏
𝑘
𝑡
𝑏
• Mixed effects model; can adjust for unmeasured confounding that is fixed over time
• Semi-parametric method; no accounting for time-dependence*
• Cannot account for multiple outcomes
Nosyk et al., Am J Epidemiol., 2009; 170 (6): 783-92.
Nosyk et al., CMAJ, 2012; 84 (6): E317-28.
R code: surv package: https://stat.ethz.ch/R-manual/R-patched/library/survival/html/Surv.html
3. Handling time-dependent exposure,
confounding
•
Marginal Structural Models
•
Context: outcome can also be a predictor
of exposure; other time-varying
confounding
•
Effect of OST in standard GEE with timevarying covariates:
–
•
OR: 1.91 (1.68, 2.19)
Effect of OST in MSM model:
–
OR: 1.68 (1.48, 1.92)
• Assumes no unmeasured confounding
• Cannot account for multiple outcomes
Robins JM, Hernan MA, Brumback B. Epidemiology 2000;11:550 –560.
Nosyk et al., AIDS. 2015. IN PRESS
Next Steps
• Continued methodological development in TTE
analytic methods needed:
– Parametric competing risks model
• Jeong and Fine, Biostatistics 2007; 8(2): 184-96.
– Competing risks CPH Frailty model
• Kauermann & Khomski, 2006. R package
‘CompetingRiskFrailty’ no longer functional
– Competing risks Marginal structural model
• None available to date
– Improved optimization algorithms for high-dimensional
multi-state Markov models
• How do differences, limitations in existing methods
affect CEA results?
Acknowledgements
•
BC-CfE Health Economics Team: Emanuel Krebs, Jeong Min, Michelle Olding, Batool
Yazdani
•
UCLA Integrated Substance Abuse Programs; Centre for Advancing Longitudinal Drug
Abuse (CALDAR) : Elizabeth Evans, Libo Li, Yih-Ing Hser
–
[NIDA Grant No. P30 DA016383]
•
Dr. Lei Liu, Northwestern University
•
An empirical investigation into recovery from illicit drug abuse using recurrent event
analytic methods
–
•
Addiction and Urban Health Research Unit (UHRI), BC-CfE
–
•
[NIDA Grant No. R01-DA033424]
[NIDA Grant No. R01-DA011591, R01-DA021525, R01-DA028532]
Seek and Treat for Optimal Prevention of HIV/AIDS (STOP-HIV/AIDS) Program Evaluation
–
[BC Ministry of Health, Healthy Living and Sport]
Questions, Comments?
Thank you for your attention.
Bohdan Nosyk: bnosyk@cfenet.ubc.ca
State of the science for TTE analysis in CEA?
Source
Recommendation
Guidelines for the economic evaluation of health
technologies: Canada [3rd Edition]. CADTH (2006).
Re.: extrapolation of short-term RCT data.
The duration and the magnitude of the clinical benefit
beyond the trial is often a critical judgment to make
regarding extrapolation.
Describe the strength of the evidence for
extrapolating data and assess uncertainty through a
sensitivity analysis.
Describe the method of elicitation to establish
parameter values, and the results of the exercise. Any
limitations of the methods, potential biases in the
parameter estimates, and caveats about the
interpretation of results, should be reported.
The Strengthening the Reporting of Observational
Studies in Epidemiology (STROBE) Statement:
Guidelines for Reporting Observational Studies.
Annals of Internal Medicine (2007).
For each variable of interest, give sources of data and
details of methods of assessment (measurement).
Describe all statistical methods.
State of the science for TTE analysis in CEA?
Source
Recommendation
Modeling Good Research Practices—Overview: A
Report of the ISPOR-SMDM Modeling Good Research
Practices Task Force-1. Value Health (2012).
Transition probabilities and intervention effects
should be derived from the most representative data
sources for the decision problem.
Conceptualizing a Model: A Report of the ISPORSMDM Modeling Good Research Practices Task Force2. Value Health (2012).
The problem conceptualization should be used to
identify key uncertainties in model structure where
sensitivity analyses could inform their impact.
The conceptual structure should be driven by the
decision problem or research question and not
determined by data availability.
State-transition modeling: a report of the ISPORSMDM modeling good research practices task force-3.
Value Health (2012).
All methods and assumptions used to derive
transition probabilities should be described.
STMs should provide clear justification for estimates
of transition probabilities and state values and their
ranges for sensitivity analysis.
As STMs allow deriving the time at which particular
transitions occur, the results can be represented as
(modeled) probability or survival curves and directly
compared with survival curves from empirical studies.
State of the science for TTE analysis in CEA?
Source
Recommendation
Consolidated Health Economic Evaluation Reporting
Standards (CHEERS)—Explanation and Elaboration: A
Report of the ISPOR Health Economic Evaluation
Publication Guidelines Good Reporting Practices Task
Force. Value Health (2013).
For model-based economic evaluations, authors
should describe and report how they estimated
parameters, for example, how they transformed
transition probabilities between events or health
states into functions of age or disease severity.
Regardless of study design, the handling of
uncertainty and the separation of heterogeneity from
uncertainty should be consistent themes.
Survival Analysis For Economic Evaluations
Alongside Clinical Trials - Extrapolation With
Patient-level Data. NICE DSU Technical Support
Document 14 (2011 - Updated 2013).
The analyst should demonstrate that a range of
parametric models [for survival analysis] have been
considered and compared.
The fit of alternative models should be assessed
systematically.
PH modelling should only be used if the proportional
hazards assumption can be clearly justified.
Scenario based sensitivity analysis should assess the
importance of duration of treatment effect
assumptions.
State of the science for TTE analysis in CEA?
Source
Recommendation
Cost-Effectiveness Analysis Alongside Clinical Trials
II—An ISPOR Good Research Practices Task Force
Report. Value Health (2015).
When modeling beyond the follow-up period for the
trial, it is important to project costs and outcomes
over the expected duration of treatment.
Parametric survival models estimated on trial data are
generally recommended for such projections, unless
models based on other data or methods are justified.
Applying Dynamic Simulation Modeling Methods in
Health Care Delivery Research—The SIMULATE
Checklist: Report of the ISPOR Simulation Modeling
Emerging Good Practices Task Force. Value Health
(2015).
The checklist identifies time-dependent and dynamic
transitions that characterize simulation modeling
methods and differentiate them from other modeling
approaches such as Markov models and decision
trees.
State of the science for TTE analysis in CEA?
Source
Recommendation
Guidelines for the economic evaluation of health technologies:
Canada [3rd Edition]. CADTH (2006).
Describe the method of elicitation to establish parameter
values, limitations of the methods, and potential biases.
The Strengthening the Reporting of Observational Studies in
Epidemiology (STROBE). Annals of Internal Medicine (2007).
Describe all sources and statistical methods.
ISPOR-SMDM Modeling Good Research Practices. Value Health
(2012).
All methods and assumptions used to derive transition
probabilities should be described.
The conceptual structure should be driven by the decision
problem and not determined by data availability.
Consolidated Health Economic Evaluation Reporting Standards
(CHEERS)—ISPOR. Value Health (2013).
Describe how transition probabilities are estimated.
The handling of uncertainty and the separation of
heterogeneity from uncertainty should be consistent themes.
Survival Analysis For Economic Evaluations. NICE DSU Technical
Support Document 14 (2011 - Updated 2013).
A range of parametric models [for survival analysis] should be
considered and fit should be assessed systematically.
Cost-Effectiveness Analysis Alongside Clinical Trials II—ISPOR.
Value Health (2015).
When modeling beyond the follow-up period, project costs and
outcomes over the expected duration of treatment.
Parametric survival models estimated on trial data are
generally recommended for such projections.
Applying Dynamic Simulation Modeling Methods in Health Care
Delivery Research—The SIMULATE Checklist. ISPOR. Value
Health (2015).
The checklist identifies time-dependent and dynamic
transitions that characterize simulation modeling methods and
differentiate them from other modeling approaches.
Nosyk et al, CMAJ. 2012; 184(6):E317-28.
The primary finding of this study was that patients experiencing
multiple treatment episodes tended to stay in treatment for progressively
longer periods in later episodes.
Nosyk et al, Am J Epidemiol. 2009; 170(6):783-92.
Decision Analytic Model
Allocated to MMT
Cycle=3
Death
Death
DAM1
MMT3
Relapse3
Cycle, j=4-6
Allocated to DAM
MMT-PD3
Abstinence
MMTj
Relapse3
Abstinence
DAMj-2
Model parameterization
• Supplemented trial TTE data with external data: BC linked health database
• Estimated weibull regressions to extrapolate TTE data
• Implemented by making use of R’s functionality with multi-dimensional
arrays described by Hawkins, Sculpher and Epstein (2005) in order to
program time dependence within each of the model states
• Multiplied adjusted hazards for episode j with adjusted hazard ratios,
drawn from frailty models, for durations of episodes j+k
• Adjusted for age, gender, state-specific mortality risks
• Assumed dirichlet distribution for transitions to multiple states
Impact of changing distributional assumption
on TTE curves
Cost, $1000CDN
Mean (95% CI)
QALYs
Mean (95% CI)
ICER
Mean (95% CI)
CS (CS, 122)
Baseline Formulation*
DAM
1096 (724, 1707)
7.9 (7.3, 8.5)
MMT
1137 (737, 1777)
7.5 (6.9, 8.0)
Exponential distributions set for TTE curves
DAM
1104 (729, 1712)
7.8 (7.1, 8.4)
MMT
1145 (738, 1812)
7.4 (6.8, 8.1)
CS (CS, 331)
*Gamma distributions set for time to discontinuation of each health state.
CPH Frailty models demonstrated durations of daily use diminished in
Successive episodes over time. MSM models revealed primary stimulant
users had more erratic longitudinal patterns of drug use, transitioning more
Rapidly between periods of treatment, abstinence, non-daily and daily use.
Nosyk et al., Drug Alcohol Depend. 2014; 140: 69-77.
1. Capturing competing risks (option b)
Parametric regression methods for competing risks:
Cumulative incidence functions for 2 competing
risks:
Probability of having Syncitium-inducing (SI)
HIV phenotype
AIDS+SI
Probability of developing AIDS
• Allows for time-dependent transition probabilities, heterogeneity and competing risks;
• Only basic control of baseline confounders
Putter H, Fiocco M, Geskus B. Stat Med. 2007; 26:2389-2430.
Jeong J-H, Fine JP. Biostatistics. 2007; 8(2): 184-96.