CALCULATING EXPECTED VALUE OF SAMPLE INFORMATION FOR SURVIVAL TRIALS:
BAYESIAN UPDATING FOR THE WEIBULL DISTRIBUTION.
Alan Brennan, , Samer Kharroubi. University of Sheffield, England. a.brennan@sheffield.ac.uk
Purpose: a generalised process for calculating partial EVSI for survival trials.
Background: Brennan et al. 1,2 and Claxton et al 3 have promoted the
EVSI concept as a measure of the societal value of research designs to help
identify optimal sample sizes for primary data collection.
CHEBS
.
Part C: Simulating a data collection exercise
Results
• Decide on Nnew, the number of new patients to study
• Decide on Dnew, the duration of followup in the new study
£1,200
• Sample from prior values for λ (λsample) and γ (γsample i.e. βsample ) i.e. partB
The Weibull Distribution for Survival Data
Survival analysis in cost-effectiveness decision models often utilises Weibull survival curves.
Typical data examines “survival” i.e. ‘time to death’ or ‘time to clinical event’
• N patients,
• X suffer an event,
• over a follow-up duration D, after which data is censored.
•Typical data for the ith patient is a pair of numbers (di, ti) where
di = 1 if died or 0 if alive or withdrew and ti = time of death / last measured date of survival.
The classic text4 determines maximum likelihood estimates (MLE) for shape (γ) and
scale (λ) parameters of the Weibull curve.
£1,000
Loop
Part D: Updating the Weibull parameters given simulated data
• Combine the prior data with the simulated data
^
• Use Newton Raphson algorithm to quantify revised estimates for
 and
SPLUS … nlimb() to minimise nonlinear functions with constraints
EVSI
^

£400
£200

• Net benefit of ‘revised decision’ | simulated data[i]: = max E NBd ,  | X

i
d
• Re-run the Loop, say 1,000 times including evaluating expression (1)

Cost-effectiveness decision models often have Weibull survival parameters alongside
cost and quality of life parameters. EVSI examines the value of gaining more accurate
estimates of these parameters for informing the decision between treatments.
The questions: “How valuable would greater sample size (Nnew) be ?”
“How valuable would longer follow-up (Dnew) be ?”
“How valuable would both greater sample size + longer follow-up be ?”
0

EVSI  E Xi max E NBd ,  | X i   max E NB(d, )
d

(1)
(2)
d
Two Nested Expectations
Option 2:
1st
Order Laplace Approximation
% survival
^
^
 is vector posterior mode of model parameters  and 
^




1st order approximation EVSI = E X  max NB d ,    max E NB(d,  )


 
d
 
 d  
^
Illustrative data
200 patients
^

= 0.904329012
VarCovar.=
Illustrative Survival Cost-Effectiveness Model
Part B: Estimating the effect of current uncertainty in Weibull parameters
To sample: Log (λ), log (γ)
Normal Parameters
600
800
0.000011972 - 0.000318771
- 0.000318771
0.009069378
Days
Weibull Fitted Survival Curve
Prior data - Kaplan Meier
0.0
-0.2
thetahat = c(0.007836397,0.904329012)
a=matrix(thetahat,ncol=2)
NormalVarCovar = log (VarCovar /c(a%*%t(a))+1)
-0.4
NormalMean=log(thetahat * exp(-0.5* diag(VarCovar)))
Log (γ)
SPLUS program :
res6months1000$samplelogth....
0.2
~
MultiVNormal (NormalMean, NormalVarCovar)
Log (λ)
-4.860
-4.855
(3)
-4.850
-4.845
res6months1000$samplelogth....
samplelogtheta[i,] = rmvnorm(1, mean=NormalMean, cov=NormalVarCovar)
-4.840
1 year
£
419
£
621
£
831
£ 1,041
£ 1,069
2 years
£
554
£
758
£
942
£ 1,080
£ 1,110
3 years
£
601
£
806
£
945
£ 1,086
£ 1,116
6 months
1.00
1.64
2.35
3.38
3.94
1 year
1.70
2.53
3.38
4.23
4.35
2 years
2.25
3.08
3.83
4.39
4.51
3 years
2.44
3.28
3.84
4.42
4.54
• Similarly doubling the duration from 6 months to 1 year increases the value of a small n=50 study
by 70%, but the value of a larger n=1,000 study by only 10%.
(4)
Posterior mode is recalculated for each simulated dataset collected Xi
T1
Weibull Parameters
lamda
0.008000
gamma
0.910000
beta
202
Implied mean
survival (days)
210.8
400
6 months
£
246
£
404
£
577
£
831
£
970
Indexed to n= 50, duration = 6 months
i
Central Estimate Model Parameters
200
Study Duration
• Doubling the sample size from n=50 to 100, increases the value of a 6 month study by 64% but
the value of a 3 years study by only 34%.
Conclusions
= 0.007836397
^

EVSI
Results
Sample
Size
50
100
200
500
1000
Only One Expectation
100 days follow-up
0
  ^ 
max  NB d , 
d

 
• Net benefit of ‘revised decision’ | simulated data[i]: =
Part A: Prior data to estimate prior Weibull parameters
Follow-up
5
• Evaluate net benefit for each treatment given new data, make ‘revised decision’
Methodology
50
100 200
500 1000
Sample Size (n)
• Calculated the overall expected value of the proposed data collection exercise

3 years
2 years
1 year
6 months
£0
• Evaluate net benefit for each treatment given new data, make ‘revised decision’
(Re-run the decision model using probabilistic Monte Carlo simulation )

= the parameters for the model (uncertain currently).
d
= set of possible decisions or strategies.
NB(d, ) = the net benefit for decision d, and parameters 
Some software packages e.g. EXCEL function Weibull(α, β) and SPLUS rweibull(λ, β)
use an alternative formulation for the scale parameter defining β as β = (1/ λ)^(1/ γ).
^
^
Mean survival =    1 1  , where  is the mathematical gamma function.
Weibull Survival Curve Prior
£600
Option 1: 2 level algorithm
These are obtained by solving two equations to define the and  , which
maximise the log of the likelihood function, usually using Newton-Raphson approach.
To quantify uncertainty in these parameters we also need J= -l’’(λ, γ)
i.e. the 2nd derivative of the likelihood function. J-1 is the variance covariance matrix.
100%
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%
£800
Part E: Quantifying the Value of the Simulated Additional data
^

• Simulate Nnew patients survival times from the Weibull distribution
SPLUS code is ….. newdata = rweibull(Nnew, λsample, βsample)
-4.835
T2
Difference
0.007836
0.904329
213
223.7
Characterisation of
Uncertainty
Standard Deviations
T1
T2
0.0000 See VarCovar
0.0000
(Part A)
12.9
Utility
0.7
0.75
0.05
0.05
0.05
Cost per day
£50
£50
£0
£10
£10
Basic cost of trt
£50
£1,000
£950
£10
£10
Model Results
LifeYears
0.5774
QALYs
0.4042
Total Cost
£10,588
Incremental Cost per QALY
Cost-Effectiveness Threshold
Net benefit
£
1,538 £
0.6128
0.4596
£12,184
1,605
0.0354
0.0554
£1,596
£28,799
£30,000
£ 66.54
Analyses
Samples sizes from 0 to 1,000
1) A simple illustrative model for two treatments, costs and benefits is used to show how Bayesian
updating for the Weibull distribution can quantify the value of different survival study options.
2) The results show diminishing returns in the value of information as sample size increases but at a
different rate to the diminishing returns as the follow-up duration is varied.
3) This methodology provides a new and valuable approach for EVSI calculation which might be
applied generally in survival studies.
1 Brennan, A., Chilcott, J. B, Kharroubi, S, O'Hagan, A. Calculating Expected Value of Perfect Information:Resolution of Conflicting Methods via a Two Level Monte Carlo Approach, presented at the 24th Annual
Meeting of SMDM, October 23rd, 2002, Washington. 2002. Submitted - Journal of Medical Decision Making
2 Brennan, A. B., Chilcott, J. B., Kharroubi, S., O'Hagan, A. A Two Level Monte Carlo Approach to Calculation
Expected Value of Sample Information: How To Value a Research Design. Presented at the 24th Annual
Meeting of SMDM, October 23rd, 2002, Washington. 2002.
3 Claxton, K, Ades, T. Efficient Research Design: An Application of Value of Information Analysis to an
Economic Model of Zanamivir. Presented at the 24th Annual Meeting of the Society for Medical Decision
Making, October 21st, 2002, Washington. 2002.
4 Collett : Modelling Survival Data in Medical Research, Chapman and Hall, 1994
5 Sweeting, J, Kharroubi, S. Some New Formulae for Posterior Expectations and Bartlett Corrections.
Sociedad de Estadistica e Investigacion Operative Test, (Accepted) 2003
Follow up from 6 months to 3 years
Acknowledgements: thankyou to Professor Tony O’Hagan for encouraging our ongoing work