A Two Level Monte Carlo Approach
To Calculating
Expected Value of Sample Information:How to Value a Research Design.
Alan Brennan
and
Jim Chilcott, Samer Kharroubi, Tony O’Hagan
University of Sheffield
IHEA June 2003
What are EVPI and EVSI?
Adoption Decision
Choose policy with greatest expected ‘payoff’
Uncertainty
Confidence interval e.g.
|--------[]---------|
implies Adoption Decision could be wrong
Perfect Information
perfect knowledge about parameter e.g.
Value of
Information
Expected Value of
Information
[]
quantify additional payoff obtained by switching
adoption decision after obtaining additional data
quantify average additional payoff obtained for
range of possible collected data
EVPI
Expected value of obtaining perfect knowledge
EVSI
Expected value of obtaining a specific sample size
Typical Process
Net benefit of alternative policies
Illustrative Model
a
Mean Model
Cost of drug
% admissions
Days in Hospital
Cost per Day
£
£
% Side effects
Change in utility if side effect
Duration of side effect (years)
£
Total QALY
Probabilistic
Sensitivity Analysis
d
e
Uncertainty in Parameter Means
Standard Deviations
Increment
Mean Model
T0
T1 (T1 over T0)
Cost of£drug 1,500 £
1,000
500
%
admissions
10%
8%
-2%
Days in Hospital6.10
5.20
0.90
Cost
400 per
£ Day
400 £
-
%
Responding 80%
70%
Utility change0.3000
if respond
0.3000
Duration
of response
3.0
3.0 (years)
% Responding
Utility change if respond
Duration of response (years)
Total Cost
Illustrative
Model
b
c
Number of Patients in the UK
1,000
Threshold cost per QALY
£10,000
Number of Patients in the UK
Threshold cost per QALY
T0
1
2%
1.00
200
Sampled Values
T1
1 £
2%
1.00
200 £
T0
1,000 £
10%
5.01
589 £
Increment
T1 (T1 over T0)
1,499 £
499
11%
0%
7.05
2.05
589 £
-
10%
-
10%
0.1000
0.5
10%
0.0500
1.0
88%
0.3119
3.1
70%
0.2120 4.1
%
Side effects 20%
25%
-5%
Change in utility -0.10
if side effect
-0.10
0.00
Duration of side 0.50
effect (years) 0.50
10%
0.02
0.20
5%
0.02
0.20
31%
-0.06
0.70
24%
-0.14
0.65 -
Total Cost
1,208
£
1,695
Total QALY
0.6175
0.7100
0.0925
Cost per
1,956
£ QALY 2,388
£5,267
£
£
487
Cost per QALY
£
Net Benefit of T1 versus
T0
Net Benefit of T1 versus
T0 £ 5,405 £437.80
£ 4,967
1,308
£
0.8394
£
1,558
£
£7,086
1,937
-18%
0.0998
1.0
-7%
-0.08
0.05
£
629
0.5948
-0.2446
3,256
-£2,570
£4,011
-£3,075
e.g. 1,000 samples …
C-E plane
Cost Effectiveness Acceptability of T1 versus T0
£2,000
100%
CEACs
£1,500
£1,000
90%
80%
70%
Probability Cost Effective
Model
Inc Cost
£500
£0
-1.4
-1.2
-1
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
-£500
60%
T1
50%
T0
40%
30%
-£1,000
20%
10%
-£1,500
0%
-£2,000
£-
£20,000
£40,000
Inc QALY
identify single or
groups of parameters
£80,000
£100,000
£120,000
Threshold (MAICER)
EVPI
£1,400
Value of Information
Partial EVPIs
£60,000
£1,200
£1,000
£800
£600
£400
£200
£-
All Six
Trial +
Utility
Utility
Study
Trial on %
Response
propose specific data collections (n)
calculate expected value
of different samples
EVI for % Responders to T0
Value of Information
Partial EVSIs
Durations
250
200
150
EVSI (n)
EVPI
100
50
0
0
50
100
Sample Size (n)
150
200
£140,000

d
NB(d, )
i
-i
= uncertain model parameters
= set of possible decisions
= net benefit (λ*QALY – Cost) for decision d, parameters 
= parameters of interest – possible data collection
= other parameters (not of interest, remaining uncertainty)
Expected net benefit
(1) Baseline decision
=
max E NB(d,  )
d
(2) Perfect Information on i
=
Partial EVSI = (3) – (1)


d
two expectations
Partial EVPI = (2) – (1)
(3) Sample Information on i

E i max E  i NB(d,  ) |  i 
=



E X max E i NB(d,  ) |  i 
d
Expected Value of Sample Information
0)Decision model, threshold, priors for uncertain parameters
1) Simulate data collection:
• sample parameter(s) of interest once ~ prior
• decide on sample size (ni)
(1st level)
• sample the simulated data | parameter of interest
2) combine prior + simulated data --> simulated posterior
3) now simulate 1000 times
parameters of interest ~ simulated posterior
unknown parameters ~ prior uncertainty (2nd level)
4) calculate best strategy = highest mean net benefit
5) Loop 1 to 4 say 1,000 times Calculate average net benefits
6) EVSI parameter set = (5) - (mean net benefit | current information)
e.g. 1000 * 1000 simulations
Bayesian Updating: Normal
Prior
0= mean, 0= uncertainty in mean (standard deviation)
I 0  1 /  02 = precision of the prior mean
2pop = patient level uncertainty ( needed for update formula)
Simulated Data
X
= sample mean (further data collection e.g. clinical trial ).
 2   2pop / n = sample variance
I s  1 /  2
= precision of the sample mean .
Simulated Posterior
N(1, 1 )
2


/n  2
 0 0   s 
pop
2
 0
1   2
1 
2





/
n
0   s
0
pop


Normal Posterior Variance – Implications
2


/n  2
pop
2
 0
1   2
  2 / n 
pop
 0

.
• 1 always < 0
• If n is very small, 1 = almost 0
• If n is very large, 1 = almost 0
Normal Bayesian Update (n=50)
Prior
Frequency (1000 samples)
300
250
Posterior after
sample 1
200
150
Posterior after
sample 2
100
Posterior after
sample 7
50
0
0%
20%
40%
60%
Response Rate (T0)
80%
100%
Bayesian Updating: Beta / Binomial
• e.g. % responders to a treatment
Prior
• % responders ~ Beta (a,b)
Simulated Data
• n cases,
• y successful responders
Simulated Posterior
• % responders ~ Beta (a+y,b+n-y)
Bayesian Updating: Gamma / Poisson
e.g. no. of side effects a patient experiences in a year
Prior
side effects per person
~ Gamma (a,b)
Simulated Data
• n samples, (y1, y2, … yn)
• from a Poisson distribution
Simulated Posterior
• mean side effects per person ~ Gamma (a+ yi , b+n)
Bayesian Updating without a Formula
• WinBUGS
• Put in prior distribution
Conjugate Distributions
Beta
Inverted Beta
• Put in data
Cauchy 1
Cauchy 2
• MCMC gives posterior
Chi
(‘000s of iterations)
Chi² 1
• Use posterior in model
Chi² 2
• Loop to next data sample Erlang
Expon.
Fisher
Gamma
• Other approximation methods
Inverted Gamma
Gumbel
Laplace
Logistic
Lognormal
Normal
Pareto
Power
Rayleigh
r-Distr.
Uniform
Student
Triangular
Weibull
EVSI Results for Illustrative Model
Value of Information
EVI for % Responders to T0
250
200
150
EVSI (n)
EVPI
100
50
0
0
50
100
Sample Size (n)
150
200
Common Properties of EVSI curve
• Fixed at zero if no sample is collected
• Bounded above by EVPI, monotonic, diminishing returns
? EVSI (n) = EVPI * [1 – exp -a*sqrt(n) ]
EVI for % Responders to T0
Value of Information
EVSI = EVPI * [1-
EXP( -0.3813 * sqrt(n) ]
250
200
EVSI (n)
150
EVPI
100
50
Exponential fit
0
0
50
100
150
Sample Size (n)
200
Correct 2 level EVPI Algorithm
0)Decision model, threshold, priors for uncertain parameters
1) Simulate data collection:
• sample parameter(s) of interest once ~ prior
• decide on sample size (ni)
(1st level)
• sample the simulated data | parameter of interest
2) combine prior + simulated data --> simulated posterior
3) now simulate 1000 times
fixed at sampled value
parameters of interest ~ simulated posterior
unknown parameters ~ prior uncertainty (2nd level)
4) calculate best strategy = highest mean net benefit
5) Loop 1 to 4 say 1,000 times Calculate average net benefits
6) EVPI parameter set = (5) - (mean net benefit | current information)
e.g. 1000 * 1000 simulations
Shortcut 1 level EVPI Algorithm
1) Simulate data collection:
• sample parameter(s) of interest once ~ prior
• decide on sample size (ni)
(1st level)
• sample the simulated data | parameter of interest
2) fix remaining unknown parameters
2) combine prior + simulated data --> simulated posterior
constant at prior mean value
3) now simulate 1000 times
E i maxofNB(d,
 i~| simulated
parameters
interest
i   i )posterior
d
unknown parameters ~ prior uncertainty (2nd level)
4) calculate best strategy = highest mean net benefit
5) Loop 1 to 4 say 1,000 times Calculate average net benefits
6) EVPI parameter set = (5) - (mean net benefit | current information)


Accurate
if .. (a) net benefit functions are linear functions of the -i for all d,i,
and (b) i and -i are independent.
How many samples for convergence ?
E V P I e s tim a te s - Im p a c t o f M o re S a m p le s
600
550
5 5 0 -6 0 0
500
5 0 0 -5 5 0
450
400
4 5 0 -5 0 0
350
4 0 0 -4 5 0
300
3 5 0 -4 0 0
250
3 0 0 -3 5 0
£334
200
2 5 0 -3 0 0
150
2 0 0 -2 5 0
100
•
•
500
750
1000
2000
5000
1000
10000
750
500
100
10
K - o u te r le ve l
10
100
1 5 0 -2 0 0
50
0
1 0 0 -1 5 0
5 0 -1 0 0
0 -5 0
J - in n e r le ve l
outer level - over 500 samples, converges to within 1%
inner level - required 10,000 samples, to converge to within 2%.
How wrong is US 1 level EVPI approach
for a non-linear model?
• E.g – squared every model parameter
• Adjusted R2 for simple linear regression = 0.86
Part b: Comparison of 1 Level US versus 2 Level US in the Two Models
150
100
Linear Model
Non-linear Model
50
T r ia l
U tility O n ly
Difference
0
T r ia l + U tility
-5 0
D u r a tio n s
-1 0 0
Main Illustrative Model
-1 5 0
-2 0 0
-2 5 0
2nd Model (non-linear)
Maximum of Monte Carlo Expectations:Upward Bias
• simple Monte Carlo estimates are unbiased
• But, bias occurs when use maximum of Monte Carlo estimates.
E{max(X,Y)} > max{E(X),E(Y)}.
E.g. X ~ N(0,1), Y ~ N(0,1)
• E{max(X,Y)} = 1.2 ,
max{E(X),E(Y)} = 1.0
• If variances of X and Y reduces to 0.5, E{max(X,Y)} = 1.08
• E{max(X,Y)} continues to fall variance reduces, but stays > 1.0
1
Partial EVPI =
K

1

max 


d 1toD J
k 1 

K
 NBd , 
J
j 1
i
j


    


i
i
k
1 L

NB(d,  l ) 
- dmax

1toD  L
 l 1

• Illustrative model:- 1000*1000 not enough to eliminate bias.
• Increasing no. of samples reduces bias, since it reduces the
variances of the Monte Carlo estimates.
Computation Issues: Emulators
•
Gaussian Processes (Jeremy Oakley, CHEBS)
•
•
•
F() ~ NB(d, )
Bayesian non-linear regression (complicated maths)
Assumes only a smooth functional form to NB(d, )
Benefits
1. Can emulate complex time consuming models with formula
i.e. speed up each sample
2. Can produce a quick approximation to inner expectation for
partial EVPI
3. Similar quick approximation for partial EVSI
but only for one parameter
Summary
1. 2 level algorithm for EVPI and EVSI is correct approach
2. Bayesian Updating for Normal, Beta, Gamma OK
Others – WinBUGS / approximations
3. There are issues of computation
4. Shortcut 1 level EVPI algorithm is accurate if …
(a) net benefit functions are linear and
(b) the parameters are independent.
5. Emulators (e.g. Gaussian Processes) can be helpful