USE OF LAPLACE APPROXIMATIONS TO SIGNIFICANTLY IMPROVE THE EFFICIENCY
OF EXPECTED VALUE OF SAMPLE INFORMATION COMPUTATIONS.
Purpose: to describe a novel process for transforming the efficiency of
partial EVSI computations in health economic decision models.
Background: Brennan et al. (1,2) Claxton et al (3) have promoted EVSI
• as a measure of the societal value of research designs
Bayes
• to identify optimal sample sizes for primary data collection.
Current Mathematical Formulation and 2 level algorithm: Partial EVSI for Parameters
Current methods involve:
• a two level Monte Carlo simulation algorithm
• a large number of calculations.

= the parameters for the model (uncertain currently).
d
= set of possible decisions or strategies.
NB(d, ) = the net benefit for decision d, and parameters 

d
Step 2: Evaluate Expected Value of the Proposed Research
(Perform the [loop] using Monte Carlo simulation a large number of times)
Expected net benefit | proposed data: =


E Xi max E NBd ,  | X i 
d
Partial Expected Value of the Proposed Sample Information (3) – (1)



EVSI  E Xi max E NBd ,  | X i   max E NB(d, )
d
d
^
2 Level Algorithm
1000 x 1000
iterations
inner expectation of
net benefit over | Xi
 

  
(1,000 outer x 1,000 inner simulations)
£1,400
(5)

 ^  b 
 ^ 
E NB  | X i   NB d ,     i- d , i NB d , i   i  i NB d , i  NB d ,  

 i 1 



 
 
E Xi  max  NB d ,    max E NB(d, )(6)
d
 
 d  
= 15 minutes
£1,000
1st order Laplace
Approximation
1000
iterations
(2)
(3)
(4)
  
2nd order term
= the posterior mode of the probability distribution for  uncertain parameters
+ and - are the solutions to a series of non-linear equations incorporating the posterior
^
^
mode  , and J the observed information (J= -l’’(  ) ) i.e.the 2nd derivative
of the prior likelihood function. J-1 is the posterior variance covariance matrix.
α+ and α-are analytic expressions in terms of the prior, the first derivative of the
likelihood function and the function v(  ) itself.
a
b
c
d
Parameters
e
g
Uncertainty in Means Patient Level Variability
Standard Deviations Standard Deviations
T0
T1
(T1 - T0)
T0
T1
T0
T1
£ 1,000 £ 1,500 £
500 £
1 £
1 £
500 £
500
10%
8%
-2%
2%
2%
25%
25%
5.20
6.10
0.90
1.00
1.00
4.00
4.00
£
400 £
400 £
£
200 £
200 £
200 £
200
Cost of drug
% admissions
Days in Hospital
Cost per Day
% Responding
Utility change if respond
Duration of response (years)
70%
0.3000
3.0
80%
0.3000
3.0
10%
-
10%
0.1000
0.5
10%
0.0500
1.0
20%
0.2000
1.0
20%
0.2000
2.0
% Side effects
25%
20%
-5%
Change in utility if side effect
-0.10
-0.10
0.00
Duration of side effect (years)
0.50
0.50
Total Cost
£ 1,208 £ 1,695 £
487
Total QALY
0.6175
0.7100
0.0925
Cost per QALY
£ 1,956 £ 2,388 £ 5,267
Net Benefit
(threshold = £10,000 per QALY)
£ 4,967 £ 5,405 £
438
10%
0.02
0.20
5%
0.02
0.20
20%
0.10
0.80
10%
0.10
0.80
Bayesian Updating: the normal case
The 2 level algorithm
0
= prior mean
0
= prior standard deviation
I0
= 1/σ02
pop
= patient level standard deviation
n.
= sample size
Bayesian Updating: the normal case
1 st order Laplace approximation
prior precision
= sample mean from N~(μ0sample, σ2pop/n)
= 2pop/n = sample variance
IS
= 1/2X = precision of the sample mean
I 0 0  I S X
I0  I S
= posterior mean
  /n  2

  2   2 / n  0 = posterior standard deviation
pop
 0

1  
2
pop
^
Posterior mode  = Posterior mean μ1
2X
1 
£400
utility observational study
£200
% response trial
0
50
100
150
200
250
Sample Size (n)
1st order Laplace
marginally below
2 level algorithm
2 level algorithm
using 1000 x 1000 is
a slight over-estimate
(150,000 simulations)
£1,400
£1,200
% response/ utility trial +
duration obsn'l study
£1,000
% response and utility trial
£800
£600
duration of response
observational study
£400
utility observational study
£200
% response trial
£0
0
50
100
150
200
250
Sample Size (n)
% Difference: (2 level Algorithm) - (1st order Laplace Approximation)
duration of
utility
response
% response
observational observational % response
n
trial
study
study
and utility trial
0
0%
0%
0%
0%
10
22%
8%
0%
13%
25
19%
2%
5%
8%
50
11%
1%
8%
5%
100
17%
1%
2%
2%
200
15%
4%
2%
3%
% response/
utility trial +
duration
obsn'l study
0%
5%
4%
4%
3%
3%
Conclusions
μ0sample = random sample from N~(0, 0)
X
duration of response
observational study
= 18 seconds
Results are very
similar order of
magnitude
f
£600
EVSI :- 1st Order Laplace Approximations
Illustrative Model with Normally Distributed Uncertain Parameters
• Two treatments T1 and T0
• normally distributed cost and benefit parameters
Parameter Mean Values given
Existing Evidence
% response and utility trial
£800
£0
Comparison
Illustrative Model
% response/ utility trial +
duration obsn'l study
£1,200
Only One Expectation
in the formula is recalculated for each simulated dataset collected Xi
(1)
^ b 
 ^   Laplace
E v  | X   v     i-  i v  i   i  i v  i  v  
(5)
  __________________________________________
 
i 1 
______


EVSI :- 2 Level Algorithm Results
Computation Times
We use Laplace approximation to evaluate the EVSI inner expectation , hence
^
Methodology: Laplace Approximation
Sweeting and Kharroubi4 have developed a 2nd order approximation
to evaluate the an expectation of function v() given available data X.
1st order term
d
outer expectation of
net benefit over Xi
Posterior mode 
This is a 2 level simulation due to 2 expectations (e.g. 1000 x 1000 model simulations)
  

E Xi max E NBd ,  | X i 
For EVSI the first term in the formula is
1st order approximate EVSI =
Decide on research design i.e. parameters to collect data on (i), sample size, etc.
•[Start loop]
• Sample the data collection:
• a) sample the true underlying value for parameter of interest (i) from its prior uncertainty
• b) sample simulated data ( Xi ) given the sampled true underlying value of i
•Synthesise existing evidence with simulated data
result is a simulated posterior probability distribution for value of parameter of interest.
•Evaluate the net benefit for each strategy given the new data and make a ‘revised decision’
(Re-run probabilistic analysis using Monte Carlo simulation on the decision model )


Results
^
Step 1: Define a Data Collection Exercise, Simulate the Variety of Possible Results
i
= the parameters of interest - we propose to collect data on these
 -i
= the other parameters (those not of interest, i.e. remaining uncertainty)
Net benefit of ‘revised decision’ | simulated data[i]: = max E NBd ,  | X

i
d
•[Loop back]
Application of Laplace Approximation to EVSI formula
EVSI (£)
Step 0: Analysis based on Current information
•Set up the decision model
•Characterise the uncertainty in each parameter with prior probability distributions
•Calculate the baseline decision and its expected net benefit
Given current information chose decision giving maximum expected net benefit.
Expected net benefit | current information =
max E NB(d,  )
.
EVSI (£)
Alan Brennan, Samer Kharroubi University of Sheffield, England. a.brennan@sheffield.ac.uk
CHEBS
(1). This novel application of Laplace approximations short-cuts the calculation of EVSI
(2). A simple illustrative model shows that EVSI calculations using the new approach are in line with
those produced by the longer 2 level Monte-Carlo sampling method.
(3). Computation time reductions depend on the number of Monte-Carlo samples used to evaluate
EVSI, but can be seen to be up to 100 times shorter using the approximation method.
(4). Application to more complex models and assessment of the value of the 2 nd order term are needed
1 Brennan, A., Chilcott, J. B, Kharroubi, S, O'Hagan, A. Calculating Expected Value of Perfect Information:Resolution of Conflicting Methods via a Two Level Monte Carlo Approach, presented at the 24th Annual
Meeting of SMDM, October 23rd, 2002, Washington. 2002. Submitted - Journal of Medical Decision Making
2 Brennan, A. B., Chilcott, J. B., Kharroubi, S., O'Hagan, A. A Two Level Monte Carlo Approach to Calculation
Expected Value of Sample Information: How To Value a Research Design. Presented at the 24th Annual
Meeting of SMDM, October 23rd, 2002, Washington. 2002.
3 Claxton, K, Ades, T. Efficient Research Design: An Application of Value of Information Analysis to an
Economic Model of Zanamivir. Presented at the 24th Annual Meeting of the Society for Medical Decision
Making, October 21st, 2002, Washington. 2002.
4 Sweeting, J, Kharroubi, S. Some New Formulae for Posterior Expectations and Bartlett Corrections.
Sociedad de Estadistica e Investigacion Operative Test, (Accepted) 2003
Acknowledgements: Particular thanks to Professor Tony O’Hagan for encouraging our ongoing work.