Appendix
CEAC/EVPI analysis and statistical inference σ𝑁𝑀𝐵(𝜆) σ𝑁𝑀𝐵(𝜆)
The CEAC can be constructed based on (re)sampling procedures, but also directly given a normally
distributed net monetary benefit (NMB) estimator ̅̅̅̅̅̅̅
𝑁𝑀𝐵 (λ) [39]:
̅̅̅̅̅̅̅ (λ) > 0}
CEAC(λ) = P{𝑁𝑀𝐵
(1)
Equation (1) thus yield the probability that the NMB estimator ̅̅̅̅̅̅̅
𝑁𝑀𝐵 is positive. Based on the central
limit theorem (CLT), the ̅̅̅̅̅̅̅
𝑁𝑀𝐵(λ) is normally distributed [48]. Based on the standard normal
̅̅̅̅̅̅̅ (λ) – NMB(λ)) / σ𝑁𝑀𝐵(λ) ~ N(0,1) the CEAC (λ) is calculated as follows
distribution with a Z score = (𝑁𝑀𝐵
[48]:
̅̅̅̅̅̅̅(λ) > 0} = P{Z > - NMB(λ) / σ𝑁𝑀𝐵(λ)}
CEAC (λ) = P{𝑁𝑀𝐵
(2)
̅̅̅̅̅̅̅(λ)} = P{Z + ̅̅̅̅̅̅̅
CEAC (λ) = P{𝑁𝑀𝐵
𝑁𝑀𝐵(λ) / σ𝑁𝑀𝐵(λ) > 0}
(3)
where Z = standard normally distributed random variable and σ𝑁𝑀𝐵(λ) = standard deviation of the NMB
estimator.
A normal cumulative density function of the CEAC is then given by:
̅̅̅̅̅̅̅ (λ) / σ𝑁𝑀𝐵(λ))
CEAC (λ) = Ф(𝑁𝑀𝐵
where Ф (•) = standard normal cumulative density function.
An estimate of CEAC (λ) is then given by:
(4)
̅̅̅̅̅̅(λ)/ σ𝑁𝑀𝐵(λ))
CEAC (λ) = Ф(𝑛𝑚𝑏
(5)
where ̅̅̅̅̅̅
𝑛𝑚𝑏(λ) and σ𝑁𝑀𝐵(λ) are sample estimates.
̅̅̅̅̅̅(λ)
Because a parametric (1-α) one-sided lower-bound confidence interval (CI) has a lower bound 𝑛𝑚𝑏
– z1-α σ𝑁𝑀𝐵(λ), the (1-α) CI where the lower bound of the interval is not above zero is given by the
inequality nmb(λ) – z1-α σ𝑁𝑀𝐵(λ) ≤ 0. By determining the cumulative density function of this term
equation 6 follows:
̅̅̅̅̅̅(λ) / σ𝑁𝑀𝐵(λ)) ≤ 1- α
Ф(𝑛𝑚𝑏
(6)
From equation (5) and (6) it follows that the CEAC (λ) is less or equal to the chosen level of confidence.
̅̅̅̅̅̅(λ) – z1-α σ𝑁𝑀𝐵(λ) and can be used to
A one-sided lower bound confidence interval is then given by 𝑛𝑚𝑏
test the null hypothesis H0: ̅̅̅̅̅̅̅
𝑁𝑀𝐵(λ) ≤ 0 (i.e., control treatment should be continued) against the
research hypothesis H1: ̅̅̅̅̅̅̅
𝑁𝑀𝐵(λ) > 0 (i.e., new treatment should be implemented). Equivalently, the p
value of a test being below or above the pre-specified level of significance can be used. A frequentist
interpretation of a CEAC is possible because the relationship between p value and CEAC can be
expressed by p(NMB (λ)) = 1 – CEAC (λ). Graphically this is reflected by the fact that the CEAC is simply
the mirror image of a p-value curve [39]. CEACs in the context of frequentist analyses can be regarded
as a particular case of Bayesian inference with use of non-informative priors [48]. Hence, the prior
probability of H0 (i.e., for a given willingness to pay a new intervention is not more cost-effective than
its comparator) being false is assumed to be 0.5.
CEACs can also be applied using a non-parametric bootstrap approach [39]. The bootstrap makes no
assumption about the distribution of data in the population. It works asymptotically, i.e., as the size of
the original data sample increases, the bootstrap sampling distribution will tend to move towards the
true sampling distribution. The bootstrap replicates of the NMB statistic nmb * b(λ), b = 1, ... , B, provide
an estimate of the sampling distribution of the NMB estimator NMB (λ), defined as
1
F(a) = 𝐵 ∑𝐵𝑏=1 𝐼(𝑛𝑚𝑏 ∗𝑏 (λ) ≤ a)
(7)
where F is a sampling distribution of the NMB estimator for any real value a and I denotes the standard
indicator function. The bootstrap estimate of the CEAC can thus be calculated as:
1
𝐵
CEAC (λ) = ∑𝐵𝑏=1 𝐼{𝑛𝑚𝑏 ∗𝑏 (λ) > 0} = 1 − 𝐹(0)
(8)
As shown for the parametric approach, there is a relationship between the CEAC and the bootstrap
confidence intervals for calculating the NMB: a (1-α) one-sided lower-bound bootstrap confidence
interval is given by 𝑛𝑚𝑏 α (λ), where 𝑛𝑚𝑏 α (λ) is the α-quantile of 𝐹, i.e., 𝑛𝑚𝑏 α (λ) = F-1(α). The
quantiles of 𝐹 are given by the [(B+1)α] and the [(B+1)(1-α)] ordered values of the bootstrap estimates
𝑛𝑚𝑏*b(λ), b = 1, . . . , B, respectively. The lower bound of a (1-α) confidence interval is not above zero
if nmbCEAC(λ) = F-1(α) ≤ 0, or equivalently if α ≤ F(0) [39].
Because CEACs may mislead policy makers by providing insufficient information on the consequences
of an incorrect decision [40,41] uncertainty is measured by the EVPI. Since perfect information would
allow avoiding the chance of making a wrong decision when adopting a novel intervention, the EVPI
can be defined as [8]:
EVPI = λ * σ𝑁𝑀𝐵(λ) * L(D0),
(9)
where D0 = │NMB(λ) NT - I0│/ σ𝑁𝑀𝐵(λ), λ = willingness to pay, σ𝑁𝑀𝐵(λ) = standard deviation of NMB, and
L(D0) = unit normal loss integral for standardized distance D0, NMB(λ) NT = mean incremental net
benefit of the new treatment, and I0 = point of indifference between new treatment and standard (I0
= 0).