Model criticism for evidence synthesis models of
infectious disease
Anne Presanis
MRC Biostatistics Unit
in collaboration with
David Ohlssen (Novartis)
David Spiegelhalter (Cambridge)
Daniela De Angelis (MRC BSU)
Acknowledgements: Paul Birrell (MRC BSU), Tony Ades (Bristol), Richard Pebody (HPA)
UCL Biostatistics Symposium
18 Oct 2012
A. M. Presanis (MRC BSU)
Conflict in complex evidence synthesis
UCL, 18 Oct 2012
1 / 26
Introduction
Outline
1
Introduction
Motivation
Evidence synthesis
2
Criticising evidence syntheses
Influence & Conflict
3
Conflict diagnostics
General method
4
Examples
HIV prevalence in women
Influenza severity
Multivariate eg: rat weights
5
Discussion
A. M. Presanis (MRC BSU)
Conflict in complex evidence synthesis
UCL, 18 Oct 2012
2 / 26
Introduction
Motivation
Motivation
The implementation and evaluation of public health policies aimed
at control and prevention of epidemics rely crucially on knowledge
of fundamental aspects of the disease of interest, such as
prevalence, incidence and severity.
These characteristics are typically not easily measurable as
little direct data are available on them.
There is plenty of indirect information on functions of these
quantities.
Estimation through the synthesis of diverse and fragmented sources
of evidence is often feasible.
A. M. Presanis (MRC BSU)
Conflict in complex evidence synthesis
UCL, 18 Oct 2012
3 / 26
Introduction
Evidence synthesis
Evidence synthesis: long-established idea
Methods for combining evidence are not new:
The Bayesian paradigm
combining prior knowledge with new
Meta-analysis
combining studies of same type
Confidence Profile Method
[Eddy et al (1992)]
combining information of different types/study designs
Multi-parameter evidence synthesis
[Spiegelhalter et al (2004), Ades & Sutton (2006)]
health technology assessment
[Spiegelhalter & Best (2003), Demiris & Sharples (2006)]
infectious disease epidemiology
[Welton & Ades (2005), Goubar et al (2008), Sweeting et al (2008), De Angelis et al (2009),
Presanis et al (2011a), Birrell et al (2011), Albert et al (2011)]
environmental epidemiology [Ryan (2008), Jackson et al (2008), Jones et al (2009)]
other fields [Henderson et al (2010), Clark et al (2010)]
A. M. Presanis (MRC BSU)
Conflict in complex evidence synthesis
UCL, 18 Oct 2012
4 / 26
Introduction
Evidence synthesis
Statistical formulation
Interest: estimation of θ = (θ1 , θ2 . . . , θk ) on the basis of a
collection of data y = (y1 , y2 . . . , yn )
Each yi provides information on
a single component of θ (“direct” data), or
a function of one or more components, i.e. on a quantity
ψi = f (θ) (“indirect” data)
Thus inference is conducted on the basis of both direct and
indirect information.
Maximum likelihood: L =
Qn
i=1 Li (yi
Bayesian: p(θ | y) ∝ p(θ) × L
A. M. Presanis (MRC BSU)
| θ)
Conflict in complex evidence synthesis
UCL, 18 Oct 2012
5 / 26
Introduction
Evidence synthesis
Graphical representation (DAG)
Basic parameters
θ1
...
θi
θi+1
...
θk
Functional parameters
ψ1
...
ψj
ψj+1
...
ψn
Data
y1
...
yj
yj+1
...
yn
A. M. Presanis (MRC BSU)
Conflict in complex evidence synthesis
UCL, 18 Oct 2012
6 / 26
Introduction
Evidence synthesis
Simple example: HIV prevalence
HIV prevalence
π
δ
π(1 − δ)
Proportion diagnosed
Prevalence of undiagnosed
infection
y1
A. M. Presanis (MRC BSU)
Conflict in complex evidence synthesis
UCL, 18 Oct 2012
7 / 26
Introduction
Evidence synthesis
Simple example: HIV prevalence
HIV prevalence
π
δ
π(1 − δ)
y1
A. M. Presanis (MRC BSU)
Proportion diagnosed
Prevalence of undiagnosed
infection
y2
Conflict in complex evidence synthesis
UCL, 18 Oct 2012
7 / 26
Introduction
Evidence synthesis
Simple example: HIV prevalence
HIV prevalence
π
δ
Prevalence of undiagnosed
infection
π(1 − δ)
y1
A. M. Presanis (MRC BSU)
y2
Proportion diagnosed
y3
Conflict in complex evidence synthesis
UCL, 18 Oct 2012
7 / 26
Introduction
Evidence synthesis
Motivation II
Evidence synthesis leads to complex probabilistic models
Combination of all available relevant data sources ideally
should lead to more precise estimates
Multiple sources informing a single parameter ⇒ potential for
conflicting evidence
Sparsity of data ⇒ parameters unidentifiable without further
model constraints, e.g. exchangeability
We can already fit such models
The next step? How do we assess and criticise them?
A. M. Presanis (MRC BSU)
Conflict in complex evidence synthesis
UCL, 18 Oct 2012
8 / 26
Criticising evidence syntheses
Influence & Conflict
What are the components of inference?
Model criticism requires understanding the role of three
components: the model structure (part of prior? [Box (1980), Efron (2010)]);
the prior; and the likelihood.
Influence & Conflict
Detection & quantification: When does heterogeneity become
conflict?
Cross-validatory / posterior / mixed predictive methods
[Rubin (1984), Gelman et al (1996), Bayarri & Berger (1999), Marshall & Spiegelhalter (2007)]
Conflict diagnostics, including prior-predictive
[Box (1980), O’Hagan (2003), Evans & Moshonov (2006), Gasemyr & Natvig (2009)]
Resolution:
Drop suspect/biased data
Expand the model to account for biased data
May require evidence expansion (external evidence) for identifiability
A. M. Presanis (MRC BSU)
Conflict in complex evidence synthesis
UCL, 18 Oct 2012
9 / 26
Conflict diagnostics
General method
Defining/measuring conflict
Existing methods can be generalised to the question:
How do we compare two sets of evidence?
prior-data [prior-predictive methods]
posterior-data [posterior-predictive methods]
data-data [conflict diagnostics]
More generally: in a DAG, consider node-splitting to carry out
posterior-posterior comparison for two disjoint sets of evidence
(data & priors) informing a single node [Marshall & Spiegelhalter (2007),
Dias et al (2010), Ohlssen & Spiegelhalter, Presanis et al (to be submitted)]
A. M. Presanis (MRC BSU)
Conflict in complex evidence synthesis
UCL, 18 Oct 2012
10 / 26
Conflict diagnostics
General method
Simple example: prior-likelihood conflict
Informative prior Reference prior Informative prior
θ
θ1
y
y
A. M. Presanis (MRC BSU)
Conflict in complex evidence synthesis
θ2
UCL, 18 Oct 2012
11 / 26
Conflict diagnostics
General method
More generally, conflict at any node θ
pa(θ)
cp(θ)
θ
si(θ)
ch(θ)
pa(θ) = Parents
ch(θ) = Children
si(θ) = Siblings
cp(θ) = Co-parents
A. M. Presanis (MRC BSU)
Conflict in complex evidence synthesis
UCL, 18 Oct 2012
12 / 26
Conflict diagnostics
General method
More generally, conflict at any node θ
pa(θ)
cp(θ)
θ
si(θ)
pa(θ)
ch(θ)
pa(θ) = Parents
ch(θ) = Children
si(θ) = Siblings
cp(θ) = Co-parents
A. M. Presanis (MRC BSU)
cp1(θ)
θ1
θ2
si(θ)
ch1(θ)
ch2(θ)
cp2(θ)
Conflict in complex evidence synthesis
UCL, 18 Oct 2012
12 / 26
Conflict diagnostics
General method
A general conflict measure c
θ1
θ2
0
δ = g(θ1) − g(θ2)
For general ‘separator’ node(s) θ, split into θ1 based on data y1 and θ2
based on data y2 . Define δ = g (θ1 ) − g (θ2 ) where g is a function such
that it is reasonable to assume a Uniform prior for g (θ1 ). Then define
c = Pr {pδ (δ|y1 , y2 ) < pδ (0|y1 , y2 )}
where pδ is the posterior density of δ.
A. M. Presanis (MRC BSU)
Conflict in complex evidence synthesis
UCL, 18 Oct 2012
13 / 26
Conflict diagnostics
General method
Defining c
Univariate node-split
For symmetric unimodal distributions p(δ|y1 , y2 ), define tail area
c = 2 min(p(δ > 0|y1 , y2 ), 1 − p(δ > 0|y1 , y2 ))
For skewed and/or multi-modal distributions, we may take a kernel
density estimate to obtain c.
Multivariate node-splits: p(δ|y1 , y2 ) multivariate normal
Define the standardised discrepancy measure
∆ = Ep (δ)T Covp (δ)−1 Ep (δ)
and compare to a X 2 distribution: c = 1 − Pr Xk2 ≤ ∆ .
A. M. Presanis (MRC BSU)
Conflict in complex evidence synthesis
UCL, 18 Oct 2012
14 / 26
Conflict diagnostics
General method
Defining c
Multivariate node-splits: p(δ|y1 , y2 ) symmetric/uni-modal
Samples δ i from posterior, calculate Mahalanobis distance from
mean
∆i = {δ i − Ep (δ)}T Covp (δ)−1 {δ i − Ep (δ)}
Then define c = Pr {∆i > ∆}, the proportion of points that are
further away from the mean than is 0.
Multivariate node-splits: p(δ|y1 , y2 ) skew/multi-modal
Obtain a kernel density estimate of the posterior to calculate the
probability that the posterior density at δ is less than (at a lower
contour than) at 0:
c = Pr {p(δ|y1 , y2 ) < p(0|y1 , y2 )}
A. M. Presanis (MRC BSU)
Conflict in complex evidence synthesis
UCL, 18 Oct 2012
14 / 26
Examples
HIV prevalence in women
Example: HIV prevalence in women
Ades & Cliffe, Medical Decision Making (2002)
Already diagnosed?
HIV infection?
Risk group?
SSA
a
Yes
Yes
c
No
1−c
Yes
IDU
b
d
Rest
1−f
Yes
g
No
1−g
No
1−d
(1 − a − b)
f
No
Yes
e
No
Yes
h
No
1−h
1−e
A. M. Presanis (MRC BSU)
Conflict in complex evidence synthesis
UCL, 18 Oct 2012
15 / 26
Examples
HIV prevalence in women
Example: HIV prevalence in women
Ades & Cliffe, Medical Decision Making (2002)
Description of data
Parameter
Proportion born SSA
p1 = a
Proportion IDU last 5 years
p2 = b
HIV prevalence, women born in SSA
p3 = c
HIV prevalence in female IDUs
p4 = d
HIV prevalence, women not born in SSA
p5 = [db + e(1 − a − b)]/(1 − a)
Overall HIV seroprevalence in pregnant women
p6 = ca + db + e(1 − a − b)
Diagnosed HIV in SSA women, out of all diagnosed HIV
p7 = fca/[fca + gdb + he(1 − a − b)]
Diagnosed HIV in IDUs, out of non-SSA diagnosed HIV
p8 = gdb/[gdb + he(1 − a − b)]
Overall proportion of HIV diagnosed
p9 = [fca + gdb + he(1 − a − b)]/[ca + db + e(1 − a − b)]
Proportion of infected IDUs diagnosed
p10 = g
Proportion of serotype B in infected women from SSA
p11 = w
Proportion of serotype B in infected women not from SSA
p12 = [db + we(1 − a − b)]/[db + e(1 − a − b)]
A. M. Presanis (MRC BSU)
Conflict in complex evidence synthesis
UCL, 18 Oct 2012
15 / 26
Examples
HIV prevalence in women
HIV eg: two types of node-split
a
b
c
d
e
...
p1
p2
p3
p4
p5
p6
n1
y1
n2
y2
A. M. Presanis (MRC BSU)
n3
y3
n4
y4
n5
y5
Conflict in complex evidence synthesis
...
...
n6
y6
...
UCL, 18 Oct 2012
16 / 26
Examples
HIV prevalence in women
HIV eg: two types of node-split
(a) basic parameter level split
a1
a2
p1
b
...
p2
n1
y1
A. M. Presanis (MRC BSU)
n2
y2
...
...
...
p5
p6
...
...
...
n5
y5
Conflict in complex evidence synthesis
n6
y6
...
...
UCL, 18 Oct 2012
16 / 26
Examples
HIV prevalence in women
HIV eg: two types of node-split
(b) data level split
p1,1
a
b
p1,2
p2
n1
y1
A. M. Presanis (MRC BSU)
...
n2
y2
...
...
...
p5
p6
...
...
...
n5
y5
Conflict in complex evidence synthesis
n6
y6
...
...
UCL, 18 Oct 2012
16 / 26
Examples
HIV prevalence in women
HIV eg: direct vs indirect evidence
δ = x1 − x2 ,
x ∈ {a, b, c, d, g , w }
100
250
c = 0.165
bw = 0.004
c = 0.01
bw = 0.001
150
80
c = 0.155
bw = 0.001
200
60
150
100
40
100
50
20
50
0
0
0.00
0.05
0.10
0.15
0.20
0
0.00
0.01
0.02
a
0.03
0.04
0.05
120
0.01
0.02
0.03
0.04
0.05
c
5
15
c = 0.009
bw = 0.002
100
0.00
b
c = 0.31
bw = 0.033
c = 0.353
bw = 0.01
4
80
10
3
60
2
40
5
1
20
0
0
0.00
0.01
0.02
0.03
d
A. M. Presanis (MRC BSU)
0.04
0.05
0
0.0
0.2
0.4
0.6
0.8
1.0
g
Conflict in complex evidence synthesis
0.00
0.10
0.20
0.30
w
UCL, 18 Oct 2012
17 / 26
Examples
HIV prevalence in women
HIV eg: data-level cross-validation
δ = px1 − px2 ,
50
40
x ∈ 1, . . . , 12
100
c = 0.169
bw = 0.003
c = 0.009
bw = 0.002
80
250
200
30
60
150
20
40
100
10
20
0
50
0
−0.06
−0.02
0.02
0
−0.01
p1
2500
2000
0.01
0.03
−0.010
p2
2500
c = 0.038
bw = 0
60
50
40
30
20
10
0
c = 0.161
bw = 0.001
−0.02
p3
c = 0.249
bw = 0
2000
0.000
4
c = 0.245
bw = 0.034
3
4
3
1500
1000
1000
2
2
500
500
1
1
0
0
0
0e+00
−5e−04
p5
4
5e−04
p6
2
1.0
4
1
0.5
2
0
6
0.0
−0.6
−0.2
0.2
p9
A. M. Presanis (MRC BSU)
0.2
0.6
0.0
p10
0.5
−0.2
0.2
p8
4
c = 0.358
bw = 0.019
3
c = 0.042
bw = 0.032
2
1
0
−0.5
−0.8
p7
8
c = 0.129
bw = 0.032
c = 0.364
bw = 0.047
0
−0.2
2.0 c = 0.314
bw = 0.05
1.5
3
0.02
p4
1500
−1e−03
c = 0.008
bw = 0.003
0
−0.3
−0.1
0.1
p11
Conflict in complex evidence synthesis
−0.6
−0.2
0.2
p12
UCL, 18 Oct 2012
18 / 26
Examples
Influenza severity
Influenza severity
CF R = cD|H × cH|S × cS|Inf
= P r{D|H} × P r{H|S} × P r{S|Inf }
sCHR = cH|S
= P r{H|S}
ICU
Death
Hospitalised
Symptomatic
Infected
A. M. Presanis (MRC BSU)
Conflict in complex evidence synthesis
UCL, 18 Oct 2012
19 / 26
Examples
Influenza severity
2009 pandemic: model for the first wave
CFR
CIR
CHR
cD|H
cI|H
cH|S
ND
NI
NH
dD
Sero-survey data, 2008
πbaseline
cS|Inf
NS
π
IAR
NInf
Sero-survey data, 2009
NP op
dH
OD
OD:H
OI:H
OH
HPA estimates of NS
based on GP consultations/virological testing
Deaths reported to CMO/HPA Web-based hospital surveillance
Full model
[Presanis et al (2011)]
A. M. Presanis (MRC BSU)
Conflict in complex evidence synthesis
UCL, 18 Oct 2012
20 / 26
Examples
Influenza severity
2009 pandemic: model for the first wave
Sero-survey data, 2008
πbaseline
cS|Inf
NS
π
IAR
NInf
Sero-survey data, 2009
NP op
Parent model
A. M. Presanis (MRC BSU)
Conflict in complex evidence synthesis
UCL, 18 Oct 2012
20 / 26
Examples
Influenza severity
2009 pandemic: model for the first wave
cD|H
ND
cI|H
NI
cH|S
NH
dD
NS
dH
OD
OD:H
OI:H
OH
HPA estimates of NS
based on GP consultations/virological testing
Deaths reported to CMO/HPA Web-based hospital surveillance
Child model
A. M. Presanis (MRC BSU)
Conflict in complex evidence synthesis
UCL, 18 Oct 2012
20 / 26
Examples
Influenza severity
NS : parent-child conflict
Child model
cD|H
ND
πbaseline
cI|H
NI
cH|S
NS1
NH
dD
cS|Inf
NS2
OD:H
OI:H
OH
2009
NP op
Parent model
dH
OD
π
IAR
NInf
2008
HPA estimates of NS
based on GP consultations/virological testing
Deaths reported to CMO/HPA Web-based hospital surveillance
A. M. Presanis (MRC BSU)
Conflict in complex evidence synthesis
UCL, 18 Oct 2012
21 / 26
Examples
Influenza severity
Conflict & influence: ln(NS )
<1
6
8
10
12
14
1.0
1−4y
1.0
5−14y
1.0
0.8
0.8
0.8
0.6
0.6
0.6
0.4
0.4
0.4
0.2
0.2
0.2
0.0
0.0
0.0
16
6
8
10
12
14
16
ln(Number symptomatic) 1.0
ln(Number symptomatic) 1.0
0.8
0.8
0.6
0.6
0.4
0.4
0.2
0.2
0.0
0.0
25−44y
A. M. Presanis (MRC BSU)
45−64y
6
8
10
12
14
ln(Number symptomatic)
65+
Conflict in complex evidence synthesis
16
15−24y
6
8
10
12
14
16
ln(Number symptomatic)
2: Parent model
3: Child model
●
UCL, 18 Oct 2012
22 / 26
Examples
Influenza severity
Conflict & influence: ln(NS )
<1
6
8
10
12
14
1.0
1−4y
1.0
5−14y
1.0
0.8
0.8
0.8
0.6
0.6
0.6
0.4
0.4
0.4
0.2
0.2
0.2
0.0
0.0
0.0
16
6
8
10
12
14
16
ln(Number symptomatic) 1.0
ln(Number symptomatic) 1.0
0.8
0.8
0.6
0.6
0.4
0.4
0.2
0.2
0.0
0.0
25−44y
A. M. Presanis (MRC BSU)
45−64y
6
8
10
12
14
ln(Number symptomatic)
65+
Conflict in complex evidence synthesis
16
15−24y
6
8
10
12
14
16
ln(Number symptomatic)
1: Full model
2: Parent model
●
3: Child model
UCL, 18 Oct 2012
22 / 26
Examples
Influenza severity
δ = ln(NS1 ) − ln(NS2 )
1.0
1.0
<1
0.8
<1
c=0
0.8
0.6
0.6
0.4
0.4
0.2
0.2
0.0
0.0
6
8
10
12
14
16
−4
−2
0
2
4
2
4
ln(Number symptomatic)
1.0
1.0
5−14y
0.8
5−14y
c = 0.02
0.8
0.6
0.6
0.4
0.4
0.2
0.2
0.0
0.0
6
8
10
12
14
16
−4
A. M. Presanis (MRC ln(Number
BSU)
Conflict in complex evidence synthesis
symptomatic)
−2
0
UCL, 18 Oct 2012
23 / 26
Examples
Influenza severity
δ = ln(NS1 ) − ln(NS2 )
0.6
c1 = 0.058
c2 = 0.063
bw = 0.5
0.5
0.4
0.3
0.2
0.1
0.0
−4
−2
0
2
4
Difference function, 65+
A. M. Presanis (MRC BSU)
Conflict in complex evidence synthesis
UCL, 18 Oct 2012
23 / 26
Examples
Multivariate eg: rat weights
Bivariate normal model for rat weights
Gelfand et al
(1990) random
linear growth
yij
µij
∼ N(µij , σ 2 )
= φia + φib tj
φi ∼ MVN2 (β, Ω)
A. M. Presanis (MRC BSU)
Conflict in complex evidence synthesis
UCL, 18 Oct 2012
24 / 26
Examples
Multivariate eg: rat weights
Bivariate normal model for rat weights
Gelfand et al
(1990) random
linear growth
yij
µij
φ1ia
Ω\i
φ\ib
φ\ia
φ2ia
φ1ib
∼ N(µij , σ 2 )
β\i
φ2ib
= φia + φib tj
φi ∼ MVN2 (β, Ω)
σ2
yij
tj
y\ij
j
A. M. Presanis (MRC BSU)
Conflict in complex evidence synthesis
UCL, 18 Oct 2012
\i
24 / 26
Examples
Multivariate eg: rat weights
Outlier: rat 9
2
0
+
+
++ +
++
+++ + + +
+
+ + + + + + + +++ +
+ +
+++
+ ++
+
+
++ +++ +
++
+
+
++
+++++++
++ +
+++
++
++++++ ++
+
+++
+
++
+
+
++++
+ ++
+
+
+
+ +
++
+++++
+
+
+
+ ++ ++
+
++++
++
++ +
+
++++
+++++
++
+
+
+
++
++
+
+
++
+
+++ +
++
++
+
+++ ++
+++++
+++++++++
+
++ +
+
+++
+ +
++ + +++ +++ ++
++ ++ + ++++ +
++++ +
+ ++ ++ +
+
+
+
+
phi2
*
−2
+
−4
Rat 9
c = 0.006
−150
−100
−50
0
50
phi1
A. M. Presanis (MRC BSU)
Conflict in complex evidence synthesis
UCL, 18 Oct 2012
25 / 26
Discussion
Discussion
Importance of understanding what is driving conclusions, particularly
when combining multiple sources of evidence (informative priors, data
from varied sources)
A number of methods available: importance of systematic model
criticism.
Conflict p-values - useful continuous scale. When does influence lead
to conflict/inconsistency?
Choice of reference priors?
Choice of function g such that δ = g (θ1 ) − g (θ2 ) is symmetric &
unimodal?
Multiple testing problem.
Once conflict detected: why is it occurring? Biases? Incorrect model?
Accommodate the inconsistency through the introduction of bias
parameters?
A. M. Presanis (MRC BSU)
Conflict in complex evidence synthesis
UCL, 18 Oct 2012
26 / 26
References
Evidence synthesis: general and HTA
D.M. Eddy, V. Hasselblad, R. Shachter Meta-Analysis by the Confidence Profile Method. Academic Press, 1992.
D.J. Spiegelhalter, N.G. Best Bayesian approaches to multiple sources of evidence and uncertainty in complex
cost-effectiveness. modelling. Statist. Med., 22(23):3687–3709, 2003.
D.J. Spiegelhalter, K.R. Abrams, J.P. Myles Bayesian approaches to clinical trials and health-care evaluation. Wiley, 2004.
A.E. Ades, A.J. Sutton Multiparameter evidence synthesis in epidemiology and medical decision-making:current
approaches. J. R. Stat. Soc. A, 169:5–35, 2006.
N. Demiris, L.D. Sharples Bayesian evidence synthesis to extrapolate survival estimates in cost-effectiveness studies.
Statist. Med., 25(11):1960–1975, 2006.
A. M. Presanis (MRC BSU)
Conflict in complex evidence synthesis
UCL, 18 Oct 2012
27 / 26
References
Evidence synthesis: epidemiology &
ecology
N.J. Welton, A.E. Ades A model of toxoplasmosis incidence in the UK: evidence synthesis and consistency of evidence. J.
R. Stat. Soc. C, 54(2):385–404, 2005.
L. Ryan Combining data from multiple sources, with applications to environmental risk assessment. Statist. Med.,
27(5):698–710, 2008.
C. Jackson, S. Richardson, N. Best Studying place effects on health by synthesising individual and area-level outcomes.
Soc. Sci. & Med., 67(12):1995–2006, 2008.
M.J. Sweeting, D. De Angelis, M. Hickman, A.E. Ades Estimating hepatitis C prevalence in England and Wales by
synthesizing evidence from multiple data sources. Assessing data conflict and model fit. Biostatistics, 9(4):715–734 2008.
D. De Angelis, M.J. Sweeting, A.E. Ades, V. Hope, M. Ramsay An evidence synthesis approach to estimating Hepatitis C
prevalence in England and Wales. Stat. Meth. Med. Res., 18(4):361–379, 2009.
D.R. Jones, J.L. Peters, L. Rushton, A.J. Sutton, K.R. Abrams Interspecies extrapolation in environmental exposure
standard setting: A Bayesian synthesis approach. Regul. Toxicol. & Pharm., 53(3):217–225, 2009.
J.S. Clark, D. Bell, C. Chu, B. Courbaud, M. Dietze, M. Hersh, J. HilleRisLambers, I. Ibáñez, S. LaDeau, S. McMahon,
J. Metcalf, J. Mohan, E. Moran, L. Pangle, S. Pearson, C. Salk, Z. Shen, D. Valle, P. Wyckoff High-dimensional
coexistence based on individual variation: a synthesis of evidence. Ecol. Monog., 80(4):569–608, 2010.
D.A. Henderson, R.J. Boys, D.J. Wilkinson Bayesian calibration of a stochastic kinetic computer model using multiple
data sources. Biometrics, 66(1):249–256, 2010.
I. Albert, E. Espie, H. de Valk, J.-B. B. Denis A Bayesian evidence synthesis for estimating campylobacteriosis
prevalence. Risk Analysis, 31(7):1141–1155, 2011.
A. M. Presanis (MRC BSU)
Conflict in complex evidence synthesis
UCL, 18 Oct 2012
28 / 26
References
Model criticism references
D.J. Spiegelhalter, N.G. Best, B.P. Carlin, A. van der Linde Bayesian measures of model complexity and fit. J. R. Statist.
Soc. B, 64(4):583–639, 2002.
G.E.P. Box Sampling and Bayes’ inference in scientific modelling and robustness. J. R. Statist. Soc. A, 143(4):383–430,
1980.
B. Efron The future of indirect evidence. Statist. Sci., 25(2):145–157, 2010.
D.B. Rubin Bayesianly justifiable and relevant frequency calculations for the applied statistician. Ann. Appl. Statist.,
12:1151–1172, 1984.
A. Gelman, X.-L. Meng, H. Stern Posterior predictive assessment of model fitness via realized discrepancies. Statistica
Sinica, 6:733–807, 1996.
M.J. Bayarri, J.O. Berger Quantifying surprise in the data and model verification. In Bayesian Statistics 6 (eds. J. M.
Bernardo, J. O. Berger, A. P. Dawid, A. F. M. Smith). OUP, 53–82, 1999.
E.C. Marshall, D.J. Spiegelhalter Identifying outliers in Bayesian hierarchical models: a simulation-based approach.
Bayesian Analysis, 2:409–444, 2007.
A. M. Presanis (MRC BSU)
Conflict in complex evidence synthesis
UCL, 18 Oct 2012
29 / 26
References
Model criticism references II
A. O’Hagan HSSS model criticism (with discussion). In Highly Structured Stochastic Systems (eds. P.J. Green, N.L.
Hjort, S. Richardson). OUP, 2003.
M. Evans, H. Moshonov Checking for prior-data conflict. Bayesian Analysis, 1(4):893–914, 2006.
J. Gasemyr, B. Natvig Extensions of a conflict measure of inconsistencies in Bayesian hierarchical models. Scand. J.
Statist., 36:822–838, 2009.
D. Ohlssen, D.J. Spiegelhalter Conflict diagnostics in directed acyclic graphs. Workshop “Explaining the results of a
complex probabilistic modelling exercise: conflict, consistency and sensitivity analysis”, 2006.
S. Dias, N.J. Welton, D.M. Caldwell, A.E. Ades Checking consistency in mixed treatment comparison meta-analysis.
Stat. Med., 29(7):932–944, 2010.
A.M. Presanis, D. Ohlssen, D.J. Spiegelhalter, D. De Angelis Conflict diagnostics in directed acyclic graphs, with
applications in Bayesian evidence synthesis. To be submitted, 2012.
A. M. Presanis (MRC BSU)
Conflict in complex evidence synthesis
UCL, 18 Oct 2012
30 / 26
References
’Flu & HIV evidence syntheses
A.M. Presanis, D. De Angelis, The New York City Swine Flu Investigation Team, A. Hagy, C. Reed, S. Riley, B.S.
Cooper, L. Finelli, P. Biedrzycki, M. Lipsitch The severity of pandemic H1N1 influenza in the United States, from April
to July 2009: A Bayesian analysis. PLoS Med, 6(12):e1000207, 2009.
A.M. Presanis, R.G. Pebody, B.J. Paterson, B.D.M. Tom, P.J. Birrell, A. Charlett, M. Lipsitch, D. De Angelis Changes in
severity of pandemic (H1N1) 2009 influenza in England: a Bayesian evidence synthesis. BMJ, 343:d5408, 2011.
P.J. Birrell, G. Ketsetzis, N.J. Gay, B.S. Cooper, A.M. Presanis, R.J. Harris, A. Charlett, X.-S. Zhang, P.J. White, R.G.
Pebody, D. De Angelis Unmasking the pandemic: a Bayesian reconstruction of influenza A/H1N1pdm dynamics in
London. PNAS, doi:10.1073/pnas.1103002108, 2011.
A. Goubar, A.E. Ades, D. De Angelis, C.A. McGarrigle, C. Mercer, P.Tookey, K.Fenton, O.N. Gill Estimates of HIV
prevalence and proportion diagnosed based on Bayesian multi-parameter synthesis of surveillance data. J.R. Stat. Soc. A,
(with discussion), 171:541–580, 2008.
A.M. Presanis, D. De Angelis, D.J. Spiegelhalter, S. Seaman, A. Goubar, A.E. Ades Conflicting evidence in a Bayesian
synthesis of surveillance data to estimate HIV prevalence. J. R. Stat. Soc. A, 171:915-937, 2008.
A.M. Presanis, O.N. Gill, T.R. Chadborn, C. Hill, V. Hope, L. Logan, B.D. Rice, V. Delpech, A.E. Ades, D. De Angelis
Insights into the rise in HIV infections in England and Wales, 2001 to 2008: a Bayesian synthesis of prevalence evidence.
AIDS, 24(18):2849–2858, 2010.
A.M. Presanis, D. De Angelis, A. Goubar, O.N. Gill, A.E. Ades Bayesian evidence synthesis for a transmission dynamic
model for HIV among men who have sex with men. Biostatistics, 12(4):666–681, 2011.
S. Conti, A.M. Presanis, M.G. van Veen, M. Xiridou, M.C. Donoghoe, A. Rinder Stengaard, D. De Angelis Modeling of
the HIV infection epidemic in the Netherlands: a multi-parameter evidence synthesis approach. Ann. Appl. Statist.,
5(4):2359–2384, 2011.
A. M. Presanis (MRC BSU)
Conflict in complex evidence synthesis
UCL, 18 Oct 2012
31 / 26