Are clinicians Bayesian:
Can a Bayesian approach
help clinical decision
making?
Martin and Laura Green
Focus
• Evidence required by decision makers
–
–
–
–
–
In this example, clinicians
Applies to
Stakeholders
Policy makers
Us!
Aim
• What motivates clinicians take up new results
from research
• How data are viewed and interpreted
– Summary of a clinical trial
– Assessment of likely uptake given clinicians vary
– Results from elicitation on use of antibiotics to ‘cure’
infection
• Conclude with usefulness of Bayesian framework
for decision making as a natural approach
The example of cattle
mastitis
Mastitis in dairy cows
•
•
•
•
Bacterial infection in the udder
Udder, 4 quarters (mammary glands)
Somatic cell count increases >200K
5 - 6 common bacterial species (Staphs,
Streps, E coli) many other possible causes
– Infection enters via teat
– Risks
• other infected cows, transmission via milking
equipment
• Environment, E. coli, Streps via contaminated ground
Common cause of disease
• Current incidence rate of clinical mastitis
(IRCM)
~50-60 cases per 100 cow-years
• Unchanged for decades
• Approx 1 in 20 cows culled pa because of
mastitis
= twice number culled for bTB each year
= total cull for cattle FMD 2001 every five years
Huge economic and welfare importance
An intervention study
testing mastitis control
in dairy herds in
England and Wales
2004 - 2005
M J Green, K A Leach, J E
Breen, L E Green, A J Bradley
Aim of research study
• Assess the effectiveness of a highly
specified mastitis control plan
– 52 randomly selected dairy herds in England
and Wales
– Representative of a target population
• > 35 clinical cases per 100 cow-years
• Funded by the Milk Development Council
Mastitis diagnosis and control plan
(MDCP)
• Targeted plan
– But addressed many putative risks
• Developed using literature
– Rare that separate risk factors tested in
controlled field trials in livestock
• A clear, structured process
– could subsequently be employed by vets
Data collected on all aspects
of farm management
• Sections:
•
•
•
•
•
•
•
•
•
•
•
•
•
General farm issues
Management between milking
Pre-milking management
Milking routine
Milking machine maintenance
Post-milking management
Dry cow management
Calving cow management
Treatment strategies
Biosecurity
Young stock management
Monitoring and recording
Nutritional management
Farm specific diagnosis of
mastitis
• Characterised patterns of mastitis
• Differentiate time of infection
• Dry period vs lactation
• Rate of clinical mastitis during lactation
• SCC patterns
• Season
• Parity (including maiden heifers)
• Recurrence rates
Heterogeneity in post calving (30 day)
incidence rates of clinical mastitis
Incidence rate of clinical
mastitis per 100 cows per 30
calved
days
Proportion
of Cows with Clinical Mastitis
0.45
0.4
0.35
0.3
0.25
0.2
0.15
0.1
0.05
0
Farms
inAscending
ascending
order
of incidence
rate
Farms in
order
of Clinical
Mastitis Incidence
Survival to clinical mastitis
Heterogeneity in rates of clinical
mastitis during lactation
1
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
1
4
7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55 58 61 64
Weeks after calving
Hierarchy of ‘importance’ of
management controls
“Diagnosis”
Must
Should
Could
10-20
Action
Points
COMPLIANCE
Clinical trial design
Time –12 mths
Time 0
Time + 12 mths
Gp 2: 26 control farms
52
Outcome: Change as
proportion of initial
level
Gp 1: 26 intervention farms
Mastitis Plan
Initial analysis
Yi = α + β1 Ii + βnCi + ei
Yi = (mastitis yrt - mastitis yrt-1) / mastitis yrt-1 on farm i
I = Indicator for intervention farm
Covatiates…
– Mean cow annual milk yield
– Mean herd size
– Year 1 IRCM /SCC
Model fit
– standardised residuals plots
– assessment of homoscedasticity
– investigation of influence & leverage
Proportional change in
incidence rate of cows affected
Results: one year follow up
97.5 %ile
*
*p=0.01
Rank by proportional change
Control Farms
Intervention Farms
3
2
Proportional change in
Proportional
change
in IRCA
affected
cows
rate of
incidence
1.4
1.2
1
0.8
0.6
0.4
0.2
0
-0.2
-0.4
-0.6
3
3
3
2
3
2
2
1
1
1
3
3
2
2
2
3
Compliance group of intervention farms
1
2
1
1
1
1
1
2
Proportional change in
incidence rate of cows affected
Rank - degree of compliance
Conclusion
• Highly specified approach
– Pattern assessment
– Targeted control strategies
– Can reduce IRCM..... if follow plan
• Clinically important...
– Welfare
– Finances
• Would the plan be adopted by vet
practitioners?
Spiegelhalter et al, 2004
• Four advantages for evaluating medical
information in a Bayesian framework
– Flexible
– Particularly for more complex ‘real life’ situations
– Efficient
– Can use all available evidence
– Ethical
– Past experiences can be exploited
– Useful
– Transparency of uncertain elements
– Natural platform to provide predictions that include full
uncertainty
• Framework to take into account clinical beliefs
Investigation of veterinary beliefs on
clinical decision making
M J Green A J Bradley, W J Browne, J E Breen,
K A Leach, L E Green, G F Medley
Making a clinical decision
• Clinicians want to know what works, not disprove
what does not work
• Have to be definite in a reasonably short time
• Decision
– May be clear cut
• Individual animal, known condition
– Often very complex
• Population level (farm or national)
• Always a degree of uncertainty
• Always context dependent
• Uptake of new knowledge dependent on evidence
and individual’s characteristics
The decision process
NEWNEW
INFORMATION
DATA
(Research
or CPD)
(Research
or
CPD)
PREVIOUSPRIOR
INFORMATION
(Research or
or
(Research
Experiences)
Experiences)
POSTERIOR
CURRENT VIEW
(with
(with
some
some
uncertainty)
uncertainty)
DELIVER DECISION
(Simplified?, politicised?,
justified?, ?probabilistic)
LOCAL CONTEXT
(Animal, population, interested
parties, ethics, real
constraints etc)
How do clinicians do this??
What they might do
What time to do?
• Read research and
undertake CPD
• Use of mental strategies
(heuristics)
• Use all available
information in all subject
areas
• New and complex knowledge
cannot be simplified – therefore
not adopted
• May not keep abreast of all
current knowledge
–
Time
• Simplify the decision
process, ignore complexity
of research
–
–
Assume generalisability
Ignore consequences of variability e.g.
2% – 80%
Evidence-based
medicine
Constraints
-Fascilities
-Business
-etc
Approach to
Economics
Mastitis Control
Vet
New
Science?
Farmer
-Ability
-Personality
-Goals
-Experience
-Psychology
-Peers
-etc
-etc
A Bayesian approach
Our question…
• Having published a clinical trial on a
mastitis control scheme:– Farm specific
– Control measures weighted
– Found a mean reduction in clinical mastitis of
20%
• How might this be interpreted by
vets?
Vets vary sceptic...enthusiast
• Own context/experiences
• Context of other science
Current context - prior beliefs
• Attempt to capture actual beliefs by
formal elicitation
• Investigate theoretically using a
“community of priors” to represent a
reasonable spectrum of possible views
{Kass and Greenhouse (1989)}
Name of Prior
Vague
Description of View Represented
No view or ability to make a choice as to what the likely
parameter values could be.
Very Sceptic
A mean effect size of 0 with a 2.5% probability that the
effect size could be a reduction in mastitis more than -
0.10.
Sceptic
A mean effect size of 0 with a 2.5% probability that the
effect size could be a reduction in mastitis more than 0.20
Mid Sceptical
A mean effect size of -0.10 with a 15% probability that
Enthusiastic
the effect size could be a reduction in mastitis greater
than -0.20 or less than 0.
Enthusiastic
A mean effect size of -0.20 with a 2.5% probability that
the effect size could be a reduction in mastitis less than 0.
Very Enthusiastic
A mean effect size of -0.30 with a 2.5% probability that
the effect size could be a reduction in mastitis less than 0.20.
“Community of Priors”
- Spectrum of reasonable clinical viewpoints
Mean
0
0
0
-0.1
-0.2
-0.3
9
8
Probability Density
7
6
5
4
3
2
1
0
-0.6
-0.4
-0.2
0.0
0.2
0.4
Prior Distributions (Proportional Reduction in Clinical Mastitis)
StDev
100
0.05
0.1
0.1
0.1
0.05
Posteriors of interest
‘Expected’ reduction in clinical mastitis
using this control scheme
– Each prior updated using trial data
Predictions of expected cost saving
(room for investment) from using the
control scheme
Financial estimates
• Cost of a case of clinical mastitis was based on
literature (Mean £212, Var £30.3)
• Assumed incidence rate of clinical mastitis of
0.5 cases per cow per year (approximate
mean value for UK farms)
Results
Prior Distributions
Mean StDev
Vague 0
100
Very Sceptic
0
0.05
0.1
Sceptic 0
-0.1
0.1
Mid Sceptic-Enthusiastic
-0.2 0.1
Enthusiastic
-0.3 0.05
Very Enthusiastic
9
8
Probability Density
Density
Probability
7
6
5
4
3
2
1
0
-0.6
-0.6
-0.4
-0.4
-0.2
-0.2
0.0
0.0
0.2
0.4
0.4
Posterior
Distributions(Proportional
(ProportionalReduction
Reduction in Clinical
Prior Distributions
ClinicalMastitis)
Mastitis)
Probability
Minimum Expected
Gain gain
expected
minimum
of of
Probability
Financial gains from the control
plan, anticipated by clinicians
with different prior beliefs
Sceptic
Very Sceptic
Vague
Enthusiastic
Mid Sceptic-Enthusiast
Cautious Sceptic
Very Enthusiastic
1
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
0
5
10
15
20
25
30
35
Minimum
Expected Gain
(£s per
cow in herd
per year)
Minimum
expected
gain
(UK£s
/cow
/yr)
40
45
50
Update views with 3 further
‘identical RCT’
Theoretical prior ... posterior
Prior … posterior
Prior … posterior
Prior … posterior
Anticipated financial gains from
implementing the control plan, three
further equivalent clinical trials
Posterior after original trial with sceptic prior
Posterior after 1 simulated trial
Posterior after 2 simulated trials
Posterior after 3 simulated trials
Enthusiastic prior at original trial
Vague prior at original trial
Probability of Minimum Expected
Gain
1
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
0
5
10
15
20
25
30
35
40
Minimum Expected Gain (£s per cow in herd per year)
45
50
The relevance of prior uncertainty…
Mean Prior = 0
Mean Prior = -0.05
Mean Prior = -0.1
Mean Prior = -0.2
Mean Prior = -0.25
Mean Prior = -0.3
Mean Prior = -0.15
0
Mean of Posterior
-0.05
-0.1
-0.15
-0.2
-0.25
-0.3
-0.35
0
0.2
0.4
0.6
0.8
Standard Deviation of Prior
1
1.2
1.4
Thoughts - dairy cow mastitis
• We may fail to convince vets to modify their
approach to management of mastitis...
– unless we understand their current views
• Do we know anything about the views of
stakeholders?
– e.g. vets for the major endemic conditions; BVD,
lameness, reproduction etc
Quantitative assessment of vet
clinical beliefs to improve preventive
healthcare for dairy cattle
Helen Higgins, Green, Huxley, O’Hagan,
Oakley, Browne, Smith, Clough
An initial study:
Dry cow antibiotic therapy in
dairy cows
?
Two main aims
1) Evaluate the variation and strength of
vets beliefs
2) Given the current beliefs of the vets,
investigate the strength of evidence
needed to ‘change their minds’?
Sampling strategy
Target population – vets who “regularly” deal with dairy cattle
Study population – 100 mile radius of Nottingham
Two-stage cluster sampling, stratified by “certCHP”
24 vets, 5 practices in England
(+ 2 clinical academics)
Qualified 9 months - 26 years
7 held extra cattle qualifications
Probabilistic elicitation
 Vets interviewed individually
 Beliefs quantified numerically as probability
density functions using “probabilistic
elicitation” (SHELF)
 Capture uncertainty
Definition of variables
• 1 = overall cure rate with intra-mammary
dry cow therapy
• 2= overall cure rate with intra-mammary dry
cow therapy AND systemic antibiotics
Interested in the joint probability

3 = the additional benefit of using systemic
antibiotics, given IDCT has failed
2 = 1 + (1- 1)3

Elicited from vets the marginal distributions:
 cure rate with DCT = f(1)
 Additional cure with systemic tx if DCT fails = f(3)

Fit a distribution from an appropriate parametric
family from summary values (min, max IQR,
median) – iterate with vet
,
Practice 2
Cure rate
with
standard
therapy
AND
systemic
antibiotic
2
Cure rate with standard dry cow therapy - 1
Practice 2
Practice 3
standard
therapy
0.8
0.6
0.2
0.4
0.6
0.8
probability of cure with treament 1
Practice 4
Practice 5
Cluster 4
0.0
0.2
0.4
0.6
0.8
probability of cure with treament 1
1.0
1.0
0.8
0.6
0.4
0.2
0.0
0.2
0.4
0.6
0.8
probaility of cure with treatment 2
1.0
Cluster 5
0.0
probaility of cure with treatment 2
2
0.4
0.0
AND
systemic
antibiotic
0.2
0.0
Cure rate
with
1.0
Cluster 3
probaility of cure with treatment 2
Practice 1
0.0
0.2
0.4
0.6
0.8
1.0
Cure rate with standard dry cow therapy - 1
probability of cure with treament 1
1.0
• Given the beliefs of the 24 vets…
• …Explore what it would take to convince the
majority of vets that it is not worth giving
systemic antibiotics
• Assume – that treatment does not work (not unrealistic)
• Assume - that an additional effect on an OR scale (odds θ2:θ1)
>1.5 (e.g. from 70% cure to 77% cure) is ‘clinically worthwhile’
• Assume - the vets need to be very (95% credible interval OR <
1.5) certain to change their action (n.b. reduces their income)
•
vet’s prior belief
likelihood of theta based on new data
the updated belief in light of the new
evidence
New data
 Synthetic, assumed :
From a single RCT trial (of different sizes) that showed no
additional benefit with systemic antibiotics.
 Synthetic data for the likelihood were generated
for different sized trials; 30, 60, 250, 500 cows
in each treatment arm.
 θ1 and θ2 were set equal so that θ3 =0; i.e. no
difference between the two treatments.
 Posterior of interest was OR θ2:θ1
0 1 2 3 4 5 6 7 8 9 10 11
Odds ratio
Vets’ elicited beliefs expressed as
95% credible intervals on an odds ratio scale
Systemic ab’s
offer a benefit
over IDCT
alone
Systemic
ab’s offer
no extra
benefit
Practice 1 Practice 2
Practice 3
Practice 4
Practice 5
Predicted belief (red) after evidence from
a single RCT involving 30 infected cows
per group
Predicted belief (red) after evidence from
a single RCT involving 250 infected cows
per group
Predicted belief (red) after evidence from
a single RCT involving 500 infected cows
per group
10 11
9
8
7
6
5
4
3
2
1
0
Odds ratio for theta 2 versus theta 1
Odds ratio
Example from a meta-analysis:
Predicted belief (red) after the evidence of 5 small
RCT ( 2 groups of 100 cows per trial)
Cluster1
Cluster2
Practice
1 Practice
2
Cluster3
Practice
3
Cluster4
Practice
4
Cluster5
Practice
5
Therefore…
 Major variations in vets beliefs
 Reasons for variation?
 To what extent does variability occur with clinical
beliefs in general?
 Many of these practitioners are “enthusiastic”
and/or “certain” about the efficacy of systemic
antibiotics
Final thoughts
If the point of research is generally to effect
change (e.g. clinical decisions)
• How often do we have a clear understanding of
the stakeholders involved?
• Current views (degree of scepticism and certainty)?
• When will they change their minds?
• Should context of stakeholders influence study
design and sample size?
Final thoughts
• Clinicians naturally Bayesian?
– Weighting of information + knowledge of uncertainty
• Bayesian analyses may be useful for clinicians if
– Transparent
– Summarise posteriors are on scales that are easily
understood
• A final ‘but’
– Link ‘what we think’ with ‘what we do’
Acknowledgements
Authors , collaborators and colleagues
Funders:
Vets and farmers
InFER