Faith, Trust and Pixie Dust
professional judgement in healthcare, is it “broken”
and can we fix it?
Professor Carl Thompson
Department of Health Sciences, University of York
Trust in professionals: UK.
Ford J BMJ 2007;335:465
©2007 by British Medical Journal Publishing Group
Trust: USA
% “high or very high”
90
80
70
60
50
40
doctors
nurses
30
20
10
0
http://www.gallup.com/poll/1654/honesty-ethics-professions.aspx#4
Judgement and Variation
65% had
tonsillectomy
46% for
tonsillectomy
1000 11yr olds
35% “to be
seen”
45% for
tonsillectomy
54% “to be
seen”
American child health association 1934
Only 65 kids left;
stop study ran
out of doctors
Fig 1
Accessing stroke rehabilitation
Source: Archives of Physical Medicine and Rehabilitation 2010; 91:788-793 (DOI:10.1016/j.apmr.2009.11.028 )
“machines that go ping”
Cognitive continuum (cf. Hammond
1996)
Cameron HM, McGoogan E. J Pathol. 1981 Apr;133(4):273-83
Some problems
• Heuristics or biases that result (Kahneman,
Tversky et al.) – e.g. availability,
overconfidence, anchoring,
representativeness.
• The alternative: fast and frugal (Gerd
Gigerenzer)
• Do our common ”solutions” make it worse?
– Groupthink
Nutrition (ANS)
•
•
•
•
•
•
•
•
27 doctors nurses and nutritionists
Large Teaching Foundation Trust
54 (49) “cases” judged
NICE guidance as a “rule”
74% rule aware, 70% using, 63% confident
1323 judgements
Feed (clinicians) = 693 (Guidelines) = 270
Accuracy/sensitivity .45 (below chance)
Journal of Human Nutrition and Dietetics
Volume 25, Issue 5, pages 427-434, 30 MAY 2012
Initiating artificial nutrition support: a clinical judgement analysis
Journal of Human Nutrition and Dietetics
Volume 25, Issue 5, pages 427-434, 30 MAY 2012 DOI: 10.1111/j.1365-277X.2012.01260.x
http://onlinelibrary.wiley.com/doi/10.1111/j.1365-277X.2012.01260.x/full#f2
Calibration curves of easy and difficult judgments under no time pressure/time pressure. Yang et
al. BMC Medical Informatics and Decision making 2012 12:113 doi:10.1186/1472-6947-12-113
Paper vs. nurses (250 from 4 countries)
Mews >4
.69
Sensitivity
P (judge+ | risk+)
>11 .65
>4 .57
1 – specificity (False positive rate)
Can we fix it?
• “Support”
– Ask a friend or friends
– Evidence based decision making/training?
– More machines that go ping (CDSS)?
• Education and Training (cognitive forcing,
feedback)
– Something about professional attitude to “rules”?
Clinical vs. actuarial “judgements”
• Neurosis vs. psychosis: rule = 70%;
clinicians 50% to 67%
• Dx of progressive brain dysfunction
based on intellectual testing: rule =
83%; mixed bag of clinicians 63%
(inexperienced) to 58%
(experienced). Knowledge of rule
results = 68% and 75%...
• Predicting survival time: experts (no
correlation); rule = modest but
significant
• Repeat…ad nauseum
RM Dawes, D Faust, PE Meehl: Science 31 March 1989:
Resistance?
• Don’t know about the evidence?
• Don’t know there is a problem?
• Don’t recognise it (“mental healthcare discourse
anyone?”)
• Russian roulette and individualisation of
probabilities
• Don’t think its as good as ME and my recall –
average drivers
• “You can’t handle the truth” (certainty pays the
bills)
Its not “broken” but needs fettling*
• If you know something about judgements and decisions
(structure, context) then you can predict and improve
them… it’s not r****t **i**ce!
• CDSS, education and training for decision making… need
cultural support
• Guidance (MRC, 2008) for developing “good”
interventions… we need to use them
www.mrc.ac.uk/complexinterventionsguidance
• Where good tools exist then implementation (science)
required
* A northern verb: to arrange, tidy up, put in order