In the 19th century health was transformed by clear, clean water.
In the 21st century health will be transformed by clean, clear knowledge.
Sir Muir Gray
70-80% of Doctors Do Not Understand
Health Statistics: What Can Be Done?
Gerd Gigerenzer
Max Planck Institute for
Human Development Berlin
Improving Health Literacy
Problem:
Most doctors and patients do not understand health statistics.
Causes:
- Failure of medical schools to teach statistical thinking.
- Biased reporting.
Solution:
1. Teach risk literacy in medical school.
2. Implement transparent risk communication.
Gigerenzer & Muir Gray (Eds.) 2011. Better doctors, better patients, better decisions. MIT Press.
I.
Most doctors do not understand
health statistics
Gigerenzer & Muir Gray (Eds.) 2011. Better doctors, better patients, better decisions. MIT Press.
"I had prostate cancer, five, six years ago. My chances of surviving
prostate cancer and thank God I was cured of it, in the United States,
82 percent. My chances of surviving prostate cancer in England, only 44
percent under socialized medicine.”
Rudy Giuliani, New Hampshire radio advertisement, October 2007
Lead Time Bias
Gigerenzer, Gaissmaier, Kurz-Milcke, Schwartz, & Woloshin 2007. Psychological Science in the Public Interest.
Overdiagnosis
Gigerenzer, Gaissmaier, Kurz-Milcke, Schwartz, & Woloshin 2007. Psychological Science in the Public Interest.
Do German Physicians Understand
5-Year Survival Rates?
65 physicians (internal medicine)
When the (same) information was framed as
Survival rates: 79% judged screening as effective
Mortality rates: 5% judged screening as effective
Lead-time-bias?
Overdiagnosis?
2 out of 65 knew
0 out of 65 knew
Wegwarth, Gaissmaier & Gigerenzer, Medical Decision Making, 2011
Do U.S. Physicians Understand
5-Year Survival Rates?
412 primary-care physicians (national sample)
Survival rates:
Mortality rates:
83% judged mortality benefit as large
28% judged mortality benefit as large
Which proves that a cancer screening test “saves lives”?
1. Screen-detected cancers have better 5-year survival. 76%
2. More cancers are detected in screened populations. 47%
3. Mortality rates are lower among screened persons.
81%
Wegwarth, Schwartz, Woloshin, Gaissmaier & Gigerenzer, Annals of Internal Medicine, 2012.
Deception by Medical Institutions
One of the most prestigious cancer centers in the US: M. D. Anderson
Confusion about progress against cancer.
Unwarranted enthusiasm for medical center.
Deception by Pink Ribbon Organizations:
Susan G. Komen
Early detection saves lives. The 5-year survival rate for
breast cancer when caught early in 98%. When it’s not?
23%.
Deception by Pink Ribbon Organizations:
Susan G. Komen
Early detection saves lives. The 5-year survival rate for
breast cancer when caught early in 98%. When it’s not?
23%.
II.
Helping Doctors and Patients
Make Sense of Health Statistics:
Fact Boxes
Schwartz, Woloshin & Welch 2009. Annals of Internal Medicine
Gigerenzer, Gaissmaier, Kurz-Milcke, Schwartz, & Woloshin 2007.
Psychological Science in the Public Interest.
Fact Box
Breast cancer screening with mammography: per 1,000 women 50+
No Screening
Screening
over 10 years
Benefits?
Breast cancer mortality
5
4
Total cancer mortality
21
21
False positives with biopsies
–
50 – 200
Unnecessary treatments
(e.g. lumpectomy)
–
2 - 10
Harms?
Gøtzsche PC & Jørgensen KJ 2013. Cochrane Database of Systematic Reviews
Woloshin S & Schwarz LM 2009. Journal of the National Cancer Institute 101(17)
www.harding-center.mpg.de
THE ART OF RISK COMMUNICATION
HOW BREAST CANCER PAMPHLETS MISLEAD WOMEN
“Breast screening has been shown to reduce the risk of
dying from breast cancer by around 35%.”
The Welsh NHS leaflet Breast
Screening
“Screening mammograms . . . reduce the chance of dying from
breast cancer by approximately 33%.”
New Zealand Breast Cancer
Foundation
“Screening saves about 1 life from breast cancer for every 200
women who are screened.”
Gigerenzer 2014 British Medical Journal
NHS
Leaflet for England
What Does a 25% Reduction of Breast Cancer
Mortality Mean?
15 Gynecologists, Department of Gynecology Luzern
Schüssler 2005, Frauenheilkunde aktuell
Prostate Cancer Early Detection
by PSA screening and digital-rectal examination.
Numbers are for men aged 50 years or older, not participating vs. participating in screening for 10 years.
1,000 men without screening:
1,000 men with screening:
P PP P P PP P
P PP P P PP P
XXXXXXXXXXXXXXXXXXXX
P
X
Men dying from prostate cancer:
8
8
Men dying from any cause:
200
200
Men that were diagnosed and treated
for prostate cancer unnecessarily:
–
20
Men without cancer that got a false
alarm and a biopsy:
–
180
Men that are unharmed and alive:
800
600
Source:
Djulbegovic, Beyth, Neuberger et al.
2010. British Medical Journal.
“I never dreamed that my discovery four decades ago would lead to
such a profit-driven public health disaster. The medical community
must confront reality and stop the inappropriate use of PSA screening.
Doing so would save billions of dollars and rescue millions of men from
unnecessary, debilitating treatments.”
Richard Ablin, 2010
discoverer of PSA (prostate specific antigen)
Harmful Overtreatment
In 2014, German gynecologists
recommended ultrasound
screening to 3 million women.
About 2 million ultrasounds were
performed.
Women paid 75 million Euro from
their own pockets.
About 35,000 healthy women lost
their ovarian.
Health insurances paid > 100
million Euros for surgery and
treatment of consequences.
www.aok.de/faktenboxen
III.
Helping Doctors and Patients
Understand Screening Tests:
Natural Frequencies
The Illusion of Certainty
What Does a Positive Test Mean?
A positive result means antibodies to HIV were
found in your blood. This means you have HIV
infection. You are infected for life and can
spread HIV to others.
You will most likely develop AIDS, but no one can
know when you will get sick. Within 10 years after
infection, about half of untreated people have
developed AIDS.
Illinois Department of Public Health
A letter from California
Conditional Probabilities
Natural Frequencies
10,000
clients
9,999
no HIV
1
HIV
1
positive
p(HIV) = 0.01%
p(positive|HIV) = 99.9%
p(negative|no HIV) = 99.99%
0
negative
1
positive
9,998
negative
Conditional Probabilities
Natural Frequencies
10,000
clients
9,999
no HIV
1
HIV
1
positive
p(HIV) = 0.01%
p(positive|HIV) = 99.9%
p(negative|no HIV) = 99.99%
0
negative
1
positive
9,998
negative
p(HIV|positive)
=
.0001x.999
.0001x.999 + .9999x.0001
Conditional Probabilities
Natural Frequencies
10,000
clients
9,999
no HIV
1
HIV
1
positive
p(HIV) = 0.01%
p(positive|HIV) = 99.9%
p(negative|no HIV) = 99.99%
0
negative
1
positive
9,998
negative
p(HIV|positive)
p(HIV|positive)
=
1
1+1
=
.0001x.999
.0001x.999 + .9999x.0001
Twenty AIDS Counseling Centers
Client:
"If one is not infected with HIV, is it possible to have a positive test result? “
Counselor:
1
2
3
4
5
6
7
8
9
10
"No, certainly not“
"Absolutely impossible“
"With absolute certainty, no“
"No, absolutely not“
"Never“
"Absolutely impossible“
"Absolutely impossible“
"With absolute certainty, no“
"The test is absolutely certain“
"No, only in France, not here"
11
12
13
14
15
16
17
18
19
20
"False positives never happen“
"With absolute certainty, no"
"With absolute certainty, no“
"Definitely not" … "extremely rare“
"Absolutely not" … "99.7% specificity“
"Absolutely not" … "99.9% specificity“
“More than 99% specificity“
“More than 99.9% specificity“
"99.9% specificity“
“No comment"
Gigerenzer, Hoffrage & Ebert (1998), AIDS CARE.
160 Gynecologists Estimate the Probability of Breast Cancer
Given a Positive Screening Mammogram
90
Estimates in %
81
10
1
Before training
Gigerenzer, Gaissmaier, Kurz-Milcke, Schwartz, & Woloshin 2007. Psychological Science in the Public Interest.
Teaching Gynecologists to Use Natural Frequencies Fosters Insight
90
Estimates in %
81
10
1
Before training
After training
Gigerenzer, Gaissmaier, Kurz-Milcke, Schwartz, & Woloshin 2007. Psychological Science in the Public Interest.
Natural Frequencies
Conditional Probabilities
1000
women
1
woman
10
cancer
9
positive
990
no cancer
1
negative
89
positive
901
negative
p(cancer|positive)
=
9
9 + 89
.01
cancer
.90
positive
.99
no cancer
.10
negative
.09
positive
p(cancer|positive)
=
.01 x .90
.01 x .90 + .99 x .09
Gigerenzer & Hoffrage Psychological Review 1995, 1999
.91
negative
Fourth-Graders Can Make Bayesian Inferences
Using Natural Frequencies
•
•
•
Out of every 20 pupils in the Ravenclaw School of Magic, 5 have a wand.
Of these 5 pupils, 4 also wear a magical hat.
Of the other 15 without wands, 12 also wear a magical hat.
Imagine a group of pupils who wear a magical hat. How many of those also have a wand?
Gigerenzer 2014. Risk Savvy. Viking
Impact on Medical Practice
Health Information Policy:
•Misleading statistics have been removed from (almost) all cancer screening
pamphlets in Germany.
• The first fact boxes published in Austria by the Tyrolean Society for General
Medicine.
Medical Education:
• Members of the Harding Center have trained 3,000 physicians in CME.
• Medical schools have begun teaching methods of risk communication (e.g.
Charité, Giessen, Mainz).
Communication Tools:
•Using “natural frequencies” (Gigerenzer & Hoffrage, Psychological Review 1995,
1999) has become a standard method in evidence-based medicine, recommended
by the Cochrane Collaboration, the International Patient Decision Aid Standards
Collaboration, and the Medicine and Healthcare Products Regulatory Agency.
Problem:
Most doctors and patients do not understand
health statistics.
Causes: Failure of medical schools.
- Biased reporting.
- Lack of education of the public.
Solutions:
1. Teach risk literacy in medical school.
2. Implement transparent risk
communication.
3. Teach health literacy in school,
starting early.
MIT Press
Problem:
Most doctors and patients do not understand
health statistics.
Causes: Failure of medical schools.
- Biased reporting.
- Lack of education of the public.
Solutions:
1. Teach risk literacy in medical school.
2. Implement transparent risk
communication.
3. Teach health literacy in school,
starting early.
RISK VS UNCERTAINTY
RISK:
How should we make decisions when all relevant alternatives,
consequences, and probabilities are known?
 STATISTICS
UNCERTAINTY:
How should we make decisions when NOT all alternatives,
consequences, and probabilities are known?
 HEURISTICS
Gigerenzer, Todd & ABC Research Group 1999. Simple Heuristics That Make Us Smart. OUP
The heart disease predictive instrument (HDPI)
Chest Pain = Chief Complaint
EKG (ST, T wave ∆'s)
History
No MI& No NTG
MI or NTG
MI and NTG
ST&T Ø
19%
27%
37%
ST
35%
46%
58%
T
42%
53%
65%
ST
54%
64%
75%
ST&T
62%
73%
80%
ST&T
78%
85%
90%
ST
21%
29%
40%
T
26%
36%
47%
ST
36%
48%
59%
ST&T
45%
56%
67%
ST&T
64%
74%
82%
ST
9%
14%
20%
T
12%
17%
25%
ST
17%
25%
35%
ST&T
23%
32%
43%
ST&T
39%
51%
62%
Chest Pain, NOT Chief Complaint
EKG (ST, T wave ∆'s)
History
No MI& No NTG
MI or NTG
MI and NTG
ST&T Ø
10%
16%
22%
No Chest Pain
EKG (ST, T wave ∆'s)
History
No MI& No NTG
MI or NTG
MI and NTG
ST&T Ø
4%
6%
10%
See reverse for definitions and instructions
Fast-and-Frugal Tree for Coronary Care Unit Allocation
ST segment changes?
no
yes
Chief complaint of
chest pain?
no
yes
Regular
nursing
bed
Any one other factor?
(NTG, MI,ST,ST,T)
no
Regular
nursing
bed
Green & Mehr (1997)
Coronary
Care
Unit
yes
Coronary
Care
Unit
Emergency Room Decisions: Admit to the Coronary Care Unit?
1
Sensitivity
Proportion correctly assigned
.9
.8
.7
.6
Physicians
.5
Heart Disease
Predictive Instrument
.4
Fast and Frugal Tree
.3
.2
.1
.0
.0
.1
.2
.3
.4
.5
.6
.7
.8
.9
False positive rate
Proportion of patients incorrectly assigned
1
Fast-and-Frugal Tree for Assessing Bank Vulnerability
Leverage ratio < 4.1%?
Yes
No
Market-based capital ratio < 16.8%
No
Yes
Loan to deposit ratio > 1.4?
No
GRE
EN
FLA
G
RED
FLAG
Yes
RED
FLAG
Leverage ratio =
RED
FLAG
Market-based capital ratio =
Tier 1 capital
Total assets
Market capitalization
Risk-weighted assets
Loan to deposit ratio =
* 100
* 100
Retail loans
Retail deposits
Aikman, Galesic, Gigerenzer, Kapadia, Katsikopoulos, Kothiyal, Murphy & Neumann 2014, Bank of England Financial Stability Paper, No. 28
A Signal-Detection-Analysis of Fast-and-Frugal Trees
Luan, Schooler & Gigerenzer
Psychological Review 2011
A Signal-Detection-Analysis of Fast-and-Frugal Trees

Bank failure

CCU allocation

Checkpoint

HIV
Luan, Schooler & Gigerenzer
Psychological Review 2011
Problem:
Most doctors and patients do not understand
health statistics.
Causes: Not irrationality, but
- Failure of medical schools.
- Biased reporting.
- Lack of education of the public.
Solutions:
1. Teach risk literacy in medical school.
2. Implement transparent risk
communication.
3. Teach health literacy in school,
starting early.
MIT Press
Problem:
Most doctors and patients do not understand
health statistics.
Causes: Not irrationality, but
- Failure of medical schools.
- Biased reporting.
- Lack of education of the public.
Solutions:
1. Teach risk literacy in medical school.
2. Implement transparent risk
communication.
3. Teach health literacy in school,
starting early.
Doctors and Patients Should Have the Right
to Clear Information
- Ethics committees should recognize that misleading the
public is a key moral issue.
- Fact Boxes should be made available
.
- Free access to the Cochrane Library
www.thecochranelibrary.com
- Honest health pamphlets.
Gigerenzer & Muir Gray (eds.) 2011. Better doctors, better patients, better decisions. MIT Press.
If you open a book on rat DM, people lack rationality, commit long lists of CI,
which are compared to stable Visual illusions. OC, base rate neglect …
All of this points to the conclusion that Homo Sapiens is a misnomer. Are
people stupid? Economist: “Human beings are fallible; lazy, stupid, greedy and
weak.”
Influential scholars argued that we are hopeless when dealing with risk: “our
species is uniformly probability-blind” …
Collect the experts, close the doors, and tell people what to do. Or find out
how to ‘nudge’ them into proper behavior.
This fatalistic message is not what you will hear this evening. The problem is
not individual stupidity but the phenomenon of a risk-illiterate society.
-Everyone can learn to deal with risk and uncertainty.
-Experts are part of the problem rather than the solution.
Gazzaniga, President’s Coucil on Bioethics (endless discussions)
NCI: President appoints the members of the NCI advisory committee directly.
Estimates in %
Colorectal Cancer
correct estimate
conditional
probabilities
Gigerenzer (2002), Reckoning with risk. Penguin. (US version: Calculated risks. Simon & Schuster.)
Estimates in %
Colorectal Cancer
correct estimate
conditional
probabilities
natural
frequencies
Gigerenzer (2002), Reckoning with risk. Penguin. (US version: Calculated risks. Simon & Schuster.)