Clinical Epidemiology – the basics
• What do the terms relative risk and absolute risk
mean? What are the advantages and
disadvantages of each?
• A new screening test is described as having a
sensitivity of 78%, a specificity of 89%,a positive
likelihood ratio of 13 and a negative likelihood
ratio of 0.05. Explain these terms.
A 2 minute exercise
• What do you know about this topic? Could you
explain what you know to others?
• What bits would you like clarifying? (Hint – do you
understand RR, OR, CIs etc)
EVIDENCE BASED MEDICINE
EBM is an approach to practicing
medicine in which the clinician is aware of
the evidence in support of his / her
clinical practice, and the strength of that
evidence.
EVIDENCE BASED HEALTHCARE
Evidence based health care promotes the
collection, interpretation, and integration of
valid, important and applicable patientreported, clinician-observed and research
derived evidence. The best available evidence,
moderated by patient circumstances and
preferences, is applied to improve the quality
of clinical judgements and facilitate effective
healthcare.
EFFECTIVE
SAFE
COST
PATIENT
FACTORS
QUALITY
OF CARE
0%
SYSTEMATIC
APPLICATION
OF THERAPY
100%
LEVELS OF EVIDENCE
• LARGE WELL DESIGNED RCT
• META ANALYSIS OF SMALLER RCTs
• CASE CONTROL AND COHORT STUDIES
• (CASE REPORTS AND CASE SERIES)
• CONSENSUS FROM EXPERT PANELS
• I THINK
Why don’t we always use an RCT then?
• Ethics
• Cost
• Feasibility
• Practicality
Why read journals?
• Need to make the best possible decisions for
patients
• Need to make the best possible decisions for
healthcare
• Need to feel confident about being “on top of the
job”
• Need to feel knowledgeable themselves and
credible with peers
WHY THE MOVE TO EBM?
• RANDOMISED CONTROLLED TRIALS PRE-1960
WERE ODDITIES
• REVIEWS AND META-ANALYSES AVAILABLE AS
ACCESSIBLE DIGESTS OF EVIDENCE
• ACCESS TO EVIDENCE VIA I.T.
• METHODOLOGICAL ADVANCEMENTS E.G. NUMBERS
NEEDED TO TREAT
EBM IS ABOUT ...
• CLINICAL EXPERIENCE, DIAGNOSTIC SKILLS AND
CLINICAL INSTINCT ARE A NECESSARY PART OF A
COMPENTENT PHYSICIAN.
• HOWEVER, CLINICAL PRACTICE BASED SOLELY UPON
CLINICAL EXPERIENCE “BECOMES TOMORROW’S BAD
JOKE”.
• “RATIONAL” TREATMENT BASED SOLELY UPON BASIC
PATHOLOGICAL PRINCIPLES MAY IN FACT BE
INCORRECT, LEADING TO INACCURATE TREATMENT.
• UNDERSTANDING CERTAIN RULES OF EVIDENCE IS
NECESSARY TO CORRECTLY INTERPRET LITERATURE ON
CAUSATION, PROGNOSIS, DIAGNOSTIC TESTS AND
TREATMENT STRATEGY.
20,000 biomedical journals in print. So why isn’t all practice
based on scientific evidence?
• Not RELEVANT
– Upstream to clinical decisions being made, e.g. animal or in vitro
studies
– Study populations and / or settings do not reflect question type,
practice population and settings.
• Not RELIABLE
– Poor study design
– Bias and confounding
– Measurement validity
– Insufficient power
BIAS
• Selection bias
• Observer bias
• Participant bias
• Withdrawal or drop out bias
• Recall bias
• Measurement bias
• Publication bias
CONFOUNDING
STUDY
COFFEE DRINKING
LUNG CANCER
SMOKING
CONFOUNDING VARIABLE
Power
The ability of the study to detect an effect if in
truth there is an effect.
An RCT may be underpowered if:-
• The duration is too short (too few events)
• It includes too few people (too few events)
• The wrong outcome was used (too few events)
• Expecting a higher level of statistical proof than is realistic
for the condition and the intervention being tested
EBM SKILLS - STATISTICS
•
CHANCE - p = 1 in 20 (0.05).
•
> 1 in 20 (0.051) = not significant
•
< 1 in 20 (0.049) = statistically
significant
•
CONFIDENCE INTERVALS
•
what is the range of values
between which we could be 95%
certain that this result would lie
if this intervention was applied
to the general population
TYPES OF STUDY - HYPOTHESIS FORMING
• CASE REPORTS / CASE SERIES
• CROSS SECTIONAL / PREVALENCE STUDIES
measure personal factors & disease states – a
snapshot
• CORRELATIONAL / ECOLOGICAL / GEOGRAPHIC
STUDIES. prevalence &/or incidence measurement in
one population c/w another pop.
TYPES OF STUDY - HYPOTHESIS TESTING
CASE CONTROL STUDIES
Exposure to Risk Factor
STUDY
Yes
Cases
No
Population
Yes
Controls
No
TIME
CASE CONTROL EXAMPLE -SMOKING & LUNG CANCER
DISEASE
Cases
Controls
EXPOSURE
Yes
a
b
EXPOSURE
No
c
d
Odds Ratio = ad/bc
(1 = no association, > 1 = possible association, < 1 = protective effect)
DISEASE
Cases
Controls
(lung cancer)
EXPOSURE
Yes
56
230
(smoking)
No
7
246
The odds ratio would therefore be 56 x 246 = 13776 = 8.6.
7 x 230
1610
TYPES OF STUDY - HYPOTHESIS TESTING
COHORT STUDIES
Exposed
Yes
No
Population
Sample
Time
Yes
Not exposed
No
COHORT STUDIES
OUTCOME
Yes
No
Exposed
a
b
Not exposed
c
d
Relative risk "How many times are exposed persons more likely to
develop the disease, relative to non-exposed persons?" i.e. the
incidence in the exposed divided by the incidence in the nonexposed.
This is expressed as
a
a+b
divided by
c
c+d
.
COHORT STUDY EXAMPLE
Deep vein thromboses (DVT) in oral contraceptive users.
(Hypothetical results).
OUTCOME (DVT)
YesNo
Exposed ( on oral contraceptive )
Not exposed (not on o.c.)
41
7
9998
10009
These results would give a relative risk of 6 significantly large enough numbers to indicate the
possibility of a real association between exposure
and outcome. However, NB biases.
RANDOMISED CONTROLLED TRIALS
Experimental
intervention
Improved
Not improved
Population
Sample
Time
Improved
Comparison
intervention
Not improved
RANDOMISED CONTROLLED TRIALS
OUTCOME
Yes
No
Comparison intervention
a
b
Experimental intervention
c
d
Absolute risk reduction: “What is the size of this effect in the
population”
Control event rate - experimental event rate
a/a+b
- c/c+d
Relative risk reduction: “ How many fewer patients will get the outcome
measured if they get active treatment versus comparison intervention”
a /a+b
-
c/c+d
a/a+b
ARR and RRR
A quick test
• In a study lasting 12 months, the death rate on
placebo was 10% and the death rate on
Marvelicoxib was 5%.
• What is the ARR?
• What is the RRR?
ARR and RRR in more detail
4S STUDY
• STABLE ANGINA OR MYOCARDIAL INFARCTION MORE THAN 6
MONTHS PREVIOUSLY
• SERUM CHOLESTEROL > 6.2mmol/l
• EXCLUDED PATIENTS WITH ARYHTHMIAS AND HEART
FAILURE
• ALL PATIENTS GIVEN 8 WEEKS OF DIETARY THERAPY
• IF CHOLESTEROL STILL RAISED (>5.5) RANDOMISED TO
RECEIVE SIMVASTATIN (20mg > 40mg) OR PLACEBO
• OUTCOME DEATH OR MYOCARDIAL INFARCTION (LENGTH OF
TREATMENT 5.4 YEARS ) WERE THE OUTCOMES
RCT EXAMPLE - 4S STUDY
OUTCOME (death)
Yes
Comparison intervention (placebo)
2223
256
Experimental intervention (simvastatin)
2221
182
No
1967
2039
The ARR is (256/2223) - (182/2221) = 0.115 - 0.082 = 0.033.
The RRR is 0.033/0.115 = 0.29 or expressed as a percentage 29%.
1/ARR = NUMBER NEEDED TO TREAT.
1/0.033 = 30.
i.e. if we treat 30 patients with IHD with simvastatin as per 4S study, in 5.4
years we will have prevented 1 death.
Another way of calculating NNTs
OUTCOME (death)
Yes
No
Comparison intervention (placebo)
2223
256
1967
Experimental intervention (simvastatin)
2221
182
2039
Prevalence of event in control group = 256/2223x100=11.5%
RRR = 29%
Now that’s magic!
NNT EXAMPLES
Intervention
Streptokinase + aspririn v.
placebo (ISIS 2)
tPA v. streptokinase
(GUSTO trial)
Simvastatin v. placebo in IHD
(4S study)
Treating hypertension in the
over-60s
Aspirin v. placebo in healthy
adults
Outcome
prevent 1 death
at 5 weeks
save 1 life with
tPA usage
prevent 1
event in 5y
prevent 1 event
in 5y
prevent MI or
death in 1 year
NNT
20
100
15
18
500
Why are RCTs the “gold standard”
Breast cancer mortality in studies of screening with mammography; women
aged 50 and over
(55 in Malmo study, 45 in UK)
Reduced RR
Increased RR
Random i s ed Tri al s
HIP
Two County
Malmo
Edinburgh
Stockholm
Geographi cal s tudy
UK
Cas e control s tudi es
BCDDP
Nijmegen
Utrecht
Florence
0.1
0.2
Relative risk
0.5
1.0
2.0
Egger M et al. Meta-analysis Spurious precision? Meta-analysis of
observational studies BMJ 1998;316:140-144
Odds ratios or relative risks?
Macfarlane J et al. BMJ 2002; 13: 105-9
Patients who took
antibiotics
Patients who did
not take
antibiotics
TOTAL
Patients who were
given a leaflet
49
55
104
Patients not given
a leaflet
63
38
101
TOTAL
112
93
205
Patients who
took antibiotics
Patients who did
not take
antibiotics
TOTAL
Patients who
were given a
leaflet
49
55
104
Patients not
given a leaflet
63
38
101
TOTAL
112
93
205
Relative risk: (49/104) / (63/101) = 0.76.
i.e the relative risk of patients taking an antibiotic
if they were given a leaflet is reduced by 24%. Also called
risk ratio.
Patients who
took antibiotics
Patients who did
not take
antibiotics
TOTAL
Patients who
were given a
leaflet
49
55
104
Patients not
given a leaflet
63
38
101
TOTAL
112
93
205
Odds ratio: (49/55) / (63/38) = 0.54.
There was a 46% reduction in the ratio of those taking
antibiotics who had a leaflet compared with the ratio of those
taking antibiotics who did not have a leaflet.
Patients who
took antibiotics
Patients who did
not take
antibiotics
TOTAL
Patients who
were given a
leaflet
49
55
104
Patients not
given a leaflet
63
38
101
TOTAL
112
93
205
Absolute risk reduction: (49/104) – (63/101) = 0.15.
Also known as the risk difference. i.e. the difference in the risk
of taking antibiotics depending on whether a leaflet was used
or not.
Patients who
took antibiotics
Patients who did
not take
antibiotics
TOTAL
Patients who
were given a
leaflet
49
55
104
Patients not
given a leaflet
63
38
101
TOTAL
112
93
205
NNT: 1 / 0.15 = 7. i.e. 7 people need to be given a leaflet
In order for 1 additional person not to take antibiotics
Jüni P, Rutjes AWS, Dieppe PA. Are selective COX 2 inhibitors superior
to traditional non steroidal anti-inflammatory drugs? BMJ 2002; 324:
1287-1288
Screening and Diagnostic Tests
SCREENING - WILSON & JUNGEN (WHO, 1968)
• IS THE DISORDER COMMON / IMPORTANT
• ARE THERE TREATMENTS FOR THE DISORDER
• IS THERE A KNOWN NATURAL HISTORY & “WINDOW OF
OPPORTUNITY” WHERE SCREENING CAN DETECT DISEASE
EARLY WITH IMPROVED CHANCE OF CURE
• IS THE TEST ACCEPTABLE TO PATIENTS
• SENSITIVE AND SPECIFIC
• GENERALISABLE
• CHEAP / COST EFFECTIVE
• APPLY TO GROUP AT HIGH RISK
Tests ain’t what they used to be
Joseph Heller Catch 22 1962
“Gus and Wes had succeeded in elevating medicine to an
exact science. All men reporting on sick call with
temperatures above 102 were rushed to hospital. All those
except Yossarian reporting on sick call with temperatures
below 102 had their gums and toes painted with gentian
violet solution and were given a laxative to throwaway in
the bushes. All those reporting on sick call with
temperatures of exactly 102 were asked to return in an
hour to have their temperatures taken again.”
A
No Disease
B
Disease
C
No
Disease
No
Disease
Percent
of
population
0
Disease
Disease
10
20
30
VALUE
Arbitrary Units
Set cut off at A
A lot of people who do not have the disease are
labeled as having it (false positives)
Set cut off at B
A lot of people who do have the disease are
labeled as not having it (false negatives)
DISEASE
Present
Positive
50
TEST
Negative
Absent
0
0
a
b
c
d
50
Measure the usefulness of the TEST by..
DISEASE
Present
Positive
45
TEST
Negative
5
Sensitivity
Absent
a
b
c
d
5
45
Specificity
Sensitivity
=
a =
a+c
45
50
=
90%
high sensy =
few false negatives
Specificity
=
d
=
b+d
45
50
=
90%
high specy =
few false positives
Test with a high specificity useful to rule in a diagnosis
e.g. before cancer chemotherapy
Test with high sensitivity useful to rule out a diagnosis
e.g. antenatal for syphilis
Sensitivity and specificity are properties of the test and
are taken into account when deciding whether to test.
But……
(and this is the hard bit so concentrate NOW)
When the test result is available the usefulness of the result
depends on:1. How good (or bad) the test was at detecting true positives
and true negatives
2. The pre-test probability of the person being tested actually
having the disease for which they are being tested.
What is the pre-test probability of someone
with dyspepsia being H pylori positive?
What is the pre-test probability of someone
with dyspepsia being H pylori negative?
The Impact of Prevalence on Predictive Value
(Bayes Theorem)
DISEASE
Positive
TEST
Negative
Present
Absent
45
5
5
Sens = 45/50
i.e. 90%
Prevalence = 50%
a b
c d
45
Spec = 45/50
i.e. 90%
Positive predictive
value
a = 45 = 90%
a+b
50
Negative predictive
value
d
= 45 = 90%
c+d
50
Watch what happens when the prevalence drops to 10%…….
DISEASE
Present
POSITIVE
TEST
9
NEGATIVE
Sensitivity =
1
9 = 90%
10
Absent
a
c
b
d
9
81
Specificity = 81 = 90%
90
PPV = 9 = 50%
18
NPV = 81 = 99%
82
This change can be described arithmetically by
likelihood ratios.
Likelihood ratios express how many more times (or less times) a test result is to
be found in diseased people compared with non-diseased people.
DISEASE
Present
Absent
Positive
a
b
Negative
c
d
TEST
LR +ve = a
a+c
b
b+d
LR -ve = c
a+c
d
b+d
Likelihood ratios - EXAMPLE
DISEASE
ABSENT
PRESENT
POSITIVE
9
a
b
9
NEGATIVE
1
c
d
81
TEST
LR +ve =
0.9 = 10
0.09
LR -ve =
0.1 = 0.12
0.81
New non-invasive tests for H. Pylori Gastritis.
Comparison with tissue-based gold standard.
Douglas O, et al. Digestive Diseases and Sciences 1996; 41:740-8
Sens.
Urea Breath Test
90
Spec
.
96
Serum Anti-bodies
74
89
LR +ve
LR -ve
22
0.10
7
0.30
Here comes another (different) magic nomogram!
Prevalence
UBT
20%
40%
Sab
20%
40%
+ve
PTP
85%
95%
60%
80%
PTP - Post-test probability
UBT - urea breath test
Sab - serum antibody test
-ve
PTP
2%
5%
5%
12%
Prevalence
UBT
20%
40%
Sab
20%
40%
+ve
PTP
85%
95%
60%
80%
PTP - Post-test probability
UBT - urea breath test
Sab - serum antibody test
-ve
PTP
2%
5%
5%
12%
H Pylori infection in a population with a 25% prevalence
MeReC Bulletin 2001; 12 (1): 1-4
Sensitivity
(%)
Specificity
(%)
Positive
predictive
value (%)
Negative
predictive
value (%)
False
positive
results (%)
False
Negative
results (%)
Breath test
(13C)
96.5
96
89
99
11
1
Breath test
(14C)
97.5
95.5
88
99
12
1
Laboratory
serological
tests
91
90
75
97
25
3
Near-patient
serological
tests
86
75.5
54
94
46
6
SUMMARY
EVIDENCE BASED MEDICINE
FORMULATE QUESTION
EFFICIENTLY
TRACK
DOWN BEST
AVAILABLE
EVIDENCE
CRITICALLY REVIEW THE
VALIDITY AND USEFULNESS
OF THE EVIDENCE
EVALUATE
PERFORMANCE
IMPLEMENT
CHANGES
IN CLINICAL
PRACTICE
“The evidence isn’t there” (whinge, moan) OR “I don’t have the
time” (whine, complain)
• Clinical Evidence
• Cochrane
• DTB, MeReC Bulletin
• PRODIGY
• Evidence Based Medicine
LIMITATIONS
• STILL LOTS OF ROOM FOR DEBATE ABOUT THE EVIDENCE
BASE
• EBM = WHAT IS BEST FOR AN INDIVIDUAL PATIENT (patient
utility)
• EVIDENCE BASED PURCHASING = BEST USE OF HEALTH
CARE RESOURCES FOR THE LOCAL POPULATION (cost
utility). i.e. knowledge of local needs, priorities and
constraints
• WHAT IF THESE CONFLICT? (Anybody want to mention beta
interferon and MS?!)
EBM VISION FROM 1996
Critical Appraisal Skills
Individual studies and reports
Systematic reviews
Individual
database
searches
Disease
specific
information
GUIDELINES
ELECTRONIC, PATIENT-SPECIFIC REMINDERS
DOCTOR
Consultation
CONSULTATION SKILLS
PATIENT
CONSULTATION SKILLS