Epidemiology Kept Simple
Chapter 12
Error in Epidemiologic
Research
7/28/2016
1
§12.1 Introduction
• View analytic studies as
exercises in measuring
association between
exposure & disease
• All such measurements are
prone to error
• We must understand the
nature of measurement
error if we are to avoid train
wrecks*
* a disaster or failure, especially one that is unstoppable or unavoidable
7/28/2016
2
Parameters and Estimates
Understanding measurement error begins by
distinguishing between parameters and estimates
Parameters
– Error-free
– Hypothetical
– Measure effect
Statistical estimates
– Error-prone
– Calculated
– Measure
association
Epidemiologic statistics are mere estimates of the
parameters they wish to estimate.
7/28/2016
3
Target Analogy
Let Φ represent the true relative risk for an
exposure and disease in a population
Replicate the study many times
Let ˆi represent the RR estimate from study i
How closely does
an estimate from
a given study
reflect the true
relative risk
(parameter)?
7/28/2016
4
Target Analogy
7/28/2016
5
Random Error
(Imprecision)
• Balanced “scattering”
• To address random
error, use
– confidence
intervals
– hypothesis tests of
statistical
significance
7/28/2016
6
Confidence Intervals (CI)
• Surround estimate with margin of error
• Locates parameter with given level of
confidence (e.g., 95% confidence)
• CI width quantifies precision of the estimate
(narrow CI  precise estimate)
7/28/2016
7
Mammagraphic Screening and
Breast Cancer Mortality
• Exposure ≡
mammographic
screening
• Disease ≡ Breast
cancer mortality
• First series
(studies 1 – 10):
women in their 40s
• Second series
(studies 11 – 15):
women in their 50s
7/28/2016
8
Hypothesis Tests
• Start with this null hypothesis
H0: RR = 1 (no association)
• Use data to calculate a test
statistic
• Use the test statistic to derive a Pvalue
• Definition: P-value ≡ probability of
the data, or data more extreme,
assuming H0 is true
• Interpret P-value as a measure of
evidence against H0 (small P gives
one reason to doubt H0)
R.A. Fisher
7/28/2016
9
Conventions for Interpretation
• Small P-value provide strong
evidence against H0 (with respect
to random error explanations only!)
• By convention
– P ≤ .10 is considered marginally
significant evidence against H0
– P ≤ .01 is considered highly
significant evidence against H0
• Do NOT use arbitrary cutoffs
(“surely, god loves P = .06”)
R.A. Fisher
7/28/2016
10
Childhood SES and Stroke
Factor
RR
P-value
Crowding
< 1.5 = 0.4
1.5 – 2.49 = 1.0 (ref.)
2.5 – 3.49 = 0.6
 3.5 = 1.0
trend P =
0.53
Tap water
0.73
P = 0.53
Toilet type
flush/not shared = 1.3
flush/shared = 1.0 (referent)
no flush = 1.0
trend P =
0.67
Ventilation
good = 1.0 (ref.)
fair = 1.7
poor = 1.7
trend P =
0.08
Cleanliness
good= 1.1
fair = 1.0 (ref.)
poor = 0.5
trend P =
0.07
(persons/room)
7/28/2016
Source: Galobardes et al., Epidemiologic Reviews, 2004, p. 14
Nonsignif.
evidence
against H0
Marginally
significant
evidence
against H0
11
Systematic Error (Bias)
• Bias = systematic error
in inference (not an
imputation of prejudice)
• Amount of bias
• Direction of bias
–Toward the null
(under-estimates risks or
benefits)
–Away from null (overestimates risks or benefits)
7/28/2016
12
Categories of Bias
• Selection bias: participants
selected in a such a way as
to favor a certain outcome
• Information bias:
misinformation favoring a
particular outcome
• Confounding: extraneous
factors unbalanced in
groups, favoring a certain
outcome
7/28/2016
13
Examples of Selection Bias
•
•
•
•
•
•
Berkson’s bias
Prevalence-incidence bias
Publicity bias
Healthy worker effect
Convenience sample bias
Read about it: pp. 229 – 231
7/28/2016
14
Examples of Information Bias
• Recall bias
• Diagnostic
suspicion bias
• Obsequiousness
bias
• Clever Hans effect
• Read about it on
page 231
7/28/2016
16
Memory Bias
• Memory is constructed rather
than played back
• Example of a memory bias is
The Misinformation Effect
≡
a memory bias that occurs when
misinformation affects people's
reports of their own memory
• False presuppositions, such as
"Did the car stop at the stop
sign?" when in fact it was a yield
sign, will cause people to
misreport actually happended
Loftus, E. F. & Palmer, J. C. (1974). Reconstruction of automobile
destruction. Journal of Verbal Learning and Verbal Behaviour, 13, 585-589.
7/28/2016
17
Recall Bias in CaseControl Studies
7/28/2016
18
Differential & Nondifferential
Misclassification
• Non-differential
misclassification:
groups equally
misclassified
• Differential
misclassification:
groups misclassified
unequally
7/28/2016
19
Nondifferential and Differential
Misclassification Illustrations
7/28/2016
20
Confounding
• A distortion brought about by extraneous variables
• From the Latin meaning “to mix together”
• The effects of the exposure gets mixed with the effects of
extraneous determinants
7/28/2016
22
Properties of a Confounding
Variable
• Associated with the
exposure
• An independent risk
factor
• Not in causal pathway
7/28/2016
23
Example of Confounding
E ≡ Evacuation
Died Survived Total
method
Helicopter 64
136
200
Road 260
840
1100
D ≡ Death
following auto
• R1 = 64 / 200 = .3200
• R0 = 260 / 1100 = .2364
accident
• RR = .3200 / .2364 = 1.35
• Does helicopter evacuation really increase the
risk of death by 35%? Surely you’re joking!
• Or is the RR of 1.35 confounded?
7/28/2016
25
Serious Accidents Only
(Stratum 1)
•
•
•
•
Died
Survived
Total
Helicopter
48
52
100
Road
60
40
100
R1 = 48 / 100 = .4800
R0 = 60 / 100 = .6000
RR1 = .48 / .60 = 0.80
Helicopter evacuation cut down on risk of death by 20%
among serious accidents
7/28/2016
26
Minor Accidents Only (Stratum 2)
Helicopter
Road
•
•
•
•
Died
Survived
Total
16
84
100
200
800
1000
R1 = 16 / 100 = .16
R0 = 200 / 1000 = .20
RR2 =.16 / .20 = 0.80
Helicopter evacuation cut down on risk of death by 20%
among minor accidents
7/28/2016
27
Accident Evacuation
Properties of Confounding
• Seriousness of accident (Confounder)
associated with method of evacuation
(Exposure)
• Seriousness of accident (Confounder) is an
independent risk factor for death (Disease)
• Seriousness of accident (Confounder) not in
causal pathway between helicopter evaluation
(Exposure) and death (Disease)
7/28/2016
28