Why is it so damn confusing?
Marc V. Gosselin
Disease or Outcome
Exposure
+
-
+
-
a
b
c
d
n
Basic Measures
• Incidence
• Estimate of risk
• Need a cohort study (follow-up)
• Cumulative
•
Number of new cases in a period
•
of time/total population at risk
Ex-5% over 5 years
• Rate
•
•
•
Number of new cases in a period
of time/total person time of
observation
Ex-8/127 person-years
Most precise measurement
Cases and diseased time are not
included in the denominator
• Prevalence
• Estimate of population burden
• Multiple study types
• (Number of existing disease
cases/total population) at a point in
time
• Ex-20% in 2013
Cases are included in denominator
Study Design
• Cohort
• Population is selected on
exposure
• Followed forward for
outcomes
• Can measure incidence
• RR or OR, RR preferred but
limited by type of analyses
• Multiple outcomes can be
evaluated
• Types
• 1) retrospective
• 2) prospective
• 3) RCT
• Case-Control
• Population is selected on
outcome
• Look back for exposures
• Can’t measure incidence
• Only OR not RR
• Multiple exposures can be
evaluated
• Always retrospective
(increased risk of bias)
• Useful for rare diseases
Study Design
• Cohort
• Case-Control
Study Design: RCT
• Traditional RCT
• Patients are randomized to
different treatment groups
• If a patient is non-adherent
or has side effects – usually
removed from the analysis
or to the control group for
analysis
• Determines efficacy – how
well the drug works in a
perfect environment
• Intention to Treat
• Patients are randomized to
different treatment groups
• If a patient is non-adherent
or has side effects – remain
in the treatment group for
analysis
• Determines effectiveness –
how well the drug works in
the real world
Study Design
• Confounding
• A confounder is a 3rd variable
related to the exposure and
the outcome that obscures
results
• Confounders can be adjusted
for during statistical analysis
(multivariate analysis)
• Result of complicated
relationships between
exposures and disease
• You can only adjust for
confounders if you measure
them and know they exist
=
Ø
• Bias
• Bias is a systematic error
inherent to the study design
that obscures results
• Bias can’t be adjusted for
during statistical analysis
• Introduced by the
investigators
• A bias towards the null (no
association) is more
acceptable (underestimating
a relationship) than a bias
towards an association
Bias and Confounding are present, to some extent, in all studies, even RCTs. The key issue is
how you and the authors interpret the results with bias and confounding in mind
Results
• OR - odds ratios
• RR - relative risk
• Odds of exposure in a case
• Risk an exposed patient will
•
•
•
•
(disease) compared to a
control
odds of exposure in cases/odds
of exposure in controls
(a/c)/(b/d) = ad/bc
Used to estimate RR in a casecontrol study (slightly
overestimates unless disease is
rare)
Only option for multivariate
logistic regression
become a case (develop
disease)
• Incidence in exposed/incidence
in unexposed
• [a/(a+b)]/[c/(c+d)]
• Requires a cohort study
Results
• Sensitivity/Specificity
• PPV/NPV
• Inherent to the test and the cut offs
• Completely depends on the
•
•
•
•
•
•
•
•
used to determine a positive test
Sensitivity
True positives/(true positive +
false negative) = a/(a+c)
A negative test rules out disease,
positive may not be helpful
Sensitive test has more false positives
Specificity
True negatives/(true negative +
false positive) = d/(d+b)
A positive test rules in disease,
negative may not be helpful
Specific test has more false negatives
prevalence of disease in the
population
• The prevalence should always be
given when presenting PPV or NPV
• Positive Predictive Value
• True positives/(true positive +
false positive) = a/(a+b)
• Negative Predictive Value
• True negatives/(true negative +
false negative) = d/(d+c)
N=1000
90% Sensitivity
90% Specificity
Prevalence = 1%
Prevalence = 10%
Disease or Outcome
Test
+
-
+
-
A
9
90
B
99
90
108
180
C
1
10
D
891
810
892
820
10
100
990
900
PPV = 8%, 50%
N = 1000
Guide to Quickly-ish Reading Research Papers
Look at who is writing the paper – conflicts of interest, specialty,
associations
1.
a.
b.
Abstract
Population – inclusion and exclusion criteria, disease severity
2.
3.
a.
b.
How does the study population compare to your patients?
Are the results generalizable? Were only the healthiest patients included?
Results – always read all of the results
4.
a.
If no association is demonstrated, it doesn’t mean one doesn’t exist.
Power and study design are important factors.
Conclusion/discussion
5.
a.
b.
6.
How could the authors bias the study?
they may not, but its helpful to consider before reading the paper
Do the authors’ conclusions reflect the study results? Some journals want
earth-shattering recommendations and conclusions that may be a bit of a
reach.
Do they discuss bias? All discussions should include this.
If concerns come up in the discussion, read detailed methods
Even though a paper is peer reviewed and published in a journal it does not mean it is peer reviewed by
an epidemiologist. I have had journals request things that are completely inappropriate.