Sensitivity Analysis
and Meta-analysis
EPI 811 Individual Presentation
Chapter 10 of Szklo and Nieto’s Epidemiology: Beyond the Basics
Anton Frattaroli
Sensitivity Analysis
• Generally, an assessment of how systematic or random errors
affect an effect estimates’ representativeness of the actual
effect (the validity of the effect estimate).
• Misclassification error is a primary inhibitor of validity and can
be difficult to correct for.
• Executed by adjusting model parameters over a reasonable
range, and observing the results.
Sensitivity Analysis Applied to
Misclassification: Example
• You: “The relative risk (RR) of coronary heart disease (CHD) for
second-hand (passive) smoke exposure in non-smokers is
between 1.15 and 1.3.”
• They: “Maybe some of your ‘non-smokers’ are actually
smokers, and your RR is too high.”
• You: “I can do a sensitivity analysis. Assume 5% of my exposed
‘non-smoker’ cases are just misclassified smokers. In that
case, the effect of CHD on active smokers would have to
exhibit a RR of 7.0 in order to entirely account for the
difference in RR. But, since the RR of CHD in smokers is 2.0,
you’re wrong.”
• They: “Try 10%.”
STATA: episensi
Sensitivity Analysis Applied to
Vaccine Effectiveness: Example
Meta-analysis
• A “quantitative approach for systematically assessing the
results of previous research in order to arrive at conclusions
about the body of research (Petitti)”.
• Unit of analysis is the study, rather than a group or individual.
Study selection is similar to the selection of subjects in a
study.
• In a meta-analysis study of the relationship of major
depression to socioeconomic class, only 51 studies were
chosen out of a 743 found.
Meta-analysis in Action!
STATA: metan
You can do it too!
Meta-analysis Styles
• Some prefer the Mantel-Haenszel method of weighting study
estimate results by the power of the study (the “fixed-effects
model”).
• Others like a “random-effects model”, which takes into account both
within-study variance and cross-study variance.
• The random-effects model is more conservative. The real difference
is in the generalizability: fixed-effects is limited to the included
studies, while random-effects can be applied to a hypothetical
“population of studies.”
• Neither model is advisable when the direction of the studies is not
consistent.
• Still some argument about the effectiveness of meta-analysis, given
the differences in participant selection and data collection methods
across studies.
• EPI 810 tip: Prospective meta-analysis can be used as an agreement
across research groups to avoid some of these pitfalls.
Questions?
• Where do I get that totally sweet Stata module?
• episens: http://ideas.repec.org/c/boc/bocode/s456792.html
• metan: http://ideas.repec.org/c/boc/bocode/s456798.html