Evidence Based Medicine
Examples
Edward G. Hamaty Jr., D.O. FACCP, FACOI
Systematic Reviews
Systematic Reviews
• Citation
• Perel P, Roberts I. Colloids versus crystalloids
for fluid resuscitation in critically ill patients.
Cochrane Database of Systematic Reviews
2007, Issue 4. Art. No.: CD000567. DOI:
10.1002/14651858.CD000567.pub3.
Systematic Reviews
• Although you have quickly found a recent
review that proves that there is no advantage
to colloid over crystalloids, your interest is
piqued and you pull one of the review articles
for further analysis.
Cochran Q (Chi-square) and I2
• Heterogeneity can be assessed using the “eyeball” test or
more formally with statistical tests such as the Cochran Q
test.
• If the Cochran chi-square (Cochran Q) is statistically
significant there is definite heterogeneity. [P<0.1](The level
of significance for Cochrane Q is often set at 0.1 due to the
low power of the test to detect heterogeneity.
• If the Cochran Q is not statistically significant, but the ratio
of Cochran Q and the degrees of freedom (Q/df) is >1 there
is possible heterogeneity.
• If the Cochran Q is not statistically significant and Q/df is <1
then heterogeneity is very unlikely.
• In the next example, Q/df is <1 (0.92/4 = 0.23) and the p
value is not significant (0.92) indicating no heterogeneity.
Cochran Q (Chi-square) and I2
• The Cochran Q test has low power to detect
heterogeneity if there are few studies in the metaanalysis and may, conversely, give a highly significant
result if it comprises many large studies, even when
the heterogeneity is unlikely to affect the conclusions.
• An Index, I2, which does not depend upon the number
of studies, the type of outcome data or the choice of
treatment effect (e.g. relative risk) can be used to
quantify the impact of heterogeneity and assess
inconsistency.
• I2 = 100x(Q – df)/Q
Cochran Q (Chi-square) and I2
• I2 represents the percentage of the total variation
across studies due to heterogeneity; it takes
values from 0% to 100%, with the value of 0%
indicating no observed heterogeneity. If there is
evidence of statistical heterogeneity, we should
proceed cautiously, investigate the reasons for its
presence and modify our approach accordingly,
perhaps by dividing the studies into subgroups of
those with similar characteristics.
Systematic Reviews- Analysis
Systematic Reviews- Analysis
http://www.cebm.utoronto.ca/practise/ca/statscal/
Systematic Reviews- Analysis
Systematic Reviews- Analysis
0.724
Using the tables will give you an idea of
the magnitude of the effect. Calculations
are more accurate.
Pooled difference in the risk of death with albumin was 6% (95% confidence
interval 3% to 9%) with a fixed effects model. These data suggest that for every 17
critically ill patients treated with albumin there is one additional death.
ARR = 0.06, NNT = 1/0.06 = 16.66 = 17
Systematic Reviews- Analysis
Systematic Reviews- Analysis
Systematic Reviews- Analysis
Diagnosis - Analysis
Diagnosis - Analysis
Diagnosis - Analysis
Are the Results Important?
Sensitivity and specificity
• Sensitivity is the proportion of people with
disease who have a positive test. (True
Positive)
• Specificity is the proportion of people free of
a disease who have a negative test. (True
Negative)
Are the Results Important?
• Using sensitivity and specificity: SpPin and SnNout
• Sometimes it can be helpful just knowing the sensitivity
and specificity of a test, if they are very high.
• If a test has high specificity, i.e. if a high proportion of
patients without the disorder actually test negative, it is
unlikely to produce false positive results. Therefore, if the
test is positive it makes the diagnosis very likely.
• This can be remembered by the mnemonic SpPin: for a test
with high specificity (Sp), if the test is positive, then it rules
the diagnosis 'in'.
• Similarly, with high sensitivity a test is unlikely to produce
false negative results. This can be remembered by the
mnemonic SnNout: for a test with high sensitivity (Sn), if
the test is negative, then it rules 'out' the diagnosis.
Are the Results Important?
• Positive Predictive Value = the proportion of
people with a positive test who have disease.
• True+/(True+ plus False+)
• Negative Predictive Value = the proportion of
people with a negative test who are free of
disease.
• True-/(True- plus False-)
Likelihood Ratios
• What do all these numbers mean? The Likelihood ratios indicate by how
much a given diagnostic test result will raise or lower the pretest
probability of the target disorder. A likelihood ratio of 1 means that the
posttest probability is exactly the same as the pretest probability.
Likelihood ratios >1.0 increase the probability that the target disorder is
present, and the higher the likelihood ratio, the greater is this increase.
Conversely, likelihood ratios <1.0 decrease the probability of the target
disorder, and the smaller the likelihood ratio, the greater is the decrease in
probability and the smaller is its final value.
• How big is a "big" likelihood ratio, and how small is a "small" one? Using
likelihood ratios in your day-to-day practice will lead to your own sense of
their interpretation, but consider the following a rough guide:
• Likelihood ratios of >10 or < 0.1 generate large and often conclusive
changes from pre- to posttest probability;
• Likelihood ratios of 5-10 and 0.1-0.2 generate moderate shifts in pre- to
posttest probability;
• Likelihood ratios of 2-5 and 0.5-0.2 generate small (but sometimes
important) changes in probability; and
• Likelihood ratios of 1-2 and 0.5-1 alter probability to a small (and rarely
important) degree.
Likelihood Ratios
• Having determined the magnitude and significance of the likelihood
ratios, how do we use them to go from pretest to posttest
probability? We cannot combine likelihoods directly, the way we
can combine probabilities or percentages; their formal use requires
converting pretest probability to odds, multiplying the result by the
Likelihood ratio, and converting the consequent posttest odds into a
posttest probability. Although it is not too difficult, this calculation
can be tedious and off-putting; fortunately, there is an easier way.
• A nomogram proposed by Fagan (Figure 1C-2) does all the
conversions and allows an easy transition from pre- to posttest
probability. The left-hand column of this nomogram represents the
pretest probability, the middle column represents the likelihood
ratio, and the right-hand column shows the posttest probability. You
obtain the posttest probability by anchoring a ruler at the pretest
probability and rotating it until it lines up with the likelihood ratio
for the observed test result.
Pre-test Prob 70% x +LR
7.5 = Post-Test Prob = 95%
Diagnosis - Analysis
Therapy - Analysis
Therapy - Analysis
Therapy - Analysis
Therapy - Analysis
Therapy - Analysis
Therapy - Analysis
Prognosis - Analysis
Prognosis - Analysis
Prognosis - Analysis
Prognosis - Analysis
Prognosis - Analysis