Maurizio A. Leone
Neurologic Clinic
University Hospital
“Maggiore della Carità”,
Novara, Italy
I International Course on Neuroepidemiology in Eastern
Europe, Chisinau, Moldova. September 24-28, 2012

•
The need for EBM
•
What is EBM ?
•
The story
•
How can I practice EBM ?
• What are the “products” of EBM ?
•
Limitations of and criticisms to EBM

On the need of
Evidence Based Medicine
1. Few medical decisions are based on evidence

Why do we need RANDOMIZED
CONTROLLED TRIALS ?
In the early 1980s newly introduced antiarrhythmics were found to be highly successful at suppressing arrhythmias.
Not until a RCT was performed was it realized that, although these drugs suppressed arrhythmias, they actually increased mortality.
The CAST trial revealed Excess mortality of
56/1000.
By the time the results of this trial were published, at least 100,000 such patients had been taking these drugs.

Caveat
Many medical decisions are taken on the basis of:
• etiopathogenetic theories
• experience
• other factors (pharma industry, other interests, ideology…)

Probability of a result favouring study drug in industry-sponsored trials
OR = 4.05 (2.98-5.51)

On the need of
Evidence Based Medicine
1. Few medical decisions are based on evidence
2. Deterioration of knowledge after graduation

Performance deteriorates ….
Determinants of the clinical decision to treat some, but not other, hypertensives:
1. Level of blood pressure.
2. Patient’s age.
3. The physician’s year of graduation from medical school.
4. The amount of target-organ damage.

On the need of
Evidence Based Medicine
1. Few medical decisions are based on evidence
2. Deterioration of knowledge after graduation
3. Delay in transferring results from research to clinical practice

Systematic review of bed rest after spinal tap
• 10 trials of bed rest after spinal puncture
– no change in headache with bed rest
– Increase in back pain
• Protocols in UK neurology units - 80% still recommend bed rest after LP
Serpell M, BMJ 1998;316:1709–10
• …evidence of harm available for 17 years preceding...
Allen, Glasziou, Del Mar. Lancet, 1999

On the need of
Evidence Based Medicine
1. Few medical decisions are based on evidence
2. Deterioration of knowledge after graduation
3. Delay in transferring results from research to clinical practice
4. Limited resources

Information epidemic: how to keep up-to-date ?
MEDLINE 2010
2,000 articles / day approx 75 new trials published every day
Bastian, Glasziou, Chalmers (2010) 75 Trials and 11Systematic Reviews a Day: How Will We Ever Keep Up? PLoS Med 7(9)

Median minutes/week spent reading about my patients (self report):
• Medical Students:
• House Officers:
• Senior House Officers:
• Registrars:
• Senior Registrars
• Consultants:
90 minutes
0 (up to 70%=none)
20 (up to 15%=none)
45 (up to 40%=none)
30 (up to 15%=none)
– Grad. Post 1975: 45 (up to 30%=none)
– Grad. Pre 1975:
30 (up to 40%=none)

• The need for EBM
•
What is EBM ?
• The story
• How can I practice EBM ?
• What are the “products” of EBM ?
• Limitations of and criticism to EBM

In Italian (Rumanian-Moldovan):
• Anything that is clear and obvious, that doesn’t need any further demonstration
In English:
• The available body of information indicating whether a belief or a proposition is true or valid (= proof, testimony, sign).

‘Background’ Questions
• About the disorder, test, treatment, etc.
2 components: a. Root* + Verb: “What causes …” b. Condition: “… stroke?”
• * Who, What, Where, When, Why, How

Background learning: basic neurosciences, neuroanatomy, neurophysiology, neuropharmacology, amd neuropathology
Foreground learning: Clinical decision-making , history taking, examining, diagnosing, and therapeutic intervening.

Evidence-Based Medicine:
The Practice
1.
Translation to an answerable question
(patient/manoeuvre/outcome).
2.
Efficient track-down of the best evidence
3.
Critical appraisal of the evidence for its validity and clinical applicability
4.
Integration of that critical appraisal with clinical expertise and the patient’s unique biology and beliefs.
5.
Evaluation of one’s performance.

• The need for EBM
• What is EBM ?
•
The story
• How can I practice EBM ?
• What are the “products” of EBM ?
• Limitations of and criticism to EBM

Evidence-based Medicine: the story
Pierre Charles Alexandre Louis (Paris, 1830): “A true science is merely a summary of facts which have no value unless they are expressed in numbers ... The statistics represent the only and fundamental basis of all the medical studies ‘. He was the promoter of “ La Societé d’Observation
Medicale”, a cultural movement supporting the concept that knowledge about a disease, its history, clinical presentation and treatment, could be derived from aggregated patient data rather than from individual experience.
Archibald Cochrane (1972): “It is surely a great criticism of our profession that we have not organized a critical summary, by specialty or subspecialty, adapted periodically, of all relevant randomized controlled trials.”
1981: (McMaster Medical School, Canada): “How to read clinical journals", a series of articles describing the strategies of critical approach to the biomedical literature. These articles are among the top reprinted in the history of biomedical literature.
Dave Sackett
1986: The focus of Sackett and coll. gradually moves from "how to read the biomedical literature" to "how to use the biomedical literature to solve clinical problems."

1993 JAMA: first article of the movement of Evidence-Based Medicine
1993: founding of the Cochrane
Collaboration, an international network to “prepare, update and disseminate systematic reviews of controlled trials on the effects of health care and, where there is no controlled trials, systematic reviews of existing evidence, however.” http://www.cochrane.org

Review Groups
• Back
• Consumers and communication
• Dementia & cognitive improvement
• Incontinence
• Infectious diseases
• Injuries
• Movement disorders
• Multiple sclerosis
• Muskuloskeletal
• Musculoskeletal injuries
• Neuromuscular disease
• Pain, palliative and supportive care
• Drugs and alcohol
• Epilepsy
• Eyes and vision
• Stroke

• The need for EBM
• What is EBM ?
• The story
•
How can I practice EBM ?
• What are the “products” of EBM ?
• Limitations of and criticism to EBM

Evidence-Based Medicine:
The Practice
1.
Translation to an answerable question
(patient/manoeuvre/outcome).
????

Structure of the clinical question
1st part
2nd part
P opulation
Clinical characteristics of the patient or of the reference patients population
3rd part
Courtesy of Luca Vignatelli

Structure of the clinical question
1st part P opulation
2nd part I ntervention
C omparator
Clinical characteristics of the patient or of the reference patients population
Therapy : drug / device / procedure
Diagnosis : test
Alternative clinical act (placebo / no therapy / other therapy / other test)

Structure of the clinical question
1st part P opulation
2nd part I ntervention
3rd part
C omparator
O utcome
T ime
Clinical characteristics of the patient or of the reference patients population
Therapy : drug / device / procedure
Diagnosis : test
Alternative clinical act (placebo / no therapy / other therapy / other test)
Clinical outcome of interest (to be increased/reduced)
Time dimension of the observation of the outcome

Formulate a research question that can be answered
Can coffee reduce daytime drowsiness?
Components: PICO(T)
•
P opulation
• I ntervention
•
C ontrol
• O utcome
•
T (Time)
Intervention
Population
In healthy adults engaging in normal activity, does coffee reduce daytime drowsiness compared with de-caffeinated coffee? Outcome
Control

Evidence-Based Medicine:
The Practice
1.
Translation to an answerable question
(patient/manoeuvre/outcome).
2.
Efficient track-down of the best evidence

Traditional information instruments
•
The expert (colleague): lack of obiectivity and completeness of information.
•
Medical books (printed): often incomplete, outdated, lack of quantitative data. Selection of scientific evidence is not systematic neither explicit; generally organized by disease and not by clinical presentations.
•
Journals: too many, fragmentation of topics, rarity of definitive studies to be transferred to clinical practice.
Traditional (narrative) reviews: same methodological shortcomings of the medical books: selection bias, self-quotation and harmony with the opinion of the author.
Randomized Clinical Trials (RCTs): difficult to adapt their results to the individual patient. Selected populations, complex patients excluded. Ideal world and peculiar competence and motivation of doctors. Often surrogate rather than clinically significant end-points, often relative and not absolute
(NNT) measures are reported.

Pyramid of the production of medical literature
GUIDELINES / HTA
SECONDARY LITERATURE
PRIMARY LITERATURE
CLINICAL STUDIES
Courtesy of Luca Vignatelli

How to organize the search of evidence in the literature
DATA-BANKS OF GUIDELINES / HTA
SECONDARY DATA-BANKS
PRIMARY DATA-BANKS
RCTs DATA-BANKS

Clinical research, synthesis of studies and recommendations
Results of research
Primary
Studies
Sinthesis

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.
Systematic Reviews
Meta-analyses

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.
Recommendations
HealthTechnology
Assessments (HTA)
Guidelines

Evidence-Based Medicine:
The Practice
1.
Translation to an answerable question
(patient/manoeuvre/outcome).
2.
Efficient track-down of the best evidence
3.
Critical appraisal of the evidence for its validity and clinical applicability
We needn’t always carry out all 5 steps to provide E-B Care

• The need for EBM
• What is EBM ?
• The story
• How can I practice EBM ?
• What are the “products” of EBM ?
• Limitations of and criticism to EBM

Guidelines are:
• Recommendations of clinical practice,
• produced through a systematic process,
• to assist physicians and patients
• in deciding which are the most appropriate method of care
• in specific clinical circumstances.

Aims of the guidelines
To ensure the highest degree of appropriateness of the interventions, reducing the possible variability in clinical decisions (due to lack of knowledge and subjectivity in the definition of strategies of care).

Basic methodological elements
• Multidisciplinarity
• Use of systematic reviews
• Explicit evaluation of quality of evidence
• Strenght of recommendations

Requirements of a guideline
• Validity
• Reproducibility
• Representativeness
• Applicability
• Flexibility
• Clarity
• Documentation
• Update

Who produce guidelines?
• National and Regional
Agencies
• Scientific Societies
(EFNS,….)
• Research Institutions
• Ad hoc Associations
• ..

Where to find guidelines?
Data Bank
National Guideline Clearinghouse™
National Library of Guidelines
(GuidelinesFinder)
CMA Infobase
International Guidelines Library
Istituzione
Agency for
Healthcare Research and Quality (USA)
NHS Evidence (UK)
Canadian Medical
Association (CMA)
G-I-N – the
Guidelines
International Network

The quality of evidence (levels of evidence) depends on:
• Appropriatness of study design
• Quality of study conduction and analysis
• Effect size
• Appropriatness and relevance of outcomes
The strength of a recommendation reflects the extent to which we can, across the range of patients for whom the recommendations are intended, be confident that desirable effects of a management strategy outweigh undesirable effects.

EFNS Guidelines: classes of evidence
•
Class I: An adequately powered prospective, randomized, controlled clinical trial with masked outcome assessment in a representative population or an adequately powered systematic review of prospective randomized controlled clinical trials with masked outcome assessment in representative populations. The following are required:
• (a) randomization concealment
• (b) primary outcome(s) is/are clearly defined
• (c) exclusion/inclusion criteria are clearly defined
• (d) adequate accounting for dropouts and crossovers with numbers sufficiently low to have minimal potential for bias
• (e) relevant baseline characteristics are presented and substantially equivalent among treatment groups or there is appropriate statistical adjustment for differences
• Class II: Prospective matched-group cohort study in a representative population with masked outcome assessment that meets a–e above or a randomized, controlled trial in a representative population that lacks one criteria a–e
• Class III: All other controlled trials (including well-defined natural history controls or patients serving as own controls) in a representative population, where outcome assessment is independent of patient treatment
• Class IV: Evidence from uncontrolled studies, case series, case reports, or expert opinion

•
Level A: (established as effective, ineffective, or harmful) requires at least one convincing class I study or at least two consistent, convincing class II studies
•
Level B: (probably effective, ineffective, or harmful) requires at least one convincing class II study or overwhelming class III evidence
•
Level C: (possibly effective, ineffective, or harmful) rating requires at least two convincing class III studies

G rades of R ecommendation A ssessment,
D evelopment and E valuation

Consensus Conference
Drafting of recommendations by a jury at the end of a presentation and consultation of experts summarizing scientific knowledge on a given topic.
Agreement among different figures regarding controversial and complex health issues, to provide patients the best quality of care in relation to available resources.

Clinical pathway: result of guidelines adaptation to local situations, with their specific organizational and management characteristics.
Protocol: the detailed outline of the steps to be followed in the treatment or the diagnosis of a patient.

What is a systematic review?
• Is a research , conducted with an explicit and repeatable method to determine what is the best available evidence on the effectiveness of an intervention or the accuracy of a diagnostic test.
• The meta-analysis is the most well-known statistical tool associated with the systematic reviews. It is used to comprehensively analyze all the observations of controlled clinical trials and to provide a summary estimate.

FEATURES TRADITIONAL
REVIEWS
SYSTEMATIC
REVIEWS
Question
Protocol often large no
Literature search strategy not specified
Selection of studies criteria not specified
Evaluation of studies
Synthesis
Inferences often focused yes explicit and reproducible inclusion/exclusion criteria variable rigorous and critical often qualitative summary quantitative summary
(meta-analyses) not always evidence-based generally evidence-based
Cook DJ et al Ann Int Med 1997; 126:320, adapted

Systematic Reviews seek to:
• Identify all relevant published and unpublished primary studies
• Select studies or reports for inclusion
• Critically appraising the quality of each study or report
• Extract data
• Quantitatively synthetise findings from individual studies or reports in an unbiased way
• Interpret the findings and present a balanced and impartial summary with due consideration of any flaws.
• SR may examine quantitative or qualitative evidence.

If we believe it, does it apply to our patient?
• Is our patient (or population) so different from those in the primary studies that the results may not apply?
• consider differences in:
- time - many things change.
- culture - both treatments and values of outcomes can be different.
- stage of illness or prevalence can affect results.

We believe it !
But —>> does it matter?
•
Is the benefit worthwhile to my patient?
Ask the patient about cultural values.
•
Are the outcomes relevant for my patient?
Think about Relative Risk Reduction vs. Absolute
Risk to your patient. Potential benefit is the
Absolute risk avoided in our patient!

Event Rate = 1 relapse at 2 years
Control
Event
Rate
(CER)
Experimen tal Event rate (EER)
RRR ARR
(CER-EER)
/CER
(CER-
EER)
IFNBMSG 76% 64%
(76-64)
/76 = 16%
NNT
1/AAR
(76-64)=
12%
100/12=8

Event Rate = 1 relapse at 2 years
Control
Event
Rate
(CER)
Experimen tal Event rate (EER)
RRR ARR
(CER-EER)
/CER
(CER-
EER)
IFNBMSG 76% 64%
Hypothesis
A
7,6% 6,4%
(76-64)
/76 = 16%
(7,6-6,4)
/7,6 = 16%
NNT
1/AAR
(76-64)=
12%
100/12= 8
(7,6-6,4) =
1,2%
100/1,2=
83

Event Rate = 1 relapse at 2 years
Control
Event
Rate
(CER)
Experimen tal Event rate (EER)
RRR ARR
(CER-EER)
/CER
(CER-
EER)
IFNBMSG 76% 64%
Hypothesis
A
Hypothesis
B
7,6% 6,4%
0,76% 0,64%
NNT
1/AAR
(76-64)
/76 = 16%
(7,6-6,4)
/7,6 = 16%
(0,76-0,64)
/0,76= 16%
(76-64)=
12%
100/12= 8
(7,6-6,4) =
1,2%
100/1,2=
83
(0,76-0,64)
= 0,12%
100/0,12=
833

For(r)est plot
No difference line
Infezioni nosocomiali
Profilassi vs controllo
Punctual estimate
Study
Abele
Aerdts
Blair
Boland
Cockerill
Finch
Jacobs 1
Kerv er
Palomar
Rocha
Sanchez
Stoutnbek
Ulrich
Verwaest1
Verwaest 2
Winter
.01
Favours treatment
.05
.2
Risk ratio
(95% CI)
0.29 (0.17,0.49)
0.07 (0.01,0.52)
0.33 (0.18,0.62)
0.82 (0.49,1.37)
0.33 (0.11,0.99)
0.69 (0.23,2.01)
0.11 (0.01,2.05)
0.15 (0.07,0.36)
0.39 (0.21,0.73)
0.32 (0.15,0.68)
0.57 (0.40,0.81)
0.61 (0.47,0.78)
0.28 (0.13,0.59)
0.53 (0.33,0.85)
0.72 (0.47,1.10)
0.18 (0.05,0.59)
Confidence Intervals
Not statistically significant results
1 5
Risk ratio
Favours control

Total (95% CI) 410/1194 301/925 100.0 0.96 (0.85, 1.08)
Test for heterogeneity

2 =18.11 df=14 p=0.2020 I 2 =23.3%
Test for overall effect z=0.68 p=0.50
The pooled analysis of the results of all studies is represented by a diamond (weighted average).
If the confidence interval includes the unity, this means that we have not found a statistically significant difference between the two treatments.

Standard Meta-analysis (Left)&
Cumulative Meta-analysis (Right)

Where to find systematic reviews?
Cochrane Database of Systematic Reviews
DARE : http://www.crd.york.ac.uk/crdweb

Health Technology Assessment
• The systematic evaluation of properties, effects, and/or impacts of health care technology
• It may address the direct, intended consequences of technologies as well as their indirect, unintended consequences
• Its main purpose is to inform technology-related policymaking in health care
• HTA is conducted by interdisciplinary groups using explicit analytical frameworks drawing from a variety of methods
(HTA glossary - www.htaglossary.net
)

Were to find Health Technology Assessments http://www.crd.york.ac.uk/crdweb/

• The need for EBM
• What is EBM ?
• The story
• How can I practice EBM ?
• What are the “products” of EBM ?
•
Limitations of and criticism to EBM

Limitations of EBM
1. Shortage of consistent scientific evidence
2. Difficulties to apply evidence to the careof individual patient
3. Economic barriers to the practice of highquality medicine
4. Need to develop new skills
5. Limited time and resources
6. Paucity of evidence that EBM “works”

Misperceptions of EBM
1.
EBM denigrates clinical expertise
2.
It ignores patients’ values and preferences
3.
It promotes a cookbook approach to medicine
4.
It is simply a cost-cutting tool
5.
It is an ivory-tower concept
6.
It is limited to medical research
7.
It leeds to therapeutic nihilism in the absence of evidence from randomized clinical trials

EBM for clinical decision-making
Individual clinical expertise
Best available evidence from medical research
Patients’ values and expectations
Sackett et al. 2000
Available resources

Best Evidence
Resources for Neurology
Guidelines (free access): American Academy of Neurology www.aan.com/go/practice/guidelines
• European Federation of Neurological Societies
(EFNS) www.efns.org
CATs Data bank: Evidence Based Neurology, University of
Western Ontario www.uwo.ca/cns/ebn/
Courtesy of Luca Vignatelli