Information Mastery
Objectives
 At the end of this seminar, participants should be able to:
 Incorporate information mastery principles into daily learning




and patient care activities
Formulate focused foreground questions in PICO format
Distinguish between disease-oriented and patient-oriented
evidence
Identify high quality evidence based on study design
Use evidence-based medical databases to research clinical
queries and to stay up to date with medical literature
Disclosures
 No financial disclosures
 Much of the material and ideas were developed by
David Slawson, MD, and Alan Shaughnessy, PharmD,
MMedEd
What is EBM?
 “Evidence-based medicine is the conscientious,
explicit, and judicious use of current best evidence in
making decisions about the care of individual patients.”
 David Sackett, 1996
Why read literature?
 Answer clinical questions
 Keep up to date
 Follow your interests
 Be the best advocate possible for patients
Limitations of EBM
 Limited evidence
 Poor quality evidence
 Individual patient differences
What determines medical
decisions?
Prior clinical
experience
Individual
patient
characteristics
Medical
Evidence
Decision
Classic EBM
 5 step approach





Ask
Search
Critically appraise
Implement
Evaluate
EBMInformation Mastery
 Traditional EBM = basic science, primary article
appraisal
 Information Mastery = applied science
 1994, Slawson and Shaughnessy
 Gather valid, relevant, patient-oriented information that is
critically appraised and apply it to practice.
 Use tools that appraise evidence for you
 Allows you to stay up do date without drowning in the vast
sea of primary journal articles.
Usefulness
 Usefulness = (relevance * validity)/work
 Relevance: Does it matter to my patient?
 Validity: How well does the study reflect the truth?
 Less work is better
 Maximize reading high quality information without sifting
through poor quality information
Clinical Questions
 Type of question determines sources to use
 Background (basic science)
 Foreground (specific clinical question)
 PICO




Patient population/problem
Intervention
Comparison
Outcome
ACTIVITY 1
 Write a PICO for each case: Patient population/problem,
Intervention, Comparison, Outcome
 Case 1: “A 65 year old man with T2DM checks his blood
sugar daily and does his best to control his blood sugar with
exercise and nutrition. He wonders if having well-controlled
blood sugars overtime will increase his lifespan.”
 Case 2: “A 40 year old woman presents with migraine
headaches that are becoming more She is reluctant to use
medications other than herbal supplements and tells you
that she just read an article about the possible benefits of
riboflavin for preventing migraines.
DOE vs POE
 Disease-oriented evidence/outcomes
 Focused around diseases/labs.
 Patient-oriented outcomes
 Focused on outcomes patients care about:
 Quality of life
 Morbidity
 Mortality
Disease-Oriented
Outcome
Patient-Oriented
Outcome
Intensive glucose lowering can Intensive glucose lowering
decrease A1c
does not decrease mortality
Beta-carotene, Vit E are good
antioxidants
Neither prevents cancer or CV
disease
Varenicline is effective for
helping patients quit smoking
Varenicline increases the risk
of adverse CV events
POEM
 Patient Oriented Evidence that Matters
 Is information relevant & does it matter? 3 criteria:
 Do patients care about the outcomes/is it patient-oriented
(quality of life, morbidity, mortality)?
 Is the intervention feasible?
 If true, will it require you to change your practice?
 Yes to all 3 = POEM
ACTIVITY 2
 Read 2 evidence summaries
 Discuss in small groups to determine if they are
POEMS
 3 criteria of POEMS
 Do patients care about the outcomes/is it patient-oriented
(quality of life, morbidity, mortality)?
 Is the intervention feasible?
 If true, will it require you to change your practice?
Evidence Hierarchy
Graded Evidence
 SORT
 Developed by AAFP
 Takes POEM into consideration
 USPSTF Graded recommendations
 GRADE
 Developed by international group of physicians
 Level of Evidence
 11 categories
 Developed at Oxford
SORT (Strength Of
Recommendation Taxonomy)
Code
Definition
A
Consistent, good-quality patient-oriented evidence
B
Inconsistent or limited-quality patient-oriented evidence
C
Consensus, disease-oriented evidence, usual practice, expert
opinion, or case series for studies of diagnosis, treatment,
prevention, or screening
Effect on Patient-Oriented
Outcomes
Symptoms (drivers license)
Functioning (visual loss)
Quality of Life (leg ulcers)
Lifespan
Effect on Disease Markers
Diabetes (Photocoagulation,
GFR, NCV)
Arthritis (x-ray, sed rate)
Peptic Ulcer (endoscopic
ulcer)
SORT
B
SORT
A
SORT
C
Effect on Risk Factors for
Disease
Improvement in markers
(blood pressure, HBA1C,
cholesterol)
Uncontrolled Observations
&
Conjecture
Physiologic Research
Preliminary Clinical
Research
Case reports
Observational studies
Validity of Evidence
Highly Controlled Research
Randomized Controlled
Trials
Systematic Reviews
USPSTF Grades
Grade
Definition
A
The USPSTF recommends the service. There is high certainty that the net
benefit is substantial
B
The USPSTF recommends the service. There is high certainty that the net
benefit is moderate or there is moderate certainty that the net benefit is
moderate to substantial.
C
The USPSTF recommends selectively offering or providing this service to
individual patients based on professional judgment and patient
preferences. There is at least moderate certainty that the net benefit is
small.
D
The USPSTF recommends against the service. There is moderate or high
certainty that the service has no net benefit or that the harms outweigh the
benefits.
I
The USPSTF concludes that the current evidence is insufficient to assess
the balance of benefits and harms of the service. Evidence is lacking, of
poor quality, or conflicting, and the balance of benefits and harms cannot
be determined.
GRADE (Grade of Recommendations,
Assessment, Development, and Evaluation)
Code Quality of Evidence
Definition
A
High
Further research is very unlikely to change our
confidence in the estimate of effect. Several high
quality studies with consistent results or one large
high quality multi-center trial
B
Moderate
Further research is likely to have an important
impact on our confidence in the estimate of the
effect and may change the estimate. One high
quality study, several studies with some limitations
C
Low
Further research is very likely to have an
important impact on our confidence in the
estimate of the effect and is likely to change the
estimate. One or more studies with severe
limitations
D
Very low
Any estimate of effect is very uncertain Expert
opinion, no direct research evidence, onre or more
studies with very severe limitations.
Level of Evidence
 Developed by Centre for Evidence-Based Medicine in
Oxford, England
 More detailed and complex
 11 levels
 1a-c (systematic reviews), 2a-c, 3a-c, 4, 5 (expert
opinion)
 http://www.cebm.net/oxford-centre-evidence-basedmedicine-levels-evidence-march-2009/
table)
(see for a
Information Mastery Toolkit
 Journals (2-3)
 Foraging services (summaries of new information)
 Hunting tools (find answers to questions)
“Foraging” Services
 Analyze articles/evidence for you and send you
summaries
 DynaMed Alerts
 EE Plus POEMS
 BMJ Clinical Evidence
 FPIN Clinical Inquiries
 ACP Journal Club
Characteristics of an Ideal
Foraging/Alert Tool
 How is the information filtered?
 Specialty specific
 Patient-oriented (relevance)
 Is the information valid?
 Backed-up by evidence (level of evidence, SORT is always best)
 Is the information summarized and easy to access?
 Comprehensive but summarized (2000-3000 words accurately in
200 words)
 Point of care (work)
 Coordinated with a hunting tool
 Is the information placed into context?
 Translational validity
 More than abstracts
Characteristics of an Ideal
Foraging/Alert Tool
 Specialty-specific
 Patient-oriented (relevance)
 Backed up by levels of evidence, SORT is best
(validity)
 Comprehensive but summarized (2000-3000 words
accurately in 200 words)
 Point of care (work)
 Coordinated with a hunting tool
Foraging/Alert Tool Risks
 Who’s paying when it is free?
 Possibly pharmaceutical companies
 Abstracts only contain no relevance/validity filter
 Examples are Journal Watch, Clinical Updates
 TAKE HOME POINT: If it’s free there may be something
wrong with it. Quality often doesn’t come free!
Foraging Tool Overview
Tool
Less work
More work
ACP Journal Club
-Specialty Specific (IM)
-Validity assessment but no
LOE
-Relevance: no POE vs
DOE, no “matters” factor
-No hunting tool
BMP Updates
-Specialty Specific
(various)
-Validity assessment but no
LOE
-Relevance: no POE vs
DOE, no “matters” factor
-No hunting tool
DynaMed Alerts
-Specialty specific (various)
-Validity assessment
-LOE
-Relevance: focuses on
patient-oriented evidence
-Coordinated hunting tool
Foraging Tool Overview
Tool
Less work
More work
Journal Watch
-Specialty Specific
(various)
-No validity assessment
-No LOE
-Relevance: no POE vs
DOE, no “matters” factor
-No hunting tool
Medscape
-Specialty Specific
(various)
-No validity assessment
-No LOE
-Relevance: no POE vs
DOE, no “matters” factor
-No hunting tool
Fig 1 Updating curves for relevant evidence (128 systematic reviews) by point of care
information summaries (log rank χ2=404, P<0.001).
Banzi R et al. BMJ 2011;343:bmj.d5856
©2011 by British Medical Journal Publishing Group
Summary of Foraging Tools
 DynaMed: Fastest to update with new information
 BMJ Clinical Evidence: Only sends valid articles
 UpToDate: e-mails article authors every 6 months to
ask for updates
 No one has looked at how accurately information is
summarized/said in the tools (are summaries valid?)
EBM “Hunting” Tools
 Point of care evidence-based tools (30-40 seconds)
 Best tools = useful = (relevance*validity)/work







DynaMed
Cochrane database
Essential Evidence Plus
BMJ Clinical Evidence
FPIN Clinical Inquiries
Trip database
Pub Med
Drilling for the Best Information
BMJ Clinical
Evidence
(therapy)
R
V W
R
V W
R
V W
R
V W
PIER
R
V W
TRIP Database
R
V W
R
V W
R
V W
Usefulness
Essential Evidence
Plus
R
V
Usefulness = Relevance X validity
Work
W
Dynamed
UpToDate
Textbooks
Medline
Clinical Jazz
 Science (EBM/structure) + Art (improvisation) = Clinical
Jazz
 Both structure and improvisation are necessary, but
there’s not good evidence for many areas of medicine,
so we have a lot of liberty to improvise!
ACTIVITY 3
 Review use of the following tools:




DynaMed
Essential Evidence Plus
TRIP Database
BMJ Clinical evidence
ACTIVITY: Medical Myths
ACTIVITY: Look-up
conference
Some Studies that I Like to Quote
http://www.youtube.com/watch?v=Ij8bPX8IINg
James McCormack, MD