InterCESS: How EBM
principles apply to the
Internet
Andrew Booth & Louise Falzon
The Ingredients of EBM
• Focused questions
• Hierarchy of
evidence
• Filters
• Problem based
learning
– Scenarios
– Anatomy of the
question
– Shared expertise
Focusing the question
• Helping requester to
break information
need down into
EBM anatomy
clarifies the
originating question.
• “OR” within
categories and
“AND” between
categories.
Understanding study
design
• Adds to your ability
to recognise “good
studies”and “bad
studies”.
• Helps you to
identify the question
being answered by
an article.
Using methodological
filters
• One line, sensitive and specific versions
• rely on publication types or occurrence
of textwords to identify
methodologically superior studies
• are also used to compile pre-filtered
database products e.g.. Cochrane
Library (clinical trials), DARE (reviews)
& NHS EED (economic evaluations)
EBM and the Internet
Why is the Internet
important for EBM?
• New paradigm rapid but
simultaneous
learning
• Publishing medium
• Delivery mechanism
• Database access
• Teaching packages
What are the dangers
of the Internet for
EBM?
• quality of
information
• lack of context
• no responsibility for
revision
• size and lack of
control
Challenges for today
• Can the Internet be used to answer patient
focused scenarios?
• Are there valuable documents that can
supplement our hard copy collections?
• Is “focusing the question” applicable to the
WWW?
• Can we devise the equivalent of “filters”?
• Can we use the WWW for teaching EBM?
The Programme
• Databases for Clinically Effective
Practice
• Full-text reports on the Web
• One-stop shops
• Focusing the question using
general search engines
Which databases?
• For Therapy questions - Trials Registers
• For Guidelines - Guidelines Databases
• For Reviews - DARE, Cochrane,
- HealthSTAR
• For Economic Studies - NHS EED, CDC
• For ALL question types - PubMed, Research
Registers, Health Technology Assessment
Reports
Full-Text reports on the
Web
Issues:
• How do you locate?
• How do you prioritise
sites to check?
• Executive summary?
• Order facility?
• All of the document?
• All documents?
• Document Types:
– HTML (In pieces?)
– Word 6.0 (Viewer
available from
Microsoft site)
– Word 8.0
– Adobe Acrobat (pdf)
(Acroread viewer
available from
Adobe)
Issues around Report
contents
• Whole report on a
general subject but
chapter on your topic
e.g. Drug therapy for
MS
• Generic names instead
of proprietary names
• Classes of drugs e.g
lipid-lowering agents
• Solutions: Do some
general Internet
searching on topic first.
What is it? What else is
it called? Is there
patient information on
topic? Has there been
FDA approval
(drugs)?etc.
Issues around report
producers
• Who are they?
– National HTA
agencies
– Regional HTA
agencies
– National R&D
projects or
programmes
– Academic centres
• But also?
– Drug company
sponsored research
units/health
deliverers
– Non-profit making
or health
maintenance
organisations (but...)
e.g. Group Health
Northwest
Two tools to help you
HTA Database
NHSCRD York
INAHTA Current
Projects in Progress
• Records for reports of
all INAHTA members.
Some of these on the
Web, others sent on
demand.
• Search Engine/
Common CRD interface
• http://nhscrd.york.ac.uk
/
• Brief title entry for all
projects being
undertaken by
INAHTA members.
• Crude search engine
• Requires follow-up to
Web sites
• http://nzhta.chmeds.ac.nz/
inahta/inahta.htm
One-stop shops
• TRIP Database
– http://www.tripdatabase.com/
• SUMSearch
– http://sumsearch.uthscsa.edu/
• NELH PC
– http://www.nelh-pc.nhs.uk/
One Stop Shops cont
• Medline Pro
– http://www.medlinepro.com/
• PubMed Clinical Queries
– http://www.ncbi.nlm.nih.gov/entrez/query/
static/clinical.html
Using a general search engine
Focusing the question
A cautionary tale I:
• Systematic search using Yahoo and
Excite search engines for sites relating
to feverish children.
• Of 41 Web pages identified only four
compared favourably with
recommendations of published
guidelines
• Impicciatore et al BMJ 1997; 314: 18751877.
Search Engines
General
• AltaVista
• Excite
• HotBot
• Infoseek
• Lycos
• Northern Lights
• Yahoo
Multi-search
• Inference Find
• Megasearch
• Meta Crawler
etcetera
A Cautionary tale - II
• The Web contains an estimated 320 million
pages of information. HotBot covers 34 %,
AltaVista 28 %, Northern Light 20 %, Excite
14 %, Infoseek 10 % and Lycos 3 %.
• "Combining six engines in this study covered
about 3.5 times more of the Web".
• Steve Lawrence and C. Lee Giles. Searching the
World Wide Web," Science, 280, April 3, 1998, p.98100.
An Evidence Based
Conclusion
• "Given that the coverage of any one
search engine is limited, the simplest
means of improving the coverage of
Web searches is to combine the results
of multiple engines, as is done with
new search engines such as
MetaCrawler (www.metacrawler.com),"
they wrote.
Cp. Also Megasearch engine at www.dogpile.com
However...........
• Mega-search engines do not usually use
Boolean logic. They are good for a search on a
single distinctive name e.g. celecoxib
• For a focused question it is better to use a
single search engine with Boolean e.g. Alta
Vista Advanced (www.altavista.com)
• Use Intervention AND Population (then
Filter, then Outcomes/Comparisons)
Example
Effectiveness/Cost effectiveness of
Propentofylline in Vascular and
Alzheimer's Disease
• POPULATION= AD/Vascular Disease
• INTERVENTION= Propentofylline [28]
• FILTERS=“technology assessment”[2],
“economic*”[4], “clinical trial*” [9]
Tips & Tricks [Alta Vista]
• Finding HTA
materials
• Limiting to a
specific domain e.g.
only sites at a UK
– Combine with a filter
such as “technology
academic institution
assessment” or
“clinical trials”
• Finding images
– Combine with image
command:
image:hernia
– Combine with host
command:
host:ac.uk
Another filtered
approach
• Use Publication
Type field in
www.omni.ac.uk
• Accessed via MeSH
• Includes:
–
–
–
–
Practice Guideline
Clinical Trial
Meta-analysis
RCT
In Conclusion
• It is possible to
practice EBM using
the Internet
• But it is much
cruder and more
time consuming
• Nevertheless
plucking a jewel
from the sewage
may just make
someone’s day!