Evidence-based
librarianship
Jessie McGowan
Senior Information Scientist, Institute of Population
Health / Ottawa Health Research Institute
Adjunct Professor, Departments of Medicine and
Family Medicine, University of Ottawa
Associate Editor, Journal of Clinical Epidemiology
Overview
•
•
•
•
What is evidence-based stuff?
What is evidence-based librarianship (EBL)?
EBL examples
Knowledge translation (KT) and EBL
• Question to ponder: Do we give users what they
want or what they need? How does the quality of
resource effect collection development policies?
Evidence-based history
• Evidence-based philosophies integrate a health professional’s
experience and knowledge with the best currently available
clinical evidence.
• It was introduced as EBM by ACP Journal Club in 1991 and this
editorial was the impetus for a a series called the Users’ Guides
to the Medical Literature in order to help clinician decide how to
incorporate these philosophies in to their daily practice.
•
•
Guyatt GH. Evidence-based medicine (editorial). ACP Journal Club suppl 1991;2:A-16.
Guyatt GH. Users/ guides to the medical literature (editorial). JAMA 1993;270(17):2096-7.
Evidence-based evolution
Evidence-based medicine
Evidence-based health care,
practice, dentistry, nursing etc.
Evidence-based medical / health
librarianship
Evidence-based librarianship
Evidence-based approach
•
•
•
•
•
1. Formation of a clinical question (s) (usually from an
existing clinical scenario). It is formed using the
following formula of PICO(T):
Patient/population
Intervention
Comparison
Outcome
Time
PICO example
In a 41 year-old women with
osteoporosis (P), does the use of
etidronate (I) versus exercies (C)
improve bone density and reduce the
risk of fractures (O)?
Evidence-based approach
2. A literature search to identify the evidence
– Biomedical database searches, textbook searches, grey
literature, hand searching etc.
3. Critical Appraisal of the literature search results
4. Application of the results to the patient
5. Quality assurance of the previous steps
Evidence-based health librarianship
• EBL began with health librarians helping health
professionals integrate EBP in their daily practice.
Andrew Booth
Evidence-based health librarianship
Many different roles evolved;
– Educators / teachers (involved in curriculum)
– Trainers (in use of library applications)
– Researchers (methods research)
– Collaborators (in grants, systematic reviews)
– Information providers (providing expert searching)
Evidence-based for all librarianship
Now its almost mainstream! There is a
CLA Evidence Based Librarianship
Interest Group (EBLIG) and a journal
Evidence Based Library and
Information Practice and an
international conference help every two
years.
"Improve library practice by utilizing the
best available evidence combined with a
pragmatic perspective developed from
working experience in librarianship."
Madge, B. Evidence Based Librarianship:
Tools we all can use. MLA 2000.
Librarianship domains
The following six domains and their definitions were developed based
upon the major areas librarians deal with in their daily practice*:
• Reference/Enquiries—providing service and access to information that
meets the needs of library users.
• Education—finding teaching methods and strategies to educate users
about library resources and how to improve their research skills.
• Collections—building a high-quality collection of print and electronic
materials that is useful, cost-effective and meets the users needs.
• Management—managing people and resources within an organization.
• Information access and retrieval—creating better systems and
methods for information retrieval and access.
• Marketing/Promotion—promoting the profession, the library and its
services to both usersand non-users.
•
Crumly E, Koufogiannakid D. Developing evidence-based librarianship: practical steps for implementation.
Health Information and Libraries Journal 2002;19:61–70.
Evidence-based librarianship
• This is about getting the ‘science’ in information science.
Steps in EBL:
1. Define the problem
• Formulate a question arising from an issue or problem in a
library or in the use of a library based philosophy
2. Find the evidence
3. Critically appraise the evidence
4. Apply the appraised evidence to the problem
5. Quality assurance – evaluate your plan
Formulate a question
• Let’s think of a few scenarios and questions!
Scenario: The quality of bibliographies in the master
level theses of music students was noted to be poorer
than in other faculties (including poor referencing, low
referencing, mistakes, etc.). The Dean of Music raised
this issue with the Chief Librarian. The Chief Librarian
suggested that perhaps database training could help.
Formulate a question
The following question was proposed: Does database
search training improve the quality of bibilographies in
Masters thesis in music students.
• Population – Master level music students
• Intervention - Database search training
• Comparison – Students with no training (maybe use
another faculty for comparison)
• Outcomes – To improve the quality of bibliographies
Search the evidence
• What are the sources of information available to
librarians? What are specific resources?
– Databases
– Textbooks
– Internet
– Unpublished (grey) literature
– Expert opinion
EVIDENCE PYRAMID
http://servers.medlib.hscbklyn.edu/ebm/2100.htm
Quality and types of librarian
literature
• Systematic reviews or Meta-analyses
• Randomized controlled trials (RCTs)
• Cohort studies - describe possible causal links and
pose probabilities of risk
• Comparative studies
• Descriptive articles (including narrative reviews)
• Surveys
• Case studies
• Qualitative research (such as focus groups,
bibliometrics)
• Case study: a description of a process, project, program,
technology implementation, organization, library service etc.
• Cohort study: a study that tracks over time a defined
population (the cohort). These groups may or may not be
exposed to factors hypothesized to influence the probability of
the occurrence of a particular disease or other outcome. Cohorts
are defined populations which, as a whole, are followed in an
attempt to determine distinguishing subgroup characteristics.
• Comparative study: a study that uses a systematic effort to
find similarities and differences between two or more observed
phenomena. An example could include comparing and
contrasting OVID MEDLINE and Pubmed.
• Data-mining/ biblio-mining: a method that involves the
discovery of meaningful patterns from data retrieved from
automated methods. Bibliomining uses the combination of data
mining with bibliometrics, statistics, and reporting tools look at
patterns in library systems.
• Descriptive survey: a survey that describes the respondent
perspectives or experiences on the questions that were asked in
a predefined manner. Citation analysis represents a variation of
the descriptive survey method.
• Focus Group: a method that generate data or information from
a meeting of individuals. This is also sometimes used to add to
information gathered from other research methods such as
questionnaires.
• Gap analysis: a method involves surveys to identify
discrepancies or gaps between individual expectations.
• Narrative review (review article): a review or overview of a
subject that has been created from the results of an expert
literature search. These can provide concise introductions to
subjects.
• Program evaluation: a method that assesses the operation or
outcomes of a program. This can be very valuable in evaluating
a program or policy at different levels in their development or
implementation.
• Randomized controlled trial (RCT): a type of primary research
that tests control and treatment groups, where the group
participants have been randomly assigned. These trials can use
individuals or groups. For example, a study could randomize
different libraries to receive an intervention (such as access to
certain resource).
• Systematic review (Meta-analysis): a review that uses
systematic and explicit methods to identify, select and critically
appraise relevant research, and to collect and analyze data from
the studies that are included in the review. Statistical methods
(meta-analysis) may or may not be used to analyze and pool the
results of the included studies. Systematic reviews (also known as
systematic overviews, evidence summaries and integrative
reviews) use recently developed scientific methods to summarize
results from multiple research studies.
Critical Appraisal
• Less is written about critical appraisal in librarianship.
It can be difficult for librarians to interpret the
evidence.
• Journals clubs can be used to improve skills*.
Pearce-Smith N. A journal club is an effective tool
for assisting librarians in the practice of
evidence-based librarianship: a case study. Health
Information and Libraries Journal. 2006 Mar; 23(1):
32-40.
Application to practice
• Now that you asked your question,
gathered your evidence, reviewed your
evidence, what do you do!!!!
Challenges for EBL
• Quality of the evidence
• Dispersion of evidence sources (e.g., education,
management, computer science)
• Skills in conducting research
• Skills in disseminating research
• Skills in interpreting research
• Time!
Andrew Booth & Anne Brice (http://www.chl.wales.nhs.uk/cpd/librarians/ebased.pdf)
EBL Resources
Evidence-based Practice for
Information Professionals: A
Handbook (Hardcover)
by Andrew Booth (Editor), Anne
Brice (Editor). 2004
ISBN-10: 1856044718
http://www.cla.ca/about/igroups/evidence_based.htm
http://ejournals.library.ualberta.ca/index.php/
EBLIP/about/submissions
EBL Example - PRESS
• PRESS – Peer Review Electronic Search Strategy
1) Develop a quality assessment checklist for
electronic search strategies used in HTAs/SRs;
2) Create a web-based peer review forum to these
searches.
• This project was graciously funded through a grant
by CADTH – Canadian Agency for Drugs and
Technologies in Health (formally CCOHTA)
• The final report to CADTH is complete and an HTA
Report is due Dec 2007/Jan 2008.
Why?
• The quality of Health Technology Assessment (HTA) reports
depends on many factors. One factor is the evidence base, the
literature and other information, upon which the HTA is based.
• Performing a high quality search of information resources
ensures accuracy and completeness of the evidence base used
in HTA reports. A current review does not exist to tell us what
elements of the search process have the most impact in the
overall quality of the resulting evidence base.
Research Components
1. a systematic review of the literature;
2. a consensus of expert opinion;
3. a peer review forum to evaluate electronic
search strategies during the initial stage of a
systematic review or health technology
assessment
Systematic review
Systematic searches
were performed to
identify evidence on
the importance of
checklist items to the
validity of the search.
7 bibliographic databases:
• MEDLINE
• LISA
• CINAHL
• The Cochrane Library’s
Methodology Register and
Methodology Reviews
• PsycINFO
• HAPI
• HealthSTAR
3 in-house databases
Systematic review results
• A total of almost 9,700 citations were retrieved.
Relevancy screening was done by 2 reviewers with
the use of SRS, a web-based systematic reviewing
platform by TrialStat.
• Over 100 relevant articles were obtained.
• Although 26 tools were identified that could
potentially be used as checklists, none were
validated for assessing electronic search strategies.
Consensus building - survey
• A survey of HTAs/SRs searchers was conducted
using Survey Monkey, a web-based survey platform.
• Fifty-eight respondents completed the survey and
allowed the elements to be ranked into three tiers of
importance based on assessment of the potential
impact of the elements on recall and precision.
Peer review forum
• Before programming the interface for a fully
operational version of the peer review forum,
10 survey respondents participated in a pilot
of a web-based peer review forum in which
10 sample searches.
• This was also used for validation testing of
the checklist and use of the web-site forum.
Implications for Decision Making
• Errors in the electronic search strategy have been demonstrated
to reduce the effectiveness of electronic search strategies used
in systematic reviews and (HTA) reports. Without assurance of a
bias-free and complete evidence-base, the true outcomes of a
systematic review cannot be tested. A checklist has been
developed to assess the electronic search strategy for errors.
• An evidence-based peer review process has been developed to
evaluate electronic search strategies for use in systematic
reviews and health technology assessment reports.
Implications for Decision Making
• As the evidence base for systematic reviews and HTA reports is
created in large part through the electronic search strategy, that
strategy should be peer reviewed before the evidence base is
created and reviewed.
• The evidence-based peer review of search strategies requires
the same body of expert knowledge needed to create search
strategies; therefore the peer review should be undertaken by
librarians or other suitably qualified and experienced individuals.
Evidence: the ultimate goal is KT
Knowledge translation is
the exchange, synthesis
and ethically-sound
application of
researcher findings
within a complex
system of relationships
among researchers and
knowledge users.
KT
• Biomedical research is consistently producing new
findings that may contribute to effective and efficient
patient care.
• The findings of such research will not change population
outcomes unless health services and health care
professionals adopt them in practice.
Grimshaw, Ward, Eccles. Oxford Handbook of Public Health.
Audiences for KT in health research
Audience
Clinical
Health
Services
Population
Health
+++
+++
+++
Professionals
+++
+++
Patients
+++
+++
Researchers
Basic
+++
Administrators
Policy Makers
Industry
…..
+++
+++
+++
+++
+++
+++
+++
What is the librarian’s role?
• This is the step beyond dissemination
• Current examples include:
– Evidence-based actionable messages
(EBAMS)
– Consumer summaries for Cochrane
reviews
– Summaries of newspaper articles
– Clinical question and answering service
Be it Resolved that
Evidence – Based
Librarianship is NOT
Hooey
Commentary. Evidence Based Library and Information Practice 2007, 2:2
• It is interesting that librarians, just like physicians, have felt
“offended” by being asked to embrace this philosophy.
• A few years after the introduction of evidence based
medicine in the literature, Sackett et al. replied to all
criticisms and verifying that the historical origins date back
to mid‐19th century Paris.
• This article is now over 20 years old and evidence based
medicine is now standard in most medical schools – we
need get on with things and catch up! Our question to
proponents is, “What is the alternative?” Just make do with
what feels good or see what Google has to say? We think
not.
Sackett, David L., William M. C. Rosenberg, J. A. Muir Gray, R. Brian Haynes, and W.
Scott Richardson. “Evidence based medicine: what it is and what it isn’t.” BMJ 312:7023
(1996): 71‐2.
Question revisit
• Question to ponder: Do we give users what they
want or what they need? How does the quality of
resource effect collection development policies?