Literature review: How to search, evaluate,
synthesize, and present the evidence
This workshop will focus on…
1. Searching through the literature for
relevant sources;
2. Evaluating the findings from your
search;
3. Synthesizing these findings; and
4. Presenting the results.
But WHY do we need to conduct a
literature review?
A literature review can help us to…
• Address an information need
• Find out whether the evaluation question has been
asked before
• Find out what has been done in similar settings
• Determine what best practices already exist
• Discover what tools might be useful in our evaluation
What is the problem we need to
solve?
The Evidence
to Practice Process
What is the question we need
answered?
Select first choice resource
where an answer may be found
(e.g. the Cochrane library)
Select second
choice resource
Design search strategy and carry
out search
Good Evidence
Appraise and summarize the
evidence
Poor or insufficient
evidence
Assess the relevance of the
evidence to your local situation
and target groups
Consider implications for your
practice and resources, and
those of the organizations you
work with
Improvement
Apply the evidence to
practice
Evaluate your
practice
No improvement
© 2003, Evidence Network: What Works for Children?
Where do we start?
Searching through the Literature
Define the Problem
1)
The target population
E.g., Age, sex, ethnic group, diagnostic group
2)
The intervention
The therapy that is of interest to your program
3)
The outcome
The changes you would like to see in your target
population
Find the Information
•
•
Academic libraries
Online databases
•
Primary source: The original publication of new data, results, and
theories
Stice, E., Shaw, H., Burton, E., & Wade, E. (2006, April). Dissonance and
healthy weight eating disorder prevention programs: a randomized efficacy
trial. Journal of Consulting and Clinical Psychology, 74(2), 263-275.
Retrieved September 18, 2008, from PubMed database.
•
Secondary source: Summarizes or comments on primary sources in
the context of the particular idea under study
Pratt, B.M. & Woolfenden, S.R. (2002, April). Interventions for preventing
eating disorders in children and adolescents. Cochrane Database of
Systematic Reviews, Issue 2. Retrieved September 18, 2008, from the
Cochrane database.
Online databases with free content…
•
Centre for Reviews and Dissemination (CRD) http://www.crd.york.ac.uk/crdweb/
•
Cochrane Database of Systematic Reviews (CDSR) http://www.cochrane.org/
•
Turning Research Into Practice (TRIP) Database http://www.tripdatabase.com
•
The International Network of Agencies for Health Technology Assessment
http://www.inahta.org/
•
British Medical Journal www.bmj.com/
•
Directory of Open Access Journals (DOAJ) http://www.doaj.org/
•
Substance Abuse and Mental Health Services Administration (SAMHSA)
http://www.samhsa.gov/ebpWebguide/index.asp
•
Centre of Excellence evidence database http://www.onthepoint.ca/EvidenceDB/
Online databases with limited free content…
•
PubMed http://www.ncbi.nlm.nih.gov/pubmed/
•
York University Health Research Guide
http://www.library.yorku.ca/ccm/rg/nk/health.jsp
•
British Medical Association (BMA)
http://www.bma.org.uk/ap.nsf/Content/Evidencebased
healthinformationon
Databases available only by subscription…
•
PsycINFO – American Psychological Association
(APA) http://www.apa.org/psycinfo/
•
Allied and Complementary Medicine Database
(AMED) http://www.bl.uk/collections/health/amed.html
•
EMBASE http://www.embase.com/
Guidelines available online…
•
Guidelines are a rich source of evaluated evidence,
particularly if the question is about treatment or
diagnosis of a relatively common medical condition
Available for free:
•
•
•
•
•
UK National Electronic Library for Health Guidelines Finder
http://rms.nelh.nhs.uk/guidelinesfinder
US National Guideline Clearing House at
http://www.guidelines.gov
Guidelines International Network (GIN) at http://www.g-i-n.net
BMJ Publishing Group’s Clinical Evidence
http://www.clinicalevidence.org/ceweb/conditions/index.jsp
Search through the Database
1. Use key words from your question as search
terms
2. Use restrictions to refine the search
•
A clearly defined patient group and intervention are
the major parameters in most subject searches
3. Search by author, journal title etc.
Approaches to research…
Qualitative
•
•
•
•
In-depth information
about people’s
experiences
Focus on subjective
meanings
Typically small samples
Emphasis on credibility
and trustworthiness
Quantitative
• Information about how
different variables are
related to one another
• Focus on objective
measurement
• Typically large samples
• Emphasis on causality,
reliability, validity
Types of studies you may find…
1.
2.
3.
4.
5.
6.
7.
8.
Systematic reviews
Randomised controlled trials
Quasi-experimental design
Evaluation studies with non-experimental designs
Case control studies
Cohort studies
Population surveys
Qualitative research
Evaluating the Results
Initial questions to consider…
•
•
•
•
•
Do the title and abstract suggest a fit
between the source and your evaluation?
Are the articles peer reviewed?
Is the study original?
Who is the study about? Is it reasonable to
expect that the results might apply to your
target group?
Are the claims made by the study
plausible?
• Have the authors addressed all outcomes of
interest?
• What does the study add to what we already
know?
• The authors’ credentials: are the authors
associated with the field of study? Do they
have relevant clinical experience?
• Are there any issues related to “researcher
bias” that are not addressed?
The Purpose and Method
•
•
•
•
•
Are the research questions clear, specific and
answerable?
Is the study design appropriate? Do the question,
method and analysis of results match up?
Is the sample appropriate?
If a comparison group was used, was it really
comparable to the group receiving the
intervention?
Did some people 'drop out' of the study, and if so,
have the authors accounted for this in their
conclusions?
Results and Conclusions
•
•
•
•
•
•
Is the material presented in a way that is
transparent and detailed in a way that can be
easily examined and evaluated?
How large is the effect of the intervention, if there
is one?
How precise is the estimate of the effect? How
likely is it that the result was due to chance?
Have the authors clearly shown how they came to
their findings?
Does the author explain study limitations?
Do the conclusions match the findings? Are the
conclusions supported by the analysis?
Synthesizing the Findings
Initial questions to consider…
•
•
•
•
•
•
What are the most relevant sources?
What are some common elements across sources?
What is unique about each source?
What are the key ideas/concepts being conveyed?
Why would this be important?
What are the limitations/gaps in the literature?
(Note: this is particularly important, because it speaks
to your contribution to the literature – i.e., your work
may help fill this gap)
Develop a Summary Table
Title and
Publication
Information
Type of
Source and
Approach
Overall Goal
Main Ideas
and
Conclusion
Limitations
Presenting the Findings
Planning the Literature Review
•
•
•
•
What are the goals of the review?
Who is the audience?
What are your key messages?
How should the review be organized?
Tips for Writing the Literature Review
•
•
•
•
•
•
Use the right word in the right place
Avoid awkward, rambling and run-on
sentences
Avoid sentence fragments
Use clear, plain language
Always use examples or statements to
substantiate your point
Feel free to use an active voice
Next Steps
•
•
•
Selecting measures?
Framework design?
Other ideas?
Visit our website for more
information:
www.onthepoint.ca