Systematic Review of the
Literature: A Novel Research
Approach
Objectives:

By the end of the session, the student will be
able to:
1. State the definition of systematic review
2. State the reasons for doing a systematic review
3. State when a systematic review is needed
4. Know the components of a systematic review
5. Know the process for a systematic review
6. Know how the systematic review process fits into
the 5 chapter dissertation format
What is a Systematic Review?


A systematic review is a rigorous and
comprehensive literature review led via a
peer-reviewed protocol making it possible to
replicate the findings.
“A review of a clearly formulated question that uses systematic
and explicit methods to identify, select, and critically appraise
relevant research, and to collect and analyze data from the
studies that are included in the review. Statistical methods
(meta-analysis) may or may not be used to analyze the results
of the included studies” (Cochrane, 2005, p. 45).
…continued


“Systematic reviews attempt to bring
the same level of rigor to reviewing
research evidence as should be used in
producing that evidence in the first
place” (Davies & Crombie, 2009, p. 1).
“The author treats the review as the
research process” (source?)
Why Do a Systematic Review?



Knowledge explosion makes keeping up with
primary research evidence impossible.
Practitioners have wide-ranging information
needs (effectiveness, meaningfulness,
feasibility, appropriateness).
Looked at individually, research studies may
offer little insight to problems; taken together
within a systematic review, a clearer and
more consistent picture will emerge.
(Davies & Crombie, 2009)
When is a Systematic Review Needed?

“….systematic reviews are most needed
whenever there is a substantive
question, several primary studies perhaps with disparate findings - and
substantial uncertainty” (Davies &
Crombie, 2009, p.3).
Components of Systematic Review





Identification of all relevant published and
unpublished evidence
Selection of studies or reports for
inclusion.
Assessment of quality of each study or
report
Synthesis of findings from the individual
studies or reports in an unbiased way
Synthesis of aggregated findings
The Systematic Review Process:
Step 1: Defining the Questions





Define the area of interest and relevant
populations.
State clear and appropriate questions.
Clearly state the objectives of the
review.
Identify the types of evidence or studies
that will help answer the questions.
State intended outcomes.
It has been said that “…the quality of the problem that
is formulated is the forerunner of the quality of the
solution that will be attained” (Getzels, as cited in
Achilles, Reynolds, & Achilles, 1997, p. 31).
In doing a systematic review, it can be said that the
quality of the question(s) that is formulated is the
forerunner of the quality of the review.
Let’s think about the characteristics of good questions
and then look at some examples.
Characteristics of Good Questions





Clear and unambiguous
Answerable
Relevant
Positively worded
Free of bias
Shouldn’t hospitals
refuse to allow smoking
anywhere on their
property, since
everybody knows that
smoking causes cancer
and heart disease?
Should hospitals refuse to allow
smoking anywhere on their
property, since everybody
knows that smoking causes
cancer and heart disease?”
Should hospitals refuse
to allow smoking within
public view?
Should hospitals offer
stop-smoking programs
to help citizens stop
smoking?
How do changes in
smoking policies of
hospitals affect smoking
rates?
What group of strategies is
associated with smoking
cessation?
The Systematic Review Process:
Step 2: Searching the Literature





Ideally, all of the published and unpublished
literature should be searched for an unbiased review.
In reality, a designated number of databases are
searched using a standardized or customized filter.
“Gray literature” (Davies and Crombie, 2009, p.4) is
searched using specialized search engines, databases,
and websites.
Expert opinion on where appropriate data may be
located is sought.
Key authors are contacted for clarification.
(Davies & Crombie, 2009)
Determine whether:


There is sufficient research to
accomplish a systematic review
A systematic review is necessary, i.e.,
there is not already a recent, relevant
systematic review
Next…



Access library databases
Select appropriate database(s), given
your question(s)
Set limits - e.g. date of publication,
human focus, language, abstracts
The Systematic Review Process:
Step 3: Assessing the Studies





Assess each study for eligibility against inclusion
criteria
Retrieve the full text of studies that meet the
inclusion criteria
Assess studies for methodological quality (ideally
conducted by two independent reviewers)
Extract findings onto a data extraction form (some
studies may be excluded even at this late stage)
Create list of included studies
(Davies & Crombie, 2009)
Screening for Inclusion

Screen all references for EACH criterion you
have established. For example:
Empirical
IV = smoking cessation strategy
DV = smoking rates (smoking cessation rates)
Relevant population
Published after 2002
In order for references to be included, all
criteria must be met.
Retrieving the Papers

Access online journals

Access hard-copy publications

Keep track of references
Assess Quality

Be fair minded. Look for strengths and
weaknesses.

Look for internal consistency.

Be thoughtful.
Extract Findings

Look for common themes.

Create list of included studies
The Systematic Review Process:
Step 4: Combining the Results


Findings from individual studies must be
aggregated. This is called “evidence
synthesis” (Davies & Crombie, 2009, p.4)
Type if evidence synthesis is selected to fit
the type of data in the review:
a. Meta-synthesis: qualitative data
b. Meta-analysis: homogenous quantitative data
c. Narrative summaries: non-homogenous
quantitative data
The Systematic Review Process: Step
5: Placing the Findings in Context

Discuss issues such as:
Quality and heterogeneity of studies
Likely impact of bias
Likely impact of chance
Applicability of findings
Systematic Review Proposal



Chapter 1: Introduction (Step 1)
Chapter 2: Literature Search (Describe
how Step 2 will be implemented.)
Chapter 3: Methodology (Describe how
Step 3 will be implemented.)
Systematic Review Dissertation





Chapter 1: Introduction (Step 1)
Chapter 2: Literature Search (Describe
the results of implementing Step 2.)
Chapter 3: Methodology: (Describe the
results of implementing Step 3.)
Chapter 4: Results (Step 4)
Chapter 5: Discussion (Step 5)
A Word of Caution…




Systematic review is far from easy
Requires careful scientific consideration
Requires meticulous and laborious
searching
Requires considerable attention to
methodological detail and analysis
(Davies & Crombie, 2009, p. 5)
Systematic Review Links


http://www.york.ac.uk/inst/crd/
CRD (Centre for Reviews and Dissemination) is part of the
National Institute for Health Research (NIHR) and is a
department of the University of York (United Kingdom). Their
website includes databases where you can search for systematic
reviews on health and social care.
http://www.thecochranelibrary.com
The Cochrane Library provides online access to Cochrane
reviews and also has a "Learn" section with various materials to
help you to develop a systematic review.
Systematic Review Links
(cont’d)


http://eppi.ioe.ac.uk/cms/
The Evidence for Policy and Practice Information and Coordinating Centre (EPPI-Centre) is part of the Social Science
Research Unit at the Institute of Education, University of
London. Their website includes databases where you can search
for systematic reviews as well as a methods and tools section
where you can find data reporting and management tools for
use in systematic reviews.
http://www.prisma-statement.org/
PRISMA stands for Preferred Reporting Items for Systematic
Reviews and Meta-Analyses. It is an evidence-based checklist of
items for inclusion in systematic reviews and meta-analyses.
A Final Word


This has been an overview.
Should you decide to pursue a
systematic review of the literature for
your dissertation, you will need to work
closely with your chair to select/develop
appropriate protocol and data
extraction forms.