Systematic Approaches to
Literature Reviewing
Presenter:
Maeve Gallagher
Adapted from presentation by Dr. Derek Richards &
Dr Mark Mathews
Workshop Overview

Explain elements of the systematic review process
Challenges in Conducting
Literature Reviews

Knowing where to start, (e.g. wide then narrow,
or narrow then wide?)

Knowing what to include

Knowing when to stop literature searching

Knowing how far to venture into the literature of
associated domains
Literature Review
What is a Systematic Literature Review?
Systematic Reviews
A review of a clearly formulated question that uses
systematic and explicit methods to identify, select
and critically appraise relevant research.
from www.sebc.bangor.ac.uk
Systematic Reviews
Readers need to be able to judge whether all of the
relevant literature is likely to have been found, and
how the quality of studies was assessed.
Key features
The systematic review process:

Explicit and transparent methods

A standard set of stages

Accountable, replicable and updateable
To summarise …
You need:

1. A clearly formulated question

2. Systematic and explicit methods

3. Critical appraisal of relevant research
1. Formulating a Problem
A Research Question should be:
- Specific
- Focused
- Clearly formulated
- Well defined
Ask questions of the question!
1. Formulating a Question - Discuss
following questions in pairs/threes
1.
Why are social networking sites harmful?
2.
How are online users experiencing or
addressing privacy issues on such social
networking sites as MySpace and Facebook?
3.
What is the effect on the environment from
global warming?
4.
How is glacial melting affecting penguins in
Antarctica?
Research Question Examples
Why are social networking sites harmful?




Doesn’t specify which social networking sites
What kind of harm the sites are causing.
Assumes that this “harm” is proven and/or accepted.
http://writingcenter.gmu.edu/?p=307
Research Question Examples
How are online users experiencing or addressing privacy
issues on such social networking sites as MySpace and
Facebook?



Specifies sites (MySpace and Facebook),
Type of harm (privacy issues)
Who the issue is harming (users).
http://writingcenter.gmu.edu/?p=307

A strong research question should never
leave room for ambiguity or interpretation.
Research Questions Examples
What is the effect on the environment from
global warming?
So broad that it couldn’t be adequately answered
in a book-length piece, let alone a standard
college-level paper.
http://writingcenter.gmu.edu/?p=307
Research Question Examples
How is glacial melting affecting penguins in
Antarctica?




narrows down to a specific cause (glacial melting),
a specific place (Antarctica)
a specific group that is affected (penguins).
http://writingcenter.gmu.edu/?p=307


When in doubt, make a research question
as narrow and focused as possible
http://writingcenter.gmu.edu/?p=307
1. Formulating a Problem
What is your research question?
Take a few moments to write down your
research question.
1. Formulating a Problem
Is Your Research Question:
- Specific?
- Focused?
- Clearly formulated?
- Well defined?
Ask questions of your question!

Exercise to do in pairs:
Take five minutes each to discuss your
research topic. Ask your partner to
interrogate your question to see if it meets all
the criteria mentioned earlier
Online students can take the time to objectively
interrogate their own question.
1.
Formulate question
Computer-based psychological treatments for
depression:
A systematic review
2. Locating and Finding Studies
2. Locating and Finding Studies
Phase 1 – Identifying the Research
 Phase 2 - Selection

Phase 1- Identify the Research
a broad but defined, systematic sweep
1. Defined search terms
2. Defined search arena
3. Other broad search limits, e.g.
language, date,
TIPS!
Document the
search protocol
and record what
research was
found
Systematically
manage the
search output,
e.g. using
endnote
Phase 1- Identify the Research
1. Defined search terms





Online self-help treatment for depression,
Web-based intervention for depression,
Online depression treatments,
Computerized (+Computerised) cognitive
behaviour therapy for depression,
Internet (+delivered) treatment for depression.
Phase 1- Identify the Research
2. Defined search arena









Databases
citation indices
reference lists from primary and review articles
grey literature
conference proceedings
research registers
the internet
individual researchers/practitioners
A search of three databases (EMBASE, PubMed, and PsychINFO
including PsychARTICLES)
Phase 1- Identify the Research
3. Other broad search limits, e.g. language,
date:
 Studies published in English in peer-reviewed
journals in the last 10 years (March 2001–March
2011)
 The authors decided that the years represented a
meaningful timeframe in terms of:
contemporary technologies
advances in multimedia
broadband developments
Phase 2 – Selection
Inclusion and Exclusion Criteria - related to
your research question
Develop inclusion or exclusion statements, these might
relate to study outcomes, research design, methods used,
population worked with etc.
e.g.
 studies with a mixed population of men and women,
 only random control trials,
 maximum exposure time of 10mins
Phase 2 – Selection
Inclusion and Exclusion Criteria - related to
your research question

Included studies using a variety of different computer-based
technologies

Study participants had to be adults (18+years) with depression (selfreport or diagnosis), established using valid and reliable measures

Studies included were published in peer reviewed journals in English in
the last 10 years

Participants could be from the general population or a clinical group so
long as depression was specifically measured.

Preliminary research into recent developments in computerized
paradigms for depression such as cognitive bias modification (CBM)
based interventions were not considered for inclusion
3. Critical appraisal of studies
“Assessing the quality of methodology is
a critical part of the systematic review process”
You will need to make decisions on all aspects of the
review and simply justify what you decide, making
sure its credible and meets the criteria, that it is
transparent, replicable and updateable.
3. Critical appraisal of studies
Computer-based psychological treatments for depression:
•Programs and their content
•Methodological characteristics
•Objectives of the studies
•Recruitment, sample types and sizes
•Eligibility criteria employed
•Outcome measures used
•Support type and communication mode in the studies
•Outcomes, support types and dropout
•Satisfaction
•Limitations
3. Critical appraisal of studies
Model to Generate Critical Thinking
Description
Who? Where?
When?
How?
What?
Topic
Why?
What next?
So what?
Evaluation
Analysis
What if?
Learning Development University of Plymouth
Comprehensive Literature Review
Do parallel
literatures exist
for this topic?
In which
subject areas
has the topic
been studied?
What are the key
concepts in this area?
How is this topic
approached by
others?
Who are
these
“others”?
Which
discussions?
Which aspects of this
work are of most
relevance to my
study?
Which subthemes?
What are the
main
perspectives
on this topic in
previous
research?
Coherent synthesis
of past and present
research in the
domain of study
What have been the
main research
questions?
What are the main
conclusions on
previous research in
this area?
Where are the gaps in
literature?
Which existing
work could be
extended?
Where is existing
knowledge “thin”?
What are the key
areas of debate in this
area?
Which work is
subject to
challenge?
Which writers?
Source: Dr Hazel Hall, Napier University
Write up literature review - Structure
Background
Purpose/Research question
Method
Findings
Discussion
Implications/Recommendations
Student Learning
Development, TCD
33
Other ideas and options
Speed reading
Endnote
Databases in my area
Data mining techniques
Recording
SR websites
Centre for Evidence-based Conservation -http://www.cebc.bangor.ac.uk/
Centre for Reviews and Dissemination (medical) -http://www.york.ac.uk/inst/crd/index.htm
Cochrane Collaboration (international- medical) -http://www.cochrane.org/
EPPI-Centre, Institute of Education - http://eppi.ioe.ac.uk/cms/
Social Policy and Social Care -http://www.york.ac.uk/inst/chp/srspsc/index.htm
If you read one article, an example here of a review of the ways studies in reviews
are appraised - http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=521688
If you prefer power-point, how about this one on mixed method reviews
http://www.ccsr.ac.uk/methods/events/challenges/documents/JamesThomasESRC
Methodologicalchallenges.ppt