Janet Harris - Updated material from Angela Harden, Three-day systematic review
workshop, K U Leuven, 6th to 8th May 2011
WHY DEVELOP A PROTOCOL?
Helps to plan the review and anticipate problems
Encourages comment and review
Aids transparency and audit
Can build in flexibility
Required by review organisations like Cochrane and
Campbell
ELEMENTS OF A REVIEW PROTOCOL
Title
Background
Review question/objectives
Inclusion and exclusion criteria
Search strategy
Screening
Quality assessment
Coding and data extraction
Synthesis methods
Conflicts of interest
EXAMPLE OF A REVIEW PROTOCOL THAT INCLUDES
QUALITATIVE RESEARCH
http://onlinelibrary.wiley.com/o/cochrane/clsysrev/articles/CD009105/frame.html
Leiknes KA, Berg RC, Smedslund G, Jarosch-von
Schweder L, Øverland S, Hammerstrøm KT, Høie B.
Electroconvulsive therapy for depression (Protocol).
Cochrane Database of Systematic Reviews 2011,
Issue 5. Art. No.: CD009105. DOI:
10.1002/14651858.CD009105.
REVIEW QUESTIONS
PICO
SPICE
•
Population
•
Setting (where? in what context?
•
Intervention
•
Perspective (who?)
•
Comparison
•
•
Outcome
Intervention (or phenomenon of
interest)
•
Comparison (what else?)
•
Evaluation (how well – what
result?
SEARCHING SYSTEMATICALLY
Common principles and challenges
Approaches to searching
Search sources
Bibliographic databases
Filters for electronic searching
Summary of key points
BUT first……..
SEARCHING IS A SKILLED
AND TIME CONSUMING ACTIVITY
Tap into the skills of information, subject and systematic
review specialists, to:
 Identify sources to search
 Deal with technicalities (e.g. of databases)
Plan time for developing and testing searches
COMMON PRINCIPLES

Searching is the act of seeking studies that might be
relevant for the review’s question

The search strategy is guided by the review question(s)

Practically constrained (by database limitations, restricted
time and resources available to reviewers)

As in other stages, reviewers:

Have a rationale for their searching methods:

Draft, test and implement a structured search plan (‘search
strategy’);

Report fully on their methods and findings.
COMMON CHALLENGES
•
Profusion of published and unpublished material
•
Much hidden: only 50% abstracts presented at
conferences are later published in full
•
Different databases use different terms to classify
studies
•
Different databases cover different, largely discrete
areas of literature
•
As individuals we are constrained by our own
disciplinary and policy area knowledge.
APPROACHES TO SEARCHING
Comprehensive searching

Considers ideal as having access to all studies that
answer the review question. An unbiased sample is next
best.
Purposive searching

E.g. to identify main themes in the literature

Searching plans may develop as evidence comes to light.
SOURCES OF QUALITATIVE RESEARCH (1)
Qualitative research may be published as:
 Journal articles
 Books and book chapters
 Conference papers
 Project reports
 Dissertations
 Any more formats?
SOURCES OF QUALITATIVE RESEARCH (2)
Qualitative research may be found through:
 Bibliographic databases
 Specialist research registers
 Library catalogues
 Conference proceedings
 Personal contacts
 Handsearching journals
 Websites
 Search engines (Google scholar)
 Reference lists of relevant studies
BIBLIOGRAPHIC DATABASES
Subject specific
 E.g. Medline, Embase, CINAHL, ERIC
Multiple disciplines
 E.g. Social Science Citation Index, ASSIA
Format specific
 Dissertation abstracts, Index to theses
SEARCHING BIBLIOGRAPHIC DATABASES
Two main approaches:
 Topic only search
 Topic plus qualitative filter
Two types of search terms
 Controlled vocabulary
 Free text
A NOTE ON TERMINOLOGY
• No agreed definition of qualitative research
• Houses a broad range of methodologies and methods
• Indexing on databases is inconsistent
• Titles and abstracts may not employ the word
‘qualitative’ but may use more specific terms
 E.g. focus groups, interviews, grounded theory, discourse
analysis, phenomenology, Glaser and Strauss, Nvivo, Nudist,
views, perspectives, experiences, stories, narrative, themes,
findings
DEFINITIONS RANGE FROM RELATIVELY SIMPLE…
“Research that derives data from observation, interviews
or verbal interactions and focuses on the meanings
and interpretations of participants”
Holloway and Wheeler (1995)
Holloway W, Wheeler S (1995) Ethical issues in qualitative nursing research, Nursing Ethics,
2:223-232
….TO ELABORATE AND COMPLEX…
Qualitative research is an umbrella term for an array of attitudes toward
and strategies for conducting inquiry that are aimed at discerning
how human beings understand, experience, interpret and produce
the social world…it encompasses richly detailed descriptions and indepth, particularised interpretations of persons and the social,
linguistic, material and other practices and events that shape their
lives and are shaped by them. Qualitative research typically includes,
but is not limited to, discerning the perspectives of….the
actors…point of view. Although both philosophically and
methodologically a highly diverse entity, qualitative research is
marked by certain defining imperatives that include a case
orientation…to analysis, sensitivity to cultural and historical context,
and reflexive accounting practices to optimise validity…..
Sandelowski M (2003) Qualitative research. In M Lewis-Beck, AE Bryman & TF Liao (Eds) The
Sage Encyclopedia of Social Science Research Methods. Thousand Oaks, Ca: Sage.
OPTIMAL SEARCH FILTERS
• Aim to balance sensitivity and specificity
• Some databases may have built in filters
• Information specialists have developed and tested
filters for others
• Lots of room for further development and evaluation build this into your reviews
OPTIMAL QUALITATIVE FILTER FOR CINAHL
Various combinations of the following tested:
 interviews.sh; interview.tw; attitude.sh; qualitative studies.sh;
qualitative stu$.mp; thematic analysis.sh; audiorecording.sh;
grounded theory.sh; study design.sh
Optimal combination:
 Interview.tw OR audiorecording.sh OR qualitative stu$.mp
sh = subject heading
tw= textword - word or phrase in title or abstract
mp = multiple posting – word or phrase in title, abstract
of subject heading
Wilczynski et al. (2007) Search strategies for identifying qualitative studies in CINAHL,
Qualitative Health Research 17:705-710.
EVALUATION OF THREE STRATEGIES ON SIX*
DATABASES
Strategy
No. of records
identified
No. relevant
to review
Controlled vocabulary
e.g. in Medline ‘Qualitative Research’,
‘Nursing Methodology Research’
3537
191
Free-text terms
40 plus commonly used terms e.g.
‘ethnograph$’, ‘lived experience$’,
‘grounded theory’
3451
172
Broad-based terms
3 free text terms ‘qualitative’, ‘findings’,
‘interview$’,
Controlled vocab term ‘Interviews’
3912
187
TOTAL
7420
262
*MEDLINE, EMBASE, CINAHL, British Nursing Index, ASSIA and SSCI
Shaw et al. (2004) Finding qualitative research: an evaluation of search strategies, BMC
Medical Research Methodology 4:5
KEY POINTS FOR DEVELOPING YOUR SEARCH
STRATEGY
• Seek the input of an information specialist asap.
• Undertake some initial scoping and searching.
• Identify relevant sources.
• Use a methods filter if your review only focuses on
qualitative research.
• Document each part of your search and its results.
• Invest in good information management systems and
people.
FURTHER READING
Sandelowski M, Barroso J (2007) Handbook for synthesising qualitative
research. New York: Springer (see chp 3)
Flemming K, Briggs M (2006) Electronic searching to locate qualitative
research: evaluation of three strategies. Journal of Advanced Nursing,
57: 95-100.
Stansfield C et al. (2010) Search wide and dig deep: Identifying ‘views’
research for systematic reviews. Paper presented at the Joint
Colloquium of the Cochrane and Campbell Collaboration, Keystone,
Colorado, USA, 18th to 22nd October.
Stansfield C et al. (forthcoming, 2012) Finding relevant studies. In Gough D,
Oliver S, Thomas J (Eds) Systematic Reviews in the Social Sciences.
London: Sage.