9.30am Choosing a Method of
Qualitative Synthesis
Mr Andrew Booth,
Reader in Evidence Based Information
Practice, ScHARR, University of Sheffield
Confusing Terminology, Variety of
Choices
• Qualitative Systematic
Review
• Qualitative MetaSynthesis
• Qualitative Research
Synthesis
Qualitative
Evidence
Synthesis
• Qualitative
Interpretive MetaSynthesis
• Best Fit Synthesis
• Critical Interpretive
Synthesis
• Framework Synthesis
• Meta-Aggregation
• Meta-Ethnography
• Meta-Interpretation
• Meta-Narrative
• Meta-Study
• Meta-Summary
• Narrative Synthesis
• Qualitative Meta-Synthesis
• Realist Synthesis
• Thematic Synthesis
Pandora’s Box!
Overview of Approaches
Gough D, Thomas J, Oliver S. Clarifying differences between review
designs and methods. Syst Rev. 2012 Jun 9;1:28. doi: 10.1186/2046-40531-28.
Bad Reasons for Choosing Method
• Frequency of Use of
Method (e.g. MetaEthnography)
• Popularity/”Sexiness” of
Method (e.g. Realist
Synthesis)
• What a friend/ colleague/
mentor has used (once!)
• Bad experiences of
others (may have been
inappropriate!)
Good Reasons for Choosing Method
•
•
•
•
•
•
Type of Question
Extent of Description versus Interpretation
Role of Theory
Type of Data
Intended Output
By Examining Methods Overviews and Published
Examples
Other Considerations:
Methodological Expertise in Team
Available Resources
Is Your Question……
• Fixed? – Pre-defined as a
PICO (PopulationIntervention-ComparisonOutcome) or SPICE
(Setting-PerspectiveInterest, Phenomenon of
– ComparisonEvaluation) – Question is
an “Anchor”
• (e.g. attached to an
Effectiveness review)
• Negotiable? – To be
explored as part of initial
review process –
Becomes clearer as you
examine data (cp.
Grounded theory
approaches) – Question is
a “Compass”
• NB. Even answering a fixed PICO
question for HTAs may require
exploration of phenomenon of
untreated/pretreated condition
(Lorenc et al, 2012)
Will You Describe or Interpret?
All Reviews figure on a
continuum between
Description and
Interpretation
• Description – What
does the data say? –
factual reporting of
“epidemiology” of
studies, themes etc…
• Reader does work of
interpretation
• Interpretation – What
does the data mean? –
“diagnosis” – subjective
interpretation of “signs
and symptoms” from
data and themes etc…
• Reviewer does work of
interpretation – may be
contested
Aggregative, Interpretive or
Integrative?
• Aggregative – (“Every
study counts”) – for
mapping, barriers and
facilitators etc – e.g.
Meta-Aggregation,
Thematic Synthesis etc
• Interpretive – (Different
perspectives, rounded
out understanding) – to
create models, possible
explanations – e.g.
Grounded Theory,
Meta-Ethnography
Integrative – (Relating effectiveness and
acceptability) – to answer what works, for whom,
under what circumstances – e.g. EPPI-Centre
Methods, Narrative Synthesis, Realist Synthesis
Will You Generate, Explore, Test
Theory (Gough et al, 2012)?
• Generate – may require
“suspension of disbelief”
– quality assessment/
value judgement may
come later (cp.
Brainstorming)Grounded Theory, Metaethnography
• Explore – looking for
patterns - Narrative
Synthesis, Thematic
Synthesis
• Test – quality assessment
differentiates wellsupported and
unsupported data Framework Synthesis
(incl. Best Fit Synthesis)
Choice of Synthesis (Adapted from Noyes & Lewin, 2011)
Decision to conduct a qualitative evidence
synthesis
Purpose of the additional
qualitative synthesis
To aggregate/ summarise/
integrate qualitative data to
address questions in relation
to a specific intervention
review
Thematic analysis without
theory generation
Meta-aggregation
Meta-summary
To interpret synthesised
qualitative evidence and develop
explanatory theory or models
Framework
synthesis
Best fit
synthesis
Metaethnography
Grounded theory
Thematic analysis
with theory
generation
Primarily to integrate and
interpret qualitative and
quantitative evidence within a
single approach or integrated
model. Can be used to develop
explanatory theory.
Realist Review
EPPI Approach
Matrix Method
Narrative Synthesis
Bayesian Synthesis
Critical Interpretive synthesis
or
Product: Aggregated findings
from source papers
Product: Explanatory theory, analytical or conceptual framework
or interpretative framework/mechanism
How Rich (“Thick”) is Your Data?
• Qualitative data from
“thin” studies (or textual
responses to surveys) will
not sustain interpretive
approaches
• Limited to MetaAggregation, Thematic
Synthesis, Framework
Synthesis, Narrative
Synthesis –type
approaches
• Rich/“Thick” reports will
sustain MetaEthnography/Grounded
Theory – may allow
selective sampling/
theoretical saturation
• NB. Is “Unit of Analysis”
Individual Study (MetaAggregation, Thematic
Synthesis) or “Body of Evidence”
(e.g. Meta-Narrative or Critical
Interpretive Synthesis
approaches) or even Theory
(Framework Synthesis/Best Fit
Synthesis)?
What is Your Intended Output?
• “output of some methods…
(Thematic Synthesis, textual
Narrative Synthesis, Framework
Synthesis, and ecological
triangulation) is more directly
relevant to policymakers and
designers of interventions
than…methods with a more
constructivist orientation (MetaStudy, Meta-Narrative, MetaEthnography, Grounded Theory,
CIS)…generally more complex
and conceptual” (Barnett-Page &
Thomas, 2009)
• Thematic Synthesis (including
Meta-Aggregation) and
Framework Synthesis produce
findings that directly inform
practitioners (Thomas &
Harden, 2009)
• Interpretive approaches (e.g.
CIS, Meta-Ethnography)
produce a model that requires
practitioners to interpret
relevance and applicability to
their own context
• Narrative Synthesis or EPPICentre (matrix) methods may
help to integrate and present
quantitative/qualitative work
What Expertise Can You Access?
• Expertise in Qualitative Research Methods
(e.g. Grounded Theory; Framework Analysis,
Thematic Analysis)
• Expertise in Synthesis Methods (incl.
Searching, Data Extraction, Quality
Assessment, Interpretation)
• Knowledge of Topic Area
Safest Options!
If…
• There is a Preexisting Theory
or
Framework….
Then
• ….Framework
Synthesis
(including Best
Fit Synthesis)
If…
• There is a
Proximate
(Close-ish!)
Theory or
Framework….
Then
• ….Best Fit
Synthesis
If…
• There is No
Theory or
Framework…
Then
• …Thematic
Synthesis (Can
also act as first
stage of MetaEthnography)
Some Examples
The Scenario
HTA Report – Systematic Review, Modelling &
Qualitative Synthesis on Group Therapy for Postnatal
Depression (UK)
• Stevenson M, Scope A, Sutcliffe P, Booth A, et al. Group
cognitive behavioural therapy for postnatal depression: a
systematic review of clinical effectiveness, cost
effectiveness and value of information analyses. Health
Technol Assess 2010;14(44).
• Scope, A., Booth, A. and Sutcliffe, P. (2012), Women’s
perceptions and experiences of group cognitive behaviour
therapy and other group interventions for postnatal
depression: a qualitative synthesis. Journal of Advanced
Nursing. doi: 10.1111/j.1365-2648.2012.05954
The Choice
• Type of Question – Acceptability of Intervention
• Extent of Description versus Interpretation – Factors Making
Group Therapy More or Less Acceptable (Descriptive)
• Role of Theory – No Theory – Trying to separate Group Effect
from Therapy Effect
• Type of Data – Very Thin Data, Small Number of Studies (n =
6) – Descriptive Case Study Accounts in Nursing Journals
• Intended Output – Alongside Effectiveness Review & Cost
Effectiveness Study – for Implementation
Other Considerations:
Methodological Expertise in Team – Novice Reviewer
Available Resources – Limited Time in Comparison to “Main”
Review
Choice = Thematic Synthesis
The Contribution
• While Group Therapy is Acceptable on
Average there are some for whom it is
unfavourable because:
– Group Comparison – Some People are Getting
Better Quicker than I am
– Group Comparison – Ms. X has been Coming Here
longer than I am and still is not any better
• Identified Key Issue: Identification of those
Most Likely to Benefit.
The Scenario
Aim: To analyse meaning and motivation of the
Wish To Hasten Death in patients with chronic
illness or advanced disease
• Monforte-Royo C, Villavicencio-Chávez C, TomásSábado J, Mahtani-Chugani V, Balaguer A (2012)
What Lies behind the Wish to Hasten Death? A
Systematic Review and Meta-Ethnography from
the Perspective of Patients. PLoS ONE 7(5):
e37117. doi:10.1371/journal.pone.0037117
The Choice
• Type of Question – Explanatory
• Extent of Description versus Interpretation – Factors
that Help to Explain a Phenomenon (Interpretative)
• Role of Theory – Theory Generation
• Type of Data – Contextually Rich – Qualitative Research
Reports – Thick Data [NB. 7 studies]
• Intended Output – Stand Alone Research Report for
Enlightenment
Other Considerations:
Methodological Expertise in Team – Large Experienced
Team with Qualitative Researcher
Available Resources – Extant Funded Project
Choice = Meta-Ethnography
The Contribution
• Wish to hasten death (WTHD) is multifactorial
construct with multiple meanings that do not
necessarily imply genuine desire to hasten one's
death or actually taking steps towards this.
• Wish to hasten death (WTHD) is a phenomenon
that appears, among patients in advanced stages
of illness, as response to extreme suffering that
affects all aspects of their human existence.
The Scenario
Aim: To evaluate benefits and constraints of
collaborative health-related research by researchers
and those affected by the issues under study and/or
those who would apply research results and to
examine how variations in programme context and
mechanisms influence the process and outcomes of
collaborative health intervention research?
Jagosh, J., Macaulay, A. C., Pluye, P., Salsberg, J., Bush, P. L.,
Henderson, J., ... & Greenhalgh, T. (2012). Uncovering the
benefits of participatory research: implications of a realist
review for health research and practice.Milbank
Quarterly, 90(2), 311-346.
The Choice
• Type of Question – Explanatory
• Extent of Description versus Interpretation – Factors that
Help to Explain Variation in Outcomes
• Role of Theory – Theory Exploration
• Type of Data – Contextually Rich – Quantitative and
Qualitative Research Reports – Thick Data [NB. Twentythree PR partnerships described in 276 publications.]
• Intended Output – Explanatory Report for Multiple Funders
Other Considerations:
Methodological Expertise in Team – Large Experienced
International Team of Experienced Researchers
Available Resources – Extant Funded Project
Choice = Realist Synthesis
The Contribution
• Used middle-range theory of partnership synergy to
demonstrate how PR can (1) ensure culturally and
logistically appropriate research, (2) enhance
recruitment capacity, (3) generate professional capacity
and competence in stakeholder groups, (4) result in
productive conflicts followed by useful negotiation, (5)
increase quality of outputs and outcomes over time,
(6) increase sustainability of project goals, and (7)
create system changes and new unanticipated projects
and activities.
• Generated new insights into benefits of PR regarding
conflicts and negotiation between stakeholders,
program sustainability and advancement,
unanticipated project activity, and generation of
systemic change.
References - 1
• Barnett-Page E, Thomas J.
Methods for the synthesis of
qualitative research: a critical
review. BMC Med Res
Methodol. 2009 Aug 11;9:59.
• Booth, A, Papaioannou, D and
Sutton, A J (2011). Systematic
Approaches to a Successful
Literature Review. SAGE
publications
• Candy B, King M, Jones L,
Oliver S. Using qualitative
synthesis to explore
heterogeneity of complex
interventions. BMC Med Res
Methodol. 2011 Aug
26;11:124.
• Dixon-Woods M, Agarwal S,
Young B, Jones D, Sutton A.
(2004) Integrative approaches
to qualitative and quantitative
evidence. London: Health
Development Agency
• Gough, D, Oliver, S, Thomas J
(2012) An Introduction to
Systematic Reviews. London:
Sage Publications.
• Lorenc, T., Pearson, M., Jamal,
F., Cooper, C. and Garside, R.
(2012), The role of systematic
reviews of qualitative evidence
in evaluating interventions: a
case study. Res. Synth.
Method, 3: 1–10.
References - 2
• Popay J, Roberts H, Sowden A,
Pettticrew M, Arai L, Rodgers
M, Britten N: Guidance on the
conduct of narrative synthesis
in systematic reviews.
http://www.lancs.ac.uk/fass/p
rojects/nssr/2007 .
• Pope C, Mays N, Popay
J: Synthesizing Qualitative and
Quantitative Health Evidence:
a Guide to Methods.
Maidenhead: Open University
Press; 2007.
• Ring N., Ritchie K, Mandava L,
Jepson R. (2011) A guide to
synthesising qualitative
research for researchers
undertaking health technology
assessment and systematic
reviews. NHS Quality
Improvement Scotland and
University of Stirling,
Edinburgh.
• Thomas J, Harden A (2009)
Methods for the thematic
synthesis of qualitative
research in systematic reviews,
BMC Medical Research
Methodology 8:45