Choosing a Method of
Qualitative Synthesis
Dr Andrew Booth
Caveat Reviewer: Pandora’s Box!
Confusing Terminology, Variety of
Choices
1.
2.
3.
Qualitative Systematic
Review
Qualitative MetaSynthesis
Qualitative Research
Synthesis
4.
5.
Qualitative
Evidence
Synthesis
Qualitative
Interpretive MetaSynthesis
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
Best Fit Synthesis
Critical Interpretive Synthesis
Framework Synthesis
Meta-Aggregation
Meta-Ethnography
Meta-Interpretation
Meta-Narrative
Meta-Study
Meta-Summary
Narrative Synthesis
Qualitative Meta-Synthesis
Realist Synthesis
19. Rapid Realist Synthesis
18. Thematic Synthesis
Decisions, Decisions!
• Novice – Bewildering variety of methods of
synthesis – compounded by choice of checklists,
conflicting guidance etcetera
• Experienced - Many authors stick to familiar
methods rather than select most appropriate
method to address question and type of evidence
• Much description of methods, little evaluation
and critique
• Much hiding behind “labels”, misuse of methods
Caveat Reviewer:
• TREAD Carefully
– Time/Timeframe
– Resources
– Expertise
– Audience & Purpose
– Data – Quality, Quantity, Thickness,
Richness
Time/Timeframe
• Most Time Consuming
– Meta-ethnography –
interpretive approach
– Realist Synthesis –
Identification of theory
and development of
programme theories
• Least Time Consuming
– Meta-Aggregation –
aggregating existing
themes
– Thematic Synthesis –
inductively coding
themes
– Framework Synthesis
• Best Fit Framework
Synthesis
“Framework-based synthesis is an important advance in conducting reviews of qualitative synthesis. The
'best fit' strategy is a variant of this approach that may be very helpful when policymakers,
practitioners or other decision makers need answers quickly, and are able to tolerate some ambiguity
about whether the answer is the very best that could be given”. Dixon-Woods M. Using frameworkbased synthesis for conducting reviews of qualitative studies. BMC Med. 2011 Apr 14;9:39.
Seven steps for meta-ethnography
Process of Realist Synthesis
from:
McCormack, B., Wright, J., Dewar,
B., Harvey, G., & Ballantine, K.
(2007). A realist synthesis of
evidence relating to practice
development: findings from the
literature analysis. Practice
Development in Health Care, 6(1),
25-55.
Resources
• Most Resource
Intensive
• Least Resource
Intensive
Expertise
• Expertise in Qualitative Research Methods
(e.g. Grounded Theory; Framework Analysis,
Thematic Analysis)
• Expertise in Synthesis Methods (incl.
Searching, Data Extraction, Quality
Assessment, Interpretation)
• Knowledge of Topic Area
Audience & Purpose
• “output of some methods…
(Thematic Synthesis, textual
Narrative Synthesis,
Framework Synthesis, and
ecological triangulation) is
more directly relevant to
policymakers and designers of
interventions than…methods
with a more constructivist
orientation (Meta-Study,
Meta-Narrative, MetaEthnography, Grounded
Theory, CIS)…generally more
complex and conceptual”
(Barnett-Page & Thomas,
2009)
• Thematic Synthesis (including
Meta-Aggregation) and
Framework Synthesis produce
findings that directly inform
practitioners (Thomas &
Harden, 2009)
• Interpretive approaches (e.g.
CIS, Meta-Ethnography)
produce a model that requires
practitioners to interpret
relevance and applicability to
their own context
• Narrative Synthesis or EPPICentre (matrix) methods may
help to integrate and present
quantitative/qualitative work
Data – Quality, Quantity, Thickness,
Richness
• Rich, Thick Data
• Meta-ethnography
characterised by data of
high methodological
quality, rich data and
systematic presentation.
• Rich/“Thick” reports
sustain MetaEthnography/Grounded
Theory – may allow
selective sampling/
theoretical saturation
• Poor, Thin Data
• Qualitative data from
“thin” studies (or textual
responses to surveys) will
not sustain interpretive
approaches
• Limited to MetaAggregation, Thematic
Synthesis, Framework
Synthesis, Meta-Summary
Narrative Synthesis –type
approaches
NB. Is “Unit of Analysis” Individual Study (Meta-Aggregation, Thematic Synthesis) or
“Body of Evidence” (e.g. Meta-Narrative or Critical Interpretive Synthesis approaches) or
even Theory (Framework Synthesis/Best Fit Synthesis)?
Overview of Approaches
Gough D, Thomas J, Oliver S. Clarifying differences between review
designs and methods. Syst Rev. 2012 Jun 9;1:28. doi: 10.1186/2046-40531-28.
Choice of Synthesis (Adapted from Noyes & Lewin, 2011)
Decision to conduct a qualitative evidence
synthesis
Purpose of the additional
qualitative synthesis
To aggregate/ summarise/
integrate qualitative data to
address questions in relation
to a specific intervention
review
Thematic analysis without
theory generation
Meta-aggregation
Meta-summary
To interpret synthesised
qualitative evidence and develop
explanatory theory or models
Framework
synthesis
Best fit
synthesis
Metaethnography
Grounded theory
Thematic analysis
with theory
generation
Primarily to integrate and
interpret qualitative and
quantitative evidence within a
single approach or integrated
model. Can be used to develop
explanatory theory.
Realist Review
EPPI Approach
Matrix Method
Narrative Synthesis
Bayesian Synthesis
Critical Interpretive synthesis
or
Product: Aggregated findings
from source papers
Product: Explanatory theory, analytical or conceptual framework
or interpretative framework/mechanism
Some Examples
The Scenario
HTA Report – Systematic Review, Modelling &
Qualitative Synthesis on Group Therapy for Postnatal
Depression (UK)
• Stevenson M, Scope A, Sutcliffe P, Booth A, et al. Group
cognitive behavioural therapy for postnatal depression: a
systematic review of clinical effectiveness, cost
effectiveness and value of information analyses. Health
Technol Assess 2010;14(44).
• Scope, A., Booth, A. and Sutcliffe, P. (2012), Women’s
perceptions and experiences of group cognitive behaviour
therapy and other group interventions for postnatal
depression: a qualitative synthesis. Journal of Advanced
Nursing. doi: 10.1111/j.1365-2648.2012.05954
The Choice
• Type of Question – Acceptability of Intervention
• Extent of Description versus Interpretation – Factors Making
Group Therapy More or Less Acceptable (Descriptive)
• Role of Theory – No Theory – Trying to separate Group Effect
from Therapy Effect
• Type of Data – Very Thin Data, Small Number of Studies (n =
6) – Descriptive Case Study Accounts in Nursing Journals
• Intended Output – Alongside Effectiveness Review & Cost
Effectiveness Study – for Implementation
Other Considerations:
Methodological Expertise in Team – Novice Reviewer
Available Resources – Limited Time in Comparison to “Main”
Review
Your Choice?
ST
The Contribution
• While Group Therapy is Acceptable on
Average there are some for whom it is
unfavourable because:
– Group Comparison – Some People are Getting
Better Quicker than I am
– Group Comparison – Ms. X has been Coming Here
longer than I am and is still not any better
• Identified Key Issue: Identification of those
Most Likely to Benefit.
The Scenario
Aim: To analyse meaning and motivation of the
Wish To Hasten Death in patients with chronic
illness or advanced disease
• Monforte-Royo C, Villavicencio-Chávez C, Tomás-Sábado J,
Mahtani-Chugani V, Balaguer A (2012) What Lies behind the
Wish to Hasten Death? A Systematic Review…. from the
Perspective of Patients. PLoS ONE 7(5): e37117.
doi:10.1371/journal.pone.0037117
The Choice
• Type of Question – Explanatory
• Extent of Description versus Interpretation – Factors
that Help to Explain a Phenomenon (Interpretative)
• Role of Theory – Theory Generation
• Type of Data – Contextually Rich – Qualitative Research
Reports – Thick Data [NB. 7 studies]
• Intended Output – Stand Alone Research Report for
Enlightenment
Other Considerations:
Methodological Expertise in Team – Large Experienced
Team with Qualitative Researcher
Available Resources – Extant Funded Project
Your Choice?
E/M
The Contribution
• Wish to hasten death (WTHD) is multifactorial
construct with multiple meanings that do not
necessarily imply genuine desire to hasten one's
death or actually taking steps towards this.
• Wish to hasten death (WTHD) is a phenomenon
that appears, among patients in advanced stages
of illness, as response to extreme suffering that
affects all aspects of their human existence.
The Scenario
Aim: To evaluate benefits and constraints of
collaborative health-related research by researchers
and those affected by the issues under study and/or
those who would apply research results and to
examine how variations in programme context and
mechanisms influence the process and outcomes of
collaborative health intervention research?
Jagosh, J., Macaulay, A. C., Pluye, P., Salsberg, J., Bush, P. L.,
Henderson, J., ... & Greenhalgh, T. (2012). Uncovering the
benefits of participatory research… Milbank Quarterly, 90(2),
311-346.
The Choice
• Type of Question – Explanatory
• Extent of Description versus Interpretation – Factors
that Help to Explain Variation in Outcomes
• Role of Theory – Theory Exploration
• Type of Data – Contextually Rich – Quantitative and
Qualitative Research Reports – Thick Data [NB. Twentythree PR partnerships described in 276 publications.]
• Intended Output – Explanatory Report for Multiple
Funders
Other Considerations:
Methodological Expertise in Team – Large Experienced
International Team of Experienced Researchers
Available Resources – Extant Funded Project
Your Choice?
S/R
The Contribution
• Used middle-range theory of partnership synergy to
demonstrate how PR can (1) ensure culturally and
logistically appropriate research, (2) enhance
recruitment capacity, (3) generate professional capacity
and competence in stakeholder groups, (4) result in
productive conflicts followed by useful negotiation, (5)
increase quality of outputs and outcomes over time,
(6) increase sustainability of project goals, and (7)
create system changes and new unanticipated projects
and activities.
• Generated new insights into benefits of PR regarding
conflicts and negotiation between stakeholders,
program sustainability and advancement,
unanticipated project activity, and generation of
systemic change.
References - 1
• Barnett-Page E, Thomas J.
Methods for the synthesis of
qualitative research: a critical
review. BMC Med Res
Methodol. 2009 Aug 11;9:59.
• Booth, A, Papaioannou, D and
Sutton, A J (2011). Systematic
Approaches to a Successful
Literature Review. SAGE
publications
• Candy B, King M, Jones L,
Oliver S. Using qualitative
synthesis to explore
heterogeneity of complex
interventions. BMC Med Res
Methodol. 2011 Aug
26;11:124.
• Dixon-Woods M, Agarwal S,
Young B, Jones D, Sutton A.
(2004) Integrative approaches
to qualitative and quantitative
evidence. London: Health
Development Agency
• Gough, D, Oliver, S, Thomas J
(2012) An Introduction to
Systematic Reviews. London:
Sage Publications.
• Lorenc, T., Pearson, M., Jamal,
F., Cooper, C. and Garside, R.
(2012), The role of systematic
reviews of qualitative evidence
in evaluating interventions: a
case study. Res. Synth.
Method, 3: 1–10.
References - 2
• Popay J, Roberts H, Sowden A,
Pettticrew M, Arai L, Rodgers
M, Britten N: Guidance on the
conduct of narrative synthesis
in systematic reviews.
http://www.lancs.ac.uk/fass/p
rojects/nssr/2007 .
• Pope C, Mays N, Popay
J: Synthesizing Qualitative and
Quantitative Health Evidence:
a Guide to Methods.
Maidenhead: Open University
Press; 2007.
• Ring N., Ritchie K, Mandava L,
Jepson R. (2011) A guide to
synthesising qualitative
research for researchers
undertaking health technology
assessment and systematic
reviews. NHS Quality
Improvement Scotland and
University of Stirling,
Edinburgh.
• Thomas J, Harden A (2009)
Methods for the thematic
synthesis of qualitative
research in systematic reviews,
BMC Medical Research
Methodology 8:45