Evidence Synthesis for Quality
Improvement Purposes
Davina Allen
Nursing & Midwifery Studies
Direction of Travel
• Understanding the challenges of evidence
synthesis for quality improvement: The
case of integrated care pathways
• Breadth of evidence types
• Diversity of questions
• New research designs
• New models of evidence synthesis
Evidence for Quality Improvement
• Quality and patient safety are policy priorities
• Quality improvement initiatives are typically
complex interventions
• Contextual factors shape effectiveness of
intervention
• Need to know how interventions work in order to
assess whether modification is required
• What works, for whom, in what circumstances and
by what mechanism?
• Challenges of evidence generation, synthesis,
translation
Integrated Care Pathways
•
•
•
•
•
•
•
Introduced into healthcare in the US in 1980s
Widespread use internationally
Multidisciplinary care management tools
Workflow system and record of care
Bring evidence to the bedside
Locally owned and maintained
Pathway development and maintenance
promotes inter-professional working
Active Ingredients?
• Development process (face to face MDT or eworking)
• ICP artefact (paper/EPR, record, guidelines,
decision aides)
• ICP implementation/facilitation (education
and training, coordinator)
• ICP maintenance (variance analysis, review
and revisions)
Illustrative Cases
Review 1
Review 2
• Allen, D. and Rixson, L. (2008)
How has the impact of 'care
pathway technologies' on service
integration in stoke care been
measured and what is the
strength of the evidence to
support their effectiveness in this
respect? International Journal of
Evidence-based Healthcare 6: 78110.
• Allen, D., Gillen, E., Rixson, L.
(2009) A systematic review of the
effectiveness of integrated care
pathways: What works, for
whom, in which circumstances?
International Journal of Evidencebased Healthcare 7: 61-74
The Volume of Literature
• Large literature on ICPs
▫ Diverse contexts
▫ Breadth of patient populations
▫ Wide range of purposes
• Review 1
▫ Search identified 10.000+ references
▫ Narrowed to stroke: 2123 papers
Restricted Evidence for Inclusion
• Review 1
▫ 2123 papers located
▫ 39 obtained for review
▫ 7 included in the study (5 studies)
• Review 2
▫ 4055 papers located
▫ 35 obtained for review
▫ 9 papers included in the study (7 studies)
The Evidence Black Box
• Limited information on the intervention
▫ Development, content, purpose, implementation
• Literature is a-theoretical
• Limited information on context
• Publication bias
New Directions
Local Service Evaluations
Alternative Research Designs
• More information on:
▫ the intervention
▫ the context
• Identify theory to link
intervention with
outcomes
• Report successes and
failures
• Reject efforts to establish
whether pathways are
unequivocally effective
• Realist evaluation
(Pawson and Tilley, 1997)
• Not pass or fail verdict
but examination of
varying impacts across
different contexts
New Directions
Realist Synthesis (Pawson)
• Realist philosophy
• Unpack the relationships
between context,
mechanism and
outcomes
• Suited to understanding
‘complex’ interventions
where different contexts
appear to produce
different expected and
unexpected effects
Meta-Narrative Synthesis
(Greenhalgh et al)
• Constructivist philosophy
• Illuminate the different
paradigmatic approaches
to a complex topic area
• Suited to topics where
there is dissent about the
nature of what is being
studied and the best
empirical approach to
studying it
Connecting for
Healthcare Improvement
• Descriptive study of nurses’ articulation work
• Realist synthesis of the literature on coordinating
interventions
• In-depth ethnographic study of
organisational/departmental interfaces and
coordinating ‘interventions’/processes
Realist Synthesis of Coordinating
Interventions (AKA swamp-dwelling)
Meta Narrative Synthesis
Realist Synthesis
• Science and technology
studies
• Computer-supported
cooperative work
• Medical Sociology
• Organizational Studies
•
•
•
•
•
•
•
•
•
Boundary spanners
Standards
Pathways
Routines
Hand-offs
Case management
MDTs
Handovers
Ward rounds
Conclusion
• Healthcare managers
need evidence-based
information for quality
improvement
• Quality improvement
interventions are
complex
• Healthcare systems
create complex contexts
• Managers need to know:
What works, in what
context and how?
• Conventional research
designs black box
relevant information
• Conventional systematic
review literature cannot
cope with heterogeneity
• New research approaches
needed
▫ Realist synthesis
• New synthesis
methodologies required
▫ Meta-narrative analysis
▫ Realist synthesis
Further reading
Realist synthesis/review
• Wong G, Greenhalgh T, Westhorp G, Pawson D: Realist methods in
medical education research: what are they and what can they
contribute? Med Educ 2012, 46: 89-96.
• Pawson, R. (2006). Evidence-based Policy. A Realist Perspective.
London: Sage.
Meta-narrative synthesis
• Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O, Peacock R:
Storylines of research in diffusion of innovation: a meta-narrative
approach to systematic review. Soc Sci Med 2005, 61: 417-430.
Join us at: www.jiscmail.ac.uk/RAMESES