Can Implementation Research Save Global Health?
May 1, 2015
David Peters, MD, MPH, DrPH
Professor & Chair Department of International Health
Johns Hopkins University Bloomberg School of Public Health
Why Global Health?
GlobalInfant
healthmortality
spendingcartogram
cartogram
Dorling D, Hennig B. 2010. Making visible injustice in health: mapping the causes of 57 million deaths. www.worldmapper.org.
Reaching the Poor in Health, Nutrition,
Population
56 Low- and Middle-Income Countries (2.8 billion population), 1990-2002
The Rediscovery and Growth of “Global
Health” in Universities
Table from: Koplan J et al. (2009) Towards a common definition of global health. Lancet 373: 199395.
4
The Quest
• To find sustainable solutions to improving the health
of disadvantaged people around the world
There are Some “Magic Bullets”
Ambitious and High Profile Interventions
7
But also Many Failed Interventions
“Our country is littered with the bones
of successful pilot projects”
African Health Minister
8
Focus of Research in Global Health:
Identifying Evidence-Based Interventions
• Many cost-effective solutions shown to save
lives
Conventional Pathway to Improving Health
1.Choose the right (cost-effective) health interventions
2.Set ambitious, common targets
3.Fund them
4.Implement interventions as designed
Do Countries Scale-Up Interventions the
Same Way?
Country-specific rates of skilled birth attendance
Random coefficient, random intercept model; each line represents one country
Why Don’t Countries Have Similar Results?
Similar goals & interventions, different contexts & results: Countryspecific rates of child mortality change
Subramanian et al. (2011). Do we have the right models for scaling up health services to achieve the
Millennium Development Goals? BMC Health Services Research
Maternal Child Health Services Cascade
(Kenya 2008-09): Hard to Implement
Effective Interventions
100
90
80
Percent Coverage
70
60
50
40
30
20
10
0
Any ANC
ANC with
Blood Test
Protected for
Tetanus
4+ ANC
Skilled Birth Post natal care Any Post natal
Fully
Attendance
in 24 hours
care
vaccinated by
12 months
Why is Research on Implementation
Needed?
But focus is on local
and national
problem-solving
14
What is Implementation Research?
Implere – to fill up or fulfill
Implementation research is the scientific inquiry
into questions concerning implementation
Implementation Research Traditions
Implementation
research tradition
Management
improvement
Operational research
Policy
implementation
Typical target
audience
Managers and teams
using improvement
strategies
Decision-makers
(executives, executive
bodies, policy-makers)
Top down: Central level
policy-makers
Bottom up: “Street
level” programme
implementers
Participatory action
research
Research participants,
community members
Dissemination and
implementation of
evidence-based
medicine
Practitioners, health
organization managers,
policy-makers not using
evidence-based
interventions
Typical research questions
Original core disciplines
How are the right services delivered to the
right clients while meeting the right
standards for quality?
Which solution provides the most rational
basis for a decision concerning the optimal
performance of a system?
Top down: How was a policy implemented,
and what contributed to reaching the
objectives or other effects?
Bottom up: Which actors are involved in
programme delivery in specific locations,
how do they understand the problem of
implementation, and what influences their
behaviour?
How can we (community members as
research participants) learn and be
empowered to take action?
Engineering, management
What promotes the integration of research
findings and evidence on interventions into
healthcare policy and practice?
Mathematics, engineering,
management
Political science, public
policy, public
administration
Non-disciplinary or transdisciplinary; influenced by
sociology, psychology,
education, and anthropology
Behaviour change
(psychology, sociology,
education), epidemiology
Peters et al. (2013). Implementation research: what is it and how to do it. BMJ
16
What is Implementation Research?
• Address any aspect of implementation
– factors affecting implementation, processes of
implementation, end products of the implementation
• May focus on strategies needed to deliver
interventions: implementation strategies
• Implementation outcome variables are part of
framework to measure how well implementation
is working
Implementation Research:
Key Characteristics
• Seeks to understand and work within “real world” or
usual practice settings
• Addresses how, why, and what is implemented and their
effects
Concern with “interventions” at several levels:
• policies, programs, or individual practices
Intention to improve health, access to health services,
quality of health services, financial protection, equity …
Implementation Strategies for Health
Programs
Peters, El-Saharty et al. 2009. Improving Health Service Delivery in Developing Countries
19
How is Implementation Research Used?
• Understand context, assess
performance, inform implementation,
strengthen health systems
• Support scale-up and integration of
interventions
• Help organizations develop capacity to
learn
Implementation research is about
Getting research into practice ...
and
Gaining knowledge from practice
21
Implementation Research is Not New
Smallpox Cases from 1960-67, and 1968-69 in 20 West African Countries:
Surveillance-Containment vs. Mass Immunization Strategy
But Many Theories and Newly Applied
Methods
23
Practical Lessons from
Implementation Experience
• Many interventions work, but there is NO BLUEPRINT, NO
MAGIC BULLET for successful implementation
Universal Principles and Processes:
• A “learning and doing” approach underlies successful
implementation of many different strategies
• Flexibility, data, and involvement of key stakeholders and
implementation support are needed to address complex
and changing conditions
How Strategies are Implemented Matters:
Odds of Success
Randomized
All
Controlled “Adequate”
Trials
Studies
Community coordination and organization
Local adaptation of the intervention
Broad-based support of various
stakeholders
Consultation and engagement of powerful
interest groups
Flexibility and modification through
stakeholder feedback
Representation from powerful interest
groups
Constraints reduction plans
Odds ratio
Odds ratio
..
4.6**
9.3
4.3 *
..
3.9 *
2.8
3.8**
..
3.4 *
2.4
3.0 *
6.7
2.7 *
* Pvalue<0.05; ** Pvalue<0.01
The Afghanistan Balanced Scorecard: Building and
Assessing Capacity to Deliver Services
Context: Rebuilding the Health Sector
Developing the Balanced Scorecard
Frontline providers, NGOs, MOPH, donors meet to
agree on:
• Purpose of Balanced Scorecard
• Domains to measure
• Unit of analysis
• Process & frequency of review/decisions
• Principles for benchmarking
• Short-listing indicators based on face validity,
importance, reliability
Afghanistan Provincial Balanced Scorecard 2004
Measure
A. Patients & Community
1 Overall Patient Satisfaction
2 Patient Perception of Quality Index
3 Written Shura-e-sehie activities in community
B. Staff
4 Health Worker Satisfaction Index
5 Salary payments current
6
7
8
9
10
11
12
13
14
15
16
C. Capacity for Service Provision
Equipment Functionality Index*
Drug Availability Index
Family Planning Availability Index
%
%
%
%
%
%
%
Laboratory Functionality Index (Hospitals & CHCs)
%
Staffing Index -- Meeting minimum staff guidelines
%
Provider Knowledge Score
Staff received training in last year
HMIS Use Index
Clinical Guidelines Index
Infrastructure Index
Patient Record Index
17 Facilities having TB register
18
19
20
21
22
23
24
%
D. Service Provision
Patient History and Physical Exam Index
Patient Counseling Index
Proper sharps disposal
Average new outpatient visits per month (BHC>750 visits)
Time spent with patient (> 9 minutes)
BPHS facilities providing antenatal care
Delivery care according to BPHS
E. Financial Systems
25 Facilities with user fee guidelines
26 Facilities with exemptions for poor patients
F. Overall Vision
27 Females as % of new outpatients
28 Outpatient visit concentration index
29 Patient satisfaction concentration index
Composite Scores
30 Upper Benchmarks Achieved
31 Lower Benchmarks Achieved
%
%
%
%
%
%
%
%
%
%
%
%
%
%
%
%
%
CI (-1 to 1)
CI (-1 to 1)
%
%
National
Median
Lower
Benchmark
Upper
Benchmark
Badakhshan
Badghis
Baghlan
Balkh
Bamyan
Farah
83.1
76.0
34.2
66.4
66.2
18.1
90.9
83.9
66.5
86.4
77.6
35.6
76.9
66.2
0.0
90.9
82.2
34.2
84.7
80.0
17.7
97.9
84.4
34.5
82.8
69.4
73.2
63.5
76.7
56.1
52.4
67.9
92.0
63.5
54.9
57.6
91.8
67.9
45.8
68.3
53.3
61.4
91.4
54.4
97.7
65.7
71.1
61.4
18.3
39.3
53.5
39.0
67.7
34.8
55.0
65.6
15.8
61.3
53.3
43.4
5.6
10.1
44.8
30.1
49.6
22.5
49.3
56.1
8.3
90.0
81.8
80.3
31.7
54.0
62.3
56.3
80.7
51.0
63.2
92.5
26.6
69.6
52.9
54.2
31.7
38.0
48.6
68.9
60.9
18.3
63.2
51.5
32.5
62.2
50.1
57.9
3.8
22.4
41.6
50.9
62.7
25.5
49.7
98.5
27.0
57.5
72.8
70.4
15.2
42.7
49.3
39.0
40.0
29.9
50.0
80.7
16.1
67.3
56.1
64.9
0.0
45.8
54.0
52.4
72.9
16.4
58.3
97.3
16.4
75.8
85.6
82.7
37.0
53.0
69.0
35.5
67.7
41.9
57.9
64.5
0.0
66.3
9.8
0.0
0.0
57.1
45.5
37.2
72.4
59.5
76.7
97.1
4.3
70.6
29.6
62.2
22.2
18.0
62.0
25.4
55.1
23.3
34.1
6.7
3.5
28.9
10.5
83.5
48.9
85.0
57.1
31.2
82.8
39.3
54.2
23.3
64.4
27.3
21.0
28.9
38.0
71.7
40.4
34.1
10.0
30.7
49.4
36.2
55.1
29.3
76.9
27.3
1.2
49.7
10.5
85.4
55.3
75.1
71.4
27.3
67.2
39.3
83.6
33.2
85.0
22.2
12.8
88.1
38.0
52.0
16.0
67.8
0.0
18.0
82.8
57.1
90.6
84.7
80.3
64.4
100.0
100.0
94.8
68.5
95.6
54.6
95.9
69.3
28.9
84.3
86.1
95.6
100.0
93.9
55.2
-0.010
0.002
46.5
0.041
0.020
59.7
-0.055
-0.018
46.9
0.021
-0.019
45.9
0.024
0.000
56.0
-0.038
0.003
55.1
0.025
-0.007
55.2
-0.076
-0.005
59.0
-0.036
0.020
17.2
82.8
10.3
75.9
30.8
89.7
17.2
86.2
6.9
79.3
6.9
86.2
20.7
86.2
34.5
96.6
31.0
72.4
Baghlan Provincial Balanced Scorecard 2004-2012/13
AFGHANISTAN HEALTH SECTOR
BPHS Balanced Scorecard 2004 - 2011/12
Domain A: Client and Community
Overall Patient Satisfaction
1 Patient Perception of Quality Index
Overall Client Satisfaction and Perceived Quality of Care Index
Written Shura-e-sehie activities in community
2
Community Involvement and Decision Making Index
Domain B: Human Resources
Health Worker Satisfaction Index
3
Revised Health Worker Satisfaction Index
4 Health Worker Motivation Index
5 Salary Payment Current
Staffing Index -- Meeting minimum staff guidelines
6
Revised Staffing Index -- Meeting minimum staff guidelines
Provider Knowledge Score
Revised Provider Knowledge Score
7
Revised Revised Provider Knowledge Score
New Provider Knowledge Score
Staff received training in last year
8
Revised Staff Received Training (in last year)
Domain C: Physical Capacity
Equipment Functionality Index
9
Revised Equipment Functionality Index
Drug Availability Index
10
Pharmaceuticals and Vaccines Availability Index
Laboratory Functionality Index (Hospitals & CHCs)
11
Revised Laboratory Functionality Index (CHCs only)
Clinical Guidelines Index
12
Revised Clinical Guidelines Index
Infrastructure Index
13
Revised Infrastructure Index
Domain D: Quality of Service Provision
Patient History and Physical Exam Index
14
Client Background and Physical Assessment Index
Patient Counseling Index
15
Client Counselling Index
Proper sharps disposal
16
Universal Precautions
17 Time Spent with Client
Domain E: Management Systems
HMIS Use Index
18
Revised HMIS Use Index
19 Financial Systems
20 Health Facility Management Functionality Index
Domain F: Overall Mission
Outpatient visit concentration index
21
New Outpatient visit concentration index
Patient satisfaction concentration index
22
New Patient satisfaction concentration index*
COMPOSITE SCORES
Percent of Upper Benchmarks Achieved
Percent of Lower Benchmarks Achieved
Overall Means (Provincial)
BAGHLAN
BENCHMARKS
LB
UB
2004 2005 2006 2007 2008
2009 2011 2012
/10 /12 /13
66.4
90.9
90.9
91.9
89.2
78.4
91.1
85.3
-
66.2
83.9
82.2
74.5
82.4
78.2
90.3
85.6
-
-
73.3
18.1
81.3
66.5
-
-
-
-
-
-
76.3
77.2
34.2
76.2
84.1
69.4
91.7
95.5
-
-
72.4
90.0
-
-
-
-
-
-
78.3
93.5
56.1
67.9
67.9
62.4
69.0
69.5
76.4
73.3
-
-
61.7
66.6
-
-
-
-
-
-
65.6
72.0
66.7
52.4
72.8
92.0
-
-
-
-
-
-
69.1
76.7
45.8
84.6
38.4
92.2
82.7
62.4
29.7
56.0
10.1
54.0
42.7
64.4
69.8
55.5
79.4
93.2
-
-
11.4
44.8
33.3
62.3
-
-
-
-
-
-
26.2
37.5
49.3
68.3
72.3
66.3
-
-
-
-
71.5
86.0
-
-
-
-
86.0
-
-
-
61.9
77.7
-
-
-
-
-
73.3
-
-
59.4
30.1
67.6
56.3
-
-
-
-
-
-
66.7
67.8
39.0
74.5
85.3
73.2
75.5
49.5
-
-
7.1
14.9
-
-
-
-
-
-
5.9
9.4
61.3
90.0
57.5
65.6
83.9
81.3
91.8
92.1
-
-
67.4
53.3
85.0
81.8
-
-
-
-
-
-
77.6
92.4
72.8
82.0
65.9
74.7
78.5
90.8
-
-
71.8
5.6
88.6
31.7
-
-
-
-
-
-
76.6
84.3
15.2
36.3
43.0
53.7
69.4
70.8
-
-
53.1
22.5
76.3
51.0
-
-
-
-
-
-
37.5
74.1
29.9
48.9
78.7
72.2
90.5
95.7
-
-
64.3
49.3
85.9
63.2
-
-
-
-
-
-
66.8
96.3
50.0
38.7
45.7
27.3
62.3
77.7
-
-
48.9
73.4
-
-
-
-
-
-
47.2
77.4
55.1
83.5
55.1
81.6
81.8
76.7
88.5
90.1
-
-
66.7
23.3
81.2
48.9
-
-
-
-
-
-
73.8
86.2
29.3
40.3
36.2
33.1
71.1
48.8
-
-
31.7
34.1
58.5
85.0
-
-
-
-
-
-
40.1
43.8
76.9
58.1
96.2
63.4
85.5
93.2
-
-
51.8
70.4
-
-
-
-
-
-
44.1
77.6
3.5
31.2
1.2
4.1
1.6
11.5
67.4
30.8
7.5
47.8
49.6
80.7
40.0
68.7
86.9
81.4
96.3
81.6
-
-
66.1
86.2
-
-
-
-
-
-
86.2
94.3
-
2.2
20.3
-
-
-
-
-
-
0.0
11.4
40.0
57.6
-
-
-
-
-
-
68.0
58.2
48.0
52.7
51.9
50.8
51.7
50.7
48.2
-
-
-
46.2
56.9
-
-
-
-
-
51.2
46.9
45.5
49.0
50.9
49.8
50.2
50.1
50.1
49.7
-
-
-
49.6
50.8
-
-
-
-
-
51.0
49.3
50.2
5.0
80.0
49.1
35.0
90.0
61.1
45.0
85.0
65.6
45.0 75.0 70.0 9.1
95.0 100.0 100.0 68.2
62.9 78.6 74.6 51.8
59.1
95.5
65.0
-
Domain: Quality of Service Provision
Indicator 17:
Time spent with Client
Provinces meeting lower benchmark (%)
Provinces meeting upper benchmark (%)
National Medians
100
80
60
40
20
18.4
18.0
6.2
7.0
2005
2006
19.7
16.1
11.8
9.6
0
2004
2007
2008
2009 /10
2011 /12
2012 /13
Assessing and Improving Performance:
Afghanistan
How the Afghanistan Balanced
Scorecard Has Been Used
• Highlight areas for increased resources, training
• Contracts with service providers: bonuses or
cancelled contracts
• Motivation for organizations and staff
• Accountability to public
• New policies and programs
• Provide basis for further research on contracting and
health financing interventions
Hansen et al. (2008) Measuring and Managing Progress in the Establishment of Basic Health Services:
The Afghanistan Health Sector Balanced Scorecard. International Journal of Health Planning and
Management 23 (2): 107-117
The Promise of Implementation
Research
• Informs practitioners, policy-makers,
consumers, researchers, funders and other
users on practical matters
• Builds capacity, ownership and accountability
• Strengthens problem-solving and learning
• Contributes to improved coverage, quality,
efficiency and effectiveness of health
interventions
35
A Proposition
If you are overseeing, sponsoring, or implementing a
health policy or program, you should have an
implementation research agenda
36
How Should We Intervene in
Global Health?
1. Listen to the voice of the disadvantaged
–
Identify levels and causes of inequality
2. Find best evidence – “evidence-informed public health” +
implementation evidence
3. Use a “systems lens” to look across the health system for
resources and effects; locally, nationally, globally
4. Customize interventions to address local capabilities and
constraints
5. Adapt and experiment, focus on implementation for
sustainable systems
6. Pay attention to governance and institution-building
7. Measure, disclose results, learn – especially to see that the
poor are benefiting and that constraints are being addressed
38