Routine Measurement of
Quality of Care
Barbara Rawlins
Senior Monitoring and Evaluation Manager,
MCHIP/Jhpiego
Maryjane Lacoste
Tanzania Country Director/MAISHA Program
Director, Jhpiego
Interventions for Impact in Essential Obstetric and Newborn Care
Africa Regional Meeting, 21–25 February 2011
Presentation Overview
 Review existing routine quality of care
measurement methods
 Describe gaps in routine quality of care data
for maternal and newborn health services
 Present a case study from Tanzania
 Discuss future measurement plans
Interventions for Impact in Essential
Obstetric and Newborn Care
Africa Regional Meeting, 21–25 February 2011
2
Measuring the Quality of Maternal
and Newborn Care is…
 Multi-faceted and complex
 Especially measurement of quality of intra-
partum care (as it includes both routine care
and management of complications)
 Usually conducted through both periodic
surveys and routine measurement
mechanisms
Interventions for Impact in Essential
Obstetric and Newborn Care
Africa Regional Meeting, 21–25 February 2011
3
Why measure the quality of maternal
and newborn care routinely?
 Program managers and policy makers need
regular and reliable data for decision making:
 Need to understand how the health system is
performing at multiple levels
 Need to know if “skilled birth attendants” are
really skilled
 Need to determine if facilities are “ready” to
provide services with respect to infrastructure,
supplies, drugs and equipment
Interventions for Impact in Essential
Obstetric and Newborn Care
Africa Regional Meeting, 21–25 February 2011
4
Key Methods for Measuring MNH
Quality of Care Routinely
1. Structured clinical observation of provider-client
interactions
 Data sources: clinical checklists applied by peers and/or
external assessors (e.g., checklists used for quality
improvement initiatives, supervision checklists)
 Data captured: compliance with clinical guidelines and
standards
2. Inventory of facility infrastructure/supplies/equipment
 Data sources: facility audit checklists, supervision reports
 Data captured: Stockouts of key medicines, broken and
absent equipment, organization of services
Interventions for Impact in Essential
Obstetric and Newborn Care
Africa Regional Meeting, 21–25 February 2011
5
Key Methods for Measuring MNH
Quality of Care Routinely (2)
3. Record review
 Data sources: Health management information
system (HMIS) reports; facility registers; patient
charts; criterion-based audits; near miss audits;
maternal and perinatal death audits; sentinel site
surveillance systems, logistics management
information systems
 Data captured: service utilization; frequency of
provision of evidence-based components of care
(e.g., for ANC– iron, TT, IPTp); management of
complications; numbers of deaths and complications
Interventions for Impact in Essential
Obstetric and Newborn Care
Africa Regional Meeting, 21–25 February 2011
6
Many Gaps in Routine MNH Quality
of Care Data
 Observational assessments of client-provider
interactions are not widely conducted on a
routine basis
 Logistics management information systems and
supervision reports only include a limited set of
facility readiness indicators, such as stockouts
 Many MNH service indicators of interest are not
captured in national HMIS, especially those
related to intrapartum care (e.g., active
management of the third stage of labor)
Interventions for Impact in Essential
Obstetric and Newborn Care
Africa Regional Meeting, 21–25 February 2011
7
Adapting Sentinel Site Surveillance
for MNH Quality of Care Monitoring
 MCHIP/Malawi and the MAISHA Program in
Tanzania are in the process of applying a sentinel
site surveillance approach for the continuous
quality monitoring of MNH services
 The MNH SSS systems are intended to:
 Complement MNH data available from national HMIS
reports
 Test the feasibility of collecting additional MNH quality
indicators at facilities on a routine basis
 Generate national support for routine collection of facilitybased quality indicators that prove feasible
Interventions for Impact in Essential
Obstetric and Newborn Care
Africa Regional Meeting, 21–25 February 2011
8
What is sentinel site surveillance?
 Sentinel sites are health facilities selected (using
specific criteria) for monitoring of key indicators
that are generally not reported up through the
national health management information system
 Sentinel site surveillance (SSS) traditionally has
been used to track disease-related indicators:
 National Malaria Control Programs and the U.S.
President’s Malaria Initiative (PMI) use SSS to
monitor trends in inpatient and outpatient malaria
cases, inpatients deaths, uptake of intermittent
preventive treatment of malaria by antenatal care
clients
Interventions for Impact in Essential
Obstetric and Newborn Care
Africa Regional Meeting, 21–25 February 2011
9
Tanzania Case Study: MNH Quality
Monitoring by the MAISHA Program
 The ACCESS/Tanzania program initiated a SSS
system for FANC in 2006/7
 A total of 30 facilities across 16 regions were selected:
10 midwifery preservice clinical sites (hospitals) and 20
facilities that received FANC inservice training
(hospitals, health centers and dispensaries)
 The SSS System was expanded under MAISHA
in 2009 to include monitoring of essential and
emergency obstetric and newborn indicators
 Now have 40 facilities across 21 regions plus Zanzibar:
19 hospitals, 12 health centers, 9 dispensaries
Interventions for Impact in Essential
Obstetric and Newborn Care
Africa Regional Meeting, 21–25 February 2011
10
Tanzania Case Study: SSS Indicators
FANC Indicators
EONC/EmONC Indicators
 % of ANC clients receiving
IPTp1 and IPT2
 % of ANC clients tested for
syphilis
 % of ANC clients treated for
hookworm
 Number of ANC clients with
Hb < 8.5g/dl
 Number of functional BP
machines with stethoscope
 Number of days SP, RPR
kits and iron out of stock
 Number of women with
obstetric complications (by
type of complication) and
number referred
 Number of maternal deaths
by cause
 Number of newborn deaths
(<24 hours, 24hours+)
 Number of stillbirths
 Number of days oxytocin,
ergometrine, misoprostol and
MgSO4 out of stock
Interventions for Impact in Essential
Obstetric and Newborn Care
Africa Regional Meeting, 21–25 February 2011
11
Tanzania Case Study: FANC Trends
Percent of ANC Clients/Facilities
100
IPT Uptake and SP Stockout Trends, October 2009
- September 2010
90
80
70
60
50
IPTp1
IPTp2
Stockout of SP
40
30
20
10
0
Oct-Dec 09 Jan-Mar 10 Apr-Jun 10 Jul-Sept 10
Interventions for Impact in Essential
Obstetric and Newborn Care
Africa Regional Meeting, 21–25 February 2011
12
Tanzania Case Study: EMOC Drug
Stockout Trends
Stockouts of EMOC Signal Function
Drugs, October 2009 – September 2010
100
Percent of Facilities
90
80
70
60
Oxytocin
MgSO4
50
40
30
20
10
0
Oct-Dec 09 Jan-Mar 10 Apr-Jun 10 Jul-Sept 10
Interventions for Impact in Essential
Obstetric and Newborn Care
Africa Regional Meeting, 21–25 February 2011
On the Horizon for MCHIP and
MAISHA
 MAISHA in Tanzania plans to expand data collection to
200+ facilities from all districts
 MCHIP/Malawi is testing a routine MNH Quality sentinel
surveillance system in the coming months
 MCHIP plans to explore how an abbreviated version of
the MNH Quality of Care facility survey can be applied
on a routine basis
Interventions for Impact in Essential
Obstetric and Newborn Care
Africa Regional Meeting, 21–25 February 2011
14
A discussion question for you…
 Are there aspects of MNH quality of care we
could be monitoring but we are not?
Interventions for Impact in Essential
Obstetric and Newborn Care
Africa Regional Meeting, 21–25 February 2011
Thank you!
MNH Quality of Care Measurement Resources:
http://www.who.int/reproductivehealth/publications
/monitoring/9789241547734/en/index.htm
http://www.rollbackmalaria.org/partnership/wg/wg_
pregnancy/docs/MIPMEFramework.pdf
Follow MCHIP on:
www.mchip.net
Interventions for Impact in Essential
Obstetric and Newborn Care
Africa Regional Meeting, 21–25 February 2011