Achieving Program Targets:
An HIV Care Cascade Approach
Molly McNairy and Bill Reidy, ICAP-NY
March 28, 2013
Webinar Overview
1.
2.
3.
4.
5.
6.
Background
Examples of low target performance
Dimensions of the problem: M&E & Clinical
Introduce a cascade approach
A case study
Toolkit inventory
Webinar Overview
1.
2.
3.
4.
5.
6.
Background
Examples of low target performance
Dimensions of the problem: M&E & Clinical
Introduce a cascade approach
A case example
Toolkit inventory
Background
• There are many reasons why a program may
face challenges reaching key targets
• Even the highest-functioning program can
have low target performance
• It is important that we address these
challenges on an ongoing basis
• Country teams have various methods for
monitoring progress to targets (e.g., ongoing
DQA, reports to funders, slide sets, URS)
URS Targets Dashboard
https://urs2.icap.columbia.edu/#dashboard
Filter by country and time period
URS Targets Dashboard
Export data to Excel sheet
Webinar Overview
1.
2.
3.
4.
5.
6.
Background
Examples of low target performance
Dimensions of the problem: M&E & Clinical
Introduce a Cascade approach
A case example
Toolkit inventory
ART Initiation: Swaziland
New and cumulative patients on ART
CDC Rapid Scale-up Year 3
12,000
Target = 11,296
by Oct 2012
Number of patients
10,000
8,000
6,000
New on ART
Cumulative on ART
4,000
2,000
0
Oct-Dec 2011
Jan-Mar 2012
Quarter
Apr-Jun 2012
Retention on ART: Mozambique
35000
30000
Target = 85%
retained
25000
59%
20000
15000
# ART patients*
50%
# ART patients retained
10000
5000
*Excludes patients who
transferred out
0
APR 2012 report to CDC
Re-counted numbers
Pediatric TB screening
One OPD facility: Tanzania
200
180
Target = 100%
screened
160
140
120
100
attended
80
screened
32%
60
25%
40
20
8%
0
week 3 jan
week 4 jan
week 1 feb
Webinar Overview
1.
2.
3.
4.
5.
6.
Background
Examples of low target performance
Dimensions of the problem: M&E & Clinical
Introduce a Cascade approach
A case example
Toolkit inventory
Low performance may have multiple and
overlapping M&E-Clinical components
M&E
• Data quality
• Data availability
• M&E system issues
Clinical
• Structural barriers
• Staffing issues
• Health system issues
Solution = must include both components
Webinar Overview
1.
2.
3.
4.
5.
6.
Background
Examples of low target performance
Dimensions of the problem: M&E & Clinical
Introduce a cascade approach
A case study
Toolkit inventory
A Cascade Approach: Why?
• A care cascade outlines the multiple steps in a
clinical pathway needed to achieve optimal
health outcomes.
• The target of interest is part of a larger cascade of
care in which the previous steps affect the target
• Improving the entire cascade will lead to
improvements in the target as well as other
targets simultaneously
• Improving the entire cascade will lead to more
sustainable improvements
Steps in the Cascade Approach
1. Identify steps in the cascade that relate to
target
2. Identify baseline data to operationalize the
cascade
3. Choose priority sites
4. Choose interventions and prioritize them
5. Use a cohort methodology to monitor
progress
1. Identify steps in the cascade that relate to
target
• The cascade’s steps are specific to the disease
(i.e. HIV, TB) and the patient population (i.e.
adults, children, pregnant women/infants).
Adult Care & Treatment
Link
ART
Eligible
McNairy, El-Sadr AIDS 2012
Tuberculosis
TB Suspect
Screen
TB Disease
Evaluate for TB
disease
TB Treatment
Retain, counsel
monitor and
support
TB Treatment Success
Prevent
recurrence,
ongoing screening
Fayorsey, Howard 2013
2. Identify Baseline Data to
Operationalize Cascade
• Where to get baseline data for a cascade?
• Routinely-reported M&E data, e.g.:
– Country aggregate databases
– URS
• Original data collection from clinics
What source to use for baseline data?
• Routinely-reported M&E data
– Advantages:
• historical data is readily available
• data available for many facilities
• collection requires no additional efforts
– Disadvantages:
• indicators not flexible (may not measure what you
need)
• data may have quality issues
– Particular danger when target shortfall is in part due to M&E
system issues
What source to use for baseline data?
• Original data collection from clinic
– Advantages:
•
•
•
•
have access to all data collected
high level of flexibility in defining set of indicators
can use highest-quality data available
may be used to compare to reported M&E data
– Disadvantages:
• burden of data collection
• lack of a large amount of historical data for comparison
• If at all possible, advisable to collect original
data to supplement routine M&E data
3. Determining & Prioritizing
Interventions
• Root cause Analysis/Driver Diagram
• Focusing Matrix
Driver Diagram
• A tool to facilitate root cause analysis
– Articulates the aim of the campaign
– Organizes primary categories for reasons
contributing to low performance
– Subdivides categories into specific reasons
– Facilitates a specific intervention tied to each
reason
An example…
Secondary Drivers Interventions
Primary Drivers
Aim
Driver Diagram
• Step 1: Aim
– Target
– Numerical goal for improvement
– Time frame
– Location (place or # of clinics)
Secondary Drivers
Aim
Increase ART
initiations by
at least 30%
in 3 months
at 15 priority
clinics
Primary Drivers
Interventions
Driver Diagram
• Step 2: Primary Drivers
– Make a list of broad categories of factors that
must be addressed to achieve aim
Secondary Drivers
Aim
Primary Drivers
Provider/Patient
Increase ART
initiations by at
least 30%
in 3 months
Supplies
(CD4/Lab)
at 15 priority
clinics
Drugs
Interventions
Driver Diagram
• Step 3: Secondary Drivers
– Specific problems under each category
• Step 4: Match specific interventions to each
driver
Aim
Primary Drivers
Provider/Patient
Increase ART
initiations by
at least 30%
in 3 months
CD4/Lab
at 15 priority
clinics
ART
Continue to fill in and complete boxes for
all secondary drivers and interventions
Secondary Drivers
Interventions
Knowledge of
WHO staging
Staging posted in
clinics, train providers
Eligible patient
but not on ART
Outreach, phone
calls, home visits
Patient refuses
ART
Assign peer counselor
Focusing Matrix
• Tool to aid in prioritizing interventions
• Uses both importance and ease of
implementation to rank priority
An example…
Focusing Matrix
Ease of Implementation
IMPORTANCE
1
(Least)
1
(Hardest)
2
3
4
5
(Easiest)
2
3
4
5
(Most)
Focusing Matrix
Ease of Implementation
IMPORTANCE
1
(Least)
2
3
4
5
(Most)
1
(Hardest)
2
3
4
5
(Easiest)
# 2 priority
most
important and
easiest to
implement –
#1 priority
Prioritizing Interventions
Example: Low ART Initiations (adult)
Proposed Intervention
A
WHO staging posted in clinics
to be reference for providers
Importance
Ease of
Implementation
1
3
2
3
4
5
5
B
Identify ART eligible patients
who have not yet initiated
ART and call them to return
5
5
C
Fix broken CD4 machines
5
1
D
Outreach ART eligible
patients at home if no show
for appointment
5
3
Assign peer counselor to
patients who refuse ART
3
3
E
IMPORTANCE
EASE of IMPLEMENTATION
Item
#
1
C
2
3
E
D
A
B
4
5
Interventions B and A should be first priority
4. Choosing Priority Sites
Highest Volume
Lowest Performance
65%
80%
42%
55%
30%
75%
20%
85%
66%
40%
35%
80%
5. Cohort Methodology to measure
change in performance towards target
• Goal is to assess impact of approach on relevant
target and cascade indicators
• Impact must be sustainable
• A cohort methodology:
1. Define cohorts of patients
2. Collect cascade data for cohort from source
documents
3. Summarize graphically
4. Review data and revisit intervention plans
5. Repeat process 2-4 periodically (e.g., every month)
Define Cohorts of Patients
• A cohort is a group of people sharing a
common trait, usually defined by a point in
time (e.g., birth cohort of people born in 1981)
• For this cascade approach, define cohort as
any patient who entered the cascade during a
specified time period, e.g.:
– Patients testing HIV-positive at Kagera Regional
Hospital during January 2013
– Patients enrolling in HIV care at RFM Hospital
during 2011
Collect cascade data for cohort from
source documents
• Operationalize the steps in relevant cascade
–
–
–
–
–
# enrolling in HIV care
# with ART eligibility assessed via CD4/WHO stage
# ART eligible
# initiating ART
# retained on ART (e.g., at 6 months, 12 months)
• Specify the best source of data for each step
• Design simple tools (paper, Excel) for abstracting and
summarizing this data
• Plan for periodic data collection
– Measuring retrospective improvements
– Measuring improvements moving foward
Summarize cohort in a graph
Pre-ART enrollment, ART eligibility, and ART initiations
100
Intervention
begins
90
Number of patients
80
70
60
50
58%
40
30
71%
20
36%
10
0
New pre-ART
Assessed for ART eligibility
Eligible for ART
ART initiations
Cohort 1
86
50
28
10 20
Cohort 2
Cohort 3
Cohort 4
Cohort 5
Summarize cohort in a graph
Pre-ART enrollment, ART eligibility, and ART initiations
100
Intervention
begins
90
Number of patients
80
70
73%
60
50
58%
40
30
58%
20
36%
10
0
New pre-ART
Assessed for ART eligibility
Eligible for ART
ART initiations
Cohort 1
86
50
28
10
Cohort 2
81
59
31
18
Cohort 3
Cohort 4
Cohort 5
Summarize cohort in a graph
Pre-ART enrollment, ART eligibility, and ART initiations
100
Intervention
begins
90
Number of patients
80
79%
70
73%
60
50
58%
40
70%
30
58%
20
36%
10
0
New pre-ART
Assessed for ART eligibility
Eligible for ART
ART initiations
Cohort 1
86
50
28
10
Cohort 2
81
59
31
18
Cohort 3
95
75
40
28
Cohort 4
Cohort 5
Summarize cohort in a graph
Pre-ART enrollment, ART eligibility, and ART initiations
100
Intervention
begins
90
Number of patients
80
79%
70
95%
73%
60
50
58%
88%
40
70%
30
58%
20
36%
10
0
New pre-ART
Assessed for ART eligibility
Eligible for ART
ART initiations
Cohort 1
86
50
28
10
Cohort 2
81
59
31
18
Cohort 3
95
75
40
28
Cohort 4
79
75
41
36
Cohort 5
Summarize cohort in a graph
Pre-ART enrollment, ART eligibility, and ART initiations
100
Intervention
begins
90
Number of patients
80
99%
79%
70
95%
73%
60
50
58%
91%
88%
40
70%
30
58%
20
10
0
New pre-ART
Assessed for ART eligibility
Eligible for ART
ART initiations
Cohort 1
86
50
28
10
Cohort 2
81
59
31
18
Cohort 3
95
75
40
28
Cohort 4
79
75
41
36
Cohort 5
91
90
55
50
Review data and revisit intervention plan
1. Review pre- and post-intervention cohort data
2. Identify successes and ongoing challenges
•
•
Take inventory of factors enabling program
improvement
Outline likely barriers to improvement
3. Consider revising intervention plan
•
•
Identify activities to keep in place, those to drop, and
any new activities to begin
Keep in mind sustainability of activities and
improvements
 Repeat this process as new cohort data
becomes available
Webinar Overview
1.
2.
3.
4.
5.
6.
Background
Examples of low target performance
Dimensions of the problem: M&E & Clinical
Introduce a cascade approach
A case study
Toolkit inventory
Case Study: ART Initiations
• ICAP Swaziland at end of Q3 reported reaching
50% of annual target for ART initiations
• Dimensions: M&E, Clinical
• The Cascade approach was implemented with the
following steps and results
1.
2.
3.
4.
5.
Identify steps in the cascade that relate to target
Identify baseline data to operationalize cascade
Choose priority sites
Choose interventions and prioritize them
Use a cohort methodology
1. Identify steps in the Cascade
1. # persons test HIV + (not reliable)
2. # persons enroll in HIV care
3. # persons assessed for ART eligibility (WHO,
CD4)
4. # persons eligible for ART
5. # persons initiated ART
2. Identify baseline data to operationalize cascade
ART cascade, Lubombo Region facilities, Oct 2011 - Mar 2012
200
180
160
Number patients
140
120
100
80
60
40
20
0
Enrolled in pre-ART
Had WHO stage/CD4 at baseline
ART eligible
Started ART
3. Choose priority sites
• 10 largest volume clinics in 3 regions = 30 sites
• Volume was defined as # of patients enrolling
in HIV care in the past quarter
Choose interventions and prioritize them
1. Identify patients with known ART eligibility
but no ART initiation and put them in a
“expectant” patient box for expert clients to
call to return to care
2. Introduce WHO Staging job aid to assist
providers to assess patients for ART eligibility
given reports of CD4 stock outs
3. Transfer reported CD4 results from lab
registers to patient charts
5. Introduce Cohort Methodology
• Identify steps in relevant cascade
• Specify the best source of data for each step
– Pre-ART register, patient HIV medical care file
• Design simple tools (paper, Excel) for abstracting and
summarizing this data
– Excel sheet for data collection/management
– Graph to display cascade data over time
• Identify cohort members
– Cohorts will be defined by month of pre-ART enrollment
• For this presentation, initial baseline cohort will include 3 months
combined
• Expect to see changes prospectively and retrospectively
• Plan for periodic data collection
Pre-ART enrollment, ART eligibility, and ART initiations
3500
Number of patients
3000
Intervention
roll-out begins
90%
2500
2000
1500
Additional post-intervention
cohort data to-be collected
80%
66%
1000
500
0
Jun-Aug cohort
New pre-ART
3217
Assessed for ART eligibility
2892
Eligible for ART
1703
ART initiations
1124 1356
Sep cohort
Oct cohort
Nov cohort
Dec cohort
Supplemental M&E Component:
Verifying national M&E data
• Collection of cascade data from sites allowed us to
re-count national reported M&E data
• Recount of site-level ART initiations showed a
substantial, systematic undercount in the national
M&E data (generated by MOH database)
• Have since implemented a system for identifying
patients not counted in M&E system, and having
their information entered into MOH database
• Also working towards improving routine M&E
processes so all patients are entered into database
• Discrepancy highlights need for routine conduct of
in-depth data quality assessments (DQA)
Re-count of ART initiation data
FY12 Quarterly ART initiations: ICAP supported sites
3,000
2,500
2,000
1,500
ICAP verified data
MOH routine M&E data
1,000
500
ICAP verified data
MOH routine M&E data
Q1
1,883
1,591
Q2
2,364
1,998
Q3
2,426
2,050
Summary 1: Results after 3 Months
• Recall that ICAP Swaziland had reached only
50% of annual ART initiations target by the
end of Q3
• Combined M&E and clinical efforts during Q4
allowed team to report reaching 81% of the
target by project year end
• Findings from efforts informed target-setting
for current year
Summary 2: work is an ongoing process
1. Identify successes and ongoing challenges
–
–
–
Lack of SOP for expert clients calling back ART-eligible
patients  develop SOP
Providers not listing f/u appt in chart or register 
investigate frequency and cause in 10 clinics (3 per
region)
Data still not systematically getting from primary clinics to
central clinics  task team with MOH
2. Revisit intervention plan
•
•
Identify activities to keep in place, those to drop, and any
new activities to begin
Keep in mind sustainability of activities and
improvements
Webinar Overview
1.
2.
3.
4.
5.
6.
Background
Examples of low target performance
Dimensions of the problem: M&E & Clinical
Introduce a cascade approach
A case study
Toolkit inventory
Toolkit
1.
2.
3.
4.
5.
6.
Cascade Approach Overview
Cohort Methodology
Driver Diagram
Focusing Matrix
URS Reports
DQA SOP
Acknowledgements
• Country team staff who are conducting
cascade approach
• Especially ICAP in Swaziland who have seen
much success
Thank you!