SOME THOUGHTS
ABOUT MONITORING THE
PERFORMANCE OF THE “SAFETY NET”
February, 2007
John Billings
NYU Center for Health and Public Service Research
Robert F. Wagner Graduate School of Public Service
WHAT I’M GOING TO TALK ABOUT
• Why the focus on “performance” of the safety net?
– Some caveats and definitions
– Some assumptions
• Some examples of using administrative data to
monitor performance
• The limitations of using administrative data
• A few suggestions (unsolicited advice)
SOME CAVEATS AND DEFINITIONS
• The focus of policy should be on: Assuring optimal
health for vulnerable populations
• We need to worry about the resources required to
assure optimal health of vulnerable populations
• These resources are the “safety net”
• Because resources are limited, it makes sense to
examine the performance of this “safety net”
• But it is important to remind ourselves that this isn’t really
a “safety net”
– We are flying without a net
– No one is particularly safe
SOME ASSUMPTIONS
• Texas is unlikely to enact a universal coverage initiative
this year, or next year, or the year after that…
• There are lots of opportunities to improve health of
vulnerable populations in addition to buying coverage
or subsidizing care
• Therefore, it is critical to have a monitoring capacity
• There is probably not a lot of money around for
monitoring things
• But it is critical to recognize the inherent limits of
administrative data
COMPUTERIZED
HOSPITAL DISCHARGE AND
ED VISIT DATA
Preventable/Avoidable Hospitalizations
Ambulatory Care Sensitive (ACS) Conditions
Conditions where timely and effective ambulatory care help
prevent the need for hospitalization
• Chronic conditions – Effective care can prevent flare-ups (asthma,
diabetes, congestive heart disease, etc.)
• Acute conditions – Early intervention can prevent more serious
progression (ENT infections, cellulitis, pneumonia, etc.)
• Preventable conditions – Immunization preventable illness
ACS Admissions/1,000
By Zip Code Area Income
New York City - Age 18-64 - 2004
50
Adms/1,000
45
R2 = .622
LowInc/HiInc = 3.65
Coef Vari = .536
Mean Rate = 16.08
40
35
30
25
20
15
10
5
Each
represents a zip code
0
0%
10%
20%
30%
40%
Percent of Households with Income <$15,000
Source: NYU Center for Health and Public Service Research
50%
60%
New York City
ACS Admissions/1,000
Age 18-64 - 2004
ACS Admissions/1,000
Age 18-64 - 2004
25 to 47
18 to 25
12 to 18
8 to 12
4 to 8
Unpopulated Areas
Source: NYU Center for Health and Public Service Research
(29)
(27)
(53)
(39)
(26)
(3)
NYU EMERGENCY DEPARTMENT
CLASSIFICATION ALGORITHM 1.0
Not preventable/avoidable
ED Care Needed
Preventable/avoidable
Emergent
Primary Care Treatable
Non-Emergent
New York City
ED Utilization Profile
Adults Age 18-64 - 1998
ED Needed
Preventable Avoidable
7.1%
Emergent - Primary
Care Treatable
34.4%
Source: NYU Center for Health and Public Service Research - UHFNYC
ED Needed - Not
PreventableAvoidable
18.8%
Non-Emergent
39.7%
Bronx
Percent of
Non-Admitted
Emergency Department Visits
That Are "Non-Emergent"
Medicaid - 1998
All Ages
Queens
Manhattan
Brooklyn
Staten Island
% ED Visits Non-Emergent
Medicaid - 1998
45% to 54%
42% to 45%
40% to 42%
20% to 39%
Unpopulated Areas
(28)
(67)
(64)
(15)
(3)
UNDERSTANDING THE CAUSES OF
VARIATION IN ACS RATES
AND ED USE
• Theory 1: It’s just New York City
– [Who cares]
– [You’re more or less a different country]
ACS Admissions/1,000
By Zip Code Area Income
Baltimore - Age 18-64 - 1999
60
R2 = .899
LowInc/HiInc = 3.90
Mean Rate = 16.93
Adms/1,000
50
40
30
20
10
Each
represents a zip code
0
0%
10%
20%
30%
40%
% Housholds Income < $15,000
Source: NYU Center for Health and Public Service Research
50%
60%
ACS Admissions/1,000
By Zip Code Area Income
St. Louis - Age 18-64 - 1999
60
R2 = .870
LowInc/HiInc = 3.50
Mean Rate = 12.53
Adms/1,000
50
40
30
20
10
Each
represents a zip code
0
0%
10%
20%
30%
40%
% Housholds Income < $15,000
Source: NYU Center for Health and Public Service Research
50%
60%
ACS Admissions/1,000
By Zip Code Area Income
Memphis - Age 18-64 - 1999
50
R2 = .887
LowInc/HiInc = 2.95
Mean Rate = 14.45
Adms/1,000
40
30
20
10
Each
represents a zip code
0
0%
10%
20%
30%
40%
50%
% Housholds Income < $15,000
Source: NYU Center for Health and Public Service Research
60%
70%
ACS Admissions/1,000
By Zip Code Area Income
San Diego - Age 18-64 - 1999
30
R2 = .650
LowInc/HiInc = 3.09
Mean Rate = 7.16
Adms/1,000
25
20
15
10
5
Each
represents a zip code
0
0%
10%
20%
30%
% Housholds Income < $15,000
Source: NYU Center for Health and Public Service Research
40%
50%
ACS Admissions/1,000
By Zip Code Area Income
HOUSTON MSA - Age 18-64 - 2002
80
R2 = .561
LowInc/HiInc = 2.71
Mean Rate = 14.57
70
Adms/1,000
60
50
40
30
20
10
Each
represents a zip code
0
0%
10%
20%
30%
40%
50%
% Housholds Income < $15,000
Source: NYU Center for Health and Public Service Research
60%
70%
ACS Admissions/1,000
By Zip Code Area Income
Denver - Age 18-64 - 2002
30
Adms/1,000
R2 = .709
LowInc/HiInc = 2.61
Mean Rate = 9.10
20
10
Each
represents a zip code
0
0%
5%
10%
15%
20%
25%
% Housholds Income < $15,000
Source: NYU Center for Health and Public Service Research
30%
35%
ACS Admissions/1,000
By Zip Code Area Income
Portland, OR - Age 18-64 - 1999
40
R2 = .739
LowInc/HiInc = 4.26
Mean Rate = 7.69
Adms/1,000
30
20
10
Each
represents a zip code
0
0%
10%
20%
30%
% Housholds Income < $15,000
Source: NYU Center for Health and Public Service Research
40%
50%
SOUTH CAROLINA
ED Utilization Profile
Adults Age 18-64 - 1997
ED Needed
Preventable Avoidable
7.1%
Emergent - Primary
Care Treatable
42.3%
Source: NYU Center for Health and Public Service Research
ED Needed - Not
PreventableAvoidable
18.8%
Non-Emergent
31.9%
Preventable/Avoidable ED Use/1,000
By Zip Code Area Income
Austin Metro Area - Age 0-17 - 2000
Austin Metro Area
Preventable/Avoidable ED Visits Per Capita
Children - Age 0-17 - 2001
300 to 546
200 to 300
120 to 200
60 to 120
12 to 60
Low Population Area*
(17)
(13)
(16)
(14)
(15)
(4)
UNDERSTANDING THE CAUSES OF
VARIATION IN ACS RATES
AND ED USE
• Theory 1: Who cares? It’s just New York
• Theory 2: It’s really pretty complicated
–
–
–
–
–
Coverage barriers
Resource supply/capacity
Economic barriers
Provider performance
Quasi-economic barriers
• Transportation
• Child care
• Lost wages
–
–
–
–
–
Barriers to social care
Limitations in community social capital
Limitations in personal social capital
Education, motivation, confidence, health beliefs
Physician practice style (Wennberg et al), etc, etc
ACS Admissions/1,000
Zip 10016 and Citywide Rates
New York City - Age 0-17 – 1982-2001
Adms/1,000
65
60
55
50
45
40
35
30
25
20
15
New York
City
10
Zip 10016
5
0
82
83
84
85
86
87
88
89
90
91
92
93
Source: SPARCS - NYU Center for Health and Public Service Research - UHFNYC
94
95
96
97
98
99
00
01
ACS Admissions/1,000
By Zip Code Area Income
New York City - Age 18-64 - 2002
50
Adms/1,000
45
R2 = .622
LowInc/HiInc = 3.65
Coef Vari = .536
Mean Rate = 16.08
40
35
30
25
20
15
10
5
Each
represents a zip code
0
0%
10%
20%
30%
40%
Percent of Households with Income <$15,000
Source: NYU Center for Health and Public Service Research
50%
60%
Bronx
Percent of
Non-Admitted
Emergency Department Visits
That Are "Non-Emergent"
Medicaid - 1998
All Ages
Queens
Manhattan
Brooklyn
Staten Island
% ED Visits Non-Emergent
Medicaid - 1998
45% to 54%
42% to 45%
40% to 42%
20% to 39%
Unpopulated Areas
(28)
(67)
(64)
(15)
(3)
Bronx
Percent of
Non-Admitted
Emergency Department Visits
That Are "Non-Emergent"
Selfpay/Uninsured - 1998
All Ages
Queens
Manhattan
% ED Visits Non-Emergent
Brooklyn
Staten Island
Self pay /Uninsured - 1998
45% to 54%
42% to 45%
40% to 42%
22% to 39%
Unpopulated Areas
(19)
(51)
(75)
(29)
(3)
ACS Admissions/1,000
By Zip Code Area Income
Miami - Age 18-64 - 1999
50
R2 = .330
LowInc/HiInc = 1.89
Mean Rate = 14.82
Adms/1,000
40
30
20
10
Each
represents a predominantly Cuban-American zip code
0
0%
10%
20%
30%
40%
50%
% Housholds Income < $15,000
Source: NYU Center for Health and Public Service Research
60%
70%
ACS Admissions/100,000
By Ward Code and Deprivation Index
London, UK - Age 15-64 - 2001/2-2002/3
2500
R2 = .387
HighDI/LowDI = 2.10
Mean Rate = 881.0
Adms/100,000
2000
1500
1000
500
Each “♦” represents a ward
0
0
10
20
30
40
50
Deprivation Index
Note: All data are for 2001/2 and 2002/3
60
70
80
ACS Admissions/1,000
Low and High Income Areas
Admissions Per 1,000
New York City MSA – Age 40-64
Adms/1,000
80.0
70.0
Low Income
Areas
60.0
50.0
40.0
30.0
High Income
Areas
20.0
10.0
0.0
82 83 84 85 86 87 88 89 90 91 92
Source: NYU Center for Health and Public Service Research
93 94 95 96 97 98 99 00 01 02 03 04
ACS Admissions/1,000
Low and High Income Areas
Admissions Per 1,000
New York City – Age 0-17
Adms/1,000
40.0
30.0
$50,000,000
Low Income
Areas
20.0
High Income
Areas
10.0
0.0
82 83 84 85 86 87 88 89 90 91 92
Source: NYU Center for Health and Public Service Research
93 94 95 96 97 98 99 00 01 02 03 04
WHAT’S GOING ON HERE?
1. It isn’t anything
2. It is something:
• It’s an improvement in clinical medicine (e.g., asthma)
• Changes in composition of NYC’s low income population
• It’s related to changes in the factors that contribute to
health disparities
–
–
–
–
–
Coverage expansion (???)
Supply expansion (???)
Service improvement: greater “competition for patients”
Changes in social context
Etc, etc, etc…
Change in ED Visits/1,000
New York City
Medicaid FFS – ADC/HR Girls Age 6mos-14yrs
1994-1999
% Change (Log Scale)
+100% -
+50% -
+25% -
Injuries
-20% -
Asthma
ACS - No
Asthma
-33% -
-50% 94
95
Source: NYU Center for Health and Public Service Research
96
97
98
99
Change in Percent of ED Visits Resulting In Admission
New York City
Medicaid FFS – ADC/HR Girls Age 6mos-14yrs
1994-1999
% Change (Log Scale)
+100% -
+50% -
+25% -
Asthma
ACS - No
Asthma
-20% -
Injuries
-33% -
-50% 94
95
Source: NYU Center for Health and Public Service Research
96
97
98
99
ACS Admissions/1,000
Low Income Areas
New York MSAs - Age 0-17
Adms/1,000
40.0
35.0
30.0
25.0
New York City
20.0
15.0
Syracuse
Buffalo
10.0
Rochester
5.0
0.0
82
83
84
85
86
87
88
89
90
Source: NYU Center for Health and Public Service Research
91
92
93
94
95
96
97
98
99
00
01
02
ACS (W/o Asthma) Admissions/1,000
Low Income Areas
California MSAs and New York City - Age 0-17
Adms/1,000
40.0
35.0
30.0
25.0
New York City
20.0
Oakland
15.0
Los Angeles
San Diego
San Francisco
10.0
5.0
0.0
83
84
85
86
87
88
89
90
Source: NYU Center for Health and Public Service Research
91
92
93
94
95
96
97
98
99
00
01
02
USING MEDICAID CLAIMS DATA
TO MONITOR PROVIDER PERFORMANCE
OUR APPROACH
• We examined fee-for-service paid Medicaid claims
• Patients are linked to their primary care provider
– Linking based on primary care visits (not ED or specialty care)
– Patients with 3+ primary care visits linked to provider having the majority of
primary care visits [“predominant provider’]
– Patients with fewer than 3 visits examined separately
• Performance of providers for their patients is then examined
GETTING BEYOND ADMINISTRATIVE DATA
IN MONITORING THE SAFETY NET
So If “Provider Performance” Matters…
What Factors Influence “Provider Performance?
•
•
•
•
•
•
•
•
•
•
•
•
•
•
Hours of operation (?)
“Cycle time” (?)
Wait time for appointment (?)
Language barriers (?)
Doctor-patient interaction [respect, courtesy, communication] (?)
Staff-patient interaction [respect, courtesy, communication] (?)
Content of care: doctor skill (?)
Content of care: patient education on self-management (?)
Staffing mix (MD type, nurse practitioner, etc.)
Staffing mix (use of medical residents)
Patient “outreach” (?)
Easy telephone access (?)
MIS systems [notification that patient is in ED] (?)
Etc, etc, etc.
Factors That Matter to Patients
“I Would Recommend This Place to My Friends”
• Things that matter most
–
–
–
–
The facility is pleasant and clean
I saw the doctor I wanted to see
The office staff were respectful and courteous
The doctor was respectful and courteous
• Things that matter somewhat
–
–
–
–
The office staff explained things in a way I could understand
The location is convenient for me
I waited a short time to see the doctor
It is easy to get an appointment when I need it
• Things that don’t seem to matter as much
–
–
–
–
The doctor spent enough time with me
The doctor/nurse/office staff listened to me carefully
It is easy to get advice by telephone
The hours are convenient
Source: NYU Center for Health and Public Service Research
FINDINGS FROM INTERVIEWS OF
ED PATIENTS
• Most patients wait a considerable amount of time before
heading to the ED
• But they are unlikely to have contacted the health care
delivery system before the visit
• Convenience is the leading reason for ED use
• Many are not well-connected to the health system
Source: NYU/UHF survey of ED patients in 4 Bronx hospitals - 1999
FINAL THOUGHTS ABOUT
MONITORING THE SAFETY NET
• It is critical to know…
– Are things getting better or worse?
– What are the biggest problems?
– Where are the biggest problems?
• Support evidence-based policy making - Use data to:
–
–
–
–
–
Identify the areas and populations in greatest need
Understand the nature and characteristics of that need
Assess impact of interventions
Learn from natural experiments
Get answers for some of things we don’t know
• Oh, and talk to patients once in a while
– They know what they want better than you do
– It is important to understand what’s driving their use patterns