VA Quality Measurement:
Is There a “Halo Effect?”
Steven M.Asch MD MPH
VA Sepulveda HSR&D COE
RAND Health
Geffen School of Medicine at UCLA
VA Performance Measurement
• Dates back to 1995
• ~30 mostly chart-based process measures,
varying evidence support
• Facility-based primary care sampling
• Feedback, public reporting
• Regional managers have financial
incentives
VA Improves 12/13 Leading
Indicators 1995- 2000
100
90
80
70
60
1995
1996
1998
1999
2000
50
40
30
20
10
0
Hgba1c
Flu vac
CRC Screen
ASA MI
Jha NEJM 348:22 2003
VA Beats Medicare 12-1 in 2000
100
90
80
70
60
50
Medicare
VA
40
30
20
10
0
Mammo
Pneumovac
DM eye
Jha NEJM 348:22 2003
What About Overall Quality?
• Leading indicator systems
– Focus QI efforts, but
– Vulnerable to gaming, resource
misallocation
• What about overall quality?
• How does VA overall quality
compare to the community?
Overall Process Quality
Measured by QATools
• Literature reviews conducted to develop
evidence-explicit indicators
• 5 Delphi expert panels convened to evaluate
recommended indicators
• 348 clinically detailed process indicators for 26
clinical areas
• Example: Patients with pre-existing coronary
disease with an LDL > 130mg/dl should begin
diet or drug therapy within 3 months.
Example Clinical Areas
General Medical Conditions
• BPH (4)
• Depression (14)
• Diabetes (13)
• Dyspepsia/PUD (8)
• Headache (21)
• Low back pain (6)
• Orthopedic conditions (10)
• Preventive care (35)
• STDs (26)
Oncologic Conditions
• Colorectal cancer (12)
• CA pain and palliation (3)
• Prostate cancer (6)
Cardiopulmonary Conditions
• Asthma (25)
• Congestive heart failure (36)
• COPD (20)
• Coronary artery disease (37)
• Hyperlipidemia (7)
• Hypertension (26)
• Pneumonia (5)
• Stroke/TIA (10)
Study Sites
•
•SEATTLE
•
SYRACUSE BOSTON
•
LANSING
•
VISN 11
CLEVELAND
•
•
•
NEWARK
INDIANAPOLIS
VISN 22
ORANGE
COUNTY
•
•
PHOENIX
LITTLE
ROCK
•
GREENVILLE
•
•
MIAMI
% Receiving Indicated Care
80
73*
67*
70
60
53
64*
58
50
56
53
44
40
Comm
VA
30
20
10
0
Overall
Chronic
Prev
Acute
% Receiving Indicated Care
90
81*
80
70
69
59
68*
63
60
52
50
Comm
VA
40
30
20
10
0
COPD
HTN
DM
Is There A “Halo Effect?”
• Does the VA advantage exist
only in specific processes
subject to VA performance
management?
• Does it extend to clinically
related conditions or areas?
• Does it extend to clinically
unrelated areas?
P. Gaugin-”Self Portrait with a Halo”
3 Mutually Exclusive Summary
Scores
 In VA measurement set (26 indicators)
 Not in VA measurement set, but in same
condition (178 indicators)
 Neither (167 indicators)
Adjusted % Receiving
Indicated Care
80
67*
70
60
58
53
50
71*
69*
51 54
42
40
Comm
VA
30
20
10
0
Overall
VA
VA
Neither
Measures Conditions
Conclusions
•
•
•
•
VA is improving on leading indicators
VA beats Medicare on leading indicators
VA beats community on broader measures
“Halo effect” of performance measurement
diminishes with clinical distance
Wins: Performance measurement
Losses: ?????
Ties: Acute care, eye exams, COPD
Errors: Hoping for too big a halo
Questions
Adjusted RR of Indicated Care
2.00
2.74*
1.75
2.04*
1.50
1.25
1.22*
1.00
0.75
0.50
0.25
0.00
VA Performance
Measures
VA Performance
Measure Conditions
Non-VA Performance
Measure Conditions
Sampling: Men>35 with 2 Visits/yr
Community
VA
Admin data
Response rate
Random digit
dial
48%
N
992
597
Study period
1/98-12/99
1/98-12/99
Sites
12 cities
26 facilities
Source
97%