Perfecting Performance Measurement in
a Private Sector System
Joe V Selby MD MPH
Division of Research
Kaiser Permanente Northern CA
Goals of This Presentation
• To Describe:
– Development of a Better Performance Metric
– Our Approach to Improving Performance
– Understandings on the Target of Performance
Improvement (i.e., physician vs. system)
Similarities between KP and VA Settings
•
•
•
•
•
Within each region, a single physician group mutually
exclusively partnered with single health plan
Primary care and specialty physicians partnered in
same physician group
Health plan owns the hospitals
Single EHR across all settings
Quality Improvement led by physician group, relies on:
– local physician champions
–
–
–
relatively intense and flexible use of data
Aggregate (not individual PCP) performance reporting
Modest use of incentives at department and medical center
level (not at individual physician level)
Potential Differences Between KP
and the VA
•
•
•
KP is a private-sector system – competes for same
group of patients (members) and physicians
KP physicians are not employees, they are partners in
medical groups that elect their leadership and negotiate
with health plan
Permanente Medical Groups have long tradition of decentralized management approach:
– Regional autonomy
– Facility autonomy
– Physician autonomy
California Cooperative Healthcare Reporting
Initiative 2008
#1 Rankings by Health Plan – All Clinical Measures
Number One Rankings Among Major Health Plans
Available in California
25
22
21
20
15
10
4
5
1
1
2
2
2
Cigna
Anthem
Blue Cross
Aetna
0
KP NCAL
KP SCAL Health Net PacifiCare
Blue
Shield
Fundamentals:
•
Monitoring and reporting simple process measures only
improves the simple processes - not the outcomes
•
Systems can use informatics to develop, measure and
report more sophisticated clinical process measures –
processes that are linked to outcomes
•
Involvement of clinicians and staff essential to understand
barriers to improvement, recognizing problems with quality
indicators
•
Interventions delivered through systems work better than
interventions aimed directly at individual physicians
20
04
20 Q4
05
20 Q1
05
20 Q2
05
20 Q3
05
20 Q4
06
20 Q1
06
20 Q2
06
20 Q3
06
20 Q4
07
20 Q1
07
20 Q2
07
20 Q3
07
20 Q4
08
20 Q1
08
20 Q2
08
20 Q3
08
Q
4
% on target medication
Current Use of Cardioprotective Medications
in High-Risk Members (n~310,000)
70
60
50
40
30
Statin Use
ACEI Use
BB Use
All Meds
KPNC PHASE Population
Treatment Intensification: Another
Evidence-based Process Measure
•
In the face of an elevated risk factor value (2 consecutive
elevated BP’s, or an elevated LDL-c or A1c), did the
clinician do one of following w/in 3 months:
– Prescribe an appropriate medication of a new class
– Increase the dose of at least one current medication
– Switch to a medication in a different class
•
Ideally, exclude patients:
– Already on maximal medical therapy
– With short life-expectancy
– With contraindications to additional med class(es)
Facility-Level Improvement in Treatment
Intensification Associated with Improvement
in SBP Control, 2001-2003 (N=35 facilities)
Change in % in Control for SBP
25
20
15
10
Spearman Corr. = .47, p=.004
5
0
0
2
4
6
8
10
12
14
16
Change in % Getting Treatment Intensification, 3 months
Selby et al, Med Care April, 2009
Linear Model* Predicting Facility Improvement
in Intermediate Outcome Control, 2001-2003
Per 1% Improvement in
Tx Intensification
P-value
Δ SBP Control
0.43 (.14)
.003
Δ LDL Control
.40 (.13)
.004
Δ A1C Control
.33 (.12)
.01
* Adjusted for patient variables and for % Tx Int. and % in Control in 2001
Selby et al, Med Care April, 2009
Intensification Feedback and Outcomes
(INFO)
• Cluster RCT in PHASE population at 8 KP medical centers
(4 intervention, 4 control facilities; ~ 13,000 eligible patients)
• Flags indicating need for treatment intensification (i.e.,
elevated SBP, A1c, LDL-c in adherent patient and no
intensification noted) placed in PHASE database
• Next recommended med or dose also placed in database
• Clinical pharmacists, nurses in each facility use this
database to outreach and follow patients
• Researchers met monthly with PHASE staff and clinicians for
6 months before intervention and during the 6-months of
intervention
INFO Study Preliminary Results
(6 months post-Intervention)
Outcomes
Control
Study
P-value
Facilities Facilities
TI In Eligible Patients:
SBP >140
LDL-c > 130
A1c >9%
51.0%
39.2%
43.9%
48.2%
36.7%
44.8%
Eligible Pts “Touched”
~43%
--
% Control @ Follow-up
SBP <140 mmHg
LDL-c <130mg/dL
A1c < 9%
55.5%
31.0%
37.1%
55.8%
33.9%
42.8%
0.02
0.08
0.67
-0.04
<0.01
Observations During Intervention
•
•
•
•
Within 3 months, intervention staff expressed frustration
that patients we were flagging were already in process,
thus creating double work
Intervention now modified to focus more narrowly on
those patients who are not intensified by the current
system within 2-3 months
Overall, control rates were 70-80% for all 3 risk factors in
both intervention and control facilities, and 40-50% of
patients intensified within 3 months. Failure to intensify
treatment applies to only a small minority of this
population.
In this context, its essential to be engaged with
clinicians/staff and to be cautious about apparent poor
performance, and ceiling effects
Examining Variation in Performance
• Where is the variation – at the physician
level or the facility level?
• Does it change as quality improves?
• What can it tell us about directing
Performance Improvement Interventions
Examining Variation in Performance
•
KP directs its QI programs at medical facilities, providing
data, modest facility-level incentives and collaboration.
Facilities respond in diverse ways, using different staffing
and strategies – “playing to their strengths.”
•
We monitored variance in 4 quality indicators from 2001
through 2006 as quality improved for each
•
Used multi-level models to measure ICCs for physicians
and facilities, which were uniformly below .08, usually
below .03, for all measures, both levels, all years
•
Also measured absolute variances at each level each
year, expressed as 10th-90th percentiles of range after
case-mix adjustment
Mean Systolic Blood Pressure (mm Hg)
150
130
140
120
Variance Changes at Facility and Physician
Level as SBP Control Improves
facilities
physicians
2001
2002
2003
2004
Physicians:10th-90th percentiles
Facilities:10th-90th percentiles
2005
Regional Mean
2006
Variance Changes at Facility and Physician
Level as Satisfaction with Care Improves
Mean Care Experience Score
4
4.5
5
facilities
3.5
physicians
2001
2002
2003
2004
Physicians:10th-90th percentiles
Facilities:10th-90th percentiles
2005
Regional Mean
2006
Proportion Receiving Screening Mammography
.5
.6
.7
.8
.9
1
Variance Changes at Facility and Physician
Level as Mammography Screening Improves
facilities
physicians
2001
2002
2003
2004
Physicians:10th-90th percentiles
Facilities:10th-90th percentiles
2005
Regional Mean
2006
140
Variance Changes at Facility and Physician
Level as LDL-c Control Improves
Mean LDL-Cholesterol (mg/dL)
110
120
130
facilities
100
physicians
2001
2002
2003
2004
Physicians:10th-90th percentiles
Facilities:10th-90th percentiles
2005
Regional Mean
2006
Conclusions
•
•
•
•
•
Low ICCs do not mean low variability, nor low potential for
quality improvement – just difficulties in measuring
individuals precisely
As quality improves, we see variation decline for 3 of 4
measures, with greater declines at facility levels
Supports the concept that organized systems are good
targets for QI
Leaves open question of how to deal with important
variation that can’t be precisely measured at individual
provider level
Suggests a “velvet glove” approach – rich in data and
discussion, but low on judgment and individual-level
incentives