Rewarding Performance:
Three-Year Results from California's
Statewide Pay-for-Performance
Experiment
Cheryl L. Damberg, PhD, Kristiana Raube, PhD, Stephanie
Teleki, PhD, and Erin dela Cruz
June 5, 2007
Financial support provided by the California Healthcare Foundation
Presentation Topics
 IHA Pay-for-Performance program design
 Year-to-year changes in performance scores
 Physician group responses to P4P post 3rd
incentive payment
 Conclusions
Academy Health, 2007
Evaluation of the IHA P4P Program
 A 5-year evaluation to assess the impact of
the IHA P4P program on:
 Changes in performance over time
 Changes in payments and the distribution of
payments over time
 The relationship between structural
characteristics and performance scores
 Physician group responses to the incentive
program
 Leadership interviews with physician groups
Academy Health, 2007
IHA P4P Program
 A statewide collaborative effort among:
 7 major health plans and 225 medical groups
 12 million commercial HMO and POS enrollees
 Measurement started in 2003 for 1st payout in 2004
 3rd payout occurred late summer 2006
Design Elements
Unit of payment
Medical groups (n=225)
# of measures
17 (clinical, patient experience, IT capability)
Data source
Administrative (plan or medical group self-report)
Min of 3.25 encounters PMPY
Earning potential
Avg. bonus of 2-3% of cap (~$2.50 per member
per month)
Scoring method
Most plans use relative rankings
Transparency
Full transparency
Academy Health, 2007
Performance Measures
MY Year 2005, Payout 2006
Clinical
 Asthma management
 Childhood immunization (MMR, VZV)
 Cancer screening (breast, cervical)
 Diabetes (HbA1c measure and control)
 LDL (screening and control: 03 cardiac; 04 cardiac and
diabetic)
Patient Experience




Timely access to care
Doctor-patient interaction/communication
Specialty care
Overall ratings of care
IT Capability
 Integrate clinical electronic data for population management
 Clinical decision making support at point of care through
electronic tools
Academy Health, 2007
Weighting of Measures in Payout Formula
Payout Year
2004
2005
2006
2007
Clinical Measures
50%
40%
50%
50%
Patient Experience with Care
40%
40%
30%
30%
IT Capabilities (add systemness
measures in 2007)
10%
20%
20%
20%
Total
100%
100%
100%
100%
x
x
Individual physician Feedback program
(optional add on bonus)
Year-to-year improvement (optional
in 06; begins 07 for all plans)
x
Academy Health, 2007
Changes in Payouts: 2004-2006
∆=47% increase in IHA portion
$160.0
$137.1
$119.5
$144.6
$120.0
Millions of
Dollars
$83.4
$80.0
$89.5
$82.0
$55.0
$40.0
$37.4
$53.7
$0.0
2004
IHA Payouts
2005
Non-IHA Payouts
2006
Total Payouts
Academy Health, 2007
Total Payments to Physician Organizations*
2004 vs. 2005
$1,600,000
$1,400,000
$1,200,000
2005
$1,000,000
$800,000
$600,000
$400,000
$200,000
$0
$0
$200,000
$400,000
$600,000
$800,000
$1,000,000
$1,200,000
$1,400,000
2004
* Note: Truncated to groups receiving less than $2 million
Academy Health, 2007
3-Year Performance Changes
2003 (2004 payout) to
2005 (2006 payout)
Academy Health, 2007
Modest Changes in Patient Experience Scores
2003
2004
Mean
Difference
Rating of Health Care
70.0%
71.4%
1.4%**
Rating of Doctor
80.0%
80.7%
0.5%
Rating of Specialist
71.0%
71.9%
0.8%
Doctor Communication
85.6%
87.0%
1.3%***
Timely Care and Access
69.5%
70.2%
1.4%***
No Problem Seeing Specialist
59.5%
61.3%
2.2%***
Measure
Statistically significant at *** p<.001 ** p < .01; * p < .05
Academy Health, 2007
Asthma: All Ages
100
90
21% point gain in
performance
80
Mean Score
70
Overall Mean
60
Quartile 1
50
Quartile 2
40
Quartile 3
Quartile 4
30
20
Reduction of 5.6% points in variation
10
0
2003
2004
2005
Measurement Year
Academy Health, 2007
Breast Cancer Screening
100
90
3.5% point gain in
performance
80
Mean Score
70
Overall Mean
60
Quartile 1
50
Quartile 2
40
Quartile 3
30
Quartile 4
20
Reduction of 2.3% points in variation
10
0
2003
2004
2005
Measurement Year
Academy Health, 2007
HbA1c Screening
100
7.7% point gain in
performance
90
80
Mean Score
70
Overall Mean
60
Quartile 1
50
Quartile 2
40
Quartile 3
Quartile 4
30
20
Reduction of 19.8% points in variation
10
0
2003
2004
2005
Measurem ent Year
Academy Health, 2007
Diabetes HbA1c Screening: 2004 vs. 2005
100
90
80
70
2005
60
50
40
30
20
10
0
0
20
40
60
80
100
2004
Academy Health, 2007
Breast Cancer Screening: 2004 vs 2005
100
90
80
70
2005
60
50
40
30
20
10
0
0
20
40
60
80
100
2004
Academy Health, 2007
IT adoption increases each year
Measurement Year 2003
Measurement Year 2004
Measurement Year 2005
Number of Groups
100
90
80
70
60
50
40
30
20
10
0
Patient Registry
Actionable Reports
HEDIS Results
By 2005, 33-44% of Groups and 68-76% of
Patients Had Data Integration Technology
Academy Health, 2007
More IT Functions are Adopted
90
80
70
60
Number of
Groups
50
40
30
20
10
0
Electronic
Prescribing
Electronic
Check of
Prescription
Interaction
Measurement Year 2003
Electronic
Retrieval of
Lab Results
Electronic
Access of
Clinical Notes
Electronic
Retrieval of
Patient
Reminders
Measurement Year 2004
Accessing
Clinical
Findings
Electronic
Messaging
Measurement Year 2005
By 2005, 1-39% of Groups; 20-64% of
Patients had Point of Care Technology
Academy Health, 2007
Physician Organization Responses to
Pay for Performance:
Findings from Leadership Interviews
Academy Health, 2007
Physician Organization Responses to the
Incentive Program
 Second round of interviews with physician
leadership (3 years into program)
 Study population: 35 physician
organizations (POs) out of a universe of 225
in CA (n=29 completed to date)
 Cross section of groups
 High, medium, and low performing Pos
 Reflects the spectrum of “winners and
losers”
 Large and small POs
 Reflects resource constraints
 Rural and urban POs
Academy Health, 2007
Support Quality Focus, but Face Constraints
• Most said the organization provides
support to addressing quality
 Mean score = 4.0 (1 to 5 scale, with 5 = a lot of support)
• Biggest constraints to improving quality:
 Technology challenges, such as lack of EMR
 Changing physician behavior
 Data issues, such as data integration, missing
information, etc.
• POs feel they are moderately successful in
monitoring their quality performance
 Mean score=3.7 ( 1-5 scale, with 5 = very successful)
Academy Health, 2007
Is the Current Incentive Level of 1-2% of
Capitation Right?
 Among those earning incentives, the amount was 2%
or less as a percentage of total capitation payments
 Mixed results on +/- ROI
 Widespread support for increasing incentives to 510% of capitation payments (26 out of 29 POs
agreed)
 This level would increase attention, provide a positive ROI
and defray set-up costs
 Some POs noted current levels have gotten their attention
and urged them to make changes
Academy Health, 2007
Most POs Believe P4P Affects Organizational
and Physician Behavior
•
•
Increased organizational accountability for quality
 New project managers, quality support, and medical
directors
Improvements in data collection, including IT adoption
 IT and data support staff
 Data mining capabilities
•
 EMRs, hardware, software, and web interfaces
Physicians are more directly managing patients and
working with administration to improve quality
 Bonuses tied to quality
 Outreach to physicians; clinical and patient satisfaction
guideline review
Academy Health, 2007
Conclusions
 Modest positive changes occurring for most
measures
 Combination of quality improvements and improvements in
data capture
 Data capture continues to challenge small groups and some
IPAs
 Challenges of how to improve patient experience
 Performance payments have grown slowly over time
 $$ at risk for performance are still a small fraction of total
payments
 Current level of incentives isn’t high enough to really get
attention of physicians
 Hard to incentivize specialists given absence of
measures
Academy Health, 2007
Will P4P Solve the Cost and Quality Problems
in the U.S. Health System?
 Improving the reliability of care received from current
level of one-sigma to six-sigma?
 Slowing the growth in healthcare costs to the rate of
growth in the GDP or general level of inflation?
 Reducing the number of deaths from medical errors
from estimated rate of >100,000/year to below
5,000/year?
 Unlikely in near term
 Need for other policy levers in conjunction with
P4P (e.g., broader performance measurement,
transparency, investments in information systems)
Academy Health, 2007