Research Objectives
Did the Premier Demonstration Project Make a
Difference?: Assessing the National Impact of P4P and
Public Reporting Initiatives
Presented at AcademyHealth 2007
Annual Research Meeting
Orlando, FL
June 5, 2007
Stephen R. Grossbart, Ph.D.
Vice President, Quality Management
Catholic Healthcare Partners
srgrossbart@health-partners.org
Copyright 2007, Catholic Healthcare Partners
• Identify the impact of the Center for Medicare
and Medicaid Services’ (CMS) “Premier
Hospital Quality Incentive Demonstration
Project” (HQID) on quality improvement
compared to other hospitals that participated in
the Hospital Quality Alliance’s (HQA) public
reporting initiative between 2004 and 2006.
Copyright 2007, Catholic Healthcare Partners
Previous Research
Improvements continue beyond Year 2
CMS/Premier HQID Project Participants Composite Quality Score:
Trend of Quarterly Median (5th Decile) by Clinical Focus Area
October 1, 2003 - June 30, 2006 (Year 1 and Year 2 Final Data, and Yr 3 YTD Preliminary)
105%
70.00%
73.13%
75%
70%
65%
60%
AMI
CABG
85.13%
86.87%
89.0%
90.5%
91.63%
93.40%
95.20%
95.92%
96.05%
96.89%
97.50%
73.1%
76.2%
78.22%
81.57%
82.98%
84.38%
86.73%
88.79%
90.00%
80%
Pneumonia
Heart Failure
Hip and Knee
Clinical Focus Area
Q4-03
Q1-04
Q2-04
Q3-04
Q4-04
Q1-05
Q2-05
Q3-05
Q4-05
Q1-06
Grossbart, 2006 Med Care Res Rev, Mar 2006:
•Heart failure composite score improved 19% in acute care hospitals within a
single multi-hospital system
•Demonstration participants outperformed matched cohort in “non-starter set”
HQA measures
64.10%
68.11%
85%
78.3%
80.0%
82.49%
82.72%
84.81%
86.30%
88.54%
89.28%
90.09%
90%
85.14%
85.92%
88.9%
90.0%
93.70%
94.89%
96.16%
97.01%
96.77%
98.28%
98.44%
Composite Quality Scor
95%
89.88%
90.06%
91.5%
92.6%
93.50%
93.36%
95.08%
95.77%
95.98%
96.14%
96.84%
100%
Q2-06
Richard A. Norling. How hospitals and health systems fit into the pay for performance puzzle.
National Pay for Performance Summit. Beverly Hills, CA, Feb 15, 2007
Copyright 2007, Catholic Healthcare Partners
•“The alignment of financial incentives with performance has significantly
affected the rate of quality improvement, resulting in an overall higher quality
of care”
Copyright 2007, Catholic Healthcare Partners
Latest findings on Premier P4P NEJM, Feb 1, 2007:
•Participants improved 2.6% to 4.1% more than matched control
group for 10 HQA “starter set” measures
•“Financial incentives can modestly increase improvement in
quality among hospitals already engaged in public reporting.”
Lindenauer PK, Remus D, Roman S, Rothberg MB, Benjamin EM, Ma A, Bratzler DW. Public
reporting and pay for performance in hospital quality improvement. N Engl J Med 2007; 356:486-96
Copyright 2007, Catholic Healthcare Partners
Lindenauer PK, Remus D, Roman S, Rothberg MB, Benjamin EM, Ma A, Bratzler DW. Public
reporting and pay for performance in hospital quality improvement. N Engl J Med 2007; 356:486-96
Copyright 2007, Catholic Healthcare Partners
1
What are we looking for from P4P?
Study Design
Placing “A Few Simple Rules” in Context
(Refreshing a Highly Viral PowerPoint Slide)
Ï Ï Value Gain
by Americans
Efficiency of Health
Benefits Spending
(Health Gain / $)
High
Ï Ï Clinical
Performance
Management
Ï Ï Market
Sensitivity to
Performance
Ï Ï Transparency
on Quality &
Cost-Efficiency
Performance
comparisons for
hospitals, MDs,
treatments &
innovations
Low
2005
“Aerobic”
professionalism,
benefits design,
innovation vetting
& P4P
Faster clinical
re-engineering
by MDs, hospitals
& innovators
Q
50 ppts
$
40 ppts,
then “5 wks” &
2.5 ppts p.a.
Q = % adherence to evidence-based rules
$ = Per capita health care spending. Includes
new investment in IT / industrial
engineering capability. Excludes impact of
inflation, aging and biomedical innovation
Evolutionary Path
2015
11
© 2006 A. Milstein MD
• The study compares the impact of the Premier HQID and the
HQA’s public reporting initiative on quality performance
between 2004 and 2006.
• 232 hospital participants in the Premier HQID are compared to
3,044 hospitals not in the project that submitted quality data as
part of the HQA’s reporting initiative.
• Focus on four clinical areas that are partly or fully included in
the HQID: acute myocardial infarction (AMI), heart failure,
pneumonia, and surgical care improvement project (SCIP).
• Reviewed both “starter set” and “non-starter set” measures
submitted to HQA. Other studies have not focused on “nonstarter set.”
Arnold Milstein, MD, MPH. You Want Me to Pay More? How Does P4P Fit With What Mainstream
Purchasers Seek?. Beverly Hills, CA, National Pay for Performance Summit, February 8, 2006.
Reproduced
with permission
Copyright 2007, Catholic
Healthcare Partners
of Arnold Milstein, MD (Mercer)
Study Design
•
•
Based on publicly available data available on the CMS Hospital Compare
web site. Data is reported quarterly on a rolling 12-month basis. The study
identified from a database of 4,661 hospitals submitting data as part of the
HQA a total of 3,276 hospitals that submitted data for all available quarters
in the study (Quarter 1, 2004 – Quarter 1, 2006) and had complete data
describing hospital characteristics.
Hospital characteristics were available from a variety of sources including
MedPAR and American Hospital Association Guide. The study controls
for variation in hospital characteristics by using multivariate regression to
control for hospital size (measured as staffed beds), the existence of
graduate medical education programs (Academic Medical Center, Council
of Teaching Hospitals Members, Other GME, non-teaching), trauma
centers, urban locations, and open heart programs to assess performance on
overall quality using a composite opportunity quality score that was
developed by Hospital Core Performance Measurement Project for the
Rhode Island Public Reporting Program for Health Care Services and
modified by Premier, Inc. for use in their demonstration project. (Premier,
2003)
Copyright 2007, Catholic Healthcare Partners
Population Studied
• Hospitals submitting performance measure
data as part of the HQA reporting initiative for
up to twenty-one performance measures for
AMI, heart failure, pneumonia, and SCIP. A
total of 3,276 hospitals had sufficient data for
analysis, including 232 of the 260 hospitals
included in the Premier HQID. All data is
available from public sources.
Copyright 2007, Catholic Healthcare Partners
Copyright 2007, Catholic Healthcare Partners
Measuring Composite Quality Score (CQS)
Measure of Improvement between time 1 (Q2 2004)
and time 8 (Q1 2006)
CQS Improvement(t1-t8) = a′ + b ′*P4PDEMO + c
′*SBEDS + d ′*GME + e ′*TRAUMA + f ′*URBAN
+ g ′*OHEART + u′
Measure of Change in CQS over Time for each
Cohort (CMS/Premier participants vs. nonparticipants)
CQS Score = a′ + b ′*TIME + c ′*SBEDS + d ′*GME +
e ′*TRAUMA + f ′*URBAN + g ′*OHEART + u′
Copyright 2007, Catholic Healthcare Partners
Current measures collected and publicly
reported (through Q3 2006)
Acute Myocardial Infarction (Heart
Attack)
• AMI-1 Aspirin at arrival*
• AMI-2 Aspirin at discharge*
• AMI-3 ACEI/ARB given for LVSD*
• AMI-4 Adult smoking cessation advice
• AMI-5 Beta blocker at discharge
• AMI-6 Beta blocker at arrival*
• AMI-7a Fibrinolysis therapy within 30
min of arrival
• AMI-8a Primary percutaneous coronary
intervention within 90 minutes of arrival
Heart Failure
• HF-1 Discharge instructions
• HF-2 LVF Assessment *
• HF-3 ACEI/ARB given for LVSD*
• HF-4 Adult smoking cessation advice
Copyright 2007, Catholic Healthcare Partners
Pneumonia
• PN-1 Oxygenation assessment*
• PN-2 Pneumococcal vaccination*
• PN-3b Initial blood culture in ER*
• PN-4 Adult smoking cessation advice
• PN-5b Antibiotic within 4 hours of arrival
• PN-6 Antibiotic selection
• PN-7 Influenza vaccination
Surgical Care Improvement Project (colon
surgery, hip and knee arthroplasty,
abdominal and vaginal hysterectomy,
cardiac surgery (including coronary artery
bypass grafts (CABG)) and vascular
surgery )
• SCIP-Inf-1a Antibiotic within 1 hr of
incision-Overall
• SCIP-Inf-3a Antibiotic disc. within 24
hrs-Overall (48 hrs for CABG)
*Hospital Quality Alliance "Starter Set
Measures"
2
12%
p-value < .001
p-value = .023
p-value < .001
p-value = .001
p-value = not calculated
9.8%
9.7%
10%
National Rates of Improvement in Composite Quality Score for Hospital
Quality Alliance Starter Set and Non-Starter Set Measures: Premier HQID
Participants vs. Non-Participants:
Q2 2004 to Q1 2006
8%
6.6%
6.5%
6.1%
6%
4.3%
4.1%
4%
3.2%
2.1%
1.5%
2%
0%
Overall
Acute Myocardial
Infarction
Pneumonia
Heart Failure
SCIP
% Increase in Composite Quality Score 2004 to 2006
p-value < .001
14%
10%
8%
4%
2%
0%
Participants
National Rates of Improvement in "Starter Set" Composite Quality Score for
HQA Measures for AMI, HF, and PN: Premier HQID Participants vs. NonParticipants
100%
100%
95%
95%
Composite Quality Score
90%
85%
80%
75%
Surgical
Measures
Added
JCAHO Web
Site Launched
Hospital Compare
Goes Live
90%
85%
80%
75%
Hospital Compare
"Beta" Web Site Available: Publishes
ALL Measures
70%
Surgical
Measures
Added
JCAHO Web
Site Launched
Hospital Compare
Goes Live
Non-Participants
06
20
Q
1
20
05
05
Q
4
20
05
Non-Participants
Copyright 2007, Catholic Healthcare Partners
Q
3
20
Q
2
20
05
04
Q
1
20
20
Participants
Q
4
Q
2
04
04
06
20
Q
1
Q
4
20
05
05
Q
3
20
20
05
05
Q
2
Q
1
20
20
04
Q
4
20
Q
3
20
Q
2
04
65%
04
65%
20
Composite Quality Score
Participants
Copyright 2007, Catholic Healthcare Partners
National Rates of Improvement in Overall Composite Quality Score for HQA
Measures for AMI, HF, and PN: Premier HQID Participants vs. NonParticipants
Participants
Copyright 2007, Catholic Healthcare Partners
National Rates of Improvement in "Non-Starter Set" Composite Quality Score
for HQA Measures for AMI, HF, and PN: Premier HQID Participants vs. NonParticipants
National Impact of P4P
• Based on publicly reported data available in
Medicare’s Hospital Compare
100%
95%
Composite Quality Score
Non-Starter Set (AMI, HF, PN)
Non-Participants
Copyright 2007, Catholic Healthcare Partners
70%
7.2%
6.8%
5.7%
6%
Starter Set (AMI, HF, PN)
Non-Participants
Hospital Compare
"Beta" Web Site Available: Publishes
ALL Measures
12.7%
p-value = .015
12%
Q
3
% Increase in Composite Quality Score 2004 to 2006
National Rates of Improvement in Composite Quality Score by Hospital
Quality Alliance Focus Area: Premier HQID Participants vs. NonParticipants: Q2 2004 to Q1 2006
90%
85%
80%
75%
70%
65%
Hospital Compare
"Beta" Web Site Available: Publishes
ALL Measures
60%
55%
Surgical
Measures
Added
JCAHO Web
Site Launched
Hospital Compare
Goes Live
Non-Participants
Copyright 2007, Catholic Healthcare Partners
06
20
Q
1
Q
4
20
05
05
Q
3
20
05
20
Q
2
Q
1
20
05
04
Q
4
20
04
20
Q
3
Q
2
20
04
50%
– Premier P4P participants had a more rapid increase in
overall performance improvement in all major clinical
areas than other hospitals submitting to Hospital Quality
Alliance (HQA) (p-value < .001)
– P4P had slight impact on performance for HQA publicly
reported ten-measure “Starter Set” (p-value .015;
Lindenauer, et. al., 2007)
– Difference most pronounced with other HQA (non-starter
set) measures (p-value < .001)
– P4P participants had higher performance rates in early
2004 and increased the spread through 2005
Participants
Copyright 2007, Catholic Healthcare Partners
3
Limitations
• Limits on data quality and availability on CMS
Hospital Compare
• Accurate identification of hospital characteristics
(e.g., size, services, infrastructure)
• Lack of information on performance
improvement infrastructure at hospitals
• Potential serial correlation in time series analysis
• Surgical data incomplete
Copyright 2007, Catholic Healthcare Partners
Conclusions
• When hospitals voluntarily agreed to submit data to CMS for
the Hospital Quality Alliance’s reporting initiative, focus
increased dramatically on 10 key “starter set” measures. The
pressure of public reporting alone improved performance in
hospitals across the nation.
• There was little difference between hospitals in the pay-forperformance demonstration project and those only
participating in the HQA reporting initiative. In contrast, in
areas of clinical measurement not included in the public
reporting initiative (non-starter set measures of AMI,
pneumonia, heart failures, and SCIP), participants in the
Premier HQID, dramatically outperformed non-participants.
Copyright 2007, Catholic Healthcare Partners
Conclusions
• The pay-for-performance demonstration clearly accelerated
performance improvement among participants across a diverse
set of measures, however, the influence of public reporting,
approached that of the pay-for-performance demonstration on
the more limited set of HQA measures that were part of the
original public reporting initiative. Improvement in general
was limited for most hospitals for only the 10 measure “starter
set.” Other closely related measures in the areas of AMI, heart
failure, and pneumonia did not experience the rate of
improvement that participants in the Premier demonstration
were able to post for the larger measure set that represented a
comprehensive bundle of evidence-based care and included in
the pay-for-performance project.
Copyright 2007, Catholic Healthcare Partners
Implications for Policy, Delivery, or
Practice
• This analysis provides evidence that both pay-for-performance
and public reporting initiatives accelerated performance
improvement among the nation’s hospitals. The analysis also
points out the need for broadly defined sets of measures that
represent performance across the full continuum of care within
each clinical condition. Understanding how providers will
optimize performance is a key to designing future public
reporting initiatives and pay-for-performance programs. This
analysis demonstrates the need for broadly defined initiatives
around clinical needs of patients to ensure that the full
“bundle” of evidence-based aspects of care is delivered to
target populations.
• Raises concerns about “sub-optimization.” Performance
should be assessed based on delivery of all appropriate care
using “bundles” rather than using composites that may reward
sub-optimal care. Bundles more likely to stimulate reengineered of care delivery processes.
Copyright 2007, Catholic Healthcare Partners
References
Grossbart SR. What's the Return? Assessing the Effect of "Pay-forPerformance" Initiatives on the Quality of Care Delivery. Med Care Res
Rev, Feb 2006; 63: 29S - 48S.
Lindenauer PK, Remus D, Roman S, Rothberg MB, Benjamin EM, Ma A,
Bratzler DW. Public reporting and pay for performance in hospital quality
improvement. N Engl J Med 2007; 356:486-96
Milstein A. You Want Me to Pay More? How Does P4P Fit With What
Mainstream Purchasers Seek?. Beverly Hills, CA, National Pay for
Performance Summit, Feb 8, 2006.
Norling RA. How hospitals and health systems fit into the pay for performance
puzzle. National Pay for Performance Summit. Beverly Hills, CA, Feb 15,
2007.
Premier, Inc. CMS HQID project composite quality score methodology
overview. Charlotte, NC, 2003.
Copyright 2007, Catholic Healthcare Partners
4