Using Performance Measurement to
Improve Quality:
Good Data Are Only the Beginning
__________________________________
Dana Gelb Safran, Sc.D.
Senior Vice President
Performance Measurement & Improvement
Blue Cross Blue Shield of Massachusetts
Presented at:
How Strong is the Evidence that Organized Processes to
Improve Quality and Efficiency Work?
AcademyHealth Annual Research Meeting
28 June 2009
Question
What is the evidence that process, structure,
organization type, and/or policy works to improve
quality and/or control costs?
Blue Cross Blue Shield of Massachusetts
2
Transformation Vision: 2016
A health care system that provides safe, timely,
effective, affordable, patient-centered care for
everyone in Massachusetts.
Blue Cross Blue Shield of Massachusetts
3
Levers of Change
Public Engagement
Legislative & Regulatory
Finance & Payment
Quality & Safety
Measurements
Governance
Information Technology
Organizational
Readiness
Blue Cross Blue Shield of Massachusetts
4
Competing Demands for Time and Attention
What gets measured
becomes
what matters
Blue Cross Blue Shield of Massachusetts
5
Changing Rates of Preventive Care Processes,
1996-2001
100.0
Diabetes Eye
Exams
80.0
Cervical
Cancer
Screening
60.0
Breast Cancer
Screening
40.0
Adolescent
Hep B
Immunization
20.0
HEDIS did not begin testing adolescent
Hepatitis B immunization rates until 1997
0.0
1996
Blue Cross Blue Shield of Massachusetts
1997
1998
1999
2000
2001
2002
Beta Blocker
Treatment
Following a
Heart Attack
6
Diabetes Care:
Process is nearing perfection, outcomes are far from it
Percent of Patients Who Have Received Recommended Screening
vs. Percent with Poor Blood Glucose Levels
100
Percent of Patients Who Have Received Recommended Screening
vs. Percent with Healthy Cholesterol Levels
100
90
90
80
80
70
70
60
60
50
50
40
40
30
30
20
10
20
0
10
S
te
ys
m
1
S
te
ys
m
2
S
te
ys
m
3
S
te
ys
m
4
S
te
ys
m
5
S
te
ys
m
Integrated Health System Entity
HbA1c Testing
6
S
te
ys
m
7
Poor HbA1c Control (>9)
0
System 1 System 2 System 3 System 4 System 5 System 6 System 7
Integrated Health System Entity
LDL-C Screening
LDL-C Control (<100 mg)
Source: MHQP, 2005 HEDIS process and outcomes measures
Blue Cross Blue Shield of Massachusetts
7
What Will Make a Medical Home Patient-Centered?
Patient - Centered
Medical Home
Care in the Context of the Whole Person
Structure
• Structure
• Process
• Values
• Life Circumstances
• Preferences
• Constraints
• Knowledge/ understanding of health, disease process
Blue Cross Blue Shield of Massachusetts
8
Challenges of Quality Improvement
“ My trouble is that the energy for this action
group died a quiet death. There really isn't
anything to report. The administrator never
really came on board and without his support
the rest of the team lost enthusiasm.”
--Participant in Patient-Centered Care Collaborative
Blue Cross Blue Shield of Massachusetts
9
Percentile Rank Adjusted
What Drives Patients’ Willingness To Recommend
And How Are We Doing (2002)
100
100
80
80
Office Staff
60
60
Clinical Team
Relationship Duration
40
40
Integration
Health
Promotion
20
20
Visit-based Continuity
Interpersonal Treatment
Knowledge
of Patient
Organizational
Access
Patient Trust
Communication
Priority Improvements
00
0.2
0.2
0
0.4
0.4
0.6
0.6
0.8
0.8
11
Correlation to Measure of Willingness to Recommend
Blue Cross Blue Shield of Massachusetts
10
Improving Patients’ Care Experiences:
Changes in 2 Important Metrics: Jan 2002 – Jan 2005
100
90
Percentile Rank Adjusted
80
Knowledge of
the Patient
70
Communication
60
2005
2005
2004
2004
2002
2002
50
40
30
Priority Improvements
20
10
0
0
0.2
0.4
0.6
0.8
1
Correlation to Measure of Willingness to Recommend
Blue Cross Blue Shield of Massachusetts
11
What Were the Critical Elements of Success?
Senior leadership vision and steadfast commitment
 “This is who we are!”
 Discussion at regular meetings and in conversations at every level of the organization
Measurement
 Regularly reported results (MD-level, practice-level, system-level)
System-level changes
 Scheduling templates, phone scripts, prioritizing continuity
External momentum toward public reporting
Whole-practice vs. Individually-Focused Improvement Strategy
 Supported by combination of group & individual-level data
Blue Cross Blue Shield of Massachusetts
12
Percentile Rank Adjusted (Statewide Data)*
Comparing Patient Experiences with Four Physicians
in Alternate Practice Settings
100
Office Staff
Organizational Access
Integration of Care
80
MD-Pt Interaction
60
Practice 1
Practice 2
(Concierge)
40
20
Integration of Care
Office Staff
MD-Pt Interaction
Organizational
Access
0
0
0.2
0.4
0.6
0.8
1
Correlation of Measure to Willingness to Recommend MD
Source
: KoBlueJM
etofal.
Patient: Patient-Centered Outcomes Research 2009.
Blue Cross
Shield
Massachusetts
13
Key components of the alternative contract model
Unique contract model:
• Physicians & hospital contracted together
as a “system” – accountable for cost &
quality across full care continuum
• Long-term (5-years)
Controls cost growth:
• Global payment for care across the
continuum
• Annual inflation tied to CPI
• Incentive to eliminate clinically wasteful
care (“overuse”)
Improved quality, safety and outcomes:
• Robust performance measure set creates
accountability for quality, safety and
outcomes across continuum
• Substantial financial incentives for high
performance (up to 10% upside)
Blue Cross Blue Shield of Massachusetts
Efficiency Opportunity
Inflation
Performance
Expanded Margin
Opportunity
INITIAL GLOBAL
PAYMENT LEVEL
Year 1
Year 2
Year 3
Year 4
Year 5
14
Defining Performance Measures for the AQC
Overarching goal: Measures should collectively advance care to the end-state vision of
safe, affordable, effective, patient-centered care
Clinical performance measures will include process, outcomes and patient care
experiences; and will encompass inpatient and ambulatory care.
AQC performance framework based on thresholds (“gates”) with the following attributes:
• Gate 1 represents performance that is at initial levels deserving of financial recognition
• Gate 5 represents the “theoretical limits” of performance (end-state vision)
• Use of gates establishes rewards for both absolute performance and for performance
improvement
• Use of gates affords “transparency” to providers regarding full scope of BCBSMA
performance priorities and expectations
Blue Cross Blue Shield of Massachusetts
15
AQC Measures - Illustration Only - Not Actual Provider Scores
Ambulatory Measures
Measure
Score
Experimental
Patient Exper.
Outcomes
Process
Depression
1 Acute Phase Rx
2 Continuation Phase Rx
Diabetes
3 HbA1c Testing (2X)
4 Eye Exams
5 Nephropathy Screening
Cholesterol Management
6 Diabetes LDL-C Screening
7 Cardiovascular LDL-C Screening
8 Breast Cancer Screening
9 Cervical Cancer Screening
10 Colorectal Cancer Screening
Preventive Screening/Treatment
Chlamydia Screening
11 Ages 16-20
12 Ages 21-25
Pedi: Testing/Treatment
13 Upper Respiratory Infection (URI)
14 Pharyngitis
Pedi: Well-visits
15 < 15 months
16 3-6 Years
17 Adolescent Well Care Visits
Diabetes
18
HbA1c in Poor Control
19
LDL-C Control (<100mg)
Hypertension
20
Controlling High Blood Pressure
Cardiovascular Disease
21
LDL-C Control (<100mg)
2.5
1.5
1.0
1.0
3.0
1.0
1.2
1.0
1.0
1.0
2.8
2.1
1.0
1.0
1.2
1.3
2.4
1.0
1.0
1.0
3.1
1.8
0.5
0.5
1.6
1.4
1.0
1.0
2.6
2.0
1.5
1.0
1.0
1.0
3.2
2.4
3.0
3.0
1.3
3.0
2.4
3.0
Patient Experiences (C/G CAHPS/ACES) - Adult 3
22 Communication Quality
1.9
23 Knowledge of Patients
1.9
24 Integration of Care
2.1
25 Access to Care
2.4
Patient Experiences (C/G CAHPS/ACES) - Pediatric 3
26 Communication Quality
1.0
27 Knowledge of Patients
1.5
28 Integration of Care
2.5
29 Access to Care
2.8
30 Experimental Measure A
31 Experimental Measure B
5.0
5.0
Weighted Ambulatory Score
Blue Cross Blue Shield of Massachusetts
Hospital Measures
Weight
Measure
Score
AMI
1 ACE/ARB for LVSD
2 Aspirin at arrival
3 Aspirin at discharge
4 Beta Blocker at arrival
5 Beta Blocker at discharge
6 Smoking Cessation
Heart Failure
7 ACE LVSD
8 LVS function Evaluation
9 Discharge instructions
10 Smoking Cessation
Pneumonia
11 Flu Vaccine
12 Pneumococcal Vaccination
13 Antibiotics w/in 4 hrs
14 Oxygen assessment
15 Smoking Cessation
16 Antibiotic selection
17 Blood culture
Surgical Infection
18 Antibiotic received
19 Received Appropriate Preventive Antibiotic(
20 Antibiotic discontinued
21
22
23
24
25
26
27
28
In-Hospital Mortality - Overall
Wound Infection
Select Infections due to Medical Care
AMI after Major Surgery
Pneumonia after Major Surgery
Post-Operative PE/DVT
Birth Trauma - injury to neonate
Obstetrics Trauma-vaginal w/o instrument
Hospital Patient Experience (H-CAHPS) Measures
29 Communication with Nurses
30 Communication with Doctors
31 Responsiveness of staff
32 Discharge Information
1.0
1.0
1.0
1.0
Weight
2.0
2.5
1.5
1.5
1.3
1.0
1.0
1.0
1.0
1.0
1.0
1.0
1.3
1.0
1.8
3.0
1.0
1.0
1.0
1.0
2.5
2.9
1.4
1.0
3.1
3.0
3.5
1.0
1.0
1.0
1.0
1.0
1.0
1.0
1.3
1.4
3.0
1.0
1.0
1.0
3.0
2.1
2.8
2.4
3.4
2.0
1.0
1.5
1.0
1.0
1.0
1.0
1.0
1.0
1.0
1.0
4.0
3.0
2.5
2.8
1.0
1.0
1.0
1.0
5.0
1.0
1.0
1.0
1.0
1.0
1.0
1.0
33 Experimental Measure C
2.2
Aggregate Score
Weighted Hospital Score
2.3
2.3
16
Performance Achievement Model
Performance Payment Model
10%
10.0%
9.0%
% Payout
8%
6%
5.0%
4%
3.0%
2.0%
2%
0%
1.0
2.0
3.0
4.0
5.0
Performance Score
Blue Cross Blue Shield of Massachusetts
17
Summary

Without measurement, we don’t know where we are on the journey

But imprecise measurement used in “high stakes” ways undermines our
collective efforts

Getting to “high stakes” implementation with reliable, valid measures does not
have to take long

Much is available and appropriate for high stakes uses already – but substantial
and important gaps in our national measurement portfolio remain

Early evidence of “improvability” is encouraging – even on measures that go
beyond “process of care”
 …but requires broad organizational engagement, leadership and
sustained effort

Getting to safe, effective, affordable, patient-centered care will require ongoing
use of valid, reliable performance measures, employed in ways that engage and
align the interests of clinicians, patients, and health care institutions.
Blue Cross Blue Shield of Massachusetts
18
For More Information
dana.safran@bcbsma.com
Blue Cross Blue Shield of Massachusetts
19
Guiding Principles in Selecting
Performance Measures for “High Stakes” Use
♦ Wherever possible, our measures should be drawn from nationally accepted standard measure
sets.
♦ The measure must reflect something that is broadly accepted as clinically important.
♦ There must be empirical evidence that the measure provides stable and reliable information at the
level at which it will be reported (i.e. individual, site, group, or institution) with available sample
sizes and data sources.
♦ There must be sufficient variability on the measure across providers (or at the level at which data
will be reported) to merit attention.
♦ The must be empirical evidence that the level of the system that will be held accountable (clinician,
site, group, institution) accounts for substantial system-level variance in the measure.
♦ Providers should be exposed to information about the development and validation of the measures
and given the opportunity to view their own performance, ideally for one measurement cycle,
before the data are used for “high stakes” purposes.
Blue Cross Blue Shield of Massachusetts
20
Barriers to Adherence
Cognitive
Financial
Logistical
Motivational
Blue Cross Blue Shield of Massachusetts
21