Change from Baseline
Performance: Practice Level
Considerations
Lynne S. Nemeth, PhD, RN
A-TRIP Research Team
Practice Partner Research
Network
• Steven M. Ornstein,
MD (PI)
• Ruth G. Jenkins, PhD
• Paul Nietert, PhD
• Chris Feifer, DrPH
• Andrea M. Wessell,
PharmD
• Heather A. Liszka,
MD, MS
• Practice Partner™,
Seattle, WA
• MUSC
• 100+ practices
Funded by the Agency for
Healthcare Research and Quality:
Aims
• Provide context for examining practice
change in a primary care quality
improvement intervention
• Describe a composite measure of quality
to evaluate change at the patient and
practice level
• Compare improvement patterns across
practices: higher baseline performance vs.
lower baseline performance
Accelerating Translation of
Research into Practice (A-TRIP)
• 4 year demonstration project
• Funded under AHRQ Partnerships for
Quality Initiative
• Practice Partner Research Network
(PPRNet) Collaborative
• Expand PPRNet’s approach to QI
125 Primary Care Practices
85 Clinical Process and Outcome Measures
114 Practices in 37 States7-1-06
Specialty of PPRNet Practices
Family Medicine
78%
Internal Medicine
18%
Multi-specialty primary care
4%
Clinicians in PPRNet
Practices
Physicians
462
Nurse Practitioners
63
Physician Assistants
51
Total
576
Size of PPRNet Practices
# of clinicians
1
2
Number of
Practices
15
28
Percentage of
Practices
13%
25%
3
4
5
17
12
12
15%
11%
11%
6 or 7
8-10
More than 10
14
7
9
12%
6%
8%
Distribution of PPRNet
Pts/Practices
PPRNet is representative of US population
PPRNet
Practices
PPRNet
Patients
U.S.
Population
Urban core
area
64%
66%
71%
Small
town/rural
area
17%
15%
10%
Suburban
area
10%
12%
9%
Large Town
area
9%
7%
10%
85 Measures in 8 Clinical
Domains
•
•
•
•
•
•
•
•
Diabetes Mellitus (13)
Heart Disease and Stroke (21)
Cancer Screening (12)
Adult Immunizations (14)
Respiratory Disease / Infectious Disease (6)
Mental Health / Substance Abuse (14)
Nutrition / Obesity (3)
Inappropriate Rx prescribing in the elderly (2)
PPRNet TRIP Intervention
Methods
• Practice Performance Reports
• Practice Site Visits
• Network Meetings
© PPRNet, 2003-2006
Practice Performance
Report
•
•
•
•
•
~80 indicators*
SPC methodology
Time trends
PPRNet benchmark (ABC)
National benchmarks (where available)
*http://www.musc.edu/PPRNet/ATRIP%20Sample%20Report/Sample%20
Report.pdf
Patient-Level Report (PLR)
•
•
•
•
•
Quarterly report
Excel Spreadsheet: 1 patient per row
Same guideline criteria as practice report
All “active” patients ≥ 18 yo
Children:
– Age 5-17: Asthma controller
– ♀ age 16-25: Chlamydia screening
PPRNet TRIP QI Model
Key Elements
• Prioritize
Performance
• Involve All Staff
• Redesign Delivery
System
• Activate the Patient
• Use EMR Tools
* Jt Comm J Qual & Safety, August 2004, 30(8):432-441.
© PPRNet, 2003
QI Activities
• Research team visited practices 2x per yr
– Guideline-based academic detailing
– Review of practice reports
– Participatory planning with clinicians and staff
• Annual network-wide QI meeting
– Updates by research team
– Best practice presentations by practices
– Small group workshops
Practice Improvement
• Study Practice Report
 Practices select indicators to target for
improvement
 Follow improvement over time
• Use PLR to identify individual patients
• Implement Quality
Improvement Cycle
PLAN
DO
ACT
STUDY
How to Rank Practice
Performance
• With many specific indicators to focus on,
how can performance be evaluated across
practices in a network or collaborative?
• A summary measure might increase the
relevance of improvement within practices
over time
• Hence, the SQUID was created
The SQUID: Algorithm
• Define processes and outcomes of
interest, regardless of target
– BP Monitoring
– LDL Monitoring
– HgbA1C Monitoring
– BP Control
– LDL Control
– HgbA1C Control
80 indicators reduced to 31 processes & 5 outcomes
Nietert et al: Implementation Science 2007, 2:11 doi:10.1186/1748-5908-2-11
The SQUID: Algorithm
• Create indicator variables (ei) that reflect whether
pt is eligible for each process and outcome
measure
– PAP Test (Women > 18 yrs old)
– CRC screening (Men & Women > 50 yrs old)
• Create indicator variables (mi) that reflect whether
pt has met target for a process/outcome,
his/her demographics and/or morbidity
– If pt has HTN, then BP should be < 140/90
– But if pt has DM, BP should be < 130/80
The SQUID: Algorithm
• E = The number of measures for which the pt is eligible
(denominator) = Σ ei
• M = The number of eligible measures for which the pt has
met his/her morbidity-specific target (numerator) = Σ mi
• Create a pt-level SQUID =
M
E
• Create a practice-level SQUID
= average of all pt-level SQUIDs
• Other SQUIDs can also be calculated:
– Provider level
– Domain-specific (e.g. DM, cancer, vaccinations)
The SQUID: Interpretation
• A patient’s SQUID reflects the proportion
of targets met for which he/she is eligible.
• A practice’s SQUID reflects the average
proportion of targets achieved by their
patients.
SQUID=Summary Quality
Index
• ~80 indicators
36 measures
Example
• 30 year old ♀; no chronic disease
eligible for 7 processes, 0 outcomes
BP monitoring ✔
Total Cholesterol
Depression Screening
Alcohol Screening
• SQUID = 3 / 7 = 0.429
PAP Smear ✔
HDL
Td vaccine ✔
Final ATRIP Results:
Change over time in the SQUID
Average Proportion of
Recommended Care
Provided
60%
p < 0.0001 for trend over time
50%
40%
(μ = +2.43% per year)
45.9%
33.7%
30%
20%
10%
0%
0
6
12
18
24
30
36
Months After Initial ATRIP Report
(Length of ATRIP Exposure)
42
Correlation with Clinical Outcomes
•
•
•
•
•
•
SBP (r = -0.17) (DM and HTN pts only)
DBP (r = -0.23) (DM and HTN pts only)
LDL (r = -0.26) (DM and CHD pts only)
HDL (r = 0.17) (DM pts only)
Triglycerides (r = -0.16) (DM pts only)
A1C (r = -0.24) (DM pts only)
Does Baseline Performance
Matter?
• Post-hoc analyses focused on whether
baseline performance significantly
influenced the observed time trends.
• Mixed linear regression models were used
to examine the interaction between
baseline strata (lower, middle, and upper
tertiles) and time, adjusting for covariates
including patient age and complexity.
SQUID Improvement Over Time, Stratified
60%
By Baseline Tertile
50%
SQUID Mean
40%
30%
20%
Lowest Baseline Tertile (Adjusted Yearly Increase = 3.2% )
Middle Baseline Tertile (Adjusted Yearly Increase = 2.2% )
10%
Highest Baseline Tertile (Adjusted Yearly Increase = 2.0% )
0%
0
2
4
6
8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42
Months of ATRIP Exposure
Proportion of Eligible Pts with Pneum Vaccine
70%
Pneum Vaccine Improvement Over Time,
Stratified By Baseline Tertile
60%
50%
Lowest Baseline Tertile (Adjusted Yearly Increase = 3.4% )
40%
Middle Baseline Tertile (Adjusted Yearly Increase = 3.3% )
Highest Baseline Tertile (Adjusted Yearly Increase = 4.9% )
30%
20%
10%
0%
0
2
4
6
8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42
Months of ATRIP Exposure
LDL Measurement Improvement Over Time,
90%
Stratified By Baseline Tertile
Proportion of Eligible Pts with LDL Measurement
80%
70%
60%
50%
40%
Lowest Baseline Tertile (Adjusted Yearly Increase = 9.4% )
30%
Middle Baseline Tertile (Adjusted Yearly Increase = 2.7% )
Highest Baseline Tertile (Adjusted Yearly Increase = 1.7% )
20%
10%
0%
0
2
4
6
8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42
Months of ATRIP Exposure
A1C Control Improvement Over Time,
Stratified By Baseline Tertile
70%
Proportion of DM Pts with A1C < 7%
60%
50%
40%
30%
20%
Lowest Baseline Tertile (Adjusted Yearly Increase = 7.1% )
Middle Baseline Tertile (Adjusted Yearly Increase = 2.9% )
10%
Highest Baseline Tertile (Adjusted Yearly Increase = 0.3% )
0%
0
2
4
6
8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42
Months of ATRIP Exposure
Discussion
• Practices with lower baseline performance
made significant improvements over time
(LDL control, HgbA1C control)
• Practices prioritize areas of focus creating
meaningful opportunities for improvement
• Practices with higher performance at
baseline may achieve increased rates of
change, as they embed a model for
improvement into practice patterns
Benefits of The SQUID Approach
• Prior to using the SQUID, it was hard for
practices to have a sense if their efforts
were paying off (some indicators
improved, some got worse).
• Increasing SQUID scores seemed to
provide them with some sense of success.
Limitations of This Approach
• Quality indicators are weighted equally.
• Some strong correlations among indicators
– Total Cholesterol & HDL
– LDL measurement & LDL control
• Does not account for patient allergies or
other contraindications to immunizations or
medications
Strengths of This Approach
• Direct interpretation, easily explained
• Can be tailored for multiple levels of
analysis
• Can help clinicians quickly identify patients
not at goals in their process of care
Issues for Further Consideration
• Should process and outcome indicators be
treated separately?
• Should there be any adjustment for more
“important” indicators?
• Should there be any adjustment for more
“difficult” indicators?
Conclusions
• SQUID provides a useful composite
measure with multiple quality indicators.
• High performance at baseline may reflect
increased exposure and experience with
the PPRNet Model for Improvement
• Lower performance at baseline, combined
with an appreciation for performance data
and a culture of learning might motivate
achievement of significant improvement