A-TRIP Research Team
Practice Partner Research
Network
Change from Baseline
Performance: Practice Level
Considerations
Lynne S. Nemeth, PhD, RN
• Steven M. Ornstein,
MD (PI)
• Ruth G. Jenkins, PhD
• Paul Nietert, PhD
• Chris Feifer, DrPH
• Andrea M. Wessell,
PharmD
• Heather A. Liszka,
MD, MS
• Practice Partner™,
Seattle, WA
• MUSC
• 100+ practices
Funded by the Agency for
Healthcare Research and Quality:
U18 HS13716
Aims
• Provide context for examining practice
change in a primary care quality
improvement intervention
• Describe a composite measure of quality
to evaluate change at the patient and
practice level
• Compare improvement patterns across
practices: higher baseline performance vs.
lower baseline performance
114 Practices in 37 States7-1-06
Accelerating Translation of
Research into Practice (A-TRIP)
• 4 year demonstration project
• Funded under AHRQ Partnerships for
Quality Initiative
• Practice Partner Research Network
(PPRNet) Collaborative
• Expand PPRNet’s approach to QI
125 Primary Care Practices
85 Clinical Process and Outcome Measures
Specialty of PPRNet Practices
Family Medicine
78%
Internal Medicine
18%
Multi-specialty primary care
4%
1
Clinicians in PPRNet
Practices
Size of PPRNet Practices
# of clinicians
Physicians
462
Nurse Practitioners
63
Physician Assistants
51
Total
576
PPRNet
Practices
PPRNet
Patients
U.S.
Population
Urban core
area
64%
66%
71%
Small
town/rural
area
17%
15%
10%
Suburban
area
10%
12%
9%
Large Town
area
9%
7%
10%
Percentage of
Practices
13%
2
28
25%
3
17
15%
4
12
11%
5
12
11%
6 or 7
14
12%
8-10
7
6%
More than 10
9
8%
85 Measures in 8 Clinical
Domains
Distribution of PPRNet
Pts/Practices
PPRNet is representative of US population
1
Number of
Practices
15
•
•
•
•
•
•
•
•
Diabetes Mellitus (13)
Heart Disease and Stroke (21)
Cancer Screening (12)
Adult Immunizations (14)
Respiratory Disease / Infectious Disease (6)
Mental Health / Substance Abuse (14)
Nutrition / Obesity (3)
Inappropriate Rx prescribing in the elderly (2)
Practice Performance
Report
PPRNet TRIP Intervention
Methods
• Practice Performance Reports
• Practice Site Visits
•
•
•
•
•
• Network Meetings
© PPRNet, 2003-2006
~80 indicators*
SPC methodology
Time trends
PPRNet benchmark (ABC)
National benchmarks (where available)
*http://www.musc.edu/PPRNet/ATRIP%20Sample%20Report/Sample%20
Report.pdf
2
Patient-Level Report (PLR)
•
•
•
•
•
Quarterly report
Excel Spreadsheet: 1 patient per row
Same guideline criteria as practice report
All “active” patients ≥ 18 yo
Children:
– Age 5-17: Asthma controller
– ♀ age 16-25: Chlamydia screening
PPRNet TRIP QI Model
Key Elements
• Prioritize
Performance
• Involve All Staff
• Redesign Delivery
System
• Activate the Patient
• Use EMR Tools
* Jt Comm J Qual & Safety, August 2004, 30(8):432-441.
QI Activities
• Research team visited practices 2x per yr
– Guideline-based academic detailing
– Review of practice reports
– Participatory planning with clinicians and staff
• Annual network-wide QI meeting
– Updates by research team
– Best practice presentations by practices
– Small group workshops
© PPRNet, 2003
Practice Improvement
• Study Practice Report
Š Practices select indicators to target for
improvement
Š Follow improvement over time
• Use PLR to identify individual patients
• Implement Quality
Improvement Cycle
PLAN
DO
ACT
STUDY
3
How to Rank Practice
Performance
• With many specific indicators to focus on,
how can performance be evaluated across
practices in a network or collaborative?
• A summary measure might increase the
relevance of improvement within practices
over time
The SQUID: Algorithm
• Define processes and outcomes of
interest, regardless of target
– BP Monitoring
– LDL Monitoring
– HgbA1C Monitoring
– BP Control
– LDL Control
– HgbA1C Control
80 indicators reduced to 31 processes & 5 outcomes
• Hence, the SQUID was created
The SQUID: Algorithm
• Create indicator variables (ei) that reflect whether
pt is eligible for each process and outcome
measure
– PAP Test (Women > 18 yrs old)
– CRC screening (Men & Women > 50 yrs old)
• Create indicator variables (mi) that reflect whether
pt has met target for a process/outcome,
his/her demographics and/or morbidity
– If pt has HTN, then BP should be < 140/90
– But if pt has DM, BP should be < 130/80
The SQUID: Interpretation
• A patient’s SQUID reflects the proportion
of targets met for which he/she is eligible.
• A practice’s SQUID reflects the average
proportion of targets achieved by their
patients.
Nietert et al: Implementation Science 2007, 2:11 doi:10.1186/1748-5908-2-11
The SQUID: Algorithm
• E = The number of measures for which the pt is eligible
(denominator) = Σ ei
• M = The number of eligible measures for which the pt has
met his/her morbidity-specific target (numerator) = Σ mi
• Create a pt-level SQUID =
M
E
• Create a practice-level SQUID
= average of all pt-level SQUIDs
• Other SQUIDs can also be calculated:
– Provider level
– Domain-specific (e.g. DM, cancer, vaccinations)
SQUID=Summary Quality
Index
• ~80 indicators
36 measures
Example
• 30 year old ♀; no chronic disease
eligible for 7 processes, 0 outcomes
BP monitoring ✔
Total Cholesterol
Depression Screening
Alcohol Screening
PAP Smear ✔
HDL
Td vaccine ✔
• SQUID = 3 / 7 = 0.429
4
Final ATRIP Results:
Change over time in the SQUID
60%
Average Proportion of
Recommended Care
Provided
•
•
•
•
•
•
p < 0.0001 for trend over time
50%
40%
Correlation with Clinical Outcomes
(μ = +2.43% per year)
45.9%
33.7%
30%
20%
10%
0%
0
6
12
18
24
30
36
SBP (r = -0.17) (DM and HTN pts only)
DBP (r = -0.23) (DM and HTN pts only)
LDL (r = -0.26) (DM and CHD pts only)
HDL (r = 0.17) (DM pts only)
Triglycerides (r = -0.16) (DM pts only)
A1C (r = -0.24) (DM pts only)
42
Months After Initial ATRIP Report
(Length of ATRIP Exposure)
Does Baseline Performance
Matter?
60%
SQUID Improvement Over Time, Stratified
By Baseline Tertile
50%
40%
SQUID Mean
• Post-hoc analyses focused on whether
baseline performance significantly
influenced the observed time trends.
20%
• Mixed linear regression models were used
to examine the interaction between
baseline strata (lower, middle, and upper
tertiles) and time, adjusting for covariates
including patient age and complexity.
70%
Lowest Baseline Tertile (Adjusted Yearly Increase = 3.2%)
Middle Baseline Tertile (Adjusted Yearly Increase = 2.2%)
10%
Highest Baseline Tertile (Adjusted Yearly Increase = 2.0%)
0%
0
2
4
6
8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42
Months of ATRIP Exposure
Pneum Vaccine Improvement Over Time,
Stratified By Baseline Tertile
LDL Measurement Improvement Over Time,
Stratified By Baseline Tertile
90%
80%
50%
Lowest Baseline Tertile (Adjusted Yearly Increase = 3.4%)
40%
Middle Baseline Tertile (Adjusted Yearly Increase = 3.3%)
Highest Baseline Tertile (Adjusted Yearly Increase = 4.9%)
30%
20%
10%
Proportion of Eligible Pts with LDL Measureme
60%
Proportion of Eligible Pts with Pneum Vaccin
30%
70%
60%
50%
40%
Lowest Baseline Tertile (Adjusted Yearly Increase = 9.4%)
30%
Middle Baseline Tertile (Adjusted Yearly Increase = 2.7%)
Highest Baseline Tertile (Adjusted Yearly Increase = 1.7%)
20%
10%
0%
0%
0
2
4
6
8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42
Months of ATRIP Exposure
0
2
4
6
8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42
Months of ATRIP Exposure
5
A1C Control Improvement Over Time,
Stratified By Baseline Tertile
70%
Proportion of DM Pts with A1C < 7%
60%
50%
40%
30%
20%
Lowest Baseline Tertile (Adjusted Yearly Increase = 7.1%)
Middle Baseline Tertile (Adjusted Yearly Increase = 2.9%)
10%
Highest Baseline Tertile (Adjusted Yearly Increase = 0.3%)
0%
0
2
4
6
Discussion
• Practices with lower baseline performance
made significant improvements over time
(LDL control, HgbA1C control)
• Practices prioritize areas of focus creating
meaningful opportunities for improvement
• Practices with higher performance at
baseline may achieve increased rates of
change, as they embed a model for
improvement into practice patterns
8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42
Months of ATRIP Exposure
Benefits of The SQUID Approach
• Prior to using the SQUID, it was hard for
practices to have a sense if their efforts
were paying off (some indicators
improved, some got worse).
• Increasing SQUID scores seemed to
provide them with some sense of success.
Strengths of This Approach
• Direct interpretation, easily explained
• Can be tailored for multiple levels of
analysis
• Can help clinicians quickly identify patients
not at goals in their process of care
Limitations of This Approach
• Quality indicators are weighted equally.
• Some strong correlations among indicators
– Total Cholesterol & HDL
– LDL measurement & LDL control
• Does not account for patient allergies or
other contraindications to immunizations or
medications
Issues for Further Consideration
• Should process and outcome indicators be
treated separately?
• Should there be any adjustment for more
“important” indicators?
• Should there be any adjustment for more
“difficult” indicators?
6
Conclusions
• SQUID provides a useful composite
measure with multiple quality indicators.
• High performance at baseline may reflect
increased exposure and experience with
the PPRNet Model for Improvement
• Lower performance at baseline, combined
with an appreciation for performance data
and a culture of learning might motivate
achievement of significant improvement
7