A-TRIP Research Team Practice Partner Research Network Change from Baseline Performance: Practice Level Considerations Lynne S. Nemeth, PhD, RN • Steven M. Ornstein, MD (PI) • Ruth G. Jenkins, PhD • Paul Nietert, PhD • Chris Feifer, DrPH • Andrea M. Wessell, PharmD • Heather A. Liszka, MD, MS • Practice Partner™, Seattle, WA • MUSC • 100+ practices Funded by the Agency for Healthcare Research and Quality: U18 HS13716 Aims • Provide context for examining practice change in a primary care quality improvement intervention • Describe a composite measure of quality to evaluate change at the patient and practice level • Compare improvement patterns across practices: higher baseline performance vs. lower baseline performance 114 Practices in 37 States7-1-06 Accelerating Translation of Research into Practice (A-TRIP) • 4 year demonstration project • Funded under AHRQ Partnerships for Quality Initiative • Practice Partner Research Network (PPRNet) Collaborative • Expand PPRNet’s approach to QI 125 Primary Care Practices 85 Clinical Process and Outcome Measures Specialty of PPRNet Practices Family Medicine 78% Internal Medicine 18% Multi-specialty primary care 4% 1 Clinicians in PPRNet Practices Size of PPRNet Practices # of clinicians Physicians 462 Nurse Practitioners 63 Physician Assistants 51 Total 576 PPRNet Practices PPRNet Patients U.S. Population Urban core area 64% 66% 71% Small town/rural area 17% 15% 10% Suburban area 10% 12% 9% Large Town area 9% 7% 10% Percentage of Practices 13% 2 28 25% 3 17 15% 4 12 11% 5 12 11% 6 or 7 14 12% 8-10 7 6% More than 10 9 8% 85 Measures in 8 Clinical Domains Distribution of PPRNet Pts/Practices PPRNet is representative of US population 1 Number of Practices 15 • • • • • • • • Diabetes Mellitus (13) Heart Disease and Stroke (21) Cancer Screening (12) Adult Immunizations (14) Respiratory Disease / Infectious Disease (6) Mental Health / Substance Abuse (14) Nutrition / Obesity (3) Inappropriate Rx prescribing in the elderly (2) Practice Performance Report PPRNet TRIP Intervention Methods • Practice Performance Reports • Practice Site Visits • • • • • • Network Meetings © PPRNet, 2003-2006 ~80 indicators* SPC methodology Time trends PPRNet benchmark (ABC) National benchmarks (where available) *http://www.musc.edu/PPRNet/ATRIP%20Sample%20Report/Sample%20 Report.pdf 2 Patient-Level Report (PLR) • • • • • Quarterly report Excel Spreadsheet: 1 patient per row Same guideline criteria as practice report All “active” patients ≥ 18 yo Children: – Age 5-17: Asthma controller – ♀ age 16-25: Chlamydia screening PPRNet TRIP QI Model Key Elements • Prioritize Performance • Involve All Staff • Redesign Delivery System • Activate the Patient • Use EMR Tools * Jt Comm J Qual & Safety, August 2004, 30(8):432-441. QI Activities • Research team visited practices 2x per yr – Guideline-based academic detailing – Review of practice reports – Participatory planning with clinicians and staff • Annual network-wide QI meeting – Updates by research team – Best practice presentations by practices – Small group workshops © PPRNet, 2003 Practice Improvement • Study Practice Report Practices select indicators to target for improvement Follow improvement over time • Use PLR to identify individual patients • Implement Quality Improvement Cycle PLAN DO ACT STUDY 3 How to Rank Practice Performance • With many specific indicators to focus on, how can performance be evaluated across practices in a network or collaborative? • A summary measure might increase the relevance of improvement within practices over time The SQUID: Algorithm • Define processes and outcomes of interest, regardless of target – BP Monitoring – LDL Monitoring – HgbA1C Monitoring – BP Control – LDL Control – HgbA1C Control 80 indicators reduced to 31 processes & 5 outcomes • Hence, the SQUID was created The SQUID: Algorithm • Create indicator variables (ei) that reflect whether pt is eligible for each process and outcome measure – PAP Test (Women > 18 yrs old) – CRC screening (Men & Women > 50 yrs old) • Create indicator variables (mi) that reflect whether pt has met target for a process/outcome, his/her demographics and/or morbidity – If pt has HTN, then BP should be < 140/90 – But if pt has DM, BP should be < 130/80 The SQUID: Interpretation • A patient’s SQUID reflects the proportion of targets met for which he/she is eligible. • A practice’s SQUID reflects the average proportion of targets achieved by their patients. Nietert et al: Implementation Science 2007, 2:11 doi:10.1186/1748-5908-2-11 The SQUID: Algorithm • E = The number of measures for which the pt is eligible (denominator) = Σ ei • M = The number of eligible measures for which the pt has met his/her morbidity-specific target (numerator) = Σ mi • Create a pt-level SQUID = M E • Create a practice-level SQUID = average of all pt-level SQUIDs • Other SQUIDs can also be calculated: – Provider level – Domain-specific (e.g. DM, cancer, vaccinations) SQUID=Summary Quality Index • ~80 indicators 36 measures Example • 30 year old ♀; no chronic disease eligible for 7 processes, 0 outcomes BP monitoring ✔ Total Cholesterol Depression Screening Alcohol Screening PAP Smear ✔ HDL Td vaccine ✔ • SQUID = 3 / 7 = 0.429 4 Final ATRIP Results: Change over time in the SQUID 60% Average Proportion of Recommended Care Provided • • • • • • p < 0.0001 for trend over time 50% 40% Correlation with Clinical Outcomes (μ = +2.43% per year) 45.9% 33.7% 30% 20% 10% 0% 0 6 12 18 24 30 36 SBP (r = -0.17) (DM and HTN pts only) DBP (r = -0.23) (DM and HTN pts only) LDL (r = -0.26) (DM and CHD pts only) HDL (r = 0.17) (DM pts only) Triglycerides (r = -0.16) (DM pts only) A1C (r = -0.24) (DM pts only) 42 Months After Initial ATRIP Report (Length of ATRIP Exposure) Does Baseline Performance Matter? 60% SQUID Improvement Over Time, Stratified By Baseline Tertile 50% 40% SQUID Mean • Post-hoc analyses focused on whether baseline performance significantly influenced the observed time trends. 20% • Mixed linear regression models were used to examine the interaction between baseline strata (lower, middle, and upper tertiles) and time, adjusting for covariates including patient age and complexity. 70% Lowest Baseline Tertile (Adjusted Yearly Increase = 3.2%) Middle Baseline Tertile (Adjusted Yearly Increase = 2.2%) 10% Highest Baseline Tertile (Adjusted Yearly Increase = 2.0%) 0% 0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 Months of ATRIP Exposure Pneum Vaccine Improvement Over Time, Stratified By Baseline Tertile LDL Measurement Improvement Over Time, Stratified By Baseline Tertile 90% 80% 50% Lowest Baseline Tertile (Adjusted Yearly Increase = 3.4%) 40% Middle Baseline Tertile (Adjusted Yearly Increase = 3.3%) Highest Baseline Tertile (Adjusted Yearly Increase = 4.9%) 30% 20% 10% Proportion of Eligible Pts with LDL Measureme 60% Proportion of Eligible Pts with Pneum Vaccin 30% 70% 60% 50% 40% Lowest Baseline Tertile (Adjusted Yearly Increase = 9.4%) 30% Middle Baseline Tertile (Adjusted Yearly Increase = 2.7%) Highest Baseline Tertile (Adjusted Yearly Increase = 1.7%) 20% 10% 0% 0% 0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 Months of ATRIP Exposure 0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 Months of ATRIP Exposure 5 A1C Control Improvement Over Time, Stratified By Baseline Tertile 70% Proportion of DM Pts with A1C < 7% 60% 50% 40% 30% 20% Lowest Baseline Tertile (Adjusted Yearly Increase = 7.1%) Middle Baseline Tertile (Adjusted Yearly Increase = 2.9%) 10% Highest Baseline Tertile (Adjusted Yearly Increase = 0.3%) 0% 0 2 4 6 Discussion • Practices with lower baseline performance made significant improvements over time (LDL control, HgbA1C control) • Practices prioritize areas of focus creating meaningful opportunities for improvement • Practices with higher performance at baseline may achieve increased rates of change, as they embed a model for improvement into practice patterns 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 Months of ATRIP Exposure Benefits of The SQUID Approach • Prior to using the SQUID, it was hard for practices to have a sense if their efforts were paying off (some indicators improved, some got worse). • Increasing SQUID scores seemed to provide them with some sense of success. Strengths of This Approach • Direct interpretation, easily explained • Can be tailored for multiple levels of analysis • Can help clinicians quickly identify patients not at goals in their process of care Limitations of This Approach • Quality indicators are weighted equally. • Some strong correlations among indicators – Total Cholesterol & HDL – LDL measurement & LDL control • Does not account for patient allergies or other contraindications to immunizations or medications Issues for Further Consideration • Should process and outcome indicators be treated separately? • Should there be any adjustment for more “important” indicators? • Should there be any adjustment for more “difficult” indicators? 6 Conclusions • SQUID provides a useful composite measure with multiple quality indicators. • High performance at baseline may reflect increased exposure and experience with the PPRNet Model for Improvement • Lower performance at baseline, combined with an appreciation for performance data and a culture of learning might motivate achievement of significant improvement 7