How to Publish Your Quality Improvement Project— Designing and Executing Your QI Project as Scholarly Activity DAVID COOPERBERG, MD DEPARTMENT OF PEDIATRICS SECTION OF HOSPITAL MEDICINE Acknowledgements SCHC Resident QI Curriculum Team Celeste Chamberlain, PhD, Director of Quality Cheryl Gebeline-Myer, MS, former Director of Process Improvement Mackenzie Frost, MD Paul Shore, MD, MS Improving Adolescent HIV Screening Team Lorena Pereira, MD Clint Steib, HIV Coordinator Peter Osgood, MD Roberta Laguerre, MD Dan Conway, MD Doug Thompson, MD, MMM Evan Weiner, MD Mario Cruz, MD (co-designed handout) Katie McPeak, MD and the Improving Time to Third Next Available Newborn Clinic Team Aarti Patel, MD and the Improving Nurse-Resident Communication Team Tess Woehrlen, MPH and the Improving Hospitalist-Primary Care Provider Handoff Team Objectives 1. Design a high-impact QI project 2. Apply the SQUIRE guideline to draft a QI manuscript Disclosure I have no relevant disclosures Disclaimer This talk is not about Quality Assurance Office of Research provides additional support and random reviews through the QA/QI division For more information: Contact: Office of Research Karen Skinner, MSN, RN, NHA, CCRP, Director QA/QI Kirtanaa Voralu BSc, MStat, Analyst Drexel University1601 Cherry Street, Suite 10-444, Philadelphia, PA 19102 Tel: 215.255.7883 | Fax: 215.255.7874 http://www.drexel.edu/research/compliance/qa/ Resources www.squire-statement.org Handout: Quality Improvement Scholarship: Taking Your QI Project to the Next Level Handout: Examples QI Posters QI Abstract accepted as platform presentation Published QI Reports Brief Review of Quality Improvement in Healthcare Systematic approach to problem identification and improvement Multidisciplinary team process Objective, data driven process Minimizing variation in processes to improve outcomes 7 Outline Steps in Designing and Executing QI Project Use Standardized Quality Improvement Reporting Excellence (SQUIRE) Guideline to Draft QI Report Manuscript Use example of Improving Adolescent HIV Screening Rates at SCHC Adequate Preparation Before You Get Started… Check with Drexel IRB Letter of determination IRB exempt Expedited IRB review Adequate Preparation CITI Training Perform Literature Review Existing programs/projects Discuss with local/regional experts in field Building Your Improvement Team Identify Key Stakeholders Stakeholder Definition: Anyone who can help affect change or may be effected by the potential interventions Consider Influence and Interest Influence Interest Option 1 High High Option 2 High Low Option 3 Low High Option 4 Low Low Jonathan Boutelle: Understanding Organizational Stakeholders for Design Success, 2004. http://www.boxesandarrows.com/view/understanding_organizational_stakeholders_for_design_success Prioritize Stakeholders Influence Interest Option 1 High High Option 2 High Low Option 3 Low High Option 4 Low Low Jonathan Boutelle: Understanding Organizational Stakeholders for Design Success, 2004. http://www.boxesandarrows.com/view/understanding_organizational_stakeholders_for_design_success Prioritize Stakeholders Influence Interest Option 1 High High Option 2 High Low Option 3 Low High Option 4 Low Low Jonathan Boutelle: Understanding Organizational Stakeholders for Design Success, 2004. http://www.boxesandarrows.com/view/understanding_organizational_stakeholders_for_design_success Prioritize Stakeholders Influence Interest Option 1 High High Option 2 High Low Option 3 Low High Option 4 Low Low Jonathan Boutelle: Understanding Organizational Stakeholders for Design Success, 2004. http://www.boxesandarrows.com/view/understanding_organizational_stakeholders_for_design_success Self- and Team-Reflection Does Do your improvement team have the requisite skills? team members represent relevant perspectives? Who else should join your improvement team? Team Roster Project Year 2 (2012-2013) Team leader: Lorena Pereira Residents: HIV Coordinators: Barbara Bungy Clint Steib Yasmin Bahora Daria Ferro Zoabe Hafeez Diamond Harris Daniel Conway Elisabeth Heal David Cooperberg Ji Kong Meyeon Shin Jill Foster Peter Osgood Katie McPeak Yesha Patel Roberta Frederick-Laguerre Lauren Weaver Doug Thompson Mentors: Getting Started Selecting a High-Impact Project High Risk High Volume Problem-Prone Evidence-based (Recognized Standard) Align with Organizational Priorities Institute of Medicine Aims for Improvement The Triple Aim Improve patient experience Improve the health of populations Reduce per capita costs of health care Berwick DM, Nolan TW, Whittington J. The Triple Aim: Care, Health, and Cost. Health Aff 2008;27:759-769 The Improvement Model What are we trying to accomplish? How will we know that a change is an improvement? What changes can we make that will result in improvement? Act Plan Study Do Langley et al The Improvement Guide 2009 Introduction Introduction Brief Literature Review Nature and Severity of Local Problem Specific Aim Measures Primary and Secondary Study Questions Brief Literature Review Incidence of AIDS has increased by 21% in youth aged 13-24 years1 Half of all new HIV infections occur in 13-24 year olds2 Risk-based testing associated with delayed diagnosis3 The 2006 CDC guideline4 Recommend HIV screening all patients ages 13-24 years if prevalence > 1/1000 Two sites describe improved screening rates in patients ages 13-64 years5 8% to 53% (New York) 3% to 17% (Louisiana) No published studies describe improvement in adolescent screening rates 2Spiegel 1AAP, Pediatrics 2011 H, Current HIV/AIDS Reports 2009 3CDC, MMWR 2011 4MMRW 2006 5Lin X, et.al, MMRW 2014 Nature and Severity of Local Problem In Philadelphia, the incidence of HIV is FIVE TIMES the national average exceeding 1/1000 Prior to 2011, HIV screening at St. Christopher’s Hospital for Children Risk based Inconsistent Not always documented The Improvement Model Aim Statement What are we trying to accomplish? How will we know that a change is an improvement? What changes can we make that will result in improvement? Act Plan Study Do Langley et al The Improvement Guide 2009 Establish Clear Goals Global Aim Specific (SMART) Aim Global Aim Improve HIV Screening Rates in Adolescents “SMART” Aim Statement Specific Measurable Action-oriented Realistic Time-bound Specific SMART Aim By June 30, 2013, 70% of patients aged 13 years and above presenting to the SCHC CCAH ambulatory clinic will have a documented annual HIV screen The Improvement Model What are we trying to accomplish? Measure How will we know that a change is an improvement? What changes can we make that will result in improvement? Act Plan Study Do Langley et al The Improvement Guide 2009 Measurement for QI You can’t improve what you can’t (or don’t) measure Measures tell a team if the changes they make are making a difference Should speed improvement, not slow it down Measurement is not the goal Definitions Process Measure: whether an activity has been accomplished (i.e. was PDSA cycle carried out as planned) Outcome Measure: relate directly to aim; offer evidence that changes are having an impact at the system level Balancing Measure: make sure that other important measure does not fall off Langley, et.al, The Improvement Guide, 2nd edition Process Measure Definition: HIV Screening in Adolescents in Ambulatory Clinic Numerator: # patients age >/= 13 years for whom HIV screening test performed within the past 12 months in the Ambulatory Clinic Denominator: # patients age >/= 13 years who present to the Ambulatory Clinic Reported monthly Other measures (not specified in example) Outcome Measure Examples would be: % of newly-diagnosed patients presenting with AIDS % of newly-diagnosed patients presenting with primary HIV infection Balancing Measure (example would be rate of pregnancy testing in female adolescents presenting to Ambulatory Clinic) Primary and Secondary Study Question Primary Study Question Can we design and implement interventions that reliably improve HIV screening in adolescent patients in the ambulatory clinic? Secondary Study Question (not stated) METHODS Methods Ethical considerations Setting Planning the Intervention Planning the Study of the Intervention Methods of Evaluation Analysis Setting Describes local context and local processes Planning the Intervention Process Map of local process Key Driver Diagram Key Driver Diagram Design Changes/ Interventions Key Drivers Aim An effective tool for breaking down complex questions or improvement goals and structuring them into smaller, more-focused “drivers” Each Key Driver Focused Answers the question, “What has to go right in order to accomplish our aim?” Stated in the affirmative Key Driver Diagram Key Drivers Aim By June 30, 2013, 70% of patients aged 13 and above presenting to the pediatric ambulatory and acute care clinics at SCHC will have documentation of a HIV screen Immunology provides the tests to each clinic and is available for each positive test Nurses screen patients and administer test Patients accept testing Residents screen patients and orders test/reviews form Administration supports program Design Changes/ Interventions Immunology monthly check-in with each clinic Provide resident/nurse incentives Nurse managers buy-in to testing Screening barriers elicited Education re: screening barriers HIV screening process defined/shared Screening incentives provided Staff signs HIV consent form/test ordered Patient fills data form in clinic Acute Retroviral curriculum established Screening barriers elicited Education re: barriers HIV screening process defined/shared Frequent poster/email reminders Screening incentives provided Administration buy-in to policy change. Routinely screen patients >/= 13 years Plan-Do-Study-Act cycle ramp The Improvement Model What are we trying to accomplish? How will we know that a change is an improvement? Interventions What changes can we make that will result in improvement? Act Plan Study Do Langley et al The Improvement Guide 2009 Standardized Process for HIV Screening NO NO Screening opportunity missed Resident screens patient, fills out form and tasks nurse HIV test offered by nurse YES Test performed Results reviewed with patient YES Patient fills out data form NO Screening opportunity missed Intervention: Reminder Cards Table of Interventions (examples) Category of Intervention Specific Intervention Setting Date Initiated Identifying Risks and Collaboration with front-line staff to identify barriers to testing Barriers Primary Pediatrics Clinic Emergency Department April 2013 April 2013 Implementing Resources HIV Coordinator hired 60-second HIV test implemented Ambulatory Clinic Emergency Department March 2012 Education Peer-to-peer review of screening process Nursing/MA Education on HIV testing kit Retroviral Curriculum for Residents Ambulatory Clinic August 2012 Ambulatory Clinic September 2012 Hospital-wide January 2013 Incentives Nursing Incentives Resident/Nursing Incentives Ambulatory Clinic Hospital-wide May 2012 March 2013 Provide Reminders Resident reminders Ambulatory Clinic December 2012 Survey/Evaluation Residents, Nurses, Medical Assistants survey of knowledge, attitudes, practices Ambulatory Clinic March 2013 Policy Medical Executive Committee Approval Hospital-wide March 2, 2013 Table of Interventions Planning the Study of the Intervention Study Design Observational time series with multiple planned sequential interventions Planning the Study of the Intervention Methods to Ensure Internal Validity of Data Improve documentation of screening tests performed (Compare to laboratory administrative data) Planning the Study of the Intervention Methods to Promote External Validity/Generalizability RELIABILITY Level 1: Straw Level 2: Wood Level 3: Brick LEVEL 1 RELIABILITY Education Training Feedback Try Harder LEVEL 2 RELIABILITY Standardize process Redundancy Decision aids LEVEL 3 RELIABILITY Alter habits Real-time Review Force-function Planning the Study of the Intervention Methods to Promote External Validity/Generalizability Level 1 reliability interventions Reminders Education Incentives Level 2 reliability interventions Standardize Policy Level screening process change 3 reliability interventions Alter everyday habits Methods of Evaluation HIV coordinator tracks testing via outpatient clinic form # tests performed patient age location of testing Analysis Statistical Process Control Charts created using QI Charts©1 Rules for Detecting Special Cause Variation2 were used to differentiate special cause from common cause variation 2Shewhart 1QI Charts©, Scoville Associates 2009 WA. The Economic Control of Quality of Manufactured Product 1931 Results Results Outcomes Results Outcomes During the study period (January 2012 – April 2013) Rate of HIV screening in Ambulatory Clinic improved from 34% to 84% 3 new patients were diagnosed with HIV 1 previously known HIV+ patient who had been lost-to-follow-up was reintroduced to specialty care Statistical Process Control Statistical Process Control Powerful tool for quality improvement projects Foundation in theory of variation Shows when changes are occurring due to Special Cause Variation v. Common Cause Variation Involves Control Limits and tests of change Plot Data in Run Order Calculate the Center Line Calculate Control Limits • Average rate +/- 3*standard deviation Langley G, The Improvement Guide 2009 100% Incentives provided to residents Ambulatory Screening Rates 90% Change in policy: age of screening lowered from 15 to 13 years 80% 70% Incentives provided to nursing staff 60% 50% 40% Nursing staff education 30% Resident education and approval to screen in Acute Care as well 20% 10% 4/1/13 3/1/13 2/1/13 1/1/13 12/1/12 11/1/12 10/1/12 9/1/12 8/1/12 7/1/12 6/1/12 5/1/12 4/1/12 3/1/12 2/1/12 1/1/12 0% Understanding Variation Common Cause Variation: variation expected within a given system Example: The high temperature in Philadelphia in March (variation from -3 degrees to 60 degrees Fahrenheit may be expected) Special Cause Variation: variation beyond what is expected in a given system Rules for Detecting Special Cause A single point outside the control limits 8 consecutive points on one side of the mean 6 consecutive points increasing or decreasing 2 of 3 consecutive points in the outer 1/3 approaching the control limit Shewhart WA. The Economic Control of Quality of Manufactured Product 1931 100% Incentives provided to residents Ambulatory Screening Rates 90% Change in policy: age of screening lowered from 15 to 13 years 80% 70% Incentives provided to nursing staff 60% 50% 40% Nursing staff education 30% Resident education and approval to screen in Acute Care as well 20% 10% 4/1/13 3/1/13 2/1/13 1/1/13 12/1/12 11/1/12 10/1/12 9/1/12 8/1/12 7/1/12 6/1/12 5/1/12 4/1/12 3/1/12 2/1/12 1/1/12 0% HIV Screening Rates: Patients >/= 13 years by Month (Jan 2012 – April 2013) Overall HIV screening rates 30% 25% 20% 15% 10% 4/… 3/… 2/… 1/… 2/… 3/… 4/… 1/… 2/… 3/… 4/… 1… 1… 1… 9/… 8/… 7/… 6/… 12… 11… 10… 9/… 8/… 7/… 6/… 5/… 4/… 3/… 2/… 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% 1/… 4/… 3/… 2/… 1/… Primary Peds Screening Rates 12… 11… 10… 9/… 8/… 7/… 6/… 5/… 4/… 3/… 2/… 1/… Ambulatory Screening Rates 5/… 3/… 2/… 1/… 4/… 3/… 2/… 0% 1/… 0% 1… 5% 1… 5% 1… 10% 9/… 10% 8/… 15% 7/… 15% 6/… 20% 5/… 20% 4/… 25% 3/… 25% 2/… 30% 1/… 30% 4/… Inpatient Screening Rates ED screening rate 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% 1/… 1… 1… 1… 9/… 8/… 7/… 6/… 5/… 4/… 3/… 2/… 0% 1/… 5% Discussion Discussion Summary Relation to other evidence Limitations Interpretation Conclusion Summary Keys to Success Inter-professional Front-line collaboration staff engagement Resident Leadership Broadcasting Peer-to-Peer ‘Great Catches’ Model of Education Relation to other evidence Recently published study (June 2014 MMWR) Improved screening rates in NY and LA in patients ages 13-64 years Still well-short of universal screening Did not specifically address adolescent HIV screening Limitations Internal Validity Data not cross-referenced with laboratory administrative data Possible that testing in ED is not documented Limitations External Validity/Generalizability Local Context Limitations External Validity/Generalizability Dedicated The HIV coordinator (funded via Gilead) Dorothy Mann Center (funded via Ryan White Fund) 6 physicians (1 immunologist, 3 infectious disease specialists, 1 general pediatrician, 1 psychiatrist) 1 physician’s assistant 1 nurse practitioner 1 licensed practicing nurse 2 social workers 2 case managers Local QI Culture Interpretation Next Steps Utilize technology (in EHR) to facilitate screening Decision aids Pre-checked orders for screening Spread this intervention through a multisite collaborative Conclusion Implications for future study of improvement interventions Links between routine screening Earlier detection/treatment Decreased rate of presenting with advanced HIV infection Decreased high-risk behaviors Decreased transmission of HIV Decreased incidence of HIV in adolescents in one community Decreased healthcare costs A multisite collaborative may accelerate improvement Other Funding Other Funding Grant from Gilead Rapid HIV tests HIV tester/data collector Dorothy Mann Center is funded by Part A and Part B Ryan White HIV/AIDS Treatment Modernization Act AIDS Activities Coordinating Office in Philadelphia Examples Examples Examples Example: Abstract Submission accepted as Platform Presentation Title: Effect of a Newborn Access Program on Third Next Available Appointment and No-Show Rates in an urban, underserved academic healthcare center Katie E McPeak, M.D.1, Deborah A Sandrock, M.D.1, David Cooperberg, M.D.1, Selima N ShulerJenkins1, Bruce A Bernstein, PhD.1 and Lee M Pachter, D.O.1. 1Section of General Pediatrics, St. Christopher's Hospital for Children, Philadelphia, PA, United States. Background: Timely access (72 hrs per AAP) to post-discharge newborn (NB) care can be challenging. Prior to our intervention, NBs in our center were scheduled into resident continuity clinics. NB scheduling problems resulted in overbooking, high no-shows, and delayed access to care. Objective: To decrease time to third next available appointments (TNAA) by 50% within 6 months. Design/Methods: A multidisciplinary team was formed, consisting of practice leadership, clinical and non-clinical staff. Utilizing planned sequential interventions, this team developed and piloted a NB clinic with a new scheduling model, aiming to have NBs seen in a single point of entry to care. TNAA is calculated weekly utilizing methods published on www.ihi.org (Institute for Healthcare Improvement). No-show rates were extracted via Next Gen EPM. We used QI Charts© to create statistical process control charts. The API rules for detecting special cause variation were applied. Results: TNAA decreased by 78% within 2 months. No-show rates decreased after intervention (9.6% vs. 12.2%, p=0.05). An additional 63 NBs were seen in 6 months, at an average charge of $145/visit. This amounts to a return on investment of roughly $18,000/year. Conclusions: Timely access to NB care was achieved, aligning our center's aim with AAP recommendations. Keys to success were: a dedicated NB team, warm-call reminders, and a supplydemand scheduling model. Significant reduction in TNAA was accomplished. While temporally related, this intervention does not demonstrate causality. Next steps are to measure sustainability and replicate this model at other centers. Future outcome measures include breastfeeding sustainability and re-admission rates for jaundice and breastfeeding issues. Results #scheduled newborn appointments # kept newborn appointments # no-show newborn appointments % No show rate (#no show/#scheduled) Pre-Intervention 1128 860 137 12.15% Post-Intervention 1132 923 109 9.63% Delta (change) Post vs. Pre Implementation +4 +63 -28 -2.52% Time to Third Next Available Appointments Examples of Published QI Reports Brady PW, Muething S, Kotagal U, et.al, Improving situation awareness to reduce unrecognized clinical deterioration and serious safety events. Pediatrics. 2013;131(1):e298-308. Volpe D, Harrison S, Damian F, et.al, Improving timeliness of antibiotic delivery for patients with fever and suspected neutropenia in a pediatric emergency department. Pediatrics. 2012;130(1):e201-10. Fischer D, Cochran KM, Provost LP, et.al, Reducing central lineassociated bloodstream infections in North Carolina NICUs. Pediatrics. 2013;132(6):e1664-71.