Diagnostics Azhar Ali, MD Frank Federico, RPh Description Delivering safe care involves understanding the root of the defects that impact safe. In order for us to understand our systems of care, we need a diagnostic journey that moves out of a model for judgment and into a model for learning. A number of data sources are available for organizations to identify the rate and type of harm that patients may be experiencing, Using the data from diagnostics can be helpful to draw conclusions to help focus improvement efforts. During this session, participants will learn and share their experiences with using diagnostics to improving safety. Objectives Describe the diagnostic journey that organizations must take in order to improve patient safety List the diagnostic tools available to identify defects Develop a plan to use diagnostics in your organization Why Are You Here? What interest brought you to a data session? What data do you use every day to improve care? Juran Trilogy QUALITY PLANNING QUALITY IMPROVEMENT QUALITY CONTROL Data Uses Data is used for several purposes in healthcare – Research – Accountability – Improvement What data uses do you see in daily work? The Three Faces of Performance Measurement Aspect Aim Improvement Accountability Research Improvement of care (efficiency & effectiveness) Comparison, choice, reassurance, motivation for change New knowledge (efficacy) Methods: • Test Observability • Bias Test observable No test, evaluate current performance Test blinded or controlled Accept consistent bias Measure and adjust to reduce bias Design to eliminate bias • Sample Size “Just enough” data, small sequential samples Obtain 100% of available, relevant data “Just in case” data • Flexibility of Hypothesis Flexible hypotheses, changes as learning takes place No hypothesis Fixed hypothesis (null hypothesis) • Testing Strategy Sequential tests No tests One large test • Determining if a change is an improvement Run & Control charts Analytic Statistics (statistical process control) No change focus (maybe compute a percent change or rank order the results) Enumerative Statistics (t-test, F-test, chi square, p-values) • Confidentiality of the data Data used only by those involved with improvement Data available for public consumption and review Research subjects’ identities protected Solberg, L I; Mosser, G; McDonald, S "The three faces of performance measurement: improvement, accountability, and research." The Joint Commission journal on quality improvement 23, No. 3 1997, pp. 135-47. Vulnerable System Syndrome Three core pathologies - Blame - Denial - And the pursuit of (the wrong kind of) excellence How can we learn about our system performance? Sources of Information Mortality Review Trigger tools Concurrent review Incident Reports Waste Report Observation Pharmacy Interactions Patient Complaints KPI and Reliability of processes Culture of safety assessment Access Diagnostic Journey Do people die unnecessarily every day in our hospitals? In order for us to understand this, we need a diagnostic journey that moves out of a model for judgment and into a model for learning. The Mortality Diagnostic – 2x2 Matrix Review most recent 50 consecutive deaths. Place them into a two by two matrix based on: - Was the patient admitted for palliative care? - Was the patient admitted to the ICU? Focus your work initially on boxes that have at least 20% of your mortality. Diagnostic – The 2 x 2 Matrix Admitted to the ICU? Yes Admitted for Palliative Care Only? No Yes Box #1 Box #2 No Box #3 Box #4 The Mortality Diagnostic - Failure to Recognize, Plan, Communicate Analyze deaths in box 3 and 4 for evidence of failure to: recognize, communicate, plan. This will help you understand the local environment. Recognize, Communicate, Plan - Failure to Recognize: Any situation in which a patient has died and there was evidence that an intervention could have been made anytime prior to the patient’s death Example: the staff was worried, change in heart rate, change in respiratory rate, change in blood pressure, change in O2 saturation or change in consciousness or neurological status that was not responded to. - Failure to Plan, such as: diagnosis, treatment, or calling a rescue team. - Failure to Communicate: Patient to staff, clinician to clinician, inadequate documentation, inadequate supervisor, leadership (no quarterback for the team), etc. The Mortality Diagnostic - The Impact of Care Evaluate ALL deaths in box 3 and box 4 to assess the estimated impact of our care on mortality: *As you review the deaths in box 3 & 4, ask yourself the questions honestly (focusing on learning, not judgment): – – Was perfect care rendered? If perfect care wasn’t rendered, could the outcome of death have been prevented if the care had been better? – What number of deaths could have been prevented? The Mortality Diagnostic - Evidence of Adverse Events Analyze deaths in box 3 and 4 for evidence of adverse events using the Global Trigger Tool. This will give some further direction to local problems. Source; Helen Lau, R.N., M.H.R.O.D., Kerry C. Litman, M.D. “Saving Lives by Studying Deaths: Using Standardized Mortality Reviews to Improve Inpatient Safety” The Joint Commission Journal on Quality and Patient Safety. September 2011 Volume 37 Number 9 Accepting the Harm Burden Concept of moving from a focus on error and the preventable to the measurement of global institutional harm whether preventable or not Definition of Harm In the IHI Global Trigger Tool, the definition used for harm is as follows: Unintended physical injury resulting from or contributed to by medical care that requires additional monitoring, treatment or hospitalization, or that results in death. New (Harm) vs. Old (Errors) Concentrates less on people more on systems Looks at all unintended results Makes measurement easier Concentrates on harm and those errors that cause harm Errors are the focus of discussion and solutions Tends to focus only on those results felt to be related to error, ignores other events Requires judgment Human found responsible for most of the errors Why Use Trigger Tools? Traditional reporting of errors, incidents, or events does not reliably occur in the best of health care cultures Voluntary methods markedly underestimate adverse events Events can be reliably detected without resorting to as yet unproven electronic surveillance methods Can be integrated into a good sampling methodology to follow event rates over time IHI Global Trigger Tool Review chart for triggers that are sensitive and specific for harm Find a trigger- was there harm? Not all triggers mean there was harm! IHI Global Trigger Tool Modules Cares (General) Critical Care Medication Surgery L&D ED Cares Module Triggers C1 Transfusion or use of blood products C2 Any Code or arrest C3 Dialysis C4 Positive blood culture C5 X-Ray or Doppler studies for emboli C6 Abrupt drop of greater than 25% in Hg or Hemtocrit C7 Patient fall C8 Decubiti C9 Readmission within 30 days C10 Restraint use C11 Infection of any kind C12 In hospital Stroke C13 Transfer to higher level of care C14 Any procedure complication C15 Other How it is Actually Done 1 - Set your timer for 20 minutes 2 - Review the coding summary (look for ecodes and obvious events) 3 - Review the discharge summary 4 - Review the lab 5 - Review the x-ray reports 6 - Review the procedure notes 7 - Any time left over, review nurse notes Example of a Trigger: Transfer to higher level of care Endoscopy Post procedure somnolent and hypotensive (BP 80) transferred to ICU Placed on Bi-Pap Received standard meperidine and midazolam for procedure Given flumazenil; stayed in unit 12 hours. Concurrent Review • Definition of Concurrent Review: - Real-time view of patient care related to the specific quality indicator being measured. • Goal: - Improve quality of care during present patient admission. • Reviewer Qualifications: - Adequate (clinical) knowledge/experience of subject matter and ability to synthesize and provide feedback. Concurrent Review Process Identify patients with a need for daily review – This can be the most challenging piece – Use IT/administrative systems when possible Review specifics of chart Analyze and synthesize information Provide feedback (with the potential for an intervention …) – One-on-one dialogue – Weekly Reports/feedback from leadership – Stats – Outliers – Review of guideline in question – Documentation issues – Staff Kudos! Pneumonia Performance: ED Measures 90 80 1 missed case per measure. Cases not picked up on concurrent review Concurrent review begins 70 BC Draw n Prior to Initial ABX Started (ED) ABX Tim ing (6 hours) Tim eline Q1 2010 Q 4 2009 Q3 2009 Q2 2009 Q1 2009 Q4 2008 Q3 2008 Q2 2008 Q1 2008 60 Baseline Performance (%) 100 Pneumonia Performance: Vaccine Measures Performance (%) 10 0 90 80 Concurrent review begins Q3 2008 70 Pneumovax Flu Vaccine 60 B aseline 2007 2008 Tim eline 2009 2 0 10 Incident/Voluntary Reports Effectiveness Findings from IHI GTT studies 34 How Much Harm ‘Global Trigger Tool’ Shows That Events in Hospitals May Be Ten Times Greater Than Previously Measured Classen DC, Resar R, Griffin F, et al. Global Trigger Tool shows that adverse events in hospitals may be ten times greater than previously measured. Health Affairs. 2011 Apr;30(4):581-589 Waste Identification Tool Ward Module Bed Occupied or Used Inappropriately Healthcare-Associated Infection Adverse Drug Event Procedure Complication Unnecessary Hospitalization Flow Delay Clinical Care Delay Patient Care Module Monitoring Invasive Devices Medications Tests Therapies Diagnosis Module Urinalysis Thyroid function studies Electrocardiogram (ECG) Chest x-ray (CXR) Metabolic panel (typically includes glucose, electrolytes, proteins, kidney function tests, and liver enzymes) Treatment Module Anticoagulation Glucose management Post-operative care for high-volume procedures Pain control Others Patient Module The Patient Module is meant to function as a reality check regarding what patient perceive as helpful and valuable in their inpatient care. Qualitative Measure Patient Module Identify five adult patients scheduled for discharge to the home setting and who are capable of participating in a brief interview. First explain the purpose of the interview and obtain permission. Be sure to inform patients as to what information will be noted and how it will be used. Patient Module Only interview patients who are willing to participate. Record some brief notes with the patient’s comments and perspectives in the worksheet. Analysis The modules and various waste streams described in this Tool represent the starting point in a journey to reduce significant waste in an acute care hospital. Observation Method Direct observation of a process in the natural setting Developed by Ken Barker at Auburn University Observation Method The method is easily understood, Data are easy to use for identifying trends and benchmarking, Data are available within hours The method is systems oriented and views errors in doses as defects in a system, The method is objective and does not assign blame Observation Method The method is defensible, with all doses being examined and errors witnessed The method enables problem-based continuing education that focuses on “our” errors The method facilitates evidence-based testing that can evaluate proposed system changes Quality can be measured quantitatively by third parties Pharmacy Interventions Source of information for medication errors and medicationrelated events Source of valuable information Not used to full potential Patient Complaints http://www.ncbi.nlm.nih.gov/books/NBK43703/ Culture of Safety Access and Flow Reliability is failure free operation over time. Failures: readmissions within 31 days related dx 25 20 15 10 5 0 1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 31 Day Readmission Analysis 100 random charts reviewed (total of 244 readmissions within 31 days for the year). Charts reviewed by physicians with a standard chart review worksheet. Worksheets reviewed and data for production defects, environmental defects extracted. Production Defects 40% 37.50% 35% 30% 25% 22.50% 20% 15% 15% 12.50% 10% 10% 5% 0% 2.50% Poor Discharge Surgical Complicatons Poor Hospital care Procedure/Rx not successful Infection Other Environmental Defects 60% 59% 50% 40% 30% 20% 15% 11% 10% 3.70% 0% Poor Outpatient Management Lack of Support LTC Facility Problems Non Compliant Patient 3.70% Unable to get meds 3.70% Unable to get appointment Poor Outpatient Management Poor outpatient pain control program (31%) Poor CHF outpatient follow up program (31%) Multiple other issues (37.5%) Defects arise from access to care, medication, self care Primary strategies care Reliability: failure free operation over time for a patient CHF ED Direct admit transfer Defects that arise over the LOS: variation from best care, Med-surg. unit Defects that arise from factors that affect care over time: Nutrition, environment, medication availability, poor discharge planning , home Home/rehab/nursing High reliability organizations are continually on the lookout for novel types of system failure and have several contingency plans. Sensemaking How does one make sense of all of the data? Meyer GS, Nelson EC, Pryor DB, James B, Swensen SJ, Kaplan GS, et al. More quality measures versus measuring what matters: a call for balance and parsimony. BMJ Qual Saf. 2012 Aug 14. A Structure of Measures & Improvement Whole System Tier 1 Macro “MesoSystem” “MesoSystem” “MesoSystem” Tier 2 Meso Project Project Project Project Project Project Tier 3 Micro Project Project A Structure of Measures & Improvement Whole System Tier 1 Macro “MesoSystem” “MesoSystem” “MesoSystem” Tier 2 Meso Project Project Project Project Project Project Tier 3 Micro Project Project Impact of Different Measures Macro System • Leadership • Support for mission and goals • Sets the agenda • Structures that assure process and outcomes in place • E.g. Skills and time for daily improvement Macro System Measures IOM Dimension Whole System Measures Safe • Adverse events that cause harm • Work days lost Effective • Hospital standardized mortality ratio • Unadjusted (raw) mortality • Functional outcomes • Readmission percentage Patient-Centered • Patient experience Timely • 3rd next appointment available; waits and delays Efficient • Patient days during the last 6 months of life • Costs per capita Equitable – for all Impact of Different Measures Meso System • Outcomes important to guide actions • Process measures guide activities to support excellent, safe care • Managers use data to set agenda to support larger organizational goals • Daily improvement Meso System Measures Aims Measures Safe • Falls this month • Fall rate • Employee injury rate Effective • Sepsis protocol implementation % • DVT bundle • Transitions home steps implemented Patient-Centered • Bedside change of shift report • Physician communication steps implemented Timely • 3rd next appointment available Efficient • Flow delays (E.g. ED to inpatient; OR room turnover) Equitable – for all Impact of Different Measures Microsystem • Process measures help focus on daily activities • What can we learn from this case – this one? • This fall; this missed lab; this wrong dose • Drives planning of work and resources • Outcomes important for focus • For day-to-day activities, focus is ‘what can I do to change all the things that need to improve?! Micro System Measures Aims Measures Safe • Falls this month • Fall rate • Employee injury rate Effective • Sepsis protocol implementation % • DVT bundle • Transitions home steps implemented Patient-Centered • Bedside change of shift report • Physician communication steps implemented Timely • Patient rooming process in clinic Efficient • Flow delays (E.g. ED to inpatient; OR room turnover) Equitable – for all The Differences Are… Where you are in the organization Primary role Scope of influence Display of Data One Example of Data Use Safe Care – Measures Measures of – • Misuse • DVT care not fully completed • Over-use • Prolonged sedation in the ICU • Under-use • Access to care; waits and delays due to flow problems http://www.psnet.ahrq.gov/popup_glossary.aspx?name=underuseoverusemisuse Types of Measures Process – Measure the effectiveness of a process, which if done well should deliver a good outcome Outcome – Measure the effectiveness of a change or intervention in delivering the desired outcome Balancing Measure Why is it important to link multiple measures? No one measure provides the complete picture Activities in one area impact results in an other Process measures should be linked through evidence to outcomes Balancing measures help understand unintended consequences Focus on the Vital Few! There are many things in life that are interesting to know. It important to work on those things that are essential to quality and safety. The challenge, therefore, is to be disciplined enough to focus on the essential, vital few. Building a cascading system of measurement 83 Resources White papers – Mortality http://www.ihi.org/knowledge/Pages/IHIWhitePapers/MoveYourDot MeasuringEvaluatingandReducingHospitalMortalityRates.aspx http://www.ihi.org/knowledge/Pages/IHIWhitePapers/ReducingHos pitalMortalityRatesPart2.aspx – Global Trigger Tool http://www.ihi.org/knowledge/Pages/IHIWhitePapers/IHIGlobalTrigg erToolWhitePaper.aspx