Using routine data to measure and promote safety and quality in hospitals Stephen Duckett Adjunct Professor, ACERH/UQ (Joint work with Dr Terri Jackson Associate Professor, ACERH/UQ) THE UNIVERSITY OF WESTERN AUSTRALIA Overview What do we know about patient safety and patient harms in Australia? What do we count as patient harm? How can we measure and monitor it? How much does it cost? How can use of routine data help us understand the economics of improving patient safety? Payment policy and patient safety What do we know about Australia? Landmark 1995 QAHC study: • • • • Careful (expensive) methods Incidence: 16.6 % of multi-day inpatient stays Revised down to 10% (comparative US study) Costs to system: $900 mil pa The newspapers: • Bundaberg • The NSW ‘Cam affair’ (Cambelltown & Camden) • WA’s Royal Commission Australian Commission on Safety & Quality in Health Care (Building on Aust Council for S&QHC (1999-2005) What and how you ‘count’ depends on why you want to count it WHY count? We need to monitor rates routinely: using hospital discharge data ICD-10-AM has specific codes: • T 80.0-88.9 ‘Complications of surgical & medical care’ • ‘End of chapter’ codes • Y 40-84.9 ‘External cause--complication of surgical or medical care’ Australia has world-class quality hospital data Victorian (and now Oz) ‘condition-onset flag’ (C-prefix) denoting: • Condition required treatment or extended LOS • Condition was not present on admission Improving patient safety requires hospital-acquired incident cases Problem of distinguishing ‘comorbidities’ from ‘adverse events’ Incident vs prevalent cases • Not GP, nursing home, admission from another hospital • Overall rate in Victoria (2000/01): 8.25% (15.9% for multi-day stays) • Only two-thirds (5.61%) were ‘Incident’ cases The ‘C-prefix’ adds valuable information • 41% of all hospital-acquired diagnoses were missed by ICD alone, eg, UTI, atrial fibrillation, pneumonia • Result: new national ‘Condition Onset’ flag on all Dx Jackson TJ, Duckett SJ, Shepheard J, & Baxter KG. ‘Measurement of adverse events using ‘incidence flagged’ diagnosis codes’ Journal of Health Services Research and Policy, 11 (1):21-25; 2006. Strengths of these data Strengths • • • • • Timely & cheap data collection Standardised definitions and coding rules Includes both dramatic and mundane Independent reporting (not bedside) Current payment incentives for thorough coding Weaknesses of these data • May miss same-day patient harms Less coding investment Symptoms appearing post-discharge • Prefix not currently audited • Coders may miss clues clinicians could spot • No judgement about ‘preventability’ Notorious inter-rater reliability problems in this judgement May be a ‘strength’ as today’s routine complication becomes tomorrow’s ‘preventable’ adverse event… Risk factors and outcomes for hospital-acquired diagnoses (HA Dx) Age is an important predictor of HA Dx • 29% of patients over 85 vs 9% in the 5-9 age group Risk of HA Dx varies considerably by medical specialty • Gastroenterology 86.0% vs ENT surgery 10.5% ALOS strongly associated with HA Dx • 11.2 days with HA Dx vs 6.9 days without ALOS both a risk factor and an outcome In-hospital mortality associated (not caused) • 4.09% with HA Dx vs 1.75% without Duckett SJ, Jackson TJ, Hong Son Nghiem (2008). Risk factors and outcomes for incident complications in multi-day admissions to Victorian hospitals (manuscript). Voluntary Sentinel Event reporting compared with routinely-coded hospitalacquired diagnoses* Study Objective: To compare two sources of data on hospital sentinel events to evaluate strengths and weaknesses of both Problems with voluntary reporting: • • • • Stigma and blame attached to involvement in adverse outcomes Resulting reluctance to report Safety-aware hospitals appear to have ‘worse’ outcomes Focus on single events rather than rates reinforces individual rather than system causation Jackson T, Moje C, Shepheard J, and McMillan A. Poster presented to the 2007 Australian Conference on Safety and Quality in Healthcare, August, 2007. Using routine data to identify sentinel events This Results SE Reporting Study Victoria National 05/06 05/06 04/05 SE1 Wrong patient or body part 13 25 53 SE2 Suicide in an inpatient unit 3 7 25 53 6 27 SE4 Intravascular gas embolism 6 0 1 SE5 Transfusion reaction 0 0 1 20 2 7 2 16 SE3 Retained instruments /material SE6 Medication error SE7(1) Maternal death (O95, O96, O97) 0 SE7(2) Any maternal death 0 Public Reporting Risk adjustment important for public comparisons • Reputation risk • Risk of patients being denied treatment • Fairness to providers Public officials expected to intervene when quality declines What do you report? • What providers have ‘done’ or • What they are now doing to improve? Any measurement approach must balance the risks ‘GOLD STANDARD’ True Positive True Negative Identified Positive Identified Negative False Negative: harm to future patients False Positive: harm to reputations Problems with public reporting using routine clinical data Timeliness – data provided on an annual basis, about 15 month delay in publication Mistrust of ‘administrative’ data • • Not ‘risk adjusted’ Insufficient clinical detail Aggregated Data – unable to detect runs Nothing about cause of differences in rates Queensland Health’s Variable Life Adjusted Display (VLAD) Plot of the cumulative difference between expected and actual outcomes over a period of time AMI VLAD - ( July 2003 - March 2006 ) VLAD Estimated Statistical Lives Gained 7 6 5 4 3 2 1 0 0 20 40 60 80 100 120 140 160 180 Case number Coory M, Duckett SJ, Sketcher-Baker K. Using control charts to monitor quality of hospital care with administrative data. International Journal for Quality in Health Care 20(1): 31-39. . Simplified Variable Life Adjusted Display (VLAD) Estimated Statistical Lives Gained Patient survives - VLAD increases by the probability of the patient dying 2 1.5 1 0.5 0 -0.5 0 1 -1 -1.5 -2 Case number 2 Simplified Variable Life Adjusted Display (VLAD) / 2 Estimated Statistical Lives Gained Patient dies: VLAD decreases by the probability of the patient surviving 2 1.5 1 0.5 0 -0.5 0 1 -1 -1.5 -2 Case number 2 Simplified Variable Life Adjusted Display (VLAD) / 3 Estimated Statistical Lives Gained 2nd patient survives: VLAD increases by the probability of the 2nd patient dying 2 1.5 1 0.5 0 0 1 2 -0.5 -1 -1.5 -2 Case number 3 Simplified Variable Life Adjusted Display (VLAD) / 4 Estimated Statistical Lives Gained 2nd patient dies: VLAD decreases by the probability of the 2nd patient surviving 2 1.5 1 0.5 0 0 1 2 -0.5 -1 -1.5 -2 Case number 3 Variable Life Adjusted Display (VLAD) Plot of the cumulative difference between expected and actual outcomes over a period of time AMI VLAD - ( July 2003 - March 2006 ) VLAD Estimated Statistical Lives Gained 7 6 5 4 3 2 1 0 0 20 40 60 80 100 Case number 120 140 160 180 Lessons: Sometimes statisticians’ methods are somewhat dubious Be wary of replicating them, even if you think you understand them Specific Clinical Indicators Developed in consultation with clinical expert groups Indicators reviewed annually and refined on basis of feedback Data from the Queensland Hospitals Admitted Patients Data Collection and the Perinatal Data Collection Sex, age and comorbidities used to risk-adjust for illness severity and co-existing conditions Clinical Indicators being developed Mortality Acute Myocardial Infarction Heart Failure Stroke Pneumonia Fractured Neck of Femur Readmission Complications of Surgery Laparoscopic Cholecystectomy Vaginal Hysterectomy Abdominal Hysterectomy Fractured Neck of Femur Colorectal Carcinoma Knee Replacement Hip Replacement Prostatectomy Acute Myocardial Infarction Heart Failure Knee Replacement Hip Replacement Depression Schizophrenia Paediatric Tonsillectomy and Adenoidectomy O&G Future Caesarean Section Rate Std Primiparae Induction of Labour Rate Std Primiparae Perineal Tears (3rd or 4th Degree) Hysterectomy – on women < 35 yo Falls Pressure ulcers Overall Surgical Complications AHRQ Patient Safety Indicators Web-based documentation of VLADs How bad is a ‘bad run’? Variable Life Adjusted Display with control limits Statistical Lives Gained 5 0 0 50 100 -5 -10 -15 VLAD Lower Control limit -20 Upper Control limit Case number 150 200 Flagging criteria • 30% higher than expected mortality– automated message emailed to the district manager and clinical lead, encouraging internal investigation and report to Area Clinical Governance Unit (50% for non-mortality flags) • 50% higher than expected – flagged to Area Clinical Governance Units to ensure they are involved in further investigation (75% non-mortality) • 75% higher than expected - identified to State Patient Safety and Quality board and in public reporting as being statistically significantly different from the average (100% non-mortality) Duckett SJ, Coory M, Sketcher-Baker K. Identifying variations in quality of care in Queensland hospitals. Medical Journal of Australia 187 (10): 571-575. VLAD charts allow tracking to specific case files Detailed data underpinning the VLAD system QH Pyramid Model of Investigation Professional Process of Care Structure of Resource Patient Case Mix Data Figure 1 : Pyramid Model of Investigation Key strength of routine data: links to patient costs Victorian and Queensland CWS data • n≈ 1.6 mil records pa • n≈ 100 large public hospitals • Validated for use in hospital funding What can cost data tell us? • Economic burden of adverse events • Economic priorities for prevention efforts • Business case for: prevention efforts medical research • Cost-effectiveness analyses for: prevention programs new patient safety devices and procedures Incident cases represent a large economic burden to the health care system Patients with at least one C-prefixed adverse event: • Stay nearly 10 days longer than other patients • Cost $ 6826 more per episode (controlling for DRG, age and comorbidity) Extrapolated to entire hospital system: • • • • At least $511.5 mil additional cost in Victoria (2003/04) Adds 18.6% to hospital expenditures Around $2 bil pa nationally Even if only 40% preventable: $200 mil pa could be saved in Vic; $800 mil nationally Ehsani JE, Jackson TJ and Duckett SJ. ‘The incidence and cost of adverse events in Victorian hospitals, 2003-04’ Medical Journal of Australia, 184;11; 5 June 2006 Re-admissions* add to this cost burden Current work on Victorian separations with a PDx in the T80-88.9 range of ICD-10-AM: • 16,734 admissions with a PDx of a ‘complication of surgical or medical care’ • $70.6 mil pa additional public expenditure on these cases in Victoria** *Includes admissions for HA Dx from primary care and nursing homes **McNair P, Jackson TJ, Borovnicar D. ‘Costs of Victorian admissions for treatment of adverse-event principal diagnoses, 2005/06’ submitted for publication, August 2008. Queensland Health P4P approach Incentives at margin ($8M out of $7B budget, 0.1% of total, larger % of specialty) Range of specialties, data collection approaches, foci Specialties: • • • • • Mental health Stroke Emergency department care COPD Medication Duckett, S.J et al (2008) ‘P4P in Australia: Queensland’s New Clinical Practice Improvement Payment’ Journal of Health Services Research and Policy 13:174-177 Mental health indicators Indicator Patients with the DRG Schizophrenia seen by a community mental health professional within 7 days following discharge from the same district mental health service provider. Recording of antipsychotic injection (depot) medication on iPharmacy for DRG Schizophrenia Payment amount etc $100 Volume: 4518 Electronic data collection in place $100 Volume: 4518 Electronic data collection in place Acute stroke indicators Indicator Payment amount etc Acute Stroke patients with acute ischaemic stroke receiving antiplatelet therapy within $50 48 hours if clinically appropriate Acute Stroke patients receiving dysphagia screen (minimum requirement) within 24 hours Volume: 6777 Manual data collection system in place $125 Volume: 8699 Manual data collection system in place Emergency indicator Indicator Payment amount etc All patients aged 65 years (or 50 years if ATSI) and $100 over who are admitted to and discharged from an Volume: 28962 emergency department to home/nursing home have Electronic data collection in place evidence of communication back to the GP/LMO Medication indicator Indicator Electronic Discharge Medication Record completed where patient is over 65 years and has a complex medication regimes Payment amount etc $100 Volume: 70000 Electronic data collection in place COPD indicator Indicator Payment amount etc Pulmonary rehabilitation program meets an acceptable standard as defined by the Statewide Chronic Obstructive Pulmonary Disease (COPD) Clinical Network. $10 000 Volume: 20 (review of unit) 12 month audit of all sites (one off data collection) Queensland Health approach Moral suasion will be used to encourage most of (80%) CPIP payment to flow to clinical unit where indicator activity is being undertaken. CPIP only to be used for non recurrent expenditure items CPIP to be used to enhance not expand service (eg increase number of operations performed) CPIP - summary Started in January 2008 Experiments with very different approaches initially Mostly Pay for process adherence/reporting rather than outcomes Can be rolled out to other indicators, add/drop indicators Stephen_duckett@health.qld.gov.au