Benchmarking Critical Care Outcomes:
Using data to drive effectiveness and efficiency
Thomas L. Higgins MD MBA
Vice Chair for Clinical Affairs, Department of Medicine, Baystate Medical Center, Springfield MA
Professor of Medicine, Surgery & Anesthesiology, Tufts University School of Medicine
5,815
0.86
nr
APACHE-IV (2002-3)
110,558
0.88
Better than expected
Mortality and LOS
0.08
20.0
Project IMPACT Data 2005-06
<0.001
SAPS-III (2002)
16,784
0.85
0.39
0.0
12
0.104
FY
0.86
11
13,152
FY
SAPS-II (1993)
5.0
10
0.31
FY
0.82
09
124,885
Hosp LOS
10.0
FY
MPM0-III (2001-4)
Hospitals within control limits
08
0.62
FY
0.84
07
12,610
ICU LOS
FY
MPM0-II (1993)
15.0
06
0.87
FY
216,626
05
ICNARC (1995-2003)
– 13.5% in 44,288 patients, APACHE-IV validation (2002-2003)
• Zimmerman et al, Crit Care Med 34:1297, 2006
25.0
FY
• Higgins et al, Crit Care Med 35:827, 2007
APACHE-II (1985)
04
Hospital Mortality Rate for ICU patients: ~6 to 19%
– 13.8% in 124,855 patients, Project IMPACT (2001-2004)
HLGOF, p
MICU + SICU Patients, BMC, 2002-2012
excludes Heart & Vascular (CVICU, CCU)
FY
•
AUROC
from Nathanson et al, Crit Care Med 2007; 35:1853
03
• Halpern and Pastores, Crit Care Med 2010; 38:67-71
n
Length of Stay Reduction
FY
93,955 CCM beds in 3,150 Hospitals (increasing 1%/yr 2000-2005)
23.2 million patient days (10.6% increase over 5 years)
Cost per day: $3518 (30.4% increase over 5 years)
Total costs: $ 81.7 Billion (44.2% increase over 5 years)
– Critical Care accounts for 13.4% of hospital costs
– 4.1% of national health expenditures
– 0.66% of GDP (rate of increase = 3.6% per year)
Model
Resource Utilization Graph
02
•
•
•
•
Current ICU Benchmarking Tools
FY
Critical Care Medicine in the US:
Big business, and growing
Worse than expected resource
utilization (Length of Stay)
AUROC = area under receiver operating curve, ideally >0.80)
HLGOF = Hosmer-Lemeshow Goodness of Fit, ideally >0.05
Central Line Associated Blood Stream
Infections (CLABSI)
Who wants to know? Patients, Families,
Physicians, Administrators, Insurers, the media…..
Driving Change
Standardized Mortality Ratio (SMR)
•
Observed Risk-Adjusted Mortality
SMR =
•
Expected Risk-Adjusted Mortality
Values 2 SD > 1.0 may indicate poor performance
Values 2 SD < 1.0 indicated superior performance
•
Major Domains of Interest
•
•
•
•
Clinical Quality
– Standardized mortality rate (observed/expected)
– ICU and hospital lengths of stay
– Complications (CR-BSI, VAP, “never” events)
– Patient and family satisfaction
Human Capital
– Engagement, turnover, morale
Financial Performance
– Revenue and Costs (Part A and Part B)
– Resource Utilization by provider
Academics: Research and Education
•
•
•
•
•
Mortality outcomes are highly dependent on presenting patient
condition
–
Unadjusted results misleading
• Mortality rate for DKA <2%
• Mortality rate for septic shock ~30%
Case-mix thus affects unadjusted overall mortality rate
Adjusted data is required for internal Quality Improvement efforts
Risk stratification helpful (but not infallible) for individual patient
prognosis
Risk-adjustment models must meet criteria for discrimination (area
under ROC >0.80) and calibration (non-significant HL-GOF)
)
Other Domains of Interest
•
Clinical Quality
–
Patient and family satisfaction – H-CAHPS Scores
Human Capital
–
Engagement, turnover, morale – Gallup EmployeeSurvey
Financial Performance
–
Revenue and Costs (Part A and Part B) – Income
Statement
–
Resource Utilization by provider – Premier Database
Academics: Research and Education –
–
Grant funding, number of publications, faculty teaching
evaluations (New Innovations)
•
•
•
Driving Change Using ICU Benchmarking
Tools
Example of calculating SMR for a hypothetical ICU
One patient each; 10 diagnoses
Accurate Risk Stratification Needed
Normalized ratios
can be created for
any outcome: e.g.:
ventilator days
In this example,
ventilator days are
higher than
predicted, indicating
an opportunity for
improvement
Interventions could
include education,
institution of “daily
wake-up”, attention
to VAP and CLABSI,
respiratory therapy
protocols, or twicedaily weaning trials
Patient Diagnosis
Predicted
Actual
DKA
2%
0
Pneumonia
Asthma
Acute MI
Septic Shock
Pneumonia
Heart Failure
Septic Shock
Ruptured AAA
Heart Failure
AVERAGE:
12%
10%
24%
30%
12%
15%
30%
65%
15%
21.5
0
0
0
1
0
0
0
1
0
0.20
•
SMR for this ICU=
Observed (20%)
Predicted (21.5%)
•
= 0.93
•
Morbidity and mortality
–
Evidence-based bundles / ordersets; CPOE, medication
scanning; alerts, early warning
–
Excess length-of-stay
–
Admission, discharge, triage policies
–
Open versus closed units
–
Ventilator weaning and sedation practices
Ventilator-associated Pneumonia
–
Ventilator “bundle” of care including HOB elevation
–
Respiratory therapy equipment and change-out policies
Catheter-related Bloodstream Infections
–
Attention to technique and tools
–
Operator training restrictions
Summary
•
•
•
•
•
•
Measuring ICU performance requires a balanced scorecard
Outcomes must be severity-adjusted
–
Tools include APACHE, MPM, SAPS
–
Endpoints include mortality, LOS
–
Normalized ratios/benchmarking can drive change
Readmission rates must also be severity-adjusted but once
adjusted do not correlate with case-mix adjusted mortality or other
quality measures, raising questions about CMS use of metric
–
Kramer et al, Crit Care Med 2013; 41:24-33
Quality metrics also include CLABSI, VAP, complications and
patient satisfaction
Track employee engagement as well as family satisfaction
Academic institutions may also track research productivity,
teaching evaluations, publications
www.postersession.com