Nurse Staffing and Failure to Rescue
Jack Needleman, PhD, FAAN
UCLA School of Public Health
Presented at
AcademyHealth
June 29, 2009
Scope of Presentation
•
•
•
The Failure to Rescue (FTR) as adopted by
 NQF in its hospital nursing sensitive performance set
 AHRQ in its Patient Safety Indicators
Reviewing and revising the FTR exclusion rules to focus on
hospital-acquired avoidable complications
Validating the revised rules as sensitive to nurse staffing
2
Why discuss Failure to Rescue
•
•
•
•
•
Adopted as nursing sensitive measure by National Quality
Forum
Incorporated into AHRQ Patient Safety Indicators
Used in research on nurse staffing and patient safety
Patients understand the concept and importance of it
Controversy about appropriate definition and concern about
patient acuity and the impact of whether the complications are
present on admission
3
Acknowledgements
•
Supported in part by
 Robert Wood Johnson Foundation Interdisciplinary Nursing
Quality Research Initiative
4
The Team
Phase 1
PIs
• Marcy Harris PhD,RN
 Mayo Clinic
• Jack Needleman PhD
 UCLA
• Peter Buerhaus PhD,RN
 Vanderbilt
Mayo Clinic
• Cynthia Leibson PhD
• Jeanine Ransom
• Shane Pankratz PhD
• Chris Farmer MD
• Kathy Dickson
Phase 2
Mayo Clinic
• Mike Malinchoc
• Catherine Vanderboom PhD,RN
• Andrew Hanson
• Suzanna Stevens
Phase 3
• Ronda Hughes PhD,RN
 AHRQ
5
History of Failure to Rescue
•
•
•
Concept initially presented by Silber et al, Medical Care 1992
 Patient characteristics associated with death after surgery
 Used by Aiken and colleagues in studies of nurse staffing
and patient deaths, JAMA 2002
Idea substantially modified by Needleman and Buerhaus for
study of nurse staffing and patient outcomes, NEJM 2002, using
the approach developed by Iezzoni and colleagues in the
Complications Screening Program (CSP)
 Select complications based on secondary diagnoses
 Restrict to “hospital acquired” via exclusion rules
Needleman-Buerhaus measure
 Adopted by NQF as part of nursing sensitive measure set
 Modified and added to AHRQ PSIs
6
NQF FTR
•
•
Renamed by NQF “Death among patients with preventable
complications” (i.e., hospital acquired complications)
Identify patients with one of 5 complications reported as
secondary diagnosis
 Exclude patients for which complication believed to be POA
 Since no “present on admission (POA)” coding in most
administrative data sets, need to
 Infer which were POA from primary diagnosis
– E.g., shock/cardiac arrest if Primary Dx trauma
– Two decades of work on exclusion rules based on expert
panels of clinicians identifying clinically related primary
diagnoses
 Analyze death rate for defined pool of patients
7
Administrator and clinician concerns
about FTR
•
•
•
Risk adjuster too complicated to implement in hospital
 In revision, will use AHRQ standard risk adjustment method
Exclusion rules do not exclude enough complications that were
comorbidities “present on admission”
 Holding clinicians/hospital responsible for conditions could
not prevent
– “POA or hospital acquired, need to manage”
– UHC study: Exclusions imperfect, deaths higher for
hospital-acquired complications
CMS and some state data system requirements for POA flags
may reduce need for exclusion rules but needed until substantial
compliance and to track changes from legacy data sets
8
Study Aims
•
•
•
Using Mayo Clinic data as gold standard for POA, develop
refined exclusion rules with goal of reducing inclusion of POAs.
 POA coded discharge abstracts
 Clinical/medical chart review
Cross validate Mayo findings in California multi-hospital data
set with POA flags
Using HCUP SID set, assess association of nurse staffing and
FTR using refined exclusion rules
9
What proportion of complications that
make up FTR are POA?
Complication
Number of cases
Proportion
POA
Acute renal failure
18,064
40%
Deep vein thrombosis
10,640
60%
GI bleed
18,654
81%
Pneumonia
27,167
33%
Shock/cardiac arrest
18,258
54%
Sepsis
16,406
55%
Source: California OSHPD data, 2000 and 2001
10
Death rates are higher for patients with
hospital-acquired complications
Percent Deaths
Complication
Hospital Acquired
POA
Acute renal failure
31%
23%
Deep vein thrombosis
14%
7%
GI bleed
17%
6%
Pneumonia
17%
14%
Shock/cardiac arrest
54%
49%
Sepsis
39%
28%
Source: California OSHPD data, 2000, 2001
11
Length of Stay is longer for patients with
hospital acquired complications
Length of Stay
Complication
Hospital Acquired
POA
Acute renal failure
15.4
11.5
Deep vein thrombosis
15.5
8.8
GI bleed
15.0
7.6
Pneumonia
16.1
11.4
Shock/cardiac arrest
12.1
9.1
Sepsis
20.1
12.8
Source: California OSHPD data, 2000, 2001
12
Implication
•
Distinguishing hospital acquired complications and present on
admission comorbidities contributes to understanding deaths
and resource use in hospitals
13
How well can exclusion rules distinguish
hospital-acquired and POA
•
•
Exclusion rule assumptions
 POA comorbidities are closely tied to primary diagnosis
 Hospital-acquired complications widely distributed among
admitting diagnoses
Implication
 Bimodal distribution
 Exclusion rules should have high sensitivity and specificity
14
Malignant neoplasm of lymph nodes
Septicima
15
POA not bimodal or tightly tied to primary DX
No clear cutoff identifies POA
Need alternative strategy for defining exclusions
Five rules tested
 No exclusions
 AHRQ (Expert panel, clinical judgment)
 50% POA cutoff
 Analyze diagnoses at 3 digit ICD9 level, and choose cutoff
percentage that optimizes balance between sensitivity &
specificity, maximizes area under the ROC (C-statistic)
 3 digit rule refined with clinical judgment applied at 4 digit
level
16
Does refining the exclusion rules improve
the FTR measure
•
•
•
Optimizes sensitivity and specificity
 Strikes a balance between false positives and false
negatives
Do rules better identify those at risk of death?
Are new definitions as/more nursing sensitive?
17
Compare Area Under ROC Curve (c-statistic)
AHRQ vs 3-digit C-stat maximizing exclusion rules
(CA data)
AUC Using
Current AHRQ
Rules
POA cutoff that
Maximizes
AUC
AUC Using
Revised
Exclusion Rule
Acute Renal Failure
0.501
40%
0.607
Deep Vein Thrombosis/PE
0.578
65%
0.644
GI Bleed
0.508
80%
0.605
Pneumonia
0.518
33%
0.579
Shock/Cardiac Arrest
0.511
54%
0.652
Sepsis
0.678
56%
0.673
Complication
18
Compare Area Under ROC Curve (c-statistic)
AHRQ and alternative exclusion rules
(CA data)
Complication
AHRQ
50%POA
Simple 3
Refined 4
Acute Renal Failure
0.501
0.590
0.607
0.614
Deep Vein Thrombosis/PE
0.578
0.630
0.644
0.648
GI Bleed
0.508
0.512
0.605
0.606
Pneumonia
0.518
0.542
0.579
0.584
Shock/Cardiac Arrest
0.511
0.650
0.652
0.662
Sepsis
0.678
0.673
0.673
0.679
Overall
0.553
0.684
0.633
0.638
19
Death rates for each condition by
exclusion rules applied
CA
Hospital
Acquired
AHRQ
50%POA
Simple 3
Refined 4
Acute Renal Failure
30.7%
26.1%
27.8%
27.3%
27.5%
Deep Vein Thrombosis/PE
13.6%
9.9%
9.8%
10.3%
10.1%
GI Bleed
16.7%
7.9%
9.7%
8.3%
8.2%
Pneumonia
17.0%
16.3%
15.9%
15.6%
15.5%
Shock/Cardiac Arrest
54.5%
49.9%
54.7%
55.7%
54.8%
Sepsis
39.5%
32.9%
36.9%
36.9%
36.8%
Overall
22.5%
17.3%
20.2%
19.5%
19.2%
Complication
20
Validation analysis
PRELIMINARY ANALYSIS
INCIDENCE RISK RATIO
No
exclusion
AHRQ
50%POA
Simple 3
Refined 4
Licensed hours/inpatient day
0.993
0.991*
0.994
0.993
0.993
RN as percent of licensed
0.710*
0.588***
0.619**
0.619**
0.612**
Nurse staffing variable
* p<0.05 **p<0.01 ** p<0.001
Hospital-level binomial regression with number of deaths regressed on number of patients
in pool, with hospital level controls for bedsize, teaching status, rural location and technology,
robust standard errors adjusted for clustering by hospital
Data sources:
Discharge data for construction of pools and deaths: HCUP NIS 2000-2003,1448 hospital-years
Staffing data: AHA staffing, adjusted using Medicare cost report data for inpatient/outpatient
nursing allocation
Other hospital variables: AHA annual survey
21
Implications and Next Steps
•
•
Next steps:
 Complete validation study
– AHRQ PSI patient level risk adjuster
– Add Silber FTR measure
– Using California data, add POA-flag based exclusions
Implications and conclusions
 Consequences of complications different from POA
comorbidities
 In designing quality measures, clinical judgment needs to be
informed by data
 Value in measures with robustness in face of measurement error
22