LESSONS FOR NURSING FROM THE AHRQ
PATIENT SAFETY INDICATOR VALIDATION
PILOT PROJECT
A d
AcademyHealth
H lth
June 26, 2010
Pat Zrelak1 PhD, CNRN
CNRN,, CNAA
CNAA--BC, Patrick Romano MD, MPH; Garth Utter
MD, MS; Dan Tancredi PhD; Ruth Baron RN, BSN; Banafsheh Sadeghi PhD,
MD, Jeff Geppert JD, MS.
Patient Safety Indicators (PSIs
(PSIs))
 Initially developed through a contract with UC
UC--Stanford
Evidence--based Practice Center
Evidence
 Set of quality indicators
 Readily available inpatient hospital discharge data
(ICD--9-CM codes)
(ICD
 Identify potentially preventable adverse events
 Little is known about the criterion validity
AHRQ PSI Validation Pilot
 Gather evidence on the criterion validity of the PSIs
based on medical record review
 Develop medical record abstraction tools
 Improve guidance on interpretation & use
 Evaluate potential refinements to the specifications
 Develop mechanisms for conducting validation studies
on a routine basis
Purpose
To review the descriptive findings from the PSIs
evaluated to date as part of the AHRQ PSI Validation
Pilot Project in the context of nursing practice.
PSIs Reviewed
AHRQ
Q
UHC
C
Postoperative pulmonary
embolism or deep vein
thrombosis
Postoperative pulmonary
embolism or deep vein
thrombosis
Selected infection due to
medical
di l care (CLABSI)
Postoperative respiratory
f il
failure
Accidental puncture and
laceration
Decubitus ulcer
Postoperative sepsis
Iatrogenic pneumothorax
AHRQ Methods
 Retrospective cross-sectional study
 US volunteer sample of 47 hospitals from 29 states
 Sampling based on administrative data
 Data included 4th Qtr 2005, 2006, & 1st Qtr 2007
 Sampling probabilities assigned using AHRQ QI
software to generate desired sample size nationally
Data Collection
 Each hospital identified chart abstractors
 Medical record abstraction tools & guidelines
– Targeting ascertainment of the event, risk
factors, evaluation & treatment, and related
outcomes
outco
es
 Training occurred via webinars
UHC Methods
 National Steering Committee for each indicator







Similarly designed instruments
Sampling included PSI positive & negative cases (1:1)
UHC ran the sampling
p g software
Each site entered data into a webweb-based database
Time--frames differed
Time
Academic Medical Centers
Organizational Survey
Medical record sample
AHRQ
Hospitals Sample
Accidental puncture and laceration
43
249
Iatrogenic pneumothorax
38
205
Postoperative PE/DVT
37
155
Selected Infection due to Medical Care
37
194
Postoperative Sepsis
33
164
Overall
47
967
Postoperative PE/DVT
Decubitus ulcer
34
32
1022
6312
Postoperative Respiratory Failure
18
609
UHC
Selected Infection due to Medical
Care (SIMC
(SIMC))

At time off study*,
* targeted infections
f
& inflammatory
f
reactions due to vascular devices, implants, & grafts
(996.62) & infection following an infusion, injection,
transfusion, or vaccination (999.3)

P iti predictive
Positive
di ti value
l 61% (95% CI
CI; 51
51-70%)
70%)

Of the 39% false p
positive cases,, 7% had an exclusionaryy
diagnosis, 20% had an infection that was POA, and 12%
had no clear documentation of a qualifying infection.
*Indicator changed in 2009 to reflect to reflect new ICD-9-CM
code 999.31 for CVC & renamed “Central Line Related
Blood Stream Infection”
Selected Infection due to Medical
Care (SIMC
(SIMC))
Number
Time to infection
All CVCs
77
27.0 ± 106
Non--tunneled CVCs
Non
64
10.7 ± 7.1
 Femoral
7
5.7 + 3.4
 Subclavian
16
11.7 + 8.4
 Internal jugular
22
10 0 + 6.1
10.0
61
 PICC
19
12.5 + 7.7
Missing values included: Line type (24/118), central line
catheter type (12/77), CL insertion site (12/77), and dwell
ti (due
time
(d to
t lack
l k off insertion
i
ti or discontinued
di
ti d dates)
d t ).
SIMC Opportunities
 Better documentation of catheter type, insertion site,
insertion and removal dates, and catheter need
 Improved site selection based on national guidelines
 Increased recognition and/or documentation of signs
and
d symptoms
t
off suspected
t d and
d confirmed
fi
d infection,
i f ti
and related interventions
–
–
–
No documentation of systemic symptoms (n=73)
Unable to determine how diagnosis was made (n=29)
Comments regarding
g
g negative
g
blood cultures
 Inexpensive tracking of CLABSI beyond the ICU
Postoperative DVT or PE
 Targets deep vein thrombosis and/or pulmonary
embolus that occur after surgery.
 PPV = 83% (95% CI: 73-95%) AHRQ
– False positives
 10% POA
 7% had no event
 PPV = 72% (95% CI: 76-99%) UHC
– False positives
 12% POA
 16% had no event
Postoperative DVT or PE

58.9% of cases lacked ACCP recommended prophylaxis
– Majority had omission of prophylaxis
– Potentially 2/3rds of VTE cases in high risk pts. & 1/6 of all cases
could have been prevented

Using median percentages, there were delays in:
– Early
y recognition
g
of DVT (20.5%)
(
) and PE (16.7%)
(
)
– Early intervention of DVT (20.0%) and PE (16.7%)


Many false positive were associated with PICCs
C
Compared
to nonnon-cases, new DVT/PE
DVT/PE
/
were discharged home
on selfself-care at nearly half the rate, were twice as likely to be
g to rehabilitation or skilled nursing,
g and were nearly
y
discharged
four times as more likely to die.
Postop DVT
DVT/PE
/PE Opportunities
 Improve processes of care that lead to improved
adherence to ACCP prophylactic guidelines
– Ex. from higher performing hospitals included prepre-printed order
sets, physician feedback, and monitors for adherence.
 Look at processes of care associated with earlier
recognition and treatment.
 Although not the target of this indicator, there are
opportunities to evaluate PICC practice
Postoperative Sepsis



Targets those who have an elective surgical
procedure and then develop sepsis
Positive Predictive Value = 41% (95% CI; 2828-54%)
False positives (59%)
– No infection (28%)
– Infection (14%) or sepsis (3%) POA
 Majority were skin infections POA that required surgery or
infectious complications from a previous hospitalization
– Non
Non--elective surgery (25%) (majority cardiac)
Postoperative Sepsis

All but 3 cases received prepre-operative AB within 1=hr

Razors were used 25% of the time in hair removal

Avg. temp was lower in the PACU (96.7o F) than OR ( 97.4o F)

17.6% were inconsistent in the temperature
p
scale used

11.8% of cases had no OR temperature recorded

Causation was most often attributed to infections of the lung
(54%) and blood stream (46%) (not mutually exclusive)

Greater than 92% of cases had some degree
g
of organ
g
dysfunction suggesting severe sepsis
Sepsis opportunities
 Timely administration of preoperative antibiotics
 Alternatives to razors in hair removal
 Improved periperi-operative temperature management
 Standardize measurement scales in all clinical areas
 Potential opportunity for earlier recognition and
t t
treatment
t
 Evaluate nursing opportunities in preventing causative
i f ti
infections
Postoperative Respiratory
Failure (PRF)
 Captures PRF as a secondary diagnosis in elective
surgery patients
 Excludes patients with major respiratory or circulatory
disorders
 PPV = 90.5% (95% C.I. 86.5
86.5--94.4%)
 False
F l positive
iti
– Non
Non--elective hospitalization (0.5%)
– Exclusionary diagnosis (4
(4.9%)
9%)
– Acute respiratory failure POA (0.8%)
– Insufficient evidence for acute respiratory
p
y failure
diagnosis (3.3%)
Postoperative Respiratory
Failure (PRF)
 Majority of cases had an abdominal surgery (54%)
and/or a surgery > 4hrs in length (51%)
 80.6% ASA Physical Status Classification of III/IV
 95.1%
95 1% severity of illness score of extreme (75%) or more
(20.1%)
 23.3% died
 22% tracheostomy
y at discharge
g
 49% had at least one other PSI
Postoperative Respiratory
Failure (PRF)
 34.4% had no documentation of a prepre-op oxygenation






assessment
94 4% of cases had general anesthesia
94.4%
40.3% with neuromuscular agents had reversal
22.5%
22 5% were re
re--intubated on the same day as surgery
80.2% developed PRF within the first week
31% had no form of breathing exercises post
post--intubation
38.7% did not have documentation of a postpost-op PaCO2
<60
60 mmHg or a PaCO2 >45
45 mmHg.
Postoperative Respiratory
Failure (PRF)
 Improve prepre-operative and postpost-operative
assessments (and/or documentation of)
 Review nursing interventions such as breathing
exercises related to maintaining pulmonary hygiene
and
d preventing
ti PRF
 Review processes of care related to neuromuscular
reversal agents including communication of such
between levels of care.
Decubitus Ulcer (DU)
 Targets hospital acquired DU
 Excludes those with a LOS < 4 days, admitted from
nursing home or other acute care facilities, and/or a
major skin disorder or paralysis.
 PPV = 33.1%.
33 1%
 False positives
– DU POA
O 53.3%
%
– No event 14.3%
Decubitus Ulcer (DU)
 96.5% skin assessment on admission
 80.9% DU risk assessment on admission
 80.9% had of skin assessment in ED
 87.2% had skin assessment in holding
g area
 25% of cases, did not have a skin assessment in the
y prior to the DU diagnosis
g
two days
 Only 9.9% DU cases had the location, dimensions,
exudate, and site documented at the time of diagnosis
Decubitus Ulcer (DU)
 Staging was limited to 64.3% of cases
 Only 13.3% of cases were stage 1 when first




diagnosed
68.9% of DU were not healed at discharged
7 5% had complete documentation of the DU at
7.5%
discharge
49.1% of at risk patients were repositioned every 22-hr
in the 2 days prior to diagnosis
19.6% were readmitted within 3030-days. 15.5% entered
hospice,
h
i
and
d 20
20.3%
3% di
died
d
DU Opportunities
 Improve frequently of skin and DU risk assessment to
include:
 Improve the quality of DU assessments to include:
–
Location, wound edges, staging, state of peri-wound skin,
pain and adherence to prevention and treatment.
 Include Ht & Wt as part of the nursing database
 Look at processes of care aimed at earlier prevention
and improved treatment
 Turn & reposition at risk patients at least every 2-hrs
Accidental puncture or laceration (APL) &
I t
Iatrogenic
i Pneumothorax
P
th
(IP)
 These indicators were not very specific to nursing
beyond what we have previously reported
–
–
–
Potential of.
of increased surveillance,
surveillance
Use of radiologic adjuncts in line placement
Opportunities for specific case reviews (such as barotrauma
associated
i t d with
ith ventilated
til t d patients)
ti t )
 Refer to published papers
papers.
Discussion
 For most indicators, except iatrogenic pneumothorax
and accidental puncture and laceration, there were
potential
t ti l opportunities
t iti for
f improving
i
i care based
b
d on
the nursing process.
– Assessment
– Nursing diagnosis
– Care planning
– Implementation of care
– Evaluation
Recognizing limitations
 Data elements available via chart review
 Time constraints (burden on collaborators)
 Inter
Inter--hospital variation
 Volunteer sample
Implications

Opportunities to improve adherence to national
guidelines

Several PSIs studied appear to be influenced by
nursing
g

PSIs may provide an additional source of inexpensive
and readily available information for evaluating the
quality of nursing care

Additional research is needed
Acknowledgments
 Funded by AHRQ
AHRQ,, U.S. Department of Health and





Human Services (Contract No. 290290-04
04--0004).
AHRQ project
j
team
– Mamatha Pancholi & John Bott
UHC team
t
– Joanne Cuny & Pradeem Sama
Project managers/PIs
– Patrick Romano MD, MPH & Jeff Geppert JD, MS
Battelle training and support team
All of the validation pilot partners
References
Utter GH, et al. Positive predictive value of the AHRQ Accidental Puncture or Laceration
Patient Safety Indicator. Ann Surg 2009; 250(6):1041-5.
Sadeghi B,
B et al.
al Cases of iatrogenic pneumothorax can be identified from ICD-9-CM
ICD 9 CM coded
data. Am J Med Qual 2010; 25(3);211-7.
White RH,
RH et al
al. How valid is the ICD
ICD-9-CM
9 CM based AHRQ Patient Safety Indicator for
postoperative venous thromboembolism? Med Care 2009; 47(12):1237-43.
White RH,
RH et al.
al Evaluation of the predictive value of ICD-9-CM
ICD 9 CM coded adminstrative data
for venous thromboembolism in the United States. Thromb Res 2010; Epub ahead of print.
Zrelak PA,
PA et al
al. Positive predictive value of the AHRQ Patient Safety Indicator for Central
Line Associated-Bloodstream Infection. J Healthcare Qual; in press.
Utter GH,
GH et al.
al Detection of Postoperative Respiratory Failure: How predictive Is the AHRQ
Patient Safety Indicator? JACS; in press.