Diagnostics
Azhar Ali, MD
Frank Federico, RPh
Description
Delivering safe care involves understanding the root of the
defects that impact safe. In order for us to understand our
systems of care, we need a diagnostic journey that moves out of
a model for judgment and into a model for learning. A number of
data sources are available for organizations to identify the rate
and type of harm that patients may be experiencing, Using the
data from diagnostics can be helpful to draw conclusions to help
focus improvement efforts. During this session, participants will
learn and share their experiences with using diagnostics to
improving safety.
Objectives
Describe the diagnostic journey that organizations must take in
order to improve patient safety
List the diagnostic tools available to identify defects
Develop a plan to use diagnostics in your organization
Why Are You Here?
What interest brought you to a data session?
What data do you use every day to improve care?
Juran Trilogy
QUALITY
PLANNING
QUALITY
IMPROVEMENT
QUALITY
CONTROL
Data Uses
Data is used for several purposes in healthcare
– Research
– Accountability
– Improvement
What data uses do you see in daily work?
The Three Faces of Performance Measurement
Aspect
Aim
Improvement
Accountability
Research
Improvement of care
(efficiency & effectiveness)
Comparison, choice,
reassurance, motivation for
change
New knowledge
(efficacy)
Methods:
• Test Observability
• Bias
Test observable
No test, evaluate current
performance
Test blinded or controlled
Accept consistent bias
Measure and adjust to
reduce bias
Design to eliminate bias
• Sample Size
“Just enough” data, small
sequential samples
Obtain 100% of available,
relevant data
“Just in case” data
• Flexibility of
Hypothesis
Flexible hypotheses, changes
as learning takes place
No hypothesis
Fixed hypothesis
(null hypothesis)
• Testing Strategy
Sequential tests
No tests
One large test
• Determining if a
change is an
improvement
Run & Control charts
Analytic Statistics
(statistical process control)
No change focus
(maybe compute a percent
change or rank order the
results)
Enumerative Statistics
(t-test, F-test,
chi square,
p-values)
• Confidentiality of
the data
Data used only by those
involved with improvement
Data available for public
consumption and review
Research subjects’ identities
protected
Solberg, L I; Mosser, G; McDonald, S "The three faces of performance measurement: improvement,
accountability, and research." The Joint Commission journal on quality improvement 23, No. 3 1997, pp. 135-47.
Vulnerable System Syndrome
Three core pathologies
- Blame
- Denial
- And the pursuit of (the wrong
kind of) excellence
How can we learn about
our system performance?
Sources of Information
Mortality Review
Trigger tools
Concurrent review
Incident Reports
Waste Report
Observation
Pharmacy Interactions
Patient Complaints
KPI and Reliability of processes
Culture of safety assessment
Access
Diagnostic Journey
Do people die unnecessarily every day in our
hospitals?
In order for us to understand this, we need a
diagnostic journey that moves out of a model for
judgment and into a model for learning.
The Mortality Diagnostic – 2x2 Matrix
Review most recent 50 consecutive deaths.
Place them into a two by two matrix based on:
- Was the patient admitted for palliative care?
- Was the patient admitted to the ICU?
Focus your work initially on boxes that have at least 20% of
your mortality.
Diagnostic – The 2 x 2 Matrix
Admitted to the ICU?
Yes
Admitted
for
Palliative
Care
Only?
No
Yes
Box #1
Box #2
No
Box #3
Box #4
The Mortality Diagnostic
- Failure to Recognize, Plan, Communicate
Analyze deaths in box 3 and 4 for evidence of failure to:
recognize, communicate, plan.
This will help you understand the local environment.
Recognize, Communicate, Plan
- Failure
to Recognize: Any situation in which a patient has
died and there was evidence that an intervention could have
been made anytime prior to the patient’s death Example: the
staff was worried, change in heart rate, change in respiratory
rate, change in blood pressure, change in O2 saturation or
change in consciousness or neurological status that was not
responded to.
- Failure to Plan, such as: diagnosis, treatment, or calling a
rescue team.
- Failure to Communicate: Patient to staff, clinician to
clinician, inadequate documentation, inadequate supervisor,
leadership (no quarterback for the team), etc.
The Mortality Diagnostic
- The Impact of Care
Evaluate ALL deaths in box 3 and box 4 to
assess the estimated impact of our care on
mortality:
*As you review the deaths in box 3 & 4, ask
yourself the questions honestly (focusing on
learning, not judgment):
–
–
Was perfect care rendered?
If perfect care wasn’t rendered, could the outcome
of death have been prevented if the care had been
better?
–
What number of deaths could have been prevented?
The Mortality Diagnostic
- Evidence of Adverse Events
Analyze deaths in box 3 and 4 for evidence of adverse events
using the Global Trigger Tool.
This will give some further direction to local problems.
Source; Helen Lau, R.N., M.H.R.O.D., Kerry C. Litman, M.D. “Saving Lives by Studying
Deaths: Using Standardized Mortality Reviews to Improve Inpatient Safety” The Joint
Commission Journal on Quality and Patient Safety.
September 2011 Volume 37 Number 9
Accepting the Harm Burden
Concept of moving from a focus on error
and the preventable to the measurement
of global institutional harm whether
preventable or not
Definition of Harm
In the IHI Global Trigger Tool, the definition used
for harm is as follows:
Unintended physical injury resulting from or
contributed to by medical care that requires
additional monitoring, treatment or hospitalization,
or that results in death.
New (Harm)
vs. Old (Errors)
Concentrates less on
people more on
systems
Looks at all unintended
results
Makes measurement
easier
Concentrates on harm
and those errors that
cause harm
Errors are the focus of
discussion and solutions
Tends to focus only on
those results felt to be
related to error, ignores
other events
Requires judgment
Human found
responsible for most of
the errors
Why Use Trigger Tools?
Traditional reporting of errors, incidents, or
events does not reliably occur in the best of
health care cultures
Voluntary methods markedly underestimate
adverse events
Events can be reliably detected without resorting
to as yet unproven electronic surveillance
methods
Can be integrated into a good sampling
methodology to follow event rates over time
IHI Global Trigger Tool
Review chart for triggers that are sensitive and specific for harm
Find a trigger- was there harm?
Not all triggers mean there was harm!
IHI Global Trigger Tool Modules
Cares (General)
Critical Care
Medication
Surgery
L&D
ED
Cares Module Triggers
C1
Transfusion or use of blood products
C2
Any Code or arrest
C3
Dialysis
C4
Positive blood culture
C5
X-Ray or Doppler studies for emboli
C6
Abrupt drop of greater than 25% in Hg or Hemtocrit
C7
Patient fall
C8
Decubiti
C9
Readmission within 30 days
C10
Restraint use
C11
Infection of any kind
C12
In hospital Stroke
C13
Transfer to higher level of care
C14
Any procedure complication
C15
Other
How it is Actually Done
1 - Set your timer for 20 minutes
2 - Review the coding summary (look for ecodes and obvious events)
3 - Review the discharge summary
4 - Review the lab
5 - Review the x-ray reports
6 - Review the procedure notes
7 - Any time left over, review nurse notes
Example of a Trigger:
Transfer to higher level of care
Endoscopy
Post procedure somnolent and hypotensive (BP 80) transferred to
ICU
Placed on Bi-Pap
Received standard meperidine and midazolam for procedure
Given flumazenil; stayed in unit 12 hours.
Concurrent Review
• Definition of Concurrent Review:
- Real-time view of patient care related to the specific
quality indicator being measured.
• Goal:
- Improve quality of care during present patient
admission.
• Reviewer Qualifications:
- Adequate (clinical) knowledge/experience of subject
matter and ability to synthesize and provide
feedback.
Concurrent Review Process
Identify patients with a need for daily review
– This can be the most challenging piece
– Use IT/administrative systems when possible
Review specifics of chart
Analyze and synthesize information
Provide feedback
(with the potential for an intervention …)
– One-on-one dialogue
– Weekly Reports/feedback from leadership
– Stats
– Outliers
– Review of guideline in question
– Documentation issues
– Staff Kudos!
Pneumonia Performance:
ED Measures
90
80
1 missed case
per measure.
Cases not
picked up on
concurrent
review
Concurrent
review begins
70
BC Draw n Prior to Initial ABX Started (ED)
ABX Tim ing (6 hours)
Tim eline
Q1 2010
Q 4 2009
Q3 2009
Q2 2009
Q1 2009
Q4 2008
Q3 2008
Q2 2008
Q1 2008
60
Baseline
Performance (%)
100
Pneumonia Performance:
Vaccine Measures
Performance (%)
10 0
90
80
Concurrent
review begins
Q3 2008
70
Pneumovax
Flu Vaccine
60
B aseline
2007
2008
Tim eline
2009
2 0 10
Incident/Voluntary Reports
Effectiveness
Findings from IHI GTT studies
34
How Much Harm
‘Global Trigger Tool’ Shows That
Events in Hospitals May Be Ten
Times Greater Than Previously
Measured
Classen DC, Resar R, Griffin F, et al. Global Trigger Tool shows that adverse
events in hospitals may be ten times greater than previously measured. Health
Affairs. 2011 Apr;30(4):581-589
Waste Identification Tool
Ward Module
Bed Occupied or Used Inappropriately
Healthcare-Associated Infection
Adverse Drug Event
Procedure Complication
Unnecessary Hospitalization
Flow Delay
Clinical Care Delay
Patient Care Module
Monitoring
Invasive Devices
Medications
Tests
Therapies
Diagnosis Module
Urinalysis
Thyroid function studies
Electrocardiogram (ECG)
Chest x-ray (CXR)
Metabolic panel (typically includes glucose, electrolytes, proteins,
kidney function tests, and liver enzymes)
Treatment Module
Anticoagulation
Glucose management
Post-operative care for high-volume procedures
Pain control
Others
Patient Module
The Patient Module is meant to function as a reality check
regarding what patient perceive as helpful and valuable in their
inpatient care.
Qualitative Measure
Patient Module
Identify five adult patients scheduled for discharge to
the home setting and who are capable of
participating in a brief interview.
First explain the purpose of the interview and obtain
permission.
Be sure to inform patients as to what information will
be noted and how it will be used.
Patient Module
Only interview patients who are willing to participate.
Record some brief notes with the patient’s comments and
perspectives in the worksheet.
Analysis
The modules and various waste streams described in this Tool
represent the starting point in a journey to reduce significant
waste in an acute care hospital.
Observation Method
Direct observation of a process in the natural setting
Developed by Ken Barker at Auburn University
Observation Method
The method is easily understood,
Data are easy to use for identifying trends and benchmarking,
Data are available within hours
The method is systems oriented and views errors in doses as
defects in a system,
The method is objective and does not assign blame
Observation Method
The method is defensible, with all doses being examined and
errors witnessed
The method enables problem-based continuing education that
focuses on “our” errors
The method facilitates evidence-based testing that can evaluate
proposed system changes
Quality can be measured quantitatively by third parties
Pharmacy Interventions
Source of information for medication errors and medicationrelated events
Source of valuable information
Not used to full potential
Patient Complaints
http://www.ncbi.nlm.nih.gov/books/NBK43703/
Culture of Safety
Access and Flow
Reliability is failure free
operation over time.
Failures: readmissions within 31 days
related dx
25
20
15
10
5
0
1
3
5
7
9 11 13 15 17 19 21 23 25 27 29 31
31 Day Readmission Analysis
100 random charts reviewed (total of 244 readmissions within 31
days for the year).
Charts reviewed by physicians with a standard chart review
worksheet.
Worksheets reviewed and data for production defects,
environmental defects extracted.
Production Defects
40%
37.50%
35%
30%
25%
22.50%
20%
15%
15%
12.50%
10%
10%
5%
0%
2.50%
Poor Discharge
Surgical
Complicatons
Poor Hospital care
Procedure/Rx not
successful
Infection
Other
Environmental Defects
60%
59%
50%
40%
30%
20%
15%
11%
10%
3.70%
0%
Poor Outpatient
Management
Lack of Support
LTC Facility
Problems
Non Compliant
Patient
3.70%
Unable to get meds
3.70%
Unable to get
appointment
Poor Outpatient Management
Poor outpatient pain control program (31%)
Poor CHF outpatient follow up program (31%)
Multiple other issues (37.5%)
Defects arise from access to care, medication, self care
Primary strategies
care
Reliability: failure free operation
over time for a patient
CHF
ED
Direct admit
transfer
Defects that
arise over the
LOS: variation
from best care,
Med-surg.
unit
Defects that arise from factors that affect care over
time: Nutrition, environment, medication availability,
poor discharge planning
, home
Home/rehab/nursing
High reliability organizations are
continually on the lookout for novel
types of system failure and have
several contingency plans.
Sensemaking
How does one make
sense of all of the data?
Meyer GS, Nelson EC, Pryor DB, James B, Swensen SJ,
Kaplan GS, et al. More quality measures versus measuring
what matters: a call for balance and parsimony.
BMJ Qual Saf. 2012 Aug 14.
A Structure of Measures & Improvement
Whole System
Tier 1
Macro
“MesoSystem”
“MesoSystem”
“MesoSystem”
Tier 2
Meso
Project
Project
Project
Project
Project
Project
Tier 3
Micro
Project
Project
A Structure of Measures & Improvement
Whole System
Tier 1
Macro
“MesoSystem”
“MesoSystem”
“MesoSystem”
Tier 2
Meso
Project
Project
Project
Project
Project
Project
Tier 3
Micro
Project
Project
Impact of Different Measures
Macro System
• Leadership
• Support for mission and goals
• Sets the agenda
• Structures that assure process and outcomes in place
• E.g. Skills and time for daily improvement
Macro System Measures
IOM Dimension
Whole System Measures
Safe
• Adverse events that cause harm
• Work days lost
Effective
• Hospital standardized mortality ratio
• Unadjusted (raw) mortality
• Functional outcomes
• Readmission percentage
Patient-Centered • Patient experience
Timely
• 3rd next appointment available; waits and delays
Efficient
• Patient days during the last 6 months of life
• Costs per capita
Equitable – for all
Impact of Different Measures
Meso System
• Outcomes important to guide actions
• Process measures guide activities to support excellent, safe care
• Managers use data to set agenda to support larger organizational
goals
• Daily improvement
Meso System Measures
Aims
Measures
Safe
• Falls this month
• Fall rate
• Employee injury rate
Effective
• Sepsis protocol implementation %
• DVT bundle
• Transitions home steps implemented
Patient-Centered • Bedside change of shift report
• Physician communication steps implemented
Timely
• 3rd next appointment available
Efficient
• Flow delays (E.g. ED to inpatient; OR room
turnover)
Equitable – for all
Impact of Different Measures
Microsystem
• Process measures help focus on daily activities
• What can we learn from this case – this one?
• This fall; this missed lab; this wrong dose
• Drives planning of work and resources
• Outcomes important for focus
• For day-to-day activities, focus is ‘what can I do to change all the things
that need to improve?!
Micro System Measures
Aims
Measures
Safe
• Falls this month
• Fall rate
• Employee injury rate
Effective
• Sepsis protocol implementation %
• DVT bundle
• Transitions home steps implemented
Patient-Centered • Bedside change of shift report
• Physician communication steps implemented
Timely
• Patient rooming process in clinic
Efficient
• Flow delays (E.g. ED to inpatient; OR room
turnover)
Equitable – for all
The Differences Are…
Where you are in the organization
Primary role
Scope of influence
Display of Data
One Example of Data Use
Safe Care – Measures
Measures of –
• Misuse
• DVT care not fully completed
• Over-use
• Prolonged sedation in the ICU
• Under-use
• Access to care; waits and delays due to flow problems
http://www.psnet.ahrq.gov/popup_glossary.aspx?name=underuseoverusemisuse
Types of Measures
Process
– Measure the effectiveness of a process, which if done well should deliver
a good outcome
Outcome
– Measure the effectiveness of a change or intervention in delivering the
desired outcome
Balancing Measure
Why is it important to link multiple measures?
No one measure provides the complete picture
Activities in one area impact results in an other
Process measures should be linked through evidence to
outcomes
Balancing measures help understand unintended consequences
Focus on the Vital Few!
There are many things in life that are interesting
to know.
It important to work on those things that are
essential to quality and safety.
The challenge, therefore, is to be disciplined
enough to focus on the essential, vital few.
Building a
cascading
system of
measurement
83
Resources
White papers
– Mortality
http://www.ihi.org/knowledge/Pages/IHIWhitePapers/MoveYourDot
MeasuringEvaluatingandReducingHospitalMortalityRates.aspx
http://www.ihi.org/knowledge/Pages/IHIWhitePapers/ReducingHos
pitalMortalityRatesPart2.aspx
– Global Trigger Tool
http://www.ihi.org/knowledge/Pages/IHIWhitePapers/IHIGlobalTrigg
erToolWhitePaper.aspx