Injecting Healthcare
with Human Factors
CRISS investigators design and evaluate
medical devices and health information
technology. We have collaborated with
the VA, other Vanderbilt centers and
outside vendors to develop and
improve the user experience.
Teaching & Training
Faculty and staff provide guidance in
theories, methods and tools related to
human factors through simulation-based
training and assessment.
Communication &
Decision Making
We investigate team communication,
coordination, adaptive problem solving,
culture and effectiveness, and individual
and group performance-shaping factors,
to generate improved clinical care
processes and outcomes.
Work Analysis &
Improvement
Using human factors engineering,
cognitive psychology, biomedical
engineering and implementation science,
CRISS studies performance during patient
care to understand how and why care
deviates from optimal.
Core
Faculty
& Staff
Matt Weinger, MD
CRISS Director
HIT & Technology
Usability, Event
Analysis
matt.weinger@vanderbilt.edu
Current Application
• Unstructured
layout and data
entry process
• Difficult manual
entry of labs
• Out-of-workflow
confirmation
• No clear patient
status indicators
Sample pre-op workspace screen.
(in development)
Overview Displays for Hybrid Nuclear Power Plants
We helped design a preincision,
forced-function electronic time-out
checklist that is displayed on a large
monitor in the OR and mediated
through existing intra-operative
nursing electronic documentation.
After use during 243,939 sequential operations at VUMC there were
zero wrong surgeries, significantly fewer than would be expected from
observed national rates, which would have been between 10 and 73.
As part of a Department of Energy Nuclear Energy University Program
(NEUP), we are assisting the Idaho National Laboratory in the design
and evaluation of overview displays in nuclear power plant (NPP)
control rooms as part of a modernization program for the nation’s NPP
fleet. Current control rooms are largely analog requiring operators to
assimilate myriad discrete data (Figure A); they do not support digital
decision support and visualization. Unfortunately, proposed large
screen overview displays (Figure B) are only economically feasible in
new NPPs. Thus, we are studying alternative strategies for overview
displays for existing largely analog control rooms.
Simulation-Based Performance Assessment
Example of complex neonatal
intensive care unit setting.
Video rating software.
Research Assistant
Professor
(Anesthesiology)
HIT UI Design &
Evaluation
Arna Banerjee, MD
Methods
This study evaluates the effects of the multidisciplinary participatory design and low-intensity
implementation of a structured NICU-to-OR handover
tool and process on the quality of handovers and the
frequency of transition-of-care related events.
To evaluate the intervention, trained observers, using
a structured assessment tool, will observe and score
actual NICU-to-OR handovers before, during and after
interventions and collect data on non-routine events.
Associate Professor of
Anesthesiology,
Assistant Professor
of Surgery
Education & Training
arna.banerjee@vanderbilt.edu
Research Associate
Professor
(Anesthesiology
and Medicine)
Healthcare System
Modeling
dan.france@vanderbilt.edu
Effect of Decision Support Tools on Team Performance in Simulated Crises
Amanda Lorinc, MD
Assistant Professor
(Pediatric
Anesthesiology)
Patient Safety,
Quality Improvement
amanda.lorinc@vanderbilt.edu
Clinicians routinely deviate from
CRISS collaborates closely with VanderThe goal of this project is to determine
published guidelines and care teams if an electronic decision support tool
bilt’s Center for Experiential Learning and
often delay initiating resuscitative
Assessment (CELA), a multipurpose, high(DST) can improve clinician performance
measures during cardiac arrest
fidelity simulation facility with 12 fully
by improving the clinical team’s NTS. In
despite the knowledge that
equipped clinical exam rooms, a 4-bed
this study, the effect on clinical teams of
adherence to consensus guidelines
ICU or ED, and an OR suite. It’s equipped
different versions of an electronic DST
improves patient survival and
with two complete control rooms, man(a version that emphasizes only TS, a
outcomes. It is increasingly evident
nequin-based simulation, and the latest
version that emphasizes only NTS and
MatthewinB.virtual
Weinger,
MD*;
Jason M.
Slagle,
PhD; Amanda Lorinc, MD; Gina Whitney, MD; Eric Porterfield,
MS;
that these
failures are the result of
reality
simulators
and
partial
a version that emphasizes both TS and
Krys Dworski,
Eva Cassidy,
and
thean
PNRE
Project Team
by the
Patient-Centered
Institute)in both
DEPARTMENT
ANESTHESIOLOGY
CELA(Funded
control room’s
view
out to test bays. Outcomes Research
deficiencies
technicalOF
skills
task trainers.
CELA aff
ords
optimal
NTS) will be evaluated during simulated
(TS) and non-technical skills (NTS).
environment for research and teaching.
emergencies.
Introduction
Figures & Tables
Results
•  A Ò non-routine eventÓ (NRE) is defined as any event that is perceived
by clinicians, patients and/or skilled observers to deviate from ideal
care for that specific patient in that specific clinical situation (Figure 1).1
Barriers toIn
Effprevious
ective Handovers
studies, we have shown that NREs:
DST screen listing technical skills
during pulseless arrest management.
•  In the 140 patients studied (age 58±14 yrs, 54% male), there were a total of 160
clinician-reported NREs and 87 patient-reported NREs. There were 214 unique NREs.
Figure 1. NRE Schematic
(41%) contained patient-reported NREs. (See examples, Figure 3)
experimental
paradigm
has
determined nearin the Intensive Care Unit (ICU) sound
Findings
With a positive predictive value
of ~27%,
false
•  Both
the patient
and at least Our
one clinician
reported an NRE
in 39 cases
(28%),
•  can be reliably collected prospectively Alarms
from physicians
80
2,3
althoughclinicians
they were rarely about
the same event.
and
nurses
in
various
care
settings
threshold
auditory perception of alarms and the signalfrequently and 85-99% of cases do not require
positive alarms are frequent, leading
Preliminary
60
•  CRNAÕ
s were most likely to report
an NREratio
(69% which
of cases),preserves
OR nurses and
surgeons performance.
•  are frequent (from 15% to 50%)
to-noise
clinician
clinical
intervention.
As
alarm
frequency
increases,
to
become
desensitized
to
alarms.
Yet,
findings identify
reported NREs in 53% and 55%, respectively, while anesthesia residents reported
40
•  are often associated with patient impact or injury 4
Results show near-threshold auditory perception of
clinicians develop ‘alarm fatigue’ resulting in
failure to respond to the infrequent
clinically
NREs least
often (34%).
common barriers
•  provide data about the nature and severity of process deficiencies
20
•  Morepatient
than half (54%) of all NREs
involved
care tasks/processes,
23% involved
alarms
is around
-27 decibels
(dB) from background
desensitization, missed alarms, and delayed
significant alarm may lead to poor
to effective
that could cause future patient injury
cliniciansÕ actions/inactions and 13% involved technology.
0
Figure for
2. P-CONES
Probes for
Ambulatory
Surgery
noise at 60 dB. Additionally, with visual offset of a patient
responses.
This
is
dangerous
the
patient
when
an
outcomes.
Fatigue
from
alarm
amplitude
and
Distractions Lack of
Many
Inadequate
Unclearabout how patients and their families/
handovers (see
•  SBAR
ThusToo
far,
we know
very little
•  The themes of the NREs are shown in Table 1.
Structure
Cooks
Preparation Expectations
•  Did anything happen that you did
•  Did you understand
•  Did anything happen to you
caregivers view Ô non-routineÕ events in perioperative
care.
monitor, there is preserved performance measured by an
alarm-provoking
event requires
intervention
nonspecifi
c alarms
not expect clinical
or want to happen?
what was supposed
that was
frustrating, from uniform uninformative
graph) and items
Reducing distractions is critical.
inconvenient, or distracting?
•  DidSince
you get the
information
you andto happen to you?
Inverse
Effi
ciency
Score
(IES
=
Response
Time/Accuracy)
but
is
inadvertently
missed.
false
alarm
alarms
is
the
post-monitor
problem
that
missed during
Conclusions
needed to know at the right time?
•  Was everything you
•  Did the people taking care of
Study Aims
needed made can be addressed
you know what youby
needed?
at -11
dB as compared
withreported
+4dBNREs
with worsening at
clinically insignificant alarm
80-99%,
•  Didrates
anything reach
happen that
caused
understanding
the
handovers. Through participatory design sessions, we have
•
In
this
preliminary
study,
40%
of
ambulatory
surgery
patients
available to you?
you stress, worry, or concern?
•  Were you worse off afterwards
•  Determine what aspects of their clinical encounters do patients and
and these
included
clinicalmore
care deviations,
communication
failures,
and performance is mainnegative
SNRs.
Thus,
clinician
practitioners
distrust
alarms,
lose
confi
dence
in
their
psychoacoustic
properties
of
alarms
and
the
than you expected
to be?
developed a tool that we hope will improve
handover
families view as Ò non-routineÓ and that reflect low care quality or
service deficiencies.
tained with alarms that are softer than background noise.
signifi
cance,
and
manifest
alarm
fatigue.
aural
perception
of
clinicians.
safety
issues.
quality and decrease significant non-routine events.
Figure 3. Sample Patient NREs
•  NRE collection from perioperative patients is feasible and appears a
100%
•  Elucidate the factors that influence the reporting of perioperative
NREs and affect the nature of the NREs that are reported.
•  I got lost in the hospital trying
to findÉ
•  Determine whether NREs obtained from patients/families add to
evidence about clinical system failure modes beyond that obtained
from clinicians caring for the same patients.
•  The lights and/or noise
disrupted my sleep.
Patient-Reported Non-Routine Events (NREs)
In a series of studies, we have
shown that perioperative
clinician-reported NREs:
v1) are frequent (≥1 NRE in
up to 40% of all care periods
studied); 2) capture a wide
cross-section of system
failures; and 3) are associated with increased clinician How everyday practice can deviate from best practice.
workload and with significant
patient physiological disturbances. NREs provide a window on system
safety and can be used as a dependent variable in safety interventions.
Dan France, PhD
Figure B: Digital control room.
Utilizing Multisensory Integration to Improve Psychoacoustic
Alarm Design in the ICU
•  Eighty-two surgical cases (59%) contained clinician-reported NREs while 57 cases
The Epidemiology of Perioperative Non-Routine Events
Clinical practice frequently varies from optimal
care, yet medical errors that do not cause
patient harm are often not reported. We
introduced to health care the concept of the
Non-Routine Event (NRE), modeled after safety
processes in the nuclear power industry where
every deviation from standard operating
procedures (SOPs) is reported and investigated.
In health care, an NRE is defined as “any aspect
of clinical care perceived by clinicians or
trained observers as deviating from optimal
care for that patient in that clinical situation.”
Based on detailed observations of operator performance
and interviews, overview display design guidelines
are being developed. We will also be doing studies of
putative overview displays in NPP control room simulators.
Numerous problems remain to be solved, including
optimal interface design to support dynamic conditions,
how best to support both routine work and rare
emergency events, integration with procedures and with
alarm systems, and practical issues like hardware display
size, number, location and control.
Patient-Reported Perioperative Non-Routine Events
Handover Tool Development, Implementation and Evaluation
Background
• Patient care transitions (i.e.
patient handovers), while vital
to patient safety, are variable
and error prone.
• Neonatal Intensive Care Unit
(NICU) patients are especially
vulnerable to medical errors
yet, to our knowledge, there
are no published studies on
ICU-to-OR transitions of care.
Figure A: Analog control room.
Center for Experiential Learning & Assessment
Participants successfully completed 75±13% of critical
performance elements. Both technical (5.0±2.1) and nontechnical (5.4±2.0) ratings were distributed across the full
1-9 scale. 32% of BCAs were rated as not performing at
the level of a consultant. Higher rated performances were
associated with academic (vs. community practice) and participant age < 50 years. If these findings reflect performance
during actual care, it calls into question the efficacy of existing systems of continuing education and training. Greater
use of simulation-based assessment and training as part of
physicians’ life-long learning may be warranted.
It is important for physicians to maintain their competence
throughout their career. We used standardized high-fidelity
simulation scenarios to assess the performance of practicing board-certified anesthesiologists (BCAs) during medical
emergencies. Consenting BCAs, who already participated
in simulation-based courses at one of 8 sites, performed
as the primary physician in standardized medical emergency scenarios. Video recorded performances were rated
by trained, independent, blinded, and experienced BCAs.
268 participants managed 294 simulated events. Intra-rater
and inter-rater reliability were 0.93 and 0.86, respectively.
Shilo Anders, PhD
shilo.anders@vanderbilt.edu
Redesigned UI
• Progressive disclosure of
modular queries
• Personalized access with
ability to audit prior entries
• Clear, easy confirmation
process
• Improved ability to scan
for pertinent positives
and alerts
Electronic Time-Out Reduces Wrong Surgeries
www.mc.vanderbilt.edu/criss
Percent of Handovers
Design & Usability
Pre-op UI Redesign
Highly interdisciplinary and collaborative, CRISS conducts basic and applied
research in healthcare informatics, patient safety and clinical quality, and designs
and evaluates health information technology, care processes and medical devices.
Laurie Lovett
Novak, PhD, MHSA
Assistant Professor
(Biomedical
Informatics)
Impact of HIT on
Work Patterns
laurie.l.novak@vanderbilt.edu
In a study funded Methods
by the Patient Centered Outcomes Research
Institute (PCORI),• CRISS
hasourcollected
NREs
from
patients,
familyOpenWe refined
previous NRE
collection
tool,
the Comprehensive
ended Non-routine Event Survey (CONES),2 for use with patients based
members and patients’
clinicians in four medical settings:
on a thematic analysis of patient/caregiver focus groups, input from our
ambulatory surgery,
interventional
cardiology,
pediatric
teamÕ
s patient representatives,
and pilot testing.
(Figureoncology,
2)
•  Preoperatively,
obtained written
consent and demographicNREs
data from
and pediatric cardiac
surgery.wePatientand clinician-reported
patients having elective ambulatory surgery (discharged within 23 hours).
were common.
•  Postoperatively, trained investigators collected NREs from the patients
and also from their
anesthesia
providers, surgeons
and perioperative
In interventional cardiology
cases,
69% contained
patient-reported
nurses.
NREs (the highest of our four settings) while 46% contained clinician•  Patients were surveyed with the Patient-specific CONES tool prior to
reported NREs. Patient
NREs
overlapped
with clinician
discharge
and rarely
then again
by phone approximately
one-week NREs.
later.
Across all settings, most patient NREs reflected deficient care
delivery processes – i.e., the care was not patient-centered.
Joseph
Schlesinger, MD
Assistant Professor
HIT, Alarm
Development,
Multisensory
Integration
joseph.j.schlessinger@vanderbilt.edu
Matt Shotwell,
PhD
Assistant Professor
(Biostatistics)
Statistical
Computing
and Inference
matt.shotwell@vanderbilt.edu
•  [Clinician] was condescending/
rude/didnÕ t listen.
•  I was Ò dumped on the
sidewalkÓ before I was
ready to go home.
•  An allergy band was
never placed on me.
•  I couldnÕ t breathe when I
woke up from surgery.
valuable source of quality and safety data.
•  Patient NREs rarely overlapped with clinician NREs.
•  My inpatient diabetes
management was Ò not very
good.Ó
Patient Chronic Illness Routines
•  Most patient NREs represented deficient care delivery processes Ð The care
provided was not patient-centered!
•  These methods show promise for understanding and evaluating patientcentered perioperative care processes.
Table 1. Key Themes from Focus Groups and their Occurrence in Cases and NREs*
% of Cases % of NREs
(n = 140)
(n = 214)
Key Non-Routine Event Theme
Diagnostic and Therapeutic Issues (unfamiliarity with the patientÕ s condition,
47.1
46.3
44.3
43.9
32.1
31.3
Environment of Care (available food choices, incorrect diet, cleanliness, etc.)
17.9
14.0
Staffing Issues (too few nurses, adequately trained providers unavailable, etc.)
18.6
13.1
15.0
10.7
5.6
8.4
mistakes and errors, diagnostic delays or misdiagnoses, etc.)
Health Care Process Deficiencies (unexpected care, failure to get access,
delays in treatment, care disruptions or variability, etc.)
Communication of Health Information (getting the wrong amount of
information, or wrong content, or mistimed delivery, etc.)
Patient-Provider Relationship (dismissal of patient concerns, not talking with
or listening to patients, being rude or inflexible, etc.)
No Relevant Focus Group Theme
* Both patient- and clinician-reported NREs included.
Key themes for patient-reported NREs in surgical setting.
Jason Slagle, PhD
Research Assistant
Professor
(Anesthesiology)
Task, Workload &
Event Analysis
jason.slagle@vanderbilt.edu
Scott Watkins, MD
Assistant Professor
(Pediatric Cardiac
Anesthesiology)
Cognitive Aids &
Decison Support
scott.watkins@vanderbilt.edu
Clinician performance at varying
signal-to-noise ratio
In four separate studies, we are examining routines used by patients
and their families to manage chronic illness in everyday life. In a
References
study &
ofAcknowledgements
cardiac patients, we explored the activities involved in self1. Weinger MB, Slagle J: Human factors research in anesthesia patient safety: Techniques to elucidate
care
routines
and
role of JAMIA
physical
artifacts
and other actors. In
factors affecting
clinical
task performance
andthe
decision-making.
2002; 9(6):
S58-63
2. Oken A, Rasmussen
MD, Slagle
JM, Jain S, Kuykendall
T, Ordonez N,who
Weingerlost
MB: A their
facilitatedhomes in a major
a
study
of
people
with
diabetes
survey instrument captures significantly more anesthesia events than does traditional voluntary
event reporting.
Anesthesiology
2007;
107(6),
909-22
flood
event,
we
examined
how routines are disrupted and the
3. Rayo M, Smith P, Weinger MB, Slagle JS: Assessing medication safety technology in the intensive
people
to re-establish routines. We examined asthma
care unit. Procstrategies
Hum Factors Ergon
Soc 2007;use
51:692-96
4. Slagle JM, Anders
S, Porterfield E, Arnold
A, Calderwood
C,
Weinger with
MB: Significant
physiological
management
routines
in
teens
asthma
and
their
parent
disturbances associated with non-routine event containing and routine anesthesia cases. J Patient
Safety (in press)
caregivers, raising questions about the role of teens’ emerging
This study was also
supported by Academic
Program
Support funds
from VUMC to
CRISS (Center
for
autonomy
on
the
stability
of
chronic
illness
routines.
Currently, we
Research and Innovation in Systems Safety).
are conducting a study of patients with diabetes to understand how
routines contribute to diabetes outcomes.
Russ Beebe Interaction Designer
Jayson Ingram Application Developer
Andrew Kline Program Coordinator
Carrie Reale, MSN, RN Informatics
Nurse Specialist
Christopher Simpson Research Manager
Jie Xu Research Fellow
Obtaining
Organizing/
Storing
Administration
Monitoring
Preventive/
Predictive
Shared elements among the many medication
managing routines produce structure, which
may increase resilience of the routines.
Sources of Research Support
Patient-Centered Outcomes Research Institute (PCORI)
Agency for Healthcare Research and Quality (AHRQ)
Technology (NIST)
National Institutes of Health (NIH)
National Institutes of Standards and
Anesthesia Patient Safety Foundation (APSF)
Veterans Affairs Health Services Research & Development
& Research (FAER)
Department of Energy (DOE)
Foundation for Anesthesia Education