Principal Investigator: Watkins, Scott
Version Date: Feb 15, 2012
Study Title: Evaluation of an Emergency Checklist for Intraoperative Pediatric Critical Events
Institution: Vanderbilt University Medical Center
Emergency Checklist Cards for Intraoperative
Pediatric Critical Events: Development and
Evaluation Using High Fidelity Simulation
Scott C. Watkins M.D.
Assistant Professor
Division of Pediatric Cardiac Anesthesia
Monroe Carell Jr. Children's Hospital at Vanderbilt
2200 Children's Way
Suite 3115
Nashville, Tn 37232
P: 615-936-6808
Shilo Anders, Ph.D.
Center for Research and Innovation in Systems Safety
Vanderbilt University Medical Center
1211 21st Avenue South
Medical Arts Building, Suite 732
Nashville, TN 37211-1212
P: 615-936-6598
Revision Date: December 1, 2003
Principal Investigator: Watkins, Scott
Version Date: Feb 15, 2012
Study Title: Evaluation of an Emergency Checklist for Intraoperative Pediatric Critical Events
Institution: Vanderbilt University Medical Center
Table of Contents:
Study Schema
1.0
Background
2.0
Rationale and Specific Aims
3.0
Inclusion/Exclusion Criteria
4.0
Enrollment/Randomization
5.0
Study Procedures
6.0
Reporting of Adverse Events or Unanticipated Problems involving Risk
to Participants or Others
7.0
Study Withdrawal/Discontinuation
8.0
Statistical Considerations
9.0
Privacy/Confidentiality Issues
10.0 Follow-up and Record Retention
Appendix A: List of Emergency Checklist Cards
Appendix B: Scenarios and Metrics
Appendix C: Survey
Appendix D: Consent for video taping
Revision Date: December 1, 2003
Principal Investigator: Watkins, Scott
Version Date: Feb 15, 2012
Study Title: Evaluation of an Emergency Checklist for Intraoperative Pediatric Critical Events
Institution: Vanderbilt University Medical Center
1.0
Background
Emergencies involving pediatric patients in the operating room are
uncommon events that require prompt and coordinated care from the
operative team to insure a safe resolution. The incidence of critical events
during pediatric procedures is unclear, although rates between 2 and 3%
have been reported (1-3). A review of the UK National Reporting and
Learning System, reviewed 96,298 critical incidents over a three year
period, of which 606 incidents occurred in the pediatric population(2). A
review of the first 10,000 anesthetics performed at a children’s hospital in
Singapore found a critical incident rate of 2.97% (1). Respiratory events,
primarily laryngospasm, were the most common, followed by
cardiovascular events, including hypotension from hemorrhage, sepsis and
arrhythmias(1). Marcus et al reported 668 incidents from a review of over
28,000 anesthetics, giving a rate of 2.4% at a single hospital in the UK(3).
Due to variation in reporting of adverse events, the true incidence is likely
under-reported. The majority of critical events in children are related to
the cardiovascular and pulmonary/airway systems, with hemorrhage,
laryngospasm and adverse medication reactions being most common (4).
The practice of pediatric anesthesia has seen a steady increase in
overall safety with the introduction of pulse oximetry and capnography as
standard monitors and the advent of safer volatile anesthetics(4). Despite
the overall increase in safety, critical events and emergencies still occur,
and often practitioners are unprepared to manage these events. The
enhanced safety of our practice may be contributing to the lack of
preparedness of practitioners, who may go long periods of time before
experiencing a critical event. Studies have demonstrated that practitioners
skills in advanced life support decay over time (5) and newer training hour
guidelines limit exposure of trainees to emergency events. Further, studies
have shown that knowledge gaps exist amongst providers in regard to
current guidelines for cardiopulmonary resuscitation(6) and that providers
often fail to adhere to best practices(7). The irregularity and
unpredictability of critical events contributes to knowledge and skill gaps in
practitioners’ ability to manage these events.
Checklists have been proposed as solutions to knowledge gaps and
failures to adhere to standards of treatment that occur during crisis
situations. Ziewacz et al, recently published a study in which they identified
Revision Date: December 1, 2003
Principal Investigator: Watkins, Scott
Version Date: Feb 15, 2012
Study Title: Evaluation of an Emergency Checklist for Intraoperative Pediatric Critical Events
Institution: Vanderbilt University Medical Center
12 frequently occurring operating room crises and created a checklist with
evidenced-based best practices for each, for use during intraoperative
emergencies in adults(7). They subsequently tested the emergency cards in
high fidelity simulator sessions with operating room teams consisting of a
surgical attending, surgical resident, anesthesia attending, anesthesia
resident, circulating nurses, and surgical technologists. Each team
performed scenarios with and without the emergency checklist cards as
cognitive aids. The primary outcome was adherence to critical steps. They
found that the group that used the checklist had a 6-fold reduction in
omission of critical management steps. The test OR teams found the crisis
checklists to be “straightforward, usable, and beneficial” (7).
The use of electronic cognitive aids has grown in popularity with the
advent of personal digital assistants (PDA) and smart phones. A recent
randomized controlled trial compared performance during simulated
cardiac arrest scenarios with and without the use of the Resuscitation
Council UK’s iResus© application on a smart phone(8). Results
demonstrated a significant improvement in performance on a standardized
test of advanced life support skills in the group using the smart phone
application. In contrast, other studies have demonstrated potential harm
with the use of electronic cognitive aids due to selection of the incorrect
algorithm (9) and a delay in initiation of chest compressions(10). This
highlights the limitation of any cognitive aid, that the user must make the
correct diagnosis and select the correct algorithm. Further, the introduction
of new technology has the potential to delay the initiation of basic life
support therapies. To date, no studies have evaluated the incorporation of
cognitive aids into anesthesia information management systems (AIMS).
2.0
Rationale and Specific Aims
The types of critical events that occur during pediatric surgery differ
from those that occur during adult procedures, with respiratory and airway
events being more common in children(3) and myocardial ischemia and
ventricular arrhythmias occurring more often in adults. Because of these
differences, the Society for Pediatric Anesthesia committee for safety and
quality has developed a series of emergency checklist cards tailored for use
by pediatric peri-operative teams. The emergency checklists cover a range
Revision Date: December 1, 2003
Principal Investigator: Watkins, Scott
Version Date: Feb 15, 2012
Study Title: Evaluation of an Emergency Checklist for Intraoperative Pediatric Critical Events
Institution: Vanderbilt University Medical Center
of potential intraoperative events including cardiovascular, airway,
pulmonary, medication reactions and equipment related events. See
Appendix A for complete list of emergency cards. The topics chosen are a
combination of common life-threatening operating room crises and rare life
threatening events affecting the pediatric population. Each emergency card
contains evidence based treatment steps and best practices as determined
by the panel of pediatric anesthesiologist and from review of relevant
literature.
This study will use emergency checklists in simulated pediatric
emergencies to test their efficacy and ease of use. Each simulated crisis
scenario will contain a set of key processes, consisting of critical and
essential steps for the successful management of the life threatening
events. We hypothesize that teams will adhere to best practices more often
when the emergency checklists (paper or electronic) are used than when
no checklist is used in simulated pediatric operating room emergencies. We
hypothesize that use of the checklist (paper or electronic) will reduce the
amount of time required for teams to make the correct diagnosis. In
addition, we believe that an electronic version of the checklist incorporated
in the AIMS will further improve adherence and time to diagnosis. During
the simulation, participants (residents and fellows) will be asked to
complete 6 scenarios related to pediatric OR emergencies either using or
not using the checklists. After which they will be asked to talk about what
they did, explain any frustrations and complete a questionnaire about their
experience. Analysis about performance and subjective data from the
questionnaires, observations and retrospective video data will highlight
main concerns around pediatric emergencies in the OR and the use and
design of the checklists and how it impacts the current workflow and
participant response.
3.0
Inclusion/Exclusion Criteria
Inclusion:
- Current residents in anesthesiology at our institution
- Fellows in pediatric anesthesiology at our institution.
4.0
Enrollment/Randomization
Revision Date: December 1, 2003
Principal Investigator: Watkins, Scott
Version Date: Feb 15, 2012
Study Title: Evaluation of an Emergency Checklist for Intraoperative Pediatric Critical Events
Institution: Vanderbilt University Medical Center
Currently, residents in anesthesiology and fellows in pediatric
anesthesiology at our institution participate in pediatric anesthesia
simulation for the purposes of training and education. These sessions are
held monthly for residents rotating through VCH and approximately every
three months for the fellows. Our intent is to recruit and consent from this
pool of residents and fellows. The scenarios will be used for the simulation
sessions as education tools, regardless of the trainees desire to participate.
Those that consent to the study will be randomized to performing the
scenarios with or without the aid of the emergency checklist cards.
5.0
Study Procedures
This study consists of a general introduction to the simulation center,
simulated emergency OR scenarios, and subjective evaluation. The study
will take place in CELA and each session will consist of 3-4 resident or fellow
anesthesia providers. As part of their current training rotations, these
individuals regularly participate in simulation training and the evaluation of
the checklist will be incorporated into this. During the general introduction
to the simulation center, participants will be informed about the
educational objectives for the simulation training including conduct,
schedule and the creation of a positive learning environment. The video
consent will also be obtained; participants choosing not to be videotaped
can still fully participate in the simulation session without any
repercussions.
The researcher will then answer any questions that the participants have
and provide a brief overview of about the first patient of the day. One
participant will be designated as the lead anesthesiologist and backup
anesthesiologists will also be designated. This will rotate such that all
participants will be the lead and backup anesthesiologist at least once and
exposed to each condition. Because it is a within-study design, if a cognitive
aid (checklist) is to be utilized for the scenario, participants will be provided
with the aid.
The participant will then continue to the simulation part of the study. There
will be 6 scenarios, which are characterized as A through F in the table
below (for complete scenario description see Appendix B). The scenarios
Revision Date: December 1, 2003
Principal Investigator: Watkins, Scott
Version Date: Feb 15, 2012
Study Title: Evaluation of an Emergency Checklist for Intraoperative Pediatric Critical Events
Institution: Vanderbilt University Medical Center
will be counterbalanced to minimize any potential order effects. Each
participant group will experience each condition, the standard practice, the
cognitive aid and the electronic cognitive aid (characterized as 1-3 on the
table below). This scheme is only illustrated for 9 groups below, so it will be
repeated as necessary to reach our N of 60 participants. The table
represents how this will be done. Additionally, each scenario will be tested
in each condition (as can be seen by the table below).
Table 1: Sample of counterbalancing for each scheduled session, where 1=no aide, 2=
aide, 3=electronic aid and A through F are patient emergencies
Scenario
Order
Participant
Group
1
2
3
4
5
6
7
8
9
First
1A
2A
3A
1C
2C
3C
1E
2E
3E
Second
1B
2B
3B
1D
2D
3D
1F
2F
3F
Third
2C
3C
1C
2E
3E
1E
2A
3A
1A
Fourth
2D
3D
1D
2F
3F
1F
2B
3B
1B
Fifth
3E
1E
2E
3A
1A
2A
3C
2C
2C
The researcher will brief the participant about the patient’s current
condition and any current concerns. The participant will be given the
opportunity to ask any questions before starting the simulation. The
simulation takes approximately 2 hours to complete all six scenarios.
Between each scenario, participants will be allowed a break if necessary
and to accommodate the training aspect of the study there will be a
debriefing after each scenario. As illustrated in Appendix B, a scoring
checklist of key metrics, consisting of critical and essential steps for the
successful management of the life threatening events, will be recorded for
each simulated scenario. The effect of the cognitive aid will be determined
by the accuracy and comprehensiveness of the participant’s responses. For
example, did they complete each step required on the checklist. In
addition, certain items on the scoring checklist will be timed to determine
the impact of the cognitive aids on time to diagnosis and time to correct
therapeutic intervention.
Revision Date: December 1, 2003
Sixth
3F
1F
2F
3B
1B
2B
3D
2D
2D
Principal Investigator: Watkins, Scott
Version Date: Feb 15, 2012
Study Title: Evaluation of an Emergency Checklist for Intraoperative Pediatric Critical Events
Institution: Vanderbilt University Medical Center
The use of videotape is essential for recording the participant’s
interactions with the emergency cards. The videotape will capture details
that observers might miss, such as facial expression and verbatim
comments. Because the study seeks to understand the usefulness and
usability of the emergency cards as well as its possible effects on the
participant’s information processing and decision making, it is not feasible
to capture all of the potential contributing information real time without
the ability to review video records of the simulations for data validation,
coding, reliability and to inform tool usability.
Following completion of the scenarios, the participant will complete
a questionnaire soliciting information about their interactions with the
checklist and demographic information (Appendix C). After completion of
the scenarios and questionnaires, the participants will be debriefed around
their participation in the study and any further evaluation of the checklist
and thanked for their time.
6.0
Reporting of Adverse Events or Unanticipated Problems involving Risk
to Participants or Others
The PI, co-investigator, and their research team will conduct data and
safety monitoring. The PI will have ultimate responsibility for data quality
and subject safety. In addition, the PI, in consultation with the research
team will regularly monitor study progress, goal achievement, and overall
research direction.
Data Monitoring. Database security is maintained using a multi-layered
approach to both limit access and the ability to alter data. There are strong
protections restricting access to the VUMC network. Local installation of
proprietary software is required to detect our server on the network. Each
authorized user has a unique username and password that allows specific
access and editing privileges. Browsing level access only is used except
when data entry/editing are explicitly intended. While multiple authorized
users can simultaneously view data, only a single specifically authorized
user can edit a given record at any time. The database is stored on a secure
server, which is also password protected and only accessible to authorized
lab personnel, in our locked laboratory offices. Database backups are
Revision Date: December 1, 2003
Principal Investigator: Watkins, Scott
Version Date: Feb 15, 2012
Study Title: Evaluation of an Emergency Checklist for Intraoperative Pediatric Critical Events
Institution: Vanderbilt University Medical Center
conducted automatically on a daily, weekly, and monthly basis. The PI will
regularly monitor the data collection and analysis process for data
appropriateness, comprehensiveness, accuracy, and timeliness.
Participant Safety Monitoring. The risks to subjects, although quite low,
are almost exclusively related to potential release of data about clinical
performance or adverse events. These are discussed in other sections of
this proposal. However, because of the sensitive nature of the research
data, the entire project team will continually monitor these issues and
evaluate in great detail any concerns raised by study participants.
This study poses minimal risk to the participants because it is being
conducted in a simulated environment where participants are being asked
to perform tasks that are routine in their everyday work. Thus, while
psychological stress, feelings of inadequacy, disclosure about participant
performance, and inconvenience of time involved with study participation
are the possible risks participants may experience, these risks are minimal
in this study.
Colleagues and supervisors will not be made aware of who has participated
in the study or participant’s performance. Additionally, the simulation
center and researchers will create a non-judgmental atmosphere and low
stress- tone of questioning after each scenario by maintaining a
professional environment and supporting the participant in their task.
Additionally, the researchers will emphasize to the participant that they can
withdraw from the study and any time without and negative consequences
to their job or reputation. Finally, participants will be reminded that the
simulation study is not testing their performance, but usefulness, usability
and performance of the new checklist.
During the data collection, one or more of the researchers will be present
to mitigate any unforeseen problems. If concerns about the study or
participant’s safety should arise, the researcher will halt the study and
review study procedures.
Participant performance will be measured in binary fashion using a list of
predetermined key steps for the management of each critical event. See
Appendix Bfor complete list of measurements for each scenario.
Revision Date: December 1, 2003
Principal Investigator: Watkins, Scott
Version Date: Feb 15, 2012
Study Title: Evaluation of an Emergency Checklist for Intraoperative Pediatric Critical Events
Institution: Vanderbilt University Medical Center
Finally, in order to minimize risk of performance disclosure, especially to
employers, participant’s personal information will be stored in a locked
filling cabinet in a locked laboratory separate from all other study materials.
The rest of the study materials will be assigned a random participant
number and be identified that way. Participant’s performance
characteristics will only be reported for the group as a whole when data is
published. Additionally, any videotaped data that is viewed outside of the
research lab will be stripped of all participant identifiers using editing
technology.
The risks to participants in this study as described above are quite minimal
compared to the potential benefits and widespread impact this study. The
potential benefits to science and humankind include a better
understanding of how cognitive aides such as the emergency cards can
impact participant performance in particular situations, especially
considering the high consequence of failure when an emergent issue occurs
in the OR. It is very difficult to study how healthcare practitioners manage
their patients during emergency events using cognitive aides because of the
unpredictable nature of emergencies and patient situations. Participants
will provide insight as to how to improve healthcare delivery in situations
that have been traditionally difficult to study.
7.0
Study Withdrawal/Discontinuation
Participants may withdraw previously signed consent forms at any time. In
addition, participants may change their selection regarding the extent to
which the video data may be used, if at all.
8.0
Statistical Considerations
Describe the statistical power of the study, the confidence intervals and the method for
analysis. Describe any possible deviations and their statistical impact.
9.0
Privacy/Confidentiality Issues
We are requesting a waiver of consent because we do not want to inform
participants that we are studying the efficacy of the cognitive aids.
Informing them that the emergency checklists are the subject of this
Revision Date: December 1, 2003
Principal Investigator: Watkins, Scott
Version Date: Feb 15, 2012
Study Title: Evaluation of an Emergency Checklist for Intraoperative Pediatric Critical Events
Institution: Vanderbilt University Medical Center
investigation might cause them to adhere to these procedures even if they
would have taken other actions in the simulation. We will tell them of their
participation at the end of the simulation session. A video consent will be
obtained at the beginning of the simulation session (Appendix D).
To protect the privacy of the research participants, signing the video
consent form and data collection will be completed at the simulation
center, which is a limited access area. The data collected will be in the form
of electronic spreadsheets, digital video records, and paper surveys; each
requires separate handling.
All participant information regardless of capture method will be accessible
only by key research personnel. Additionally, all electronic information will
be stored on password-protected computers. All video materials (if
recorded to permanent media) and paper surveys will be stored in a locked
cabinet and de-identified using a random case and subject number. A
password-protected crosswalk file that links case number to participant ID
will be kept separately by a research administrator and no investigator of
research assistant will have access to this information once the case record
is complete and locked.
Upon completion of the study, all coded paper records will be destroyed.
All data will remain secured, de-identified and no individual subject data
will be disseminated. Video records where participants gave consent will be
used for the purposes specified by the video informed consent. The
participants can agree to have videotapes used by designated research
personnel to conduct the research project (i.e. video review of data).
Additionally, they may agree to additional uses of video images and
videotapes shown including: scientific print and electronic publications,
meetings of scientists and educators, classrooms of trainees for educational
purposes, public presentation to non-scientific groups, and on radio or
television. Consent is obtained for each of these individually (See appendix
D). Since the simulation scenarios will take place in groups of 3-4
individuals, the treatment of the videotapes will be to that of the most
conservative, such that if one individual consents only if obscured then this
will be how the videotape will be handled. Additionally participants have
Revision Date: December 1, 2003
Principal Investigator: Watkins, Scott
Version Date: Feb 15, 2012
Study Title: Evaluation of an Emergency Checklist for Intraoperative Pediatric Critical Events
Institution: Vanderbilt University Medical Center
the ability to consent only if their face is obscured to be unrecognizable on
the video or photographic images used. Video labels will have no
identification information about the participant and will not be identified
by name, but their likeness could appear in presentations if consent was
obtained. If participants only agree to the possible video uses if the face is
obscured these videos will use different colored labels to reflect this choice.
Videotapes in which participants did not consent to use for research
purposes will be not be recorded at all.
Participant’s information will be identified with a random number that is
generated using a customized random number generator rather than any
identifying characteristics. Any identifying information will be stripped and
removed from any materials collected (e.g. researcher notes and
participant surveys). Digital video will be retained for the duration of the
research project and subsequently destroyed.
10.0
Follow-up and Record Retention
See response to question 9.
11.0
References
1.
Tay CL, Tan GM, Ng SB. Critical incidents in paediatric anaesthesia: an audit of
10 000 anaesthetics in Singapore. Paediatr Anaesth 2001;11:711-8.
MacLennan AI, Smith AF. An analysis of critical incidents relevant to pediatric
anesthesia reported to the UK National Reporting and Learning System, 20062008. Paediatr Anaesth 2011;21:841-7.
Marcus R. Human factors in pediatric anesthesia incidents. Paediatr Anaesth
2006;16:242-50.
Paterson N, Waterhouse P. Risk in pediatric anesthesia. Paediatr Anaesth
2011;21:848-57.
Semeraro F, Signore L, Cerchiari EL. Retention of CPR performance in
anaesthetists. Resuscitation 2006;68:101-8.
Heitmiller ES, Nelson KL, Hunt EA, Schwartz JM, Yaster M, Shaffner DH. A
survey of anesthesiologists' knowledge of American Heart Association Pediatric
Advanced Life Support Resuscitation Guidelines. Resuscitation 2008;79:499-505.
Ziewacz JE, Arriaga AF, Bader AM, Berry WR, Edmondson L, Wong JM, Lipsitz
SR, Hepner DL, Peyre S, Nelson S, Boorman DJ, Smink DS, Ashley SW,
Gawande AA. Crisis checklists for the operating room: development and pilot
testing. J Am Coll Surg 2011;213:212-7 e10.
2.
3.
4.
5.
6.
7.
Revision Date: December 1, 2003
Principal Investigator: Watkins, Scott
Version Date: Feb 15, 2012
Study Title: Evaluation of an Emergency Checklist for Intraoperative Pediatric Critical Events
Institution: Vanderbilt University Medical Center
8.
9.
10.
Low D, Clark N, Soar J, Padkin A, Stoneham A, Perkins GD, Nolan J. A
randomised control trial to determine if use of the iResus(c) application on a
smart phone improves the performance of an advanced life support provider in a
simulated medical emergency. Anaesthesia 2011;66:255-62.
Zanner R, Wilhelm D, Feussner H, Schneider G. Evaluation of M-AID, a first aid
application for mobile phones. Resuscitation 2007;74:487-94.
Merchant RM, Abella BS, Abotsi EJ, Smith TM, Long JA, Trudeau ME, Leary
M, Groeneveld PW, Becker LB, Asch DA. Cell phone cardiopulmonary
resuscitation: audio instructions when needed by lay rescuers: a randomized,
controlled trial. Ann Emerg Med 2010;55:538-43 e1.
Appendix A: List of Emergency Checklist Cards
Appendix B: Scenarios and Metrics
Appendix C: Survey
Appendix D: Consent for video taping
Revision Date: December 1, 2003