The NCSBN National
Simulation Study: Results
from Year 1
2012 NCSBN Scientific Symposium
September 11, 2012
Arlington, VA
Study Collaborators
Jennifer Hayden, MSN, RN
NCSBN, Project Director
Pamela Jeffries, PhD, RN, FAAN, ANEF
Johns Hopkins University
Suzan Kardong-Edgren, PhD, RN, ANEF
Boise State University
Washington State University
Nancy Spector, PhD, RN
NCSBN
Study Phases
Phase I
• National survey of simulation use
in prelicensure nursing programs
Phase II
• Multi-site, randomized, controlled
study of three levels of simulation
use across the nursing curricula
Phase III
• Longitudinal follow-up of nursing
graduates into clinical practice
Phase II-Multi-site, randomized study of
simulation/clinical experiences
Purpose
Determine if there are differences in clinical
competency among graduating nursing
students with 50%, 25% or up to 10% of
traditional clinical hours substituted with
simulation experiences.
Study Groups
 Control Group (clinical as usual)
Up to 10% of clinical time can be in
simulation
 25% simulation in place of clinical
 50% simulation in place of clinical
Research Questions
1. Are there differences in clinical competency
among graduating nursing students in the three
study groups?
2. Are there differences in knowledge among
graduating nursing students in the three study
groups?
3. Are their perceived differences in how well learning
needs are met in the clinical and simulation
environments among the three study groups?
4. Are there differences in clinical competency among
the three study groups in each of the core clinical
courses?
Null Hypothesis
There are no differences between the
study groups in knowledge, clinical
competency, and learning needs being
met.
Substitution Rate
 1 hour of traditional clinical is being replaced
with 1 hour of simulation time
 Time spent in a simulation lab either as an
active participant or active observer of a
simulated scenario followed by debriefing
 This time in the simulation lab is counted
towards required clinical hours for
graduation
Core Clinical Courses
Randomization will be maintained in the core
clinical courses
 Foundations
 Medical/Surgical Nursing
 Advanced Medical/Surgical Nursing
 Obstetrical Nursing
 Pediatric Nursing
 Mental Health/Psychiatric Nursing
 Community Health/Public Health Nursing
Study Will Evaluate:
 Clinical competency
Creighton Competency Evaluation Instrument
(CCEI)
 Nursing knowledge
ATI Comprehensive Assessment and Review
Program
 How well student learning needs were met
in the clinical and simulation environments
Clinical Learning Environment Comparison Survey
(CLECS)
Defining Simulation
Operational definition: An activity or event
replicating clinical practice. For the purpose
of this study, simulation includes multiple
types of scenario based simulation using
high fidelity manikins, medium fidelity
manikins, standardized patients, role
playing, and computer based simulations.
NLN/Jeffries Simulation Framework
Study Management at each School
 Team Leader
 Study Team
 Trained in the protocol
 Trained in the debriefing technique
 Large part in the development of the simulation
curriculum
Clinical Faculty Training
 Clinical faculty rate students each week of
clinical and each day of simulation using the
Creighton Competency Evaluation
Instrument
 Each semester clinical faculty undergo
online training in the use of the instrument
 Calculate inter-rater reliability and intra-rater
reliability each semester
 Clinical faculty review objectives and
“norm” what are competent/not competent
expectations
Simulation Day
 Entire clinical group reports to the lab with
the instructor
 Assigned roles: primary nurse, secondary
nurse, family member, documenter, med
nurse, observers
 15-20 minutes for scenario followed by 20+
minutes debriefing
 Clinical instructors rate their students with
the CCEI
 2-3 clinical groups in the lab at the same
time
Stations




Manikin scenarios
Standardized patients/patient actors
Skills Stations
Group Activities
 Layered Learning Critical Thinking Cards
 Sensory kits
 Group Therapy in a Box
 Computerized scenarios
 ATI Real Life
 Virtual Clinical Excursions
Scenarios Available
Number of Scenarios
Available
Course
Foundations
26
Medical-Surgical
64
Advanced Medical-Surgical
46
Obstetrical
42
Pediatrics
67
Psychiatric-Mental Health
24
Community Health
23
Total
292
Student “Safety”
 Safety data collected each day of simulation
and each week of clinical
 Oversight Committee at each school
Reviews school level safety data, mid-term
grades, final grades, ATI scores, feedback from
faculty and study team
 Data Safety Monitoring Board established
Meets monthly to review school level data and
national data to ensure participants’ education is
not being jeopardized by being in the study
Participant Eligibility




18 yo or older
Provide informed consent
Full-time RN student
Complete the nursing program with study
cohort
Ineligibility
 Licensed as LPN/VN or RN
 Accelerated RN student
The Students
Gender:
Age:
Race:
86% Female
56% 18-24
28% 25-34
16% 35+
83% White 8% Black
6% Asian
1% Native American
1% Hawaiian/PI 2% Multiracial
Ethnicity:
18% Hispanic
Student Demographics
Previous Degree: 35% (295)
62% (181) Bachelor’s Degree
41% (120) Associate Degree
3% (9) Master’s Degree
Certified Nursing Assistant: 16% Yes
Paramedic/EMT: 19
Military Experience:
28 Yes
8 Medical Corps
Study Numbers
847 Students consented and randomized
411 ADN
436 BSN
54 Failed a course
39 ADN
15 BSN
96 Withdrew
40 ADN
56 BSN
697 Continuing on to Year 2
332 ADN
365 BSN
82% rate of continuing in the study
A closer look at attrition
Clinical Instructor Demographics
(N=102)
Gender:
Race:
95% Female
Ethnicity:
Age:
9% Hispanic
20% Less than 35
24% 35-44
18% 45-54
39% 55+
79% White
7% Asian
6% Black
1% Native American
Clinical Instructor Demographics
Teaching experience: 94% Yes
average 9.0 years (SD 8.7)
Current position:
43% Instructor
29% Adjunct faculty
13% Asst or Associate Professor
4% Professor
14% Other
Clinical Instructor Simulation Experience
21% No previous experience
55% Observed simulation
47% Run simulation scenarios
50% Debriefed students after simulations
38% Assessed students who have
participated in simulations
23% Written simulation scenarios
33% Participated in formal simulation training
Clinical Hours
Range of total clinical hours w/o capstone:
520-776 hours
25%= 130-194 hours of sim
50%= 260-388 hours of sim
What have we learned so far…
 Study Model is working






Use of a trained simulation team
Clear objectives
15-20 minute scenarios
Having clinical instructors observe
Having students observe their peers
Debriefing
 Skeptical faculty are now on board
Incidental Findings
 Informal training/norming of clinical faculty-more inclusion of adjunct faculty
 Bonding of students in their clinical group
 Clinical instructors are using DML in their
post-conferences
 Johns Hopkins University—focus groups of
clinical instructors and how this experience
is informing their teaching
Study Timeline
Phase I
Phase II
Phase III
• October 2009 survey development began
• January 2010 survey mailed
• October 2010 results published
• November 2010 sites selected, first training meeting held
• March 2011 research protocol finalized
• April and June 2011 second and third training meetings held
• Summer 2011 students enrolled into study
• August 2011 semester 1 of study begins
• May 2013 semester 4 of study ends
• April 2013 students consented for longitudinal follow-up study
• May 2013 students graduate
• December 31, 2014 data collection completed
Current Status of Phase II
 Study Students are in the senior year of
their programs
 Students will graduate in May 2013
 We will collect NCLEX pass rates through
December 31, 2013
 Final results are expected in Spring 2014
Looking Ahead to Phase III
 Phase III will follow graduates of the study
sites into clinical practice
 Purpose of Phase III is to evaluate how well
students were prepared for professional
practice
 Evaluations will include:





Clinical competencies
Safety competencies
Job satisfaction
Retention
Stress
What can regulators take away
 Who is conducting the sims? Do they have
training, what kind?
 What kind of debriefing?
 How will students spend their time while in
sim?
 How long are sims?
 Are there clear objectives for each sim?
 Evaluations/graded?
 Is the sim lab accredited?
www.ncsbn.org/2094.htm
jhayden@ncsbn.org