International Nursing Association for Clinical Simulation and Learning.org
Report on the NBSBN National Simulation Study
Julia Greenawalt, PhD, BSN
Assistant Professor at Indiana University of Pennsylvania
Originally published in the INACSL Quarterly E-Newsletter (August 2014)
The NCSBN departments of Research and Regulatory Innovations had collaborated to conduct a landmark, national,
multi-site research study. The findings and conclusions from this study were shared first at the NCSBN Annual Meeting
held in Chicago, August 13-15, 2014. This study was led by NCSBN staff members Jennifer Hayden, MSN, RN
associate, Research; Kevin Kenward, PhD, director, Research and Nancy Spector, PhD, RN, director, Regulatory
Innovations; and two national experts in the area of simulation in nursing education: Pamela Jefferies, DNS, RN,
FAAN, Johns Hopkins University; and Suzan Kardong-Edgren, PhD, RN, ANEF, CHSE, Boise State University and VP for
Research for the INACSL. The study had three phases. Phase one consisted of surveying national cohorts of prelicensure nursing students from January-March 2010. Phase two then consisted of conducting a randomized,
controlled, multi-site, longitudinal study of three levels of simulation use in lieu of clinical hours. Students were
randomly assigned to a group where 25 percent of the time was normally spent in simulation, a group where 50
percent of the time normally spent at clinical sites would be spent in simulation, or a group that received business as
usual or less than 10% of clinical time in simulation. Phase three followed a cohort of these students into the clinical
environment for the first 6 months after graduation. This longitudinal follow-up of graduates into their first year of
practice will focus on retention of new nurses and clinical judgment after graduation.
The review of literature confirmed that there were many studies utilizing simulation, but most had small sample sizes,
no control group, lacked randomization, and displayed a vast variability in study design limiting the generalizability of
the findings to the discipline and a noted absence of longitudinal studies. Thus, a significant gap was revealed in
translating the outcomes of these studies to the current workforce. A study commenced, initiated by the NCSBN that
improved upon the previously noted short-comings.
The research questions asked were:
1. Does substituting clinical hours with 25% and 50% simulation impact educational outcomes (knowledge,
clinical competency, critical thinking and readiness for practice) assess at the end of the undergraduate
nursing program?
2. Are there course by course differences in nursing knowledge, clinical competency, and perception of learning
needs being met among undergraduate students when traditional clinical hours are substituted with 25% and
50% simulation?
3. Are there differences in first-time NCLEX pass rates between students that were randomized into a control
group, 25% and 50% of traditional clinical substituted with simulation?
4. Are there differences in clinical competency, critical thinking and readiness for practice among the new
graduate nurses from the three study groups?
5. Are there differences among new graduates from the three study groups in acclimation to the role of the
professional nurse?
The theoretical design used to guide the study was the updated NLN/Jeffries Simulation. The research design used a
randomized, controlled, large-scale, multi-site longitudinal study which had a follow-up component to it. Three study
groups were analyzed, from ten selected schools from across the United States of America. Students were examined
from seven common nursing courses within a nursing program; fundamentals of nursing, med-surg nursing, advanced
med-surg nursing, maternal-newborn nursing, pediatrics, mental health nursing and community health nursing.
Reliable and valid instruments were used for the study; nursing knowledge was measured with the ATI Content
Mastery Series™, clinical competency was measured using the Creighton Competency Evaluation Instrument (CCEI),
the New Graduate Nurse Performance Survey and the National Council Licensure Exam™ (NCLEX), whether learning
needs were met was measured by the Clinical Learning Environment Comparison Survey (CLECS), critical thinking was
measured using the Critical Thinking Diagnostic, and acclimation to the role of the RN was measured using patient
International Nursing Association for Clinical Simulation and Learning.org
loads, workplace stress, whether the RN left his/her first nursing position and noting the charge nurse responsibilities
taken on within the first year of practice. Additionally, participants were measured using the Global Assessment of
Clinical Competency and Readiness for Practice. Simulation was operationalized for this study as being inclusive of
medium and high fidelity, employing standardized patients, utilization of role playing and/or computer based critical
thinking simulations, and using skills stations.
The findings revealed that there were 847 participants that consented to enroll in the study.
Figure: Sample Dispersion
Eighty-six percent were female, eighty-four percent were white, sixteen percent had previous experience as a nursing
assistant, and the average age was 26.3 years (SD 8.0, range 18-60). While the study employed best-practices
adhering to the noted INACSL Standards of Best Practice, employment of high quality simulations was noted, the
debriefing method was grounded in educational theory and the study was conducted using trained and dedicated
simulation faculty some limitations still must be acknowledged. Participating schools were not randomly selected, only
one method of simulations was conducted and the quality of the clinical experiences was not addressed. Additionally,
the end of program preceptors and nurse managers were not blinded as to a study group and the distribution of end
of program and nurse manager surveys relied on the study participants.
The findings concluded that up to 50% simulation can be effectively substituted for traditional clinical experience in all
core courses across the pre-licensure curriculum, that 50% simulation can be effectively used in various program
types, in different geographic areas in urban and rural settings with good educational outcomes, that NCLEX pass
rates were unaffected by the substitution of simulation throughout the curriculum, that all three groups (control, 25%
and 50%) were equally prepared for entry into practices as a new graduate registered nurse, and that policy decisions
regarding the use and amount of simulation in nursing needs to be dependent upon the utilization of best practices in
simulation. There was a statistically significant difference on the end of course ATI™ examinations between the
groups, with the 50% group scoring better than the other two groups.
The implication of these findings to Boards of Nursing reveal that to achieve equivalent results, schools need: to use
formally trained and dedicated simulation faculty, to use the standards of best practice in simulation from INACSL, to
use a theory-based debriefing method using subject matter experts, and that the simulation environment needs to be
comprised of equipment and supplies which create a realistic environment.
This study empirically supports what many simulationists have felt; that quality simulation can be used to enhance
our current learning environment; but safe-guards, resources, and quality simulation practices need to be employed
to garner quality outcomes. As a simulationist, how will you embrace continuing education for yourself? As a
simulation director-how can you educate and advocate for your simulation lab to your superior, chair, dean, president
and provost of your school? Maybe you need to advocate to your Department of Education or to your state
representative. We are all in this to enhance the outcome at the bedside level or within the workforce environment, to
increase the safety and quality of care delivered ultimately to the patient. What role as a simulation aficionado will you
employ towards this end? The full study report is now available free of charge from the Journal of Nursing Regulation.