Running head: AN INTEGRATIVE LITERATURE REVIEW: IN HEALTHCARE
An Integrative Literature Review: In Healthcare Education, Are Computer Screen-Based
Simulators As Compared To High-Fidelity Patient Simulators As Effective In Relation To
Learning Outcomes
Beth Shaffer
University of Central Florida
1
AN INTEGRATIVE LITERATURE REVIEW: IN HEALTHCARE
2
Abstract
With the increase in online education in healthcare, the use of computer screen-based simulation
will allow students the opportunity for simulation that they may not otherwise have. To
determine if computer screen-based simulation is a viable option to high fidelity patient
simulation in healthcare education, a literature review was performed. Nine studies met
inclusion and exclusion criteria, with 823 nursing and medical students participating. Three
themes emerged from the findings, clinical performance, knowledge, and satisfaction.
Conflicting results were identified for clinical performance, therefore, high fidelity patient
simulation remains to be the gold standard and more research is needed to evaluate the
effectiveness of computer screen-based simulation. Computer screen-based simulation is an
option for activities involving a lower level cognitive knowledge. More studies are needed to
evaluate knowledge at higher cognitive level for computer screen-based simulation. High
fidelity patient simulation has been shown to have a higher satisfaction rating than computer
screen-based simulation. While computer simulation should not replace the interactive clinical
experience in the undergraduate program, there are many uses that may be appropriate for its
use.
Keywords: computer screen-based simulation, high fidelity patient simulation, education,
healthcare
AN INTEGRATIVE LITERATURE REVIEW: IN HEALTHCARE
3
Table of Contents
Abstract ........................................................................................................................................... 2
Significance and Background .......................................................................................................... 4
Research Question. ......................................................................................................................... 5
Method ........................................................................................................................................... 6
Search Strategy.......................................................................................................................................... 6
Definitions ................................................................................................................................................. 7
Inclusion and Exclusion Criteria................................................................................................................. 7
Coding........................................................................................................................................................ 8
Validity of Findings .................................................................................................................................... 8
Quality ....................................................................................................................................................... 8
Findings ........................................................................................................................................... 8
Study Characteristics ................................................................................................................................. 8
Sample Characteristics .............................................................................................................................. 9
Themes .................................................................................................................................................... 10
Clinical Performance ............................................................................................................................ 10
Knowledge ........................................................................................................................................... 10
Satisfaction .......................................................................................................................................... 11
Recommendations ........................................................................................................................ 11
Clinical Performance ............................................................................................................................... 12
Knowledge ............................................................................................................................................... 12
Satisfaction .............................................................................................................................................. 13
Conclusions ................................................................................................................................... 13
References .................................................................................................................................... 15
Appendix A .................................................................................................................................... 19
Appendix B .................................................................................................................................... 26
AN INTEGRATIVE LITERATURE REVIEW: IN HEALTHCARE
4
An Integrative Literature Review: In Healthcare Education, Are Computer Screen-Based
Simulators As Compared To High-Fidelity Patient Simulators As Effective In Relation To
Learning Outcomes
Simulation is being used in healthcare education as an adjunct to live patient interaction.
With the different modalities of learning in healthcare including online learning, traditional
classroom, clinical setting, skills labs, and simulation labs, it is important to review the literature
on the different options available so that faculty and schools can provide their students with the
best education. High-fidelity patient simulation has been proven to be an effective method in
nursing clinical education with the landmark study by The National Council of State Boards of
Nursing (NCSBN) proving that simulation may be substituted for up 50% of pre-licensure
clinical education (Hayden, Smiley, Alexander, Kardong-Edgren, & Jeffries, 2014).
With the increasing rates of online education, computer screen-based simulation should
be considered. While much research has been analyzed on high-fidelity patient simulation to
live patients, there are few comparative analyses on computer screen-based simulation to highfidelity patient simulation. This integrative literature review examines studies on computer
screen-based simulation to high-fidelity patient simulation and the effects on students learning
outcomes.
Significance and Background
Simulation provides a way for educators to evaluate a student’s ability to critically think,
problem solve, and communicate to others (Blevins, 2014). Studies have shown the
effectiveness of high-fidelity patient simulation in nursing programs as an option to replace
clinical hours or enhance clinical hours (Hayden et al., 2014). As of 2014, many states now
allow up to 50% of nursing clinical hours to be replaced with simulation ("National League for
AN INTEGRATIVE LITERATURE REVIEW: IN HEALTHCARE
5
Nursing," n.d.). It has also been utilized in advanced nurse practice programs, medical school
programs and used for recertification in different specialties. High-fidelity patient simulation is
desirable as it closely emulates reality in its desire for students to feel immersed in the
simulation.
In 2011, over 6.7 million students were enrolled in at least one online class ("Online
Learning Consortium," 2012). As of 2013, there were 130 online nursing graduate programs and
as of 2009, there were 129 fully online RN-BSN programs with up to 400 of the 692 RN-BSN
programs being offered partially online (American Association of Colleges of Nursing [AACN],
2014; Brooks & Morse, 2014; Kolowich, 2010).
With growing technology, there are advancements being made in web-based simulation.
Online virtual environments are being created for students to feel immersed in the environment
as an avatar responding in real time to a clinical experience in a virtual world (Youngblood et al.,
2008). In addition, web-based interactive simulation training programs are increasingly being
produced over the past several years (Johnson et al., 2014). Utilizing web-based simulation
would allow online RN-BSN students an avenue to complete clinical requirements and health
assessment requirements that are required by some institutions. Web-based simulation also
allows participants to access it from any where in the world, at any time, for a reduced cost that
may be beneficial to many disciplines in healthcare.
Research Question
In healthcare education, are computer screen-based simulators as compared to highfidelity patient simulators as effective in relation to learning outcomes? A comprehensive
review of the literature was completed to answer this question.
AN INTEGRATIVE LITERATURE REVIEW: IN HEALTHCARE
6
Method
In order to understand if there were differences in learning outcomes in healthcare
education utilizing computer screen-based simulation versus high-fidelity patient simulators a
comprehensive search was completed to identify published and unpublished research studies,
written in English, between 2008-2015.
Search Strategy
An initial search of Cumulative Index of Nursing and Allied Health Literature [CINAHL]
Plus, Medline, Academic Search Premier, PsychINFO, and Education Resources Information
Center [ERIC], using the key search terms, simulat*, “virtual reality”, virtual patients, nurs*,
student*, undergrad, baccalaureate*, and bachelor yielded 1,427 results. Therefore, additional
search terms were added, undergrad*, bachelor*, BSN, “virtual patient”, web-based, online,
computer*, mannequin, manikin, “human patient”, didactic*, instruct*, teach which yielded 112
results. After peer review of key search terms, healthcare and eval* were added yielding 114
results. After duplicates were removed the search yielded 63 results. After review, 29 articles
were rejected based on titles and abstracts not related to both computer screen-based simulation
and high-fidelity simulation. Therefore, 34 articles remained for further review. After review
and abstract only studies removed, three met inclusion criteria for this analysis. An independent
search of the Clinical Simulation in Nursing journal from January 2008 through March 2015
yielded 11 possible articles and after review, four studies met inclusion criteria for this analysis.
An additional independent review of Cook et al. (2013) systematic review and meta-analysis
reference list yielded no additional results that met inclusion criteria. After a continued search of
reference list from Consorti, Mancuso, Nocioni, and Piccolo (2012) and Google scholar, two
more studies were found that met inclusion criteria.
AN INTEGRATIVE LITERATURE REVIEW: IN HEALTHCARE
7
Definitions
All studies included in the integrative literature review include simulation used for the
purpose of teaching in healthcare education. Simulation is utilized in healthcare education to
increase students knowledge and practical skills in a subject matter, such as, critical thinking, a
particular skill, and advanced training by using different modalities other than real-life situations
(Aebersold & Tschannen, 2013). According to Aebersold and Tschannen (2013), high-fidelity
simulation utilizes “computerized manikins”, mid-fidelity simulation includes “standardized
patient’s, computer programs, or video games”, low-fidelity simulation consists of “role play,
non-computerized manikins or task-trainers”, and virtual simulation is an online simulation
environment (Table 1).
Computer screen-based simulation in this review only consists of
simulation that is seen on a computer screen and may or may not include real time virtual reality.
It does not include virtual reality hepatic task trainers that may be attached to the computer.
Computer screen-based simulation can also be considered low, moderate, or high fidelity
simulation. Due to the lack of definitions in computer screen-based simulation, this review will
define computer screen-based simulation as CD-ROM software that utilizes Power
Point slides, videos, and questions as seen in the study by Johnson, Ramos-Alarilla, Harilal,
Case, and Dillon (2012) and NurseSquared software utilizing the Electronic Health Record as
seen in the study by Wilson, Klein, and Hagler (2014) as low fidelity simulation. The use of
virtual patients simulations, virtual worlds, and virtual reality skills simulation without a hepatic
trainer are considered high fidelity simulation.
Inclusion and Exclusion Criteria
Inclusion criteria were any research studies considering participants in healthcare,
computer screen-based simulation, virtual simulation, and high fidelity standardized patients;
AN INTEGRATIVE LITERATURE REVIEW: IN HEALTHCARE
8
studies published between 2008-2015; peer-reviewed, and were written in the English language.
Exclusion criteria for studies inappropriate to this review included not examining outcomes
between computer screen-based simulations to high fidelity standardized patient simulation;
included partial task trainers to virtual reality simulation; and comparison of computer screenbased simulation to live standardized patients.
Coding
After reviewing the studies, the following coding themes emerged from the findings,
clinical performance, knowledge, satisfaction, and confidence. These are all based on student
feedback and testing of performance.
Validity of Findings
The Quelly Tool for Validity (2007) was used to validate the findings of the studies. Five
studies met a Level 3, high quality and four were a Level 2, moderate quality (Quelly, 2007).
Quality
Article quality and level of evidence was determined using criteria published by Melnyk
and Fineout-Overholt (2011).
According to Melnyk and Fineout-Overholt (2011) level of
evidence, eight studies met a Level II quality of evidence from well-designed random controlled
trials and one study from a Level III quality of evidence from well-designed controlled trial
without randomization (Cooper et al., 2015).
Findings
All nine studies compared computer screen-based simulation with high-fidelity patient
simulation in healthcare education. Three themes emerged from the findings, clinical
performance, knowledge, and satisfaction.
Study Characteristics
AN INTEGRATIVE LITERATURE REVIEW: IN HEALTHCARE
9
Two of the studies were randomized, experimental, pretest-posttest design, four were
quasi-experimental designs, two were prospective, pretest, posttest randomized design, and one
was an after-only experimental design. Five of the studies involved nursing students
participating in a critically ill patient scenario (Arnold, Johnson, Tucker, Chesak, & Dierkhising,
2013; Cooper et al., 2015; Howard, 2013; Liaw, Chan, Chen, Hooi, & Siau, 2014; Wilson et al.,
2014). One study involved advanced practice nurses caring for acutely ill patients in a simulated
scenario (Johnson et al., 2014). One study involved nursing students practicing Foley insertion
(Smith & Hamilton, 2015). The final study involved military healthcare personnel caring for
combat injuries (Johnson et al., 2012). One study was unpublished from a dissertation at Capella
University (Howard, 2013).
Sample Characteristics
Sample sizes ranged from 20 to 427. Of the nine studies there were 823 participants, with
an average number of 47 participants, excluding the one study of 427 participants. Seven were
conducted in the United States, one in Australia, and one in Singapore. Six studies involved
nursing students, one study involved, advanced practice students (APN), one study involved
licensed practical nurses (LPN), and one study involved medical students. Out of eight of the
studies the mean age was 25; 599 participants were female; and 163 were male. Age and gender
were not provided in one of the studies reviewed.
Limitations to the studies included small sample size, convenience samples, and no
longitudinal studies. Other limitations included different operators of the manikins, completed at
different settings, and different test measures. Limitations due to technology includes lack of
student knowledge to technology, lack of clarification when needed, less exposure to virtual
AN INTEGRATIVE LITERATURE REVIEW: IN HEALTHCARE
10
patients, technology mishaps, and some screen-based technology were of a lower caliber than
others in different studies.
Themes
Clinical Performance. In all nine studies clinical performance is measured as an
outcome variable. In five of the studies, no significant difference is seen between the computer
screen-based simulation group and the high-fidelity patient simulation group in clinical
performance [0% - 3% difference in range of test scores] (Arnold et al., 2013; Howard, 2013;
Liaw et al., 2014; Smith & Hamilton, 2015; Youngblood et al., 2008). The study by Cooper et
al. (2015) rated clinical performance as “moderate” with both modalities, but high fidelity patient
simulation achieved 49% on the performance checklist, while the computer screen-based
simulation group achieved 69% on the performance checklist. The remaining three studies
revealed that clinical performance was better with high fidelity patient simulation than computer
screen-based simulation; test scores ranged from 66% - 84% and 50% - 63% respectively
(Johnson et al., 2014; Johnson et al., 2012; Wilson et al., 2014). However, in two of the three
studies that high fidelity patient simulation had better clinical performance, low fidelity
computer screen-based simulation was used as a comparison (Johnson et al., 2012; Wilson et al.,
2014). Based on the results obtained from these nine studies, there is not a clear answer on
whether computer screen-based simulation is as effective as high fidelity patient simulation on
learning outcomes as they indicate conflicting results.
Knowledge. In four of the studies knowledge is measured as an outcome variable. In
two of the studies, no significant difference between groups was found (Arnold et al., 2013;
Johnson et al., 2014). Arnold et al. (2013) noted both groups with significant improvement in
posttest scores with the computer screen-based group scoring higher at 80% than the high fidelity
AN INTEGRATIVE LITERATURE REVIEW: IN HEALTHCARE
11
patient group at 76%. Johnson et al. (2014) noted by self-assessment that knowledge improved
significantly in both groups, posttest scores for the computer group were 76% and 79% for the
high fidelity group. Johnson et al. (2012) noted that the high-fidelity patient simulation group
scored higher on cognitive posttest scores (59%) than the computer screen-based simulation
group (49%). Cooper et al. (2015) checked posttest scores in only the computer screen-based
simulation group, which did show significant improvement (9% increase). While not all studies
directly evaluated knowledge, those that did had a positive outcome with both modalities of
simulation leading to inconclusive evidence to answer the research question of whether computer
screen-based simulation is as effective as high fidelity patient simulation on learning outcomes.
Satisfaction. Three studies measured student satisfaction as an outcome variable. All
rated high fidelity patient simulation higher in satisfaction (Arnold et al., 2013; Cooper et al.,
2015; Youngblood et al., 2008). A wider range was noted with Arnold et al. (2013) than Cooper
et al. (2015) and Youngblood et al. (2008), with 72% - 94%; 92% - 95%; 94% - 100%,
respectively. However, Copper et al. (2015) noted that while there was a significant difference
the overall effect size was small and that naturally the high fidelity patient simulation group
would have overall higher scores with team work and ‘face-to-face’ debriefing as this was not
part of the criteria in the computer screen-based simulation. Liaw et al. (2014) did note that 87%
were satisfied with the overall computer screen-based simulation experience, but did not obtain
satisfaction scores from the high fidelity patient simulation for comparison. With such a small
group utilizing satisfaction as an outcome measure, it is hard to obtain whether users are satisfied
with computer screen-based simulation. However, based on the results, participants prefer high
fidelity patient simulation as compared to computer screen-based simulation.
Recommendations
AN INTEGRATIVE LITERATURE REVIEW: IN HEALTHCARE
12
After review of all nine articles, three themes emerged, clinical performance, knowledge,
and satisfaction. Recommendations for each area are given according to a Strength of
Recommendation Taxonomy (SORT) scale by that has been modified by Shaffer (2015) to
accommodate nursing simulation in education as seen in Appendix B (Ebell et al., 2004).
Clinical Performance
High fidelity patient simulation should be recommended over computer screen-based
simulation as a preferred method for nursing education simulation in clinical performance. The
strength of recommendation is an A based on a modified SORT taxonomy to nursing education
as seen in Appendix B. However, with conflicting results and over half of the studies showing
no significant difference in performance with both groups under evaluation and one study
showed a higher performance with computer screen-based simulation, computer screen-based
simulation should also be considered a credible option for online students in supplemental
programs such as, RN-BSN, MSN and ACLS renewal. Future research is needed to determine if
clinical performance in computer screen-based simulation is sustainable over time with retention
in learning (Liaw et al., 2014). More studies are needed to directly compare computer screenbased simulation to high fidelity simulation in clinical performance with newer technology.
Knowledge
Computer screen-based simulation as well as high fidelity patient simulation should be
recommended for student learning at the lower cognitive level. The strength of recommendation
is an A based on a modified SORT taxonomy to nursing education as seen in Appendix B.
Practice experiences that may apply at this level include, documentation, communication,
pharmacology, enhance lecture content or be used in a “flipped classroom”, and to prepare
students for high fidelity patient simulation (Foronda, 2014). Limitations include difference in
AN INTEGRATIVE LITERATURE REVIEW: IN HEALTHCARE
13
educational delivery, differing testing materials, and differing evaluation methods used. Faculty
needs to consider incorporating computer screen-based simulation into the curriculum and
become educated on its potential uses. Future research is needed to evaluate knowledge at a
higher cognitive level in comparison of the two modalities.
Satisfaction
High fidelity patient simulation should be recommended for student learning in clinical
simulation, as student satisfaction is rated higher in hands on interaction and higher level of
satisfaction with the debriefing period, which is essential to the learning process. The strength of
recommendation is an A based on a modified SORT taxonomy to nursing education as seen in
Appendix B. More studies are needed to evaluate different methods of debriefing that may be
effective for computer screen-based simulation. Limitations include inadequate social
interaction with computer screen-based simulation and group debriefing that leads to retention in
learning. Future research needs to incorporate ways to bring others into a group dynamic in
computer screen-based simulation, such as, discussion forums after the simulation.
Conclusions
This literature review supports the recommendations of the National League for Nursing
Response to NCSBN Simulation Study (n.d.) for recommendations of high fidelity patient
simulation use for clinical experiences in nursing education. With the steady increase in online
nursing programs and classes being offered throughout the country, computer screen-based
simulation should be considered as a viable option to high fidelity patient simulation in RNBSN, MSN, and doctoral programs, as well as recertification courses, and as an adjunct to high
fidelity patient simulation. More studies are needed to determine the best uses for computer
screen-based simulation and direct studies comparison of simulation modalities are also needed.
AN INTEGRATIVE LITERATURE REVIEW: IN HEALTHCARE
There are many opportunities for computer screen-based simulation that could be utilized to
benefit students and nursing programs, such as, opportunities that may incorporate
interdisciplinary collaboration among nursing, medical, and other professional modalities;
capstone simulation; hybrid simulation; and used as a “flipped classroom” (Foronda, 2014).
14
AN INTEGRATIVE LITERATURE REVIEW: IN HEALTHCARE
15
References
Aebersold, M., & Tschannen, D. (2013, May 31). Simulation in nursing practice: The impact on
patient care. The Online Journal of Issues in Nursing, 18(2).
http://dx.doi.org/10.3912/OJIN.Vol18No02Man06
American Association of Colleges of Nursing [AACN]. (2014). Degree completion programs for
registered nurses: RN to master’s degree and RN to baccalaureate programs. Retrieved
from www.aacn.nche.edu/media-relations/fact-sheets/degree-completion-programs
Arnold, J. J., Johnson, L. M., Tucker, S. J., Chesak, S. S., & Dierkhising, R. A. (2013).
Comparison of three simulation-based teaching methodologies for emergency response.
Clinical Simulation in Nursing, 9, e85-e93. http://dx.doi.org/10.1016/j.ecns.2011.09.004
Blevins, S. (2014, March-April). The impact of simulation on patient care. MEDSURG Nursing,
23(2), 120-121.
Brooks, E., & Morse, R. (2014, January 7). Methodology: Best online nursing programs
rankings. US News. Retrieved from www.usnews.com/education/onlineeducation/articles/2014/01/07/methodology-best-online-nursing-programs-rankings-2014
Changing course: Ten years of tracking online education in the United States. (2012). Retrieved
from http://onlinelearningconsortium.org/survey_report/changing-course-ten-yearstracking-online-education-united-states/
Consorti, F., Mancuso, R., Nocioni, M., & Piccolo, A. (2012). Efficacy of virtual patients in
medical education: A meta-analysis of randomized studies. Computers & Education, 59,
1001-1008. http://dx.doi.org/10.1016/j.compedu.2012.04.017
Cook, D. A., Hamstra, S. J., Brydges, R., Zendejas, B., Szostek, J. H., Wang, A. T., ... Hatala, R.
(2013). Comparative effectiveness of instructional design features in simulation-based
AN INTEGRATIVE LITERATURE REVIEW: IN HEALTHCARE
16
education: Systematic review and meta-analysis. Medical Teacher, 35, e844-e875.
http://dx.doi.org/10.3109/0142159X.2012.714886
Cooper, S., Cant, R., Bogossian, F., Kinsman, L., Bucknall, T., & FIRST ACT Research Team
(2015). Patient deterioration education: Evaluation of face-to-face simulation and esimulation approaches. Clinical Simulation in Nursing, 11, 97-105.
http://dx.doi.org/dx.doi.org/10.1016/j.ecns.2014.10.010
Ebell, M. H., Siwek, J., Weiss, B. D., Susman, J., Ewigman, B., & Bowman, M. (2004). Strength
of recommendation taxonomy (SORT): A patient-centered approach to grading evidence
in the medical literature. Journal of the American Board of Family Practice, 17(1), 5967.
Foronda, C. (2014). Strategies to incorporate virtual simulation in nurse education. Clinical
Simulation in Nursing, 10(8), 412-418.
http://dx.doi.org/http://dx.doi.org/10.1016/j.ecns.2014.03.005
Hayden, J. K., Smiley, R. A., Alexander, M., Kardong-Edgren, S., & Jeffries, P. R. (2014, July).
The NCSBN national simulation study: A longitudinal, randomized, controlled study
replacing clinical hours with simulation in prelicensure nursing education [Supplement].
Journal of Nursing Regulation, 5(2), S1-S64. Retrieved from
https://www.ncsbn.org/JNR_Simulation_Supplement.pdf
Howard, B. J. (2013). Computer-based versus high-fidelity mannequin simulation in developing
clinical judgement in nursing education (Doctoral dissertation). Retrieved from proquest
Johnson, A., Ramos-Alarilla, J., Harilal, K., Case, D., & Dillon, E. (2012). HPS more effective
than CD-ROM for improving cognition and performance. Clinical Simulation in Nursing,
8, e443-e449. http://dx.doi.org/10.1016/j.ecns.2011.04.006
AN INTEGRATIVE LITERATURE REVIEW: IN HEALTHCARE
17
Johnson, M. P., Hickey, K. T., Scopa-Goldman, J., Andrews, T., Boerem, P., Covec, M., &
Larson, E. (2014). Manikin versus web-based simulation for advanced practice nursing
students. Clinical Simulation in Nursing, 10(10), e317-323.
http://dx.doi.org/http://dx.doi.org/10.1016/j.ecns.2014.02.004
Kolowich, S. (2010). Online cure for the nursing crisis. Retrieved from
www.insidehighered.com/news/2010/02/02/nursing
Liaw, S. Y., Chan, S. W., Chen, F. G., Hooi, S. C., & Siau, C. (2014). Comparison of virtual
patient simulation with mannequin-based simulation for improving clinical performance
in assessing and managing clinical deterioration: Randomized controlled trial. Journal of
Medical Internet Research, 16(9), e214. http://dx.doi.org/10.2196/jmir.3322
Melnyk, B. M., & Fineout-Overholt, E. (2011). Evidence-based practice in nursing and
healthcare: A guide to best practice. Philadelphia, PA: Lippincott, Williams & Wilkins.
National League for Nursing Response to NCSBN Simulation Study. (n.d.). Retrieved from
http://www.nln.org/facultyprograms/NCSBNStudyResponsefinal.pdf
Quelly, S. (2007). Determining quality and validity of findings. Unpublished manuscript.
Retrieved from
https://webcourses.ucf.edu/courses/1083521/files/42186842?module_item_id=8004827
Smith, P. C., & Hamilton, B. K. (2015). The effects of virtual reality simulation as a teaching
strategy for skills preparation in nursing students. Clinical Simulation in Nursing, 11, 5258. http://dx.doi.org/dx.doi.org/10.1016/j.ecns.2014.10.001
Wilson, R. D., Klein, J. D., & Hagler, D. (2014, January/February). Computer-based or human
patient simulation-based case analysis: Which works better for teaching diagnostic
AN INTEGRATIVE LITERATURE REVIEW: IN HEALTHCARE
reasoning skills? Nursing Education Perspectives, 35(1), 14-18.
http://dx.doi.org/10.5480/11-515.1
Youngblood, P., Harter, P. M., Srivastava, S., Moffett, S., Heinrichs, W. L., & Dev, P. (2008).
Design, development, and evaluation of an online virtual emergency department for
training trauma teams. Simulation in Healthcare, 3(3), 146-153.
18
AN INTEGRATIVE LITERATURE REVIEW: IN HEALTHCARE
19
Appendix A
Literature Evaluation Table
Name: Beth Shaffer
PICO Question: In healthcare education (P), are computer screen-based simulators (I) as compared to high-fidelity patient simulators
(C) as effective in relation to learning outcomes (O)?
Search Strategies: Databases: Cumulative Index of Nursing and Allied Health Literature [CINAHL] Plus, Medline, Academic Search
Premier, PsychINFO, and Education Resources Information Center [ERIC]. Key Search Terms: Simulat*, “virtual reality”, virtual
patients, nurs*, student*, undergrad, baccalaureate*, and bachelor, undergrad*, bachelor*, BSN, “virtual patient”, web-based, online,
computer*, mannequin, manikin, “human patient”, didactic*, instruct*, teach, healthcare and eval*. Years included: Years 2008present. Limits: Peer-reviewed and written in the English language.
Search Outcome: A total of 114 studies were found and after review nine met inclusion criteria. The remaining articles were
excluded due to not directly measuring the elements in the PICO question.
Literature Evaluation Table
Citation
Arnold et al.
(2013)
United States
Patient Group
and Sample
Size
N=33
RN’s
Group1: n= 9
(Low Fidelity
Simulation)
Group 2: n= 9
Study Design
and Level of
Evidence
Randomized,
Experimental,
pretest-posttest,
control-group
design
Level II
(Melnyk &
Outcome
Variables
Key Results
DATA
Knowledge
scores similar,
computer-based
simulation rated
least satisfied
on the Student
satisfaction and
self-confidence
The computer-based group
(CI: 80 [71,89])) had a
higher posttest score than
the high-fidelity group (CI:
76 [68,84]). The highfidelity group (CI 54 [48,
60]) had a higher
confidence score than the
Themes
Validity
Level 3
(High
quality)
(Quelly,
2007)
Clinical
Performance
No difference (p =
.44)
Knowledge
AN INTEGRATIVE LITERATURE REVIEW: IN HEALTHCARE
(Computerbased
Simulation)
FineoutOverholt, 2011)
in learning
(SSSL)
Group 3: n= 10
(High-fidelity
Simulation)
Cooper et al.
(2015)
Australia
N= 427
Phase 1: n= 97
(LaboratoryBased
Simulation)
Phase 2: n=
330 (WebBased esimulation)
Final year
Nursing
Students
Quasiexperimental
design;
convenience
sample
Level III
(Melnyk &
FineoutOverholt, 2011)
Moderate
clinical
performance for
both groups;
improvement in
skills for both
groups; higher
satisfaction
levels with
face-to-face
group
20
computer-based group (CI
52 [46, 58]). The
computer-based group ( CI
3.6 [3.4, 3.8]) had a
significantly lower overall
mean SSSL score than the
high-fidelity group ( CI 4.7
[4.5, 4.9]).
Phase 2-web-base groupsmall effect around 9%
improvement on clinical
knowledge.
Phase 1 face-to-face
performance criteria
(49%); Phase 2 Web-based
(69%).
Skill gain (knowledge,
confidence, and
competence)- significantly
improved (face-to-face, p =
.000; Web based, p = .000)
total mean gains 6.39 for
face –to-face and 4.73 for
Web-based
Satisfaction
High fidelity group higher
than computer group ( p =
No difference
( p < .001)
Satisfaction
Computer group
lower than high
fidelity group
(p < .001)
Level 2
(Moderat
e quality)
(Quelly,
2007)
Clinical
Performance
Computer group
higher (69% of
performance
criteria) vs High
fidelity group
(49% of
performance
criteria)
Knowledge
Significant
improvement to
computer group (p
= .000) Effect size
small- 9%
AN INTEGRATIVE LITERATURE REVIEW: IN HEALTHCARE
21
.000)
Effect size was small.
High fidelity group Mean=
33.15; SD=2.26
Computer group Mean=
32.05; SD= 3,70
CI= 0.48- 1.71
Howard (2013) N = 47
n = 22
United States
(High-fidelity
mannequin
simulation)
n = 25
(Computerbased
simulation)
Johnson et al.
(2014)
United States
Second-year
associate
degree nursing
students
N = 32
APN students
without prior
Quasiexperimental,
randomized
convenience
sample
improvement.
High fidelity group
not measured.
Satisfaction
High fidelity group
higher than
computer group
( p ..000) Effect
size-small
No differences
in clinical
judgment
between groups
Mean scores for
Mannequin group was
125.84 and Computer
group was 131.63.
Level 2
(Moderat
e quality)
(Quelly,
2007)
Clinical
Performance
Both with
improved
clinical
performance
Both groups showed
significant improvement in
observed performance after
training (manikin, 52% vs.
Level 3
(High
quality)
(Quelly,
Clinical
Performance
No difference
(p > .05)
Level II
(Melnyk &
FineoutOverholt, 2011)
Quasiexperimental;
pretest, posttest;
Randomized,
High fidelity group
AN INTEGRATIVE LITERATURE REVIEW: IN HEALTHCARE
clinical
rotations
convenience
sample
Level II
(Melnyk &
FineoutOverholt, 2011)
Johnson et al.
(2012)
United States
N= 119
n = 35
(HPS)
n = 45
(CD-ROM)
n = 39
(Control group)
U.S. Army
Licensed
Practical Nurse
Prospective,
pretest-posttest,
experimental,
mixed design
with
randomization;
convenience
sample
Level II
(Melnyk &
FineoutOverholt, 2011)
and selfassessed
knowledge;
Manikin group
scored higher in
observed
performance
HPS provided
more “realism”
than the CDROM group;
HPS was
superior in the
CP (combat
performance)
instrument
which relates to
“realism”
22
70%; p < 0.001; Web, 51%
vs. 63%; p < 0.001)
Self-assessed knowledge
(Manikin, 54% vs. 79%; p
< 0.001; Web, 59% vs.
76%; p < 0.02)
Manikin group
significantly improved
scores on self-assessment
of practice ability after
training (47% vs. 75%; p =
0.001)
Post training observed
performance- Manikin
scored significantly higher
than Web (70% vs. 63%; p
=0.02)
The HPS group had
significantly higher scores
than the CD-ROM group
and control group (p<.05).
The CD-ROM group had
significantly better scores
than the control group
(p<.05)
2007)
higher than
computer group ( p
= .02)
Knowledge
(Self assessment)
No difference
(p = .02)
Level 3
(High
quality)
(Quelly,
2007)
Clinical
Performance
High fidelity group
higher than
computer group ( p
= .000)
Knowledge
High fidelity group
scored higher than
computer group
(p = .037)
AN INTEGRATIVE LITERATURE REVIEW: IN HEALTHCARE
23
Course
Liaw et al.
(2014)
Singapore
N = 61
n = 31
Experiment
Group (Virtual
Patient
Simulation)
n = 30
Control Group
(Mannequin
Simulation)
High fidelity group
Mean- 58.96
Computer group
Mean- 49.22
Prospective,
randomized
controlled trial
with pretestposttest design
Level II
(Melnyk &
FineoutOverholt, 2011)
Posttest scores
with Virtual
group
decreased;
Virtual group
participant’s
satisfied with
simulation,
quality and
information.
First posttest scores from
pretest scores were both
with a significant increase
with virtual simulation (P
<.0001) and Mannequin
(P<.05).
Second posttest scores for
the Virtual group
decreased significantly
((P<.05). No significant
difference (P=.94) for the
mannequin group.
Level 3
(High
quality)
(Quelly,
2007)
Clinical
Performance
No difference
(Between first and
second posttests)
( p = .17)
No difference
(Over time
between groups)
(p = .12)
Satisfaction
Smith et al.
(2015)
N = 20
n = 10
After-only
experimental
design
Control group
No statistically significant
spent more time difference between groups.
practicing; all
The EG’s visual analog
Level 2
(Moderat
e quality)
Computer group
satisfied- rated
6.06 out possible
7-point scale.
High fidelity group
satisfaction not
rated.
Clinical
Performance
AN INTEGRATIVE LITERATURE REVIEW: IN HEALTHCARE
United States
Experiment
Group (Virtual
Reality –VR)
n = 10
Control Group
(non human
models)
Wilson et al.
(2014)
ADN nursing
students
N = 54
BSN students
United States
Youngblood et
al. (2008)
United States
N = 30
Virtual ED
simulator group
(n = 16)
PS simulator
Level II
(Melnyk &
FineoutOverholt, 2011)
Quasiexperimental
crossover
design with
random
assignment
within intact
groups
Level II
(Melnyk &
FineoutOverholt, 2011)
Pilot study,
Prospective,
pretest-posttest,
Randomized,
experimental
design
Level II
24
participants in
EG completed
the skill on the
first attempt,
whereas 9 out
of 10 completed
in CG on first
attempt
score (92.85) was greater
than the CG’s ((86.10) and
the EG’s overall
performance score grading
tool score (94,92) was
greater than the CG’s
(92.77).
The CG spent more time
practicing in minutes than
the EG.
(Quelly,
2007)
No difference
(p = .054)
Performance
better in HPS
The HPS group had
significantly better
performance compared to
the CBS group, t(47)= 4.35; p < .001.
Level 3
(High
quality)
(Quelly,
2007)
Clinical
Performance
Both Virtual ED or PS
system showed significant
improvement in
performance between
pretest and posttest cases
(P<0.05) The EMCRM
rating scale had an internal
Level 2
(Moderat
e
quality)
(Quelly,
2007)
Clinical
Performance
Improvement in
performance
between pretest
and posttest
scores with
both modalities;
users satisfied
High fidelity group
higher than
computer group
(p < .001)
No difference
(P = .40)
AN INTEGRATIVE LITERATURE REVIEW: IN HEALTHCARE
group (n = 14).
Medical
students
(Melnyk &
FineoutOverholt, 2011)
with both
simulations
25
consistency of 0.96 and an
interrater reliability of the
rating scale was 0.71
Satisfaction
High fidelity group
(100%) marginally
higher than
computer group
(94%)
AN INTEGRATIVE LITERATURE REVIEW: IN HEALTHCARE
Appendix B
Modified Strength of Recommendation Taxonomy (SORT) Table to Education in
Healthcare (Table 2)
Strength of
Recommendation
A
B
C
Definition
Recommendation based on consistent and good quality studentoriented evidence *
Recommendation based on inconsistent or limited quality studentoriented evidence *
Recommendation based on consensus, usual practice, opinion, and
instructor preferred methods.
* Student-oriented evidence measures knowledge, skills, clinical performance,
satisfaction, and student confidence.
See Figure 1 below for modified algorithm for determining strength of recommendation
in healthcare education.
26
AN INTEGRATIVE LITERATURE REVIEW: IN HEALTHCARE
27
Figure 1.
Modified algorithm for determining Strength of Recommendation in healthcare
education
Is this a key recommendation for
educators regarding use as an
instructor approach that merits a
label?
No
Strength of
Recommendation
not needed
Yes
Is the recommendation based on
student-orient evidence (i.e. an
improvement in knowledge, skills,
clinical performance, satisfaction,
student confidence)





No
Strength of
Recommendation =
c
Yes
Is the recommendation based
on consensus, usual practice,
opinion, and instructor preferred
methods.
Yes
No
Is the recommendation based on
one of the following:
• National League for Nursing
(NLN) recommendation
• American Association of
Colleges of Nursing
(AACN) recommendation
• Consistent findings from at least
two good quality
randomized controlled
trials or a systematic
review/meta-analysis of
same
• Consistent findings from at least
two good quality cohort
studies or systematic
review/meta-analysis of
same
Yes
Strength of
Recommendation = A
No
Strength of
Recommendation
=B