Joanie Selman, MSN, RN
Stephen F. Austin State University School of Nursing

1. Discuss core components to a healthcare simulation.
2. Discuss the importance of outcomes evaluation and challenges to traditional assessments.
3. Discuss the importance of validity, reliability and feasibility as it relates to assessment.
4. Discuss types of assessments and their application in healthcare education.

DeWitt School of Nursing at
Stephen F. Austin State University
Nacogdoches, Texas
Designated as a Laerdal Center of
Educational Excellence

 9,000 square foot simulation lab opened Jan. 2010
* 10 room Med-Surg lab
*4 room OB/pedi lab and Nursery
*ER with 12 gurney spaces
*Assessment lab with 10 exam tables
*14-station control room

 5-3Gs
 1-SimMan
 1-SimNewB
 2-SimBaby
 17 VitalSims
 6-Virtual IVs
 Coming soon- SimMom, SimKid

Start simple.
Do not overwhelm yourself.
1.
Assessment
2.
3.
4.
Planning
Intervention
Evaluation

• Assess the learning needs of your students to build scenario objectives.
– Start broad and narrow it down.
• Look at …..
– Common disease/complication from your area.
• Choose problem that is it crucial for students know how to handle.
– School’s objectives
– Course objectives
– Unit objectives
– Objectives related to specific disease process

• Med-surg area
• Chest Pain
• Very Common chief complaint of clients
• Students must learn interventions to care for this client
• Not enough clinical time for every student to experience a client with chest pain.

• Plan the objectives for the scenario so that you can achieve the goal for your students.
• Based on objectives of
– The School
– The Course
– The Unit
– What do I want for the students to accomplish
(do) in this scenario?
Remember Students learn by DOING !

Example-Planning phase for chest pain (acute coronary syndrome) scenario
• School of nursing outcome objectives
– Graduates of the program will:
• Apply knowledge of the physical, social, and behavioral sciences in the provision of nursing care based on theory and evidence based practice.
• Course objectives
– Students will:
• Apply concepts from nursing, scientific, and humanistic disciplines to the care of an adult client in medical-surgical situations.
• Unit objectives
– Students will:
• Apply knowledge of pathophysiology of cardiovascular diseases to the plan of care for clients with cardiovascular disease, and vascular disorders.
• Formulate an appropriate nursing plan of care for clients with selected cardiovascular Disease, and vascular disorders.

Specific to this scenario:
• What do I want the students to learn by “doing” ?
• What interventions do I want to “see” the students accomplish?
• What are best-practices that the students should demonstrate?
• What do I want to see in terms of prioritization and critical thinking?

• Demonstrates infection control practices.
• Identifies signs and symptoms of ACS by proper assessment
• Identifies primary nursing dx
• Prioritizes and implements Physician Orders appropriately and safely by applying cardiac chest pain protocol
• Initiates relevant cardiac and respiratory monitoring
• Implements the "5 rights" of medication administration
• Recognizes signs and symptoms of adverse reaction to nitroglycerin tablets
• Implements therapeutic communication
• Demonstrate teaching techniques to help clients understand lifestyle choices that effect health.

• From the scenario objectives, make an assessment tool in the form of a
“check-off” list for instructors and student observers to use during simulation.

• You have your objectives
• You know what you want to see
• You know what you want the students to do

-- Remember --
Objectives guide everything
– Mannequin actions
– Vital signs, set-up
– “Patient” voice
– MD orders
– Medications
– Chart contents
– Available supplies
– Pre-Sim activities

You’re finished planning and writing
• Pre-Sim Activities/Assignment
• Running the Sim
• Debriefing
Researchers suggest that equal time be spent on each.

 Lecture on the topic
 Help students prepare
 Assignment to complete on topic before the SIM
 article in a journal
 case study

“dress rehearsal” with other students or faculty

 Keep it real. Room, mannequin, pt voice
 Pre-brief students
 act exactly like a real life setting
 give them report – like at a hospital
 Give a few minutes after report for them to plan as a team before starting scenario
 As mannequin voice give “hints” if student is really struggling.
 Give student “Observer” check-list

 Faculty facilitate the discussion.
 Guide students in reflective learning.
 Have observer use assessment tool.
 Let observer identify positives as well as negatives of the simulation activity.
 Constructive feedback is important.
 Redirect students away from poor choices.
 Praise students for positive choices.
 Group discusses application of scenario to clinical practice

• Did the students meet the objectives?
• Did this teaching strategy work?
• How do I assess their learning?
Faculty Evaluation of students learning
Traditional way of evaluation is an exam.
Simulation necessitates non-traditional evaluation.

Use the assessment tool of the instructor and the observer as a tool to measure the student’s accomplishment of the objectives of the
SIM.

 The assessment tool has internal validity….
 it was created from the scenario objectives, therefore it is measuring what you intend to measure
 The assessment tool has external validity…
 it is based on real world interventions and expectations of nurses in specific medical occurrences.

• The assessment tool has reliability.
– It is consistent
• From group to group
• From observer to observer
• From instructor to observer
Constant, Repeatable and Objective
Evaluation

Assessment tool items can be analyzed and data generated for a quantitative evaluation “view”
• For example:
– 90 % introduced themselves to the patient
– 92 % instructed patient to take NTG under tongue
– 85% assessed blood pressure before giving NTG
Beneficial to instructor to identify gaps in learning

• Evaluation of Student’s perception of the scenario and learning experience.
– Use surveys for the students to complete soon after the SIM.
– Short, concise and easy to answer.

Question
Disagree No opinion Agree
1. I understand the purpose and objectives of the simulation.
0.00%
2. The scenario(s) resembled a real-life situation.
1.33%
3. The simulation provided a variety of ways to learn the material. 0.00%
0.00%
0.00%
5.33%
100.00%
98.67%
94.67%
4. I had a chance to work with my peers during the simulation.
0.00%
5. The simulation helped me learn to prioritize.
6. The debriefing session was helpful as a learning activity.
7. I was able to participate in the debriefing session.
8. The feedback was constructive.
0.00%
1.33%
1.33%
0.00%
9. The simulation will help me better care for healthcare clients.
0.00%
10. The simulation made me feel more confident in caring for healthcare clients.
2.67%
5.33%
1.33%
0.00%
1.33%
2.67%
0.00%
4.00%
94.67%
98.67%
98.67%
97.33%
97.33%
100.00%
93.33%

• Have a free text comment section on the evaluation.
Actual student comments…….
“I really enjoy SIMs and I always learn a lot from them. I wish we had more and I will continue to learn and feel more comfortable when entering a patient's room.”
“I always learn things that I know I won’t forget. It is good to have these simulations because we don't always come across some of these situations when we are actually in clinicals. Because of these simulations, I know i will be prepared when I come across the same problems.”
“The feedback in the debriefing session was very informative without being condemning. Good job!”

Feasibility
Reasonable to use SIMs?
 Cons:
 Mannequins costly
 Labor intensive for faculty
 Instructor training needed
 Space needed
 Pros:
 Helps with shortage of clinical opportunities
 Reproducibility of clinical experience
 Great teaching tool for Tactile/Kinesthetic learners
 Opportunity for students to gain confidence
 Encourages teamwork
 Prepares students for the workforce

With current expectations of students, school administrators and employers,
Feasible or not…
Simulations are the future and are needed to help fulfill these expectations.

– To the instructor
– To the students
– To the future employer
– To the client

Contact information
Joanie Selman, MSN, RN selmand@sfasu.edu
936-674-7896
DeWitt School of Nursing
Stephen F. Austin State University
Nacogdoches, Texas