Simulating healthcare practice
for assessment – a case study
approach for iCMA
development
Dr Verina Waights
Lecturer, Faculty of Health and Social Care
COLMSCT CETL Teaching Fellow
The Open University
OpenCetl 2009
eAssessment in higher education
• primarily use multiple choice (Slater 2007)
• may be limited when assessing learning in complex
professional situations
‘Traditional modes of assessment of knowledge are seen as
inadequate because they fail to assess students‘ capability
in the authentic activities of their discipline.’
Laurillard (2002)
2
Background
Nurses make clinical decisions in response to changes in each
patient's condition, which may occur within very small time-frames
The ability to make clinical judgements depends on both a sound
theoretical background and good decision-making skills, which are
strongly influenced by their context (Bucknall 2000)
As the complexity of decision tasks increases, so context-specific
knowledge determines the effectiveness of decision-making (Botti
& Reeve 2003)
Healthcare professionals need to develop clinical decision-making
skills without compromising patient safety.
3
Aims
• develop a web-based tool to provide decision-making
opportunities, building on Laurillard (2002): who suggests that:
‘Traditional modes of assessment of knowledge are seen as
inadequate because they fail to assess students‘ capability in the
authentic activities of their discipline’.
• help student nurses develop practice-based decision-making
skills in a context-driven virtual environment
• assess nursing students’ decision-making skills
• provide feedback on their progress that is contexual,
individualised to each student’s learning journey and timely
4
Research methods
Students’ perceptions of their decision-making skills and
efficacy of tool will be evaluated through:
• Individual completion of an online questionnaire to enable
quantitative and qualitative evaluation
• Assessment of their success after two attempts in using the
tool
5
Case study
Context: Miss Alice Phelps*, an independent 80 year old lady
with an ulcer on her left lower leg.
Realistic: case prepared in collaboration with a practice nurse
in a GP surgery to ensure validity and mimic reality
Relevant: Wound healing and clinical decision-making are
integral components of Nursing Pre-registration programme
of study
Ethical: consent obtained from the patient and anonymity
assured
*pseudonym
6
Case study
7
Example Reflective log
8
Scoring
Scoring at each decision point depends on the type of decision
as outlined below and whether it is the first or second
attempt at using the tool:
e.g. 1st attempt: a 5 b 3 c 0
2nd attempt: a 3 b 2 c 0
The final score is an average of the two attempts at the tool.
9
Findings - score
11 students completed tool/33 students partly completed
tool. (73%) improved across two attempts.
Average
score (%)
Number Score
of
decreased
students over 2
attempts
40 - 49
1
50 -59
2
60 -69
3
Score
stayed the
same over
2 attempts
Score
increased
over 2
attempts
X
XX
X
XX
70 -79
80 -89
2
90-99
3
XX
X
XX
10
Findings to date - questionnaire
Strongly
agree
Agree
Neutral
9
5
1
12
Resources
Explored thoroughly / variety
valuable / made learning
memorable
15
2
10
Reflective Log
Easy to use / Useful to reflect
during tasks / useful to
discuss with other students /
useful learning tool
18
4
Tool
Easy to use / realistic
Disagree
Strongly
disagree
2
1
11
Finding to date - questionnaire
Strongly
agree
Agree
Neutral
12
8
4
17
Learning experience
New information / Better
prepared for practice / Useful
for self-assessment / Useful
for group discussion
19
4
12
10
2
Confidence
Improved across task /
Feedback useful and
informative / More confident
in practice
Decision-making
Range of resources reflects
practice / Useful in other
practice areas / Useful to
have another attempt after
‘real’ practice experience
Disagree
Strongly
disagree
12
Students felt using tool:• Increased their confidence in decision-making and caring
for leg ulcers - support learner engagement
‘a very helpful tool in confidence-building’
‘helped a great deal - more please!’
• Would assist in their practice - acknowledge the learning
context
‘‘it made you think about what you were doing for
the patient’
13
Students found using tool:• Positive, motivating experience - challenge learners
‘very keen to expand my new and developing
skills, these questions taxed me!’
Developed their knowledge and skills - provide practice
‘in care planning and prioritising care on the ward’
‘this tool would be very useful in every
clinical/practice setting’
14
Question types in Moodle
• Single response
• Multiple response
• Numeric
• Short answer
• True/False
15
16
17
18
19
20
21
22
23
24
25
Case Study – Agnes Henderson’s
story
Biographical details/ Health information given as narrative
Agnes’ story
1-6
Agnes’ condition deteriorates
7-11
Agnes’ continuing care
13 – 15
End of the story
26
Challenges
•
Limited number of question types
•
Incorporating flash media
•
Opening links in new windows
•
Developing LTS processes
•
Developing iCMA guidance notes
•
Time heavy initially
27
Opportunities
• Follow patient/client over time
• Mimic reality of practice
•Assess range of skills/knowledge within each question
• Rehearse skills before going into practice
• Creative and fun to develop
28