Assessing clinical decision
making skills using an
interactive online tool.
Verina Waights & Ali Wyllie
of Health and Social Care
Fellows
Faculty
COLMSCT CETL
OU Conference ‘Making Connections’ (30th April 2008)
Quality eLearning strand
Outline
Context of Project
Overview principles of quality eLearning experience
Outline characteristics of CDM tool
Demo CDM
Findings to date
Looking to the future
2
Background
Nurses make clinical decisions in response to changes in each
patient's condition, which may occur within very small time-frames
The ability to make clinical judgements depends on both a sound
theoretical background and good decision-making skills
Clinical decisions are strongly influenced by the context in which
they are made (Bucknall 2000)
As the complexity of decision tasks increases, so context-specific
knowledge determines the effectiveness of decision-making (Botti
& Reeve 2003)
3
Aims
• to develop a web-based tool to provide decision-making
opportunities, building on Laurillard (2002): who suggests that:
‘Traditional modes of assessment of knowledge are seen as
inadequate because they fail to assess students‘ capability in the
authentic activities of their discipline’.
• to assess nursing students’ decision-making skills in a contextdriven virtual environment.
4
What makes quality eLearning
Boud and Prosser’s (2002) principles for designing a high
quality learning environment suggests learning activities
should:
•
support learner engagement - build on prior knowledge
•
acknowledge the learning context – integral with study
•
seek to challenge learners – active participation
•
provide practice – through individual and/or group activities
5
Principles of good feedback
practice
Nicol and Milligan (2007:72) seven principles of good feedback
practice. Principle 6: Good feedback practice ‘provides
opportunities to close the gap between current and desired
performance’.
Boud’s (2000:158) arguments about closing the gap can be
viewed in 2 ways:
•
It is about supporting students in the act of production of a
piece of work
•
It is about providing opportunities to repeat the ‘task –
performance – feedback cycle’.
6
Pedagogical approach
Based on principles of case-based learning design and
experiential learning (Kolb 1984).
Students’ options are scaffolded
Learning is student - led and asynchronous
Feedback is contexual, individualised to student’s learning
journey and timely
Intention is to assist students to practice in risk-free
environment until ready to practice in work setting.
7
Research methods
Students’ perceptions of their decision-making skills and
efficacy of tool will be evaluated through:
• Individual completion of an online questionnaire to enable
quantitative and qualitative evaluation
• Qualitative output from video observation and ‘Think aloud’
techniques with students’ interacting in the IET Userlab
8
Case study: patient with a leg ulcer
Realistic: case prepared in collaboration with a practice nurse
to mimic reality
Relevant: Wound healing and clinical decision-making are
integral components of Nursing Programme of study
Ethical: consent obtained from the patient and anonymity
assured
Context: Miss Alice Phelps, an independent 80 year old lady
with an ulcer on her left lower leg.
9
Case study – patient with leg ulcer
10
Narrative
•
The story evolves over a series of consultations depending
on the treatment the patient receives.
•
At each stage students consult resources to find out about
their patient’s condition then determine the best treatment
from 3 available options. Their decisions result in the
patient progressing through successive consultations until
an end point is reached.
•
They are encouraged to have a second attempt at the
maze if their first attempt is unsuccessful.
•
Images of the consultations and patient’s final outcome
create a visual narrative within the maze
11
Practice Resources
The students have a range of media resources to consult:

Audios of consultations between the nurse and patient

Photos of leg ulcers

Instruction sheets on possible treatments

Patient charts and records

Patient records

Laboratory reports

Links to the BNF (British National Formulary)
12
Reflective Log
The Reflective Log captures each decision point and the
student’s reasons for their choice.
At the end of each attempt the student can review their
pathway and reflect on their decisions and reasoning.
The final attempt additionally provides author feedback on
decision-making at each stage.
13
Example Reflective log
14
Scoring
Scoring at each decision point depends on the type of decision
as outlined below and whether it is the first or second
attempt:
For example:
1st attempt: a 5 b 3 c 0
2nd attempt: a 3 b 2 c 0
The final score is an average of the two attempts at the maze.
15
Findings to date - score
7 students completed tool/8 students partly completed tool
Score
stayed the
same over
2 attempts
Score
increased
over 2
attempts
Average
score (%)
Number Score
decreased
of
students over 2
attempts
45 – 49
1
50 – 59
1
X
60 – 69
2
XX
2
XX
X
70 – 79
80 – 89
90 -99
1
X
16
Findings to date - questionnaire
Strongly
agree
Agree
1)Easy to use
3
2
2)Realistic
3
1
3
2
Neutral
Disagree
Strongly
disagree
1
1
Tool
1
Resources
3)Explored all
thoroughly
4)Cursory
glance only
2
5)Variety was
valuable
2
3
6)They made
learning
memorable
2
2
1
1
17
Strongly
agree
Agree
1) Easy to use
2
3
2)Useful to
reflect during
tasks
2
3
3)Useful to
discuss with
other students
2
3
4)Useful
learning tool
2
2
Neutral
Disagree
Strongly
disagree
Reflective Log
1
18
Strongly
agree
Agree
Neutral
Disagree Strongly
disagree
Learning
experience
1) New
3
information re
decisionmaking
2
2) New
2
information re
leg ulcers
2
1
3) Better
prepared for
practice
2
2
1
4) Useful for
selfassessment
4
5) Useful for
group
discussion
3
2
6) Could feed
into TMA
3
2
1
19
Very
Quite Neutral
3
1
1
4
1
Not very Not at
all
Motivating
1) Context
2) Log
3)Audios
2
3
4)Images
3
1
1
5) Texts
3
1
1
6) Linked
questions
2
3
7) Case study
2
2
Missing
data
20
Strongly Agree Neutral
agree
Disagree Strongly
disagree
Decisionmaking
1) Range of
resources
reflects
practice
4
1
2) Useful in
3
other practice
areas
1
1
3) Useful to
2
have another
attempt after
‘real’ practice
experience
2
1
21
Summary to date
Students felt using tool:• Increased their confidence in decision-making and caring
for leg ulcers – support learner engagement
• would assist in their practice – acknowledge the learning
context
• Positive, motivating experience – challenge learners
• Developed their knowledge and skills - provide practice
22
Looking toward the future
The approach piloted in this project can be applied in part or
whole in other courses, programmes or disciplines.
The tool can be combined with other VLE tools, such as online
discussion forums and voting tools, to create flexible and
collaborative learning and assessment sequences.
The Reflective Log can become part of a TMA.
Our findings are being used to inform future assessment
strategies within HSC. We are particularly interested in:
Contextualised CMA questions; Narratives; Embedded
multiple media; Case-based CMAs; Decision-Making;
Confidence-indicator tool.
23