Assessing clinical decision
making skills using an online
interactive tool
Ali Wyllie & Verina Waights
Faculty of Health and Social Care
COLMSCT CETL Fellows
Open CETL Conference: Building bridges
24-25th September 2008
Centre for Open Learning of Mathematics, Science, Computing and Technology (COLMSCT)
Outline
Context of Project
Quality eLearning experience
Outline characteristics of CDM tool
Demo CDM
Findings to date
Looking to the future
Centre for Open Learning of Mathematics, Science, Computing and Technology (COLMSCT)
2
Background
Nurses make clinical decisions in response to changes in each
patient's condition, which may occur within very small time-frames
The ability to make clinical judgements depends on both a sound
theoretical background and good decision-making skills
Clinical decisions are strongly influenced by the context in which
they are made (Bucknall 2000)
As the complexity of decision tasks increases, so context-specific
knowledge determines the effectiveness of decision-making (Botti
& Reeve 2003)
Centre for Open Learning of Mathematics, Science, Computing and Technology (COLMSCT)
3
Aims
to develop a web-based tool to provide decision-making
opportunities, building on Laurillard (2002): who suggests that:
‘Traditional modes of assessment of knowledge are seen as
inadequate because they fail to assess students‘ capability in the
authentic activities of their discipline’.
to assess nursing students’ decision-making skills in a contextdriven virtual environment.
Centre for Open Learning of Mathematics, Science, Computing and Technology (COLMSCT)
4
Quality eLearning for learner
enagagement
Boud and Prosser’s (2002) principles for designing a high
quality learning environment suggests learning activities
should:
support learner engagement - build on prior knowledge
acknowledge the learning context – integral with study
seek to challenge learners – active participation
provide practice – through individual and/or group activities
Centre for Open Learning of Mathematics, Science, Computing and Technology (COLMSCT)
5
Pedagogical approach
Based on principles of case-based learning design and
experiential learning (Kolb 1984).
Students’ options are scaffolded
Learning is student - led and asynchronous
Feedback is contexual, individualised to student’s learning
journey and timely (Nicol and Milligan 2007: 72, Principle 6)
Intention is to assist students to practice in risk-free
environment until ready to practice in work setting.
Centre for Open Learning of Mathematics, Science, Computing and Technology (COLMSCT)
6
Research methods
Students’ perceptions of their decision-making skills and
efficacy of the tool will be evaluated through:
• Individual completion of an online questionnaire to
enable quantitative and qualitative evaluation
• Qualitative output from video observation and ‘Think
aloud’ techniques with students’ interacting in the IET
Userlab
Centre for Open Learning of Mathematics, Science, Computing and Technology (COLMSCT)
7
Case study: patient with a leg ulcer
Context: Miss Alice Phelps, a (fictitious) 80 year old lady
with an ulcer on her left lower leg.
Realistic: case prepared in collaboration with a practice
nurse to mimic reality
Ethical: consent obtained from the ‘real’ patient and
anonymity assured
Relevant: Wound healing and clinical decision-making are
integral components of Nursing Programme of study.
Centre for Open Learning of Mathematics, Science, Computing and Technology (COLMSCT)
8
Case study – patient with leg ulcer
Centre for Open Learning of Mathematics, Science, Computing and Technology (COLMSCT)
9
Narrative
The story evolves over a series of consultations depending on
the treatment the patient receives.
At each stage students consult resources to find out about
their patient’s condition.
They determine the best treatment from 3 available options.
Their decisions result in the patient progressing through
successive consultations until an end point is reached.
They are encouraged to have a second attempt at the maze if
their first attempt is unsuccessful.
Images of the consultations and patient’s final outcome create
a visual narrative within the maze
Centre for Open Learning of Mathematics, Science, Computing and Technology (COLMSCT)
10
Practice Resources
The students have a range of media resources to consult:

Audios of consultations between the nurse and patient

Photos of leg ulcers

Instruction sheets on possible treatments

Patient charts and records

Patient records

Laboratory reports

Links to the BNF (British National Formulary)
Centre for Open Learning of Mathematics, Science, Computing and Technology (COLMSCT)
11
Reflective Log
The Reflective Log captures each decision point and the
student’s reasons for their choice.
At the end of each attempt the student can review their
pathway and reflect on their decisions and reasoning.
The final attempt additionally provides author feedback on
decision-making at each stage.
Centre for Open Learning of Mathematics, Science, Computing and Technology (COLMSCT)
12
Example Feedback in Reflective log
Reflective log
Miss Alice Phelps, a lively and independent 80 year old lady came to the surgery on
17th February with an ulcer on her left lower leg. How would you help her?
You chose: A. Take a swab and apply a viscopaste dressing
Why: The wound assessment chart indicates a venous ulcer
Feedback: The correct decision here would be to apply a viscopaste dressing and
take a swab to check for infection. It would also be an acceptable decision to apply a
silver dressing to prevent infection. Algae-based dressings are unsuitable in this
instance because they are used to absorb exudates and Miss Alice Phelps’ ulcer has
only low levels of exudate.
Miss Alice Phelps has returned to your clinic.
You chose: A. Prescribe penicillin and apply a viscopaste dressing
Why: It needs an antibiotic and penicillin is broad spectrum.
Feedback: The correct decision here would be to prescribe erythromycin and apply a
viscopaste dressing as there are bacteria present in the ulcer. Although S.aureus is
sensitive to penicillin as well as to erythromycin, you should not prescribe penicillin to
Miss Phelps because her records show that she is allergic to penicillin.
Centre for Open Learning of Mathematics, Science, Computing and Technology (COLMSCT)
13
Scoring
Scoring at each decision point depends on the type of decision as
outlined below and whether it is the first or second attempt:
For example:
1st attempt: a 5 b 3 c 0
2nd attempt: a 3 b 2 c 0
The final score is an average of the two attempts at the maze.
Centre for Open Learning of Mathematics, Science, Computing and Technology (COLMSCT)
14
Demo
http://learn.open.ac.uk/mod/resourcepage/view.php?id=79337
&superfluous=357
Centre for Open Learning of Mathematics, Science, Computing and Technology (COLMSCT)
15
Findings to date - score
11 students completed tool/27 students partly completed tool
Average
score (%)
Number Score
of
decreased
students over 2
attempts
40 - 49
1
50 -59
2
60 -69
3
Score
stayed the
same over
2 attempts
Score
increased
over 2
attempts
X
XX
X
XX
70 -79
80 -89
2
90-99
3
XX
X
XX
Centre for Open Learning of Mathematics, Science, Computing and Technology (COLMSCT)
16
Findings to date - questionnaire
Strongly
agree
Agree
1)Easy to use
5
2
2)Realistic
4
3
6
2
Neutral
Disagree
Strongly
disagree
2
1
Tool
1
Resources
3)Explored all
thoroughly
4
4)Cursory
glance only
5)Variety was
valuable
3
5
6)They made
learning
memorable
3
4
1
1
Centre for Open Learning of Mathematics, Science, Computing and Technology (COLMSCT)
17
Strongly
agree
Agree
Neutral
1) Easy to use
4
4
2)Useful to
reflect during
tasks
2
6
3)Useful to
discuss with
other students
2
4
2
4)Useful
learning tool
2
4
2
Disagree
Strongly
disagree
Reflective Log
Centre for Open Learning of Mathematics, Science, Computing and Technology (COLMSCT)
18
Strongly Agree Neutral
agree
Disagree Strongly
disagree
Learning
experience
4
1) New
information re
decisionmaking
4
2
2) New
information re
leg ulcers
5
1
3) Better
prepared for
practice
3
4
1
4) Useful for
selfassessment
5
2
1
5) Useful for
group
discussion
3
4
1
6) Could feed
into TMA
4
2
2
Centre for Open Learning of Mathematics, Science, Computing and Technology (COLMSCT)
19
Very
Quite Neutral
1) Context
4
3
1
2) Log
1
6
1
3)Audios
3
3
4)Images
6
1
1
5) Texts
6
1
1
6) Linked
questions
3
5
7) Case study
3
4
Not very Not at
all
Motivating
Centre for Open Learning of Mathematics, Science, Computing and Technology (COLMSCT)
20
Strongly Agree Neutral
agree
Disagree Strongly
disagree
Decisionmaking
5
3
4
2) Useful in
other practice
areas
3
1
3
3) Useful to
have another
attempt after
‘real’ practice
experience
4
1
1) Range of
resources
reflects
practice
Centre for Open Learning of Mathematics, Science, Computing and Technology (COLMSCT)
21
Students felt using CDM tool:
Increased their confidence in decision-making and caring for
leg ulcers - support learner engagement
‘a very helpful tool in confidence-building’
‘helped a great deal - more please!’
Would assist in their practice - acknowledge the learning
context
‘I found that it made you think about what you are doing for
the patient’.
Seek to challenge learners – active participation
‘very keen to expand my new skills, these questions taxed
me!’
Centre for Open Learning of Mathematics, Science, Computing and Technology (COLMSCT)
22
Looking toward the future
The approach piloted in this project can be applied in part or
whole in other courses, programmes or disciplines.
The tool can be combined with other VLE tools, such as online
discussion forums and voting tools, to create flexible and
collaborative learning and assessment sequences.
The Reflective Log can become part of a TMA.
Our findings are being used to inform future assessment
strategies within HSC. We are particularly interested in:
Contextualised CMA questions; Narratives; Embedded
multiple media; Case-based CMAs; Decision-Making;
Centre for Open Learning of Mathematics, Science, Computing and Technology (COLMSCT)
23
Contact
If you are interested in our work we can be contacted on:
Ali Wyllie (a.j.wyllie@open.ac.uk)
Verina Waights (v.waights@open.ac.uk)
Centre for Open Learning of Mathematics, Science, Computing and Technology (COLMSCT)
24