GPSim
A report on a pilot study into the use of
Virtual Patients for GP Continuing
Professional Development.
Prepared for: Avon Somerset and Wiltshire Cancer Services
Author:
Simon Messer
Principal Lecturer,
University of the West of England
Date 30th June 2012
Table of Contents
1.0
Introduction ...........................................................................................
2
2.0
Results...................................................................................................
3
2.1
Demographic Data.................................................................................
4
2.2
Opinions of the usefulness of the tool....................................................
4
2.3
Opinions on e-learning and the use of Virtual Patients..........................
11
2.4
Additional open comments .....................................................................
16
3.0
Conclusion ..............................................................................................
22
1
1.0 Introduction
Gpsim is a simulation focussed on providing Continuing Professional Development
(CPD) for GPs around early signs of cancer. The innovation comes from using a set
of Virtual Patients that provide an engaging online, multimedia and interactive
experience. Commissioned by the Avon Somerset & Wiltshire Cancer services
(ASWCs) network and authored by GPs, the cases are focussed on increasing
awareness of early signs for survival as well as the support available to GPs from the
network. The cases are presented as branched systems that offer the learner
various pathways and outcomes depending upon the decisions that they make. The
learner is able to engage in a conversation with each patient, represented as an
avatar, as well as access pertinent data such as medical records to inform their
decisions. The patient set consists of mix of cases, some with early signs of cancer
and some without so that the consultation is not a foregone outcome.
The strategic drivers for developing online resources for CPD are well documented.
This idea arose from reviewing an example of existing provision for CPD around
early cancer signs that was considered very two dimensional (text only) and also
predictive in that the learner was cued to an early signs of cancer outcome. It was
conjectured that creating a simulation situated in primary care using a set of virtual
patients, would improve the learning experience. Its inception was coincident with
the National Awareness and Early Diagnosis Initiative (NAEDI) and in particular, The
"Signs for Survival" (S4S) project run by ASWCs. The project was subsequently
commissioned as part of S4S.
In all, six virtual patient cases were created, including two non-cancer cases, which
were presented to participants in two groups of three containing two cancer and one
placebo cases. Participants in the pilot were recruited by ASWCs as part of the S4S
project from practices across the network region. In total, seventy two GPs
participated in the pilot, working through three cases. Evaluation was conducted
using a self administered on-line questionnaire that captured both quantitative and
qualitative data. The results were evaluated using statistical and thematic analysis. It
should be noted that qualitative comments from the survey are exactly reproduced
here without correction of any spelling or syntax errors that may be presented.
2
2.0 Results
2.1 DEMOGRAPHIC DATA
There were 72 participants. Gender and age are presented in Figures 1 & 2
respectively.
Figure 1: Participant gender
Figure 2: Participant age
2.2 OPINIONS OF THE USEFULNESS OF THE TOOL
This part of the survey was intended to gauge opinion about the usefulness of the
simulation tool to increase awareness of early signs of cancer.
2.2.1 I felt that the virtual patients created for this project increased my
awareness of tools to help assess suspected cancer
The quantitative data relating to this statement are presented in Figure 3. 79.2% of
participants either strongly agreed or agreed with the statement. The emerging
theme from the qualitative data is that participants found the inclusion of the
Hamilton risk assessment tool in the simulation very useful. E.g.

I found the first case- with Hamilton Score and guidelines very interesting

I had never heard of the Hamilton assessment tools.

The Hamilton risk assessment was useful to be made aware of
3
Additional qualitative comments are presented in Table 1. Although a number of
participants were already aware of and use assessment tools, clearly, the simulation
served to increase awareness for the majority of participants, especially with respect
to the recently introduced Hamilton risk assessment tool.
Figure 3: Increase awareness of assessment tools.
Table 1: Comments










I liked the virtual patient feel.
Realistic cases MacMillan guidance was new and very useful for me
Have a personal interest in this anyway so would be hard to increase my awareness
It was useful to know that spirometry was available to help detect restrictive lung disease, when
often we think of it looking just for obstructive disease
though I already knew of the risk tables
am aware of the 2ww guidelines for all suspected cancerns
Very useful links and resources for each question.
I already use the tool
Re enforeced my learning need and read up the assessment tools and NICE guidelines again,
so very helpful.
the supporting information in the pages was useful to read through. Perhaps a 'worked
example' of using the Hamilton rules?
4
2.2.2 I felt that the simulation tool increased my confidence in referring
patients as TWW
The quantitative data in response to this statement are presented in Figure 4. 63.4%
of respondents either strongly agreed or agreed with the statement.
Figure 4: Increased confidence in referring as 2WW.
Table 2: increased confidence in referring patients as TWW










I think it increased my confidence that making TWW referrals
useful at the end of such cases to clearly state 2WW guidelines
Because we are doing a survey looking at investigating cancer one assumes that all these 3
have cancer and when doing the survey are looking and expecting it to be cancer. The situation
is completely different in the surgery we see a large number of people with cough, headache,
mild anaemia and you know when you see them in surgery that they are not very likely to have
cancer at all. Even the tools are not helpful- we are trying to pick up cancer at an early stage- if
the cancer has have resulted in anaemia or weight loss then its likely to be more advancedGood links to guidelines and liked the summaries of relevant guidelines given at the end of the
cases.
Very realistic simulation - did give you the sense of having a 'real patient' scenario
Useful interactive tool.
the ask the expert box was good but it wasnt clear if i should look at this initially with the
medical record, or whether it would have been better placed at the end as feedback?
Through the information and links provided
Yes raised awareness about TWW and when to refer.
If anything it slightly muddied the waters a little.
5
2.2.3 I felt that the simulation tool helped increase my awareness of the late
effects from previous cancer treatments.
The quantitative data for participants’ attitude to this statement are presented in
Figure 5. 51.4% of respondents either strongly agreed or agreed with the statement.
The median lies in the ‘disagree’ option.
Figure 5: Increased awareness of late effects from previous cancer treatments.
It is a relevant to restate here that the participants were placed into one of two
groups, each having three cases to play. Only Group 1 was exposed to a case that
had a previous history of cancer and this has affected the results. Cross-tabulation
analysis of the data is presented in Table 3.
6
Table 3: Crosstabulation analysis of group attitude to the statement: I felt that the simulation tool
helped increase my awareness of the late effects from previous cancer treatments.
Count
Answer
Neither Agree
Strongly agree
Group
Agree
nor disagree
Disagree
Strongly disagree
Total
1
17
15
0
0
0
32
2
0
3
8
19
4
34
Un-classified
1
0
0
3
0
4
18
18
8
22
4
70
Total
Chi-Square Tests
Asymp. Sig. (2Value
df
sided)
60.115a
8
.000
Likelihood Ratio
80.630
8
.000
Linear-by-Linear
7.548
1
.006
Pearson Chi-Square
Association
N of Valid Cases
70
Four email addresses provided as part of the survey data did not link to those used
for the survey and are therefore could not be classified in a particular group. There is
a significant association between Group and attitudes expressed to the statement
with Pearson’s Chi-square being:
χ2 (8) = 60.115, p < 0.001
All those in Group 1, who were exposed to the case with previous cancer, either
strongly agreed or agreed with the statement, whilst those in Group 2 or unclassified
were either neutral or in disagreement with the statement. This is supported in the
qualitative data where a number of comments suggested the participant did not have
access to a case with previous cancer. Eg.

I don't think that any of the 3 cases had had previous cancer treated

do not think this was covered in the 3 simulation sessions i did

There were no cases of this in the simulation tool.
Additional qualitative comments are presented in Table 4.
7
Table 4: Increased awareness of late effects from previous cancer treatments.














I wasn't aware of the the risks of mantle radiotherapy.
useful raising awareness of increased risk of breast ca post radiotherapy
The links to information about post radiotherapy risk and also the scenario concerning a woman
who had had previous treatment were useful.
I was not aware of the later risks of mantle radiotherapy but would have liked more information
on late effects of both chemo and radiotherapy in general. This is a knowledge gap that I wasn't
really aware of before taking part in this project.
Good reminder on the effects of radiothaerapy
I was not aware of an 8 times risk of breast cancer.
It was interesting to be reminded about radiotherapy as a cause of other health problems
did not seem to offer any info regarding this
this was definitely a learning point in the breast cancer case
Through the information provided
Previously did not consider previous cancer treatments in TWW decisions.
Good feedback at end of questions - excellent.
Yes I wasn't aware of the breast 8x increase in breast cancer risk post-radiotherapy
not fully aware of the statistics with this. Wasn't going down breast cancer route initially but the
sim sort of wound me up there by only really giving me that option in the end!
2.2.4 I felt that the simulation tool increased my awareness of the importance
of early diagnosis.
The quantitative data for participants’ attitude to this statement are presented in
Figure 6. 76.1% respondents either strongly agreed or agreed with the statement.
Figure 6: Increased awareness of the importance of early diagnosis.
8
A number of those providing additional comments stated they were already aware of
this. E.g.

I hope that we are all fully aware of the importance of early detection/diagnosis

I already feel aware of the importance of this,

No more than I was already aware of this.
Additional comments are provided in Table 5. It is relevant to note that several VPs
are mentioned in these comments, indicating that the subject was covered in more
than one of the cases presented.
Table 5: Increased awareness of the importance of early diagnosis












Especially in considering cancer in the younger population where we tend to think of more
common diagnoses and delay
Useful to try a case more than once and see the differences in outcome depending on the
action taken in that one consultation
It was a reminder to always be on the lookout for suspicious signs and symptoms.
The statistics at the end of the breast cancer case were very useful.
well highlighted in the lung cancer case
case 1 was most useful
I agree but given the advanced symptoms (weight loss, anaemia, 5m change in bowels) of each
of the cancer patients I do not think that they were early diagnoses!
Earlier diagnosis often means better prognosis
It highlighted the importance of not being led astray by the patient's past medical history (e.g.
the diagnosis of bowel cancer in the patient with IBS).
The statistics of changes in mortality rate for Sarah highlighted this to me.
Picking up on pt cues and concerns and more data gathering help lead to eliciting red flags that
were not mentioned initially by the Pt.
The feedback from the lung cancer case outlined the importance of early diagnosis
2.2.5 In general, the simulation increased my knowledge of the support
available to obtain further information (e.g. from ASWCS and National
Guidance)
The quantitative data for participants’ attitude to this statement are presented in
Figure 7. 77.8% of respondents either strongly agreed or agreed with the statement.
9
Figure 7: Increased knowledge of support available from ASWCs and nationally.
There were a limited number of additional comments to this statement. These are
presented in Table 6.
Table 6: Increased knowledge of support available from ASWCs and nationally









Have already attended a local ASWCS training event a few months ago and this further
enhances the message from that event.
I was fairly well aware of NICE guidelines but it was good to see summaries.
these i have now bookmarked - had some previously but not all
I was aware
i think it would have been better to demonstrate the resources available very simply online
before the patient started talking. i wasn't aware of them during the whole first patient
simulation.
Useful links to national guidelines
Went through the rescourses available during the excercise and re enforced my learning.
I was aware of the guideance and broadly what it contained already.
already aware of it
10
2.3 OPINIONS ON E-LEARNING AND THE USE OF VIRTUAL
PATIENTS
This part of the survey was intended to gauge opinion about e-learning and in
particular the use of Virtual Patient cases for online simulation.
2.3.1 I feel that I benefit from being able to learn online
The quantitative data for participants’ attitude to this statement are presented in
Figure 8. 94.4% respondents either strongly agreed or agreed that they benefited
from online learning.
Figure 8: Benefit from online working.
Additional comments are presented in Table 7. A number of themes are evident
including those of interactivity, and ease of access as well as the cases requiring
some improvements, which are discussed further in section XX below.
11
Table 7: Benefit from online working.
























Enjoyed VP's Nice to do in own time and online Very easy to use
can do in own time and quite realistic
Online learning is the most time efficient and easy to access method for me to do my revision
etc
Especially as can start and stop to fit around other work, and can study at work or home.
in general although a disadvantage is that often time is not set aside for online learning and so
it may not get done
fits in well with lifestyle
difficult that you cant always ask the q's you would normally.
Thus was a fun exercise in a bite size piece.
could not agree more
I find e-learning to be rewarding, self-directed and often extremely useful in altering my clinical
practice. I think with some alterations these simulations could do the same. There are a lot of
similarities with doctors.net modules but with fancy graphics.
Very useful learning tool
Technology has meant some excellent resources are now available online.
Really think interactive online learning suits my learning style.
Good way to learn
Can do at own speed in own time and able to go over scenarios multiple times.
In scenarios like this ie interactive.
Great to use virtual patients and I liked how it worked althought the options were too restrictive.
All the Qs should be available wherever in the scenerio you were.
SOme of the rescources on lone are availble on courses, but as I cannt attned all, I use on line
for my learning needs.I do some conference learning online via medscape too( American
rescourse).
Online self paced and self-timed learning is a verfy useful modality, and lends itself well to this
subject matter I feel.
I often use online resources to learn but I found this tool was a little clunky in this early stage
and not as rewarding as I would like.
Especially when interactive and time efficient. Independent of drug company sales pitch.
easy to accommodate into my CPD
If the scenarios are develped and better structured
Being able to dip in and out of resources like this are invaluable to me and will often find myself
doing some cbd at home in the form of elearning.
2.3.2 Case based learning using virtual patients is a good way of engaging
with online learning
The quantitative data for participants’ attitude to this statement are presented in
Figure 9. 91.7% respondents either strongly agreed or agreed that Virtual Patients
were a good way of engaging with online learning
12
Figure 9: Virtual patients are a good way of engaging with online learning.
The lack of options available during the scenarios (relating to what can be said to the
patient) emerged as a theme here. This is discussed in more detail in section XX
below. Additional comments about this statement are presented in table 8.
Table 8: Virtual patients are a good way of engaging with online learning.
















I liked these tools but I found the first one most helpful.
I really like the principal of this as a learning tool
Enjoyed this format - had never experienced simulated videos like this before.
Seeing the Avitar made the experience more realistic and more interesting.
i didn't find this tool useful sorry
absolutely
Doing a virtual clinic where there are right and wrong answers for each patient is very
unrealistic.
Interactive learning much more engaging and keeps you far more focused than simply reading
guidlines or a textbook
If the technology allows.
Think case-based learning is especially useful.
Better interactivity than sim sessions in reality.
Vet realistic pt, you can talk to them and practice your communication skills.
definitely useful.
I think having a speaking (albeit virtually) patient rather than text is quite an engaging method,
however, and I would like to see more of this type of learning.
I personally do not find this way of learning very easy and am concerned it will be used in the
future as a substitute for face to face patient contact
I felt engaged
13
2.3.3 Working through the virtual patient cases was useful for my CPD
The quantitative data for participants’ attitude to this statement are presented in
Figure 10. 70.8% respondents either strongly agreed or agreed that this simulation
was useful for their CPD.
Figure 10: the VP cases were useful for CPD
There were nine additional responses, reproduced in Table 9 below.
Table 9: the VP cases were useful for CPD









Not really
Not sure what this means.
cold be very ueful but the options were limiting and not always logical/realistic in the path they
led you
GP ST3 so not applicable
It would be good if it could produce some sort of certicficate I could post in my educational log.
An interesting and novel way to learn. Useful for identifying needs for additional learning.
I will enter this excercise in my e portfolio.
Links to succinct and relevant resources essential. Space for personal reflective notes, which
could be saved, would be useful
I am afraid that unless you can issue some certificate to demonstrate completion of
patient/module etc with estimated time spent then not really valuable as demonstrable cpd.
However for my own learning I would certainly dip into it from time to time.
14
2.3.4 I would like to see more of these cases being developed for GP CPD
The quantitative data for participants’ attitude to this statement are presented in
Figure 11. 88.9% respondents either strongly agreed or agreed that this simulation
was useful for their CPD.
Figure 11: More VP cases developed
Again, the issue of technical limitations/ restricted options is an identifieable theme in
the qualitative data. E.g.

I would prefer it to be real people rather than computer generated as more realistic and some
technical glitches in what questions were available for you to ask but overall really good.

very useful but the computing needs some tweaking i found it frustrating having to choose
from a list of options when i would have chosen none of the options but would have done
something that was an option 2 questions ago.

Perhaps with more expanded options for the doctor to chose, and a more fluid way to conduct
the consultation, although I suspect that may be a technical challenge?
Additional comments are reproduced in Table 10.
15
Table 10: More VP cases developed

Yes I think this could be a useful tool especially for raising awareness in cases of less common
situations. Also some scenarios managing not clear cut situations would be helpful (slightly
looser bowels in someone not meeting age criteria, recurrence of bowels symptoms a year
after investigation - when to reinvestigate. Also I think cancer in patients who had cancer as a
child in an important area to look at
Think they are a really good interactive idea.
I think it is a useful format. However the cases are so much more straightforward than real lifewhen you have 10 minutes, lots of letters , blood results and masses of previous consultations
and medications, QOF indicators, interruptions for registrars, receptionists urgent requests.
It seems a bit of a tricky situation, not being able to ask which q you want to ask, it throws your
natural thinking patterns, then you miss tings you would normally ask.
These cases are very relevent to my work.
I would like the opportunity to take part in further cases.
and for nurse practitioners
Yes ,more cases please.Dermatology wouldbe helpful.
Very useful. I am currently on maternity leave and this type of exercise is perfect for keeping up
to date with things.
In time I'm sure these would be a really good resource and certainly something I would come
back to. However, I think there are improvements that could be made to make it more realistic.
More useful for medical students/foundation doctors
With certificates!!











2.4 ADDITIONAL OPEN COMMENTS
This part of the survey was designed to allow participants the opportunity to
comment on the good aspects of the simulation as well as identify any issues and
make suggestions for improvement. The resulting qualitative data was subjected to
thematic analysis and the major themes are presented below.
2.4.1 What are the good aspects of using this learning tool?
Seven major themes were identified from the 69 responses received to this question.
These are summarised in Figure 12.
Respondents identified the provision of supplimentary information within the
simulation as being of benefit. This includes medical notes, letters and test results as
well as links to external web resources, NICE guidance and cancer assessment
tools.

Is closer to real life I liked the supplementary info eg medication, letters, NICE guidance..

Good availability of background information / letters etc to support the consultation - this
made it more realistic.

There are good references and signposts to other resources

Simulates a GP consultation quite well. Like the access to medical records etc at the
side, so I can choose when to look at them, as in real life.
16
Figure 12: The good aspects of this learning tool
The interactive nature of the virtual patient tool was also positively commented on:

Interactive learning - closer to real life with virtual patients than just reading information.

The interactive nature of it, and the simulation of "real life" GP consultations.

Very interactive with good cases and supporting information

Interactive, interesting and novel approach.

Interactive and therefore easier to learn on a case based situation
The interactive nature of the simulation had a positve effect on respondents in terms
of an providing an authentic learning experience and increased sense of realism. It
was acknowledged that the cases reflected real life:

I liked the different options available to create your own consultation "journey". This
made it more realistic

They were good cases, and they created quite realistic scenarios.

case 1 (meadows) actually quite realistic!

Realsitic Patient similer to real life
17

The cases were very fair and common, the actors voices were very good, the user
interface acceptable, the answers given by the patients had just the right degree of
ambiguity that you get in practice (excellent choice of wording)

Real patient simulations to think about real life situations and responses.

Felt like a reasonably lifelike interaction
The accessibility theme refers to the learners’ ability to access the simulation at a
time and place of their chosing:

Can be done from home in my own time. Gave me time to think about the case and lots of
useful information at the end of each case.

Freedom to do in own time

Able to complete this in my own time

Flexible - can be used at a convenient time

Useful to be able to complete at your own convenience
It was acknowledged that this was a better tool for e-learning than the usual
resources, and the feedback in the cases was very useful

Better than dry learning modules.

Less dry than reading guidelines - makes you think more than just reading

I like the virtual patient concept above a text based vignette very much.

Good feedback

Feedback information at the end.

It was nice to have positive feedback when you'd reached the right conclusion.
It was felt that the length of time needed to engage with the learning was beneficial:

Short enough cases to be informative but have choice to spend a few minutes doing one or a
longer time doing several

time efficient.

Easy to use, clear, does not take too long
Several respondents commented that they found the simulation fun to do:

It was fun, relevent, short, accesible.

It doesn't feel like work- enjoyable

Is fun to do,
18
2.4.2 What are the weak or negative aspects of using this learning tool?
Four themes were identified from the 67 responses received to this question. These
are summarised in Figure 13.
Overwhelmingly, respondents had issues with the limited choice of options available
to them.
Figure 13: The weak or negative aspects of this learning tool
This was particularly so as they progressed through the simulation and found that
questions to ask the patient that were previously available to them were no longer
accessible.E.g.

Occassionally the question I wanted to ask which had been present on the previous page was
not available if I asked a different question.

The option trees for the questions you could ask where somewhat restrictive. I found that if I
asked one question, I was sometimes denied the option of asking about another symptom
that was previously available.

I felt that there was too much restriction in the questions one can ask.

The options given sometimes were not how I would conduct a consultation and sometimes I
wanted to ask a question or do something next and it wasn't an option.
Several respondents found the simulations to be unrealistic, a view partly influenced
by the limited choice of options. E.g.

not realistic becuase of limiting options
19

doesnt properly mimic a gp consultation though it is very good that can open records whilst
the patient talks
Four comments were received regarding technical issues including poor audio
quality and the simulation being slow to load. A helpline was established in order to
deal with such issues during the pilot and the problems were identified as relating to
the user having very old versions of Internet Explorer.

I had a lot of technical problems, NHS computers arent always using the most up to date
software!
Three respondents felt that the simulations, being focussed on early signs of cancer,
cued their responses accordingly. E.g.

you tend to suspect they are pointing towards cancer more because of the nature of the
learning tool so I wonder if this clouds the decision making process.
2.4.3 What suggestions do you have for improving this learning tool?
Eight themes were identified from the 62 responses received to this question. These
are summarised in Figure 14.
Figure 14: Suggestions for improving this learning tool
Not surprisingly, given the issues highlighted in the previous section, the major
theme identified relates to the improvement of the options available to the learner at
each stage of the simulation. E.g.
20

Have more options for questions with each screen

Making sure all the options are present, otherwise consultation more difficult
Some respondents went on to suggest that a solution to this issue would be to
provide the opportunity for entering free text questions.

If there was a way of using free text instead of multiple choice options that may make it even
more realistic but realise this may not be possible. No other specific improvements, was a
very good tool.
Others suggested that having the ability to go back and revisit a particular node in
the simulation would also provide a solution:

I would be happier to have more flexibility in going back and forth and be allowed to form my
own sequence of questions from a list
There was a significant call for more cases to be produced and with increased
complexity:

More variety of case and publicising widely!

Wider variety of cases including paediatric.(appreciate just pilot currently)

More cases, more complex

Some more challenging cases with more subtle symptoms/signs/investigation findings that
meet 2ww referral criteria

More situations - I got better at using the tool as I learnt how it works.

Ovarian cancer would be a good one to do in light of new NICE guidelines
Alongside this was a call for more supporting information to be made available at the
end of each case:

Provide some extra learning points at the end to sum up the case.

If after several attempts the case has not been resolved it would be helpful to be rerouted to
an explanation of the case, so that one can learn from one's mistakes

a little more information after each case would probably help consolidate learning
Finally, a number of respondents re-affirmed that it was a good tool:

This is a very useful tool.

The patients are realistic, and the speed of speech is good.

I think this is a great tool
In addition to the themes identified above, there were several individual suggestions
that were identified has having potential to add value to the tool:

Certificates if thinking about credible cpd option (kitemarking ?)

Option to save as you go along.
21

Didnt immediately notice the links to pmhx, meds etc on right hand side - perhaps these could
be made more obvious?
3.0 CONCLUSION
Evidence shows that learners benefitted from interacting with the case based
simulations that have served to:





Highlight the importance of early diagnosis
Increase awareness of cancer assessment tools
Increase confidence in 2WW referral
Increase knowledge of available support to GPs
Increase awareness of the late effects of cancer
The study has shown that participants feel they benefit from online working and that
the virtual patient cases have provided a useful addition to CPD activity. Participants
have found their interactive nature engaging, and have highlighted the benefits of
them being online, quick and fun to do.
The dominant issue with the tool has been identified as relating to the lack of options
available as one progresses through a case, with learners expressing frustration at
not being able to ask questions previously available to them. This is not a fault of the
tool itself or the concept of virtual patients, but relates to a lack of experience in case
authoring. It should be noted that this was the first time this particular approach to
using virtual patients was used and that all the authors for this pilot had never
undertaken this type of work before. With the experience gained from this pilot, it is
expected that future iterations of this model will redress this issue.
Overall, the use of virtual patient cases for CPD around the early signs of cancer has
been shown to be very favourably received with the pilot participants. There was
significant agreement that case based learning using virtual patients provides an
authentic learning solution online and that further cases would be beneficial.
22