Designing a Virtual Patient for
Communication Training
April Barnes, M.S., Ph.D. Candidate1,2
Jennifer Cloud-Buckner., Ph.D. Candidate1,2
Jennie Gallimore, Ph.D.1,2,3
Phani Kidambi, Ph.D.1
Rosalyn Scott, M.D., M.S.H.A.1,2,3
Ohio Center of Excellence in Human-Centered Innovation1
Department of Biomedical, Industrial, and Human Factors Engineering2
Department of Surgery, Boonshoft School of Medicine3
Wright State University, Dayton, OH, USA
Source: http://research.bidmc.harvard.edu/VPtutorials/default.htm
Navigation
Menu
•Information presented to user through video or text
•Best for training clinical reasoning/decision-making skills (Cook & Triola 2009)
Picture/video
of patient
•Develop high–fidelity, interactive VP
•Realistic appearance (3D, animated, full body, non-verbal
behavior)
•Speech recognition
•Natural, conversational capability
•Animated facial expressions, gestures
•Adaptive responses
•Emotion Detection
• Develop training related to communication skill
performance
•Evaluate learner performance
•Provide constructive feedback
INPUT
Speech
Recognition
Learning
Objective,
Scenario
Development
Evaluation/
Coding of
Context in
Communication
Model
Communication
Analysis for
Learner
Feedback
Selection of
Responses
(emotion, nonverbal, verbal
VP Output
Signal
Processing
(tone,
inflection)
Key word
Processing
(learning
algorithm)
• Multidisciplinary team of subject-matter
experts and experienced clinicians
• Extensive literature review
• Observation of real SP training and
performance for iterative VP design
• Prototype VP
•
•
•
•
Most software is Freeware
Speech recognition
Script matching based on keyword in user query
Randomly selects 1 of 3 responses to each question
• Develop rubric for performance evaluation
Communication models
• Cognitive-Affective Model of Organizational
Communication Systems (CAMOCS)
• Roter Interaction Analysis System (RIAS)
• Extensive
research article
cites 301
sources in
Inputs
business
organizational task, distance,
values, norms
communication
• Main factors of
communication
complexity:
inputs,
process,
impact
Process
goals, media,
strategies,
message form
Communication
Complexity
Impact
mutual
understanding,
relationship
• Commonly used measurement framework of
healthcare communication
• Classifies task-focused communication and socialemotional communication
• Coding scheme for video or audio of physicianpatient interactions
• Utterances divided into >40 classifications, plus 12
global dimensions of socio-emotional affect
(D. Roter, 2006; D. Roter & Larson, 2002).
Personal remarks, social
conversation
Laughs, tells jokes
Shows approval
Gives compliment
Shows agreement or
understanding
Empathy
Shows concern or worry
Shows concern or worry
Reassures, encourages
or shows optimism
Legitimizes
Partnership
Self-Disclosure
Shows disapproval
Shows criticism
Asks for reassurance
Transition words
Gives orientation,
instructions
Paraphrase/Checks for
understanding
Bid for repetition
Requests for services
Asks for understanding
Asks for opinion
Asks questions
Closed/open-ended
Medical condition
Therapy
Lifestyle
Psychosocial-Feelings
Gives information
Counsels or directs
behavior
Design the system to support the
needed impact, goals, strategies,
media characteristics, inputs and
learning outcomes.
Representative case of Mr. Y and Dr. X:
65-year-old white male with no
significant past medical history
Coughing for 3 months (no fever,
infection, chills)
Former smoker
Possible mass on chest x-ray
Analysis Components
Affective distance
Adjusting to feedback
Interactivity
Tasks
Shared understanding
Contextualized content
Explicit directions
Goals
Cognitive distance
Adjusting to feedback in
communicating a difficult
diagnosis
Analysis Components
Physician must be sensitive to body
language and patient’s reactions to
moderate how much information is
delivered in the initial diagnosis.
For example, if Mr. Y dismisses the
urgency of the news, Dr. X may give
a more explicit explanation of why
these tests are needed and why the
timing of them is important.
Affective distance
Adjusting to feedback
Interactivity
Tasks
Shared understanding
Contextualized content
Explicit directions
Goals
Cognitive distance
Shared understanding
Need shared knowledge between
participants to improve dialogue
Analysis Components
Affective distance
Contextualized, explicit content
The surgeon may want to explicitly present
treatment options, with various risks and
percentages associated with them.
Adjusting to feedback
Interactivity
Tasks
Shared understanding
Contextualized content
Explicit directions
Goals
Cognitive distance
Message Goal
• In a follow-up appointment, oncologist
discovers that Mr. Y has not been getting
all of his chemo pills; Mrs. Y had
postponed a couple of doses because it
was making her husband too sick.
• When physicians want to instruct or
influence difficult patients, they may
want to use highly formal language with
explicit instructions so that they can
better convey the importance to the
patient of a particular course of
treatment.
Analysis Components
Affective distance
Adjusting to feedback
Interactivity
Tasks
Shared understanding
Contextualized content
Explicit directions
Goals
Cognitive distance
Cognitive distance
• A physician explaining a complex
diagnosis to a patient with limited
medical understanding will require
more explicit explanations, more
formal information, and probably
multiple methods of presenting
information (visual, verbal) for the
patient to get then and to reference
later.
Analysis Components
Affective distance
Adjusting to feedback
Interactivity
Tasks
Shared understanding
Contextualized content
Explicit directions
Goals
Cognitive distance
• Conduct study: comparison of training with SP
alone to training with VP and SP
• Measures: same used to evaluate performance
using SP
• Move from prototype to build a VP in a gaming
environment with more realistic non-verbal
movements
• Development of the virtual human is being
created in an Army project to develop learning
for cross-cultural competencies focusing on nonverbal behaviors
•Prototype Proof of Concept
•Haptek SDK and body models from Haptek
•Free speech recognition – Microsoft Speech
•Free synthetic speech generation
•JAVA
•New System Under Development
• Unreal Tournament SDK game engine for virtual
environment
• Stereoscopic 3D display
• Maya 3D object editing software for body and object
creation
• FaceFX for visual expressions and matching speech
phonemes with mouth movements.
• Custom creation of different looks and custom developed
facial action movements not available in FaceFX.
• Natural Speaking Professional for speech recognition.
• Ipisoft and Playstation video cameras (6) for creating natural
body movements into characters.
•Future adds
•Learning software development for interpreting
speech and providing feedback vs discrete
scripted feedback.
•Measures of learner interaction
•Eye tracking (when not using 3D stereo)
•Face tracking
•Speech context
•Emotion detection (facial and verbal)
Thank You!
Questions?
•
•
•
•
•
•
•
•
•
•
•
•
•
Accreditation Council for Graduate Medical Education (ACGME) (2005). Advancing Education in Interpersonal and
Communication Skills: An educational resource from the ACGME Outcome Project. Retrieved from
http://www.acgme.org/outcome/implement/interperComSkills.pdf.
Association of American Medical Colleges. (1999). Contemporary Issues in Medicine: Communication in Medicine.
Report 3 of the Medical School Objectives Project. Washington, DC
Cook, D.A. & Triola, M. M. (2009). Virtual patients: a critical literature review and proposed next steps. Medical
Education, 43(4), 303-311.
Stone, M. & Silen, W. A 50 year-old woman with Lower Abdominal Pain. Retrieved from
http://research.bidmc.harvard.edu/VPtutorials/default.htm.
Issenberg, S.B., McGaghie, W.C., Petrusa, E.R., Gordon, D.L. & Scalese, R.J. (2005). Features and uses of high-fidelity
medical simulations that lead to effective learning: a BEME systematic review*. Medical Teacher, 27(1), 10-28.
Makoul , G. (2001). Essential elements of communication in medical encounters: the Kalamazoo consensus statement.
Academic Medicine. 76:390-393.
Paul, D. L. (2006). Collaborative activities in virtual settings: A knowledge management perspective of telemedicine.
Journal of Management Information Systems, 22(4), 143-176.
Roter, D., & Larson, S. (2002). The Roter Interaction Analysis System (RIAS): Utility and flexibility for analysis of
medical interactions. Patient Education and Counseling, 46(4), 243-251.
Roter, D. (2006). The Roter Method of Interaction Process Analysis. Retrieved May 1, 2009, from
http://rias.org/manual.pdf
Smothers, V., Azan, B., Ellaway, R.(2010). MedBiquitous Virtual Patient Specifications and Description Document.
Retrieved from http://www.medbiq.org/working_groups/virtual_patient/VirtualPatientDataSpecification.pdf
Stone, M. & Silen, W. A 50 year-old woman with Lower Abdominal Pain. Retrieved from
http://research.bidmc.harvard.edu/VPtutorials/default.htm.
Te'eni, D. (2001). Review: A cognitive-affective model of organizational communication for designing IT. MIS Quarterly,
25(2), 251-312.
Toussaint, P., Verhoef, J., Vliet Vlieland, T., & Zwetsloot-Schonk, J. (2004). The impact of ICT on communication in
healthcare. Proceedings of MEDINFO’04