Speaking of Virtual Patients:
What are they, really?
A reasonable question???
Thomas B. Talbot, MD, MS, FAAP
MedVR Group
USC Institute for Creative Technologies
Let’s talk about what a virtual patient is……

Let’s talk about what a virtual patient is

Virtual Patients we can measure (standards based)
– Case Presentations
– Interactive Patient Scenarios

Virtual Patients that don’t fit
–
–
–
–
–

2
Virtual Patient Games
High Fidelity Software Simulations
High Fidelity Manikin Simulators
Human Standardized Patients
Dialog Agents & Virtual Standardized Patients
The future of Virtual Standardized Patients
CASE PRESENTATION
3
CASE PRESENTATION
4
Type of Virtual Patient
Type of VirtualCommon
Patient
Names
Teaching Applications
Learner Skills Evaluated
Interactivity
CASE PRESENTATION
Online Virtual Patient
Basic Knowledge, Clinical Reasoning
Knowledge, Pattern Recognition
Very Low
Consistency of Experience & Evaluation
High
Flexibility to Recover from Learner Errors
None
Suitable for Game Approach?
None
Authoring Challenge
Low
Core Technology
5
Enabling Technology
Text, Multimedia
HTML, Authoring Tools
INTERACTIVE PATIENT SCENARIO
6
7
Type of Virtual Patient
Type of VirtualCommon
Patient
Names
Teaching Applications
Learner Skills Evaluated
Interactivity
INTERACTIVE PATIENT SCENARIO
Virtual Patient, Multimedia Sim,
Branching Scenario
Clinical Reasoning
Case Knowledge, Clinical Reasoning
Low-Moderate
Consistency of Experience & Evaluation
High
Flexibility to Recover from Learner Errors
Low
Suitable for Game Approach?
Moderate
Authoring Challenge
Moderate
Core Technology
8
Enabling Technology
Branching, Multimedia
HTML, C, Actionscript, Multimedia Authoring Tools
INOTS
9
VIRTUAL PATIENT GAME
10
Type of Virtual Patient
Type of VirtualCommon
Patient
Names
VIRTUAL PATIENT GAME
VR Simulation, Virtual Patient Avatar
Teaching Applications
Procedures, high-risk scenarios,
team training
Learner Skills Evaluated
Clinical & Procedural Reasoning,
Physical Diagnosis
Interactivity
Moderate-High
Consistency of Experience & Evaluation
Moderate
Flexibility to Recover from Learner Errors
Moderate
Suitable for Game Approach?
High
Authoring Challenge
High
Core Technology
11
Enabling Technology
State-Machines, Branching
Game Engine
High Fidelity Software Simulation
12
13
Type of Virtual Patient
Type of VirtualCommon
Patient
Names
Teaching Applications
Learner Skills Evaluated
HIGH FIDELITY
SOFTWARE SIMULATIONS
Medical Simulation, VR Surgical Simulator
Complex Procedures, Surgeries,
High-risk Scenarios
Procedural Skills, Critical Interventions
Interactivity
High
Consistency of Experience & Evaluation
High
Flexibility to Recover from Learner Errors
High
Suitable for Game Approach?
Moderate
Authoring Challenge
Very High
Core Technology
14
Enabling Technology
Physiology Engine,
Virtual Reality Tissue Model
Game Engine, Haptics, Anatomy Data
15
Type of Virtual Patient
Type of VirtualCommon
Patient
Names
Teaching Applications
Learner Skills Evaluated
Interactivity
HIGH FIDELITY MANIKINS
Manikin, Task Trainer
Procedures, Team Training, Codes
Clinical Protocols, Physical Diagnosis,
Team Performance
Moderate
Consistency of Experience & Evaluation
Low-Moderate
Flexibility to Recover from Learner Errors
Moderate
Suitable for Game Approach?
Authoring Challenge
Core Technology
16
Enabling Technology
Low
Moderate
Varies
Microcontrollers, sensors, actuators,
wireless data
HUMAN STANDARDIZED PATIENTS
17
Type of Virtual Patient
Type of VirtualCommon
Patient
Names
Teaching Applications
Learner Skills Evaluated
HUMAN STANDARDIZED PATIENTS
Standardized Patient, Human Actor
Patient History, Physical Exam,
Patient Counseling
Interview Skills, Physical Diagnosis
Interactivity
Very High
Consistency of Experience & Evaluation
Very Low
Flexibility to Recover from Learner Errors
High
Suitable for Game Approach?
None
Authoring Challenge
Low
Core Technology
None
18
Enabling Technology
Written Script, Video Recorder,
Human Monitor
19
Type of Virtual Patient
Type of VirtualCommon
Patient
Names
Teaching Applications
Learner Skills Evaluated
Interactivity
VIRTUAL STANDARDIZED PATIENTS
Virtual Human, Conversational Agent
Patient History, Patient Counseling
Interview Skills
Very High
Consistency of Experience & Evaluation
High
Flexibility to Recover from Learner Errors
High
Suitable for Game Approach?
High
Authoring Challenge
Very High
Core Technology Dialogue AI, Natural Language Processing
20
Enabling Technology
3D Game Engine, Speech, Non-verbal behavior
generation, Speech Recognition
Video Time
21
Simcoach Standard Patient Studio

$8M of DoD Advanced Technology R&D Project

A large public trial and distribution of the technology over 4 years

Ambitious scope, yet uses mature technology to mitigate risk

Usable, refined tools for the public
– Free
– Goal is to create a “critical mass” with the medical education community

Serious emphasis on student assessment

Designed with ‘lessons learned’ from prior work in mind

Desire to replicate the “gold standard” or surpass it
22
What it is:

Conversational Virtual Standardized Patient (The “Clinic”)
– Includes plethora of ICT technology:
 Avatars, Natural Language Understanding, Artificial Intelligence Dialogue
System, Automated Non-Verbal Behavior Generation, Emotional
Expression, Trust-based Responses
– Works through web-browsers & tablets

Online Authoring System (The Studio)
– Guided authoring system targeting medical educators
 Leverages prepared ‘personalities’ that are modified
 Learner ‘questions’ are mostly pre-packaged because they are the most
difficult aspect of patient authoring
 Two-phase “open questioning” / “review of system” model
 Simple assessment authoring integrated into tools
– Shared assets with forum and user rating system
23
The Standard Patient Studio Team

ICT Teams
–
MedVR Group


–
Simcoach Group

–
Eric Forbell
Integrated Virtual Human Team


–
Rizzo, Talbot, Williams and others
Screenwriting Team
Arno Hartholt
Includes Art Team
Assessment Group

Chad Lane

External Partners
–
Breakaway Ltd,

–
USC Keck School of Medicine

–
Dr. Win May and others
Uniformed Services University School
of Medicine


National Capitol Area Simulation Center
Gil Muniz, Alan Liu, et al.
– National Board of Medical
Examiners


24
Serious game development company,
Timonium, MD
Steve Clyman
Galen Buckwalter
Human Standardized Patients

The “Gold Standard” dialogue-based training system
– Rely on student initiative to drive encounter dialogue, ask questions,
dynamically follow leads
– Close approximation to actual patient encounters
– Genuine physical exam and findings

Disadvantages
–
–
–
–
–

High verbal, but low factual accuracy
Expensive, difficult retention
Some pathologies & ages unavailable
Few opportunities to access them
Uncontrolled variation, subjective assessment
There is no desire to replace or eliminate HSPs
– HSPs have inherent advantages
25
Lessons from the past regarding Virtual Patients
Lack Dialogue
Are inflexible
Not Readily Authorable
Lack of content ‘critical mass’
Emphasize things that computer
sims aren’t good for
Target populations that do not need
the training
Lack critical interactive technologies
Too expensive, too proprietary
Avatar is often superfluous
Do not emulate gold-standard
approach
Lack automated & well designed
assessment
Lack breadth
26
Attempted things that were too
complicated
SPS: Achieving A Gold Standard Interaction
1.
Reading the chart
2.
The patient interview (VSP session)
1.
2.
3.
4.
5.
3.
Greeting the patient
Obtaining the chief complaint
Establishing Rapport
Open questioning
Specific questioning (Review of Systems)
Patient Examination (single text page or VSP session)
1.
2.
Physical Examination
Review Laboratory or Radiology information
4.
Make a Diagnosis and Treatment Plan (single menu page)
5.
Counsel the patient on the diagnosis and treatment plan (VSP session)
6.
Assessment (visual summary of skills and progress)
1.
2.
27
Errors of commission and omission
System’s estimate of progress on learning skills and sub-skills
GREY: Lower Emphasis Areas BLACK: Higher Emphasis Areas
A Variety of Encounter Types
28
The Patient Interview

Virtual Human Encounter
–
–

On-Screen Educational Guides
–
–

Full ICT Virtual Human Capabilities
Typed Input, Verbal & Nonverbal Output
 Natural Language Recognition
 Speech recognition year 3
Assessment Indicators
 Rapport, Achievements
 Not used for evaluation oriented patients
Virtual Attending Physician (VAP)
 Automatically asks a question when pressed
Conversation
–
–
–
29
Free-text Question/Answer model
 Patient may ask a multiple choice question
Open to Closed Questioning
Answers rapport dependent
Virtual Standardized Patient Interview Details

Based on pre-authored “Personalities”
–
–
–

Variety of appearances
–
–
–

Educator will be able to select sex, face, skin, habitus, clothing, etc
Non-verbal tone and baseline rapport is selectable
Respiratory rate/depth can be set or based upon physiology engine data at authoring
Guided authoring by medical educators
–
–
–
30
Will create a very well-built out patient with negative/normal responses to a wide variety of
medical questions
At least five personality variants will be created
Most educator-authored patients will be based off these personalities
First, enter chief complaint, diagnosis
Second, enter patient descriptive dialoge in response to open-ended questioning
Third, select items from “Review of Systems” and history lists, alter responses for
appropriate items and select items for evaluation scoring
Patient Interview Editor (Open Phase)
31
32
Virtual Attending Physician (VAP)

A Virtual Human Encounter
– White Coat Attending
– Simulated Socratic Session

Multiple-choice INOTS-like encounter
– Choices with consequences
– Unique feedback based upon learner response

Multiple Uses
– Patient diagnosis & evaluation
– Teaching points
– Test for understanding
33
Physical Examination Phase

Different than other physical exams
– Features are resource constrained
– Will not attempt to do things that can now be done better via other means.

Multimedia interface
– Pictures, sounds, videos or animations can be loaded
– Think “X-Rays, rashes, auscultation, etc”
– Lab/rad results available (physiology engine derived or author entered)
 Available via menu or command line

Unique Virtual Human Encounter
– Non-verbal, non-dialogue
– Responds to commands to perform maneuvers
 Neurological & Musculoskeletal focused
 “Stand on one leg and close your eyes”
– Easy “check the box” based authoring
34
Patient Diagnosis & Ordering

Simple One Page Multimedia Form
– Select diagnosis from choices
– Select options to order for patient plan
 Auto-populates labs/radiology from physical exam phase
35
Patient Counseling

Virtual Human Session
– Dialogue Based, Multiple-Choice or both? (undecided)
– Resource Constrained
– Optional

What the learner should do
– Explain the diagnosis & treatment plan
– Answer questions
– Assess for patient understanding
36
Assessment Screen: Robust & Graphical
– Interview Phase






–
Rapport & Trust
Open/Closed question ratio
Must Ask Items (critical)
Nice to have asked items (secondary)
Acts of commission (bad things you said)
Efficiency
VAP Encounter

Score-based feedback or no Feedback
– Physical Examination Phase


Critical & secondary items selected
Efficiency score
– Counseling Phase

Undetermined (limited assessment in this version)
– Diagnosis & Treatment

37
Direct feedback on selections made vs. optimal
Initial Use Cases:
“Bread & Butter” Medical Student / Intern Cases
1. General Communications Skills
a. Patient Introductions
b. Rapport
c. Ensuring that patients understand
your instructions
2. Common Acute Care Patients
a. Infectious Diseases (URI, AGE, STD,
Pneumonia, influenza, etc.)
3. The Internal Medicine Patient (The big five)
a. Type II Diabetes
b. Hypertension
c. COPD
d. Heart Failure
e. Thyroid
f. Combined pathology
4. Psychological Screening
a. Depression
b. Alcoholism
c. ADHD
38
4. Psychological Screening (cont'd)
d. Sexual History
e. Suicide
5. Surgery
a. Pre-operative visit
b. Post-operative visit
6. Neurology
a. Mini Mental Status Exam
b. Seizure history
c. Parkinson’s
d. Headache History
e. Sleep History
7. Pediatrics (optional)
8. Women
a. GYN History
b. Pregnancy
Development Plan

Development Phase (18-24 months)
–
–
–
–

Public Use Phase (24 months)
–
–
–
–

39
Construct tools, interface
Build prototype baseline-personality
Initial cases built & secondary personalities authored
Research-based question database optimization
Open to second-level partners for first three months
Open publically
Public case authoring, student use, forum community
Public presentations & seminars at national events
Reporting
Research Questions

Standard Patient Encounters
–
–
–
–
–
–
40
How useful are virtual standardized patient encounters?
What are the comparative advantages and disadvantages of VSPs compared to human
standardized patients?
What kinds of patients, diagnoses or other characteristics are best/worst for VSP
encounters?
What phase of the clinical encounter is preferred by educators for a VSP encounter?
How do students respond to the presence of a virtual attending physician (VAP) quizzing
and guiding them? Is it useful?
How can VSP encounters best be blended with live standardized patients?
 What are the cost implications?
 How can use of VSPs free up human resources to expand availability of human
standardized patients (HSPs)? If so, what will HSPs being doing more of?
Research Questions

Capabilities
–
–
–

Other Results
–
–
–
–
41
What capabilities are in greatest practical use?
What capabilities are not being used? Why?
What are the additional capabilities desired by the medical education community?
Number of cases and patients authored
Student assessment and performance data
Author and student feedback
Project impact results
Research Questions

Authoring
–
–
–

Assessment
–
–
–
–
–
42
Can medical educators successfully author compelling VSP interactions when this formerly
required teams of experts?
What are the unexpected uses and author populations for this technology? Nurses, allied
health, dentists, pharmacists, sales, customer service?
What are the limitations for educator authored VSPs?
 What features requires more sophisticated authors?
 How can authoring be improved to extend these domains to less sophisticated
authors?
Are VSP encounter assessments equal or superior to traditional HSP encounter
assessments? How?
Is there value to offering a third person perspective replay?
How can an automated assessment be successful in graphically depicting a concise
assessment of student performance that has utility for guiding future student performance?
How does VSP encounter performance change with repetition? What is the optimal
number of repetitions?
Are VSPs and the assessment good enough for use in state-mandated testing?
STANDARD PATIENT STUDIO SUMMARY

The Simcoach Standard Patient Studio uses ICT’s
unparalleled technology

The impact of SPS will be revolutionary

SPS will be a special national resource

SPS will closely approximate HSP encounters, but with some
advantages

SPS will be useful for both learning & evaluation

SPS will provide objective student data
43
CONCLUDING COMMENTS

There are lots of different things called ‘Virtual Patients”

Various technologies tend to be stronger for specific types of
medical training. Choose accordingly.

Use the easiest technology that achieves the learning
objective

The technology is a tool, not an end goal in itself

Assessment is really important

Newer types don’t fit into the Medbiquitous model, but we can
probably find practical solutions for this.
44