CAN INTERACTING WITH A
VIRTUAL HUMAN MAKE YOU
A BETTER HUMAN?
BENJAMIN LOK, PH.D.
UNIVERSITY OF FLORIDA
April 21st, 2015
Learning Objectives




Define is a virtual human
Identify components of virtual humans
Identify approaches to creating virtual human
Understand current and research applications of
virtual humans
Conflict

I have a financial interest in Shadow Health, Inc.
Shadow Health, Inc. develops virtual patient
software for nursing education.
Virtual Environment Fidelity
Virtual Iraq, USC, ICT
Virtual Environment Fidelity
Ford Vehicle Simulator
Virtual Environment Fidelity
Flatworlds, USC, ICT

Virtual environments have
high fidelity
Virtual Iraq, USC, ICT
Ford Vehicle Simulator
Flatworlds, USC, ICT
Goal of Talk

I propose the next area of VR and computer graphics research will
focus on virtual humans

How virtual humans affect people

How people are using virtual humans

Research directions of virtual humans

Motivate you to explore virtual humans in your own research
Image courtesy of Skip Rizzo, University of Southern California
Impact of VH in environments


https://www.youtube.com/watch?v=VZHQj0ttD44&
feature=youtu.be
https://www.youtube.com/watch?v=YL3hnFco1U0&f
eature=youtu.be
Example Virtual Human Interaction

Video courtesy of Skip Rizzo, University of Southern California
LO: Identify what is a virtual human


A virtual character in human form
Sometimes referred to a:
 Virtual
character
 Digital human
 Avatars – have come to mean representations of users
LO: Identify components of a
virtual human

Virtual humans are composed of
 3d
models
 Content (Audio, Gestures, Script)

There exist many VH generation systems for 3d
models
 Make
Human
 Autodesk Character Generator

Output a common file format (e.g. fbx)
Components of interacting with a
virtual human
Common technologies




Display – screen displays, HMDs, environment
displays
Audio – standard speakers
Sensing – Kinect, cameras, microphone, Wizard of
Oz approaches
Cognition – AI systems, NLP/NLU, Wizard of Oz
approaches
Creating a virtual human






1. Create a virtual human model
using a 3d modeling program
2. Create the conversation corpus
for a virtual human (e.g.
virtualpeoplefactory.com)
3. Record audio and animations
4. Integrate with a rendering engine
(e.g. Unity3D, Unreal Engine, OGRE)
5. Integrating sensing system (e.g.
keyboard/mouse, microphone,
Kinect)
6. Integrate cognition system (NLP,
event-based system, rule-based
system)
Can interacting with a virtual
human make you a better person?
Dr. Gregory House
Good with medical knowledge
Not so good with interacting with people
DerekRoss
Shepherd
Dr. Doug
Good with medical knowledge
Good with interacting with people
LO: Virtual humans applications
and research

Virtual humans as interaction partners

How can they affect us?

Teach us?

Change us?
Affect: Bias

Would health profession students treat these virtual
human patients differently?
Images from the Virtual Patients project at verg.cise.ufl.edu
Affect: Paranoia
Images courtesy of Mel Slater, University of Barcelona
Affect: Social Facilitation

Zanbaka, ICAT 2004
Affect: Phobia treatment

Fear of public speaking (Pertaub 2002, Virtually
Better)
Affect: Emotions and Ethics
Affect: Social norms

What would you do if she sneezed?
Pixels mean different things

What do you see?
Training

Think of tasks that everyone does almost
everyday…

Interact with another person

Yet training for this is very limited.

Humans are social creatures!
How do people train to interact
with others?




Lectures
 Human resources training
Case studies
Videos
Role-play
 With other students, instructors, actors
 Actors -> gold standard for many fields
 Medicine
 Military
Training with a VH

Impacts
 Education (teachers with students)
 Military (leadership training)
 Law Enforcement (police officers and suspects)
Justine Cassell – Carnegie Mellon University, USC ICT
Real change in behavior
Would you be influenced more by?
Baylor & Plant 2005
Avatars affect us

Being affected by your avatar
 Weight
loss
 Energy consumption

Bailenson, Stanford
 http://www.stanford.edu/~bailenso/
Virtual Patient
http://cg.alexandra.dk/tag/simulation/ http://labyrinth.sgul.ac.uk/mnode.asp?id=qgxlrdbu3lpfvf4jesngxlrdbtpr9kq
Virtual human patients train
students on

Eliciting important
information


E.g. “What medications
are you taking?”
Developing
communication skills
Organization of inquiry
 Diagnostic reasoning

Virtual humans provide:






Opportunities for practice
Exposure to infrequent –
yet critical – scenarios
Standardization
Patient variability
Feedback
Diagnostic reasoning
Virtual patients in education



Misconception: virtual patients would replace standardized
patients
Complements standardized patients
 Presentation of abnormal findings (e.g. cranial nerves)
 Vulnerable populations (e.g. pediatrics, mental health)
 Practice before seeing SPs (e.g. intimate exams)
Are assisted by standardized patients
 Standardized patients would support VPs (talk to SP,
physical exam on VP)
 Standardized patients can play many roles (children,
mentally disabled)
Virtualpatientsgroup.com




8 universities, 30+ researchers, 9 years of VP research
Create virtual patients
Deploy virtual patients
Enable


Curricular building and integration of training scenarios
Teaching and training with




Variety of scenarios
Variety of patients
After-action review systems
Looking for new collaborators


Write papers
Obtain nationally-competitive funding.
Virtual Patients

Sample Applications
 Abnormal
findings
 Intimate exams
 Team training

Educational Benefits
 Feedback
 Understand
your students
NERVE: The Neurological Examination
Rehearsal Virtual Environment

Medical students






Use neurological tests to diagnose a patient with a vision disorder
Discuss patient history
Conduct physical exams
Includes lecture notes, simulation, and evaluation
Receive additional exposure to patients with abnormal findings (Cranial
Nerve 2,3,4,5,6,7,12)
Each scenario has




Male African-American
Female African-American
Male Caucasian
Female Caucasian
Physical Examinations of Virtual Human
Patients

Mixed reality humans


Combine virtual humans with
mannequin simulators
Intimate Exams



Clinical breast exam
Prostate Exam
Students can


Practice physical examination and
communication skills
Real-time feedback of exam
performance



Coverage
Pressure
Conversation topics
Team Training



NSF-supported collaboration with UVA
Simulate team scenarios using a combination of
humans and virtual humans
Video
Virtual People Factory



www.virtualpeoplefactory.com
Web-based interface to virtual humans
Deployed Early 2008
 50+
active developers
 4000+ interactions
 100,000+ utterances
Vulnerable populations, bias, lowEnglish profeciency
Current state of virtual patients


Still in its infancy
Need to research:
 Topics
to apply VPs
 Quantify benefits

Would enable new tool to address
 Cultural
competency
 Patient Centered care
 Team training

Accomplished through:
 Curricular
integration
Interactive Demo

Build a virtual patient

How long and how much
 If
you want to build a VP similar to an existing VP, then
we’ve had students build VPs in approximately 5 hours
 If you want to build a new one, several weeks.
 Our online resources (www.virtualpatientsgroup.com)
are available free for research purposes
Future Implications
•
Revolutionize interpersonal
training
–
–
•
Culture
Communication Skills
Help people with communication
skills deficiencies
–
–
–
–
Fear of public speaking
Social phobias (e.g. paranoia)
Autism
Bias
Images courtesy of Sabarish Babu – Clemson University
So Can Virtual Humans Make You a
Better Person?

If you want them to, we know they can
 Affect
 You
you
can learn from interacting them
 Change
your behavior
I hope this talk motivated you to…




Explore virtual humans in your research
As a community, have more people interact with
your research
Embrace the new wave of technology to interact
and distribute
Inspire people outside this room
Thank You!
Build your own virtual patients: www.virtualpatientsgroup.com
Contact: lok@cise.ufl.edu
Support: National Science Foundation and National Institutes of Health
After-Action Review by Educators
•
IPSVizn
–
–
•
Web-based interface
Educators can review student
interactions
Experts can
–
–
–
Identify trends (mean of class)
Identify outliers
Evaluate topics covered
Commercialization

If you want to start a company…
 Find
the right people
 Get a business person
 Find the right people
 Find the right people
 Expect 60-80 hour work day
 Know your markets/competition
Mobile Distribution of Simulation
Deploy simulations via mobile
platforms
 Android app, released December
2010, over 4600 downloads

 In
Android Market, search for
“Virtual Patient”
Image from www.virtualpatientsgroup.com
Museum of Science
and Industry


Science museum in Tampa, FL
Integrate a VH interaction


Public health literacy
Research


About 4000 people per year
enter our exhibit
About n=~400 per year are
usable datapoints for studies
Repositories

MedEdPORTAL
 Peer

reviewed medical education resource
400 institutions downloads in 10 months