Aaron Kotranza, Ben Lok, Juan Cendan
University of Florida
Colleges of Medicine and Engineering
Gainesville, FL
MedBiquitous Annual Conference, London, 2010
Presentation 39558

Neurological Exam Rehearsal Virtual Environment (NERVE)

Virtual Patient targets abnormal physical exam findings of
the cranial nerves

Interaction includes:
 Verbal communication with natural speech
 Gestures
 Virtual tool interactions

Patient Vision Feedback (PVF) allows the user to share the
patient visual experience

Cranial nerve palsy/injury is uncommon but
dramatic and critical physical finding

Cannot be feigned by SP

Opportunities for clinical teaching are rare
 Identified as a particularly relevant niche for
virtual human representation

Traditional lecture, textbookvideo didactics, some simulation
systems
 Fundoscopic exams
 UC Davis disembodied eyes
system
▪ http://cim.ucdavis.edu/eye
s/version15/eyesim.html
▪ Rick Lasslo,M.D., M.S.,
Gary Henderson, PhD, and
John Keltner, M.D. , UC
Davis School of Medicine

NERVE uses a life-size virtual patient with cranial nerve
injury

Learner performs examination using
 Speech
 Virtual tools (ophthalmoscope, eye chart, hand and fingers)
manipulated by a Nintendo Wii Remote (WiiMote)

Representation from the VP
 Restricted eye motion
 Double vision
 Relevant history
VP modeled in Autodesk
Maya
 Rendered with open
source Ogre 3D
 VP portrayed on a 52” LCD
 User wears IR fiducials for
location/angle of vision
tracking

 Naturalpoint Optitrack

Real-time experience
 Allows learner to view a virtual room through the
eyes of the patient

Goals
 Provide learner with more information for
understanding how a particular CN injury affects
the patient’s vision
 Provide insight into the patient’s overall wellbeing
and safety

Learner greets VP and queries with natural speech
 Virtual People Factory

Learner performs examination
 Ophthalmoscope with and without light
 Fundoscopic exam
 Hand fingers tool for ROM
 Convergence
 Peripheral Vision
 Eye chart with “cover one eye”

Typical exchange between user and virtual patient can occur in natural spoken language or in text-box

Also able to reproduce on command








Head tilt
Chin to chest
Smile, frown
Stick out tongue
Raise eyebrows and wink
Puff out cheeks
Turn head side-to-side
Collision detection allows examination of Cranial Nerve 5
(virtual finger and facial sensation)
•
•
•
Eye movement requires coordination of 6
muscles
Characteristic presentation when abnormal
would require tremendous computational
expense
Developed our own model that is not
physically based, but consistent with
physiologic presentation



Visual model uses
interpolation of 8
cardinal eye
movements (left, upleft,…, down)
Can be visualized as a
2-D pitch-yaw plane
For each eye the model
defines a set of 8
vector criteria

Typical Range Examination
 Red/Green boxes as teaching tools





Shaped like a hand-held tool and high degree of freedom
control
Bluetooth, 11 buttons, 3 orthogonal accelerometers, 45
degree field-of-view IR camera that tracks up to 4 points
Displays information with integrated LEDs, speaker and
vibration
100Hz update and sub-centimeter accuracy
Inexpensive

Ophthalmoscope

Example: “Contact”
with eye leads to
vibration and
complaint from patient

Trigger button used to
create light for pupil
examination

Exam of the pupil using laptop
 Can be done with WiiMote

Head mounted
display

Virtual room is
rendered twice
reflecting the
perspective of
each eye

18 students in second year of medical school
 Had prior experience with “normal” neurological
exam
 Had not examined a patient with active pathology

9 students experienced PVF prior to examining the
patient
 The other 9 did so after data collection

Two participants at a time
 Completed experience survey
 NERVE and PVF were explained
by instructor (Author AK)
 Participant PV donned a
stereoscopic HMD displaying
the virtual room while
participant NPV examined the
NERVE
▪ Both participants are
synchronized to the same
patient exam and
environment
“PV” Passive – Sees
what the patient sees…
“NPV” Active – Interviews
and examines patient
• Takes H&P
Participant • Nerve
NPV –
Injured CN3
Active
Examiner
• Sees what
the patient
Participant is seeing
PV –
during the
Passive
H&P
Viewer
Both
users
complete
survey,
switch
roles
• Takes
H&P
PV Now • Nerve
is Active
Injure
Examiner d CN6
NPV
Now is
Passive
Viewer
• Sees
what
the
patient
is seeing
during
the H&P
Both
users
complete
final
survey
2 participants never experienced double vision
 One in each group
 ? Technical, individual …unclear and removed
from analysis
 Correct diagnosis was established equally by both
groups
 9 of 9 in PV; 8 of 9 in NPV (p=NS)


PVF provided enough data to accurately diagnose
the CN injury
 81% were able to make the Dx using PVF only
▪ 7/8 in PV group identified CN3 palsy
▪ 6/8 in NPV group identified CN6 palsy
▪ 2 = 5.1; p<0.05

Concern for Patient Safety
 Affective performance (perspective taking leading to
concern for patient wellbeing) evaluated as expressed
(verbal or written) concern for safety
▪ E.g. Informing the VP that he should not drive home,
etc.
▪ 7/8 learners in PV expressed concern vs. 0/8 in the NPV
group (p<0.005 Fisher’s exact)
▪ 5 on post patient vision survey
 1 of five told the patient directly during the exchange
▪ 3 on the overall survey
 1 expressed concern in both surveys

Note
 All participants were primed to think about
driving as the opening statement from the VP was
▪ “I was driving home from work and all of the
sudden the lines on the road started to cross.”

NERVE presents a life-sized virtual human
presenting with a cranial nerve palsy
 Novel vector methodology provides efficient
modeling
 Content validity has been completed on prior
testing with 32 students and residents

Students can arrive at the correct diagnosis using
traditional examination skills as well as “through the
eyes of the patient” technique

Patient safety will be augmented by giving the
examiner a patient-perspective of the presenting
condition

Currently developing and testing a webdeployable version of NERVE
 Will not initially support voice recognition but can
type-in-box
 Will not support patient vision
 Widely deployable

Dr. Cendan and Dr. Lok receive support from
NIH NLM R03-LM009646.
100%
90%
2
2
2
2
2
2
4
80%
3
2
3
4
70%
60%
8
4
4
50%
40%
9
30%
10
9
8
8
6
20%
4
10%
0%
Prepared for patient
encounter
I enjoyed the VP
interview
The VP appears
authentic
Agree
Stimulates asking
questions
Neutral
I found this a
worthwhile
educational learning
experience
Disagree
Would use as
practice tool
Better than reading
about it