Biofeedback in
Virtual Reality applications and
Gaming
Bonie Rosario, Jr.
Sebastian Osorio
Tom Iancovici
University of Massachusetts Lowell
Intro to Biosensors
16.441.201
Prof. Xingwei Wang
Outline
What is Biofeedback?
 Electroencephalogram
 The Galvanic Skin Response and Heart
Rate Variability
 Biosensor Research
 Research findings
 Our Conclusions

What is Biofeedback?

Biofeedback = the technique of using
monitoring devices to furnish information
regarding an automatic bodily function,
such as heart rate or blood pressure, in
an attempt to gain some voluntary
control over these functions[1].
Electroencephalogram

The electroencephalography is defined as a graphic representation of the
potential difference between two different cerebral locations plotted
over time[3].
EEG activity can be subdivided into
various types of frequency rhythm bands.
Research has indicated that different
EEG frequency bands are associated
with different mental states. The major
five are alpha, theta, beta delta and
gamma.
EEG signals tend to have amplitude in
the range of 2V to 100 V
EEG Commercial Products
There are several affordable
commercial products EEG
Systems. Two of the most popular
and user-friendly are produced by
NeuroSky and Emotiv.
NeuroSky MindWave
Emotiv EPOC
A recent study compared the quality
score of an EEG System based on price,
functionality and usability. In terms of
usability, Emotiv’s EPOC scored the
highest.
How EEG Signals are Measured: Hardware
An example of a Two-channel
EEG amplifier
An example of an
EEG device
prototype
Emotive EPOC Hardware and Software
The Emotive EPOC EEG
Headset samples the EEG
signal at a rate of 120
Samples per Second (SPS)
using an Analog-to-Digital
Converter (ADC). It samples
signals from 14 different
electrodes and filters out
artifacts, or noise, outside
the bandwidth range of 0.2 –
45 Hz. It then is
Emotiv EPOC headset Specification
How EEG Signals are Measured: Software

The Emotiv EPOC Headset
I. Expressive Suite: User facial recognition
II. Affective Suite: Monitors player emotional states in real-time
II. Cognitive Suite: Reads and interprets a player’s conscious thoughts and intent
Cognitive Suite
Expressive Suite
How EEG Signals are Measured: Software
SDK of the Research Edition
Affective Suite
GSR and HRV Biosensor
http://www.hsphysicsteacher.com/electricc
urres/13%20Electric%20Current%20&%20R
esistance/13.06%20galvanic%20skin%20resp
onse.jpg

http://www.healthy-heartmeditation.com/images/IOMTechnology.jpg
The heart rate and the galvanic skin response are very
accurate manifestations of human reaction such as
boredom, stress, and excitement. They are also very
easy to measure using non-intrusive techniques.
How GSR and HRV are Measured
Stress
HRV= Heart Rate
Variable
• Sweat
glands are
activated
HRV
• Increased
Heart
Rate
Low
resistance
pathway
GSR=Voltage/Current
Also known as Instantaneous
Skin Resistance
• Apply small
voltage to
human skin to
induce a current
What About Current Input Devices?
Mouse and Keyboard are relatively easy
to use and are recognizable by the public
 “Typing” and “Clicking” seem to be the
most efficient ways to interact with a
computer.

Multi-Biosensor Gaming System
Research by the
University of
Saskatchewan, Canada
 Combines

◦ EMG (Electrical Activation
of Muscle Tissue)
◦ GSR (Galvanic Skin
Response)
◦ HRS (Heart Rate
Variability)
◦ Respiration
◦ Temperature
◦ Eye Movement / Gaze
Source: Nacke, L.E., Kalyn, M., Lough, C., Mandryk,
R.L. 2011. “Biofeedback Game Design: Using
Direct and Indirect Physiological Control to
Enhance Game Interaction,” CHI 2011,Vancouver,
BC, Canada.
Purpose of Research
Evaluate how feasible it is to incorporate
biofeedback peripherals with current
computer gaming hardware and software
 Determine which kinds of biosensors feel
natural to what types of activities

Study
10 participants who are not affluent with
video games
 Play a Two-Dimensional Shooting Game

◦ Using only a traditional video game controller
◦ Using a controller enhanced with biofeedback
peripherals
◦ Using a controller enhanced with biofeedback
peripherals mapped to a different set of
functions
Gameplay
Increasing enemy sprite size
Movement and flamethrower length
Gameplay
Snowfall and final boss battle
Medusa’s Gaze
Testing Different Biosensors







Mechanic
Target size
Speed/jump
Weather/boss
Flamethrower
Avatar control
Medusa’s Gaze
Cond. 1
RESP
EKG
TEMP
GSR
Gamepad
Gaze
Cond. 2
GSR
EMG
EKG
RESP
Gamepad
Gaze
General Results





9 out of the 10 like the biosensors
“made for a very immersive game, out of what
is basically just a very simple platform shooter.”
“the sensors added a new dimension to the
game, and gave a greater sense of involvement.”
“variation, and more enjoyment while playing
because there are always new skills to improve
on.”
“sensors [made the] game complicated.”
More General Results
[Preferred biosensor for each activity]


“I like the idea of using multiple physiological
inputs. Distributing the functions around the
body is intuitive in some cases.”
“[It] basically boils down to an "extended
controller" where the buttons are not buttons
but other actions, similar to waving a Wii
remote.”
GSR Feedback
“I liked that it was always a challenge to
control just with my thoughts […] and forced
me to use a part of my brain I wouldn't
normally use in a video game.”
 “I disliked the fact that one of the only ways
that I found I was able to use the GSR was
by biting my lip which isn't actually all that
fun after it starts hurting.”
 -overreaction

EKG Feedback
I couldn't control as instantly as the others the effect from it tended to last over longer
periods of time.”
 “better suited to changing the game context
than what the character is doing.”

EMG Feedback
“It was fairly easy to use. It was effective and
worked.”
 “[…] having this sensor tied to
jumping/speed felt natural”
 - muscle strain during repeated use

Respiratory Feedback
“It was neat to see the immediate reaction
from my body to the game.”
 “[It] felt very natural, particularly when it was
tied to target size in the game […]. It's one I
felt I could control to a fine degree.”
 Immediate Reaction

Temperature Feedback
“It was easy to use for short periods of time
but hard to remember to breathe deep into
the sensor […].”
 “I like […] when it was tied to weather […],
because it felt like a natural thing to do. […]”
 “Breathing rapidly to increase temperature
also brought my heart rate up.”
 -intrusiveness

Gaze Feedback
“Now that was just cool […] I liked being
able to roast one frozen combatant while
immobilizing another.”
 “I found it frustrating when you would look at
a platform intending to jump on it and end
up freezing it.”

-Problems of overfocusing
Additional Feedback
“[The] muscle and breathing sensors were
simple enough that they were practically like
a new button on the controller for me, but
very awesome ones since rather than tapping
a button, it was an instinctive action.”
 “The breathing sensor and GSR sensor felt
like controllers though, because I was very
aware they were attached to me.The EKG
and EMG were completely unnoticeable and
fun to use…”

Researchers’ Conclusions
Biofeedback devices made games more
interactive and more fun
 Preferred sensors were one that were
easy to control and produces noticeable
results in real-time
 Sensors were appreciated when mapped
to activities that felt natural

Our Findings

EEG allows direct control without
physical contact
◦ Assistive Technology
◦ Accuracy Issues + set up
◦ Cost

GSR and HRV reflect users’ state of mind
◦ More intelligent gaming systems
◦ Guides gamers to feel particular emotions
◦ Personalizes each user’s gaming experience
Conclusions
Innovative
 Not enough research
 Not cost effective yet
 Unlimited potential
 Fun and “out of the box” applications of
biomedical theory

Demo
Demo
 Campus Catalyst Competition

◦ Entrepreneurship Showcase
◦ May 5, 2011 at Merrimack College
References



Nacke, L.E., Kalyn, M., Lough, C., Mandryk, R.L. 2011. “Biofeedback
Game Design: Using Direct and Indirect Physiological Control to
Enhance Game Interaction,” CHI 2011,Vancouver, BC, Canada.
[1]"Biofeedback - Definition of Biofeedback by the Free Online
Dictionary, Thesaurus and Encyclopedia." Dictionary, Encyclopedia
and Thesaurus - The Free Dictionary. Web. 26 Feb. 2011.
<http://www.thefreedictionary.com/biofeedback>.
[2]Tarvainen, M.P.; Karjalainen, P.A.; Koistinen, A.S.;ValkonenKorhonen, M.V.; , "Principal component analysis of galvanic skin
responses," Engineering in Medicine and Biology Society, 2000.
Proceedings of the 22nd Annual International Conference of the IEEE ,
vol.4, no., pp.3011-3014 vol.4, 2000
doi: 10.1109/IEMBS.2000.901513
URL:
http://xplorebcpaz.ieee.org/stamp/stamp.jsp?tp=&arnumber=90151
3&isnumber=19495