Research Methods in
Human-Computer Interaction
Chapter 13
Measuring the
Human
©2010 John Wiley and Sons
www.wileyeurope.com/college/lazar
Chapter 13
Participant as data source
•
•
•
•
Physical and emotional measurements
can give significant into how we use
computers
Possibly data that would be hard to get
in other ways
Many approaches
Range of cost, complexity, and
invasiveness
©2010 John Wiley and Sons
www.wileyeurope.com/college/lazar
Chapter 13
Examples
•
•
•
Mouse/keyboard
Eye-tracking
Measurements of physical and
emotional responses
–
•
•
•
Galvanic skin response, blood volume,
heart rate measurement
EKG/EMG
Brain scan? FMRI
Cost-benefit tradeoffs
©2010 John Wiley and Sons
www.wileyeurope.com/college/lazar
Chapter 13
Eye-Tracking
•
•
•
•
•
Where does the user look?
Where does she expect to find data?
When is she lost/confused?
Cameras or other sensors track position
or orientation of eyes or other parts of
body
Transform raw data into detailed
descriptions of “paths” of eye gaze
©2010 John Wiley and Sons
www.wileyeurope.com/college/lazar
Chapter 13
Eye-Gaze Challenges
•
Eyes constantly in motion
–
–
•
“Dwell” - relatively little motion indicating
focus on a target
–
•
Rapid eye motions – saccades – help us
focus
Larger motions indicate change of focus
Thresholds for identifying a “dwell”?
Can be applied at larger scale
©2010 John Wiley and Sons
www.wileyeurope.com/college/lazar
Chapter 13
Head tracking for large displays and
virtual environments
Ball, North, & Bowman, 2007
©2010 John Wiley and Sons
www.wileyeurope.com/college/lazar
Chapter 13
Challenges
•
•
•
Expense of equipment
Low-cost alternatives based on USB
webcams may be possible
Hard to interpret
–
–
•
Where you are looking is useful
But it must be coordinated with what you
are looking at
Overlay trail on screen shot indicating
path of user gaze
©2010 John Wiley and Sons
www.wileyeurope.com/college/lazar
Chapter 13
Web Page with Eye-tracking
trails
Card, et al, 2001
©2010 John Wiley and Sons
www.wileyeurope.com/college/lazar
Chapter 13
Uses for eye-gaze
•
Pointing and selecting
–
•
Impact of interface design
–
–
•
Assistive technology
Placement of target in list of links
Length of text summaries for search
results
Eye movement during menu selection
–
Fixation vs. movements in specific
directions - “sweeps”
©2010 John Wiley and Sons
www.wileyeurope.com/college/lazar
Chapter 13
Dimensions of Eye-Tracking Designs
•
Hypothesis-driven experiments
–
–
•
Well-defined tasks, measures, etc.
Do users look at security indicators on
web browsers? (Whalen & Inkpen, 2005)
Exploratory, open-ended research
–
“Let's see what they look at”?
©2010 John Wiley and Sons
www.wileyeurope.com/college/lazar
Chapter 13
Physiological Tools
•
•
Bodies change behavior with stimuli
Measurable differences when we are
–
–
–
•
Excited
Frustrated
Aroused
Measurements from bodies can be used
to understand these responses
©2010 John Wiley and Sons
www.wileyeurope.com/college/lazar
Chapter 13
Psychophysiology
•
•
Combination of physiological signals
with traditional study of task
performance and subjective responses
Goal: get fine-grained assessments of
subjective responses
–
–
As they happen, rather than later (via
surveys)
Replace recall with measurement,
subjective response with quantitative
measurement
©2010 John Wiley and Sons
www.wileyeurope.com/college/lazar
Chapter 13
Types of Physiological Data
•
Current flow
–
•
•
•
every cell is an electrical system
Blood flow
Heart Rate
Respiration Rate
©2010 John Wiley and Sons
www.wileyeurope.com/college/lazar
Chapter 13
Two Classes of Sensors
•
Electrodes
–
–
•
Direct measurements of electrical
signals
Galvanic skin response
Transducers
–
–
Convert mechanical or physical
measurements to electrical form
Pressure sensors to measure posture
©2010 John Wiley and Sons
www.wileyeurope.com/college/lazar
Chapter 13
Challenges
•
•
No single notion of arousal
Stimulus-Response specificity
–
•
Certain stimuli may lead to increases in
some measures alongside decreases in
others
Need to understand the range of
potential responses that might be
associated with stimuli used in your
tasks.
©2010 John Wiley and Sons
www.wileyeurope.com/college/lazar
Chapter 13
Where are measurements made?
•
•
•
•
•
•
•
Electrodermal Activity/Galvanic Skin
Response – fingers/toes
Blood volume pressure – finger
Electrocardiography (EKG) – chest/abdomen
Respiration – Thorax
Muscular/Skeletal Positioning – Varied
Electromyography (EMG) – Jaw/Face
Electroencephalography (EKG)/Evoked
responses - head
©2010 John Wiley and Sons
www.wileyeurope.com/college/lazar
Chapter 13
Electrodermal Activity/
Galvanic Skin Response (GSR)
•
•
•
Measurement of flow of electricity
through the skin
Pair of electrodes on fingers
Conductance levels linked to fear,
sadness, arousal, cognitive activity, and
frustration
©2010 John Wiley and Sons
www.wileyeurope.com/college/lazar
Chapter 13
Galvanic Skin Response (GSR)
Sensors
•
Thought Technology
(http://www.thoughttechnology.com/sensors.htm)
©2010 John Wiley and Sons
www.wileyeurope.com/college/lazar
Chapter 13
Cardiovascular
•
Stimuli may cause complex changes in
heart behavior
–
•
Variability in heart rate, blood pressure,
and blood-volume pressure
Used to measure stress, mental effort,
fear, happiness, and anger
©2010 John Wiley and Sons
www.wileyeurope.com/college/lazar
Chapter 13
Blood Volume Pressure (BVP)
•
•
•
•
Light-absorption properties of capillaries
in finger
Changes in pressure lead to changes in
reflected light
Correlate with stimuli that provoke
anxiety
Also can be used to infer changes in
heart rate
©2010 John Wiley and Sons
www.wileyeurope.com/college/lazar
Chapter 13
Electrocardiography (EKG)
•
•
Source: electrical current that causes
heart to beat
Measure
–
–
–
Heart rate
Interval between beats
Heart-rate variability
©2010 John Wiley and Sons
www.wileyeurope.com/college/lazar
Chapter 13
Respiration
•
•
Arousal makes us breather faster
Some emotions can cause irregular
breathing
•
Use sensors attached to thorax or
integrated into clothing
•
Measure expansion and contraction of
chest
©2010 John Wiley and Sons
www.wileyeurope.com/college/lazar
Chapter 13
Muscular and Skeletal Position
Sensing
•
•
How do we sit? How do we move?
Various sensors measure different types
of movement and positioning
–
–
–
–
–
•
Fiber optics
Accelerometers
Computer vision
Foam sensors in clothing
Pressure Sensors
Nintendo Wii..
©2010 John Wiley and Sons
www.wileyeurope.com/college/lazar
Chapter 13
Electromyography (EMG)
•
•
•
•
Measure muscle tension
Measurements on jaw
– Tensions associated with clenching
Eyebrows or cheeks
– Frowns or smiles
– Mild positive emotions lead to lower
readings over the eyebrow and higher
over cheek
Also sadness, fear, happiness
©2010 John Wiley and Sons
www.wileyeurope.com/college/lazar
Chapter 13
Brain Activity
•
•
•
•
Brain imaging – detailed displays, but
expensive
Electroencephalography (EEG)
– Helmet with electrodes on scalp
Functional MRI (fMRI) – identify regions of the
brain activated in response to specific stimuli
Functional near-infrared spectroscopy (fNIRS)
– measure reflective characteristics of skull,
scalp, and brain)
– Measure cognitive load in tasks?
©2010 John Wiley and Sons
www.wileyeurope.com/college/lazar
Chapter 13
Data Collection Challenges
•
Electrodes and sensors can be difficult
to use correctly
–
–
•
Get proper training
Work with an experienced professional
May cause some discomfort and
unease
–
–
Electrodes should only be attached by
someone of the same gender?
Be explicit in informed consent (Chapter
14)
©2010 John Wiley and Sons
www.wileyeurope.com/college/lazar
Chapter 13
More Data Collection Challenges
•
Electromyography needles placed in
skin
–
–
Safe, but “Don't try this at home”
Use electrodes placed on skin instead
©2010 John Wiley and Sons
www.wileyeurope.com/college/lazar
Chapter 13
Data Interpretation Challenges
•
Signals are very noisy
–
–
•
•
Distortion
Interference
Comparison to individual “baseline”
measurements necessary
Magnitude of responses may be
influenced by baseline
–
Small increase in heart beat if heart is
already beating quickly
©2010 John Wiley and Sons
www.wileyeurope.com/college/lazar
Chapter 13
More Interpretation Challenges
•
•
Habituation – response to stimulus
decreases after repeated presentation
Use signal processing techniques to
filter and “clean” data
–
•
But, you need to know how to use them
Data granularity
–
–
Averages OK for overall impressions
Specific responses require synchronize
measurement stream with user actions
©2010 John Wiley and Sons
www.wileyeurope.com/college/lazar
Chapter 13
Synchronization Challenges
•
•
•
Multiple data streams
Actions on the computer (log file)
Timing of physiological measurements
–
•
Often collected on a second computer..
Modified mouse with signals to two
computers?
©2010 John Wiley and Sons
www.wileyeurope.com/college/lazar
Chapter 13
Example Experimental setup
•
•
•
•
•
•
Two computers: one for stimulus, one
for recording physiological
measurements
Mouse with two outputs
Additional display for clock
Video camera
Blood Volume Pressure and Galvanic
Skin Response sensors
Scheirer, et al., 2002
©2010 John Wiley and Sons
www.wileyeurope.com/college/lazar
Chapter 13
Example Experimental Setup
•
Scheirer, et al., 2002
©2010 John Wiley and Sons
www.wileyeurope.com/college/lazar
Chapter 13
Analysis Challenges
•
•
Modeling to classify types of
responses?
Determining the emotional state
associated with a response?
–
•
•
Happiness? Sadness? Disgust? Fear?
Data mapping from measured signals to
emotional states is inconclusive
Mixed or incomplete signals?
©2010 John Wiley and Sons
www.wileyeurope.com/college/lazar
Chapter 13
Design Questions?
•
Do the benefits of physiological
measurement outweigh the costs?
•
Would easier methods (post-task
questionnaires & observations) provide
data that is almost as good, at a lower
cost?
©2010 John Wiley and Sons
www.wileyeurope.com/college/lazar
Chapter 13
Example: Multi-modal Input
•
Galvanic Skin Response (GSR) to test
emotional response to speech, gesture, and
multimodal interfaces
•
Response levels lowest for multimodal, then
speech, gestures
•
Total response increases with task complexity
•
GSR peaks correlated with stressful or
frustrating events
(Shi, et al. 2007)
©2010 John Wiley and Sons
www.wileyeurope.com/college/lazar
Chapter 13
Example: Video Games
•
GSR, EKG, cardiovascular rate, respiration,
facial EMG
•
Measure responses to computer games
played against computer and against a friend
•
Playing against a friend was more exciting
– Most had higher GSR and facial EMG
– No differences on cardiovascular and
respiratory measures
(Mandryk & Inkpen, 2004)
©2010 John Wiley and Sons
www.wileyeurope.com/college/lazar
Chapter 13
End-of-chapter
• Summary
• Discussion questions
• Research design exercise
©2010 John Wiley and Sons
www.wileyeurope.com/college/lazar
Chapter 13