Introduction: EyeMovements and EyeTracking
Irina A. Sekerina (Higher School of Economics and CUNY)
Workshop on Reading in Cyrillic ● 21 September, 2015
1
Roadmap
A. Introduction: History, Terminology,
Background
B. Eye-Tracking Applications
C. Techniques and Equipment
D. Eye-Tracking in Psycholinguistics:
Reading
E. Models of Reading
2
A.
Introduction:
History,
Terminology,
Background
A1. Some recent history
A2. Scanpaths and visual attention
A3. The eye
A4. Types of eye movements
3
 Alfred Luk’yanovichYarbus (1965)*:
1914-1986
A1: Some
[Recent]
History
 demonstrated sequential, but variable, viewing
patterns over particular image regions
 Noton and Stark (1971)**:
 showed that participants tend to fixate
identifiable regions of interest, containing
“informative details”;
 coined term “scanpath” describing eye
movement patterns
* Ярбус, A. Л. (1965). Роль движений глаз в процессе зрения. Наука.
Yarbus, A. L. (1967). Eye Movements and Vision. New York: Plenum Press.
** Noton, D., & Stark, L. (1971). Scanpaths in eye movements during pattern perception.
Science, 171(3968), 308-311.
4
“Не ждали
[Unexpected]”
(by Ilya Repin,
1884)
5
Yarbus’ early scanpath
recording:
 1: examine at will
A2: Scanpaths
and Visual
Attention
 2: estimate wealth
 3: estimate ages
 4: guess previous
activity
 5: remember clothing
 6: remember position
 7: time since last visit
6
A2:
Scanpaths of
a portrait
7
A3: The Eye
The eye—“the world’s
worst camera”
 suffers from
numerous optical
imperfections...
 ...endowed with
several
compensatory
mechanisms
8
А3: Human
visual field
 The retina is a light
sensitive structure inside
of the eye responsible
for transforming light
into signals, which are
later converted into an
image by the visual
cortex in the brain.
 The fovea is a section of
the retina that contains
a high density of both
kinds of light receptor
cells found in the eye,
i.e. Cone and Rod cells.
9
Eye movements are mainly used to
reposition the fovea
Five main classes of eye movements:
A4: Types
of eye
movements
 Saccadic (saccades and fixations)
 Smooth pursuit
 Vergence
 Vestibular
 Physiological nystagmus
Other types of movements are nonpositional (adaptation, accommodation)
10
A4: Saccades
Rapid eye movements between fixations
used to reposition fovea
Voluntary and reflexive
Range in duration from 10 ms – 100 ms
Effectively blind during transition
Deemed ballistic (pre-programmed) and
stereotyped (reproducible)
11
A4: Fixations
Possibly the most
important type of eye
movement for
attentional
applications
 90% viewing time is
devoted to fixations
 duration: 150 ms – 600 ms
Duchowski, A. T. (2007).
Eye-Tracking Methodology.
2nd Ed. 360 p.
ISBN: 978-1-84628-808-7
12
B.
Applications
B1. Ergonomics and Human Factors
B2. Marketing and Advertising
B3. Websites
B4. Psychology, Psychophysics,
Neuroscience
13
Wide variety of eye tracking applications
exist, each class increasingly relying on
advanced graphical techniques:
B.
Applications
 Advertising
 Human Factors
 Displays
 HCI & Collaborative Systems
 Virtual Reality
 Psychology, Psychophysics,
Neuroscience
14
Applications range from usability studies
to testing effectiveness of cockpit
displays
B2:
Ergonomics
and Human
Factors
 Examples:
 evaluation of tool icon groupings
 comparison of gaze-based and mouse interaction
 organization of click-down menus
 testing electronic layout of pilots’ visual flight rules
 testing simulators for training effectiveness
15
B3:
Marketing
and
Advertising
Applications range from assessing ad
effectiveness (copy testing) in various
media (print, images, video, etc.) to
disclosure research (visibility of fine print)
Examples:
 eye movements over print media (e.g., magazines)
 eye movements over TV ads, web pages, etc.
16
B3:
Marketing
and
Advertising
Scanpaths over printed magazine ads
17
 Most people view websites
in a “F” shaped flow.
 First they scan the page at
the top, from left to right.
B3: Websites
 Then the eyes go back to
the left and down the
page.
 They again scan to the
right and back along the
same pattern.
18
B4:
Psychology,
Psychophysics, and
Neuroscience
Applications range from basic research in
vision science to investigation of visual
exploration in aesthetics (e.g., perception
of art).
Examples:
 psychophysics: spatial acuity, contrast, sensitivity
 perception: reading, natural scenery, ...
 neuroscience: cognitive load, with fMRI an ERP
 psycholinguistics
19
B4:
Perception of
Art
(Rembrandt’s
Anatomy
Lesson)
(Duchowski, 2007)
(a) Aesthetic group
(b) Semantic group
 small but visible differences in scanpaths
 similar sets of fixated image features
20
C.
Techniques
and
Equipment
C1. Electro-oculography
C2. Scleral contact lens/search coil
C3. Video-based combined pupil and
corneal reflection
21
Two broad applications of eye movement
monitoring/recording techniques:
C.
Techniques
and
Equipment
 measuring position of eye relative to the
head
 measuring orientation of eye in space, or
the “point of regard” (POR)—used to
identify fixated elements in a visual scene
The most widely used apparatus for
measuring the POR is the video-based
corneal reflection eye-tracker
22
 First method for objective eye movement
measurements using corneal reflection
reported in 1901
C1:
Techniques
 Techniques using contact lenses to improve
accuracy developed in 1950s (invasive)
 Remote (non-invasive) trackers rely on visible
features of the eye (e.g., pupil)
 Fast image processing techniques have
facilitated real-time video-based systems
23
C1: Electroоculography
Relies on measurement of
skin’s potential differences,
using electrodes placed
around the eye
• most widely used
method some 30 years
ago (still used today)
• similar to electromechanical motioncapture
• measures eye
movements relative to
head position
• not generally suitable
for POR measurement
(unless head is also
tracked)
24
C2: Scleral
Contact Lens/
Search Coil
search coil embedded
in contact lens and
electromagnetic field
frames
• possibly most
precise
• similar to electromagnetic position/
orientation trackers
used in motioncapture
25
C2: Scleral
Contact
Lens/Search
Coil
• highly accurate, but limited
Example of scleral
measurement range (~5°)
suction ring insertion:
 most intrusive
method
 insertion of lens
requires care
 wearing of lens
causes discomfort
• measures eye movements
relative to head position
• not generally suitable for POR
measurement (unless head is
also tracked)
26
C3: VideoBased
Combined
Pupil/ Corneal
Reflection
H
Head-mounted videobased eye tracker
• most suitable for
(graphical) interactive
systems, e.g., VR
• Monocular and
binocular systems
27
SMI
C3: Existing
Eye-Trackers
ISCAN
TOBII
Remote system
ASL head-mounted
ISCAN child headmounted
28
C3: Existing
Eye-Trackers
2013 next to a
student’s poster
1997 “KindergartenPath” article
29
 A light source is used to
cause reflection patterns
on the cornea and pupil
of the test person.
C3: POR
Method
 A camera will then be
used to capture an
image of the eye.
 The direction of the
gaze is then calculated
using the angles and
distances.
30
C3: How it
Works
31
D. EyeTracking in
Psycholinguistics:
Reading
D1. Visual World Eye-Tracking Paradigm
D2. Dual-Purkinjie Eye-Tracking in
Reading
D3. Examples of Eye Movements in
Reading (Tobii)
D4. Eye-Movement Parameters in
Reading
32
Cooper (1974)* and Tanenhaus et al. (1995)**
D1: The
Visual World
Eye-Tracking
Paradigm
 The Mind-Eye hypothesis
 Relationship between eye fixations and the meaning
of concurrently spoken sentence
 Using this relationship as a research tool in cognitive
psychology and psycholinguistics
 Applications:
 Speech perception and memory
 Language processing
*Cooper, R. (1974). The control of eye fixation by the
meaning of spoken language. Cognitive Psychology, 6, 84-107.
**Tanenhaus, M. K., Spivey-Knowlton, M. J., Eberhard, K. M., & Sedivy, J. C.
(1995). Integration of visual and linguistic information in spoken language
comprehension. Science, 268(5217), 1632-34.
Федорова, О. В. (2008). Методика регистрации движений глаз Визуальный мир:
шанс для сближения психолингвистических традиций. Вопросы языкознания, 6,
98–120.
33
 DPI trackers measure
rotational and
translational eye
movements
D2: DualPurkinjie
Eye-Trackers
for Reading
 Dual-Purkinje image
(DPI) eye tracker
 so-called generation-V
trackers measure the 1st
and 4th Purkinje images
 1st and 4th reflections
move together through
same distance upon eye
translation, but separate
upon eye rotation
 highly precise
 used to be expensive
and difficult to set up
34
 Purkinje images appear
as small white dots in
close proximity to the
(dark) pupil
D2: Pupil and
Purkinje
Images
 tracker calibration is
achieved by measuring
user gazing at properly
positioned grid points
(usually 5 or 9)
 tracker interpolates POR
on perpendicular screen
in front of user
Pupil and Purkinje images
as seen by eye tracker’s
camera
35
D3: Tobii
Examples of
Eye
Movements
in Reading
A gaze replay, recorded at 300Hz using
the Tobii TX300 eye tracker, of a
participant in a reading study:
https://www.youtube.com/watch?v=VBTZNydUh0w
Eye-tracking with Tobii (4-year-old child)
http://www.youtube.com/watch?v=VFIZDZwdf-0
36
D4. EyeMovement
Parameters
1. Studying eye movements per se to
learn about reading
2. Using eye movements in reading as a
means to infer cognitive processes
(e.g., language processing)
37
D4. EyeMovement
Parameters
1. Saccade latency: 150-175 ms;
2. Perceptual span: ~4 symbols to the left,
~15 symbols to the right;
3. Skipping words: 2-3-letter words are
skipped 75%, 8-letter words are never
skipped;
4. Regressions: 10-letter spaces happen
because of problems in understanding
text
38
D4. Eye
Movements
and Word
Recognition
in Reading
1.
2.
3.
4.
5.
6.
7.
Word frequency
Word familiarity
Age of acquisition
Number of meanings
Morphology
Predictability
Plausibility
39
E. Models of
Reading
E1. Types of Models of Reading
E2. E-Z Model
E3. SWIFT
40
E. Models of
Reading
Models of reading behavior try to explain
how the eye movement control system
makes two fundamental decisions
involved in reading:
 When the eyes should move
 Where the gaze should land
41
E1. Types of
Models of
Reading
1. Oculomotor control models
2. Cognitive control models
1. E-Z Reader
2. SWIFT
42
E-Z Reader
model
 http://raynerlab.ucsd.edu/
Keith_Rayner.html
E1: E-Z
Reader
Model
 Rayner, K. (1978). Eye movements in
reading and information processing.
Psychological Bulletin, 85, 618-660.
 Rayner, K. (1998). Eye movements in
reading and information processing:
20 years of research. Psychological
Bulletin, 124, 372-422
Keith Rayner (UCSD)
1943-2015
43
E2. E-Z
Reader
1. 50 ms: visual uptake phase
2. 75-100 ms: L1 phase of lexical access
3. Saccade planning
4. L2 phase of lexical access
Accounts for frequency, predictability,
spillover effects, covert attention,
importance of parafoveal preview
44
Reinhold Kliegl (University of Potsdam,
Germany): SWIFT Model
E3: SWIFT
http://www.psych.uni-potsdam.de/people/kliegl/
45
Reinhold
Kliegl’s
Lecture 1
11:30am-12:30pm
“Distributed Processing During Fixation
Durations in Reading”
46