September 4, 2014
Computer
1 Vision
Lecture 2: Vision, Attention, and Eye Movements
Stimuli in
receptive
field of
neuron
September 4, 2014
Computer
2 Vision
Lecture 2: Vision, Attention, and Eye Movements
Cat V1 (striate
cortex)
Orientation
preference
map
Ocular
dominance
map
September 4, 2014
Computer
3 Vision
Lecture 2: Vision, Attention, and Eye Movements
September 4, 2014
Computer
4 Vision
Lecture 2: Vision, Attention, and Eye Movements
Structure of NNs (and some ANNs)
• In biological systems, neurons of similar
functionality are usually organized in separate
areas (or layers).
• Often, there is a hierarchy of interconnected layers
with the lowest layer receiving sensory input and
neurons in higher layers computing more complex
functions.
• For example, neurons in macaque visual cortex
have been identified that are activated only when
there is a face (monkey, human, or drawing) in the
macaque’s visual field.
September 4, 2014
Computer Vision
Lecture 2: Vision, Attention, and Eye Movements
5
“Data Flow Diagram”
of Visual Areas in
Macaque Brain
Blue:
motion perception
pathway
Green:
object recognition
pathway
September 4, 2014
Computer Vision
Lecture 2: Vision, Attention, and Eye Movements
6
Receptive Fields in Hierarchical Neural Networks
neuron A
September 4, 2014
receptive field of A
Computer Vision
Lecture 2: Vision, Attention, and Eye Movements
7
Receptive Fields in Hierarchical Neural Networks
neuron A
in top layer
September 4, 2014
receptive field of A in input layer
Computer Vision
Lecture 2: Vision, Attention, and Eye Movements
8
September 4, 2014
Computer Vision
Lecture 2: Vision, Attention, and Eye Movements
9
Face aftereffect – Thanks to Arash Afraz for the slides!
September 4, 2014
Computer Vision
Lecture 2: Vision, Attention, and Eye Movements
10
Visual Illusions
Visual Illusions demonstrate how we perceive an “interpreted version” of
the incoming light pattern rather that the exact pattern itself.
September 4, 2014
Computer Vision
Lecture 2: Vision, Attention, and Eye Movements
11
Visual Illusions
He we see that the squares A and B from the previous image actually have
the same luminance (but in their visual context are interpreted differently).
September 4, 2014
Computer Vision
Lecture 2: Vision, Attention, and Eye Movements
12
Visual Attention
• Visual attention is the selective allocation of visual
processing resources.
• For example, we can focus our attention on a
particular object of interest in the visual field.
• Visual processing of that object is enhanced while
being rather shallow for other objects.
• Also, we can respond more quickly and accurately to
changes in an attended region.
• This prioritization is necessary due to our limited
processing resources.
September 4, 2014
Computer Vision
Lecture 2: Vision, Attention, and Eye Movements
13
Visual Attention
The attentional cueing task introduced by Michael
Posner gives insight into the dynamics of visual
attention.
Subjects are instructed to fixate on the central cross.
One of two boxes (left or right) flashes to capture the
subject’s attention (an automatic, involuntary
response).
After some a short delay (stimulus onset asynchrony SOA) an asterisk appears in one of the boxes.
The subject has to report as quickly as possible in
which box the asterisk appeared.
September 4, 2014
Computer Vision
Lecture 2: Vision, Attention, and Eye Movements
14
The Posner Attention Task
x
September 4, 2014
Computer Vision
Lecture 2: Vision, Attention, and Eye Movements
15
The Posner Attention Task
x
September 4, 2014
Computer Vision
Lecture 2: Vision, Attention, and Eye Movements
16
The Posner Attention Task
x
September 4, 2014
Computer Vision
Lecture 2: Vision, Attention, and Eye Movements
17
The Posner Attention Task
*
September 4, 2014
x
Computer Vision
Lecture 2: Vision, Attention, and Eye Movements
18
The Posner Attention Task
x
September 4, 2014
Computer Vision
Lecture 2: Vision, Attention, and Eye Movements
19
The Posner Attention Task
For short SOAs (< 200 ms), subjects respond faster if
flash and asterisk appear on the same side.
 Cueing of attention to relevant location allows
faster response.
For longer SOAs, subjects respond more slowly if
flash and asterisk appear on the same side.
 Inhibition-of-Return mechanism makes attention
less likely to remain on the side of the flash until the
asterisk appears.
September 4, 2014
Computer Vision
Lecture 2: Vision, Attention, and Eye Movements
20
Eye Movements
Eye Muscles
September 4, 2014
Computer Vision
Lecture 2: Vision, Attention, and Eye Movements
21
Types of Eye Movement
Fixations:
The eye is almost motionless, for example, while
reading a single, short word.
The information from the scene is almost entirely
acquired during fixation.
Duration varies from 100-1000 ms, typically between
200-600 ms.
Typical fixation frequency is about 3 Hz.
Fixations are interspersed with saccades.
September 4, 2014
Computer Vision
Lecture 2: Vision, Attention, and Eye Movements
22
Types of Eye Movement
Saccades:
Quick “jumps” that connect fixations
Duration is typically between 30 and 120 ms
Very fast (up to 700 degrees/second)
Saccades are ballistic, i.e., the target of a saccade
cannot be changed during the movement.
Vision is suppressed during saccades to allow stable
perception of surroundings.
Saccades are used to move the fovea to the next
object/region of interest.
September 4, 2014
Computer Vision
Lecture 2: Vision, Attention, and Eye Movements
23
Types of Eye Movement
Smooth Pursuit Eye Movements:
Smooth movement of the eyes for visually tracking a
moving object
Cannot be performed in static scenes
(fixation/saccade behavior instead)
September 4, 2014
Computer Vision
Lecture 2: Vision, Attention, and Eye Movements
24
Why Eye-Movement Research?
About eye movements and visual attention:
Usually, saccades follow shifts of attention to
provide high acuity at the attended position.
It is possible to look at an object without paying
attention to it (staring).
It is possible to shift attention without eye
movement (covert shifts of attention).
It is impossible to perform a saccade while not
shifting attention.
During specific, natural tasks it is reasonable to
assume that saccades follow shifts of attention.
September 4, 2014
Computer Vision
Lecture 2: Vision, Attention, and Eye Movements
25
Why Eye-Movement Research?
The investigation of visual attention, in turn, is at
the core of cognitive science.
• Studying visual attention yields insight into
general attentional mechanisms.
• It can provide information on a person’s stream
of conscious and unconscious processing while
solving a task.
• Attention is closely linked to the concept of
consciousness.
• Attentional mechanisms could improve artificial
vision systems.
September 4, 2014
Computer Vision
26
Lecture 2: Vision, Attention, and Eye Movements
Eye-Movement Studies
Eye movements while watching a girl’s face
(early study by Yarbus, 1967)
September 4, 2014
Computer Vision
Lecture 2: Vision, Attention, and Eye Movements
27
Eye-Movement Studies
Eye movements as
indicators of cognitive
processes (Yarbus):
trace 1: examine at will
trace 2: estimate wealth
trace 3: estimate ages
trace 4: guess previous
activity
trace 5: remember clothing
trace 6: remember position
trace 7: time since last visit
September 4, 2014
Computer Vision
Lecture 2: Vision, Attention, and Eye Movements
28
Eye-Movement Studies
Visual scan paths on instruments/dashboards – studies for the
improvement of human-computer interfaces
September 4, 2014
Computer Vision
Lecture 2: Vision, Attention, and Eye Movements
29
Eye-Movement Studies
Gaze trajectory measurement for the optimization of
web page layout
September 4, 2014
Computer Vision
Lecture 2: Vision, Attention, and Eye Movements
30
Eye-Movement Studies
Improving advertisements with eye-movement studies
September 4, 2014
Computer Vision
Lecture 2: Vision, Attention, and Eye Movements
31
Selectivity in Complex Scenes
September 4, 2014
Computer Vision
Lecture 2: Vision, Attention, and Eye Movements
32
Face Recognition
Gaze-contingent window deteriorates face recognition, allows
to identify relevant visual information.
September 4, 2014
Computer Vision
Lecture 2: Vision, Attention, and Eye Movements
33