BCS/CVS 504
Fall 2003
Motion perception
and
its neural mechanisms
Tatiana Pasternak
Departments of Neurobiology & Anatomy,
Brain & Cognitive Science,
and Center for Visual Science
Motion perception and its neural mechanisms I
• What is motion?
• Functions of motion
• Psychophysics of motion
• Early cortical processing
What is motion?
Change in position over time
90o
45o
135o
Direction
0o
180o
225o
315o
270o
Speed
degrees/second
Functions of Motion
Perceiving real moving objects
Image segmentation
Motion plays a role in pattern vision
Cue to depth
Time to collision from expanding optic flow
A stimulus to drive eye movements
Functions of Motion
Kinetic depth
Biological motion
Form from motion
Note: to see the QuickTime files that normally appear here, download the full compressed version of the presentation
from the same page from which this file was originally linked.
Functions of Motion
Optic flow
Time to collision from
expanding optic flow
Stimuli Commonly Used for Measuring Motion Perception
Sinusoidal Gratings
QuickTime™ and a Sorenson Video decompressor are needed to see this picture.
•Spatial frequency
•Temporal Frequency (speed)
•Contrast
•Noise masking
Random dots
Note: to see the QuickTime file that normally
appears here, download the full compressed
version of the presentation from the same
page from which this file was originally linked.
•∆x, spatial displacement
•∆t, temporal interval between displacements
•dot density
•% correlation
•local directions
Examining Spatiotemporal Properties of Directional Mechanisms
Detection
Direction Discrimination
vs
vs
Measure contrast threshold
Direction Discrimination Is Optimal with
Coarse Stimuli, Moving at Higher Speeds
Sensitivity Ratio
(direction/detection)
1
2 c/deg
Coarse
gratings
Fine
gratings
0.9
0.8
8 c/deg
0.7
0.6
slow
0
fast
5
10
15
Temporal Frequency (cycles/sec)
From Watson et al 1980
Discrimination of Stimulus Speeds
base speed
vs
faster
or
slower
Measure minimal discriminable difference in speed
Speed Discrimination Is Most Accurate at Higher Speeds
Speed Discrimination
(∆V/ V) x 100
20 %
15 %
10 %
5%
0%
0
5
10
15
20
Speed (deg/sec)
25
30
Properties of Motion Mechanisms
Motion perception is best at:
low spatial frequencies (coarse)
high temporal frequencies (higher speeds)
Measuring Motion Perception With Random Dots
Amplitude Spectrum of Random Dots
Random-dot stimulus
Normalized Amplitude
1
0.1
0.01
0.1
1
10
Spatial Frequency (c/deg)
Most of the energy in a random-dot stimulus is at lower spatial frequencies
Apparent (Sampled) Motion
Continuous Motion
Apparent Motion
∆x
∆x spatial displacement of elements in a display
Dmax maximal displacement seen as coherent motion
∆t
temporal interval between spatial displacements
Spatial limit of motion perception
Small displacement
of elements
Large (x 6) displacement
of elements
Qui ckTi me™ and a Animation decompr essor are needed to see thi s picture.
What is the direction of displacement?
Apparent Motion
Identify orientation of a bar
formed by correlated displacement
Spatial limit for motion
increases with eccentricity
From Baker and Braddick (1985)
Temporal limit for apparent motion: 100-150 msec
10o ecc
Dmax
5o ecc
1o ecc
interstimulus interval ∆t (msec)
From Baker and Braddick (1985)
Apparent Motion
Spatial limit for motion depends on:
Eccentricity
Spatial composition of the stimulus
Temporal limit for motion
100 - 150 msec
Measuring Complex Motion Perception
Coherence
100%
Note: to see the QuickTime file that normally
appears here, download the full compressed
version of the presentation from the same page
from which this file was originally linked.
50%
25%
Measuring perception of motion coherence
Coherence
100%
50%
Percent Correct (%)
100
Monkey 240
90
80
70
Threshold
60
4
5
6 7 8 9 10
20
25%
Motion Coherence (%)
Extracting coherent motion from random-dot stimuli
no correlation
50% correlation
Monkey
From Newsome and Pare 1988
100 % correlation
Human
From Baker et al 1999
Stimuli Used to Measure Motion Integration
Direction Range: 0 deg
Note: to see the QuickTime file that normally appears here, download the full compressed version of
the presentation from the same page from which this file was originally linked.
Stimuli Used to Measure Motion Integration
Direction Range: 240 deg
Note: to see the QuickTime file that normally appears here, download the full compressed version
of the presentation from the same page from which this file was originally linked.
Stimuli Used to Measure Motion Integration
Direction Range: 360 deg
Note: to see the QuickTime file that normally appears here, download the full compressed
version of the presentation from the same page from which this file was originally linked.
Motion integration threshold
100
Percent Correct
90
80
70
Range Threshold
60
50
0
90
180
270
330
Direction Range (deg)
360
Recording Neural Activity of a Single Cortical Neuron
Visual Field
microelectrode
Fixation spot
Visual Cortex
Note: to see the QuickTime file
that normally appears here,
download the full compressed
version of the presentation
from the same page from
which this file was originally
linked.
Recording Neural Activity of a Single Cortical Neuron
Visual Field
microelectrode
Fixation spot
Visual Cortex
Note: to see the QuickTime
file that normally appears
here, download the full
compressed version of the
presentation from the same
page from which this file was
originally linked.
Recording Neural Activity of a Single Cortical Neuron
Visual Field
microelectrode
Note: to see the QuickTime file
that normally appears here,
download the full compressed
version of the presentation from
the same page from which this
file was originally linked.
Fixation spot
Visual Cortex
Recording Neural Activity of a Single Cortical Neuron
Visual Field
microelectrode
Receptive field
Note: to see the QuickTime file
that normally appears here,
download the full compressed
version of the presentation from
the same page from which this
file was originally linked.
Fixation spot
Visual Cortex
Direction Selective Responses
Visual Field
Preferred direction
Fixation spot
400
Firing rate (spikes/sec)
Note: to see the QuickTime
file that normally appears
here, download the full
compressed version of the
presentation from the same
page from which this file
was originally linked.
200
0
0
Time (ms)
500
Direction Selective Responses
Visual Field
Anti-preferred direction
Fixation spot
400
Firing rate (spikes/sec)
Note: to see the QuickTime
file that normally appears
here, download the full
compressed version of the
presentation from the same
page from which this file
was originally linked.
200
0
0
Time (ms)
500
Directionally Selective Neuron
Preferred Direction
Anti-Preferred Direction
Firing rate (spikes/sec)
400
200
0
0
Time (ms)
500
0
Time (ms)
500
Visually Responsive Areas in the Monkey Brain
LIP
VP
Percent Selective Neurons (%)
Incidence of Directionally Selective Neurons in Monkey Cortex
100
MT
75
50
V3
25
V1
V2
0
V4
Directionally selective neurons are present in several regions of visual cortex
but are most numerous in area MT
adapted from Felleman & Van Essen (1987)
and Levitt et al 1997
Visual Pathways in Monkey Cortex
Parietal Cortex
7a
STP
VIP
LIP
MT
V1
V2
MST
V3
V4
TEO
TE
Temporal Cortex
Widespread Loss of Cortical Directional Selectivity
in Strobe-Reared Cats
100
Percent Cells
80
V1
LS cortex
normal
60
normal
40
20
strobe-reared
strobe-reared
0
Very few cortical neurons in cats reared in the environment
illuminated by an 8 Hz strobe flash are selective for stimulus
direction.
from Spear et al (1985)
from Pasternak et al (1985)
Testing Sensitivity of Directional Mechanisms
Detection
Direction Discrimination
vs
vs
Measure contrast threshold
Widespread Loss of Cortical Directional Selectivity
Affects Sensitivity for Direction Discrimination
Contrast Sensitivity
Detection
100
10
1
Direction Discrimination
Normal
Normal
Reduced DS
Reduced DS
Loss of Directional Selectivity Affects
Motion Coherence Thresholds
Direction Range (deg)
360
270
Normal
Reduced
DS
Normal
180
90
0
Reduced
DS
100%
25%
Coherence (%)
Cortical Directional Selectivity and Motion Perception
Effects of strobe-rearing
Cats reared in 8 Hz stroboscopic illumination show selective loss of over 90% of
neurons selective for motion direction.
Such animals detect moving targets at near normal contrasts but require 10 times
higher contrasts to identify their direction of motion.
Such animals show severe deficits in discriminating directions of coherent motion
masked by noise.
The residual direction sensitivity is likely to be determined by the activity of a
small number of remaining directionally selective neurons.
A small number of directionally selective cells can support normal direction
discrimination with highly visible targets (high contrast and no noise)
This result demonstrates the involvement of directionally selective neurons in
direction discrimination.
Directionally Selective Neuron in Area V1
245
90
60
135
250
255
260
70
240
45
60
75
235
55
80
230
50
180
0
315
225
265
65
215
35
270
Hawken et al 1988
Non-Directional Neuron in Area V1
270
90
90
275
95
265
280
100
260
285
105
255
290
110
250
305
125
235
85
45
135
80
75
0
180
70
55
315
225
270
This neurons is selective for stimulus orientation
but is not selective for stimulus direction
Hawken et al 1988
Direction Selectivity of V1 Neurons projecting to MT
Neurons Projecting to MT are localized to layers 4B and 6
and show directional selectivity similar to that of area MT.
Movshon & Newsome, 1998
Spatiotemporal Properties
of a Directionally Selective V1 Neuron Projecting to MT
Prefer lower spatial frequencies
Low-pass temporal frequency
response
High contrast sensitivity
4.8 Hz
0.9 c/deg
0.9 c/deg
4.8 Hz
Movshon & Newsome, 1998
Physiological Basis of Motion Perception II
Processing of motion beyond striate cortex
Direction selective neuron
Reichardt (1961) detector
Responds most strongly to one
preferred direction but little
to motion in the opposite direction
Stimulus
Receptive
field
Response
Note: to see the QuickTime
file that normally appears
here, download the full
compressed version of the
presentation from the same
page from which this file
was originally linked.
Response
Responds to luminance variations over space and time.
DS neurons collects inputs from neighboring regions.
A delay (∆t) assures that the two signals will arrive
simultaneously at the DS neuron.
Cortical Area MT
Parietal Cortex
7a
STP
VIP
LIP
MT
V1
V2
A small area (~60 mm2)
MST
Receives direct inputs from
layer 4B of V1, from the thick
stripes of V2 and from V3.
V3
Projects to MST and other
components of parietal cortex
V4
TEO
TE
Temporal Cortex
Receptive Fields in Area MT
MT neurons have medium-sized receptive
fields (5-10 fold larger than in V1) which
increase with eccentricity.
Contains a coarse retinotopic map of the
contralateral visual field.
fovea
Most MT neurons are not selective for form
but are tuned to:
direction of motion
speed of motion
binocular disparity (depth)
from Tanaka, 1998
Receptive Fields in MT Are Larger Than in V1
MT
V1
MT Neurons Are Selective for the Direction of Motion
but not Color and Orientation
Preferred direction
From Tanaka, 1998
Direction and speed selectivity in MT
Direction selectivity
MT contains neurons tuned to all directions
and a wide variety of speeds
Speed selectivity
from de Angelis
Over 90% of MT neurons are tuned
for direction and speed of motion
http://thalamus.wustl.edu/Neural_Systems_03/
Direction selectivity in MT
Note: to see the QuickTime
file that normally appears
here, download the full
compressed version of the
presentation from the same
page from which this file
was originally linked.
From Maunsell et al, 1983
J.A. Movshon
Anatomical Organization of Direction Selectivity in MT
Cortical layers
Electrode penetration
MT neurons are clustered
according to their directional
Axis of motion (degrees)
preferences.
Track distance (µm)
from Albright et al 1984
Directional Columns in MT
There is a topographic map across
the surface of MT
A region 0.5 mm x 0.5 mm
contains columns tuned to
directions covering 360o
from Albright et al 1984
Speed Tuning of an MT Neuron
From Maunsell et al 1983
Speed Tuning of four MT Neurons
From Maunsell et al 1983
Can MT neurons tuned for speed explain
psychophysical performance?
Speed Discrimination
(∆V/ V) x 100
20 %
15 %
10 %
5%
0%
0
5
10 15 20 25 30
Speed (deg/sec)
We discriminate differences in speed that are much smaller than differences
in speed leading to a change in firing rate.
It is unlikely that a single neuron can signal speed differences at threshold
Responses of MT neurons to relative motion
Background movement
Slit movement
When the surround moves in the same direction as the central bar,
many MT neurons decrease their firing.
Selectivity of MT Neurons for Relative Motion
center dots move
background dots
inhinition (%)
Normalized response (%)
stationary
Maximal inhibition occurs when
background & center dots move in the
same direction (no relative motion)
Faciliation %)
Maximal response occurs when
the center dots move (no relative motion)
center dots
move in
optimum
direction
Background
direction
varies
Direction of movement of background dots
Direction of movement of center dots
From Allman et al, 1985
Selectivity of MT Neurons for Relative Motion
Bar moves at optimum speed
background velocity varies
Inhibition (%)
Normalized response (%)
Bar moves
dots stationary
Bar velocity (deg/sec)
Background velocity (deg/sec)
Maximal inhibition occurs when background dots and the
center bar move at the same speed (no relative motion)
MT neurons signal relative motion
Neurons fire most when the motions of stimuli in the
receptive field center and surround do not match
Spatiotemporal properties of MT receptive fields
Psychophysics
Spatial limit for apparent motion
Temporal limit for apparent motion
10o ecc
Dmax
5o ecc
1o ecc
interstimulus interval ∆t (msec)
Do spatiotemporal properties of MT neurons match psychophysics?
Spatial Limit for Directional Selectivity in MT
Stimulus
Receptive
field
stroboscopic slit
∆x
preferred
null
Direction
selectivity
At large spatial displacements, MT neurons are no longer direction selective
From Mikami et al, 1986
Temporal Limit for Directional Selectivity in MT
Stimulus
stroboscopic slit
Receptive
field
At temporal intervals > 100msec, MT neurons
stop showing direction selectivity
From Mikami et al, 1986
Spatial Limit for Directional Selectivity in MT
Increases with Eccentricity
Stimulus
MT
stroboscopic slit
Psychophysical
data
V1
Spatial limits (Dmax) for direction selectivity in MT are larger than in V1
Dmax for single MT neurons and measured psychophysically is similar
From Mikami et al, 1986
Temporal Limit for Direction Selectivity
Does Not Change with Eccentricity
Stimulus
stroboscopic slit
V1
MT
Temporal limits for direction selectivity in MT and V1 are similar
From Mikami et al, 1986
Psychophysical thresholds match spatial
properties of MT neurons
MT
Motion threshold
V1
From Newsome et al 1986
Spatial & Temporal Limits for Direction Selectivity in MT
• Spatial limit (Dmax) for MT neurons increases with
eccentricity
• Temporal limit is constant (~100msec) and independent of
eccentricity
• These properties of MT neurons are consistent with
spatiotemporal limits for apparent motion measured
psychophysically
Motion integration in MT
240o
Average Activity (spikes/sec)
0o
360o
100
Preferred
80
MT neurons compute
global direction of motion
60
40
Anti-Preferred
20
baseline
activity
N=127
0
0
90
180
270
Direction Range (deg)
360
Aperture Problem and Motion Integration
Small receptive fields face aperture problem
Aperture problem
True direction of motion is not perceived
through a small aperture
Proposed solution
edge 1
edge 2
True direction of motion can be extracted with
two apertures
Solution of the aperture problem requires
integration of local directional signals
Plaid stimuli are often used to study motion integration
component
pattern
+
component
=
Perceiving pattern motion
Component gratings must match in:
Contrast
Note: to see the QuickTime
file that normally appears
here, download the full
compressed version of the
presentation from the same
page from which this file
was originally linked.
Spatial frequency
Speed
How do MT neurons respond to plaid stimuli?
V1 Neurons Projecting to MT Respond Only to the
Components of the Plaid
pattern
component
From Movshon & Newsome, 1996)
Responses to Plaids in MT
pattern
Some MT neurons respond to pattern motion.
These neurons integrate local motion signals
Some MT neurons show the same coherence thresholds
as the monkey
no correlation
behavior
50% correlation
100 % correlation
neuron
neuron
behavior
Some MT neurons show lower coherence
thresholds and some higher than the monkey
Sensitivity of Many MT Neurons is Equal to Psychophysically
Measured Sensitivity of the Monkey
Neuron more sensitive
Neuron less sensitive
from
Britten et al, 1992
Psychophysical decisions may be based on a relatively small number of neurons
This view has since been modified by Britten & Newsome (1998):
psychophysical decisions near threshold are based on activity of the
population of MT neurons
Properties of MT neurons
•
MT contains representation of the contralateral receptive field
receptive fields are localized and increase with eccentricity
•
Most MT neurons are selective for direction & speed of stimulus motion
•
Directionally selective neurons are organized in columns
•
Responses of MT neurons are strongly modulated by stimuli placed
in the surround (relative motion)
•
MT neurons respond to relatively low spatial frequency, have high
contrast sensitivity and are broadly tuned to temporal frequency
•
Spatial limits of MT receptive fields increase with eccentricity
•
Maximal temporal interval for direction selectivity is about 100ms
•
MT neurons are capable of integrating local motion vectors
•
Many properties of MT neurons parallel perceptual phenomena
•
Signals provided by MT neurons are used in the performance of motion tasks
Processing of Visual Motion in Area MST
MST
MST receives extensive inputs from MT and has two main subdivisions
MSTd (dorsal)
Most neurons have very large
receptive fields that cross the
midline.
Respond to complex patterns
of motion including translation,
rotation, expansion/contraction,
and various combinations of
these.
MSTv (ventral)
Neurons have medium to large
receptive fields
Respond to motion of objects much
like MT neurons
Receptive Fields in Area MST
Receptive fields in MST are larger than those in MT and include the fovea
Responses of a Typical MSTd Neuron
MSTd neurons prefer movement
over a wide visual field
MSTd Neurons Prefer Complex Patterns of Motion
MST neurons are thought to encode patterns of motion, or optic
flow, that are created by self-motion.
Optic flow stimuli
Focus of expansion stimuli
Note: to see the
QuickTime file that
normally appears here,
download the full
compressed version of
the presentation from the
same page from which
this file was originally
linked.
Some uses of optic flow information
Heading estimation
When you translate through your environment,
the focus of expansion can give
information about where you are headed.
Postural adjustment
optic flow gives powerful visual feedback that
can be used to adjust posture and complements
vestibular information
Responses of MSTv Neurons
Stationary center
MT cell
These neurons respond to
relative motion and occlusion
ventral MST cell
MSTv respond to relative
movement of small objects
Separate subsystems for
processing of object motion and field motion?