Perceptual systems:
Central visual pathways
Kandell et. al., Princ. of Neural Sci.
Outline
A. Subcortical pathways (retina, LGN)
– Magno- & Parvo- streams
B.
Primary visual cortex (V1)
– Partial integration of M and P
– Gateway to Ventral/Dorsal
C.
Ventral pathway (V4, IT)
– Object processing
D. Dorsal pathway (MT, IP)
– Motion processing
E.
Summary
Retinal ganglion cells (RGCs):
Quick Summary
• RGC’s
– Output of the retina
(“optic nerve”)
Retinal ganglion cells (RGCs):
Quick Summary
• RGC’s
– Output of the retina
(“optic nerve”)
– Center-surround
receptive fields
Kandell et. al., Princ. of Neural Sci.
RGCs: Two principal cell-types
Midget RGC receptive field tiling
Parasol RGCs receptive field tiling
Send axons to LGN to form
Parvocellular (‘P’) pathway
Send axons to LGN to form
Magnocellular (‘M’) pathway
RGCs: Two principal cell-types
Midget RGC receptive field tiling
Parasol RGCs receptive field tiling
Send axons to LGN to form
Parvocellular (‘P’) pathway
Send axons to LGN to form
Magnocellular (‘M’) pathway
RGCs: Two principal cell-types
Midget RGC receptive field tiling
P pathway characteristics:
-Smaller RFs, so better spatial resolution
-Selective for color (red-green opponency)
Parasol RGCs receptive field tiling
M pathway characteristics:
-Better temporal resolution
-More sensitive to low light levels
-Monochromatic (black-white opponency)
Lateral geniculate nucleus (LGN)
• Thalamus is the meeting
point of sensory organs
• LGN is the visual portion:
• Receives direct input
from RGC cells
Lateral geniculate nucleus (LGN)
• LGN receives direct input
from RGC cells
• Receptive fields have very
similar RFs to their RGC
inputs:
– Circular
– Center-surround
• Reflected in anatomy…
Kandell et. al., Princ. of Neural Sci.
Optic chiasm
RGCs in left hemifield send
axons to right LGN, and
vice-versa
Kandell et. al., Princ. of Neural Sci.
LGN lamina: 3 layers for each eye
-Contralateral = the eye
at the opposite side of
brain
-Ipsilateral = the eye at
the same side of the
brain
-Contra and ipsi inputs
are segregated by LGN
layers
Parvocellular (P channel)
Kandell et. al., Princ. of Neural Sci.
Magnocellular (M channel)
LGN lamina: 4 Parvo & 2 Magno layers
Inputs from RGC Midget cells:
-Smaller RFs
-Selective for Color
Inputs from RGC Parasol cells:
-Larger RFs
-Monochromatic
-Respond to faster changes
Parvocellular (P-channel)
-Similar RFs to Midget inputs
Magnocellular (M-channel)
-Similar RFs to Parasol inputs
Kandell et. al., Princ. of Neural Sci.
LGN lamina: 4 Parvo & 2 Magno layers
All combinations
transmitted to cortex:
M-Contra, M-Ipsi,
P-Contra, P-Ipsi
Kandell et. al., Princ. of Neural Sci.
Subcortical pathways summary:
• Unique cell-types in the retina form the
first stages of 2 major parallel pathways:
Magno- and Parvo• M and P pathways carry unique spatiotemporal-chromatic information
• LGN RFs are very similar to those in the
retina (one reliable synapse from RGC)
• 6 layers in the LGN to maintain
segregation of eye input and M/P input.
• Subcortical pathways are trying to take
basic features about the visual scene
and reliably transmit this information to
cortex.
Outline
A.
Subcortical pathways (retina, LGN)
– Magno- & Parvo- streams
B.
Primary visual cortex (V1)
– Partial integration of M and P
– Gateway to Ventral/Dorsal
C.
Ventral pathway (V4, IT)
– Object processing
D. Dorsal pathway (MT, IP)
– Motion processing
E.
Summary
Kandell et. al., Princ. of Neural Sci.
Laminar architecture of cortex
• The primate cortex is ~2mm
thick
• 6 layers within this thickness
Laminar architecture of cortex
Outputs from
layers II/III
• The primate cortex is ~2mm
thick
• 6 layers within this thickness
• Feedforward circuit:
– Layer IV, then layer II/III, then
next cortical area
Inputs to
layer IV
Laminar architecture of V1
Outputs from
layers II/III
V1
Inputs to
layer IV
Kandell et. al., Princ. of Neural Sci.
Laminar architecture of V1
V1
Spatial-temporal-chromatic
information of visual world is still
segregated anatomically within the
input layers of cortex: i.e. Parallel
pathways are still segregated
Kandell et. al., Princ. of Neural Sci.
Laminar architecture of V1
V1
Finally, this information is partially
combined once it hits the output
layers of V1.
Spatial-temporal-chromatic
information of visual world is still
segregated anatomically within the
input layers of cortex: i.e. Parallel
pathways are still segregated
Kandell et. al., Princ. of Neural Sci.
V1 ocular dominance
• Eye inputs are also
segregated at the input
layer of V1
• A neuron’s response
preference for one eye
over another is called
“ocular dominance”
• In V1, there are
alternating bands of
ocular dominance
across the cortical
surface… “map”
1mm
LGN
V1
Kandell et. al., Princ. of Neural Sci.
Primary visual cortex (V1):
Orientation tuning arises
• LGN inputs are not tuned for
edges
• V1 neurons, one synapse from
LGN, have tuning for edges of a
particular orientation:
“orientation tuning”
V1 orientation tuning
Stimulu
s
Response
V1 orientation tuning
• Classic Hubel & Weisel model:
Orientation tuning arises from
an alignment of LGN inputs
LGN
V1
Kandell et. al., Princ. of Neural Sci.
V1 orientation maps
Similar tuning
within column
• Columnar organization:
similar response
properties at all depths
of a vertical penetration
V1 orientation maps
• Tuning progresses
smoothly across
horizontal penetration
V1 orientation maps
1mm
Kandell et. al., Princ. of Neural Sci.
V1 orientation maps
• Continuous maps
of orientation
preference:
“pinwheels”
• Consistent
preference though
the depth of
cortex: columnar
architecture
Kandell et. al., Princ. of Neural Sci.
V1 retinotopic maps
• Each point of the visual
field maps on to a local
group of neurons in V1.
• Retinotopy = Remapping
of retinal image onto
cortical surface
• Foveal region uses more
of V1 (greater
magnification factor)
(V1)
Classic V1 maps: summary
Ocular dominance map
Orientation map
Retinotopic map
1mm
Why have maps? How are they created?
Kandell et. al., Princ. of Neural Sci.
Classic V1 maps: summary
Ocular dominance map
Orientation map
Retinotopic map
1mm
Why have maps? How are they created?
Nobody knows for certain Kandell et. al., Princ. of Neural Sci.
Primary visual cortex (V1):
Summary
• Pathways segregated in LGN are still
segregated at input layers of V1
– M & P separated by sublayers in IVC
– Eye inputs segregated into columns
• Tuning for edges arises at first synapse:
orientation selectivity.
• Maps: orientation, ocular dominance,
retinotopy.
Preview to higher areas
• V1 is the “trunk” of the visual cortex
hierarchy: V1 output forms dorsal and
ventral streams.
• M-to-Dorsal (mostly)
• P-to-Ventral (mostly)
Outline
A.
Subcortical pathways (retina, LGN)
– Magno- & Parvo- streams
B.
Primary visual cortex (V1)
– Partial integration of M and P
– Gateway to Ventral/Dorsal
C.
Ventral pathway (V4, IT)
– Object processing
D. Dorsal pathway (MT, IP)
– Motion processing
E.
Summary
Kandell et. al., Princ. of Neural Sci.
Ventral pathway:
Preview
• Two principal paths:
1) V1-V2-V4-TEO-IT
2) V1-V4-TEO-IT
• Involved in object identification.
Called the “what” pathway.
• Late stages of inferior temporal
(IT) cortex contain the celebrated
“face cells”
Kandell et. al., Princ. of Neural Sci.
Ventral pathway: Receptive
field hierarchy
• Subcortical pathways compute
local differences
• V1 identifies edges
• V1 projects to V2 & V4: What
might V4 neurons represent?
?
Ventral pathway: Receptive
field hierarchy
• Subcortical pathways compute
local differences
• V1 identifies edges
• V1 projects to V2 & V4: What
might V4 neurons represent?
– Curvature, contour integration,
foreground/background.
Ventral pathway
IT represents complex objects, such as faces
Desimone et. al. 1984
Ventral pathway: Convergence of
inputs at each stage
• We don’t know how this
selectivity is built from the
circuitry, but we have very
general (i.e. crude) ideas.
• Each region receives
convergent input from the
previous region (e.g. V1-toV4), thus progressively
increasing the size and
complexity of receptive fields
V4
V1
Ventral pathway: Convergence of
inputs at each stage
Ventral pathway: Convergence of
inputs at each stage
What happens when
we scale or rotate the
image?
Ventral pathway: Convergence of
inputs at each stage
Simple models predict that
the IT neurons will no longer
respond, but instead IT
neurons demonstrate scale,
shift, and rotational
invariance.
i.e. IT responds to particular
objects regardless of simple
transformations.
Ventral pathway: Convergence of
inputs at each stage
-Lots of models attempting to explain
object recognition in the ventral
pathway, particular invariance (many
useful engineering applications):
-None of them come close to the
performance of the primate visual
system.
200 ms to IT spikes;
Also human response time
?
Ventral Pathway: Summary
• Receives input from both M and P subcortical
pathways, but mostly P inputs.
• Progressive increase in the complexity of
spatial selectivity. Neurons are ultimately
responsive to holistic objects in IT.
• IT neurons also exhibit well-known scale, shift,
rotation invariance. How does this happen?
Kandell et. al., Princ. of Neural Sci.
Outline
A.
Subcortical pathways (retina, LGN)
– Magno- & Parvo- streams
B.
Primary visual cortex (V1)
– Partial integration of M and P
– Gateway to Ventral/Dorsal
C.
Ventral pathway (V4, IT)
– Object processing
D. Dorsal pathway (MT, IP)
– Motion processing
E.
Summary
Kandell et. al., Princ. of Neural Sci.
Dorsal pathway
(preview)
• The “where” pathway
• Main input from M-pathway
circuits within V1.
• V1-to-MT-to-IP
• MT (aka V5) neurons are highly
selective to direction of motion.
• IP neurons tell the motor system
how to respond to the
environment.
Kandell et. al., Princ. of Neural Sci.
Medial Temporal Area (MT):
Motion Selectivity
Motion selectivity seen at 2
stages of dorsal hierarchy:
1) Neurons in layer IVB of V1
2) Neurons in MT (IVB of V1
is major MT input)
So what is MT doing??
IVB of V1
MT
Both stages are direction selective
Medial Temporal Area (MT)
computes global motion
You could easily identify the
motion of this object.
Born & Bradley., Annu Rev Neurosci
Medial Temporal Area (MT)
computes global motion
Apparent motion
Actual motion
-Now, it would appear that movement
is in the direction perpendicular to the
unmasked edge.
-This is what a V1 neuron “sees”. V1
neurons only detect local motion.
Born & Bradley., Annu Rev Neurosci
Medial Temporal Area (MT)
computes global motion
Apparent motion
Actual motion
-MT neurons have much larger
receptive fields than V1, so they can
integrate multiple cues in the image.
MT neurons can detect global motion
of the object.
Parietal cortex:
A ridiculously brief summary
• MT passes its output to the
intraparietal cortical areas. Each
IP area is responsible for a
different “class” of movement.
• In addition to unparalleled
object recognition (ventral
pathway), primates have great
hand-eye coordination so that
we can use tools, or hit a
baseball (dorsal pathway).
• IP uses visual input to tell the
motor system how to interact
with the environment.
Differences between dorsal and
ventral pathway, summarized by lesion
examples
• Patients with lesions of the parietal cortex
cannot properly adjust their grip (width or
angle) to grab an object. But they can
describe the shape of the object in detail.
• Patients with lesions of the ventral path
cannot describe the object but can easily
adjust their grip to grab it.
Summary
Kandell et. al., Princ. of Neural Sci.
Things I ignored (to name a few)
• Many aspects of the circuitry
– Local cortical circuits
– Intercortical feedback
• Contextual influences
• Attention
Much we still do not know
• We do not have a descriptive or mechanistic
model that predicts response properties of
downstream visual areas, or behavior.
• A descriptive model would vastly transform
technology: the primate visual system is far
superior to anything that engineers can build.
• A mechanistic model is the ultimate goal (a
more difficult one) as this would allow us to
address neurological disorders systematically.
Thats all!
Ventral pathway: Convergence of
inputs at each stage
IT