16.899A: Physiology (contd)
Lavanya Sharan
January 24th, 2011
Before we start, a few caveats
• A lot is not known about how the human
visual system works.
• We (Alyosha + Lavanya) don’t know a lot
about physiology.
• But, before you worry, a few lines from Marr…
Slide source: Nancy Kanwisher & Jim DiCarlo
We care about ‘big picture’
• In this class, we are interested in the underlying
software/algorithm/computations
• Not in specifics of the `particular hardware’
• Want back pocket models for various components of the
human visual system
– Very few of these exist.
• Our closest cousins: computational neuroscientists/cognitive
scientists/psychophysicists
Overview of `particular hardware’
•
•
•
•
•
•
•
Retina, LGN
What are visual areas?
Tools for studying human visual system
Area V1
Beyond area V1
What and where pathways
Summary of what (and how little) we know
Primary Visual Pathway
1. Retina
2. Thalamus
– Lateral Geniculate Nucleus
(LGN)
•
divided into magno and
parvo layers
3. Primary visual cortex (V1)
4. Extrastriate visual areas
•
Each visual hemifield
projects to the opposite
hemisphere
Slide source: Jody Culham
7
Slide source: Nancy Kanwisher & Jim DiCarlo
Primary Visual Pathway
1. Retina
2. Thalamus
– Lateral Geniculate Nucleus
(LGN)
•
divided into magno and
parvo layers
3. Primary visual cortex (V1)
4. Extrastriate visual areas
•
Each visual hemifield
projects to the opposite
hemisphere
Slide source: Jody Culham
What is a Visual Area?
1. Function
– an area has a unique pattern of responses to different stimuli
2. Architecture
– different brain areas show differences between cortical
properties (e.g., thickness of different layers, sensitivity to
various dyes)
3. Connectivity
– Different areas have different patterns of connections with
other areas
4. Topography
– many sensory areas show topography (retinotopy, somatotopy,
tonotopy)
– boundaries between topographic maps can indicate
boundaries between areas (e.g., separate maps of visual
space in visual areas V1 and V2
Slide source: Jody Culham
Why are there so many visual areas?
MAGNO
• quick and dirty
PARVO
• slow and detailed
Slide source: Jody Culham
Source: Felleman & Van Essen, 1991
Source: Mapping the MInd cover image
More brain, more visual areas
Slide source: Jody Culham
Why not a really big visual area?
• As areas become larger, longer interconnections are required
• Limits on cortical thickness and connections may constrain
max area size
Slide source: Jody Culham
Parallel processing is more efficient
Teach neural network to identify
“what” and “where”
One neural network with 18
nodes (~neurons) devoted
to both tasks
versus
One neural networks with two
streams of 9 nodes each
(total = 18)
After 300 training trials, the two
stream model outperformed
the single-system model
Slide source: Jody Culham
Rueckl, Cave & Kosslyn, 1989
Different Tasks Require Different Information
• different regions may need to use different coding
systems
dorsal stream:
viewer-centred
ventral stream:
object-centred
Slide source: Jody Culham
Wiring Constraints
Source: Van Essen, 1997
David Van Essen proposes that as the brain develops, areas that are richly
interconnected will be pulled together to form a gyrus (and those that are weakly
interconnected form sulci).
Slide source: Jody Culham
Optimized Connections
Multidimensional Scaling
• strength of connections can be used to infer spatial layout
• expected layout of visual areas matches anatomy amazingly well
Parietal
Occipital
Temporal
Malcolm Young
Slide source: Jody Culham
Tools for mapping human areas
• Neuropsychological Lesions
• Temporary Disruption
• transcranial magnetic stimulation
(TMS)
• Electrical and magnetic signals
• electroencephalography (EEG)
• magnetoencephalography (MEG)
• Brain Imaging
• positron emission tomography (PET)
• functional magnetic resonance
imaging (fMRI)
Slide source: Jody Culham
Slide source: Nancy Kanwisher & Jim DiCarlo
Overview of `particular hardware’
•
•
•
•
•
•
•
Retina, LGN
What are visual areas?
Tools for studying human visual system
Area V1
Beyond area V1
What and where pathways
Summary of what (and how little) we know
Cortical Receptive Fields
Single-cell recording from visual cortex
David Hubel & Thorston Wiesel
© Stephen E. Palmer, 2002
Cortical Receptive Fields
Single-cell recording from visual cortex
Time
© Stephen E. Palmer, 2002
Cortical Receptive Fields
Three classes of cells in V1
Simple cells
Complex cells
Hypercomplex cells
© Stephen E. Palmer, 2002
Cortical Receptive Fields
Simple Cells: “Line Detectors”
B. Dark Line Detector
Firing
Rate
Horizontal Position
© Stephen E. Palmer, 2002
Cortical Receptive Fields
Simple Cells: “Edge Detectors”
C. Dark-to-light Edge Detector
Firing
Rate
D. Light-to-dark Edge Detector
Firing
Rate
Horizontal Position
Horizontal Position
© Stephen E. Palmer, 2002
Cortical Receptive Fields
Constructing a line detector
Retina
Receptive Fields
LGN
CenterSurround
Cells
© Stephen E. Palmer, 2002
Cortical Receptive Fields
Complex Cells
STIMULUS
NEURAL RESPONSE
00o
Time
© Stephen E. Palmer, 2002
Cortical Receptive Fields
Complex Cells
STIMULUS
NEURAL RESPONSE
o
60
0
Time
© Stephen E. Palmer, 2002
Cortical Receptive Fields
Complex Cells
STIMULUS
NEURAL RESPONSE
o
90
0
Time
© Stephen E. Palmer, 2002
Cortical Receptive Fields
Complex Cells
STIMULUS
NEURAL RESPONSE
o
120
0
Time
© Stephen E. Palmer, 2002
Cortical Receptive Fields
Constructing a Complex Cell
Retina
Receptive Fields
Cortical Area V1
Simple Cells
© Stephen E. Palmer, 2002
Cortical Receptive Fields
Hypercomplex Cells
© Stephen E. Palmer, 2002
Cortical Receptive Fields
Hypercomplex Cells
© Stephen E. Palmer, 2002
Cortical Receptive Fields
Hypercomplex Cells
© Stephen E. Palmer, 2002
Cortical Receptive Fields
Hypercomplex Cells
“End-stopped” Cells
© Stephen E. Palmer, 2002
Cortical Receptive Fields
“End-stopped” Simple Cells
© Stephen E. Palmer, 2002
Cortical Receptive Fields
Constructing a Hypercomplex Cell
RETINA
Receptive Fields
CORTICAL AREA V1
Complex Cell
End-stopped Cell
© Stephen E. Palmer, 2002
Overview of `particular hardware’
•
•
•
•
•
•
•
Retina, LGN
What are visual areas?
Tools for studying human visual system
Area V1
Beyond area V1
What and where pathways
Summary of what (and how little) we know
Overview of visual areas
Logothetis 1999; from http://psychology.uwo.ca/fMRI4Newbies/RetinotopicandEarlyVisualAreas.html
Macaque & human visual areas are similar
Tootell et al. 2003; from http://psychology.uwo.ca/fMRI4Newbies/RetinotopicandEarlyVisualAreas.html
Slide source: Nancy Kanwisher & Jim DiCarlo
Retinotopy
(Tootell et al. 1982)
Adjacent parts of visual field are mapped to adjacent parts of cortex.
Not all visual areas have retinotopy, may be graded.
Slide source: Nancy Kanwisher & Jim DiCarlo
Slide source: Nancy Kanwisher, Jim DiCarlo, David Heeger
Slide source: Nancy Kanwisher, Jim DiCarlo, David Heeger
Two visual pathways
The two visual processing streams for different visual percepts:
“What” (ventral stream)- object recognition
• main input from “slow and detailed” parvo system
“Where” or “How” (dorsal stream) - spatial perception, motor planning
• main input from “quick and dirty” magno system
Slide source: Jody Culham
Source: Mishkin & Ungerleider, 1982
Two visual pathways
The two visual processing streams for different visual percepts:
“What” (ventral stream)- object recognition
• main input from “slow and detailed” parvo system
“Where” or “How” (dorsal stream) - spatial perception, motor planning
• main input from “quick and dirty” magno system
Slide source: Jody Culham
Source: Mishkin & Ungerleider, 1982
The “What” Pathway
Other Visual Areas
• contain more
complex receptive
fields
Temporal Lobe
• contains many
specialized areas for
recognizing various
things
body motion
faces
places
bodies
objects
Slide source: Jody Culham
The “Where” or “How” Pathway
grasping and reaching
attention
head movements
Parietal Lobe
• contains many specialized areas
for using vision to guide actions in
space
motion
perception
eye movements
Slide source: Jody Culham
Slide source: Nancy Kanwisher & Jim DiCarlo

Summary
• Low-level areas
Filter banks, SIFT, HOG for color, orientation,
spatial frequencies, motion…

• High-level areas
Desired output from computer vision systems
e.g., segmentation, robust object/scene/texture recognition,
motion understanding and planning…


• Middle-level area
Where the magic happens
– No one (neuroscientists, psychologists, computer scientists, etc.)
really understands this stage of processing.
more, come find us for pointers to papers/books/readings and
• For
people to talk to.
