Chapter 4:
The Visual Cortex and Beyond
Overview of Questions
• How can brain damage affect a person’s
perception?
• Are there separate brain areas that determine
our perception of different qualities?
• How has the operation of our visual system
been shaped by evolution and by our day-today experiences?
Pathway from Retina to Cortex
• Signals from the retina travel through the
optic nerve to the
– Lateral geniculate nucleus (LGN)
– Primary visual receiving area in the
occipital lobe (the striate cortex)
– And then through two pathways to the
temporal lobe and the parietal lobe
– Finally arriving at the frontal lobe
Figure 4.1 (a) Side view of the visual system, showing the three major sites along the primary visual
pathway where processing takes place: the eye, the lateral geniculate nucleus, and the visual receiving
area of the cortex. (b) Visual system seen from underneath the brain showing how some of the nerve fibers
from the retina cross over to the opposite side of the brain at the optic chiasm.
Processing in the LGN
• LGN cells have center-surround receptive
fields.
• Major function of LGN is to regulate neural
information from the retina to the visual
cortex.
– Signals are received from the retina, the
cortex, the brain stem, and the thalamus.
– Signals are organized by eye, receptor
type, and type of environmental
information.
Figure 4.2 (a) Inputs and outputs of an LGN neuron. The neuron receives signals from the retina and also
receives signals from the cortex, from elsewhere in the thalamus (T), from other LGN neurons (L), and from
the brain stem. Excitatory synapses are indicated by Y’s and inhibitory ones by T’s. (b) Information flow into
and out of the LGN. The sizes of the arrows indicate the sizes of the signals. (Part a adapted from Kaplan,
Mukherjee, & Shapley, 1993.)
Organization in LGN
• Each LGN has six layers
– Each layer receives signals from only one eye
• Layers 2, 3, and 5 receive signals from the
ipsilateral eye
• Layers 1, 4, and 6 receive signals from the
contralateral eye
• Thus, each eye sends signals to both LGNs
and the information for each eye is kept
separated.
Figure 4.3 Cross section of the LGN showing layers. Red layers receive signals from the ipsilateral (same
side of the body) eye. Blue layers receive signals from the contralateral (opposite side) eye.
Maps: Representing Spatial Layout
• Retinotopic map - each place on the retina
corresponds to a place on the LGN
• Determining retinotopic maps - record from
neurons with an electrode that penetrates the
LGN obliquely
– Stimulating receptive fields on the retina
shows the location of the corresponding
neuron in the LGN
Figure 4.4 Points A, B, and C on the cup create images at A, B, and C on the retina and cause activation at
points A, B, and C on the lateral geniculate nucleus (LGN). The correspondence between points on the
LGN and retina indicates that there is a retinotopic map on the LGN.
Figure 4.5 Retinotopic mapping of neurons in the LGN. The neurons at A, B, and C in layer 6 of the LGN
have receptive fields located at positions A’, B’, and C’ on the retina. This mapping can be determined by
recording from neurons encountered along an oblique electrode track. Also, neurons along a perpendicular
electrode track all have their receptive fields on about the same place on the retina.
The Map on the Striate Cortex
• Cortex shows retinotopic map, too.
– Electrodes that recorded activation from a
cat’s visual cortex show:
• Receptive fields on the retina that
overlap also overlap in the cortex.
• This pattern is seen using an oblique
penetration of the cortex.
Figure 4.13 Retinotopic mapping of neurons in the cortex. When the electrode penetrates the cortex
obliquely, the receptive fields of the neurons recorded from the numbered positions along the track are
displaced, as indicated by the numbered receptive fields; neurons near each other in the cortex have
receptive fields near each other on the retina.
Neurons in Striate Cortex
• Simple cortical cells
– Side-by-side receptive fields
– Respond to spots of light
– Respond best to bar of light oriented along
the length of the receptive field
• Orientation tuning curves
– Shows response of simple cortical cell for
orientations of stimuli
Figure 4.6 (a) The receptive field of a simple cortical cell. (b) This cell responds best to a vertical bar of
light that covers the excitatory area of the receptive field. The response decreases as the bar is tilted so that
it also covers the inhibitory area. (c) Orientation tuning curve of a simple cortical cell for a neuron that
responds best to a vertical bar (orientation = 0). (From Hubel & Wiesel, 1959.)
Neurons in Striate Cortex - continued
• Complex cells
– Like simple cells
• Respond to bars of light of a particular
orientation
– Unlike simple cells
• Respond to movement of bars of light in
specific direction
Figure 4.8 (a) Response of a complex cell recorded from the visual cortex of a cat. The stimulus bar is
moved back and forth across the receptive field. The cell fires best when the bar is positioned with a
specific orientation and is moved in a specific direction (*). (From Hubel and Wiesel, 1959.) (b) Response of
an end-stopped cell recorded from the visual cortex of the cat. The stimulus is indicated by the light area on
the left. This cell responds best to a medium-sized corner that is moving up (*). (From “Receptive fields and
functional architecture in two non-striate visual areas (18 and 19) of the cat,” by D. H. Hubel and T. N.
Wiesel, 1965, Journal of Neurophysiology, 28, 229-289.)
Neurons in Striate Cortex - continued
• End-stopped cells
– Respond to:
• Moving lines of specific length
• Moving corners or angles
– No response to:
• Stimuli that are too long
Feature Detectors
• Neurons that fire to specific features of a
stimulus
• Pathway away from retina shows neurons
that fire to more complex stimuli
• Cells that are feature detectors:
– Simple cortical cell
– Complex cortical cell
– End-stopped cortical cell
Table 4.1 Properties of neurons in the optic nerve, LGN and cortical neurons
Selective Adaptation
• Neurons tuned to specific stimuli fatigue
when exposure is long.
• Fatigue or adaptation to stimulus causes
– Neural firing rate to decrease
– Neuron to fire less when stimulus
immediately presented again
• Selective means that only those neurons that
respond to the specific stimulus adapt.
Method for Selective Adaptation
• Measure sensitivity to range of one stimulus
characteristic
• Adapt to that characteristic by extended
exposure
• Re-measure the sensitivity to range of the
stimulus characteristic
Stimulus Characteristics for Selective
Adaptation
• Gratings are used as stimuli
– Made of alternating light and dark bars
– Angle relative to vertical can be changed to
test for sensitivity to orientation
– Difference in intensity can be changed to
test for sensitivity to contrast
Figure 4.9 (a) Gratings that vary in orientation. (b) A vertical grating. The contrast is high for the gratings on
the left, low for the ones on the right.
Method for Contrast Sensitivity
• Measure contrast threshold by decreasing
intensity of grating until person can just see it.
• Calculate the contrast sensitivity by taking
1/threshold.
• If threshold is low, person has high contrast
sensitivity.
Figure 4.10 Procedure for a selective adaptation experiment. See text for details.
Method for Orientation Sensitivity
•
•
•
•
•
Use a high contrast grating
Measure sensitivity to different orientations
Adapt person to one orientation
Re-measure sensitivity to all orientations
Psychophysical curve should show selective
adaptation for specific orientation if neurons
are tuned to this characteristic.
Figure 4.11 (a) Results of a psychophysical selective adaptation experiment. This graph shows that the
participant’s adaptation to the vertical grating causes a large decrease in her ability to detect the vertical
grating when it is presented again, but less effect on gratings that are tilted to either side of the vertical. (b)
Orientation tuning curve of the simple cortical cell from Figure 4.6.
Selective Rearing Experiments
• Animals are reared in environments that contain only
certain types of stimuli
– Neurons that respond to these stimuli will become
more predominate due to neural plasticity.
– Blakemore and Cooper (1970) showed this by
rearing kittens in tubes with either horizontal for
vertical lines.
– Both behavioral and neural responses showed the
development of neurons for the environmental
stimuli.
Figure 4.12 (a) Striped tube used in Blakemore and Cooper’s (1970) selective rearing experiments. (b)
Distribution of optimal orientations for 52 cells from a cat reared in an environment of horizontal stripes, on
the left, and 72 cells from a cat reared in an environment of vertical stripes, on the right (From Blakemore &
Cooper, 1970.)
Brain Imaging Techniques
• Positron emission tomography (PET)
– Person is injected with a harmless
radioactive tracer
– Tracer moves through bloodstream
– Monitoring the radioactivity measures
blood flow
– Changes in blood flow show changes in
brain activity
Brain Imaging Techniques - continued
• PET - subtraction method
– Brain activity is determined by:
• Measuring activity in a control state
• Measuring activity in a stimulation state
• Subtracting the control activity from the
stimulation activity
Figure 4.16 The subtraction technique that is used to interpret the results of brain imaging experiments.
See text for explanation.
Brain Imaging Techniques - continued
• Functional magnetic resonance imaging (fMRI)
– Hemoglobin carries oxygen and contains a ferrous
molecule that is magnetic
– Brain activity takes up oxygen, which makes the
hemoglobin more magnetic
– fMRI determines activity of areas of the brain by
detecting changes in magnetic response of
hemoglobin
• Subtraction technique is used like in PET
Figure 4.17 (a) Red and blue areas show the extent of stimuli that were presented while a person was in an
fMRI scanner. (b) Red and blue indicates areas of the brain activated by the stimulation in (a). (From
Dougherty et al., 2003.)
Maps and Columns in the Striate Cortex
• Cortical magnification factor
– Fovea has more cortical space than
expected
• Fovea accounts for .01% of retina
• Signals from fovea account for 8% to
10% of the visual cortex
• This provides extra processing for highacuity tasks.
Figure 4.14 The magnification factor in the visual system: The small area of the fovea is represented by a
large area on the visual cortex.
Organization in Columns - continued
• Visual cortex shows:
– Location columns
• Receptive fields at the same location on
the retina are within a column
– Orientation columns
• Neurons within columns fire maximally
to the same orientation of stimuli
• Adjacent columns change preference in
an orderly fashion
• 1 millimeter across the cortex represents
entire range of orientation
Figure 4.19 When an electrode penetrates the cortex perpendicularly, the receptive fields of the neurons
encountered along this track overlap. The receptive field recorded at each numbered position along the
electrode track is indicated by a correspondingly numbered square.
Figure 4.20 All of the cortical neurons encountered along track A respond best to horizontal bars (indicated
by the red lines cutting across the electrode track.) All of the neurons along track B respond best to bars
oriented at 45 degrees.
Figure 4.21 If an electrode is inserted obliquely into the cortex, it crosses a sequence of orientation
columns. The preferred orientation of neurons in each column, indicated by the bars, changes in an orderly
way as the electrode crosses the columns. The distance the electrode is advanced is exaggerated in this
picture.
Organization in Columns - continued
• Visual cortex shows (cont.)
– Ocular dominance columns
• Neurons in the cortex respond
preferentially to one eye.
• Neurons with the same preference are
organized into columns.
• The columns alternate in a left-right
pattern every .25 to .50 mm across the
cortex.
Organization in Columns - continued
• Visual cortex shows (cont.)
– Hypercolumns contain:
• A single location column
• Left and right dominance columns
• A complete set of orientation columns (0
to 180 degrees)
• This is called the “ice-cube” model.
Figure 4.22 Schematic diagram of a hypercolumn as pictured in Hubel and Wiesel’s ice-cube model. The
light area on the left is one hypercolumn, and the darkened area on the right is another hypercolumn. The
darkened area is labeled to show that it consists of one location column, right and left ocular dominance
columns, and a complete set of orientation columns.
Figure 4.24 How a tree creates an image on the retina and a pattern of activation on the cortex. See text for
details.
Figure 4.25 How the trunk of the tree pictured in Figure 4.24 would activate a number of different orientation
columns in the cortex.
Lesioning or Ablation Experiments
• First, an animal is trained to indicate
perceptual capacities.
• Second, a specific part of the brain is
removed or destroyed.
• Third, the animal is retrained to determine
which perceptual abilities remain.
• The results reveal which portions of the brain
are responsible for specific behaviors.
What and Where Pathways
• Ungerleider and Mishkin experiment
– Object discrimination problem
• Monkey is shown an object
• Then presented with two choice task
• Reward given for detecting the target
object
– Landmark discrimination problem
• Monkey is trained to pick the food well
next to a cylinder
What and Where Pathways - continued
• Ungerleider and Mishkin (cont.) - Using
ablation, part of the parietal lobe was
removed from half the monkeys and part of
the temporal lobe was removed from the
other half.
– Retesting the monkeys showed that:
• Removal of temporal lobe tissue
resulted in problems with the landmark
discrimination task - What pathway
• Removal of parietal lobe tissue resulted
in problems with the object
discrimination task - Where pathway
Figure 4.26 The two types of discrimination tasks used by Ungerleider and Mishkin. (a) Object
discrimination: Pick the correct shape. Lesioning the temporal lobe (shaded area) makes this task difficult.
(b) Landmark discrimination: Pick the food well closer to the cylinder. Lesioning the parietal lobe makes this
task difficult. (From Mishkin, Ungerleider, & Macko, 1983.)
Figure 4.27 The monkey cortex, showing the what, or ventral pathway from the occipital lobe to the
temporal lobe, and the where, or dorsal pathway from the occipital lobe to the parietal lobe. The where
pathway is also called the how pathway. (From Mishkin, Ungerleider, & Macko, 1983.)
What and Where Pathways - continued
• What pathway also called doral pathway
• Where pathway also called ventral pathway
• Both pathways:
– originate in retina and continue through two
types of ganglion cells in the LGN.
– have some interconnections.
– receive feedback from higher brain areas.
What and Where Pathways - continued
• Where pathway may actually be “How”
pathway
– Dorsal stream shows function for both
location and for action.
– Evidence from neuropsychology
• Single dissociations: two functions
involve different mechanisms
• Double dissociations: two functions
involve different mechanisms and
operate independently
Table 4.2 A double dissociation
What and How Pathways Neuropsycholgical Evidence
• Behavior of patient D.F.
– Damage to ventral pathway due to gas
leak
– Not able to match orientation of card with
slot
– But was able to match orientation if she
was placing card in a slot
– Other patients show opposite effects
– Evidence shows double dissociation
between ventral and dorsal pathways
Figure 4.29 Performance of D.F. and a person without brain damage for two tasks: (a) judging the
orientation of a slot; and (b) placing a card through the slot. See text for details. (From Milner & Goodale,
1995.)
What and How Pathways - Further Evidence
• Rod and frame illusion
– Observers perform two tasks: matching
and grasping
• Matching task involves ventral (what)
pathway
• Grasping task involves dorsal (how)
pathway
– Results show that the frame orientation
affects the matching task but not the
grasping task.
Figure 4.30 (a) Rod and frame illusion. Both small lines are oriented vertically. (b) Matching task and
results. (c) Grasping task and results. See text for details.
Modularity: Structures for Faces, Places,
and Bodies
• Module - a brain structure that processes
information about specific stimuli
– Inferotemporal (IT) cortex in monkeys
• Responds best to faces with little
response to non-face stimuli
– Temporal lobe damage in humans results
in prosopagnosia.
Figure 4.32 (a) Monkey brain showing location of the inferotemporal (IT) cortex. (b) Human brain showing
location of the fusiform face area (FFA), which is located under the temporal lobe.
Figure 4.33 Size of response of a neuron in the monkey’s IT cortex that responds to face stimuli but not to
nonface stimuli. (Based on data from Rolls & Tovee, 1995.)
Modularity: Structures for Faces, Places,
and Bodies - continued
• Evidence from humans using fMRI and the
subtraction technique show:
– Fusiform face area (FFA) responds best to
faces.
– Parahippocampal place area (PPA)
responds best to spatial layout.
– Extrastriate body area (EBA) responds
best to pictures of full bodies and body
parts.
Figure 4.35 fMRI responses of the human brain to various types of stimuli: (a) areas that were most strongly
activated by houses, faces, and chairs; (b) all areas activated by each type of stimulus. (From Alumit Ishai,
Leslie G. Ungerleider, Alex Martin, & James V.Haxby,”The representation of objects in the human occipital
and temporal cortex,” Journal of Cognitive Neuroscience, 12:2 (2000), 35-51.) © 2000 by the
Massachusetts Institute of Technology.)
Evolution and Plasticity: Neural
Specialization
• Evolution is partially responsible for shaping
sensory responses:
– Newborn monkeys respond to direction of
movement and depth of objects
– Babies prefer looking at pictures of
assembled parts of faces
– Thus “hardwiring” of neurons plays a part
in sensory systems
Evolution and Plasticity: Neural
Specialization - continued
– Experience-dependent plasticity in humans
• Brain imaging experiments show areas
that respond best to letters and words.
• fMRI experiments show that training
results in areas of the FFA responding
best to:
–Greeble stimuli
–Cars and birds for experts in these
areas
Figure 4.36 (a) Greeble stimuli used by Gauthier. Participants were trained to name each different Greeble.
(b) Brain responses to Greebles and faces before and after Greeble training. (Reprinted by permission from
Macmillan Publishers Ltd: Nature Neuroscience, 2 568-573. From Figure 1a, p. 569, from Gauthier, I., Tarr,
M. J., Anderson, A. W., Skudlarski, P. L., & Gore, J. C., Activation of the middle fusiform “face area”
increases with experience in recognizing novel objects.” 1999.)