Early Vision
Bruce Draper
Department of Computer
Science
Colorado State University
Overview

“The Biomimetic Vision Trilogy”
1.
Selective Attention
–
–
2.
Early Vision
–
–
3.
Understanding the problem
Last week
Understanding the literature
Today
Ventral vision
–
–
Understanding object recognition
March 9 (next week)
General Theme

Vision evolved to serve the needs of animals
–
Vision is action oriented (it guides behavior)


–
Actions may be immediate (e.g. grasp, navigate)
Actions may be delayed (“perception”)
Vision is not one system

As animals became more complex, more and more visual
capabilities evolved in separate systems
Note: this is not a new idea. See The Visual Brain in Action by
Milner & Goodale 1996; The Metaphorical Brain by Arbib 1972; or
even Cybernetics by Weiner 1948.
Input: The Eye(s)
Start at the beginning:
 Lens focuses light
 Iris serves as aperture
 Retina contains receptors
 Optic nerve transmits to brain
S. Palmer. Vision Science. P. 27
Lens, iris are controlled by muscles under the control of
the brain
Retina as
Processor
• Five cell types:
• receptor (rod/cones)
Species dependent
• horizontal
• bipolar cells
• amacrine cells
• ganglion cells
• Its inside out!
• Blind spot where optic
nerve passes through
retina
S. Palmer. Vision Science. P. 30
Retina (cont.)

Ganglion Cells
–
The first cells to produce spike discharges


–
–
Other retinal cells use graded potentials
Spikes are needed for long distance communication
On-center off-surround
Off-center on-surround
Two Types of Ganglion Cells

P ganglion cells in primates (like Y cells in cats):
–
–
–
–

Large receptive fields (low frequency?)
Transient response
Fast transmission
Receive inputs from all colors and from rods
P ganglion cells (like X cells in cats):
–
–
–
–
Small receptive fields (high frequency?)
Sustained response
Medium transmission
Color opponent channels
Fields of View & Stereo

Right hemisphere
receives the left visual
field from both eyes
–
–

And vice-versa
Splitting the field of view
supports disparity
computations
High resolution in fovea,
lower elsewhere
–
Fovea is ±2° (thumbnail at
arms length)
Visual Projections


The eye + optic nerve is a shared device
There are eleven projections (“endpoints”) of the
optic nerve:
–
Retinogeniculate Projection


–
Onto LGNd (Lateral Geniculate Nucleus, dorsal) and then to V1
(a.k.a. primary visual cortex, striate cortex)…
This path is dominant in people; barely evident in nonmammals
Retinotectal Projection



Onto Superior Colliculus, then the Pulvinar Nucleus, then LGN,
V1, MT, higher level centers…
This path is dominant in non-mammals; evolutionarily older
Involved in eye movements, motion, tracking
Projections (LGN & S.C.)
S. Palmer. Vision Science. P. 25.
Projections (II)

At least 8 more (minor) projections!
–
Retina Suprachiasmatic Nucleus



–
Retina Nucleus of the Optic Tract (NOT)

–
Optokinetic nystagmus
Retina  Accessory Optic Nuclei

–
Circadian rhythms
SN is part of the hypothalamus (like LGNd)
SN receives multi-modal projections
Visual control of posture, locomotion
Retina  Pretectum

Pupillary Light Reflex
Two LGNd Channels

P cells in the retina project to the two magnocellular
(“large cell”) layers in the LGN.
–

P cells in the retina project to the four parvocellular
(“small cell”) layers in the LGN.
–

Livingstone & Hubel: color-blind, fast, high contrast
sensitivity, low spatial resolution
L&H: color selective, slow, low contrast sensitivity, high
spatial resolution
LGNd also has interlaminar layers with unknown
role/properties
–
Receives projections from optic nerve, S.C.
Right at the levels of cells; wrong at the level of populations
Primary Visual Cortex (V1)

First cortical visual area
–




Columnar (like all cortex)
Retinotopically mapped
Ocular dominance columns
Edges (Gabor filters), color,
disparity & motion maps
Connects to other
retinotopic areas (V2, V3,
MT)
http://webvision.med.utah.edu/imageswv/capas-cortex.jpg
Tootell, et al. 1982.
Proof of Retinotopic Mapping
Pattern flashed (like a
strobe) in front of monkey
injected with sugar dye
Left primary visual
cortex of the same
monkey
Single Cell Recordings in V1
Two Channels in V1

P  Magnocellular LGN Layers  layer 4C of V1.
–
Projects further to layer 4B




Motion direction selective
Orientation selectivity & binocular
No color
P  Parvocellular LGN  4C of V1
–
Projects to layers 2 & 3.

Layers 2 & 3 subdivide into “blobs” and “interblobs”:
–
–
–
–
–
Blobs are color selective, simple receptive fields
No orientation/movement direction selectivity, binocularity.
Prefer low frequencies, have small Magnocellular input
Interblobs have reduced (non-zero) color selectivity
Binocular, high-frequency, orientation selective
Jones & Palmer 1987
Simple Cells

The first orientation selective cells found in
V1 were labeled “simple cells”
–
Well approximated by Gabor functions with fixed
orientations, scales and phases
Complex Cells

The second set of orientation selective cells
were called complex cells
–
–
Well modeled as combining 90° out-of-phase
Gabor responses (quadrature pairs)
Captures energy at a particular orientation &
scale
even
filter

+
odd
filter


Organization of cells in V1
Hubel & Weisel proposed the following
organization for cells in V1
Well, a little more complex…
Ocular dominance
columns
Color-coded orientation
Sensitivity columns
More on V1




Only about 27% of V1 cells are orientation selective
About 70% of orientation selective cells are complex
cells
Orientation selective cells also include end-stopped
cells (a.k.a. hypercomplex cells), and grating cells.
Non-orientation selective cells include cells that
respond to:
–
–
–
Color (Hue/Sat maps?)
Disparity
Motion
V1 Connections
V1 is the starting point
of cortical visual
processing.
Dorsal projections lead
to somatosensory and
motor control areas
Ventral projections lead
toward associative
memories
From Van Essen 1992. Image can be found at
http://webvision.med.utah.edu/imageswv/Visual-Cortex1.jpg
Anatomical Maps of Visual Cortex
1983 Version
1990 Version
Two Visual Subsystems

In 1982, Ungeleider & Mishkin propose that
there are two primary visual pathways in
humans and primates:
–
The dorsal (or “what”) pathway

–
Ends in the posterior parietal cortex
The ventral (or “where”) pathway

Ends in the inferotemporal cortex
Visualizing Two Subsystems
D. Milner & M. Goodale, The Visual Brain in Action, p. 22