twelve
visual perception
Processing in the human visual system
(maybe, sort of)
salience
eye
attention
object
recognition
early
vision
(IT cortex)
(striate cortex)
contour
detection
grouping
“what” pathway
spatial
memory
(PP cortex)
“where” pathway
The eye

Lens



Photoreceptors




Depth of field
Chromatic aberration
Rods
Cones
Fovea, macula, and
periphery
Retinal processing



Gain control
Edge enhancement?
Simple motion detection
lens/iris
rods
cones
retinal
ganglion
Photoreceptors

Rods




Cones




Found mostly in the macula and periphery
Very sensitive to light
But don’t detect color
Found in the fovea
Less sensitive
Color sensitive
Colors seem to fade in low light
Trichromacy

Having different cones for every
possible wavelength would be bad

We just have three kinds of cones





“Blue” cones: short wavelengths
“Green” cones: intermediate
wavelengths
“Red” cones: long wavelengths
However, their responses overlap
The eye reduces all the
wavelengths at a given pixel to
just the total “amount” of “red”,
“green”, and “blue”
Components of a color image
Evolution of the color system
S
to cortex
Evolution of the color system
S
L
intensity
+
−
chroma (B-Y)
to cortex
cool/warm
Intensity and chromaticity
Color image
Intensity (R+G+B)
Chromaticity (B-Y)
white = cool
black = warm
grey = neurtral
Evolution of the color system
S
I
L
intensity
+
−
−
chroma (B-Y)
chroma (R-G)
to cortex
cool/warm
Intensity and chromaticity
Color image
Intensity
Chromaticity (B-Y)
Chromaticity (R-G)
white = cool
black = warn
grey = neutral
white = red
black = green
grey = balanced
Code
[define gray
[i → [color i i i]]]
[define signed-gray
[i → [gray [+ 128
[∕ i 2]]]]]
[define b-y
[c → [− [blue c]
[∕ [+ [red c] [green c]]
2]]]]
[map-bitmap [c → [signed-gray [b-y c]]]
cones]
Code
[define r-g
[c → [− [red c] [green c]]]]
[map-bitmap [c → [signed-gray [r-g c]]]
cones]
Image-level processing




Pixel-level processing
Massively parallel
Preattentive
Very fast
Contrast versus absolute intensity



The visual system
responds less to the
amount of light or color
Than to variations in the
amount
This is also known as
contrast
Contrast detection
inhibitory region
(light prevents firing)
excitatory region
(light stimulates firing)


Many of the neurons in
the visual system are
designed to respond to
contrast
Center/surround
receptive fields


Stimulated by the presence
of light in a specific area
Inhibited by its presence in
the area immediately
around it
− + −
Edge detection



When applied to
image intensity
(R+G+B),
Center/surround
processing responds
to lightness contrast
Which often occurs at
object edges
Scale space

Contrast detection is
performed at many
different scales
Oriented edge detection

Other neurons



Are stimulated by light
in an elongated region
And inhibited by it in
the region next to it
These can detect the
orientation of edges
−
+
Oriented edges
Constancy

Illumination varies over a
surface


in intensity
in chroma

But you (usually don’t
perceive the color of the
surface as changing

This is called perceptual
constancy


Color constancy
Lightness constancy
Georges de la tour,
The Repentant Magdalene, C. 1640
Failures of lightness constancy
Failures of lightness constancy
Failures of lightness constancy
Failures of color constancy
Failures of color constancy
Failures of color constancy
Failures of color constancy
Depth cues



The world is 3D
But the image is 2D
Your vision system uses
depth cues to recover
lost information





Stereopsis
Motion
Linear perspective
Aerial perspective
Relative size
Occlusion


The interruption of edges of
one object by the edges of
another object
Shows the interrupting object
is in front
C. Coles Phillips,
Brittish Tank on Fifth Avenue, 1917
Stereo vision


Your eyes are in slightly
different positions
So they receive slightly
different images


The disparity between
images decreases with
distance (simple case)
So by matching features
between the two eyes


Your brain computes
disparity
And hence, depth
Motion parallax

Objects farther away seem to move slower




So the background seems to remain stable
And the foreground moves
Or, the opposite, if the camera tracks the foreground
This is frequently used in cell animation
Linear perspective


Parallel lines converge at a
vanishing point
One-point perspective




Two-point perspective



One set of lines converging
Gives a sense of immersion in
the scene
Background seems to pop out
Two sets of converging lines
The object seems to pop out
Three-point perspective


Rarely used
Very dramatic
Aerial perspective

Surfaces get hazier
as they recede into
the distance
Relative size


Apparent size
decreases with
distance
So
Texture gradients

Textures get finer as objects recede
Gestalt theory

Theory of how we organize
components of a percept into
wholes


Law of prägnanz (simplicity)


Max Wertheimer, Kurt Koffka,
Wolfgang Köhler
We choose the simplest possible
interpretation of a scene
Formulated laws of grouping
Disks or pacman?
Grouping by proximity
Grouping by similarity
Grouping by continuity
Grouping by closure
Grouping by common fate
Grouping by common fate
Figure and ground

We tend to separate the
scene into



We use a number of
tricks to do it


One or more objects
(“figure”), in front of or on
top of
A ground plane or
background (“ground”)
Depth, grouping
But it can be hard in
degenerate cases
Attention

Remember our fovea is only about 2°


Attention is driven both top-down by
cognition



Task
Knowledge, expectations
And bottom-up by






So we have to view scenes by scanning
the fovea over the image
Size
Intensity
Local contrast
Motion
Odd-man-out
As well as recency
(don’t look if we’ve just looked there)
Attention and scale space


Attention starts
at coarse
(large) scales
And narrows to
finer scales
Attention and intensity

All things being equal,
our attention is driven
to intense stimuli



Bright lights
Saturated colors
Rapid motion
Attention and color


Warm, bright colors
draw attention
Cooler, darker colors
tend to recede into
the background
Claude Monet, Sunrise, 1872
Attention and contrast

But one of the biggest
drivers of attention is
contrast


We focus on unusual
elements of the scene
Elements that are
different from their
neighbors
Attention follows curves
Attention follows perspective and depth
Top-down attention

Attention is also
driven by
expectations




If I see something
face-like
I look for the facial
features
Or vice-versa
And by task
Change blindness


We only really see what the
fovea is fixating
If anything changes, we don’t
notice unless we see motion



Supply
Pattern
Continuity in films


Oh, and we don’t see motion
while our eye is moving …
Pretty woman
Neisser et al’s demo
Simons and Chabris’ demo
Work area
Adapted from Ballard et al.
Shape recognition



We don’t really know
how this works
It’s really hard …
And there are a lot of
high level semantic
effects