Visual Cognition
If we had the
sensory
apparatus of
some other of the
earth's
organisms,
'reality' would
seem quite
different - Irvin
Rock
Visual Cognition: In Humans
Camera eye
Compound eye
The Problem
• How to turn an upside down, 2D, warped
mirror-reflection and turn it into a right-side
up, straightened, aligned 3D world?
Fields of View Across Ecological Niches
• Birds have the highest resolution of visual
acuity, with cones eight times smaller than
ours.
• Basic feature design for predators (cats):
Both eyes in front
• Prey (rabbits, horses): eyes on the sides of
the head
Sensitivity to Light Across Biological Organisms
•
•
Rattlesnakes detect in infrared:
bees detect ultraviolet light.
“Function Follows Form”
• Two (mostly) distinct visual systems with :
–
–
–
–
–
Two regions of the eye
Two photoreceptors
Two perceptual characteristics (acuity/sensitivity)
Two convergence ratio
Two neural pathways & target regions
Two Photoreceptors:
Rods & Cones
• Rods
– Less dense
– Sensitive: brightness, movement
• Cones
– Dense
– Sensitive: acuity / edges, color
• Trade-off of Sensitivity for Acuity
• The Purkinge
Shift
Regions of the Eye
Retina: Whole inner eye
containing all photoreceptor
cells.
• Fovea indentation on retina,
mostly Cones.
– fine discrimination; colors &
detail.
• Periphery area outside fovea,
mostly rods
– Sensitivity to brightness,
movement
–
Optic disk: “Design Flaw” place
where axons exit eye forming
optic nerve.
Rods & Cones: Regions of the Eye & Convergence Ratio
Cones-specialized for color vision
& detail fovea.
Rods-sensitive to light periphery
• 126 million receptors total with 6
million cones.
• Cones = 5% of photoreceptors, but
25% of brain dedicated to them.
Specialization of light processing
determined by convergence ratio.
Rods-big receptive fields
Cones-small receptive fields
How Does Visual Information Flow?
• Lateral Inhibition
Information Flow: Lateral Inhibition
• Mach Bands
Luminance Comparison
Both squares are identical in luminance gradient.
The four squares are identical in luminance gradient.
Portions indicated by arrows have the same mean luminance.
Rods & Cones: Convergence Ratio
Specialization of light processing determined by “convergence ratio.”
Many rods converge on a single retinal ganglion = Sensitivity
Few Cones converge on a single retinal ganglion = acuity (detail)
Rods-big receptive fields
cones-small receptive fields
What is a receptive field of retinal
ganglion cells?
Kuffler (1953) presented spots of light to retina cells
in the cat & recorded their responses.
• The cells have a Concentric circle configuration!
• usually called center-surround cells
• On-center, off-surround cell has an “excitatory
center,” & “inhibitory surround”
• Off-center, on-surround cell has an “inhibitory
center” & “excitatory surround”
Receptive fields
Receptive Fields: Seeing a line
Receptive Fields: Seeing Movement
A computer emulation of "edge detection" using retinal receptive fields. Oncentre and off-centre stimulation is shown in red and green respectively.
"Red on centre green off centre" by Own work by Simpsons contributor - originally uploaded to the English language wikipedia.
Licensed under Public Domain via Wikimedia Commons ttp://commons.wikimedia.org/wiki/File:Red_on_centre_green_off_centre.png#mediaviewer/File:Red_on_centre_green_off_centre.png
Receptive fields at work.
Receptive fields at work.
As your fovea lands on a white
intersection, the black corners of the
neighboring squares fall on the retina,
with its larger receptive fields. With
each of the four corners falling into a
receptive field, the receptive field
sums to ‘dark’.
How the Hermann Grid Illusion Works
• (from a student response) The retina contains collections of
photoreceptors, some of which are activated by light and
others which are activated in the absence of light. The two
types are usually arranged to encircle each other, dark ones
around light ones and vice versa, and are spread throughout
the retina. The gray spots appear in the intersection of the grid
due to the competing effects of the dark and light
photoreceptors.
• When looking at the grid as a whole, the majority of which is
dark, it causes the more numerous dark photo receptors to
activate, overiding the light activated ones, and causing a
subtle darkening effect. When you focus directly on the white
space at the intersections between the squares, it narrows the
field of vision, with the smaller resulting receptive fields able
to “fit” within allowing the light detecting photoreceptors to
function without interference from the dark activated ones.
Receptive fields in Art
Mona Lisa’s beguiling
smile results from the large
receptive fields signaling
darkness at the corner of
her mouth, drawing your
attention, directing your
eye to foveate on the
corner. The fovea has
smaller receptive fields,
which do not sum with
darkness, making the smile
look like it is disappearing.
Apparent movement of the streams is
created by afterimages as our eyes shift to
examine the picture.
Two Visual Systems
• Geniculostriate & Tectopulvinar
Visual Pathways
• 1. Geniculostriate pathway– optic chiasm LGN  Primary Visual Cortex  Ventral
“What”
– Signals from fovea & cones (mostly); Parvo
– involved in pattern perception, color vision
• 2. Tectopulvinar pathway– optic chiasm---superior colliculus---Lateral Posterior Pulvinar-- Visual Cortex  Dorsal “Where/How”; Magno
– Signals from periphery & rods (mostly)
– detection of light; spatial orientation
Information Flow
Optic Chiasm
Retinal- Lateral
Calcarine Geniculate
pathway Nucleus
Dorsal Pathway
Parvocellular
Retina (rods &
cones)
Magnocellular
Visual
Cortex 1
(V1)
Tectal
Pathway
Superior
Colliculus
Ventral
Pathway
Parvo and Magno Cellular Pathways:
Example of Double Dissociation
• Lesions to the parvo-cellular pathway affect
perception for color, and fine detail (small spatial
frequencies); Lesions to magno-cellular pathway do
not.
• Lesions to magno-cellular pathway affect perception
of movement, brightness contrast (flicker); Lesions to
parvo-cellular pathway do not
• An engineer’s
box diagram of
the neural
circuitry in visual
perception, from
the retina to the
hippocampus.
Information Flow
• Topographic
Information Flow
• Visual Hemifields
measurement
• cones: vision under bright illumination
– Cone time constant=100ms, 1,000 photons can be
presented in a brief period (say, 1000 photons within a 1
ms period) or over a long period (such as 200 photons in
each of five 20 ms period) for the same visual effect. The
cones cannot tell the difference.
– Rod time constant=400 ms, meaning the photon catch
extends over a longer time interval working like a slow
shutter speed on a camera (well, cameras once had shutters
that had a “click” that digital cameras artificially reproduce
even though there is nothing to “click” any more.)
Sensation vs Perception
• The difference between sensation and
perception is the difference between light and
color
• It requires senses to detect light
• It requires perception and cognition to see
color
• Color happens both in the eye and in the brain
Measurement in Vision Science
• light measured in nanometers (nm)
nan·o·me·ter (năn'ə-mē'tər)
– 1nm=1billionth meter
– eyes sense from 360nm to 780nm on the infinite
scale of wavelength – nothing.
– “light” is whatever energy falls in that range “light” by definition, is anthromorphized.
• 1st feature of light
– wavelength:480=blue, 540=green,
565=yellow,590=red
Electromagnetic Radiation
What are the 2 properties of light that
influence visual perception?
• 1. Wavelength is associated with our
perception of color.
• 2. Intensity is associated with our perception
of “brightness.”
Visual Cognition:
Describing Light
• Hue or Value (Color)
• Luminance
(Brightness)
• Saturation (Purity)
We have three cone wavelengths
• 1. Short wavelength: peaks at 419 nm (blues).
• 2. Medium wavelength: peaks at 531 nm (greens).
• 3. Long wavelength: peaks at 558 nm (reds).
• The primary colors are blue, green, & red
Additive Color mixing with lights
Results of Additive Color Mixing
Two Theories of Color Cognition
• Young-Helmholtz
Trichromacy Theory
• Proposed in 1802 and
confirmed in 1983
– You should be able to create
any color by combining three
basic colors, red, green and
blue.
• Evidence: three different
colored pigments in the
fovea (electron micropscope)
Two Theories of Color Cognition
Trichromacy does not explain:
• Incompatible colors cannot be seen. Why can’t we
see certain colors (reddish-green, bluish-yellow)
• Color afterimages.
Opponent Process
We have 3 opposing mechanisms: red-green, yellowblue, & black-white. These are called
complimentary colors & put together they produce
yellow or white.
+
Opponent Process in a Movement
Illusion: Waterfall Effect
• http://video.google.com/videoplay?docid=6294268981850523944&ei=r5P
RSNGPD6fcqAPS48y6Ag&q=spiral+visual+illusion&vt=lf&hl=en
• http://video.google.com/videoplay?docid=2927422796086500362&vt=lf&hl=en
Color Perception
• Each set of colored rectangles, against the solid background, is
the same
•
http://www.youtube.com/watch?v=mf5otGNbkuc&feature=em-subs_digest-newavtr-vrecs
Assume Surrounding Colors are
Constant
In this illusion, the second card from the left
seems to be a stronger shade of pink in the top
picture. In fact they are the same color, but the
brain changes its assumption about color due to
the color cast of the surrounding photo.
Color-blindness
• Results whenever we are either missing one
of our cones or one of our cones doesn’t
work properly.
Color Helps To Recognize Objects
Color Helps To Recognize Objects
Color Helps To Recognize Objects
Color Helps To Recognize Objects
Color Helps To Recognize Objects
Constancies and Illusions: Perceiving
Size, Shape and Depth
Figure 1.6 The 2-D image size (on the retina) is a function of S.
Visual sizes and distances are measured in terms of degrees of
visual angle (hunk of an arc) for convenience
Cues to Size
Visual Angle
Universal Unit of Measure
If you want to know the distance between bc given that q is a given angle,
take the tangent of q1and multiply it by the distance d. This calculation
gives you the distance from a to b To determine the distance between b
and c, multiply the calculated distance from a to b by 2 and you know the
distance bc.
• |ba| = tan(θ) x d (4)
• |ba| = tan(θ ÷2) x d
• |bc| = 2 x |ba|
• |bc| = 2 x (tan(θ ÷ 2) x d)
Size Illusions
Mueller-Lyer Illusion
Ebbinghaus Illusion
Figure 1.2. Vision as a useful construction of the mind
·
Which two tabletops have identical dimensions on the page?
·
Tabletops A and B are identical in size on the page except for rotation.
·
Human vision sees tabletops B and C as similar because of a three-dimensional
interpretation of the world that is built into our brains and through which we see.
Monocular & Binocular
Cues to Depth
Monocular cues to depth:
Learned & Environmental
– Occlusion
– shading (assume light from
above and most shapes are
convex)
– T-junctions (crossbar is
nearer, stem is farther)
– Perspective
– Texture gradient,
– Height in the plane
– Eye-height
– Haze
– Familiar size
Binocular cues to depth:
Innate & Biological
– Convergence
– (Interpupillary distance –
IPD),
– accommodation,
– pupil contraction
Perceiving Depth and Size
• Monocular cues to
depth or Pictorial
Cues
– Linear perspective (a)
– Texture gradient (b)
– Interposition (c)
– Relative height in the
image (d)
76
Monocular cues
to size
Monocular cues
to depth
•Texture Gradient
•Occlusion
•Perspective
Relative Size. When
viewing two objects,
the farther object will
appear smaller even
though the objects
are the same size.
Far away objects
often appear less
clear in color and
detail due to haze.
On a clear day far
away objects may
seem very close,
closer than they
actually are.
Anamorphism: Tricks with Perspective
The Ambassadors. Painting by the Holbein. What is the object on the
ground?
Forced
Perspective
Binocular Cues to Depth:
Convergence & Retinal
Dis[arity
Human eyes are spaced apart on
average by 7cm. The brain gets two
slightly different images of the same
scene.
Convergence, is a depth cue
provided by the brain, turning the
eyes towards each other, the greater
the convergence, the greater the
proximity of the object.
Light from a scene falls on points in the retina of each eye in
mirror reflection, with a greater difference in relative position
for points from the scene that are closer. The greater the
disparity, the closer the object.
• http://deutsch.ucsd.edu/psychology/pages.php?
i=203
• Illusions with sound
• Motion perception:
https://www.youtube.com/watch?v=1F5ICP9S
YLU
• Optical flow:
• https://www.youtube.com/watch?v=qhoCYetpnM
The End
• Back-up Slides
Sensitivity to Light Across Biological Organisms
• Insects have complex eyes and
simple brains. Equipped with
pairs of compound eyes each
made of thousands of simple
light sensitive lenses, Each
lens registers the brightness of
whatever is directly in front of
it . A picture is built up from
which the insect can detect
objects in space and can detect
movement very readily. A fly
watching a reel film will see
the frames for each picture.
Sensitivity to Light Across Biological Organisms
•
•
Sunflowers and other plants turn to face the
sun
Earth worms do not have eyes, but will die if
exposed to too much sun, so there skin cells
senses photons as pressure, that cause the
worm to move in the right direction, a form of
attention called phototropism.
Sensitivity to Light Across Biological Organisms
• Spiders’ eyes have some
similarities with ours
• Their eyes are rigidly fixed on
their heads and look in different
directions outward and upwards
and they overlap to give a wide
view.
• Their enormous lenses let all the
available light through, so they
can see in one tenth of the light
we need to see,
• Spiders react to very small
movements, but they can only
see for about 15 cm.
Parts of the eye
• Outer Layer
• 1. Sclera: White fibrous
layer
• 2. Cornea: the clear
protruding structure in the
front of the eye that is
curved.
• Bends light rays & is
responsible for 70-80% of
eye’s focusing ability.
Middle layer (contd.)
• 3. Iris: smooth ring of muscle with a central
opening (pupil). Gives us our eye color
• 4. Pupil: Pupil changes in size depending on
intensity of light.
-Intense light, small constricted pupils
-Dim light, dilated pupils
• 5. Lens: focuses light on retina (convex).
-lens is round (nearby objects)
-lens is flatter for distant objects
Middle Layer
• 1. Choroid: vascularized
layer that provides
nutrition for retinal cells.
Is located between the
retina & sclera.
• 2. Pigment epithileum:
a black pigment found
between the choroid &
retina.
• **Traps photons from
light to prevent scattering
of photons along the
retina, which reduces
distortion.
measurement
• time
– the amount of light shining on the eye depends on how long
the eyes sits still.
• a single cycle of neural activity in response to a signal takes
between 1-10 milliseconds, 1 millisecond=1/1000 second. Between
the retina and the cerebral cortex, there are 4 cycles, or 40 ms of a
lapse time. the minimum time for a motor response is 200ms, a fifth
of a second.
• neurons can track pulsating light only up to 60 cycles, 8 ms on and
8 ms off. Individual movie frames change every 30 ms (33 cycles
per second) and we can’t see them.
– Eyeblink duration is around 100 ms.
measurement
• 2nd feature of light
– intensity or amount, photon catch = the number of
photons that actually make contact with light
sensitive receptors in the eye
• candela=intensity of light emitted from a candle onto a
square meter at one meter, 1/1000 watt bulb with a peak
wavelength of 555 nm.
• Lightness=the degree of surface reflectance (24),
• brightness=the intensity of the source of light
How is a dark adaptation experiment
conducted?
• Step 1: The subject (S) is exposed to bright light of
a known intensity.
• Step 2: S is then placed in total darkness & asked to
detect a spot of light (controlled luminance).
• Step 3: S’s detection threshold is plotted as a
function of time spent in dark.
Primary Visual Cortex
measurement
• Vision’s various jobs
– determine the combination of brightness and lightness that
generates the total photon catch at the eye
• is it a black paper illuminated by bright light, or white paper under
dim light?
– Both pieces of paper may reflect similar amounts of light to the eye
but usually the black reflects more so why aren’t we fooled into
seeing it as white?
• Space
– candela is defined in terms of space
• the angle formed by rays extending from the lens to the object
equals the angles formed by rays extending from the lens to the
retina.
• visual angle is measured in degrees, minutes, seconds (see next)
measurement
• limitations
– space constants of the eye are limited by the size
of, and interaction among, cones and rods.
• constant, the point at which the eye cannot distinguish
whether the number of photons were concentrated in a
small region of the retina or were distributed over a
large region
– space and time constants are limited by the size
and interaction among cones an rods
• the closer to the center the eye, the smaller the constant
(the eye can “see” more in daylight) (27).
Visual Cognition: The
Basics
How do our eyes adapt to the dark?
• The photopic & scotopic systems adjust
their sensitivity as a function of time in the
dark.
• The cones become more sensitive during
the first 5-10 minutes after being in the
dark.
• Rods continue becoming more sensitive
over period of 20-30 minutes.