Psy/Orf 322
Human – Machine Interactions
Lecture 16
Spring 2004
Psy/Orf 322 Human-Machine Interactions
Spring 2005
Prof. Alain L. Kornhauser
Human Vision: The System and its Behavior
Web References:
http://webvision.med.utah.edu/
http://www.driesen.com/visual_system.htm
http://www.cis.rit.edu/mcsl/faq/faq1.shtml (good link on questions of vision and color.
http://www.redherring.com/mag/issue86/mag-mutant-86.html (interesting link on tetrachromats)
Cellular Model of Biological Visual Processing (The human eye):
Image Data (Photons)
Captured by Receptors (Rods & Cones) (Retina)
Pass Thru layers of cells (Preprocessing of the Data) (Retina)
Transmitted over distance (Ganglia (optic nerve))
Higher – level processing (Visual Cortex)
02/16/16 Psy/Orf 322 Human –Machine Interactions
Spring 2005
Retina and Phototransduction:
Retina:
Retina is derived from neuroectoderm; (cellular structure that gives rise to the brain)
Retina is sufficiently complex to be a small brain
(not a photographic plate, rather operates on contrast and differences)
Cellular Structure of Retina:
5 classes of neurons (Figure); organized in 3 physical layers, producing 2 distinct
synaptic interactions :
Neurons:
Rods & Cones (Photoreceptors, ~108) Densities
Bipolar (flat cone & rod varieties); 11 varieties
Horizontal cells (couple many rods & cones)
Amacrine (couple many Bipolars to a single ganglia)
Ganglion (off-center and on-center varieties, ~106]
Physical Layers:
Outer plexiform layer:
Receptors (Rods & Cones), bipolar, horizontal
Light must pass through other cellular layers to reach them
02/16/16 Psy/Orf 322 Human –Machine Interactions
Spring 2005
Page 2
Inner plexiform layer: Bipolar, Amacrine, ganglia
Ganglia layer: Ganglion
Synaptic interactions (Information preprocessing)
direct route: receptor, bipolar, ganglion
Splits visual signals into 2 separate channels (successive ON & OFF pathways):
a. lighter than background
b. darker than background
integration route: horizontal cells integrate and transfer info from distant receptors to
bipolar-ganglion cell pathway
Creates simultaneous contrast of visual objects (forms a receptive field with a center contrasted
to an inhibitory surround)
Operation:
Continual movement in a sequence of saccades
Each takes about 30 msec to jump to a new point of regard
Dwells there for 230 [70~700] msec
Low resolution, Peripheral vision ~ 180 deg.
High resolution, Fovea ~ 2 deg.
Fovia; intervening neural layers are pealed back
Blind spot: nasal to fovea, where optic nerve fibers leave retina: no photoreceptor
What a Frog’s Eye tells the Frog’s brain
02/16/16 Psy/Orf 322 Human –Machine Interactions
Spring 2005
Page 3
2 major information flow pathways:
Cones: detect color. 3 varieties: RGB, not very sensitive, day vision, Details of color vision
Rods: low light gray-scale receptors(0.64x107). 20 times as many as cones, (1.2x108)
Cones: less sensitive, faster response, directionally sensitive, connected to more individualized
neural channels leads to higher acuity.
Rods: longer photoreceptors allow them to capture more light (more sensitive), are affected by
more scattered light and many rods synapse on the same bipolar cell, thus, they have poor spatial
resolution; “firing rate” changed upon achieving a threshold of stimulus.
They transduce light into electrical signals through synaptic activity:
Surprisingly: light inhibits and darkness excites photo-receptor cells!
Most cells in visual system show continued synaptic activity (discharge) even in the absence
of illumination. Nominal synaptic rate of .001 sec.
Appropriate stimuli modulates background synaptic activity, (increasing or decreasing)
Rods & cones make direct synaptic contact with bipolar cells. Horizontal and amacrine cells
mediate between lateral interactions between receptors and ganglion. Ganglion cells project to the
lateral geniculate nucleus and the superior colliculus in the visual cortex.
A single cone synapses on two separate bipolar cell channels: one is excited by light activation (on02/16/16 Psy/Orf 322 Human –Machine Interactions
Spring 2005
Page 4
center (depolarizing)) and the other is inhibited by light activation (off-center (hyper-polarizing)).
Horizontal Cells: deliver to the ganglion response of more distant photoreceptors than those
directly connected to the bipolar.
This leads to Center-Surround Antagonism; Direct light on the center of a receptive field is
antagonized by direct light on the surround of its receptive field. This is important in determining
borders (27-7)
Ganglion cells : on-center and off-center variety
Also have morphologically and functionally different subsets that serve the same photo receptors in
parallel
X ganglion: medium sized cell bodies, narrow dendritic fields: high visual acuity (28-7)
(80%)
Y ganglion: largest cell bodies, large dendritic arborization: respond to large targets, perform
initial crude evaluation (10%)
W ganglion: small cell bodies, large arborization, project to the superior colliculus: are
involved in head and eye movement (10%)
Anatomy of the Central Visual Pathways
See Fig 28-1 for visual fields,
Lens inverts the image on the retina (28-2),
Nasal portion of the visual field projects on the temporal portion of the retina, etc.
02/16/16 Psy/Orf 322 Human –Machine Interactions
Spring 2005
Page 5
Left optic track contains complete representation of right hemifield of vision. (28-3)
Injury to portions of brain lead to defects in visual fields (28-10)
Processing form and movement:
Retina is mapped in the lateral geniculate nucleus and the visual cortex.(29-1, 29-4)
Lateral geniculate nucleus enhances the antagonism between the center and its surround. (29-5)
Receptive field of neurons in the visual cortex have the following 3 features:
1. correspond to a specific retinal position
2. have discrete excitatory and inhibitory zones
3. have specific axis of orientation (see fig. 29-7)
Appearance of an object depends not on the intensity of the light source but on the contrast
between the object and its surround (29-6)
Fig 29-8 clearly describes the orientation of the receptive field of a simple cell in the primary
visual cortex.
Basic anatomy, Feedback pathways
Illusions
Edge extension
Kanizsa Triangle
ExtendLine
YellowCircles (They now charge for these)
02/16/16 Psy/Orf 322 Human –Machine Interactions
Spring 2005
Page 6
NotaSpiral (They now charge for these)
Perspective
Depth1, depth2
Front/back
Lightness Perceptions
2 shades of green
Recognition
Old/young womana. 6Beers 4 dots
HowIsThisPossible
ABC?
Liar?
ColorTest
Crazy
Animated Illusions (Do these at home)
Also these
Eyetricks.com
Alzheimers Fs
Count the dots
Dizzy
Bending lines
FreakySpiral
MC Escher
02/16/16 Psy/Orf 322 Human –Machine Interactions
Spring 2005
Page 7
Views of Viewing:
Computational Modeling Vision and Visual Cognition
A. Kornhauser
Model the process of vision and visual cognition in two sequential phases:
Low-level & High-level vision
Low-level Vision
 1st stage of processing; input is array of intensity levels; available are past arrays.
 Objective of low-level: Segmentation and decomposition of a scene into constituent
independent parts/objects.
 Need to find what belongs with what
 Recovers properties of physical environment
Edges/Boundaries
Surface orientation
Depth of field, Stereopsis
Material properties/Textures/Color
Motion in space (Relative motion train moving or station moving?)
 Seems to be executed independent of domain and task
02/16/16 Psy/Orf 322 Human –Machine Interactions
Spring 2005
Page 8
Given a scene on two different occasions with different tasks/goals, same low-level
process applied
 Process applied is mostly independent of specific object knowledge
Motion perception uses assumption about the continuity and rigidity of objects but
not that it is a pen or a hat. The use of rigidity in the perception of motion is present
at the age of 5 months.
 Done in parallel / simultaneously in large portions of the visual field
 Process can be modeled as sub-processes (Marr):
3 stage process:
a. Primal sketch (edge detection) (input)
b. Modular processes of Shape extraction, Texture, Color, Binocular, Motion
c. 2.5 D sketch (view-centered representation)
Given: Array of intensity values.
1st step is to normalize the values (eye works on relative intensity)
next steps (going on in parallel if possible, somewhat independent)
 Edge detection:
find zero crossing of 2nd derivative of image intensity,
need convolution of filter to “smooth out noise”
02/16/16 Psy/Orf 322 Human –Machine Interactions
Spring 2005
Page 9
 Texture segmentation:
Can be viewed either as
Statistical property (variation of spatial intensity levels), or
A class of primitive elements (template matching): textons
 Binocular Stereo:
Must solve the correspondence problem: what matches with what?
Important Principles:
 If corresponding primitives can be found: triangularization computes depth
 Epipolar line constraints: A primitive in the left eye can only be matched with
primitives lying on the (epipolar) line on the right eye. (fig 1.6b)
 Direction of Gaze (relative orientation of the eyes)can also compute depth
 Changes in Direction of Gaze:
 changes in direction of gaze changes the complexity of correspondence!
Correspondence is simplest if eyes are fixating directly on object. (fig 1.9)
 Color:
Color constancy under varying illumination levels:
Perceived color of a patch is largely independent of illumination (intensity), but
does depend on the color of neighboring patches. (Red London bus shade or
sun)
Implications on shape, depth
02/16/16 Psy/Orf 322 Human –Machine Interactions
10
Spring 2005
Page
 Motion:
Solve the correspondence problem over a temporal sequence of images
1. Motion Measurement: construct a Velocity Field, 2. Recovery of Structure
Long Range motion correspondence: Minimal mapping: finds the
correspondence that minimizes a distance function while ensuring that there will
be at least one match.
Short Range motion correspondence:
Match neighboring point: “easy”
Match neighboring contours: Aperture problem:
get good information on velocity field normal to the contour,
get bad/no information on velocity field tangential to the contour (1.14)
(good for collision avoidance)
Higher-Level Vision:
Pattern Recognition: problems
Independent of scale
Independent of orientation
Partially occluded
Representing Images:
02/16/16 Psy/Orf 322 Human –Machine Interactions
11
Spring 2005
Page
Nonaccidental properties: 1.Smooth Continuation, 2.Cotermination,
Parallelism, 4. Symmetry
3.
Complex entities invite decomposition into simple parts.
Recognition-by-components / Geons
24 geons, 4 attribute x-sections,
108 viewpoint invariant relations
3 relative aspect ratios (larger, smaller, equal)
Then: 3 geons give 109 combinations
Extension of scene perception, Transversality principle: matching vertices
Object Recognition:
Parallel distributed processing
Visual attention
02/16/16 Psy/Orf 322 Human –Machine Interactions
12
Spring 2005
Page