Segmentation and Grouping
• Gestalt approach
– Problem - We don’t perceive local events in an
image - we perceive more global figures
– Elucidate principles which determine grouping
of local “things” in an image into figures
• Proximity
• Similarity
• Pragnanz
– good continuation
– symmetry
Proximity
Similarity
Colinearity
Real world problems to which we can apply
gestalt principles
• Segmentation
– determining where objects are in an image and
what their boundaries are.
• Grouping
– grouping together stuff as part of the same
object; for example, across occluders.
Boundary detection
(Local)
Boundary detection
(Local)
• Luminance edges
Boundary detection
(Local)
• Luminance edges
– Use center-surround receptive fields
Finding meaningful contours in
image
• Local edge detection
– Problems - false targets, misses
Finding meaningful contours in
image
• Local edge detection
– Problems - false targets, misses
• Solution 1: use other cues
– Texture
– Motion
– Disparity
Boundary detection
(Local)
• Luminance edges
– Use center-surround receptive fields
• Texture edges
Boundary detection
(Local)
• Luminance edges
– Use center-surround receptive fields
• Texture edges
– Sharp changes in orientation, scale of textures
Boundary detection
(Local)
• Luminance edges
– Use center-surround receptive fields
• Texture edges
– Sharp changes in orientation, scale of textures
• Disparity edges
Left eye
Right eye
Boundary detection
(Local)
• Luminance edges
– Use center-surround receptive fields
• Texture edges
– Sharp changes in orientation, scale of textures
• Disparity edges
Left eye
-
Right eye
=
000000000000000000000
000000000000000000000
000000222222222000000
000000222222222000000
000000222222222000000
000000222222222000000
000000000000000000000
000000000000000000000
Boundary detection
(Local)
• Luminance edges
– Use center-surround receptive fields
• Texture edges
– Sharp changes in orientation, scale of textures
• Disparity edges
• Motion edges
Boundary detection
(Local)
• Luminance edges
– Use center-surround receptive fields
• Texture edges
– Sharp changes in orientation, scale of textures
• Disparity edges
• Motion edges
Paradigm
• Look for textures which “pop-out” to
observers.
• Characterize texture properties which
support texture pop-out - fill in the blank:
– A figure pops-out from the background if its
__________ (property of texture) differs from
that of the background.
• Logic:
– Pop-out is the result of automatic, low-level
segmentation processes.
Texture properties which the visual system
uses to do segmentation
• Brightness
• Contrast
• Scale
• Orientation
Segmentation by Contrast
Segmentation by scale
Segmentation by orientation
Real-world Justification for these
properties
• Most objects in a scene will differ in at least
one, and probably more of these properties.
• When an object’s texture doesn’t differ from
that of it’s background it is camouflaged.
• But why only these and not others?
Mechanisms for texture segmentation
• Texture is a semi-local property of an image
– Texture is the “micro-pattern” in an image
– An individual point in an image cannot have
texture, but a small region can.
• Complex cells are good coders of texture
properties
– have local receptive fields, but aren’t sensitive
to position of a pattern within the receptive
fields
– Signals how much oriented “stuff” falls within
their receptive fields
Cortical images
• Treat a set of cortical cells with the same
receptive field properties as an image. The
activity of the cell whose receptive field is
centered at a given position of the visual
field is the “intensity” of the cortical image.
• Have cortical images for each combination
of orientation and scale preferences.
Complex cell images
• A cortical image made by looking at the
firing rates of complex cells with the same
orientation and scale preferences.
• Example:
– Fine-scale, vertical complex cell image • firing rates of complex cells with small, vertical
receptive fields.
• An image of the fine-scale vertical “stuff” in an
image.
• Problem – How does visual system resolve ambiguities in
local measures of image intensity changes to
decide what is part of a contour and what isn’t?
– How does the visual system integrate local edge
information into global figures?
• Phenomenal window into visual processing
of contours:
– illusory contours and amodal completion
Illusory contours and amodal completion
are flip sides of the same coin
• Amodal completion - Filling in boundaries
of objects behind occluders.
• Illusory contours - Filling in boundaries OF
occluders.
• The appearance of illusory contours usually
coincides with the appearance of amodally
completed boundaries.