Outline
• Theoretical approaches to computer vision
– Visual perception as information processing
• Problems in Computer Vision
–
–
–
–
Classification
Segmentation
Recognition
Motion analysis
Visual Perception as an Inverse Problem
• Retinal images are generated by the light
reflected from the 3-D world
– The image formation is determined by the laws
of optics
– The area of image rendering is called computer
graphics
• Vision as an inverse problem
– Get from optical images of scenes back to
knowledge of the objects that gave rise to them
5/29/2016
Visual Perception Modeling
2
Vision as a Heuristic Process
• Visual system makes a lot of assumptions about the
nature of the environment and conditions under
which it is viewed
– These assumptions constrain the inverse problem enough
to make it solvable most of the time
– The resulting solution will be veridical if the assumptions
are true
– Vision is a heuristic process in which inferences are made
about the most likely environmental condition that could
have produced a given image
5/29/2016
Visual Perception Modeling
3
Perception as Bayesian Inference
• Images I are observations
• Scene properties S are not known
• p(S) specifies the prior knowledge about the scene
– The knowledge you have without looking at the image
• Bayes’ rule
p( I | S ) p( S )
p( S | I ) 
p( I )
p( S | I )  p( I | S ) p( S )
5/29/2016
Visual Perception Modeling
4
Four Stages of Visual Processing
•
•
•
•
Image-based stage
Surface-based stage
Object-based stage
Category-based stage
5/29/2016
Visual Perception Modeling
5
Image-based Stages
• Most theorists agree that initial stage is not
the only representation based on a twodimensional retinal organization
– It includes image-processing operations
• Local edge and line detection
• Region detection
• Correspondence between left and right eyes
– Marr called this representation primal sketches
• Raw primal sketch
• Full primal sketch
5/29/2016
Visual Perception Modeling
6
Representation in Early Vision
• Local spatial/frequency representation
– The representation should be
• Local
• Orientation-tuned
• Frequency-tuned
– Gabor filters
– Wavelet transformation
• Image compression
5/29/2016
Visual Perception Modeling
7
Gabor Filters
5/29/2016
Visual Perception Modeling
8
Surface-based Stage
• Recovery of intrinsic properties of visible
surfaces
• Surface layout
– The spatial distribution of visible surfaces within
the 3-D environment
• Explicit surface-based representation
– 2.5-D sketch
– Intrinsic images
• Intrinsic properties to surfaces
5/29/2016
Visual Perception Modeling
9
Surface-Based Stage – cont.
• Surface primitives
– Local patches of 2-D surface within a 3-D space
• Three-dimensional geometry
– Projective geometry
• Viewer-centered reference frame
5/29/2016
Visual Perception Modeling
10
Surface-Based Stage – cont.
• Cues for surface representation
–
–
–
–
Stereopsis
Motion parallax
Shading and shadows
Pictorial properties
•
•
•
•
5/29/2016
Texture
Size
Shape
Occlusion
Visual Perception Modeling
11
Object-Based Stage
• Some form of true 3-D representation
– Includes unseen and occluded surfaces
• Explicit representations of whole objects
• Two ways of constructing object
representation
– Extend the surface-based representation
– Infer 3-D objects from 2-D images
5/29/2016
Visual Perception Modeling
12
Object-Based Stage – cont.
• Volumetric primitives
– Descriptions of truly 3-D volumes
• Three-dimensional geometry
– Geometry in 3-D space
• Object-based reference frame
– Spatial relations among the volumetric primitives
are represented by intrinsic structures among
volumetric structures
5/29/2016
Visual Perception Modeling
13
Category-Based Stage
• Final stage concerns with recovering fully the
functional properties of objects
– Functional properties through categorization
– Properties directly from visible characteristics
5/29/2016
Visual Perception Modeling
14
Top-down vs. Bottom-up Processes
• Bottom-up processing
– Data driven processing
– Take a lower-level representation as input and
create or modify a higher-level representation
• Top-down processing
– Expectation-driven processing
– Processes that take a higher-level representation
as input and produce or modify a lower-level
representation
5/29/2016
Visual Perception Modeling
15
Neural Network Approaches
• Neural networks are based on the
assumptions that human vision depends
heavily on the massively parallel structure of
neural circuits in the brain
– Multiple Layer Perceptrons
• Input layer
• Hidden layer
• Output layer
5/29/2016
Visual Perception Modeling
16
Problems in Computer Vision
• Given a matrix of numbers representing an image,
or a sequence of images, how to generate a
perceptually meaningful description of the matrix?
– An image can be a color image, gray level image, or other
format such as remote sensing images
– A two-dimensional matrix represents a single image
– A three-dimensional matrix represents a sequence of
images
• A video sequence is a 3-D matrix
• A movie is also a 3-D matrix
5/29/2016
Visual Perception Modeling
17
Image Classification
• Given some types through examples, identify
the type of a new image
5/29/2016
Visual Perception Modeling
18
Image Segmentation
• Partition the images into homogenous regions
– Widely studied problem
– A very difficult problem
– An important problem
A texture image
5/29/2016
Visual Perception Modeling
19
Object Recognition
• Object recognition
– Recognize objects in a constrained environment
– Identify objects from images
A cheetah image
5/29/2016
Visual Perception Modeling
20
Video Sequence Analysis
• Motion analysis
– Compute motion from
images
– Motion segmentation
• Video sequence analysis
– Derive models
automatically
– Enhanced TV
5/29/2016
Visual Perception Modeling
21