Digital Image Processing
Segmentation
Material in this presentation is largely based on/derived from
presentations by: Sventlana Lazebnik, and Noah Snavely
Brent M. Dingle, Ph.D.
Game Design and Development Program
Mathematics, Statistics and Computer Science
University of Wisconsin - Stout
2015
Lecture Objectives
• Previously
– Image Manipulation and Enhancement
•
•
•
•
Filtering
Interpolation
Warping
Morphing
– Image Compression
– Image Analysis
• Edge Detection
• Smart Scissors
• Stereo Image Processing
• Today
– Segmentation
Segmentation Relation
• Segmentation methods touch on and use
many previous topics
– Representation Methods
– Manipulation Methods
– Human Perception and Psychology
Segmentation Goals
• Group similar looking pixels together
for efficiency of additional processing
• Superpixels
– Learning a classification model for segmentation, Ren and Malik, ICCV
2003.
Segmentation Goals
• Separate image into coherent objects
• Berkeley segmentation database
– http://www.eecs.berkeley.edu/Research/Projects/CS/vision/grouping/segbench/
image
segmentation
Gestalt Psychology
• Human minds ‘group’ things
– Our perception is affected by this behavior
subjective contours
occlusion
familiar configuration
http://en.wikipedia.org/wiki/Gestalt_psychology
Emergence
• Find the dog
http://en.wikipedia.org/wiki/Gestalt_psychology
Gestalt Factors
These factors are intuitively obvious to humans
BUT are difficult to code into a computer
Segmentation via Clustering
• Concept:
– Cluster similar pixels/features together
• Color being an obvious choice
source: K. Grauman
K-Means Clustering
• K-means clustering is based on the intensity or color of
pixels
– Essentially is a vector quantization of the image attributes
(intensity or color)
• Notice the clusters need not be spatially localized
Image
Intensity-based clusters
Color-based clusters
source: S. Lazebnik
Segmentation via Clustering
– Cluster similar pixels/features together
• Color PLUS LOCATION
source: K. Grauman
Cluster Color AND Location
• Clustering based on (r, g, b, x, y) values leads to greater
spatial coherence
source: S. Lazebnik
Summary: K-means Segmentation
• Good
– Simple
– Converges to local
minimum of the error
function
• Bad
–
–
–
–
–
Uses lots of memory
Human picks K
Sensitive to initialization
Sensitive to outliers
Only finds ‘sphere-like’
clusters
source: S. Lazebnik
Summary: K-means Segmentation
• Good
– Simple
– Converges to local
minimum of the error
function
• Bad
–
–
–
–
–
Uses lots of memory
Human picks K
Sensitive to initialization
Sensitive to
Sensitive
to outliers
outliers
Only finds ‘sphere-like’
clusters
source: S. Lazebnik
Questions So Far?
• Questions on K-Means Segmentation?
Mean Shift Clustering
• An advanced and versatile method of clusteringbased segmentation
http://www.caip.rutgers.edu/~comanici/MSPAMI/msPamiResults.html
Mean Shift: A Robust Approach toward Feature Space Analysis,
D. Comaniciu and P. Meer, PAMI 2002.
Mean Shift Algorithm
• Seeks modes or local maxima of density in the
feature space
image
Feature space
(L*u*v* color values)
L = luminance
u and v are spatial coordinates
source: S. Lazebnik
Mean shift
Search
window
Center of
mass
Mean Shift
vector
Slide by Y. Ukrainitz & B. Sarel
Mean shift
Search
window
Center of
mass
Mean Shift
vector
Slide by Y. Ukrainitz & B. Sarel
Mean shift
Search
window
Center of
mass
Mean Shift
vector
Slide by Y. Ukrainitz & B. Sarel
Mean shift
Search
window
Center of
mass
Mean Shift
vector
Slide by Y. Ukrainitz & B. Sarel
Mean shift
Search
window
Center of
mass
Mean Shift
vector
Slide by Y. Ukrainitz & B. Sarel
Mean shift
Search
window
Center of
mass
Mean Shift
vector
Slide by Y. Ukrainitz & B. Sarel
Mean shift
Search
window
Center of
mass
Slide by Y. Ukrainitz & B. Sarel
Mean Shift Clustering
• Define Cluster as
– all data points in the attraction basin of a mode
• Define Attraction Basin as
– the region for which all trajectories lead to the same mode
Slide by Y. Ukrainitz & B. Sarel
Mean Shift Clustering / Segmentation
•
•
•
•
Find Features (color, gradients, texture…)
Initialize windows at individual feature points
Perform mean shift for each window until convergence
Merge windows that end near the same ‘peak’ or mode
source: S. Lazebnik
Example: Mean Shift Results
http://www.caip.rutgers.edu/~comanici/MSPAMI/msPamiResults.html
Mean Shift Results (c1)
Mean Shift Results (c2)
Summary: Mean Shift
• Good
–
–
–
–
Does not assume spherical clusters
Takes a single parameter (window size)
Finds variable number of nodes
Robust outliers
• Bad
– Output depends on window size
– Computationally expensive
– Does not scale well with dimension of feature space
Questions So Far?
• Questions on Mean Shift
Clustering/Segmentation?
More Questions?
• Beyond D2L
– Examples and information
can be found online at:
• http://docdingle.com/teaching/cs.html
• Continue to more stuff as needed
Extra Reference Stuff Follows
Credits
• Much of the content derived/based on slides
for use with the book:
– Digital Image Processing, Gonzalez and Woods
• Some layout and presentation style derived/based
on presentations by
–
–
–
–
–
–
Donald House, Texas A&M University, 1999
Sventlana Lazebnik, UNC, 2010
Noah Snavely, Cornell University, 2012
Xin Li, WVU, 2014
George Wolberg, City College of New York, 2015
Yao Wang and Zhu Liu, NYU-Poly, 2015