Prof Trivedi ECE253A Notes for Students
only
Digital Image Processing
Digital Image Processing
December 2 -4 , 2019:
Image Segmentation
(Chapter 10 Textbook)
• Basic Segmentation Approaches: Region based and Boundary Based
• Segmentation by Clustering
• Boundary Based Segmentation
• Linear Segment Detection by Hough Transforms
• Edges and Edge Linking
• Linear Segment Detection by Hough Transforms
• Circles and Ellipses by Hough Transforms
Quiz 4: Dec
(Chapter 10 Textbook)
• Edges and Edge Linking
• Boundary Based Segmentation
4th
Image Segmentation
2019
• Circles and Ellipses by Hough Transforms
Quiz 4: Dec 4th 2019
HW 4: Due Day of the Finals, Dec, 12th , 2019
HW 4: Due Day of the Finals, Dec, 12th , 2019
Image Segmentation: What do you see??
Image Segmentation: What do you see??
Color “patches” “Image Segmentation: What do you see??
Image Segmentation: What do you see??
https://boofcv.org/index.php?title=File:Example_Image_Segmentation_Color.jpg
Segmented image where each region is assigned the average color of pixels inside.
1
Prof Trivedi ECE253A Notes for Students
only
Clustering: Multidimensional Thresholding
Feature 3
K-Means Clustering
•
• Nonparametric
• Multidimensional multiclass
extension of thresholding
Initialization: Given K categories, N points in feature
space. Pick K points randomly; these are initial cluster
centers (means) m1, …, mK. Repeat the following:
Assign each of the N points, xj, to clusters by nearest mi
(make sure no cluster is empty)
2. Re-compute mean mi of each cluster from its member points
3. If no mean has changed, stop
1.
+++
+++
•
Feature 2
Effectively carries out gradient descent to minimize:
⎧
2⎫
x
−
µ
⎨
∑
∑
j
i ⎬
i∈clusters ⎩ j∈elements of i'th cluster
⎭
Feature 1
Slide credit: Christopher Rasmussen
K-Means
Example: 3-means Clustering
Minimizing squared distances to the center implies
that the center is at the mean:
Derivative of error is
zero at the minimum
Anil Jain, 2008
2
Prof Trivedi ECE253A Notes for Students
only
Example: 3-means Clustering
Example:
3-means
Example: 4-means
Clustering:
WhatClustering
should be the desired clustering?
Selection of Initial Cluster Centers:
--- Does it matter?
from
Duda et al. 2005
How to judge quality of
clustering at any stage,
especially at the final
termination stage?
Convergence in 3 steps
Visualization from Prof. Andrey Shabalin
Input Color Image
Clusters on intensity
Clusters on color
K-means clustering using intensity alone and color alone
3
Prof Trivedi ECE253A Notes for Students
only
Segmentation by Clustering (5-Means)
Segmentation by Clustering (K-Means)
Segmentation by Clustering (3-Means)
Segmentation by Clustering (K-Means)
https://ilab.cs.ucsb.edu/index.php/component/content/article/12/31
4
Prof Trivedi ECE253A Notes for Students
only
Segmentation by Clustering (K-Means)
https://ilab.cs.ucsb.edu/index.php/component/content/article/12/31
Skin Tone Segmentation
In order to segment human skin regions from non-skin regions
based on color, we need a reliable skin color model that is
adaptable to people of different skin colors and to different
lighting conditions [1].
The common RGB representation of color images is not suitable
for characterizing skin-color.
In the RGB space, the triple component (r, g, b) represents not
only color but also luminance.
Luminance may vary across a person's face due to the ambient
lighting and is not a reliable measure in separating skin from nonskin region [2].
http://www-cs-students.stanford.edu/~robles/ee368/skincolor.html
Segmentation by Clustering (K-Means)
https://ilab.cs.ucsb.edu/index.php/component/content/article/12/31
Skin Tone Segmentation
Luminance can be removed from the color representation in the
chromatic color space. Chromatic colors, also known as "pure"
colors in the absence of luminance, are defined by a
normalization process shown below:
r = R/(R+G+B)
b = B/(R+G+B)
Note: Color green is redundant after the normalization because r+g+b = 1.
It is also well suited in this case to segment skin regions from non-skin regions. The
color distribution of skin colors of different people was found to be clustered in a
small area of the chromatic color space. Although skin colors of different people
appear to vary over a wide range, they differ much less in color than in brightness. In
other words, skin colors of different people are very close, but they differ mainly in
intensities [1].
http://www-cs-students.stanford.edu/~robles/ee368/skincolor.html
5
Prof Trivedi ECE253A Notes for Students
only
Skin Tone Segmentation (parametric)
Skin Tone Segmentation (parametric)
http://www-cs-students.stanford.edu/
~robles/ee368/skincolor.html
http://www-cs-students.stanford.edu/~robles/ee368/skincolor.html
Skin Tone Segmentation (parametric)
Digital Image Processing
Image Segmentation
(Chapter 10 Textbook)
• Boundary Based Segmentation
• Edges and Edge Linking
• Linear Segment Detection by Hough Transforms
• Circles and Ellipses by Hough Transforms
Quiz 4: Dec 4th 2019
HW 4: Due Day of the Finals, Dec, 12th , 2019
http://www-cs-students.stanford.edu/~robles/ee368/skincolor.html
6
Prof Trivedi ECE253A Notes for Students
only
Boundaries of Objects: How to evaluate ??
Where are Object Boundaries?
Berkeley Segmentation Data Set and Benchmarks 500 (BSDS500)
This new dataset is an extension of the BSDS300, where the
original 300 images are used for training / validation and 200
fresh images, together with human annotations, are added for
testing.
Each image was segmented by five different subjects on average.
Performance is evaluated by measuring Precision / Recall on
detected boundaries and three additional region-based metrics.
Boundaries of Objects
Boundaries of Objects from Edges
Marked by many users
Brightness Gradient (Edge detection)
• Missing edge continuity, many spurious edges
http://www.eecs.berkeley.edu/Research/Projects/CS/vision/grouping/segbench/bench/html/images.html
7
Prof Trivedi ECE253A Notes for Students
only
Boundaries of Objects from Edges: Some Options
Preprocessing Edge Images: Morphological Operations
• Preprocessing Edge Images
• Fitting Lines and Curves to Edges
• The Hough Transform
Edge detection
and Thresholding
Noisy edge image
Incomplete boundaries
Image
Shrink and Expand
Thinning
Finding lines in an image: Hough Transform
Finding lines in an image for Lane Keep System
Joel McCall, Mohan M. Trivedi, "Video Based Lane Estimation and
Tracking for Driver Assistance: Survey, System, and Evaluation",
IEEE Transactions on Intelligent Transportation Systems, 2006.
• Elegant method for direct object detection
• Edges need not be connected
• Complete object need not be visible
• Key Idea: Edges VOTE for the possible model
Hart, P. E., "How the Hough Transform was Invented”, IEEE Signal Processing
Magazine, November, 2009.
8
Prof Trivedi ECE253A Notes for Students
only
Finding lines in an image: LISA running on I-5
Finding lines in an image
R. Satzoda, M. Trivedi, "Vision-based Lane Analysis: Exploration of
Issues and Approaches for Embedded Realization,” Advances in
Embedded Computer Vision, Springer, 2014.
Better Parameterization
NOTE:
Finding lines in an image
y
−∞ ≤ m ≤ ∞
( xi , yi )
Large Accumulator
More memory and computations
Improvement:(Finite Accumulator Array Size)
Line equation: ρ
Here
Given points
= − x cosθ + y sin θ
0 ≤ θ ≤ 2π
0 ≤ ρ ≤ ρ max
( xi , yi )
find
( ρ ,θ )
Image Space
x
ρ
?
Hough Space Sinusoid
Hough Space
θ
9
Prof Trivedi ECE253A Notes for Students
only
Hough transform algorithm
Typically use a different parameterization
–
–
–
d is the perpendicular distance from the line to the origin
θ is the angle this perpendicular makes with the x axis
Why?
Basic Hough transform algorithm
1. Initialize H[d, θ]=0
2. for each edge point I[x,y] in the image
for θ = 0 to 180
H[d, θ] += 1
3. Find the value(s) of (d, θ) where H[d, θ] is maximum
4. The detected line in the image is given by
Finding lines in an image: Hough Transform
Real World Example
Original
Edge Detection
Parameter Space
Lines
10
Prof Trivedi ECE253A Notes for Students
only
Finding lines in an image: Hough Transform
Sign Detection from Real-world Images
Finding lines in an image: Hough Transform
Hough Based Lane Detector: Example
11
Prof Trivedi ECE253A Notes for Students
only
Line segments from first 15 highest peaks above threshold of 50
Setting a threshold on rho and theta for lanes
12
Prof Trivedi ECE253A Notes for Students
only
Theta
Rho
Mechanics of the Hough transform
Setting Threshold on (x,y), (rho,theta)
Issues:
How big should the cells be?
Too big, and we merge quite
different lines;
• How many lines?
– Count the peaks in the
Hough array
– Treat adjacent peaks as a
single peak
Too small, and noise causes lines • Which points belong to each
to be missed
line?
– Search for points close to
the line
– Solve again for line and
iterate
13
Prof Trivedi ECE253A Notes for Students
only
Finding Circles by Hough Transform
Generalizations of Hough Transform
Equation of Circle:
2
2
( xi − a) + ( yi − b) = r
Analytic Form
Parameters
Equation
Line
ρ, θ
xcosθ+ysinθ=ρ
Circle
x0, y0, ρ
(x-xo)2+(y-y0)2=r2
Parabola
x0, y0, ρ, θ
(y-y0)2=4ρ(x-xo)
Ellipse
x0, y0, a, b, θ
(x-xo)2/a2+(y-y0)2/b2=1
2
If radius is known: è 2D Hough Space
Accumulator Array
A(a, b)
Hough
Canny
Based
Edge
Circle
Detector:
Detector:
Example
Example
Hough Based Circle Detector: Example
>I=imread( iris.bmp );
>c=edge(I, canny );
>[x,y]=find(c==1);
1146
EE465: Introduction to Digital Image Processing Copyright Xin Li'20031148
14
Prof Trivedi ECE253A Notes for Students
only
Hough Lines
Basedand
Circle
Detector:
Example
Hough Based
Circle
Detector:
Example
Hough Based Circle Detector: Example
1151
1152
Finding Circles by Hough Transform: Courtesy Kevan !!
Hough Based Circle Detector: Example
1153
15
Prof Trivedi ECE253A Notes for Students
only
Finding Circles by Hough Transform: Courtesy Kevan !!
Finding Coins (Continued)
Note that because the
quarters and penny are
different sizes, a
different Hough
transform (with
separate accumulators)
was used for each circle
size.
Coin finding sample images from: Vivek Kwatra
Finding Coins
Original
Finding Coins (Continued)
Edges (note noise)
Penn
y
Quarters
16
Prof Trivedi ECE253A Notes for Students
only
Snakes (Active Contours)
Snake is an energy minimizing, deformable spline influenced by constraint and image
forces that pull it towards object contours. Snakes are greatly used in applications like
object tracking, shape recognition, segmentation, edge detection, stereo matching.
Snakes may be understood as a special case of general technique of matching a
deformable model to an image by means of energy minimization.[1] Snake is an “active”
model as it always minimizes its energy functional and therefore exhibits dynamic
behavior.
A simple elastic snake is thus defined by
a set of n points
an internal elastic energy term
an external edge based energy term
One may visualize the snake as a rubber band of arbitrary shape that is deforming with
time trying to get as close as possible to the object contour. Snakes do not solve the
entire problem of finding contours in images, but rather, they depend on other
mechanisms like interaction with a user, interaction with some higher level image
understanding process, or information from image data adjacent in time or space. In
general, Snake is placed near the object contour. It will dynamically move towards
object contour by minimizing its energy iteratively.
Snakes (Active Contours)
Snake is an energy minimizing, deformable spline influenced by
constraint and image forces that pull it towards object contours.
Snakes are greatly used in applications like object tracking, shape
recognition, segmentation, edge detection, stereo matching.
A simple elastic snake is defined by
1) a set of n points,
2) an internal elastic energy term and
3) external edge based energy term
Snakes: Active Contour Models, M Kass, A Witkin, D Terzopoulos,
Snakes: Active Contour Models, M Kass, A Witkin, D Terzopoulos,
http://en.wikipedia.org/wiki/Active_contour_model
http://en.wikipedia.org/wiki/Active_contour_model
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.124.5318&rep=rep1&type=pdf
Snakes (Active Contours)
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.124.5318&rep=rep1&type=pdf
Snakes (Active Contours)
One may visualize the snake as a rubber band of arbitrary shape
that is deforming with time trying to get as close as possible to
the object contour.
In general, Snake is placed near the object contour.
It will dynamically move towards object contour by minimizing its
energy iteratively.
Snakes: Active Contour Models, M Kass, A Witkin, D Terzopoulos,
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.124.5318&rep=rep1&type=pdf
http://en.wikipedia.org/wiki/Active_contour_model
Images taken from http://www.cs.bris.ac.uk/home/xie/
content.htm
17