Lecture 9 (4.7.08)
Image Segmentation
Shahram Ebadollahi
4/8/2008
DIP ELEN E4830
1
Lecture Outline
1
Skeletonization
1
1
1
1
Image Segmentation – Introduction
Thresholding
Edge Segmentation and Linking
1
1
1
Extension to Gray-level images -- GSAT
Hough Transform
Region-based Approach
Using Morphology for Segmentation
1
4/8/2008
Watershed Algorithm
2
Skeletonization (Medial Axis Transform)
B is a “Maximal Disc” in set X if there are no
other discs included in X and containing B
Notion of “Maximal Disc”
Skeleton is the loci of the centers of all
“maximal discs”
S ( X ) = 1 {ε kB ( X ) \ γ B [ε kB ( X )]}
k ≥0
4/8/2008
3
Skeletonization
K
S ( X ) = 1 Sk ( X )
Notion of “Maximal Disc”
k =0
S k ( X ) = ε kB ( X ) − γ B (ε kB ( X ))
ε kB ( X ) = ε B (ε B (2 (ε B ( X )))
K = max{k | ε kB ( X ) ≠ φ }
Reconstruction
K
X = 1 δ kB ( S k ( X ))
k =0
δ kB ( X ) = δ B (δ B (2 (δ B ( S k ( X ))))
4/8/2008
Skeleton is the loci of the
centers of all “maximal discs”
4
Gray-level SAT
(Skeletonization for Gray-level Images)
4/8/2008
5
GSAT
• GSAT is the extension of skeletonization to gray-level images
• GSAT is the locus of the centers of maximal discs whose
plane is perpendicular to the gray-level axis and which fit in the
region above or below the topographic map of image
• Symmetry surface for gray-level == skeleton for bi-level
• Represent the collection of symmetry surfaces as a graph 2
GSAT graph
• nodes:
• g_min:
gray-level at bottom of surface path
• delta_g: difference in gray-level between top and bottom
• p_avg: average location of maximal discs on the path
• n_avg: average size of maximal discs on the path
4/8/2008
6
GSAT - Implementation
4/8/2008
7
Dn ( f )
Sn ( f )
4/8/2008
8
Image Segmentation by Region
Growing on GSAT graph
4/8/2008
9
Digital Image Processing
Low level
Mid level
 x1 
x 
x =  2
3
 
 xP 
High level
4/8/2008
10
Image Segmentation - Intro
Goal decompose image into regions Rk such that:
K
f = 1 Rk
k =1
Ri 4 R j = φ i ≠ j
H ( Rk ) = true ∀k ∈ {1, 2, K }
H ( Ri 1 R j ) = false i ≠ j
There are various approaches:
• edge-based vs. region-based
• global vs. local
• feature – texture, motion, color
4/8/2008
11
Thresholding
Input image
segmented
T
f ( x, y )
g ( x, y )
1 if
g ( x, y ) = 
0 if
f ( x, y ) > T
f ( x, y ) ≤ T
•
Global Thresholding:
T is constant everywhere in the image
•
Variable Thresholding:
T varies
4/8/2008
•
Local Thresholding:
T changes based on the neighborhood properties
•
Adaptive Thresholding: T changes based on the coordinates
12
Thresholding - pitfalls
Effect of Noise
Effect of Illumination
4/8/2008
13
Global Thresholding
1.
T=T0
2.
Segment using T
3.
Get average gray-level for region G1 (f > T) and region G2 (f <= T)
4.
T_new = average of average of gray-levels
5.
Repeat until convergence
Input image
segmented
T=80.8
T=30
4/8/2008
T=42.5
T=46.7
T=52.1
14
Optimal Threshold
• Pose problem as:
minimizing error of assigning pixels in image to two or more groups
Pr( f ( x, y ) = z | ( x, y ) ∈ FG )
Pr( f ( x, y ) = z | ( x, y ) ∈ BG )
Pr( f ( x, y ) = z ) = Pr([ f ( x, y ) = z , ( x, y ) ∈ BG ] ∨ [ f ( x, y ) = z, ( x, y ) ∈ FG ])
Pr( f ( x, y ) = z ) = Pr(( x, y ) ∈ FG ) Pr( f ( x, y ) = z | ( x, y ) ∈ FG ) +
Pr(( x, y ) ∈ BG ) Pr( f ( x, y ) = z | ( x, y ) ∈ BG )
Pr( f ( x, y ) = z ) = PFG p1 ( z ) + PBG p2 ( z )
4/8/2008
where,
PFG + PBG = 1
15
Optimal Threshold (cont.)
T
E1 (T ) =
∫ p2 ( z )dz
Probability of classifying a BG
pixel as FG by mistake
−∞
∞
E2 (T ) = ∫ p1 ( z )dz
Probability of classifying a FG
pixel as BG by mistake
T
Overall probability of error:
E (T ) = PBG E1 (T ) + PFG E2 (T )
Topt = arg min E (T )
Optimal threshold minimizes the
probability of misclassification
T
Gaussian densities case:
p( z ) =
PFG
e
2π σ FG
−
( z − µ FG ) 2
2
2σ FG
+
PBG
e
2π σ BG
−
( z − µ BG ) 2
2
2σ BG
Topt
µ FG + µ BG
P
σ2
=
+
ln( FG )
µ FG − µ BG PBG
2
2
2
σ FG
= σ BG
=σ 2
4/8/2008
16
Optimal Threshold – Otsu’s method
• Otsu defines the measure of “goodness” of a threshold based on
how well it can separate two (or more) classes (regions)
σ
k∈{0 ,2, L −1} σ
k * = arg max
Goal
Between-class
Variance
Total Variance
2
B
2
T
Measure of goodness
σ B2 = ω1 ( µ1 − µT ) 2 + ω2 (µ 2 − µT )2 = ω1ω2 (µ 2 − µ1 )2
L −1
2
σ T2 = ∑ pi (i − µT )
i =0
Thresholding in MATLAB
using Otsu’s method for
determining the threshold:
I=imread('coins.png');
level=graythresh(I);
BW=im2bw(I,level);
imshow(BW)
4/8/2008
17
Adaptive Thresholding – Image Partitioning
Global threshold
obtained using
iterative algorithm
Global threshold
obtained using
Otsu’s method
Under what condition this method fails?
4/8/2008
Bimodal
histograms
18
Adaptive Thresholding – Moving Average
1 k +1
1
(
)
(zk +1 − zk −n )
m(k + 1) =
z
=
m
k
+
∑
i
n i = k + 2− n
n
original
4/8/2008
Otsu
Moving average
19
Image Derivatives
• 1st order derivative produces
“thick” edges
• 2nd order derivative has stronger
response to fine edges
• 2nd order derivative produces
double edge response at ramp
and step transitions in intensity
• 2nd order derivative’s sign can be
used to find out if going from dark
to light or vice versa
f ' ( x) = f ( x + 1) − f ( x)
f " ( x) = f ( x + 1) + f ( x − 1) − 2 f ( x)
4/8/2008
20
Edge detection – gradient operator
 ∂f 
 g x   ∂x 
∇f =   =  ∂f 
g y   
 ∂y 
M ( x, y ) = g x2 + g y2
Edge magnitude
 gx 
α (x, y ) = tan  
 g y 
Edge direction
−1
Gradient operator for discrete images
g x = f ( x + 1, y ) − f ( x, y )
g y = f ( x, y + 1) − f ( x, y )
4/8/2008
21
2-D gradient operators
g x = ( z9 − z5 )
g y = ( z8 − z 6 )
g x = ( z7 + z8 + z9 ) − ( z1 + z 2 + z3 )
g y = ( z3 + z6 + z9 ) − ( z1 + z 4 + z7 )
g x = ( z7 + 2 z8 + z9 ) − ( z1 + 2 z 2 + z3 )
g y = ( z3 + 2 z6 + z9 ) − ( z1 + 2 z 4 + z7 )
4/8/2008
for diagonal edges
22
Edge detection
Averaged prior to edge detection
After thresholding
4/8/2008
23
Edge detection: Marr-Hildreth method
* Edge detection operator should be “tunable” to
detect edges at different “scales”
∇ 2G
LoG
∂2
∂2
∇ = 2+ 2
∂x
∂y
2
G ( x, y ) = e
−
x2 + y2
2σ 2
 x + y − 2σ 
∇ 2 G ( x, y ) = 
e
4
σ


2
4/8/2008
2
2
−
x2 + y2
2σ 2
24
Edge detection: Marr-Hildreth method
[
]
g ( x, y ) = ∇ 2G ( x, y ) * f ( x, y ) = ∇ 2 [G (x, y )* f ( x, y )]
In practice
- sample Gaussian function [nxn samples]
- convolve with image f(x,y): image smoothing
- convolve result with Laplacian mask
- find zero-crossings of g(x,y)
How?
g ( x, y )
Low zerocrossing
threshold
4/8/2008
Higher zerocrossing
threshold
25
Edge detection: Canny method
* An optimal method for detecting step edges corrupted by white noise
•Goal Satisfy the following 3 criteria:
• Detection: should not miss important edges
• Localization: distance between the actual and located position of the
edge should be minimal
• One response: only one response (edge) to a single actual edge
•Algorithm
• Step 1: Smooth input image with a Gaussian filter
• Step 2: Compute the gradient magnitude and angle images
f s ( x, y ) = G ( x, y ) * f ( x, y )
G ( x, y )= e
4/8/2008
−
x2 + y 2
2σ 2
M ( x, y ) = g x2 + g y2
 gx 
α (x, y ) = tan  
 g y 
−1
26
Edge detection –
Step 3: nonmaxima suppression
1. Find direction d k closest to α ( x, y )
2. If value of M(x,y) is less than at
least one of its neighbors along d k ,
let g N ( x , y ) = 0 , otherwise
g N ( x, y ) = M ( x, y )
4/8/2008
27
Edge detection –
Step 4: Reducing false edge points
(hysteresis thresholding)
Strong edges:
g NH ( x, y ) = g N ( x, y ) ≥ TH
g NL ( x, y ) = g N ( x, y ) ≥ TL
Weak edges:
g NL ( x, y ) = g NL ( x, y ) − g NH ( x, y )
1. Locate unvisited edge pixel in the strong edge image
2. Mark as valid edge pixels all the weak pixels that are connected to above
pixel in the neighborhood
3. If all nonzero pixels in strong edge image havee been visited continue,
otherwise go to (1)
4. Set to zero all pixels in weak edge image not already marked
5. Append all remaining non-zero pixels from weak edge image to strong
edge image
4/8/2008
28
Edge detection: Canny method
4/8/2008
29
Hough Method for Curve Detection
Goal: find subset of points
in the image that lie on a
straight line
yi = axi + b
b = − xi a + yi
y j = ax j + b
b = −x ja + y j
( a, b)
Accumulator matrix
A( p, q)
A( p, q ) = M
⇒
M points in image
with slope ap and
4/8/2008
intercept bq
Algorithm:
1.
For each point (x,y) in image
determine ap and bq that
satisfy the line equation
2.
Increment A(p,q) by 1
30
Hough Method: Normal line representation
x cos θ + y sin θ = ρ
Algorithm:
4/8/2008
1.
Quantize parameter space
2.
Initialize the accumulator matrix
3.
For each point in the edge image, iterate on
choices of angle bins and find distance.
Increase corresponding accumulator bins
4.
Lines in image correspond to the local
maxima in the accumulator array
31
Hough Method: Matlab
image
accumulator
hough()
houghlines()
houghpeaks()
4/8/2008
32
4/8/2008
33
Edge-based segmentation:
Problems
1
1
1
1
Spurious edges due to noise and low quality
image. Difficult to identify spurious edges.
Dependent on local neighborhood information
No notion of higher order organization of the
image
Gaps and discontinuities in the linked edges
4/8/2008
34
Split & Merge Method
1. Pick a grid structure and
homogeneity property H
2. If for any region R, H(R)=false,
split region into 4
3. If for any neighboring regions
H(R1UR2)=true, merge regions
into single region
4. Stop when no more split or merge
H ( Rk ) = true cnt (| z j − mk |≤ 2σ k ) ≥ 0.8
4/8/2008
35
Watershed
4/8/2008
Goal: Find watershed lines
Build dams to prevent water flow from one
catchment basin to other
36
Boundaries between flooded
catchment basins
4/8/2008
37
Segmentation using Watershed
4/8/2008
38
Segmentation as Clustering in
Feature Space
Feature Space
(Color, Texture)
B
G
R
Clustering algorithms:
4/8/2008
Issues/Choices:
• K-means
• Feature
• Gaussian Mixture Model
• Distance
• Neural Network
• Method
39
K-means Clustering Algorithm
1
1
1
1
Select K initial cluster centers:
1 3
2
C1 , C2 , 2, C K
Assign each pixel
representation xi to nearest
cluster Ck
Recomputer cluster centroids
for all clusters
Iterate until convergence
Change in clusters and
migration of centroids in
consecutive steps
4/8/2008
40