Chap 7
Image Segmentation
Image Segmentation

Edge-Based Segmentation


Pixel-based direct classification methods


The edge information is used to determine boundaries
of objects
The estimation methods derived from the histogram
statistics of the image are used
Region-based methods

Pixels are analyzed directly for a region growing
process based on a pre-defined similarity criterion
Edge-based image segmentation
-Edge Detection Operations

The gradient magnitude and directional
information can be obtained by convolving the
edge detection masks with the image
Sobel operation
 1
G x   2
  1
1
G y   0
 1
0 1
0 2
0 1 
2
1
0
0 
 2  1
M  G x2  G y2  G x  G y
Second  order gradient ( Laplacian)
G L ( 4)
G L (8 )
0
  1
 0
 1
  1
 1
1 0 
4  1
 1 0 
 1  1
8  1
 1  1
Edge-based image segmentation
-Edge Detection Operations

Laplacian is very sensitive to noise

It is beneficial to apply a smoothing filter(Gaussian)
first before taking a Laplacian of the image


Laplacian of Gaussian (LOG)
The image obtained by convolving the LOG mask with
the original image is analyzed for zero-crossing to detect
edges.
h( x, y )   2 g x, y   f x, y 
  2 g x, y   f x, y 
 x 2  y 2  2 2


4

 x 2  y 2 

2 2
 
e


 f  x, y 
[Gonzalez]
Edge-based image segmentation
-Edge Detection Operations

LOG operation
Sobel
LoG
Threshold = 0
[Gonzalez]
Edge-based image segmentation
-Boundary Tracking

The closed regions can be formed by the edge-linking
procedures following the edge-detection
The neighborhood bj+1 of edge point bj can be found to be a boundary
point if it satisfied the following conditions.
 
eb j 1   T1
eb j   eb j 1   T2
 b j    b j 1  mod 2  T3
e b j  T1
Where e is the magnitude and  the edge orientation
Edge-based image segmentation
Hough Transform
Haugh Transform
x - y space
yi  axi  b
[Gonzalez]
parameter space

b   xi a  yi
Edge-based image segmentation
Hough Transform
O
r
p
Gradient e
Figure 7.2. A model of the object shape to be detected in the image using Hough transform. The
vector r connects the Centroid and a tangent point p. The magnitude and angle of the vector r
are stored in the R-table at a location indexed by the gradient of the tangent point p.
Gray-Level Thesholding
1
g ( x, y )  
0
if f ( x, y )  T
if f ( x, y )  T
Optimal Global Thresholding

The parametric distribution based methods can be applied to
the histogram of an image

Assume the histogram has two Gaussian distributions
p( z )  P1 p1 ( z )  P2 p2 ( z )
P(z)
Background
object
z
Optimal Global Thresholding
Class 2
P2(z)
P(z)
p( z )  P1 p1 ( z )  P2 p2 ( z )
p1 ( z ) : The Gaussian distribution of class 1
Class 1
P1(z)
object
Background
p2 ( z ) : The Gaussian distribution of class 2
P1  P2  1
E1
a pixel f ( x,y ) can be classified to class 1 or class 2 of the segmented image g ( x,y )
 Class 1 (object )
g ( x, y )  
Class 2 (background)
if f ( x, y )  T
if f ( x, y )  T
E2
z
T
The error probabilit ies of misclassif ying a pixel
T
E1 (T ) 
 p ( z)dz
: The probabilit y of erroneously classifying a class 1 pixel to class 2
 p ( z)dz
: The probabilit y of erroneously classifying a class 2 pixel to class 1
1


E2 (T ) 
2
T
The overall probabilit y of error in pixel classification using the threshold T
E (T )  P2 E1 (T )  P1 E2 (T )
E (T )  P1 (T ) E1 (T )  P2 (T ) E2 (T )
Mistake in textbook
Optimal Global Thresholding
E (T )  P2 E1 (T )  P1E2 (T )

(7.14)
For image segmentation, the objective is to find an optimal threshold T
that minimizes the overall probability of error in pixel classification.

The process requires the parameterization of the probability density
distributions to find the 1, 1, and 2, 2
p( z ) 

P1
2  1
e ( z  1 )
2
2 12

P2
2  2
e ( z   2 )
2
2 22
(7.15 )
The optimal global threshold T can be determined by finding a general solution
the minimizes the equation (7.14) with mixture distribution in Equation (7.15)
Optimal Global Thresholding

(7.14) thus satisfies the following quadratic expression
AT 2  BT  C  0
where
A   12   22
B  2( 1 22   2 12 )
C   12  22   22 12  2 12 22 ln( 2 P1 /  1 P2 )

If the variances of both classes can be assumed to be equal to
T
1   2
2
P 
2

ln  2 
1   2  P1 
The determined threshold
2
Pixel Classification Through Clustering

The approach is useful when images with pixels
representing a feature vector consisting of multiple
parameters of interest are to be segmented.


Parameters: Gray value, RGB components, contrast and
local texture measurements for each pixel
Clustering may produce disjoint regions with holes or
regions with a single pixel


Post-processing algorithm is usually applied
region growing, pixel connectivity or rule-based algorithm
Pixel Classification Through Clustering
k-means Clustering



Partition d-dimentional data into k clusters
The objective function providing the desired properties of the
distribution of feature vectors of clusters
The algorithm is quite sensitive to the initial cluster assignment
and the choice of the distance measure. (See the following
algorithm for more)
Pixel Classification Through Clustering
k-means Clustering
Algorithm
1.
Select the number of clusters k with initial cluster centroids vi ;
i=1,2,…,k
2.
Partition the input data points into k clusters by assigning each data
point xj to the closest cluster centroid vi .(ex. Euclidean distance)
d ij  x j  vi
3.
Compute a cluster assignment matrix U representing the partition
of the data points with the binary membership value of the jth data
point to the ith cluster such that U=[uij], where
uij
clusters(k )
points(n)
1 0 0 0
0 1 1 0


0 0 0 1
uij  0,1
for all i, j
k
u
i 1
n
ij
 1 for all j and 0 
u
j 1
ij
 n for all i
Pixel Classification Through Clustering
k-means Clustering
Algorithm (cont.)
4.
Re-computer the centroids using the membership values as
n
u
vi 
uij
ij x j
j 1
n
u
for all i
clusters(k )
ij
j 1
5.
points(n)
1 0 0 0
0 1 1 0


0 0 0 1
If cluster centroids does not change, stop; otherwise go to
step 2
The k-means clustering method optimizes the sum-of-squared-error
based objective function Jw(U,v)
k
J w (U , v) 
n

i 1 j 1
x j  vi
2
Pixel Classification Through Clustering
Fuzzy k-means Clustering

Utilize an adaptable membership value (uij) that can be updated
c
J w (U , v) 
n

c
uijm d ij2

i 1 j 1
n

uijm
x j  vi
2
i 1 j 1
c : the number of clusters
n : the number of data vectors
uij : the fuzzy membership
m : the fuzziness index
0  uij  1 for all i,j
c
u
i 1
n
ij
1
for all j and 0 
u
j 1
ij
 n for all i
Region Growing



Examine pixels in the neighborhood based on a predefined similarity criterion
The neighborhood pixels with similar properties are
merged to form closed regions for segmentation
Guarantee the segmented regions of connected
pixels
Region Growing

Two criteria


A similarity criterion that defines the basis for inclusion of
pixels in the growth of the region
A stopping criterion stops the growth of the region
1. Determine the seed for each region
( the pixels with gray 255 are selected
to be seeds in this case)
2. Choose criteria for region growing
• The absolute gray-level difference
between any pixels and seed
• The detected pixel had to be 8connected to at least one pixel in
that region
[Gonzalez]
Region
Growing
Center Pixel
Pixels satisfying the similarity criterion
Pixels not satisfying similarity criterion
1st iteration
3x3 neighborhood, 100% (9/9) growth satisfied
2nd iteration
5x5 neighborhood , 56%(9/16) growth satisfied
3rd iteration
7x7 neighborhood, %(6/24) growth satisfied
The stopping criterion : the minimum percentage is 30%
Segmented region
Region -Merging

An image is partitioned into a large number of
potential homogeneous regions


Be examined for homogeneity of pre-defined property as
gray values, contrast, texture…
Two neighborhood regions with similar property are merged
Case 1
Case 2
Region -Splitting

Examine the heterogeneity of a predefined property
of the entire region in terms of its distribution and
the mean, variance, minimum and maximum values.


Heterogeneous region R is split into two or more regions R1,
R2, …, Rn
Rule-based splitting and quad-tree based splitting
1. Each region, Ri ; i  1,2,...,n is connected.
n
2.
R
i
 R.
i 1
3. Ri  R j  O for all i, j; i  j.
4. H ( Ri )  TRUE for i  1,2,...,n .
5. H ( Ri  R j )  FALSE for i  j ,
where H ( Ri ) is a logical predicate for the homogeneity criterion on the region Ri .
Region -Splitting

An image with quad region-splitting process
R1
R3
R21
R22
R23
R24
R41
R42
R43
R44
R
R1
R21
R22
R2
R23
R3
R24
R4
R41
R42
Quad-tree structure
R43
R44
Recent advances in segmentation
The large variability in anatomical structures needs of a
reliable, accurate, and diagnostically useful segmentation
 Model-based estimation methods
 Rule-based methods
Estimation-Model Based Adaptive Segmentation
-Multi-level Adaptive Segmentation (MAS)
Define Classes
Determination of Model Parameters
(From a set of
manually segmented and labeled images)
Classification of Image
Pixels Using Model
Signatures
Parameter
Relaxation
No
No
All pixels
classified?
Yes
Identification of Tissue
Classes and Pathology
Formation of
a New Class
Necessary?
Yes
Image Segmentation Using Neural Networks
-Neural Network



Neural Network do not require underlying class probability
distribution for accurate classification
The decision boundaries for pixel classification are adapted
through an iterative training process (supervised neural network)
Neural Network paradigms

Backpropagation, Radial Basis Function, and Self-Organizing
Feature Maps
Image Segmentation Using Neural Networks
-Neural Network

Important works for neural network



Selection of a useful set of features as input data for classification
Selection of the training examples
The structure of the network
Artificial Neural Network: Backpropagation
Output Results
y 2L
y1L
y nL
Output Layer
Neurons
Layer L
.
.
.
Hidden Layer
Neurons
Layer 1
x1
x2
xn
x3
Input Features
1
Image Segmentation Using Neural Networks
-Backpropropagation Neural Network
x1
w1
x2
w2
S
 n

y  F   wi x i  wn 1 ÷
 i 1

wn
Non-Linear Activation Function F
wn+1
xn
1
bias
 n

y  F
wi xi  wn 1 


 i 1

F ( y ) can be defined as a sigmoidal function
1
F y 
1  e y

Figure 7.11. A basic computational neural element or Perceptron for classification
Image Segmentation Using Neural Networks
-Backpropropagation Neural Network


y k   F W k y k 1 for k  1,2,... L
Architectural graph
where y k  is the output of the kth layer
 y1k  
 x1 
 k  
x 
 y2 
 2
y 0    . ; y k    . 
 k  
 
 yn 
 xn 
 1 
 1 


and
k 
 w11
 k 
 w21

k
W  .

 wnk1


k  . w k 
w12
1n
k  . w k 
w22
2n
.
.
.
k 
wnk2 . wnn
.
Input HiddenOutput
layer layer layer
Signal-flow graph
y0=-1
w1kn1 


k
w2n 1 
. 


k
wn n 1 


wj0(n)
yi(n)
F.
wji(n)vj(n)
yj(n)
Layer k
Layer k+1
Image Segmentation Using Neural Networks
-Backpropropagation Neural Network
Least Mean Squared(LMS) error algorithm for training the neural
network
1.Assign random weights in the range of [-1,+1] to all weights wijk
2.For each classified pattern pair {y(0),yL} in the train set, do the following steps:
a. Compute the output values of each neural element using the current weight matrix.
b. Find the error e(k) between the computed output vector and the desired out vector.
c. Adjust the weight matrix using the change dW(k) computed as dW(k) =ae(k)[y(k-1)] for
all layers =1,…,L, where a is the learning rate that can be set between 0 and 1.
3.Repeat Step 2 for all classified pattern pairs in the training set until the error
vector for each training example is sufficiently low or zero.
RBF Network
Output
Linear
Combiner
RBF Unit
1
RBF Unit
2
Sliding
Image
Window
RBF Unit
n
Input
Image
RBF
Layer
RBF Network

The final output of the network f(x)
k
f ( x) 
 w (x, i)
i
i 1
 x  ci 

exp  
2 i 

with (x, i ) 
K
 xcj
exp  

2 j
j 1






ci : the K centers
wi : the K weighted connections from the hidden layer units to the output unit.
RBF Network

Within the basic topology, the system can be
represented in state-space from y=(F,w),
where F is the matrix of activation functions
  ( x1 ,1) ...  ( x1 , K ) 

F   ...
...
...

 ( x N ,1) ...  ( x N , K )
The weight van be calculated from the Moore-Penrose inverse

w  FT  F a  I

1
 FT  y
where a is a small value such as 0.1
RBF NN Based Segmentation