Chapter 7
Image Segmentation
Chuan-Yu Chang (張傳育)Ph.D.
Dept. of Electronic Engineering
National Yunlin University of Science & Technology
chuanyu@yuntech.edu.tw
Office: ES709
Tel: 05-5342601 Ext. 4337
Introduction


Image segmentation refers to the process of
partitioning an image into distinct regions by
grouping together neighborhood pixels based
on some pre-defined similarity criterion.
Segmentation is a pixel classification
technique that allows the formation of regions
of similarities in the image.
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
2
Introduction (cont.)

Image segmentation methods can be broadly
classified into three categories:

Edge-based methods


Pixel-based direct classification


The edge information is used to determine boundaries of
objects. The boundaries are then analyzed and modified to
form closed regions belonging to the objects in the image.
Heuristics or estimation methods derived from the histogram
statistics of the image are used to form closed regions
belonging to the objects in the image.
Region-based methods

Pixels are analyzed directly for a region growing process
based on a pre-defined similarity criterion to form closed
regions belonging to the objects in the image.
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
3
Image Segmentation






Edge-Based Segmentation
Gray-level Thresholding
Pixel Clustering
Region Growing and Splitting
Artificial Neural Network
Model-Based Estimation
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
4
Edge-Based Segmentation

Edge-Based approaches




use a spatial filtering method to compute the first-order or
second-order gradient information of the image.
Sobel, directional derivative masks can be used to compute
gradient information of the image.
Laplacian mask can be used to compute second-order
gradient information of the image.
For segmentation purpose


Edges need to be linked to form closed regions
Gradient information of the image is used to track and link
relevant edges.
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
5
Introduction

Image segmentation algorithm generally are
based on one of two basic properties of intensity
values:

Discontinuity


Similarity


Partition an image based on abrupt changes in intensity.
Partitioning an image into regions that are similar according
to a set of predefined criteria.
There are three basic types of gray-level
discontinuities

Points, lines, and edges.
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
6
Detection of discontinuities

最常用來偵測discontinuity的方法是對整張影像計算
mask區域內的積之和(sum of product)
R  w1 z1  w2 z2  ...  w9 z9
9
  wi zi
i 1
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
7
Detection of discontinuities

Point detection

一個孤立的點，其灰階值一定和其周圍的點有相當明顯的差
異。
飛機渦輪
葉片的X光
影像
R T
Mask 計 算
後的結果
取(c)圖最大灰
階 的 90% 作 為
threshold 後 的
結果
T為非負值的Threshold
有小破洞
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
8
Detection of discontinuities

Line detection


令R1, R2, R3, 及R4分別代表以下圖mask計算後的結果，如果
影像中某點的|Ri|>|Rj|，則表示此點比較像第i種線條的方向。
若要偵測影像中某方向的線條時，可以該方向的mask進行運
算，再取threshold後，留下的就是結果。
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
9
Detection of discontinuities


We are interested in finding all the lines that are one pixel thick and are
oriented at -45.
Use the last mask shown in Fig. 10.3.
強度最強的
line
偵測到的45度
line
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
10
Detection of discontinuities

Edge detection


An edge is a set of connected pixels that lie on
the boundary between two regions.
The “thickness” of the edge is determined by
the length of the ramp.

Blurred edges tend to be thick and sharp edges tend
to be thin.
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
11
Detection of discontinuities

The first derivative is positive at the points of transition
into and out of the ramp as we move from left to right
along the profile.



It is constant for points in the ramp,
It is zero in areas of constant gray level.
The second derivative is positive at the transition
associated with the dark side of the edge, negative at
the transition associated wit the light side of the edge,
and zero along the ramp and in areas of constant gray
level.
一階微分的大小可用來檢測是否存在邊界點，
二階微分的正負值可用來決定邊是在亮邊或暗邊。
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
12
Edge detection (cont.)

Two additional properties of the second
derivative around an edge:


It produces two values for every edge in an image.
An imaginary straight line joining the extreme
positive and negative values of the second
derivative would cross zero near the midpoint of
the edge.

The zero-crossing property of the second derivative is
quit useful for locating the centers of thick edges.
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
13
Detection of discontinuities

The entire transition from black to white is a single edge.
s=0
Image and gray-level profiles
of a ramp edge
s=0.1
First derivative image and the
gray-level profile
s=1
Second derivative
image and the graylevel profile
s=10
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
14
Edge detection (cont.)



The second derivative is even more sensitive to noise.
Image smoothing should be a serious consideration prior to the
use of derivatives in applications .
Summaries of edge detection
 To be classified as a meaningful edge point, the transition in gray
level associated with that point has to be significant stronger than
the background at that point.



Use a threshold to determine whether a value is “significant” or not.
We define a point in an image as being an edge point if its two
dimensional first-order derivative is greater than a specified threshold.
A set of such points that are connected according to a predefined
criterion of connectedness is by definition an edge.
Edge segmentation is used if the edge is short in relation to the
dimensions of the image.
 A key problem in segmentation is to assemble edge
segmentations into linger edges.
If we elect to use the second-derivative to define the edge points
in an image as the zero crossing of its second derivative.


醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
15
Detection of discontinuities

Gradient operator

The gradient of an image f(x,y) at location (x,y) is defined as
the vector
 f 
G x   
f      fx 
G y   
 y 

the gradient vector points is the direction of maximum rate of
change of f at coordinates (x,y).
The magnitude of the vector denoted ∇f, where

f  mag(f )  G  G

2
x

1
2 2
y
The direction of the gradient vector denoted by
 Gy
 Gx
 ( x, y )  tan 1 



The angle is measured with respect to the x-axis.
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
16
Detection of discontinuities

Roberts operator



Prewitt operator



Gx=(z9-z5)
Gy=(z8-z6)
Gx=(z7+z8+z9)-(z1+z2+z3)
Gy=(z3+z6+z9)-(z1+z4+z7)
Sobel operator


Gx=(z7+2z8+z9)-(z1+2z2+z3)
Gy=(z3+2z6+z9)-(z1+2z4+z7)
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
17
Detection of discontinuities

由於Eq.(10.1-4)的計算較耗時，因此梯度的計算通常
以下式來近似：
f  G x  G y
用來偵測對角邊界的
Prewitt及Sobel
mask。
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
18
Detection of discontinuities
Example 10-4 原始影像未smooth，造成一些細微的邊界點。
Fig. 10.10 shows the response of the two components of the gradient, |Gx| and |Gy|.
The gradient image formed the sum of these two components.
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
19
Detection of discontinuities
Example 10-4 原始影像經5*5 averaging filter ，移除細微的邊界點。
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
20
Detection of discontinuities
Example 10-4 原始影像經Sobel 對角線邊界mask處理後的結果，
強化的45度角的邊界點。




The horizontal and vertical Sobel masks respond about equally well to
edges oriented in the minus and plus 45° direction.
If we emphasize edges along the diagonal directions, the one of the mask
pairs in Fig. 10.9 should be used.
The absolute responses of the diagonal Sobel masks are shown in Fig.
10.12.
The stronger diagonal response of these masks is evident in these figures.
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
21
Detection of discontinuities

The Laplacian of a 2-D function f(x,y) is a second-order
derivative defined as
2 f 2 f
 f  2  2
x
y
2

For a 3x3 region, one of the two forms encountered most
frequently in practice is (水平與垂直邊)
2 f  4z5  z2  z4  z6  z8 

A digital approximation including the diagonal neighbors is
given by (含水平、垂直、及對角邊)
2 f  8z5  z1  z2  z3  z4  z6  z7  z8  z9 
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
22
Detection of discontinuities

雖然Laplacian operator對於影像亮度的變化有反應，
但因為下列缺點，仍很少應用在實際的邊界偵測：




對雜訊太敏感
產生雙邊緣
無法偵測出邊的方向
因此，Laplacian operator在影像分割的角色在於


使用過零點(zero-crossing)的特性找出邊緣的位置
幫助確定一個像素點是在暗側還是亮側。
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
23
Detection of discontinuities

Laplacian of a Gaussian (LoG)

The Laplacian is combined with smoothing as a precursor to
finding edges via zero-crossing, consider the function
h(r )  e



r2
2s 2
where r2=x2+y2 and s is the standard deviation.
Convolving this function with an image blurs the image, with
the degree of blurring being determined by the value of s.
The Laplacian of h is
r2
 r 2  s 2   2s 2
2
 h( r )   
e
2
s



The function is commonly referred to as the “Laplacian of a
Gaussian” (LoG), sometimes is called the ”Mexican hat”
function
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
24
Detection of discontinuities
LoG中，Gaussian函數的目的在於smooth影像，
而Laplacian運算子則以ZC來找出邊的位置。
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
25
Detection of discontinuities

Comparing Figs/ 10.15(b) and (g)


The edges in the zero-crossing image are thinner
than the gradient edges.
The edges determined by zero-crossings form
numbers closed loops.


“spaghetti effect” is one of the most serious drawbacks
of this method.
The major drawback is the computation of zero
crossing.
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
26
Detection of discontinuities
ZC的偵測，可將LoG中的正值設為白色，
負值設為黑色，以得到近似的結果。
Gradient edge與LoG edge的差異：
1. ZC的邊比gradient的邊細。
2. ZC的邊有義大利麵的現象發生，
(region化)
3. ZC的計算量相當大
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
27
Edge Linking and Boundary Detection


Edge detection的目的在於找出邊界上的點，但這些點很
可能會有斷裂的現象，因此需要邊的連接(edge linking)
來將邊界點組合成有意義的邊界。
Edge linking



Based on pixel-by-pixel search to find connectivity among
the edge segmentations.
The connectivity can be defined using a similarity criterion
among edge pixels.
Local Processing


分析每個被標記為邊界點的pixel在一個小區域(3x3, 5x5)的
特徵，把所有相似的點連接起來，這些具有相似特性的像素
點就形成了邊界。
建立邊界點相似性的兩個主要特性：


用來產生邊界點的梯度運算子反應的強度，Eq.(10.1-4)
梯度向量的方向， Eq. (10.1-5)
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
28
Edge Linking and Boundary Detection


像素點(x,y)和(x0,y0)的梯度相似性
f ( x, y)  f ( x0 , y0 )  E
where E is a nonegativethreshold
像素點(x,y)和(x0,y0)的梯度角度相似性
 ( x, y)   ( x0 , y0 )  A

where A is a nonegativeangle threshold
如果上面兩個條件均滿足，則將(x0,y0)與(x,y)
連接起來。
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
29
Edge Linking and Boundary Detection



Example 10-6: the objective is to find rectangles whose sizes
makes them suitable candidates for license plates.
The formation of these rectangles can be accomplished by
detecting strong horizontal and vertical edges.
Linking all points, that had a gradient value greater than 25 and
whose gradient directions did not differ by more than 15°.
使用垂直的
Sobel operator
使用水平的
Sobel operator
分別對圖(b)及(c)
進行edge linking
的動作，將梯度
大於25，且角度
小於15度的點連
起來。
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
30
Boundary Tracking

A* search algorithm
 Select an edge pixel as the start node of the boundary and
put all of the successor boundary pixels in a list, OPEN
 If there is no node in the OPEN list, stop; otherwise
continue.
 For all nodes in the OPEN list, compute the cost function
t(z) and select the node z with the smallest cost t(z).
Remove the node z from the OPEN list and label it as
CLOSED. The cost function t(z) may be computed as
t ( z )  c ( z )  h( z )
k
c(k ) 
 sz
i 2
   d z j 
k
i 1 , z i
j 1
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
31
Boundary Tracking (cont.)




If z is the end node, exit with the solution path by tracking
the pointers; otherwise continue.
Expand the node z by finding all successors of z. If there is
no successor , goto Step 2; otherwise continue.
If a successor zi is not labeled yet in any list, put it in the list
OPEN with updated cost as
c(zi)=c(z)+s(z, zi)+d(zi) and a pointer to its predecessor z.
If a successor zi is already labeled as CLOSED or OPEN,
update its value by
c’(zi)=min[c(zi),c(z)+s(z,zi)]
Put those CLOSED successors whose cost function x’(zi) s
were lowered, in the OPEN list and redirect to z the
pointers from all nodes whose cost were lowered. Goto
Step 2.
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
32
Boundary Tracking (cont.)
Start Node
End Node

Figure 7.1. Top: An edge map with magnitude and direction
information; Bottom: A graph derived from the edge map with a
minimum cost path (darker arrows) between the start and end
nodes.
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
33
Edge Linking and Boundary Detection

Global Processing via the Hough Transform

點是否被連接，端視該點是否處於一特定形狀的曲線而定。
在xy平面上，a’表通過(xi, yi)和(xj, yj)的直線，b’表截距
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
34
Edge Linking and Boundary Detection

Hough Transform





將參數空間細分成accumulator cell
一開始所有的accumulator cell均設為0
對影像上的每一點(xk,yk) ，將參數a設成a軸上每一個允許的細分值，
用等式b=-xka+yk計算對應的b值。
將得到的b值化整到最近的b軸
若ap的結果為bq，則A(p,q)=A(p,q)+1
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
35
Edge Linking and Boundary Detection

當直線趨近垂直時，斜率趨近無窮大，因此將直線方
程式改以下式表示：
x cos  y sin   
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
36
Edge Linking and Boundary Detection
X, Y平面上有
五個點(1, 2, 3,
4, 5)
X, Y平面
上五個點
(1, 2, 3, 4,
5) ，在
平面的曲
線
從交點A知
道， 點1, 3,
5)共線。
交點B表示
點2,3,4共線
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
37
Edge Linking and Boundary Detection

Edge-linking based on Hough transform





計算影像的梯度，並取threshold以得到binary
image。
訂出平面的範圍。
對二元化影像進行Hough Transform。
檢查accumulator cell中較高的值
依其對應的找出直線。
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
38
Edge Linking and Boundary Detection
Sobel的結果
跑道有斷格
空中紅外線
影像，有兩
個停機坪及
跑道
Hough
Transform結果
從accumulator
cells 中找出前三
大值
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
39
Edge Linking and Boundary Detection

Global Processing via Graph-Theoretic Techniques



將邊界點的線段以圖形法表示，找尋最低成本的路徑來代表重要的
邊界。
此法在雜訊影像中有不錯的效果。但演算法複雜、耗時。
Graph G=(N,U)








N: node的集合
U: N的元素對集合
U 的元素(ni,nj)對稱為arc，ni,稱為parent， nj稱為successor。
確認successor的過程稱為expansion。
Level 0只有一個點稱為start或root。最後的level稱為goal。
Cost (ni,nj)為每個arc的成本
一序列的節點稱為路徑(path)
n1到nk的Path成本(cost)定義成
k
c   cni 1 , ni 
p
q
i 2
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
40
Edge Linking and Boundary Detection
每一個edge element定義成
灰階值
座標
成本
c( p, q)  H   f ( p)  f (q)
其中，H為影像中最大的灰階值，
f(x)為x點的灰階值
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
41
Edge Linking and Boundary Detection






By convention, the point p is on the right-hand side of the direction
of travel along edge elements.
To simplify, we assume that edges start in the top row and
terminate in the last row.
p and q are 4-neighbors.
An arc exists between two nodes if the two corresponding edge
elements taken in succession can be part of an edge.
The minimum cost path is shown dashed.
Let r(n) be an estimate of the cost of a minimum-cost path from s
to n plus an estimate of the cost of that path from n to a goal node;
r n   g n   hn 

(10.2-7)
Here, g(n) can be chosen as the lowest-cost path from s to n found
so far, and h(n) is obtained by using any variable heuristic
information.
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
42
Edge Linking and Boundary Detection
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
43
Edge Linking and Boundary Detection

Graph search algorithm






Step1: Mark the start node OPEN and set g(s)=0.
Step 2: If no node is OPEN exit with failure; otherwise, continue.
Step 3: Mark CLOSE the OPEN node n whose estimate r(n) computed from
Eq.(10.2-7) is smallest.
Step 4: If n is a goal node, exit with the solution path obtained by tracing
back through the pointets; otherwise, continue.
Step 5: Expand node n, generating all of its successors (If there are no
successors go to step 2)
Step 6: If a successor ni is not marked, set
r ni   g n   cn, ni 

Step 7: if a successor ni is marked CLOSED or OPEN, update its value by
letting
g ' ni   min g ni , g n  cn, ni 
Mark OPEN those CLOSED successors whose g’ value were thus lowered
and redirect to n the pointers from all nodes whose g’ values were lowered.
Go to Step 2.
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
44
Edge Linking and Boundary Detection
Example 10-9: noisy chromosome silhouette(有雜訊的染色體影像)
The edge is shown in white, superimposed on the original image.
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
45
Thresholding

Thresholding
 To select a threshold T, that separates the objects form
the background.


Then any point (x,y) for which f(x,y)>T is called an object point;
otherwise, the point is called a background point.
Multilevel thresholding


Classifies a point (x,y) as belonging to one object class if T1
<f(x,y) ≤T2, and to the other object class if f(x,y) >T2
And to the background if f(x,y) ≤T2
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
46
Thresholding


In general, segmentation problems requiring multiple thresholds are
best solved using region growing methods.
The thresholding may be viewed as an operation that involves tests
against a function T of the form
T  T x, y, px, y , f x, y 




where f(x,y) is the gray-level of point (x,y) and p(x,y) denotes some
local property of this point.
A threshold image g(x,y) is defined as
1 if f ( x, y )  T
g  x, y   
0 if f ( x, y )  T
Thus, pixels labeled 1 correspond to objects, whereas pixels labeled
0 correspond to the background.
When T depends only on f(x,y) the threshold is called global. If T
depends on both f(x,y) and p(x,y), the threshold is called local.
If T depends on the spatial coordinates x and y, the threshold is called
dynamic or adaptive.
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
47
The role of illumination

An image f(x,y) is formed as the product of a reflectance
component r(x,y) and an illumination component i(x,y).
f x, y   ix, y r x, y 



(10.3-4)
In ideal illumination, the reflective nature of objects and
background could be easily separable.
However, the image resulting from poor illumination could
be quit difficult to segment.
Taking the natural logarithm of Eq.(10.3-3)
z x, y   ln f x, y 
 ln ix, y   ln r x, y 
 i' x, y   r ' x, y 
(10.3-5)
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
48
The role of illumination

From probability theory,

If i’(x,y) and r’(x,y) are independent random
variables, the histogram of z(x,y) is given by the
convolution of the histograms of i’(x,y) and r’(x,y).
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
49
Pixel-based Direct Classification Methods
電腦產生的反射函數
f ( x, y)  i( x, y)r ( x, y)
影像f(x,y)可看作是反射
量r(x,y)和照度i(x,y)的乘
積。
電腦產生的照度函數
Fig. (a)*Fig(c)
物體和背景的反射特性，
使她們容易被分割，但
差的照明，會使產生的
影像難以分割。
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
50
Thresholding

Basic global thresholding






Select an initial estimate for T
Segment the image using T
 G1 : consisting of all pixels with gray level values > T
 G2 : consisting of all pixels with gray level values <= T
Compute the average gray level values m1 and m2 for pixels
in regions G1 and G2.
Compute a new threshold value
 T=0.5*(m1 + m2)
Repeat step 2 through 4 until the difference in T in
successive iterations is smaller than a predefined
parameter T0.
The parameter T0 is used to stop the algorithm after
changes become small.
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
51
Gray-Level Thesholding
1
g ( x, y)  
0
if f ( x, y)  T
if f ( x, y)  T
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
52
Basic Global Thresholding


To partition the image histogram by using a single global
threshold T.
Segmentation is then accomplished by scanning the image
pixel by pixel and labeling each pixel as object or
background, depending on whether the gray level of that
pixel is great or less than the value T.
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
53
Basic Global Thresholding

Fig. (a) is the original image, (b) is the image histogram.
 The clear valley of the histogram.
 Application of the iterative algorithm resulted in a value of 125.4
after three iterations starting with the average gray level and T0=0.
 The result obtained using T=125 to segment the original image is
shown in Fig. (c).
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
54
Thresholding

Basic adaptive thresholding



Imaging factors such as uneven illumination can transform a
perfectly segmentable histogram into a histogram that cannot be
partitioned effectively by a single global threshold.
To divide the original image into subimages and then utilize a
different threshold to segment each subimage.
The key issues are


How to subdivide the image?
How to estimate the threshold for each resulting subimages?
Global threshold手
動將T設在山谷處。
將原始影像根據亮度的變
化分成16區塊。
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
55
Thresholding
灰階值得分佈極不
均勻，全域門限法
注定失敗
圖10.30(c)的(1,2)及
(2,2)子影像
分成更多的子影像
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
56
Thresholding (cont.)

Optimal Global and Adaptive Thresholding






Estimating threshold that produce the minimum average
segmentation error.
Suppose that an image contains only two principal graylevel regions.
Let z denote gray-level values.
We can view these values as random quantities, and their
histogram may be considered an estimate of their
probability density function, p(z).
The overall density function id the sum or mixture of two
densities in the image.
If the form of the densities is known or assumed, it is
possible to determine an optimal threshold for segmenting
the image into two distinct regions.
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
57
Thresholding (cont.)

Optimal Global and Adaptive Thresholding

The mixture probability density function describing the overall
gray-level variation in the image is
p( z )  P1 p1 ( z )  P2 p2 ( z )

(10.3-5)
Here, P1 and P2 are the probabilities of occurrence of the two
classes of pixels.
Assuming that any given pixel belongs either to an object or to the
background,
P1  P2  1
(10.3-6)
background
object
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
E1(T)
E2(T)
58
Thresholding (cont.)



An image is segmented by classifying as background all pixels with
gray levels greater than a threshold T. All other pixels are called
object pixels.
The main objective is to select the value of T that minimizes the
average error in making the decisions that a given pixel belongs to
an object or to the background.
The probability of erroneously classifying a background point as an
object point is
E1 (T )  
T


p2 ( z )dz
The probability of erroneously classifying a object point as an
background point is

E2 (T )   p1 ( z )dz
T

將背景誤認為物體的機率。 (10.3-7)
將物體誤認為背景的機率。 (10.3-8)
The overall probability of error is
E(T )  P2 E1 (T )  P1E2 (T )
總誤差
(10.3-9)
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
59
Thresholding (cont.)


To find the threshold value for which this error is minimal requires
differentiating E(T) with respect to T and equating the result to 0.
The result is 為求得最小誤差的門限值T，須求E(T)對T的偏微分。
P1 p1 (T )  P2 p2 (T )
(10.3-10)
This equation is solved for T to find the optimum threshold. If
P1=P2, the optimum threshold is where the curves for p1(z) and p2(z)
intersect.
Obtaining an analytical expression for T requires that we know the
equations for the two PDFs.

Estimating these densities in practice is not always feasible. One of
the principal densities used is the Gaussian density, which is
completely characterized by two parameters, the mean and the
variance (以Gaussian密度函數來近似p(z))
pz  
P1
2 s 1

e
 z  m1 2
2s 12

P2
2 s 2

e
 z  m 2 2
2s 22
(10.3-11)
where m1 and s are the mean and variance of the Gaussian density of
one class of pixels.
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
60
Thresholding (cont.)

Using this equation in the general solution of Eq.(10.3-10) results
in the following solution for the threshold T: (10.3-10)式的通解)
AT 2  BT  C  0
where
A  s 12  s 22

B  2 m1s 22  m 2s 12

(10.3-12)
(10.3-13)
C  s 12 m 22  s 22 m12  2s 12s 22 lns 2 P1 / s 1 P2 


Since a quadratic equation has two possible solution, two
threshold values may be required to obtain the optimal solution.
If the variances are equal, a single threshold is sufficient:
T

m1  m 2
2
P 
s2

ln 2 
m1  m 2  P1 
(10.3-14)
If P1=P2, the optimal threshold is the average of the means
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
61
Thresholding (cont.)


Instead of assuming a functional form for p(z), a minimum
mean-square-error approach may be used to estimate a
composite gray-level PDF of an image from the image
histogram.
The mean square error between the (continuous) mixture
density p(z) and the (discrete) image histogram h(zi) is
ems

1 n
   pzi   hzi 2
n i 1
(10.3-15)
where an n-point histogram is assumed.
The principal reason for estimating the complete density is to
determine the presence or absence dominant modes in the
PDF.

Two dominant modes typically indicate the presence of edges in
the image (or region) over which the PDF is computed.
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
62
Thresholding

Two segmented MR brain images using a gray
value threshold T=166 (top) and T=225 (bottom)
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
63
Thresholding
原始的心臟影像，打有顯
影劑，目的在於描繪出左
心室的輪廓。
前處理:
(1)每個像素點經log function轉換
(2)將打藥前與打藥後的影像相減，以消
除脊椎部份。
(3)將多張心臟影像平均以消除雜訊。
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
64
Thresholding (cont.)






Each preprocessed image was subdivided into 49 regions by placing a 7x7
grid with 50% overlap over each image.
Each of the 49 resulting overlapped regions contained 64x64 pixels.
The histogram for region A is bimodal, indicating the presence of a boundary.
The histogram for region B is unimodal, indicating the absence of two
markedly distinct regions.
After all 49 histograms were computed, a test of bimodality was performed to
reject the unimodal histograms.
The remaining histograms were then fitted by bimodal Gaussian density
curves [see Eq. (10.3-11)] using a conjugate gradient hill-climbing method to
minimize the error function given in Eq(10.3-15).
Block A和Block B的
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
影像histogram
65
Thresholding
(cont.)
The x’s and o’s in Fig. 10.34 (a) are two fits to the histogram shown in





black dots.
The optimal thresholds were then obtained by using Eqs.(10.3-12)
and .(10.3-13)
For non-bimodal regions, the thresholds were obtained by interpolating
these thresholds.
Then a second interpolation was carried out point by point using
neighboring threshold values so that, at the end of the procedure, every
point in the image had been assigned a threshold.
Finally, a binary decision was carried out for each pixel using the rule
1 if f( x,y) Txy
f  x, y   
otherwise
0
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
66
Thresholding

Use of boundary characteristics for histogram
improvement and local thresholding


如果histogram的peak是高、窄、對稱，而且被深的山谷給分
隔，則選到好的threshold的機會較大。
改善histogram形狀的方法是只考慮接近物體與背景邊界的像
素點。
0 f  T

sx, y    f  T and  2 f  0
 f  T and  2 f  0

非邊界點標為0
邊界點的深色邊標為+
邊界點的亮色邊標為-
包含有物體的水平/垂直掃描線具有如下的結構：
(…)(-,+)(o or +)(+, -)(…)
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
67
Thresholding (cont.)





Figure 10.36 shows the labeling produced by Eq. (10.3-16) for an
image of a dark, underlined stroke written on a light background.
The information obtained with this procedure can be used to
generate a segmented, binary image in which 1’s correspond to
object of interest and 0’s correspond to the background.
The transition from a light background to a dark object must be
characterized by the occurrence of a – followed by a + in s(x,y).
The interior of the object is composed of pixels that are labeled
either 0 or +.
A horizontal or vertical scan line containing a section of an object
has the following structure:
(…)(-,+)(o or +)(+, -)(…)
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
68
Thresholding
具風景背景的支票
Example 10.14
梯度大於5的梯度
histogram，具有相同高
度，及由顯著山谷所分
開的特性
取梯度histogram的山谷中間值，
當成threshold後的結果。
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
69
Thresholding

Thresholds based on several variables

Multispectral thresholding


相當於3D空間中找尋分類點。
例如：在RGB影像中分別根據RGB來分類。
以接近臉的灰
階來分割
將color影像
以單色顯示。
分割紅色的成分
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
70
Pixel Classification Through Clustering


Image can be segmented by pixel classification
through clustering of all features of interest.
As the image is classified into cluster classes,
segmented regions are obtained by checking the
neighborhood pixels for the same class label.


Clustering may produce disjoint regions with holes or
regions with a single pixel.
A post-processing algorithm such as region growing, pixel
connectivity or rule-based algorithm is usually applied to
obtain the final segmented regions.
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
71
Pixel Classification Through Clustering




Clustering is the process of grouping data points with similar
feature vectors together in a single cluster while data points with
dissimilar feature vectors are placed in difference clusters.
The similarity of feature vectors can be represented by an
appropriated distance measure such as Euclidean or
Mahalanobis distance.
Each cluster is represented by its mean (centeroid) and variance
(spread) associated with the distribution of the corresponding
feature vectors of the data points in the cluster.
The formation of clusters is optimized with respect to an objective
function involving pre-specified distance and similarity measures
along with additional constraints such as smoothness.
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
72
Pixel Classification Through Clustering

K-means Clustering


To partition d-dimensional data into k clusters
Initially places k clusters at arbitrarily selected
cluster centroids vi; i=1,2,…,k and modifies
centroids for the formation of new cluster shapes
optimizing the objective function.
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
73
Pixel Classification Through Clustering

The k-means clustering algorithm


Step 1: Select the number of clusters k with initial cluster
centroids vi; i=1,2,…,k
Step 2: partition the input data points into k clusters by
assigning each data point xj to the closest cluster centroid vi
using the selected distance measure,(Euclidean distance)
dij  x j  vi

Step 3: Compute a cluster assignment matrix U representing
the partition of the data points with the binary membership
value of the jth data point to the ith cluster such that
uij  0,1 for all i, j
k
 uij  1
i 1
n
for all j and 0  uij  n for all i
j 1
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
74
Pixel Classification Through Clustering

Step 4: Re-compute the centroids using the membership
values as
n
 uij x j
vi 
j 1
n
 uij
for all i
j 1


Step 5:If cluster centroids or the assignment matrix does not
change from the previous iteration, stop; otherwise go to
Step 2.
The k-means clustering method optimizes the sum-ofsquared-error based objective function
k
n
J w U , v    x j  vi
2
i 1 j 1
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
75
Pixel Classification Through Clustering

Fuzzy c-Means Clustering


The k-mean clustering method utilizes the hard binary
values for the membership of a data point to the cluster.
The fuzzy c-means clustering method utilizes an
adaptable membership value that can be updated
based using the distribution statistics of the data points
assigned to the cluster minimizing the following
objective function
c
n
c
n
J m U , v    uijm d ij2   uijm x j  vi

2
i 1 j 1
1 j 1
Fuzziness index
can be idefined
between 1 and very
large value for the highest level of fuzziness
(maximum allowable variance within a cluster)
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
76
Pixel Classification Through Clustering

The membership values in the fuzzy c-means
algorithm can be defined as
0  uij  1 for all i, j
c
 uij  1
i 1
n
for all j and 0  uij  n for all i
j 1
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
77
Pixel Classification Through Clustering

Adaptive FCM Algorithm


Using a weighted derivative information.
The objective function is minimized for clustering, the
minimization of the derivative information provides smooth
homogenous clusters.
c
n
J ACFM U , v    
i 1 j 1
2
uijm
x j  g j vi
n
R
n
R
R
 1  Dr  g   2  Dr  Ds  g 2j
j 1 r 1
2
j
j 1 r 1 s 1
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
78
Pixel Classification Through Clustering

The adaptive fuzzy c-means algorithm


Step 1: Start with initial centroids vi; i=1,2,…,c for c clusters
and set the initial gain field gj=1 for all j belonging to the
image data.
Step 2: Compute membership values as
uij 
x j  g j vi
c

i 1

2 / m 1
x j  g j vi
 2 / m 1
Step 3:Compute new centroids as
n
 uijm g j x j
vi 
j 1
n
 uijm g 2j
, for all i  1,2,..., c
j 1
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
79
Pixel Classification Through Clustering

Step 4: Compute a new gain field by solving the following
equation for gj
 xi , vi   g j  uijm vi , v j   1 H1  g  j  2 H 2  g  j
c
i 1
c
uijm
i 1
with
R

H1   Dr  Dr
r 1
R


j

H 2   Dr  Dr   Ds  Ds
r 1


j
Step 5: If there is no difference in the centroids from the
previous iteration, or there is not a significant difference in
the membership values, the algorithm has converged, stop;
otherwise go to Step 2.
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
80
Region-Based Segmentation



Region-growing based segmentation algorithms
examine pixels in the neighborhood based on a predefined similarity criterion.
The neighborhood pixels with similar properties are
merged to form closed regions for segmentation.
The difference between the region and thresholdingbased segmentation


Region-growing methods guarantee the segmented regions
of connected pixels.
Threasholding-based segmentation methods may yield
region with holes and disconnected pixels.
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
81
Region-Based Segmentation

Basic Formulation



Let R represent the entire image region.
We may view segmentation as a process that partitions R into n
subregions, R1, R2,…,Rn
The following conditions are met:
n
(a )
R
i
i 1
R
所有的像素點必須屬於任一區域
(b) Ri is a connectedregion,i  1, 2,..., n
(c) Ri  R j   for all i and j , i  j
(d) P( Ri )  TRUE for i  1,2,...,n
(e) PRi  R j   FALSE for i  j
區域內的點必須相連
區域和區域間沒有相連
區域內的像素點有相同的性質。
區域 Ri和Rj有不同的特性
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
82
Region-Based Segmentation

Region Growing



Group pixels or subregions into regions based on predefined
criteria.
Start with a set of “seed” points and from these grow regions
by appending to each seed those neighboring pixels that have
properties similar to the seed.
Problems about region growing:
 Selecting a set of seed points
 Selecting of similarity criteria depends not only on the
problem under consideration, but also on the type of image
data available.
 The formulation of a stooping rule.

Growing a region should stop when no more pixels satisfy the criteria
for inclusion in that region.
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
83
Region-Based Segmentation
Example 10-16 焊接點的檢測
有缺陷的焊接點
的灰階值有傾向
255的趨勢
選擇灰階值為
255的點當seed
point
將和種子點灰階
值差異小於65的
點”長”出來。
Criteria for region growing:
(1) The absolute gray-level difference between any pixel and the seed had to be less than 65.
醫學影像處理實驗室(Medical
Image
Lab.)
Changin Ph.D.
(2)To be included
in one of the regions, the pixel had
to beProcessing
8-connected
to atChuan-Yu
least one pixel
that
region
84
Region-Based Segmentation

Problems having multimodal histograms generally are best
solved using region-based approach.

Histogram of Fig.10.40 .(無法清楚分離物體與背景)
 This problem cannot be solved effectively by any reasonably
general method of automatic threshold selection based on
gray levels alone.
 The use of connectivity was fundamental in solving the
problem.
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
85
Region
Growing
Center Pixel
Pixels
satisfying the
Pixels not satisfying
similarity
the similarity
criterion
criterion
3x3 neighborhood
5x5 neighborhood
7x7 neighborhood
Segmented region
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
86
Region-Based Segmentation

A T-2 weighted MR brain image (top) and the segmented
ventricles (bottom) using the region-growing method.
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
87
Region-Based Segmentation

Region-Splitting



Region-splitting methods examine the
heterogeneity of a predefined property of the
entire region in terms of its distribution and the
mean, variance, minimum and maximum values.
If the region is evaluated as heterogeneous, the
original region is split into two or more regions.
The region-splitting process continues until all
regions satisfy the homogeneity criterion
individually.
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
88
Region-Based Segmentation


In the region-splitting process, the original region R is split
into R1, R2,…,Rn subregions such that the following
conditions are met.
1. Each region, Ri; i=1,2,…,n is connected.
n
2.  Ri  R
i 1
3. Ri  R j  0 for all i, j; i  j
4. PRi   TRUE for i  1,2,..., n


5. P Ri  R j  FALSEfor i  j
where P(Ri) is a logical predicate for the homogeneity
criterion on the region Ri.
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
89
Region-Based Segmentation

Quad-tree based Region Splitting and Merging


一開始將影像分割成任意子影像，然後再合併或分割
成滿足條件的segmentation。
將整張影像R連續的分割成更小的1/4影像，直到任何
區域Ri, P(Ri)=TRUE


從整張圖R開始，如果P(R)=FALSE (表示區域 R內的pixel有
不同的灰階值) ，則將R分成四個子影像。
如果子影像的P為FALSE則繼續分割成四個子影像。
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
90
Region-Based Segmentation
Region Splitting and Merging
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
91
Region-Based Segmentation

Region Splitting and Merging



Split into four disjoint quadrants any region Ri for
which P(Ri)=FALSE.
Merge any adjacent regions Rj and Rk for which
P(Rj U Rk)=TRUE.
Stop when no further merging or splitting is
possible.
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
92
Region-Based Segmentation
若Ri中有80%的像素點具有
|zj-mi|<=2si的特性。則P(Ri)=TRUE。
原始楓葉影像
Threshold後，葉柄不見了。
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
93
Estimation-Model Based Adaptive Segmentation



The multi-level adaptive segmentation (MAS)
method was used to segment and classify
multiparameter MR brain images into a large
number of classes of physiological and pathological
interest.
The MAS method is based on estimation of
signatures for each segmentation class for pixel-bypixel classification.
It can create new classes, if necessary, when there
is a significant discrepancy between the measured
parameters of the image and the model signatures.
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
94
Estimation-Model Based Adaptive Segmentation
Define Classes
Determination of Model Parameters
(From a set of
manually segmented and labeled images)
Classification of Image
Pixels Using Model
Signatures
Parameter
Relaxation
No
No
All pixels
classified?
Formation of
a New Class
Necessary?
Yes
Yes
Identification of Tissue
Classes and Pathology
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
95
Estimation-Model Based Adaptive Segmentation



Let’s present a multi-parameter image to be segmented with a
pair of random variables
{Cmn , Xmn}
where Cmn is the class of the pixel mn, Xmn is a d-dimensional
random variable of pixel mn.
To develop an estimation model, the required parameters can be
obtained using a priori knowledge or manual segmentation on a
set of training image.
The i-th class mean vector is
uˆ i 


1
xj
n j
The dimension of xj corresponds to the number of image
modalities used in the analysis.
The covariance matrix of class i is
1
ˆ i 
( x j  uˆ i )( x j  uˆ i ) t

n 1 j
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
96
Estimation-Model Based Adaptive Segmentation

Given that Cmn= i, the distribution of Xmn is
estimated to obey the general multivariate normal
distribution described by the density function
pˆ i ( x) 

  ( x  uˆi ) 
exp


1/ 2
d /2 ˆ
ˆ
2

(
x

u
)
i 
 i
(2 )  i
1
Four transition matrices can be estimated
pij1 (m, n)  PC mn  j | C m,n 1  i
pij 2 (m, n)  PC mn  j | C m1,n  i
pij3 (m, n)  PC mn  j | C m,n 1  i
pij 4 (m, n)  PC mn  j | C m1,n  i
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
97
Estimation-Model Based Adaptive Segmentation

Transition probabilities can be given by
pij1 (m, n) 
 pix | C  j, C  i
 pix | C  i
m ,n 1
b


where b{pix|P} denotes the number of pixels with the property P.
Because of the relatively small number of brain images in the
training set, averaging over a small neighborhood of pixels
around the pixel of interest mn is performed
pij1
pix | C  j, C  i


(m, n) 
  pix | C  i
b

m,n1
mn
n
b

m ,n 1
mn
b
m,n1
n
where n{pix|P} indicates the averaging over the neighborhood.
The equilibrium transition probabilities are estimated as
pix | C  i


 (m n) 
  pix
b
mn
n
i
b
n
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
98
Estimation-Model Based Adaptive Segmentation



To perform maximum likelihood discriminant analysis, the
class random variable Cmn is assumed to constitute a Kstate Markov random field.
Rows and Columns of Cmn constitute segments of K-state
Markov chains.
Chains are specified by the KxK transition matrix
pij  PCmn  j | Cm,n1  i

The probability of a pixel belonging to a specific class i is
PCmn  i | xkl , (k , l )  N (m, n)
where N(m,n) is some neighborhood of the pixel (m,n).
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
99
Estimation-Model Based Adaptive Segmentation

The conditional probabilities can be expressed as

PC  i, X | X
,X
PC  i | x , (k , l )  N (m, n) 

PX | X
,X
and
mn
mn
mn
m1,n
m,n1
kl
mn
m1,n
m,n1
PCmn  i | xkl , (k , l )  N (m, n)

where
PX mn | Cmn  i, X m1,n , X m,n1 PCmn  i | X m1,n , X m,n1 
PX mn | X m1,n , X m,n1 
P | X m1,n , X m,n1 P | X m1,n P | X m,n1P | X m1,n P | X m,n1
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
100
Estimation-Model Based Adaptive Segmentation

Using class condition independence of xmn and applying Bayesian
estimate, Eq(7.38) can be rewritten as
PCmn  i | xkl , (k , l )  N (m, n)

PX mn | Cmn  iPX m1,n , X m,n1 | Cmn  iPCmn  i
PX mn | X m1,n , X m,n1 PX m1,n , X m,n1 
with
PX m1,n | Cmn i  i   PX m1,n | Cmn  jPCm1,n  j | Cmn  i  H m1,n (i)
N
j 1

With substitutions, the probability of the current pixel mn,
belonging to class i given the characteristics of the pixels in the
neighborhood of mn
PCmn  i | xkl , (k , l )  N (m, n)

PCmn  i | X mn PX mn H m1,n (i) H m1,n (i) H m,n1 (i)
PX mn | X m1,n , X m,n1 PX m1,n , X m,n1 
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
101
Estimation-Model Based Adaptive Segmentation






The brain model consists of class probabilities, transition
probabilities, mean vectors, and covariance matrices of class
clusters in d dimensions space.
After the model is developed, an image to be segmented is
automatically classified on a pixel-by-pixel basis using the model
signatures.
The multiparameter measurement vector of each pixel in the image
to be classified as a particular class if its distance from the model
signature is less than some predefined threshold.
The thresholds are selected to classify pixels with relatively pure
content of the corresponding class.
The next level of processing deals with the data on the
neighborhood level.
For the remaining unclassified pixels, a spatial neighborhood is
identified.
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
102
Estimation-Model Based Adaptive Segmentation

The spatial distribution of the neighboring pixels that have been
classified in the first step is now taken into consideration with the
class and transitional probabilities.



This processing step allows relaxation of the model signature
parameters to help classification of those pixels which are in the
neighborhood of already classified pixels.
After the relaxation step, there may still be some unclassified
pixels that belong to the edges of tissue regions.
For boundary pixels, it is necessary to perform unmixing.


Unmixing is the process of determining the set of classes, and the
contribution of each, that compose boundary or mixed pixels.
The unmixing process uses the labels of the already classified
neighborhood pixels to determine the candidate set of classes that
compose the unclassified pixel.
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
103
Estimation-Model Based Adaptive Segmentation
(a) Proton Density MR and
(b) perfusion image of a patient
48 hours after stroke.
Segmented by a model of 15 tissue classes
Results of MAS method with 4x4 pixel
probability cell size and 4 pixel wide averaging.
(a) pixel classification as obtained on the
(b) basis of maximum probability,
(c) (b) as obtained with p>0.9.
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
104
Image Segmentation Using Neural Networks





Neural networks do not require underlying class probability
distribution for accurate classification.
The decision boundaries for pixel classification are adapted
through an iterative training process.
Neural networks learns from examples in the training set in which
the pixel classification task has already been performed using
manual methods.
A non-linear mapping function between the input features and the
desired output for labeled examples is learned by neural
networks without using any parameterization.
After the learning process, a pixel in a new image can be
classified for segmentation by the neural network
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
105
Image Segmentation Using Neural Networks



It is important to select a useful set of features to be
provided to the neural network as input data for
classification.
The selection of training examples is also very
important, as they should represent a reasonably
complete statistical distribution of the input data.
The structure of the network and the distribution of
training examples play a major role in determining
its performance for accuracy, generalization and
robustness.
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
106
Backpropagation Neural Network for Classification




The neural element may receive its input from an input
vector or other neural elements.
A weighted sum of these inputs constitutes the argument
of a non-linear activation function such as a sigmoidal
function.
The resulting thresholded value of the activation function
is the output of the neural elements.
The output is distributed along weighted connections to
other neural elements.
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
107
Neural Network Element
x1
w1
x2
w2

 n

y  F   wi xi  wn 1 
 i 1

wn
wn+1
xn
1
Non-Linear Activation
Function F
 n

y  F   wi xi  wn1 
 i 1

醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
108
Artificial Neural Network: Backpropagation
y 2L
y1L
y nL
Output Layer
Neurons
Hidden Layer
Neurons
x1
x2
x3
xn
1
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
109
Backpropagation Neural Network for Classification

To learn a specific pattern of input vectors for
classification, an iterative learning algorithm is used with
a set of pre-classified training examples that are labeled
with the input vectors and their respective class outputs.



LMS algorithm, (Widrow-Hoff Delta Rule)
The learning algorithm repeatedly presents input vectors
of the training set to the network and forces the network
output to produce the respective classification output.
Once the network converges on all training examples to
produce the respective desired classification outputs, the
network is used to classify new input vectors into the
learned classes.
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
110
Backpropagation Neural Network for Classification

The Least Mean Squared (LMS) error algorithm can be
implemented to train a feedforward neural network using
the following steps:


Step 1: Assign random weights in the range of [-1, +1] to all
weights.
Step 2: For each classified pattern pair {y(0), yL} in the
training set, do the following steps:




Compute the output values of each neural element using the
current weight matrix.
Find the error e(k) between the computed output vector and the
desired output vector for the classified pattern pair.
Adjust the weight matrix using the change Δw(k)= e(k) [y(k-1)]
Step 3: Repeat Step 2 for all classified pattern pairs in the
training set until the error vector for each training examples
is sufficiently low or zero.
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
111
Backpropagation Neural Network for Classification

Two problems with the BPNN


It is sensitive to the selection of initial weights and noise in
the training set that can cause the algorithm to get stuck in
local minima in the solution pace.
To find optimal network architecture with the consideration
of an optimal number of hidden layers and neural elements
in each of the hidden layers.

Fahlman proposed a cascade-correlation neural network
architecture to find the best architecture by correlating the
error patterns with the inclusion or deletion of a neural element
in the hidden layers based on the learning vectors in the
training set.
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
112
Radial Basis Function (RBF) Neural Network for
Classification







The RBF is based on the principle of regularization theory for
function approximation.
It does not suffer from problems like the BPNN.
The RBF provides more reliable and reproducible results.
The basic RBF network contains an input layer for input signal
distribution, a hidden layer to provide the radial basis function
processing, and an output layer.
The RBF representation can be obtained by clustering the input
training data to obtain the centroid vector, ci for all clusters.
A radial basis function is applied to the Euclidean distance
between the input vector xj and its own centroid.
The output of each radial basis function unit is then weighted and
summed through a linear combiner to provide the final output of
the network.
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
113
Radial Basis Function (RBF) Neural Network for
Classification

The final output of the network f(x) can be given as
K
f  x    wi   x, i 
i 1
 x  cj 

exp 

2s j 


with   x, i  
K
 x  cj 

exp 


2s j 
j 1



The matrix of activation function
   x1 ,1 ...   x1 , K  


F   ...
...
... 
   x ,1 ...   x , K 
N
N


醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
114
Radial Basis Function (RBF) Neural Network for
Classification

The weight

w  F  F   I



T

1
 FT  y
The major issues for implementation of the RBF network are
the location of the centroids and the structure of the radial
basis function.
The locations of the radial basis functions depend upon the
data being presented to the network.
The optimal number of the clusters can be obtained from the
statistical evaluation of the population density and variance
parameters of initial clusters through an adaptive K-means or
fuzzy clustering method.
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
115
RBF Network
Output
Linear
Combiner
RBF Unit
1
RBF Unit
2
Sliding
Image
Window
RBF Unit
n
RBF
Layer
Input
Image
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
116
Segmentation of Arterial Structure in
Digital Subtraction Angiograms


Specific features with information about gray
values, edges and local contrast of the pixels
are needed for effective segmentation using
a neural network based classifier.
The feature vector consists of the gray values
of the center pixel and its neighbors,
combined with the contrast value at the
center pixel.
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
117
RBF NN Based Segmentation
醫學影像處理實驗室(Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
118