電腦視覺
Computer and Robot Vision I
Chapter2: Binary Machine Vision:
Thresholding and Segmentation
Instructor: Shih-Shinh Huang
1
Contents
 Introduction
 Thresholding
 Connected Components Labeling
 Signature Segmentation and Analysis
2
Computer and Robot Vision I
Introduction
2.1 Introduction
3
2.1 Introduction
Binary Machine Vision
 Binary Image
 Binary Value 1: Part of Object
 Binary Value 0: Background Pixel
 Definition of Binary Machine Vision
 Generation and analysis of such a binary image
4
2.1 Introduction
Binary Machine Vision
 Thresholding
 It is the first step of binary machine vision
 It is a labeling operation
 Connected Components / Signature Analysis
 They are multilevel vision grouping techniques.
 They make a transformation from image pixels to
more complex units.
• Regions
• Segments
5
Computer and Robot Vision I
Introduction
2.2 Thresholding
6
2.2 Thresholding
Introduction
 What is Thresholding ?
 It is a labeling operation.
 It assigns a binary value to each pixel.
• Binary Value 1: pixels have higher intensity values
• Binary Value 0: pixels have higher intensity values
T 128
  
7
2.2 Thresholding
Introduction
 Mathematical Formulation
1
B (r , c)  
0
if
I (r , c)  T
if
I (r , c)  T
 ( r , c ) : row and column
 I (.,.) : gray-level intensity image
 T
: intensity threshold
 B (.,.) : binary intensity image
8
2.2 Thresholding
Introduction
T 5
I (.,.)
B (.,.)
How to select an appropriate threshold ?
9
2.2 Thresholding
Introduction
 Approaches
 Global Thresholding: use a global value to make the pixel
distinction in the image.
 Local Thresholding: use spatial varying threshold to label
the local pixels.
T1
T2
T3
T4
T
Image
Image
10
2.2 Thresholding
Histogram
 Definition of Histogram h (.)
h ( m )  # {( r , c ) | I ( r , c )  m }
number of elements
m spans each gray level value
e.g. 0 - 255
 Histogram Probability P ( I ) : I  0 ,1,..., 255
P(I ) 
h(I )
RC
11
2.2 Thresholding
Histogram
 Examples
h(m )
h(m )
12
2.2 Thresholding
Histogram
h (m )
13
2.2 Thresholding
Histogram
T=110
T=130
14
T=150
T=170
2.2 Thresholding
Within-Group Variance
 Observations
 A group is a set of pixels with intensity homogeneity.
 Homogeneity is measured by the use of variance
• High homogeneity group has low variance
• Low homogeneity group has high variance
 Objective
 Select a dividing score such that the weighted sum
of the within-group variances is minimized.
15
2.2 Thresholding
Within-Group Variance
 Definition: weighted sum of group variances
 W ( t )  q1 ( t ) 1 ( t )  q 2 ( t ) 2 ( t )
2
2
2
 q1 ( t ) : probability for the group with values  t
 q 2 ( t ) : probability for the group with values  t
(t )
: variance for the group with values  t
 2 (t )
: variance for the group with values  t


2
1
2
16
2.2 Thresholding
Within-Group Variance
 Objective Formulation
 Find a threshold t
*
which minimizes  W ( t )
2
t  arg min  W ( t )
*
2
t
q1 ( t ) 

P (i )
 1 (t ) 
i  t 1
 (t ) 
P (i )
 2 (t ) 

 [ i   1 ( t )] P ( i ) / q1 ( t )
2
i 1
I
I
I


iP ( i ) / q 1 ( t )
2
1
i 1
i 1
q 2 (t ) 
t
t
iP ( i ) / q 2 ( t )
 (t ) 
2
2
 [i  
it i
i  t 1
17
2
( t )] P ( i ) / q 2 ( t )
2
2.2 Thresholding
Within-Group Variance
 Implementation Issue
 Step1: For t=0,…,255
2
2
 Step2: Compute q1 ( t ) ,q 2 ( t ) , 1 ( t ) , and  2 ( t )
 Step3: Compute  W ( t )
2
 W ( t )  q1 ( t ) 1 ( t )  q 2 ( t ) 2 ( t )
2
2
2
2
 Step4: If  W ( t ) is less than the value in the
previous iteration t *  t
All variables should be re-compute at each iteration.
18
2.2 Thresholding
Within-Group Variance
 Implementation Issue
 Speed-Up Formulation
T
I

2

 (i   )
2
P (i )
i 1
 
 iP ( i )
i 1
t

2


T
(i   1 (t )   1 (t )   ) P (i ) 
2
i 1
 ( i  

(i   2 (t )   2 (t )   ) P (i )
2
i  t 1
t

2


( t ))  2 ( i   1 ( t ))(  1 ( t )   )  (  1 ( t )   ) P ( i )  .......
1
2
2
i 1
19
2.2 Thresholding
Within-Group Variance
t
 Implementation Issue
 (i  
( t ))(  1 ( t )   ) P ( i )  0
i 1
 Speed-Up Formulation
 ( i  
1
t


2
2
2
i 1
 ( i  
t

2


( t ))  (  1 ( t )   ) P ( i )
1
2
2
i 1
 ( i  
t


( t ))  2 ( i   1 ( t ))(  1 ( t )   )  (  1 ( t )   ) P ( i )  .......
1

( t ))  (  2 ( t )   ) P ( i )
2
2
2
i 1
20
2.2 Thresholding
Within-Group Variance
 Implementation Issue
 Speed-Up Formulation
t

2


( i   1 ( t )) P ( i )  [ 1 ( t )   ] q1 ( t )
2
2
i 1
I


( i   2 ( t )) P ( i )  [ 2 ( t )   ] q 2 ( t )
2
2
i  t 1

2

 q1 ( t ) 1 ( t )  q 2 ( t ) 2 ( t )

2
2

 q1 ( t )[  1 ( t )   ]  q 2 ( t )[  2 ( t )   ]
2
21
2

2.2 Thresholding
Within-Group Variance
 Implementation Issue
 Speed-Up Formulation


 q1 ( t ) 1 ( t )  q 2 ( t ) 2 ( t )
2
2

2

  q1 ( t )  1 ( t )  q 2 ( t )  2 ( t )
 q1 ( t )[  1 ( t )   ]  q 2 ( t )[  2 ( t )   ]
2

2


2
  w  q1 ( t )[  1 ( t )   ]  q 2 ( t )[  1 ( t )   ]

2
  w  q1 ( t )[ 1  q1 ( t )][  1 ( t )   2 ( t )]
2
2
2
22
2

2.2 Thresholding
Within-Group Variance
 Implementation Issue
 Speed-Up Formulation

2
  w  q1 ( t )[ 1  q1 ( t )][  1 ( t )   2 ( t )]
constant
2
minimize
maximize
t  arg max q1 ( t )[1  q1 ( t )][  1 ( t )   2 ( t )]
*
23
2.2 Thresholding
Within-Group Variance
 Implementation Issue
 Speed-Up Formulation
t  arg max q1 ( t )[1  q1 ( t )][  1 ( t )   2 ( t )]
*
• We have recursive form to compute optimal threshold.
q 1 ( t  1)  q 1 ( t )  P ( t  1)
 1 ( t  1) 
q 1 ( t )  1 ( t )  ( t  1) P ( t  1)
q1 ( t  1)
24
2.2 Thresholding
Within-Group Variance
 Example
25
2.2 Thresholding
Kullback Information Distance
 Assumption
 The observations come from a weighted mixture of
two Gaussians distributions.
• Gaussian Distribution of Background ( u 1 ,  1 )
2
• Gaussian Distribution of Object
f (i ) 
q1
1
2

e
1 i  1 2
(
)
2 1

26
(u 2 ,  )
2
2
q2
2
2

e
1 i2 2
(
)
2 2
2.2 Thresholding
Kullback Information Distance
( u1 ,  1 )
2
(u 2 ,  2 )
2
Background
Gaussian Distribution
Object
Gaussian Distribution
27
2.2 Thresholding
Kullback Information Distance
 Objective Formulation
 Determine a threshold T that results in two Gaussian
distributions which minimize Kullback divergence
I
J 

P ( i ) log[
P (i )
]
f (i )
i 1
• P(I) : observed histogram distribution
• f(I) : a mixture of Gaussian distributions determined by T
f (i ) 
q1
 1 2
e
1 i  1 2
 (
)
2 1
28

q2
 2 2
e
1 i2 2
 (
)
2 2
2.2 Thresholding
Kullback Information Distance
 Objective Formulation
 Known Parameter: Observed Histogram P (.)
P (1), P ( 2 ),... P ( I )
 Unknown Parameter: Two Gaussian Distributions
( q 1 ,  1 ,  1 ), ( q 2 ,  2 , 
f (i ) 
q1
 1 2
e
1 i  1 2
 (
)
2 1

2
)
q2
 2 2
29
e
1 i2 2
 (
)
2 2
2.2 Thresholding
Kullback Information Distance
 Solution Derivation
I
J 

P ( i ) log[


]
f (i )
i 1
I
P (i )
P ( i )  log P ( i )  log f ( i ) 
i 1
I


I
P ( i ) log P ( i ) 
i 1

i 1
Constant
30
P ( i ) log f ( i )
2.2 Thresholding
Kullback Information Distance
 Solution Derivation
I
arg min
 P ( i ) log[
i 1
P (i )
I
]  arg min 
f (i )
 P ( i ) log
i 1
 Assumption: The modes are well separated.
1 i  1 2
 (
)

q1
2
1
e

  1 2
f (i )  
1 i 2 2
 (
)
q2
2
2

e

2
 2
31
i  t
i  t
f (i )
2.2 Thresholding
Kullback Information Distance
 Solution Derivation
I
H ( t )    P ( i ) log f ( i )
i 1
t
H (t )    P (i )
i 1
H (t ) 
q1
 1 2
1  log 2 
2

1
2

e
1 i  1 2
(
)
2 1
I

 P (i ) 
i  t 1
 q1 log q1  q 2 log q 2
 q1 log  1  q 2 log  2 
32
q2
2
2

e
1 i2 2
(
)
2 2
2.2 Thresholding
Kullback Information Distance
 Implementation Issue
 Step1: For t=0,…,255
2
2
 Step2: Compute q1 ( t ) ,q 2 ( t ) , 1 ( t ) , and  2 ( t )
 Step3: Compute H (t )
I
H ( t )    P ( i ) log f ( i )
i 1
 Step4: If H (t ) is less than the value in the
previous iteration t *  t
33
2.2 Thresholding
Kullback Information Distance
 Example
34
2.2 Thresholding
Kullback Information Distance
Within Group Variance
(Otsu)
Kullback Information
(Kittler-Illingworth)
35
Computer and Robot Vision I
Introduction
2.3 Connected Component
Labeling
36
2.3 Connected Component Labeling
Introduction
 Description
 Connected Components labeling is a grouping
operation.
 It performs the unit change from pixel to region or
segment.
 All pixels are given the same identifier
• Have value binary 1
• Connect to each other
37
2.3 Connected Component Labeling
Introduction
 Terminology
 label: unique name or index of the region
 connected components labeling: a grouping
operation
 pixel property: position, gray level or brightness
level
 region property: shape, bounding box, position,
intensity statistics
38
2.3 Connected Component Labeling
Connected Component Operators
 Definition of Connected Component
 Two pixels p and q belong to the same
connected component C if there is a sequence of
1-pixels ( p 0 , p1 ,... p n ) , where
•
p0  p
•
pn  q
• p i 1 , p i : i  1,..., n are neighbor
39
2.3 Connected Component Labeling
Connected Component Operators
 Neighborhood Types
4-connected
Original Image
8-connected
40
Connected
Components
2.3 Connected Component Labeling
Connected Component Algorithms
 Common Features
 Process a row of image at a time
 Assign a new labels to the first pixel of each
component.
 Propagate the label of a pixel to its neighbors to
the right or below it.
41
2.3 Connected Component Labeling
Connected Component Algorithms
 Common Features
• What label should be assigned to A
• How does the algorithm keep track of the equivalence
of two labels
• How does the algorithm use the equivalence information
to complete the processing
42
2.3 Connected Component Labeling
Algorithm1: Iterative Algorithm
 Algorithm Steps
 Step1 (Initialization): Assign an unique label to
each pixel.
 Step2 (Iteration) : Perform a sequence of topdown and bottom-up label propagation.
• Use no auxiliary storage
• Computational Expensive
43
2.3 Connected Component Labeling
Algorithm2: Classic Algorithm
 Two-Pass Algorithm
 Pass 1:
• Perform label assignment and label propagation
• Construct the equivalence relations between labels when
two different labels propagate to the same pixel.
• Apply resolve function to find the transitive closure of all
equivalence relations.
 Pass 2:
• Perform label translation.
44
2.3 Connected Component Labeling
Algorithm2: Classic Algorithm
 Example:
{2=4}
{3=5}
{1=5}
45
2.3 Connected Component Labeling
Algorithm2: Classic Algorithm
 Example:
 Resolve Function
{2=4} {3=5}
{1=5}
{2=4} {1=3=5}
•Computational Efficiency
•Need a lot of space to store equivalence
46
Computer and Robot Vision I
Introduction
2.4 Signature Segmentation
and Analysis
47
2.4 Signature Segmentation and Analysis
Introduction
 Description
 Signature analysis perform unit
change from the pixel to the segment.
 It was firstly used in character
recognition
 Definition of Signature
 The signature, which is a projection,
is the histogram of the non-zero pixels
of the masked image.
48
2.4 Signature Segmentation and Analysis
Introduction
 General Signatures
 Vertical Projection
 Horizontal Projection
 Diagonal Projection
h ( r )  # {( x , y ) | ( x , y )  1, x  r }
v ( r )  # {( x , y ) | ( x , y )  1, y  c}
49
2.4 Signature Segmentation and Analysis
Signature Segmentation
 Steps
 Thresholding: generate the binary image.
 Projection Computation: compute the vertical,
horizontal, or diagonal projections.
 Projection Segmentation: divide the image into
several segments or regions according to the
signatures.
50
2.4 Signature Segmentation and Analysis
Signature Segmentation
51
2.4 Signature Segmentation and Analysis
Signature Segmentation
52
2.4 Signature Segmentation and Analysis
Signature Segmentation
OCR: Optical Character Recognition
MICR: Magnetic Ink Character Recognition
53
Computer and Robot Vision I
The End