Segmentation and Perceptual Grouping




The problem
Gestalt
Edge extraction: grouping and completion
Image segmentation
Camouflage
Kanizsa Triangle
The image of this cube contradicts the optical image
Perceptual Organization

Atomism, reductionism:



Perception is a process of decomposing an
image into its parts.
The whole is equal to the sum of its parts.
Gestalt (Wertheimer, Köhler, Koffka 1912)

The whole is larger than the sum of its parts.
Mona Lisa
Mona Lisa
Gestalt Principles

Proximity
Gestalt Principles


Proximity
Similarity
Gestalt Principles



Proximity
Similarity
Continuity
Gestalt Principles



Proximity
Similarity
Continuity

Closure
Gestalt Principles



Proximity
Similarity
Continuity


Closure
Common Fate
Gestalt Principles



Proximity
Similarity
Continuity



Closure
Common Fate
Simplicity
Smooth Completion




Isotropic
Smoothness
Minimal curvature
Extensibility
Elastica
E ()  min  k ( s)ds
2


Elastica is not scale invariant
    l   l , k 
E () 
1

E ( )
k

Elastica

Scale invariant measure
Ei ()  min l  k (s)ds
2


Approximation
Ei ()  4(    1 2 )
2
1
1
2
2
2
Finding lines from points
Parametric methods: RANSAC
RANSAC


RANdom SAmple Concensus
Complexity:



Need to go over all pairs: O(n2)
For each pair check how many more points are
consistent: O(n)
Total complexity: O(n3 )
RANSAC


Another application of RANSAC:
Find transformation between images
Example: compute homography




Compute homography for every 4 pairs of
corresponding points
Choose the homography that best explains the
image
m4n4 sets should be tested
Another example: compute epipolar lines

How many correspondences are needed?
Hough Transform
Hough Transform


Linear in the number of points
Describe lines as
y  mx  n

Or better

Prepare a 2D table
x cos   y sin   c
c
θ
Hough Transform
c
+1
+1
+1
+1
+1
θ
Hough Transform
c
13
16
θ
What if we want to find circles?
Curve Salience
Saliency Network
Encourage
 Length
 Low curvature
 Closure
Saliency Network
Tensor Voting




Every edge element votes to all its circular edge
completions
Vote attenuates with distance: e-αd
Vote attenuates with curvature: e-βk
Determine salience at every point using principal
moments
Tensor Voting
Stochastic Completion Field

Random walk:
x  cos 
y  sin 


N (0,  2 )
In addition, a particle may die with probability:
e
1/ r
Stochastic Completion Fields
Stochastic Completion Fields

Most probable path:
  k 2 ( s )ds   ds
with



1
2
1
   log(  2 )
r

2
Can be implemented as a convolution
Stochastic Completion Fields
Stochastic Completion Fields
Snakes

Given a curve Г(s)=(x(s),y(s)), define:
1
      E (( s ))ds
with
0
E (( s ))  Eimage  Eint  Eext
Eimage  I ( x, y )


  ( s) 2
Eint   ( s )
s
s
2
Eext  ...
2
2
Extremum: Calculus of Variation


Given a functional
T
  x  s     E ( x, x)ds
0
A condition for a local extrimum is obtained using the
Euler-Lagrange equation
   x  E d  E 

 
0
 x( s) x ds  x 

Curve evolution is defined
x( s, t ) E d  E 

 
0
t
x ds  x 

Solution obtained when
x( s, t )
0
t
Curve evolution
Level Set Methods
S ( x, y; t )
Curve defined implicitly
by S ( x, y; t )  0
Curve Evolution
Curve Evolution
Shortest Path
Image Segmentation: Thresholding
Histogram
1200
1000
800
600
400
200
0
0
50
100
150
200
250
Thresholding
Thresholding
125
99
156
S-T Min-Cut/Max Flow
S-T Min-Cut/Max Flow
t
S
Normalized Cuts

Given a graph G=(V,E), define

W = {wij} weights

D = diag{di}, di   wij
j


L = D - W Laplacian
Let V  A  B, we seek to solve
 cut ( A, B)
cut ( A, B) 



min
assoc( B,V ) 
V  A B  assoc ( A, V )
Normalized Cuts

This can be show to be equivalent to
T
u Lu
min T
u u Du
with
 1

ui   k
 1  k


vi  A
vi  B
and u T D1  0
With these constrains the problem is NP-hard.
Without the constraint the solution is obtained
through the generalized eigenvalue problem
Lu   Du
Normalized Cuts

Dividing into two segments:



Partition determined by the eigenvector with the
second smallest eigenvalue
We need to pick a threshold
Dividing into more than two segments:



Pick several thresholds.
Divide each segment recursively.
Pick the best few eigenvectors and then perform
k-means.
Texture Examples
Filter Bank
Textons
image
textons
texton
assignment
Normalized Cuts
Mean Shift Segmentation
Mean Shift Segmentation



Given an image, convert it to a function that is
inversely related to edgeness
Perform mean shift from every pixel
Cluster pixels that lead to the same peak
Mean Shift Segmentation
Summary


Local processing is often insufficient to separate
objects
We reviewed several approaches for


curve extraction, completion
region segmentation
Preattentive: Parallel
Preattentive: Parallel
Attention: Serial