Kapitel 13
Interactive Segmentation
 Live-wire approach
 Random walker segmentation
Kapitel 13 “Interactive Segmentation” – p. 1
Live-wire approach
Live-wire approach (intelligent scissors):
 The user interactively picks a seed point on the boundary.
 Then, a live-wire is displayed in real time from the initial point to any
subsequent position taken by the cursor.
 The entire 2D boundary is specified by means of a set of live-wire
segments in this manner.
The detection of segments is formulated as a graph search problem,
which finds the globally optimal (minimum-weight) path between an
initial start pixel and an end pixel.
Kapitel 13 “Interactive Segmentation” – p. 2
Live-wire approach
E.N. Mortensen and W.A. Barrett: Intelligent scissors for image
composition. SIGGRAPH, 191-198, 1995.
Kapitel 13 “Interactive Segmentation” – p. 3
Exkurs: Dijkstra‘s algorithm
Problem: finding shortest paths from a source vertex to all other vertices
in a graph.
Dijkstra's algorithm: Greedy solution to the single-source shortest path
problem. It works on both directed and undirected graphs. All edges
must have nonnegative weights.
Input: Weighted graph G = {E, V} and start vertex s∈V, such that all
edge weights are nonnegative.
Output: Lengths of shortest paths (or the shortest paths themselves)
from the given start vertex s∈V to an end node t∈V (or all other vertices).
Kapitel 13 “Interactive Segmentation” – p. 4
Exkurs: Dijkstra‘s algorithm
Step 1: Assign the start node with distance zero and mark it as visited.
Assign all other vertices with infinity and mark them unvisited.
Step 2: Consider the most recently visited node X with distance Dx. For
each of its unvisited neighbors Y with distance Dy, replace Dy by
min(Dx+ weight(X,Y), Dy)
Step 3: Choose the unvisited vertex with the smallest distance and mark
it visited.
Step 4: Repeat Steps 2 and 3 until the end node is marked visited.
Step 5: Go backwards through the graph, retracing the minimum-weight
path from the end node to the start node.
Kapitel 13 “Interactive Segmentation” – p. 5
Exkurs: Dijkstra‘s algorithm
Example: Find the minimum-weight path from node s
to node t.
s
2
1
5
2
3
1
2
3
4
t
Kapitel 13 “Interactive Segmentation” – p. 6
Exkurs: Dijkstra‘s algorithm
2
1
5
2
1
2
3
1
2
2
3
4
5
5
2
1
3
5
6
2
4
1
2
3
2
1
2
3
3
4
1
4
4
6
5
3
2
2
1
3
1
3
1
7
4
2
5
2
3
3
∞
3
4
2
3
4
3
5
4
3
1
32
3
2
1
∞
4
1
23
5
2
3
3
2
1
3
5
1
1
∞
2
3
1
1
2
2
3
2
4
5
Kapitel 13 “Interactive Segmentation” – p. 7
Exkurs: Search algorithm from AI
Search is a classical AI problem and many solutions exists (e.g. A*)
Kapitel 13 “Interactive Segmentation” – p. 8
Live-wire approach
Example:
Kapitel 13 “Interactive Segmentation” – p. 9
Live-wire approach
Example: Composition
E.N. Mortensen and W.A. Barrett: Intelligent scissors for image
composition. SIGGRAPH, 191-198, 1995.
Kapitel 13 “Interactive Segmentation” – p.10
Live-wire approach
Example: A cursor snap mechanism forces the mouse point to the pixel
of maximum gradient magnitude within a user-specified neighborhood.
Live-wire segment snaps to a boundary as the free point moves (via
cursor movement). The path of the free point is shown in white. Livewire segments from previous free point positions are shown in green.
Kapitel 13 “Interactive Segmentation” – p.11
Live-wire approach
Example: Continuous snap–drag of a live-wire to a coronary edge (the
boundary is completed in about 2s)
E.N. Mortensen and W.A. Barrett:
Interactive live-wire boundary
extraction. Medical Image Analysis,
1(4): 331-341.
Kapitel 13 “Interactive Segmentation” – p.12
Live-wire approach
Reproducibility for live-wire and manual
tracing tool. Eight users extracted five
object boundaries five times with the livewire tool and the same object boundaries
three times with the manually tracing tool.
Manual tracing takes 2-3 times longer.
The boundaries are virtually identical
regardless of which user is performing
the task!
Kapitel 13 “Interactive Segmentation” – p.13
Live-wire approach
Extension to 3D: segment 3D volume data or time sequences of 2D
images.
A.X. Falcao and J.K. Udupa: A 3D generalization of user-steered
live-wire segmentation. Medical Image Analysis, 4(4), 389-402, 2000.
The user specifies contours via live-wiring on a few slices that are
orthogonal to the natural slices of the original data. If these slices are
selected strategically, then one obtains a sufficient number of seed
points in each natural slice which enable a subsequent automatic
optimal boundary detection therein.
Kapitel 13 “Interactive Segmentation” – p.14
Random walker segmentation
Given labeled pixels, for each pixel: What is the probability that a
random walker starting from this pixel first reaches each set of labels?
L.Grady: Random walks for image segmentation. IEEE-TPAMI, 28:
1768–1783, 2006.
unseeded (unlabeled) pixel
(a) A two-region image
edge weight:
similarity between two nodes, based on
e.g., intensity gradient, color changes
(c) A 4-connected lattice topology
seeded (labeled) pixels
(b) Use-defined seeds for each region
low-weight edge (sharp color gradient)
(d) An undirected weighted graph
Kapitel 13 “Interactive Segmentation” – p.15
Random walker segmentation
The algorithm labels an unseeded pixel in following steps:
Step 1. Calculate the probability that a random walker starting at an
unseeded pixel x first reaches a seed with label s
0.97
0.90
0.97
0.97
0.90
0.85
0.15
0.85
0.15
0.85
0.15
0.10
0.10
0.03
0.03
0.03
0.03
0.03
0.03
Probability that a random walker starting from
each unseeded node first reaches red seed
0.10
0.10
0.15
0.85
0.15
0.85
0.15
0.85
0.90
0.97
0.97
0.90
0.97
Probability that a random walker starting from
each unseeded node first reaches blue seed
Kapitel 13 “Interactive Segmentation” – p.16
Random walker segmentation
Step 2. Label each pixel with the most probable seed destination
(0.97,0.03)
(0.90,0.10)
(0.97,0.03)
(0.97,0.03)
(0.90,0.10)
(0.85,0.15)
(0.15,0.85)
(0.85,0.15)
(0.15,0.85)
(0.85,0.15)
(0.15,0.85)
(0.10,0.90)
(0.03,0.97)
(0.03,0.97)
(0.10,0.90)
(0.03,0.97)
A segmentation corresponding to region boundary is obtained by
biasing the random walker to avoid crossing sharp color gradients
Kapitel 13 “Interactive Segmentation” – p.17
Random walker segmentation
partially labeled image
green
segmented image
Probabilities
red
yellow
blue
Kapitel 13 “Interactive Segmentation” – p.18
Random walker segmentation
Algorithm summary:
1. Generate weights based on image intensities.
2. Build Laplacian matrix.
3. Solve system of linear equations for each label.
4. Assign pixel (voxel) to label for which it has the highest probability.
Kapitel 13 “Interactive Segmentation” – p.19
Random walker segmentation
Weight generation:
The random walker is governed by edge weights (e.g., probabilities
of reaching a pixel pj from its neighboring pixel pi).
f(): intensity function
β: influences how quickly the probability decreases
The probability is higher the more likely the two pixels belong to the
same region.
Kapitel 13 “Interactive Segmentation” – p.20
Random walker segmentation
Laplacian matrix L:
di : degree of pixel pi
It is shown that finding the probabilities for a walk starting at some
pixel to arrive at some seed pixel is related to minmizing:
Kapitel 13 “Interactive Segmentation” – p.21
Random walker segmentation
Solution of system of linear equations:
Partition the pixels into two sets, VM (all marked/seed pixels, regardless
of their label) and VU (unseeded). The pixels in L and x are ordered
such that seed pixels are first and unseeded pixels are second. Then,
the optimization problem becomes:
LM: edge weighted among the marked pixels
LU: edge weighted among the unmarked pixels
Kapitel 13 “Interactive Segmentation” – p.22
Random walker segmentation
To minimize it, the equation is differentiated with respect to unknown XU:
This is set to zero for finding the minumum of D(XU), resulting a system
of |VU| linear equations:
XM: 1 for the label under consideration, and 0 for all other labels
(remember: we need to solve XU for each label)
Kapitel 13 “Interactive Segmentation” – p.23
Random walker segmentation
Cardiac segmentation across modalities
Kapitel 13 “Interactive Segmentation” – p.24
Random walker segmentation
Segmentation of objects with varying size, shape and texture
Kapitel 13 “Interactive Segmentation” – p.25
Random walker segmentation
Segmentation of natural images
Kapitel 13 “Interactive Segmentation” – p.26
Random walker segmentation
 Author webpage:
http://cns.bu.edu/~lgrady
 Random walker paper:
http://cns.bu.edu/~lgrady/grady2006random.pdf
 Random walker MATLAB code:
http://cns.bu.edu/~lgrady/random_walker_matlab_code.zip
 Random walker demo page:
http://cns.bu.edu/~lgrady/Random_Walker_Image_Segmentation.html
Kapitel 13 “Interactive Segmentation” – p.27