Random Walk with Restart (RWR)
for Image Segmentation
Sungsu Lim
AALAB, KAIST
Image Segmentation

Computer vision: make machine to see or to understand/
interpret the scenes (images & videos) like human do.

Image segmentation is one of the most challenging issues
in computer vision.

Two major difficulties of conventional algorithms: weak
boundary problem & texture problem.

Semi-supervised segmentation approaches are preferred
since user inputs can reduce the ambiguity in images.
2011-02-08
RWR for image segmentation
2
Random Walks for Image Segmentation

RW (L. Grady, PAMI2006): In image segmentation, random
walks are used to determine the labels (i.e., “object” or
“background”) to associate with each pixel.

K-way image segmentation: given user-defined seeds
indicating regions of the image belonging to k objects.
Each seed specifies a location with a user-defined label.

We can use hitting time or commute time as relevance
score between two nodes (seed and unlabeled pixel).
By assigning each pixel to the label for which the best
value is calculated.
2011-02-08
RWR for image segmentation
3
Random walk with restart
9
10
12
2
8
1
11
3
4
6
5
7
4
Example of RW
What if we star
t at a different
node?
Start node
2011-02-08
RWR for image segmentation
5
RWR for Classification

Simple idea: use RW for classification
RW with start no
des being labeled
points in class A
2011-02-08
RW with start n
odes being labele
d points in class B
Nodes frequented more by RW(A)
belongs to class A, otherwise they b
elong to B
6
RWR for Image Segmentation

Limitation of RW: only considers the local relationship
between the pixel and that border. (more prone to hit
popular nodes)

RWR (Kim, Lee and Lee, ECCV2008): a new generative
image segmentation algorithm based on Random Walks
with Restart (Pan,Yang and Faloutsos, KDD2004)

Most previous semi-supervised image segmentation
algorithms focus on the inter-label discrimination, but it
introduce a generative model for image segmentation.
2011-02-08
RWR for image segmentation
7
Generative Model
8
Random Walk with Restart

Imagine a network, and starting at a specific node, you
follow the edges randomly.

But (perhaps you’re afraid of wondering too far) with
some probability, you “jump” back to the starting node
(restart!).
If you record the number of times
you land on each node, what would
that distribution look like?
9
Random Walk with Restart

The walk distribution r satisfies a simple equation:
Equivalent to the
well-known Googl
e PageRank if all n
odes are start nod
es! (e is uniform)
Transition matrix
(relevance vector)
π  (1  c ) P π  c e
“Keep-going”
probability
(damping factor)
2011-02-08
Seed vector
(start nodes)
Ranking
vector
RWR for image segmentation
Restart
probability
10
Example of RWR
Iterative update until convergence
π  (1  c ) P π
t
t 1
 0.13 
 0 1/3 1/3 1/3 0 0 0 0



0.10
1/3 0 1/3 0 0 0 0 1/4



 0.13 
 1/3 1/3 0 1/3 0 0 0 0



 0.22 
 1/3 0 1/3 0 1/4 0 0 0
 0.13 
 0
0
0 1/3 0 1/2 1/2 1/4



0
0
0 1/4 0 1/2 0
 0.05 
 0
 0.9 
 0.05 
 0
0
0
0 1/4 1/2 0 0



0 1/4 0 0 0
 0.08 
 0 1/3 0



0.04
0
0
0
0
0 0 0 1 /4



 0.03 
 0
0
0
0
0 0 0 0



0
0
0
0 0 0 1/4
 0.04 
 0
 0.02 
 0
0
0
0
0 0 0 0



nx1
11
2011-02-08
 ce
  0.13 
0


 
0 0 0
0
0.10
0


 
0
0 0 0
0   0.13 


 
0 0 0
0   0.22 
1 



0
0 0 0
0
0.13


 
0 0 0
0   0.05 
0
 0.1 



0
0 0 0
0
0.05


 
1/2 0 1/3 0   0.08 
0


 
0 1/3 0 0
0.04
0


 
0
1 /2 0 1/3 1/2   0.03 


 
0 1/3 0 1/2   0.04 
0



0
0 1/3 1/3 0   0.0 2 
 
0 0 0
nxn
0
9
1
2
8
3
10
12
11
4
5
6
7
nx1
RWR for image segmentation
11
Use of RWR
1
π  c ( I  (1  c ) P ) e

Linear solution:

It can be reformulated as

π
 c (1  c )
t
t
P e
(0  c  1 )
t0
As t increases weight
becomes smaller.

Weighted average
of all probability
Restart
probability
It considers all relations at all scales in the image.
2011-02-08
RWR for image segmentation
12
Use of RWR

π
 c (1  c )
t
t
P e
t0

As restarting probability c decreases, coarser scale is
more emphasized in likelihood term.
2011-02-08
RWR for image segmentation
13
Energy minimization framework

Quadratic energy (cost) minimization:
similar to the formulation of RWR
π *  arg min E ( π )
π
E (π) 
P
ij
(πi  π j ) 
2
  (π
i, j
E
π
i
 e)
2
i
 π  P π   (π  e)  0
π  (1  c ) P π  c e
2011-02-08
( c
RWR for image segmentation

1 
)
14
Experimental Results
2011-02-08
RWR for image segmentation
15
Applications

1. Data-Driven RWR (Kim, Lee and Lee, ICIP2009)
It use the restart probability matrix. The restarting probability of
each pixel depend on its edgeness, generated by Canny edge
detector.

2. High-order RWR (multi-layer graph model)
2011-02-08
RWR for image segmentation
16