advertisement

Random Walk with Restart (RWR) for Image Segmentation Sungsu Lim AALAB, KAIST Image Segmentation Computer vision: make machine to see or to understand/ interpret the scenes (images & videos) like human do. Image segmentation is one of the most challenging issues in computer vision. Two major difficulties of conventional algorithms: weak boundary problem & texture problem. Semi-supervised segmentation approaches are preferred since user inputs can reduce the ambiguity in images. 2011-02-08 RWR for image segmentation 2 Random Walks for Image Segmentation RW (L. Grady, PAMI2006): In image segmentation, random walks are used to determine the labels (i.e., “object” or “background”) to associate with each pixel. K-way image segmentation: given user-defined seeds indicating regions of the image belonging to k objects. Each seed specifies a location with a user-defined label. We can use hitting time or commute time as relevance score between two nodes (seed and unlabeled pixel). By assigning each pixel to the label for which the best value is calculated. 2011-02-08 RWR for image segmentation 3 Random walk with restart 9 10 12 2 8 1 11 3 4 6 5 7 4 Example of RW What if we star t at a different node? Start node 2011-02-08 RWR for image segmentation 5 RWR for Classification Simple idea: use RW for classification RW with start no des being labeled points in class A 2011-02-08 RW with start n odes being labele d points in class B Nodes frequented more by RW(A) belongs to class A, otherwise they b elong to B 6 RWR for Image Segmentation Limitation of RW: only considers the local relationship between the pixel and that border. (more prone to hit popular nodes) RWR (Kim, Lee and Lee, ECCV2008): a new generative image segmentation algorithm based on Random Walks with Restart (Pan,Yang and Faloutsos, KDD2004) Most previous semi-supervised image segmentation algorithms focus on the inter-label discrimination, but it introduce a generative model for image segmentation. 2011-02-08 RWR for image segmentation 7 Generative Model 8 Random Walk with Restart Imagine a network, and starting at a specific node, you follow the edges randomly. But (perhaps you’re afraid of wondering too far) with some probability, you “jump” back to the starting node (restart!). If you record the number of times you land on each node, what would that distribution look like? 9 Random Walk with Restart The walk distribution r satisfies a simple equation: Equivalent to the well-known Googl e PageRank if all n odes are start nod es! (e is uniform) Transition matrix (relevance vector) π (1 c ) P π c e “Keep-going” probability (damping factor) 2011-02-08 Seed vector (start nodes) Ranking vector RWR for image segmentation Restart probability 10 Example of RWR Iterative update until convergence π (1 c ) P π t t 1 0.13 0 1/3 1/3 1/3 0 0 0 0 0.10 1/3 0 1/3 0 0 0 0 1/4 0.13 1/3 1/3 0 1/3 0 0 0 0 0.22 1/3 0 1/3 0 1/4 0 0 0 0.13 0 0 0 1/3 0 1/2 1/2 1/4 0 0 0 1/4 0 1/2 0 0.05 0 0.9 0.05 0 0 0 0 1/4 1/2 0 0 0 1/4 0 0 0 0.08 0 1/3 0 0.04 0 0 0 0 0 0 0 1 /4 0.03 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1/4 0.04 0 0.02 0 0 0 0 0 0 0 0 nx1 11 2011-02-08 ce 0.13 0 0 0 0 0 0.10 0 0 0 0 0 0 0.13 0 0 0 0 0.22 1 0 0 0 0 0 0.13 0 0 0 0 0.05 0 0.1 0 0 0 0 0 0.05 1/2 0 1/3 0 0.08 0 0 1/3 0 0 0.04 0 0 1 /2 0 1/3 1/2 0.03 0 1/3 0 1/2 0.04 0 0 0 1/3 1/3 0 0.0 2 0 0 0 nxn 0 9 1 2 8 3 10 12 11 4 5 6 7 nx1 RWR for image segmentation 11 Use of RWR 1 π c ( I (1 c ) P ) e Linear solution: It can be reformulated as π c (1 c ) t t P e (0 c 1 ) t0 As t increases weight becomes smaller. Weighted average of all probability Restart probability It considers all relations at all scales in the image. 2011-02-08 RWR for image segmentation 12 Use of RWR π c (1 c ) t t P e t0 As restarting probability c decreases, coarser scale is more emphasized in likelihood term. 2011-02-08 RWR for image segmentation 13 Energy minimization framework Quadratic energy (cost) minimization: similar to the formulation of RWR π * arg min E ( π ) π E (π) P ij (πi π j ) 2 (π i, j E π i e) 2 i π P π (π e) 0 π (1 c ) P π c e 2011-02-08 ( c RWR for image segmentation 1 ) 14 Experimental Results 2011-02-08 RWR for image segmentation 15 Applications 1. Data-Driven RWR (Kim, Lee and Lee, ICIP2009) It use the restart probability matrix. The restarting probability of each pixel depend on its edgeness, generated by Canny edge detector. 2. High-order RWR (multi-layer graph model) 2011-02-08 RWR for image segmentation 16