IEEE International Conference on Computer Vision (ICCV), 2015, Santiago, Chile Minimum Barrier Salient Object Detection at 80 FPS J I A N M I N G Z H A N G , S TA N S C L A R O F F, Z H E L I N , X I A O H U I S H E N , B R I A N P R I C E , RADOMIR MECH IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015 1 Salient Object Detection Generate a Saliency Map for segmenting significant objects in an image. Input Saliency Map IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015 2 Bottom-up Saliency Cues Contrast/Rarity Image Boundary Connectivity Frequency salient Appearance Space IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015 3 Previous Works Rairity/Contrast Boundary Connectivity SO [Zhu et al. CVPR’14] State-of- AMC [Jiang et al. ICCV’13] the-art GS [Wei et al. ECCV’12] RC [Cheng et al. TPAMI’15] IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015 4 Previous Works Rairity/Contrast Boundary Connectivity SO [Zhu et al. CVPR’14] State-of- AMC [Jiang et al. ICCV’13] the-art GS [Wei et al. ECCV’12] RC [Cheng et al. TPAMI’15] IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015 Using Super-pixel? Yes Yes Yes Yes 4 Previous Works Rairity/Contrast Boundary Connectivity SO [Zhu et al. CVPR’14] State-of- AMC [Jiang et al. ICCV’13] the-art GS [Wei et al. ECCV’12] Fast Using Super-pixel? Yes Yes Yes RC [Cheng et al. TPAMI’15] Yes FT [Achanta et al CVPR’09] No HC [Cheng et al. CVPR’11] No IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015 4 Previous Works Rairity/Contrast Boundary Connectivity SO [Zhu et al. CVPR’14] State-of- AMC [Jiang et al. ICCV’13] the-art GS [Wei et al. ECCV’12] Fast Using Super-pixel? Yes Yes Yes RC [Cheng et al. TPAMI’15] Yes FT [Achanta et al CVPR’09] No HC [Cheng et al. CVPR’11] No Proposed No IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015 4 Contributions FastMBD, a fast approximate Minimum Barrier Distance (MBD) [Strand et al. CVIU’13] transform algorithm, with error bound analysis. A salient object detection method based on FastMBD, which achieves state-of-the-art performance and is one order of magnitude faster! 77 FPS IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015 5 System Overview MBD Transform Input L MBD-L a MBD-a b MBD-b Post-processing MBD Sal-Map Backgroundness IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015 6 Measuring Boundary Connectivity by Distance Transform Compute the distance for each pixel w.r.t. the image boundary Seed Set Shortest Path IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015 7 MBD vs Geodesic Distance In what follows, we consider a single-channel image. MBD [Strand et al. CVIU’13] Geodesic + IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015 = 8 MBD vs Geodesic Distance MBD is robust to small pixel value fluctuation MBD + = = Geodesic IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015 9 FastMBD A raster-scanning iterative algorithm Share the same spirit of the raster-scanning geodesic distance transform [Toivanen, Pattern Recognition Letters, 1996] Highly efficient in practice (2ms/scan for a 320X240 image!) IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015 10 Algorithm For each visited pixel x: 1. Check each of the 4-connected neighbors y x IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015 11 Algorithm For each visited pixel x: 1. Check each of the 4-connected neighbors 2. Minimize the path cost 3. Update: ◦ ◦ ◦ D(x), cost of current assigned path U(x), highest value on assigned path L(x), lowest value on assigned path IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015 D(y) U(y) L(y) D(x) U(x) L(x) 11 Some Analysis … FastMBD returns an upper-bound of the MBD for each pixel. FastMBD will eventually converge. Q: Is the converged solution of FastMBD equal to the exact MBD transform? A: No [Falcao et al. TPAMI’04]. IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015 12 An Error Bound Result Definition: Maximum Local Difference is the maximum absolute pixel value difference between a pair of pixels that share an edge or a corner on an image . Measures maximal local discontinuity Under a mild condition, the approximation error of the converged solution of FastMBD is bounded by . The more continuous the image, the smaller the error. IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015 13 Empirical Evaluation of FastMBD PASCAL-S IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015 14 Minimum Barrier Salient Object Detection 1. For each color channel, compute the MBD map by FastMBD 2. Post-process the averaged MBD map a) Morphological smoothing b) Multiplying a center-distance map c) Contrast enhancement Input Averaged MBD Map IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015 Output 15 Leveraging Backgroundness 1. For each color channel, compute the MBD map by FastMBD 2. Compute the Image Boudnary Contrast (IBC) Map 3. Post-process the averaged MBD map + IBC Map Averaged MBD Map IBC Map Output + IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015 16 Computing the IBC Map Image boundary region is mostly background. Mahalanobis Distance to Mean Color Compute mean color and color covariance IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015 17 Computing the IBC Map IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015 17 Enhancement by the IBC Map W/O IBC Map IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015 With IBC Map 18 Experiments Datasets MSRA10K DUTOmron ECSSD PASCAL-S #Images 10000 5168 1000 850 Difficulty + +++ ++ +++ Compared Methods SO [Zhu et al. CVPR’14] FT [Achanta et al. CVPR’09] AMC [Jiang et al. ICCV’13] HC [Cheng etal. CVPR’11] HS [Yan et al. CVPR’13] MB+ (full system) GS [Wei et al. ECCV’12] MB (w/o IBC map) RC [Cheng et al. TPAMI’15] GD (Same as MB but uses Geodesic Distance) SIA [Cheng et al. ICCV’13] IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015 19 Performance Precision-Recall Curve Comparison with the State-of-the-art Comparison with Other Baselines IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015 20 Performance Weighted Fβ [Margolin et al. CVPR’14] IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015 21 Speed Excluding IO time Using a singe thread 0.6 MB+ Mean Weighted Fb MB 0.5 0.4 0.3 0.2 0 20 40 60 80 100 FPS IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015 22 Qualitative Evaluation IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015 23 Conclusion Propose to use MBD to measure image boundary connectivity Present FastMBD, a fast approximation MBD transform algorithm Achieve state-of-the-art performance at a substantially reduced computational cost (~80 FPS) An executable program is available on our website http://www.cs.bu.edu/groups/ivc/fastMBD/ IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015 24 Performance Area Under the Curve (AUC) IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015 25 Performance mean Average Error (mAE) IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015 26 Performance Controlling Post-processing IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015 27 Algorithm For each visited pixel x: 1. Check each of the 4-connected neighbors 2. Minimize the path cost 3. Update D(x), L(x), U(x) IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015 D(y) U(y) L(y) D(x) U(x) L(x) 11