IEEE International Conference on Computer Vision (ICCV), 2015, Santiago, Chile
Minimum Barrier Salient Object
Detection at 80 FPS
J I A N M I N G Z H A N G , S TA N S C L A R O F F, Z H E L I N , X I A O H U I S H E N , B R I A N P R I C E ,
RADOMIR MECH
IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015
1
Salient Object Detection
Generate a Saliency Map for segmenting significant objects in an image.
Input
Saliency Map
IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015
2
Bottom-up Saliency Cues
Contrast/Rarity
Image Boundary Connectivity
Frequency
salient
Appearance Space
IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015
3
Previous Works
Rairity/Contrast
Boundary
Connectivity
SO [Zhu et al. CVPR’14]
State-of- AMC [Jiang et al. ICCV’13]
the-art GS [Wei et al. ECCV’12]
RC [Cheng et al. TPAMI’15]
IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015
4
Previous Works
Rairity/Contrast
Boundary
Connectivity
SO [Zhu et al. CVPR’14]
State-of- AMC [Jiang et al. ICCV’13]
the-art GS [Wei et al. ECCV’12]
RC [Cheng et al. TPAMI’15]
IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015
Using
Super-pixel?
Yes
Yes
Yes
Yes
4
Previous Works
Rairity/Contrast
Boundary
Connectivity
SO [Zhu et al. CVPR’14]
State-of- AMC [Jiang et al. ICCV’13]
the-art GS [Wei et al. ECCV’12]
Fast
Using
Super-pixel?
Yes
Yes
Yes
RC [Cheng et al. TPAMI’15]
Yes
FT [Achanta et al CVPR’09]
No
HC [Cheng et al. CVPR’11]
No
IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015
4
Previous Works
Rairity/Contrast
Boundary
Connectivity
SO [Zhu et al. CVPR’14]
State-of- AMC [Jiang et al. ICCV’13]
the-art GS [Wei et al. ECCV’12]
Fast
Using
Super-pixel?
Yes
Yes
Yes
RC [Cheng et al. TPAMI’15]
Yes
FT [Achanta et al CVPR’09]
No
HC [Cheng et al. CVPR’11]
No
Proposed
No
IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015
4
Contributions
FastMBD, a fast approximate Minimum Barrier Distance (MBD) [Strand
et al. CVIU’13] transform algorithm, with error bound analysis.
A salient object detection method based on FastMBD, which achieves
state-of-the-art performance and is one order of magnitude faster!
77 FPS
IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015
5
System Overview
MBD Transform
Input
L
MBD-L
a
MBD-a
b
MBD-b
Post-processing
MBD
Sal-Map
Backgroundness
IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015
6
Measuring Boundary Connectivity
by Distance Transform
Compute the distance for each pixel w.r.t. the image boundary
Seed Set
Shortest Path
IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015
7
MBD vs Geodesic Distance
In what follows, we consider a single-channel image.
MBD [Strand et al. CVIU’13]
Geodesic
+
IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015
=
8
MBD vs Geodesic Distance
MBD is robust to small pixel value fluctuation
MBD
+
=
=
Geodesic
IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015
9
FastMBD
A raster-scanning iterative algorithm
Share the same spirit of the raster-scanning geodesic distance transform
[Toivanen, Pattern Recognition Letters, 1996]
Highly efficient in practice (2ms/scan for a 320X240 image!)
IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015
10
Algorithm
For each visited pixel x:
1. Check each of the 4-connected
neighbors
y
x
IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015
11
Algorithm
For each visited pixel x:
1. Check each of the 4-connected
neighbors
2. Minimize the path cost
3. Update:
◦
◦
◦
D(x), cost of current assigned path
U(x), highest value on assigned path
L(x), lowest value on assigned path
IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015
D(y)
U(y)
L(y)
D(x)
U(x)
L(x)
11
Some Analysis …
FastMBD returns an upper-bound of the MBD for each pixel.
FastMBD will eventually converge.
Q: Is the converged solution of FastMBD equal to the exact MBD
transform?
A: No  [Falcao et al. TPAMI’04].
IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015
12
An Error Bound Result
Definition: Maximum Local Difference
is the maximum absolute
pixel value difference between a pair of pixels that share an edge or a
corner on an image .
Measures maximal local discontinuity
Under a mild condition, the approximation error of the
converged solution of FastMBD is bounded by
.
The more continuous the image, the smaller the error.
IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015
13
Empirical Evaluation of
FastMBD
PASCAL-S
IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015
14
Minimum Barrier Salient
Object Detection
1.
For each color channel, compute the MBD map by FastMBD
2.
Post-process the averaged MBD map
a) Morphological smoothing
b) Multiplying a center-distance map
c) Contrast enhancement
Input
Averaged MBD Map
IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015
Output
15
Leveraging Backgroundness
1.
For each color channel, compute the MBD map by FastMBD
2.
Compute the Image Boudnary Contrast (IBC) Map
3.
Post-process the averaged MBD map + IBC Map
Averaged MBD Map
IBC Map
Output
+
IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015
16
Computing the IBC Map
Image boundary
region is mostly
background.
Mahalanobis Distance
to Mean Color
Compute mean color
and color covariance
IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015
17
Computing the IBC Map
IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015
17
Enhancement by the IBC Map
W/O IBC Map
IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015
With IBC Map
18
Experiments
Datasets
MSRA10K
DUTOmron
ECSSD
PASCAL-S
#Images
10000
5168
1000
850
Difficulty
+
+++
++
+++
Compared Methods
SO [Zhu et al. CVPR’14]
FT [Achanta et al. CVPR’09]
AMC [Jiang et al. ICCV’13]
HC [Cheng etal. CVPR’11]
HS [Yan et al. CVPR’13]
MB+ (full system)
GS [Wei et al. ECCV’12]
MB (w/o IBC map)
RC [Cheng et al. TPAMI’15]
GD (Same as MB but uses
Geodesic Distance)
SIA [Cheng et al. ICCV’13]
IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015
19
Performance
Precision-Recall Curve
Comparison with the State-of-the-art
Comparison with Other Baselines
IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015
20
Performance
Weighted Fβ
[Margolin et al. CVPR’14]
IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015
21
Speed
Excluding IO time
Using a singe thread
0.6
MB+
Mean Weighted Fb
MB
0.5
0.4
0.3
0.2
0
20
40
60
80
100
FPS
IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015
22
Qualitative Evaluation
IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015
23
Conclusion
Propose to use MBD to measure image boundary connectivity
Present FastMBD, a fast approximation MBD transform algorithm
Achieve state-of-the-art performance at a substantially reduced
computational cost (~80 FPS)
An executable program is available on our website
http://www.cs.bu.edu/groups/ivc/fastMBD/
IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015
24
Performance
Area Under the Curve (AUC)
IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015
25
Performance
mean Average Error (mAE)
IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015
26
Performance
Controlling Post-processing
IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015
27
Algorithm
For each visited pixel x:
1. Check each of the 4-connected
neighbors
2. Minimize the path cost
3. Update D(x), L(x), U(x)
IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015
D(y)
U(y)
L(y)
D(x)
U(x)
L(x)
11