Object Segmentation Based on
Multiple Features Fusion and Conditional Random Field
CASIA_IGIT
National Laboratory of Pattern Recognition(NLPR)
Institute of Automation, Chinese Academy of Sciences(CASIA)
Reporter ： Kun Ding （丁昆）
2013.10.17

• System Overview
• System Characteristics
Outline
• Results and Conclusions

• System Overview
• System Characteristics
Outline
• Results and Conclusions

System Overview
• Object Segmentation Pipeline
Superpixel Segmentation Feature Extraction SVM Classification
Feature
Engineering
Features Input Image Superpixels
GrabCut
Probabilistic Output Final Results
Stage 1 :
Superpixel Classification
Stage 2 :
Pixel-based CRF Smoothing

System Overview
• Superpixel Classification
• Superpixel Segmentation
• Graph-based image segmentation
• Feature Extraction:
• To be detailed in next section
• SVM Classification[1]
• RBF kernel with Probabilistic Output

System Overview
• Pixel-Based CRF Smoothing
• Fusing several kinds of information as data term
• Solving with GrabCut with only a few iterations
Binarize
First
Iteration
Second
Iteration
SVM Probabilistic Output CRF Smoothing Output

Outline
• System Overview
• System Characteristics
• Results and Conclusions

System Characteristics
• Superpixel Segmentation --
Efficient Graph-Based Image Segmentation[2]
• Fast, property of edge-preserving
• Speeding up the whole procedure
• Improving the separability between foreground and background
Superpixels and their edge-preserving property

System Characteristics
• Feature Engineering –
Superpixel-Based Multiple Features Fusion
Gradient Dense SIFT[3][4] dictionary with Bag-of-Words description
Texture
Multi-scale LBP histogram
Color and skin RGB histogram and HS histogram with skin detection
Geometrical Position, direction and roundness
Saliency Color spatial distribution, multi-scale local and global contrast
Results of Object Detection
Probability derived from AdaBoost, with manifold ranking[6] refinement
PCA

System Characteristics
• Feature Engineering –
Superpixel-Based Multiple Features Fusion
• Illustration of object detection
Object Detection result Rectangle Density as Probability Refined with Manifold Ranking

System Characteristics
• Pixel-Based CRF Smoothing – GrabCut[7]
• Modified data term
• Solving by maxflow iteratively
SVM Result Object Detection
Result
GMM Result for Foreground and Background
CRF Smoothing Output

• System Overview
• System Characteristics
Outline
• Results and Conclusions

Conclusion and Results Exhibition
• Results Exhibition

Conclusion and Results Exhibition
• Conclusion
• Superpixel classification
• Feature fusion works
• CRF smoothing improves the results of SVM
• Object parts sometimes lost
• Context information is inadequate

Selected References
[1] C.-C. Chang and C.-J. Lin. LIBSVM: a library for support vector machines,
2001. Software available at http: //www.csie.ntu.edu.tw/˜cjlin/libsvm.
[2] Felzenszwalb P F, Huttenlocher D P. Efficient graph-based image segmentation[J]. International Journal of Computer Vision, 2004, 59(2):
167-181.
[3] Lowe D G. Distinctive image features from scale-invariant keypoints[J].
International journal of computer vision, 2004, 60(2): 91-110.
[4] Vedaldi A, Fulkerson B. VLFeat: An open and portable library of computer vision algorithms[C]//Proceedings of the international conference on Multimedia. ACM, 2010: 1469-1472.

Selected References
[5] Liu T, Yuan Z, Sun J, et al. Learning to detect a salient object[J].
Pattern Analysis and Machine Intelligence, IEEE Transactions on, 2011,
33(2): 353-367.
[6] Chuan Yang, Lihe Zhang, Huchuan Lu, Minghsuan Yang, Saliency
Detection via Graph-Based Manifold Ranking, CVPR2013, P3166-3173
[7] Rother C, Kolmogorov V, Blake A. Grabcut: Interactive foreground extraction using iterated graph cuts[C]//ACM Transactions on Graphics
(TOG). ACM, 2004, 23(3): 309-314.

CASIA_IGIT
Leader : Ying Wang （王颖）
Members : Kun Ding （丁昆）
Huxiang Gu （谷鹄翔）
Yongchao Gong （宫永超）
E-mails : {ywang, kding, hxgu, yongchao.gong}@nlpr.ia.ac.cn