advertisement

Higher-Order Correlation Clustering for Image Segmentation Sungwoong Kim Department of EE, KAIST Daejeon, South Korea [email protected] Sebastian Nowozin Microsoft Research Cambridge Cambridge, UK [email protected] Pushmeet Kohli Microsoft Research Cambridge Cambridge, UK [email protected] Chang D. Yoo Department of EE, KAIST Daejeon, South Korea [email protected] Abstract For many of the state-of-the-art computer vision algorithms, image segmentation is an important preprocessing step. As such, several image segmentation algorithms have been proposed, however, with certain reservation due to high computational load and many hand-tuning parameters. Correlation clustering, a graphpartitioning algorithm often used in natural language processing and document clustering, has the potential to perform better than previously proposed image segmentation algorithms. We improve the basic correlation clustering formulation by taking into account higher-order cluster relationships. This improves clustering in the presence of local boundary ambiguities. We first apply the pairwise correlation clustering to image segmentation over a pairwise superpixel graph and then develop higher-order correlation clustering over a hypergraph that considers higher-order relations among superpixels. Fast inference is possible by linear programming relaxation, and also effective parameter learning framework by structured support vector machine is possible. Experimental results on various datasets show that the proposed higher-order correlation clustering outperforms other state-of-the-art image segmentation algorithms. 1 Introduction Image segmentation, a partitioning of an image into disjoint regions such that each region is homogeneous, is an important preprocessing step for many of the state-of-the-art algorithms for high-level image/scene understanding for three reasons. First, the coherent support of a region, commonly assumed to be of a single label, serves as a good prior for many labeling tasks. Second, these coherent regions allow a more consistent feature extraction that can incorporate surrounding contextual information by pooling many feature responses over the region. Third, compared to pixels, a small number of larger homogeneous regions significantly reduces the computational cost for a successive labeling task. Image segmentation algorithms can be categorized into either non-graph-based or graph-based algorithms. Some well-known non-graph-based algorithms represented by mode-seeking algorithms such as the K-means [1], mean-shift [2], and EM [3] are available, while well-known graph-based algorithms are available as the min-cuts [4], normalized cuts [5] and Felzenszwalb-Huttenlocher (FH) segmentation algorithm [6]. In comparison to non-graph-based segmentations, graph-based segmentations have been shown to produce consistent segmentations by adaptively balancing local 1 judgements of similarity [7]. Moreover, the graph-based segmentation algorithms with global objective functions such as the min-cuts and normalized cuts have been shown to perform better than the FH algorithm that is based on the local objective function, since the global-objective algorithms benefit from the global nature of the information [7]. However, in contrast to the min-cuts and normalized cuts which are node-labeling algorithms, the FH algorithm benefits from the edge-labeling in that it leads to faster inference and does not require a pre-specified number of segmentations in each image [7]. Correlation clustering is a graph-partitioning algorithm [8] that simultaneously maximizes intracluster similarity and inter-cluster dissimilarity by solving the global objective (discriminant) function. In comparison with the previous image segmentation algorithms, correlation clustering is a graph-based, global-objective, and edge-labeling algorithm and therefore, has the potential to perform better for image segmentation. Furthermore, correlation clustering leads to the linear discriminant function which allows for approximate polynomial-time inference by linear programming (LP) and large margin training based on structured support vector machine (S-SVM) [9]. A framework that uses S-SVM for training the parameters in correlation clustering has been considered previously by Finley et al. [10]; however, the framework was applied to noun-phrase and news article clusterings. Taskar derived a max-margin formulation for learning the edge scores for correlation clustering [11]. However, his learning criterion is different from the S-SVM and is limited to applications involving two different segmentations of a single image. Furthermore, Taskar does not provide any experimental comparisons or quantitative results. Even though the previous (pairwise) correlation clustering can consider global aspects of an image using the discriminatively-trained discriminant function, it is restricted in resolving the segment boundary ambiguities caused by neighboring pairwise relations presented by the pairwise graph. Therefore, to capture long-range dependencies of distant nodes in a global context, this paper proposes a novel higher-order correlation clustering to incorporate higher-order relations. We first apply the pairwise correlation clustering to image segmentation over a pairwise superpixel graph and then develop higher-order correlation clustering over a hypergraph that considers higher-order relations among superpixels. The proposed higher-order correlation clustering is defined over a hypergraph in which an edge can connect to two or more nodes [12]. Hypergraphs have been previously used to lift certain limitations of conventional pairwise graphs [13, 14, 15]. However, previously proposed hypergraphs for image segmentation are restricted to partitioning based on the generalization of normalized cut framework, which suffer from a number of problems. First, inference is slow and difficult especially with increasing graph size. A number of algorithms to approximate the inference process have been introduced based on the coarsening algorithm [14] and the hypergraph Laplacian matrices [13]; these are heuristic approaches and therefore are sub-optimal. Second, incorporating a supervised learning algorithm for parameter estimation under the spectral hypergraph partitioning framework is difficult. This is in line with the difficulties in learning spectral graph partitioning. This requires a complex and unstable eigenvector approximation which must be differentiable [16, 17]. Third, utilizing rich region-based features is restricted. Almost all previous hypergraph-based image segmentation algorithms are restricted to use only color variances as region features. The proposed higher-order correlation clustering overcomes all of these problems due to the generalization of the pairwise correlation clustering and enables to take advantages of using a hypergraph. The proposed higher-order correlation clustering algorithm uses as its input a hypergraph and leads to a linear discriminant function. A rich feature vector is defined based on several visual cues involving higher-order relations among superpixels. For fast inference, the LP relaxation is used to approximately solve the higher-order correlation clustering problem, and for supervised training of the parameter vector by S-SVM, we apply a decomposable structured loss function to handle unbalanced classes. We incorporate this loss function into the cutting plane procedure for S-SVM training. Experimental results on various datasets show that the proposed higher-order correlation clustering outperforms other state-of-the-art image segmentation algorithms. The rest of the paper is organized as follows. Section 2 presents the higher-order correlation clustering for image segmentation. Section 3 describes large margin training for supervised image segmentation based on the S-SVM and the cutting plane algorithm. A number of experimental and comparative results are presented and discussed in Section 4, followed by a conclusion in Section 5. 2 i k yik k yijk j yij y jk j (a) yj k (b) Figure 1: Illustrations of a part of (a) the pairwise graph (b) and the triplet graph built on superpixels. 2 Higher-order correlation clustering The proposed image segmentation is based on superpixels which are small coherent regions preserving almost all boundaries between different regions, since superpixels significantly reduce computational cost and allow feature extraction to be conducted from a larger homogeneous region. The proposed correlation clustering merges superpixels into disjoint homogeneous regions over a superpixel graph. 2.1 Pairwise correlation clustering over pairwise superpixel graph Define a pairwise undirected graph G = (V, E) where a node corresponds to a superpixel and a link between adjacent superpixels corresponds to an edge (see Figure 1.(a)). A binary label yjk for an edge (j, k) ∈ E between nodes j and k is defined such that { 1, if nodes j and k belong to the same region, yjk = (1) 0, otherwise. A discriminant function, which is the negative energy function, is defined over an image x and label y of all edges as ∑ Simw (x, j, k)yjk F (x, y; w) = (j,k)∈E = ∑ ⟨w, ϕjk (x)⟩yjk = ⟨w, (j,k)∈E ∑ ϕjk (x)yjk ⟩ = ⟨w, Φ(x, y)⟩ (2) (j,k)∈E where the similarity measure between nodes j and k, Simw (x, j, k), is parameterized by w and takes values of both signs such that a large positive value means strong similarity while a large negative value means high degree of dissimilarity. Note that the discriminant function F (x, y; w) is assumed to be linear in both the parameter vector w and the joint feature map Φ(x, y), and ϕjk (x) is a pairwise feature vector which reflects the correspondence between the jth and the kth superpixels. An image segmentation is to infer the edge label, ŷ, over the pairwise superpixel graph G by maximizing F such that ŷ = argmax F (x, y; w) (3) y∈Y E where Y is the set of {0, 1} that corresponds to a valid segmentation, the so called multicut polytope. However, solving (3) with this Y is generally NP-hard. Therefore, we approximate Y by means of a common multicut LP relaxation [18] with the following two constraints: (1) cycle inequality and (2) odd-wheel inequality. When producing the segmentation results based on the approximated LP solutions, we take the floor of a fractionally-predicted label of each edge independently for simply obtaining valid integer solutions that may be sub-optimal. Even though this pairwise correlation clustering takes a rich pairwise feature vector and a trained parameter vector (which will be presented later), it often produces incorrectly predicted segments due to the segment boundary ambiguities caused by limited pairwise relations of neighboring superpixels (see Figure 2). Therefore, to incorporate higher-order relations, we develop higher-order correlation clustering by generalizing the correlation clustering over a hypergraph. 2.2 Higher-order correlation clustering over hypergraph The proposed higher-order correlation clustering is defined over a hypergraph in which an edge called hyperedge can connect to two or more nodes. For example, as shown in Figure 1.(b), one 3 (a) (b) (c) (d) Figure 2: Example of segmentation result by pairwise correlation clustering. (a) Original image. (b) Ground-truth. (c) Superpixels. (d) Segments obtained by pairwise correlation clustering. can introduce binary labels for each adjacent vertices forming a triplet such that yijk = 1 if all vertices in the triplet ({i, j, k}) are in the same cluster; otherwise, yijk = 0. Define a hypergraph HG ∪ = (V, E) where V is a set of nodes (superpixels) and E is a set of hyperedges (subsets of V) such that e∈E = V. Here, a hyperedge e has at least two nodes, i.e. |e| ≥ 2. Therefore, the hyperedge set E can be divided into two disjoint subsets: pairwise ∪ edge set Ep = {e ∈ E | |e| = 2} and higherorder edge set Eh = {e ∈ E | |e| > 2} such that Ep Eh = E. Note that in the proposed hypergraph for higher-order correlation clustering all hyperedges containing just two nodes (∀ep ∈ Ep ) are linked between adjacent superpixels. The pairwise superpixel graph is a special hypergraph where all hyperedges contain just two (neighboring) superpixels: Ep = E. A binary label ye for a hyperedge e ∈ E is defined such that { 1, if all nodes in e belong to the same region, (4) ye = 0, otherwise. Similar to the pairwise correlation clustering, a linear discriminant function is defined over an image x and label y of all hyperedges as ∑ F (x, y; w) = Homw (x, e)ye e∈E ∑ ∑ ∑ = ⟨w, ϕe (x)⟩ye = ⟨wp , ϕep (x)⟩yep+ ⟨wh , ϕeh (x)⟩yeh =⟨w, Φ(x, y)⟩ (5) e∈E ep ∈Ep eh ∈Eh where the homogeneity measure among nodes in e, Homw (x, e), is also the inner product of the parameter vector w and the feature vector ϕe (x) and takes values of both signs such that a large positive value means strong homogeneity while a large negative value means high degree of nonhomogeneity. Note that the proposed discriminant function for higher-order correlation clustering is decomposed into two terms by assigning different parameter vectors to the pairwise edge set Ep and the higher-order edge set Eh such that w = [wpT , whT ]T . Thus, in addition to the pairwise similarity between neighboring superpixels, the proposed higher-order correlation clustering considers a broad homogeneous region reflecting higher-order relations among superpixels. Now the problem is how to build our hypergraph from a given image. Here, we use unsupervised multiple partitionings (quantizations) from baseline superpixels. We obtain unsupervised multiple partitionings by merging not pixels but superpixels with different image quantizations using the ultrametric contour maps [19]. For example, in Figure 3, there are three region layers, one superpixel (pairwise) layer and two higher-order layers, from which a hypergraph is constructed by defining hyperedges as follows: first, all edges (black line) in the pairwise superpixel graph from the first layer are incorporated into the pairwise edge set Ep , while hyperedges (yellow line) corresponding to regions (groups of superpixels) in the second and third layers are included in the higher-order edge set Eh . Note that we can further decompose the higher-order term in (5) into two terms associated with the second layer and the third layer, respectively, by assigning different parameter vectors; however for simplicity, this paper aggregates all higher-order edges from all higher-order layers into a single higher-order edge set assigning the same parameter vector. 2.2.1 LP relaxation for inference An image segmentation is to infer the hyperedge label, ŷ, over the hypergraph HG by maximizing the discriminant function F such that ŷ = argmax F (x, y; w) y∈Y 4 (6) Higher-order layer Higher-order layer Superpixel (pairwise) layer (a) Superpixel (pairwise) layer (b) (c) Figure 3: Hypergraph construction from multiple partitionings. (a) Multiple partitionings from baseline superpixels. (b) Hyperedge (yellow line) corresponding to a region in the second layer. (c) Hyperedge (yellow line) corresponding to a region in the third layer. where Y is also the set of {0, 1}E that corresponds to a valid segmentation. Since the inference problem (6) is also NP-hard, we relax Y by (facet-defining) linear inequalities. In addition to the constraints placed on pairwise labels such that the cycle inequality and odd-wheel inequality hold pairwise correlation clustering, we augment the constraints for labels on the higher-order edges, called higher-order inequalities, for a valid segmentation; there is no all-one pairwise labels in a region for which the higher-order edge is labeled as zero (non-homogeneous region), and when a region is labeled as one (homogeneous region), all pairwise labels in that region should be one. These higher-order inequalities can be formulated as yeh ≤ yep , ∀ep ∈ Ep |ep ⊂ eh , ∑ (1 − yeh ) ≤ (1 − yep ). (7) ep ∈Ep |ep ⊂eh Indeed, the LP relaxation to approximately solve (6) is formulated as ∑ ∑ argmax ⟨wp , ϕep (x)⟩yep + ⟨wh , ϕeh (x)⟩yeh y s.t. ep ∈Ep ∀ e ∈ E(= Ep ∪ (8) eh ∈Eh Eh ), ye ∈ [0, 1], ∀ ep ∈ Ep , cycle inequalities, odd-wheel inequalities [18], ∀ eh ∈ Eh , higher-order inequalities (7). Note that the proposed higher-order correlation clustering follows the concept of soft constraints: superpixels within a hyperedge are encouraged to merge if a hyperedge is highly homogeneous. 2.2.2 Feature vector We construct a 771-dimensional feature vector ϕe (x) by concatenating several visual cues with different quantization levels and thresholds. The pairwise feature vector ϕep (x) reflects the correspondence between neighboring superpixels, and the higher-order feature vector ϕeh (x) characterizes a more complex relations among superpixels in a broader region to measure homogeneity. The magnitude of w determines the importance of each feature, and this importance is task-dependent. Thus, w is estimated by supervised training described in Section 3. 1. Pairwise feature vector (611-dim): ϕep = [ϕcep ; ϕtep ; ϕsep ; ϕeep ; ϕvep ; 1]. • Color difference ϕc : The 26 RGB/HSV color distances (absolute differences, χ2 distances, earth mover’s distances) between two adjacent superpixels. 5 • Texture difference ϕt : The 64 texture distances (absolute differences, χ2 -distances, earth mover’s distances) between two adjacent superpixels using 15 Leung-Malik (LM) filter banks [19]. • Shape/location difference ϕs : The 5-dimensional shape/location feature proposed in [20]. • Edge strength ϕe : The 1-of-15 coding of the quantized edge strength proposed in [19]. • Joint visual word posterior ϕv : The 100-dimensional vector holding the joint visual word posteriors for a pair of neighboring superpixels using 10 visual words and the 400-dimensional vector holding the joint posteriors based on 20 visual words [21]. e tm 2. Higher-order feature vector (160-dim): ϕeh = [ϕva eh ; ϕeh ; ϕeh ; 1]. • Variance ϕva : The 14 color variances and 30 texture variances among superpixels in a hyperedge. • Edge strength ϕe : The 1-of-15 coding of the quantized edge strength proposed in [19]. • Template matching score ϕtm : The color/texture and shape/location features of all regions in the training images are clustered using k-means with k = 100 to obtain 100 representative templates of distinct regions. The 100-dimensional template matching feature vector is composed of the matching scores between a region defined by a hyperedge and templates using the Gaussian RBF kernel. In each feature vector, the bias (=1) is augmented for proper similarity/homogeneity measure which can either be positive or negative. 3 Structural learning The proposed discriminant function is defined over the superpixel graph, and therefore, the groundtruth segmentation needs to be transformed to the ground-truth edge labels in the superpixel graph. For this, we first assign a single dominant segment label to each superpixel by majority voting over the superpixel’s constituent pixels and then obtain the ground-truth edge labels. Using this ground-truth edge labels of the training data, the S-SVM [9] is performed to estimate the n parameter vector. Given N training samples {xn , yn }N n=1 where y is the ground-truth edge labels for the nth training image, the S-SVM [9] optimizes w by minimizing a quadratic objective function subject to a set of linear margin constraints: min w,ξ s.t. N ∑ 1 ∥w∥2 + C ξn 2 n=1 (9) ⟨w, ∆Φ(xn , y)⟩ ≥ ∆(yn , y) − ξn , ∀n, y ∈ Y\yn , ξn ≥ 0, ∀n where ∆Φ(xn , y) = Φ(xn , yn ) − Φ(xn , y), and C > 0 is a constant that controls the trade-off between margin maximization and training error minimization. In the S-SVM, the margin is scaled with a loss ∆(yn , y), which is the difference measure between prediction y and ground-truth label yn of the nth image. The S-SVM offers good generalization ability as well as the flexibility to choose any loss function [9]. The cutting plane algorithm [9, 18] with LP relaxation for loss-augmented inference is used to solve the optimization problem of S-SVM, since fast convergence and high robustness of the cutting plane algorithm in handling a large number of margin constraints are well-known [22, 23]. A loss function is usually a non-negative function, and a loss function that is decomposable is preferred, since it enables the loss-augmented inference in the cutting plane algorithm to be performed efficiently. The most popular loss function that is decomposable is the Hamming distance which is equivalent to the number of mismatches between yn and y at the edge level in this correlation clustering. Unfortunately, in the proposed correlation clustering for image segmentation, the number of edges which are labeled as 1 is considerably higher than that of edges which are labeled as 0. This unbalance makes other learning methods such as the perceptron algorithm inappropriate, since it leads to the clustering of the whole image as one segment. This problem due to the unbalance also 6 PRI VOI SCO BDE 0.84 11 0.6 3 0.82 10 0.5 0.8 2.5 9 0.78 0.4 2 0.76 20 25 30 35 Average number of regions Multi -NCut 40 8 20 25 30 35 Average number of regions gPb-owt -ucm 40 0.3 gPb-Hoiem 20 25 30 35 Average number of regions 40 Corr -Cluster -Pairwise 20 25 30 35 Average number of regions 40 Corr -Cluster -Higher Figure 4: Obtained evaluation measures from segmentation results on the SBD. occurs when we use the Hamming loss in the S-SVM. Therefore, we use the following loss function: ) ) ∑( ∑( ∆(yn , y)= Rp yenp +yep − (Rp + 1)yenp yep +D Rh yenh +yeh − (Rh + 1)yenh yeh (10) ep ∈Ep eh ∈Eh where D is the relative weight of the loss at higher-order edge level to that of the loss at pairwise edge level. In addition, Rp and Rh control the relative importance between the incorrect merging of the superpixels and the incorrect separation of the superpixels by imposing different weights to the false negative and the false positive. Here, we set both Rp and Rh to be less than 1 to overcome the problem due to the unbalance. 4 Experiments To evaluate segmentations obtained by various algorithms against the ground-truth segmentation, we conducted image segmentations on three benchmark datasets: Stanford background dataset [24] (SBD), Berkeley segmentation dataset (BSDS) [25], MSRC dataset [26]. For image segmentation based on correlation clustering, we initially obtain baseline superpixels (438 superpixels per image on average) by the gPb contour detector and the oriented watershed transform [19] and then construct a hypergraph. The function parameters are initially set to zero, and then based on the S-SVM, the structured output learning is used to estimate the parameter vectors. Note that the relaxed solutions in loss-augmented inference are used during training, while in testing, our simple rounding method is used to produce valid segmentation results. Rounding is only necessary in case we obtain fractional solutions from LP-relaxed correlation clustering. We compared the proposed pairwise/higher-order correlation clustering to the following state-of-theart image segmentation algorithms: multiscale NCut [27], gPb-owt-ucm [19], and gPb-Hoiem [20] that grouped the same superpixels based on pairwise same-label likelihoods. The pairwise samelabel likelihoods were independently learnt from the training data with the same 611-dimensional pairwise feature vector. We consider four performance measures: probabilistic Rand index (PRI) [28], variation of information (VOI) [29], segmentation covering (SCO) [19], and boundary displacement error (BDE) [30]. As the predicted segmentation is close to the ground-truth segmentation, the PRI and SCO are increased while the VOI and BDE are decreased. 4.1 Stanford background dataset The SBD consists of 715 outdoor images with corresponding pixel-wise annotations. We employed 5-fold cross-validation with the dataset randomly split into 572 training images and 143 test images for each fold. Figure 4 shows the obtained four measures from segmentation results according to the average number of regions. Note that the performance varies with different numbers of regions, and for this reason, we designed each algorithm to produce multiple segmentations (20 to 40 regions). Specifically, multiple segmentations in the proposed algorithm were obtained by varying Rp (0.001∼0.2) and Rh (0.1∼1.0) in the loss function during training (D=10). Irrespective of the measure, the proposed higher-order correlation clustering (Corr-Cluster-Higher) performed better than other algorithms including the pairwise correlation clustering (Corr-Cluster-Pairwise). Figure 5 shows some example segmentations. The proposed higher-order correlation clustering yielded the best segmentation results. In specific, incorrectly predicted segments by pairwise correlation clustering were reduced in the segmentation results obtained by higher-order correlation clustering 7 Original image Ground-truth Multi -NCut gPb-owt-ucm gPb-Hoiem Corr-Cluster Pairwise Corr-Cluster Higher Figure 5: Results of image segmentation. Table 1: Quantitative results on the BSDS test set and on the MSRC test set. Test set Multi-NCut gPb-owt-ucm gPb-Hoiem Corr-Cluster-Pairwise Corr-Cluster-Higher PRI 0.728 0.794 0.724 0.806 0.814 BSDS VOI SCO 3.043 0.315 1.909 0.571 3.194 0.316 1.829 0.585 1.743 0.599 BDE 14.257 11.461 14.795 11.194 10.377 PRI 0.628 0.779 0.614 0.773 0.784 MSRC VOI SCO 2.765 0.341 1.675 0.628 2.847 0.353 1.648 0.632 1.594 0.648 BDE 11.941 9.800 13.533 9.194 9.040 owing to the consideration of higher-order relations in broad regions. Regarding the runtime of our algorithm, we observed that for test-time inference it took on average around 15 seconds (graph construction and feature extraction: 14s, LP: 1s) per image on a 2.67GHz processor, whereas the overall training took 10 hours on the training set. Note that other segmentation algorithms such as the multiscale NCut and the gPb-owt-ucm took on average a few minutes per image. 4.2 Berkeley segmentation dataset and MSRC dataset The BSDS contains 300 natural images split into the 200 training images and 100 test images. Since each image is segmented by multiple human subjects, we defined a single probabilistic (real-valued) ground-truth segmentation of each image for training in the proposed correlation clustering. The MSRC dataset is composed of 591 natural images. We split the data into 45% training, 10% validation, and 45% test sets, following [26]. The performance was evaluated using the clean ground-truth object instance labeling of [31]. On average, all segmentation algorithms were set to produce 30 disjoint regions per image on the BSDS and 15 disjoint regions per image on the MSRC dataset. As shown in Table 1, the proposed higher-order correlation clustering gave the best results on both datasets. Especially, the obtained results on the BSDS are similar or even better than the best results ever reported on the BSDS [32, 19]. 5 Conclusion This paper proposed the higher-order correlation clustering over a hypergraph to merge superpixels into homogeneous regions. The LP relaxation was used to approximately solve the higher-order correlation clustering over a hypergraph where a rich feature vector was defined based on several visual cues involving higher-order relations among superpixels. The S-SVM was used for supervised training of parameters in correlation clustering, and the cutting plane algorithm with LP-relaxed inference was applied to solve the optimization problem of S-SVM. Experimental results showed that the proposed higher-order correlation clustering outperformed other image segmentation algorithms on various datasets. The proposed framework is applicable to a variety of other areas. Acknowledgments This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MEST) (No.2011-0018249). 8 References [1] T. Kanungo, D. Mount, N. Netanyahu, C. Piatko, R. Silverman, and A. Wu, “An efficient k-means clustering algorithm: Analysis and implementation,” PAMI, vol. 24, pp. 881–892, 2002. [2] D. Comaniciu and P. Meer, “Mean shift: A robust approach toward feature space analysis,” PAMI, vol. 24, pp. 603–619, 2002. [3] C. Carson, S. Belongie, H. Greenspan, and J. Malik, “Blobworld: image segmentation using expectationmaximization and its application to image querying,” PAMI, vol. 24, pp. 1026–1038, 2002. [4] F. Estrada and A. Jepson, “Spectral embedding and mincut for image segmentation,” in BMVC, 2004. [5] J. Shi and J. Malik, “Normalized cuts and image segmentation,” PAMI, vol. 22, pp. 888–905, 2000. [6] P. Felzenszwalb and D. Huttenlocher, “Efficient graph-based image segmentation,” IJCV, vol. 59, pp. 167–181, 2004. [7] F. Estrada and A. Jepson, “Benchmarking image segmentation algorithms,” IJCV, vol. 85, 2009. [8] N. Bansal, A. Blum, and S. Chawla, “Correlation clustering,” Machine Learning, vol. 56, 2004. [9] I. Tsochantaridis, T. Joachims, T. Hofmann, and Y. Altun, “Large margin methods for structured and independent output variables,” JMLR, vol. 6, 2005. [10] T. Finley and T. Joachims, “Supervised clustering with support vector machines,” in ICML, 2005. [11] B. Taskar, “Learning structured prediction models: a large margin approach,” Ph.D. thesis, Stanford University, 2004. [12] C. Berge, Hypergraphs, North-Holland, Amsterdam, 1989. [13] L. Ding and A. Yilmaz, “Image segmentation as learning on hypergraphs,” in Proc. ICMAL, 2008. [14] S. Rital, “Hypergraph cuts and unsupervised representation for image segmentation,” Fundamenta Informaticae, vol. 96, pp. 153–179, 2009. [15] A. Ducournau, S. Rital, A. Bretto, and B. Laget, “A multilevel spectral hypergraph partitioning approach for color image segmentation,” in Proc. ICSIPA, 2009. [16] F. Bach and M. I. Jordan, “Learning spectral clustering,” in NIPS, 2003. [17] T. Cour, N. Gogin, and J. Shi, “Learning spectral graph segmentation,” in AISTATS, 2005. [18] S. Nowozin and S. Jegelka, “Solution stability in linear programming relaxations: Graph partitioning and unsupervised learning,” in ICML, 2009. [19] P. Arbeláez, M. Maire, C. Fowlkes, and J. Malik, “Contour detection and hierarchical image segmentation,” PAMI, vol. 33, pp. 898–916, 2011. [20] D. Hoiem, A. A. Efros, and M. Hebert, “Recovering surface layout from an image,” IJCV, 2007. [21] D. Batra, R. Sukthankar, and T. Chen, “Learning class-specific affinities for image labelling,” in CVPR, 2008. [22] T. Finley and T. Joachims, “Training structural SVMs when exact inference is intractable,” in ICML, 2008. [23] A. Kulesza and F. Pereira, “Structured learning with approximate inference,” in NIPS, 2007. [24] S. Gould, R. Fulton, and D. Koller, “Decomposing a scene into geometric and semantically consistent regions,” in ICCV, 2009. [25] C. Fowlkes, D. Martin, and J. Malik, The Berkeley Segmentation Dataset and Benchmark (BSDB), http://www.cs.berkeley.edu/projects/vision/grouping/segbench/. [26] J. Shotton, J. Winn, C. Rother, and A. Criminisi, “Textonboost: joint apprearence, shape and context modeling for multi-class object recognition and segmentation,” in ECCV, 2006. [27] T. Cour, F. Benezit, and J. Shi, “Spectral segmentation with multiscale graph decomposition,” in CVPR, 2005. [28] W. M. Rand, “Objective criteria for the evaluation of clustering methods,” Journal of the American Statistical Association, vol. 66, pp. 846–850, 1971. [29] M. Meila, “Computing clusterings: An axiomatic view,” in ICML, 2005. [30] J. Freixenet, X. Munoz, D. Raba, J. Marti, and X. Cufi, “Yet another survey on image segmentation: Region and boundary information integration,” in ECCV, 2002. [31] T. Malisiewicz and A. A. Efros, “Improving spatial support for objects via multiple segmentations,” in BMVC, 2007. [32] T. Kim, K. Lee, and S. Lee, “Learning full pairwise affinities for spectral segmentation,” in CVPR, 2010. 9