Multiplicative Bounds for Metric Labeling M. Pawan Kumar École Centrale Paris Joint work with Phil Torr, Daphne Koller Metric Labeling Variables V = { V1, V2, …, Vn} Metric Labeling Variables V = { V1, V2, …, Vn} Metric Labeling wabd(f(a),f(b)) θb(f(b)) wab ≥ 0 θa(f(a)) d is metric Va minf E(f) Vb = Σa θa(f(a)) + Σ(a,b) wabd(f(a),f(b)) Variables V = { V1, V2, …, Vn} Labels L = { l1, l2, …, lh} Labeling f: { 1, 2, …, n} {1, 2, …, h} Metric Labeling Va minf E(f) Vb = Σa θa(f(a)) + Σ(a,b) wabd(f(a),f(b)) NP hard Low-level vision applications Approximate Algorithms Minka. Expectation Propagation for Approximate Bayesian Inference, UAI, 2001 Murphy et al. Loopy Belief Propagation: An Empirical Study, UAI, 1999 Winn et al. Variational Message Passing, JMLR, 2005 Yedidia et al. Generalized Belief Propagation, NIPS, 2001 Besag. On the Statistical Analysis of Dirty Pictures, JRSS, 1986 Boykov et al. Fast Approximate Energy Minimization via Graph Cuts, PAMI, 2001 Komodakis et al. Fast, Approximately Optimal Solutions for Single and Dynamic MRFs, CVPR, 2007 Lempitsky et al. Fusion Moves for Markov Random Field Optimization, PAMI, 2010 Chekuri et al. Approximation Algorithms for Metric Labeling, SODA, 2001 Goemans et al. Improved Approximate Algorithms for Maximum-Cut, JACM, 1995 Muramatsu et al. A New SOCP Relaxation for Max-Cut, JORJ, 2003 Ravikumar et al. QP Relaxations for Metric Labeling, ICML, 2006 Alahari et al. Dynamic Hybrid Algorithms for MAP Inference, PAMI 2010 Kohli et al. On Partial Optimality in Multilabel MRFs, ICML, 2008 Rother et al. Optimizing Binary MRFs via Extended Roof Duality, CVPR, 2007 . . . Outline • Linear Programming Relaxation • Move-Making Algorithms • Comparison • Rounding-based Moves Integer Linear Program Minimize a linear function over a set of feasible solutions Indicator xa(i) {0,1} for each variable Va and label li Number of facets grows exponentially in problem size Linear Programming Relaxation Indicator xa(i) {0,1} for each variable Va and label li Schlesinger, 1976; Chekuri et al., 2001; Wainwright et al., 2003 Linear Programming Relaxation Indicator xa(i) [0,1] for each variable Va and label li Schlesinger, 1976; Chekuri et al., 2001; Wainwright et al., 2003 Outline • Linear Programming Relaxation • Move-Making Algorithms • Comparison • Rounding-based Moves Move-Making Algorithms Space of All Labelings f Expansion Algorithm Variables take label lα or retain current label Slide courtesy Pushmeet Kohli Boykov, Veksler and Zabih, 2001 Expansion Algorithm Variables take label lα or retain current label Status: InitializeSky Expand Ground House with Tree Slide courtesy Pushmeet Kohli Tree Ground House Sky Boykov, Veksler and Zabih, 2001 Outline • Linear Programming Relaxation • Move-Making Algorithms • Comparison • Rounding-based Moves Multiplicative Bounds f*: Optimal Labeling f: Estimated Labeling Σa θa(f(a)) + Σ(a,b) sabd(f(a),f(b)) ≥ Σa θa(f*(a)) + Σ(a,b) sabd(f*(a),f*(b)) Multiplicative Bounds f*: Optimal Labeling f: Estimated Labeling Σa θa(f(a)) + Σ(a,b) sabd(f(a),f(b)) ≤ Σa θa(f*(a)) + B Σ(a,b) sabd(f*(a),f*(b)) Multiplicative Bounds Expansion LP Potts 2 2 Truncated Linear 2M 2 + √2 Truncated Quadratic 2M O(√M) Metric 2M O(log h) M = ratio of maximum and minimum distance Outline • Linear Programming Relaxation • Move-Making Algorithms • Comparison • Rounding-based Moves – Complete Rounding – Interval Rounding – Hierarchical Rounding Complete Rounding Treat xa(i) [0,1] as probability that f(a) = i Cumulative probability ya(i) = Σj≤i xa(j) r 0 ya(1) ya(2) ya(i) Generate a random number r (0,1] Assign the label next to r ya(k) ya(h) = 1 Complete Move Va Vb θab(i,k) = sabd(i,k) NP-hard Complete Move Va Vb d’(i,k) ≥ d(i,k) θab(i,k) = sabd’(i,k) d’ is submodular Complete Move Va Vb d’(i,k) ≥ d(i,k) θab(i,k) = sabd’(i,k) d’ is submodular Complete Move New problem can be solved using minimum cut Same multiplicative bound as complete rounding Multiplicative bound is tight Outline • Linear Programming Relaxation • Move-Making Algorithms • Comparison • Rounding-based Moves – Complete Rounding – Interval Rounding – Hierarchical Rounding Interval Rounding Treat xa(i) [0,1] as probability that f(a) = i Cumulative probability ya(i) = Σj≤i xa(j) 0 ya(1) ya(2) ya(i) Choose an interval of length h’ ya(k) ya(h) = 1 Interval Rounding Treat xa(i) [0,1] as probability that f(a) = i Cumulative probability ya(i) = Σj≤i xa(j) r ya(i) Choose an interval of length h’ ya(k) REPEAT Generate a random number r (0,1] Assign the label next to r if it is within the interval Interval Move Choose an interval of length h’ Va Vb θab(i,k) = sabd(i,k) Interval Move Choose an interval of length h’ Add the current labels Va Vb θab(i,k) = sabd(i,k) Interval Move Choose an interval of length h’ Add the current labels d’(i,k) ≥ d(i,k) d’ is submodular Solve to update labels Va Vb Repeat until convergence θab(i,k) = sabd’(i,k) Interval Move Each problem can be solved using minimum cut Same multiplicative bound as interval rounding Multiplicative bound is tight Kumar and Torr, NIPS 2008 Outline • Linear Programming Relaxation • Move-Making Algorithms • Comparison • Rounding-based Moves – Complete Rounding – Interval Rounding – Hierarchical Rounding Hierarchical Rounding L1 l1 l2 L2 l3 l4 l5 L3 l6 l7 l8 Hierarchical clustering of labels (e.g. r-HST metrics) l9 Hierarchical Rounding L1 l1 l2 L2 l3 l4 l5 L3 l6 l7 Assign variables to labels L1, L2 or L3 Move down the hierarchy until the leaf level l8 l9 Hierarchical Rounding L1 l1 l2 L2 l3 l4 l5 Assign variables to labels l1, l2 or l3 L3 l6 l7 l8 l9 Hierarchical Rounding L1 l1 l2 L2 l3 l4 l5 Assign variables to labels l4, l5 or l6 L3 l6 l7 l8 l9 Hierarchical Rounding L1 l1 l2 L2 l3 l4 l5 Assign variables to labels l7, l8 or l9 L3 l6 l7 l8 l9 Hierarchical Move L1 l1 l2 L2 l3 l4 l5 L3 l6 l7 l8 Hierarchical clustering of labels (e.g. r-HST metrics) l9 Hierarchical Move L1 l1 l2 L2 l3 l4 l5 L3 l6 l7 Obtain labeling f1 restricted to labels {l1,l2,l3} l8 l9 Hierarchical Move L1 l1 l2 L2 l3 l4 l5 L3 l6 l7 Obtain labeling f2 restricted to labels {l4,l5,l6} l8 l9 Hierarchical Move L1 l1 l2 L2 l3 l4 l5 L3 l6 l7 Obtain labeling f3 restricted to labels {l7,l8,l9} l8 l9 Hierarchical Move L1 L2 L3 f3(a) f3(b) f2(a) f2(b) f1(a) f1(b) Va Vb Move up the hierarchy until we reach the root Hierarchical Move Each problem can be solved using minimum cut Same multiplicative bound as hierarchical rounding Multiplicative bound is tight Kumar and Koller, UAI 2009 Conclusion MoveMaking LP Potts 2 2 Truncated Linear 2 + √2 2 + √2 Truncated Quadratic O(√M) O(√M) Metric O(log h) O(log h) M = ratio of maximum and minimum distance Open Problems • Moves for general rounding schemes • Higher-order energy functions • Better comparison criterion Questions? http://www.centrale-ponts.fr/personnel/pawan