A Polynomial-Time Cutting-Plane Algorithm for Matchings Karthekeyan Chandrasekaran Harvard University Joint work with L. Vegh and S. Vempala Cutting Plane Method π₯0 π₯2 π π₯3 π₯1 π₯4 π₯πππ‘ π ππΌ π = {π₯ ∈ π π : π΄π₯ ≤ π} ππΌ = conv−hull π ∩ π π Cutting Plane Method 1. Starting LP. Start with the LP relaxation of the given IP to obtain basic optimal solution π₯ 2. Repeat until π₯ is integral: a. Add Cuts. Find a linear inequality that is valid for the convex hull of integer solutions but violated by π₯ and add it to the LP b. Re-solve LP. Obtain basic optimal solution π₯ Cutting Plane Method ο§ Proposed by Dantzig-Fulkerson-Johnson 1954 ο§ Several cut-generation procedures ο Gomory cuts [Gomory 1958 ] ο Intersection cuts [Balas 1971 ] ο Disjunctive cuts [Balas 1979 ] ο Split cuts [Cook-Kannan-Schrijver (1990)] ο MIR Inequalities [Nemhauser-Wolsey (1990)] ο Lift-and-project methods [Sherali-Adams (1990), Lovász-Schrijver (1991), Balas-Ceria-Cornuéjols (1993)] ο§ Closure properties of polytopes [Chvátal (1973)] ο Chvatal-Gomory rank ο§ Cutting plane proof system [Chvátal-Cook-Hartmann (1989)] ο§ Implemented in commercial IP solvers Requirements – Efficient and Correct ο§ Efficient cut-generation procedure [Gomory ο§ Convergence to integral solution [Gomory 1958 ] 1958 ] ο§ Fast convergence (number of cuts to reach integral solution) ο§ Is the cutting plane method efficient for polynomial-time [Gomory 1958 ] οsolvable At most combinatorial 2π cuts for 0/1-IP problems? ο No hope for faster theoretical convergence using Gomory cuts ο§ Theoretical bound on the number of steps to convergence of ο Practical implementations seem to be efficient for certain cutting plane method for such problems? problems Question: Can we explain the efficiency in practical implementations? Minimum-Cost Perfect Matching ο§ Problem: Given an edge cost function in a graph, find a minimum cost perfect matching Edmonds ο§ IP formulation: Gabow , Lawler Galil-Micali-Gabow min π π’π£ π₯ π’π£ π’π£∈πΈ π₯ πΏ π’ = 1 ∀π’ ∈ π (degree constraints) π₯≥0 (non-negativity constraints) π₯ ∈ ππΈ Gabow Gabow-Tarjan π₯ πΏ π’ β π₯ π π:π=π’π£ LP Relaxation (Bipartite Relaxation) min π π’π£ π₯ π’π£ π’π£∈πΈ ο In bipartite graphs, the solution isπ₯ the a perfect matching πΏ π’indicator = 1 ∀π’vector ∈ π of (degree constraints) π₯≥0 (non-negativity constraints) in the graph π₯ ∈ ππΈ ο In non-bipartite graphs, the solution is not necessarily integral • Solution is half-integral and supported by a disjoint union of edges and odd cycles 1 2 1 2 1 1 2 1 2 π₯π = 12 1 2 1 2 1 2 1 1 Polyhedral Characterization [Edmonds 1965 ] min π’π£∈πΈ π π’π£ π₯ π’π£ π₯ πΏ π’ = 1 ∀π’ ∈ π π₯≥0 π₯ πΏ π ≥ 1 ∀π ⊆ π, π πππ π₯ πΏ π β (degree constraints) (non-negativity constraints) (odd-set inequalities) π₯ π π=π’π£:π’∈π, π£∈πβπ ο§ Odd-set inequalities ο Gomory cuts derived from bipartite relaxation [Chvátal 1973 ] ο§ Efficient separation oracle exists ο For any given point π₯, we can efficiently verify if π₯ satisfies all constraints and if not output a violated constraint [Padberg-Rao 1982 ] Cutting Plane Algorithm [Padberg − Rao 1982 ] Starting LP. Start with the bipartite relaxation to obtain basic optimal solution π₯ Repeat until π₯ is integral: 1. 2. a. b. Add Cuts. Find an odd-set inequality that is violated by π₯ using the Padberg-Rao procedure and add it to the LP Re-solve LP. Obtain basic optimal solution π₯ ο§ Easy to find cuts in the first step ο Starting LP optimum is half-integral and supported by a disjoint union of odd cycles and edges ο§ Half-integral structure is not preserved in later steps - need Padberg-Rao for cut generation 1 3 1 2 1 2 1 2 ππ = 1 ∀π 1 2 1 2 1 2 1 2 1 2 π 3 1 2 1 2 1 3 1 3 1 3 π 3 1 3 π 3 1 3 π 3 1 3 Cutting Plane Algorithm [Padberg − Rao 1982 ] ο§ Discussed by Lovász-Plummer 1986 ο§ Implementations by Grötschel-Holland 1985 , Trick 1987 , Fischetti-Lodi 2010 ο Computationally efficient Result: A cutting plane algorithm for minimum-cost perfect matching ο Convergence guarantee: π π log π rounds of cuts for an π-vertex graph ο Black-box LP solver after each round of cut addition ο Intermediate LP optima are half-integral Outline ο§ Cutting Plane Method for Integer Programs ο§ Perfect Matching and Linear Constraints ο§ Results and Techniques ο New Cutting Plane Algorithm for Matching ο Analysis Overview • Half-integral Structure • Progress ο§ Coupling with a new Combinatorial Primal-Dual Algorithm for Matching New Cutting Plane Algorithm Primal - ππΉ Min π’π£∈πΈ ππ’π£ π₯π’π£ π₯ πΏ π’ = 1 ∀π’ ∈ π π₯ πΏ π ≥ 1 ∀π ∈ πΉ π₯≥0 Dual - π·πΉ Max π∈π∪πΉ Π(π) Π(π) ≤ π(π’π£) ∀π’π£ ∈ πΈ π∈π∪πΉ:π’π£∈πΏ π 1 Π π ≥0∀π ∈ πΉ 1. Perturb the integral cost function by adding π to edge π 2 2. Starting LP. Bipartite relaxation (πΉ = ∅) 3. Repeat until π₯ is integral (a) Retain old cuts. Find a specific dual optimal solution Π to π·πΉ . π» ′ = π ∈ πΉ: Π π > 0 (b) Choose new cuts. (c) Re-solve LP. Find an optimal solution π₯ to ππΉ To ensure |Π πuniqueness −Πππππ£ π of | Min π∈πΉ intermediate optima |π| Final integral optimum to perturbed cost function is also optimum to original cost function An Example πΉ = {π1 , π2 , π3 } Π π1 > 0 x π =1 π» ′ = π1 , π3 x π =1 x π =1 x π =1 x π =1 2 Π π2 = 0 Π π3 > 0 New Cutting Plane Algorithm Primal - ππΉ Min π’π£∈πΈ ππ’π£ π₯π’π£ π₯ πΏ π’ = 1 ∀π’ ∈ π π₯ πΏ π ≥ 1 ∀π ∈ πΉ π₯≥0 Dual - π·πΉ Max π∈π∪πΉ Π(π) Π(π) ≤ π(π’π£) ∀π’π£ ∈ πΈ π∈π∪πΉ:π’π£∈πΏ π 1 Π π ≥0∀π ∈ πΉ 1. Perturb the integral cost function by adding π to edge π 2 2. Starting LP. Bipartite relaxation (πΉ = ∅) 3. Repeat until π₯ is integral (a) Retain old cuts. Find a specific dual optimal solution Π to π·πΉ . π» ′ = π ∈ πΉ: Π π > 0 (b) Choose new cuts. (c) Re-solve LP. Find an optimal solution π₯ to ππΉ An Example πΆ πΉ = {π1 , π2 , π3 } Π π1 > 0 x π =1 π» ′ = π1 , π3 x π =1 π» ′′ = πΆ x π =1 x π =1 πΆ x π =1 2 Π π2 = 0 Π π3 > 0 New Cutting Plane Algorithm Primal - ππΉ Min π’π£∈πΈ ππ’π£ π₯π’π£ π₯ πΏ π’ = 1 ∀π’ ∈ π π₯ πΏ π ≥ 1 ∀π ∈ πΉ π₯≥0 Dual - π·πΉ Max π∈π∪πΉ Π(π) Π(π) ≤ π(π’π£) ∀π’π£ ∈ πΈ π∈π∪πΉ:π’π£∈πΏ π Π π ≥0∀π ∈ πΉ 1 1. Perturb the integral cost function by adding π to edge π 2 2. Starting LP. Bipartite relaxation (πΉ = ∅) 3. Repeat until π₯ is integral (a) Retain old cuts. Find a specific dual optimal solution Π to π·πΉ . π» ′ = π ∈ πΉ: Π π > 0 (b) Choose new cuts. For each cycle πΆ ∈ supp π₯ , define πΆ as the union of π πΆ and the inclusionwise maximal sets of π»′ intersecting π πΆ π» ′′ = {πΆ: πΆ ∈ supp π₯ } (c) Set the new πΉ = π» ′ ∪ π»′′ (d) Re-solve LP. Find an optimal solution π₯ to ππΉ Analysis Overview ο§ Laminarity: Intermediate LPs are defined by a laminar family πΉ of odd sets π [⇒At most 2 odd-set inequalities in intermediate LPs] (i) Structural Guarantee: Intermediate LP optima are half-integral and supported by a disjoint union of odd cycles and edges [⇒Cut-generation in π(π) time] (ii) Progress: The number of odd cycles odd(π₯) in the support of the intermediate LP optima π₯ ο Non-increasing ο Decreases by one in at most π 2πππ π₯ rounds of cut addition [⇒Number of rounds of cut addition is π π log π ] Analysis Overview (ii) Cut Retention: If odd π₯ remains the same in iterations π, π + 1, … , π then all cuts (ii) Progress: Theπ,number cycles odd(π₯) in the added in iterations π + 1, …of, πodd are retained upto the π’thsupport iterationof the intermediate LP optima π₯ ο Non-increasing One cut for each cycle ⇒ In π’th iteration, number of cuts ≥ j − π odd π₯ π π ο Decreases in at most rounds of cutinequalities) addition in any iteration ≤ Laminarityby⇒one Maximum number of cuts (odd-set 2πππ π₯ 2 π [⇒Number of rounds of cut additionπ − is π π log π πππ π₯ π≤] 2 π π≤π+ 2πππ(π₯) Analysis Overview (i) Half-integral Structure: Intermediate LP optima are half-integral and supported by a disjoint union of odd cycles and edges (ii) Cut Retention: If odd π₯ remains the same in iterations π, π + 1, … , π then all cuts added in iterations π, π + 1, … , π are retained upto the π’th iteration Half-integral Structure ο§ Conjecture 0: All intermediate solutions are Half-integral ο§ Conjecture 1: Half-integral if odd-set inequalities correspond to a laminar family 1 3 1 2 1 2 1 2 ππ = 1 ∀π 1 2 1 2 1 2 1 2 1 2 π 3 1 2 1 2 Hey look – the optima are not unique! 1 3 1 3 1 3 π 3 1 3 π 3 1 3 π 3 1 3 Half-integral Structure ο§ Conjecture 2: Half-integral if the optimum is unique and odd-set inequalities correspond to a laminar family Half-integral Structure Primal - ππΉ Min π’π£∈πΈ ππ’π£ π₯π’π£ π₯ πΏ π’ = 1 ∀π’ ∈ π π₯ πΏ π ≥ 1 ∀π ∈ πΉ π₯≥0 Dual - π·πΉ Max π∈π∪πΉ Π(π) Π(π) ≤ π(π’π£) ∀π’π£ ∈ πΈ π∈π∪πΉ:π’π£∈πΏ π Π π ≥0∀π ∈ πΉ ο§ Lemma: If π₯ is unique, πΉ is laminar, and π·πΉ has an F-critical dual optimal solution, then π₯ is half-integral ο§ Definition. Π is a F-critical dual solution to π·πΉ if ∀π ∈ πΉ: Π(π) > 0, the induced graph over π using the tight edges wrt Π is πΉ-factor-critical π1 ο For every π’ ∈ π, there exists a perfect matching ππ’ ο§ ο§ ππ’ covers all vertices in π β π’ ππ’ ∩ πΏ π ≤ 1 ∀ π ∈ πΉ, π ⊆ π π π2 Half-integral Structure Say we have a primal-dual optimal pair π₯, Π such that Π is F-critical. π’ Π π£ 1/4 π£ 1/4 Π π π 1/4 1/4 1/2 Π π >0 Tight Edge 1/2 π₯′′ π₯ π = 1 3 π₯ π = 2 3 • Projection of primal-dual optimal pair gives primal-dual optimal pair in the contracted instance • The image π₯′ of π₯ in the contracted graph is the unique optimum in the contracted instance ο Extend π₯′′ to an optimal solution π¦ in the original graph satisfying all constraints ο Since π₯ is unique, π¦ = π₯ Half-integral Structure Say we have a primal-dual optimal pair π₯, Π such that Π is F-critical. 1/4 π π’ π£Π π£ 1/4 1/4 1/2 1/4 +1/2 1/41/4 Π π 1/4 1/2 1/4 1/4 1/4 1/4 1/2 1/2 1/4 1/4 1/4+1/2 π₯′′ • Projection of primal-dual optimal pair gives primal-dual optimal pair in the contracted instance • The image π₯′ of π₯ in the contracted graph is the unique optimum in the contracted instance ο isExtend π₯′′ tothen an optimal • If π₯’ half-integral so is π₯ solution π¦ in the original graph satisfying all constraints ο we Since π₯ is unique, = Ππ₯ π > 0, the image π₯′ is the unique optimal solution to the • Once contract all π ∈π¦ πΉ: bipartite relaxation ⇒ π₯′ is half-integral New Cutting Plane Algorithm Primal - ππΉ Min π’π£∈πΈ ππ’π£ π₯π’π£ π₯ πΏ π’ = 1 ∀π’ ∈ π π₯ πΏ π ≥ 1 ∀π ∈ πΉ π₯≥0 Dual - π·πΉ Max π∈π∪πΉ Π(π) Π(π) ≤ π(π’π£) ∀π’π£ ∈ πΈ π∈π∪πΉ:π’π£∈πΏ π Π π ≥0∀π ∈ πΉ 1 1. Perturb the integral cost function by adding π to edge π 2 2. Starting LP. Bipartite relaxation (πΉ = ∅) 3. Repeat until π₯ is integral (a) Retain old cuts. Find a specific dual optimal solution Π to π·πΉ . π» ′ = π ∈ πΉ: Π π > 0 (b) Choose new cuts. For each cycle πΆ ∈ supp π₯ , define πΆ as the union of π πΆ and the inclusionwise maximal sets of π»′ intersecting π πΆ π» ′′ = {πΆ: πΆ ∈ supp π₯ } (c) Set the new πΉ = π» ′ ∪ π»′′ (d) Re-solve LP. Find an optimal solution π₯ to ππΉ Analysis Overview (i) Half-integral Structure: Intermediate LP optima are half-integral and supported by a disjoint union of odd cycles and edges (ii) Cut Retention: If odd π₯ remains the same in iterations π, π + 1, … , π then all cuts added in iterations π, π + 1, … , π are retained upto the π’th iteration Proof of Cut Retention: Coupling with the intermediate solutions of a Half-Integral Primal-Dual Algorithm for matching ο The choice of specific dual optimal solution to retain cuts comes from this coupling Edmonds’ Algorithm (Bipartite) Primal - ππΉ Max π’π£∈πΈ ππ’π£ π₯π’π£ π₯ πΏ π’ = 1 ∀π’ ∈ π π₯ πΏ π ≥ 1 ∀π ∈ πͺ π₯≥0 Dual - π·πΉ Min π∈π∪πͺ Π(π) Π(π) ≤ π(π’π£) ∀π’π£ ∈ πΈ π∈π∪πͺ:π’π£∈πΏ π Π π ≥0∀π ∈ πͺ + • Maintain a dual feasible solution and a primal infeasible solution + + • Primal solution is on tight edges (dual constraint is tight) + + + + + + + + + • Dual solution uses only sets from some laminar family πΉ • Build alternating trees from exposed vertices using tight edges − − • If there is a primal-augmenting path (alternating path between two exposed vertices), then augment the primal • If not, modify the dual - Increase dual values on nodes at even distance from exposed nodes and decrease on nodes at odd distance simultaneously while maintaining dual feasibility • New edges will become tight wrt dual • Proof of optimality: Complementary slackness • Proof of poly-time: either support of primal solution or alternating tree increases in size + + + + Edmonds’ Algorithm (Non-bipartite) Primal - ππΉ Max π’π£∈πΈ ππ’π£ π₯π’π£ π₯ πΏ π’ = 1 ∀π’ ∈ π π₯ πΏ π ≥ 1 ∀π ∈ πͺ π₯≥0 π Dual - π·πΉ Min π∈π∪πͺ Π(π) Π(π) ≤ π(π’π£) ∀π’π£ ∈ πΈ π∈π∪πͺ:π’π£∈πΏ π Blossom Π π ≥0∀π ∈ πͺ π ο§ Alternating path starting and ending at the same exposed vertex ο§ If such a path exists, then ο contract the blossom ο add the vertex set π of the blossom to πΉ with Π π = 0 ο§ While modifying dual: If Π π decreases to zero, then deshrink the node π, remove π from πΉ and continue Edmonds’ Algorithm (Non-bipartite) ο§ Proof of optimality: Complementary slackness ο§ Proof of poly-time: ο If a set S is shrunk, then it is never deshrunk before the next augmentation ο Number of steps between any two primal-augmentation is π(π) • Between primal-augmentations, the algorithm can - Either grow alternating tree - Or shrink a blossom - Or deshrink a blossom Half-Integral Primal-Dual Algorithm Edmonds’ Algorithm Half-Integral Algorithm ο Intermediate Primal – integral ο Intermediate Primal – half-integral ο Unique way to augment primal ο 3 ways to augment primal ο Build an alternating tree using tight edges and ο Build an alternating tree using tight edges and repeatedly attempt to augment primal and change dual values until there are no more exposed nodes repeatedly attempt to augment primal and change dual values until there are no more exposed nodes ο Deshrink if dual value on a set decreases to ο Deshrink if dual value on a set decreases to zero ο Shrink if a set forms a blossom zero ο Augment primal if a set forms a blossom Half-Integral Primal-Dual Algorithm Primal Augmentations 1 2 1 2 1 2 1 2 1 2 1 2 Half-Integral Primal-Dual Algorithm Edmonds’ Algorithm Half-Integral Algorithm ο Intermediate Primal – integral ο Intermediate Primal – half-integral ο Unique way to augment primal ο 3 ways to augment primal ο Build an alternating tree using tight edges and ο Build an alternating tree using tight edges and repeatedly attempt to augment primal and change dual values until there are no more exposed nodes Node Covering Phase repeatedly attempt to augment primal and change dual values until there are no more exposed nodes ο Deshrink if dual value on a set decreases to ο Deshrink if dual value on a set decreases to zero zero ο Shrink if a set forms a blossom Cycle Shrinking Phase ο Augment primal if a set forms a blossom ο After covering all exposed nodes, shrink all odd cycles to exposed nodes and proceed again until no more odd cycles Half-Integral Primal-Dual Algorithm Primal Augmentations 1 2 1 2 1 2 1 2 1 2 1 2 Half-Integral Primal-Dual Algorithm ο§ Optimality: Complementary slackness ο§ Poly-time convergence: ο Each Node Covering Phase is poly-time [consists of primal augmentation, deshrinking or extension of alternating trees] ο At the end of a Node Covering Phase, • the number of odd cycles in the primal is non-increasing • if the number of odd cycles is non-decreasing, then any set that is shrunk stays shrunk [similar to the even/odd parity argument for Edmonds’ algorithm] ο Laminarity ⇒ the number of odd cycles should decrease in at most Phases π 2 Cycle Shrinking Future Directions ο§ Implications of the Dual-based cut-retention procedure for other poly-time solvable combinatorial problems ο Combinatorial polytopes with Chvátal rank one (Edmonds-Johnson matrices) ο§ Efficient cutting plane algorithms for optimization over ο two-matroid-intersection polytope ο subtour elimination polytope Questions? Thank you