Linear and Integer Programming III
– Integer Programming
• Applications
• Algorithms
Page 1

minimize: c T x subject to: Ax ≤ b x ≥ 0 x  Z n
Related Problems
– Mixed Integer Programming (MIP)
– Zero-one programming
– Integer quadratic programming
– Integer nonlinear programming
Page 2

• Introduced in 1951 (Dantzig)
• TSP as special case in 1954 (Dantzig)
• First convergent algorithm in 1958 (Gomory)
• General branch-and-bound technique 1960
(Land and Doig)
• Frequently used to prove bounds on approximation algorithms (late 90s)
Page 3

• Has become “dominant” over linear
• Saves industry billions of dollars/year
• Can solve 10,000+ city TSP problems
• 1 million variable LP approximations
• Branch-and-bound, cutting plane, and separation all used in practice
• General purpose packages do not tend to work as well as with linear programming -- knowledge of the domain is critical.
Page 4

• Facility location
Locating warehouses or franchises (e.g., a Burger King)
• Set covering and partitioning
Scheduling airline crews
• Multicommodity distribution
Distributing auto parts
• Traveling salesman and extensions
Routing deliveries
• Capital budgeting
• Other applications
VLSI layout, clustering
Page 5

Integer (zero-one) Program: maximize subject to: c T x ax ≤ b x binary where: b = maximum weight c i a i
= utility of item i
= weight of item i x i
= 1 if item i is selected, or 0 otherwise
The problem is NP-hard.
Page 6

Find shortest tours that visit all of n cities.
courtesy:
Applegate ,
Bixby ,
Chvátal
, and Cook
Page 7

minimize: i n n 

1 j

1 c ij x ij subject to: n  ji j
 x

0 x ij x ij

 i
 n
(path enters and leaves) c ij x ij
= c ji
= distance from city i to city j
(assuming symmetric version) if tour goes from i to j or j to i, and 0 otherwise
Anything missing?
Page 8

minimize: subject to: t i j n i n n 

1 j

1


1 x ij
 c ij
1 x ij
 t j i n 

1
 x ij nx ij

1
 n

1
1
 i
 n
1
 j
 n
(out degrees = 1)
(in degrees = 1)
2
 i , j
 n
(??) c ij x ij t i
= distance from city i to city j
= 1 if tour visits i then j, and 0 otherwise (binary)
= “potential” of node i, real number to solve for
Page 9

The last set of constraints: prevents “subtours”: t i
 t j
 nx ij
 n

 i
j
 n
Consider a cycle that goes from some node 4 to 5, t
4
– t
5
+ nx
4,5
≤ n-1  t
5
≥ t
4
+ 1 + n(x
4,5
-1)  t
5
≥ t
4
+ 1
Thus, t has to increase by at least 1 along each edge of the cycle that does not include vertex 1.
(On the other hand, if x
4,5
=0, we have t
4
- t
5
 n-1, which just limits distance from vertex 5 to 4 to at most n-1.)
Therefore, if a cycle of length m contains vertex 4 but does not go through vertex 1, t
4
≥ t
4
+ m, a contradiction.
Every cycle must go through vertex 1.
Integer Programming Formulation of Traveling Salesman Problems,
C. E. Miller, A. W. Tucker, and R. A. Zemlin. Journal of the ACM ,
Volume 7, Number 4, October 1960, pp. 326-329.
Page 10

Many “Real World” applications based on the TSP.
– They typically involve more involved constraints
– Not just routing type problems.
Consider a drug company with k drugs they can make at a lab. They can only make the drugs one at a time. The cost of converting the equipment from making drug i to drug j is c ij
Best solutions are based on IP (but not MTZ formulation)
– Applegate, Bixby, et. al., have solutions for more than 15K cities in Germany
> 150,000 CPU hours ( more info )
– Involves “branch-and-bound” and “cutting planes”
Page 11

Find cheapest sets that cover all elements
Courtesy: Darmstadt
University of Technology
Page 12

Given m sets and n items:
A ij




Columns = sets
Rows = items c

j x j




Set covering: minimize: c T x subject to: Ax ≥ 1, x binary
Set partitioning: minimize: c T x subject to: Ax = 1, x binary
Page 13

set members cost s s
2 s
3 s
4 s
5 s
6 s
7
1
{a,c,d}
{b,c}
{b,e}
{a,d}
{c,e}
{b,c,e}
{c,d}
.5
.2
.3
.1
.2
.6
.2
A





1
1
0

1

 0
1
0
1
0
1
1
0
1
0
0
0
1
0
1
0
0
0
1
0
1
1
0
0
0
1
Best cover: s
2
, s
4
Best partition: s
4
, s
5
, s
6
= .5
= .7
0
0
1
1
0







Page 14

Applications:
• Facility location.
Each set is a facility (e.g., warehouse, fire station, emergency response center).
Each item is an area that needs to be covered
• Crew scheduling.
Each set is a route for a particular crew member
(e.g. NYC->Pit->Atlanta->NYC).
Each item is a flight that needs to be covered.
Page 15

Many constraints are expressible with integer programming:
– logical constraints (e.g., x implies not y)
– k out of n
– piecewise linear functions
– … and many more
Page 16

Logical constraints (x
1
, x
2
Either x
1
If x
1 k out of n or x
2 then x
2
Combining constraints: binary):
 -x
1
 x
1
- x
2
– x
2
≤
≤ 0
-1
 ∑
1≤i≤n x i
= k
Either a
1 x ≤ b
1 or a
2 x ≤ b
2
 a
1 x - My ≤ b
1 a
2 x - M(1-y) ≤ b
2 y is a binary variable, M needs to be “large”, a
1
, a
2
, and x can be vectors
Page 17

• Discrete variables: x i x i
=∑
1≤i≤n z i k i and ∑ in {k
1≤i≤n z
• Piecewise linear functions: i
1
• Create new binary vars z j
, k
2
, …, k n
} and add constraints
= 1
Constraint for x i • If x i
≥ 1 then a
• Convert to (x i i x i
≤ b i
< 1) or (a and use method from i x i
≤ b i
) previous page x i
0 1
Page 18

• Covering Constraints: Ax ≥ b (non-negative a ij
• Packing constraints: Ax ≤ b (non-negative a ij
’s)
’s)
• Connectivity constraints (e.g., for network design):
• flow formulation: to connect s and t, buy enough edges to support a unit s-t flow.
• Cut constraints (e.g., for clustering):
• Distance formulation: e.g., separate p and q
Variables d, x. Edge costs/lengths c(e): d(p, v) ≤ d(p, u) + c(e)x(e) for each edge e=(u,v) d(p,v) ≥ 0 for each v d(p,q) ≥ ∞ u c(e) v
Page 19

1. Use a linear program
– round to integer solution (what if not feasible?)
2. Search
– Branch and bound (integer ranges)
– Implicit (0-1 variables)
3. Cutting planes
– Many variants
Page 20

• LP solution is an upper bound on IP solution
(assuming maximization)
• If LP is infeasible then IP is infeasible
• If LP solution is integral (all variables have integer values), then it is the IP solution.
Page 21

1. Some LP problems will always have integer solutions
• transportation problem (send finished goods from factories to customers)
• assignment problem (assign workers to jobs)
• min-cost network flow (w/ integer capacities)
These are problems with a unimodular matrix A
(integer matrix invertible over the integers, det(A) =  1).
2. Solve as linear program and round. Can violate constraints, and be non-optimal. Works OK if
• integer variables take on large values
• accuracy of constraints is questionable
Page 22

Lets first consider 0-1 programs.
Exponential solution: try all {0,1} n
Branch-and-bound solution: x
2
= 0 x
1
= 0 x
3
= 0 x
1
= 1 x
3
= 1 x
2
= 1
Traverse tree keeping current best solution.
If it can be shown that a subtree never improves on the current solution, or is infeasable, prune it.
Page 23

minimize: z = c T x, subject to: Ax ≤ b, x ≥ 0, x  {0,1} n
Assume all elements of c are non-negative function ZO
// x f r
(A, b, c, x f
, z*)
: a fixed setting for a subset of the variables
+ 0 // set unconstrained variables to zero
if (Ax ≤ b) then return c T x pick an unconstrained variable x z z
0
1
// z* is the cost of current best solution x = x
if (c T x ≥
* = ZO
* = ZO
1 r r z*) or (no feasible completion of x
*
(A, b, x
(A, b, x f f
 {x
 {x i i
= 1}, c, z
0 from x
*) f
) return z*
function ZO(A, b, c) = ZO r
(A, b, c,  , ∞ )
Page 24

Checking for feasible completions: check each constraint and find if minimum of left is greater than right.
Example: x f
= {x
1
= 1, x
3 and one of the constraints is
3x
1
+ 2x
2
= 0}
≤ 2 then
3 + 2x
2
– x
3
2x
2 which is impossible.
+ x
4
- 0 + x
4
+ x
4
≤ 2
≤ -1
Page 25

The zero-one version is sometimes called “implicit enumeration” since it might enumerate all possibilities.
An integer version cannot branch on all possible integer values for a variable. Even if the integer range is bounded, it is not practical.
Will “bound” by adding inequalities to split the two branches.
Since solutions are integral, each split can remove a strip of width 1 of non-integral settings for a variable
Page 26

maximize: z = c T x subject to: Ax ≤ b, x ≥ 0, x  Z n function IP
// A r e
(A
, b e e
, b e
, c, z*) are A and b with additional constraints
// z* is the cost of current best solution z, x, f = LP(A e
,b e
,c) // f indicates whether feasible
if not(f) or (z < z*) return z*
if (integer(x)) return z pick a non-integer variable x z l
* = IP(extend A e z g
* = IP(extend A
return z g
* e
,b
,b e e with x i i
’ from x
≤ with –x i
 x i
’  , c, z*)
≤  x i
’  , c, z l
*)
function IP(A, b, c) = IP r
(A, b, c, ∞ )
Page 27

z feasible
l u
Find optimal solution.
Cut along y axis, and make two recursive calls
Page 28

z l u
Find optimal solution.
Solution is integral, so return it as current best z*
Page 29

z u l
= z*
Find optimal solution. It is better than z*.
Cut along x axis, and make two recursive calls
Page 30

z
Infeasible, Return.
u l
Page 31

z l u u
= z* l
Find optimal solution. It is better than z*.
Cut along y axis, and make two recursive calls
Page 32

z l u
= z*
Find optimal solution. Solution is integral and better than z*. Return as new z*.
Page 33

z l u
= z*
Find optimal solution. Not as good as z*, return.
Page 34

The idea is to start with a “relaxation” R of the problem and then add constraints on the fly to find an actual feasible solution in S.
new constraint
= S
= R
S new constraint
Example 1 relaxation
Example 2
A “linear” relaxation
Page 35

minimize: z = c T x, subject to x  S
function CP(R, c)
// R a relaxed set of constraints Ax ≤ b s.t. S  polytope(Ax ≤ b) repeat: x = LP(R,c)
if x  S return x find an inequality r satisfied by S, but violated by x (r separates x from S)
R = R
U
{r}
Can add multiple inequalities on each iteration
Page 36

z feasible
New plane
Note that we are removing a corner, and no integer solutions are being excluded.
Page 37

Method 1: Gomory cuts (1958)
– Cuts are generated from the LP Tableau
Each row defines a potential cut
– Guaranteed to converge on solution
– General purpose, but inefficient in practice
Method 2: problem specific cuts (templates)
– Consider the problem at hand and generate cuts based on its structure
– A template is a problem specific set of cuts
(probably of exponential size) which S satisfies.
Each round picks a cut from this set.
Page 38

We consider some example templates used in solutions of the Traveling Salesman Problem.
Recall that x ij indicates the edge from v i
Assume the symmetric TSP: x ij
= x ji
Consider subsets of vertices U  V.
 V-U to v j define: d x
(U) =  x ij
, v i
 U, v
(i.e., the number of times path crosses into/out of U)
Note: For any U, d x
Degree Constraints: d x
(U) is even.
({v i
}) = 2, 1 ≤ i ≤ n
Subtour Constraints: d x
(U) ≥ 2, U  V A template
There are an exponential number of these
Page 39

A set of contraints (a template) is facet-defining for
S if each constraint is on a facet of the convex hull of S.
We would like templates which are facet defining since, intuitively, they will more quickly constrain us to the boundary of S.
The subtour template is facet defining.
In practice the subtour inequalities are not enough to constrain the solution to integral solutions.
Are there other sets of facet defining constraints?
Page 40

Blossom inequalities (Edmonds 1965):
Defined by H (handle) and T
T k ≥ 3 and odd, |T i i
 T j
| = 2
=  |H  T i
| = 1, |T
1
, …, T i k
(teeth) satisfying:
\ H| = 1 d x
(H) +  i=1 k d x
(T i
) ≥ 3k + 1
1
T
1
H
T
3
T
2
Why? If d x
(T i
) = 2, then T contributes 1 to d x
(H).
Otherwise, d x
(T i
Minimum when d x
) ≥ 4
(T for all i. Since k is odd, d d x x
(H) ≥ k implies
(H) ≥ k +1 i
) = 2
Page 41 i

Comb inequalities (Grotschel 1977)
Just generalizes T i element of each T to be any size. At least one i has to be in and out of H.
H
T
1
T
3
T
2 d x
(H) +  i=1 k d x
(T i
) ≥ 3k + 1
Page 42

Picking the right set of templates, and applying them in the right way is the art of solving NP-hard problems with integer programming.
Different problems have different templates.
One needs to find good algorithms for selecting a member of a template that separates x from S
(can be quite complicated on its own).
Cutting planes often used in conjunction with branch and bound.
Can interleave template cuts with Gomory cuts (e.g., use Gomory cuts when the set of template cuts
“dries out”).
Page 43

• Good formulations, heuristics and theory
Goal: to get LP solution as close as possible to IP solution
Disaggregation, adding constraints (cuts)
• Preprocessing
Automatic methods for reformulation
Some interesting graph theory is involved
• Cut generation (branch-and-cut)
Add cuts during the branch-and-bound
• Column generation
Improve formulation by introducing an exponential number of variables.
Page 44