Week 10
1
Integer Linear Programming
So far we considered LP’s with continuous variables. However sometimes decisions are intrinsically restricted to integer numbers. E.g., in the long term
planning, decision variables that model acquisition of a number of machines for
a future production process, or ordering of airplanes, cars etc., are bound to
be integer. Therefore an important class of optimization problems is so-called
mixed integer linear programming problems (MILP):
min cT x + dT y
s.t. Ax + Dy = b
x, y ≥ 0
x integer.
Without continuous variables y it is called Integer LP (ILP). Ignoring the integrality restriction gives the so-called LP-relaxation. Solving the LP-relaxation
and rounding is usually not too bad in case the optimal solution has high numbers. But in case the x-values of the optimal LP-solution are between 0 and 1
then rounding may make a significant difference and indeed lead to unacceptable infeasibilities. This is all the more important since integer variables that
are restricted to have value either 0 or 1, so-called binary variables, enhance the
modelling power of ILP enormously.
1.1
Modelling with binary variables
Binary variables are used to represent a choice: take an action or not, process a
job on a particular machine or not, select an item in your knapsack or not.
Facility location
• m clients are to be served from facilities to be established on any of n
candidate locations
• cj is the cost of opening a facility at location j, j = 1, . . . , n
• dij is the cost of serving client i from facility j
• facilities have unlimited capacity w.r.t. serving clients
• select locations for facilities and assign each client to a facility opened such
as to minimize total cost
We introduce binary variables
• yj , which gets value 1 if a facility is opened on location j and value 0
otherwise, j = 1, . . . , n;
1
• xij , which gets value 1 if client i is served from a facility at location j,
and 0 otherwise, i = 1, . . . , m, j = 1, . . . , n
The ILP-formulation
Pn
Pm Pn
min
cj yj + i=1 j=1 dij xij
Pj=1
n
s.t.
j=1 xij = 1,
xij ≤ yj ,
xij , yj ∈ {0, 1}.
∀i,
∀i, ∀j,
(1)
The second set of constraints prohibit service from a facility that is not open.
We can apply a trick with a large enough number M to get to a more compact
ILP-formulation of the facility location problem:
Pn
Pm Pn
min
cj yj + i=1 j=1 dij xij
Pj=1
n
s.t.
xij = 1,
∀i,
Pj=1
(2)
m
x
≤
M
y
,
∀j,
j
i=1 ij
xij ≤ yj ∈ {0, 1}.
Again, the second constraints prohibit service from a facility that is not open,
and if the facility is open then M is chosen so big that there is no restriction on
the number of clients to be serve from it. Notice that M = m suffices here.
We could be tempted to conclude that the more compact formulation is the
better of the two. However, we will exhibit a criterion for the “strength” of an
ILP-formulation, under which, in fact, formulation (1) is stronger.
Clearly, both formulations have the same optimal solution with value that we call
Z IP . For formulations (1) and (2) the LP-relaxation gives an optimal solution
with value, respectively, Z F L1 and Z F L2 , both less than or equal to Z IP . A
closer look at the formulations reveals that
Z F L2 ≤ Z F L1 ≤ Z IP .
Notice that,
Pm in the LP-relaxation, any solution that satisfies xij ≤ yj , ∀i, also
satisfies i=1 xij ≤ myj , but not necessarily the other way round (think of a
solution with value yj = 1/2).
Thus the feasible region of the LP-relaxation of (2) contains the feasible region
of the LP-relaxation of (1), whereas both contain the convex hull of the feasible
(integer) points of (1) and (2). It is easy to see that all extreme points of the
convex hull of a set of integer points are integer. This means that if we had
the formulation in terms of linear inequalities of the convex hull of the feasible
points of the facility location problem, i.e., the formulation whose LP-relaxation
has only integer extreme points, then we could simply solve the facility location
problem by Simplex or any other LP-method.
2
In Chapter 11 we will see solution methods that are based on solutions for the
LP-relaxations and for these methods the closer the feasible region of the LPrelaxation is to the convex hull of the integer feasible solutions the better. In
this context we call formulation (1) stronger than formulation (2) since the feasible region of the LP-relaxation of (1) ⊂ the feasible region of the LP-relaxation
of (1). We do not qualify one polyhedron as stronger than another if it is not
fully contained in the other one.
Disjunctive constraints
The trick with big M is also applicable to model the following situation. Suppose
that at least one of two constraints
aT1 x ≥ b1 ,
aT2 x ≥ b2
has to be satisfied. Introduce binary variable y and a large enough number M
and set
aT1 x ≥ b1 − M y,
aT2 x ≥ b2 − M (1 − y)
y ∈ {0, 1}.
If M is large enough then y = 1 implies that the first restriction is always satisfied, and y = 0 that the second restriction is always satisfied. This can be
generalised as is asked in Exercise 10.1.
Making ILP-formulations of problems is a craft, in which some people are very
good. Please read for yourself the examples for job sequencing with set-up times
and for the allocation of fellowships and make Exercise 10.4.
Minimum spanning tree
Minimum Spanning Tree (MST): Given a graph G = (V, E) and (integer)
weights on the edges c : E → ZZ, find a spanning tree of edges of G with minimum total weight of its edges.
We skipped this problem in Section 7.10. It is extremely easily solved by a
greedy method that orders the edges according to non-decreasing weight and
selects them in this order unless the next edge in the order makes a cycle with
the previously selected edges, in which case it is not selected.
However, from the viewpoint of learning about strength of a ILP-formulation
we will study two ILP-formulations of the MST which at first sight look equally
good. In both formulations we use decision variables xe getting value 1 if they
3
are in the tree and 0 otherwise.
In the first formulations we use notation E(S) = {{i, j} | i ∈ S, j ∈ S}, for all
edges entirely inside a subset S of the nodes.
P
min Pe∈E ce xe
s.t. Pe∈E xe = n − 1, ∀S ⊂ V, S 6= ∅, V
∀S ⊂ V, S 6= ∅, V
e∈E(S) xe ≤ |S| − 1,
xe ∈ {0, 1}, ∀e ∈ E.
(3)
The first constraint ensures the right number of edges in the solution and the
second group of constraints ensures that there are no cycles (they have become
known in the literature under the name subtour elimination constraints. In the
LP-relaxation we replace the constraints xe ∈ {0, 1} by 0 ≤ xe ≤ 1, and call its
feasible Psub .
In the second formulation we use notation δ(S) = {{i, j} | i ∈ S, j ∈
/ S}
for the cutset of subset of nodes S, i.e., the edges that have one vertex in S
and the other outside S. Clearly a spanning tree must have at least one edge
in the cutset of any subset of the nodes. Indeed this leads to the following
formulation:
P
min Pe∈E ce xe
s.t. Pe∈E xe = n − 1, ∀S ⊂ V, S 6= ∅, V
(4)
∀S ⊂ V, S 6= ∅, V
e∈δ(S) xe ≥ 1,
xe ∈ {0, 1}, ∀e ∈ E.
Also here, in the LP-relaxation we replace the constraints xe ∈ {0, 1} by 0 ≤
xe ≤ 1, and call its feasible Pcut .
Theorem 1.1 Psub ⊂ Pcut
2
Proof. Proof as in the book.
The example in the book shows that Pcut can have fractional extreme points.
It shows a fractional solution that satisfies all constraints of the LP-relaxation
and has a value that is superior to the optimal integer value. The point does not
satisfy all constraints of the LP-relaxation of formulation (3), since it contains
a cycle with sum of x-values 2 i.o. the required 3.
In fact it can be shown that Psub is exactly equal to the convex hull of all spanning trees. Hence, solving the LP-relaxation of (3) solves the problem MST.
The only obstacle is that the LP-relaxation has 2n − 1 constraints. However,
the separation problem is solvable by solving a min-cut problem and therefore the LP-relaxation can be solved in polynomial time through the ellipsoid
method. Of course, one would never do so in practice, given a pure MSTproblem instance, but it could be a nice alternative in case of MST with some
side constraints.
4
Perfect matching
Given a graph G = (V, E), a matching is a subset of the edges that are vertex
disjoint (no two edges are incident to the same vertex). Given a graph with an
even number of vertices, a perfect matching is a matching such that the edges
cover all vertices, i.e., each vertex is coupled to one other vertex.
Given graph G = (V, E) and weights on edges c : E → ZZ, find a perfect matching
of minimum total weight.
The {0, 1}-LP formulation is simply, using the notation of δ introduced before,
P
min Pe∈E ce xe
s.t.
∀v ∈ V
e∈δ(v) xe = 1,
xe ∈ {0, 1}, ∀e ∈ E.
This allows to set values 21 on all edges of odd cycles, while in fact at least
one vertex on an odd cycle should have an edge to some vertex outside the
cycle. This observation can be captured in constraints. For each odd size subset
S ⊂V,
X
xe ≥ 1.
e∈δ(S)
Adding these constraints to the ones given before and relaxing the integrality
constraints defines the polytope of all perfect matchings as you can read in the
book Theory of linear and integer programming by Alexander Schrijver. This
is not an easy proof! To prove that the convex hull of all perfect matchings is
contained in the polytope is rather straightforward. But to prove the converse
is far from trivial. Again, a formulation with an exponential number of constraints, which can be separated in polynomial time. In this case, the standard
solution method is a primal-dual method based on the extended formulation.
Please read for yourself the two alternative formulation of the Travelling Salesman Problem and make Exercise 10.14.
Example of problematic ILP-formulation
That it is craftmanship I recently encountered in a Bachelor’s project at the
VU. It concerns a problem from biology, specifically from phylogeny. We are
to construct the phylogeny tree of a set of species (or subspecies) from data
that are given in the form of rooted binary trees on three leaves, as in Fig. 1.
The unique rooted triplet (triplet for short) on a leaf set {x, y, z} in which the
parent of x and y is a proper descendant of the lowest common ancestor of x
and z is denoted by xy|z (which is identical to yx|z). The triplet in the figure is
xy|z. We say that a directed binary tree containing, amongst others, leaves x, y
and z is consistent with triplet xy|z (and conversely xy|z is consistent with the
tree) if it contains a subdivision of xy|z, i.e., the rooted binary tree xy|z can be
5
x
y
z
Figure 1: One of the three possible triplets on the leaves x, y and z. Note that,
we adopt the convention to direct all arcs downwards, away from the root.
obtained from the tree by a series of contractions (of edges) and deletions (of
vertices).
Clearly the triplets xy|z and xz|y cannot be consistent with one tree. But also
the triplets xy|z, xw|z and zw|y cannot be consistent with one tree. Due to
measurement errors the triplets are not all perfect, hence inconsistencies may
occur. The optimization problem then emerges:
Max Phylogenetic Tree: Given a set of triplets on n species, find the
binary tree that contains all n species as its leaves and that is consistent with a
maximum number of triplets.
It is remarkably hard to find a correct ILP-formulation for this problem. In fact
the only one we have is indirect, in the sense that it gives the set of consistent
triplets in an optimal tree, without giving the tree. Given a set of triplets known
to allow for a consistent tree, such a tree can be constructed by a recursive
algorithm in O(n log n) time. The ILP-formulation itself is based on a theorem
that has been proved by various authors:
Theorem 1.2 A binary tree is not consistent with a triplet set if and only if
there is a subset of at most four leaves such that the triplets on these four leaves
are not all consistent with the tree.
Thus, we only have to make sure that for every subset of four leaves the triplets
that we select can be made consistent. For the ILP-formulation
we introduce a
n
variable tab|c for each possible triplet (there are 3
of them), getting value
3
1 meaning that the triplet is chosen to be consistent and 0 otherwise. Then for
each set a, b, c we add the restriction
tab|c + tac|b + tbc|a = 1,
and for each ordered set a, b, c, d we add the restriction
tab|c + tac|d − tbc|d ≤ 1
enforcing that is the first two triplets are consistent then the third one also needs
to be consistent. Given input triplet set T we set the objective
X
tab|c .
ab|c∈T
6
There are some remarkable features. The number of constraints is growing in
the O(n4 ) which means that problems with more than 40 species become problematic, just by the size of the input, for a desktop version of CPLEX. It is
worth investigating how much redundancy there is i the formulation and diminish the number of constraints. Or if the separation problem can be solved more
efficiently than checking all O(n4 ) constraints, i.e., without needing to specify
all of them. Secondly, in all cases from practice that we had the optimal solution
of the LP-relaxation appeared to be integer-valued. The polytope certainly has
not only integral extreme points, since then we would have shown that P = N P .
But maybe it has the {0, 1, 12 }-property?
Material of Week 10 from [B& T]
Chapter 10
Exercises of Week 10
10.1, 10.4, 10.11, 10.13, 10.14 and if you like 10.8
Next time in two weeks: April 20
Chapter 11
7