Integer Programming
(整数规划)
孙小玲教授
(xls@fudan.edu.cn)
复旦大学管理学院
时间：2012.7.23-2012.8.26
地点：台湾交通大学
1 / 38
Lecture 1: Introduction
(3 units)
Outline
I
Course introduction
I
What is integer programming?
I
Simple examples
I
Classification of integer programming problems
I
Course description
I
Examples from combinatorial optimization
2 / 38
Course introduction
I
32 units (45 minutes per unit) in 5 weeks.
I
There will be 4 assignments.
I
Scores will be based on the quality of the assignments.
I
Course website:
http://my.gl.fudan.edu.cn/teacherhome/xlsun
3 / 38
What is integer programming?
I
Operations Research (O.R.) is the discipline of applying
advanced analytical methods to help make better decisions.
Science of Better.
I
Mathematical Programming (Optimization) is a branch of
OR. It is about how to make decision (with regard to some
criteria) from some set of available alternatives.
I
Integer Programming is about decision making with discrete
or integer variables.
4 / 38
Why bother to use integer decision variable?
I
If the variable is associated with a physical entity that is
indivisible, then it must be integer, for example, number of
airplanes to produce, number of shares of stock (roundlot) ...
I
If a “Yes” or “No” decision has to be made (0 or 1).
I
In most of these cases the continuous approximation to the
discrete decision is not accurate enough for practical purposes.
5 / 38
Simple Examples
Example 1 (Stone Problem).
Given n stones of positive integer weights (i.e., given n positive
integers a1 , . . . , an ), check whether you can partition these stones
into two groups of equal weight, i.e., check whether a linear
equation
n
X
ai xi = 0
i=1
has a solution with xi ∈ {−1, 1}, ∀i.
(How to model it to an optimization problem?)
6 / 38
Example 2 (Knapsack Problem)
A burglar has a knapsack of size b. He breaks into a store that
carries a set of items n. Each item has profit cj and size aj . What
items should the burglar select in order to optimize his heist?
Let
½
1, Item j is selected
xj =
0, Otherwise
7 / 38
The problem can be modeled as a 0-1 integer program:
max
s.t.
n
X
j=1
n
X
c j xj
aj xj ≤ b,
j=1
xj ∈ {0, 1}, j = 1, . . . , n.
8 / 38
Example 3 (Assignment Problem).
I
n people available to carry out n jobs. Each person is assigned
to carry out one job. The cost for person i to do job j is cij .
The problem is to find a minimum cost assignment.
½
xij =
I
1, if person i does job j
0, otherwise
The problem can be formulated as:
min
n X
n
X
cij xij
i=1 j=1
s.t.
n
X
xij = 1, i = 1, . . . , n
j=1
n
X
xij = 1, j = 1, . . . , n, xij ∈ {0, 1}.
i=1
9 / 38
Some Real-World Optimization Problems
I
Train scheduling
I
Airline crew scheduling
I
Highway pavement maintenance and rehabilitation
I
Production planing
I
Electricity generation planning
I
Telecommunications
I
Cutting problem
10 / 38
Classification of Integer Programming
I
We call the problem mixed integer programming due to the
presence of continuous variables.
I
In some problems x are allowed to take on values only 0 or 1.
Such variables are called binary. Integer programs involving
only binary variables are called binary integer programs
(BIPs). (x ∈ Bn : {0, 1}n , or x ∈ {−1, 1}n )
I
Pure Integer Linear Programming:
(IP)
I
min {c T x | Ax ≤ b, x ∈ Zn+ }.
Mixed 0-1 Programming:
(MIP)
min{c T x + hT y | Ax + Gy ≤ b, x ∈ Bn , y ∈ Rp+ }.
11 / 38
I
A general (nonlinear) integer programming problem can be
formulated as:
(NLIP)
min f (x)
s.t. gi (x) ≤ bi , i = 1, . . . , m,
x ∈ X,
I
where f and gi , i = 1, . . ., m, are real-valued functions on
Rn , and X is a finite subset in Zn , the set of all integer points
in Rn .
Mixed-integer nonlinear programming problem
(MNLIP)
min f (x, y )
s.t. gi (x, y ) ≤ bi , i = 1, . . . , m,
x ∈ X, y ∈ Y,
where f and gi , i = 1, . . ., m, are real-valued functions on
Rn+q , X is a finite subset in Zn , and Y is a continuous subset
in Rq .
12 / 38
How Hard is Integer Programming?
I
First thought (a bit naive):
I
I
I
I
I
Total enumeration: For Stone Problem and 0-1 Knapsack
Problem. To list all the feasible points, a super computer with
speed 108 (Yi) basic operations per second needs:
n = 30, 230 ≈ 109 , 10 seconds.
n = 60, 260 ≈ 1018 , 360 years
n = 100, 2100 ≈ 1030 , 4 × 1014 years
...
13 / 38
I
Round off: Solve the continuous optimization and round the
solution to its nearest integer points.
I
Heuristic methods. Greedy and local search methods. For 0-1
knapsack problem. Ranking the ratio of the profit to weight
as:
cj
cj
cj1
≥ 2 ≥ ··· ≥ n .
aj1
aj2
ajn
Choose the items in the order j1 , . . ., jn , and stop when taking
the next item to the bag exceeds the total capacity b.
I
Optimal solution? (If not, construct an counterexample)
14 / 38
Deep thoughts
I
I
I
I
I
I
Most of the integer programs are NP-complete or NP-hard,
which means the problem is “as difficult as a combinatorial
problem can be”, if we knew an efficient algorithm for one
problem, we would be able to convert it to an efficient
algorithm to solve any other combinatorial problem.
Solving the linear programming relaxation results in a lower
bound on the optimal solution to the IP.
Rounding to a feasible integer solution may be difficult or
impossible.
The optimal solution to the LP relaxation can be arbitrarily
far away from the optimal solution to the MIP (except totally
unimodular case).
Solving general integer programs can be much more difficult
than solving linear programs or convex optimization problems.
This is more than just an empirical statement. There is a
whole theory surrounding it (you will learn some soon)
15 / 38
Course objectives
I
I
I
I
I
Be able to use integer variables for formulating complex
mathematical models in management science, industrial
engineering and transportation science and learn how to use
various software package for solving integer programming
models.
Understand the concepts of NP-Completeness and
NP-Hardness and the difficulty of integer programming
problems using the tool of complexity theory.
Be able to use the basic methodology for the solution of
integer programs.
Understand the basic concepts of polyhedral theory and valid
inequalities and how to integrate the theory to the solution
methods for integer programming.
Understand the advanced methods based on column
generation and Dantzig-Wolfe decomposition for large-scale
integer programming.
16 / 38
Course Modules
I
Module 1 (IP Basic)
I
I
I
I
Modeling problems with integer decision variables.
Branch-and-Bound method. The very basics of the
“workhorse”algorithm for solving IPs
Software for IP (Matlab, CLPEX, AIMMS)
Module 2 IP Theory
I
I
Complexity Theory. What makes a problem “hard”or
“easy”The complexity classes P, and NP, and
NP-completeness
Polyhedral Theory. Dimension, facets, polarity, and the
equivalence of separation and optimization.
17 / 38
I
Module 3 Basic Algorithms
I
I
I
I
Module 4 Advanced Algorithms
I
I
I
I
Branch-and-bound
Dynamic programming
Cutting plane method
Lagrangian Relaxations
Column generation and Dantzig-Wolfe decomposition
Branch-and-Price.
Module 5 Nonlinear IP
I
I
Quadratic 0-1 optimization, Max-cut problem
Quadratic knapsack problem
18 / 38
Combinatorial Optimization Problems
Example 1 (Set Covering Problem)
I
Given a certain number of regions, the problem is to decide
where to install a set of emergency service centers. For each
possible center the cost of installing a service center, and
which regions it can be service are known. The goal is to
choose a minimum cost set of service centers so that each
region is covered.
I
Let M = {1, . . . , m} be the set of regions, and N = {1, . . . , n}
the set of potential centers. Let Sj ⊆ M be the regions that
can be serviced by a center at j ∈ N, and cj its installing cost.
I
Define a 0-1 incidence matrix A such that aij = 1 if i ∈ Sj and
aij = 0 otherwise. Let
½
1, if center j is selected
xj =
0, otherwise
19 / 38
I
The problem can be formulated as
min
s.t.
n
X
c j xj
j=1
n
X
aij xj ≥ 1, i = 1, . . . , m
j=1
x ∈ {0, 1}n .
20 / 38
Example 2: Facility location problem
I Given a set N = {1, . . . , n} of potential facility locations and
a set of clients M = {1, . . . , m}. A facility placed at j costs fj
for j ∈ N. The profit from satisfying the demand of client i
from a facility at j is cij . The problem is to choose a subset of
the locations at which to place facilities, and then to assign
the clients to these facilities so as to maximize the total profit.
I Let xj = 1 if a facility is placed at j and xj = 0 otherwise.
I Let yij be the fraction of the demand of client i that is
satisfied from a facility at j.
I The problem can be modeled as as
X
XX
max
cij yij −
fj xj
i∈M j∈N
s.t.
X
j∈N
yij = 1,
∀i ∈ M,
j∈N
yij ≤ xj ,
∀i ∈ M, j ∈ N,
n
x ∈ {0, 1} , y ∈ <mn
+ .
21 / 38
Example 3: Traveling Salesman Problem (TSP)
I
I
A traveling salesman must visit each of n cities before
returning home. He knows the distance between each of the
cities and wishes to minimize the total distance traveled while
visiting all of the cities.
Problem: In what order should he visit the cities?
Figure: Traveling salesman problem
22 / 38
I
Let the distance from city i to city j be cij . Let
½
1, if he goes directly from i to j
xij =
0, otherwise
I
Constraints:
I
He leaves city i exactly once:
X
xij = 1, i = 1, . . . , n.
j6=i
I
He arrives at city j exactly once:
X
xij = 1, j = 1, . . . , n.
i6=j
23 / 38
I
Subtour elimination constraints:
XX
xij ≥ 1, ∀S ⊂ N = {1, . . . , n}, S 6= ∅,
i∈S j6∈S
or
XX
xij ≤ |S| − 1, ∀S ⊂ N, 2 ≤ |S| ≤ n − 1.
i∈S j∈S
I
TSP can be formulated as
n X
n
X
min
cij xij
i=1 j=1
s.t.
X
xij = 1, i = 1, . . . , n,
j6=i
X
xij = 1, j = 1, . . . , n,
i6=j
XX
xij ≤ |S| − 1, ∀S ⊂ N, 2 ≤ |S| ≤ n − 1,
i∈S j∈S
x ∈ {0, 1}n .
24 / 38
Figure: TSP 1000 cities
25 / 38
Figure: Optimal tour for the TSP 1000 cities
26 / 38
I
A problem related to TSP is the vehicle routing problem: a
number of vehicle located at a central depot has to serve a set
of customers which geographically surround the depot. Each
vehicle has a given capacity and each customer has a a given
demand. The goal is to minimize the total travel distance.
Figure: Vehicle routing problem
27 / 38
Example 4 (Generalized Assignment Problem)
Given m machines and n jobs, find a least cost assignment of jobs
to machines not exceeding the machine capacities. Each job j
requires aij units of machine i’ s capacity bi . The problem can be
modeled as
min
s.t.
m X
n
X
cij xij
i=1 j=1
n
X
aij xj ≤ bi , ∀i,
j=1
m
X
xij = 1, ∀j,
(machine capacity)
(assign all jobs)
i=1
xij ∈ {0, 1}, ∀i, j
28 / 38
Example 5: Minimum cost network flows
I
A digraph G = (V , A) with arc capacity hij (integer), for
(i, j) ∈ A, demand bi at node i ∈ V , unit flow cost cij for
(i, j) ∈ A.
I
Find feasible integer flow with the minimum cost.
I
xij : the flow for arc (i, j), Vi+ = {k | (i, k) ∈ A},
Vi− = {k | (k, i) ∈ A}.
I
The model:
min
X
cij xij
(i,j)∈A
s.t.
X
k∈Vi+
xik −
X
xki = bi , i ∈ V
k∈Vi−
0 ≤ xij ≤ hij , xij integer.
29 / 38
Example 6: Shortest Path Problem
Given two nodes s, t ∈ V and nonnegative arc costs cij for
(i, j) ∈ A, find a minimum cost flow from s to t.
30 / 38
The problem can be formulated as
X
min
cij xij
(i,j)∈A
s.t.
X
xik −
k∈Vi+
X
k∈Vi+
xki = 1, i = s
k∈Vi−
xik −
k∈Vi+
X
X
X
xki = 0, i ∈ V \ {s, t}
k∈Vi−
xik −
X
xki = −1, i = t
k∈Vi−
xij ∈ {0, 1}, (i, j) ∈ A.
31 / 38
Example 7: Maximum flow problem
I
Given two nodes s, t ∈ V and nonnegative arc capacities hij
for (i, j) ∈ A, find a maximum flow from s to t.
I
Adding a backward arc from t to s with hij = +∞. The
Maximum flow problem can be formulated as
max xts
X
X
s.t.
xik −
xki = 0, i ∈ V ,
k∈Vi+
k∈Vi−
0 ≤ xij ≤ hij , (i, j) ∈ A.
32 / 38
Nonlinear Integer Programming Models
Example 1: Portfolio Optimization
I
A market with n securities. An investor with initial wealth W0
seeks to improve his wealth status by investing his wealth into
these n risky securities and into a risk-free asset (e.g., a bank
account).
I
Xi : the random return per lot of the i-th security
(i = 1, . . . , n) before deducting associated transaction costs.
I
The mean and covariance:
µi = E(Xi ), and σij = Cov(Xi , Xj ),
I
i, j = 1, . . . , n.
xi : the integer number of lots the investor invests in the i-th
security. Portfolio vector: x = (x1 , . . . , xn )T .
33 / 38
I
The random return: Ps (x) =
variance of Ps (x) are
Pn
i=1 xi Xi .
The mean and
n
n
X
X
s(x) = E[Ps (x)] = E[
xi Xi ] =
µi xi
i=1
i=1
and
V (x) = Var(Ps (x))
n
X
= Var[
xi Xi ]
i=1
=
n X
n
X
xi xj σij = x T Cx,
i=1 j=1
I
I
where C = (σij )n×n is the covariance matrix.
Let r be the interest rate of the risk-free asset.
P
Transaction cost: c(x) = ni=1 ci (xi ).
34 / 38
I
Total expected return of portfolio:
n
X
R(x) = s(x) + rx0 −
ci (xi )
i=1
=
n
X
[(µi − rbi )xi − ci (xi )] + rW0 .
i=1
I
Budget constraint:
b T x ≤ W0 + Ub
I
where b = (b1 , . . . , bn )T and Ub is the upper borrowing limit
from the risk-free asset.
Integer programming model:
(MV )
min V (x) = x T Cx
n
X
s.t. R(x) =
[(µi − rbi )xi − ci (xi )] + rW0 ≥ ε,
i=1
T
U(x) = b x ≤ W0 + Ub ,
x ∈ X = {x ∈ Zn | li ≤ xi ≤ ui }.
35 / 38
Example 2: Maximum Cut
I
I
I
Let G be an n-node graph, and let the arcs (i; j) of the graph
be associated with nonnegative “weights” aij (aij = aji ≥ 0).
A cut (S, S 0 ) is a partition of the n nodes in two disjoint set.
The problem is to find a cut of the largest possible weight,
i.e., to partition the set of nodes in two parts S, S 0 so that
the total weight of all arcs linking S and S’ is maximized
36 / 38
I
Let xi = 1 for i ∈ S, xi = −1 for i ∈ S 0 .
I
Total weight of cut (S, S 0 ) is:


n
X
1 1
aij (1 − xi xj ) (Why ?)
2 2
i,j=1
=
I
1
4
n
X
aij (1 − xi xj ).
i,j=1
The Maxcut problem can be modeled as
max
n
1 X
aij (1 − xi xj )
4
i,j=1
s.t. x ∈ {−1, 1}n .
37 / 38
I
Goemans and Williamson (1995)’s brilliant result on the
quality of semidefinite program (SDP) relaxation of Maxcut
problem:
fopt
0.878. ≤
≤1
fSDP
I
Moreover, they showed that a 0.878. approximate (feasible)
solution can be obtained by a randomized method to Maxcut
problem!
(2000 AMS-MPS Fulkerson Prize)
38 / 38