Linear Programming and Network Optimization
Zongpeng Li
Department of Computer Science
University of Calgary
Zongpeng Li – p.1/28
Outline
• Hello World linear program
• The power of LP
• LP models in network optimization
• LP duality
• Solving LPs
• Beyond LP
Zongpeng Li – p.2/28
Hello World
maximize 2x + y
s.t. :
x
y
x+y
≤ 2
≤ 2
≤ 3
x, y ≥ 0
Zongpeng Li – p.3/28
Hello World
y
2
1
0
1
2
x
Zongpeng Li – p.4/28
The power of LP
But we all know the world is nonlinear.
— Harold Hotelling, 1948
Zongpeng Li – p.5/28
The power of LP
But we all know the world is nonlinear.
1. If you have a problem that satisfies the axioms (of LP), then
use it. If it does not, then don’t.
— John von Neumann, 1948
2. Much more problems can be modelled using LPs than
suggested by intuition.
3. LP constitutes building blocks for nonlinear programming
Zongpeng Li – p.6/28
LP model: max-flow
B
5/10
C
8/9
5/5
0/4
S
8/10
A
3/3
0/6
5/8
T
5/5
D
• Maximum rate we can push flows from S to T in a given
capacitied flow network.
• flow-rate/link-capacity
Zongpeng Li – p.7/28
LP model: max-flow
→
χ = f (T S)
Maximize
Subject to:
(
→
→
→
f (uv) ≤ C(uv)
∀ uv6=T S
P
P
→
→
f
(
uv)
=
f
(
vu)
∀u
v∈N (u)
v∈N (u)
→
f (uv) ≥ 0
→
∀ uv
Zongpeng Li – p.8/28
Totally unimodular LPs
• Totally unimodular: every square sub-matrix of the
coefficient matrix has determinant of 1 or -1.
• Totally unimodular LPs always have integral optimal
solutions.
• The node-arc incidence matrix of a directed network is
totally unimodular!
Zongpeng Li – p.9/28
LP model: min-cut
P
Minimize
Subject to:
(
→
→
uv
C(uv)y(uv)
→
→
→
y(uv) + p(v) ≥ p(u) ∀ uv6=T S
p(T ) − p(S) ≥ 1
→
y(uv) ≥ 0
→
∀ uv
• Max-cut cannot be modelled as a simple LP; it is NP-hard.
• Elegant approximation algorithm of max-cut based on
semidefinite programming.
Zongpeng Li – p.10/28
LP model: min-cost flow
P
Minimize
→
→
uv
→
w(uv)f (uv)
Subject to:

→


 f (T S) = d
→
→
→
f (uv) ≤ C(uv)
∀ uv6=T S

P

→
→
 P
f
(
uv)
=
f
(
vu)
∀u
v∈N (u)
v∈N (u)
→
f (uv) ≥ 0
→
∀ uv
Zongpeng Li – p.11/28
LP model: shortest path
P
Minimize
Subject to:
(
→
→
uv
→
w(uv)f (uv)
→
f (T S) = 1
P
P
→
→
v∈N (u) f (uv) =
v∈N (u) f (vu) ∀u
→
f (uv) ≥ 0
→
∀ uv
Zongpeng Li – p.12/28
LP model: the assignment problem
• Assign n objects to n persons, 1-to-1 mapping
• Each object o worths v(i, o) to each person i
• Goal: maximize “total happiness”
Maximize
P P
i
Subject to:
f (i, o) ≥ 0
o f (i, o)v(i, o)
( P
o f (i, o) = 1 ∀i
P
i f (i, o) = 1 ∀o
∀i, ∀o
• Totally unimodular LP, integral optimal solution
• primal-dual algorithm design, the celebrated auction
algorithm
Zongpeng Li – p.13/28
LP model: max-rate multicast with network coding
Given network coding, a multicast rate x is feasible in a directed
network iff it is feasible as an independent unicast to every
receiver. [Ahlswede et al. IT 2000][Koetter and Médard TON 2003]
S
S
S
a
replication
point
a
a
T2
T1
T2
T1
T2
T1
S
S
S
b
a
a
T1
T2
T1
T2
T1
b
a+b
a
a+b
encoding
point
a+b
b
T2
Zongpeng Li – p.14/28
LP model: max-rate multicast with network coding
Maximize
Subject to:
χ

→

χ ≤ fi (Ti S)




→
→
fi (uv) ≤ c(uv)
P
P
→
→



v∈N (u) fi (uv) =
v∈N (u) fi (vu)

→
 →
c(uv) + c(vu) ≤ C(uv)
→
→
c(uv), fi (uv), χ ≥ 0
∀i
(1)
→
uv6=Ti S
→
∀i, ∀
∀i, ∀u
∀uv 6= Ti S
(2)
(3)
(4)
→
∀i, ∀ uv
Zongpeng Li – p.15/28
LP model: max-rate multicast without network coding
Minimize
Subject to:
P
t f (t)
X
f (t) ≤ c(e)
∀e
t:e∈t
f (t) ≥ 0
∀t
• Don’t be misguided by the seeming simplicity of the LP.
• It has exponentially many variables.
• We know a network instance with 16 nodes only, having
∼ 50 million different trees.
• But, what else can we do? It’s an NP-hard problem.
Zongpeng Li – p.16/28
Primal and dual LPs
Minimize
c1 x1 + c2 x2 + c3 x3
Subject to:



 a11 x1 + a12 x2 + a13 x3 ≥ b1
a21 x1 + a22 x2 + a23 x3 ≥ b2



a31 x1 + a32 x2 + a33 x3 ≥ b3
x1 , x2 , x3 ≥ 0
Maximize
b1 y1 + b2 y2 + b3 y3
Subject to:
↔ y1
↔ y2
↔ y3



 a11 y1 + a21 y2 + a31 y3 ≤ c1
a12 y1 + a22 y2 + a32 y3 ≤ c2



a13 y1 + a23 y2 + a33 y3 ≤ c3
↔ x1
↔ x2
↔ x3
y1 , y2 , y3 ≥ 0
• Poor student vs. greedy drug store owner
• Student: satisfying vitamin intaking needs with minimal
budget
• Store owner: maximizing revenue while maintaining
competitiveness
Zongpeng Li – p.17/28
LP duality
• Every feasible solution in the primal (minimization) provides
a lower-bound for the dual (maximization) and vice versa.
• If the primal is feasible and has optimal solutions, then so
does the dual; furthermore, their optimal objective function
values must be the same.
• Every max-min theorem (that I know of) in graph theory,
combinatorial optimization and game theory can be derived
as a corollary of the LP duality theorem and/or the matroid
union theorem.
Zongpeng Li – p.18/28
Complementary slackness
• Let x∗ and y ∗ be a pair of corresponding optimal primal and
dual solutions
• y1∗ > 0 ⇒ a11 x∗1 + a12 x∗2 + a13 x∗3 = b1 , and so on
• The shadow price is nonzero only if the resource supply is
tight
• Generalization into nonlinear programming: the
Karush-Kuhn-Tucker (KKT) conditions
Zongpeng Li – p.19/28
An example application of LP duality and CS
Enforcing minimum-cost multicast routing, Li and Williamson,
2007.
• Min-cost multicast, flows selfishly route themselves through
cheapest paths available
• Formulate primal and dual LPs
• Use shadow prices to allocate edge costs and set edge
taxes
• Each optimal flow can be thus enforced; proof of Nash
Equilibrium based on CS conditions
Zongpeng Li – p.20/28
Solving LPs: the simplex method
• Walk along a sequence of vertice, on the polyhedron
boundary
• with improved objective value at each step
• multiple “better neighbors”, which to choose?
• The pivot rule
Zongpeng Li – p.21/28
Solving LPs: the interior-point method
• Walk within the polytope
• Each step, walk towards a new feasible solution in the
polytope
• which had better not be too close to the boundary
• being close to the optimum is naturally good
• Model the above concerns using barrier functions and
potential functions
Zongpeng Li – p.22/28
Solving LPs: the ellipsoid method
• Solve optimization by solving feasibility, through binary
search.
• Enclose the feasibility polytope using an ellipsoid
• either verify feasibility using a separation oracle
• or cut the ellipsoid into two halves and enclose the feasible
half using a smaller ellipsoid
• Claim infeasibility when the ellipsoid becomes small
enough.
• Why ellipsoid? Why not a sphere? What about other
geometric shapes?
Zongpeng Li – p.23/28
Solving LPs: problem specific methods
• Tailor the simplex algorithm: the network simplex algorithm
• Lagrange relaxation and subgradient optimization
◦ Assume a network flow LP with an extra side constraint
◦ Can relax the side constraint and solve the smaller
network flow LP using highly optimized algorithms
◦ Trade-off: need to solve a sequence of these
◦ Can help in distributed protocol design
Zongpeng Li – p.24/28
Solving LPs: realworld experiences
• 1000 variables/constraints ? — that’s easy
• 1 million variables/constraints ? — that’s OK
• 1 billion variables/constraints ? — no way
• For general LPs: simplex and interior-point algorithms can
compete with each other
• For LPs with a network background: interior-point
algorithms might perform much better (personal experience)
• Ellipsoid algorithms are of theoretical interest mostly
Zongpeng Li – p.25/28
Solving LPs: software available
• GNU glpk, http://www.gnu.org/software/glpk/
◦ free
◦ simplex, interior-point, branch-and-cut
• CPLEX, http://www.ilog.com/products/cplex/
◦ simplex, interior-point, integer programming, quadratic
programming
• CVX, http://www.stanford.edu/ boyd/cvx/
◦ free
◦ Matlab library
◦ solves “disciplined” convex programs
Zongpeng Li – p.26/28
Liner integer programming: layered multicast
Maximize
Subject to:
P P
i
i
l
.x
k
k
k
(9)
 P
→
→
i (vu)]
i (uv)

=0
−
f
[f

v∈N
(u)
k
k

→
→

 f i (uv)
≤ fk (uv)
k
P
→
→

k fk (uv) ≤ C(uv)


→

i
 i
xk+1 ≤ xik ≤ fk (lTki S)
i →
fk (uv), fk (uv)
→
≥ 0,
xik
∈ {0, 1}
∀k, ∀i, ∀u
→
∀k, ∀i, ∀ uv
→
∀ uv
∀k = 1..L − 1, ∀i
→
∀k, ∀i, ∀ uv
Zongpeng Li – p.27/28
Semidefinite/Vector programming: max cut
Quadratic formulation of max cut (x(u) = 1 if u is in the source
component; otherwise x(u) = −1):
P
1
Maximize
uv∈E 2 (1 − x(u)x(v))
Subject to:
x(u) ∈ {1, −1}, ∀u
Vector programming relaxation:
P
1
Maximize
uv∈E 2 (1 − x(u)x(v))
Subject to:
||x(u)||n = 1, ∀u
x(u) ∈ Rn , ∀u; n ∈ Z+
Zongpeng Li – p.28/28