MEIO/UPC-UB : NETWORK FLOWS
Network Flows
UPCOPENCOURSEWARE number 34414
Topic 8: Lagrangian Relaxation
F.-Javier Heredia
This work is licensed under the Creative Commons AttributionNonCommercial-NoDerivs 3.0 Unported License. To view a copy of this
license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/ or send a
letter to Creative Commons, 444 Castro Street, Suite 900, Mountain View,
California, 94041, USA.
• Lagrangian Relaxation and subgradient optimization:
– Lagrangian Duality.
– Solving the dual problem.
– Subgradient optimization.
• Applications
– TSP
– Generalized assignment
– Facility Location
• Lagrangian Relaxation and LPs.
• Source material:
– R.K. Ahuja, Th.L. Magnanti, J. Orlin “Network Flows”, chap. 15.
– J. Orlin “Network Optimization” http://ocw.mit.edu/courses/sloanschool-of-management/15-082j-network-optimization-fall-2010/
F.-Javier Heredia http://gnom.upc.edu/heredia
J. Orlin http://jorlin.scripts.mit.edu/
MCNFP- 2
Lagrangian Relaxation
MEIO/UPC-UB : NETWORK FLOWS
8.- Lagrangian Relaxation
• In solving network flow problems, we not only
solve the problem, but we provide a guarantee
that we solved the problem.
• Guarantees are one of the major contributions
of an optimization approach.
• But what can we do if a minimization problem is
too hard to solve to optimality?
• Sometimes, the best we can do is to offer a
lower bound on the best objective value. If the
bound is close to the best solution found, it is
almost as good as optimizing.
F.-Javier Heredia http://gnom.upc.edu/heredia
J. Orlin http://jorlin.scripts.mit.edu/
MCNFP- 3
Lagrangian Relaxation
MEIO/UPC-UB : NETWORK FLOWS
On bounding in optimization
• Decomposition based approach.
• Start with
• Easy constraints
• Complicating Constraints.
• Put the complicating constraints into the
objective.
• We will obtain a lower bound on the optimal
solution for minimization problems.
• In many situations, this bound is close to the
optimal solution value.
F.-Javier Heredia http://gnom.upc.edu/heredia
J. Orlin http://jorlin.scripts.mit.edu/
MCNFP- 4
Lagrangian Relaxation
MEIO/UPC-UB : NETWORK FLOWS
Overview
Primal problem (P):
z*
= min { cx | Ax = b, x ∈ X }
Lagrangian problem (P(µ)):
L(µ)
= min { cx + µ(Ax - b) | x ∈ X }
Lagrangian Dual problem (D): L*
Lemma 16.1.:
= max { L(µ) | µ ∈ ℜn }
For all vectors µ, L(µ) ≤ z*.
For all vectors µ, L(µ) ≤ L* ≤ z*.
Lemma 16.2.: (weak duality):
Property 16.3 (Optimality test):
1.
If L(µ) = c’x for some primal feasible x
⇒ x and µ are optimal (P) and (D) resp.
1.
If, for some µ, x*=argmin P(µ) is primal feasible
⇒ x and µ are optimal (P) and (D) resp.
F.-Javier Heredia http://gnom.upc.edu/heredia
J. Orlin http://jorlin.scripts.mit.edu/
MCNFP- 5
Lagrangian Relaxation
MEIO/UPC-UB : NETWORK FLOWS
The Lagrangian Multiplier Problem
(with equality constraints)
Primal problem (P ≤):
z*
= min { cx | Ax ≤ b, x ∈ X }
Lagrangian problem (P ≤(µ)):
L(µ)
= min { cx + µ(Ax - b) | x ∈ X }
Lagrangian Dual problem (D ≤): L*
= max { L(µ) | µ ≥ 0, µ ∈ ℜn }
Property 16.4.: If, for some µ, the solution x*=argmin P(µ) is:
1.
Primal feasible.
2. Satisfies the complementary slackness condition
µ(Ax - b) = 0
Then x* is optimal for the primal problem (P≤)
F.-Javier Heredia http://gnom.upc.edu/heredia
J. Orlin http://jorlin.scripts.mit.edu/
MCNFP- 6
Lagrangian Relaxation
MEIO/UPC-UB : NETWORK FLOWS
The Lagrangian Multiplier Problem
(with inequality constraints)
2
(1,10)
1
(1,1)
4
(2,3)
(1,7)
(10,1)
(1,2)
(5,7)
(10,3)
6
(2,2)
3
(12,3)
5
Find the shortest path from node 1 to node 6 with a
transit time at most 10
F.-Javier Heredia http://gnom.upc.edu/heredia
J. Orlin http://jorlin.scripts.mit.edu/
MCNFP- 7
Lagrangian Relaxation
MEIO/UPC-UB : NETWORK FLOWS
The Constrained Shortest Path Problem
• Given: a network G = (N,A)
•
cij
cost for arc (i,j)
•
tij
Z* = Min
s. t.
traversal time for arc (i,j)
∑
c xij
( i , j )∈ A ij
 1 if i = s

x
−
x
=
∑ j ij ∑ j ji  −1 if i = t
 0 otherwise

∑ ( i , j )∈A tij xij ≤ T
Complicating constraint
=
xij 0 or 1 for all ( i , j ) ∈ A
F.-Javier Heredia http://gnom.upc.edu/heredia
J. Orlin http://jorlin.scripts.mit.edu/
MCNFP- 8
Lagrangian Relaxation
MEIO/UPC-UB : NETWORK FLOWS
Constrained Shortest Paths: LP Formulation
Suppose that X = {x1, x2, x3, …, xK}. Possibly K is
exponentially large; e.g., X is the set of paths from node
s to node t.
L(µ)
=
min
cx + µ(Ax - b) = (c + µA)x - µb
subject to
L(µ)
=
x ∈ X.
min {(c + µA)xk - µb : k = 1 to K}
F.-Javier Heredia http://gnom.upc.edu/heredia
J. Orlin http://jorlin.scripts.mit.edu/
MCNFP- 9
Lagrangian Relaxation
MEIO/UPC-UB : NETWORK FLOWS
An alternative representation
L(µ) =
min {(c + µA)xk - µb : k = 1 to K }
Determine L* = max { L(µ) : µ ≥ 0 }
F.-Javier Heredia http://gnom.upc.edu/heredia
J. Orlin http://jorlin.scripts.mit.edu/
MCNFP- 10
Lagrangian Relaxation
MEIO/UPC-UB : NETWORK FLOWS
Solving the Lagrange Multiplier Problem
We now list all K
paths from 1 to 6.
2
(1,10)
1
(1,1)
4
(2,3)
(1,7)
(10,1)
(1,2)
(5,7)
(10,3)
6
(2,2)
3
(12,3)
5
P
cP
tP
cP+ µ (tP – 14)
1-2-4-6
3
18
3+4µ
1-2-5-6
5
15
5+ µ
1-2-4-5-6
14
14
14
etc.
F.-Javier Heredia http://gnom.upc.edu/heredia
J. Orlin http://jorlin.scripts.mit.edu/
MCNFP- 11
Lagrangian Relaxation
MEIO/UPC-UB : NETWORK FLOWS
Suppose we want the
min cost path from 1 to
6 with transit time at
most 14.
J. Orlin http://jorlin.scripts.mit.edu/
MCNFP- 12
Lagrangian Relaxation
MEIO/UPC-UB : NETWORK FLOWS
F.-Javier Heredia http://gnom.upc.edu/heredia
30
1-3-4-6
20
1-2-4-6
1-3-4-5-6
1-2-4-5-6
1-2-5-6
1-3-2-4-6
10
0
L* = max (L(µ): µ ≥ 0)
-10
0
1
2
3
4
Lagrange Multiplier µ
1-3-2-4-5-6
1-3-2-5-6
1-3-5-6
5
Figure 16.3 The Lagrangian function for T = 14.
F.-Javier Heredia http://gnom.upc.edu/heredia
J. Orlin http://jorlin.scripts.mit.edu/
MCNFP- 13
Lagrangian Relaxation
Composite Cost
MEIO/UPC-UB : NETWORK FLOWS
Paths
L(µ)
=
min {(c + µA)xk - µb : k = 1 to K }
In the algorithm S will be a subset of {1, 2, …, K}
Let LS(µ) = min {(c + µA)xk - µb : k ∈ S}
LS(µ) ≥ L(µ).
So, L*S ≥ L*.
1.
2.
3.
4.
5.
Initialize S
Find µ* that maximizes L*S = LS(µ).
Find xk ∈ X that minimizes (c + µ*A)x
If k ∈ S, quit with the optimal solution
Else, add k to S, and return to step 2.
F.-Javier Heredia http://gnom.upc.edu/heredia
J. Orlin http://jorlin.scripts.mit.edu/
At end,
L*S = LS(µ*)
= L(µ*)
≤ L*.
MCNFP- 14
Lagrangian Relaxation
MEIO/UPC-UB : NETWORK FLOWS
Solving the Lagrange Multiplier Problem
Paths
30
1-2-4-6
3 + 18 µ
20
2.1, 40.8
10
0
1-3-5-6 24 + 8 µ
-10
0
1
2
3
4
Lagrange Multiplier µ
5
F.-Javier Heredia http://gnom.upc.edu/heredia
J. Orlin http://jorlin.scripts.mit.edu/
MCNFP- 15
Lagrangian Relaxation
Composite Cost
MEIO/UPC-UB : NETWORK FLOWS
We start with the paths 1-2-4-6, and 1-3-5-6
which are optimal for L(0) and L(∞).
2
22
1
The optimum path
is 1-3-2-5-6
3.2
4
8.3
12.1
5.2
6
19.7
16.3
15 + 10 µ
15.7
6.2
3
18.3
5
F.-Javier Heredia http://gnom.upc.edu/heredia
J. Orlin http://jorlin.scripts.mit.edu/
MCNFP- 16
Lagrangian Relaxation
MEIO/UPC-UB : NETWORK FLOWS
Set µ = 2.1 and solve the constrained shortest path
problem
Paths
30
1-2-4-6
3+4µ
20
10
1.5, 9
0
1-3-2-5-6
1-3-5-6
-10
0
1
2
3
4
Lagrange Multiplier µ
5
F.-Javier Heredia http://gnom.upc.edu/heredia
J. Orlin http://jorlin.scripts.mit.edu/
15 - 4 µ
24 - 6 µ
MCNFP- 17
Lagrangian Relaxation
Composite Cost
MEIO/UPC-UB : NETWORK FLOWS
Add Path 1-3-2-5-6 and reoptimize
The optimum path
is 1-2-5-6.
2
16
1
2.5
4
6.5
11.5
11.5
4
6
15.5
14.5
5
3
16.5
5
F.-Javier Heredia http://gnom.upc.edu/heredia
J. Orlin http://jorlin.scripts.mit.edu/
MCNFP- 18
Lagrangian Relaxation
MEIO/UPC-UB : NETWORK FLOWS
Set µ = 1.5 and solve the constrained shortest path
problem
Paths
30
1-2-4-6
3+4µ
1-2-5-6
5+µ
1-3-2-5-6
1-3-5-6
15 - 4 µ
24 - 6 µ
20
10
2, 7
0
-10
0
1
2
3
4
Lagrange Multiplier µ
5
F.-Javier Heredia http://gnom.upc.edu/heredia
J. Orlin http://jorlin.scripts.mit.edu/
MCNFP- 19
Lagrangian Relaxation
Composite Cost
MEIO/UPC-UB : NETWORK FLOWS
Add Path 1-2-5-6 and reoptimize
2
21
1
3
4
8
15
12
5
6
19
16
The optimum
paths are
1-2-5-6 and
1-3-2-5-6
6
3
18
5
F.-Javier Heredia http://gnom.upc.edu/heredia
J. Orlin http://jorlin.scripts.mit.edu/
MCNFP- 20
Lagrangian Relaxation
MEIO/UPC-UB : NETWORK FLOWS
Set µ = 2 and solve the constrained shortest path
problem
Paths
30
1-2-4-6
3+4µ
1-2-5-6
5+µ
1-3-2-5-6
1-3-5-6
15 - 4 µ
24 - 6 µ
20
10
2, 7
0
-10
0
1
2
3
4
Lagrange Multiplier µ
5
F.-Javier Heredia http://gnom.upc.edu/heredia
J. Orlin http://jorlin.scripts.mit.edu/
MCNFP- 21
Lagrangian Relaxation
Composite Cost
MEIO/UPC-UB : NETWORK FLOWS
There are no new paths to add.
µ* is optimal for the multiplier problem
L(µ)
=
min {(c + µA)xk - µb : k = 1 to K }
Let S be a subset of {1, 2, …, K}
Let LS(µ) = min {(c + µA)xk - µb : k ∈ S}
1.
2.
3.
4.
5.
Initialize S
Find µ* that maximizes L*S = LS(µ).
Find xk ∈ X that minimizes (c + µ*A)x
If k ∈ S, quit with the optimal solution
Else, add k to S, and return to step 2.
F.-Javier Heredia http://gnom.upc.edu/heredia
J. Orlin http://jorlin.scripts.mit.edu/
LS(µ) ≥ L(µ).
So, L*S ≥ L*.
At end,
L*S = LS(µ*)
= L(µ*)
≤ L*.
MCNFP- 22
Lagrangian Relaxation
MEIO/UPC-UB : NETWORK FLOWS
Solving the Lagrange Multiplier Problem
Let LS(µ) = min {(c + µA)xk - µb : k ∈ S}
Find µ* that maximizes LS(µ).
 This is a linear program (see next slide)
Find xk ∈ X that minimizes (c + µ*A)x
 This is optimizing over X
Usually X is chosen so that optimizing over X is
something that “easy” to do
e.g., finding a minimum cost spanning tree
or finding a minimum cost flow
F.-Javier Heredia http://gnom.upc.edu/heredia
J. Orlin http://jorlin.scripts.mit.edu/
MCNFP- 23
Lagrangian Relaxation
MEIO/UPC-UB : NETWORK FLOWS
Solving the Lagrange Multiplier Problem
min (a1, a2, …, an) = max { w : w ≤ aj for j = 1 to n }
min (4, 9, 3) = max { w : w ≤ 4; w ≤ 9; w ≤ 3 }.
LS(µ) = min {(c + µA)xk - µb : k ∈ S}
LS(µ) = max {w : w ≤ (c + µA)xk - µb for k ∈ S}
L*S
= max
w
w ≤ (c + µA)xk - µb for k ∈ S
µ≥0
F.-Javier Heredia http://gnom.upc.edu/heredia
J. Orlin http://jorlin.scripts.mit.edu/
MCNFP- 24
Lagrangian Relaxation
MEIO/UPC-UB : NETWORK FLOWS
Determining L*S by solving an LP
The Lagrangian multiplier problem L* = max µ L(µ) with
L(µ) = {min cx + µ(Ax - b) : x ∈ X} is equivalent to the
linear programming problem:
L*
= max
w
w ≤ (c + µA)xk - µb for k = 1 to K
µ≥0
F.-Javier Heredia http://gnom.upc.edu/heredia
J. Orlin http://jorlin.scripts.mit.edu/
MCNFP- 25
Lagrangian Relaxation
MEIO/UPC-UB : NETWORK FLOWS
Theorem 16.5
• Another major solution technique for solving the
Lagrange Multiplier Problem is subgradient
optimization.
• Based on ideas from non-linear programming.
• It converges (often slowly) to the optimum.
F.-Javier Heredia http://gnom.upc.edu/heredia
J. Orlin http://jorlin.scripts.mit.edu/
MCNFP- 26
Lagrangian Relaxation
MEIO/UPC-UB : NETWORK FLOWS
Subgradient optimization
Subgradient
k +1
k
k
k
max
f
(
x
)
;
x
:
=
x
+
θ
∇
f
(
x
)
• Gradient methods:
n
x∈
"Let θ s.t.: lim
f ( x + θ d ) − f ( x)
θ
θ →∞
= ∇f ( x)d then
if ∇f ( x)d > 0, there exists a small enough θ > 0
s.t.: f ( x + θ d ) > f ( x)"
L( µ ) ; µ
• Subgradient methods: µmax
≥ 0,∈
m
k +1
:=  µ + θ ∂L( µ ) 
k
k
k
+
k
– Subgradient ∂L( µ k ) : let x be unique opt. sol of L( µ k )
L( µ k ) = min {c ' x + µ ( Ax − b) : x ∈ X } = c ' x k + µ ( Ax k − b )
⇒ ∂L( µ k ) =
xk
unique sol.
= ∇L( µ k ) = ( Ax k − b )
⇒ µ k +1 :=µ k + θ k ( Ax k − b )
F.-Javier Heredia http://gnom.upc.edu/heredia
J. Orlin http://jorlin.scripts.mit.edu/
MCNFP- 27
Lagrangian Relaxation
MEIO/UPC-UB : NETWORK FLOWS
Subgradient Method:
Steplenght θ
• Subgradient methods: max L( µ ) ; µ
µ∈
k +1
m
:=  µ + θ ∂L( µ ) 
k
k
k
+
– Convergence property (Bertsekas, prop. 6.3.1) :
Any dual solution µ* satisfies that
µ k +1 − µ * ≤ µ k − µ *
for any steplength θ k such that
0 < θ < 2  L * − L( µ ) 
– Practical update :
k
θ =
λ UB − L( µ k ) 
λ 0 := 2
k
k
k
Ax − b
k
2
arg min L( µ k )
Ax − b ; x k =
k
2
λ k +1 := λ k 2 if L( µ k ) has not been improved in the last t iterations
F.-Javier Heredia http://gnom.upc.edu/heredia
J. Orlin http://jorlin.scripts.mit.edu/
MCNFP- 28
Lagrangian Relaxation
MEIO/UPC-UB : NETWORK FLOWS
Subgradient Method:
Generic algorithm
Begin
Initialize µ0, λ0, UB and t. k:=0 ;
While some convergence criteria are not met do
Compute the dual function: L( µk )→ xk,
If xk feasible (P) update, if possible, UB;
Compute the subgradient:
∂L(µk )= Axk-b;
θk:= λk [UB- L( µk )] / ||(Axk-b)||2;
Update the dual variables: µ k+1:= [ µ k + θk·(Axk-b) ]+;
If L(µk) not improved in the last t iter: λk+1 := λk / 2;
k:=k+1;
End While
End
Compute the steplength:
F.-Javier Heredia http://gnom.upc.edu/heredia
J. Orlin http://jorlin.scripts.mit.edu/
MCNFP- 29
Lagrangian Relaxation
MEIO/UPC-UB : NETWORK FLOWS
Subgradient Method:
Exemple
• µ0:=0, λ0 :=0.8 and UB:=24 (path 1-3-5-6).
• k=0:
– L(µ0=0) := 3 (x0 =1-2-4-6, infeasible (P)).
– ∂L(µ0 ) := Ax0-b = tp-T = 18 – 14 = 4
– θ0 := λ0 [UB- L( µ0 )] / ||(Ax0-b)||2=0.8*[24-3]/16=1.05
– µ 1 := [ µ 0 + θ0·(Ax0-b) ]+ = [0+1.05*4]+ = 4.2; λ1 := λ0 = 0.8 ;
• k=1:
– L(µ1=4.2) := 15 + 4.2*(10-14) = -1.8 (x1=1-3-2-5-6)
– x1 feasible (P): UB:=15.
– ∂L(µ1 ) := Ax1-b = tp-T = 10 – 14 = -4
– θ1 := λ1 [UB- L( µ1 )] / ||(Ax1-b)||2=0.8*[15-1.8]/16=0.84
– µ 2 := [ µ 1 + θ1·(Ax1-b) ]+ = [4.2+0.84*(-4)]+ = 0.84; λ2 := λ1 = 0.8 ;
F.-Javier Heredia http://gnom.upc.edu/heredia
J. Orlin http://jorlin.scripts.mit.edu/
MCNFP- 30
Lagrangian Relaxation
MEIO/UPC-UB : NETWORK FLOWS
Subgradient Method:
Complete execution
F.-Javier Heredia http://gnom.upc.edu/heredia
J. Orlin http://jorlin.scripts.mit.edu/
MCNFP- 31
Lagrangian Relaxation
MEIO/UPC-UB : NETWORK FLOWS
Subgradient Method:
Embedded Network Structure
Networks with side
constraints
minimum cost flows
shortest paths
Traveling Salesman
Problem
assignment problem
minimum cost spanning tree
Vehicle routing
assignment problem
variant of min cost spanning tree
Network design
shortest paths
Two-duty operator
scheduling
shortest paths
minimum cost flows
Multi-time
production planning
shortest paths / DPs
minimum cost flows
F.-Javier Heredia http://gnom.upc.edu/heredia
J. Orlin http://jorlin.scripts.mit.edu/
MCNFP- 32
Lagrangian Relaxation
MEIO/UPC-UB : NETWORK FLOWS
Application
INPUT: n cities, denoted as 1, . . . , n
cij = travel distance from city i to city j
OUTPUT: A minimum distance tour.
F.-Javier Heredia http://gnom.upc.edu/heredia
J. Orlin http://jorlin.scripts.mit.edu/
MCNFP- 33
Lagrangian Relaxation
MEIO/UPC-UB : NETWORK FLOWS
Travelling Salesman Problem (TSP)
• A collection of arcs is a tour if
1. There are two arcs incident to each node
2. The red arcs (those not incident to node 1) form
a spanning tree in G\1.
1
F.-Javier Heredia http://gnom.upc.edu/heredia
J. Orlin http://jorlin.scripts.mit.edu/
MCNFP- 34
Lagrangian Relaxation
MEIO/UPC-UB : NETWORK FLOWS
Representing the TSP problem
Let A(j) be the arcs incident to node j.
Let X denote all 1-trees, that is, there are two arcs incident to
node 1, and deleting these arcs leaves a tree.
∑cx
=
∑ x
z * = min
e
e
e∈A(j)
e
e
2=
for each j 1 to n
P
x∈ X
L( µ ) = min
µ
c
∑ e e xe − 2 n
x∈ X
P(µ)
where for e = (i,j),
ceµ = ce + µ i + µ j
F.-Javier Heredia http://gnom.upc.edu/heredia
J. Orlin http://jorlin.scripts.mit.edu/
MCNFP- 35
Lagrangian Relaxation
MEIO/UPC-UB : NETWORK FLOWS
A Lagrangian Relaxation for the TSP
• This Lagrangian Relaxation was formulated by
Held and Karp [1970 and 1971].
• Seminal paper showing how useful Lagrangian
Relaxation is in integer programming.
• The solution to the Lagrange Multiplier Problem
gives an excellent solution, and it tends to be
“close” to a tour.
F.-Javier Heredia http://gnom.upc.edu/heredia
J. Orlin http://jorlin.scripts.mit.edu/
MCNFP- 36
Lagrangian Relaxation
MEIO/UPC-UB : NETWORK FLOWS
More on the TSP
1
F.-Javier Heredia http://gnom.upc.edu/heredia
J. Orlin http://jorlin.scripts.mit.edu/
MCNFP- 37
Lagrangian Relaxation
MEIO/UPC-UB : NETWORK FLOWS
An optimal spanning tree for the Lagrangian problem
L(µ*) for optimal µ* usually has few leaf nodes.
1
S
In a tour, the number of arcs with both
endpoints in S is at most |S| - 1 for |S| < n
F.-Javier Heredia http://gnom.upc.edu/heredia
J. Orlin http://jorlin.scripts.mit.edu/
MCNFP- 38
Lagrangian Relaxation
MEIO/UPC-UB : NETWORK FLOWS
Towards another Lagrangian Relaxation
∑cx
=
∑ x
z * = min
e
e
e
e∈A(j)
∑
L( µ ) = min
e∈S
e
2=
for each j 1 to n
xe ≤ | S | −1 for each strict subset S of N
µ
c
∑ e e xe − 2 n
∑
∑
e∈S
e
xe ≤ | S | −1 for each strict subset S of N
xe = n
µ
c
where for e = (i,j), e = ce + µ i + µ j
A surprising fact: this relaxation gives exactly the
same bound as the 1-tree relaxation for each µ.
F.-Javier Heredia http://gnom.upc.edu/heredia
J. Orlin http://jorlin.scripts.mit.edu/
MCNFP- 39
Lagrangian Relaxation
MEIO/UPC-UB : NETWORK FLOWS
Another Lagrangian Relaxation for the TSP
1
1
2
aij = the amount of
processing time of
job i on machine j
2
3
3
4
4
xij = 1 if job i is processed
on machine j
= 0 otherwise
Job i gets processed.
5
Set I of
jobs
Set J of
machines
Machine j has at most dj
units of processing
F.-Javier Heredia http://gnom.upc.edu/heredia
J. Orlin http://jorlin.scripts.mit.edu/
MCNFP- 40
Lagrangian Relaxation
MEIO/UPC-UB : NETWORK FLOWS
Generalized assignment problem ex. 16.8
Ross and Soland [1975]
Minimize
∑ ∑
i∈I
c x
j∈J ij ij
=
∑ j∈J xij 1
∑
a x ≤ dj
i∈I ij ij
xij ≥ 0 and integer
(16.10a)
for each i ∈ I
(16.10b)
for each j ∈ J
(16.10c)
for all ( i , j ) ∈ A
(16.10d)
Generalized flow with integer constraints.
Class exercise: write two different Lagrangian relaxations.
F.-Javier Heredia http://gnom.upc.edu/heredia
J. Orlin http://jorlin.scripts.mit.edu/
MCNFP- 41
Lagrangian Relaxation
MEIO/UPC-UB : NETWORK FLOWS
Generalized assignment problem ex. 16.8
Ross and Soland [1975]
Consider a set J of potential facilities
• Opening facility j ∈ J incurs a cost Fj.
• The capacity of facility j is Kj.
Consider a set I of customers that must be served
• The total demand of customer i is di.
• Serving one unit of customer i’s from location j
costs cij.
customer
potential facility
F.-Javier Heredia http://gnom.upc.edu/heredia
J. Orlin http://jorlin.scripts.mit.edu/
MCNFP- 42
Lagrangian Relaxation
MEIO/UPC-UB : NETWORK FLOWS
Facility Location Problem ex. 16.9
Erlenkotter 1978
F.-Javier Heredia http://gnom.upc.edu/heredia
J. Orlin http://jorlin.scripts.mit.edu/
MCNFP- 43
Lagrangian Relaxation
MEIO/UPC-UB : NETWORK FLOWS
A pictorial representation
F.-Javier Heredia http://gnom.upc.edu/heredia
J. Orlin http://jorlin.scripts.mit.edu/
MCNFP- 44
Lagrangian Relaxation
MEIO/UPC-UB : NETWORK FLOWS
A possible solution
Formulate the facility location problem as an integer
program. Assume that a customer can be served by
more than one facility.
Suggest a way that Lagrangian Relaxation can be
used to help solve this problem.
Let xij be the amount of demand of customer i
served by facility j.
Let yj be 1 if facility j is opened, and 0 otherwise.
F.-Javier Heredia http://gnom.upc.edu/heredia
J. Orlin http://jorlin.scripts.mit.edu/
MCNFP- 45
Lagrangian Relaxation
MEIO/UPC-UB : NETWORK FLOWS
Class Exercise
Minimize
∑∑ c
i∈I j∈J
subject to
∑x
j∈J
ij
i∈I
xij + ∑ F j y j
j∈J
=1
∑d x
i
ij
ij
≤ K j yj
for all i ∈ I
for all j ∈ J
0 ≤ xij ≤ 1
for all i ∈ I and j ∈ J
y j = 0 or 1
for all j ∈ J
F.-Javier Heredia http://gnom.upc.edu/heredia
J. Orlin http://jorlin.scripts.mit.edu/
MCNFP- 46
Lagrangian Relaxation
MEIO/UPC-UB : NETWORK FLOWS
The facility location model
Theorem 16.6. Suppose that we apply the
Lagrangian relaxation technique to solving a linear
program P’ defined as
minimize
subject to
cx
Ax = b
Dx ≤ q
x≥0
P’
By relaxing the constraint Ax = b. Then the optimal
value L* of the Lagrangian multiplier problem equals
the optimal objective value of P’.
Proof. See AMO.
F.-Javier Heredia http://gnom.upc.edu/heredia
J. Orlin http://jorlin.scripts.mit.edu/
MCNFP- 47
Lagrangian Relaxation
MEIO/UPC-UB : NETWORK FLOWS
Lagrangian Relaxation and LPs
• Advantage of solving the Lagrangian Relaxation
• The relaxed problem may be much easier to
solve
• The relaxed problem may decompose into
smaller problems
• Comment on Theorem 16.6
• We will use it to develop a stronger result on
Lagrangian Relaxations based on convex hulls.
F.-Javier Heredia http://gnom.upc.edu/heredia
J. Orlin http://jorlin.scripts.mit.edu/
MCNFP- 48
Lagrangian Relaxation
MEIO/UPC-UB : NETWORK FLOWS
On Lagrangian Relaxation and LPs
Suppose that X = {x1, x2, …, xK} is a finite set.
Vector x is a convex combination of X = {x1, x2, …, xK}
if there is a feasible solution to
x = ∑ k =1 λk x k
K
∑
K
k =1
λk = 1
λk ≥ 0 for k =
1 to K
The convex hull of X is H(X) = { x : x can be
expressed as a convex combination of points in X.}
F.-Javier Heredia http://gnom.upc.edu/heredia
J. Orlin http://jorlin.scripts.mit.edu/
MCNFP- 49
Lagrangian Relaxation
MEIO/UPC-UB : NETWORK FLOWS
Convex Hulls
The convex Hull H(X) is the smallest LP
feasible region that contains all points of X.
F.-Javier Heredia http://gnom.upc.edu/heredia
J. Orlin http://jorlin.scripts.mit.edu/
MCNFP- 50
Lagrangian Relaxation
MEIO/UPC-UB : NETWORK FLOWS
Convex Hulls in two dimensions
1. The set H(X) is a polyhedron, that is, it can be
expressed as H(X) = {x : Ax ≤ b} for some matrix A
and vector b.
2. Each extreme point of H(X) is in X. If we minimize
{cx : x ∈ H(X)}, the optimum solution lies in X.
3. Suppose X ⊆ Y = {x : Dx ≤ c and x ≥ 0}.
Then H(X) ⊆ Y.
F.-Javier Heredia http://gnom.upc.edu/heredia
J. Orlin http://jorlin.scripts.mit.edu/
MCNFP- 51
Lagrangian Relaxation
MEIO/UPC-UB : NETWORK FLOWS
Property 16.7
The optimal objective function value L* of the
Lagrangian multiplier problem equals the optimal
objective function value of the linear program
minimize
subject to
cx
Ax = b
x ∈ H(X )
minimize
subject to
cx
Ax = b
x∈ X
P*
P
First, note that P* ≠ P
F.-Javier Heredia http://gnom.upc.edu/heredia
J. Orlin http://jorlin.scripts.mit.edu/
MCNFP- 52
Lagrangian Relaxation
MEIO/UPC-UB : NETWORK FLOWS
Theorem 16.8
max x1
s.t.
x1 – x2 ≤ 1
x ∈ H(X)
is different
from
max x1
s.t.
x1 – x2 ≤ 1
x∈X
x1
F.-Javier Heredia http://gnom.upc.edu/heredia
J. Orlin http://jorlin.scripts.mit.edu/
MCNFP- 53
Lagrangian Relaxation
MEIO/UPC-UB : NETWORK FLOWS
Graphical representation
minimize
subject to
cx
Ax = b
x ∈ H(X )
P*
Proof. For multiplier µ, the Lagrangian Relaxation is:
=
L( µ ) min
cx + µ ( Ax − b )
subject to x ∈ X
which by Property 16.7 is equivalent to:
=
L( µ ) min
cx + µ ( Ax − b )
subject to x ∈ H ( X )
F.-Javier Heredia http://gnom.upc.edu/heredia
J. Orlin http://jorlin.scripts.mit.edu/
(16.2)
MCNFP- 54
Lagrangian Relaxation
MEIO/UPC-UB : NETWORK FLOWS
Theorem 16.8. The optimal objective function value L* of
the Lagrangian multiplier problem equals the optimal
objective function value of the linear program P*.
(16.2)
(16.2) can be expressed as a linear program.
Moreover, (16.2) is a Lagrangian relaxation for:
minimize
subject to
cx
Ax = b
x ∈ H(X )
P*
By Theorem 16.6, the optimal solution for the
Lagrangian multiplier problem for P is equal to the
optimal solution value for P*.
F.-Javier Heredia http://gnom.upc.edu/heredia
J. Orlin http://jorlin.scripts.mit.edu/
MCNFP- 55
55
Lagrangian Relaxation
MEIO/UPC-UB : NETWORK FLOWS
L( µ ) min
cx + µ ( Ax − b )
=
subject to x ∈ H ( X )
MEIO/UPC-UB : NETWORK FLOWS
Theorem 16.9
Suppose we solve the Lagrangian multiplier
problem for
Proof.
min
cx
Ax = b
Dx ≤ q
x ≥ 0 x integer
Then z0 ≤ L*, where
z0 =
min
s.t
cx
Ax = b
Dx ≤ q
x≥0
L*=
min
s.t
cx
Ax = b
x ∈ H(X )
H(X ) =
H ({ x : Dx ≤ q , x ≥ 0, x integer})
⊆ { x : Dx ≤ q , x ≥ 0}
F.-Javier Heredia http://gnom.upc.edu/heredia
J. Orlin http://jorlin.scripts.mit.edu/
MCNFP- 56
Lagrangian Relaxation
s.t.
The feasible region
for Dx ≤ q, x ≥ 0
The feasible region
for x ∈ H(X)
The feasible points
for
Dx ≤ q, x ≥ 0,
x integer
are in black.
F.-Javier Heredia http://gnom.upc.edu/heredia
J. Orlin http://jorlin.scripts.mit.edu/
MCNFP- 57
Lagrangian Relaxation
MEIO/UPC-UB : NETWORK FLOWS
An Example of Theorem 16.9
Suppose X = {x : Dx ≤ q , x ≥ 0, x integer}.
We say that X satisfies if the integrality property if the
following LP has integer solutions for all d
minimize
subject to
dx
Dx ≤ q
x≥0
Fact: if X satisfies the integrality property then
min (dx: Dx ≤ q , x ≥ 0)
= min (dx : x ∈ X)
= min (dx : x ∈ H(x))
F.-Javier Heredia http://gnom.upc.edu/heredia
J. Orlin http://jorlin.scripts.mit.edu/
MCNFP- 58
Lagrangian Relaxation
MEIO/UPC-UB : NETWORK FLOWS
Integrality Property
If the Lagrangian subproblem of the optimization
problem P satisfies the integrality property then
L* = z0, the solution to the LP relaxation of P.
F.-Javier Heredia http://gnom.upc.edu/heredia
J. Orlin http://jorlin.scripts.mit.edu/
MCNFP- 59
Lagrangian Relaxation
MEIO/UPC-UB : NETWORK FLOWS
Theorem 16.10
∑ ∑
=
∑ x
Minimize
c x
j∈J ij ij
i∈I
j∈J
∑
ij
1
a x ≤ dj
i∈I ij ij
xij ≥ 0 and integer
(16.10a)
for each i ∈ I
(16.10b)
for each j ∈ J
(16.10c)
for all ( i , j ) ∈ A
(16.10d)
If we relax (16.10c), the bound for the Lagrangian multiplier
problem is the same as the bound for the LP relaxation.
If we relax (16.10b), the LP does not satisfy the integrality
property, and we should get a better bound than z0.
F.-Javier Heredia http://gnom.upc.edu/heredia
J. Orlin http://jorlin.scripts.mit.edu/
MCNFP- 60
Lagrangian Relaxation
MEIO/UPC-UB : NETWORK FLOWS
Example: Generalized Assignment