Approximation Algorithms
Thomas Rothvo
Institute of Mathematis
EPFL, Lausanne
Spring 2010
1 / 292
Introdution (page 3)
Steiner tree (page 8)
k-Center (page 13)
Traveling Salesman Problem (page 22)
The Capaitated Vehile Routing Problem (page 31)
Set Cover (page 35)
Set Cover via LPs (page 38)
Insertion: Linear Programming (page 44)
Weighted Vertex Cover (page 60)
Insertion: Algorithmi probability theory (page 66)
Minimizing Congestion (page 75)
Knapsak (page 91)
Multi Constraint Knapsak (page 96)
Bin Paking (page 102)
The algorithm of Karmarkar & Karp (page 109)
Minimum Makespan Sheduling (page 128)
Sheduling on Unrelated Parallel Mahines (page 134)
Multiproessor Sheduling with Preedene Constraints (page 143)
Eulidean TSP (page 148)
Tree Embeddings (page 174)
Introdution into Primal dual algorithms (page 199)
Steiner Forest (page 204)
Faility Loation (page 227)
Insertion: Semidenite Programming (page 244)
MaxCut (page 254)
Max2Sat (page 263)
Budgeted Spanning Tree (page 270)
k-Median (page 281)
2 / 292
Part 1
Introdution
Soure:
Approximation Algorithms (Vazirani, Springer Press)
3 / 292
Why approximation algorithms?
Task: Solve NP-hard optimization problem A
! no eÆient algorithm (unless NP = P)
Possible approahes:
exponential time algorithms ! some theory but too slow
and no lower bounds
I heuristi ! fast, easy but no guarantee, not muh theory
I approximation algorithms ! rih theory in many ases
good lower bounds
Running times: n = number of objets in instane, B biggest
appearing number, " > 0 onstant
I exponential: 2n ; n B
I polynomial: n2 ; n100 ; n log B; n 21=" ; nO(1=")
I
O (1=")
4 / 292
Basi denitions
Denition
Let be an optimization problem and I is instane for A.
Then OP T(I ) is the value of the optimum solution.
Denition
Let 1. A is an -approximation algorithm for a
minimization problem if
A(I ) OP T (I ) 8 instanes I
where A(I ) is the value of the solution, that A returns for I .
I Typial values for : 1:5; 2; O(1); O(log n)
I Usually we omit and I in OP T (I )
I For a maximization problem: A(I ) 1 OP T (I )
I Attention: Sometimes in literature < 1 for maximization
problems. For example 12 -apx means A(I ) 21 OP T(I )
5 / 292
Denition PTAS
Denition
A" is a polynomial time approximation sheme (PTAS) for a
minimization problem if
A" (I ) (1 + ") OP T (I ) 8 instanes I
and for every xed " > 0, the running time of A" is polynomial
in the input size.
Typial running times: O(n="); 21=" n2 log2 (B ); nO(1=")
O (1=")
6 / 292
Denition FPTAS
Denition
A" is a fully polynomial time approximation sheme (FPTAS)
for a minimization problem if for every " > 0
A" (I ) (1 + ") OP T (I )
8 instanes I
and the running time of A" is polynomial in the input size and
1=".
I
Typial running time: O(n3 ="2 )
7 / 292
Part 2
Steiner tree
Soure:
Approximation Algorithms (Vazirani, Springer Press)
8 / 292
Steiner Tree
Problem: Steiner tree
I Given: Undireted graph G = (V; E ), metri ost funtion
: E ! Q + , terminals R V
I Find: Minimum ost tree T onneting all terminals R:
OP T
= minf(T ) j T spans Rg
P
I (T ) := e2T e
I metri: 8u; v; w 2 V : uw uv + vw
(triangle inequality)
Steiner tree
Steiner node
terminal
9 / 292
Steiner tree (2)
Fat
If R = V , then Steiner Tree is just the Minimum Spanning
Tree Problem whih an be solved optimally by piking
greedily the heapest edges (without losing a yle).
Algorithm:
(1) Compute the minimum spanning tree T on R
(2) Return T
Theorem
The algorithm gives a 2-approximation.
10 / 292
Proof of approximation guarantee
I Claim: 9 spanning tree of ost 2 OP T
I Let T be optimum Steiner tree
I Double the edges of T I Observe: Degrees now even ) 9 Euler tour E visiting eah
terminal
Theorem (Euler)
Given an undireted, onneted graph G = (V; E ). Then G has
an Euler tour (tour ontaining eah edge exatly one) if and
only if jÆ(v)j is even for all v 2 V .
I
I
Shortut E suh that eah terminal is visited one
Remove an edge ) spanning tree of ost 2 (T )
T
)
)
11 / 292
State of the art
Known results:
I There is a 1.39-approximation.
I For quasi-bipartite graphs (no Steiner nodes inident):
1:22-apx
I No < 96
95 -apx unless NP = P.
12 / 292
k
Soure:
Part 3
-Center
Approximation Algorithms (Vazirani, Springer Press)
13 / 292
k
-Center
Problem: k-Center
I Given: Undireted, metri graph G = (V; E ), k 2 N .
Dene
`(v; F ) := min uv
u2F
I
Find: k many enters F V that minimize the maximum
distane from any v 2 V to the nearest enter:
OP T = min maxf`(v; F )g
F V;jF j=k v2V
14 / 292
The algorithm
Algorithm:
(1) Guess OP T
(2) F := ;
2 fuv j u; v 2 V g
(3) REPEAT
(4) IF 9v 2 V : `(v; F ) > 2 OP T THEN F := F [ fvg
ELSE RETURN F
15 / 292
The algorithm
Algorithm:
(1) Guess OP T
(2) F := ;
2 fuv j u; v 2 V g
(3) REPEAT
(4) IF 9v 2 V : `(v; F ) > 2 OP T THEN F := F [ fvg
ELSE RETURN F
2 OP T
F
3
b
16 / 292
The algorithm
Algorithm:
(1) Guess OP T
(2) F := ;
2 fuv j u; v 2 V g
(3) REPEAT
(4) IF 9v 2 V : `(v; F ) > 2 OP T THEN F := F [ fvg
ELSE RETURN F
2 OP T
F
3
b
b
2F
17 / 292
Guessing
For simpliity we sometimes guess parameters:
Algorithm with guessing:
(1) Guess a parameter m
(2) : : : ompute a solution S
(3) return S
using m : : :
Algorithm without guessing:
(1) FOR all hoies of m DO
(2) : : : ompute a solution S (m) : : :
(3) return the best found solution S (m)
I
I
Still polynomial if the domain of m is polynomial
Typial guesses: OP T , O(1) many nodes in a graph
18 / 292
The analysis
Theorem
One has jF j k and `(v; F ) 2 OP T for all v 2 V .
I `(v; F ) 2 OP T , otherwise algo would not have stopped.
I Remains to show jF j k.
I Let F V; jF j = k be optimum solution.
I Observe: uv > 2 OP T 8u; v 2 F : u 6= v
I Hene the enters in F that serve u and v must be
dierent ) jF j jF j k.
2 OP T
b
OP T
b
b
b
b
opt enters F 19 / 292
Dominating Set
Problem: Dominating Set
I Given: Undireted graph G = (V; E )
I Find: Dominating set U V of minimum size
OP TDS = minfjU j j U
V; U [
[
u2U
Æ(u) = V g
dominating set
Theorem
Given (G; k), it is NP-hard to deide, whether OP TDS
k.
20 / 292
Hardness of k-Center
Theorem
Unless NP = P, for all " > 0, there is no (2 ")-approximation
algorithm for k-Center.
Let (G; k) be DominatingSet instane.
Suppose A is a (2 ")-algorithm for k-Center
Dene omplete graph G0 on nodes V with
(
1 (u; v) 2 E
(u; v) :=
2 otherwise
I 9 DS of size k ) k-Center solution with value 1
I 9k-Center solution with value 1 ) 9 DS of size k
I Run A on G0 :
I
I
I
I A(G0 ) < 2 ) A(G0 ) = 1 ) answer to DS instane is YES
I A(G0 ) 2 ) answer is NO
21 / 292
Part 4
Traveling Salesman Problem
Soure:
Approximation Algorithms (Vazirani, Springer Press)
22 / 292
TSP
Problem: Traveling Salesman Problem (Tsp)
I Given: Undireted graph G = (V; E ) with metri ost
: E ! Q+
I Find: Minimum ost tour visiting all nodes
min
tour :V !V
nX
v2V
(v; (v))
o
23 / 292
A 2-approximation for TSP
Algorithm:
(1)
(2)
(3)
(4)
Compute an MST T on G
Double the edges in T
Compute Euler tour E using edges in T
Shortut to obtain a tour Theorem
Algorithm yields a 2-apx.
Let be optimum tour
9 a spanning tree on G of ost (T ) OP T (just delete an
arbitrary edge from )
I Degrees are even after doubling, hene E exists and
(E ) 2 OP T
I () 2 OP T (G is metri, hene shortutting does not
inrease the ost)
I
I
24 / 292
A 3=2-approximation for TSP
Algorithm (Christodes):
(1) Compute an MST T
(2)
(3)
(4)
Find min ost perfet mathing M on nodes V odd V
with odd degree in T
Find Euler tour in T [ M .
Return obtained by shortutting the Euler tour
M
M
3
3
2 V odd
T
Reminder
A perfet mathing in an undireted graph G0 = (V 0; E 0 ) is an
edge set M E 0 with jÆM (v)j = 1 8v 2 V 0. The heapest
perfet mathing an be found in poly-time.
25 / 292
A 3=2-approximation for TSP (2)
Theorem
The algorithm gives a 3=2-apx.
I Again (T ) OP T
I V odd := fv 2 V j jÆT (v)j oddg.
I Claim: jV odd j is even beause
jV odd j 2
X
v2V
2T
odd
jÆT (v)j 2
X
v2V
jÆT (v)j 2 0
V odd
26 / 292
A 3=2-approximation for TSP (3)
I Let be optimum tour. Obtain shortutted tour odd on
V odd : (odd ) OP T .
I Partition odd into 2 mathings M1 ; M2 on V odd
I Let M 2 fM1 ; M2 g be the heaper of both mathings
I (M ) 21 (odd ) 21 OP T
I In T [ M all nodes have even degree, hene T [ M ontains
an Euler tour of ost (T ) + (M ) 32 OP T .
27 / 292
A 3=2-approximation for TSP (3)
I Let be optimum tour. Obtain shortutted tour odd on
V odd : (odd ) OP T .
I Partition odd into 2 mathings M1 ; M2 on V odd
I Let M 2 fM1 ; M2 g be the heaper of both mathings
I (M ) 21 (odd ) 21 OP T
I In T [ M all nodes have even degree, hene T [ M ontains
an Euler tour of ost (T ) + (M ) 32 OP T .
odd
28 / 292
A 3=2-approximation for TSP (3)
I Let be optimum tour. Obtain shortutted tour odd on
V odd : (odd ) OP T .
I Partition odd into 2 mathings M1 ; M2 on V odd
I Let M 2 fM1 ; M2 g be the heaper of both mathings
I (M ) 21 (odd ) 21 OP T
I In T [ M all nodes have even degree, hene T [ M ontains
an Euler tour of ost (T ) + (M ) 32 OP T .
2 M2 M1 3
M1 3
2 M2
29 / 292
Open Problems on TSP
Open Problem
I Is there a < 3=2-apx for TSP?
I Held-Karp LP relaxation is onjetured to have integrality
gap 4=3.
5381
I No ( 5380
")-apx even if e 2 f1; 2g
30 / 292
Part 5
The Capaitated Vehile Routing
Problem
Soure:
Bounds and Heuristis for apaitated routing problems
(Haimovih, Rinnooy Kan)
http://www.jstor.org/stable/3689422
31 / 292
The Capaitated Vehile Routing Problem
Problem: Cvrp
I Given: Undireted graph G = (C [ frg; E ) with metri
osts : E ! Q + , depot r, lients C and vehile apaity k
I Find: A tour of minimal ost whih visits all lients at
least one, but must revisit the depot after eah k lient
visits
lient
r
k lients
Assume: jC j = Z k (otherwise add lients at the depot)
32 / 292
A 5=2-apx for Cvrp
Algorithm:
(1)
(2)
(3)
(4)
Compute a 3=2-approximate TSP tour on lients
Let v0; : : : ; vn 1 be lients in visiting order
Choose randomly a starting node vi
Starting from vi revisit r every k many lients (i.e.
augment the tour with edges r ! vi; vi 1 ! r if i k i ) to
obtain a Cvrp solution 0
vi +k 1
:::
vi +1
0
r
vi
vi 1
33 / 292
The analysis
Lemma
E [AP X ℄ 52 OP T
I Opt. TSP tour osts OP TTSP OP T
I Pr[need edge (r; vi )℄ = k2
P
I E [AP X ℄ () + k2 v2C (r; v)
I
Look at a subtour in optimum Cvrp
solution. Send k=2 lients
[ounter-℄lokwise to r: edges in
subtour
used k=2 times
P
) v2C (v; r) k2 OP T
E [AP X ℄ () +
hene () 32 OP T
r
subtour
2 X (r; v) 3 OP T + 2 k OP T = 5 OP T
k v2C
2
k 2
2
34 / 292
Part 6
Set Cover
Soure:
Approximation Algorithms (Vazirani, Springer Press)
35 / 292
Set Cover
Problem: Set Cover
I Given: Elements U := f1; : : : ; ng, sets S1 ; : : : ; Sm U
ost (Si )
I Find:
OP T
= I f1min
;:::;mg
nX
i2I
(Si ) j
[
i2I
Si = U
with
o
Greedy algorithm:
(1) I := ;
(2) WHILE not yet all elements overed DO
(3) prie(S ) := jS n S(S ) S j
i2I i
(4) I := I [ f set S with minimum prie(S )g
Theorem
The greedy algorithm yields a O(log n)-approximation.
36 / 292
Analysis
I Let e1 ; : : : ; en be elements in the order of overing.
I Suppose S (S 2 I ) newly overed ek ; : : : ; e`
z
n k+1 elements
}|
{
e1 ; e2 ; e3 ; : : : ; ek ; : : : ; ej ; : : : ; e` ; : : : ; en
|
{z
overed by S
}
Dene prie(ej ) := prie(S ) for j 2 fk; : : : ; `g.
Consider the iteration, when S was hosen: Still n k + 1
elements where unovered and it was still possible to over
them all at ost OP T . Sine S minimizes the prie:
OP T
OP T
prie(ej ) = prie(ek ) n k+1 n j +1
I Finally
n
n
n
X
X
X
OP T
AP X = prie(ej ) = OP T 1j = O(log n)OP T
n j +1
I
I
j =1
j =1
j =1
37 / 292
Part 7
Set Cover via LPs
Soure:
Approximation Algorithms (Vazirani, Springer Press)
38 / 292
A linear program for SetCover
Introdue deision variables
(
1 take set Si
xi =
0 otherwise
Formulate SetCover as integer linear program:
min
m
X
i=1
(Si )xi
X
i:j 2Si
I
(ILP )
xi
1 8j 2 U
xi
2 f0; 1g 8i
Cheapest Set Cover solution = best (ILP ) solution
39 / 292
The LP relaxation
We relax this to a linear program
min
m
X
i=1
(Si )xi
X
i:j 2Si
xi
(LP )
1 8j 2 U
0 xi 1 8i
I (LP ) an be solved in polynomial time (see next hapter)
I Let OP Tf be value of optimum solution
I Of ourse OP Tf OP T
I Integrality gap
(n) :=
OP T (I )
instanes jIj=n OP Tf (I )
sup
40 / 292
The algorithm
Algorithm:
(1) Solve (LP ) ! x opt. frational solution
(2) (Randomized rounding:) FOR i = 1; : : : ; m DO
(3) Pik Si with probability minfln(n) xi ; 1g
(4) (Repairing:) FOR every not overed element j 2 U
heapest set ontaining j
pik the
41 / 292
Analysis
Theorem
E [AP X ℄ (ln(n) + 1) OP Tf
Consider an element j 2 U :
Pr[j not overed in (2)℄ =
1+yey
=
Y
i:j 2Si
Y
i:j 2Si
Y
i:j 2Si
Pr[Si not piked in (2)℄
(1 ln(n) xi )
e ln(n)xi
1 due to LP ineq.
zP }|
{
e ln(n) i:j 2Si xi
1
e ln(n) =
n
42 / 292
Analysis (2)
I Cost of randomized rounding:
E [ost in (2)℄
=
I
m
X
i=1
m
X
i=1
Pr[Si piked in (2)℄ (Si )
ln(n)xi (Si ) = ln(n) OP Tf
Cost of repairing step: In step (3), we pik n times with
prob. n1 a set of ost OP Tf . Hene
1
E [ost of step (3)℄ n OP Tf = OP Tf
n
I
By linearity of expetation
E [AP X ℄ = E [ost in (2)℄+E [ost in (3)℄ (ln(n)+1)OP Tf
43 / 292
Part 8
Insertion: Linear Programming
Soure:
Geometri Algorithms and Combinatorial Optimization
(Grotshel, Lovasz, Shrijver)
44 / 292
Linear programs
Let A 2 Rmn ; b 2 Rm ; 2 Rn then
max T x
Ax
xi
x2
opt. sol.
b
0 8i
aTi x bi
x1
is alled a linear program. Alternatively one might have
I min instead of max
I no non-negativity xi 0
I Ax = b
More terminology
I onv(fx; yg) := fx + (1 )y j 2 [0; 1℄g
I Set Q Rn onvex if 8x; y 2 Q : onv(fx; yg) Q
I A set P is alled a polyhedron if P = fx 2 Rn j Ax bg
I If P bounded (9M : P [ M; M ℄n ) then P is a polytope.
45 / 292
Verties
Let P = fx 2 Rn j Ax bg be a polyhedron.
Denition
A point x 2 P is alled a vertex if there is a 2 Rn suh that
x is the unique optimum solution of maxfT x j x 2 P g.
Alternative names: basi solution, extreme point.
x2
x P
x1
46 / 292
Alternative haraterisations
Lemma
Let x 2 P = fx 2 Rn j Ax bg. The following statements are
equivalent
I x is a vertex
I There are no y; z 2 P with (x ; y; z pairwise dierent) and
x 2 onvfy; z g
I There is a linear independent subsystem A0 x b0 (with n
onstraints) of Ax b s.t. fx g = fx 2 Rn j A0 x = b0 g.
x2
x
P
aTi x bi
aTj x bj
x1
47 / 292
Not every polyhedron has verties
Example: The polyhedron P = fx 2 R2 j x1 + x2 1g does
not have any verties.
x1 + x2 1
x2
P
x1
Lemma
Any polytope has verties.
Lemma
Any polyhedron P
Rn with non-negativity onstraints
xi 0 8i = 1; : : : ; n has verties.
48 / 292
Support of vertex solutions
Lemma
Let x be a vertex of
= fx 2 Rn j aTj x bj 8j = 1; : : : ; m; xi 0 8ig
Then jfi j xi > 0gj m (#non-zero entries #onstraints).
P
(0; 12 )
x2
P
=
x 2 R2
P
4
x
+
6
x
3
1
2
j x1 0; x2 0
x1
(0; 0)
Proof: There is a subsystem I; J with jJ j + jI j = n and
fx g = fx j aTj x = bj 8j 2 J ; xi = 0 8i 2 I g. Hene
jI j = n jJ j n m.
( 34 ; 0)
49 / 292
Linear programming is doable in polytime
Theorem
Given A 2 Q mn ; b 2 Q m ; 2 Q n , there is an algorithm whih
solves
maxfT x j Ax bg
in time polynomial in n; m and the enoding length of A; b; .
The algorithm returns an optimum vertex solution if there is
any.
Polynomial here means that the number of bit operations is
bounded by a polynomial (Turing model).
I Enoding length (= #bits used to enode an objet) for
I
I
I
I
I
I
integer 2 Z: hi := dlog2 (jj + 1)e + 1.
rational number = pqP2 Q : hi := hpi + hqi
vetor 2 Q n : hi := ni=1 hi i
inequality aT x Æ: hai + hÆi
P P
matrix A = (aij ) 2 Q mn : hAi := mi=1 nj=1 haij i
50 / 292
The ellipsoid method
Input: Fulldimensional polytope P Rn
Output: Point in P
(1) Find ellipsoid E1 P with enter z1
(2) FOR t = 1; :::; 1 DO
(3) IF zt 2 P THEN RETURN zt
(4) Find hyperplane aT x = Æ through zt suh that
P
fx j aT x < Æg
(5) Compute ellipsoid Et+1 Et \ fx j aT x Æg with
vol(Et+1 ) = (1 (1)
n )vol(Et )
zt
Et
P
zt+1
Et+1
aT x = Æ
51 / 292
The ellipsoid method (2)
Problem: Separation Problem for P :
I Given: y 2 Q n
I Find: a 2 Q n with aT y > aT x 8x 2 P (or assert y 2 P ).
y
P
a
Rule of thumb
If one an solve the Separation Problem for P Rn in
poly-time, then one an solve maxfT x j x 2 P g eÆiently.
Important: The number of inequalities does not play a role.
Espeially we an optimize in many ases even if the number of
inequalities is exponential.
52 / 292
Theorem
Let P Rn be a polyhedron that an be desribed as
P = fx 2 Rn j Ax bg with A 2 Q mn ; b 2 Q m , and let 2 Q n
be an objetive funtion. Let ' be an upper bound on
I the enoding length of eah single inequality in Ax b.
I the dimension n
I the enoding length of .
Suppose one an solve the following problem in time poly('):
Separation problem: Given y 2 Q n with enoding
length poly(') as input. Deide, whether y 2 P . If not
nd an a 2 Q n with aT y > aT x 8x 2 P .
Then there is an algorithm that yields in time poly(') either
I x 2 Q n attaining maxfT x j x 2 P g (x will be a vertex if
P has verties)
I P empty
I Vetors x; y 2 Q n with x + y 2 P 8 0 and T y 1.
Here running times are w.r.t. the Turing mahine model.
53 / 292
Weak duality
Observation
Consider the LP maxfT x j x 2 P g with
P = fx 2 Rn j Ax bg. Let y 0. Then (yT A)x yT b is a
feasible inequality for P (i.e. (yT A)x yT b 8x 2 P ). In fat, if
yT A = T , then
T x = (yT A)x yT b 8x 2 P
Example: maxfx1 + x2 j x1 + 2x2 6; x1 2; x1 x2 1g
Optimum solution: x = (2; 2) with T x = 4.
x1 + x2 133
x
+
2
x
6
1
2
2
3 ( x1 +2x2 6)
x
0 ( x1
2)
1 ( x1
P
x2 1)
3
x1 +x2 133 4:33 x1 x2 1
x 2
1
54 / 292
Weak duality (2)
Theorem (Weak duality)
Let A 2 Rmn ; b 2 Rm ; 2 Rn . Then
fbT y j yT A{z= T ; y 0g}
maxfT x{zj Ax bg} min
|
|
(P )
(D)
given that both systems are feasible.
If (P ) is the primal program, then (D) is the dual program
to (P ).
I Note: The dual of the dual is the primal.
I
55 / 292
Strong duality
Theorem (Strong duality I)
Let A 2 Rmn ; b 2 Rm ; 2 Rn . Then
maxfT x j Ax bg = minfbT y j yT A = T ; y 0g
given that both systems are feasible.
Theorem (Strong duality II)
Let A 2 Rmn ; b 2 Rm ; 2 Rn . Then
maxfT x j Ax b; x 0g = minfbT y j yT A T ; y 0g
given that both systems are feasible.
56 / 292
Hand-waving proof of strong duality
Claim
Let x be optimum solution of maxfT x j Ax bg.
is a y 0 with yT A = T and yT b = T x .
aTi1 x bi1
I
I
Let a1; : : : ; am be rows of A.
Let I := fi j aTi x = big be
the tight inequalities.
Then there
ai1
C
a i2
b
x
b
aTi2 x bi2
I Suppose for ontradition 2= f i ai yi j yi 0; i 2 I g =: C
I Then there is a 2 Rn with T > 0; aTi 0 8i 2 I .
I Walking in diretion improves objetive funtion.
But x was optimal. Contradition!
P
57 / 292
Hand-waving proof of strong duality
Claim
Let x be optimum solution of maxfT x j Ax bg.
is a y 0 with yT A = T and yT b = T x .
aTi1 x bi1
I
I
Let a1; : : : ; am be rows of A.
Let I := fi j aTi x = big be
the tight inequalities.
I 9y 0 : yT A = T
yT b
Then there
ai1
b
x
b
C
a i2
aTi2 x bi2
and yi = 0 8i 2= I (we only use tight
inequalities)
m
X
T
T
T
T
yi |(bi {zaTi x}) = 0
x = y b y Ax = y (b Ax ) =
|{z}
i=1 =0 if i=2I
=0 if i2I
58 / 292
Complementary Slakness
Warning: Primal and dual are swithed here.
Theorem (Complementary slakness)
Let x be a solution for
(P ) : minfT x j Ax b; x 0g
and y a solution for
(D) : maxfbT y j AT y ; y 0g:
Let ai be the ith row of A and aj be its j th olumn. Then x
and y are both optimal , both following onditions are true
I Primal omplementary slakness: xj > 0 ) (aj )T y = j
I Dual omplementary slakness: yi > 0 ) aTi x = bi
59 / 292
Part 9
Weighted Vertex Cover
Soure:
Approximation Algorithms (Vazirani, Springer Press)
60 / 292
Vertex Cover
Problem: Weighted Vertex Cover
I Given: Undireted graph G = (V; E ), node weights
: V ! Q+
I Find: Subset U V suh that every edge is inident to at
least one node in U and Pv2U (v) is minimized.
vertex over
Consider the LP X
min (v)xv
v2V
xu + xv 1 8 (u; v) 2 E
xv 0 8v 2 V
61 / 292
Half-integrality
Lemma
Let x be a basi solution of (LP ). Then xv 2 f0; 12 ; 1g for all
v 2 V , i.e. x is half-integral.
Suppose x is not half-integral, i.e. not both sets are empty:
o
n
n
1o
1
V+ := v j < xv < 1 ; V := v j 0 < xv <
2
2
I It suÆes to show that x an be written as onvex
ombination x = 21 y + 21 z for 2 dierent feasible (LP )
solutions y; z.
I
xv2
V
3 v1
xv1 = 0:3
xv2 = 0:7
v2 2 V +
1
0
y
x
0
(LP )
z
1
xv1
62 / 292
Half-integrality (2)
I Dene
8
>
<xv + " xv 2 V+
and
yv := xv " xv 2 V
>
:
xv
otherwise
V
"
"
V+ xv 2 f0; 21 ; 1g
+"
+0
+"
+0
8
>
<xv
" xv 2 V+
zv := xv + " xv 2 V
>
:
xv
otherwise
V
+"
+"
V+ xv 2 f0; 21 ; 1g
"
+0
"
+0
+"
I Tight edges (u; v) 2 E : xv + xu = 1 drawn solid
I Constraints satised by y; z for " > 0 small enough.
"
63 / 292
The Algorithm
Algorithm:
(1) Compute an optimum basi solution x to (LP )
(2) Choose vertex over U := fv j xv > 0g
Theorem
U is a vertex over of ost
2 OP Tf .
Proof.
Clearly U is feasible. Furthermore
X
X
X
(v) = dxv e(v) 2 xv (v) = 2 OP Tf :
v2U
v2V
v2V
64 / 292
Inapproximability
Theorem (Khot & Regev '03)
There is no polynomial time (2 ")-apx unless Unique Games
Conjeture is false.
Unique Games Conjeture
For all " > 0, there is a prime p := p(") suh that the following
problem is NP-hard:
I Given: Equations xi p aij xj for some (i; j ) pairs
I
Distinguish:
I
I
Yes:
No:
max satisable fration 1 "
max satisable fration "
Example:
x1
x2
13 4 x3
13 9 x1
:::
65 / 292
Part 10
Insertion: Algorithmi probability
theory
Soure:
Probability and Computing (Mitzenmaher & Upfal,
Cambridge Press)
66 / 292
Probability theory
Denition
A (disrete) probability spae onsists of
I A (ountable) sample spae modelling all possible
outomes of a random proess.
I A probability funtion Pr : 2
! R suh that
(a) 0 Pr[E ℄ 1 8E (b) Pr[
℄ = 1
() For any (ountable) sequene of pairwise disjoint events
E1 ; E2 ; : : : Pr
h[ i X
E = Pr[E ℄
i1
i
i1
i
Denition (Random variable)
A funtion X : ! R is alled a random variable.
67 / 292
Probability theory (2)
Denition (Expetation)
Let X : ! R be a random variable.
E [X ℄ =
X
i
Then
i Pr[X = i℄
Lemma (Linearity of expetation)
Let X1 ; : : : ; Xn : ! R random variables with nite
expetations. Then
E
n
hX
i=1
Xi
i
=
n
X
i=1
E [Xi ℄
68 / 292
Probability theory (3)
Lemma (Independene)
Random variables X1 ; : : : ; Xn are alled independent if
8I f1; : : : ; ng : 8xi : Pr
h\
i2I
i
(Xi = xi) =
Y
i2I
Pr[Xi = xi℄
Lemma
Let X1 ; : : : ; Xn independent random variables. Then
E
n
hY
i=1
Xi
i
=
n
Y
i=1
E [Xi ℄
69 / 292
Probability theory (4)
Lemma (Union bound)
Let E1 ; : : : ; En be events
Pr
n
h[
i=1
Ei
i
n
X
i=1
Pr[Ei℄
70 / 292
Probability theory (5)
Lemma (Markov bound)
Let X 0 be a random variable. Then
Pr[X a℄ E [aX ℄
Proof.
The value E [X ℄ is
E [X ℄ = E [X j X a℄ Pr[X a℄ + E [X j X < a℄ Pr[X < a℄
|
{z
}
{z
} | {z }
|
a
a Pr[X a℄
0
0
71 / 292
Probability theory (6)
Theorem (Chernov bound)
Let X1 ; : : : ; Xn be independent random variables with
Xi 2 f0; 1g and X := X1 + : : : + Xn . For any Æ > 0 one has
Pr[X (1 + Æ)E [X ℄℄ eÆ
(1 + Æ)1+Æ
E [X ℄
72 / 292
Let t := ln(1 + Æ) > 0, pi := Pr[Xi = 1℄. Note that E [Xi ℄ = pi.
Pr[X (1 + Æ)E [X ℄℄ e mon.in.
=
Pr[etX et(1+Æ)E[X ℄ ℄
Markov
E [etX ℄
tx
()
Æ)E [X ℄
et(1+
Qn
E [ i=1 etXi ℄
et(1+Æ)E [X ℄
Qn
tXi
X1 ;:::;Xn indep
i=1 E [e ℄
=
et(1+Æ)E [X ℄
Qn
Æpi
()
i=1 e
Æ)E [X ℄
et(1+
Pn
eÆ i=1 pi
=
et(1+Æ)E [X ℄
E [X ℄
Pn
E [X ℄= i=1 pi
eÆ
=
(1 + Æ)(1+Æ)
E [etXi ℄ = pi |{z}
et1 +(1 pi ) |{z}
et0 = 1 + Æpi eÆpi
=1+Æ
=1
73 / 292
Probability theory (7)
Theorem (Variants of Chernov bound)
Let X1 ; : : : ; Xn 2 f0; 1g be independent random variables with
and X := X1 + : : : + Xn and 0 < Æ 1. Then
I Let E [X ℄, then
Pr[X (1 + Æ)℄ e
Æ2 =2
I Let E [X ℄, then
Pr[X (1
2
Æ)℄ e Æ =2
74 / 292
Part 11
Minimizing Congestion
Randomized rounding: A tehnique for provably good
algorithms and algorithmi proofs (Raghavan, Tompson)
Soure:
http://www.springerlink.om/ontent/n16347864k45367w/fulltext.pdf
75 / 292
Minimizing Congestion
Problem: MinCongestion
I Given: Direted graph G = (V; E ) with demand pairs
(si; ti ) si; ti 2 V , i = 1; : : : ; k
I Find: si -ti paths Pi that minimize the ongestion
max
jfi : e 2 Pigj
e2E
s1
t1
s2
t2
s3
t3
76 / 292
Minimizing Congestion
Problem: MinCongestion
I Given: Direted graph G = (V; E ) with demand pairs
(si; ti ) si; ti 2 V , i = 1; : : : ; k
I Find: si -ti paths Pi that minimize the ongestion
max
jfi : e 2 Pigj
e2E
s1
P1
s2
s3
P2
P3
ongestion of 2
t1
t2
t3
77 / 292
A ow-based LP formulation of MinCongestion
min C
(8LP )
>
v = si
<1
X
X
fi(e)
fi (e) =
1 v = ti
>
:
e2Æ (v)
e2Æ (v)
0 otherwise
+
k
X
i=1
f1 (e) = 12
on red e
s1
f2 (e) = 12
on blue e
s2
f3 (e) = 12
on green e
s3
fi (e)
C 8e 2 E
C
fi (e)
1
0 8i 8e 2 E
f1(e) = 12
e
t1
t2
C = 32
t3
78 / 292
Path Deomposition
I Input: s-t ow f : E ! Q + (without direted yles)
I Output: Paths p1 ; : : : ; pm with values v1 ; : : : ; vm 0
(1) i := 1
(2) WHILE f 6= 0 DO
(3)
(4)
(5)
(6)
Let pi be any s-t path in fe j f (e) > 0g
vi := minff (e) j e 2 pi g
f (e) := f (e) vi 8e 2 pi
i := i + 1
3
2
1
s
2
t
3
79 / 292
Path Deomposition
I Input: s-t ow f : E ! Q + (without direted yles)
I Output: Paths p1 ; : : : ; pm with values v1 ; : : : ; vm 0
(1) i := 1
(2) WHILE f 6= 0 DO
(3)
(4)
(5)
(6)
Let pi be any s-t path in fe j f (e) > 0g
vi := minff (e) j e 2 pi g
f (e) := f (e) vi 8e 2 pi
i := i + 1
p1 : v1 = 2
3
2
1
s
2
t
3
80 / 292
Path Deomposition
I Input: s-t ow f : E ! Q + (without direted yles)
I Output: Paths p1 ; : : : ; pm with values v1 ; : : : ; vm 0
(1) i := 1
(2) WHILE f 6= 0 DO
(3)
(4)
(5)
(6)
Let pi be any s-t path in fe j f (e) > 0g
vi := minff (e) j e 2 pi g
f (e) := f (e) vi 8e 2 pi
i := i + 1
1
0
1
s
2
t
3
81 / 292
Path Deomposition
I Input: s-t ow f : E ! Q + (without direted yles)
I Output: Paths p1 ; : : : ; pm with values v1 ; : : : ; vm 0
(1) i := 1
(2) WHILE f 6= 0 DO
(3)
(4)
(5)
(6)
Let pi be any s-t path in fe j f (e) > 0g
vi := minff (e) j e 2 pi g
f (e) := f (e) vi 8e 2 pi
i := i + 1
1
0
1
s
2
t
3p2 : v2 = 1
82 / 292
Path Deomposition
I Input: s-t ow f : E ! Q + (without direted yles)
I Output: Paths p1 ; : : : ; pm with values v1 ; : : : ; vm 0
(1) i := 1
(2) WHILE f 6= 0 DO
(3)
(4)
(5)
(6)
Let pi be any s-t path in fe j f (e) > 0g
vi := minff (e) j e 2 pi g
f (e) := f (e) vi 8e 2 pi
i := i + 1
0
0
0
s
2
t
2
83 / 292
Path Deomposition
I Input: s-t ow f : E ! Q + (without direted yles)
I Output: Paths p1 ; : : : ; pm with values v1 ; : : : ; vm 0
(1) i := 1
(2) WHILE f 6= 0 DO
(3)
(4)
(5)
(6)
Let pi be any s-t path in fe j f (e) > 0g
vi := minff (e) j e 2 pi g
f (e) := f (e) vi 8e 2 pi
i := i + 1
0
s
p3 : v3 = 22
0
0
t
2
84 / 292
Path Deomposition
I Input: s-t ow f : E ! Q + (without direted yles)
I Output: Paths p1 ; : : : ; pm with values v1 ; : : : ; vm 0
(1) i := 1
(2) WHILE f 6= 0 DO
(3)
(4)
(5)
(6)
Let pi be any s-t path in fe j f (e) > 0g
vi := minff (e) j e 2 pi g
f (e) := f (e) vi 8e 2 pi
i := i + 1
0
0
0
s
0
t
0
85 / 292
Path Deomposition
I Input: s-t ow f : E ! Q + (without direted yles)
I Output: Paths p1 ; : : : ; pm with values v1 ; : : : ; vm 0
(1) i := 1
(2) WHILE f 6= 0 DO
(3)
(4)
(5)
(6)
Let pi be any s-t path in fe j f (e) > 0g
vi := minff (e) j e 2 pi g
f (e) := f (e) vi 8e 2 pi
i := i + 1
p1 : v1 = 2
3
s
p3 : v3 = 22
2
1
t
3p2 : v2 = 1
86 / 292
Path Deomposition
Lemma
The algorithm deomposes the ow in s-t paths p1 ; : : : ; pm with
m jE j.
X
e2Æ+ (s)
f (e) =
m
X
i=1
vi
and
X
i:e2pi
vi = f (e) 8e 2 E
I f remains a ow throughout the algorithm.
I In eah iteration there is an edge, where the ow drops
down to 0.
87 / 292
An approximation algorithm for MinCongestion
Algorithm
(1) Solve (LP ) ! ows f1; : : : ; fk fra. ongestion OP Tf
(2) FOR i = 1; : : : ; k DO
P
(3) apply path deomposition to fi ! (pij ; vji ) ( j vji = 1 8i)
(4) Choose Pi among pij 's with Pr[Pi = pij ℄ = vji
Theorem
With probability
1
1 the ongestion is O( ln n ) OP Tf .
ln ln n
n
Consider any edge e 2 E .
Let Xie 2 f0; 1g be the random variable, saying whether the
si -ti path uses e. X1e ; : : : ; Xke are independent!
P
I Let X e := ki=1 Xie be the number of paths, rossing e.
P
P
I E [X e ℄ = ki=1 Pr[Xie ℄ = ki=1 fi (e) OP Tf .
| {z }
I
I
=fi (e)
88 / 292
Proof (2)
Pr
h
z
Xe > =:
Æ
}|
{
E [X e ℄ i
log n
log log n +1
z }| {
OP Tf
1
z }| {
Æ OP Tf
e
ÆÆ
e
3
=
n big
=
Pr
h_
e2E
Xe > 6
lnlnlnnn
!
ln n
ln ln n
lnln n ln n
exp lnlnln n lnln n
ln n exp 12 lnln n lnln
n
1
ln n
ln ln n
ln n
ln ln n
n=2
ln n
1 1
lnln n OP Tf jE j n3 n
i
89 / 292
Inapproximability
Theorem (Andrews & Zhang - JACM'08)
There is no log1 " n-apx unless NP ZPTIME(npolylog(n) ).
90 / 292
Part 12
Knapsak
Soure:
Approximation Algorithms (Vazirani, Springer Press)
91 / 292
Knapsak
Problem: Knapsak
I Given: n objets with weight wi 2 Q + and prot pi 2 Q + ,
size G 2 Q +
I Find: Subset of objets, maximizing the prot and not
exeeding the weight bound:
o
nX
X
pi j wi G
OP T = max
I f1;:::;ng
i2I
i2I
92 / 292
A dynami program for Knapsak
Dynami program:
(1) Assume restrited prots pi 2 f0; : : : ; B g
(2) Compute table entries
o
nX
X
wj j pj b
T (i; b) = min
I f1;:::;ig
j 2I
j 2I
= minimum weight needed for a subset of the rst i
objets to obtain a prot of at least b
using dynami programming
T (i; b) = min T (i 1; b); T (i 1; b pi ) + wi 8i 8p = 0; : : : ; B
| {z } |
{z
}
(3)
don't take i
take i
Reonstrut I leading to maxfb 2 N 0 j T (n; b) Gg
Observation
The algorithm nds optimum solutions in time O(n B ).
93 / 292
The FPTAS
Algorithm:
(1) Sale prots s.t. pmax = n="
(2) Round p0i := bpi
(3) Compute and return optimum solution I for weights p0i
94 / 292
Analysis of FPTAS
Theorem
Let 0 < " 12 . The algo gives a
(1 + 2")-apx in time O(n2=").
I W.l.o.g. OP T pmax = n=" (we an delete objets that
even alone do not t into the knapsak)
I Let I be optimum solution for original prots. Let OP T 0
be optimum value for prots p0. Then
X
X
X
OP T 0 p0i = bpi pi jI j OP T n
i2I I
i2I i2I T
(1 ")OP T 1OP
+ 2"
Let I be solution found by dynami program:
X
X
OP T
pi p0i = OP T 0 1 + 2"
i2I
i2I
0
I B = maxfpi g n=" hene the running time is O(n2 =")
95 / 292
Part 13
Multi Constraint Knapsak
Soure:
Folklore
96 / 292
Multi Constraint Knapsak
Problem: Multi Constraint Knapsak (Mk)
I Given: n objets with prots pi 2 Q + and k many budgets
Bj . Objet i has requirement aji 2 Q + w.r.t. budget j .
I Find: Subset of objets, maximizing the prot and not
exeeding any budget:
nX
o
X
OP T = max
pi j aji Bj 8j = 1; : : : ; k
I f1;:::;ng
i2I
i2I
For arbitrary k there is no n1 "-apx: Take an
Independent Set instane G = (V; E ). For eah edge
e = (u; v) add an \edge budget onstraint"
aeu = aev = 1; Be = 1. Then OP T = OP T .
1
max x1 +x2 +x3
1x1 +1x2 +0x3 1
)
3
0x1 +1x2 +1x3 1
xi 2 f0; 1g
2
I
IS
97 / 292
A PTAS for k = O(1)
Algorithm:
(1) Guess the d k" e items Ilarge in the optimum solution with
maximum prot
(2) Let x be optimum basi solution to the following LP
max
n
X
i=1
n
X
i=1
xi pi
aji xi
Bj 8j = 1; : : : ; k
= 1 8i 2 Ilarge
= 0 8i 2= Ilarge : pi > minfpj j j 2 Ilargeg
1 8i = 1; : : : ; n
(3) Output I := fi j xi = 1g.
xi
xi
0 xi
98 / 292
The Analysis
Theorem
For onstant k the algorithm has polynomial running time.
Furthermore AP X (1 ")OP T .
I The produed solution is learly feasible
I LP OP T (sine we guess elements from OP T )
I Observation: jfi j 0 < xi < 1gj k sine x is a basi
solution and appart from 0 : : : 1 there are only k
onstraints.
I For i with 0 < xi < 1 one has pi k" OP T
AP X
n
X
X
i=1
i:0<xi <1
bxi pi LP
|
{z
pi
k k" OP T
}
OP T k k" OP T = (1 ")OP T
99 / 292
Hardness of MultiConstraintKnapsak
Theorem
There is no FPTAS for MultiConstraintKnapsak even for
2 budgets, unless NP = P.
Problem: Partition
P
I Given: Numbers a1 ; : : : ; an 2 N , S := ni=1 ai ,
m 2 f1; : : : ; ng
P
I Find: I f1; : : : ; ng : jI j = m; i2I ai = S=2
I
I
Reall: Partition is NP-hard.
Dene Mk instane with 2 onstraints:
maxPPn ni=1 xi
i=1 xi ai S=2
Pn
1
x
(
i=1 i S ai ) S (m 2 )
xi 2 f0; 1g 8i = 1; : : : ; n
100 / 292
Proof
I
I
I
Claim: 9 Partition solution
, OP T m
) Suppose 9I : jI j = m; Pi2I ai = S=2. Then this is a
Mk solution of value m sine
X
X
1)
(S ai) = mS
ai = S (m
2
Mk
i2I
i2I
( Let I be Mk solution of value m.
X
X
1: onstr.
2: onst.
S
jI j S S2 jI j S
a i = (S a i ) m S
2
i2I
| {z }
I
I
I
S=2
i2I
HenePjI j = m. Then ineq. holds with "="
Thus i2I ai = S=2.
Now suppose for ontradition we would have an FPTAS
1 . Then the FPTAS would
for Mk: Then hoose " := n+1
give an optimum solution for the instane resulting from
the Partition redution.
101 / 292
Part 14
Bin Paking
Soure:
Combinatorial Optimization: Theory and Algorithms
(Korte, Vygen)
102 / 292
Bin Paking
Problem: BinPaking
I Given: Items with sizes a1 ; : : : ; an 2 [0; 1℄
I Find: Assign items to minimum number of bins of size 1.
OP T
I
n
= min k j 9I1[_ : : : [_ Ik = f1; : : : ; ng : 8j :
X
i2Ij
o
ai 1
Dene size(I ) = Pi2I ai
103 / 292
First Fit
First Fit algorithm:
(1) Start with empty bins
(2) FOR i = 1; : : : ; n DO
(3) Assign
Pitem i to the bin B with least index suh that
ai + j2B aj 1
Lemma
Let m P
be the number of used bins. Then
m 2 ni=1 ai + 1 2 OP T + 1.
All but m 1 bins must be lled with 12 (otherwise we
would not have opened a new bin):
1
n
X
1
ai (m 1) 0:5
2
i=1
0
Pn
bin 1 : : :
bin m
I Hene m 2 i=1 ai + 1.
I
104 / 292
Linear Grouping
I Input: Instane I = (a1 ; : : : ; an ), k 2 N
I Output: Instane I 0 = (a01 ; : : : ; a0n ) with a0i ai and k
dierent item sizes
(1) Sort a1 a2 : : : an
(2) Partition items into k onseutive groups of dn=ke items
(the last group might have less items)
(3) Let a0i be the size of the largest item in i's group
group 1 group 2
group k
I
I0
a1 a2
0
:::
an
1
a01 = a02 = : : :
105 / 292
Linear Grouping (2)
Lemma
OP T (I 0 ) OP T (I ) + dn=ke.
Consider solution OP T (I ). Assign item a0i of group j to a
spae for item in group j + 1
I Assign largest dn=ke items to their own bin
group 1 group 2
group k
I
I
I0
a1 a2
0
:::
an
1
a01 = a02 = : : :
106 / 292
An asymptoti PTAS
Algorithm of Fernandez de la Vega & Lueker:
(1) Let I = fi j ai > "g be set of large items (other items are
small)
(2) Apply linear grouping with k = 1="2 groups to I ! I 0
(3) Compute an optimum distribution of I 0
(4) Distribute the small items over the used bins using First Fit
Lemma
The algorithm runs in polynomial time and uses at most
(1 + 2")OP T + 1 bins.
Let b1; : : : ; b1=" dierent item sizes in I 0.
Possible bin ongurations
P = fp 2 f0; : : : ; 1="g1=" j bT p 1g. jPj (1="2 )1=".
I Solution is desribed by (np)p2P (np = how many times
shall I pak a bin with onguration p?), np 2 f0; : : : ; ng
I n(1=" ) possibilities for (np )p2P .
I
I
2
2
2 1="
107 / 292
An asymptoti PTAS (2)
I We need OP T (I 0 ) + # of bins additionally opened for the
small items
I Note that
OP T (I 0 ) OP T (I )+djI j"2 e OP T (I )+d"OP T (I )e = (1+2")OP T
using OP T (I ) Pi2I ai " jI j and OP T OP T (I ).
I Suppose we need to open an additional bin for small items.
Let m be total number of used bins. Then all but one bin
are lled to 1 ". Hene
OP T
and
m
m
X
i=1
OP T
1 "
ai (1 ") (m
1)
+ 1 (1 + 2")OP T + 1
108 / 292
Setion 14.1
The algorithm of Karmarkar & Karp
109 / 292
The Algorithm of Karmarkar & Karp
Theorem (Karmarkar, Karp '82)
One an ompute a BinPaking solution with
OP T + O(log2 n) many bins in polynomial time.
I
Assume ai Æ := n1 (again one an distribute items that
are smaller than n1 after distributing the large items.
110 / 292
The Gilmore-Gomory LP-relaxation
I Let bi 2 N now the number of items of size ai
I n = number of dierent item sizes
P
I m := ni=1 bi = total number of items
I P = fp 2 Zn+ j aT p 1g set of feasible patterns
I Variable xp = # of bins paked with pattern p
Primal
min
1T x
X
p2P
(P (P ))
xp p
b
max yT b
x
0
y
pT y
Dual
(D(P ))
1 8p 2 P
0
I # var. polynomial
# var. exponential
I # onstr. exponential
# onstr. polynomial
Idea: Solve the dual with Ellipsoid!
I
I
111 / 292
Example
Dual
I Item sizes a1 = 0:3; a2 = 0:4
max
31y + 0
7y2 1
I # of items b1 = 31; b2 = 7
00 11 1
1
02
1
I Set
of
patterns
P
=
0 0 1 2 1 2 3
B1 1C
B1C
1; 2; 1; 1; 0; 0; 0
Primal
min 1T x
( 10 20 11 12 01 02 03 ) x ( 317 )
x
I
0
Opt basi solution is
x = (0; 0; 0; 7; 0; 0; 173 )
2 1Ay
10
20
30
y
1A
0
1
1
1
1:0 y2
0
1
3 1
0:8
0 0
0
0:6
2
0:4
b 1
1 0
1
0:2 D(P ) 21
0 0 0:2 0:4 0:6 0:8 1:0y1
112 / 292
Weak Separation Problem
"-Weak Separation Orale for P
Input:
z 2 Qn
R n , obj.ft. 2 Q n
Vetor
Output: One of the following
I Case (A): Vetor a with aT x aT z 8x 2 P
I Case (B): Point y 2 P with T y T z 2"
Case (A):
Case (B):
a
2"
z
P
P
I
If z 2 P , just return z (! ase (B)).
z
y
113 / 292
Grotshel-Lovasz-Shrijver Algorithm
I Input: 2 Q n ; x0 2 Q n ; "; r; R 2 Q + :
B (x0 ; r) P B (x0 ; R)
I Output: y 2 P with T y OP Tf "
(1) Ellipsod E0 := B (x0; R) with enter z0 := x0, y := x0
(2) FOR t = 0; : : : ; poly DO
(4) Submit zt to "-weak separation orale
(5) Case (A) ! a: Compute Et+1 Et \ fx j aT x aT ztg
(6) Case (B) ! y 2 P :
(7) IF T y > T y THEN y := Ty
(8) Compute Et Et x x
+1 \ f
j
Input/
Output:
T zt g
"
r x0
R
P
y
114 / 292
Grotshel-Lovasz-Shrijver Algorithm
I Input: 2 Q n ; x0 2 Q n ; "; r; R 2 Q + :
B (x0 ; r) P B (x0 ; R)
I Output: y 2 P with T y OP Tf "
(1) Ellipsod E0 := B (x0; R) with enter z0 := x0, y := x0
(2) FOR t = 0; : : : ; poly DO
(4) Submit zt to "-weak separation orale
(5) Case (A) ! a: Compute Et+1 Et \ fx j aT x aT ztg
(6) Case (B) ! y 2 P :
(7) IF T y > T y THEN y := Ty
(8) Compute Et Et x x
+1 \ f
Case (A):
j
T zt g
a
zt
Et
P
115 / 292
Grotshel-Lovasz-Shrijver Algorithm
I Input: 2 Q n ; x0 2 Q n ; "; r; R 2 Q + :
B (x0 ; r) P B (x0 ; R)
I Output: y 2 P with T y OP Tf "
(1) Ellipsod E0 := B (x0; R) with enter z0 := x0, y := x0
(2) FOR t = 0; : : : ; poly DO
(4) Submit zt to "-weak separation orale
(5) Case (A) ! a: Compute Et+1 Et \ fx j aT x aT ztg
(6) Case (B) ! y 2 P :
(7) IF T y > T y THEN y := Ty
(8) Compute Et Et x x
+1 \ f
Case (A):
j
T zt g
a
zt
Et
P
Et+1
116 / 292
Grotshel-Lovasz-Shrijver Algorithm
I Input: 2 Q n ; x0 2 Q n ; "; r; R 2 Q + :
B (x0 ; r) P B (x0 ; R)
I Output: y 2 P with T y OP Tf "
(1) Ellipsod E0 := B (x0; R) with enter z0 := x0, y := x0
(2) FOR t = 0; : : : ; poly DO
(4) Submit zt to "-weak separation orale
(5) Case (A) ! a: Compute Et+1 Et \ fx j aT x aT ztg
(6) Case (B) ! y 2 P :
(7) IF T y > T y THEN y := Ty
(8) Compute Et Et x x
+1 Case (B):
\ f
j
2"
b
T zt g
Et
zt
P y
117 / 292
Grotshel-Lovasz-Shrijver Algorithm
I Input: 2 Q n ; x0 2 Q n ; "; r; R 2 Q + :
B (x0 ; r) P B (x0 ; R)
I Output: y 2 P with T y OP Tf "
(1) Ellipsod E0 := B (x0; R) with enter z0 := x0, y := x0
(2) FOR t = 0; : : : ; poly DO
(4) Submit zt to "-weak separation orale
(5) Case (A) ! a: Compute Et+1 Et \ fx j aT x aT ztg
(6) Case (B) ! y 2 P :
(7) IF T y > T y THEN y := Ty
(8) Compute Et Et x x
+1 Case (B):
\ f
j
2"
b
T zt g
Et
zt
P y
Et+1
118 / 292
Analysis
Theorem
Let OP Tf = maxfT x j x 2 P g. The GLS algorithm nds a
y 2 P with T y OP Tf ".
" "
I Suppose for ontradition this is false.
2 2
I Let x 2 P be opt. sol.; ' input size.
I Inequalities from ase (A) never ut
y
points from P
r
I Ineq. from ase (B) never ut points
U U0
x
0
better than OP Tf 2" (otherwise we
would have found a suitable y )
P
I Let U := onvfB (x0 ; r); x g and
"
0
T
U = fx 2 U j x OP Tf 2 g. By standard volume
bounds: vol(U 0) ( 21 )poly(') . But U 0 Et 8t. After
t
0
poly(') many it. vol(Et ) = (1 (1)
n ) vol(E0 ) < vol(U ).
Contradition!
x
119 / 292
A useful observation
Observation
Consider a run of the GLS algorithm for P Rn whih yields
y 2 P . Let aT1 x b1 ; : : : ; aTN x bN be the inequalities whih
the orale are returned for Case (A).
I Eah aTi x bi is feasible for P
I T y maxfT x j aTi x bi 8i = 1; : : : ; N g "
"
P
y
aTi x bi
120 / 292
Solving D(P )
Lemma
Suppose ai Æ. Then we an nd a feasible solution y to D(P )
of value OP Tf 1 in time polynomial in n; m; 1Æ .
I
Apply GLS algo for " := 1. Choose y0 = ( 2Æ ; : : : ; 2Æ ).
B y0 ;
I
Æ (Æ;:::;Æ)T p1
2
We use Pni=1 pi 1Æ for
any feasible pattern
p 2 P sine ai Æ
y2
Æ
D(P ) B (y0 ; n)
D (P )
y0
Æ
yT p 1
2
Æ
y1
121 / 292
Solving D(P ) (2)
I We solve "-weak separation problem for z 2 Q n .
I If zi < 0 ! Case (A) (inequality zi 0 violated)
I If zi > 1 ! Case (A) (inequality z T ei 1 violated)
I Round z down to nearest multiple of 21m and term this
vetor y. Solve p = argmaxfyT p j p 2 Pg
(Knapsak with prots from 0; 1 21m ; 2 21m ; : : : ; 1)
Case yT p > 1:
Case yT p 1:
I Then z T p yT p > 1
I Then y 2 D(P ). And
! Case (A).
z T b yT b m 21m = 21 = 2" .
y2
! Case (B)
y2
y z
z
yT p 1
y
yT p 1
D(P )
y1
D(P )
I GLS yields a solution y
mit
bT y y1
OP Tf 1.
122 / 292
Finding a near optimal basi solution for P (P )
Theorem
Suppose ai Æ. Then we an nd a basi solution x for P (P )
of value OP Tf + 1 in time polynomial in n; m; 1Æ .
I
I
Run GLS to obtain sol. y to D(P ) with bT y OP Tf 1
Let yT p 1; p 2 P 0 be inequalities returned by yorale for
2
ase (A). P 0 P has polynomial size and
y
valid for D(P) T D(P )
b y D(P 0 )
I Compute optimum basi solution x
1T x = P (P 0 ) duality
= D(P 0 )
I
y
b
2 P0 0
D(P )
P
1 (1)
y
for P (P 0 ) in poly-time.1
D( )
(1)
D(P ) + 1 duality
= P (P ) + 1
x is also a (non-optimal) basi solution for P (P )
123 / 292
Geometri Grouping
P
I Input: Instane I = (a1 ; : : : ; an ), size(I ) = ni=1 ai bi n,
I
(1)
(2)
ai Æ
Rounded up instane I 0 with n=2 di. item sizes
OP Tf (I 0 ) OP Tf (I ) plus waste of O(log 1Æ )
Sort items w.r.t. sizes e1 e2 : : : em (ai appears bi
times)
Let
G1 = fe1 ; : : : ; e` g be minimal set of items with
P
i2G ei 2, then ontinue with G2 ,: : :. Let `i := jGi j be
number of items in Gi
`1
`2 `3 `4 `5
Remove rst and last
group ! waste
I
From Gi throw away
smallest `i `i+1
I0
items ! waste
waste
Round up items in Gi
0
to largest item ! I
G1
G2 G3 G4 G5
Output:
1
1
(3)
(4)
(5)
124 / 292
Geometri Grouping (2)
Lemma
Size of waste is O(log 1Æ ).
I
I
I
I
Size of 1st and last group is O(1)
Consider group Gi . Total size of items in Gi is 3.
Num of groups is n=2. Cleary 2Æ `1 `2 : : :.
The ni := `i `i+1 smallest items in Gi have size 3 n` .
`
X
X
waste 3 n` i 3 1j ` 2
= =Æ O(log 1Æ )
i i
j =1
`i items of total size 3
i
i
1
1
Gi
ni items of total size 3 n`ii
125 / 292
The algorithm
Algorithm:
(1)
(2)
(3)
(4)
Compute a basi solution x to P (P ) with 1T x OP Tf + 1
Buy bxp times pattern p, let I be remaining instane
Apply geometri grouping to I (with n dierent item sizes)
! I 0 (with n=2 dierent item sizes)
Reurse
Theorem
One has AP X
OP Tf + O(log2 n).
Sine x is basi solution,
jfp j xp > 0gj n.
After (2) size(I ) Pp(xp bxp) n. P
Let xt be solution x in iteration t. We buy pbxtp bins,
but OP Tf dereases by the same quantity.
I We pay in total OP Tf + total waste. We have O(log n)
reursions; in eah reursion we have a waste of
O(log 1Æ ) = O(log n).
I
I
I
126 / 292
State of the art
I Computing OP T exatly is NP-hard even if the numbers
ai are unary enoded (i.e. BinPaking is strongly
NP-hard).
Open question
One an ompute a Bin Paking solution with OP T + 1
bins in poly-time?
Mixed Integer Roundup Conjeture
One has OP T dOP Tf e + 1.
127 / 292
Part 15
Minimum Makespan Sheduling
Soure:
Approximation Algorithms (Vazirani, Springer Press)
128 / 292
Minimum Makespan
Problem: Minimum Makespan Sheduling
I Given: n jobs, job j has proessing time pj . Number m of
mahines.
I Find: Assign jobs to mahines to minimize the makespan.
n X oo
n
pj
max
OP T =
min
I [_ :::[_ I =f1;:::;ng i=1;:::;m
j 2Ii
m
1
makespan
0
j pj
1 2
:::
m
129 / 292
A PTAS for Minimum Makespan Sheduling
Algorithm:
(1) Guess OP T
(2) Call job with pj > " OP T large and small otherwise !
sub-instane I of large jobs
(3) Round proessing times pj for large jobs down to multiple
of OP T "2 ! instane I 0 with proessing times p0j
(4) Distribute rounded large jobs I 0 suh that makesepan is
OP T
(5) Distribute small jobs onseutively on least loaded mahine
130 / 292
Analysis
Lemma
The algorithm runs in polynomial time and produes a
makespan of at most (1 + ")OP T .
Large jobs with rounded proessing times an be
distributed optimally in polynomial time sine: 1="2
dierent job sizes, at most 1=" large jobs per mahine,
hene O((1="2 )1=" ) many ways how to pak a mahine,
hene nO((1=" ) ) possible solutions.
I Clearly OP T (I 0 ) OP T (I ) OP T . Let Ii set of jobs on
most loaded mahine (attaining the makespan).
I Case: Small jobs don't in. makespan. No small job in Ii .
I
2 1="
X
j 2Ii
pj X
j 2Ii
(p0
j
+ " "OP
| {z T})
p0j
P
0 OP T
j Ii pj
2
(1 + ")OP T
131 / 292
Analysis (2)
P
I OP T m1 nj=1 pj = average load
I Case: Small jobs do in. makespan.
Then all mahines are
lled up to makespan " OP T OP T . Hene
makespan (1 + ")OP T
small job
makespan
0
" OP T
OP T
1
::: i ::: m
132 / 292
Hardness
Lemma
There is no FPTAS for
unless NP = P.
Minimum Makespan Sheduling
Reall that given a BinPaking instane
I = (a1 ; : : : ; an ); ai 2 N unary enoded and m; B 2 N , it is
NP-hard to deide, whether m bins of size B suÆe to
pak the items.
I Suppose there is an FPTAS for Minimum Makespan
Sheduling. Take items as jobs, m as number of
mahines and " := P 1a +1 . Then the FPTAS would give
an exat answer.
opt. makespan B , 9 Bin Paking solution with m bins.
I
n
i=1 i
133 / 292
Part 16
Sheduling on Unrelated Parallel
Mahines
Soure:
Approximation Algorithms (Vazirani, Springer Press)
134 / 292
Sheduling on Unrelated Parallel Mahines
Problem: Unrelated Mahine Sheduling
I Given: Jobs J = f1; : : : ; ng, mahines M = f1; : : : ; mg.
Running job j on mahine i takes a proessing time pij .
I Find: Assign jobs to mahine to minimize the makespan.
OP T
n
= I [_ :::[_ Imin=f1;:::;ng i=1max
;:::;m
nX
m
1
job j
makespan
j 2Ii
pij
oo
pij
0
1
:::
i
:::
m
135 / 292
How NOT to solve the problem
LP:
min T
X
i2M
X
j 2J
xij
pij xij
xij
Variables:
= 1 8j 2 J
T 8i 2 M
0 8i 8j
xij
T
8
>
<
1 job j is assigned
= > to mahine i
:
0 otherwise
= makespan
Example: 1 job with exeution time pi1 = m, 8i = 1; : : : ; m
Integer solution: x11 = 1
Frational solution: xi1 = m1
T =m
T
=1
::: m
1 :::
I Integrality gap of m
1
:::
::: m
136 / 292
A 2-approximation
Algorithm:
(1) Guess OP T
(2) Compute basi solution x to
X
xij = 1 8j 2 J
i2M
X
j 2J
pij xij
OP T 8i 2 M
= 0 for i; j with pij > OP T
0 8i 2 M 8j 2 J
(3) xij = 1 ) assign job j to mahine i
(4) For not yet assigned jobs: Assign j to a mahine i with
0 < xij < 1 s.t. every mahine reeives at most 1 extra job
xij
xij
137 / 292
Analysis
Theorem
The algorithm runs in polynomial time and the makespan is at
most OP T + maxfpij j xij > 0g 2 OP T .
1 0:7 1
Running time is learly polynomial:
We solve a poly size LP in (2) and solve
a maximum mathing problem in (4). 0:8 0:3
2 0:2 2
I Let H = (J [ M; E ) with
E := f(j; i) j 0 < xij < 1g. For laim on
1
makespan we need to show that E
3
3
ontains a
0:4
fj not assigned in (3)g-perfet
mathing.
4 x = 0:6 4
I
ij
J
M
138 / 292
Analysis
Theorem
The algorithm runs in polynomial time and the makespan is at
most OP T + maxfpij j xij > 0g 2 OP T .
1
Running time is learly polynomial:
We solve a poly size LP in (2) and solve
a maximum mathing problem in (4).
2
I Let H = (J [ M; E ) with
E := f(j; i) j 0 < xij < 1g. For laim on
makespan we need to show that E
3
ontains a
fj not assigned in (3)g-perfet
mathing.
4
1
J
M
I
2
3
4
139 / 292
Assigning the frational jobs (1)
Claim
E ) of H . Then
Consider a onneted omponent (J [ M;
x = (xij )(j;i)2E is still a basi solution of the subsystem LP (E ).
X
i2M
X
j 2J
xij
pij xij
= 1 8j 2 J (LP (E ))
T
X
j 2= J
pij xij
8i 2 M
0 xij 1 8(j; i) 2 E
Reason: If x 2 onv(fy(1) ; y(2) g) then
x = (x ; x^ ) 2 onv(f(y(1) ; x^ ); (y(2) ; x^ )g).
Contradition.
1
J
2
1 E
M
2
3
3
4
4
J
M
140 / 292
Assigning the frational jobs (2)
I x is basi solution, hene
=#onstr. in LP (E )
z }| {
jE j = jf(j; i) j 0 < xij < 1gj jJj + jM j #nodes in E
I But E is onneted, thus E is a tree + 1 extra edge.
I Jobs have degree 2, hene leaves must be mahines. As
long as there are mahine-leaves i, assign a j with xij > 0
to i and remove both, i and j .
I A single even length job-mahine yle (potentially)
remains. Extrat a mathing and we are done.
J
E :
M
J
M
M
J
M
M
J
M
J
M
141 / 292
State of the art
Exerise
There is no (3=2 ")-apx for Unrelated Mahine
Sheduling unless NP = P.
Open Problem 1
Is there a 3=2-apx?
Open Problem 2
A (2
")-apx is still unknown even for the Restrited
Assignment Problem where pij 2 fpj ; 1g.
Theorem (Ebenlendr, Kral, Sgall '08)
There is a 1:75-apx for the Restrited Assignment
Problem if eah job j is admissible on 2 mahines.
142 / 292
Part 17
Multiproessor Sheduling with
Preedene Constraints
Soure:
I Graham (1966): Bounds for ertain multiproessor anomalies
(Bell Systems Tehnial Journal).
I Leture notes of Chandra Chekuri
http://www.s.illinois.edu/lass/sp09/s598s/Letures/leture_6.pdf
143 / 292
Multiproessor Sheduling with Preedene
Constraints
Problem: PreSheduling (P j pi ; pre j Cmax)
I Given: Jobs J1 ; : : : ; Jn , job Ji has proessing time pi ,
preedene relation , # of mahines m
I Find: (Non-preemptive) shedule of the jobs on m
mahines respeting the preedene order and minimizing
the makespan
I Ji J` means that Ji has to be nished, before J` is
allowed to start.
Ji
Input:
Ji J`
p`
J`
Solution:
1.
..
m
makespan
144 / 292
The algorithm
Graham's List Sheduling:
(1) FOR t = 1; : : : DO
(2) IF a mahine j 2 f1; : : : ; mg is idle at t
AND all predeessors of some (not yet proessed) job Ji are
already nished
THEN shedule Ji on mahine j starting from t
I
In other words: At any time, just start a job whenever
possible.
145 / 292
The analysis
Theorem
The makespan of the produed shedule is at most
2 OP T
I Find a sequene (w.l.o.g. after reordering) J1 ; : : : ; Jk s.t.
I Jk is the last job of the whole shedule that nishes
I J1 J2 : : : Jk (hain in the partial order )
I Ji is the predeessor of Ji+1 that is nished last
1.
..
m
J1
J2
:::
Jk
makespan
After Ji nished Ji+1 is startet as soon as a mahine is
available. Hene between Ji is nished and Ji+1 begins, all
mahines must be fully busy.
I length of all busy periods OP T
I Length of hain J1 ; : : : ; Jk is OP T
I Makespan length hain + busy period 2 OP T
I
146 / 292
Hardness
Theorem (Svensson - STOC'10)
For every xed " > 0, there is no (2 ")-apx unless a variant of
the Unique Games Conjeture is false.
Open Problem
What is the omplexity status of P 3 j pi = 1; pre j Cmax (i.e.
PreSheduling with unit proessing times and 3 mahines)?
Known:
I 4=3-apx.
I P 2 j pi = 1; pre j Cmax is poly-time solvable
147 / 292
Part 18
Eulidean TSP
Polynomial-time Approximation Shemes for Eulidean
TSP and other Geometri Problems (Arora '98, Link)
Soure:
148 / 292
Eulidean Travelling Salesman Problem
Problem: EulideanTSP
I Given: Points v1 ; : : : ; vn 2 Q 2 in the plane.
I Find: Minimum ost tour visiting all nodes
min
tour :V !V
n
nX
i=1
kvi v(i) k2
o
vj =
xj yj
p(xi
Goal:
Find a PTAS!
vi =
kvi vj k2
=
xj )2 (yi yj )2
xi yi
149 / 292
A random bounding box
I Choose a minimal square S ontaining all points.
I W.l.o.g. this square is [ L2 ; L℄2 with L = n=" 2 2N after
saling. Hene OP T L = n=".
I Choose a; b 2 f1; : : : ; L=2g randomly.
I Let R = [a; a + L℄ [b; b + L℄ S be the randomly shifted
bounding box.
L
R
L
S
L
2
L
b
(0; 0)
aL
2
L
150 / 292
Disretization
I Move all points v to nearest point in Z2.
I Changes the ost of any tour by 2n 2" OP T
(sine OP T L = n=")
1
151 / 292
The dissetion
I Divide the L L bounding box into 4 squares of size L2 L2
I Divide eah L2 L2 square into 4 squares of size L4 L4
I Reurse, until unit size squares are reahed
I Size 2L 2L squares are level i squares
I A line segment is on level i, if it is the boundary of a level i
square but not of a level i 1 square
I A grid line is on level i, if it onsists of level i segments
i
i
L
level 2 square
level 0 square
1
level 1 square
152 / 292
The dissetion
I Divide the L L bounding box into 4 squares of size L2 L2
I Divide eah L2 L2 square into 4 squares of size L4 L4
I Reurse, until unit size squares are reahed
I Size 2L 2L squares are level i squares
I A line segment is on level i, if it is the boundary of a level i
square but not of a level i 1 square
I A grid line is on level i, if it onsists of level i segments
i
i
L
level 1 segment
1
level 2 segment
153 / 292
The dissetion
I Divide the L L bounding box into 4 squares of size L2 L2
I Divide eah L2 L2 square into 4 squares of size L4 L4
I Reurse, until unit size squares are reahed
I Size 2L 2L squares are level i squares
I A line segment is on level i, if it is the boundary of a level i
square but not of a level i 1 square
I A grid line is on level i, if it onsists of level i segments
i
i
L
level 1 grid line
1
level 2 grid line
154 / 292
Basi idea
I Method: Use dynami programming.
I Idea: Consider a level i square Q in the dissetion.
For all
ways how OP T an interset Q, ompute the heapest
extension inside Q that visits all nodes in Q (using that we
omputed similar information already for all smaller
squares).
Q
opt. tour
The number of possibilities how OP T an
ross Q might be exponential/innite.
I Solution: Limit this number.
I DiÆulty:
155 / 292
Basi idea
I Method: Use dynami programming.
I Idea: Consider a level i square Q in the dissetion.
For all
ways how OP T an interset Q, ompute the heapest
extension inside Q that visits all nodes in Q (using that we
omputed similar information already for all smaller
squares).
Q
opt. tour
The number of possibilities how OP T an
ross Q might be exponential/innite.
I Solution: Limit this number.
I DiÆulty:
156 / 292
Portals
I On any level i line segment, plae 1" log L many
level i portals (plus one per orner)
I Distane of onseutive level i portals is 2L log" L
i
r
r
r
r
r
r
r
r
r
r
r
r
r
r
r
r
r
r
r
r
r
r
r
r
r
r
r
r
r
r
r
r
r
r
r
r
r
r
r
r
r
r
r
r
r
2Li log" L
r
r
r
r
r
r
r
r
r
r
r
r
r
r
"
r
r
r
r
1 log L portals
r
r
r
r
r
r
r
r
r
r
r
r
r
L=2i
r
r
r
r
r
level i square
portal
157 / 292
Well rounded tours
r
r
r
Denition
I
r
r
A tour is alled well-rounded
tour if:
I It leaves and enters squares
only at portals.
I Eah square is entered at most
4 times.
"
r
r
r
r
r
r
r
r
r
r
r
r
opt. tour
r
r
r
r
r
r
r
r
r
r
r
r
r
r
r
r
Eah square has "
The number
of times that a well-rounded tour an leave/enter a square
is bounded by ( 4" log L + 4)O(1=") (whih is polynomial).
4 log L + 4 many portals.
Theorem (Struture Theorem)
There is always a well-rounded tour of ost
(1 + O("))OP T .
158 / 292
Well rounded tours
r
r
r
Denition
A tour is alled well-rounded
tour if:
I It leaves and enters squares
only at portals.
I Eah square is entered at most
4 times.
"
r
r
r
r
r
r
r
r
r
r
r
r
r
r
r
r
r
r
r
r
r
r
r
r
r
well rounded tour
I Eah square has 4" log L + 4 many portals. The number
of times that a well-rounded tour an leave/enter a square
is bounded by ( 4" log L + 4)O(1=") (whih is polynomial).
r
r
Theorem (Struture Theorem)
There is always a well-rounded tour of ost
r
r
r
(1 + O("))OP T .
159 / 292
Relation OP T vs. number of rossings
I For the optimum tour and a grid line `, let t(; `) be the
number of times that rosses `.
1 X t(; `) OP T p2 X t(; `)
3
grid lines `
grid lines `
I OP T = (1) #rossings
I Goal: Turn opt. tour into a well-rounded tour,
suh that the expeted
ost
inrease is O(") P` t(; `)
I Alternatively: Average
ost inrease per rossing
must be O(1) "
`
t(; `) = 4
160 / 292
Bending edges through portals
I Consider a rossing of the
optimum tour at a grid line `
i
I Pr[line ` is at level i℄ = 2
L
I If line ` is at level i, we have to
bend edge through the nearest
portal and loose 2Li log" L
I The expeted length inrease is
log
XL
=
r
2Li log" L
r
0
r
r
`
Pr[` at level i℄ portal distane at level i
i=0
log
XL i
2 L " 2"
L 2i log L
i=0
161 / 292
Pathing Lemma
Lemma
Given a TSP tour , rossing a line segment ` of length s an
arbitrary number of times. 9 tour 0 rossing ` at most 2 times
whih an be obtained by adding segments of length 6s.
Cut at `. Let L1; : : : ; Lt be endpoints on the
left side, R1; : : : ; Rt end points on the right.
Imagine their distane to ` as 0. Say t is even
(other ase is similar).
I Add tours on Li 's and on Ri 's of ost 2s eah.
I Add mathings (L2i 1 ; L2i ); (R2i 1 ; R2i ) for
2i < t and 2 edges (Lt 1 ; Rt 1 ); (Lt ; Rt ) of total
ost 2s.
I Degree of V [ fLi ; Ri j i = 1; : : : ; tg is even.
Graph is again onneted. Hene there is a tour
visiting all nodes (at least one).
I
s
`
162 / 292
Pathing Lemma
Lemma
Given a TSP tour , rossing a line segment ` of length s an
arbitrary number of times. 9 tour 0 rossing ` at most 2 times
whih an be obtained by adding segments of length 6s.
Cut at `. Let L1; : : : ; Lt be endpoints on the
left side, R1; : : : ; Rt end points on the right.
Imagine their distane to ` as 0. Say t is even L1 R1
(other ase is similar).
I Add tours on Li 's and on Ri 's of ost 2s eah.
I Add mathings (L2i 1 ; L2i ); (R2i 1 ; R2i ) for
2i < t and 2 edges (Lt 1 ; Rt 1 ); (Lt ; Rt ) of total
ost 2s.
Lt Rt
I Degree of V [ fLi ; Ri j i = 1; : : : ; tg is even.
Graph is again onneted. Hene there is a tour `0
visiting all nodes (at least one).
I
b
b
b
b
b
b
b
b
b
b
b
b
s
163 / 292
Pathing Lemma
Lemma
Given a TSP tour , rossing a line segment ` of length s an
arbitrary number of times. 9 tour 0 rossing ` at most 2 times
whih an be obtained by adding segments of length 6s.
Cut at `. Let L1; : : : ; Lt be endpoints on the
left side, R1; : : : ; Rt end points on the right.
Imagine their distane to ` as 0. Say t is even L1 R1
(other ase is similar).
I Add tours on Li 's and on Ri 's of ost 2s eah.
I Add mathings (L2i 1 ; L2i ); (R2i 1 ; R2i ) for
2i < t and 2 edges (Lt 1 ; Rt 1 ); (Lt ; Rt ) of total
ost 2s.
Lt Rt
I Degree of V [ fLi ; Ri j i = 1; : : : ; tg is even.
Graph is again onneted. Hene there is a tour `0
visiting all nodes (at least one).
I
b
b
b
b
b
b
b
b
b
b
b
b
s
164 / 292
Pathing Lemma
Lemma
Given a TSP tour , rossing a line segment ` of length s an
arbitrary number of times. 9 tour 0 rossing ` at most 2 times
whih an be obtained by adding segments of length 6s.
Cut at `. Let L1; : : : ; Lt be endpoints on the
left side, R1; : : : ; Rt end points on the right.
Imagine their distane to ` as 0. Say t is even L1 R1
(other ase is similar).
I Add tours on Li 's and on Ri 's of ost 2s eah.
I Add mathings (L2i 1 ; L2i ); (R2i 1 ; R2i ) for
2i < t and 2 edges (Lt 1 ; Rt 1 ); (Lt ; Rt ) of total
ost 2s.
Lt Rt
I Degree of V [ fLi ; Ri j i = 1; : : : ; tg is even.
Graph is again onneted. Hene there is a tour `0
visiting all nodes (at least one).
I
b
b
b
b
b
b
b
b
b
b
b
b
s
165 / 292
Reduing the number of rossings (1)
MODIFY
Proedure:
I Input: Grid line ` on level i
I Output: Tour 0 rossing eah segment of ` at most 1="
times
(1) FOR j = log L downto i DO
(2) FOR all level j segments DO
(3) IF segment is rossed > 1=" times
THEN redue # rossings to 2 via Pathing Lemma
j = log L
)
`
j =i+1
:::
)
`
j=i
)
`
Output:
)
`
`
166 / 292
Reduing the number of rossings (2)
I Starting from optimum tour, we apply MODIFY to all
horizontal and vertial grid lines.
I Now onsider a xed grid line `. Want to show:
E [ost for rossing redution at `℄ O(") t(; `)
I Let `;j be number of times that MODIFY is applied to level
j segments of grid line `
I Eah appliation of MODIFY redues the number of rossings
of ` by 1=" 2 21" (assuming " 1=4). Hene
X
t(; `)
`;j 1=(2") = 2" t(; `)
j 0
The ost inrease of a single rossing redution on level j is
I Thus
X
L
E [ost inrease at ` j ` at level i℄ `;j 6 j
2
I
6 2Lj (by Pathing Lemma).
j i
167 / 292
Reduing the number of rossings (3)
E [ost for rossing redution at `℄
X
=
Pr[` at level i℄ E [ost inrease at ` j ` at level i℄
i0
reordering
=
P
j 0 `;j
2"t(;`)
I
2
L
X i X
i0
6
j i
`;j 6
X `;j X
i
j 0
12 L
2j
2j ij 2
X
j 0
| {z }
22j
`;j
24" t(; l)
9 well-rounded tour of ost (1 + O(")) OP T
168 / 292
The dynami program (1)
I Table entries:
A(Q; (s1 ; t1 ); : : : ; (sq ; tq ))
= ost of heapest extension of q subtours to well-rounded tour
visiting all nodes in Q suh that subtour i goes from si to ti
8 squares Q 8q 2 f0; : : : ; 4="g 8 portals si; ti of Q
I Number of table entries:
r
I O(n log L) many non-empty
squares Q
t1 r
I There are O( 1" log n)O(1=")
many ways to hoose O(1=")
r
portals out of O( 1" log L)
portals
s1 r
I
s2r
r
r
Q
r
r
t2
r
s3
r
Total number of entries:
O(n(log n)O(1=") )
r
r
r
r
t3
r
169 / 292
The dynami program (1)
I Table entries:
A(Q; (s1 ; t1 ); : : : ; (sq ; tq ))
= ost of heapest extension of q subtours to well-rounded tour
visiting all nodes in Q suh that subtour i goes from si to ti
8 squares Q 8q 2 f0; : : : ; 4="g 8 portals si; ti of Q
I Number of table entries:
r
I O(n log L) many non-empty
squares Q
t1 r
I There are O( 1" log n)O(1=")
many ways to hoose O(1=")
r
portals out of O( 1" log L)
portals
s1 r
I
s2r
r
r
Q
r
r
t2
r
s3
r
Total number of entries:
O(n(log n)O(1=") )
r
r
r
r
t3
r
170 / 292
The dynami program (2)
Lemma
The best well rounded tour an be omputed in O(n(log n)O(1=") )
Compute table entries bottom-up (starting with smallest
squares)
s2
I For entry A(Q; (s1 ; t1 ); : : : ; (sq ; tq )):
Let Q1; : : : ; Q4 be the subsquares of
t1
Q. Guess (i.e. try out all
1
2
ombinations) the visited portals of
Q1 ; : : : ; Q4 and their order
! O( 1" log n)O(1=") ombinations
s1
3
4
I Look up table entries for Q1 ; : : : ; Q4
to determine ost.
t3
I
r
Q Q
Q Q
r
r
r
Q
r
r
r
r
t2
r
s3
r
r
r
r
r
r
r
r
r
r
r
r
r
r
r
r
r
r
r
r
171 / 292
The dynami program (2)
Lemma
The best well rounded tour an be omputed in O(n(log n)O(1=") )
Compute table entries bottom-up (starting with smallest
squares)
s2
I For entry A(Q; (s1 ; t1 ); : : : ; (sq ; tq )):
Let Q1; : : : ; Q4 be the subsquares of
t1
Q. Guess (i.e. try out all
1
2
ombinations) the visited portals of
Q1 ; : : : ; Q4 and their order
! O( 1" log n)O(1=") ombinations
s1
3
4
I Look up table entries for Q1 ; : : : ; Q4
to determine ost.
t3
I
r
Q Q
Q Q
r
r
r
Q
r
r
r
r
t2
r
s3
r
r
r
r
r
r
r
r
r
r
r
r
r
r
r
r
r
r
r
r
172 / 292
Generalizations
Advantages of this approah:
Appliable for many graph optimization problems, when
nodes are points in the Eulidean plane (like Steiner
Tree, k -Median, Steiner Forest, k -Tsp, k -Mst.
I Works for general `p-metries (like maximums-norm)
I Extends to any onstant dimension
I (Theoretially) nie dependene on "
I
Theorem (Arora '98)
Let d 2 N ; " > 0; p 2 N [ f1g be xed onstants. Then there is
an expeted (1 + ")-apx for TSP if the nodesPare points in Rd
and distanes are measured
as kv ukp := ( di=1 jvi ui jp )1=p
p
d
1
in time n(O(log n))O( d1=") . This an be derandomized by
inreasing the running time by a fator of O(n=").
173 / 292
Part 19
Tree Embeddings
Soure:
A tight bound on approximating arbitrary metris by tree
metris (Fakharoenphol, Rao, Talwar:
Link)
174 / 292
Tree metri
Denition (Tree metri)
Given nodes V , spanning tree T , edge osts (e) 8e 2 T . Then
dT : V V ! Q + with
dT (u; v) := length of u v path in T
is alled a tree metri.
dT (u; v)
T
(e)
v
u
175 / 292
Motivation
Many optimization problems are easy on
trees: Steiner tree, Tsp, k-Tsp, Steiner Forest, : : :
I Question: Can we for any node set V and metri
d : V V ! Q + , nd a tree metri dT suh that
d(u; v) dT (u; v) d(u; v) 8u; v 2 V
for a small distortion ?
I Motivation:
u d(u; v) v
I Possible approah: For some graph optimization
problem, ompute tree T . Then solve problem on tree
optimally (or get O(1)-apx). Obtain a -apx (or O()-apx)
for original problem.
176 / 292
Motivation
Many optimization problems are easy on
trees: Steiner tree, Tsp, k-Tsp, Steiner Forest, : : :
I Question: Can we for any node set V and metri
d : V V ! Q + , nd a tree metri dT suh that
d(u; v) dT (u; v) d(u; v) 8u; v 2 V
for a small distortion ?
I Motivation:
T
dT (e)
u d(u; v) v
I Possible approah: For some graph optimization
problem, ompute tree T . Then solve problem on tree
optimally (or get O(1)-apx). Obtain a -apx (or O()-apx)
for original problem.
177 / 292
One good, one bad news
Bad news:
1
Theorem (Rabinovith, Raz '95)
Any tree embedding for an n-yle
must have distortion (n).
1
1
1
1
Good news:
1
a random edge.
1 1 1 II Delete
For u; v 2 V with d(u; v) = k one has
1 1
1 1 dTT (u; v) = n k with probability knk and
d (u; v) = k with probability 1 n .
1
I Expeted distortion is at most 2 sine:
k
k
E [dT (u; v)℄ = (n k) + 1
k 2k
n
n
|
{z
k
}
|
{z
k
}
178 / 292
The Theorem
Theorem (Fakharoenphol, Rao, Talwar '03)
Given any metri (V; d), one an nd randomly (in time O(n2 ))
a tree metri (V [ U; dT ) suh that
I d(u; v) dT (u; v) 8u; v 2 V (i.e. dT dominates d)
I E [dT (u; v)℄ O(log n) d(u; v) 8u; v 2 V
That means the tree metri has an expeted O(log n) distortion.
The tree will ontain extra nodes U , whih were not
ontained in the original nodeset.
Remark:
2U
T
3
u d(u; v) v
179 / 292
Preliminaries
Assumptions:
I 2Æ = maxu;v2V fd(u; v)g
I d(u; v) > 1 8u 6= v
is diameter
Denition
A set system S is alled laminar if for every S1; S2 2 S one has
either S1 \ S2 = ; or S1 S2 or S2 S1 .
Idea: Obtain a random laminar family.
180 / 292
Clustering
Algorithm:
(1) Choose a random permutation on nodes V
(2) Choose 2 [0; 1℄ uniformly at random
(3) DÆ := fV g
(4) FOR i = Æ 1 DOWNTO 0 DO
(5) Assign every node to rst node (w.r.t. order ) that has
distane 2 2i 1
(6) All nodes that are assigned to the same node and are in the
same luster (in Di+1 ) form a new luster of Di
(7)
(2)
(4)
DÆ :
b
b
b
b
b
b
(8)
(1)
b
(6)
(3)
b
(5)
181 / 292
Clustering
Algorithm:
(1) Choose a random permutation on nodes V
(2) Choose 2 [0; 1℄ uniformly at random
(3) DÆ := fV g
(4) FOR i = Æ 1 DOWNTO 0 DO
(5) Assign every node to rst node (w.r.t. order ) that has
distane 2 2i 1
(6) All nodes that are assigned to the same node and are in the
same luster (in Di+1 ) form a new luster of Di
(7)
(2)
(4)
DÆ :
b
b
b
b
b
b
(8)
(1)
b
(6)
(3)
b
(5)
182 / 292
Clustering
Algorithm:
(1) Choose a random permutation on nodes V
(2) Choose 2 [0; 1℄ uniformly at random
(3) DÆ := fV g
(4) FOR i = Æ 1 DOWNTO 0 DO
(5) Assign every node to rst node (w.r.t. order ) that has
distane 2 2i 1
(6) All nodes that are assigned to the same node and are in the
same luster (in Di+1 ) form a new luster of Di
(7)
(2)
(4)
Di :
b
b
b
b
b
b
(8)
(1)
b
(6)
(3)
b
(5)
183 / 292
Clustering
Algorithm:
(1) Choose a random permutation on nodes V
(2) Choose 2 [0; 1℄ uniformly at random
(3) DÆ := fV g
(4) FOR i = Æ 1 DOWNTO 0 DO
(5) Assign every node to rst node (w.r.t. order ) that has
distane 2 2i 1
(6) All nodes that are assigned to the same node and are in the
same luster (in Di+1 ) form a new luster of Di
(7)
(2)
(4)
Di :
b
b
b
b
b
b
(8)
(1)
b
(6)
(3)
b
(5)
184 / 292
Clustering
Algorithm:
(1) Choose a random permutation on nodes V
(2) Choose 2 [0; 1℄ uniformly at random
(3) DÆ := fV g
(4) FOR i = Æ 1 DOWNTO 0 DO
(5) Assign every node to rst node (w.r.t. order ) that has
distane 2 2i 1
(6) All nodes that are assigned to the same node and are in the
same luster (in Di+1 ) form a new luster of Di
(7)
(2)
(4)
b
b
b
b
b
b
(8)
(1)
b
(6)
(3)
b
(5)
185 / 292
Clustering
Algorithm:
(1) Choose a random permutation on nodes V
(2) Choose 2 [0; 1℄ uniformly at random
(3) DÆ := fV g
(4) FOR i = Æ 1 DOWNTO 0 DO
(5) Assign every node to rst node (w.r.t. order ) that has
distane 2 2i 1
(6) All nodes that are assigned to the same node and are in the
same luster (in Di+1 ) form a new luster of Di
(7)
(2)
(4)
b
b
b
b
b
b
(8)
(1)
b
(6)
(3)
b
(5)
186 / 292
Clustering
Algorithm:
(1) Choose a random permutation on nodes V
(2) Choose 2 [0; 1℄ uniformly at random
(3) DÆ := fV g
(4) FOR i = Æ 1 DOWNTO 0 DO
(5) Assign every node to rst node (w.r.t. order ) that has
distane 2 2i 1
(6) All nodes that are assigned to the same node and are in the
same luster (in Di+1 ) form a new luster of Di
(7)
(2)
(4)
D0 :
b
b
b
b
b
b
(8)
(1)
b
(6)
(3)
b
(5)
187 / 292
Clustering
Algorithm:
(1) Choose a random permutation on nodes V
(2) Choose 2 [0; 1℄ uniformly at random
(3) DÆ := fV g
(4) FOR i = Æ 1 DOWNTO 0 DO
(5) Assign every node to rst node (w.r.t. order ) that has
distane 2 2i 1
(6) All nodes that are assigned to the same node and are in the
same luster (in Di+1 ) form a new luster of Di
(7)
D0 ; : : : ; DÆ :
(2)
(4)
b
b
b
b
b
b
(8)
(1)
b
(6)
(3)
b
(5)
188 / 292
Dening the tree metri
I Eah luster beomes an extra node
I Insert edge of ost 2i between S 2 Di ; S 0 2 Di 1 if S 0 S
DÆ
...
Di
Di 1
...
D0
I
2i 2i 2i
2i
2i
2V
Note that in the last iteration (i = 0) we assign eah node
to a luster enter at distane 2 20 1 1. Hene the
lusters of D0 are indeed singletons (sine
d(u; v) > 1 8u 6= v).
189 / 292
T
d
dominates d
Lemma
The tree metri dT dominates d, i.e. d(u; v) dT (u; v)8u; v 2 V
I
I
I
I
Suppose u; v are in the same Di luster, but separated by
Di 1
Cluster in Di have diameter 2 2 2i 1 2i+1
On the other hand dT (u; v) 2 2i .
Hene d(u; v) 2i+1 dT (u; v)
DÆ
...
Di
Di 1
...
D0
2i 2i 2i
u
2i
v
2i
2V
190 / 292
Proof of O(log n) average distortion
Lemma
For any u; v 2 V : E [dT (u; v)℄ = O(log n) d(u; v)
If only one of the nodes u; v is assigned to enter w in an
iteration i, then we say w uts edge (u; v) at level i.
I We want to harge the u-v distane to that luster enter
that uts the u-v edge
X
dTw (u; v) :=
2i+2
I
I
Then
i:w uts (u;v) at level i
dT (u; v) X
w2V
dTw (u; v)
sine: Suppose u; v are separated by DPi i(i.e.
they are in the
+1
T
same Di+1 luster). Then d (u; v) j=0 2 2j 2 2i+2 .
But in iteration i, we nd 2 luster enters w; w0 that ut
edge (u; v), for both dTw (u; v); dTw0 (u; v) 2i+2.
191 / 292
Proof of O(log n) average distortion (2)
I Assume w.l.o.g. that d(u; ws ) < d(v; ws ).
I Let w1 ; w2 ; : : : be nodes in inreasing distane from u
I ws an ut (u; v) only if
I
I
(A)
(B)
9 level i, where d(u; ws ) 2 2i 1 < d(v; ws )
u is assigned to ws
ws 1
ws
2 2
i
1
2 2i
w1
u
1
w2
ws 2
v
192 / 292
Proof of O(log n) average distortion (3)
I Assume for a seond: 9i : 2i 1 d(u; ws ) < d(v; ws ) < 2i .
I Then there is only one level i at whih ws might ut (u; v)
I By triangle inequality, the length of the interval
[d(u; ws ); d(v; ws )℄ is
d(v; ws ) d(u; ws ) d(u; v):
I Logsale length of interval is at most log2 2 2+d(u;v) .
i 1
2
+
d(u; v) log (1+x)2x d(u; v)
Pr[(A)℄ log2
2 2i 1
2i 1
d(u; ws )
d(v; ws )
Standard:
:::
20
2i 1
2i
21
0
1
i 1
i 1
2
Logsale:
193 / 292
Proof of O(log n) average distortion (3)
I Assume for a seond: 9i : 2i 1 d(u; ws ) < d(v; ws ) < 2i .
I Then there is only one level i at whih ws might ut (u; v)
I By triangle inequality, the length of the interval
[d(u; ws ); d(v; ws )℄ is
d(v; ws ) d(u; ws ) d(u; v):
I Logsale length of interval is at most log2 2 2+d(u;v) .
i 1
2
+
d(u; v) log (1+x)2x d(u; v)
Pr[(A)℄ log2
2 2i 1
2i 1
d(u; ws )
d(v; ws )
Standard:
i
1
2
2
2
:::
20
2i 1
2i
21
0 1
1
1
luster sizes
Logsale:
i 1
i 1
2
194 / 292
Proof of O(log n) average distortion (4)
I Next, ondition on (A).
Pr[u assigned to wsj(A)℄ Pr[ws 1st of w1; : : : ; ws w.r.t. ℄ = 1s
I If (A) & (B ) happen, this inurs ost of 2i+2 .
I Hene
d(u; v ) d(u; v) 1
E [dTw (u; v)℄ 2i+2 2 i 1 = O
2
s
s
I For general ase: Let Æi be length of
i ℄ Then applying the arguments
[d(u; ws ); d(v; ws )℄ \ [2i 1 ; 2P
)
T
for eah Æi: E [dw (u; v)℄ i Æi O( 1s ) O( d(u;v
s ).
I Then
nX2 nX2
d(u; v) T
T
= O(log n)d(u; v)
E [d (u; v)℄ E [d (u; v)℄ = O
s
s
s=1
ws
s=1
s
195 / 292
Distortion must be (log n)
Denition (Expander graph)
An undireted graph G = (V; E ) is alled an (n; d; )-expander
graph if
I jV j = n
I onstant degree: deg(v) = d 8v 2 V
I edge expansion
jÆ(S )j
= min
1jS jn=2 jS j
I
I
Random d-regular graphs are good expanders w.h.p.
The diameter of expanders is (log n).
Theorem (Bartal '96)
A randomized tree embedding of any (n; d; )-expander graph
(d; onstants) must have an edge with expeted distortion of
(log n).
196 / 292
Steiner nodes are not really neessary
Theorem (Gupta '01)
Given a weighted tree T = (V; E; ), where the node set
V = R[_ S onsists of required verties R and Steiner nodes S .
Then in linear time, one an nd a weighted tree
T = (R; E ; ) suh that
dT (u; v) dT (u; v) 8 dT (u; v)
where dT and dT are the indued tree metries.
2S
T
2R
T
197 / 292
Derandomization
Theorem (FRT + Gupta + Charikar et al.)
Given a omplete graph G = (V; E ) with metri ost funtion
: E ! Q + . One an nd deterministially, in polynomial time:
spanning trees T1 ; : : : ; Tq on V , osts di : Ti ! Q + and
probabilities i > 0, 1 + : : : + q = 1 where q = poly(n). Then
I For u; v 2 V and i = 1; : : : ; q one has (u; v) dTi (u; v)
I For any u; v 2 V one has
q
X
i=1
Here dTi : V
i dTi (u; v) O(log n) (u; v):
V ! Q+
is the tree metri indued by Ti and di .
198 / 292
Part 20
Introdution into Primal dual
algorithms
Soure:
Approximation Algorithms (Vazirani, Springer Press)
199 / 292
A generi problem
Situation: We want to approximate a problem, whih (in
many ases) is of the form
min
n
X
j =1
n
X
j =1
Examples so far:
Cover,: : :
j xj
aij xj
xj
bi 8i = 1; : : : ; m
2 f0; 1g 8j = 1; : : : ; n
,
,
Set Cover Steiner tree Vertex
200 / 292
A primal-dual pair
Primal "overing" LP:
min
n
X
j =1
n
X
j =1
j xj
aij xj
xj
(P )
bi 8i = 1; : : : ; m
0 8j = 1; : : : ; n
Dual "paking" LP:
max
m
X
i=1
m
X
i=1
bi yi
(D )
aij yi
j 8j = 1; : : : ; n
yi
0 8i = 1; : : : ; m
201 / 292
A generi Approximation algorithm
Generi primal-dual algorithm:
(1) x := 0; y = 0
(2) WHILE x not feasible DO
(3) Inrease dual variables in a suitable way until some dual
onstraint j beomes tight
(4) Set xj := 1
(5) RETURN x
Generi analysis:
I Show: At the end x is integer and feasible for primal
I Show: At the end y is feasible for dual
P
P
I Show: nj=1 j xj m
i=1 bi yi ( is the apx fator)
primal solutions
dual solutions
0
Pm
i=1 bi yi
OP Tf
OP T
fator of Pn
j =1 j xj
202 / 292
Relaxed omplementary slakness
Lemma
Let ; 1. Let x; y be primal/dual feasible solutions obtained
by the algorithm. If
P
(A) Relaxed primal ompl. slak.: xj > 0 )
j m
i=1 aij yi
Pn
(B) Relaxed dual ompl. slak.: yi > 0 ) j=1 aij xj bi
Then AP X OP Tf .
I
Let AP X be the ost of the produed solution. Then
AP X
=
(B)
n
X
j =1
j xj
m
X
i=1
m
X
n
(A) X
yi bi
xj aij yi
i=1
j =1
y dual feasible
=
m
X
i=1
yi
n
X
j =1
aij xj
OP Tf
203 / 292
Part 21
Steiner Forest
Soure:
Approximation Algorithms (Vazirani, Springer Press)
204 / 292
Steiner Forest
Problem: Steiner Forest
I Given: Undireted graph G = (V; E ), edge ost
: E ! Q + , terminal pairs (s1 ; t1 ); : : : ; (sk ; tk )
I Find: Minimum ost subgraph F onneting all terminal
pairs:
OP T
= Fmin
E
nX
s1
3
3
s2
e2F
(e) j 8i = 1; : : : ; k : F
8
3
8
t1
3
5
onnets si and ti
o
s3
6
3
t2
5
t3
205 / 292
Steiner Forest
Problem: Steiner Forest
I Given: Undireted graph G = (V; E ), edge ost
: E ! Q + , terminal pairs (s1 ; t1 ); : : : ; (sk ; tk )
I Find: Minimum ost subgraph F onneting all terminal
pairs:
OP T
= Fmin
E
nX
s1
3
3
s2
e2F
(e) j 8i = 1; : : : ; k : F
8
3
8
t1
3
5
t2
5
o
s3
F
3
onnets si and ti
6
t3
206 / 292
The LP relaxation
I For any S V dene ut requirement
(
1 if 9i : jS \ fsi; tigj = 1
f (S ) =
0 otherwise
Primal LP relaxation:
X
min exe
e2E
X
e2Æ(S )
Dual LP:
max
X
S V
(P )
xe
f (S ) 8S V
xe
0 8e 2 E
f (S )yS
X
S :e2Æ(S )
(D)
yS
e 8e 2 E
yS
0 8S V
207 / 292
Preliminaries
I For F E; S V : ÆF (S ) = ffu; vg 2 F j u 2 S; v 2= S g
I A ut S V is violated by F E , if there is a terminal
pair (si; ti ) with jfsi ; tig \ S j = 1 but ÆF (S ) = ;
I A ut S is ative w.r.t. F , if S is violated and minimal
(i.e. there is no subset S 0 S that is also violated).
I P
An edge e is tight w.r.t. a dual solution (yS )S if
S :e2Æ(S ) yS = e
(i.e. if the dual onstraint of e satised with equality).
208 / 292
The algorithm
(1) F := ;, y := 0
(2) WHILE 9 violated ut DO
(5)
(3) Inrease simultaneously yS for all ative uts S , until some
edge e gets tight
(4) Add the tight edge e to F
Compute an arbitrary minimal feasible solution F 0 F
209 / 292
The ative uts
Lemma
The ative uts w.r.t. F
E are onneted omponents of F .
I Consider ative ut S (S minimal, f (S ) = 1, ÆF (S ) = ;).
I ÆF (S ) = ; ) onneted omponents of F are either fully
ontained in S or fully outside
I S is violated, hene there is a pair jfsi ; ti g \ S j = 1
I The onneted omponent of F inside S that ontains si is
also violated. Hene, S is a single onneted omponent (or
we would have a ontradition).
F
ti
F
S
impossible
si
F
S
impossible
orret
210 / 292
Example
s1
6
6
9
16
12
s2
t1
20
19
12
t2
211 / 292
Example
s1
t1
20
6
6
9
16
12
19
12
s2
t2
ative set
212 / 292
Example
edges added to F
6
s1
6
6
6
9
16
12
6
t1
20
s2
19
12
t2
6
yS = 6 for S = fs2 g
213 / 292
Example
6
s1
t1
20
6
16
12
8
s2
6
6
2
9
2
19
12
t2
8
214 / 292
Example
6
s1
t1
20
6
16
12
8
s2
6
6
2
1
9
3
19
12
t2
9
215 / 292
Example
edge not neessary
6
s1
t1
20
6
16
12
8
6
6
2
2
9
3
19
12
s2
t2
9
F at the end of WHILE loop
216 / 292
Example
6
s1
t1
20
6
6
16
12
8
6
2
2
s2
9
3
19
12
t2
9
Solution F 0
217 / 292
Feasibility
Lemma
F 0 is a feasible solution.
I Let F be the solution at the end of the WHILE loop.
I F is feasible, beause there is no violated ut.
I We do not delete neessary edges, hene F 0 is also
feasible.
Lemma
y is dual feasible, i.e.
P
S :e2Æ(S ) yS
e for all e 2 E .
Eah
time that an edge e gets tight (i.e.
P
S :e2Æ(S ) yS = e ), we add it to F .
I We inrease yS only for violated uts { not for uts
ontaining edges of F .
I
218 / 292
The main analysis (1)
Lemma
Let y be the dual solution at the end of the algorithm. Then
AP X =
X
e2F 0
I
e e tight
=
X
X
e2F 0
e 2
X
e2F 0 S :e2Æ(S )
yS
X
S V
=
yS 2 OP Tf :
X
S V
()
jÆF 0 (S )j yS X
S V
2yS
Consider any iteration i. Let be the amount by whih the
dual variables yS were inreased. We show (*) by proving
X
jÆF 0 (S )j 2 #ative sets in it.i
S
ative in it.i
219 / 292
The main analysis (2)
I Consider an intermediate iteration i with intermediate F .
I Remark: F 0 nF might ontain edges that are added later
F nF 0 might ontain edges that are deleted at the end.
I Claim:
X
jÆF 0 (S )j 2 #ative sets in iteration i
I
S
ative in it.i
Shrink onneted omponents of F ! H 0 (S beomes node
vS ). Nodes vS steming from ative uts S are ative nodes,
others are inative nodes
ative S
I H0
F
is a forest. Degrees are preserved.
inative S
220 / 292
The main analysis (2)
I Consider an intermediate iteration i with intermediate F .
I Remark: F 0 nF might ontain edges that are added later
F nF 0 might ontain edges that are deleted at the end.
I Claim:
X
jÆF 0 (S )j 2 #ative sets in iteration i
I
S
ative in it.i
Shrink onneted omponents of F ! H 0 (S beomes node
vS ). Nodes vS steming from ative uts S are ative nodes,
others are inative nodes
ative S
I H0
F
F0
is a forest. Degrees are preserved.
inative S
221 / 292
The main analysis (2)
I Consider an intermediate iteration i with intermediate F .
I Remark: F 0 nF might ontain edges that are added later
F nF 0 might ontain edges that are deleted at the end.
I Claim:
X
jÆF 0 (S )j 2 #ative sets in iteration i
I
S
ative in it.i
Shrink onneted omponents of F ! H 0 (S beomes node
vS ). Nodes vS steming from ative uts S are ative nodes,
others are inative nodes
H0 :
ative vS
0
F
inative vS
I H 0 is a forest. Degrees are preserved.
222 / 292
The main analysis (2)
H0 :
ative vS
F0
inative vS
Consider non-singleton leaf vS . Edge to vS was not deleted.
Hene f (S ) = 1. But then S was ative (sine S is a
onneted omponent of F at iteration i).
I Average degree over all nodes in a forest is 2 (sine #
edges # nodes) and eah edge ontributes at most 2 to
the degrees.
I Inative nodes are inner nodes of degree 2, hene average
degree of ative nodes average degree 2.
I
223 / 292
Deleting redundant edges is ruial
:::
1
1
1
s1
1
1
1
v1
4
1
1
vn
t1
1
2
Without the pruning step at the end of the
algorithm, the solution would ost n + 4 instead of 4.
Observation:
224 / 292
Conlusion
Theorem
The primal dual algorithm produes a 2-approximation in time
O(n2 log n).
Remark: The algorithm works whenever the requirement
funtion f : 2V ! f0; 1g is proper, that means
I f (V ) = 0
I f (S ) = f (V nS ) (symmetry)
I If A; B V are disjoint and f (A [ B ) = 1 then f (A) = 1
or f (B ) = 1.
Note: Funtion f for Steiner Forest is proper.
225 / 292
State of the art
96 -approximation algorithm unless NP = P
I There is no 95
(same ratio as for the speial ase of Steiner tree).
I There is still no better than 2-approximation known.
I The integrality gap of the onsidered LP is in fat exatly
2.
I There is also no other LP formulation known, whih might
have a smaller gap.
226 / 292
Part 22
Faility Loation
Soure:
Approximation Algorithms (Vazirani, Springer Press)
227 / 292
Faility Loation
Problem: Faility Loation
I Given: Failities F , ities C , opening ost fi for every
faility i. Metri ost ij for onneting ity j to faility i.
I Find: Set of failities I and an assignment : C ! I of
ities to opened failities, minimizing
the total ost:
o
nX
X
OP T = min
fi + (j );j
I F;:C !I
i2I
F
C
j
j 2C
Without the metri
assumption, the problem
beomes (log n)-hard.
I We assume w.l.o.g. ij ; fi 2 Z+
I Remark:
ij
i
fi
228 / 292
Faility Loation
Problem: Faility Loation
I Given: Failities F , ities C , opening ost fi for every
faility i. Metri ost ij for onneting ity j to faility i.
I Find: Set of failities I and an assignment : C ! I of
ities to opened failities, minimizing
the total ost:
o
nX
X
OP T = min
fi + (j );j
I F;:C !I
i2I
F
C
j
j 2C
Without the metri
assumption, the problem
beomes (log n)-hard.
I We assume w.l.o.g. ij ; fi 2 Z+
I Remark:
ij
i
fi
229 / 292
The primal dual pair
Primal LP: min X ij xij + X fi yi
P
i;j
i2F xij
xij
xij
yi
Dual LP:
max
j
j 2C ij
j
ij
P
X
j 2C
i2F
1
yi
0
0
8j 2 C
8i 2 F 8j 2 C
8i 2 F 8j 2 C
8i 2 F
j
ij + ij
fi
0
0
8i 2 F 8j 2 C
8i 2 F
8j 2 C
8i 2 F 8j 2 C
Intuition:
I j is the amount that ity j "pays" in total.
I ij is what ity j "pays" to open faility i.
230 / 292
The algorithm - Phase 1:
(1) Initially all ities are unonneted
(2) := 0; := 0; Ft := ;
(3) WHILE not all ities are onneted DO
(4) FOR ALL unonneted ities j DO
(5)
Inrease j (by 1 per time unit)
(6)
For tight edges j = ij + ij inrease also ij
(7) IF Pj ij = fi (new) THEN
(8)
open faility i temporarily (Ft := Ft [ fig)
(9)
FOR ALL ities j where edge (i; j ) is tight DO
(10)
onnet ity to faility i
(11)
faility i is onnetion witness of j : w(j ) := i
Phase 2:
(1) Let H = (Ft ; E 0 ) with (i; i0 ) 2 E 0
if 9j 2 C : ij ; i0 j > 0
(2) Open a maximal independent set I Ft
(3) FOR ALL j 2 C DO
(4) IF 9j 2 I : ij > 0 THEN '(j ) := i (j diretly onn.)
(5) ELSE IF w(j ) 2 I THEN '(j ) := w(j ) (j diretly onn.)
(6) ELSE '(j ) := a neighbour of w(j ) in H (j indir. onn.)
231 / 292
Example:
Phase 1 - Time: 0
1 = 0
2 = 0
3 = 0
4 = 0
C
10
1
3
5
3
1
10
5
3
1
1
F
f1 = 4
f2 = 5
3
f3 = 2
232 / 292
Example:
Phase 1 - Time: 1
1 = 1
2 = 1
3 = 1
4 = 1
C
10
1
3
5
3
1
10
5
3
1
1
=0
=0
F
f1 = 4
=0
=0
3
f2 = 5
f3 = 2
233 / 292
Example:
Phase 1 - Time: 2
1 = 2
2 = 2
3 = 2
4 = 2
C
10
1
3
5
3
1
10
5
3
1
1
=1
=1
F
f1 = 4
=1
=1
3
f2 = 5
f3 = 2
234 / 292
Example:
Phase 1 - Time: 3
onn.: w(1) = 1; 1 = 3
onn.: w(2) = 1; 2 = 3
onn.: w(3) = 1; 3 = 3
4 = 3
C
10
1
3
5
3
1
10
5
3
1
1
=2
=2
=0
F
f1 = 4 temp.
opened
=2
=2
f2 = 5
=0
3
=0
f3 = 2
235 / 292
Example:
Phase 1 - Time: 4
onn.: w(1) = 1; 1 = 3
onn.: w(2) = 1; 2 = 3
onn.: w(3) = 1; 3 = 3
onn.: w(4) = 2; 4 = 4
C
10
1
3
5
3
1
10
5
3
1
1
=2
=2
=0
F
f1 = 4 temp.
opened
f2 = 5 temp.
opened
=2
=2
=1
3
=1
f3 = 2
236 / 292
Example:
Phase 2: Graph H
C
F
f1 = 4 temp.
opened
H
f2 = 5 temp.
opened
f3 = 2
237 / 292
Example:
Phase 2: The solution
C
F
2 I (faility opened)
f3 = 2
238 / 292
Analysis
Theorem
P
P
P
One has j 2C '(j );j + i2I fi 3 j 2C j .
We aount the dual "payments"
fj
:= payment for opening :=
j := payment for onnetion
:=
(
'(j );j
(
0
'(j );j
j
if j diretly onneted
if j is indiretly onn.
if j diretly onneted
if j is indiretly onn.
Claim: j = fj + j .
I
I
For indiretly onneted ities: lear
For diretly onneted ities: j = '(j);j + '(j);j beause
edge ((j ); j ) was tight.
239 / 292
Bounding the opening osts
Lemma
The dual pries pay for the opening ost, i.e.
X
i2I
fi =
X f
j 2C
j :
A faility i 2 I was temporarily opened beause Pj ij = fi
All j with ij > 0 must be diretly onneted to i beause:
We opened an independent set in H in Phase 2, hene any
i0 2 Ft with i0 j > 0 is not in I
I Thus all j with ij > 0
i2I
ij > 0
X
X
i temp opened
f
j =
ij
= fi
I
I
j :(j )=i
I
j :ij >0
The
laim X
follows
from X
X
X
f
j =
fj = fi
j 2C
i2I j :(j )=i
j
i0 j > 0
2H
i0 2= I
i2I
240 / 292
Bounding the onnetion ost
Lemma
For any ity j 2 C one has '(j );j
3j .
If j diretly onneted, then even j = '(j);j . Next,
suppose j is indiretly onneted.
I Then there is an edge (w(j ); (j )) 2 H (sine j was
indiretly onneted).
I This edge implies that there is a j 0 2 C with
'(j );j 0 > 0; w(j );j 0 > 0.
I
j
j0
tight: j w(j);j
w(j);j > 0
0
(i);j > 0
0
w(j ) 2= I
2H
(j ) 2 I
241 / 292
Bounding the onnetion ost (2)
j
tight: j w(j);j
w(j);j > 0
w(j ) 2= I
Event w(j);j > 0 only
j0
2H
happened if j w(j);j . For
(i);j > 0
(j ) 2 I
the same reason: j0 w(j);j0
and j0 (j);j0 .
I Claim j j 0 : Consider the time t, when w(j ) was
temporarily opened. Sine w(j ) is onnetion witness of j ,
j t. At this time t, it was w(j );j 0 > 0 (sine if
w(j );j 0 = 0 at that time, then w(j );j 0 = 0 forever). At the
latest at this time t, also j 0 was onneted and j0 stopped
growing. Hene j t j0 .
I Then
metri ineq.
|w{z(j);j} + |w{z
(j );j
(j );j}0 + |({z
j );j 0 3j = 3j
}
I
0
0
j
j0 j j0 j
242 / 292
Conlusion
Theorem
The algorithm produes a 3-approximation in time
O(m log(m)), where m = jC j jF j is the number of edges.
State of the art:
Theorem (Byrka '07)
There is a 1:499-apx for
I
Faility Loation
.
The integrality gap for the onsidered LP lies in
[1:463; 1:499℄.
Theorem
There is no polynomial time 1:463-apx for
unless NP DTIME(nO(log log n) ).
Faility Loation
243 / 292
Part 23
Insertion: Semidefinite Programming
Soure:
Approximation Algorithms (Vazirani, Springer Press)
244 / 292
Positive denite matries
Denition (positive semidenite Matrix)
A matrix A 2 Rnn is alled positive semi-denite if
8x 2 Rn : xT Ax 0:
Theorem (Diagonalization)
Let A 2 Rnn be symmetri (i.e. aij = aji), then A is
diagonalizable, i.e. one an write
0
1
0 .
.. 1 1 0 : : : 0 0 : : : v : : :
..
1
.
B
C
C B 0 2 : : : 0 C B
.
.
A=B
:
:
:
v
v
C 1
nA B
...
.
: : : : : :
: : :A
..
..
. {z . } 0 0 : : : n | : : : v{zn : : :
|
{z
}
|
=L
=LT
=D
1
C
A
}
where vi 2 Rn is orthonormal Eigenvetor for Eigenvalue i , i.e
Avi = i vi , kvi k2 = 1; viT vj = 0 8i 6= j .
245 / 292
Some useful results
Lemma
Let A 2 Rnn be a symmetri matrix (vi orthonormal
Eigenvetor for i ). Then the following statements are
equivalent
(1) 8x 2 Rn : xT Ax 0
(2)
(3)
I
I
I
i 0 8i
There is W
2 Rnn with A = W T W
T
(1) ) (2): 0 viT Avi = viT (i vi) = i v|{z}
i vi = i
=1p
p T
p p
(2) ) (3): A = LDLT = L D DLT = ( DLT )T |( DL
)
{z }
=:W
(3) ) (1): For any x 2 Rn :
xT Ax = xT (W T W )x = (W x)T (W x) 0
Remark: Matrix W an be found by Cholesky deomposition
in O(n3 ) arithmeti operations (if p ounts as 1 operation). 246 / 292
The semidenite one
0
Write Y 0 if Y is positive semidenite.
The set
fY 2 Rnn j Y 0; Y symmetrig = onefxxT j x 2 Rn g
is a onvex, non-polyhedral one.
I Def.:
I Fat:
247 / 292
A semidenite program
Given:
I
I
Obj. funtion vetor C = (ij )1i;jn 2 Q nn
Linear onstraints Ak = (akij )1i;jn 2 Q nn ; bk 2 Q
max
X
i;j
X
i;j
I
ij yij
akij yij
Y
Y
bk 8k = 1; : : : ; m
symmetri
0
Frobenius inner produt: C Y := Pni=1 Pnj=1 ij yij
248 / 292
A semidenite program
Given:
I
I
Obj. funtion vetor C = (ij )1i;jn 2 Q nn
Linear onstraints Ak = (akij )1i;jn 2 Q nn ; bk 2 Q
max C Y
Ak Y
Y
Y
I
bk 8k = 1; : : : ; m
symmetri
0
Frobenius inner produt: C Y := Pni=1 Pnj=1 ij yij
249 / 292
Pathologial situations
I Case: All solutions might be irrational. x =
unique solution of
0
1 x 0 01
Bx 2 0 0 C
B
C
0 0 2x 2 A 0
0 0 2 x
4
3
2
1
0
1
2
1
3
4
p
2 is the
2
p
1 2 2
x
I Case: All sol. might have exponential
enoding
1
x
i
1
. Then
length. Let Q1 (x) = x1 2; Qi (x) := x
i 1 xi
0
Q1 (x)
0 ::: 0 1
B 0
Q2 (x) : : :
0 C
B
C
Q(x) := B
0
... C
A
0
0 : : : Qn(x)
if and only if Q1(x); : : : ; Qn(x) 0. I.e. x1 2 0 and
xi x2i 1 , hene xn 22 1 .
:::
:::
...
n
250 / 292
Solvability of Semidenite Programs
Theorem
Given rational input A1 ; : : : ; Am ; b1 ; : : : ; bm ; C; R and " > 0,
suppose
= maxfC Y j Ak Y bk 8k; Y symmetri; Y 0g
is feasible and all feasible points are ontained in B (0; R). Then
SDP
one an nd a Y with
Ak Y bk +"; Y symmetri; Y 0
suh that C Y SDP ". The running time is polynomial in
the input length, log(R) and log(1=") (in the Turing mahine
model).
251 / 292
Solving the separation problem
I Remark: We show that we an solve the separation
problem, ignore numerial inauraies.
I Let infeasible Y be given, we have to nd a separating
hyperplane.
(1) Case Ak Y < bk : return "Ak Y bk violated"
(2) Case Y not symmetri: Find the i; j with yij < yji. Return
"yij yji violated".
(3) Case Y not positive semidenite. Find eigenvetor v with
Eigenvalue < 0, i.e. Y v = v. Then
X
viT vj yij = vT Y v < 0
i;j
hene return "Pi;j viT vj yij 0 violated".
252 / 292
Vetorprograms
Idea:
,
,
SDP:
max
X
i;j
X
i;j
Y symmetri and Y 0
9W = (v1 ; : : : ; vn) 2 Rnn : W T W
9v1; : : : ; vn 2 Rn : yij = viT vj
=Y
Vetor program:
max
ij yij
akij
yij bk 8k
Y
Y
sym.
0
X
i;j
X
i;j
ij viT vj
akij viT vj
vi
bk 8k
2
Rn
8i
Observation
The SDP and the vetor program are equivalent.
253 / 292
Part 24
MaxCut
Soure:
I Approximation Algorithms (Vazirani, Springer Press)
I Improved Approximation Algorithms for Maximum Cut and
Satisability Problems Using Semidenite Programming
(Goemans, Williamson) (link)
254 / 292
Problem denition
Problem: MaxCut
I Given: Complete undireted graph G = (V; E ), edge
weights w : E ! Q +
I Find: Cut maximizing the weight of separated edges
OP T
= max
S V
i
e2Æ(S )
w(e)
o
S
wij = 1
j
n X
0
0
255 / 292
A vetor program
I Choose deision variable for any node i 2 V :
(
( 1; 0; : : : ; 0) i 2 S
vi =
( 1; 0; : : : ; 0) i 2= S
I An exat MaxCut vetor program:
X wij
(1 viT vj )
max
2
(i;j )2E
viT vi = 1 8i = 1; : : : ; n
vi 2 Rn 8i = 1; : : : ; n
vi = (1; 0; : : : ; 0) 8i = 1; : : : ; n
I Then
z
=1 if (i;j )2Æ(S ); 0 o.w.
1
wij (1
2
(i:j )2E
X
}|
viT vj
| {z }
{
= 1 if (i;j )2Æ(S )
+1 o.w.
)=
X
(i;j )2Æ(S )
wij
256 / 292
A vetor program (2)
The relaxed vetor program:
X wij
max
(1 viT vj )
(i;j )2E
2
viT vi
vi
vj
0
1
os()
=viT vj
b
wij
= 1 8i = 1; : : : ; n
2 Rn 8i = 1; : : : ; n
wij
2
vi
(1 os())
257 / 292
A physial interpretation
I n vetors on n-dim unit ball.
I Repulsion fore of wij between vi and vj
Example:
Graph G
3
4
SDP solution:
v4
2
wij = 1
v2
1
v5
4
5
v1
v3
5
I OP T = 4
I For SDP solution, plae v1 ; : : : ; v5 equidistantly on 2-dim.
subspae. SDP = 5 12 (1 os( 45 )) 4:52
I Hene integrality gap 1:13.
258 / 292
The algorithm
Algorithm:
(1) Solve MaxCut vetor program ! v1 ; : : : ; vn 2 Q n
(More preisely: Solve the equivalent SDP, obtain a matrix
Y 2 Q nn . Apply Cholesky deomposition to Y to obtain
v1 ; : : : ; vn )
(2) Choose randomly a vetor r from n-dimensional unit ball
(3) Choose ut S := fi j vi r 0g
Theorem
P
E [ (i;j )2Æ(S ) wij ℄ 0:87 OP T (i.e. the algorithm gives an
expeted 1:13-apx).
259 / 292
Proof
Consider 2 vetors vi; vj with angle
Let R a be the 1-dim.
intersetion of the n 1-dim.
hyperplane x r = 0 with the plane
spanned by vi ; vj
I a has a random diretion
I vi ; vj are separated
, they lie on dierent sides of line aR
, a lies in one of the 2 gray ars of
angle I Pr[vi and vj separated℄ = 2 2 = I Expeted ontribution to AP X is wij I
2 [0; ℄.
vj
(i; j ) 2 Æ(S ) :
aR
0 aR
vi
(i; j ) 2= Æ(S ) :
vj
vi
a
r
b
vj
0 vi
260 / 292
Proof (2)
I Expeted ontribution of edge (i; j ) to AP X is wij I Contribution of edge (i; j ) to SDP is wij 12 (1 os())
E [AP X ℄
=
0:878:
min
1
0 2 (1 os())
SDP
1
2
1
0:87
=
=
(1 os())
1 (1
2
os())
261 / 292
State of the art
Theorem (Khot, Kindler, Mossel, O'Donnell '05)
There is no polynomial time < 1:138-approximation algorithm
(unless the Unique Games Conjeture is false).
I
That means the presented approximation is the best
possible.
262 / 292
Part 25
Max2Sat
Soure:
Approximation Algorithms (Vazirani, Springer Press)
263 / 292
Problem denition
Problem: Max2Sat
V
I Given: SAT formula C 2C C on variables x1 ; : : : ; xn . Eah
lause C ontains at most 2 literals.
I Find: Truth assignment maximizing the number of satised
lauses
OP T =
max
a=(a ;:::;a )2f0;1g
1
n
n
C
2 C j C true for assignment a
I Example:
(
x1 _ x2 ) ^(x1 _ x2 ) ^ (x1 _ x2 ) ^ (x1 _ x2 ) ^ x1
| {z }
lause
Optimal assignment: a = (0; 1) with 4 satised lauses.
Problem is NP-hard though testing wether all
lauses an be satised is easy.
I Remark:
264 / 292
A quadrati program
I Goal: Write Max2Sat as quadrati program
X
max aij (1 + yiyj ) + bij (1 yiyj )
i;j
yi2 = 1
yi 2 Z
for suitable oeÆients aij ; bij .
I Here yi = 1 xi true; yi = 1 xi false
I Let y0 := 1 be auxiliary variable.
I Write
(
1 if lause C true for y
v(C ) =
0 otherwise
I For lauses with 1 literal
1 + y0yi ; v(xi ) = 1 y0yi
v(xi ) =
2
2
265 / 292
A quadrati program (2)
I For lause xi _ xj
v(xi _ xj )
= 1
v(xi ) v(xj ) = 1
1
=1
y0 yi
2
1 2y0yj
z}|{
1
= 4 (3 + y0yi + y0yj y02 yiyj )
= 1 +4y0yi + 1 +4y0yj + 1 4yiyj
I Similar for xi _ xj and xi _ xj .
I We
obtain promised oeÆients aij ; bij by summing up
P
C 2C v (C ).
I Now: Relax the quadrati program to a (solvable) vetor
program.
266 / 292
The algorithm
Algorithm:
(1) Solve MaxCut vetor program
X max
aij (1 + vi vj ) + bij (1
0i<j n
vi2 = 1 8i = 0; : : : ; n
vi 2 Rn+1
(2) Choose randomly a vetor r from
n-dimensional unit ball
(3) Let yi := 1 for all i that are on
the same side of the hyperplane
x r = 0 as v0 (the "truth" vetor)
v i vj )
false
vj
0
aR
v0
vi
true
Theorem
Let AP X := #satised lauses. Then E [AP X ℄ 0:87 SDP .
267 / 292
Analysis
Case: Term bij (1 vi vj ) with angle between vi ; vj
I Contribution to E [AP X ℄: 2bij Pr[yi 6= yj ℄ = 2bij I Contribution to Vetor program: bij (1 os())
= 0:878
I Gap: min0 1 2os(
))
Case: Term aij (1 + vi vj ) with angle between vi ; vj
I Contribution to E [AP X ℄: 2aij Pr[yi = yj ℄ = 2aij (1
I Contribution to Vetor program: aij (1 + os())
2(1 =) 0:878
I Gap: min0 1+os(
))
Case: yi 6= yj
vj
0 vi
vj
vi a
b
r
)
Case: yi = yj
vj
aR
0 vi
aR
268 / 292
State of the art
Theorem (Feige, Goemans '95)
There is a 1:0741-apx for Max2Sat.
Theorem (Lewin, Livnat, Zwik '02)
There is a 1:064-apx for Max2Sat.
Theorem (Hastad '97)
There is no 1:0476-apx for
Max2Sat
(unless NP = P).
Theorem (Khot, Kindler, Mossel, O'Donnell '05)
There is no polynomial time 1:063-apx for Max2Sat (unless
the Unique Games Conjeture is false).
269 / 292
Part 26
Budgeted Spanning Tree
Soure:
The Constrained Minimum Spanning Tree Problem
(Goemans, Ravi) (link)
270 / 292
The Budgeted Spanning Tree problem
Problem: Budgeted Spanning Tree
I Given: Undireted graph G = (V; E ) with edge osts
: E ! Q + and edge lengths ` : E ! Q + . Budget B .
I Find: Spanning tree T minimizing the ost, while not
exeeding the budget
n X
o
X
e j `e B OP T =
max
spanning tree T
e2T
(3; 1)
(1; 2)
ost (e)
(0; 2)
e2T
(2; 1)
(2; 1)
length `(e)
B=4
271 / 292
The Budgeted Spanning Tree problem
Problem: Budgeted Spanning Tree
I Given: Undireted graph G = (V; E ) with edge osts
: E ! Q + and edge lengths ` : E ! Q + . Budget B .
I Find: Spanning tree T minimizing the ost, while not
exeeding the budget
n X
o
X
e j `e B OP T =
max
spanning tree T
e2T
(3; 1)
(1; 2)
ost (e)
(0; 2)
T
e2T
(2; 1)
(2; 1)
length `(e)
B=4
272 / 292
Budgeted Spanning Tree is NP-hard
Reall that Partition is (weakly) NP-hard:
Problem: Partition
P
I Given: Numbers a1 ; : : : ; an 2 N , S := ni=1 ai
P
I Find: I f1; : : : ; ng : i2I ai = S=2
Redution to
Budgeted Spanning Tree
:
(a1 ; 0) (a2 ; 0)
(an; 0)
:::
(0; a1 ) (0; a2 )
(0; an )
I Budget B := S=2. There is a feasible tree T of ost
(T ) B; `(T ) B if and only if there is a Partition
solution.
I Problem also NP-hard for simple graphs (our algorithm
will also work for multigraphs).
I Reall: The Spanning Tree problem without a budget is
easy.
273 / 292
Lagrangian Relaxation
Original problem:
minT (T )
T spanning tree
`(T ) B
Lagrangian Relaxation:
minT (T ) + z (`(T )
T spanning tree
B)
:= OP TLR (z)
:= OP T
Lemma
For any Lagrange multiplier z 0: OP TLR (z ) OP T .
I
Let T be the optimum solution: (T ) = OP T; `(T ) B .
Then
OP T
z
0
}|
{
z (`(T ) B ) OP TLR (z )
= (T ) (T ) + |{z}
| {z }
0
0
274 / 292
Solving the Lagrangian relaxation
Lemma
A sol. z ; T1 ; T2 an be omputed in poly-time where OP TLR =
OP TLR (z ) is attained by T1 ; T2 , `(T1 ) B `(T2 ).
I Assume w.l.o.g. (e); `(e) 2 Z. amax := maxf(e); `(e)g
I For any spanning tree T , let gT (z ) := (T ) + z (`(T ) B )
I OP TLR (z ) = minT fgT (z )g. Hene OP TLR (z ) is onave.
OP TLR
gT1 (z )
b
OP TLR (z )
z
OP TLR (z ) attained by T : OP TLR (z ) attained by T :
(T )OP TLR ; `(T )B
(T )OP TLR ; `(T )B
gT2 (z )
n amax
z
275 / 292
Solving the Lagrangian relaxation (2)
I For a given z , hoose 0 (e) := (e) + z `(e), then
OP TLR (z ) = min f(T )+z (`(T ) B )g = min f0 (T )g z B
sp.tree T
sp.tree T
I OP TLR (0) OP TLR (z )
I OP TLR (n amax ) 0 (if there is no tree with budget < B ,
then MST w.r.t. 0(e) := `(e) + na1 (e) is optimal).
I Perform binary searh (needs O(log(n amax )) iterations):
max
(1) L := 0; R := n amax 1
(2) WHILE jL Rj 4n2 a2max DO
(3) z := L R
(4) T := M ST for ost funtion 0 (e) := (e) + z `(e)
(5) IF `(T ) > B THEN L := z ELSE R := z
+
2
z := rational number in [L; R℄ with min. denominator
T1 := argminT fgT (z ")g
T2 := argminT fgT (z + ")g (" := 8n2 1a2max should suÆe)
I Use: z 2 Zq for some q 2 f1; : : : ; 4n2 a2maxg
(6)
(7)
(8)
276 / 292
An example
gfe2 g (z ) = 0 + (2
e1 : (2; 0)
e2 : (0; 2)
e3 : (3=2; 1)
B=1
2
gfe3 g (z )
OP TLR = 11
0
1) z
OP TLR (z )
0
z
z 1= 1
2
gfe g (z ) = 2 + (0 1) z
1
I
In this example OP T = 23 ; OP TLR = 1
277 / 292
Obtaining 2 trees diering in 2 edges
Lemma
One an nd opt. Lagrange solutions T1 ; T2 with
`(T1 ) B; `(T2 ) B whih dier in exatly 2 edges.
I Let S0 ; Sk the trees returned by the
S0
Sk
algorithm with `(S0) B; `(Sk ) B
e0
e1
that dier in jSk S0j :=
jSk nS0j + jS0nSk j = 2k edges
I Let e0 2 S0 be edge maximizing 0 (e) := (e) + z `(e).
There is an edge e1 2 Sk nS0 suh that S1 := S0 nfe0 g [ fe1 g
is a spanning tree. Sine 0 (S0 ) = 0 (Sk ), 0 (e0 ) 0(e1 ).
On the other hand 0 (S1) 0(S0 ) sine S0 has minimal
0 -ost. Hene 0 (S1 ) = 0 (S0 ) and jS1 S0 j = 2(k 1).
I We iterate this to obtain S0 ; : : : ; Sk with
0 (S0 ) = 0 (S1 ) = : : : = 0 (Sk ) and jSi Si+1 j = 2 8i.
I Sine `(S0 ) B; `(Sk ) B there must be a pair
(T1 ; T2 ) := (Si; Si+1 ) with `(Si) B; `(Si+1) B .
278 / 292
T
2
is not that bad
Lemma
Let z ; T1 ; T2 be opt. Lagrange solutions, `(T1 ) B; `(T2 ) B
s.t. jT1 T2 j = 2. Then (T2 ) OP T + max .
I
Reall that
0
}|
(T1 ) (T1 ) + z (`(T1 )
z
|
I
{z
0
{
B}) = OP TLR (z ) OP T
Let e1 ; e2 be edges with T2 = (T1 nfe1 g) [ fe2 g. Then
(T2 ) = (T1 ) (e1 ) + (e2 ) OP T + max
| {z } | {z } | {z }
OP T
0
max
279 / 292
A PTAS
Lemma
There is a PTAS for
Budgeted Spanning Tree
.
Guess the 1=" many edges of maximum ost in the
optimum solution.
I Contrat them. Now max " OP T in the remaining
instane.
I
State of the art:
It is not know, whether there is an FPTAS for Budgeted
Spanning Tree.
I [Hong et al.℄ an nd a tree T with
(T ) (1 + ")OP T; `(T ) (1 + ")B in poly(n; 1=") (i.e. a
biriteria FPTAS).
I
280 / 292
Part 27
-Median
k
Soure:
Approximation Algorithms (Vazirani, Springer Press)
281 / 292
k
-Median
Problem: k-Median
I Given: Failities F , ities C , parameter k 2 N . Metri ost
ij for onneting ity j to faility i.
I Find: Set of at most k failities I and an assignment
: C ! I of ities to opened failities, minimizing the
onnetion ost:
OP T
:= I F;jImin
jk;:C !I
i2I
(j );i
F
C
j
X
ij
i
282 / 292
k
-Median
Problem: k-Median
I Given: Failities F , ities C , parameter k 2 N . Metri ost
ij for onneting ity j to faility i.
I Find: Set of at most k failities I and an assignment
: C ! I of ities to opened failities, minimizing the
onnetion ost:
OP T
:= I F;jImin
jk;:C !I
X
i2I
(j );i
F
C
k
j
ij
i
283 / 292
Integer
program
:
P
P
min i2F j2C xij ij
P
x
8j 2 C
i2F ij = 1
xij yi
8i 2 F 8j 2 C
P
y
k
i
i2F
yi ; xij 2 f0; 1g 8i 2 F 8j 2 C
= OP T
Lagrangian
RelaxationP(z 0) : P
P
minP i2F j2C xij ij + z i2F yi k
8j 2 C
i2F xij = 1
xij yi
8i 2 F 8j 2 C
yi ; xij 2 f0; 1g 8i 2 F 8j 2 C
=
optimum faility loation
value for instane with fi := z
=: OP TF L(z)
=: OP TLR (z)
zk
284 / 292
Approximating the Lagrangean Relaxation (1)
Reall the previous result:
Theorem
One an ompute a Faility Loation solution in poly-time,
with
onnetion ost + 3 faility ost 3 OP TF L:
Let F L(z) F be the set of failities, opened by
approximation algorithm if fi := z for all failities i 2 F .
I For F 0 F and j 2 C let (F 0 ; j ) := mini2F 0 fij g be the
distane of ity
j to nearest faility in F 0
P
I Let (F 0 ) := j 2C (F 0 ; j ) be the onnetion ost of a
Faility Loation or k -Median solution F 0 .
I
285 / 292
Approximating the Lagrangean Relaxation (2)
jF L(z)j = # failities opened by apx-algo
jF2 j
k
jF1 j
sol F2 : infeasible/ (F2 ) 3 OP T
sol F1 : feasible/ (F1 ) 3 OP T
z
z
I jF L(0)j = jF j k; limz!1 jF L(z )j = 1 k
I By binary searh in the interval [0; jC j maxi;j fij g℄, nd
z 0, where jF L(z )j k jF L(z + ")j
I Let F1 := F L(z + "); F2 := F L(z ) be the obtained
approximate solutions (we ignore the "-term from now on,
sine it an be made exponentially small).
286 / 292
Bounding the ost of F1; F2
Lemma
Choose 0 1 with jF1 j + (1
)jF2 j = k. Then
(F1 ) + (1 ) (F2 ) 3 OP T:
Sine we use a (3; 1)-apx algo for Faility Loation:
(F1 ) + 3z jF1 j 3 OP TF L (z )
(F2 ) + 3z jF2 j 3 OP TF L (z )
I Adding both inequalities with oeÆient and 1 , resp.:
(F1 ) + (1 )(F2 ) + 3z (jF1 j + (1 )jF2 j)
|
{z
}
I
=k
I
3 OP TF L(z) = 3 OP TLR (z) + 3z k
The 3zk term anels out and
(F1 ) + (1 )(F2 ) 3 OP TLR (z ) 3 OP T
287 / 292
Combining F1 and F2 (1)
Lemma
We an randomly hoose a subset I F1 [ F2 of size jI j k of
ost E [(I )℄ 6 OP T .
We want to hoose I s.t.
E [(I; j )℄ 2 (F1 ; j ) + (1 ) (F2 ; j ) :
Then
X
X
E [(I )℄ =
E [(I; j )℄ 2 (F1 ; j ) + (1 ) (F2 ; j )
I
j 2C
j 2C
2 |( (F1 ) + (1{z ) (F2 ))} 6 OP T
3OP T
288 / 292
Combining F1 and F2 (2) F1 (i ;i ) F2
Case (1): With prob 1 :
(F ;j )
F20
+
(
F
;j
)
0
I Choose F2 F2 with jF1 j
i1
i2
jF20 j = jF1 j so that for
jF2 j
any faility i1 2 F1 , also
(F1 ; j ) (F2 ; j ) i3
j
the faility i2 2 F2
F200
0
minimizing i ;i is in F2
I Choose F200 F2 nF20 with jF200 j = k jF1 j uniformly at
random. Open I := F20 [ F200 .
I Let i1 2 F1 and i3 2 F2 be nearest failities to j . Suppose
i3 2= F20 (other ase later).
I Note that Pr[i3 2 I ℄ = jFk j jFjFj j = 1 . Hene
E [(I; j )℄ Pr[i3 2 I ℄ (i3 ; j ) +Pr[i3 2= I ℄ (i2 ; j )
| {z }
| {z } | {z } | {z }
1
3
1
2
1
2
1
2
1
=1 (F2 ;j )
=
(1 + ) (F2 ; j ) + 2 (F1 ; j )
(F2 ; j ) + 2 (F1 ; j )
2(F1 ;j )+(F2 ;j )
289 / 292
Combining F1 and F2 (2)
Case (2): With prob :
I Choose I := F1 [ F200
I Then
i3 2= I ℄ (i1 ; j )
E [(I; j )℄ Pr[i{z
3 ; j}) +Pr[
3 2 I}℄ |(i{z
| {z } | {z }
|
=1 (F2 ;j )
=
(F1 ; j ) + (1 )(F2 ; j )
F1
jF1 j
i1
(F1 ; j )
(F1 ;j )
F2
F20
(F2 ; j ) i
3
j
00
F2
jF2 j
290 / 292
Combining F1 and F2 (3)
I Overall:
:
E [(I; j )℄
(2)℄} Pr[ase
(1)℄} |E [(I; j{z) in (1)℄} +Pr[ase
|
{z
{z
|
=1 |( + 2(1
{z
2
I
2(F1 ;j )+(F2 ;j )
)) (F1 ; j ) + (1
}
=
E [(I; j ) in (2)℄
|
{z
}
(F1 ;j )+(1 )(F2 ;j )
) (1 + ) (F2 ; j )
| {z }
2
(For ase i3 2 F20 : E [(I; j )℄ (F1 ; j ) + (1
)(F2 ; j )).
291 / 292
The main result
Theorem
There is an expeted 6-approximation for k-Median in
polynomial time (whih an be easily derandomized).
State of the art:
Theorem (Arya et al.)
One an obtain a (3 + ")-apx in time O(n2=" ).
I
I
Algorithm uses loal searh.
The natural LP relaxation has an integrality gap of 3, but
no algorithm is known that ahieves this value.
292 / 292