Approximation Algorithms
Pasi Fränti
17.10.2013
Approximation algorithms
• Optimal algorithm too slow
• Heuristic algorithm too uncertain
• Provide result with guarantee:
f OPT  f
f OPT

f  result of approximation algorithm
fOPT  result of optimalalgorithm
Bin packing problem
• Consider a set of bins (capacity 1) and
a set of items xi of size wi[0,1].
• GOAL: pack the items into as few bins as
possible
• Finding the optimal solution is a NP-hard
problem (exponential time)
Bin packing example
Greedy algorithm
Items: [0.4, 0.7, 0.5, 0.2, 0.9, 0.6]
0.9
1.0
0.4
w
i
0.9
0.9
0.7
 f OPT  f  2   wi
0.6
 f  f OPT  2   wi   wi   wi

f  f OPT
f OPT
w


w
i
i
1
Traveling Salesman Problem
• ASSUMPTION: All pairwise distances exist
and they obey triangular inequality:
w(a,c) ≤ w(a,b)+w(b,c)
• GOAL: find the shortest possible route that
visits each city once and returns to the
origin city
• Finding optimal route is slow (NP-hard)
Approximation algorithm
TSP-appro(V, E)
T  MST(V, E);
R  WALK-OF-TREE(T);
//depth-first
 A-B-C-B-H-B-A-D-E-F-E-G-E-D-A
S  SHORTCUTS(R);
// skip redundant
 A-B-C-H-D-E-F-G-A
C
A
D
B
F
H
E
G
Algorithm example
Graph with N=8 nodes
A
D
E
B
F
C
H
G
Algorithm example
Minimum spanning tree
2
A
11.9
D
2
2
B
F
2
C
5
H
2
E
2
G
Algorithm example
Tree traversal
2
A
23.8
2
2
D
2
2
2
B
2
C
F
5
2
5
A-B-C-B-H-B-A-D-E-F-E-G-E-D-A
H
2
E
2 2
2
G
Algorithm example
Shortcuts
A
19.1
D
20
2
17
2
B
A-B-C-B-H-B-A-D-E-F-E-G-E-D-A
2
F
C
5
2
H
E
2
G
Algorithm example
Final result
A
19.1
D
E
B
F
C
A-B-C-H-D-E-F-G-A
H
G
Analysis of error bound
1-approximation algorithm
Tree traversal includes all edges twice:
f  f R  2  f MST
Cutting one link from TSP is spanning tree,
but not necessarily the minimum:
f MST  fTSP
Tree traversal includes all edges twice:
 f  2  f MST  2  fTSP

fTSP  f
fTSP

fTSP  2  fTSP
fTSP
1
  1
Christofides algorithm
Christofides(V, E)
T  MST(V, E);
Find Euler tour:
D  FindOddNodes(T);
M  PerfectMatch(D);
H  T+M;
R  EulerTour(H);
S  SHORTCUTS(R);
Christofides example
Minimum spanning tree
2
A
11.9
D
2
2
B
F
2
C
5
H
2
E
2
G
Christofides example
Detect the nodes with odd degree
A
D
E
B
F
C
H
G
Christofides example
Matching the nodes
A
D
E
B
F
2
C
5
H
2
G
Christofides example
Merging MST and the matching
A
D
E
B
F
C
H
G
Christofides example
Euler tour
A
17.0
2
D
2
2
B
2
C
A-B-C-B-H-F-E-G-E-D-A
2
F
5 5
H
2
E
2
2
G
Christofides example
Shortcuts
A
15.5
2
D
2
E
2
B
F
2
C
A-B-C-B-H-F-E-G-E-D-A
2∙2
5
5
H
2
G
Christofides example
Shortcuts
A
D
E
B
F
C
H
G
Error bounds for Christofides
MST + Matching
Christofides produces TSP which is at most 50%
longer than the optimum:
f  1.5  fOPT
Proof: The algorithm combines MST with the
minimum-weight perfect matching:
f  f MST  f Match
Shortcuts can only shorten the tour.
Error bounds for Christofides
Links from matching
Let v1, v2, …, v2m be the odd nodes of T.
Skipping the rest of the nodes in TSP
gives path that consists of two matchings:
M1 ={(v1, v2), …, (v2m-1, v2m)}
M2 ={(v2, v3), …, (v2m, v1)}
Due to optimality of matching:
2  f Match  f M1  f M 2  fTSP
Odd nodes
TSP
 f Match  0.5  fTSP
Links complementing tour
Links from matching
v2
v1
v3
v2m
v2m-1
v4
…
Error bounds for Christofides
0.5-approximation algorithm
Cutting one link from TSP is spanning tree,
but not necessarily the minimum:
f MST  fTSP
Combining this with the matching:
f  f MST  f Match  fTSP  0.5  fTSP  1.5  fTSP
Thus we have:

fTSP  f
fTSP

fTSP  1.5  fTSP
fTSP
 0.5
  0.5
Empty space for notes
Pseudo polynomial knapsack
Input:
Weight of N items {w1, w2, ..., wn}
Cost of N items {c1, c2, ..., cn}
Knapsack limit s
Goal:
Select for knapsack: {x1,x2,…xn} so that
n
w c
i 1
i
i
s
n
c  x
i 1
i
i
 max!
• Problem is NP hard
• Pseudo polynomial solution exist
• Used for approximation algorithm
Pseudo polynomial knapsack
Dynamic programming algorithm
KNAPSACK (c[1,n], w[1,n], s} RETURNS x[1,n]
If same cost prune
R  {(∅,0)}
Include if
out weaker solution
item fits in
FOR i 1 TO n DO
FOR (S,c)  R DO
IF ΣjS wj+wi ≤ s THEN R  R∪{(S∪{i},c+ci)}
FOR (S,c), (S’,c)  R DO
IF ΣjS wj ≤ ΣjS’ wj THEN R  R\{(S,c)}
ELSE R  R\{(S’,c)}
(S,c)  (S,c’)R for which c’ is max!
FOR i1 TO n DO IF iS THEN x[i]1 ELSE x[i]0
RETURN x;
Pseudo polynomial knapsack
Time complexity
Sample input:
wi={1,1,2,4,12}
ci ={1,2,2,10,4}
s=15
All combinations:
∅ {1} {2} {1,2} {3} {1,3} {2,3} {1,2,3} …
1
2
3
Time complexity:
O(nc) where c = cost of optimal solution
…but c  O(2 )
n
Approximation algorithm
Scaling the input
Scale
down
input
Original input:
c1
c2
c3
 cn
Modified input:
c1
d
Solve by DP
y1
y2
y3

c2
d
c3
d

cn
d
Solve by DP
yn
wi ={1,1,2,4,12}
ci ={1,2,2,10,4}
S =15
x1
x2
x3
 xn
wi
={1,1,2,4,12}
ci/d ={1,1,1,5,2}
S
=15
Approximation algorithm
Error bound
Optimal
for original
Optimal
for scaled
∙ d/d
ci
c   ci yi   ci xi  d   xi
d
Truncate
 ci 
 ci 
 d     xi  d     yi
d 
d 
z ≥ z-1
 ci 
 d     1 yi   ci  d  yi
d

  ci yi   dyi  c  nd
=c
Σyi≤ n
xi optimal for scaled
and thus bigger
Approximation algorithm
nd/c-approximation algorithm
Result of algorithm:
f  ci xi
Optimal result:
fopt  c

f opt  f
f opt

c  c  nd 
c
nd

c
• Choosing d = ε∙c/n, we can get any ε-approximation
• Value c is not known but upper limited: c ≤ n∙cmax
• Time complexity: T n  nc d  n2   n3 
Note:
 0  1  
Empty space for notes
Acknowledgements
Contributions by Sharon Guardado and Pekka T.
Kilpeläinen to TSP example and its proof appreciated.