Approximation Algorithms
Guo QI, Chen Zhenghai, Wang Guanhua, Shen Shiqi,
Himeshi De Silva
Introduction
Shen Shiqi
Definition
Approximation Algorithm
•
Return the solutions to optimization problem
Comparison
•
Approximation algorithm: provably close to optimal
•
Heuristic: may or may not find a good solution
Background
NP Problem
•
A set of decision problems in which given a solution, you can check it in polynomial time.
NP-hard Problem
•
A problem is NP-hard if all other problems in NP can be polynomially reduced to it.
•
At least as hard as the hardest problem in NP problems
NP-complete Problem
•
A problem is NP-complete if it is in NP and NP-hard.
Approximation Ratio (approximation factor)
•
Ratio between the result obtained by the algorithm and the optimal cost
•
K-approximation algorithm
Steiner Tree Problem
Assume we are given:
• a graph
,
• edge weights
,
• a set of required nodes R,
• a set of steiner nodes S.
Assume that
. The Steiner Tree Problem is to find a subset
Steiner points and a spanning tree
of minimum weight.
The weight of Tree
is defined to be:
of the
3 variants of Steiner Tree Problem
Euclidean
•
weights refers to the Euclidean distance from u and v
Metric
•
a metric distance function d: VxV → R which satisfies the following properties:
Non-negativity: for all u, v ∈V, d(u, v)≥0
Identity: for all u∈V, d(u, u)=0
Symmetric: for all u, v ∈V, d(u, v)=d(v, u)
Triangle Inequality: for all u, v, w∈V, d(u, v)+d(v, w)≥d(u, w)
General
•
weights can be arbitrary
Steiner Tree Problem → Metric Steiner Tree Problem
Theorem:
There is an approximation factor preserving reduction from the Steiner tree
problem to the metric Steiner tree problem.
Proof:
• A instance I of the Steiner tree problem, consisting of graph G=(V,E)
• Generate a complete undirected graph G’ on vertex set V. The cost of edge (u,v)
in G’ is the smallest cost of u-v path in G
• Find the Steiner Tree T’ in G’
• Replace each edge (u,v) in T’ by corresponding path in G
• Delete edges if there are cycles ⇒ Obtain the Steiner Tree T for G
Minimum Spanning Tree
Definition of Minimum Spanning Tree
Given a graph G=(V, E) and edge weights
such that:
•
the subgraph
•
the sum of edge weights
, find a subset of the edges
is a spanning tree
is minimized.
Minimum Spanning Tree
Kruskal’s algorithm
•
create an empty set A
•
for each vertex v in V :
edge
ae
cd
ab
be
bc
ec
ed
weight
1
2
3
4
5
6
7
A={(a,e),(c,d)}
A={(a,e),(c,d),(a,b),(b,c)}
A={(a,e)}
A={(a,e),(c,d),(a,b)}
A={}
{a,e,b,c,d}
{a,e,b},
{c,d}
{a},
{b},
{c},
{d},
{a,e},
{b},
{c,d}
{c},
{d}{e}
MAKE-SET(v)
•
sort E in nondecreasing order by weight w
•
for each (u, v) taken from the sorted list :
if u and v are not in the same set :
1
a
then add (u, v) into A
4
UNION(u, v)
3
e
6
7
if all nodes are in same set:
break
b
5
c
2
d
Minimum Spanning Tree vs. Steiner Tree
c
5 km
a
2 km
5 km
b
e
5 km
D=17 km
D=16 km
d
c
3 km
a
2 km
b
4 km
4 km
3 km
d
e
MST-based Approximation Algorithm
Wang Guanhua
MST-based Approximation Algorithm
Method: Perform minimum spanning tree(MST) algorithm on Required
vertices without any steiner vertices.
Kruskal O(m*log n)
MST algorithms
Prim O(n^2)
The cost of a MST on R is within 2*OPT
Theorem: For a set of required nodes R, a set of Steiner nodes S, and a
metric distance function d. Consider an Optimal Steiner Tree(R+S, d) of cost
OPT. The cost of a minimum spanning tree of (R, d) is within 2*OPT.
Cost(MST) ≤ 2*OPT
Euler tour & Eulerian Graph
Euler tour: An Euler tour is a trail which starts and ends at the same graph
vertex. In other words, it is a graph cycle which uses each graph edge exactly
once. (Also named by Eulerian cycle, Eulerian circuit, Euler circuit, etc.)
degree = 3 (odd)
Eulerian Graph: An Eulerian graph is a graph containing an Euler tour.
Theorem: A connected graph has an Euler tour if and only if it has no graph
vertices of odd degree.
Hamiltonian cycle
Hamiltonian cycle: Is a graph cycle through a graph that visits each node
exactly once.
Proof of 2*OPT
● An example: An undirected, complete graph
1
1
d
1
h
1
1
g
f
b
1
1
1
c
a
e
Green nodes: Required vertices
Red nodes: Steiner vertices
Assume: All edges drawn have distance 1(e.g. d to f), All edges not drawn have distance 2
(e.g. a to e) .
Proof of 2*OPT
● An example: An undirected, complete graph
1
1
d
1
h
1
1
g
f
b
1
1
1
c
a
e
Green nodes: Required vertices
Red nodes: Steiner vertices
Assume: All edges drawn have distance 1(e.g. d to f), All edges not drawn have distance 2
(e.g. a to e) .
Proof of 2*OPT
● An example: An undirected, complete graph
1
1
d
1
h
1
1
g
f
b
1
1
1
a
c
2
e
Green nodes: Required vertices
Red nodes: Steiner vertices
Assume: All edges drawn have distance 1(e.g. d to f), All edges not drawn have distance 2
(e.g. a to e) .
Step 1: Optimal Steiner tree
● Optimal Steiner tree T.
1
1
1
d
1
g
f
a
h
1
1
b
1
1
1
2
1
g
c
h
1
1
a
e
d
f
1
1
c
e
OPT=Cost(T) = 8
The cost is the sum of distance of all edges: 1 + 1 + 1+ 1 + 1 + 1 + 2 = 8
b
Step 2
● Obtain an Eulerian graph by doubling edges.
a
1
a
2
1
1
g
h
e
1
d
f
c
Optimal steiner tree
OPT = 8
1
b
d
1
f
2
h
1
1
1
1
g
doubling edges
1
1 1
2
1
1
e
1
c
Eulerian graph
Cost(EG) = 16
1
1
b
Step 3
●
Find an Euler Tour by traversing edges of Eulerian graph in depth first search
(DFS) order .
a
g
d
f
h
e
c
Eulerian tour
b
Step 3
●
Find an Euler Tour by traversing edges of Eulerian graph in depth first search
(DFS) order .
a
g
d
f
h
e
c
Eulerian tour
b
Step 3
●
Find an Euler Tour by traversing edges of Eulerian graph in depth first search
(DFS) order .
a
g
d
f
h
e
c
Eulerian tour
b
Step 3
●
Find an Euler Tour by traversing edges of Eulerian graph in depth first search
(DFS) order .
a
g
d
f
Euler tour
h
e
c
Cost(ET) = 16
b
Step 4
●
Remove Steiner vertices from Euler Tour
a
1
1 1
1
g
1
1
d
1
f
a
2
2
1
1
e
1
c
1 2
2
h
h
Remove g
e
2
1
1
b
d
(a,d)
(d,f)
(f,a)
1
1
1
(a,g)-> (g,d)
(d,g)-> (g,f)
(f,g)-> (g,a)
2
f
1
c
b
Step 4
●
Remove Steiner vertices from Euler Tour
a
2
2
h
1
1
f
c
2
2
2
1
1
2
d
a
2
e
2
Remove h
2
2
1
b
d
2
f
Cost = 15
(a,h)-> (h,c)
(c,h)-> (h,b)
(b,h)-> (h,a)
(a,c)
(c,b)
(b,a)
e
c
b
Step 5
● Obtain a Hamiltonian cycle by removing the duplicates.
a
2
2
e
e
Remove duplicates
2
2
1
f
2
2
2
2
d
a
2
c
2
1
b
d
a -> d -> f -> a -> c -> b -> a -> e -> a
“Short-cutting”
(f,a)-> (a,c)
(b,a)-> (a,e)
f
2
c
2
a -> d -> f -> c -> b -> e -> a
(f,c)
(b,e)
Hamiltonian cycle
b
Step 5
● Obtain a Hamiltonian cycle by removing the duplicates.
a
a
Because
of triangle inequality, the shortcuts
2
do
2 not increase the cost.
2
f
e
e
Remove duplicates
2
2
1
d
2
2
2
2
c
2
1
b
d
a -> d -> f -> a -> c -> b -> a -> e -> a
“Short-cutting”
(f,a)-> (a,c)
(b,a)-> (a,e)
f
2
c
2
a -> d -> f -> c -> b -> e -> a
(f,c)
(b,e)
Hamiltonian cycle
b
Step 5
● Obtain a Hamiltonian cycle by removing the duplicates.
a
2
2
2
2
2
2
e
e
Remove duplicates
2
2
1
d
a
2
f
c
2
1
b
d
a -> d -> f -> a -> c -> b -> a -> e -> a
“Short-cutting”
(f,a)-> (a,c)
(b,a)-> (a,e)
Cost = 15
f
2
c
2
a -> d -> f -> c -> b -> e -> a
(f,c)
(b,e)
Hamiltonian cycle
Cost (HC)= 11
b
Step 6
● Step 4: Remove any one arbitrary of the highest cost edge . It becomes to a
spanning tree ST.
a
a
2
2
2
e
e
Remove (a,e)
2
1
d
f
2
c
Hamiltonian cycle
Cost(HC) = 11
2
2
1
b
d
f
2
c
Spanning Tree
Cost (ST)= 9
2
b
Step 5
● Step 4: Remove any one arbitrary of the highest cost edge . It becomes to a
spanning tree ST.
a
a
2
2
2
e
e
Remove (a,e)
2
1
d
f
2
c
2
2
1
b
d
f
2
c
Spanning Tree
Hamiltonian cycle
Cost(HC) = 11
Cost (ST)= 9
Cost(MST)
≤ Cost (ST)
2
b
Conclusion
● Conclusion: The minimum spanning tree has cost at most twice the cost of
the optimal spanning tree T.
Minimum spanning tree
Cost(MST)
≤
Spanning tree
cost(ST)
Hamiltonian cycle
≤
cost(HC)
Cost(MST) ≤ 2*OPT
2*Optimal steiner tree
≤
2*OPT
Metric Travelling Salesman
Problem
Himeshi De Silva
The Travelling Salesman Problem (TSP)
Given a complete graph G=(V, E) with non-negative edge
costs, find a minimum cost tour visiting each vertex exactly
once
(A complete graph is a simple (no loops/multiple edges) undirected graph where every pair of distinct
vertices is connected by a unique edge)
(A tour is a path which visits all vertices of a graph and returns to its starting vertex)
❏ NP-Hard
TSP example
2
6
3
1
5
➩
4
Thick edges = 1, thin edges= 2
Cost of solution = 6
❏ No constant factor approximation algorithm for TSP in general
Metric TSP
❏ Metric TSP - a relaxed version of the problem allows for approximation
❏ Assumption: The edges of the graph satisfy the triangle inequality
2
Cost(6,4) ≤ Cost(6,5) + Cost(5,4)
6
3
1
Still NP-complete, but no
longer harder to approximate!
5
4
Factor 2 algorithm
Example: Step 1
1. Find an MST, T, of G.
2
2
6
3
1
5
4
Thick edges = 1, thin edges= 2
➩
6
3
1
5
4
Example: Step 2
2. Double every edge in the MST to obtain an Eulerian graph
2
6
3
1
5
4
Example: Step 3
3. Find an Eulerian tour, 𝛵, on the graph
2
1→2→1→4→1→3→1→5→1→6→1
6
3
1
5
4
Example: Step 4
4. Output the tour that visits vertices of the graph in the order of their first
appearance in 𝛵. Let C be this tour.
2
T: 1→2→1→4→1→3→1→5→1→6→1
6
3
1
5
⇓
C: 1→2→4→3→5→6→1
4
“Short-cutting”
Result
C: 1→2→4→3→5→6→1
2
6
3
1
Cost of the tour = 1 + 2 + 1 + 2 + 1 + 1 = 8
5
4
Factor 2 Analysis
Proof:
(important observation) Delete any edge from an optimal solution
to TSP forms a spanning tree.
Cost(T) ≤ OPT
(OPT - cost of optimal TSP solution)
Factor 2 Analysis
Proof:
Eulerian tour contains every edge of MST twice:
Cost(𝒯) = 2*Cost(T)
Factor 2 Analysis
Proof:
Eulerian tour contains every edge of MST twice:
Cost(𝒯) = 2*Cost(T)
“Short-cutting” steps obey the triangular inequality:
Cost(C) ≤ Cost(𝒯)
Factor 2 Analysis
Combining all,
Cost(T) ≤ OPT
Cost(𝒯) = 2*Cost(T) ≤ 2*OPT
Cost(C) ≤ Cost(𝒯)
Cost(C) ≤ 2*OPT
Improving the factor to 3/2 - Christofides’ Algorithm
Chen Zhenghai
Improving the factor
Double an MST -> Eulerian graph -> Eulerian tour -> Short-cutting
Cheaper way to get Eulerian graph?
A graph has an Euler tour iff all its vertices has even degrees
Only need to be concerned about the odd degree vertices
Minimum weight perfect matching
Perfect matching: Every vertex of the graph is incident to exactly
one edge of the matching.
Input Graph
Minimum matching 1
Minimum matching 2
Handshaking Lemma (Euler)
The size of V ′ (odd degree vertices set) is even in undirected
graph.
Proof. Every edge contributes 2 degree. So the total degree is
equal to 2|E|, which is even. Thus the number of odd degree
vertices is also even.
Metric TSP —Christofides’ Algorithm
1.Find a MST, T, of G.
2.Let V′ be the vertices of odd degree in T.
3.Compute a minimum weight perfect matching M on V ′ .
4.Let G′ = T ∪ M. The graph G′ is Eulerian
5.Find an Eulerian tour, 𝒯, in G′
6.Output the tour C that visits the vertices in the order of their first
appearance in 𝒯
Tight example for Christofides’ algorithm
Input:
complete graph whose edge
weights obey the triangle
inequality
Tight example for Christofides’ algorithm
1.Find a MST, T, of G.
Tight example for Christofides’ algorithm
1.Find a MST, T, of G.
2.Let V′ be the vertices of odd
degree in T.
Tight example for Christofides’ algorithm
1.Find a MST, T, of G.
2.Let V′ be the vertices of odd
degree in T.
3.Compute a minimum weight
perfect matching M on V ′ .
Tight example for Christofides’ algorithm
1.Find a MST, T, of G.
2.Let V′ be the vertices of odd
degree in T.
3.Compute a minimum weight
perfect matching M on V ′ .
Tight example for Christofides’ algorithm
1.Find a MST, T, of G.
2.Let V′ be the vertices of odd
degree in T.
3.Compute a minimum weight
perfect matching M on V ′ .
4.Let G′ = T ∪ M. The graph G′ is
Eulerian
Tight example for Christofides’ algorithm
5.Find an Eulerian tour, 𝒯, in G′
D -> E -> A -> B -> C -> A -> D
Tight example for Christofides’ algorithm
5.Find an Eulerian tour, π, in G′
6.Output the tour C that visits the
vertices in the order of their first
appearance in π
D -> E -> A -> B -> C -> A -> D
----------------------------------------D -> E -> A -> B -> C -> D
Cont’d Theoretical Analysis and Applications
Guo Qi
Metric TSP - Factor 3/2
Lemma Let V’ ⊆ V, such that |V’| is even, and let M be a minimum
cost perfect matching on V’. Then, Cost(M) ≤ OPT/2.
Proof: (Similar technique: “short-cutting”)
Consider an optimal TSP tour of G, let
𝜏’ be the tour on V’ obtained by
short-cutting TSP tour. By triangle
inequality: Cost(𝜏’) ≤ OPT.
Metric TSP - Factor 3/2
Note 𝜏’ is the union of two
perfect matchings on V’. At
least one of the matchings has
cost ≤ OPT/2. Thus, Cost(M) ≤
OPT/2.
Metric TSP - Factor 3/2
Proof:
MST lower bound:
Cost(T) ≤ OPT.
Perfect Matching lower bound:
Cost(M) ≤ OPT/2.
Hence,
Cost(𝒯) = Cost(T) + Cost(M) ≤ 3/2* OPT.
Last, after short-cutting, Cost(C) ≤ Cost(𝒯) ≤ 3/2* OPT.
Applications - Productions of PCBs
● Problems: Logical Design, Physical
Design Correctness, Placement of
Components, Drilling, …
● Example: 442 holes to drill
Printed Circuit Board (PCB)
Applications - Productions of PCBs
Correct modelling of a printed
circuit board drilling problem:
Optimize length of a move of
the drilling head
http://www.math.uwaterloo.c
a/tsp/vlsi/index.html
Applications - Productions of PCBs
Significant Improvements via TSP (Padberg & Rinaldi)
Before
After
Applications - Productions of PCBs
Simens-Problem PCB da1
Applications - Tour Route Design
http://gebweb.net/optimap/
https://itunes.apple.com/us/app/concor
de-tsp/id498366515?mt=8
Appendix
Lower bound on TSP solution
❏ The lower bound is the cost of an MST in the graph
❏ Deleting any edge from an optimal solution to TSP gives us a spanning
tree of the graph
i.e. cost of MST ≤ cost of TSP solution
If approximation ratio is 2,
cost of solution ≤ 2*cost of MST ≤ 2*cost of TSP solution
Approximation of TSP
For any polynomial time computable function ɑ(n), TSP cannot be approximated
within a factor ɑ(n) of unless P = NP
Proof As contradiction assume that there exists a factor ɑ(n) polynomial time approximation
algorithm. We show that such an algorithm can be used to solve the Hamiltonian cycle
problem (Given a graph decide whether a Hamiltonian cycle exists)
Hamiltonian cycle - a cycle in an undirected or directed graph that visits each vertex exactly
once
The Hamiltonian cycle problem H, can be transformed into the TST G as follows.
Approximation of TSP contd.
If H has no tour, then any tour T of G has cost cost(T) > ɑ(n).n. This includes the tour
found by the approximation algorithm.
If H has a tour, then G has a tour T* with cost(T*) = n. Since the algorithm is a ɑ(n)approximation algorithm, it produces a tour T with cost(T) ≤ ɑ(n).n Clearly T is also
a tour in H, since it can not traverse any edge with cost ɑ(n).n in G.
Therefore, the algorithm is a polynomial time algorithm which can be used to
decide the Hamilton Cycle problem contradicting P ≠ NP.
Time complexity of Factor 2 algorithm
Complexity Analysis:
m: number of edges;
n: number of nodes;
Kruskal’s Algo
O(m*log n)
Constant
O(m) DFS
O(m)
Metric TSP - Factor 3/2
Complexity Analysis:
m: number of edges;
n: number of nodes;
n’: number of nodes in V’;
Kruskal’s Algo
O(m*log n)
O(n’^4) Blossom Algo
O(m) DFS
O(m)