Approximation Algorithms
Outlines
 Why
approximation algorithm?
 Approximation ratio
 Approximation vertex cover problem
 Approximation traveling salesman
problem (TSP)
 Other interesting problems
2301681
Approximation Algorithms
2
3 approaches for NP-complete Problems
 For
small inputs, an exponential algorithm is
OK.
 Try to solve important special cases in
polynomial time.
 Find near-optimal solutions in polynomial
time (worst case or on average).
 An algorithm that returns near-optimal
solutions is called an approximation
algorithm.
2301681
Approximation Algorithms
3
Approximation Ratio
If, for any input of size n for a problem, the
cost C of the solution produced by an
algorithm is within a factor of ρ(n) of the cost
C∗ of an optimal solution (where max(C/C∗ ,
C∗/C) ≤ ρ(n)), we say that,
 the algorithm has an approximation ratio of
ρ(n) or
 the algorithm is a ρ(n)-approximation
algorithm.
2301681
Approximation Algorithms
4
On Approximation Ratio

maximization problem



0 < C ≤ C∗
C ∗ /C gives the factor by which the cost of an optimal
solution is larger than the cost of the approximate
solution.
minimization problem


0 < C∗ ≤ C
C/C ∗ gives the factor by which the cost of the
approximate solution is larger than the cost of an
optimal solution.
A 1-approximation algorithm produces an optimal
solution.
 large approximation ratio => a solution can be
much worse than optimal.

2301681
Approximation Algorithms
5
Polynomial-time approximation algorithms
may:
 have small constant approximation
ratios.
 have approximation ratios that grow as
functions of the input size n.
 achieve increasingly smaller
approximation ratios by using more and
more computation time.
2301681
Approximation Algorithms
6
Approximation scheme
 An
approximation algorithm that takes
as input both
 an
instance of the problem, and
 a value  > 0 such that for any fixed , the
scheme is a (1 + )-approximation
algorithm.
2301681
Approximation Algorithms
7
Polynomial-time Approximation Scheme
An approximation scheme is
 a polynomial-time approximation scheme

if for any fixed > 0, the scheme runs in time
polynomial in the size n of its input instance.
a
fully polynomial-time approximation
scheme
if it is an approximation scheme and its running
time is polynomial both in 1/ and in the size n of
the input instance.
 Ex: running time of O((1/)2 n3).

2301681
Approximation Algorithms
8
Approximate Vertex Cover
Vertex Cover

A vertex cover of an undirected graph G = (V, E)
is a subset V’ ⊆V such that if (u, v) is an edge of G,
then either u∈V or v∈V.

The size of a vertex cover is the number of
vertices in it.

An optimal vertex cover is a vertex cover of
minimum size in an undirected graph.

The vertex-cover problem is to find an optimal
vertex cover in a given undirected graph.
2301681
Approximation Algorithms
10
Example of Vertex Covers
A
B
C
D
F
2301681
E
A
B
C
D
E
F
Approximation Algorithms
11
Approximate Vertex Cover
AVC(G)
C←
E ← E[G]
while E  
do Choose an arbitrary edge (u,v) from E
C ← C ∪ {u, v}
E=E-{(x, y)| x=u or x=v or y=u or y=v}
return C
2301681
Approximation Algorithms
12
Find Approximted Vertex Covers (1)
C←
E ← E[G]
while E  
do Choose an arbitrary
edge (u,v) from E
C ← C ∪ {u, v}
E = E - {(x, y)|
x=u or x=v or
y=u or y=v}
return C
2301681
A
B
C
D
Approximation Algorithms
E
F
13
Find Approximted Vertex Covers (2)
C←
E ← set of edges in G
while E  
do Choose an arbitrary
edge (u,v) from E
C ← C ∪ {u, v}
E = E - {(x, y)|
x=u or x=v or
y=u or y=v}
return C
2301681
b
c
d
a
e
f
b
c
d
a
e
f
Approximation Algorithms
g
g
14
Approximation ratio of AVC (1)
AVC is a polynomial-time 2-approximation algorithm.
Proof
From the algorithm, AVC runs in O(V+E), which is
polynomial time.
Let C* be a set of optimal vertex cover.
Let C be the set of vertices returned by AVC.
C is a vertex cover because AVC loops until all
edges in the graph is removed.
Next, we will show that |C|/|C*| ≤ 2.
2301681
Approximation Algorithms
15
Approximation ratio of AVC (2)
Let A be the set of all edges chosen in the
loop.
Since all edges incident on the chosen
nodes in C are removed, no two edges in A
share an endpoint.
An edge is chosen in the loop when neither
of its endpoints are already in C.
Thus, |C|=2|A|.
2301681
Approximation Algorithms
16
Approximation ratio of AVC (3)
For any vertex v in C*, there is at least one edge in
A with v as an endpoint (because C* is a cover).
And, no two edges in A share an endpoint.
Thus, |A| ≤ |C*|.
Thus, |C|=2|A|≤ 2|C*|.
That is, |C|/|C*| ≤ 2.
Thus, AVC has approximation ratio of 2.
2301681
Approximation Algorithms
17
Traveling Salesman
Problem
Traveling salesman problem (TSP)
 Given
a complete undirected graph G=(V, E)
that has a nonnegative integer cost c(u, v)
associated with each edge (u, v)∈ E, find a
hamiltonian cycle (a tour) of G with minimum
cost.
C(A) = c(u, v)
(u,v)∈A
 TSP
2301681
is an NP-complete problem.
Approximation Algorithms
19
TSP with triangle inequality

The cost function c satisfies the triangle
inequality if for all vertices u, v,w ∈ V,


Example of cost functions that satisfy the triangle
inequality


c(u, w) ≤ c(u, v) + c(v, w) .
Euclidean distance in the plane.
Even with the triangle inequality, TSP is NPcomplete.
2301681
Approximation Algorithms
20
Approximation TSP w triangle inequality
ATSP(G, c)
Select a vertex r∈V [G] to be a “root” vertex
Compute a minimum spanning tree T for G
from root r using MST-PRIM(G, c, r).
Let L be the list of vertices visited in a
preorder tree walk of T.
Return the hamiltonian cycle H that visits
the vertices in the order L.
2301681
Approximation Algorithms
21
Graph whose cost is the
euclidean distance
2301681
Minimum spanning tree
with root a.
Approximation Algorithms
From: Cormen, Rivest, Leiserson and Stein,
Introduction to Algorithms, MIT Press, 2001.
Example: Approximate TSP (1)
22
Preorder traversal and
preorder walk
2301681
Tour obtained from
preorder walk
Approximation Algorithms
From: Cormen, Rivest, Leiserson and Stein,
Introduction to Algorithms, MIT Press, 2001.
Example: Approximate TSP (2)
23
Approximate tour
2301681
Shortest tour
Approximation Algorithms
From: Cormen, Rivest, Leiserson and Stein,
Introduction to Algorithms, MIT Press, 2001.
Example: Approximate TSP (3)
24
ATSP is 2-approximation algorithm (1)
APPROX-TSP-TOUR is a 2-approximation algorithm
for the traveling-salesman problem with the
triangle inequality.
Proof
Let H* denote an optimal tour for the given set of
vertices.
If an edge is removed from H*, the result is a
spanning tree, say Ts. Thus, c(Ts) ≤ c(H*).
Let T be a minimum spanning tree.
Then, c(T) ≤ c(Ts).
Thus, c(T) ≤ c(H*).
2301681
Approximation Algorithms
25
ATSP is 2-approximation algorithm (2)
Let W be a full walk of T.
Then, W lists the vertices when they are first
visited and also whenever they are returned to
after a visit to a subtree.
Since the full walk traverses every edge of T
exactly twice, c(W) = 2c(T ).
From c(T) ≤ c(H*) and c(W) = 2c(T ),
we get c(W) ≤ 2c(H*).
2301681
Approximation Algorithms
26
ATSP is 2-approximation algorithm (3)
However, W is not a tour, since it visits some
vertices more than once.
By the triangle inequality, we can delete a visit to
any vertex from W and the cost does not increase.
(If a vertex v is deleted from W between visits to u
and w, the resulting ordering specifies going
directly from u to w.)
By repeatedly applying this operation, we can
remove from W all but the first visit to each vertex.
2301681
Approximation Algorithms
27
ATSP is 2-approximation algorithm (4)
Let H be the cycle corresponding to this preorder
walk.
H is a hamiltonian cycle, because every vertex is
visited exactly once.
Since H is obtained by deleting vertices from the
full walk W, c(H) ≤ c(W).
c(H) ≤ 2c(H*) since c(H) ≤ c(W) & c(W) ≤ 2c(H*).
That is, APPROX-TSP-TOUR is a 2-approximation
algorithm.
2301681
Approximation Algorithms
28
MAX-3-CNF satisfiability
A randomized approximation
algorithm
MAX-3CNF-Satisfiability

Given a 3-CNF expression , satisfiability, find an
assignment of the variables that maximizes the
number of clauses evaluating to 1.

According to the definition of 3-CNF satisfiability,
we require each clause to consist of exactly three
distinct literals.

We further assume that no clause contains both a
variable and its negation.

We now show that randomly setting each variable
to 1 with probability 1/2 and to 0 with probability
1/2 is a randomized 8/7-approximation algorithm.
2301681
Approximation Algorithms
30
Randomized MAX-3-CNF-SAT is 8/7-approximate (1)
Given an instance of MAX-3-CNF satisfiability with
m clauses and variables x1, x2, . . . , xn, the
randomized algorithm that independently sets
each variable to 0/1 with prob. 1/2 is a randomized
8/7-approximation algorithm.
Proof
Suppose that we have independently set each
variable to 1 with probability 1/2 and to 0 with
probability 1/2.
2301681
Approximation Algorithms
31
Randomized MAX-3-CNF-SAT is 8/7-approximate (2)
For i = 1, 2, . . . , n, we define the indicator random
variable
Yi = I {clause i is satisfied} ,
so that Yi = 1 as long as at least one of the literals
in the ith clause has been set to 1.
Since no literal appears more than once in the
same clause, and since we have assumed that no
variable and its negation appear in the same
clause, the settings of the three literals in each
clause are independent.
2301681
Approximation Algorithms
32
Randomized MAX-3-CNF-SAT is 8/7-approximate (3)
A clause is not satisfied only if all three of its
literals are set to 0, and so Pr {clause i is not
satisfied} = (1/ 2)3 = 1/ 8.
Thus, Pr {clause i is satisfied} = 1 − 1/ 8 = 7/ 8.
Therefore, E [Yi ] = 7/ 8.
Let Y be the number of satisfied clauses overall,
so that Y = Y1 + Y2 +· · · +Ym. Then, we have
m
E [Y ] = E [ ∑ Yi ]
i=1
2301681
Approximation Algorithms
33
Randomized MAX-3-CNF-SAT is 8/7-approximate (4)
m
E [Y ]
= E [ ∑ Yi ]
i =1
m
= ∑ E [Yi ] (by linearity of expectation)
i=1
=
m
∑ 7/8
i=1
= 7m/8 .
Since m is an upper bound on the number of
satisfied clauses, the approximation ratio is at
most m/(7m/ 8) = 8/ 7.
2301681
Approximation Algorithms
34
Approximating weighted
vertex cover
Using linear programming
Minimum-weight vertex-cover problem

For any vertex cover V′ ⊆ V, we define the weight of the
vertex cover
w(V′) = ∑ w(v).
vV ′


Given an undirected graph G = (V, E) in which each vertex
vV has an associated positive weight w(v), find a vertex
cover of minimum weight.
We cannot apply



the algorithm used for unweighted vertex cover, nor
random solution.
Solution:


2301681
Use linear programming to compute a lower bound on the weight of
the minimum-weight vertex cover.
Round this solution and use it to obtain a vertex cover.
Approximation Algorithms
36
Objective/Constraint for Linear Programming
Minimize
w(V′) = ∑ w(v) x(v).
vV ′
subject to
x(u) + x(v) ≥ 1
for each v  V
(every edge must be covered)
x(v) ≤ 1 for each v  V
x(v) ≥ 0 for each v V.
(a vertex is either in or not in the cover)
2301681
Approximation Algorithms
37
AMINVC
AMINVC(G,w)
C=Ø
computex, an optimal solution to the linear
program
for each v  V
do
ifx(v) ≥ 1/2
then C = C  {v}
return C
2301681
Approximation Algorithms
38
The approx.-ratio of AMINVC is 2 (1)
AMINVC is 2-approximation algorithm for the
minimum-weight vertex-cover problem.
Proof
Let C* be an optimal solution to the minimumweight vertex-cover problem.
Let z* be the value of an optimal solution to the
linear program.
Since an optimal vertex cover is a feasible solution
to the linear program,
z* ≤ w(C*).
2301681
Approximation Algorithms
39
The approx.-ratio of AMINVC is 2 (2)
Next, we claim that by rounding the fractional
values of the variablesx(v), we produce a set C
that is a vertex cover and satisfies w(C) ≤ 2z*.
To see that C is a vertex cover, consider any edge
(u, v)  E.
Because x(u) + x(v) ≥ 1, at least one ofx(u)
andx(v) is at least 1/2.
Then, at least one of u and v will be included in the
vertex cover, and so every edge will be covered.
2301681
Approximation Algorithms
40
The approx.-ratio of AMINVC is 2 (3)
Now we consider the weight of the cover. We have
z* =
∑ w(v) ·x(v)
vV
≥ ∑ w(v)·x(v)
≥ ∑ w(v)·(1/2)
vV:x(v)≥1/2
vV:x(v)≥1/2
= ∑ w(v)·(1/2)
= (1/2)·∑ w(v)
vC
vC
= w(C)/2
Thus, w(C) ≤ 2z* ≤ 2w(C*)
AMINVC is a 2-approximation algorithm.
2301681
Approximation Algorithms
41
Subset Sum
Fully polynomial-time
approximate scheme
Exponential-time Exact Subset Sum
EXACT-SUBSET-SUM(S, t)
n ← |S|
L0 ← 0
for i ← 1 to n
do
Li ← MERGE-LISTS(Li−1, Li−1 + xi ) O(2n)
remove from Li every element a, a>t
return the largest element in Ln
Max. size of Li is 2n.
2301681
Approximation Algorithms
43
Example
Let S = {1, 4, 5} and t =8.
L0 = 0
L1 = 0, 1 ,
L2 = 0, 1, 4, 5 ,
L3 = 0, 1, 4, 5, 6, 9, 10 .
2301681
n ← |S|
L0 ← 0
for i ← 1 to n
do Li ← MERGE(Li−1,Li−1+xi )
Li+1= Li – {a|a Li, a>t}
return the largest element in
Ln
Approximation Algorithms
44
fully polynomial-time approximation scheme
Approximation by trimming each list Li after it is
created.
 If two values in L are close to each other, then one
of them can be removed.
 Let δ, 0 < δ < 1, be a trimming parameter.
 x approximates y with trimming parameter δ if y/(1
+ δ) ≤ x ≤ y .
 To trim a list L by δ


2301681
remove an element z from L if there is an element x in L′
which approximates y.
Approximation Algorithms
45
Trimming a list: Example
Let δ = 0.1 and
L = 10, 11, 12, 15, 20, 21, 22, 23, 24, 29.
To trim L, we obtain
L′ = 10, 12, 15, 20, 23, 29.
11 is represented by 10
21 and 22 are represented by 20
24 is represented by 23.
2301681
Approximation Algorithms
46
APPROX-SUBSET-SUM
APPROX-SUBSET-SUM(S, t, )
n = |S|
L0 = 0
for i = 1 to n
do Li = MERGE-LISTS(Li−1, Li−1 + xi )
Li = TRIM(Li, /2n)
Li+1= Li – {a| a Li and a>t }
Find the largest value, z*, in Ln
return z*
2301681
Approximation Algorithms
47