Introduction - Optimization and Approximation
Introduction:
We say that an algorithm is efficient if it runs in polynomial time.
Denote P – the set of problems for which exist an efficient algorithm.
Denote NP - the set of problems for which deciding if a suggested solution is a correct
solution can be done efficiently.
Examples:
1. Vertex Cover:
Let G  G (V , E ) be a graph, U  V is called a vertex cover if
e  E , e  (u, v) : u U or v  U
Given a graph G and U  V checking if U is a vertex cover can be done efficiently
(by checking if the definition holds).
2. Bin Packing:
Given a set of items U 
u , u ,..., u  and a size s(u) 
1
2
n
for each u U and a
U ,U ,...U is a partition of U such
that the sum of the sizes of the item in each U is B or less - can be done efficiently.
positive integer bin capacity B , checking if
1
2
k
i
3. Set Cover:
Given a Collection C of subsets of a finite set S, the cost of each
C - checking if
i
C '  C is a set cover
(i.e. a subset of C such that every element in S belongs to at least one member of C')
can be done efficiently.
Unsolved problem in computer science: P  NP ?
The Need for Approximations
As we stated, P  NP is an open problem.
Moreover, there are thousands of problems in NP that we don't know if they can be solved
in polynomial time. Some of these problems are very common to come by and we want to
be able to find a solution for them.
Problem: it is unknown if these problems can even by solved efficiently. How can we find a
solution in reasonable time?
Answer: instead of finding an exact solution we will find a good approximation, finding an
approximation for problems in NP can sometimes be done in polynomial time – even if it's
unknown if the problem itself is in P.
NPO problems
An NP optimization problem A is a fourtuple (I,sol,m,goal) such that :
1.
I is the set of the instances of A and it is recognizable in polynomial time.
2.
Given an instance x of I, sol(x) denotes the set of feasible solutions of x. These
solutions are short, that is, a polynomial p exists such that, for any y such that
y  sol ( x ) :| y || p (| x |) .
Moreover, it is decidable in polynomial time whether, for any x and for any y
such that y || p (| x |) : y  sol ( x ) .
3.
Given an instance x and a feasible solution y of x, m(x,y) denotes the positive
integer measure of y. The function m is computable in polynomial time and is
also called the objective function.
4.
goal {min, max} .
Examples:
We will now explain how problems (1) - (3) from the introduction are NP optimization
problems:
1. Vertex cover:
1. I  {G | G  G (V , E ) is a graph}
2. sol (G)  {U  V | G  G(V , E ), U is a vertex cover}
3. m(G, U ) | U |
4. goal  min
2. Bin Packing:
1. I = All Finite set U of items, a size s(u) 
for each u U
and a positive integer bin capacity B
2. sol ((U , s, B))  A partition of U into disjoint sets
U ,U ,...U
1
2
k
such that the sum of the items in each U i is B or less
3. m((U,s,B),sol_for_(U,s,B)) = The number of used bins
in sol, i.e., the number of disjoint sets, k .
4. goal  min
3. Set Cover:
1. I  All Collection C of subsets of a finite set S
2. sol ((C , S , Cost _ func)) =
A set cover for S i.e., a subset C'  C
such that every element in S belongs
to at least one member of C '
3. m((C , S , Cost _ func), C ') 
 Cost _ func( x)
xc '
4. goal  min
Approximation algorithm
Denote OPT ( x) an optimal solution for x i.e. y  sol ( x) : m( x, y ) reaches the goal .
Let A be a minimization problem (maximization problem), an algorithm is said to be  factor
approximation algorithm for A if on each instance x of I, the algorithm produces a solution s
for x, such that OPT ( x)  m( x, s )   * OPT ( x) (
OPT ( x)

 m( x, s )  OPT ( x) )
Set Cover:
An approximation algorithm for vertex covers with   2 :
We will start with a reminder:
Let G  G (V , E ) be a graph, a matching is M  E without common vertices.
A matching M is said to be a maximal matching if for any edge not in M is added to M, it is
no longer a matching.
An efficient algorithm for finding a maximal matching:

M 

For every e  E :

If e doesn't have common vertices with the edges in M than
M 
M

{e}
Output M.
It holds that:
1. The algorithm is polynomial
2. Invariant – M is always a matching
3. e  E \ M : M
{e} is not a matching.
Hence, this is an efficient algorithm for finding a maximal matching.
Now that we know how to find a maximal matching, a surprisingly simple factor-2
approximation algorithm is the following:

Find a maximal matching M.

Output B: the set of vertices which is the collection of vertices of the edges in M.
It holds that:
1. The algorithm is polynomial (since the algorithm for finding a maximal matching is
polynomial).
2. B is a vertex cover: let e  (u , v)  E , assume that u  B and v  B - then M
{e}
is a matching in contradiction to property (3) of the algorithm for finding a maximal
matching.
3. B is not 'too big' (formally: | B | 2 | OPT (G ) | ) – denote B * a minimal vertex cover,
B * cover all the edges in E , especially it cover all the edges in M.
Since any vertex cover B ' (and especially B * ) has to pick at least one of each edge
in M, and since all the edges in M doesn't have common vertices than | M || B ' | .
As stated above, this is true for any vertex cover B ' and thus true for B * .
Hence: | B | 2 | M | 2 | B* | 2 | OPT (G) | .
Bin Packing
An approximation algorithm for Minimum Bin packing with   2 :
u u ,...u
For 1,
2
:
n

If there is a bin with that

Otherwise, add a new bin and insert
u
i
u
can be inserted into – insert
u
i
i
into that bin.
into the new bin.
It holds that:
1. The algorithm is polynomial.
2. The algorithm outputs a correct solution.
3. Analysis :
Claim: In all iterations – there is at most one bin that is less than half full.
Proof by induction:
When we add the first bin the claim is clear.
Assume that the claim is true for the first i iterations and consider iteration i+1:

If
B
s(u i )  2
then if
u
i
is inserted into an existing bin the claim clearly
holds.
Otherwise,
u
i
was inserted to a new bin and since
B
s(u )  2
i
and the
claim still holds.

If
B
s(u )  2 then :
i

If there was a bin that

holds.
Otherwise, there were exactly 0 bins that were less than half full
(since
u
i
can be inserted into the claim clearly
B
s(u )  2 , if there was a bin that was less than half full we
i
could insert
u
i
into that bin).
The algorithm adds a new bin and now there is at most 1 bin that is
less than half full.
Let k be number of bins the algorithm found, and let k ' be the minimal number of
bins needed.
Since k' is minimal it holds that: k '  k .
n
 s(u )  k ' B .
It also holds that:
i 0
i
Since the algorithm is correct, and since there is at most one bins that is less than
n
half full:
B
 s(u )  (k  1) 2
i 0
i
thus (k  1)
B n
  s(u )  k ' B hence
i
2 i 0
B
k 1
 k 'B 
 k '  k  1  2k '  k  2k '
2
2
k
k'
2
(k  1)
Set Cover
We will start with a key definition:
Let D  S , the cost-effectives of a set C' is the average cost at which it covers new elements
i.e.
Cost (C ')
|C'
D|
An approximation algorithm for Minimum Set Cover with  
H
n
n
where
1
:
k 1 k
Hn

1. D 
2. While D  S
2.1. Find the most cost effective set, say C'
Cost (C ')
2.2. Let  
|C'
, i.e the cost effective of C', Pick C' and for each
D|
e  C '\ D, price(e) 

3. Output the picked sets
Correctness:
The algorithm clearly produces a correct solution in is with polynomial time complexity.
We will give an analysis that determines  :
Number the elements of S in the order of which they were covered (resolving ties arbitrary).
Let
e , e ,..., e
1
2
n
be the numbering.
Lemma: for each k  {1,..., n} : price(
OPT
e )  n  k 1
k
Proof:
The algorithm is assigning a price price(e) to each element of S where that price is the cost
of the set C' used to cover e divided by the number of elements newly covered by the set C'.
There exists an optimal solution which chooses a set of sets with total cost OPT.
As that solution covers all elements, it certainly covers whatever elements have not yet been
covered.
Covering the rest of the elements at cost OPT would mean covering each element at costeffectiveness
OPT
.
|D|
Suppose we pick a set from the optimal solution which covers at least one remaining
element. Note that we are choosing the set with the best cost-effectiveness, so its costeffectiveness must be at least as good as the average cost-effectiveness of the sets in the
optimal solution of
OPT
.
|D|
as at least k-1 elements have already been covered. | D | n  (k  1)  n  k  1
e )
Thus: price(
k
OPT
OPT

| D | n  k 1
Using the lemma we get:
n
n
n
n
OPT
1
1
1 1
1
 OPT 
 OPT (  ...   )  OPT 
n
2 1
k 1 n  k  1
k 1 n  k  1
k 1 k
 price(e k )  
k 1
Approximation algorithm
APX - NPO problem A belongs to the class APX if it is approximable within  for some
constant   1
Examples: Vertex Cover, Bin Packing, Set Cover
Approximation scheme – Let A be an NPO problem. An algorithm T is said to be an
approximation scheme for A if, for any instance x of A and for any rational   0 : T ( x,  )
returns a feasible solution of x whose performance ratio is at most 1  
PTAS - An NPO problem A belongs to the class PTAS if it admits a Polynomial-Time
Approximation Scheme
Note: PTAS algorithms are polynomial in instance size - Includes approximation scheme with
1
1


time complexity of the type 2 P (| x |) or |x|
(where P is a polynomial)
FPTAS - An NPO problem A belongs to the class FPTAS if it admits a Fully Polynomial-Time
Approximation Scheme. That is, an approximation scheme whose time complexity is
polynomial in both |x| and
1

(where |x| is the instance size).