Approximation Algorithms for
NP-hard Combinatorial Problems
Magnús M. Halldórsson
Reykjavik University
www.ru.is/~mmh/ewscs13/lec1.ppt
Post-doc & Ph.D. Positions available
• Part of research project titled
“Algorithms for Wireless Scheduling”
Why? Motivation

Many computational problems are NP-hard
Most
• Instead of looking for the best solution, we may want to
find a solution that is good enough
– Instead of an optimal solution, we seek approximations
Why study approximation algorithms?
• Study valuable, general-purpose problem
solving techniques
• Interesting combinatorics, from a CS
viewpoint
• Related to various other paradigms:
– Online algorithms
– Streaming algorithms
– Distributed algorithms
What I expect that you already know
• Discrete structures and problems
– Graphs and networks
• Basic knowledge of algorithms
• Some experience with analysis of algorithms
– E.g. why Dijkstra’s algorithm works
• Bonus:
– Mathematical programming
– Basic probability theory
Plan today
• Get comfortable with our main problems
• Overview the algorithmic strategies that we
will examine
• Look at the most naive approximation
algorithms
Fundamental problems
• There are millions of optimization problems
• We focus on a few fundamental problems:
– The techniques developed can be relatively easily
transferred to other problems
• The classic graph problems, w/o connectivity
Problems we shall consider
•
•
•
•
•
•
Independent set
Vertex cover
Chromatic number
Dominating set
Domatic number
Max Cut
• And sometimes weighted versions
Graphs and Notation
•
•
•
•
•
n=9
m=13
=4
=2
d=26/9
Independent sets
• Find: An independent set S of vertices
• Objective: Maximize |S|
Independent sets
• Find: An independent set of vertices
• Maximize: Size
Intervals
& corresponding interval
graph
Interval selection
& corresponding interval
graph
Interval selection
& corresponding interval
graph
k=2
distance-k-independent set in graph G
= independent set in the power graph Gk
(u,v)  E(Gk)  distG(u,v)  k
Matching
• Input: Graph.
• Goal. Find maximum cardinality matching.
16
A
1
B
2
C
3
D
4
E
5
Vertex Cover
• S is a vertex cover if every edge has at least
one endpoint in S
• Objective: Minimize |S|
Vertex Cover
• S is a vertex cover if every edge has at least
one endpoint in S
• Objective: Minimize |S|
Map coloring
Austria and its neighbors
?
Graph Coloring
Chromatic Number
(G) = 3
Where to locate icecream stands?
• The Icecream Stands
problem:
• Every kid should have
access to an icecream
stand in the next street.
How many stands are needed?
• A dominating set for a graph is a set of
vertices whose neighbors, along with
themselves, constitute all the vertices in the
graph.
Domatic number
• Find: A partition (coloring) of the vertices
such that each color is a dominating set
• Objective: Maximize
A
the number of colors
B
F
C
E
The MAX CUT problem
Input. A graph.
Feasible solution. A set S of vertices.
Value of solution. Number of edges cut.
Objective. Maximize.
Weighted versions
• In Weighted Independent Set:
– Given a graph G with weights on vertices
– Find an independent set with maximum total
weight
• Weights on vertices:
– Ind.set, Vertex cover, Dominating set
• Weights on edges
– Max Cut
Algorithmic strategies
•
•
•
•
•
•
•
Greedy
Subgraph removal
Local search
Probabilistic method
Local ratio
Linear programming rounding
Semi-definite programming
Core issue: Bounding OPT
• We want to compare „our“ solution, ALG, to
OPT, the optimal solution
– I.e., show that ALG   * OPT
– But OPT is hard to get a handle on (NP-hard!)
• Instead, we compare ALG to an easier
property that bounds OPT:
–  (I) OPT(I), and
– ALG(I)  (I)
Greedy
• Previously known as „myopic“
• In each round, the algorithm makes the action
that is best according to some criteria
Subgraph removal
• An independent set can contain at most 1
vertex from a clique
• If we remove all big cliques, the problem may
become easier.
• If OPT was large, it is still fairly large after
removing the cliques
Local search
• Can I make small, „local“ changes to my
solution, to get a better solution?
– Ex. 1: „Shortcuts“
– Ex. 2: If I throw out 1 vertex, can I replace it with
2?
Probabilistic method
• A „random“ solution is often pretty good
– Sometimes close to best possible
– Sometimes it needs minor changes
• Basic facts from probability:
– Product rule
– Union bound
– Linearity of expectation
LP rounding
• Linear programming is a powerful hammer
– The most general purpose solving method
• All of our problems can be written as Integer
Programming problems
– X_i = 1, if v_i is in the independent set,
– X_i = 0, if v_i is not in the set
• LP: Allow any value in the range [0,1]
– Solvable in polynomial time
• „Rounding“: Use LP-value to choose the binary
0/1 value
Semi-definite programming
• „Vector programming“ assigns each vertex a
vector in n-dimensional space
– LP value = 1-dimensional vector
• Use nearness or farness
to decide on rounding
Performance ratio
• An algorithm A is  -approximate if, for every
input instance I,
or
A(I)    OPT(I)
(maximization)
A(I)    OPT(I)
(minimization)
VC - Approximation Algorithm
• C
• E’  E
• while E’  
– do let (u,v) be an arbitrary edge of E’
– C  C  {u,v}
– remove from E’ every edge incident to
either u or v.
• return C.
Demo
Compare this cover to the
one from the example
Polynomial Time
• C
• E’  E
• while E’   do
O(n2)
O(n2)
– let (u,v) be an arbitrary edge of E’
– C  C  {u,v}
– remove from E’ every edge incident to
either u or v
• return C
O(1)
O(n)
Correctness
The set of vertices our algorithm returns is clearly a vertex-cover, since
we iterate until every edge is covered.
How Good an Approximation is it?
Observe the set of edges our algorithm chooses
no common vertices!  any VC contains 1 in each
our VC contains both, hence at most twice as large
Another algorithm
Algorithm 1.2 (Cardinality vertex cover)

Find a maximal matching in G and output the set of
matched vertices
• Given a graph G=(V,E), a subset of the edges
M  E is a matching if no two edges of M share an
endpoint.
• A matching is maximal if no more edges can be
added
• The previous solution is a maximal matching!
Linking ALG to OPT
(relating the approximation to the optimal)
Approximate
solution
Maximal
Maximal
matching
matching
?
Optimal
solution
Lower bounding OPT
• OPT : an optimal solution (vertex cover) (also its size)
• ALG : solution found by our algorithm
• M : maximal matching
• We argued a lower bound for OPT in terms of M:
OPT ≥ |M|
• Thus, maximal matching is the combinatorial structure that is
relevant here.
• Performance analysis follows from this:
– Since, ALG = 2 |M|, the performance ratio is at most 2
Can we improve this approximation?
• Can we improve the analysis for this
algorithm?
• Can we make this algorithm more clever?
– What if we carefully choose the maximal
matching?
• Can we find a better algorithm?
Tight example #1
OPT picks 1 vertex
ALG selected 2
But, this is only one small graph.
We want a family of graphs, of infinite size
Tight example #2
A matching and the
algorithm solution
An optimal solution
• Complete bipartite graph Kn,n (here n=4)



Any maximal matching contains all the 2n vertices
So the algorithm solution always contains 2n verts.
Optimal solution contains only n vertices
“This was a too simple solution”
• It is simple, but not stupid. Many seemingly smarter
heuristics can give a far worse performance, in the worst case.
• It is easy to complicate heuristics by adding more special cases
or details. For example, if you just picked the edge whose
endpoints had highest degree, it doesn’t improve the worst
case. It just makes it harder to analyze.
Naive approximations
• Bound ALG and OPT, independently
• Forms the baseline, with which we compare
Naïve approximation of
Dominating Set
• ALG  n
• OPT  n/(+1)
Naïve approximation of
Domatic Partition
• ALG  1
• OPT   +1
Naïve approximation of Coloring
• ALG  +1
• OPT  2
Naïve approximation of
Independent Set
• ALG  n/(+1)
• OPT  n
Acknowledgment
• Much of the presentations are adapted from
slides on the web
• In this talk, material on Vertex Cover was
taken from:
– www.cs.tau.ac.il/~safra/Complexity/appalgo.ppt
Problem #1: Min-degree greedy
• What is the most natural algorithm for the
Independent Set problem?
• „Min-degree greedy“:
– 1. Find a vertex of minimum degree;
2. delete its neighbors;
3. repeat (until the graph is empty)
• A) How good/bad is it on general graphs?
– Construct a bad instance for that algorithm
– As a function of n
• B) Does it do better than +1 ?.
Problem #2: Max Cut
• What (easy) approximations can we give for
Max Cut?