COMP 482: Design and
Analysis of Algorithms
Spring 2013
Lecture 20
Prof. Swarat Chaudhuri
Recap: Project Selection
can be positive or negative
Projects with prerequisites.
Set P of possible projects. Project v has associated revenue pv.

–
some projects generate money: create interactive e-commerce interface,
redesign web page
–


others cost money: upgrade computers, get site license
Set of prerequisites E. If (v, w)  E, can't do project v and unless
also do project w.
A subset of projects A  P is feasible if the prerequisite of every
project in A also belongs to A.
Project selection. Choose a feasible subset of projects to maximize
revenue.
2
Recap: Project Selection: Prerequisite Graph
Prerequisite graph.
Include an edge from v to w if can't do v without also doing w.
{v, w, x} is feasible subset of projects.
{v, x} is infeasible subset of projects.



w
w
v
x
feasible
v
x
infeasible
3
7.10 Image Segmentation
Image Segmentation
Image segmentation.
Central problem in image processing.
Divide image into coherent regions.


Ex: Three people standing in front of complex background scene.
Identify each person as a coherent object.
5
Image Segmentation
Foreground / background segmentation.
Label each pixel in picture as belonging to
foreground or background.
V = set of pixels, E = pairs of neighboring pixels.
ai  0 is likelihood pixel i in foreground.
bi  0 is likelihood pixel i in background.
pij  0 is separation penalty for labeling one of i
and j as foreground, and the other as background.





Goals.
Accuracy: if ai > bi in isolation, prefer to label i in foreground.
Smoothness: if many neighbors of i are labeled foreground, we
should be inclined to label i as foreground.
Find partition (A, B) that maximizes:



foreground
background
6
Image Segmentation
Properties of the problem.
Maximization.
No source or sink.
Undirected graph.



Turn into minimization problem.

Maximizing
is equivalent to minimizing

or alternatively
7
Q1
…Can you solve this problem using max flow or min-cut?
8
Image Segmentation
Formulate as min cut problem.
G' = (V', E').
Add source to correspond to foreground;
add sink to correspond to background
Use two anti-parallel edges instead of
undirected edge.


pij
pij
pij

aj
s
i
pij
j
t
bi
G'
9
Image Segmentation
Consider min cut (A, B) in G'.
A = foreground.

cap(A, B) =

å a j + å bi +
jÎB
iÎ A
å pij
if i and j on different sides,
pij counted exactly once
(i, j) Î E
iÎ A, j Î B
Precisely the quantity we want to minimize.
aj
i
s
A
pij
j
t
bi
G'
10
7.12 Baseball Elimination
Baseball Elimination
Against = rij
Team
i
Wins
wi
Losses
li
To play
ri
Atl
Phi
NY
Mon
Atlanta
83
71
8
-
1
6
1
Philly
80
79
3
1
-
0
2
New York
78
78
6
6
0
-
0
Montreal
77
82
3
1
2
0
-
Which teams have a chance of finishing the season with most wins?
Montreal eliminated since it can finish with at most 80 wins, but
Atlanta already has 83.
wi + ri < wj  team i eliminated.
Only reason sports writers appear to be aware of.
Sufficient, but not necessary!




12
Baseball Elimination
Against = rij
Team
i
Wins
wi
Losses
li
To play
ri
Atl
Phi
NY
Mon
Atlanta
83
71
8
-
1
6
1
Philly
80
79
3
1
-
0
2
New York
78
78
6
6
0
-
0
Montreal
77
82
3
1
2
0
-
Which teams have a chance of finishing the season with most wins?
Philly can win 83, but still eliminated . . .
If Atlanta loses a game, then some other team wins one.


Remark. Answer depends not just on how many games already won and
left to play, but also on whom they're against.
13
Baseball Elimination
Baseball elimination problem.
Set of teams S.
Distinguished team s  S.
Team x has won wx games already.
Teams x and y play each other rxy additional times.
Is there any outcome of the remaining games in which team s
finishes with the most (or tied for the most) wins?





14
Baseball Elimination: Max Flow Formulation
Can team 3 finish with most wins?
Assume team 3 wins all remaining games  w3 + r3 wins.
Divvy remaining games so that all teams have  w3 + r3 wins.


1-2
1
team 4 can still
win this many
more games
1-4
2
games left
1-5
s
r24 = 7
2-4


4
w3 + r3 - w4
t
2-5
5
game nodes
4-5
team nodes
15
Baseball Elimination: Max Flow Formulation
Theorem. Team 3 is not eliminated iff max flow saturates all edges
leaving source.
Integrality theorem  each remaining game between x and y added
to number of wins for team x or team y.
Capacity on (x, t) edges ensure no team wins too many games.


1-2
1
team 4 can still
win this many
more games
1-4
2
games left
1-5
s
r24 = 7
2-4


4
w3 + r3 - w4
t
2-5
5
game nodes
4-5
team nodes
16
NP-completeness and computational
intractability
18
Algorithm Design Patterns and Anti-Patterns
Algorithm design patterns.
Greed.
Divide-and-conquer.
Dynamic programming.
Duality.
Reductions.
Local search.
Randomization.




Ex.
O(n log n) interval scheduling.
O(n log n) FFT.
O(n2) edit distance.
O(n3) bipartite matching.



Algorithm design anti-patterns.
NP-completeness.
PSPACE-completeness.
Undecidability.



O(nk) algorithm unlikely.
O(nk) certification algorithm unlikely.
No algorithm possible.
19
8.1 Polynomial-Time Reductions
Classify Problems According to Computational Requirements
Q. Which problems will we be able to solve in practice?
A working definition. [Cobham 1964, Edmonds 1965, Rabin 1966]
Those with polynomial-time algorithms.
Yes
Probably no
Shortest path
Longest path
Matching
3D-matching
Min cut
Max cut
2-SAT
3-SAT
Planar 4-color
Planar 3-color
Bipartite vertex cover
Vertex cover
Primality testing
Factoring
21
Classify Problems
Desiderata. Classify problems according to those that can be solved in
polynomial-time and those that cannot.
Provably requires exponential-time.
Given a Turing machine, does it halt in at most k steps?
Given a board position in an n-by-n generalization of chess, can black
guarantee a win?


Frustrating news. Huge number of fundamental problems have defied
classification for decades.
This chapter. Show that these fundamental problems are
"computationally equivalent" and appear to be different manifestations
of one really hard problem.
22
Polynomial-Time Reduction
Desiderata'. Suppose we could solve X in polynomial-time. What else
could we solve in polynomial time?
don't confuse with reduces from
Reduction. Problem X polynomial reduces to problem Y if arbitrary
instances of problem X can be solved using:
Polynomial number of standard computational steps, plus
Polynomial number of calls to oracle that solves problem Y.


Notation. X  P Y.
computational model supplemented by special piece
of hardware that solves instances of Y in a single step
Remarks.
We pay for time to write down instances sent to black box 
instances of Y must be of polynomial size.
Note: Cook reducibility.


in contrast to Karp reductions
23
Polynomial-Time Reduction
Purpose. Classify problems according to relative difficulty.
Design algorithms. If X  P Y and Y can be solved in polynomial-time,
then X can also be solved in polynomial time.
Establish intractability. If X  P Y and X cannot be solved in
polynomial-time, then Y cannot be solved in polynomial time.
Establish equivalence. If X  P Y and Y  P X, we use notation X  P Y.
up to cost of reduction
24
Reduction By Simple Equivalence
Basic reduction strategies.
 Reduction by simple equivalence.
 Reduction from special case to general case.
 Reduction by encoding with gadgets.
Independent Set
INDEPENDENT SET: Given a graph G = (V, E) and an integer k, is there a
subset of vertices S  V such that |S|  k, and for each edge at most
one of its endpoints is in S?
Ex. Is there an independent set of size  6? Yes.
Ex. Is there an independent set of size  7? No.
independent set
26
Vertex Cover
VERTEX COVER: Given a graph G = (V, E) and an integer k, is there a
subset of vertices S  V such that |S|  k, and for each edge, at least
one of its endpoints is in S?
Ex. Is there a vertex cover of size  4? Yes.
Ex. Is there a vertex cover of size  3? No.
vertex cover
27
Vertex Cover and Independent Set
Claim. VERTEX-COVER P INDEPENDENT-SET.
Pf. We show S is an independent set iff V  S is a vertex cover.
independent set
vertex cover
28
Vertex Cover and Independent Set
Claim. VERTEX-COVER P INDEPENDENT-SET.
Pf. We show S is an independent set iff V  S is a vertex cover.





Let S be any independent set.
Consider an arbitrary edge (u, v).
S independent  u  S or v  S  u  V  S or v  V  S.
Thus, V  S covers (u, v).





Let V  S be any vertex cover.
Consider two nodes u  S and v  S.
Observe that (u, v)  E since V  S is a vertex cover.
Thus, no two nodes in S are joined by an edge  S independent set. ▪
29
Reduction from Special Case to General Case
Basic reduction strategies.
 Reduction by simple equivalence.
 Reduction from special case to general case.
 Reduction by encoding with gadgets.
Set Cover
SET COVER: Given a set U of elements, a collection S1, S2, . . . , Sm of
subsets of U, and an integer k, does there exist a collection of  k of
these sets whose union is equal to U?
Sample application.
m available pieces of software.
Set U of n capabilities that we would like our system to have.
The ith piece of software provides the set Si  U of capabilities.
Goal: achieve all n capabilities using fewest pieces of software.




Ex:
U = { 1, 2, 3, 4, 5, 6, 7 }
k=2
S1 = {3, 7}
S4 = {2, 4}
S2 = {3, 4, 5, 6}
S5 = {5}
S3 = {1}
S6 = {1, 2, 6, 7}
31
Vertex Cover Reduces to Set Cover
Claim. VERTEX-COVER  P SET-COVER.
Pf. Given a VERTEX-COVER instance G = (V, E), k, we construct a set
cover instance whose size equals the size of the vertex cover instance.
Construction.
Create SET-COVER instance:
– k = k, U = E, Sv = {e  E : e incident to v }
Set-cover of size  k iff vertex cover of size  k. ▪


VERTEX COVER
a
e7
e2
e3
e4
e6
f
k=2
SET COVER
b
c
e5
e1
e
U = { 1, 2, 3, 4, 5, 6, 7 }
k=2
Sa = {3, 7}
Sb = {2, 4}
Sc = {3, 4, 5, 6}
Sd = {5}
Se = {1}
Sf= {1, 2, 6, 7}
d
32
Q2: Hitting set
HITTING SET: Given a set U of elements, a collection S1, S2, . . . , Sm of
subsets of U, and an integer k, does there exist a subset of U of
size  k such that U overlaps with each of the sets S1, S2, . . . , Sm?
Show that SET COVER polynomial reduces to HITTING SET.
33