..
IN210 − lecture 9
non-solvable
Not acceptable
...
Undecidable
EXP TIME
PSPACE
intractable
Co
NP
NP
= complete
problems
P
Task I
well-solved
;
Create map of problems
• abstraction (
)
• techniques (diagonalisaztion/reduction)
• insights
• complete/characteristic problems
Task II
Organize/study problems
• place abstract/real-world problems on the
map
• ’walk’ the map (get to know the places &
’countries’)
Autumn 1999
1 of 13
IN210 − lecture 9
Task III
Organize/study solutions
• ’classical’ (worst-case/best solution)
approaches
• ’alternative’ approaches to algorithm
design/analysis
— approximation
— average-case
— randomized/probabilistic
• ’alternative’ machines
— parallell computers
— quantum computers
— ???
Task IV
Organize/study other issues
• logic ↔ computation
• expressive power of
— programming languages
— query languages
— logic
• cryptography
Autumn 1999
2 of 13
IN210 − lecture 9
Example problem: TSP
T RAVELING S ALES P ERSON ( TSP) is the most
studied optimization problem.
Bergen 450
Lillehammer
420
160
540
Stavanger
120
Oslo
830
Instance: Distance matrix
Oslo Bergen Stav. Lilleh.
Oslo
0
450
540 120
450
0
160 450
Bergen
540
160
0
830
Stavanger
450
830
0
Lillehammer 120
Question:
• (decision version) Given integer K, is there
a tour of length K or less?
• (search version) Given integer K, find a
tour of length K or less.
• (optimization version) Find the shortest
tour.
Autumn 1999
3 of 13
IN210 − lecture 9
Is TSP N P-equivalent?
TSP is N P-hard
Proof: H AMILTONICITY ∝TSP
a
d
b
c
a
b
c
d
∝
a
2
1
2
1
b
1
2
1
2
c
2
1
2
1
d
1
2
1
2
K = n(= 4)
TSP- DECISION is N P-complete
• TSP ∈ N P: ’guess’ tour
• TSP ∈ N P-hard
TSP- SEARCH is N P-easy
(T, K)
RT
TSPd
alg.
Y
a tour
in T or
’No’
N
TSP- SEARCH algorithm
Autumn 1999
4 of 13
IN210 − lecture 9
TSP- OPT is N P-easy
Proof:
• use binary search and TSP- DECISION
algorithm to find the optimal K:
P
0
00
— K = 1, K = i=1,...,n∧j=i,...,n dist(i, j)
—
K
K0
YES
K 00
NO
— Repeat:
00
0
If TSP-dec(T, K +K
2 ) = YES
then K 00 ← K else K 0 ← K
• use TSP- SEARCH algorithm to find the tour
of length K
⇒ TSP is N P-equivalent
Insight: Although based on formal languages
(decision problems) and TMs, the ’map’
captures/represents what is essential.
Autumn 1999
5 of 13
IN210 − lecture 9
Focus on task III: Organize
algorithms/solutions
We look at different algorithm-design
paradigms:
• Today: ’Classical’ worst-case/best solution
approaches
• Next three classes: ’Alternative’
approaches
Worst-case/best solution
approaches
P
Autumn 1999
• Branch-and-bound
• Dynamic programming
• Greedy
• Matching
• Linear programming
• Dynamic programming
6 of 13
IN210 − lecture 9
Subset systems
A subset system is a triple (E, w, τ ) where
E is a set
w is weights on the elements, w : E → Z+
τ is a system of subsets closed with respect to
inclusion (any subset of a set s in τ is also
in τ ): s ∈ τ ∧ s0 ⊆ s ⇒ s0 ∈ τ
Combinatorial problem associated with a
subset system:
Find s ∈ τ with largest sum of weights.
Example: M AX . S PANNING T REE
In M AX . S PANNING T REE we ask, given a
weighted graph G, for the subtree (subgraph
without cycles) of G that has the highest sum
of weights.
M AX . S PANNING T REE as a subset system:
E is the set of edges in G.
w is weights of the edges.
τ is the set of subtrees of G. τ is closed with
respect to inclusion, because removing an
edge from a subtree of G, we still have a
subtree of G.
Autumn 1999
7 of 13
IN210 − lecture 9
M AX . S PANNING T REE
• solved by a greedy algorithm in time
O (n log n):
7
9
8
7
9
3
6
5
2
M AX . M ATCHING
• greedy algorithms don’t work
• solved by an “iterative-improvement”
3
algorithm in time O n
3
1
4
3
TSP
• N P-complete
• best-known algorithm is exponential time
(exhaustive search)
Autumn 1999
8 of 13
IN210 − lecture 9
Generic greedy algorithm
I=∅
while E 6= ∅ do
Pick largest-weight element e
of E;
Remove e from E;
If I + e ∈ τ then I := I + e;
end while
A subset system whose combinatorial
optimization problem can be solved by a
greedy algorithm is called a matroid.
Autumn 1999
9 of 13
IN210 − lecture 9
Insights
1 SAT
1 DM
1 C OLORING
↓
1 matroid
↓
greedy
solutions
2 SAT
2 DM
2 C OLORING
↓
intersection of
2 matroids
↓
generic
matroid
intersection
algorithm
3 SAT
3 DM
3 C OLORING
↓
intersection of
3 matroids
↓
N P-complete
TSP as an intersection of 3 matroids
(directed graph G, and a TSP-path)
1st matroid: Same as for M IN S PANNING T REE
2nd matroid: Subtrees where each node has
at most 1 outgoing edge (all edges in G get
weight=1)
3rd matroid: Subtrees where each node has
at most 1 ingoing edge (all edges in G get
weight=1)
In addition the intersection tree (solution)
must have exactly n − 1 edges.
Autumn 1999
10 of 13
IN210 − lecture 9
L INEAR P ROGRAMMING (LP
In the LP we want to
• minimize ~c~x (i.e. c1x1 + c2x2 + . . . cnxn) such
that
• A~x ≥ ~r (A is a m × n integer matrix, ~r is a
m-vector of integers)
• ~x ≥ 0 (xi are real numbers)
Example: Diet problem
~x = diet
i = food type
xi = amount of food i
ci = cost of food i per unit
Aij = amount of nutrient j in food i
rj = required amount of nutrient j
• Simplex algorithm is not polynomial in
worst-case, but efficient “in practice”.
• Ellipsoid algorihm is polynomial in
worst-case, but not as fast as Simplex “in
practice”.
Autumn 1999
11 of 13
IN210 − lecture 9
I NTEGER L INEAR P ORGRAMMING
(ILP)
• minimize ~c~x such that
• A~x ≥ ~r
• ~x ≥ 0
• xi are integers
Theorem 1 ILP is N P-hard.
Proof: Reduction from TSP
• distance matrix with elements cij
• Variable xij assosiated with edge (i, j)
• xij = 1 if edge (i, j) in the tour, and 0 if not
Pn
• minimize z = i,j=1;i6=j cij xij
• 0 ≤ xij ≤ 1 and xij integer
Pn
• i=0 xij = 1, for j = 0, . . . n
Pn
• j=0 xij = 1, for i = 1, . . . n
• + a constraint that excludes tours
consisting of “subtours”
Autumn 1999
12 of 13
IN210 − lecture 9
Geometric interpretation
x3
x2
Simplex
LP:
c
x1
• It can be shown that all feasible solutions
(= points in a n-dim. space) must be inside
the convex hull.
OPT LP
ILP:
OPT ILP
Algorithm for ILP:
• solve LP-opt (use Simplex)
• find ILP-opt (it should not be too far away)
Note: This algorithm is not pol. time because
the ILP-opt point can be exponential far
away the from LP-opt point given by
Simplex/Ellipsoid (arbitrary small angle . . . ).
Autumn 1999
13 of 13