advertisement

NP-completeness Algorithms and Networks Today • Complexity of computational problems • Formal notion of computations • NP-completeness – Why is it relevant? – What is it exactly? – How is it proven? • P vs. NP • Some animals from the complexity zoo 2 Algorithms and Networks: NP-completeness 1 Introduction Easy problems / Hard problems • Finding the shortest simple path between vertices v and w in a given graph • Determine if there is an Euler tour in a given graph • Testing 2-colorability • Satisfiability when each clause has two literals 4 • Finding the longest simple path between vertices v and w in a given graph • Determine if there is a Hamiltonian circuit in a given graph • Testing 3-colorability • Satisfiability when each clause has three literals Algorithms and Networks: NP-completeness Fast and slow • Algorithms whose running time is polynomial in input size • Algorithms whose running time is exponential in input size • Or worse… Or in between??? 5 Algorithms and Networks: NP-completeness EXPTIME • Many problems appear not to have a polynomial time algorithm • For a few, we can proof that each algorithm needs exponential time: – EXPTIME hardness, in particular generalized games (Generalized Go, Generalized Chess) • For most, including many important and interesting problems, we cannot. 6 Algorithms and Networks: NP-completeness NP-completeness • Theory shows relations and explains behavior of many combinatorial problems • From many fields: – – – – – – 7 Logic Graphs, networks, logistics, scheduling Databases Compiler optimization Graphics … Algorithms and Networks: NP-completeness We need formalisation! • Formal notion of – – – – 8 problem instance decision problem computation running time Algorithms and Networks: NP-completeness 2 Abstract and concrete problems Different versions of problems • Decision problems – Answer is yes or no Focus on decision problems • Optimization problems – Answer is a number • Construction problems – Answer is some object (set of vertices, function, …) 10 Algorithms and Networks: NP-completeness Abstract versus concrete problems • Abstract: – Talks about graphs, logic formulas, applications, ... • Concrete: – Inputs are sets of strings in finite alphabet 11 Algorithms and Networks: NP-completeness Abstract problem instances • Computers work with bit strings • Problems are described using objects: – G is a graph, … – Given a logic formula, … – Is there a clique of size at least k, ... • We must map objects to bit strings (or another encoding like Gödel numbers...) 12 Algorithms and Networks: NP-completeness Formal problem instances 0 1 1 0 0 0 1 0 1 2 4 7 8 1 0 1 0 0 0 0 0 1 1 0 1 1 1 0 0 0 0 1 0 1 0 0 0 3 6 5 0 0 1 1 0 0 0 0 0 0 1 0 0 0 1 0 1 0 0 0 0 1 0 1 0 0 0 0 0 0 1 0 13 Algorithms and Networks: NP-completeness Abstract decision problems • Abstract decision problem: – Set of instances I • Objects like graphs, integers, logic formulas, etc. – Subset of I: instances where the answer to the problem is YES. 14 Algorithms and Networks: NP-completeness Encoding / concrete problem • Encoding: mapping of set of instances I to bitstrings in {0,1}* • Concrete (decision) problem: – Subset of {0,1}* • With encoding, an abstract decision problem maps to a concrete decision problem 15 Algorithms and Networks: NP-completeness 3 P and NP Complexity Class P FORMAL • Class of languages L, for which there exists a deterministic Turing Machine deciding whether i L, using running time O(p(|i|)) for some polynomial p 17 INFORMAL • Class of decision problems that have polynomial time algorithms solving them Algorithms and Networks: NP-completeness P • An algorithm solves a problem – Decides if string in {0,1}* belongs to subset • Time: deterministic, worst case • Algorithm uses polynomial time, if there is a polynomial p such that on inputs of length n the algorithm uses at most p(n) time. – Size of input x is denoted |x|. • P is the class of decision problems that have an algorithm that solves it in polynomial time 18 Algorithms and Networks: NP-completeness Using P for abstract problems • Abstract problem (with encoding) is in P, if the resulting problem is in P • In practice: encoding and corresponding concrete problem is assumed very implicitly • For polynomiality, encoding does not matter! – If we can transform encodings in polynomial time to each other • Details: see e.g., chapter 34 of Introduction to Algorithms 19 Algorithms and Networks: NP-completeness Language of a problem • Decision problem as a language: – Set of all yes-instances • P is the set of all languages that have a polynomial time decision algorithm 20 Algorithms and Networks: NP-completeness Verification algorithm • Verification algorithm has two arguments: – Problem input – Certificate (“solution”) • Answers “yes” or “no” • Checks if 2nd argument is certificate for first argument for studied problem • The language verified by the verification algorithm A is – {i | there is an c with A(i,c)= true} 21 Algorithms and Networks: NP-completeness Complexity Class NP Two equivalent definitions of NP • Class of languages L, for which there exists a Non-Deterministic Turing Machine deciding whether i L, using running time O(p(|i|)) • Class of languages L, for which there exists a Deterministic Turing Machine verifying whether i L, using a polynomial sized certificate c, and using running time O(p(|i|)) – 22 This definition we use: details follow Algorithms and Networks: NP-completeness NP • Problems with polynomial time verification algorithm and polynomial size certificates • Problem L belongs to the class NP, if there exists a 2-argument algorithm A, with – A runs in polynomial time – There is a constant d such that for each i, i is a yesinstance to L, if and only if there is a certificate c with • A(i,c) = true • |c| = O(|i|d) 23 Algorithms and Networks: NP-completeness Many problems are in NP • Examples: Hamiltonian Path, Maximum Independent Set, Satisfiability, Vertex Cover, … • Al of these have trivial certificates (set of vertices, truth assignment, …) • In NP (not trivial): Integer Linear Program 24 Algorithms and Networks: NP-completeness P NP • If A decides L in polynomial time, then as verification algorithm, compute – B(i,c) = A(i) – “We do not need a certificate”. • Famous open problem: P = NP ?? Or not?? 25 Algorithms and Networks: NP-completeness 4 Reducibility Reducibility • Language L1 is polynomial time reducible to language L2 (or: L1 P L2 ), if there exists a polynomial time computable function f : {0,1}* {0,1}* such that – For all x {0,1}*: • x L1 if and only if f(x) L2 27 Algorithms and Networks: NP-completeness Lemma Lemma: If L1 P L2 then if L2 P, then L1 P. Proof-idea: run an algorithm for L2 on f(i) for input i to problem L1. Also: If L1 P L2 then if L2 NP, then L1 NP. 28 Algorithms and Networks: NP-completeness 5 NP-completeness and the Cook-Levin theorem NP-completeness A language L is NP-complete, if 1. L NP 2. For every L’ NP: L’ P L A language L is NP-hard, if 1. For every L’ NP: L’ P L • 30 NP-hardness sometimes also used as term for problems that are not a decision problem, and for problems that are ‘harder than NP’ Algorithms and Networks: NP-completeness What does it mean to be NP-complete? • Evidence that it is (very probably) hard to find an algorithm that solves the problem – Always – Exact – In polynomial time 31 Algorithms and Networks: NP-completeness CNF-Satisfiability • Given: expression over Boolean variables in conjunctive normal form • Question: Is the expression satisfiable? (Can we give each variable a value true or false such that the expression becomes true). – CNF: “and” of clauses; each clause “or” of variables or negations (xi or not(xj)) 32 Algorithms and Networks: NP-completeness Cook-Levin theorem • Satisfiability is NP-complete – Most well known is Cook’s proof, using Turing machine characterization of NP. 33 Algorithms and Networks: NP-completeness Brief sketch of ideas in proof of Cook-Levin theorem • Suppose problem Q is in NP • Build a Turing Machine that – gets on an input tape a string (i, c), with c a certificate of size O(|i|d) – Runs the verification algorithm A in O(|i|q) time – d, q constants • Make a logic formula that expresses “A halts in an accepting state on (i,c)” – Variables: • a(t,r,s): true if at time t, the r’s symbol on the tape is symbol s • b(t,a): true if the machine is at time t at state a • c(t,r): true if at time t, the reading head at the tape is at position r – Lots of clauses that express that the variable act like a Turing Machine tells them to do 34 Algorithms and Networks: NP-completeness 5 Proving that problems are NP-complete Proving problems NP-complete Lemma 1. Let L’ P L and let L’ be NP-complete. Then L is NP-hard. 2. Let L’ P L and let L’ be NP-complete, and L NP. Then L is NP-complete. 36 Algorithms and Networks: NP-completeness 3-Sat • 3-Sat is the special case of CNF-Satisfiability where each clause has exactly three literals • Lemma: CNF-Satisfiability P 3-Sat – Clauses with one or two literals: • Use two extra variables p and q • Replace 2-literal clause (x or y) by (x or y or p) and (x or y or not(p)) • Similarly, replace 1-literal clause by 4 clauses – Clauses with more than three literals: • Repeat until no such clauses – For (l1 or l2 or … lr) add new variable t and take as replacement clauses (l1 or l2 or t) and (not(t) or l3 or … or lr) 37 Algorithms and Networks: NP-completeness 3-Sat is NP-complete • Membership in NP • Reduction – 3-Sat is important starting problem for many NP-completeness proofs 38 Algorithms and Networks: NP-completeness Clique • Given: graph G=(V,E), integer k • Question: does G have a clique with at least k vertices? Clique is NP-complete. In NP … easy! NP-hardness: using 3-sat. 39 Algorithms and Networks: NP-completeness Reduction for Clique Clause: {x1,not(x2),x3} • One vertex per literal x1 not(x2) x3 per clause • Edges between x1 vertices in different … clauses, except edges x2 between xi and not(xi) not(x3) • If m clauses, look for Clause: {x1,x2,not(x3)} clique of size m 40 Algorithms and Networks: NP-completeness Correctness • There is a satisfying truth assignment, if and only if there is a clique with m vertices • =>: Select from each clause the true literal. The corresponding vertices form a clique with m vertices. • <=: Set variable xi to true, if a vertex representing xi is in the clique, otherwise set it to false. This is a satisfying truth assignment: – The clique must contain one vertex from each 3 vertices representing a clause. – It cannot contain a vertex representing xi and a vertex representing not(xi). 41 Algorithms and Networks: NP-completeness Independent set • Independent set: set of vertices W V, such that for all v,w W: {v,w} E. • Independent set problem: – Given: graph G, integer k – Question: Does G have an independent set of size at least k? • Independent set is NP-complete 42 Algorithms and Networks: NP-completeness Independent set is NP-complete • In NP. • NP-hard: transform from Clique. • W is a clique in G, if and only if W is an independent set in the complement of G (there is an edge in Gc iff. there is no edge in G). 43 Algorithms and Networks: NP-completeness How do I write down this proof? • Theorem. Independent Set is NP-complete. • Proof: The problem belongs to NP: as certificates, we use sets of vertices; we can check in polynomial time for a set that it is a clique, and that its size is at least k. To show NP-hardness, we use a reduction from Clique. Let (G,k) be an input to the clique problem. Transform this to (Gc,k) with Gc the complement of G. As G has a clique with at least k vertices, if and only if Gc has an independent set with k vertices, this is a correct transformation. The transformation can clearly be carried out in polynomial time. QED 44 Algorithms and Networks: NP-completeness Writing an NP-completeness proof • State the theorem • Proof starts (or ends) with showing that problem belongs to NP. – Sometimes (e.g., in journal paper: trivial). Only if it is. – Otherwise, explain the certificates and how / that they can be checked in polynomial time • State which known NP-complete problem you transform from – Careful: do not go in the wrong direction • Explain the transformation • Give the proof: input to original problem is Yes-instance, if and only if transformed input is Yes-input for known NP-complete problem – Note: you need to proof this in two directions • Phrase: transformation can be carried out in polynomial time, hence problem is NP-complete QED 45 Algorithms and Networks: NP-completeness Vertex Cover • Set of vertices W V with for all {x,y} E: x W or y W. • Vertex Cover problem: – Given G, find vertex cover of minimum size = vertex cover 46 Algorithms and Networks: NP-completeness Vertex cover is NP-complete • In NP. • NP-hard: transform from independent set. • W is a vertex cover in G, if and only if V-W is an independent set in G. 47 Algorithms and Networks: NP-completeness 3-coloring • 3-Coloring – Given: Graph G=(V,E) – Question: Can we color the vertices with 3 colors, such that for all edges {v,w} in E, the color of v differs from the color of w 1 2 2 3 3 1 2 • 3-coloring is NPcomplete 48 Algorithms and Networks: NP-completeness Proof of NP-completeness of 3-coloring true false • In NP: … We noemen • Transformation from 3-sat: de kleuren: C C, true, false • We build a graph in 3 steps: 1. Take a clique with 3 not not vertices true, false, C x1 x 2 x1 x2 2. Take two adjacent vertices for each variable 49 Algorithms and Networks: NP-completeness Continue l1 3. For each clause {l1,l2,l3}, take the following gadget C true 50 l2 l3 Algorithms and Networks: NP-completeness Finish • One can show: • If the formula is satisfiable, then there is a 3coloring of G • If there is a 3-coloring of G, then the formula is satisfiable • In both cases, the intuition is a literal vertex is colored true, iff we take it to be true in the formula • … argue transformation takes polynomial time; QED 51 Algorithms and Networks: NP-completeness 6 Three techniques for NPcompleteness proofs 52 Algorithms and Networks: NP-completeness Techniques for proving NP-hardness 1) Local replacement 2) Restriction 3) Component Design 53 Algorithms and Networks: NP-completeness 6.1 Local replacement proofs Technique 1: Local replacement • Form an instance of our problem by – Taking an instance of a known NP-complete problem – Making some change “everywhere” – Such that we get an equivalent instance, but now of the problem we want to show NP-hard 55 Algorithms and Networks: NP-completeness Examples of Local Replacement • We saw or will see: – – – – 56 3-Satisfiability Independent Set TSP Vertex Cover Algorithms and Networks: NP-completeness Example Local Replacement • Planar 3-coloring – Given: Planar graph G=(V,E) – Question: Can we color the vertices of G with 3 colors, such that adjacent vertices have different colors? • Plan: transform from 3-coloring – – – – 57 Take arbitrary graph G Draw G on the plane So, we possibly get some crossings Replace the crossings by something clever Algorithms and Networks: NP-completeness Clever reduction • Garey et al found the following: a a d c d c b 58 b Algorithms and Networks: NP-completeness Property of the gadget • If a and b have the same color, we cannot color the gadget with 3 colors • If c and d have the same color, we cannot color the gadget with 3 colors • Otherwise, we can • ... Some additional details (basically, repeat the step) when an edge has more than one crossing … 59 Algorithms and Networks: NP-completeness 7 Restriction proofs Technique 2: Restriction • Take the problem. • Add a restriction to the set of instances. NOT to the problem definition! • Show that this is a known NP-complete problem 61 Algorithms and Networks: NP-completeness Restriction: Weighted Vertex Cover • Weighted vertex cover – Given: Graph G=(V,E), for each vertex v V, a positive integer weight w(v), integer k. – Question: Does G have a vertex cover of total weight at most k? • NP-complete – In NP. – NP-hardness: set all weights to 1 (VC). 62 Algorithms and Networks: NP-completeness Restriction: Knapsack • Knapsack – Given: Set S of items, each with integer value v and integer weight w, integers W and V. – Question: is there a subset of S of weight no more than W, with total value at least V? • NP-complete – In NP – NP-hardness: set all weights equal to their values (Subset sum) 63 Algorithms and Networks: NP-completeness 9 Component design proofs Technique 3: Component design • Build (often complicated) parts of an instance with certain properties • Glue them together in such a way that the proof works • Examples: Clique, Hamiltonian Circuit 65 Algorithms and Networks: NP-completeness Hamiltonian circuit • Given: Graph G • Question: does G have a simple cycle that contains all vertices? 66 Algorithms and Networks: NP-completeness NP-completeness of Hamiltonian Circuit • HC is in NP. • Vertex Cover P Hamiltonain Circuit: complicated proof (component design) – Widgets – Selector vertices – Given a graph G and an integer k, we construct a graph H, such that H has a HC, if and only if G has a VC of size k. 67 Algorithms and Networks: NP-completeness Widget • For each edge {u,v} [u,v,1] we have a widget Wuv [v,u,1] [u,v,6] [v,u,6] 68 Algorithms and Networks: NP-completeness Only possible ways to visit all vertices in widget 69 Algorithms and Networks: NP-completeness Selector vertices • We have k selector vertices s1, …, sk • These will represent the vertices selected for the vertex cover 70 Algorithms and Networks: NP-completeness Connecting the widgets • For each vertex v we connect the widgets of the edges {v,w}. Suppose v has neighbors x1, …, xr: add edges {[v,x1,6],[v,x2,1]}, {[v,x2,6],[v,x3,1]}, …, {[v,xr-1,6],[v,xr,1]}. 71 Algorithms and Networks: NP-completeness Connecting the selector vertices to the widgets s i • Each selector vertex is attached to the first neighbor widget of each vertex, i.e. to vertex [v,x1,1] and to the last neighbor widget [v,xr,6] Vertex in example has degree 2 72 Algorithms and Networks: NP-completeness Correctness of reduction • Lemma: G has a vertex cover of size (at most) k, if and only if H has a Hamiltonian circuit. 73 Algorithms and Networks: NP-completeness Finally • The reduction takes polynomial time. • So, we can conclude that Hamiltonian Circuit is NP-complete. 74 Algorithms and Networks: NP-completeness TSP • NP-completeness of TSP by local replacement: – In NP. – Reduction from Hamiltonian Circuit: • • • • Take city for each vertex Take cost(i,j) = 1 if {i,j} E Take cost(i,j) = 0, if {i,j} E G has HC, if and only if there is a TSP-tour of length 0. • Remark: variant with triangle inequality: use weights 2, 1 and n 75 Algorithms and Networks: NP-completeness 10 Weak and strong NP-completeness 76 Algorithms and Networks: NP-completeness Problems with numbers • Strong NP-complete: – Problem is NP-complete if numbers are given in unary • Weak NP-complete: – Problem is NP-complete if numbers are given in binary, but polynomial time solvable when numbers are given in unary 77 Algorithms and Networks: NP-completeness Examples • Subset-sum – Given: set of positive integers S, integer t. – Question: Is there a subset of S with total sum t? • Weak NP-complete. (Solvable in pseudo-polynomial time using dynamic programming: O(nt) time…) • 3-Partition – Given: set of positive integers S, (integer t). – Question: can we partition S into sets of exactly 3 elements each, such that each has the same sum (t)? • Strong NP-hard. • t must be the sum of S divided by |S|/3 = number of groups • Starting point for many reductions 78 Algorithms and Networks: NP-completeness Remark • Easily made mistake: reductions from subset sum that create exponentially large instances • Subgraph Isomorphism for degree 2 graphs – Given: Graphs G and H, such that each vertex in G and H has degree at most 2 – Question: Is G a subgraph of H? • NP-hardness proof can be done with 3-PARTITION 79 Algorithms and Networks: NP-completeness Subgraph isomorphism for forests • Input: Forest F =(V,E), forest H = (W,F) • Question: Is there an injection f: W -> V, such that for all {v,w} in F, {f(v),f(w)} is in E, i.e., is H isomorphic to a subgraph of F? • Theorem: Subgraph isomorphism for forests is NP-complete 80 Algorithms and Networks: NP-completeness Proof • In NP • NP-complete: transform from 3-partition • Suppose we have numbers a(1), …, a(3m) with total sum mt. • F is forest with m paths, each with t vertices • H is forest with 3m paths, with lengths a(1), …, a(3m) 81 Algorithms and Networks: NP-completeness 11 Some discussion How to prove NP-completeness • Is there an easy proof by reduction? • Is there a problem that “looks like our problem” – try to do local replacement • Try component design: – Good starting problems are: 3-Sat; special cases of 3-sat; 3-partition; subset sum; simple graph problems (like clique, IS, VC, Coloring) – … puzzle … 83 Algorithms and Networks: NP-completeness Discussion • Is P NP? (who thinks so?) • www.claymath.org/prizeproblems/pvsnp.htm : one of the millennium problems • Why so hard to prove? P • What to do with problems that are NPNP NP-complete? complete • Other complexity notions… 84 Algorithms and Networks: NP-completeness P vs NP is hard to prove • P = NP? Hard to design poly algorithm... • Current mathematical knowledge does not suffice to prove P != NP: – “Natural Proofs” can not separate P from NP (Razborov & Rudich, 1993) – PA = NPA, but PB != NPB for some oracles A and B, so diagonalisation can not separate P from NP (Baker, Gill, & Solovay, 1975) 85 Algorithms and Networks: NP-completeness 12 A Few Animals from The Complexity Zoo Much more classes • In Theoretical Computer Science, a large number of other complexity classes have been defined • Here, we give an informal introduction to a few of the more important ones • There is much, much, much more… 87 Algorithms and Networks: NP-completeness coNP • Complement of a class: switch “yes” and “no” • coNP: complement of problems in NP, e.g.: NOT-HAMILTONIAN – Given: Graph G – Question: Does G NOT have a Hamiltonian circuit UNSATISFIABLE – Given: Boolean formula in CNF – Question: Do all truth assignments to the variable make the formula false? 88 Algorithms and Networks: NP-completeness PSPACE • All decision problems solvable in polynomial space • Unknown: is P=PSPACE? • Savitch, 1970: PSPACE = NPSPACE – NPSPACE: solvable with non-deterministic program in polynomial time • PSPACE-complete, e.g., – generalized Tic-Tac-Toe, generalized Reversi, – Some puzzles that need an exponential number of moves – Quantified Boolean formula’s (QBF): x1x2x3x4 x1 x2 x3 x2 x3 89 Algorithms and Networks: NP-completeness EXPTIME • Decision problems that can be solved in exponential time • P is unequal EXPTIME (Stearns, Hartmanis, 1965) • EXPTIME complete problems: – Generalized chess, generalized checkers, generalized go (Japanese drawing rule) – Given a Turing Machine M and integer k, does M halt after k steps? 90 Algorithms and Networks: NP-completeness And a few more • NEXPTIME: non-deterministic exponential time • EXPSPACE = NEXPSPACE: exponential space 91 Algorithms and Networks: NP-completeness Graph Isomorphism • Discussed in another lecture • Given two graphs, are they isomorphic? • In NP, not known to be NP-complete; not known to be in P • Several problems are equaly hard: Isomorphism-complete 92 Algorithms and Networks: NP-completeness NC • NC: “Nicks class”, after Nick Pippinger • Talks about the time to solve a problem with a parallel machine • Model: we have a polynomial number of processors, that use the same memory – Variants depending on what happens when processors try to read or write the same memory location simultaneously 93 Algorithms and Networks: NP-completeness NC – the definition • NC: decision problems that can be solved with a PRAM (Parallel Random Access Machine) with polynomial number of processors in polylogarithmic time – O( (log n)d) for some constant d • Unknown: P=NC? • P-complete problems are expected not to be in NC. An example is – Linear Programming (formulated as decision problem) 94 Algorithms and Networks: NP-completeness Counting • #P: (“Sharp-P”) • Problems that outputs a number • The precise definition will not be given here. Think as: “what is the number of certificates for this instance”, with polynomial checking of certificates • #P-complete e.g.: – Number of satisfying truth assignments of 3SAT-formula – Number of Hamiltonian circuits in a graph – Number of perfect matchings in a given graph • PP is a related class (vaguely: “decide if the number of solutions is at most given number k”) 95 Algorithms and Networks: NP-completeness On PP and #P • Inference: – Given: probabilistic network, observations O, variable X, value x, value p in [0,1] – Question: Pr(X = x | O ) <= p? • Decision variant of problem from course Probabilistic Reasoning • Is PP-complete; variants are #P-complete • PP-hard and #P-hard problems are probably not polynomial... 96 Algorithms and Networks: NP-completeness LSPACE or L • Problems can be solved with only logarithmic extra space: – You can read the input as often as you want – You may use only O(log n) extra memory • E.g.: Q(1) pointers to your input • NL: non-deterministic logspace... 97 Algorithms and Networks: NP-completeness More • The Polynomial Time Hierarchy • Oracles • UP: unique solutions 98 Algorithms and Networks: NP-completeness