CSC 413/513: Intro to
Algorithms
NP Completeness
Problems and Instances
● Problem: Sort(A)
● Problem instance: Sort([4,2,7,1,6,9])
● Input encoding
■ Bin(4) concat Bin(2) concat Bin(7) …
■ Size: n lg x
○ n: size of A
○ lg x: max. number of bits required to represent A[i], for
any i
○ We say size = ϴ(n), or simply n
NP-Completeness
● Some problems are intractable:
as the input size increases, we are unable to
solve them in reasonable time
● What constitutes reasonable time? Standard
working definition: polynomial time
■ On an input of size n the worst-case running time
is O(nk) for some constant k
■ Polynomial time: O(n2), O(n3), O(1), O(n lg n)
■ Not in polynomial time: O(2n), O(nn), O(n!)
Polynomial-Time Problems
● Are some problems solvable in polynomial
time?
■ Of course: every algorithm we’ve studied provides
polynomial-time solution to some problem
○ Except DP algorithm to 0-1 knapsack
■ We define P to be the class of problems solvable in
polynomial time
○ That is, all instances of such problems are polynomially
solvable (e.g., sorting)
■ These are the “easy” problems
Polynomial-Time Problems
● Are all problems solvable in polynomial time?
■ No: Turing’s “Halting Problem” is not solvable by any
computer, no matter how much time is given
○ But we often detect infinite loops in a program by
inspection, and hence “solve” the halting problem
○ However, there are instances that are not solvable, even by
our brains
■ Such problems are clearly intractable, not in P
■ Similarly, there are problems that can be solved, but
we haven’t found algorithms that will take any
instance of the problem and solve it in polynomial
time
NP-Complete Problems
● The NP-Complete problems are an interesting
class of problems whose status is unknown
■ No polynomial-time algorithm has been
discovered for an NP-Complete problem, for all
instances of it
■ No suprapolynomial lower bound has been proved
for some instance of any NP-Complete problem,
either
● We call this the P = NP question
■ The biggest open problem in CS
An NP-Complete Problem:
Hamiltonian Cycles
● An example of an NP-Complete problem:
■ A hamiltonian cycle of an undirected graph is a
simple cycle that contains every vertex
■ The hamiltonian-cycle problem: given a graph G,
does it have a hamiltonian cycle?
Describe a naïve algorithm
for solving the hamiltoniancycle problem. Running
time?
P and NP
● As mentioned, P is set of problems that can be
solved in polynomial time
● NP (nondeterministic polynomial time) is the
set of problems that can be solved in
polynomial time by a nondeterministic
computer
■ What the hell is that?
Nondeterminism
● Think of a non-deterministic computer as a computer
that magically “guesses” a solution, and then has to
verify that it is correct
■ If a solution exists, n.d. computer always guesses it
■ One way to imagine it: a parallel computer that can freely
spawn an infinite number of processes
○ Have one processor work on each possible solution
○ All processors attempt to verify that their solution works, in parallel
○ If a processor finds it has a working solution
■ So: NP = problems verifiable in polynomial time
■ Note: P  NP (think of verifying the output of sorting)
P and NP
● Summary so far:
■ P = problems that can be solved in polynomial time
■ NP = problems for which a solution can be verified in
polynomial time
■ P  NP
■ Unknown whether P = NP (most suspect not)
● Hamiltonian-cycle problem is in NP:
■ Don’t know how to solve in polynomial time
■ But, easy to verify solution in polynomial time (How?)
Intuition behind NP-Complete
problems
● Many problems exhibit structure that is
exploited
■ E.g., recursive structure in sorting, optimal
substructure in problems solved by DP, greedy etc.
● But, some problems seem to have no
exploitable structure
Does not
have optimal
substructure
■ E.g., longest simple path in a graph
○ Contrast with shortest paths in graphs, which has the
optimal substructure
■ Typically, these are the NP-Complete problems
What about 0-1 Knapsack?
● Has optimal substructure
■ But input contains a number (W), and the optimal
substructure depends on the magnitude of this
number, not its size
■ Algorithm turns out to be exponential in the size of
W (although polynomial in the magnitude of W)
■ Its DP Algorithm is called pseudo-polynomial
● 0-1 knapsack is called weakly NP-complete
NP-Complete Problems
● We will see that NP-Complete problems are
the “hardest” problems in NP:
■ If any one NP-Complete problem can be solved in
polynomial time…
○ …then every NP-Complete problem can be solved in
polynomial time…
○ …and in fact every problem in NP can be solved in
polynomial time (which would show P = NP)
■ Thus: solve hamiltonian-cycle in O(n100) time,
you’ve proved that P = NP. Retire rich & famous.
A Simplification
● Henceforth, we will only talk about decision
problems, with boolean answers
■ Most problems of interest can be posed as decision
problems
■ E.g., SHORTEST_PATH(G,u,v)
■ Its decision problem is PATH(G,u,v,k): is there a path
between u and v involving no more than k edges?
■ Can solve SHORTEST_PATH by calling PATH
repeatedly with decreasing k, until the answer is “No”
● So, “solve” ≡ “decide”
How to Verify?
● How do you verify the answer to a decision problem
(in polynomial time)?
● Need a certificate
■ E.g., certificate for PATH(G,u,v,k) would be the path:
<u,v1,v2,…,v>
● How do you verify the certificate?
■ Check length ≤ k
■ Check first vertex is u, last vertex is v
■ Check all path-edges actually exist
● How much time?
Why only Decision Problems?
● So that problems can be considered as
languages
■ Then, “algorithm” ≡ Turing machine
■ Consider the binary encoding of a problem instance (i.e., the
input)
■ Collect all input binary strings for which the answer/output is
“Yes”
■ This set is a language
○ A machine that decides the language also solves the
problem
● So, “decision problem” ≡ “language”
Reduction
● The crux of NP-Completeness is reducibility
■ Informally, a problem P can be reduced to another
problem Q if any instance of P can be “easily
rephrased” as an instance of Q, the solution to which
provides a solution to the instance of P
○ What do you suppose “easily” means?
○ This rephrasing is called transformation
■ Intuitively: If P reduces to Q, P is “no harder to
solve” than Q
Polynomial time
Polynomial Reduction
• Polynomial reduction can be used for polynomial time solution
• If A polynomially reduces to B, and we know a polynomial
algorithm to B, then we can solve A in polynomial time as well.
Reducibility
● An example:
■ A: Given a set of Booleans, is at least one TRUE?
■ B: Given a set of integers, is their sum > 0?
■ Transformation: Given (x1, x2, …, xn) construct (y1,
y2, …, yn) where yi = 1 if xi = TRUE, yi = 0 if xi =
FALSE
● Another example:
■ Solving linear equations is reducible to solving
quadratic equations
○ How can we easily use a quadratic-equation solver to
solve linear equations?
NP-Hard and NP-Complete
● If P is polynomial-time reducible to Q, we
denote this P p Q
● Definition of NP-Hard and NP-Complete:
■ If all problems R  NP are reducible to P, then P
is NP-Hard
■ We say P is NP-Complete if P is NP-Hard
and P  NP
● If P p Q and P is NP-Complete, Q is also
NP- Complete
Why Prove NP-Completeness?
● Though nobody has proven that P != NP, if
you prove a problem NP-Complete, most
people accept that it is probably intractable
● Therefore it can be important to prove that a
problem is NP-Complete
■ Don’t need to come up with an efficient algorithm
■ Can instead work on approximation algorithms
Proving NP-Completeness
● What steps do we have to take to prove a
problem P is NP-Complete?
■ Pick a known NP-Complete problem Q
■ Reduce Q to P
○ Describe a transformation that maps an arbitrary
instance of Q to some instance of P, s.t. “yes” for P =
“yes” for Q
○ Prove the transformation works
○ Prove the transformation takes polynomial time
■ Oh yeah, prove P  NP (What if you can’t?)
Coming Up
● Given one NP-Complete problem, we can prove
many interesting problems NP-Complete
■ Graph coloring (recall the wrestler-rivalry HW
problem)
■ Hamiltonian cycle
■ Hamiltonian path
■ Knapsack problem
■ Traveling salesman
■ Job scheduling with penalties
■ Many, many more