NP-Completeness
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong
http://www.cs.cityu.edu.hk/~helena
7. NP-Completeness - 1
NP-Completeness
Coming up
Intractability
P,NP,NP-Complete,P<>NP
NP-Completeness Proofs
• Informal Illustration
• Key Concepts
• Formal definitions: P,NP,NP-Complete
• First NP-Complete Problem
• NP-Completeness proofs and Examples
(Chap 34)
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong
http://www.cs.cityu.edu.hk/~helena
7. NP-Completeness - 2
Polynomial-time Alg., Tractable
Polynomial-time algorithms:
On inputs of size n, the worst-case running time is
O(nk) for some constant k.
There are some problems that cannot be solved in
polynomial-time.
Eg. Turing’s famous Halting Problem cannot be solved by
any computer, no matter how much time is given:
Halting Problem: The problem of determining in
advance whether a particular program or
algorithm will terminate (will not run forever).
Example_A(…)
For i = 1 to n
..
while (i < j*k)
if (j is ..)
return true
…
return false
Polynomial-time algorithms are “Tractable”, “Easy”, eg. O(n), O(1),
O(n3), O(n lg n)
vs Superpolynomial time: “Intractable”, “Hard”, eg. O(2n),O(nn),O(n!)
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong
http://www.cs.cityu.edu.hk/~helena
7. NP-Completeness - 3
P, NP, NPC
3 classes of problems:
Class P:
Solvable in Polynomial-time ( O(nk) ) for some constant k.
Class NP:
“Verifiable” in polynomial-time
Given a “certificate” of a solution, we can verify that the
certificate is correct in O(nk) time.
Hamiltonian-cycle problem: Find a simple cycle that contains
each vertex of a given directed graph.
A certificate would be a path of n vertices.
It is easy to check that this certificate is correct, in O(nk) time.
A problem solvable in polynomial-time must be verifiable in
polynomial time. ie. P  NP.
Class NPC (The class of NP-complete problems):
These problems are in NP and are as “hard” as any
problem in NP, eg. Hamiltonian-cycle
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong
http://www.cs.cityu.edu.hk/~helena
7. NP-Completeness - 4
P, NP, NPC
Class P
Solvable in
Polynomial-time
We are interested in whether
P = NP ? --- (*)
Class NP
Verifiable in
polynomial-time
If we can find a polynomialverifiable problem that is proved
not polynomial-time solvable,
then we can disprove (*).
NP
P
We know P  NP
P=NP
NP
P
But so far we cannot find any.
Class NPC (The class of NP-complete problems):
These problems are
There are many problems in NPC. So it is important
hardest ones in NP (no
to know whether these problems are tractable.
other NP problem is
harder then these
Moreover, if a polynomial time solution can be found
problems)
for any one of them, then all NP problems are
polynomial-time solvable (hence * can be proved).
However, so far no such solution is discovered.
NP
P=NP
CS3381
Des & Anal
of Alg (2001-2002
SemA)
So,
many
believe
that
P

NP
(NP-complete
problems
are
intractable.)
http://www.cs.cityu.edu.hk/~helena
City Univ of HK / Dept of CS / Helena Wong
7. NP-Completeness - 5
P, NP, NPC
Class P: Solvable in Polynomial-time
Class NP: “Verifiable” in polynomial-time
Class NPC (The class of NP-complete problems):
These problems are in NP and are as “hard” as any problem in NP.
…
So, many people believe that P  NP (NP-complete problems are intractable.)
Why study NP-completeness?
• When we can establish a problem as NP-complete, we provide
good evidence for its intractability.
• Then we better spend the time to develop an approximation
algorithm, or solve a tractable special case.
• Interestingly, several NP-complete problems are similar to some
tractable ones.
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong
http://www.cs.cityu.edu.hk/~helena
7. NP-Completeness - 6
More examples
1. Shortest vs longest simple paths
Single-source shortest paths can be found in O(VE) time.
But finding longest simple path between 2 vertices is NP-complete
2. Euler tour vs hamiltonian cycle
Euler tour: traverses each edge of a connected, directed graph exactly once (O(E))
Hamiltonian cycle: a simple cycle that contains each vertex of a given graph:
NP-complete
3. 2-CNF satisfiability vs 3-CNF satisfiability (k-conjunctive normal form)
A boolean formula is satisfiable if there is some assignment of the 0/1
variable values that makes the formula results in “1”. Example:
2-CNF : P
OR
x1
x2
AND
OR
x1
x3
3-CNF : NPC
OR
OR
x2 x3
x1 x2 x4 x1 x3
2 elements
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong
AND
OR
OR
x4
x2 x3 x1
3 elements
http://www.cs.cityu.edu.hk/~helena
7. NP-Completeness - 7
“Polynomial time” Algorithms
By the way, why do we study
polynomial time algorithms?
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong
http://www.cs.cityu.edu.hk/~helena
7. NP-Completeness - 8
Why study polynomial-time algorithms
From experience,
once a polynomial-time algorithm for a
problem is discovered (eg. (n100)),
more efficient algorithm often follow.
ProgramA(…)
For i = 1 to n
while (i < j*k)
For i = 1 to n
For i = 1 to n
..
while (i < j*k)
For i = 1 to n
..
while (i < j*k)
if (j is ..)
For i = 1 to n
while (i < j*k)
For i = 1 to n
..
while (i < j*k)
if (j is ..)
return false
…
ProgramA(…)
For i = 1 to n
..
while (i < j*k)
For i = 1 to n
..
while (i < j*k)
if (j is ..)
return true
if (j is ..)
return true
…
return false
..
while (i < j*k)
if (j is ..)
return true
if (j is ..)
A problem that can be solved in polynomial time
in one model (one computer architecture) can
be solved in polynomial time in another.
Polynomials are closed under
,
, ..
Example:
if an algorithm only include polynomial-time
execution of basic instructions and a constant
number of calls to polynomial-time subroutines,
the running time of the composite algorithm is polynomial.
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong
http://www.cs.cityu.edu.hk/~helena
O(n2)
ProgramA(…)
For i = 1 to n
..
while (i < j*k)
For i = 1 to n
..
while (i < j*k)
return true
if (j is ..)
return true
…
return false
ProgramA(…)
For i = 1 to n
..
while (i < j*k)
For i = 1 to n
if (j is ..)
return true
…
return false
ProgramA(…)
For i = 1 to n
..
while (i < j*k)
while (i < j*k)
while (i < j*k)
while (i < j*k)
return false
O(n lg n)
7. NP-Completeness - 9
How to prove NP-Completeness
How do we know a problem
is NP-complete?
We’ll have an informal illustration first:
• The concept of “Polynomial-time reducibility”
- For comparing the hardness of problems.
• How to prove a problem to be NP-complete
(the first NP-complete problem).
• Based on some already known NP-complete
problems, how to prove some other problems
are NP-complete.
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong
http://www.cs.cityu.edu.hk/~helena
7. NP-Completeness - 10
How to prove NP-Completeness
Informal Illustration
Given 2 problems: P1 and P2,
• We map an input problem instance
of P1 to P2 ,
• Then solve for P2,
• And then obtain the answer for P1
directly from the answer from P2.
Instance of P1
Solve
for P1 Map
Polynomial-time
instance of P1 to
Instance
Reduction
of P2
Instance of P2
If the mapping (reduction) is in
polynomial time, then we say:
Solve for P2
P1 is polynomial-time reducible to P2.
Answer of
instance of P2
We can use P2 as a subroutine of P1, with
only a polynomial amount of overhead.
Answer of
instance of P1
Note that, if we can solve a problem P1 by mainly solving P2, this does not
imply P1 is harder than P2. Indeed, we can say the P2 algorithm is a powerful
engine that can solve for P1, although maybe P1 is only a piece of cake.
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong
http://www.cs.cityu.edu.hk/~helena
7. NP-Completeness - 11
How to prove NP-Completeness
Informal Illustration
Note that, if we can solve a problem P1 by mainly solving P2, this does not
imply P1 is harder than P2. Indeed, we can say the P2 algorithm is a powerful
engine that can solve for P1, although maybe P1 is only a piece of cake.
NP-complete problems are the hardest ones of all NP problems.
One method to prove that a particular problem is NP-complete:
Given a particular problem that can
essentially be used as a subroutine for all
problems in NP, with only a polynomial
amount of overhead.
(Polynomial-time Reduction)
=> This problem is harder than or at least
equally hard as all NP problems.
“This problem is
”
=> If this problem is also in NP, then, this
problem is NP-complete.
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong
http://www.cs.cityu.edu.hk/~helena
Instance of a
problem in NP
Solve for the
problem in NP
Polynomial-time
Reduction
Instance of a
subroutine problem
An problem
used as subroutine
Answer of the
subroutine problem
Answer of the
problem in NP
7. NP-Completeness - 12
How to prove NP-Completeness
Informal Illustration
Note that, if we can solve a problem P1 by mainly solving P2, this does not
imply P1 is harder than P2. Indeed, we can say the P2 algorithm is a powerful
engine that can solve for P1, although maybe P1 is only a piece of cake.
NP-complete problems are the hardest ones of all NP problems.
After we prove that a particular problem is NP-complete,
Given this NP-complete problem, eg., P1, we can prove
some others are NP-complete:
To prove the NP-completeness for P2:
If we can show that P2 can essentially be used as a
subroutine for P1, with only a polynomial amount of
overhead. (Polynomial-time Reduction)
=> P2 is harder than or at least equally hard as P1.
=> P2 is harder than or at least equally hard as all NP
problems. “P2 is NP-hard”
=> If P2 is also in NP, then, P2 is NP-complete.
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong
http://www.cs.cityu.edu.hk/~helena
Instance of P1
(NP-complete)
Solve for P1
(NP-complete)
Polynomial-time
Reduction
Instance of P2
(in NP)
P2
Answer of P2
(in NP)
Answer of P1
(NP-complete)
7. NP-Completeness - 13
How to prove NP-Completeness
Informal Illustration
Example: Assume we already know that the Hamiltonian Cycle Problem is NP-complete:
Hamiltonian Cycle Problem:
Traveling Salesman Problem:
whether there is a simple cycle containing
each vertex of a given undirected graph
(simple cycle => each vertex visited once)
whether there is a simple cycle containing
each vertex of a given complete graph with
edge costs and has total cost  K?
H Problem:
T Problem:
1
2 1
2 2
1
1
1 2
1
We can reduce the H problem to the T problem as shown.
Then we solve the T problem with k=number of edges in the H problem.
Then we use the result of the T problem to answer the H problem.
Since we already know that the H problem is NP-Complete.
We can conclude that the T http://www.cs.cityu.edu.hk/~helena
problem is
.7. NP-Completeness - 14
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong
How to prove NP-Completeness
Let’s go to a formal study of
NP-Completeness proofs.
To prove a problem is NP-complete, we need to
show “how hard it is”.
3 Key concepts for the proofs.
A. Decision problems vs. optimization problems
B. Reductions
C. A first NP-complete problem
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong
http://www.cs.cityu.edu.hk/~helena
7. NP-Completeness - 15
How to prove NP-Completeness
3 Key concepts for the proofs
A. Decision problems vs. optimization problems
NP-Completeness applies directly to decision problems, not optimization
ones. But optimization problems usually have a related decision problem.
Optimization problems
each feasible solution has
a value to be optimized
eg. SHORTEST-PATH
Decision problems
answer is simply yes or no.
PATH(G,u,v,k)
1.run SHORTEST-PATH(G,u)
2.compare result of SHORTEST-PATH with k
if
then
is easy,
is easy as well.
eg. PATH(G,u,v,k) :
Likewise, “a decision problem is hard”
implies “the related opt. problem is hard”.
whether a path exists
between 2 given vertices
having at most k edges.
Hence the theory of NP-completeness
often has implications for optimization
problems as well.
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong
http://www.cs.cityu.edu.hk/~helena
7. NP-Completeness - 16
How to prove NP-Completeness
3 Key concepts for the proofs
B. Reductions

• We have 2 decision problems: A and B.
• We have a a polynomial-time reduction
algorithm that transforms any instance  of A
into some instance  of B such that the
answers (yes/no) to  and  are the same.
• “Problem A is hard” => “
”
• “A is NP-Complete” => “
if B is also in NP.
”
An algorithm
to decide A
Polynomial-time
reduction algorithm

An algorithm
to decide B
yes
yes
no
no
The reduction method for NP-Completeness proofs:
Given an NP-Complete problem A, we can prove problem B as NP-Complete by
1. Prove that B is in NP.
2.
Then, find a polynomial-time reduction algorithm from A to B.
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong
http://www.cs.cityu.edu.hk/~helena
7. NP-Completeness - 17
How to prove NP-Completeness
3 Key concepts for the proofs
C. A first NP-Complete Problem

An algorithm
to decide A
Polynomial-time
reduction algorithm

2. Then, find a polynomial-time reduction algorithm
from A to B.
no
yes
no
The notation of
polynomial-time reducibility:
Eg. A p B
Given an NP-Complete problem A, we can prove
problem B as NP-Complete by
1. Prove that B is in NP.
An algorithm
to decide B
yes
The reduction method for NP-Completeness proofs:
p
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong
• We need a first NP-Complete problem
in order to prove a different problem
NP-Complete.
• Our first NP-Complete problem:
Circuit-Satisfiability Problem
http://www.cs.cityu.edu.hk/~helena
7. NP-Completeness - 18
In the followings, we will
(1) Formalize the definitions of P, NP, and NPComplete
(2) Illustrate NP-Complete proofs.
(3) Introduce some NP-Complete problems.
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong
http://www.cs.cityu.edu.hk/~helena
7. NP-Completeness - 19
Encoding, Polynomial-time solvable
Encoding of a set A of abstract objects
is a mapping from A to the set of
binary strings.
The PATH problem :
Eg. We can map {0,1,2,3,4,..15}
to {0000,0001,0010,..1111}
Polygons, graphs, programs
-- All can be encoded as binary strings.
u
v
Eg., we can encode each one
such instance as
G,u,v,k = “00101100…0101”
• A PATH algorithm is to map every such G,u,v,k to {1,0}, ie. {yes,no}
For some binary string that does not represent any relevant problem
instance, the PATH algorithm maps it to “0” or “no”.
• A problem is
if there is an algorithm that solves it
in time O(nk), for some constant k, where n = the size of the encoded
binary string representation of each problem instance.
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong
http://www.cs.cityu.edu.hk/~helena
7. NP-Completeness - 20
Accepted vs Decided, Class P
• Decision problems : map the problem instances to {0,1} (ie. True/False).
• Given an algorithm A and an input instance x, we can say that
A
x if A(x) returns 1. A
x if A(x) returns 0.
• Consider all inputs x1,x2,.. xn that results “1”.
Suppose L is a set such that L = {x1,x2,.. xn}.
Then L is accepted by A if every element xi in L is accepted by A.
(But A may loop forever for an element x L without rejecting it.)
L is decided by A if every element xi in L is accepted by A and
every element not in L is rejected by A.
For Turing’s Halting Problem, there exists an accepting algorithm, but no
decision algorithm exists.
• The complexity class P : A problem belongs to P if and only if it has a
polynomial time decision algorithm.
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong
http://www.cs.cityu.edu.hk/~helena
7. NP-Completeness - 21
Verification Algorithm, CLASS NP
• Consider the decision problem PATH, for a given problem instance
x = <G,u,v,k>, we need to determine whether it is true or false.
If we are also given a path from u to v (a certificate) that helps to prove
for k, then the verification of x is easy.
• A
A is a 2-input algorithm that outputs
whether a given certificate verifies a given problem instance x:
A(x,certificate) = 1, or A(x,certificate) = 0
Eg. bool Verify_Path(<G,u,v,k>,a_path)
• The complexity class NP: A problem belongs to NP if and only if it
has a polynomial-time verification algorithm, A(x, certificate), such that
|certificate| = O(|xc|).
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong
http://www.cs.cityu.edu.hk/~helena
7. NP-Completeness - 22
NP-Completeness and the Proofs
• A problem, A, is NP-Complete:
CIRCUIT-SAT
(1) A  NP, and
(2) L p A, for every L NP. [NP-hard]
SAT
• We also say that A belongs to NPC
3-CNF-SAT
• Some famous NP-Complete problems:
CIRCUIT-SAT, SAT, 3-CNF-SAT, CLIQUE, ..
• Alternatively, we can conclude that a
problem, A, is NP-Complete if:
(1)
(2’)
, and
, for some L NPC [=>
CLIQUE
SUBSET-SUM
VERTEX-COVER
] HAM-COVER
CIRCUIT-SAT is proved using (1) and (2).
TSP
SAT, 3-CNF-SAT, CLIQUE.. are proved using (1) and (2’).
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong
http://www.cs.cityu.edu.hk/~helena
7. NP-Completeness - 23
CIRCUIT-SAT
First NP-complete Problem
Circuit-Satisfiability
Problem
(CIRCUIT-SAT)
x1
x2
Given a boolean combinational
circuit composed of AND, OR,
and NOT gates, is it satisfiable?
x
z
x x
0
1
x3
If we assign 1, 1, 0 to x1, x2, and x3, the output will be 1.
=> The circuit is satisfied by some
=> It is satisfiable.
However, if A is replaced by an AND gate, we can’t find any
“assignment” of (x1, x2, x3) to make it satisfied. => It is not satisfiable.
http://www.cs.cityu.edu.hk/~helena
1
0
x
z
y
x y xy
A
Output
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong
Elements:
0
0
1
1
0
1
0
1
0
0
0
1
x
z
y
x y xy
0
0
1
1
0
1
0
1
0
1
1
1
7. NP-Completeness - 24
CIRCUIT-SAT
First NP-complete Problem
x1
x2
Output
x3
• The
of a boolean combinational circuit
= no. of boolean combinational elements + no. of wires
• To
such a circuit into a binary string:
we can devise a graph like encoding
• If there are k inputs, then there are 2k possible assignments. By
simply checking all these assignments => O( ).
• By proving that it is NP-Complete, we can claim that “there is strong
evidence that no polynomial-time algorithm exists” for CIRCUIT-SAT.
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong
http://www.cs.cityu.edu.hk/~helena
7. NP-Completeness - 25
CIRCUIT-SAT First NP-complete Problem
Part A of the proof: CIRCUIT-SATNP
CIRCUIT-SAT is NP-Complete
2 parts of the proof:
A. CIRCUIT-SAT NP
B. CIRCUIT-SAT is NP-hard
x1
x2
Output
x3
Part A of the proof: CIRCUIT-SATNP
Consider an algorithm:
bool Verify_Circuit_Sat( Input_Circuit,
Certificate/*an assignment to the wires, eg. x1, x2, x3*/)
This algorithm computes values of the logic gates based on the
assignment. If circuit output is 1 (0), then the algorithm returns 1 (0).
This is a
algorithm for CIRCUIT-SAT
Since we can find such an algorithm for CIRCUIT-SAT, we say that
CIRCUIT-SAT can be verified in polynomial time, and
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong
http://www.cs.cityu.edu.hk/~helena
7. NP-Completeness - 26
.
CIRCUIT-SAT First NP-complete Problem
Part B of the proof: CIRCUIT-SAT is NP-hard
Part B of the proof: CIRCUIT-SAT is NP-hard
• Show that every NP problem
is polynomial-time reducible to
CIRCUIT-SAT.
• When a program is being executed,
the computer changes the content
of the computer memory by running
program statements.
• The computer hardware can be
constructed as a boolean
combinational circuit.
One Configuration
of Computer Memory
Config1
Another Configuration
of Computer Memory
Config2
Another Configuration
of Computer Memory
Config3
• Hence correspondingly we can
construct a boolean combinational
circuit that maps Configi to Configi+1
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong
Transformed by
the computer
by running first
statement
Run
second
statement
Configi
a boolean
combinational circuit
Configi+1
http://www.cs.cityu.edu.hk/~helena
7. NP-Completeness - 27
CIRCUIT-SAT First NP-complete Problem
Part B of the proof: Every NP problem is polynomial-time reducible to CIRCUIT-SAT
Part B of the proof: CIRCUIT-SAT is NP-hard
• If the program is a 2-input
verification program, then,
Config1 is:
Encoded_Input Certificate
Program code
• Suppose T(n) = worst case
running time. Then the solution
can be obtained by evaluating
Config1, Config2, Config3, ..
ConfigT(n).
• If the exact execution time is
452, then, Config452 = Config453
= Config454 = … = ConfigT(n) .
Config1
Program code
Variables, etc.
Config2
Program code
Variables, etc.
ConfigT(n)
Program code
• In the “final” configuration,
there is a memory location
storing the result (1/0).
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong
Variables, etc.
Variables, etc.
0/1 Output
http://www.cs.cityu.edu.hk/~helena
7. NP-Completeness - 28
CIRCUIT-SAT First NP-complete Problem
Part B of the proof: Every NP problem is polynomial-time reducible to CIRCUIT-SAT
Part B of the proof: CIRCUIT-SAT is NP-hard
Let L be any problem in NP, we need to describe a polynomial-time algorithm F to reduce
every input instance of L to a circuit C such that result of L = result of C.
For any NP problem, it must have a 2-input polynomial-time verification algorithm A.
Suppose T(n) = worst case running time of A.
Execution of A:
Encoded_Input Certificate
?
Program
Variables,
code of A
etc.
Config1
Config1
Circuit1
Circuit1
Config2
Config2
Circuit2
Circuit2
Circuit2
CircuitT(n)
CircuitT(n)
CircuitT(n)
Circuit1
0/1
polynomial_time_F
Encoded Input + “bool A(Encoded_input,Certificate)”
A boolean combinational
circuit C such that “
http://www.cs.cityu.edu.hk/~helena
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong
7. NP-Completeness - 29
”
CIRCUIT-SAT First NP-complete Problem
Part B of the proof: Every NP problem is polynomial-time reducible to CIRCUIT-SAT
Part B of the proof: CIRCUIT-SAT is NP-hard
For each NP problem, we design F in this way:
Bool_Combinational_Circuit polynomial_time_F(encoded_input_of_an_NP_instance)
1 n = size of encoded_input_of_an_NP_problem_instance
2 Compute T(n) of alg. A using n, where A is the verification program of the NP_problem
3 Based on a computer capable of running A, create T(n) copies of the corresponding
boolean combinational circuits, such that circuiti will map configi to configi+1.
4 Create the whole_circuit by wiring the outputs of circuit1 to circuit2, and outputs of
circuit2 to circuit3, etc.
5 Wire the “Program Code” pins of circuit1 according to A’s code.
6 Wire the “Encoded_input” pins of circuit1 according to
encoded_input_of_an_NP_problem_instance.
7 Ignore (ground) all outputs of whole_circuit, except the 0/1 output of circuitT(n).
8 Return whole_circuit.
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong
Polynomial time of n
http://www.cs.cityu.edu.hk/~helena
7. NP-Completeness - 30
NP-Completeness and the Proofs
After some problems have been proved NP-Complete,
we can prove for other problems more easily:
CIRCUIT-SAT
A common approach to prove that a problem,
A, is NP-complete:
Show
SAT
•
•
3-CNF-SAT
1. Prove A  NP
2. Select a known NP-complete problem K
CLIQUE
3. Describe an algorithm that maps every
instance of A to an instance of K
VERTEX-COVER
4. Prove that the results for both instances
(yes or no) in (3) are the same
HAM-COVER
5. Prove that the algorithm in (3) runs in
polynomial time [ Polynomial-time Reducibility ]
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong
http://www.cs.cityu.edu.hk/~helena
SUBSET-SUM
TSP
7. NP-Completeness - 31
The SAT Problem
Given a boolean formula of n boolean
variables, m boolean connectives, and required
parenthesis, is it satisfiable?
Formula-Satisfiability
Problem
(SAT)
Boolean functions:
P
T
T
F
F
Q
T
F
T
F
PQ PQ
T
T
F
T
F
T
F
F
P
F
F
T
T
PQ
T
F
T
T
P Q
T
F
F
T
Example: ((x1  x2)  (( x1  x3)  x4))   x2
It is satisfiable by the assignment: <x1=0, x2=0, x3=1, x4=1>:
((0  0)   (( 0  1)  1))   0
= (1  (1  1))  1
=
=
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong
http://www.cs.cityu.edu.hk/~helena
7. NP-Completeness - 32
The SAT Problem
Part A of the proof: SATNP
SAT is NP-Complete
2 parts of the proof:
Example:
((x1  x2)  (( x1  x3)  x4))   x2
A. SAT NP
B. SAT is NP-hard
Part A of the proof: SATNP
Consider an algorithm:
bool Verify_Sat( Input_Boolean_Formula,
Certificate/*an assignment to the variables, eg. x1, x2, x3*/)
This algorithm replaces each variable in the formula with its corresponding
values and evaluate the expression.
This is a 2-input, polynomial-time verification algorithm for SAT.
Since we can find such an algorithm for SAT, we say that SAT can be
verified in polynomial time, and
.
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong
http://www.cs.cityu.edu.hk/~helena
7. NP-Completeness - 33
The SAT Problem
Part B of the proof: SAT is NP-hard
Part B of the proof: SAT is NP-hard
Show that CIRCUIT-SAT p SAT
ie. Any instance of circuit satisfiability can be reduced in polynomial time to
an instance of formula satisfiability.
Reduction by simple translation,
Sometimes it is easy:
s
t
z
x
y
z = (s t)  ((x y))
Original circuit: 4 gates + 8 wires
Resultant formula: 4 variables + 4 operators
But not always:
t
x
y
z
z = (t  (xy))  ((xy))
Original circuit: 4 gates + 8 wires
Resultant formula: variables + operators
Reason of complexity: Fan-out > 1
Indeed, such a reduction is not in polynomial time.
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong
http://www.cs.cityu.edu.hk/~helena
7. NP-Completeness - 34
The SAT Problem
Part B of the proof: CIRCUIT-SAT p SAT
Part B of the proof: SAT is NP-hard
x1
x2
x5
Since we need to show that “Any
instance of circuit satisfiability can be
reduced in
to an
instance of formula satisfiability.”
We design a more clever method:
Step 1. For each gate, formulate it with
the operation on its incident
wires.
eg. x10  (x7  x8  x9)
Step 2. “AND” all the formulas of the
gates.
x6
x3
x4
x8
x9
x10
x7
The formula:
x10  (x4   x3)
 (x5  (x1  x2))
 (x6  x4)
 (x7  (x1  x2  x4))
 (x8  (x5  x6))
 (x9  (x6  x7))
 (x10  (x7  x8  x9))
A CIRCUIT-SAT instance with m gates and n wires
Polynomial time
CS3381 Des & Anal of Alg (2001-2002 SemA)
A SAT instance
(size = polynomial in the size of7. CIRCUIT-SAT)
http://www.cs.cityu.edu.hk/~helena
City
Univ of HK / Dept of CS / Helena Wong
NP-Completeness - 35
reduction
The 3-CNF-SAT Problem
3CNF-Satisfiability Problem (3-CNF-SAT)
Given a boolean formula in
3-CNF, is it satisfiable?
A literal: eg. x1 or x1 (an occurrence of a variable or its negation)
Conjunctive normal form
(CNF)
Eg. (x1x2)(x3x2x4)x1
2-CNF
Eg. (x1x2)(x3x4)(x1x3)
3-CNF
Eg. ( x1x1x2)
(x3x2x4)
(x1x3x4)
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong
One or more literals “” together
x? x?
x? x?
x? x?
3-CNF
or?x
or?  …)(x
or?x
or?  …)(x
or ?x
or?  …)..
(x
x? x?
x? x?
x? x?
x? x?
x? x?
x? x?
(x
or?x
or?)(x
or?x
or ?)(x
or?x
or ?)..
x? x? x? x? x? x?
x? x? x?
x? x? x?
x? x? x?
or?x
or?x
or?)(x
or?or??x
or?)(x
or?x
or?x
or?)..
(x
x? x? x? x? x? x? x? x? x?
http://www.cs.cityu.edu.hk/~helena
7. NP-Completeness - 36
The 3-CNF-SAT Problem
Part A of the proof: 3-CNF-SATNP
3-CNF-SAT is NP-complete
2 parts of the proof:
A. 3-CNF-SAT NP
B. 3-CNF-SAT is NP-hard
A 3-CNF Example:
(x1x1x2)(x3x2x4)(x1x3x4)
Part A of the proof: 3-CNF-SATNP
Consider an algorithm:
bool Verify_3_CNF_SAT( Input_Boolean_Formula,
Certificate /*an assignment to the variables*/ )
This algorithm replaces each variable in the formula with its corresponding
values and evaluate the expression.
This is a 2-input, polynomial-time verification algorithm for 3-CNF-SAT.
Since we can find such an algorithm for 3-CNF-SAT, we say that 3-CNFSAT can be verified in polynomial time, and 3-CNF-SATNP.
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong
http://www.cs.cityu.edu.hk/~helena
7. NP-Completeness - 37
The 3-CNF-SAT Problem
Part B of the proof: 3-CNF-SAT is NP-hard
Part B of the proof: 3-CNF-SAT is NP-hard
Show that SAT p 3-CNF-SAT
ie. Any instance of formula
satisfiability can be reduced in
polynomial time to an instance
of 3-CNF formula satisfiability.
A formula Example:
((x1  x2)   ((x1  x3 )  x4 ))  x2
A 3-CNF Example:
(x1x1x2)(x3x2x4)(x1x3x4)
The Reduction :
Step 1: Create a binary “parse” tree for the formula.
Step 2: Rewrite it in the form:
y1  (y1
 (y2
 (y3
 (y4
 (y5
 (y6
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong
 (y2  x2))
 (y3  y4))
 (x1  x2 ))
  y5)
 (y6  x4))
 ( x1  x3))
http://www.cs.cityu.edu.hk/~helena
y1

y2
x2
y3  y4

x1

x2

x4
 y6
x1
y5
x3
7. NP-Completeness - 38
Part B of the proof: 3-CNF-SAT is NP-hard
Show that SAT p 3-CNF-SAT
ie. Any instance of formula satisfiability
can be reduced in polynomial time to an
instance of 3-CNF formula satisfiability.
A formula: eg. ((x1  x2)   ((x1  x3 )  x4 ))  x2
3-CNF eg. (x1x1x2)(x3x2x4)(x1x3x4)…
The Reduction :
Step 3: Change each sub-clause of the following to an OR of literals:
y1  (y1  (y2  x2))
Eg. y1 y2 x2 (y1 (y2  x2)) (y y x )
1
2
2
 (y2  (y3  y4))
1 1 1
0
1 1 0
1
 (y3  (x1  x2 ))
(y1y2x2)
1 0 1
0
1 0 0
0
 (y4   y5)
0 1 1
1
(y1y2x2)
 (y5  (y6  x4))
0 1 0
0
0 0 1
1
 (y6  ( x1  x3))
(y1y2 x2)
0 0 0
1
We can rewrite (y1 (y2  x2)) as:
 [ (y1y2x2) (y1y2x2)  (y1y2x2)  (y1y2 x2) ]
By DeMorgan’s laws:
=> (y1y2x2)  (y1y2x2)  (y1y2x2)  (y1y2 x2)
=> (y1  y2  x2) (y1  y2  x2)  (y1  y2  x2)  (y1  y2  x2)
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong
http://www.cs.cityu.edu.hk/~helena
7. NP-Completeness - 39
Part B of the proof: 3-CNF-SAT is NP-hard
Show that SAT p 3-CNF-SAT
ie. Any instance of formula satisfiability
can be reduced in polynomial time to an
instance of 3-CNF formula satisfiability.
A formula: eg. ((x1  x2)   ((x1  x3 )  x4 ))  x2
3-CNF eg. (x1x1x2)(x3x2x4)(x1x3x4)…
The Reduction :
Step 3: Change each sub-clause to an OR of literals:
(Con’td)
y1  (y1  (y2  x2))
 (y2  (y3  y4))
 (y3  (x1  x2 ))
 (y4   y5)
 (y5  (y6  x4))
 (y6  ( x1  x3))
We can rewrite (y1 (y2  x2)) as:
(y1  y2  x2) (y1  y2  x2)  (y1  y2  x2)  (y1  y2  x2)
Apply the same
method to other
sub-clauses:
Polynomial time
reduction
y1  (y1y2x2)(y1y2x2)(y1y2x2)(y1y2 x2)
 (y2  (y3  y4))
 (y3  (x1  x2 ))
 (y4   y5)
 (y5  (y6  x4))
 (y6  ( x1  x3))
A SAT instance
A 3-CNF-SAT instance
http://www.cs.cityu.edu.hk/~helena
7. NP-Completeness
such that result
of SAT instance = result of 3-CNF-SAT
instance- 40
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong
NP-Completeness and the Proofs
Recall:
CIRCUIT-SAT
SAT
3-CNF-SAT
CLIQUE
SUBSET-SUM
VERTEX-COVER
HAM-CYCLE
TSP
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong
http://www.cs.cityu.edu.hk/~helena
7. NP-Completeness - 41
The Clique Problem
A complete graph
u
v
A Clique in an undirected graph
G=(V,E) is a complete subgraph of G.
u
y
x
v
z
w
y
x
Clique Problem
Optimization problem: Find a clique of maximum size in a graph.
Decision problem: Whether a clique of a given size k exists in the graph.
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong
http://www.cs.cityu.edu.hk/~helena
7. NP-Completeness - 42
The CLIQUE Problem
Part A of the proof: CLIQUENP
CLIQUE is NP-complete
2 parts of the proof:
A. CLIQUE NP
B. CLIQUE is NP-hard
u
v
z
Part A of the proof: CLIQUE NP
w
y
x
Consider an algorithm:
bool Verify_CLIQUE(
Input_Graph,
Certificate /*a set of vertices in the input graph*/ )
This algorithm checks whether the set of vertices in the certificate are
linked up as a complete graph.
This is a 2-input, polynomial-time verification algorithm for CLIQUE.
Since we can find such an algorithm for CLIQUE, we say that CLIQUE can
be verified in polynomial time, and CLIQUENP.
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong
http://www.cs.cityu.edu.hk/~helena
7. NP-Completeness - 43
The CLIQUE Problem
Part B of the proof: CLIQUE is NP-hard
Part B of the proof: CLIQUE is NP-hard
Show that 3-CNF-SAT p CLIQUE
A 3-CNF Example:
(x1x2x3)(x1x2x3)(x1x2x3)
ie.Any instance of 3-CNF formula satisfiability can
be reduced in polynomial time to an instance of CLIQUE.
x1
x
2
x
3
The Reduction :
Describe a 3-CNF formula with k sub-clauses as:
(l11  l12  l13)  (l21  l22  l23)  .. (lk1  lk2  lk3)
Reduce it to a clique problem such that it is
satisfiable if and only if a corresponding graph
has a clique of size k.
Step 1: Represent each “term”
lr
i
as a vertex
v ri .
Step 2: Create the edges for any two
vertices: vri and vsj if:
r  s and lri is not the negation of lsj
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong
http://www.cs.cityu.edu.hk/~helena
x
1
x1
x2
x2
x3
x3
Polynomial time reduction
A 3-CNF-SAT instance
A CLIQUE instance
s.t. result of 3-CNFSAT instance = result
of7.CLIQUE
instance
NP-Completeness
- 44
The VERTEX-COVER Problem
A Vertex-Cover in an undirected graph is a set
of vertices that cover all edges in the graph.
A vertex Covers
a set of edges:
u
v
z
u
w
y
x
v
z
u
w
y
x
v
z
w
y
x
Vertex-Cover = {w,z}
Vertex-Cover Problem
Optimization problem: Find a vertex-cover of minimum size in a graph.
Decision problem: Whether a graph has a vertex-cover of a given size k.
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong
http://www.cs.cityu.edu.hk/~helena
7. NP-Completeness - 45
The VERTEX-COVER Problem
Part A of the proof: VERTEX-COVERNP
VERTEX-COVER is NP-complete
2 parts of the proof:
A. VERTEX-COVER NP
B. VERTEX-COVER is NP-hard
u
v
z
w
y
x
Part A of the proof: VERTEX-COVER NP
Consider an algorithm:
bool Verify_VERTEX_COVER( Input_Graph,
Certificate /*a set of vertices*/ )
This algorithm checks whether the set of vertices in the certificate cover all
edges in the input graph.
This is a 2-input, polynomial-time verification algorithm for VERTEX-COVER.
Since we can find such an algorithm for VERTEX-COVER, we say that VERTEXCOVER can be verified in polynomial time, and VERTEX-COVER NP.
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong
http://www.cs.cityu.edu.hk/~helena
7. NP-Completeness - 46
The VERTEX-COVER Problem
Part B of the proof: VERTEX-COVER is NP-hard
Part B of the proof: VERTEX-COVER is NP-hard
Show that CLIQUE p VERTEX-COVER
ie.Any instance of CLIQUE satisfiability can be reduced
in polynomial time to an instance of VERTEX-COVER.
u
v
z
w
y
x
The Reduction :
For any CLIQUE satisfiability problem < G=(V,E), k>, we can create a
VERTEX-COVER problem <G’,|V|-k> such that G’ is the complement of G.
Example:
G
G’ = complement of G
CLIQUE problem:
VERTEX-COVER
u
v
u
v
In the graph
problem:
G=(V,E), can we
In the graph G’, can
z
w
z
w
find a clique of
we find a clique of
size k=4?
size |V|-k=2?
y
x
y
x
Polynomial
A CLIQUE instance
time
CS3381 Des & Anal of Alg (2001-2002 SemA)
reduction
VERTEX-COVER
instance
s.t. result of CLIQUE inst. = result
VERTEX-COVER
http://www.cs.cityu.edu.hk/~helena
City Univ of HK / Dept A
of CS
/ Helena Wong
7. of
NP-Completeness
- 47inst.
The HAM-CYCLE Problem
A Hamiltonian Cycle in an
undirected graph is a simple cycle
that contains each vertex in V.
u
v
z
The HAM-CYCLE
Problem
Does a graph have a
hamiltonian cycle?
w
y
x
HAM-CYCLE is NP-complete
2 parts of the proof:
A. HAM-CYCLE NP
B. HAM-CYCLE is NP-hard
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong
http://www.cs.cityu.edu.hk/~helena
7. NP-Completeness - 48
The HAM-CYCLE Problem
Part B of the proof: HAM-CYCLE is NP-hard
Part A of the proof: HAM-CYCLE NP
Consider an algorithm:
bool Verify_HAM-CYCLE (Input_Graph,Certificate /*a seq of vertices of a ham-cycle*/ )
It checks: 1. The certificate covers all vertices in input_graph exactly once.
2. The certificate is a valid path (each vertex links to next one).
3. An edge joins the first and the last vertex in the certificate.
This is a 2-input, polynomial-time verification algorithm for HAM-CYCLE.
Since we can find such an algorithm for HAM-CYCLE, we say that
HAM-CYCLE can be verified in polynomial time, and HAM-CYCLE NP.
Part B of the proof: HAM-CYCLE is NP-hard
Show that VERTEX-COVER p HAM-CYCLE
ie.Any VERTEX-COVER instance can be reduced in polynomial time to a HAM-CYCLE instance.
The Reduction : Please refer to the text book. Chp 34.5.3
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong
http://www.cs.cityu.edu.hk/~helena
7. NP-Completeness - 49
NP-Completeness
Summary
Intractability
CIRCUIT-SAT
P,NP,NP-Complete,P<>NP
NP-Completeness Proofs
• Informal Illustration
• Key Concepts
•
•
•
•
Formal definitions: P,NP,NPC
First NP-Complete Problem
NP-Completeness proofs
Examples
CS3381 Des & Anal of Alg (2001-2002 SemA)
City Univ of HK / Dept of CS / Helena Wong
http://www.cs.cityu.edu.hk/~helena
SAT
3-CNF-SAT
CLIQUE
SUBSET-SUM
VERTEX-COVER
HAM-CYCLE
TSP
7. NP-Completeness - 50