Reductions &
NP-Completeness
•See 2001 slides
•More Classes for 4111
•More Detailed Steps for 6111
Jeff Edmonds
York University
Thanks
Intro & COSC 3101/4111/6111
Reductions &
NP-Completeness
More Classes
For 4111
Jeff Edmonds
York University
COSC 3101/4111/6111
Non-Deterministic Machines
Acceptable
Computable
A Problem P is said to be
computable/decidable/recursive
•if on every input the machine
halts and give the correct answer.
Non-Deterministic Machines
Acceptable
Computable
Three equivalent definitions for a problem P
Acceptable:  TM M
•On every yes input, the machine halts
and gives the correct answer.
•On every no input, the machine
could halt and gives the correct answer
or could run for ever.
•Witnesses:
 computable Valid such that P(I) =  S Valid(I,S)
•Enumerable:  TM M
•Every yes input I, is eventually printed.
•No no input is ever printed.
Non-Deterministic Machines
Acceptable
Is there a language
that can be accepted
but not computed?
Halting
Halting(M,I)
= Does TM M halt
on input I.
Turing proves uncomputable.
Alg: Run M on I.
If it halts halt and say “yes”
If it does not halt run forever.
Computable
Non-Deterministic Machines
The Post Correspondence Problem (PCP)
•The input is a finite collection of dominos
b a
ca abc
I = ca , ab , a , c
•A solution is a finite sequence of the dominos
a b ca a abc
ab ca a ab c
I can give a
solution as a
•So that the combined string on the top
witness.
is the same as that on the bottom
abcaaabc
abcaaabc
Non-Deterministic Machines
Acceptable
But how big is
this witness?
PCP?
With
Repeats
Computable
Exp
a b ca a abc
ab ca a ab c
NP
PCP?
Without
Repeats
•If dominos can’t be repeated,
|S|  |I|
•If dominos can be repeated,
|S| can be arbitrarily long.
I can give a
solution as a
witness.
Poly
Non-Deterministic
Machines
Acceptable
CoAcceptable
Halting
Not Halting
Computable
Non-Deterministic
Yes instances I
•have accepting
computations
•have witness/solutions.
No instances I
•Do not.
Co-Non-Deterministic
No instances I
•have accepting
computations
•have witness/solutions.
Yes instances I
•Do not.
Exp
CoNP
NP
SAT
not SAT
Poly
GCD
Reductions &
NP-Completeness
•Non-Deterministic Poly-Time
•Reductions
•NP vs Co-NP Preserving Reductions
•NP-Complete
•K-Clique vs K-Independent Set
In More Depth
•12 Steps
•Any P ≤ Sat
•3-Col ≤ 3-Sat
•Sat ≤ 3-Col
•More Problems
Jeff Edmonds
•12 Steps in More Depth
York University
COSC 3101/4111/6111
Non-Deterministic
Poly-Time Decision Problems (NP)
•An optimization problem
•Each solution is either valid or not (no cost)
•The output is
•Yes, it has a valid solution.
•No, it does not
•the solution is not returned
•Eg: Given graph and integer <G,k>,
does G have a clique of size k?
Non-Deterministic Poly-Time (NP)
•Key: Given
•an instance I
(= <G,k>)
•and a solution S (= subset of nodes)
•there is a poly-time alg Valid(I,S) to test
whether or not S is a valid solution for I.
•Poly-time in |I| not in |S|. |S| can't be too big.
Not Valid
Valid
k=4
Formal definition:
Prob  NP iff  poly time Valid
such that Prob(I) =  S Valid(I,S)
Non-Deterministic Poly-Time (NP)
•Key:
•If the instance has a valid solution
•A non-deterministic (fairy god mother)
could prove it to you by giving you
such a solution as a witness.
•You could check that it is valid.
•You could convince your boss.
Valid
k=4
Non-Deterministic Poly-Time (NP)
•Key:
•If the instance does not have a valid solution
•A non-deterministic (fairy god mother)
could prove it to you by giving you
????
•You have no way to convince your boss.
k=5
Non-Deterministic Poly-Time (NP)
•Example: 3-Col:
•Instance: A graph G.
•Solution: Colouring C nodes of G with 3 colours
such that every edge has two colours.
•G is a Yes instance if there is such a colouring.
•Given an instance G and a solution C,
there is a poly-time alg Valid(G,C) to test
whether or not C is a valid 3-colouring of G.
 3-Col  NP.
Col
G
Col
Non-Deterministic Poly-Time (NP)
•Example: Not 3-Col:
•Instance: A graph G.
•Solution: Colouring C nodes of G with 3 colours
such that every edge has two colours.
•G is a No instance if there is not such a colouring.
•Given an instance G and a solution C,
there is a poly-time alg Valid(G,C) to test
whether or not C is a valid 3-colouring of G.
 Not 3-Col  NP.
 Not 3-Col  Co-NP.
I can't witness
Yes instances
G
Col
Non-Deterministic Poly-Time (NP)
Computable
Exp
CoNP
NP
3-Col
Jack Edmonds
Not 3-Col
Steve Cook
Definition is asymmetric.
There is a witness that
a graph has a 3-Col.
There is no known witness that
a graph has no 3-Col.
Poly
Known
GCD
Non-Deterministic Poly-Time (NP)
•Example: Sorting
•Instance: A list of numbers X=[x1,…xn].
•Solution: Sorted order X’.
•Given an instance X and a solution X’,
there is a poly-time alg Valid(X,X’) to test
whether or not X’ is a sorting of X.
 Sorting  NP.
Sorting is not a decision (Yes/No) problem.
X=[5,3,7,2] X’=[2,3,5,7]
Non-Deterministic Poly-Time (NP)
•Example: Parity
•Instance: A list of bits X=[x1,…xn].
•X is a Yes Instance if the number of xi=1 is odd.
 Parity  Poly-Time  NP.
X=[0,1,0,1,1,0]
parity=1
What
Don’t
do Ibegive
lazy.
as a
You can
witness?
solve this on
your own.
Non-Deterministic Poly-Time (NP)
Computable
Exp
Games
CoNP
NP
3-Col
Jack Edmonds
Not 3-Col
Steve Cook
Poly
Parity  Poly-Time  NP.
Known
Parity
Non-Deterministic Poly-Time (NP)
•Example: Airplane Wing:
•Instance: Requirements I of the wing.
•Solution: A description S of how to make the wing.
•I is a Yes instance if there is such a wing.
•Given an instance I and a proof S,
there is a poly-time alg Valid(I,S) to test
whether or not S is a valid solution for I.
Airplane Wing  NP.
I = [weight, lift, cost, …]
Non-Deterministic Poly-Time (NP)
•Example: Math Truth:
•Instance: A math statement I
•Solution: A proof S.
•I is a Yes instance if there is such a proof.
•Given an instance I and a proof S,
there is a poly-time alg Valid(I,S) to test
whether or not S is a valid solution for I.
in |I|
Math Truth  NP.
not in |S|
Witness is way too big!
I = [ a,b,c,r  3 ar+br cr]
Non-Deterministic Poly-Time (NP)
•Example: Math Truth:
•Instance: A math statement I
•Solution: A proof S.
•I is a Yes instance if there is such a proof.
•Math Statement I can encode whether a TM stops
•Halting problem poly Math Truth
 Math Truth  Undecidable.
I = [ a,b,c,r  3 ar+br cr]
Non-Deterministic Poly-Time (NP)Math Truth
Computable
Halting
Exp
Games
CoNP
NP
3-Col
Turing 1936
Not 3-Col
Poly
Known
GCD
Non-Deterministic Poly-Time (NP)Math Truth
Acceptable
Computable
Halting
Exp
Games
CoNP
NP
3-Col
If the program does halt,
I can give you its computation
as a witness.
It the statement
does have a proof,
I could give you a proof
as a witness.
Not 3-Col
Poly
Known
GCD
Reductions
Palg ≤poly Poracle
•Reduction: Design a fast algorithm for one computational
problem, using a supposedly fast algorithm for another
problem as a subroutine.
NP vs Co-NP Preserving Reductions
Computable
Exp
CoNP
NP
3-Col
Jack Edmonds
Not 3-Col
Steve Cook
Definition is asymmetric.
There is a witness that
a graph has a 3-Col.
There is no known witness that
a graph has no 3-Col.
Poly
Known
GCD
NP vs Co-NP Preserving Reductions
3-Col ≤poly Not 3-Col
<G>
No
G is not 3-Colorable
<G>
BUILD:
3-Col
Yes
G is not 3-colorable
GIVEN:
Not 3-Col
NP vs Co-NP Preserving Reductions
3-Col ≤poly Not 3-Col
<G>
Yes
G is 3-Colorable
<G>
BUILD:
3-Col
Poly time
If not poly time
No
G is 3-colorable
GIVEN:
Not 3-Col
If poly time
Not poly time
NP vs Co-NP Preserving Reductions
3-Col ≤poly Not 3-Col
<G>
Yes
G is 3-Colorable
<G>
BUILD:
3-Col
If NP-complete
No
G is 3-colorable
GIVEN:
Not 3-Col
???
NP vs Co-NP Preserving Reductions
Computable
Exp
CoNP
NP
3-Col
Jack Edmonds
Not 3-Col
Steve Cook
Definition is asymmetric.
There is a witness that
a graph has a 3-Col.
There is no known witness that
a graph has no 3-Col.
If NP-complete
Poly
Known
GCD
???
NP vs Co-NP Preserving Reductions
Palg ≤poly Poracle
Cook Reduction:
Design any fast algorithm for Palg using a
supposed fast algorithm for Poracle as a subroutine.
Karp Reduction:
The algorithm for Palg calls that for Poracle only once
Yes  Yes & No  No
NP vs Co-NP Preserving Reductions
Palg ≤poly Poracle
Karp Reduction:
Yes  Yes
& No  No
We will only consider reductions of this simple form.
Because they preserve NP vs Co-NP
NP-Complete Problems
Problem Pnew is NP-Complete
•Pnew not too hard.
•Pnew  NP
Test in poly-time
if a given solution
is valid
Computable
Exp
NP
Sat
complete
Pnew
Poly
Known
GCD
NP-Complete Problems
Problem Pnew is NP-Complete
Computable
•Pnew not too hard.
Exp
•Pnew  NP
•Pnew sufficiently hard.
•PNP, P ≤poly Pnew
NP
•Easier: Sat ≤poly Pnew
Sat
•Cook: P ≤poly Sat complete
Pnew
Poly
Known
GCD
K-Clique vs K-Independent Set
Clique: Given <G,k>, does G contains a k-clique?
Independent Set: Given <G,k>, does G contains a k-Ind Set?
A K-independent set
A K-clique is a set of K nodes
is a set of K nodes
with all edges between them.
with no edges between them.
K-Clique vs K-Independent Set
Clique: Given <G,k>, does G contains a k-clique?
Independent Set: Given <G,k>, does G contains a k-Ind Set?
Brute Force: Try out all n choose k possible subsets.
If k = (n) then 2(n) subsets to check
If k=3 then only O(n3)
K-Clique vs K-Independent Set
Clique ≤poly Indep Set
<G,k>
G has a k clique
or not
<G*, k>
BUILD:
Clique
Oracle
G* has a k Indep. set
or not
GIVEN:
Indep.
Set
Oracle
K-Clique vs K-Independent Set
Clique ≤poly Indep Set
<G,k>
G has a k clique
or not
<G*, k>
Proof of correctness:
G* has a k Indep. set
BUILD:
Our oracle says
yes to <G,k> or not
Clique
iff Old oracle
says yes to <G*,k>
iff G* has Oracle
a k indep. set
iff G has a k clique
GIVEN:
Indep.
Set
Oracle
K-Clique vs K-Independent Set
G
G*
G* has edge
if and only if
G does not
K-Clique vs K-Independent Set
G
This graph
contains a
clique
of size 4.
if and only if
G*
This graph
contains an
independent
set of size 4.
12 Steps
Steps for proving that Pnew is NP-Complete
1) I am addicted to solving NP-Complete Problems
2) I trust in my higher power to help
A witness
12 Steps
Steps for proving that Pnew is NP-Complete
0) Pnew  NP
1) What to reduce it to
2) What is what
3) Direction of reduction & Code
4) Look for similarities
5) Instance Map
6) Solution Map
7) Valid to Valid
8) Reverse Solution Map
9) Valid to Valid
10) Working Algorithm
11) Running Time
12 Steps
Step 0: Pnew  NP
Formal definition:
Pnew  NP iff  poly time Valid
such that Pnew(I) =  S Valid(I,S)
Design poly-time algorithm Valid(I,S)
which determines whether a given solution S
is valid for a given instance I.
12 Steps
Step 1: What to reduce it to
Palg ≤poly Poracle
Choose a problem Pis NP-comp
that is as similar to Pnew as possible and
that is already known to be NP-Complete.
12 Steps
Step 2: What is what
Pnew = Independent Set
Pis NP-comp = Clique
Inew
Iis NP-comp
Snew
k=3
Sis NP-comp
k=4
12 Steps
Step 3: Direction of reduction & Code
Reduce Pnew to Pis NP-comp or Pis NP-comp to Pnew?
Pis NP-comp ≤poly Pnew
Palg ≤poly Poracle
12 Steps
Step 3: Direction of reduction & Code
12 Steps
Step 4: Look for similarities
Clique
Independent Set
Both instances are graphs: nodes & edges
A clique solution is a subset of the nodes with edges.
An Ind Set solution is a subset of the nodes without edges.
12 Steps
Step 5: Instance Map
12 Steps
Step 5: Instance Map
Ialg
Ioracle = InstanceMap(Ialg)
12 Steps
Step 5: Instance Map
Ialg
no yes
Yes mapped to Yes
No to No
Ioracle
no yes
12 Steps
Step 6: Solution Map
12 Steps
Step 6: Solution Map
Ialg
Potential Solution
= SolutionMap(S).
Ioracle
Potential Solution S.
12 Steps
Step 7: Valid to Valid
Ialg
Valid
Salg = SolutionMap(Soracle)
Ioracle = InstanceMap(Ialg)
If Soracle is valid
solution for Ioracle,
then Salg is valid
solution for Ialg
Valid
Soracle
12 Steps
Step 8: Reverse Solution Map
Ialg
Potential Solution S.
Ioracle
Not part of code,
but of proof.
Potential Solution
= ReverseSolutionMap(S).
12 Steps
Step 9: Valid to Valid
Ialg
Valid
Salg
Ioracle = InstanceMap(Ialg)
If Salg is valid
solution for Ialg,
then Soracle is valid
solution for Ioracle
Valid
Soracle = ReverseSolutionMap(Salg)
12 Steps
Step 10: Working Algorithm
Our Algalg says yes to Ialg
iff old Algoracle says yes to Ioracle
iff Ioracle has a valid solution Soracle
iff Ialg has a valid solution Salg
iff Ialg is a yes instance.
6&7
Steps 8&9
12 Steps
Step 11: Running Time
Algalg(Ialg) is poly time if
the following are poly time:
Algoracle(Ioracle)
Ioracle = InstanceMap(Ialg)
Salg = SolutionMap(Soracle)
Assumed
Your job
Palg ≤poly Poracle
or Palg
Poracle
Scheduling
Graph
Colouring
Clique
Independent Set
Circuit
Satisfiability
Any NP-Problem
This will prove that
Cir-SAT is NP-Complete.
Reduction
Any NP-problem ≤poly Cir-SAT
I
I don’t even know what
problem I am trying to solve!!!
?
BUILD:
Oracle for
arbitrary
NP Problem
GIVEN:
Oracle for
Cir-Sat
We need to solve some unknown NP-Problem.
What do we know about it?
Formal definition:
Parbitrary  NP iff  poly time Valid
such that Parbitrary(I) =  S Valid(I,S)
We have a poly-time alg Valid(I,S) to test
whether or not S is a valid solution for I.
Valid
k=4
Reduction
Any NP-problem ≤poly Cir-SAT
I
Please, give me
a solution S
such that
Valid(I,S) is true.
BUILD:
Oracle for
arbitrary
NP Problem
That looks like
a Turing Machine.
I only know about
circuits
GIVEN:
Oracle for
Cir-Sat
The Circuit Satisfiability Problem
An instance is a circuit C.
A solution is an assignment X = [F,T,F…].
C(X) evaluates to T or F.
x1 F
x2T
x3F
AND
AND
F
F
OR
F
OR
NOT
F
OR
T
One bit output
No feedback
The Circuit Satisfiability Problem
An instance is a circuit C.
A solution is an assignment X = [F,T,F…].
A valid solution has C(X) = True.
x1 F
x2F
x3F
AND
AND
F
F
OR
F
OR
NOT
T
OR
F
Given a circuit,
does it have a
satisfying assignment?
The Circuit Satisfiability Problem
Turing (and friends) prove that
any algorithm computed by a
JAVA program in poly-time
can be computed by a
Turing Machine in poly-time.
Very Powerful
Cook proves that any algorithm
computed by a Turing Machine
in time T(n) can be computed
by a family of circuits of size [T(n)]2.
But clearly, circuits compute.
Reduction
Any NP-problem ≤poly Cir-SAT
I build a circuit VI(S)
equivalent to Valid(I,S)
I
Please, give me
a solution S
such that
VI(S) is true.
BUILD:
Oracle for
arbitrary
NP Problem
Thanks for the circuit.
But what is S?
I can only find
assignments.
GIVEN:
Oracle for
Cir-Sat
Reduction
Any NP-problem ≤poly Cir-SAT
I
I build a circuit VI(S)
I decode X equivalent
into S
to Valid(I,S)
and am done.
Please, give me
an assignment X
such that
VI(X) is true.
BUILD:
Oracle for
arbitrary
NP Problem
My pleasure.
Here: X
GIVEN:
Oracle for
Cir-Sat
12 Step Program
Let's be more formal.
And flow the 12 steps.
Step 0: Cir-SAT  NP
Design poly-time algorithm ValidCir-SAT(C,X)
which determines whether a given assignment X
is valid for a given circuit C.
Easy: Evaluate C(X).
Step 1: What to reduce it to
Palg ≤poly Cir-Sat
Choose a problem Pis NP-comp
that is as similar to Cir-Sat as possible and
that is already known to be NP-Complete.
Have none.
Any NP-problem ≤poly Cir-SAT
We have a poly-time alg Valid(I,S) to test
whether or not S is a valid solution for I.
Step 2: What is what
Pnew
= Circuit-Sat
Inew
x1 F
AND
Snew
x2F
OR
NOT
OR
OR
Iarbitrary = I
x3F
AND
Parbitrary = some NP problem
Sarbitrary = S
Step 3: Direction of reduction & Code
Given oracle for Cir-Sat,
we need to be able to solve any NP-Complete problem.
Parbitrary
Cir-Sat
Step 4: Look for similarities
x1F
AND
x2F
x3F
AND
S
I
OR
NOT
OR
?
OR
Step 5: Instance Map
Step 5: Instance Map
I  circuit
We have a poly-time alg Valid(I,S) to test
whether or not S is a valid solution for I.
Let Validn(I,S) be a circuit:
•I is a bit string representing an instance I.
•S is a bit string representing a solution S.
•Outputs T if S is an encoding of a valid solution of I.
Eg: Clique
G,k
I
S
{u.v}  E OR uS OR vS
|S|  k
And over pairs of nodes
Step 5: Instance Map
I  circuit
Given an instance I
Circuit VI(S) = Valid(I,S)
Valid
I
(I,S)
nV
I(S)
S
I
hard wired
Outputs T if
S is an encoding of
a valid solution S of I
Step 6: Solution Map
Step 6: Solution Map
S  assignment
X=[T,F,F,T,F,T]
X is viewed as a bit string S representing a solution S.
If X is not a bit string
representing a solution
then “what ever”
I
VI(S)
S
I
hard wired
Outputs T if
S is an encoding of
a valid solution S of I
Step 6: Solution Map
S  assignment
X=[T,F,F,T,F,T]
X is viewed as a bit string S representing a solution S.
solution S
I
VI(S)
S
I
hard wired
Outputs T if
S is an encoding of
a valid solution S of I
Step 7: Valid  Valid
VI(X) = T
Valid(I,S) = Validn(I,S) = VI(X) = T
S is a valid solution of I.
I
VI(S)
S
I
hard wired
Outputs T if
S is an encoding of
a valid solution S of I
Step 8: Rev. Sol. Map
solution  assignment
solution S
S bit string representing a solution S
X=[T,F,F,T,F,T]
I
VI(S)
S
I
hard wired
Outputs T if
S is an encoding of
a valid solution S of I
Step 9: Valid  Valid
S is a valid solution of I.
VI(X) = Validn(I,S) = Valid(I,S) = T
VI(X) = T
I
VI(S)
S
I
hard wired
Outputs T if
S is an encoding of
a valid solution S of I
Reduction
Any NP-problem ≤poly Cir-SAT
I Yes/No
i.e.  S, S is a valid solution for I
VI
BUILD:
Opt.
problem
satisfiable
or not
i.e.  X, VI(X)
GIVEN:
Alg for
circuit
problem
Palg ≤poly Poracle
or Palg
Poracle
Scheduling
Clique
3-Col
?
Graph
Colouring
Independent Set
Circuit
Satisfiability
Any NP-Problem
3-Col
Graph Col: Given graph G & k
can G be coloured with k colours?
3-Col:
Given graph G
can G be coloured with 3 colours?
3-Col ≤poly Graph Col
If you can decide whether you can colour with k colours,
then you can decide whether you can colour with 3 colours.
G,k
C
G
C
Palg ≤poly Poracle
or Palg
Poracle
3-Col
Scheduling
Graph
Colouring
Clique
Independent Set
3-SAT
?
Circuit
Satisfiability
Any NP-Problem
3-SAT
Cir Sat: Given circuit C
does it have a satisfying assignment X?
3-SAT: Given an expression:
•A circuit consisting of a big AND of clauses
•Each clause is the OR of at most 3 literals
•Each literal is a variable or its negation.
does it have a satisfying assignment X?
FT F F T F
FT F F T F
xoryorz AND xorwora AND …
3-SAT
Cir Sat: Given circuit C
does it have a satisfying assignment X?
3-SAT: Given an expression:
does it have a satisfying assignment X?
3-Sat ≤poly Cir Sat
If you can decide whether any circuit is satifiable,
then you can decide whether a circuit that happens to be
an expression is satisfiable.
FT F F T F
FT F F T F
xoryorz AND xorwora AND …
Palg ≤poly Poracle
or Palg
Poracle
3-Col
Scheduling
Graph
Colouring
Clique
Independent Set
3-SAT
?
Circuit
Satisfiability
Any NP-Problem
3-Col ≤poly 3-SAT

Graph Col ≤poly Cir SAT
Step 5: Instance Map
Graph G  Expression V
Given an instance G
Circuit VG(C) = Valid(G,C)
Encoding C of C:
u is a node
r is a colour
x<u,r> = T
if node u is colour r.
G
C
hard wired
Outputs T if
C is an encoding of
a valid 3-colouring C of G
Step 5: Instance Map
Graph G  Expression V
Given an instance G
Circuit VG(C) = Valid(G,C)
G
hard wired
C
Encoding C of C:
u is a node
x<u, r> = F OR x<v, r> = F
r is a colour
x<u,r> = T
if node u is colour r. And over edges <u,v> and colours r
clauses with 2 literals
Step 5: Instance Map
Graph G  Expression V
Given an instance G
Circuit VG(C) = Valid(G,C)
G
hard wired
C
Encoding C of C:
u is a node
x<u, r> =T OR x<u, g>=T OR x<u, b>=T
r is a colour
x<u,r> = T
And over node u
if node u is colour r.
clauses with 3 literals
Reduction
3-Col ≤poly 3-SAT
I Yes/No
i.e.  C, C is a valid 3-colouring for G
VG
BUILD:
Opt.
problem
satisfiable
or not
i.e.  X, VG(X)
GIVEN:
Alg for
circuit
problem
Palg ≤poly Poracle
or Palg
Poracle
3-Col
3-SAT
?
Scheduling
Graph
Colouring
Clique
Independent Set
Circuit
Satisfiability
Any NP-Problem
This will prove that 3-Col is NP-Complete.
And so are Scheduling,
Graph-Colouring, and
3-SAT.
Sat ≤poly 3-Col
Computable
Given a circuit,
find a satisfying
assignment.
Exp
NP
SAT
complete
Poly
Steve Cook 1971
SAT is NP-Complete!
Known
GCD
Sat ≤poly 3-Col
Given a circuit,
find a satisfying
assignment.

Colour each node
with 3 colours.
Nodes with lines
between them must
have different colours.
Two problems that are cosmetically different,
but substantially the same
Circuit Satisfiablity ≤poly 3-Col
This will prove that 3-Col is NP-Complete.
Step 0: 3-Col  NP
Design poly-time algorithm Valid3-Col(G,C)
which determines whether a given 3-colouring C
is valid for a given graph G.
Easy: Check that each edge is bi-chromatic.
Step 1: What to reduce it to
Palg ≤poly 3-Col
Choose a problem Pis NP-comp
that is as similar to 3-Col as possible and
that is already known to be NP-Complete.
Pis NP-comp = Cir-SAT
Step 2: What is What
3-Colouring
•instance = graph
•solution = green/red/blue
to each node.
Circuit Problem:
•instance = circuit
of gates
•solution = true/false
to each variable
FT
G
T
x y
z
OR
NOT
Col
OR
Output
Step 3: Direction of reduction & Code
Given an oracle for
3-colorability, how can you
quickly solve circuit SAT?
BUILD:
Alg for
circuit
problem
GIVEN:
Alg for
3-colour
problem
Reduction
Circuit ≤poly 3-Colouring
C
satisfiable Graph composed of
or not
gadgets that mimic the
gates in C
BUILD:
Alg for
circuit
problem
colourable
or not
GIVEN:
Alg for
3-colour
problem
Step 4: Look for Similarities
Circuit Problem:
•instance = circuit
of gates
•solution = true/false
to each variable
•true/false
to internal wires
FT
3-Colouring
•instance = graph
of gadgets
•solution = green/red/blue
?
to each node.
T
x y
z
OR
NOT
G
OR
Output
Col
A Graph Named
“Gadget”
Steven Rudich had computer
search for graph with some key
properties.
Graph Colouring
Colour each node.
Nodes with lines
between them must
have different
colours.
No
Could you do it with
2 colours?
Graph Colouring
Colour each node.
Nodes with lines
between them must
have different
colours.
Yes.
Could you do it
with 3 colours?
How do we know?
Because we colored it!
Y
X
Output
x y
OR
Output
variables&wires
gate ~ gadget
~ nodes
T
F
Y
X
Output
FT
x y
OR
Output
T
true ~ green red
false ~ red
yellow
? ~ yellow green
Consider your
colouring
T
F
Y
X
Output
FT
x y
OR
Output
T
true ~ green red
false ~ red
yellow
? ~ yellow green
Re-map your
colouring
F
T
Y
X
?
Output
FT
x y
OR
Output
T
gate function
~ gadget “function”
X
Y
Or
F
F
F
F
T
T
T
F
T
T
T
T
T
F
Consider your
colouring
Y
X
F
F
F
?
Output
F F
x y
OR
Output
F
gate function
~ gadget “function”
X
Y
Out
F
F
F
F
T
T
F
T
T
T
F
Y
X
T
F
T
Output
FT
x y
OR
Output
T
Consider your
colouring
gate function
~ gadget “function”
X
Y
F
F
F
T
T
F
T
T
Out
T
T
F
Y
X
F
T
T
T F
Output
x y
OR
Output
T
Consider your
colouring
gate function
~ gadget “function”
X
Y
F
F
F
T
T
F
T
T
Out
T
T
F
Y
X
T
T
T
Output
TT
x y
OR
Output
T
Consider your
colouring
gate function
~ gadget “function”
X
Y
F
F
F
T
T
F
T
T
Out
T
T
F
Consider your
colouring
Y
X
Or - Gadget
Output
FT
x y
OR
Output
T
gate function
~ gadget “function”
X
Y
Or
F
F
F
F
T
T
T
F
T
T
T
T
T
F
Proof
Y
X
T
F
Or - Gadget
T
Output
FT
x y
OR
Output
T
gate function
~ gadget “function”
X
Y
F
F
F
T
T
F
T
T
Or
T
T
F
Proof
Y
X
F
F
Or - Gadget
F
Output
F F
x y
OR
Output
F
gate function
~ gadget “function”
X
Y
Out
Or
F
F
F
F
T
T
F
T
T
T
F
X
NOT - Gadget
Output
X
NOT
F
T
T
F
Reduction
Circuit ≤poly 3-Colouring
C
satisfiable Graph composed of
or not
gadgets that mimic the
gates in C
BUILD:
Alg for
circuit
problem
colourable
or not
GIVEN:
Alg for
3-colour
problem
Step 5: Instance Map
Step 5: Instance Map
x y
z
OR
NOT
OR
Output
circuit  graph
Step 5: Instance Map
x y
z
OR
NOT
circuit  graph
y
x
OR
Output
variables&wires
~ nodes
Output
z
Step 5: Instance Map
x y
z
OR
NOT
circuit  graph
y
x
OR
Output
One ~pallet
for
gate
gadget
“defining” truth.
Output
z
Step 5: Instance Map
x y
z
circuit  graph
y
x
second input
first input
OR
NOT
The Graph Named
“OR-Gadget”
OR
output
Output
gate ~ gadget
Output
z
Step 5: Instance Map
x y
z
circuit  graph
y
x
second input
first input
OR
NOT
OR
output
Output
gate ~ gadget
Output
z
Step 5: Instance Map
x y
z
circuit  graph
y
x
z
input
OR
NOT
The Graph Named
“NOT-Gadget”
OR
output
Output
gate ~ gadget
Output
Step 5: Instance Map
x y
z
circuit  graph
y
x
z
input
OR
NOT
OR
output
Output
gate ~ gadget
Output
Step 5: Instance Map
x y
z
OR
NOT
circuit  graph
y
x
z
OR
first input
Output
gate ~ gadget
second input
The Graph Named
“OR-Gadget”
Output
output
Step 5: Instance Map
x y
z
OR
NOT
circuit  graph
y
x
z
OR
first input
Output
gate ~ gadget
Output
output
second input
Step 5: Instance Map
x y
z
OR
NOT
OR
Output
One extra edge
from “False”
to “Output”
circuit  graph
Step 5: Instance Map
x y
z
OR
NOT
OR
Output
Done
circuit  graph
Step 6: Solution Map
assignment  colouring
F
T
F F
T
x y
z
OR
NOT
OR
x
Assume these are
the colours
of these nodes.
Output
Output
y
z
Step 7: Valid  Valid
F F
T
x y
z
OR
NOT
correspond
F
T
x
OR Gadget
F
correspond
OR
Output
Output
y
z
Step 7: Valid  Valid
F F
correspond
T
T
F
x
x y
z
OR
NOT
y
z
Not Gadget
F
F
correspond
OR
Output
Output
Step 7: Valid  Valid
T
F F
T
x
x y
z
OR
NOT
F
F
F
correspond
OR
Output F
OR Gadget
correspond
Output
y
z
Step 7: Valid  Valid
T
F F
T
F
x
x y
z
OR
NOT
y
z
F
F
OR
Output F
One extra edge
from “False”
to “Output”
Output
Oops this is
a bad coloring
Step 6: Solution Map
assignment  colouring
F
T
F T
T
x y
z
OR
NOT
OR
x
Assume these are
the colours
of these nodes.
Output
Output
y
z
Step 7: Valid  Valid
F T
correspond
T
T
F
x
x y
z
OR
NOT
F
T
OR
correspond
correspond
Output T
correspond
Output
y
z
Step 7: Valid  Valid
T
F T
T
F
x
x y
z
OR
NOT
y
z
F
T
OR
Output T
One extra edge
from “False”
to “Output”
Output T
Can’t be F
Step 7: Valid  Valid
T
F T
T
F
x
x y
z
OR
NOT
F
T
OR
Output T Must be T
valid colouring
correspond

Output T
satisfying assignment
y
z
Step 8: Rev. Sol. Map
F T
T
x y
z
OR
NOT
assignment  colouring
y
x
OR
Output
Rest of colouring
follows
Output
z
Step 9: Valid  Valid
F T
T
x y
z
OR
NOT
y
x
OR
Output
Edges in Gadgets
bi-chromatic
Output
z
Step 9: Valid  Valid
F T
T
y
x
x y
z
OR
NOT
z
OR
Output T valid
correspond
Edge?
Extra Edge
bi-chromatic
Output
Must be T
Step 9: Valid  Valid
F T
T
x y
z
OR
NOT
y
x
OR
Output T valid
satisfying assignment

valid colouring
Output
z
Step 5-9
F T
T
x
x y
z
OR
NOT
F
T
OR
Output T
Satisfiability of this circuit
=
Output
3-colourability of this graph
y
z
Satisfiability of this circuit
=
3-colourability of this graph
Reduction
Circuit ≤poly 3-Colouring
C
satisfiable Graph composed of
or not
gadgets that mimic the
gates in C
BUILD:
Alg for
circuit
problem
colourable
or not
GIVEN:
Alg for
3-colour
problem
Palg ≤poly Poracle
or Palg
Poracle
3-Col
Scheduling
Graph
Colouring
Clique
Independent Set
3-SAT
Circuit
Satisfiability
Any NP-Problem
This proves that 3-Col is NP-Complete.
And so are Scheduling,
Graph-Colouring, and
3-SAT.
Palg ≤poly Poracle
or Palg
Poracle
3-Col
Scheduling
Clique
Graph
Colouring
?
Independent Set
3-SAT
Circuit
Satisfiability
Any NP-Problem
True but we will not do this here.
Palg ≤poly Poracle
or Palg
Poracle
3-Col
Scheduling
Graph
Colouring
Clique
Independent Set
3-SAT
Circuit
Satisfiability
Any NP-Problem
NP-Complete Problems
If we can solve one of these quickly,
then we can solve all NP-problems quickly.
Palg ≤poly Poracle
or Palg
Poracle
3-Col
Scheduling
Graph
Colouring
Clique
Independent Set
3-SAT
Circuit
Satisfiability
Any NP-Problem
NP-Complete Problems
If we can prove one takes exponential time,
then all NP-complete problems take exponential time.
3-Col
Scheduling
Graph
Colouring
Clique
Independent Set
3-SAT
Circuit
Satisfiability
Any NP-Problem
NP-Complete Problems
?
Network Flows
Yes: If we can solve Sat fast,
we can solve Network Flow fast,
because we can solve Network Flow fast.
3-Col
Scheduling
Graph
Colouring
Clique
Independent Set
3-SAT
Circuit
Satisfiability
Any NP-Problem
NP-Complete Problems
?
Network Flows
No: We can solve Network Flow fast,
but we cannot solve Sat fast.
3-Col
3-SAT
Scheduling
Graph
Colouring
Clique
Independent Set
Circuit
Satisfiability
Any NP-Problem
NP-Complete Problems
Network Flows
?
Bipartite Matching
Yes: We use the fast alg
for Network Flow
to give a fast alg
for Bipartite Matching.
3-Col
Scheduling
Graph
Colouring
Clique
Independent Set
NP-Complete Problems
3-SAT
Circuit
Satisfiability
Any NP-Problem
?
Math Truth
Network Flows
Bipartite Matching
Halting problem poly Math Truth
 Math Truth  Undecidable.
3-Col
Scheduling
Graph
Colouring
Clique
Independent Set
3-SAT
Circuit
Satisfiability
Any NP-Problem
NP-Complete Problems
Network Flows
Bipartite Matching
Yes: If we can solve
Math Truth fast,
we can solve Sat fast.
?
Math Truth
Undecidable
12 Steps
In More Detail
Steps for proving that Pnew is NP-Complete
•Designed after collecting lots of wrong answers.
•Understand and follow them carefully.
•Keep the steps separate, because errors happen when they
are intertwined.
12 Steps
In More Detail
Steps for proving that Pnew is NP-Complete
1) Pnew  NP
2) What is what
3) Direction of reduction & Code
4) Look for similarities
5) Instance Map
6) Solution Map
7) Valid to Valid
8) Reverse Solution Map
9) Valid to Valid
10) Working Algorithm
11) Running Time
12 Steps
Step 3: Direction of reduction & Code
Reduce Pnew to Pis NP-comp or Pis NP-comp to Pnew?
People get this wrong.
Don’t memorize it. Go through it each time.
•I want to prove Pnew is hard
•hence that Pis NP-comp poly Pnew
•hence that Pis NP-comp is easy
•hence I need to design an algorithm for Pis NP-comp
•hence I start with an input instance Iis NP-comp for Pis NP-comp
•to get help, I must map it to an instance Inew for Pnew
Pis NP-comp ≤poly Pnew
Palg ≤poly Poracle
12 Steps
If and Only If
Palg ≤poly Poracle
Ialg
Ioracle
Salg
Soracle
•Step 5
Ioracle = InstanceMap(Ialg)
•Steps 6,7: Salg = SolutionMap(Soracle)
•Steps 8,9: Soracle = ReverseSolutionMap(Salg)
Not part of code,
but of proof.
12 Steps
If and Only If
Algoracle gives the correct answer (whether yes or no)
 Our algorithm gives the correct answer.
Algoracle says yes to Ioracle
 Ioracle has a valid solution Soracle
 Ialg has a valid solution Salg
 Ialg is a yes instance.
Steps 6&7
Algoracle says no to Ioracle
 Ioracle does not have a valid solution Soracle
 Ialg does not have a valid solution Salg
 Ialg is a no instance.
Take to contra positive.
No witness
to map!
12 Steps
If and Only If
Algoracle gives the correct answer (whether yes or no)
 Our algorithm gives the correct answer.
Algoracle says yes to Ioracle
 Ioracle has a valid solution Soracle
 Ialg has a valid solution Salg
 Ialg is a yes instance.
Steps 6&7
Algoracle says yes to Ioracle
 Ioracle has a valid solution Soracle
 Ialg has a valid solution Salg
 Ialg is a yes instance.
Steps 8&9
Take to contra positive.
12 Steps
Wider net
Ialg
Salg = SolutionMap(Soracle)
Ioracle = InstanceMap(Ialg)
Every solution Soracle
needs to be considered.
Not just those you
intended when you
created I oracle.
Soracle
12 Steps
Wider net
Ialg
Potential Solution
= SolutionMap(S).
Ioracle = InstanceMap(Ialg)
To be safe,
cast a bigger net.
Potential Solution S.
12 Steps
Wider net and Separate steps 6 & 7
It is best to separate the steps of
•defining the entire solution Salg being mapped to
in a well defined and clear way.
•the proof that Salg is a valid solution.
A false proof
•defines part of Salg
•proves that this makes some aspect of Salg valid.
•Later to make some other aspect of Salg valid,
•it redefines the same part of Salg in an inconsistent way.
12 Steps
Step 5: Instance Map
12 Steps
Step 5: Instance Map
Ialg
no yes
Yes mapped to Yes
No to No
Ioracle
no yes
ok mapped
to twice
ok not mapped to
12 Steps
Step 5: Instance Map
Ialg
no yes
InstanceMap
does not know if
Ialg is a Yes
instance.
Ioracle
no yes
12 Steps
Step 5: Instance Map
Ialg
no yes
InstanceMap
does not know if
Ialg is a Yes
instance.
Ioracle
no yes
A useful mapping
but not poly-time.
12 Steps
Forward vs Reverse Solution Map
Ialg
Potential Solution
= SolutionMap(S).
Ioracle
Potential Solution S.
12 Steps
Forward vs Reverse Solution Map
Ialg
Potential Solution S.
Ioracle
Potential Solution
= ReverseSolutionMap(S).
12 Steps
Forward vs Reverse Solution Map
Salg
Salg = SolutionMap(Soracle)
Soracle = ReverseSolutionMap(Salg)
In this case, bijective
Soracle
12 Steps
Forward vs Reverse Solution Map
Salg
Salg = SolutionMap(Soracle)
Soracle = ReverseSolutionMap(Salg)
ok not mapped to
ok mapped
Even the unexpected solutions
to twice
need to be mapped.
Soracle
The End
An early version of these slides
produced by
Steven Rudich
from Carnegie Mellon University
Things Cut Out
of More Detailed NP Talk
Computational Complexity Theory
Computational Complexity Theory is
the study of how much of a given
resource (such as time, space,
parallelism, randomness, algebraic
operations, communication, or quantum
steps) is required to perform the
computations that interest us the most.
2 CRAYOLAS
Given a graph G, what is a
fast algorithm to decide
if it can be 2-colored?
PERSPIRATION; BRUTE FORCE:
Try out all 2n ways of 2 coloring G.
Sorry
Things Cut Out
of intro theory talk
Boss assigns task:
Given a computer program, will it halt?
Everyday industry asks these questions.
Example: Halting Problem
loop a = 1 … 1,000,000
s := s+a
end loop
loop a > 0
s := s+a
end loop
Halts
Does not halt
Example: Halting Problem
loop a,b,c,z > 2
Proof of not halting =
exit when az + bz = cz Proof of
Fermat’s Last Theorem
end loop
loop
chaotic behavior
exit when
special event
end loop
Chaotic behavior =
not knowing what it will do
without doing it.
How can we know it will
not halt without running it
forever?
Your answer:
Sorry. No algorithm exists that tells
you for every computer program
whether it halts!
Halting Problem
Suppose there was a working algorithm:
Halt(P,I) = whether program P halts on input I
Construct from it another algorithm:
Nhalt(P) = if(Halt(P,P)) loop forever else halt
= halts ↔ program P does not halt on input P.
Paradox: Nhalt(Nhalt) = ?
program Nhalt halts on input Nhalt
↔
program Nhalt does not halt on input Nhalt
Contradiction!
Your answer:
Sorry. No algorithm exists that tells
you for every computer program
whether it halts!
Or any other useful thing
about what the program
does.
Research in Computer Science
•Vision
•Graphics
•Robotics
•Systems
•Data Bases
•Networks
•Real Time
•Artificial Intelligence
•Theory
Theoretical
Computer Science
Useful
Fun
Doable by you
So you want to be a computer
scientist?
•Micro soft packages.
•Programming
It is important to
learn theory!
Boss assigns task:
Given courses students want,
schedule them to minimize conflicts.
Everyday industry asks these questions.
Your answer:
Um? Tell me what to code.
The demand for mundane programmers is diminishing.
Your answer:
I learned this great algorithm that will
work.
Soon all known algorithms
will be available in libraries.
Your answer:
I can develop a new algorithm for you.
Great thinkers
will always be needed.
Alternatively, your answer:
Sorry. Very likely no fast algorithm
exists that always finds best
schedule.
(else worthy of wealth & Nobel prize)
This too will
impress your boss.
The future belongs to the
computer scientist who has
A large collection of tools to draw
on.
Rudich www.discretemath.com
The future belongs to the
computer scientist who has
Creativity
The future belongs to the
computer scientist who has
Logical
Thinking
The future belongs to the
computer scientist who has
Abstract
thinking.
The future belongs to the
computer scientist who has
It is important to
learn theory!
Andy Mirzaian
Suprakash Datta
Franck Van Breugel
Jeff Edmonds
Patrick Dymond
Zbigniew Stachniak
Joseph Liu
Theoretical
Computer Science
George Tourlakis
Research
Eric Ruppert
Theoretical Computer Science at
York
•Complexity Theory
•Algorithms for solving problem
•Determining time and memory used
•Proving can’t do better
Theoretical Computer Science at
York
•Many machines working together
•Different speed clocks
•Different memory and programs
•Communication Networks
•Parallel Algorithms
Theoretical Computer Science at
York
•Other Topics
•Large-scale scientific computing
•Sparse matrix technology
•Computational geometry
•Combinatorial optimization
•Graph algorithms
•Computational logic,
•Knowledge representation
Some Typical Ideas in
Theoretical Computer Science
Computer Science
Started with Theory
The theory of computation and
algorithms was developed well before the
hardware was invented.
This is one of the reasons that
progress was made so fast when the
hardware was developed.
Please feel free to ask questions!
Professors at York
are accessible.
A Hotdog
A combination of pork,
grain, and sawdust, …
Constraints:
•Amount of moisture
•Amount of protean,
•…
The Hotdog Problem
Given today’s prices,
what is a fast algorithm
to find the cheapest hotdog?
Abstraction
There are deep ideas within the simplicity.
=
Goal: Understand and think about complex
things in simple ways.
There are deep ideas within the simplicity.
Rudich www.discretemath.com
Abstract Out Essential Details
Amount to add: x1, x2, x3, x4
Cost: $29, $8, $1, $2
Cost of Hotdog:
Constraints:
•moisture
•protean,
•…
29x1 + 8x2 + 1x3 + 2x4
3x1 + 4x2 – 7x3 + 8x4  12
2x1 - 8x2 + 4x3 - 3x4  24
-8x1 + 2x2 – 3x3 - 9x4  8
x1 + 2x2 + 9x3 - 3x4  31
Abstract Out Essential Details
Minimize:
Subject to:
29x1 + 8x2 + 1x3 + 2x4
3x1 + 4x2 – 7x3 + 8x4  12
2x1 - 8x2 + 4x3 - 3x4  24
-8x1 + 2x2 – 3x3 - 9x4  8
x1 + 2x2 + 9x3 - 3x4  31
A Fast Algorithm
For decades people thought that there
was no fast algorithm.

Minimize:
Subject to:
Then one was found!
29x1 + 8x2 + 1x3 + 2x4
3x1 + 4x2 – 7x3 + 8x4  12
2x1 - 8x2 + 4x3 - 3x4  24
-8x1 + 2x2 – 3x3 - 9x4  8
x1 + 2x2 + 9x3 - 3x4  31
Theoretical Computer Science
finds new algorithms every day.
Doable by You
How can I solve problems that smart people have
worked hard on?
Doable by You
•We will teach you how to do it.
•You will come upon new
problems that no one has
thought about before.
•Hence, you will have the first
chance to solve them.
Theoretical Computer Science
Think
Be Creative
Arithmetic Sum
Gauss
S=1+2+3+...+n
Rudich www.discretemath.com
1
+
2
+
3
+
...
+
n-1 +
n
=
S
= number of white dots.
1
2........n
1
+
2
+
n
+
n-1 +
+
...
+
n-1 +
n
=
S
= number of white dots
n-2 +
...
+
2
1
=
S
= number of yellow dots
3
n ....... 2 1
+
1
2........n
1
+
2
+
n
+
n-1 +
+
...
+
n-1 +
n
=
S
= number of white dots
n-2 +
...
+
2
1
=
S
= number of yellow dots
3
+
n
2S dots
= n(n+1) dots in
the grid
n
n
n
n
n (n  1)
S
2
n
n+1 n+1 n+1
n+1 n+1
Perspiration Search
Try all possible combinations
to find the cheapest
Too many to try.
With 50 ingredients,
number of valid combinations is
more than the number of atoms.