The Halting Problem
• Can we design a program that, given any other
program and its input, tells whether that
program will halt when run on that input?
•
Output of this program, called HALT, when applied to
program P and input I, is “halts” if P(I) eventually
halts
•
Output of this program is “never halts” if P(I) never
halts
• Theorem (Alan Turing 1937):
No computer program can
solve the halting problem.
Proof by Contradiction
• Assume the hypothesis to be proven is false, i.e.
there is a computer program that can solve the
halting problem
• Show that this assumption leads to a
contradiction
• Do it by creating a program and an input to it
that generate this contradiction
Proof by Contradiction
• Suppose there was a computer program,
called HALT, which solved the halting problem.
• We can write a new program that uses HALT and
leads to a contradiction:
Program NonConformist (Program P)
If ( HALT(P) = “never halts” ) Then
Halt
Else
Do While (1 > 0)
Print “Hello!”
End While
End If
End Program
Proof by Contradiction
Program NonConformist (Program P)
If ( HALT(P) = “never halts” ) Then
Halt
Else
Do While (1 > 0)
Print “Hello!”
End While
End If
End Program
• Does NonConformist(NonConformist) halt?
• Note: It calls HALT(NonConformist)
• Yes? Means HALT(NonConformist) = never halts
• No? That means HALT(NonConformist) = halts
• Contradiction: There exists a program
(NonConformist) for which the HALT program
gives the wrong answer
Undecidability
• Halting Problem:
Given a computer program P and input “input”:
• Output “halts” if P(input) eventually halts
• Output “never halts” if P(input) never halts
• We’ve shown that the Halting Problem is
Undecidable – no computer program can ever
solve it, no matter how powerful the
computer is.
Undecidability
• Many other problems are undecidable, too.
• Moreover, the same ideas that we used to
prove undecidability can be used to prove a
very disturbing statement.
GÖDEL’S
INCOMPLETENESS THEOREM
• Until Gödel came along, Mathematicians
were searching for the “one true logical
framework,” i.e. a framework in which
every true statement can be proved.
• Their faith was shattered by Gödel.
• For more on this and its philosophical
implications, see Gödel, Escher, Bach by
Douglas Hofstadter.
GÖDEL’S
INCOMPLETENESS THEOREM
• In 1931, Gödel stunned the world by proving
that in any logical framework in which you
can express basic facts about numbers,
there exists a true statement that cannot
be proved in that framework.
• “Not every true statement has a proof”
GÖDEL’S
INCOMPLETENESS THEOREM
• In 1931, Gödel stunned the world by proving
that in any logical framework in which you
can express basic facts about numbers,
there exists a true statement that cannot
be proved in that framework.
• But wait, if it can’t be proved, how do we
know it is true?
GÖDEL’S
INCOMPLETENESS THEOREM
• We can prove that it is true by “jumping”
outside the original logical framework to a
“larger” logical framework…
GÖDEL’S
INCOMPLETENESS THEOREM
• We can prove that it is true by “jumping”
outside the original logical framework to a
“larger” logical framework…
• But that larger logical framework also has
a true statement that cannot be proved!
GÖDEL’S
INCOMPLETENESS THEOREM
• We can prove that it is true by “jumping”
outside the original logical framework to a
“larger” logical framework…
• But that larger logical framework also has
a true statement that cannot be proved!
• …and so on, and so on…
• No matter when we stop, there will always
be some true statement that doesn’t have
a proof!
GÖDEL’S
INCOMPLETENESS THEOREM
• Until Gödel came along, Mathematicians
were searching for the “one true logical
framework,” i.e. a framework in which
every true statement can be proved.
• Their faith was shattered by Gödel.
• For more on this and its philosophical
implications, see Gödel, Escher, Bach by
Douglas Hofstadter.
More Undecidable Problems?
Post's Correspondence Problem (PCP)
• An instance of PCP of size s is a finite
set of pairs of strings (gi , hi) [for i =
1...s; s>=1] over some alphabet .
• A solution is a sequence i1 i2 ... in of
selections from each set of strings (gi , hi)
such that the strings gi1gi2 ... gin and hi1hi2
... hin formed by concatenation are
identical.
Sample PCP
•g1 =
•g2 =
•g3 =
•g4 =
aba
bbab
baaa
a
h1 =
h2 =
h3 =
h4 =
abaa
abab
a
bb
So, 1,3,1,2 would correspond to
• aba baaa aba bbab from g’s
• abaa a abaa abab from h’s
Not a solution!
Another Sample PCP
•g1 =
•g2 =
•g3 =
•g4 =
aba
bbab
baaa
a
h1 =
h2 =
h3 =
h4 =
abaa
abab
a
bb
1,4,2,1,3 corresponds to
• aba a bbab aba baaa from g
• abaa bb abab abaa a from h
Solution!
PCP is undecidable?
•PCP shown to be undecidable by Post in
1946.
• What about PCP with limited-size inputs
• PCP with size 2 has been proved decidable.
• PCP with size 7 has been proved undecidable
• The decidablility of problems with size between 3
and 6 is still pending.
What Computers Can’t Do
In Your Lifetime
• We’ve now seen examples of problems that
computers can’t solve, even if computers have
unlimited speed and unlimited time.
• Are there more real world problems (eg that
arise in business, science, … ) that can be solved
but take far too much time to be solved in
practice?
“Search” Problems
• We’ll focus on what we’ll call “search problems”.
(In Computer Science terms, we’ll be talking
about what are termed NP problems)
• Intuitively, a search problem is a problem where
you are looking for something which you can
recognize quickly if you find it.
• Recognizing a good solution is easy
• Problem is finding it
Example: Coloring
Example: Coloring with 3 colors
• Suppose we are given a
collection of circles (nodes).
• Some circles connect to others
by edges, forming a graph
• Rule: No two connected
circles can have the same color.
• You only have three colors
(Green, Red, Yellow)
• Is there a valid coloring?
• Note: easy to check validity.
Hard to find?
Search Problems:
More Precise Definition
• Intuitively, a search problem is a problem where
you are looking for something which you can
recognize quickly if you find it.
• A bit more precisely, we require that there is a
small circuit that can quickly check the validity
of a solution.
• For coloring, it would just be a circuit that
checks that for every pair of connected circles,
the colors are different.
Another Example:
Traveling Salesperson Problem
• A saleswoman wants to visit n different cities.
• She knows the costs associated with flights
between each city.
• Can she visit all the cities spending less than $B
in total?
• Note: Easy to check that a given flight plan
visits all the cities and costs less than $B.
• Seems hard to find the flight plan …
Search Problems…
• Unfortunately, we don’t know how to solve these
and many other search problems with a
computer in our lifetimes for large inputs.
•
Large graphs, large number of cities
• For many years, computer scientists wondered
which search problems could be solved, and
which couldn’t.
• But just because computer scientists couldn’t
solve the Coloring problem for 40 years doesn’t
mean it is impossible, right?
•
Proof of Fermat’s Last Theorem took over 300 years
Search Problems…
• Can we prove that it is impossible to solve
them quickly for large inputs?
• Unfortunately, not yet
• But now we are much more confident that
they really are impossible to solve quickly
• How do we make progress toward determining
an answer?
Breakthrough: Reduction
• A surprise:
• In the early 70’s, Cook, Levin, and Karp
showed us that if we can solve the Coloring
problem quickly, then we can solve ALL search
problems quickly!
• But general search problems are defined in
terms of circuits (that can validate their
solutions), not colors…
• Need to map circuits to graph coloring problems
• Then we can map the circuit for any search
problem to the corr. graph coloring problem
Coloring and Circuits
• Suppose we think of :
• Green as meaning True.
• Red as meaning False
• Yellow as meaning nothing
What is the circuit for this
graph?
Input
T
X
So this is
a graph
for a NOT
F
Valid coloring
for this graph
leads to a
NOT gate
Output
NOT X
T
F
Y
X
Output
X
Y
F
F
F
T
T
F
T
T
T
F
Y
X
F
T
Output
X
Y
F
F
F
T
T
F
T
T
T
F
Y
X
Output
X
Y
F
F
F
T
T
F
T
T
T
F
Y
X
Output
X
Y
F
F
F
T
T
F
T
T
T
F
Y
X
Output
X
Y
F
F
F
T
T
F
T
T
T
F
Y
X
Output
X
Y
F
F
F
T
T
F
T
T
T
T
F
Y
X
Output
X
Y
F
F
F
F
T
T
T
F
T
T
T
T
T
F
Y
X
=
Output
OR
X
Y
OR
F
F
F
F
T
T
T
F
T
T
T
T
Coloring !
• And you thought coloring was for kids..
• In fact, we can encode any circuit into a
collection of connected circles (graph) waiting to
be colored.
• Any valid coloring for the graph conforms to the
circuit (which verifies a search problem)
• Thus, we can efficiently reduce any search
problem to Coloring.
• If we can solve Coloring quickly, we can solve
any search problem quickly!
Many more
• Traveling Salesperson Problem is also as “hard”
as any search problem.
• Search Problems with this property are called
complete problems
• Although we don’t know how to prove that they
are hard, we know that if find a way to solve one
of them quickly, we can solve all search problems
quickly!
• Gives us more confidence that they really are
hard.
Recap: Hard search problems
• A hard search problem is a problem where
• It is hard to find a solution
• It is easy to check possible solutions for
validity
• A complete search problem is a problem that is
as hard as any search problem
• Search problem is believed to be hard because
• No one found a way to solve any of the
complete search problems quickly
What does “Quickly” mean?
• In time polynomial in the size of the input
• E.g. if one could solve an n-city TSP in time
– n2+2n+5
– or even n2000+7n1999+…+8n2+5n+6
– Then these are a Polynomial-time algorithms (quick)
• But if one can only solve it in time:
– 2n
– or 2n+6n-1+…+3
– Then these are Exponential-time algorithms (slow)
2
n
versus
n
2
• n = 10
– n2 = 100
– 2n = 1024
• n = 100
– n2 = 10,000
– 2n = (1024)10
> 1 trillion * 1 trillion * 1 million
Hence we call exponential-time algorithms slow
Classes of search problems
• In computer-science terminology:
• NP = All Search Problems
• P = Problems we can solve quickly
• We believe that P  NP, i.e. not every search
problem can be solved quickly on a computer.
NP-Complete Problems
• Coloring is complete
• In particular, we can reduce solving any
search problem to finding a valid coloring for
some collection of circles!
• So, if we could solve Coloring quickly, then
P = NP
• That’s why we believe Coloring can’t be solved
quickly by any computer.
• We call such problems NP-Complete.