Spring 2016
CISG 5115
Theory of Computation
Chapter 3
Church-Turing Thesis
Donghyun (David) Kim
Department of Mathematics and Computer Science
North Carolina Central University
Some slides are in courtesy of Prof. Verma at University of Houston
1
Universal Models of
Computation
• Models of computation that have power equivalent to
that of an idealized general-purpose computer or,
models of computation characterize problem-solving
by humans and machines.
• how is the input (output) represented?
• how does the computational model compute?
• what is the cost (in time and space) ?
• Use models to determine what can and cannot be
computed.
2
Spring 2016 CISG 5115
Donghyun (David) Kim
Department of Mathematics and Physics
North Carolina Central University
Issues of Computability
• “What problems can be solved on a given model of
computation?”
• Halting Problem - a classical unsolvable problem:
• “Given a description of a program, decide whether the
program will finish running or continue run forever.”
• Another one: “Does the program return the correct
answer?”
3
Spring 2016 CISG 5115
Donghyun (David) Kim
Department of Mathematics and Physics
North Carolina Central University
A Contradiction Argument for
Halting Problem
• Suppose there exists an algorithm P for the Halting Problem.
•
•
boolean function P (string prog, data) {
}
P returns T (“prog" stops on “data") or F (“prog" doesn't stop)
• Now consider the following function Q.
•
boolean function Q (string x) {
if (P(x, x) == F) return F;
while (1) ;
}
• Q(x) ?
•
•
Q(x) stops iff P(x,x) returns F, which happens iff P(x,x) doesn't stop on x.
Q(x) doesn't stop iff P(x,x) returns T, which happens iff P(x,x) stops on x.
• Contradiction
•
•
P (Q , Q) returns T => Q(Q) stops => P (Q,Q) does not stop and return F.
P (Q, Q) returns F => Q(Q) does not stop => P (Q,Q) stops and return T.
Spring 2016 CISG 5115
Donghyun (David) Kim
4
Department of Mathematics and Physics
North Carolina Central University
3.1 Turing Machines
5
Spring 2016 CISG 5115
Donghyun (David) Kim
Department of Mathematics and Physics
North Carolina Central University
Turing Machine (TM)
• Proposed by Alan Turing in 1936.
• Mimic the problem-solving mechanism of a scientist
• unbounded supply of paper, pencils, erasers and thinks.
• write/lookup/modify notes
• a finite number of mental states: finite volume of brain and
discrete thought processes.
• make decision based on notes and the scientist's current
mental state
• TM: has an unlimited and unrestricted memory (an infinite
tape).
• read/write tape
• move left/right
• rejecting/accepting states take immediate effect.
• Equivalent in power with a real computer.
Spring 2016 CISG 5115
Donghyun (David) Kim
6
Department of Mathematics and Physics
North Carolina Central University
Turing Machine – cont’
Infinite tape (unlimited memory)
. . .
Bi-direction
Read/Write
Tape head
(read/write)
Finite
State
Control
• Initially, a tape contains only the input string and is
blank everywhere else.
7
Spring 2016 CISG 5115
Donghyun (David) Kim
Department of Mathematics and Physics
North Carolina Central University
Turing Machine – cont’
8
Spring 2016 CISG 5115
Donghyun (David) Kim
Department of Mathematics and Physics
North Carolina Central University
Turing Machine – cont’
• We can design TM’s for computing functions from
strings to strings
• We can design TM’s to decide languages
• using special states accept/reject or by writing Y/N on
tape.
• We can design TM’s to accept languages.
• if TM halts at an accept status (after consuming the
whole string), the string is accepted
9
Spring 2016 CISG 5115
Donghyun (David) Kim
Department of Mathematics and Physics
North Carolina Central University
Example of TM for
B  {w# w | w {0 ,1} }
*
• We want a Turing machine M to accept if its input is a
member of B and to reject otherwise.
• Our goal is to determine whether the input is a
member of B – whether the input comprises two
identical strings separated by a # symbol.
• The input is too long for you to remember it all, but
you are allowed to move back and forth over the input
and make marks on it.
10
Spring 2016 CISG 5115
Donghyun (David) Kim
Department of Mathematics and Physics
North Carolina Central University
Example of TM for
– cont’
B  {w# w | w {0 ,1}*}
• Design of M
• It makes multiple passes over the input string with the
read-write head.
• On each pass, it matches one of the characters on each
side of the #symbol.
• To keep track of which symbols have been checked
already, M crosses off each symbol as it is examined.
• If it crosses off all the symbols, that means that
everything matched successfully, and M goes into an
accept state. Otherwise, reject.
0
1
1
0
#
0
1
1
0
∎
1
1
0
∎
cross off 0
X
1
Spring 2016 CISG 5115
Donghyun (David) Kim
1
0
#
0
11
Department of Mathematics and Physics
North Carolina Central University
Example of TM for
– cont’
B  {w# w | w {0 ,1}*}
• Algorithm of M on input string w
• Zig-zag across the tape to corresponding positions on
either side of the # symbol to check whether these
positions contain the same symbol. If they do not, or if no
# is found, reject. Cross off symbols as they are checked
to keep track of which symbols correspond.
• When all symbols to the left of the # have been crossed
off, check for any remaining symbols to the right of the #.
If any symbols remain, reject; otherwise accept.
12
Spring 2016 CISG 5115
Donghyun (David) Kim
Department of Mathematics and Physics
North Carolina Central University
Example of TM for
– cont’
B  {w# w | w {0 ,1}*}
0
1
1
0
#
0
1
1
0
∎
x
1
1
0
#
0
1
1
0
∎
x
1
1
0
#
x
1
1
0
∎
x
1
1
0
#
x
1
1
0
∎
x
x
1
0
#
x
1
1
0
∎
x
x
x
∎
accept

x
x
Spring 2016 CISG 5115
Donghyun (David) Kim
x
x
#
x
13
Department of Mathematics and Physics
North Carolina Central University
Formal Definition of a TM
• A TM is a 7-tuple M  (Q, , ,  , q0 , qaccept , qreject ) where
•
•
•
•
Q: finite set of states
Σ: finite input alphabet (not containing the blank symbol)
Γ: finite tape alphabet (Σ ⊆ Γ), including the blank symbol
 : Q    Q    {L, R}
e.g.:  (q, a)  (r , b, L)
• q0  Q : start state
• qaccept  Q : the accept state
• qreject  Q : the reject state, where qaccept  qreject
14
Spring 2016 CISG 5115
Donghyun (David) Kim
Department of Mathematics and Physics
North Carolina Central University
Operations of TM
• Copy input to tape.
• Start on the leftmost symbol on the tape.
• Proceed according to the transition rules.
• (Never move off the left end of the tape.)
• Continue until it
enters the accept state,
or the reject state,
or goes on forever.
Spring 2016 CISG 5115
Donghyun (David) Kim
15
Department of Mathematics and Physics
North Carolina Central University
Configurations of TM
• Configuration: a snap shot (3 items)
• current state
• current tape contents
• current head location
• e.g. uqv, 10110 q10110
• 101100110 on the tape, q1 is current state, head is on 3rd 0
• Start configuration: q0 w
• Halting configurations (does not yield further
configurations)
• accepting configuration: accept state
• rejecting configuration: reject state
Spring 2016 CISG 5115
Donghyun (David) Kim
16
Department of Mathematics and Physics
North Carolina Central University
Language of TM
• Yield: configuration C1 yields C2 if
the TM can go from C1 to C2 in one step.
e.g. If  (q, a)  (r , b, L) , then
ubqav yields urbbv
ubqav  ubrav  ubrbv  urbbv
• A TM M accepts a string w:
M goes from the start configuration to an accepting
configuration on input w.
• The language of M (the language recognized by M):
L(M )  {w | w  * and M accepts w}
17
Spring 2016 CISG 5115
Donghyun (David) Kim
Department of Mathematics and Physics
North Carolina Central University
Turing-recognizable Languages
• Definition: A language is Turing-recognizable
(recursively enumerable, r.e.) if some Turing machine
recognizes (accepts) it, i.e.,
• A TM has three possible outcomes: accept, reject, or
not halt (loop)
• If L is regular., we can determine if w L , but not
necessarily if w L . Thus, we don't have an algorithm
for determining whether or not w L .
(not halt = either loop or still computing?)
18
Spring 2016 CISG 5115
Donghyun (David) Kim
Department of Mathematics and Physics
North Carolina Central University
Turing-decidable Languages
• Decider: TMs that halt on all inputs.
• A TM M decides a language L if M recognizes L and M
is a decider.
• Definition: A language is Turing-decidable (recursive) if
some TM decides it, i.e.,
{L :  TM M such that L  L(M) and M haltson allinputs}
• If L is recursive, then ∃ an algorithm (TM) for
determining whether or not w L .
19
Spring 2016 CISG 5115
Donghyun (David) Kim
Department of Mathematics and Physics
North Carolina Central University
Example
• The description of TM M 2 that decides A  {0 2 | n  0},
the language consisting of all strings of 0s whose
length is a power of 2.
n
• M 2 = “On input string w”
1.
2.
3.
4.
5.
Sweep left to right across the tape, crossing off (x)
every other 0
If in stage 1 the tape contained a single 0, accept
If in stage 1 the tape contained more than a single 0 and
the number of 0s was odd, reject
Return the head to the left-hand end of the tape
Go to stage 1
20
Spring 2016 CISG 5115
Donghyun (David) Kim
Department of Mathematics and Physics
North Carolina Central University
3.2 Variants of Turing
Machines
21
Spring 2016 CISG 5115
Donghyun (David) Kim
Department of Mathematics and Physics
North Carolina Central University
Turing Machine: Variations
• A number of variations of TM possible such as:
• a TM with possible “stay put” moves of the head;
•  : Q    Q    {L, R, S}
• a TM with a tape in finite in both directions;
• a TM with multiple tapes;
• a non-deterministic TM.
• It turns out that such variations have no more computation
power than the “regular” TM as we defined it (equivalent!)
– recognize the same class of languages
• To prove this, it is enough to show that there is a regular TM
simulates a particular variation, i.e., they always produce the
same result for the same input.
22
Spring 2016 CISG 5115
Donghyun (David) Kim
Department of Mathematics and Physics
North Carolina Central University
Turing Machine: Simulation
• Note that configuration of a TM can be encoded by a
(finite) string that other TM can read and write on its
tape. Furthermore, the transition function of a TM can
be also described by a (finite) string. These
observations lead to idea of encoding and running the
whole TM by another TM.
23
Spring 2016 CISG 5115
Donghyun (David) Kim
Department of Mathematics and Physics
North Carolina Central University
Multi-tape TMs
• It is an ordinary Turing machine with several tapes.
• Initially, the input appears on tape 1, and the others start
out blank.
• Several tapes, each has its own head for reading and
writing.
•  : Q   k  Q   k {L, R}k
•  (qi , a1,, ak )  (q j , b1,, bk , L, R,, L)
• Theorem: Every multitape TM has an equivalent single
tape TM.
Proof. Simulate a multitape TM with a single tape TM:
use one tape to represent all tapes
24
Spring 2016 CISG 5115
Donghyun (David) Kim
Department of Mathematics and Physics
North Carolina Central University
Multi-tape TMs – cont’
 #
 # #
w  w1w2  wn #w 1 w2  wn # 
0
1
0
1
⊔
a
a
a
⊔
⋯
b
a
⊔
⋯
M
S
#
1
1
0
1
0
#
a
a
𝑎
#
𝑏
a
#
⊔
⋯
25
Spring 2016 CISG 5115
Donghyun (David) Kim
Department of Mathematics and Physics
North Carolina Central University
⋯
Nondeterministic TMs
• Nondeterministic computation | several possibilities to
proceed
Q{ L , R}
•  :Q  2
•  (qi , a1 )  {( q j , b1 ), (qk , c1 ), }
• Theorem: Every nondeterministic TM has an equivalent
deterministic TM.
• Proof. Simulate a nondeterministic TM with a
deterministic TM: try all possible derivations.
26
Spring 2016 CISG 5115
Donghyun (David) Kim
Department of Mathematics and Physics
North Carolina Central University
Nondeterministic TMs – cont’
0
0
1
0
⊔
⋯
x
x
#
0
1
x
⊔
⋯
simulation tape
(for a branch of
nondeterministic
computation)
1
2
3
3
2
3
1
2
1
input tape (never altered)
D
address tape (keeps track of D’s location in N’s
nondeterministic computation tree)
Spring 2016 CISG 5115
Donghyun (David) Kim
1
⊔
⋯
27
Department of Mathematics and Physics
North Carolina Central University
Enumerators
• A TM with an attached printer. (loosely speaking)
• Start with a blank input tape
• The language enumerated is all the strings it prints out.
• Theorem: A language is Turing-recognizable if and only
if some enumerator enumerates it.
• M = on input w,
1.
2.
Run E. Every time that E outputs a string, compare it with
w.
If w ever appears in the output of E, accept.
• Construct an enumerator E as follows:
1.
2.
*
Run M for each possible s  
If M accept s, print out
Spring 2016 CISG 5115
Donghyun (David) Kim
28
Department of Mathematics and Physics
North Carolina Central University
3.3 The Definition of
Algorithm
29
Spring 2016 CISG 5115
Donghyun (David) Kim
Department of Mathematics and Physics
North Carolina Central University
Algorithms
• Algorithms have been around since ancient times.
Informally, it is a collection of simple instructions for
carrying out some task. They can be called
“procedures” or “recipes”.
• One of the famous algorithms was given by Euclid
around 300 BC (now called Euclid's algorithm) for
finding the greatest common divisor (GCD) of two
integers.
• However, there was no rigorous definition of the
algorithm until the twentieth century
Spring 2016 CISG 5115
Donghyun (David) Kim
30
Department of Mathematics and Physics
North Carolina Central University
Hilbert's Tenth Problem
• In 1900, at the International Congress of
Mathematicians, David Hilbert posed a list of open
problems as a challenge for the coming century. The
tenth problem was about an algorithm.
• He asked to “devise” a procedure for answering to the
question whether a given polynomial such as
coefficient
term
6 x yz  3 xy2  x3 10
3
2
possess an integer root, i.e., a set of integer values of
the variables at which the polynomial takes zero value
(e.g., x = 5, y = 3, z = 0 is an integer root of the above
polynomial).
31
Spring 2016 CISG 5115
Donghyun (David) Kim
Department of Mathematics and Physics
North Carolina Central University
Hilbert's Tenth Problem – cont’
• By asking to devise a procedural way to answer this
question, Hilbert implicitly assumed that such a
procedure (algorithm) exists.
• There was no way to come up with a conclusion
without the definition of algorithm.
32
Spring 2016 CISG 5115
Donghyun (David) Kim
Department of Mathematics and Physics
North Carolina Central University
Church-Turing thesis
• The definition of the algorithm came in the 1936
papers of Alonzo Church and Alan Turing. Their
definitions, using 𝜆-calculus and Turing machines
respectively, turned out to be equivalent.
• Since then the connection between the informal
notion of an algorithm and the precise definition is
called the Church-Turing thesis.
• In 1970, Yuri Matijasevic showed there no such
algorithm exists.
Spring 2016 CISG 5115
Donghyun (David) Kim
33
Department of Mathematics and Physics
North Carolina Central University
Church-Turing thesis – cont’
• Hilbert's Tenth Problem basically asks to construct a
decider for the language:
D  { p | p isa polynomial withan integerroot}.
• That is, for any given polynomial p, we need to decide
whether p  D (in a finite number of steps).
• That is, Hilbert’s tenth problem asks in essence
whether the set D is decidable.
34
Spring 2016 CISG 5115
Donghyun (David) Kim
Department of Mathematics and Physics
North Carolina Central University
Example: Finding an integer root
• A Turing machine for finding an integer root of a given
polynomial (of a single variable x for simplicity, i.e.
4 x 3  2 x 3  x  7 ) can be informally described as:
• Evaluate a given polynomial p(x) at x set successively to
the values 0, 1, -1, 2, -2, 3, -3, …
• If at any such value of x the value p(x) is zero, then accept.
• Q: Is this Turing machine recognizer or decider?
35
Spring 2016 CISG 5115
Donghyun (David) Kim
Department of Mathematics and Physics
North Carolina Central University
Example: Finding an integer root
• Q: Is this Turing machine recognizer or decider?
• For a univariate polynomial, it is possible to compute an
upper bound for an absolute value of its integer root. If
no root found below this bound, the polynomial has no
integer roots, and its TM can reject (making it a decider
solving the problem).
• However, as shown by Matijasevic, for multivariate
polynomials no such bound can be computed, making it
impossible to have a decider for this problem in general
case.
36
Spring 2016 CISG 5115
Donghyun (David) Kim
Department of Mathematics and Physics
North Carolina Central University
Description of (Turing Machine)
Algorithms
• Algorithms can be described in three ways:
• formal: Explicit description of a Turing machine (its
states, transition function etc.);
• implementation: Description of TM using English prose,
i.e., how TM moves head, read/writes symbols etc.
(without explicitly specifying its states and transition
function);
• high-level: Description of an algorithm using English
prose, ignoring implementation details
• We start with the first two levels and later will switch
to the third one, when you feel confident on how to go
from one level of algorithm description to another.
37
Spring 2016 CISG 5115
Donghyun (David) Kim
Department of Mathematics and Physics
North Carolina Central University