CS240 • Language Theory and Automata • Fall 2015
Turing Machines
Turing Machines
After Alan M. Turing (1912–1954)
(real picture this time!)
In 1936, Turing introduced his
abstract model for computation in
his article "On Computable Numbers, with an
application to the Entscheidungsproblem”
At the same time, Alonzo Church published
similar ideas and results
However, the Turing model has become the
standard model in theoretical computer science
Turing Machines
• Turing machine (TM) = formal model
of a computer running a particular
program
• We use the simplicity of the TM
model to prove formally that there
are specific problems (languages)
that the TM cannot solve
The TM
• Similar to a finite automaton but with an
unlimited and unrestricted memory
– Infinite tape is unlimited memory
– Tape head can read and write symbols and move around the
tape
– Initially tape contains only the input string and is blank
everywhere else
– To store information, TM can write on the tape
– TM continues computing until it decides to produce an output
– Outputs accept and reject obtained by entering designated
accepting and reject states
– If TM does not enter an accepting or reject state, it will go on
forever, never halting
Schematic of a TM
CONTROL
      a
b
a
b
    
Summary of differences
between TM and FA
1. TM can both read from tape and write on
it
2. Read-write head can move both to the left
and right
3. The tape is infinite
4. The special states for accepting and
rejecting take immediate effect
5. TM need not read the entire input to
accept
Three Possible Outcomes
• TM may halt and accept the
input
• TM may halt and reject the
input
• TM may not halt at all
Example
TM M1 for testing membership in the
language B = {wcw | w  {a,b}*}
• M1 should accept if its input is a member of B
• To understand better, imagine yourself in M1's
place
– Standing on a mile-long input tape consisting of millions of
characters
– Goal is to determine if the input comprises two identical
strings of as and bs separated by a c
– Input is too long for you to remember it all, but you can move
back and forth over the input and make marks on it
– Obvious strategy is….
• …zig zag back and forth to corresponding
places on the two sides of the c and determine
whether or not they match
• Use marks to keep track of which places
correspond
Use same strategy to design M1:
– Make multiple passes over the input string with read-write head
– On each pass, matches one of the characters on each side of
the c
– Cross off each symbol as it is examined
– If all symbols crossed off, then the string is in the language
– If a mismatch discovered, enter a reject state
Algorithm
1.
Scan the input to be sure that it contains a single c
•
2.
If not, reject
Zig-zag across the tape to corresponding positions on
either side of the c to check whether they contain the
same symbol
•
•
3.
If not, reject
Cross off symbols as they are checked to keep track of which
symbols correspond
When all symbols to left of the c have been crossed off,
check to see if any symbols remain on the right
•
•
If yes, reject
Otherwise, accept
Snapshots while computing
a b b a a a c a b b a a a . . .
x b b a a a c x b b a a a . . .
x b b a a a c x b b a a a . . .
x x b a a a c x b b a a a . . .
x x b a a a c x x b a a a . . .
x x x x x x c x x x x x x . . .
accept
Formal TM
• M = (Q,,,,q0,B,F) where:
–
–
–
–
–
Q = finite set of states
 = tape alphabet
   = input alphabet
B () in  -  = blank
q0 in Q = start state; qaccept  Q = accepting states;
qreject  Q rejecting states
–  takes a state and tape symbol, returns a new state,
replacement symbol (both state and symbol might not
change) and a direction L/R for head motion
Transition function 
• The heart of the definition of the TM is the
transition function  because it tells us how
the machine gets from one step to the next
• Form of  : Q    Q    {L,R}
– When the machine is in a certain state q  Q and
head is over a tape square containing a symbol a
 , and if (q,a) = (r,b,L), the machine writes the
symbol b replacing the a, and goes to state r
• The third component is either L or R, indicating whether
head then moves left or right--here, it moves left
Three language classes
– Recursive = TM halts for all input strings
(decides a language)
• Sipser: "Turing-decidable”
– Recursively Enumerable = TM can accept the
strings in the language but cannot tell for certain
that a string is not in the language
• Sipser: "Turing-recognizable”
– Non-RE = no TM can even recognize the
members of the language in the RE sense
N.B. : BEING RECURSIVE, RECURSIVELY
ENUMERABLE, OR NON-RE IS A PROPERTY OF THE
LANGUAGE, NOT THE TM THAT ACCEPTS IT
Example
•
•
Nontrivial examples are hard to come by
Here's a TM that checks that its third symbol is
0, accepts if so, and runs forever if not
M = ({p, q, r, s, t}, {0, 1}, {0, 1, B}, p, B,
{s},{})
1.
2.
3.
4.
5.
(p,X) = (q, X, R) for X = 0, 1
(q,X) = (r, X, R) for X = 0, 1
(r,0) = (s, 0, L)
(r,1) = (t, 1, R)
(t,X) = (t, X, R) for X = 0, 1, B
ID's of a Turing Machine
• The ID (instantaneous description)
captures what is going on at any moment:
the current state, the contents of the tape,
and the position of the tape head
– Keep things finite by dropping all symbols to
the left of the string and to the right of the
rightmost nonblank
• Subtle point: although there is no limit on how
far right the head may move and write nonblanks, at any finite time, the TM has visited
only a finite prefix of the infinite tape
Notation
q says:

•  is the tape contents to the left of the
head
• The state is q
•  is the nonblank tape contents at or to
the right of the tape head
• One move indicated by ; zero, one, or
more moves represented by*

M = ({p, q, r, s, t}, {0, 1}, {0, 1, B}, p, B, {s},{})
•
(p,X) = (q, X, R) for X = 0, 1
•
(q,X) = (r, X, R) for X = 0, 1
•
(r,0) = (s, 0, L)
•
(r,1) = (t, 1, R)
•
(t,X) = (t, X, R) for X = 0, 1, B
Example


• With input 0101, the sequence of ID's of
the example TM is: p0101 0q101
01r01 0s101

– At that point it halts, since state s has no move
when the head is scanning 1

• With input 0111 the sequence is: p0111
0q111 01r11 011t1 0111t
0111Bt …



 
– The TM never halts, but continues to move right
More examples
TM that recognizes the language of
all strings of 0s whose length is a
n
2
power of 2: A={0 | n ≥ 0}
M2 = On input string w:
1. Sweep left to right across the tape, crossing off
every other 0
2. If in stage 1 the tape contains a single 0,
accept
3. If in stage 1 the tape contains more than a
single 0 and the number of 0s was odd, reject
4. Return the head to the left end of the tape
5. Go to stage 1
Formal description
M2 = (Q,,,,q1,qaccept,qreject)
Q = (q1, q2, q3, q4, q5, qaccept,qreject)

= {0}
 is a blank

= {0,x,}
q1 = start state
qaccept = accept state
qreject = reject state
Transition function 
0L
xL
q5
xR
xR
q1
0  R
q2
0x,R
q3
0x,R
0R
xR
xR
qreject
qaccept
q4
0L
xL
Sample run
q5
xR
xR
q1
Input: 0000
0  R
q2
0x,R
q3
0x,R
0R
xR
xR
qreject
q10000
q5x0x
xq5xx
q2000
q5 x0x
q5xxx
xq300
q2x0x
q5xxx
x0q40
xq20x
q2xxx
x0xq3
xq3xx
xq2xx
x0q5x
xxxq3
xxq2x
xq50x 
xxq5x
xxxq2
xxxqaccept
qaccept
q4
Example
TM to decide {aibjck | i x j = k and i,j,k ≥ 1
M3 = On input string w:
1. Scan the input left to right to be sure it is a
Tricky; have to
member of a*b*c* and reject if it isn't
2. Return the head to the left end of the tape mark at beginning
3. Cross off an a and scan to the right until a b
occurs. Shuttle between the bs and cs, crossing
off one of each until the bs are gone
4. Restore the crossed off bs and repeat stage 3 if
there is another a to cross off. If all the as are
crossed off, check on whether all the cs are
crossed off. If yes, accept; otherwise, reject
Example
Element distinctness problem
List of strings over {0,1} separated by #'s, job is to accept if all
strings are different
E = {#x1#x2#…xk | each xi  {0,1}* and xi  xj for each i  j}
M3 = On input string w:
1. Place a mark on top of the leftmost tape symbol (marked
symbols are simply new symbols added to the tape alphabet). If
that symbol is not a #, reject
2. Scan right to the next # and place a second mark of top of it, If
no # is encountered before a blank symbol, only x1 was present
so accept
3. By zig-zagging, compare the two strings to the right of the
marked #'s. If they are equal, reject
4. Move the rightmost of the two marks to the next # symbol to the
right. If no # is encountered before a blank symbol, move the
leftmost mark to the next # to its right and the rightmost mark to
the # after that. This time, if no # is available for the rightmost
mark, all strings have been compared so accept
Variants of Turing Machines
• Many alternative definitions of TMs exist
• The original model and its reasonable
variants have the same power
– Recognize the same languages
• E.g., TM that can move left, right, or
"stay put"
– No new power though because could always
simulate the "stay put" move with two moves, one
to the left and one to the right, in the original TM
– This example is key to showing equivalence of TM
variants
A lot of models for computation turn out to be
equivalent (especially variants of Turing
machines). To show they are equivalent, give a
way to simulate one model with the other.
Multitape TMs
• Allow the TM to have some finite
number of tapes k, with a head for
each tape
– Move is a function of the state and the
symbol scanned by each tape head
– Action = new state, new symbol for each
tape, and a head motion (L, R)
– First tape holds the input, other tapes are
initially blank
Every Multitape TM has an
equivalent single tape TM
Proof: show how to convert multitape TM M to
an equivalent single tape TM S
– If M has k tapes, S simulates the effect of k tapes
by storing the same information on its single
tape
– Uses a new symbol # as a delimeter to separate
the contents of the different tapes
– S must also keep track of the locations of the
heads on each tape
• Writes a tape symbol with a dot above it to mark where the
head on that tape would be
• Dotted symbols are simply new symbols added to the tape
alphabet
0
1
0
1
0
a
a
a

…
b
a

M
…

S
•
•
•
# 0 1 0 1 0 # a a a # b a # …
…
Definition of S
On input w = w1 …wn:
1.
First S puts its tape into the format that represents all k tapes of M.
The formatted tape contains
2.
3.
• w …w #• #• #…#
#w
1 2
n
To simulate a single move, S scans its tape from the first # (marks
the left end) to the (k+1)st # (marks the right end) to determine the
symbols under the virtual heads. Then S makes a second pass to
update the tapes according to the way that M's transition function
dictates.
If at any point S moves one of the virtual heads to the right onto a #,
this action signifies that M has moved the corresponding head onto
the previously unread blank portion of that tape. So S writes a blank
symbol on this tape cell and shifts the tape contents, from this cell to
the rightmost #, one unit to the right. Then it continues the simulation
as before.
Nondeterministic TM
• Let the TM have a finite set of
choices of move
• Components of NDTM same as TM
• Transition function may indicate that
more than one move is possible
NDTM
• Arbitrarily chooses move when more
than one possibility exists
• Accepts if there is at least one
computation that terminates in an
accepting state
– Existence of other computations that halt in nonaccepting states or fail to halt altogether is
irrelevant
NDTM
LNDTM = wcz | c is immediately preceded or followed by the string ab
cc,R
bb,R
aa,R
 R
q0
Three choices on
c
cc,R
q1
bb,R
aa,R
q2
q3
q4
cc,L
bb,L
q5
aa,L
q6
q7
Processes input until a c is encountered; then, can stay in state q1, enter state
q2 to determine if the c is followed by ab, or enter q5 to determine if the c is
preceded by ab
NTM Computation
Evolution of the NTM
represented by a tree
of configurations
C5
If there is (at least)
one accepting leaf,
then the TM accepts
C1
C2
t=1
C4 t=2
C3
t=3
C6

"reject"
"accept"
Simulating Nondeterministic
TMs with Deterministic Ones
We want to search every path down the tree
for accepting configurations
Bad idea: "depth first"
- This approach can get lost in never-halting paths
Good idea: "breadth first”
- For time step 1,2,… we list all possible
configurations of the non-deterministic TM
- The simulating TM accepts when it lists an
accepting configuration
Breadth First

C1
t=1
Let b be the maximum number
of children for a node
1 C
2
2
C3 3 C4 t=2
Any node in the tree can
1,2
1,1
3,1
t=3
be uniquely identified by
C5 C6
 "reject"
a string  {1,…,b}*
Example: location of the
rejecting configuration is (3,1)
"accept"
With the lexicographical listing , (1), (2),…, (b), (1,1),
(1,2),…,(1,b), (2,1),… etc., we cover all nodes
Every nondeterministic TM has an
equivalent deterministic TM
Proof:
Let M be the nondeterministic TM on input w
Let M' be the deterministic TM
•
1.
2.
3.
The simulating TM M’ uses three tapes:
T1 contains the input w
T2 simulates the tape content of M on w at a given node
T3 keeps track of the current location in the nondeterministic computation
tree
Steps in the simulation:
1. Empty string (symbolizing address of the root) is written on T3
2. Input string on T1 is copied to T2
3. The computation of M defined by the sequence on T3 is simulated on T2
4. If no more symbols remain on T3, a nondeterministic choice is invalid, or a
rejecting configuration is found, abort this branch and go to step 6
5. If an accepting configuration is found, the computation of M' halts and accepts
the input
6. If the computation did not halt in step 4, the next sequence is generated on T3
and the computation continues at step 3
Robustness
Just like k-tape TMs, nondeterministic Turing machines
are not more powerful than simple TMs:
Every nondeterministic TM has an equivalent
3-tape Turing machine, which – in turn – has an
equivalent 1-tape Turing machine.
Hence
A language L is recursively enumerable (Turing recognizable) if
and only if some nondeterministic TM recognizes it
A language L is recursive (decidable) if and only if some
nondeterministic TM decides it
Enumerating Languages
So far, the Turing machines were recognizers
When a TM E generates the words of a language,
E is an enumerator (cf. "recursively enumerable”)
A Turing machine E, enumerates the language L
if it prints an (infinite) list of strings on the tape
such that all elements of L will appear on the tape,
and all strings on the tape are elements of L
E starts on an empty input tape. The
strings can appear in any order; repetition
is allowed.
Enumerating = Recognizing
A language L is recursively enumerable
if and only if some enumerator enumerates
it
Proof (short version)
1.
Show that if we have an enumerator E that enumerates
language A, a TM M recognizes A
M: On input w
• Run E and check the strings it generates
• If w is produced, then accept
2.
Now show if a TM M recognizes A, we can construct
enumerator E for A
E: Ignore the input
• Repeat the following for i = 1,2,3,…
– Let s1,s2,… be a listing of all strings  *
– Run M on s1,…,sj
– If any computations accept, print out the corresponding
s
A is recursively enumerable iff A =
L(E) for some enumerator E
• Proof (need to prove both directions)
1. Convert E to an ordinary recognizer M
•
•
•
•
Given E, we construct a TM M with E built inside it
Have M leave the input string w alone
M moves to the blank portion of the tape and runs E
When E decides to print something out, M takes a look to
see if the string is w
– If not, then M keeps simulating E
– If the string is w, then M accepts
M
w
E
x
YES
NO
x = w?
If M doesn’t find a match, it may go on forever—this is
okay
ACCEP
T
2. Convert M to enumerator E
–
The idea is to feed all possible strings to M in
some reasonable order, for instance,
lexicographic order ε, 0, 1, 00, 01, 10, 11
– However, we have to be careful…
– Suppose M is running on 101
• If M accepts 101, then we print it out
• If M halts and rejects 101, then E should move on to the
next string
• The only problem is when M runs forever
• What is E supposed to do? E doesn’t know M is going
forever!
• Can’t get hung up running M on 101
– We need to check the rest of the strings
• Solution
– Run M for a few steps on any given string, and if
hasn’t halted then move on, and come back to it
later
– Share time among all strings where computation
hasn’t ended
– Run more and more strings for longer and longer
– More precisely
• For k = 1,2,3,..., E runs M on the first k strings for k steps
• If M ever accepts some string s, then print s
Other Computational Models
We can consider many other 'reasonable'
models of computation: DNA computing,
neural networks, quantum computing…
Experience teaches us that every such model
can be simulated by a Turing machine
Church-Turing Thesis
The intuitive notion of computing and algorithms
is captured by the Turing machine model
A Brief History of Turing
Machines
• Why are Turing machines so important, and why do
we use them as a model for a general- purpose
computer?
• The concept of a Turing machines dates back to the
1930’s
– One of a number of different models of computation that tried to capture
effective computability, or algorithm, as we would now say
– Other researchers came up with other models to capture computation, e.g.,
Alonzo Church developed lambda calculus
• It wasn’t obvious that these different models are
equivalent, i.e., that they capture the same class of
computations
– However, they do!
Brief History Con’t
• Nowadays we have programming
languages
– Can today’s more “advanced” programming
languages (e.g., Python) do more than, say, a
FORTRAN program?
– It has a lot of new features compared to old boring
DO-loops
– It is conceivable that as we add more constructs to
a programming language, it becomes more
powerful, in the sense of computing functions and
recognizing languages
Brief History Con’t
• However, anything you can do with
one language you can do with
another
– It might be easier to program in one than another,
or one might run faster
• How can we show we can do the same
thing with Python as FORTRAN?
– Convert Python programs into FORTRAN, or
convert FORTRAN programs to Python
– Simulate one language with the other
– This “proves” that they have the same
computational power
Brief History Con’t
• That’s what the researchers of computation
theory did
• They gave ways to simulate Turing machines
by lambda calculus, lambda calculus by
Turing machines, as well as different
variations of these models
• They found that all these models were
doing the same thing
• Similarly, several variations of Turing
machines all have the same computational
power
Importance of the
Church-Turing Thesis
The Church-Turing thesis marks the end of
a long sequence of developments that concern
the notions of "way-of-calculating", "procedure",
"solving", "algorithm”
Goes back to Euclid's GCD algorithm (300 BC)
For a long time, this was an implicit notion that
defied proper analysis
"Algorithm"
After Muhammad ibn Musa al-Khwarizmi
(770 – 840)
His "Al-Khwarizmi on the Hindu Art of
Reckoning" describes the decimal system
(with zero), and gives methods for calculating
square roots and other expressions
Algebra comes from the title of his book Kitab al-jabr wa'l-muqabala (Book of
Restoring and Balancing). The term "al-jabr" is translated as "restoring".
Restoring, in this case, referred to the method of taking a subtracted quantity
from one side and placing it to the other side of an equation
Algorithm
• The notion of algorithm is a natural,
robust notion
• This was a major step forward in our
understanding of what computation is
– It’s almost saying something about the physical
universe: there is nothing we can build in the
physical world that is more powerful than a Turing
machine
Hilbert's 10th Problem
In 1900, David Hilbert (1862–1943) proposed
his Mathematical Problems (23 of them)
Hilbert's 10th problem is the
Determination of the solvability of a
Diophantine equation
•
An indeterminate polynomial equation where variables are
integers only
Given a Diophantine equation with any number of unknown
quantities and with rational integral numerical coefficients:
devise a process according to which it can be
determined by a finite number of operations whether the
equation is solvable in rational integers
Hilbert's 10th Problem
• Suppose we want to solve a polynomial
equation 3x2 + 17x - 22 = 0
– Easily done
– But suppose we don’t want to know if a polynomial equation
has a root, but whether it has a root where variables are
integers
– Furthermore, we allow variables with several variables
– This makes things a lot harder: e.g., could have
17xy2 + 2x - 21z5 + xy + 1=0
• Is there an assignment of integers in x, y, z
such that this equation is satisfied?
Hilbert's 10th Problem
• Hilbert asked:
– Is there a finite procedure which concludes after
some finite number of steps, that tells us whether a
given polynomial has an integer root?
• Put in the modern framework:
Let D = {p | p is a multivariable polynomial that has a
solution (root) in integers}
Is D decidable?
Hilbert's 10th Problem
• Without a precise notion of procedure, there
was no hope of answering the question
– Hilbert originally said, give a “finite procedure”
– There was no notion that there might not be a
procedure!
• It took 35 years before the problem could be
addressed because we needed a formal
notion of procedure to prove there is none
• Here, the Church-Turing Thesis played a
fundamental role
(Un)solving Hilbert's 10th
Hilbert's "…a process according to which it can
be determined by a finite number of operations…"
needed to be defined in a proper way
This was done in 1936 by Church and Turing
The impossibility of such a process for
exponential equations was shown by Davis,
Putnam and Robinson
Matijasevic proved that Hilbert's 10th problem
is unsolvable in 1970
Terminology for TMs
• From now on we speak of TMs but real focus
is on algorithms
– TM is a precise model for the definition of algorithm
• Possible descriptions of algorithms/TMs:
– Formal (as in previous slides): lowest level of detail
– Implementation description: use prose to describe how the
TM works (way it moves its head, stores data on tape), no
details of states and transitions
– High-level description: use prose to describe the algorithm,
ignore implementation details
This is sufficient from
now on
Format and Notation
• Input to TM is always a string
– Must represent other objects as strings
• Strings can represent anything!
– TM may be programmed to decode the
representation to so that it is interpreted as
intended
– Notation for the encoding of an object into its
representation: <O>
• For several objects, <O1,O2…,Ok>
– Example
• Let A = {<G> | G is a connected undirected graph}
• Then A is the language consisting of all strings
representing undirected graphs