Lecture 21
Variations of the Turing Machines – Cont’d
Nondeterministic Turing Machines
Universal Turing Machine
COT 4420
Theory of Computation
Section 10.2, 10.3, 10.4
Slides from Prof. Busch
Same power doesn’t imply same speed:
n n
L = {a b }
Standard Turing machine:
Go back and forth
O(n 2 )
2-tape machine:
2
O(n ) time
times to match the a’s with the b’s
O(n) time
Read all the a’s and copy them onto tape 2
Match the b’s on tape 1 to a’s on tape 2
Multidimensional Turing Machines
2-dimensional tape
y
☐
☐
c
☐
a
b
☐
MOVES: L,R,U,D
U: up D: down
HEAD
Position: +2, -1
x
Theorem:
Proof:
Multidimensional machines
have the same power with
Standard Turing machines
1. Multidimensional machines
simulate Standard Turing machines
2. Standard Turing machines
simulate Multi-Dimensional machines
1. Multidimensional machines simulate
Standard Turing machines
Trivial: Use one dimension
2. Standard Turing machines simulate
Multidimensional machines
Standard machine simulating 2-dimensional
machine:
• Use a two track tape
• Store symbols in track 1
• Store coordinates in track 2
2-dimensional machine
y
☐
☐
c
☐
a
x
b
☐
Standard Machine
c
a
b
1 # 1 # 2 # − 1 # − 1# 1#
q1
q1
symbol
address
Standard machine:
Repeat for each transition followed
in the 2-dimensional machine:
1. Update current symbol
2. Compute coordinates of next position
3. Go to new position
Nondeterministic Turing
Machines
Nondeterministic Turing Machines
a → b, L
q2
q1
a → c, R
q3
Allows Non Deterministic Choices
Time 0
☐
a b c
☐
Time 1
q1
a → b, L
Choice 1
q2
☐
b b c
☐
q2
q1
a → c, R
Choice 2
q3
☐
c b c
q3
☐
Input string w is accepted if
there is a computation:
q0w
Initial configuration
*
⊢
x qf y
Final Configuration
Any accept state
There is a computation:
Theorem:
Proof:
Nondeterministic machines
have the same power with
Standard Turing machines
1. Nondeterministic machines
simulate Standard Turing machines
2. Standard Turing machines
simulate Nondeterministic machines
1. Nondeterministic Machines simulate
Standard (deterministic) Turing Machines
Trivial: every deterministic machine
is also nondeterministic
2. Standard (deterministic) Turing machines
simulate Nondeterministic machines:
Deterministic machine:
• Keep track of all possible computations in a 2dimensional tape
(equivalent to standard Turing machine with one tape)
All possible computation paths
Initial state
Step 1
Step 2
Step i
reject
accept
infinite
path
Step i+1
NonDeterministic machine
a → b, L
Time 0
q2
☐
q1
a b c
☐
q1
a → c, R
q3
Deterministic machine
#
#
#
#
# # #
a b c
q1
# # #
# #
#
#
#
current
configuration
NonDeterministic machine
Time 1
a → b, L
q2
☐
b b c
☐
Choice 1
c b c
☐
Choice 2
q2
q1
☐
a → c, R
q3
q3
Deterministic machine
# # # # # #
#
b b c #
# q2
#
#
c b c #
q3
#
#
Computation 1
Computation 2
Deterministic Turing machine
Repeat
Execute a step in each computation:
if there are two or more choices:
1. Replicate configuration
2. Change the state in the replicas
Until either the input string is accepted
or rejected in all configurations
Remark:
The simulation in the deterministic machine
takes exponential time compared to the
nondeterministic machine
Universal Turing Machine
A Turing Machine is restricted to carry out
one particular type of computation.
Real Computers however are re-programmable
Solution:
Universal Turing Machine
Attributes:
• Reprogrammable machine
• Simulates any other Turing Machine
Universal Turing Machine Mu simulates any
Turing Machine M
Input of Universal Turing Machine:
• Description of transitions of M
• Input string of w
Three tapes
Tape 1
Description of M
Universal
Turing
Machine Mu
Tape 2
Tape Contents of M
Tape 3
State of M
Tape 1
Description of M
We describe Turing machine M
as a string of symbols:
We encode M as a string of symbols
Alphabet Encoding
Symbols:
Encoding:
a1
a2
a3
a4
1
11
111
1111

State Encoding
States:
q1
q2
q3
q4
Encoding:
1
11
111
1111
Head Move Encoding
Move:
L
R
Encoding:
1
11

Transition Encoding
Transition:
δ (q1, a ) = (q2 , b, L)
Encoding:
1 0 1 0 11 0 11 0 1
separator
Turing Machine Encoding
Transitions:
δ (q1, a ) = (q2 , b, L)
δ (q2 , b) = (q3 , c, R)
Encoding:
1 0 1 0 11 0 11 0 1 00 11 0 1 10 111 0 111 0 11
separator
Every Turing Machine can be represented by a
string of 0’s and 1’s.
Therefore:
The set of Turing machines
forms a language:
each string of this language is
the binary encoding of a Turing Machine
Language of Turing Machines
L = { 010100101,
00100100101111,
111010011110010101,
…… }
(Turing Machine 1)
(Turing Machine 2)
……
Review: Countable and
Uncountable sets
• What about |ℤ|=?
…, -4, -3, -2, -1, 0, 1, 2, 3, 4, …
8
6
4
2 0 1
3 5 7
• A set S is called countable iff |S| = |ℕ|
There is a one to one correspondence (injection) of
elements of the set to positive integers (1,2,3,…)
• A set S is called uncountable iff |ℕ| < |S|
Example: The set of rational numbers
is countable
Rational numbers:
1 3 7
, , , 
2 4 8
1
1
1
2
1
3
2
1
2
2
2
3
3
1
3
2
4
1


1
4


Rational Numbers:
1 1 2 1 2
, , , , , 
1 2 1 3 2
Correspondence:
Positive Integers:
1, 2, 3, 4, 5, 
We Showed:
The set of rational numbers is countable
by describing an enumeration procedure
(enumerator)
for the correspondence to natural numbers
Definition
Let S be a set of strings on some alphabet
An enumerator for S is a Turing Machine
that generates (prints on tape)
all the strings of S one by one
and
each string is generated in finite time
strings
s1, s2 , s3 ,  ∈ S
Enumerator
Machine for S
output
(on tape)
Finite time:
s1, s2 , s3 , 
t1, t2 , t3 ,
Enumerator Machine
Time 0
Configuration
◊ ◊
q0
Time t1
prints s1
x1 # s1
qs
Time t2
prints s2
x2 # s2
qs
Time t3
prints s3
x3 # s3
qs
Observation:
If for a set S there is an enumerator,
then the set is countable
The enumerator describes the
correspondence of S to natural numbers
Theorem:
Proof:
The set of all Turing Machines
is countable
Any Turing Machine can be encoded
with a binary string of 0’s and 1’s
Find an enumeration procedure
for the set of Turing Machine strings
Enumerator:
Repeat
1. Generate the next binary string
of 0’s and 1’s in proper order
2. Check if the string describes a
Turing Machine
if YES: print string on output tape
if NO: ignore string
Binary strings
Turing Machines
0
1
00
01

1 0 1 0 11 0 11 0 0
1 0 1 0 11 0 11 0 1
s1

s2
1 0 11 0 1010010101101
1 0 1 0 11 0 11 0 1
1 0 11 0 1010010101101

End of proof
A Language that is not accepted by Turing
Machine
We will prove that there is a language L’
which is not accepted by any Turing machine
Technique:
Turing machines are countable
The set of all Languages is uncountable
(there are more languages than Turing Machines)
Theorem:
If S is an infinite countable set, then the
powerset 2S of S is uncountable.
The powerset 2S is a set of all subsets:
Example:
Application of Cantor’s Theorem
The set of all strings over alphabet {0,1}
S = { 0,1}* = { λ, 0, 1, 00, 11, 01, 10, …}
infinite and countable
Application of Cantor’s Theorem
The set of all strings over alphabet {0,1}
S = { 0,1}* = { λ, 0, 1, 00, 11, 01, 10, …}
infinite and countable
A language is a subset of S:
L = { 00, 01, 0111000}
Application of Cantor’s Theorem
The set of all strings over alphabet {0,1}
S = { 0,1}* = { λ, 0, 1, 00, 11, 01, 10, …}
infinite and countable
The powerset of S contains all languages:
2S = { {λ}, {0}, {01}, {01,11,011110}, …}
uncountable
The limits of computation
• The set of Turing machines is a language over
alphabet {0,1} and is countable.
• The set of languages is uncountable.
– There are more languages than Turing
machines
– Therefore, there exists languages that are
not acceptable by Turing machines.
Languages accepted
by Turing machines:
X countable
All possible languages:
S uncountable
2
Therefore:
X ≠2
(since X ⊆ 2 , we get
S
X ⊂2
S
S
)
Conclusion:
There is a language L’ not accepted
by any Turing Machine:
X ⊂2
S
∃L′ ∈ 2
S
and L′ ∉ X
Non Turing-Acceptable Languages
L′
Turing-Acceptable
Languages