A Universal Turing Machine
Fall 2006
Costas Busch - RPI
1
A limitation of Turing Machines:
Turing Machines are “hardwired”
they execute
only one program
Real Computers are re-programmable
Fall 2006
Costas Busch - RPI
2
Solution:
Universal Turing Machine
Attributes:
• Reprogrammable machine
• Simulates any other Turing Machine
Fall 2006
Costas Busch - RPI
3
Universal Turing Machine
simulates any Turing Machine
M
Input of Universal Turing Machine:
Description of transitions of
Input string of
Fall 2006
M
M
Costas Busch - RPI
4
Tape 1
Three tapes
Description of
Universal
Turing
Machine
M
Tape 2
Tape Contents of
M
Tape 3
Fall 2006
Costas Busch - RPI
State of
M
5
Tape 1
Description of
We describe Turing machine
as a string of symbols:
We encode
Fall 2006
M
M
M as a string of symbols
Costas Busch - RPI
6
Alphabet Encoding
Symbols:
a
b
c
d
Encoding:
1
11
111
1111
Fall 2006
Costas Busch - RPI

7
State Encoding
States:
q1
q2
q3
q4
Encoding:
1
11
111
1111

Head Move Encoding
Move:
L
R
Encoding:
1
11
Fall 2006
Costas Busch - RPI
8
Transition Encoding
Transition:
 (q1, a)  (q2 , b, L)
Encoding:
1 0 1 0 11 0 11 0 1
separator
Fall 2006
Costas Busch - RPI
9
Turing Machine Encoding
Transitions:
 (q1, a)  (q2 , b, L)
 (q2 , b)  (q3 , c, R)
Encoding:
1 0 1 0 11 0 11 0 1 00 11 0 110 111 0 111 0 11
separator
Fall 2006
Costas Busch - RPI
10
Tape 1 contents of Universal Turing Machine:
binary encoding
of the simulated machine
M
Tape 1
1 0 1 0 11 0 11 0 10011 0 1 10 111 0 111 0 1100
Fall 2006
Costas Busch - RPI
11
A Turing Machine is described
with a binary string of 0’s and 1’s
Therefore:
The set of Turing machines
forms a language:
each string of this language is
the binary encoding of a Turing Machine
Fall 2006
Costas Busch - RPI
12
Language of Turing Machines
(Turing Machine 1)
L = { 010100101,
00100100101111,
111010011110010101,
(Turing Machine 2)
……
…… }
Fall 2006
Costas Busch - RPI
13
Countable Sets
Fall 2006
Costas Busch - RPI
14
Infinite sets are either:
Countable
or
Uncountable
Fall 2006
Costas Busch - RPI
15
Countable set:
There is a one to one correspondence
of
elements of the set
to
Natural numbers (Positive Integers)
(every element of the set is mapped to a number
such that no two elements are mapped to same number)
Fall 2006
Costas Busch - RPI
16
Example: The set of even integers
is countable
Even integers:
(positive)
0, 2, 4, 6, 
Correspondence:
Positive integers:
1, 2, 3, 4, 
2n corresponds to n  1
Fall 2006
Costas Busch - RPI
17
Example: The set of rational numbers
is countable
Rational numbers:
Fall 2006
1 3 7
, , , 
2 4 8
Costas Busch - RPI
18
Naïve Approach
Nominator 1
Rational numbers:
1 1 1
, , ,
1 2 3
Correspondence:
Positive integers:
1, 2, 3, 
Doesn’t work:
we will never count
numbers with nominator 2:
Fall 2006
Costas Busch - RPI
2 2 2
, , ,
1 2 3
19
Better Approach
Fall 2006
1
1
1
2
1
3
2
1
2
2
2
3
3
1
3
2
4
1
1
4




Costas Busch - RPI
20
Fall 2006
1
1
1
2
1
3
2
1
2
2
2
3
3
1
3
2
4
1
1
4




Costas Busch - RPI
21
Fall 2006
1
1
1
2
1
3
2
1
2
2
2
3
3
1
3
2
4
1
1
4




Costas Busch - RPI
22
Fall 2006
1
1
1
2
1
3
2
1
2
2
2
3
3
1
3
2
4
1
1
4




Costas Busch - RPI
23
Fall 2006
1
1
1
2
1
3
2
1
2
2
2
3
3
1
3
2
4
1
1
4




Costas Busch - RPI
24
Fall 2006
1
1
1
2
1
3
2
1
2
2
2
3
3
1
3
2
4
1
1
4




Costas Busch - RPI
25
Rational Numbers:
1 1 2 1 2
, , , , , 
1 2 1 3 2
Correspondence:
Positive Integers:
Fall 2006
1, 2, 3, 4, 5, 
Costas Busch - RPI
26
We proved:
the set of rational numbers is countable
by describing an enumeration procedure
(enumerator)
for the correspondence to natural numbers
Fall 2006
Costas Busch - RPI
27
Definition
Let
S be a set of strings (Language)
An enumerator for S is a Turing Machine
that generates (prints on tape)
all the strings of S one by one
and
each string is generated in finite time
Fall 2006
Costas Busch - RPI
28
strings
s1, s2 , s3 ,  S
Enumerator
Machine for S
output
(on tape)
Finite time:
Fall 2006
Costas Busch - RPI
s1, s2 , s3 , 
t1, t2 , t3 ,
29
Enumerator Machine
Time 0
Configuration
 
q0
prints s1
Time t1
x1 # s1
qs
Fall 2006
Costas Busch - RPI
30
Time t 2
prints s2
x2 # s2
qs
Time t3
prints s3
x3 # s3
qs
Fall 2006
Costas Busch - RPI
31
Observation:
If for a set S there is an enumerator,
then the set is countable
The enumerator describes the
correspondence of S to natural numbers
Fall 2006
Costas Busch - RPI
32
Example: The set of strings
is countable
S  {a , b,c }

Approach:
We will describe an enumerator for
Fall 2006
Costas Busch - RPI
S
33
Naive enumerator:
Produce the strings in lexicographic order:
s1 
s2 

a
aa
aaa
aaaa
......
Doesn’t work:
strings starting with b
will never be produced
Fall 2006
Costas Busch - RPI
34
Better procedure:
Proper Order
(Canonical Order)
1. Produce all strings of length 1
2. Produce all strings of length 2
3. Produce all strings of length 3
4. Produce all strings of length 4
..........
Fall 2006
Costas Busch - RPI
35
s1 
s2 

Produce strings in
Proper Order:
Fall 2006
a
b
c
aa
ab
ac
ba
bb
bc
ca
cb
cc
aaa
aab
aac
......
Costas Busch - RPI
length 1
length 2
length 3
36
Theorem:
The set of all Turing Machines
is countable
Proof: Any Turing Machine can be encoded
with a binary string of 0’s and 1’s
Find an enumeration procedure
for the set of Turing Machine strings
Fall 2006
Costas Busch - RPI
37
Enumerator:
Repeat
1. Generate the next binary string
of 0’s and 1’s in proper order
2. Check if the string describes a
Turing Machine
if YES: print string on output tape
if NO: ignore string
Fall 2006
Costas Busch - RPI
38
Binary strings
Turing Machines
0
1
00
01

1 0 1 0 11 0 11 0 0
1 0 1 0 11 0 11 0 1
s1

s2
1 0 11 0 1010010101101

Fall 2006
1 0 1 0 11 0 11 0 1
1 0 11 0 1010010101101
End of Proof
Costas Busch - RPI
39
Uncountable Sets
Fall 2006
Costas Busch - RPI
40
We will prove that there is a language L
which is not accepted by any Turing machine
Technique:
Turing machines are countable
Languages are uncountable
(there are more languages than Turing Machines)
Fall 2006
Costas Busch - RPI
41
Definition:
A set is uncountable
if it is not countable
We will prove that there is a language
which is not accepted by any Turing machine
Fall 2006
Costas Busch - RPI
42
Theorem:
If
S is an infinite countable set, then
the powerset
S
2 of S is uncountable.
(the powerset 2 S is the set whose elements
are all possible sets made from the elements of S )
Fall 2006
Costas Busch - RPI
43
Proof:
Since
S is countable, we can write
S  {s1, s2 , s3 ,}
Elements of
Fall 2006
Costas Busch - RPI
S
44
Elements of the powerset 2 S have the form:

{s1, s3}
{s5 , s7 , s9 , s10}
……
Fall 2006
Costas Busch - RPI
45
We encode each element of the powerset
with a binary string of 0’s and 1’s
Powerset
element
Binary encoding
s1
s2
s3
s4 
{s1}
1
0
0
0

{s2 ,s3}
0
1
1
0

{s1,s3, s4}
1
0
1
1 
(in arbitrary order)
Fall 2006
Costas Busch - RPI
46
Observation:
Every infinite binary string corresponds
to an element of the powerset:
Example:
Corresponds to:
Fall 2006
10 0 111 0 
{s1, s4 , s5 , s6 ,}  2
Costas Busch - RPI
S
47
Let’s assume (for contradiction)
S
that the powerset 2 is countable
Then:
we can enumerate
the elements of the powerset
S
2  {t1, t2 , t3 ,}
Fall 2006
Costas Busch - RPI
48
suppose that this is the respective
Powerset
element
Fall 2006
Binary encoding
t1
1
0
0
0
0 
t2
1
1
0
0
0

t3
1
1
0
1
0

t4
1
1
0
0
1 


Costas Busch - RPI
49
Take the binary string whose bits
are the complement of the diagonal
t1
1
0
0
0
0 
t2
1
1
0
0
0

t3
1
1
0
1
0

t4
1
1
0
0
1 
Binary string:
t  0011
(birary complement of diagonal)
Fall 2006
Costas Busch - RPI
50
The binary string
corresponds
to an element of
S
the powerset 2 :
Fall 2006
t  0011
t  {s3, s4,}  2
S
Costas Busch - RPI
51
Thus, t
must be equal to some ti
t  ti
However,
the i-th bit in the encoding of t is
the complement of the i-th bit of ti , thus:
t  ti
Contradiction!!!
Fall 2006
Costas Busch - RPI
52
Since we have a contradiction:
The powerset
S of
2
S is uncountable
End of proof
Fall 2006
Costas Busch - RPI
53
An Application: Languages
Consider Alphabet : A  {a, b}
The set of all Strings:
S  {a, b}  { , a, b, aa, ab, ba, bb, aaa, aab,}
*
infinite and countable
(we can enumerate the strings
in proper order)
Fall 2006
Costas Busch - RPI
54
Consider Alphabet : A  {a, b}
The set of all Strings:
S  {a, b}  { , a, b, aa, ab, ba, bb, aaa, aab,}
*
infinite and countable
Any language is a subset of
S:
L  {aa, ab, aab}
Fall 2006
Costas Busch - RPI
55
Consider Alphabet : A  {a, b}
The set of all Strings:
S  A  {a, b}  { , a, b, aa, ab, ba , bb , aaa, aab,}
infinite and countable
*
*
The powerset of
S contains all languages:
2  {,{}, {a}, {a, b}, {aa, b},..., {aa, ab, aab}, }
uncountable
S
Fall 2006
Costas Busch - RPI
56
Consider Alphabet : A  {a, b}
countable
M3 
M1 M 2
Turing machines:
accepts
Languages accepted
By Turing Machines:
L1
Denote: X  {L1, L2 , L3 ,}
countable
Fall 2006
Costas Busch - RPI
L3
L2
countable
Note:

X 2
S
S  {a, b } 
*
57
Languages accepted
by Turing machines:
X countable
S
All possible languages: 2 uncountable
Therefore:
since X  2S , we have
Fall 2006
Costas Busch - RPI
X 2
X  2S
S

58
Conclusion:
There is a language L not accepted
by any Turing Machine:
X 2
S
L  2
S
and L  X
(Language L cannot be described
by any algorithm)
Fall 2006
Costas Busch - RPI
59
Non Turing-Acceptable Languages
L
Turing-Acceptable
Languages
Fall 2006
Costas Busch - RPI
60
Note that:
X  {L1, L2 , L3 ,}
is a multi-set (elements may repeat)
since a language may be accepted
by more than one Turing machine
However, if we remove the repeated elements,
the resulting set is again countable since every element
still corresponds to a positive integer
Fall 2006
Costas Busch - RPI
61