CDT314
FABER
Formal Languages, Automata
and Models of Computation
Lecture 15-1
Mälardalen University
2012
1
1
Countable Sets
Based on course materials by Prof. Costas Busch
2
Infinite sets are either:
• Countable
• Uncountable
3
Countable set:
There is a one to one correspondence
between
elements of the set
and
positive integers
4
Example:
The set of even integers
is countable
Even integers:
0, 2, 4, 6, 
Correspondence:
Positive integers:
2n
1, 2, 3, 4, 
corresponds to
n 1
5
Example:
The set of rational numbers
is countable
Rational numbers:
1 3 7
, , ,
2 4 8
6
Naive Approach
Rational numbers:
1 1 1
, , ,
1 2 3
Correspondence:
Positive integers:
1, 2, 3, 
Doesn’t work:
we will never count numbers with nominator 2
2 2 2
, , ,
1 2 3
7
Better Approach
1
1
1
2
1
3
2
1
2
2
2
3
3
1
3
2
4
1
1
4




8
1
1
1
2
1
3
2
1
2
2
2
3
3
1
3
2
4
1
1
4




9
1
1
1
2
1
3
2
1
2
2
2
3
3
1
3
2
4
1
1
4




10
1
1
1
2
1
3
2
1
2
2
2
3
3
1
3
2
4
1
1
4




11
1
1
1
2
1
3
2
1
2
2
2
3
3
1
3
2
4
1
1
4




12
1
1
1
2
1
3
2
1
2
2
2
3
3
1
3
2
4
1
1
4




13
Rational Numbers:
1 1 2 1 2
, , , , ,
1 2 1 3 2
Correspondence:
Positive Integers:
1, 2, 3, 4, 5, 
14
We proved:
the set of rational numbers is countable
by giving
an enumeration procedure
15
Definition
Let S be a set of strings
An enumeration procedure for S is a
Turing machine that generates
any string of S in finite number of steps.
16
strings
s1, s2 , s3 ,  S
Enumeration
Machine for S
output
Finite time:
s1, s2 , s3 , 
t1, t2 , t3 ,
17
Enumeration Machine
Configuration
Time 0
# #
q0
Time t1
x1  s1
qs
18
Time t 2
x2  s2
qs
Time t3
x3  s3
qs
19
A set is countable if there is an
enumeration procedure for it
20
Example:
The set of all strings
is countable.

{a, b, c}
We will describe the enumeration procedure.
21
Naive procedure:
Produce the strings in lexicographic order:
a
aa
aaa
...
Doesn’t work!
Strings starting with b will never be produced.
22
Better procedure: Proper Order
Produce all strings of length 1
Produce all strings of length 2
Produce all strings of length 2
..........
23
Produce strings:
Proper Order
a, b, c
aa
ab
ac
ba
bb
bc
ca
cb
cc
aaa
aab
aac
......
Length 1
Length 2
Length 3
......
24
Theorem:
The set of all Turing Machines
is countable.
Proof:
Any Turing machine is a finite string
Encoded with a sequence of 0’s and 1’s.
Find an enumeration procedure
for the set of Turing Machine strings.
25
Enumeration Procedure:
Repeat
1. Generate the next string of 0’s and 1’s
in proper order
2. Check if the string defines a
Turing Machine
if YES: print string on output
if NO: ignore string
26
Uncountable Sets
27
Definition:
A set is uncountable if it is not countable
28
Theorem:
Let S be an infinite countable set.
The powerset 2
S
of S is uncountable.
The power set of natural numbers has the same cardinality as the set of real numbers.
(Using the Cantor–Bernstein–Schröder theorem, it is easy to prove
that there exists a bijection between the set of reals and the power set of the natural
numbers).
29
Proof:
Since S is countable, we can write
S  {s1, s2 , s3 ,}
Element of S
30
Elements of the powerset have the form:
{s1, s3}
{s5 , s7 , s9 , s10 }
31
We encode each element of the power set
with a string of 0’s and 1’s *
Encoding
Powerset
s
s
s
s
3
1
2
4 
element
{s1}
1
0
0
0

{s2 , s3 }
0
1
1
0

{s1,s3, s4}

1
0
1

1

*Cantor’s diagonal argument
32
Let’s assume the contrary, that the
powerset is countable.
We can enumerate the elements of the
powerset.
33
Powerset
element
Encoding
t1
1
0
0
0
0 
t2
1
1
0
0
0

t3
1
1
0
1
0

t4
1
1
0
0
1 


34
Take the powerset element
whose bits are the complements
of the diagonal.
35
t1
1
0
0
0
0 
t2
1
1
0
0
0

t3
1
1
0
1
0

t4
1
1
0
0
1 
New element:
0011
(Diagonal complement)
36
The new element must be some ti
This is impossible:
The i-th bit must be the complement
of itself.
37
We have contradiction!
Therefore the powerset is uncountable.
38
Theorem:
Let S be an infinite countable set.
S
The powerset 2 of S is uncountable.
39
Application: Languages
Alphabet: {a, b}
Set of Strings:
S  {a, b}  { , a, b, aa, ab, ba, bb, aaa, aab,}
*
infinite and countable
Powerset: all languages
2  {{},{a},{a, b}{aa, ab, aab},}
L1 L2 L3
L4

S
uncountable
40
Languages: uncountable
L1
L2
L3
M1
M2
M3

Lk

?
Turing machines: countable
There are infinitely many more languages
than Turing machines!
41
There are some languages not accepted
by Turing Machines.
These languages cannot be described
by algorithms.
42
Recursively Enumerable Languages
and
Recursive Languages
43
Definition:
A language is recursively enumerable
if some Turing machine accepts it.
44
Let L be a recursively enumerable language
and M be the Turing Machine that accepts it.
For a string w :
if
w L
then M halts in a final state
if
w L
then M halts in some state
or loops forever
45
Definition:
A language is recursive
if some Turing machine accepts it
and halts on any input string.
In other words:
A language is recursive if there is
a membership algorithm for it
46
Let L be a recursive language
and M be the Turing Machine that accepts it.
For a string w :
if w L then M halts in a final state.
if w L then M halts in a non-final state.
47
We will prove:
1. There is a specific language
which is not recursively enumerable.
2. There is a specific language
which is recursively enumerable
but not recursive.
48
Non Recursively Enumerable
Recursively Enumerable
Recursive
49
First we prove:
• If a language is recursive then
there is an enumeration procedure for it.
• A language is recursively enumerable
if and only if
there is an enumeration procedure for it.
50
Theorem:
if a language L is recursive then
there is an enumeration procedure for it.
51
Proof:
Enumeration Machine
~
M
Enumerates all
strings of input alphabet
M
Accepts L
52
Enumeration procedure
Repeat:
~ generates a string w
M
M
checks if
w L
YES: print w to output
NO:
ignore w
End of proof
53
Theorem:
if language L is recursively enumerable
then there is
an enumeration procedure for it.
54
Proof:
Enumeration Machine
~
M
Enumerates all
strings of input alphabet
M
Accepts L
55
NAIVE APPROACH
Enumeration procedure
~
Repeat: M generates a string w
M checks if
w L
YES: print w to output
NO:
ignore w
w L
Problem: If
machine M may loop forever
56
BETTER APPROACH
~ generates first string w
M
1
M executes first step on w1
~ generates second string w
M
2
M executes first step on w2
second step on w1
57
~ Generates third string
M
M
w3
executes first step on
w3
second step on w2
third step on
w1
And so on............
58
w1
w2
w3
w4
1




2




3





Move

59
If for string w
machine M halts in a final state
then it prints w on the output.
End of proof
60
Theorem:
If for language L
there is an enumeration procedure
then L is recursively enumerable.
61
Proof:
Input Tape
w
Machine that
accepts L
Enumerator
for L
Compare
62
Turing machine that accepts L
For input string w
Repeat:
• Using the enumerator,
generate the next string of L
• Compare generated string with w
If same, accept and exit loop
End of proof
63
Question:
This is not a membership algorithm.
Why?
Answer:
The enumeration procedure
may not produce strings in proper order
64
We have shown:
A language is recursively enumerable
if and only if
there is an enumeration procedure for it.
65
A Language which
is not
Recursively Enumerable
66
We search for a language that
is not Recursively Enumerable.
This language is not accepted by any
Turing Machine.
67
Consider alphabet {a}
Strings:
a, aa, aaa, aaaa, 
1
a a
2
a
3
a
4

68
Consider Turing Machines
that accept languages over alphabet {a}
They are countable:
M1, M 2 , M 3 , M 4 , 
69
Example language accepted by M i
L( M i )  {aa, aaaa, aaaaaa}
2
4
6
L( M i )  {a , a , a }
Alternative representation
1
2
a
a
L( M i ) 0
1
a
3
0
4
a
1
0
a
5
a
6
1
a
7
0


70
1
a
a
2
3
a
L ( M1 )
0
1
0
1

L( M 2 )
1
0
0
1

L( M 3 )
0
1
1
1

L( M 4 )
0
0
0
1

a
4

71
Consider the language
i
i
L  {a : a  L( M i )}
L consists of the 1’s on the diagonal
72
1
2
3
4

a
a
L ( M1 )
0
1
0
1

L( M 2 )
1
0
0
1

L( M 3 )
0
1
1
1

L( M 4 )
0
0
0
1

a
3
a
4
L  {a , a ,}
73
Consider the language
i
i
i
i
L
L  {a : a  L( M i )}
L  {a : a  L( M i )}
L
consists from of 0’s on the diagonal
74
1
a
a
2
a
L ( M1 )
0
1
0
1

L( M 2 )
1
0
0
1

L( M 3 )
0
1
1
1

L( M 4 )
0
0
0
1

1
3
a
4

2
L  {a , a ,}
75
Theorem:
Language L is not recursively enumerable.
76
Proof:
Assume on the contrary that
L is recursively enumerable
There must exist some machine M k
that accepts L
L( M k )  L
77
1
a
a
2
a
L ( M1 )
0
1
0
1

L( M 2 )
1
0
0
1

L( M 3 )
0
1
1
1

L( M 4 )
0
0
0
1

3
a
4

Question: M k  M1 ?
78
1
a
a
2
a
L ( M1 )
0
1
0
1

L( M 2 )
1
0
0
1

L( M 3 )
0
1
1
1

L( M 4 )
0
0
0
1

3
a
4

1
Answer:
M k  M1
a  L( M k )
1
a  L ( M1 )
79
1
a
a
2
a
L ( M1 )
0
1
0
1

L( M 2 )
1
0
0
1

L( M 3 )
0
1
1
1

L( M 4 )
0
0
0
1

3
a
4

Question: M k  M 2 ?
80
1
a
a
2
a
L ( M1 )
0
1
0
1

L( M 2 )
1
0
0
1

L( M 3 )
0
1
1
1

L( M 4 )
0
0
0
1

3
a
4

2
Answer:
Mk  M2
a  L( M k )
2
a  L( M 2 )
81
1
a
a
2
a
L ( M1 )
0
1
0
1

L( M 2 )
1
0
0
1

L( M 3 )
0
1
1
1

L( M 4 )
0
0
0
1

3
a
4

Question: M k  M 3 ?
82
1
a
a
2
a
L ( M1 )
0
1
0
1

L( M 2 )
1
0
0
1

L( M 3 )
0
1
1
1

L( M 4 )
0
0
0
1

3
a
4

3
Answer:
M k  M3
a  L( M k )
3
a  L( M 3 )
83
Similarly:
M k  Mi
for any
i
Because either:
i
a  L( M k )
i
a  L( M i )
i
or
a  L( M k )
i
a  L( M i )
84
Therefore the machine
M kcannot exist
CONTRADICTION!!!
The language L
is not recursively enumerable.
End of proof
85
Observation:
There is no algorithm that
describes L
(otherwise it would be accepted by
a Turing Machine)
86
A Language
which is Recursively Enumerable
and not Recursive
87
We want to find a language which
Is recursively
enumerable
There is a
Turing Machine
that accepts
the language
But not
recursive
The machine
doesn’t
necessarily halt
on any input
88
We will prove that the language
i
i
L  {a : a  L( M i )}
Is recursively enumerable
but not recursive.
89
Theorem:
The language
i
i
L  {a : a  L( M i )}
is recursively enumerable
90
Proof:
We will give a Turing Machine that
accepts L
91
Turing Machine that accepts L
For any input string w
• Write
wa
i
• Find Turing machine M i
(using the enumeration procedure
for Turing Machines)
• Simulate M i on input ai
• If M i accepts, then accept w
End of proof
92
Observation:
Recursively enumerable
i
i
L  {a : a  L( M i )}
Not recursively enumerable
i
i
L  {a : a  L( M i )}
(Thus, not recursive)
93
Theorem:
The language
i
i
L  {a : a  L( M i )}
is not recursive.
94
Proof:
Assume on the contrary that L is recursive.
Then L is recursive:
Take the Turing Machine M that accepts L
M halts on any input
If M accepts then reject
If M rejects then accept
95
Therefore:
L recursive
But we know:
L not recursively enumerable
thus, not recursive
CONTRADICTION!
Therefore, L is not recursive
End of proof
96
Non Recursively Enumerable
L
Recursively Enumerable
L
Recursive
97