1 / 32
Computing and Complexity Theory
Chapter 3: Decidability
J. Roland
Quantum Information & Communication
2 / 32
Outline
1
Computability theory
Introduction
Decidable languages
Undecidability
Computability theory
Outline
1
Computability theory
Introduction
Decidable languages
Undecidability
Introduction
3 / 32
Computability theory
Introduction
Models of Computation
x1
x2
x3
...
xn
0: LOAD 1
1: ADD =3
2: STORE 7
3: LOAD =77
4: STORE 7
5: JUMP 100
...
Program
Counter
x1
Input tape
x2
x3
...
x2
y3
...
Input tape
Memory
r0
r0
r1
Program
Counter
r2
r3
r4
...
y1
xn
Memory
Program
Output tape
r1
LOAD 1
r2
ADD =3
r3
STORE 7
r4
LOAD =77
...
y1
x2
y3
...
Output tape
4 / 32
Computability theory
Introduction
Theory of Computation
I
f
O
Turing Machines (and, by equivalence, also
RAMs and RASPs) define partial functions
~M  : Σ∗ → Γ∗ ∪ {⊥}
┴
I
f
O
If a TM M halts on all inputs, then it defines a
total function
~ M  : Σ ∗ → Γ∗
In the Theory of Computation, we want to get a better insight into the total functions that can be defined by means of TMs.
5 / 32
Computability theory
Introduction
Computational problems
Definition
A computational problem can be formalized
as a triple (Σ, Γ, P ) where
Σ is a finite input alphabet;
Γ is a finite output alphabet;
P : Σ∗ → Γ∗ is a total function.
Example: On input x:
if x = hG , s , t i, i.e., x is the string representation of graph G with s and t two
nodes in G, then P (x ) needs to be the shortest path from s to t in G;
if x is not such a valid representation, then P (x ) = ε. (bogus output)
So, the triple (Σ, Γ, P ) is a declarative specification of what an algorithm should
do.
Question
Given (Σ, Γ, P ) does there exist an algorithm (i.e., a TM M) that solves this problem
(i.e., ~M (x ) = P (x ) for all x ∈ Σ∗ )?
6 / 32
Computability theory
Introduction
7 / 32
Decision problems
Definition
A decision problem is a language (set of
string) L ⊆ Σ∗ over some alphabet Σ∗ .
TM M solves decision problem L if it
decides L , i.e., if L(M ) = L and M halts
on every input.
So, L is solvable if it is decidable.
Decision problems are boolean computational problems!
Decision problem L is decidable if and only if the computational problem (Σ, {TRUE}, P )
is solvable where
P (x ) = TRUE if x ∈ L ;
P (x ) = ε otherwise.
Computability theory
Introduction
Computational vs decision problems
Definition
Given a a computational problem (Σ, Γ, P ), we
define the following decision problem
LP := {x #y | x ∈ Σ∗ , y ∈ Γ∗ , P (x ) = y }
Theorem
If LP is undecidable then P is not algorithmically solvable.
Proof (by contraposition)
Given a TM M for P, construct a TM N that decides LP as follows. On input x #y:
N first runs M as a subroutine to compute P (x )
then compares y with P (x ) and accepts if these are equal; rejects otherwise.
Question
Can we also deduce a TM that solves (Σ, Γ, P ) from a TM that decides LP ?
8 / 32
Computability theory
Introduction
Other direction?
Definition
Given a a computational problem (Σ, Γ, P ), we
define the following decision problem
LP := {x #y | x ∈ Σ∗ , y ∈ Γ∗ , P (x ) = y }
Question
Can we also deduce a TM that solves (Σ, Γ, P ) from a TM that decides LP ?
Example:
Let (Σ, Γ, P ) be the computational problem that on input hG , s , t i computes the
shortest path from s to t in graph G.
If you have a TM N that decides LP , then you can create M that solves (Σ, Γ, P ):
M simply enumerates all simple paths of G, one by one, and for every such
path y, it calls N on hG , s , t i#y.
I It outputs y if N accepts this input, otherwise it continues with the next path.
I
9 / 32
Computability theory
Introduction
Computational vs decision problems (2)
Definition
Given a a computational problem (Σ, Γ, P ), we
define the following decision problem
LP := {x #y | x ∈ Σ∗ , y ∈ Γ∗ , P (x ) = y }
Theorem
If LP is decidable then P is algorithmically solvable.
Proof
Given a TM N that decides LP , construct a TM M that solves (Σ, Γ, P ) as follows.
On input x:
Enumerate all y ∈ Γ∗ , one by one; first those of length 0, then those of length 1,
then those of length 2, . . . .
While enumerating, for each current y, run N on x #y; if N accepts, output y and
accept, otherwise continue with the next y.
10 / 32
Computability theory
Introduction
Therefore ...
In what follows, we focus exclusively on decision problems.
11 / 32
Computability theory
Outline
1
Computability theory
Introduction
Decidable languages
Undecidability
Decidable languages
12 / 32
Computability theory
Decidable languages
Examples of decidable languages
See book section 4.1
ADFA = {hB , w i | B is a DFA that accepts input string w }.
ANFA = {hB , w i | B is a NFA that accepts input string w }.
ACFG = {hB , w i | B is a CFG that generates string w }.
...
13 / 32
Computability theory
Outline
1
Computability theory
Introduction
Decidable languages
Undecidability
Undecidability
14 / 32
Computability theory
Undecidability
The canonical undecidable language: ATM
Definition
Define ATM := {hM , w i | TM M accepts string w }.
Theorem
ATM is undecidable.
Proof
(See later, we are not ready to prove this yet)
15 / 32
Computability theory
Undecidability
ATM is (at least) Turing-recognizable
Definition
Define ATM := {hM , w i | TM M accepts string w }.
Theorem
ATM is Turing-recognizable.
Proof
On input hM , w i where M is a TM and w is a string:
1
Simulate M on input w. (similar to how, e.g., we simulated a RASP on a
RAM)
2
If M ever enters its accept state, accept; if M ever enters its reject state,
reject.
Note that this machine loops on input hM , w i if M loops on w, and hence
does not decide ATM .
16 / 32
Computability theory
Undecidability
Perhaps surprisingly.
Theorem
Some languages are not Turing-recognizable.
2 Key observations:
1
A TM can only recognize a single language
Language L1
rec
ogn
i
ze
TM M
,
Language L2
2
There are more languages than TMs
ize
ogn
rec
17 / 32
Computability theory
Undecidability
Huh?
What do you mean there are more languages
than TMs?
There are infinitely many languages
There are infinitely many TMs
Question
How do you compare the size of two infinite sets?
Answer:
They have the same size if it is possible to construct a bijection (or one-to-one and
onto correspondence) between them.
Bijection f : N → E:
n
f(n)
Example:
1
2
The set N = {1, 2, 3, . . . } of natural
2
4
numbers and the set of even
3
6
numbers E = {2, 4, 6, 8, . . . } have the
4
8
same size.
...
...
18 / 32
Computability theory
Undecidability
Countable and uncountable sets
What do you mean there are more languages
than TMs?
There are infinitely many languages
There are infinitely many TMs
Definition
A set A is countable if it is finite or it has the same size as N, the set of natural
numbers.
An infinite set that is not countable is called uncountable. Such a set is simply
too big compared to N.
We will show:
The set {M1 , M2 , . . . } of all TMs is countable.
The set {L1 , L2 , . . . , } of all languages is uncountable.
19 / 32
Computability theory
Undecidability
20 / 32
Example of countable set
Theorem
n
The set Q+ = { m
| n ∈ N, m ∈ N} of all positive rational numbers is countable.
Proof
To find a correspondence with N, it suffices to find a way to list (enumerate) the elements of Q
without repetition. Then the first element hence listed corresponds to 1, the second element to 2,
and so on.
The listing is as follows
Computability theory
Undecidability
21 / 32
The set of TMs is countable
Proposition
The set Σ∗ of all strings over Σ is countable, for any finite alphabet Σ.
Proof
To find a correspondence with N, it suffices to find a way to list (enumerate) the elements of Σ∗
without repetition. Then the first element hence listed corresponds to 1, the second element to 2,
and so on.
To list the elements of Σ∗ , first enumerate all strings over Σ∗ of length 0, then all strings of length
1, then all strings of length 2, . . . .
Corollary
The set {M1 , M2 , . . . } of all TMs is countable
Proof
Every TM M can be represented as a string hM i over a finite alphabet Σ.
If when listing the elements of Σ∗ we simply omit those strings that are not legal encodings of
TMs, we get a listing of {M1 , M2 , . . . }.
Computability theory
Undecidability
Example of uncountable set
Theorem
The set R of real numbers is uncountable.
Proof (by contradiction)
Suppose that f is a bijection from N to R.
We construct x ∈ R that is not paired with any element of N.
I
Idea We choose the nth digit of x to the right of the decimal point to be different from the
corresponding digit of f (n)
I
Since x is different from f (n) for any n, it cannot appear in the list
Therefore f cannot be a bijection, which yields the desired contradiction.
22 / 32
Computability theory
Undecidability
The set of infinite binary sequences is uncountable
Definition
An infinite binary sequence is an unending sequence of 0s and 1s. Let B be the set of all infinite
binary sequences.
Proposition
The set B of all infinite binary sequences is uncountable.
Proof
Completely similar to how we showed that R is uncountable.
23 / 32
Computability theory
Undecidability
24 / 32
The set of languages is uncountable
Theorem
The set L of all languages over finite alphabet Σ is uncountable.
Proof
We show that L and B have the same size. Since B is uncountable, so is L.
Let s1 , s2 , . . . , be a listing of Σ∗ (this exists since Σ∗ is countable)
To each language L ⊆ Σ∗ we associate a unique sequence in B.
I
I
The i-th bit in this sequence is 1 if si ∈ A and 0 if si < A .
Call this sequence the characteristic sequence of L , denoted χL .
Then f : L → B such that f (L ) is the characteristic sequence of L is a bijection between L and B.
Σ∗
L
χL
=
=
=
{
{
ε,
0
0,
0,
1
1,
0
00,
00,
1
01,
01,
1
10,
11,
0
0
000,
000,
1
001,
001,
1
...
...
...
}
}
Computability theory
Undecidability
In conclusion
Theorem
Some languages are not Turing-recognizable
Proof
The set {M1 , M2 , . . . } of all TMs is countable.
The set {L1 , L2 , . . . , } of all languages (decision problems) is uncountable.
25 / 32
Computability theory
Undecidability
26 / 32
Undecidability of ATM
Theorem
ATM = {hM , w i | TM M accepts string w } is undecidable
Proof (by contradiction)
Assume, for the purpose of obtaining a contradiction, that TM H decides ATM .
Then,



accept if M accepts w
H (hM , w i) = 

reject if M rejects w or loops on w
Now construct a new TM D that uses H as a subroutine. D = “On input hM i with
M a TM:
1
2
Run H on input hM , hM ii.
Output the opposite of what H outputs. That is, if H accepts, reject ; and if H
rejects, accept .”
Note that by definition, D always halts since H always halts
Computability theory
Undecidability
27 / 32
Undecidability of ATM (2)
Theorem
ATM = {hM , w i | TM M accepts string w } is undecidable
Proof (cont’d)
So,



accept if M rejects hM i or loops on hM i
D (hM i) = 

reject if M accepts hM i
Question: what does D do when run on hD i?
Possiblity 1: D accepts hD i (that is, D (hD i) = accept)
Then, D rejects hD i or loops on hD i, that is, D (hD i) = reject. Contradiction!
I Possibility 2: D rejects hD i (that is, D (hD i) = reject)
Then, D accepts hD i, that is, D (hD i) = accept. Contradiction!
I
Computability theory
Undecidability
Interpretation in terms of diagonalization
Theorem
ATM = {hM , w i | TM M accepts string w } is undecidable.
Proof
This is essentially a diagonalization argument.
Behavior of Mi on input hMj i
28 / 32
Computability theory
Undecidability
Interpretation in terms of diagonalization
Theorem
ATM = {hM , w i | TM M accepts string w } is undecidable.
Proof
This is essentially a diagonalization argument.
Behavior of H on hMi , hMj ii
28 / 32
Computability theory
Undecidability
Interpretation in terms of diagonalization
Theorem
ATM = {hM , w i | TM M accepts string w } is undecidable.
Proof
This is essentially a diagonalization argument.
D (hMi i) is the opposite of H (hMi , hMi ii)
28 / 32
Computability theory
Undecidability
co-Turing recognizability
Definition
Let L be a language over finite alphabet Σ∗ . The complement of L is the language
L = Σ∗ \ L , which consists of all strings not in L .
Definition
A language L is called co-Turing recognizable if L is Turing recognizable.
Theorem
A language L is decidable iff both L and L are Turing recognizable.
29 / 32
Computability theory
Undecidability
Decidability and recognizability
Theorem
A language L is decidable iff both L and L are Turing recognizable.
Proof
⇒ If L is decidable, then so is L . Moreover, any language that is decidable is also
Turing-recognizable.
⇐ Let M1 be a recognizer for L and M2 a recognizer for L . The following TM M is a
decider for L :
M = “On input w:
1
2
Run both M1 and M2 on input w in parallel (example: use two tapes, alternate
between running one step of M1 on tape 1 and one step of M2 on tape 2)
If at any step, M1 accepts, accept; and if at any step M2 accepts, reject .”
30 / 32
Computability theory
Undecidability
A Turing-Unrecognizable language
Corollary
ATM is not Turing-recognizable.
Proof (by contradiction)
Assume, for contradiction, that ATM is Turing-recognizable
We have previously proved that ATM is Turing-recognizable
Therefore, by the previous theorem, ATM is decidable
But we have previously proved that ATM is undecidable. Contradiction
31 / 32
Computability theory
Undecidability
References
These slides: published on the Université Virtuelle
I
Based on slides by S. Vansummeren
Introduction to the Theory of Computation, M. Sipser (3rd Edition International Edition, Cengage Learning, 2013)
I
Section 4.2 Undecidability
32 / 32