TDDC95
Introduction to the Theory of Computation
Lecture 4
Gustav Nordh, IDA
gustav.nordh@liu.se
2011-09-14
Outline
Turing Machines
The Church-Turing Thesis
The method of diagonalization
Problems that cannot be solved by computers
Undecidability of the Halting Problem
Reductions
The Incompleteness Theorem via Undecidability
Computability
What can be computed?
Computability
What can be computed?
read/write tape
0 1 1 0 ···
control
Computability
What can be computed?
read/write tape
0 1 1 0 ···
control
Computability
What can be computed?
read/write tape
0 1 1 0 ···
control
Computability
What can be computed?
read/write tape
0 1 1 0 ···
control
Computability
What can be computed?
read/write tape
0 1 1 0 ···
control
Computability
What can be computed?
read/write tape
1 1 1 0 ···
control
Computability
What can be computed?
read/write tape
1 0 1 0 ···
control
Turing machine
Alan Turing (1912-1954)
Definition of a Turing machine
Definition
A Turing machine is a 7-tuple (Q, Σ, Γ, δ, q0 , qaccept , qreject )
where
Q is the finite set of states
Σ is the finite input alphabet not containing the blank
symbol B
Γ is the finite tape alphabet where B ∈ Γ and Σ ⊆ Γ
δ : Q × Γ → Q × Γ × {L, R} is the transition function
q0 ∈ Q is the start state
qaccept ∈ Q is the accept state
qreject ∈ Q is the reject state
Comparision with finite automata
A Turing machine can both write on the tape and read from
it
The read-write head can move both to the left and right
The tape is infinite
The special states for rejecting and accepting take effect
immediately
Turing machine computation
Initially the machine recieves the input on the leftmost part
of the tape
Computation proceeds according to the transition function
The computation continues until the machine enters the
accept or reject states at which point it halts.
The machine may continue forever without entering the
accept or reject states, in which case we say that the
machine loops.
Turing-recognizable, decidable
Definition
The collection of strings that a Turing machine M accepts is the
language recognized by M, denoted L(M)
Turing-recognizable, decidable
Definition
The collection of strings that a Turing machine M accepts is the
language recognized by M, denoted L(M)
Definition
A language is Turing-recognizable if some Turing machine
recognizes it
Turing-recognizable, decidable
Definition
The collection of strings that a Turing machine M accepts is the
language recognized by M, denoted L(M)
Definition
A language is Turing-recognizable if some Turing machine
recognizes it
Definition
A language is decidable if some Turing machine recognizes it
and rejects all strings that are not in the language
Turing machines, decidable
Definition
A language is decidable if some Turing machine recognizes it
and rejects all strings that are not in the language
Example
Consider a Turing machine M with Σ = {0, 1} that works as
follows: M accept all strings of even length and loop on all
strings of odd length.
Turing machines, decidable
Definition
A language is decidable if some Turing machine recognizes it
and rejects all strings that are not in the language
Example
Consider a Turing machine M with Σ = {0, 1} that works as
follows: M accept all strings of even length and loop on all
strings of odd length.
Is L(M) decidable?
Turing machines, decidable
Definition
A language is decidable if some Turing machine recognizes it
and rejects all strings that are not in the language
Example
Consider a Turing machine M with Σ = {0, 1} that works as
follows: M accept all strings of even length and loop on all
strings of odd length.
Is L(M) decidable?
YES! For example by the Turing machine M 0 which accept all
strings of even length and reject all strings of odd length.
Describing Turing machines
Machine code
Assembly code
Java code
Pseudo code
Algorithm description
Describing Turing machines
Formal description (e.g., State diagram)
Implementation level description
Algorithm description
Describing Turing machines: State diagram
Describing Turing machines: Implementation level
Example
Describe a Turing machine that recognizes the language
L = {0n 1n 2n | n ≥ 0}.
1
Scan the input from left to right and make sure it is of the
form 0∗ 1∗ 2∗ (if it is not, then reject)
2
Return the head to the left end of the tape
3
If there is no 0 on the tape, then scan right and check that
there are no 1’s and 2’s on the tape and accept (should a 1
or 2 be on the tape, then reject)
4
Otherwise, cross of the first 0 and continue to the right
crossing of the first 1 and the first 2 that is found (should
there be no 1 or no 2 on the tape, then reject)
5
Go to Step 2
Describing Turing machines: Algorithm description
An algorithm description is a list of simple instructions for
solving/computing some task.
Describing Turing machines: Algorithm description
An algorithm description is a list of simple instructions for
solving/computing some task.
If the goal of the algorithm description is to convince the
reader that the task can be solved/computed, then “simple
instructions” means “can be carried out by a Turing
machine”.
Describing Turing machines: Algorithm description
An algorithm description is a list of simple instructions for
solving/computing some task.
If the goal of the algorithm description is to convince the
reader that the task can be solved/computed, then “simple
instructions” means “can be carried out by a Turing
machine”.
Algorithm descriptions are similar to mathematical proofs
The goal of a mathematical proof is to convince the reader
that the truth of a mathematical statement follows from the
basic axioms.
The goal of an algorithm description is to convince the
reader that a task/problem can be solved by Turing
machines/computers.
Describing Turing machines: Algorithm description
Example
Describe an algorithm for recognizing the language
L = {0n 1n 2n | n ≥ 0}.
Describing Turing machines: Algorithm description
Example
Describe an algorithm for recognizing the language
L = {0n 1n 2n | n ≥ 0}.
1
Check that the input is of the form 0∗ 1∗ 2∗ . Then count the
number of 0’s, 1’s, and 2’s. If they are the same, accept.
Otherwise, reject.
Algorithm
Muhammad ibn Musa al-Khwarizmi (780-850)
Alternatives to Turing machines?
Why are Turing machines a good model for computation?
Alternatives to Turing machines?
Why are Turing machines a good model for computation?
There should be more powerful machines, right?
Alternatives to Turing machines?
Alonzo Church (1903-1995)
Church-Turing thesis
Intuitive notion of computation
equals
Turing machine computation
Consequences of the Church-Turing thesis
The details of the computational model are not important
Consequences of the Church-Turing thesis
The details of the computational model are not important
Opens up the possibility to prove that some problems are
not solvable by computers/Turing machines
Consequences of the Church-Turing thesis
The details of the computational model are not important
Opens up the possibility to prove that some problems are
not solvable by computers/Turing machines
Humans can be simulated by Turing machines!?
The universe is a Turing machine!?
Testing the Church-Turing thesis
Theorem
Nondeterministic Turing machines can be simulated by
deterministic Turing machines
Definition of a nondeterministic Turing machine
Definition
A nondeterministic Turing machine is a 7-tuple
(Q, Σ, Γ, δ, q0 , qaccept , qreject ) where
Q is the finite set of states
Σ is the finite input alphabet not containing the blank
symbol B
Γ is the finite tape alphabet where B ∈ Γ and Σ ⊆ Γ
δ : Q × Γ → P(Q × Γ × {L, R}) is the transition function
q0 ∈ Q is the start state
qaccept ∈ Q is the accept state
qreject ∈ Q is the reject state
Definition of a nondeterministic Turing machine
Definition
A nondeterministic Turing machine is a 7-tuple
(Q, Σ, Γ, δ, q0 , qaccept , qreject ) where
Q is the finite set of states
Σ is the finite input alphabet not containing the blank
symbol B
Γ is the finite tape alphabet where B ∈ Γ and Σ ⊆ Γ
δ : Q × Γ → P(Q × Γ × {L, R}) is the transition function
q0 ∈ Q is the start state
qaccept ∈ Q is the accept state
qreject ∈ Q is the reject state
A nondeterministic Turing machine accepts its input w if at least
one of the states explored is an accept state.
Configuration of a Turing machine
Given a Turing machine M operating on an input w: the current
state, current tape contents, and current position of the
read/write head is the current configuration of M.
Configuration of a Turing machine
Given a Turing machine M operating on an input w: the current
state, current tape contents, and current position of the
read/write head is the current configuration of M.
Example
00q5 10 represent the configuration where the tape contents is
0010, the state is q5 , and the position of the head is over the 1.
Configuration of a Turing machine
Given a Turing machine M operating on an input w: the current
state, current tape contents, and current position of the
read/write head is the current configuration of M.
Example
00q5 10 represent the configuration where the tape contents is
0010, the state is q5 , and the position of the head is over the 1.
The start configuration is q0 w
An accept configuration is one where the state is qaccept
A reject configuration is one where the state is qreject
Testing the Church-Turing thesis
Theorem
Nondeterministic Turing machines can be simulated by
deterministic Turing machines
Proof.
Given a nondeterministic Turing machine N we construct a
deterministic Turing machine D such that D accepts input w if
and only if N accepts w. D works as follows:
Testing the Church-Turing thesis
Theorem
Nondeterministic Turing machines can be simulated by
deterministic Turing machines
Proof.
Given a nondeterministic Turing machine N we construct a
deterministic Turing machine D such that D accepts input w if
and only if N accepts w. D works as follows:
1
Given input w. Beginning with the start configuration of N
on input w, D explores the computation tree of N on input
w.
2
If D encounters an accept configuration of N, then D
accepts w.
3
If D has explored the whole computational tree of N
without finding an accept configuration, then D rejects w.
Testing the Church-Turing thesis
Theorem
Nondeterministic Turing machines can be simulated by
deterministic Turing machines
Proof.
Given a nondeterministic Turing machine N we construct a
deterministic Turing machine D such that D accepts input w if
and only if N accepts w. D works as follows:
1
Given input w. Beginning with the start configuration of N
on input w, D explores the computation tree of N on input
w in breadth first manner (i.e., level by level).
2
If D encounters an accept configuration of N, then D
accepts w.
3
If D has explored the whole computational tree of N
without finding an accept configuration, then D rejects w.
Decidable languages
ADFA = {hB, wi | B is a DFA that accepts input string w}
Decidable languages
ADFA = {hB, wi | B is a DFA that accepts input string w}
Theorem
ADFA is decidable
Decidable languages
ADFA = {hB, wi | B is a DFA that accepts input string w}
Theorem
ADFA is decidable
Proof.
Let M be a Turing machine that works as follows:
1
Check that the input hB, wi is a legal encoding of a DFA B
and string w (otherwise reject)
2
Simulate B on input w
3
If the simulation ends in an accept state (of B), then M
accepts the input hB, wi
4
If the simulation ends in a nonaccepting state (of B), then
M rejects the input hB, wi
Diagonalization
Diagonalization
Georg Cantor (1845-1918)
Diagonalization
Theorem
Some languages are not Turing-recognizable
Diagonalization
Theorem
Some languages are not Turing-recognizable
Proof idea.
1
The set of all Turing machines is countable
2
The set of all languages is uncountable
3
Since each Turing machine recognize exactly one
language, there are languages that are not recognized by
any Turing machine
Diagonalization
Lemma
The set of all Turing machines is countable
Diagonalization
Lemma
The set of all Turing machines is countable
Proof.
1
The set of all strings Σ∗ (for any alphabet Σ) is countable
A list of all strings in Σ∗ can be written down by listing all
strings of length 0, length 1, length 2, ...
2
3
Each Turing machine M can be encoded as a string hMi
over Σ
By omitting those strings in Σ∗ which are not legal
encodings of Turing machines, we get a list of all Turing
machines
Diagonalization
Lemma
The set of all languages over Σ is uncountable
Diagonalization
Lemma
The set of all languages over Σ is uncountable
Proof idea.
1
Each language over Σ can be represented by its
characteristic sequence (an infinite binary sequence).
Diagonalization
Lemma
The set of all languages over Σ is uncountable
Proof idea.
1
Each language over Σ can be represented by its
characteristic sequence (an infinite binary sequence).
2
Each infinite binary sequence can be seen as a
characteristic sequence for a language over Σ. Hence,
there is a one-to-one correspondence between infinite
binary sequences and languages over Σ.
Diagonalization
Lemma
The set of all languages over Σ is uncountable
Proof idea.
1
Each language over Σ can be represented by its
characteristic sequence (an infinite binary sequence).
2
Each infinite binary sequence can be seen as a
characteristic sequence for a language over Σ. Hence,
there is a one-to-one correspondence between infinite
binary sequences and languages over Σ.
3
The set of infinite binary sequences is uncountable (by a
simple diagonalization proof)
Diagonalization
Lemma
The set of all languages over Σ is uncountable
Proof idea.
1
Each language over Σ can be represented by its
characteristic sequence (an infinite binary sequence).
2
Each infinite binary sequence can be seen as a
characteristic sequence for a language over Σ. Hence,
there is a one-to-one correspondence between infinite
binary sequences and languages over Σ.
3
The set of infinite binary sequences is uncountable (by a
simple diagonalization proof)
4
Hence, the set of all languages over Σ is uncountable
Diagonalization
Theorem
Some languages are not Turing-recognizable
Proof.
1
The set of all Turing machines is countable
2
The set of all languages is uncountable
3
Since each Turing machine recognize exactly one
language, there are languages that are not recognized by
any Turing machine
Undecidability
Undecidability
Undecidability
Hilbert’s 10th problem:
Construct an algorithm that given a polynomial, determine
whether the polynomial has an integral root
Undecidability
Hilbert’s 10th problem:
Construct an algorithm that given a polynomial, determine
whether the polynomial has an integral root
Example
6x 3 yz 2 + 3xy 2 − x 3 − 10
Undecidability
Hilbert’s 10th problem:
Construct an algorithm that given a polynomial, determine
whether the polynomial has an integral root
Example
6x 3 yz 2 + 3xy 2 − x 3 − 10
root: x = 5, y = 3, z = 0
Undecidability
Hilbert’s 10th problem:
Construct an algorithm that given a polynomial, determine
whether the polynomial has an integral root
Example
6x 3 yz 2 + 3xy 2 − x 3 − 10
root: x = 5, y = 3, z = 0
1970: Hilbert’s 10th problem is undecidable!
Undecidability
Undecidability
Software verification: Given a computer program and a
specification of how the program is supposed to work,
verify that the program performs as specified.
Undecidability
Software verification: Given a computer program and a
specification of how the program is supposed to work,
verify that the program performs as specified.
Impossible! This problem is not solvable by computers
Undecidability
Construct a tool/debugger that can tell whether a Java
program will go into an infinite loop on input w.
Undecidability
Construct a tool/debugger that can tell whether a Java
program will go into an infinite loop on input w.
Impossible! This problem is undecidable
Undecidability
Construct a tool/debugger that can tell whether a Java
program will go into an infinite loop on input w.
Impossible! This problem is undecidable
There is no Turing machine U that given a Turing machine
M and string w can determine whether M will loop on input
w
Undecidability
ATM = {hM, wi | M is a Turing machine that accepts w}
Undecidability
ATM = {hM, wi | M is a Turing machine that accepts w}
Theorem
ATM is Turing-recognizable
Undecidability
ATM = {hM, wi | M is a Turing machine that accepts w}
Theorem
ATM is Turing-recognizable
Proof.
Let U be a Turing machine that works as follows:
1
Check that the input hM, wi is a legal encoding of a Turing
machine M and string w (otherwise reject)
2
Simulate M on input w
3
If M enters the accept state, then U accepts hM, wi
4
If M enters the reject state, then U rejects hM, wi
Undecidability
ATM = {hM, wi | M is a Turing machine that accepts w}
Theorem
ATM is Turing-recognizable
Proof.
Let U be a Turing machine that works as follows:
1
Check that the input hM, wi is a legal encoding of a Turing
machine M and string w (otherwise reject)
2
Simulate M on input w
3
If M enters the accept state, then U accepts hM, wi
4
If M enters the reject state, then U rejects hM, wi
Note that U does not decide ATM !
Undecidability: Universal Turing machine
Undecidability: Universal Turing machine
Undecidability: Universal Turing machine
#include<iostream>
#define Z atoi(b[a++])
int main(int a,char**b){
int c=a=1,d=0,e=2;
char*f,*g=0,*h,*i=0;
while(c){if(i<=g|i>=g+e)f=g,g=(char*)
calloc(e,2),i=f?(memcpy(g+e/2,f,e),
i-f+e/2+g):g+e,h=f?h-f+e/2+g:i,e*=2;
a=Z-c|Z-*i%2?a+3:(c=Z,*i=Z|48,i+=Z,h-=i<h,++d,1);
}printf("%s\n%d",h,d);}
Undecidability of the halting problem
ATM = {hM, wi | M is a Turing machine that accepts w}
Undecidability of the halting problem
ATM = {hM, wi | M is a Turing machine that accepts w}
Theorem
ATM is undecidable
Undecidability of the halting problem
ATM = {hM, wi | M is a Turing machine that accepts w}
Theorem
ATM is undecidable
Proof.
1
2
Assume that ATM is decidable by a Turing machine H
Construct a new Turing machine D which takes hMi as
input and works as follows:
Run H on hM, hMii and output the opposite of what H
outputs
3
Running D on input hDi results in a contradiction because
D rejects hDi if D accepts hDi, and D accepts hDi if D
rejects hDi
An explicit language which is not Turing-recognizable
ATM = {w | w ∈
/ ATM }
An explicit language which is not Turing-recognizable
ATM = {w | w ∈
/ ATM }
Theorem
ATM is not Turing-recognizable
An explicit language which is not Turing-recognizable
ATM = {w | w ∈
/ ATM }
Theorem
ATM is not Turing-recognizable
Proof.
1
Assume with the aim of reaching a contradiction that ATM
is Turing-recognizable.
An explicit language which is not Turing-recognizable
ATM = {w | w ∈
/ ATM }
Theorem
ATM is not Turing-recognizable
Proof.
1
Assume with the aim of reaching a contradiction that ATM
is Turing-recognizable.
2
Let M1 be a Turing machine recognizing ATM and M2 a
Turing machine recognizing ATM .
An explicit language which is not Turing-recognizable
ATM = {w | w ∈
/ ATM }
Theorem
ATM is not Turing-recognizable
Proof.
1
Assume with the aim of reaching a contradiction that ATM
is Turing-recognizable.
2
Let M1 be a Turing machine recognizing ATM and M2 a
Turing machine recognizing ATM .
3
On input w run both M1 and M2 on w in parallel
An explicit language which is not Turing-recognizable
ATM = {w | w ∈
/ ATM }
Theorem
ATM is not Turing-recognizable
Proof.
1
Assume with the aim of reaching a contradiction that ATM
is Turing-recognizable.
2
Let M1 be a Turing machine recognizing ATM and M2 a
Turing machine recognizing ATM .
3
On input w run both M1 and M2 on w in parallel
4
If M1 accepts, then reject. If M2 accepts, then accept.
5
This shows that ATM is decidable, which is a contradiction
Reductions
Reductions
Definition
A function f : Σ∗ → Σ∗ is computable if some Turing machine M
on every input w halts with f (w) on its tape
Reductions
Definition
Language A is mapping reducible to language B if there is a
computable function f : Σ∗ → Σ∗ such that for every w
w ∈ A iff f (w) ∈ B
The function f is called a reduction from A to B
If A is mapping reducible to B then we write A ≤m B
Reductions
Theorem
If B is decidable and A ≤m B, then A is decidable
Reductions
Theorem
If B is decidable and A ≤m B, then A is decidable
Proof.
Let MB be a Turing machine that decides B and f the reduction
from A to B. Given input w:
1
Compute f (w)
2
Run MB on f (w), accept if MB accepts f (w), and reject if
MB rejects f (w)
Reductions
Theorem
If A is undecidable and A ≤m B, then B is undecidable
Reductions
Theorem
If A is undecidable and A ≤m B, then B is undecidable
Proof.
Assume with the aim of reaching a contradiction that B is
decidable. Let MB be a Turing machine that decides B and f the
reduction from A to B. Given input w:
1
Compute f (w)
2
Run MB on f (w), accept if MB accepts f (w), and reject if
MB rejects f (w)
3
This shows that A is decidable, which is a contradiction
Reductions
Theorem
If B is Turing-recognizable and A ≤m B, then A is
Turing-recognizable
Reductions
Theorem
If A is not Turing-recognizable and A ≤m B, then B is not
Turing-recognizable
Reductions
ETM = {hMi | M is a Turing machine and L(M) = ∅}
Reductions
ETM = {hMi | M is a Turing machine and L(M) = ∅}
Theorem
ETM is undecidable
Reductions
ETM = {hMi | M is a Turing machine and L(M) = ∅}
Theorem
ETM is undecidable
Proof.
By reduction from ATM (Theorem 5.2 in Sipser)
Reductions
EQTM = {hM1 , M2 i | M1 and M2 are TMs and L(M1 ) = L(M2 )}
Reductions
EQTM = {hM1 , M2 i | M1 and M2 are TMs and L(M1 ) = L(M2 )}
Theorem
EQTM is undecidable
Reductions
EQTM = {hM1 , M2 i | M1 and M2 are TMs and L(M1 ) = L(M2 )}
Theorem
EQTM is undecidable
Proof.
ETM ≤m EQTM :
Reductions
EQTM = {hM1 , M2 i | M1 and M2 are TMs and L(M1 ) = L(M2 )}
Theorem
EQTM is undecidable
Proof.
ETM ≤m EQTM :
1
Let M2 be a Turing machine such that L(M2 ) = ∅
Reductions
EQTM = {hM1 , M2 i | M1 and M2 are TMs and L(M1 ) = L(M2 )}
Theorem
EQTM is undecidable
Proof.
ETM ≤m EQTM :
1
Let M2 be a Turing machine such that L(M2 ) = ∅
2
Given a Turing machine M, let f (hMi) = hM, M2 i
Reductions
EQTM = {hM1 , M2 i | M1 and M2 are TMs and L(M1 ) = L(M2 )}
Theorem
EQTM is undecidable
Proof.
ETM ≤m EQTM :
1
Let M2 be a Turing machine such that L(M2 ) = ∅
2
Given a Turing machine M, let f (hMi) = hM, M2 i
3
hMi ∈ ETM iff L(M) = ∅ iff L(M) = L(M2 ) iff
hM, M2 i ∈ EQTM
Incompleteness Theorem via Undecidability
Kurt Gödel (1906-1978)
Incompleteness Theorem via Undecidability (S, 6.2)
Theorem
(Informally) There are true mathematical statements that
cannot be proved
Incompleteness Theorem via Undecidability (S, 6.2)
Theorem
(Informally) There are true mathematical statements that
cannot be proved
Proof idea.
1
The language of all true mathematical statements is
undecidable (by reduction from ATM )
Incompleteness Theorem via Undecidability (S, 6.2)
Theorem
(Informally) There are true mathematical statements that
cannot be proved
Proof idea.
1
The language of all true mathematical statements is
undecidable (by reduction from ATM )
2
The language of all provable statements is Turing
recognizable by a Turing machine M
Incompleteness Theorem via Undecidability (S, 6.2)
Theorem
(Informally) There are true mathematical statements that
cannot be proved
Proof idea.
1
The language of all true mathematical statements is
undecidable (by reduction from ATM )
2
The language of all provable statements is Turing
recognizable by a Turing machine M
3
Assume that all true statements are provable
Incompleteness Theorem via Undecidability (S, 6.2)
Theorem
(Informally) There are true mathematical statements that
cannot be proved
Proof idea.
1
The language of all true mathematical statements is
undecidable (by reduction from ATM )
2
The language of all provable statements is Turing
recognizable by a Turing machine M
3
Assume that all true statements are provable
4
Given a statement Φ, run M in parallel on Φ and ¬Φ. One
of them is true and thus (by assumption) provable. If Φ is
provable then Φ is true and if ¬Φ is provable then Φ is false.
Incompleteness Theorem via Undecidability (S, 6.2)
Theorem
(Informally) There are true mathematical statements that
cannot be proved
Proof idea.
1
The language of all true mathematical statements is
undecidable (by reduction from ATM )
2
The language of all provable statements is Turing
recognizable by a Turing machine M
3
Assume that all true statements are provable
4
Given a statement Φ, run M in parallel on Φ and ¬Φ. One
of them is true and thus (by assumption) provable. If Φ is
provable then Φ is true and if ¬Φ is provable then Φ is false.
5
So M decides the truth of Φ. This is a contradiction (with 1
above)
Summary on Computability (what to focus on)
Turing machines and the Church-Turing thesis
Diagonalization and the undecidability of the halting
problem
Mapping reducibility