Lecture 25:
Turing Machines
Some Languages are not Context-Free
At the end of the last section, we used the Pumping Lemma for Context Free Languages to prove
that there are languages that are not context free.
This means that languages exist for which there is no corresponding push-down automata that
recognizes them and there is no context-free grammar (CFG) that generates them.
Just as we did when we found non-regular languages, we need to devise a new model of
computation with sufficient computational power to recognize and/or generate members of the
class (or classes) of languages that are not context free.
Introducing
the Turing Machine
Our new, more powerful model of computation is the Turing Machine (TM). A TM is made up
of a finite set of states and transitions called the state control (similar to the state controls in
FSA's and PA's), but there are a few differences which we will enumerate here:
1. A TM can write to the input string as well as read from it. We will refer to this
I/O string as a symbol tape.
2. The TM has a read/write head that can move in both directions on the tape.
3. The tape is infinite. (Although the input strings were also arbitrarily large in
FSA and PA they could not use them as a data store.
4. Special states for accepting or rejecting the input string take effect immediately.
That is, we don't necessarily have to wait until we reach the end of the input string
in order to make a decision to accept or reject a candidate member.
Example
Before we give a formal definition of TMs we will look at an example TM in action.
Consider the language B={w#w|w is in {0,1}*}. In words we say that B is the set of all
strings over the alphabet {#,0,1} that is comprised of two identical strings of 0's and 1's
separated by a # symbol.
Since we do not know the length of the candidate strings we will design our TM to mark a
symbol (0 or 1) move to the corresponding position on the other side of the # symbol and
verify that the same symbol is in this position. This symbol is then marked and we return to
the next symbol in the first string of 0's and 1's.
This process is repeated until we consume all the 0's and 1's or until a non-matching symbol
is found.
Turing Machine
Operating on the Symbol Tape
Another example
Consider a TM that recognizes the language consisting of all strings of 0's whose length is an
integer power of 2.
n
A={0 2 | n >=0}.
Let's take a look at the state control for this TM.
start state
reject state
transition rule
accept
state
cur_state,new_state,read_symbol,write_symbol,move_dir
From the start state we move to state q2 on the first zero, writing a blank _ in its place and
moving the read/write head to the right (R).
From q2 if we encounter a blank _ we accept immediately since the string 0 contains 20=1
zeros. If we encounter another 0 we write an x and move to the right (R).
From here we alternate between states q3
and q4 replacing every other 0 with an x
until we reach a blank _
If we encounter a blank _ while in q4 we reject
immediately since we have encountered an odd
number of 0's. If we encounter a blank while in q3
we have encountered an even number of 0's so we
move to state q5, where we return to the beginning of
the input string, replacing the first 0 with an x.
Starting over we return to the q3, q4 cycle, this time ignoring the x's encountered and replacing
every other 0 with an x. This process in repeated until all the 0's are consumed or until an odd
number of remaining 0's is encountered. The only number of 0's that can be repeated divided
by two until a single 0 remains is an integer power of 2 number of 0's.
We have seen two simple examples of TM's.
Now it is time to introduce a more complete definition of Turing Machines.
Turing Machine
a formal definition
Formally a Turing Machine (TM) is a 7-tuple (Q,S,G,d,qo,qa,qr), where Q,S,G are all finite
sets and,
1. Q is the set of states.
2. S is the input alphabet not containing the special blank symbol _,
3. G is the tape alphabet, including _ and S is a subset of G.
4. d: QxG->QxGx{L,R} (L and R indicate the direction to move the read/write head).
5. q0 is in Q and is the start state.
6. qaccept or qa is in Q and is the accept state (only one needed).
7. qreject or qr is in Q and is the reject state (only one needed).
Turing Machine Behavior
A TM receives input w = w1w2...wn (i.e. strings from S*) on the leftmost n positions of the
tape. Note that the tape is infinite (semi-infinite, actually) and the the remaining positions are
filled with blanks.
This is important to the computational power of the TM since it gives an unlimited amount of
storage space to hold the results of computations.
The set of input symbols S does not include the blank symbol _ so that the TM can know when
it has reached the end of the input string. As the TM computes, its current state changes
according to the transition rules.
Since the TM can write on the tape, it can store information there as well.
Finally the read/write head of the TM can be in any position on the tape. Any of these factors
can affect the eventual behavior of the TM so we define the configuration of the TM as its
(1) current state,
(2) current tape contents and the
(3) current read/write head position.
These three elements completely define the current configuration of the TM so that, starting
from this configuration, the TM will always exhibit the same behavior.
Executing a Turing Machine Program
In the start configuration, the read/write head of the TM is point to the first symbol on the tape,
the input tape is initialized with the input string followed by all blanks and the TM state control
is in the start state q1.
In an accepting configuration the current state of the configuration is qaccept. In a rejecting
configuration the current state is qreject.
Accepting and rejecting configuration are halting configurations, which means that the TM
stops when entering one of these configurations.
A TM accepts input w is a sequence of configurations exists starting at the start state, ending in
the accept state and the sequence of configurations are follow valid transitions.
The collection of strings accepted by some TM M is said to be the language of M also denoted
by L(M).
Turing-recognizable & Turing-decidable
Definition: A language is Turing-recognizable is some TM recognizes it or accepts it.
Definition: A language is Turing-decidable or simply decidable if some TM either
accepts it or rejects it.
A language is decidable if there exits a TM which eventually halts for every member of the
language input to the TM. Every decidable language is Turing-recognizable but not all
Turing-recognizable languages are decidable.
What does this mean? For one thing, it means that there are languages for which members
are accepted by a TM but some non-members may never to rejected by the TM. In other
words the TM that recognizes the language may never halt while scanning a non-member of
the language.
This does not mean that the TM does not halt for every non-member string, but rather that
there exists some non-member strings for which the TM does not halt.
Building Turing Machines
Usually we will employ pseudo code or textual descriptions of TM's as step algorithms rather than
explicitly listing the elements of the 7-tuple. Consider the following examples:
Example: Describe a TM that recognizes the language L={aibjck | ixj=k and i,j,k >= 1}. In words
L is the set of all strings of over the alphabet {a,b,c} in which a string of a's is followed by a string
of b's which is, in turn followed by a string of c's and the number of c's is equal to the product of
the numbers of a's and b's. Let M be a TM scanning the string w:
1. Scan the w to be sure that it is a member of a*b*c* and reject if it is not.
2. Return the beginning of w.
3. Move back and forth between the b's and the c's marking one of the b's and one of the c's
until all of the b's have been marked. If you exhaust the c's before this is accomplished then
reject w.
4. Reset all the b's back to their unmarked condition. Check to see if there are any more
a's. If not, then check to see if all the c's have been marked. If not reject otherwise
accept. If there are more a's then mark one of them and return the the beginning of the
string and repeat Step 3.
We can see that the computational model of a TM can embody the operations implied by this
description.
Example: Describe a TM that recognizes a language for which each member is a string of segments
of 0's and 1's separated by # symbols and for which no two segments are the same. This is also called
the element distinctness problem and is described symbolically as,
D={#X1#X2#...#Xn | Xi is in (0+1)* and Xi/=Xj for each i/=j}
In this example we will include two tape symbols # and _, called marked and unmarked. Let M be a
TM scanning the string w:
1. Mark the first symbol in w. If the first symbol is a blank _ then accept. If the first symbol is a #,
continue, otherwise reject.
2. Scan right to the next # and mark it. If a blank _ is encountered then accept since there was only
one segment of (0+1)*.
3. Scan back and forth comparing corresponding symbols and the two segments being
compared. When a non-matching symbol is encountered continue to Step 4. If the ends of two
segments being compared is reached this means the two segments are the same and we reject.
4. Unmark the rightmost marked # and mark the next # to the right if one exists and return to Step
3. If the marked # is at the end of the string, unmark the leftmost marked # and mark the next # to
the right. If this # is at the end of the string then accept. Else mark the next # and return to Step 3.
In this example we are comparing every segment to every other segment which would be an order
O(n2) algorithm if implemented on a standard (real) computer.
Multitape Turing Machines
A multitape Turing machine is the same as a single-tape TM except that it has multiple tapes to
read to and write from. One of the tapes holds the input string while the other can be used to hold
intermediate values during the TM's computation.
Theorem: Every multitape TM has an equivalent single tape TM.
This theorem says that we may assume multiple tapes and multiple read/write heads in order to
simplify a derivation or a demonstration that a TM recognizes a particular language. Just as with
the interchanging of DFA's, NFA's and regular expression's. We can use the TM model that best
fits a particular problem and be assured that we are working with the same computation power as
is exhibited by a single-tape TM.
Nondeterministic Turing Machines
A nondeterministic TM is a TM in which the transition function d is replaced with
d:QxG -> P(QxGx{L,R})
The computation of a nondeterministic TM is a tree whose branches represent different
sequences of valid state transitions through its state control. If all paths terminate then the TM
rejects the string being scanned. If one or more of the sequences leads to the accept state the
TM accepts.
Theorem: For every nondeterministic TM there is an equivalent deterministic TM.
Enumerators
An enumerator is a TM attached to a printing device. The TM can send strings to the printer to
be printed. The purpose of the enumerator is to list or enumerate all the members of a
language. An enumerator may generate members of a language in any order and it may repeat
some members before all the members of the language are enumerated.
Theorem: A language is Turing-recognizable if and only if some enumerator
enumerates it.
Comparing Turing Machines to Real Computers
There are many other models of computation equivalent in computational power to Turing
machines. The two essential elements are:
1. Unrestricted access to unlimited memory, and
2. The model of computation can perform a finite amount of work in a single step.
How do real-world computers compare to TM's? One major difference is that the program of a TM
is hard-wired into the state control while the program of a real-world computer is part of the
memory store. That is, the input includes the instructions for execution as well as the string being
evaluated. So what do we place into the state control? Rather than hard-wiring the program to
recognize a particular language we place the operations required to execute the instructions placed
on the tape. The portion of the real-world computer corresponding to the state control is the
CPU. The hard-wired component in the CPU perform the FETCH-DECODE-EXECUTE operations
on each instruction loaded in a designated location on the tape.
Hilbert's 10 Problem
In the 3rd century A.D., Diophantus of Alexandria
wrote a book on the subject of finding integer or
whole number solutions to arithmetic expressions.
In the summer of 1900, in a address to the Second
International Congress of Mathematicians, David
Hilbert introduced a list of 23 major challenges to
mathematicians for the future.
Some of these problems remain unsolved, others
have led to new mathematical theories and one
(problem number 10) is at the core of theoretical
computer science.
Hilbert's 10th problem is stated as:
Is there a computer algorithm by which Diophantine
equations can be tested to see if any solutions exist?
17th Century Printing of Diophantus'
Arithmetica translated into Latin
Does an Algorithm Exist for Hilbert's 10 Problem?
Of course, at a meeting of mathematicians in 1900, the problem did not directly refer to computers
or algorithms. In Hilbert's words the 10th problem is stated:
Given a Diophantine equation with any number of unknown quantities and with
rational integral numerical coefficients: to devise a process according to which it
can be determined by a finite number of operations whether the equation is
solvable in rational integers.
It's important to note that Hilbert is not asking for a method to find the solutions but rather for a
procedure to determine if the equation has integer solutions.
This process should be a well-defined formal procedure that could be implemented as a program
for a computer and that would be guaranteed to work for every Diophantine expression.
We refer to such processes as algorithms.
Ref: Martin Davis & Reuben Hersh, "Hilbert's 10th Problem." Scientific American Inc.1973.
There is nothing special about a Diophantine equation. Actually, the term refers to the
admissible solutions. For example, the equation x2 + y2 - 2 = 0 has infinitely many solutions
is we do not restrict valid solutions to integers.
-3
-2
3
3
2
2
1
1
-1
0
1
2
3
-3
-2
-1
0
-1
-1
-2
-2
-3
-3
x2  y 2  2  0
1
2
3
x2  y 2  3  0
Ref: Martin Davis & Reuben Hersh, "Hilbert's 10th Problem." Scientific American Inc.1973.
Fermat's Last Theorem
A famous family of Diophantine equations has the form xn + yn = zn where n may equal 2, 3, 4
or any larger integer. If n is equal to 2, the equation is satisfied by the lengths of the sides of
any right triangle and is called the Pythagorean theorem. One solution is the set of numbers
x=3, y=4, z=5.
If n > 2 this equation is known a Fermat's equation. Pierre de Fermat thought he had proved
that these equations have no positive whole-number solutions. In the margin of his copy of
Diophantus' Arithmetica he wrote that he had found a "marvelous proof" that was unfortunately
too long to be written down in that space. He never made the proof public if he actually had
one.
No correct proof was found for 357 years, until one was finally published by Andrew Wiles in
1995. The term "Last Theorem" resulted because all the other theorems and results proposed by
Fermat were eventually proved or disproved, either by his own proofs or by those of other
mathematicians, in the two centuries following their proposition. Although it is a theorem now
that it has been proved, the status of Fermat's Last Theorem before then, in spite of the name,
was that of a conjecture, a mathematical statement whose status (true or false) had not been
conclusively settled.
http://en.wikipedia.org/wiki/Fermat's_Last_Theorem
Wiles and Taylor Prove Fermat's Last Theorem
Although Wiles had reviewed his argument beforehand with a Princeton colleague, Nick Katz,
he soon discovered that the proof contained a gap. There was an error in a critical portion of
the proof which gave a bound for the order of a particular group.
Wiles and his former student Richard Taylor spent almost a year trying to repair the proof,
under the close scrutiny of the media and the mathematical community. In September 1994,
they were able to complete the proof by using a very novel approach in the troublesome part
of the argument.
Taylor and Wiles's proof is extremely technical in that it relies on the mathematical techniques
developed in the twentieth century, most of which would be totally alien to mathematicians
who had worked on Fermat's Last Theorem only a century earlier.
Fermat's alleged "marvelous proof", on the other hand, would have had to be fairly
elementary, given the state of the mathematical knowledge at the time, and so could not have
been the same as Wiles's.
Most mathematicians and science historians doubt that Fermat had a valid proof of his
theorem for all exponents n, as it seems unlikely there is an elementary proof.
http://en.wikipedia.org/wiki/Fermat's_Last_Theorem
The Green Light Machine
x
0
0
1
0
-1
1
-1
1
-1
y
0
1
0
-1
0
1
1
-1
-1
xx22++yy22--22==00
x2  y 2  2  0
x
0
0
1
0
-1
1
-1
1
-1
:
:
y
0
1
0
-1
0
1
1
-1
-1
:
:
x2 + y2 - 3 = 0
x2  y 2  3  0
Ref: Martin Davis & Reuben Hersh, "Hilbert's 10th Problem." Scientific American Inc.1973.
The Green Light Red Light Machine
AKA - The Hilbert Machine
Suppose S stands for a set of integers. S is listable if a green-light machine can be built that
will do the following job: accept any integer as an input, and as an output turn on a green light
after a finite number of steps if and only if the input (the integer) belongs to S.
For example, the set of even numbers is listable. In this case the machine would divide the
input by 2 and turn on a green light if the remainder is 0.
The set S is computable if a green light red light machine can be built to do a more difficult
job: accept any integer as input and, after a finite number of steps, turn a green light if the
integer is in S and a red light if the integer is not in S.
By this definition, the set of even numbers is computable.
2 -S2 = 0
x2 +n y->
n = 23
22
Ref: Martin Davis & Reuben Hersh, "Hilbert's 10th Problem." Scientific American Inc.1973.
The Computable and the Listable
Let S denote the complement of S, that is, the set of all integers that do not belong to S. If in
the two examples S is the set of even integers, then S is the set of odd integers. We can prove
that if S is computable, S and S are both listable.
In other words, If a green light red light machine exists for S, then there exists a green-light
machine for S and a green-light machine for S.
To build a green-light machine for S, just unscrew the red build of the green light red light
machine.
To build a green-light machine for S, move the green bulb to the socket for the red light.
The converse is also true: If S and S are listable then S is computable. That is, if a green-light
machine exists for S and S, then a green light red light machine can be built for S. Just take the
green light machine for S and replace the green bulb with a red one and then hook the two
machine in parallel.
2 -S2 = 0
x2 +n y->
Ref: Martin Davis & Reuben Hersh, "Hilbert's 10th Problem." Scientific American Inc.1973.
Some Sets are Listable but not Computable
There is a set K for which there exists a green light machine but for which there is no green light
machine for K, the complement of K.
Let each green light machine be described in detail in an owner's manual. This manual explains
exactly how the machine is constructed and how it operates. Every green light machine has a
manual and every manual is numbered M1, M2, . . . and so on.
Now we define the set K as the set of numbers n such that the nth machine lights up if it receives
n as an input. That is, the number 1 belongs to K iff M1 lights up when 1 is enters into its input,
the number 2 belongs to K iff M2 lights up when 2 is entered, and so on...
...
n->S1
n->S2
list of all green machines
Ref: Martin Davis & Reuben Hersh, "Hilbert's 10th Problem." Scientific American Inc.1973.
Building a Green Light Machine for K
Our green light machine for K is built in the following manner: We build a new green light
machine modified to accept both the value n and the manual for the nth green light machine. This
new green light machine reads the manual and reconfigures itself to produce the same result as the
nth green light machine.
...
...
list of all green machine manuals
For example we check to see if 3,781 is a member of
the set K. We load the manual M3,781, the value 3,781
and turn the crank. If green light machine 3,781
accepts 3,781 the 3,781 is a member of the set K.
n->SM
3,781
Ref: Martin Davis & Reuben Hersh, "Hilbert's 10th Problem." Scientific American Inc.1973.
There is no Green Light Machine for K
Imagine that such a machine existed. Since K is the complement of K, this machine would
light up for an input say 297, iff M297 does not light up for 297. Therefore the machine for K
is not M297. Of course this argument holds for any n. So the green machine for K cannot be a
member of the list of all green machines, therefore...
K is not listable
And so, there is no green light red light machine for K.
There can never be a formal procedure (an algorithm or a machine program) for sorting K
from K. Therefore there is an example of a problem that can never be solved by
mechanical means.
This is a major theorem of recursive function theory, established in the 1930's
..
by Godel, Church, Post, Kleene and Turing.
Ref: Martin Davis & Reuben Hersh, "Hilbert's 10th Problem." Scientific American Inc.1973.
There is no Green Light Machine for K: Redux
For those who are still feeling uncertain about this proof idea. Consider the alternatives. If the
green light machine for K exists, then it must be in the list of green light machines somewhere,
say position p. If it is a green light machine, its manual will then be Mp.
When we input to this green light machine its own manual Mp and the number p, does it light
up? By the definition of K, this machine must light up iff p is not a member of the set of
numbers that cause the pth green light machine to light...But since this is the pth green light
machine and the machine lights, then p is a member of the set defining K and therefore p cannot
be a member of K...an apparent contradiction.
The Alternatives
1. There is no green light machine for K. - Traditional interpretation of this result.
2. There is no way to write Mp. - Essentially the same argument as 1.
3. We cannot build the modified green light machine for K. - That is, the set S is not well defined.
4. The green light machines are not listable. - There is no one-to-one correspondence between the
natural numbers and the set of green light machines
Decidability
We have learned about a model of computation called a Turing Machine (TM) that can be used to
recognize and/or generate wide variety of languages. We also learned that the part of the TM that
determines the particular language being recognized can be expressed as an algorithm.
Now we will investigate the power of algorithms and their limitations. We will discover that
some problems can be well defined but not solved using algorithmic methods or TM's.
Decidable Languages and Decidable Problems
Just as we learned that there are some languages that are not decidable we will see that there are
undecidable problems as well. The relationship between decidable problems and decidable
languages is a subtle one.
The acceptance problem for DFAs of testing whether a particular finite automaton accepts a
given string can be expressed as a language, ADFA. This language contains the encoding of all
DFAs together with strings that the DFAs accept.
In general, let
ADFA = { <B,w> | B is a DFA that accepts input string w}
The problem of telling whether a DFA B accepts an input w is the same as the problem of testing
whether <B,w> is a member of the language ADFA.
Theorems for Decidable Languages
Theorem: ADFA is a decidable language. The proof of this theorem is simple.
We only need to describe a TM that decides ADFA. Let M be a TM M. The input to M is <B,w>
which includes a description of B followed by the string w. We Simulate B on input w and, if the
simulation ends in an accept state we accept <B,w> otherwise we reject <B,w>.
We can prove a similar theorems for non deterministic finite automata and regular expressions.
This is not surprising since NFA's and regular expressions have been shown to be equivalent to
DFA's in terms of their computational power.
Theorem: ANFA is a decidable language.
Theorem: AREX is a decidable language.
Emptiness Testing
We want to show that the problem of determining if two DFA's recognize the same language is
decidable. In order to do this we first need to establish a rather odd theorem called emptiness
testing. We need to show that a TM can test to see if a DFA accepts any strings at all. That is
we want to prove that
EDFA = {<A> | A is a DFA and L(A)=f}
To accomplish this we define a TM T that reads <A> as its input, marks the start state of A and
then continues to mark every state that can be reached from a marked state along a valid
transition. Once all reachable states are marked, T accepts if an accept state of the DFA is
marked, otherwise T rejects.
Theorem: EDFA is decidable.
Deciding if Two Turing Machines Accept the Same Language
Now that we have proved we can decide if a DFA accepts no string, we can show that we can
decide if two DFA's accept the same language.
We wish to prove that
EQDFA = {<A,B> | A and B are DFA's and L(A) = L(B)}
Construct a new DFA C from A and B such that C accepts only those strings accepted by A or
B but not both. That is,
L(C) = (L(A) and ~L(B)) or (~L(A) and L(B))
We now define a TM F that takes <A,B> as its input and builds the DFA C. We now apply the
TM for emptiness testing to <C>. If this TM accepts <C> we accept (i.e. conclude that the
symmetric difference of L(A) and L(B) is empty), otherwise we reject.
Theorem: EQDFA is a decidable language.
Are Context-Free Languages Decidable?
Next we want to see if the class of context-free languages is decidable.
ACFG = {<G,w> | G is a CFG that generates string w}.
For CFG G and string w we must test to see if G can generate w. We have to be careful here
since we must use a method of derivation that does not result in an infinite number of steps. That
is, if w can be generated by G it must be accomplished in a finite number of steps and if w cannot
be generated by G we must determine this fact in a finite number of steps.
1. Convert G into an equivalent CNF grammar.
2. List all derivations possible for the number of steps required to derive w.*
3. If any of the derivation generate w, accept, otherwise reject.
* How to we determine the number of steps in 2 above and how do we know that this number is finite?
The Number of Derivations is Finite
Recall that all rules in CNF grammars are of the form
A->BC
A->a
so we can determine the exact number of steps (application of rules) needed to generate any string
w where w is made up of n terminal symbols. Starting with A we need to apply n-1 rules of the
form A -> BC to obtain n non-terminals, then we apply n rules of the form A -> a to replace the n
non terminals with terminals. Therefore the operation in 2 requires us to list all derivations
reachable from exactly 2n-1 steps, which completes the proof.
Theorem: ACFG is a decidable language.
Emptiness Test for CFG's
Just as with the regular languages we want to show that we can decide if a particular CFG can
generate any strings at all.
ECFG = {<G> | G is a CFG and L(G)=f}
Since we don't know the length of the strings we would generate, we cannot simply attempt to
generate all strings to show that G can generate no strings. We need a way to analyze the
grammar directly to determine if any strings can be generated. This is OK since the grammar
itself must be finite.
Let R be a TM reading <G> as its input where G is a CFG in Chomsky Normal Form (CNF).
1. Mark all terminals symbols in the grammar G.
2. Repeatedly mark every occurrence of any variables that appear on the LHS of any rule in
which all symbols (terminals or variables) on the RHS are marked.
3. When no more variables are marked in 2, check to see if the start symbol has been marked. If
not accept, otherwise reject.
Showing that Two CFG's Generate the Same Language
Giddy with our successes, we might attempt to show that,
EQCFG = {<G,H> | G and H are CFLs and L(G) = L(H)}.
But we would discover that this is not possible. It is rather involved to show a formal proof
for this fact but it is related to the fact that context-free grammars are not closed under
complementation.
That is, if L(G) is a context-free language it is not necessarily true that ~L(G) is also context
free. In general,
Theorem: EQCFG is not decidable.
Every CFL is Decidable
It is straightforward to prove that,
Theorem: Every CFL is decidable.
Given a CFG G that generates A we need to design a TM MG that decides A. Since we already
have a theorem stating that ACFG is a decidable language we can construct a TM that contains
a copy of G.
This TM reads a string w and performs the following operations, 1. Run the TM that decides
ACFG = {<G,w> | G is a CFG that generate string w}. 2. If this TM accepts then accept
otherwise reject.
A Complete (?) Description of Levels of Computation
Now we can fill in our computational power diagram for the relationship between the classes
of languages.
In summary, we have models of computation for regular languages, context-free languages, and a
class of context-sensitive languages called decidable. We have found that there are types of
languages whose members can be recognized by TM's but some non-members cannot be proved
non-members.
Halting Problem
We are now ready to look at one of the most philosophically important theorems of the theory of
computation and formal axiomatic systems in general. This theorem is called the halting
problem.
Given an computer program and a precise description of what that program is supposed to do,
there may not be an automated technique for demonstrating the correctness of the program. We
will need to review a number of theorems in preparation for our study of the halting problem.
First we will look at ATM, which is the problem of testing whether a TM accepts a given input
string.
ATM = {<M,w> | M is a TM and M accepts w}
We note that ATM is Turing-recognizable since we can always devise a TM U that loops on
<M,w> whenever M is a TM that loops on w.
The Universal Turing Machine
We call U the Universal Turing Machine. This machine was first proposed by Alan Turing as
a machine that could simulate any other TM using the description of the machine. In fact,
Turing's idea of a universal TM was influential in the development of the first stored-program
computers.
Correspondences & Countable Sets
Given two sets A and B and a function f from A to B, we say that f is one-to-one if it never maps
different elements in a and b to the same place. That is f(a) /= f(b) whenever a /= b.
We say that f is onto if it reaches every element in B from an element in A. That is for every
element b in B there is an element a in A such that f(a) = b.
We say that A and B are the same size if there is a one-to-one and onto function f : A -> B.
A function that is one-to-one and onto is called a correspondence. In a correspondence, every
element of A maps to a unique element of B and each element of B has a unique element of A
mapping to it. A correspondence gives us a way to pair the element of A with the elements of B.
We can devise explicit correspondences between finite sets, but when the sets are infinite we
demonstrate a correspondence by establishing an ordering between the two sets that ensures that
every member of one set corresponds to a member of the other.
We can show a correspondence between the natural numbers N (1,2,3,. . .) with the positive
integers divisible by 7 (7, 14, 21, . . .) In general we can develop a correspondence between N
and any subset of N. Weird...
Definition: A set A is said to be countable if either it is finite or it has the same size as N.
Levels of Infinity
We can find a correspondence between N and the rational numbers Q.
As we have seen there are infinite sets
that do not correspond to N. These sets
are said to be uncountable.
Real
numbers are an example of an
uncountable set.
Cantor's Diagonalization Method is used
to prove that the real numbers in the
range (0.0,1.0) are uncountable.
Remember our problem of 0.4999999...
= 0.500000...? A method has been
suggested to avoid this problem.
When choosing the number that is not in the correspondence list, just never select the digits 0 or 9
when constructing the value x that is not in the list.
A Case Against the Axiom of Choice
Recall that our concern with diagonalization was in the construction of the correspondence in
the first place. We needed to create the list and then use it to construct a value x to prove that
the list could not be created.
The problem is not whether the value x is in the list but whether proof by contradiction can be
used at all in this case. In a proof by contradiction we show that the direct result of accepting
the contradictory statement is an impossible result such as A=~A or 0=1, etc.
In this case the contradiction is demonstrated by using the result (i.e. the correspondence
between N and (0.0..1.0) (which we believe to be impossible) to infer the existence of another
value x which can only exist if it can be constructed from a correspondence that cannot exist.
Specifically we may argue that the proof of contradiction may not rely on the construction of
an object or the derivation an entity from some other non-existent object or entity. Analogous
to the case of 0/0, we can ascribe to the non-existent entity any properties we wish.
Given the statement,
If pigs have wings, then I have a zillion dollars.
It is valid to prove that pigs do not have wings by showing that I do not have a zillion dollars.
However it is not valid to argue that, with a zillion dollars I could construct a flying swine, and
since I have not performed such a construction pigs cannot have not have wings...
ATM is Undecidable
To prove that ATM is undecidable we first assume that it is decidable (i.e. U exists and decides
ATM) and then obtain a contradiction. Suppose that U is a decider for ATM. This means that U
halts and accepts when M accepts w and U halts and reject when M fails to accept w.
We construct another TM D that calls U as a subroutine and does the opposite of U. That is, it
accepts when U rejects and rejects when U accepts.
D(<M>) accepts if M does not accept <M>
D(<M>) rejects is M accepts <M>
Now we run D on its own description
D(<D>) accepts if D does not accept <D> and
D(<D>) rejects is D accepts <D>
No matter what D does, it is forced to do the opposite, which is a contradiction. Thus neither TM
D or TM U can exist.
co-Turing-Recognizable
If a language is Turing-recognizable and its complement is Turing-recognizable then it is said to be
co-Turing-recognizable.
Theorem: A language is decidable if and only if it is both Turing-recognizable and
co-Turing-recognizable.
We have shown (sort of) that ATM is Turing-recognizable but not decidable. From these two results
we must conclude that ~ATM is not Turing-recognizable. Otherwise would be decidable.
Theorem: ~ATM is not Turing-recognizable
• Therefore (if you believe the diagonalization proof) there is something else, a class of languages
that cannot be recognized at all.
• There is no universal Turing machine.
• There are true statements that cannot be proved true.
• There are statements that can neither be proved true or false. There are correct programs that
cannot be proved correct.
• And finally, there are well-defined problems for which no program can be written to solve them.
THE
END
??