CIT 596 – Theory of Computation
1
Theory of Computation: An Introduction
Traditionally, the study of theory of computation comprises three central areas:
automata, computability, and complexity.
These areas are linked by the question:
What are the fundamental capabilities and limitations of computers?
In this course, we will study the fundamentals of these three central areas of theory of computation.
c Marcelo Siqueira — Spring 2005
CIT 596 – Theory of Computation
2
Theory of Computation: An Introduction
The central question of complexity theory is
What makes some problems computationally hard and others easy?.
Remarkably, we still have not been able to answer this question!
An important research achievement, though, was the definition of a
scheme to classify problems according to their “level of difficulty” (difficulty is precisely expressed in this scheme).
What does a computer scientist do when faced with a “hard” computational problem?
Cryptography: sometimes, hardness is very desirable!
c Marcelo Siqueira — Spring 2005
CIT 596 – Theory of Computation
3
Theory of Computation: An Introduction
The central question of computability theory is
Which problems can be solved by computers and which ones cannot?
Even before the invention of “real computers”, Kurt Gödel, Alan Turing,
and Alonzo Church discovered that certain basic problems cannot be
solved by computers!
For instance, computers cannot solve the problem of determining whether
a mathematical statement is true or false.
The fact that this discovery is older than the birth of real computers suggests that we do not have to study semicondutors or integrated circuits
in order to understand “computation”.
Actually, we can do that by using “abstract models” of computers.
c Marcelo Siqueira — Spring 2005
CIT 596 – Theory of Computation
4
Theory of Computation: An Introduction
Automata theory deals with the definitions and properties of “abstract
models” of computers.
These abstract models are models of computation that can be defined
mathematically.
Some of these models are (believed to be) as powerful as real computers, whereas others can be defined more simply and are less powerful.
The simplest model of computation is the finite automaton or finitestate machine.
This simple model has several applications in computer science, including text processing, compilers, and interface design.
c Marcelo Siqueira — Spring 2005
CIT 596 – Theory of Computation
5
Theory of Computation: An Introduction
Another important model is the pushdown automaton, which is more
powerful than the finite automaton.
This model provides a natural way of determining whether the sintax
of a statement in a high-level programming language is correct, and
therefore it is heavily used in compiler construction and computational
linguistics.
Finally, there is the Turing machine, named after Alan Turing, the English mathematician who invented it.
There is general agreement that Turing machines are able to carry out
any algorithm whatsoever.
c Marcelo Siqueira — Spring 2005
CIT 596 – Theory of Computation
6
Theory of Computation: An Introduction
How do these models of computation work?
These models actually solve decision problems, i.e., computational
problems for which every specific instance can be answered “yes” or
“no”.
For instance, “Given a positive integer n, is it a prime?”
The input value n is encoded as a string of digits, and a computation
that solves the problem starts with this string and ends with a “yes” or
“no”.
This encoding process is no different from what actually happens with
the input data in real computers. Everything is converted into strings of
0’s and 1’s, isn’t it?
c Marcelo Siqueira — Spring 2005
CIT 596 – Theory of Computation
7
Theory of Computation: An Introduction
Not all computational problems are decision problems. However, in the
case of a problem requiring an answer more complicated than “yes” or
“no”, there is often a decision problem whose solution is comparable to
that of the original.
For instance, if f is a function, being able to answer the question “Given
x and y, is y = f (x)?” is tantamount to being able to compute f (x) for
an arbitrary x.
Furthermore, the more powerful computational models are able to deal
with more general kinds of problems.
An extremely interesting point is the relationship between models of
computation and languages.
c Marcelo Siqueira — Spring 2005
CIT 596 – Theory of Computation
8
Theory of Computation: An Introduction
We can think of a decision problem as a language recognition problem.
For instance, for the primality decison problem (“Given a positive integer
n, is n a prime?”), we can view ALL strings for which our model answers
“yes” as a “language”.
So, our decision problem can be viewed as the one of determining
whether an arbitrary string is in the language recognized by the model
of computation.
In our example, the recognition problem is to determine whether a given
string (enconding a number) belongs to the language of ALL strings representing prime numbers.
c Marcelo Siqueira — Spring 2005
CIT 596 – Theory of Computation
9
Theory of Computation: An Introduction
Distinct models of computation recognize (resp. solves) languages (resp.
decision problems) of different complexity.
Finite automata recognize regular languages.
Pushdown automata recognize context-free languages.
Turing machines recognize recursively enumerable languages.
Regular languages are simpler than context-free languages, which in
turn are simpler than recursively enumerable languages. This seems
very natural! Why?
c Marcelo Siqueira — Spring 2005
CIT 596 – Theory of Computation
10
Theory of Computation: An Introduction
We can also study “languages” from a generation point of view, rather
than recognition.
From the generation point of view, we use a formalism known as a grammar.
A grammar contains a set of rules that can be applied to generate all
strings in a language.
Grammars are extremely useful for specifying and representing programming languages.
In order to build a compiler for a programming language, we must specify the language in some way. Grammars are pretty handy and often
used for that.
c Marcelo Siqueira — Spring 2005
CIT 596 – Theory of Computation
11
Theory of Computation: An Introduction
As you may expect, we also classify grammars according the complexity
of the languages they can generate.
Regular languages are generated by regular grammars.
Context-free languages are generated by context-free grammars.
Recursively enumerable languages are generated by unrestricted grammars.
This hierarchy of grammars and languages was created by Noam Chomsky, and it is known as Chomsky hierarchy.
Yeah, this is the same Noam Chomsky you may have heard about in
the past four years within a completely different context... He is a former
PhD student of UPenn!
c Marcelo Siqueira — Spring 2005