CoSc3132 – Complexity Theory
-------------------------------------------------------------------------------------------------------------------------
Overview
This is the first “pure” course in theoretical computer science. We will be concerned with the
following basic questions in computer science:
 What is a computation?
 Is there a universal model of computation?
 Can everything be comput ed?
 Is there a relation between computations and mathematics?
 Can we identify problems that are not computable?
 What resources are needed to perform a certain computation?
 Can we identify computationally hard problems?
More than physical devices we are interested in the history of concepts in computing. We
are interested in very basic questions. Let us ask the following more important questions.
When was the first algorithm devised?
This is a very difficult question. Since, algorithms were not studied the way we do them today.
Perhaps, the first algorithm dev ised was unary addition thousands of years ago in some cave. The
limbobo bone type devices were used to execute that algorithm.
The concept of an algorithm is very well developed in Euclid’s school.
Two striking examples are:
1. The Euclidean Algorithm to compute the GCD. It is a non -trivial algorithm, in the sense
that it is not clear at all why it computes the GCD.
2. The constructions by ruler can compass are examples of precise algorithms. The
elementary steps are well defined and the tasks are well defin ed.
The most important question in theory of computation is:
When was this question asked?
In 1900 David Hilbert and eminent mathematician was invited to give a
lecture at the international congress of mathematicians. In his talk he posed
21 questions as a challenge to 20th century mathematicians. His problems
spanned several areas from mathematical analysis, logic, foundation of set
----------------------------------------------------------------------------------------------------------------------------- ----
CoSc3132 – Complexity Theory
------------------------------------------------------------------------------------------------------------------------theory and geometry. However, one of question the 10th problem was v ery
interesting.
Turing Machines and Algorithms
Luckily the mathematical notion of what an algorithm and computation is was already developed
for Matiyasevich. Let us ask the most fundamental question in theory of computation.
Algorithms and Computation
The seeds of answer to this question were also sown by Hilbert. In 1 928, Hilbert who wanted to
systematize all of mathematics had posed The Entscheidung’s problem He wanted an algorithm
that would take a mathematical assertion and would decide if the assertion was true or not!!!!
That would be an amazing algorithm (if it e xisted).
In 1936, Alan Turing, Alonzo Church showed that the Entscheidung’s problem was unsolvable. The
two most famous papers in this regard are:
Alonzo Church, "An unsolvable problem of elementary number theory", American Journal of
Mathematics, 58 (1936), pp 345 - 363
Alan Turing, "On computable numbers, with an application to the Entscheidung’s problem",
Proceedings of the London Mathematical Society, Series 2, 42 (1936), pp 230 - 265.
We will work with Turing’s model. Turing came up with the idea o f computing machines that we
now call Turing machines. He proposed that every computation that we can perform
mechanically could be performed by a Turing machine. The good thing about Turing machines
was:
“Turing machines are not an intuitive notion like that of an algorithm. But they are
mathematically defined objects. Thus one can prove things about them. One can prove their
existence and non-existence also. One can study them with the tools of mathematics and make
precise claims about them.”
Computability Theory
Computability Theory will deal with the following questions:





What is a computation? Can we co me up with a precise definition of an algorithm or a
computation? (The answer is yes).
Can everything be computed? (The answer is no.)
Can we characterize problems that cannot be computed? ( The answer is it is hard to do
that but in certain cases we can identify them and prove that they are not computable).
We will study computable and incomputable problems in various different areas.
We will study computability and logic. We will prove Godel’s incompleteness theorem.
Godel’s incompleteness theorem
----------------------------------------------------------------------------------------------------------------------------- ----
CoSc3132 – Complexity Theory
------------------------------------------------------------------------------------------------------------------------Godel’s theorem is a corner stone in mathematical logic and foundations of mathematics. I t
roughly states that:
“In any (axiomatic) system that includes numbers. There are always statements that are true but
not provable.”
This has had profound impact of mathematics. We will prove this theorem in our course using
tools from computability th eory. We will discuss this in detail when the time comes.
Complexity Theory
Complexity theory is much more practical than computability theory. The basic idea is now we
not only want an algorithm but an efficient algorithm to solve problems.
Complexity Theory will deal with the following questions:
1. Given a problem can we devise an efficient algorithm to solve that problem?
2. Can we characterize problems that have efficient algorithms?
3. Can we characterize problems that are on the boundary of being efficiently solvable and
intractable?
4. Can we prove that problems though computable are intractable?
5. Can we prove that certain problems which are practically significant are intractable?
6. Is there a unified theory of problems that are hard to solve?
One of the most important concepts in Complexity Theory is that of NP -completeness. In 1971
Cook and Levin proved a very important theorem in the following papers:
Stephen Cook (1971). "The Complexity of Theorem Proving Procedures", Proceedings of the third
annual ACM symposium on Theory of computing, 151 –158.
Leonid Levin (1973). "Universal’nye perebornye zadachi". Problemy Peredachi Informatsii 9 (3):
265 English translation, "Universal Search Problems", in B. A. Trakhtenbrot (1984).
"A Survey of Russian Approaches to Perebor (Brute -Force Searches) Algorithms". Annals of the
History of Computing 6 (4): 384400.
This theorem showed that if one given problem (called Satisfiability) could be solved efficiently
then many others could also be solved efficiently. In fact, a whole infinite class of problems can
be solved efficiently. This is very strong evidence that Satisfiability is a hard to solve problem. We
are still looking for a proof!!!!!!!
In complexity theory we will talk about the celebrated
P vs NP question.
Which has a $1000,000.00 award on it. We will see why the amount stated here is not enough
and why this question has far reaching consequences in computer science.
----------------------------------------------------------------------------------------------------------------------------- ----