CoSc3132 – Complexity Theory ------------------------------------------------------------------------------------------------------------------------- Overview This is the first “pure” course in theoretical computer science. We will be concerned with the following basic questions in computer science: What is a computation? Is there a universal model of computation? Can everything be comput ed? Is there a relation between computations and mathematics? Can we identify problems that are not computable? What resources are needed to perform a certain computation? Can we identify computationally hard problems? More than physical devices we are interested in the history of concepts in computing. We are interested in very basic questions. Let us ask the following more important questions. When was the first algorithm devised? This is a very difficult question. Since, algorithms were not studied the way we do them today. Perhaps, the first algorithm dev ised was unary addition thousands of years ago in some cave. The limbobo bone type devices were used to execute that algorithm. The concept of an algorithm is very well developed in Euclid’s school. Two striking examples are: 1. The Euclidean Algorithm to compute the GCD. It is a non -trivial algorithm, in the sense that it is not clear at all why it computes the GCD. 2. The constructions by ruler can compass are examples of precise algorithms. The elementary steps are well defined and the tasks are well defin ed. The most important question in theory of computation is: When was this question asked? In 1900 David Hilbert and eminent mathematician was invited to give a lecture at the international congress of mathematicians. In his talk he posed 21 questions as a challenge to 20th century mathematicians. His problems spanned several areas from mathematical analysis, logic, foundation of set ----------------------------------------------------------------------------------------------------------------------------- ---- CoSc3132 – Complexity Theory ------------------------------------------------------------------------------------------------------------------------theory and geometry. However, one of question the 10th problem was v ery interesting. Turing Machines and Algorithms Luckily the mathematical notion of what an algorithm and computation is was already developed for Matiyasevich. Let us ask the most fundamental question in theory of computation. Algorithms and Computation The seeds of answer to this question were also sown by Hilbert. In 1 928, Hilbert who wanted to systematize all of mathematics had posed The Entscheidung’s problem He wanted an algorithm that would take a mathematical assertion and would decide if the assertion was true or not!!!! That would be an amazing algorithm (if it e xisted). In 1936, Alan Turing, Alonzo Church showed that the Entscheidung’s problem was unsolvable. The two most famous papers in this regard are: Alonzo Church, "An unsolvable problem of elementary number theory", American Journal of Mathematics, 58 (1936), pp 345 - 363 Alan Turing, "On computable numbers, with an application to the Entscheidung’s problem", Proceedings of the London Mathematical Society, Series 2, 42 (1936), pp 230 - 265. We will work with Turing’s model. Turing came up with the idea o f computing machines that we now call Turing machines. He proposed that every computation that we can perform mechanically could be performed by a Turing machine. The good thing about Turing machines was: “Turing machines are not an intuitive notion like that of an algorithm. But they are mathematically defined objects. Thus one can prove things about them. One can prove their existence and non-existence also. One can study them with the tools of mathematics and make precise claims about them.” Computability Theory Computability Theory will deal with the following questions: What is a computation? Can we co me up with a precise definition of an algorithm or a computation? (The answer is yes). Can everything be computed? (The answer is no.) Can we characterize problems that cannot be computed? ( The answer is it is hard to do that but in certain cases we can identify them and prove that they are not computable). We will study computable and incomputable problems in various different areas. We will study computability and logic. We will prove Godel’s incompleteness theorem. Godel’s incompleteness theorem ----------------------------------------------------------------------------------------------------------------------------- ---- CoSc3132 – Complexity Theory ------------------------------------------------------------------------------------------------------------------------Godel’s theorem is a corner stone in mathematical logic and foundations of mathematics. I t roughly states that: “In any (axiomatic) system that includes numbers. There are always statements that are true but not provable.” This has had profound impact of mathematics. We will prove this theorem in our course using tools from computability th eory. We will discuss this in detail when the time comes. Complexity Theory Complexity theory is much more practical than computability theory. The basic idea is now we not only want an algorithm but an efficient algorithm to solve problems. Complexity Theory will deal with the following questions: 1. Given a problem can we devise an efficient algorithm to solve that problem? 2. Can we characterize problems that have efficient algorithms? 3. Can we characterize problems that are on the boundary of being efficiently solvable and intractable? 4. Can we prove that problems though computable are intractable? 5. Can we prove that certain problems which are practically significant are intractable? 6. Is there a unified theory of problems that are hard to solve? One of the most important concepts in Complexity Theory is that of NP -completeness. In 1971 Cook and Levin proved a very important theorem in the following papers: Stephen Cook (1971). "The Complexity of Theorem Proving Procedures", Proceedings of the third annual ACM symposium on Theory of computing, 151 –158. Leonid Levin (1973). "Universal’nye perebornye zadachi". Problemy Peredachi Informatsii 9 (3): 265 English translation, "Universal Search Problems", in B. A. Trakhtenbrot (1984). "A Survey of Russian Approaches to Perebor (Brute -Force Searches) Algorithms". Annals of the History of Computing 6 (4): 384400. This theorem showed that if one given problem (called Satisfiability) could be solved efficiently then many others could also be solved efficiently. In fact, a whole infinite class of problems can be solved efficiently. This is very strong evidence that Satisfiability is a hard to solve problem. We are still looking for a proof!!!!!!! In complexity theory we will talk about the celebrated P vs NP question. Which has a $1000,000.00 award on it. We will see why the amount stated here is not enough and why this question has far reaching consequences in computer science. ----------------------------------------------------------------------------------------------------------------------------- ----