TM, DTM, NP, NPC, BB(n)
C(L, t), reducibility
Advanced Algorithms
By Me Dr. Mustafa Sakalli
March, 19, 2012.
• Halting Problem
• Superpolynomial time algorithms
• Easy and hard problems
• Some interesting problems
Shortest vs. longest simple paths
Euler tour vs. hamiltonian cycle
2-CNF satisfiability vs. 3-CNF satisfiability
• The classes P and NP and NP-completeness
Class P, Polynomial-time algorithms: problems that are solvable
in polynomial time. On inputs of size n, the worst-case running
time is O(nk) for some constant k.
Class NP: those problems that are "verifiable" in polynomial
time.
Class NPC: those problems that are in NP and are as "hard"
as any problem in NP.
Introduction
• Computational problems for which
– fast algorithms have never been found,
– but neither have such algorithms been proved to be unattainable.
• Examples:
– The primality-testing problem, for which the best-known algorithm on the brink
of polynomial time, and
– The integer-factoring problem, for which the known algorithms in a more
primitive condition.
• Many such problems, not just a list of seemingly difficult computational
problems, in fact bound together by strong structural ties. Called the NPcomplete problems:
• The traveling salesman problem (‘TSP’): Given n points in the plane
(‘cities’), and a distance D. Queston: Is there a tour that visits all n of the
cities, returns to its starting point, and has total length D?
• Graph coloring: Given a graph G and an integer K. Can the vertices be
properly colored in  K of colors? Kn such possibilities, n the number of the
vertices.
• Independent set: Given a graph G and an integer K. Is there a V(G)
containing an independent set of K vertices?
Introduction
• Bin packing: Given a finite set S of positive integers, and an
integer N (the number of bins). Does there
• exist a partition of S into N or fewer subsets such that the
sum of the integers in each subset is K? In
• other words, ‘pack’ the integers of S into at most N bins with
the capacity K?
• These are hard problems, but there can be easy instances, ie
if the graph G happens to have no edges at all, or a very few
of them, then it will be very easy to figure out a possible
coloring, or determine the presence of an independent set of
K vertices.
• Suppose a manufacturer gives a warranty of the worst case
performance for independent set, nK …ie 1000n8 …
Adjacency matrix dimension n, the element number would be
n2, hence the input size B=n1/2. B bits. Time T=BK/2 , which is
polynomial if K would have been fixed. But our focus is all the
instances of the independent set problem.
Some properties of the NPcomplete problems
• The problems computationally very difficult, for any of
which no polynomial time algorithms have been found.
Which means If a fast algorithm could be found for one of
which then there would be fast algorithms for all of them.
• But not any proof given yet indicating that polynomial
time algorithms for these problems do not exist. Similarly,
if it could be proved that no fast algorithm exists for one
of the NP-complete problems, then there could not be a
fast algorithm for any other of those problems.
• "The question of the existence or nonexistence of
polynomial-time algorithms for the NP-complete problems
probably rates as the principal unsolved problem that
faces theoretical computer science today.“ Page: 105.
• Our next task will be to develop the formal
machinery that will permit us to give
precise definitions of all of the concepts
that are needed. In the remainder of this
section we will discuss the additional ideas
informally, and then in section 5.2 we’ll
state them quite precisely.
Decision problems vs. optimization problems
Optimization problems, find the feasible solution with the best value.
Decision problems, verifying the membership with the answer of "yes" or "no"
•
Casting an optimization problem into a related decision problem
– by imposing a bound on the value of it, to be optimized.
– complexity of optimization problem is equal to complexity of related
decision problem.
– Shortest path: does there exist a path from vertex u to v of length at most
k?
Reductions
A procedure that transforms any instance α of decision problem A into
some instance β of decision problem B with the characteristics:
1 The transformation takes polynomial time.
2 The answers are the same. That is, the answer for α is "yes" if and
only if the answer for β is also "yes."
A string of an Instance: for a particular problem
Algorithm for the reduction, Polynomial-time reduction algorithm:
Idea behind a decision and optimization problems
• Many of the problems can be phrased as decision
problems or as optimization problems.
• A decision problem is one that asks only for a yes-or-no
answer:
– Can this graph be 5-colored?
– Is there a tour of length  15 miles?
– Is there a set of 67 independent vertices?
• An optimization problem is one that asks the value that
provides on of the feasible solution.
– What is the smallest number of colors with which G can be
colored?
– What is the length of the shortest tour of these cities?
– What is the size of the largest independent set of vertices in G?
• Usually if we find a fast algorithm for a decision problem
then with just a little more work we will be able to merge it,
to solve the corresponding optimization problem.
Idea behind a decision and optimization problems
• Example: suppose we have an algorithm that solves the
decision problem for graph coloring, and what we want is
the solution of the optimization problem (the chromatic
number).
• A G of 100 vertices, Q: Can the graph be 50-colored?
• If so, then the 1 cr  50. Split the range by 2 for every
run, and keep on asking the question? Yes for second half,
and no for the first half.
• After O(log n) steps we will have found the cr. Hence if
there is a fast way to do the decision problem then there is
a fast way to do the optimization problem. The converse is
true or obvious. Linear Optimization, Simplex, Ellipsoidal
so on.
• Our Attention is on Decision Problems.
What is a language and the class
P, and class NP?
•
Decision problem asking: Question: If a given word (the input string) does or does not
belong to a certain language. The language is the totality of words for which the
answer is ‘Y.’
•
The graph 3-coloring language. Question: Does the vertex adjacency matrix of a
graph represent, if it is 3-colorable. A 3-colorability computation is therefore nothing
but an attempt to discover whether a given word belongs to the dictionary.
•
•
What is the class P?
We say that a decision problem belongs to the class P if there is an algorithm A and a
number c such that for every instance I of the problem the algorithm A will produce a
solution in time O(Bc),
Briefly, P is the set of easy decision problems. Examples: Determining
•
–
–
–
–
–
–
Relative primality of given two integers.
Divisibility of an integer by another.
K=2 colorablity of a G.
A flow of value greater than K in a network.
K - connectedness of a G.If a G be disconnected by the removal of K or fewer edges.
A matching of K – or more edges in a bipartite G.
What is a language and the class
P, and class NP?
• What is the class P?
• We say that a decision problem belongs to the class P if
there is an algorithm A and a number c such that for
every instance I of the problem the algorithm A will
produce a solution in time O(Bc),
• Briefly, P is the set of easy decision problems. Examples:
Determining
–
–
–
–
–
Relative primality of given two integers.
Divisibility of an integer by another.
K=2 colorablity of a G.
A flow of value greater than K in a network.
K - connectedness of a G.If a G be disconnected by the removal
of K or fewer edges.
– A matching of K – or more edges in a bipartite G.
• NP is a class of Decision Problems for which it is easy to check the
correctness of a claimed answer. Not addressing to find a solution, but
only verifying that an alleged solution is correct.
• A decision problem Q belongs to NP if there is an algorithm A that does
the following:
– Associated with each word of the language Q (i.e.,with each instance I for
which the answer is ‘Yes’) there is a certificate C(I) such that when the pair
(I,C(I)) are input to algorithm A it recognizes that I belongs to the language Q.
– If I is some word that does not belong to the language Q then there is no
choice of certificate C(I) that will cause A to recognize I as a member of Q.
– Algorithm A operates in polynomial time. Verifiable
• An analogy of the distinction between the classes of P and NP.
– Reading through a difficult proof of some mathematical theorem, and
verifying its correctness, which is much easier than going through the
invention of the idea as it happened.
– Some proofs are extremely time consuming even to check (the proof of the 4color theorem!), and some computational problems are not even known to
belong to NP, let alone to P.
• The problems in class P are the problems easy to find a solution, and
those in NP are the problems where it’s easy to check a solution that
may have been very tedious to find.
Consider the graph coloring problem to be the decision problem Q.
Certainly the graph coloring problem is not known to be in P. However, in
NP.
• A hard method of constructing certificates that proves it.
• Suppose G is some graph K-colorable.
• The certificate of G is a list of the colors assigned to each vertex in some
proper K-coloring of the vertices.
• How to produce that list?
• Not easy to construct a certificate. One needs to solve a hard problem for
every entry of G. DP memoization?.
• Then checking the correctness of an alleged answer is
– either show that the color of each vertex is assigned a proper color of Kcoloring,
• or go to hard path to provide a certificate, and only check first that every
vertex has only one color (as listed) and check that no more than K
colors have been used altogether, and finally check that every edge e of
G is terminated by two different colors. All checking takes a polynomial
time, and hence the graph coloring problem belongs to NP.
For the tsp, suppose we produce a certificate that contains a tour, whose
total length is K.
• The checking algorithm A would then verify that the tour really does visit
all of the cities and really does have total length K. without seeking all
possible K solutions through each of the vertices. Polynomial.
• The TSP, therefore, also belongs to NP.
• How could a problem fail to belong to NP?’
• Try this decision problem: an instance I of a G with n cities and a positive
number K.
• The question is ‘Is it true that there is not a tour of all of these cities
whose total length is less than K?’ This is a kind of a negation of the
tsp.
• Does the negated TSP belong to NP? If so, there must be an algorithm A
and a way of making a certificate C(I) for each instance I so that absence
of such a tour can be verified. Any certificate or even any algorithm to
generate C is known yet. Therefore, not known if this negation of the
tsp belongs to NP.
• Problems belong to NP but not immediately obvious? Pratt’s
algorithm is a method of producing a certificate with a quick
check that a given integer is prime.
• The decision problem ‘Given n, is it prime?’ is thereby
revealed to belong to NP, although that fact wasn’t obvious at
a glance.
• 1903 a speaker announced in a math convention that 267 - 1 =
multiplication of two long prime numbers. Listeners must
work this one out, polynomial checking A the claimed result,
and determining yes or no.
• But in fact if one needs work very hard to find these
numbers.
• Verifying compositeness of n, Certification C(n), r  1, s  1, n
= rs.
• Checking algorithm for a pair of primes, verify that
– r  1, s  1
– n = rs.
The hard part is to convince an audience that "a certain integer n is a prime
number”.
• The rules: Allowed to do any immense amount of calculation beforehand,
and the
• results can be listed on a certificate C(n) that accompanies the integer n.
However, the audience, will need to do only a polynomial amount of
computation in order to convince themselves that n is prime. Or accept
the fact.
A primality-checking algorithm A with the following properties:
1. A ( n, C(n)): Inputs to A are the integer n and a certain certificate C(n).
2. If n is prime then A for the given inputs (n, C(n)) results on the output ‘n is
prime.’
3. If n is not prime then for every possible certificate C(n) the action of A on
the inputs (n, C(n)) results in the output ‘primality of n is not verified.’
4. Algorithm A runs in polynomial time.
Now the question is, does such a procedure exist for primality verification?
The answer is affirmative. Every prime number has a succinct certificate
1975, Pratt. This has a great importance for the developments of NPC.
• The next lemma is a kind of converse to ‘Fermat’s little theorem’.
Fermat’s little theorem.
• Lemma. Let p be a positive integer. Suppose
there is an integer x such that xp−1 = 1 (mod p)
and such that for all divisors d of (p − 1), d < (p −
1), we have xd  1 (mod p). Then p is prime.
• Proof: gcd(x, p) = 1, and suppose g = gcd(x, p).
Then x = gg', p = gg''.
• Since xp−1 = 1(mod p) we have xp−1 = 1+tp and
(gg')(p−1) − tgg'' = 1. The left side is a multiple of
g. The right side is not, unless g = 1.
Fermat’s little theorem.
•
•
•
•
•
•
•
•
•
Lemma. Let p be a positive integer. Suppose there is an integer x such that xp−1 = 1
(mod p) and such that for all divisors d of (p − 1), d < (p − 1), we have xd  1 (mod p).
Then p is prime.
Proof: p | np − n
Case: Let p | n, then, obviously, p | np − n, and we are done.
Case p ¬| n.
Consider n, 2n, 3n, . . . , (p − 1)n
By the Division Algorithm we have
n = p k1 + r1,
p | n - r1,
…,
(p-1)n = p k(p-1) + r(p-1),
p | (p-1)n - r(p-1),
•
•
0<r(p-1)p-1
r(p-1)0, since otherwise p | i n, therefore p | i, or p | n,
•
•
•
Let a, b, c, d, and p be integers such that
p | (a − b) and p | (c − d).
Then p | (ac − bd).
•
•
•
p | [(p − 1)! np−1 − (p − 1)!].
p | (p − 1)! (np−1 − 1).
p ¬| (p − 1)!, then p | (np−1 − 1).
For the tsp, suppose we produce a certificate that contains a tour, whose total length is K.
•
The checking algorithm A would then verify that the tour really does visit all of the cities and really does
have total length K. without seeking all possible K solutions through each of vertices.
•
•
•
•
The TSP, therefore, also belongs to NP.
How could a problem fail to belong to NP?’
Try this decision problem: an instance I of a G with n cities and a positive number K.
The question is ‘Is it true that there is not a tour of all of these cities whose total length is less than K?’
This is a kind of a negation of the tsp.
•
Does the negated TSP belong to NP? If so, there must be an algorithm A and a way of making a
certificate C(I) for each instance I so that absence of such a tour can be verified. Any certificate or even
any algorithm to generate C is known yet. Therefore, not known if this negation of the tsp belongs to
NP.
•
Problems belong to NP but not immediately obvious? Pratt’s algorithm is a method of producing a
certificate with a quick check that a given integer is prime.
The decision problem ‘Given n, is it prime?’ is thereby revealed to belong to NP, although that fact wasn’t
obvious at a glance.
•
•
•
•
•
•
•
•
•
•
It is very clear that PNP. Indeed if Q  P is some decision problem then we can verify membership in the
language Q with the empty certificate. That is, we don’t even need a certificate in order to do a quick
calculation that checks membership in the language because the problem itself can be quickly solved.
It seems natural to suppose that NP is larger than P. That is, one might presume that there are problems
whose solutions can be quickly checked with the aid of a certificate even though they can’t be quickly
found
in the first place.
No example of such a problem has ever been produced (and proved), nor has it been proved that no
such problem exists. The question of whether or not P=NP is the one that we cited earlier as being
perhaps
the most important open question in the subject area today.
It is fairly obvious that the class P is called ‘the class P’ because ‘P’ is the first letter of ‘Polynomial
Time.’ But what does ‘NP’ stand for? Stay tuned. The answer will appear in section 5.2.
• Whereevery I travel this world, epics lyrics.
Uncountable SET-not enumerable
• A set is uncountable if its cardinal number is larger than that of the set of
all cardinal numbers.
• Its characterization iff any holds..
– No injective correspondence (f) from X to the set of natural numbers.
– X is nonempty and there is surjective function from the set of natural
numbers to X.
– The cardinality of X is neither finite nor equal to the cardinality of natural
numbers!!
– The cardinality of X is strictly grater than cardinality of natural number.
• If cardinality of subset of a set Y is infinite, than set Y is uncountable.
• Cantor’s diagonal argument.
• Uncountable sets
– R, the cardinality of R (c or 2N0, ]1 - beth-one) is called cardinality of the
continuum. ]2 beth-two cardinality of more uncountable numbers.
– Cantor set that is an uncountable subset of R and has Hausdorff dimension
number between 0 and 1. (Fact: Any subset of R of Hausdorff dimension
greater than 0 must be uncountable).
– Earlier Cantor poses question if ]1 = N1 (aleph1), cardinality of ordinal
numbers,
– First in the list of Hilbert’s 23 questions, 1900.
Schubfachprinzip drawer principle
• n pigeons > m nests. 10 to 9, at least one nest
must be shared. At least one box must contain
no fewer than ceiling(n/m), and at least one box
must contain no more than floor(n/m).
• Collision probability with a uniform probability of
1/m is equal to
• 1-{m(m-1)..(m-n+1)}/mn
• If the cardinality of set A is greater than those of
set B, then there is NO injection from A to B.
Numbers that cannot be counted cannot be computed, but
infinite c programs can be counted.
• static void Main() 1/7..
• { Console.Write("0.");
while (true) Console.Write("142857");
• }
• Pi
• …
Cantor’s Diagonal Argument
Let T be a set of consisting all infinite sequences, and its
permutations of S=f(i).
• Characteristic function of sequence is a function f(i)=1.
• If X has characteristic function f, then its complement is 1f(i).
Sequence S is countable since possible to associate only
one number n of N for every element of the sequence S.
And T is countable..
• s1 = {1…..0 ….},  f1(i) mapping f(i)
• s2 = {……., ….},  f2(i)
• s3 =
• ..
• Thus new s is not in {s1, s2, s3, ...}.
This contradicts the assumption that the list has contained
all the sequences.
Cantor’s Diagonal Argument
• All the possible countable numbers must be
included in the set, but always possible to obtain
another s0=cmplt{diag(T)}=cmplt(fn(n))=cmplt(sn,n),
which is s0  T. Cmplt referes complement of.
• Therefore T is uncountable, since if S0. included in,
there will be another one which cannot be placed
in one-to-one correspondence with the set of N.
• There is no bijection between N and T, but T may
be subcountable.
• Although both sets are infinite in conclusion there
are more infinite sequences of real numbers than
that could be mapped with natural numbers.
Cantor’s Diagonal Argument
• Some function will not yield bijection
condition, f(t)=0.t, check if t=1000.. And its
cmpt(t).. Two different sequences
corresponds to one number. But possible to
modify function and avoid this situation that
takes us to.. I am not sure how to prove, but
it says that R and T have the same
cardinality which is called generalized
cardinality of continuum which is also given
as cardinality between |S| and |P(s)|.
Cardinality between integers and reals is
called continuum hypothesis.
Inconsistency of notions.
• P(S) is the Power of a set, defined as all the subsets of S. Cantor’s
theorem states that cardinality of P(S) is larger then those of S.
• Suppose f is a function from S to P(S),
• T={s  S: sf(s)}, second statement is from Cantor’s
diagonalization. Either s is in T (by the definition, from the same
definition f(s)T, then |T|+|f(s)||P(s)|) or if s is not in T, which is a
violation of the definition, but also and |T|+|s||P(s)|.
• Inconsistency of “set of all sets that do not contain themsleves in the
set”.
• If S were the set of all sets, then P(S) would be bigger than {S and a
subset of S}.
• Similar to Russell's Paradox, based on an unrestricted
comprehension scheme, is contradictory.
• Russell’s Paradox if let D = {x| x not in x}.
Then D in D iff D not in D, a contradiction.
• For example, the conventional proof of the unsolvability of the
halting problem is essentially a diagonal argument of Cantors arg.
• Also, diagonalization was originally used to show the existence of
arbitrarily hard complexity classes and played a key role in early
attempts to prove P does not equal NP. In 2008, diagonalization was
used to "slam the door" on Laplace's demon.1
Russell's paradox
• The set M is the set of all sets that do not contain themselves as
members. Does M contain itself?
• If it does, it is not a member of M according to the definition. If it does not,
then it has to be a member of M, again according to the definition of M.
Therefore, both of the statements "M is a member of M " and "M is not a
member of M " lead to contradictions.
For example:
• "A is an element of M if and only if A is not an element of A".
• M = {s  S| sf(s)}, or M = {x|xx}.
• 1) List of all lists that do not contain themselves.
If the "List of all lists that do not contain themselves" contains itself, then it
does not belong to itself and should be removed. However, if it does not
list itself, then it should be added to itself.
2) Barber paradox
The paradox considers a town with a male barber who shaves all and only
those men who do not shave themselves.
Function x  x < 2 is the set of all numbers less
than 2.
Set membership is via application: e member-of S iff
S(e) is true.
Since (Function x  x < 2)(1) is true, 1 is in this
"set".
Now consider P = "the set of all sets that do not
contain themselves as members"!:
P = Function x  Not(x)(x) (Note, it may make
sense to have a set with itself as member: the set
{{{{...}}}}, infinitely receding, has itself as a
member; this only happens in so-called non-wellfounded set theory).
Now, is P P? Namely is P a member of itself? This is
written:
(function x  not(x x)) (function x  not(x x)) --if this
were viewed as a D program, it would loop
forever:
Compute Not((Function x  Not(x x))(Function x 
Not(x x))))
Now, notice we have P is a member of itself if and
only if it isn't, a contradiction!
The computational realization of the paradox is that
the predicate cannot compute to true or false, so
its not a sensible logical statement.
To avoid this problem, Russell developed his theory
of types.
Avoiding Russell's paradox: type theory
and axiomatic set theory
• Old classic set of self-referential paradoxical statements causing
ambiguities that are neither true nor false could be avoided.
• In mathematical terms. the set of all sets that are members of the
themselves, permitting self-referential sets, allowing the set of all
sets that contain themselves as a member.
• But the set of all sets that do not contain themselves as a member?
are they a member of themselves.
• To avoid paradox, Russell with Whitehead propose a type theory, in
which sentences were arranged hierarchically.
• This definition avoids the paradox: the definition of R must now
define R as a set of type k set containing all sets of type k − 1 and
below that do not contain themselves as members. Since R is a type
k set, it cannot contain itself, since it cannot contain any type k sets.
This avoids the possibility of having to talk about “the set of all sets
that are not members of themselves”, because the two parts of the
sentence are of different types - that is, at different levels.
Avoiding Russell's paradox: type theory
and axiomatic set theory
• This definition avoids the paradox: the definition of R
must now define R as a set of type k set containing all
sets of type k − 1 and below that do not contain
themselves as members. Since R is a type k set, it
cannot contain itself, since it cannot contain any type k
sets. This avoids the possibility of having to talk about
“the set of all sets that are not members of themselves”,
because the two parts of the sentence are of different
types - that is, at different levels.
• Another approach to avoid such types of paradoxes was
an axiomatic set theory, proposed by Ernst Zermelo.
This theory determines what operations were allowed
and when.
Some Logics used in
axiomatizations of mathematics
• A logic usually refers to a set of rules about constructing
valid sentences. Here are a few logics. Propositional
logic concerns sentences such as (p ¬q)  (¬p  r)
where p, q, r are boolean variables. Recall that the SAT
problem consists of determining the satisfiability of such
sentences. In first order logic, the symbols allowed are
relation and function symbols as well as quantification
symbols  and . For instance, the statement xS(x) x
is a first order sentence in which x is quantified
universally, S() is a unary relation symbol and  is a
binary relation. Finally, second order logic allows
sentences in which one is allowed quantification over
structures, i.e., functions and relations. An example of a
second order sentence is SxS(x)  x, where S is a
unary relation symbol.
Corollary: There are uncountably
many subsets of N. D:\godel.html
• Hence one must accept that not possible to comprehend
the list of comprehensible sequences.
• The same applies to "sequences which God can
comprehend". Thus omniscience has some limits.
Godel showed that a formal system that is both complete and
consistent could not be created since he proved that any
formal system which is complete cannot be prevented from
including self-referential statements. The way that happens
is through something called models - a formal system is only
valid if there is a model which fits its system of rules. If a
system is capable of being represented by a model which is
complete, then it is also capable of being represented by a
model which encodes self-referential statements, and thus
comes the self referential paradox.
Alan Turing and others were interested in with a mechanical
perfect complete computing device. If such a system exists,
then one can use a computer program with the rules of that
system to enumerate every true statement - or can write a
program which can take any statement as input, and tell
whether or not that statement is true. REDUCTION.
Which was something not possible as proved by Godel. But
Turing and others found an easier way to prove its
impossibility: the halting problem.
The halting problem considers a simple question: Write a
program which looks at any arbitrary computer programs,
and determines whether or not any other prg will eventually
stop when it is run for some input.
Computing system S(p,i), which computes a function with a
pair of inputs that are a program p, and an input i for that
program of p, then, we are writing a program P which will
take a pair (Sj(q,i), j) as an input, and gives an answer of
TRUE if S(q,i) eventually halts.
By feeding the program itself we drive it into a never ending
status which means halting problem cannot return an answer
which means it will never return a solution, therefore problem
is unsolvable. Contradiction that can be also justified with
Cantor’s digitalization, that is also Godel’s incompleteness.
P() = S(i, i)+ 1, if S(i, i)={0, 1}  P={1,0}.
Question a program that is going to generate P, cannot
generate P, therefore S is incomplete.
• S is a collection of the objects that are members of the set S.
• fs(i) is a function which takes input iN, and returns “True 1 or
False 0" (decision) for values that are/aren't members of the
set, respectively.
Define a function g which just asks, what does fs return if you
give it ‘sS’ as an input?
• 1) If fs(s) returns true, then g(s) returns True; if fs(s) returns
False, then g(s) also returns False (declining f’s wrong
decision, since sS).
• 2) Think reverse scenario. sS, If fs(s) returns True, then g(s)
must return False.
• 1- g(s) is then a confirmation function for the set of all sets
that do contain themselves as members.
• 2- g(s) is also a confirmation function for the set of all sets
that donot contain themselves as members.
• Is g a member of itself? There are two different valid
functions fitting to the definition of g - one which contains
itself, and the other which doesn't contain itself: a flip of g: g'.
• Is g a member of itself? There are two different valid
functions fitting to the definition of g - one which contains
itself, one which doesn't. but just a flip of g: g'.
• g‘(s) is the function for the set of all sets that do not
contain themselves as members. if g'(s) returns true, s
does not contain itself as member of S, that is s S.
• Suppose s=g’, flipping regions.
• Inconsistency of g'. (Russel’s paradox): if g'(g') returns
true, then g' is a member of itself, which is wrong, so
g'(g') should have been false. And if g'(g') returns false,
then that means that g' is not a member of itself, which
means that g'(g') should have returned true.
• We say that the elements of a set can be counted if they can
be listed in a single sequence. N, neg or positive wouldn’t
matter.
• Cantor’s D.
• Anything that can be computed according to a finite list of
rules, can be computed by one of TM. Consistent
• Briefly, a Turing machine can be thought of as a black box,
which performs a calculation of some kind on an input
number. If the calculation reaches a conclusion, or halts
then an output number is returned.
• One of the consequences of Turing's theory is that there is a
Universal (GENERIC) Turing machine, in other words one
which can simulate all possible Turing machines. This
means that we can think of the Turing machines as
countable and listed T1, T2,... by a Universal Machine
through a sort of alphabetical listing. Turing used this to
describe his own version of Gödel's Theorem: that there is
no mechanical procedure for telling whether a Turing
machine will halt on a given input: the Halting Problem.
• The set of functions is uncountable, the set of turing machines
(programs) is countable, therefore there are more functions than
programs. By the diagonal argument Turing proves that the
existence of a TM that decides the halting problem is contradictory.
• Let's represent the result of using the nth Turing machine, Tn on the
input i as Tn(i). Suppose that there was a rule or procedure for
deciding whether or not Tn(i) halts for all values of n and i.
• But then by a similar diagonalising procedure, we can define a new
Turing machine, say D, which will halt for all inputs and return the
following output for input i:
• 0 if Ti(i) does not halt.
• Ti(i)+1 if Ti(i) does halt.
• But this machine D must be one of those machines, in other words it
must be Td for some d. However, we just defined it to give a different
answer from Td with input d. Contradiction.
• The extra sophistication here over the original diagonalising
argument lies in all the listing done is itself computable and any
machine Tn may or may not halt in carrying out its computations.
None of this enters into Cantor's original diagonal argument.
Computable sequences
• A sequence f(i) is computable if there is a program (Machine) for
given input i computes f(i).
• Gödel proved his Completeness Theorem, namely that a
formula is provable from the axioms iff it is valid.
• Godel's First InCompleteness Theorem. Any adequate
axiomatizable theory is incomplete. In particular the sentence
"This sentence is not provable" is true but not provable in the
theory. Enumerate
• Godel's Second Incompleteness Theorem. In any consistent
axiomatizable theory (axiomatizable means the axioms can be
computably generated) which can encode sequences of numbers
(and thus the syntactic notions of "formula", "sentence", "proof") the
consistency of the system is not provable in the system.
• The Liar Paradox. "Truth" for English sentences is not definable in
English. Proof. Suppose it is. Then so is its complement "False".
Let s be the sentence "This sentence is false" .
Since the phrase "This sentence" refers to s, we have
s iff "This sentence is false" iff "s is false" iff not s.
A contradiction to the statement. “Everything I say is a lie, I am lying”
What can be computable in principle
•
•
•
•
•
Alonzo Church defined the Lambda calculus,
Kurt Gödel defined Recursive functions,
Stephen Kleene defined Formal systems,
Markov defined, Markov algorithms,
Emil Post and Alan Turing defined abstract machines now known as
Post machines and Turing machines.
• Church Turing thesis. Anything computable with these computational
methodologies is computable by a Turing machine.
• Earlier 1900, David Hilbert believed that all of mathematics could be
precisely axiomatized. Once this LIST was completed, there would
be an “effective procedure”, i.e., an algorithm that would accept any
precise mathematical statement as input, and, after a finite number
of steps, it would reach to decision whether the statement was true
or false. YES or NO statement.
• Hilbert was introducing what is called now a decision procedure for
all of mathematics.
Satisfiable and Valid Structures
• “Hilbert considered the validity problem for first-order logic”. First-order
logic is a mathematical language in which most mathematical
statements can be formulated. This is a special case of decision
problem.
• Every statement in first-order logic has a precise meaning in every
appropriate logical structure, i.e., it is true or false in each such
structure. Those statements that are true in every appropriate structure
are called valid. Those statements that are true in some structure are
called satisfiable.
• Notice that a formula, Φ, is valid iff its negation, ¬Φ, is not satisfiable.
• Hilbert calls First Order Logic as entscheidungsproblem.
• In a textbook, Principles of Mathematical Logic by Hilbert and
Ackermann, the authors wrote, “The Entscheidungsproblem is solved
when we know a procedure that allows for any given logical expression
to decide by finitely many operations its validity or satisfiability. SAT or
3SAT.. Etc..
• Axioms and Inference rules.
• 1930, Gödel’s presents a complete axiomatization of first-order logic,
based on the Principia Mathematica by Whitehead and Russell
• He proves in his Completeness Theorem, that a formula is provable
from the axioms iff it is valid.
• In particular, since the axioms are easily recognizable, and rules of
inference very simple, there is a mechanical procedure that can LIST
out all proofs.
• Each line in a proof is either an axiom, or a inference following from the
previous lines by one of the simple rules.
• For any given string of characters, we can tell if it is a proof.
• Thus we can systematically list all strings of characters and check
whether each one is a proof. If so, then we can add the proof's last line
to our list of theorems.
•
In this way, we can list out all theorems, i.e., exactly all the valid
formulas of first-order logic, can be listed out by a simple mechanical
procedure.
• More precisely, the set of valid formulas is the
range of a computable function. In modern
terminology we say that the set of valid formulas of
first-order logic is recursively enumerable (r.e.).
• “Yes, Φ is valid.” However, if Φ were not valid then
we might never find this fact out. The list of all the
non-valid formulas, or the list of all satisfiable
formulas.
• Gödel’s Incompleteness Theorem: there is no
complete and computable axiomatization of the
first-order theory of the natural numbers. That is,
there is no reasonable list of axioms from which we
can prove exactly all true statements of number
theory (Gödel 1931).
• Church and Turing independently prove that the
entscheidungsproblem is unsolvable. Turing’s
unsolvability of halting problem.
Application of halting: Goldbach’ s
conjecture
• Conjecture: (even n)  (n2), can be
represented with two prime numbers p and q as
n=p+q.
• That could be disproven via counterexample by
simulating an n-state TM, such that quits (halts-if
the result is TRUE) if finds a counterexample, an
even n  (p+q), {p, q}primes. Then the
conjecture will be disproven.
• If the result is FALSE, then the conjecture is
proven. That means halting algorithm for this
problem must not halt.
• FSM, Busy Beaver. Next week.