Notables in LinkedList History
Dr. Philip Cannata
1
Dr. Philip Cannata
2
Notables in LinkedList History
Aristotle
Categories and Syllogism
Object Oriented Concepts and Syllogistic
Logic.
Euclid
5 axioms of Geometry in the
“The Elements”
“The Elements” was responsible for the
notion of Certainty until the discovery of
non-Euclidian Geometry in the 19th
Century.
Gottlob Frege
Modern mathematical logic
Propositional Calculus and Predicate
Calculus
Giuseppe Peano
The principles of arithmetic,
presented by a new method.
First attempt at an axiomatization of
mathematics/arithmetic.
Georg Cantor
Theory of sets, sizes of Infinity,
and paradoxes.
Mathematics can be axiomatized using set
theory.
David Hilbert, Alfred
Whitehead, and
Bertrand Russell
Principia Mathematica
A 30 year attempt to restore Certainty to
mathematics by formalizing it using set
theory and logic and to prove that
mathematics is “consistent”.
Thoralf Skolem
Primitive Recursive Functions:
A reaction to the works of Cantor, Hilbert,
Whitehead and Russell in which the
notion of Infinity is abandoned.
Dr. Philip Cannata
3
Notables in LinkedList History
Kurt Gödel
First Incompleteness
Theorem
In any formal system capable of expressing primitive
recursive functions/arithmetic the statement “I am a
statement for which there is no proof” can be generated.
Second Incompleteness
Theorem
No formal system capable of expressing primitive
recursive functions/arithmetic can prove its own
consistency.
Restored the notion of Certainty but in a manner very different from that envisioned by Hilbert,
Whitehead and Russell.
Alonso
Church
Alan Turing
ChurchTuring Thesis
Recursive Functions
Functions which are defined for every input. Infinity returns.
Lambda Calculus
A language for defining functions and function application
inspired by Gödel's recursive functions.
Undecidability of
equivalence
There is no algorithm that takes as input two lambda
expressions and determines if they are equivalent.
Turing Machine and
Halting Problem
Created a theoretical model for a machine, (Turing machine),
that could carry out calculations from inputs. There is in
general no way to tell if it will halt (undecidability of halting.)
Recursive Functions = Effectively Computable = Computational Completeness =
calculable on a Turing machine = Turing-Complete (e.g., Lambda Calculus).
For more information see http://www.cs.utexas.edu/~ear/cs341/automatabook/
Haskell Currie Combinator Logic
Dr. Philip Cannata
Developed a Turing-Complete language based solely
upon function application of combinators.
4
Notables in LinkedList History
John McCarthy
The lisp programming language.
1960 paper: “Recursive Functions
of Symbolic Expressions and Their
Computation by Machine”, see
class calendar for a copy.
McCarthy took concepts from Gödel's incompleteness proof (substitution), lambda calculus (function
definition and function application) and combinator logic (car, cdr, and cons as primitive operations on
linked-lists)
(let ((l (cons 'a (cons 'b '())))) (let ((first (lambda (x) (car x)))) (first l)))
Dr. Philip Cannata
5
Good Books to Have for a Happy Life J
From Frege to Gödel:
Dr. Philip Cannata
My Favorite
6
Supplemental Material
(The following pages are not required for this course unless explicitly stated
otherwise by Dr. Cannata in, for instance, Review Notes for an Exam.)
Dr. Philip Cannata
7
Aristotle’s syllogistic logic
If it’s raining outside, then the grass is wet.
Its raining outside
Therefore, the grass is wet.
If it’s raining outside, then the grass is wet.
The grass is wet.
Therefore, its raining outside.
Dr. Philip Cannata
Aristotle
Valid argument
Fallacy
8
Euclid’s “The Elements”
Five axioms (postulates)
Dr. Philip Cannata
Euclid
9
Propositional Logic
P
Q
False
False
False
P
Q
P || Q
P
¬P
False
False
False
False
False
True
True
False
False
True
True
True
False
True
False
False
True
False
True
True
True
True
True
True
True
P
False
False
True
True
Q
P
Q
False True
True True
False False
True True
Q
P<=>Q
Dr. Philip Cannata
P && Q
→
P
False
False
True
False
True
False
True
False
False
True
True
True
Frege
10
Reasoning with Truth Tables
Proposition: ((P ∨ Q)∧((¬P)→Q))
P
Q
(P ∨ Q)
(¬P)
False False
False
True
False
False
False True
True
True
True
True
True
False
True
False
True
True
True
True
True
False
True
True
If prop is True when all
variables are True:
P, Q
((P∨Q)∧((¬P)→Q))
((¬P)→Q)
Some True: prop is Satisfiable*
If they were all True: Valid / Tautology
A Truth
double turnstile
Dr. Philip Cannata
((P∨Q)∧((¬P)→Q))
All False: Contradiction
(not satisfiable*)
*Satisfiability was the first known NP-complete (see next slide) problem
Frege
11
The class P consists of all those
decision problems (see Delong,
page 159) that can be solved on a
deterministic sequential machine
in an amount of time that is
polynomial in the size of the
input; the class NP consists of all
those decision problems whose
positive solutions can be verified
in polynomial time given the right
information.
A problem is NP-hard if an algorithm for solving it can be translated into one for solving any NP-problem (nondeterministic
polynomial time) problem. NP-hard therefore means "at least as hard as any NP-problem," although it might, in fact, be harder.
Complexity Theory
•  If there is a polynomial algorithm for any NP-hard problem, then there are
polynomial algorithms for all problems in NP, and hence P = NP;
•  If P ≠ NP, then NP-hard problems have no solutions in polynomial time,
while P = NP does not resolve whether the NP-hard problems can be solved
in polynomial time;
•  A common mistake is to think that the NP in NP-hard stands for nonpolynomial. Although it is widely suspected that there are no polynomialtime algorithms for NP-hard problems, this has never been proven.
Moreover, the class NP also contains all problems which can be solved in
polynomial time.
Dr. Philip Cannata
NP-Complete
12
Tautological Proof using Propositional Logic
It’s either summer or winter.
If it’s summer, I’m happy.
If it’s winter, I’m happy.
Is there anything you can uncompress from this?
The above statements can be written like this: ( (s ∨ w) ^ (s -> h) ^ (w -> h) ) -> h
This is a Haskell proof of this Tautology
valid3 :: (Bool -> Bool -> Bool -> Bool) -> Bool
valid3 bf = and [ bf r s t| r <- [True,False],
s <- [True,False],
t <- [True,False]]
LOGIC> valid3 (\ s w h -> ((s || w) && (s ==> h) && (w ==> h)) ==> h)
True
Another form of a un-compression (proof):
Dr. Philip Cannata
( (p ∨ q) ^ (¬p ∨ r) ^ (¬q ∨ r) ) -> r
( (p ^ ¬p) ∨ (r ^ q) ^ (¬q ∨ r) ) -> r
(F ∨ (q ^ ¬q) ∨ r) -> r
r -> r
¬r ∨ r
.: T
Frege
13
Why Reasoning with Truth Tables is Infeasible
Works fine when there are 2 variables
{T,F} × {T,F} = set of potential values of variables
2 × 2 lines in truth table
Three variables — starts to get tedious
{T,F} × {T,F} × {T,F} = set of potential values
2 × 2 × 2 lines in truth table
Twenty variables — definitely out of hand
2 × 2 × … × 2 lines (220)
You want to look at a million lines?
If you did, how would you avoid making errors?
Hundreds of variables — not in a million years
à A need for Predicate Logic. We’ll look at this with Prolog.
Dr. Philip Cannata
Frege
14
Predicate Logic Example in Prolog
Prolog file “04 prolog examples.p”
Run prolog with the file:
parent(hank,ben).
parent(hank,denise).
parent(irene,ben).
parent(irene,denise).
parent(alice,carl).
parent(ben,carl).
parent(denise,frank).
parent(denise,gary).
parent(earl,frank).
parent(earl,gary).
$ gprolog
GNU Prolog 1.4.1
By Daniel Diaz
Copyright (C) 1999-2012 Daniel
Diaz
| ?- ['04 prolog examples.p'].
| ?- grandparent(A, B).
A = hank
B = carl
A = hank
B = frank
grandparent(X,Z) :- parent(X,Y) , parent(Y,Z).
This prolog statement says X is the grandparent of Z if
X is the parent of Y and Y is the parent of Z.
A = hank
B = gary
A = irene
B = carl
A = irene
B = frank
A = irene
B = gary
Dr. Philip Cannata
Frege
15
Peano's Axioms
1. Zero is a number.
2. If a is a number, the successor of a is a number.
3. zero is not the successor of a number.
4. Two numbers of which the successors are equal are themselves
equal.
5. (induction axiom.) If a set S of numbers contains zero and also
the successor of every number in S, then every number is in S.
Peano's axioms are the basis for the version of number theory
known as Peano arithmetic.
Dr. Philip Cannata
Peano
16
Peano's Arithmetic
Dr. Philip Cannata
Peano
17
Cantor Set Theory
Dr. Philip Cannata
Cantor
18
Cantor Diagonalization
Create a new number from the diagonal by adding 1 and changing
10 to 0.
The above example would give .960143…
Now try to find a place for this number in the table above, it can’t
be the first line because 8 != 9, it can’t be the second line because
5 != 6, etc. to infinity. So this line isn’t in the table above and
therefore was not counted. è The real numbers are not countable.
Dr. Philip Cannata
Cantor
19
Set Theory Paradoxes
Suppose there is a town with just one barber, who is male. In this town, every man keeps himself
clean-shaven, and he does so by doing exactly one of two things:
shaving himself; or
going to the barber.
Another way to state this is that "The barber is a man in town who shaves all those, and only
those, men in town who do not shave themselves."
From this, asking the question "Who shaves the barber?" results in a paradox because according
to the statement above, he can either shave himself, or go to the barber (which happens to be
himself). However, neither of these possibilities is valid: they both result in the barber shaving
himself, but he cannot do this because he only shaves those men "who do not shave themselves".
Dr. Philip Cannata
Cantor and Russell
20
Principia Mathematica
Dr. Philip Cannata
Whitehead and Russell
21
Principia Mathematica
Dr. Philip Cannata
Whitehead and Russell
22
Primitive Recursive Functions and Arithmetic
(see “A Profile of Mathematical Logic” by Howard Delong – pages 152 – 160)
A Sequence of Functions from definitions on DeLong, page 157:
Book notation
Relation notation
Arithmetic notation
f1(x) = x’
(x (+ x 1))
f1 is the successor
function
f2(x) = x
(x x)
f2 is the identity
function with i = 1
f3(y, z, x) = z
(y z x z)
f3 is the identity
function with i = 2
f4(y, z, x) = f1(f3(y,z,x))
(y z x ((x (+ x 1)) (y z x z)))
f4 is defined by
substitution for f1 and f3
This is how you would do this in lisp
(let ((f1 (lambda (x) (+ x 1))) (f3 (lambda (y z x) z))) (let ((f4 (lambda (y z x) (f1 (f3 y z x))))) (f4 2 4 6)))
f5(0, x) = f2(x)
f5(y’, x) = f4(y, f5(y,x), x)
(0 x ( x x))
(not doable yet)
f5 is defined by recursion
and f2 and f4
f5 is primitive recursive addition
(let ((f1 (lambda (x) (+ x 1))) (f2 (lambda (x) x)) (f3 (lambda (y z x) z))) (let ((f4 (lambda (y z x) (f1 (f3 y z x))))) (letrec ((f5 (lambda (a b) (if
(= a 0) (f2 b) (f4 (- a 1) (f5 (- a 1) b) b))))) (f5 2 3))))
Dr. Philip Cannata
Skolem
23
Gödel Numbering
1!
3!
5!
7!
9!
11!
13!
17!
19!
23!
‘0’!
‘’’!
‘-’!
‘=>’!
‘V’!
‘(‘!
‘)!
‘x’!
‘y’!
‘z’!
29!
31!
37!
41!
43!
47!
53!
…!
‘=‘!
‘+’!
‘.’!
‘x1’!
‘y1’!
!
‘z1’!
!
‘z2’!
!
…!
1 = (0)’ = 211 x 31 x 513 x 73
Dr. Philip Cannata
Gödel
24
Gödel’s First Incompleteness Theorem
See Gödel's paper on the class calendar.
Dr. Philip Cannata
Gödel
25
Two Lambda Calculus Examples (beta-reduction)
1) start with:
(((λf.λg.λx.(f (g x)) λs.(s s)) λa.λb.b) λx.λy.x)
and reduce as follows:
((λg.λx.(λs.(s s) (g x)) λa.λb.b) λx.λy.x)
(λx.(λs.(s s) (λa.λb.b x)) λx.λy.x)
(λs.(s s) (λa.λb.b λx.λy.x))
((λa.λb.b λx.λy.x) (λa.λb.b λx.λy.x))
(λb.b (λa.λb.b λx.λy.x))
(λa.λb.b λx.λy.x)
λb.b
2) start with:
(λh.((λa.λf.(f a) h) h) λf.(f f))
and reduce as follows:
((λa.λf.(f a) λf.(f f)) λf.(f f))
(λf.(f λf.(f f)) λf.(f f))
(λf.(f f) λf.(f f))
(λf.(f f) λf.(f f))
repeats indefinitely
Dr. Philip Cannata
Church
26
Lambda Calculus
Lambda Calculus!
λx.x!
λs.(s s)!
λfunc.λarg.(func arg)!
def identity = λx.x!
def self_apply = λs.(s s) !
def apply = λfunc.λarg.(func arg)!
def select_first = λfirst.λsecond.first!
def select_second =λfirst.λsecond.second!
def cond= λe1.λe2.λc.((c e1) e2)!
def true=select_first !
def false=select_second!
def not= λx.(((cond false) true) x)!
Or def not= λx.((x false) true)!
def and= λx.λy.(((cond y) false) x)!
Or def and= λx.λy.((x y) false)!
def or= λx.λy.(((cond true) y) x)!
Or def or= λx.λy.((x true) y)!
Dr. Philip Cannata
Church
27
Lambda Calculus Arithmetic (Church Numerals)
def true = select_first
def false = select_second
def
def
def
def
zero =
succ =
pred =
iszero
def
def
def
def
def
def
identity = λx.x
self_apply = λs.(s s)
apply = λfunc.λarg.(func arg)
select_first = λfirst.λsecond.first
select_second =λfirst.λsecond.second
cond= λe1.λe2.λc.((c e1) e2)
λx.x
λn.λs.((s false) n)
λn.(((iszero n) zero) (n select_second))
= λn.(n select_first)
one = (succ zero)
(λn.λs.((s false) n) zero)
λs.((s false) zero)
two = (succ one)
(λn.λs.((s false) n) λs.((s false) zero))
λs.((s false) λs.((s false) zero))
three = (succ two)
(λn.λs.((s false) n) λs.((s false) λs.((s false) zero)))
λs.((s false) λs.((s false) λs.((s false) zero)))
(iszero zero)
(λn.(n select_first) λx.x)
(λx.x select_first)
select_first
Dr. Philip Cannata
(iszero one)
(λn.(n select_first) λs.((s false) zero) )
(λs.((s false) zero) select_first)
((select_first false) zero)
Church
28
Lambda Calculus Arithmetic
def addf = λf.λx.λy.
if iszero y
then x
else f f (succ x)(pred y)
ADDITION
def add = λx.λy.
if iszero y
then x
else addf addf (succ x)(pred y)
add one two
(((λx.λy.
if iszero y
then x
else addf addf (succ x)(pred y)) one) two)
if iszero two
then one
else addf addf (succ one)(pred two)
Multiplication
def multf = λf.λx.λy.
if iszero y
then zero
else add x (f x (pred y)))
def recursive λf.(λs.(f (s s)) λs.(f (s s)))
def mult = recursive multf = λx.λy
if iszero y
then zero
else add x ((λs.(multf (s s)) λs.(multf (s s))) x (pred y))
Church-Turing thesis: no
formal language is more
powerful than the lambda
calculus or the Turing machine
which are both equivalent in
expressive power.
addf addf (succ one)(pred two)
((((λf.λx.λy
if iszero y
then x
else f f (succ x)(pred y)) addf) (succ one))(pred two))
if iszero (pred two)
then (succ one)
else addf addf (succ (succ one))(pred (pred two))
addf addf (succ (succ one)) (pred (pred two))
((((λf.λx.λy
if iszero y
then x
else f f (succ x)(pred y)) addf) (succ (succ one)))(pred (pred
two)))
if iszero (pred (pred two))
then (succ (succ one)
else addf addf (succ (succ (succ one))) (pred (pred (pred two)))
(succ (succ one))
Three
Dr. Philip Cannata
Church
29
Turing Machine
Dr. Philip Cannata
Turing
30
Combinator Logic
A function is called primitive recursive if there is a finite sequence of functions ending with f
such that each function is a successor, constant or identity function or is defined from preceding
functions in the sequence by substitution or recursion.
Combinators
s f g x = f x (g x)
k x y
= x
b f g x = f (g x)
c f g x = f x g
y f
= f (y f)
cond p f g x = if p x then f x else g x
-- Some Primitive Recursive Functions on Natural Numbers
addition x z = y (b (cond ((==) 0) (k z)) (b (s (b (+) (k 1))
) (c b pred))) x
multiplication x z = y (b (cond ((==) 0) (k 0)) (b (s (b (pradd1) (k z)) ) (c b pred))) x
exponentiation x z = y (b (cond ((==) 0) (k 1)) (b (s (b (prmul1) (k x)) ) (c b pred))) z
factorial x
= y (b (cond ((==) 0) (k 1)) (b (s (prmul1)
) (c b pred))) x
No halting problem here but not Turing complete either
Implies recursion or bounded loops, if-then-else constructs and run-time stack.
see the BlooP language in
Dr. Philip Cannata
Haskell Currie
31
John McCarthy’s Takeaways
-- Primitive Recursive Functions on Lists are more interesting than Primitive Recursive Functions on
Numbers
length x = y (b (cond ((==) []) (k 0)) (b (s (b (+) (k 1)) ) (c b cdr))) x
sum x = y (b (cond ((==) []) (k 0)) (b (s (b (+) (car)) ) (c b cdr))) x
product x = y (b (cond ((==) []) (k 1)) (b (s (b (*) (car)) ) (c b cdr))) x
map f x = y (b (cond ((==) []) (k [])) (b (s (b (:) (f) ) ) (c b cdr))) x
-- map (\ x -> (car x) + 2) [1,2,3] or
-- map (\ x -> add (car x) 2) [1,2,3]
-- A programming language should have first-class functions as (b p1 p2 . . . Pn), substitution, lists with car,
cdr and cons operations and recursion.
car (f:r) = f
cdr (f:r) = r
cons is : op
John’s 1960 paper: “Recursive Functions of Symbolic Expressions
and Their Computation by Machine” – see copy on class calendar.
Dr. Philip Cannata
McCarthy
32