Computation
Binary Numbers
• Decimal numbers
• Binary numbers
http://schoolscience.rice.edu/duker/robots/binarynumber.html
Representing Text
• Decide how many characters we need to represent.
• Determine the required number of bits.
• Ascii: 7 bits. Can encode 27 = 128 different symbols.
Ascii
http://www.neurophys.wisc.edu/comp/docs/ascii.html
Representing Text
Four …
01000110 01101111 01110101 01110010
When We Need More Characters
What about things like:
简体字
When We Need More Characters
What about things like:
简体字
Answer: Unicode: 32 bits. Over 4 million characters.
http://www.unicode.org/charts/
Digital Images
RGB
The red channel
RGB
The green channel
RGB
Red
Green
Blue
Representing Pictures
Representing Sounds
Representing Programs
public static TreeMap<String, Integer> create() throws IOException
public static TreeMap<String, Integer> create() throws IOException
{ Integer freq;
String word;
TreeMap<String, Integer> result = new TreeMap<String,
Integer>();
JFileChooser c = new JFileChooser();
int retval = c.showOpenDialog(null);
if (retval == JFileChooser.APPROVE_OPTION)
{ Scanner s = new Scanner( c.getSelectedFile());
while( s.hasNext() )
{ word = s.next().toLowerCase();
freq = result.get(word);
result.put(word, (freq == null ? 1 : freq + 1));
}
}
return result;
}
}
The Roots of Modern Technology
5thc B.C. Aristotelian logic
invented
1642
Pascal built an
adding machine
1694
Leibnitz reckoning
machine
The Roots, continued
1834
Charles Babbage’s
Analytical Engine
Ada writes of the engine, “The
Analytical Engine has no
pretensions whatever to
originate anything. It can do
whatever we know how to
order it to perform.”
The picture is of a model built in the late 1800s by Babbage’s son
from Babbage’s drawings.
The Roots: Logic
1848 George Boole The Calculus of Logic
chocolate and  nuts and mint
chocolate
nuts
mint
Boolean Logic
P
Q
P
PQ
PQ
PQ
PQ
True
True
False
True
True
True
True
True
False
False
True
False
False
False
False
True
True
True
False
True
False
False
False
True
False
False
True
True
Using Boolean Logic
P
Q
P
P  Q P  Q (P  Q) Q P  ((P  Q) Q)
True
True
False True
True
True
False
True
False
False True
False
False
False
False
True
True
True
True
True
True
False
False
True
False
True
True
True
How Big Are the Truth Tables?
P
Q
R
True
True
True
True
True
False
True
False
True
True
False
False
False
True
True
False
True
False
False
False
True
False
False
False
2n
1200000
1000000
800000
600000
400000
200000
0
1
3
5
7
9
11
13
15
17
19
21
Increased Power of Quantified Logic
•
Bear(Smoky).
•
x (Bear(x)  Animal(x)).
•
x (Animal(x)  Bear(x)).
•
x (Animal(x)  y (Mother-of(y, x))).
•
x ((Animal(x)  Dead(x))  Alive(x)).
Using Quantified Logic (FOL)
We can represent what we know as a set of logical formulas
and then manipulate them.
Facts in English
(1) Marcus was a man.
(2) Marcus was a Pompeian.
(3) All Pompeians were Romans.
(4) Caesar was a ruler.
(5) All Romans were either loyal to Caesar or hated him.
(6) Everyone is loyal to someone.
(7) People only try to assassinate rulers they are not loyal to.
(8) Marcus tried to assassinate Caesar.
Facts in First Order Logic (FOL)
(1) Marcus was a man.
man(Marcus)
(2) Marcus was a Pompeian.
Pompeian(Marcus)
(3) All Pompeians were Romans.
x Pompeian(x)  Roman(x)
(4) Caesar was a ruler.
ruler(Caesar)
(5) All Romans were either loyal to Caesar or hated him.
x Roman(x)  loyalto(x, Caesar)  hate(x, Caesar)
(6) Everyone is loyal to someone.
x y loyalto(x, y)
(7) People only try to assassinate rulers they are not loyal to.
x y person(x)  ruler(y)  tryassassinate(x, y)  loyalto(x, y)
(8) Marcus tried to assassinate Caesar.
tryassassinate(Marcus, Caesar)
Search
Breadth-First Search
Is this a good idea?
Depth-First Search
Scalability
Solving hard problems
requires search in a large
space.
To play master-level
chess requires searching
about 8 ply deep. So
about 358 or 21012 nodes
must be examined.
Growth Rates of Functions
The Advent of the Computer
1945 ENIAC The first electronic digital computer
1949 EDVAC
The first stored
program computer
Moore’s Law
http://www.intel.com/technology/mooreslaw/
How Much Computer Power Might It Take?
http://www.frc.ri.cmu.edu/~hpm/book97/ch3/index.html
How Much Compute Power is There?
From Hans Moravec, Robot Mere Machine to Transcendent Mind 1998.
Kurweil’s Vision
http://www.pocket-lint.co.uk/news/news.phtml/12920/13944/Computersmatch-humans-by-2030.phtml
Limits to What We Can Compute
Are there fundamentally uncomputable things?
• Does God exist?
• What’s the best way to run a country?
• Does this puzzle have a solution?
Mathematics in the Early 20th Century
1900 Hilbert’s program and the effort to formalize
mathematics
1931 Kurt Gödel’s paper, On Formally Undecidable
Propositions
1936 Alan Turing’s paper, On Computable Numbers with
an application to the Entscheidungs problem
What Can Algorithms Do?
1.
Can we make all true statements theorems?
2.
Can we decide whether a statement is a theorem?
Gödel’s Incompleteness Theorem
Kurt Gödel showed, in the proof of his
Incompleteness Theorem [Gödel 1931], that the
answer to question 1 is no. In particular, he
showed that there exists no decidable
axiomatization of Peano arithmetic that is both
consistent and complete.
The Entscheidungsproblem
Does there exist an algorithm to decide, given an arbitrary
sentence w in first order logic, whether w is valid?
Given a set of axioms A and a sentence w, does there exist
an algorithm to decide whether w is entailed by A?
Given a set of axioms, A, and a sentence, w, does there exist
an algorithm to decide whether w can be proved from A?
Turing Machines
At each step, the machine must:
● choose its next state,
● write on the current square,
● move left or right.
and
An Example
The Church-Turing Thesis
All formalisms powerful enough to describe everything we
think of as a computational algorithm are equivalent.
• Turing machines
• Lambda calculus
• Standard programming languages like Java
• Conway’s game of life
• DNA computing
The Game of Life
Playing the game
The rules:
● A dead cell with exactly three live neighbors becomes
a live cell (birth).
● A live cell with two or three live neighbors stays alive
(survival).
● In all other cases, a cell dies or remains dead
(overcrowding or loneliness).
A game halts iff it reaches some stable configuration.
Back to the Entscheidungsproblem
Theorem: The Entscheidungsproblem is unsolvable.
Proof: (A variant of Turing’s proof)
1.
2.
3.
4.
Given a Turing machine M, we can construct a logical
formula F that is true iff M ever halts.
If there were a solution to the Entscheidungsproblem,
then we could determine the truth of F and thus be
able to decide whether M ever halts.
But there is no procedure for determining (in the
general case) whether M halts.
So there is no solution to the Entscheidungsproblem.
The Halting Problem
Turing machine, M
input string, w
Does M halt on w?
Accept
Reject
An Example of a Similar Question
Does this program halt on all inputs?
times3(x: positive integer) =
While x  1 do:
If x is even then x = x/2.
Else x = 3x + 1.
Let’s try it.
The Halting Problem Is Undecidable
Proof: If it were decidable, then some TM MH would
decide it. MH would implement the specification:
halts(<M: string, w: string>) =
If <M> is a Turing machine description
and M halts on input w
then accept.
Else reject.
Trouble
Trouble(x: string) =
if halts(x, x) then loop forever, else halt.
What is Trouble(<Trouble>)?
What is halts(<Trouble, Trouble>)?
● If halts reports that Trouble(<Trouble>) halts,
Trouble ________.
● If halts reports that Trouble(<Trouble>) does not halt,
Trouble __________.
Another Undecidable Problem
The Post Correspondence Problem
A PCP Instance With a Simple Solution
i
X
Y
1
b
aab
2
abb
b
3
aba
a
4
baaa
baba
A PCP Instance With a Simple Solution
i
X
Y
1
b
aab
2
abb
b
3
aba
a
4
baaa
baba
Solution: 3, 4, 1
Another PCP Instance
i
X
Y
1
11
011
2
01
0
3
001
110
A PCP Instance With No Simple Solution
i
X
Y
1
1101
1
2
0110
11
3
1
110
A PCP Instance With No Simple Solution
i
X
Y
1
1101
1
2
0110
11
3
1
110
Shortest solution has length 252.
A Tiling Problem
Given a finite set T
of tiles of the
form:
Is it possible to tile
an arbitrary
surface in the
plane?
A Set of TilesThat Cannot Tile the Plane
Is the Tiling Problem Decidable?
Wang’s conjecture: If a given set of tiles can be used to tile an
arbitrary surface, then it can always do so periodically. In other
words, there must exist a finite area that can be tiled and then
repeated infinitely often to cover any desired surface.
But Wang’s conjecture is false.
The Implications
• The Entscheidungs problem is undecidable.
• There’s no black box reasoning engine for FOL.
• Would quantum computing change the picture?
• Does undecidability doom our attempt to make artificial
copies of ourselves?
Is Decidability Enough?
Boolean Logic, Again
P
Q
P
P  Q P  Q (P  Q) Q P  ((P  Q) Q)
True
True
False True
True
True
False
True
False
False True
False
False
False
False
True
True
True
True
True
True
False
False
True
False
True
True
True
How many steps would it take a deterministic Turing machine
to examine this table?
Nondeterminism
Nondeterministically Deciding SAT
P
Q
P
P  Q P  Q (P  Q) Q P  ((P  Q) Q)
True
True
False True
True
True
False
True
False
False True
False
False
False
False
True
True
True
True
True
True
False
False
True
False
True
True
True
How many steps in a single path of the process of examining
this table?
P and NP
• P – problems that are solvable deterministically in
polynomial time.
• NP – problems that are solvable nondeterministically in
polynomial time.
The Traveling Salesman Problem
15
25
10
28
20
4
8
40
9
7
3
23
Given n cities and the distances between each pair of
them, find the shortest tour that returns to its starting point
and visits each other city exactly once along the way.
The Traveling Salesman Problem
15
25
10
28
20
4
8
40
9
7
3
23
Given n cities:
Choose a first city
Choose a second
Choose a third
…
n
n-1
n-2
n!
The Traveling Salesman Problem
Can we do better than n!
● First
city doesn’t matter.
● Order doesn’t matter.
So we get (n-1!)/2.
The Growth Rate of n!
2
2
11
479001600
3
6
12
6227020800
4
24
13
87178291200
5
120
14
1307674368000
6
720
15
20922789888000
7
5040
16
355687428096000
8
40320
17
6402373705728000
9
362880
18
121645100408832000
10
3628800
19
2432902008176640000
11
39916800
36
3.61041
The Traveling Salesman Problem
Solving it Nondeterministically
15
25
10
28
20
4
8
40
9
7
3
23
SAT and Traveling Salesman
D
ND
2n
n
Traveling Salesman n!
n
SAT
P and NP
• P – problems that are solvable deterministically in
polynomial time.
• NP – problems that are solvable nondeterministically in
polynomial time.
• NP-complete – NP problems that are at least as hard as
every other NP-complete problem.
Growth Rates of Functions, Again
Does Complexity Doom AI?