CS-INFO 372:
Explorations in Artificial Intelligence
Prof. Carla P. Gomes
gomes@cs.cornell.edu
Introduction
http://www.cs.cornell.edu/courses/cs372/2008sp
Carla P. Gomes
INFO372
INFO372 – Explorations in
Artificial Intelligence
Course Administration
Lectures: Tuesday and Thursday - 10:10 - 11:25
Location: Phillips Hall, room 307
Lecturer: Prof. Gomes
Office: 5133 Upson Hall
Phone: 255 9189
Email: gomes@cs.cornell.edu
Administrative Assistant: Beth Howard
(bhoward@cs.cornell.edu)
5136 Upson Hall, 255-4188
TAs: Robert Xiao rkx2@cornell.edu
Yunsong Guo <guoys@cs.cornell.edu>
Web Site: http://www.cs.cornell.edu/courses/cs372/2008sp
Carla P. Gomes
INFO372
Office Hours
TAs:
Robert Xiao rkx2@cornell.edu
Yunsong Guo guoys@cs.cornell.edu
TBA
TBA
Prof. Gomes:
Office: 5133 Upson Hall
Wednesdays 12:00 – 1:00 p.m.
If you need to meet with me at a different time please
schedule an appointment by email.
Carla P. Gomes
INFO372
Grades
Midterm
(30%)
Homework
(25%)
Participation
(5%)
Final
(40%)
Homework is very important. It is the best way for you to learn the
material. You are encouraged to discuss the problems with your
classmates, but all work handed in should be original, written by
you in your own words. No late homework will be accepted
Carla P. Gomes
INFO372
Textbook
Artificial Intelligence: A Modern Approach (AIMA)
(Second Edition) by Stuart Russell and Peter Norvig
Artificial Intelligence : A New Synthesis
By Nils Nilsson
Principles of Constraint Programming
By Krzysztof Apt
Linear Programming by Vasek Chvatal
Carla P. Gomes
INFO372
Overview of this Lecture
• Course Administration
• What is Artificial Intelligence?
• Course Themes, Goals, and Syllabus
Carla P. Gomes
INFO372
What is Artificial Intelligence (AI)?
What is Intelligence?
Historical Perspective of AI
State-of-the-art and Challenges
Carla P. Gomes
INFO372
What is AI?
Ambitious goals:
– understand “intelligent” behavior
– build “intelligent” agents
Carla P. Gomes
INFO372
What is Intelligence?
• Intelligence:
– “the capacity to learn and solve problems”
(Webster dictionary)
– the ability to act rationally
• Artificial Intelligence:
– build and understand intelligent entities
– synergy between:
philosophy, psychology, and cognitive science
computer science and engineering
mathematics and physics
Carla P. Gomes
INFO372
AI Leverages
from Different Disciplines
Philosophy
e.g., foundational issues in logic, methods of reasoning,
mind as physical system, foundations of learning,
language, rationality
Computer science and engineering
e.g., complexity theory, algorithms, logic and inference,
programming languages, and system building (hardware
and software).
Mathematics and physics
e.g., statistical modeling, continuous mathematics, Markov
models, statistical physics, and complex systems.
and others, e.g., cognitive science, neuroscience, economics,
psychology, linguistics, statistics…
Carla P. Gomes
INFO372
AI:
Historical Perspective
Obtaining an understanding of the human mind is one of the
final frontiers of modern science.
Founders:
George Boole (1779-1848), Gottlob Frege (1848-1925), and Alfred Tarski
(1902-1983)
formalizing the laws of human thought
Alan Turing (1912-1954) , John von Neumann (1903-1957), Claude
Shannon (1916-2001)
thinking as computation
John McCarthy (1927- ), Marvin Minsky (1927 - ) , Herbert Simon (19162001), and Allen Newell (1927-1992)
the start of the field of AI (1959)
Carla P. Gomes
INFO372
In 1936, Alan Turing, a British mathematician, showed
that there exists a relatively simple universal computing
device that can perform any computational process.
Computers use such a universal model.
Alan
Turing
Turing Machine
(abstraction)
Turing also showed the limits of computation – some problems
cannot be computed even with the most powerful computer and even
with unlimited amount of time – e.g., Halting problem.
Halting Problem
• The halting problem is a decision problem which
can be stated as follows:
– Given a description of a program and a finite input,
decide whether the program finishes running or will
run forever, given that input.
• Alan Turing proved in 1936 that a general
algorithm to solve the halting problem for all
possible program-input pairs cannot exist. We say
that the halting problem is undecidable.
Carla P. Gomes
INFO372
Acting humanly: Turing Test
Alan Turing
•
Turing (1950) "Computing machinery and intelligence":
"Can machines think?"  "Can machines behave intelligently?"
– Operational test for intelligent behavior: the Imitation Game
AI system passes
if interrogator
cannot tell which one
is the machine
•
•
•
Predicted that by 2000, a machine might have a 30% chance of fooling a lay
person for 5 minutes
Anticipated all major arguments against AI in following 50 years
Suggested major components of AI: knowledge, reasoning, natural language
understanding, learning
•
Carla P. Gomes
INFO372
Some Famous Imitation Games
• 1960s
ELIZA Joseph Weizenbaum
– Rogerian psychotherapist
• 1990s
ALICE
• Loebner prize
– win $100,000 if you pass the test
Carla P. Gomes
INFO372
ELIZA:
impersonating a Rogerian psychotherapist
1960s ELIZA Joseph Weizenbaum
http://www-ai.ijs.si/eliza/eliza.html
Eliza:
a friend you could never have before
Eliza: Hello. I am ELIZA. How can I help you?
You: Well I feel sad
Eliza: Do you often feel sad?
You: not very often, but it's becoming more common
Eliza: Please go on.
Carla P. Gomes
INFO372
Chat Bot – ALICE
(AI Foundation)
•
•
•
•
Human: hi my name is Carla
ALICE: How are you doing, Carla.
Human: what is your name?
ALICE: My name is ALICE.
Human: what is your full name?
ALICE: ALICE = Artificial Linguistic Internet Computer Entity.
Human: when were you born?
ALICE: I was connected to the net on November 23, 1995.
Human: Have you passed the Loebner Prize?
ALICE: The Loebner Prize is an annual Turing Test sponsored by New York
philanthropist Hugh Loebner. Dr. Wallace and ALICE won it in 2000. The
ALICE program was ranked "most human computer" by the panel of
judges.
Human: who is Dr. Wallace?
ALICE: He is a famous computer scientist, the author of ALICE, and a
graduate of Carnegie Mellon.
Human: Where does Dr. Wallace work?
ALICE: he works from his home in San Francisco.
Carla P. Gomes
INFO372
Acting humanly: Turing Test
Turing test  identified key research areas in AI:
•
•
•
•
Natural Language Processing – to communicate with the machine;
Knowledge Representation – to store and manipulate information;
Automated reasoning – to use the stored information to answer questions
and draw new conclusions;
Machine Learning – to adapt to new circumstances and to detect and
extrapolate patterns.
but does a machine need to act humanly
to be considered intelligent?
Carla P. Gomes
INFO372
Different Approaches
I Building exact models of human cognition
view from psychology and cognitive science
II Developing methods to match or exceed human
performance in certain domains, possibly by
very different means  e.g., Deep Blue;
Focus of INFO372 (most recent progress).
Carla P. Gomes
INFO372
Man vs. Machiens The Hardware
• The brain
– a neuron, or nerve cell, is the basic information processing unit
(10^11 )
– many more synapses (10^14) connect the neurons
– cycle time: 10^(-3) seconds (1 millisecond)
• How complex can we make computers?
– 10^8 or more transistors per CPU
– supercomputer: hundreds of CPUs, 10^10 bits of RAM
– cycle times: order of 10^(-9) seconds (1 nanosecond)
Carla P. Gomes
INFO372
Computer vs. Brain
Carla P. Gomes
INFO372
Carla P. Gomes
INFO372
• Conclusion
– In near future we can have computers with as many processing
elements as our brain, but:
far fewer interconnections (wires or synapses)
much faster updates.
Fundamentally different hardware may require fundamentally
different algorithms!
– Very much an open question.
Carla P. Gomes
INFO372
What is AI?
Human-like
Intelligence
Thought/
Reasoning
Behavior/
Actions
“Ideal” Intelligent/
Rationally
Thinking
humanly
Thinking
Rationally
Acting
Humanly
Acting
Rationally
Carla P. Gomes
INFO372
What's involved in Intelligence?
A) Ability to interact with the real world
to perceive, understand, and act
speech recognition and understanding
image understanding (computer vision)
B) Reasoning and Planning
INFO 372
modelling the external world
problem solving, planning, and decision making
ability to deal with unexpected problems, uncertainties
C) Learning and Adaptation
We are continuously learning and adapting.
We want systems that adapt to us!
Carla P. Gomes
INFO372
State-of-the-art
Reasoning and Planning in AI
A few examples…
Carla P. Gomes
INFO372
1997:
Deep Blue beats the World Chess Champion
vs.
I could feel human-level intelligence across the room
-Gary Kasparov, World Chess Champion (human…)
Carla P. Gomes
INFO372
Deep Blue vs. Kasparov
Game 1: 5/3/97:
Kasparov wins
Game 2: 5/4/97:
Deep Blue wins
Game 3: 5/6/97:
Draw
Game 4: 5/7/97:
Draw
“I felt a new kind of
Intelligence” ( across
the board from him)
Kasparov 1997
Game 5: 5/10/97:
The value of IBM’s stock
Draw
Increased by $18 Billion!
Game 6: 5/11/97:
Deep Blue wins
One of the most famous modern computers,
Deep Blue, which defeated Gary Kasparov at chess.
Carla P. Gomes
INFO372
How Intelligent is Deep Blue?
• Saying Deep Blue doesn't really think about chess
is like saying an airplane doesn't really fly because
it doesn't flap its wings.
- Drew McDermott
Carla P. Gomes
INFO372
On Game 2
(Game 2 - Deep Blue took an early lead.
Kasparov resigned, but it turned out he could
have forced a draw by perpetual check.)
This was real chess. This was a game any human
grandmaster would have been proud of.
Joel Benjamin grandmaster, member Deep Blue team
Carla P. Gomes
INFO372
Kasparov on Deep Blue
• 1996: Kasparov Beats Deep Blue
“I could feel --- I could smell --- a new kind
of intelligence across the table.”
• 1997: Deep Blue Beats Kasparov
“Deep Blue hasn't proven anything.”
Carla P. Gomes
INFO372
Game Tree Search
• How to search a game tree was independently
invented by Shannon (1950) and Turing (1951).
• Technique called: MiniMax search.
• Evaluation function combines material & position.
Carla P. Gomes
INFO372
Game Tree Search
Carla P. Gomes
INFO372
History of Search Innovations
•Shannon, Turing
•Kotok/McCarthy
•MacHack
•Chess 3.0+
•Belle
•Cray Blitz
•Hitech
•Deep Blue
Minimax search
1950
Alpha-beta pruning 1966
Transposition tables 1967
Iterative-deepening 1975
Special hardware
1978
Parallel search
1983
Parallel evaluation
1985
All of the above 1997
Carla P. Gomes
INFO372
Transposition Tables
• Introduced by Greenblat's Mac Hack (1966)
• Basic idea: caching
– once a board is evaluated, save it in a hash table (data structure that
associates keys with values), avoid re-evaluating.
– called “transposition” tables, because different orderings (transpositions)
of the same set of moves can lead to the same board.
– Form of root learning (memorization)
– Don’t repeat blunders  can’t beat the computer twice in a row using
same moves
Deep Blue --- huge transposition tables (100,000,000+),
must be carefully managed.
Carla P. Gomes
INFO372
Special-Purpose and Parallel Hardware
•
•
•
•
Belle (Thompson 1978)
Cray Blitz (1993)
Hitech (1985)
Deep Blue (1987-1996)
– Parallel evaluation: allows more complicated evaluation
functions
– Hardest part: coordinating parallel search
– Deep Blue never quite plays the same game, because of
“noise” in its hardware!
Carla P. Gomes
INFO372
Deep Blue
• Hardware
– 32 general processors
– 220 VSLI chess chips
• Overall: 200,000,000 positions per second
– 5 minutes = depth 14
• Selective extensions - search deeper at unstable
positions
– down to depth 25 !
Carla P. Gomes
INFO372
Tactics into Strategy
• As Deep Blue goes deeper and deeper into a
position, it displays elements of strategic
understanding. Somewhere out there mere tactics
translate into strategy. This is the closest thing
I've ever seen to computer intelligence. It's a very
weird form of intelligence, but you can feel it. It
feels like thinking.
– Frederick Friedel (grandmaster), Newsday, May 9, 1997
Carla P. Gomes
INFO372
1996 - EQP:
Robbin’s Algebras are all boolean
A mathematical conjecture (Robbins conjecture) unsolved for decades
The Robbins problem was to determine whether one
particular set of rules is powerful enough to capture all of
the laws of Boolean algebra. One way to state the Robbins
problem in mathematical terms is:
Can the equation not(not(P))=P be derived from the
following three equations?
[1] P or Q = Q or P,
[2] (P or Q) or R = P or (Q or R),
[3] not(not(P or Q) or not(P or not(Q))) = P.
[An Argonne lab program] has come up with a major mathematical
proof that would have been called creative if a human had thought of it.
New York Times, December, 1996
http://www-unix.mcs.anl.gov/~mccune/papers/robbins/
Carla P. Gomes
INFO372
1999: Remote Agent takes
Deep Space 1 on a galactic ride
Goals
Scripts
Scripted
Executive
ESL
Mission-level
actions &
resources
Generative
Planner &
Scheduler
Generative
Mode Identification
& Recovery
component models
Monitors
Real-time Execution
Adaptive Control
Hardware
For two days in May, 1999, an AI Program called Remote Agent
autonomously ran Deep Space 1 (some 60,000,000 miles from earth)
Carla P. Gomes
INFO372
2000: SCIFINANCE
synthesizes programs for financial modeling
• Develop pricing models
for complex derivative
structures
• Involves the solution of a
set of PDEs (partial
differential equations)
• Integration of objectoriented design, symbolic
algebra, and plan-based
scheduling
Carla P. Gomes
INFO372
Proverb 1999: Solving Crossword Puzzles as
Probabilistic Constraint Satisfaction
Proverb solves
crossword puzzles
better than most
humans
Michael Littman et a. 99
Carla P. Gomes
INFO372
Robocup @ Cornell
199
http://www.mae.cornell.edu/raff/MultiAgentSystems/MultiAgentSystems.htm
2005 Autonomous Control:
DARPA GRAND CHALLENGE
October 9, 2005
Stanley and the Stanford RacingTeam
were awarded 2 million dollars for being the
first team to complete the 132 mile
DARPA Grand Challenge course (Mojave Desert).
Stanley finished in just under 6 hours 54 minutes
and averaged over 19 miles per hours on the course.
Carla P. Gomes
INFO372
Carla P. Gomes
INFO372
DARPA - Urban Challenge (2007)
• The Urban Challenge features autonomous ground
vehicles maneuvering in a mock city environment,
executing simulated military supply missions
while merging into moving traffic, navigating
traffic circles, negotiating busy intersections, and
avoiding obstacles.
Carla P. Gomes
INFO372
Carla P. Gomes
INFO372
Many Other Applications
•
•
•
•
•
•
•
•
•
•
Financial planning
Marketing
E-business
Telecommunications
Manufacturing
Operations Management
Production Planning
Transportation Planning
System Design
Health Care
Carla P. Gomes
INFO372
Course Themes, Goals, and Syllabus
Carla P. Gomes
INFO372
Goals of INFO 372
Focus of Info 372: Problem Solving
Introduce the students to a range of computational modeling
approaches and solution strategies using examples from AI and
Information Science.
Formalisms:
Logical representations;
Constraint-based languages,
Mathematical programming;
Multi-agent formalisms (including adversarial games);
Solution strategies:
Logical inference;
General complete backtrack search;
Local search;
Dynamic Programming;
Carla P. Gomes
INFO372
Goals of INFO 372
Special models:
Satisfiability (SAT); Maximum SAT; Horn
Constraint Satisfaction; Binary Constraint Satisfaction;
Mixed Integer Programming, Linear Programming and
Network Flow Models;
Themes:
Expressiveness and efficiency tradeoffs of the various representation
formalisms
Students learn about the tradeoffs in modeling choices.;
Concrete examples to move from one representation modeling
formalism to another formalism;
Carla P. Gomes
INFO372