Lecture 10 of 41
Game Tree Search (Concluded)
Introduction to Logical Agents
Monday, 13 September 2004
William H. Hsu
Department of Computing and Information Sciences, KSU
http://www.kddresearch.org
http://www.cis.ksu.edu/~bhsu
Reading:
Sections 6.4-6.8, Russell and Norvig 2e
Next class (Wednesday): Sections 7.1-7.4
CIS 730: Introduction to Artificial Intelligence
Kansas State University
Department of Computing and Information Sciences
Lecture Outline
•
Today’s Reading
– Sections 6.4-6.8, Russell and Norvig 2e
– Recommended references: Rich and Knight, Winston
•
Reading for Next Class: Chapter 6, Russell and Norvig
•
More on Game Tree Search
– Review: framework; minimax algorithm; quiescence detection, horizon effect
– Alpha-Beta pruning
– Expectiminimax
•
Applications: Current “Hot” Problems
– “Harder” games: Go
– Interactive agents and games
• Economically-founded multiagent systems, negotiation
• Simulated societies
•
Next Week: Knowledge Representation – Logics and Production Systems
CIS 730: Introduction to Artificial Intelligence
Kansas State University
Department of Computing and Information Sciences
Review:
Minimax Algorithm
Adapted from slides by S. Russell, UC Berkeley
CIS 730: Introduction to Artificial Intelligence
Figure 5.2 p. 125 R&N
Kansas State University
Department of Computing and Information Sciences
Alpha-Beta (-) Pruning:
Definitions
Adapted from slides by S. Russell, UC Berkeley
CIS 730: Introduction to Artificial Intelligence
Kansas State University
Department of Computing and Information Sciences
Alpha-Beta (-) Pruning:
Example [1]
≥3
MAX
MIN
Warning: Gratuitous use of
Microsoft® PowerPoint ® XP 
≤2
3
≤ 14 ≤5
2
MAX
3
12
8
2
Adapted from slides by S. Russell, UC Berkeley
CIS 730: Introduction to Artificial Intelligence
14
5
2
Figure 5.6 p. 131 R&N
Kansas State University
Department of Computing and Information Sciences
Alpha-Beta (-) Pruning:
Example [2]
What are ,  values here?
≥3
MAX
MIN
≤2
3
≤ 14 ≤5
2
MAX
3
12
8
2
Adapted from slides by S. Russell, UC Berkeley
CIS 730: Introduction to Artificial Intelligence
14
5
2
Figure 5.6 p. 131 R&N
Kansas State University
Department of Computing and Information Sciences
Alpha-Beta (-) Pruning:
Modified Minimax Algorithm
Adapted from slides by S. Russell, UC Berkeley
CIS 730: Introduction to Artificial Intelligence
Kansas State University
Department of Computing and Information Sciences
Expectiminimax
Adapted from slides by S. Russell, UC Berkeley
CIS 730: Introduction to Artificial Intelligence
Kansas State University
Department of Computing and Information Sciences
Alpha-Beta (-) Pruning:
Properties
•
Perfect Play (TBD from Perfect Information)
– Common framework: two-player (or multi-player), zero-sum
– Deterministic or known probability distribution function, totally observable
– Minimax algorithm given ability to search to arbitrary ply
•
Assumptions about
– … game?
– … opponent?
•
Specifications and Ramifications
– Combinatorics: finite depth (m), breadth (b) of search
– Resource limits: need for static evaluation function
•
Practical Issues
– Effects of depth cutoff
– Need for pruning
Adapted from slides by S. Russell, UC Berkeley
CIS 730: Introduction to Artificial Intelligence
Kansas State University
Department of Computing and Information Sciences
Digression:
Learning Evaluation Functions [1]
•
Learning = Improving with Experience at Some Task
– Improve over task T,
– with respect to performance measure P,
– based on experience E.
•
Example: Learning to Play Checkers
– T: play games of checkers
– P: percent of games won in world tournament
– E: opportunity to play against self
•
Refining the Problem Specification: Issues
– What experience?
– What exactly should be learned?
– How shall it be represented?
– What specific algorithm to learn it?
•
Defining the Problem Milieu
– Performance element: How shall the results of learning be applied?
– How shall the performance element be evaluated? The learning system?
CIS 730: Introduction to Artificial Intelligence
Kansas State University
Department of Computing and Information Sciences
Digression:
Learning Evaluation Functions [2]
•
Possible Definition
– If b is a final board state that is won, then V(b) = 100
– If b is a final board state that is lost, then V(b) = -100
– If b is a final board state that is drawn, then V(b) = 0
– If b is not a final board state in the game, then V(b) = V(b’) where b’ is the best
final board state that can be achieved starting from b and playing optimally
until the end of the game
– Correct values for minimax / alpha-beta, but not operational
•
Choosing a Representation for the Static Evaluation Function
– Collection of rules?
– Neural network?
– Polynomial function (e.g., linear, quadratic combination) of board features?
– Other?
•
A Representation for Learned Function
– Vˆ b   w 0  w 1bp b   w 2 rp b   w 3 bk b   w 4 rk b   w 5 bt b   w 6 rt b 
– bp/rp = number of black/red pieces; bk/rk = number of black/red kings;
bt/rt = number of black/red pieces threatened (can be taken on next turn)
CIS 730: Introduction to Artificial Intelligence
Kansas State University
Department of Computing and Information Sciences
State of The Art:
Learning to Play Games
•
“Easy” Games: Nim, Tic-Tac-Toe
– Nim: known Grundy function
– TTT: ditto (easy to play, prove; hard to store game tree)
•
“Cracked” Games: Go-Moku, Checkers, Go Subgames
– Go-Moku aka Connect Five
– Some limited Go (smaller than 19x19) or specialized games (Guess – see SciAm)
•
Games where Best Human Players Have Been Defeated by AI
– Reversi / Othello (1988, Lee et al)
– Backgammon (1992, Tesauro)
– Checkers (1994)
– Chess (1997, Campbell et al)
•
“Harder” Games
– Go (see Computer Go, Ing Prize pages)
– Others
CIS 730: Introduction to Artificial Intelligence
Kansas State University
Department of Computing and Information Sciences
Current Research Topics in Games and AI
•
Game Theory
– Common ramework: two-player (or multi-player), zero-sum
– Deterministic or known probability distribution function, totally observable
– Minimax algorithm given ability to search to arbitrary ply
•
Negotiation
– … game?
– … opponent?
•
Specifications and Ramifications
– Combinatorics: finite depth (m), breadth (b) of search
– Resource limits: need for static evaluation function
•
Practical Issues
– Effects of depth cutoff
– Need for pruning
CIS 730: Introduction to Artificial Intelligence
Kansas State University
Department of Computing and Information Sciences
Resources for Term Project
•
Print Resources on The Web (You May Cite Some as Print References)
– CiteSeer (largest online citation database): http://citeseer.nj.nec.com
– See also: Google (author search), E-journals (JAIR, JMLR), societies (AAAI, AUAI)
•
Resources for AI Projects
– USENET: many AI-related groups
• comp.ai, comp.ai.edu, comp.ai.{neural-nets | genetic | fuzzy}
• Other groups on vision, optical character recognition (OCR)
– Watch CIS730 Yahoo! Group (see Part 3 of 3, project guidelines)
•
Don’t Forget Interlibrary Loan!
– K-State: off-campus borrowing facilities (http://www.lib.ksu.edu)
– Arrangements for remote (NTU) students: contact instructor
•
Instructor Contact Info
– ICQ #28651394, cis730ta, cis730-dist-L@www.kddresearch.org
– Voice: (785) 532-6350 x29 [Office], (785) 539-7180 [Home 18:00 – 20:00 CDT]
– Mobile phone contact: call Delores Winfough (785) 532-6350
– Schedule: http://www.kddresearch.org/Personal/bhsu/schedule.txt
CIS 730: Introduction to Artificial Intelligence
Kansas State University
Department of Computing and Information Sciences
Ideas for Term Project
•
Rogue versus Angband
– Idea: discrete domain, single-agent, control and target acquisition problem
– Rog-O-Matic: http://www.cs.princeton.edu/~appel/papers/rogomatic.html
– Angband Borg: http://www.phial.com/angborg/
•
Empire versus Intergalactics
– Idea: continuous domain, multi-agent, competitive, control and target acquisition
– Empire: http://gamespot.com/gamespot/features/pc/history_spaceempire/
– Intergalactics: http://www.intergalactics.net
•
Robocode versus Space Pirates
– Idea: continuous domain, multi-agent, competitive, control and target acquisition
– Robocode: http://www.alphaworks.ibm.com/tech/robocode
– Space Pirates: http://www.lostlogic.com
•
Hey! Are Computer Games All This Course Is About?
– No!
– Diagnostic expert systems, data mining, intelligent IR
– Many many more… (see web group)
CIS 730: Introduction to Artificial Intelligence
Kansas State University
Department of Computing and Information Sciences
Advice About Term Project [1]
•
Start Early!
– Recall “How to Get an A in This Course” (Lecture 0)
– See below: start ball rolling ASAP
•
Ask Questions on Course Web Group (ksu-cis730-fall2001)
– On online fora
• Course web group (ksu-cis730-fall2001) – check progress with classmates!
• USENET (caveat: know what you are asking, what you will do with answers)
• Mailing lists (ditto) – data mining; Uncertainty in AI; many, many others
– Directly question instructor
• By remote: see contacts, previous page
• in person: before class, Wednesday office hours (with project partner, by self)
– Do footwork
• Not everything is on web (Inter-Library Loan – excruciatingly slow)
• Not everything on web is well-indexed, easily accessible
•
Stay Coordinated with Project Partner
•
Don’t Waste Too Much Time on Nonessentials (Display/GUI)
CIS 730: Introduction to Artificial Intelligence
Kansas State University
Department of Computing and Information Sciences
Advice About Term Project [2]
•
Evaluation Criteria
– Originality, significance, scope
• Nontriviality of problem: very important!
• Originality of solution: reimplementation / rehashing of common “applet”?
• Generality of results
– Functionality
– Writing quality
• Proposal is not end of writing!
• Conference/workshop-style report much more important
•
How to Focus on Essentials
– Keep list of milestones, track schedule slippage carefully
– Report to instructor and get feedback (no formal interim reports, but post)
– Know what is essential
• Ounce of prevention: good planning, design, specification, antibugging
• Don’t lose time on interfaces, et cetera: scoring not based on “KLOC”
• Leave time for debugging and testing!
CIS 730: Introduction to Artificial Intelligence
Kansas State University
Department of Computing and Information Sciences
Knowledge Bases
Adapted from slides by S. Russell, UC Berkeley
CIS 730: Introduction to Artificial Intelligence
Kansas State University
Department of Computing and Information Sciences
Simple Knowledge-Based Agent
Adapted from slides by S. Russell, UC Berkeley
CIS 730: Introduction to Artificial Intelligence
Figure 6.1 p. 152 R&N
Kansas State University
Department of Computing and Information Sciences
Representing Intelligent Agent (IA) Problems:
PAGE Formulation
•
Percepts
– What can agent observe?
– What can sensors tell it?
•
Actions
– What actuators does agent have?
– In what context are they applicable?
•
Goals
– What are agents goals? Preferences (utilities)?
– How does agent evaluate them (check environment, deliberate, etc.)?
•
Environment
– What are “rules of the world”?
– How can these be represented, simulated?
CIS 730: Introduction to Artificial Intelligence
Kansas State University
Department of Computing and Information Sciences
Wumpus World PAGE Description
Adapted from slides by S. Russell, UC Berkeley
CIS 730: Introduction to Artificial Intelligence
Figure 6.2 p. 154 R&N
Kansas State University
Department of Computing and Information Sciences
Wumpus World Characterization
•
Deterministic?
– In general: is there any uncertainty about
• Outcomes of actions?
• Initial conditions? (Can still be deterministic domain; uncertainty is agent’s)
– Yes, deterministic… outcomes exactly specified
•
Fully Accessible?
– Global perception?
– No, not fully accessible… can only observe locally
•
Static?
– Obstacles, other “Non-Player Character (NPC)” agents can move?
– Yes, static… wumpus, pits cannot move
•
Discrete?
– NB: not representation; actual domain (why important?)
– Yes, discrete
Adapted from slides by S. Russell, UC Berkeley
CIS 730: Introduction to Artificial Intelligence
Kansas State University
Department of Computing and Information Sciences
Exploring A Wumpus World
OK

OK
OK
KSU
Adapted from slides by S. Russell, UC Berkeley
CIS 730: Introduction to Artificial Intelligence
Figures 6.3-6.4 p. 156 R&N
Kansas State University
Department of Computing and Information Sciences
Wumpus World Reasoning Problems:
Uncertainty and Coercion
Adapted from slides by S. Russell, UC Berkeley
CIS 730: Introduction to Artificial Intelligence
Kansas State University
Department of Computing and Information Sciences
Logic in General
Adapted from slides by S. Russell, UC Berkeley
CIS 730: Introduction to Artificial Intelligence
Kansas State University
Department of Computing and Information Sciences
Ontology and Epistemology:
Describing Types of Logic
Adapted from slides by S. Russell, UC Berkeley
CIS 730: Introduction to Artificial Intelligence
Figure 6.7 p. 166 R&N
Kansas State University
Department of Computing and Information Sciences
Summary Points
•
Games as Search Problems
– Frameworks
– Concepts: utility, reinforcements, game trees
– Static evaluation under resource limitations
•
Family of Algorithms for Game Trees: Minimax
– Static evaluation algorithm
• To abritrary ply
• To fixed ply
• Sophistications: iterative deepening, pruning
– Credit propagation
• Intuitive concept
• Basis for simple (delta-rule) learning algorithms
•
State of The Field
•
Uncertainty in Games: Expectiminimax and Other Algorithms
CIS 730: Introduction to Artificial Intelligence
Kansas State University
Department of Computing and Information Sciences
Terminology
•
Game Graph Search
– Frameworks: zero-sum, multi-player
– Concepts
• Utility and representations (e.g., static evaluation function)
• Reinforcements: possible role for machine learning
• Game tree: node/move correspondence, search ply
• Important distinction: perfect play versus perfect information
•
Family of Algorithms for Game Trees: Minimax
– Propagation of credit
– Imperfect decisions
– Issues
• Quiescence
• Horizon effect
•
Alpha-Beta Pruning
•
Expectiminimax: Extension to Uncertain Games
CIS 730: Introduction to Artificial Intelligence
Kansas State University
Department of Computing and Information Sciences