Strategic Planning for Unreal
Tournament© Bots
Héctor Muñoz-Avila
Todd Fisher
Department of Computer Science and Engineering
Lehigh University
USA
Overview
• Motivation
• Unreal Tournament© (UT)
• HTN Planning
 Why HTN Planning
• Related Work
• HTN Encodings of UT Bot Strategies
• Built-in predicates and Java Bots
• Final Remarks
Motivation # 1: Test Environment for
Reactive Planning
Some research issues on reactive environments…
• Plan generation versus plan execution
• Case-Based Planning
 Fast retrieval of adequate plans
 Real-time plan adaptation
• Learning
 Learning retrieval criteria
 Learning domain knowledge
 …
Lack of
adequate testing
environment
Motivation # 2: Declarative Encoding of
Gaming Strategies
• Many games provide scripting languages (e.g., Lua
scripting language)
•Games can be useful to teach programming concepts
(e.g., IBM’s robocode)
•Is it possible to create a declarative language for
encoding gaming strategies?
 RBSs are an alternative (but can be messy)
 How about for controlling teams of Bots?
Avatar controlled by a computer program
Unreal Tournament©
• Online FPS developed by Epic Games Inc.
•Various gameplay modes including team deathmatch
and capture the flag
•Provides a client-server
architecture for controlling
bots
•High-level script language:
UnrealScript (Sweeney, 2004)
Hierarchical Task Network (HTN) Planning
Principle: complex tasks are decomposed into simpler tasks. The
goal is to decompose compound tasks into primitive tasks, which
define actions changing the world.
Travel(Lehigh,NRL)
Travel from Lehigh to NRL
alternative
methods Travel by car
Travel by plane
Enough money for air
fare available
Enough money
for gasoline
Seats available
Roads are passable
Travel(Lehigh,Newark Airport)
Drive(Lehigh, Newark Park)
Bus(Newark Park,airport)
Fly(Newark, BWI)
Travel(BWI, NRL)
Taxi(BWI, NRL)
Why HTN Planning?
• HTN planning is provable more expressive than STRIPS
planning (Erol et al, 1994)
• Hierarchical planning is natural in many domains
e.g., military planning
Strategic
National
Strategic
Theater
Operational
• Can be very fast!
Tactical
JCS / NCA
CINC
JTF
Related Work
• Hierarchical planning can be useful in games (Wallace,
2003)
• Bridge Baron game uses HTN ideas (Smith et al., 1998).
• Games have the potential to become testbed for different AI
algorithms (Laird & van Lent, 1999).
• Soar Bots for UT using the Soar Architecture (Laird &
Duchim, 2000)
• Tielt: testbed for investigating and evaluating learning
techniques (Aha & Molineaux, 2004)
Client-Server Architecture for UT Bots
• The UT server provides sensory information about events in
the UT world and controls all the gameplay events.
• Event-driven: The client program uses this information to
decide commands controlling the behavior of a Bot and
passes them to the server.
• Two types of messages:
 Asynchronous: special events
 Synchronous: regular intervals
• Variants: Enhanced server (UC/ISI), Java Bots (CMU), Soar
Bots (U. of Michigan).
HTN for Controlling Team of Bots
Idea:
• Use methods to encode gaming strategies
• Use operators to encode gaming actions
UT task: Domination
C
B
A
Strategy:
secure most
locations
UT action: move Bot1 to
location B
Technical Difficulties and Solutions
UT task: Domination
Strategy:
secure most
locations
UT action: move Bot1 to
location B
2. Dealing with
contingencies while
Bots are executing
actions
1. Declarative language
for expressing
conditions
Solution to 1: use builtin predicates to
represent conditions
Solution to 2: use Java
Bots to represent
actions (event-driven)
Example
P
C
RP
B
HTN Method
Head: Domination
Preconditions:
LocsGeographTogether(P,RP)
Divide3Groups(T1,T2,T3)
Subtasks:
CoverLocations(T1,P)
PatrolLocations (T2,P)
HarrassLocations(T3,RP)
A
The HTNs are represented in XML
Strategy Change vs. Strategy Modification
•We are exploring 2 alternatives:
Select alternative strategy when similarity of current
strategy falls below a certain threshold
Modify strategy using plan adaptation techniques
On-going/Short-Term Future Directions
• Selection of alternative strategy modification versus strategy
modification
• Support the Tielt testbed:
 Use HTNs and process models (TMKL) to encode game
model
 Learn conditions for retrieving strategies
Final Remarks
• Our approach can be used to accomplish two seemingly
contradictory goals:
 The Bot needs to react quickly in a highly dynamic
environment.
 The Bot must contribute to the grand strategy to win the
game.
 A grand strategy is laid out in the HTNs and event-driven
programming allows the Bots to react in highly dynamic
environments while contributing to the grand task.