Finite State Machine for Games
Fall 2012
Ref: Chenney, CS679 lectures
AI Game Programming Wisdom 2
Outline
AI and Game
Introduction/examples
Design


Intuition
State-based
Implementation
Extending


Fall 2012
Stack-based
Fuzzy-state machine
2
What is AI? (and is NOT)
AI is the control of non-human entities in a game



The other cars in a car game
The opponents and monsters in a shooter
Your units, your enemy’s units and your enemy in a RTS
game
But, typically does not refer to passive things that
just react to the player and never initiate action


Fall 2012
That’s physics or game logic
For example, the blocks in Tetris are not AI, nor is a flag
blowing in the wind
3
AI in the Game Loop
AI is updated as part of the game loop,
after user input, and before rendering
Fall 2012
4
AI Update Step
The sensing phase determines
the state of the world


May be very simple - state
changes all come by messaging
Or complex - figure out what is
visible, where your team is, etc
The thinking phase decides
what to do given the world

Sensing
Game
Engine
Thinking
The core of AI
The acting phase tells the
animation what to do
Fall 2012
AI Module
Acting
5
AI gets called at a fixed rate
AI by Polling
Senses:

looks to see what has been changed in the world
then acts on it
Generally inefficient and complicated

Fall 2012
Different characters might require different polling
rate
6
Event Driven AI
does everything in response
to events in the world
• Events triggered by a message
•
basically, a function gets called when a
message arrives, just like a user interface)
• Example messages:



Fall 2012
You have heard a sound
Someone has entered your field of view
A certain amount of time has passed, so
update yourself
7
AI Techniques in Games
Basic problem: Given the state of the world,
what should I do?
A wide range of solutions in games:

Finite state machines, Decision trees, Rule
based systems, Neural networks, Fuzzy logic
A wider range of solutions in the academic
world:


Fall 2012
Complex planning systems, logic programming,
genetic algorithms, Bayes-nets
Typically, too slow for games
8
Expressiveness
What behaviors can easily be defined, or defined at
all?
Propositional logic:



Statements about specific objects in the world – no variables
Jim is in room7, Jim has the rocket launcher, the rocket
launcher does splash damage
Go to room8 if you are in room7 through door14
Predicate Logic:





Fall 2012
Allows general statement – using variables
All rooms have doors
All splash damage weapons can be used around corners
All rocket launchers do splash damage
Go to a room connected to the current room
9
Finite State Machines (FSMs)
A set of states that the agent can be in
Connected by transitions that are triggered
by a change in the world
Normally represented as a directed graph,
with the edges labeled with the transition
event
Ubiquitous in computer game AI
Fall 2012
10
Formal Definitions (N. Philips)
"An abstract machine consisting of a set of states
(including the initial state), a set of input events, a
set of output events, and a state transition function.
The function takes the current state and an input
event and returns the new set of output events and
the next state. Some states may be designated as
"terminal states".
The state machine can also be viewed as a function
which maps an ordered sequence of input events into
a corresponding sequence of (sets of) output events.
Finite State Automaton: the machine with no output
Fall 2012
11
FSM with Output: vending
machines
OJ & AJ for 30 cents
State table
Fall 2012
12
Vending Machine: state diagram
Fall 2012
13
FSM and Game
Game character behavior can be
modeled (in most cases) as a sequence
of different “mental state”, where
change is driven by the actions of
player/other characters, …
Natural choice for defining AI in games
Fall 2012
14
FSM Examples
Fall 2012
15
Fall 2012
16
Fall 2012
17
Fall 2012
18
PacMan State Machine
Timer <= 0
PacMan eats a
Power Pill
Timer <= 0
Collision with
GhostBox
Fall 2012
Collision with
PacMan
19
Actions of States
Roam;
If PacMan gets close,
PathTo (PacMan)
Timer—
Move back-and-forth
Fall 2012
Timer--;
PathAwayFrom
(PacMan)
PathTo (GhostBox)
20
Ex: predator vs. prey
Prey (laalaa)
Sees predator
Idle
(stand,wave,…)
No more threat
Dead
Fall 2012
Flee
(run)
captured
21
Predator (Raptor)
Idle
(stand)
Tdining>5
Tidle > 5
Hungry
(wander)
Dining
Prey captured
Pursuit
(run)
Prey in sight
Tpursuit > 10
Fall 2012
22
Idling LaaLaa
This page illustrates:
hierarchical state,
Non-deterministic state transition
Target arrived
Wander
(set random target)
Stand
Tstand>4
20%
R 30%
50%
Wave
Twave>2
Fall 2012
23
FSM Design
Quake2 Examples
Intuitive thinking:
model the events and
state changes
Quake2 uses 9 different states:
standing, walking, running, dodging, attacking, melee,
seeing the enemy, idle and searching.
Incomplete
design
Fall 2012
25
Quake: Rocket
Fall 2012
26
Shambler monster
Fall 2012
27
FSM Advantages
Very fast – one array access
Expressive enough for simple behaviors or characters
that are intended to be “dumb”
Can be compiled into compact data structure


Dynamic memory: current state
Static memory: state diagram – array implementation
Can create tools so non-programmer can build
behavior
Non-deterministic FSM can make behavior
unpredictable
Fall 2012
28
FSM Disadvantages
Number of states can grow very fast

Exponentially with number of events: s=2e
Number of arcs can grow even faster: a=s2
Propositional representation


Difficult to put in “pick up the better powerup”, “attack the
closest enemy”
Expensive to count: Wait until the third time I see enemy,
then attack
 Need extra events: First time seen, second time seen, and
extra states to take care of counting
Fall 2012
29
FSM
Control System Implementation
FSM Implementation
Fall 2012
31
Previous Example
Fall 2012
32
Code 1
Fall 2012
Ad hoc implementation
33
Code 1p
Fall 2012
34
Code 2
Fall 2012
Structure, Readable, maintainable
35
Hierarchical …
Fall 2012
36
FSM Extensions
Advanced Issues
Hierarchical FSM
Non-deterministic FSM
Swapping FSM
Fall 2012
38
Hierarchical FSMs
What if there is no simple action for a state?
Expand a state into its own FSM, which explains what
to do if in that state
Some events move you around the same level in the
hierarchy, some move you up a level
When entering a state, have to choose a state for its
child in the hierarchy



Fall 2012
Set a default, and always go to that
Or, random choice
Depends on the nature of the behavior
39
Stack-based FSM
Pushdown automaton
(PDA)
History stack: Remember
previous state; create
characters with a
memory …
Fall 2012
40
Goal-based vs. State-based
There is also a slight derivative to the statebased engine, but it used in more
complicated games like flight simulators and
games like MechWarrior. They use goal based engines - each entity within the game
is assigned a certain goal, be it 'protect base',
'attack bridge', 'fly in circles'. As the game
world changes, so do the goals of the various
entities.
Fall 2012
41
Processing Models
Polling



FSM update
frequency
Easy to implement
and debug
Inefficiency (Little
Red example)
Event-driven




Fall 2012
Publish-subscribe
messaging system
Game engine sends
event messages to
individual FSMs
An FSM subscribes only
to the events that have
the potential to change
the current state
Higher efficiency, nontrivial implementation
42
Efficiency and Optimization
In AI, FSM is the most efficient
technology available
Yet, there is always room for
improvement

Fall 2012
Level of Detail: depending on the condition
(e.g., distance with player), use different
FSM, or different update frequency
43
References
Web references:





www.gamasutra.com/features/19970601/build_brains_into_
games.htm
csr.uvic.ca/~mmania/machines/intro.htm
www.erlang/se/documentation/doc4.7.3/doc/design_principles/fsm.html
www.microconsultants.com/tips/fsm/fsmartcl.htm
http://www.angelfire.com/dragon/letstry/tutorials/dfa/
Game Programming Gems Sections 3.0 & 3.1

Fall 2012
It’s very very detailed, but also some cute programming
44
Well, the sad news …
Behavior trees
Fall 2012
45
Fall 2012
46
Additional Materials
Fall 2012
47
Intuitive Design
Say, a simple teletube baby has three
states: idle, run, and wave
Scenario:


Fall 2012
When an idle laalaa sees a butterfly, it
waves to it. When the butterfly flies away,
it returns to idle
When an idle laalaa sees a mouse, it flees
away. When the mouse is no longer in
sight, it returns to idle
48
Laalaa
flee
mouse
~mouse
How to make sure the design
complete? I.e., all states and
transitions are considered
idle
butterfly
~butterfly
Fall 2012
wave
Is there any systematic way to
develop an FSM?
49
Quake Bot Example (first-cut)
Types of behavior to capture:




Wander randomly if don’t see or hear an enemy
When see enemy, attack
When not see enemy and hear an enemy, chase
enemy
When die, re-spawn (new a bot from start)
Events: see enemy, hear enemy, die
States: wander, attack, chase, spawn
Fall 2012
50
Remark
With 3 events, potentially there should be 23
states:

(E,S,D)=(0,0,0),(1,0,0),(0,1,0), …,(1,1,1)
Some doesn’t make sense

E.g., ESD = 111
Name and create a state for the ones that we
want to consider




Fall 2012
Wander (ESD=000)
Chase (ESD=010)
Attack (ESD=1x0), x for dont-care
Die (ESD=xx1)
51
FSM (first-cut)
Attack
1x0
~E
start
Problem: Can’t go directly from
attack to chase. Why not?
E
Events:
D
Wander
000

S
Chase
010
~S

D
D
Spawn
xx1

States:
E SD



Fall 2012
E: see an enemy
S: hear a sound
D: die
E: enemy in sight
S: sound audible
D: dead
52
FSM (first-cut)
Attack+S
110
~S
Attack
100
~E
start
E
S
D
D
Wander
000
~E
Chase
010
~S
Spawn
xx1
E
Events:

S


D
D
Extra state
needs to be
defined
States:
E SD



Fall 2012
E: see an enemy
S: hear a sound
D: die
E: enemy in sight
S: sound audible
D: dead
53
Quake Bot Example (refined)
Types of behavior to capture:




Wander randomly if don’t see or hear an enemy
When see enemy, attack
When not see enemy and hear an enemy, chase
enemy
When die, respawn
Extensions:


Fall 2012
When health is low and see an enemy, retreat
When see power-ups during wandering, collect
them [hierarchical FSM]
54
Example FSM with Retreat
Attack-ES
E,-D,S,-L
S
Attack-E
E,-D,-S,-L
-S
L
• States:
Retreat-S
-E,-D,S,L
L
-L
E
-E
E E
Wander-L
-E,-D,-S,L
L
-L
-L
E
L
Retreat-ES
E,-D,S,L
-S
-L
S
-E
Wander
-E E
-E,-D,-S,-L
Retreat-E
E,-D,-S,L
– E: enemy in
sight
– S: sound
audible
– D: dead
– L: Low health
A lot more states
got added
D D
D
Fall 2012
D
Spawn
D
(-E,-S,-L)
Chase
-E,-D,S,-L
S
55
Hierarchical FSM Example
Wander
Attack
~E
E
~S
Pick-up
Powerup
Chase
S
Start
Turn Right
D
Spawn
~E
Go-through
Door
Note: This is not a complete FSM

Fall 2012

All links between top level states still
exist
Need more states for wander 56
Non-Deterministic Hierarchical
FSM (Markov Model)
Attack
Approach
Aim &
Slide Right
& Shoot
.3
Aim &
Slide Left
& Shoot
Fall 2012
.3
.4
.3
.3
Start
.4
Aim &
Jump &
Shoot
Adds variety to actions
Have multiple transitions
for the same event
Label each with a
probability that it will be
taken
Randomly choose a
transition at run-time
Markov Model: New
state only depends on
the previous state
57