Automated Planning
The Classical State Transition Model
Jonas Kvarnström
Automated Planning Group
Department of Computer and Information Science
Linköping University
jonas.kvarnstrom@liu.se – 2015
Recall the three levels:
Practical representation language  PDDL
Many convenient high-level constructs for practical modeling
Possibly a large, complex language
Started
here
Formal representation language  Classical Rep. (book)
"The simplest representation that is (more or less) reasonable to use for modeling“
Not too far from how we think about planning,
still reasonable to use for simple domains and examples,
but fewer concepts to keep track of
Underlying formal model
Concepts should be as few, as simple and as clearly defined as possible
Good for analysis, correctness proofs, understanding what planning is
Could be cumbersome for modeling
What
about this?

A classical planning problem indirectly specifies:
 (1) A finite set of states
▪ Any assignment of values to the ground atoms
is one state
▪ Each state has structure
(represented by atoms,
illustrated here as pegs/…)
Towers of Hanoi
3 disks
27 discrete states
s17 top(1) is true
top(2) is false
top(3) is false
on(1,1) is false
on(1,2) is true
…
3
jonkv@ida
The State Space (1)

A classical planning problem indirectly specifies:
 (2) A finite set of state transitions
▪ If action a is applicable in s
and leads to s2:
A transition ss2
labeled with action a
▪ Actions have structure
(preconds, effects, …)
This is the
problem’s
state space!
4
jonkv@ida
The State Space (2)
Where an action takes you
depends on
where you started!
All orange arrows could be
the PDDL action “(move
diskA Peg1 Peg3)”
5
…and in some states
an action is not applicable
at all
jonkv@ida
The State Space (3)
Towers of Hanoi
3 disks
27 states
Known initial state
Single goal state
6
jonkv@ida
The State Space (4)
A larger
(but still tiny)
example…
Most state spaces
are far less regular,
can have dead ends, …
Given a start state s0,
some states can be
unreachable
(can even have disjoint
components)
7
Towers of Hanoi
7 disks
2187 discrete states
jonkv@ida
The State Space (5)

8
Example: Unable to return
crack(egg5)
Can never return to the leftmost part of the state space
jonkv@ida
The State Space (6)

Small part of the state space for a DWR problem
9
jonkv@ida
The State Space (7)
Instead of this…
11
Suppose we only model this!
s0 (initial)
s2
s1
s3
s5
All orange arrows are
move(diskA, Peg1, Peg3)
with precond formulas, effects
s4
s6
s7
s8
All orange arrows are
A4722
Same problem – ignore any structure "inside" states/actions!
jonkv@ida
STS 1: Introduction

12
Formally: a restricted state transition system Σ = (S,A,γ)
Finite set of world states
Finite set of actions
State transition function, where |γ(s,a)| ≤ 1
 S = { s0, s1, … }:
 A = { a0, a1, … }:
 γ: S × A  2S:
▪ If γ(s,a) = {s’},
then whenever you are in state s,
you can execute action a
and you end up in state s’
▪ If γ(s,a) = ∅ (the empty set),
then a cannot be executed in s
S = { s0, s1, … }
A = { take1, put1, … }
γ: S × A  2S
γ(s0, take2) = { s1 }
γ(s1, take2) = ∅
This also describes the entire state space!
s0
take2
put2
s1
put3
take3
s2
jonkv@ida
STS 2: Restricted State Transition System

13
We can now formally define the classical planning problem
 Let Σ = (S,A,γ) be a state transition system
 Let s0 ∈ S
 Let Sg ⊆ S
satisfying the assumptions A0 to A7
(called a restricted state transition system in the book)
be the initial state
be the set of goal states
 Then, find a sequence of transitions
labeled with actions [a1, a2, …, an]
that can be applied starting at s0
resulting in a sequence of states [s1, s2, …, sn]
such that sn ∈ Sg
start
goal
goal
goal
jonkv@ida
STS 3: Classical Planning Problem

Assumption A5: Sequential plans
 No concurrency
 No if-then conditions
 …
Bildbehandling,
Plan
reglerteknik, ...
1. Move disk 1 from B to A
2. Move disk 2 from B to C
3. Move disk 1 from A to C
4. …
A simple sequence!
14
jonkv@ida
STS 4: Correspondence to Plan Structure

Some disagreements about terminology…
 In the book: Any sequence of actions σ = ⟨a1, a2, …, an⟩ is a plan
▪ Does not have to be executable
 If it is executable, it is called… an executable plan!
▪ Let γ (s,a) be the state resulting from executing action a in state s
▪ There exist states s0, s1 , …, sn such that
γ (s0,a1) = { s1 }
γ (s1,a2) = { s2 }
…
γ (sn–1,an) = { sn }
▪ Some others only consider executable plans to be plans
 A plan is a solution if it is executable
and ends in a state sn satisfying the goal
In the exam, we will make clear which variation we mean!
15
jonkv@ida
STS 5: Plans and Solutions
16
The STS information is sufficient
to determine if a plan solves the problem
Objects, predicates, preconditions, effects, …
can be seen as distractions to be avoided
when doing a formal analysis
…but very useful when actually solving the problem!
jonkv@ida
STS 6: Sufficient
Real World
Abstraction
Approximation
Simplification
17
Real World
+ current
problem
Abstraction
Approximation
Simplification
Formal Model:
Restricted State Transition System
Σ = (S,A,γ)
Planning Problem
P = (Σ, s0, Sg)
Tells us: How the world works
(Only those aspects
that we need in our model
in order to solve
interesting problems!)
Tells us:
Which specific problem to solve
jonkv@ida
STS 7: Transition System vs. Problem
s0
take2
s1
put3
put2

Σ = (S,A,E,γ)
 E = { move13, … }
 Requires different
problem definitions,
different algorithms!
s2
take3
put1
Multiple outcomes:
put1 may drop the disk!
s3
 γ: S × (A ∪ E)  2S
▪ γ(s0, put2) = { s1 }
▪ γ(s0, take2) = ∅
▪ γ(s1, put1) = { s3, s4 }
 More expressive
18
s4
take2
s5
Possible exogenous event:
Someone else may move a disk!
move13
s6
jonkv@ida
STS 8: Extended Alternative
20
The state transition model is ”as simple as possible”
Few concepts involved: unstructured states & actions + transition function
Every classical planning problem can be transformed into this
No additional concepts are necessary!
STS: Simple concepts
STS: Lack of structure
 Gives some insights into
what classical planning really is
 More work required to specify
a real planning problem
 We can analyze algorithms
relative to the model,
prove them correct,
and be certain of our conclusions
 Less information available
for a planner to work with!
Both are useful!
jonkv@ida
Why both Language and Model?
(:action MOVE
:parameters (?disk ?source ?dest)
:precondition
(and (clear ?disk)
(on ?disk ?source)
(clear ?dest)
(smaller ?disk ?dest))
:effect
(and (on ?disk ?dest)
(not (on ?disk ?source))
(not (clear ?dest))
(clear ?source))))
21
Without internal structure:
γ(s0, A1) = s2,
γ(s6, A1) = s5,
γ(s7, A1) = s4,
…
s0
s2
s1
s3
s5
s4
s6
s7
s8
jonkv@ida
Why both Language and Model? (2)
Instead of this…
22
The STS really contains this:
s0
s2
s1
s3
s5
No information
about what
differs between
s0 and s1
s4
s6
s7
s8
jonkv@ida
Why both Language and Model? (3)
23
jonkv@ida
Transformation: First-order classical  STS
Input 1: Planning domain
Defines
the set of
states
in the
formal
model
(STS)
Object Types:
There are UAVs, boxes …
Predicates:
Every UAV has a maxSpeed, …
Operators:
Definition of fly, pickup, …
Input 2: Problem instance
Objects:
Current UAVs are {UAV1,UAV2}
Initial State:
Box locations, …
Goal:
Box b1 at location l1, …
Defines
transitions
between
states
in the
formal
model
(STS)
Defines
initial and
goal
states

"Intuitive" explanation:
Integer
An actual number,
a member of ℤ
25
"Integer Statements"
A way of expressing a number
using a specific language
"10" chars "1" "0"
"X"
"0x0A"
"2*5"
"√100"
"2+2+2+2+2"
"p#c"
[in some made-up language]
…
jonkv@ida
Introduction

26
"Intuitive" explanation:
Problem
The actual structure:
A state transition system,
an initial state, a set of goal states
s0
s2
s1
s3
s5
s4
s6
s7
s8
Problem Statement
A way of writing down the problem
Many problem statements
correspond to the same problem
– Could list all states, actions, transitions
– PDDL domain + problem definition
(:action MOVE
:parameters (?disk ?source ?dest) …)
– PDDL propositional domain:
(:action MOVE-A-1-2 …)
– General Problem Solver language
–…
jonkv@ida
Introduction (2)
The "real domain"
27
A way of writing it down
Real World
Abstraction
Approximation
State Trans Syst
Σ = (S,A,γ)
Equivalence
First-order language L defined
by predicates, objects
Operators
O
States have no internal structure
States are sets of atoms, induced
by the predicates and objects in L
Actions are unstructured symbols
Operators are structured, have
preconditions and effects
State transitions are unstructured
(γ specified by state / action symbols)
Each operator specifies part of γ,
through its preconds and effects
jonkv@ida
Problems and Problem Statements 1
The "real problem"
28
A way of writing it down
Real World
+ current
problem
Abstraction
Approximation
Planning Problem
P = (Σ, s0, Sg)
Equivalence
Language L defined by
predicates, objects
Problem
Statement
P=(O,s0,g)
Specifies the ID of the initial state:
s0
Specifies the true atoms in the init state: {
attached(p1,loc1), in(c1,p1), … }
Specifies a set of possible goal states:
Sg = { s0, s1, s2, s20, s21, s22, s4912, … }
Specifies a set of literals that must hold:
g = { in(c1,p2), in(c2,p2) , … }
Often seen as a conjunctive goal formula
jonkv@ida
Problems and Problem Statements 2
29
Difference in size!
Real World
+ current
problem
Abstraction
Approximation
Planning Problem
P = (Σ, s0, Sg)
Equivalence
Language L defined by
predicates, objects
Problem
Statement
P=(O,s0,g)
Trillions of states in Σ = (S,A,γ)
would be a rather small
planning problem
Thousands of constants and predicates
in L would be a rather large
classical planning problem statement
Trillions of state transitions in γ
would also correspond to a small
planning problem
Hundreds of operators
would correspond to a very large
classical planning problem statement
jonkv@ida
Problems and Problem Statements 3
30
Planning algorithms work with the problem statement!
Real World
+ current
problem
Abstraction
Approximation
Planning Problem
P = (Σ, s0, Sg)
Equivalence
Working with the problem statement
gives direct access to
structured states, structured
operators, …
 Allows the problem to be analyzed
and treated at a higher level
Language L
Problem
Statement
P=(O,s0,g)
Planner
Plan
⟨a1, a2, …, an⟩
jonkv@ida
Problems and Problem Statements 4
Practical
representation language
Additional/different
convenient constructs
for practical modeling
Formal
representation language
Closer to how we think
"The simplest representation
that is (more or less) reasonable
to use for modeling"
Underlying formal model
Concepts as simple as possible
Good for analysis, correctness proofs,
understanding what planning is
Cumbersome for modeling
Misc.
Misc.
Preconditions
Effects
Extensions
Separation: Domain / instance
PDDL object types
Formulas: Disjunctions, …
Conditional effects, …
Timing, action costs, …
Objects
Fact atoms
State
Operators
Preconditions
Effects
{ car1, car2, car3, loc1, loc2 }
{ at(car1,loc1),at(car1,loc2),… }
Set of true atoms
drive(loc1, loc2) – with params
{ at(car1,loc1), ¬broken(car1) }
{ ¬at(car1,loc1), at(car1,loc2) }
This indirectly defines γ(s,a)!
States
Actions
s1 … s1000000000,
a1 … a10000 – no structure!
Transition
function
defining the result of an action,
γ(currentstate, action)=newstate
Goals
{s1,s3,s282} – set of end states
Covered in the book
Summary…