Backward Search
Jonas Kvarnström
Automated Planning Group
Department of Computer and Information Science
Linköping University
jonas.kvarnstrom@liu.se – 2015

2
Classical Planning: Find a path in a finite graph
If we are here
initial
We’ve seen
state space search
in this direction:
What can the next
action be?
Forward search,
initial  goal
Where do we end up?
goal
goal
goal
Known initial state
Deterministic actions
 Every node corresponds to one well-defined state
jonkv@ida
Repetition: Forward State Space Search

3
jonkv@ida
Backward Search
Classical Planning: Find a path in a finite graph
initial
Where would we
have to be before?
What about
this direction:
Backward,
goal  initial?
What can the previous
action be?
goal
goal
goal
If we want to be here
Forward search uses applicable actions
Backward search uses relevant actions:
Must achieve some requirement, must not contradict any requirement
What
stack(A,B)
needs
—————
What the goal
needs, but
stack(A,B) did
not achieve
We want to achieve:
holding(A)
clear(B)
—————
ontable(C)
ontable(D)
…
4
We want
to achieve
this…
A
B
C
D
jonkv@ida
Backward Search: Intuitions
We want to achieve:
holding(A)
clear(B)
—————
ontable(C)
ontable(D)
…
We want to achieve:
holding(D)
—————
ontable(C)
ontable(D)
…
5
We want
to achieve
this…
A
B
C
D
jonkv@ida
Backward Search: Intuitions (2)

Classical planning:
 Given a known state, applying an action gives single possible result
at(home)
drive-to(shop)
at(shop)
 Given an action and a desired result, there can be
many possible starting states
at(home)
at(work)
at(restaurant)
drive-to(shop)
at(shop)
7
jonkv@ida
Backward and Forward: Asymmetric!
Intuitive successor node:
Before:
Must be holding A
B must be clear
…
stack(A,B) is relevant:
It could be the last action
in a plan achieving this goal…
Formal successor node:
8
jonkv@ida
Backward Search: More Details
Intuitive goal:
A
B
C
D
Formal goal:
clear(B), on(B,C), ontable(C)
clear(D), ontable(D)
holding(A)
Applying stack(A,B) here
leads to the goal state
Suppose there is a
single goal state:
clear(B), on(B,C), ontable(C)
clear(D), ontable(D)
holding(A), handempty
Applying stack(A,B) here
leads to the goal state
sg = { clear(A), on(A,B),
on(B,C), ontable(C),
clear(D), ontable(D),
handempty }
clear(B), on(B,C), ontable(C)
clear(D), ontable(D)
holding(A), on(A,B)
Applying stack(A,B) here
leads to the goal state
Every node must correspond to many possible states!
9
sg = { clear(A), on(A,B), on(B,C),
ontable(C), clear(D), ontable(D),
handempty }
stack(A,B)
{
{clear(B), on(B,C), ontable(B)
clear(D), ontable(D), holding(A) },
{ clear(B), on(B,C), ontable(B)
clear(D), ontable(D)
holding(A), handempty },
{ clear(B), on(B,C), ontable(B)
clear(D), ontable(D)
holding(A), on(A,B) }, …
}
putdown(D)
A search node
corresponds to
a set of states
from which we
know how to
reach the goal
{
{ … },
{ … },
{ … },
…
}
jonkv@ida
Search Space
10
sg = { clear(A), on(A,B), on(B,C),
ontable(C), clear(D), ontable(D),
handempty }
Note: The above defined one goal state
sg = { clear(A), on(A,B), on(B,C), ontable(C), clear(D), ontable(D), handempty }
Exactly these atoms are true!
Even if we started with one goal state, we will eventually get several

Let's allow many goal states from the beginning, as in the state transition system:
Sg = set of goal states
jonkv@ida
Search Space
11
Forward search in the state space:
γ(s,a)
= { the state resulting from executing a in state s }
a is applicable to s iff
precond+(a) ⊆ s and
s ∩ precond–(a) = ∅
Γ(s)
= { γ(s,a) | a ∈ A and a is applicable to s }
Which state do I end up in?
Successors are created
only from applicable actions
All possible successor states
Backward search, if a node is a set of goal states g = { s1,…,sn }
γ-1(g,a)
= { s | a is applicable to s and
γ(s,a) satisfies some goal state in g }
a is relevant for g iff
it contributes to g and
it does not destroy any part of g
Γ-1(g)
= { γ-1 (g,a) | a ∈ A and a is relevant for g }
Regression through an action:
Which states could I start from?
Successors are created
only from relevant actions
(vague definition, clarified later)
All possible successor subgoals
jonkv@ida
State Sets and Regression
12
If we have a single goal state:
Sg = { sg }
stack-1(A,B)
putdown-1(D)
γ-1(Sg, stack(A,B))
{
{clear(B), on(B,C), ontable(B)
clear(D), ontable(D), holding(A) },
{ clear(B), on(B,C), ontable(B)
clear(D), ontable(D)
holding(A), handempty },
γ-1(Sg, putdown(D))
Storing (and
regressing over)
arbitrary sets of
goal states is
too expensive!
…
}
Γ-1(Sg)
{
{ … },
{ … },
{ … },
…
}
jonkv@ida
Search Space
Use the classical goal representation!
The (sub)goal of a search node is an arbitrary set of ground goal literals:
g = { in(c1,p3), …, in(c5,p3), ¬on(A,B) }
As usual, can't represent arbitrary sets of states – but works for:
Classical goals (already sets of ground literals)
Classical effects (conjunction of literals)
Classical preconditions (conjunction of literals)
13
jonkv@ida
Classical Representation
14
Use the classical goal representation!
The (sub)goal of a search node is an arbitrary set of ground goal literals:
g = { in(c1,p3), …, in(c5,p3), ¬on(A,B) }
All goals except effects(a)
must already have been true
precond(a)
must have been true,
so that a was applicable
γ-1(g,a)
= ((g – effects(a)) ∪ precond(a)), representing
{ s | a is applicable to s and γ(s,a) satisfies g }
Γ-1(g)
= { γ-1 (g,a) | a ∈ A and a is relevant for g }
Backward / regression:
Which states could I start from?
Efficient: Just one set (partial state), not a set of sets (set of states)!
a is relevant for g iff
g ∩ effects(a) ≠ ∅ and
g+ ∩ effects–(a) = ∅ and
g– ∩ effects+(a) = ∅
All successor subgoals
Contribute to the goal
(add positive or negative literal)
Do not destroy it
jonkv@ida
Classical Representation and Regression
15
Regression Example: Classical Representation
on(B,C)
clear(B)
clear(A)
ontable(A)
handempty
Not
actually
reachable
from init
state!
on(B,C)
holding(A)
clear(B)
on(B,C)
clear(B)
on(A,B)
clear(A)
handempty
Initial state:
on(A,C)
clear(A)
ontable(B)
clear(B)
clear(D)
ontable(C)
ontable(D)
handempty
Relevant:
Achieves
on(A,B), does
not delete
any goal fact
(g – effects(a))
on(A,B)
precond(a)
holding(B)
clear(C)
The goal is
not
already
achieved…
Goal:
on(A,B)
on(B,C)
Relevant:
Achieves
on(B,C), does
not delete
any goal fact
Represents
many
possible
goal states!
jonkv@ida
Regression
Forward search
…
16
Backward search
…
I can reach this node
from the initial state…
But what comes next?
Can I reach the goal?
Efficiently?
on(B,C)
clear(B)
on(A,B)
clear(A)
handempty
I can reach the goal
from this node…
But what comes before?
Can I reach it from s0?
Efficiently?
on(A,B)
on(B,C)
jonkv@ida
Backward and Forward Search: Unknowns
Forward search
Backward search
…
clear(A)
on(A,B)
ontable(B)
handempty
clear(A)
on(A,B)
ontable(B)
handempty
17
…
clear(A)
on(A,B)
handempty
Reach a node with the same state
 can prune
If preconditions and goals are positive:
Reach a node with a subset of the facts
 can prune
clear(A)
on(A,B)
ontable(B)
clear(A)
on(A,B)
ontable(B)
handempty
Reach a node with the same or stronger goal
 can prune
jonkv@ida
Backward and Forward Search: Pruning
FORWARD SEARCH

Problematic when:
18
BACKWARD SEARCH

Problematic when:
 There are many applicable actions
 There are many relevant actions
 Blind search knows
 Blind search knows
 high branching factor
 need guidance
if an action is applicable,
but not if it will contribute
to the goal
 high branching factor
 need guidance
if an action contributes to the goal,
but not if you can achieve its
preconditions
Blind backward search
is generally better than blind forward search:
Relevance tends to provide better guidance than applicability
This in itself is not enough to generate plans quickly!
jonkv@ida
Backward and Forward Search: Problems

19
Let’s take a look at expressivity:
 Suppose we have disjunctive preconditions
▪ (:action travel
:parameters (?from ?to – location)
:precondition (and (at ?from) (or (have-car) (have-bike)))
:effects
(and (at ?to) (not (at ?from))))
 How do we apply such actions backwards?
▪ More complicated
(at pos1)
disjunctive
(or (have-car)
goals to achieve?
(have-bike))
▪ Additional
branching?
(at pos1)
(have-car)
(at pos1)
(have-bike)
(at pos2)
(at pos2)
Similarly for existentials ("exists block [ on(block,A) ]"): One branch per possible value
Some extensions are less straight-forward in backward search (but possible!)
jonkv@ida
Backward and Forward Search: Expressivity

21
Even with conjunctive preconds:
(clear A)
(on-table A)
(handempty)
(clear A)
(on A B)
(handempty)
(holding A)
(clear B)
(clear C)
(clear A)
(on A C)
(handempty)
(clear A)
(on A D)
(handempty)
 High branching factor
 No reason to decide now
which block to unstack A from
jonkv@ida
Lifted Search (1)
(:action pickup
:parameters (?x)
:precondition (and (clear ?x) (on-table ?x)
(handempty))
:effect
(and (not (on-table ?x))
(not (clear ?x))
(not (handempty))
(holding ?x)))
(:action unstack
:parameters (?top ?below)
:precondition (and (on ?top ?below)
(clear ?top) (handempty))
:effect
(and (holding ?top)
(clear ?below)
(not (clear ?top))
(not (handempty))
(not (on ?top ?below))))

22
General idea in lifted search:
 Keep some variables uninstantiated (non-ground  lifted)
(clear A)
(on-table A)
(handempty)
Next step:
How to check for actions
achieving (on A ?x)?
(clear A)
(on A ?x)
(handempty)
Requires unification – see
the book, fig 4.3
Applicable to other types of planning!
Will be seen again in Partial Order Causal Link planning
(holding A)
(clear B)
(clear C)
jonkv@ida
Lifted Search (2)

24
Consider hm heuristics using forward search:
Need
∆m(s1, g),
∆m(s2, g),
∆m(s3, g),
Need
∆m(s0, g)
s0
A
C B D
s1
A
C B D
A
C
B
A
C B
s2
D
Ds3
Need
∆m(s4, g),
∆m(s5, g)
A
C
Bs4
D
B
A
C
s5
D
A
B
C
D
jonkv@ida
Forward Search with hm
Goal:
clear(A)
on(A,B)
on(B,C)
ontable(C)
clear(D)
ontable(D)
cost 0
cost 2
cost 2
cost 0
cost 0
cost 0
stack(A,B)
holding(A)
clear(B)
cost 1
25
stack(B,C)
holding(B)
clear(C)
cost 0
cost 1
unstack(A,C)
handempty
clear(A)
cost 0
cost 0
More calculations show:
This is expensive…
D
cost 1
on(A,C)
cost 0
Search continues: This is cheaper!
unstack(A,D)
handempty
clear(A)
on(A,D)
A
B
C
pickup(B)
handempty
clear(B)
cost 0
cost 0
unstack(A,C)
handempty
clear(A)
on(A,C)
A
C B D
current: clear(A), on(A,C), ontable(C), clear(B), ontable(B), clear(D), ontable(D), handempty
Calculations depend very much on the entire current state!
New search node  new current state  recalculate ∆m from scratch
jonkv@ida
Forward Search with hm: Illustration

In backward search:
Need
∆m(s0, g3),
∆m(s0, g4),
∆m(s0, g5)
New search node 
same starting state 
use the old ∆m values
for previously
encountered
goal subsets
s0
A
C B D
g3
B
C D A
g4
B
C
A
B
C
A
D
g5
D
26
∆1(s0, g1)
is the max of
∆1(s0, ontable(C)),
∆1(s0, ontable(D)),
∆1(s0, clear(A)),
…,
∆1(s0, holding(A))
Need
∆m(s0, g1),
∆m(s0, g2)
g1
A
B
C
A
B
C
D
g2
D
∆1(s0, g0)
is the max of
∆1(s0, ontable(C)),
∆1(s0, ontable(D)),
∆1(s0, clear(A)),
∆1(s0, clear(D)),
∆1(s0, on(A,B)),
…
Need
∆m(s0, g0)
A
B
C
g0
D
jonkv@ida
Backward Search with hm

Results:
 Faster calculation of heuristics
 Not applicable for all heuristics!
▪ Many other heuristics work better with forward planning
27
jonkv@ida
HSPr, HSPr*
Planning using Planning Graphs
Jonas Kvarnström
Automated Planning Group
Department of Computer and Information Science
Linköping University
jonas.kvarnstrom@liu.se – 2015
30
BACKWARD SEARCH


We know if the effects of an action
can contribute to the goal
Don't know if we can reach a state
where its preconditions are true
so we can execute it
at(home)
have-heli
at(LiU)
…
at(home)
have-shoes
One solution: Heuristics. But other methods exist…
jonkv@ida
Recap: Backward Search

31
Suppose that:
 We are interested in sequential plans with few actions
 We can quickly calculate the set of reachable states at time 0, 1, 2, 3, …
 We can quickly test formulas in such sets

Then we can:
 Use this to prune the backward search tree!
jonkv@ida
Reachable States

32
First step: Determine the length of a shortest solution
 Doesn't tell us the actual solutions:
We know states, not paths
Time 4
Time 3
Time 2
Time 1
Time 0
Initial
state
These 10
states are
reachable
These 200
states are
reachable
These
4000
states are
reachable
Goal not
satisfied
Goal not
satisfied in
any state
Goal not
satisfied in
any state
Goal not
satisfied in
any state
These
12000
states are
reachable
Goal satisfied
in 14 of the states:
All shortest solutions
must have 4 actions!
jonkv@ida
Reachable States

33
Second step: Backward search with pruning
Time 4
Time 3
Time 2
Time 1
Time 0
Initial
state
These 10
states are
reachable
These 200
states are
reachable
These
4000
states are
reachable
Regression:
at(home)
have-heli
These
12000
states are
reachable
One
relevant
action:
fly-heli
Not true (reachable) in any of the 4000 states at time 3!
We know preconds can’t be achieved in 3 steps  backtrack
at(LiU)
…
True in 14 states
jonkv@ida
Backward Search with Pruning

34
There must be some other way of achieving at(LiU)!
Time 4
Time 3
Time 2
Time 1
Time 0
Initial
state
These 10
states are
reachable
Continue backward search
as usual,
using reachable states
to prune the search tree
These 200
states are
reachable
These
4000
states are
reachable
Regression:
at(home)
have-shoes
These
12000
states are
reachable
Another
relevant
action:
walk
Preconds are achievable at time 3!
(True in several states)
at(LiU)
…
True in 14 states
jonkv@ida
Backward Search with Pruning (2)
35
Problem: Fast and exact computation is impossible!

”Suppose we could quickly calculate all reachable states…”
 In most cases, calculating exactly the reachable states
would take far too much time and space!
 Argument as before
("otherwise planning would be easy, and we know it isn't")
jonkv@ida
Problem

37
Solution: Don’t be exact!
 Quickly calculate an overestimate
 Anything outside this set is definitely not reachable – still useful for pruning
Time 4
Time 3
Time 2
Time 1
Time 0
Initial
state
These 15:
possibly
reachable
10
These
500:
possibly
reachable
Includes
200
These
15000:
possibly
reachable
Includes
all 4000
truly
reachable
…out of a billion?
These
42000:
possibly
reachable
Includes
all 12000
truly
reachable
jonkv@ida
Possibly Reachable States (1)

38
Planning algorithm:
 Keep calculating until we find a timepoint
where the goal might be achievable
Time 2
Time 1
Time 0
Initial
state
These 15:
possibly
reachable
These
500:
possibly
reachable
Goal not
satisfied
Goal not
satisfied in
any state
Goal not
satisfied in
any state
Time 3
These
15000:
possibly
reachable
Includes
all 4000
truly
reachable
Satisfied in
37 possibly reachable states:
A shortest solution must have
at least 3 actions!
jonkv@ida
Possibly Reachable States (2)

39
Backward search will verify what is truly reachable
 In a much smaller search space than plain backward search
Time 3
Time 2
These
15000:
possibly
reachable
Time 1
Time 0
Initial
state
These 15:
possibly
reachable
These are all of the
relevant actions…
(Of course we don't
always detect the bad
choice in a single time
step!)
These
500:
possibly
reachable
Includes
all 4000
truly
reachable
Preconds not
satisfied…
fly-heli
at(LiU)
…
Preconds not
satisfied…
walk
at(LiU)
…
The goal seems
to be reachable here,
but we can’t be sure
(overestimating!)
jonkv@ida
Possibly Reachable States (3)

40
Extend one time step and try again
Time 4
 A larger number of states may be truly reachable then
Time 3
Time 2
Time 1
Time 0
Initial
state
These 15:
possibly
reachable
These
500:
possibly
reachable
Continue backward search,
using reachable states
to prune the search tree
 smaller search space
Pruning still
possible!
These
42000:
possibly
reachable
These
15000:
possibly
reachable
Includes
all 4000
truly
reachable
Preconds not
satisfied…
Seems OK!
Includes
all 12000
truly
reachable
fly-heli
at(LiU)
…
walk
at(LiU)
…
jonkv@ida
Possibly Reachable States (4)

41
This is a form of iterative deepening search!
Time step = state step
(1 time step  0 actions)
Classical problem P
Loop for i = 1, 2, …
Try to find a plan
with i time steps
(including the
initial state)
no solution
with i-1 actions exists
Plan!
{ solutions to P } =
{ solutions to P with i time steps | i ∈ ℕ, i >= 0 }
jonkv@ida
Iterative Search
An efficient representation
for possibly reachable states

43
A Planning Graph also considers possibly executable actions
 Useful to generate states – also useful in backwards search!
k+1 proposition levels
Which propositions may possibly hold in each state?
Initial
state
1200
200
4000
10 possibly 8 possibly 47 possibly
possibly
possibly
possibly
executable reachable executable
reachable executable reachable
actions
actions
states
actions
states
states
k action levels
Which actions may possibly be executed in each step?
jonkv@ida
Planning Graph

44
GraphPlan’s plans are sequences of sets of actions
  Fewer levels required!
Initial
state
1200
200
4000
10 possibly 8 possibly 47 possibly
possibly
possibly
possibly
executable reachable executable
reachable executable reachable
actions
actions
states
actions
states
states
load(Package1,Truck1),
load(Package2,Truck2),
load(Package3,Truck3)
Can be executed in
arbitrary order
drive(T1,A,B),
drive(T2,C,D),
drive(T3,E,F)
unload(Package1,Truck1),
unload(Package2,Truck2),
unload(Package3,Truck3)
Arbitrary
order
Can be executed in
arbitrary order
Not necessarily in parallel – original objective was a sequential plan
jonkv@ida
GraphPlan: Plan Structure

Running example due to Dan Weld (modified):
 Prepare and serve a surprise dinner,
take out the garbage,
and make sure the present is wrapped before waking your sweetheart!
s0 = {clean, garbage, asleep}
g = {clean, ¬garbage, served, wrapped}
Action
Preconds Effects
cook()
serve()
wrap()
carry()
roll()
clean()
clean
dinner
asleep
garbage
garbage
¬clean
dinner
served
wrapped
¬garbage, ¬clean
¬garbage, ¬asleep
clean
45
jonkv@ida
Running Example
 Time 0:
▪ s0
 Time 1:
▪ cook
▪ serve
▪ wrap
▪ carry
▪ roll
▪ clean
▪ cook+wrap
▪ cook+roll
▪ …
 Time 2:
▪ cook/cook
▪ cook/serve
▪ cook/wrap
▪ cook/carry
▪ cook/roll
▪ cook/clean
▪ wrap/cook
 {clean, garbage, asleep}
46
Can’t calculate and store
all reachable states,
one at a time…
 {clean, garbage, asleep, dinner}
 impossible
 {clean, garbage, asleep, wrapped}
Let’s calculate
 {asleep}
reachable literals
 {clean}
instead!
 impossible
 {garbage, clean, asleep, dinner, wrapped}
 {clean, dinner}
 {clean, garbage, asleep, dinner}
 {clean, garbage, asleep, dinner, served}
 {clean, garbage, asleep, dinner, wrapped}
 {asleep, dinner}
 {clean, dinner}
 not possible
…
jonkv@ida
Reachable States
Time 0:
 s0 = {garbage, clean, asleep}


Time 1:






cook
(serve)
wrap
carry
roll
(clean)
 s1 = {garbage, clean, asleep, dinner}
not applicable
 s2 = {garbage, clean, asleep, wrapped}
 s3 = {asleep}
 s4 = {clean}
not applicable
No need to consider sets of actions:
All literals made true
by any combination of ”parallel” actions
are already there
State
level 0
Action
level 0
garbage
State
level 1
garbage
¬garbage
clean
asleep
¬dinner
We can't reach all combinations
of literals in state level 1…
¬served
But we can reach only such combinations!
Can not reach a state where served is true
¬wrapped
clean
Depending on which actions
we choose here…

47
jonkv@ida
Reachable Literals (1)
¬clean
asleep
¬asleep
dinner
¬dinner
¬served
wrapped
¬wrapped

48
Planning Graph Extension:
State
level 0
 Start with one ”state level”
▪ Set of reachable literals
garbage
 For each applicable action
▪ Add its effects
to the next state level
▪ Add edges to preconditions
and to effects
for bookkeeping (used later!)
Action
cook()
serve()
wrap()
carry()
roll()
clean()
Precond
clean
dinner
asleep
garbage
garbage
¬clean
Effects
dinner
served
wrapped
¬garbage, ¬clean
¬garbage, ¬asleep
clean
But wait!
Some propositions are missing…
Action
level 1
State
level 1
carry
¬garbage
roll
¬clean
clean
asleep
¬asleep
cook
dinner
¬dinner
wrap
¬served
wrapped
¬wrapped
jonkv@ida
Reachable Literals (2)

Depending on the actions chosen,
facts could persist
from the previous level!
49
State
level 0
Action
level 1
garbage
To handle this consistently:
maintenance (noop) actions
One for each literal l
 Precond = effect = l

Action
cook()
serve()
wrap()
carry()
roll()
clean()
noopdinner
noopnotdin
…
Precond
clean
dinner
asleep
garbage
garbage
¬clean
dinner
¬dinner
Effects
dinner
served
wrapped
¬garbage, ¬clean
¬garbage, ¬asleep
clean
dinner
¬dinner
State
level 1
garbage
carry

jonkv@ida
Reachable Literals (3)
clean
¬garbage
clean
roll
asleep
¬clean
asleep
¬asleep
cook
¬dinner
dinner
¬dinner
wrap
¬served
¬served
wrapped
¬wrapped
¬wrapped

Now the graph is sound
 If an action might be executable,

50
State
level 0
it is part of the graph
 If a literal might hold
in a given state,
it is part of the graph
garbage
But it is quite “weak”!
asleep
Action
level 1

carry
clean
¬garbage
clean
roll
¬clean
asleep
¬asleep
cook
¬dinner
We need more information
 In an efficiently useful format
State
level 1
garbage
 Even at state level 1,
it seems any literal
except served
can be achieved
jonkv@ida
Reachable Literals (4)
dinner
¬dinner
wrap
¬served
 Mutual exclusion
¬served
wrapped
¬wrapped
¬wrapped
No mutexes at state level 0:
We assume a consistent initial state!
State
level 0
51
Action
level 1
garbage
Two actions in a level are mutex
if their effects are inconsistent
Can’t execute them in parallel, and
order of execution is not arbitrary
 carry / noop-garbage,
 roll / noop-garbage,
roll / noop-asleep
 cook / noop-¬dinner
 wrap / noop-¬wrapped
State
level 1
garbage
carry
clean
¬garbage
clean
roll
asleep
¬clean
asleep
carry / noop-clean
▪ One causes garbage,
the others cause not garbage
jonkv@ida
Mutex 1: Inconsistent Effects
¬asleep
cook
¬dinner
dinner
¬dinner
wrap
¬served
¬served
wrapped
¬wrapped
¬wrapped
Two actions in one level are mutex
if one destroys a precondition
of the other
52
State
level 0
garbage
 carry is mutex with noop-garbage
▪ carry deletes garbage
▪ noop-garbage needs garbage
 …
State
level 1
garbage
carry
Can’t be executed in arbitrary order
 roll is mutex with wrap
▪ roll deletes asleep
▪ wrap needs asleep
Action
level 1
jonkv@ida
Mutex 2: Interference
clean
¬garbage
clean
roll
asleep
¬clean
asleep
¬asleep
cook
¬dinner
dinner
¬dinner
wrap
¬served
¬served
wrapped
¬wrapped
¬wrapped
Two propositions are mutex
if one is the negation of the other
Can’t be true at the same time…
State
level 0
53
Action
level 1
garbage
jonkv@ida
Mutex 3: Inconsistent Support
State
level 1
garbage
carry
clean
¬garbage
clean
roll
asleep
¬clean
asleep
¬asleep
cook
¬dinner
dinner
¬dinner
wrap
¬served
¬served
wrapped
¬wrapped
¬wrapped
Two propositions are mutex
if they have inconsistent support
All actions that achieve them
are pairwise mutex
in the previous level
¬asleep can only be achieved by roll,
wrapped can only be achieved by wrap,
and roll/wrap are mutex
 ¬asleep and wrapped are mutex
State
level 0
54
Action
level 1
garbage
garbage
carry
clean
¬garbage
clean
roll
asleep
¬clean
asleep
¬asleep
cook
¬clean can only be achieved by carry,
dinner can only be achieved by cook,
and carry/cook are mutex
 ¬clean and dinner are mutex
State
level 1
¬dinner
dinner
¬dinner
wrap
¬served
¬served
wrapped
¬wrapped
¬wrapped
jonkv@ida
Mutex 4: Inconsistent Support



55
Two actions at the same action-level are mutex if
 Inconsistent effects: an effect of one negates an effect of the other
 Interference: one deletes a precondition of the other
 Competing needs: they have mutually exclusive preconditions
Otherwise they don’t interfere with each other
Recursive
 Both may appear at the same time step in a solution plan
Two literals at the same state-level are mutex if
 Inconsistent support: one is the negation of the other,
or all ways of achieving them are pairwise mutex
propagation
of mutexes
jonkv@ida
Mutual Exclusion: Summary

Is there a possible solution?

Goal g =
{clean, ¬garbage, served, wrapped}

No: Cannot reach a state
where served is true
in a single (multi-action) step
56
State
level 0
Action
level 1
garbage
State
level 1
garbage
carry
clean
¬garbage
clean
roll
asleep
¬clean
asleep
¬asleep
cook
¬dinner
dinner
¬dinner
wrap
¬served
¬served
wrapped
¬wrapped
¬wrapped
jonkv@ida
Early Solution Check
State
level 0
Action
level 1
garbage
carry
clean
State
level 1
asleep
cook
¬dinner
¬wrapped
State
level 2
garbage
¬garbage
¬garbage
¬clean
carry
roll
clean
¬clean
asleep
asleep
¬asleep
¬asleep
dinner
¬dinner
wrap
¬served
Action
level 2
garbage
clean
roll
57
cook
serve
wrap
dinner
¬dinner
served
¬served
¬served
wrapped
wrapped
¬wrapped
¬wrapped
All goal literals are present in level 2, and none of them are mutex!
jonkv@ida
Expanded Planning Graph
State
level 0
Action
level 1
garbage
carry
clean
59
State
level 1
asleep
cook
¬dinner
We seem to have 6 alternatives
at the last layer (3*1*1*2): wrap
¬served
3 ways of achieving ¬garbage
1 way of achieving clean
1 ¬wrapped
way of achieving served
2 ways of achieving wrapped
State
level 2
garbage
garbage
¬garbage
¬garbage
clean
roll
Action
level 2
¬clean
carry
clean
roll
¬clean
asleep
asleep
¬asleep
¬asleep
dinner
¬dinner
cook
serve
wrap
dinner
¬dinner
served
¬served
¬served
wrapped
wrapped
¬wrapped
¬wrapped
g = {clean, ¬garbage, served, wrapped}
jonkv@ida
Solution Extraction (1)
State
level 0
Action
level 1
garbage
carry
clean
State
level 1
asleep
cook
¬dinner
Here:
wrap
An inconsistent alternative
¬served
(mutex
actions chosen)…
Action
level 2
State
level 2
garbage
garbage
¬garbage
¬garbage
clean
roll
¬wrapped
60
¬clean
carry
clean
roll
¬clean
asleep
asleep
¬asleep
¬asleep
dinner
¬dinner
cook
serve
wrap
dinner
¬dinner
served
¬served
¬served
wrapped
wrapped
¬wrapped
¬wrapped
g = {clean, ¬garbage, served, wrapped}
jonkv@ida
Solution Extraction (2)
State
level 0
Action
level 1
garbage
carry
clean
61
State
level 1
asleep
cook
¬dinner
This is one of the consistent
combinations – a successor wrap
in this
¬served backwards search
“multi-action”
The choice is a backtrack point!
(May¬wrapped
have to backtrack, try to
achieve ¬garbage by roll instead)
State
level 2
garbage
garbage
¬garbage
¬garbage
clean
roll
Action
level 2
¬clean
carry
clean
roll
¬clean
asleep
asleep
¬asleep
¬asleep
dinner
¬dinner
cook
serve
wrap
dinner
¬dinner
served
¬served
¬served
wrapped
wrapped
¬wrapped
¬wrapped
g = {clean, ¬garbage, served, wrapped}
jonkv@ida
Solution Extraction (3)
State
level 0
Action
level 1
garbage
carry
clean
62
State
level 1
asleep
cook
¬dinner
garbage
¬garbage
¬garbage
¬clean
roll
clean
¬clean
asleep
¬asleep
¬asleep
dinner
The successor has a new goal
wrap
The goal
is not mutually exclusive
¬wrapped
(if so, it would be unreachable)
carry
asleep
¬dinner
This goal consists of all
¬served
preconditions of the chosen action
(Not a backtrack point)
State
level 2
garbage
clean
roll
Action
level 2
cook
serve
wrap
dinner
¬dinner
served
¬served
¬served
wrapped
wrapped
¬wrapped
¬wrapped
Successor goal = {clean, garbage, dinner, wrapped}
jonkv@ida
Solution Extraction (4)
State
level 0
Action
level 1
garbage
carry
clean
63
State
level 1
asleep
cook
¬dinner
garbage
¬garbage
¬garbage
¬clean
One possible solution:
1) cook/wrap
¬wrappedin any order,
2) serve/roll in any order
carry
roll
clean
¬clean
asleep
asleep
¬asleep
¬asleep
dinner
¬dinner
wrap
¬served
State
level 2
garbage
clean
roll
Action
level 2
cook
serve
wrap
dinner
¬dinner
served
¬served
¬served
wrapped
wrapped
¬wrapped
¬wrapped
jonkv@ida
Solution Extraction (5)
The set of goals we are
trying to achieve
64
The level of the state si,
starting at the highest level
procedure Solution-extraction(g,i)
A form of backwards search,
if i=0 then return the solution
but only among the actions in the graph
(generally much fewer, esp. with mutexes)!
nondeterministically choose
a set of non-mutex actions
("real" actions and/or maintenance actions)
statestateactionto use in state s i–1 to achieve g
level
level
level
(must achieve the entire goal!)
i-1
i
i
if no such set exists then fail (backtrack)
g’ := {the preconditions of the chosen actions}
Solution-extraction(g’, i–1)
end Solution-extraction
jonkv@ida
Solution Extraction 6

65
Possible literals:
 What is achieved is always carried forward by no-ops
  Monotonically increase over state levels

Possible actions:
 Action included if all precondition literals exist in the preceding state level
  Monotonically increase over action levels
State
level 0
Action
level 1
garbage
carry
clean
State
level 1
asleep
cook
State
level 2
garbage
garbage
¬garbage
¬garbage
clean
roll
Action
level 2
¬clean
carry
roll
clean
¬clean
asleep
asleep
¬asleep
¬asleep
dinner
cook
dinner
jonkv@ida
Important Properties

66
Mutex relationships:
 Mutexes between included literals monotonically decrease
▪ If two literals could be achieved together in the previous level,
we could always just “do nothing”, preserving them to the next level
 (Mutexes to newly added literals can be introduced)

At some point, the Planning Graph “levels off”
 After some time k all levels are identical
 (Why?)
A planning graph is exactly what we described here –
not some arbitrary planning-related graph
jonkv@ida
Important Properties (2)
Top: Real state reachability for a Rover problem ("avail" facts not shown)
Bottom: Planning Graph for the same problem (mutexes not shown)
At each step, an overestimate of reachable properties / applicable actions!

Again, a form of iterative deepening:
Classical problem P
Loop for i = 1, 2, …

68
Try to find a plan
with i time steps
(including the initial
state)
Plan!
Therefore, GraphPlan is optimal in the number of time steps
 Not very useful: We normally care much more about
▪ Total action cost
▪ Number of actions (special case where action cost = 1)
▪ Total execution time (”makespan”)
load(Package1,Truck1),
load(Package2,Truck2),
load(Package3,Truck3)
drive(T1,A,B),
drive(T2,C,D),
drive(T3,E,F)
unload(Package1,Truck1),
unload(Package2,Truck2),
unload(Package3,Truck3)
jonkv@ida
Parallel Optimality