jonkv@ida
Why Automated Planning?
y
g
ƒ
2
2007-12-12
What we start with:
jonkv@ida
Why Automated Planning?
y
g
ƒ
ƒ A system controlled at a low level
▪
▪
▪
▪
▪
Jonas Kvarnström
KPLAB / AIICS / IDA
Introduction to
A t
Automated Planning
t d Pl
i
▪
▪
▪
▪
▪
▪
ƒ
4
2007-12-12
Take off / land
Fly from point (x1,y1) to point (x2,y2) at altitude z
H
Hover in the current position
i h ii
Position the camera at angle θ
T k i t
Take a picture
…
ƒ Higher level...
Hi h l l
Barometric pressure (altitude)
( l
d )
Inertial navigation + GPS
C l Color camera
…
jonkv@ida
Why planning?
yp
g
What we build using this functionality:
ƒ Meaningful higher‐level actions
Engine thrust / RPM
Blade angles
Pi h Pitch angle
l
Yaw angle
…
ƒ A set of sensors
▪
▪
▪
▪
▪ ...but not high enough!
jonkv@ida
Why planning?
yp
g
5
2007-12-12
jonkv@ida
Why planning?
yp
g
ƒ Deliver a set of packages to their destinations,
to their destinations
What we want:
3
2007-12-12
6
2007-12-12
ƒ Patrol an area in search of potential forest fires
using all available resources (UAVs/UGVs)
ƒ A system capable of solving even higher‐level goals
▪ Take pictures of all buildings in an area,
while avoiding no‐fly‐zones
jonkv@ida
A Specialized Solution
p
ƒ
7
2007-12-12
All tasks can be solved using specialized methods
ƒ Example: Take pictures of all buildings
▪ Use ”Travelling Salesman” algorithms to find shortest path
▪ Use TSP solution to generate flight commands
Picture Problem
Picture
Planner
Picture Plan
jonkv@ida
A Specialized Solution
p
ƒ
But specialization means less flexibility
ƒ What if…
▪
▪
▪
▪
▪
2007-12-12
jonkv@ida
A General Solution
ƒ
Automated planners are domain‐independent
are domain independent
ƒ No a priori domain knowledge in algorithms / code!
you want to deliver a couple of crates at the same time?
you want to survey an area (take pictures of the ground)?
you have two UAVs
h UAV and a UGV
d UGV (ground vehicle)?
(
d hi l )?
you have to consider refueling?
you have dynamic no‐fly‐zones (”don’t fly there at 15:00‐16:00”)?
h d
i fl
(”d ’t fl th t 6 ”)?
Picture
Planner
(TSP)
8
Picture+
Delivery
Pl
Planner
Picture
Planner
w/ fuel
/ f l
ƒ Planners use generic search methods
▪
▪
▪
▪
▪
Can be applied to the UAV domain
Can be applied to transportation problems
Can be applied to satellite operations
Can be applied to the blocks world
…
9
2007-12-12
jonkv@ida
Planning Domains
g
ƒ
10
2007-12-12
jonkv@ida
Problem Instances
▪ Object Types: There are UAVs, crates, coordinates, …
▪ World model: Objects can be at coordinates,
UAVs have altitudes,
UAVs have altitudes
a crate can be carried or on the ground,
we may have or not have a picture of a building, …
y
p
g
▪ Operators:
Take off, hover, fly to point, pick up crate, …
(with preconditions and effects)
General
Planner
2007-12-12
▪ Perform these actions,
with these timing constraints,
in order to achieve the goal
▪ For optimizing planners: As cheaply / quickly / … as possible
▪ For satisfising planners: Try to find a plan that is “good enough”
ƒ Current World State
ƒ Goal State
ƒ Object Types
ƒ World Model
ƒ Operators
General
Planner
jonkv@ida
12
2007-12-12
ƒ Output: A plan
ƒ Current World State
ƒ Goal State
ƒ Object Types
ƒ World Model
ƒ Operators
13
Plans
▪ Actual objects: 2 UAVs (A and B), 10 crates (C1 … C10)
▪ Actual state:
Crate 1 at position (300,370), …
Crate 1 at position (300
370) ▪ Goal:
2 crates of supplies should be at (400,200),
UAV 2 should be at (100,127),
( , 7),
Should have pictures of all buildings in area X
ƒ Input 1: High‐level specification of a planning domain
Many Domains, One Planner
y
,
2007-12-12
jonkv@ida
ƒ Input 2: High‐level specification of a problem instance
Two types of input to domain independent planners
Two types of input to domain‐independent planners
ƒ Object Types
ƒ World Model
ƒ Operators
11
General
Planner
A plan that uses
available actions
and resources to
achieve the goal
jonkv@ida
Advantages and Disadvantages
g
g
ƒ
14
2007-12-12
Advantages of automated planning:
ƒ One (large) planner + n domain definitions,
UAV Problem
rather than n
h h different hard‐coded implementations
diff
h d d d i l
i
UAV Domain
General
Planner
▪ Easier to specify a domain than to implement from scratch
▪ Less information, and at a higher level Î
L i f
i d hi h l l Î less error‐prone
l ▪ Improvements to the planner Î all n domains benefit
UAV Plan
Pl
ƒ Flexibility
Fl ibili and elaboration tolerance
d l b
i l
I: Classical Planning
g
▪ Easier to change than a hard‐coded optimized implementation
ƒ
Satellite Problem
Satellite Domain
Ad
Advantages
of optimized implementations:
f i i d i l
i
ƒ More efficient – if you have the time to optimize
Satellite Plan
General
Planner
jonkv@ida
Formalization
ƒ
16
2007-12-12
To solve the planning problem we need to design:
To solve the planning problem, we need to design:
A representation for planning domains
2. A representation for problem instances
3. A structure for plans
p
4. An algorithm that generates a plan
1.
ƒ Current World State
ƒ Goal State
jonkv@ida
Expressivity and Efficiency (1)
p
y
y( )
ƒ
Planner
A plan that uses
available actions
and resources to
achieve the goal
2007-12-12
The more expressive the input language
The more expressive the input language…
ƒ …the more faithfully we can represent a domain
▪
▪
▪
▪
▪
Conditional effects (”if there is gas, then the engine starts”)
Action timing (”flying from A to B takes 100 minutes”)
Di j
Disjunctive preconditions (”device must be in mode A, B or C”)
i di i
(”d i b i d A B C”)
Incomplete knowledge (”rather warm”, ”don’t know where B is”)
P ti ll k
Partially known outcomes (”will end up at point Z +/‐
t
(” ill d t i t Z / 10 meters”)
t ”)
ƒ Not too difficult to represent
N t t diffi lt t t
ƒ Object Types
ƒ World Model
ƒ Operators
17
ƒ Very expressive languages available from other areas
▪ Knowledge representation (KR)
▪ Reasoning about action and change (RAC)
ƒ How about ”importing” these languages?
jonkv@ida
Expressivity and Efficiency (2)
p
y
y( )
ƒ
Problem: Representation is only the first step!
ƒ Also need to reason using the representation!
ƒ KR and RAC deal with specific reasoning problems
▪ Prediction – if I do this, what will happen?
▪ Postdiction – I did this and that happened; what can I conclude?
▪ ...
ƒ Planning is a different problem
▪ If I want this to happen, what should I do?
▪ Need to consider many different courses of action, not just one
▪ What is easily handled in prediction may be difficult in planning
18
2007-12-12
jonkv@ida
Expressivity and Efficiency (3)
p
y
y (3)
ƒ
19
2007-12-12
The more expressivity we support
The more expressivity we support...
jonkv@ida
Classical Planning
g
ƒ
ƒ …the more complex the algorithms will have to be!
20
2007-12-12
Restrictions / assumptions in ”classical planning”
Restrictions / assumptions in classical planning
jonkv@ida
Modeling Around Restrictions
g
ƒ
ƒ The world can be described using finite states
▪ High performance is also desirable
▪ Many efficient strategies are only usable with lower expressivity
▪ But we can use fixed precision (0.000 to 9999.999)
ƒ Operators are completely defined
ƒ Operators are deterministic and completely defined
▪ Improving performance, trying to retain expressivity if possible
▪ Improving expressivity, trying to retain performance if possible
Some restrictions can be “modeled around”
Some restrictions can be modeled around
ƒ No real‐valued numbers (infinite domains)
▪ A fixed number of properties are modeled
▪ No real‐valued numbers
ƒ Planning research advances on (at least) two fronts
(
)
21
2007-12-12
▪ But we can build tolerance margins into operator execution:
“go to crate 4 at (100,200)” interpreted as “go to c4 near (100,200)”
▪ No unknown effects (throwing dice, placing crate 1 at pos4 ± 5 m)
ƒ The initial state is completely defined
Th i i i l i l l d fi d
ƒ The initial state is completely defined
▪ Some facts about the world can be left out of the model
▪ Some partly unknown facts can be handled using tolerance margins
▪ Some boolean facts can be given the domain “yes, no, unknown”
(but technically speaking the fact is then “definitely unknown”)
(but technically speaking, the fact is then definitely unknown )
▪ Must know all relevant facts about the world – strong assumption!
ƒ Single‐step actions
▪ Instantaneous transition from invocation to result state
ƒ Only the actions in the plan can change the world state
ƒ No other agents can modify the world
▪ No other agents can modify the world
g
y
▪ No events taking place outside our control
▪ But if we monitor the execution, we can detect changes and recover
B t if it th ti d t t h
d You are a robot...
...this is what you see...
y
One Step Back…
A less complex planning domain example
A
C
B
jonkv@ida
...this is what you want...
y
...and this is what you can do.
y
ƒ
A
B
ƒ
ƒ
26
2007-12-12
Pick up a block
jonkv@ida
Silly Toy Domains?
y y
ƒ
27
2007-12-12
Is this just a silly toy domain?
ƒ but only if it is clear (there is no other block on top of it).
ƒ In one sense, yes – just a pile of blocks, nothing complex
Put a block that you are holding on the table.
Put a block you are holding on another block
ƒ In one sense, no: A distilled version of a specific difficulty!
ƒ but only if the destination is clear.
ƒ Example below:
p
▪ Almost all facts are correct: A on B on C, A is clear, F on the table, …
▪ But we are very far from achieving the goal
▪ Put all blocks on the table before rebuilding!
A
B
A
C
A
A
C
B
B
C
B
C
C
D
D
E
E
F
F
jonkv@ida
Required Input
q
p
Formalizing the Blocks World
ƒ
29
2007-12-12
Recall the required input:
jonkv@ida
The Blocks World Domain
ƒ
The blocks world domain consists of:
ƒ Types
A specification of the planning domain (blocks world)
2. A specification of the problem instance (blocks A/B/C) A Classical Planning Domain
30
2007-12-12
1.
▪ None – only blocks
ƒ Predicates
▪
▪
▪
▪
ƒ Current World State
ƒ Goal State
ƒ Object Types
ƒ World Model
ƒ Operators
Planner
ƒ
31
2007-12-12
Operators:
A
ƒ pickup(x) – pick up a block from the table
C
A
C
ƒ
B
The current problem instance:
A
ƒ Objects
C
32
2007-12-12
jonkv@ida
Correct?
ƒ
B
▪ {A, B, C}
▪
▪
▪
▪
▪ precond: holding(x)
▪ effect:
¬holding(x) & ontable(x) & clear(x)
ƒ unstack(x,y) – pick up block x, which is now on block y
on(A,C)
ontable(B), ontable(C)
clear(A), clear(B)
All other predicate instances are false!
ƒ Goal
▪ precond: on(x,y) & clear(x)
▪ effect:
¬on(x,y) & ¬clear(x) & holding(x) & clear(y)
▪ on(A,B), on(B,C)
A
ƒ Begin with the initial state
C
B
▪ precond: on(A,C) & clear(A)
▪ effect:
¬on(A,C) & ¬clear(A) & holding(A) & clear(C)
▪ result:
ontable(B), ontable(C), clear(B), holding(A), clear(C) A
ƒ Now pickup(C) is executable
A
C
▪ precond: clear(C) & ontable(C)
▪ effect:
¬clear(C) & ¬ontable(C) & holding(C)
▪ result:
ontable(B), clear(B), holding(A), holding(C)
B
C
B
ƒ Oops…
▪ precond: holding(x) & clear(y)
▪ effect:
on(x,y) & clear(x) & ¬holding(x) & ¬clear(y)
▪ The robot should only be able to carry one block at a time!
jonkv@ida
34
2007-12-12
How to express that “the hand is free”?
How to express that the hand is free ?
ƒ With sufficient expressivity: !exists x. holding(x)
▪ But many classical planners cannot handle quantified preconditions
▪ Often hard to remedy: algorithms depend on having no quantifiers!
▪ “handempty” means the same as “!exists x. holding(x)”
▪ handempty holds in the initial state
Let’s test a candidate plan!
Let
s test a candidate plan!
ƒ Now unstack(A,C) is executable
ƒ stack(x,y) – put down block x on block y
ƒ Additional predicates
33
2007-12-12
▪ on(A,C), ontable(B), ontable(C), clear(A), clear(B)
ƒ Initial state
ƒ putdown(x) –
( ) put down a block on the table
ƒ
B
jonkv@ida
Blocks World Problem Instance
▪ precond: clear(x) & ontable(x)
▪ effect:
¬clear(x) & ¬ontable(x) & holding(x)
Expressivity
p
y
– block x is on block y
– block x is on the table
– block x can be picked up
– the robot’s hand is holding block x
A plan that uses
available actions
and resources to
achieve the goal
jonkv@ida
Blocks World Operators
p
on(x,y)
ontable(x)
clear(x)
holding(x)
jonkv@ida
Blocks World Operators, Again
p
, g
ƒ
Operators:
A
ƒ pickup(x) – pick up a block from the table
C
35
2007-12-12
jonkv@ida
Confusing Terminology
g
gy
ƒ
B
▪ precond: handempty & clear(x)
▪ effect:
!handempty & !clear(x) & holding(x)
ƒ pickup(x) – usually; generic, can be instantiated
ƒ
ƒ putdown(x) –
( ) put down a block on the table
▪ precond: holding(x)
▪ effect:
handempty & !holding(x) & ontable(x) & clear(x)
ƒ unstack(x,y) – pick up block x, which is now on block y
An operator is almost always a schema
An operator instance is always instantiated
ƒ pickup(A) – specific, no free variables
ƒ
An action is usually an operator instance
ƒ pickup(A) –
pickup(A) usually
▪ precond: handempty & on(x,y) & clear(x)
▪ effect:
!handempty & !on(x,y) & !clear(x) & holding(x) & clear(y)
ƒ pickup(x) – sometimes
ƒ stack(x,y) – put down block x on block y
▪ precond: holding(x) & clear(y)
▪ effect:
handempty & on(x,y) & clear(x) & !holding(x) & !clear(y)
ƒ
An action instance is always instantiated
ƒ pickup(A)
36
2007-12-12
jonkv@ida
Domain Definition Languages
g g
Domain Definition Languages
ƒ
38
2007-12-12
Need a formal language!
jonkv@ida
PDDL Predicates: BW example
p
ƒ
Domains are named in a LISP like definition
Domains are named in a LISP‐like definition
ƒ
ƒ Specify (declaratively!) what problem you want solved
From Semi‐Formal to Formal Specifications
▪ Language specifies syntax – how to write
▪ Language specifies semantics – exactly what it means
▪ Language specifies restrictions – what is allowed
ƒ
(define (domain blocksworld)
…
)
Predicates must be explicitly declared
ƒ (define (domain blocksworld)
ƒ STRIPS (Stanford Research Institute Problem Solver)
ƒ ADL: Action Definition Language (Pednault, 1989)
g g
9 9
▪ Considerably more expressive
)
ƒ Most common current language: PDDL
▪ Planning Domain Definition Language
C
▪ We will start with the STRIPS level of expressivity
ƒ
40
2007-12-12
A STRIPS level PDDL operator is described by:
A STRIPS‐level PDDL operator is described by:
ƒ
41
2007-12-12
Problem instance has a name belongs to a domain
Problem instance has a name, belongs to a domain
ƒ Effects: What is changed in the successor state
▪ Atoms made true (added) / made false (deleted).
A
2007-12-12
The goal is a set of atoms required to be true
ƒ Most planners handle “don’t care” goal incompleteness
▪ “A should be on top of B, but I don’t care where C is”:
Require on(A,B); don’t mention on(…,C), on(C,…), clear(C), …
▪ (Often cannot handle handle “A is on B or
A is on B or on C
on C”, “A is not
A is not on B
on B”))
ƒ Goal description may correspond to more than one state
A
A
B
B
C
C
{on(A,B), clear(A)}
Current world state: represented by true atoms
▪ Therefore, atoms that are not mentioned are assumed to be false
▪ No need to specify everything that does not hold!
▪ (define (problem ABC)
(d fi ( bl ABC)
(:domain blocksworld)
(:objects A B C)
(:init (on A C) (ontable C) (ontable B) (clear A) (clear B)
handempty)
…))
▪ (define (problem MyThreeBlockProblem)
p
y
(:domain blocksworld)
(:objects A B C)
…
)
C
A
B
C
B
jonkv@ida
PDDL Goals
ƒ
42
2007-12-12
ƒ Many planners need complete initial state information
jonkv@ida
▪ We don’t
W d ’ assume that unmentioned atoms must be false!
h i
d b f l !
ƒ
ƒ Objects must be explicitly declared
▪ (define (domain blocksworld)
(:predicates …)
(:action pick‐up
:parameters (?X)
:precondition (and (handempty) (ontable ?X) (clear ?X))
:effect (and (not (handempty)) (not (ontable ?X))
(not (clear ?X))
(holding ?X))))
ƒ
jonkv@ida
PDDL States: BW example
p
▪ (define (problem MyThreeBlockProblem)
(:domain blocksworld)
…
)
▪ A conjunction of (positive) atoms
PDDL Goals
B
jonkv@ida
PDDL States: BW example
p
ƒ Precondition: Must hold for an action to be applicable
43
Colon before many keywords,
in order to avoid collisions
when new keywords are added
A
Can be used to model STRIPS domains or more complex domains
▪ Can be used to model “STRIPS domains” or more complex domains
jonkv@ida
Variables prefixed with “?”
(
(:predicates
di t
((on ?X ?Y) (ontable ?X)
?X ?Y) (
bl ?X)
(clear ?X) (holding ?X) (handempty))
…
▪ One of the first planners (1969); quite restrictive input language
PDDL Actions: BW Example
p
39
2007-12-12
44
2007-12-12
Goal for our example instance
A
ƒ One possibility:
B
C
▪ (define (problem ABC)
(:domain blocksworld)
…
(:goal (and (on A B) (on B C) (ontable C) (clear A))))
ƒ Another possibility (the simplest specification):
▪ (define (problem ABC)
(:domain blocksworld)
…
(:goal (and (on A B) (on B C))))
▪ Note: (ontable C) is still guaranteed; where else would C be?
N
(
bl C) i ill d h l ld C b ?
▪ Note: (clear A) is still guaranteed; what could be on top of A?
jonkv@ida
Example Domain: Logistics
p
g
ƒ
Example 2: The logistics domain
ƒ Goal: A number of packages should be moved
f
from their initial to their final locations.
h i i i i l h i fi l l
i
ƒ Move packages within a city using trucks.
ƒ Move packages between cities using airplanes.
ƒ Packages can only be transferred between trucks and g
y
airplanes at certain airport locations.
45
2007-12-12
jonkv@ida
Example Domain: Logistics
p
g
46
2007-12-12
(define (domain logistics)
(:predicates
(incity ?L ?C) (at ?X ?L) (in ?P ?V))
jonkv@ida
Example Domain: Logistics
p
g
ƒ
2007-12-12
Solution 1: Type predicates
jonkv@ida
Explicit Types
p
yp
ƒ
(:requirements :typing)
(:types truck airplane – vehicle
airport – location package vehicle
location city
Problem: No type checking!
• load plane7 onto city5 at truck2
p
7
y5
• drive city9 from truck1 to airplane2
• Demented syntax:
subtype subtype … ‐
b
b
supertype
• All “top‐level” types must be declared last!
dec
a ed ast!
(:predicates (incity ?L – location ?C – city)
(at ?X – package ?L – location)
(in ?P – package ?V – vehicle))
(:action load
:parameters (?P – package ?V – vehicle ?L – location)
:precondition (and (at ?P ?L) (at ?V ?L))
:effect (and (in ?P ?V) (not (at ?P ?L)))
…)
(:action load
:parameters (?P ?V ?L)
p
(
)
:precondition (and (package ?P) (vehicle ?V) (location ?L)
(at ?P ?L) (at ?V ?L))
:effect (and (in ?P ?V) (not (at ?P ?L))))
ff t ( d (i ?P ?V) ( t ( t ?P ?L))))
…)
(:action drive‐truck
:parameters (?T ?L1 ?L2 ?C)
:precondition (and (incity ?L1 ?C) (incity ?L2 ?C) (at ?T ?L1))
:effect (and (not (at ?T ?L1)) (at ?T ?L2)))
48
2007-12-12
Solution 2: True types (modern planners)
ƒ (define (domain logistics)
(define (domain logistics)
(d
fi (d
i l i ti )
(:predicates
(
(incity ?L ?C) (at ?X ?L) (in ?P ?V)
y
)(
)(
)
(location ?L) (airport ?A) (vehicle ?V) (truck ?T)
(airplane ?P) (package ?P))
(:action load
:parameters (?P ?V ?L)
:precondition (and (at ?P ?L) (at ?V ?L))
:effect (and (in ?P ?V) (not (at ?P ?L)))
…)
47
jonkv@ida
Plan Structures
If that's the Problem
If that
s the Problem,
What s a Solution?
What's a Solution?
ƒ
50
2007-12-12
The choice of plan structure depends on:
jonkv@ida
The Linear Plan
ƒ
ƒ The abilities of the execution system
Plan Structures
51
2007-12-12
Linear (sequential) plans:
ƒ Just a simple sequence
▪ Can it perform many tasks at once, or only one?
ƒ The desired solution properties
▪ Is timing important?
unstack(A,C)
ƒ The abilities of the algorithms
putdown(A)
pickup(B)
stack(B,C)
▪ Most algorithms are tightly bound to
plan structures
▪ May settle for a less expressibe plan structure,
if this allows you to use an algorithm
which is better in other respects
jonkv@ida
The Linear Plan: Logistics
g
ƒ
A linear plan for logistics:
52
2007-12-12
jonkv@ida
The (Simple) Parallel Plan
(
p )
ƒ
Simple parallel plans (”Graphplan parallelism”):
Simple parallel plans (
Graphplan parallelism ):
53
2007-12-12
jonkv@ida
Temporal Plans
p
ƒ
54
2007-12-12
Temporal plans: Explicit timing
ƒ Nine actions in sequence; does not realize T1..3 are ƒ Actions within a step can be executed in any order,
ƒ Predict the amount of time required for each action
iindependent
d
d
ƒ This structure is more suitable for a single agent
or in parallel
i ll l
ƒ Must not start executing the next step
until all actions in the previous step have finished
ƒ Problem: Predictions may not be correct
load(P1,T1)
drive(T1,A,B)
unload(P1,T1)
load(P2,T2)
drive(T2,C,D)
unload(P2,T2)
load(P1,T1),
load(P2,T2),
load(P3,T3)
load(P3,T3)
drive(T3,E,F)
unload(P3,T3)
drive(T1,A,B),
drive(T2,C,D),
drive(T3,E,F)
▪ Is the plan still correct if drive(T1,A,B) ends at 420? At 620?
[0,10] load(P1,T1)
[10,570] drive(T1,A,B)
[570,580] unload(P1,T1)
[0,10] load(P2,T2)
[10,420] drive(T2,C,D)
[420,430] unload(P2,T2)
[0,10] load(P3,T3)
[10,845] drive(T3,E,F)
[845,855] unload(P3,T3)
unload(P1,T1),
unload(P2,T2),,
unload(P3,T3)
jonkv@ida
The Partial‐Order Plan
ƒ
55
2007-12-12
Partial order plans are less constrained
Partial‐order plans are less constrained
jonkv@ida
Further Plan Structures
ƒ
ƒ Can be executed in parallel or in sequence
56
2007-12-12
Other plan structures exist
Other plan structures exist...
ƒ ...but this is all we will cover in the course.
ƒ Quality metrics: number of actions, critical path length, …
UAV1: Take off
UAV2: Take off
UAV1: Move crate 1 to carrier 1
UAV2: Fly to carrier 1
II: Planning as Search
g
UAV1: Move crate 2 to carrier 1
UAV1: Move to pos7
UAV2: Pick up
p carrier 1
UAV1: Take picture
UAV1: Fly home
jonkv@ida
Achieving g Your Goals
ƒ
58
2007-12-12
How do we achieve the given goals?
jonkv@ida
Can We Recover?
ƒ
ƒ In general, can’t immediately see what actions are “good”
B
C
Therefore: Think before you act!
ƒ
B
B
C
ƒ Then begin executing it!
ƒ
ƒ Possibly heuristics / other means of improving efficiency
ƒ Sometimes actions are expensive or even irreversible
ƒ …and now B should be on C… oops!
▪ Crack an egg (can’t get it back)
C k ( ’t t it b k)
▪ Buy a car (can sell it, but probably for less money)
▪ Drive (fuel is consumed)
▪ Most dead ends are not this obvious, or cheap to correct
M d d d hi b i
h jonkv@ida
Progression Search (1)
g
( )
ƒ
61
2007-12-12
Search method #1: Progression (Forward‐chaining)
(Forward chaining)
jonkv@ida
Progression Search (2)
g
( )
ƒ
62
2007-12-12
Four potential ways of achieving the goal:
ƒ 1) We may already have achieved the goal (not now)
what actions are applicable,
h i
li bl
and where will they take me?”
ƒ Begin at the initial state
ƒ Search forward, hoping to find a goal state
ƒ 2) Unstack(A,C), then continue
ƒ 3) Pickup(B), then continue
3
p
ƒ 4) Pickup(D), then continue
Example:
p
A
C B D
Goal
A
B
A
C
B
D
C
D
jonkv@ida
Progression Search (3)
g
(3)
ƒ
ƒ “If I’m at this state now,
ƒ Initial state
A set of search states
A branching rule, which generates successors
A i i i l An initial state
A set of goal states, or a goal criterion identifying goal states
ƒ A search algorithm, specifying the search order
A h l i h if i h h d
But can’t you recover from executing bad actions?
ƒ Sometimes, yes
Sometimes yes
ƒ
Planning is often viewed as search which requires:
Planning is often viewed as search, which requires:
▪
▪
▪
▪
ƒ Verify that it will satisfy the goals
A
A
60
2007-12-12
ƒ A search space:
▪ Don’t execute – reason hypothetically
ƒ Example: A should be on B, so let’s start there…
C
2007-12-12
jonkv@ida
Planning as Search
g
ƒ Find (in some way) a complete plan
▪ Actions that seem good may lead to dead ends
A
59
Hypothetical reasoning: What if we unstack(A
C)?
Hypothetical reasoning: What if we unstack(A,C)?
▪
▪
▪
▪
▪
1) Already achieved the goal? No.
2) May stack(A,C) –
2) May stack(A
C) cycle.
cycle
3) May stack(A,B), continue
4) May stack(A,D), continue
5) May putdown(A), continue A
A
C B D
C B D
A
C
B
A
C B
D
D
A
B
C
63
2007-12-12
A
C B D
A
C
A
C B D
A
C B D
A
C B D
B
D
D
A
C B
D
C B D A
A
B
C
D
jonkv@ida
Progression Search (4)
g
(4)
ƒ
64
2007-12-12
Result: A search tree structure
jonkv@ida
Depth First
p
ƒ
65
2007-12-12
jonkv@ida
Breadth First
66
2007-12-12
ƒ Blind search: Breadth first
Remaining choice: search algorithm
ƒ Decides which actions to apply / simulate, in which order
ƒ Blind search: Depth first
cycle
G
cycle
G
G
▪ Each node is associated with a world state
▪ Branching rule: Given a node, apply an applicable action
B
hi l Gi d l li bl i
G
G
jonkv@ida
Why is Planning Hard?
y
g
ƒ
67
2007-12-12
Seems simple so why is planning hard?
Seems simple –
jonkv@ida
Why is Planning Hard?
y
g
ƒ
ƒ The number of states grows exponentially!
68
2007-12-12
A larger example:
▪
▪
▪
▪
▪
ƒ 3 trucks, 2 planes, 10 packages, 10 locations (2 airports):
each truck at one of 10 locations: 310
each plane at one of 2 locations: 22
each package in one of 5 vehicles or at one of 10 locations: 1015
A total of 2.36 * 1020 states
7 seconds/year Î
Search 10
h 9 nodes per second, 3*10
d
d
d
Î
7500 years to cover the entire search space
ƒ
ƒ
70
2007-12-12
▪ Blocks: Easy to see if a block must still be moved; count them!
▪ Logistics: Number of packages not yet at their destinations
ƒ Domain‐independent: Used for all domains
▪ Based on an automated analysis of the problem structure,
the operators and preconditions, and so on
ƒ Usually not perfect (can’t always show the best way to go)
ƒ A*, hill climbing, many specialized methods
The search space contains “stupid” sequences
Blocks world: Unstacking towers that are already “finished”
Bl k Blocks world: Putting blocks on towers that will need to be removed
ld P i bl k h ill d b d
Logistics: Moving packages that are at their destinations
…
Blind search is not goal‐directed!
ƒ Must try to find the best actions to apply
▪ Best = most likely to lead towards the goal
B lik l l d d h l
▪ Not always as obvious as in these examples…
jonkv@ida
Heuristic Search (2)
( )
ƒ
71
2007-12-12
A domain independent heuristic:
A domain‐independent
▪ Note: Only local information used
‐ on(A,C)
‐ clear(A)
‐ clear(C)
+ holding(A)
+ handempty
ontable(C)
on(A,C)
on(C,A)
on(A,B)
clear(A)
clear(B)
clear(C)
A
C B
C B
8
A
C
B
‐ clear(B)
+ ontable(B)
( )
+ holding(B)
+ handempty
72
2007-12-12
The perfect solution?
ƒ No! Need much more sophisticated heuristics…
‐ on(A,B)
A
C B
‐ handempty
h d
+ clear(A)
7 ‐holding(A)
C B A
10
Heuristic Search (3)
(3)
ƒ Can still fail…
5 ‐ holding(A)
7
A
jonkv@ida
ƒ
ƒ Count the number of predicates with the “wrong” value
ƒ Domain‐dependent: Developed for a specific domain
…together with heuristic search methods
▪
▪
▪
▪
▪
▪
▪ Use every particle in the known universe as a computer (1080),
assume every particle can search 1020 nodes per second,
nodes per second
Î 10392 years
ƒ For each node, estimates the cost to reach the goal
ƒ
With these numbers can we do planning at all?
With these numbers, can we do planning at all?
ƒ What about more computers?
Wh t b t t ?
One approach: heuristic functions
▪ Example: Can’t always choose the optimal block to unstack
▪ Would be equivalent to solving the original planning problem!
69
2007-12-12
ƒ Yes – as long as we don’t do blind search!
each truck at one of 60 locations: 3060
each plane at one of 10 locations: 1010
each package in one of 40 vehicles or at one of 60 locations: 100
h k i f hi l f 6 l
i
200
A total of approximately 10500 states
S
Search 10
h 9 nodes per second, < 10
d d 8 seconds per year Î
d Î 10483 years
jonkv@ida
Heuristic Search (1)
( )
jonkv@ida
Can Planning be Done?
g
ƒ 30 trucks, 10 planes, 200 packages, 60 locations (10 ap):
▪ Blind search only works for tiny problem instances
▪
▪
▪
▪
▪
G
▪ Same search tree
▪ Different search order
▪ Can also use iterative deepening and other blind search methods
‐ handempty
py
+ clear(A)
+ ontable(A)
C
A
B
ontable(B)
on(A,B)
on(A,C)
on(B,C)
clear(B)
A
C B D
8
A
C B D
5
A
C
5
B
A
C B
D
D
9
A
C
B
A
C
B
D
6
6
D
A
B
C
D
jonkv@ida
Heuristic Search (4)
(4)
ƒ
73
2007-12-12
Domain dependent heuristic: More knowledge
Domain‐dependent
jonkv@ida
Domain‐Dependent Guidance
p
ƒ
▪ Logistics: Analyze the (metric) goal distances for packages
▪ Blocks: 2 * the number of Bl k * th b f “unheld” blocks that must be moved
h ld” bl k th t t b d
+ (if holding a block: 1)
+ (if holding a block whose destination is not ready,
(
g
y,
and moving it does not free the destination of another block: 2)
A
Must move A, B
2*2 + 0+0 = 4
C B D
A
C B D
A
C
B
A
C B
D
D
ƒ
75
2007-12-12
Can also use other search spaces
ƒ Regression, partial order, etc
ƒ All strategies / search spaces have their own strengths and weaknesses
ƒ Sometimes a small set of common sense rules will suffice
A
C B D
Must move A
2*1+ 1+2 = 5
A
C
B
A
C B
D
D
D
Moves a block that had to be moved,,
in order to put something else on C.
GOOD.
B did not block anything; its destination
is not yet ready. BAD.
A
B
C
As above. BAD.
jonkv@ida
ƒ
Avoiding Blind Search
g
▪ Don't unload packages before you reach the destination
▪ Don't drive to a destination where there's nothing to pick up
A
Regression Search (1)
g
( )
jonkv@ida
ƒ Rules that help the planner recognize "stupid" actions
C B D
A
B
C
2007-12-12
Other forms of domain specific guidance:
Other forms of domain‐specific
Must move B
2*1 + 1+0 = 3
Must move A, B
,
2*2 + 1+2 = 7
74
76
2007-12-12
Search method #2: Regression
D
jonkv@ida
Regression Search (2)
g
( )
ƒ
77
2007-12-12
jonkv@ida
Regression Search (3)
g
(3)
ƒ
Three potential ways of achieving the goal:
ƒ “If this is the goal,
ƒ 1) It might already have been achieved – not in this case
what action could I use to achieve part of the goal,
h i ld I hi f h l
and what would I have to do before that?”
ƒ Begin at the goal
ƒ Search backwards, hoping to find the initial state
ƒ 2) The last action was stack(A,B)
How may we achieve the state where we’re holding A?
How may we achieve the state where we
re holding A?
ƒ pickup(A)
ƒ unstack(A
D)
unstack(A,D)
ƒ unstack(A,B)
▪ Cycle!
y
▪ Before this, we’d have the same situation,
except A would be held.
ld b h ld
78
2007-12-12
B
C D A
ƒ 3) The last action was putdown(D)
▪ Before, D would have been held.
ƒ
ƒ Initial state
B
C
Goal
D
A
B
A
C
B
B
C
A
Example:
p
D
C
A
C B D
D
A
B
C
D
A
B
C
D
A
C B D
A
B
C
A
D
D
A
B
C
A
B
C
D
D
A
B
C
D
jonkv@ida
Search Spaces for Planning
p
g
ƒ
Other search spaces:
ƒ Parallel progression and regression
▪ Start from both ends; hope to meet in the middle
ƒ Partial‐order planning
p
g
▪ Don’t commit to placing A before B,
if the actions can be invoked in any order
y
(don’t care whether truck1 or truck2 drives first!)
ƒ Encoding: Transform into another problem, e.g. SAT
ƒ Iterative repair: "debug" complete plan
ƒ …
ƒ Consult the course book for more information!
jonkv@ida
ADL example 1: Quantifiers
p
79
2007-12-12
Extensions to STRIPS
ƒ
81
2007-12-12
Why is there a “clear” predicate in STRIPS
Why is there a clear predicate in STRIPS‐BW?
BW?
ƒ Can pick up x if nothing is on top of it.
▪ Precondition (not (exists ?Y) (on (?Y ?X))) – impossible in STRIPS
▪ No quantifiers allowed!
ƒ Must add the “clear” predicate to keep track
“
of this state!
▪ Provide a definition of clear() in the initial state
▪ Make clear(x) false when you put something on x
▪ Make clear(x) true when you remove something from x
ƒ ADL and many modern planners allow quantifiers
▪ The “clear” predicate is not strictly needed
▪ Still often used, due to tradition or readability –
or possibly works better with some heuristic functions…
ibl k b tt ith h i ti f
ti
jonkv@ida
ADL Example 2
p
ƒ
82
2007-12-12
Why different ”pickup” and ”unstack” operators?
Why different pickup and unstack operators?
jonkv@ida
ADL in PDDL
ƒ
ƒ Suppose you only had a single “grab(x)” operator
ƒ Quantified and conditional effects.
ƒ
Must specify that you want ADL extensions
▪ ((define ((domain blocksworld))
(:requirements :adl) … … )
▪ Precondition on(x,y), so you know x is
P
di i ( ) k
i on top of another block
f h bl k
▪ Only executable if for the correct y, so you know which block
▪ Effect not on(
Effect not on(x,y), so the intended instance of ) so the intended instance of “on” becomes false
on becomes false
Eff
(f ll (?Y) ((when
h ((on ?X ?Y) ((not ((on ?X ?Y))))
Effect: (forall
(when initial-condition effect)
I iti l
Initial‐condition evaluated before execution
diti l t d b f ti
Effect becomes true after evaluation
Example Domains and Complicating Factors
ƒ Quantified preconditions and goals,
ƒ STRIPS requires a separate unstack(x,y) operator
▪
▪
▪
▪
III: Real‐World Planning
l
ld l
i
PDDL supports ADL style extensions
PDDL supports ADL‐style extensions
ƒ Negation and disjunction in preconditions and goals,
▪ What you want to say is: if x is initially on top of some block,
then x will no longer be on top of that block after grab(x).
▪ There is no way of expressing this!
ƒ Modeling grab(x) in ADL:
83
2007-12-12
A
B
C
jonkv@ida
Reconfiguration in a Chemical Processing Plant
86
Ex: Planning Satellite Operations
g
p
Reconfigure plant, eco gu e p a t,
Sc
Schedule observations, edu e obse at o s,
e.g. from standby mode to running, in g,
short time.
Most change caused g
by processes running over time.
Simple actions (e.g.
opening a valve) have complex effects.
maintenance activities etc.
Resources very tight
y g
Pre‐scheduled events enable/disable actions
Complex goal priorities
Goals and resources change unpredictably
jonkv@ida
Ex: Supply Restoration
pp y
Power network failure
Locate fault and reconfigure network.
Very high uncertainty about the initial state; a "trial‐and‐
error" approach is necessary.
Sensors and actions are also unreliable.
unreliable
jonkv@ida
Time and Resources
Time and Resources
ƒ
89
2007-12-12
Actions have different duration
jonkv@ida
Time and Resources
ƒ
Actions require resources which are limited
Actions require resources, which are limited
ƒ Planners must take this into account,
ƒ Planners must take this into account,
or risk generating plans of low quality
i k i l f l li
ƒ Many packages can be loaded while flying a plane,
not just one
ƒ Sometimes optimizing on resource usage is desirable
(load package1 truck2)
(fly‐airplane plane7 CPH SEA‐TAC)
5:30 am
87
4:00 pm
or risk generating plans that fail due to lack of resources
i k i l h f il d l k f 90
2007-12-12
jonkv@ida
Time and Resources
ƒ
91
2007-12-12
Events happen independently at certain times
jonkv@ida
Time and Resources
ƒ
ƒ Must be able to model this to create plans for such 92
2007-12-12
Uncertainty
Goals may have deadlines
ƒ To model this, planners need support for:
d
domains
i
▪ Time and action durations
▪ Time‐limited goals
ƒ To generate plans efficiently:
▪ Heuristics that ensure the planner backtracks
when achieving a goal before its deadline is hopeless
h hi i l b f i d dli i h
l
jonkv@ida
Uncertainty in Planning
y
g
ƒ
ƒ
94
2007-12-12
Classical planning assumes certainty / determinism
jonkv@ida
Conformant Plans
ƒ
95
2007-12-12
Sometimes knowing the exact state is not essential
Sometimes, knowing the exact state is not essential
ƒ For the initial state
ƒ In these cases, conformant planners can be used
ƒ For action effects
ƒ Apply only actions whose preconditions are definitely true
Sometimes this constraint must be relaxed
jonkv@ida
Contingent / Conditional Plans
g
/
ƒ
▪ Must reason about what we will know later
ƒ Can take results into account in sequential plans
▪ Skip actions whose preconditions are false or unknown
▪ Open a safe: Read combination from note; dial what you read
ƒ
▪ Initial state: List true atoms and false atoms; the rest are unknown
;
▪ Alternatively: Support arbitrary formulas, such as (or a b)
▪ …
ƒ More expressive: Conditional plans
Example: VCR domain
p
ƒ Goal: Tape at 00:20
lookat(B)
ƒ Operators: ▪ Effect ranges: after moveto(10,20), we have x in [9,11], y in [19,21]
g
( , ),
[9, ], y [ 9, ]
▪ Probabilities: start‐car() results in a running engine 97% of the time
▪ …
▪ G
Go to position n (must know current position);
t iti (
t k
t iti )
rewind completely (counter := 0).
jonkv@ida
97
2007-12-12
Actions may fail
May have additional unintended consequences
C
¬clear(B)?
)?
unstack(B,D)
jonkv@ida
Action Failure (2)
( )
ƒ
ƒ
98
2007-12-12
Information about the initial state may be false
Other agents may be changing the world
X
B
clear(B)?
ƒ Plan: Rewind completely; wind to 00:20
ƒ We still have well‐defined constraints, though relaxed!
ƒ
▪ If‐then conditions; possibly loops
▪ Example: We know on(B,D); we don’t know whether clear(B)
ƒ Initial state: Don’t know the current tape position
ƒ Action effects not always completely determined
ƒ
Planners may support sensory actions
ƒ The result will only be known at execution time
ƒ Initial state not always completely determined
Action Failure
96
2007-12-12
A
A
B
B
C
C
jonkv@ida
Approaches to Handling Failure
pp
g
ƒ
Partial solution: Execution Monitoring
ƒ Monitor what happens – completely or partially
ƒ Compare with what should happen
ƒ Signal failures and attempt recovery
g
p
y
▪ Try the same action again
▪ Plan repair or complete replanning given new information
▪ …
99
2007-12-12
jonkv@ida
TALplanner
p
TALplanner
ƒ
101
2007-12-12
TALplanner: A planner developed at IDA
jonkv@ida
TALplanner Example Domain
p
p
ƒ
ƒ Forward‐chaining and uses depth‐first search
▪
▪
▪
▪
102
2007-12-12
Example Domain: ZenoTravel
ƒ Planes move people between cities (board, debark, fly)
Allows us to use highly expressive models
Arbitrary preconditions
C di i
Conditional effects
l ff
Timing, concurrency and resources
ƒ Planes have limited fuel level; must refuel
ƒ Example instance:
p
▪ 70 people
▪ 35 planes
▪ 30 cities
ƒ Depth first requires guidance and goal‐directedness
D h fi i id
d l di
d
▪ No built‐in heuristics
▪ Uses domain‐dependent rules
d
d
d
l to guide towards the goal
d
d h
l
▪ Can also specify domain‐dependent heuristics
ƒ Did well in international competitions
Did ll i i
i
l ii
jonkv@ida
ZenoTravel Problem Instance
ƒ
103
2007-12-12
A smaller problem instance
jonkv@ida
What Just Happened?
pp
ƒ
104
2007-12-12
No additional domain knowledge specified yet!
ƒ Pure depth first…
jonkv@ida
Control Rules
ƒ
initial node
TALplanner uses control rules
ƒ The plan generates a sequence of states
105
2007-12-12
at(p1 city4)
at(p1, city4)
at(p2, city4)
▪ Plan:
step 1: board(p1, plane1, city4),
step 2: board(p2, plane1, city4)
▪ States: time 0: initial state
time 1: after p1 boarding
time 2: after p2 boarding
p
g
board(p1, plane1, city4)
in(p1, plane1)
at(p2 city4)
at(p2, city4)
ƒ Control rules are formulas
that must hold in this state sequence
q
ƒ Rules can also refer to the goal
▪ goal(f) true if f must hold in all goal states
one of the
g
goal nodes
jonkv@ida
Multiple Uses of Control Rules
p
ƒ
106
2007-12-12
Control rules have multiple uses
ƒ Prune parts of the tree that do not contain solutions
▪ No need to investigate dead ends
▪ Some domains, like ZenoTravel, have no dead ends
ƒ Prune parts of the tree that contain “bad” solutions
Prune parts of the tree that contain bad solutions
▪ Plans that achieve the goal, but are unnecessarily expensive
▪ Applicable to this domain!
pp
ƒ Prune parts of the tree that contain redundant solutions
▪ In optimizing planning, we don’t stop when the first plan is found
p
gp
g
p
p
▪ No point in visiting many plans with equal cost
b d( l
board(p2, plane1, city4)
it )
in(p1, plane1)
in(p2, plane1)
jonkv@ida
Control Rules
ƒ
107
2007-12-12
First problem in the example:
jonkv@ida
Control Rules
ƒ
108
2007-12-12
Second problem in the example:
ƒ Passengers debark whenever possible.
ƒ Passengers board planes, even at their destinations
ƒ Rule: "At any timepoint, if a passenger debarks, he is at his ƒ Rule: "At any timepoint, if a passenger boards a plane, he goal.”
was not at his destination.”
ƒ #control :name "only‐board‐when‐necessary"
ƒ #control :name "only‐debark‐when‐in‐goal‐city"
y
g
y
forall t, person, aircraft [
[ ] (p
[t] in(person, aircraft) →
f)
[t+1] in(person, aircraft) exists city [
[t] at(aircraft, city) goal(at(person, city)) ] ]
forall t, person, aircraft [
([t] !in(person, aircraft) [t+1] in(person, aircraft)) →
exists city1, city2 [
[t] at(person, city1) goal(at(person, city2)) l( (
)) city1 != city2 ] ]
jonkv@ida
Zeno Travel, second attempt
,
p
109
2007-12-12
jonkv@ida
What's Wrong This Time?
g
ƒ
ƒ
110
2007-12-12
forall t, aircraft, city [
[t] at(aircraft, city) →
([ ] (
([t+1] at(aircraft, city)) |
f
)) |
exists city2 [
city2 != city &
([t+1] at(aircraft, city2)) &
[t] reasonable‐destination(aircraft, city2) ]]
ƒ #define [t] reasonable‐destination(aircraft, city):
[t] has‐passenger‐for(aircraft, city) |
exists person [
[t] at(person, city) &
[t] in‐wrong‐city(person) ] |
g yp
|
goal(at(aircraft, city)) &
[t] empty(aircraft) &
[t] all persons at their destinations or in planes ]
[t] all‐persons‐at‐their‐destinations‐or‐in‐planes ]
ƒ 1. A passenger’s destination
1. A passenger s destination
ƒ 2. A place where a person wants to leave
ƒ 3. The airplane
3 The airplane’s destination
s destination
jonkv@ida
112
2007-12-12
jonkv@ida
Positive Side Effects
ƒ
113
2007-12-12
Positive effects of using domain knowledge:
ƒ Better performance than domain‐independent heuristics
▪ When you
When you’ve found the right control rules!
ve found the right control rules!
ƒ Can use more detailed domain models
▪ With control rules, this has a smaller impact on performance
With control rules this has a smaller impact on performance
ƒ Can generate better plans
▪ Can prune non
Can prune non‐solutions
solutions, but also bad
but also bad solutions
ƒ
Negative effects of using domain knowledge:
ƒ Requires more knowledge
ƒ Requires more domain engineering
ƒ Rules are sometimes easy to find, sometimes difficult
Rules are sometimes easy to find sometimes difficult
ƒ
As always: A tradeoff!
Control Rules
ƒ #control :name "planes‐always‐fly‐to‐goal“
#control :name planes always fly to goal
Only constrained passengers
Forgot to constrain airplanes
ƒ Which cities are reasonable destinations?
Zeno Travel, third attempt
,
p
jonkv@ida
111
2007-12-12