AT&T Labs
Cornell University

Focus: Classical state-space planning, extended with parallel actions
Core problem - similar computational issues arise in many other models
• Reactive plans
• Planning with uncertainty and utilities
• Continuous processes
• Metric time
Many results directly extend to richer formalisms
2

Find a sequence of operators that transform an initial state to a goal state
State = complete truth assignment to a set of variables (fluents)
Goal = partial truth assignment (set of states)
Operator = a partial function State

State
• specified by three sets of variables: precondition, add list, delete list
3

Operators may be applied in parallel when all orderings are well defined and equivalent
(Op1 || Op2)(s) = Op2(Op1(s)) = Op1(Op2(s))
A special form of non-linear plans
• Only allows parallel actions, not parallel action sequences
• Easy to serialize
4

I. Domain-independent planning: PSPACEcomplete or worse
(Chapman 1987; Bylander 1991; Backstrom 1993)
II. Domain-dependent planning: NP-complete or worse
(Chenoweth 1991; Gupta and Nau 1992)
III. Approximate planning: NP-complete or worse
(Selman 1994)
5

Domain-independent planning systems can only generate relatively short plans.
Practical systems minimize or eliminate search by employing
• Search control rules , hand-tailored to the search engine and the particular search space
(Sacerdoti 1975, Slaney 1996, Bacchus 1996)
• Pre-compiling entire state-space to a reactive finite-state machine
(Agre & Chapman 1997, Williams & Nayak 1997)
Scaling remains problematic when state space is large or not well understood!
6

• Planning as first-order theorem proving
(Green 1969) computationally infeasible
• STRIPS
(Fikes & Nilsson 1971) very hard
• Partial-order planning
(modal truth criteria)
(Tate 1977, Chapman 1985, McAllester 1991, Smith & Peot 1993) can be more efficient, but still hard
(Minton, Bresina, & Drummond 1994)
• SATPLAN: planning as propositional reasoning
7

SAT encodings are designed so that plans correspond to satisfying assignments
Use recent efficient satisfiability procedures
(systematic and stochastic) to solve
Evaluation performance on benchmark instances
8

problem description axiom schemas instantiate length mapping instantiated propositional clauses plan interpret satisfying model
SAT engine(s)
(New implementation blackbox takes STRIPS input - these experiments based on older handcreated “state-based” encodings)
9

Propositional CNF: no variables or quantifiers
Sets of clauses specified by axiom schemas
• fully instantiated before problem-solving
Discrete time, modeled by integers
• state predicates : indexed by time at which they hold
• action predicates : indexed by time at which action begins
– each action takes 1 time step
– many actions may occur at the same step fly(Plane, City1, City2, i)
 at(Plane, City2, i +1)
10

A solution is specified by any model (satisfying truth assignment) of the conjunction of the axioms describing the initial state, goal state, and operators
Easy to convert back to a STRIPS-style plan
11

Systematic, complete procedures
• Depth-first backtrack search
(Davis, Putnam, & Loveland 1961)
– unit propagation, shortest clause heuristic
• One of the best implementations: ntab
(Crawford & Auton 1997)
– see also: satz (Li), CSAT (Dubois), MODOC (van Gelder)
Stochastic, incomplete procedures
• Inspired by Mins-Conflict algorithm
(Adorf & Johnson 1990; Minton, Johnson, Philips, & Laird
1992)
• GSAT (Selman et. al 1993)
• Current fastest: Walksat (Selman & Kautz 1993)
– greedy local search + noise to escape local minima
12

Extension of Graphplan test set blocks world - up to 18 blocks, 10 19 states logistics - complex, highly-parallel transportation domain.
Logistics.d:
• 2,165 possible actions per time slot
• 10 16 legal configurations (2 2000 states)
• optimal solution contains 74 distinct actions over
14 time slots
Problems of this size never previously handled by state-space planning systems
13

100000
10000
1000
100
10
1
0.1
0.01
rocket.a rocket.b log.a log.b log.c log.d
Graphplan ntab/STRIPS w alksat/STRIPS ntab/STATE w alksat/STATE
14

More flexible than forward or backward chaining
• Systematic : most unit propagation at most highly constrained states
• Stochastic : iterative repair
Space for time tradeoff
• Less overhead since does not have to instantiate variable during search
Randomized algorithms less likely to get trapped along bad paths
15

Efficient representations and (randomized) SAT engines extend the range of domainindependent planning
Ways to further improvement :
• Better general search algorithms
• Incorporate (more) domain dependent knowledge
16

* About domain itself
• a truck is only in one location
• airplanes are always at some airport
* About good plans
• do not remove a package from its destination location
• do not unload a package and immediate load it again
X About how to search
• plan air routes before land routes
• work on hardest goals first
17

Such information is traditionally incorporated in the planning algorithm itself
– or in a special programming language
Instead: we propose expressing as additional declarative axioms
• Problem instance: operator axioms + initial and goal axioms + heuristic axioms
• Domain knowledge
 constraints on search and solution spaces
• Independent of any search engine strategy
NOT logic programming!
18

1. Entailed by operator axioms: conflicts and derived effects fly(plane,d1,i) and fly(plane,d2,i) conflict
2. Entailed by operators + initial state axioms: state invariants a truck is at only one location
3. Entailed by operators + initial + goal + length: optimality conditions do not return a package to a location
4. New constraints on problem instance: simplifying assumptions
Once a truck is loaded, it should immediately move
19

Invariant: A truck is at only one location at(truck,loc1,i) & loc1
 loc2

 at(truck,loc2,i)
Optimality: Do not return a package to a location at(pkg,loc,i) &
 at(pkg,loc,i+1) & i<j

 at(pkg,loc,j)
Simplifying: Once a truck is loaded, it should immediately move
 in(pkg,truck,i) & in(pkg,truck,i+1) & at(truck,loc,i+1)

 at(truck,loc,i+2)
20

Does it work?
• Additional axioms might just blow up instance with redundant information
Is effect independent of search engine?
• Simplifying assumptions might hurt stochastic methods by reducing solution density
Can we predict the most useful level of heuristic axioms?
• What is relation of difficulty to problem size?
21

Observation: Original BW examples for SATPLAN included some state invariant axioms
• After simplification by unit propagation, invariant axioms make instance smaller and easier to solve
(Ernst, Millstein, Weld 1997)
Experiment: 3 versions of each instance
• minimal set of axioms
• + state invariants
• + optimality (“don’t move a block twice in a row”) bw.b.ops
bw.b.state
bw.b.heur
bw.d.ops
bw.d.state
bw.d.heur
vars walksat ntab
1635
1865
1876
13.08
2.22
0.75
5.13
0.66
0.38
8282 >48 hr >48 hr
9023 688 >48 hr
9042 643 2651
22

h1: Optimality conditions
• Once a package leaves a location, it never returns h2, h3: Simplifying assumptions
• A package is never in any city other than its origin or destination cities
– rules out solutions where packages are transferred between airplanes in an intermediate city
• Once a vehicle is loaded, it should immediately move
– rules out solutions where vehicles are loaded incrementally h4: More optimality conditions
• A package never leaves its destination city
23

1200
1100
1000
900
800
700
600
500 none h1 h2 h3 h4 log.a
log.b
log.c
24

16000
14000
12000
10000
8000
6000
4000 none
h1 h2 h3 h4 log.a
log.b
log.c
25

8
7
6
5
4
3
2
1
0 none h1 h2 h3 h4 log.a
log.b
log.c
log.d
26

100000
10000
1000
100
10
1
0.1
none h1 h2 h3 h4 log.a
log.b
log.c
27

Does it work?
• YES, 10 to 100+ times speedup
Is effect independent of search engine?
• YES, same heuristics best for systematic and stochastic engines - but needs more investigation
Can we predict the most useful level of heuristic axioms?
• USUALLY point at which problem size is minimized after simplification by unit propagation
(40% - 70% reduction)
• BUT larger simplified instances still easier than
“no heuristic” case
28

Introspection
• Try to capture “obvious” inferences that are hard to deduce (McAllester)
Compilation
• Fix operators and initial or goal state, generate tractable equivalent theory (Kautz & Selman)
• In worse case, tractable theory is exponentially large but can be efficient in practice
EBL
(Minton, Kambhampati)
Static analysis
(Smith, Etzioni, Knoblock, Peot)
• How to adapt to generate constraints on plans instead of explicit control rules?
29

Easy to encode domain-specific knowledge in the planning as satisfiablity frame
• Key to next order-of-magnitude scaling
Purely declarative assertions
• Not logic programming!
Makes clear logical status of different heuristics
• invariants, optimality, simplifying
• entailed
 admissible
Heuristics are independent from the SAT engine
• Can use same axioms for radically different problem solvers
30

Dramatic speedup possible
• Stochastic: speedup 6X - 10X
• Systematic: speedup 100X +
For largest, hardest problems, additional domain knowledge allows systematic solvers to become competitive
• but in some domains stochastic solvers still show better scaling
31

100000
10000
1000
100
10
1
0.1
0.01
log.a
log.b
log.c
log.d
w alksat ntab satz
32

Automatic generation of axioms from STRIPS input
• efficient interleaving instantiation and pruning
(Kambhampati 1997)
Randomized systematic search
• broad coverage across domains
• robust against instances that are hard for deterministic strategies (“heavy tails”)
(Gomes,
Selman, Kautz 1997) blackbox
• adapts graphplan for generating wffs
• randomized version of satz (Li 1997)
• now solves log.d in 1 minute directly from STRIPS input, without hand-encodings or domain knowledge!
33

blackbox version 9B command line: blackbox -o logistics.pddl -f logistics_prob_d_len.pddl
-solver compact -l -then satz -cutoff 25 -restart 10
----------------------------------------------------
Converting graph to wff
6151 variables
243652 clauses
Invoking simplifier compact
Variables undetermined: 4633
Non-unary clauses output: 139866
----------------------------------------------------
Invoking solver satz version satz-rand-2.1
Wff loaded
[1] begin restart
[1] reached cutoff 25 --- back to root
[2] begin restart
[2] reached cutoff 25 --- back to root
[3] begin restart
[3] reached cutoff 25 --- back to root
[4] begin restart
[4] reached cutoff 25 --- back to root
[5] begin restart
**** the instance is satisfiable *****
**** verification of solution is OK **** total elapsed seconds = 25.930000
----------------------------------------------------
Begin plan
1 drive-truck_ny-truck_ny-central_ny-po_ny
34

Begin plan
1 drive-truck_ny-truck_ny-central_ny-po_ny
1 drive-truck_sf-truck_sf-airport_sf-po_sf
1 load-truck_package5_bos-truck_bos-po
1 drive-truck_pgh-truck_pgh-airport_pgh-central_pgh
1 fly-airplane_airplane2_pgh-airport_sf-airport
1 load-truck_package6_bos-truck_bos-po
2 load-truck_package2_pgh-truck_pgh-central
2 load-truck_package4_ny-truck_ny-po
2 load-truck_package7_ny-truck_ny-po
2 load-truck_package3_pgh-truck_pgh-central
2 drive-truck_bos-truck_bos-po_bos-airport_bos
2 load-airplane_package8_airplane2_sf-airport
2 fly-airplane_airplane1_pgh-airport_sf-airport
2 drive-truck_la-truck_la-po_la-airport_la
3 fly-airplane_airplane2_sf-airport_bos-airport
3 unload-truck_package6_bos-truck_bos-airport
3 drive-truck_pgh-truck_pgh-central_pgh-airport_pgh
3 fly-airplane_airplane1_sf-airport_pgh-airport
3 unload-truck_package5_bos-truck_bos-airport
3 drive-truck_ny-truck_ny-po_ny-airport_ny
3 drive-truck_sf-truck_sf-po_sf-airport_sf
4 unload-truck_package3_pgh-truck_pgh-airport
4 unload-truck_package2_pgh-truck_pgh-airport
4 unload-truck_package4_ny-truck_ny-airport
4 load-airplane_package6_airplane2_bos-airport
4 load-airplane_package5_airplane2_bos-airport
4 drive-truck_la-truck_la-airport_la-po_la
4 drive-truck_bos-truck_bos-airport_bos-central_bos
4 unload-truck_package7_ny-truck_ny-airport
5 drive-truck_ny-truck_ny-airport_ny-po_ny
5 drive-truck_bos-truck_bos-central_bos-po_bos
5 load-airplane_package2_airplane1_pgh-airport
5 drive-truck_la-truck_la-po_la-central_la
5 drive-truck_pgh-truck_pgh-airport_pgh-po_pgh
5 load-airplane_package3_airplane1_pgh-airport
5 fly-airplane_airplane2_bos-airport_ny-airport
6 drive-truck_sf-truck_sf-airport_sf-central_sf
6 unload-airplane_package6_airplane2_ny-airport
6 load-airplane_package4_airplane2_ny-airport
6 drive-truck_la-truck_la-central_la-po_la
6 drive-truck_bos-truck_bos-po_bos-airport_bos
6 load-airplane_package7_airplane2_ny-airport
6 drive-truck_ny-truck_ny-po_ny-airport_ny
6 unload-airplane_package8_airplane2_ny-airport
6 fly-airplane_airplane1_pgh-airport_sf-airport
6 load-truck_package1_pgh-truck_pgh-po
7 fly-airplane_airplane2_ny-airport_la-airport
7 fly-airplane_airplane1_sf-airport_bos-airport
7 load-truck_package9_sf-truck_sf-central
7 load-truck_package6_ny-truck_ny-airport
7 drive-truck_bos-truck_bos-airport_bos-central_bos
7 drive-truck_pgh-truck_pgh-po_pgh-airport_pgh
7 load-truck_package8_ny-truck_ny-airport
8 drive-truck_sf-truck_sf-central_sf-po_sf
8 fly-airplane_airplane2_la-airport_pgh-airport
8 unload-truck_package1_pgh-truck_pgh-airport
8 drive-truck_bos-truck_bos-central_bos-po_bos
8 drive-truck_ny-truck_ny-airport_ny-central_ny
8 fly-airplane_airplane1_bos-airport_la-airport
8 drive-truck_la-truck_la-po_la-airport_la
9 unload-airplane_package7_airplane2_pgh-airport
9 unload-truck_package8_ny-truck_ny-central
9 unload-airplane_package5_airplane2_pgh-airport
9 unload-truck_package9_sf-truck_sf-po
9 unload-airplane_package3_airplane1_la-airport
9 unload-truck_package6_ny-truck_ny-central
9 drive-truck_pgh-truck_pgh-airport_pgh-po_pgh
9 load-airplane_package1_airplane2_pgh-airport
10 drive-truck_ny-truck_ny-central_ny-po_ny
10 fly-airplane_airplane2_pgh-airport_bos-airport
10 load-truck_package3_la-truck_la-airport
10 fly-airplane_airplane1_la-airport_ny-airport
10 drive-truck_pgh-truck_pgh-po_pgh-airport_pgh
11 drive-truck_bos-truck_bos-po_bos-airport_bos
11 drive-truck_ny-truck_ny-po_ny-airport_ny
11 unload-airplane_package2_airplane1_ny-airport
11 drive-truck_la-truck_la-airport_la-central_la
11 drive-truck_sf-truck_sf-po_sf-airport_sf
11 unload-airplane_package1_airplane2_bos-airport
11 load-truck_package7_pgh-truck_pgh-airport
11 load-truck_package5_pgh-truck_pgh-airport
12 drive-truck_sf-truck_sf-airport_sf-po_sf
12 load-truck_package1_bos-truck_bos-airport
12 fly-airplane_airplane2_bos-airport_la-airport
12 load-truck_package2_ny-truck_ny-airport
12 fly-airplane_airplane1_ny-airport_pgh-airport
12 drive-truck_pgh-truck_pgh-airport_pgh-po_pgh
12 unload-truck_package3_la-truck_la-central
13 drive-truck_ny-truck_ny-airport_ny-po_ny
13 load-truck_package3_la-truck_la-central
13 load-truck_package9_sf-truck_sf-po
13 drive-truck_bos-truck_bos-airport_bos-po_bos
13 unload-truck_package5_pgh-truck_pgh-po
13 unload-airplane_package4_airplane2_la-airport
14 unload-truck_package9_sf-truck_sf-po
14 unload-truck_package1_bos-truck_bos-po
14 unload-truck_package7_pgh-truck_pgh-po
14 unload-truck_package2_ny-truck_ny-po
14 unload-truck_package3_la-truck_la-central
End plan
35

Future extensions to blackbox:
• Specification of heuristic axioms in the
STRIPS-style input
– state invariants - straightforward
– optimality conditions - temporal logic?
• Automatic generation of heuristic axioms
DOWNLOAD latest version from www.research.att.com/~kautz/blackbox
36

• in most general case, can cover most forms of problem solving
• special case of program synthesis
• scheduling : fixes set of actions, need to find optimal total ordering
- scheduling problems often (near) linear, may be solvable by linear relaxation techniques
- planning problems typically highly non-linear, require combinatorial search
37

Autonomous systems
Many NASA applications:
• Deep Space One Remote Agent
• Long-range mission planning
• Communication planning & scheduling
Softbots - software robots
• Internet agents , program assistants
• AI “characters” in games, entertainment
User modeling / discourse
Synthesis, diagnosis, experiment design, bugfinding (goal = undesirable state), ...
38

Start with random initial assignment.
Pick a random unsatisfied clause.
Select and flip a variable from that clause:
• With probability p , pick a random variable.
• With probability 1-p , pick greedily
– a variable that minimizes the number of unsatisfied clauses
Repeat to predefined maximum number flips; if no solution found, restart.
39