Introduction
Our Approach
Results
Conclusions
Learning Generalized Plans
Using Abstract Counting
Siddharth Srivastava, Neil Immerman, Shlomo Zilberstein
Twenty Third Conference on Artificial Intelligence
17th July, 2008
Introduction
Our Approach
Results
Conclusions
Overview
Introduction
Our Approach
Abstraction Mechanism
Algorithm for Learning Generalized Plans
Results
Conclusions
Introduction
Our Approach
Results
Conclusions
4
L
2
L
5
L
1
L
3
L
Plans vs Algorithms
Plans and Algorithms
Introduction
Our Approach
Results
Conclusions
Plans and Algorithms
4
L
2
L
5
L
1
L
3
L
Plans vs Algorithms
Move Truck to Dock
While #(undelivered crate)>0
Load a crate
Find crate’s destination
Move truck to destination
Unload crate
Move Truck to Dock
Move Truck to Garage
Introduction
Our Approach
Results
Conclusions
Plans and Algorithms
Finding Algorithm-like Plans
Variants of this problem have been of continued interest.
Recurring Hurdles
Problem definition: unknown numbers
Plans with loops: finding loops
Plans with loops: reasoning about loops (Plan correctness)
Myth Systematic approach =⇒ undecidable
(cf. automated programming)
We identify a tractable piece of this problem.
Introduction
Our Approach
Results
Conclusions
Plans and Algorithms
Generalized Plans
A formalization of algorithm-like plans.
Start
Move(Truck,Dock)
Choose x: crate(x)
Connected, directed
graph.
delivered(x)
ch
x
SetDest(Garage)
Nodes → actions.
Start/terminal nodes.
su
Load(x)
ScanDest(x)
Edges → conditions.
no
DriveToDest()
DriveToDest()
Stop
Unload(x)
SetDest(Dock)
DriveToDest()
Introduction
Our Approach
Results
Conclusions
Abstraction Mechanism
Algorithm for Learning Generalized Plans
Our Approach
Learn from an example plan
Recognize loops through loop invariants
Use abstraction to identify similar states for determining
invariants
Introduction
Our Approach
Results
Conclusions
Abstraction Mechanism
Algorithm for Learning Generalized Plans
Representation: States as Logical Structures
Dock(1)
delivered, Crate
10
Garage(2)
Crate
Truck(3)
Crate(6)
4
Dock
1
Crate(10)
at(3,1)
..
.
at
2
Garage
at
at
at
Crate
5
6
Crate
3
Truck
Loc
7
dest
dest, at
Loc
dest 8
dest, at
9
delivered, Crate
V =
{Garage1 , Dock1 , Loc1 , Truck1 , Crate1 , delivered1 , at2 , in2 , dest2 }
|S| = {1, 2, . . . 10}
delivered(10)
Integrity constraints specify legal structures.
Introduction
Our Approach
Results
Conclusions
Abstraction Mechanism
Algorithm for Learning Generalized Plans
Representation: Actions
Precondition: formula in FO(TC).
Action operators = structure transformers
Predicate updates
p0 (x̄)
= (¬p(x̄) ∧ ∆+
p (x̄))
∨
(p(x̄) ∧ ¬∆−
p (x̄))
mv(A,B):
topmost0 (x) = (¬topmost(x) ∧ on(A, x))
∨
(topmost(x) ∧ x 6= B).
Introduction
Our Approach
Results
Conclusions
Abstraction Mechanism
Algorithm for Learning Generalized Plans
Review: Need for Abstraction
Idea: collapse similar states together.
Makes identifying invariants (recurring properties) easy.
Use an abstraction mechanism.
We use an abstraction scheme from static analysis.
Introduction
Our Approach
Results
Conclusions
Abstraction Mechanism
Algorithm for Learning Generalized Plans
Abstraction Using 3-Valued Logic
TVLA [Sagiv et al., 2002]: Three Valued Logic Analysis
Abstraction predicates: chosen unary predicates.
Values of all abstraction predicates on an element define its
role.
Collapse elements of the same role into summary
elements.
Relations involving summary elements may become
indefinite.
C4
C3
C2
C1
dest
L3
L2
L1
Abstraction
dest
Location
Crate
States from infinitely many instances 7→ finite set of abstract states
Introduction
Our Approach
Results
Conclusions
Abstraction Mechanism
Algorithm for Learning Generalized Plans
Precision in Action Updates
Predicate update formula:
p0 (x̄) = (¬p(x̄) ∧ ∆+
p)
(p(x̄) ∧ ¬∆−
p)
∨
TVLA’s focus+coerce operations: make structure precise wrt
a user defined formula (automatically determined in our
approach).
φ
R
S0
fφ
φ
R
S1
φ constrained to be unique.
Use this for sensing actions too.
φ
R
R
S2
R
S3
Introduction
Our Approach
Results
Conclusions
Abstraction Mechanism
Algorithm for Learning Generalized Plans
Learning Generalized Plans
We recognize loop invariants by tracing example plans in the
abstract state space.
Algorithm for Learning Generalized Plans
Change action arguments to their roles in the example plan.
Apply resulting plan to abstraction of the given start state.
Find loops in the resulting state and action sequence.
Introduction
Our Approach
Results
Conclusions
Abstraction Mechanism
Algorithm for Learning Generalized Plans
Tracing
Crate
delivered, Crate
Crate
delivered, Crate
Chosen,Crate
Loc
dest, at
Loc
dest
Dock
at
at
at
at
Crate
dest, at
dest
Loc
Loc
dest
Dock
at
at
dest, at
delivered, Crate
dest
Crate
dest, at
Crate
Garage
Truck
delivered, Crate
at
Dock
at
Crate
dest
dest
Garage
Chosen,Crate
Truck
Loc
at
dest, at
Chosen
Truck
at
Dock
=
dest
Loc
dest, at
Chosen
Garage
delivered, Crate
Crate
dest
at
at
Garage
delivered, Crate
Truck
delivered, Crate
findDest(); Load(); setTarget(destLoc); Drive(); Unload(); Choose(Crate)
Introduction
Our Approach
Results
Conclusions
Abstraction Mechanism
Algorithm for Learning Generalized Plans
Tracing
Crate
delivered, Crate
Chosen,Crate
dest
at
at
Dock
at
Crate
Chosen,Crate
Crate
Crate
Truck
at
Dock
at
Dock
at
dest
at
at
at
Crate
dest
Garage
delivered, Crate
dest, at
Loc
Crate
Dock
at
at
dest, at
delivered, Crate
dest
Crate
Truck
Loc
at
Dock
at
Crate
dest
dest
Garage
at
Dock
at
dest, at
=
dest
delivered, Crate
Dock
dest, at
dest, at
Garage
Chosen
Truck
dest, at
delivered, Crate
Loc
Loc
at
Garage
delivered, Crate
Crate
dest
delivered, Crate
dest, at
Truck
delivered, Crate
Loc
Chosen
Truck
dest, at
Garage
Chosen,Crate
Truck
at
Garage
Dock
dest, at
delivered, Crate
dest, at
delivered, Crate Loc
Loc
dest
dest, at
Loc
dest, at
dest
Loc
dest
Chosen, Crate
Truck
delivered, Crate
dest, at
Crate
Crate
at
Garage
Loc
dest
Loc
at
Garage
Crate
delivered, Crate
Loc
Crate
delivered, Crate
findDest(); Load(); setTarget(destLoc); Drive(); Unload(); Choose(Crate)
at
delivered, Crate
Truck
Introduction
Our Approach
Results
Conclusions
Abstraction Mechanism
Algorithm for Learning Generalized Plans
Tracing
at
Dock
at
Crate
dest
Loc
dest
at
Dock
at
at
Loc
Truck
Loc
Dock
dest, at
dest, at
Garage
Chosen
Garage
Truck
dest
at
Chosen, Crate
Garage
Crate
dest
delivered, Crate
at
delivered, Crate
Truck
Introduction
Our Approach
Results
Conclusions
Abstraction Mechanism
Algorithm for Learning Generalized Plans
Finding Preconditions
In generalized planning, correctness ≡ applicability.
Classify branches on the basis of role counts; propagate
these counts backwards.
Need for doing this constrains predicate update formulas.
φ
R
S0
fφ
φ
R
φ
R
R
S1
S2
#R = 1
#R > 1
φ constrained to be unique and satisfiable
Introduction
Our Approach
Results
Conclusions
Abstraction Mechanism
Algorithm for Learning Generalized Plans
Finding Preconditions
In generalized planning, correctness ≡ applicability.
Classify branches on the basis of role counts; propagate
these counts backwards.
Need for doing this constrains predicate update formulas.
φ constrained to be unique and satisfiable
(#
Plan Pre­condition
S1
S2
Si
Sj
Sk
R
>
1)
Introduction
Our Approach
Results
Conclusions
Problems
Outputs
Performance
Problem Domains
Delivery
Assembly and Transport
Ln
Dock
L1
L2
L1
(a)
L2
L3
Striped Block Tower
L4
Van: Capacity 1
Truck: Capacity 2
(b)
Start
End
(c)
Introduction
Our Approach
Results
Conclusions
Results: Delivery
Problems
Outputs
Performance
setDest(dock)
Drive()
choose(crate)
findDest()
Load()
setDest(crateDest)
Drive()
Unload()
setDest(dock)
Drive()
choose(crate)
#(cra
te
findDest()
Load()
deliv
ered
)=1
findDest()
setDest(crateDest)
Load()
Drive()
setDest(crateDest)
Unload()
Drive()
Unload()
setDest(garage)
Drive()
Learned plan for unit delivery
Introduction
Our Approach
Results
Conclusions
Problems
Outputs
Performance
Results: Transport
mvToL3()
mvToL2()
chooseVehicle(Van)
chooseItem(monitor;atL1)
Load()
#(m
on
ito
r
at
L1
)=
mvToL3()
Unload()
mvToL1()
chooseVehicle(Truck)
chooseItem(server;atL2)
Load()
mvToL3()
chooseItem(monitor;atL3)
Load()
mvToL4()
Unload()
1
Load()
mvToL3()
Unload()
mvToL1()
chooseVehicle(Truck)
chooseItem(server;atL2)
Load()
mvToL3()
chooseItem(monitor;atL3)
Load()
mvToL4()
Introduction
Our Approach
Results
Conclusions
Results: Blocks
Problems
Outputs
Performance
Choose(topmost)
Choose1(blue;onTable;topmost)
mvToTable()
Choose2(red;onTable;topmost;base)
Move()
Choose(topmost)
#(blue
onTable
topmost) = 1
Choose1(red;onTable;topmost)
mvToTable()
mvToTable()
Choose(topmost)
Choose2(blue;topmost)
mvToTable()
Move()
Choose1(blue;onTable;topmost)
Choose(topmost)
Choose2(red;topmost)
mvToTable()
Choose(topmost)
#(red
onTable
Move()
topmost) = 1
mvToTable()
Choose1(red;onTable;topmost)
mvToTable()
Choose(topmost)
Choose2(blue;topmost)
mvToTable()
Move()
Choose1(blue;onTable;topmost)
#(blue
onTable
Choose2(red;topmost)
topmost) = 1
Move()
Choose1(blue;onTable;topmost)
Choose2(red;topmost)
Move()
Move()
Introduction
Our Approach
Results
Conclusions
Problems
Outputs
Performance
Results: Running Times
Planning Times
300
Execution Time Breakups
Transport: SGPlan5
SATPLAN06
Aranda
Blocks: SGPlan5
SATPLAN06
Aranda
250
20
EgRunTVLA
AbsRunTVLA
10
LoopFinding
Tracing
5
200
Time (s)
Time (s)
15
150
2GB
100
0
Delivery
Transport
Problem
Blocks
50
0
0
5
10
15
20
25
30
Number of Items of Each Kind
35
40
Introduction
Our Approach
Results
Conclusions
Conclusions
Novel algorithm for generalizing plans and finding loops.
Identified a class of domains where our methods are proven
to work (extended-LL).
No need for plan annotations/parameterization etc.
Work in Progress/Future Directions
Plan synthesis
Extensions beyond extended-LL domains
Plan evaluation.
Introduction
Our Approach
Results
Conclusions
Existing Approaches
Other research along this direction
Plan compilation: Triangle tables [Fikes et al., 1972], case
based planning [Hammond, 1989]
Explanation based learning of plans (B AGGER 2) [Shavlik,
1990]
Extracting plan templates (D ISTILL) [Winner et al., 2003],
planning with loops (K PLANNER) [Levesque, 2005]
Introduction
Our Approach
Results
Conclusions
Extended-LL Domains
Look like linked lists upon abstraction.
Theorem
In “extended-LL” domains, we can compute all the branch
conditions and propagate them backwards to get preconditions for
plans with simple loops.
We can find complete generalized plans through search in these
domains!
Defined as a set of syntactic constraints on action update
formulae making sure that action updates don’t require
more precision than is availabe in abstract structures.
Predicate change formulas which need focusing are
role-specific, uniquely satisfiable.