Introduction Framework Planning Algorithms Results
Planning with Unknown Object Quantities and
Properties
Siddharth Srivastava
Joint work with Neil Immerman and Shlomo Zilberstein
University of Massachusetts, Amherst
Eighth Symposium on Abstraction, Reformulation and
Approximation
7 − 10 July, 2009
Siddharth Srivastava
Planning with Unknown Object Quantities and Properties
Introduction Framework Planning Algorithms Results
Overview
Introduction
Framework
Concrete Representation
Abstract Representation for Belief States
Actions on Belief States
Planning Algorithms
Plan Generalization
Plan Merging
Preconditions
Results
Siddharth Srivastava
Planning with Unknown Object Quantities and Properties
Introduction Framework Planning Algorithms Results
Conditional Planning
Conditional Planning
Test Type
paper
collect In PaperContainer
pick Next
Siddharth Srivastava
collect In GlassContainer
pick Next
Test Type
paper
glass
Test Type
glass
paper
glass
Planning with Unknown Object Quantities and Properties
Introduction Framework Planning Algorithms Results
Conditional Planning
Conditional Planning
Need to merge, maintain history
Siddharth Srivastava
Planning with Unknown Object Quantities and Properties
Introduction Framework Planning Algorithms Results
Conditional Planning
Conditional Planning
Test Type
glass
collect In PaperContainer
Pick Next
paper
collect In GlassContainer
Pick Next
Need to merge, maintain history
Progress and termination of loops
Siddharth Srivastava
Planning with Unknown Object Quantities and Properties
Introduction Framework Planning Algorithms Results
Concrete Representation Abstraction Actions on Belief States
Concrete States as First-Order Structures
V = {object1 , bin1 , isGlass1 , isPaper1 , in2 , empty1 , collected1 , forGlass1 , forPaper1 }
isPaper
isGlass
2
2
object
object
in
bin
in
1
1
bin
[[object(2)]] =1
[[object(2)]] =1
[[isPaper(2)]] = 1
[[bin(1)]] = 1
[[in(2,1)]] = 1
[[isGlass(2)]] = 1
[[bin(1)]] = 1
[[in(2,1)]] = 1
S1
S2
Siddharth Srivastava
Planning with Unknown Object Quantities and Properties
Introduction Framework Planning Algorithms Results
Concrete Representation Abstraction Actions on Belief States
Action Operators
Precondition: formula in FO.
Predicate updates
tuples added to p
0
p (x̄) = (¬p(x̄) ∧
∆+
p (x̄))
tuples retained in p
∨
(p(x̄) ∧ ¬∆−
p (x̄))
Use formula evaluation to compute action effect.
Frame axioms/successor state axioms (situation calculus) using a
double vocabulary.
Siddharth Srivastava
Planning with Unknown Object Quantities and Properties
Introduction Framework Planning Algorithms Results
Concrete Representation Abstraction Actions on Belief States
Example: The Collect Action
Collect(o,c)
object(o) ∧ container(c) ∧ (isGlass(o) ↔ forGlass(c)) ∧
∃b(bin(b) ∧ in(o, b) ∧ robotAt(b))
in0 (u, v) := (in(u, v) ∧ u 6= o) ∨
(¬in(u, v) ∧ u = o ∧ v = c)
empty0 (u) := (empty(u) ∧ u 6= c) ∨ in(o, u)
collected0 (u) := collected(u) ∨ o = u
isPaper
isPaper
Collect(o,c)
o
o
object
in
bin
b
object, collected
in
c
container, forPaper, empty
b
c
container, forPaper
bin, empty
Siddharth Srivastava
Planning with Unknown Object Quantities and Properties
Introduction Framework Planning Algorithms Results
Concrete Representation Abstraction Actions on Belief States
Abstraction Using 3-Valued Logic
isPaper
isGlass
2
2
object
object
in
bin
in
1
bin
1
Use 3−Valued logic to abstract as:
isPaper
isGlass
2
object
in
bin
= 1/2
1
TVLA: [Sagiv et al., 2002]
Siddharth Srivastava
Planning with Unknown Object Quantities and Properties
Introduction Framework Planning Algorithms Results
Concrete Representation Abstraction Actions on Belief States
Abstraction Using 3-Valued Logic
isPaper
isGlass
2
2
object
object
in
bin
in
1
bin
1
Focus and coerce w.r.t {isPaper(x)}
isPaper
Integrity Constraint:
isGlass
2
object
Objects are either paper or glass
in
bin
1
Implementation of “sensing” actions
Siddharth Srivastava
Planning with Unknown Object Quantities and Properties
Introduction Framework Planning Algorithms Results
Concrete Representation Abstraction Actions on Belief States
Abstraction Using 3-Valued Logic
isPaper
isGlass
isPaper
2
isGlass
4
1
isGlass
6
object
object
in
bin
isPaper
object
in
bin
Siddharth Srivastava
3
in
bin
5
Planning with Unknown Object Quantities and Properties
Introduction Framework Planning Algorithms Results
Concrete Representation Abstraction Actions on Belief States
Abstraction Using 3-Valued Logic
isPaper
isGlass
isPaper
2
isGlass
4
isGlass
6
object
object
in
bin
isPaper
object
in
1
bin
3
in
bin
5
Canonical Abstraction
isPaper
isGlass
V
object
in
bin
= Summary element
U
Siddharth Srivastava
Planning with Unknown Object Quantities and Properties
Introduction Framework Planning Algorithms Results
Concrete Representation Abstraction Actions on Belief States
Abstraction Using 3-Valued Logic
isPaper
isPaper
isGlass
object
object
object
in
in
in
1
bin
isGlass
2n
4
2
bin
isPaper
isGlass
3
bin
2n−1
Concretization
isPaper
isGlass
Integrity Constraint:
Each bin has a unique object
V
object
in
bin
U
Siddharth Srivastava
Planning with Unknown Object Quantities and Properties
Introduction Framework Planning Algorithms Results
Concrete Representation Abstraction Actions on Belief States
Abstraction Using 3-Valued Logic: Summary
TVLA [Sagiv et al., 2002]: Three Valued Logic Analysis
Abstraction predicates: unary predicates.
Element’s role = set of abstraction predicates satisfied
Collapse elements of a role into summary elements.
Use integrity constraints to retreive concrete states.
Siddharth Srivastava
Planning with Unknown Object Quantities and Properties
Introduction Framework Planning Algorithms Results
Concrete Representation Abstraction Actions on Belief States
Action Application on Belief States
isPaper
isGlass
isPaper
2
1
object
in
in
bin
isGlass
2n
object
in
bin
isPaper
isGlass
4
object
3
bin
isPaper
2n−1
isGlass
Axn: CollectAnObject()
V
object
in
bin
U
Siddharth Srivastava
Planning with Unknown Object Quantities and Properties
Introduction Framework Planning Algorithms Results
Concrete Representation Abstraction Actions on Belief States
Action Application on Belief States
A simpler case: only paper objects and containers
isPaper
isPaper
collected
o
object
Collect(o,c)
o
b
bin
object
in
in
in
c
container, forPaper, empty
b
c
container, forPaper
bin
empty
Underspecified action application
Siddharth Srivastava
Planning with Unknown Object Quantities and Properties
Introduction Framework Planning Algorithms Results
Concrete Representation Abstraction Actions on Belief States
Drawing Out Action Arguments
φ
Role i
S0
φ
Focus
Role i
φ
Role i
Role i
S2
S1
Role i
S3
Coerce
φ
Role i
S1
φ
Role i
Role i
S2
φ constrained to be unique and satisfiable
Siddharth Srivastava
Planning with Unknown Object Quantities and Properties
Introduction Framework Planning Algorithms Results
Concrete Representation Abstraction Actions on Belief States
Drawing Out Action Arguments
φ
Role i
S0
Dr
aw
ou
t
an
e
lem
en
t la
be
ll
φ
φ
ed
φ
Role i
S1
Role i
Role i
S2
φ constrained to be unique and satisfiable
Siddharth Srivastava
Planning with Unknown Object Quantities and Properties
Introduction Framework Planning Algorithms Results
Concrete Representation Abstraction Actions on Belief States
Summary of Action Application
Draw-out action arguments prior to application
Use focus and coerce to create cases for properties of drawn
out of objects.
Branches caused:
Classifiable, e.g #R {S} > 1
Unpredictable at planning-time, e.g. “object type=paper”
Siddharth Srivastava
Planning with Unknown Object Quantities and Properties
Introduction Framework Planning Algorithms Results
Plan Generalization Plan Merging Preconditions
Plan Generalization
Use abstract structures to recognize loop invariants in example concrete plans.
Example Execution
2 objects of each type collected;
2 bins remaining
#
S0
#
goToNextBin()
S1
#
senseType()
preProc−Paper()
S2
#
S3
collectPaper()
#
S4
Tracing
S5
S0
goToNextBin()
#{bin}: −1
#{bin, visited}: +1
#{bin}(S 0) > 1
S7
S1
senseType()
S2
preProc−Paper()
#{object}: −1
#{object, isPaper}:+1
Not role−count classifiable
S3
collectPaper()
#{object, isPaper}: −1
#{object, isPaper,
collected}: +1
#{container, forPaper
empty}: −1
#{container, forPaper}: +1
#{bin,visited}: −1
#{bin,visited,empty}: +1
S0
Developed for completely observable settings [Srivastava et al., 2008]
Siddharth Srivastava
Planning with Unknown Object Quantities and Properties
Introduction Framework Planning Algorithms Results
Plan Generalization Plan Merging Preconditions
Merging Generalized Plans
Find Loops
collectPaper()
goToNextBin()
S0
goToNextBin()
S1
S5
senseType()
S2
S7
preProc−Paper()
Plan for Unhandled Structure
S3
#
S7
preProc−Glass()
#
S9
collectGlass()
#
S 10
goToNextBin()
#
S 11
S6
senseType()
S8
Generalize and Merge
collectPaper()
S6
senseType()
S5
goToNextBin()
S0
goToNextBin()
S1
senseType()
senseType()
S8
S2
S7
preProc−Paper()
preProc−Glass()
S3
S9
collectGlass()
S 10
S 12
goToNextBin()
A single plan may not explore all possibilities.
Construct problem instances from unsolved belief states.
Solve them using classical planners.
Siddharth Srivastava
Planning with Unknown Object Quantities and Properties
Introduction Framework Planning Algorithms Results
Plan Generalization Plan Merging Preconditions
Finding Preconditions
Branches solve only some members of abstract structures.
Classify branches using role counts; propagate backwards.
Need for doing this constrains predicate update formulas.
Siddharth Srivastava
Planning with Unknown Object Quantities and Properties
Introduction Framework Planning Algorithms Results
Example Results
Initial: p0 = k{paper, collected}k; pc0 = k{empty,container,forPaper}k;
g0 , gc0 : similar for glass; b0 = k{bin}k
Loops 1 & 2
Loop 1
goToNextBin()
goToNextBin()
senseType()
senseType()
2
paper
apply−PaperPreProc(obj)
apply−PaperPreProc(obj)
collect−Paper−Cont(obj)
collect−Paper−Cont(obj)
apply−GlassPreProc(obj)
collect−Glass−Cont(obj)
goToNextBin()
senseType()
paper
senseType()
paper
apply−PaperPreProc(obj)
apply−PaperPreProc(obj)
collect−PaperCont(obj)
collect−PaperCont(obj)
Final: p0 + l1 ; pc0 − l1 ; b0 − l1
Solves 1 instance out of every
2n
Siddharth Srivastava
Final: p0 + l1 ; pc0 − l1 ; g0 +
l2 ; gc0 − l2 ; b0 − l1 − l2
2n−1 + 1 out of every 2n
Planning with Unknown Object Quantities and Properties
Introduction Framework Planning Algorithms Results
Transport Domain
Siddharth Srivastava
Planning with Unknown Object Quantities and Properties
Introduction Framework Planning Algorithms Results
Transport Domain: Results
Initial undelivered counts:
m0 = k{monitor, atD2}k; s0 = k{server, atD1}k
Loop 1
Loops 1 & 2
mv(T1, L)
load(server, T1)
unload(T1)
mv(T1, L)
load(server, T1)
mv(T1,D1)
mv(T1,D1)
load(LCD, T2)
load(LCD, T2)
mv(T2,L)
mv(T2,L)
unload(T1)
server lost
mv(T2, D1)
load(server,T2)
server present
load(server, T2)
mv(T2, D3)
mv(T2,L)
load(server, T2)
unload(T2)
mv(T2, L)
mv(T2, D2)
mv(T2, D3)
unload(T2)
mv(T2, L)
mv(T2, D2)
Final: mo − l1 ; s0 − l1
Siddharth Srivastava
Final: m0 − l1 ; so − l1 − k1
Planning with Unknown Object Quantities and Properties
Introduction Framework Planning Algorithms Results
Solutions: Recycling and Transport
goToNextBin()
mv(T2, L)
senseType()
2
apply−GlassPreProc(obj)
mv(T2, D2)
load(kind1, T1)
2
forkLift(kind1, T1)
apply−PaperPreProc(obj)
collect−Glass−Cont(obj)
collect−Paper−Cont(obj)
mv(T1, L)
unload(T1)
goToNextBin()
mv(T1,D1)
mv(T2,L)
3
3
load(kind2, T2)
mv(T2, D1)
load(kind1,T2)
4
forkLift(kind1, T2)
senseType()
load(kind1, T2)
4
apply−GlassPreProc(obj)
collect−Glass−Cont(obj)
mv(T2,L)
2
forkLift(kind1, T2)
apply−PaperPreProc(obj)
mv(T2, D3)
unload(T2)
collect−PaperCont(obj)
Transport
Recycling
Siddharth Srivastava
Planning with Unknown Object Quantities and Properties
Introduction Framework Planning Algorithms Results
Conclusions
An approach for representing unknown quantities for
planning.
Methods for finding generalized plans with branches and
loops.
Automatic computation of preconditions for many kinds of
nested loops in a broad class of domains.
Siddharth Srivastava
Planning with Unknown Object Quantities and Properties
Introduction Framework Planning Algorithms Results
Example Results: Domain Coverage
CFF-soln7
Gen(Eg 1)
Gen(Eg 1+2)
Gen(Eg 1..3)
Gen(Eg 1..4)
1
D(n>=8) = 1
Coverage (D)
0.8
0.6
D(n>=8) = 0.5
0.4
D(n>=8) = 0.25
0.2
0
0
5
10
15
20
Max Number of Bins (n)
25
30
35
Dπ (n) = |Sπ (n)|/|T (n)|
Siddharth Srivastava
Planning with Unknown Object Quantities and Properties
Introduction Framework Planning Algorithms Results
Merging Generalized Plans: Algorithm
1
2
3
4
5
6
7
8
9
Input: Existing plan Π, eg trace tracei
Output: Extension of Π
if Π = ∅ then
Π ← tracei
return Π
end
repeat
mpΠ , mpt ← findMergePoint(Π, tracei , bpΠ , bpt )
if mpΠ found and not first iteration then
attachEdges(Π, tracei , bpt , mpt , mpΠ , bpΠ )
end
if mpΠ found then
bpΠ , bpt ← findBranchPoint(Π, tracei , mpΠ , mpt )
end
until new bpΠ or mpΠ not found
10 return Π
Algorithm 1: A RANDA-Merge
Siddharth Srivastava
Planning with Unknown Object Quantities and Properties
Introduction Framework Planning Algorithms Results
Related Work
Plans with Loops
[Winner and Veloso, 2007]: no preconditions or sensing
actions, but use partial ordering.
[Levesque, 2005]: single planning parameter, limited
preconditions.
[Cimatti et al., 2003]: “hard” loops.
Planning with unknown quantities:
[Milch et al., 2005]: action operators not provided.
Siddharth Srivastava
Planning with Unknown Object Quantities and Properties
Introduction Framework Planning Algorithms Results
Plan Generalization: Example
Crate
delivered, Crate
Crate
delivered, Crate
Chosen,Crate
Loc
dest, at
Loc
dest
Dock
at
at
at
at
Crate
dest, at
dest
Loc
Loc
dest
Dock
at
at
dest, at
delivered, Crate
dest
Crate
dest, at
Crate
Garage
Truck
delivered, Crate
at
Dock
at
Crate
dest
dest
Garage
Chosen,Crate
Truck
Loc
at
dest, at
Chosen
Truck
at
Dock
=
dest
Loc
dest, at
Chosen
Garage
delivered, Crate
Crate
dest
at
at
Garage
delivered, Crate
Truck
delivered, Crate
findDest(); Load(); setTarget(destLoc); Drive(); Unload(); Choose(Crate)
Siddharth Srivastava
Planning with Unknown Object Quantities and Properties
Introduction Framework Planning Algorithms Results
Plan Generalization: Example
at
Dock
crate
dest
Location
at
Dock
crate
dest
Truck
Truck
S1
chosen;crate
at
dest, at
delivered;crate
Garage
Garage
Dock
Location
at
at
dest
Location
at
Dock
dest, at
S2
crate
dest
Location
dest, at
Choose(crate)
at
at
delivered;crate
Garage
delivered;crate
Garage
Truck
Truck
S4
S3
Siddharth Srivastava
Planning with Unknown Object Quantities and Properties
Introduction Framework Planning Algorithms Results
References I
Cimatti, A., Pistore, M., Roveri, M., and Traverso, P. (2003).
Weak, strong, and strong cyclic planning via symbolic model
checking.
Artif. Intell., 147(1-2):35–84.
Levesque, H. J. (2005).
Planning with loops.
In Proc. of IJCAI, pages 509–515.
Milch, B., Marthi, B., Russell, S. J., Sontag, D., Ong, D. L.,
and Kolobov, A. (2005).
Blog: Probabilistic models with unknown objects.
In Proc. of IJCAI, pages 1352–1359.
Siddharth Srivastava
Planning with Unknown Object Quantities and Properties
Introduction Framework Planning Algorithms Results
References II
Sagiv, M., Reps, T., and Wilhelm, R. (2002).
Parametric shape analysis via 3-valued logic.
ACM Transactions on Programming Languages and Systems,
24(3):217–298.
Srivastava, S., Immerman, N., and Zilberstein, S. (2008).
Learning generalized plans using abstract counting.
In Proc. of AAAI, pages 991–997.
Winner, E. and Veloso, M. (2007).
LoopDISTILL: Learning domain-specific planners from
example plans.
In Workshop on AI Planning and Learning, ICAPS.
Siddharth Srivastava
Planning with Unknown Object Quantities and Properties