Introduction Our Approach Results Conclusions Learning Generalized Plans Using Abstract Counting Siddharth Srivastava, Neil Immerman, Shlomo Zilberstein Twenty Third Conference on Artificial Intelligence 17th July, 2008 Introduction Our Approach Results Conclusions Overview Introduction Our Approach Abstraction Mechanism Algorithm for Learning Generalized Plans Results Conclusions Introduction Our Approach Results Conclusions 4 L 2 L 5 L 1 L 3 L Plans vs Algorithms Plans and Algorithms Introduction Our Approach Results Conclusions Plans and Algorithms 4 L 2 L 5 L 1 L 3 L Plans vs Algorithms Move Truck to Dock While #(undelivered crate)>0 Load a crate Find crate’s destination Move truck to destination Unload crate Move Truck to Dock Move Truck to Garage Introduction Our Approach Results Conclusions Plans and Algorithms Finding Algorithm-like Plans Variants of this problem have been of continued interest. Recurring Hurdles Problem definition: unknown numbers Plans with loops: finding loops Plans with loops: reasoning about loops (Plan correctness) Myth Systematic approach =⇒ undecidable (cf. automated programming) We identify a tractable piece of this problem. Introduction Our Approach Results Conclusions Plans and Algorithms Generalized Plans A formalization of algorithm-like plans. Start Move(Truck,Dock) Choose x: crate(x) Connected, directed graph. delivered(x) ch x SetDest(Garage) Nodes → actions. Start/terminal nodes. su Load(x) ScanDest(x) Edges → conditions. no DriveToDest() DriveToDest() Stop Unload(x) SetDest(Dock) DriveToDest() Introduction Our Approach Results Conclusions Abstraction Mechanism Algorithm for Learning Generalized Plans Our Approach Learn from an example plan Recognize loops through loop invariants Use abstraction to identify similar states for determining invariants Introduction Our Approach Results Conclusions Abstraction Mechanism Algorithm for Learning Generalized Plans Representation: States as Logical Structures Dock(1) delivered, Crate 10 Garage(2) Crate Truck(3) Crate(6) 4 Dock 1 Crate(10) at(3,1) .. . at 2 Garage at at at Crate 5 6 Crate 3 Truck Loc 7 dest dest, at Loc dest 8 dest, at 9 delivered, Crate V = {Garage1 , Dock1 , Loc1 , Truck1 , Crate1 , delivered1 , at2 , in2 , dest2 } |S| = {1, 2, . . . 10} delivered(10) Integrity constraints specify legal structures. Introduction Our Approach Results Conclusions Abstraction Mechanism Algorithm for Learning Generalized Plans Representation: Actions Precondition: formula in FO(TC). Action operators = structure transformers Predicate updates p0 (x̄) = (¬p(x̄) ∧ ∆+ p (x̄)) ∨ (p(x̄) ∧ ¬∆− p (x̄)) mv(A,B): topmost0 (x) = (¬topmost(x) ∧ on(A, x)) ∨ (topmost(x) ∧ x 6= B). Introduction Our Approach Results Conclusions Abstraction Mechanism Algorithm for Learning Generalized Plans Review: Need for Abstraction Idea: collapse similar states together. Makes identifying invariants (recurring properties) easy. Use an abstraction mechanism. We use an abstraction scheme from static analysis. Introduction Our Approach Results Conclusions Abstraction Mechanism Algorithm for Learning Generalized Plans Abstraction Using 3-Valued Logic TVLA [Sagiv et al., 2002]: Three Valued Logic Analysis Abstraction predicates: chosen unary predicates. Values of all abstraction predicates on an element define its role. Collapse elements of the same role into summary elements. Relations involving summary elements may become indefinite. C4 C3 C2 C1 dest L3 L2 L1 Abstraction dest Location Crate States from infinitely many instances 7→ finite set of abstract states Introduction Our Approach Results Conclusions Abstraction Mechanism Algorithm for Learning Generalized Plans Precision in Action Updates Predicate update formula: p0 (x̄) = (¬p(x̄) ∧ ∆+ p) (p(x̄) ∧ ¬∆− p) ∨ TVLA’s focus+coerce operations: make structure precise wrt a user defined formula (automatically determined in our approach). φ R S0 fφ φ R S1 φ constrained to be unique. Use this for sensing actions too. φ R R S2 R S3 Introduction Our Approach Results Conclusions Abstraction Mechanism Algorithm for Learning Generalized Plans Learning Generalized Plans We recognize loop invariants by tracing example plans in the abstract state space. Algorithm for Learning Generalized Plans Change action arguments to their roles in the example plan. Apply resulting plan to abstraction of the given start state. Find loops in the resulting state and action sequence. Introduction Our Approach Results Conclusions Abstraction Mechanism Algorithm for Learning Generalized Plans Tracing Crate delivered, Crate Crate delivered, Crate Chosen,Crate Loc dest, at Loc dest Dock at at at at Crate dest, at dest Loc Loc dest Dock at at dest, at delivered, Crate dest Crate dest, at Crate Garage Truck delivered, Crate at Dock at Crate dest dest Garage Chosen,Crate Truck Loc at dest, at Chosen Truck at Dock = dest Loc dest, at Chosen Garage delivered, Crate Crate dest at at Garage delivered, Crate Truck delivered, Crate findDest(); Load(); setTarget(destLoc); Drive(); Unload(); Choose(Crate) Introduction Our Approach Results Conclusions Abstraction Mechanism Algorithm for Learning Generalized Plans Tracing Crate delivered, Crate Chosen,Crate dest at at Dock at Crate Chosen,Crate Crate Crate Truck at Dock at Dock at dest at at at Crate dest Garage delivered, Crate dest, at Loc Crate Dock at at dest, at delivered, Crate dest Crate Truck Loc at Dock at Crate dest dest Garage at Dock at dest, at = dest delivered, Crate Dock dest, at dest, at Garage Chosen Truck dest, at delivered, Crate Loc Loc at Garage delivered, Crate Crate dest delivered, Crate dest, at Truck delivered, Crate Loc Chosen Truck dest, at Garage Chosen,Crate Truck at Garage Dock dest, at delivered, Crate dest, at delivered, Crate Loc Loc dest dest, at Loc dest, at dest Loc dest Chosen, Crate Truck delivered, Crate dest, at Crate Crate at Garage Loc dest Loc at Garage Crate delivered, Crate Loc Crate delivered, Crate findDest(); Load(); setTarget(destLoc); Drive(); Unload(); Choose(Crate) at delivered, Crate Truck Introduction Our Approach Results Conclusions Abstraction Mechanism Algorithm for Learning Generalized Plans Tracing at Dock at Crate dest Loc dest at Dock at at Loc Truck Loc Dock dest, at dest, at Garage Chosen Garage Truck dest at Chosen, Crate Garage Crate dest delivered, Crate at delivered, Crate Truck Introduction Our Approach Results Conclusions Abstraction Mechanism Algorithm for Learning Generalized Plans Finding Preconditions In generalized planning, correctness ≡ applicability. Classify branches on the basis of role counts; propagate these counts backwards. Need for doing this constrains predicate update formulas. φ R S0 fφ φ R φ R R S1 S2 #R = 1 #R > 1 φ constrained to be unique and satisfiable Introduction Our Approach Results Conclusions Abstraction Mechanism Algorithm for Learning Generalized Plans Finding Preconditions In generalized planning, correctness ≡ applicability. Classify branches on the basis of role counts; propagate these counts backwards. Need for doing this constrains predicate update formulas. φ constrained to be unique and satisfiable (# Plan Pre­condition S1 S2 Si Sj Sk R > 1) Introduction Our Approach Results Conclusions Problems Outputs Performance Problem Domains Delivery Assembly and Transport Ln Dock L1 L2 L1 (a) L2 L3 Striped Block Tower L4 Van: Capacity 1 Truck: Capacity 2 (b) Start End (c) Introduction Our Approach Results Conclusions Results: Delivery Problems Outputs Performance setDest(dock) Drive() choose(crate) findDest() Load() setDest(crateDest) Drive() Unload() setDest(dock) Drive() choose(crate) #(cra te findDest() Load() deliv ered )=1 findDest() setDest(crateDest) Load() Drive() setDest(crateDest) Unload() Drive() Unload() setDest(garage) Drive() Learned plan for unit delivery Introduction Our Approach Results Conclusions Problems Outputs Performance Results: Transport mvToL3() mvToL2() chooseVehicle(Van) chooseItem(monitor;atL1) Load() #(m on ito r at L1 )= mvToL3() Unload() mvToL1() chooseVehicle(Truck) chooseItem(server;atL2) Load() mvToL3() chooseItem(monitor;atL3) Load() mvToL4() Unload() 1 Load() mvToL3() Unload() mvToL1() chooseVehicle(Truck) chooseItem(server;atL2) Load() mvToL3() chooseItem(monitor;atL3) Load() mvToL4() Introduction Our Approach Results Conclusions Results: Blocks Problems Outputs Performance Choose(topmost) Choose1(blue;onTable;topmost) mvToTable() Choose2(red;onTable;topmost;base) Move() Choose(topmost) #(blue onTable topmost) = 1 Choose1(red;onTable;topmost) mvToTable() mvToTable() Choose(topmost) Choose2(blue;topmost) mvToTable() Move() Choose1(blue;onTable;topmost) Choose(topmost) Choose2(red;topmost) mvToTable() Choose(topmost) #(red onTable Move() topmost) = 1 mvToTable() Choose1(red;onTable;topmost) mvToTable() Choose(topmost) Choose2(blue;topmost) mvToTable() Move() Choose1(blue;onTable;topmost) #(blue onTable Choose2(red;topmost) topmost) = 1 Move() Choose1(blue;onTable;topmost) Choose2(red;topmost) Move() Move() Introduction Our Approach Results Conclusions Problems Outputs Performance Results: Running Times Planning Times 300 Execution Time Breakups Transport: SGPlan5 SATPLAN06 Aranda Blocks: SGPlan5 SATPLAN06 Aranda 250 20 EgRunTVLA AbsRunTVLA 10 LoopFinding Tracing 5 200 Time (s) Time (s) 15 150 2GB 100 0 Delivery Transport Problem Blocks 50 0 0 5 10 15 20 25 30 Number of Items of Each Kind 35 40 Introduction Our Approach Results Conclusions Conclusions Novel algorithm for generalizing plans and finding loops. Identified a class of domains where our methods are proven to work (extended-LL). No need for plan annotations/parameterization etc. Work in Progress/Future Directions Plan synthesis Extensions beyond extended-LL domains Plan evaluation. Introduction Our Approach Results Conclusions Existing Approaches Other research along this direction Plan compilation: Triangle tables [Fikes et al., 1972], case based planning [Hammond, 1989] Explanation based learning of plans (B AGGER 2) [Shavlik, 1990] Extracting plan templates (D ISTILL) [Winner et al., 2003], planning with loops (K PLANNER) [Levesque, 2005] Introduction Our Approach Results Conclusions Extended-LL Domains Look like linked lists upon abstraction. Theorem In “extended-LL” domains, we can compute all the branch conditions and propagate them backwards to get preconditions for plans with simple loops. We can find complete generalized plans through search in these domains! Defined as a set of syntactic constraints on action update formulae making sure that action updates don’t require more precision than is availabe in abstract structures. Predicate change formulas which need focusing are role-specific, uniquely satisfiable.