Hierarchical Planning
Hierarchical Planning
 Principle
 hierarchical organization of 'actions'
 complex and less complex (or: abstract) actions
 lowest level reflects directly executable actions
Procedure
 planning starts with complex action on top
 plan constructed through action decomposition
 substitute complex action with plan of less complex actions
(pre-defined plan schemata; or learning of plans/plan
abstraction)
 overall plan must generate effect of complex action
Hierarchical Planning
•
Hierarchical Planning / Plan Decomposition
•
Plans are organized in a hierarchy. Links between nodes
at different levels in the hierarchy denote a
decomposition of a “complex action” into more primitive
actions (operator expansion).
Example:
move (x, y, z)
operator
expansion
pickup (x, y)
putdown (x, z)
The lowest level corresponds to executable actions of the
agent.
Hierarchical Plan - Example
Travel (source, dest.)
Take-Plane
Take-Bus
Take-Car
Goto (bus, source) Buy-Ticket (bus) Hop-on (bus) Leave (bus, dest.)
Goto (counter) Request (ticket) Pay (ticket)
Components of Hierarchical Planning
 Artificial intelligence (AI) hierarchical planning usually entails the
following essential elements:
 High-Level Goals: High-level goals provide the initial direction
for the planning process and guide the decomposition of tasks into
smaller sub-goals.
 Tasks: Tasks are actions that need to be performed to accomplish
the high-level goals.
 Sub-Goals: Sub-goals are intermediate objectives that contribute to
the accomplishment of higher-level goals. Sub-goals are derived
from decomposing high-level goals into smaller, more manageable
tasks.
 Hierarchical Structure: Hierarchical planning organizes tasks and
goals into a hierarchical structure, where higher-level goals are
decomposed into sub-goals, and sub-goals are further decomposed
until reaching primitive actions that can be directly executed.
 Task Dependencies and Constraints: Hierarchical planning
considers dependencies and constraints between tasks and subgoals. These dependencies determine the order in which tasks
need to be executed and any preconditions that must be satisfied
before a task can be performed.
 Plan Representation: Plans in hierarchical planning are
represented as hierarchical structures that capture the sequence
of tasks and sub-goals required to achieve the high-level goals.
This representation facilitates efficient plan generation,
execution, and monitoring.
 Plan Evaluation and Optimization: Hierarchical planning
involves evaluating and optimizing plans to ensure they meet the
desired criteria, such as efficiency, feasibility, and resource
utilization. This may involve iteratively refining the plan
structure or adjusting task priorities to improve performance.
Hierarchical Planning in Autonomous
Driving
 Let's consider an example of autonomous driving car, here
hierarchical planning is employed in the following manner:
1. High-Level Goal: safely navigate from A to B, following the
traffic rules
2. Major Steps:
 Route Planning: determine optimal route to B
 Obstacle Avoidance: identify obstacles like vehicle,
people, etc.
 Traffic Signal Recognition: detect traffic signals and signs
 Lane Keeping: stay in the designated lane and adjust the
vehicle's position to avoid collision
3. Minor Steps:
 Route Planning:
 Map Analysis: analyze maps to find the optimal route
 Traffic Prediction: predict traffic patterns to avoid traffic jams.
 Obstacle Avoidance:
 Sensor Data Processing: process the data from onboard sensor to
detect nearby objects
 Path Planning: generate paths to avoid obstacles
 Traffic Signal Recognition:
 Image Recognition: analyze images to detect traffic lights
 Traffic Rule Interpretation: interpret and detect signal to determine
the action
 Lane Keeping:
 Lane Detection: use computer vision algorithms to detect lane
markings
 Control Systems: adjust the speed, steering, break command to keep
the vehicle within the detected lane.
4. Hierarchical Planning:
 First Level Plan: Define the high-level goals and major steps,
such as route planning, obstacle avoidance, traffic signal
recognition, and lane keeping.
 Second Level Plan: Break down each major step into
subtasks and minor steps, as described above, to handle the
complexity of each component.
 Third Level Plan: Further decompose the minor steps into
detailed actions and algorithms necessary to execute them
effectively.
Hierarchical Planning in Robotics
 Let's consider an example of hierarchical planning applied to
a robotic arm tasked with assembling electronic devices:
1. High-Level Goal: ensemble electronic devices following the
provided assembly process
2. Major Steps:
 Identifying Components: identify and local the
components required for assembling the device.
 Planning Assembly Sequence: determine optimal
sequence of assembly steps to minimize assembly time and
maximize efficiency.
 Manipulation and Grasping: manipulate the arm of the
robot to grasp and manipulate the electronic components.
 Quality Control: ensure the quality of assembly step and
detect and correct any errors.
3. Minor Steps:
 Identifying Components:
 Object Recognition: recognize components using computer vision
 Inventory Management: maintain inventory of the components
 Planning Assembly Sequence:
 Task Planning: break the process into smaller and sequential tasks
 Motion Planning: planning the arm movement to perform
assembling
 Manipulation and Grasping:
 Grasping strategy: determine optimal grasping pose based on
shape and side of the components.
 Path Generation: generate smooth trajectories for the robotic arm
 Quality Control:
 Vision Inspection: inspect assembled components using cameras
 Feedback Control: implement feedback mechanism to adjust
assembly actions.
4. Hierarchical Planning:
 First Level Plan: Define the high-level goals and major steps,
including component identification, assembly sequence
planning, manipulation and grasping, and quality control.
 Second Level Plan: Break down each major step into
subtasks and minor steps, such as object recognition,
inventory management, task planning, motion planning,
grasping strategy, path generation, vision inspection, and
feedback control.
 Third Level Plan: Further decompose the minor steps into
detailed actions and algorithms necessary to execute them
effectively, such as specific image processing algorithms for
object recognition or control algorithms for adjusting robotic
arm movements based on inspection results.
Example:
 Consider a robot tasked with cleaning a house:
 High-Level Plan: Clean the house.
 Intermediate Sub-Goals:
 Clean the living room.
 Clean the kitchen.
 Clean the bedroom.
 Low-Level Actions (for each room):
 Move to the room.
 Pick up trash.
 Vacuum the floor.
 Dust the surfaces.
 Hierarchical Planning Process:
 Define the High-Level Goal: Start with a high-level goal,
such as "Clean the house."
 Decompose the Goal: Break down the high-level goal into
intermediate sub-goals, such as "Clean the living room,"
"Clean the kitchen," and "Clean the bedroom."
 Plan for Sub-Goals: Further decompose each sub-goal into
specific low-level actions. For example, "Clean the living
room" may involve moving to the living room, picking up
trash, vacuuming the floor, and dusting surfaces.
 Execute Plans: Execute the low-level actions to achieve the
sub-goals. Once all sub-goals are accomplished, the highlevel goal is achieved.
Hierarchical Planning Techniques
 In robotic arm assembly, hierarchical planning techniques like
HTN planning, hierarchical reinforcement learning (HRL),
HTNs, and hierarchical state space search ensure efficient
execution.
 HTN Planning: Organizes major steps hierarchically,
facilitating task decomposition.
 Hierarchical Reinforcement Learning (HRL): Learns
hierarchical policies for manipulation and grasping,
optimizing performance.
 HTNs (Hierarchical Task Networks): Structures minor steps
systematically, ensuring accurate component handling.
 Hierarchical State Space Search: Optimizes assembly
sequences considering constraints, ensuring efficiency.
Extensions and Modifications
to Basic Planning Methods
ADL - Action Definition Language
ADL
Can be seen as extension of the STRIPS language.
Contains typing of parameters (sorts).
Allows explicit expression of negation.
Allows equality of terms in precondition formula.
Example:
Fly (p: plane; from: airport; to: airport; c: cargo)
precondition: at(p,to)  at(c,to)  in(c,p)  tofrom
effect: at(p,to)  at(c,to)  at(p,from)  at(c,from)
From Russell & Norvig, Chapter 11
Resource Constraints in Planning
 Resources
 physical quantities, e.g. money, fluids etc.
 time
 Integrate Measures into Action Description and Planning
 representation of physical quantities and reasoning /
calculation,
 e.g. for buy-action: effect: cash := cash – price (x)
 time system / time logic,
 e.g. go-to-action: effect: time := time + 30 (Minutes)
 Backtracking on Constraint Violation
Other Issues in Planning
Disjunctive Preconditions
 planning with alternatives
Disjunctive Effects
 parallel future worlds to consider
All-Quantified Variables (in preconditions and effects)
 only for finite, static Universe of objects
Conditional Planning
 action depends on conditions
 specified concretely only at plan execution time
 typically based on percepts/sensor information
 integrate into partial order planning with threats
Real World Agents 1
 Consider Sensors and Effectors
 perception of environment, e.g. vision
 ensure correspondence between internal map of robot and
environment, e.g. locating robot
 low-level body control, e.g. Motion Control (behaviour
routines, e.g. Fuzzy or Neural Network Controller)
 other sensor information for body control and environment
mapping, e.g. bumpers, radar
 sensors for other information channels and cognitive
processes, e.g. speech – language
Real World Agents 2
 Low-level Processing and Control
 Motion Control
 Audio Recording and low-level analysis
 Medium-level Processing
 Navigation / Route Planning
 Robot Location
 Higher-level Processing
 Speech Recognition, NLP, ...
 Strategies, Planning
 BDI (Belief-Desire-Intention) - Architecture
Real World Agents 3
Multi-Agents
 Language / Communication
 communicating agents
 Mental Models of other Agents
 cooperating agents
 Strategies
 cooperating agents
 Deontic Systems
 Trust
Additional References
Nils J. Nilsson: Artificial Intelligence – A New Synthesis. Morgan
Kaufmann, San Francisco, 1998.
Konolidge, K. and K. Myers: The Saphira Architecture for
Autonomous Mobile Robots (Robot Soccer Class Project)
Guzzoni, D. et al.: Many Robots Make Short Work. (AAAI’96
Robot Competition - Meeting Scheduling)
Martina Veloso, MIT (RoboCup)