ARTIFICIAL INTELLIGENCE
[INTELLIGENT AGENTS PARADIGM]
PLANNING AGENTS
Professor Janis Grundspenkis
Riga Technical University
Faculty of Computer Science and Information Technology
Institute of Applied Computer Systems
Department of Systems Theory and Design
E-mail: Janis.Grundspenkis@rtu.lv
Simple Planning Agents (1)
• PLANNING AGENT generates goals
to achieve, constructs plan that
achieves its goal, executes this plan
until it is finished, and then begins
again with a new goal
Simple Planning Agents (2)
• What is the difference between
problem-solving agent and planning
agent?
– Differences are in representations of
• goals
• states
• actions
– Differences are in the representation
and construction of action sequences
(search for solutions)
Simple Planning Agent’s
Program
> Percept
TELL (KB, MAKE PERCEPT SENTENCE (percept))
CURRENT STATE DESCRIPTION (KB)
> Goal generation
ASK (KB, MAKE GOAL QUERY)  GOAL
> Plan construction
CALL PLANNING ALGORITHM (CURRENT STATE
DESCRIPTION, GOAL, KB)  PLAN
> Action
First action from the PLAN
TELL (KB, MAKE ACTION SENTENCE (action))
Return: action
Basic Elements of a SearchBased Problem-Solver (1)
• Representation of actions
– Actions are described by programs that
generate successor state descriptions
• Representation of states
– States are simple data structures.
– All state descriptions are complete. (A
complete description of initial state is given.
Actions are represented by a program that
generates complete state descriptions)
Basic Elements of a SearchBased Problem-Solver (2)
• Representation of goals
– Agent has information in the form of goal test
and the heuristic function
• Representation of solutions (plans)
– A solution is sequence of actions beginning
from the initial state, and ending at a goal
state
Basic Elements
of Planning Agent
• Planning algorithms use descriptions in some
formal language, usually first-order logic
• Representation of actions
– Logical descriptions of preconditions and effects
• Representation of states
– Set of sentences of first-order logic
• Representation of goals
– Set of sentences of first-order logic
• Representation of plans
– Direct connections between states and actions
Three Key Ideas Behind
Planning
1. “Open up” the representation of states, goals
and actions
2. The planner is free to add actions to the plan
wherever they are needed (instead of growing
sequence of actions starting at the initial state)
3. Most parts of the world are independent from
the other parts
Divide-and-conquer strategy is used,
because it is easier to solve several small
sub-problems rather than one big problem
Representations for Planning
• Representations for states and
goals
• Representations for actions
• Situation Space
• Plan Space
• Representations for Plans
• Solutions
Representations
for States and Goals (1)
• STRIPS language is used in most
planners
• In STRIPS language states are
represented by conjunctions of
predicates (possibly negated) with
constants as their terms
Example:
At(Shop)  Have(Money)  Have(Computer)
Representations
for States and Goals (2)
•
In STRIPS language goals or represented by
conjunctions of predicates with constants and
variables used as their terms
Examples:
At(home)  Have(Computer)
At(x)  Sells(x, Computer)
> Variables are assumed to be existentially
quantified
• Representations of initial states and goals are
used as inputs of planning systems
Representations
for Actions (1)
• STRIPS operators consist from
three components:
–The action description
• Within the planner it is only the name
of a possible action
• Agent returns it to the environment in
order to do something
Representations
for Actions (2)
–The precondition
• It is a conjunction of predicates (not
negated) that must be true before the
operator can be applied
–The effect of an operator
• It is a conjunction of predicates
(possibly negated) that describes how
the situation changes when the
operator is applied
STRIPS Operator (1)
• Syntax:
Operator (ACTION: action’s name,
PRECONDITION: p(x)  q(y)  ...
EFFECT: r(y)  w(x)  ...)
where p and q denotes any predicates; p and w
may be changed to r and/or w in EFFECT
> An operator with variables is called operator
schema
> All variables are assumed universally
quantified
STRIPS Operator (2)
• Graphical notation (example):
PRECONDITION: At(home), Path(home, shop)
ACTION:
 Go(shop)
EFFECT:
At(shop), At(home)
Situation Space
for Planning (1)
• Situation space is represented by a
graph
– Nodes represent possible situations in the
world
– Arcs represent transitions between situations.
Transitions occur if operators are applied one
at a time
• A path through the situation space from
the initial situation to the goal situation
constitutes a plan
Situation Space
for Planning (2)
• Situation space planner searches
through the situation space
– Progression planner searches forward
from the initial situation to the goal
situation
– Regression planner searches
backward from the goal situation to the
initial situation
Plan Space for Search (1)
• Space of Plans
– Search starts with partial plan
(incomplete plan)
– Partial plan is expanded until a
complete plan that solves the problem
– The final plan is the solution, and the
path taken to reach it is irrelevant
Plan Space for Search (2)
• Operators on plans
Operators are: adding a step, ordering steps, binding
variables, etc.
– Refinement operators add constraints to a partial
plan
• If partial plans are representations for complete,
fully constrained plans, then refinement operators
eliminate some plans from this set
• Refinement operators never add new plans to this
set
– Modification operators are anything that is not a
refinement operator
Types of Plans (1)
• Partially ordered plans
Some steps are ordered (before or
after) with respect to each other
and other steps are unordered
• Totally ordered plans
Plans consist of a simple list of steps
Types of Plans (2)
• Linearization of a plan P is a
totally ordered plan derived from
P by adding ordering constraints
• Fully instantiated plans
Plans in which every variable is
bound to a constant
Formal Definition of a Plan (1)
• A plan is a data structure
consisting of four components:
–A set of plan steps. Each step is
one of the operators for the problem
–A set of step ordering constraints.
notation: Si  Sj
(step Si must occur sometimes before Sj)
Formal Definition of a Plan (2)
–A set of variable binding
constraints
Variable may be substituted by
another variable or a constant
–A set of causal links
Notation: Si  Sj
(the purpose of Si for Sj)
Representations of Plans (1)
• The initial plan
– Describes the unsolved problem
– Consists from two steps START and
FINISH
– Ordering constraint START  FINISH
– The START step has no preconditions.
– The START step’s effect is to add all the
propositions that are true in the initial
state
Representations of Plans (2)
– The START step has null actions associated
with it
– The FINISH step has the goal state as its
precondition
– The FINISH step has no effects
– The FINISH step has null actions associated
with it
• The planner starts with the initial plan and
manipulate it until a plan that is a solution
is constructed
Solution (1)
(a Complete, Consistent Plan)
• A complete plan
–A plan in which every precondition
of every step is achieved by some
other step
–A step achieves a condition if the
condition is one of the effects of the
step, and if no other step can
possibly cancel out the condition
Solution (2)
(a Complete, Consistent Plan)
• A consistent plan
– A plan in which there are no
contradiction in the ordering or binding
constraints
– A contradiction occurs when both Si  Sj
and Sj  Si hold, or both v = A and v = B
hold (v is variable, and A, B are
constants). Both  and = are transitive
Solution (3)
(a Complete, Consistent Plan)
• A solution is a plan that an agent
can execute
• A solution guarantees
achievement of the goal
A Partial-Ordered Planning
Example (1)
• A partial-order regression planner
searches through plan space
• The planner starts with an initial plan
representing the start and finish steps
Start
At(Home), Sells(Record shop, CD), Sells(Baker’s shop, Bread)
Have(CD), Have(Bread), At(home)
Finish
A Partial-Ordered Planning
Example (2)
• The planner adds one more step on each
iteration
• To keep the search focused, the planner
only considers adding steps that allow to
achieve a precondition that has not yet
been achieved (causal links keep track
on it)
• If the search leads to an inconsistent
plan, it backtracks and searches
another branch
A Partial-Ordered Planning
Example (3)
• Definition of the initial state
– Operator (ACTION: Start,
EFFECT: At(Home),
Sells(Record shop, CD),
Sells(Baker’s shop, Bread)
• Definition of the final state
– Operator (ACTION: Finish,
PRECONDITION: Have(CD),
Have(Bread), At(Home))
A Partial-Ordered Planning
Example (4)
• Definition of the actions
–Operator (ACTION: Go(shop),
PRECONDITION: At(Home),
EFFECT: (At(shop)  At(Home))
–Operator (ACTION: Buy(x),
PRECONDITION: At(shop) 
Sells(shop, x), EFFECT: Have(x))
A Partial-Ordered Planning
Example (5)
• The first step: a partial plan that
achieves two preconditions of Finish
state
Ordering constraints
Start
At(shop), Sells(shop, CD)
At(shop), Sells(shop, Bread)
Buy(CD)
Causal
links
Ordering
constraint
Have(CD)
Buy(Bread)
Have(Bread)
Finish
At(Home)
Ordering
constraint
A Partial-Ordered Planning
Example (6)
• Remark: Just to simplify the
graphical representation, in the
following Figures ordering
constraints are supposed to be
depicted underneath of the
causal links
A Partial-Ordered Planning
Example (7)
• The second step: a partial plan
that achieves two preconditions
of Finish state
Start
At(Record shop),
Sells(Record shop, CD)
At(Baker’s shop),
Sells(Baker’s shop, Bread)
Buy(CD)
Have(CD)
Buy(Bread)
Have(Bread)
Finish
At(Home)
A Partial-Ordered Planning
Example (8)
• The third step: extension of the plan by
choosing two Go actions
Start
At(x)
At(x)
Go(Record shop)
Go(Baker’s shop)
At(Baker’s shop),
Sells(Baker’s shop, Bread)
At(Record shop),
Sells(Record shop, CD)
Buy(CD)
Have(CD)
Buy(Bread)
Have(Bread)
Finish
At(Home)
A Partial-Ordered Planning
Example (9)
• The fourth step: a dead end
Start
At(Home)
At(Home)
Go(Record shop)
Go(Baker’s shop)
At(Baker’s shop),
Sells(Baker’s shop, Bread)
At(Record shop),
Sells(Record shop, CD)
Buy(CD)
Have(CD)
Buy(Bread)
Have(Bread)
Finish
At(Home)
A Partial-Ordered Planning
Example (10)
• Remark: The step Go(Record shop)
adds the condition At(Record shop),
but it also deletes the condition
At(Home), that is, now agent can not
go from home to the baker’s shop
(whichever Go step comes first will
delete the At(Home) precondition
on the other step)
A Partial-Ordered Planning
Example (11)
• The fifth step: back up to try a different choice at
some earlier point in the planning process
Start
At(Home)
At(Record Shop)
Go(Baker’s shop)
Go(Record shop)
At(Baker’s shop),
Sells(Baker’s shop, Bread)
At(Record shop),
Sells(Record shop, CD)
Buy(CD)
Have(CD)
Buy(Bread)
Have(Bread)
Finish
At(Home)
A Partial-Ordered Planning
Example (12)
• Remark: Go(Baker’s shop) step
threatens the At(Record shop)
precondition of the Buy(CD) step.
Constraining Go(Baker’s shop) step
to come after Buy(CD) step will not
allow the agent to go from the record
shop to the baker’s shop without first
buying the CD
A Partial-Ordered Planning
Example (13)
• The sixth step: adding a Go(Home) step
Start
At(Home)
At(Record Shop)
Go(Baker’s shop)
Go(Record shop)
At(Record shop),
Sells(Record shop, CD)
Buy(CD)
At(Baker’s shop),
Sells(Baker’s shop, Bread)
Buy(Bread)
At(Baker’s shop)
Go(Home)
Have(CD)
Have(Bread)
Finish
At(Home)
A Partial-Ordered Planning
Example (14)
• Remark: Notice that Go(Home)
step is ordered to be after
Buy(Bread) step
A Partial-Ordered Planning Example (15)
• A complete solution plan (totally ordered)
Start
At(Home)
Go(Record shop)
At(Record shop), Sells(Record shop, CD)
Buy(CD)
At(Record shop)
Go(Baker’s shop)
At(Baker’s shop, Sells(Baker’s shop, Bread)
Buy(Bread)
At(Baker’s shop)
Go(Home)
Have(CD), At(Home), Have(Bread)
Finish
A Partial-Ordered Planning (1)
• Backtrack for trying another choice when
dead end is reached
– Causal links in a partial plan are protected
links
Definition: A causal link is protected by
ensuring that threats, that is, steps that
might delete the protected condition, are
ordered to come before or after the
protected link
A Partial-Ordered Planning (2)
• Example
Step 1
Step 3
PRECONDITION
Step 2
PRECONDITION
A causal link
Step1Step2
is threatened
by Step 3
because one
effect of Step 3
is to delete
condition
A Partial-Ordered Planning (3)
• Threat resolving by addition of ordering
constraints
> Demotion
Step 3
> Promotion
Step 1
precondition
precondition
Step 1
precondition
Step 2
Step 2
Step 3
precondition
A Partial-Order Planning
Algorithm (1)
• POP algorithm
– Starts with minimal partial plan (starts with
goals that must be achieved)
– Extends the plan by achieving a precondition
of a needed step (find operators)
(Some operator that achieves the precondition
is chosen either from the existing steps of the
plan or from the set of operators)
A Partial-Order Planning
Algorithm (2)
– Records the causal link for the newly
achieved precondition
– If the new step threatens an existing
causal link or an existing step threatens
the new causal link the threat resolving
is applied
– If at any point the algorithm fails to find a
relevant operator or resolve a threat, it
backtracks to a previous choice point
– If all the preconditions of all the steps
are achieved, it is a solution
A Partial-Order Planning
Algorithm (3)
• POP is a regression planner. It
is sound and complete