Cyrille Berger package sample; import java.awt.Color; import java.awt.Graphics2D; import java.awt.Polygon; import java.util.Collection; import java.util.HashSet; import rescuecore2.misc.gui.ScreenTransform; import rescuecore2.standard.view.StandardViewLayer; import rescuecore2.view.RenderedObject; public class SampleLayer extends StandardViewLayer {
@Override
public String getName() { return "SampleLayer"; }
@Override
public Collection<RenderedObject> render(Graphics2D g,
ScreenTransform arg1, int arg2, int arg3) {
// TODO Auto-generated method stub
Collection<RenderedObject> objects = new HashSet<RenderedObject>();
int count = 4;
int[] xs = new int[count];
int[] ys = new int[count];
xs[0] = 100; ys[0] = 100;
xs[1] = 1000; ys[1] = 100;
xs[2] = 1000; ys[2] = 2000;
xs[3] = 100; ys[3] = 2000;
Polygon shape = new Polygon(xs, ys, count);
g.setColor(Color.black);
g.fill(shape);
objects.add(new RenderedObject(null, shape));
return objects;
}
}
3 / 77
Be aware, they are multiple class with the same name, make sure you import the right stuff (for instance if "obj instanceof Road"
How the larger group will be formed?
For the visualisation, add a layer
2 / 77
1
AI Programming: Introduction
2
Introduction to RoboRescue
3
Agents and Agents Architecture
4
Communication
5
Multiagent Decision Making
6
Cooperation And Coordination 1
7
Cooperation And Coordination 2
8
Machine Learning
9
Knowledge Representation
10
Putting It All Together
4 / 77

Acquire knowledge on what is an agent.
Acquire knowledge on what are the different agent architecture and how they take decision
5 / 77
Agents
An Overview of Decision Making
Agent Architectures
Deliberative Architecture
Reactive Architecture
Hybrid
Summary
6 / 77
Agents are autonomous : capable of acting independently and exhibiting control over their internal state
An agent is a (computer) system that is situated in some environment and that is capable of autonomous action in this environment in order to meet its delegated objectives .
8 / 77

Should an agent be able to learn ?
Should an agent be intelligent ?
9 / 77
Reactivity
Intelligent agents are able to perceive their environment, and respond in a timely fashion to changes that occur in it in order to meet its delegated objectives.
Proactivity
Intelligent agents are able to exhibit goal-directed behavior by taking the initiative in order to meet its delegated objectives.
Social Ability
Intelligent agents are capable of interacting (cooperating, coordinating and negotiating) with other agents (and possible humans) in order to meet its delegated objectives.
11 / 77
10 / 77
Cooperation is working together as a team to achieve a shared goal .
Often prompted either by the fact that no one agent can achieve the goal alone, or that cooperation will obtain a better result (e.g., get result faster).
Coordination is managing the interdependencies between activities .
Negotiation is the ability to reach agreements on matters of common interest .
Typically involves offer and counter-offer, with compromises made by participants.
12 / 77

The philosopher Daniel Dennett coined the term intentional system to describe entities “ whose behaviour can be predicted by the method of attributing belief, desires and rational acumen ”.
Is it legitimate or useful to attribute beliefs , desires , and so on, to computer systems?
With very complex systems a mechanistic explanation of its behaviour may not be practical or available . But, the more we know about a system, the less we need to rely on animistic, intentional explanations of its behaviour.
As computer systems become ever more complex, we need more powerful abstractions and metaphors to explain their operation
— low level explanations become impractical. The intentional stance is such an abstraction, which provide us with a convenient and familiar way of describing, explaining, and predicting the behaviour of complex systems.
13 / 77
Structural
Relation to previous level
Machine
Language
Program
Structured
Programming
Subroutine
Object-
Oriented
Programming
Object
Bound unit of program
Subroutine
+ persistent local state
Agent-
Oriented
Programming
Agent
Object + independent thread of control + initiative
15 / 77
Object Oriented Programming vs Multi-Agent Systems
Object-Oriented Programming
Objects are passive , i.e. an object has no control over method invocation
Objects are designed for a common goal
Typically integrated into a single thread
Multi-Agent Systems
Agents are autonomous , i.e.
pro-active
Agents can have diverging goals, e.g.
coming from different organizations
Agents have own thread of control
14 / 77
Based on the agent definition:“ An agent is an entity whose state is viewed as consisting of mental components such as beliefs, capabilities, choices, and commitments.
”
The mental constructs will appear in the programming language itself.
The semantics will be related to the semantics of the mental constructs.
A computation will consist of agents performing speech-acts on each other.
16 / 77

AGENT0 is implemented in LISP
Each agent in AGENT0 has 4 components: a set of capabilities (things the agent can do) a set of initial beliefs a set of initial commitments (things the agent will a set of commitment rules
The key component, which determines how the agent acts, is the commitment rule set
17 / 77
One rule could be: if I receive a message from agent which requests me to do action at time , and I believe that: agent is currently a friend
I can do the action
At time , I am not committed to doing any other action then commit to doing action at time
19 / 77
Each commitment rule contains a message condition a mental condition an action
On each ‘agent cycle’...
The message condition is matched against the messages the agent has received
The mental condition is matched against the beliefs of the agent
If the rule fires, then the agent becomes committed to the action (the action gets added to the agent’s commitment set)
18 / 77
COMMIT(
( agent, REQUEST, DO(time, action)
), ;;; msg condition
( B, [now, Friend agent] AND
CAN(self, action) AND
NOT [time, CMT(self, anyaction)]
), ;;; mental condition
self,
DO(time, action)
)
20 / 77

Explicit decision
Decision trees
Rules
Automata
Single agent task specification languages
Decision theoretic decision
Markov Decision Processes (MDP)
Partially Observable Markov Decision Processes (POMDP)
Declarative (logic-based) decision
Theorem Proving
Planning
Constraint satisfaction
22 / 77
Explicit
Mutual modeling
Norms
Organizations and Roles
Multiagent task specification languages
Decision theoretic
Decentralized POMDPs (Dec-POMDP)
Game theoretic
Auctions
Declarative
Multiagent planning
Distributed constraint satisfaction
23 / 77

“[A] particular methodology for building [agents]. It specifies how
… the agent can be decomposed into the construction of a set of component modules and how these modules should be made to interact.” (P. Maes 1991)
25 / 77
Three types: deliberative (symbolic/ reactive hybrid.
26 / 77
We define a deliberative (or reasoning) agent or agent architecture to be one that: contains an explicitly represented, symbolic model of the world and makes decisions via symbolic reasoning.
Views agents as knowledge-based
We can say that a deliberative agent makes an action in three steps:
Sense
Plan
Act
28 / 77

“Practical reasoning is a matter of weighing conflicting considerations for and against competing options, where the relevant considerations are provided by what the agent desires/values/cares about and what the agent believes.” Bratman
29 / 77
Agents need to determine ways of achieving intentions.
If I have an intention to φ you would expect me to devote resources to deciding how to bring about
Intentions provide a filter for adopting other intentions, which must not conflict.
If I have an intention to φ, you would not expect me to adopt an intention ψ such φ and ψ are mutually exclusive.
31 / 77
Human practical reasoning consists of two activities: deliberation - deciding what state of affairs we want to achieve; means-ends reasoning - deciding how to achieve these states of affairs.
The output of deliberation is intentions.
30 / 77
Agents believe their intentions are possible.
An agent believes there is at least some way that the intentions could be brought about.
Agents do not believe they will not bring about their intentions.
It would not be rational of me to adopt an intention to φ if I believed φ was not possible.
32 / 77

Under certain circumstances, agents believe they will bring about their intentions.
It would not normally be rational of me to believe that I would bring my intentions about; intentions can fail.
Moreover, it does not make sense that if I believe φ is inevitable that I would adopt it as an intention.
Agents track the success of their intentions, and are inclined to try again if their attempts fail.
If an agent's first attempt to achieve φ fails, then all other things being equal, it will try an alternative plan to achieve
φ.
33 / 77
Intentions drive means-ends reasoning
Intentions persist
Intentions constrain future deliberation
Intentions influence beliefs upon which future practical reasoning is based
35 / 77
Agents need not intend all the expected side effects of their intentions.
If I believe φ ⇒ ψ and I intend that φ, I do not necessarily intend ψ also. (Intentions are not closed under implication.)
This last problem is known as the side effect or package deal problem. I may believe that going to the dentist involves pain, and I may also intend to go to the dentist - but this does not imply that I intend to suffer pain!
34 / 77
Given: a representation of goal/intention to achieve a representation of actions it can perform a representation of the environment generate a plan to achieve the goal.
36 / 77

1 while true
2
observe the world
3
update internal world
4 model
5
6
7
deliberate about what intentions to achieve next
use means-ends reasoning to get a plan for the intention
execute the plan end while
1 while true
2
get
3 next
4
B :=
I := deliberate(B);
5
P :=
6
execute(P);
7 end
37 / 77
The deliberate function can be decomposed into two distinct functional components: option generation in which the agent generates a set of possible alternatives represent option generation via a function, options, which takes the agent’s current beliefs and current intentions, and from them determines a set of options (= desires).
filtering in which the agent chooses between competing alternatives, and commits to achieving them.
In order to select between competing options, an agent uses a filter function.
39 / 77
How does an agent deliberate?
begin by trying to understand what the options available to you are choose between them, and commit to some.
Chosen options are then intentions.
38 / 77
1 while true
2
get
3 next percept
4
I :=
5 deliberate(B);
P := plan(B,I);
6
execute(P);
7 end
3
4
5
1 while true do
2
get next percept p;
B := brf(B,p);
D := options(B,I);
I := filter(B,D,I);
6
P := plan(B,I);
7
execute(P);
8 end while
40 / 77

Cleaning robot
Percepts p = {dirt, X,
Actions A =
{turnRight, forward,
Start: (0,0,North)
Goal: searching and cleaning dirt
41 / 77
After filtering the intention is:
{clean, 0,
Plan P:
{turnRight, forward, forward,
43 / 77
Beliefs B are:
{dirt, 0, 2} {dirt, 1,
{pos, 0, 0,
Options D:
{clean, 0,
{clean, 1,
42 / 77
Blind commitment a blindly committed agent will continue to maintain an intention until it believes the intention has actually been achieved. Blind commitment is also sometimes referred to as fanatical commitment.
Single-minded commitment a single-minded agent will continue to maintain an intention until it believes that either the intention has been achieved, or else that it is no longer possible to achieve the intention.
Open-minded commitment an open-minded agent will maintain an intention as long as it is still believed possible.
44 / 77

An agent has commitment both to ends
(i.e. of wishes to bring about), and means
(i.e., the mechanism via which the agent wishes to achieve the state of affairs).
Currently, our agent control loop is overcommitted, both to means and ends.
Modification: replan if ever a plan goes wrong.
45 / 77
Still overcommitted to intentions: never stops to consider whether or not its intentions are appropriate.
Modification: stop to determine whether intentions have succeeded or whether they are impossible (singleminded commitment).
47 / 77
2
1 while true
get next percept p;
3
B :=
4
D :=
5 options(B,I);
I := filter(B,D,I);
6
P := plan(B,I);
7 execute(P);
8 end
1
2
3
4
5
6
7
8
9 while true
get next percept
B :=
D :=
I := while not empty(P) do a := first(P); execute(a); get next percept p;
10 B := brf(B,p);
11 if not sound(P,B,I) then
12 P := plan(B,I);
13 end while
14 end
46 / 77
2
1 while true do
3
get next percept p;
4
B := brf(B,p);
5
D := options(B,I);
6
I := filter(B,D,I);
7
while not empty(P) do
a := first(P); execute(a);
8
P := rest(P);
9
get next percept p;
10
B := brf(B,p);
11
if not sound(P,B,I) then
12
P := plan(B,I);
13
end while
14 end while
2
1 while true
3
get next percept
4
B :=
5
D :=
6
I := while not empty(P) or succeeded(B,I) or
7 impossible(B,I) do
8
a := first(P);
9
P :=
10
11
get next percept
B :=
12
if not sound(P,B,I)
P :=
13
end
14 end
48 / 77

Our agent gets to reconsider its intentions once every time around the outer control loop, i.e., when: it has completely executed a plan to achieve its current intentions; or it believes it has achieved its current intentions; or it believes its current intentions are no longer possible.
This is limited in the way that it permits an agent to reconsider its intentions.
Modification: Reconsider intentions after executing every action.
49 / 77
But intention reconsideration is costly! A an agent that does not stop to reconsider its intentions sufficiently often will continue attempting to achieve its intentions even after it is clear that they cannot be achieved, or that there is no longer any reason for achieving them; an agent that constantly reconsiders its attentions may spend insufficient time actually working to achieve them, and hence runs the risk of never actually achieving them.
Solution: incorporate an explicit meta-level control component, that decides whether or not to reconsider.
51 / 77
1
2
3
4
5
6
7
8 while true
get next percept
B :=
D :=
I :=
while not empty(P) or succeeded(B,I) or impossible(B,I) do
a := first(P);
P :=
9
10
get next percept
11
12
B :=
if not sound(P,B,I)
P :=
13
end
14 end
2
1 while true
3
get next percept
4
B :=
5
D :=
6
I :=
while not empty(P) or succeeded(B,I) or
7 impossible(B,I)</font>
8
a := first(P);
9
P :=
10
11
get next percept
B :=
12
D := options(B,I);
I := filter(B,D,I);
13
if not sound(P,B,I) then
14
15
P := plan(B,I);
end end
50 / 77
1
2
3
4
5
6
7
8 while true
get next percept
B :=
D :=
I :=
while not empty(P) or succeeded(B,I) or impossible(B,I) do
a := first(P);
P :=
9
10
get next percept
11
12
B :=
D :=
I :=
13
if not sound(P,B,I)
14
P :=
15
end
16 end
2
1 while true do
3
get next percept p;
4
B := brf(B,p);
5
D := options(B,I);
6
I := filter(B,D,I);
while not empty(P) or succeeded(B,I) or
7 impossible(B,I)</font> do
a := first(P); execute(a);
8
P := rest(P);
9
get next percept p;
10
B := brf(B,p);
11 if reconsider(B,I) then
12
D := options(B,I); I := filter(B,D,I);
13
if not sound(P,B,I) then
14
P := plan(B,I);
15 end while
16 end while 52 / 77

Kinny and Georgeff's experimentally investigated effectiveness of intention reconsideration strategies.
Two different types of reconsideration strategy were used: bold agents never pause to reconsider intentions, and cautious agents stop to reconsider after every action.
Dynamism in the environment is represented by the rate of world change, ɣ.
53 / 77
The representation/reasoning problem
How to symbolically represent information about complex real-world entities and
How to translate the perceived world into an accurate, adequate symbolic description, in time for that description to be useful
… vision, speech recognition, learning.
How to get agents to reason with this information in time for the results to be useful
… knowledge representation, automated reasoning, planning.
During computation, the dynamic world might change and thus the solution not valid anymore!
How to represent temporal information, e.g., how a situation changes over time?
55 / 77
If ɣ is low (i.e., the environment does not change quickly), then bold agents do well compared to cautious ones. This is because cautious ones waste time reconsidering their commitments while bold agents are busy working towards - and achieving their intentions.
If ɣ is high (i.e., the environment changes frequently), then cautious agents tend to outperform bold agents.
This is because they are able to recognize when intentions are doomed, and also to take advantage of serendipitous situations and new opportunities when they arise.
54 / 77
There are many unsolved problems associated with symbolic AI.
“Most of what people do in their day to day lives is not problem-solving or planning, but rather it is routine activity in a relatively benign, but certainly dynamic, world.” (Brooks, 1991)
These problems have led some researchers to question the viability of the whole paradigm, and to the development of reactive
Although united by a belief that the assumptions underpinning mainstream AI are in some sense wrong, reactive agent researchers use many different techniques.
56 / 77

Brooks has put forward three theses:
2
1
Intelligent behaviour can be generated without explicit representations of the kind that symbolic
AI proposes.
Intelligent behaviour can be generated without explicit abstract reasoning of the kind that
3 symbolic AI proposes.
Intelligence is an emergent property of certain complex systems.
59 / 77
An agent must cope appropriately and in a timely fashion with changes in its environment.
An agent should be robust with respect to its environment.
An agent should be able to maintain multiple goals and switch between
An agent should do something, it should have some purpose in being.
58 / 77
Situatedness and embodiment: The world is its own best model and it gives the agent a firm ground for its reasoning.
Intelligence and emergence: “Intelligent” behaviour arises as a result of an agent's interaction with its environment. Also, intelligence is “in the eye of the beholder”; it is not an innate, isolated property.
60 / 77

To illustrate his ideas, Brooks built some robots based on his subsumption architecture.
A subsumption architecture is a hierarchy of task-accomplishing behaviours.
Each behaviour is a rather simple rule-like structure.
Each behaviour “competes” with others to exercise control over the agent.
61 / 77
63 / 77
Traditional decomposition into functional modules:
Decomposition based on task achieving behaviors:
62 / 77
Lower layers represent more primitive kinds of behaviour, (such as avoiding obstacles), and have precedence over layers further up the hierarchy.
The resulting systems are, in terms of the amount of computation they do, extremely simple.
Some of the robots do tasks that would be impressive if they were accomplished by symbolic AI systems.
64 / 77

65 / 77
The local environment must contain enough information to make a decision.
Hard to take non-local information into account.
Behavior emerges from interactions ⇒ How to engineer the system in the general case?
How to model long-term decisions?
How to implemented varying goals?
Hard to engineer, especially large systems with many layers that interacts.
67 / 77
Simplicity, i.e. modules have high expressiveness
Computational tractability
Robustness against failure, i.e.
possibility of modeling redundancies
Overall behavior emerges from interactions
66 / 77

A hybrid system is neither a completely deliberative nor completely reactive approach.
An obvious approach is to build an agent out of two
(or more) subsystems: a deliberative one, containing a symbolic world model, which develops plans and makes decisions in the way proposed by symbolic AI; and a reactive one, which is capable of reacting to events without complex reasoning.
Often, the reactive component is given some kind of precedence over the deliberative one. This kind of structuring leads naturally to the idea of a layered architecture.
69 / 77
In a layered architecture, an agent's control subsystems are arranged into a hierarchy, with higher layers dealing with information at increasing levels of abstraction.
A key problem in layered architectures is what kind of control framework to embed the agent's subsystems in, to manage the interactions between the various layers.
Horizontal layering - Layers are each directly connected to the sensory input and action output. In effect, each layer itself acts like an agent, producing suggestions as to what action to perform.
Vertical layering - Sensory input and action output are each dealt with by at most one layer each.
70 / 77
71 / 77 72 / 77

Properties
Internal state (using symbolic representation)
Search-based decision making
Goal directed
Benefits
Nice and clear (logics) semantics
Easy to analyze by proving properties
Problems
Can’t react in a timely manner to events that requires immediate actions. Intractable algorithms.
Hard to create a symbolic representation from continuous sensor data. The anchoring problem.
75 / 77
Originally (1956-1985), pretty much all agents designed within AI were symbolic reasoning agents
Its purest expression proposes that agents use explicit logical reasoning in order to decide what to do
Problems with symbolic reasoning led to a reaction against this — the so-called reactive agents movement, 1985–present
From 1990-present, a number of alternatives proposed: hybrid architectures, which attempt to combine the best of reasoning and reactive architectures
74 / 77
Properties
No explicit world model
Rule-based decision making
Benefits
Efficient
Robust
Problems
The local environment must contain enough information to make a decision.
Easy to build small agents, hard to build agents with many behaviors or rules. Emergent behavior.
76 / 77

Properties
Tries to combine the good parts of both reactive and deliberative architectures.
Usually layered
Benefits
Attacks the problem on different abstraction
Has the benefits of both architecture
Problems
Hard do combine the different
77 / 77