SAC 2002 Tutorial
Henry Hexmoor
Svet Brainov
University of Arkansas
Engineering Hall, Room 328
Fayetteville, AR 72701
University at Buffalo
210 Bell Hall
Buffalo, NY 14260
Hexmoor&Braynov
Contents
Multiagents: Formal and Economic
Morning: Basics

I. Introduction: from DAI to Multiagecy
1. History and perspectives on multiagents (Henry)
2. Architectural theories (Henry)
3. Agent Oriented Software Engineering (Henry)
4. Mobility, reliability, and fault-tolerance (Henry)

II. Enabling Technologies
5. Game Theoretic and Decision Theoretic Agents(Svet)
6. Communication, security (Svet)
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov

Multiagents: Formal and Economic
Contents: Continued
Afternoon: Issues
III. Enabling Technologies
7. Social attitudes: Values, norms, obligations, dependence, control,
responsibility, roles (Henry)
8. Benevolence, Preference, Power, Trust (Svet)
9. Communication, Security(Svet)
10. Agent Adaptation and Learning (Svet)

IV. Closing
11. Trends and open questions (Svet)
12. Concluding Remarks (Svet)
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Outline
1. History and perspectives on multiagents
2. Agent Architecture
3. Agent Oriented Software Engineering
4. Mobility
5. Autonomy and Teaming
6. Game Theoretic and Decision Theoretic Agents
7. Social attitudes: Values, norms, obligations, dependence,
control, responsibility, roles
8. Benevolence, Preference, Power, Trust.
9. Communication, Security
10. Agent Adaptation and Learning
11. Trends and Open questions
12. Concluding Remarks
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Definitions
1.
An agent is an entity whose state is viewed as consisting of
mental components such as beliefs, capabilities, choices,
and commitments. [Yoav Shoham, 1993]
2. An entity is a software agent if and only if it
communicates correctly in an agent communication
language. [Genesereth and Ketchpel, 1994]
3. Intelligent agents continuously perform three functions:
perception of dynamic conditions in the environment;
action to affect conditions in the environment; and
reasoning to interpret perceptions, solve problems, draw
inferences, and determine actions. [Hayes-Roth, 1995]
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Definitions
5. An agent is anything that can be viewed as (a)Perceiving
its environment, and (b) Acting upon that
environment [Russell and Norvig, 1995]
6. A computer system that is situated in some environment
and is capable of autonomous action in its environment to
meet its design objectives. [Wooldridge, 1999]
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Agents: A working definition
An agent is a computational system that interacts with one
or more counterparts or real-world systems with the
following key features to varying degrees:
• Autonomy
• Reactiveness
• Pro-activeness
• Social abilities
e.g., autonomous robots, human assistants, service agents
The need is for automation and distributed use of online resources
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Test of Agenthood [Huhns and Singh, 1998]
“A system of distinguished agents should
substantially change semantically if a distinguished
agent is added.”
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Agents vs. Objects
“Objects with attitude” [Bradshaw, 1997]

Agents are similar to objects since they are
computational units that encapsulate a state and
communicate via message passing

Agents differ from objects since they have a
strong sense of autonomy and are active versus
passive.
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Agent Oriented Programming, Yoav Shoham
AOP principles:
1. The state of an object in OO programming has no generic
structure. The state of an agent has a “mentalistic” structure:
it consists of mental components such as beliefs and
commitments.
2. Messages in object-oriented programming are coded in an
application-specific ad-hoc manner. A message in AOP is
coded as a “speech act” according to a standard agent
communication language that is application-independent.
SAC 2002 Tutorial
3/13/02
Multiagents: Formal and Economic
Agent Oriented Programming
Extends Peter Chen’s ER model, Gerd Wagner
Hexmoor&Braynov
•
Different entities may belong to different epistemic categories.
There are agents, events, actions, commitments, claims, and
objects.
•
We distinguish between physical and communicative
actions/events. Actions create events, but not all events are created
by actions.
•
Some of these modeling concepts are indexical, that is, they depend
on the perspective chosen: in the perspective of a particular agent,
actions of other agents are viewed as events, and commitments of
other agents are viewed as claims against them.
SAC 2002 Tutorial
3/13/02
Multiagents: Formal and Economic
Agent Oriented Programming
Extends Peter Chen’s ER model, Gerd Wagner
Hexmoor&Braynov
•
•
•
In the internal perspective of an agent, a commitment refers to a
specific action to be performed in due time, while a claim refers to
a specific event that is created by an action of another agent, and
has to occur in due time.
Communication is viewed as asynchronous point-to-point message
passing. We take the expressions receiving a message and sending a
message as synonyms of perceiving a communication event and
performing a communication act.
There are six designated relationships in which specifically agents,
but not objects, participate: only an agent perceives environment
events, receives and sends messages, does physical actions, has
Commitment to perform some action in due time, and has Claim
that some action event will happen in due time.
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Agent Oriented Programming
Extends Peter Chen’s ER model, Gerd Wagner
•
•
•
•
An institutional agent consists of a certain number of (institutional,
artificial and human) internal agents acting on behalf of it. An
institutional agent can only perceive and act through its internal
agents.
Within an institutional agent, each internal agent has certain rights
and duties.
There are three kinds of duties: an internal agent may have the duty
to full commitments of a certain type, the duty to monitor claims of a
certain type, or the duty to react to events of a certain type on behalf
of the organization.
A right refers to an action type such that the internal agent is
permitted to perform actions of that type on behalf of the
organization.
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Agent Typology
Multiagents: Formal and Economic
• Human agents: Person, Employee, Student, Nurse, or
Patient
• Artificial agents: owned and run by a legal entity
• Institutional agents: a bank or a hospital
• Software agents: Agents designed with software
• Information agent: Data bases and the internet
• Autonomous agents: Non-trivial independence
• Interactive/Interface agents: Designed for interaction
• Adaptive agents: Non-trivial ability for change
• Mobile agents: code and logic mobility
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Agent Typology
Multiagents: Formal and Economic
• Collaborative/Coordinative agents: Non-trivial ability
for coordination, autonomy, and sociability
• Reactive agents: No internal state and shallow
reasoning
• Hybrid agents: a combination of deliberative and
reactive components
• Heterogenous agents: A system with various agent
sub-components
• Intelligent/smart agents: Reasoning and intentional
notions
• Wrapper agents: Facility for interaction with nonSACagents
2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Multi-agency
A multi-agent system is a system that is made up of
multiple agents with the following key features among
agents to varying degrees of commonality and adaptation:
• Social rationality
• Normative patterns
• System of Values
e.g., HVAC, eCommerce, space missions, Soccer, Intelligent Home,
“talk” monitor
The motivation is coherence and distribution of resources.
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Applications of Multiagent Systems

Electronic commerce: B2B, InfoFlow, eCRM

Network and system management agents: E.g., The
telecommunications companies

Real-time monitoring and control of networks: ATM

Modeling and control of transportation systems: Delivery

Information retrieval: online search

Automatic meeting scheduling

Electronic entertainment: eDog
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Applications of Multiagent Systems (cont.)

Decision and logistic support agents:Military and Utility
Companies

Interest matching agents: Commercial sites like Amazon.com

User assistance agents: E.g., MS office assistant

Organizational structure agents: Supply-chain ops

Industrial manufacturing and production: manufacturing cells

Personal agents: emails

Investigation of complex social phenomena such as evolution of
roles, norms, and organizational structures
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Summary of Business Benefits
• Modeling existing organizations and dynamics
• Modeling and Engineering E-societies
• New tools for distributed knowledge-ware
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Three views of Multi-agency
Constructivist: Agents are rational in the sense of Newell’s principle
of individual rationality. They only perform goals which bring them
a positive net benefit without regard to other agents. These are selfinterested agents.
Sociality: Agents are rational in the Jennings’ principle of social
rationality. They perform actions whose joint benefit is greater than
its joint loss. These are self-less, responsible agents.
Reductionist: Agents which accept all goals they are capable of
performing. These are benevolent agents.
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Multi-agency: allied fields
DAI
MAS: (1) online social laws, (2) agents may adopt goals and adapt beyond any problem
DPS: offline social laws
CPS: (1) agents are a ‘team’, (2) agents ‘know’ the shared goal
• In DAI, a problem is being automatically decomposed
among distributed nodes, whereas in multi-agents, each agent
chooses to whether to participate.
• Distributed planning is distributed and decentralized action
selection whereas in multi-agents, agents keep their own
copies
a plan that might include others.
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Multi-agent assumptions and goals
• Agents have their own intentions and the system has
distributed intentionality
• Agents model other agents mental states in their own
decision making
• Agent internals are of less central than agents interactions
• Agents deliberate over their interactions
• Emergence at the agent level and at the interaction level are
desirable
• The goals is to find some principles-for or principled ways to
explore interactions
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Origins of Multi-agent systems
• Carl Hewitt’s Actor model, 1970
• Blackboard Systems: Hearsay (1975), BB1, GBB
• Distributed Vehicle Monitoring System (DVMT, 1983)
• Distributed AI
• Distributed OS
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
MAS Orientations
Sociology
Computational
Organization
Theory
Databases
Formal AI
Economics
Distributed
Problem
Solving
Psychology
Systems
Theory
SAC 2002 Tutorial
Cognitive
Science
Distributed
Computing
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Conferences
• ICMAS 96, 98, 00, 02
• Autonomous Agents 96, 97, 98, 99, 00, 02
• CIA, ATAL, CEEMAS
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Multi-agents in the large versus in the small
• In the small: (Distributed AI) A handful of “smart”
agents with emergence in the agents
• In the large: 100+ “simple” agents with emergence in the
group: Swarms (Bugs) http://www.swarm.org/
SAC 2002 Tutorial
3/13/02
Autonomy
Purposefulness
Henry Hexmoor’s Tree of
Research Issues
Learning
Action Selection
Timeliness
Habituation
Commitment
Skill formation
Automaticity
Perception
Teamwork
Cooperation
Social attitudes
Values, Norms,
Obligations
Inference
Agents
Hexmoor&Braynov
Multiagents: Formal and Economic
Outline
1. History and perspectives on multiagents
2. Agent Architecture
3. Agent Oriented Software Engineering
4. Mobility
5. Autonomy and Teaming
6. Game Theoretic and Decision Theoretic Agents
7. Social attitudes: Values, norms, obligations, dependence,
control, responsibility, roles
8. Benevolence, Preference, Power, Trust.
9. Communication, Security
10. Agent Adaptation and Learning
11. Trends and Open questions
12. Concluding Remarks
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Abstract Architecture
states
action
action
actions
Environment
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Architectures
• Deduction/logic-based
• Reactive
• BDI
• Layered (hybrid)
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Abstract Architectures

An abstract model: <States, Action, S*A>

An abstract view
S
= {s1, s2, …} – environment states
A=

{a1, a2, …} – set of possible actions
This allows us to view an agent as a function
action : S*  A
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Logic-Based Architectures
These agents have internal state
 See and next functions and model decision making by a set of
deduction rules for inference

see : S  P
next : D x P  D
action : D  A
Use logical deduction to try to prove the next action to take
 Advantages

Simple,

elegant, logical semantics
Disadvatages
Computational
complexity
Representing the real world
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Reactive Architectures

Reactive Architectures do not use

symbolic world model

symbolic reasoning

An example is Rod Brooks’s subsumption architecture

Advantages
 Simplicity,

computationally tractable, robust, elegance
Disadvantages
 Modeling
SAC 2002 Tutorial
limitations, correctness, realism
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Reflexive Architectures:
simplest type of reactive architecture

Reflexive agents decide what to do without
regard to history – purely reflexive
action : P  A

Example - thermostat
action(s) =
SAC 2002 Tutorial
{
off
on
if temp = OK
otherwise
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Reflex agent without state
(Russell and Norvig, 1995)
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Reflex agent with state (Russell and Norvig, 1995)
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Goal-oriented agent:
a more complex reactive agent (Russell and Norvig, 1995)
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Utility-based agent:
a complex reactive agent (Russell and Norvig, 1995)
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
BDI: a Formal Method
• Belief: states, facts, knowledge, data
• Desire: wish, goal, motivation (these might conflict)
• Intention: a) select actions, b) performs actions, c)
explain choices of action (no conflicts)
• Commitment: persistence of intentions and trials
• Know-how: having the procedural knowledge for carrying out a task
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Belief-Desire-Intention
Environment
belief
revision
sense
act
Beliefs
generate
options
Desires
SAC 2002 Tutorial
filter
Intentions
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Why is BDI a Formal Method?
• BDI is typically specified in the language of modal logic with
possible world semantics.
• Possible worlds capture the various ways the world might develop.
Since the formalism in [Wooldridge 2000] assumes at least a KD
axiomatization for each of B, D, and I, each of the sets of possible
worlds representing B, D and I must be consistent.
• A KD45 logic with the following axioms:
• K: BDI(a, f j, t)  (BDI(a, f, t)  BDI(a, j, t))
• D: BDI(a, f, t)  not BDI(a, not f, t)
• 4: B(a, f, t)  B( B(a, f, t) )
• 5: (not B(a, f, t))  B( not B(a, f, t))
• K&D is the normal modal system
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
A simplified BDI agent algorithm
1. B = B0;
2. I := I0;
3. while true do
4.
get next percept r;
5.
B := brf(B, r);
// belief revision
6.
D:=options(B,D,I,O);
// determination of desires
7.
I := filter(B, D, I,O);
// determination of intentions
8.
p := plan(B, I);
9.
execute p
// plan generation
10. end while
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Correspondences
• Belief-Goal compatibility:
Des  Bel
• Goal-Intention Compatibility:
Int  Des
• Volitional Commitment:
Int Do  Do
• Awareness of Goals and Intentions:
Des  BelDes
Int  BelInt
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Layered Architectures

Layering is based on division of behaviors into automatic
and controlled.

Layering might be Horizontal (I.e., I/O at each layer) or
Vertical (I.e., I/O is dealt with by single layer)

Advantages are that these are popular and fairly intuitive
modeling of behavior

Dis-advantages are that these are too complex and nonuniform representations
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Outline
1. History and perspectives on multiagents
2. Agent Architecture
3. Agent Oriented Software Engineering
4. Mobility
5. Autonomy and Teaming
6. Game Theoretic and Decision Theoretic Agents
7. Social attitudes: Values, norms, obligations, dependence,
control, responsibility, roles
8. Benevolence, Preference, Power, Trust.
9. Communication, Security
10. Agent Adaptation and Learning
11. Trends and Open questions
12. Concluding Remarks
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Agent-Oriented Software Engineering
 AOSE
is an approach to developing software using
agent-oriented abstractions that models high level
interactions and relationships.
 Agents
are used to model run-time decisions about the
nature and scope of interactions that are not known
ahead of time.
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Designing Agents:
Multiagents: Formal and Economic
Recommendations from H. Van Dyke Parunak’s (1996) “Go to the Ant”: Engineering Principles from Natural
Multi-Agent Systems, Annals of Operations Research, special issue on AI and Management Science.
1. Agents should correspond to things in the problem domain rather than to abstract
functions.
2. Agents should be small in mass (a small fraction of the total system), time (able to
forget), scope (avoiding global knowledge and action).
3. The agent community should be decentralized, without a single point of control or
failure.
4. Agents should be neither homogeneous nor incompatible, but diverse. Randomness
and repulsion are important tools for establishing and maintaining this diversity.
5. Agent communities should include a dissipative mechanism to whose flow they
can orient themselves, thus leaking entropy away from the macro level at which
they do useful work.
6. Agents should have ways of caching and sharing what they learn about their
environment, whether at the level of the individual, the generational chain, or the
overall community organization.
7. Agents should plan and execute concurrently rather than sequentially.
SAC 2002 Tutorial
3/13/02
Organizations
Human organizations are several agents, engaged in multiple
goal-directed tasks, with distinct knowledge, culture, memories,
history, and capabilities, and separate legal standing from that of
individual agents
Computational Organization Theory (COT) models information
production and manipulation in organizations of human and
computational agents
Hexmoor&Braynov
Multiagents: Formal and Economic
Management of Organizational Structure

Organizational constructs are modeled as entities
in multiagent systems

Multiagent systems have built in mechanisms for
flexibly forming, maintaining, and abandoning
organizations

Multiagent systems can provide a variety of
stable intermediary forms in rapid systems
development
SAC 2002 Tutorial
3/13/02
7.2.1 Agent and Agency
Hexmoor&Braynov
Multiagents: Formal and Economic
AOSE Considerations

What, how many, structure of agent?

Model of the environment?

Communication? Protocols? Relationships?
Coordination?
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Stages of Agent-Oriented Software Engineering
A.
Requirements: provided by user
B.
Analysis: objectives and invariants
C.
Design: Agents and Interactions
D.
Implementation: Tools and techniques
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
KoAS- Bradshaw, et al
Knowledge
(Facts) represent Beliefs in which the agent
has confidence about
Facts and Beliefs may be held privately or be shared.
Desires represent goals and preferences that motivate
the agent to act
Intentions represent a commitment to perform an
action.
There is no exact description of capabilities
Life cycle: birth, life, and death (also a Cryogenic state)
Agent Types: KaOS, Mediation (KaOS and outside) ,
Proxy (mediator between two KAOS agents), Domain
Manager (agent registration), and Matchmaker (mediator
of services)
Omitted: Emotions, Learning, agent relationships,
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Gaia- Wooldridge, et al
The Analysis phase:
 Roles model:
- Permissions (resources)
- Responsibilities (Safety properties and Liveliness
properties)
- Protocols
 Interactions model: purpose, initiator, responder, inputs,
outputs, and processing of the conversation
The Design phase:
Agent model
 Services model
Acquaintance model
Omitted: Trust, Fraud, Commitment, and Security.
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
TAEMS: Keith Decker and Victor Lesser

The agents are simple processors.
Internal structure of agents include (a) beliefs
(knowledge) about task structure, (b) states, (c)
actions, (d) a strategy which is constantly being
updated, of what methods the agent intends to
execute at what time.


Omitted: Roles, Skills or Resources.
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
BDI based Agent-Oriented Methodology
(KGR) Kinny, Georgeff and Rao
External viewpoint: the social system
structure and dynamics.

Agent Model + Interaction Model.

Independent of agent cognitive model and
communication

Internal viewpoint: the Belief Model, the Goal
Model, and the Plan Model.

Beliefs: the environment, internal state, the
actions repertoire

Goals: possible goals, desired events

Plans: state charts

SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
MaSE – Multi-agent Systems Engineering, DeLoach
Domain Level Design (Use AgML for Agent type
Diagram, Communication Hierarchy Diagram, and
Communication class Diagrams.)
 Agent Level Design (Use AgDL for agent
conversation)
 Component Design AgDL
 System Design AgML
 Languages:
AgML (Agent Modeling Language- a graphical
language)
AgDL (Agent Definition Language- the system level
behavior and the internal behavior of the agent)
 Rich in communication, poor in social structures

SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Scott DeLoach’s MaSE
Roles
Tasks
Agent Class
Diagram
Sequence
Diagrams
Conversation
Diagram
Internal Agent
Diagram
Deployment
Diagram
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
The TOVE Project (1998) ; Mark Fox, et al.
• Organizational hierarchy: Divisions and sub-divisions
• Goals, sub-goals, their hierarchy (using AND & OR)
• Roles, their relations to skills, goals, authority, processes, policies
• Skills, and their link to roles
• Agents, their affiliation with teams and divisions Commitment, Empowerment
• Communication links between agents: sending and receiving information.
Communication at three levels: information, intentions (ask, tell, deny…), and
conventions
(semantics). Levels 2 & 3 are designed using speech act.
• Teams as temporary group of agents
• Activities and their states, the connection to resources and the constraints.
• Resources and their relation to activities and activities states
• Constraints on activities (what activities can occur at a specific situation and a
specific time)
• Time and the duration of activities. Actions occur at a point in time and they
have duration.
• Situation
Shortcomings: central decision making
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Agent-Oriented Programming (AOP): Yoav Shoham
AGENT0 is the first AOP and the logical component of this
language is a quantified multi-modal logic.
•
Mental state: beliefs, capabilities, and commitments (or
obligations).
•
Communication: ‘request’ (to perform an action), ‘unrequest’
(to refrain from action), and ‘inform’ (to pass information).
•
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
The MADKIT Agent Platform Architecture:
Olivier Gutknecht Jacques Ferber

Three core concepts : agent, group, and role.

Interaction language

Organizations: a set of groups
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Outline
1. History and perspectives on multiagents
2. Agent Architecture
3. Agent Oriented Software Engineering
4. Mobility
5. Autonomy and Teaming
6. Game Theoretic and Decision Theoretic Agents
7. Social attitudes: Values, norms, obligations, dependence,
control, responsibility, roles
8. Benevolence, Preference, Power, Trust.
9. Communication, Security
10. Agent Adaptation and Learning
11. Trends and Open questions
12. Concluding Remarks
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Mobile Agents
[Singh, 1999] A computation that can change its location of execution
(given a suitable underlying execution environment), both
code
program state
[Papaioannou, 1999] A software agent that is able to migrate from one
host to another in a computer network is a mobile agent.
[IBM] Mobile network agents are programs that can be dispatched from
one computer and transported to a remote computer for execution.
Arriving at the remote computer, they present their credentials and
obtain access to local services and data. The remote computer may also
serve as a broker by bringing together agents with similar interests and
compatible goals, thus providing a meeting place at which agents can
interact.
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Mobile Agent Origins
- Batch Jobs
- Distributed Operating System (migration is
transparent to the user.)
- Telescript [General Magic, Inc. USA, 1994]
migration of an executing program for use of
local resources
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
A paradigm shift: Distributed Systems versus mobile code
Instead of masking the physical location of a component, mobile
code infrastructures make it evident.
Code mobility is geared for Internet-scale systems ... unreliable
Programming is location aware ...location is available to the programmer
Mobility is a choice ...migration is controlled by the programmer or at runtime by the
agent
Load balancing is not the driving force ...instead flexibility, autonomy and
disconnected operations are key factors
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
A paradigm comparison:
2 Components, 2 Hosts, a Logic, a Resource, Messages, a Task
Remote Computation
In remote computation, components in the system are static, whereas
logic can be mobile. For example, component A, at Host HA,
contains the required logic L to perform a particular task T, but does
not have access to the required resources R to complete the task. R
can be found at HB, so A forwards the logic to component B, which
also resides at HB. B then executes the logic before returning the
result to A. E.g., batch entries.
HA
L, T
HA
L
SAC 2002 Tutorial
HB
R
L
result
HB Compute
R
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
A paradigm comparison:
2 Components, 2 Hosts, a Logic, a Resource, Messages, a Task
Code on Demand
In Code on Demand, component A already has access to resource R.
However, A (or any other components at Host A) has no idea of the
logic required to perform task T. Thus, A sends a request to B for it
to forward the logic L. Upon receipt, A is then able to perform T. An
example of this abstraction is a Java applet, in which a piece of code
is downloaded from a web server by a web browser and then
executed.
HA
R
Compute
SAC 2002 Tutorial
HA
R
HB
L
Send L
L
HB
L
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
A paradigm comparison:
2 Components, 2 Hosts, a Logic, a Resource, Messages, a Task
Mobile Agents
With the mobile agent paradigm, component A already has the logic L required to
perform task T, but again does not have access to resource R. This resource can be
found at HB. This time however, instead of forwarding/requesting L to/from another
component, component A itself is able to migrate to the new host and interact locally
with R to perform T. This method is quite different to the previous two examples, in this
instance an entire component is migrating, along with its associated data and logic. This
is potentially the most interesting example of all the mobile code abstractions. There are
currently no contemporary examples of this approach, but we examine its capabilities in
the next section.
HA
L
HA
L
SAC 2002 Tutorial
HB
R
A moves
A returns
HB
R
Compute
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
A paradigm comparison:
2 Components, 2 Hosts, a Logic, a Resource, Messages, a Task
Client/Server
Client/Server is a well known architectural abstraction that has been
employed since the first computers began to communicate. In this
example, B has the logic L to carry out Task T, and has access to resource
R. Component A has none of these, and is unable to transport itself.
Therefore, for A to obtain the result of T, it must resort to sending a
request to B, prompting B to carry out Task T. The result is then
communicated back to A when completed.
HA
HA
HB
L, R
request
result
SAC 2002 Tutorial
HB
L, R
Compute
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Problems in distributed Systems: J. Waldo
Latency: Most obvious, Least worrisome
Memory: Access, Unable to use pointers, Because memory
is both local and remote, call types have to differ, No
possibility of shared memory
Partial Failure: Is a defining problem of distributed
computing, Not possible in local computing,
Concurrency: Adds significant overhead to programming
model, No programmer control of method invocation
order
we should treat local and remote objects differently.
Waldo, J., Wyant, G., Wollrath, A., Kendall, S., “A note on
distributed computing”, Sun Microsystems Technical Report SML
94-29, 1994.
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Mobile Agent Toolkit from IBM: Basic concepts
Aglet. An
aglet is a mobile Java object that visits aglet-enabled hosts in a computer
network. It is autonomous, since it runs in its own thread of execution after arriving at a
host, and reactive, because of its ability to respond to incoming messages.
Proxy. A proxy is a representative of an aglet. It serves as a shield for the aglet that
protects the aglet from direct access to its public methods. The proxy also provides
location transparency for the aglet; that is, it can hide the aglet’s real location of the
aglet.
Context. A context is an aglet's workplace. It is a stationary object that provides a
means for maintaining and managing running aglets in a uniform execution
environment where the host system is secured against malicious aglets. One node in a
computer network may run multiple servers and each server may host multiple contexts.
Contexts are named and can thus be located by the combination of their server's address
and their name.
Message. A message is an object exchanged between aglets. It allows for synchronous
as well as asynchronous message passing between aglets. Message passing can be used
by aglets to collaborate and exchange information in a loosely coupled fashion.
Future reply. A future reply is used in asynchronous message-sending as a handler to
receive a result later asynchronously.
Identifier. An identifier is bound to each aglet. This identifier is globally unique and
immutable throughout the lifetime of the aglet.
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Mobile Agent Toolkit from IBM: Basic operations
Creation. The
creation of an aglet takes place in a context. The new aglet is assigned
an identifier, inserted into the context, and initialized. The aglet starts executing as soon
as it has been successfully initialized.
Cloning. The cloning of an aglet produces an almost identical copy of the original
aglet in the same context. The only differences are the assigned identifier and the fact
that execution restarts in the new aglet. Note that execution threads are not cloned.
Dispatching. Dispatching an aglet from one context to another will remove it from its
current context and insert it into the destination context, where it will restart execution
(execution threads do not migrate). We say that the aglet has been “pushed” to its new
context.
Retraction. The retraction of an aglet will pull (remove) it from its current context and
insert it into the context from which the retraction was requested.
Activation and deactivation. The deactivation of an aglet is the ability to temporarily
halt its execution and store its state in secondary storage. Activation of an aglet will
restore it in a context.
Disposal. The disposal of an aglet will halt its current execution and remove it from its
current context.
Messaging. Messaging between aglets involves sending, receiving, and handling
messages synchronously as well as asynchronously.
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Outline
1. History and perspectives on multiagents
2. Agent Architecture
3. Agent Oriented Software Engineering
4. Mobility
5. Autonomy and Teaming
6. Game Theoretic and Decision Theoretic Agents
7. Social attitudes: Values, norms, obligations, dependence,
control, responsibility, roles
8. Benevolence, Preference, Power, Trust.
9. Communication, Security
10. Agent Adaptation and Learning
11. Trends and Open questions
12. Concluding Remarks
SAC 2002 Tutorial
3/13/02
Autonomy
•Target and Context: Autonomy is only meaningful in terms of
specific targets and within given contexts.
•Capability: Autonomy only makes sense if an agent has a capability
toward a target. E.g, a rock is not autonomous
•Sources of Autonomy:
Endogenous: Self liberty, Desire, Experience, Motivations
Exogenous: Social, Deontic liberty, Environments
•Implementations: Off-line and by design, Online with fixed cost
analysis, Online learning
Perspectives on Autonomy
Cognitive Science and AI
Organizational Science
Communication
Software Engineering
Hexmoor&Braynov
Multiagents: Formal and Economic
Autonomy and Communication
Detection
and expression of autonomies requires sharing
understanding of social roles and personal relationships among the
participating agents, e.g., agents with positive relationships will
would change their autonomies to accommodate one another
The
form of the directive holds clues for autonomy, e.g., specificity
in “Do x with a wrench and slowly.”
The
content of the directive and the responses to it contribute to the
autonomy, e.g., “Do x soon.”
An
agent’s internal mechanism for autonomy determination affects
the detection, expression, and harmony of autonomies, e.g., an
agent’s moods, drives, temperaments, …
SAC 2002 Tutorial
3/13/02
Situated Autonomy and Action Selection
enablers
sensory
data
communications
beliefs
situated
autonomy
physical
goal
physical act
intention
communication
goal
communication
intention
Shared Autonomy between an Air Traffic Control assistant
agent and the human operator- 1999
Autonomy Computation
Collision:
Autonomy = (CollisionPriority / 4.0) +
(((|CollisionPriority – 4.0|) * t) / T)
Landing:
If 3.0 <= LandingPriority <= 4.0:
Autonomy = 1.0
If LandingPriority < 3.0:
Autonomy = (LandingPriority/4.0) +
(((|LandingPriority – 4.0|) * t) / 2)
Team- Building Intuition
•Drivers on the road are generally not a team
•Race driving in a “draft” is a team
•11 soccer players declaring to be a team are a team
•Herding sheep is generally a team
Agents change their autonomy, roles, coordination strategies
•A String Quartet is a team
Well organized and practiced
Team- Phil Cohen, et al
Phil Cohen, et al:
Shared goal and shared mental states
Communication in the form of Speech Acts is required for team formation
Steps to become a team:
1. Weak Achievement Goal (WAG) relative to q and with respect to a team
to bring about p if either of these conditions holds:
•The agent has a normal achievement goal to bring about p; that is, the agent does
not yet believe that p is true and has p eventually being true as a goal.
•The agent believes that p is true, will never be true, or is irrelevant (that is, q is
false), but has as a goal that the status of p be mutually believed by all the team
members.
2. Joint Persistent Goal (or JPG) relative to q to achieve p just in case
1. They mutually believe that p is currently false;
2. They mutually know they all want p to eventually be true;
3. It is true (and mutual knowledge) that until they come to mutually believe either that
p is true, that p will never be true, or that q is false, they will continue to mutually
believe that they each have p as a weak achievement goal relative to q.
Team- Phil Cohen, et al
•Requiring Speech Act Communication is too strong
•Requiring Mutual Knowledge is too strong
•Requiring agents to remain in a team until everyone knows about
the team-qualifying condition is too strong
Team- Munindar Singh
<agents, social commitments, coordination relationships>
Social commitments: <debtor, creditor, context, discharge condition>
Operators: Create, Discharge, Cancel, Release, Delegate, Assign
Coordination relationships about events:
e is required by f
e disables f
e feeds or enables f
e conditionally feeds f
…
Team- Michael Wooldridge
With respect to agent i’s desires j there is potential for cooperation iff:
1. there is some group g such that i believes that g can jointly achieve j;
and either
2. i can’t achieve j in isolation; or
3. i believes that for every action a that it can perform that achieves j,
it has a desire of not performing a.
i performs speech act FormTeam to form a team iff:
1. i informs team g that the team J-can j; and
2. i requests team g to perform j
Team g is a PreTeam iff:
1. g mutually believe that it J-can j;
2. g mutually intends j
Team- Michael Wooldridge
•Onset of cooperative attitude is independent of knowing about
specific individuals
•Assuming agent knows about g is hard too simplistic
•Requiring Speech Act Communication is too strong
•Requiring Mutual Knowledge is too strong
Team- [Hexmoor and Beavers 2001]
Necessary components of a team:
•Ability
•Objective
•Awareness
•Attitude: Cooperation and Responsibility
Working Conditions:
•Agent’s anticipatory power of team sensing and acting
•Laws, Norms, Conventions, Commitments
Motivations for team formation:
•Shortcomings in ability
•Efficiency
•Failure/fault tolerance
Hexmoor&Braynov
Multiagents: Formal and Economic
Outline
1. History and perspectives on multiagents
2. Agent Architecture
3. Agent Oriented Software Engineering
4. Mobility
5. Autonomy and Teaming
6. Game Theoretic and Decision Theoretic Agents
7. Social attitudes: Values, norms, obligations, dependence,
control, responsibility, roles
8. Benevolence, Preference, Power, Trust.
9. Communication, Security
10. Agent Adaptation and Learning
11. Trends and Open questions
12. Concluding Remarks
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Game-Theoretic and Decision-Theoretic Agents

Utilitarian rationality: every agent has an utility function
representing agent’s preferences over different alternatives. Every
agent is concerned with maximizing his expected utility

Decision theory deals with situations in which one or more agents
must make choices among given alternatives (Anatol Rapoport).
Every choice has outcomes. Agents have preferences for different
outcomes.

Decision making:



under certainty.
under risk.
under uncertainty.
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Game-Theoretic and Decision-Theoretic Agents (cont.)

Utility theory can be used in both decision making under
risk (where the probabilities are explicitly given) and in
decision making under uncertainty (where the
probabilities are not explicitly given).

Three different approaches:



Descriptive approach.
Normative.
Prescriptive approach.
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Foundations of Utility: Lottaries
Let A1 and A2 be any two events. Let 0p1. Then by
(pA1,(1-p)A2) we mean the lottery which has the two
possible outcomes A1 and A2 with probabilities p and 1-p
respectively.
A B A is preferred or indifferent to B
A B A is indifferent to B
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Axioms of Utility

Theorem [Luce and Raiffa, 1957]. If the preference
relation is complete, transitive, continuous, and
monotonic, then there exists an utility function U, such
that:
L L*, iff U(L)U(L*)
L L*, iff U(L)=U(L*)
OR
 Theorem [Fishburn, 1970]. If the preference relation is
a weak order on X and X/ is countable, then there is a
real-valued function U on X such that:
L L*, iff U(L)U(L*)
L L*, iff U(L)=U(L*)
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Utility of Money

The St. Petersburg Paradox (Daniel Bernoulli):
How much would you pay to play the following game? A fair coin is
continually tossed until it lands on heads. If the coin lands on heads
on the nth throw, you receive 2n dollars.
EU=2.(1/2)+4(1/4)+8(1/8)+…=1+1+1+…..

Risk attitudes:



Risk averse (concave utility function)
U(pA1+(1-p)A2) > pU(A1)+(1-p)U(A2)
Risk seeking (convex utility function)
U(pA1+(1-p)A2) < pU(A1)+(1-p)U(A2)
Risk neutral (linear utility function)
U(pA1+(1-p)A2) = pU(A1)+(1-p)U(A2)
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Decision Making Using Infinite Beliefs

Infinite beliefs arise naturally. An agent’s optimal decision
depends on what he believes the other bidders will do, which
in turn depends on what he believes the other bidders believe
about him, and so on. This leads to infinite regress of beliefs.

Problems:
 How to represent infinitely nested beliefs
 How to reason and make decision with infinite beliefs.

Our solution [Brainov, Sandholm, 2000]:
 We generalize the principle of backward induction to the
case of infinite belief trees.
 We identified a class of infinite belief trees that allow finite
representation.
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Infinite Belief Trees (cont.)
Cutting Infinite Belief Trees
[Gmytrasiewicz and Durfee, 95]
t1
p
1-p
a
t1
q
t1
.
.
.

t2
1-q
q
t2
t1
. .
. .
. .

1-q
t2
.
.
.
No information
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Infinite Belief Trees (cont.)
Backward Induction
Step 4: Solve the first level
i’s level
p
Step 3: Solve the second level
Step 2: Solve the third level
p
t1
Step 1: Solve the bottom level
SAC 2002 Tutorial
1-p
t2
t1
p
1-p
t1
t2
1-p p
t2 t
1
p
j’s level
1-p
t2
t
1
1-p
p
t2 t1
1-p p
t2 t1
i’s level
1-p
t2
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Infinite Belief Trees (cont.)
Representation of Infinite Belief Trees with Finite
Graphs

Two nodes of v1 and v2 of belief tree i are identical iff:

they are labeled with the same tk, tkT,
 v1 and v2 are both on an even or an odd-numbered
level of reflection,
 v1 and v2 have the same
successors,
 every two arcs starting at v1 and v2, that point to
the same successor, are labeled with equal
probabilities.

An elementary contraction of a graph G is obtained by
identifying two identical nodes v1 and v2 by removing v1
and v2, and by adding a new node v adjacent to those
nodes to which v1 and v2 were adjacent.
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Infinite Belief Trees (cont.)
Reasoning on Graphs

A graph G is contractible to a graph G’ if G’ can be
obtained from G by applying elementary contractions.

Proposition: If a belief tree is contractible to a graph,
then the graph is pointed and accessible.

Proposition: For every accessible pointed graph G there
exists a belief tree that is contractible to G.

A strategy labeling is balanced if the strategy associated
with each node is a best response to the strategies
associated with the successor nodes, given the
probabilities assigned to the successors.
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Infinite Belief Trees (cont.)
Balanced Strategy Labeling for Infinite Tree
t1 S1
i’s level of reflection
p
j’s level of reflection
SAC 2002 Tutorial
1-p
t1 S2
p
i’s level of reflection
a
t1 S4
. .
. .
. .
a
1-p
t2 S3
p
S5 t2 t1 S6
. . . .
. . . .
. . . .

1-p
S7 t2
. .
. .
. .
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Infinite Belief Trees (cont.)
Balanced Strategy Labeling for Infinite
Graph
S1
Agent i
p
1-p
S3
S2
t1
Agent j
p
Agent i
SAC 2002 Tutorial
S4
t1
t2
p
p
p
1-p
1-p
1-p
1-p
t2
S5
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Example: Auction Analysis

English (first-price open cry). An ascending auction where the
price is successively raised until one bidder remains. That
bidder wins the object and pays his final price.

Dutch. A descending auction where the auctioneer starts at a
very high price and then lowers the price gradually. The first
bidder who stops him takes the object at the current price.

First-price sealed-bid auction. Each bidder independently
submits a bid without knowing others’ bids. The highest
bidder wins and pays his bid.

Second-price sealed-bid auction. Each bidder independently
submits a bid without knowing others’ bids. The highest
bidder wins and pays the amount of the second highest bid.
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Example (cont.)

Revenue equivalence theorem [William Vickrey, 1961]: The firstprice sealed bid, second-price sealed bid, English and Dutch
auctions are all optimal selling mechanisms provided that
they are supplemented by optimally set reserve price.

Simple Auction Setting:
 Isolated auction for a single indivisible object with two risk
neutral bidders.
 Two possible valuations of the object: t1 and t2 (t1<t2). t1
and t2 are independent random variables with objective
distribution p=(1/2,1/2).
 Each bidder knows his own valuation, but is uncertain
about his rival’s evaluation.
 p is common knowledge.
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Example (cont.)
Revenue Equivalence Theorem fails:
 Proposition. When there does not exist common knowledge
about private beliefs, the revenue equivalence theorem ceases
to hold. The bidder’s expected utility is different in the first
price sealed bid auction and Vickrey auction.
Agent i
Agent i
p
p
1-p
1/2
Agent j
t2
t1
Agent j
1/2 1/2
t1
SAC 2002 Tutorial
t.2 t1.
. .
. .
t2
1/2
1/2
1/2
1/2
t.1
.
.
1-p
t.2
.
.
Agent i
t1
1/2
1/2
1/2
1/2
1/2
t2
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Outline
1. History and perspectives on multiagents
2. Agent Architecture
3. Agent Oriented Software Engineering
4. Mobility
5. Autonomy and Teaming
6. Game Theoretic and Decision Theoretic Agents
7. Social attitudes: Values, norms, obligations, dependence,
control, responsibility, roles
8. Benevolence, Preference, Power, Trust.
9. Communication, Security
10. Agent Adaptation and Learning
11. Trends and Open questions
12. Concluding Remarks
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Agent as a member of a group...
agent
honors
handles
partakes
roles
obligations
member of
goals
specifies
plans
contains norms
values
(terminal
goals)
SAC 2002 Tutorial
relies on
partakes
shares
group
3/13/02
Values
• "value" might mean:
(a)assessment of usefulness of an object or
action relative to a purpose, I.e.,
"(instrumental) evaluations", E.g., ="this
knife is good for chip carving ",
(b) absolute assessment of desirability of
something, I.e, “principles”, E.g., "honesty is
good"
• Adding value to an agent enables it to generate
internal desires as well as adds a level of
behavior predictability for other agents.
Norms
•Involve two or more agents.
Each agent
understands and shares them.
• Agents have power to not choose them.
• There is no direct rational account of them
available to the agents.
• The bearer experiences an implicit or an explicit
sanction or rewards for adoption.
Norms
Norm = (O, R, G, A)
 O is the content of the norm set. These can at least one
goal to do or at least one state that it may avoid.
 R is the sanction that may result from not following the
norm.
 G is the agent’s goal that invokes the norm set when the
agent chooses to consider other agents
• A is a set of mental stances along with degrees for each.
The mental stances are characterized by using the notions of
belief, desire, and intention. The degrees of mental stances
forms a required pattern against which we will match the
agent’s actual mental stances.
Obligations
• Obligations capture all forms of social influence.
• Obligations have a strong deontological and
motivational senses (more so than norms)
• Obligations are frequently assumed to have
penalties associated with the failure to meet the
obligation. We make no such assumption; some
obligations may have sanctions and some may not.
Dependence and Control
• An agent a may depend on another agent b for
performing an action when agent b is obligated (with
consent) to agent a for performing the action.... More to
come
• An agent a controls an agent b in a given domain when
(a) agent b adopts goals set by agent a in the domain, and
(b) agent a monitors agent b about the goal and gives it
feedback
• If furthermore, agent b incorporates the feedback from
agent a, the control is master-slave.
Responsibilities
• There are several types of responsibility:
(a)Responsibility to concerns an agent’s obligation
to perform an action.
(b)Responsibility for concerns an agent’s obligation
to see that a state of affairs obtains.
(c) Responsibility about is the agent’s obligation to
behave in accordance with its principles, which
is general, abstract, and typically with respect to
an agent’s immutable values.
Responsibilities, CAST project [Yen, et al. 2001]
•Agents are represented as nodes of a graph.
•One type of labeled directed edge is between two
agents (A t B), and it represents that A delegates
t to B or conversely B is responsible to A with
respect to t.
•The delegation relationships is non-reflexive, antisymmetric, and transitive. The transitive property
can be used to establish implied relationships.
Roles
• Several agents can adopt it individually, independently, and
concurrently. One agent may adopt several simultaneously. Several
agents may adopt it as a group. In general we will call this the
adopter.
• It is meaningful in the social context of other agents including (a)
the adopter’s relationship to other agents and groups, (b) the agent’s
mental attitudes about the social relationships, and (b) the available
norms including obligations and responsibilities.
• There are typical capabilities associated with the adopter. If the
adopter is loses these abilities then the efficacy of the role is
jeopardized.
Roles
•Networks of roles are more clearly seen in role-based
access control.
•Role hierarchy and role grouping are useful for selecting
subsequent roles [Moffett and Lupu, 1999, Na and Cheon,
2000].
The big picture
Norms
Values
Obligationsab (i.e., responsibility)
Dependenceba
Delegationba
Powerab
Trustba
Autonomyb
Autonomyb + Autonomya
Controlab
Hexmoor&Braynov
Multiagents: Formal and Economic
Outline
1. History and perspectives on multiagents
2. Agent Architecture
3. Agent Oriented Software Engineering
4. Mobility
5. Autonomy and Teaming
6. Game Theoretic and Decision Theoretic Agents
7. Social attitudes: Values, norms, obligations, dependence,
control, responsibility, roles
8. Benevolence, Preference, Power, Trust.
9. Communication, Security
10. Agent Adaptation and Learning
11. Trends and Open questions
12. Concluding Remarks
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Social Interaction
Self-Interest and Benevolence in Multiagent Interaction
[Brainov, 1994, 1999]

Limitations of self-interested behavior:
 Free-rider problem
 Not always efficient allocation of tasks and resources

Benevolent agents are important when
 An agent represents preferences of non-selfish users
 An agent represents collective users
 Several agents represent one user

Malevolent agents are important in adversarial environments

Model of social attitudes
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Power and Dependence in Multiagent Plans

Joint Plans:
N={1,2,…,n}
(p1,p2,…,pn)
pN-i
-
Ui
-
the set of all agents
a joint plan
the plan of the group of
agents N-{i}
utility of agent i
(p1,p2,…,pn)
=
(pN-i, pi)
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Power and Dependence in Multiagent Plans
Models of Power and Dependence

Social dependence network [Castelfranchi, Conte, Sichman,
Demazeau]
 Only two types of dependence (resource and action
dependence)
 Individual dependence (an agent depends on another
agent

Dependence as an element of power [Brainov, 1998;
Brainov&Sandholm, 1999]
 Decision-oriented model of dependence
 Group dependence (an agent depends on a group, group
depends on an agent, group depends on a group)
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Elements of Power
Means
Base
Cost
Power
Amount
SAC 2002 Tutorial
Scope
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Power and Dependence in Multiagent Plans
Individual Dependence

An agent i depends on another agent j in a joint plan if agent j
can gain at the expense of agent i agent by deviating from the
plan.

Formally:
In a joint plan (pN-j,pj) agent i depends on agent j if there
exists plan p*j such that for every plan p’N-j of the other
agents:
Ui(p’N-j,p*j) < Ui(pN-j,pj),
Uj(pN-j,p*j)  Uj(pN-j,pj).
The plan p*j is said to be gainful for agent j at the
expense
of agent i. To indicate that the gainful plan
depends on
the initial plan (pN-j,pj), we denote it by
gain
pj (i,pN-j,pj).
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Example
j
i
Help!
j
SAC 2002 Tutorial
i
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Example
I can help
you.
I can also
help you.
j
k
i
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Power and Dependence in Multiagent Plans
A Group Depends on an Agent
[Brainov and Sandholm,99]
A group S depends on an agent j if every member of S depends
on j for the same plan p*j.
 Formally:
In a joint plan (pN-j,pj) a group of agents S, SN-j, depends on
agent j if there exists a plan p*j such that for every plan p’N-j of
the other agents:
Uk(p’N-j,p*j) < Uk(pN-j,pj) for every kS
Uj(pN-j,p*j)  Uj(pN-j,pj).
The plan p*j is said to be gainful for agent j at the expense of
group S. To indicate that the gainful plan depends on the
initial plan (pN-j,pj), we denote it by pjgain(S,pN-j,pj).
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Example
We have to
move the
block.
I can help
you.
j
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Power and Dependence in Multiagent Plans
An Agent Depends on a Group

An agent i depends on a group S, if



i depends on every member of the group (given that the rest of
the group do not change their plans),
the group S is minimal.
Formally:
In a joint plan (pN-j,pj) agent i depends on a group of agents S,
iS, if
(i) (group requirement) agent i depends conditionally on
every agent k, kS, with the tacit consent of the group
S-k;
(ii) (minimality requirement) for every agent k, kS, every
conditionally gainful plan pkgain(i, pN-S, pS-k, pk/pS-k) and
every plan p’N-S, it holds that every agent m, mS-k, can
gain without harming agent i in the plan (p’N-S, pS-k,
pkgain(i,pN-S,pS-k,pk/pS-k))
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Example
i
j
k
j
k
Help!
i
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Example
j
k
i
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Power and Dependence in Multiagent Plans

A joint plan is individually stable if no single agent can deviate
from the plan.

A joint plan is coalitionally stable if no group of agents can
deviate from the plan in a way that benefits all its members.

Proposition. Every Pareto optimal joint plan which is based on
reciprocal dependence is individually and coalitionally stable.

Proposition. If in a Pareto optimal joint plan (pS,pj):
(i) pS is based on reciprocal dependence,
(ii) group S depends on agent j,
(iii) agent j depends on some agent i, iS,
then the plan (pS,pj) is individually and coalitionally stable.
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Social Attitudes
Agent typology
Altruistic
SAC 2002 Tutorial
Self-interested
Malevolent
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Social Interaction (cont.)
Benevolent Agents
An
agent is benevolent towards other agents if its utility
function depends positively on the utility functions of other
agents:
 Formally:
An agent a is benevolent towards the agents 1,…,k iff:
Ua=Ua(Xa, U1(X1),…,Uk(Xk))
∂Ua
0
∂X a
∂
Ua
0
∂
Uj
For all j=1,…,k
Here Xa represents the benefits of agent a. U1,…,Uk are
utility functions of agents towards whom agent a is
altruistic. X1,…,Xk represent the benefits of these agents.
Example:
agents with a common goal, agents that serve one
user, team of agents, etc.
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Social Interaction (cont.)
Self-Interested Agents


An agent is self-intersted if its utility function does
not depend on the utility functions of other agents:
Formally:
An agent i is self intersted iff:
Ui=Ui(Xi, U1(X1),…,Uk(Xk))
∂Ui
0
∂Xi
∂
Ui
0
∂
Uj
For all j=1,…,k
Here Xi represents the benefits of agent i.
U1,…,Uk are utility functions of other agents.

Example: agents that serve different users, etc.
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Social Interaction (cont.)
Malevolent Agents
An
agent is malevolent towards other agents if its utility
function depends negatively on the utility functions of other
agents:
 Formally:
An agent m is malevolent towards the agents 1,…,k iff:
Um=Um(Xm, U1(X1),…,Uk(Xk))
∂
Um
0
∂
Xm
∂
Um
0
∂
Uj
For all j=1,…,k
Here Xm represents the benefits of agent m. U1,…,Uk are
utility functions of agents towards whom agent m is
malevolent.
Example:
agents with
antagonistic
competitors, virtual battlefield, etc.

SAC 2002 Tutorial
goals,
rivals,
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Social Interaction (cont.)
Degree of Attitude
degree of attitude aij of agent i towards agent j is
defined as follows:
The
Ui Ui
αij 
/
Uj Xi
We call an agent i:
 Self-biased, if aij<1;
 Other-biased, if aij>1;
 Neutral, if aij=1.
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Social Interaction (cont.)
Derived Social Attitudes



If agent i has a social attitude towards agent j and agent j has
a social attitude towards agent k, then the combined effect of
these two attitudes is called a derived attitude of agent i
towards agent k.
Proposition. If aij is the degree of attitude of agent i towards
agent j and ajk is the degree of attitude of agent j towards
agent k, then the degree of derived attitude, aik, of agent i
towards agent k is:
aik=aijajk
Proposition. If aij is the degree of attitude of agent i towards
agent j, ajk is the degree of attitude of agent j towards agent k,
and aik is the degree of attitude of agent i towards agent k,
then the degree, a*ik, of the final derived attitude of agent i
towards agent k satisfies the following condition:
a*ik=aijajk+aik
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Example
friends
enemies
?
enemies
SAC 2002 Tutorial
friends
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Social Interaction (cont.)
Optimal Group Attitudes

Proposition. Let
Ui(p)=worthi(gi)+aijworthj(gj)-costi(p)- aijcostj(p)
Uj(p)= ajiworthi(gi)+worthj(gj)- ajicosti(p)-costj(p)

where p is a joint plan of agents i and j, aij, aji are constants
and aij>0, aji>0. If the cost functions of both agents are
differentiable on the set of all joint plans P, and if there exists
an equilibrium joint plan p*P, then:
aijaji=1
Example: Conflict of attitudes:
If one agent wants to help another agent, the first one has to
be benevolent enough to provide help and the second agent
has to be selfish enough to receive help. If both agents are
other-biased, then both of them will want to help and nobody
will want to receive help.
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Social Interaction (cont.)
Solving Social Situations
1/2
-1/3
3
Incomplete situation
1/2
-1/3
2
-3
3
1/3
Possible completion
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Social Interaction (cont.)
Socially Responsible Agents
[Jennings and Campos, 97]
Principle of Social Rationality: If a member of a
responsible society can perform an action whose joint
benefit is greater than its joint loss, then it may select
that action.
D(M,a) - W(M,a)  d(M,a) -w(M,a)
Members may perform actions for which their
member benefit is less than their member loss, if the
society gets more in total than it loses.
 Members may perform actions which bring them
personal benefit, but which are detrimental to the
overall society. This is possible if the member benefit
is greater than the societal loss.

SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Social Interaction (cont.)
Helping Agents
[Cesta, Miceli and Rizzo, 99]
Lonely agents: ignore one another.There is no
interaction among them. Their goal is always to
individually find food.
 Social agents: in case of danger, their goal
generator activates the goal of looking for help; when
hungry, their goal is to find food; and finally, in case of
normal state, if there are any visible needy agents, the
goal of giving help is activated; otherwise, they go on
looking for food.

In an experimental environment social agents
overperform lonely agents.
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Social Interaction (cont.)
Probabilistic reciprocity
[Sen, 96]
Adaptive, probabilistic, reciprocity-based policy for deciding
which other agent to cooperate with.
 Allows agents to initiate cooperative relationships.
 Use a mechanism to compare cooperation costs\allow agents
to be inclined to help other agents with whom there is a
favorable balance of help.
 Flexible adjustment of help-giving behavior based on current
work-load.
1
Pr( i , k , j , l ) 
k
c ijkl - βcavg
 Bik
1  exp
r
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Trust

Reasons for trust:




Trust as:



Time asymmetry: some agents have to accomplish their part of the
transaction before other agents have taken actions.
Lack of power: an agent does not have the power to control actions of
other agents.
Inability to conclude perfect contracts: contract are usually incomplete
or indefinite.
Expectation of partner’s competence
Expectation of partner’s benign intent
Models of trust:



Castelfranchi, Falcone [1999,2000]
Marsh [1994]
Brainov, Sandholm [1999,2001]
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Trust (cont.)
Contracting Problem
The seller produces some commodity and sells it to
the buyer. The transaction is secured by a contract
stating the quantity and the selling price.
 - the level of trustworthiness of the buyer
 - the level of trustworthiness of the seller
In our case =1, 01. The seller is not completely
trustworthy.
a - the buyer’s estimate of 
 - the seller’s estimate of 
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Trust (cont.)
Optimal Trust
[S. Brainov, T. Sandholm , 1999]

Proposition. If a=, then the quantity exchanged maximizes
the function the social welfare.

Proposition. The quantity maximizing the social welfare is the
maximum possible output.

Proposition. When the trust placed matches trustworthiness
(a=) the seller and the buyer maximize their utilities.

The social welfare, the volume of trade and agents’ utilities are
maximized if the buyer’s trust matches the seller’s
trustworthiness. It is NOT necessary for the seller to be
completely trustworthy
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Trust (cont.)
Improving Trustworthiness by Advanced Payments
Proposition. If a< and the agents choose an
advanced delivery contract, then the quantity
exchanged maximizes the social welfare, the buyer
makes a complete advance delivery, and the volume
of trade is maximal.
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Trust (cont.)
Incentive Compatible Mechanism for Trust Revelation

If the buyer asks the seller about his level of
trustworthiness, the seller may lie.

Incentive compatibility: the seller has sufficient
incentives to reveal truthfully his level of
trustworthiness.

The mechanism:

First, the seller declares his level of
trustworthiness  .
the seller’s declaration  , the buyer
chooses the quantity to be exchanged q=q( )
 Using
 The
price P is exogenously determined by
market forces.
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Trust (cont.)

Proposition. If the seller’s cost function is twice
differentiable and convex, there exists exactly one
quantity function q() that makes the seller reveal
truthfully his level of trustworthiness.

Proposition [Braynov, 2001]. If the seller’s cost
function is convex and the buyer’s value function is
concave, then the mechanism is individually
rational for both the buyer and the seller (i.e., they
will benefit by participating in the mechanism).
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Trust (cont.)
Advantages of the Mechanism




It does not require the estimation of other agents’
trustworthiness. For example, agents do not need to
reason, plan and search for estimates of other agents’
trustworthiness.
It eliminates the need to speculate on other agents’
intentions and beliefs. This could simplify individual
decision-making and save some deliberation costs.
It may reduce the cost of trust management. Since agents
are reporting truthfully their levels of trustworthiness,
recording, analyzing and aggregating reputation
information is simplified.
It could eliminate many market failures caused by the
lack or inaccuracy of information. Truthful estimates of
the agents’ trustworthiness are available for every
transaction.
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Outline
1. History and perspectives on multiagents
2. Agent Architecture
3. Agent Oriented Software Engineering
4. Mobility
5. Autonomy and Teaming
6. Game Theoretic and Decision Theoretic Agents
7. Social attitudes: Values, norms, obligations, dependence,
control, responsibility, roles
8. Benevolence, Preference, Power, Trust.
9. Communication, Security
10. Agent Adaptation and Learning
11. Trends and Open questions
12. Concluding Remarks
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Communication
Speech Act Theory
[Searle, 70; Austin, 62]
Natural language communication consists of speech
acts such as requests, suggestions, commitments,
replies. etc.
Utterances:
propositional utterances
illocutionary utterances
perlocutionary utterances
intention to affect behavior
intention to interact
reference to the world
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Communication (cont.)
Knowledge Query and Manipulation Language (KQML)
(KQML-performative
:sender
<word>
:receiver
<word>
:language <word>
:ontology <word>
:reply-with <expression>
:in-reply-to <expression>
:content
<expression>...)
SAC 2002 Tutorial
semantics
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Communication (cont.)





KQML communication could be asynchronous or
synchronous.
KQML is a communication protocol for both agents
and application programs.
KQML separates the domain semantics from the
semantics of the communication protocol.
Both the sender and the receiver have to
understand KQML and use the same ontology.
The network infrastructure is not a part of KQML:
agents cannot locate one another. They use
facilitators.
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Communication (cont.)
Knowledge Interchange Format (KIF)
[Genesereth, 1991]
KIF is a prefix version of first order predicate
calculus with extensions to support nonmonotonic
reasoning and definitions.
(=> (bird joe)
(can_fly joe))
(interested john '(can_fly ?x))
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Communication (cont.)
Ontologies
An ontology is a formal, explicit specification
of a shared conceptualization [Gruber, 93].

Domain ontologies

Metadata ontologies

Generic or common sense ontologies

Representational ontologies
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Communication (cont.)
Ontology languages

CYC [Lenat&Guha, 90]: based on the first-order
predicate calculus. Uses constants, predicates,
variables, formulas, functions and quantifiers.
Quantification is also allowed over predicates,
functions, arguments and formulas.

Ontolingua [Gruber, 93]: this is an object-oriented
approach.
(define-relation name (?A1 ?A2)
:def (KIF fornula))

Frame logic [Kifer et al., 95]: a language for
specifying object-oriented databases.
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Security

Host security: protecting the hosting computer, its services,
data and its reputation

Mobile agent security: protecting agent code, state, data and
reputation.

Security threats:

Disclosure of information

Denial of service

Corruption of information

Masquerading

Unauthorized Access

Repudiation

Eavesdropping

Alternation
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Security (cont.)
Home
Platform
Agent
Agent
Platform
Network
Agent
Platform
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Security (cont.)
An agent attacking an agent platform
An agent platform attacking an agent
Thread categories:
An agent attacking another agent
on the same platform
Other entities attacking an agent
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Security (cont.)
Protection
Software-Based Fault Isolation: Application modules are isolated
into distinct fault domains. Sandboxing.

Safe Code Interpretation: commands considered harmful can be
either made safe for or denied to an agent.

Signed Code: signing code or data with digital signature. Microsoft
Authenticode.

State Appraisal: the author produce and digitally signs state
appraisal functions.

Path histories: to keep authenticatable record of the prior platforms
visited by an agent.

Proof Carrying Code: the author produces a formal proof that the
program possesses safety properties previously stipulated by the
consumer.

SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Security (cont.)
Proof Carrying Code: the author produces a formal proof that
the program possesses safety properties previously stipulated
by the consumer.

Partial result encapsulation: encapsulation of the results of an
agent’s actions at each platform visited.

Mutual Itinerary Recording: when moving between agent
platforms, an agent conveys the last platform, current platform,
and the next platform information to the cooperating peer agent.

Replication: rather than a single copy of an agent performing a
computation, multiple copies of the agent are used.


Shared secrets and interlocking.
Execution tracing: faithful recording of the agent’s behavior
during its execution on each platform.

SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Security (cont.)
Environmental key generation: upon encountering an
environmental condition a key is generated which is used to
unlock some executable code cryptographically.

Computing with encrypted functions (executable encrypted
functions): the platform executes a program implementing an
encrypted function without being able to discern the original
function.


Using dummy items and functions.

Watermarks and steganographic techniques.

Smart-card solutions.
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Outline
1. History and perspectives on multiagents
2. Agent Architecture
3. Agent Oriented Software Engineering
4. Mobility
5. Autonomy and Teaming
6. Game Theoretic and Decision Theoretic Agents
7. Social attitudes: Values, norms, obligations, dependence,
control, responsibility, roles
8. Benevolence, Preference, Power, Trust.
9. Communication, Security
10. Agent Adaptation and Learning
11. Trends and Open questions
12. Concluding Remarks
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Multiagent Learning
Learning agent: an agent that can improve its behavior by
analyzing its own and others’ experience.
Agent’s abilities:
 An agent is able to sense the environment to some
extent.
 An agent is able to take actions that affect the state of
the environment.
Important elements:
 Learning element: responsible for making
improvements.
 Performance element: responsible for selecting actions.
 Performance standard: fixed, beyond the control of the
agent.
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Multiagent Learning (cont.)
Environment:
 Accessible: the agent percept at each step will
identify the state it is in.
 Inaccessible or partially accessible
Supervised learning: there is a supervisor, who
provides both the actions and their results.
Reinforcement learning: the feedback is in the form of
reward(reinforcement). An agent learn its utility
functions on states.Every agent tries to maximize
its (expected) utility.
Q-Learning: an agent learns an action-value function
giving the expected utility of taking a given action.
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Multiagent Learning (cont.)
Rewards:




Can be received only in terminal states.
Can be received in any state.
Can be components of actual utility.
Can be hits as to the actual utility.

Centralized learning: the learning process is
executed by a single agent and does not require
any interaction with other agents.

Decentralized learning:several agents or groups of
agents are engaged in the learning process. Agents
may have the same or different learning goals.
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Multiagent Learning (cont.)
Layered learning
[Stone P., 2000; Stone P. Veloso M., 2000]




Layered learning is designed for domains that are
too complex for learning a mapping directly from
the input to the output representation.
Layered learning uses a bottom-up incremental
approach to hierarchical task decomposition.
Learning occurs separately at each level.
Each learning layer directly affects the learning at
the next layer. A learned subtask can affect the
subsequent layer by: constructing the set of
training examples; providing the features used for
learning; or pruning the output set.
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Multiagent Learning (cont.)
Layered learning in robotic soccer

Layer 1: An individual skill: ball Interception. Intercept
opponent shots or passes as well as receive passes
from teammates

Layer 2: Multiagent behavior: pass evaluation. Agents
(both teammates and opponents) are equipped with the
previously learned ball-interception behavior.

Layer 3: Collaborative and adversarial team behavior:
pass selection. The input space is drastically reduced
the input space with the help of the previously learned
decision tree.
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Multiagent Learning (cont.)
Team-Partitioned Opaque-Transition RL
[Stone P., Veloso M., 1999]
Team Partitioning: Each agent explores a
separate partition of the state space without
any knowledge of state values in other
partitions. Agents are learning in parallel.
 Opaque-transition setting: since teammates
and opponents can affect the outcome of an
action, state transitions are opaque to an
agent.

SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Multiagent Learning (cont.)
State generalization: every state is generalized to
a feature vector v using the state generalization
function f:SV.
 Value function learning: the feature vector v is
used to estimate the expected reward for taking an
action in a state – Q(V,A)  R.
Q(v,a)=Q(v,a)+a(r-Q(v,a))
 Action selection: an action is chosen for execution
and its reward is used to update Q. An action is
chosen either randomly when exploring, or
according to maximum Q-value when eploiting.

SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Multiagent Learning (cont.)
Improving Opponent Models into Adversary Search
[Carmel and Markovitch, 96]
S
: S2S
j:S S
f
d
-the set of all possible states
- the successor function
- the opponent model
- utility function
- a depth limit

f (s)

M (s, d, f , j)   max (f (s' ))
s'(s)
 max (M (j(s' ), d - 2, f , j))
s'(s)
SAC 2002 Tutorial
d0
d 1
d 1
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Multiagent Learning (cont.)
Let M0(<f0>,d)(s) - regular minimax algorithm
M0(<f0>,d)(s)=M(s,d,f0, M0(<-f0>,d-1))
M1(<f1,f0>,d)(s)=M(s,d,f1, M0(<f0>,d-1))
……………………………………..
Mn(<fn,...,f0>,d)(s)=M(s,d,fn, Mn-1(<fn-1….f0>,d-1))
Definition: A player is a pair as follows:
• Given a utility function f, P=(f, NIL) is a player with a
modeling-level 0.
• Given a utility function f and a player O (with modelinglevel n-1), P=(f,O) is a player with a modeling-level n.
Different players: (f0,NIL), (f1,(f0,NIL)), (f2,
(f1,(f0,NIL))),…….
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Multiagent Learning (cont.)
Game Tree
a
c
b
d
h
f2= 8
f1= 6
f0= 4
SAC 2002 Tutorial
e
i
f2= -4
f1= 6
f0= -8
j
f2= 4
f1= -8
f0= 10
f
k
f2= 7
f1= -7
f0 = 3
l
f2= -6
f1 = 7
f0= -4
m
f2= 1
f1= -2
f0 = 4
g
n
f2= 10
f1= -4
f0 = 4
o
f2= 2
f1= 0
f0= 6
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Multiagent Learning (cont.)
Learning in market environments
[Vidal, Durfee, 96]
Several buyers and sellers. These agents exchange
goods by paying some price psg. When a buyer wants
to buy a good g, he will advertise this fact. Each seller
that sells that good will give his bid in the form of a
price psg . The buyer will pick one of these and will
pay the seller. The seller will then return the specified
good.
Vbg(p,g) - value function, returns the value that the
buyer b assigns to that particular good at that
particular price and quality.
psg-csg - the seller's profit
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Multiagent Learning (cont.)
Types of agents:
0-level agents: they do not model the behavior of the other agents.
1-level agents: they model other agents as 0-level agents.
2-level agents: they model other agents as 1-level agents

If buyers are 0-level agents and sellers are 1-level agents, seller can
pretend to be high-quality goods sellers by bidding high prices and
thus obtain substantial profits at the expense of the buyers.

If the buyers are 1-level agents, they learn to buy from sellers who
can provide them with the highest value.

If the buyers and the sellers are 1-level agents, 1-level sellers suffer,
because they assume buyers are 0-level agents and try to over-price
their goods.
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Outline
1. History and perspectives on multiagents
2. Agent Architecture
3. Agent Oriented Software Engineering
4. Mobility
5. Autonomy and Teaming
6. Game Theoretic and Decision Theoretic Agents
7. Social attitudes: Values, norms, obligations, dependence,
control, responsibility, roles
8. Benevolence, Preference, Power, Trust.
9. Communication, Security
10. Agent Adaptation and Learning
11. Trends and Open questions
12. Concluding Remarks
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Current Trends
Pervasive and emerging agent applications: agent
mediated e-commerce, emotional agents, embodied
agents, virtual characters, conversational agents, etc.


Standardization efforts: FIPA.

New Initiatives: semantic web initiative.
 Agent
tournaments: RoboCup, Trading Agent
Competition.
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Outline
1. History and perspectives on multiagents
2. Agent Architecture
3. Agent Oriented Software Engineering
4. Mobility
5. Autonomy and Teaming
6. Game Theoretic and Decision Theoretic Agents
7. Social attitudes: Values, norms, obligations, dependence,
control, responsibility, roles
8. Benevolence, Preference, Power, Trust.
9. Communication, Security
10. Agent Adaptation and Learning
11. Trends and Open questions
12. Concluding Remarks
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Concluding Remarks

There are many uses for
 Agents
 Agent-based
Systems
 Agent Frameworks

Many open problems area available
 Theoretical
issues for modeling social elements such as
autonomy, power, trust, dependency, norms,
preference, responsibilities, security, …
 Adaptation and learning issues
 Communication and conversation issues
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
Further Explorations




DAI-List@engr.sc.edu
Agents.umbc.edu
http://www.AgentLink.org/
http://www.multiagent.com/
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
References
Austin J. How to do Things with Words. Calderon, Oxford, 1962.
Braynov S.. Incentive Compatible Mechanism for Trust Revelation. IJCAI’01
Workshop on Economic Agents, Models and Mechanisms, 2001.
Brainov S., Sandholm T. Reasoning about Others: Representing and Processing
Infinite Belief hierarchies, ICMAS’2000.
Brainov S., T. Sandholm T. Contracting with Uncertain Level of Trust. In
Proceedings of the ACM Conference on E-Commerce, 1999.
Brainov S., T. Sandholm T. Power, Dependence and Stability in Multiagent Plans.
In Proc. Of AAAI’99, pp: 11-16, 1999.
Brainov S. Altruistic Cooperation Between Self-Interested Agents, European
Conference on AI, pp:519-523, 1996.
S. Brainov S. The Role and the Impact of Preferences on Multiagent Interaction,
ATAL’99, 1999.
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
References (cont.)
Carmel D., Markovitch S. Incorporating Opponent Model into Adversary Search, In
Proc. Of AAAI’96, pp: 120-125, 1996.

Castelfranchi C., Falcone R., (2000), Trust and Control: A Dialectic Link, Applied
Artificial Intelligence journal, Special Issue on "Trust in Agents" Part1, Castelfranchi C.,
Falcone R.,Firozabadi B., Tan Y. (Editors), Taylor and Francis 14 (8), 2000.

Castelfranchi, C., Falcone, R. (1999). The Dynamics of Trust: from Beliefs to Action,
Autonomous Agents Î99 Workshop on "Deception, Fraud and Trust in Agent Societies",
Seattle, USA, May 1, pp.41-54.

Cesta A., Miceli M., Rizzo P. Proceedings of the International Workshop on the Design
of Cooperative Systems. Proceedings of the International Workshop on the Design of
Cooperative Systems, 1995.


Genesereth M. Knowledge Interagent Format. In Proc. Of KR-91, pp: 238-249, 1991.

Gmytrasiewicz, P., Durfee, E. A Rigorous, Operational Formalization of Recursive
Modeling, In Proceedings of the First International Conference on Multi-Agent Systems,
pages 125-132, 1995.
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
References (cont.)
Gruber T. A Translation Approach to Portable Ontology Specification, Knowledge
Acquisition, 5:199-220, 1993.

Hexmoor, H., Holmback H., and Duncan, L. Detecting, Expressing, and Harmonizing
Autonomy in Communication Between Social Agents, 2001 AAAI spring symposium on
Robust Autonomy, Stanford, AAAI press.

Hexmoor H. and Harry Duchscherer, H. Shared Autonomy and Teaming: A preliminary
report. In Proceedings of Workshop on Performance Metrics for Intelligent Systems, NIST,
2000, Washington, DC.

Hexmoor, H. A Cognitive Model of Situated Autonomy, In Proceedings of PRICAI-2000
Workshop on Teams with Adjustable Autonomy, Australia.

Hexmoor, H. Weakly Dependent Agents May Rely on Their Luck or Collaboration: A Case
for Adaptation, In AISB-2001, York, UK.

Hexmoor, H. and Duchscherer, H. Efficiency as Motivation for Teaming, In Proceedings of
FLAIRS 2001, AAAI press.

SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
References (cont.)
Hexmoor, H. and Beavers, G. Towards Teams of Agents, In Proceedings of the International
Conference in Artificial Intelligence, H. R. Arabnia, (ed), (IC-AI'2001), Las Vegas, CSREA
Press.

Hexmoor H. and Zhang, X. Norms, Roles, and Simulated RoboCup, In 2nd workshop on norms
and institutions in multiagent systems, (Agents 2001), Montreal, CA, ACM press.

Jennings N., Campos J. Towards a Social Level Characterization of Socially Responsible
Agents. in IEE Proceedings on SoftwareEngineering, 144 (1),11-25, 1997,


Fishburn P. Utility Theory for Decision Making. Wiley, New York, 1970.
Kifer M., lausen G., Wu J. Logical Foundations of Object-Oriented and frame-Based
languages. Journal of the ACM, 42, 1995.


Lenat D., Guha R. Building Large Knowledge-Based Systems. Addison-Wesley, 1990.

Luce R D, Raiffa H. Games and Decisions, Wiley, New York, 1957.
Marsh S. Formalising Trust as a Computational Concept. Ph.D Thesis, University of Stirling,
April 1994


Searle J. Speech Acts: An Essay in the Philosophy of Language., Cambridge, 1970.
SAC 2002 Tutorial
3/13/02
Hexmoor&Braynov
Multiagents: Formal and Economic
References (cont.)
Sen S. Reciprocity: a foundational principle for promoting cooperative behavior among selfinterested agents'' , in Proc. of the Second International Conference on Multiagent Systems, pp.
322-329, 1996.

Sichman J., Rosaria C., Demazeau Y., Castelfranchi C. A social reasoning mechanism based
on dependence networks. In: Proceedings of the 11th European Conference on Arti- cial
Intelligence, 1994.

Stone P., TPOT-RL Applied to Network Routing. Seventeenth International Conference on
Machine Learning, 2000.

Stone P., Veloso M. Layered Learning. Eleventh European Conference on Machine Learning,
2000.

Stone P., Veloso M. Team-Partitioned, Opaque-Transition Reinforcement Learning, Agents’99,
1999.

Vickrey, W. Counterspeculation, Auctions, and Competitive Sealed Tenders, Journal of
Finance 16, 8-37, 1961.

Vidal J., Durfee E. The Impact of Nested Agent Models in an Information Economy. In Proc.
Of the ICMAS’96, pp:377-384, 1996.

SAC 2002 Tutorial
3/13/02