Deliberative Automated Negotiators Using Fuzzy Similarities
C. Sierra
IIIA (CSIC), Spain.
Sierra@iiia.csic.es
P. Faratin
N.R. Jennings
QMW
University of London
P.Faratin@qmw.ac.uk
Introduction
Automated agents are autonomous entities which decide for themselves what, when, and under what conditions their actions should be performed. Since agents
have no direct control over others, they must persuade
others to act in a particular manner. The type of persuasion we consider in this paper is negotiation which
we dene as a process by which a joint decision is made
by two or more parties. The parties rst verbalise contradictory demands and then move towards agreements
(Pruitt 1981).
For negotiation agents must be provided with the capability to represent and reason about, within their information and resource bounds, both their internal and
their external world and with the capacity to interact
according to a normative protocol. It is this individual
agent modelling which has been the central focus of the
work reported in this paper.
This paper extends our previous work, reported in
(Faratin, Sierra, & Jennings 1998), on negotiation models in the following way. The agent architecture has
been updated from a purely responsive mechanisms to
include new higher level deliberative mechanisms, involving the generation of trade os and the manipulation of the set of issues under negotiation by means
of a fuzzy similarity measure. Due to the complexity
of interactions and the nature and number of possible settlements, automated negotiators require problem
solving mechanisms which are more sophisticated than
simple response mechanisms. The negotiation protocol
has been updated to account for these new mechanisms.
More generally speaking, this paper advances the state
of the art in negotiation by designing components of
a negotiation architecture which allows agents to be
both responsive and deliberative and thus participate
in more varied types of negotiation processes.
The deliberative component of the individual agent
architecture is expanded on which describe evaluation
and oer generation mechanisms. An example real
world scenario is then introduced to clarify the concepts introduced in the model. Finally, we present the
conclusions reached and future avenues of research.
Agent Negotiation Architecture
The main contribution of the research reported here is
the use of fuzzy techniques for the specication of a
QMW
University of London
N.R.Jennings@qmw.ac.uk
negotiation architecture that structures the individual
agent's reasoning throughout the problem solving. Negotiation is often characterised by the diculty faced
by agents in establishing crisp decisions. For example, preferences, comparison of contracts or evaluation
of contracts may be vague. Thus, the use of fuzzy
techniques appears very natural to extend a classical
negotiation model. In this paper we'll see the use of
fuzzy techniques to compare contracts exchanged between agents. More concretely we'll generate trade-os
as contracts that are `similar' to contracts oered by opponents by means of a fuzzy similarity measure. Also,
we'll use this measure to compute which issue to include
in a negotiation process. Future work will include the
modelling of fuzzy preferences, and the fuzzy qualitative modelling of weights or issues' importance.
Rational behaviour is assumed to consist of maximisation of some value function (Raia 1982). Given this
rationality stance, the decisions faced by agents in negotiation are often a combination of: oer generation
decisions (what initial oer should be generated, what
counter oer should be given in situations where the
opponent's oer is unacceptable), and evaluatory decisions (when negotiation should be abandoned, and
when an agreement is reached). The solution to these
decision problems is captured in the agent architecture.
The mechanisms which assist an agent with evaluation
of oers is described rst, followed by two deliberative
mechanisms and an example.
Evaluation Mechanism
The evaluation process involves computing the
value/score of a proposal or a contract. When an agent
a receives an oer x from b at time t, xtb!a , over a set of
issues J , (x = (xj1 ] : : : xjn ]) where ji 2 J ), it rates
the overall contract value using the following weighted,
linear, additive scoring function:
V a (x) =
P
X wa V a(xji ])
in
1
ji ji
(1)
where wjai is the importance (or weight) of issue ji
such that 1in wjai = 1. Given that the set of negotiation issues can dynamically change, agents need
to dynamically change the values of the weights. The
score of value xj ] for agent a, given the domain of
acceptable values Dj , is modelled as a scoring function Vja : Dj ! 0 1]. For convenience, scores are
bounded to the interval 0 1] and the scoring functions
are monotonous for quantitative issues. Note that our
formulation assumes scores of issues are independent.
Given the score of the oered contract, the contract
evaluation function will determine whether to accept
or reject the contract or whether to generate a new
contract to propose back to the other agent.
Trade O Mechanism
A trade o is where one party lowers its score on some
issues and simultaneously demands more on other issues. Thus, a trade o is a search for a new contract to
propose that is equally valuable to the previous oered
contract, but which may benet the other party.
This decision making mechanism is costly since it involves searching all, or a subset of, possible contracts
with the same score as the previously oered contract
and the selecting the a new contract to propose that is
the \closest" to the opponent's last oer. The search
is initiated by rst generating new contracts that lie
on what is called the iso-value (or indierence) curves
(Raia 1982). Because all the potential contracts lie on
the same iso-value curve the agent is indierent between
them. More formally, an iso-curve is dened as:
Denition 1 Given a scoring value , the iso-curve set
at degree for agent a is dened as:
isoa () = fx j V a (x) = g
(2)
The selection of which contract to oer is then modelled as a \closeness function". The theory of fuzzy
similarity can be used to model \closeness". The best
trade o is the one that is the most similar contract on
the iso-curve to the opponent's last oer, since it may
be benecial to the other party. This evaluation is uncertain since other party's evaluation is not known by
the proposing agent. A trade o can now be dened as:
Denition 2 Given an oer, x, from agent a to b, and
a subsequent counter oer, y, from agent b to a,, with
= V a (x), a trade o for agent a with respect to y is
dened as:
tradeo a (x y) = arg max fSim(z y)g
z2isoa ()
(3)
where the similarity, Sim, between two contracts is
dened as a weighted combination of the similarity of
the issues:
Denition 3 The similarity between two contracts x
and y over the set of issues J is dened as:
Sim(x y) =
P
X waSimj (xj] yj])
j 2J
j
(4)
With, j2J wja = 1. Simj is the similarity function for
issue j .
Following the results from (Valverde 1985), a similarity function, that is, a function that satises the
axioms of reexivity, symmetry, and t-norm transitivity, can always be dened as a conjunction (modelled
as the inmum) of appropriate fuzzy equivalence relations induced by a set of criteria functions hi . A criteria
function is a function that maps from a given domain
into values in 0 1].
The similarity between two values for issue j ,
Simj (x y) is dened as:
Denition 4 Given a domain of values Dj , the similarity between two values x y 2 Dj is dened as:
Simj (x y) =
^
(hi (x) $ hi (y))
im
1
(5)
where fh1 : : : hm g is a set of comparison criteria with
hi : Dj ! 0 1], and $ is an equivalence operator.
Simple examples of the equivalence operator,$, are
h(x) $ h(y) = 1; j h(x) ; h(y) j or h(x) $ h(y) =
min(h(y)=h(x) h(x)=h(y)).
Issue Set Mechanisms
Our other deliberation mechanism is issue set manipulation. Negotiation processes are directed and centred around the resolution of conicts over a set of issues J . It is assumed that agents begin negotiation
with a prespecied set of \core" issues, J core J , and
possibly other mutually agreed non-core set members,
J :core J . Alterations to J core is not permitted. However, elements of J :core can be altered dynamically.
Agents can add or remove issues into J :core as they
search for new possible and up to now unconsidered
solutions.
If J t is the set of issues being used at time t (where
t
J = fj1 : : : jn g), J ; J t is the set of issues not being
used at time t, and xt = (xj1 ] : : : xjn ]) is a0 s current oer to b at time t, then issue set manipulation is
dened through two operators: add and remove.
The add operator assists the agent in selecting an
issue j 0 from J ; J t , and an associated value xj 0 ], that
gives the highest score to the agent.
Denition 5 The best issue to add to the set J t is dened as:
(6)
add(J t ) = arg j2max
f max V a (xt :xj ])g
J ;J t
xj ]2Dj
where : stands for concatenation.
An issue's score evaluation is also used to dene the
remove operator in a similar fashion. This operator
assists the agent in selecting the best issue to remove
from the current negotiation set J t .
Denition 6 The best issue to remove from the set J t
(from a0 s perspective), is dened as:
remove(J t ) = arg j 2Jmax
fV a (x)g
(7)
t ;J core
i
with x = (xt j1 ] : : : xt ji;1 ] xt ji+1 ] xt jn ])
The remove operator can also be dened in terms
of the aforementioned similarity function. This type
of similarity-based remove operator selects from two
given oers x, from agent a to b, and y, from agent b
to a, which issue to remove in order to maximise the
similarity between x and y. Therefore, this mechanism
can be considered as more cooperative. We dene this
similarity based remove operator as:
Denition 7 The best issue to remove from a0 s perspective from the set J t is dened as:
remove(J t ) = arg j 2Jmax
fsim(
i t ;J core
(xj1 ] : : : xji;1 ] xji+1 ] : : : xjn ])
(yj1 ] : : : yji;1 ] yji+1 ] : : : yjn ]))g
(8)
It is not possible to dene a similarity-based add operator since the introduction of an issue does not permit
an agent to make comparisons with the opponent's last
oer (simply because there is no value oered over that
issue).
Agents deliberate over how to combine these add and
remove operators in a manner that maximises some
measure | such as the contract score. However, a
search of the tree of possible operators to nd the optimum set of issues may be computationally expensive.
To overcome this problem we intend to implement anytime algorithms and use the negotiation time limits to
compute a, possibly sub-optimal, solution. Another
computational requirement of these mechanisms is the
need for an agent to dynamically recompute the issue
weights. We dene the re-computation of weights by
rst specifying the importance of the added issue, Ij ,
with respect to the average importance of other issues.
Then:
Denition 8 The weight of added issue j , wj , is dened as:
wj = (n ; I1)j + I
j
wi = (1 ; wj )wi 8i 2 fi1 : : : in g i 6= j (9)
where wj is the importance of the issue j , n is the new
number of issues, wi is the old weight for issue i and
wi is its new weight after the inclusion of issue j . Re0
0
computation of weights when an issue is removed in
turn is dened simply as re-normalising the remaining
weights:
Denition 9 The weight of the remaining issues i after an issue j has been removed is dened as:
(10)
wi = 1 ;1w wi
j
0
An Illustrative Example
The concepts and processes outlined above will be described using an example involving negotiation between
the European Union (EU) and Morocco over shing
Issue
Reservation
Weight
Zone
fAll Central Boundariesg
0.2
Quantity
20 2]
0.35
Ships
30 5]
0.2
Price
100 5]
0.25
Figure 1: Core Negotiation Parameters for EU
Issue
Reservation
Weight
Zone
fBoundaries Central Allg
0.3
Quantity
1 10]
0.2
Ships
1 10]
0.1
Price
50 200]
0.4
Figure 2: Core Negotiation Parameters for Morocco
rights o the coast of Morocco. Negotiation between
these parties involves reaching agreements over access
rights as well as shing conditions which Morocco affords EU shing boats o its coastline.
Negotiation Parameters
Figures 1 and 2 detail the \core" set of issues involved
in negotiation for EU and Morocco respectively. Reservation values specify the ranges of acceptable values for
an issue and the weight of the issue signies the level of
importance of that issue.
The issue Zone represents the sectors of the coastal
regions where shing is permitted by Morocco. The
values of this issue are qualitatively subdivided into regions, where shing eets can sh anywhere (all), or
the central regions of the area (central) or on the outskirts of the region (boundaries). Quantity, the most
important issue for EU, represents the total tonnage
of sh (in units of millions) the shipping eet is permitted to catch. Like Zone, Ships is a qualitative issue
which represents the number of the ships allowed to sh
within Zone. Finally, Price, the most important issue
for Morocco, is the amount of money the EU will pay
Morocco for the right to sh within Zone.
Non-core issue types, which EU or Morocco can include into negotiation respectively, and their respective
parameters, are given in gures 3 and 4. Trade represents the amount of discount (in percentage) Morocco
can obtain through the sale of sh caught by EU to
Morocco. Seasons, the least important non-core issue to Morocco, qualitatively represents the seasons
where Morocco can aord EU shing rights in its territorial waters |W A Sp Su, represent winter, autumn,
spring and summer respectively. Finally, Fish represents the type of sh Morocco will permit EU boats to
catch and ranges from Tuna to Octopus and Cuttlesh.
Finally, how agents value the contracts proposed to
them is given by the value function. For the purpose
of exposition, the value of the oer x for quantitative
Issue
Reservation
Weight
Trade
10 80]
0.2
Seasons
fSu Sp A W g
0.4
Fish fTuna Octopus Cuttlefishg 0.4
Figure 3: Non Core Negotiation Parameters for EU
Issue
Reservation
Weight
Trade
90 15]
0.4
Season
fW A Sp Sug
0.1
Fish fTuna Octopus Cuttlefishg 0.3
Figure 4: Non Core Negotiation Parameters for Morocco
issues i is modelled as a simple linear function, dened
as:
maxx ;;minmin
if increasing
(11)
if decreasing
Because the values of issues fQuantity Shipsg increase with increasing levels of the oer, these issues
are increasing in value for EU (and conversely decreasing for Morocco). Alternatively, the values of issues
fPrice Tradeg decrease with increasing levels of the offer and therefore decrease in value for EU (but increase
for Morocco). The value functions for qualitative issues
fZone Ships Season Fishg are discrete in nature and
is represented in gure 5 for both EU and Morocco.
V (xi ) =
i
i
i
xi ;min
i
1 ; max
;
i mini
i
Issue Trade-O Negotiation
Assume EU begins the negotiation, oering Morocco
a contract which allows EU to sh in all zones, for
15 tones, using 18 ships for a price of 55 units,
All 15 18 55]. Using equation (1), the qualitative
scores in gure 5, and equation (11), EU scores the
value for this contract to be 0:7705. Further assume
that Morocco evaluates this contract to be unacceptEU
Issue
Reservation
Score
Zone fAll Central Boundariesg
f1 0:6 0:1g
Ships
30 5]
0:04=ship
Season
fW A Sp Sug
0:2 0:6 0:8 1]
Fish fTuna Octopus Cuttlefishg 1 0:8 0:1]
Morocco
Issue
Reservation
Score
Zone fAll Central Boundariesg
f0:1 0:6 1g
Ships
30 5]
0:11=ship
Season
fW A Sp Sug
1 0:8 0:6 0:2]
Fish fTuna Octopus Cuttlefishg 0:1 0:8 1]
Figure 5: Qualitative Values for EU & Morocco
Issue
Criteria Function
Zone
1 0:5 0]
Quantity
0:4=Ship
Ships
Offer ; min=max ; min
Price 1 ; (Offer ; min=max ; min)
Figure 6: Comparison Criteria for EU
able and therefore counter-proposes with the contract
Boundaries 8 4 100].
EU decides to oer Morocco a contract which is a
trade-o over some issues and possibly more acceptable to Morocco. A contract trade-o for EU begins
by generating all/subset of contracts that lie on the indierence curve (using equation 3). Three such points
are:
Central 15 18 24:6] All 13 18 19:2] All 15 17 51:7]
where EU has traded-o Zone for Price in the rst contract (shing in central zone only but paying less to Morocco), Quantity for Price in the second (reduced tonnage for less payment) and Ships for Price in the third
(reduced number of ships and less payments). Note,
since these are indierence contract points the value of
each of these contracts is the same as EUs rst oer,
namely 0:7705.
Figure 6 shows the comparison criteria that EU uses
for computing the similarity (using equation 5) between
each iso-contract issue and the corresponding issue
in Morocco's last oer, namely Boundaries 8 4 100].
Offer, in gure 6, refers to the number of ships Morroco or EU make to one another. The equivalence operator for comparing two values of the criteria, used in
equation 5, is 1; j h(x) ; h(y) j.
Given the above iso-contracts and criteria functions, the most similar iso-contract to Morocco's
last oer is then computed, using equation 4,
to be 0:45 0:38 and 0:43 for the iso-contracts
Central 15 18 24:6] All 13 18 19:2] All 15 17 51:7]
respectively.
Therefore, EU oers Morocco
Central 15 18 24:6] since this is the closest EU
iso contract to Morocco's last oer.
Issue Inclusion Negotiation
Assume now that Morocco evaluates, using equation 1,
EUs rst oer (Central 15 18 24:6]) to be unacceptable, and decides to include an issue into the core negotiation set. Using equation 9, and the importance levels
of non-core issues in table 4, the new set of weights after individually adding Trade, Season and Fish into
the existing set of issues are, 0:27 0:18 0:09 0:36 0:09],
0:29 0:2 0:1 0:39 0:02] and 0:28 0:19 0:09 0:37 0:07]
respectively. The nal step in deciding which issue to
include into the core negotiation set is achieved by individually adding each non-core issue into the core set and
then, using the updated weights, computing the value
for the new contract (equation 6). The contract whose
overall value is the greatest is then selected. In this case
the inclusion of Trade, Season and Fish generates contracts with overall contract value of 0:56, 0:52, and 0:55
respectively. Therefore, Morocco begins a sub-dialogue
with EU to include the issue Trade into the original dialogue. If EU accepts the inclusion of this issue then Morocco will oer the contract Boundaries 8 4 100 90],
allowing EU to sh within the boundaries of the Morocco coastline, for 8 tonnage of sh, using 4 ships, and
a payment of 100 units. Finally, Morocco also demands
a trade agreement with EU for the EU sale of sh to
Morocco at a 90% discount rate.
Related Work
The central focus of the work reported here, has been
the use of fuzzy techniques for the design of a negotiation agent architecture for structured interactions in
real environments. Our work is closely related to the
Contract Net Protocol (Davis & Smith 1988), where a
protocol is used for modelling interactions. However,
unlike the CNP we do not assume agents are cooperative and negotiation is an iterative process involving
more than two interchanges of oers. Iterative negotiation, over multiple issues and agents, is modelled by
the PERSUADER system through the concepts of argumentation and mediation (Sycara 1989). However,
negotiation, as dened in this paper, is a mutual selection of outcome and precludes any intervention by
outside parties. Other systems such as KASBAH have
attempted to actually engineer a real world application
(Chavez & Maes 1996). However, negotiation in KASBAH is between semi-autonomous agents which negotiate over a single issue and agents are semi-autonomous
|the system models only a subset of the decision making which is involved in negotiation and the user makes
all the other decisions.
Conclusions
This paper has presented a distributed negotiation
model which coordinates agent interactions. Mechanisms have been proposed for nding solutions using
fuzzy logic and based on realistic assumptions that are
practical and model the complex nature of negotiation.
The direction for future research will be primarily
focused at empirical evaluation of the developed model
to determine its properties.
References
Chavez, A., and Maes, P. 1996. Kasbah: An agent marketplace for buying and selling goods. Proceedings of the
First International Conference on Practical Applications
of Intelligent Agents and Multi-Agent Technology.
Davis, R., and Smith, R. 1988. Negotiation as a metaphor
for distributed problem solving. In Bond, A., and Gasser,
L., eds., Readings in Distributed Articial Intelligence.
Morgan Kaufmann Publishers, Inc. 333{357.
Faratin, P. Sierra, C. and Jennings, N. 1998. Negotiation
decision functions for autonomous agents. Robotics and
Autonomous Systems 24(3{4):159{182.
Pruitt, D. G. 1981. Negotiation Behavior. Academic Press.
Raia, H. 1982. The Art and Science of Negotiation. Cambridge, USA: Harvard University Press.
Sycara, K. 1989. Multi-agent compromise via negotiation.
In Gasser, L., and Huhns, M., eds., Distributed Articial
Intelligence Volume II, 119{139. San Mateo, California:
Morgan Kaufmann.
Valverde, L.
1985.
On the structure of Findistinguishability. Fuzzy Sets and Systems 17:313{328.