Laboratorul de Inteligenta Artificiala
si Sisteme Multi-Agent
Catedra Calculatoare
Universitatea Politehnica
Bucuresti
An Adaptive Negotiation Multi-Agent System for eCommerce Applications
PhD Thesis Proposal
PhD Student: Serban Radu, AI-MAS Laboratory, Computer Science Department,
University “Politehnica” of Bucharest
Supervisor: Prof. Adina Florea, AI-MAS Laboratory,
Department, University “Politehnica” of Bucharest
Computer Science
1 Introduction
A software agent is considered as an entity with goals, capable of actions endowed
with domain knowledge and situated in an environment. Multi-agent systems are
suitable for the domains that involve interactions between different people or
organizations with different goals.
The thesis proposes the development of a multi-agent system for an online
negotiation process, in which every agent is composed from a knowledge base and
an inference engine. The knowledge base will have a structure and a set of rules.
The agents will negotiate, based on a negotiation language, which contains a set of
primitives, and also based on negotiation criteria, represented as rules. The
negotiation process will be improved using innovative learning algorithms. The use
of learning techniques will be investigated, in order to allow agents to reuse their
negotiation experience to improve the final outcomes. The learning mechanism will
develop strategies in negotiation. Experimental results on specific problems will
evaluate the quality of the learning algorithms.
2 State of the Art
Negotiation surrounds our lives. Research in the field of automated negotiations
has suggested the design and use of automated negotiators, on one hand to allow
facilitation of the negotiation process by human negotiators and, on the other hand,
to provide automated agents that can negotiate on behalf of humans [10].
Negotiation in electronic commerce is the process in which two or more parties
multilaterally bargain resources for mutual intended gain, using the tools and
techniques of electronic commerce [2]. Automated negotiations take place when
the negotiating function is performed by computers.
When conceptualizing automated negotiations, negotiation protocols (or
mechanisms) and negotiation strategies have to be distinguished. The protocol
defines ”rules of encounter” between negotiation participants by specifying the
requirements that enable their interaction. The strategy defines the behavior of
participants aiming at achieving a desired outcome. This behavior must be
consistent with the negotiation protocol, and usually aims at maximizing individual
”gains” of each of negotiation participants [1].
There are three major classes of approaches to automated negotiation in the multiagent domain: game-theoretic approach, heuristic-based approach, and
argumentation-based approach.
Game-theoretic approaches of automated negotiation assume that agents have
unbounded computational resources and that the space of outcomes is completely
known. In most realistic environments, these assumptions fail, due to the limited
processing and communication capabilities of the information systems.
Heuristic-based approaches overcome the shortcomings of game-theoretic
approaches, but they have a number of disadvantages. The models lead to
outcomes that are sub-optimal, because they adopt an approximate notion of
rationality and because they do not examine the full space of possible outcomes.
Also, it is very difficult to predict how the system and the agents will behave.
The negotiation heuristic strategies are domain dependent and time consuming. A
reinforcement learning approach may permit the negotiator to learn which
negotiation primitive to use in a certain state of the negotiation [6].
Argumentation-based approaches attempt to overcome the limitations of other
approaches, by allowing agents to exchange additional information, or to argue
about their beliefs and other mental attitudes during the negotiation process. In the
context of negotiation, an argument is a piece of information that may allow an
agent to justify its negotiation position or to influence another agent’s negotiation
position [12].
From the transactional point of view, there are the following types of electronic
commerce: business-to-business (B2B), business-to-consumer (B2C), consumerto-consumer (C2C), consumer-to-business (C2B), non-business (using the Internet
by non-profit organizations), intra-business. Most of the e-commerce applications
are B2C or B2B [13].
In order to analyze the tasks of an intelligent agent, it is used the client buyer
behavior (CBB) model, for modeling the consumer behavior [14]. The CBB model
considers that the B2C e-commerce takes place in seven steps: needs
identification, product brokerage, buyers’ coalition formation, seller brokerage,
negotiation, purchase and delivery, services evaluation. Using this model, it is
supposed that the intelligent agent acts as an intermediate in five of these steps:
needs identification, product brokerage, buyers’ coalition formation, seller
brokerage and negotiation.
The main techniques used for product brokerage are: characteristics filtering,
collaborative filtering and constrained-based filtering [7]. Characteristics filtering
select the products based on associated keywords (characteristics). Collaborative
filtering refers to sending personalized recommendations to an agent, based on
similarities between different profiles of users’ preferences. Constrained-based
filtering implies an agent which specifies the imposed constrains upon the product.
Systems that use software agent technologies are proving to be effective in helping
users make better decisions when buying or selling over the Internet. Software
agents can also play an important role in providing automation and support for the
negotiation stage of online trading.
Agent-based systems have been implemented, which support various stages of
online trade: product brokering (determining what to buy), merchant brokering
(determining who to buy from), and negotiation, where all parties involved
communicate in order to reach an agreement on the terms of transactions. An
example of such system is Kasbah (Chavez and Maes-1996). More sophisticated
automated trading systems have been proposed, which offer multi-attribute
intelligent matching, such as Tete-a-Tete (Guttman, Moukas and Maes-1997) and
ITA (Kowalczyk and Bui-2000) [11].
An ontology is a way of categorizing objects such that they are semantically
meaningful to a software agent. Ontology defines the relationship between similar
products.
A negotiation framework should specify a negotiation protocol, in order to constrain
the use of the language. A protocol is a formal set of conventions governing the
interaction among participants [12]. The negotiation protocol defines the formal
interaction between the negotiators, whether the negotiation is done only once
(one-shot) or repeatedly, and how the exchange of offers between the agents is
conducted [9]. The negotiation strategy is an important issue. If one agent’s
negotiation strategy is known to the other agent, the first agent may be at a
significant disadvantage. Strategy means an analysis of the counterpart’s
negotiation strategy and offering a responding proposal.
Bazaar (Zeng and Sycara-1996) is an experimental system for updating negotiation
offers between two intelligent agents during bilateral negotiations. It models
negotiation as a sequential decision making task, and uses Bayesian probability as
the underlying learning mechanism. The price is used as the issue of negotiation.
Most preceding studies on the negotiation systems are not for an automated
negotiation system, but for a Negotiation Support System, that supports the
negotiations between buyers and sellers. The reason is that in case of multi-issue
negotiation, it is not easy to evaluate many negotiation issues, consequently
making it difficult to develop an automated negotiation system.
Under current e-commerce environment, an automated negotiation system is
critical in dealing with complex problems and diverse changes in business
environment [3]. The multi-agent frameworks have simple functions, such as the
generation of agent, supporting the conversation between agents and
management of agents, but they don’t have the function to support negotiation
attributes.
Existing multi-agent frameworks have no function to define the category and
relationship between products. These functions are necessary if the conditions of a
negotiation don’t coincide. Negotiation messages should be prepared and used to
bring a better result of the negotiation.
Negotiation can be viewed as a process of cooperative and competitive decision
making between self-interested agents in the presence of incomplete information. It
is a process in which disputing agents decide how to divide the gains from
cooperation [5].
The agents have limited information about the preferences and constraints of each
other. They make decisions according to available information about private
preferences, constraints and individual negotiation strategies. The agents
exchange information in the form of offers. An offer is a complete solution, which is
currently preferred by an agent, given its preferences, constraints and the
negotiation history of offers and counteroffers.
An agreement takes place when a particular offer is accepted by all negotiation
parties. During the negotiation process, the range of possible offers of each agent
changes according to the current information available. These ranges typically
reduce to the final agreement, or if they become empty, a deal is not possible and
the negotiation ends unsuccessfully. Therefore, negotiation is typically an iterative
process of evaluating the offers, updating the available options, and making the
counteroffers, according to the individual negotiation strategies.
An automated negotiation system has three steps: the estimation step, the
negotiation making step, the negotiation concluding step. In the stage of an
estimate, upon the buyer’s request for a written estimate, the seller reviews his
process planning, performing cost accounting, and suggests his estimate to the
buyer. During the stage of making negotiations and concluding negotiations, one
agent evaluates the other agent’s negotiation proposal, building one’s own
negotiation strategy and sending his proposal to the other.
The cooperation between agents should be made by means of a common
message through which the request for a work will be able to be made and the
result to be sent. In general, messages between agents under the multi-agent
environment are based on FIPA-ACL and KQML.
Effective and efficient multi-issue negotiation requires an agent to have some
indication of its opponent’s preferences. However, in competitive domains, such as
e-commerce, an agent will not reveal this information and so the best that can be
achieved is to learn some approximation of it through the negotiation exchanges
[4].
The negotiation proposals between the seller and the buyer are based on XML,
which makes possible to represent multi-attribute negotiation items.
Machine learning is widely adopted in negotiation learning mechanisms.
Negotiation parties use various learning techniques to speculate and update their
knowledge about environments and other parties, and these knowledge are good
support to their later decisions. The past negotiation records play an important role
in all these learning techniques, because they compose the training data sets for
learning algorithms. As the larger training set probably implies more accurate
learning results and more profits, negotiation parties have the desire to obtain more
related negotiation records. If several negotiation parties have the same
negotiation learning objective and meanwhile they do not have any direct profit
conflicts (e.g., a group of buyers all conduct bilateral negotiations with a third party
supplier), sharing their negotiation records will bring all of them benefits [8].
Real-world negotiation scenarios such as those found in B2B environment are
characterized by combinatorial complex negotiation spaces, tough negotiation
deadlines, limited information about the opponents, and volatile negotiator
preferences. Therefore, practical negotiation systems must be equipped with
effective learning mechanisms to automatically acquire domain knowledge from the
negotiation environments and continuously adapt to the dynamic negotiation
contexts [15].
The framework will be composed of a facilitator and a mediator. The facilitator
performs the function of relaying messages and managing agents, and the
mediator provides the information on facilitator’s location. The facilitator is in
charge of a server function to relay the message between agents, and the mediator
is in charge of a server function to support the facilitators.
Rule-based approaches represent a technique to parameterize the negotiation
design space in multi-agent systems. Rules can be used for describing both
strategies and mechanisms of automated negotiations [1].
3 Research Goals and Challenges
The main goal of the thesis is to develop a set of adaptive negotiation strategies, a
model of autonomous entities that use these strategies and an associated
implementation based on multi-agent system technology. The proposed model and
implementation are aimed to support the development of advanced e-commerce
applications in which the users are represented by autonomous agents that can
adapt their negotiation strategies to both context and user preferences.
The proposed strategies and model will be validated by building an e-commerce
environment that supports the development of several business models and ecommerce scenarios.
When designing the model, the following requirements for a negotiation protocol
should be fulfilled:
1.
Be sufficiently formal that automated entities can interact using it.
2.
Support negotiation about simple and complex objects.
3.
Be sufficiently general that a variety of different market mechanisms (e.g. 1-1
negotiation, combinatorial auctions, exchanges) can be expressed as specific
instances of it.
4.
Support security mechanisms and protocols that enable participants to do
business in a trusted way.
5.
Allow, but not require, the existence of a third party to arbitrate a given
negotiation (e.g. an auctioneer in an auction)
6.
Support existing ways of doing business, as well as permitting more radical
approaches in the future.
Regarding the negotiation strategies, some decision taking assistance techniques
will be used in the thesis, such as rule-based reasoning and machine learning
techniques. A specific agent will be programmed to work in many business
domains and will be possible to reconfigure the agent’s business domains at
execution time.
The environment will allow automated negotiations, offering support for the use of
various existing negotiation models, together with their respective negotiation
strategies. In this way it is possible to integrate different models, approximating the
automated negotiations to the way the real world works.
Multi-criteria techniques from decision theory will be introduced in the evaluation,
by the agents, of the exchanged offers and counter-offers and an associated
communication protocol will be developed to support the negotiation protocol.
Individual agents will use multi-issue negotiation by exchanging offers and counteroffers, until they either reach a consensus that satisfies each party's private
preferences and constraints, or they run out of offers and the negotiation fails.
These agents will use multi-attribute utility theory and constraint based reasoning
for the evaluation and generation of offers. Multi-attribute negotiation is important
for agents to reach agreements on multiple issues, but it is much more complicated
than a single-attribute negotiation
The architecture for the negotiation environment will be flexible and open, in which
the number of buyers and sellers could be changed during execution time. In a
flexible way, the negotiator agent could grow in the number of negotiation
strategies and also in the number of business domains it is involved. These
characteristics will be implemented in the environment using ontology concepts
and rule-based systems.
A flexible and interoperable system, where knowledge, negotiation protocol and
strategies are explicitly represented in rules will be developed.
The agents will be rule based and the rules will express both the knowledge of the
domain and the negotiation strategy. The rules will be expressed in a language like
RuleML or JESS, for ensuring the interoperability among agents.
An agent will be modeled as a knowledge based system and will be composed
from the knowledge base, the inference engine and the control part. The
knowledge base will have two components, one component to represent the rules
and knowledge about the objects of the domain, and also one component to
represent explicitly the negotiation rules and strategy.
Practical applications of the automated negotiation multi-agent system could be
designed for an automated credit assignment system to clients. There will be a
knowledge base with all the possible types of credits from all the banks in a
country. The agents will negotiate with the banks on behalf of humans. In this way,
persons will choose the best credit for their needs, which fulfills their requirements.
The application will have an intuitive interface and will contain the attributes upon
which the credit is assigned. Depending on the priority of attributes, given by
humans, the agents will negotiate for them, learning new rules during negotiation
steps. The rules will be added to the knowledge base, thus improving the future
behavior of the agents during negotiations. Learning customer preferences during
negotiation requires a learning algorithm, since the training data is gained during
the negotiation process. The learning algorithm should filter unnecessary offers
quickly and thus should speed up the negotiation process considerably. The
agents should learn patterns of negotiations, depending on different types of clients
and their requirements, and should negotiate with the bank’s agents accordingly.
Also, the agents could learn new negotiation strategies and discover new rules to
be applied, based on feedback received from humans.
Agents will learn the preferences of customers over time and will negotiate based
on this new knowledge. The customer’s preferences will be specified as conditional
rules and the agents will learn and guide the bank’s agents to create well-targeted
offers.
In order to negotiate successfully, agents need to consider each others’ agents
preferences and generate offers accordingly. Agents may learn about each others’
preferences over time and through interactions. As agents learn about each others’
preferences, they can provide better-targeted offers and thus enable faster
negotiation.
During the learning process, customer’s requests are accepted as a positive
example, whereas the bank’s counter-offers that are rejected by the consumer are
taken as negative examples. According to the learning algorithm, bank’s services
are filtered, so that the bank selects a service as an offer from a smaller set of
possible services.
4 Research Reports
Two research reports will be developed.
Adaptive Negotiation Strategies for e-Commerce
The first report will present the state of the art in automated negotiation and
learning techniques, and will propose a set of strategies to be followed for
achieving adaptive negotiation. E-Commerce applications will drive the features of
these strategies.
A Multi-Agent System Model for Adaptive Negotiation
The second report will develop the multi-agent system able to perform automated
negotiation, with agents able to adapt to different contexts and user preferences. It
will also propose a demonstrator for the system architecture.
Bibliography
[1] C. Badica, A. Badita, M. Ganzha, Implementing rule-based mechanisms for
agent-based price negotiations, Proceedings of the 2006 ACM Symposium on
Applied Computing, p. 96-100, 2006
[2] C. Beam, A. Segev, Automated Negotiations: A Survey of the State of the Art,
CMIT Working Paper 97-WP-1022, 1997
[3] H.R. Choi, H.S. Kim, S.G. Hong, Y.J. Park, Y.S. Park, M.H. Kang,
Implementation of Framework for Developing Multi-Agent based Automated
Negotiation Systems, Proceedings of the 7th International Conference on Electronic
Commerce, p. 306-315, 2005
[4] R. Coehoorn, N. Jennings, Learning an Opponent’s Preferences to Make
Effective Multi-Issue Negotiation Trade-Offs, Proceedings of the 6th International
Conference on Electronic Commerce, p. 59-68, 2004
[5] S. Fatima, M. Wooldridge, N.R. Jennings, Approximate and Online Multi-Issue
Negotiation, Proceedings of the 6th International Joint Conference on Autonomous
Agents and Multi-Agent Systems, p. 951-958, 2007
[6] A.M. Florea, E.Kalisz, A Negotiation Learning Model for Open Multi-Agent
Environments, International Journal of Computing Anticipatory Systems (IJCAS),
Vol. 20, 2008, CHAOS, Liege, Belgium, p.121-130, 2008
[7] R.H. Guttman, P. Maes, Agent-mediated electronic commerce. A survey, The
Knowledge Engineering Review no. 13(2), p. 147-159, 1998
[8] R. Lau, Adaptive Negotiation Agents for E-business, Proceedings of the 7th
International Conference on Electronic Commerce, p. 271-278, 2005
[9] R. Lin, S. Kraus, Can Automated Agents Proficiently Negociate with Humans ?,
Communications of the ACM, vol. 53, issue 1, 2010
[10] R. Lin, Y. Oshrat, S. Kraus, Investigating the Benefits of Automated
Negotiations in Enhancing People’s Negotiation Skills, Proceedings of the 8th
International Conference on Autonomous Agents and Multi-Agent Systems, Vol. 1,
p. 345-352, 2009
[11] I. Rahwan, R. Kowalczyk, H.H. Pham, Intelligent Agents for Automated Oneto-Many e-Commerce Negotiation, Proceedings of the 25th Australasian
Conference on Computer Science, p. 197-203, 2002
[12] I. Rahwan, S.D. Ramchurn, N.R. Jennings, P. McBurney, S. Parsons, L.
Sonenberg, Argumentation-Based Negotiation, The Knowledge Engineering
Review, Vol. 18, p. 343-375, 2003
[13] C.M. Sachi, B. O’Leary, The role of Internet auctions in the expansion of B2B
markets, Industrial Marketing Management no. 31, p. 103-110, 2002
[14] Y. Ye, J. Liu, A. Moukas, Agents in electronic commerce, Electronic
Commerce Research no. 1(1-2), p. 9-14, 2001
[15] S. Zhang, F. Makedon, Privacy Preserving Learning in Negotiation,
Proceedings of the 2005 ACM Symposium on Applied Computing, p. 821-825,
2005