Chapter 15 Bargaining

Negotiation may involve:

Exchange of information

Relaxation of initial goals

Mutual concession
1
Mechanisms, Protocols, Strategies

Negotiation is governed by a mechanism or a
protocol:

defines the ”rules of encounter” between the agents

the public rules by which the agents will come to
agreements.

The deals that can be made

The sequence of offers and counter-offers that can be made
2
Negotiation Mechanism
Negotiation is the process of reaching agreements on matters of
common interest. It usually proceeds in a series of rounds, with
every agent making a proposal at every round.
Issues in negotiation process:
• Negotiation Space: All possible deals that agents can make, i.e., the
set of candidate deals.
• Negotiation Set : deals that are pareto optimal and individually
rational
• Negotiation Protocol: – rules to determine the process: how/when a
proposal can be made, when a deal has been struck, when the
negotiation should be terminated, and so.
• Negotiation Strategy: When and what proposals should be made.
• Agreement Deal: rule to determine when a deal has been struck.
7-3
Typical Goals of Negotiation





Efficiency – not waste utility. Pareto Optimal
Stability – no agent have incentive to deviate
from dominant strategy
Simplicity – low computational demands on
agents
Distribution – no central decision maker
Symmetry (possibly) – may not want agents to
play different roles.
4
Negotiation Protocol









Who begins
Take turns
single or multiple issues
Build off previous offers
Give feed back (or not). Tell what utility is (or not)
Obligations – requirements for later
Privacy (not share details of offers with others)
Allowed proposals you can make as a result of
negotiation history
process terminates (hopefully)
5
Simple Model




The rewards to be gained from negotiation are
fixed and divided between the two parties
Suppose (x, 1-x) represents the portion of the
utility each person gets.
If the number of rounds are
fixed, a person can

propose (, 1-) on the last round. Theoretically,
a person would rather have  than nothing.
Even in multiple rounds: If you knew that the
other agent wouldn’t offer you more than , it
would be in your best interest to accept it. (So
we have a pair of Nash Equilibrium strategies)6
Impatient players – shrinking pie





Discount factor of  (a fraction in [0,1]) is applied
to the utility.
At time t, the value of slice x is tx
The larger value of , the more patient the player
is.
Typically the players will have different values
for , say 1 and 2
if player 1 offers player 2, 2, he can do no
better than accept it.
7

The following offer could be accepted in the
first time step (assuming agent 1 would make
the first proposal):
[ (1- 2)/(1-12), 2(1-1)/(1-12)]
Example: 1 =.8, 2 = .9 yields:
(.36, .64)
Example: 1 =.8, 2 = .8 yields:
(.56, .44)
8
Negotiation Process 1

Negotiation usually proceeds in a series of
rounds, with every agent making a proposal at
every round.

Communication during negotiation:
Proposal
Counter Proposal
Agenti concedes
Agenti
Agentj
9
Negotiation Process 2

Another way of looking at the negotiation
process is (can talk about 50/50 or 90/10
depending on who ”moves” the farthest):
Proposals by Ai
Point of
Acceptance/
aggreement
Proposals by Aj
10
Single issue negotiation


Like money
Symmetric (If roles were reversed, I would benefit the
same way you would)



If one task requires less time, both would benefit equally by
taking less time
utility for a task is experienced the same way by whomever is
assigned to that task.
Non-symmetric – we would benefit differently if roles
were reversed

negotiate about who picks up an item, but you live closer to
the store
11
Multiple Issue negotiation




Could be hundreds of issues (cost, delivery date, size,
quality)
Some may be inter-related (as size goes down, cost
goes down, quality goes up?)
Not clear what a true concession is (larger may be
cheaper, but harder to store or spoils before can be
used)
May not even be clear what is up for negotiation (I
didn’t realize having a bigger office was an option) (on
the job…Ask for stock options, travel compensation,
work from home, 4- day work week.)
12
How many agents are involved?



One to one
One to many (auction is an example of one
seller and many buyers)
Many to many (could be divided into buyers
and sellers, or all could be identical in role –
like officemate)

n(n-1)/2 number of pairs
13
Jointly Improving Direction method
Iterate over
 Mediator helps players criticize a tentative agreement
(could be status quo)
 Generates a compromise direction (where each of
the k issues is a direction in k-space)
 Mediator helps players to find a jointly preferred
outcome along the compromise direction, and then
proposes a new tentative agreement.
14
Various Domains
Worth Oriented Domain –
maximize value to all
State Oriented Domain
Get to a state we both agree to
Task Oriented Domain
Typical Negotiation Problems
Task-Oriented Domains(TOD): each agent has set of tasks that it has
to achieve. The target of a negotiation is to minimize the cost of
completing the tasks by divvying them up differently.
State Oriented Domains(SOD): each agent is concerned with moving
the world from an initial state into one of a set of goal states. The target
of a negotiation is to achieve a common goal. Main attribute: actions
have side effects (positive/negative). TOD is a subset of SOD.
Agents can unintentionally achieve one another’s goals. Negative
interactions can also occur. Utility = worth of goal – cost to achieve it
Worth Oriented Domains(WOD): agents assign a worth to each
potential state (via a function), which captures its desirability for the
agent. The target of a negotiation is to maximize mutual worth (rather
than worth to individual). Superset of SOD.
Rates the acceptability of final states. Allows agents to compromise
on their goals.
16
Negotiation Domains:Task-oriented
”Domains in which an agent’s activity can be defined in terms of
a set of tasks that it has to achieve”, (Rosenschein & Zlotkin, 1994)

An agent can carry out the tasks without interference (or
help) from other agents – such as ”who will deliver the mail”

Any agent can do any task.

Tasks redistributed for the benefit of all agents
17
Types of deals




Conflict deal: keep the same tasks as had
originally
Pure – divide up tasks
Mixed – we divide up the tasks, but we decide
probabilistically who should do what
All or Nothing (A/N) - Mixed deal, with added
requirement that we only have all or nothing
deals (one of the tasks sets is empty)
18
Examples of TOD
Parcel Delivery:
Several couriers have to deliver sets of parcels to different cities. The
target of negotiation is to reallocate deliveries so that the cost of travel
for each courier is minimal.
 Database Queries:
Several agents have access to a common database, and each has to
carry out a set of queries. The target of negotiation is to arrange queries
so as to maximize efficiency of database operations (Join, Projection,
Union, Intersection, …) . e.g., You are doing a join as part of another
operation, so please save the results for me.

19
Consider tasks. 1 delivers to a. 2 delivers to both. Must return
home. Can’t find a deal where both win. Try mixed deal.
Distribution Point
3
city a
3
2
city b
Cost function:
c()=0
c({a})=6
c({b})=6
c({a,b)}=8
Utility for agent 1 (org {a}):
1. Utility1({a}, {b}) = 0
2. Utility1({b}, {a}) = 0
Utility for agent 2 (org {ab}):
1. Utility2({a}, {b}) = 2
2. Utility2({b}, {a}) = 2
3. Utility1({a, b}, ) = -2
3. Utility2({a, b}, ) = 8
4. Utility1(, {a, b}) = 6
…
4. Utility2(, {a, b}) = 0
…
20
What mixed deals are possible if
splitting utility is our goal?
21
Consider deal 3 with probability







({},{ab}):p means agent 1 does {} with p probability
and {ab} with (1-p) probability.
What should p be - to be fair to both (equal utility)
(1-p)(-2) + p6 = utility for agent 1
(1-p)(8) + p0 = utility for agent 2
(1-p)(-2) + p6= (1-p)(8) + p0
-2+2p+6p = 8-8p => p=10/16
If agent 1 does no deliveries 10/16 of the time, it is
fair.
Note how a mixed deal allows us to be fair.
22
Incomplete Information


Don’t know tasks of others in TOD.
Solution



Exchange missing information
Penalty for lie
Possible lies

False information


Hiding letters (don’t admit part of your job)
Lie about letters (claim work that isn’t required)



decoy – produce if needed
phantom – can’t produce, caught in lie
Not carry out a commitment (trust: misrepresent or
unreliable)
23
Difficult to think about




So many situations
So many kinds of lies
So many kinds of deals
Approach – divide into special cases so we
can draw conclusions
24
Subadditive Task Oriented Domain
Cost of whole is ≤ cost of parts


for finite X,Y in T, c(X U Y) <= c(X) + c(Y)).
Examples of subadditive (c(X U Y) < c(X) + c(Y)).):
 Deliver to one, saves distance to other (in a tree arrangement if
have to return home)

Example of subadditive TOD (c(X U Y) = c(X) + c(Y)).

deliver in opposite directions
–doing both saves nothing

Not subadditive: doing both actually costs more than the sum of
the pieces. Example: One way delivery. Doing both causes
backtracking.
25
Tasks that don’t exist

We call producible tasks decoy tasks (no risk of being
discovered). Unproducible non-existent tasks are
called phantom tasks.

Example decoy: Need to pick something up at store.
(Can think of something for them to pick up, but if you
are the one assigned, you won’t bother to make the
trip.)
Example phantom: Need to deliver load (but
recipient won’t accept unwanted item)

26
What if phantom task to furthest point?
What if hidden task to b?
Agent 1: f and b (hides)
Agent 2: e
Must return to postoffice
28
Incentive compatible Mechanism
Are the rules (in terms of allowable deals) we establish
sufficient to produce truth telling?



L there exists a
beneficial lie in
some encounter
T  There exists no
beneficial lie.
T/P  Truth is
dominant if the
penalty for lying is
stiff enough.
Example indicates a case where lying helps. (Assume we deliver and return to
post office.) Can you see it? Who lies? What is lie?
29
Explanation of arrow

If it is never beneficial in a mixed deal
encounter to use a phantom lie (with
penalties), then it is certainly never beneficial
to do so in an all-or-nothing mixed deal
encounter (which is just a subset of the mixed
deal encounters).
30
Concave (a special case of subadditive):
c(YU Z) –c(Y) ≤c(XU Z) –c(X)
•X is subset of Y
•The cost that tasks Z add to set of tasks Y
cannot be greater than the cost Z add to a
subset of Y
•Expect it to add more to subset (as is smaller)
•At seats – is postmen domain concave ?
Example: Y is in pacman shape, X is nodes in
polygon.
adding Z adds 0 to X (as was going that
way anyway) but adds 2 to its superset Y
(as was going around loop)
• Concavity implies sub-additivity
•Modularity: c(YU Z) –c(Y) =c(XU Z) –c(X)
•Modularity implies concavity
y
Concave Task Oriented Domain


We have 2 tasks X and Y, where X is a subset of Y
Another set of task Z is introduced
 c(YU Z) –c(Y) ≤c(XU Z) –c(X)
32
Explanation of Previous Chart



Arrows indicate reasons we know this fact (diagonal
arrows are between domains). For example, What is true
of a phantom task, may be true for a decoy task in same
domain as a phantom is just a decoy task we don’t have
to create.
Similarly, what is true for a mixed deal may be true for an
all or nothing deal (in the same domain) as a mixed deal
is an all or nothing deal where one choice is empty. The
direction of the relationship may depend on truth (never
helps) or lie (sometimes helps).
The relationships can also go between domains as subadditive is a superclass of concave and a super class of
modular.
33
Modular TOD



c(X U Y) = c(X) + c(Y) - c(X Y).
X and Y are sets of tasks
Notice modular encourages truth telling, more than others
34
Implied relationship between cells
Implied relationship between domains (slanted arrows).
L means lying may be beneficial
T means telling the truth is always beneficial
T/P Truth telling is beneficial if penalty for being caught is great
35
Incentive Compatible Facts
(assume we must return to post office)
Fact1: in SubadditiveTOD, any Optimal
Negotiation Mechanism over all or nothing
deals, “hiding” lies are not beneficial
 Ex: A1 = {b,c}, A2 = {a,b}
 A1hides letter to c, his utility doesn’t
increase.
 If he tells truth : p=1/2
 Expected util ({abc}{})1/2 = 5
 Lie: p=1/2 (as both still go around). This is
obviously bad as you deliver all at the same
rate, but never deliver nothing.
 Expected util (for 1) ({abc}{})1/2 = ½(0) +
½(2) = 1 (as has to deliver to c)
1
4
4
1
Fact 2 Modular, all or nothing, decoy
encourages truth telling.
Both deliver to e and b. Suppose agent 2 lies about
having a delivery to c.
Under Truth, p (prob of agent 1 doing everything)
would be ½. Utility = 3.
Under Lie:
If we assign p ({ebc}, ) p
agent 1 utility -2*p + 6(1-p)
Agent 2 (under lie) 8p+0(1-p)
-2*p + 6(1-p)= 8p+0(1-p)
-8p+6 = 8p p=6/16 (so 2 is worse off)
Obviously, we
can’t prove we
need to tell the
truth from an
example.
Originally each is assigned five places











Fact3:in Modular TOD, any ONM over Mixed deals,
“Hide” lies can be beneficial. A1={acdef} A2={abcde}
A1 hides his letter to node a
Under truth, each delivers to three.
Under truth Util({fae}{bcd})1/2 = 4 (save going to two)
Under lie, can we divide as ({efd}{cab})p ?
Try lie again, under ANOTHER division ({ab}{cdef})p
p(4) + (1-p)(0) = p(2) + (1-p)(6)
4p = -4p + 6
p = 3/4
Utility is actually
3/4(6) + 1/4(0) = 4.5
39
Conclusion


In order to use Negotiation Protocols, it is
necessary to know when protocols are appropriate
TOD’s cover an important set of Multi-agent
interaction
40
Various Domains
Worth Oriented Domain –
maximize value to all
State Oriented Domain
Get to a state we both agree to
Task Oriented Domain
State Oriented Domain





Goals are acceptable final states (superset of TOD)
Example – Slotted blocks world -blocks cannot go anywhere on table – only in slots
(restricted resource)
Have side effects - agent doing one action might hinder or help another agent.
Example in blocks world, on (white,gray) has side effect of clear(2).
Negotiation : develop joint plans (what they each do) and schedules for the agents, to
help and not hinder other agents
Note how this simple change (slots) makes it so two workers get in each other’s way
even if goals are unrelated.
42
Assumptions of SOD






Agents will maximize expected utility (will prefer 51%
chance of getting $100 than a sure $50)
Agent cannot commit himself (as part of current
negotiation) to behavior in future negotiation.
No explicit utility transfer (no side-payment that can be
used to compensate one agent for a disadvantageous
agreement)
Inter-agent comparison of utility: common utility units
Symmetric abilities (all can perform tasks, and cost is
same regardless of agent performing)
Binding commitments
43
Achievement of Final State



Goal of each agent is represented as a set of states that they
would be happy with.
Looking for a state in intersection of goals
Possibilities:
 (GREAT) Both can be achieved, at gain to both (e.g. travel to
same location and split cost)
 (IMPOSSIBLE) Goals may contradict, so no mutually acceptable
state (e.g., both need the car)
 (NEED ALT) Can find common state, but perhaps it cannot be
reached with the primitive operations in the domain (could both
travel together, but may need to know how to pickup another)
 (NOT WORTH IT) Might be a reachable state which satisfies both,
but may be too expensive – unwilling to expend effort (i.e., we
could save a bit if we car-pooled, but is too complicated for so little
gain).
44
Examples: Cooperative
Each is helped by joint plan




Slotted blocks world: initially white block is at 1 and
black block at 2. Agent 1 wants black in 1. Agent 2
wants white in 2. (Both goals are compatible.)
Assume pick up is cost 1 and set down is one.
Mutually beneficial – each can pick up at the same
time, costing each 2 – Win – as didn’t have to move
other block out of the way!
If done by one, cost would be four – so utility to each
is 2.

45
Examples: Compromise
Both succeed, but worse for both than if other agent was
not present.

Slotted blocks world: initially white block is at 1 and black block at
2, two gray blocks at 3.





Agent 1 wants black in slot 1, but not on table.
Agent 2 wants white in slot 2, but not directly on table.
Alone, agent 1 could just pick up black and place on white.
Similarly, for agent 2. But would undo others goal.
But together, all blocks must be picked up and put down. Best
plan: one agent picks up black, while other agent rearranges (cost
6 for one, 2 for other)
Can both be happy, but unequal roles.

46
Example: conflict





I want black on white (in slot 1)
You want white on black (in slot 1)
Can’t both win. Could flip a coin to decide who wins.
Better than both losing. Weightings on coin needn’t
be 50-50.
May make sense to have person with highest worth
get his way – as utility is greater. (Would
accomplish his goal alone) Efficient but not fair?
What if we could transfer half of the gained utility to
the other agent? This is not normally allowed, but
could work out well.
47
Negotiation Domains:
Worth-oriented – more flexible

”Domains where agents assign a worth to each potential
state (of the environment), which captures its desirability for
the agent”, (Rosenschein & Zlotkin, 1994)

agent’s goal is to bring about the state of the environment with
highest value

we assume that the collection of agents have available a set of
joint plans – a joint plan is executed by several different agents

Note – not ”all or nothing” – but how close you got to goal.
48
Worth Oriented Domain




Rates the acceptability of final states
Allows partially completed goals
Negotiation : a joint plan, schedules, and goal relaxation. May
reach a state that might be a little worse that the ultimate
objective
Example – Multi-agent Tile world (like airport shuttle) – isn’t just
a specific state, but the value of work accomplished
49
How can we calculate Utility?

Weighting each attribute


Rating/ranking each attribute


Utility = {Price*60 + quality*15 + support*25}
Price : 1, quality 2, support 3
Using constraints on an attribute


Price[5,100], quality[0-10], support[1-5]
Try to find the pareto optimum
50
What if choices don’t benefit others fairly?




Suppose there are two states that satisfy both
agents.
State 1: one has a utility of 6 for one agent
and 3 for the other.
State 2: utility of both agents 4.
State 1 is better (overall), but state 2 is more
equal. How can we get cooperation (as why
should one agent agree to do more)?
51
Resource Allocation



In the TOD, suppose we have multiple agents
who possess tasks and we want to reallocate
by paying them to do the task (a side
payment).
So each agent currently has the tasks:
(a,b,c,d,e) and after reallocation the tasks
become (d,e,a,b,c) with payments (x,y,z,q,r)
52

There are several protocols:






O-contracts – One contract(1-1) Agent moves one
task at a time
C-contracts – Cluster contracts (1-1) Agent moves
many tasks at a time
S-contracts - Swap contracts (1-1) two agents swap
tasks (and make side payments)
M-contract – Multi-agent (n-n) n agents transfer a
single task (and make side payments)
All of these have the downside that you may get
worse before you get better.
Better if there are no restrictions on how the deals are
formed at various steps. Then just accept if it is
better for you.
53
The Monotonic Concession Protocol – One
direction, move towards middle
Rules of this protocol are as follows. . .
 Negotiation proceeds in rounds.
 On round 1, agents simultaneously propose a deal from the negotiation set
Can re-propose same deal.
 Agreement is reached if one agent finds that the deal proposed by the
other is at least as good or better than its proposal.
 If no agreement is reached, then negotiation proceeds to another round of
simultaneous proposals.
 An agent is not allowed to offer the other agent less (in term of utility ) than
it did in the previous round. It can either stand still or make a concession.
Assumes we know what the other agent values.
 If neither agent makes a concession in some round, then negotiation
terminates, with the conflict deal.
 Meta data may be present: explanation or critique of deal.
54
Condition to Consent to an Agreement
If both of the agents finds that the deal proposed by the
other is at least as good or better than the proposal it
made – randomly pick one.
Utility1(2) Utility1(1)
and
Utility2(1) Utility2(2)
55
The Monotonic Concession Protocol
Advantages:


Symmetrically distributed (no agent plays a special role or
gets to go first)

Ensures convergence - It will not go on indefinitely
Disadvantages:


Can result in conflict deal – when a better solution is
possible

Inefficient – no quarantee that an agreement will be
reached quickly
56
The Zeuthen Strategy – a refinement of
monotonic protocol
Q: What should my first proposal be?
A: the best deal for you among all possible deals in the
negotiation set. (Is a way of telling others what you
value.)
Agent 1's best deal
agent 2's best deal
57
The Zeuthen Strategy
Q: Who should compromise?
A: The one who has the most to lose by a conflict deal
Agent 1 has more to lose
As he’s already gained
agent 2 has less to
lose from conflict
58
Example of Zeuthan


In interviewing for Women’s center
director, the candidate we were most
interested in was approached.
She started by asking for




$10K more money
Job for husband
Tenured full professor in academic department
Gold parking pass for terrace
59
What was her strategy?





Clearly Zeuthan
Advantages: she had something to concede
and we knew what she valued
Disadvantage: could be thought of as too
much so that the committee removes her from
the pool.
Have had students make “initial request” that
backfired as seemed totally off-base.
If you realize someone is using this strategy,
you might NOT be offended.
60
The Zeuthen Strategy
Q: If I concede, then how much should I concede?
A: Enough to change the balance of risk (who has more to
lose). (Otherwise, it will just be your turn to concede
again at the next round) Not so much that you give up
more than you needed to
Q: What if both have equal risk?
A: Both concede.
Obviously, if people’s utility was completely known, we
would just split the utility down the middle and be done.
The difficulty is that utilities are not always linear or
61
known to the other person.
About MCP and Zeuthen Strategies
Advantages:


Simple and reflects the way human negotiations work.

Stability – in Nash equilibrium – if one agent is using the strategy, then
the other can do no better than using it him/herself.
Disadvantages:


Computationally expensive – players need to compute the entire
negotiation set.

Communication burden – negotiation process may involve several
steps.
62
A one-shot Negotiation Protocol Protocol: both agents suggest an agreement; the offer giving a
higher product of utilities wins (flip a coin in case of a tie)
 Obvious strategy: amongst the set of agreements with maximal
product of utilities, propose the one that is best for you
 Properties: This mechanism is:



efficient - Pareto optimal
stable: no agent has an incentive to deviate from the strategy
simple: only one round is required
 Produces similar outcomes as MCP/Zeuthen, but it is a much
simpler mechanism.

63
One step Negotiation Protocol

Look at all choices in the negotiation set
Consider the product of the utilities
Pick the solution with the highest product of utilities

P() – Product of the two agent utilities from 

What is achieved?


a
b
5
0
2
3
1
4
0.5
7
0
9
64