Preference Elicitation in
Combinatorial Auctions:
An Overview
Tuomas Sandholm
[For an overview, see review article by
Sandholm & Boutilier in the textbook
Combinatorial Auctions, MIT Press 2006,
posted on course home page]
Setting
Combinatorial auction: m items for sale
• Private values auction, no allocative externalities
– So, each bidder i has value function, vi: 2m  R
• Free disposal
• Unique valuations (to ease presentation)
Another complex problem in
combinatorial auctions:
“Revelation problem”
• In direct-revelation mechanisms (e.g. VCG), bidders bid
on all 2#items combinations
– Need to compute the valuation for exponentially many
combinations
• Each valuation computation can be NP-complete local planning problem
• For example if a carrier company bids on trucking tasks: TRACONET
[Sandholm AAAI-93, …]
– Need to communicate the bids
– Need to reveal the bids
• Loss of privacy & strategic info
Revelation problem …
• Agents need to decide what to bid on
– Waste effort on counter-speculation
– Waste effort making losing bids
– Fail to make bids that would have won
• Reduces economic efficiency & revenue
? for
$ 1,000 for
$ 1,500 for
What info is needed from
Elicitor
an agent depends on what
Clearing algorithm
others have revealed
Elicitor decides what to ask
next based on answers it
has received so far
Conen & Sandholm IJCAI-01 workshop on Econ. Agents, Models & Mechanisms, ACMEC-01
Elicitor skeleton
• Repeat:
– Decide what to ask (and from which bidder)
– Ask that and propagate the answer in data structures
– Check whether you know the optimal allocation of
items to agents. If so, stop
Conen & Sandholm IJCAI workshop-01, ACMEC-01
Incentive to answer elicitor’s queries
truthfully
• Elicitor’s queries leak information across agents
• Thrm. Nevertheless, answering truthfully can be made
an ex post equilibrium [Conen&Sandholm ACMEC-01]
– Elicit enough to determine optimal allocation overall, and for
each agent removed in turn
– Use VCG pricing
• Push-pull mechanism
• If a bidder can endogenously decide which bundles for
which bidders to evaluate, then no nontrivial mechanism
– even a direct revelation mechanisms - can 1) be truthpromoting, and 2) avoid motivating an agent to compute
on someone else’s valuation(s) [Larson&Sandholm AAMAS-05]
First generation of elicitors
• Rank lattice based elicitors
[Conen & Sandholm IJCAI-01 workshop, ACMEC-01, AAAI-02, AMEC-02]
Rank Lattice
Rank of Bundle Ø A B AB
for Agent 1
4 2 3 1
for Agent 2
4 3 2 1
[1,1]
[1,2]
[1,3]
[1,4]
[2,1]
[2,2]
[2,3]
[2,4]
[3,1]
[3,2]
[3,3]
[3,4]
[4,1]
[4,2]
[4,3]
[4,4]
Infeasible
Feasible
Dominated
A search algorithm for the rank lattice
Algorithm PAR “PAReto optimal“
OPEN  [(1,...,1)]
while OPEN  [] do
Remove(c,OPEN); SUC  suc(c);
if Feasible(c) then
PAR  PAR  {c}; Remove(SUC,OPEN)
else foreach node  SUC do
if node  OPEN and Undominated(node,PAR)
then Append(node,OPEN)
• Thrm. Finds all feasible Pareto-undominated allocations (if
bidders’ utility functions are injective, i.e., no ties)
• Welfare maximizing solution(s) can be selected as a postprocessor by evaluating those allocations
– Call this hybrid algorithm MPAR (for “maximizing” PAR)
Value-Augmented Rank Lattice
Value of Bundle Ø A B AB
for Agent 1
0 4 3 8
for Agent 2
0 1 6 9
9
[1,3]
8
[1,4]
17
14
[1,2]
[1,1]
10
13
[2,1]
[2,2]
[2,3]
[2,4]
12
[3,1]
[3,2]
[3,3]
[3,4]
[4,4]
[4,1]
[4,2]
[4,3]
9
Search algorithm family for the valueaugmented rank lattice
Algorithm EBF “Efficient Best First“
OPEN  {(1,...,1)}
loop
if |OPEN| = 1 then c  combination in OPEN
else
M  {k  OPEN | v(k) = maxnode  OPEN v(node) }
if |M|  1  node  M with Feasible(node) then return node
else choose c  M such that c is not dominated by any node  M
OPEN  OPEN \ {c}
if Feasible(c) then return c
else foreach node  suc(c) do
if node  OPEN then OPEN  OPEN  {node}
• Thrm. Any EBF algorithm finds a welfare maximizing allocation
• Thrm. VCG payments can be determined from the information
already elicited
Best & worst case elicitation effort
• Best case: rank vector (1,...,1) is feasible
– One bundle query to each agent, no value queries
– VCG payments are all 0
• Thrm. Any EBF algorithm requires at worst (2#items #bidders
– #bidders#items)/2 + 1 value queries
– Proof idea. Upper part of the lattice is infeasible and not less in
value than the solution
• Not surprising because in the worst case, finding a
provably (even approximately) optimal allocation requires
exponentially many bits to be communicated no matter
what query types are used and what query policy is used
[Nisan&Segal J. Economic Theory 2006]
– We will prove this later
EBF minimizes feasibility checks
• Def: An algorithm is admissible if it always finds a
welfare maximizing allocation
• Def: An algorithm is admissibly equipped if it only has
– value queries, and
– a feasibility function on rank vectors, and
– a successor function on rank vectors
• Thrm: There is no admissible, admissibly equipped
algorithm that requires fewer feasibility checks (for every
problem instance) than an (arbitrary) EBF algorithm
MPAR minimizes value queries
• Thrm. No admissible, admissibly equipped
algorithm (that calls the valuation function for
bundles in feasible rank vectors only) will require
fewer value queries than MPAR
• MPAR requires at most #bidders#items value
queries
Differential-revelation
• Extension of EBF
• Information elicited: differences between valuations
– Hides sensitive value information
• Motivation: max ∑ vi(Xi)  min ∑ [vi(r-1(1)) – vi(Xi)]
– Maximizing sum of value  Minimizing difference between
value of best ranked bundle and bundle in the allocation
• Thrm. Differences suffice for determining welfare
maximizing allocations & VCG payments
• 2 low-revelation incremental ex post incentive compatible
mechanisms ...
Differential elicitation ...
• Questions (start at rank 1)
– “tell me the bundle at the current rank”
– “tell me the difference in value of that bundle and
the best bundle“
• increment rank
• Natural sequence: from “good” to “bad” bundles
Policy-independent elicitor
algorithms
What query should the elicitor
ask next ?
• Simplest answer: value query
– Ask for the value of a bundle vi(b)
• How to pick b, i?
Random elicitation
• Asks randomly chosen value queries whose
answer cannot yet be inferred
• Thrm. If the full-revelation mechanism makes Q
value queries and the best value-elicitation policy
makes q queries, random elicitation makes on
average
value queries
– Proof idea: We have q red balls, and the remaining balls
are blue; how many balls do we draw before removing
all q red balls?
Hudson & Sandholm AMEC-02, AAMAS-04
Random elicitation
• Not much better than theoretical bound
queries
queries
4 items
2 agents
80
1000
60
100
40
10
1
20
2 3
4
5
6 7 8 9 10
items
2
3
4
agents
5
6
Querying random allocatable
bundle-agent pairs only…
• Bundle-agent pair (b,i) is allocatable if some yet
potentially optimal allocation allocates bundle b to agent i
• How to pick (b,i)?
– Pick a random allocatable one
• Asking only allocatable bundles means throwing out some
queries
• Thrm. This restriction causes the policy to make at worst
twice as many expected queries as the unrestricted random
elicitor. (Tight)
– Proof idea: These ignored queries are either
• Not useful to ask, or
• Useful, but we would have had low probability of asking it, so no big
difference in expectation
Querying random allocatable
bundle-agent pairs only…
• Much better
– Almost (#items / 2) fewer queries than unrestricted random
– Vanishingly small fraction of all queries asked !
– Subexponential number of queries
queries
queries
80
1000
60
100
40
10
1
20
2 3
4
5
6 7 8 9 10
items
2
3
4
agents
5
6
Best value query elicitation policy so far
Focus on allocations that have highest upper bound.
Ask a (b,i) that is part of such an allocation and among
them, pick the one that affects (via free disposal) the largest number
of bundles in such allocations.
Omniscient elicitor
Fraction of
values queried
before provably
optimal
allocation found
1
2 agents
0.8
3 agents
4 agents
0.6
0.4
0.2
0
1
2
3
4
5
6
7
8
9
10
Number of items for sale
Optimal elicitor implementable, but utterly intractable.
Hudson & Sandholm AMEC-02, AAMAS-04
Worst-case number of bits transmitted
(nondeterministic model)
• Exponential (even to approximately optimally allocate the
items within ratio better than 1/2) [Nisan & Segal JET-06; see also
CS-friendly version from Nisan’s home page]
L is the number of items
Proof.
Restricted preferences
Even worst-case number of queries is polynomial
when agents’ valuation functions fall within
certain natural classes…
Read-once valuations
Returns sum of c highest-valued inputs
if at least k inputs are positive,
0 otherwise
PLUS
MAX
GATEk,c
ALL
ALL
1000
500
400
200
100
150
• Thrm. If an agent has a read-once valuation function, the number
of value queries needed to elicit the function is polynomial in items
• Thrm. If an agent’s valuation function is approximable by a readonce function (with only MAX and PLUS nodes), elicitor finds an
approximation in a polynomial number of value queries
Zinkevich, Blum & Sandholm ACMEC-03
Toolbox valuations
• Items are viewed as tools
• Agent can accomplish multiple goals
– Each goal has a value & requires some subset of tools
– Agent’s valuation for a package of items is the sum of
the values of the goals that those tools allow the agent to
accomplish
• E.g. items = medical patents, goals = medicines
• Thrm. If an agent has a toolbox valuation function,
it can be elicited in O(#items #goals) value queries
Zinkevich, Blum & Sandholm ACMEC-03
Computational complexity of finding
an optimal allocation after elicitation
• Thrm. Given one agent with an additive valuation fn
and one agent with a read-once valuation fn, allocation
requires only polynomial computation
• Thrm. With 2 agents with read-once valuations (even
with just MAX, SUM, and ALL gates), it is NP-hard to
find an allocation that is better than ½ optimal
• Thrm. Given 2 agents with toolbox valuations having
s1 and s2 terms respectively, optimal allocation can be
done in computation time poly(m, s1+s2)
Zinkevich, Blum & Sandholm ACMEC-03
2-wise dependent valuations
3
Node = item
0
3
m items
1
-2
1
0+1+2 = 3
2
• Prop. If an agent has a 2-wise dependent valuation function,
elicitor finds it in m(m+1)/2 queries
• Thrm. If an agent’s valuation function is approximately 2-wise
dependent, elicitor finds an approximation in m(m+1)/2 queries
– Thrm. Every super-additive valuation function is approximately 2-wise
dependent
• Thrm. These results generalize to k-wise dependent valuations
using O(mk) queries
Conitzer, Sandholm & Santi Draft-03, AAAI-05
Gk = k-wise dependent valuations
• G 1  G2  …  Gm
• G1 = linear valuations: Easy to elicit & allocate
• Gk where k ≥ 2 is a constant: Easy to elicit, NPhard to allocate
– if graph cycle free (i.e. forest), allocation polytime
• Gg(m) where g(m) is an arbitrary (sublinear) fn
s.t. g(m) approaches infinity as m approaches
infinity: Hard to elicit & NP-hard to allocate
• Gm contains all valuation fns
Conitzer, Sandholm & Santi Draft-03
Combining polynomially elicitable classes
• Thrm. If class C1 (resp. C2) is elicitable using
p1(m) (resp. p2(m)) queries, then C1 union C2 is
elicitable in p1(m) + p2(m) + 1 queries. Tight
Santi, Conitzer, Sandholm COLT-04
In some settings, learning only a tiny part of
valuation fns suffices to allocate optimally
Blum, Jackson, Sandholm & Zinkevich JMLR-04
• Consider 2 agents with valuations f and g
– Each has some subsets of items that he likes
– Each such subset is of size log m
– Agent’s valuation is 1 if he gets a set of items that he likes, 0 otherwise
• Since there are
bundles of size log m, some members of this class
cannot be represented in poly(m) bits => can require super-polynomial
number of queries to learn an agent’s valuation fn
• But… Thrm. Optimal allocation can be determined in poly(m) queries
– Proof: Try random partitions of items into two equal-sized sets
– Derandomization: A set of assignments to m boolean variables is (m,k)universal if for every subset of k variables, the induced assignments to those
variables cover all 2k settings. Naor and Naor (1990) give efficient
constructions of such sets using only 2O(k) log m assignments. We can use k =
O(log m), so the construction is polynomial time and space. Each of these
assignments corresponds to a partition of items, and we ask f and g for their
valuations on each one and take the best.
In some settings, learning only a tiny part of
valuation fns suffices to allocate optimally…
• There can be super-polynomial power even when valuation
fns have short descriptions
• Let each agent have some distinguished bundle S’
• Agent’s valuation is
1 for all bundles of size ≥ |S’|, except for S’ itself
0 otherwise
• Prop. It can take
value queries to learn such a
valuation fn
• Thrm. With two agents with such valuation fns, the optimal
allocation can be determined in 4 + log2 m value queries
– Proof. First find |S’| in log2 m + 1 queries using binary search. Then
make 3 arbitrary queries of size |S’|. At most 1 of them can return 0.
Call the other two sets T and T’. We then query the other agent for MT; if it returns 1, then T, M-T is an optimal allocation. Otherwise, T’,
M-T’ is optimal.
Blum, Jackson, Sandholm & Zinkevich JMLR-04
Power of interleaving queries among
agents
• Observation: In general (not just in
combinatorial auctions), we can elicit
without interleaving within a number of
queries that is exponential in q
– where q is the number of queries used when
eliciting with interleaving.
• Proof: Contingency plan tree is (merely)
exponential in the number of queries
Other results on elicitation
• Interleaving value & order queries [Hudson & Sandholm
AMEC-02, AAMAS-04]
• Bound-approximation queries [Hudson & Sandholm
AMEC-02, AAMAS-04]
• Elicitation in exchanges (for multi-robot task allocation)
[Smith, Sandholm & Simmons AAAI-02 workshop]
• Eliciting bid-taker’s non-price preferences in
(combinatorial) reverse auctions [Boutilier, Sandholm,
Shields AAAI-04]
Demand queries
“If the prices (on items or some bundles) were p,
which bundle would you buy?”
Value queries vs. demand queries
• A value query can be simulated by a polynomial
number of demand queries [Blumrosen&Nisan EC-05, see also
their SIAM J. Computing 2010 paper]
– Proof. Elicit value of adding one item at a time into the bundle. The marginal
value of each such addition is done via binary search on that item’s price.
• A demand query cannot be simulated in a polynomial
number of value queries [Blumrosen&Nisan EC-05]
• There exists restricted CAs where optimal allocation
can be found in poly bits, but exponential number of
demand (and thus value) queries are needed [Nisan &
Segal TARK-05]
Ascending combinatorial auctions
• Demand queries
– Per-item prices vs. bundle prices
– Discriminatory vs. nondiscriminatory prices
• Exponential communication complexity, but
polynomial in special classes (e.g., when items are
substitutes) [Nisan-Segal JET-06]
– To allocate optimally, enough info has to be elicited to
determine the minimal competitive equilibrium prices
[Parkes; Nisan-Segal JET-06]
• Could also use descending prices
Ascending combinatorial auctions…
•
Thm [Blumrosen & Nisan JET-10]. To achieve efficiency, the number of trajectories in
an ascending item-price CA may have to be exponential in the number of items
–
•
even if the trajectories can be interleaved and what is done on a trajectory can depend on what happened on
other trajectories
Thm [Blumrosen & Nisan JET-10]. Any anonymous ascending (even bundle-price) CA
may fail to find an efficient allocation
XOR-bidding language
[Sandholm ICE-98, IJCAI-99]
• ({umbrella}, $4) XOR
({raincoat}, $5) XOR
({umbrella,raincoat}, $7) XOR …
• Bidder’s valuation is the highest-priced
term, of the terms whose bundle the bidder
receives
Power of bundle prices
• Thrm. [Lahaie & Parkes ACMEC-04] Using bundleprice demand queries (even when only poly(m)
bundles are priced) and value queries, an XORvaluation can be learned in O(m2 #terms) queries
• Thrm. [Blum, Jackson, Sandholm, Zinkevich COLT-03,
JMLR-04] If the elicitor can use value queries and
item-price demand queries only, then 2Ω(√m)
queries are needed in the worst case
– even if each agent’s XOR-valuation has only O(√m)
terms
Conclusions on preference elicitation
in combinatorial auctions
• Reduces the number of local plans needed
• Capitalizes on multi-agent elicitation
• Truth-promoting push-pull mechanism
Future research on preference elicitation
•
•
•
•
Scalable general elicitors (in queries, CPU, RAM)
New polynomially elicitable valuation classes
More powerful queries, e.g. side constraints
Using models of how costly it is to answer
different queries [Hudson & Sandholm AMEC-02, AAMAS-04]
– Strategic deliberation [Larson & Sandholm]
• Other applications (e.g. voting [Conitzer & Sandholm AAAI-02,
EC-04])
Tradeoffs between
1.
2.
3.
4.
5.
Agent’s evaluation complexity
Amount revealed to the auctioneer (crypto)
Amount revealed to other agents (vs. to elicitor)
Bits communicated
Elicitor’s computational complexity (knowing when
to terminate, what to ask next)
6. Elicitor’s memory usage (e.g., implicit candidate list)
7. Designer’s objective
•
•
Designing for specific prior & eliciting using the prior
Terminating before optimal allocation, …