Expressiveness in mechanisms and its relation to efficiency:
Our experience from $40 billion of combinatorial multiattribute auctions, and recent theory
Tuomas Sandholm
Professor
Computer Science Department
Carnegie Mellon University
Founder, Chairman, Chief Scientist
CombineNet, Inc.
Outline
• Practical experiences with expressiveness
• Domain-independent measure of expressiveness
– Theory on how it relates to efficiency
• Application of the theory to sponsored search
• Expressive ad (e.g., banner) auction that spans
time
Sourcing before 2000
Pros
Manual
negotiation
Cons
Expressive => win-win
Unstructured, nontransparent
Implementable solution
Sequential => difficult,
suboptimal decisions
1-to-1 => lack of competition
Basic
reverse
auction
Structured, transparent
Simultaneous negotiation
with all suppliers
Global competition
Bidding on predetermined lots is
not expressive => ~ 0-sum game
Lotting effort
Small suppliers can’t compete
Unimplementable solution
Bidding complexity & exposure
Expressive commerce
• Expressive bidding
• Expressive allocation evaluation
Expressive bidding
• Package bids of different forms
• Conditional discount offers of different forms (general trigger conditions,
effects, combinations & sequencing)
• Discount schedules of different forms
• Side constraints, e.g. capacity constraints
• Multi-attribute bidding – alternates
• Detailed cost structures
• All of these are used in conjunction
– Don’t have to be used by all
Benefits of expressive bidding
Pareto improvement in allocation
1. Finer-grained matching of supply and demand (e.g. less empty driving)
2. Exposure problems removed => better allocation & lower cost
3. Capacity constraints => suppliers can bid on everything
4. No need to pre-bundle => better bundling & less effort
5. Fosters creativity and innovation by suppliers
6. Collaborative bids
=> lower prices and better supplier relationships
Academic bidding languages unusable
(in this application)
•
•
•
•
•
•
OR [S. 99]
XOR [S. 99]
Fully expressive
OR-of-XORs [S. 99]
XOR-of-ORs [Nisan 00]
OR* [Fujishima et al. 99, Nisan 00]
Recursive logical bidding languages
[Boutilier & Hoos 01]
Expressive allocation evaluation
• Side constraints
–
–
–
–
–
Counting constraints
Cost constraints
Unit constraints
Mixture constraints
…
• Expressions of how to evaluate bidder and bid attributes
Example of expressive allocation evaluation
Benefits of expressive allocation evaluation
1. Operational & legal constraints captured
=> implementable allocation
2. Can honor prior contractual obligations
3. Speed to contract: months  weeks
– $ savings begin to accrue earlier
– Effort savings
Clearing (aka. winner determination) problem
• Allocate (& define) the business
– so as to minimize cost (adjusted for buyer’s preferences)
– subject to satisfying all constraints
• Even simple subclass NP-complete & inapproximable [S.,
Suri, Gilpin & Levine AAMAS-02]
• We solve problems ~100x bigger than competitors, on all
dimensions:
•
•
•
•
> 2,600,000 bids
> 160,000 items (multiple units of each)
> 300,000 side constraints
> 1,000 suppliers
• Avg 20 sec, median 1 sec, some instances take days
• Speed & expressiveness: huge competitive advantage
CombineNet events so far
• > 500 procurement events
– $2 million - $1.6 billion
– The most expressive auctions ever conducted
• Total transaction volume > $40 billion
• Created 12.6% savings for customers
– Constrained; Unconstrained was 15.4%
• Suppliers also benefited
– Positive feedback (win-win, expression of efficiencies,
differentiation, creativity)
– Un-boycotting
– They recommend use of CombineNet to other buyers
Applied to many areas
Chemicals
Marketing
Transportation
Aromatics
Solvents
Cylinder Gasses
Colorants
Media buy
Corrugate Displays
Printed Materials Promotional
Items
Packaging
Technology
Cans & Ends
Corrugated Boxes
Corrugated Displays
Flexible Film
Folding Cartons
Labels
Plastic Caps/Closures
Shrink/Stretch Film
Security Cameras
Computers
Airfreight
Ocean Freight
Dray
Truckload
Less-than-truckload (LTL)
Bulk
Small Parcel
Intermodal
3PLs
Ingredients/Raw Mat.
Medical
Sugars/Sweeteners
Meat/Protein
Services
Pre-press
Temporary Labor
Shuttling/Towing
Warehousing
Pharmaceuticals
Medical/surgical supplies
Miscellaneous
Office Supplies
Industrial Parts/Materials
Bulk Electric
Fasteners
Filters
Leased Equipment
MRO
Pipes/Valves/Fittings/Gauges
Pumps
Safety Supplies
Steel
Broader trend toward expressiveness
Amazon.c & New Egg o offer bundles of items (ca. 2000)
Facebook increases expressiveness of privacy control (2006)
“…we did a bad job of explaining what
the new features were and an even
worse job of giving you control of
them…. This is the same reason we
have built extensive
privacy settings — to give
you even more control over
who you share your information with.”
CD+Tunes adds option for users to rent movies (2007)
Airlines charge extra for baggage, food & choice seats (2008)
Prediction/insurance markets becoming more expressive
Is more expressiveness always better?
• Not always for revenue!
Expressive mechanism: vi(
)? vi ( )? vi (
An inexpressive mechanism: vi (
)?
)?
Is more expressiveness always better
for efficiency?
• And what is expressiveness, really?
[Benisch, Sadeh & S. AAAI-08]
What makes a mechanism expressive?
A straw man notion
Item bid auction
$5 $2
Expression space 2
Combinatorial auction
$5 $2
$6
Expression space 3
What makes a mechanism expressive?
Prop: Dimensionality of expression space does not suffice
Proof intuition [based on work of Georg Cantor, 1890] :
a
Expression space
3
Mapping
b
Expression space 1
Work on informational complexity in mechanisms [Hurwicz, Mount, Reiter 1970s…]
puts technical restrictions that preclude such mappings
Our notion: Expressive mechanisms
allow agents more impact on outcome
An agent’s impact is a measure of the outcomes it can
choose between by altering only its own expression
$Y
$4
A C
B D
$6
$X $Y
$X
$4 $6
Uncertainty introduces the need for greater impact
$Y
A,A
A,C
C,C
A,B
A,D
C,D
B,B
B,D
D,D
$4
$6
$X
$X $Y
$4 $6
Uncertainty introduces the need for greater impact…
$Y
A,A
A,C
C,C
A,B
A,D
C,D
B,B
B,D
D,D
$X $Y
$7
$3
$X
$7 $3
Uncertainty introduces the need for greater impact…
$Y
A,A
A,C
C,C
A,B
A,D
C,D
B,B
B,D
D,D
$X $Y
$7
$4
$3
$6
$4 $6
$7 $3
$X
• 10 outcome pairs but only 9 regions
• In this example the impact vector B,C can’t be expressed
Expressive mechanisms
$Y
$Y
$Z
$Y
A,A
Some
Z>0
$Y
A,A
Z=0
A,C
Region
B,C
B,B
B,D
A,C
C,C
A,B
A,D
C,D
B,B
C,C
B,D
D,D
B,BD,D
$X
$X $Y
$Z
$X
$X
A,BExtra A,D
A,B
A,A
C,D
• In combinatorial
auction all 10
pairs can
be
B,D
D,D
expressed
$4 $6
$7 $3
$X
• Our measure of expressiveness for one agent (semi-shattering):
how many combinations of outcomes can he choose among
• Not just for combinatorial allocation problems because outcomes can be anything
• Captures multi-attribute considerations as well
An upper bound on a mechanism’s
best-case efficiency
• We study a mechanism’s
efficiency when agents cooperate
• It bounds the efficiency of any
equilibrium
• It allows us to avoid computing
equilibrium strategies
• It allows us to restrict our analysis
to pure strategies only
?
?
Theorem: the upper bound on efficiency for an optimal
mechanism increases strictly monotonically as more
expressiveness (# of expressible impact vectors) is allowed
(until full efficiency is reached)
Proof intuition: induction on the number of expressible
impact vectors; each time this is increased at least one
more efficient outcome is allowed
Theorem: the upper bound on efficiency for an optimal
mechanism can increase arbitrarily when any increase in
expressiveness (# of expressible impact vectors) is allowed
Proof intuition: construct preference distributions that
ensure at least one type makes each combination of
outcomes arbitrarily more efficient than any others
The bound can always be met
Theorem: for any outcome function, there exists at least
one payment function that yields a mechanism that
achieves the bound's efficiency in Bayes-Nash equilibrium
Proof intuition: if agents are charged their expected
imposed externality (i.e., the inconvenience that they cause
to other agents in the potentially inexpressive mechanism),
then making expressions that maximize social welfare is an
optimal strategy for each agent given that the others do so
as well
Application to sponsored search
[Benisch, Sadeh & S. Ad Auctions Workshop 2008]
Heterogeneous bidder preferences
Bidder utility
$0.25
$0.15
Prototypical
value advertiser
Prototypical
brand advertiser
$0.05
-$0.05 0%
-$0.15
20%
40%
60%
80%
100%
Rank
percentile
Mechanisms we compared
Google,
Yahoo!,
Microsoft, …
Inexpressive
Rank mechanism
1
2
$4
3
4
Our proposal
1
2
3
4
Premium
mechanism
$5
$4
Expressiveness
Fully expressive
mechanism
1
$5
2
$4
3
$3
4
$2
Best-case expected efficiency
100%
90%
80%
70%
60%
50%
Inexpressive
mechanism
Premium Fully expressive
mechanism
mechanism
Expressive ad (e.g., banner) auctions
that span time,
and
model-based online optimization for clearing
[Boutilier, Parkes, S. & Walsh AAAI-08]
Prior expressiveness
• Typical expressiveness in existing ad auctions
– Acceptable attributes
– Per-unit bidding (per-impression/per-clickthrough (CT))
– Budgets
– Single-period expressiveness (e.g., 1 day)
• Most prior research assumes this level of
expressiveness
Campaign-level expressiveness
• Advertising campaigns express preferences over a sequence of allocations
–
–
–
–
–
–
–
Minimum targets: pay only if 100K impressions in a week
Tiered preferences: $0.20 per impression up to 30K, $0.50 per impression for more
Temporal sequencing: at least 20K impressions per day for 14 days
Substitution: either NYT ($0.90) or CNN ($0.50) but not both
Smoothness: impressions vary by no more than 20% daily
Long-term budget: spend no more that $250k in a month
Exclusivity
• Additional forms of expressiveness
– Advertiser’s choice of impression/CT/conversion pricing (or combination)
– Target audience (e.g., demographics) rather than indirectly via web site properties
Value of optimization under sequential
expressiveness
• Bidder 1: bids $1 on A, $0.50 on B, budget $50k
• Bidder 2: bids $0.50 on A, budget $20k
• Traditional first-price auction: $52.3k revenue
Bidder 1: 45.45k
Supply of A
50k
t0
Supply of B
Bidder 2: 4.55k
0
t1
10k
Bidder 1: 9.09k
...
t2
10k
...
Value of optimization under sequential
expressiveness
• Bidder 1: bids $1 on A, $0.50 on B, budget $50k
• Bidder 2: bids $0.50 on A, budget $20k
• Optimal allocation: $70k revenue
Bidder 2: 40k
Bidder 1: 10k
Supply of A
50k
t0
Supply of B
0
t1
10k
t2
10k
Bidder 1: 80k
...
...
Stochastic optimization problem
•
•
•
•
•
•
Advertising channels C
Supply distribution of advertising channels PS
Set of campaigns B
Spot market demand distribution PD
Time horizon T
Can be modeled as Markov Decision Process (MDP)
– But how do we make it scale?
Scalable optimization with real-time
response
• Huge number of possible events => infeasible to compute full policy
contingent on all future states
• Cannot reoptimize policy in real time at every event
• Optimize-and-dispatch architecture [Parkes and S., 2005]
– Periodically compute policies with limited contingencies (e.g., stop dispatching
when budget reached)
– Dispatch in real time
• Policy form: xti,j - fraction of channel i allocated to campaign j at time t
• Optimize over coarse time periods (e.g., minutes, hours)
– Tradeoff between optimization speed and optimality
– Finer-grained in near-term, coarse-grained in long-term
Channels
• A channel is an aggregation of properties (web pages or spots on them)
• Constructed automatically based on campaigns
• Lossless aggregation: two web pages are in the same channel if
indistinguishable from the point of view of bids
• Example:
–
–
–
–
Bid 1: NY Times (NYT)
Bid 2: Medical article (Med)
Channels: (NYT ∧ Med), (NYT ∧ ¬Med), (¬ NYT ∧ Med)
Non-NYT pages grouped together, non-Med pages grouped together
• We can also perform lossy abstraction to avoid exponential blowup
Algorithms for stochastic problem
• Infeasible to solve the MDP
– Huge state space – cross product of individual
campaign states
– High-dimensional continuous action space
• Our approaches:
– Deterministic optimization
– Online stochastic optimization
Deterministic optimization
•
•
•
•
Replace uncertain channel supply with expectations
Formulate the problem as a mixed-integer program (MIP)
Solving a MIP is much faster than an MDP
Our winner determination algorithms can solve very large problems [S. 2007]
• Solutions may be far from optimal if supply distributions have high variance
– Does not adequately account for risk
– Can be mitigated by periodic reoptimization
Sample-based online stochastic optimization [van Hentenryck & Bent 06]
• Compute only next action, rather than entire policy
– Informed by what we might do in the future
– Recompute at each time period
• Sample-based
– Solve w.r.t. samples from distributions
• Extremely effective when good deterministic
algorithms exist
• Requires that domain uncertainty is exogenous
– Distribution of future events doesn’t depend on decisions
– Roughly true for advertising: allocation of ads should have
little effect on supply of channel
REGRETS algorithm [Bent & van Hentenryck 04]
λ1t
Time t
Action
Value
xt,1
f(xt,1)
xt,2
f(xt,2)
xt,3
.
.
.
xt,n
f(xt,n)
...
λ1T
Sample λ1
Optimal solution
x1t , x1t 1 ,, x1T
λ2t
λ2t+1 λ2t+2 λ2t+3 λ2t+4
...
λ2T
Sample λ2
Optimal solution
x 2t , x 2t 1 ,  , x 2T
f(xt,3)
.
.
.
λ1t+1 λ1t+2 λ1t+3 λ1t+4
Choose xt,i that
maximizes f(xt,i)
.
.
.
λKt
λKt+1 λKt+2 λKt+3 λKt+4
...
Sample λK
Optimal solution
x Kt , x Kt 1 ,  , x KT
λKT
REGRETS algorithm [Bent & van Hentenryck 04]
Lower bound on Q-values
for action xt at time t
λ1t
Q1t ( x t )
λ2t
λ1T
λ2t+1 λ2t+2 λ2t+3 λ2t+4
...
λ2T
Sample λ2
Optimal solution
x 2t , x 2t 1 ,  , x 2T
x t , x2t 1 ,, x2T
Q2t ( x t )
.
.
.
...
Sample λ1
Optimal solution
x1t , x1t 1 ,, x1T
x t , x1t 1 ,, x1T
f ( xt )  
λ1t+1 λ1t+2 λ1t+3 λ1t+4
.
.
.
λKt
λKt+1 λKt+2 λKt+3 λKt+4
...
x t , x Kt 1 , , x KT
Sample λK
Optimal solution
QKt ( x t )
x Kt , x Kt 1 ,  , x KT
λKT
REGRETS doesn’t apply to ad auctions
• Requires set of possible first-period decisions to be small
• Our dispatch policies are continuous
• Even a discretization of our continuous decision space
would be huge: dimensionality = |C||B||Discretization|
Our extension of REGRETS to continuous action spaces
Combining MIP
λ1t
x t , x1t 1 ,, x1T
Q1t ( x t )
1
max
xt
K
t
t
Q
(
x
 k )
k K
Q2t ( x t )
.
.
.
...
λ1T
Sample λ1
Optimal solution from MIP:
x1t , x1t 1 ,, x1T
λ2t
x t , x2t 1 ,, x2T
λ1t+1 λ1t+2 λ1t+3 λ1t+4
λ2t+1 λ2t+2 λ2t+3 λ2t+4
...
λ2T
Sample λ2
Optimal solution from MIP:
x 2t , x 2t 1 ,  , x 2T
.
.
.
λKt
λKt+1 λKt+2 λKt+3 λKt+4
...
λKT
x t , x Kt 1 , , x KT
Sample λK
Optimal solution from MIP:
QKt ( x t )
x Kt , x Kt 1 ,  , x KT
Revenue: Flat bids
Method
Unimodal supply
Bimodal supply
Bid-all
25,687 ± 436
14,004 ± 141
Myopic
30,256 ± 437
15,890 ± 175
Deterministic
42,365 ± 581
22,385 ± 227
Stochastic
42,237 ± 581
22,774 ± 238
Revenue: Bonus bids
Method
Unimodal supply
Bimodal supply
Deterministic
100,266 ± 3,555
55,901 ± 1,887
Stochastic
149,423 ± 3,204
65,065 ± 2,356
Conclusions & future research
•
•
Expressive mechanisms are practical & provide huge benefits
Needed to develop natural concise expressiveness forms
– Prior academic bidding languages not usable
•
For efficiency, can add any expressiveness forms
– Will help & can help an arbitrary amount; Bound can be met in BNE
•
Uncertainty about others => need more expressiveness
– Unlike in work solely on dominant-strategy mechanisms [Ronen 01], [Holzman et al. 04],
[Blumrosen & Feldman 06]
•
Sponsored search
– GSP seems to run at a large inefficiency
– Most of it fixable by our “premium mechanism”
•
Expressive (banner) ad auctions that span time
–
–
–
–
–
–
•
Optimize-and-dispatch framework
Channel aggregation
Deterministic optimization with re-optimization
Online sample-based optimization – extended to continuous action space
Optimization provides significant benefits, even with no added expressiveness
Stochastic especially beneficial with non-linear preferences
Most helpful expressiveness forms for other apps?
– Agents’ preferences as input, our methodology can be used to evaluate different mechanisms
– Psychological burden: expressing more vs. expressing strategically (e.g., chopsticks)