Robust Price of
Anarchy Bounds via
Smoothness Arguments
Tim Roughgarden
Stanford University
1
Price of Anarchy
Definition: price of anarchy (POA) of a game
(w.r.t. some objective function, eq concept):
equilibrium objective fn value
optimal obj fn value
the closer to 1
the better
2
Price of Anarchy
Definition: price of anarchy (POA) of a game
(w.r.t. some objective function, eq concept):
the closer to 1
the better
equilibrium objective fn value
optimal obj fn value
2x
s
0
5
12
5x
cost = 14+14 = 28
2x
t
12
0
s
5
t
5x
cost = 14+10 = 24
3
The Need for Robustness
Meaning of a POA bound: if the game is at an
equilibrium, then outcome is near-optimal.
4
The Need for Robustness
Meaning of a POA bound: if the game is at an
equilibrium, then outcome is near-optimal.
Problem: what if can’t reach equilibrium?
• (pure) equilibrium might not exist
• might be hard to compute, even centrally
– [Fabrikant/Papadimitriou/Talwar], [Daskalakis/
Goldbeg/Papadimitriou], [Chen/Deng/Teng], etc.
• might be hard to learn in distributed way
Worry: are our POA bounds “meaningless”?
5
Fix #1: Provable Convergence
One Good Research Agenda: prove that
“natural learning dynamics” converge
(quickly) to equilibrium.
• [Hart/Mas-Collell], [Even-Dar/Kesselman/Mansour], [Fisher/
Racke/Voecking], [Chien/Sinclair], [Awerbuch et al], [EvenDar/Mansour/Nadav], [Kleinberg/Piliouras/Tardos], etc.
Problem: this “best-case scenario” has
limited reach
• non-existence/complexity issues
• lower bounds on convergence time of natural
dynamics (e.g., [Skopalik/Voecking STOC 08])
6
Fix #2: Robust POA Bounds
High-Level Goal: worst-case bounds that
apply even to non-equilibrium outcomes!
• best-response dynamics, pre-convergence
– [Mirrokni/Vetta 04], [Goemans/Mirrokni/Vetta 05],
[Awerbuch/Azar/Epstein/Mirrokni/Skopalik 08]
• correlated equilibria
– [Christodoulou/Koutsoupias 05]
• coarse correlated equilibria aka “price of
total anarchy” aka “no-regret players”
– [Blum/Even-Dar/Ligett 06],
[Blum/Hajiaghayi/Ligett/Roth 08]
7
Abstract Setup
• n players, each picks a strategy si
• player i incurs a cost Ci(s)
Important Assumption: objective function is
cost(s) := i Ci(s)
Key Definition: A game is (λ,μ)-smooth if, for
every pair s,s* outcomes (λ > 0; μ < 1):
i Ci(s*i,s-i) ≤ λ●cost(s*) + μ●cost(s)
[(*)]
8
Smooth => POA Bound
Next: “canonical” way to upper bound POA
(via a smoothness argument).
• notation: s = a Nash eq; s* = optimal
Assuming (λ,μ)-smooth:
cost(s) = i Ci(s)
[defn of cost]
≤ i Ci(s*i,s-i)
[s a Nash eq]
≤ λ●cost(s*) + μ●cost(s)
[(*)]
Then: POA (of pure Nash eq) ≤ λ/(1-μ).
9
Why Is Smoothness Stronger?
Key point: to derive POA bound, only needed
i Ci(s*i,s-i) ≤ λ●cost(s*) + μ●cost(s)
[(*)]
to hold in special case where s = a Nash eq
and s* = optimal.
Smoothness: requires (*) for every pair s,s*
outcomes.
– even if s is not a pure Nash equilibrium
10
Some Smoothness Bounds
• atomic (unweighted) selfish routing
[Awerbuch/Azar/Epstein 05], [Christodoulou/Koutsoupias 05],
[Aland/Dumrauf/Gairing/Monien/Schoppmann 06], [Roughgarden 09]
• nonatomic selfish routing
[Roughgarden/Tardos 00],[Perakis 04] [Correa/Schulz/Stier Moses 05]
• weighted congestion games
[Aland/Dumrauf/Gairing/Monien/Schoppmann 06],
[Bhawalkar/Gairing/Roughgarden 10]
• submodular maximization games
[Vetta 02], [Marden/Roughgarden 10]
• coordination mechanisms
[Cole/Gkatzelis/Mirrokni 10]
11
Application: Out-of-Equilibria
Definition: a sequence s1,s2,...,sT of outcomes
is no-regret if:
• for each player i, each fixed action qi:
– average cost player i incurs over sequence no
worse than playing action qi every time
– simple hedging strategies can be used by
players to enforce this (for suff large T)
Theorem: [Roughgarden STOC 09] in a (λ,μ)smooth game, average cost of every noregret sequence at most [λ/(1-μ)] x cost
of optimal outcome.
12
Smooth => POTA Bound
• notation: s1,s2,...,sT = no regret; s* = optimal
Assuming (λ,μ)-smooth:
t cost(st) = t i Ci(st)
[defn of cost]
13
Smooth => POTA Bound
• notation: s1,s2,...,sT = no regret; s* = optimal
Assuming (λ,μ)-smooth:
t cost(st) = t i Ci(st)
= t i [Ci(s*i,st-i) + ∆i,t]
[defn of cost]
[∆i,t:= Ci(st)- Ci(s*i,st-i)]
14
Smooth => POTA Bound
• notation: s1,s2,...,sT = no regret; s* = optimal
Assuming (λ,μ)-smooth:
t cost(st) = t i Ci(st)
= t i [Ci(s*i,st-i) + ∆i,t]
[defn of cost]
[∆i,t:= Ci(st)- Ci(s*i,st-i)]
≤ t [λ●cost(s*) + μ●cost(st)] + i t ∆i,t [(*)]
15
Smooth => POTA Bound
• notation: s1,s2,...,sT = no regret; s* = optimal
Assuming (λ,μ)-smooth:
t cost(st) = t i Ci(st)
= t i [Ci(s*i,st-i) + ∆i,t]
[defn of cost]
[∆i,t:= Ci(st)- Ci(s*i,st-i)]
≤ t [λ●cost(s*) + μ●cost(st)] + i t ∆i,t [(*)]
No regret: t ∆i,t ≤ 0 for each i.
To finish proof: divide through by T.
16
Why Important?
• bound on no-regret
sequences implies bound
on inefficiency of mixed
and correlated equilibria
• bound applies even to
sequences that don’t
converge in any sense
no regret
correlated eq
mixed Nash
pure
Nash
• no regret much weaker than reaching equilibrium
• e.g., if every player uses “multiplicative weights”
then get o(1) regret in poly-time
17
Tight Game Classes
Theorem: [Roughgarden STOC 09] for every set C,
unweighted congestion games with cost
functions restricted to C are tight:
maximum [pure POA] = minimum [λ/(1-μ)]
congestion games
w/cost functions in C
(λ ,μ): all such games
are (λ ,μ)-smooth
18
Tight Game Classes
Theorem: [Roughgarden STOC 09] for every set C,
unweighted congestion games with cost
functions restricted to C are tight:
maximum [pure POA] = minimum [λ/(1-μ)]
congestion games
w/cost functions in C
(λ ,μ): all such games
are (λ ,μ)-smooth
• weighted congestion games [Bhawalkar/
Gairing/Roughgarden ESA 10] and submodular
maximization games [Marden/Roughgarden CDC
10] are also tight in this sense
19
Further Applications
no regret
correlated eq
mixed Nash
approximate
Nash
pure
Nash
bestresponse
dynamics
Theorem: in a (λ,μ)-smooth game, everything in
these sets costs (essentially) λ/(1-μ) x OPT.
20
More Applications?
Theorem: [Nadav/Roughgarden 10] Consider a
(λ,μ)-smooth game for optimal choices of λ,μ.
Then precisely the “aggregate” coarse
correlated equilibria have cost ≤ λ/(1-μ) x OPT.
•essentially, bound holds if and only if the average
(rather than per-player) regret is non-positive
Proof: convex duality.
21
When Is a POA Bound a
Smoothness Proof?
Need to show: for every pair s,s* outcomes:
i Ci(s*i,s-i) ≤ λ●cost(s*) + μ●cost(s)
Generally sufficient: prove POA bound for pure
Nash equilibria such that:
• invoke best-response condition once/player
• hypothetical deviation by i independent of s-i
Non-example: network creation games
[Fabrikant et al], [Albers et al], [Demaine et al], etc.
22
Application: POA of NonTruthful Mechanisms
Mechanism Design: design protocol with
desirable outcome despite selfish participants
• example: Vickrey (second-price) auction
• focus thus far on "truthful" mechanisms
Non-truthful mechanisms: motivated by
• simplicity (e.g., sponsored search auctions)
• low communication complexity
• low computational complexity
23
Application: POA of NonTruthful Mechanisms
Fact: plausible outcomes of non-truthful
mechanisms = Bayes-Nash equilibria
• POA results for simple non-truthful auctions
in [Christodoulou/Kovacs/Schapira ICALP 08] and
[Borodin/Lucier SODA 10]
•
first bound POA of Nash equilibria, then "by
magic" same bound holds for Bayes-Nash
Fact: these are automatic consequences
of smoothness.
24
POA of Non-Truthful
Mechanisms (continued)
[Paes Leme/Tardos FOCS 10]
•
•
POA upper bounds for welfare of
Generalized Sponsored Search auction
different bounds for pure (1.618), mixed (4),
and Bayes-Nash eq (8) (without smoothness)
Open Questions:
• compute best smoothness bound
• prove a lower bound separating the bestpossible pure vs. Bayes-Nash POA
25
POA of Non-Truthful
Mechanisms (continued)
[Bhawalkar/Roughgarden 10]
•
POA upper bounds for welfare of subadditive
combinatorial auctions with “item bidding”
–
•
•
generalizes [Christodoulou/Kovacs/Schapira 08]
pure Nash: non-smooth upper bound of 2
Bayes-Nash: lower bound of 2.01, upper
bound of 2 ln m [m = # goods]
Open Questions:
• is Bayes-Nash POA = O(1)?
26
Local Smoothness and
Splittable Congestion Games
Local smoothness: require smoothness condition
only for outcomes that are “close”
–
•
•
•
•
assume strategy sets = convex subset of Rn
can only decrease optimal value of λ/(1-μ)
[Harks 08]: local smoothness gives improved
POA bounds for splittable congestion games
[Roughgarden/Schoppmann 10]: matching lower
bounds => first tight bounds in this model
[RS10]: local smoothness bounds extend to
correlated eq but not to no-regret outcomes!
27
Splittable Congestion Games:
Open Questions
Open Question #1: determine optimal POA
bounds for no-regret sequences.
Open Question #2: is (the distribution of)
every no-regret sequence a convex
combination of pure Nash equilibria?
Open Question #3: prove something non-trivial
about the POA of symmetric splittable
congestion games.
28