A Primal-Dual Approach to
Online Optimization Problems
Online Optimization Problems
• input arrives “piece by piece” (“piece” is called request)
• upon arrival of a request - has to be served immediately
• past decisions cannot be revoked
how to evaluate the performance of an online algorithm?
– if, for each request sequence,
cost(online) ≤ r x cost(optimal offline)
– then online algorithm is r-competitive
Road Map
Introducing the framework:
• Ski rental
• Online set cover
• Virtual circuit routing
The general framework:
• {0,1} covering/packing linear programs
• General covering/packing linear programs
Recent results:
• The ad-auctions problem
• Weighted caching
The Ski Rental Problem
• Buying costs $B.
• Renting costs $1 per day.
Problem:
• Number of ski days is not known in advance.
Goal: Minimize the total cost.
Ski Rental – Integer Program
1 - Buy
zi
x
0 - Don't Buy
1 - Rent on day i

0 - Don't rent on day i
k
min Bx   zi
i 1
Subject to:
For each day i:
x  zi  1
x, zi {0,1}
Ski Rental – Relaxation
P: Primal Covering
k
min Bx   zi
i 1
For each day i: x  zi  1
x, zi  0
D: Dual Packing
k
max  yi
i 1
For each day i: yi  1
k
y
i 1
i
B
Online setting:
• Primal: New constraints arrive one by one.
• Requirement: Upon arrival, constraints should
be satisfied.
• Monotonicity: Variables can only be increased.
Ski Rental – Algorithm
P: Primal Covering
D: Dual Packing
k
k
min Bx   zi
i 1
For each day i: x  zi  1
max  yi
i 1
For each day i: yi  1
x, zi  0
Initially x 0
Each new day (new constraint):
if x<1:
 zi  1-x
 x  x(1+ 1/B) + 1/(c*B)
 yi  1
k
y
i 1
- ‘c’ later.
i
B
Analysis of Online Algorithm
Proof of competitive factor:
1. Primal solution is feasible.
2. In each iteration, ΔP ≤ (1+ 1/c)ΔD.
3. Dual is feasible.
Conclusion: Algorithm is (1+ 1/c)-competitive
Initially x 0
Each new day (new constraint):
if x<1:
 zi  1-x
 x  x(1+ 1/B) + 1/(c*B)
 yi  1
- ‘c’ later.
Analysis of Online Algorithm
1. Primal solution is feasible.
If x ≥1 the solution is feasible.
Otherwise set: zi  1-x.
2. In each iteration, ΔP ≤ (1+ 1/c)ΔD:
Algorithm:
If x≥1, ΔP =ΔD=0
When new constraint
Otherwise:
arrives, if x<1:
zi1-x
• Change in dual: 1
x x(1+ 1/B) + 1/c*B
• Change in primal:
y 1
BΔx + zi = x+ 1/c+ 1-x = 1+1/c
i
Analysis of Online Algorithm
3.
Dual is feasible:
k
Need to prove:  yi  B
i 1
We prove that after B days x≥1
x is a sum of geometric sequence
a1 = 1/(cB), q = 1+1/B
B
B
 1
 1
1


1
1   1



1  B
B
x


cB  1 
c
1


1


B


Algorithm:
When new constraint
arrives, if x<1:
zi1-x
x x(1+ 1/B) + 1/c*B
yi1
B
 1
c  1    1  e  1
 B
1
e
1 
c e 1
Randomized Algorithm
0
1
X:
X1 X2
X3
X4
•
Choose d uniformly in [0,1]
•
Buy on the day corresponding to the “bin” d falls in
•
Rent up to that day
Analysis:
•
Probability of buying on the i-th day is xi
•
Probability of renting on the i-th day is at most zi
Key Idea for Primal-Dual
Primal: Min i ci xi
Dual: Max t bt yt
Step t, new constraint:
a1x1 + a2x2 + … + ajxj ≥ bt
New variable yt
+ bt yt in dual objective
xi  (1+ ai/ci) xi
yt  yt + 1
(mult. update)
(additive update)
 primal cost =
=  Dual Cost
The Online Set-Cover Problem
• Elements: e1, e2, …, en
• Set system: s1, s2, … sm
• Costs: c(s1), c(s2), … c(sm)
Online Setting:
• Elements arrive one by one.
• Upon arrival elements need to be covered.
• Sets that are chosen cannot be “unchosen”.
Goal: Minimize the cost of the chosen sets.
Set Cover – Linear Program
P: Primal Covering
D: Dual Packing
min  c( s) x( s)
max  y(e)
eE
sS
e  E
 x( s )  1
s es
s  S
 y(e)  c(s)
es
Online setting:
• Primal: constraints arrive one by one.
• Requirement: each constraint is satisfied.
• Monotonicity: variables can only be increased.
Set Cover – Algorithm
P: Primal Covering
D: Dual Packing
min  c( s) x( s)
max  y(e)
eE
sS
e  E
 x( s )  1
s es
s  S
 y(e)  c(s)
es
Initially x(s) 0
When new element arrives, while  x(s)  1:
• y(e)  y(e)+1
s es
• .s e  s x(s)  x( s) 1  1/ c( s)  1/ m  c( s)

 

Analysis of Online Algorithm
Proof of competitive factor:
1. Primal solution is feasible.
2. In each iteration, ΔP ≤ 2ΔD.
3. Dual is (almost) feasible.
Conclusion: We will see later.
Initially x(S) 0
When new element e arrives, while
 x(s)  1:
s es
• y(e)  y(e)+1
• . s e  s x(s)  x( s) 1  1/ c( s)   1/  m  c( s) 
Analysis of Online Algorithm
1. Primal solution is feasible.
We increase the primal variables until the
constraint is feasible.
Initially x(S) 0
When new element e arrives, while
 x(s)  1:
s es
• y(e)  y(e)+1
• . s e  s x(s)  x( s) 1  1/ c( s)   1/  m  c( s) 
Analysis of Online Algorithm
2. In each iteration, ΔP ≤ 2ΔD.
In each iteration:
• ΔD = 1
 x( s )
1 
P =  c(s)  x( s)   c(s)  


c
(
s
)
m

c
(
s
)
s|es
s|es


  x(s)   1/m  2
s|es
s|es
Initially x(S) 0
When new element e arrives, while
 x(s)  1:
s es
• y(e)  y(e)+1
• . s e  s x(s)  x( s) 1  1/ c( s)   1/  m  c( s) 
Analysis of Online Algorithm
3.
Dual is (almost) feasible:
•
We prove that: s  S ,
 y(e)  c(s)O(log m)
eS
•
If y(e) increases, then x(s) increases (for e in S).
•
x(s) is a sum of a geometric series:
a1 = 1/[mc(s)], q = (1+ 1/c(s))
Initially x(S) 0
When new element e arrives, while
 x(s)  1:
s es
• y(e)  y(e)+1
• . s e  s x(s)  x( s) 1  1/ c( s)   1/  m  c( s) 
Analysis of Online Algorithm
 After c(s)O(log m) rounds:


1
1  1/ c(s)
1
x( s ) 
m  c( s )
1  1/ c(s) 1
c ( s ) O (log m )
1  1/ c( s)


c ( s ) O (log m )
m
 1
1
We never increase a variable x(s)>1!
Initially x(S) 0
When new element e arrives, while
 x(s)  1:
s es
• y(e)  y(e)+1
• . s e  s x(s)  x( s) 1  1/ c( s)   1/  m  c( s) 
Conclusion
•
The dual is feasible with cost 1/O(log m) of the primal.
 The algorithm produces a fractional set cover that is
O(log m)-competitive.
•
Remark: No online algorithm can perform better in
general.
What about an integral solution?
•
Round fractional solution. (With O(log n) amplification.)
•
Can be done deterministically online [AAABN03].
•
Competitive ratio is O(log m log n).
Online Virtual Circuit Routing
Network graph G=(V, E)
capacity function u: E Z+
Requests: ri = (si, ti)
• Problem: Connect si to ti by a path, or reject the
request.
• Reserve one unit of bandwidth along the path.
• No re-routing is allowed.
• Load: ratio between reserved edge bandwidth and
edge capacity.
• Goal: Maximize the total throughput.
Routing – Linear Program
y (ri , p) = Amount of bandwidth allocated for ri on path p
P(ri ) - Available paths to serve request ri
max 
ri

pP ( ri )
y(ri , p)
s.t:
For each ri:
For each edge e:

pP ( ri )
y(ri , p)  1
 
ri
pP ( ri ) e p
y(ri , p)  u (e)
Routing – Linear Program
P: Primal Covering
D: Dual Packing
max 
min  u (e) x(e)   z (ri )
eE
ri
ri
ri , p  P(ri ) :
 x(e)  z(r )  1
ep
i
ri
e :

pP ( ri )

pP ( ri )
y (ri , p)  1
 
ri
y(ri , p)
pP ( ri ) e p
y(ri , p)  u(e)
Online setting:
• Dual: new columns arrive one by one.
• Requirement: each dual constraint is satisfied.
• Monotonicity: variables can only be increased.
Routing – Algorithm
P: Primal Covering
D: Dual Packing
max 
min  u (e) x(e)   z (ri )
eE
ri
ri
ri , p  P(ri ) :
 x(e)  z(r )  1
ep
i
ri
e :

pP ( ri )

pP ( ri )
y (ri , p)  1
 
ri
y(ri , p)
pP ( ri ) e p
y(ri , p)  u(e)
Initially x(e) 0
When new request arrives, if p  P(ri ),  x(e)  1:
ep
 z(ri)  1

1 
1

. e  p : x ( e)  x (e)  1 

 u (e)  n  u (e)
 y(ri,p)  1
Analysis of Online Algorithm
Proof of competitive factor:
1. Primal solution is feasible.
2. In each iteration, ΔP ≤ 3ΔD.
3. Dual is (almost) feasible.
Conclusion: We will see later.
Initially x(e) 0
When new request arrives, if p  P (ri ), x(e)  1:
ep
 z(ri)  1

1 
1
 .e  p : x(e)  x(e) 1 

 u (e)  n  u (e)
 y(ri,p)  1

Analysis of Online Algorithm
1. Primal solution is feasible.
If p  P(ri ),  x(e)  1: the solution is feasible.
ep
Otherwise: we update z(ri)  1
Initially x(e) 0
When new request arrives, if p  P (ri ),
 x(e)  1:
ep
 z(ri)  1

1 
1

. e  p : x ( e)  x (e)  1 

 u (e)  n  u (e)
 y(ri,p)  1
Analysis of Online Algorithm
In each iteration: ΔP ≤ 3ΔD.
If p  P(ri ) :  x(e)  1 ΔP = ΔD=0
ep
Otherwise:
ΔD=1
2.
P   u (e)x(e)  z (ri )
e p
 x ( e)
1 
  u ( e) 

 1  3
e p
 u ( e) n  u (e ) 
Initially x(e) 0
When new request arrives, if p  P (ri ),
 x(e)  1:
ep
 z(ri)  1

1 
1

. e  p : x ( e)  x (e)  1 

 u (e)  n  u (e)
 y(ri,p)  1
Analysis of Online Algorithm
3. Dual is (almost) feasible.
We prove:
• For each e, after routing u(e)O(log n) on e, x(e)≥1
x(e) is a sum of a geometric sequence
x(e)1 = 1/(nu(e)), q = 1+1/u(e)
 After u(e)O(log n) requests:
u ( e ) O (log n )
u ( e ) O (log n )


1 
1 
 1 1 
 1  u (e) 

u
(
e
)
1


x ( e) 


n  u (e )
n

1 
 1  u ( e)   1


1
1
New Results via P-D Approach: Routing
Previous results (routing/packing):
• [AAP93] – Route O(log n) fraction of the optimal
without violating capacity constraints.
Capacities must be at least logarithmic.
• [AAFPW94] – Route all the requests with load of at
most O(log n) times the optimal load.
Observation [BN06] – Both results can be described
within the primal-dual approach.
New Results via P-D Approach: Routing
We saw a simple algorithm which is:
• 3-competitive and violates capacities by O(log n) factor.
Can be improved [Buchbinder, Naor, FOCS06] to:
• 1-competitive and violates capacities by O(log n) factor.
Non Trivial.
Main ideas:
• Combination of ideas drawn from casting of previous
routing algorithms within the primal-dual approach.
• Decomposition of the graph.
• Maintaining several primal solutions which are used to
bound the dual solution, and for the routing decisions.
New Results via P-D Approach: Routing
Applications [Buchbinder, N, FOCS 06]:
• Can be used as “black box” for many objective
functions and in many routing models:
– Previous Settings [AAP93,APPFW94].
– Maximizing throughput.
– Minimizing load.
– Achieving better global fairness results
(Coordinate competitiveness).
Road Map
Introducing the framework:
• Ski rental
• Online set cover
• Virtual circuit routing
The general framework:
• {0,1} covering/packing linear programs
• General covering/packing linear programs
Recent results:
• The ad-auctions problem
• Weighted caching
Online Primal-Dual Approach
• Can the offline problem be cast as a linear
covering/packing program?
• Can the online process be described as:
– New rows appearing in a covering LP?
– New columns appearing in a packing LP?
Yes ??
• Upon arrival of a new request:
– Update primal variables in a multiplicative way.
– Update dual variables in an additive way.
Online Primal Dual Approach
Next Prove:
1. Primal solution is feasible (or nearly feasible).
2. In each round, ΔP ≤ c ΔD.
3. Dual is feasible (or nearly feasible).
Got a fractional solution, but need an integral
solution ??
•
•
Randomized rounding techniques might work.
Sometimes, even derandomization (e.g., method of
conditional probabilities) can be applied online!
Online Primal-Dual Approach
Advantages:
1. Generic ideas and algorithms applicable to many
online problems.
2. Linear Program helps detecting the difficulties of the
online problem.
3. General recipe for the design and analysis of online
algorithms.
4. No potential function appearing “out of nowhere”.
5. Competitiveness with respect to a fractional optimal
solution.
General Covering/Packing Results
What can you expect to get?
•
For a {0,1} covering/packing matrix:
– Competitive ratio O(log D) [BN05]
(D – max number of non-zero entries in a constraint).
Remarks:
• Fractional solutions.
•
Number of constraints/variables can be exponential.
•
There can be a tradeoff between the competitive ratio
and the factor by which constraints are violated.
General Covering/Packing Results
• For a general covering/packing matrix [BN05] :
Covering:
– Competitive ratio O(log n)
(n – number of variables).
Packing:
– Competitive ratio O(log n + log [a(max)/a(min)])
a(max), a(min) – maximum/minimum non-zero entry
Remarks:
•
Results are tight.
Special Cases
The max number of non-zero entries in a
constraint is a constant?
• You can get a constant ratio.
The max number of non-zero entries in a
constraint is 2?
• Calls for an e/(e-1)-ratio.
Examples:
• Ski rental, Online matching, Ad-Auctions.
Known Results via P-D Approach
Covering Online Problems (Minimization):
•
O(log k)-algorithm for weighted caching [BBN07]
•
Ski rental, Dynamic TCP Acknowledgement
•
Parking Permit Problem [Meyerson 05]
•
•
Online Set Cover [AAABN03]
Online Graph Covering Problems [AAABN04]:
–
–
–
–
Non-metric facility location
Generalized connectivity: pairs arrive online
Group Steiner: groups arrive online
Online multi-cut: (s,t)--pairs arrive online
Known Results via P-D Approach
Packing Online Problems (maximization):
•
Online Routing/Load Balancing Problems [AAP93,
AAPFW93, BN06].
•
•
•
General Packing/routing e.g. Multicast trees.
Online Matching [KVV91] – Nodes arrive one-by-one.
Ad-Auctions Problem [MSVV05] – In a bit …
Road Map
Introducing the framework:
• Ski rental
• Online set cover
• Virtual circuit routing
The general framework:
• {0,1} covering/packing linear programs
• General covering/packing linear programs
Recent results:
• The ad-auctions problem
• Weighted caching
What are Ad-Auctions?
You type in a search engine: Vacation Eilat
You get:
And …
Advertisements
Algorithmic
Search results
How do search engines sell ads?
• Each advertiser:
– Sets a daily budget
– Provides bids on interesting keywords
• Search Engine (on each keyword):
– Selects ads
– Advertiser pays bid if user clicks on ad.
Goal (of Search engine):
Maximize revenue
Mathematical Model
• Buyer i:
– Has a daily budget B(i)
• Online Setting:
– Items (keywords) arrive one-by-one.
– Each buyer gives a bid on each of the items
(can be zero)
• Algorithm:
– Assigns each item to some interested buyer.
Assumption:
Bids are small compared to the daily budget.
Ad-Auctions – Linear Program
B(i) – Budget of buyer i
b(i,j) – bid of buyer i on item j
I - Set of buyers.
J - Set of items.
y (i, j ) = 1  j-th ad-auction is sold to buyer i.
max  b(i, j ) y (i, j )
iI jJ
s.t:
For each item j:
 y(i, j )  1
iI
For each buyer i:
Each item is
Buyers
do not
sold once.
exceed their
budget
 b(i, j ) y(i, j )  B(i)
jJ
Results
[MSVV FOCS 05]:
• (1-1/e)-competitive online algorithm.
• Bound is tight.
• Analysis uses tradeoff revealing family of LP’s - not
very intuitive.
Our Results [Buchbinder, Jain, N, 2007]:
• A different approach based on the primal-dual method:
very simple and intuitive … and extensions.
• Techniques are applicable to many other problems.
The Paging/Caching Problem
Set of n pages, cache of size k<n.
Request sequence of pages 1, 6, 4, 1, 4, 7, 6, 1, 3, …
If requested page is in cache, no penalty.
Else, cache miss!
And load page into cache, (possibly) evicting some page.
Goal: Minimize the number of cache misses.
Main Question: Which page to evacuate?
Previous Results: Paging
Paging (Deterministic) [Sleator Tarjan 85]:
• Any det. algorithm >= k-competitive.
• LRU is k-competitive (also other algorithms)
• LRU is k/(k-h+1)-competitive if optimal has cache of size h<k
Paging (Randomized):
• Rand. Marking O(log k) [Fiat, Karp, Luby, McGeoch, Sleator,
Young 91].
• Lower bound Hk [Fiat et al. 91], tight results known.
• O(log(k/k-h+1))-competitive algorithm if optimal has
cache of size h<k [Young 91]
The Weighted Paging Problem
One small change:
• Each page i has a different load cost w(i).
• Models scenarios in which the cost of bringing pages is
not uniform:
Main memory, disk, internet …
web
Goal
• Minimize the total cost of cache misses.
Weighted Paging (Previous Work)
Randomized
Deterministic
Paging
Weighted Paging
Lower bound k
LRU
k competitive
k-competitive [Chrobak, Karloff,
Payne, Vishwanathan 91]
k/(k-h+1) if opt’s cache size h
k/(k-h+1)
O(log k) Randomized
Marking
O(log k) for two weight classes
O(log k/(k-h+1))
[Young 94]
[Irani 02]
No o(k) algorithm known even
for 3 weight classes.
The k-server Problem
• k servers lie in an n-point metric space.
• Requests arrive at metric points.
• To serve request: Need to move some server there.
Goal: Minimize total movement cost
• Paging = k-server on a uniform metric.
(every page is a point, page in cache iff server on the point)
• Weighted paging = k-server on a weighted star metric.
The k-server Problem
• k servers lie in an n-point metric space.
• Requests arrive at metric points.
• To serve request: Need to move some server there.
Goal: Minimize total movement cost
(2k-1) Det. Work Function Alg
[Koutsoupias, Papadimitriou 95]
Randomized. No o(k) known
(even for very simple spaces).
Best lower bound (log k)
(widely believed conjecture)
Our Results
Weighted Paging (Randomized):
(Bansal, Buchbinder, N., FOCS 2007)
• O(log k)-competitive algorithm for weighted paging.
• O(log (k/k-h+1))-competitive if opt’s cache size h<k.
Much simpler than previous approaches.
Metrical Task System (Randomized):
• O(log N)-competitive algorithm on a weighted star metric.
• Closely related to k-server problem (details in paper …)
Further Research
Generalized Caching
• Pages have both sizes and fetching costs.
• Motivation: Web-Caching
Special models:
• Bit model: Fetching cost proportional to size (minimize traffic)
• Fault model: Fetching cost is uniform (minimize number of
times a user has to wait for a page)
Results (Bansal, Buchbinder, N., 2008):
• O(log k)-competitive algorithms for Bit and Fault models.
• O(log2k)-competitive algorithm for the general model.
• Requires new interesting ideas and interesting analysis in
the rounding phase.
Further Research
• More applications.
• Extending the general framework
beyond packing/covering.
• The k-server problem?
Thank you