Decomposable Optimisation Methods
LCA Reading Group, 12/04/2011
Dan-Cristian Tomozei
Convexity
• Convex function
• Unique minimum over convex domain
2
Roadmap
•
•
•
•
•
(Sub)Gradient Method
Convex Optimisation crash course
NUM
Basic Decomposition Methods
Implicit Signalling
3
Roadmap
•
•
•
•
•
(Sub)Gradient Method
Convex Optimisation crash course
NUM
Basic Decomposition Methods
Implicit Signalling
4
(Sub)gradient method
• Unconstrained convex optimisation problem
• If objective is differentiable,
• Else,
• Gain sequence
– Constant
– Diminishing
5
Roadmap
•
•
•
•
•
(Sub)Gradient Method
Convex Optimisation crash course
NUM
Basic Decomposition Methods
Implicit Signalling
6
Constrained Convex Optimisation
• “Primal” formulation
• Convex constraints  unique solution
• Lagrangian
• “Dual” function
– For all “feasible” points – lower bound
– Slater’s condition  zero duality gap
7
Optimality conditions
• “Primal” and “dual” formulations
Primal variables
Dual variables
(i.e., Lagrange multipliers)
Optimum
• Karush-Kuhn-Tucker (KKT)
8
Roadmap
•
•
•
•
•
(Sub)Gradient Method
Convex Optimisation crash course
NUM
Basic Decomposition Methods
Implicit Signalling
9
Network Utility Maximisation
• Population of users
• Concave utility functions (e.g., rates)
• Typical formulation (e.g., [Kelly97]):
– Network flows of rates
– Physical links of max capacity
– Routing matrix
– Dual variables = congestion shadow prices
10
Roadmap
•
•
•
•
•
(Sub)Gradient Method
Convex Optimisation crash course
NUM
Basic Decomposition Methods
Implicit Signalling
11
Dual Decomposition
• Coupling constraint
• To decouple – simply write the dual objective
• Iterative dual algorithm:
– Each user computes
– Use a gradient method to update dual variables, e.g.,
12
Primal Decomposition
• Coupling variable
• To decouple – consider fixed coupling variable
• Iterative primal algorithm:
– Solve individual problems and get partial optima
– Update primal coupling variable using gradient method
13
Implementation issues
• Certain problems can be decoupled
• Dual decomposition  dual algorithm
– Primal vars (rates) depend directly on dual vars (prices)
– Price adaptation relies on current rates
– Always closed form?
• Primal decomposition
– The other way around…
• Do we really need to keep track of both primal and
dual variables? Can duals be “measured” instead?
14
Roadmap
•
•
•
•
•
(Sub)Gradient Method
Convex Optimisation crash course
NUM
Basic Decomposition Methods
Implicit Signalling
15
Multipath unicast min-cost live streaming
• Graph
• Supported rate region
• Network cost function
– Unsupported rate allocation 
– Marginal cost positive and strictly increasing
• Source s wants to send data to receiver r at rate  at
minimum cost
– Supported  min-cut is at least
16
Optimisation formulation
• Write Lagrangian
• Primal-dual provably converges to optimum
17
Is it that hard?
• Recall
• Dual variables have queue-like evolution!
• We already queue packets!
18
Implicit Primal-Dual
• Rate control via
• Rate on link (i,j)
– Increase prop to backlog difference
– Decrease prop to marginal cost (measurable – RTT, …)
• Influence of parameter s
– Small  closer optimal allocation, huge queue sizes
– Large  manageable queue sizes, optimality trade-off
19
Conclusion
• Finding a fit-all recipe is hard
• We can handle some cases
• Specific formulations may lead to nice protocols
• See also
– R. Srikant’s “Mathematics of Internet Congestion Control”
– Kelly, Mauloo, Tan - ***
– Palomar, Chiang - ***
20
Questions
21