Stochastic Network Optimization
(a 1-day short course)
l
a
d
b
1
c
Michael J. Neely
University of Southern California
http://www-rcf.usc.edu/~mjneely/
Lectures based on the following text:
L. Georgiadis, M. J. Neely, L. Tassiulas, Resource Allocation
And Cross-Layer Control in Wireless Networks, Foundations
And Trends in Networking, vol. 1, no. 1, pp. 1-144, 2006.
Free PDF download of text on the following web pages:
http://www-rcf.usc.edu/~mjneely/
A hardcopy can be ordered for around $50 from NOW publishers:
A brief history of Lyapunov Drift for Queueing Systems:
Lyapunov Stability:
Tassiulas, Ephremides [91, 92, 93]
P. R. Kumar, S. Meyn [95]
McKeown, Anantharam, Walrand [96, 99]
Kahale, P. E. Wright [97]
Andrews, Kumaran, Ramanan, Stolyar, Whiting [2001]
Leonardi, Mellia, Neri, Marsan [2001]
Neely, Modiano, Rohrs [2002, 2003, 2005]
Joint Stability with Utility Optimization was the Big Open
Question until:
Tsibonis, Georgiadis, Tassiulas [infocom 2003] (special structure net,
linear utils)
Neely, Modiano [thesis 2003, infocom 2005] (general nets and utils)
Eryilmaz, Srikant [infocom 2005] (downlink, general utils)
Stolyar [Queueing Systems 2005] (general nets and utils)
Lyapunov drift for joint stability and performance
optimization:
Neely, Modiano [2003, 2005] (Fairness, Energy)
Georgiadis, Neely, Tassiulas [NOW Publishers, F&T, 2006]
Neely [Infocom 2006, JSAC 2006] (“Super-fast” delay tradeoffs)
Alternate Approaches to Stoch. Performance Optimization:
Tsibonis, Georgiadis, Tassiulas [2003] (special structure net,
linear utils)
Eryilmaz, Srikant [2005] (Fluid Model Transformations)
Stolyar [2005] (Fluid Model Transformations)
Lee, Mazumdar, Shroff [2005] (Stochastic Gradients)
Lin, Shroff [2004] (Scheduling for static channels)
Lyapunov drift for joint stability and performance
optimization:
Neely, Modiano [2003, 2005] (Fairness, Energy)
Georgiadis, Neely, Tassiulas [NOW Publishers, F&T, 2006]
Neely [Infocom 2006, JSAC 2006] (“Super-fast” delay tradeoffs)
Yields Explicit Delay Tradeoff Results!
Alternate Approaches to Stoch. Performance Optimization:
Tsibonis, Georgiadis, Tassiulas [2003] (special structure net,
linear utils)
Eryilmaz, Srikant [2005] (Fluid Model Transformations)
Stolyar [2005] (Fluid Model Transformations)
Lee, Mazumdar, Shroff [2005] (Stochastic Gradients)
Lin, Shroff [2004] (Scheduling for static channels)
Network Model --- The General Picture
lij
Flow Control
Decision Rij(t)
own
other
3 Layers:
• Flow Control (Transport)
• Routing (Network)
• Resource Alloc./Sched. (MAC/PHY)
*From: Resource Allocation and Cross-Layer Control in Wireless Networks by
Georgiadis, Neely, Tassiulas, NOW Foundations and Trends in Networking, 2006
Network Model --- The General Picture
own
other
3 Layers:
1) Flow Control (Transport)
2) Routing (Network)
3) Resource Alloc./Sched. (MAC/PHY)
*From: Resource Allocation and Cross-Layer Control in Wireless Networks by
Georgiadis, Neely, Tassiulas, NOW Foundations and Trends in Networking, 2006
Network Model --- The General Picture
own
other
“Data Pumping Capabilities”:
(mij(t)) = C(I(t), S(t))
3 Layers:
1) Flow Control (Transport)
2) Routing (Network)
3) Resource Alloc./Sched. (MAC/PHY)
Control Action
(Resource
Allocation/Power)
I(t)
Channel
State
Matrix
I
*From: Resource Allocation and Cross-Layer Control in Wireless Networks by
Georgiadis, Neely, Tassiulas, NOW Foundations and Trends in Networking, 2006
DIVBAR for Mobile Networks
= Stationary Node
= Locally Mobile Node
= Fully Mobile Node
= Sink
DIVBAR for Mobile Networks
= Stationary Node
= Locally Mobile Node
= Fully Mobile Node
= Sink
DIVBAR for Mobile Networks
= Stationary Node
= Locally Mobile Node
= Fully Mobile Node
= Sink
DIVBAR for Mobile Networks
= Stationary Node
= Locally Mobile Node
= Fully Mobile Node
= Sink
DIVBAR for Mobile Networks
= Stationary Node
= Locally Mobile Node
= Fully Mobile Node
= Sink
DIVBAR for Mobile Networks
= Stationary Node
= Locally Mobile Node
= Fully Mobile Node
= Sink
DIVBAR for Mobile Networks
= Stationary Node
= Locally Mobile Node
= Fully Mobile Node
= Sink
DIVBAR for Mobile Networks
= Stationary Node
= Locally Mobile Node
= Fully Mobile Node
= Sink
DIVBAR for Mobile Networks
= Stationary Node
= Locally Mobile Node
= Fully Mobile Node
= Sink
DIVBAR for Mobile Networks
= Stationary Node
= Locally Mobile Node
= Fully Mobile Node
= Sink
DIVBAR for Mobile Networks
= Stationary Node
= Locally Mobile Node
= Fully Mobile Node
= Sink
Application 3: Mesh Networks
Communiation Pairs: 0
1, 2
3, … , 8
9
Halfway through the simulation, node 0 moves (non-ergodically)
from its initial location to its final location. Node 9 takes a Markov
Random walk.
0
1
2
8
7
4
6
3
5
9
Full throughput is maintained throughout, with noticeable delay
increase (at “new equilibrium”), but which is independent of mobility
timescales.
More simulation results for the mesh network:
Flow control included (determined by parameter V)
Course Outline:
1. Intro and Motivation (done)
2. Magic of Telescoping Sums
3. Discrete Time Queues
4. Stability and Capacity Regions
5. Lyapunov Drift
6. Max-Weight Algorithms for Stability
(imperfect scheduling, robust to non-ergodic traffic)
(downlinks, backpressure, multi-hop, mobile)
Lyapunov Drift with Performance Optimization
8. Energy Problems
9. Virtual Queues and Power Constraints
10. General Constrained Optimization
11. Other Utilities: Flow control and Fairness
12. Modern Network Models
A (very un-impressive) “Magic Trick”:
5
12
1
11
-3
8
9
7
?
A (very un-impressive) “Magic Trick”:
5
12 - 5 = 7
12
1
11
-3
8
9
7
?
A (very un-impressive) “Magic Trick”:
5
12 - 5 = 7
12
1
11 - 12 = -1
11
-3
8
9
7
?
A (very un-impressive) “Magic Trick”:
5
12 - 5 = 7
12
1
11 - 12 = -1
8 - 11 = -3
11
-3
8
9
7
?
A (very un-impressive) “Magic Trick”:
5
12 - 5 =
12
1
11 - 12 =
8 - 11 =
11 ? - 8 =
-3
8
9
7
?
7
-1
-3
??
A (very un-impressive) “Magic Trick”:
5
12 - 5 =
12
1
11 - 12 =
8 - 11 =
11 ? - 8 =
-3
7 - ? =
8
9
7
?
7
-1
-3
??
??
A (very un-impressive) “Magic Trick”:
5
12 - 5 =
12
1
11 - 12 =
8 - 11 =
11 ? - 8 =
-3
7 - ? =
9 - 7 =
8
9
7
?
7
-1
-3
??
??
2
A (very un-impressive) “Magic Trick”:
5
12 - 5 = 7
12
1
11 - 12 = -1
8 - 11 = -3
11 ? - 8 = ??
-3
7 - ? = ??
9 - 7 = 2
-3 - 9 = -12
8
9
7
?
A (very un-impressive) “Magic Trick”:
5
12 - 5 = 7
12
1
11 - 12 = -1
8 - 11 = -3
11 ? - 8 = ??
-3
7 - ? = ??
9 - 7 = 2
-3 - 9 = -12
8
1
(-3)
=
4
9
7
?
A (very un-impressive) “Magic Trick”:
5
12 - 5 = 7
12
1
11 - 12 = -1
8 - 11 = -3
11 ? - 8 = ??
-3
7 - ? = ??
9 - 7 = 2
-3 - 9 = -12
8
1
(-3)
=
4
9
5 - 1 = 4
?
7
A (very un-impressive) “Magic Trick”:
5
12 - 5 = 7
12
1
11 - 12 = -1
8 - 11 = -3
11 ? - 8 = ??
-3
7 - ? = ??
9 - 7 = 2
-3 - 9 = -12
8
1
(-3)
=
4
9
5 - 1 = 4
?
7
A (very un-impressive) “Magic Trick”:
5
12 - 5 = 7
12
1
11 - 12 = -1
8 - 11 = -3
11 2 - 8 = -6
-3
Zero!
7 - 2 = 5
9 - 7 = 2
-3 - 9 = -12
8
1
(-3)
=
4
9
5 - 1 = 4
?=2
0
7
A (very un-impressive) “Magic Trick”:
5
12 - 5 = 7
12
1
11 - 12 = -1
8 - 11 = -3
11 2 - 8 = -6
-3
Zero!
7 - 2 = 5
9 - 7 = 2
-3 - 9 = -12
8
1
(-3)
=
4
9
5 - 1 = 4
?=2
0
7
A (very un-impressive) “Magic Trick”:
5
12 - 5 = 7
12
1
11 - 12 = -1
8 - 11 = -3
11 2 - 8 = -6
-3
Zero!
7 - 2 = 5
9 - 7 = 2
-3 - 9 = -12
8
1
(-3)
=
4
9
5 - 1 = 4
?=2
0
7
A (very un-impressive) “Magic Trick”:
5
12 - 5 = 7
12
1
11 - 12 = -1
8 - 11 = -3
11 2 - 8 = -6
-3
Zero!
7 - 2 = 5
9 - 7 = 2
-3 - 9 = -12
8
1
(-3)
=
4
9
5 - 1 = 4
?=2
0
7
A (very un-impressive) “Magic Trick”:
5
12 - 5 = 7
12
1
11 - 12 = -1
8 - 11 = -3
11 2 - 8 = -6
-3
Zero!
7 - 2 = 5
9 - 7 = 2
-3 - 9 = -12
8
1
(-3)
=
4
9
5 - 1 = 4
?=2
0
7
Fact (Telescoping Sums):
Let t {0, 1, 2, 3, …}
Let f(t) be any function of discrete time t
Define: D(t) = f(t+1) - f(t) (difference)
M-1
Then:
t=0
D(t) = f(M) - f(0)
f(1)-f(0) + f(2)-f(1) + f(3)-f(2)+…
D(M-2) D(M-1)
D(0) D(1) D(2)
0
1
2
3
M-2
M-1
M
A single discrete time queue:
U(t)
A(t)
m(t)
Discrete time t = {0, 1, 2, …}
U(t) = Queue Backlog (“Unfinished Work”)
at time t [units = “bits,” real number]
A(t) = Arrivals during slot t
m(t) = Transmission rate on slot t (bits/slot)
A single discrete time queue:
U(t)
A(t)
m(t)
Discrete time t = {0, 1, 2, …}
U(t) = Queue Backlog (“Unfinished Work”)
at time t [units = “bits,” real number]
A(t) = Arrivals during slot t
m(t) = Transmission rate on slot t (bits/slot)
*Alternate Units: U(t) = packets (integer), m(t) = packets/slot
A single discrete time queue:
U(t)
A(t)
m(t)
Dynamic Equation:
U(t+1) = max[U(t) - m(t), 0] + A(t)
A single discrete time queue:
U(t)
A(t)
m(t)
Dynamic Equation:
U(t+1) = max[U(t) - m(t), 0] + A(t)
<OR>
U(t+1) = U(t) - m(t) + A(t)
A single discrete time queue:
U(t)
A(t)
m(t)
Dynamic Equation:
U(t+1) = max[U(t) - m(t), 0] + A(t)
<OR>
U(t+1) = U(t) - m(t) + A(t)
(where m(t) = ?? )
A single discrete time queue:
U(t)
A(t)
m(t)
Dynamic Equation:
U(t+1) = max[U(t) - m(t), 0] + A(t)
<OR>
U(t+1) = U(t) - m(t) + A(t)
(where m(t) = min[m(t) , U(t)] )
U(t)
m(t)
A(t)
U(t+1) = U(t) - m(t) + A(t)
=> U(t+1) - U(t) = A(t) - m(t)
(*)
Basic Sample Path Equality (t1 < t2):
t2-1
U(t2) - U(t1) =
t=t1
t2-1
A(t) -
t=t1
m(t)
Proof: Use equation (*) with the telescoping sums fact.
A(t)
m(t)
U(t)
Important Sample Path Equality and Inequalities:
t2-1
U(t2) - U(t1) =
=>
=>
U(t2) - U(t1) >
U(t) >
t=t1
t2-1
A(t) -
t2-1
t2-1
A(t) -
t=t1
t-1
t=t1
m(t)
t=t1
m(t)
t-1
A(t) -
t=0
t=0
m(t)
(if U(0) = 0)
A(t)
U(t)
m(t)
A(t), m(t) are Discrete time Stochastic Processes
(t = {0, 1, 2, …})
Queue Stability:
A discrete time queue U(t) is strongly stable
if:
t-1
1
lim sup t
E{U(t)} <
t
t=0
A(t)
U(t)
m(t)
A(t), m(t) are Discrete time Stochastic Processes
(t = {0, 1, 2, …})
call this “stability” for simplicity
Queue Stability:
A discrete time queue U(t) is strongly stable
if:
t-1
1
lim sup t
E{U(t)} <
t
t=0
Define this as U
A(t)
m(t)
U(t)
Theorem: If m(t) < mmax for all t, and if U(t)
is strongly stable, then:
lim
t
E{U(t)}
t
= 0
Proof: See Appendix D of:
M. J. Neely, “Energy Optimal Control for Time
Varying Wireless Networks,” IEEE Transactions
on Information Theory, vol. 52, no. 7, July 2006.
A(t)
U(t)
m(t)
Corollary: Suppose m(t) < mmax for all t, and:
(1)
(2)
t
1
t
lim
1
t
lim
t
t-1
t=0
t-1
t=0
E{m(t)} = mav
E{A(t)} = l
Then if U(t) is strongly stable, we must have:
>
l
mav
Necessary Condition
for Stability (intuitive)
A(t)
m(t)
U(t)
Proof: We know that (assuming U(0)=0):
U(t) >
t-1
t-1
A(t) -
t=0
-Take Expectations…
t=0
m(t)
A(t)
U(t)
m(t)
Proof: We know that (assuming U(0)=0):
E{U(t)} >
t-1
t=0
t-1
E{A(t)} -
-Take Expectations
-Divide by t…
t=0
E{m(t)}
A(t)
U(t)
m(t)
Proof: We know that (assuming U(0)=0):
E{U(t)} > 1
t
t
t-1
1
E{A(t)} t
t=0
-Take Expectations
-Divide by t
-Take limits…
t-1
t=0
E{m(t)}
A(t)
U(t)
m(t)
Proof: We know that (assuming U(0)=0):
t-1
E{U(t)} > 1
1
E{A(t)} t t=0
t
t
>
0
l
mav
-Take Expectations
-Divide by t
-Take limits
t-1
t=0
E{m(t)}
A(t)
U(t)
m(t)
*Corollary (Alternate form):
Suppose m(t) < mmax for all t. Then if U(t) is
strongly stable, we must have:
lim inf 1
t
t
t-1
t=0
E{m(t) - A(t)} > 0
*Holds in cases when time averages of A(t) and/or m(t)
do not necessarily exist. Proof = similar (exercise).
Thus (assuming l, mav are defined):
>
l
mav is a necessary condition for stability.
A(t) ~ l
U(t)
m(t) ~ mav
We want a sufficient condition for stability
(need more structure on A(t), m(t)…)
Arrival Process A(t) is admissible with rate l if:
t-1
1
lim
E{A(t)}
=
l
t
t
t=0
| H(t)}
>
E{A(t)2
2
Amax
(some Amax <
For any d>0, there is a T such that for any t0:
{
t=0
A(t0+t) | H(t0) }
t
H(t) = Past History up to time t
>
1
E T
T-1
l +d
)
Server Process m(t) is admissible with rate m if:
t-1
1
lim
E{m(t)}
=
m
t
t
t=0
>
m(t)
(some mmax <
mmax for all t
)
For any d>0, there is a T such that for any t0:
1
E T
{
T-1
t=0
A(t0+t) | H(t0) } > m - d
t
H(t) = Past History up to time t
Fact: iid probabilistic splitting (with probability p)
of an admissible process A(t) with rate l yields
an admissible process of rate pl.
A(t) ~ l
p
Asplit(t) ~ pl
1-p
Proof: Simple Exercise
Asplit(t) = A(t) with
prob p, 0 else.
A(t)
U(t)
m(t)
Theorem (Queue Stability): Suppose that
A(t) is admissible with rate l
m(t) is admissible with rate m
Then:
>
(a) l
m
is necessary for stability of U(t)
(b) l < m is sufficient for stability of U(t)
A(t)
U(t)
m(t)
Theorem (Queue Stability): Suppose that
A(t) is admissible with rate l
m(t) is admissible with rate m
Then:
>
(a) l
m
is necessary for stability of U(t)
(b) l < m is sufficient for stability of U(t)
[Proof of (b): T-slot Lyapunov drift, Lemma 4.4 of text]
Definition: A network of queues is
strongly stable if all individual queues
are strongly stable.
A simple routing problem for two queues:
U1(t)
m1
A(t)
(admissible,
rate l)
U2(t)
m2
Rules:
(1) Every slot t, must put all new A(t) arrivals
of that slot to either queue 1 or queue 2.
(2) Must make decision immediately (no
data storage at the decision point).
A simple routing problem for two queues:
U1(t)
m1
A(t)
(admissible,
rate l)
U2(t)
m2
Definition: The Capacity Region L is the
closure of the set of all rates l for which
a stabilizing routing algorithm exists
(subject to the given rules).
A simple routing problem for two queues:
m1
U1(t)
A(t)
(admissible,
rate l)
U2(t)
>
0
l
m1 + m2
>
Claim: L = {l | 0
m2
m1 + m2}
l
A simple routing problem for two queues:
m1
U1(t)
A(t)
(admissible,
rate l)
U2(t)
0
>
Must show 2 things:
(i) l < m1+ m1 sufficient
l
>
Claim: L = {l | 0
m2
m1 + m2}
(ii) l < m1+ m1 necessary
m1 + m2
l
U1(t)
A(t)
(admissible,
rate l)
U2(t)
m1
m2
Proof of Necessity: Under any routing algorithm
we must have (for all slots t):
U1(t) + U2(t) >
Take Expectations…
t-1
t=0
A(t) - (m1 + m2)t
U1(t)
A(t)
(admissible,
rate l)
U2(t)
m1
m2
Proof of Necessity: Under any routing algorithm
we must have (for all slots t):
E{U1(t) + U2(t)} >
Take Expectations
Divide by t…
t-1
t=0
E{A(t)} - (m1 + m2)t
A(t)
U1(t)
(admissible,
rate l)
U2(t)
m1
m2
Proof of Necessity: Under any routing algorithm
we must have (for all slots t):
1
>
E{U1(t) + U2(t)}
t
t
Take Expectations
Divide by t
Take limits…
t-1
t=0
E{A(t)} - (m1 + m2)
U1(t)
A(t)
(admissible,
rate l)
U2(t)
m1
m2
Proof of Necessity: Under any routing algorithm
we must have (for all slots t):
1
>
E{U1(t) + U2(t)}
t
t
lim
t
t-1
t=0
E{A(t)} - (m1 + m2)
E{U1(t) + U2(t)} > l - (m1 + m2)
t
Must be 0 if stable!
A(t)
U1(t)
(admissible,
rate l)
U2(t)
m1
m2
Proof of Sufficiency: Suppose l < m1 + m2.
Must construct a stabilizing routing policy.
Any ideas?
A(t)
(admissible,
rate l)
U1(t)
U2(t)
m1
m2
Proof of Sufficiency: Suppose l < m1 + m2.
Must construct a stabilizing routing policy.
Probabilistic (iid) splitting will suffice for this proof:
Every slot t:
-Put A(t) in queue 1 with probability m1 /(m1 + m2)
-Put A(t) in queue 2 with probability m2 /(m1 + m2)
l
lm1
(m1 + m2)
U1(t)
m1
lm2
U2(t)
m2
(m1 + m2)
Proof of Sufficiency: Suppose l < m1 + m2.
Must construct a stabilizing routing policy.
Probabilistic (iid) splitting will suffice for this proof:
Every slot t:
-Put A(t) in queue 1 with probability m1 /(m1 + m2)
-Put A(t) in queue 2 with probability m2 /(m1 + m2)
Then both queues are stable.
Other definitions of stability in the literature:
1
lim sup t
t
t-1
t=0
E{U(t)} <
lim lim sup 1
M
t
t
lim
t
E{U(t)}
t
t-1
t=0
(strongly
stable)
Pr[U(t) > M] = 0
= 0
(mean rate
stable)
Sample path variations on definitions of stability:
1
lim sup t
t
t-1
t=0
U(t) <
lim lim sup 1
M
t
t
lim
t
U(t)
t
t-1
t=0
= 0
(w.p.1)
1{U(t) > M} = 0
(w.p.1)
(w.p.1) (rate stable)
Other definitions of stability in the literature:
1
lim sup t
t
t-1
t=0
E{U(t)} <
lim lim sup 1
M
t
t
lim
t
E{U(t)}
t
t-1
(strongly
stable)
Pr[U(t) > M] = 0
t=0
= 0
(mean rate
stable)
Note: All these definitions give the same capacity
region L for single and multi-queue networks!
A(t)
U(t)
m(t)
Example of a “Bad” definition of stability:
(Bad) Definition: Suppose we say a queue U(t)
is stable if it empties infinitely often (so that
U(t)=0 for arbitrarily large t).
Corresponding Definition for Networks:
A network of queues is stable if all individual
queues are stable (according to the above
definition of stability).
m1
U1(t)
A(t)
(admissible,
rate l)
m2
U2(t)
What is the capacity region L for the simple
2-queue routing problem under this (bad)
definition of stability? (where a queue
is stable if it empties infinitely often)
0
>
Claim: L = {l | 0
l <
m1 + m2
}
…Why?
l
m1
U1(t)
A(t)
(admissible,
rate l)
m2
U2(t)
(bad def.) Stable = Empties Infinitely Often
Must find a routing algorithm that stabilizes
both queues for any finite l.
0
>
Claim: L = {l | 0
l <
m1 + m2
}
…Why?
l
m1
U1(t)
A(t)
(admissible,
rate l)
m2
U2(t)
(bad def.) Stable = Empties Infinitely Often
Route to queue 1 until queue 2 empties,
then switch, vice versa!
0
>
Claim: L = {l | 0
l <
m1 + m2
}
…Why?
l
Course Outline:
1. Intro and Motivation (done)
2. Magic of Telescoping Sums
3. Discrete Time Queues
4. Stability and Capacity Regions
5. Lyapunov Drift
6. Max-Weight Algorithms for Stability
(imperfect scheduling, robust to non-ergodic traffic)
(downlinks, backpressure, multi-hop, mobile)
Lyapunov Drift with Performance Optimization
8. Energy Problems
9. Virtual Queues and Power Constraints
10. General Constrained Optimization
11. Other Utilities: Flow control and Fairness
12. Modern Network Models
Lyapunov Drift for Multi-Queue Systems
Let U(t) = (U1(t), U2(t), …, UK(t)) be a vector
of queue backlogs that evolve in discrete time
according to some probability law.
Let L(U(t)) be a non-negative function of U(t),
called a Lyapunov function.
U(t) = (U1(t), …, UK(t))
Lyapunov Drift for Multi-Queue Systems
For this course, we use quadratic Lyapunov
K
functions:
2
L(U(t)) =
i=1
Ui(t)
U(t) = (U1(t), …, UK(t))
Lyapunov Drift for Multi-Queue Systems
For this course, we use quadratic Lyapunov
K
functions:
2
L(U(t)) =
i=1
U2
U1
Ui(t)
Useful Properties:
1. L(0) = 0 iff
network empty
2. L(U) “large” implies
“network congested”
3. L(U) small implies
all queues small.
Lyapunov Drift for Multi-Queue Systems
For this course, we use quadratic Lyapunov
K
functions:
2
L(U(t)) =
i=1
Ui(t)
Definition: The 1-slot conditional
Lyapunov Drift D(U(t)) is defined:
D(U(t)) = E{L(U(t+1)) - L(U(t))| U(t)}
*[better notation: D(U(t), t) ]
Theorem (Lyapunov Drift for Stability):
Let L(U) be a non-negative function of U.
Suppose there are positive constants B, e such
that for all t and all U(t):
>
D(U(t))
B- e
K
i=1
Ui(t)
Then all queues are strongly stable, and:
denote this as:
K
t=0 i=1
E{Ui(t)}
K
Ui
i=1
>
lim sup 1
t
t
t-1
B
e
>
D(U(t))
K
B- e
i=1
Ui(t)
Intuition: If the above condition holds, then for
any d>0, we have:
>
D(U(t))
-d
whenever
K
i=1
B+d
>
Ui(t)
e
U2
U1 + U2 = B+d
e
Negative drift region
U1
>
D(U(t))
K
B- e
i=1
Ui(t)
Intuition: If the above condition holds, then for
any d>0, we have:
>
D(U(t))
-d
whenever
K
i=1
B+d
>
Ui(t)
e
U2
U1 + U2 = B+d
e
Negative drift region
U1
>
D(U(t))
K
B- e
i=1
Ui(t)
Intuition: If the above condition holds, then for
any d>0, we have:
>
D(U(t))
-d
whenever
K
i=1
B+d
>
Ui(t)
e
U2
U1 + U2 = B+d
e
Negative drift region
U1
D(U(t)) = E{L(U(t+1)) - L(U(t))| U(t)}
>
D(U(t))
B- e
K
i=1
Ui(t)
(*)
Proof (Lyapunov drift): Eq. (*) implies:
E{L(U(t+1)) - L(U(t))| U(t)}
B- e
K
>
i=1
Ui(t)
Take Expectations of Both sides (iterated expect…)
(with respect to distribution of U(t),
use fact: E{E{X|Y}} = E{X})
D(U(t)) = E{L(U(t+1)) - L(U(t))| U(t)}
>
D(U(t))
B- e
K
i=1
Ui(t)
(*)
Proof (Lyapunov drift): Eq. (*) implies:
E{L(U(t+1))} - E{L(U(t))}
B- e
K
>
i=1
E{Ui(t)}
Take Expectations of Both sides (iterated expect.)
M-1
Use Magic of Telescoping Sums…
t=0
D(U(t)) = E{L(U(t+1)) - L(U(t))| U(t)}
>
D(U(t))
B- e
K
i=1
Ui(t)
(*)
Proof (Lyapunov drift): Eq. (*) implies:
E{L(U(M))} - E{L(U(0))}
MB - e
M-1 K
t=0 i=1
E{Ui(t)}
>
Take Expectations of Both sides (iterated expect.)
Use Magic of Telescoping Sums
Use non-negativity of L(U)…
D(U(t)) = E{L(U(t+1)) - L(U(t))| U(t)}
>
D(U(t))
B- e
K
i=1
Ui(t)
(*)
Proof (Lyapunov drift): Eq. (*) implies:
- E{L(U(0))}
MB - e
M-1 K
t=0 i=1
E{Ui(t)}
>
Take Expectations of Both sides (iterated expect.)
Use Magic of Telescoping Sums
Use non-negativity of L(U)
Divide by Me…
D(U(t)) = E{L(U(t+1)) - L(U(t))| U(t)}
>
D(U(t))
B- e
K
i=1
Ui(t)
(*)
Proof (Lyapunov drift): Eq. (*) implies:
-
1
eM
E{L(U(0))}
B e
1 M-1 K
M
t=0 i=1
E{Ui(t)}
>
Take Expectations of Both sides (iterated expect.)
Use Magic of Telescoping Sums
Use non-negativity of L(U)
Divide by Me
D(U(t)) = E{L(U(t+1)) - L(U(t))| U(t)}
>
D(U(t))
B- e
K
i=1
Ui(t)
(*)
Proof (Lyapunov drift): Eq. (*) implies:
-
eM
>
Rearrange
terms
1
E{L(U(0))}
B e
1 M-1 K
M
t=0 i=1
E{Ui(t)}
Take Expectations of Both sides (iterated expect.)
Use Magic of Telescoping Sums
Use non-negativity of L(U)
Divide by Me
D(U(t)) = E{L(U(t+1)) - L(U(t))| U(t)}
>
D(U(t))
B- e
K
i=1
Ui(t)
(*)
Proof (Lyapunov drift): Eq. (*) implies:
K
Ui
i=1
Take the limsup as t
>
t=0 i=1
E{Ui(t)}
>
1 M-1 K
M
B + E{L(U(0))}
eM
e
B
e
Applications of the Lyapunov Drift Theorem:
A(t)
U(t)
m(t)
Application 1: Prove the l < m sufficiency theorem:
l < m is sufficient for stability of U(t)
We’ll consider the special case when:
m(t) i.i.d. over slots, E{m(t)} = m
A(t) i.i.d. over slots, E{A(t)} = l (where l < m)
E{A(t)2} = E{A2}
[General non-i.i.d. case proved in Lemma 4.4 of text]
Claim:
l < m
A(t)
=>
U(t) is stable
U(t)
m(t)
Proof (iid case): Define L(U(t)) = U(t)2
U(t+1) = max[U(t) - m(t), 0] + A(t)
Useful fact: If queueing law (*) holds, then
>
U(t+1)2
U(t)2 + A(t)2 + m(t)2
- 2U(t)[m(t) - A(t)]
Because (max[U - m, 0])2 <= (U-m)2
(*)
Claim:
l < m
A(t)
=>
U(t) is stable
U(t)
m(t)
Proof (iid case): Define L(U(t)) = U(t)2
>
U(t+1)2
U(t)2 + A(t)2 + m(t)2
- 2U(t)[m(t) - A(t)]
Take conditional expectations given U(t)…
Claim:
l < m
=>
A(t)
U(t) is stable
U(t)
m(t)
Proof (iid case): Define L(U(t)) = U(t)2
U(t)2 + A(t)2 + m(t)2
- 2U(t)[m(t) - A(t)]
E{U(t+1)2|U(t)}
>
>
U(t+1)2
U(t)2 + E{A2} +E{m2}
- 2U(t)[m - l]
Take conditional expectations given U(t)
Claim:
l < m
=>
A(t)
U(t) is stable
U(t)
m(t)
Proof (iid case): Define L(U(t)) = U(t)2
U(t)2 + A(t)2 + m(t)2
- 2U(t)[m(t) - A(t)]
E{U(t+1)2|U(t)}
D(U(t))
>
=>
>
>
U(t+1)2
U(t)2 + E{A2} +E{m2}
- 2U(t)[m - l]
E{A2} +E{m2} - 2U(t)[m - l]
>
D(U(t))
e>0
B
We have:
E{A2} +E{m2} - U(t)2[m - l]
Recall Lyapunov Drift Theorem:
U
i=1
K
Ui
i=1
>
Thus:
B- e
>
Then:
>
D(U(t))
If:
K
Ui(t)
B
e
E{A2} + E{m2}
2[m - l]
(iid A(t), m(t))
>
U
E{A2} + E{m2}
2[m - l]
(iid case)
U
m
>
Little’s Theorem: W
l
E{A2} + E{m2}
2 l[m - l]
>
U
E{A2} + E{m2}
2[m - l]
(iid case)
Idea for general admissible A(t), m(t) (non iid)
case:
Define the T-slot Lyapunov drift (T>0, integer):
D(U(t)) = E{L(U(t+T)) - L(U(t)) | U(t)}
Delay and backlog expression given in text (Lemma 4.4).
Roughly, backlog bound increased by an amount T proportional
to time required to reach “near steady state”
Application of Lyapunov drift to the simple
Routing Problem:
m(t) i.i.d. over slots
A(t)
i.i.d. rate l
U1(t)
m1(t)
U2(t)
m2 (t)
UK(t)
mK(t)
Control Variables: Choose {Ai(t)} for each t so that:
A1(t) + A2(t) + … + AK(t) = A(t) (routing constraint)
Application of Lyapunov drift to the simple
Routing Problem:
m(t) i.i.d. over slots
A(t)
i.i.d. rate l
If l interior to L:
For some e>0, {pi}:
lpi + e < mi (all i)
p1+…+ pK = 1
U1(t)
m1(t)
U2(t)
m2 (t)
UK(t)
mK(t)
Can easily show:
Capacity region = L = {l | 0 < l < m1 + … + mK}
A(t)
U2(t)
m1(t)
m2 (t)
UK(t)
mK(t)
U1(t)
If l interior to L:
For some e>0, {pi}:
lpi + e < mi (all i)
p1+…+ pK = 1
A1(t) + A2(t) + … + AK(t) = A(t) (routing constraint)
By Fact , there exists a stationary randomized
routing policy Ai*(t) such that:
E{Ai*(t)} = lpi < mi - e
(Just Choose Ai*(t) = A(t) with prob pi, Aj*(t) = 0 all other j )
However, let’s build a dynamic policy…
U2(t)
m1(t)
m2 (t)
UK(t)
mK(t)
U1(t)
A(t)
If l interior to L:
For some e>0, {pi}:
lpi + e < mi (all i)
p1+…+ pK = 1
Let’s build a dynamic policy via Lyapunov drift:
L(U(t))=
K
i=1
2
Ui(t) , D(U(t))= E{L(U(t+1))-L(U(t))|U(t)}
Ui(t+1) = max[Ui(t) - mi(t), 0] + Ai(t)
>
Ui(t+1)2
Sum over all i,
Take expectations…
Ui(t)2 + Ai(t)2 + mi(t)2
- 2Ui(t)[mi(t) - Ai(t)]
A(t)
>
D(U(t))
U2(t)
m1(t)
m2 (t)
UK(t)
mK(t)
U1(t)
If l interior to L:
For some e>0, {pi}:
lpi + e < mi (all i)
p1+…+ pK = 1
E{A2}+E{m12+…+ mK2}
K
-2
i=1
Ui(t)E{mi(t) - Ai(t)|U(t)}
m(t) i.i.d. over slots
E{mi(t)|U(t)} = mi
A(t)
>
D(U(t))
U2(t)
m1(t)
m2 (t)
UK(t)
mK(t)
U1(t)
If l interior to L:
For some e>0, {pi}:
lpi + e < mi (all i)
p1+…+ pK = 1
E{A2}+E{m12+…+ mK2}
K
-2
i=1
Ui(t)mi + 2E{
DYNAMIC POLICY: Choose {Ai(t)}
K
Minimize:
i=1
K
i=1
Ui(t)Ai(t)|U(t)}
to minimize RHS.
Ui(t)Ai(t)
Subject to: A1(t) + A2(t) + … + AK(t) = A(t)
Fact: Let f(X, I) be a function of a random
variable X and a control decision I (where I
We want to choose I such that:
E{f(X, I)} is minimized.
We can observe the random X before
choosing I I . Then the minimizing policy
is to choose I*X I that minimizes f(X, I).
Proof: We know f(X, I*X) >= f(X, I),
thus: E{f(X, I*X)} >= E{f(X, I)}
I).
A(t)
>
D(U(t))
U2(t)
m1(t)
m2 (t)
UK(t)
mK(t)
U1(t)
If l interior to L:
For some e>0, {pi}:
lpi + e < mi (all i)
p1+…+ pK = 1
E{A2}+E{m12+…+ mK2}
K
-2
i=1
Ui(t)mi + 2E{
DYNAMIC POLICY: Choose {Ai(t)}
K
Minimize:
i=1
K
i=1
Ui(t)Ai(t)|U(t)}
to minimize RHS.
Ui(t)Ai(t)
Subj. to: A1(t) + A2(t) + … + AK(t) = A(t)
Join-theShortest Queue!
(JSQ)
A(t)
D (U(t))
>
JSQ
U2(t)
m1(t)
m2 (t)
UK(t)
mK(t)
U1(t)
E{A2}+E{m12+…+ mK2}
K
-2
If l interior to L:
For some e>0, {pi}:
lpi + e < mi (all i)
p1+…+ pK = 1
i=1
Ui(t)mi + 2E{
K
i=1
}
JSQ
Ui(t)Ai (t)|U(t)
Given current queue backlog, JSQ minimizes the
RHS over all alternative routing decisions that can
be implemented on slot t! (Let Ai*(t) be any other…)
A(t)
D (U(t))
>
JSQ
U2(t)
m1(t)
m2 (t)
UK(t)
mK(t)
U1(t)
E{A2}+E{m12+…+ mK2}
K
-2
If l interior to L:
For some e>0, {pi}:
lpi + e < mi (all i)
p1+…+ pK = 1
i=1
Ui(t)mi + 2E{
K
i=1
Ui(t)A*i (t)|U(t)}
Given current queue backlog, JSQ minimizes the
RHS over all alternative routing decisions that can
be implemented on slot t! (Let Ai*(t) be any other…)
A(t)
D (U(t))
>
JSQ
U2(t)
m1(t)
m2 (t)
UK(t)
mK(t)
U1(t)
E{A2}+E{m12+…+ mK2}
K
-2
If l interior to L:
For some e>0, {pi}:
lpi + e < mi (all i)
p1+…+ pK = 1
i=1
Ui(t)mi + 2
K
Ui(t)E{A*i (t)|U(t)}
i=1
Given current queue backlog, JSQ minimizes the
RHS over all alternative routing decisions that can
be implemented on slot t! (Let Ai*(t) be any other…)
A(t)
D (U(t))
>
JSQ
U2(t)
m1(t)
m2 (t)
UK(t)
mK(t)
U1(t)
E{A2}+E{m12+…+ mK2}
K
-2
If l interior to L:
For some e>0, {pi}:
lpi + e < mi (all i)
p1+…+ pK = 1
i=1
Ui(t)mi + 2
K
Ui(t)E{A*i (t)|U(t)}
i=1
Recall: By Fact , there exists a stationary
randomized routing policy Ai*(t) such that:
E{Ai*(t)} = lpi < mi - e
A(t)
D (U(t))
>
JSQ
U2(t)
m1(t)
m2 (t)
UK(t)
mK(t)
U1(t)
E{A2}+E{m12+…+ mK2}
K
-2
If l interior to L:
For some e>0, {pi}:
lpi + e < mi (all i)
p1+…+ pK = 1
i=1
Ui(t)mi + 2
K
Ui(t)E{A*i (t)|U(t)}
i=1
Recall: By Fact , there exists a stationary
randomized routing policy Ai*(t) such that:
E{A*i (t)|U(t)}= E{Ai*(t)} = lpi < mi - e
Independent of queue backlog!
A(t)
D (U(t))
>
JSQ
U2(t)
m1(t)
m2 (t)
UK(t)
mK(t)
U1(t)
E{A2}+E{m12+…+ mK2}
K
-2
If l interior to L:
For some e>0, {pi}:
lpi + e < mi (all i)
p1+…+ pK = 1
i=1
Ui(t)mi + 2
K
Ui(t)(mi - e)
i=1
Recall: By Fact , there exists a stationary
randomized routing policy Ai*(t) such that:
E{A*i (t)|U(t)}= E{Ai*(t)} = lpi < mi - e
Independent of queue backlog!
A(t)
D (U(t))
>
JSQ
U2(t)
m1(t)
m2 (t)
UK(t)
mK(t)
U1(t)
E{A2}+E{m12+…+ mK2}
K
-2
If l interior to L:
For some e>0, {pi}:
lpi + e < mi (all i)
p1+…+ pK = 1
i=1
Ui(t)mi + 2
K
Ui(t)(mi - e)
i=1
Recall: By Fact , there exists a stationary
randomized routing policy Ai*(t) such that:
E{A*i (t)|U(t)}= E{Ai*(t)} = lpi < mi - e
Independent of queue backlog!
U2(t)
m1(t)
m2 (t)
UK(t)
mK(t)
U1(t)
A(t)
D (U(t))
>
JSQ
If l interior to L:
For some e>0, {pi}:
lpi + e < mi (all i)
p1+…+ pK = 1
E{A2}+E{m12+…+ mK2}
-2e
K
i=1
Ui(t)
By Lyapunov drift theorem…
Ui
i=1
>
=>
K
E{A2}+E{m12+…+ mK2}
2e
An extended routing problem:
A1(t)
U1(t)
U2(t)
mj(t)
A2(t)
Uj+2(t)
mj+1(t)
mj+2 (t)
UK(t)
mK(t)
Uj+1(t)
AS(t)
m1(t)
m2 (t)
m1(t)
m2 (t)
mj(t)
mj+1(t)
mj+2 (t)
mK(t)
A1(t)
A2(t)
AS(t)
e
e
Capacity Region L
(lots of linear constraints)
Ui(t+1) = max[Ui(t) - mi(t), 0] +
s
Asi(t)
Exercise: Show…
>
D(U(t))
K
B - 2
i=1
+
s
Ui(t)mi
E{
K
i=1
Ui(t) Asi(t) | U(t)}
Minimize RHS: Individual JSQ! (decoupled by user)
Prove thruput-optimality by showing it performs better than any
stationary-randomized (queue independent) algorithm Asi*(t).
Ui(t+1) = max[Ui(t) - mi(t), 0] +
s
Asi(t)
Exercise: Show…
>
D(U(t))
K
B - 2
i=1
+
s
Ui(t)mi
E{
K
i=1
Ui(t) Asi(t) | U(t)}
Minimize RHS: Individual JSQ!
Alternative alg: E{
s
Asi*(t)} = mi - e
Ui(t+1) = max[Ui(t) - mi(t), 0] +
s
Exercise: Show…
>
D(U(t))
B - 2
K
i=1
Ui(t)mi
K
+ 2
i=1
Ui(t)(mi - e )
Minimize RHS: Individual JSQ!
Alternative alg: E{
s
Asi*(t)} = mi - e
Asi(t)
m1(t)
m2 (t)
A2(t)
mj(t)
mj+1(t)
mj+2 (t)
AS(t)
mK(t)
(1) The Stationary Randomized Alg:
A1(t)
Advantages:
Disavantages:
-No Queue Backlog
Needed
-Need to know statistics
-Pre-compute rate splits
-Centralized
-Cannot adapt
m1(t)
m2 (t)
A2(t)
mj(t)
mj+1(t)
mj+2 (t)
AS(t)
mK(t)
(1) The Dynamic (Drift Minimizing) Alg:
A1(t)
Advantages:
-No a-priori rate splits
-No knowledge of statistics
-Decentralized
-Adapts easily to changes
-Better Delay
Disadvantages:
Queue Backlog needed
every slot
m1(t)
m2 (t)
A2(t)
mj(t)
mj+1(t)
mj+2 (t)
AS(t)
mK(t)
(1) The Dynamic (Drift Minimizing) Alg:
A1(t)
Advantages:
-No a-priori rate splits
-No knowledge of statistics
-Decentralized
-Adapts easily to changes
-Better Delay
Disadvantages:
-Queue Backlog needed
every slot
Robust to Non-Ergodic Traffic Variation:
Arrivals i.i.d. E{A(t)} = l(t), but l(t) changes
arbitrarily every slot
e
l(t)
e
Robust to Non-Ergodic Traffic Variation:
Arrivals i.i.d. E{A(t)} = l(t), but l(t) changes
arbitrarily every slot
K
>
D(U(t)) B- 2
i=1
Ui(t)mi +
e
l(t)
e
s
E{
K
i=1
Ui(t)Asi*(t)}
(m i - e )
regardless of t
Robust to Non-Ergodic Traffic Variation:
Arrivals i.i.d. E{A(t)} = l(t), but l(t) changes
arbitrarily every slot
>
D(U(t)) B- 2
K
i=1
Ui(t)mi +
e
e
l(t)
s
E{
K
i=1
Ui(t)Asi*(t)}
(m i - e )
regardless of t
Robust to Non-Ergodic Traffic Variation:
Arrivals i.i.d. E{A(t)} = l(t), but l(t) changes
arbitrarily every slot
>
D(U(t)) B- 2
K
i=1
Ui(t)mi +
e
e
l(t)
s
E{
K
i=1
Ui(t)Asi*(t)}
(m i - e )
regardless of t
Robust to Non-Ergodic Traffic Variation:
Arrivals i.i.d. E{A(t)} = l(t), but l(t) changes
arbitrarily every slot
K
>
D(U(t)) B- 2
i=1
Ui(t)mi +
e
l(t)
e
s
E{
K
i=1
Ui(t)Asi*(t)}
(m i - e )
regardless of t
m1(t)
m2 (t)
A2(t)
mj(t)
mj+1(t)
mj+2 (t)
AS(t)
mK(t)
(1) The Dynamic (Drift Minimizing) Alg:
A1(t)
Advantages:
-No a-priori rate splits
-No knowledge of statistics
-Decentralized
-Adapts easily to changes
-Better Delay
Disadvantages:
-Queue Backlog needed
every slot
Not Necessary!
Let Usi(t) be the source s estimate of queue
backlog Ui(t).
Assume: |Usi(t) - Ui(t)| < s
i=1
K
s
s
Thus…
i=1
K
>
s
Asi(t)Usi(t)
>
K
i=1
JSQ
Asi(t)Usi(t)
JSQ
Asi(t)Usi(t) + s Atot(t)
=>
i=1
Asi(t)Usi(t)
D(U(t))
>
s
K
>
K
s
i=1
JSQ
Asi(t)Usi(t) + s Atot(t)
B + sltot - 2
+ 2
K
K
i=1
i=1
Ui(t)mi
Ui(t)(mi - e )
Thus: Backlog estimates do not affect throughput.
Only Increase delay according to sltot
Method for General Stability Problems with
Lyapunov Drift:
i
Ui2
Compute Lyap Drift: D(U(t))
>
Define L(U) =
B + RHS
Design Alg. To minimize RHS
>
Show RHS
RHS*
(where RHS* corresponds to a stationary randomized
algorithm over which stability can be achieved)
Properties of the Resulting Min Drift Algorithm
(for general networks):
>
D(U(t))
B + s + RHS
Also stabilizes for non-i.i.d. traffic/channels/mobility.
Robust to Changes in System Statistics (non-ergodic)
Can use out-of-date queue backlog info (delay tradeoff…)
Also: “Imperfect Scheduling” for gRHS solutions,
get within factor g of the capacity region.
Proof of these 4 properties, see: M. J. Neely, “Dynamic Power Allocation
And Routing for Satellite and Wireless Networks with Time Varying
Channels,” Ph.D. Dissertation, Mass. Inst. Tech, 2003. [Also NOW text]
Imperfect Scheduling Result (gRHS solutions) of
Recent Interest for distributed MAC Design!
>
D(U(t))
B + s + RHS
L
gL
See also alternate techniques for imperfect scheduling results:
Lin, Shroff [Infocom 2005] (factor of 2 results via MSM)
Chaporkar, Kar, Sarkar [Allerton 2005]
Wu, Srikant [CDC 2005, INFOCOM 2006]
Imperfect Scheduling Result (gRHS solutions) of
Recent Interest for distributed MAC Design!
>
D(U(t))
B + s + RHS
L
Also:
DARPA CB-MANET
gL
See also alternate techniques for imperfect scheduling results:
Lin, Shroff [Infocom 2005] (factor of 2 results via MSM)
Chaporkar, Kar, Sarkar [Allerton 2005]
Wu, Srikant [CDC 2005, INFOCOM 2006]
Course Outline:
1. Intro and Motivation (done)
2. Magic of Telescoping Sums
3. Discrete Time Queues
4. Stability and Capacity Regions
5. Lyapunov Drift
6. Max-Weight Algorithms for Stability
(imperfect scheduling, robust to non-ergodic traffic)
(downlinks, backpressure, multi-hop, mobile)
Lyapunov Drift with Performance Optimization
8. Energy Problems
9. Virtual Queues and Power Constraints
10. General Constrained Optimization
11. Other Utilities: Flow control and Fairness
12. Modern Network Models
Example System with Scheduling Constraints
(Opportunistic Scheduling):
S1(t) {ON, OFF}
A1(t) ~ l1
A2(t) ~ l2
S2(t) {ON, OFF}
Ii(t) = Control Decision (0/1), I1(t) + I2(t) = 1
mi(t) =
{
1
0
if Si(t) = ON and Ii(t)=1
if Si(t) = OFF
A1(t) ~ l1
S1(t) {ON, OFF}
A2(t) ~ l2
S2(t) {ON, OFF}
Let qi = Pr[Si(t) = ON]
(S1(t), S2(t) indep. for simplicity)
l2
q2
l1+ l2 = q1 + (1-q1) q2
L
q1
l1
A1(t) ~ l1
S1(t) {ON, OFF}
A2(t) ~ l2
S2(t) {ON, OFF}
Ui(t+1) = max[Ui(t) - mi(t), 0] + Ai(t)
D(U(t)) B- 2E{
>
mi(t) =
{
K
i=1
Ui(t)mi(t)| U(t)} + E
1 if Si(t) = ON and Ii(t)=1
0 if Si(t) = OFF
=>
K
i=1
Ui(t)li
LCQ Alg!
LCQ = “Longest Connected Queue”
[Tassiulas, Ephremides Trans. Information Theory 1993]
A1(t) ~ l1
S1(t) {ON, OFF}
A2(t) ~ l2
S2(t) {ON, OFF}
LCQ Algorithm is robust:
l2
q2
l1+ l2 = q1 + (1-q1) q2
l(t)
q1
l1
A1(t) ~ l1
S1(t) {ON, OFF}
A2(t) ~ l2
S2(t) {ON, OFF}
LCQ Algorithm is robust:
l2
q2
l1+ l2 = q1 + (1-q1) q2
l(t)
q1
l1
A1(t) ~ l1
S1(t) {ON, OFF}
A2(t) ~ l2
S2(t) {ON, OFF}
LCQ Algorithm is robust:
l2
q2
l1+ l2 = q1 + (1-q1) q2
l(t)
q1
l1
A1(t) ~ l1
S1(t) {ON, OFF}
A2(t) ~ l2
S2(t) {ON, OFF}
LCQ Algorithm is robust:
l2
q2
l1+ l2 = q1 + (1-q1) q2
l(t)
q1
l1
General Multi-Hop Networks (links (a,b)):
1
N
3
2
S(t) = S1
4
5
S(t) = Topology State at time t
(possibly varying channels, node locations)
General Multi-Hop Networks (links (a,b)):
1
N
3
2
S(t) = S2
4
5
S(t) = Topology State at time t
(possibly varying channels, node locations)
General Multi-Hop Networks (links (a,b)):
1
N
3
2
S(t) = S3
4
5
S(t) = Topology State at time t
(possibly varying channels, node locations)
General Multi-Hop Networks (links (a,b)):
1
N
3
2
S(t) = S4
4
5
S(t) = Topology State at time t
(possibly varying channels, node locations)
General Multi-Hop Networks (links (a,b)):
1
3
2
4
N
S(t) = S5
5
S(t) = Topology State at time t
(possibly varying channels, node locations)
General Multi-Hop Networks (links (a,b)):
1
3
2
4
N
S(t) = S6
5
S(t) = Topology State at time t
(possibly varying channels, node locations)
General Multi-Hop Networks (links (a,b)):
1
3
2
4
N
S(t) = S7
5
S(t) = Topology State at time t
(possibly varying channels, node locations)
General Multi-Hop Networks (links (a,b)):
1
2
N
3
4
S(t) = S8
5
S(t) = Topology State at time t
(possibly varying channels, node locations)
General Multi-Hop Networks (links (a,b)):
1
2
N
3
4
S(t) = S9
5
S(t) = Topology State at time t
(possibly varying channels, node locations)
General Multi-Hop Networks (links (a,b)):
1
2
N
3
4
S(t) = S9
5
S(t) = Topology State at time t
(possibly varying channels, node locations)
I(t) = General Network-Wide Resource Allocation
Decision (Power, Servers, Frequency Bands, etc.)
Constraint: I(t) I
m(t) = (mab(t)) = C(I(t), S(t)) (matrix valued)
C(I(t), S(t)) = General Transmission Rate Function
General Multi-Hop Networks (links (a,b)):
1
2
N
3
4
S(t) = S9
5
control decision
topology state
m(t) = (mab(t)) = (Cab(I(t), S(t)))
C(I(t), S(t)) = “Data Pumping” Ability of Network
over one-hop links
Controller (Resource Allocation):
Observe S(t), Make decision I(t)
I
Examples: Static Wireline Network
1
2
N
3
4
5
(Cab(I(t), S(t))) = (Cab)
constant function:
-no S(t) variation
-no I(t) resource allocation
Examples: Wireless Downlink with Fading Channels
A1(t) ~ l1
S1(t) {ON, OFF}
A2(t) ~ l2
S2(t) {ON, OFF}
Ci(I(t), S(t))
=
{
1 if Si(t) = ON and Ii(t)=1
0 if Si(t) = OFF
I = {(I1, I2) | Ii {0, 1}, I1 + I2 = 1}
Examples: Static Wireless Network with Scheduling
2
N
4
1
3
No S(t) variation
5
I(t) = Network-wide Server Scheduling Decision
I(t) I = {Link Activation Sets}
Cij(I(t)) =
{
mij(t) if link (I,j) part of I(t)
0
else
This is the Model of the seminal Tassiulas, Ephremides 1992,
Which introduced MWM and Backpressure!
Examples: Static Wireless Network with SINR
2
N
4
1
3
SINRij(t) =
I(t) = P(t)
P
No S(t) variation
5
aijPi(t)
Nj +
akjPk(t)
k=i
Cij(I(t), S(t)) = log(1 + SINRij(P))
Examples: Mobile Wireless Network with SINR
I(t) = P(t)
2
N
4
1
3
P
S(t) = Topology State
Model from Neely,
Modiano, Rohrs
[JSAC 2005]
5
Example f() rate-power curve:
Modulation Bits/Symbol SIR/Symbol
2 PAM
1
.25D2
4 QAM
2
.50D2
16 QAM
4
1.25D2
64 QAM
6
5.25D2
Bit Rate
Cij(I(t), S(t)) = fij(SINRij(P(t), S(t)))
f(SINR)
SINR
Examples: Mobile Wireless Network with SINR
I(t) = P(t)
2
N
4
1
3
P
S(t) = Topology State
Model from Neely,
Modiano, Rohrs
[JSAC 2005]
5
Example f() rate-power curve:
Modulation Bits/Symbol SIR/Symbol
2 PAM
1
.25D2
4 QAM
2
.50D2
16 QAM
4
1.25D2
64 QAM
6
5.25D2
Bit Rate
Cij(I(t), S(t)) = fij(SINRij(P(t), S(t)))
f(SINR)
SINR
Examples: Mobile Wireless Network with SINR
I(t) = P(t)
2
N
4
1
3
P
S(t) = Topology State
Model from Neely,
Modiano, Rohrs
[JSAC 2005]
5
Example f() rate-power curve:
Modulation Bits/Symbol SIR/Symbol
2 PAM
1
.25D2
4 QAM
2
.50D2
16 QAM
4
1.25D2
64 QAM
6
5.25D2
Bit Rate
Cij(I(t), S(t)) = fij(SINRij(P(t), S(t)))
f(SINR)
SINR
Examples: Mobile Wireless Network with SINR
I(t) = P(t)
2
N
4
1
3
P
S(t) = Topology State
Model from Neely,
Modiano, Rohrs
[JSAC 2005]
5
Example f() rate-power curve:
Modulation Bits/Symbol SIR/Symbol
2 PAM
1
.25D2
4 QAM
2
.50D2
16 QAM
4
1.25D2
64 QAM
6
5.25D2
Bit Rate
Cij(I(t), S(t)) = fij(SINRij(P(t), S(t)))
f(SINR)
SINR
Examples: Mobile Wireless Network with SINR
I(t) = P(t)
2
N
4
1
3
P
S(t) = Topology State
Model from Neely,
Modiano, Rohrs
[JSAC 2005]
5
Example f() rate-power curve:
Modulation Bits/Symbol SIR/Symbol
2 PAM
1
.25D2
4 QAM
2
.50D2
16 QAM
4
1.25D2
64 QAM
6
5.25D2
Bit Rate
Cij(I(t), S(t)) = fij(SINRij(P(t), S(t)))
f(SINR)
SINR
Examples: Mobile Wireless Network with SINR
I(t) = P(t)
2
N
4
1
3
P
S(t) = Topology State
Model from Neely,
Modiano, Rohrs
[JSAC 2005]
5
Example f() rate-power curve:
Modulation Bits/Symbol SIR/Symbol
2 PAM
1
.25D2
4 QAM
2
.50D2
16 QAM
4
1.25D2
64 QAM
6
5.25D2
Bit Rate
Cij(I(t), S(t)) = fij(SINRij(P(t), S(t)))
f(SINR)
SINR
Examples: Mobile Wireless Network with SINR
I(t) = P(t)
2
4
N
S(t) = Topology State
1
3
P
Model from Neely,
Modiano, Rohrs
[JSAC 2005]
5
Example f() rate-power curve:
Modulation Bits/Symbol SIR/Symbol
2 PAM
1
.25D2
4 QAM
2
.50D2
16 QAM
4
1.25D2
64 QAM
6
5.25D2
Bit Rate
Cij(I(t), S(t)) = fij(SINRij(P(t), S(t)))
f(SINR)
SINR
Examples: Mobile Wireless Network with SINR
I(t) = P(t)
2
1
4
3
N
P
S(t) = Topology State
Model from Neely,
Modiano, Rohrs
[JSAC 2005]
5
Example f() rate-power curve:
Modulation Bits/Symbol SIR/Symbol
2 PAM
1
.25D2
4 QAM
2
.50D2
16 QAM
4
1.25D2
64 QAM
6
5.25D2
Bit Rate
Cij(I(t), S(t)) = fij(SINRij(P(t), S(t)))
f(SINR)
SINR
Examples: Mobile Wireless Network with SINR
2
1
I(t) = P(t)
4
N
3
P
S(t) = Topology State
Model from Neely,
Modiano, Rohrs
[JSAC 2005]
5
Example f() rate-power curve:
Modulation Bits/Symbol SIR/Symbol
2 PAM
1
.25D2
4 QAM
2
.50D2
16 QAM
4
1.25D2
64 QAM
6
5.25D2
Bit Rate
Cij(I(t), S(t)) = fij(SINRij(P(t), S(t)))
f(SINR)
SINR
Examples: Mobile Wireless Network with SINR
2
1
I(t) = P(t)
4
N
3
P
S(t) = Topology State
Model from Neely,
Modiano, Rohrs
[JSAC 2005]
5
Example f() rate-power curve:
Modulation Bits/Symbol SIR/Symbol
2 PAM
1
.25D2
4 QAM
2
.50D2
16 QAM
4
1.25D2
64 QAM
6
5.25D2
Bit Rate
Cij(I(t), S(t)) = fij(SINRij(P(t), S(t)))
f(SINR)
SINR
Other Examples:
Cij(I(t), S(t))
I(t) can include OFDM frequency band
selection, etc.
I = {Iij} , Iij = {Red, Green, Blue}
i
j
General Multi-Hop Problem:
1
l(3)
2
l(N)
2
2
N
3
4
5
l(N)
2
l(3)
2
Ai(c)(t) = Exogenous data arriving to source i on slot t
destined for node c (“commodity c data”)
Assume Ai(c)(t) admissible with rate l(c)
i
Simplest case: Assume time independence,
so E{Ai(c)(t) } = l(c)
i
General Routing and Transmission Scheduling:
link (a,b)
a
b
a
A closeup
view at
timeslot t
b
= “Red Commodity”
Timeslotted system t = {0, 1, 2, …}
Ui(c)(t) = Queue Backlog at node i of commodity c
(i.e., final destination = node c)
General Routing and Transmission Scheduling:
link (a,b)
a
b
a
A closeup
view at
timeslot t
b
= “Red Commodity”
Every Slot t, observe S(t) (topology) and U(t) (backlogs)
Resource Allocation: Choose I(t) I
Determines transmission rates mij(t) = Cij(I(t), S(t))
Routing: Choose mij(c)(t) (routing decisions) such that:
>
c
mij(c)(t)
mij(t)
Lyapunov Drift Analysis:
Un(c)(t+1)
n, c
>
L(U(t)) =
(Un(c)(t))2
max[Un(c)(t) -
b
+ An(c)(t) +
Exogenous arrivals
mnb(c)(t), 0]
a
man(c)(t)
Endogenous arrivals
Lyapunov Drift Analysis:
>
Un(c)(t+1)
max[Un(c)(t) -
b
mnb(c)(t), 0]
+ An(c)(t) +
>
D(U(t))
- 2
n, c
B + 2
a
man(c)(t)
Un(c)(t)ln(c)
n, c
Un(c)(t)E{
Switch the sums
b
mnb(c)(t) -
b
mnb(c)(t) | U(t)}
Lyapunov Drift Analysis:
>
Un(c)(t+1)
max[Un(c)(t) -
b
mnb(c)(t), 0]
+ An(c)(t) +
>
D(U(t))
- 2
n, c
-2E{
B + 2
Un(c)(t)E{
a
man(c)(t)
Un(c)(t)ln(c)
n, c
b
mnb(c)(t) -
b
mnb(c)(t) | U(t)}
Switch the sums
mab(c)(t)[Ua(c)(t) - Ub(c)(t)] |
a,b,c
Maximize this every slot t!
U(t)}
Given U(t), S(t), schedule to maximize:
a,b,c
mab(c)(t)[Ua(c)(t) - Ub(c)(t)]
Wab(c)(t) = Differential Backlog
Wab*(t) = max differential backlog (over all c)
cab*(t) = the optimal commodity for (a,b) link
link (a,b)
a
b
a
A closeup
view at
timeslot t
b
= Optimal Commodity for link (a,b)
on slot t (maximizes diff. backlog)
Resource Allocation: Observe S(t), W*(t). Choose I(t) to:
Max:
a,b
Cab(I(t), S(t)) Wab*(t) , s.t. I(t) I
Routing: Send commodity cab*(t) data over link (a,b),
with rate mab(t) = Cab(I(t), S(t)). (if not enough, idle fill)
Wab*(t) = max differential backlog (over all c)
cab*(t) = the optimal commodity for (a,b) link
link (a,b)
a
b
a
A closeup
view at
timeslot t
b
= Optimal Commodity for link (a,b)
on slot t (maximizes diff. backlog)
Define: Capacity Region L is the closure of the set of all
data rate matrices (li(c)) that can be stabilized by some
Routing and Resource allocation algorithm for the
given network structure C(I(t), S(t)).
l2
l1
Theorem: If (li(c)) is strictly interior to L, then the
dynamic routing policy (“DRPC”) stabilizes the
network and yields:
>
n, c
Un(c)
B
2e
l2
e
l
(li(c)+e) L
e
l1
Theorem: If (li(c)) is strictly interior to L, then the
dynamic routing policy (“DRPC”) stabilizes the
network and yields:
>
n, c
Un(c)
B
2e
l2
-Robust to changes
-Can take out-of-date
queue backlog
-Imperfect scheduling
gRHS => gL
l
l1
Theorem: If (li(c)) is strictly interior to L, then the
dynamic routing policy (“DRPC”) stabilizes the
network and yields:
>
n, c
Un(c)
B
2e
l2
-Robust to changes
-Can take out-of-date
queue backlog
-Imperfect scheduling
gRHS => gL
l
l1
Theorem: If (li(c)) is strictly interior to L, then the
dynamic routing policy (“DRPC”) stabilizes the
network and yields:
>
n, c
Un(c)
B
2e
l2
-Robust to changes
-Can take out-of-date
queue backlog
-Imperfect scheduling
gRHS => gL
l
l1
Theorem: If (li(c)) is strictly interior to L, then the
dynamic routing policy (“DRPC”) stabilizes the
network and yields:
>
n, c
Un(c)
B
2e
l2
-Robust to changes
-Can take out-of-date
queue backlog
-Imperfect scheduling
gRHS => gL
l
l1
Implementation 1: A Mobile Wireless Network Under Dynamic Routing
and Power Control (DRPC), [Neely, Modiano, Rohrs JSAC 2005]
Modulation Bits/Symbol SIR/Symbol
2 PAM
1
.25D2
4 QAM
2
.50D2
16 QAM
4
1.25D2
64 QAM
6
5.25D2
Bit Rate
-10 Wireless Mobiles
-Markov Random Walks
-Omnidirectional Radiation
-Transmission Rate is a function
of Signal-to-Interference Ratio
(SIR) as below.
f(SIR)
SIR
Total Avg. Backlog
Average Number of Packets/Node versus Exogenous Arrival
Rate lambda for:
-2-hop Relay [Grossglausser, Tse TON 2002]
-DRPC Centralized, Distributed [Neely, Modiano, Rohrs JSAC 2005]
Rate l of each source
Course Outline:
1. Intro and Motivation (done)
2. Magic of Telescoping Sums
3. Discrete Time Queues
4. Stability and Capacity Regions
5. Lyapunov Drift
6. Max-Weight Algorithms for Stability
(imperfect scheduling, robust to non-ergodic traffic)
(downlinks, backpressure, multi-hop, mobile)
Lyapunov Drift with Performance Optimization
8. Energy Problems
9. Virtual Queues and Power Constraints
10. General Constrained Optimization
11. Other Utilities: Flow control and Fairness
12. Modern Network Models
Lyapunov Drift with Performance Optimization (Motivation):
l1
S1(t) {ON, OFF}
l2
S2(t) {ON, OFF}
Additional flow control decisions!
Maximize:
L
i=1
Subject to:
gi(ri)
r L
0 < ri < li
l2
l1
general concave
utility functions
of thruput
Lyapunov Drift with Performance Optimization (Motivation):
l1
S1(t) {ON, OFF}
l2
S2(t) {ON, OFF}
Additional flow control decisions!
Maximize:
L
i=1
Subject to:
gi(ri)
r L
0 < ri < li
l2
l1
general concave
utility functions
of thruput
A brief history of Lyapunov Drift for Queueing Systems:
Lyapunov Stability:
Tassiulas, Ephremides [91, 92, 93]
P. R. Kumar, S. Meyn [95]
McKeown, Anantharam, Walrand [96, 99]
Kahale, P. E. Wright [97]
Andrews, Kumaran, Ramanan, Stolyar, Whiting [2001]
Leonardi, Mellia, Neri, Marsan [2001]
Neely, Modiano, Rohrs [2002, 2003, 2005]
Joint Stability with Utility Optimization was the Big Open
Question until:
Tsibonis, Georgiadis, Tassiulas [infocom 2003] (special structure net,
linear utils)
Neely, Modiano [thesis 2003, infocom 2005] (general nets and utils)
Eryilmaz, Srikant [infocom 2005] (downlink, general utils)
Stolyar [Queueing Systems 2005] (general nets and utils)
Comparison of previous algorithms:
(1) Tassiulas Alg. MWM (max Uimi)
(2) Borst Alg. [Borst Infocom 2003] (max mi/mi)
(3) Tse Alg. [Tse 97, 99, Kush 2002] (max mi/ri)
Curves from [Neely, Modiano, Li, INFOCOM 2005]
Lyapunov drift for joint stability and performance
optimization:
Neely, Modiano [2003, 2005] (Fairness, Energy)
Georgiadis, Neely, Tassiulas [NOW Publishers, F&T, 2006]
Neely [Infocom 2006, JSAC 2006] (“Super-fast” delay tradeoffs)
Alternate Approaches to Stoch. Performance Optimization:
Tsibonis, Georgiadis, Tassiulas [2003] (special structure net,
linear utils)
Eryilmaz, Srikant [2005] (Fluid Model Transformations)
Stolyar [2005] (Fluid Model Transformations)
Lee, Mazumdar, Shroff [2005] (Stochastic Gradients)
Lin, Shroff [2004] (Scheduling for static channels)
Lyapunov drift for joint stability and performance
optimization:
Neely, Modiano [2003, 2005] (Fairness, Energy)
Georgiadis, Neely, Tassiulas [NOW Publishers, F&T, 2006]
Neely [Infocom 2006, JSAC 2006] (“Super-fast” delay tradeoffs)
Yields Explicit Delay Tradeoff Results!
Alternate Approaches to Stoch. Performance Optimization:
Tsibonis, Georgiadis, Tassiulas [2003] (special structure net,
linear utils)
Eryilmaz, Srikant [2005] (Fluid Model Transformations)
Stolyar [2005] (Fluid Model Transformations)
Lee, Mazumdar, Shroff [2005] (Stochastic Gradients)
Lin, Shroff [2004] (Scheduling for static channels)
A1(t)
A2(t)
m1(t)
m2(t)
Example: Can either be idle, or allocate 1 Watt to a single queue.
Assume identical rate-power functions for i = 1, 2:
Lyapunov Optimization -- The theory:
Let U(t) be a discrete time vector process that
evolves according to some probability law. Let
f(t) and g(t) be related stochastic processes. If
there is a non-negative function L(U) such that
for all U(t), t, we have:
>
D(U(t))
E{g(t)| U(t)} - E{f(t)| U(t)}
Then:
t=0
E{f(t)}
>
1
t
t-1
1
t
t-1
E{L(U(0))}
E{g(t)} +
t
t=0
Lyapunov Optimization -- The theory:
Let U(t) be a discrete time vector process that
evolves according to some probability law. Let
f(t) and g(t) be related stochastic processes. If
there is a non-negative function L(U) such that
for all U(t), t, we have:
>
D(U(t))
E{g(t)| U(t)} - E{f(t)| U(t)}
Then:
t
t
t=0
E{f(t)}
>
lim inf 1
t-1
lim inf 1
t
t
t-1
E{g(t)}
t=0
Lyapunov Optimization -- The theory:
Let U(t) be a discrete time vector process that
evolves according to some probability law. Let
f(t) and g(t) be related stochastic processes. If
there is a non-negative function L(U) such that
for all U(t), t, we have:
>
D(U(t))
E{g(t)| U(t)} - E{f(t)| U(t)}
Then:
t
t
t=0
E{f(t)}
>
lim sup 1
t-1
lim sup 1
t
t
t-1
E{g(t)}
t=0
Proof:
E{g(t)| U(t)} - E{f(t)| U(t)}
E{L(U(t+1))} - E{L(U(t))}
>
E{L(U(M))}-E{L(U(0))}
>
>
D(U(t))
M-1
t=0
=>
t=0
E{f(t)}
>
M-1
M-1
t=0
E{g(t)} - E{f(t)}
E{g(t)} -
M-1
E{f(t)}
t=0
E{g(t)} + E{L(U(0))}
Theorem: Let U(t) = Vector Queue Process. Let
L(U) be a non-negative function. Let x(t) > xmin be a
penalty process, let V>0. If for all t, U(t) we have:
>
D(U(t))+VE{x(t)|U(t)}
B - E{h(U(t))|U(t)} + Vx*
for some constant B, some non-neg function h(), and
some target penalty x*, then:
limsup 1
t
t
E{h(U(t))}
t=0
t-1
E{x(t)}
t=0
>
t
t
>
limsup 1
t-1
B + V(x* - xmin)
x* + B/V
>
D(U(t))
>
D(U(t))+VE{x(t)|U(t)}
B - E{h(U(t))|U(t)} + Vx*
B + Vx* - VE{x(t) + h(U(t))|U(t)}
f(t)
g(t)
t=0
t-1
E{h(U(t))}
t=0
=>
1
t
t-1
E{x(t)}
t=0
B + Vx*
B + V(x* - xmin)
>
1
t
VE{x(t) + h(U(t))}
>
=>
1
t
>
=>
t-1
x* + B/V
Example: One-Hop Net with C(I(t), S(t)).
Penalty process x(I(t)). Channels S(t) iid over t
A1(t)
A2(t)
C1(I(t), S(t)) C2(I(t), S(t))
AK(t)
S={Totally Awesome}
CK(I(t), S(t))
Objective: Stabilize all queues with minimum time
average penalty.
Example: One-Hop Net with C(I(t), S(t)).
Penalty process x(I(t)). Channels S(t) iid over t
A1(t)
A2(t)
AK(t)
e
C1(I(t), S(t)) C2(I(t), S(t))
e
CK(I(t), S(t))
Assume optimality can be achieved over the class of
stationary randomized policies that choose I*(t) as a
(possibly random) function of S(t) only (not queue backlog).
(True for penalty processes of power expenditure, etc.)
Example: One-Hop Net with C(I(t), S(t)).
Penalty process x(I(t)). Channels S(t) iid over t
A1(t)
A2(t)
AK(t)
e
C1(I(t), S(t)) C2(I(t), S(t))
Assume that (l + e)
e
CK(I(t), S(t))
L. Consider stat. alg for (l + e).
Then: E{C(I*(t), S(t))}
>
l+e
E{x(I*(t))} = x*(e)
Min avg. penalty
To stabilize l + e
A1(t)
A2(t)
C1(I(t), S(t)) C2(I(t), S(t))
AK(t)
CK(I(t), S(t))
Dynamic Policy: use L(U) = (1/2)
k
>
D(U(t))
B +
k
Uk(t)lk
Uk(t)E{Ck(I(t),S(t))| U(t)}
k
U k2
Recall: From Lyap Opt. Theorem, we want to get:
>
D(U(t))+VE{x(t)|U(t)}
B - E{h(U(t))|U(t)} + Vx*
Dynamic Policy:
k
>
D(U(t))
B +
k
Uk(t)lk
Uk(t)E{Ck(I(t),S(t))| U(t)}
Recall: From Lyap Opt. Theorem, we want to get:
>
D(U(t))+VE{x(t)|U(t)}
B - E{h(U(t))|U(t)} + Vx*
Dynamic Policy:
k
>
D(U(t))+VE{x(I(t))|U(t)}
B+
k
Uk(t)lk
Uk(t)E{Ck(I(t),S(t))| U(t)}+ VE{x(I(t))|U(t)}
Minimize RHS
Recall: From Lyap Opt. Theorem, we want to get:
>
D(U(t))+VE{x(t)|U(t)}
B - E{h(U(t))|U(t)} + Vx*
Dynamic Policy:
k
>
D(U(t))+VE{x(I(t))|U(t)}
B+
k
Uk(t)E{Ck(I(t),S(t))| U(t)}+ VE{x(I(t))|U(t)}
B+
>
k
k
Uk(t)lk
Uk(t)lk
Uk(t)E{Ck(I*(t),S(t))| U(t)}+ VE{x(I*(t))|U(t)}
I*(t) = Stationary Randomized Policy
Recall: From Lyap Opt. Theorem, we want to get:
>
D(U(t))+VE{x(t)|U(t)}
B - E{h(U(t))|U(t)} + Vx*
Dynamic Policy:
k
>
D(U(t))+VE{x(I(t))|U(t)}
B+
k
Uk(t)lk
Uk(t)E{Ck(I*(t),S(t))| U(t)}+ VE{x(I*(t))|U(t)}
I*(t) = Stationary Randomized Policy
E{C(I*(t), S(t))}
>
l+e
E{x(I*(t))} = x*(e)
Recall: From Lyap Opt. Theorem, we want to get:
>
D(U(t))+VE{x(t)|U(t)}
B - E{h(U(t))|U(t)} + Vx*
Dynamic Policy:
E{C(I*(t), S(t))}
>
>
D(U(t))+VE{x(I(t))|U(t)}
B-
l+e
E{x(I*(t))} = x*(e)
k
Uk(t)e + Vx*(e)
Recall: From Lyap Opt. Theorem, we want to get:
>
D(U(t))+VE{x(t)|U(t)}
B - E{h(U(t))|U(t)} + Vx*
Dynamic Policy: (assume x(I(t)) > 0)
limsup 1
t
t
t-1
k
Uk(t)e + Vx*(e)
e
E{x(t)}
t=0
>
k
Uk
B-
B + Vx*(e)
>
=>
>
D(U(t))+VE{x(I(t))|U(t)}
x*(e) + B/V
t-1
e
E{x(t)}
>
limsup 1
t
t
>
k
Uk
B + Vx*(e)
x*(e) + B/V
t=0
Holds for any e such
That:
e
>
>
0
emax
e
e
Take opposite extremes for e to optimize the bounds!
t-1
e
E{x(t)}
>
limsup 1
t
t
>
k
Uk
B + Vx*(e)
x*(e) + B/V
B + Vxmax
emax
x* + B/V
t=0
Holds for any e such
That:
e
>
>
0
emax
e
e
Take opposite extremes for e to optimize the bounds!
What is the policy?
k
Uk(t)E{Ck(I(t),S(t))| U(t)}+ VE{x(I(t))|U(t)}
Minimize RHS
Policy [Neely Infocom 05, IT 06]: Observe U(t), S(t)
Max:
k
Subject to:
Uk(t)Ck(I(t),S(t)) - Vx(I(t))
I(t)
I
m1(t)
A2(t)
V
m2(t)
Av Backlog
A1(t)
Av Power
Example Simulation:
Two-queue downlink
with {G, M, B} channels
Average Power
Multi-Hop Example Simulation:
Average Backlog
Lyapunov Optimization -- Virtual Queues:
l1
(i) Peak power constraint:
P(t) P
l2
L = Region of all supportable input rate vectors l
Consider Maximizing Throughput Subject to Peak and
Average Power Constraints.
Lyapunov Optimization -- Virtual Queues:
l1
(i) Peak power constraint:
P(t) P
(ii) Avg. power constraint:
l2
L = Region of all supportable input rate vectors l
Consider Maximizing Throughput Subject to Peak and
Average Power Constraints.
Lyapunov Optimization -- Virtual Queues:
l1
(i) Peak power constraint:
P(t) P
(ii) Avg. power constraint:
l2
L = Region of all supportable input rate vectors l
Consider Maximizing Throughput Subject to Peak and
Average Power Constraints.
To remove the average power constraint
,
we create a virtual power queue with backlog X(t).
L
Dynamics:
X(t+1) = max[X(t) - Pav, 0] +
A1(t)
A2(t)
i=1
Pi(t)
AL(t)
L
i=1
C1(I(t), S(t)) C2(I(t), S(t))
CK(I(t), S(t))
Pi(t)
Pav
Observation: If we stabilize all original queues and the virtual power
queue subject to only the peak power constraint P(t) P , then
the average power constraint will automatically be satisfied.
Control policy: [EECA, Neely IT 06] Show case when P restricts
power options to full power to one queue, or idle (general case in paper).
A1(t)
m1(t)
A2(t)
AL(t)
m2(t)
Choose queue i that maximizes:
mL(t)
Ui(t)mi(t) - X(t)Ptot
Whenever this maximum is positive. Else, allocate no power at all.
Then iterate the X(t) virtual power queue equation:
L
X(t+1) = max[X(t) - Pav, 0] +
i=1
Pi(t)
Performance of Energy Constrained Control Alg. (ECCA):
Theorem: Finite buffer size B, input rate l
L or l
L
(r1*,…, rL*) = optimal
vector
(r1, …, rL) = achieved
thruput vec.
L
(a) Thruput:
i=1
ri
L
i=1
ri*
- C/(B - Amax)
(b) Total power expended over any interval (t1, t2)
Pav(t2-t1) + Xmax
where C, Xmax are constants independent of rate vector and channel
statistics.
C = (Amax2 + Ppeak2 + Pav2)/2
Course Outline:
1. Intro and Motivation (done)
2. Magic of Telescoping Sums
3. Discrete Time Queues
4. Stability and Capacity Regions
5. Lyapunov Drift
6. Max-Weight Algorithms for Stability
(imperfect scheduling, robust to non-ergodic traffic)
(downlinks, backpressure, multi-hop, mobile)
Lyapunov Drift with Performance Optimization
8. Energy Problems
9. Virtual Queues and Power Constraints
10. General Constrained Optimization
11. Other Utilities: Flow control and Fairness
12. Modern Network Models
Let y(t) = -x(t) = Reward process!
Theorem: Let U(t) = Vector Queue Process. Let
L(U) be a non-negative function. Let x(t) > xmin be a
penalty process, let V>0. If for all t, U(t) we have:
>
D(U(t))+VE{x(t)|U(t)}
B - E{h(U(t))|U(t)} + Vx*
for some constant B, some non-neg function h(), and
some target penalty x*, then:
limsup 1
t
t
E{h(U(t))}
t=0
t-1
E{x(t)}
t=0
>
t
t
>
limsup 1
t-1
B + V(x* - xmin)
x* + B/V
Goal: Develop joint flow control, routing, resource allocation
wired network
wireless
sensor network
5
8
l91
l93
ln
(c)
0
6
9
QuickTime™ and a
TIFF (Uncompressed) decompressor
are needed to see this picture.
7
4
l1
1
2
3
l48
l42
Rn(c)(t)
l2
Un(c)(t)
gnc(rnc) = concave utility functions
util
L = Capacity region (considering all routing, resource alloc. policies)
r
Comparison of previous algorithms:
(1) MWM (max Uimi)
(2) Borst Alg. [Borst Infocom 2003] (max mi/mi)
(3) Tse Alg. [Tse 97, 99, Kush 2002] (max mi/ri)
wired network
wireless
sensor network
5
8
l91
l93
ln
(c)
0
6
9
QuickTime™ and a
TIFF (Uncompressed) decompressor
are needed to see this picture.
7
4
1
2
3
l48
l42
Rn(c)(t)
Un(c)(t)
Approach: Put all data in a reservoir before sending into
network. Reservoir valve determines Rn(c)(t) (amount
delivered to network from reservoir (n,c) at slot t).
Optimize dynamic decisions over all possible valve
control policies, network resource allocations, routing
to provide optimal fairness.
Part 1: Optimization with infinite demand
wired network
wireless
sensor network
5
8
l91
l93
ln
(c)
0
6
9
QuickTime™ and a
TIFF (Uncompressed) decompressor
are needed to see this picture.
7
4
1
l1
2
3
l48
l42
Rn(c)(t)
Un(c)(t)
Assume all active sessions infinitely backlogged
(general case of arbitrary traffic treated in part 2).
l2
Cross Layer Control Algorithm (CLC1):
(1) Flow Control: At node n, observe queue backlogs Un(c)(t)
for all active sessions c.
(c1)(t)
R
(c1)
n
ln
Rest of Network
(c2)(t)
R
(c2)
n
ln
Un(c)(t)
(where V is a parameter that affects network delay)
(2) Routing and Scheduling:
link l
cl*(t) =
(
(similar to the original Tassiulas differential backlog
routing policy [1992])
(3) Resource Allocation: Observe channel states S(t). Allocate
resources to yield rates m(t) such that:
Maximize:
l
Wl*(t)ml(t)
Such that:
m(t) GS(t)
Theorem: If channel states are i.i.d., then for any V>0
and any rate vector l (inside or outside of L),
l1
optimal point r *
msym
msym
Avg. delay:
Fairness:
(where
)
Special cases:
(for simplicity, assume only 1 active session per node)
1. Maximum throughput and the threshold rule
Linear utilities: gnc(r) = anc r
ln(c)
Rn(c)(t)
Un(c)(t)
(threshold structure similar to Tsibonis [Infocom 2003] for
a downlink with service envelopes)
(2) Proportional Fairness and the 1/U rule
logarithmic utilities: gnc(r) = log(1 + rnc)
ln
(c)
Rn(c)(t)
Un(c)(t)
Mechanism Design and Network Pricing:
greedy users…each naturally solves the following:
Maximize: gnc(r) - PRICEnc(t)r
Such that :
0
r
Rmax
This is exactly the same algorithm if we use the following
dynamic pricing strategy:
PRICEnc(t) = Unc(t)/V
Framework for Stochastic Network Optimization with
General Utilities/Costs, Penalties/Rewards: (Ch 5 of text)
[See also Stolyar 05 via fluid models]
U(t) = Vector queue backlog process
x(t) = Vector process of “penalties”
y(t) = Vector process of “rewards”
f(x) convex cost , g(y) concave utility
q(x) vector function, convex in each dimension
p(y) vector function, concave in each dimension
Framework for Stochastic Network Optimization with
General Utilities/Costs, Penalties/Rewards: (Ch 5 of text)
[See also Stolyar 05 via fluid models]
U(t) = Vector queue backlog process
x(t) = Vector process of “penalties”
y(t) = Vector process of “rewards”
Maximize: g( y ) - f( x )
q( x )
>
Subject to:
b1
p( y ) > b2
Queue Stability
Maximize: g( y ) - f( x )
q( x )
>
Subject to:
b1
p( y ) > b2
Queue Stability
Maximizing a function of a time average is different
from maximizing a time average of a function!
Technique: Virtual Queues: Introduce new constraint (and queue):
Auxiliary
variables
g(t)
Z(t)
y(t)
reward vector
g( g ) - f( x )
Maximize: g( y ) - f( x )
q( x )
>
Subject to:
b1
and
y > g
p( y ) > b2
Queue Stability
Maximizing a function of a time average is different
from maximizing a time average of a function!
Technique: Virtual Queues: Introduce new constraint (and queue):
Auxiliary
variables
g(t)
Z(t)
y(t)
reward vector
DIVBAR
source 1
source 2
source 3
-Multi-Node Wireless Network (possibly mobile)
-Operates in slotted time (t = 0, 1, 2, …)
-Broadcast Advantage [Wieselthier,Nguyen,Ephremides 99]
-Channel Errors
-Time Varying Transmission Success
Probabilities qab(t)
Example: Suppose Source 1 transmits…
a
d
b
1
source 1
c
source 2
source 3
-Multi-Node Wireless Network (possibly mobile)
-Operates in slotted time (t = 0, 1, 2, …)
-Broadcast Advantage [Wieselthier,Nguyen,Ephremides 99]
-Channel Errors
-Time Varying Transmission Success
Probabilities qab(t)
Example: Suppose Source 1 transmits…
There are 4 possible receivers…
a
d
b
1
source 1
c
source 2
source 3
-Multi-Node Wireless Network (possibly mobile)
-Operates in slotted time (t = 0, 1, 2, …)
-Broadcast Advantage [Wieselthier,Nguyen,Ephremides 99]
-Channel Errors
-Time Varying Transmission Success
Probabilities qab(t)
Example: Suppose Source 1 transmits…
Each with different success probs…
a
d
b
1
source 1
c
source 2
source 3
-Multi-Node Wireless Network (possibly mobile)
-Operates in slotted time (t = 0, 1, 2, …)
-Broadcast Advantage [Wieselthier,Nguyen,Ephremides 99]
-Channel Errors
-Time Varying Transmission Success
Probabilities qab(t)
Example: Suppose Source 1 transmits…
Only 3 successfully receive…
a
d
b
1
source 1
c
source 2
source 3
-Multi-Node Wireless Network (possibly mobile)
-Operates in slotted time (t = 0, 1, 2, …)
-Broadcast Advantage [Wieselthier,Nguyen,Ephremides 99]
-Channel Errors
-Time Varying Transmission Success
Probabilities qab(t)
Multi-Receiver Diversity!
a
d
b
1
source 1
c
source 2
source 3
Fundamental Questions:
1) How to Fully Utilize Multi-Receiver Diversity?
2) How to Maximize Throughput? Minimize Av. Power?
3) How to choose which node takes charge of the packet?
4) Should we allow redundant forwarding of different
copies of the same packet?
5) How to schedule multiple traffic streams?
GeRaF:
a
d
1
source 1
b
c
A Hot Topic Area:
Zorzi and Rao: “Geographic Random Forwarding”
(GeRaF) [IEEE Trans. on Mobile Computing, 2003].
Biswas and Morris: “Extremely Opportunistic Routing”
(EXOR) [Proc. of Sigcomm, 2005].
Baccelli, et. al. [IEEE Trans. Information Theory 2006]
GeRaF:
a
d
1
source 1
b
c
A Hot Topic Area:
Zorzi and Rao: “Geographic Random Forwarding”
(GeRaF) [IEEE Trans. on Mobile Computing, 2003].
“Closest-to-Destination” Heuristic
Biswas and Morris: “Extremely Opportunistic Routing”
(EXOR) [Proc. of Sigcomm, 2005].
Baccelli, et. al. [IEEE Trans. Information Theory 2006]
GeRaF:
a
d
1
source 1
e
b
c
A Hot Topic Area:
Zorzi and Rao: “Geographic Random Forwarding”
(GeRaF) [IEEE Trans. on Mobile Computing, 2003].
“Closest-to-Destination” Heuristic
Biswas and Morris: “Extremely Opportunistic Routing”
(EXOR) [Proc. of Sigcomm, 2005].
Baccelli, et. al. [IEEE Trans. Information Theory 2006]
possible deadlock
GeRaF:
a
d
1
source 1
e
b
c
A Hot Topic Area:
Zorzi and Rao: “Geographic Random Forwarding”
(GeRaF) [IEEE Trans. on Mobile Computing, 2003].
“Closest-to-Destination” Heuristic
Biswas and Morris: “Extremely Opportunistic Routing”
(EXOR) [Proc. of Sigcomm, 2005].
Baccelli, et. al. [IEEE Trans. Information Theory 2006]
EXOR:
h7
d
h6
1
h8
source 1
h
h1 h2 3
a
h5
h1
b
h4
h14
h16
h22
h
h21 h23 24
h
h19 h h20
h9 h11 15
19
h
17
h13
h18
c
h
12
h10
h25
A Hot Topic Area:
Zorzi and Rao: “Geographic Random Forwarding”
(GeRaF) [IEEE Trans. on Mobile Computing, 2003].
“Closest-to-Destination” Heuristic
Biswas and Morris: “Extremely Opportunistic Routing”
(EXOR) [Proc. of Sigcomm, 2005].
“Fewest Expected Hops to Destination” Heuristic
(using a traditional shortest path based on error probs)
How to achieve thruput and energy optimal routing?
A Big Challenge: Complexity!
Example: Suppose a node transmits a packet, and
there are k potential receivers…
1
2
3
k
Then there are 2k possible outcomes. An optimal
algorithm must specify a contingency plan for each
possible outcome.
How to achieve thruput and energy optimal routing?
A Big Challenge: Complexity!
Example: Suppose a node transmits a packet, and
there are k potential receivers…
1
2
3
k
Then there are 2k possible outcomes. An optimal
algorithm must specify a contingency plan for each
possible outcome.
How to achieve thruput and energy optimal routing?
A Big Challenge: Complexity!
Example: Suppose a node transmits a packet, and
there are k potential receivers…
1
2
3
k
Then there are 2k possible outcomes. An optimal
algorithm must specify a contingency plan for each
possible outcome.
How to achieve thruput and energy optimal routing?
A Big Challenge: Complexity!
Example: Suppose a node transmits a packet, and
there are k potential receivers…
1
2
3
k
Then there are 2k possible outcomes. An optimal
algorithm must specify a contingency plan for each
possible outcome.
How to achieve thruput and energy optimal routing?
A Big Challenge: Complexity!
Example: Suppose a node transmits a packet, and
there are k potential receivers…
1
2
3
k
Then there are 2k possible outcomes. An optimal
algorithm must specify a contingency plan for each
possible outcome.
How to achieve thruput and energy optimal routing?
A Big Challenge: Complexity!
Example: Suppose a node transmits a packet, and
there are k potential receivers…
1
2
3
k
Then there are 2k possible outcomes. An optimal
algorithm must specify a contingency plan for each
possible outcome.
1
2
3
k
Further Challenges:
1) How to Handle Multiple Simultaneous Transmissions?
2) How to Handle Multiple Traffic Sessions?
3) How to Handle Mobility and/or Time Varying
Channel Success Probabilities?
DIVBAR Results
ExOR
[Biswas, Morris 05]
DIVBAR, E-DIVBAR
[Neely, Urgaonkar
2006]
DIVBAR Results
ExOR
[Biswas, Morris 05]
DIVBAR, E-DIVBAR
[Neely, Urgaonkar
2006]
DIVBAR Results
ExOR
[Biswas, Morris 05]
DIVBAR, E-DIVBAR
[Neely, Urgaonkar
2006]
ExOR
[Biswas, Morris 05]
DIVBAR, E-DIVBAR
[Neely, Urgaonkar
2006]
The Optimality of Two Prices: Maximizing
Revenue in a Stochastic Network
AP
Larger Network
Profit
F(cost(t), S(t))
F(M(t),p(t))
Price
Allerton 2007
Longbo Huang
Michael J. Neely
*Sponsored in part
by the DARPA ITMANET Program
University of Southern California
The Revenue Maximization Problem
for an Access Point (AP)
AP
Users
Q(t)
Larger Net
Users: -Want to connect to Larger Net via AP
-Send Data to AP in reaction to price p(t)
AP:
-Wants to Maximize Time Average Revenue
-Sets a price p(t) every slot t
-Must transmit data (incurring its own cost)
Motivation: We want to solve a general stochastic
Max Revenue problem for wireless service providers
Previous Work: (Utility Optimization via “prices”)
-MacKie-Mason, Varian [JSAC 1995]
-Kelly [1997]
-Low, Lapsley [TON 1999]
Price = Lagrange Multiplier
(Used only to facillitate the utility optimization)
Motivation: We want to solve a general stochastic
Max Revenue problem for wireless service providers
Previous Work (Max Revenue):
[Paschalidis, Tsitsiklis: TON 2000]
[Acemoglu, Ozdaglar: CDC 2004]
[Basar, Srikant: INFOCOM 2002]
[Marbach, Berry: INFOCOM 2002]
[Keon, Anadalingam, TON 2003]
[Zhang, Dey, Tan, INFORMS 2007]
Dynamic Programming
Downlinks
Static
Optimization
Max Revenue
with Qos
[Musacchio, Walrand: TON 2006]
[Lam, Chiu, Lui: Trans. On Computers 2007]
AP pricing
Game
Theory
Motivation: We want to solve a general stochastic
Max Revenue problem for wireless service providers
Previous Work (Max Revenue):
[Paschalidis, Tsitsiklis: TON 2000]
[Acemoglu, Ozdaglar: CDC 2004]
[Basar, Srikant: INFOCOM 2002]
[Marbach, Berry: INFOCOM 2002]
[Keon, Anadalingam, TON 2003]
[Zhang, Dey, Tan, INFORMS 2007]
Dynamic Programming
Downlinks
Static
Optimization
Max Revenue
with Qos
[Musacchio, Walrand: TON 2006]
[Lam, Chiu, Lui: Trans. On Computers 2007]
Typical Prior Assumptions:
(1) Time invariant system
(2) Concave Utility/Demand
function
(3) ‘‘One shot”,No queueing
AP pricing
However, we have:
(1) Stochastic Wireless Net.
(2) General (non-concave)
Demand Function
(3) Queueing
Game
Theory
Need new models &
new algorithms !!
Network Model:
R(t)
Q(t)
Time Slotted System: t = {0, 1, 2, ….}
User “Demand State” M(t):
-Models “willingness-to-pay”
-Examples: i) M(t) = # users at time t , ii) M(t) = {LOW, MED, HIGH}
User “Demand Function” F(M(t), p(t)):
F: (M(t), p(t))
E{R(t)} = Expected Arriving Data
in reaction to M(t), p(t)
Channel State S(t) and rate-cost function:
F(cost(t), S(t)) = m(t) = Transmission rate (bits/slot)
R(t)
Q(t)
Assumptions on Demand State M(t):
1. M(t) is Markovian: finite state space, ergodic
2. AP knows current value of M(t) on each slot t
3. AP does not necessarily know transition
or steady state probabilities
M(t)
4. R(t) = Random Arrivals, E{R(t)} = F(M(t), p(t)) , R(t) < Rmax
R(t) is conditionally indep of past given M(t), p(t)
R(t) = Random Arrivals,
E{R(t)} = F(M(t), p(t))
R(t)
Q(t)
-F(M, p) is continuous over p for all M
(possibly non-linear, non-concave)
-p(t)
P = Compact Set of Feasible
Prices (possibly discrete)
-Assume 0 < p(t) < pmax
E[R(t)]
Assumptions on Demand Function F(M(t), p(t)):
Price
-AP knows Demand Function F(m, p)
Example: If M(t) = # users, then F(M(t), p(t)) might be M(t)F(p(t))
R(t)
Q(t)
Assumptions on S(t) and Rate-Cost Function:
-S(t) is Markovian (finite state space, ergodic)
-AP knows current value of S(t)
-AP does not necessarily know transition probabilities
or steady state probabilities.
-m(t) = F(cost(t), S(t)) (general, continuous)
cost(t)
C = Compact Set
Assume there is a max transmission rate mmax.
F(cost(t),S(t))
Cost(t)
If the AP wants to admit new data, it gives an “OPEN
MARKET” signal (Z(t) = 1). It chooses a price p(t) and
informs the users. Users react by sending packets.
Demand Function:
E[R(t)] =F(M(t),P(t))
Price $P(t)
AP
E[R(t)]
Larger Network
Arrival R(t)
Business Decision: Z(t)=1
Price
If the AP doesn’t want to admit new data, it sends a “CLOSED”
signal (Z(t)=0). Users will NOT send any packets.
Demand Function:
E[R(t)] =F(M(t),P(t))
CLOSED
AP
E[R(t)]
Mesh Network
Arrival 0
Business Decision: Z(t)=0
Price
-The AP queues the data (Q(t) = backlog).
-Observes channel state S(t).
-Makes Resource Allocation decision cost(t) C.
-Transmission rate = m(t) = F(cost(t), S(t))
Z(t)R(t)
F(cost(t),S(t))
Q(t)
Partially controlled Arrival Rate
Controlled Departure Rate
F(cost(t),S(t))
Instantaneous Profit:
Profit(t) = Z(t)R(t)p(t) - cost(t)
Goal: Maximize Time Average Profit
while maintaining queue stability
Cost(t)
The optimum avg. profit can be obtained with exact
steady state distribution information for M(t) and S(t)
Theorem: (Max Profit with Stability)
The max average profit Profit* is the solution to the max problem
max
s.t.
Pr ofit = Incomeav - Cost av
Immediate corollary: A stationary
randomized STAT* algorithm achieves
 (m)  (m)
(m)
(m) 
Incomeav = Em   a k F (m, pk ) pk  the optimum avg. profit.
k =1


  (s)

Cost av = ES   k cos t k( s ) 
 k =1

 (m)  (m)
  (s)

(m) 
lav = Em   a k F (m, pk )  m av = ES   k F(cos t k( s ) , S )
k =1


 k =1

0   (m)  1
0  pk( m )  pmax k , m 0  cos t k( s )  Cmax k , s

a
k =1
(m)
k
= 1m


k =1
(s)
k
= 1s
Proved by using
Caratheodory’s thm
The Optimality of Two Prices Theorem
Theorem:
l*= avg. rate of STAT*, Income*=avg. income of STAT*
For each M(t)=m, exists 2 business-price points (Z1(m), p1(m)),
(Z2(m), p2(m)), and probabilities q1(m) & q2(m) s.t.
l*= Em[q1(m) F(m, p1(m)) + q2(m) F(m, p2(m))]
Income*=Em[q1(m) p1(m) F(m, p1(m)) + q2(m) p2(m) F(m, p2(m))]
Proof :
Caratheodory’s thm + its extension
+ Continuity of F(m,p)
For any F(m,p) demand function, AP
can maximize its revenue by using
two prices per M(t)
pF(m,p)
(0,0)
F(m,p)
Proof Sketch for special case when:
-price set P is topologically connected (ex: 0 < p < pmax)
-only one M(t) state, so F(M(t), p(t)) = F(p(t))
Stationary Randomized Algorithm from Theorem 1 yields:
l* = E{Z(t)F(p(t))}
Income* = E{Z(t)p(t)F(p(t))}
Thus, point (l* , Income*) is in the convex hull of the 2-d set:
W = { ( ZF(p) ; ZpF(p) ) | Z in {0, 1}, p in P}
Thus, (l* , Income*) can be achieved by a convex combination
of at most 3 points in W. But…
The point (l* , Income*) is in the convex hull of the 2-d set:
W = { ( ZF(p) ; ZpF(p) ) | Z in {0, 1}, p in P}
Extension to Caratheodory’s Theorem (Hanner, Radstrom 1951):
If W is the union of at most two connected components, then any point
in the convex hull can be achieved by a convex combination of at
most 2 points in W.
The point (l* , Income*) is in the convex hull of the 2-d set:
W = { ( ZF(p) ; ZpF(p) ) | Z in {0, 1}, p in P}
Extension to Caratheodory’s Theorem (Hanner, Radstrom 1951):
If W is the union of at most two connected components, then any point
in the convex hull can be achieved by a convex combination of at
most 2 points in W.
The point (l* , Income*) is in the convex hull of the 2-d set:
W = { ( ZF(p) ; ZpF(p) ) | Z in {0, 1}, p in P}
Extension to Caratheodory’s Theorem (Hanner, Radstrom 1951):
If W is the union of at most two connected components, then any point
in the convex hull can be achieved by a convex combination of at
most 2 points in W.
The point (l* , Income*) is in the convex hull of the 2-d set:
W = { ( ZF(p) ; ZpF(p) ) | Z in {0, 1}, p in P}
W = {(0, 0)} U {( F(p) ; pF(p)) | p in P}
Image of a connected set through a continuous function is connected!
This example shows that using a single price does NOT always lead to
optimal average profit, thus the number Two is tight
Q(t)
Constant
m=2.28

4
0  p 1


F ( p) =  - 6 p + 10 1 < p  1.5
 2
20 1.5 < p  10
- 17 p + 17
Single price with constant Z(t)=1
P*=5 at A=(5,50/17)
ProfitSingle= 50/17≈2.9412
lA = 10/17 ≈0.5882<<2.28 !!
Single price with varying Z(t)
Profitsingle≤3.42
A=(5,50/17) B=(1.2867,2.9285) C=(1.2867,2.28)
This example shows that using a single price does NOT always lead to
optimal average profit, thus the number Two is tight
Q(t)
Constant
m = 2.28
A choice of 2 prices: p1=31/30 &
p2=5, each with prob. 0.5
ProfitTwo= 0.5F(p1)p1+0.5F(p2)p2
= 0.5*(3.8*31/30+5*10/17)
≈ 3.4339
> ProfitSingle
lA
= 0.5F(p1)+0.5F(p2)
= 0.5*(3.8+10/17)
≈ 2.1941<2.28
Single price not always enough.
Need Two prices!
A=(5,50/17) B=(1.2867,2.9285) C=(1.2867,2.28)
This example shows that using a single price does NOT always lead to
optimal average profit, thus the number Two is tight
Q(t)
Constant
m = 2.28
A choice of 2 prices: p1=31/30 &
p2=5, each with prob. 0.5
ProfitTwo= 0.5F(p1)p1+0.5F(p2)p2
= 0.5*(3.8*31/30+5*10/17)
≈ 3.4339
> ProfitSingle
lA
= 0.5F(p1)+0.5F(p2)
= 0.5*(3.8+10/17)
≈ 2.1941<2.28
Using p1=1 & p2=5.15,
max Profitav= 3.4676
A=(5,50/17) B=(1.2867,2.9285) C=(1.2867,2.28)
Our dynamic Pricing and Transmission Scheduling Algorithm
PTSA achieves the optimum average profit
PTSA: For a control parameter V (affects a delay tradeoff)…
Admission Control: Observe M(t) & Q(t), choose
P(t)=argmax p in P [ VpF(M(t),p)-2Q(t)F(M(t),p) ]
Whenever the maximum is positive, Z(t)=1,announce P(t);
Else Z(t)=0,send “CLOSED”
Cost/Transmission: Observe S(t) & Q(t), choose
Cost(t)=argmax cost in C [2Q(t)F(cost,S(t)) - Vcost(t)]
Our dynamic Pricing and Transmission Scheduling Algorithm
PTSA achieves the optimum average profit
PTSA: For a control parameter V (affects a delay tradeoff)…
Admission Control: Observe M(t) & Q(t), choose’
P(t)=argmax p in P [ VpF(M(t),p)-2Q(t)F(M(t),p) ]
Whenever the maximum is positive, Z(t)=1,announce P(t);
Else Z(t)=0,send “CLOSED”
Note: For F(m,p)=mF(p), the AP
can choose p(t) without info of M(t)
Demand-Blind Pricing!
Performance of the joint Pricing and Transmission Scheduling
Algorithm (PTSA)
Theorem:
Backlog Bound: Q(t) ≤Qmax = VPmax+Rmax
Achieved Average Profit: ProfitPTSA≥ Profit*- O(logV/V)
Profit*
Qmax
V
Proof Technique: Use General Framework for
Stochastic Network Optimization developed in:
L. Georgiadis , M. J. Neely, and L. Tassiulas,
“Resource Allocation and Cross-Layer Control in Wireless
Networks,” Foundations and Trends in Networking,
Vol. 1, no. 1, pp.1-144, 2006.
Available via free download at : http://www-rcf.usc.edu/~mjneely/www
or on NOW Publishers Webpage as free issue (vol. 1 in series).
Plug for my UCSD Talk:
ITA “Short Course” on stochastic network optimization:
Friday, October 5, 2007
CAL-IT2
9am-5pm
http://ita.ucsd.edu/events.php
Simulation setup of PTSA
Recall:

4
0  p 1


F ( p) =  - 6 p + 10 1 < p  1.5
 2
20 1.5 < p  10
p
+
 17
17
4
F(p)
0
price
10
Constant F(cost(t),S(t))=2.28 with no cost; constant M(t)=m
Arrival: (1) R(t) = F(m,p(t)) – deterministic case
(2) R(t) = 2F(m,p(t)) with prob. ½ Bernoulli
0
else
-random case
Simulation results of PTSA
average queue backlog
Backlog
Avg. Profit
Simulation results of PTSA
Prices chosen by PTSA: “Two-value” phenomenon
More simulation results of PTSA
Setup: Two Demand states, {High, Low}
Constant service rate 2.28 with zero cost

4
0  p 1


F LOW ( p ) =  - 6 p + 10
1 < p  1.5
 2
20 1.5 < p  10
p
+

17
 17

0 p2
 10 - p

F High ( p) =  - 6 p + 20 2 < p  3
 1
17 3 < p  10
p
+

7
 7
0.4
0.6
Low
High
0.4
0.6
More simulation results of PTSA
More simulation results of PTSA
Prices chosen when M(t)=Low
Prices chosen when M(t)=High
Aggregate rate 0.2704
Aggregate income 1.4601
Aggregate rate 2.0096
Aggregate income 6.3508
PTSA finds the optimum share
This work characterizes the max avg. profit of APs in wireless mesh
nets and offers a revenue maximizing algorithm
The max avg. profit &‘‘The
Optimality of Two Prices”
Theorem
Joint Pricing and Transmission
Scheduling Algorithm (PTSA)
Questions or Comments?