Lecture 3
Preview of Markov Process
• A sequence of random variables X1 , X2,….,Xn
,….. such that
– Xi+1 is independent of X1,….Xi-1 given Xi
– Pr(Xi+1 = ai+1/ Xi = ai, …., X1 = a1) = Pr(Xi+1 = ai+1/ Xi =
ai )
• Discrete time discrete state markov chain
• So a markov chain evolution can be specified by
– Initial states
– Transition probabilities Pr(Xi+1 = ai+1/ Xi = ai ) = pi,i+1
0.5
1
0
2
1.0
0.5
0.5
0.5
State A communicates with state B if there is a positive probability
path from A to B
A set of states is closed if all states in the set communicate
with each other, and no state in the set communicates to any
state outside the set, e.g., {0, 1}
A state a is open if it communicates to some other state
which does not communicate to a, e.g., {2}
A state has a period d if the process can only return to this state
in intervals which are multiples of d
More precisely, di is the g.c.d. of {k: pi ik > 0},
Where pi ik is Pr(Xk = i/ X0 = i)
0.5
1
0
d0= 1
0.5
0.5
0.5
1.0
1
0
1.0
d0 = 2
Period of any two communicating states are equal
A state is aperiodic if its period is 1
Consider the wireless system
Let the system have random arrivals, i.e., the
number of arrivals for the different sessions is
random
Arrival process for each session is i.i.d (independent and
identically distributed)
Number of arrivals of session i in slot t is independent of
the number of arrivals of session i or any other session in
past or future slots, and also independent of number of
arrivals of any other session in the same slot
Probability that a packet arrives for session i in any slot t is
qi
Under this assumption the queue length process at the nodes
(values of the Biu(t) s for all sessions i and nodes u) constitute a
markov process
Session 1
u
e1
v
e2
x
e3
y
Session 2
(B1u(t) B1v(t) B1x(t) B2v(t) B2x(t) B2y(t) ) is a markov chain
This markov chain consists of one closed set, and some
other open states (prove it!)
Periodicity of the closed set is 1 (prove it!) (under a minor
assumption on the values of the arrival probabilities, qi s)
How many states does this markov chain have?
If the system is markov with an aperiodic closed set, and
other open states, and
Suppose we can find a scalar function V(x) of vector x such
that E(V(B(t + 1))/ V(B(t) = x)) 0 for all states x, except a
finite number of states and V(x) 0 for all states x,
Then the probability distribution of the markov chain
converges to a probability distribution with finite expectation
This means the system stabilizes!
The function V(x) we choose is icixi2
Problem Complexity
Most optimization problems can be expressed as
decision problems (yes/no answer)
For example, shortest path problem can be expressed as, is there
a path with weight less than k in a graph G?
P is the class of decision problems which can be decided in
polynomial time
Language: has certain membership (a, b, c
belongs to English)
Language L = { (G, k): G has spanning tree of
weight at most k}
A language is in P if its membership can be tested
in polynomial complexity
How about {(G, k): G has an independent set of size
at least k}?
A language is in NP (non-deterministic polynomial
complexity) if given a member of the language, and a
certificate to prove membership, we can verify the
certificate in polynomial complexity
Both previous examples are in NP
How about {G: G has a unique hamiltonian cycle}?
A language is NP-hard, if it can be solved in
polynomial complexity, then all other
languages in NP can also be solved in
polynomial complexity,
i.e., any other language in NP can be
transformed in to this language in polynomial
complexity
Note that a NP-hard language does not necessarily
belong to NP.
Example: {(G, k): G does not have a path of weight
higher than k}.
A language is NP-complete if it is in NP and is NP-hard
NP-complete
P
Is P = NP?
probably not
NP
NP-hard
Technique for proving that a problem is NP-hard
There are already problems proved to be NP-hard. Show
that if you can solve this problem (problem we are trying to
prove NP-hard) in polynomial complexity, you can solve
such a known problem in polynomial complexity using the
solution of this problem.
We will do such a reduction.
Quality of Service Routing
• Quality of Service
– Delay
– Packet loss
• Packet loss
– Transmission error
– Buffer overflow
• Delay consists of
– Queueing delay
– Transmission delay
– Propagation delay
• First two can be related to bandwidth
• If we guarantee a certain bandwidth for a session
on each of its links, its queueing and transmission
delay can be upper-bounded
• Congestion related packet loss can also be upperbounded for certain guaranteed bandwidth
• Using Shannons channel capacity theorem
transmission error related packet loss can also be
upper-bounded
So quality of service requirement translates to bandwidth
requirement in every link
Given a network with finite bandwidths on the links, and a
connection with a certain bandwidth requirement, find a
path which gives that bandwidth to the session in every
link,
If there is no such path detect the condition
Path 1
Source
1
1.5
0.5
2.5
Path 2
Destination
Generalization
There is a reward associated with every link
Reward of a path is the minimum reward on a link
Find a path with required reward
Pseudo-code
Delete any link which has bandwidth less than the
required amount
Find any path between the source and the destination in
the remaining link
First step has complexity O(E)
Using depth first search the second step has complexity
O(V + E)
Overall complexity O(V + E)
Find an ``efficient’’ algorithm which gives the path with
the highest reward
Offline QoS Routing
We have a set of connection requests, each with a
source, destination and a bandwidth requirement
Is it possible to satisfy all the requests?
Sum of the bandwidths used by connections in a
link should not exceed link capacity
NP-complete
Polynomial complexity computable if we are
allowed to split paths for a single connection
If we are not allowed to split paths, then need to study
approximation algorithms
heuristics
Widest path heuristic
Initially residual bandwidth of a link is the
entire link bandwidth
Route a connection through the largest reward
path
Find the residual bandwidth in each link of the network
Residual bandwidth Residual bandwidth – bandwidth used
by the connection
Route another connection
If a connection can not be routed, backtrack.
Re-route the previous connection through the ``second
best`` path, and now try routing this connection.
Limit the number of backtracks
If the number of allowed backtracks increase,
Computation time increases
Better results are expected
A more Efficient Heuristic
Start with the most difficult demand
Demand is high
All routes for the demand have low
bandwidths
Why should we start with the most difficult task?
Choose a good route for the demand
Go to the next most difficult demand
Backtrack a limited number of times, whenever demand can
not be routed.
Abstraction and Constraint Satisfaction techniques
for planning bandwidth allocation
Christian Frei (Swiss Federal Institute of
Technology (EPFL)), Boi Faltings (Swiss Federal
Institute of Technology (EPFL)) ,
INFOCOM 2000
A group of nodes constitute a -blocking island, if any
node in the set is connected to another in the set by path
of bandwidth at least .
a
112
b
128
d
j
96
20
N4
256
16
c 22
58
i
68
10
N3
28
e
k
76
f
60
N1
N4
64-blocking islands
g
9
118
N2
N5
56-blocking islands
16
58
22
10
N3
28
N1
h
N2
60
N6
N4
N5
16-blocking islands
16
58
22
10
N3
28
N2
60
N1
N6
N7
If two nodes are in a -blocking island, then there is a
path of bandwidth  between them.
N7
16
N6
N5
56
64
(b, c) most difficult
8
N1
64
a b c d e
N2
N3
N4
i j
k
32
f g h
If the lowest common ancestor of two end points is level , then any path for
routing the demand can not have more than level +1 bandwidth. So higher the
lowest common ancestor, more difficult is the demand.
However, this does not consider the amount of the demand
The most difficult demand is the one with the least difference
between the least common ancestor bandwidth and the demand
amount
Form the -blocking islands for all bandwidth levels 
Start with  = highest value of the bandwidth
Choose any vertex, add all vertices connected by an edge
of b.w.  or higher to this vertex, further add all vertices
similarly connected to the new vertices and so on. When
you can not add any more, you end up with a -blocking
island.
Start with a vertex which is not in any -blocking island,
and repeat the same till all vertices are exhausted.
Repeat the entire procedure with  = highest value of
the bandwidth and so on.
Choose the most difficult demand first
The best route for the demand is the one that lies
completely in the lowest level blocking island which
consists of the two end points.
This is because connectivity between any two nodes
in a -blocking island is at least .
Choose the next most difficult demand
Backtrack if necessary
Artificial intelligence based technique.
Online Routing of Demands
• Route demands as and when they arrive
– Demands may be blocked
• Previous technique works well when all demands
are known in advance.
• You can still use this for online routing, but you
may need to reroute a lot of existing connections
as future connections come in
• Any good online strategy must use some
information about future demands
– For example potential source destinations may be
known
Minimum Interference Routing with
application to MPLS traffic engineering
Murali Kodialam (Bell Labs, Lucent
Technologies), T. V. Lakshman (Bell Labs,
Lucent Technologies)
INFOCOM 2000
There are a few source-destination pairs in the
network.
Demands are multiples of some constants
When a demand arrives, route it through a path which
has a lot of residual bandwidth , and which
Does not overlap a lot with routes between other
source destination pairs
Path 1
Source 2
Source 1
Destination 1
Path 2
Destination 2
The objective is to reduce blockings of potential future
demands as much as possible.
Network flows
Given a network and a source and a destination find the maximum
traffic the source can send to the destination, while not exceeding
the link bandwidths, and using as many routes as it wishes to.
Path 1
Source
1
0.5
1.5
2.5
Path 2
Destination
Max flow = 1.5
A valid flow is a traffic the source can send to the destination,
while not exceeding the link bandwidths, and using as many
routes as it wishes to.
Flow  Max Flow
A cut is a partition of the vertex set into two sets,
A and B such that A and B do not overlap
A contains the source
Path 1
a
B contains the destination
Source
For blue cut, A = {source}, B =
{destination, a , b}
For red cut, A = {source, b}, B =
{destination, a }
1
0.5
1.5
2.5
b
Path 2
Destination
Capacity of a cut is the total bandwidth of the edges who have
one end in set A and the other end in set B
(edges which cross the cut)
Path 1
a
Source
1
0.5
1.5
2.5
b
Path 2
Capacity of the red cut = 3.5
Capacity of the blue cut is 1.5
Destination
Any flow is upper bounded by the capacity of any cut
This is because flow is the amount of traffic you are
sending from source to destination,
Source is in one set of the cut
Destination is in the other set of the cut
The two sets of the cuts do not overlap
So any flow from the source to destination must ``flow”
across the cut, and hence can not exceed the cut capacity
Max flow from source to destination  Min.
cut capacity
Max flow from source to destination = Min. cut capacity
This means if we reduce the bandwidths of any link
in any min. cut even slightly, maximum flow across
the source to destination is reduced.
Whereas this does not hold for other edges in the
network. We can always find a small amount such
that reducing the link capacities of the edges not in
any min. cut by that amount, does not hurt the max.
flow
Edges in the min. cuts are critical to ``connectivity’’ of
the source and the destination.
Any edge in any min. cut will be denoted as a ``critical
edge.’’
Incidentally, max flow and all min. cuts are polynomial
complexity computable.
If link bandwidths are integers, then max flow
allocation algorithm assigns integer flows to all links
So, max flow with the constraint that flow in any link
must be integer is polynomial complexity computable.
However, given n source destination pairs, and a flow
vector (f1, ….fn) (fi is the flow for session i) , the
question whether we can route this flow vector without
violating the capacity constraints, using only integer
flows in the links, and as many routes as we wish to, is
NP-hard!
Interference Measure
Interference caused by a path is measured by computing the
max flow in the residual capacity graph between all source
destination pairs.
Let demand B be routed along path p
Look at the residual capacity graph after routing the
demand along p (subtract bandwidth B from all edges in p)
Find the maximum flow between each source destination
pair in the residual capacity graph.
Consider the minimum value of these individual
maxflows. This is the interference caused by the path p
Another measure of the interference can be the
lexicographic ordering of these maxflows
Another measure is the weighted sum of these
maxflows.
Choose the path p which maximizes the interference
measure
Higher the interference measure, less is the interference
The problem is NP-hard
heuristics
MIRA
While routing a new demand, one should avoid the
critical edges of other source destination pairs as
much as possible.
So when a demand arrives, look at the network with the
residual capacities.
Find the max. flow of each source destination pair
separately (polynomial complexity)
Find the critical edges for every source destination pair
(polynomial edges)
Remove edges with residual capacity < demand
Give weights to all the critical edges of all the source
destination pairs.
Weights to edges which are not critical for any
source destination will be 0 or very small positive
numbers
Use Dijkstra to find the minimum weight path for the new
demand.
Note that the minimum weight path will have the requisite
bandwidth. Why?
The min. weight path would try to avoid critical edges as
much as possible?
Do we gain anything by giving small nonzero weights to the
non-critical edges instead of 0 weights?
Weights for critical edges could be same or different for
different source destination.
For example, weight could be largest for the most
valued flow. This would avoid using the critical edges
of the most valued flow by other flows as much as
possible, and reduce the blocking for the most valued
flow.
Weight could be the largest for the source destination pair with
least max. flow in the residual capacity graph (this has the largest
blocking possibility, so its critical edges should be preserved the
most).
Weight could be in lexicographic order of max. flow values,
the one with least value of max flow gets much larger
weight for its critical edges than others, the one with with
second min. value of max flow gets second largest weight
for its critical edges (much larger than the others), etc….
How do we compute all the critical edges in the graph?
• Typically, routing decisions are not taken for every
connection, but at intervals.
• For this case, lexicographic weighing of edges
performs better.
• Also, we should have non-negligible weights for
edges which are not critical, but ``close to
critical.’’
– The capacities in these edges allow increasing flows,
but only by a small amount 
Minimum Interference Routing with
application to MPLS traffic engineering
Koushik Kar (University of Maryland),
Murali Kodialam (Bell Labs, Lucent
Technologies), T. V. Lakshman (Bell Labs,
Lucent Technologies)
JSAC Nov. 2000