Network Analysis
A brief introduction on queues, delays, and tokens
Lin Gu, lingu@acm.org
Computer Networking: A Top Down Approach 6th edition. Jim Kurose and
Keith Ross,. Part of the slides are adapted from course companion materials.
Queueing theory



How long is the queue? How much time will I wait?
A branch of applied probability theory
Applications in








Telecommunications
Traffic control
Predicting computer performance
Health services (e.g. control of hospital bed assignments)
Airport traffic, airline ticket sales
Layout of manufacturing systems.
Communication networks
Why do we want to know the characteristics of queues?
Queues and queueing

Oct. 16, 2010: 1.03 million
visitors in World Expo., Shanghai

If you are one of them, you wonder




How many people ahead of me?
How soon can I get into the pavilion?
8 hours? 12 hours?
Ultimately, organizer starts to
persuade visitors to leave (drop
packets)
Every second, millions of packets
flow into a switch/router

Would you be able to tell



How long is the queue?
How much is the delay?
Would you like to drop packets?
Queuing theory for studying networks
View network as collections of queues


FIFO data-structures
Queuing theory provides probabilistic analysis of these
queues
Examples:






4
Average length
Average waiting time
Probability queue is at a certain length
Probability a packet will be lost
Little’s Law
System
Arrivals

Little’s Law:
Mean number tasks in system = mean arrival rate x mean
response time


Departures
Observed before, Little was first to prove
Applies to any system in equilibrium, as long as nothing in
black box is creating or destroying tasks
5
Proving Little’s Law
Arrivals
Packet 3
#
2
1
# in 3
System 2
1
Departures
1 2 3 4 5 6 7 8
Time
J = Shaded area = 9
Time in 3
System 2
1
6
1 2 3 4 5 6 7 8
Time
Same in all cases!
1 2 3
Packet #
Definitions
J: “Area” from previous slide
N: Number of jobs (packets)
T: Total time
l: Average arrival rate





N/T
W: Average time job is in the system


= J/N
L: Average number of jobs in the system


7
= J/T
The Little’s Law
# in 3
System 2
(L) 1
=
Time in 3
System 2
(W)
1
1 2 3 4 5 6 7 8
Time (T)
J  TL  NW
L  ( TN )W
L  (l )W
8
1 2 3
Packet # (N)
Model Queuing System
Queuing System
Queue
Queuing System
Server
Server System
Use Queuing models to



9
Describe the behavior of queuing systems
Evaluate system performance
Characteristics of a queueing model

Arrival process: the sequence of requests for service,
often specified in terms of inter-arrival time


Service mechanism: # of servers and service time


The distribution that determines how the tasks arrive in the system.
The distribution that determines the task processing time
Queue discipline: disposition of blocked customers
(customers who find all servers busy)
Kendall notation







A/B/m/N –S
A – distribution of inter-arrival time
B – distribution of service time
m – number of servers
N – max capacity (∞ if omitted)
S – queue discipline (FIFO if omitted)
Distributions





M: stands for "Markovian", implying exponential distribution for
service times or inter-arrival times.
D: Deterministic (e.g. fixed constant)
Ek: Erlang with parameter k
Hk: Hyperexponential with parameter k
G: General (anything)
Kendall Notation Examples

M/M/1:



M/M/m


Poisson arrivals and exponential service, 1 server, infinite capacity
(also assumed infinite population), FCFS (FIFO)
The simplest ‘realistic’ queue
Same, but m servers
G/G/3/20

12
General arrival and service distributions, 3 servers, 17 queue slots
(20-3)
Poisson Process
For a Poisson process with average arrival rate l , the
probability of seeing n arrivals in time interval  t
e  lt (lt ) n
Pr( n) 
n!
E (n)  lt
(lt ) 2
Pr(0)  e
 1  lt 
...  1  lt  o(t )  Pr(0)  1  lt
2!
2
(
l

t
)
Pr(1)  ltelt  lt[1  lt 
...]  lt  o(t )  Pr(1)  lt
2!
Pr(  2)  ...  0
 lt
13
Poisson process & exponential distribution

Inter-arrival time t (time between arrivals) in a Poisson
process follows exponential distribution with
parameter l
Pr(t )  le
E (t ) 
14
1
l
 lt
Analysis of M/M/1 queue

Given:
•
•

l: Arrival rate of jobs (e.g., packets on input link)
m: Service rate of the server (e.g., packets on output link)
Solve:




15
L: average number of jobs in the queuing system
Lq : average number of jobs in the queue
W: average waiting time in whole system
Wq: average waiting time in the queue
M/M/1 queue model
L
Lq
l
m
1
m
Wq
W
16
Solving queuing systems


4 unknowns: L, Lq W, Wq
Relationships:





L=lW
Lq=lWq (steady-state argument)
W = Wq + (1/m)
If we know L or Lq, we can find the others
Finding L can be hard.

In general:
P
 n


L   nPn
n 0
: the probability that the number of items in the system is n
Can we find a closed form expression of Pn and L , given
only l and m?
17
Equilibrium conditions
Define Pn (t ) to be the probability of having n items in the system at time t
l
l
n-1
m
l
n
m
l
n+1
m
m
P0 (t  t )  P0 (t )[(1  lt )  mtlt ]  P1 (t )[( mt )(1  lt )]
Pn (t  t )  Pn (t )[(1  mt )(1  lt )  mtlt ]  Pn 1 (t )[( mt )(1  lt )]  Pn 1 (t )[( lt )(1  mt )]
P0 (t  t )  P0 (t )
 lP0 (t )  mP1 (t )
t
Pn (t  t )  Pn (t )
 lPn 1 (t )  (l  m ) Pn (t )  mPn 1 (t )
t
For it to stablize, we assume
18
Pn (t  t )  Pn (t )
0
t  
t
l  m , lim Pn (t )  Pn , lim
t  
Equilibrium conditions
lP0  mP1
(l  m ) Pn  lPn 1  mPn 1
2
P1 
n-1
n
l
l
l
P0 , P2    P0, Pn    P0
m
m
m

l
ρ  , then
m
P0 
1

n
ρ

n 0
19
n
l
n
m
m
n
l
P

1
,
then
P

n
0 
 m   1,  P0 
n 0
n 0 


l
l
1
l
 

n 0  m 

n


l
1

ρ
1
n



 m    ρ  1  ρ  1  ρ ρ  1
n 0 
n 0


 1  ρ and Pn  ρ n 1  ρ 
l
n+1
m
m
Solving for L



n 0
n 0

n 0
L   nPn   n n (1   )  (1   ) n n
L  (1   ) n n 1
n 0



L  L  (1   )(  n   (n  1)  )  (1   )(     )
n
n 0
 
L(1   )  (1   )
L
20

(1  )
 mll

1 
n
n2
n
n2
Solving W, Wq and Lq
   
 W       
W
Wq
L
l

l
m l
1
m
Lq  lWq  l
21
1
1
m l
l
l
1
m l
m
l
m ( m l )

l
m ( m l )
l2
m ( m l )
Response Time vs. Utilization
Waiting vs. Utilization
0.25
W(sec)
0.2
0.15
0.1
0.05
0
0
0.2
0.4
0.6
 %
W
22
1
m l
0.8
1
1.2
Summary of M/M/1


The simplest ‘realistic’ queue: Poisson
arrivals and exponential service, 1 server,
infinite capacity (also assumed infinite
population), FCFS (FIFO)
Variables:
•
l: Arrival rate of jobs (packets on input link)
•
m: Service rate of the server (output link)
For the queue to stablize, we assume
•
Pn  ρ 1  ρ 
n
E (n)  lt
Pr(1)  lt
l
m
l  m , lim Pn (t )  Pn , lim
t 
L: average number of jobs in the queuing system
P0  1  ρ
23
ρ
e  lt (lt ) n
Pr( n) 
n!
Pr(0)  1  lt
t 

L   nPn 
n 0

(1  )
•
Lq : average number of jobs in the queue
Lq  lWq  m ( ml l )
•
W: average waiting time in whole system
W
•
Wq: average waiting time in the queue
Wq  W  m1 
Pn (t  t )  Pn (t )
0
t

m l
2
L
l

l
1
m l
l
m ( m l )
Example

On a network gateway, measurements show that the
packets arrive at a mean rate of 125 packets per second
(pps) and the gateway takes about 2 ms to forward a
packet. Assuming an M/M/1 model, what is the probability
of buffer overflow if the gateway had only 13 buffer units?
How many buffer units are needed to keep packet loss
below one packet per million?



mean arrival rate (l): 125 packets/s
mean response time (1/m): 2 ms
Assuming M/M/1:




24
What is the gateway’s utilization?
What is the probability of n packets queued in the gateway?
What is the mean number of packets queued in the gateway?
The number of buffers so P(overflow) is <10-6?
Example

Service rate μ = 1/0.002=500

Gateway utilization ρ = λ/μ = 0.25

Prob. of n packets in system (gateway) =
(1  ρ)ρ n  0.75(0.25) n

Mean number of packets in system (gateway) =
ρ
0.25

 0.33
1  ρ 0.75
25
Example

Probability of buffer overflow with 13 buffer units:
= P(more than 13 packets in gateway)
= ρ14 = 0.2514 = 3.73x10-9
= 3.73 packets per billion packets

To limit the probability of loss to less than 10-6:
ρ n1  106

n  log 10
26
6
/ log 0.25 1
Policing Mechanisms
Token Bucket: limit input to specified Burst Size and Average
Rate.




Bucket can hold b tokens
Tokens generated at rate r token/sec unless bucket full
Over interval of length t: number of packets admitted less than or
equal to (r t + b).
At most b packets can be transmitted in a burst (within a short
period of time).
Policing Mechanisms (more)

token bucket, WFQ combine to provide guaranteed
upper bound on delay, i.e., QoS guarantee!
arriving
traffic
token rate, r
bucket size, b
WFQ
per-flow
rate, R
D = b/R
max
Appendix
29
CS352 Fall,2005
Example application of queuing theory
Which one is better, multiple-line or single-queue?
 We can prove using queuing theory that : throughput improves increases
30
with one queue instead of separate lines
