Introduction to Queueing
Theory
COMP5416
Advanced Network Technologies
Courtesy of R.Martin, Rutgers Univ
Queuing Theory
• View network as collections of queues
– FIFO data-structures
• Queuing theory provides probabilistic analysis of
these queues
• Examples:
– Average length
– Probability queue is at a certain length
– Probability a packet will be lost
School of Information Technologies
M/M/1-2
Little’s Formula (aka. Little’s Law)
Arrivals
Departures
System
• Little’s Law:
– Mean number tasks in system = arrival rate x mean
residence time
• Observed before, Little was first to prove
• Applies to any system in equilibrium, as long as
nothing in black box is creating or destroying tasks
School of Information Technologies
M/M/1-3
Example using Little’s Formula
• Consider 40 customers/hour visit Hungry Jack
Restaurant
– Average customer spend 15 minutes in the restaurant
• What is average number of customers at at given
time?
r  Tr  40cust / hr  0.25hr / cust  10
School of Information Technologies
M/M/1-4
Model Queuing System
• Use Little’s formula on complete system and parts to
reason about average time in the queue
School of Information Technologies
M/M/1-5
Kendal Notation
•
Six parameters in shorthand (x/x/x/x/x/x)
–
1.
2.
3.
4.
5.
6.
First three typically used, unless specified
Arrival Distribution
Service Distribution
Number of servers
Total Capacity (infinite if not specified)
Population Size (infinite)
Service Discipline (FCFS/FIFO)
School of Information Technologies
M/M/1-6
Distributions
•
•
•
•
•
M: Exponential
D: Deterministic (e.g. fixed constant)
Ek: Erlang with parameter k
Hk: Hyperexponential with param. k
G: General (anything)
• M/M/1 is the simplest ‘realistic’ queue
School of Information Technologies
M/M/1-7
Kendal Notation Examples
• M/M/1:
– Exponential arrivals and service, 1 server, infinite capacity
and population, FCFS (FIFO)
• M/M/n
– Same as above, but n servers
• G/G/3/20/1500/SPF
– General arrival and service distributions, 3 servers, 17
queue slots (20-3), 1500 total jobs, Shortest Packet First
School of Information Technologies
M/M/1-8
M/M/1 Queue
Single Isolated Link
•Assume messages arrive at the channel according to a Poisson
process at a rate  messages/sec.
•Assume that the message lengths have a negative exponential
distribution with mean 1/n bits/message.
•Channel transmits messages from its buffer at a constant rate
c bits/sec
•The buffer associated with the channel can considered to be
effectively of infinite length.
=> an M/M/1 queue.
School of Information Technologies
M/M/1-9
M/M/1
• The first M says that the interarrival times to the queue
are negative-exponential
(ie Markovian or memoryless);
• The second M says that the service times are negativeexponential (ie Markovian or memoryless again);
• The 1 says that there is a single server at the queue.
• Shorthand notation for "a queue with Poisson arrivals,
negative exponentially distributed message lengths, a
single server, and infinite buffer space".
School of Information Technologies
M/M/1-10
• Arrival rate  messages/sec,
• Message lengths neg exp, mean 1/n bits/message,
• Transmission rate c bits/sec,
– ie messages transmitted at rate m = cn messages/sec.
• Define the state of the system as the total number of
messages at the link (waiting + being transmitted).
• The system is therefore a Markov chain, since both the
arrival and message length distributions are memoryless.


1
0
m


2
m
m
School of Information Technologies
M/M/1-11
M/M/1 queue model
r
w
Tw
1/m
Tr
School of Information Technologies
M/M/1-12
Solving queuing systems
• Given:
– : Arrival rate of jobs (packets)
– m: Service rate of the server (output link)
• Solve:
– r: average number of items in queuing system
– Tr: avg. time an item spends in whole system
– Tw: avg. time an item waits in the queue
School of Information Technologies
M/M/1-13
Equilibrium conditions
Boundary for flux balance

0

1
m


2
m
m
Boundary for global balance
 pi 1  m pi i  1,2,
School of Information Technologies
M/M/1-14
• Solving the flux balance equations recursively,
we get
2
pi 
i



pi 1  pi  2     p0
m
m
m

Define occupancy  
m
 pi   i p0
School of Information Technologies
M/M/1-15
 

i

p0   


 i 0 
1
 (1   )
pi  (1   )  i i  0,1,2,
(normalising condition)
(geometric distribution)
Note:
Solution valid only for < 1 (stability condition)
For  1, there is no steady state!
School of Information Technologies
M/M/1-16
M/M/1 Queue Length Distribution
0 .5
0 .5
0 .4
0 .4
0 .3
0 .3
0 .2
0 .2
0 .1
0 .1
0
0
0
1
2
3
4
5
6
05
7
8
9
0
1
2
3
5
5
6
7
8
9
 09
School of Information Technologies
M/M/1-17
M/M/1 Mean Number in the System


i 0
i 0
E{R}   i pi  (1   ) i i
 (1   )


(1   ) 2

1 
School of Information Technologies
M/M/1-18
M/M/1 Mean Delay

Mean delay  E{TR }   E{TR | i} pi
i 0

1
 i 1
E{R}  1 1  
 pi 
  

m
m
i 0  m 
1 1
E{TR } 
M/M/1 queue
m 1 
10.00

E{T}
8.00
6.00
4.00
2.00
0.00
0
0.25
0.5
0.75
1

School of Information Technologies
M/M/1-19
M/M/1 Mean Waiting Time
• The waiting time is defined as the time in the
queue, excluding service time:
1  1  1 1  
   

E{TW }  E{TR }   
m m  1   m m 1  
1
School of Information Technologies
M/M/1-20
Summary of M/M/1 (infinite buffer)

  1
m

E{R} 
1 
1 1
E{TR } 
m 1 
1 
E{TW } 
m 1 
School of Information Technologies
M/M/1-21
Little’s Formula
• If  messages/sec enter a “system”,
and each spends, on average, W seconds there,
then the total number in the “system” must be
given by  W by a simple flow argument.
1
1 /m 1
• Proof for M/M/1: E{TR } 

 E{R}
m 1   1  
 E{R}   E{TR }
1
School of Information Technologies
M/M/1-22
M/M/1/n – Finite Buffer


1
0



2
m
m
m
n
m
pi   i p 0 i  0,1, , n
 n
1

1 

p0   

n 1


1


 i 0 
1 
i
pi 

i  0,1,, n
n 1
1 
i
School of Information Technologies
M/M/1-23
M/M/1/n
• Probability that the buffer will be blocked:
PB  Pr{message is blocked }  pn 
1 
1 
n

n 1
School of Information Technologies
M/M/1-24
M/M/1/n example
Example :  = 0.5
• Assume that we want
• Choose n so that
PB  103
1  0 .5
1  0.5n 1
0 .5 n 
0.5n 1 
• i.e. choose n = 9
0.5n 1
1  0.5n 1
 10 3
10  3
1  10 3
n  1  9.96
School of Information Technologies
M/M/1-25
M/M/1/n – validity of solution
• Solution for the finite buffer queue is valid for
all 
– not just for  < 1 as was the case for the infinite
buffer.
• This is because the finite buffer copes with
overloads by blocking messages, rather than by
creating a backlog of messages.
School of Information Technologies
M/M/1-26
M/M/1/n – Little’s formula
To use Little's formula for the finite buffer case,
note that the rate of non-blocked messages is given
by:
   (1  PB )
and it is only these messages which contribute to
the buffer size. This implies that:
E{R}   E{TR }   (1  PB ) E{TR }
School of Information Technologies
M/M/1-27
Summary of M/M/1/n


m
(can be  1 or < 1)
E{R}   (1  PB ) E{TR }
(1   )  n
PB 
1   n 1
(1   ) n i
E{R}   ipi 
i
n 1 
1   i 0
i 0
n
School of Information Technologies
M/M/1-28
State dependent Queues
Let
i= arrival rate when the system is in state i
mi = service rate when the system is in state i
0
1
0
m
Balance Equations:
with solution



2
m
m
i 1 pi 1  mi pi i  1,2,
01  i 1
pi 
p0 i  1,2,
m1m 2  mi
School of Information Technologies
M/M/1-29
M/M/2
i   i  0,1,
• State dependent with

m i  1
mi  
2m i  2,3,


1
0
m
 pi 

2
m
i
m (2m )
i 1
p0 
m
i
2
i 1 i
m
p0
School of Information Technologies
M/M/1-30
M/M/1 vs M/M/2
Compare M/M/2 (2 servers – each at rate m) to that of a single link at rate 2m .
E{TR }1 server, rate 2 m 
E{TR }2 servers, each ratem
1 1
2m 1  
1

m (1   )(1   )
E{TR }1 server, rate 2 m
E{TR }2 servers, each ratem
1
(M/M/1 result)
(M/M/2 result)
1 

1
2
Conclusion: Single fast server is always more efficient!
School of Information Technologies
M/M/1-31
0
0.
12
5
0.
17
5
0.
22
5
0.
27
5
0.
32
5
0.
37
5
0.
42
5
0.
47
5
0.
52
5
0.
57
5
0.
62
5
0.
67
5
0.
72
5
0.
77
5
0.
82
5
0.
87
5
0.
92
5
0.
97
5
Mean Delay (units of single packet service
times)
M/M/1 vs M/M/2
M/M/1 vs M/M/2
4
3
2
M/M/2: 2 servers, each rate m
1
M/M/1: 1 server, rate 2m
0
Occupancy
School of Information Technologies
M/M/1-32
M/M/


1
0
m


2
m
i
i
A
pi 
p0  e  A
i!
m i i!
m


 with A  
m

(a Poisson distribution)
School of Information Technologies
M/M/1-33
M/M/1 delay distribution
pTR (t )  m (1   )e m (1  )t
 (an exponential distribution)
• Probability of delay within certain limit:
Pr(TR  t )  1  e  m (1  )t
School of Information Technologies
M/M/1-34
Example
Suppose that packets arrive at a link at a Poisson rate of
250 packets/second, and the packet lengths are
exponentially distributed with mean length 1000
bits/packet. Assume that the buffer length is sufficiently
large that it can be assumed to be effectively infinite.
Find the required capacity of the link in bits/second so that
the following criteria are satisfied:
(1) Mean delay  10 msec
(2) Pr{delay  20 msec}  0.10
School of Information Technologies
M/M/1-35
Solution
250 1000 250000

 c  250,000 bps
c
c
1 1
Mean Delay : E{TR } 
m 1 
1000
1

c 1  250000
c
1000

 0.010
c  250000
Occupancy :  
c  350,000bps
School of Information Technologies
M/M/1-36
Solution
Percentile of Delay : Pr{TR  t}  e  m (1  )t
Pr{TR  20 msec}  e

c  250000 20
1000 1000
 0.10
c  365,129 bps
In order to satisfy both criteria, we need to choose
capacity >365,129 bps.
(The next highest standard rate is 384 kbps.)
School of Information Technologies
M/M/1-37