Lecture 10:
Queueing Theory
Queueing Analysis

Jobs serviced by the system resources

Jobs wait in a queue to use a busy server
queue
server
Queueing Network Models

Open Network Model
Jobs arrive in the system from an
external resource and depart the
system

Closed Network Model
Has a fixed number of jobs that
circulate among various queues
and servers
Queueing Analysis











s
m
l
r
r
w
q
n
U
a
d
average service time
average service rate (1/s)
average arrival rate ((number of jobs arrive in time T) / T)
traffic intensity (l/m)
average response time (waiting time + service time)
average waiting time in queue
average number of jobs in the queue
number of jobs in the system (waiting & being serviced)
utilization (busy time / total time)
number of arrivals within interval T
number of departures within interval T
Operational Analysis
Job arrival rate (l)
Average arrival rate
Utilization
Computer
system
l=
Job service rate (m)
(requests/s)
U=
U=sl
U=
Utilization Law
=r
Traffic intensity
Operational Analysis
Ex: A single disk system
Average time to service each request is 600 ms.
What is the maximum possible request rate it can tolerate?
Operational Analysis
Utilization
U=sl
U=
=r
Traffic intensity
r>1
arrival rate > service rate
number of jobs in the queue increase without bound
r<1
arrival rate < service rate
stable system
Operational Analysis
Little’s Law
D=wlT
D=qT

q=wl
Little’s Law
Operational Analysis
queue
q=lw
U=ls
server
n=lr
Operational Analysis
Ex:
Average response time of a web server is 11 ms.
Request rate is 450 requests/s
What is the average number of requests that are buffered in
the system?
Stochastic Analysis
Kendall’s Notation
A/S/c
A – arrival process
most common arrival process is Poisson process:
interarrival times follow exponential distribution
exponential distribution is memoryless
S – service process
Poisson service process is the most common
c – number of servers
Stochastic Analysis
Interarrival time: Time between successive jobs arriving at the
queue
Ex:
Single queue
Exponential interarrival time
Exponential service time
M/M/1
queue
server
M/M/1
Markov Chain
State transition diagram if a birth-death process:
M/M/1
Flow Equations
Pi – probability of being in state i
Flow from state n-1 to state n:
Flow from state n to state n-1:
l Pn-1
m Pn
l Pn-1 = m Pn
Pn = r Pn-1 = rn P0
M/M/1
P0 =
r < 1,
1/rn
1/rn
Pn = rn (1-r)
converges to 1-r
M/M/1
Average number of jobs in the system:
E[n] =
nPn = r /(1-r)
Response time:
r=
=
Average number of jobs waiting in the queue:
Average waiting time in the queue:
q = r2/(1-r)
w = r/m(1-r)
Probability of finding k or more jobs in the system:
Pr(K≥k) = rk
M/M/1
Ex:
In a web server, the time between requests follow exponential
distribution with mean time between requests of 8 ms.
Time required by the server to process each request is also
exponential distribution with average service time 5 ms.
a.
b.
c.
What is the average response time observed?
How much faster must the server process requests to
halve this average response time?
How many queue entries are necessary to buffer requests
so that no more than one request per billion is lost?
M/M/c
queue
server
M/M/c
...
c – number of servers
l – job arrival rate
n – jobs in the system (both on server & in queue)
m – service rate for each server
M/M/c
If all servers are busy, mean service rate is: cm
If some servers are idle, mean service rate is:nm
(n<c)
M/M/c
Probability of finding n jobs in the system:
(cr)nP0 / n!
n=1,2,… c
Pn =
(cr)nP0 / c! cn-c n>c
Total traffic intensity: r = l/(cm)
Probability of all servers being busy:
k = Pr(K≥c) = (cr)cP0 / c!(1-r)
(utilization of each server)
M/M/c
Average number of jobs in the system:
E[n] = cr + rk/(1-r)
Average number of jobs waiting in the queue:
q = (r k) / (1-r)
Little’s law:
r = 1/m + k/cm(1-r)
Average time jobs wait in the queue:
w = k/cm(1-r)
M/M/c
Ex:
How does the response time in M/M/1 queue example change
if the number of servers in the system is 4?