Queueing Systems
Copyright ©: Nahrstedt, Angrave, Abdelzaher, Caccamo
1
Copyright ©: Nahrstedt, Angrave, Abdelzaher
Content of This Lecture

Goals:


Introduction to Principles for Reasoning
about Process Management/Scheduling
Things covered in this lecture:

Introduction to Queuing Theory
2
Copyright ©: Nahrstedt, Angrave, Abdelzaher
Process States Finite State
Diagram
3
Copyright ©: Nahrstedt, Angrave, Abdelzaher
Queueing Model

Random Arrivals modeled as Poisson process

Service times follow exponential distribution
4
Copyright ©: Nahrstedt, Angrave, Abdelzaher
Discussion

If a bus arrives at a bus stop every 15
minutes, how long do you have to wait
at the bus stop assuming you start to
wait at a random time?
5
Copyright ©: Nahrstedt, Angrave, Abdelzaher
Discussion


The mean value is (0+15)/2 = 7.5 minutes
What assumption have you made about the
distribution of your arrival time?
6
Copyright ©: Nahrstedt, Angrave, Abdelzaher
Discussion



The mean value is (0+15)/2 = 7.5 minutes
What assumption have you made about the
distribution of your arrival time?
The above mean assumes that your arrival time to
the bus station is uniformly distributed within [0, 15]
7
Copyright ©: Nahrstedt, Angrave, Abdelzaher
Queuing Theory (M/M/1 queue)
Server
ARRIVAL RATE 
(Poisson process)
Input Queue


SERVICE RATE 
the distribution of inter-arrival times between two consecutive arrivals is
exponential (arrivals are modeled as Poisson process)
service time is exponentially distributed with parameter

8
M/M/1 queue


The M/M/1 queue assumes that arrivals are a Poisson process and the
service time is exponentially distributed.
Interarrival times of a Poisson process are IID (Independent and Identically
Distributed) exponential random variables with parameter

- independent from each other!
Arrival times:
- each interarrival i follows
an exponential distribution
1
2
t
Arrival rate 
Service rate 
CPU
Appendix: exponential
distribution


If is the exponential random variable describing the distribution of interarrival times between two consecutive arrivals, it follows that:
A(t )  P{  t}  1  e

 t
cumulative distribution
function (cdf)
The probability density function (pdf) is:
d
a(t )  A(t )  e t
dt

0
Arrival rate 
Service rate 
CPU
t
Probability to have the first
arrival within  is 1-e-  
Copyright ©: Nahrstedt, Angrave, Abdelzaher
Queueing Theory

Queuing theory assumes that the queue is in a steady state

M/M/1 queue model:




Poisson arrival with constant average arrival rate  (customers per unit time)
Each arrival is independent.
Interarrival times are IID (Independent and Identically Distributed) exponential
random variables with parameter 
What are the odds of seeing the first arrival
before time t?
P{  t}  1  e t
See http://en.wikipedia.org/wiki/Exponential_distribution
for additional details
Copyright ©: Nahrstedt, Angrave, Abdelzaher
Analysis of Queue Behavior

Poisson arrivals: probability n customers arrive within time interval t is
e
 t
t 
n
n!
12
Copyright ©: Nahrstedt, Angrave, Abdelzaher

Analysis of Queue Behavior
n
 t
e t 
Probability n customers arrive within time interval t is:
n!

Do you see any connection between previous formulas and the above one?
13
Copyright ©: Nahrstedt, Angrave, Abdelzaher

Analysis of Queue Behavior
n
 t
e t 
Probability n customers arrive in time interval t is:
n!


Do you see any connection between previous formulas and the above one?
Consider the waiting time until the first arrival. Clearly that time is more
than t if and only if the number of arrivals before time t is 0.
e


P t 
 t
t 0
0!
 e  t
P  t   1  P  t   1  e t
14
Copyright ©: Nahrstedt, Angrave, Abdelzaher
Little’s Law in queuing theory

The average number L of customers in a stable system is equal to the average
arrival rate λ times the average time W a customer spends in the system

It does not make any assumption about the specific probability distribution followed by the
interarrival times between customers

Wq= mean time a customer spends in the queue

 = arrival rate

Lq =  Wq

W = mean time a customer spends in the entire system (queue+server)

L=W
number of customers in queue
 number of customers in the system
In words – average number of customers is arrival rate times average waiting time
15
Copyright ©: Nahrstedt, Angrave, Abdelzaher
Analysis of M/M/1 queue
model

  1


Server Utilization:

mean time Ws a customer spends in the server is 1/, where  is the service rate.
According to M/M/1 queue model, the expected number of customers in the
Queue+Server system is:

L

1 
Quiz: how can we derive the average time W in the system, and the average
time Wq in the queue?

16
Copyright ©: Nahrstedt, Angrave, Abdelzaher
Analysis of M/M/1 queue
model
Quiz: how can we derive the average time W in the system, and the average time
Wq in the queue?
 Use Little’s theorem




Time in the system is:
Time in the queue is:
1
W 
 
Wq 

Try to derive them using
Little’s Law!
 
Number of customers in the queue is:
2
Lq 
1 
17
Copyright ©: Nahrstedt, Angrave, Abdelzaher
Hamburger Problem

7 Hamburgers arrive on average every time unit

8 Hamburgers are processed by Joe on average every unit
1.
Av. time hamburger waiting to be eaten? (Do they get cold?) Ans = ????
2.
Av number of hamburgers waiting in queue to be eaten? Ans = ????
Queue
8
7
18
Copyright ©: Nahrstedt, Angrave, Abdelzaher
Hamburger Problem

7 Hamburgers arrive on average every time unit

8 Hamburgers are processed by Joe on average every unit
1)
2)
How long is a hamburger waiting to be eaten? (Do they get cold?) Ans = 7/8
time units
How many hamburgers are waiting in queue to be serviced? Ans = 49/8
Queue
8
7
19
Copyright ©: Nahrstedt, Angrave, Abdelzaher
Example: How busy is the
server?
μ=3
λ=2
20
Copyright ©: Nahrstedt, Angrave, Abdelzaher
Example: How busy is the
server?
μ=3
λ=2
66%
21
Copyright ©: Nahrstedt, Angrave, Abdelzaher
How long is an eater in the
system?
μ=3
λ=2
22
Copyright ©: Nahrstedt, Angrave, Abdelzaher
How long is an eater in the
system?
μ=3
λ=2
1 = 1/(3-2)= 1
W 
 
23
Copyright ©: Nahrstedt, Angrave, Abdelzaher
How long is someone in the
queue?
μ=3
λ=2
Copyright ©: Nahrstedt, Angrave, Abdelzaher
How long is someone in the
queue?
μ=3
λ=2

Wq 
 .66 .66
  3 2
25
Copyright ©: Nahrstedt, Angrave, Abdelzaher
How many people in queue?
μ=3
λ=2
Copyright ©: Nahrstedt, Angrave, Abdelzaher
How many people in queue?
μ=3
λ=2
2 .662

Lq 

1.33
1  1.66
27
Copyright ©: Nahrstedt, Angrave, Abdelzaher
Interesting Fact

As  approaches one, the queue length
becomes infinitely large.
28
Copyright ©: Nahrstedt, Angrave, Abdelzaher
Until Now We Looked at Single
Server, Single Queue
ARRIVAL RATE 
Server
Input Queue
SERVICE RATE 
29
Copyright ©: Nahrstedt, Angrave, Abdelzaher
Sum of Independent Poisson Arrivals
ARRIVAL RATE 1
ARRIVAL RATE 2

Server
Input Queue
SERVICE RATE 
If two or more arrival processes are independent and Poisson with parameter λi,
then their sum is also Poisson with parameter λ equal to the sum of λi
= 1+  2
30
Copyright ©: Nahrstedt, Angrave, Abdelzaher
As long as service times are
exponentially distributed...
SERVICE RATE 1
Server
ARRIVAL RATE 
Input Queue
Combined =1+2
Server
SERVICE RATE 2
31
Copyright ©: Nahrstedt, Angrave, Abdelzaher
Question: McDonalds Problem
λ
λ
λ
μ
μ
μ
A) Separate Queues per Server

λ
λ
λ
μ
μ
μ
B) Same Queue for Servers
Quiz: if WA is waiting time for system A, and WB is waiting time for system
B, which queuing system is better (in terms of waiting time)?
32