Mean Value Analysis
and Related Techniques
Raj Jain
Washington University in Saint Louis
Saint Louis, MO 63130
Jain@cse.wustl.edu
Audio/Video recordings of this lecture are available at:
http://www.cse.wustl.edu/~jain/cse567-06/
Washington University in St. Louis
CSE567M
34-1
©2008 Raj Jain
Overview
1. Analysis of Open Queueing Networks
2. Mean-Value Analysis
3. Approximate MVA
4. Balanced Job Bounds
Washington University in St. Louis
CSE567M
34-2
©2008 Raj Jain
Analysis of Open Queueing Networks
!
!
!
!
!
Used to represent transaction processing systems, such as
airline reservation systems, or banking systems.
Transaction arrival rate is not dependent on the load on the
computer system.
Arrivals are modeled as a Poisson process with a mean arrival
rate λ.
Exact analysis of such systems
Assumption: All devices in the system can be modeled as
either fixed-capacity service centers (single server with
exponentially distributed service time) or delay centers
(infinite servers with exponentially distributed service time).
Washington University in St. Louis
CSE567M
34-3
©2008 Raj Jain
Analysis of Open Queueing Networks
!
!
!
!
For all fixed capacity service centers in an open queueing
network, the response time is:
Ri = Si (1+Qi)
On arrival at the ith device, the job sees Qi jobs ahead
(including the one in service) and expects to wait Qi Si
seconds. Including the service to itself, the job should expect a
total response time of Si(1+Qi) .
Assumption: Service is memory-less (not operationally
testable) ⇒ Not an operational law
Without the memory-less assumption, we would also need to
know the time that the job currently in service has already
consumed.
Washington University in St. Louis
CSE567M
34-4
©2008 Raj Jain
Mean Performance
Assuming job flow balance, the throughput of the system is
equal to the arrival rate:
X=λ
! The throughput of ith device, using the forced flow law is:
Xi = X Vi
! The utilization of the ith device, using the utilization law is:
Ui = Xi Si = X Vi Si = λ Di
! The queue length of ith device, using Little's law is:
Qi = Xi Ri = Xi Si(1+Qi) =Ui(1+Qi)
Or
Qi = Ui/(1-Ui)
! Notice that the above equation for Qi is identical to the
equation for M/M/1 queues.
!
Washington University in St. Louis
CSE567M
34-5
©2008 Raj Jain
Mean Performance
!
The device response times are:
!
In delay centers, there are infinite servers and, therefore:
!
Notice that the utilization of the delay center represents the
mean number of jobs receiving service and does not need to be
less than one.
Washington University in St. Louis
CSE567M
34-6
©2008 Raj Jain
Example 34.1
!
!
File server consisting of a
CPU and two disks, A and B.
With 6 clients systems:
Washington University in St. Louis
CSE567M
34-7
©2008 Raj Jain
Example 34.1 (Cont)
Washington University in St. Louis
CSE567M
34-8
©2008 Raj Jain
Example 34.1 (Cont)
!
Device utilizations using the utilization law are:
Washington University in St. Louis
CSE567M
34-9
©2008 Raj Jain
Example 34.1 (Cont)
!
The device response times using Equation 34.2 are:
!
Server response time:
!
We can quantify the impact of the following changes:
Washington University in St. Louis
CSE567M
34-10
©2008 Raj Jain
Example 34.1 (Cont)
!
Q: What if we increase the number of clients to 8?
⇒ Request arrival rate will go up by a factor of 8/6.
!
Conclusion: Server response time will degrade by a factor of
6.76/1.406= 4.8
Washington University in St. Louis
CSE567M
34-11
©2008 Raj Jain
Example 34.1 (Cont)
Q: What if we use a cache for disk B with a hit rate of
50%, although it increases the CPU overhead by 30%
and the disk B service time (per I/O) by 10%.
! A:
!
Washington University in St. Louis
CSE567M
34-12
©2008 Raj Jain
Example 34.1 (Cont)
!
The analysis of the changed systems is as follows:
!
Thus, if we use a cache for Disk B, the server response time
will improve by (1.406-1.013)/1.406 = 28%.
Washington University in St. Louis
CSE567M
34-13
©2008 Raj Jain
Example 34.1 (Cont)
!
Q: What if we have a lower cost server with only one disk
(disk A) and direct all I/O requests to it?
!
A: the server response time will degrade by a factor of
3.31/1.406 = 2.35
Washington University in St. Louis
CSE567M
34-14
©2008 Raj Jain
Mean-Value Analysis (MVA)
!
!
!
!
!
!
Mean-value analysis (MVA) allows solving closed queueing
networks in a manner similar to that used for open queueing
networks
It gives the mean performance. The variance computation is not
possible using this technique.
Initially limit to fixed-capacity service centers. Delay centers
are considered later. Load-dependent service centers are also
considered later.
Given a closed queueing network with N jobs:
Ri(N) = Si (1+Qi(N-1))
Here, Qi(N-1) is the mean queue length at ith device with
N-1 jobs in the network.
It assumes that the service is memoryless
Washington University in St. Louis
CSE567M
34-15
©2008 Raj Jain
Mean-Value Analysis (MVA)
Since the performance with no users ( N=0 ) can be
easily computed, performance for any number of users
can be computed iteratively.
! Given the response times at individual devices, the
system response time using the general response time
law is:
!
!
The system throughput using the interactive response
time law is:
Washington University in St. Louis
CSE567M
34-16
©2008 Raj Jain
Mean-Value Analysis (MVA)
The device throughputs measured in terms of jobs per
second are:
Xi(N)= X(N) Vi
! The device queue lengths with N jobs in the network
using Little's law are:
Qi(N)= Xi(N) Ri(N)= X(N) Vi Ri(N)
! Response time equation for delay centers is simply:
Ri(N) = Si
! Earlier equations for device throughputs and queue
lengths apply to delay centers as well.
Qi(0)=0
!
Washington University in St. Louis
CSE567M
34-17
©2008 Raj Jain
Example 34.2
!
!
!
!
Consider a timesharing system
Each user request makes ten I/O
requests to disk A, and five I/O
requests to disk B.
The service times per visit to disk A and
disk B are 300 and 200 milliseconds,
respectively.
Each request takes two seconds of CPU
time and the user think time is four seconds.
Washington University in St. Louis
CSE567M
34-18
©2008 Raj Jain
Example 34.2 (Cont)
!
!
Initialization:
" Number of users: N=0
" Device queue lengths: QCPU=0 , QA=0 , QB = 0
Iteration 1:
" Number of users: N=1
" Device response times:
"
System Response time:
Washington University in St. Louis
CSE567M
34-19
©2008 Raj Jain
Example 34.2 (Cont)
System Throughput: X=N/(R+Z)=1/(6+4)=0.1
" Device queue lengths:
"
!
Iteration 2:
" Number of users: N=2
" Device response times:
Washington University in St. Louis
CSE567M
34-20
©2008 Raj Jain
Example 34.2 (Cont)
"
System Response time:
System Throughput: X=N/(R+Z)=2/(7.4+4)=0.175
" Device queue lengths:
"
Washington University in St. Louis
CSE567M
34-21
©2008 Raj Jain
MVA Results for Example 34.2
!
!
!
!
MVA is applicable only if the network is a product form network.
This means that the network should satisfy the conditions of job flow
balance, one step behavior, and device homogeneity.
Also assumes that all service centers are either fixed-capacity service
centers or delay centers.
In both cases, we assumed exponentially distributed service times.
Washington University in St. Louis
CSE567M
34-22
©2008 Raj Jain
Approximate MVA
!
!
!
Useful for large values of N
Schweitzer's approximation:
" Estimate the queue lengths with N jobs and computing the
response times and throughputs. The values so computed
can be used to re-compute the queue lengths.
" Assumes that as the number of jobs in a network increases,
the queue length at each device increases proportionately.
Analytically:
Washington University in St. Louis
CSE567M
34-23
©2008 Raj Jain
Approximate MVA (Cont)
!
In particular, this implies:
Or
!
MVA equations can, therefore, be written as follows:
Washington University in St. Louis
CSE567M
34-24
©2008 Raj Jain
Approximate MVA (Cont)
If the new values of Qi are not close to the old values
⇒ continue iterating.
! If they are sufficiently close, we stop.
!
Washington University in St. Louis
CSE567M
34-25
©2008 Raj Jain
Example 34.3
!
Consider again the timesharing system of Example 34.1. Let us
analyze this model using Schweitzer's approximation when
there are 20 users on the system. The stopping criterion is to
stop when the maximum absolute change in every queue length
is less than 0.01.
The system parameters are:
!
Z=4 , and N=20
!
Washington University in St. Louis
CSE567M
34-26
©2008 Raj Jain
Example 34.3 (Cont)
!
To initialize the queue lengths, we assume that the 20 jobs are
equally distributed among the three queues of CPU, disk A, and
disk B.
!
Iteration 1:
" Device response times:
"
System Response time:
Washington University in St. Louis
CSE567M
34-27
©2008 Raj Jain
Example 34.3 (Cont)
System throughput: X=N/(R+Z)=20/(44+4)=0.42
" Device queue lengths:
"
"
Maximum absolute change in device queue lengths:
Washington University in St. Louis
CSE567M
34-28
©2008 Raj Jain
Example 34.3 (Cont)
Washington University in St. Louis
CSE567M
34-29
©2008 Raj Jain
Example 34.3 (Cont)
!
!
Common mistake: a small error in throughput does not imply
that the approximation is satisfactory. The same applies to
device utilizations, and the system response time.
In spite of a small error in any of these, the error in the device
queue lengths may be quite large.
Washington University in St. Louis
CSE567M
34-30
©2008 Raj Jain
Example 34.3 (Cont)
!
!
!
Note that the throughput reaches close to its final value within
five iterations, while the response time reaches close to its final
value within six iterations.
Queue lengths take the longest to stabilize.
Notice, that for all values of N , the error in throughput is
small; the error in response time is slightly larger; and error in
queue lengths is the largest.
Washington University in St. Louis
CSE567M
34-31
©2008 Raj Jain
Balanced Job Bounds
!
!
!
!
A system without a bottleneck device is called a balanced
system.
Balanced system has a better performance than a similar
unbalanced system
⇒ Allows getting two sided bounds on performance
An unbalanced system's performance can always be improved
by replacing the bottleneck device with a faster device.
Balanced System: Total service time demands on all devices
are equal.
Washington University in St. Louis
CSE567M
34-32
©2008 Raj Jain
Balanced Job Bounds (Cont)
!
Thus, the response time and throughput of a time-sharing
system can be bounded as follows:
!
Here, Davg=D/M is the average service demand per device.
These equations are known as balanced job bounds.
These bounds are very tight in that the upper and lower bound
are very close to each other and to the actual performance.
For batch systems, the bounds can be obtained by substituting
Z=0
!
!
!
Washington University in St. Louis
CSE567M
34-33
©2008 Raj Jain
Balanced Job Bounds (Cont)
Assumption: All service centers except terminals are
fixed-capacity service centers.
! Terminals are represented by delay centers. No other
delay centers are allowed because the presence of
delay centers invalidates several arguments related to
Dmax and Davg .
!
Washington University in St. Louis
CSE567M
34-34
©2008 Raj Jain
Derivation of Balanced Job Bounds
Steps:
1. Derive an expression for the throughput and response time of a
balanced system.
2. Given an unbalanced system, construct a corresponding `best
case' balanced system such that the number of devices is the
same and the sum of demands is identical in the balanced and
unbalanced systems. This produces the upper bounds on the
performance.
3. Construct a corresponding `worst case' balanced system such
that each device has a demand equal to the bottleneck and the
number of devices is adjusted to make the sum of demands
identical in the balanced and unbalanced systems. This
produces the lower bounds on performance.
Washington University in St. Louis
CSE567M
34-35
©2008 Raj Jain
Derivation (Cont)
!
!
!
!
Any timesharing system can be divided into two subsystems:
the terminal subsystem consisting of terminals only, and the
central subsystem consisting of the remaining devices.
Consider a system whose central subsystem is balanced in the
sense that all M devices have the same total service demand:
Here, D is the sum of total service demands on the M
devices.
Device response times using the mean-value analysis:
Washington University in St. Louis
CSE567M
34-36
©2008 Raj Jain
Derivation (Cont)
!
Since, the system is balanced, all Qi 's are equal, and we have:
!
!
Here, Q(j) (without any subscript) denotes the total number of
jobs in the central subsystem when there are j jobs in the
system.
The number of jobs in the terminal subsystem is j-Q(j) .
!
The system response time is given by:
!
or
Washington University in St. Louis
CSE567M
34-37
©2008 Raj Jain
Derivation (Cont)
A non-iterative procedure to bound Q(N) is based on
the following arguments.
!
!
If we replace the system with N workstations so that each user
has his own workstation and the workstations are identical to
the original system, then the new environment would have a
better response time and better throughput.
The new environment consists of N single user systems and
is, therefore, easy to model. Each user spends its time in cycles
consisting of Z units of time thinking and D units of time
computing. Each job has a probability D/(D+Z) of being in
the central subsystem (not at the terminal).
Washington University in St. Louis
CSE567M
34-38
©2008 Raj Jain
Derivation (Cont)
Now consider another environment like the previous
one except that each user is given a workstation that is
N times slower than the system being modeled. This
new environment has a total computing power
equivalent to the original system, but there is no
sharing.
! The users would be spending more time in the central
subsystem. That is:
!
Washington University in St. Louis
CSE567M
34-39
©2008 Raj Jain
Derivation (Cont)
!
The two equations above combined together result in the
following bounds on the number of jobs at the devices
!
In terms of response time, this results in the following bounds:
!
This completes the first step of the derivation.
Washington University in St. Louis
CSE567M
34-40
©2008 Raj Jain
Derivation (Cont)
!
Step 2 :Suppose we have an unbalanced system such that the
service demands on ith device is Di
!
Bound on the performance of the unbalanced system:
!
and
!
The expressions on the right-hand side are for the balanced
system. This completes the second step of the derivation.
Washington University in St. Louis
CSE567M
34-41
©2008 Raj Jain
Derivation (Cont)
!
Step 3: Consider a balanced system in which M'=D/Dmax
devices have nonzero demands, each equal to Dmax ; the
remaining devices have zero demands and can, therefore, be
deleted from the system.
!
The expressions on the left-hand side are for the balanced
system.
Combining equations with asymptotic bounds we get the
balanced job bounds.
!
Washington University in St. Louis
CSE567M
34-42
©2008 Raj Jain
Example 34.4
!
For the timesharing system of Example 34.1
" DCPU = 2 , DA = 3 , DB = 1 , Z = 4
" D = DCPU + DA + DB = 2+3+1 = 6
" Davg = D/3 = 2
" Dmax = DA = 3
!
The balanced job bounds are:
Washington University in St. Louis
CSE567M
34-43
©2008 Raj Jain
Example 34.4 (Cont)
Washington University in St. Louis
CSE567M
34-44
©2008 Raj Jain
Example 34.4 (Cont)
Washington University in St. Louis
CSE567M
34-45
©2008 Raj Jain
Summary
1. Open queueing networks of M/M/1 or M/M/∞ can be
analyzed exactly
2. MVA allows exact analysis of closed queueing networks.
Given performance of N-1 users, get performance for N users.
3. Approximate MVA is used when there is a large number of
users. Assume queue lengths for a system with N users and
compute, response time, throughput, and queue lengths.
4. Balanced Job bounds: A balanced system with Di = Davg will
have better performance and an unbalanced system with some
devices at Dmax and others at 0 will have worse performance.
Washington University in St. Louis
CSE567M
34-46
©2008 Raj Jain
Homework 34
For a timesharing system with two disks (user and system), the
probabilities for jobs completing the service at the CPU were found
to be 0.75 to disk A, 0.15 to disk B, and 0.1 to the terminals. The
user think time was measured to be 5 seconds, the disk service
times are 30 milliseconds and 25 milliseconds, while the average
service time per visit to the CPU was 40 milliseconds.
Using the queueing network model shown in Figure 33.8:
a. Use MVA to compute system throughput and response time for
N=1,…,5 interactive users
b. Use Schweitzer’s approximation to MVA with N=5 users. Use a
starting value of 5/3 for each of the three device queue lengths and
stop after 5 iterations
c. Write the expressions for balanced job bounds on the system
throughput and response time of the system and compute the
bounds for up to N=5 users.
Washington University in St. Louis
CSE567M
34-47
©2008 Raj Jain