CTMC Sample Problems
Problem 1
A small company has two identical PCs that are running at the same time. The time until either PC fails
is exponentially distributed with a mean of 1/λ. When a PC fails, a technician starts repairing it
immediately. The two PCs fail independently of each other. The time to repair a failed PC is
exponentially distributed with a mean of 1/μ. As soon as the repair is completed, the PC is brought
back online and is assumed to be as good as new.
a. Give the state-transition-rate diagram of the process.
b. What is the fraction of time that both machines are down?
Problem 2
Customers arrive at Mike’s barber shop according to a Poisson process with rate λ customers per hour.
Unfortunately Mike, the barber, has only five chairs in his shop for customers to wait when there is
already a customer receiving a haircut. Customers who arrive when Mike is busy and all the chairs are
occupied leave without waiting for a haircut. Mike is the only barber in the shop, and the time to
complete a haircut is exponentially distributed with a mean of 1/μ hours.
a. Give the state-transition-rate diagram of the process.
b. What is the probability that there are three customers waiting in the shop?
c. What is the probability that an arriving customer leaves without receiving a haircut?
d. What is the probability that an arriving customer does not have to wait?
Problem 3
A small company has two PCs A and B. The time to failure for PC A is exponentially distributed with a
mean of 1/λA hours, and the time to failure for PC B is exponentially distributed with a mean of 1/λB
hours. The PCs also have different repair times. The time to repair PC A when it fails is exponentially
distributed with a mean of 1/μA hours, and the time to repair PC B when it fails is exponentially
distributed with a mean of 1/μB hours. There is only one repair person available to work on both
machines when failure occurs, and each machine is considered to be as good as new after it has been
repaired.
a. Give the state-transition-rate diagram of the process.
b. What is the probability that both PCs are down?
c. What is the probability that PC A is the first to fail given that both PCs have failed?
d. What is the probability that both PCs are up?
Problem 4
Lazy Chris has three identical lightbulbs in his living room that he keeps on all the time. Because of his
laziness, Chris does not replace a lightbulb when it fails. (Maybe Chris does not even notice that the
bulb has failed!) However, when all three bulbs have failed, Chris replaces them at the same time. The
lifetime of each bulb is exponentially distributed with a mean of 1/λ, and the time to replace all three
bulbs is exponentially distributed with a mean of 1/μ.
a. Give the state-transition-rate diagram of the process.
b. What is the probability that only one lightbulb is working?
c. What is the probability that all three lightbulbs are working?
Problem 5
A switchboard has two outgoing lines serving four customers who never call each other. When a
customer is not talking on the phone, he or she generates calls according to a Poisson process with rate
λ calls per minute. Call lengths are exponentially distributed with a mean of 1/μ minutes. If a customer
finds the switchboard blocked (i.e., both lines are busy) when attempting to make a call, he or she
never tries to make that particular call again; that is, the call is lost.
a. Give the state-transition-rate diagram of the process.
b. What is the fraction of time that the switchboard is blocked?
Problem 6
A service facility can hold up to six customers who arrive according to a Poisson process with a rate of λ
customers per hour. Customers who arrive when the facility is full are lost and never make an attempt
to return to the facility. Whenever there are two or fewer customers in the facility, there is only one
attendant serving them. The time to service each customer is exponentially distributed with a mean of
1/μ hours. Whenever there are three or more customers, the attendant is joined by a colleague, and
the service time is still the same for each customer. When the number of customers goes down to two,
the last attendant to complete service will stop serving. Thus, whenever there are two or less
customers in the facility, only one attendant can serve.
a. Give the state-transition-rate diagram of the process.
b. What is the probability that both attendants are busy attending to customers?
c. What is the probability that neither attendant is busy?
Problem 7
A taxicab company has a small fleet of three taxis that operate from the company’s station. The time it
takes a taxi to take a customer to his or her location and return to the station is exponentially
distributed with a mean of 1/μ hours. Customers arrive according to a Poisson process with average
rate of λ customers per hour. If a potential customer arrives at the station and finds that no taxi is
available, he or she goes to another taxicab company. The taxis always return to the station after
dropping off a customer without picking up any new customers on their way back.
a. Give the state-transition-rate diagram of the process.
b. What is the probability that an arriving customer sees exactly one taxi at the station?
c. What is the probability that an arriving customer goes to another taxicab company?
Problem 8
Consider a collection of particles that act independently in giving rise to succeeding generations of
particles. Suppose that each particle, from the time it appears, waits a length of time that is
exponentially distributed with a mean of 1/λ and then either splits into two identical particles with
probability p or disappears with probability 1 − p. Let X(t), 0 ≤ t <∞, denote the number of particles
that are present at time t.
a. Find the birth and death rates of the process.
b. Give the state-transition-rate diagram of the process.
Problem 9
An assembly line consists of two stations in tandem. Each station can hold only one item at a time.
When an item is completed in station 1, it moves into station 2 if the latter is empty; otherwise it
remains in station 1 until station 2 is free. Items arrive at station 1 according to a Poisson process with
rate λ. However, an arriving item is accepted only if there is no other item in the station; otherwise it is
lost from the system. The time that an item spends at station 1 is exponentially distributed with mean
1/μ1, and the time that it spends at station 2 is exponentially distributed with mean 1/μ2. Let the state
of the system be defined by (m, n), where m is the number of items in station 1 and n is the number of
items in station 2.
a. Give the state-transition-rate diagram of the process.
b. Calculate the limiting state probabilities pmn.
Problem 10
Trucks bring crates of goods to a warehouse that has a single attendant. It is the responsibility of each
truck driver to offload his truck, and the time that it takes to offload a truck is exponentially distributed
with mean 1/μ1. When a truck is offloaded, it leaves the warehouse and takes a time that is
exponentially distributed with mean 1/λ to return to the warehouse with another set of crates. When a
truck driver is done with offloading his truck, the warehouse attendant takes an additional time that is
exponentially distributed with mean 1/μ2 to arrange the crates before the next truck in line can start
offloading. Assume that there are N trucks that bring crates to the warehouse. Denote the state of the
system by (m, n), where m is the number of trucks in the system and n is the state of the attendant: n =
1 if the attendant is busy arranging the crates and n = 0 otherwise.
a. Formulate the problem as a continuous-time Markov chain, specifying the state variables and giving
the state-transition-rate diagram of the process.
b. From the diagram identify the steady-state probability that the attendant is idle with no truck in the
warehouse.
Problem 11
Consider a system consisting of two birth and death processes labeled system 1 and system 2.
Customers arrive at system 1 according to a Poisson process with rate λ1, and customers arrive at
system 2 according to a Poisson process with rate λ2. Each system has two identical attendants. The
time it takes an attendant in system 1 to serve a customer is exponentially distributed with mean 1/μ1,
and the time it takes an attendant in system 2 to serve a customer is exponentially distributed with
mean 1/μ2. Any customer that arrives when the two attendants in its group are busy can receive
service from the other group, provided that there is at least one free attendant in that group;
otherwise it is lost. Let the state of the system be denoted by (m, n), where m is the number of
customers in system 1 and n is the number of customers in system 2. Give the state-transition-rate
diagram of the process and specify the probability that a customer receives service from a group that is
different from its own group.
Problem 12
Cars arrive at a parking lot according to a Poisson process with rate λ. There are only four parking
spaces, and any car that arrives when all the spaces are occupied is lost. The parking duration of a car is
exponentially distributed with mean 1/μ. Let pk(t) denote the probability that k cars are parked in the
lot at time t, k = 0, 1, . . . , 4.
a. Give the differential equation governing pk(t).
b. What are the steady-state values of these probabilities?
c. What is m14, the mean first passage time to state 4 given that the process started in state 1?