A Stochastic Approach
The problem we will consider here is how a disease spreads through a population. Let us suppose that
there is a certain troop of 10 monkeys. At time 𝑡 = 0, one of them somehow contracts a disease that is
contagious among monkeys. We will suppose that the average daily rate of disease transmission is 5%
for every possible monkey pair that exists in which one is infected and one is not.
Thus, if we let 𝑀 be the number of infected monkeys among the ten, and if we let 𝑡 be the number of
days since the disease was introduced to the troop, then a differential equation modeling the spread of
the disease is:
𝑑𝑀
= 0.05𝑀(10 − 𝑀)
𝑑𝑡
Interested students may want to solve this differential equation, given the initial condition 𝑀 = 1 at
time 𝑡 = 0. We will not solve it here, but a slope field of this differential equation along with its solution
is shown below.

sick monkeys












time (days)




















That solution is continuous and deterministic: it says that at any moment in time, we can predict exactly
how many monkeys will be infected. There is a good chance that the number will be a non-integer. Let
us now consider a stochastic approach to the problem.
We will say that there are 10 distinct states that the monkey troop can be in at any point in time, each
one indicating how many monkeys are infected—anywhere from one to all ten. We’ll identify these
states as 𝑆1 , … , 𝑆10.
Our model says that the average daily rate at which 𝑀 (the number of sick monkeys) changes is equal to
0.05𝑀(10 − 𝑀). That is equivalent to saying that over a very short time period Δ𝑡, the probability that
a new monkey is infected is approximately equal to 0.05𝑀(10 − 𝑀)Δ𝑡.1
Note that our model does not allow for monkey recovery. Thus if we are presently in state 𝑆𝑛 , then
after a short period of time Δ𝑡, there are only two possible states we can be in: we may still be in state
𝑆𝑛 , if no infections were transmitted; or we may be in state 𝑆𝑛+1 , if the disease was indeed transmitted.
Our new question of interest is: at a particular time 𝑡 > 0, what is the probability that we are in each
state 𝑆𝑛 , 𝑛 = 1, 2, … , 10?
Let us consider just the state 𝑆3 (chosen arbitrarily). To be in that state at time 𝑡 + Δ𝑡, we must have
previously been either in state 𝑆3 and “gone nowhere,” or else we must have been in state 𝑆2 and seen
an infection occur. Thus, the probability of being in state 𝑆3 at time 𝑡 is equal to:
𝑃3 (𝑡 + Δ𝑡) = 𝑃3 (𝑡) ⋅ [1 − 0.05(3)(10 − 7)Δ𝑡] + 𝑃2 (𝑡) ⋅ 0.05(2)(10 − 2)Δ𝑡
where 𝑃𝑛 (𝑡), 𝑛 = 1, … , 10 are functions indicating the probabilities of being in the various states 𝑆𝑛 at
time 𝑡.
We may expand the right-hand side of the equation...
𝑃3 (𝑡 + Δ𝑡) = 𝑃3 (𝑡) − 𝑃3 (𝑡) ⋅ 0.05(3)(10 − 3)Δ𝑡 + 𝑃2 (𝑡) ⋅ 0.05(2)(10 − 2)Δ𝑡
… and subtract 𝑃3 (𝑡) from both sides…
𝑃3 (𝑡 + Δ𝑡) − 𝑃3 (𝑡) = −𝑃3 (𝑡) ⋅ 0.05(3)(10 − 3)Δ𝑡 + 𝑃2 (𝑡) ⋅ 0.05(2)(10 − 2)Δ𝑡
… and divide both sides by Δ𝑡…
𝑃3 (𝑡 + Δ𝑡) − 𝑃3 (𝑡)
= −𝑃3 (𝑡) ⋅ 0.05(3)(10 − 3) + 𝑃2 (𝑡) ⋅ 0.05(2)(10 − 2)
Δ𝑡
… and take the limit of both sides …
lim
Δ𝑡→0
𝑃3 (𝑡 + Δ𝑡) − 𝑃3 (𝑡)
= lim −𝑃3 (𝑡) ⋅ 0.05(3)(10 − 3) + 𝑃2 (𝑡) ⋅ 0.05(2)(10 − 2)
Δ𝑡→0
Δ𝑡
..and the result is a differential equation.2
1
By considering only very short time periods, we reduce to effectively zero the probability that two different
transmissions will occur during the same time period. This simplifies our solution considerably.
2
Recall that we assumed that because Δ𝑡 was very small, the probability of two new infections occurring in the
same time period was negligible. More rigorously, we could have included possible transitions from one state to
another that involved 𝑞 > 1 new infections occurring. The probability of such a transition occurring would have
included (Δ𝑡)𝑞 as a factor, and even after dividing both sides of our equation by Δ𝑡, we would have been left with
a factor of at least Δ𝑡. In the limit, all terms like this would have gone to zero.
𝑑
(𝑃 (𝑡)) = −1.05𝑃3 (𝑡) + 0.8𝑃2 (𝑡)
𝑑𝑡 3
We may find it notationally convenient to hide the argument 𝑡 from these functions; but we must
remember that 𝑃𝑛 is a function of 𝑡 for all 𝑛 = 1, … , 10.
𝑑𝑃3
= −1.05𝑃3 + 0.8𝑃2
𝑑𝑡
It may be helpful to think of probability as “flowing” from one state to another. The −1.05𝑃3 is the
probability that is flowing “out” of state 𝑆3 and into state 𝑆4 . The 0.08𝑃 is the probability that is flowing
“into” state 𝑆3 from state 𝑆2 .
You can draw a “flow diagram” of this system and it might look something like this:
The only flow rates that have been included on this diagram are those into and out of state 𝑆3 , but the
others are not difficult to derive. Notice that states 𝑆1 and 𝑆10 are a little bit different from the others.
They are called “boundary states” because they either do not have probability flowing in, or do not have
probability flowing out.
The full system of differential equations for this problem is shown below, following a table in which the
𝑑𝑀
flow rates for 𝑀 monkeys are computed based on the differential equation 𝑑𝑡 = 0.05𝑀(10 − 𝑀).
𝑀
1
2
3
4
5
6
7
8
9
10
𝑑𝑀
= 0.05𝑀(10 − 𝑀)
𝑑𝑡
0.45
0.80
1.05
1.2
1.25
1.2
1.05
0.80
0.45
0
𝑑𝑃1
= −0.45𝑃1
𝑑𝑡
𝑑𝑃2
= 0.45𝑃1 − 0.80𝑃2
𝑑𝑡
𝑑𝑃3
= 0.80𝑃2 − 1.05𝑃3
𝑑𝑡
𝑑𝑃4
= 1.05𝑃3 − 1.2𝑃4
𝑑𝑡
𝑑𝑃5
= 1.2𝑃4 − 1.25𝑃5
𝑑𝑡
𝑑𝑃6
= 1.25𝑃5 − 1.2𝑃6
𝑑𝑡
𝑑𝑃7
= 1.2𝑃6 − 1.05𝑃7
𝑑𝑡
𝑑𝑃8
= 1.05𝑃7 − 0.80𝑃8
𝑑𝑡
𝑑𝑃9
= 0.80𝑃8 − 0.45𝑃9
𝑑𝑡
𝑑𝑃10
= 0.45𝑃9
𝑑𝑡
It is worth noting that all these derivatives add up to zero, meaning that the “flow” of probability must
be entirely contained within this set of ten possible states.
A numerical solution to the monkey problem
This system of differential equations may appear difficult to solve. But Euler’s method—or any of
several possible numerical approaches—may be employed to solve the system numerically. Note that
the initial conditions are 𝑃1 (0) = 1 and 𝑃𝑛 (0) = 0 for 𝑛 = 2, … , 10; i.e., there is a 100% chance that
there is exactly one sick monkey at time 𝑡 = 0, and there is a 0% chance that there is more than one sick
monkey at time 𝑡 = 0.
Using Euler’s method to solve the system of differential equations yields the following plots for 𝑃1 (𝑡),
𝑃2 (𝑡), …, 𝑃10 (𝑡).
1
probability
0.8
0.6
0.4
0.2
0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
time (days)
Because the curves run together a bit in the middle, it may be helpful to look at just a subset of the
functions 𝑃𝑛 (𝑡). Below is a plot showing just 𝑃1 (𝑡), 𝑃5 (𝑡), and 𝑃10 (𝑡).
22
1
probability
0.8
0.6
0.4
0.2
0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
time (days)
P1
P5
P10
In this plot we can see that by the time 4 days have passed, there is only about a 20% chance that there
will still be only one sick monkey. indeed, by day 8 there is at least a 25% chance that all the monkeys
will be sick.
Another graph that might be instructive is the “expected” or “average” number of monkeys sick at any
given time, which is given by the function:
10
̂ (𝑡) = ∑ 𝑛 ⋅ 𝑃𝑛 (𝑡)
𝑀
𝑛=1
That function is shown below.
expected number of sick monkeys
10
8
6
4
2
0
1
2
3
4
5
6
7
8
9
10
11
12
time (days)
13
14
15
16
17
18
19
20
21
22
Notice how similar this looks to the deterministic solution.
It is also possible to solve this system of equations analytically, although the technique is beyond the AP
Calculus syllabus. It is not beyond AP Calculus students, however, and if you want to try it, you are
𝑑𝑃
encouraged to do so. Notice that the first differential equation in the collection, 1 = −0.45𝑃1 , may be
𝑑𝑡
solved easily. The second solution depends upon the first; the third upon the second; and so on.
A useful technique for solving the system is called “integrating factors” and you can research it easily
online. Your teacher may also be able to help you with integrating factors.