1
Introduction to Stochastic Processes — 2007, TAU
N black balls and N white balls are placed in two urns so that each contains N balls.
After each unit of time one ball is selected at random from each urn, and the two balls thus
selected are interchanged. Let the number of black balls in the first urn denote the state
of the system. Write down the transition matrix of this Markov chain and find the unique
stationary distribution. Is the chain reversible in equilibrium?
..........................................................................................
The first urn contains i black balls and N − i white balls; the second urn contains i white
balls and N − i black balls. The transition i → i + 1 occurs if the ball selected from the first
urn is white and the ball selected from the second urn is black. Thus,
pi,i+1 =
N −i N −i
(N − i)2
·
=
.
N
N
N2
Similarly,
i(N − i)
i2
; pi,i = 2
.
2
N
N2
The chain must be reversible in equilibrium, since its graph has no loops. The stationary
probabilities µi satisfy the detailed balance condition
pi,i−1 =
µi pi,i+1 = µi+1 pi+1,i .
We have
2
pi,i+1
N −i
µi+1
=
=
;
µi
pi+1,i
i+1
2 2
2 2
N
N N −1
µ1
N
N
µ2
=
=
·
=
=
;
;...
µ0
1
µ0
1
2
1
2
2
µi
N
=
(by induction).
µ0
i
Therefore
µi =
In fact,
N 2
0
+
N 2
1
+···+
N 2
N
=
N 2
0
2N
N
+
N 2
i
N 2
+···
1
, thus, µi =
+
2
N 2
N
(Ni )
.
(2N
N )
.