The Collins Case Again∗
Halvor Mehlum†
Abstract
In this note, I follow up Turner’s (2002) discussion of the Collins Case. I derive
some results, which correct an oversight in Turner’s paper. The discussion illustrates
the general point that it is important for the analysis of statistical data to know
exactly how the data was collected.
1
Introduction
In this journal of December 2002, T. Rolf Turner discusses the interesting Collins Case.
In California in 1968 a couple with certain characteristics committed a robbery. A couple
with identical characteristics was arrested and put on trial. In court the prosecutor
claimed that the characteristics would appear in a randomly chosen couple with the
probability 1/12,000,000. The suspected couple was convicted based on a flawed argument
called the ”transposition fallacy” or the ”prosecutor’s fallacy” - since the probability of
having the requisite characteristics is extremely low, the probability that a couple with
these characteristics is the guilty couple must be extremely high. The California Supreme
∗
I am grateful to a referee and the Editor for clarifying my argument.
Department of Economics, University of Oslo P.O. Box 1095, Blindern N-0317 Oslo, Norway. E-mail:
halvor.mehlum@econ.uio.no.
†
1
The Collins Case
2
Court later overturned the conviction. The reasoning in the Supreme Court appeal was
in part based on the following argument: Given that there is at least one couple with
the characteristics (the guilty couple), and given that the number of couples in the area
is several millions, the probability that there is more than one such couple is quite high.
Hence, it may well be that the suspected couple is different from the guilty couple. Turner
brings this argument an essential step further by asking the crucial question: “What is the
probability that the suspect is innocent given the evidence?” In providing his answer he
builds on a ‘beads in an urn’ analogy: “We suppose that there is a (large) urn containing
a number of white, and possibly some red, beads. A red bead (the couple observed by
the eyewitnesses [. . . ]) is observed and then put back (escapes) into the urn. Later, a red
bead is drawn from the urn. What is the probability that it is the guilty red bead, i.e.
the same red bead that was first seen?” (Turner 2002, p. 82)
In his analysis, Turner answers the question by building on the statistical reasoning
used in the Supreme Court. The only information Turner extracts from the circumstances
is that there is at least one red bead. Turner is thus not in line with the insight from
parts of the literature related to the Collins Case and a similar stylised version - the
so-called “Island Problem.” The main contributions to this are due to Yellin (1979),
Eggleston (1983), Dawid (1994), Balding and Donnelly (1995) and Dawid and Mortera
(1996). All these authors argue that the prior probability distribution of the number of
couples fitting the characteristics should be updated, but none of them accepts arguments
similar to Turner’s. In the language of the ‘beads in an urn’ model, we should update our
prior probability distribution of red beads as we observe a red bead, but the updating
should go further than just eliminating the possibility of there being zero red beads. I
will explain this argument using the helpful ‘beads in an urn’ analogy. By this starting
3
The Collins Case
point a more detailed insight is gained compared to the direct Bayes’ approach used in
the literature referred to above.
2
The ‘beads in an urn’
Consider an urn containing N beads. Now, a robbery happens: One bead, the robberbead, is drawn at random from the urn. All beads are equally likely to be drawn. The
colour of the robber-bead is observed and it happens to be red. Let this part of the
evidence be denoted F1 = ‘bead drawn at random is red’.
The reader may question the assumption that the robber-bead is drawn at random, and
may rather have an intuitive feeling that this bead is somehow “self-selecting”. Careful
consideration of the problem will, however, convince the reader that this assumption is the
appropriate one. In order to think in terms of probability we must frame our discussion
in the context of a sequence of experiments, and in this context, the assumption that the
first bead is drawn at random is what makes sense.
The question now is: How should F1 affect our belief regarding the number of red
beads in the urn? When the bead we draw at random is seen to be red, we will adjust our
belief about the likely number of red beads in the urn. The only knowledge we have before
picking a red bead is that red beads appear with (unknown) probability pr . Assuming that
N is large and pr is small the distribution of the number of red beads X is approximately
Poisson, with E (X) = λ = Npr and P (F1 ) = E (X) /N = λ/N. Given the evidence F1
the distribution of X may be updated using Bayes’ formula:
P (X = n|F1 ) = P (X = n)
e−λ λn n/N
e−λ λn−1
P (F1 |X = n)
=
=
P (F1 )
n! λ/N
(n − 1)!
(1)
Hence, given the evidence F1 , the updated distribution of X is equal to the Poisson
4
The Collins Case
Figure 1: Collins case, updating the distribution.
P
.
.
.
.
.
..
.....
...
..
. ...
...
... ..
...
.. ..
. .
...
. ......
...
. ....
...
. .
...
. ....
...
. ....
...
.
. ......
...
.
.....
...
.
..
...
.
.. .
...
.
.. ...
...
.
.. .
.
...
.. .
.. ..
.
...
.. ..
.
...
..
.
...
.. ..
.
...
. .
.
...
. 0 .... ..
...
.
.. ...
.
...
.. ..
.
..
...
.
.. ..
...
.. ..
.
...
.. .
.
...
..
.
.. ....
...
.
..
...
.
.. .
.
...
..
.
...
.. .....
.
...
...
.
...
...
...
.
...
..
...
.
...
.
...
...
...
.
..
...
...
.
....
.
.
....
...
....
.
.
...
.....
.
.
.....
.
.....
...
..
.
.....
...
.
.....
.....
.
...
..... .
..
...
..... .
......
.......
...
1
.
......
...
......
..
...
.
....
. ............
.......
.......
.
.
.
.
....
........
.
.......
......
.........
.
.....
...........
......
.
..............
........ .. ....... .....
.
....................
.. ....... ..
...........
.
......................................
..... .......
.
...........................................
...........
............
...............
....................
..................
...
.
P (X = n|F )
P (X = n)
P (X = n|F )
1
2
n
3
distribution for X − 1.1 This is in contrast to the evidence Turner utilizes that may be
formulated F0 =‘there is at least one red bead’. In his case the updated distribution is
simply a rescaling of the a priori distribution
e−λ λn /n!
P (X = n|F0 ) =
,
1 − e−λ
where 1 − e−λ is the probability P (X ≥ 1) . Note that Turner only utilizes part of the
initial evidence, as F0 is a proper subset of F1 . The different updated distributions
are illustrated in Figure 1, using the numbers from the Collins Case which give rise to
λ = 1/6.. The figure shows that the updating based on F0 implies shifting the distribution
upwards by the factor 1/P (X ≥ 1), while the updating based on F1 implies shifting the
distribution to the right by one unit. The fact that updating based on F1 implies shifting
1
This and all the following results also go through also when using the exact binomial formula. I
stick to the Poisson distribution in the following, both because it is simpler and in order to stay close to
Turner’s algebra.
5
The Collins Case
the distribution to the right by one unit is also seen from (1), where it is clear that
P (X = n|F1 ) = P (X = n − 1) .
(2)
The intuitive reason for this result is as follows. When a bead, that happens to be red, is
drawn at random from the urn, the number of red beads among the N −1 remaining beads
is distributed as Bin(N − 1, pr ) . Hence, if the red bead is returned the total number of red
beads in the urn is distributed as 1+Bin(N − 1, pr ). The solution in (2) follows from the
Poisson approximation, when a large N and small pr imply that (N − 1) pr ≈ Npr = λ.
Now a search for the robber-bead is conducted. A search through the urn is carried
out until a red bead, the suspected bead, is found. If no record is made of the time used
or the number of beads screened, no additional evidence is gained in the process of finding
the suspected bead.2 The question of guilt, G, is the question of whether the suspected
bead is identical to the robber-bead. The larger the number of red beads, the lower is
the probability of guilt. For a given number of red beads, X = n, the probability of guilt
is 1/n. When using the updated distribution of X from (1) and calculating the sum over
all n, the probability of guilt is therefore
N
X
1
N
X
1 e−λ λn−1
P (X = n|F1 ) =
⇐⇒
n
n (n − 1)!
n=1
!
ÃN
1 − e−λ
1
1 X e−λ λn
−λ
=
−e
=
P (G|F1 ) =
λ n=0 n!
λ
E (X|X ≥ 1)
P (G|F1 ) =
n=1
Hence, the probability of guilt P (G|F1 ) has an explicit solution and happens to be identical to the inverse of E (X|X ≥ 1). This is different from Turner’s solution P (G|F0 ) =
2
Balding and Donnelly (1995) and Dawid and Mortera (1996) discus more complex search strategies.
The Collins Case
6
E (X −1 |X ≥ 1). Given this particular relationship between the expressions for P (G|F1 )
and P (G|F0 ) it follows from Jensen’s inequality that P (G|F1 ) < P (G|F0 ). This is
confirmed when using the Collins Case example of λ = 1/6 where it turns out that
P (G|F1 ) = 0.9211 while P (G|F0 ) = 0.9587.
3
Discussion
The difference between P (G|F1 ) and P (G|F0 ) illustrates the general point that the analysis of statistical data must take into account exactly how the data was collected. An
interesting question is ”Are there conditions under which Turner’s calculations, based on
P (G|F0 ) , would be justified?” One possible justification could be the assumption that,
by some mechanism inside the urn, only red beads can be picked from the urn (perhaps
due to a minor difference in size between red and other beads). Then, if a red bead is
drawn the only information regarding the distribution is that X ≥ 1 i.e. F0 . In that
case P (G|F0 ) would be the appropriate probability of guilt. In the Collins Case this
assumption is clearly wrong. It would imply that it is only couples with the exact Collinscharacteristics that can commit a robbery. It may be conceivable that such couples are
more likely robbers than other couples, but in that case such evidence should be presented
to the court. As long as such evidence is not presented, the principle of innocence until
proven guilty implies that the evaluation of the evidence should start out from the premise
that all couples are equally likely robbers. Hence, the equal prior likelihood implied by
the ‘beads in an urn’ analogy is appropriate.
The Collins Case
7
References
Balding; David J. and Peter Donnelly (1995) “Inference in Forensic Identification” Journal
of the Royal Statistical Society. Series A (Statistics in Society), 158 (1), 21-53.
Dawid, A. P. (1994) “The Island Problem: Coherent Use of Identification Evidence”
Aspects of Uncertainty. A Tribute to D. V. Lindley, Wiley, New York, 159—170.
Dawid, A. P. and J. Mortera (1996) “Coherent Analysis of Forensic Identification Evidence” Journal of the Royal Statistical Society. Series B (Methodological), 58 (2),
425-443.
Eggleston, R (1983) Evidence, proof and probability 2nd ed. , London: Weidenfeld and
Nicolson.
Turner, T. R. (2002) “The Collins Case: A ‘Beads in Urns’ Model” The Mathematical
Scientist, 27, 80-84.
Yellin, J. (1979) “Review of 0 Evidence, Proof and Probability (by R. Eggleston)” Journal
of Economic Literature, 17, 583-584.