Probability and Justice
You have one ticket for the National Lottery
What is the probability of winning a jackpot ?
3
2
6
5
1
4
×
× ×
×
×
49 48 47 46 45 4 4
1
13 983 816
1 in 14 million
Probability and Justice
• Unlikely events (like winning the lottery)
do occur
• When such an event has occurred, what
matters is the relative probabilities of the
different possible explanations
- your lottery win was honest
- you cheated
Probability and Justice
• You win the jackpot but when you go to
claim your winnings …
• … you are arrested
• You are told “1 in 14 million is so
unlikely that you must have cheated”
• You are convicted and sent to prison
What is the error in the argument ?
Probability and Justice
Honest
:
1
:
13 985 816
1
:
Cheating
0
0
Probability and Justice
Probability and Justice
The Prosecutor’s Fallacy
1 in 73
14 million
The probability that the accused is
innocent
=
The probability that the evidence
occurred naturally (ie by chance)
Stated to be the probability of 2 cases of
Sudden Infant Death Syndrome (cot death)
occurring in the same family
So unlikely that the parents must have
committed murder
The prosecutor’s fallacy was used to send
mothers who had suffered the grief of losing
children to prison
1
Probability and Justice
Probability and Justice
SIDS
:
Murder
1
73 000 000
:
1
150 000 000
0.673
Probability and Justice
Where did 1 in 73 million come from ?
A study showed that the probability, p, of
a randomly selected infant suffering
SIDS is given by
1
8543
<
p
<
1
1303
Probability and Justice
• Apart from the prosecutor’s fallacy this
incorporates many other mistakes
- It assumes that cases of SIDS within a
family are independent
- It assumes the worst case probability
- It ignores family size
:
0.327
Probability and Justice
It was argued that therefore the
probability of two such deaths in a family
can be as small as
1
1
1
×
=
8543 8543
72 982 849
1 in 73 m illion
Probability and Justice
• There is growing evidence of genetic links in
cases of SIDS
• It is now suggested that following 1 case, the
probability of a 2nd case is 1 in 200
• If so, for some parents, the probability of
suffering 2 cases of SIDS would be just 1 in 40
thousand.
1
1
1
×
=
200 200
40 000
2
Probability and Justice
Probability and Justice
Who was to blame ?
•
•
•
•
Probability and Justice
Could it happen again ?
Yes, with Shaken Baby Syndrome
Campaigners believe that up to 300
people are either wrongly in prison or
have had their children taken away from
them.
Probability and Justice
The case of Job v Halifax plc
• The withdrawals took place in February
2006
• The case came to court in April 2009
• The judgement was given in June 2009
Social services
The medical profession
The legal system
Our school curriculum
Probability and Justice
The case of Job v Halifax plc
• Mr Job had a total of £2100 withdrawn
from his account from two ATMs in
Reading. 7 transactions took place.
• He denies making or authorising the
withdrawals
Probability and Justice
The case of Job v Halifax plc
The judge decided against Mr Job
“… the absence of a history of fraudulent attacks
on on-line chip and PIN transactions, and the
absence of any evidence of systems failure
indicated that the transactions could be taken
at face value …”
3
Probability and Justice
The case of Job v Halifax plc
• The judge did not reach any conclusions as to
how the withdrawals were made, only that:
- they were made by him, or
- by someone authorised by him, or
- by gross negligence in that he had enabled
someone else to use the card and the third
party knew the PIN
4
Probability and Justice
Roger Porkess
roger.porkess@ mei org.uk
01803 840343
2
Probability and Justice
Introduction
This article looks at some issues which can arise when arguments based on
probability are used in courts of law. It begins with an example based on the National
Lottery.
To take part in the National Lottery on any Wednesday or Saturday, you pay £1 and
select six different numbers between 1 and 49 (inclusive). Six numbers, also between
1 and 49, are then drawn at random, without replacement. If they are the same as your
six numbers, you are a jackpot winner.
Imagine that you are holding your lottery ticket and watching the draw on television.
•
The probability that the first number drawn is one of your 6 numbers is
6
.
49
•
If the first number drawn was one of yours, the probability that the next
5
number drawn is one of your 5 remaining numbers is
.
48
So the probability that you have both of the first two numbers drawn is
6 5
5
× =
.
49 48 392
Similarly the probability that you have all six of the numbers drawn is
6 5 4 3 2 1
1
,
× × × × ×
=
49 48 47 46 45 44 13 983 816
or about 1 in 14 million.
Now imagine this situation. All six of the numbers drawn match your own and so you
should have won the jackpot. However, when you go to claim the prize, you are
arrested. You are told
“1 in 14 million is such a small probability that you must have been cheating.”
How would you defend yourself against that accusation ?
You might, perhaps, say that you are innocent until proved guilty and that, unless
someone can show you cheated, there can be no proof of your guilt. You might also
point out that unlikely events do occur.
Probability and Justice
Roger Porkess
3
Now imagine that these arguments were ignored, that you were taken to court, found
guilty and sent to prison, solely on the grounds that 1 in 14 million is a very small
probability. You would be shocked that such an obvious miscarriage of justice could
happen.
The prosecutor’s fallacy
This example demonstrates the prosecutor’s fallacy. It is based on the following
argument.
The probability that the accused
is innocent
=
The probability of the evidence
occurring naturally, i.e. by chance
This argument is wrong.
In the lottery example:
•
the evidence is that you have matched all six numbers;
•
the probability of the evidence occurring by chance is 1 in 14 million;
•
this is not the probability that you are innocent.
To assess the probability that someone is innocent (or guilty) when an unlikely event
has occurred, you need to accept that the event has occurred, look at the possible
explanations and then compare the probabilities of these explanations.
In the lottery example, there are two possible explanations:
•
you matched the six numbers by chance;
•
you cheated.
Since there is no known method of cheating the lottery, the best estimate of the
probability of that explanation is zero.
So the ratio of the probabilities of the two explanations is
Chance
1
13 983 816
:
Cheating
:
0 .
Since, in this example, you have in fact matched the six numbers, one of these
explanations must be correct. So the total of their actual probabilities must be 1.
Scaling the above ratio gives these probabilities as
Chance
1
Probability and Justice
:
:
Cheating
0.
Roger Porkess
4
In this imaginary example, common sense tells us that sending someone to prison for
winning the lottery is wrong. The probability calculations merely confirm the
obvious. However, the prosecutor’s fallacy has not always been understood by the
courts and this has led to serious miscarriages of justice.
Notice that there is nothing special about the figure of 14 million. The prosecutor’s
fallacy would still be logically incorrect if the probability were 1 in 50 million, 1 in
100 million or 1 in 73 million.
Sudden Infant Death Syndrome
The figure of 1 in 73 million has received considerable publicity in recent years. It
was the figure stated in court by an expert witness as the probability of two cot deaths
occurring naturally in the same family.
The term “cot death” refers to a situation where an apparently healthy baby or infant
goes to sleep and never wakes up. This is described medically as Sudden Infant Death
Syndrome (SIDS) and that term is used in the rest of this article.
In a number of cases over recent years, courts accepted the argument that a
probability of 1 in 73 million was so small that it was safe to find mothers guilty of
murder rather than accepting the alternative explanation of SIDS. The courts thus
made the same mistake, that of accepting the prosecutor’s fallacy, as that in the
example about the lottery. However, there is an important difference; this was real,
not imaginary. Women who had suffered the terrible pain of losing their children were
then subjected to prison sentences.
The figure of 1 in 73 million for two cases of SIDS is, as we will explain, wrong. But
even if it were correct, it should have been compared with the probability of either or
both of the children being murdered by their parents. Sadly some parents do harm,
and even kill, their children and so this probability is not zero. For the sake of
argument, suppose that it is 1 in 150 million.
If these figures were correct, the following calculation should have been presented to
the courts.
Probability and Justice
Roger Porkess
5
1
.
73 000 000
The probability of either or both of the children being murdered by their parents is
1
.
150 000 000
The probability of two cases of SIDS in a family is
So the ratio of the probabilities of the two possible causes is
SIDS
1
73 000 000
:
:
Murder
1
150 000 000
The probability of the correct explanation being SIDS is 0.673.
The probability of the correct explanation involving murder is 0.327.
Clearly a murder conviction based on these figures alone would not be safe.
At the time of writing this article, three of the women who suffered this injustice
(Sally Clark, Angela Cannings and Donna Anthony) have been released on appeal; a
fourth (Trupti Patel) was found not guilty after a long trial.
To the extent that statistical evidence was considered in the successful appeals,
attention was focused more on the unreliability of the figure of 1 in 73 million than on
the original trial courts having failed to take account of the prosecutor’s fallacy.
This figure of 1 in 73 million was based on the findings of a study of infant deaths
between 1993 and 1996, which concluded that the probability of an infant suffering
1
1
and
. Within this range, the probability increases if the
SIDS is between
8543
1303
parents smoke and decreases if they are affluent.
The smallest probability in the range was then used to give the probability of two
cases of SIDS.
1
1
1
1
×
=
≈
8543 8543 72 982 849 73 000 000
While the arithmetic is correct, there are many fundamental errors in the assumptions
underlying this calculation. Three of these are now considered.
Probability and Justice
Roger Porkess
6
1.
Range of values
1
, in the range of
8543
possible values for the probability of a child suffering from SIDS. It is entirely
inappropriate that it should be used as a typical value.
The estimate of 1 in 73 million was based on the extreme value,
It was then argued in court that “In England, Wales and Scotland there are about
700 000 live births a year, so it is saying that by chance that happening will occur
about once every hundred years”. This is another example of invalid reasoning but,
nevertheless, “Once in 100 years” has also become a well known saying.
However, if the other extreme value of the probability had been taken, and the
calculation carried out in the same invalid way, the 100 years would, instead, have
been 2½ years.
Such different figures would have quite different impacts on a jury. Once in 100 years
is genuinely unlikely; once in 2½ years could well be the case before you.
2.
Genetic links
In the case of SIDS, it seems possible that genetic factors are involved. As part of the
evidence at Angela Cannings’s appeal, it was revealed that there had been several
unexplained infant deaths in her extended family, going back over a number of
generations. One of the witnesses in Trupti Patel’s trial was her 80-year old
grandmother who had travelled from India to tell the court that 5 of her 12 children had
died in their first few months.
If, as seems likely, genetic factors are involved, SIDS cases can be expected to run in
families. For such a family, the probability of a case of SIDS is much greater than for
the population as a whole. Certainly the probability of two SIDS cases in such a family
would be very much greater than 1 in 73 million.
3.
Family size
The calculation of 1 in 73 million was based on a family with exactly 2 children.
Suppose, however, that a family has 5 children and that 1 in 8543 is the appropriate
probability. Then, if there are no genetic links, the probability of exactly 2 cases of
SIDS would be given by
2
3
⎛ 1 ⎞ ⎛ 8542 ⎞
C2 ⎜
⎟ ⎜
⎟ .
⎝ 8543 ⎠ ⎝ 8543 ⎠
5
In this expression, the binomial coefficient 5C2 is the number of selections of 2
2
⎛ 1 ⎞
children out of 5, ⎜
⎟ is the probability that 2 of them suffer from SIDS and
⎝ 8543 ⎠
Probability and Justice
Roger Porkess
7
3
⎛ 8542 ⎞
⎜
⎟ is the probability that 3 do not. This works out to be about 1 in 7.3 million
⎝ 8543 ⎠
rather than 1 in 73 million.
Misleading defence arguments
There are two sides to justice. On the one hand, it is clearly very undesirable that an
innocent person is convicted but, on the other hand, we also do not want to allow
guilty people to be acquitted and so given the opportunity to re-offend. The incorrect
use of statistics when considering DNA evidence may lead to an incorrect acquittal. In
that case a serial murderer or rapist could be free to attack further victims.
So it is important to be aware of invalid arguments that might be used by defence
lawyers. One of these, known as the defender’s fallacy, is illustrated in the following
example.
A burglary occurs at a house on an island. A witness says that she saw a red sports car
parked at the house at the time when the burglary was taking place. Ted Jones, who
has a red sports car, is arrested and charged with the crime. At his trial, his defence
lawyer argues “There are 5 red sports cars on this island, so the probability that the
car belonged to the accused is 1 in 5. It is 80% certain that he is innocent”.
One problem with this argument is that it assumes that each of the 5 cars in question
is equally likely to have been the one involved in the burglary. This may not be true.
There may be evidence that places some or all of the other cars elsewhere at the time,
and there may be other evidence that places Ted Jones near the scene of the crime. It
is only if there is no other evidence that it is reasonable to assume equal probabilities.
Understanding probability and statistics
The cases relating to SIDS in this article highlight some of the difficulties that can
arise when statistical arguments are presented in court.
In the original trial of Sally Clark, the judge, in effect, took the view that the members
of the jury were capable of weighing the arguments for themselves. The conduct of
the case was reviewed at the first appeal, which was unsuccessful; the appeal court
took the same view of the jury’s competence. At the second appeal, which was
successful, new medical evidence was presented which undermined the original
conviction and the case collapsed before the use of statistics was raised. However, in
the written judgement for this second appeal, the court took the view that the original
jury should not have been expected to understand the statistical arguments.
In Angela Cannings’s original trial, the defence and prosecution agreed not to
introduce statistics at all so as to avoid the possibility of confusing the jury.
Any suggestion that a jury is unable to follow an argument because it involves
statistics is deeply worrying. It means that justice may not be done.
Probability and Justice
Roger Porkess
8
It is the responsibility of the judge to ensure that technical matters are explained in a
way that is open to the jury and so it is crucially important that judges are competent
to carry out this role.
For many years now, all school students have learnt some probability and statistics.
There are many reasons why this is important. These include developing the ability to
be a competent jury member should the need arise.
Could it happen again ?
The cases of the mothers whose children suffered SIDS raise the question of whether
there are other comparable groups of people who have been convicted for crimes that
did not occur.
In at least one case the answer would seem to be “Yes”. The term Shaken Baby
Syndrome, or SBS, was coined in the 1970s to describe a particular type of injury,
involving bleeding within the brain, suffered by mistreated babies. However, similar
injuries can have other causes, like falling off a settee.
In the courts, one expert witness claimed that in 95% of cases the baby had been
mistreated. Although this figure was not supported by any data, it has become widely
accepted, just like the 1 in 73 million. (A mistreated baby will usually show other
symptoms, making it possible to distinguish between cases of systematic mistreatment
and accidents or natural causes; many experts now believe that the correct figure is
much lower.)
As a result of the 95% figure, many parents who took sick children into hospital found
themselves charged with mistreating them. When parents protested their innocence,
they were sometimes told “Denial is highly indicative of abuse” (the words of a
well known American paediatrician).
Some of the babies in question died and in some of those cases their parents are still
in prison, found guilty of murder. Other babies survived and in those cases their
parents’ trials were held in the secret family courts rather than the open public courts
used for murder trials. Consequently records of the proceedings are not available to
the public and so cannot be questioned. A common outcome was that children were
taken away from their parents. Many such children now live with foster or adoptive
parents, and will never be returned to their natural parents, even if they are entirely
innocent.
Those committed to supporting these unfortunate parents claim that about 300 adults
are currently suffering injustice in the name of SBS, some in prison falsely convicted
of murder and others separated from their children.
Probability and Justice
Roger Porkess
9
Probability and Justice
Questions
Probability and Justice
Roger Porkess
10
Questions
1.
An island in the Pacific Ocean runs a lottery which is organised on the same
principles as that in the UK. However, instead of choosing six numbers
between 1 and 49, contestants choose four numbers between 1 and 100
(inclusive). Find the probability that, in this lottery, a particular selection of
four numbers wins the jackpot.
2.
Show how to obtain the figures 0.673 and 0.327 towards the top of page 5.
3.
Show how the figure of 2½ years in line in paragraph 3 of page 6 was
obtained.
4.
Each of the 6 children in a family has, independently, a probability of
1
of contracting a hereditary disease.
120
Use the method illustrated in the section on Family Size on page 6 to find the
probability that exactly 4 of the 6 children contract the disease.
5.
A specimen of blood of type AB is found at the scene of a crime on an island
with a population of 400 adults. The police have records of the blood types of
a few of those living on the island, one of whom, Jed Smith, also has blood of
type AB. The police arrest Jed Smith for the crime and the case comes to trial.
At the trial, the prosecution make the following statements.
“Only three in a hundred people have blood from group AB. So the probability
that Jed Smith is innocent is only 1 in 33.3.”
6.
(i)
Identify the fault in this argument
(ii)
Show how the defence could argue that the probability of Jed Smith’s
1
.
guilt is
12
(iii)
What sort of evidence could the prosecution present to argue that the
1
probability of Jed Smith’s guilt is greater than
?
12
Explain why the statement “Denial is highly indicative of abuse” on page 8
should not be used in a fair trial.
Probability and Justice
Roger Porkess
11
Answers
4
3 2 1
1
× × × =
100 99 98 97 3 921 225
1.
2.
The ratio of the probabilities is
1
:
73 000 000
This is the same ratio as
150
:
223
or
1
150 000 000
73
223
0.6726… : 0.3273…
which round to the given numbers.
1
1
×
× 700 000 = 0.412...
1303 1303
3.
1
= 2.425... ≈ 2 12
0.412...
4
⎛ 1 ⎞ ⎛ 119 ⎞
C4 ⎜
⎟ ⎜
⎟
⎝ 120 ⎠ ⎝ 120 ⎠
2
6
4.
0.000 000 0711
5. (i)
The prosecutor’s fallacy.
(ii) “With a population of 400 adults on the island, you would expect 12 people to
have type AB blood. So the probability that the blood at the scene of the
1
.”
crime came from Jed Smith is
12
(iii) Either
Evidence that places Jed Smith near the scene of the crime at
the time of the crime
Or
Evidence that provides alibis for other people with blood group AB.
6.
If the accused denies mistreating a child this would be taken to indicate guilt.
If the accused admits mistreating the child this too would imply guilt.
So the accused is being assumed to be guilty.
Probability and Justice
Roger Porkess