Lecture 11:
Reporting and communicating
Bayes factors
Communication of numerical results where the context is typically
non-numeric
• The end-users of statistical inference are vey often less convenient
with using numbers (quite typical for lawyers)
• Understanding of probabilistic reasoning should not be taken for
granted
• We need tools to demonstrate what a Bayes factor means
• Moreover, Bayes factors are often difficult to assign (estimate) with
high precision
• Reporting the magnitude of a Bayes factor on an ordinal scale is often
to be preferred in opposite to providing its numerical value
The forensic evidence evaluation is made in a context represented
by two competing propositions/hypotheses
Hm : The incriminating proposition (often referred to as the
main proposition)
• can state a specific source of a recovered material
• can state a specific activity that gave rise to the recovered material
• can state that the recovered material belongs to a certain category
(forged pass-ports, ignitable liquids, toxic materials,…)
Ha : The alternative proposition. Comprises the relevant
opposite to Hm
The scale of conclusions used at the Swedish
National Forensic Centre (NFC)
Level
Magnitude of the Bayes
factor/likelihood ratio
Verbal equivalent
+4
at least one million
extremely strong support for Hm
+3
between 6000 and one million
+2
between 100 and 6000
+1
between 6 and 100
support to some extent for Hm
0
between 1/6 and 6
neither support for Hm nor for Ha
–1
between 1/100 and 1/6
–2
between 1/6000 and 1/100
–3
between 1/(one million) and 1/6000
–4
at most 1/(one million)
strong support for Hm
support for Hm
support to some extent for Ha
support for Ha
strong support for Ha
extremely strong support for Ha
Symmetric scale:
The intention is to answer – as far as possible – the question put by the
commissioner. This question is what is usually reformulated as the main
proposition Hm.
Why the limits 6, 100, 6000 and 1 million?
Should the number of levels have been 7 on both sides of zero, then
a logarithmic scale would have been a proper choice.
The levels here were obtained by requiring that
• level +2 should give a posterior probability P(Hm | E ) of at least
0.99 when the prior probability P(Hm) is at least 0.5
• the limit for level +4 should be one million – corresponds with a
reasonable limit for DNA evidence with respect to a Swedish
population
…and by then forcing a logarithmic increase between levels
Pros and cons with an ordinal scale of conclusions compared
to reporting the explicit Bayes factor/likelihood ratio
Pros:
• The use of a scale enables the use of elicited Bayes factors/likelihood ratios
with low precision – it is sufficient to ensure that a certain limit is crossed
• The end-users get used to a limited number of evidentiary “strengths” and do
not have to consider numerical differences from case to case – What is the
significant difference between a Bayes factor of 350 and a Bayes factor of 300
?
• As was previously mentioned, a majority of forensic casework types still lacks
explicit data to assign Bayes factors with a high accuracy
Cons:
• The interval lengths of the scale are by necessity strongly increasing with
level  Large Bayes factors (e.g. around 100000) become underestimated by
just ensuring the lower level
Who is the end-user?
In a substantial part of the forensic literature the enduser is referred to as the court, the jury or the judge
In Sweden, it is not the judge (or the court) that requests the
forensic analysis – it is the leader of the preliminary investigation,
very often a prosecutor.
Hence, who should at first understand the conclusions from a
forensic analysis is the prosecutor – in order to deem on its value for
the prosecution.
When educating and communicating with an end-user…
…it is difficult to centre on the meaning of the Bayes factor/ likelihood ratio!
In general, when a request of a forensic analysis is made, the commissioner
would expect a categorical answer
– Was it this gun that fired the bullet?
– The forensic findings support that the gun fired the bullet (level +2).
– Do you mean that you are not 100 % sure that it was this gun?
– I do not address the proposition, only the forensic findings under each
proposition. They are less probable if there was another gun that fired the
bullet.
– Then, since you have up to +4 in your scale, shall I take it that your +2
means you are 50 % certain that the bullet was fired from this gun?
Instead of always addressing the likelihood ratio in the education,
if we try to address Bayes theorem…
– The reported conclusion +2 means that the forensic findings are at least 100
times more probable if this gun fired the bullet compared to if another gun fired
it. Hence you should multiply your prior odds with 100.
– What is that, prior odds???
– Your prior odds are how much you believe in that it was this gun compared to
how much you believe it was another gun based on other information in your
investigation.
– But what I believe shouldn’t affect your conclusions, should it? My standpoint
is that the conclusion from a forensic scientist should be objective and selfstanding.
– It is objective, yes. But it can never be self-standing when it comes to the
probability that this gun actually fired the bullet. It is always relative to the
propositions.
– I do not follow this at all? Must I be a mathematician to understand your
conclusions?
However scientifically sound Bayes theorem is when it come to
reasoning about evidentiary strength we must never forget that…
• Jurists very often – there are good exceptions though – are
clever people that used to dislike mathematics in school
• It is not yet easy (and not opportune) to quantify the prior
odds of a proposition
• Taking the reasoning up to court level would meet great
cultural objections against the vision that guilt in a case
should be verified using mathematical calculations
• Today – and for quite a long while – the forensic laboratory
cannot come up with accurate likelihood ratios in several
forensic case types to “feed” the mathematical exercise
These points, however, do not prevent us from…
• developing methods for assigning Bayes factors/
likelihood ratios based on rigorous mathematics
• showing theoretically how different choices of prior
odds would affect the final conclusions about the
propositions (e.g. via Bayesian networks)
• trying to alleviate the mathematical exercise by
illustrating it on simpler form
It could be good to start with the ultimate goal of the request for a
forensic analysis:
There is a question: Was it this gun that fired the bullet?
The question is (usually) put because the investigation
indicates that this could be the case.
Re-formulate the question as a proposition:
Hm : This gun fired the bullet!
The ultimate goal is to find the veracity (the posterior
probability) of this proposition…
…without disregarding the indication coming from the
investigation itself!!
Many people would agree on that when trying to prove that
something holds, other possibilities must (successively) be excluded.
Other possibilities are represented by an alternative proposition.
That proposition may be:
Ha : Another gun fired the bullet!
…but it is important that the chosen Ha must comprise all relevant
alternatives to Hm and…
…that the commissioner and the laboratory works
with the same alternative proposition!
Graphical illustration of Bayes theorem
P H h 

P H a 
P E H h 

P E H a 
prior odds
P H h E 
P H a E 
posterior odds
high
high
even
even
Bayes factor, B
low
low
If B is higher than 1
 Upward arrow
 The Findings support
the main proposition Hm
If B is lower than 1
 Downward arrow
 The Findings support
the alternative proposition
Ha
If B is equal to 1
 Horizontal arrow
 The propositions are
equally supported
With one specific pair of main and alternative hypothesis the findings always give
the same Bayes factor, i.e. the same angle of the arrow.
The degree of veracity, however depends on where the arrow starts, i.e. what the
prior odds are.
prior odds
posterior odds
very high
medium high
medium high
even
medium low
low
Depicting the scale of conclusions with the arrow model
Even prior
odds
Level
Degree of veracity
P(Hh | E )
+4 :
+3:
+2:
+1:
> 0.999999
> 0.9998
> 0.99
> 0.86
0:
between 0.14 and
0.86
Lower limit of the
Bayes factor
106  B
6000  B
100  B
6B
(1/6 < B < 6)
When the evidentiary strength is reported in terms of a level of this
scale it may suffice for the commissioner to consider whether the
prior odds are “lower than” , “about equal to” or “higher than” 1.
The magnitude of the posterior probability may still be appreciated.
Nordgaard A., Ansell R., Drotz W., Jaeger L. (2012) Scale of conclusions for the value of
evidence. Law, Probability and Risk 11(1): 1-24.
And when the alternative proposition is changed?...
There may be subsequent arguments for a more narrow proposition:
Ha2 : Another gun of same calibre as the seized gun fired the bullet!
Since the propositions change, the likelihood ratio must change – it
generally decreases with a narrower alternative.
May be difficult to comprehend for someone who has taken the
view that the forensic conclusion is “categorical”.
But if you accept the principle of exclusion, you must admit that
there is a difference between trying to exclude Ha and trying to
exclude Ha2
The eventual degree of veracity may however be less influenced
(sometimes not at all) which can be easily communicated with the
arrow model:
B with Ha
B with Ha2
Not only the likelihood ratio changes with the alternative
proposition, so does the prior odds.