Understanding Science
2. Bayes’ Theorem
© Colin Frayn, 2012
www.frayn.net
Recap
• Assumptions of science
a) Underlying laws
b) Accurate senses
c) Occam’s Razor
• Absolute proof
– Can be achieved with mathematical claims
– Difficult or impossible for scientific laws
• Spectrum of certainty
– Science moves theories on the spectrum
• Scientific Theories
– Empirical models
– Well tested, predictive, falsifiable
© Colin Frayn, 2012
www.frayn.net
Clarifications
• “False” does not imply “completely wrong”
– E.g. Newtonian Physics vs. Relativity
– E.g. the Flat Earth theory, the Spherical Earth theory
• Carl Sagan’s Dragon
– Can we show it doesn’t exist?
– Should we bother?
• Predictive laws versus specific statements
– “There are no dragons”
© Colin Frayn, 2012
www.frayn.net
Introduction
• New evidence arrives
– What does that do?
– Moving around the spectrum of
certainty
• Prior knowledge
– Did you see Elvis?
• When could we call something a
“fact”?
– A scientific fact is “near enough”!
© Colin Frayn, 2012
www.frayn.net
Examples
• On trial for murder
– DNA testing
– Very accurate
– …but a very large population
• A rare disease
– Rare disease or rare misdiagnosis
– Intuition doesn’t help
© Colin Frayn, 2012
www.frayn.net
Organic Gravity – An Example
Organic gravity
”Gravity only acts on organic things”
Vs.
Newtonian gravity
“Gravity acts identically on every type of object”
• Test 1 – drop an apple
– Both theories are equal
• Test 2 – drop a stone
– Newtonian gravity wins
© Colin Frayn, 2012
www.frayn.net
In More Detail
• Let’s look at what we just did
• Test 1 didn’t really help
– It didn’t differentiate
– It provided equal support to each
• Test 2 solved the issue
– Distinguished between the proposals
– Provided support to Newtonian theory
© Colin Frayn, 2012
www.frayn.net
Equal Support
• What do we do when we cannot
distinguish between two possibilities?
• Look at the prior probability of each
• Example: Diagnosing a rare disease
1. The patient has a rare disease
2. The test was wrong
© Colin Frayn, 2012
www.frayn.net
Putting it all together...
Probability of a Hypothesis
given the Evidence
P(H|E)
Depends on...
1. The support that E gives to H
2. The prior probability of H
© Colin Frayn, 2012
www.frayn.net
Finally, Bayes’ Theorem
P (H | E) = P (E | H) * P (H)
P (E)
Prior
Posterior
Support
© Colin Frayn, 2012
www.frayn.net
Evidential Support
• “How much does evidence E support
hypothesis H?”
– P(E|H)/P(E)
• Eating garlic scares away vampires
– Given that I don’t see any vampires
• P(E) = 1
– Vampires don’t exist!
• P(E|H) is also 1
– So test is useless
– That is, it has no differentiating power
© Colin Frayn, 2012
www.frayn.net
Non-discriminating Evidence
P(
|
) = P(
1
|
P(
) * P(
)
Posterior probability is equal to the prior
i.e. We’ve learned nothing whatsoever
© Colin Frayn, 2012
www.frayn.net
)
Priors
• “What is the chance that our hypothesis might be
true ignoring the new evidence?”
– P(H)
• A “flat prior” means “no preference”
– P(H) is the same for all hypotheses
• The “status quo”
– E.g. “Elvis is alive”
– … or any other conspiracy theory
– … or and pseudoscientific claim
© Colin Frayn, 2012
www.frayn.net
Organic Gravity Revisited
• Dropping an apple gave no preference
– P(H) = 0.5 for both
P(Newtonian | Stone Falls) = 1
P(Stone Falls1| Newtonian) * P(Newtonian)
0.5
P(Stone
0.5Falls)
P(Organic | Stone Falls) = 0
0 | Organic) * P(Organic)
P(Stone Falls
0.5
P(Stone
0.5Falls)
© Colin Frayn, 2012
www.frayn.net
Assumptions
• Assumption of completeness
– Don’t have to make this assumption
– Though we do need some way to calculate P(E)
• Assumption that the evidence was
accurate
– Can factor this into P(E|H)
• Assumption that you understand your
models
– Do you really know P(E|H)?
© Colin Frayn, 2012
www.frayn.net
Summary
• Bayes theorem allows us to update
hypotheses in response to evidence
• It evaluates the support that
evidence gives for a hypothesis
• It underlies all of science
© Colin Frayn, 2012
www.frayn.net