Simplicity as a
Surrogate
John D. Norton
Department of History and Philosophy of Science
Center for Philosophy of Science
University of Pittsburgh
Center for Philosophy of Science
University of Pittsburgh, November 27, 2012.
1
The Claim of this Talk
In so far as it has any
epistemic power…
Simplicity is a surrogate
for background facts or
assumptions that warrant the
relevant inductive inference.
Application of the
Elliot Sober has been
material theory of
induction to simplicity.
defending this view of
simplicity for decades.
2
How it
works
3
Bird Tracks
What caused these tracks?
One bird walking?
Two coordinated one-legged
birds hopping?
Many one-legged birds
touching down just once?
4
Simplicity as an Epistemic Criterion
Rule I. We are to admit no more causes of
natural things than such as are both true and
sufficient to explain their appearances.
To this purpose the philosophers say that Nature does
nothing in vain, and more is in vain when less will
serve; for Nature is pleased with simplicity, and affects
not the pomp of superfluous causes.
Rule II. Therefore to the same natural effects we
must, as far as possible, assign the same causes.
As to respiration in a man and in a beast; the descent of
stones in Europe and in America; the light of our
culinary fire and of the sun; the reflection of light in the
earth, and in the planets.
Isaac Newton, Rules of
reasoning in philosophy
“Nature is pleased with simplicity,
and affects not the pomp of
superfluous causes.”
…ONE bird.
5
Bird Tracks again
What caused these tracks?
One bird walking a lot?
Many birds birds each
walking a little?
Rule II. Therefore to the
same natural effects we
must, as far as possible,
assign the same causes.
…ONE bird?
…MANY birds?
(in ONE flock).
6
Background knowledge…
…is what
really decides,
but we use
simplicity talk
to avoid
having to
explain lots of
little details.
ONE bird
since we know that
coordinated one-legged
birds hopping are very rare.
MANY birds in a flock
since we know that single
birds do not like to walk
about a lot.
7
A Brief Farewell to the
Metaphysics of
Simplicity
8
Nature is Simple
“…I would like to state a proposition
that at present cannot be based upon
anything more than upon
a faith in the simplicity, i.e.,
intelligibility, of nature: there
are no arbitrary constants of this
kind…”
Autobiographical Notes.
Our experience hitherto justifies us in
believing that nature
is the
realization of the simplest
conceivable mathematical
ideas.”
On the Methods of Theoretical Physics, 1933.
9
Nature is NOT Simple.
The term “simple” is vague. No
continuum gas
single meaning broad enough to
support a universal metaphysics of
simplicity.
Ontic simplicity: fewest entities.
Aesthetic judgments of
simplicity are made post hoc
and reflect the achievement of
comfort with a new theory.
Descriptive simplicity.
molecular gas
one entity
1023 entities
infinitely
many parts
finitely many
parts
General relativity
in 1920
"...the complications of the
theory of relativity are
altogether too much...I fear it
will always remain beyond
my grasp..."
General relativity
in 1973
“Einstein’s theory of
gravity is simple; Newton’s
is complex.”
Misner, Thorne and Wheeler,
1973
Hale, 1920
Nature is NOT NOT Simple, either.
10
Curve Fitting
11
Hierarchy of Functions
quartic
cubic
quadratic
linear
constant
Real least
squares fit
to the data.
Choose the simplest that works.
12
Distinct projects
Data Compression
The mark of truth
Present experimental data in a
compact usable form. Most
engineering uses of curve fitting.
versus
Search
More efficient to
check the simpler
hypotheses first,
independently of
whether the truth is
simple or not.
Simplicity is pragmatic,
not an indicator of truth.
Simplicity strips away confounding
error noise to reveal truth.
13
Background Assumptions make simplicity is a mark of truth.
I. Error model holds
error
laden
data
= true + error
curve
Fails in data compression in
engineering applications.
There may be no true curve.
y=
Fails for density of primes
true =
data
error
laden
curve
+ error
density
of
primes in
0 to x
Background Assumptions make simplicity a mark of truth.
Reparametrize
II. The right
parameterization
is used.
III. Order
hierarchy matches
the strength, likelihood of
processes, causes.
1, x, x2, x3, x4, x5, x6, x7, …
rescale z = x3
1,
z,
z2, …
The right parametrization welladapted to the true processes.
For cyclic processes,
first fit periodic function
sin (t) = x – (1/3!) t3 + (1/5!) t5 - …
before any finite order polynomial in t.
Simplicity in curve fitting is a surrogate for these background assumptions.
15
II. and III. Combined.
Reparameterize
same data with
z = sin-1x
Data generated by true curve y=x
True curve
y = sin z = z – (1/3!)z3 + (1/5!)z5 - …
cannot be found in finite ascent of
polynomial hierarchy.
16
Curve Fitting
Illustrated
17
Fitting trajectories to planets, comets…
Background assumptions
Fit ellipse,
hyperbola,
parabola.
(Not straight
line.)
Fit ellipse whose
elements change
with time.
Advancing perihelion
Newton’s theory of gravity holds.
Object deflected by sun.
No other object exerts a perceptible
deflecting force.
There must be another object
deflecting.
1846: successful prediction of Neptune
for perturbations in Uranus.
1915: anomalous motion of Mercury
explained by general relativity.
Background assumption fails.
18
Harmonic analysis of tides: the toy theory
19
Harmonic analysis of tides: the real theory
Joe S. Depner, “Mathematical Description of Oceanic Tides,” 2012
20
Physical Basis of 37 Harmonic Constituents Fitted
21
Model Selection
22
Which Model?
cubic
quadratic
linear
constant
Less simple models
eventually perform
better by overfitting
= conforming to error
noise.
23
Akaike Information Criterion
cubic
quadratic
Which
model?
Unbiased estimator
of average
performance of
fitted curve,
distribution over all
data sets
“Performance”
= log likelihood of data
linear
constant
=
Performance
of fitted curve,
distribution on
particular data
set at hand
inflated by
overfitting
-
Dimension of
model
containing
fitted curve,
distribution
(lack of)
simplicity
penalty
24
Akaike Information Criterion
No posit of simplicity
or principle of
parsimony is assumed.
The bias correction follows
from ordinary statistical
modeling.
No general principle of
parsimony is derived.
Results hold only for those
systems presumed.
Simplicity
description
is an
imprecise
surrogate
for
The analysis could
proceed without any overt
talk of simplicity.
We introduce it since we find it
a comfortable way to describe
Akaike’s very simple formula.
the precise
procedure of
bias correction.
25
Values, Virtues…
26
Accuracy, Consistency, Scope, Simplicity,
Fruitfulness…Explanatory Power
Are they properly called…
Criteria?
for theory choice that
might lead us to the truth
Virtues, values?
of theories selected by the
scientific community.
Sought because they
might lead us to the truth.
Prized as ends in
themselves.
Whether they do this is a
matter of further analysis.
Virtues, values are
endpoints of analysis.
Whether they do is imposed
on us by the external world.
Virtues, values are agreed
upon by social convention.
“Virtues, values” encodes a skepticism that the
criteria are not guides to the truth.
“Criteria” is neutral.
27
Does the Difference Really Matter?
virtues,
values
criteria
“science and values”
“science,
criteria for theory choice
and ethical values”
“Objectivity, Value
Judgment, and
Theory Choice.”
“Objectivity, CriteriaBased Judgment and
Theory Choice.”
28
Conclusion
29
The Claim of this Talk
In so far as it has any
epistemic power…
Simplicity is a surrogate
for background facts or
assumptions that warrant the
relevant inductive inference.
Application of the
Elliot Sober has been
material theory of
induction to simplicity.
defending this view of
simplicity for decades.
30
31