How do you make an inference example?
Inference is using observation and background to reach a logical conclusion. You probably practice
inference every day. For example, if you see someone eating a new food and he or she makes a face,
then you infer he does not like it. Or if someone slams a door, you can infer that she is upset about
something.
What is inference?
Inference may be defined as the process of drawing conclusions based on evidence and reasoning. It lies
at the heart of the scientific method, for it covers the principles and methods by which we use data to
learn about observable phenomena. This invariably takes place via models. Much of science is modelbased, meaning that we construct a model of some phenomenon and use it to make predictions of the
data we expect to observe under certain conditions. By comparing predictions with the actual data, we
can determine how well the model explains the data and hence the phenomenon. This may lead us to
reject entirely some models, to improve (and then reassess) others, and perhaps finally to declare one
as the "best" model (so far). Models are constructed using accepted theoretical principles, prior
knowledge and expert judgement. Inference is the process by which we compare the models to the data.
This normally involves casting the model mathematically and using the principles of probability to
quantify the quality of match.
By way of illustration, let us consider -- somewhat anachronistically -- models for the motion of the
planets in the solar system as seen from the Earth's surface. Two prominent alternatives are (1) the
geocentric model, in which the the Earth is considered fixed and the Sun, Moon and planets orbit
around it, and (2) the heliocentric model, in which the Sun is considered fixed and the Earth and planets
orbit around it (the Moon continues to orbit the Earth). Ignoring the specific details of the various
flavours of these general models (Aristotelean, Ptolemaic, Copernican, Keplerian etc.), there is actually a
form of equivalence of the two models, because one can be transformed into the other via a noninertial coordinate transformation. But when defined in their natural coordinates, these two models
look very different.
Inference can typically be divided into two parts: model fitting and model comparison. In one flavour of
the geocentric model, the planets move on regular circular orbits with the Sun at the center. Several
parameters describe the motion of each planet (radius, period, inclination, phase). Model fitting is the
process by which the values of these parameters are determined from a set of observational data. As all
data are noisy to a greater or lesser degree, this involves uncertainty, and this is best quantified using
probability. That is, uncertainty in the data corresponds to uncertainty in the model parameters. The
most general method of inference is to determine the probability distribution function (PDF), P(θ | D, M),
where D denotes the data and θ the parameters of model M. This is generally a multidimensional PDF
and cannot usually be evaluated analytically. The arsenal of inference includes numerous numerical
tools for evaluating it, Markov Chain Monte Carlo (MCMC) methods being one of the most popular. This
is time consuming, however, and approximate techniques for determining summary information (such
as the mean and covariance) are sometimes employed. Depending on both the quantity and quality of
the data, as well as the suitability of the model, P(θ | D, M) may be more or less peaked around a
narrow range of parameters, indicating a well-determined, low uncertainty solution for that model.
We will usually want to know how good a model is, either in a general sense or at some specific fitted
values of the parameters. This is actually an ill-posed question if we only consider one model, because
we then have no alternative which could explain the data better, so any data must be ascribed to our
one-and-only model. Fundamentally, therefore, we actually always compare models and try to identify
the "best" one (according to some criteria). At the minimum we at least consider a "background" model
as an implicit alternative. Taking the example of detecting an emission line in a spectrum where we have
just one model for the location and shape of the line, an implicit alternative model is no line, e.g. just a
constant spectrum. But often we will have other alternative models, e.g. with multiple lines, or lines
with different shapes.
We rarely believe that our models are perfect, so it actually makes no sense to ask whether a particular
model is the "correct" one. This is reinforced by the fact that data are noisy - have a random component
which cannot be predicted by a model - so we expect some deviation between data and predictions.
Therefore, we can only quantify the relative quality of models (including a possible background model).
We cannot establish the absolute quality of a model. Incidentally, model comparison is often performed
poorly in the literature, in part through the use of over-simplified, straw-man background models, the
weakness of which helps to artificially promote almost any other model.
This brings us to the second pillar of inference, model comparison, the goal of which is to identify which
of the models under consideration best explains the data. Returning to the example of planetary orbits,
consider that we have a set of observations of planetary positions (two-dimensional sky-coordinates) at
known dates. A good geocentric model can actually predict most possible observations very well; better,
in fact, than a heliocentric model with circular orbits. More generally, because we can geometrically
transform the predictions of a heliocentric model into those of a geocentric model, both models could
be equally good at predicting these data. By making a geocentric model more complex (add more
epicycles) we make make it fit the data better and better. (Think of fitting a curve to ten points in a twodimensional space: Unless the points are collinear, a cubic curve can always be adjusted to fit better -- in
terms of sum of square residuals -- than a straight line.) If we have additional reasons to prefer a
geocentric model (such as lack of stellar parallaxes, Aristotle or biblical interpretations), then the
geocentric model appears to be favoured.
Yet there is something important missing from this chain of reasoning. We know that increasingly
complex models can be made to fit any data set, but we consider such models to be increasingly
contrived. Thus in addition to predictive power, consideration of the plausibility of models must also be
a fundamental part of inference. Plausibility is often (but not always) equated with parsimony, in which
case we adopt what is often called Occam's razor: we should prefer a simple solution when a more
complicated one is not necessary. We should therefore apply some kind of "complexity control" to our
models. In terms of the historical development of theories of planetary motion, the shift in preference
from a geocentric to a heliocentric model was not just due to the improved data. It was also due to a
increased willingness to question the assumption that the Earth could not move, as well as the choice to
give more weight to the fact that the Sun's motion is suspiciously synchronized with the motion of the
planets in the geocentric model. The former was a consequence of an intellectual revolution reaching
far beyond astronomy; the latter was essentially a plausibility argument. Both demonstrate the
unavoidable importance to the process of inference of prior information, that information which goes
beyond just the data we are explicitly using in the modelling.
The preceding description of inference is the Bayesian one. This is the only self-consistent, logical
approach to inference based on probability. A probabilistic approach is essential, because dealing with
observational data means dealing with uncertainty: Data are noisy (we cannot measure planetary
positions exactly) and our samples are incomplete (we cannot measure all points in an orbit). Probability
is arguably the most powerful means of dealing with uncertainty. Some scientists take issue with priors,
but this does not to deny their existence. It just highlights the difficulty we face in practice of
encapsulating prior information in terms of probabilities, which is a scientific problem to be tackled, not
to be shied away from.
Bayesian inference has undergone a significant renaissance in astronomy in the past twenty years. This
is partly due to the increase of available computing power, for one frequently has to perform highdimensional numerical integration. Once methods for doing this became tractable, astronomers became
aware of the need for self-consistent, logical data analysis, as opposed to quick, simple -- but often
wrong -- statistical tests or recipes. Doing inference properly is a vitally important area in all of science,
but in particular in astronomy where, lacking the ability to perform experiments or obtain in situ data,
we are limited to remote observations. Vast amounts of money, time and effort are invested into
building powerful instruments, so a commensurate effort should be made to ensure that sensible things
are done with the data. Unfortunately this is not always the case, and many publications in the literature
draw incorrect conclusions because of flawed inference.
This is not always due to ignorance or even lack of effort. The principles of inference may be straight
forward, but the practice is considerably more complex: What models should I consider? How do I set
up and parametrize these models? Which data do I take into account? What is the appropriate noise
model? How do I define my priors and test the sensitivity to these? How do we measure model
complexity or plausibility? How do I explore a high dimensional posterior PDF in an acceptable time
scale? What new data should I acquire in order to help distinguish between models I have tested? How
do I use the results of the analysis to improve the models or propose new ones? These are questions
which can only be answered in the context of specific problems, and are the focus of applied research in
inference.
Coryn Bailer-Jones
March 2012