What is Random?
“The two philosophies have different views of what is random. To the frequentist, the
population parameters are fixed (but unknown), and the observed data is random, with
sampling distributions that give the probabilities of observing various outcomes based on the
values of certain population parameters. But in the Bayesian view, the observed data is fixed
(after all, we know what we saw); it’s the population parameters that are random and have
probability distribution functions associated with them based on the observed outcomes.”
Spades vs. Shovels
“Using a spade for some jobs and shovel for others does not require you to sign up to a lifetime
of using only Spadian or Shovelist philosophy, or to believing that only spades or only shovels
represent the One True Path to garden neatness.”
Prosecutor’s Fallacy
“Mixing up P(A|B) with P(B|A) is the Prosecutor’s Fallacy; a small probability of evidence given
innocence need NOT mean a small probability of innocence given evidence.”
____
Hoff 2009
Chapter 1
Y = “sample space”; the set of all possible datasets
y = subset of the population; a dataset.
 = “parameter space”; the set of all possible population characteristics
 = “numerical values of population characteristics”
“The idealized form of Bayesian learning begins with a numerical formulation of joint beliefs
about y and , expressed in terms of probability distributions over Y and .”
(p.8) Where n is the sample size and w is the confidence in our prior beliefs. For theta-hat
(estimator of theta), as the sample size n increases, we see that our estimation increasingly
centers on y-bar (the sample mean), and less on theta-naught (our prior beliefs). This makes
total sense; with large sample sizes, we don’t need Bayesian approaches as much, since the
sample mean increasingly represents the population mean. One problem with OLS methods is
that there is no means of detecting when the sample size is too small to make predictions.
Chapter 2
(p.16) “Bayes’ rule does not determine what our beliefs should be after seeing the data, it only
tells us how they should change after seeing the data.”
(p. 29)