Bayesian Statistics, Modeling & Reasoning
What is this course about?
P548: Intro Bayesian Stats with Psych Applications
Instructor: John Miyamoto
01/04/2016: Lecture 01-1
Note: This Powerpoint presentation may contain macros that I wrote to help me
create the slides. The macros aren’t needed to view the slides. You can disable or
delete the macros without any change to the presentation.
Outline
• What is Bayesian inference?
• Why are Bayesian statistics, modeling & reasoning
relevant to psychology?
• What is Psych 548 about?
• Explain Psych 548 website
• Intro to R
• Intro to RStudio
• Intro to the R to BUGS interface
Psych 548, Miyamoto, Win '16
Lecture probably
ends here
2
Bayes Rule – What Is It?
• Reverend Thomas Bayes, 1702 – 1761
English Protestant minister & mathematician
• Bayes Rule is fundamentally important to:
♦
Bayesian statistics
♦
Bayesian decision theory
♦
Bayesian models in psychology
P Data|Hypothesis P(Hypothesis)
P(Hypothesis|Data) =
P(Data)
P(Data) 
n
 P  Data | Hypothesisi  P  Hypothesisi 
i 1
Psych 548, Miyamoto, Win '16
Bayes Rule – Why Is It Important?
3
Bayes Rule – Why Is It Important?
• Bayes Rule is the optimal way to update the probability of hypotheses
given data.
• The concept of "Bayesian reasoning“: 3 related concepts
♦
Concept 1: Bayesian inference is a model of optimal learning
from experience.
♦
Concept 2: Bayesian decision theory describes optimal strategies
for taking actions in an uncertain environment.
Optimal gambling strategies.
♦
Concept 3: Bayesian reasoning represents the uncertainty of events
as probabilities in a mathematical calculus.
• Concepts 1, 2 & 3 are all consistent with the use of the term,
"Bayesian", in modern psychology.
Psych 548, Miyamoto, Win '16
Bayesian Issues in Psychology
4
Bayesian Issues in Psychological Research
• Does human reasoning about uncertainty conform to Bayes Rule?
Do humans reason about uncertainty as if they are manipulating
probabilities?
♦
These questions are posed with respect to infants & children,
as well as adults.
• Do neural information processing systems (NIPS) incorporate
Bayes Rule?
• Do NIPS model uncertainties as if they are probabilities.
Psych 548, Miyamoto, Win '16
Four Roles for Bayesian Reasoning in Psychology Research
5
Four Roles for Bayesian Reasoning in Psychology
1. Bayesian statistics: Analyzing data
♦
E.g., is the slope of the regression of grades on IQ the same for boys as
for girls?
♦
E.g., are there group differences in an analysis of variance?
Psych 548, Miyamoto, Win '16
Four Roles …. (Continued)
6
Four Roles for Bayesian Reasoning in Psychology
1. Bayesian statistics: Analyzing data
2. Bayesian decision theory – a theory of strategic action.
How to gamble if you must.
3. Bayesian modeling of psychological processes
4. Bayesian reasoning – Do people reason as if they are
Bayesian probability analysts? (At macro & neural levels)
♦
Judgment and decision making – This is a major issue.
♦
Human causal reasoning – is it Bayesian or quasi-Bayesian?
♦
Modeling neural decision making – many proposed models have a
strong Bayesian flavor.
Psych 548, Miyamoto, Win '16
Four Roles …. (Continued)
7
Four Roles for Bayesian Reasoning in Psychology
1. Bayesian statistics: Analyzing data
2. Bayesian decision theory – a theory of strategic action.
How to gamble if you must.
3. Bayesian modeling of psychological processes
4. Bayesian reasoning – Do people reason as if they are
Bayesian probability analysts? (At macro & neural levels)
Psych 548:
Focus on Topics (1) and (3).
Include a little bit of (4).
Psych 548, Miyamoto, Win '16
Graphical Representation of Psych 548 Focus on Stats/Modeling
8
Graphical Representation of Psych 548
Psych 548
Bayesian Statistics
& Modeling:
R & JAGS
Psych 548, Miyamoto, Win '16
Bayesian Models
in Cognitive Psychology
& Neuroscience
Graph & Text Showing the History of S, S-Plus & R
9
Brief History of S, S-Plus, & R
• S – open source statistics program
created by Bell Labs (1976 – 1988 – 1999)
Ancestry of R
• S-Plus – commercial statistics program,
S
refinement of S (1988 – present)
• R – free open source statistics
program (1997 – present)
♦
currently the standard computing
framework for statisticians worldwide
Many contributors to its development
♦
Excellent general computation.
Powerful & flexible.
Great graphics.
Multiplatform: Unix, Linux, Windows, Mac
User must like programming
♦
♦
♦
Psych 548, Miyamoto, Win '16
S-Plus
R
BUGS, WinBUGS, OpenBUGS, JAGS
10
BUGS, WinBUGS, OpenBUGS & JAGS
• Gibbs Sampling & Metropolis-Hastings Algorithm
Two algorithms for sampling from a hard-to-evaluate probability
distribution.
“BUGS” includes all of these.
• BUGS – Bayesian inference Under Gibbs Sampling (circa 1995)
• WinBUGS - Open source (circa 1997)
♦
Windows only
• OpenBUGS – Open source (circa 2006)
♦
Mainly Windows. Runs within a virtual Windows machine on a Mac.
• JAGS – Open source (circa 2007)
♦
Multiplatform: Windows, Mac, Linux
• STAN – Open source (circa 2012)
○
Multiplatform: Windows, Mac, Linux
Psych 548, Miyamoto, Win '16
Basic Structure of Bayesian Computation with R & OpenBUGS
11
Basic Structure of Bayesian Computation
rjags
runjags
R
JAGS
rjags functions
data preparation
rjags functions
analysis of results
BRugs
R2WinBUGS
rstan
R
Psych 548, Miyamoto, Win '16
BRugs functions
Brugs functions
Computes
approximation to the
posterior distribution.
Includes diagnostics.
OpenBUGS/
WinBUGS/
Stan
Outline of Remainder of the Lecture: Course Outline & General Information
12
RStudio
• Run RStudio
• Run R from within RStudio
Psych 548, Miyamoto, Win '16
13
Remainder of This Lecture
• Take 5 minute break
• Introduce selves
• Psych 548: What will we study?
• Briefly view the Psych 548 webpage.
• Introduction to the computer facility in CSSCR.
• Introduction to R, BUGS (OpenBUGS & JAGS), and RStudio
Psych 548, Miyamoto, Win '16
5 Minute Break
14
5 Minute Break
• Introduce selves upon return
Psych 548, Miyamoto, Win '16
Course Goals
15
Course Goals
• Learn the theoretical framework of Bayesian inference.
• Achieve competence with R, OpenBUGS and JAGS.
• Learn basic Bayesian statistics
♦
Learn how to think about statistical inference from a Bayesian standpoint.
♦
Learn how to interpret the results of a Bayesian analysis.
♦
Learn basic tools of Bayesian statistical inference - testing for convergence,
making standard plots, examing samples from a posterior distribution.
--------------------------------------------------------------Secondary Goals
♦
Bayesian modeling in psychology
♦
Understand arguments about Bayesian reasoning in the psychology
of reasoning. The pros and cons of the heuristics & biases movement.
Psych 548, Miyamoto, Win '16
Kruschke Textbook
16
Kruschke, Doing Bayesian Data Analysis
Kruschke, J. K. (2014). Doing bayesian data analysis, second edition: A
tutorial with R, JAGS, and Stan. Academic Press.
• Excellent textbook – worth the price ($90 from Amazon)
• Emphasis on classical statistical test problems from a Bayesian
perspective. Not so much modeling per se.
♦
Binomial inference problems, anova problems, linear regression problems.
Computational Requirements
• R & JAGS (or OpenBUGS)
• A programming editor like Rstudio is useful.
Psych 548, Miyamoto, Win '16
Chapter Outline of Kruschke Textbook
17
Kruschke, Doing Bayesian Data Analysis
• Ch 1 – 4: Basic probability background (pretty easy)
• Ch 5 – 8: Bayesian inference with simple binomial models
♦
Conjugate priors, Gibbs sampling & Metropolis-Hastings algorithm
♦
OpenBUGS or JAGS
• Ch 9 – 12: Bayesian approach to hierarchical modeling,
model comparison, & hypothesis testing.
• Ch 13: Power & sample size (omit
)
• Ch 14: Intro generalized linear model
• Ch 15 – 17: Intro linear regression
• Ch 18 – 19: Oneway & multifactor anova
• Ch 20 – 22: Categorical data analysis, logistic regression, probit
regression, poisson regression
Psych 548, Miyamoto, Win '16
Lee & Wagenmakers, Bayesian Graphical Modeling
18
Bayesian Cognitive Modeling
Lee, M. D., & Wagenmakers, E. J. (2014). Bayesian cognitive modeling:
A practical course. Cambridge University Press.
♦
Michael Lee:
♦
E. J. Wagenmaker: http://users.fmg.uva.nl/ewagenmakers/BayesCourse/BayesBook.html
♦
Equivalent Matlab & R code for book are available at the
Psych 548 website and at Lee or Wagenmaker's website.
http://www.socsci.uci.edu/~mdlee/bgm.html
• Emphasis is on Bayesian models of psychological processes
rather than on methods of data analysis. Lots of examples.
Psych 548, Miyamoto, Win '16
Chapters in Lee & Wagenmakers
19
Table of Contents in Lee & Wagenmakers
Psych 548:, Miyamoto, Win ‘16
Computer Setup in CSSCR
20
CSSCR Network & Psych 548 Webpage
• Click on /Start /Computer.
The path & folder name for your Desktop is:
C:\users\NetID\Desktop
(where "NetID" refers to your
NetID)
• Double click on MyUW on your Desktop.
Find Psych 548 under your courses and
double click on the Psych 548 website.
Is this information
obsolete?
• Download files that are needed for today's class.
Save these files to
♦
C:\users\NetID\Desktop
Note that Ctrl-D takes you to your Desktop.
• Run R or RStudio.
Psych 548, Miyamoto, Win '16
Psych 548 Website - END
21
Psych 548 Website
• Point out where to download the material for today’s class
• Point out pdf’s for the textbooks.
Psych 548, Miyamoto, Win '16
NEXT: Time Permitting ......
22
General Characteristics of Bayesian Inference
• The decision maker (DM) is willing to specify the prior probability of the
hypotheses of interest.
• DM can specify the likelihood of the data given each hypothesis.
• Using Bayes Rule, we infer the probability of the hypotheses given the
data
Psych 548, Miyamoto, Win '16
Comparison Between Bayesian & Classical Stats - END
23
How Does Bayesian Stats Differ from Classical Stats?
Bayesian: Common Aspects
Classical: Common Aspects
• Statistical Models
• Statistical Models
• Credible Intervals
– sets of
• Confidence Intervals
–
which parameter values are tenable after
viewing the data.
parameters that have high posterior
probability
Bayesian: Divergent Aspects
Classical: Divergent Aspects
• Given data, compute the full posterior
probability distribution over all
parameters
• No prior distributions in general, so
this idea is meaningless or selfdeluding.
• Generally null hypothesis testing is
nonsensical.
• Null hypothesis te%sting
• Posterior probabilities are
meaningful; p-values are half-assed.
• MCMC approximations are
sometimes useful but not for
computing posterior distributions.
• MCMC approximations to posterior
distributions.
Psych 548, Miyamoto, Win '16
• P-values
Sequential Presentation of the Common & Divergent Aspects
24
How Does Bayesian Stats Differ from Classical Stats?
Bayesian: Common Aspects
Classical: Common Aspects
• Statistical Models
• Statistical Models
• Credible Intervals
– sets of
parameters that have high posterior
probability
• Confidence Intervals
–
which parameter values are tenable after
viewing the data.
Bayesian: Divergent Aspects
Classical: Divergent Aspects
• Given data, compute the full posterior
probability distribution over all
parameters
• No prior distributions in general, so
this idea is meaningless or selfdeluding.
• Generally null hypothesis testing is
nonsensical.
• Null hypothesis te%sting
• Posterior probabilities are
meaningful; p-values are half-assed.
• MCMC approximations are
sometimes useful but not for
computing posterior distributions.
• MCMC approximations to posterior
distributions.
Psych 548, Miyamoto, Win '16
• P-values
END
25
How Does Bayesian Stats Differ from Classical Stats?
Bayesian: Common Aspects
Classical: Common Aspects
• Statistical Models
• Statistical Models
• Credible Intervals
– sets of
parameters that have high posterior
probability
• Confidence Intervals
–
which parameter values are tenable after
viewing the data.
Bayesian: Divergent Aspects
Classical: Divergent Aspects
• Given data, compute the full posterior
probability distribution over all
parameters
• No prior distributions in general,
so this idea is meaningless or
self-deluding.
• Generally null hypothesis testing
is nonsensical.
• Null hypothesis testing
• Posterior probabilities are
meaningful; p-values are half-assed.
• MCMC approximations are
sometimes useful but not for
computing posterior distributions.
• MCMC approximations to posterior
distributions.
Psych 548, Miyamoto, Win '16
• P-values
END
26
How Does Bayesian Stats Differ from Classical Stats?
Bayesian: Common Aspects
Classical: Common Aspects
• Statistical Models
• Statistical Models
• Credible Intervals
– sets of
parameters that have high posterior
probability
• Confidence Intervals
–
which parameter values are tenable after
viewing the data.
Bayesian: Divergent Aspects
Classical: Divergent Aspects
• Given data, compute the full posterior
probability distribution over all
parameters
• No prior distributions in general, so
this idea is meaningless or selfdeluding.
• Generally null hypothesis testing is
nonsensical.
• Null hypothesis testing
• Posterior probabilities are
meaningful; p-values are half-assed.
• MCMC approximations are
sometimes useful but not for
computing posterior distributions.
• MCMC approximations to posterior
distributions.
Psych 548, Miyamoto, Win '16
• P-values
END
27