Stat 544 Exam 1
March 8, 2012
I have neither given nor received unauthorized assistance on this examination.
____________________________________
Signature
____________________________________
Name Printed
There are 12 parts on this exam. I will score every part out of 10 points and take your best
10 of 12 scores. (Budget your time accordingly.)
1
1. Consider the data model for an observable X taking values in  0,1 with probability density
for a real parameter   0 .
  1 x
f  x |   
0

for 0  x  1
otherwise
a) Derive the Jeffreys prior for this model. (Remember that one form of the Fisher information in
d2
a 1-parameter model is  E
ln f  X |   .) Is this prior proper?
d 2
b) If the posterior distribution for  is proper, find a 95% HPD credible region for  . If it is not
proper, show that it is not proper. (Remember that x  exp( ln x) .)
2
2. This problem concerns a simple discrete observable, Y , and several issues of Bayes analysis for
its mean. We'll entertain two basic models with conditional distributions of Y given a parameter
(  or  ) as below.
y
f  y |   1
3
2
1
.1
.1
.8
Model 1
f  y |   2
f  y |   3
y
.3
.4
.3
3
2
1
.2
.2
.6
Model 2
f  y |   1
f  y |   2
.1
.1
.8
.8
.1
.1
Consider first an analysis based on Model 1 alone.
a) Define the parametric function 1    E Y |   . Find the 3 values 1 1 , 1  2  , and 1  3 .
1 1  ____________
1  2   ____________
1  3  ____________
Suppose that I use a prior distribution for  that is uniform on 1, 2,3 .
b) Suppose that I observe Y  3 . What is the value of the posterior mean of 1   (given this
observation)?
3
c) Find the value of "DIC" for this model (Model 1 with a uniform prior on  ) if Y  3 is

observed. (Use D  y  in place of its MCMC-based estimate D
 y  .)
g
g
Now consider both original data models, Model 1 and Model 2. For a prior on  in Model 1, we'll
continue to use a uniform prior. And now we'll also use a uniform prior for  on 1, 2 .
d) Find the value of a Bayes factor for comparing these two Bayes models (data model plus prior)
if Y  3 is observed.
4
Parallel to what we did in Model 1, define a parametric function in Model 2 by  2     E Y |   .
e) (Bayesian Model Averaging) Consider now the possibility of using both models (perhaps
because we're unsure about which is most appropriate). Let M be a model index (1 or 2).
We'll suppose that what is of interest is the mean of Y regardless of which model is acting.
That is, consider
  I  M  1 1    I  M  2 2   
( I statement  is 1 or 0 depending upon whether "statement" is respectively true or false.)
If I use a uniform prior on 1, 2 for M , what is the posterior mean of  given that I observe
Y  3?
5
3. (Truncation) Consider again Model 1 from the previous problem, but now add the issue of
truncation. That is, suppose that there is a parameter   1, 2,3 and a two-parameter model for Y
with probability mass functions
 f  y | 
if y  
 

f  y |  ,     f  t |  
 t 1

0
otherwise
( f  y |  ,  3  f  y |   and truncation has no effect for   3 . If   1 the distribution of Y is
concentrated on Y  1 . And if   2 there is no possibility that Y  3 and for each  one divides
the tabled probabilities for Y  1 and Y  2 on page 3 by their sum to make distributions over
1, 2 .)
Suppose that a priori  and  are independent and both are uniform on 1, 2,3 . If Y  2 is
observed, what is the marginal posterior distribution of  ?
6
4. In a BUGS simulation where some joint distribution for X   X 1 , X 2 , , X k  has been specified,
the software will not allow a user to define a deterministic function g  X  and then specify as
"data" a value for that function, unless g  X  is a coordinate function (i.e. X 1 , X 2 , or X k ). For
example, one may not make the assignment
Y <- X1 + X2
in the model specification, and then subsequently use data like
list(Y=3)
Presumably the reason for this is that while it's clear how to use f  3, x2 , x3 , , xk  as a conditional
density for  X 2 , X 3 , , X k  given X 1  3 , it's not even completely obvious in general how to think
about a density on the part of k - space where g  x   3 . A way to possibly use data like g  x   3
in BUGS might be to suppose that g  x  plus a small random error is known to be 3 .
a) Use this idea and write BUGS code to find (approximately) the conditional distributions of
X 1 , X 2 , and  given X 1  X 2  1 in a model where
1. given  , X 1 and X 2 are independent, X 1  Beta  ,1 and X 2  Beta 1,   , and
2.   Exp 1 .
(Give both the model code and the data list.)
7
b) Vardeman ran some code like you are supposed to write for part a) and got results below. What
do the plots indicate about the MCMC in OpenBUGS? (Do you like the plots? If not, what do
they indicate will be needed in order to produce reliable results based on the simulations?) Why
(in retrospect) is what you see on the plots not surprising?
8
5. We consider here a Bayes analysis of some (incomplete) binomial time series data. That is,
suppose that conditioned on values p1 , p2 , , p20 the variables X 1 , X 2 , , X 20 are independent
Binomial  50, pi  variables, and we adopt a random walk model for the logits of the pi time series.
That is for 1 ,  2 , ,  20 iid N  0,  2  let
i
si    j
j 1
and for constant  suppose that
exp    si 
 p 
ln  i     si , that is, pi 
1  exp    si 
 1  pi 
(For what it is worth, one might call this two-parameter model a "hidden Markov model.") To
complicate things further, we'll suppose that only some of the X i are observed, namely
X 1  3, X 3  15, X 4  11, X 8  18, X 10  20, X 13  26, X 15  21, X 20  13 , and set as our goal inference
for the pi . Some BUGS code and resulting summaries follow this page.
a) How does pˆ 20   X 20 / 50   .26 compare to the posterior median of p20 ? Explain why this is
reasonable in light of the observed values X i .
b) Which pi seems to be known least precisely? Explain how you make this judgment. Then
explain why this is reasonable in light of the observed values X i .
9
model {
logsigma~dflat()
sigma-exp(logsigma)
tau-exp(-2*logsigma)
for (i in 1:20) {
epsilon[i] ~ dnorm(0,tau)
}
sum[1] <- epsilon[1]
for (i in 2:20) {
sum[i]<- sum[i-1] + epsilon[i]
}
mu ~ dnorm (0, .0001)
for (i in 1:20) {
logt[i] <- mu + sum[i]
}
for (i in 1:20) {
p[i] <- exp(logt[i])/(1 + exp(logt[i]))
}
for (i in 1:20) {
X[i] ~ dbin(p[i],50)
}
}
list(X=c(3,NA,15,11,NA,NA,NA,18,NA,20,NA,NA,26,NA,21,NA,NA,NA,NA,13))
list(epsilon=c(0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0),mu=0,logsigma=1)
10
X[2]
X[5]
X[6]
X[7]
X[9]
X[11]
X[12]
X[14]
X[16]
X[17]
X[18]
X[19]
mu
p[1]
p[2]
p[3]
p[4]
p[5]
p[6]
p[7]
p[8]
p[9]
p[10]
p[11]
p[12]
p[13]
p[14]
p[15]
p[16]
p[17]
p[18]
p[19]
p[20]
sigma
mean
8.914
13.56
15.01
16.34
18.87
21.3
22.63
22.44
19.64
18.31
16.99
15.63
-2.018
0.1255
0.1779
0.2433
0.2396
0.2709
0.2997
0.3265
0.353
0.3775
0.3985
0.4262
0.4525
0.4793
0.4488
0.4176
0.392
0.3663
0.3397
0.3124
0.2827
0.4046
sd
3.941
5.173
5.749
5.61
5.38
5.926
5.991
5.631
6.054
6.545
6.414
5.78
0.6437
0.0465
0.0580
0.0479
0.0465
0.0838
0.0962
0.0915
0.0562
0.0838
0.0578
0.0964
0.0981
0.0633
0.0886
0.0592
0.1009
0.1129
0.1103
0.0961
0.0599
0.1938
MC_error
0.02338
0.02735
0.02886
0.02834
0.0238
0.02606
0.02679
0.02553
0.02465
0.02884
0.02609
0.02417
0.008275
4.236E-4
4.113E-4
2.304E-4
1.987E-4
4.684E-4
5.251E-4
5.063E-4
2.357E-4
3.77E-4
2.382E-4
4.272E-4
4.733E-4
3.336E-4
3.904E-4
2.556E-4
4.315E-4
4.736E-4
4.459E-4
3.942E-4
2.684E-4
0.002107
val2.5pc
2.0
5.0
5.0
6.0
9.0
10.0
11.0
12.0
8.0
6.0
6.0
5.0
-3.484
0.04676
0.07621
0.1595
0.1538
0.1257
0.1338
0.1644
0.2484
0.2235
0.2905
0.2463
0.2719
0.3597
0.2854
0.3059
0.2018
0.1587
0.1387
0.138
0.1717
0.1497
median
9.0
13.0
15.0
16.0
19.0
21.0
22.0
22.0
19.0
18.0
17.0
15.0
-1.943
0.1219
0.1744
0.2397
0.2378
0.2638
0.2911
0.3194
0.3509
0.3726
0.3969
0.4205
0.4459
0.478
0.444
0.416
0.3871
0.3595
0.3334
0.3075
0.2808
0.3656
val97.5pc
17.0
25.0
28.0
29.0
30.0
34.0
35.0
34.0
32.0
32.0
31.0
28.0
-0.9702
0.2255
0.3014
0.347
0.3367
0.4609
0.522
0.5321
0.4687
0.5593
0.5153
0.6365
0.6657
0.6061
0.6397
0.5374
0.6109
0.6171
0.5847
0.5208
0.4052
0.8875
start
1000
1000
1000
1000
1000
1000
1000
1000
1000
1000
1000
1000
1000
1000
1000
1000
1000
1000
1000
1000
1000
1000
1000
1000
1000
1000
1000
1000
1000
1000
1000
1000
1000
1000
sample
59170
59170
59170
59170
59170
59170
59170
59170
59170
59170
59170
59170
59170
59170
59170
59170
59170
59170
59170
59170
59170
59170
59170
59170
59170
59170
59170
59170
59170
59170
59170
59170
59170
59170
11