Answer Key:
a. Let’s derive the logarithm of the posterior distribution
f  yi |  0 , 1  
e  i iyi
,    0 , 1   1, i  e 0  1i
yi !
 18

P   0 , 1 | data    f  yi |  0 , 1      0 , 1 
 i 1

 18 e  i iyi 
 
 1
 i 1 yi ! 
e

18
 i
i 1
18

i 1
yi
i
18
18
i 1
i 1
log  P   0 , 1 | data     i   yi log  i 
18
   yi log  i   i 
i 1
18
log  P   0 , 1 | data      yi   0  1i   e 0  1i 
i 1
b. Write an R function to compute the logarithm of the posterior density of
 0 , 1 
> Post.loglike <- function(data, beta){
b0 = beta[1]
b1 = beta[2]
y = data[,"y"]
Month = data[,"Month"]
lambda = b0 + b1*Month
return(sum(y*lambda - exp(lambda)))
}
c. Suppose we are interested in estimating the posterior mean and standard deviation for the
intercept and the slope. Through using optim() (method = "BFGS") with the data, find the
maximum likelihood estimate for
 0 , 1  and the corresponding estimated matrix of variance
covariance, using the starting value for
0  1 and 1  1 .
MOM is a vector with values of 1 in it.
> Bayes.mle1 <- optim(par = c(MOM), fn = Post.loglike, data = data,
method = "BFGS", control = list(fnscale = -1), hessian = TRUE)
> MLE = Bayes.mle1$par
> MLE
[1] 2.80320484 -0.08377386
The MLE of
0  2.80320484
and the MLE of
1  0.08377386 .
> cov <- - solve(Bayes.mle1$hessian)
The estimated covariance matrix of
 0 , 1 
is
> cov
[,1]
[,2]
[1,] 0.021950882 -0.0020675731
[2,] -0.002067573 0.0002822018
d. Using the output of optim(), and the multivariate t distribution as the proposal distribution, set
the seed to be 1234, and the data, construct a metropolis random walk algorithm for simulating
from the posterior density. Compute the posterior mean, median, mode, and standard deviation
for
0
and
1 , using 10000 iterations. The multivariate t distribution can be compute in R using
the following function rmvt(n, sigma=cov*(df-2)/df, df=df) where df = 16, in library(rmvt).
> run_metropolis_MCMC <- function(startvalue, cov, df, data, iterations){
nAccepted = 0
nRejected = 0
burnIn = ceiling( 0.0 * iterations )
chain = matrix(NA, nrow = iterations + 1, ncol = 2)
chain[1,] = startvalue
for (i in 1:iterations){
proposal = rmvt(1, sigma=cov*(df-2)/df, df=df)
probab = exp(Post.loglike(data = data, beta =
- Post.loglike(data = data, beta
if (runif(1) < probab){
chain[i+1,] = proposal + chain[i,]
if ( i > burnIn ) { nAccepted = nAccepted +
}else{
chain[i+1,] = chain[i,]
if ( i > burnIn ) { nRejected = nRejected +
}
proposal + chain[i,])
= chain[i,]))
1 }
1 }
}
return(list(chain,nAccepted, nRejected, burnIn ))
}
> set.seed(1234)
> df <- dim(data)[1]-2
> startvalue <- Bayes.mle1$par
> chain = run_metropolis_MCMC(startvalue, cov = cov, df = df, data= data,
iterations=100000)
> nAccepted <- chain[2]
> trajectory <- chain[[1]]
> burnIn <- chain[[3]]
> # Extract the post-burnIn portion of the trajectory.
> acceptedTraj = trajectory[ (burnIn+1) : dim(trajectory)[1], ]
> trajLength = dim(acceptedTraj)[1]
> Mean = apply(acceptedTraj, 2, mean)
> Median <- apply(acceptedTraj, 2, median)
> SD <- apply(acceptedTraj, 2, sd)
> densCurve1 = density( acceptedTraj[,1] , adjust=2 )
> densCurve2 = density( acceptedTraj[,2] , adjust=2 )
> Mode = c(densCurve1$x[which.max(densCurve1$y)],
densCurve2$x[which.max(densCurve2$y)])
> names <- c("Intercept", "Slope")
> Sum.Stat <- cbind(Mean, Median, Mode, SD)
> Summary <- data.frame(names, Sum.Stat)
> Summary
names
Mean
Median
Mode
SD
1 Intercept 2.80178299 2.80290193 2.80460000 0.14678168
2
Slope -0.08441321 -0.08423112 -0.08366501 0.01664822
The output above gives us the Mean, Median, Mode, and standard deviation of t
he intercept and slope.
e. Compare the result in part d) with what we get in part c) using optim().
> sqrt(diag(cov))
[1] 0.14815830 0.01679886
In part c), the estimated standard deviation for
0  0.14815830
and
1  0.01679886 .
Intercept
Slope
Part d)
Mean
2.80178299
-0.08441321
SD
0.14678168
0.01664822
Part c)
Mean
2.80320484
-0.08377386
SD
0.14815830
0.01679886
From this table, we see that the estimated mean, and standard deviation of the intercept and slope are
pretty much closed.
f.
Estimate the credible interval.
> HDI <- matrix(NA, ncol = 2, nrow = 2)
> colnames(HDI) <- c("HDI_Lower", "HDI_Upper")
> Parameters <- c("intercept", "Slope")
> HDI <- data.frame(Parameters, HDI)
> for(i in 1:2){
HDI[i,2:3] <- HDIofMCMC(sampleVec = acceptedTraj[,i], credMass=0.95)
}
> HDI
Parameters HDI_Lower
HDI_Upper
1 intercept 2.5207164 3.09140864
2
Slope -0.1161668 -0.05075874
The output above gives us the upper and lower bound of the HDI for
g. Construct the histogram of
0
and
0
and
1 .
1 with the density overlay on the top, with Mean, Mode,
Median and standard deviation the graph.
> layout( matrix(1:2,ncol=2) )
> par(mar=c(3,4,2,1),mgp=c(2,0.7,0))
> library(coda)
> border <- "skyblue"
> col <- "skyblue"
> for(j in 1:2){
histinfo = hist( acceptedTraj[,j] , freq=F, border=border , xlab = Parameter
s[j],
main=bquote( list( "SD" == .(round(sd(acceptedTraj[,j]),3)) ,
"Median" == .(round(median(acceptedTraj[,j]),3)),
"Mean" == .(round(mean(acceptedTraj[,j]),1)) ) ))
histinfo = hist( acceptedTraj[,j] , plot=F )
densCurve <- density( acceptedTraj[,j] , adjust=2 )
lines( densCurve$x , densCurve$y , type="l" , lwd=2, col = "red" )
cenTendHt = 0.9*max(histinfo$density)
cvHt = 0.7*max(histinfo$density)
ROPEtextHt = 0.55*max(histinfo$density)
# Display central tendency:
mn = Mean[j]
med = Median[j]
mo = Mode[j]
text( mo , cenTendHt ,
bquote(mode ==.(signif(mo,3))) , adj=c(.5,0) , cex=1.5 )
# Display the HDI.
credMass <- 0.95
cex <- 1.5
HDItextPlace=0.7
lines( HDI[j,2:3] , c(0,0) , lwd=4 )
text( mean(as.numeric(HDI[j,2:3])) , 0 , bquote(.(100*credMass) * "% HDI" ) ,
adj=c(.5,-1.7) , cex=cex )
text( HDI[j,2:3][1] , 0 , bquote(.(signif(HDI[j,2:3][1],3))) ,
adj=c(HDItextPlace,-0.5) , cex=cex )
text( HDI[j,2:3][2] , 0 , bquote(.(signif(HDI[j,2:3][2],3))) ,
adj=c(1.0-HDItextPlace,-0.5) , cex=cex )
}