The Latest Development of Objective Bayesian Analysis
Ning Ji1 ,Yi Dai2
School of Mechanical Engineering, Tianjin University of Technology and Education, Tianjin 300222, China
(1jining5225015@163.com, 2dai1845@qq.com)
Abstract - The 2011 International Workshop on
Objective Bayes Methodology has been held in East China
Normal University last year. This paper selects three
experts’ reports to introduce the latest development of
Objective Bayesian Analysis. Professor James Berge applied
the reference prior in estimating the parameters of Gaussian
random field and is shown to yield a proper posterior
distribution. Professor Ferreira used the theory of Objective
Bayesian to solve the parameters of Exponential Power
Regression Models and recommended the Jeffreys prior in
three cases. Professor Lai-sheng Wei introduced the loss
function for Empirical Bayes test problem for variance
components in random effects models and the recommended
noninformative prior distribution is given, obtained very
good result. Those three experts’ arguments will be
introduced in detail in this paper.
Keywords - Gaussian random field, Jeffreys prior,
Noninformative prior, Objective Bayesian analysis, random
effects models, reference prior
I. INTRODUCTION
Bayesian method is developed based on Bayes
theorem for solving statistical problems systematically [1].
Bayesian analysis as a subjective theory is a kind of
general view, but it is not very accurate either in history
or in reality. In fact the first Bayesian experts, Thomas
Bayes and Laplace, use a constant prior distribution for
the unknown parameters based on the Bayesian analysis.
How to make the prior probability generates the
subjective and objective Bayesian School. Objective
Bayesian School's main content is the use of
noninformative prior distribution. Most of them are using
Jeffreys prior distribution [2]. Maximum entropy prior is
another commonly used noninformative prior distribution
in practice [2]. Details will be discussed in this paper.
II. Reference prior application to Gauss random field
Professor James Berge comes from the University of
Duke introduced the development history of Objective
Bayesian Analysis and pointed out that the objective
Bayesian analysis has been applied to many fields of
Bayesian analysis. It is characterized by use of the
following prior.
Noninformative prior. The largest frequency that we
use in practice is constant prior, Jeffreys prior and the
maximum entropy prior. The reference prior is often
emphasized in the latest statistics literature. The basic
thought of the reference prior is that: firstly, extraction the
test data zk   x1, x2 , , xk  from the population
p x  
distribution
;
then
introduce
a
kind
of
measure I zk ; p   which expressed that under the prior
distribution of p   , the amount of information that
sample data zk can provided. The sample data zk defined
by I zk ; p   can provide the prior  k   which
contains the maximum amount of information and
obtained the corresponding posterior distribution  k  x  .
And then in limit significance (use Kullback-Leibler
distance) to define the limit   x   lim  k  x  , we call
k 
  x  is reference posterior distribution. Finally we
define    which satisfying   x   p  x     as
reference prior distribution [3].
Objective
Bayesian
analysis
method
can
successfully, which has been confirmed, but it will also
appear some problems in practice.
Constant prior can yield improper posteriors [4]. For
instance, we use the Objective Bayesian analysis method
to estimate the mean and the variance for the Gaussian
random field Z ( s ), s  D , D  R l ; we usually select the
constant prior or Jeffreys prior. At the same time
parameter  will be introduced through the process of
estimation to control the smoothness and roughness of the
random field. Unknown regression parameter  will be
also introduced. Professor James Berge gives the


likelihood function for parameters  , 2 , as following:
L(  , , ; Z ) 
2



n
2


1

2 exp 
(1)

( Z  X  )1  Z  X   
 2

In which the symbol Z is the observed data. And the
2 2
1
2
prior density form:


  , 2 , 
  
 2 
a
a  R
(2)
Professor James Berge demonstrated that for the
prior (2)with    
distribution of
1
v
  , 2 , 
and any a , the posterior
is improper. For the model
described above with sampling distribution (1) and prior

distribution(2), the posterior distribution of  , 2 ,

1
is

1 
1  2
1    1    1
p
p 

 
 l1   , , p   
 p
proper if and only if [4]:
0   LI ( )   d  
(3)

This follows immediately from(3), since LI ( )  c0
as   0 and
1
is not integrable at 0.

1
(4)
In which,
 

Wv     1P
 v 

is
the

K v si  s j
nn

1
X 1

matrix ，  ,ij  K v si  s j

，
III. Objective Bayesian Analysis for Exponential
Power Regression Models
Professor Ferreira form University of Missouri,
applied the Objective Bayesian Analysis method to
parameter solution for Exponential Power Regression
Models and recommended the Jeffreys prior in three
cases.
Exponential Power distribution [5]:


2
p





(5)
In which:

  y   ，      ，   0 , p  1
(8)
proper posterior. Both prior  J and  l lead to improper
posteriors because of their relatively heavy tail in term of
p when p   .
In practice, if the heavy tail behavior of the data
observed, to reduce the influence of outliers and increases
the robustness of analysis, we may directly fix
1
J
and  l priors may also lead to proper
1
posteriors.
Professor Lai-sheng Wei from University of Science
and Technology of China introduced the loss function for
Empirical Bayes test problem for variance components in
random effects models and the recommended
noninformative prior distribution is given, obtained very
good result.
The random effects model is as following [6]:
Yij    i  eij ； i  1, , a ； j  1, , b



In which i ~ N 0, 22 ， eij ~ N 0,12

There are many methods on estimating the variance
components 11 and  22 such as ANOVA [7], Maximum
likelihood [8], Restrict Maximum likelihood and Bayesian
method [9].
Professor Wei demonstrated that if under the non
informative prior:
 
  2   2
EP(0, , p)
Based on the Fisher information, three default priors
for the parameter   , , p  are derived.
Jeffreys prior is:
 J   , , p  
 1
   
  p
1  p 
 
1  2
  1   
p 
 
Professor Ferreira demonstrated that  l can yield a
Model:
yi  X i  i , i
3
p2
1
2
IV. Empirical Bayes estimation of variance
components in Random Effects Model
Prior (4) yields a proper posterior distribution.





 l   , , p  
1  p  2 .Then 
  corr Z  s  , Z u  .
 
   1 1 p y  
1
f  y  , , p  
exp   
2

 
 

1
2

1
2

2
a =1，
  v   tr Wv2  
tr Wv  



n  p





Independent Jeffreys prior associated with the
grouping    ,   ,  p  :
1
For the above mentioned problems, James Berge
proposed the use of a reference prior. The ultimate prior
form is derived:
P  I  X X -1X
(7)
(9)
And under the loss function

  

L  2 , d  w  2  d1  12

   d2   22  
2
2
(10)
It can obtain very good result.
k
1

1   2 
1 
1  2
 2    1    1    1
p   
p 
p 

k 1


p
(6)
Independent Jeffreys prior associated with the
grouping    ,  , p  :
V. CONCLUSION
According to the questionnaire investigation results
distributed to famous scholars from international
statisticians field, Jordan, chairman of international
Bayesian statistical society (ISBA), announced five
important unanswered questions in the Bayesian theory
method, the sorting of which is [10]: (1) model selection
and hypothesis testing; (2) Calculation and statistics; (3)
the relationship between Bayesian and frequency; (4)
prior; (5) Nonparametric and semiparametric; How to
make the prior probability generates the subjective and
objective Bayesian School. Choosing prior in subjective
way is the focus of the study for Objective Bayesian
School. Use the noninformative prior is the main content
for Objective Bayesian School and has been made great
achievement. The Jeffreys prior, the maximum entropy
prior and the reference prior are the largest frequency
prior that we used in practice. Now many scholars are still
committed to the study of the noninformative, believing
that through joint efforts, Objective Bayesian School
certainly could have greater achievements.
ACKNOWLEDGMENT
This work was supported by the National Natural
Science Foundation of China (50875186) and Major
projects of national science and technology
(2009ZX04014-013).
REFERENCES
[1]
Xi-zhi Wu, Modern Bayesian statistic .Beijing, China
Statistics Press, 2000, pp. 11–20.
[2] James O. Berger, Statistical decision theory and Bayesian
analysis. New York, Springer-Verlag, 1985, pp. 96–
98,100–104.
[3] Yong Li, “Studying on selecting theory of prior
distribution” Master. dissertation, Applied mathematics.
Southwest Jiaotong University. China, 2006.
[4] James O. Berger, Victor De Oliveira, and Bruno Sanso,
“Objective Bayesian analysis of spatially correlated data”,
Statistical mathematics. Duke University. USA, 2000.
[5] Marco A.R.Ferreira, “Objective Bayesian analysis for
exponential power regression models”, presented at the 9th
Annu. The 2011 International Workshop on Objective
Bayes Methodology. Shanghai. 2011.
[6] Lai-sheng Wei, “The superiorities of Bayes estimation and
empirical Bayes estimation of variance components in
random effects model”, presented at the 9th Annu. The
2011 International Workshop on Objective Bayes
Methodology. Shanghai. 2011.
[7] Hui-min Hu, Rong Yang and Xing-zhong Xu, “The
generalized p-value of the one-way layout analysis of
variance model”, Beijing Institute of Technology. China,
2007.
[8] Shi-song Mao, Xiao-long Pu and Zhong Liu, Statistical
models and methods for lifetime data. Beijing, China
Statistics Press, 1998, pp. 171–172.
[9] Shi-song Mao, Bayesian statistics. Beijing, China
Statistics Press, 2005, pp. 36-39.
[10] Jordan M I, “What are the open problems in Bayesian
statistics”. The ISBA Bulletin, 2011, pp.1-4.