Homework
Homework 1
1. M. D. Computing describes the use of Bayes’ theorem and the use of
conditional probability in medical diagnosis. Prior probabilities of
diseases are based on the physician’s assessment of such things as
geographical location, seasonal influence, occurrence of epidemics,
and so forth. Assume that a patient is believed to have one of two
diseases, denoted D1 and D2 with PD1   0.6 and PD2   0.4 and
that medical research has determined the probability associated with
each symptom that may accompany the diseases. Suppose that given
diseases D1 and D2 , the probabilities that the patient will have
symptoms S1 , S 2 or S 3 are as follows.
D1
D2
S1
S2
S3
0.15
0.80
0.1
0.15
0.15
0.03
After a certain symptom is found to be present, the medical diagnosis
may be aided by finding the revised probabilities of each particular
disease. Compute the posterior probabilities of each disease given the
following medical findings.
(a) The patient has symptom S1 .
(b) The patient has symptoms S 2 or S 3 .
(c) For the patient with symptom S1 in part (a), suppose we also find
symptom S 2 . What are the revised probabilities of D1 and D2 .
1
2. Let X 1 , X 2 ,, X n ~ N  ,  2  , with  2 known. For the prior
distribution for  , we shall assume the normal distribution N  , 2 .
(a) Find the maximum likelihood estimate.
(b) Find the posterior distribution of  and posterior mode as estimate
of .  .
(c) Compare the estimates in (a) and (b).
3. Let X 1 , X 2 ,, X n ~ P  ,
e   xi
f  xi |   
xi !
For the prior distribution for  , we shall assume the gamma distribution
   ~ G  ,  ,    

1
 


 1
e .
(a) Find the maximum likelihood estimate.
(b) Find the posterior distribution of  and posterior mean as estimate
of .  .
(c) Compare the estimates in (a) and (b).
Solution 1:
1.
(a) By Bayes’ theorem,
P( D1 | S1 ) 
PD1 PS1 | D1 
0.6  0.15

 0.2915
PD1 PS1 | D1   PD2 PS1 | D2  0.6  0.15  0.4  0.8
and
P( D2 | S1 )  1  P( D1 | S1 )  0.7805 .
(b)Suppose S 2  S 3   . Then,
2
P(S 2 )  PS 2  D1   PS 2  D2   PD1 PS 2 | D1   PD2 PS 2 | D2 
 0.6  0.1  0.4  0.15  0.12
P(S 3 )  PS 3  D1   PS 3  D2   PD1 PS 3 | D1   PD2 PS 3 | D2 
,
 0.6  0.15  0.4  0.03  0.102
and
P(S 2  S 3 )  PS 2   PS 3   0.12  0.102  0.222
Also,
P( D1  ( S 2  S 3 ))  P( D1  S 2 )  ( D1  S 3 )   PD1  S 2   PD1  S 3 
 PD1 PS 2 | D1   PD1 PS 3 | D1   0.6  0.1  0.6  0.15
 0.15
and
P( D2  ( S 2  S 3 ))  P( D2  S 2 )  ( D2  S 3 )   PD2  S 2   PD2  S 3 
 PD2 PS 2 | D2   PD2 PS 3 | D2   0.4  0.15  0.4  0.03
 0.072
Therefore,
P( D1 | S 2  S 3 ) 
PD1  S 2  S 3  0.15

 0.676
PS 2  S 3 
0.222
P( D2 | S 2  S 3 ) 
PD2  S 2  S 3  0.072

 0.324 .
PS 2  S 3 
0.222
and
(c)
We can use PD1 | S1  and PD2 | S1  as new revised probability for the
diseases. That is,
PD1 , reviesed   PD1 | S1   0.2195
and
PD2 , reviesed   PD2 | S1   0.7805 .
Thus,
P( D1 | S 2 ) 

PD1 , revised PS 2 | D1 
PD1 , revised PS 2 | D1   PD2 , revised PS 2 | D2 
0.2195  0.1
 0.1578
0.2195  0.1  0.7805  0.15
and
P( D2 | S 2 )  1  PD1 | S 2   1  0.1578  0.8422 .
3
2.




X 1 , X 2 , , X n ~ N  ,  2 ,  2 known.  ~ N  , 2 .
(a) The likelihood function is
 1
 1
2 


l  | x1 , x2 ,  , xn    
exp 
x


i
2
 
 2
i 1  2 

n
n
 1
 1 

 exp 
2
 2  
 2
n
 x
i
i 1
 
2



.
The log-likelihood function is
log l  | x1 ,, xn    n log
 log l  | x1 ,, xn 
1
 2


 ˆ  x



2  
n
 x
i 1
i
n
1
2
2
 x
i 1
i
   0
(b)
f  | x1 , x2 ,  , xn 
   l  | x1 , x2 ,  , xn 
 1
 1
2
 exp  2      exp 
2
 2

 2
2


x


 i





 exp 





2
2



2
 x 

n
2
  
2
   n


  2 2



n
2
2

 2  

n

2


2
 x  
n
 N 
2
2

  
n





n
i 1











  2 2

n 
, 2
2

   n 


4

 
2
Therefore, the posterior distribution is
2


2
 x  
n
N 
2
2

  
n

  2 2

n 
, 2
2
.
   n 


Thus, the mode of the posterior distribution is
2

x 
n
2
.
 2  n
2
(c)
x  2  

 2


n
x


 2 2

 
n



2
n 
2
 2 2
 
 2  n
n

2


 .


The larger  2 is (i.e., the prior  varies dramatically), the less
contribution of the prior mean  is for the Bayes estimate. On the other
2
hand, the larger
is (i.e., the m.l.e. X varies dramatically), the less
n
contribution of the m.l.e. x based on the data is for the Bayes estimate.
3.
(a) The likelihood function is
n
 xi
e   xi e n  i 1
l  | x1 , x2 ,  , xn   
 n
.
x
!
i 1
i
x
!
i
n
i 1
The log-likelihood function is
 n

 n

log l  | x1 ,, xn    n    xi  log    log   xi !
 i 1 
 i 1

n
 log l  | x1 ,, xn 

 n 

 ˆ  x
5
x
i 1

i
0
(b)
f  | x1 , x2 ,, xn 
   l  | x1 , x2 ,, xn 
n


e 
 -1

 xi
n

e  n
i 1




   x i 1   n  1 

i 1
e


n
1

 Gamma    xi ,
1

i 1
n










Therefore, the posterior distribution is


n
1

Gamma    xi ,
1

i 1
n







.


Thus, the mean of the posterior distribution is
n
x
i 1
i
n

1
.

(c)
n
x
i 1
i
n

1

 n 
 1 
 x  

 
.
n


1
n


1




The larger n is (i.e., more data), the more contribution of the m.l.e. x
based on the data is for the Bayes estimate. On the other hand, the larger
 is (i.e., the prior  varies dramatically), the less contribution of the
prior mean  for the Bayes estimate.
6
Homework 2
4. Let X have a Poisson   distribution,   0, , A  0,  . The loss
function is L , a     a 2 . Consider decision rules of the form
 x  cx . Assume the prior is     e  .
(a) Calculate   , a  , and find the Bayes action. (Note that this is the
optimal Bayes action for the no-data problem in which x is not
observed).
(b) Find R ,   .
(c) Show that  is inadmissible if c  1 .
(d) Find r  ,   .
(e) Find the value of c which minimizes r  ,  
(f) Is there a best rule of the form  in terms of the minimax principle?
5. A insurance company is faced with taking one of the following 3
actions: a1 : increase sales force by 10%; a 2 : maintain present sales
force; a 3 : decrease sales force by 10%.
Depending on whether or not the economy is good ( 1 ), mediocre (  2 ), or
bad (  3 ), the company would expect to lose the following amounts of
money in each case:
a1
a2
a3
1
2
3
-10
-5
-3
-5
-5
-2
1
0
-1
(a) Determine if each action is admissible or inadmissible.
(b) The company believes that  has the probability distribution
 1   0.2,   2   0.3,   3   0.5 . Order the action according to their
7
Bayesian expected loss (equivalent to Bayes risk, here), and state the
Bayes action.
(c) Order the actions according to the minimax principle and find the
minimax action.
6. A professional baseball team is concerned about attendance for the
upcoming year. They must decide whether or not to implement a 0.5
million dollar promotional campaign. If the team is a contender, they
feel that 4 million in attendance revenues will be earned (regardless of
whether or not the promotional campaign is implemented). Letting 
denote the team’s proportion of wins, they feel the team will be a
contender if   0.6 . If   0.6 , they feel their attendance revenues
will be 1 5 million dollars without the promotional campaign, and
2
10
 million dollars with the promotional campaign. It is felt that
3
 have a Uniform0,1 distribution.
(a) Describe A,  , and L , a  .
(b) What is the Bayes action?
(c) What is the minimax action?
7. The owner of a ski shop must order skis for the upcoming season.
Orders must be placed in quantities of 25 pairs of skis. The cost per
pair of skis is $50 if 25 are ordered, $45 if 50 are ordered, and $40 if
$75 are ordered. The skis will be sold at $75 per pair. Any skis left
over at the end of the year can be sold (for sure) at $25 per pair. If the
owner runs out of skis during the season, he will suffer a loss of
“goodwill” among unsatisfied customers. He rates this loss at $5 per
unsatisfied customer. For simplicity, the owner feels that demand for
the skis will be 30, 40, 50, or 60 pair of skis, with probabilities 0.2, 0.4,
0.2, and 0.2, respectively.
(a) Describe A,  , the loss matrix, and the prior distribution.
(b) Which actions are admissible?
(c) What is the Bayes action?
(d) What is the minimax action?
8
Solution 2:
1.
(a)


  , a   E  L , a   E    a 2  E   2  2a  a 2 
 
 E   2  2aE     a 2  2  2a  a 2  a  1  1
2
since    ~ Gamma(1,1), E     1, E   2   E     Var     12  1  12  2 .
2
As a  1 ,   , a  achieves its minimum. Therefore, 1 is the Bayes
action.
(b)



R ,    E L ,    E   cx   E  2  2cx  c 2 x 2
2
 


  2 E 1  2cE x  c 2 E x 2   2  2c    c 2    2
 c  1  2  c 2
2
(c)
Since
R ,  

 2c  1 2  2c  0  c 
c
 1
  2 R ,  

2



2


2


0



0
2

c


R ,   achieves its minimum at c 

 1
. As c  1,   0 ,
R ,  c   c  1  2  c 2  c 2    R , x .
2
Therefore,  c is inadmissible if c  1 .
(d)

r  ,    E  R ,    E  c  1  2  c 2
 
2

 c  1 E   2  c 2 E     2c  1  c 2
2
2
(e)
9

r  ,  
2
 4c  1  2c  6c  4  0  c 
c
3
  2 r  ,  




6

0
2

c


(f)
Since
sup R ,    sup c  1 
2

2


 c 2   ,
there is no minimax estimate!!
2.
(a)
In no data problem, R , a   L , a  . Since
R1 , a1   R1 , a2   R1 , a3 
R 2 , a1   R 2 , a2   R 2 , a3 
R 3 , a1   R 3 , a2   R 3 , a3 
a1 , a2 , a3
,
are all admissible.
(b)
  , a1   0.2   10  0.3   5  0.51  3
  , a2   0.2   5  0.3   5  0.50  2.5
.
  , a3   0.2   3  0.3   3  0.5 1  1.7
Therefore, the Bayes action is
(c)
Since
a1 .
sup R , a1   1, sup R , a2   0, sup R , a3   1

the minimax action is


a3 .
3.(a)
10
a1 : without th e promotiona l campaign 
A
,   0,1
a
:
with
the
promotiona
l
campaign
 2

 1  5 , 0    0.6
L , a1   
,

4
,
0.6



1

  10 
 2 
  0.5, 0    0.6
L , a2    
3 
  3.5,
0.6    1
(b)
1
  , a1   E L , a1    L , a1   d

0
0.6
1
0
0.6
   1  5   1d    4  1d
 1.5  1.6  3.1
1
  , a2   E L , a2    L , a2   d

0
10 

   1.5 
  1d    3.5  1d
3


0
0.6
 1.5  1.4  2.9
0.6
Since
(c)
1
  , a2     , a1  , a1
is the Bayes action.
sup R , a1   sup L , a1   1 as   0




sup R , a2   sup L , a2   1.5 as   0 .
Since
sup R , a2   sup R , a1  ,

a2

is the minimax action.
11
4.
(a)
A  a1 , a2 , a3 ,   1 , 2 , 3 , 4 
and the loss matrix is
1
2
3
4
where
a3
a1
a1
a2
a3
-600
-550
-500
-450
-500
-1000
-1500
-1450
-375
-875
-1375
-1875
a2
is to order 25 pairs of skis,
is to order 75 pairs of skis and
pairs of skis,
2
1
is to order 50 pairs of skis,
represents the demand will be 30
represents the demand will be 40 pairs,
represents the demand will be 50 pairs, and
4
3
represents the demand
will be 60 pairs. The prior distribution is
 1   0.2,  2   0.4,  3   0.2,  4   0.2 .
(b)
a1 , a2 , a3
are all admissible.
(c)
  , a1   0.2   600  0.4   550  0.2   500  0.2   450
 530
  , a2   1090,   , a3   1075
The Bayes action is
a2 .
(d)
sup R , a1   450, sup R , a2   500, sup R , a3   375

The minimax action is


a2 .
12
Homework 3
1. A large shipment of parts is received, out of which 5 are tested for
defects. The number of defective parts, X, is assumed to have a
Binomial 5,  . From past shipments, it is known that  ~ Beta1,9 . Find
the Bayes rule (or Bayes estimate) under loss
(a) L , a     a 2 (b) L , a    a (c) L , a  
   a, if   a
2a   , if   a
(d) L , a   
  a 2
 1   
.
2. In the IQ example, where X ~ N  ,100 and  ~ N 100,225 . Assume it
is important to detect particularly high or low IQs. Indeed the
weighted loss
  1002

L , a     a  exp 
900 ,

2
is deemed appropriate. Find the Bayes rule.
3. A device has been created which can supposedly classify blood as type
A, B, AB, or O. The device measures a quantity X, which has density
f x |    e  x  , x   .
If 0    1, the blood is of type AB; if 1    2 , the blood is of type A; if
2    3 , the blood is of type B; and if 3   , the blood is of type O. In
the population as a whole,
    e  ,  0
. The loss in
misclassifying the blood is given in the following table.
AB
A
B
AB
0
1
1
A
1
0
2
B
1
2
0
O
3
3
3
If x  4 is observed, what is the (posterior) Bayes action.
13
O
2
2
2
0
Solution 3:
1.
 5
1  9 11
5 x
f x |      x 1    ,    
 1   91  91   8
19
 x
 f  | x   f x |       x 1   
13 x
 Beta x  1,14  x 
Therefore, the posterior density is f  | x ~ Betax  1,14  x .
(a)
 x   E f  |x    
x 1
x 1

x  1  14  x 15 .
(b)
Given x  0 , the posterior density is f  | x ~ Beta1,14. Thus, the
Bayes estimate is the posterior median of
f  | x ~ Beta1,14 .
Suppose the posterior median is denoted by m, then
15 11
 1
141
13
14 m
0 114 1    d  0 141    d  14   14 1    0
m
m
 1  1  m  
14
1
14
1
2
1
1
 m  1     1  214
 2
(c)

  a 2
1
L , a  
 w  
, g     . Thus, the
 1   
 1   
Bayes estimate is
E f  |x  w g  
 x  
.
E f  |x  w 
Since
14
 1

 1 
E f  |x  w g    E f  |x  
   E f  |x  

 1    
 1   
1
15
13 x

 x  1    d
1   x  114  x 
0
1

15
x  113  x 
14


 x  1   12 x d

x  114  x 
14
x  113  x 
0
1

15
x  113  x 
14


x  114  x 
14
13  x
and
 1 
E f  |x  w   E f  |x  

 1   
1
15
13 x

 x  1    d
 1    x  114  x 
0
1

15
x 13  x 
13


 x1  1   12 x d

x  114  x 
13
x 13  x 
0
1

15
x 13  x  13 14


x  114  x 
13
x13  x 
then
14
w g   .  13  x  x
E
 x  
13 14
E f  |x  w 
13 .
x13  x 
f  | x 
(d)
Given
x  0, k0  1, k1  2 ,
 k0 
100

  100% 
% percentile of
3
 k 0  k1 
(b), suppose
100
% percentile of
3
the
Bayes
estimate
is
f  | x ~ Beta1,14. Similar to
f  | x ~ Beta1,14 is p. Then,
15
the
15
1
141
14 p
14
11





1


d



1


 1  1  p  
0 114
0
3
p
1
 2  14
 p 1  
 3
2.
X ~ N  ,100,  ~ N 100,225
100
100  225 
 225
 9 x  400 900 
 f  | x  ~ N 
x
,
,
  N

100

225
100

225
100

225
13
13 



L , a     a  e
2
Since
then
 9002
900
 w   e
   1002 
1
f  | x 
w    exp 
E


 900  2  900







1
2  900




900

, g    

   9 x  400 2 
1
13 d
exp 
2

900
13
13


   9 x  400  2


2 




13   100
1
1
exp   

 d
900
900
2
900



2 
13
2
13
 
 


 9002
13

1
2  900

 
 exp c



 
 exp c


 
  1 1   9 x  400  2
2 


exp  
13



2


100

 
 d
13

2
900



13

1 1

exp  
11 2  2 9 x  200  c d
 2 900

2
  1 11  2

9 x  200  9 x  200   
exp  

  d
  2 
2
900
11
11
900

  

2 

11
1
  1 11  9 x  200 2 
exp  

d


2
900
11

 

2  900
11
1
 exp c 
and similarly
16
,
   1002 
1
f  | x 
g  w    exp 
E


 900  2  900



   9 x  400 2 
1
13 d
exp 
2

900
13


13

   9 x  400  2


2 




13   100
1
1
 
exp   

 d
900
900
2
900




2 
13
2
13
 
 


1
 
2  900


13

1
 
2  900

  
 exp c
13


  1 1   9 x  400  2
2 
exp  
13 
 2  100  d


13

 2 900  
1
2  900


 1 1

exp  
11 2  2 9 x  200  c d
 2 900

11
2
  1 11  2

9 x  200  9 x  200   
exp  

  d
  2 
2
900
11
11

  


  1 11  9 x  200 2 
 exp c  
exp  
d
 
11  
 2 900 

2  900
11
 


1
  9 x 11200
 exp c  
then
 x  
E
f  | x 
E
w g   . 
w 
  9 x 11200 9 x  200

exp c 
11
exp c 
f  | x 

3.
f x |    e  x  ,     e   f x,   f x |      e  x ,0    x
f x,  e  x 1
mx    e d  xe , x  0  f  | x  
 x 


m
x
xe
x
0
x
x
x
Then,
17
0 11 3 5

4
4
1 0  2  3 6
  , A  E f  |x  L , A 

4
4
1 2  0  3 6
  , B   E f  |x  L , B  

4
4
2220 6
  , O   E f  |x  L , O  

4
4
  , AB   E f  |x  L , AB  
So, the posterior Bayes action is AB.
Homework 4
1. If X ~ Gamma ,2  and
n
2
   

1
1
     1
e

~ Inverse Gamma ,   .
Find the Bayes estimator of  under loss

  a 2
L , a  
.
2

( If a random variable Z ~ Inverse Gamma ,   , then
E Z  
1
   1
,VarZ  
1
   1   2
2
2
,  2;
1
~ Gamma ,  
Z
2. A business man must decide how to finance the investment of some
business. It costs $100000 to invest. The man has available 3 options:
a1 --finance the investment himself; a 2 --accept $70000 from investors in
return for paying them 50% of the business profits; a 3 --accept $120000
from investors in return for paying them 90% of the business profits. The
business profits will be $3 , where  is the number of goods sold in the
business. From past data, it is believed that   0 with probability 0.9,
18
while the   0 have density    

0.1
e 300000 I 0 ,    .
300000
A financial assessment is performed to determine the likelihood of goods
sold . The assessment tells which status x1 , x2 , x3 is present. It is known
that the probabilities of the xi given  are
f x1 |    0.8e

100000



100000 

, f x 2 |    0.2, f x3 |    0.81  e



(a) For mometary loss, what is the Bayes action if x1 is observed?
For mometary loss, what is the Bayes action if x 2 is observed?
Solution 4:
1. Please replace the original loss function with

  a 2
L , a  
2

 w  
1
2
, g     .
Note that for a random variable Z ~ Inverse Gamma ,   , then
1

1

1
1
1
1
E    
e z dz  
e z dz
  1
  11
Z
z







z



z
  0
0
  1

 

1
   1 
1
0
z
 11
e
1
z
dz 
  1
,
 
 
 1 
E 2  
Z 


0
1
1
1

e z dz 
2
  1
z   z
  2  2

 
   1 2

1
   2 
0
and
19
2

1
    z 
 2 1
e
1
z
dz
0
z   2 1
e
1
z
dz 
  2  2
 
1
E 

1
Z 

 1    1 2   1 .
E 2 
Z 
The posterior is
f  | x   f  x |    

 1
1
  x  n 2 1

exp 
exp 
x
  1
n 


2






n





 2  2
 2
1
1


n

   1
2

  1  x 1 
exp     
   2  
1

n

   1
2





1


exp
  2  
 
 
  2  x  
n
2 

 Inverse Gamma    ,
2  x 
2
The Bayes estimate under the loss function is

f  | x   1
f  | x   1 
E


E
 2 
 
E f  |x  w g  
 x  
.

1
E f  |x  w 


1
E f  |x   2 
E f  |x   2 
 
 
1
2  x


n
 2    n   2 
 

    1
2
2
2


x






2  x
n  2  2
Note:
Homework 5, 1, (a) can be solved by finding the posterior, which is
also distributed as inverse gamma!!
20
2.
(a)
The joint distribution function is
f  x1 ,   f x1 |    

 0 
0
.
9

0
.
8

exp

  0.72 (probabili ty,   0)

100000 


0.8  exp      0.1  1  exp      0.02  1  exp    ,  0

300000
75000
 100000 
 300000 
 75000 
Since

m x1   0.72   0.02 
0
1
  
 exp 
d
75000
 75000 
 0.72  0.02  0.74
the posterior is
f  | x1  
f  x1 , 
m x1 
36

(probabili ty,   0)


37

1
1
  
 
 exp 
,  0

37
75000
75000



Since the loss function for a1 is
 100000,   0
,
L , a1   
100000 - 3 ,   0
the Bayesian (posterior) expected loss for a1 is
E f  |x  L , a1 

36
1
1
  
 100000   100000  3   
 exp 
 d
37
37
75000
75000

 .
0
3600000  125000 3475000


37
37
21
Similarly, the loss function for a 2 is
 30000,   0
L , a 2   
30000 - 1.5 ,   0
the Bayesian (posterior) expected loss for a 2 is
E f  |x  L , a2 

36
1
1
  
  30000   30000  1.5   
 exp 
 d
37
37 75000
 75000 
0

1080000  82500 997500

37
37
Since the loss function for a 3 is
  20000,   0
L , a3   
 20000 - 0.3 ,   0
the Bayesian (posterior) expected loss for a 3 is
E f  |x  L , a3 

36
1
1
  
  20000    20000  0.3   
 exp 
 d
37
37 75000
 75000 
0

 720000  42500  762500

37
37
Therefore, the Bayes action is
a3 .
(b) Similar to (a),
The joint distribution function is
f  x 2 ,   f x 2 |    
0.9  0.2  0.18 (probabili ty,   0)


1
1

  
  
0.2  0.1 
 exp 
 0.02 
 exp 

,  0

300000
300000
 300000 
 300000 
22
Since

m x2   0.18   0.02 
0
1
  
 exp 
d
300000
 300000 
 0.18  0.02  0.2
the posterior is
f  | x2  
f  x2 ,  
m x2 
9

(probabili ty,   0)


10

1
1
  
 
 exp 
,  0

10
300000
300000



Since the loss function for a1 is
 100000,   0
,
L , a1   
100000 - 3 ,   0
the Bayesian (posterior) expected loss for a1 is
E f  |x  L , a1 

9
1
1
  
  100000   100000  3   
 exp 
 d
10
10 300000
 300000 
0

900000  800000 100000

10
10
Similarly, the loss function for a 2 is
 30000,   0
L , a 2   
30000 - 1.5 ,   0
the Bayesian (posterior) expected loss for a 2 is
23
E f  |x  L , a2 

9
1
1
  
  30000   30000  1.5   
 exp 
 d
10
10 300000
 300000 
0

270000  420000  150000

10
10
Since the loss function for a 3 is
  20000,   0
L , a3   
 20000 - 0.3 ,   0
the Bayesian (posterior) expected loss for a 3 is
E f  |x  L , a3 

9
1
1
  
  20000    20000  0.3   
 exp 
 d
10
10 300000
 300000 
0

 180000  110000  290000

10
10
Therefore, the Bayes action is
a3 .
Homework 5
3. A production lot of 5 electronic components is to be tested to
determine  , the mean lifetime. A sample of 5 components is drawn, and
the lifetimes X 1 ,, X 5 are observed. It is known that X i ~ Exponential ( ) .
From past records it is known that, among production lots,  is
distributed according to an InverseGamma(10,0.01) distribution. The 5
observations are 15, 12, 14, 10, 12.
(a) Find the generalized maximum likelihood estimate of  , the posterior
24
mean of  , and their respective posterior variance.
(b) Find the approximate 90% HPD credible set, using the normal
approximation to the posterior.
4. The weekly number of fires, X, in a town has a Poisson ( )
distribution. It is desired to find a 90% HPD credible set for  . Nothing
is known a priori about  , so the noninformative prior
 ( ) 
1

,0     is deemed appropriate. The number of fires observed
for 5 weekly periods was 0, 1, 1, 0, 0.
(a) What is the desired credible set?
(b) Find the approximate 90% HPD credible set, using the normal
approximation to the posterior.
5. Suppose X ~ N ( ,1) , and that a 90% HPD credible set for  is
desired. The prior information is that  has a symmetric unimodal
density with median 0 and quartiles  1 . The observation is x  6 .
(a) If the prior information is modeled as a N (0,2.19) prior, find the 90%
HPD credible set.
(b) If the prior information is modeled as a Cauchy(0,1) prior, find the
90% HPD credible set.
6. A large shipment of parts is received, out of which 5 are tested for
defects. The number of defective parts, X, is assumed to have a
Binomial (5, ) distribution. From past shipments, it is known that  has
a Beta (1,9) prior distribution. Find the 95% HPD credible set for  , if
x  0 is observed.
5. From path perturbations of a nearby sun, the mass  of a neutron star
is to be determined. 5 observations 1.2, 1.6, 1.3, 1.4 and 1.4 are obtained.
Each observation is (independently) normally distributed with mean 
and unknown variance  2 . A priori nothing is known about  and  2 ,
25
so the noninformative prior density  ( ,  2 ) 
1
2
is used. Find a 90%
HPD credible set for  .
Solution 5:
1. X i ~ Exponential   and    ~ Inverse Gamma10,0.01 Then,
f  | x1 ,  , x5      f  x1 ,  , x5 |  

1
  xi 
 1  5 1
exp


 exp 
10
101
10 0.01 
 0.01  i 1 
  
1



151
1

151
5
1 1

exp 
  xi  

i 1

   0.01





1
exp 
 
 
1
 
5
  100 
xi



i 1
 














1
 Inverse Gamma 15,
5

100   xi

i 1









Since  xi  63 , thus f  | x1 , , x5  ~ Inverse Gamma15,
5
i 1
1 
.
163 
(a) Since
  163 
  163 
exp 
exp 


15
163
 
 


L   f  | x1 ,  , x5  


15
14!
 16
 1 
151
15


 163 
 163
 ln L  
 16 ln    ln 14!  15 ln 163

d ln L  163 16
 2 
0
d


163
 ˆ 
 10.1875
16

26
The posterior mean is

1
   1
1

1
 15  1
163

163
 11.6429
,
14
and the posterior variance is
V
1
1

 10.4274
2
2
.
 2   1   2  1 
2





15

1

15

2


 163 


2
2
V  V    ˆ  10.4274  11.6429  10.1875  12.546
(b) The approximate 90% HPD credible set for  is
  z


V  11.6429  1.645  10.4274  6.331,16.955
2
2. X i ~ Poisson   and     . Then,
1

f  | x1 ,  , x5      f  x1 ,  , x5 |  
exp    xi
 
 i 1
xi !
5
1
5

1

 xi
  i 1  exp  5 
5
 xi 1
  i 1

 
 exp 
 1

 5






1
 5
 Gamma   xi , 
5
 i 1
5
(b) Since
 x  0  1  1  0  0  2 , the posterior mean is
i 1
i
1
5
    2   0.4 ,
and the posterior variance is
27
2
2
1
V    2    
.
25
5
2
Therefore, the approximate 90% HPD credible set for  is
  z
2

V  0.4  1.645 

2 
   0.065,0.8653
25 
3.
(a)
X ~ N  ,1 and    ~ N 0,2.19 . Then,
f  | x      f  x |  
   x   2 
  2 
exp

exp 


2
2  2.19 





2  2.19
2
2
 
2.19 x  
 
   
3.19  


 exp

 2.19  
 2   3.19  

 

 2.19 x 2.19 
 N
,

 3.19 3.19 
Since x  6 , the posterior mean is  
posterior variance is V 
  z
2
2.19 x 2.19  6

and the
3.19
3.19
2.19
. Thus, 90% HPD credible set for  is
3.19
 2.19  6  
V 
  1.645 
 3.19  
2.19 
  2.7561,5.4821
3.19 
4. X ~ Binomial 5,  and    ~ Beta1,9 . Then,
28
f  | x      f  x |  

1  9  11
91  5 
5 x
  1      x 1   
19 
 x
  x 1   
14 x 1
   x 11 1   
14 x 1
 Beta  x  1,14  x 
Since x  0 , the posterior mean is  

x 1
1


   x  1  14  x 15 and
the posterior variance is
V
x  114  x

14


.
2
2
    1    x  1  14  x  1x  1  14  x 16 152
Thus, the approximate 95% HPD credible set for  is
  z
2
14
 1 
V     1.96 
16  152
 15  
5. X i ~ N  ,  2  and   ,  2  ~


1
2

   0.0556,0.1889


. Then,

 
f  , 2 | x    , 2 f x |  , 2


   xi   2  
exp



2 2
1 5 


 2 

 i 1 
2 





5

 xi   2 



1

 7 exp  i 1

2 2




Further,
29
f  | x    f  ,  2 | x d 2
 5
2 
   xi    

1
d 2
  7 exp  i 1 2

2














1
1

d 2

exp
5
2 2 1

 
 

 
2
2


5

2 



x





i
  i 1






2
 5 

   5

2 
 2
   xi    
 i 1

5
2
   x i    
 i 1

5
2

1




2
 5 

  5

2 
 2
   xi    
 i 1

5
2
 
2
5 1
2










1

 d 2
exp

 

 
2
2


5

2 



x





  i 1 i


5
2
-5
5
5
5
2 

2
2
2
2
 5  x     xi  x     xi      xi  x  x    
i 1
i 1

  i 1

Note that
t
5   x 
5

 x  x 
i 1
2
i
  x  ~ t
V
5
5 1
5
where V 
 x
i 1
 x
2
i
4
is the sample variance.
30
51
 t4
,
Thus, a 1001   % HPD credible set is
x  t 4,
Since x 

V
V
V
  x  t 4,
, x  t 4,

2
2
5 
5
5
2
6.9
and V  0.022 , a 90% HPD credible set for  is
5
x  t 4,
2
V  6.9  

   2.132 
5  5  
0.022 
  1.2386,1.5214
5 
Note:
For X i ~ N  ,  2 , i  1, , n, and   ,  2  ~
1
2
. Then,
-n
n
2

2
2
f  | x   n  x     xi  x   .
i 1


Further,
n   x 
n

 x  x 
i 1
2
i
  x  ~ t
V
n
n 1
,
n 1
n
where V 
 x
i 1
 x
2
i
n 1
.
Homework 6
7. For 2. (a) and 3. (b) in homework 5, please write programs to
calculate 90% HPD credible sets (not approximate).
8. A large shipment of parts is received, out of which 5 are tested for
defects. The number of defective parts, X, is assumed to have a
31
Binomial (5, ) distribution. From past shipments, it is known that  has
a Beta (1,9) prior distribution. Suppose x  0 is observed. It is desired to
test H 0 :   0.1 versus H1 :   0.1. Find the posterior probabilities of the
two hypotheses, the posterior odds ratio, and the Bayes factor.
9. The waiting time for a bus at a given corner at a certain time of day is
known to have a Uniform0,  distribution. It is desired to test
H 0 : 0    15 versus H1 :   15 . From other similar routes, it is known
that  has a Pareto5,3 distribution. If waiting times of 10, 3, 2, 5, and
14 are observed at the given corner, calculate the posterior probability, the
posterior odds ratio, and the Bayes factor.
(If X ~ Paretox0 ,  , then f x | x0 ,  
E X  
  x0 
 1
 
x0  x 
x0
if   1
 1
I  x0 ,  x  .
)
Solution 6:
2.
x 0
X ~ Binomial 5, , ~ Beta1,9  f  | x  ~ Beta1  x,9  5  x  
 Beta1,14
Then,
 0  P  0.1 | x  0 
15
 114 1   
0.1
11
141
0
and
1  1   0  0.914 .
The posterior odds ratio is
 0 1  0.914
14

 0.9  1 .
14
1
0.9
32
d  1  0.9
14
Since
 0  P  0.1 
10
 119 1   
0.1
91
11
d  1  0.9
9
0
and
 1  1   0  0.99 ,
the Bayes factor is
0
B
0
1

1
0.9 14  1
0.9 9  1 .
3.
4
X ~ Uniform0, , ~ Pareto5,3  f  | x1 , .x5  
3  5  5
  
5  

4
3  5  5
5 5     d
Then,
9
8 5
 0  P0    15 | x1 ,, x5      d  1  38
5  
5
15
and
1  1   0  38 .
The posterior odds ratio is
 0 1  38
8

 3  1 .
8
1
3
Since
4
3 5
 0  P0    15     d  1  33
5  
5
15
and
 1  1   0  33 ,
the Bayes factor is
33
8 5
  
5  
9
0
B
0
1

1
38  1
33  1 .
Homework 7
10. Theory predicts that  , the melting point of a particular substance
under a pressure of 10 6 atmospheres, is 4.01. The procedure for
measuring this melting point is fairly inaccurate, due to the high pressure.
Indeed it is known that an observation X has a N ( ,1) distribution. 5
independent experiments give observations of 4.9, 5.6, 5.1, 4.6, and 3.6.
The prior probability that   4.01 is 0.5. The remaining values of  are
given the density g   .
(a) Assume g   ~ N 4.01,1 . Formulate and conduct a Bayesian test of the
proposed theory.
(b) Calculate the p-value against H 0 :   4.01 .
(c) Calculate the lower bound on the posterior probability of H 0 :   4.01
for any g   , and find the corresponding lower bound on the Bayes
factor.
(d) Assume g   ~ N 4.01, 2  , and graph the posterior probability of
H 0 :   4.01 as a function of  2 .
11.Let yi   0   1 xi   i ,  i ~ N 0,  2 , i  1,, n . Suppose the prior for
 0 ,  1 ,  2 is   0 ,  1 ,   
1

.
(a) Find the MLE for  0 ,  1 .
(b)Find the posterior density f  0 , 1 | y1 ,, y n  for  0 ,  1 .
34
(c) Find the posterior density f  | y1 ,, yn  for  and calculate the
posterior mean.
Solution 7:
1. The hints are given in the following:
(a) Please calculate  0 ,1 for the hypothesis
H 0 :   4.01 v.s. H1 :   4.01 .
(b) Please use classical t-test to obtain the p-value!!
(c) Please use the theorem (p.32, course notes) to obtain the lower
bounds.
(d) Please use the formula on p. 28 (course notes) to solve this problem.
2.
(a) The MLE is the least square estimate
n
ˆ0  y  ˆ1 x , ˆ1 
 x
i
i 1
 x  yi  y 
n
 x
i
i 1
.
 x
2
(b)

y i ~ N  0   1 xi ,  2

and   0 ,  1 ,   
1
Then,

f  0 , 1 ,  | y1 ,  , y n     0 , 1 ,   f  y1 ,  , y n |  0 , 1 ,  
 1 n
 yi   0  1 xi 2 
exp
n
2 

 
 2 i 1

2
1
 1 n
 n1 exp 
yi  ˆ0  ˆ1 xi   0  ˆ0  1  ˆ1 xi 
2 

 2 i 1


1

1


1
 1
 n1 exp 
2

 2


1
 1
 n1 exp 
2

 2

 


 

2
 n
ˆ
ˆ
ˆ
 yi   0  1 xi  n  0   0
 i 1
n

ˆ   ˆ

2



0
0
1
1  xi

i 1





S  0 , 1 

35
 
  
2
1
 ˆ1
 x
n
i 1
2
i







where
n

S  0 , 1    yi  ˆ 0  ˆ1 xi
i 1



 n  0  ˆ 0
2

 x
  
2
ˆ
1  1
 x
n
i 1
2
i
,
n
 2  0  ˆ 0  1  ˆ1
i 1
i
n
 x
ˆ0  y  ˆ1 x , ˆ1 
i
i 1
 x  yi  y 
n
 x
i 1
 x
2
i
f  0 ,  1 | y1 ,, y n    f  0 ,  1 , | y1 ,, y n d


 2
1
exp

d
n 1
2
1
2
  2 S  0 ,  1  
1

 
2

 
 2 S 1  0 ,  1 

n

 2 S 1  0 ,  1 
n

1
2
2
  2S 
n
   2
 2
n 1
2
 S  0 ,  1 
n
1

0 , 1 
n
2

 2
1
exp  2
d
1
  2 S  0 ,  1  


2
is a bivariate Student’s t-distribution.
(c)
f  | y1 ,, y n    f  0 ,  1 ,  | y1 ,, y n d 0 d 1


1
 1
  n 1 exp  2

 2





 n
ˆ  ˆ x 2  n   ˆ
y



i
0
1 i
0
0

i 1

n

ˆ   ˆ

2



0
0
1
1  xi

i 1




2  2  2 
1
 ˆ1
 x
n
i 1
2
i
1

 
2



d 0 d 1




n

2 
ˆ 2    ˆ
n



0
1
1  xi  
 0
i 1


d 0 d 1
n


ˆ
ˆ
  2  0   0  1   1  xi  
i 1





1
n
2

2



exp  n  2 s

2
2  1
 1


exp  2
n 1
2



 2


2


exp  n  2 s
2
2 


n 1

  

 
t
 1
exp  2   ˆ  1   ˆ
 2
2


exp  n  2 s
2
2




n 1

36
 d d

0
1
where
 y
n


t
   0 ,  1 t , ˆ  ˆ 0 , ˆ1 , s 2 

 n
-1
  n
 x
i

i 1
i 1
i
 ˆ 0  ˆ1 xi

2
,
n2


i 1

n
2
xi


i 1
n
x
i
Since
2


exp  n  2s
2
2 
n2
2

f  2 | y1 ,, y n 

Inverse
Gamma(
,
)
n2
2

1
2


n

2
s
2 2


 
the posterior mean is
 
  f 
1
E  | y1 ,, y n   E   2 2 | y1 ,, y n     2


 n  3
1


2   n  2 2



 s
 n  2 2 


 2 
1
2
2

| y1 ,, y n d 2
Homework 8
12. Determine the Jeffreys noninformative prior for the unknown
parameter in each of the following distributions:
(a) Poisson  ; (b) Binomial n,  ; (c) Negative Binomial m,  ;
(d) Gamma ,   (  known).
13.Determine the Jeffreys noninformative prior for the unknown vector
of parameters in each of the following distributions:
(a) Multinomail n, p1 ,, pk  ; (b) Gamma ,   (  ,  unknown).
(c) N  ,  2 ;
37
Solution 8:
1..
(a) Poisson       
1

(b) Binomial n,      
1
 1   
(c) Negative Binomial m,      
1
 1


 ''    '  
 2
(d) Gamma ,  ,  known     
 
   ,
2
where  '   and  ''   are the first and the second derivatives of   ,
respectively.
2.
k
1
(a) Multinomia l n, p1 , , pk     p1 , , pk    pi
2
i 1
(b)

 1
Gamma ,      ,     2







2
2
(c) N  ,     ,  


          
  1 
 
 2

     
   

''
'
2
1
2
1
2
Homework 9
Observations yi , i  1,, n , are independent given two parameters 
and  2 , and normally distributed with respective means xi and
38
variance  2 , where the xi are specified explanatory variables.
(a) Suppose  and  2 are unknown and the prior is   ,  2  
1
2
.
 Find the marginal posterior density of  2 .
 Find the posterior mean based on the above marginal posterior.
 Find the generalized maximum likelihood estimate of  2 based on
the marginal posterior.
(b) Suppose  2  1 is known and the prior is     1.
 Find the marginal posterior density of  .
 Find the generalized maximum likelihood estimate of  based on
the above estimate.
 For the following data,
xi
1
-1
yi
3
2
0
0
-1
3
1
2
and the hypothesis H 0 :   0.98 v.s. H1 :   0.98 , find the posterior
probability for H 0 and the posterior odds ratio.
Solution 9:
1..
(a) yi ~ N xi ,  2  and   ,  2  


1
2
Then,

 
f  ,  2 | y1 ,, y n    ,  2 f y1 ,, y n |  ,  2
 1 n
 yi  xi 2 
exp

2
n
2 
 
 2 i 1

1
 1 n
 n  2 exp 
y i  ˆxi    ˆ xi
2 

2

i 1


1

1



1

n2
1

n2

 

 1  n
exp 
y i  ˆxi
2 
2

 i 1


2

    ˆ   x
2
 1 
n  1s 2    ˆ
exp 
2 
 2 
39
  
2
n
i 1
 x
2
n
i 1
2
i



2
i



where
n
ˆ 
x y
i
i 1
n
i
x
i 1
 y
n
, s2 
i 1
i

 ˆxi
2
.
n 1
2
i
Thus,




f  2 | y1 ,, y n   f  ,  2 | y1 ,, y n d

1

n 2

 1 
exp  2 n  1s 2    ˆ
 2 
  n  1s 2 
exp 

2 2 


n 1

2 n
i 1
2
i
 2
d

 n 2
   xi
i 1
exp  i 1 2   ˆ
2 
 2

n
x

 x
2
i


2




  n  1s 2 
exp 

2 2 


n 1

 n 1

2

 Inverse Gamma
,
2 
 2 n  1s 
The posterior mean is


E  2 | y1 , , y n 
n  1s 2
1

.
2
n3
 n 1 

 1
n  1s 2  2 
The generalized M.L.E. can be obtained by solving
  n  1s 2  n  1 
2 





1
log




2 2
 log f  2 | y1 , y 2 ,  , y n 
 2




2
2


2
n  1s   n  1  1 1  0


 2
2 4
 2


 ˆ
2


n  1s 2

n 1
(b)
yi ~ N xi ,1 and     1 Then,
40
f  | y1 ,, y n      f  y1 ,, y n |  
1 n
2
 exp    y i  xi  

 2 i 1
2
1 n
 exp   y i  ˆxi    ˆ xi 

 2 i 1
 

 
1  n
 exp   y i  ˆxi
 2  i 1

    ˆ   x  
 n 2
   xi
  ˆ
 exp  i 1
2



2




2
2 n
i 1
2
i






1

 N  ˆ , n


2
  xi 
 i 1 
Since
5
5
i 1
i 1
 xi yi  0,  xi2  4 , the posterior distribution is
 1
f  | y1 ,  , y 5  ~ N  0,  .
 4
The posterior probability for H 0 is


 

P   0.98 | y1 ,, y5   P
 1.96 | y1 ,, y5   P Z  1.96  0.95 ,
1

 2

where Z is the standard normal random variable. The posterior odds ratio
0.95
 19 .
then is
1  0.95
41