Homework 2
Prob 2.2.10
N is a Poisson random variable whose rates are given as k1 and k2 under hypotheses
H1 and H0, respectively. The a priori probabilities are equally likely, i.e.,
(1)
Pr H1   Pr H 0   0.5
1. Prove that the number of events in the interval T is a “sufficient statistic”
From the problem, we have
H1 : Pr( n  N H1 ) 
H1 : Pr( n  N H 0 ) 
k1T N
N!
k 0T N
N!
e  k1T
(2)
e  k 0T
Hence, likelihood ratio is given by
N  
Pr( n  N H1 )
Pr( n  N H 0 )
k1T N

e  k1T
N!
k 0T N  k 0T
e
N!
(3)
N
k 
  1  e  k1  k 0 T
 k0 
(4)
As a result, we obtain the LRT as
H1
N
 k1   k1  k 0 T 
  e


 k0 
(5)
H0
Taking Ln(.) both sides, we obtain
H1
k  
N ln  1  ln    k1  k 0 T ,
 k0  
(6)
H0
or
H1
N
 ln    k1  k 0 T
,
 ln( k1 )  ln k 0 
(7)
H0
if k1  k 0 , and
H0
N
 ln    k1  k 0 T
 ln( k1 )  ln k 0 
H`
(8)
if k1  k 0 .
From (7) and (8), we have proven.
2. Assuming equal costs for the possible errors, derive the appropriate LRT and
threshold
Possible errors are deciding H1 given that H0 is true and deciding H0 given that H1 is
true. Hence, we have
(9)
C01  C10
The threshold becomes
P C  C00  C10  C00 
(10)
  0 10

P1 C01  C11  C10  C11 
(7) and (8) change to
H1
N
 ln C10  C00   ln( C01  C11 )  k1  k 0 T

ln( k1 )  ln k 0 
(11)
H0
if k1  k 0 , and
H0
N
 ln C10  C00   ln( C01  C11 )  k1  k 0 T

ln( k1 )  ln k 0 
(12)
H`
if k1  k 0 .
3. Probability of error are the special case of Bayes’ risk when C01  C10  1
and C11  C00  0 .
Hence, the LRT becomes
H1
N


d
(13)
d
(14)
H0
if k1  k 0 , and
H0
N


H`
 k1  k 0 T 

if k1  k 0 where  d  ceil 
 ln( k1 )  ln k 0  
Hence, the probabilities of error can be obtained as
pe  Pr H 0 PF  PrH1 PM
(15)
(16)
1
PrN  d H 0   PrN  d H1  ,
2
1   d 1k T N  k 0T  d 1k1T N  k1T
 1   0
e
 
e
2 
N
!
N
!
N 1
N 1
pe 
 0.5 
1  d 1k 0T N e

2 N 1
 k 0T
 k1T N e  k1T
N!

,


(17)
Prob. 2.2.15
Prove that
2
 X2


1 
  erfc *  X   1 exp   X , X  0
1 
 exp  
 2 
 2 
X 2 
X 2
X2




1
Using definition in (66) in the textbook, we have

erfc *  X    X

1
e
2

  X
z2
2

1
z 2
(1)
dz
de
z2
2
(2)
Using partial integration, which is given by
 udv  uv   vdu ,
we obtain
erfc *  X   
z2

de 2
1
z 2
(3)

z2
 1  2 1
 X
e
d
z
2
X
erfc *  X  
X 2
X
1

de 2   X
2

z
z2
e 2 dz .

2
(4)
z2
e 2  0.

1
For positive z,
1
2
Because the second integral on the right hand side
z 2 2
(RHS) is always positive, we have the upper bound.
Eq. (4) can rewritten again as
erfc *  X  
1
X2
1

de 2   X
3

z2
de 2

X 2
z 2
Let us consider only the second integral in (5). Using partial integration, we obtain

X
1
z
3
2
z2

de 2 
1
z
3
2
z2

e 2


 X
1
2
(5)
z2
1
,
e 2d
3

z
X

1
X2

1

e 2  X
4
z2
e 2 dz ,

X 2
z 2
With the same procedure used to obtain the upper bound, the above equation becomes
3
1

X 3
z
z2
de 2  

2
X2
e 2

1
X 3 2
.
(6)
Substituting (6) in (5), we have
erfc *  X  
Hence, the lower bound is obtained.
1
X 2
X2
de 2 

1
X
3
X2
e 2

2
.
(7)
Prob. 2.3.2
Using

M 1 M 1

    Pj Cij Z pr H R H j dR ,
i
j
i 0 j 0
(1)
show that Bayes’ test is equivalent to compute
M 1
 i   Cij Pr H j R 
(2)
j 0
and choose smallest
Bayes’ test chooses Zi that minimizes the risk function. In other words, we want to
minimize the risk function. Starting from (1), we have
M 1 M 1
 
i 0 j 0
M 1 M 1
 




 P j Cij Z i pr H j R H j dR
 Cij Z i P j pr H j R H j dR
i 0 j 0
(3)
M 1 M 1
 
 Z i pr R Cij Pr H j R dR
i 0 j 0
M 1
M 1
i 0
j 0


 pr R   Z  Cij Pr H j R dR.
i
Since terms inside the integrant are all positive, the multiplication of them are also
positive. Hence, we have
M 1
M 1


min   min  pr R   Z  Cij Pr H j R dR 


i 0 i j 0
(4)
M 1
 M 1

  min  Z  Cij Pr H j R dR 
 i j  0

i 0




To minimize (4), we need to assign Zi to the one that has the smallest value of
M 1
 Cij PrH j R dR .
(QED)
j 0
2. If
Cii  0
Cij  C
show that
Compute

,
Pr H j R
(5)

and choose the largest is equivalent to the test in (1).
(6)
Eq (2) can be rewritten as
M 1
M 1
j 0
j  0, j  i
 i   Cij Pr H j R  
C

M 1

j  0, j  i
M 1


C Pr H j R

Pr H j R


.
(7)

Using the fact that  Pr H j R  1, (7) becomes
j 0
M 1


Pr H j R  C 1  Pr H i R 
(8)
Minimization (8) is equivalent to maximization Pr H i R  .
(QED)
i  C

j  0, j  i
Prob. 2.3.3
 R  m 2 
k

pr H R H k  
exp  
k
2


 2
2


1
(1)
where
m1  2m,
m2   m,
m3  0,
(2)
m4  m,
m5  2m.
Hypotheses are equally likely and criterion is minimum probability of error.
To minimize probability of error, we use the MAP test which is written as


(3)
H i  arg max Pr H k R  .
 Hk

Using the definition of conditional probability, the above equation can be rewritten as

 p r H R H k Pr H k  
k


H i  arg max 
(4)


p r ( R)
 Hk



Since pr (R) is independent of choices of Hk, we have




H i  arg max pr H R H k Pr H k  
k
 Hk

Since hypotheses are equally likely, Eq (5) reduces to


H i  arg max p r H R H k  ,
k
 Hk


 1
 R  m 2  
k

 ,

 arg max
exp  
2

 
 k   2
2

 




 R  m 2  
k
 ,
 arg max  exp  
2

 
 k 
2

 



 R  m 2 
k
,
 arg min 
2


k

 2



 arg min R  mk .
 k


(5)

(6)
Answer #1
H1
H2
H3
-3m/2
3m/2
m/2
-1m/2
H5
H4
2. Compute probability of error.
Pr1   PrR  3m / 2 H1  ,
 R  2m 2 
dR .
exp  
2


 2
2


(7)
 z2 
dz  erfc * m / 2  .
exp  


2
2


(8)

  3m / 2
1
Let z = (R+2m)/, we obtain

Pr 1   m 2
1
Next, consider H2
Pr2   PrR  3m / 2 H 2   PrR  m / 2 H 2 
 R  m 2 
dR
exp  
2


 2
2


,
2

R  m dR
1

  m / 2
exp  

 2
2 2 

 3m / 2
  
(9)
1
(10)
Let z = (R+m)/, we obtain
 z2 
dz
Pr 2  
exp  
 2 
2


,
2

1
z

dR
 m / 2
exp  
 2 
2


2
Since, exp(-z /2) is symmetric function, we have
 m / 2
 
1
(11)
 z2 
dz
Pr 2  
exp  
 2 
2


 z2 
1

dR
 m / 2
exp  
 2 
2


 2erfc *m / 2 

m / 2
1
(12)
Considering H3
Pr3   PrR  m / 2 H 3   PrR  m / 2 H 3 
Using the sane procedure in H2, we have
Pr3   2erfc * m / 2 
Next, consider H4, we have
Pr4   PrR  m / 2 H 4   PrR  3m / 2 H 4 
Using the sane procedure in H2, we obtain
Pr4   2erfc * m / 2 
(13)
(14)
(15)
(16)
Finally, consider H5, we have
Pr5   PrR  3m / 2 H 5 
Using the same procedure in H1, we obtain
Pr 5   erfc * m / 2 
(17)
(18)
Using (8), (12), (14), (16), and (18), the probability of error is written as
5
Pr    Pr H i  Pr i 
i 1
1 2  2  2 1
erfc * m / 2 
5
8
 erfc * m / 2 
5

(ANS)
2.3.5

R2
1  R2
p r , r H R1 , R2 H k   21k  2k 1 exp   1  2
1 2
k
 2  2  2
2k
 1k


,


(1)
where
2
2
 11
  21
  n2 ,
2
 12
  s2   n2 ,
2
 22
  n2 ,
2
 13
  n2 ,
2
 23
  s2   n2 .
(2)
The cost matrix is
0 1 1 
C (i, j )  1 0  
1  0 
(3)
PrH 2   PrH 3   p. Define l1  R12 and l 2  R22 .
1. Find the test.
Let  o2   s2   n2 . Using Eq. (102) in the textbook, the likelihood ratios are given by
1 R1 , R2  
pr , r H R1 , R2 H 2 
1 2
2
,
pr , r H R1 , R2 H1 
1 2
1

R22 

2 
 2   12  22


,
2 
 1  R2
R
211 21 1 exp   12  22 
 2   11  21 
 R2  1
n 
1 
1 



,

exp



2
2 


2



 o
n 
 o


2
212 22 1 exp  1  R12
and
 2 R1 , R2  

pr , r H R1 , R2 H 2 
1 2
3
pr , r H R1 , R2 H1 
1 2
1
 1  R2
R22 
1

213 23  exp   2  2 
 2   13  23 

 1  R2
R22 
1
1


211 21  exp   2  2 
 2   11  21 
1
(4)

  n
o
 R2  1

1 
 exp  2 
.

 2   o2  n2 

(5)
Using (103) by replacing H k by H k 1 in the textbook, we have
H2 ,H3
 R2  1
 

1  

p n  exp  1 

(1  2 p)  p  1 n
 2   o2  n2  
o 
o
H ,H
1
 R2  1

1 
 exp  2 


 2   o2  n2 

3
or

p n
o
H 2 ,H3
 l  1


1  
 exp  1 


(1  2 p)  p  1 n
 2   o2  n2  

o
H ,H
1
 l

 exp  2
 2

 1
1 



  2  2 
n 
 o
3
 l  1
 l
1 
  1    exp  2
exp  1 

 2   o2  n2 
 2
 1
1 



  2  2 
n 
 o
H 2 , H3


H1, H 3
(1  2 p)   o

p   n



(6)
Using (104) by replacing H k by H k 1 in the textbook, we have

p n
o
H3 , H 2
 R2  1


1  

 exp  2 

(1  2 p)  p  1 n
 2   o2  n2  

o

 H ,H
1
2
 R2  1

1 

 exp  1 

2
2 

2


  o  n 

or

p n
o
 l

 exp  2
 2

 l
exp  2
 2
 1
1 



  2  2 
n 
 o
H3 ,H 2


(1  2 p)  p  1 n

o
H1, H 2
 l  1
 1
1 
1 

  1    exp  1 



  2  2 
 2   o2  n2 
n 
 o
H3 ,H 2


H1, H 2
 l  1

1 
 exp  1 


 2   o2  n2 

(1  2 p)   o

p   n
Using (104) by replacing H k by H k 1 in the textbook, we have

p  n
o
H 3 , H1
 R2  1


1  

 exp  2 

p  n
 2   o2  n2  

o

 H ,H
2
1
 R2  1

1 

 exp  1 

 2   o2  n2 






(7)
Taking log both sides, we have
H 3 , H1
R22  1
R12  1
1  
1 


,




2   o2  n2  
2   o2  n2 
H 2 , H1

R22
H 2 , H1
R12

,


2
2
H 3 , H1
H 3 , H1


R22
R12 ,
H 2 , H1
H 3 , H1

l2

l1 .
(8)
H 2 , H1
We introduce two additional parameters which are defined as
 l  1
1 
,
Z1  exp  1 

 2   o2  n2 
 l
Z 2  exp  2
 2
(9)
 1
1 

.

  2  2 
n 
 o
(10)
Eq. (6) and (7) become
H 2 ,H3

Z1  1   Z 2

H1, H 3
(1  2 p)   o

p   n

  

(11)
H3 , H 2

Z 2  1   Z1


(12)
H1, H 2
It is simple to show that (8) leads to
H 3 , H1
Z2


Z1
H 2 , H1
Using (11)-(13), we can draw decision boundary as
(13)
Z2
/(1-)
(13)
(11)

H3
H2
H1
(12)

l2
/(1-)
Z1
/(1-)
l1
/(1-)
(8)

(7)
H3
H2
(6)
H1

2. Find probability of error
Pr(1 )  1  PrH1 H1 
Pr(1 )  1 
 / 2     1 Z1


-


 p z1 , z 2 H 1 Z1 , Z 2 H1 dZ 2 dZ1
0
  Z1  / 1 
0

 p z1 , z 2 H 1 Z1 , Z 2 H1 dZ 2 dZ1
 / 2  
(14)
0
Pr(2 )  1  PrH 2 H 2 
Pr(2 )  1 

Z1


 / 2     Z1  / 1 
p z , z H Z1 , Z 2 H 2 dZ 2 dZ1
1 2
2
(14)
 Z1
-   p z , z H Z1 , Z 2 H 2 dZ 2 dZ1
1 2 1
 0
Pr(3 )  1  PrH 3 H 3 
Pr(3 )  1 
 / 2  

0

-



  1 Z1
p z , z H Z1 , Z 2 H 23 dZ 2 dZ1
1 2
2

 p z1 , z 2 H 1 Z1 , Z 2 H 3 dZ 2 dZ1
(16)
 / 2   Z1
where
R1
Z1
p z , z H Z1 , Z 2 H k  
1 2
k
R2
Z1
R1
Z 2
R , R H  .
p
R2 r1 , r2 H k 1 2 k
Z 2
(17)
3
Pr    Pr H i  Pr i  .
(18)
i 1
#3. if   0 , problem reduce to
 l  1
 l
1 
  exp  2
exp  1 

 2   o2  n2 
 2
 1
1 



  2  2 
n 
 o
H 2 ,H3


H1, H 3
(1  2 p)   o

p   n



(19)
 l
exp  2
 2
H3 , H 2
 l  1
 1
1 
1   (1  2 p)   o


  exp  1 



  2  2 
  2  2  
2
p

 n
n 
n 
 o
 o

H1, H 2



(20)
H 3 , H1
l2


(21)
l1
H 2 , H1
Combining H2 and H3 to single hypothesis Hc, (19)-(21) become
 l  1
 l
1 
  exp  2
exp  1 

 2   o2  n2 
 2
which is exactly in (2.2.17)
HC
 1
1   (1  2 p)   o




  2  2  
p
n
n 
 o
H1

 , (22)
