General Classes of Lower Bounds on
Outage Error Probability and MSE in
Bayesian Parameter Estimation
Tirza Routtenberg
Dept. of ECE, Ben-Gurion University of the Negev
Supervisor: Dr. Joseph Tabrikian
Outline

Introduction

Derivation of a new class of lower bounds on the
probability of outage error

Derivation of a new class of lower bounds on the
MSE

Bounds properties: tightness conditions, relation
to the ZZLB

Examples

Conclusion
Introduction
Bayesian parameter estimation
Goal: to estimate the unknown parameter θ
based on the observation vector x.
Assumptions:

θ and x are random variables

The observation cdf Fx and
f |x  | x 
posterior pdf f |x  | x  are known
Applications:
Radar/Sonar, Communication, Biomedical, Audio/speech,…

Introduction
Parameter estimation criteria
Mean-square error (MSE) E  ˆ   2 


Probability of outage error Pr  ˆ    h 
2


CC ˆˆ
100
90

80
70
60
50
40
30
20
10
0
-10
h / 2
-5
0
5
10
h/2
ˆˆ

Introduction
Parameter estimation criteria
Provides meaningful
information in the presence
of large errors case.

Dominated by the all error
distribution.
Large-errors
Small
errors
SNR
Prediction of the operation
region.
Probability of
outage error


Threshold
MSE
Advantages of the probability
of outage error criterion:
Large-errors
Threshold
Small
errors
SNR
Introduction
MMSE estimation
The minimum MSE is attained by MMSE:
ˆMMSE
2

ˆ     E  | x.
 arg min
E



ˆ
f |x  | x 
ˆMMSE

Introduction
h-MAP estimation
The h-MAP estimator is
h
2
h
ˆ 
2
ˆhMAP  arg max
ˆ 

f |x  | x 
ˆ 
f |x  | x  d
ˆhMAP

The corresponding minimum probability of h-outage error is
h
ˆ 


h
 ˆ
2
min
Pr        1  E max
f  | x  d  .
ˆ ˆ  h  |x
ˆ

2

2


Performance lower bounds
Motivation
Performance analysis

Threshold prediction

System design

Feasibility study
PERFORMANCE
MEASURE

Threshold
bound
SNR or number of
samples
Performance lower bounds
Bounds desired features
Computational simplicity

Tightness

Asymptotically coincides with the optimal performance

Validity: independent of the estimator.
PERFORMANCE
MEASURE

Threshold
bound
SNR or number of
samples
Previous work:
probability of outage error bounds

Most of the existing bounds on the probability of
outage error are based on the relation to the
probability of error in decision procedure
(binary/multiple).

Kotelnikov inequality - lower bound for uniformly
distributed unknown parameter.
Previous work: Bayesian MSE bounds
Bayesian MSE bounds
Weiss–Weinstein class
The covariance inequality
• Bayesian Cramér–Rao (Van Trees, 1968)
• Bayesian Bhattacharyya bound
(Van Trees 1968)
• Weiss–Weinstein (1985)
• Reuven-Messer (1997)
• Bobrovski–Zakai (1976)
Ziv-Zakai class
Relation to probability of error
in decision problem
• Ziv–Zakai (ZZLB) (1969)
• Bellini–Tartara (1974)
• Chazan–Zakai–Ziv (1975)
• Extended ZZLB (Bell, Steinberg,
Ephraim,Van Trees,1997)
General class of outage error
probability lower bounds
The probability of outage error
h
 ˆ
Pr        E 
2

 
h / 2
h/2
?
ˆ  
(Reverse) Hölder inequality for p  1
1
1




1

p
E  uh  x,  vh  x,    E  uh  x,  p  E  vh  x,  1 p  .




p
Taking
h

ˆ
1





uh  x,   
2

0 otherwise
h / 2
h/2
ˆ  
General class of outage error
probability lower bounds
h

Pr  ˆ     
2

 ˆ  h2

1  E   ˆ h f |x  | x  vh  x,   d  E
 2

1
p
p 1
p
1


1 p
v
x
,

 
 h


Objective: obtain valid bounds, independent of ˆ .
General class of outage error
probability lower bounds
Theorem:
A necessary and sufficient condition to obtain a valid
bound which is independent of the estimator, is that the
function
f |x  | x  vh  x,  
is periodic in θ with period h, almost everywhere.
General class of outage error
probability lower bounds
Using Fourier series representation
f |x  | x  vh  x,  

 a  x, h  e
k
k 
i
2 k

h
,
a  x, h   l  
k
2
the general class of bounds is
h

Pr  ˆ      Bh ,
,p
2

2
1
p
1
p
p 1
Bh  1  h E  a0  x, h   E
,p
2
p 1
p
1


p
2 k
i
 1 p
  
p 1


h
a
x
,
h
e
f

|
x
d








k

|
x
  k 





Example: Linear Gaussian model
The model
x    n,  ~ N   ,  2  , n ~ N  0,  n2  ,
The minimum h-outage
error probability:

1  erf h / 2 2 2| x

The single coefficient
bound:
ak  x, h   0 k  0
Bh /2, p  1 
1/ p
h
 2 
2
 |x
B  max  Bh /2,1 , 0 
1/2 p
 p 1 


 p 
p 1
2p

2
 |x
2 n2
 2
   n2
The tightest subclass of lower bounds
■
The bound is maximized w.r.t.
a  x, h  for given p
k
Convergence condition:
There exists l0h(θ,x), α>0 such that for all │l│≥│l0h(θ,x)│
f |x   lh | x  
1
l
This mild condition guaranties that
converges for every p≥1.
a.e.
 1


l 
f |x
p
p 1
  lh | x 
The tightest subclass of lower bounds
Under the convergence condition, the tightest bounds are
h

Pr  ˆ      Bhopt
/ 2, p
2

 f   lh | x 
 |x
p
p1
l
Repeat for all x
and   0, h
h

1  E   f |x   lh | x 
l
 0
p
p 1
d 

p 1


p
p



h


 1  E     f |x p 1   lh | x   d  ,
0 

l 




h – sampling period
p 1
The tightest subclass of lower bounds
Under the convergence condition, the tightest bounds are
h

Pr  ˆ      Bhopt
/ 2, p
2

p 1


p
p



h


 1  E     f |x p 1   lh | x   d  ,
0 

l 




Properties:
■ The bound exists p  1.
■ The bound becomes
tighter by decreasing p.
■ For p→1+, the tightest
bound is
opt
h /2,1
B
h

 1  E  max f |x   lh | x d  .
 0 l

h – sampling period
p 1
General class of MSE lower bounds
The probability of outage error and MSE are related via:
• Chebyshev's inequality
2
h
4  ˆ
 ˆ
Pr        2 E    


2 h

• Known probability identity
E  ˆ  


2
 1


2


0
h

h Pr  ˆ     dh
2

General class of MSE lower bounds
New MSE lower bounds can be obtained by using
2
1 
h



ˆ
E      h Pr  ˆ     dh

 2 0
2

and lower bounding the probability of outage error
2
1 


ˆ
E  

hBh dh.

 2 0
For example:
■
General class of MSE bounds:
■
The tightest MSE bound:
2
1 


ˆ
E      hBh /2, p dh

 2 0
2
h
1  



ˆ
E      h 1  E  max f |x   lh | x d   dh

 2 0 
 0 l
 
General class of lower bounds on
different cost functions
Arbitrary cost function C(·) that is non-decreasing and
differentiable satisfies


1  h  ˆ
h


ˆ
E C       C   Pr       dh

 2 0 2 
2
Thus, it can be bounded using lower bounds on the
probability of outage error


1  h


ˆ
E C       C  Bh dh

 2 0 2
Examples: the absolute error, higher moments of the error.
Properties: Relation to the ZZLB
Theorem
The proposed tightest MSE bound is always tighter
than the extended ZZLB.
The extended ZZLB is
1   h 

hE    min  f |x   lh | x  , f |x   lh  h | x  d  dh

0
2
 0 l 

The tightest proposed MSE bound can be rewritten as
1   h 
 
hE     f |x   lh | x   max f |x   lh | x   d dh

0
0
l
2
 
  l 
Properties: Relation to the ZZLB
ZZLB
a1 ,..., aN
4
min  al , al 1 

N 1
l 1
2
2
The proposed bound
8
2
min  al , al 1 
1
1
7
4
a1 ,..., aN
max out
1

6
N
a  max ak
l 1 l
k
2
4
8
2
1
1
7
7
14
For any converging sequence of non-negative numbers

 a  max a
l 
l
k
k


 min  a , a 
l 
l
l 1
Therefore,
1   h 
 
hE     f |x   lh | x   max f |x   lh | x   d dh  ZZLB

0
0
l
2
 
  l 
Properties: unimodal symmetric pdf
Theorem:
A. If the posterior pdf f θ| x(θ| x) is unimodal, then the
proposed tightest outage error probability bound
coincides with the minimum probability of outage error
for every h>0.
B. If the posterior pdf f θ| x(θ| x) is unimodal and
symmetric, then the proposed tightest MSE bound
coincides with the minimum MSE.
Example 1
Statistics
 x  2x
e 1

 21
f | x  | x   
x
 x  22
 2 e
 2
1  1, 2  10
1 w.p. 0.5
x
2 w.p. 0.5
 0
 0
Example 2
The model
x   n
Statistics
1
f    13  6
6

n ~ N 0,  2
 2  100

Conclusion

The concept of probability of outage error criterion is
proposed.

New classes of lower bounds on the probability of
outage error and on the MSE in Bayesian parameter
estimation were derived.

It is shown that the proposed tightest MSE bound is
always tighter than the Ziv-Zakai lower bound.

Tightness of the bounds:

Probability of outage error- condition: Unimodal posterior pdf.

MSE – condition: Unimodal and symmetric posterior pdf.