COMBINATION OF CORRELATED RANDOM MEASURES
FOR IMPROVING THE ESTIMATION OF A VARIABLE
http://www.essi.fr/~leroux/two_correlated_measures.doc
In the very interesting introduction of the following book
Maybeck, Peter S., Stochastic Models, Estimation, and Control, Vol. 1, ACADEMIC PRESS,
1979. (Introduction available on the famous Kalman Filter website
http://www.cs.unc.edu/~tracker/media/pdf/SIGGRAPH2001_CoursePack_08.pdf
Appendix B1-Maybeck)
the author proposes a formulation of the combination of two random independent measures of
a variable and shows that this combination increases the accuracy of the estimation of the
variable, yielding an original approach to the Kalman filter construction. The present exercise
is based on the same approach but takes into account the possible correlation between the two
measures
The following problem is treated: we consider two different measures of a random variable.
These two measures yield a random result. This couple of variables is Gaussian, characterized
by the mean values (m,m), the variances (2,2) and the correlation coefficient r. This
correlation coefficient is supposed to be positive. 1. How is it conceivable to combine these
two measures in order to improve the accuracy of the measured variable estimation? (A
typical situation is the following: a phenomenon is measured by two different methods; the
first one yields a result (the mean) with a given incertitude (the variance); the second yields a
different result with a different incertitude; these two measurements are correlated and the
correlation coefficient is also known.). 2. Furthermore show that this combination yields the
minimal variance estimation of the measured variable.
In the case of a couple of Gaussian variables, the quadratic form of the exponent has the
following expression
r
 1
 2
1
 1 1  2







.
x

m
,
y

m
.
1
2  r
1
2(1  r 2 )

2
 1  2  2


.  x  m1   .
   y  m 
2


(1)
Considering that it is the same random variable that is measured, y is equal to x and the
exponent becomes

1
2(1  r 2 )
. x  m1 ,  x  m2 
r
 1
 2
 1  2
. 1
 r
1

2
 1  2  2
1


 .   x  m1   .
   x  m 
2


(2)
The computation of the mean and variance of the new random variable is deduced from the
fact that it is the same variable that is measured: in the previous expression,

 1
2r
1
2
2










x

m

x

m
.
x

m

x

m
1
1
2
2 ,
2  2
2
2(1  r )   1
2
1  2

1
(3)
the inverse of the coefficient of x2 yields the new variance; then the coefficient of x yields the
new mean. So, the inverse of the new variance is
1
2

so that
 1
2r
1

.


(1  r 2 )   12  1  2  22

,


(4)
 .
.
  (1  r )
  2 r   
(5)
1
2
2
2
2
1
2
2
2
2
1
2
1
where the denominator is positive. We note that 2 is less than min(2,2): supposing that
2 <2 and naming  / = g  1, we see that (fig.1):

1
(1  r )
s(r , g ) 
 (1  r )

 1.

1  2rg  g
(1  r )  r  g 
2
2
2
2
2
(6)
2
2
1
g
s
s
g
g
r
r
r
Fig. 1: Representations of the function s(r, g) of eq. (6).
The equality holds when g = r: there is always a reduction of the variance except when
=r. When the two variances are equal,
  (1  r )
2
2
 12
2(1  r )
2
 (1  r )
 12
2
.
(7)
If the correlation coefficient is equal to one, the two measures must present the same
characteristics; in particular the two mean values and the two variances must be equal; in
that special case the new variance is equal to the old ones.
The expression of the mean value is deduced from
m
2

 1

r
1




.
m

m

m

m
1
2
2.
2  2 1
2
(1  r )   1
2
1  2

1
(8)
Using the expression of the variance (5)
m

or
m  m1 



 2 .  2  r. 1 .m1   1 .  1  r. 2 .m2
.
 22  2r 1  2   12


 1 .  1  r. 2
.m2  m1  .
 2 .  2  r. 1   1 .  1  r. 2




(9)
(10)
When there is no improvement as considered above (=r.), we note that m reduces to x
When >/rthe weight of m in the linear combination (9-10) is negative. When the two
variances are equal,
m
m1  m2
.
2
(11)
2. A linear combination of the form (10) of the two correlated variables yields the
minimal variance estimator: consider the linear combination:
z  x  a. y  x 
(12)
of the two initial random variables. Its mean value is
m(a)  m1  a.m2  m1 
(13)
and its variance is
 z2  Ez  m1  a.m2  m1 2  Ex  a y  x   m1  a.m2  m1 2 ,
or
 z2  E1  a x  m1   a y  m2 2 ,
(14)
(15)
that we express as a function of a, ,  and r:
 z2  1  a 2 12  2a1  a r1 2  a 2 22 .
3
(16)
The variance
 z2 is minimal when
a
   r  ,
  2r   
1
1
(17)
2
2
2
1
1
2
2
So, the mean value of the combination is given by eq. (10), and the value of the minimum is
 
min  z2   12 
 12  1  r 2 2 ,
 12  2r 1 2   22
(18)
that can be written in the form (5).
July 5, 2008
Comments are welcome, concerning especially examples of application on real data: mailto
Joel Le Roux leroux@polytech.unice.fr
Another obtention of the solution :
Note that it is possible to obtain this result by applying the usual approach (variance of a sum
of independent random variables) after an intermediate computation of a variable v
independent of x

(21)
v  y  r . . x,

(22)

  

2
1
E (v.x)  E   y  r. .x .x   E  y.x   r. .E  x ,
  


2
2
1
1
2

E (v.x)  r. .  r. .  0.

(23)
2
2
1
2
1
1
The variance of
v
is

 
E (v )  E   y  r. .x  ,

  

2
 2





2
2
2
E (v )  E  y  2.r. .x. y   r.  .x 2 ,

1
  1  


  

E (v )     2.r. .r. .   r.  .


  

2
2
(24)
2
1
(25)
2
2
2
2
2
2
1
2
1
1
E (v )    2.r .  r . ,
2
2
2
2
2
2
2
E (v )   .1  r .
2
2
2
(This expression will be used later in eq.(33).)
4
2
2
2
2
1

,


(26)
(27)
(28)
We consider again the linear combination
z  x  a. y  x ,
(29)
where we replace y by its value as a linear combination of the independent variables x and v
y  v  r.

. x,

(30)
2
1



z  x  a. v  r. .x  x ,



(31)
2
1


 
z  1  a.1  r.  .x  a.v.
 


The variance of z is the sum of the variances of two independent variables


 
  1  a.1  r.   .  a . .1  r ,
 


(the second term of the sum comes from eq. (28).)
2
(32)
1
2
2
2
2
z
2
2
1
2
(33)
2
1
  1  a    2a1  a r   a  .
2
2
2
z
2
1
1
2
2
2
(34)
The minimum of  z2 is obtained when
 21  a   21  a r   2ar   2a  0,
(35)
a  2r   a      r.   0,
(36)
2
2
1
1
2
1
2
1
2
2
2
1
2
a
2
1
1
2
   r. 
.
  2 r   
1
1
2
2
2
1
1
yielding the same result as above(17).
5
2
2
(37)