PROBABILITY AND STATISTICS
FOR ENGINEERING
One Function of Two Random Variables
Hossein Sameti
Department of Computer Engineering
Sharif University of Technology
One Function of Two Random Variables
 X and Y : Two random variables
 g(x,y): a function
 We form a new random variable Z as
Z  g ( X , Y ).
 Given the joint p.d.f f XY ( x, y ), how does one obtain f Z (z ), the p.d.f of Z ?
Practical Viewpoint
 A receiver output signal usually consists of the desired signal buried in noise
 the above formulation in that case reduces to Z = X + Y.
 We have:
FZ ( z )  P Z ( )  z   P g ( X , Y )  z   P( X , Y )  Dz 


x , yDz
f XY ( x, y )dxdy,
 Where Dz in the XY plane represents the region where g ( x, y )  z .
 Dz need not be simply connected
Y
 First find the region FZ (z) for every z,
 Then evaluate the integral there.
Dz
Dz
Dz
X
Example
 Z = X + Y. Find
f Z (z ).
FZ ( z )  P  X  Y  z   

y  

z y
x  
f XY ( x, y )dxdy,
 Integrating over all horizontal strips (like the one in figure) along the x-axis
 We can find f Z (z ) by differentiating FZ (z) directly.
y
x z y
x
Recall - Leibnitz Differentiation Rule
Suppose
H ( z) 
Then

b( z )
a( z)
h ( x, z )dx.
b ( z ) h ( x , z )
dH ( z ) db( z )
da ( z )

h b( z ), z  
h a ( z ), z   
dx.
a
(
z
)
dz
dz
dz
z
Using the above,
 
z  y f
  z y

XY ( x, y ) 
f Z ( z)    
f XY ( x, y )dx  dy    f XY ( z  y, y )  0  
dy

 z 


z








f XY ( z  y, y )dy.
Example – Alternate Method of Integration
FZ ( z )  
f Z ( z) 

x  

zx
y  
y
f XY ( x, y )dxdy,

dFZ ( z )

x  
dz


x  
  z x

  y  f XY ( x, y )dy dx
 z

yzx
x
f XY ( x, z  x )dx.
If X and Y are independent, then f XY ( x, y )  f X ( x) fY ( y )
f Z ( z)  

y  
f X ( z  y ) fY ( y )dy  

x  
f X ( x ) fY ( z  x )dx.
This integral is the standard convolution of the functions f X (z ) and fY (z )
expressed in two different ways.
Example – Conclusion
 If two r.vs are independent, then the density of their sum equals the
convolution of their density functions.
 As a special case, suppose that
- f X ( x)  0 for x  0
- fY ( y )  0 for y  0,
 then using the figure we can determine the new limits for Dz .
y
(z,0)
x z y
(0, z )
x
Example – Conclusion
FZ ( z )  
In that case
or
f Z ( z)  


y 0 z  x 0

z y
z
z
y 0

z y
x 0
f XY ( x, y )dxdy
z

 f ( z  y, y )dy, z  0,

f XY ( x, y )dx  dy  0 XY


0,
z  0.

On the other hand, by considering vertical strips first, we get
FZ ( z )  
or
f Z ( z)  
z
x 0
z
x 0

z x
y 0
f XY ( x, y )dydx
 z f ( x ) f ( z  x )dx, z  0,
Y
f XY ( x, z  x )dx   y 0 X

0,
z  0,
if X and Y are independent random variables.
Example
 X and Y are independent exponential r.vs
 with common parameter ,
 let Z = X + Y.
 Determine f Z (z ).
Solution
f X ( x)  e xU ( x ),
z
f Z ( z)    e
0
2 x
e
fY ( y )  e yU ( y ),
 ( z  x )
dx  e
2 z

z
0
dx  z2ezU ( z ).
Example
 This example shows that care should be taken in using the convolution
formula for r.vs with finite range.
 X and Y are independent uniform r.vs in the common interval (0,1).
 let Z = X + Y.
 Determine f Z (z ).
Solution
 Clearly, Z  X  Y  0  z  2
 There are two cases of z for which the shaded areas are quite different in
shape and they should be considered separately.
Example – Continued
y
y
x z y
x z y
x
(a ) 0  z  1
FZ ( z )  

z
y 0
z
y 0
2

z y
x 0
1 dxdy
( z  y )dy
z
 , 0  z  1.
2
( b) 1  z  2
x
FZ ( z )  1  PZ  z 
 1 
 1 
1
y  z 1
1
y  z 1

1
x z y
1 dxdy
(1  z  y )dy
(2  z ) 2
 1
, 1  z  2.
2
Example – Continued
 So, we obtain
0  z  1,
dFZ ( z )  z
f Z ( z) 

dz
2  z, 1  z  2.
 By direct convolution of f X (x ) and fY ( y ), we obtain the same result.
 In fact, for 0  z  1
z
f Z ( z )   f X ( z  x) fY ( x)dx   1 dx  z.
0
f X ( z  x) fY ( x)
f X ( z  x)
fY (x)
x
1
z 1
z
0  z 1
x
z
x
Example – Continued
f Z ( z)  
and for 1  z  2
1
z 1
1 dx  2  z.
f X ( z  x)
fY (x)
x
1
z 1
f X ( z  x) fY ( x)
z
x
z 1
x
1
1 z  2
f Z (z )
This figure shows f Z (z ) which agrees with the
convolution of two rectangular waveforms as well.
0
1
2
z
Example
Let
Z  X  Y . Determine its p.d.f f Z (z ).
Solution
FZ ( z )  P X  Y  z   

y  

z y
x  
y
f XY ( x, y )dxdy
y
x y  z
x yz
and hence
fZ ( z) 
x

dFZ ( z )
 

y   z
dz


z y
x 


f XY ( x, y )dx  dy  
f XY ( y  z, y )dy.


If X and Y are independent,

f Z ( z)   f X ( z  y) f Y ( y)dy  f X ( z )  f Y ( z)

which represents the convolution of f X ( z ) with fY (z ).
Example – a special case
Suppose
f X ( x)  0, x  0, and fY ( y)  0, y  0.
y
For z  0,
FZ ( z )  
and for z  0,
FZ ( z )  

y 0

y  z

z y
x 0

z y
x 0
f XY ( x, y )dxdy
xz y
x
z
z
f XY ( x, y )dxdy
y
After differentiation, this gives
  f ( z  y , y )dy ,
  XY
f Z ( z )   0 
f ( z  y , y )dy ,

   z XY
xz y
z  0,
z  0.
z
x
Example
Given Z = X / Y, obtain its density function.
Solution
 X /Y  z
- X  Yz if Y  0,
- X  Yz if
Y  0.
A  Y  0
X / Y  z
__
 Since A  A  , by the partition theorem, we have
X / Y  z  ( X / Y  z )  ( A  A )
 ( X / Y  z )  A ( X / Y  z )  A 
 and hence by the mutually exclusive property of the later two events
P  X / Y  z   P  X / Y  z, Y  0  P  X / Y  z, Y  0
 P  X  Yz , Y  0  P  X  Yz , Y  0 .
Example – Continued
y
y
x  yz
x  yz
x
x
P X  Yz , Y  0
P X  Yz , Y  0
 Integrating over these two regions, we get
FZ ( z )  

y 0

yz
f XY ( x, y )dxdy  
x  
0
y  


x  yz
f XY ( x, y )dxdy.
 Differentiation with respect to z gives

0
0

f Z ( z )   yf XY ( yz , y )dy   (  y ) f XY ( yz , y )dy

  | y | f XY ( yz , y )dy,

   z  .
Example – Continued
If X and Y are nonnegative random variables, then the area of integration
reduces to that shown in this figure:
y
x  yz
x
This gives
or
FZ ( z )  

y 0

yz
x 0
f XY ( x, y )dxdy
 y f ( yz , y )dy,
z  0,
f Z ( z )  0 XY

0,
otherwise.
Example
X and Y are jointly normal random variables with zero mean so that
f XY ( x, y ) 

1
21 2 1  r
2
e
 x 2 2 rxy y 2
1



2 (1 r 2 )   12  1 2  22




.
Show that the ratio Z = X / Y has a Cauchy density function centered at r 1 /  2 .
Solution
Using

0
0

f Z ( z )   yf XY ( yz , y )dy   (  y ) f XY ( yz , y )dy

  | y | f XY ( yz , y )dy,
   z  .

and the fact that
f Z ( z) 
f XY ( x, y)  f XY ( x, y), we obtain
2
21 2 1  r
2


0
ye
 y 2 / 2 02
 02 ( z )
dy 
,
2
1 2 1  r
Example – Continued
where
Thus
 ( z) 
2
0
z2
 12

1  r2
2 rz
 1 2

1
.
 22
 1 2 1  r 2 / 
f Z ( z)  2
,
2
2
2
 2 ( z  r 1 /  2 )   1 (1  r )
which represents a Cauchy r.v centered at r 1 /  2 .
Integrating the above from   to z, we obtain the corresponding
distribution function to be
FZ ( z ) 
1 1
 z  r 1
 arctan 2
.
2
2 
1 1  r
Example
Z  X 2  Y 2 . Obtain f Z (z ).
Solution
We have
FZ ( z )  
z
y  z

z y2
f XY ( x, y )dxdy.
x  z  y 2
But, X 2  Y 2  z represents the area of a circle with radius
FZ ( z )  P X 2  Y 2  z   

X 2 Y 2  z
z , and hence
f XY ( x, y )dxdy.
y
z
X 2 Y 2  z
This gives after repeated differentiation
f Z ( z)  
z
y  z
1
2 z y
2
f
XY

z
( z  y 2 , y )  f XY (  z  y 2 , y ) dy.
 z
x
Example
2
X and Y are independent normal r.vs with zero Mean and common variance  .
Determine f Z (z ) for Z  X 2  Y 2 .
Solution
Direct substitution with r  0,  1   2   gives
f Z ( z) 


z
y  z
e  z / 2
 2
1
e  z / 2

 ( z  y 2  y 2 ) / 2 2 
2
e
 dy 
2
2 
2

 2

2 zy 
1
2

 /2
0
2

z
0
1
zy
2
dy
z cos 
1  z / 2 2
d 
e
U ( z ),
2 2
z cos 
where we have used the substitution y 
z sin  .
Result - If X and Y are independent zero mean Gaussian r.vs with common
variance  2 , then X 2  Y 2 is an exponential r.vs with parameter 2 2 .
z.
Example
Let Z  X 2  Y 2 . Find f Z (z ).
Solution
This corresponds to a circle with radius

FZ ( z ) 
f Z ( z) 

z
z
z
z y
2
2
z
y  z
f
XY

z. Thus
z2  y2
x 
z2  y2
f XY ( x, y )dxdy.

( z 2  y 2 , y )  f XY (  z 2  y 2 , y ) dy.
( )
*
If X and Y are independent Gaussian as in the previous example,
z
z
1
2 z  z 2 / 2 2 z
1
( z 2  y 2  y 2 ) / 2 2
f Z ( z )  2
e
dy

e
dy
2

2
2 2 2
2
2
0
0

z y
z y

2z
 2
e z
2
/ 2 2

/2
0
2
2
z cos
z
d  2 e  z / 2 U ( z ),
z cos

Rayleigh distribution
Example – Conclusion
 If W  X  iY , where X and Y are real, independent normal r.vs with zero
mean and equal variance,
 then the r.v W  X 2  Y 2 has a Rayleigh density.
 W is said to be a complex Gaussian r.v with zero mean, whose real and
imaginary parts are independent r.vs.
 As we saw its magnitude has Rayleigh distribution.

What about its phase   tan 1 
X
Y

?

Example – Conclusion
 Let U  tan   X / Y ,
 U has a Cauchy distribution with
fU (u ) 
1/
,
2
u 1
   u  .
 As a result
1 /  ,   / 2     / 2,
1
1
1/ 
f ( ) 
fU (tan  ) 


otherwise .
| d / du |
(1 / sec 2  ) tan 2   1  0,
 The magnitude and phase of a zero mean complex Gaussian r.v has
Rayleigh and uniform distributions respectively.
 We will show later, these two derived r.vs are also independent of each
other!
Example
 What if X and Y have nonzero means  X and Yrespectively?
Solution
 Since f XY ( x, y ) 
1
2
2
e
[( x   X ) 2  ( y  Y ) 2 ] / 2 2
,
substituting this into ( ), and letting
*
x  z cos  , y  z sin  ,  
ze  ( z   ) / 2
f Z ( z) 
2 2
2
we get
2
 X2  Y2 ,  X   cos  , Y   sin  ,
2
/2

 /2
e
z cos(  ) /  2
 e  z cos(  ) / 
2
d
Rician p.d.f.
3/2
ze ( z   ) / 2  /2 z cos(  ) /  2
z cos(  ) /  2

e
d


e
d 

2




/2

/2


2
2
2
2
ze ( z   ) / 2
 z 

I
 2 ,
0
2 2
 
2
where
I 0 ( ) 
1
2

2
0
2
2
e cos(  ) d 
1



0
e cos d
the modified Bessel
function of the first
kind and zeroth order
Application
 Fading multipath
 situation where there is
- a dominant constant component (mean)
- a zero mean Gaussian r.v.
Multipath/Gaussian
Line of sight
noise
signal (constant)
a

 The constant component may be the line of sight signal and the zero
mean Gaussian r.v part could be due to random multipath components
adding up incoherently (see diagram below).
 The envelope of such a signal is said to have a Rician p.d.f.
Example
Z  max( X , Y ), W  min( X , Y ). Determine f Z (z ).
Solution
 nonlinear operators
 special cases of the more general order statistics.
 We can arrange any n-tuple X 1 , X 2 ,  , X n , such that:
X (1)  X ( 2 )    X ( n ) ,
X (1)  min  X 1 , X 2 , , X n  ,
X ( n )  max  X 1 , X 2 , , X n  .
Example - continued

X1, X 2 ,  , X n
represent r.vs,
 The function X ( k ) that takes on the value x( k ) in each possible
sequence x1 , x2 ,  , xn  is known as the k-th order statistic.
 ( X (1) , X (2) , , X ( n ) ) represent the set of order statistics among n
random variables.
 R  X ( n )  X (1) represents the range, and when n = 2, we have the
max and min statistics.
Example - continued
 Since
 we have
X ,
Z  max( X , Y )  
Y ,
X Y,
X Y,
FZ ( z )  P max( X , Y )  z   P X  z, X  Y   Y  z, X  Y 
 P  X  z, X  Y   P Y  z, X  Y ,
 since ( X  Y ) and ( X  Y ) form a partition.
y
xz
x y
X z
x
X Y
(a ) P( X  z, X  Y )
y
y
x y
X Y
( z, z )
yz
x
x
Y z
(b) P(Y  z, X  Y )
(c )
Example - continued
From the rightmost figure,
FZ ( z)  P X  z,Y  z   FXY ( z, z).
If X and Y are independent, then
FZ ( z)  FX ( z) FY ( z)
and hence
f Z ( z)  FX ( z) fY ( z)  f X ( z) FY ( z).
Similarly
Thus
Y ,
W  min( X , Y )  
X ,
X Y,
X  Y.
FW ( w)  Pmin( X , Y )  w  PY  w, X  Y    X  w, X  Y .
Example - continued
y
y
x y
xw
y
x y
( w, w)
yw
(a)
x
x
x
(b)
(c)
FW ( w)  1  P W  w   1  P  X  w, Y  w 
 FX ( w)  FY ( w)  FXY ( w, w) ,
Example
X and Y are independent exponential r.vs with common parameter .
Determine fW (w) forW  min( X , Y ).
Solution
FW ( w)  FX ( w)  FY ( w)  FX ( w) FY ( w)
 We have
 and hence
fW ( w)  f X ( w)  fY ( w)  f X ( w) FY ( w)  FX ( w) fY ( w).
 But f X ( w)  fY ( w)  e
 w
, and FX ( w)  FY ( w)  1  e w , so that
fW ( w)  2ew  2(1  e  w )e  w  2e 2 wU ( w).
 Thus min ( X, Y ) is also exponential with parameter 2.
Example
X and Y are independent exponential r.vs with common parameter .
Define Z  min( X , Y ) / max( X ,Y ). Determine f Z (z ).
Solution
We solve it by partitioning the whole space.
X /Y ,
Z 
Y / X ,
X Y,
X  Y.
FZ ( z )  P  Z  z   P  X / Y  z, X  Y   P Y / X  z , X  Y 
 P  X  Yz, X  Y   P Y  Xz, X  Y  .
Since X and Y are both positive random variables in this case, we have 0  z  1.
Example - continued
y
FZ ( z )  

0

yz
x 0
f XY ( x, y )dxdy  


0
0

0

xz
f XY ( x, y )dydx.
y 0
x  yz
f Z ( z )   y f XY ( yz , y )dy   x f XY ( x, xz)dx
x
(a)

  y f XY ( yz , y )  f XY ( y, yz )  dy
0


y

x y
  y2 e  ( yz  y )  e  ( y  yz ) dy
0
 22 

0
2
  (1 z ) y
ye
dy 
(1  z ) 2
 2
, 0  z  1,

  (1  z ) 2
 0,
otherwise .


0
y  xz
ue u dy
x
(b)
2
x y
f Z (z )
1
z
Example – Discrete Case
 Let X and Y be independent Poisson random variables with
parameters  and  respectively.
1
2
 Let Z  X  Y . Determine the p.m.f of Z.
Solution
 Z takes integer values
 For any n  0, 1, 2, , X  Y  n gives only a finite number of
options for X and Y.
 The event { X  Y  n} is the union of (n + 1) mutually exclusive
events Ak given by
Ak   X  k , Y  n  k ,
Example – continued
As a result
 n

P( Z  n )  P( X  Y  n )  P   X  k , Y  n  k 
 k 0

n
  P( X  k , Y  n  k ) .
k 0
If X and Y are also independent, then
P  X  k , Y  n  k   P( X  k ) P(Y  n  k )
and hence
n
P( Z  n )   P( X  k , Y  n  k )
k 0
n
 e
 1
k 0
e
( 1  2 )
1k
k!
e
 2
e ( 1 2 )

(n  k )!
n!
n2k
( 1  2 ) n
,
n!
n
n!
1k n2k

k 0 k! ( n  k )!
n  0, 1, 2, , .
Example – conclusion
 Thus Z represents a Poisson random variable with parameter
1  2 .
 Sum of independent Poisson random variables is also a Poisson random
variable whose parameter is the sum of the parameters of the original
random variables.
 Such a procedure for determining the p.m.f of functions of discrete
random variables is somewhat tedious.
 As we shall see, the joint characteristic function can be used in this
context to solve problems of this type in an easier fashion.