3. Several Random Variables
3.1
Two Random Variables
3.2
Conditional Probability--Revisited
3.3
Statistical Independence
3.4
Correlation between Random Variables
Standardized (or zero mean normalized) random variables
3.5
Density Function of the Sum of Two Random Variables
3.6
Probability Density Function of a Function of Two Random Variables
3.7
The Characteristic Function
Concepts

Two Dimensional Random Variables

Probability in Two Dimensions, Conditional Probability--Revisited

Statistical Independence

Two Dimensional Statistics, Correlation between Random Variables

Density Function of the Linear Combination of Two Random Variables

Multi-input Electrical Circuits

Simulating Convolution Integrals
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Spring 2015
1 of 30
ECE 3800
3. Several Random Variables
Sum and Difference of random variables
Z  X Y
Z  a  X  b Y  c
Z  a  cos2  f  t    N
Applications:



Repeated trials
Signal plus noise
Measurements with noise
Products of Random Variables
Z  X Y
Z  A  cos2  f  t   
Applications:


Two- and three-dimension measurements
Coordinate change
Product and Sum of Random Variables
Z  A  cos2  f  t    N
Applications:





Signal detection and estimation
Coordinate rotation
Aircraft flight dynamics
Two- and three-dimension measurements, movements, navigation, etc.
Plant modeling, leading to multivariable state control.
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Spring 2015
2 of 30
ECE 3800
3-1. Joint Probability Distribution Function (PDF)
Probability Distribution Function:The probability of the event that the observed random variable
X is less than or equal to the allowed value x and that the observed random variable Y is less than
or equal to the allowed value y.
F  x, y   Pr  X  x, Y  y 
The defined function can be discrete or continuous along the x- and y-axis. Constraints on the
probability distribution function are:
for    x   and    y  
1. 0  F  x, y   1,
2. F  , y   F  x,   F  ,   0
3. F ,    1
4. F  x, y  is non-decreasing as either x or y increases
5. F  x,    FX  x  and F , y   FY  y 
Joint Probability Density Function (pdf)
The derivative of the probability distribution function is the density function
 2 FX x 
f  x, y  
xy
Properties of the pdf include
for    x   and    y  
1. f  x, y   0,
 
2.
  f x, y   dx  dy  1
 
Note: the “volume” of the 2-D density function is one.
y
3. F  x, y  
4.
f X x  
x
  f u, v  du  dv
 

 f x, y   dy
and f Y  y  
 f x, y   dx


5. Pr  x1  X  x2 , y1  Y  y 2  

y 2 x2
  f x, y   dx  dy
y1 x1
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Spring 2015
3 of 30
ECE 3800
Expected Values
 
  g x, y   f x, y   dx  dy
E g  X , Y  
  
All expected values may be computed using the Joint pdf. There are some “new” relationships.
Correlation and Covariance between Random Variables
The definition of correlation was given as
 
  x  y  f x, y   dx  dy
EX  Y  
 
But most of the time, we are not interested in products of mean values (observed when X and Y
are independent) but what results when they are removed prior to the computation. Developing
values where the random variable means have been extracted, is defined as computing the
covariance
E  X  E X   Y  E Y  
 
  x   X    y  Y   f x, y   dx  dy
 
This gives rise to another factor, when the random variable means and variances are used to
normalize the factors or correlation/covariance computation. For example, the following
definition – correlation coefficient based on the normalized covariance
 X   X
  E 
  X
  Y  Y
  
  Y
 

 x  X
 


 X
 

  y  Y
  
  Y

  f x, y   dx  dy

When random variables are modified this way, they are called the standardized variables and
have zero mean and a unit variance.
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Spring 2015
4 of 30
ECE 3800
An alternate expression for the correlation coefficient is derived by performing the multiplication
 X   X   Y   Y
  
  X    Y
  E 

 X  Y   X  Y   Y  X   X   Y 

  E 

 X  Y



 
 x  y   X  y  Y  x   X  Y
 X  Y
  
  

  f  x, y   dx  dy

 
 














x
y
f
x
,
y
dx
dy

y
f
x
,
y
dx
dy
 

X

1
  
  



 
 

 X  Y 
  Y    x  f x, y   dx  dy   X   Y    f  x, y   dx  dy 


  
  



 

1




x
y
f
x
,
y
dx
dy

y
f
y
dy

x  f  x   dx   X   Y 










 
X
Y


 X   Y  



 

1

   x  y  f  x, y   dx  dy   X   Y   Y   X   X   Y 
 X   Y  


E x  y    X   Y
 X  Y
An alternate method to “skip the integrals” …
 X   X   Y   Y
  

X

  Y

  E 
 X  Y   X  Y   Y  X   X   Y 

  E 

 X  Y



The expected value is a linear operator … constants remain constants and sums are sums …

E  X  Y    X  E Y    Y  E  X    X   Y
 X  Y

E  X  Y    X   Y   Y   X   X   Y
 X  Y

E  X  Y    X   Y
 X  Y
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Spring 2015
5 of 30
ECE 3800
For standardized random variables, the correlation coefficient can be solved for as the correlation
value.

E x  y    X   Y E x  y   0  0
 E x  y 

 X  Y
1  1Y
For either X or Y a zero mean variable,

E x  y   0 E  X  Y 

 X  Y
 X  Y
For independent random variables …

E X  Y    X   Y  X   Y   X   Y

0
 X  Y
 X  Y
Standardized variables and have zero mean and a unit variance.
This is similar to using the normal density/distribution for a Gaussian. The standardized or
normalized R.V. must have a zero mean and unit variance (normalized).
 X  X
  
 X

 Y  Y
 and   

 Y



Note that now
Y  Y 
 X  X 
E    E 

  0 and E    E 
 Y 
 X 
and
 X   X
E      E 
  X
  Y  Y
  
  Y
E     

1
 
E  X   X   Y   Y 
  X   Y
E X  Y    X   Y

 X  Y
Remember: this is generalized for all R.V, not just Gaussian/Normal R.V.
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Spring 2015
6 of 30
ECE 3800
There are also computations based on the sum and difference of these random variables that can
be computed.
E     E   E   0  0  0

 
E      E  2  2       2

2

  
 
E       1  2    1
E       2  1   
E      E  2  2  E      E  2
2
2
2
and the variance is the same value


Var      E      E      2  1   
2
2
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Spring 2015
7 of 30
ECE 3800
Uniform Density Example
The uniform density function in two dimensions can be defined as:
1
1
1
1
1
f X ,Y  x, y  
 1, for   x  and   y 
1 1
2
2
2
2
0
,
else
Determine the density in y
fY  y 
x2
 f x, y   dx
X ,Y
x1
1
2
 
f Y  y    1  dx  x 12  1   1  1
2
2
2
1
1
2
f Y  y   1,
for 
1
1
y
2
22
f X  x   1,
for 
1
1
x
2
2
Similarly
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Spring 2015
8 of 30
ECE 3800
Distribution
FX ,Y  x, y  
FX ,Y  x, y  
y
y x
  f x, y   dx  dy
  
x

1 
1
  1  dx  dy   x  2    y  2 
1 1
2
2
Detailed distribution in the entire x,y-plane

0,

 x  1    y  1 ,
 


2 
2

1

FX ,Y  x, y    x  ,
2


1
 y  ,
2


1,

1
1
and y  
2
2
1
1
1
1
for   x  and   y 
2
2
2
2
for x  
for 
1
1
1
 x  and  y
2
2
2
1
1
1
 x and   y 
2
2
2
1
1
for  x and  y
2
2
for
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Spring 2015
9 of 30
ECE 3800
Correlation
EX  Y  
 
  x  y  f x, y   dx  dy
  
1
EX  Y  
2
1
2
  x  y  dx  dy
1 1
2 2
 x2
EX  Y    y  
2
1
2

1
2
1

2
2
2
1
  dy   y   1   1   dy

2
2


2 1
1 
2
2
1

2
0  y2
E X  Y    
2 2

EX  Y  

 0
2
2
    1   1 
2 
2  2
1 
2
1

2

1
  x2  x1    y 2  y1   0
4
These are independent, as
f  x, y   f X  x   f Y  y 
Statistical Independence
f  x, y   f X  x   f Y  y 
Exercise 3-1.2
f X ,Y  x, y   6  exp 2  x  3  y ,
0
for 0  x and 0  y
,
else
f X  x   2  exp 2  x ,
for 0  x
f Y  y   3  exp 3  x ,
for 0  y
E  X  Y   E  X   E Y   X  Y 
1 1 1
 
2 3 6
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Spring 2015
10 of 30
ECE 3800
Conditional Probability (with multiple r.v.)
Using the Probability Distribution Function (PDF), define
Pr  X  x | M  F  x, y 

FX  x | Y  y  
Pr M 
FY  y 
Another way.
FX  x | y1  Y  y 2  
F  x, y 2   F  x, y1 
FY  y 2   FY  y1 
Leading to,
f X x | Y  y  
f  x, y 
fY y
fY  y | X  x 
f  x, y 
f X x 
and
From these equations, it can be seen that
f  x, y   f  x | Y  y   f Y  y   f  y | X  x   f X  x 
To derive the multiple variable Bayes Theorem, use
f  x, y   f  x | Y  y   f Y  y   f  y | X  x   f X  x 
resulting in
f x | Y  y  
f  y | X  x   f X x 
fY  y 
f  y | X  x 
f x | Y  y   fY  y 
f X x 
or
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Spring 2015
11 of 30
ECE 3800
Examples
Exercise 3-2.1
Two R.V., X and Y, have a joint pdf of the form
f X ,Y  x, y   k   x  2  y ,
for 0  x  1 and 0  y  1
Find everything ….
k Coefficient
 
1

  
1 1
f  x, y   dx  dy    k   x  2  y   dx  dy
0 0
1
1
x

1

1  k     2  y  x   dy  k     2  y   dy
2
2

0
0
0
1
2
1
1
1
y2 
1
3
1  k    y  2    k    2    k   
2 0
2
2
2
2
k
2
3
Marginal density functions of X and Y
1

2
2  x2
f Y  y      x  2  y   dx     2  y  x 
3
3  2
0
0
2 1
 1 4
fY y      2  y     y
3 2
 3 3
1
1
y2 
2
2 
f X  x      x  2  y   dy    x  y  2  
3
3 
2 0
0
2
f X  x     x  1
3
1
Joint Distribution of X and Y
y x
2
FX ,Y  x, y      u  2  v   du  dv
3
0 0

2  x2
FX ,Y  x, y     
 2  v  x   dv
3  2

0
y
FX ,Y  x, y  

2  x2
   y  x  y 2 ,
3  2

for 0  x  1 and 0  y  1
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Spring 2015
12 of 30
ECE 3800
The conditional Probability that X is greater than ½ given that Y=½.
f X ,Y  x, y 
f X |Y  x | Y  y  
fY y 
2
 x  2  y 
2  x  2  y 
f X |Y  x | Y  y   3

1 4
1 4 y
 y
3 3
2  x  2  y 
1 1  4  y  dx 
1
1
FX ,Y   X | Y   
2
2
1
2
1
1
FX ,Y   X | Y   
2
2
1
 3  x  1  dx
1
2
2
1

2
2  x2
1 3  x  1  dx  3   2  x  1
2
1
2
1  2  1   1 1   2  3 5  2  7  7
1
FX ,Y   X | Y       1               
2  3  2   8 2   3  2 8  3  8  12
2
The conditional Probability that Y is less than or equal to ½ given that X=½.
f X ,Y  x, y 
fY|X  y | X  x 
f X x 
2
 x  2  y 
x  2 y
fY|X  y | X  x  3

2
x 1
  x  1
3
1
1

FY | X  Y  | X   
2
2

x  2 y
1 x  1  dx 
1
2
1
1

FY | X  Y  | X   
2
2

1
1
2

0
1
 2 y
2
 dx 
1
1
2
1
2
 3  1  4  y   dx
1
0
1
2
y2 
1 
1
0 3  1  4  y   dx  3   y  4  2 
0
2
1
1 1 1
1 1
1

FY | X  Y  | X       4     1 
2
2 3 2
8 3
3

Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Spring 2015
13 of 30
ECE 3800
Exercise 3-3.1
f X ,Y  x, y   k  exp x  y  1,
for 0  x   and 1  y  
There is no a, unless the problem was supposed to be stated as …
f X ,Y  x, y   k  exp x  y  a ,
for 0  x   and 1  y  
but then k and a are not necessarily computed separately as 1 and 1.!
Overall, the correct pdf is
f X ,Y  x, y   exp x  y  1,
for 0  x   and 1  y  
f X ,Y  x, y   exp x   exp y  1,
for 0  x   and 1  y  
f X  x   exp  x ,
for 0  x  
f Y  y   exp  y  1,
for 1  y  
Correlation
EX  Y  
 
  x  y  f x, y   dx  dy
  

E  X  Y     x  y  exp x  y  1  dx  dy
1 0
expax 
 ax  1
a2
 x  expax   dx 

 exp0 

E  X  Y   exp 1   y  exp y   
 0  1  dx
2
  1

1

E  X  Y   exp 1   y  exp y   1  dx
1
 exp 1

E X Y   exp 1  
  1  1  exp 1  exp 1  2  2
2
  1

E X  Y   E X   1 E Y   1  1  1  1  1  2
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Spring 2015
14 of 30
ECE 3800
Exercise 3-3.2
Assume X and Y independent
f X  x   0.5  exp x  1 ,
for    x  
f Y  y   0.5  exp y  1 ,
for    y  
Find PrX Y  0 ? That is, the product of the random variables is positive.
PrX  Y  0  PrX  0  PrY  0  PrX  0  PrY  0
PrX  Y  0  1  Fx 0   1  FY 0  Fx 0  FY 0
Find the distribution for X and Y based on the ranges defined for the absolute value
FX  x  
x
 0.5  exp x  1   dx

For for    x  1 and for 1  x  
for    x  1
FX  x  
for 1  x  
x
x
 0.5  expx  1  dx
FX x   0.5   0.5  exp1  x   dx

1
exp1  x 
1
1
FX  x   0.5  0.5  1  exp1  x 
exp x  1
1

FX x   0.5  exp x  1
F X  x   0 .5 
x
F X  x   0 .5  0 .5 
x
FX 0  FY 0  0.5  exp 1  0.1839
PrX  0  PrY  0  1  Fx 0  1  FY 0  0.6660
PrX  0  PrY  0  Fx 0  FY 0  0.0338
PrX  Y  0  1  Fx 0  1  FY 0   Fx 0  FY 0  0.6660  0.0338  0.6998
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Spring 2015
15 of 30
ECE 3800
Exercise 3-4.1 – Again the text is wrong
Two R.V. have means of 1 and variances of 1 and 4, respectively. Their correlation coefficient is
0.5.
Find everything …..
EX   1
EY   1
 X2  1
 Y2  4
 
 
E X 2   X2  E X   1  1  2
E Y 2   Y2  E Y   4  1  5
2
2
EX  Y    X   Y 1

 X  Y
2
EX  Y      X   Y   X  Y

E X Y  

 
1
1  2  1 1  2
2
  
 
E  X  Y   E X 2  2  X  Y  Y 2  E X 2  2  E X  Y   E Y 2
2


 E  X  Y    E X  Y 
E  X  Y   2  2  2  5  11
2
 X2 Y
2
2

2
X Y
for EX  Y   EX   EY   2
 11  2   7
2
Alternately,
 X2 Y   X2   Y2  2     X   Y
1
2
 X2 Y  1  4  2   1  2  7
Differences

  
 
E  X  Y   E X 2  2  EX  Y   E Y 2  2  4  5  3
2
 X2 Y  3  0 2  2 for E X  Y   E X   EY   0
or
 X2 Y   X2   Y2  2     X   Y  1  4  2  3
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Spring 2015
16 of 30
ECE 3800
Density Function of the Sum of Two Random Variables
The summing (differencing) of random variables creates a new random variable.
Expect that the resulting probability distribution and density function are different (except when
Gaussians are involved).
Z  X Y
We want to find
Pr Z  z   FZ z   Pr  X  Y  z 
For the experiment, let
Z  X Y
Figure in two dimensions with the line z=x+y ….
We know the joint density function, then any probability can be computed as
y
F  x, y  
x
  f u, v  du  dv
 
We know that x  z  y , which says that    x  z  y and we can allow    y   as the
limitation in x is incorporated in the first inequality. Therefore,
FZ  z   FXY  z  y,   
 z y
  f u, y   du  dy
 
If we assume that X and Y are statistically independent, f  x, y   f X  x   f Y  y  :

z y
FZ  z    f Y  y     f X  x  dx  dy

 


Taking the derivative to find the density function results in
d FZ  z  d FZ  z  dx 
f Z z  


 f Y  y    f X  z  y   1  dy
dz
dx
dz 
f Z z  

 f  y   f z  y   dy
Y
X

Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Spring 2015
17 of 30
ECE 3800
Interpreting the math: this is the convolution of the two density functions for X and Y!
The sum of random variables density function may be formed/derived as either of the two
equations (equivalent convolution results)
f Z z  

 f X x  fY z  x  dx


f Z  z    f Y  y   f X  z  y   dy

Thus, there are two equivalent forms for the density function.
f Z z  




 f X x   fY z  x  dx   fY  y   f X z  y   dy
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Spring 2015
18 of 30
ECE 3800
Example #1: Z  X  Y
Uniform densities in X and Y
f X  x   1, for 
1
1
1
1
and f Y  y   1, for   y 
x
2
2
2
2
Forming the density in z
f Z z  




 f X x  fY z  x  dx   fY  y   f X z  y   dy
f Z z  
0.5
 f X z  y   dy
 0.5
For  1  z  0 ,
f Z z  
z  0.5
z  0.5
 0.5
 0.5
0.5
0.5

f X  z  y   dy 

z  0.5
1  dy  y  0.5  z  0.5   0.5)   1  z
For 0  z  1 ,
f Z z  
 f X z  y   dy  1 dy  y z  0.5  0.5  z  0.5)  1  z
0.5
z  0 .5
z  0 .5
The convolution of two rectangles makes a triangle.
The bounds move from +/- 0.5 to +/-1.
Figure in two dimensions with the line z=x+y … and convolution figure.
Application: This describes the error in the least significant bit of numbers that are rounded to
integers! Initially a uniform distribution between +/- 0.5.
After summing numbers that are rounded off, the density function of the error changes … the
maximum possible error increases and the shape “distributes”.
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Spring 2015
19 of 30
ECE 3800
Example #2: Z  X  Y
Uniform densities in X and exponential density in Y
f X  x   1, for 0  x  1 and f Y  y   exp y  for 0  y
Forming the density in z
f Z z  


 f X x  fY z  x  dx   fY  y   f X z  y   dy


f Z z  
1
 fY z  x  dx
0
Draw the convolution figure to define regions of integration. I. The zero region – no overlap
(z<0). II. The beginning to overlap region (0<z<1). and III. The overlapping region that extends
to infinity(1<z).
For 0  z  1 ,
f Z z  
z
z
 fY z  x  dx   exp z  x  dx  exp z  x 0  exp z  z   exp z 
z
0
0
f Z  z   1  exp z 
For 1  z   ,
f Z z  
1
1
 fY z  x  dx   exp z  x  dx  exp z  x 0  exp z  1  exp z 
1
0
0
f Z  z   exp
 z   e 1
1

Therefore,
1  exp z 
f Z z   
exp z   e  1
for 0  z  1
for 1  z
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Spring 2015
20 of 30
ECE 3800
Example #3: Z  X  Y
Independent Gaussian densities in X and Y
f X x  
   x   2 
X
 exp
 and f Y  y  
2
2   X
 2   X

1
   y   2 
Y
 exp 

2
2   Y
 2   Y

1
Forming the density in z
f Z z  





 f X x  fY z  x  dx   fY  y   f X z  y   dy
   x   2 
  z  x   2 
1
X
Y
f Z z  
 exp 
 exp 

 dx
2
2
2   X
2  Y
 2  X
 2   Y






f Z z  
1
   x   X 2   z  x   Y 2 

  exp 

  dx
2
2   Y2
2
2   X   Y   2   X

1

1
To solve, complete the square in x and remember that the Gaussian pdf integrates to 1.0. The
remaining elements of the equation (terms not in x) transfer outside the integral.
Once completed, the result should be:
f Z z  
   z   X   Y 2 
 exp 

2
2
2   X2   Y2 
 2   X   Y  
1
This is a Gaussian that has a variance that is the square root of the sum of the variances of the
individual Gaussians (larger – spreading out, but still Gaussian) and the mean is the sum of the
individual means.
This result (and the extension to multiple random variables) is a major reason why Gaussian
Noise models are so widely used! This is also part of an empirical proof of the central limit
theorem (Section 2-5).
The MATALB example demonstrates convolved exponential pdfs becoming Gaussian-like.
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Spring 2015
21 of 30
ECE 3800
%%
% Gauseeian Convolution
% p. 140 Cooper and McGillam
%
clear
close all;
x=0:0.1:5;
f=exp(-x);
g=f;
figure;
plot(x,f);
hold on;
for ii = 1:10
g=0.1*conv(f,g);
y=0.1*(0:length(g)-1);
plot(y,g);
end
xlabel('y');
ylabel('g(y)')
axis([0 20 0 1]);
hold off;
figure
plot(y,g);
xlabel('y');
ylabel('g(y)')
1
0.25
0.9
0.8
0.2
0.7
0.15
g(y)
g(y)
0.6
0.5
0.4
0.1
0.3
0.2
0.05
0.1
0
0
2
4
6
8
10
y
12
14
16
18
20
0
0
10
20
30
y
40
50
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Spring 2015
22 of 30
ECE 3800
60
Probability Density Function of a Function of Two Random Variables
Functions of random variables create new random variable.
As before, expect that the resulting probability distribution and density function are different.
Assume that there is a function that combines two random variables and that the functions
inverse exists. Define the function as
Z  1  X , Y  and W   2  X , Y 
and the inverse as
X   1 Z ,W  and Y   2 Z ,W 
The original pdf is f x, y  with the derived pdf in the transform space of g z, w .
Then it can be proven that:
Pr  z1  Z  z 2 , w1  W  w2   Pr  x1  X  x2 , y1  Y  y 2 
or equivalently
w2 z 2
y2 x2
w1 z1
y1 x1
  g z, w  dz  dw    f x, y   dx  dy
Empirically, since the density function must integrate to one for infinite bounds, the
“transformed” portion of one density must have the same “volume” as the original density
function.
Using an advanced calculus theorem to perform a transformation of coordinates.
x
g z , w  f  1  z, w, 2 z, w  z
y
z
w2 z 2
w2 z 2
w1 z1
w1 z1
x
w  f   z , w,  z, w  J
1
2
y
w
  g z, w  dz  dw    f  1 z, w, 2 z, w  J  dz  dw
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Spring 2015
23 of 30
ECE 3800
Example #4: Z  X  Y and let W  X , an arbitrary selection
Then,
z  x  y and w  x describes the forward transformation, and
y
z
and x  w describes the inverse transformation.
w
The Jocobian is
x
J  z
y
z
x
0
w  1
y
w
w
1
z  1  1   1

w
w
w2
Therefore,
 z  1
g  z , w  f  w,  
 w w
and integrating for all w to find z,
g z  


g  z , w  dw 




1
 z
 f  w,   dw
w  w
As may be expected, the integral may need to be solved using numerical methods.
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Spring 2015
24 of 30
ECE 3800
Example #5: Compute the area where the nominal 10 cm square IC die size varies by
independent uniform random variables allowing a +/- 0.5% tolerance. Then:
f XY  x, y  
1 1

0 .1 0 .1
for 9.95  x, y  10.05
Derive the area density function ( Z  X  Y ).
Then,
z  x  y and w  x describes the forward transformation, and
y
z
and x  w describes the inverse transformation, and
w
 z  1
g  z , w  f  w,  
 w w
Therefore,
g z  



1
 z
 f  w,   dw 
w  w

 z  10 
1
 w  10 
 w
  dw
 100  rect 
  rect 

w
0
.
1
0
.
1








 z  10 
1
 w  10 
 w
  dw
 rect 
g  z   100 
  rect 
w
 0.1 
 0 .1 




for 9.95  w  10.05 and 9.95 
z
 10.05
w
or 9.95  w  z  10.05  w
z


9.95
 z 
1
100 

 dw  100  ln

2
w
9
.
95



9.95
g z   
10.05

 z 
1

100 
 dw  100  ln
w

 10.05 2 
z

10.05


for 9.95 2  z  9.95  10.05

for 9.95  10.05  z  10.05 2
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Spring 2015
25 of 30
ECE 3800
Integrating to find the distribution
z

 z 
100 
  dz
ln
2

9
.
95



9.95 2
G z   
z

 z 
0.5  100 
  dz
ln

 10.05 2 
9.9510.05



for 9.95 2  z  9.95  10.05
for 9.95  10.05  z  10.05 2



 z 
  z  9.95 2 
for 9.95 2  z  9.95  10.05
100   z  ln
2
 9.95 






 z 

  z
G z   
 z  ln

 10.05 2 



for 9.95  10.05  z  10.05 2

0.5  100  
 9.95  10.05  1  ln 9.95  10.05  


 



 10.05 2  



To determine the area within 0.5% bounds,
Pr 100  0.995  z  100  1.005
Pr 100  0.995  z  100  1.005  G100.5  G 99.5


 100.5 
  100.5
 z  ln

 10.05 2 


Pr 100  0.995  z  100  1.005  100  

 9.95  10.05  1  ln 9.95  10.05  

 


 10.05 2  




 99.5 
  99.5  9.95 2 
 100   z  ln
 9.95 2 


Pr 100  0.995  z  100  1.005  0.8751  0.1251  0.7500
G(z) PDF
101.00
100.90
100.80
100.70
100.60
100.50
100.40
100.30
100.20
z
100.10
100.00
99.90
99.80
99.70
99.60
99.50
99.40
99.30
99.20
99.10
99.00
Probability
1.2
1
0.8
0.6
0.4
0.2
0
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Spring 2015
26 of 30
ECE 3800
The Characteristic Function
Whenever you see convolution and transformation, expect to see equations related to either the
Fourier Transform or Laplace Transform!
The characteristic function of a random variable X is defined as:
 u   Eexp j  u  X 
or

 u  
 f x  exp j  u  x  dx

The inverse of the characteristic function is then defined as:
1
f x  
2

  u   exp j  u  x   du

Application: The density function of two summed random variables, where
 X u  

 f X x  exp j  u  x  dx
and Y u  

 fY  y   exp j  u  y   dy


Since we already know
f Z z  




 f X x   fY z  x   dx   fY  y   f X z  y   dy
We can compute
 Z u    X u   Y u 
and solve for the density function as
1
f Z z  
2

  X u   Y u   exp j  u  z   du

Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Spring 2015
27 of 30
ECE 3800
Example #6 Repeat example #2 using the characteristic functions
Uniform densities in X and exponential density in Y
f X  x   1, for 0  x  1 and f Y  y   exp y  for 0  y
1
exp j  u  x 
exp j  u   1

 X u   1  exp j  u  x   dx 
j u
j u
0
1

0


exp j  u  y  y 
1
1
Y u   exp y   exp j  u  y   dy 


j  u 1
j  u 1 1 j  u
0

0
Then
 Z u    X u   Y u 
 exp j  u   1  1 
exp j  u   1




j u

 1  j  u  j  u  1  j  u 
 Z u   
Solving for the density function
1
f Z z  
2
f Z z  
1
2
1

2


  X u   Y u   exp j  u  z   du

for 0  z

exp j  u   1
 exp j  u  z   du
j  u  1  j  u 

exp j  u   z  1
1
 du 
2
j  u  1  j  u 






exp j  u  z 
 du
j  u  1  j  u 
Find it for extra credit … but it should result in
1  exp z 
f Z z   
exp z   e  1
for 0  z  1
for 1  z
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Spring 2015
28 of 30
ECE 3800
Computing the first moment using the characteristic function:
Differentiating the characteristic function by u:
 u  
d  u 

du

 f x   exp j  u  x  dx


 f x   j  x  exp j  u  x  dx

Evaluating the function at u=0
d  u 

du u  0




 f x    j  x   exp0  dx  j   x  f x   dx  j  EX 
Computing other moments is performed similarly, where:
d n  u 
du
du

u 0
 f x    j  x 
n

 exp j  u  x   dx


d n  u 
n
n

  j  x
n
 f  x   dx  j 
n


x
n
 
 f x   dx  j n  E X n

Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Spring 2015
29 of 30
ECE 3800
The Joint Characteristic Function
Whenever you see convolution and transformation, expect to see equations related to either the
Fourier Transform or Laplace Transform!
The characteristic function of a random variable X and Y is defined as:
 u, v   Eexp j  u  X  v  Y 
or
 u, 
 
  f x, y  exp j  u  x  v  y  dx  dy
 
The inverse of the characteristic function is then defined as:
f  x, y  
 
2 2  
1
 XY u, v   exp j  u  x  v  y   du  dv
Note that
 2  u, v 
E X  Y   j 2 
uv

u v 0
 2  XY u, v 
uv
u v 0
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Spring 2015
30 of 30
ECE 3800