3. Several Random Variables
3.1
Two Random Variables
3.2
Conditional Probability--Revisited
3.3
Statistical Independence
3.4
Correlation between Random Variables
Standardized (or zero mean normalized) random variables
3.5
Density Function of the Sum of Two Random Variables
3.6
Probability Density Function of a Function of Two Random Variables
3.7
The Characteristic Function
Concepts

Two Dimensional Random Variables

Probability in Two Dimensions, Conditional Probability--Revisited

Statistical Independence

Two Dimensional Statistics, Correlation between Random Variables

Density Function of the Linear Combination of Two Random Variables

Multi-input Electrical Circuits

Simulating Convolution Integrals
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Spring 2015
1 of 18
ECE 3800
Density Function of the Sum of Two Random Variables
The summing (differencing) of random variables creates a new random variable.
Expect that the resulting probability distribution and density function are different (except when
Gaussians are involved).
Z  X Y
We want to find
Pr Z  z   FZ z   Pr  X  Y  z 
For the experiment, let
z  X Y
We know the joint density function, then
y
x
  f u, v  du  dv
F  x, y  
 
for x  z  y ,
FZ  z   FXY  z  y,   
 z y
  f u, y   du  dy
 
If we assume that X and Y are statistically independent, f  x, y   f X  x   f Y  y  :
FZ  z  
z y


fY  y  

 f X x dx  dy

Taking the derivative to find the density function using
d FZ  z 

f Z z  
dz

 fY  y   f X z  y   dy

Interpreting the math: this is the convolution of the two density functions!
Alternately, the density function may also be formed as
f Z z  

 f X x  fY z  x  dx

Thus, there are two equivalent forms for the density function.
f Z z  




 f X x  fY z  x  dx   fY  y   f X z  y   dy
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Spring 2015
2 of 18
ECE 3800
HW 3-2.4
A random signal X can be observed only in the presence of independent additive
noise N. The observed quantity is Y=X+N. The joint probability density function of X and Y is
 

f X ,Y x, y   K  exp  x 2  4  x  y  y 2 ,
a) Find a general expression for the best estimate of X as a function of the observation Y=y.
b) If the observed value of Y is y=3, find the best estimate of X.
f  x, y  f  y | x   f X  x 

fY  y
fY  y
f x | y  
Since we have the joint density function, we need the marginal density of Y.

fY  y 
 f x, y   dx,
X ,Y

fY  y 

 K  exp x
2

 4  x  y  y 2  dx




 

f Y  y   K  exp 3  y 2   exp  x 2  4  x  y  4  y 2  dx






f Y  y   K  exp 3  y 2   exp   x  2  y   dx
2

Force the integral to be Gaussian, for 2   2  1 and   2  y


   x  2  y 2
2  
1

 exp
1
2  2
  2  


f Y  y   K  exp 3  y 2 


f Y  y   K  exp 3  y 2 
2
2

  dx


 K    exp3  y 2 
To find K, the integral will become infinite. Therefore, this is not a valid density function, and
the problem is improperly posed!
Nuts …
Ways to fix the problem …
 

f X ,Y  x, y   K  exp  x 2  a  x  y  y 2 ,
 
for  2  a  2
f X ,Y  x, y   K  exp  x 2  4  x  y 2

Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Spring 2015
3 of 18
ECE 3800
Let’s work with
 

f X ,Y  x, y   K  exp  x 2  a  x  y  y 2 ,
for a  1
Then,
fY  y 

 K  exp x
2
 x  y  y 2   dx

Since we have the joint density function, we need the marginal density of Y.
fY  y 

 f x, y   dx,
X ,Y

fY  y 

 K  exp x
2
 x  y  y 2   dx


 
1

 3

f Y  y   K  exp   y 2    exp   x 2  x  y   y 2    dx
4

 4
   
2

 
1  
 3 2

f Y  y   K  exp   y    exp   x   y   dx
2  
 4
   
1
Force the integral to be Gaussian, for 2   2  1 and     y
2

   x   2
1
 3 2  2  

 exp
f Y  y   K  exp   y  
2
1

2


 4


 2 

  dx


 3
 2
 3

 K    exp   y 2 
f Y  y   K  exp   y 2  
2
 4

 4

Notice that this is now a Gaussian density function. To determine the value of K, integrate.

 y2 
  dy
1   f Y  y   dy  K     exp 

3


4

Force the integral to be Gaussian, for 2   2  3
1   f Y  y   dy  K   
 0
4 and

  y2 
2  
1
  dx

 exp
2 
1
 2  
  2  
1   f Y  y   dy  K    2 
K
3
3
 K  
8
2
2
 3
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Spring 2015
4 of 18
ECE 3800
Therefore,
fY  y 
2
 3

 3

   exp   y 2  
 exp   y 2 
 3
3
 4

 4

2
Then,
2
 
 exp  x 2  x  y  y 2

f  x, y    3
f x | y  

2
fY  y

 3
 exp   y 2 
3

 4
 
f  x, y 
1
1


 exp   x 2  x  y   y 2  
f x | y  
fY  y
4


 
2
 
f  x, y 
y  
1

f x | y  

 exp   x  
 
fY  y
2  


b) If the observed value of Y is y=3, find the best estimate of X.
E X | y  3 

 x  f x | y  3  dx

2
 
3  

E X | y  3   x 
 exp   x    dx
 
2  



2
2


 
 
1
3
3  
1
3  

3


    exp   x    dx
E X | y  3 
  x    exp   x    dx 
 
 
2
2  
2  
 
  2 


3
Force the second integral to be Gaussian, for 2   2  1 and   
2

1

2
 1
 

 3
3

E X | y  3 
   exp   x     
 
2  
  2
2 


1
 3
E X | y  3  0  
2 

   x   2
 2
1
 

 exp
2
2  2  

 2 

  dx


 2
3
 

2
2

Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Spring 2015
5 of 18
ECE 3800
Probability Density Function of a Function of Two Random Variables
Functions of random variables create new random variable.
As before, expect that the resulting probability distribution and density function are different.
Assume that there is a function that combines two random variables and that the functions
inverse exists.
Z  1  X , Y  and W   2  X , Y 
and the inverse
X   1 Z ,W  and Y   2 Z ,W 
The original pdf is f x, y  with the derived pdf in the transform space of g z, w .
Then it can be proven that:
Pr  z1  Z  z 2 , w1  W  w2   Pr  x1  X  x2 , y1  Y  y 2 
or equivalently
w2 z 2
y2 x2
w1 z1
y1 x1
  g z, w  dz  dw    f x, y   dx  dy
Empirically, since the density function must integrate to one for infinite bounds, the
“transformed” portion of one density must have the same “volume” as the original density
function.
Using an advanced calculus theorem to perform a transformation of coordinates.
x
g z , w  f  1  z, w, 2 z, w  z
y
z
w2 z 2
w2 z 2
w1 z1
w1 z1
x
w  f   z , w,  z, w  J
1
2
y
w
  g z, w  dz  dw    f  1 z, w, 2 z, w  J  dz  dw
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Spring 2015
6 of 18
ECE 3800
Example #4: Z  X  Y and let W  X , an arbitrary selection
Then,
z  x  y and w  x describes the forward transformation, and
y
z
and x  w describes the inverse transformation.
w
The determinant of the Jocobian is
x
J  z
y
z
x
0
w  1
y
w
w
1
z  1  1   1

w
w
w2
Therefore,
x
g z , w  f  1  z, w, 2 z , w  z
y
z
x
w  f   z , w,  z , w  J
1
2
y
w
 z  1
g  z , w  f  w,  
 w w

Then, integrating for all w to find z (remember that Fz , w  z,    Fz  z  and f z x   dFZ z
g z  




 g z, w  dw  
dz
1
 z
 f  w,   dw
w  w
As may be expected, the integral may need to be solved using numerical methods.
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Spring 2015
7 of 18
ECE 3800
,
Example #5: Compute the area where the nominal 10 cm square IC die size varies by
independent uniform random variables allowing a +/- 0.5% tolerance. Then:
f X ,Y  x, y  
1 1

, for 9.95  x, y  10.05
0 .1 0 .1
Derive the area density function ( Z  X  Y ).
Then,
z  x  y and w  x describes the forward transformation, and
z
and x  w describes the inverse transformation, and
y
w
 z  1
g  z , w  f X ,Y  w,  
 w w
Therefore,
 z  10 
1
 w  10 
  dw, for 9.95  w, z  10.05
 w
g  z     100  rect 
  rect 
w
w
 0.1 
 0.1 




g z   100 
leading to
 z  10 
1
 w  10 
  dw, for 9.95  w, z  10.05
 w
rect
rect




 w
 0.1 
w
 0.1 



9.95  w  z  10.05  w and 9.95 2  z  10.05 2
using wmin and wmax to establish the bounds on z. Finally, we have
z


9.95
 z 
1
100 

 dw  100  ln

2
w
9
.
95



9.95
g z   
10.05

 z 
1

100 
 dw  100  ln
2
w

 10.05 
z

10.05


for 9.95 2  z  9.95  10.05

for 9.95  10.05  z  10.05 2
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Spring 2015
8 of 18
ECE 3800
Integrating to find the distribution
z

 z 
100 
  dz
ln
2

9
.
95



9.95 2
G z   
z

 z 
G 9.95  10.05  100 
  dz
ln

 10.05 2 
9.9510.05



for 9.95 2  z  9.95  10.05
for 9.95  10.05  z  10.05 2
and



 z 
  z  9.95 2 
for 9.95 2  z  9.95  10.05
100   z  ln
2
 9.95 






 z 

  z
G z   
 z  ln

 10.05 2 




G 9.95  10.05  100  





9
.
95
10
.
05





 9.95  10.05  1  ln 10.05 2  


 



Pr 100  0.995  z  100  1.005
Pr 100  0.995  z  100  1.005  G100.5  G 99.5
Pr 100  0.995  z  100  1.005  G 9.95  10.05


 100.5 
  100.5
100.5  ln

 10.05 2 


 100  

 9.95  10.05  1  ln 9.95  10.05  

 


 10.05 2  




 99.5 
  99.5  9.95 2 
 100  99.5  ln
 9.95 2 


Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Spring 2015
9 of 18
ECE 3800


 9.95  10.05 
  9.95  10.05  9.95 2 
Pr 100  0.995  z  100  1.005  100  9.95  10.05  ln
 9.95 2 




 100.5 
  100.5

 z  ln
 10.05 2 


 100  

 9.95  10.05  1  ln 9.95  10.05  

 


 10.05 2  




 99.5 
  99.5  9.95 2 
 100  99.5  ln
 9.95 2 




 9.95  10.05 
  9.95  10.05  9.95 2 
Pr 100  0.995  z  100  1.005  100  9.95  10.05  ln
 9.95 2 




 100.5 
  100.5

 z  ln
 10.05 2 


 100  

 9.95  10.05  1  ln 9.95  10.05  

 


 10.05 2  




 99.5 
  99.5  9.95 2 
 100  99.5  ln
 9.95 2 


Pr 100  0.995  z  100  1.005  0.8751  0.1251  0.7500
G(z) PDF
1.2
Probability
1
0.8
0.6
0.4
0.2
z
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Spring 2015
10 of 18
ECE 3800
101.00
100.90
100.80
100.70
100.60
100.50
100.40
100.30
100.20
100.10
100.00
99.90
99.80
99.70
99.60
99.50
99.40
99.30
99.20
99.10
99.00
0
HW 3-4.2
Two statistically independent random variables, X and Y, have variances of
2
2
 x  9 and  y  25 . Two new variables are defined by
U  3 X  4 Y
V  5  X  2 Y
a) Find the variance of U and V.
b) Find the correlation coefficient of U and V.
E U   E 3  X  4  Y   3   x  4   Y
  

E U   9  E X   E 24  X  Y   16  E Y 
E U   9  E X   24      16  E Y 
  9  E X   24      16  E Y  3    4   
 9  E X   24      16  E Y  9    24      16   
  9  E X     16  E Y      9    16  
E U 2  E 3  X  4  Y 
2
2
2
2
2
2
x
2
U
 U2
y
2
2
2
x
y
x
2
y
2
x
2
2
Y
2
2
x
2
U
2
x
2
2
x
2
X
y
y
y
2
Y
 U2  9  9  16  25  81  400  481
Similarly
E V   E 5  X  2  Y   5   x  2   Y
 V2  25   X2  4   Y2
 V2  25  9  4  25  225  100  325
Notice that equation (3-28) would have been easier!
b) Find the correlation coefficient of U and V.
E U  V      U   V  U  V  E 3  X  4  Y   5  X  2  Y 
  481  325  3   X  4  Y   5   X  2  Y   E15  X 2  14  X  Y  8  Y 2 



  481  325  15   X 2  14   X  Y  8  Y 2  15   X2   X 2  14   X  Y  8   Y2  Y 2
  481  325  15   X2  8   Y2  15  9  8  25

135  200
481  325

 65
 0.1644
395.38
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Spring 2015
11 of 18
ECE 3800

Another way to do this is to create another composite random variable.
Z  2  V  U  2  5  X  2  Y   3  X  4  Y   13  X
But we know from equation (3-28)
 Z2  4   V2   U2  2    2   V    U   132   X2
Therefore
132   X2  4   V2   U2

2   V    U 

13 2  9  4  325  481
2  2  481  325

 260
 0.1644
1581.52
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Spring 2015
12 of 18
ECE 3800
Example 3-5.2 An ECE student on main campus attempts to catch the 9am bus every morning
although his arrival time at the bus stop is a random variable that is uniformly distributed
between 8:55 am and 9:05 am. The bus’ departure time from the bus stop is also a random
variable that is uniformly distributed between 9am and 9:10 am.
a) Find the probability density function of the time interval between the student’s arrival at the
bus stop and the bus’ departure time.
b) Find the probability that the student will catch the bus.
c) Find the probability that the student will catch the bus with 3 minutes to spare.
Scaling so that 9am is “zero”
1
 ,
f S s   10
0,
Let
5  s  5
else
1
 ,
and f B b   10
0,
0  b  10
else
Z  BS

f Z z    f S s   f B  z  s   ds

Determine the appropriate regions
f Z z  
z
1
1
1
1
 10  10  ds,
for  5  z  5
5
15

 10  10  ds,
for 5  z  15
z
f Z z  
1
z
 s   5 ,
100
1
15

 s  z ,
100
1
f Z z  
  z  5,
100
1

 15  z ,
100
for  5  z  5
for 5  z  15
for  5  z  5
for 5  z  15
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Spring 2015
13 of 18
ECE 3800
Determine the probability distribution function
z
1
FZ  z  
  5  x   dx,
100 5
for  5  z  5
z
 0 .5 
1
 15  x   dx,
100 5
for 5  z  15
2

1 
z2
 5

FZ  z  
 5 z 
 5   5 
100 
2
2

,


2
2
z
5 
1 
 0 .5 
 15  z 
 15  5  ,
100 
2
2
1  z2
25 
FZ  z  
   5  z  ,
100  2
2 
 0 .5 
for  5  z  5
for 5  z  15
for  5  z  5
1   125
z2 
  
 15  z  ,
100 
2
2 
for 5  z  15
b) Find the probability that the student will catch the bus. To catch the bus Z positive
PrZ  0  1  FZ 0
PrZ  0  1 
25 
1  02
   5  0  
2
100  2
PrZ  0  1 
25
 0.875
200
c) Find the probability that the student will catch the bus with 3 minutes to spare.
PrZ  3  1  FZ 3
PrZ  0  1 
PrZ  0  1 
25 
1  32
   5  3  
2 
100  2
9  30  25 136

 0.68
200
200
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Spring 2015
14 of 18
ECE 3800
The Characteristic Function
Whenever you see convolution and transformation, expect to see …
equations related to either the Fourier Transform or Laplace Transform!
The characteristic function of a random variable X is defined as:
 u   Eexp j  u  X 
or

 u  
 f x  exp j  u  x  dx

The inverse of the characteristic function is then defined as:
1
f x  
2

  u   exp j  u  x   du

Application: The density function of two summed random variables, where
 X u  

 f X x  exp j  u  x  dx
and Y u  

 fY  y   exp j  u  y   dy


Since we already know
f Z z  




 f X x   fY z  x   dx   fY  y   f X z  y   dy
We can compute
 Z u    X u   Y u 
and solve for the density function as
1
f Z z  
2

  X u   Y u   exp j  u  z   du

Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Spring 2015
15 of 18
ECE 3800
Example #6 Repeat example #2 using the characteristic functions
Uniform densities in X and exponential density in Y
f X  x   1, for 0  x  1 and f Y  y   exp y  for 0  y
1
exp j  u  x 
exp j  u   1

 X u   1  exp j  u  x   dx 
j u
j u
0
1

0


exp j  u  y  y 
1
1
Y u   exp y   exp j  u  y   dy 


j  u 1
j  u 1 1 j  u
0

0
Then
 Z u    X u   Y u 
 exp j  u   1  1 
exp j  u   1




j u

 1  j  u  j  u  1  j  u 
 Z u   
Solving for the density function
1
f Z z  
2
f Z z  
1
2
1

2


  X u   Y u   exp j  u  z   du

for 0  z

exp j  u   1
 exp j  u  z   du
j  u  1  j  u 

exp j  u   z  1
1
 du 
2
j  u  1  j  u 






exp j  u  z 
 du
j  u  1  j  u 
For me, this is harder than the other way … due to the “discontinuity” in the function.
Therefore, choose (guess at ) the easiest way to perform the convolution.
Find the derivation for extra credit … but it should result in
1  exp z 
f Z z   
exp z   e  1
for 0  z  1
for 1  z
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Spring 2015
16 of 18
ECE 3800
Computing the first moment using the characteristic function:
Differentiating the characteristic function by u:

 f x   exp j  u  x  dx
 u  
d  u 

du


 f x   j  x  exp j  u  x  dx

Evaluating the function at u=0
d  u 

du u  0




 f x    j  x   exp0  dx  j   x  f x   dx  j  EX 
d  u 
 j  E X 
du u  0
Computing other moments is performed similarly, where:
d n  u 
du
du

u 0
 exp j  u  x   dx

  j  x
n

 f x    j  x 
n


d n  u 
n
n

 f  x   dx  j 
n
du
x
n
 
 f x   dx  j n  E X n

 
d n  u 
n

 jn  E X n
u 0
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Spring 2015
17 of 18
ECE 3800
The Joint Characteristic Function
Whenever you see convolution and transformation, expect to see equations related to either the
Fourier Transform or Laplace Transform!
The characteristic function of a random variable X and Y is defined as:
 u, v   Eexp j  u  X  v  Y 
or
 XY u, v  
 
  f x, y   exp j  u  x  v  y   dx  dy
 
The inverse of the characteristic function is then defined as:
f  x, y  
 
2 2  
1
 XY u , v   exp j  u  x  v  y   du  dv
Note that
 2  u, v 
E X  Y   j 
uv
2
u v 0
 2  XY u, v 

uv
u v 0
Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System
Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9.
B.J. Bazuin, Spring 2015
18 of 18
ECE 3800