3. Several Random Variables 3.1 Two Random Variables 3.2 Conditional Probability--Revisited 3.3 Statistical Independence 3.4 Correlation between Random Variables 3.5 Density Function of the Sum of Two Random Variables 3.6 Probability Density Function of a Function of Two Random Variables 3.7 The Characteristic Function Concepts Two Dimensional Random Variables Probability in Two Dimensions, Conditional Probability--Revisited Statistical Independence Two Dimensional Statistics, Correlation between Random Variables Density Function of the Linear Combination of Two Random Variables Multi-input Electrical Circuits Simulating Convolution Integrals Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9. B.J. Bazuin, Spring 2015 1 of 18 ECE 3800 Joint Probability Distribution Function (PDF) Probability Distribution Function:The probability of the event that the observed random variable X is less than or equal to the allowed value x and that the observed random variable Y is less than or equal to the allowed value y. F x, y Pr X x, Y y The defined function can be discrete or continuous along the x- and y-axis. Constraints on the probability distribution function are: 1. 0 F x, y 1, for x and y 2. F , y F x, F , 0 3. F , 1 4. F x, y is non-decreasing as either x or y increases 5. F x, FX x and F , y FY y Analogies: a 2-dimensional probability moving from scalars to vectors (2 or more elements) Calc 3 as compared to Calc 1 & 2 Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9. B.J. Bazuin, Spring 2015 2 of 18 ECE 3800 Joint Probability Density Function (pdf) The derivative of the probability distribution function is the density function f x, y 2 FX x xy Properties of the pdf include 1. f x, y 0, for x and y 2. f x, y dx dy 1 Note: the “volume” of the 2-D density function is one. y 3. F x, y x f u, v du dv 4. f X x f x, y dy and f Y y 5. Pr x1 X x2 , y1 Y y 2 f x, y dx y 2 x2 f x, y dx dy y1 x1 Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9. B.J. Bazuin, Spring 2015 3 of 18 ECE 3800 Expected Values E g X , Y g x, y f x, y dx dy All expected values may be computed using the Joint pdf Correlation The definition of a new expected value … Correlation EX Y x y f x, y dx dy A value describing the relationship – how correlated – two random variables are to each other. Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9. B.J. Bazuin, Spring 2015 4 of 18 ECE 3800 Uniform Density Example The uniform density function in two dimensions can be defined as: f X ,Y x, y 1 , x 2 x1 y 2 y1 0 for x1 x x 2 and y1 y y 2 , else Determine the density in y fY y x2 f x, y dx X ,Y x1 x2 x2 1 x fY y dx x 2 x1 y 2 y1 x x 2 x1 y 2 y1 x1 1 fY y x2 x1 x2 x1 y 2 y1 1 , y 2 y1 for y1 y y 2 Similarly f X x 1 , x 2 x1 for x1 y x 2 Correlation EX Y x y f x, y dx dy E X Y y 2 x2 x y1 x1 2 x y dx dy x1 y 2 y1 x2 y EX Y x2 x1 y 2 y1 2 y1 y2 x EX Y 2 y2 x1 1 2 x 2 x1 y 2 y1 2 2 2 EX Y x2 x1 y 2 y1 2 2 y 2 2 1 2 y x 2 x1 dy dy x x y y 2 2 1 2 1 y x1 1 x2 x x y 2 y 2 1 1 2 2 2 2 y y1 2 y1 y2 1 x 2 x1 y 2 y1 4 Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9. B.J. Bazuin, Spring 2015 5 of 18 ECE 3800 Exercise 3-1.1 Part a and b would appear to have the same result. It would seem that the “random errors” should be independent, so parts the 2nd and 3rd solution values make sense. Where the first solution value came from, I do not know. Exercise 3-1.2 f X ,Y x, y A exp 2 x 3 y , 0 , for 0 x and 0 y else Determine A 1 0 0 f x, y dx dy A exp 2 x 3 y dx dy exp 2 x 1 A exp 3 y exp 2 x dx dy A exp 3 y dy 2 0 0 0 0 1 1 exp 3 y 1 1 1 A exp 3 y dy A A 3 2 0 2 2 3 0 A6 Determine the Distribution Function y x F x, y 6 exp 2 x 3 y dx dy 0 0 x F x, y 6 exp 3 y exp 2 x dx dy 0 0 y 1 exp 2 x F x, y 6 exp 3 y dy 2 2 0 y 1 exp 2 x 1 exp 3 y F x, y 6 3 1 exp 2 x 1 exp 3 y 2 3 2 Then, 1 1 3 F x , y 1 exp 1 1 exp 2 4 4 Finally, determine the correlation Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9. B.J. Bazuin, Spring 2015 6 of 18 ECE 3800 E X Y x y f x, y dx dy EX Y x y 6 exp 2 x 3 y dx dy 0 0 x expa x Given expa x a x 1 a2 exp 2 x exp 3 y EX Y 6 2 x 1 3 y 1 4 9 0 0 1 1 E X Y 6 2 0 1 3 0 1 4 9 1 1 1 E X Y 6 4 9 6 Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9. B.J. Bazuin, Spring 2015 7 of 18 ECE 3800 Conditional Probability (Again, with multiple r.v.) Using the Probability Distribution Function (PDF), define Pr X x | M F x, y FX x | Y y FY y Pr M Another way. FX x | y1 Y y 2 F x, y 2 F x, y1 FY y 2 FY y1 Leading to, f X x | Y y f x, y fY y fY y | X x f x, y f X x and These are different from the probability of a continuous distribution taking on a single value in X and Y… FX X x 0 or FY Y y 0 An engineering derivation follows: F x, y y F x, y F x, y y F x, y y lim FX x | Y y lim F y y F y y 0 FY y y FY y y 0 Y Y y x FX x | Y y F x, y FY y y f u, y du y fY y [Note: Equ(3-9) is in error, the integral is to x not infinity] Then FX x | Y y 2 F x, y x FY y yx y f x, y fY y Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9. B.J. Bazuin, Spring 2015 8 of 18 ECE 3800 The corresponding conditional density function is f x | Y y f x, y fY y f y | X x f x, y f X x and similarly it can be shown that [Note: Equ (3-13) is in error, the denominator is a variable in x.] From these equations, it can be seen that f x, y f x | Y y f Y y f y | X x f X x The joint density total probability concepts can define the x and y marginal densities. f X x f x, y dy and f Y y f x, y dx Then from the conditional density relationship with the joint density f x, y f x | Y y f Y y f y | X x f X x we can replace the joint density functions in the total probability equations to define the pdf densities of x and y based on the conditional densities as fY y f x, y dx f y | X x f X x dx or f X x f x, y dy f x | Y y fY y dy Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9. B.J. Bazuin, Spring 2015 9 of 18 ECE 3800 To derive the multiple variable Bayes Theorem, use f x, y f x | Y y f Y y f y | X x f X x resulting in f x | Y y f y | X x f X x fY y f y | X x f x | Y y fY y f X x or Note: the joint probability density function completely specifies: both marginal density functions and both conditional density functions. Warning: The example on 127 and 128 does not make sense without information from p. 136-137. In particular, Eq. 3-17 implies that you know how to form fy(y) when this concept has not been introduced. It should be revisited after discussing the pdf of “the sum of two random variables”. Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9. B.J. Bazuin, Spring 2015 10 of 18 ECE 3800 Statistical Independence f x, y f X x f Y y Why have already seen this in the two examples presented. Joint Uniform Densities f X ,Y x, y 1 , x 2 x1 y 2 y1 for x1 x x 2 and y1 y y 2 Where we proved fY y 1 , y2 y1 for y1 y y2 f X x 1 , x 2 x1 for x1 y x 2 and Therefore 1 1 , f X ,Y x, y f X x f Y y x x y y 1 2 1 2 for x1 x x 2 and y1 y y 2 Exercise 3-1.2 f X ,Y x, y A exp 2 x 3 y , for 0 x and 0 y Where it turns out that f X ,Y x, y 2 exp 2 x 3 exp 3 y , for 0 x and 0 y As fY y f x, y dx 2 exp 2 x 3 exp 3 y dx 1 exp 2 x f Y y 3 exp 3 y 2 exp 2 x dx 3 exp 3 y 2 2 2 0 f Y y 3 exp 3 y and f X x 2 exp 2 x Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9. B.J. Bazuin, Spring 2015 11 of 18 ECE 3800 Definition of correlation for X and Y independent E X Y x y f x, y dx dy x f X x dx y f Y y x dy E X Y E X E Y X Y As another consequence, the conditional density is simplified as f x | Y y f y | X x f X x fY y f X x f X x fY y fY y f y | X x f x | Y y fY y f X x fY y fY y f X x f X x and similarly [Note: p.131 first equation is in error, the denominator is the density function.] In general, if independence can be establish, or even assumed, the computations to be performed become much easier! Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9. B.J. Bazuin, Spring 2015 12 of 18 ECE 3800 Example p. 126 revisited for independence and correlation. f X ,Y x, y 6 1 x 2 y , 5 for 0 x 1 and 0 y 1 Where the marginal densities are 1 6 6 x3 f Y y 1 x 2 y dx x y 5 5 3 0 0 6 y f Y y 1 5 3 1 and 1 6 6 y2 f X x 1 x 2 y dy y x 2 5 5 2 0 0 1 6 x2 f X x 1 5 2 Note that f x, y f X x f Y y , Therefore the variables are not independent! From computations: E Y 7 3 9 7 . EY2 and E X . E X2 15 10 20 25 Then the correlation value is EX Y x y f x, y dx dy 1 1 1 3 y2 6 6 2 2 y E X Y x y 1 x y dx dy x x dx 5 00 5 2 3 0 1 6 6 x2 x4 1 2 1 E X Y x x dx 5 2 3 5 4 12 0 E X Y 6 1 1 1 5 4 12 5 And again EX Y X Y which would be the case for independents random variables. Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9. B.J. Bazuin, Spring 2015 13 of 18 ECE 3800 Example 3-3.1 f X ,Y x, y k exp x y 1, for 0 x and 1 y There is no a, unless the problem was supposed to be stated as … f X ,Y x, y k exp x y a , for 0 x and 1 y but then k and a are not necessarily computed separately as 1 and 1.! Overall, the correct pdf is f X ,Y x, y exp x y 1, for 0 x and 1 y Correlation EX Y x y f x, y dx dy E X Y x y exp x y 1 dx dy 1 0 expax ax 1 a2 x expax dx exp0 0 1 dx E X Y exp 1 y exp y 2 1 1 E X Y exp 1 y exp y 1 dx 1 exp 1 1 1 exp 1 exp 1 2 2 E X Y exp 1 2 1 Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9. B.J. Bazuin, Spring 2015 14 of 18 ECE 3800 Exercise 3-3.2 Assume X and Y independent f X x 0.5 exp x 1 , for x f Y y 0.5 exp y 1 , for y Find PrX Y 0 ? That is, the product of the random variables is positive. PrX Y 0 PrX 0 PrY 0 PrX 0 PrY 0 PrX Y 0 1 Fx 0 1 FY 0 Fx 0 FY 0 find the distribution for X and Y based on the ranges defined for the absolute value FX x x 0.5 exp x 1 dx For for x 1 and for 1 x for x 1 FX x for 1 x x x 0.5 expx 1 dx FX x 0.5 0.5 exp1 x dx 1 exp1 x 1 1 FX x 0.5 0.5 1 exp1 x exp x 1 1 FX x 0.5 exp x 1 F X x 0 .5 x F X x 0 .5 0 .5 x FX 0 FY 0 0.5 exp 1 0.1839 Pr X Y 0 1 Fx 0 1 FY 0 Fx 0 FY 0 0.6998 Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9. B.J. Bazuin, Spring 2015 15 of 18 ECE 3800 Correlation and Covariance between Random Variables The definition of correlation was given as x y f x, y dx dy EX Y But most of the time, we are not interested in products of mean values (observed when X and Y are independent) but what results when they are removed prior to the computation. Developing values where the random variable means have been extracted, is defined as computing the covariance E X E X Y E Y x X y Y f x, y dx dy This gives rise to another factor, when the random variable variances are used to normalize the factors or covariance computation as: X X E X Y Y Y x X X y Y Y f x, y dx dy This equation defines the correlation coefficient or normalized covariance, the modified random variables are called the standardized variables and have zero mean and a unit variance. An alternate expression for the correlation coefficient is derived by performing the multiplication X X Y Y X Y E X Y X Y Y X X Y E X Y x y X y Y x X Y X Y f x, y dx dy x y f x , y dx dy y f x , y dx dy X 1 X Y Y x f x, y dx dy X Y f x, y dx dy Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9. B.J. Bazuin, Spring 2015 16 of 18 ECE 3800 1 x y f x , y dx dy y f y dy x f x dx X Y X Y X Y 1 x y f x, y dx dy X Y Y X X Y X Y E x y X Y X Y An alternate method to “skip the integrals” … X X Y Y X Y X Y X Y Y X X Y E X Y E The expected value is a linear operator … constants remain constants and sums are sums … E X Y X E Y Y E X X Y X Y E X Y X Y Y X X Y X Y E X Y X Y X Y Simplifications for random variables that are inherently zero mean with a unit variance, E x y For either X or Y a zero mean variable, E X Y X Y and for independent random variables … E X Y X Y X Y X Y 0 X Y X Y Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9. B.J. Bazuin, Spring 2015 17 of 18 ECE 3800 Standardized variables and have zero mean and a unit variance. This is similar to using the normal density/distribution for a Gaussian. The standardized or normalized R.V. must have a zero mean and unit variance (normalized). X X X Y Y and Y Note that now Y Y X X E E 0 and E E Y X and X X E E X 1 E X X Y Y X Y Y Y Y E E X Y X Y X Y Remember: this is generalized for all R.V, not just Gaussian/Normal R.V. There are also computations based on the sum and difference of these random variables that can be computed. E E E 0 0 0 E E 2 2 2 2 E 1 2 1 E 2 1 E E 2 2 E E 2 2 2 2 and the variance is the same value Var E E 2 1 2 2 Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9. B.J. Bazuin, Spring 2015 18 of 18 ECE 3800