Chapter 2: Random Variables 2.1. Concept of a Random Variable 2.2. Distribution Functions 2.3. Density Functions 2.4. Mean Values and Moments 2.5. The Gaussian Random Variable 2.6. Density Functions Related to Gaussian 2.7. Other Probability Density Functions 2.8. Conditional Probability Distribution and Density Functions 2.9. Examples and Applications Random variable: A real function whose domain is that of the outcomes of an experiment (sample space, S) and whose actual value is unknown in advance of the experiment. From: http://en.wikipedia.org/wiki/Random_variable A random variable can be thought of as the numeric result of operating a non-deterministic mechanism or performing a non-deterministic experiment to generate a random result. Unlike the common practice with other mathematical variables, a random variable cannot be assigned a value; a random variable does not describe the actual outcome of a particular experiment, but rather describes the possible, as-yet-undetermined outcomes in terms of real numbers. Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9. B.J. Bazuin, Spring 2015 1 of 25 ECE 3800 Probability Distribution Function (PDF) Probability Distribution Function:The probability of the event that the observed random variable X is less than or equal to the allowed value x. FX x Pr X x The defined function can be discrete or continuous along the x-axis. Constraints on the probability distribution function are: 1. 0 FX x 1, for x 2. FX 0 and FX 1 3. FX is non-decreasing as x increases 4. Pr x1 X x2 FX x 2 FX x1 Note: This is also known as the cumulative distribution function or cdf in many references! (this avoids a major problems with language that is coming soon …) From: http://en.wikipedia.org/wiki/Cumulative_distribution_function In probability theory, the cumulative distribution function (abbreviated cdf) completely describes the probability distribution of a real-valued random variable, X. For every real number x, the cdf is given by FX x Pr X x where the right-hand side represents the probability that the variable X takes on a value less than or equal to x. The probability that X lies in the interval (a, b) is therefore F(b) − F(a) if a ≤ b. It is conventional to use a capital F for a cumulative distribution function, in contrast to the lowercase f used for probability density functions and probability mass functions. Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9. B.J. Bazuin, Spring 2015 2 of 25 ECE 3800 Probability Density Function (pdf) The derivative of the probability distribution function f X x lim FX x FX x 0 dFX x dx An interpretation is f X x dx Pr x X x dx FX x dx FX x Properties of the pdf include 1. f X x 0, for x 2. f x dx 1 X 3. F X x x f X x dx 4. Pr x1 X x 2 x2 f x dx X x1 From: http://en.wikipedia.org/wiki/Probability_density_function In mathematics, a probability density function (pdf) serves to represent a probability distribution in terms of integrals. A probability density function is everywhere non-negative and its integral from −∞ to +∞ is equal to 1. If a probability distribution has density f(x), then intuitively the infinitesimal interval [x, x + dx] has probability f(x) dx. Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9. B.J. Bazuin, Spring 2015 3 of 25 ECE 3800 Probability Mass Function (pmf) The probability that a discrete random variable takes on an exact value is defined as the pmf. f X x Pr X x f X x FX x FX x Note that for discrete random variables, the probability distribution function, PDF, is not continuous at the discrete inputs of interest. Properties of the pdf include 1. f X x 0, for x 2. f X u 1 u 3. F X x x f X u u 4. Pr x1 X x 2 x2 f X u u x1 From: http://en.wikipedia.org/wiki/Probability_mass_function In probability theory, a probability mass function (abbreviated pmf) gives the probability that a discrete random variable is exactly equal to some value. A probability mass function differs from a probability density function in that the values of the latter, defined only for continuous random variables, are not probabilities; rather, its integral over a set of possible values of the random variable is a probability. Note: The textbook does not differentiate between the probability density function and probability mass function. Notice that in the definitions, the pmf represents the actual probability while the pdf is defined in terms of the derivative of the “distribution” function (PDF or cdf). If you wish to pursue correct mathematical derivations, use pmf and pdf and cdf. If you just intend to apply this concept to engineering problems, you can do it like the textbook. … Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9. B.J. Bazuin, Spring 2015 4 of 25 ECE 3800 Functions of random variables In engineering analysis, many times one random variable is a function of a second random variable, for example, random power derived from a random voltage circular area derived from a random measurement of the diameter Think of Y gX … what can be described for the probability density functions (pdf) of Y and X? Since the PDF is the integral of the pdf, we should have: f X x dx f Y y dy For Y g X a monotonically increasing function of X, the new pdf should be related to the previous pdf as: f Y y f X x dx dy For Y g X a monotonically decreasing function of X, the new pdf must be increasing. As a result the new pdf should be related to the previous pdf as: f Y y f X x dx dy Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9. B.J. Bazuin, Spring 2015 5 of 25 ECE 3800 Amplitude scaling (an amplifier) a random variable … Y A X f Y y f X x dx dy dx 1 dy A Therefore, y 1 fY y f X A A Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9. B.J. Bazuin, Spring 2015 6 of 25 ECE 3800 f X x e x u x Text example: FX x e v x v 0 e x 1 1 e x , for 0 x Y X3 With What is the new pdf for Y f Y y f X x dx dy d y 3 dx 1 y 23 dy dy 3 1 1 1 1 3 2 f Y y e y u y 3 y 3 3 fY y 1 y 13 2 3 e y u y 3 Now for a hard one … Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9. B.J. Bazuin, Spring 2015 7 of 25 ECE 3800 Example Page 62 and HW 2-3.3 Y X2 Let the functional relationship be If we “plug and chug” using past formula f Y y f X x dx dy There will be a problem/uniqueness if X is allowed to be both positive and negative; we map 2 different X values into the same value of Y! This may be easier to consider based on the Distribution (PDF) and definitions. The probability and PDF definition of Y y occurs when X 2 y . This is equivalently to the following bounds y X y where y is nonnegative; see Fig. 2-9. Y X 2 x y x y The event is null when y is negative. Thus, based on the definition of the PDF we must have 0, FY y FX y F y , X y0 y0 Differentiating with respect to y, d FY y fY y dy d d y FX y f X x y y fX x y y dy dy d FX dy Therefore fY y fX y f y , y 0 X 2 y 0, fY y f X 2 y y f y , X 2 y y0 y0 Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9. B.J. Bazuin, Spring 2015 8 of 25 ECE 3800 Continuing on to the homework problem, f x x exp 2 x , We will have fx for x y exp 2 y and f y exp 2 y exp 2 y x and fY y exp 2 y exp 2 y 2 y exp 2 y y Resulting in 0, f Y y exp 2 y , y y0 y0 Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9. B.J. Bazuin, Spring 2015 9 of 25 ECE 3800 Multiple solutions due to mapping. (from Alberto Leon-Garcia, “Probability, Statistics, and Random Processes For Electrical Engineering, 3rd ed.”, Pearson Prentice Hall, Upper Saddle River, NJ, 2008, ISBN: 013-147122-8 ) If the equation has n solutions, x0 , x1, , xn , then f Y y0 will be equal to n terms of the type on the right-hand side of the solution. We now show that this is generally true by using a method for directly obtaining the pdf of Y in terms of the pdf of X. Consider a nonlinear function Y g X such as the one shown in Fig. 4.13. Consider the event C y y Y y dy and let By be its equivalent event. For y indicated in the figure, the equation g X y has three solutions x1 , x2 , and x3 and the equivalent event By has a segment corresponding to each solution: By x1 X x1 dx1 x2 X x2 dx2 x3 X x3 dx3 The probability of the event C y is approximately P C y f Y y dy where dy is the length of the interval y Y y dy . Similarly, the probability of the event By is approximately P B y f X x1 dx1 f X x2 dx2 f X x3 dx3 Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9. B.J. Bazuin, Spring 2015 10 of 25 ECE 3800 Since C y and By are equivalent events, their probabilities must be equal. By equating Eqs. (4.71) and (4.72) we obtain P C y f Y y dy P B y f X x1 dx1 f X x2 dx2 f X x3 dx3 therefore f Y y f X x1 dx1 dy f X x2 dx2 dy f X x3 dx3 dy written generically as fY y k f X x f X x dx dy dy k dx x x x xk k It is clear that if the equation g x y has n solutions, the expression for the pdf of Y at that point is given by these Eqs., and contains n terms. Example 4.35 Let Y X 2 as in Example 4.34. For y 0 the equation has two solutions, x0 y and x1 y , so Eq. (4.73) has two terms. Since dy dx 2 x , Eq. (4.73) yields fY y k f y f y f X x X X dy 2 y 2 y dx x x k Also of note is that dx d 1 y dy dy 2 y which when substituted then yields the correct result again. f Y y f X x dx k dy x xk fX y 2 1 y f y 2 1 y X (from Alberto Leon-Garcia, “Probability, Statistics, and Random Processes For Electrical Engineering, 3rd ed.”, Pearson Prentice Hall, Upper Saddle River, NJ, 2008, ISBN: 013147122-8 ) Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9. B.J. Bazuin, Spring 2015 11 of 25 ECE 3800 Example 4.36 Amplitude Samples of a Sinusoidal Waveform (from Alberto Leon-Garcia) Let Y cos X where X is uniformly distributed in the interval (0,2π]. Y can be viewed as the sample of a sinusoidal waveform at a random instant of time that is uniformly distributed over the period of the sinusoid. Find the pdf of Y. We have f X x We have 1 , 0 x 2 2 It can be seen in Fig. 4.14 that for 1 y 1 the equation y cos x has two solutions in the 1 interval of interest, x0 cos y and x1 2 cos 1 y . We will need the derivative, dy sinx0 sin cos1 y 1 y 2 dx x0 To determine the density of Y, we have to points and therefore 1 1 f X x 1 2 fY y 2 , 1 y 1 k dy 1 y2 1 y2 1 y2 dx x x k Therefore, And, fY y 1 1 y2 0, 1 sin 1 y FY y , 2 1, , 1 y 1 y 1 1 y 1 1 y Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9. B.J. Bazuin, Spring 2015 12 of 25 ECE 3800 Defining proper PDF/pdf functions (when you know the general shape) FX x 1 tan 1 x Example: Determine the values of and . Known values at FX 0 and FX 1 FX 1 tan 1 0 FX 1 0 2 1 0 2 2 FX 1 tan 1 1 FX 1 1 2 1 2 1 2 Therefore, FX x 1 2 1 2 1 tan 1 x 2 and f X x 1 1 1 x2 Note: Any strictly positive valued function can be a pdf. Even a function with a section that is purely positive for a finite region can be a “bounded in x” pdf. Once the “function” is defined, the appropriate magnitude must be determined to meet the pdf and PDF criteria. Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9. B.J. Bazuin, Spring 2015 13 of 25 ECE 3800 Mean Values and Moments Mean Value: the expected mean value of measurements of a process involving a random variable. This is commonly called the expectation operator or expected value of … and is mathematically described as: X E X x f X x dx For laboratory experiments, the expected value of a voltage measurement can be thought of as the DC voltage. For discrete random variables, the integration becomes a summation and X EX x f X x x x Pr X x x General concept of an expected value In general, the expected value of a function is: E g X g X f x dx X E g X g X f X x x E g X g X Pr X x x gX f X x dx Estimating a parameter: If we know the expected value, you have a simple estimate of future expected outcomes. xˆ X E X Or for y g x yˆ E y Eg X Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9. B.J. Bazuin, Spring 2015 14 of 25 ECE 3800 Moments The moments of a random variable are defined as the expected value of the powers of the measured output or … x X EX n n f X x dx n x Xn E Xn n x f X x x n Pr X x x Therefore, the mean or average is sometimes called the first moment. Expected Mean Squared Value or Second Moment The mean square value or second moment is x X EX 2 2 2 f X x dx X2 E X2 x 2 Pr X x x Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9. B.J. Bazuin, Spring 2015 15 of 25 ECE 3800 Central Moments The central moments are the moments of the difference between a random variable and its mean. X X n x X E XX n f X x dx n Notice that the first central moment is 0 … X X 1 x X f E XX X x dx X X x f X X 1 X 1 x dx X f X x dx X X 1 0 The second central moment is referred to as the variance of the random variable … XX 2 2 x X E XX 2 2 f X x dx The square root of the variance is the standard deviation, σ X X 2 Note that the variance may also be computed as: 2 E X X EX X X X 2 2 E X 2 2 X X X 2 2 E X 2 2 X E X X 2 E X 2 2 X X X 2 2 2 E X 2 X E X 2 E X 2 2 2 X2 X 2 The variance is equal to the 2nd moment minus the square of the first moment.. Another estimate for future outcomes, is the value that minimizes the mean squared error. 2 2 min E error min E X xˆ Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9. B.J. Bazuin, Spring 2015 16 of 25 ECE 3800 Exercise 2-3.1 and 2-4.1: A probability density function f X x 5 e K x u x Determine the value of K 1 f X x dx 1 5 e K x dx 0 1 1 5 e K x K 0 1 1 1 1 5 e K e K 0 5 K K K K 5 Therefore f X x 5 e 5 x u x As an extension or generalization, note that f X x K e K x u x Probability X>1 Pr X 1 1 1 f X x dx 1 Pr X 1 1 5 e 5 x dx 0 1 5 Pr X 1 1 e 5 x 0 5 5 5 Pr X 1 1 1 e 1 e 0.0067 Probability X≤0.5 Pr X 0.5 0.5 5 e 5 x dx 0 0 .5 5 Pr X 0.5 e 5 x 0 5 2.5 2.5 Pr X 0.5 1 e 1 1 e 0.9179 Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9. B.J. Bazuin, Spring 2015 17 of 25 ECE 3800 First Moment f X x 5 e 5 x u x for X EX x5e 5 x dx 0 Integral Table Formula x e a x X EX 5 e a x a x 1 a2 e 5x 52 X EX 5 x 1 0 5 5 2 1 5 As an extension or generalization, note for f X x K e K x u x we have X E X 1 K Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9. B.J. Bazuin, Spring 2015 18 of 25 ECE 3800 Example p. 65: Text offset uniform density x 20 0 1 f X x 20 0 20 x 40 40 x Mean Value (average DC value) X E X x f X x dx X EX 40 20 40 1 1 x dx x dx 20 20 20 1 x2 X E X 20 2 X E X 40 20 1 40 2 20 2 1 1600 400 1 1200 30 20 2 2 20 2 2 20 2 Mean Square x X EX 2 2 20 X EX 2 1 x3 20 3 X 2 E X 2 40 20 f X x dx 40 X2 E X2 2 2 x2 1 dx 20 1 40 3 20 3 1 4 3 2 3 10 3 1 20 3 3 2 3 3 10 1 64 8 10 3 56 10 2 933.3 1 2 3 3 10 6 Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9. B.J. Bazuin, Spring 2015 19 of 25 ECE 3800 Variance or Standard Deviation (average AC value squared) XX 2 2 x X E XX 2 2 f X x dx 2 2 X X E XX x 30 2 40 2 20 X X 2 2 E X X x 1 dx 20 40 2 2 60 x 900 20 2 X X 2 E XX 2 X X 2 1 dx 20 933.3 60 30 900 1 2 E X X 33.3 2 or 2 X2 X 2 2 933 .3 30 2 933.3 900 33.3 The variance or stand deviation is then 33.3 5.77 The bulk of the density function appears at in the range defined by 30+/-5.77. Pr X X X X f x dx X Pr X X X X 1 20 dx X X 2 X 20 20 10 0.577 Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9. B.J. Bazuin, Spring 2015 20 of 25 ECE 3800 Three Coins Experiment: Flip two Coins and count the number of heads S Pair TTT , HTT , THT , TTH , HHT , HTH , THH , HHH S 0,1,2,3 n Pr A occuring k times in n trials p n k p k q n k k And the probability mass function, f X x Pr X x , is then 1 , 8 3 , 8 f X x 3 , 8 1 8, 0, For FX x Pr X x for x 0 for x 1 for x 2 for x 3 else for x 0 0, 18 , FX x 4 , 8 7 8 , 1, for 0 x 1 for 1 x 2 for 2 x 3 for 3 x Mean Value X EX x Pr X x x 3 X E X x Pr X x x 0 1 3 3 1 3 6 3 12 X E X 0 1 2 3 1 .5 8 8 8 8 8 8 Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9. B.J. Bazuin, Spring 2015 21 of 25 ECE 3800 Second Moment (mean squares) x X2 E X2 2 Pr X x x 1 3 3 1 3 12 9 24 X 2 E X 2 0 2 12 2 2 3 2 3 8 8 8 8 8 8 Variance 2 X2 X 2 2 3 1.5 2 3 2.25 0.75 0.866 The bulk of the density function appears at 1.5+/-0.866. Pr X X X X Pr X x x X Pr X X X Pr X x 2 x 1 3 3 6 0.75 8 8 8 Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9. B.J. Bazuin, Spring 2015 22 of 25 ECE 3800 Triangular density function Assume that a random variables probability density function is triangular and can be described as for x 1 for 1 x 0 for 0 x 1 for 1 x 0, 1 x, f X x 1 x, 0, Find the probability distribution function. The definition FX x x f v dv X v For x 1 For 1 x 0 FX x 0 FX x x 1 v dv v 1 x v 2 FX x v 2 1 1 x2 1 x 2 1 x FX x x 2 2 2 2 For 0 x 1 1 FX x 2 x 1 v dv v0 v2 1 FX x v 2 2 x 0 1 1 x 2 x2 x FX x x 2 2 2 2 For 1 x FX x 1 Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9. B.J. Bazuin, Spring 2015 23 of 25 ECE 3800 Therefore, for x 1 0, 2 x x 1 , 2 FX x 2 2 1 x 2 x 2 , 1, for 1 x 0 for 0 x 1 for 1 x 1.2 1 0.8 0.6 0.4 0.2 0 -0.2 -2 -1.5 -1 -0.5 0 0.5 1 1.5 2 Mean Value (average DC value) X E X x f X x dx X EX 0 1 1 0 x 1 x dx x 1 x dx 0 1 x 2 x3 x 2 x3 X E X 2 2 3 3 1 0 1 1 1 1 X E X 0 2 3 2 3 Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9. B.J. Bazuin, Spring 2015 24 of 25 ECE 3800 Mean Square X 2 x EX 2 2 f X x dx 0 X 2 x EX 2 1 2 1 x dx x 2 1 x dx 1 0 0 1 4 x3 x 4 3 x x X2 E X2 3 3 4 4 1 0 1 1 1 1 1 1 1 X 2 E X 2 2 3 4 6 3 4 3 4 Variance or Standard Deviation (average AC value squared) XX 2 2 x X E XX 2 2 f X x dx 2 X2 X 2 2 1 1 02 6 6 Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9. B.J. Bazuin, Spring 2015 25 of 25 ECE 3800