Use of moment generating functions Definition Let X denote a random variable with probability density function f(x) if continuous (probability mass function p(x) if discrete) Then mX(t) = the moment generating function of X E etX tx e f x dx if X is continuous etx p x if X is discrete x The distribution of a random variable X is described by either 1. The density function f(x) if X continuous (probability mass function p(x) if X discrete), or 2. The cumulative distribution function F(x), or 3. The moment generating function mX(t) Properties 1. mX(0) = 1 2. mXk 0 k th derivative of mX t at t 0. k E X k E X 3. k k k x f x dx k x p x mX t 1 1t 2 2! t 2 X continuous X discrete 3 3! t 3 k k! t k . 4. Let X be a random variable with moment generating function mX(t). Let Y = bX + a Then mY(t) = mbX + a(t) = E(e [bX + a]t) = eatmX (bt) 5. Let X and Y be two independent random variables with moment generating function mX(t) and mY(t) . Then mX+Y(t) = mX (t) mY (t) 6. Let X and Y be two random variables with moment generating function mX(t) and mY(t) and two distribution functions FX(x) and FY(y) respectively. Let mX (t) = mY (t) then FX(x) = FY(x). This ensures that the distribution of a random variable can be identified by its moment generating function M. G. F.’s - Continuous distributions Name Continuous Uniform Exponential Gamma 2 d.f. Normal Moment generating function MX(t) ebt-eat [b-a]t for t < t for t < t 1 1-2t /2 for t < 1/2 t+(1/2)t22 e M. G. F.’s - Discrete distributions Name Discrete Uniform Bernoulli Binomial Geometric Negative Binomial Poisson Moment generating function MX(t) et etN-1 N et-1 q + pet (q + pet)N pet 1-qet pet k t 1-qe (et-1) e Moment generating function of the gamma distribution mX t E etX tx e f x dx where 1 x x e f x 0 x0 x0 e f x dx mX t E e tX tx 1 x e x e dx 0 1 t x x e dx 0 tx using or ba a 1 bx 0 a x e dx 1 a a 1 bx 0 x e dx ba then 1 t x mX t x e dx 0 t t t Moment generating function of the Standard Normal distribution mX t E etX tx e f x dx where f x 1 e 2 x2 2 thus mX t e tx 1 e 2 x2 2 dx 1 e 2 x2 tx 2 dx We will use 0 mX t e 1 e 2 x2 tx 2 x 2 2tx 2 1 e 2 b t2 2 2 x a 2b2 dx 1 dx 1 e dx 2 2 x t x 2 2 tx t 2 t 2 t2 1 2 1 2 2 2 e e dx e e dx 2 2 Note: 2 3 4 x x x ex 1 x 2! 3! 4! 2 3 t t t2 2 2 2 t mX t e 2 1 2 2! 3! 2 2 4 6 t t t 1 2 3 2 2 2! 2 3! Also mX t 1 1t 2 2! t 2 2 2m t m 2 m! 3 3! t 3 Note: 2 3 4 x x x ex 1 x 2! 3! 4! 2 3 t t t2 2 2 2 t mX t e 2 1 2 2! 3! 2 2 Also 4 2 6 2m t t t t 1 2 3 m 2 2 2! 2 3! 2 m! 2 2 3 3 mX t 1 1t t t 2! 3! k k th moment x k f x dx Equating coefficients of tk, we get k 0 if k is odd and 2 m 1 for k 2m then m 2 m! 2m ! hence 1 0, 2 1, 3 0, 4 3 Using of moment generating functions to find the distribution of functions of Random Variables Example Suppose that X has a normal distribution with mean and standard deviation . Find the distribution of Y = aX + b Solution: 2t 2 t mX t e 2 2 2 at at bt maX b t e mX at e e bt a b t e 2 2 a 2t 2 2 = the moment generating function of the normal distribution with mean a + b and variance a22. Thus Y = aX + b has a normal distribution with mean a + b and variance a22. Special Case: the z transformation Z X 1 X aX b 1 Z a b 0 2 1 2 2 2 2 Z a 1 Thus Z has a standard normal distribution . Example Suppose that X and Y are independent each having a normal distribution with means X and Y , standard deviations X and Y Find the distribution of S = X + Y Solution: mX t e mY t e X t Y t X2 t 2 2 Y2 t 2 2 Now mX Y t mX t mY t e X t X2 t 2 2 e Y t Y2 t 2 2 or m X Y t e X X t 2 X Y2 t 2 2 = the moment generating function of the normal distribution with mean X + Y and 2 2 variance X Y Thus Y = X + Y has a normal distribution 2 2 with mean X + Y and variance X Y Example Suppose that X and Y are independent each having a normal distribution with means X and Y , standard deviations X and Y Find the distribution of L = aX + bY Solution: mX t e X t X2 t 2 mY t e 2 Y t Y2 t 2 2 Now maX bY t maX t mbY t mX at mY bt e X at X2 at 2 2 e Y bt Y2 bt 2 2 or a t 2 maX bY t e a X b X 2 X b2 Y2 t 2 2 = the moment generating function of the normal distribution with mean aX + bY 2 2 2 2 and variance a X b Y Thus Y = aX + bY has a normal distribution with mean aX + BY and 2 2 2 2 variance a X b Y Special Case: a = +1 and b = -1. Thus Y = X - Y has a normal distribution with mean X - Y and variance 1 2 X 1 Y X Y 2 2 2 2 2 Example (Extension to n independent RV’s) Suppose that X1, X2, …, Xn are independent each having a normal distribution with means i, standard deviations i (for i = 1, 2, … , n) Find the distribution of L = a1X1 + a1X2 + …+ anXn Solution: mX i t e Now ma1 X1 i t i2t 2 (for i = 1, 2, … , n) 2 an X n t ma X t 1 1 mX1 a1t e 1 a1t man X n t mX n ant 12 a1t 2 2 e n an t n2 an t 2 2 or ma1 X1 an X n t e a11 ... an n a t 2 2 2 2 ... a 1 1 n n t 2 2 = the moment generating function of the normal distribution with mean a11 ... an n 2 2 2 2 and variance a1 1 ... an n Thus Y = a1X1 + … + anXn has a normal distribution with mean a11 + …+ ann 2 2 2 2 and variance a1 1 ... an n Special case: a1 a2 1 2 12 12 1 an n n 12 2 In this case X1, X2, …, Xn is a sample from a normal distribution with mean , and standard deviations ,and 1 L X1 X 2 X n n X the sample mean Thus Y x a1 x1 ... an xn n 1 n x x1 ... 1 n has a normal distribution with mean x a11 ... an n 1 ... 1 n n and variance x2 a12 12 ... an2 n2 2 1 1 1 2 2 2 ... n n n n n 2 2 2 Summary If x1, x2, …, xn is a sample from a normal distribution with mean , and standard deviations ,then x the sample mean has a normal distribution with mean x and variance 2 x 2 n standard deviation x n Sampling distribution of x 0.4 0.3 Population 0.2 0.1 0 20 30 40 50 60 The Central Limit theorem If x1, x2, …, xn is a sample from a distribution with mean , and standard deviations ,then if n is large x the sample mean has a normal distribution with mean x and variance 2 x 2 standard deviation x n n Proof: (use moment generating functions) We will use the following fact: Let m1(t), m2(t), … denote a sequence of moment generating functions corresponding to the sequence of distribution functions: F1(x) , F2(x), … Let m(t) be a moment generating function corresponding to the distribution function F(x) then if lim mi t m t for all t in an interval about 0. i then lim Fi x F x for all x. i Let x1, x2, … denote a sequence of independent random variables coming from a distribution with moment generating function m(t) and distribution function F(x). Let Sn = x1 + x2 + … + xn then mSn t mx1 x2 = m t xn t =mx t mx t 1 2 mxn t n x1 x2 now x n xn Sn n n t t or mx t m 1 t mSn m Sn n n n Let z x n then mz t e n n t x n nt mx e n t nt m n t and ln mz t t n ln m n n n t t2 Let u or n and n 2 2 u u n t t Then ln mz t t n ln m n t 2 t2 2 2 2 ln m u u u n t 2 ln m u u 2 u2 Now lim ln mz t lim ln mz t n u 0 t2 ln m u u 2 u 0 t2 lim lim 2 u 0 u2 m u m u 2u using L'Hopital's rule m u m u m u t 2 lim 2 u 0 m u 2 2 2 using L'Hopital's ru m u m u m u t m u 2 2 lim 2 u 0 t m 0 m 0 2 2 2 2 2 using L'Hopital's rule again 2 t E x E xi t2 2 2 2 2 2 i 2 2 t thus lim ln mz t and lim mz t e n n 2 t2 2 Now m t e t2 2 Is the moment generating function of the standard normal distribution Thus the limiting distribution of z is the standard normal distribution i.e. lim Fz x n x 1 e 2 u2 2 du Q.E.D.