MTH206 - Sem 1 2020/21 Handout on discrete
MGF
Giovanni Merola
June 25, 2020
Discrete distributions
Reminder
Binomial Theorem
r n k (n−k)
( a + b)n = ∑
a b
k
k =0
Taylor expansion
∞
f (x) =
∑
k =0
f (k) ( a )
( x − a)k
k!
Pay attention to the fact that at k = 0 f (k) ( a)/k! = f ( a).
Maclaurin expansion is Taylor for a = 0
∞
f (x) =
∑
k =0
If you don’t remember it, try it out for
n = 2, 3
Why?
f 00 (0) 2
f ( k ) (0) k
f ( k ) (0) k
x = f (0) + f 0 (0) x +
x +···+
x +···
k!
2
k!
We will need the Maclaurin expansion of h(w) = (1 − w)−r , w >
−1, 1. Consider that h(0) = 1 and
h0 (w)|0 = (−r )(−1)(1 − w)−(r+1) |0 = r,
h00 (w)|0 = r (r + 1)(1 − w)−(r+2) |0 = r (r + 1),
..
.
h(k) (w)|0 = r (r + 1) · · · , (r + k − 1) = (r + k − 1)!/(r − 1)!
Then, the Maclaurin expansion is
(1 − w )
−r
∞ (r + k − 1) ! k
r+k−1
w = ∑
wk
= ∑
(
r
−
1
)
!k!
r
−
1
k =0
k =0
∞
Binomial
If p is the probability of success, and X the number of successes in n
trials
n x
f ( x ) = P( X = x ) =
p (1 − p)n− x , x = 0, 1, 2, . . . , n.
x
∑nx=0 f ( x ) = 1 because, by the Binomial theorem, it is equal to ( p +
(1 − p)n = 1.
where does the binomial coefficient
come from?
mth206 - sem 1 2020/21 handout on discrete mgf
The Moment Generating Function is
n
n x
M(t) = E(etx ) = ∑ etx
p (1 − p ) n − x =
x
x =0
2
n
n
∑ x ( e t p ) x (1 − p ) n − x =
x =0
[(1 − p) + pet ]n , −∞ < t < ∞
M0 (t) = np[(1 − p) + pet ]n−1 so µ = E( X ) = M (0)0 = np.
Use M00 (t) to verify that σX2 = np(1 − p)
Geometric and Negative Binomial distributions
Both distributions are still defined from a sequences of independent
Bernulli variables P( Xi = 0) = q = 1 − p, P( Xi = 1) = p.
Geometric distribution
A Geometric variable, Y, is the number of trials required to observe the first success.
The first success will happen at the kth trial if the first (k − 1)
trials were failures and the last was a success. Since the trials are
independent, the probability mass function is given by
f (y) = P(Y = y) = pqy−1 , y = 1, 2, . . .
It is easy to see that the sum of the probabilities is equal to 1, by
y−1 = p ∞ qz = p/ (1 − q ) = p/p = 1.
considering that ∑∞
∑ z =0
y=1 pq
The CDF can be easily found as
F ( y ) = P (Y ≤ k ) =
k
k −1
y =1
z =0
1 − qk
∑ pqy−1 = p ∑ qz = p 1 − q
Caution. In some cases the Geometric
variable is defined as the number of
failures before the first success. That
is Y − 1, with repect to our definition.
there are good arguments in favour of
both deefinitions.
This is similar to the Binomial but now
the order is fixed. The first success must
happen at the last trial. We can see this
also as two experiments with different
stopping rules. The Binomial stops after
n trials, the geometric after the first
success.
= 1 − qk
Moments For as simple as the distribution looks, finding the moments requires a little care.
direct way
E (Y ) =
∞
∞
y =1
y =1
∑ ypqy−1 = ∑ (y − 1 + 1) pqy−1 =
∞
∑ (y − 1) pqy−1 +
y =1
∞
∑
pqz = q
z =0
∞
p
∑ zpqz−1 + 1 − q
=
z =1
qE(Y ) + 1.
Therefore, E(Y ) = qE(Y ) + 1 implies that E(Y ) = 1/(1 − q) = 1/p.
Computing the variance of Y directly requires doing a similar
decomposition twice and it is a little more tricky. See, for example,
your MTH113 textbook Section 4.8.
Taking derivatives
y−1 is simple to compute.
The series ∑∞
y=1 yq
Consider that
∞
∞
y =1
z =0
1
S. Ross A first course in probability.
q
∑ qy = ∑ qz − 1 = 1 − q − 1 = 1 − q .
∞
k
0
0
∑∞
x =1 = ∑ k =1 x + x − x .
mth206 - sem 1 2020/21 handout on discrete mgf
By taking derivatives, we have
y
∂ ∑∞
y =1 q
q
∞
∑ yq
=
∂q
y −1
=
∂ 1− q
=
∂q
y =1
1
1
= 2.
2
(1 − q )
p
y −1 =
Hence, p ∑∞
= 1p .
y=1 yq
p2
For the variance, we consider the second factorial moment,
E[Y (Y − 1)] and take derivatives again.
p
y
∂2 ∑ ∞
y =1 q
∂q2
∞
=
∑
y ( y − 1 ) q y −2 =
y =1
∂ (1−1q)2
=
∂q
2
2
= 3.
(1 − q )3
p
So,
M (Y(2) ) = E(Y (Y − 1)] = E(Y 2 ) − E(Y ) = p
∞
∑ y ( y − 1 ) q y −1 =
y =1
∞
pq
∑ y ( y − 1 ) q y −2 =
y =1
2q
2pq
= 2.
p3
p
We now conclude that
Var ( X ) = E(Y (Y − 1)] + E(Y ) − [ E(Y )]2 =
2q
1
1
q
+ − 2 = 2
2
p
p
p
p
Using the moment generating function The MGF of a Geometric
variable is equal to
M (t) = E(ety ) =
∞
∞
y =1
y =1
pet
∑ ety pqy−1 = pet ∑ (qet )y−1 = 1 − qet
So,
E (Y ) = M 0 ( 0 ) =
pet (1 − qet ) + pet qet
(1 − qet )2
=
t =0
pet
(1 − qet )2
=
t =0
p
1
= .
p
p2
And
E(Y 2 ) = M00 (0) =
Therefor Var (Y ) =
1+ q
p2
−
pet (1 − qet ) + 2pet qet
(1 − qet )3
1
p2
=
=
t =0
1+q
.
p2
q
.
p2
Negative Binomial distribution
Probability of the r-th success at the x-th trial.
x−1 r
f ( x ) = P( X = x ) =
p (1 − p) x−r , x = r, r + 1, . . .
r−1
The sum of the probabilities is 1. To see this, recall the Maclaurin
expansion of (1 − w)−r in which we substitute k = x − r Then,
Remember the last trial must be a
success
That is, x = k + r, because x =
r, r + 1, · · · and k = 0, 1, . . .
3
mth206 - sem 1 2020/21 handout on discrete mgf
(1 − w ) −r =
∞
∑
k =0
r+k−1
wk =
r−1
∞
∑
x =r
x−k−1
w x −r
r−1
x − k −1
x −r =
By setting w = (1 − p) this equal to ∑∞
x =r ( r −1 )(1 − p )
−
r
−
r
(1 − (1 − p)) = p , which is like the Negative Binomial probability mass function without the constant pr . Hence the sum of the
Negative Binomial probabilities is equal to 1.
Moments The mean and variance can be found directly but solving the summations takes a long time.
I’ll show two ways.
Using the MGF The MGF is found as
∞
x−k−1 r
M (t) = ∑ e xt
p (1 − p ) x −r =
r
−
1
x =r
∞ x−k−1
( pet )r
t r
( pe ) ∑
[qet ] x−r =
r−1
(1 − qet )r
x =r
x − k −1
x −r = (1 − w ) −r
Because ∑∞
x =r ( r −1 ) w
So,
M0 (t) = rpr etr [1 − qet ]−r + pr etr qet [1 − qet ]−r−1 =
And,
rpr etr
[1 − qet ]r+1
check details in the textbook
r2 pr etr [1 − qet ] + rpr etr [(r + 1)qet ]
M (t) =
.
[1 − qet ](r+2)
00
Hence
r
p
rq
E( X ) = M00 (0) = 2
p
E ( X ) = M 0 (0) =
Using the moments of a Geometric variable
We know that a Negative Binomial variable, X,is the sum of r independent Geometric variables, X = ∑ri=1 Yi , with Xi i.i.d. Geom(p),
each with E(Yi ) = 1/p and Var (Yi ) = q/p2 .
Therefore,
r
E( X ) = ∑ E(Yi ) =
i =1
r
r
p
E( X ) = ∑ Var (Yi ) ==
i =1
rq
p2
d( pe x )t /dt = ( pe x )t
4