Topic 1 Fundamentals of Probability (Wooldridge, Appendix B)
Tutorial Exercise 1: In mathematics, the symbol ∑ (the Greek letter “Sigma”) denotes the
summation operator, which is defined as the sum of a sequence of numbers, as follows:
𝑥 = 𝑥 +𝑥 +⋯+ 𝑥 .
Show the following results are true:
(i)
∑
𝑐𝑥 = 𝑐 ∑
(ii)
∑
𝑐 = 𝑛𝑐
(iii)
∑
(𝑥 + 𝑦 ) = ∑
𝑥
𝑥 +∑
𝑦
Learning outcome: The purpose of this exercise is to practise our skills on working with the
summation operator.
Solution: We merely apply the definition of the summation operator throughout.
(i)
∑
𝑐𝑥 = 𝑐𝑥 + 𝑐𝑥 + ⋯ + 𝑐𝑥 = 𝑐(𝑥 + 𝑥 + ⋯ + 𝑥 ) = 𝑐 ∑
(ii)
∑
𝑐 = 𝑐 + 𝑐 + ⋯ + 𝑐 = 𝑐(1 + 1 + ⋯ + 1) = 𝑐𝑛 = 𝑛𝑐.
(iii)
∑
(𝑥 + 𝑦 ) = (𝑥 + 𝑦 ) + (𝑥 + 𝑦 ) + ⋯ + (𝑥 + 𝑦 )
= 𝑥 + 𝑦 +𝑥 +𝑦 +⋯+ 𝑥 +𝑦
= 𝑥 + 𝑥 + ⋯+ 𝑥 + 𝑦 + 𝑦 +⋯+ 𝑦
=
𝑥 +
𝑦.
𝑥.
Tutorial Exercise 2: Let 𝑌 and 𝑋 be two discrete random variables. Prove the following results:
(i)
(ii)
(iii)
(iv)
(v)
(vi)
(vii)
For any constant 𝑐, 𝐸(𝑐) = 𝑐.
For any constants 𝑎 and 𝑏, 𝐸(𝑎𝑋 + 𝑏) = 𝑎𝐸(𝑋) + 𝑏.
𝐸(𝑌|𝑋) = 𝐸(𝑌) if 𝑌 and 𝑋 are independent.
For any constant 𝑐, 𝑉𝑎𝑟(𝑐) = 0
For any constants 𝑎 and 𝑏, 𝐸(𝑎𝑋 + 𝑏𝑌) = 𝑎𝐸(𝑋) + 𝑏𝐸(𝑌).
𝑉𝑎𝑟(𝑎𝑋 + 𝑏𝑌) = 𝑎 𝑉𝑎𝑟(𝑋) + 𝑏 𝑉𝑎𝑟(𝑌) + 2𝑎𝑏𝐶𝑜𝑣(𝑋, 𝑌).
𝐶𝑜𝑣(𝑎𝑋, 𝑏𝑌) = 𝑎𝑏𝐶𝑜𝑣(𝑋, 𝑌).
Learning outcome: The purpose of this exercise is to deepen our understanding on the
properties of the expectation operator (expected value), the variance and the covariance.
Hint: Given a random discrete random variable 𝑋, let 𝑔(𝑋) be a new discrete random variable,
for some real-valued function 𝑔(. ). Make use of the definition of the expected value of 𝑔(𝑋),
which is given by
𝐸[𝑔(𝑋)] = 𝑔(𝑥 )𝑓(𝑥 ) + 𝑔(𝑥 )𝑓(𝑥 ) + ⋯ + 𝑔(𝑥 )𝑓(𝑥 ) =
𝑔 𝑥 𝑓 𝑥 .
Since (i) is a special case of (ii), you may complete (ii) first.
Solution: We apply the above formula repeatedly. In particular,
(i)
let 𝑔(𝑋) = 𝑐 + 𝛼𝑋 = 𝑐 + 0𝑋 = 𝑐, where 𝛼 = 0. We have
𝐸[𝑔(𝑋)] = 𝑔(𝑥 )𝑓(𝑥 ) + 𝑔(𝑥 )𝑓(𝑥 ) + ⋯ + 𝑔(𝑥 )𝑓(𝑥 )
= (𝑐 + 𝛼𝑥 )𝑓(𝑥 ) + (𝑐 + 𝛼𝑥 )𝑓(𝑥 ) + ⋯ + (𝑐 + 𝛼𝑥 )𝑓(𝑥 )
= (𝑐 + 0𝑥 )𝑓(𝑥 ) + (𝑐 + 0𝑥 )𝑓(𝑥 ) + ⋯ + (𝑐 + 0𝑥 )𝑓(𝑥 )
= 𝑐𝑓(𝑥 ) + 𝑐𝑓(𝑥 ) + ⋯ + 𝑐𝑓(𝑥 )
= 𝑐[𝑓(𝑥 ) + 𝑓(𝑥 ) + ⋯ + 𝑓(𝑥 )]
=𝑐
𝑓 𝑥
= 𝑐,
where the last equality holds because ∑ 𝑓 𝑥
𝑃(𝑋 = 𝑥 ) + 𝑃(𝑋 = 𝑥 )+. . . +𝑃(𝑋 = 𝑥 ) = 1.
(ii)
= 𝑓(𝑥 ) + 𝑓(𝑥 ) + ⋯ + 𝑓(𝑥 ) =
Let 𝑔(𝑋) = 𝑎𝑋 + 𝑏. We have
𝐸[𝑔(𝑋)] = 𝑔(𝑥 )𝑓(𝑥 ) + 𝑔(𝑥 )𝑓(𝑥 ) + ⋯ + 𝑔(𝑥 )𝑓(𝑥 )
= (𝑎𝑥 + 𝑏)𝑓(𝑥 ) + (𝑎𝑥 + 𝑏)𝑓(𝑥 ) + ⋯ + (𝑎𝑥 + 𝑏)𝑓(𝑥 )
= 𝑎𝑥 𝑓(𝑥 ) + 𝑏𝑓(𝑥 ) + 𝑎𝑥 𝑓(𝑥 ) + 𝑏𝑓(𝑥 ) + ⋯ + 𝑎𝑥 𝑓(𝑥 ) + 𝑏𝑓(𝑥 )
= 𝑎𝑥 𝑓(𝑥 ) + ⋯ + 𝑎𝑥 𝑓(𝑥 ) + 𝑏𝑓(𝑥 ) + ⋯ + 𝑏𝑓(𝑥 )
=𝑎
𝑥 𝑓 𝑥 +𝑏
𝑓 𝑥
= 𝑎𝐸(𝑋) + 𝑏.
(iii)
We have
𝑓
|
(𝑦|𝑥) =
𝑓 , (𝑥, 𝑦) 𝑓 (𝑥)𝑓 (𝑦),
=
= 𝑓 (𝑦).
𝑓 (𝑥)
𝑓 (𝑥)
Where the second equality is due to the fact that 𝑌 and 𝑋 are independent. Therefore,
𝐸(𝑌|𝑋 = 𝑥) = 𝑦 𝑓
|
(𝑦 |𝑥) + 𝑦 𝑓
|
(𝑦 |𝑥) + ⋯ + 𝑦 𝑓
|
(𝑦 |𝑥)
= 𝑦 𝑓 (𝑦 ) + 𝑦 𝑓 (𝑦 ) + ⋯ + 𝑦 𝑓 (𝑦 )
=
(iv)
𝑦𝑓 𝑦
= 𝐸(𝑌).
Let 𝑔(𝑋) = 𝑐 + 0𝑋 = 𝑐 ⇒ [𝑔(𝑋)] = 𝑐 . The variance of 𝑔(𝑋) is given by
𝑉𝑎𝑟[𝑔(𝑋)] = 𝐸[𝑔(𝑋) ] − 𝐸 𝑔(𝑋)
We have already shown that 𝐸 𝑔(𝑋) = 𝑐 ⇒ 𝐸 𝑔(𝑋)
.
= 𝑐 . On the other hand,
𝐸[𝑔(𝑋) ] = [𝑔(𝑥 )] 𝑓(𝑥 ) + [𝑔(𝑥 )] 𝑓(𝑥 ) + ⋯ + [𝑔(𝑥 )] 𝑓(𝑥 )
= 𝑐 𝑓(𝑥 ) + 𝑐 𝑓(𝑥 ) + ⋯ + 𝑐 𝑓(𝑥 )
=𝑐
∴ 𝑉𝑎𝑟(𝑐) = 𝑐 − 𝑐 = 0.
𝑓 𝑥
=𝑐 .
(v)
Let 𝑔(𝑋, 𝑌) = 𝑎𝑋 + 𝑏𝑌. We have
𝐸[𝑔(𝑋, 𝑌)] =
𝑎𝑥 + 𝑏𝑦 𝑓
,
𝑥 ,𝑦
,
=
=𝑎
𝑎𝑥 + 𝑏𝑦 𝑓
𝑥
𝑓
=𝑎
,
𝑥 ,𝑦 + 𝑏
𝑥 𝑓 (𝑥 ) + 𝑏
𝑥 ,𝑦
,
𝑦
𝑓
,
𝑥 ,𝑦
𝑦𝑓 𝑦
= 𝑎𝐸(𝑋) + 𝑏𝐸(𝑌),
where the second-to-last equality is due to the Law of Total Probability, which states
that ∑ 𝑓 , 𝑥 , 𝑦 = 𝑓 (𝑥 ) and ∑ 𝑓 , 𝑥 , 𝑦 = 𝑓 (𝑦 ).
(vi)
We have
𝑉𝑎𝑟(𝑎𝑋 + 𝑏𝑌) = 𝐸 (𝑎𝑋 + 𝑏𝑌) − 𝐸 (𝑎𝑋 + 𝑏𝑌)
= 𝐸[𝑎𝑋 + 𝑏𝑌 − 𝑎𝐸(𝑋) − 𝑏𝐸(𝑌)]
= 𝐸 𝑎 𝑋 − 𝐸(𝑋) + 𝑏 𝑌 − 𝐸(𝑌)
= 𝐸 𝑎 𝑋 − 𝐸(𝑋)
=𝑎 𝐸
𝑋 − 𝐸(𝑋)
+ 𝑏 𝑌 − 𝐸(𝑌)
+𝑏 𝐸
𝑌 − 𝐸(𝑌)
+ 2𝑎𝑏 𝑋 − 𝐸(𝑋) 𝑌 − 𝐸(𝑌)
+ 2𝑎𝑏𝐸 𝑋 − 𝐸(𝑋) 𝑌 − 𝐸(𝑌)
= 𝑎 𝑉𝑎𝑟(𝑋) + 𝑏 𝑉𝑎𝑟(𝑌) + 2𝑎𝑏𝐶𝑜𝑣(𝑋, 𝑌),
where the second equality is due to part (v) of this exercise, whereas the fourth
equality is due to the binomial theorem, or otherwise (𝑎 + 𝑏) = (𝑎 + 𝑏)(𝑎 + 𝑏) =
𝑎 + 𝑎𝑏 + 𝑏𝑎 + 𝑏 = 𝑎 + 𝑎𝑏 + 𝑎𝑏 + 𝑏 = 𝑎 + 2𝑎𝑏 + 𝑏 .
(vii)
We have
𝐶𝑜𝑣(𝑎𝑋, 𝑏𝑌) = 𝐸 𝑎𝑋 − 𝐸(𝑎𝑋) 𝑏𝑌 − 𝐸(𝑏𝑌)
= 𝐸 𝑎𝑋 − 𝑎𝐸(𝑋) 𝑏𝑌 − 𝑏𝐸(𝑌)
= 𝐸 𝑎 𝑋 − 𝐸(𝑋) 𝑏 𝑌 − 𝐸(𝑌)
= 𝑎𝑏𝐸 𝑋 − 𝐸(𝑋) 𝑌 − 𝐸(𝑌)
= 𝑎𝑏𝐶𝑜𝑣(𝑋, 𝑌).
where the second and fourth equalities are due to part (ii) of this exercise.