Chapter 5 Expectations
主講人:虞台文
Content

Introduction

Expectation of a Function of a Random Variable

Expectation of Functions of Multiple Random Variables

Important Properties of Expectation

Conditional Expectations

Moment Generating Functions

Inequalities

The Weak Law of Large Numbers and Central Limit Theorems
Chapter 5 Expectations
Introduction
有夢最美
有夢最美
Definition  Expectation
The expectation (mean), E[X] or X, of a random
variable X is defined by:
 xi pX (xi ) if X is discrete
 i
E[ X ]   X   
  xf X ( x)dx if X is continuous
 
Definition  Expectation
The expectation (mean), E[X] or X, of a random
variable X is defined by:
 xi pX (xi ) if X is discrete
 i
E[ X ]   X   
  xf X ( x)dx if X is continuous
 
provided that the relevant sum or integral is absolutely convergent, i.e.,
x
i
i
p X (xi )   or



x f X ( x)dx  .
有些隨機變數不存在期望值。
若存在則為一常數。
Definition  Expectation
The expectation (mean), E[X] or X, of a random
variable X is defined by:
 xi pX (xi ) if X is discrete
 i
E[ X ]   X   
  xf X ( x)dx if X is continuous
 
provided that the relevant sum or integral is absolutely convergent, i.e.,
x
i
i
p X (xi )   or



x f X ( x)dx  .
 xi pX (xi ) if X is discrete

E[ X ]   X   i
  xf X ( x)dx if X is continuous
 
Example 1
Let X denote #good components in the experiment.
 4  3 
 

x  3  x 

p X ( x) 
, x  0,1, 2,3
7
 
 
 3
x
0
1
2
3
p X ( x)
1
35
12
35
18
35
4
35
60
18
4

E[ X ]  0  351  1 12

2


3

35  1.7
35
35
35
 xi pX (xi ) if X is discrete

E[ X ]   X   i
  xf X ( x)dx if X is continuous
 
Example 2

 20, 000
20, 000
1 
E[ X ]   x 
dx  
dx  20, 000x
 200
3
2
100
100
100
x
x
 xi pX (xi ) if X is discrete

E[ X ]   X   i
  xf X ( x)dx if X is continuous
 
Example 3



1
1 
1

1
?





x
(
x

1)
x
x

1


x 1
x 1
1 1   1 1   1 1 
       
1 2   2 3   3 4 
1
驗證此為一
正確之pdf
1
1
1
 
x( x  1) x x  1
 xi pX (xi ) if X is discrete

E[ X ]   X   i
  xf X ( x)dx if X is continuous
 
Example 3


1
1
1 1 1
E[ X ]   x 

   
x( x  1) x 1 x  1 2 3 4
x 1

Chapter 5 Expectations
Expectation of a Function
of a Random Variable
The Expectation of Y=g(X)
E[Y ]  E  g ( X )
 g ( xi ) pX (xi ) if X is discrete
 i
 
  g ( x) f X ( x)dx if X is continuous
 
The Expectation of Y=g(X)
E[Y ]  E  g ( X )
 g ( xi ) pX (xi ) if X is discrete
 i
 
  g ( x) f X ( x)dx if X is continuous
 
 g ( xi ) pX (xi ) if X is discrete

E  g ( X )   i
  g ( x) f X ( x)dx if X is continuous
 
Example 4
9
E  g ( X )  E[2 X 1]   (2 x  1) p X ( x)
x4
 7  121  9  121  11 14  13  14  15  16  17  16
 12.67
 g ( xi ) pX (xi ) if X is discrete

E  g ( X )   i
  g ( x) f X ( x)dx if X is continuous
 
Example 5
E  g ( X )  E[4 X  3] 
 13 
2
1
 4x
3
2
1
3 1

(4 x  3) x 2 dx
 3x dx  13  x  x
2
4
3

2
1
8
E  g ( X )
某些g(X)吾人特感興趣
Moments
E  X k 
第k次動差
X  E  X 
E ( X   X ) 
k
第ㄧ次動差謂之均數(mean)
第k次中央動差
Var[ X ]    E ( X   X ) 
2
X
2
第二次中央動差謂之變異數(variance)
 g ( xi ) pX (xi ) if X is discrete

E  g ( X )   i
  g ( x) f X ( x)dx if X is continuous
 
均數、變異數與標準差
X  E  X 
 xi p X (xi ) if X is discrete

  i
  xf X ( x)dx if X is continuous
 
Var[ X ]    E ( X   X ) 
2
X
2
 ( xi   X )2 pX (xi ) if X is discrete

  i
  ( x   X )2 f X ( x)dx if X is continuous
 
X :為標準差
X ~ B(n, p)
Example 6
E[X]=?
n x
pX ( x)    p (1  p)n  x , x  0,1,
 x
Var[X]=?
,n
n
n x
n x
n x
E[ X ]   x   p (1  p)
  x   p (1  p)n  x
x 0  x 
x 1  x 
n
n 1 n  1
 n  1 x

 x 1
n x
n 1 x
 n 
p
(1

p
)

n
p
(1

p
)




x

1
x
x 1 
x 0 


n
 n  1 x
n 1
n 1 x
 np  
p
(1

p
)
 np  p  1  p  
 np

x 0  x 
n 1
X ~ B(n, p)
Example 6
E[X]=?np Var[X]=?
n x
pX ( x)    p (1  p)n  x , x  0,1,
 x
,n
n
n x
n x
n x
E[ X ]   x   p (1  p)
  x   p (1  p)n  x
x 0  x 
x 1  x 
n
n 1 n  1
 n  1 x

 x 1
n x
n 1 x
 n 
p
(1

p
)

n
p
(1

p
)




x

1
x
x 1 
x 0 


n
 n  1 x
n 1
n 1 x
 np  
p
(1

p
)
 np  p  1  p  
 np

x 0  x 
n 1
X ~ B(n, p)
Example 6
E[X]=?np
Var[X]=?npq
n x
pX ( x)    p (1  p)n  x , x  0,1, , n
 x
n
2 n x
Var[ X ]    x  np    p (1  p )n  x
x 0
 x
n
n
n x
 n x
 n x
n x
n x
2 2
  x   p (1  p)  2np  x   p (1  p)  n p    p (1  p)n  x
x 0
x 0  x 
x 0  x 
 x
n
2
n2 p 2  np(1  p)
 np (1  p )
 npq
2n 2 p 2
n2 p 2
X ~ Exp()
1
Example 7
1
E[X]=?

Var[X]=? 2

f X ( x)   e   x , x  0

E[ X ]    xe
0
Var[ X ]   

0
 x
dx 
1

 (3)
 (2)
(1)
3
2

2


1   x
1   x

2  x
 x
 x   e dx   0 x e dx  20 xe dx   0 e dx


2
2
1
1
 2  2  2  2




Summary of Important Moments of
Random Variables
Chapter 5 Expectations
Expectation of Functions of
Multiple Random Variables
The Expectation of Y = g(X1, …, Xn)
E[Y ]  E  g ( X1 ,

x
 1


 
 g(x ,
1
, xn ) p (x1 ,
, xn )
, xn ) f ( x1 ,
, xn )dx1
, X n )
discrete case
xn



g ( x1 ,
dxn
continuous case
Example 8
p(x, y)
Y 1 2 3
X
1 0 16 16
2 16 0 16
3 16 16 0
E[ X  Y ]  16 (1  2)  (1  3)  (2  1)  (2  3)  (3  1)  (3  2) 
4
E[ XY ]  16 (1 2)  (1 3)  (2 1)  (2  3)  (3 1)  (3  2)

11
3
Example 9
E[Y / X ]  
1

2
0 0
1 1 2
y x(1  3 y 2 )
2
dxdy  0 0 y (1  3 y )dxdy
4
x
4
1
5
1
1 2 3 4
2
  y (1  3 y )dy   y  y  
2 0
8 0 8
4
1
Chapter 5 Expectations
Important Properties
of Expectation
Linearity
E1. E[ k ]  k
常數之期望值為常數
E2. E[a1 X 1  a2 X 2 
 an X n ]
 a1 E[ X 1 ]  a2 E[ X 2 ] 
 an E[ X n ]
X1, X2, …, Xn間不須具備任何條件，上項特性均成立。
E1. E[ k ]  k
E2. E[a1 X 1  a2 X 2 
Example 10
 an X n ]
 a1 E[ X 1 ]  a2 E[ X 2 ] 
 an E[ X n ]
令X與Y為兩連續型隨機變數，證明E[X+Y] = E[X]+E[Y].
E[ X  Y ]  


 

 
 
 
( x  y) f ( x, y)dxdy
xf ( x, y)dxdy  



 
yf ( x, y)dxdy








  x  f ( x, y)dydx   y  f ( x, y)dxdy




  xf X ( x)dx   yfY ( y)dy
 E[ X ]  E[Y ]
E1. E[ k ]  k
E2. E[a1 X 1  a2 X 2 
A Question
 an X n ]
 a1 E[ X 1 ]  a2 E[ X 2 ] 
 an E[ X n ]
令X與Y為兩連續型隨機變數，證明E[X+Y] = E[X]+E[Y].
E[ XY ]  E[ X ]E[Y ]
Independence
E3. If random variables X1, . . ., Xn are independent, then


E  X i    E  X i 
 i 1  i 1
n
n
E3. X1
Example 11
··· Xn
 n
 n
E  X i    E  X i 
 i 1  i 1
令X與Y為兩獨立之連續型隨機變數，證明E[XY] = E[X]E[Y].
E[ XY ]  



 
  yf
 
 
 


Y
xyf ( x, y)dxdy
xyf X ( x) fY ( y)dxdy

( y) xf X ( x)dxdy


 E[ X ] yfY ( y)dy

 E[ X ]E[Y ]
E3. X1
A Question
··· Xn
 n
 n
E  X i    E  X i 
 i 1  i 1
令X與Y為兩獨立之連續型隨機變數，證明E[XY] = E[X]E[Y].
X
Y
E[ XY ]  E[ X ]E[Y ]
E[ XY ]  E[ X ]E[Y ]
Example 12 X  Y
X
1
Y
0
p ( x, y )
1/ 4
1
0
1/ 4
0
1
1/ 4
0 1
1/ 4
x
p X ( x)
1 0 1
1
4
1
2
1
4
y
1 0 1
pY ( y ) 14 12 14
E[ XY ]  0  E[ X ]E[Y ]
E[ X ]  0
E[Y ]  0
p(0, 0)  0  14  pX (0) pY (0)
A Question
E[ X  Y ]  E[ X ]  E[Y ]
?
Var[ X  Y ]  Var[ X ]  Var[Y ]
X
Y  E[ XY ]  E[ X ]E[Y ]
Define
Cov( X , Y )  E  X  E[ X ]Y  E[Y ] 
The Variance of Sum
Var[ X  Y ]


 E  X  E[ X ]  Y  E[Y ] 
 E  X  E[ X ]  Y  E[Y ]  2  X  E[ X ]Y  E[Y ]
E
 X  Y  E[ X  Y ]
2
2
2
2
2
2



 E  X  E[ X ]  E Y  E[Y ]   2E  X  E[ X ]Y  E[Y ]




 Var[ X ]  Var[Y ]  2Cov( X , Y )
The Variance of Sum
Var[ X  Y ]
 Var[ X ]  Var[Y ]  2Cov( X , Y )
Cov( X , Y )  E  X  E[ X ]Y  E[Y ] 
The Covariance
差積之期望值
Cov( X , Y )  E  X  E[ X ]Y  E[Y ] 
Cov( X , Y )  E[ XY ]  E[ X ]E[Y ]
The Covariance
Cov( X , Y )  E  X  E[ X ]Y  E[Y ] 
 E  XY  XE[Y ]  YE[ X ]  E[ X ]E[Y ]
 E[ XY ]  E[ X ]E[Y ]  E[ X ]E[Y ]  E[ X ]E[Y ]
 E[ XY ]  E[ X ]E[Y ]
Cov( X , Y )  E[ XY ]  E[ X ]E[Y ]
Example 13
X Y  E[ XY ]  E[ X ]E[Y ]
 Cov( X , Y )  E[ X ]E[Y ]  E[ X ]E[Y ]  0
Cov( X , Y )  E[ XY ]  E[ X ]E[Y ]
A Question
X Y
Cov( X , Y )  0
Cov( X , Y )  E[ XY ]  E[ X ]E[Y ]
Properties Related to Covariance
E4. X
Y  Cov( X , Y )  0
E5. Var[ X ]  Cov( X , X )  E[ X ]   E[ X ]
2
Cov( X , X )  E ( X  E[ X ])( X  E[ X ])
 E ( X  E[ X ]) 2 
 Var[ X ]
2
Cov( X , Y )  E[ XY ]  E[ X ]E[Y ]
Properties Related to Covariance
Y  Cov( X , Y )  0
E4. X
E5. Var[ X ]  Cov( X , X )  E[ X ]   E[ X ]
2
2
Fact: Var[aX ]  a Var[ X ]
2
2
2
2
2
2
2


 E  aX    E[aX ]  a E[ X ]  a  E[ X ]



 a E[ X ]   E[ X ]
2
2
2

Cov( X , Y )  E[ XY ]  E[ X ]E[Y ]
Properties Related to Covariance
E4. X
Y  Cov( X , Y )  0
E5. Var[ X ]  Cov( X , X )  E[ X ]   E[ X ]
2
2
E6. Cov( X , Y )  Cov(Y , X )
E7. X
X1
Y  Var[ X  Y ]  Var[ X ]  Var[Y ]
X n  Var[ X1 
 X n ]  Var[ X1 ] 
 Var[ X n ]
Example 14






m
m  1   x
m  1   x
E[ X ] 
x
x
e
dx

x
e dx


0
0
( )
( )
  (m   ) (m    1) (  1)


m 
( ) 
m
Example 14
E[ X ] 
m
(m    1)
m

E[ X ] 

Var[ X ] 
(  1)
E[ X ] 
2
(  1)
2
(  1)
2

 
   2
 
2
Cov( X , Y )  E[ XY ]  E[ X ]E[Y ]
More Properties on Covariance
E8.
m n
n
 m

Cov  i X i ,   jY j    i  j Cov( X i , Y j )
j 1
 i 1
 i 1 j 1
n
m


m
  n
 E    i X i   jY j   E    i X i  E    jY j 
j 1
 i 1
  j 1
 i 1


m n

 m
 n
  i  j E  X iY j    i E  X i      j E Y j  
i 1 j 1
 i 1
  j 1

m
n
m
n
 i  j E  X iY j   i  j E  X i  E Y j 
i 1 j 1
m
n
i 1 j 1

 i  j E  X iY j   E  X i  E Y j 
i 1 j 1

More Properties on Covariance
E8.
m n
n
 m

Cov  i X i ,   jY j    i  j Cov( X i , Y j )
j 1
 i 1
 i 1 j 1
E9.
 m
 m 2
Var    i X i   i Var[ X i ]  2i j Cov( X i , X j )
 i 1
 i 1
i j
m
 m
 m m
 Cov  i X i ,  j X j   i j Cov( X i , X j )
j 1
 i 1
 i 1 j 1
m
 i2Var[ X i ]  i j Cov( X i , X j )
i 1
i j
E[ X ]  2, E[Y ]  1, E[ Z ]  3,
Var[ X ]  4, Var[Y ]  1,Var[ Z ]  5,
Cov( X , Y )  2, Cov( X , Z )  0, Cov(Y , Z )  2
Example 16
U  3 X  2Y  Z
V  X  Y  2Z
E[U ]  ?
Var[U ]  ?
Cov(U , V )  ?
E[ X ]  2, E[Y ]  1, E[ Z ]  3,
Var[ X ]  4, Var[Y ]  1,Var[ Z ]  5,
Cov( X , Y )  2, Cov( X , Z )  0, Cov(Y , Z )  2
Example 16
U  3 X  2Y  Z
V  X  Y  2Z
E[U ]  ?
Var[U ]  ?
Cov(U , V )  ?
E[U ]  E[3 X  2Y  Z ]  3E[ X ]  2 E[Y ]  E[ Z ]
 623  7
Var[U ]  Var[3 X  2Y  Z ]
 9Var[ X ]  4Var[Y ]  Var[ Z ]
12Cov( X , Y )  4Cov(Y , Z )  6Cov( X , Z )
 9  4  4 1  5  12  (2)  4  2  6  0  61
E[ X ]  2, E[Y ]  1, E[ Z ]  3,
Var[ X ]  4, Var[Y ]  1,Var[ Z ]  5,
Cov( X , Y )  2, Cov( X , Z )  0, Cov(Y , Z )  2
Example 16
U  3 X  2Y  Z
V  X  Y  2Z
E[U ]  ?
Var[U ]  ?
Cov(U , V )  ?
Cov(U ,V )  Cov(3 X  2Y  Z , X  Y  2Z )
 3Cov( X , X )  3Cov( X , Y )  6Cov( X , Z )
 2Cov(Y , X )  2Cov(Y , Y )  4Cov(Y , Z )
+ Cov( Z , X )  Cov( Z , Y )  2Cov( Z , Z )
 3  4  3  (2)  6  0
 2  (2)  2 1  4  2
 0  2  25
8
Theorem 1 Schwartz Inequality
 E[ XY ]
2
 E[ X ]E[Y ]
2
2
 E[ XY ]
2
 E[ X ]E[Y ]
2
2
Theorem 1 Schwartz Inequality
Pf) 0  E  X  Y 2   E  X 2  2 XY   2Y 2 




 E[ X 2 ]  2 E[ XY ]   2 E[Y 2 ]
求ㄧ=*使E具有最小值
E
令 0
 2 E[ XY ]  2 E[Y 2 ]

E
E[ XY ]
* 
E[Y 2 ]
 E[ XY ]
2
 E[ X ]E[Y ]
2
2
Theorem 1 Schwartz Inequality
Pf) 0  E  X  Y 2   E  X 2  2 XY   2Y 2 




 E[ X 2 ]  2 E[ XY ]   2 E[Y 2 ]
E[ XY ]
* 
E[Y 2 ]
E
2
 E[ XY ] 
E[ XY ]
2
E
[
XY
]

E
[
Y
]
0  E *  E[ X ]  2

2
2 
E[Y ]
 E[Y ] 
2
 E[ X
2
E[ XY ]

] 2
2
E[Y ]
2
E[ XY ]


2
E[Y ]
2
 E[ X
2
E[ XY ]

]
E[Y 2 ]
2
 E[ XY ]
2
 E[ X ]E[Y ]
2
2
Theorem 1 Schwartz Inequality
 E[ XY ]
Pf) 0  E  X  Y 2  2 E  X 2  2 XY   2Y 2 




E[ XY ]
* 
22
E[Y ]
 E[ XE ]
2 ]   2 E[Y 2 ]
 E[ X 2 ]  2 E[ XY
E[Y ]
2
 E[ XY ] 
E[ XY ]
2
E
[
XY
]

E
[
Y
]
0  E *  E[ X ]  2

2
2 
E[Y ]
 E[Y ] 
2
 E[ X
2
E[ XY ]

] 2
2
E[Y ]
2
E[ XY ]


2
E[Y ]
2
 E[ X
2
E[ XY ]

]
E[Y 2 ]
2
 E[ XY ]
2
 E[ X ]E[Y ]
2
2
Corollary
2
Cov
(
X
,
Y
)
 Var[ X ]Var[Y ]
E10.
Pf)
2
2
E
[
(
X

E
[
X
])
]
)
(
Y

E
[
Y
]
])
)

E
[
(
X

E
[
X
]
)
]
E
[
(
Y

E
[
Y
]
)
]


2
Cov( X , Y )2  Var[ X ]Var[Y ]
Correlation Coefficient
 X ,Y
Cov( X , Y )

Var[ X ] Var[Y ]
E11. 1   X ,Y  1
Cov( X , Y )2  Var[ X ]Var[Y ]
Correlation Coefficient
 X ,Y
Cov( X , Y )

Var[ X ] Var[Y ]
Fact: X Y   X ,Y  0
E11. 1   X ,Y  1
 X ,Y
Cov( X , Y )

Var[ X ] Var[Y ]
E11.
1   X ,Y  1
Correlation Coefficient
E12. Y  aX  b   X ,Y
Pf)
1 a  0

1 a  0
Cov( X , Y )  Cov( X , aX  b)  aCov( X , X )  Cov( X , b)  aVar[ X ]
2
Var[Y ]  Var[aX ]  Var[b]  a Var[ X ]
0

a0
a Var[ X ]
Var[Y ]  

a Var[ X ] a  0
  X ,Y
0
1 a  0

1 a  0
X
Y
E[ X ]  1, E[Y ]  2,
Var[ X ]  3, Var[Y ]  4
Example 18
Var[3 X  2Y  1]  ?
Var[ XY ]  ?
 X Y , X Y  ?
X
Y
E[ X ]  1, E[Y ]  2,
Var[ X ]  3, Var[Y ]  4
Example 18
Var[3 X  2Y  1]  ?
Var[ XY ]  ?
 X Y , X Y  ?
Var[3 X  2Y  1]  9Var[ X ]  4Var[Y ]  27  16  43
Var[ XY ]  E[ X Y ]   E[ XY ]
2
2
2
 E[ X ]E[Y ]   E[ X ]E[Y ]
2
 Var[ X ]  E[ X ]
2
2
Var[Y ]  E[Y ]   E[ X ] E[Y ]
  3  12  4  22   12  22
 28
2
2
2
2
X
Y
E[ X ]  1, E[Y ]  2,
Var[ X ]  3, Var[Y ]  4
Example 18
 X Y , X Y
Var[3 X  2Y  1]  ?
Var[ XY ]  ?
 X Y , X Y  ?
Cov( X  Y , X  Y )
1


7
Var[ X  Y ] Var[ X  Y ]
Cov( X  Y , X  Y )  Cov( X , X )  Cov(Y , Y )  Var[ X ]  Var[Y ]  1
Var[ X  Y ]  Var[ X ]  Var[Y ]  7
Var[ X  Y ]  Var[ X ]  Var[Y ]  7
2 X: #
Y: #
Example 19
 X ,Y  ?
2 X: #
Y: #
Example 19
 X ,Y  ?
 X ,Y 
Method 1:
E[ X ]  0  101  1 53  2  103  65
E[Y ]  0  103  1 53  2  101  54
E[ X 2 ]  02  101  12  53  22  103  95
E[Y 2 ]  02  103  12  53  22  101  1
E[ XY ]  1 53  53
X
p(x, y)
Var[ X ]  E[ X 2 ]   E[ X ]  95  36

25
9
25
Var[Y ]  E[Y 2 ]   E[Y ]  1  16
25 
9
25
2
2
Cov( X , Y )
Var[ X ] Var[Y ]
9
Cov( X , Y )  E[ XY ]  E[ X ]E[Y ]  53  24


25
25
Y
0
1
2
0
1
2
0
0
0
1
10
3
5
3
10
0
0
0
 X ,Y  1
2 X: #
Y: #
Example 19
 X ,Y  ?
Method 2:
Facts:
X  Y  2  Y  X  2
Y  aX  b   X ,Y
1 a  0

1 a  0
 X ,Y  1
Chapter 5 Expectations
Conditional
Expectations
Definition  Conditional Expectations
E[Y | X  x]
 E[Y | x]  Y | x
 yi pY | X (yi | x) discrete case
 i
 
  yfY | X ( y | x)dy continuous case
 
Facts
E[Y | X  x]
 E[Y | x]  Y | x
 a function of X (x)
 yi pY | X (yi | x) discrete case
 i
 
  yfY | X ( y | x)dy continuous case
 
E13.
E  E[Y | X  x]  E[Y ]
See text for
the proof
E[Y | x]  Y |x
 yi pY | X (yi | x) discrete case

  i
  yfY | X ( y | x)dy continuous case
 
Conditional Variances
Var[Y | x]  
2
Y |x


 E Y  Y | x  x 


2

 E Y x   E Y x 
2

2
E[Y | x]  Y |x
Example 20
fY | X ( y | x ) 
f ( x, y )
f X ( x)
2(2 x  y )

, 0  x, y  1
1 4x
1
f X ( x)  23  (2 x  y)dy
0
 (1  4 x), 0  x  1
1
3
 yi pY | X (yi | x) discrete case

  i
  yfY | X ( y | x)dy continuous case
 

Var[Y | x]  E Y x   E Y x 
2
fY | X ( y | 0.5)  23 ( y  1), 0  y  1
E[Y | 0.5] 
2
3

1
0
y( y  1)dy  95
1
E[Y 2 | 0.5]  23  y 2 ( y  1)dy  187
0
13
Var[Y | 0.5]  187   95   162
2

2
Chapter 5 Expectations
Moment
Generating
Functions
動差母函數
Moment Generating Functions
Moments
Moments
Moment Generating Functions
The moment generating function MX(t) of a
random variable X is defined by
M X (t )  E e 
Xt
The domain of MX(t) is all real numbers such that eXt has finite expectation.
M X (t )  E e 
Xt
Example 21
X ~ B(n, p), M X (t )  ?
 n  x n x
p X ( x)    p q , x  0,1,
 x
,n
q  1 p
n
n
 n  x n x
n
t x n x
t
M X (t )   e   p q      pe  q   pe  q 
x 0
x 0  x 
 x
n
xt
M X (t )  E e 
Xt
Example 22
X ~ ( ,  ), M X (t )  ?
  x 1e x
f X ( x) 
, x0
( )
   xt  1  x
    1  x (  t )
M X (t ) 
e x e dx 
x e
dx


( ) 0
( ) 0


  ( )
t
     t  




1


 
 

( ) (  t )




t
 


 

Summary of Important Moments of
Random Variables
為何MX(t) 會生動差?
Moment Generating Functions
The moment generating function MX(t) of a
random variable X is defined by
M X (t )  E e 
Xt
The domain of MX(t) is all real numbers such that eXt has finite expectation.
Moment Generating Functions
Xt

M X (t )  E e 
 X
X2 2 X3 3
 E 1  t 
t 
t 
2!
3!
 1!
 1
EX 



E  X  2 E  X  3
t
t 
t 
1!
2!
3!
2
3
Moment Generating Functions
k
2
1
E  X k   ?
0
M X (t )  1 
E[ X 0 ]
0!
EX 
E[ X 1 ]
1!
E[ X 2 ]
2!
E[ X k ]
k!
E  X  2 E  X  3
t
t 
t 
1!
2!
3!
2
3
Moment Generating Functions
k
2
1
E  X k   ?
0
M X (t )  1 
E[ X 0 ]
0!
EX 
E[ X 1 ]
1!
E[ X 2 ]
2!
E[ X k ]
k!
E  X  2 E  X  3
t
t 
t 
1!
2!
3!
2
3
Moment Generating Functions
M X (t )  1 
EX 
1!
M X (t )  E  X  
t
E  X 2 
2!
E  X 2 
M X (t )  E  X 2  
1!
t
E  X 3 
1!
t2 
E  X 3 
E  X 3 
2!
t2 
E  X   M X (0)
E  X 2   M X (0)
t
E  X   M
k
3!
t3 
(k )
X
(0)
Example 23
Using MGF to find the
means and variances of
X ~ B(n, p )
Y ~ ( ,  )
Using MGF to find the
means and variances of
Example 23
X ~ B(n, p )
Y ~ ( ,  )
Using MGF to find the
means and variances of
Example 23
X ~ B(n, p )
Y ~ ( ,  )
Using MGF to find the
means and variances of
Example 23
X ~ B(n, p )
Y ~ ( ,  )
Using MGF to find the
means and variances of
Example 23
X ~ B(n, p )
Y ~ ( ,  )
Correspondence or Uniqueness Theorem
Let X1, X2 be two random variables.
M X1 (t )  M X 2 (t )
FX1 ( x)  FX 2 ( x)
f X1 ( x )  f X 2 ( x )
Example 24
Example 24
X
B(n, p)  M X (t )   pe  q 
t
X
B(5, 0.7)
E[ X ]  np  3.5
Var[ X ]  npq  1.05
n
Example 24
X
P( )  M X (t )  e
Y
P(4)
E[Y ]  4
Var[Y ]  4

 1 et

Example 24
X
pet
G( p)  M X (t ) 
1  qet
Z
G (0.3)
E[ Z ]  10 / 3
Var[ Z ]  70 / 9
Example 24
X
 t  2t 2 / 2
N (  ,  )  M X (t )  e e
2
W N (2, 22 )
E[W ]  2
Var[W ]  22
Theorem  Linear Translation
Y  aX  b  M Y (t )  ebt M X (at )
Pf)
M Y (t )  E eYt   E e( aX b )t 
 E e e
bt
X ( at )
 ebt M X (at )
  e E e
bt
X ( at )

Theorem  Convolution
X1 X 2
M X1  X 2 
Pf)
M X1  X 2 
...
 Xn
Xn
(t )  M X1 (t ) M X 2 (t )
( X1  X 2 

(
t
)

E
e
 Xn

X1t
X 2t



 E e  E e 
 M X1 (t ) M X 2 (t )
X nt

E e 
M X n (t )
 X n )t

M X n (t )
X1 X 2
Example 25
...
 1 
X i ~ Exp  
Xn
 i 
n
Xi
Z  2nY / 
Y 
i 1 ni
E[Y ]  ?
Z ~?
X1 X 2
Example 25
...
 1 
X i ~ Exp  
Xn
 i 
n
Xi
Z  2nY / 
Y 
i 1 ni
E[Y ]  ?
Z ~?
E  X i   i
n
n
X
1
X

 i
i 
E[Y ]  E      E     E  X i 
 i 1 ni  i 1  ni  i 1 ni
n
n

i 1
n


X1 X 2
Example 25
...
 1 
X i ~ Exp  
Xn
 i 
n
Xi
Z  2nY / 
Y 
i 1 ni
E[Y ]  ?
Z ~?
M Z (t )  ?M 2 nY /  (t )  ?M Y  2n
 t
n
n
M Y (t )  
? M X i (t )  ? M X i  ni1 t 
i 1
i 1
ni
Y  aX  b  M Y (t )  ebt M X (at )
X1 X 2
M X1  X 2 
...
 Xn
Xn
(t )  M X1 (t ) M X 2 (t )
M X n (t )
X1 X 2
...
Example 25
 1 
X i ~ Exp  
Xn
 i 
n
Xi
Z  2nY / 
Y 
i 1 ni
E[Y ]  ?
Z ~?
M Z (t )  ?M 2 nY /  (t )  ?M Y 
n
2n


t  1  2t 
n
M Y (t )  
? M X i (t )  ? M X i  ni1 t 
i 1
n
ni
i 1
  1   t / n   1   t / n 
i 1
M X i (t )  ?(1  i t )1
1
n
n
X1 X 2
...
Example 25
 1 
X i ~ Exp  
Xn
 i 
n
Xi
Z  2nY / 
Y 
i 1 ni
E[Y ]  ?
Z ~?
M Z (t )  ?M 2 nY /  (t )  ?M Y 
2n


t  1  2t 
Z ~   n,
Z~
2
2n
n
1
2

(2k )!
E[ X ]  k
2 k!
2k
Example 26
E[ X 2 k 1 ]  0
k  0,1, 2,
X ~?
(2k )!
E[ X ]  k
2 k!
2k
Example 26
M X (t )  E e Xt   1 

E  X k 
k 0
k!
M X (t )  
EX 
1!
t
2!
E  X 2 k 
k 0
(2k )!
k  0,1, 2,
X ~?
E  X 2 

tk  
E[ X 2 k 1 ]  0
t2 
E  X 3 

t 2k  
k 0
3!
t3 
E  X 2 k 1 
(2k  1)!
0
t 2 k 1
(2k )!
E[ X ]  k
2 k!
2k
E[ X 2 k 1 ]  0
Example 26
M X (t )  E e Xt   1 
EX 
1!
t
k  0,1, 2,
X ~?
E  X 2 
2!
t2 
E  X 3 
t3 
3!
(2k )!2 k 1
k
2
k
 E X 
 E X
 2 k  E2kkX! 2 k  2 k 1



k

t t
M X (t )  
t 
t 
k!
(2k )!
(2kk)! 1)!
k 0
k 0
kk00 (2
1 2k
1 t 
 k t   
k 0 2 k !
k 0 k !  2 


X ~ N (0,1)
2
k
e
t2 / 2
0
Theorem of Random Variables’ Sum
Theorem of Random Variables’ Sum


We have proved the above five using probability
generating functions.
They can also be proved using moment generating
functions.
Theorem of Random Variables’ Sum
X1  X 2 
 X n ~  ( n,  ) ?
表何意義?
Theorem of Random Variables’ Sum
M X i (t )  1   
t
M X1  X 2 
 Xn
X1  X 2 
1
(t )  ?1   
t
n
 X n ~ ?(n,  )
Theorem of Random Variables’ Sum
X1  X 2 
 X n ~ (1 
表何意義?
 n , ) ?
Theorem of Random Variables’ Sum
X1  X 2 
? (1 
 Xn ~ 
M X i (t )  1   
t
M X1  X 2 
 Xn
 n ,  )
 i
(t )  ?1   
t
 (1   n )
Theorem of Random Variables’ Sum
X i ~ ( 12 , 12 )  X1 
 X n ~ ( n2 , 12 )
12
 n2
X i ~ ( , )  X1 
mi
2
1
2
m2
i
 X n ~ (
m1   mn
2
m2 
1
 mn
, 12 )
Theorem of Random Variables’ Sum
X1  X 2 
 Xn
表何意義?
Theorem of Random Variables’ Sum
X1  X 2 
 Xn
~ N?(1 
 n ,  12 
  n2 )
i t  i2t 2 / 2
M X i (t )  e e
M X1  X 2 
 Xn
(t )  ?e
( 1   n ) t (12   n2 ) t 2 / 2
e
Theorem of Random Variables’ Sum
Chapter 5 Expectations
Inequalities
Theorem  Markov Inequality
Let X be a nonnegative random variable with E[X] = .
Then, for any t > 0,
P( X  t ) 

t
僅知一次動差對機率値之評估
I(X )  0
E[ X ]  
P( X  t ) 

t
t 0
Theorem  Markov Inequality
0
Define Y  
t
X t
X t
 P( X  t )
 pY ( y )  
 P( X  t )
A discrete random variable
y0
y t
E[Y ]  0  P( X  t )  t  P( X  t )  E[ X ]
Why?
I(X )  0
E[ X ]  
P( X  t ) 

t
t 0
Theorem  Markov Inequality
0
Define Y  
t
X t
X t
 P( X  t )
 pY ( y )  
 P( X  t )
A discrete random variable
y0
y t
E[Y ]  0  P( X  t )  t  P( X  t )  E[ X ]  
I(X )  0
E[ X ]  
Example 27
P( X  t ) 

t
t 0
MTTF  Mean Time To Failure
I(X )  0
E[ X ]  
Example 27
P( X  t ) 

t
t 0
MTTF  Mean Time To Failure
E[ X ]  100
E[ X ]  1  100  X ~ Exp(0.01)
東方不敗，但精確性差
By Markov
P( X
P( X
P( X
P( X
 90)  100 / 90  1.11
 100)  100 /100  1
 110)  100 /110  0.9
 200)  100 / 200  0.5
By Exponential Distribution
P( X
P( X
P( X
P( X
 90)  e0.9  0.40657
 100)  e1  0.36788
 110)  e1.1  0.33287
 200)  e2  0.13534
I(X )  0
E[ X ]  
P( X  t ) 

t
t 0
Theorem  Chebyshev's Inequality
P  X  X  t  

t
2
X
2
知一次與二次動差對機率値之評估
I(X )  0
E[ X ]  
P( X  t ) 

t
t 0
Theorem  Chebyshev's Inequality
P  X  X  t  
2

P  X  X  t   P  X  X 


t
2
X
2
2

E  X  X  


2

t
2

t
P  X  X  t  

t
2
X
2
Theorem  Chebyshev's Inequality
1
P  X   X  k X   2
k
1
P  X   X  k X   1  2
k
Facts:
X
X
X
 0.934
 0.889
 0.75
0
X
X
X
X
X
X
X ~ N (  , P ) X   X  t  
2

t
2
X
2
Theorem  Chebyshev's Inequality
1
P  X   X  k X   2
k
1
P  X   X  k X   1  2
k
Facts:
X
X
X
 0.934
 0.889
 0.75
0
X
X
X
X
X
X
Example 28
P  X  X
1
 k X   2
k
Example 28
此君必然上榜
  60
 6
P  X   X  4 X   1/16
X  84
X    24  4



1000  161  62.5  70






Chapter 5 Expectations
The Weak Law of Large Numbers
and
Central Limit Theorems
The Parameters of a Population
X

A population

2
We may never have the chance to
know the values of parameters in a
population exactly.
E[ X i ]  
Var[ X i ]  
2
Sample Mean
iid: identical independent distributions
X

A population

iid random variables
X1 ,
, Xn
Sample Mean
2
X
1
n
 X1 
 Xn 
E[ X i ]  
Var[ X i ]  
2
Expectation & Variance of X
E  X   E  1n  X 1 
X


A population

1
n
EX 
1
 X n  
 E  X n 

Var  X   Var  1n  X 1 

1
n2
Var  X  
1
 X n  
 Var  X n 
2 /n
2
X
1
n
 X1 
 Xn 
X  1n  X1 
 Xn 
Expectation & Variance of X
X

E  X   
A population

Var  X    / n
2
X
2


X
X  1n  X1 
 Xn 
Expectation & Variance of X
X

E  X   
A population

Var  X    / n
2
X
2


X
Theorem 
Weak Law of Large Numbers
Let X1, …, Xn be iid random variables having
finite mean .
X

1
n
 X1 
 Xn 

lim P X      0, for any   0.
n 
Chebyshev's Inequality
P  X  X  t  
Theorem 
Weak Law of Large Numbers
 X2
t2
Let X1, …, Xn be iid
 X2random variables having
P X  X    2
finite mean .

2
P X    2 .
n
 X   ,  X2   2 / n


X


1
n
 X1 

 Xn 

lim P X      0, for any   0.
n 
Central Limit Theorem
Let X1, …, Xn be iid random variables having
finite mean  and finite nonzero variance 2.
Sn  X 1 
 X n , X  1n Sn
1. lim S n ~ N (n , n 2 )
n 
3. lim X ~ N (  ,  2 / n)
n 
S n  n
X 
2. lim
~ N (0,1) 4. lim
~ N (0,1)
n 
n   /
n
n
X1
Xn
E[ X i ]   , Var[ X i ]   2
X 
lim
~ N (0,1)
n   /
n
Central Limit Theorem
Let X1, …, Xn be iid random variables having
finite mean  and finite nonzero variance 2.
Sn  X 1 
 X n , X  1n Sn
1. lim S n ~ N (n , n 2 )
n 
3. lim X ~ N (  ,  2 / n)
n 
S n  n
X 
2. lim
~ N (0,1) 4. lim
~ N (0,1)
n 
n   /
n
n
X1
Xn
E[ X i ]   , Var[ X i ]   2
X 
lim
~ N (0,1)
n   /
n
Central Limit Theorem
X 
Define Z n 

/ n
M Z n (t )  ?M
 Yi
 n Xi n  
n  n  n 
n
Xi  
1 n
i 1
 i 1
 1

Yi



n i 1 
n i 1
t / n 
n
M
MY t 

t

?


Y
i 1
i
i
E[Yi ]
E[Yi 2 ] 2 E[Yi 3 ] 3
t
t 
t 
M Yi  t   1? 
1!
2!
3!
X 
E[Yi ]  E  i
 0



2

X


 i
 
2
E[Yi ]  E 
  1
   
1 2 E[Yi 3 ] 3
 1 t 
t 
2!
3!
X1
X 
lim
~ N (0,1)
n   /
n
Xn
E[ X i ]   , Var[ X i ]   2
Central Limit Theorem
 n Xi n  
n  n  n 
n
Xi  
1 n
i 1
 i 1
 1

Yi



n i 1 
n i 1
X 
Define Z n 

/ n
M Z n (t )  ?M
 Yi


1 t 2 E[Yi 3 ] t 3


t / n  1 
3/ 2
3! n
 2! n


1 2 E[Yi ] 3
M
t



1

t 
t 
M Y  t   ? Yi

i

3!
i 1
 2!
n
3
E[Yi ]
E[Yi 2 ] 2 E[Yi 3 ] 3
t
t 
t 
M Yi  t   1? 
1!
2!
3!






n
n
1 2 E[Yi 3 ] 3
 1 t 
t 
2!
3!
X1
Xn
E[ X i ]   , Var[ X i ]   2
X 
lim
~ N (0,1)
n   /
n
Central Limit Theorem
X 
Define Z n 

/ n
 n Xi n  
n  n  n 
n
Xi  
1 n
i 1
 i 1
 1

Yi



n i 1 
n i 1

E[Yi ] t
1 t
M Z n (t )  ?1 


3/ 2
3! n
 2! n
2
3
3
=0 as n



n

1 ?
X1
Xn
E[ X i ]   , Var[ X i ]   2
X 
lim
~ N (0,1)
n   /
n
Central Limit Theorem
X 
Define Z n 

/ n
 n Xi n  
n  n  n 
n
Xi  
1 n
i 1
 i 1
 1

Yi



n i 1 
n i 1

E[Yi ] t
1 t
M Z n (t )  ?1 


3/ 2
3! n
 2! n
2
3
3




1 t 2 E[Yi 3 ] t 3
ln M Zn (t )  n ln 1 


3/ 2
3! n
 2! n
n
e
ln M Zn ( t )

1 t 2 E[Yi 3 ] t 3
ln 1 


3/ 2

3! n
 2! n


n 1

當時n分子分母均趨近0



X1
Xn
E[ X i ]   , Var[ X i ]   2
X 
lim
~ N (0,1)
n   /
n
Central Limit Theorem
X 
Define Z n 

/ n
 n Xi n  
n  n  n 
n
Xi  
1 n
i 1
 i 1
 1

Yi



n i 1 
n i 1
n
e
ln M Zn ( t )


E[Yi ] t
1 t
M Z n (t )  ?1 

 
3/ 2
3! n
 2! n


 分子分母均
1 t 2 E[Yi 3 ] t 3
ln 1 


 對n微分一次
3/ 2
2!
n
3!
n

lim ln M Z n (t )  lim 
n 
n 
n 1
 1 2 2 3 E[Yi 3 ] 5/ 2 3

1

n
t

n
t



1 t 2 E [Yi3 ] t 3
2!
2
3!
1




2! n
3! n3 / 2
2
 lim

t
/2

2
n 
n
2
3
3
X1
Xn
E[ X i ]   , Var[ X i ]   2
X 
lim
~ N (0,1)
n   /
n
Central Limit Theorem
X 
Define Z n 

/ n
 n Xi n  
n  n  n 
n
Xi  
1 n
i 1
 i 1
 1

Yi



n i 1 
n i 1

E[Yi ] t
1 t
M Z n (t )  ?1 


3/ 2
3! n
 2! n
2
lim ln M Z n (t )  t 2 / 2
n 
3
3



n
e
ln M Zn ( t )
lim M Zn (t )  e
n 
t2 / 2
X1
Xn
E[ X i ]   , Var[ X i ]   2
X 
lim
~ N (0,1)
n   /
n
Central Limit Theorem
X 
Define Z n 

/ n
 n Xi n  
n  n  n 
n
Xi  
1 n
i 1
 i 1
 1

Yi



n i 1 
n i 1

E[Yi ] t
1 t
M Z n (t )  ?1 


3/ 2
3! n
 2! n
2
3
3



n
e
ln M Zn ( t )
lim Z n lim
~ MN ((0,1)
t)  e
n 
lim ln M Z n (t )  t 2 / 2
n 
t2 / 2
n 
Zn
Central Limit Theorem
Let X1, …, Xn be iid random variables having
finite mean  and finite nonzero variance 2.
Sn  X 1 
 X n , X  1n Sn
1. lim S n ~ N (n , n 2 )
n 
3. lim X ~ N (  ,  2 / n)
n 
S n  n
X 
2. lim
~ N (0,1) 4. lim
~ N (0,1)
n 
n   /
n
n
Normal Approximation
By the central limit theorem, when a sample
size is sufficiently large (n > 30), we can use
normal distribution to approximate certain
probabilities regarding to the sample or the
parameters of its corresponding population.
E[ X i ]  10
Var[ X i ]  100
Example 29
Let Xi represent the lifetime of ith bulb
We want to find
X1 
P( X 1 
 X 50
P( X 1 
X i ~ Exp(0.1)
 X 50  365)  ?
(50, 0.1)  N (500,5000)
n > 30
 365  500 
 X 50  365)   
  (1.91)  0.028
 5000 
E[ X i ]  0.3
Example 30
Var[ X i ]  0.21
Let X ~ B(50, 0.3). P( X  20)  ?
Let X i ~ B(1, 0.3), i  1,
 X  X1 
,50 are iid
 X 50 ~ B(50, 0.3)  N (15,10.5)
n > 30
 20.5  15 
P( X  20)   
   1.6973  0.95514
 10.5 
Example 30
 20.5  15 
P( X  20)   

10.
5


Let X ~ B(50, 0.3). P( X  20)  ?
0.14
X ~ B(50, 0.3)
0.12
0.1
 N (15, 10.5)
0.08
0.06
20.5
0.04
0.02
0
20