2-1 Review of Fundamentals of Probabilities
Random Variables and Distributions
Random variables:
將隨機試驗的結果，透過隨機變數轉換成數字形式
Probability Distributions
1. Probability mass function, pmf
機率質量函數
P (X = xi ) = pi
用於離散型隨機變數，例如：次數、人數
每點機率皆為0~1, 機率和＝1
機率密度函數
1. Probability density function, pdf
P (a < X < b)
單點沒有機率，是計算某區間的機率
2. Cumulative distribution function, cdf累積分配函數
F (x) = P (X ≤ x)
若F是連續函數，則X是連續型隨機變數。
若F是「跳躍」的，則X是離散型隨機變數
因為是累積分配函數，所以F is nondecreasing
分量數
Quantiles
τ = F (q(τ )) → q(τ ) = F −1 (τ )
0<τ <1
。例如：第0.5個分量數＝第50百分位數，q(0.9)就是第九十百分位數
分量數：從機率找q值
cdf: 從q找機率
Joint, Marginal and Conditional Distributions
2-1 Review of Fundamentals of Probabilities
1
if X and Y are discrete random variables, the joint pdf: fXY (x, y) = P (X = x, Y = y)
固定x的值）:
marginal pdf of x(
fX (x) = {
∑ P (X = x, Y = y) for discrete X and Y
∫ fXY (x, y)dy
for continuous X and Y
不能從Ｘ和Y各自的邊際pdf得知聯合pdf
Independence
⊥⊥
if X and Y are independent (X ⊥
⊥ Y ) then,
fXY (x, y) = fX (x) ∗ fY (y)
Conditional distribution
(x,y)
fY ∣X (y∣x) = fXY
fX (x)
if X and Y are independent, then fY ∣X (y∣x) = fY (y)
Summarizing Distributions
Expectation
⊥ Y , E[XY]=E[X]E[Y]
if X ⊥
推導：
E[XY ] = ∑ xy ∗ P (X = x, Y = y)
= ∑ xP (X = x) ∗ yP (Y = y)
= ∑ E[Y ]xP (X = x)
= E[Y ] ∑ xP (X = x) = E[X]E[Y ]
Jensen’s inequality
E[g(X)] > g(E[X]), if g is convex
E[g(X)] = g(E[X]), if g is linear function
E[g(X)] < g(E[X]), if g is concave
2-1 Review of Fundamentals of Probabilities
2
Variation
population variance:
2
σX
= Var[X] = E[(X − μX )2 ] = E[X 2 − 2Xμx + μ2X ] = E[X 2 ] − 2μ2X + μ2X
= E[X 2 ] − μ2X = E[X 2 ] − E[X]2
population standard deviation:
sd[X] =
2
σX
Var[X] =
Properties of Variance
Var[X] = E[X 2 ] − E[X]2
，常數的變異數為零
Var[aX + b] = a2 Var[X]
sd[aX + b] = |a| sd[X]
Var[aX + bY ] = a2 Var[X] + b2 Var[Y ] + 2abCov[X, Y ]
Standardization
A random variable X has E[X] = μ, Var[X] = σ 2
Standardizing X:
Z=
X −μ
→ E[Z] = 0, Var[Z] = 1
σ
，離差和為零
E[X − μ]
1
1
Var[ X−μ
σ ] = σ 2 Var[X − μ] = σ 2 Var[X] = 1
Covariance and Correlation
1. Covariance of X and Y:
σXY = COV [X, Y ] = E[(X − μX )(Y − μY )] = E[XY − XμY − Y μX + μX μY ] = E[XY ] − 2μX μY + μX μY
= E[XY ] − μX μY = E[XY ] − E[X]E[Y ]
2-1 Review of Fundamentals of Probabilities
3
if X and Y are independent, E[XY]=E[X]E[Y], COV[X, Y]=0
Properties of covariance and Correlation:
Cov[X, X] = Var[X]
Cov[X, Y ] = Cov[Y , X]
Cov[X, Y ] = E[XY ] − E[X]E[Y ]
Cov[aX + b, cY + d] = acCov[X, Y ]
if X and Y are independent, Cov[X, Y ] = 0
2. Correlation coefficient
COV [X,Y ]
，會介於-1~1之間
ρXY = sd[X]sd[Y ] = σσXXYσY
在相關係數中，隨機變數的放大或縮小不影響其值大小，僅對正負號有影響
3. Slope of the regression line
Y = a + bX
COV [X, Y ] = COV [X, a + bX] = COV [X, a] + COV [X, bX] = bCOV [X, X] = bVar[X]
Cov[X,Y ]
→ b = Var[X]
Linear dependence
if X and Y are independent, then covariance equal 0
X ⊥⊥ Y → E[XY ] = E[X]E[Y ] → Cov[X, Y ] = 0
但是 共變異數為零，不一定代表兩變數獨立： Cov[X, Y ] ⇏ X ⊥⊥ Y ，僅代表沒有線性關係，不代表沒有非線性關係
2-1 Review of Fundamentals of Probabilities
4
Var[∑ Xi ] = ∑ Var[Xi ]
Conditional Expectations
Discrete Y: E[Y ∣X = x] = ∑ yfY ∣X (y∣x)
Continuous Y: E[Y ∣X = x] = ∫ yfY ∣X (y∣x)dy
條件期望值也是一個隨機變數
Properties:
CE.1: E[c(X)∣X] = c(X)
CE.2: E[a(X)Y + b(X)∣X] = a(X)E[Y ∣X] + b(X)
CE.3: if X ⊥
⊥ Y , then E[Y ∣X] = E[Y ]
證明：
E[Y ∣X] = ∑ y ∗ fY ∣X (y∣x) = ∑ y ∗ fY (y) = E[Y ]
CE.4: Law of Iterated Expectation
E[E[Y ∣X] = E[Y ]
CE.5:
Conditional Variance
Var[Y ∣X = x] = E[(Y − E[Y ∣X = x])2 ]
= E[Y 2 − 2Y E[Y ∣X = x] + E[Y ∣X = x]2 ∣X = x] = E[Y 2 ∣X = x] − 2E[Y ∣X = x]2 + E[Y ∣X = x]2
= E[Y 2 ∣X = x] − (E[Y ∣X = x])2
if X and Y are independent, we know E[Y ∣X] = E[Y ], then Var[Y ∣X = x] = E[Y 2 ] − E[Y ]2 = Var[Y ]
若兩變數獨立，代表Y的條件變異數，與X無關，仍為自己的變異數
The Normal and Related Distributions
後面10頁是各種分配，先略過
If X
～N (μ, σ ), then aX + b～N (aμ + b, a σ )
2
2 2
If X and Y are jointly normally distributed, then
X ⊥⊥ Y ⇔ Cov[X, Y ] = 0
互相獨立且符合常態分配的隨機變數，其線性組合也會符合常態分配
2-1 Review of Fundamentals of Probabilities
5
2-1 Review of Fundamentals of Probabilities
6
