ECE 226: Master Final Exam Cheat
Sheet
Outer x bounds: 0to 1. Inner
y bounds: 0 to x/2.
R 1 R x/2
P (Y < X/2) = 0 0 6y dy dx.
With Guided Solving Mini-Examples for High-Yield Topics
5. Covariance & Correlation
1. Probability Basics & Counting
Covariance: Cov(X, Y ) = E[XY ] − E[X]E[Y ]
)
Correlation: ρ = Cov(X,Y
(−1 ≤ ρ ≤ 1)
σX σY
Var of Sums: V ar(X + Y ) = V ar(X) + V ar(Y ) +
2Cov(X, Y )
De Morgan’s Laws: (A ∪ B)c = Ac ∩ B c and (A ∩ B)c =
Ac ∪ B c
Inclusion-Exclusion: P (A∪B) = P (A)+P (B)−P (A∩B)
Conditional Prob: P (A|B) =
PP (A ∩ B)/P (B)
Law of Total Prob: P (A) =
P (A|Bi )P (Bi )
i )P (Bi )
Bayes’ Rule: P (Bi |A) = PPP(A|B
(A|Bk )P (Bk )
Independence: A, B indep
⇐⇒
P (A ∩ B) =
P (A)P (B).
n!
Combinations (Unordered, No rep): nk = k!(n−k)!
n!
Permutations (Ordered, No rep): Pkn = (n−k)!
MINI-EXAMPLE: Covariance Properties
Rules: Cov(X, X) = V ar(X). Cov(X, constant) = 0.
Linearity: Cov(aX + b, cY + d) = ac · Cov(X, Y ).
Cov(X, Y + Z) = Cov(X, Y ) + Cov(X, Z).
6. Conditional Expectation
2. Random Variables (RVs) & Expectation
(x,y)
Conditional PDF: fX|Y (x|y) = fXY
fY (y) .
Law of Iterated Expectations: E[X] = E[E[X|Y ]].
CDF: FX (x) = P (X ≤ x). Limits:
F (−∞)
R = 0, F (∞) = 1.
P
Expected Value: E[X]
=
xp
(x)
or
xfX (x)dx
X
R
P
LOTUS: E[g(X)] =
g(x)pX (x) or g(x)fX (x)dx
Variance: V ar(X) = E[(X − E[X])2 ] = E[X 2 ] − (E[X])2
Linearity: E[aX + b] = aE[X] + b
Var Scaling:
V ar(aX + b) = a2 V ar(X)
7. Bivariate Normal Distribution
If X and Y are jointly normal:
• Any linear combo aX + bY is also normally distributed.
• Uncorrelated (Cov = 0) ⇐⇒ Independent. (Only
true here!)
• Conditional Exp: E[X|Y = y] = µX + ρ σσX
(y − µY )
Y
2
• Conditional Var: V ar(X|Y = y) = σX (1 − ρ2 )
3. Common Distributions
Binomial(n,p): P (X = k) = nk pk (1 − p)n−k .
µ = np, σ 2 = np(1 − p).
Geometric(p): P (X = k) = (1 − p)k−1 p. µ = 1/p, σ 2 =
1−p
p2 .
MINI-EXAMPLE: Bivariate Normal Tricks
Problem: X, Y jointly normal. µX = µY = 0, V ar(X) =
1. X is independent of X + Y . Find Cov(X, Y ).
Solution: Because they are jointly normal, independent
means covariance is 0. So, Cov(X, X + Y ) = 0.
Expand using linearity: Cov(X, X) + Cov(X, Y ) = 0.
Substitute Cov(X, X) = V ar(X) = 1.
1 + Cov(X, Y ) = 0 =⇒ Cov(X, Y ) = −1.
k −λ
e
Poisson(λ): P (X = k) = λ k!
. µ = λ, σ 2 = λ.
2
(b−a)
1
2
Uniform(a,b): f (x) = b−a
. µ = a+b
2 ,σ =
12 .
−λx
2
Exponential(λ): f (x) = λe
. µ = 1/λ, σ = 1/λ2 .
– Memoryless: P (X > s + t | X > s) = P (X > t)
Normal(µ, σ 2 ): Z = X−µ
∼ N (0, 1). FX (x) = Φ x−µ
.
σ
σ
8. Limit Theorems & Continuity Correction
4. Joint Distributions & Marginalization
P
Sample Mean: X̄n = n1
Xi . µX̄ = µ, V ar(X̄) = σ 2 /n.
WLLN: X̄n converges
P in probability to µ.
2
CLT: Sum Sn =
Xi is approx N (µsum = nµ, σsum
=
2
nσ ).
−nµ
Standardize to use tables: Z = Sσn √
.
n
Joint CDF Rule: F (y1 , z1 ) + F (y2 , z2 ) ≥ F (y1 , z2 ) +
F (y2 , z1 )
Independence: fXY (x, y) = fX (x)fY (y) for all x, y.
MINI-EXAMPLE: Finding Marginals & Bounds
Problem: fXY (x, y) = cy for 0 ≤ y ≤ x ≤ 1. Find fX (x).
1. Find Bounds: We want to eliminate y. Look at the
inequality 0 ≤ y ≤ x. The floor for
h y2 is
ix0, the ceiling is x.
Rx
cy
2. Integrate: fX (x) = 0 cy dy = 2
= 2c x2 .
MINI-EXAMPLE: CLT with Continuity Correction
Problem: K48 is binomial with n = 48, p = 3/4. Approximate P (30 ≤ K48 ≤ 42) with continuity correction.
1. Find Mean/Var: µ = 48(3/4) = 36. σ 2 = 48(3/4)(1/4) =
9. σ = 3.
2. Apply Correction: Stretch bounds by 0.5 outwards!
P (29.5 ≤ N ≤ 42.5).
3. Standardize: Zlow = 29.5−36
= −2.17. Zhigh =
3
0
3. Find c: The total probability must be 1. Integrate the
marginal over all possible X (0 ≤ x ≤ 1).
R1
1 = 0 2c x2 dx = 6c [x3 ]10 = c/6 =⇒ c = 6.
4. Prob of a Region (Y < X/2): Double integral!
1
42.5−36
= 2.17.
3
4. Table Lookup: Φ(2.17) − Φ(−2.17).
9. Probability Bounds (Inequalities)
Markov: If X ≥ 0, a > 0 =⇒ P (X ≥ a) ≤ E[X]
a .
Chebyshev: For a > 0 =⇒ P (|X − µ| ≥ a) ≤ V ar(X)
.
a2
Chernoff: P (X ≥ a) ≤ mins>0 e−sa MX (s) (where
MX (s) = E[esX ]).
p
Cauchy-Schwarz: |E[XY ]| ≤ E[X 2 ]E[Y 2 ].
Jensen’s: If g(x) is convex (e.g. x2 ), E[g(X)] ≥ g(E[X]).
10. Random & Poisson Processes
Mean Func: µX (t) = E[X(t)] (A deterministic function
of t).
Autocorrelation: RX (t1 , t2 ) = E[X(t1 )X(t2 )].
Autocovariance:
CX (t1 , t2 )
=
RX (t1 , t2 ) −
µX (t1 )µX (t2 ).
Poisson Process N (t) (rate λ):
• Has independent, stationary increments.
• Arrivals in interval of length τ follows P oisson(λτ ).
• Expected arrivals in interval τ is λτ .
• Variance of arrivals in interval τ is λτ .
2