CS 70
Cheatsheet
Note 1 (Propositional Logic)
•
•
•
•
𝑃 =⇒ 𝑄 ≡ ¬𝑃 ∨ 𝑄
𝑃 =⇒ 𝑄 ≡ ¬𝑄 =⇒ ¬𝑃
¬(𝑃 ∧ 𝑄) ≡ ¬𝑃 ∨ ¬𝑄
¬(𝑃 ∨ 𝑄) ≡ ¬𝑃 ∧ ¬𝑄
Note 7 (Modular Artithmetic)
•
•
•
•
• 𝑥 −1 (modular inverse) exists mod 𝑚 if and only if gcd(𝑥, 𝑚) = 1
𝑃 ∧ (𝑄 ∨ 𝑅) ≡ (𝑃 ∧ 𝑄) ∨ (𝑃 ∧ 𝑅)
• Extended Euclidean Algorithm:
𝑃 ∨ (𝑄 ∧ 𝑅) ≡ (𝑃 ∨ 𝑄) ∧ (𝑃 ∨ 𝑅)
𝑥 𝑦 b𝑥/𝑦c 𝑎 𝑏
¬(∀𝑥𝑃(𝑥)) ≡ ∃𝑥¬𝑃(𝑥)
– new 𝑎 = old 𝑏 j k
¬(∃𝑥𝑃(𝑥)) ≡ ∀𝑥¬𝑃(𝑥)
35 12
2
−1 3 answer
– new 𝑏 = 𝑎 − 𝑏 𝑦𝑥
12 11
1
1 −1
– if gcd(𝑥, 𝑦) = 1, then 𝑎 =
11 1
11
0 1
𝑥 −1 mod 𝑦, 𝑏 = 𝑦 −1 mod 𝑥
start
1 0
Note 2/3 (Proofs)
• Direct proof
• Proof by contraposition
• Proof by cases
• Proof by induction
– Base case (prove smallest case is true)
– Inductive hypothesis (assume 𝑛 = 𝑘 true for weak induction, assume 𝑛 ≤ 𝑘 true for strong induction)
– Inductive step (prove 𝑛 = 𝑘 + 1 is true)
• Pigeonhole principle
– Putting 𝑛 + 𝑚 balls in 𝑛 bins =⇒ ≥ 1 bin has ≥ 2 balls
Note 4 (Sets)
gcd
• 𝒫(𝑆) = powerset/set of all subsets; if |𝑆| = 𝑘, |𝒫(𝑆)| = 2 𝑘
• One to one (injection); 𝑓 (𝑥) = 𝑓 (𝑦) =⇒ 𝑥 = 𝑦
• Onto (surjection); (∀𝑦∃𝑥)( 𝑓 (𝑥) = 𝑦); “hits" all of range
• Bijection: both injective and surjective
Note 5 (Countability & Computability)
• Countable if bijection to N
• Cantor-Schroder-Bernstein Theorem: bijection between 𝐴 and 𝐵 if
there exists injections 𝑓 : 𝐴 → 𝐵 and 𝑔 : 𝐵 → 𝐴
• Cantor diagonalization: to prove uncountability, list out possibilities,
construct new possibility different from each listed at one place (ex.
reals ∈ (0, 1), infinite binary strings, etc)
• 𝐴 ⊆ 𝐵 and 𝐵 is countable =⇒ 𝐴 is countable
• 𝐴 ⊇ 𝐵 and 𝐴 is uncountable =⇒ 𝐵 is uncountable
• Infinite cartesian product sometimes countable (∅ × ∅ × · · · ), sometimes
uncountable ({0, 1}∞ )
• Halting Problem: can’t determine for every program whether it halts
(uncomputable)
• Reduction of TestHalt(P, x) to some task (here, TestTask)
– define inner function that does the task if and only if P(x) halts
– call TestTask on the inner function and return the result in
TestHalt
Note 6 (Graph Theory)
• 𝐾 𝑛 has 2 edges
• Handshaking lemma: total degree = 2𝑒
• Trees: (all must be true)
– connected & no cycles
– connected & has 𝑛 − 1 edges (𝑛 = |𝑉 |)
– connected & removing an edge disconnects the graph
– acyclic & adding an edge makes a cycle
• Hypercubes:
– 𝑛-length bit strings, connected by an edge if differs by exactly 1 bit
– 𝑛-dimensional hypercube has 𝑛2𝑛−1 edges, and is bipartite (even
vs odd parity bitstring)
• Eulerian walk: visits each edge once; only possible if connected and
all even degree or exactly 2 odd degree
• Eulerian tour: Eulerian walk but starts & ends at the same vertex;
only possible if all even degree and connected
• Planar graphs
– 𝑣+ 𝑓 = 𝑒+2
P𝑓
– 𝑖=1 𝑠𝑖 = 2𝑒 where 𝑠𝑖 = number of sides of face 𝑖
– 𝑒 ≤ 3𝑣 − 6 if planar (because 𝑠𝑖 ≥ 3)
– 𝑒 ≤ 2𝑣 − 4 if planar for bipartite graphs (because 𝑠𝑖 ≥ 4)
– nonplanar if and only if the graph contains 𝐾5 or 𝐾3,3
– all planar graphs can be colored with ≤ 4 colors
𝑛(𝑛−1)
• Chinese Remainder Theorem:
– find bases 𝑏 𝑖 that are ≡ 1 mod 𝑚 𝑖 and ≡ 0 mod 𝑚 𝑗 for 𝑗 ≠ 𝑖
Q
→ 𝑏 𝑖 = 𝑐 𝑖 (𝑐−1
𝑖≠ 𝑗 𝑚 𝑗
𝑖 modQ𝑚 𝑖 ) where 𝑐 𝑖 =
P
– 𝑥 ≡ 𝑎 𝑖 𝑏 𝑖 (mod
𝑚𝑖 )
Q
– solution is unique mod 𝑚 𝑖
– 𝑚 𝑖 must be pairwise relatively prime in order to use CRT
Note 8 (RSA)
• Scheme: for primes 𝑝, 𝑞, find 𝑒 coprime to ( 𝑝 − 1)(𝑞 − 1)
– public key: 𝑁 = 𝑝𝑞 and 𝑒
– private key: 𝑑 = 𝑒 −1 mod ( 𝑝 − 1)(𝑞 − 1)
– encryption of message 𝑚: 𝑚 𝑒 (mod 𝑁) = 𝑦
– decryption of encrypted message 𝑦: 𝑦 𝑑 (mod 𝑁) = 𝑚
• Fermat’s Little Theorem (FLT): 𝑥 𝑝 ≡ 𝑥 (mod 𝑝), or 𝑥 𝑝−1 ≡ 1
(mod 𝑝) if 𝑥 coprime to 𝑝
• Prime Number Theorem: 𝜋(𝑛) ≥ ln𝑛𝑛 for 𝑛 ≥ 17, where 𝜋(𝑛) = # of
primes ≤ 𝑛
• Breaking RSA if we know 𝑑:
– we know 𝑑𝑒 − 1 = 𝑘 ( 𝑝 − 1)(𝑞 − 1), where 𝑘 ≤ 𝑒 because
𝑑 < ( 𝑝 − 1)(𝑞 − 1)
– so 𝑑𝑒−1
𝑘 = 𝑝𝑞 − 𝑝 − 𝑞 − 1; 𝑝𝑞 = 𝑁, so we can find 𝑝, 𝑞 because we
know 𝑑, 𝑒, 𝑘
Note 9 (Polynomials)
• Property 1: nonzero polynomial of degree 𝑑 has at most 𝑑 roots
• Property 2: 𝑑 + 1 pairs of points (𝑥𝑖 distinct) uniquely defines a polynomial of degree at most 𝑑
• Lagrange Interpolation:
Q
𝑥−𝑥
– Δ𝑖 (𝑥) = 𝑖≠ 𝑗 𝑥𝑖 −𝑥𝑗𝑗
P
– 𝑃(𝑥) = 𝑖 𝑦 𝑖 Δ𝑖 (𝑥)
• Secret Sharing (normally under 𝐺𝐹 ( 𝑝)):
– 𝑃(0) = secret, 𝑃(1), . . . , 𝑃(𝑛) given to all people
– 𝑃(𝑥) = polynomial of degree 𝑘 − 1, where 𝑘 people are needed to
get the secret
• Rational Root Theorem: for 𝑃(𝑥) = 𝑎 𝑛 𝑥 𝑛 + · · · + 𝑎 0 , the roots of 𝑃(𝑥)
𝑝
that are of the form 𝑞 must have 𝑝 | 𝑎 0 , 𝑞 | 𝑎 𝑛
Note 10 (Error Correcting Codes)
• Erasure Errors: 𝑘 packets lost, message length 𝑛; need to send 𝑛 + 𝑘
packets because 𝑃(𝑥) of degree 𝑛 − 1 needs 𝑛 points to define it
• General Errors: 𝑘 packets corrupted, message length 𝑛; send 𝑛 + 2𝑘
packets
• Berlekamp Welch:
– 𝑃(𝑥) encodes message (degree 𝑛 − 1)
– 𝐸 (𝑥) constructed so that roots are where the errors are (degree 𝑘);
coefficients unknown
– 𝑄(𝑥) = 𝑃(𝑥)𝐸 (𝑥) (degree 𝑛 + 𝑘 − 1)
– substitute all (𝑥𝑖 , 𝑟 𝑖 ) into 𝑄(𝑥𝑖 ) = 𝑟 𝑖 𝐸 (𝑥𝑖 ), make system of equations
𝑄 ( 𝑥)
– solve for coefficients; 𝑃(𝑥) = 𝐸 ( 𝑥)
1
CS 70
Cheatsheet
Note 11 (Counting)
• 1st rule of counting: multiply # of ways for each choice
• 2nd rule of counting: count ordered arrangements, divide by # of
ways
to get unordered
to order
𝑛!
• 𝑛𝑘 = 𝑘!(𝑛−𝑘)!
= # ways to select 𝑘 from 𝑛
• Stars and bars:
𝑛 objects, 𝑘 groups → 𝑛 stars, 𝑘 − 1 bars
𝑛+𝑘−1
→ 𝑛+𝑘−1
𝑘−1 =
𝑛
• Zeroth rule of counting: if bijection between
𝐴 and 𝐵, then | 𝐴| = |𝐵|
P
• Binomial theorem: (𝑎 + 𝑏) 𝑛 = 𝑛𝑘=0 𝑛𝑘 𝑎 𝑘 𝑏 𝑛−𝑘
𝑛−2
𝑛 𝑘
• Hockey-stick identity: 𝑘+1
= 𝑛−1
𝑘 + 𝑘 +···+ 𝑘
P
(−1) 𝑘
• Derangements: 𝐷 𝑛 = (𝑛 − 1)(𝐷 𝑛−1 + 𝐷 𝑛−2 ) = 𝑛! 𝑛𝑘=0 𝑘!
• Principle of Inclusion-Exclusion: | 𝐴1 ∪ 𝐴2 | = | 𝐴1 | + | 𝐴2 | − | 𝐴1 ∩ 𝐴2 |
More generally, alternate add/subtract
all
√
𝑛combinations
• Stirling’s approximation: 𝑛! ≈ 2𝜋𝑛 𝑛𝑒
Note 12 (Probability Theory)
• Sample points = outcomes
• Sample space = Ω = all possible outcomes
• Probability space: (Ω, P(𝜔)); (sample space, probability function)
P
• 0 ≤ P(𝜔) ≤ 1, ∀𝜔 ∈ Ω; 𝜔 ∈Ω 𝑃(𝜔) = 1
1 , ∀𝜔 ∈ Ω
• Uniform probability: P(𝜔) = |Ω|
P
• P( 𝐴) = 𝜔 ∈ 𝐴 P(𝜔) where 𝐴 is an event
# sample points in 𝐴
| 𝐴|
• If uniform: P( 𝐴) = # sample points in Ω = |Ω|
• P( 𝐴) = 1 − P( 𝐴)
Note 13 (Conditional Probability)
𝜔)
• P(𝜔 | 𝐵) = P(
𝑃 (𝐵) for 𝜔 ∈ 𝐵
𝐴∩𝐵)
→ P( 𝐴 ∩ 𝐵) = P( 𝐴 | 𝐵) P(𝐵)
• P( 𝐴 | 𝐵) = P(P(𝐵)
• Bayes’ Rule:
P(𝐵 | 𝐴) P( 𝐴)
P(𝐵 | 𝐴) P( 𝐴)
.
P( 𝐴 | 𝐵) =
=
P(𝐵)
P(𝐵 | 𝐴) P( 𝐴) + P(𝐵 | 𝐴) P( 𝐴)
• Total Probability Rule (denom of Bayes’ Rule):
𝑛
𝑛
∑︁
∑︁
P(𝐵) =
P(𝐵 ∩ 𝐴𝑖 ) =
P(𝐵 | 𝐴𝑖 ) P( 𝐴𝑖 )
𝑖=1
𝑖=1
for 𝐴𝑖 partitioning Ω
• Independence: P(
S𝐴 ∩ 𝐵) = P(P𝐴) P(𝐵) or P( 𝐴 | 𝐵) = P( 𝐴)
𝑛 𝐴 ≤ 𝑛 P( 𝐴 )
• Union bound: P 𝑖=1
𝑖
𝑖
𝑖=1
Note 16 (Geometric/Poisson Distributions
• Geometric distribution: P(𝑋 = 𝑖) = exactly 𝑖 trials until success with
probability 𝑝; use 𝑋 − 1 for failures until success
– Memoryless Property: P(𝑋 > 𝑎 + 𝑏 | 𝑋 > 𝑎) = P(𝑋 > 𝑏); i.e.
waiting > 𝑏 units has same probability, no matter where we start
• Poisson distribution: 𝜆 = average $ of successes in a unit of time
– 𝑋 ∼ Pois(𝜆), 𝑌 ∼ Pois(𝜆) independent: 𝑋 + 𝑌 ∼ Pois(𝜆 + 𝜇)
– 𝑋 ∼ Bin(𝑛, 𝜆𝑛 ) where 𝜆 > 0 is constant, as 𝑛 → ∞, 𝑋 → Pois(𝜆)
Note 20 (Continuous Distributions)
• Probability density function:
– ∫𝑓 𝑋 (𝑥) ≥ 0 for 𝑥 ∈ R
∞
– −∞ 𝑓 𝑋 (𝑥) d𝑥 = 1
∫𝑥
• Cumulative density function: 𝐹𝑋 (𝑥) = P(𝑋 ≤ 𝑥) = −∞ 𝑓 𝑋 (𝑡) d𝑡,
d 𝐹 (𝑥)
𝑓 𝑋 (𝑥) = d𝑥
𝑋
∫∞
• Expectation: E[𝑋] = −∞ 𝑥 𝑓 𝑋 (𝑥) d𝑥
∫∞
• LOTUS: E[𝑔(𝑋)] = −∞ 𝑔(𝑥) 𝑓 𝑋 (𝑥) d𝑥
• Joint distribution: P(𝑎 ≤ 𝑋 ≤ 𝑏, 𝑐 ≤ 𝑌 ≤ 𝑑)
– ∫𝑓 𝑋𝑌 ∫(𝑥, 𝑦) ≥ 0, ∀𝑥, 𝑦 ∈ R
∞ ∞
– −∞ −∞ 𝑓 𝑋𝑌 (𝑥, 𝑦) d𝑥 d𝑦 = 1
∫∞
• Marginal distribution: 𝑓 𝑋 (𝑥) = −∞ 𝑓 𝑋𝑌 (𝑥, 𝑦) d𝑦; integrate over all 𝑦
• Independence: 𝑓 𝑋𝑌 (𝑥, 𝑦) = 𝑓 𝑋 (𝑥) 𝑓𝑌 (𝑦)
𝑓𝑋𝑌 ( 𝑥,𝑦)
𝑓𝑋 ( 𝑥)
• Conditional probability: 𝑓 𝑋 | 𝐴 (𝑥) = P(
𝑓𝑌 ( 𝑦)
𝐴) , 𝑓 𝑋 |𝑌 (𝑥 | 𝑦) =
• Exponential distribution: continuous analog to geometric distribution
– Memoryless property: P(𝑋 > 𝑡 + 𝑠 | 𝑋 > 𝑡) = P(𝑋 > 𝑠)
– Additionally, P(𝑋 < 𝑌 | min(𝑋, 𝑌 ) > 𝑡) = P(𝑋 < 𝑌 )
– If 𝑋 ∼ Exp(𝜆 𝑋 ), 𝑌 ∼ Exp(𝜆𝑌 ) independent, then
𝑋
min(𝑋, 𝑌 ) ∼ Exp(𝜆 𝑋 + 𝜆𝑌 ) and P(𝑋 ≤ 𝑌 ) = 𝜆𝑋𝜆+𝜆
𝑌
• Normal distribution (Gaussian distribution)
2
2
– If 𝑋 ∼ N (𝜇 𝑋 , 𝜎𝑋 ), 𝑌 ∼ N (𝜇𝑌 , 𝜎𝑌 ) independent:
𝑍 = 𝑎𝑋 + 𝑏𝑌 ∼ N (𝑎𝜇 𝑋 + 𝑏𝜇𝑌 , 𝑎 2 𝜎𝑋2 + 𝑏 2 𝜎𝑌2 )
P𝑛
𝑋𝑖 , all 𝑋𝑖 iid with mean 𝜇,
• Central Limit Theorem: if 𝑆 𝑛 = 𝑖=1
variance 𝜎 2 ,
𝑆𝑛
𝜎2
𝑆 𝑛 − 𝑛𝜇
≈ N 𝜇,
;
≈ N (0, 1).
√
𝜎
𝑛
𝜎 𝑛
Note 17 (Concentration Inequalities, LLN)
]
• Markov’s Inequality: P(𝑋 ≥ 𝑐) ≤ E[𝑋
𝑐 , if 𝑋 nonnegative, 𝑐 > 0
Note 14 (Random Variables)
• Bernoulli distribution: used as an indicator RV
• Binomial distribution: P(𝑋 = 𝑖) = 𝑖 successes in 𝑛 trials, success
probability 𝑝
– If 𝑋 ∼ Bin(𝑛, 𝑝), 𝑌 ∼ Bin(𝑚, 𝑝) independent, 𝑋 +𝑌 ∼ Bin(𝑛+𝑚, 𝑝)
• Hypergeometric distribution: P(𝑋 = 𝑘) = 𝑘 successes in 𝑁 draws w/o
replacement from size 𝑁 population with 𝐵 objects (as successes)
• Joint distribution: P(𝑋 = 𝑎, 𝑌 = 𝑏)
P
• Marginal distribution: P(𝑋 = 𝑎) = 𝑏 ∈𝐵 P(𝑋 = 𝑎, 𝑌 = 𝑏)
• Independence: P(𝑋 = 𝑎, 𝑌 = 𝑏) = P(𝑋 = 𝑎) P(𝑌 = 𝑏)
P
• Expectation: E[𝑋] = 𝑥 ∈X 𝑥 P(𝑋 = 𝑥)
P
• LOTUS: E[𝑔(𝑋)] = 𝑥 ∈X 𝑔(𝑥) P(𝑋 = 𝑥)
• Linearity of expectation: E[𝑎𝑋 + 𝑏𝑌 ] = 𝑎 E[𝑋] + 𝑏 E[𝑌 ]
• 𝑋, 𝑌 independent: E[𝑋𝑌 ] = E[𝑋] E[𝑌 ]
Note 15 (Variance/Covariance)
• Variance: Var(𝑋) = E[(𝑋 − 𝜇) 2 ] = E 𝑋 2 − E[𝑋] 2
– Var(𝑐𝑋) = 𝑐2 Var(𝑋), Var(𝑋 +𝑌 ) = Var(𝑋) + Var(𝑌 ) + 2 Cov(𝑋, 𝑌 )
– if indep: Var(𝑋 + 𝑌 ) = Var(𝑋) + Var(𝑌 )
• Covariance: Cov(𝑋, 𝑌 ) = E[𝑋𝑌 ] − E[𝑋] E[𝑌 ]
Cov(𝑋 ,𝑌 )
• Correlation: Corr(𝑋, 𝑌 ) = 𝜎𝑋 𝜎𝑌 , always in [−1, 1]
• Indep. implies uncorrelated (Cov = 0), but not other way around, ex.
(
𝑋=
1
−1
𝑝 = 0.5
,
𝑝 = −0.5


1



𝑌 = −1


0

𝑋 = −1, 𝑝 = 0.5
𝑋 = −1, 𝑝 = 0.5
𝑋=1
2
|𝑌 | 𝑟 ]
• Generalized Markov: P(|𝑌 | ≥ 𝑐) ≤ E[ 𝑐𝑟 for 𝑐, 𝑟 > 0
Var(𝑋 )
• Chebyshev’s Inequality: P(|𝑋 − 𝜇| ≥ 𝑐) ≤ 𝑐 2 for 𝜇 = E[𝑋],
𝑐>0
√︁
– Corollary: P(|𝑋 − 𝜇| ≥ 𝑘𝜎) ≤ 𝑘12 for 𝜎 = Var(𝑋), 𝑘 > 0
• Confidence intervals:
Var( 𝑝)
ˆ
– For proportions, P(| 𝑝ˆ − 𝑝| ≥ 𝜀) ≤ 𝜀 2 ≤ 𝛿, where 𝛿 is the
confidence level (95% interval → 𝛿 = 0.05)
𝑝 (1− 𝑝)
– 𝑝ˆ = proportion of successes in 𝑛 trials, Var( 𝑝)
ˆ =
𝑛
1
– =⇒ 𝑛 ≥ 4𝜀 2 𝛿
2
– For means, P 𝑛1 𝑆 𝑛 − 𝜇 ≥ 𝜀 ≤ 𝑛𝜎𝜀 2 = 𝛿
P𝑛
– 𝑆 𝑛 = 𝑖=1 𝑋𝑖 , all 𝑋𝑖 ’s iid mean 𝜇, variance 𝜎 2
– =⇒ 𝜀 = √𝜎 , interval = 𝑆 𝑛 ± √𝜎
𝑛𝛿
𝑛𝛿
– With CLT,
√
√ P(| 𝐴𝑛 − 𝜇| ≤ 𝜀) = P
( 𝐴𝑛 −𝜇) 𝑛
𝜎
𝜀 𝑛
≤ 𝜎
√ 𝜀 𝑛
≈ 1 − 2Φ − 𝜎 = 1 − 𝛿.
2
Here, 𝐴𝑛 = 𝑛1 𝑆 𝑛 and CLT gives 𝐴𝑛 ≈ N 𝜇, 𝜎𝑛 ; use inverse cdf
to get 𝜀
• Law of large numbers: as 𝑛 → ∞, sample average of iid 𝑋1 , . . . 𝑋𝑛
tends to population mean
CS 70
Cheatsheet
Note 24 (Markov Chains)
• Markov chain = sequence of states
• 𝑋𝑛 = state at time step 𝑛
• X = state space (finite)
• 𝜋 𝑛 = distribution of states at time step 𝑛
• Markov property (memoryless): P(𝑋𝑛+1 = 𝑖 | 𝑋𝑛 , 𝑋𝑛−1 , . . . , 𝑋0 ) =
P(𝑋𝑛+1 = 𝑖 | 𝑋𝑛 )
• Transition matrix (P): transition probabilities, from row to column
• 𝜋 𝑛 = 𝜋0 P𝑛
• Invariant distribution: 𝜋 = 𝜋P; solve balance equations in addition to
P
𝜋(𝑖) = 1
• Irreducible Markov chain: any state can be reached from any other
• All irreducible Markov chains have a unique invariant distribution:
∑︁
1 𝑛−1
1{𝑋𝑚 = 𝑖}.
𝜋(𝑖) = lim
𝑛→∞ 𝑛
𝑚=0
That is, average time spent at state 𝑖 = 𝜋(𝑖)
• Periodicity: let 𝑑 (𝑖) = gcd{𝑛 > 0 : P(𝑋𝑛 = 𝑖 | 𝑋0 = 𝑖) > 0}
– If 𝑑 (𝑖) > 1, periodic with period 𝑑
– If 𝑑 (𝑖) = 1, aperiodic
– If irreducible, all states have the same 𝑑 (𝑖)
• Aperiodic irreducible Markov chains will always converge to the
invariant distribution
• Hitting time: # of steps before first reaching state 𝑖
– Let 𝛽( 𝑗) denote this value, starting at state 𝑗; set up system of
equations and solve
• P( 𝐴 before 𝐵): similarly, let 𝛼( 𝑗) denote this probability, starting at
state 𝑗; set up system of equations and solve (use TPR)
• Similar problems: let 𝑓 (𝑖) denote # of steps or probability, starting at
state 𝑖; set up equations and solve
• 2-state Markov chain:
𝑎
1−𝑎
1
2
1−𝑏
𝑏
𝑏
1−𝑎
𝑎
𝑎
P=
and 𝜋 = 𝑎+𝑏 𝑎+𝑏
𝑏
1−𝑏
Other
• In CS70, naturals and whole numbers both include 0
𝑛
• Sum of finite geometric series: 1−𝑟
1−𝑟 𝑎 where 𝑟 is ratio, 𝑎 is first term,
𝑛 is number of terms
• Memoryless property independence: if 𝑋, 𝑌 both memoryless (i.e.
geometric or exponential), then min(𝑋, 𝑌 ) and max(𝑋, 𝑌 ) − min(𝑋, 𝑌 )
are
independent
P𝑛
P
P𝑛
• E 𝑆 2𝑛 = 𝑖=1
E 𝑋𝑖2 + 2 𝑖≠ 𝑗 E 𝑋𝑖 𝑋 𝑗 = 𝑖=1
E 𝑋𝑖2 , where
P𝑛
𝑆 𝑛 = 𝑖=1 𝑋𝑖 ; useful for variance of indicators
• Coupon Collector Problem:
– 𝑛 distinct items, each with equal probability; 𝑋𝑖 = # of tries before
𝑖th new item, given 𝑖 − 1 already
P
– 𝑆 𝑛 = 𝑋𝑖 = total tries before getting all items
– We have 𝑋𝑖 ∼ Geom( 𝑛−𝑖+1
𝑛 ) because 𝑖 − 1 old items, so 𝑛 − 𝑖 + 1
new items
– Hence,
𝑛
𝑛
𝑛 1
∑︁
∑︁
∑︁
𝑛
E[𝑆 𝑛 ] =
E[𝑋𝑖 ] =
=𝑛
≈ 𝑛(ln 𝑛 + 0.5772)
𝑖=1
𝑖=1 𝑛 − 𝑖 + 1
𝑖=1 𝑖
3
CS 70
Table 1: Common Discrete Distributions
Distribution
Parameters
PMF (P(𝑋 = 𝑘))
Uniform
Uniform(𝑎, 𝑏)
Bernoulli
Bernoulli( 𝑝)
1
𝑏−𝑎+1
(
1 𝑝
0 1− 𝑝
Binomial
Bin(𝑛, 𝑝)
𝑛 𝑘
𝑝 (1 − 𝑝) 𝑛−𝑘
𝑘
Geometric
Geom( 𝑝)
𝑝(1 − 𝑝) 𝑘−1
Poisson
𝜆 𝑘 𝑒 −𝜆
Pois(𝜆)
CMF (P(𝑋 ≤ 𝑘))
Expectation (E[𝑋])
𝑘 −𝑎+1
𝑏−𝑎+1
𝑎+𝑏
2
Variance (Var(𝑋))
(𝑏 − 𝑎
+ 1) 2
−1
12
𝐾 𝑁 −𝐾 𝑘
𝑛−𝑘
𝑁
𝑛
Hypergeometric(𝑁, 𝐾, 𝑛)
𝑋 ∈ [𝑎, 𝑏]
—
𝑝
𝑝(1 − 𝑝)
𝑋 ∈ {0, 1}
—
𝑛𝑝
𝑛𝑝(1 − 𝑝)
𝑋∈N
1 − (1 − 𝑝) 𝑘
1
𝑝
1− 𝑝
𝑝2
𝑋∈N
—
𝜆
𝜆
𝑋∈N
𝐾 (𝑁 − 𝐾)(𝑁 − 𝑛)
𝑁 2 (𝑁 − 1)
𝑋∈N
𝑘!
Hypergeometric
Support
—
𝑛
𝐾
𝑁
𝑛
4
Distribution
Uniform
Exponential
Normal/Gaussian
PDF ( 𝑓 𝑋 (𝑥))
Parameters
CDF (𝐹𝑋 (𝑥) = P(𝑋 ≤ 𝑥))
Expectation (E[𝑋])
Uniform(𝑎, 𝑏)
1
𝑏−𝑎
𝑥−𝑎
𝑏−𝑎
Exp(𝜆)
𝜆𝑒 −𝜆𝑥
1 − 𝑒 −𝜆𝑥
𝑎+𝑏
2
1
𝜆
Φ(𝑥)
𝜇
N (𝜇, 𝜎 2 )
√
(𝑥 − 𝜇) 2
exp −
2𝜎 2
2𝜋𝜎
1
Variance (Var(𝑋))
(𝑏
− 𝑎) 2
12
1
𝜆2
𝜎2
Support
𝑋 ∈ [𝑎, 𝑏]
𝑋 ∈ [0, ∞)
𝑋∈R
Cheatsheet
Table 2: Common Continuous Distributions