Chapter 1: Algorithm Analysis
When counting big(O), pay attention to three things
● Nested loops
● Loop indexing (linearly or multiplicatively)
● Function calls
Function calls that are not nested within loops are usually not important.
Big O, Big theta, Big Omega:
https://www.youtube.com/watch?v=A03oI0znAoc
https://www.youtube.com/watch?v=Nd0XDY-jVHs
https://www.youtube.com/watch?v=NI4OKSvGAgM
https://www.youtube.com/watch?v=mwN18xfwNhk
https://www.youtube.com/watch?v=WlBBTSL0ZRc
https://www.youtube.com/watch?v=9TlHvipP5yA
https://www.youtube.com/watch?v=9SgLBjXqwd4
https://www.youtube.com/watch?v=p1EnSvS3urU
Make sure you can recognize these algorithms when you see them and identify the
various time complexities:
Algorithm
O(N) : worst Ω(N): best
Θ(N):
average
Space
Quick sort
(pivot, recursive)
O(n^2)
O(n log(n))
O(n log(n))
O(n)
Merge sort
O(n log(n))
O(n log(n))
O(n log(n))
O(n)
Heap sort
O(n log(n))
O(n log(n))
O(n log(n))
O(1)
Smooth sort
O(n log(n))
O(n)
O(n log(n))
O(1)
Bubble sort
(double looping)
O(n^2)
O(n)
O(n^2)
O(1)
Insertion sort
(item before i)
O(n^2)
O(n)
O(n^2)
O(1)
Selection sort
(update minimum index)
O(n^2)
O(n^2)
O(n^2)
O(1)
Chapter 2: Number Theory and Cryptography
Modular Arithmetic
Mod for negative numbers:
-17 mod 10 = q * 10 + r => q = -2, r = 3
Congruence modulo
https://www.khanacademy.org/computing/computer-science/cryptography/modarithmetic/e/congr
uence-relation
which means that 26 mod 5 = 11 mod 5
Modular multiplication
(A * B) mod C = (A mod C * B mod C) mod C (property)
Modular exponentiation
For A^B mod C
Step 1: Divide B into powers of 2 by writing it in binary
117 = 1110101 = 2^0 + 2^2 + 2^4 + 2^5 + 2^6 (locations where there is 1, backwards)
5^117 = 5^( 2^0 + 2^2 + 2^4 + 2^5 + 2^6)
5^117 mod 19 = (5^1 * 5^4 * 5^16 * 5^32 * ^64) mod 19
Step 2: Calculate mod c of the powers of two <= B
Step 3: Use modular multiplication property (see above) to combine the calculated mod C
values.
________________________________________
Modular inverse
Find (197)^(-1) mod 3000 or solve for d: 197d = 1 mod 3000
Step 1: Find the gcd of 3000 and 197
Step 2: Express 1 as the difference between multiples of 3000 and 197
Step 3: Find the coefficient of 197 and the inverse is going to be (coefficient mod 3000).
Step 1: Find the GCD of 3000 and 197 using Euclid’s algorithm
3000 = 15(197) + 45
197=4(45) + 17
45=2(17) + 11
17 = 1(11) + 6
11 = 1(6) + 5
6 = 1(5) + 1 (the answer is 1. If not 1, then there will be no inverse)
5 = 1(5) + 0
Step 2: Express 1 as the difference between multiples of 3000 and 197
Starting from the penultimate step from above, rewrite as follows
1 = 533(197) - 35(3000)
Step 3: identify the coefficient for 197, which is 533. The inverse is 533 mod 3000 = 533
Find the multiplicative inverses of 3 (mod 26), 3 (mod 7), 8 (mod 11)
________________________________________
Euclidean Algorithm
https://www.youtube.com/watch?v=JUzYl1TYMcU
Extended Euclidean Algorithm
https://www.youtube.com/watch?v=OyRzpScJuvE
1. Extended euclidean algorithm expresses the GCD as a linear combination of the factors.
2. If the GCD is one, then the multiplicative inverse can be obtained by taking the modulus
on both sides of the equation by mod (e.g., 25^-1 mod 100 – 100 should be the mod)
Numbers in Different Bases
Fast Exponentiation Algorithms
RSA Algorithm
https://www.youtube.com/watch?v=2Z3toEiY5lI&t=42s
1. consider two large prime numbers p, q
2. calculate phi(n) = (p-1) (q-1)
3. assume e such that gcd(e, phi(n)) = 1
4. assume d such that d*e mod (phi(n)) = 1
5. public key = {e, n}, private key = {d, n}
6. encryption: cipher = m^e mod n
7. decryption: message = c^d mod n
Chapter 3: Recursion and Induction
Homogeneous recurrence relations
https://www.youtube.com/watch?v=aHw7hAAjbD0
https://www.youtube.com/watch?v=moQvLm8JlIc
https://www.youtube.com/watch?v=TvFCDWEUjZs
Recursive Structures
Chapter 4: Counting
Don't Memorise on Permutations and Combinations
https://www.youtube.com/playlist?list=PLCZFcMzDIQmc07NPz4jPG28tlgA7_blzJ
*Practice all problems and challenges until you can answer them without reviewing the
instructions.*
Sum and Product Rule
The product rule states that the cardinality of individual sequences (number of elements) can be
multiplied to determine the total number of possible sequences that could be created from the
individual sequences.
●
For example, to determine the number of ways you could choose two cards—a red card
and spade—from a standard deck of cards, you would identify the cardinality of each
sequence and multiply the numbers. The number of red cards is 26 and the number of
spades is 13. In this example there are 338 possible ways to choose two cards. 26 x 13
= 338.
The sum rule requires a bit more thought than the product rule. When there is an either/or
choice to be made, the number of possibilities are calculated by adding the cardinalities.
However, if there is any overlap in choices, the number of common choices must be subtracted.
●
For example, to determine the number of ways you could choose either a heart or a
face card from a standard deck of cards, you would add the cardinality of the heart cards
and the cardinality of face cards. In this example, there are three common choices
between these sets. The equation to obtain the result is 13 + 12 - 3 = 22.
Permutations and Combinations - https://www.youtube.com/watch?v=c5It9wxTpKY
Check the pdf practice sheets, lots of examples for permutations and combinations.
https://www.khanacademy.org/math/precalculus/probcomb/combinations/e/permutations_and_c
ombinations_2
Indistinguishable balls and distinguishable bins
The number of ways to place n indistinguishable balls into m distinguishable bins:
n = number of objects
m = varieties of objects
Important: Stars are the number of items you want to select, bars are the varieties you
can select from minus 1. When order matters, you want to select bars rather than stars
from the total.
1. In a donut shop, there are 20 types of donuts. How many ways can we select 12 different
donuts to take home?
Stars and Bars method:
https://www.youtube.com/watch?v=zquaPqHFpS0
We have 11 bars to separate donuts into 12 varieties. We need to choose 12 donuts out of 31
(20 donuts + 11 bars). The answer is 31 choose 12. Notice c(31, 12) = c(31, 19). We get the
same answer like the formula method
2. How many ways are there to order 6 coffees from three options: latte, americano, and drip?
Formula method: (6+3-1) chooses (3-1)
Stars and bars method: stars=6, bars=2 -> (6+2) chooses 6
3: Your favorite bagel place has 10 varieties of bagels (plain, poppy seed, sesame seed, salt,
onion, parmesan, everything, blueberry, cinnamon raisin, and cheddar jalapeño). How many
ways are there to order half a dozen bagels?
Stars and bars method:
The answer is: 15 (10 + 5 bars) chooses 6
4. A donut shop sells 8 different varieties of donuts, one of which is chocolate. How many
selections of 12 donuts are possible if
a. There are no restrictions
Stars and bars method: 12 stars, 7 bars -> (12 + 7) chooses 12
Formula method: n=12, m=8 -> (12 + 8 - 1) chooses ( 8 -1 )
b. The selection must include at least one donut of each kind?
8 donuts from 8 kinds. 4 donuts left out of 8 kinds.
Formula method: n=4, m=8 -> (4+8-1) choose 8-1
Stars and bars method: 4 stars, 7 bars -> (4 + 7) chooses 4
c. The selection must include exactly 2 chocolate donuts?
2 chocolate donuts. 10 donuts left out of 7 kinds.
Formula method: n=10, m=7 -> (10+7-1) chooses (7-1)
Stars and bars method: 10 stars, 6 bars -> (10 + 6) chooses 10
d. The shop has only 2 chocolate donuts left?
No chocolate option: 12 donuts, 7 kinds -> (12 + 7-1) chooses 12
1 chocolate option: 11 donuts, 7 kinds -> (11 + 7-1) chooses 11
2 chocolate option: 10 donuts, 7 kinds -> (10 + 7-1) chooses 10
Also see 13, 14 on unit 4 supplemental practice
(14 on unit 4): There are 15 varieties of donuts sold at a bakery. How many ways are there to
select a dozen donuts?
Stars and bars method: 12 stars, 15-1 bars -> (12+15-1) chooses 12
(15 on unit 4): Susan must do exactly 100 push ups in the course of a seven day week. How
many different schedules are there for her to do her push ups if she must do at least 20 on each
weekend day (Saturday and Sunday)?
if she must do >= 20 on Sat and Sunday, that leaves us 100-20-20=60 pushups to distribute
across 7 days.
Stars and bars method: stars: 60, bars 7-1 -> (60+7-1) chooses 60
Important: Stars are the number of items you want to select, bars are the varieties you
can select from minus 1. When order matters, you want to select bars rather than stars
from the total
(16 on unit 4): How many ways can we add three non-negative integers such that they sum to
9?
Stars: 9, bars: 3-1=2 -> (9+3-1) chooses 2
(this is question on the PA) A grocery store stocks 1-gallon cartons of skim milk, 1% milk, 2% milk,
and whole milk. A customer is asked to buy 10 gallons of milk. The customer needs to buy at least
one carton of each type of milk. How many different ways can the kinds of milk to buy be selected?
At least one carton of each type, that leaves us with 6 gallons milk to distribute across four
varieties. c(6+4-1, 4-1) = c(9, 3) = 84
Inclusion/Exclusion
https://www.youtube.com/watch?v=4qd9JkYVGBU
When a collection of sets are disjoint—that is, mutually disjoint—counting the total number of
elements is simple; just add the cardinalities.
The inclusion-exclusion principle is applied when two or more sets intersect (are not
disjoint). Language hint like: x, y, or both (|A U B| = |A| + |B| - |A ∩ B|)
To count the total number of elements in two sets that intersect:
● Sum the cardinalities of each set
● Subtract the cardinality of the intersection
To count the total number of elements in three sets, of which two intersect:
● Sum the cardinalities of all three sets
● Calculate the cardinality of the intersection of all three sets
● Calculate the cardinality of the three intersections of two sets at a time
● Subtract the cardinality of all the intersection sets
|A ∪ B ∪ C| = |A| + |B| + |C| - |A ∩ B| - |B ∩ C| - |A ∩ C| + |A ∩ B ∩ C|
To find the cardinality of an intersection, use the complement of a set.
Binomial Theorem
https://www.youtube.com/watch?v=1pSD8cYyqUo
Binomial Theorem (find the coefficient)
https://youtu.be/XPmrvRiTruw?t=184
Pigeonhole Principle
Weak Form: If k+1 objects (pigeons) are placed in k boxes (holes), then at least one box must
contain two or more of the objects.
Generalized Form: If N objects are placed into k boxes (N>K), then there is at least one box
containing ceil(n/k) objects.
Example: How many people must be in a group in order to guarantee at least three people in
the group were born in the same month?
⌈N/12⌉=3 => N = 25
Chapter 5: Discrete Probability
Three fair coins are tossed. What is the probability that exactly two are tails?
There are three tails c(3, 2). There are 2^3 elements in the sample space. c(3, 2)/2^3 = 3 / 8.
Five cards are dealt (without replacement) from a well-shuffled 52 card standard deck. Find the
probability of the following:
a. Being dealt 5 spades
1 / 4 * 12 / 51 * 11/50 * 10/49 * 9/48 = 0.00049
Or c (13, 5) / c(52, 5)
b. Being dealt all the same suit
c(4, 1) c(13, 5) / c(52, 5) = 0.00198
A drawer has 7 socks. 4 socks are black and 3 are white socks. John randomly pulls out 4
socks. Find the probability of the following:
a. All 4 socks are black
b. Exactly 2 are white
c. At least 3 are white
d. At most 2 are black
b. c(3, 2)c(4, 2) / c(7, 4) = 3*6/45 = 18/35
c. c(3, 3) c(4, 1) / c(7, 4) = 4 / 35
d. c(4,1)c(3,3)/c(7,4) + c(4, 2)c(3,2)/c(7,4) = 22/35
(this is on the PA)A random experiment consists of tossing a fair six-sided die repeatedly.
How many tosses are required to be certain that the probability that at least one '6' appears
is greater than or equal to 1/2?
So at least one ‘6’ appears, the complement would be no ‘6’ appears (1-⅙ = ⅚), then the
probability of at least one ‘6’ appears greater than or equal to ½ is 1-(⅚)^n >= 1/2 . We
solve for n with this inequality, we get n = 4.