Chapter 1: Algorithm Analysis When counting big(O), pay attention to three things ● Nested loops ● Loop indexing (linearly or multiplicatively) ● Function calls Function calls that are not nested within loops are usually not important. Big O, Big theta, Big Omega: https://www.youtube.com/watch?v=A03oI0znAoc https://www.youtube.com/watch?v=Nd0XDY-jVHs https://www.youtube.com/watch?v=NI4OKSvGAgM https://www.youtube.com/watch?v=mwN18xfwNhk https://www.youtube.com/watch?v=WlBBTSL0ZRc https://www.youtube.com/watch?v=9TlHvipP5yA https://www.youtube.com/watch?v=9SgLBjXqwd4 https://www.youtube.com/watch?v=p1EnSvS3urU Make sure you can recognize these algorithms when you see them and identify the various time complexities: Algorithm O(N) : worst Ω(N): best Θ(N): average Space Quick sort (pivot, recursive) O(n^2) O(n log(n)) O(n log(n)) O(n) Merge sort O(n log(n)) O(n log(n)) O(n log(n)) O(n) Heap sort O(n log(n)) O(n log(n)) O(n log(n)) O(1) Smooth sort O(n log(n)) O(n) O(n log(n)) O(1) Bubble sort (double looping) O(n^2) O(n) O(n^2) O(1) Insertion sort (item before i) O(n^2) O(n) O(n^2) O(1) Selection sort (update minimum index) O(n^2) O(n^2) O(n^2) O(1) Chapter 2: Number Theory and Cryptography Modular Arithmetic Mod for negative numbers: -17 mod 10 = q * 10 + r => q = -2, r = 3 Congruence modulo https://www.khanacademy.org/computing/computer-science/cryptography/modarithmetic/e/congr uence-relation which means that 26 mod 5 = 11 mod 5 Modular multiplication (A * B) mod C = (A mod C * B mod C) mod C (property) Modular exponentiation For A^B mod C Step 1: Divide B into powers of 2 by writing it in binary 117 = 1110101 = 2^0 + 2^2 + 2^4 + 2^5 + 2^6 (locations where there is 1, backwards) 5^117 = 5^( 2^0 + 2^2 + 2^4 + 2^5 + 2^6) 5^117 mod 19 = (5^1 * 5^4 * 5^16 * 5^32 * ^64) mod 19 Step 2: Calculate mod c of the powers of two <= B Step 3: Use modular multiplication property (see above) to combine the calculated mod C values. ________________________________________ Modular inverse Find (197)^(-1) mod 3000 or solve for d: 197d = 1 mod 3000 Step 1: Find the gcd of 3000 and 197 Step 2: Express 1 as the difference between multiples of 3000 and 197 Step 3: Find the coefficient of 197 and the inverse is going to be (coefficient mod 3000). Step 1: Find the GCD of 3000 and 197 using Euclid’s algorithm 3000 = 15(197) + 45 197=4(45) + 17 45=2(17) + 11 17 = 1(11) + 6 11 = 1(6) + 5 6 = 1(5) + 1 (the answer is 1. If not 1, then there will be no inverse) 5 = 1(5) + 0 Step 2: Express 1 as the difference between multiples of 3000 and 197 Starting from the penultimate step from above, rewrite as follows 1 = 533(197) - 35(3000) Step 3: identify the coefficient for 197, which is 533. The inverse is 533 mod 3000 = 533 Find the multiplicative inverses of 3 (mod 26), 3 (mod 7), 8 (mod 11) ________________________________________ Euclidean Algorithm https://www.youtube.com/watch?v=JUzYl1TYMcU Extended Euclidean Algorithm https://www.youtube.com/watch?v=OyRzpScJuvE 1. Extended euclidean algorithm expresses the GCD as a linear combination of the factors. 2. If the GCD is one, then the multiplicative inverse can be obtained by taking the modulus on both sides of the equation by mod (e.g., 25^-1 mod 100 – 100 should be the mod) Numbers in Different Bases Fast Exponentiation Algorithms RSA Algorithm https://www.youtube.com/watch?v=2Z3toEiY5lI&t=42s 1. consider two large prime numbers p, q 2. calculate phi(n) = (p-1) (q-1) 3. assume e such that gcd(e, phi(n)) = 1 4. assume d such that d*e mod (phi(n)) = 1 5. public key = {e, n}, private key = {d, n} 6. encryption: cipher = m^e mod n 7. decryption: message = c^d mod n Chapter 3: Recursion and Induction Homogeneous recurrence relations https://www.youtube.com/watch?v=aHw7hAAjbD0 https://www.youtube.com/watch?v=moQvLm8JlIc https://www.youtube.com/watch?v=TvFCDWEUjZs Recursive Structures Chapter 4: Counting Don't Memorise on Permutations and Combinations https://www.youtube.com/playlist?list=PLCZFcMzDIQmc07NPz4jPG28tlgA7_blzJ *Practice all problems and challenges until you can answer them without reviewing the instructions.* Sum and Product Rule The product rule states that the cardinality of individual sequences (number of elements) can be multiplied to determine the total number of possible sequences that could be created from the individual sequences. ● For example, to determine the number of ways you could choose two cards—a red card and spade—from a standard deck of cards, you would identify the cardinality of each sequence and multiply the numbers. The number of red cards is 26 and the number of spades is 13. In this example there are 338 possible ways to choose two cards. 26 x 13 = 338. The sum rule requires a bit more thought than the product rule. When there is an either/or choice to be made, the number of possibilities are calculated by adding the cardinalities. However, if there is any overlap in choices, the number of common choices must be subtracted. ● For example, to determine the number of ways you could choose either a heart or a face card from a standard deck of cards, you would add the cardinality of the heart cards and the cardinality of face cards. In this example, there are three common choices between these sets. The equation to obtain the result is 13 + 12 - 3 = 22. Permutations and Combinations - https://www.youtube.com/watch?v=c5It9wxTpKY Check the pdf practice sheets, lots of examples for permutations and combinations. https://www.khanacademy.org/math/precalculus/probcomb/combinations/e/permutations_and_c ombinations_2 Indistinguishable balls and distinguishable bins The number of ways to place n indistinguishable balls into m distinguishable bins: n = number of objects m = varieties of objects Important: Stars are the number of items you want to select, bars are the varieties you can select from minus 1. When order matters, you want to select bars rather than stars from the total. 1. In a donut shop, there are 20 types of donuts. How many ways can we select 12 different donuts to take home? Stars and Bars method: https://www.youtube.com/watch?v=zquaPqHFpS0 We have 11 bars to separate donuts into 12 varieties. We need to choose 12 donuts out of 31 (20 donuts + 11 bars). The answer is 31 choose 12. Notice c(31, 12) = c(31, 19). We get the same answer like the formula method 2. How many ways are there to order 6 coffees from three options: latte, americano, and drip? Formula method: (6+3-1) chooses (3-1) Stars and bars method: stars=6, bars=2 -> (6+2) chooses 6 3: Your favorite bagel place has 10 varieties of bagels (plain, poppy seed, sesame seed, salt, onion, parmesan, everything, blueberry, cinnamon raisin, and cheddar jalapeño). How many ways are there to order half a dozen bagels? Stars and bars method: The answer is: 15 (10 + 5 bars) chooses 6 4. A donut shop sells 8 different varieties of donuts, one of which is chocolate. How many selections of 12 donuts are possible if a. There are no restrictions Stars and bars method: 12 stars, 7 bars -> (12 + 7) chooses 12 Formula method: n=12, m=8 -> (12 + 8 - 1) chooses ( 8 -1 ) b. The selection must include at least one donut of each kind? 8 donuts from 8 kinds. 4 donuts left out of 8 kinds. Formula method: n=4, m=8 -> (4+8-1) choose 8-1 Stars and bars method: 4 stars, 7 bars -> (4 + 7) chooses 4 c. The selection must include exactly 2 chocolate donuts? 2 chocolate donuts. 10 donuts left out of 7 kinds. Formula method: n=10, m=7 -> (10+7-1) chooses (7-1) Stars and bars method: 10 stars, 6 bars -> (10 + 6) chooses 10 d. The shop has only 2 chocolate donuts left? No chocolate option: 12 donuts, 7 kinds -> (12 + 7-1) chooses 12 1 chocolate option: 11 donuts, 7 kinds -> (11 + 7-1) chooses 11 2 chocolate option: 10 donuts, 7 kinds -> (10 + 7-1) chooses 10 Also see 13, 14 on unit 4 supplemental practice (14 on unit 4): There are 15 varieties of donuts sold at a bakery. How many ways are there to select a dozen donuts? Stars and bars method: 12 stars, 15-1 bars -> (12+15-1) chooses 12 (15 on unit 4): Susan must do exactly 100 push ups in the course of a seven day week. How many different schedules are there for her to do her push ups if she must do at least 20 on each weekend day (Saturday and Sunday)? if she must do >= 20 on Sat and Sunday, that leaves us 100-20-20=60 pushups to distribute across 7 days. Stars and bars method: stars: 60, bars 7-1 -> (60+7-1) chooses 60 Important: Stars are the number of items you want to select, bars are the varieties you can select from minus 1. When order matters, you want to select bars rather than stars from the total (16 on unit 4): How many ways can we add three non-negative integers such that they sum to 9? Stars: 9, bars: 3-1=2 -> (9+3-1) chooses 2 (this is question on the PA) A grocery store stocks 1-gallon cartons of skim milk, 1% milk, 2% milk, and whole milk. A customer is asked to buy 10 gallons of milk. The customer needs to buy at least one carton of each type of milk. How many different ways can the kinds of milk to buy be selected? At least one carton of each type, that leaves us with 6 gallons milk to distribute across four varieties. c(6+4-1, 4-1) = c(9, 3) = 84 Inclusion/Exclusion https://www.youtube.com/watch?v=4qd9JkYVGBU When a collection of sets are disjoint—that is, mutually disjoint—counting the total number of elements is simple; just add the cardinalities. The inclusion-exclusion principle is applied when two or more sets intersect (are not disjoint). Language hint like: x, y, or both (|A U B| = |A| + |B| - |A ∩ B|) To count the total number of elements in two sets that intersect: ● Sum the cardinalities of each set ● Subtract the cardinality of the intersection To count the total number of elements in three sets, of which two intersect: ● Sum the cardinalities of all three sets ● Calculate the cardinality of the intersection of all three sets ● Calculate the cardinality of the three intersections of two sets at a time ● Subtract the cardinality of all the intersection sets |A ∪ B ∪ C| = |A| + |B| + |C| - |A ∩ B| - |B ∩ C| - |A ∩ C| + |A ∩ B ∩ C| To find the cardinality of an intersection, use the complement of a set. Binomial Theorem https://www.youtube.com/watch?v=1pSD8cYyqUo Binomial Theorem (find the coefficient) https://youtu.be/XPmrvRiTruw?t=184 Pigeonhole Principle Weak Form: If k+1 objects (pigeons) are placed in k boxes (holes), then at least one box must contain two or more of the objects. Generalized Form: If N objects are placed into k boxes (N>K), then there is at least one box containing ceil(n/k) objects. Example: How many people must be in a group in order to guarantee at least three people in the group were born in the same month? ⌈N/12⌉=3 => N = 25 Chapter 5: Discrete Probability Three fair coins are tossed. What is the probability that exactly two are tails? There are three tails c(3, 2). There are 2^3 elements in the sample space. c(3, 2)/2^3 = 3 / 8. Five cards are dealt (without replacement) from a well-shuffled 52 card standard deck. Find the probability of the following: a. Being dealt 5 spades 1 / 4 * 12 / 51 * 11/50 * 10/49 * 9/48 = 0.00049 Or c (13, 5) / c(52, 5) b. Being dealt all the same suit c(4, 1) c(13, 5) / c(52, 5) = 0.00198 A drawer has 7 socks. 4 socks are black and 3 are white socks. John randomly pulls out 4 socks. Find the probability of the following: a. All 4 socks are black b. Exactly 2 are white c. At least 3 are white d. At most 2 are black b. c(3, 2)c(4, 2) / c(7, 4) = 3*6/45 = 18/35 c. c(3, 3) c(4, 1) / c(7, 4) = 4 / 35 d. c(4,1)c(3,3)/c(7,4) + c(4, 2)c(3,2)/c(7,4) = 22/35 (this is on the PA)A random experiment consists of tossing a fair six-sided die repeatedly. How many tosses are required to be certain that the probability that at least one '6' appears is greater than or equal to 1/2? So at least one ‘6’ appears, the complement would be no ‘6’ appears (1-⅙ = ⅚), then the probability of at least one ‘6’ appears greater than or equal to ½ is 1-(⅚)^n >= 1/2 . We solve for n with this inequality, we get n = 4.