Recurrences It continues… Jeff Chastine Recurrences • When an algorithm calls itself recursively, its running time is described by a recurrence. • A recurrence describes itself in terms of its value on smaller inputs • There are four methods of solving these: substitution, iteration, recursion tree, or master method Jeff Chastine What it looks like T(n) = { Θ(1) 2T(n/2) + Θ(n) if n = 1 if n > 1 • This is the recurrence of MERGE-SORT • What this says is that the time involved is 1 if n = 1, • Else, the time involved is 2 times calling on half the size of the array, plus n time to merge sorted sub-arrays Jeff Chastine Substitution • Similar to induction • Guess solution, and prove it holds true for next call • Powerful method, but can sometimes be difficult to guess the solution! Jeff Chastine Substitution • Example: – T (n) = 2T (n/2) + n – Guess that T (n) = O(n lg n) • We must prove that T (n) cn lg n | c > 0 • Assume it holds for n/2 as well T (n/2) = c(n/2) lg (n/2) + n // Rewrite using n/2 T (n) 2 (c(n/2) lg (n/2)) + n // Substitution = cn lg (n/2)) + n // Math stuff = cn (lg n – lg 2) + n = cn lg n – cn + n cn lg n c 1 Note: means ‘for all’ Jeff Chastine Subtleties • Let T (n) = T (n/2) + T(n/2) + 1 • Assume that T (n) = O(n) • Then T (n/2) = c(n/2) T (n) c(n/2) + c(n/2) + 1 = cn + 1 Which does not imply T (n) cn • Here, we’re correct, but off by a constant! Jeff Chastine Subtleties • We strengthen our guess: T (n) cn – b T (n) (c(n/2) – b) + (c(n/2) – b) + 1 = cn – 2b + 1 cn – b b > 1 Jeff Chastine One Last Example Jeff Chastine Iteration Method • • • • • Expand out the recurrence T(n) = 3T(n/4) + n T(n) = 3(3T(n/16) + (n/4)) + n T(n) = 3((3T(n/64) + 3(n/16))+ (n/4))) + n T(n) < n + 3n/4 + 9n/16 + 27n/64 + … ∞ ( ) nΣ i=0 3 4 i 1 = n 1 - 3/4 =n 1 1 4 Jeff Chastine = 4n Recursion Tree Method • A recursion tree is built • We sum up each level, • Total cost = number of levels * cost at each level • Usually used to generate a good guess for the substitution method • Could still be used as direct proof • Example: T (n) = 3T (n/4) + (n2) Jeff Chastine T(n) Jeff Chastine cn2 T(n/4) T(n/4) Jeff Chastine T(n/4) cn2 c(n/4)2 T(n/16) T(n/16) T(n/16) c(n/4)2 c(n/4)2 T(n/16) T(n/16) T(n/16) T(n/16) T(n/16) T(n/16) Jeff Chastine cn2 c(n/4)2 c(n/16)2 c(n/16)2 c(n/16)2 c(n/4)2 c(n/4)2 c(n/16)2 c(n/16)2 c(n/16)2 c(n/16)2 c(n/16)2 c(n/16)2 T(1) T(1) T(1) T(1) T(1) T(1) T(1) T(1)T(1) T(1) T(1) T(1) T(1) T(1) T(1) T(1) T(1) Jeff Chastine cn2 cn2 c(n/4)2 c(n/16)2 c(n/16)2 c(n/16)2 c(n/4)2 c(n/4)2 3/16 cn2 c(n/16)2 c(n/16)2 c(n/16)2 c(n/16)2 c(n/16)2 c(n/16)2(3/16)2cn2 T(1) T(1) T(1) T(1) T(1) T(1) T(1) T(1)T(1) T(1) T(1) T(1) T(1) T(1) T(1) T(1) T(1) (nlog43) Jeff Chastine Questions • How many levels does this tree have? • The subproblem at depth i is n/4i • When does the subproblem hit size 1? – n/4i = 1 – n = 4i – lg4n = i • Therefore, the tree has lg4n levels • The cost at each level is 3ic(n/4i)2 • The last level has 3log4n nodes = nlog43 Jeff Chastine The Master's Method When it has this form: T(n) = aT(n/b) + f(n) • If f (n) = Ο(nlogb a-ε) for some constant ε>0, then T (n) = Θ (nlogb a) • If f (n) = Θ(nlogb a), then T (n) = Θ (nlogb algn) • If f (n) = Ω (nlogb a+ε) for some constant ε>0, then T (n) = Θ (f (n)) Jeff Chastine • T(n) = 9T(n/3) + n – a = 9, b = 3 thus nlogb a = nlog3 9= n2 • T(n) = T(2n/3) + 1 – a = 1, b = 3/2 thus nlogb a = nlog3/2 1 = n0 = 1 • Problem: – T(n) = 2T(n/2) + nlgn Jeff Chastine