EECS 3101 Prof. Andy Mirzaian STUDY MATERIAL: • [CLRS] Appendix A, chapter 4 • Lecture Note 3 2 Summations: ๐ ๐ ๐ = ๐ 1 + ๐ 2 + โฏ + ๐ ๐ = Θ( ? ) ๐=1 ๐ 2 ๐ = Θ ๐ 2 ๐ ๐=1 Recurrence Relations: ๐ ๐ = ๐ ๐ = ๐ ๐ − 1 + ๐(๐) 0 2๐ ๐ − 1 + 1 0 ∀๐ ≥ 1 ∀๐ < 1 ∀๐ ≥ 1 ∀๐ < 1 ๐ โน ๐ ๐ = ๐(๐) ๐=1 โน ๐ ๐ = Θ 2๐ 3 TOPICS Summations & Recurrence Relations arise in the running time analysis of algorithms. ๏ง SUMMATIONS: ๏ Classic Methods: Arithmetic, Geometric, Harmonic, Bounded Tail ๏ Approximating Summation by Integration ๏ Summation by Parts ๏ ASSUME: ASymptotic SUmmation Made Easy ๏ง RECURRENCE RELATIONS: ๏ Iteration method ๏ Recursion tree method ๏ Divide-&-Conquer Recurrence: The Master Method ๏ Guess-&-Verify method ๏ Full History Recurrences ๏ Variable Substitution method 4 SUMMATION f : Ν ๏ฎ R๏ซ S (n) ๏ฝ n ๏ฅ f ( i ) ๏ฝ f (1) ๏ซ f ( 2 ) ๏ซ f ( 3) ๏ซ ๏ ๏ ๏ ๏ซ f ( n ) i ๏ฝ1 5 Classic Methods ๏ง Arithmetic (or polynomial): f(n) = n: 1 + 2 + ··· + n f(n) = n2: 12 + 22 + ··· + n2 f(n) = nd: 1d + 2d + ··· + nd S(n) = Q(n f(n)) = n(n+1)/2 = Q( n2 ) = n(n+1)(2n+1)/6 = Q( n3 ) = Q( nd+1 ), for any constant real d > -1 ๏ง Geometric (or exponential): S(n) = Q(f(n)) f(n) = xn: 1 + x + x2 + ··· + xn = (xn+1 –1)/(x –1) for any constant real 0๏น|x|๏น1 = Q(xn) if x > 1 (Geometric increasing) f(n) = 2n: 1+ 2 + 22 + ··· + 2n = Q(2n) (with x = 2) ๏ง Harmonic: S(n) = Q(log n) f(n) = 1/n: H(n) = 1 + 1/2 + 1/3 + ··· + 1/n ≈ ln n = Q(log n) ๏ง Bounded Tail: S(n) = Q(1) 1 + x + x2 + ··· + xn = (1 – xn+1)/(1-x) = Q(1) if 0<x<1. (Geometric decreasing) f(n) = 2-n: 1 + 1/2 + 1/4 + ··· + 1/2n = Q(1) (with x = ½) f(n) = n-2: 1 + 1/22 + 1/ 32 + ··· + 1/n2 = Q(1) (Arithmetic decreasing) 6 Approximating f๏ญ : ๐(๐) by ๐ ๐ฅ ๐๐ฅ f monotonically non-decreasing f(x) x m-1 m ๏ฒ n m ๏ญ1 m+1 m+2 f ( x )dx ๏ฃ n ... ๏ฅ f (i ) i๏ฝm n ๏ฃ n+1 ๏ฒ n ๏ซ1 m f ( x )dx 7 Approximating f๏ฏ : ๐(๐) by ๐ ๐ฅ ๐๐ฅ f monotonically non-increasing f(x) x m-1 m ๏ฒ n m ๏ญ1 m+1 m+2 f ( x )dx ๏ณ n ... ๏ฅ f (i ) i๏ฝm n ๏ณ n+1 ๏ฒ n ๏ซ1 m f ( x )dx NOTE: direction of inequalities changed 8 ๐(๐) by Approximating ๐ ๐ ๐ ↑โถ ๐ ๐ฅ ๐๐ฅ ≤ ๐−1 ๐ ↓โถ ๐+1 ๐ ๐ ≤ ๐ ๐ ๐ฅ ๐๐ฅ ≥ ๐−1 S(n) = f(1) + f(2) + … + f(n) ๐ ๐ฅ ๐๐ฅ ๐ ๐=๐ ๐ ๐ ๐ฅ ๐๐ฅ ๐+1 ๐ ๐ ≥ ๐=๐ ๐ ๐ฅ ๐๐ฅ ๐ (m = 1) Example 1: S(n) = 13 + 23 + … + n3 . f(x) = x3 : f ๏ญ and ๏ฒ f(x) dx = ๏ฒ x3 dx = x4 / 4. n4 / 4 ๏ฃ S(n) ๏ฃ ( (n+1)4 – 1)/ 4 ๏ S(n) = Q(n4) Example 2: H(n) = 1 + 1/2 + 1/3 + … + 1/n . f(x) = 1/x : f ๏ฏ and ๏ฒ f(x) dx = ๏ฒ dx /x = ln x. 1 + ln n ๏ณ H(n) ๏ณ ln (n+1) ๏ H(n) = Q(log n) Caution against dividing by zero! Separate f(1) from the summation. 9 Summation by Parts: Arithmetic f(n) = n : S(n) = f(1) + f(2) + f(3) + ··· + f(n). ๏ฃ n·n ๏ฃ O(n2) S(n) = 1 + 2 + 3 + ······································································· +n + (๏ซn/2๏ป –1) + ๏ซn/2๏ป + (๏ซn/2๏ป +1) + (๏ซn/2๏ป + 2) + ๏ซn/2๏ป ๏ฉn/2๏น ๏ณ ๏ฉn/2๏น · ๏ฉn/2๏น ๏ณ W(n2) ๏ณ0 ๏ณ W(n2) ๏ S(n) = Q(n2) 10 Summation by Parts: Harmonic ๏ฃ 1+ ๏ฉlog n๏น 1 <1 < 2(1/2)=1 < 4(1/4) =1 ๏ฃ O(log n) < 8(1/8) = 1 H (n) ๏ฝ 1 ๏ซ 12 ๏ซ 13 ๏ซ 14 ๏ซ 15 ๏ซ 16 ๏ซ 17 ๏ซ 18 ๏ซ 19 ๏ซ 101 ๏ซ 111 ๏ซ 121 ๏ซ 131 ๏ซ 141 ๏ซ 151 ๏ซ 161 ๏ซ ๏ ๏ ๏ ๏ซ 1 ½ >2(1/4) = ½ > 4(1/8) = ½ ๏ณ 1 + ½ ๏ซlog n๏ป 1 n > 8(1/16) = ½ ๏ณ W(log n) ๏ H(n) = Q(log n) 11 ASSUME : ASymptotic SUmmation Made Easy f: N ๏ฎ ๏+ : S(n) = f(1) + f(2) + f(3) + ··· + f(n) maxn(f) = max { f(1) , f(2) , … , f(n) } minn(f) = min { f(1) , f(2) , … , f(n) } aven(f) = average {f(1) , f(2), … , f(n) } S(n) = n · aven(f) 0 ๏ฃ minn(f) ๏ฃ aven(f) ๏ฃ maxn(f) f๏ญ ๏ minn(f) = f(1) , maxn(f) = f(n) f๏ฏ ๏ minn(f) = f(n) , maxn(f) = f(1) max{ n · minn(f), maxn(f) } ๏ฃ S(n) ๏ S(n) = Q( g(n) · maxn(f)) ๏ฃ n · maxn(f) for some 1 ๏ฃ g(n) ๏ฃ n ASSUME: A method for finding g(n) (and hence, S(n)). 12 ASSUME on monotone f(n) f: N ๏ฎ ๏+ S(n) = f(1) + f(2) + f(3) + ··· + f(n) FACT: f is monotone ๏ exactly one of the following holds: (1) (2) (3) (4) f(n) = 0 f(n) = Q(1) f(n) = w(1) f(n) = o(1) ๏ขn and f๏ญ and f๏ฏ and f(1) > 0 Cases (1) & (2): g(n) = Q(n), S(n) = Q(g(n) f(n)) = Q(n f(n)) Case (3): Considered next. Case (4): Similar methods apply. Exercise. 13 ASSUME for f๏ญ & f(n) = w(1) f S(n) = f(1) + ··· + f(n) f(n) f(m) ๏ฃ cf(n) ๏ฃ f(m+1) f(n)g(n) 0<c<1 m n g(n) Step 1) Find m such that log f(m) = log f(n) – Q(1) Step 2) g(n) ๏ฌ ๏ฉn – m๏น THEOREM: f๏ญ & g๏ญ [ Note: 1 ๏ฃ g(n) ๏ฃ n ] ๏ S(n) = Q(f(n)g(n)). 14 ASSUME: examples Polynomial: f(n) = nd (const. d ๏ณ 0) : log f(m) = log f(n) – Q(1) d log m = d log n – d = d log n/2 m = n/2 ๏ g(n) = n–m = n/2. f๏ญ & g๏ญ ๏ S(n) = Q(f(n)g(n)) = Q(nd+1). Exponential: f(n) = 2n : log f(m) = log f(n) – Q(1) m = n – 1 ๏ g(n) = n–m = 1 f๏ญ & g๏ญ ๏ S(n) = Q(f(n)g(n)) = Q(2n). 15 log(1 – x) = Q( ln(1 – x) ) = – Q(x) for x = o(1) Taylor Series Expansion : f (1) ( 0 ) f ( 2 ) ( 0) 2 f ( 3 ) ( 0) 3 f ( x ) ๏ฝ f ( 0) ๏ซ x๏ซ x ๏ซ x ๏ซ๏๏๏ 1! 2! 3! 2 3 4 ln(1 ๏ญ x ) ๏ฝ ๏ญ x ๏ญ x2 ๏ญ x3 ๏ญ x4 ๏ญ ๏ ๏ ๏ f (x) = ln(1– x) : Example: log(n ๏ญ Q (log n )) log n ๏ฝ log n ๏จ1 ๏ญ Q ๏จ n ๏ฉ๏ฉ log n ๏ฝ log n ๏ซ log๏จ1 ๏ญ Q ๏จ n ๏ฉ๏ฉ log n ๏ฝ log n ๏ญ Q ๏จ n ๏ฉ ๏ฝ (log n ) ๏จ1 ๏ญ Q ๏จ 1n ๏ฉ๏ฉ ๏ ๏ [ take x ๏ฝ Q ๏จ log n n ๏ฉ] 16 ASSUME: examples Super-polynomial sub-exponential 1: f(n) = 2 ๐ : log f(m) = log f(n) – Q(1) ๐ = ๐ – 1 [Now square both sides] 2 m = ( ๐ – 1) = n–2 ๐+1 ๏ g(n) = n–m = 2 ๐ –1 = Q( ๐ ). f๏ญ & g๏ญ ๏ S(n) = Q(f(n)g(n)) = Q( ๐2 ๐ ). Super-polynomial sub-exponential 2: f(n) = 2n/log n : = log f(n) – Q(1) m/log m = n/log n – Q(1) [Multiply across by log m ๏ป log n ] m = n – Q(log n) [Verify this satisfies (*) with “=” ] m/log m = (n – Q(log n)) / log (n – Q(log n)) = ( n – Q(log n) ) / (log n)(1 – Q(1/n)) [See previous page] = ( n/log n – Q(1) ) / (1 – Q(1/n)) = n/log n – Q(1) [Why? Multiply sides by 1 – Q(1/n)] log f(m) (*) ๏ g(n) = n – m = Q(log n) f๏ญ & g๏ญ ๏ S(n) = Q(f(n)g(n)) = Q(2n/log n log n). 17 ASSUME: Lower Bound Proof THEOREM: f๏ญ & g๏ญ ๏ S(n) = f(1) + ··· + f(n) = Q(f(n)g(n)). f f(n) f(m) ๏ฃ cf(n) ๏ฃ f(m+1) f(m+1)g(n) m m+1 n g(n) Lower bound: S(n) ๏ณ S(n) – S(m) = f(m+1) + f(m+2) + ··· + f(n) ๏ณ shaded rectangular area = f(m+1) g(n) ๏ณ c f(n) g(n) ๏ณ W( f(n) g(n) ). since f๏ญ 18 ASSUME: Upper Bound Proof THEOREM: f๏ญ & g๏ญ ๏ S(n) = f(1) + ··· + f(n) = Q(f(n)g(n)). f f(n0) cf(n0) ๏ณ f(n1) c2f(n0) ๏ณ cf(n1) ๏ณ f(n2) g(n0)f(n0) g(n2)f(n2) g(n1)f(n1) n3 g(n2) n2 g(n1) n1 g(n0) n0 = n Upper bound: S(n) ๏ฃ sum of shaded rectangular areas = g(n0)f(n0) + g(n1)f(n1) + g(n2)f(n2) + ··· since f๏ญ ๏ฃ g(n) [ f(n0) + f(n1) + f(n2) + ··· ] since g๏ญ ๏ฃ g(n)f(n) [ 1 + c + c2 + ··· ] f(ni+1) ๏ฃ c f(ni) ๏ฃ g(n)f(n) / (1 – c) classic geometric decreasing sum ๏ฃ O( f(n) g(n) ). 19 Caution! To guarantee conclusion of the theorem, its premise must hold. The premise insists that not only f๏ญ, but also g๏ญ holds. f๏ญ without g๏ญ is possible. The following is an example: f(n) ๏ฝ 2 2 ๏ซ lo g lo g n ๏ป f(n) is a step function sandwiched between n and ๐. Why is “g๏ญ” not true? However, with Simple Analytical Functions (SAF), i.e., functions composed of only constants, arithmetic, exponential, logarithm, and functional composition, such a “bizarre” situation will not arise, and hence, the statement requiring “g๏ญ” can be omitted from the premise of the theorem. 20 Simple Analytical Function (SAF) SAF : a function composed of a finite number of applications of constants, arithmetic, exponential, logarithm, & functional composition. Example: 4 n 2 n log log n ๏จ1 ๏ซ 6(log n ) / n ๏ฉ2 ๏ซ 7 n FACTS: A SAF f(n) has the following properties: a) log f(n) and ๏ถf(x)/๏ถx are also SAFs. b) has only a finite # of roots. (does not oscillate) c) is asymptotically sign invariant. (past its last real root) d) is asymptotically monotonic. (part (c) on its derivative) e) In ASSUME: f(n)๏ญ ๏ g(n)๏ญ f) when comparing SAFs asymptotically, “๏คn0๏ขn๏ณn0” can be replaced by the phrase “for infinitely many n” (“for i.m. n” for short). E.g., “for all n that are positive integer powers of 2”. (“past last real root”) Our short article says more on this topic. The next table shows a summary of ASSUME on SAF. 21 f(n) : a SAF S(n) = f(1) + f(2) + ··· + f(n) Sum Type ๐(๐) Geometric ๐(๐) ๐ (or exponential) Arithmetic (or polynomial) quasi Harmonic Bounded Tail ๐๐ฏ ๐ −๐ Example f(n) ๐บ(๐) 32๐ log ๐ ๐3 Θ(๐(๐)) ๐3.5 log ๐ Θ(๐๐(๐)) ๐ฅ๐จ๐ ๐ ๐ ๐ฏ ๐ ๐ −๐ ๐ −๐ 3 log ๐ ๐ ๐4 23๐ , 1 ๐2+log ๐ Θ log ๐+1 ๐ ๐๐ ๐ > −1 Θ(log log ๐) ๐๐ ๐ = −1 Θ 1 ๐๐ ๐ < −1 Θ(1) 22 A Simple form The SAFs we encounter, usually have the following simple form: f ( n ) ๏ฝ Q๏จt n log n ๏ฉ n d e for some real constants t>0, d, e. We classify S(n) for such f(n) as: t>1 t=1 Geometric (increasing) d > -1 Arithmetic d = -1 quasi Harmonic d < -1 Bounded tail (polynomially decreasing) 0<t<1 Bounded tail (geometrically decreasing) 23 RECURRENCE RELATIONS ๏ฌ2T๏จ n2 ๏ฉ ๏ซ Q(n) for n ๏พ O(1) T ( n) ๏ฝ ๏ญ for n ๏ฝ O(1) ๏ฎO(1) ๏ T (n) ๏ฝ Q(n log n) 24 The Iteration Method for n ๏ฝ 0 ๏ฌ 0 T( n ) ๏ฝ ๏ญ ๏ฎ T ( n ๏ญ 1 ) ๏ซ f ( n ) for n ๏ณ 1 The iteration method (or unwinding the recurrence): T(n) = f(n) + T(n –1) = f(n) + f(n-1) + T(n –2) = f(n) + f(n-1) + f(n-2) + T(n –3) f(n) is the driving function of the recurrence = … = f(n) + f(n-1) + f(n-2) + ··· + f(3) + f(2) + T(1) = f(n) + f(n-1) + f(n-2) + ··· + f(3) + f(2) + f(1) + T(0) = f(n) + f(n-1) + f(n-2) + ··· + f(3) + f(2) + f(1). Example: f(n) = Q(n) f(n) = Q(2n) ๏ T(n) = Q(n2). ๏ T(n) = Q(2n). 25 The Iteration Method for n ๏ผ 1 ๏ฌ0 T( n ) ๏ฝ ๏ญ n ๏ฎ 2T ๏จ 2 ๏ฉ ๏ซ n for n ๏ณ 1 T(n) = n + 2T(n/2) = n + 2 [n/2 + 2T(n/22)] = 2n + 22 T(n/22) = 2n + 22 [n/22 + 2 T(n/23)] = 3n + 23 T(n/23) = 3n + 23 [n/23 + 2T(n/24)] = 4n + 24 T(n/24) = … = kn + 2k T(n/2k) take k = 1+ ๏ซlog n๏ป, T(n/2k) = 0 = n(1 + ๏ซlog n๏ป) = Q(n log n ) 26 The Iteration Method ๏ฌ0 T( n ) ๏ฝ ๏ญ 2 n ๏จ ๏ฉ 2 T ๏ซ n ๏ฎ 2 for n ๏ผ 1 for n ๏ณ 1 T(n) = n2 + 2T(n/2) = n2 + 2 [(n/2)2 + 2T(n/22)] = n2(1+ ½) + 22 T(n/22) = n2(1+ ½) + 22 [(n/22)2 + 2 T(n/23)] = n2(1+ ½ + ½2) + 23 T(n/23) = n2(1+ ½ + ½2) + 23 [(n/23)2 + 2T(n/23)] = n2(1+ ½ + ½2 + ½3) + 24 T(n/24) = … = n2 (1+ ½ + ½2+ ½3 + ··· + ½k ) + 2k+1 T(n/2k+1) take k = ๏ซlog n๏ป, T(n/2k+1) = 0 = n2 (1+ ½ + ½3 + ½5 + ··· + ½k ) = n2 · Q(1) geometric decreasing = Q(n2 ) 27 The Recursion Tree Method for n ๏ผ 1 ๏ฌ0 T( n ) ๏ฝ ๏ญ n ๏ฎ2T ๏จ 2 ๏ฉ ๏ซ f ( n ) for n ๏ณ 1 T(n) f(n) f(n) T(n/2) T(n/2) f(n/2) 2f(n/2) f(n/2) T(n/4) T(n/4) T(n/4) T(n/4) f(n/4) f(n/4) f(n/4) f(n/4) T(n/8) f(n/8) T(n/8) T(n/8) f(n/8) T(n/8) T(n/8) f(n/8) f(n/8) f(n/8) T(n/16) T(n/8) T(n/8) f(n/8) f(n/8) 4f(n/4) T(n/8) f(n/8) 8f(n/8) T(n/16) T ( n) ๏ฝ f ( n) ๏ซ 2 f ( n 2 ) ๏ซ 4 f ( n 4 ) ๏ซ 8 f ( n 8 ) ๏ซ ๏ ๏ ๏ ๏ฝ ๏ซlog n ๏ป ๏ฅ i ๏ฝ0 2i f ๏จ ๏ฉ n 2i 28 The Recursion Tree Method for n ๏ผ 4 ๏ฌ0 T( n ) ๏ฝ ๏ญ n n ๏จ ๏ฉ ๏จ ๏ฉ ๏ซ n for n ๏ณ 4 T ๏ซ T ๏ฎ 4 2 CLAIM: T(n) = Q(n). Lower bound: T(n) ๏ณ W(n) obvious T(n) n n T(n/2) n/4 n/2 T(n/16) T(n/8) n/16 T(n/64) n/64 n/8 T(n/8) T(n/4) n/8 n/4 log2 n log4 n T(n/4) T(n/32) T(n/32) T(n/16) T(n/32) T(n/16) T(n/16) T(n/8) n/32 n/16 n/16 n/8 n/32 n/32 n/16 T(n/256) ๏ฃ 3n/4 ๏ฃ (3/4)2n ๏ฃ (3/4)3n T(n/16) T (n) ๏ฃ n ๏ซ ๏จ3 4 ๏ฉn ๏ซ ๏จ3 4 ๏ฉ n ๏ซ ๏จ3 4 ๏ฉ n ๏ซ ๏ ๏ ๏ ๏ 2 ๏จ 3 ๏ฉ Upper bound ๏ฝ n 1 ๏ซ ๏จ3 4 ๏ฉ ๏ซ ๏จ3 4 ๏ฉ ๏ซ ๏จ3 4 ๏ฉ ๏ซ ๏ ๏ ๏ ๏ฃ 4n ๏ฃ O(n). 2 3 29 The Recursion Tree Method for n ๏ผ 4 ๏ฌ0 T( n ) ๏ฝ ๏ญ n n ๏จ ๏ฉ ๏จ ๏ฉ ๏ซ n for n ๏ณ 4 T ๏ซ T ๏ฎ 4 2 This is a special case of the following recurrence: T(n) = T(an) + T(bn) + Q(n) where 0 ๏ฃ a < 1 and 0 ๏ฃ b < 1 are real constant parameters. FACT: (1) a ๏ซ b ๏ผ 1 (2) a ๏ซ b ๏ฝ 1 (3) a ๏ซ b ๏พ 1 ๏ T(n) = Q(n) [linear] ๏ T(n) = Q(n log n) ๏ T(n) = Q(nd) [super-linear poly] where d > 1 is the unique constant that satisfies the equation ad + bd = 1. [See the guess-&-verify method later & Exercise 9.] 30 Divide-&-Conquer MergeSort is the prototypical divide-&-conquer algorithm. Here is a high level description: Algorithm MergeSort( S ) Pre-Condition: § John von Neumann [1945] input S is a sequence of numbers Post-Condition: output is a sorted permutation of the input sequence Base: if |S| ๏ฃ 1 then return S Divide: Divide S into its left and right halves S=๏กL, R๏ฑ, |L|๏ป|R|๏ป ½|S| Conquer: L’ ๏ฌ MergeSort( L ); R’ ๏ฌ MergeSort( R ) Combine: S’ ๏ฌ Merge( L’, R’ ) Output: return S’ end 31 MergeSort Example INPUT 23 15 15 23 41 82 41 82 15 23 41 82 37 92 37 92 66 31 31 66 31 37 66 92 15 23 31 37 41 66 82 92 34 73 34 73 25 19 19 25 19 25 34 73 33 15 15 33 16 58 16 58 15 16 33 58 15 16 19 25 33 34 58 73 15 15 16 19 23 25 31 33 34 37 41 58 66 73 82 92 OUTPUT 32 Algorithm MergeSort( A[1..n] ) B[0..n+1] ๏ฌ auxiliary array for merging MS(1,n) end MERGE Loop Invariant: procedure MS( s, f ) § sort A[s..f] Base: Divide: Conquer: Combine: n merged so far A s s-1 f k i m-1 m end j left half f+1 ๏ฅ ๏ฅ B if s ๏ณ f then return m ๏ฌ ๏ฉ(s+f)/2๏น MS( s, m –1 ) ; MS( m, f ) Merge( s, m, f ) right half Merge time = Q(n), where n = f – s +1. procedure Merge( s, m, f ) B[s-1 .. m-2] ๏ฌ A[s,m-1] B[m .. f] ๏ฌ A[m,f] B[m-1] ๏ฌB[f+1]๏ฌ๏ฅ § barrier sentinel i ๏ฌ s-1 ; j ๏ฌ m for k ๏ฌ s .. f do if B[i] ๏ฃ B[j] then A[k] ๏ฌ B[i]; i++ else A[k] ๏ฌ B[j]; j++ end 33 Algorithm MergeSort( A[1..n] ) B[0..n+1] ๏ฌ auxiliary array for merging MS(1,n) end MergeSort Recurrence: T(n) = Q(1) ๏ขn ๏ฃ 1 T(n) = T( ๏ซn/2๏ป ) + T( ๏ฉn/2๏น ) + Q(n) ๏ขn>1 procedure MS( s, f ) § sort A[s..f] Base: Divide: Conquer: Combine: if s ๏ณ f then return m ๏ฌ ๏ฉ(s+f)/2๏น MS( s, m –1 ) ; MS( m, f ) Merge( s, m, f ) end procedure Merge( s, m, f ) Simplified Recurrence: T(n) = 2 T(n/2) + Q(n) for n > 1 T(n) = Q(1) for n ๏ฃ 1 Solution: T(n) = Q( n log n). B[s-1 .. m-2] ๏ฌ A[s,m-1] B[m .. f] ๏ฌ A[m,f] B[m-1] ๏ฌB[f+1]๏ฌ๏ฅ § barrier sentinel i ๏ฌ s-1 ; j ๏ฌ m for k ๏ฌ s .. f do if B[i] ๏ฃ B[j] then A[k] ๏ฌ B[i]; i++ else A[k] ๏ฌ B[j]; j++ end 34 Divide-&-Conquer Recurrence ๐ ๐ = ๐๐ ๐๐ + ๐(๐) Θ 1 ∀๐ > 1 ∀๐ ≤ 1 a = # of recursive calls (a > 0) n/b = size of each sub-instance called recursively (b > 1) f(n) = cost of divide + combine steps (every thing except the recursive calls). In MergeSort we have: a = 2 recursive calls of size n/b = n/2 each. Cost of divide = Q(1) Cost of Combine, i.e., merge, is Q(n). So, f(n) = Q(n). 35 ๏ฌaT ( bn ) ๏ซ f ( n ) T( n ) ๏ฝ ๏ญ ๏ฎQ( 1 ) ๏ขn ๏พ 1 ๏ขn ๏ฃ 1 f(n) f(n) a f(n/b) f(n/b2) Recursion Tree: f(n/b) f(n/b2) af(n/b) f(n/b) f(n/b2) a2f(n/b2) f(n/b2) k = ๏ฉlogbn๏น f(n/bk-1) Q(1) f(n/bk-1) Q(1) Q(1) Q(1) ak-1f(n/bk-1) h(n) = Q(n logba ) ak a logb n = n logb a (compare their logarithms) h(n) = ak ·Q(1) = Q(ak) = Q(a๏ฉlogbn๏น ) = Q๏จalogbn) = Q๏จnlogba) k = ๏ฉlogbn๏น = Q( log n). 36 ๏ฌaT ( bn ) ๏ซ f (n) T ( n) ๏ฝ ๏ญ ๏ฎQ(1) ๏ขn ๏พ 1 ๏ขn ๏ฃ 1 f(n) T(n) = S(n) + h(n) = Q( max{S(n), h(n)}) h(n) = Q๏จ n k ๏ญ1 S ( n) ๏ฝ ๏ฅ a i f i ๏ฝ0 af(n/b) logba ) S(n) = ๏จ๏ฉ S a2f(n/b2) n bi ๏ฝ f (n) ๏ซ af ( bn ) ๏ซ a 2 f ( bn2 ) ๏ซ ๏ ๏ ๏ ๏ซ a k ๏ญ1 f ๏จ ๏ฉ n b k ๏ญ1 ak-1f(n/bk-1) h(n) = Q(n logba ) a logb n = n logb a (compare their logarithms) h(n) = ak ·Q(1) = Q(ak) = Q(a๏ฉlogbn๏น ) = Q๏จalogbn) = Q๏จnlogba) k = ๏ฉlogbn๏น = Q( log n). 37 ๏ฌaT ( bn ) ๏ซ f (n) ๏ขn ๏พ 1 T ( n) ๏ฝ ๏ญ ๏ขn ๏ฃ 1 ๏ฎQ(1) T(n) = S(n) + h(n) Special Solution: contribution of the internal nodes of the recursion tree. Homogeneous Solution: contribution of the leaves of the recursion tree. ๏ฌah( bn ) ๏ขn ๏พ 1 h ( n) ๏ฝ ๏ญ ๏ฎQ(1) ๏ขn ๏ฃ 1 Define: Q ( n) ๏ฝ S ( n) , h( n) h(n) = Q๏จ n logba ) r ( n) ๏ฝ ๏ฌQ( bn ) ๏ซ r (n) ๏ขn ๏พ 1 Q ( n) ๏ฝ ๏ญ ๏ขn ๏ฃ 1 ๏ฎ0 f ( n) h( n) T(n) = h(n)( Q(n) + 1 ) Q(n) ๏ฝ r(n) ๏ซ r๏จbn ๏ฉ ๏ซ r ๏จ ๏ฉ๏ซ r ๏จ ๏ฉ๏ซ ๏ ๏ ๏ n b2 n b3 f(n) is a SAF ๏ r(n) is a SAF 38 Q ( n) ๏ฝ S ( n) , h( n) r ( n) ๏ฝ f ( n) h( n ) T(n) = h(n)( Q(n) + 1 ) ๏ r(n) is a SAF โช Let n = bk for integer k = Q(log n). ๏ฆ k i ๏ถ Q(n) ๏ฝ Q๏ง ๏ฅ r b ๏ท ๏จ i ๏ฝ1 ๏ธ r(n) r(bi) Q(n)=Q( ? ) T(n)=Q( ? ) W(n+e) W(b+ie) r(n) f(n) O(n-e) O(b-ie) 1 h(n) Q(loge n) Q(ie) r(n) ๏ k f(n)๏log n Q(log-1 n) Q(1/i) log k h(n)๏log log n O(loge n) O(i-e) 1 h(n) f(n) is a SAF e > -1 e < -1 ๏จ ๏ฉ 39 ๏ฌaT ( bn ) ๏ซ f ( n ) T( n ) ๏ฝ ๏ญ ๏ฎQ( 1 ) ๏ขn ๏พ 1 ๏ขn ๏ฃ 1 h(n) = Q๏จ n logba ) THE MASTER THEOREM Suppose f(n) is a SAF, and e ๏พ 0 & e are constant reals. ๐(๐) / ๐(๐) ๐ป(๐) ๐ ๐+๐บ f(n) Compare with CLRS. See Exercise 4. h(n) » ๐ฏ ๐ฅ๐จ๐ ๐ ๐ (e > -1) ๐ฏ ๐ฅ๐จ๐ ๐ ๐ (e = -1) ๐ ๐ฅ๐จ๐ ๐ ๐ (e < -1) ๐ ๐−๐บ f(n) ๏ป h(n) f(n) « h(n) ๐ฉ(๐(๐)) ๐ฉ(๐(๐) log ๐) = ๐ฉ(๐log๐๐ log ๐+1 ๐) ๐ฉ(โ(๐) log log ๐) = ๐ฉ(๐log๐๐ log log ๐) ๐ฉ(โ(๐)) = ๐ฉ(๐log๐๐) 40 ๏ฌaT ( bn ) ๏ซ f ( n ) T( n ) ๏ฝ ๏ญ ๏ฎQ( 1 ) ๏ขn ๏พ 1 ๏ขn ๏ฃ 1 f(n) = Q( tn nd log e n) T(n) = Q(f(n)) bd > a bd =a ๏ d = logb a bd < a 0<t<1 constants t > 0, d, e. T(n) = Q(f(n)) t>1 t=1 h(n) = Q๏จ n logba ) e > -1 T(n) = Q(f(n) log n) e = -1 T(n) = Q(h(n) log log n) e < -1 T(n) = Q(h(n)) T(n) = Q(h(n)) T(n) = Q(h(n)) 41 CSE 3101 log ๐ฅ๐ฆ = log ๐ฅ + log ๐ฆ log ๐ฅ ๐ฆ = ๐ฆ log ๐ฅ log ๐ฆ ๐ฅ = log ๐ฅ / log ๐ฆ log(1 ± ๐ฅ) = ±Θ(๐ฅ) for ๐ฅ = ๐(1) For f ↑: ๐ ๐๐ ๐=1 ๐ ๐ ๐ ๐ฅ ๐๐ฅ ≤ ๐−1 Sums & Recurrences Θ ๐๐+1 = Θ(log ๐) Θ 1 Fact Sheet if d > −1 if d = −1 if d < −1 ๐+1 ๐ ๐ ≤ ๐=0 ๐ For f ↓: ๐ ๐=๐ ๐ฅ๐ = ๐ ๐ ๐ฅ ๐๐ฅ ๐ ๐ฅ ๐๐ฅ ≥ ๐−1 ASSUME [ ASymptotic SUmmation Made Easy ] ๐ ๐ = ๐๐=1 ๐ ๐ ๐๐๐ ๐ ๐ ↑ = ๐ 1 g(n) ← ๐ − ๐ ๐ฅ ๐+1 − 1 if 0 ≠ |๐ฅ| ≠ 1 ๐ฅ−1 Θ ๐ฅ๐ if x > 1 Θ 1 if 0 < x < 1 ๐ ๐+1 ๐ ๐ ≥ ๐=๐ where log ๐ ๐ = log ๐ ๐ − Θ 1 : ๐ ↑ ๐๐ง๐ ๐ ↑ ⇒ ๐ ๐ง = ๐ฏ ๐ ๐ง ๐ ๐ง ๐ Sum Type ๐(๐) : ๐ ๐บ๐จ๐ญ ๐บ ๐ = Geometric 2Ω(๐) ๐ฃ( ๐ ๐ ) Arithmetic ๐Θ 1 ๐ฃ( ๐ ๐ ๐ ) Θ(๐ Harmonic ๐−Ω 1 Bounded Tail MASTER METHOD: ๐ ๐ โถ ๐(๐) ๐ ๐ โซ ๐(๐) ๐ ๐ ≈ ๐(๐) ๐ ๐ โช ๐(๐) −1 −1 ๐ป(๐) = ๐(๐) ๐(๐) ๐ log ๐) −1 , ๐=๐ ๐ฃ ๐ฅ๐จ๐ ๐+๐ ๐ ๐ฏ ๐ฅ๐จ๐ ๐ฅ๐จ๐ ๐ ๐ฏ ๐ e = a real constant ๐๐ ๐๐ฃ๐๐ ๐ ๐−1 log ๐ ๐ , ๐ < −1 ๐๐ป ๐๐ + ๐(๐) ∀๐ > ๐ , ๐ฃ(๐) ∀๐ ≤ ๐ ๐ ๐ฅ ๐๐ฅ ๐ . ๐(๐) ๐๐ ๐ > −๐ ๐๐ ๐ = −๐ ๐๐ ๐ < −๐ ๐ฃ(๐) โ(๐) = Θ ๐log๐ ๐ , ๐ ๐ โถ ๐ ๐๐จ๐ญ, constants ๐ > 0 & ๐. ๐ป ๐ = = Ω(๐๐ ) ๐ฃ( ๐ ๐ ) Θ(log ๐ ๐ ) , ๐ > −1 ๐ฃ( ๐ ๐ ๐ฅ๐จ๐ ๐ ) Θ log −1 ๐ , ๐ = −1 ๐ฃ( ๐ ๐ ๐ฅ๐จ๐ ๐ฅ๐จ๐ ๐ ) ๐ ๐−๐ ๐ฃ( ๐ ๐ ) ๐๐ ๐๐ฃ๐๐ ๐(log ๐ ๐ ) , ๐ < −1 42 ๏ฌaT ( bn ) ๏ซ f ( n ) T( n ) ๏ฝ ๏ญ ๏ฎQ( 1 ) ๏ขn ๏พ 1 ๏ขn ๏ฃ 1 h(n) = Q๏จ n logba ) Example 1: Assume a = 2, b = 4, f(n) = Q(nd log2 n). Express T(n) based on the constant parameter d. Solution: logb a = log4 2 = ½ ๏ h(n) = Q( ๐) d > ½ ๏ T(n) = Q(f(n)) = Q(๐๐ ๐๐๐2 ๐) d = ½ ๏ T(n) = Q(f(n) log n) = Q( ๐ ๐๐๐3 ๐) d < ½ ๏ T(n) = Q(h(n)) = Q( ๐) 43 ๏ฌaT ( bn ) ๏ซ f ( n ) T( n ) ๏ฝ ๏ญ ๏ฎQ( 1 ) ๏ขn ๏พ 1 ๏ขn ๏ฃ 1 h(n) = Q๏จ n logba ) Example 2: Assume b = 3, f(n) = Q(n4 / log n). Express T(n) based on the constant parameter a. Solution: h(n) = Q(nlog3a) , bd = 34 = 81 a < 81 ๏ T(n) = Q(f(n)) = Q(n4 / log n) a = 81 ๏ T(n) = Q(h(n) log log n) = Q(n4 log log n) a > 81 ๏ T(n) = Q(h(n)) = Q(nlog3a) 44 Guess-&-Verify BASIC INGRIDIANTS: ๏ Guess a solution to the recurrence. ๏ Verify it by mathematical induction. The induction variable must be a natural number. E.g., height of the recursion tree (max recursion depth), or size of the recursion tree (# times you apply the recurrence). n may also be a candidate if it “stays” integer. ๏ If you spot a place where the verification does not go through, examine it to help revise your guess, and try again. ๏ One key guiding principle, taken with a grain of salt, is to first make sure the leading term (with its constant coefficient) matches on the LHS and RHS of the recurrence. That will show you to guess higher or lower the next time! After figuring out the leading term, you can apply the same principle to find the lower order terms. 45 Guess-&-Verify: Example 1 ๐ ๐ = • Clearly ๐ ๐ = Ω ๐2 . • Guess # 1: 4๐ Is ๐ 2 + ๐2 1 ๐๐๐ ๐ > 1 ๐๐๐ ๐ ≤ 1 ๐ ๐ = ๐ ๐2 ? ๐ ๐ = ๐๐2 + ๐ ๐ const. ๐ > 0, ๐ ๐ = ๐ ๐2 lower order terms (to be determined) • Plug this guess in the recurrence and verify: ๐๐2 +๐ ๐ ? =? 4 ๐ ๐ 2 2 • LHS leading term < RHS leading term +๐ ๏ ๐ ๐ ? =? 4๐ ๐ 2 + ๐2 ๐ 2 + ๐2 = (๐ + 1)๐2 + 4๐ ๐ 2 guess is too low. 46 Guess-&-Verify: Example 1 ๐ ๐ = • Guess # 2: 4๐ ๐ 2 + ๐2 1 ๐๐๐ ๐ > 1 ๐๐๐ ๐ ≤ 1 ๐ ๐ = ๐๐2+๐ + ๐ ๐ const. ๐ > 0, ๐ > 0, ๐ ๐ lower order terms (to be determined) • Plug this guess in the recurrence and verify: ๐๐2+๐ +๐ ๐ ? =? 4 ๐ ๐ 2+๐ 2 • LHS leading term > RHS leading term +๐ ๏ ๐ ๐ ? =? 4๐ ๐ 2 + ๐2 ๐ 2 1 2๐ ๐๐2+๐ + 4๐ + ๐2 = ๐ 2 +๐2 guess is too high. 47 Guess-&-Verify: Example 1 ๐ ๐ = • Guess # 3: 4๐ ๐ 2 + ๐2 ๐๐๐ ๐ > 1 1 ๐ ๐ = ๐๐2 log ๐ + ๐ ๐ ๐๐๐ ๐ ≤ 1 const. ๐ > 0, ๐ ๐ lower order terms (to be determined) ๐ ๐ ? =? 4๐ • Plug this guess in the recurrence and verify: ๐๐2 log ๐ +๐ ๐ ? =? 4 ๐ ๐ 2 ๐ log 2 2 ๐ 2 +๐ = ๐๐2 log ๐ + 4๐ • LHS leading term = RHS leading term ๐ 2 ๐ 2 + ๐2 + ๐2 + (1 − ๐)๐2 ๏ correct guess of high order term. • Solve the residual recurrence for ๐(๐) (with revised boundary condition): ๐ ๐ = 4๐ ๐ 2 + (1 − ๐)๐2 = 4๐ • Solution: ๐ ๐ = Θ ๐2 and ๐ 2 (must have 1 − ๐ โฏ 0, set ๐ = 1, why?) ๐ ๐ = ๐2 log ๐ + Θ ๐2 48 Guess-&-Verify: Example 2 Recurrence: T(n) = 2T( ๏ซn/2๏ป ) + n2 , T(n) ๏ณ n2 ๏ณ W(n2) is obvious. Let’s see if O(n2) is an upper-bound: Guess: T(n) ๏ฃ an2 + bn + c Basis (n = 0): T(0) = 0 ๏ฃ T(0) = 0 a02 (for some constants a>0, b, c to be determined) + b0 + c we need c๏ณ0 Ind. Step (n ๏ณ1): T(n) = 2T( ๏ซn/2๏ป ) +n2 ๏ฃ 2 [ a ๏ซn/2๏ป2 + b ๏ซn/2๏ป + c ] + n2 ๏ฌ๏พ by induction hypothesis if n is even ๏พ๏พ๏พ๏ฎ = 2 [ a(n/2)2 + b(n/2)+c ] + n2 ๏ฌ๏พ not “๏ฃ” for odd n, if b<0 = (1+ a/2)n2 + bn + 2c ๏ฃ an2 + bn + c ๏ฌ๏พ๏พ๏พ๏พ๏พ our guessed upper-bound We need: for all even n ๏ณ1: an2 + bn + c ๏ณ (1+ a/2)n2 + bn + 2c, i.e., (a/2 – 1)n2 – c ๏ณ 0. 49 Guess-&-Verify: Example 2 Recurrence: T(n) = 2T( ๏ซn/2๏ป ) + n2 , T(n) ๏ณ n2 ๏ณ W(n2) is obvious. Let’s see if O(n2) is an upper-bound: Guess: T(n) ๏ฃ an2 + bn + c Basis (n = 0): T(0) = 0 ๏ฃ T(0) = 0 a02 (for some constants a>0, b, c to be determined) + b0 + c we need c๏ณ0 Ind. Step (n ๏ณ1): T(n) = 2T( ๏ซn/2๏ป ) +n2 ๏ฃ 2 [ a ๏ซn/2๏ป2 + b ๏ซn/2๏ป + c ] + n2 ๏ฌ๏พ by induction hypothesis if n is odd ๏พ๏พ๏พ๏ฎ = 2 [ a((n-1)/2)2 + b((n-1)/2)+c ] + n2 = (1+ a/2)n2 + (b-a)n + (2c-b+a/2) ๏ฃ an2 + bn + c ๏ฌ๏พ๏พ๏พ๏พ๏พ our guessed upper-bound We need: for all odd n ๏ณ1: an2 + bn + c ๏ณ (1+ a/2)n2 + (b-a)n + (2c-b+a/2), i.e., (a/2 – 1)n2 + an + (b – c – a/2) ๏ณ 0. We need: for all even n ๏ณ1: an2 + bn + c ๏ณ (1+ a/2)n2 + bn + 2c, i.e., (a/2 – 1)n2 – c ๏ณ 0. a=2 b = -1 c=0 works for all n ๏ณ 0. n2 ๏ฃ T(n) ๏ฃ 2n2 - n 50 Guess-&-Verify: Example 2 Recurrence: T(n) = 2T( ๏ซn/2๏ป ) + n2 , T(0) = 0 Exercise 8: Using this same method, verify the following tighter LOWER BOUND: ๏ขn๏ณ1: 2n2 – n – 2n log n ๏ฃ T(n) ๏ฃ 2n2 – n. ๏ T(n) = 2n2 – n – O(n log n). For more examples of guess-&-verify see Lecture Note 3 & Sample Solutions …. a=2 b = -1 c=0 works for all n ๏ณ 0. n2 ๏ฃ T(n) ๏ฃ 2n2 - n 51 Full History Recurrence Such recurrences often arise in average-case analysis. Example 1 : T (n) ๏ฝ ๏ฅi ๏ฝ0 T (i ) ๏ซ f (n), n ๏ญ1 ๏ขn ๏ณ 0 Values of T(n) for some small n: T(0) = f(0) T(1) = T(0) + f(1) = f(0) + f(1) T(2) = T(0) + T(1) + f(2) = 2f(0) + f(1) + f(2) T(3) = T(0) + T(1) + T(2) + f(3) = 4f(0) + 2f(1) + f(2)+ f(3) T(4) = T(0) + T(1) + T(2) + T(3) + f(4) = 8f(0) + 4f(1) + 2f(2) + f(3) + f(4) 2 Example 2 : T (n) ๏ฝ n ๏ฅ n ๏ญ1 i ๏ฝ0 T (i ) ๏ซ n, ๏ขn ๏ณ 0 This is the QuickSort expected time recurrence (details shown later in LS5) 52 Full History Recurrence Example 1: T ( n ) ๏ฝ ๏ฅi๏ฝ0 T (i ) ๏ซ f ( n ), ๏ขn ๏ณ 0 n๏ญ1 A Solution Method: n ๏ฌ n -1: T ( n ๏ญ1) ๏ฝ ๏ฅi๏ฝ0 T ( i ) ๏ซ f ( n ๏ญ1), ๏ขn ๏ญ1๏ณ 0 (i.e., ๏ขn ๏ณ1 ) n ๏ญ2 Subtract: T ( n ) ๏ญ T ( n ๏ญ1) ๏ฝ T ( n ๏ญ1) ๏ซ f ( n ) ๏ญ f ( n ๏ญ1), ๏ขn ๏ณ1 ๏ฌ2T ( n ๏ญ1) ๏ซ ๏f ( n ) ๏ญ f ( n ๏ญ1) ๏ ๏ขn ๏ณ1 Rearrange: T ( n ) ๏ฝ ๏ญ for n ๏ฝ 0 ๏ฎf ( 0 ) Now there is only one appearance of T(.) on the RHS. Continue with conventional methods. 53 Variable Substitution: Example 1 Sometimes we can considerably simplify the recurrence by a change of variable. T(n) = T(n/2) + log n [assume the usual boundary condition T(O(1))=O(1).] Change of variable: n = 2m, i.e., m = log n. Now T(n) = T(2m). It is some function of m. Rename it S(m). That is, T(n) = T(2m) = S(m). The recurrence becomes: S(m) = S(m-1) + m. By the iteration method: S(m) = Q(m2). So, T(n) = T(2m) = S(m) = Q(m2) = Q(log2 n). T(n) = Q(log2 n). 54 Variable Substitution: How? Recurrence T( n ) g(m) Boundary Condition = T( n/2 ) + f(n) g(m-1) T(n) = 0 for n < 1 g(m) Rename: n = g(m), n/2 = g(m-1), S(m) = T(g(m)), h(m) = f(g(m)) Now solve 2 recurrences: 1) g(m) = 2g(m-1), g(0) = 1 ๏ฌ our choice 2) ๏ g(m) = 2mg(0) = 2m . S(m) = S(m-1) + h(m) ๏ S(m) = h(0)+h(1)+ ··· +h(m). S(0) = T(g(0)) = T(1) = f(1) = h(0) Now back substitute: T(n) = T(g(m)) = S(m) = S(g-1(n)), where g-1 is functional inverse of g. See another example on the next page 55 Variable Substitution: Example 2 T(๐) = T( ๐) + 1 g(m) g(m-1) Rename: n = g(m), ๐ = g(m-1), S(m) = T(g(m)). g (m) ๏ฝ g(m ๏ญ1) ๏ฝ g (m ๏ญ 2) ๏ฝ g(m ๏ญ 3) ๏ฝ ๏ ๏ ๏ ๏ฝ g (0) 2 22 g(0) = 2 ๏ n = g(m) = 22m S(m) = S(m-1)+1 S(0) = T(2) = Q(1) 23 2m ๏ m = log log n. ๏ S(m) = Q(m) = Q(log log n). T(n) = Q(log log n). 56 Exercises 57 1. For each pseudo-code below derive the simplified asymptotic running time in Q(?) notation. (a) for i ๏ฌ 1 .. n do for j ๏ฌ 1 .. 2*i do print( i+j ) (b) for i ๏ฌ 1 .. n do for j ๏ฌ 2*i .. n do print( i+j ) (c) for i ๏ฌ 1 .. n do j๏ฌ1 while j ๏ฃ i do j ๏ฌ j+3 (d) for i ๏ฌ 1 .. n do j๏ฌ1 while i+j ๏ฃ n do j ๏ฌ j+3 (e) for i ๏ฌ 1 .. n do j๏ฌ1 while j ๏ฃ i do j ๏ฌ j+j (f) for i ๏ฌ 1 .. n do j๏ฌ1 while j*j ๏ฃ i do j ๏ฌ j+1 (g) for i ๏ฌ 1 .. n do j๏ฌ2 while j ๏ฃ n do j ๏ฌ j*j (h) for i ๏ฌ 1 .. n do j๏ฌ2 while j ๏ฃ i do j ๏ฌ j*j (i) for i ๏ฌ 1 .. n do j๏ฌ1 while i*j ๏ฃ n do j ๏ฌ j+1 (j) for i ๏ฌ 1 .. n do j๏ฌ1 while i*i ๏ณ j do j ๏ฌ j+1 for i ๏ฌ 1 .. n do j๏ฌn while i < j*j do j ๏ฌ j – 2 (l) for i ๏ฌ 1 .. n do j๏ฌn while i ๏ฃ j*j do j ๏ฌ j div 2 (k) 58 59 4. The Master Theorem: CLRS versus these Slides: The case “f(n)/h(n) = W(n๏ซe)” of the Master Theorem [page 94 of CLRS] has the following extra condition: (*) if af(n/b) < cf(n) for some constant c < 1 and all sufficiently large n. The same theorem on page 40 of these slides instead uses the condition: (**) f(n) is a SAF. Show that (**) implies (*) in the case “f(n)/h(n) = W(n๏ซe)” . [Hint: Consider the SAF r(n) = f(n)/h(n) ๏ฝ W(n+e). So, r(bi) = W(b+ie) is increasing at least exponentially as a function of i. Then, with some extra derivation, show that r(n/b) / r(n) < c < 1 for some constant c and all sufficiently large n. c = b –e/2 works. ] 5. Exercise 6 of Lecture Note 3 (page 9) solves the recurrence T(n) = 3T(n/3 +5) + n/2 by the guess-&-verify method. Such recurrences can also be solved by the variable substitution method. Consider the somewhat generalized recurrence T(n) = a T(n/b + c) + f(n), where a>0, b>1, and c are constants, and f(n) is the driving function which is a SAF. a) Show the substitution S(n) = T(n + cb/(b-1)) results in the new recurrence S(n) = aS(n/b) + f(n + cb/(b-1)). [Now this can be solved by the Master Theorem.] b) Use this method to solve the recurrence T(n) = 4T(n/2 +7) + n2. 60 (m) T (n) ๏ฝ ๏ฅ n ๏ญ1 i ๏ฝ0 i ๏ T (i ) ๏ซ n 61 (n) ๐ ๐ = ๐ ๐−1 ๐ ๐−1 +1 (o) ๐ ๐ = ๐−1 ๐ ๐ ๐−1 +1 62 8. Show that the recurrence ๐ ๐ = 2๐ ๐ 0 = 0 ๐ 2 + ๐2 , has the solution: ๐(๐) = 2๐2 – ๐ – ๐(๐ ๐๐๐ ๐). 9. Show that the recurrence ๐ a๐ + ๐ b๐ + ๐ ๐ ๐ = ๐ ๐๐๐ ๐ ≥ ๐๐ ๐๐๐ 0 ≤ ๐ < ๐๐ for any real constants 0 ๏ฃ a < 1, 0 ๏ฃ b < 1, c > 0, ๐๐ > 0, has the following solutions: a) a ๏ซ b ๏ผ 1 ๏ T(n) = Q(n) b) a ๏ซ b ๏ฝ 1 ๏ T(n) = Q(n log n) c) a ๏ซ b ๏พ 1 ๏ T(n) = Q(nd), where d > 1 is the unique constant that satisfies the equation: ad + bd = 1. 63 64 12. Derive the most simplified answers to the following summations. Mention which methods you used to derive your solution. a) log ๐ ๐=1 b) ๐ ๐=1 (2๐+7)5 2๐ ๐=1 (3๐+4) 8๐ +3๐ 5 (9๐−1)3 c) 2๐ ๐=1 2๐ +๐ ๐ ๐=1 ๐4 d) ๐ ๐=1 ๐4 ๐=1 e) ๐2 ๐=1 log ๐ ๐=1 2๐ ๐ + 7๐ 2 5๐ + 4๐ 6 log ๐ = Θ ? . = Θ ? . = Θ ? . (๐ 2 +๐)4 2๐ log ๐+5 ๐ 73๐ +5 ๐ 2 2๐ 3 +9๐ log ๐ 2 = Θ ? . = Θ(? ). 65 13. Recursion Time Analysis: A certain recursive algorithm takes an input list of n elements. Divides the list into n sub-lists, each with n elements. Recursively solves each of these n smaller sub-instances. Then spends an additional Θ(n) time to combine the solutions of these sub-instances to obtain the solution of the main instance. As a base case, if the size of the input list is at most a specified positive constant, then the algorithm solves such a small instance directly in Θ(1) time. a) Express the recurrence relation that governs T n , the time complexity of this algorithm. b) Derive the solution to this recurrence relation: T n = Θ(? ). Mention which methods you used to derive your solution. 66 END 67