CS 253: Algorithms
Chapter 4
Divide-and-Conquer
Recurrences
Master Theorem
Credit: Dr. George Bebis
Recurrences and Running Time

Recurrences arise when an algorithm contains recursive calls to itself

Running time is represented by an equation or inequality that
describes a function in terms of its value on smaller inputs.
T(n) = T(n-1) + n

What is the actual running time of the algorithm? i.e. T(n) = ?

Need to solve the recurrence
◦ Find an explicit formula of the expression
◦ Bound the recurrence by an expression that involves n
Example Recurrences

T(n) = T(n-1) + n
Θ(n2)
Recursive algorithm that loops through the input to eliminate one item

T(n) = T(n/2) + c
Θ(lgn)
Recursive algorithm that halves the input in one step

T(n) = T(n/2) + n
Θ(n)
Recursive algorithm that halves the input but must examine every item
in the input

T(n) = 2T(n/2) + 1
Θ(n)
Recursive algorithm that splits the input into 2 halves and does a
constant amount of other work
BINARY-SEARCH

Finds if x is in the sorted array A[lo…hi]
Alg.: BINARY-SEARCH (A, lo, hi, x)
if (lo > hi)
1
2
3
4
return FALSE
2 3 5 7
mid  (lo+hi)/2
if x = A[mid]
lo
return TRUE
if ( x < A[mid] )
BINARY-SEARCH (A, lo, mid-1, x)
if ( x > A[mid] )
BINARY-SEARCH (A, mid+1, hi, x)
5
6
9
7
8
10 11 12
mid
hi
Example 1
A[8] = {1, 2, 3, 4, 5, 7, 9, 11}
lo = 1
hi = 8
x=7
1
2
3
4
5
6
7
8
1
2
3
4
5
7
9
11
5
6
7
1
2
3
4
5
7
9
mid = 4, lo = 5, hi = 8
8
11
mid = 6, A[mid] = x
Found!
Example 1I
A[8] = {1, 2, 3, 4, 5, 7, 9, 11}
lo = 1 hi = 8
x=6
1
2
3
4
5
6
7
8
1
2
3
4
5
7
9
11
high
low
1
lo = 1, hi = 8, mid = 4. A[4]=4
2
3
4
5
7
9
11
lo = 5, hi = 8, mid = 6, A[6]=7
high
low
1
2
3
4
5
7
9
11
lo = 5, hi = 5, mid = 5, A[5]=5
1
2
3
4
5
7
9
11
lo = 6, hi = 5  NOT FOUND!
high
low
Analysis of BINARY-SEARCH
Alg.: BINARY-SEARCH (A, lo, hi, x)
if (lo > hi)
return FALSE
mid  (lo+hi)/2
if x = A[mid]
return TRUE
if ( x < A[mid] )
BINARY-SEARCH (A, lo, mid-1, x)
if ( x > A[mid] )
BINARY-SEARCH (A, mid+1, hi, x)
T(n) = c + T(n/2)
constant time: c1
constant time: c2
constant time: c3
same problem of size n/2
same problem of size n/2
Methods for Solving Recurrences

Iteration method

Recursion-tree method

Master method
8
The Iteration Method
Convert the recurrence into a summation and solve it using a known series
Example:
T(n) = c + T(n/2)
T(n) = c + T(n/2)
= c + c + T(n/4)
= c + c + c + T(n/8)
= c + c + c + c + T(n/24)
Assume n=2k then k = lg n and
T(n) = c + c + c + c + c + … + T(n/2k)
(k times)
T(n) =
T(n) = clg n
k *c
+ T(1)
Iteration Method – Example 2
T(n) = n + 2T(n/2)
Assume n=2k  k = lg n
T(n) = n + 2T(n/2)
= n + 2(n/2 + 2T(n/4))
= n + n + 4T(n/4)
= n + n + 4(n/4 + 2T(n/8))
= n + n + n + 8T(n/8)
T(n) = 3n + 23T(n/23)
= kn + 2kT(n/2k)
= nlgn + nT(1)
T(n) = O(nlgn)
Methods for Solving Recurrences

Iteration method

Recursion-tree method

Master method
11
The recursion-tree method
Convert the recurrence into a tree:
◦ Each node represents the cost incurred at various levels of recursion
◦ Sum up the costs of all levels
Used to “guess” a solution for the recurrence
Example 1
W(n) = 2W(n/2) + n2

Subproblem size at level i = n/2i

At level i: Cost of each node = (n/2i)2

h = Height of the tree  n/2h=1  h = lgn

Total cost at all levels:
lg n
W (n) 

i0
n
2
2
i
lg n
n
2

i0
 W(n) = O(n2)
i
i
# of nodes = 2i

1
1
1
2
2
2
 2n
   n    n
1
2
i0  2 
1
2
Total cost = (n2/2i)
Example 2
T(n) = 3T(n/4) + cn2

Subproblem size at level i = n/4i

At level i: Cost of each node= c(n/4i)2 # of nodes= 3i Total cost =cn2(3/16)i

h = Height of the tree  n/4h=1  h = log4n

Total cost at all levels:
log 4 n  1
T (n) 

i0
i
(last level has 3log4n = nlog43 nodes)
 3 
log 3
2

 cn   n 4 
 16 
 T(n) = O(n2)


 
i0
i
1
 3 
log 3
log 3
2
2
2
cn   n 4  O ( n )

 cn   n 4 
3
 16 
1
16




Example 3
W(n) = W(n/3) + W(2n/3) + n
• The longest path from the root to a leaf:
n  (2/3)n  (2/3)2 n  …  1
(2/3)in =1

i = log3/2n
• Cost of the problem at level i = n
• Total cost:
W ( n )  n (log 3 / 2 n )  n
lg n
 O ( n lg n )
lg(3 / 2)
via further analysis W(n) =Θ(nlgn)
15
Methods for Solving Recurrences

Iteration method

Recursion-tree method

Master method
16
Master Theorem

“Cookbook” for solving recurrences of the form:
n
T ( n )  aT    f ( n )
b
where a  1, b  1, and f(n)  0
Idea: compare f(n) with nlogba

f(n) is asymptotically smaller or larger than nlogba by a
polynomial factor n
OR

f(n) is asymptotically equal with nlogba
Master Theorem
n
T ( n )  aT 
b

  f (n)

where a  1, b  1, and f(n)  0
Case 1: if f(n) = O(nlogba -) for some  > 0, then: T(n) = (nlogba)
Case 2: if f(n) = (nlogba), then: T(n) = (nlogba lgn)
Case 3: if f(n) = (nlogba +) for some  > 0, and if
af(n/b) ≤ cf(n) for some c < 1 and all sufficiently large n, then:
regularity condition
T(n) = (f(n))
18
Example 1
Case 1: if f(n) = O(nlogba -) for some  > 0, then: T(n) = (nlogba)
Case 2: if f(n) = (nlogba), then: T(n) = (nlogba lgn)
Case 3: if f(n) = (nlogba +) for some  > 0, and if
af(n/b) ≤ cf(n) for some c < 1 and all sufficiently large n, then: T(n) =(f(n))
T(n) = 2T(n/2) + n
a = 2, b = 2, log22 = 1
Compare nlogba=n1 with f(n) = n
f(n) = (nlogba=n1)  Case 2
 T(n) = (nlogba lgn) = (nlgn)
19
Example 2
Case 1: if f(n) = O(nlogba -) for some  > 0, then: T(n) = (nlogba)
Case 2: if f(n) = (nlogba), then: T(n) = (nlogba lgn)
Case 3: if f(n) = (nlogba +) for some  > 0, and if
af(n/b) ≤ cf(n) for some c < 1 and all sufficiently large n, then: T(n) =(f(n))
T(n) = 2T(n/2) + n2
a = 2, b = 2, log22 = 1
Compare nlog22=n1 with f(n) = n2
f(n) = (nlog22+)  Case 3
(* need to verify regularity cond.)
a f(n/b) ≤ c f(n)  2 n2/4 ≤ c n2  (½ ≤ c <1)

T(n) = (f(n)) = (n2)
Example 3
Case 1: if f(n) = O(nlogba -) for some  > 0, then: T(n) = (nlogba)
Case 2: if f(n) = (nlogba), then: T(n) = (nlogba lgn)
Case 3: if f(n) = (nlogba +) for some  > 0, and if
af(n/b) ≤ cf(n) for some c < 1 and all sufficiently large n, then: T(n) =(f(n))
T(n) = 2T(n/2) + √n
a = 2, b = 2, log22 = 1
Compare n with f(n) = n1/2
f(n) = O(n1-)  Case 1
 T(n) = (nlogba) = (n)
21
Example 4
Case 1: if f(n) = O(nlogba -) for some  > 0, then: T(n) = (nlogba)
Case 2: if f(n) = (nlogba), then: T(n) = (nlogba lgn)
Case 3: if f(n) = (nlogba +) for some  > 0, and if
af(n/b) ≤ cf(n) for some c < 1 and all sufficiently large n, then: T(n) =(f(n))
T(n) = 3T(n/4) + nlgn
a = 3, b = 4, log43 = 0.793
Compare n0.793 with f(n) = nlgn
f(n) = (nlog43+)  Case 3
Check regularity condition:
3(n/4)lg(n/4) ≤ (3/4)nlgn = c f(n),
 T(n) = (nlgn)
(3/4 ≤ c < 1)
Example 5
Case 1: if f(n) = O(nlogba -) for some  > 0, then: T(n) = (nlogba)
Case 2: if f(n) = (nlogba), then: T(n) = (nlogba lgn)
Case 3: if f(n) = (nlogba +) for some  > 0, and if
af(n/b) ≤ cf(n) for some c < 1 and all sufficiently large n, then: T(n) =(f(n))
T(n) = 2T(n/2) + nlgn
a = 2, b = 2, log22 = 1

Compare n with f(n) = nlgn
Is this a candidate for Case 3 ?

Case 3: if f(n) = (nlogba +) for some  > 0
In other words, If nlgn ≥ n1.n for some  > 0

lgn is asymptotically less than n for any  > 0 !!
Therefore, this in not Case 3!
(somewhere between Case 2 & 3)
Exercise:
T(n) = 2T(n/2) + nlgn

Use Iteration Method to show that
T(n) = nlg2n
T(n) = nlgn + 2T(n/2)
= nlgn + 2(n/2*lg(n/2) + 2T(n/4))
= nlgn + nlg(n/2) + 22T(n/22)
= ….
Changing variables
T(n) = 2T(
n
) + lgn
Try to transform the recurrence to one that you have seen before
◦ Rename:
m = lgn  n = 2m
T (2m) = 2T(2m/2) + m
◦ Rename: k=m and S(k) = T(2k)
S(k) = 2S(k/2) + k  S(k) = (k*lgk)
T(n) = T(2m) = S(m) = (mlgm)= (lgn*lglgn)
**See Page 86 of the Textbook.
25
Exercise:
T(n) = 2T(

n
) + lgn
Use Iteration Method to show that
T(n) = (lgn*lglgn)
T(n) = lgn + 2T(n1/2)
= lgn + 2(lg(n1/2) + 2T(n1/4))
= lgn + lgn + 22T(n1/4)
= lgn + lgn + … + 2kT(2)
k=lglgn
26
Appendix A
Summations
n

k  1 2  3  n 
n ( n  1)
2
k 1
n

k 1  2  3   n 
2
2
2
2
2
n ( n  1)( 2 n  1)
6
k 1
k
n ( n  1)
2
n
3
1  2  3   n 
3
3
3
3
4
k 1
n

2
x  1 x  x   x 
k
2
n
k 0
n
Harmonic
Series
Hn 

k 1
1
k
x
n 1
1
n
If | x | 1 then lim
x 1
 1
n 
1
2

1
3

1
n

k 0
 ln n  O (1)
x 
k
1
1 x