6.042/18.062J Mathematics for Computer Science
Tom Leighton and Marten van Dijk
October 24, 2006
Notes for Recitation 14
1
The Akra-Bazzi Theorem
Theorem 1 (Akra-Bazzi, strong form). Suppose that:
⎧
is defined
for 0 ≤ x ≤ x0
⎪
⎨
k
T (x) = �
⎪
ai T (bi x + hi (x)) + g(x) for x > x0
⎩
i=1
where:
� a1 , . . . , ak are positive constants
� b1 , . . . , bk are constants between 0 and 1
� x0 is “large enough” in a technical sense we leave unspecified
� |g � (x)| = O(xc ) for some c ∈ N
� |hi (x)| = O(x/ log2 x)
Then:
x
� �
�
p
T (x) = Θ x 1 +
1
where p satisfies the equation
�k
p
i=1 ai bi
��
g(u)
du
up+1
= 1.
The only difference between the strong and weak forms of Akra-Bazzi is the appearance
of this hi (x) term in the recurrence, where hi (x) represents a small change in the size of the
subproblems (O(x/ log2 x)). Notice that, despite the change in the recurrence, the solution
T (x) remains the same in both the strong and weak forms, with no dependence on hi (x)! In
algorithmic terms, this means that small changes in the size of subproblems have no impact
on the asymptotic running time.
Example: Let’s compare the Θ bounds for the following divide-and-conquer recurrences.
�n�
�� n ��
Ta (n) = 3T
+n
Tb (n) = 3T
+n
3
3
Recitation 14
2
For Ta (n) we have a1 = 3, b1 = 1/3, g(n) = n, p = 1.
For Tb (n) we have the same parameters as for Ta (n), plus h1 (n) = �n/3� − n/3.
Using the strong Akra-Bazzi form, the h1 (n) falls out of the equation:
� n
u
Ta (n) = Tb (n) = Θ(n(1 +
du)) = Θ(n log n).
2
1 u
The addition of the ceiling operator changes the value of n/3 by at most 1, which is easily
O(n/ log2 n). So floor and ceiling operators have no impact on the asymptotic solution to a
recurrence.
2
TriMergeSort
We noted in lecture that reducing the size of subproblems is much more important to the
speed of an algorithm than reducing the number of additional steps per call. Let’s see if we
can improve the Θ(n log n) bound on MergeSort from lecture.
Let’s consider a new version of MergeSort called TriMergeSort, where the size n list is
now broken into three sublists of size n/3, which are sorted recursively and then merged.
Since we know that floors and ceilings do not affect the asymptotic solution to a recurrence,
let’s assume that n is a power of 3.
1. How many comparisons are needed to merge three lists of 1 item each?
Solution. 3. For example, a merge of lists {4},{5}, and {2} compares 4 with 5 and
4 with 2, adding 2 to the final list. Then it compares 4 with 5 and adds 4 to the final
list. Finally, it appends the remaining 5. (This could be made more efficient, but
let’s not worry about that here.)
�
2. In the worst case, how many comparisons are needed to merge three lists of n/3 items,
where n is a power of 3?
Solution. 2(n − 2) + 1. The worst case occurs if the first list empties when there is
exactly 1 item in each of the other two. Prior to this, each of the other n − 2 numbers
requires 2 comparisons before going into the big list. After this, we only need 1 more
comparison between the 2 leftover items.
�
3. Define a divide-and-conquer recurrence for this algorithm. Let T (n) be the number of
comparisons to sort a list of n items.
Solution. T (n) = 3T (n/3) + 2n − 3.
�
4. We could analyze the running time of this using plug-and-chug, but let’s try AkraBazzi. First, what is p?
Recitation 14
3
�k
p
Solution. p = 1. Using
i=1 ai bi = 1 with a1 = 3 and b1 = 1/3, we get the
constraint that 3(1/3)p = 1.
�
5. Does the condition |g � (x)| = O(xc ) hold for some c ∈ N ?
Solution. Yes. g(n) = 2n − 3, so |g � (n)| = 2 = O(nc ) for c = 0.
�
6. Determine the theta bound on T(n) by integration.
Solution.
� �
��
� n
g(u)
p
T (n) =Θ n 1 +
du
p+1
1 u
� �
��
� n
2u − 3
=Θ n 1 +
du
u2
1
��
� �
� n
� n
2
3
=Θ n 1 +
−
du
2
1 u
1 u
� �
�n 3 �n ��
�
�
=Θ n 1 + 2 log u� + �
u 1
1
� �
��
3
=Θ n 1 + 2 log n + − 3
n
=Θ (2n log n − 2n + 3)
=Θ (n log n)
�
7. Turns out that any equal partition of the list into a constant number of sublists c > 1
will yield the same theta bound. Can you see why?
Solution. Given a constant size partition, the recurrence will always be in the form:
�
�
c−2
�
T (n) = cT (n/c) + (c − 1)(n − (c − 1)) +
i
i=1
The first term creates the constraint that c(1/c)p = 1, which always gives p = 1. The
second term will always be Θ(n), which dominates the final bound after integration.
Therefore, no matter what c > 1 we choose, T (n) = Θ(n log n).
�
Recitation 14
4
Guessing a Particular Solution
A general linear recurrence has the form:
f (n) = b1 f (n − 1) + b2 f (n − 2) + . . . + bd f (n − d) + g(n)
One step in solving this recurrence is finding a particular solution. This is a function f (n) that
satisfies the recurrence equation, but may not be consistent with the boundary conditions.
Here’s a recipe to help you guess a particular solution:
� If g(n) is a constant, guess that f (n) is some constant c. Plug this into the recurrence
equation and see if any constant actually works. If not, try f (n) = bn + c, then
f (n) = an2 + bn + c, etc.
� More generally, if g(n) is a polynomial, try a polynomial of the same degree. If that
fails, try a polynomial of degree one higher, then two higher, etc. For example, if
g(n) = n, then try f (n) = bn + c and then f (n) = an2 + bn + c.
� If g(n) is an exponential, such as 3n , then first guess that f (n) = c3n . Failing that, try
f (n) = bn3n + c3n and then an2 3n + bn3n + c3n , etc.
In practice, your first or second guess will almost always work.
Recitation 14
5
Linear Recurrences
Find closed-form solutions to the following linear recurrences.
1.
T0 = 0
T1 = 1
Tn = Tn−1 + Tn−2 + 1
Solution. Following the guide to solving linear recurrences:
(a) First, we find the general solution to the homogeneous recurrence. The charac­
teristic equation is r2 − r − 1 = 0. The roots of this equation are:
√
1+ 5
r1 =
2√
1− 5
r2 =
2
(b) Using the roots, we write down the homogeneous solution in the form
�
�
√ �n
√ �n
1+ 5
1− 5
Tn = A
+B
.
2
2
(c) Next, we need a particular solution to the inhomogenous recurrence. Since the
inhomogenous term is constant, we guess a constant solution, c. So replacing the
T (n) terms in Tn = Tn−1 + Tn−2 + 1 by c, we require
c = c + c + 1,
namely, c = −1. That is, Tn = −1 is a particular solution to the equation.
(d) Putting it together, the complete solution to the recurrence is the homogenous
solution plus the particular solution:
�
Tn = A
�
√ �n
√ �n
1+ 5
1− 5
+B
−1
2
2
(e) All that remains is to find the constants A and B. Substituting the initial condi­
tions gives a system of linear equations.
0 = A�
+B−1 �
�
√ �
√
1+ 5
1− 5
1 = A
+B
−1
2
2
Recitation 14
6
The solution to this linear system is:
√
5+3 5
A =
10√
5−3 5
B =
10
Therefore, the complete solution to the recurrence is
�
√ �n
√ � �
√ �n �
√ � �
5+3 5
1+ 5
5−3 5
1− 5
·
·
Tn =
+
− 1.
10
2
10
2
�
2.
S0 = 0
S1 = 1
Sn = 6Sn−1 − 9Sn−2
Solution. The characteristic polynomial is r2 − 6r + 9 = (r − 3)2 , so the solution
is of the form A3n + Bn3n for some constants A and B. Setting n = 0, we have
0 = S0 = A30 + B · 0 ·30 = A. Setting n = 1, we have 1 = S1 = A31 + B · 1 · 31 = 3B,
so B = 1/3. That is,
1
Sn = 0 · 3n + · n3n = n3n−1 .
3
�
MIT OpenCourseWare
http://ocw.mit.edu
6.042J / 18.062J Mathematics for Computer Science
Fall 2010
For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.