1 A Master Theorem for Recurrences
Akra and Bazzi [AB98] recently gave a closed form solution for a large class of recurrences.
Theorem 1 Let
f (x) =
(
h(x)
for 1 ≤ x ≤ x0
x
a f ( b ) + g(x) for x > x0
(1)
where
1. a > 0, b > 1, and x0 ≥ b are constants,
2. x ≥ 1 is a real number,
3. d1 ≤ h(x) ≤ d2 for some positive constants d1 and d2 and all x with 1 ≤ x ≤ x0 , and
4. g is a nonnegative function satisfying the polynomial growth condition, i.e., there are positive constants c1 and c2 such that
c1 g(x) ≤ g(u) ≤ c2 g(x) for all x > x0 and u ∈ [x/b, x] .
5. (technical condition)
I :=
Z x0
g(u)
u p+1
1
du < ∞
Let p be the unique real number for which a/b p = 1. Then
Z x
g(u)
p
f (x) = Θ x 1 +
du
.
p+1
1 u
Before we enter the proof, we give some examples.
• If f (x) = 2 f (x/2) + x, then p = 1 and
Z
f (x) = Θ x(1 +
x
1
u
du) = Θ(x log x) .
u2
• If f (x) = 3 f (x/2) + x2 , then p = log 3 and
Z x
u2
log 3
du) = Θ(xlog 3 (1 + x2−log 3 )) = Θ(x2 ) .
f (x) = Θ x (1 +
1+log 3
1 u
• If f (x) = f (x/2) + log x, then p = 0 and
Z x
log u
0
du) = Θ(1 + log2 x) = Θ(log2 x) .
f (x) = Θ x (1 +
1
1 u
1
Proof: For any x > x0 , we have x/b < x and x/b > x0 /b ≥ 1. Thus f (x) is well-defined. We
need to prove
Z x
Z x
g(u)
g(u)
p
p
C·x 1+
du ≤ f (x) ≤ D · x 1 +
du
(2)
p+1
p+1
1 u
1 u
for positive constants C and D. We prove the upper bound and leave the lower bound to exercise 1. The proof of the upper bound is by induction on the index of x, the smallest integer k such
that x/bk ≤ x0 .
Assume first that 1 ≤ x ≤ x0 . Then
R
p 1 + x g(u) du
x
p
1 u p+1
x
,
f (x) = h(x) ≤ d2 ≤ d2
p ≤ d2
p
min(1, x0 )
min(1, x0 )
since x p assumes its minimum in [1, x0 ] at either x = 1 (if p ≥ 0) or x = x0 (if p < 0) and since
the value of the integral is nonnegative. Thus the upper bound in equation (2) holds for any D
for which D ≥ d2 / min(1, x0p ).
Assume next that x > x0 . The index of x/b is one smaller than the index of x and hence we
may use the upper bound in equation (2) for x/b. We obtain:
x
f (x) = a f ( ) + g(x)
b
Z x/b
x p g(u)
1+
du
+ g(x)
≤ a·D·
b
u p+1
1
Z x/b
g(u)
p
du + g(x)
= D·x 1+
u p+1
1
Z x
g(u)
p
≤ D·x 1+
du
p+1
1 u
definition of f
induction hypothesis
since a/b p = 1
provided we can establish the inequality
g(x) ≤ D · x p
Z x
g(u)
x/b
u p+1
du .
The inequality is a simple consequence of the polynomial-growth assumption. Observe that
g(u) ≥ c1 g(x) for x/b ≤ u ≤ x and u p+1 ≤ max((x/b) p+1 , x p+1 ) and hence
D·x
p
Z x
g(u)
x/b
u p+1
c1 g(x)
max((x/b) p+1, x p+1 )
c1 (1 − 1/b)
g(x)
= D·
max((1/b) p+1, 1)
≥ g(x)
du ≥ D · x p (x − x/b)
provided that D ≥ max((1/b) p+1, 1)/(c1(1 − 1/b)). This completes the induction step.
2
p
We satisfy both constraints on D by setting D = max(d2 , d2 /x0 , b/(c1 (b − 1)), 1/(c1(b −
1)b p)).
How does the value I := x p
a table of integrals1 yields:
Rx
1
g(u)/u p+1 du depend on the relation of g and p? Inspection of
1 x
Rx 1
1 u1+ε du = −
ε uε i1
h k+1
x
R x lnk u
ln
u
du =
Thus
1 u
R x uδ lnk
1 u
= Θ(1)
ε >0
= Θ(lnk+1 x)
k≥0
ix h
du = Θ uδ lnk u
= Θ(xδ lnk x)
δ > 0 and k ≥ 0
k+1
1
1
• f (x) = Θ(x p ), if g(x) = O(x p−ε ) for some ε > 0,
• f (x) = Θ(g(x) logk+1 x) if g(x) = Θ(x p logk ) for some k ≥ 0, and
• f (x) = Θ(g(x)) if g(x) = Θ(x p+δ logk ) for some δ > 0 and k ≥ 0.
It is instructive to inspect the value I∆ := x p
I∆ = x p
Z x/bi−1
g(u)
x/bi
u p+1
du = x p
R x/bi−1
x/bi
g(u)/u p+1 du. For g(x) = x p , we have
Z x/bi−1
1
x/bi
u
x/bi−1
du = x p [ln u]x/bi
= x p ln b ,
i.e., all levels of the recursion contribute essentially the same amount to f (x). If g(x) = ω (x p ),
the outermost level (i = 0) contributes most, and if g(x) = o(x p ), the bottommost level (i = logb x)
contributes most.
Exercise 1 Prove the lower bound of Theorem 1.
Let us see a second proof. We first express f (x) as a sum.
Lemma 1 Let k be the minimal integer with x/bk ≤ x0 . Then
f (x) = ak f (x/bk ) +
∑
ai g(x/bi ) = ak h(x/bk ) +
∑
ai g(x/bi ) .
0≤i≤k
0≤i<k
Proof: Either by repeated substitution or by induction on k.
Let us next study the two terms in the expression for f (x). Since 1 ≤ x/bk ≤ x0 , we have
d1 ≤ h(x/bk ) ≤ d2 and hence
ak h(x/bk ) = Θ(ak ) = Θ(x p )
1 See
for example, the Wikipedia entry on lists of integrals.
3
where the last equality can be established by taking logarithms on both sides and observing that
k log a = (log x)/(log b) log a and p log x = (log a)/(log b) log x.
We turn to the sum S := ∑0≤i<k ai g(x/bi ) Define x∗ as x/bk . Then x/bi = x∗ bk−i and hence
S = ak
g(x∗ b j )
g(x∗ bk−i )
g(x∗ b j )
p
p
=
x
=
x
∑ aj
∑ ak−i
∑ (b p) j ==: x p S′
1≤ j≤k
0≤i<k
1≤ j≤k
where the second equality uses the substitution j = k − i and the third equality uses a = b p . Since
sums are harder to evaluate than integrals, we want to turn the sum into a integral. Consider a
single term g(x∗ b j )/(b j ) p . We want it to be the value of an integral from x∗ b j−1 to x∗ b j . The
length of this integral is x∗ (b j − b j−1 ) = x∗ b j (1 − 1/b). Thus
g(x∗ b j ) (x∗ ) p g(x∗ b j )
∗ p b
=
=
(x
)
(b j ) p
(x∗ b j ) p
b−1
Z x∗ b j
x∗ b j−1
g(x∗ b j )
∗ p b
du
=
(x
)
(x∗ b j ) p · (x∗ b j )
b−1
Z x∗ b j
x∗ b j−1
g(x∗ b j )
du .
(x∗ b j ) p+1
With
g(x∗ b j )
h(u) := ∗ j p+1
(x b )
∗ j−1
for x b
b
< u ≤ x b , we have S = (x )
b−1
∗ j
′
∗ p
Z x
x∗
h(u) du .
For x∗ b j−1 < u ≤ x∗ b j , we have
g(x∗ b j ) (x∗ b j ) p+1
h(u)
=
·
= Θ(1) · Θ(1) = Θ(1) .
g(u)/u p+1
g(u)
u p+1
For the first fraction, this follows from the polynomial growth property of g, and for the second
fraction, this follows from the polynomial growth property of x1+p . Thus
Z x
Z x
g(u)
′
∗ p b
S = (x )
h(u) du = Θ
du .
p+1
b − 1 x∗
x∗ u
and we completed the alternative proof of the AB-theorem. We should remark that the alternative
proof does not generalize.
We come to extensions. In Theorem 1, f (x) is defined as a times the value of f at x/b. More
generally, we can define f (x) as a linear combination of smaller values.
(
h(x)
for 1 ≤ x ≤ x0
(3)
f (x) =
x
k
∑i=1 ai f ( bi ) + g(x) for x > x0
where k is an integer constant and ai > 0 and bi > 1 are real constants. In order to make f well
defined, we require x0 ≥ maxi bi . Then 1 ≤ x/bi < x for all x ≥ x0 and all i. We define p as the
unique real number for which ∑i ai (1/bi ) p = 1.
Theorem 2 (Akra-Bazzi) Under the assumptions stated above,
Z x
g(u)
p
du
.
f (x) = Θ x 1 +
p+1
1 u
4
Proof: The proof is analogous to the proof of Theorem 1 and delegated to the exercises. It
again uses induction on the index of x, the smallest integer k such that x/(mini bi )k ≤ x0 .
Exercise 2 Prove Theorem 2
Although Theorem 2 handles a broad class of recurrences, it does not cover a common form
of recurrence arising in the analysis of algorithms. For example, in the recurrence for the running
time of Karatsuba’s algorithm, we reduced TK (n) to TK (⌈n/2⌉ + 1). The following extension of
Theorem 2 deals with these variations. Consider
(
h(x)
for 1 ≤ x ≤ x0
f (x) =
(4)
∑ki=1 ai f ( bxi + hi (x)) + g(x) for x > x0
where k is an integer constant and ai > 0 and bi > 1 are real constants. The hi are functions with
|hi (x)| ≤ x/(log1+ε x) for some ε > 0 and all x > x0 . In order to make f well defined, we require
1 ≤ x/bi + hi (x) < x/b for all i and x > x0 and some b > 1. As before, we define p as the unique
real number for which ∑i ai (1/bi ) p = 1.
Theorem 3 (Leighton) Under the assumptions stated above and some more technical conditions on x0 (see [Lei])
Z x
g(u)
p
f (x) = Θ x 1 +
du
.
p+1
1 u
Proof: See [Lei]
For the Karatsuba recurrence we would use k = 3, ai = 1, bi = 2 and hi (x) = ⌈x/2⌉ + 1 − x/2
for 1 ≤ i ≤ k.
References
[AB98] M. Akra and L. Bazzi. On the solution of linear recurrence equations. Computational
Optimization and Applications, 10(2):195–210, 1998.
[Lei]
T. Leighton. Notes on better master theorems for divide and conquer recurrences. http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.
1.1.39.1636.
5