Lecture on Recurrence Relations II
We know how to solve recurrences of the form an = 3an−1 , or an = 3an−1 − 4an−2 . How about
an = 3an−1 − 4n.
Such a recurrence is called nonhomogeneous,
an = c1 an−1 + c2 an−2 + · · · + ak cn−k + F (n).
while the associated homogeneous recurrence is
an = c1 an−1 + c2 an−2 + · · · + ak cn−k
The general solutions of them are closely related.
(p)
Theorem 1. Let {an } be a particular solution of the nonhomogeneous linear recurrence relation
with constant coefficient
an = c1 an−1 + c2 an−2 + · · · + ak cn−k + F (n).
(1)
Then every solution of (1) is of the form
(p)
an = a(h)
n + an .
(h)
where {an } is the general solution of the homogeneous recurrence.
Example 1. Find the general solution of an = 2an−1 − 3, with initial value a0 = 5.
(h)
General solution for the homogeneous part: an = α1 2n .
(p)
Try to get a particular solution. Say, if a0 = 3, then a1 = 3, a2 = 3, . . . . So an = 3. Thus
an = α1 2n + 3.
Using the initial value a0 = 5, we have α1 (20 ) + 3 = 5, hence α1 = 2, and an = 2n+1 + 3.
Example 2. Find the general solution of an = 2an−1 + 3n .
(p)
(p)
For an , try an = t3n . Get t = 3. So a particular solution is 3 · 3n . Thus
an = α1 2n + 3n+1 .
Summary. To solve a nonhomogeneous recurrence,
(h)
1. Find the general solution of the homogeneous part an .
2. (Experiment part) Get one particular solution of the nonhomogeneous recurrence. Call it
(p)
an .
3. The general solution for the nonhomog recurrence is
(p)
an = a(h)
n + an
∗h)
4. If there is initial values, solve the coefficients α in an .
1
How to “get one particular solution”? – Try a function of the similar form, with some unknowns.
If one fails, multiply by n and try again. e.g. an = 3an−1 + 3n . ( Ans: an = c3n + n3n ).
Remark: Write down the trial solution for
(p)
1. an = 2an−1 + (−3)n ; should try an = c(−3)n .
(p)
2. an = 2an−1 − 3n ; should try an = c3n .
Summary to find the trial solution:
The general rule is that the trial solution looks like the nonhomogeneous part.
Form of F (n)
trial solution
constant
constant
polynomial of degree k general polynomial of degree k
exponential cȧn
constant multiple of an
sin(kn) or cos(kn)
A sin(kn) + B cos(kn)
sum/product of above sum/product of corresponding terms
If a term fail, multiply by n and try again.
Complexity and Master Theorem
Sometimes we cannot find a suitable trial solution. Sometimes we can not solve the homogeneous
recurrence at all. If the function is coming from complexity of an algorithm, usually we are only
concerned about the asymptotic of the function f (n), i.e., the big-Oh or big-Theta term.
For example, in merge sort, we have
n
f (n) = 2f ( ) + n.
2
In Divide and Conquer, we typically have
n
f (n) = af ( ) + h(n),
b
where b > 1 and a ≥ 1.
Theorem 2 (MASTER THEOREM). Let a ≥ 1 and b > 1 be constants. f (n) is an increasing
function satisfying
n
f (n) = af ( ) + h(n).
b
Then f (n) is bounded asymptotically as follows.
1. If h(n) = O(nlogb a− ) for some > 0, then f (n) = Θ(nlogb a ).
2. If h(n) = Ω(nlogb a+ ) for some > 0, then f (n) = Θ(h(n)).
3. If h(n) = Θ(nlogb a ), then f (n) = Θ(nlogb a log n).
Remark: Why do we compare
nlogb a
v.s
h(n)
Since nlogb a is the solution of the homogeneous part f (n) = af ( nb ).
Just let n = bk and f (n) = f (bk ) = g(k). The recurrence becomes g(k) = ag(k − 1). Then
g(k) = Cak = C · alogb n = C · nlogb a .
Then the theorem says,
2
• If h(n) grows faster, then f (n) = Θ(h(n)).
• If nlogb a grows faster, then f (n) = Θ(nlog ba ).
• If these two terms are of the same size, then f (n) = Θ(h(n) log n) = Θ(nlogb A log n).
Example 3. f (n) = 2f ( n2 ) + n.
h(n) = n; a = b = 2, so logb a = 1. nlogb a = n = Θ(h(n)).
So f (n) = Θ(n log n).
√
Example 4. f (n) = 2f ( n2 ) + n.
h(n) = n.5 ; a = b = 2, so logb a = 1. nlogb a = n n.5+ .
So f (n) = Θ(n).
Example 5. f (n) = 2f ( n2 ) + n2 .
h(n) = n2 ; a = b = 2, so logb a = 1. nlogb a = n n2−
So f (n) = O(n2 ).
Example 6. f (n) = 9f ( n3 ) + n
h(n) = n. a = 9 and b = 3. So logb a = 2. h(n) = n n2− . So f (n) = Θ(n2 ).
Example 7. f (n) = 3f ( n4 ) + n log n
h(n) = n log n. a = 3 and b = 4. log4 3 ∈ (0, 1). Hence h(n) = n log n nlog4 3+ . So
f (n) = Θ(n log n).
Example 8. f (n) = 2f (n/2) + n log n.
h(nn log n and nlogb a = n. This is a case that is not covered by the theorem.
There is an additional case that we know the answer:
If h(n) = nlogb a (log n)k , then f (n) = Θ(nlogb a (log n)k+1 ).
Hence the answer is n(log n)2 .
Note: The MASTER THEOREM in the textbook, page 479 is a special case of this version.
3