Recurrence Relations
Analysis of recursive algorithms, such as:
int factorial (int n) {
if (n==0) return 1;
else return (n * factorial(n-1));
}
Let tn be the number of multiplications needed to calculate factorial(n). Then tn = tn-1 + 1.
The equation, tn = tn-1 + 1, is an example of a recurrence equation or recurrence relation. Recurrence relations
are usually accompanied by initial condition(s). For the factorial function above, the initial condition would
be written as t0 = 0; which indicates that no multiplications are calculated when n is 0.
The first few values for the above recurrence relation (beginning with the initial condition) are:
t0 = 0
t1 = t0 + 1 = 0 + 1 = 1
t2 = t1 + 1 = 1 + 1 = 2
t3 = t2 + 1 = 2 + 1 = 3
t4 = t3 + 1 = 3 + 1 = 4
From this pattern we might guess that a non-recurrence expression for tn might be tn = n. Such an expression
is called a solution to the recurrence relation. Many times we can guess at the solution of a recurrence
relation by looking for a pattern in the first few iterations.
If a potential or candidate solution is found by observation, we still need to prove that it does, indeed, “solve”
the recurrence relation. Such proofs often use an induction style of proof. When proving something by
induction, one first shows that candidate solution satisfies a base case. For instance, it is obvious that for n =
0, it is true that tn = 0.
The next step in a proof by induction is to state the “induction hypothesis”. In the induction hypothesis, one
makes the assumption that the candidate solution is true for the ith case (i  0). For example, for this problem
one would simply state the assumption that ti = i .
The final step (sometimes called the “induction step”) is to show that assuming the induction hypothesis is
true (the solution is true for the ith case) that the solution is true for the i+1 case. For our example problem,
one would show that: ti+1 = t(i+1)-1 + 1 = ti + 1 = i + 1 DONE!
Examples:
a. Solve the recurrence relation: tn = tn/2 + 1, where n > 1 is a power of 2 and t1 = 1 (This recurrence
relation occurs with the worst case lookup using a binary search.)
t1 = 1
t2 = t2/2 + 1 = t1 + 1 = 1 + 1 = 2
t4 = t4/2 + 1 = t2 + 1 = 2 + 1 = 3
t8 = t8/2 + 1 = t4 + 1 = 3 + 1 = 4
t16 = t16/2 + 1 = t8 + 1 = 4 + 1 = 5
From observation, a candidate solution would be: tn = log2(n) + 1
Using proof by induction: Base case: t1 = 1
Induction Hypothesis: Assume ti = log2(i) + 1
Induction Step:
t2i = t2i/2 + 1 = ti + 1 = (log2(i) + 1) + 1 = log2(i) + log2(2) + 1
= log2(2i) + 1
Done!
Page 1
b. Solve tn = 7tn/2, where n > 1 and n is a power of 2 and t1 = 1.
t1 = 1
t2 = 7t2/2 = 7t1= 7
t4 = 7t4/2 = 7t2 = 7*7 = 72
t8 = 7t8/2 = 7t4 = 7*72 = 73
t16 =7 t16/2 = 7t8 = 7*73 = 74
Candidate: tn = 7log2(n)
Proof: Base:
t1 = 1 = 7log2(1)
Induction Hypothsis: ti = 7log2(i)
Induction Step:
t2i = 7t2i/2 = 7ti = 7 * 7log2(i) = 7(1+log2(i)) = 7(log2(2) + log2(i)) = 7log2(2i)
c. Solve tn = 2tn/2 + n – 1, where n > 1 and n is a power of 2 and t1 = 0.
t1 = 0
t2 = 2t1 + 2 – 1 = 2*0 + 2 – 1 = 1
t4 = 2t2 + 4 – 1 = 2*1 + 4 – 1 = 5
t8 = 2t4 + 8 – 1 = 2*5 + 8 – 1 = 17
t16 = 2t8 + 16 – 1 = 2*17 + 16 – 1 = 49
Solution? (We’ll return to this problem later!)
Homogeneous Linear Recurrences
A homogeneous linear recurrence is any recurrence that can be put in the following format:
a0tn + a1tn-1 + … + aktn-k = 0 where each ai is a constant (i = 0 ... k)
These are linear because each ti term appears only to the 1st power (no tn2, tn*tn-1, or tn/2). They
are homogeneous because the right-hand side of the equation is equal to 0.
Example: Analysis of the recursive function to find the nth Fibonacci number leads to a
homogeneous linear recurrence relation.
int Fib(int n) {
if (n<=1) return n;
else return ( fib(n-1) + fib(n-2) );
}
Let tn = number of additions needed to find Fib(n). Then tn = tn-1 + tn-2, with t0 = 0 & t1 = 1.
This can be converted to the format: tn – tn-1 – tn-2 = 0, with t0 = 0 & t1 = 1. The first few terms
are:
t2 = t1 + t0 = 1 + 0 = 1
t3 = t2 + t1 = 1 + 1 = 2
t4 = t3 + t2 = 2 + 1 = 3
t5 = t4 + t3 = 3 + 2 = 5
Page 2
Before we find a solution to this homogeneous linear recurrence relation, let’s try an easier problem.
Let’s solve the recurrence relation: tn – 5tn-1 + 6tn-2 = 0, for n > 1 with t0 = 0 and t1 = 1.
If we set tn = rn, then a root, r, of the polynomial: rn – 5rn-1 + 6rn-2 = 0, will give us a solution to the
recurrence relation.
rn – 5rn-1 + 6rn-2 = 0  rn-2 ( r2 -5r + 6) = 0  rn-2 ( r-2) (r-3) = 0
Thus, the roots are r=0, r=2, and r=3; and some of the solutions to the recurrence relation should be
tn = 0, tn = 2n, and tn = 3n
Verify: tn = 0:
Verify: tn = 2n:
Verify: tn = 3n:
0 – 5*0 +6*0 = 0
2n – 5*2n-1 + 6*2n-2 = 2n – 5*2n-1 +3*2n-1 = 2n -2*2n-1 = 2n – 2n = 0
3n – 5*3n-1 + 6*3n-2 = 3n – 5*3n-1 + 2*3n-1 = 3n – 3*3n-1 = 3n – 3n = 0
tn = 0 is a vacuous solution that fails to satisfy the initial conditions. We’ll eliminate it.
If tn = 2n and tn = 3n are solutions, then so is tn = 2n + 3n. Can you verify this?
In fact, tn = C12n + C23n is a solution for any constants C1 & C2. We can use the initial conditions
(t0=0, t1=1) to find the particular solution that satisfies the initial conditions:
tn = C12n + C23n, t0 = 0 and t1 = 1  0 = C1 + C2 and 1 = 2C1 + 3C2
Solving these two equations for the two unknowns yields: C1 = -1 and C2 = +1. Thus, the particular
solution that satisfies the initial conditions is:
tn = 3n – 2n
You should be able to verify that the equation: tn = 3n - 2n (n 0) generates the same sequence as the
recurrence relation: tn = 5tn-1 - 6tn-2 for n > 1 with t0 = 0 and t1 = 1.
Definition: Given a homogenous linear recurrence relation: a0tn + a1tn-1 + … + aktn-k = 0, the
characteristic equation is defined as a0rk + a1rk-1 + … + akr0 = 0
What are the characteristic equations for the following recurrence relations?
Recurrence Relation
Characteristic Equation
5tn – 7tn-1 + 6tn-2 = 0
tn – 3tn-1 – 4tn-2 = 0
tn – tn-1 – tn-2 = 0
Theorem: If the characteristic equation has k distinct roots: r1, r2, …, rk, the recurrence relation has
solution: tn = C1r1n + C2r2n + … Ckrkn
Theorem: If the characteristic equation has a root, r, of multiplicity = m, then tn = rn, tn = nrn, tn =
n2rn, tn = n3rn, … tn = nm-1rn are all solutions to the recurrence relation.
Page 3
Examples:
a) (Back to the Fibonacci example) tn – tn-1 – tn-2 = 0, with t0 = 0 and t1 = 1
Characteristic equation: r2 – r – 1 = 0
Roots: r1 = (1 + 5 ) / 2 and r2 = (1 - 5 ) / 2
n
 1 5 


 + C2  1 5 
General Solution: tn = C1 
 2 



 2 
n
Applying initial conditions yields the following 2 equations in 2 unknowns:
0 = C1 + C2
n
n
 1 5 
 1 5 

 + C2 
1 = C1


2
2




Which yields C1 = 1 5
and C2 =  1 5
n
n
1 5   1 5 
Specific Solution: tn = 
2  
2 
5
b) tn – 4tn-1 + 4tn-2 = 0, where n > 0 and t0 = 0 & t1 = 1
Characteristic Equation:
Roots:
General Solution:
Applying initial conditions:
Which yields:
Specific Solution:
r2 – 4r + 4 = 0
r = 2 with multiplicity of 2
tn = C12n + C2n2n
0 = C1
1 = 2C1 + 2C2
C1 = 0 & C2 = ½
tn = ½ n 2n = n2n-1
Page 4
Nonhomogeneous Linear Recurrences
A homogeneous linear recurrence is any recurrence that can be put in the following format:
a0tn + a1tn-1 + … + aktn-k = f(n) where each ai is a constant (i = 0 ... k) and f(n) is any function in n.
There is no known general method for solving nonhomogeneous linear recurrences. However
there is a method for the special case: a0tn + a1tn-1 + … + aktn-k = bn p(n), where b is a constant
and p(n) is a polynomial in n.
Examples: tn – 3tn-1 = 4n
tn – 3tn-1 = 4n (2n+1)
The 1st example can be made homogeneous by the applying the following trick:
tn – 3tn-1 = 4n  tn-1 – 3tn-2 = 4n-1
Also
tn – 3tn-1 = 4n  t n – 3 t n1 = 4n-1
4
4
Subtracting the 1st equation from the 2nd yields: t n – 7t n1 + 3tn-2 = 0
4
4
Multiplying by 4: tn – 7tn-1 + 12tn-2 = 0
The characteristic equation for this homogeneous recurrence is: r2 -7t + 12 = 0
Which factors into: (r-3)(r-4) = 0
Thus the general solution is: tn = C13n + C24n
Using the initial conditions, t0 = 0 and t1 = 4, we get C1 = -4 and C2 = 4
Thus, the specific solution is: tn = 4n+1 – 4(3n)
Notice that the “–4(3n)” term would have been present if the original recurrence relation had
been the homogeneous relation: tn – 3tn-1 = 0. The other term (4n+1) comes from the function on
the right-hand side of the original recurrence relation. This leads us to the following theorem:
Theorem: A recurrence relation in the form: a0tn + a1tn-1 + … + aktn-k = bn p(n), has a
characteristic equation in the form: (a0rk + a1rk-1 + … + ak)(r-b)d+1 = 0, where b is the constant
found in the recurrence relation and d is the degree of p(n). Thus the characteristic equation is
composed of
i) the characteristic equation corresponding to a homogeneous version of the recurrence
relation, and
ii) (r-b)d+1 which has a root of multiplicity d+1.
If there is more than one term like bn p(n) on the right side, each one contributes a product such
as (r-b)d+1 to the characteristic equation.
Page 5
Examples:
a) tn – 3tn-1 = 4n (2n+1) with initial conditions t0 = 0 and t1 = 12.
Characteristic Equation:
(r-3)(r-4)2 = 0
Which has roots:
r = 3 and r = 4 with multiplicity 2
General solution:
tn = C13n + C24n + C3n4n
We need to find a value for t2 so that we can derive 3 equations in 3 unknowns:
t2 – 3t1 = 42(4+1) = 16*5 =80 and, thus, t2 = 3t1 + 80 = 3(12) + 80 = 116
Solving for C1, C2, and C3:
Thus, the specific solution is:
C1  C 2  0




 3C1  4C 2  4C 3  12 
9C  16C  32C  116
2
3
 1


C1 = 20, C2 = -20, C3 = 8
tn = 20(3n) – 20(4n) + 8n4n
b) tn – tn-1 = n - 1 where n > 0 and t0 = 0
Characteristic Equation: (r-1)(r-1)2 = 0  (r-1)3 = 0
Which has roots:
r = 1 with multiplicity 3
General solution:
tn = C11n + C2n1n + C3n21n = C1 + C2n + C3n2
Given t0 = 0:
t1 = t0 + (1-1) = 0 + 0 = 0
t2 = t1 + (2-1) = 0 + 1 = 1
Solving for C1, C2, and C3:
Thus, the specific solution is:
C1  0




 C1  C 2  C 3  0 
C  2C  4C  1
2
3
 1


C1 = 0, C2 = -½, C3 = ½
n2  n
n n2
n( n  1)
tn = - +
=
=
2
2
2
2
c) tn – 2tn-1 = n + 2n where n > 1 and t1 = 0
Characteristic Equation: (r-2)(r-1)2(r-2)1 = 0
Which has roots:
r = 1 with multiplicity 2 and r= 2 with multiplicity 2
General solution:
tn = C11n + C2n1n + C32n + C4n2n = C1 + C2n + C32n + C4n2n
Given t1 = 0:
t2 = 2t1 + 2 + 22 = 0 + 2 + 4 = 6
t3 = 2t2 + 3 + 23 = 2*6 + 3 + 8 = 23
t4 = 2t3 + 4 + 24 = 2*23 + 4 + 16 = 46 + 20 = 66
 C1  C 2  2C 3  2C 4  0 
 C  2C  4C  8C  6 


2
3
4
Solving for C1, C2, and C3:  1

 C1  3C 2  8C 3  24C 4  23 
C1  4C 2  16C 3  64C 4  66
Thus, the specific solution is:

C1 = -2, C2 = -1,
C3 = ½, C4 = 1
tn = -2 – n + ½ (2n)+ n2n = 2n-1 + n2n – n – 2
Page 6
d) Let’s return to the example left unfinished on page 2:
Solve tn = 2tn/2 + n – 1, where n > 1 and n is a power of 2 and t1 = 0.
This is not a linear recurrence relation because of the tn/2 term. However, by using two
substitutions, we can convert this into a linear recurrence relation.
First, since n is defined to always be a power of 2, we’ll replace n with 2k. Thus, the new
recurrence relation is: t2 k  2t2 k  2k  1 = 2t2 k 1  2k  1 , where k > 0 and t 2 0 = 0.
2
Since terms such as t 2 k are tedious to work with, we’ll make another substitution.
Let Sk = t 2 k .
We now have the recurrence relation: Sk = 2Sk-1 +2k – 1 or
Sk – 2Sk-1 = 2k - 1, where k > 0 and S0 = 0,
which is a linear nonhomogeneous recurrence relation. The right-hand side of the equation may
be rewritten as 2k(1) + 1k(-1), where (1) and (-1) are trivial functions in k. Thus using the
thereom for nonhomogeneous recurrence relations, the characteristic equation is:
(r – 2) (r – 2) (r – 1) = 0
The first factor is derived from the homogeneous contribution while the other two factors follow
directly from the thereom. Thus, the roots are 1 and 2 (with multiplicity 2).
General Solution: t 2 k = Sk = C11k + C22k + C3k2k
Since n = 2k, k = log n. Thus, the general solution is: tn = C1 + C2n + C3n log n, where n  1
and n is a power of 2.
 C1  C2  0 


Solving for C1, C2, and C3: C1  2C2  2C3  1
C  4C  8C  5
2
3
 1

Thus, the specific solution is: tn = 1 – n + n log n
where n  1 and n is a power of 2.

C1 = 1, C2 = -1, C3 = 1
or tn = n log n – (n – 1),
You should verify that this solution generates the same sequence of values that is generated by
the original recurrence relation.
Page 7
Master Theorem
A very common type of recurrence relation occurs with Divide-and-Conquer type of algorithms. In
fact, they are so common that a master theorem has been proven that addresses these problems and
provides an order of complexity formula. That is, if the recurrence relation matches a common
pattern, we can find the  complexity of the solution. Note, the theorem does not provide the exact
solution; but, after all, most of the time all we really need is the  complexity.
Master Theorem: Given a recurrence relation of the form: tn = atn/b + f(n) , where f(n) is in (nd)
with d ≥ 0, then
For example, let’s once again consider the recurrence relation on the previous page: tn = 2tn/2 + n – 1.
For this problem, a = 2, b = 2, and since n-1 is in (n1), d = 1. Thus, a = bd; therefore the solution
for the recurrence relation belongs to (n1 log n) = (n log n). DONE!
What are the order of growth for solutions of the following recurrence relations?
a. tn = 4tn/2 + n , t1 = 1
b. tn = 4tn/2 + n2 , t1 = 1
c. tn = 4tn/2 + n3 , t1 = 1
Page 8