Mathematical Analysis
of Recursive Algorithm
CSG3F3
Lecture 7
Set up a recurrence relations
and initial condition
2
Example 1
ALGORITHM F(n)
//computes n! recursively
//input: a nonnegative integer n
//output: the value of n!
if n=0 return 1
else return F(n-1)*n
3
Recurrence relation and initial condition for
the algorithm’s number of multiplication
M(n):
M(n)=M(n-1)+1
M(0)=0
for n>0,
4
M(n)=M(n-1)+1 substitute M(n-1)=M(n-2)+1
=[M(n-2)+1]+1=M(n-2)+2 substitute M(n-2)=M(n-3)+1
=[M(n-3)+1]+2=M(n-3)+3
General formula: M(n)=M(n-i)+i
5
• For n=0,
M(n)=M(n-1)+1=…=M(n-i)+i=…
=M(n-n)+n=n
6
Techniques for solving recurrence
• Substitution
• Characteristic equation
• Induction
7
Solving Recurrences by
Substitution
• Recurrence:
• Work backward:
8
Substitution (cont.)
• Substitution:
9
Problem 1
1
1
Note: 1 + + ⋯ + ≈ ln 𝑛
2
𝑛
10
Problem 2
ALGORITHM Power(x, n)
// Computes x to the power of n.
// Input: x is a real number, n
an integer >= 0
// Output: Returns x to the power of
n.
if n=0 return 1
else return x * Power(x, n-1)
11
Problem 2 (cont.)
• Analysis
Basic operation
M(n) = 1 + M(n-1)
M(0) = 0
: multiplication
: recurrence relation
: initial condition
12
Problem 2 (cont.)
• Solving the recurrence relation
M(n) = 1 + M(n-1)
= 1 + [M(n-2) + 1 ]
= 2 + M(n -2)
= 2 + [M(n -3) + 1 ]
= 3 + M(n -3)
=:
= k + M(n-k)
13
Problem 2 (cont.)
M(n) = k + M(n-k)
= n + M(n-n)
= n + M(0)
=n +0
=n
Order of growth M(n)O(n)
14
Problem 3: Binary Recursive
Algorithm BinRec(n)
//input: a positive decimal integer n
//output: the number of binary digits in
n’s binary representation
If n=1 return 1
Else return BinRec([n/2])+1
17
A(2 k ) = A(2 k-1 ) + 1
= [A(2 k-2 ) + 1] + 1 = A(2 k-2 ) + 2
= [A(2 k-3 ) + 1] + 2 = A(2 k-3 ) + 3
……
= A(2 k-i ) + i
……
= A(2 k-k ) + k
=k
A(2 0 ) = A(1) = 0
= log2 n
n = 2k
A(n) = log2 n O(log n)
18
Characteristic Equation
• Homogeneous Linear Recurrences
• k and ai are constant
• It’s called “linear” because every term ti
appears only to the first power
• It’s called “homogeneous” because the linear
combination is equal to 0.
22
Characteristic (cont.)
• The following are homogeneous linear
recurrence equations
23
Characteristic (cont.)
• The Fibonacci Sequence is defined as follows:
• Subtract tn-1 and tn-2 from both sides, we get
homogeneous linear recurrence equation
24
Characteristic (cont.)
• Definition
25
Characteristic (cont.)
• Example
26
Characteristic (cont.)
• Theorem. Let the homogeneous linear recurrence
equation with constant coefficients
be given. If its characteristic equation
has k distinct solutions r1, r2,…,rk, then the only
solutions to the recurrence are
27
Characteristic (cont.)
•
Consider the following recurrence
1.
Obtain the characteristic equation
28
Characteristic (cont.)
2.
Solve the characteristic equation
3.
Apply the theorem to get the general solution:
4.
Determine the values of the constants
29
Characteristic (cont.)
The solution to this system is c1 =1/5, c2 =-1/5
5.
Substitute the constant into the general
solution to obtain the particular solution:
30
Exercise
• Fibonacci Sequence
31
Multiplicity root
• Theorem. Let r be a root of multiplicity m of the
characteristic equation for a homogeneous linear
recurrence with constant coefficients. Then
are all solutions to the recurrence.
32
Exercises
33
Characteristic (cont.)
• Nonhomogeneous Linear Recurrences
• k and ai are constant, f(n) is a function other than
the zero function.
• Develop a method for solving the common case
where b is a constant and p(n) is a polynomial in n.
34
Characteristic (cont.)
• Theorem. A nonhomogeneous linear recurrence of
the form
can be transformed into homogeneous linear
recurrence
where d is the degree of p(n).
35
Characteristic (cont.)
1.
2.
3.
36
Characteristic (cont.)
•
Solve the following recurrence
1.
Obtain the characteristic eq for the
corresponding homogeneous eq:
38
Characteristic (cont.)
2.
Obtain a term from the nonhomogeneous part of
the recurrence
the term is
39
Characteristic (cont.)
3.
Apply theorem to obtain the characteristic eq.
4.
Solve the characteristic equation
the roots are r=3 and r=4, and the root r=4 has
multiplicity 2
40
Characteristic (cont.)
5.
Apply theorem to get the general solution
We must find another initial condition. Because
and t1 =12,
41
Characteristic (cont.)
6.
7.
Determine the value of the constants
Substitute the constants into the general
solution to obtain
42
Exercises
43
Change of variables
The recurrence relation:
n
T ( n)  T    1
2
T (1)  1
for n  1, n a power of 2
44
Change of variables
1.
Set
n=2k, which means k= 2logn
2. Substitute 2k for n to obtain
k


2
k
T 2  T    1
 2
 T 2k 1  1
 
 
45
Change of variables
3. Next set: tk = T(2k) in 2) to obtain
the new recurrence
tk = tk-1 + 1
4. General solution
tk = c1 + c2k
46
Change of variables
5. Substitute T(2k) for tk in the general
solution:
T(2k) = c1 + c2k
6. Substitute n for 2k and lg n for k:
T(n) = c1 + c2 lg n
47
Change of variables
T(n) = 1 + lg n
48
Exercises
49
•
Neapolitan, Richard. Kumarss
Naimipour. Foundations of
algorithms. D.C Heath and Company.
1996
50
Next..
Brute Force Technique
51