The Design and Analysis of Algorithms
Chapter 2:
Fundamentals of the
Analysis of Algorithm
Efficiency
Mathematical Analysis of Non-recursive and
Recursive Algorithms
Section 2.3. Mathematical Analysis of
Non-recursive Algorithms
Steps in mathematical analysis of nonrecursive algorithms





Decide on parameter n indicating input size
Identify algorithm’s basic operation
Determine worst, average, and best case for input
of size n
Set up summation for C(n) reflecting algorithm’s
loop structure
Simplify summation using standard formulas (see
Appendix A)
2
Example: Selection sort 1


Input: An array A[0..n-1]
Output: Array A[0..n-1] sorted in ascending order
for i  0 to n-2 do
min  i
for j = i + 1 to n – 1 do
if A[j] < A[min]
min  j
swap A[i] and A[min]
3
Example: Selection sort 2
Basic operation: comparison
Inner loop:

n-1
S(i) =  1 = (n-1) – (i + 1) + 1 = n – 1 – i
j = i+1
Outer loop:
n-2
n-2
n-2
n-2
C(n) =  S(i) =  (n – 1 – i) =  (n – 1) –  i
i=0
i=0
i=0
i=0
n
Basic formula:
i
= n(n+1) / 2
i=0
C(n) = (n – 1 )(n -1 ) – (n-2)(n-1)/2 = (n – 1) [2(n – 1) – (n – 2)] / 2 =
= (n – 1) n / 2 = O(n2)
4
Section 2.4. Mathematical Analysis of
Recursive Algorithms
Steps in mathematical analysis of nonrecursive algorithms





Decide on parameter n indicating input size
Identify algorithm’s basic operation
Determine worst, average, and best case for input of
size n
Set up a recurrence relation and initial condition(s)
Solve the recurrence to obtain a closed form or
estimate the order of magnitude of the solution (see
Appendix B)
5
Important Recurrence Types

One (constant) operation reduces problem size by one.
T(n) = T(n-1) + c
T(1) = d
Solution: T(n) = (n-1)c + d
linear

A pass through input reduces problem size by one.
T(n) = T(n-1) + cn
T(1) = d
Solution: T(n) = [n(n+1)/2 – 1] c + d
quadratic

One (constant) operation reduces problem size by half.
T(n) = T(n/2) + c
T(1) = d
Solution: T(n) = c log n + d
logarithmic

A pass through input reduces problem size by half.
T(n) = 2T(n/2) + cn
T(1) = d
Solution: T(n) = cn log n + d n
n log n
6
Example 1: Factorial
n! = n*(n-1)!
0! = 1
Recurrence
relation:
T(n) = T(n-1) + 1
T(1) = 1
Telescoping:
T(n) = T(n-1) + 1
T(n-1) = T(n-2) + 1
T(n-2) = T(n-3) + 1
…
T(2) = T(1 ) + 1
Add the equations and cross equal
terms on opposite sides:

T(n) = T(1) + (n-1) =
= n
7
Example 2: Binary Search
Recurrence Relation
T(n) = T(n/2) + 1, T(1) = 1
 Telescoping
T(n/2) = T(n/4) + 1
…
T(2) = T(1) + 1


Add the equations and cross equal terms on opposite sides:
T(n) = T(1) + log(n) = O(log(n))
8
Master Theorem: A general divideand-conquer recurrence
T(n) = aT(n/b) + f (n)
a < bk
a = bk
a > bk
where f (n) ∈ Θ(nk)
T(n) ∈ Θ(nk)
T(n) ∈ Θ(nk log n )
T(n) ∈ Θ(n to the power of (logba))
Note: the same results hold with O instead of Θ.
9