Chapter 2
Recursive Algorithms
Chapter Outline
•
•
•
•
•
•
Analyzing recursive algorithms
Recurrence relations
Closest pair algorithms
Convex hull algorithms
Generating permutations
Recursion and stacks
2
Prerequisites
• Before beginning this chapter, you
should be able to:
– Read and create recursive algorithms
– Identify comparison and arithmetic
operations
– Use basic algebra
3
Goals
• At the end of this chapter you should be
able to:
– Create the recurrence relation for a
recursive algorithm
– Convert a simple recurrence relation into
closed form
– Explain the closest pair algorithm
4
Goals (continued)
– Explain the convex hull algorithm
– Generate permutations both recursively
and iteratively
– Explain the relationship between recursion
and stacks
5
Recursive Algorithms
• Recursive algorithms solve the problem
by solving smaller versions of the
problem
– If the smaller versions are only a little
smaller, the algorithm can be called a
reduce and conquer algorithm
– If the smaller versions are about half the
size of the original, the algorithm can be
called a divide and conquer algorithm
6
Recursive Algorithm Analysis
• The analysis depends on
– the preparation work to divide the input
– the size of the smaller pieces
– the number of recursive calls
– the concluding work to combine the results
of the recursive calls
7
Generic Recursive Algorithm
Recurse( data, N, solution )
// data
a set of input values
// N
the number of values in the set
// solution
the solution to this problem
if N ≤ SizeLimit then
DirectSolution( data, N, solution )
No. of smaller sets
else
DivideInput( data, N, smallerSets,
smallerSizes, numberSmaller )
for i = 1 to numberSmaller do
Recurse(smallerSets[i], smallerSizes[i],
smallSolution[i])
end for
CombineSolutions(smallSolution,
numberSmaller, solution)
end if
8
Generic Recursive Algorithm Analysis
• The recursive algorithm does the following amount of work:
for N  SizeLimit
DIR ( N )

numberSmaller
REC( N )  
DIV( N ) 
 REC(smallerSiz es[i])  COM ( N ) for N  SizeLimit

i 1

• Where
– DIR(N) is the amount of work done by the direct solution
– DIV(N) is the amount of work done by the division of the
problem
– COM(N) is the amount of work done to combine the
solutions
9
Recursive Algorithms (Example)
Factorial( N ){
// Assuming N >= 1
if (N == 1) then
// Size limit: 1
return 1
// Direct solution
else
smaller = N-1
// Div(N)
answer = Factorial(smaller) // Rec(smaller)
return (N * smaller)
// Com(N)
end if
}
Divide and Conquer Example
Largest( list, start, end ){
(start,end):
if (start ≥ end) then
return list[end]
end if
(1,4)  (1,2) & (3,4)
(1,2)  (1,1) & (2,2)
(3,4)  (3,3) & (4,4)
middle = (start + end) / 2
first = Largest( list, start, middle )
second = Largest( list, middle+1, end )
if (first > second) then
11
return first
7 5 9 11
else
return second
7 5
9 11 s=? s=11
end if
f=7 f=?
7
}
f=7
5
s=5
9
f=9
11
s=1111
Divide and Conquer Example Analysis
• This algorithm does:
– One addition and division to calculate
middle
– Two recursive calls with lists about half the
original size
– One comparison to decide between the
largest of the first half and the largest of
the second half
12
Divide and Conquer Example Analysis
• The number of comparisons C(n) of list values
of size n done by this example is given by:
(ignoring the comparison between start and end)
0
for n  1

C(n)  
2Cn 2  1 for n  1
13
Recurrence Relation Form
• A recurrence relation is a recursive form
of an equation, for example:
T (1)  3
T (n)  T (n  1)  2
• A recurrence relation can be put into an
equivalent closed form without the
recursion
14
Converting Recurrence Relations
• Begin by looking at a series of
equations with decreasing values of n:
T(n)  T(n  1)  2
T(n  1)  T (n  2)  2
T(n  2)  T(n  3)  2
T(n  3)  T (n  4)  2
T(n  4)  T(n  5)  2
15
Converting Recurrence Relations
• Now, we substitute back into the first
equation:
T(n)  T(n  1)  2
T(n)  (T(n  2)  2)  2
T(n)  ((T(n  3)  2)  2)  2
T(n)  (((T(n  4)  2)  2)  2)  2
T(n)  ((((T(n  5)  2)  2)  2)  2)  2
16
Converting Recurrence Relations
• We stop when we get to T(1):
T(n)  T(n  1)  2
T(n)  (T(n  2)  2)  2

T(n)  (((T(1)  2)  2)  2)  2
• How many “+ 2” terms are there?
Notice we increase them with each
substitution.
17
Converting Recurrence Relations
• We must have n – 1 of the “+ 2” terms
because there was one at the start and
we did n – 2 substitutions:
n 1
T(n)  T(1)   2
i 1
• So, the closed form of the equation is:
T(n)  3  2(n  1)
• See pp.36 & 38 for more examples
18
• Example 2 (see pp.36-38)
T( n)  2T (n  2) - 15 for n  2
T(2)  40
T (1) 40
• Example 2 (see pp.38-40)
T( n)  
5
if n  4
4T( n / 2)  1 otherwise
19
Approximating Recurrence Relations
• For recurrence relations of the form
T (N) = a * T (N / b) + f (N)
• where f (N) = Θ ( N d ) with d >= 0, the closed
form can be approximated by:



 N d

T( N )   N d lg N
 N logb a




if a  b d
if a  b d
if a  b d
20
• Example 1: T( N )  2 T( N / 2)  1
a  2, b  2; f ( N )  1  N  d  0
0
a  bd
case 3

T( N )   N
logb a
  N
log2 2
  N 
21
• Example 2: T( N )  T( N / 2)  N
a  1, b  2; f ( N )  N  N  d  1
1
a  bd
case 1
   N   N 
T( N )   N
d
1
22
• Example 3: T( N )  2 T( N / 2)  N  1
a  2, b  2; f ( N )  N  1  d  1
a  bd
case 2


T( N )   N lg N  N lg N 
d
23
Closest Pair Problem
• Given a set of N points in space, which
two are the closest?
• A brute force method calculates the
distance between every pair of points
using:
2
2
d p1, p2  
x2  x1    y2  y1 
• But this does (N 2 – N) / 2 distance
calculations
24
Brute Force Algorithm
smallDist =
∞
for i = 1 to N do
for j = i+1 to N do
dist = sqrt((xi – xj)2 + (yi – yj)2)
if dist < smallDist then
smallDist = dist
first = i
second = j
end if
end for
end for
25
A Divide and Conquer Solution
• Divide the set of points in into a right and left half
• Find the closest pair in the right and left half
(recursively)
• Determine the shortest (ds) of these two distances
• Check if there is a pair of points within ds units but on
opposite sides of the dividing line that are closer
26
Divide and Conquer Example
27
Divide and Conquer Example
ds
ds
ds
ds
m-ds m m+ds
28
Divide and Conquer Example
ds
29
Divide and Conquer Example
30
Divide and Conquer Closest Pair (Part 1)
ClosestPair( Px, Py, d, p1, p2)
//
//
//
//
//
Px
Py
d
p1
p2
the
the
the
the
the
points sorted by x coordinate
points sorted by y coordinate
shortest distance between points p1 and p2
first point
second point
Px
if sizeOf(Px)  3 then
find d, p1, and p2 by brute force
return
Lx
Rx
end if
construct Lx, Ly, Rx, Ry
ClosestPair(Lx, Ly, dl, L1, L2)
ClosestPair(Rx, Ry, dr, R1, R2)
Py
Ly
Ry
// Next slide
31
Divide and Conquer Closest (Part 2)
d = minimum(dl, dr)
if d == dl then
p1 = L1
p2 = L2
else
Mx & My consist of points whose x-coordinates
p1 = R1
are in the range [m-d, m+d].
p2 = R2
Mx: consist of those points sorted by x-coor.
end if
My: consist of those points sorted by y-coor.
construct Mx and My
for i = 1 to sizeOf(Mx)-1 do
for j = i+1 to i+7 (with j  sizeOf(My)) do
temp = distance(Myi, Myj)
if temp < d then
d = temp
p1 = Myi
p2 = Myj
end if
end for
end for
32
Divide and Conquer Closest Point Analysis
•
The points are divided into two halves
•
The combination step looks at the seven points closest to those points
within ds of the dividing line. In the worst case, N/2 points could be
in this strip.
This part takes (N) time.
•
•
Let T(N) be the computational time to solve the problem of N points
Therefore, the recurrence relation is
T(N) = 2 * T(N/2) + (N)
(see the Master method, a=2, b=2, d=1)
•
This algorithm is  (N lg N)
33
Convex Regions
• A region is convex if a line connecting
every pair of points in the region lies
entirely within the region
34
Convex Hull
• A convex hull for a set of points is the
smallest convex region including all of
the points
• All of the points
lie on one side
of each edge
of the convex
hull
35
Brute Force Convex Hull
• Consider the line defined by the first pair of
points
• If all of the other points lie on one side of that line,
the line is part of the convex hull (N-2 points)
• Repeat this process for every other pair of points
( N(N-1) / 2 pairs )
• This process is O(N3)
36
A Divide and Conquer Solution
• The leftmost and rightmost points are on the convex
hull
• Use the line between these points to divide the set of
points into an upper and lower set
• Find the convex hull of the two parts
• The convex hull of the entire set is the convex
hulls of these two parts without the dividing line
37
A Divide and Conquer Solution
• To find the convex hull of each part
– Find the point farthest away from the dividing line
– If all the points in this part are inside the triangle formed
with this point, this is the convex hull
– If there are points outside one of these two edges,
recursively find the convex hull of those points
– If necessary, repeat for the other edge
38
Divide and Conquer Example
ps
pe
39
Divide and Conquer Example
pf
ps
pe
pf
40
Divide and Conquer Example
41
Divide and Conquer Convex Hull
quickHull(S, ps, pe)
if S == {} then
return [] // the empty list
else
pf = point of S farthest from the dividing line
PR = set of points to the right of the line
between ps and pf
PL = set of points to the right of the line
between pf and pe
return quickHull(PR, ps, pf) + [pf]
+ quickHull(PL, pf, pe)
end if
42
Divide and Conquer
Convex Hull Analysis
• If the algorithm divides the points into two
halves, the recurrence relation is
T(N) = 2 * T(N/2) + (N)
and the closed form is (N lg N)
• In the worst case, all the points wind up on
one side of each dividing line and the
algorithm is then O(N2)
43
Permutations
• The permutation of a set of elements is an
ordering of those elements
• The permutations of the numbers from 1 to 3
are [1, 2, 3], [1, 3, 2], [2, 1, 3], [2, 3, 1], [3, 1,
2], and [3, 2, 1]
• If there are N elements in the set, there are
N ! permutations
44
Recursive Permutations
• Swap pairs of list values during the recursion
• Output the list when the “bottom” of the
recursion is reached
• The algorithm by Heap will permute the
elements at the front of the list and will then
swap in the next element
• It then permutes the front elements again
45
Recursive Permutations Algorithm
heapPermute(n)
// list is a global variable
if n == 0 then
output list
else
for i = 1 to n do
heapPermute(n-1)
if n is odd then
swap list[1] and list[n]
else
swap list[i] and list[n]
end if
end for
end if
46
Iterative Permutations
• Permutations can be listed in lexical
order
• The position within this list is a
permutation’s rank in the range [0,
N! – 1]
• It is possible to iteratively generate the
permutation from this rank number by
using the factorial of the values from 1
to N
47
Iterative Permutations Algorithm
list[size] = 1
for j = 1 to size – 1 do
d = (rank mod f[j + 1]) / f[j]
rank = rank – d * f[j]
list[size – j] = d+1
for i = size – j + 1 to size do
if list[i] > d then
list[i] = list[i] + 1
end if
end for
end for
48
Recursion and Stacks
• Every subprogram has an Activation Record
(AR) that includes the space needed for
parameters, local variables, a return value,
and a place to return control when done
• Every time a subprogram is called during
execution, an instance of this AR (ARI) is
created and placed on the system stack
49
Recursion and Stacks
• When a subprogram completes,
its AR instance (ARI) is
removed from the
system stack
• For recursive subprograms,
there will be multiple
ARIs on the stack –
one for each call
50
Recursion and Stacks
• Tracing recursive subprograms is easier if
you simulate the system stack by drawing
boxes when subprograms are called and
crossing them out when the subprogram
finishes
51