Lecture 4. Paradigm #2 Recursion
 Last time we discussed Fibonacci numbers F(n), and Alg
 fib(n)
if (n <= 1) then return(n)
else return(fib(n-1)+fib(n-2))
 The problem with this algorithm is that it is woefully
inefficient. Let T(n) denote the number of steps needed
by fib(n). Then T(0) = 1, T(1) = 1, and T(n) = T(n-1) +
T(n-2) + 1. It is now easy to guess the solution
T(n) = 2 F(n+1) - 1
Proof by induction: Clearly the claim is true for n = 0, 1.
Now assume it is true for all n < N; we prove it for n = N:
T(N) = T(N-1) + T(N-2) + 1
= (2 F(N) - 1) + (2 F(N-1) - 1) + 1
= 2(F(N)+F(N-1)) - 1
= 2 F(N+1) - 1
 We know F(n) = Θ(an), a=(1+√5)/2
Memoization
 A trick called "memoization“ can help recursion, store function values
as they are computed. When the function is invoked, check the
argument to see if the function value is already known; that is, don't
recompute. Some programming languages even offer memoization as
a built-in option. Here are two Maple programs for computing Fibonacci
numbers; the first just uses the ordinary recursive method, and the
second uses memoization (through the command "option remember").
 A recursive one:
f := proc(n)
if n ≤ 1 then n else f(n - 1) + f(n - 2) end if
end proc;
 One that looks recursive, but that uses memoization:
g := proc(n)
option remember;
if n ≤ 1 then n else g(n - 1) + g(n - 2) end if
end proc;
 When you run these and time them, you'll see the amazing difference in
the running times. The "memoized" version runs in linear time.
Printing permutations
 Sometimes, recursion does provide simple & efficient
solutions. Consider the problem of printing out all
permutations of {1,2,...,n} in lexicographic order.
 Why might you want to do this? Well, some
combinatorial problems have no known efficient
solution (such as traveling salesman), and if you want
to be absolutely sure you've covered all the
possibilities for some small case.
 How can we do this? If we output a 1 followed by all
possible permutations of the elements other than 1;
then a 2 followed by all possible permutations of the
elements other than 2; then a 3 followed by .... etc.,
we'll have covered all the cases exactly once.
Printing all permutations …
 So we might try a recursive algorithm. What should the input
parameter be? If we just say n, with the intention that this gives
all permutations of {1,2,..., n} that's not going to be good
enough, since later we will be permuting some arbitrary subset
of this. So you might think that the input parameters should be
an arbitrary set S. But even this is not quite enough, since we
will have to choose an arbitary element i out of S, and then print
i followed by all the permutations of S - { i }. But if we don't want
to store all the permutations of S - { i } before we output them we
need some way to tell the program that when it goes and prints
all the permutations of S - { i }, it should print i first, preceding
each one. This suggests making a program with two
parameters: one will be the fixed "prefix" of numbers that is
printed out, and the second the set of remaining numbers to be
permuted.
Printing permutations
 printperm(P,S)
/* P is a prefix, S is a nonempty set */
if S={x} then print Px;
else for each element i of S do
printperm( (P,i), S - { i });
 There are n! permutations. So any program must spend O(n*n!)
time to print all the permutations, each taking n steps.
 Let me give a simple amortizing counting argument. We will be
printing n*n! symbols. Each time the "else" statement is
executed, we charge O(1) to that "i". This particular "i" in the
n*n! symbols (note i appears in different permutations many
times, but we are just referring to one instance of such "i") gets
charged only once. Summing up, the total time is O(n · n!).
Paradigm #3: Divide-and-conquer
 Divide et impera [Divide and rule]
-- Ancient political maxim cited by Machiavelli
-- Julius Caesar (102-44BC)
 The paradigm of divide-and-conquer:
-- DIVIDE problem up into smaller problems
-- CONQUER by solving each subproblem
-- COMBINE results together to solve
the original problem
Divide & Conquer: MergeSort
 Example: merge sort (an O(n log n) algorithm
for sorting) (See CLR, pp. 28-36.)
 MERGE-SORT(A, p, r)
/* A is an array to be sorted. This algorithm
sorts the elements in the subarray A[p..r] */
if p < r then
q := floor( (p+r)/2 )
MERGE-SORT(A, p, q)
MERGE-SORT(A, q+1, r)
MERGE(A, p, q, r)
MergeSort continues ..
 Let T(n) denote the number of comparisons
performed by algorithm MERGE-SORT on an
input of size n. Then we have
T(n) = 2T(n/2) + n
expanding …
= 2k T(n/2k) + kn
….
= O(nlogn)
--- when k=logn.
Another way:
 Prove T(2k) = (k+1)2k by induction:
 It is true for k = 0. Now assume it is true for k;
we will prove it for k+1.
 We have
T(2k+1) = 2T(2k) + 2k+1 (by recursion)
= 2(k+1)2k + 2k+1 (by induction)
= (k+2) 2k+1,
and this proves the result by induction.
Divide & conquer: multiply 2 numbers
 Direct multiplication of 2 n-bit numbers takes
n2 steps. Note, we assume n is very large,
and each register can hold only O(1) bits.
 When do we need to multiply two very large
numbers?




In Cryptography and Network Security
message as numbers
encryption and decryption need to multiply
numbers
My comment: but really: none of above seems to be a good
enough reason. Even you wish to multiply a number of 1000
digits, an O(n2) alg. is good enough!
How to multiply 2 n-bit numbers

************
************
************
************
************
************
************
************
************
************
************
************
************
************
************************
(n ) bit operations
2
History:
AN Kolmogorov
1903-1987
 1960, AN Kolmogorov (we will meet him again later in




this class) organized a seminar on mathematical
problems in cybernetics at MSU.
He conjectured Ω(n2) lower bound for multiplication
and other problems.
Karatsuba, then 25 year old student, proposed the
nlog3 solution in a week, by divide and conquer.
Kolmogorov was upset, he discussed this result in
the next seminar, which was then terminated.
The paper was written up by Kolmogorov, but
authored by Karatsuba (who did not know before he
received reprints) and was published in Sov Phys.
Dol.
Can we multiply 2 numbers faster?
 Karatsuba's 1962 algorithm for multiplying
two n bit numbers in O(n1.59) steps.
 Suppose we wish to multiply two n-bit
numbers X Y. Let X = ab, Y = cd where a, b,
c, d are n/2 bit numbers.
 Then XY = (a · 2n/2 + b)(c · 2n/2 + d)
= (ac)2n + (ad + bc)2n/2 + bd
X=
a
b
Y=
c
d
Multiplying 2 numbers
 So we have broken the problem up to 4
subproblems each of size n/2. Thus,
T(2k) = 4T(2k-1) + c 2k
4T(2k-1) = 16 T(2k-2) + 4c2k-1 = 42T(2k-2)+c2k+1
...
4k-1 T(2) = 4k T(1) + 4k-1 · c · 2
 Now T(1) = 1, so T(2k) = 4k + c(2k + 2k+1 + ... +
22k-1) ≤ 4k + c 4k = (4k)(c+1).
 This gives T(n) = O(n2)! No improvement!
Multiplying 2 numbers
 But Karatsuba did not give up. He observed:
XY = (2n + 2n/2)· ac + 2n/2· (a-b) · (d-c) + (2n/2 + 1)· bd
 Now, we have broken the problem up into only 3
subproblems, each of size n/2, plus some linear
work. This time it should work!
K(n) ≤ 3K(n/2) + cn
≤ c(3k+1 - 2k+1)
 Putting n = 2k, we see that for n a power of 2, we get
 K(n) ≤ c(3 lg n + 1) – 2 lg n + 1 )
= c(3 nlg 3 - 2 n)
 Here we have used the fact that alog(b) = blog(a).
 Since lg 3 is about 1.58496, this gives us a O(n1.59) algorithm
 Note: Using FFT. Schonhage and Strassen: O(nlogn loglogn) in
1971. In 2007, this was slightly improved by Martin Furer
Divide & conquer: finding max-min
 Problem: finding both the maximum and
minimum of a set of n numbers.
 Obvious method: first compute the maximum,
using n-1 comparisons; then discard this
maximum, and compute the minimum of the
remaining numbers, using n-2 comparisons.
Total work: 2n-3 comparisons.
Maxmin by divide & conquer
 MAXMIN(S)
/* find both the maximum and minimum elements of a set S */
if S = {a} then
min=a, max=a;
else if S = {a < b}
min=a, max=b;
else /* |S| > 2 */
divide S into 2 subsets, S1 and S2, such that S1 has floor(n/2)
elements and S2 has ceil(n/2) elements
(min1, max1) := MAXMIN(S1);
(min2, max2) := MAXMIN(S2);
min = min(min1, min2);
max = max(max1, max2);
return (max,min);
Time complexity
 T(n) = 1 when n =2
 T(n) = 2T(n/2) + 2, otherwise.
 This gives T(n) = 3n/2 – 2, when n is a power
of 2.