Divide and Conquer
Faculty Name: Ruhi Fatima
Topics Covered
• Divide and Conquer
• Matrix multiplication
• Recurrence
Divide and Conquer
• Divide the problem into a number of subproblems that are smaller
instances of the same problem.
• Conquer the subproblems by solving them recursively. If the
subproblem sizes are small enough, however, just solve the
subproblems in a straightforward manner.
• Combine the solutions of the subproblems into the solution for the
original problem.
• When the subproblems are large enough to solve recursively, we
call that the recursive case. Once the subproblems become small
enough that we no longer recursive, we say that the recursion
“bottoms out” and that we have gotten down to the base case.
Matrix multiplication
• Let A, B and C be n  n matrices
C = AB
C(i, j) =  A(i, k)B(k, j)
1 k  n
• The straightforward method (Brute Force Method) to perform
a matrix multiplication requires 8 multiplications and 4
additions resulting O(n3) time complexity.
Divide-and-conquer approach
• C = AB
C11 C12 = A11 A12
B11 B12
C21 C22 = A21 A22
B21 B22
C11 = A11 B11 + A12 B21
C12 = A11B12 + A12 B22
C21 = A21 B11 + A22 B21
C22 = A21 B12 + A22 B22
• Time complexity:
 b
,n2
T(n) = 
8T(n/2)+cn2 , n > 2 (# of additions : n2)
We get T(n) = O(n3)
Divide and Conquer ( Cont..)
• Derive a recurrence to characterize the running time of
SQUAREMATRIX-MULTIPLY-RECURSIVE.
• Let T(n) be the time to multiply two n x n matrices using this
procedure.
• In the base case, when n = 1, we perform just the one scalar
multiplication in line 4, and so
( Cont..)
• The recursive case occurs when n > 1.
• In lines 6–9, we recursively call SQUARE-MATRIX-MULTIPLYRECURSIVE a total of eight times.
• Because each recursive call multiplies two n/2 x n/2 matrices, thereby
contributing T (n/2) to the overall running time, the time taken by all
eight recursive calls is 8T(n/2).
• Also must account for the four matrix additions in lines 6–9.
• Each of these matrices contains n2/4 entries, and so each of the four
matrix additions takes Θ(n2) time.
• The total time for the recursive case, therefore, is the sum of the
partitioning time, the time for all the recursive calls, and the time to
add the matrices resulting from the recursive calls:
• Combining equations (4.15) and (4.16) gives us the recurrence for
the running time of SQUARE-MATRIX-MULTIPLY-RECURSIVE:
Strassen’s matrix multiplication Method
1. Divide the input matrices A and B and output matrix C into
n/2 x n/2 submatrices. This step takes Θ(1) time by index
calculation, just as in SQUARE-MATRIX-MULTIPLY.
2. Create 10 matrices S1, S2, ….,S10, each of which is n/2 x n/2
and is the sum or difference of two matrices created in step 1.
All 10 matrices can be created in Θ(n2) time.
3. Using the submatrices created in step 1 and the 10 matrices
created in step 2, recursively compute seven matrix products
P1, P2, ….,P7. Each matrix Pi is n/2 x n/2 .
4. Compute the desired submatrices C11,C12, C21, C22 of the result
matrix C by adding and subtracting various combinations of the
Pi matrices. We can compute all four submatrices in Θ(n2)
time.
Cont…
C11
C12
C21
C22
=
=
=
=
A11 B11
A11B12
A21 B11
A21 B12
• P1 = A11 S1
P2 = S2 B22
P3 = S3 B11
P4 = A22 S4
P5 = S5 S6
P6 = S7 S8
P7 = S9 S10
• C11
C12
C21
C22
=
=
=
=
P5
P1
P3
P5
+
+
+
+
+
+
+
+
A12
A12
A22
A22
B21
B22
B21
B22
S1
S3
S5
S7
S9
=
=
=
=
=
B12 - B22 , S2 = A11 + A12
A21 + A22 , S4 = B21 - B11
A11 + A22 , S6 = B11 + B22
A12 - A22 , S8 = B21 + B22
A11 - A21 , S10= B11 + B12
= A11B12 - A11B22
= A11B22 + A12B22
= A21B11 + A22B11
= A22 B21 – A22B11
= A11 B11 + A11B22 + A22 B11 + A22B22
= A12 B21 + A12B22 - A22 B21 - A22B22
= A11 B11 + A11B12 - A21 B11 - A21B12
P4 - P2 + P 6
P2
P4
P1 - P3 - P 7
Time Complexity of Strassens
• 7 multiplications and 18 additions or subtractions
• Time complexity:
T(n)
b
,n2
7T(n/2)+an2 , n > 2

= 
T ( n )  an 2  7T ( n / 2)
 an 2  7( a ( n2 ) 2  7T ( n / 4)
 an 2 
7
4
an 2  7 2 T ( n / 4)


 an 2 (1 
7
4
 ( 74 ) 2    ( 74 ) k 1 )  7 k T (1)
 cn 2 ( 74 ) log2 n  7log2 n , c is a constant
 cn 2 ( 74 )log2 n  n log2 7  cn log2 4 log2 7 log2 4  n log2 7
 O( n log2 7 )  O( n 2.81 )
Recurrence
• A recurrence is an equation or inequality that describes a function in
terms of its value on smaller inputs.
EXAMPLE : Compute the factorial function F(n) = n! for an arbitrary
nonnegative integer n. Since n!= 1 . . . . . (n − 1) . n = (n − 1)! . n for n ≥ 1
and 0!= 1 by definition, we can compute F(n) = F(n − 1) . n with the following
recursive algorithm.
ALGORITHM F(n)
//Computes n! recursively
//Input: A nonnegative integer n //Output: The value of n!
if n = 0 return 1
else return F(n − 1) ∗ n
• The basic operation of the algorithm is multiplication, whose number of
executions we denote M(n). Since the function F(n) is computed according
to the formula F(n) = F(n − 1) . n for n > 0, F(0) = 1.
M(n) = M(n − 1) + 1 for n > 0, M(0) = 0.
• So, two recursively defined functions are dealed. The first is the factorial
function F(n) itself, the second is the number of multiplications M(n)
needed to compute F(n) by the recursion.
Methods for solving recurrences
There are three methods for solving recurrences
• Substitution method, guess a bound and then use mathematical
induction to prove our guess correct.
• Recursion-tree method converts the recurrence into a tree whose
nodes represent the costs incurred at various levels of the recursion.
We use techniques for bounding summations to solve the
recurrence.
• Master method provides bounds for recurrences of the form
T(n)= aT(n/b) + f(n)
where a≥1, b >1, and f (n) is a given function. A recurrence of the form
in equation characterizes a divide and-conquer algorithm that creates a
subproblems, each of which is 1/b the size of the original problem, and
in which the divide and combine steps together take f (n) time
Substitution Method
• The substitution method for solving recurrences (divide and
conquer) comprises two steps:
1. Guess the form of the solution.
2. Use mathematical induction to find the constants and show
that the solution works.
• Substitute the guessed solution for the function when applying
the inductive hypothesis to smaller values; hence the name
“substitution method.” This method is powerful, but we must be
able to guess the form of the answer in order to apply it.
• Substitution method to establish either upper or lower bounds
on a recurrence.
Cont…
• Example: Determine an upper bound on the recurrence
• We guess that the solution is T(n) =O(n lg n).
• The substitution method requires us to prove that T(n)≤cn lg n
for an appropriate choice of the constant c > 0.
• Substituting into the recurrence yields
Recursion Tree
• In a recursion tree, each node represents the cost of a single
subproblem somewhere in the set of recursive function
invocations. We sum the costs within each level of the tree to
obtain a set of per-level costs, and then we sum all the perlevel costs to determine the total cost of all levels of the
recursion.
• For example, let us see how a recursion tree would provide a
good guess for the recurrence
• We start by focusing on finding an upper bound for the
solution. Because we know that floors and ceilings usually do
not matter when solving recurrences (here’s an example of
sloppiness that we can tolerate), we create a recursion tree for
the recurrence, having written out the implied constant
coefficient c > 0.
Example of recursion tree
Solve T(n) = T(n/4) + T(n/2) + n2:
n2
n2
(n/2)2
(n/4)2
(n/8)2
(n/8)2
(n/4)2
…
(n/16)2
5 n2
16
25 n 2
256
Q(1)
Total =
n
2
= Q(n2)

   
2
5
5
1  16  16
geometric series
Appendix: geometric series
1 x  x
2
 x
1  x  x2   
n
1  x n 1

1 x
1
1 x
for |x| < 1
5 3
16
for x  1


Master Theorem
• The master method provides a “cookbook” method for solving
recurrences of the form
T (n) = aT (n/b) + f (n) ; where a ≥ 1 and b > 1 are
constants and f (n) is an asymptotically positive function.
• Master method depend on the Master Theorem
Cont…
Example 1: Solve the recurrence T(n) = 9T(n/3) +n.
•
Solution: In the given recurrence a = 9, b = 3, f(n) = n, and thus
𝑛log𝑏 𝑎 = 𝑛log3 9 = Θ 𝑛2 .
Since f(n) = O (𝑛log3 9−ϵ ), where ϵ = 1. So, by applying case 1 we
get T(n)=Θ(𝑛log3 9 ) = Θ 𝑛2 .
• Example 2: Consider T( n) = T ( 2n/3) +1
Solution: In the above a = 1, b = 3/2, f(n) = 1, and 𝑛log𝑏 𝑎 = 𝑛log3/2 1 =
𝑛0 = 1. So, case 2 is to be applied, which results the solution to the
recurrence is T(n) =Θ (lg n).
• Example 3: Consider the recurrence T(n) = 3 T(n/4) + n lg n.
Solution: In the above a = 3, b = 4, f(n) = n lg n, and 𝑛log𝑏 𝑎 = 𝑛log4 3 =
.
𝑂 𝑛0 793 = 1. Since f(n) = Ω (𝑛log4 3+ϵ ) , where ϵ ≈ 0.2, case 3
applies if we can show that the regularity condition holds for f( n). For
sufficiently large n, we have that
af( n/b ) = 3(n/4)lg (n/4) ≤ (3/4) lg (n/4) ≤ (3/4)n lg n = c f(n) for c=3/4.
Cont…
Example 4: Solve the recurrence T(n) = 2 T(n/2) + n lg n.
•
Solution: The master method does not apply to the above recurrence.
Even though it appears to have the proper form: a = 2, b = 2, f(n)=nlgn,
and 𝑛log𝑏 𝑎 = n.
It might be mistakenly as case 3, since f (n) = n lg n is asymptotically
larger than 𝑛log𝑏 𝑎 = n. The problem is that it is not polynomially larger.
The ratio f (n ) / 𝑛log𝑏 𝑎 = n lg n/n = lg n is asymptotically less than n for
any positive constant .
Consequently, the recurrence falls into the gap between case 2 and
case 3.
Example 5: Let’s use the master method to solve the recurrences
T (n)= 2T(n /2 )+ Θ(n) , characterizes the running times of the
divide-and-conquer algorithm.
Here, we have a = 2, b = 2, f (n) = Θ(n) and thus we have that
𝑛log𝑏 𝑎 = 𝑛log2 2 = n.
Case 2 applies, since f (n) = Θ(n), and so we have the solution
T(n) = Θ(n lg n).
Cont…
Example 5: Let’s use the master method to solve the recurrences
T (n)= 8T(n /2 )+ Θ(n2) , describes the running times of the divideand-conquer algorithm for matrix multiplication.
Now we have a = 8, b = 2, and f (n) = Θ(n2), and so
𝑛log𝑏 𝑎 = 𝑛log2 8 = 𝑛3.
Since 𝑛3 is polynomially larger than f (n) (i.e., f (n) = O(n3-ϵ) for ϵ = 1),
case 1 applies, and T (n) = Θ(n3).
Example 6: Consider recurrence T (n)= 7T(n /2 )+ Θ(n2), which
describes the running time of Strassen’s algorithm.
Here, we have a =7, b = 2, f (n) = Θ(n2), and thus 𝑛log𝑏 𝑎 = 𝑛log2 7 .
Rewriting log 2 7as lg 7 and recalling that 2:80 < lg7 < 2:81, we see that
f (n) = O(nlg 7-ϵ) for ϵ = 0.8.
Again, case 1 applies, and we have the solution T (n) = Θ(nlg 7).
Proof of Master Theorem
• The proof appears in two parts.
• The first part analyzes the master recurrence, under the simplifying
assumption that T (n) is defined only on exact powers of b > 1, i.e,
for n = 1, b, b2, .....
• The second part shows how to extend the analysis to all positive
integers n; it applies mathematical technique to the problem of
handling floors and ceilings.
• The proof for exact powers :The first part of the proof of the
master theorem analyzes the recurrence T (n) = aT(n/b) + f(n) ,
under the assumption that n is an exact power of b > 1, where b
need not be an integer.
• The analysis is broken into three lemmas.
• The first reduces the problem of solving the master recurrence to
the problem of evaluating an expression that contains a summation.
• The second determines bounds on this summation.
• The third lemma puts the first two together to prove a version of the
master theorem for the case in which n is an exact power of b.
Proof of Master Theorem
• Lemma 1