Introduction To Algorithms (4th edition) by Thomas
H. Cormen:
Exercise Solutions
2
Contents
1
4.Divide-and-Conquer
1.1 [4.1] Multiplying Square Matrices . . . . . . . . . . . . .
1.2 4.2 Strassen’s Algorithm for Matrix Multiplication . . .
1.3 4.3 Substitution Method for Solving Recurrences . . . .
1.4 4.4 The Recursion-Tree Method For Solving Recurrences
3
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
5
5
7
8
8
4
Chapter 1
4.Divide-and-Conquer
1.1
[4.1] Multiplying Square Matrices
Exercise 4.1-1
Generalize MATRIX-MULTIPLY-RECURSIVE to multiply n × n matrices for which n is not necessarily an exact power of 2. Give a recurrence describing its running time. Argue that it runs in Θ(n3 )
time in the worst case.
Solution. By letting m = 2⌈lg(n)⌉ and augmenting the n × n matrix with zeros in the additional (m − n)
rows and columns, we can effortlessly apply our original recursive algorithm to the new m × m matrix
where m is an exact power of 2:
GENERALIZED-MATRIX-MULTIPLY-RECURSIVE(A, B, C, n)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
if n == 1
//Base case.
C11 = C11 + A11 × B11
return
//Divide
if lg (n) ∈
/N
let A′ , B ′ and C ′ be new empty m × m matrices where m = 2⌈lg(n)⌉
for i = (m − n) + 1 to m
for j = 1 to n
let k = i−(m − n)
A′ij = Akj
Bij′ = Bkj
′
′
setzero(A′kj , A′jk , Bkj
, Bjk
)
else
let A′ , B ′ , C ′ , and m equal to A, B, C, and n respectivly
Partition A′ , B ′ , andC ′ into m/2 x m/2 submatrices
A11, A12, A21, A22; B11, B12, B21, B22; and C11, C12, C21, C22; respectively
//Conquer
MATRIX−MULTIPLY−RECURSIVE(A11, B11, C11, m/2)
MATRIX−MULTIPLY−RECURSIVE(A11, B12, C12, m/2)
MATRIX−MULTIPLY−RECURSIVE(A21, B11, C21, m/2)
MATRIX−MULTIPLY−RECURSIVE(A21, B12, C22, m/2)
MATRIX−MULTIPLY−RECURSIVE(A12, B21, C11, m/2)
MATRIX−MULTIPLY−RECURSIVE(A12, B22, C12, m/2)
MATRIX−MULTIPLY−RECURSIVE(A22, B21, C21, m/2)
MATRIX−MULTIPLY−RECURSIVE(A22, B22, C22, m/2)
5
The constraint that lg(n) ∈
/ N is applied only once, causing each of the two inner loops to execute
n + 1 times. This contributes a total cost of Θ(1) + 2Θ(n2 ) = Θ(n2 ) (including the base case). As a
result, the total cost of the original algorithm, T (n) = 8T (n/2) + Θ(n2 ), remains unchanged. And since
O(n2 ) = O(n3 ), case 1 of the master theorem applies with a = 8, b = 2, nlogb a = n3 , and f (n) = Θ(n2 ).
Therefore, T (n) = Θ(n3 ).
Exercise 4.2-1
How quickly can you multiply a kn × n matrix (kn rows and n columns) by an n × kn matrix, where
k ≥ 1, using MATRIX-MULTIPLY-RECURSIVE as a subroutine? Answer the same question for
multiplying an n × kn matrix by a kn × n matrix. Which is asymptotically faster, and by how much?
Solution. The former will result in a kn × kn = k 2 (n × n) matrix, contributing a total cost of k 2 Θ(n3 ) =
Θ(k 2 n3 ). The later on the other hand, will result in an n × n matrix, And since the number of columns
is k times the number of rows in each matrix, we have to perform k matrix multiplications of two n by n
matrices (regarding the summations), thereby contributing a total of kΘ(n3 ) = Θ(kn3 ). And thus, clearly
the later is faster by a factor of k.
Exercise 4.1-3
Suppose that instead of partitioning matrices by index calculation in M AT RIX − M U LT IP LY −
RECU RSIV E, you copy the appropriate elements of A, B, and C into separate n/2 × n/2 submatrices A11, A12,A21, A22; B11, B12, B21, B22; and C11, C12, C21, C22, respectively. After the
recursive calls, you copy the results from C11, C12, C21, and C22 back into the appropriate places in
C. How does recurrence (4.9) change, and what is its solution?
Solution. Doing so will simply require us to adjust two main steps in the algorithm:
1. Partitioning: This involves looping twice and copying 2n2 elements into 2n2 distinct sub-matrices which
will result in Θ(n2 ) time complexity.
2. Conquering: Similarly, combining the sub-matrices will result in Θ(n2 ).
Thus (-regarding the base case), Doing so will result in T (n) = 8T (n/2) + 2Θ(n2 ) = 8T (n/2) + 2Θ(n2 )
time complexity.
Exercise W
rite pseudo-code for a divide−and−conquer algorithm MATRIX−ADD−RECURSIVE that sums two
n × n matrices A and B by partitioning each of them into four n/2 × n/2 sub-matrices and then recursively summing corresponding pairs of submatrices. Assume that matrix partitioning uses Θ(1)−time
index calculations. Write a recurrence for the worst-case running time of MATRIX−ADD−RECURSIVE,
and solve your recurrence. What happens if you use Θ(n2 )−time copying to implement the partitioning
instead of index calculations?
Solution.
MATRIX-ADD-RECURSIVE(A, B, C, n)
1
2
3
4
5
6
7
8
9
10
11
12
if n == 1
//Base case.
C11 = A11 + B11
return
//Divide
Partition A′ , B ′ , andC ′ into n/2 x n/2 submatrices
A11, A12, A21, A22; B11, B12, B21, B22; and C11, C12, C21, C22; respectively
//Conquer
MATRIX−ADD−RECURSIVE(A11, B11, C11, n/2)
MATRIX−ADD−RECURSIVE(A12, B12, C12, n/2)
MATRIX−ADD−RECURSIVE(A21, B21, C21, n/2)
MATRIX−ADD−RECURSIVE(A22, B12, C22, n/2)
6
1.2
4.2 Strassen’s Algorithm for Matrix Multiplication
Exercise 4.2-1
Use Strassen’s algorithm to compute the matrix product. Show your work.
1 3
7 5
6 8
4 2
Solution. P 1 = 6, P 2 = 8, P 3 = 72, P 4 = −10, P 5 = 48, P 6 = −12, P 7 = −84 and C11 = 18C12 =
14C21 = 62C22 = 66 Thus C =
18 14
62 66
Exercise 4.2-2
Write pseudo-code for Strassen’s algorithm.
Solution.
STRASSEN-ALGORITHM(A, B, C, n)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
if n == 1
//Base case.
C11 = C11 + A11 × B11
return
//Divide
Partition A′ , B ′ , andC ′ into m/2 x m/2 submatrices
A11, A12, A21, A22; B11, B12, B21, B22; and C11, C12, C21, C22; respectively
//Compute S matrices
MATRIX−ADD−RECURSIVE(B12, −B22, S1, n/2)
MATRIX−ADD−RECURSIVE(A11, A12, S2, n/2)
MATRIX−ADD−RECURSIVE(A21, A22, C, n/2)
...
Matrix−Add−Recursive(B11, B12, S10, n/2)
//Compute P matrices
Strassen−Algorithm(A11, S1, P1, n/2)
Strassen−Algorithm(S2, B22, P2, n/2)
...
Strassen−Algorithm(S9, S10, P7, n/2)
//Combine...
Matrix−Add−Recursive(P5, P4, C11, n/2)
Matrix−Add−Recursive(−1P2, P6, C11, n/2)
...
Matrix−Add−Recursive(−1P3, −1P7, C22, n/2)
Exercise 4.2-3
What is the largest k such that if you can multiply 3 × 3 matrices using k multiplications (not assuming
commutativity of multiplication), then you can multiply n × n matrices in o(nlg7) time? What is the
running time of this algorithm?
7
Solution. Computing the product of 3 by 3 matrices using k multiplications, will result in k sub-matrices
of n/3 sizes, giving us the recurrence:
T (n) = kT (n/3) + Θ(1)
From case 1 of the master theorem, we conclude the solution T (n) = Θ(nlog3 k ). Thus, T (n) = o(nlg7 ) iff
nlog3 k < nlg7 , simplifying for k we get k < 3lg7 , and therefore, k = 21.
Exercise 4.2-4
V. Pan discovered a way of multiplying 68 × 68 matrices using 132,464 multiplications, a way of
multiplying 70 × 70 matrices using 143,640 multiplications, and a way of multiplying 72 × 72 matrices
using 155,424 multiplications. Which method yields the best asymptotic running time when used in a
divide-and-conquer matrix-multiplication algorithm? How does it compare with Strassen’s algorithm?
Answer: The second method which runs asymptotically faster than Strassen’s algorithm since lg7 >
log70 143640.
1.3
4.3 Substitution Method for Solving Recurrences
Exercise 4.3-1
Use the substitution method to show that each of the following recurrences defined on the reals has
the asymptotic solution specified:
(a) T (n) = T (n − 1) + n has solution T (n) = O(n2 ).
(b) T (n) = T (n/2) + Θ(1) has solution T (n) = O(lgn).
(c) T (n) = 2T (n/2) + n has solution T (n) = Θ(nlgn).
(d) T (n) = 2T (n/2 + 17) + n has solution T (n) = O(nlgn).
(e) T (n) = 2T (n/3) + Θ(n) has solution T (n) = Θ(n).
(f ) T (n) = 4T (n/2) + Θ(n) has solution T (n) = Θ(n2 ).
Solution.
(a) Suppose that T (n − 1) ≤ c(n − 1)2 . Substituting for T (n − 1) in the original recurrence we get:
T (n) ≤ c(n − 1)2 + n
(1.1)
2
≤ cn − 2cn + c + n
(1.2)
≤ cn2 − (n(2c − 1) − c)
(1.3)
2
Where the last step holds for all n ≥
c
2c−1
≤ cn
(1.4)
= O(n2 )
(1.5)
and for all c > 1/2 for which c ∈ R
(c)
1.4
4.4 The Recursion-Tree Method For Solving Recurrences
Exercise 4.4-1
For each of the following recurrences, sketch its recursion tree, and guess a good asymptotic upper
bound on its solution. Then use the substitution method to verify your answer.
(a). T (n) = 2T ( n2 ) + n3 .
(b). T (n) = 4T ( n3 ) + n.
8
(c). T (n) = 4T ( n2 ) + n.
(d). T (n) = 3T (n − 1) + 1.
Solution.
(a) A recursion tree for the recurrence a is modeled as follows...:
cn3
c( 2n1 )3
c( 2n2 )3
c( 2n3 )3
c( 2n1 )3
c( 2n2 )3
c( 2n3 )3
c( 2n3 )3
c( 2n2 )3
c( 2n3 )3
c( 2n3 )3
c( 2n3 )3
( 2n2 )3
( 2n3 )3
( 2n3 )3
Θ(1)Θ(1)Θ(1)Θ(1)Θ(1)Θ(1)Θ(1)Θ(1)Θ(1)....
...Θ(1)
Where c is some arbitrary constant. Let i represent the i − th level starting from 0,1,2... up to log2 (⌊ nn0 ⌋)
where 0 ≤ n0 ≤ n is some threshold constant such that the recurrence is well defined. Then the total cost
can be found by summing up all the nodes of the tree:
log2 (⌊ nn ⌋)
X0
c
i=0
log2 (⌊ nn ⌋)
n3
2i ( i ) = cn3
8
X0
∞
(
i=0
X 1
4
1
) ≤ cn3
( )i ≤ cn3 = O(n3 )
i
4
4
3
(1.6)
i=0
Where the last step holds as long as 0 < c < n. To see that our solution holds; Suppose that T ( n2 ) =
O(( n2 )3 ), then:
n
n
T ( ) ≤ c0 ( )3
2
2
(1.7)
n
T (n) = 2c0 ( )3 + n3 = c0 n3 + n3 = (c0 + 1)n3 ≤ dn3 = O(n3 )
2
(1.8)
substituting for T ( n2 ), we obtain:
where the last step holds for all d ≥ (c0 + 1) and for all 0 < c0 < n.
(b) A recursion tree for the recurrence b is modeled as follows...:
cn
c( 3n1 )
c( 3n2 )
c( 3n1 )
c( 3n2 )
c( 3n2 )
c( 3n1 )
c( 3n2 )
c( 3n2 )
c( 3n2 )
Θ(1)Θ(1)Θ(1)Θ(1)Θ(1)Θ(1)Θ(1)Θ(1)....
c( 3n1 )
c( 3n2 )
c( 3n2 )
...Θ(1)
Where c is some arbitrary constant. Let i represent the i − th level starting from 0,1,2... up to log3 (⌊ nn0 ⌋)
where 0 ≤ n0 ≤ n is some threshold constant such that the recurrence is well defined. Then the total can
be found by summing up all the nodes of the tree:
log3 (⌊ nn ⌋)
c
X0
i=0
n
4i ( i ) = cn
3
log3 (⌊ nn ⌋)
X0
i=0
(
∞
X 4
4i
) ≤ cn
( )i ≤ −3cn = O(n)
i
3
3
i=0
9
(1.9)
Where the last step holds as long as 0 > c > −n. To see that our solution holds; Suppose that T ( n3 ) = O( n3 ),
then:
n
n
T ( ) ≤ c0 ( )
3
3
(1.10)
substituting for T ( n3 ) in the recurrence a, we obtain:
n
4
4
T (n) = 4c0 ( ) + n = c0 n + n = n( c0 + 1) ≤ dn = O(n)
3
3
3
(1.11)
Where the last step holds for all d ≥ 43 c0 + 1 and for all 0 < c0 < n.
(c) the O(n) upper bound for the recurrence c is easily derived and verified using the same logic as in
a and b...
Exercise 4.5-1
Use the master method to give tight asymptotic bounds for the following recurrences.
a. T (n) = 2T (n/4) + 1.
√
b. T (n) = 2T (n/4) + n.
√ 2
c. T (n) = 2T (n/4) + nlg n.
d. T (n) = 2T (n/4) + n.
e. T (n) = 2T (n/4) + n2 .
Solution. Note that in all of the recurrences, we’re given that a = 2, b = 4, and nlogb a =
√
n.
(a) Since f (n) = O(1) = O(nlog4 2−1 ), case 1 of the master theorem is applied. Therefore, T (n) =
√
Θ(nlog4 2 ) = Θ( n).
√
(b) Since f (n) = O( n) = O(nlog4 2 ∗ 1), case 2 of the master theorem is applied. Therefore, T (n) =
√
Θ(nlog4 2 lgn) = Θ( nlgn).
(c) Since f (n) = Ω(n) = Ω(nlog4 2+2 ), and
Exercise 4.4-4
Use a recursion tree to justify a good guess for the solution to the recurrence T (n) = T (αn) + T ((1 −
α)n) + Θ(n), where α is a constant in the range 0 ≤ α ≤ 1.
Solution.
10