Divide-and-Conquer
7 2 I 9 4
2 4 7 9
7 I 2 9 2 7
7
7
2
2
Divide-and-Conquer
9
9I 4
4 9
9
4
4
0
Divide-and-Conquer
•
Divide-and conquer is a general
algorithm design paradigm:
●
●
•
quicksort
thisstep
Divide: divide the input data S in
two or more disjoint subsets S1, S2,
…
Recur: solve the subproblems
recursively
Conquer: combine the solutions for
S1, S2, …, into a solution for S
●
•
sort
generally wetrymerge
andbinarysearchfor
The base case for the recursion are
subproblems of constant size
Analysis can be done using
recurrence equations
algorithm
your
time
running
the
of
combine step can drastically affect
the
Divide-and-Conquer
0
Merge-Sort Review
•
●
●
●
Merge-sort on an input
sequence S with n
elements consists of three
steps:
Divide: partition S into two
sequences S1 and S2 of
about n/2 elements each
Recur: recursively sort S1
and S2
Conquer: merge S1 and S2
into a unique sorted
sequence
Algorithm mergeSort(S)
Input sequence S with n
elements
Output sequence S sorted
if S.size() > 1 handlesbasecase
(S1, S2) partition(S, n/2) Divide
mergeSort(S1)
mergeSort(S2) recur
S merge(S1, S2) Conquer
Olnlog n
Divide-and-Conquer
0
Recurrence Equation Analysis
•
•
•
The conquer step of merge-sort consists of merging two sorted sequences,
each with n/2 elements and implemented by means of a doubly linked list,
takes at most bn steps, for some constant b.
Likewise, the basis case (n < 2) will take at b most steps.
Therefore, if we let T(n) denote the running time of merge-sort:
b
⎧
T (n) = ⎨
⎩2T ( n / 2) + bn
•
●
if n < 2
if n ≥ 2
We can therefore analyze the running time of merge-sort by finding a
closed form solution to the above equation.
1
That is, a solution that has T(n) only on the left-hand side.
Divide-and-Conquer
0
ofeachsub problem thatoccursduring recursion
size
1
TC
Tin
that
of
occur
recursions
Tm
n
Tbs n
running time
oftheremainingalgorithm
ofsubproblems present
2Tm Yz
th
Tb I
t 1
where m is merge sort
a recurrence equation issolved to determine the runningtime ofthealgorithm
Solving a recurrence equation
we can do this in 3 ways
Substitution thisis done until
yougettothe basecase iterativesubstitution
recursion tree
master method
Tbs n
Basecase
Tb I
t 1
seepg 116forgreedyalgorithms
Tin
Y
2 T
TIN
th
T
212T E
22 T
n
E
2n
242T I 23
23 T
E
21092
T
n T i
n
Tcn
0
Yz
2h
In
ng
t n log n
th leg
k
n
n log n
t log
n
n
2 T
Ye
2T
E
Mz
Mz
Iterative Substitution
•
In the iterative substitution, or “plug-and-chug,” technique, we iteratively
apply the recurrence equation to itself and see if we can find a pattern:
T ( n ) = 2T ( n / 2) + bn
= 2( 2T ( n / 22 )) + b( n / 2)) + bn
= 22 T ( n / 22 ) + 2bn
= 23 T ( n / 23 ) + 3bn
= 24 T ( n / 24 ) + 4bn
= ...
= 2i T ( n / 2i ) + ibn
•
•
Note that base, T(n)=b, case occurs when 2i=n. That is, i = log n.
So,
T ( n ) = bn + bn log n
•
Thus, T(n) is O(n log n).
Divide-and-Conquer
0
Verifying I know how iterative substitution works by redoing
the example above maybe you're replacing T
Tin
TIM
Tin
T
Yaz
2T
2T
n 2
E
212T Y
tbh
b
finding T k
t b Nz tbn
byplugging in
into Tin
rewrite Tin
using the
calculated value
TL
by
for
findingTCM
byplugging in
Mzinto Tin
Anotherexample
of iterativesubstitution
iterations
Iff
I
Tin
Tin
Tin 1
Tin 1
2
Tin
8
Tch I
TIN 2
1
8
8
T n 2
8
The Recursion Tree
•
Draw the recursion tree for the recurrence relation and look for a pattern:
b
⎧
T (n) = ⎨
⎩2T ( n / 2) + bn
anodes
of
if n < 2
if n ≥ 2
rootoftherecursiontree
time
depth
T’s
size
0
1
n
bn
1
2
n//2
bn
i
2i
n//2i
bn
…
…
…
…
leafts
Total time = bn + bn log n
thebasecase
Divide-and-Conquer
(last level plus all previous levels)
0
Using
the recursiontree to analyze the runningtime ofbinarysearch
T Ya
Tin
depth
O
N
I
2
i
logon
I
Running
i
I
I
Y
TI
I
E
TI
I
1
Tt
I
whatisthis
Time
Master Method
•
Many divide-and-conquer recurrence equations have the
form:
ofsub
•
c
⎧
T (n) = ⎨
I
⎩aT ( n / b) + f ( n )
pith
if n < d
if n ≥ d
runningtime the
nonrecursive point
of Tereusportion
The Master Theorem:
runningtime
sizeofrecursion
of
1. if f (n) is O(n logb a −ε ), then T (n) is Θ(n logb a )
2. if f (n) is Θ(n logb a log k n), then T (n) is Θ(n logb a log k +1 n)
log b a +ε
3. if f (n) is Ω(n
), then T (n) is Θ( f (n)),
provided af (n / b) ≤ δf (n) for some δ < 1.
Divide-and-Conquer
0
identify
Case
flu
2
n
compare
and compare it n 9b
Estrictupperboundoff n
109b a
Case3
identify
E
flu
power
Case 2
T
of
and compare it n 9b
n
9a b
I strictlowerboundoff n
of flu
power
ex
T Ya t log u
n
nlegb
flu
a
I
n
Thn
log n
0
The Master Theorem suggests
1
if
the recursive part dominatestherunning time ofthealgorithm
3 the non recursive part dominates the runningtime ofthe
algorithm
Master Method, Example 1
c
⎧
T (n) = ⎨
⎩aT ( n / b) + f ( n )
•
The form:
•
The Master Theorem:
if n < d
if n ≥ d
1. if f (n) is O(n logb a −ε ), then T (n) is Θ(n logb a )
2. if f (n) is Θ(n logb a log k n), then T (n) is Θ(n logb a log k +1 n)
3. if f (n) is Ω(n logb a +ε ), then T (n) is Θ( f (n)),
provided af (n / b) ≤ δf (n) for some δ < 1.
•
Example:
T (n) = 4T (n / 2) + n
Solution: logba=2, so case 1 says T(n) is O(n2).
Divide-and-Conquer
0
1 figure out what is
i
2
compare
n
1096
it
flu
fin
n 95
n 924
z
n
flu
w
01h19
D WZ
Th
6
2
2
confused
a
E
K
6
This is case I in MasterTheorem
Therefore
4
9
D n
109
1
Master Method, Example 2
c
⎧
T (n) = ⎨
⎩aT ( n / b) + f ( n )
•
The form:
•
The Master Theorem:
if n < d
if n ≥ d
1. if f (n) is O(n logb a −ε ), then T (n) is Θ(n logb a )
2. if f (n) is Θ(n logb a log k n), then T (n) is Θ(n logb a log k +1 n)
3. if f (n) is Ω(n logb a +ε ), then T (n) is Θ( f (n)),
provided af (n / b) ≤ δf (n) for some δ < 1.
•
Example:
T (n) = 2T (n / 2) + n log n
Solution: logba=1, so case 2 says T(n) is O(n log2 n).
Divide-and-Conquer
0
Master Method, Example 3
c
⎧
T (n) = ⎨
⎩aT ( n / b) + f ( n )
•
The form:
•
The Master Theorem:
if n < d
if n ≥ d
1. if f (n) is O(n logb a −ε ), then T (n) is Θ(n logb a )
2. if f (n) is Θ(n logb a log k n), then T (n) is Θ(n logb a log k +1 n)
3. if f (n) is Ω(n logb a +ε ), then T (n) is Θ( f (n)),
provided af (n / b) ≤ δf (n) for some δ < 1.
•
Example:
T (n) = T (n / 3) + n log n
Solution: logba=0, so case 3 says T(n) is O(n log n).
Divide-and-Conquer
0
A
plogab
flu
flat
I
b
n
3
1093
no
fortheassignment
notatthisraninea
nhan
naga
n
Master Method, Example 4
c
⎧
T (n) = ⎨
⎩aT ( n / b) + f ( n )
•
The form:
•
The Master Theorem:
if n < d
if n ≥ d
1. if f (n) is O(n logb a −ε ), then T (n) is Θ(n logb a )
2. if f (n) is Θ(n logb a log k n), then T (n) is Θ(n logb a log k +1 n)
3. if f (n) is Ω(n logb a +ε ), then T (n) is Θ( f (n)),
provided af (n / b) ≤ δf (n) for some δ < 1.
•
Example:
T (n) = 8T (n / 2) + n
2
Solution: logba=3, so case 1 says T(n) is O(n3).
Divide-and-Conquer
0
Master Method, Example 5
c
⎧
T (n) = ⎨
⎩aT ( n / b) + f ( n )
•
The form:
•
The Master Theorem:
if n < d
if n ≥ d
1. if f (n) is O(n logb a −ε ), then T (n) is Θ(n logb a )
2. if f (n) is Θ(n logb a log k n), then T (n) is Θ(n logb a log k +1 n)
3. if f (n) is Ω(n logb a +ε ), then T (n) is Θ( f (n)),
provided af (n / b) ≤ δf (n) for some δ < 1.
•
Example:
T (n) = 9T (n / 3) + n
3
Solution: logba=2, so case 3 says T(n) is O(n3).
Divide-and-Conquer
0
b 3
a 9
plogba
fln
n
0939
n
it is case 3
95143
9143
Ign
Yzn
N2
Master Method, Example 6
c
⎧
T (n) = ⎨
⎩aT ( n / b) + f ( n )
•
The form:
•
The Master Theorem:
if n < d
if n ≥ d
1. if f (n) is O(n logb a −ε ), then T (n) is Θ(n logb a )
2. if f (n) is Θ(n logb a log k n), then T (n) is Θ(n logb a log k +1 n)
3. if f (n) is Ω(n logb a +ε ), then T (n) is Θ( f (n)),
provided af (n / b) ≤ δf (n) for some δ < 1.
•
Example:
T ( n ) = T ( n / 2) + 1
(binary search)
Solution: logba=0, so case 2 says T(n) is O(log n).
Divide-and-Conquer
0
Master Method, Example 7
c
⎧
T (n) = ⎨
⎩aT ( n / b) + f ( n )
•
The form:
•
The Master Theorem:
if n < d
if n ≥ d
1. if f (n) is O(n logb a −ε ), then T (n) is Θ(n logb a )
2. if f (n) is Θ(n logb a log k n), then T (n) is Θ(n logb a log k +1 n)
3. if f (n) is Ω(n logb a +ε ), then T (n) is Θ( f (n)),
provided af (n / b) ≤ δf (n) for some δ < 1.
•
Example:
T (n) = 2T (n / 2) + log n
(heap construction)
Solution: logba=1, so case 1 says T(n) is O(n).
Divide-and-Conquer
0