The Design and Analysis of Algorithms
Chapter 4:
Divide and Conquer
Master Theorem, Mergesort,
Quicksort, Binary Search, Binary Trees
Chapter 4.
Divide and Conquer Algorithms







Basic Idea
Merge Sort
Quick Sort
Binary Search
Binary Tree Algorithms
Closest Pair and Quickhull
Conclusion
2
Basic Idea

Divide instance of problem into two or
more smaller instances

Solve smaller instances recursively

Obtain solution to original (larger) instance
by combining these solutions
3
Basic Idea
a problem of size n
subproblem 1
of size n/2
subproblem 2
of size n/2
a solution to
subproblem 1
a solution to
subproblem 2
a solution to
the original problem
4
General Divide-and-Conquer
Recurrence
T(n) = aT(n/b) + f (n) where f(n)  (nd), d  0
Master Theorem: If a < bd,
If a = bd,
If a > bd,
 Examples:
T(n)  (nd)
T(n)  (nd log n)
T(n)  (nlog b a )
T(n) = T(n/2) + n  T(n)  (n )
Here a = 1, b = 2, d = 1,
a < bd
T(n) = 2T(n/2) + 1  T(n)  (n log 2 2 ) = (n )
Here a = 2, b = 2, d = 0, a > bd
5
Examples

T(n) = T(n/2) + 1  T(n)  (log(n) )
Here a = 1, b = 2, d = 0,
a = bd

T(n) = 4T(n/2) + n  T(n)  (n log 2 4 ) = (n 2 )
Here a = 4, b = 2, d = 1,
a > bd


T(n) = 4T(n/2) + n2  T(n)  (n2 log n)
Here a = 4, b = 2, d = 2,
a = bd
T(n) = 4T(n/2) + n3  T(n)  (n3)
Here a = 4, b = 2, d = 3, a < bd
6
Mergesort

Split array A[0..n-1] in two about equal halves
and make copies of each half in arrays B
and C

Sort arrays B and C recursively

Merge sorted arrays B and C into array A
7
Mergesort
8
Mergesort
9
Mergesort
8 3 2 9 7 1 5 4
8 3 2 9
8 3
8
7 1 5 4
2 9
3
2
3 8
7 1
9
2 9
5 4
7
1
5
1 7
2 3 8 9
4
4 5
1 4 5 7
1 2 3 4 5 7 8
9
10
Mergesort Code
mergesort( int a[], int left, int right)
{
if (right > left)
{
middle = left + (right - left)/2;
mergesort(a, left, middle);
mergesort(a, middle+1, right);
merge(a, left, middle, right);
}
}
11
Analysis of Mergesort

T(N) = 2T(N/2) + N

All cases have same efficiency: Θ(n log n)

Space requirement: Θ(n) (not in-place)

Can be implemented without recursion (bottom-up)
12
Features of Mergesort
All cases: Θ(n log n)
 Requires additional memory Θ(n)
 Recursive
 Not stable (does not preserve previous
sorting)
 Never used for sorting in main memory,
used for external sorting

13
Quicksort
Let S be the set of N elements
 Quicksort(S):
If N = 0 or N = 1, return
Pick an element v - pivot, in S
Partition S - {v} into two disjoint sets:
S1 = {x є S - {v}| x ≤ v}, and
S2 = {x є S - {v}| x ≥ v}
Return {quicksort(S1), followed by v,
followed by quicksort(S2)}
14
Quicksort


Select a pivot (partitioning element)
Median of three algorithm

Take the first, the last and the middle elements and sort
them (within their positions).

Choose the median of these three elements. Hide the
pivot – swap the pivot and the next to the last element.

Example:
After sorting
Hide the pivot
5891427
1895427
1892457
15
Quicksort

Rearrange the list so that all the elements in the first s
positions are smaller than or equal to the pivot and all
the elements in the remaining n-s-1 positions are larger
than or equal to the pivot. Note: the pivot does not
participate in rearranging the elements.

Exchange the pivot with the first element in the second
subarray — the pivot is now in its final position

Sort the two subarrays recursively
16
left – the smallest valid index
right – the largest valid index
if( left + 10 <= right)
{
int i = left, j = right - 1;
for ( ; ; )
{
while (a[++i] < pivot) {}
while (pivot < a[--j] ) {}
if (i < j) swap (a[i],a[j]);
else break;
}
swap (a[i], a[right-1]);
quicksort ( a, left, i-1);
quicksort (a, i+1, right);
}
else
insertionsort (a, left, right);
17
Analysis of Quicksort

Best case: split in the middle — Θ(n log n)
T(N) = 2T(N/2) + N

Worst case: sorted array
T(N) = T(N-1) + N

Average case: random arrays — Θ(n log n)
— Θ(n2)
Quicksort is considered the method of choice for internal
sorting of large files (n ≥ 10000)
18
Features of Quicksort






Best and average case: Θ(n log n) ,
Worst case: Θ(n2), it happens when the pivot is
the smallest of the largest element
In-place sorting, additional memory only for
swapping
Recursive
Not stable
Never used for life-critical and missioncritical applications, unless you assume the
worst-case response time
19
Binary Search

Very efficient algorithm for searching in sorted array:
K
vs
A[0] . . . A[m] . . . A[n-1]

If K = A[m], stop (successful search);

otherwise, continue searching by the same
method
in A[0..m-1]
if K < A[m],
and in A[m+1..n-1] if K > A[m]
20
Analysis of Binary Search

Time efficiency T(N) = T(N/2) + 1
 Worst
case:
 Best case:
O(log(n))
O(1)
Optimal for searching a sorted array
 Limitations: must be a sorted array
(not linked list)

21
Binary Tree Algorithms

Binary tree is a divide-and-conquer ready structure
Ex. 1: Classic traversals
(preorder, inorder, postorder)

Algorithm Inorder(T)
if T  
Inorder(Tleft)
print(root of T)
Inorder(Tright)

Efficiency: Θ(n)
22
Binary Tree Algorithms
Ex. 2: Computing the height of a binary tree
TL
TR
h(T) = max{ h(TL), h(TR) } + 1 if T   and h() = -1
Efficiency: Θ(n)
23