Sorting
Text
•
Read Shaffer, Chapter 7
Sorting
•
•
O(N2) sorting algorithms:
– Insertion, Selection, Bubble
O(N log N) sorting algorithms
– HeapSort, MergeSort, QuickSort
Assumptions
• Array of elements
• Contains only integers
• Array contained completely in memory
2
O(N )
Sorting Algorithms
Insertion Sort
Selection Sort
Bubble Sort
Insertion Sort
Pseudo-code Algorithm
public static void insertionSort(Comparable a[]) {
int j;
for (int p=1; p<a.length; p++) {
Comparable tmp = a[p];
for (j=p; j>0 && tmp.compareTo(a[j-1])<0; j--)
a[j] = a[j-1];
a[j]=tmp;
} // p
} // insertionSort
Insertion Sort: Step Through

i
for (int p=1; p<a.length; p++) {
Comparable tmp = a[p];
for (j=p; j>0 && tmp.compareTo(a[j-1])<0; j--)
a[j] = a[j-1];
a[j]=tmp;
} // p
Insertion Sort Strategy: Start with p=1. In each
pass of the outer loop, determine where the pth
element should be inserted in the sorted
|
subarray. Make room for it, if necessary, by
sorted | unsorted 
sliding sorted elements down one. When
th
:
0 | 1
2
3
4
5 appropriate slot is found, insert p element.
Increment p and repeat.
a :
|
15 | 4
|
13
2
21
10
Insertion Sort: Step Through
for (int p=1; p<a.length; p++) {
Comparable tmp = a[p];
for (j=p; j>0 && tmp.compareTo(a[j-1])<0; j--)
a[j] = a[j-1];
a[j]=tmp;
} // p
i :
0
a :
15
p (insert pth element into sorted array)

1
2
3
4
5
4
13
2
21
10
Insertion Sort: Step Through
for (int p=1; p<a.length; p++) {
Comparable tmp = a[p];
for (j=p; j>0 && tmp.compareTo(a[j-1])<0; j--)
a[j] = a[j-1];
a[j]=tmp;
} // p
i :
0
p

1
a :
15
4
tmp=4
2
3
4
5
13
2
21
10
Insertion Sort: Step Through
for (int p=1; p<a.length; p++) {
Comparable tmp = a[p];
for (j=p; j>0 && tmp.compareTo(a[j-1])<0; j--)
a[j] = a[j-1];
a[j]=tmp;
} // p
i :
a :
0
15
p

1
j
4
tmp=4
2
3
4
5
tmp < a[j-1]!
13
2
21
10
Insertion Sort: Step Through
for (int p=1; p<a.length; p++) {
Comparable tmp = a[p];
for (j=p; j>0 && tmp.compareTo(a[j-1])<0; j--)
a[j] = a[j-1];
a[j]=tmp;
} // p
i :
a :
0
15
p

1
j
15
tmp=4
2
3
4
5
Copy a[j-1] down!
13
2
21
10
Insertion Sort: Step Through
for (int p=1; p<a.length; p++) {
Comparable tmp = a[p];
for (j=p; j>0 && tmp.compareTo(a[j-1])<0; j--)
a[j] = a[j-1];
a[j]=tmp;
} // p
i :
a :
0
j
15
p

1
2
3
4
5
j==0, exit inner loop.
15
tmp=4
13
2
21
10
Insertion Sort: Step Through
for (int p=1; p<a.length; p++) {
Comparable tmp = a[p];
for (j=p; j>0 && tmp.compareTo(a[j-1])<0; j--)
a[j] = a[j-1];
a[j]=tmp;
} // p
i :
a :
0
j
4
p

1
2
3
4
5
Copy tmp.
15
tmp=4
13
2
21
10
Insertion Sort: Step Through
for (int p=1; p<a.length; p++) {
Comparable tmp = a[p];
for (j=p; j>0 && tmp.compareTo(a[j-1])<0; j--)
a[j] = a[j-1];
a[j]=tmp;
} // p
|
 sorted | unsorted 
i :
0
1 | 2
3
4
5
|
a :
4 15 |13
2 21 10
|
Insertion Sort: Step Through
for (int p=1; p<a.length; p++) {
Comparable tmp = a[p];
for (j=p; j>0 && tmp.compareTo(a[j-1])<0; j--)
a[j] = a[j-1];
a[j]=tmp;
} // p
i :
0
1
a :
4
15
p (insert pth element into sorted array)

2
3
4
5
13
2
21
10
Insertion Sort: Step Through
for (int p=1; p<a.length; p++) {
Comparable tmp = a[p];
for (j=p; j>0 && tmp.compareTo(a[j-1])<0; j--)
a[j] = a[j-1];
a[j]=tmp;
} // p
i :
a :
0
4
1
15
tmp=13
p

2
j
13
3
4
5
tmp < a[j-1]!
2
21
10
Insertion Sort: Step Through
for (int p=1; p<a.length; p++) {
Comparable tmp = a[p];
for (j=p; j>0 && tmp.compareTo(a[j-1])<0; j--)
a[j] = a[j-1];
a[j]=tmp;
} // p
i :
a :
0
4
1
15
tmp=13
p

2
j
15
3
4
5
Copy a[j-1] down!
2
21
10
Insertion Sort: Step Through
for (int p=1; p<a.length; p++) {
Comparable tmp = a[p];
for (j=p; j>0 && tmp.compareTo(a[j-1])<0; j--)
a[j] = a[j-1];
a[j]=tmp;
} // p
i :
a :
0
4
1
j
15
tmp=13
p

2
3
4
5
tmp >= a[j-1], exit loop!
15
2
21
10
Insertion Sort: Step Through
for (int p=1; p<a.length; p++) {
Comparable tmp = a[p];
for (j=p; j>0 && tmp.compareTo(a[j-1])<0; j--)
a[j] = a[j-1];
a[j]=tmp;
} // p
i :
a :
0
4
1
j
13
tmp=13
p

2
3
4
5
Copy tmp!
15
2
21
10
Insertion Sort: Step Through
for (int p=1; p<a.length; p++) {
Comparable tmp = a[p];
for (j=p; j>0 && tmp.compareTo(a[j-1])<0; j--)
a[j] = a[j-1];
a[j]=tmp;
} // p
i :
a :
|
 sorted | unsorted 
0
1
2 | 3
4
5
|
4 13 15 | 2 21 10
|
Insertion Sort: Step Through
for (int p=1; p<a.length; p++) {
Comparable tmp = a[p];
for (j=p; j>0 && tmp.compareTo(a[j-1])<0; j--)
a[j] = a[j-1];
a[j]=tmp;
} // p
i :
0
1
2
p

3
4
5
Continue …
a :
4
13
15
2
21
10
Insertion Sort: Analysis
public static void insertionSort(Comparable a[]) {
int j;
for (int p=1; p<a.length; p++) {
Comparable tmp = a[p];
for (j=p; j>0 && tmp.compareTo(a[j-1])<0; j--)
a[j] = a[j-1];
a[j]=tmp;
} // p
} // insertionSort
Count comparisons
Assume a.length == n
In general, for a given p==i the number of comparisons performed in the inner
loop is i (from j=p downto j>0)
p:
1 2 3 4
… i … (n-1)
max #comparisons:
1 2 3 4
… i … (n-1)
 total number of comparisons ≤ 1 + 2 + 3 + … + (n-1) = (n-1)n/2
Selection Sort
Pseudo-code Algorithm
public static void selectionSort(Comparable a[]) {
for (int p=0; p<a.length-1; p++) {
Comparable min = a[p];
int minIndex = p;
for (int j=p+1; j<a.length; j++) {
if min.compareTo(a[j])>0 {
minIndex = j;
min = a[j];
} // new min found
} // j
swap(a,p,minIndex);
} // p
} // selectionSort
Selection Sort: Step Through
public static void selectionSort(Comparable a[]) {
for (int p=0; p<a.length-1; p++) {
Comparable min = a[p];
int minIndex = p;
for (int j=p+1; j<a.length; j++) {
if min.compareTo(a[j])>0 {
minIndex = j;
min = a[j];
} // new min found
} // j
swap(a,p,minIndex);
} // p
} // selectionSort
Selection Sort Strategy: In each pass of the outer
|
| unsorted 
i : | 0
1
2
a : | 15
4 13
|
loop, select smallest value in unsorted subarray
(i.e., from pth element on). Swap smallest element
with pth element. Increment p and repeat.
3
2
4
21
5
10
Selection Sort: Analysis
public static void selectionSort(Comparable a[]) {
for (int p=0; p<a.length-1; p++) {
Comparable min = a[p];
int minIndex = p;
for (int j=p+1; j<a.length; j++) {
if min.compareTo(a[j])>0 {
minIndex = j;
min = a[j];
} // new min found
} // j
swap(a,p,minIndex);
} // p
} // selectionSort
Count comparisons. Assume a.length == n
In general, for a given p the number of comparisons performed in the inner loop
is (from j=p+1 to j<a.length) = (n-p-1)
p:
0
1
2
…
i
… (n-3)(n-2)
max #comparisons:
(n-1)(n-2)(n-3) … (n-i-1) …
2
1
 total number of comparisons ≤ (n-1)+(n-2)+ … + 2 + 1 = (n-1)n/2
Bubble Sort
Pseudo-code Algorithm
public static void bubbleSort(Comparable a[]) {
for (int p=a.length-1; p>0; p--) {
for (int j=0; j<p; j++)
if (a[j].compareTo(a[j+1])>0)
swap(a,j,j+1);
} // p
} // bubbleSort
Bubble Sort: Step Through
public static void bubbleSort(Comparable
for (int p=a.length-1; p>0; p--) {
for (int j=0; j<p; j++)
if (a[j].compareTo(a[j+1])>0)
swap(a,j,j+1);
} // p
} // bubbleSort
i :
a :
0
15
1
4
a[]) {
Bubble Sort Strategy: Outer loop starts with
bottom of array (i.e. p=a.length-1). In each pass
of outer loop, “bubble” largest element down by
swapping adjacent elements (i.e., a[j] and a[j+1])
from the top whenever a[j] is larger. Decrement p
|and repeat.
 unsorted |
2
3
4
5 |
13
2 21 10 |
|
Bubble Sort: Analysis
public static void bubbleSort(Comparable a[]) {
for (int p=a.length-1; p>0; p--) {
for (int j=0; j<p; j++)
if (a[j].compareTo(a[j+1])>0)
swap(a,j,j+1);
} // p
} // bubbleSort
Count comparisons. Assume a.length == n
In general, for a given p==i the number of comparisons performed in the inner
loop is i (from j=0 to j<p)
p:
(n-1) (n-2) (n-3) …
i
… 2
1
max #comparisons:
(n-1) (n-2) (n-3) …
i
… 2
1
 total number of comparisons ≤ (n-1)+(n-2) + … + 2 + 1 = (n-1)n/2
O(N log N) Sorting Algorithms
HeapSort
MergeSort
QuickSort
HeapSort
• Strategy and Back-of-the-Envelope Analysis
– Insert N elements into a Heap
• Each insert takes O(log N) time
• Inserting N elements takes O(N log N) time
– Remove N elements from a Heap
• Each delete takes O(log N) time
• Removing N elements takes O(N log N) time
MergeSort
Pseudo-code Algorithm
// Merge two sorted arrays into a single array
public static Comparable[] merge (Comparable a[], Comparable b[]) {
int i=0; int j=0; int k=0;
while (i<a.length && j<b.length) {
if (a[i]<b[j]) {
c[k] = a[i];
// merge a-value
i++;
} // a < b
else
c[k] = b[j];
// merge b-value
j++;
} // b <= a
k++;
} // while
// continued next slide
} // mergeSort
MergeSort
Pseudo-code Algorithm
if (i==a.length) // a-values exhausted, flush b
while(j<b.length) {
c[k] = b[j];
j++;
k++;
} // flush b-values
else
// b-values exhausted, flush a
while(i<a.length) {
c[k] = a[j];
i++;
k++;
} // flush a-values
return c;
} // mergeSort
// c contains merged values
MergeSort: Step Through
• Start with two sorted sets of values
a:
3
7
8
19
b:
2
5
6
10
c:
24
25
MergeSort: Step Through
• Merge
a:
3
7
8
19
b:
_
5
6
10
c:
2
24
25
MergeSort: Step Through
• Merge
a:
_
7
8
19
b:
_
5
6
10
c:
2
3
24
25
MergeSort: Step Through
• Merge
a:
_
7
8
19
b:
_
_
6
10
c:
2
3
5
24
25
MergeSort: Step Through
• Merge
a:
_
7
8
19
b:
_
_
_
10
c:
2
3
5
6
24
25
MergeSort: Step Through
• Merge
a:
_
_
8
19
b:
_
_
_
10
c:
2
3
5
6
24
7
25
MergeSort: Step Through
• Merge
a:
_
_
_
19
b:
_
_
_
10
c:
2
3
5
6
24
7
8
25
MergeSort: Step Through
• Merge
a:
_
_
_
19
b:
_
_
_
_
c:
2
3
5
6
24
7
8
25
10
Exit first
loop
MergeSort: Step Through
• Merge
a:
_
_
_
_
b:
_
_
_
_
c:
2
3
5
6
24
7
8
25
10
Flush a-values
19
MergeSort: Step Through
• Merge
a:
_
_
_
_
b:
_
_
_
_
c:
2
3
5
6
_
7
8
25
10
Flush a-values
19
24
MergeSort: Step Through
• Merge
a:
_
_
_
_
b:
_
_
_
_
c:
2
3
5
6
_
7
8
_
10
Flush a-values
19
24
25
MergeSort: Step Through
• Merge
a:
_
_
_
_
b:
_
_
_
_
c:
2
3
5
6
_
7
8
_
10
Return c-array
19
24
25
MergeSort: Text Example
• Start with array of elements
a:
5
9
1
0 12 15
7
8 11 13 16 24 10
4
3
2
MergeSort: Text Example
• Merge 1-element lists  2-element list
a:
5
9
1
0 12 15
7
8 11 13 16 24 10
4
3
2
 b:
5
9
0
1 12 15
7
8 11 13 16 24
4 10
2
3
MergeSort: Text Example
• Merge 2-element lists  4-element list
b:
5
9
0
1 12 15
 a:
0
1
5
9
7
7
8 11 13 16 24
8 12 15 11 13 16 24
4 10
2
2
4 10
Note that we move values from b to a in this pass.
3
3
MergeSort: Text Example
• Merge 4-element lists  8-element list
a:
0
1
5
9
7
8 12 15 11 13 16 24
 b:
0
1
5
7
8
9 12 15
2
3
2
3
4 10
4 10 11 13 16 24
Note that we move values from a to b in this pass.
MergeSort: Text Example
• Merge 8-element lists  16-element list
b:
0
1
5
7
8
9 12 15
2
 a:
0
1
2
3
4
5
9 10 11 12 23 15 16 24
7
8
3
4 10 11 13 16 24
Note that we move values from b to a in this pass.
QuickSort
• See Weiss, §7.7
• Key: Partitioning, Figures 7.13 – 7.14
• Example:
i:
a:
… 20 21 22 23 24 25 26 27 28 29 30 31 32 33 …
… 19 24 36 9 7 16 20 31 26 17 19 18 23 14 …
 quickSort( a, 23, 31);
QuickSort: Partitioning
i:
a:
|
|
… 20 21 22|23 24 25 26 27 28 29 30 31|32 33 …
… 19 24 36| 9 7 16 20 31 26 17 19 18|23 14 …
|
|
 quickSort( a, 23, 31 );
left = 23
right = 31
Assume CUTOFF=5
QuickSort: Partitioning
i:
a:
|
|
… 20 21 22|23 24 25 26 27 28 29 30 31|32 33 …
… 19 24 36| 9 7 16 20 19 26 17 18 31|23 14 …
|
|
 quickSort( a, 23, 31 );
left = 23
right = 31
pivot = 18
i=23, j=30
After call to median3
QuickSort: Partitioning
i:
a:
|
|
… 20 21 22|23 24 25 26 27 28 29 30 31|32 33 …
… 19 24 36| 9 7 16 20 19 26 17 18 31|23 14 …
|
i
j
|
 quickSort( a, 23, 31 );
left = 23
right = 31
pivot = 18
After statement 6 of
Figure 7.14
QuickSort: Partitioning
i:
a:
|
|
… 20 21 22|23 24 25 26 27 28 29 30 31|32 33 …
… 19 24 36| 9 7 16 17 19 26 20 18 31|23 14 …
|
i
j
|
 quickSort( a, 23, 31 );
left = 23
right = 31
pivot = 18
After statement 8 of
Figure 7.14
QuickSort: Partitioning
i:
a:
|
|
… 20 21 22|23 24 25 26 27 28 29 30 31|32 33 …
… 19 24 36| 9 7 16 17 19 26 20 18 31|23 14 …
|
j i
|
 quickSort( a, 23, 31 );
left = 23
right = 31
pivot = 18
Just before statement 10
of Figure 7.14
QuickSort: Partitioning
i:
a:
|
|
… 20 21 22|23 24 25 26 27 28 29 30 31|32 33 …
… 19 24 36| 9 7 16 17 18 26 20 19 31|23 14 …
|
|
 quickSort( a, 23, 31 );
left = 23
right = 31
pivot = 18
After statement 10 of
Figure 7.14
QuickSort: Analysis
N elements in original array  log N height
Each level is created by partitioning  O(N) time per pass
Total time to create tree = time to perform QuickSort == O(N log N)
Assuming tree is balanced  assume good pivots are selected