Sorting Text • Read Shaffer, Chapter 7 Sorting • • O(N2) sorting algorithms: – Insertion, Selection, Bubble O(N log N) sorting algorithms – HeapSort, MergeSort, QuickSort Assumptions • Array of elements • Contains only integers • Array contained completely in memory 2 O(N ) Sorting Algorithms Insertion Sort Selection Sort Bubble Sort Insertion Sort Pseudo-code Algorithm public static void insertionSort(Comparable a[]) { int j; for (int p=1; p<a.length; p++) { Comparable tmp = a[p]; for (j=p; j>0 && tmp.compareTo(a[j-1])<0; j--) a[j] = a[j-1]; a[j]=tmp; } // p } // insertionSort Insertion Sort: Step Through i for (int p=1; p<a.length; p++) { Comparable tmp = a[p]; for (j=p; j>0 && tmp.compareTo(a[j-1])<0; j--) a[j] = a[j-1]; a[j]=tmp; } // p Insertion Sort Strategy: Start with p=1. In each pass of the outer loop, determine where the pth element should be inserted in the sorted | subarray. Make room for it, if necessary, by sorted | unsorted sliding sorted elements down one. When th : 0 | 1 2 3 4 5 appropriate slot is found, insert p element. Increment p and repeat. a : | 15 | 4 | 13 2 21 10 Insertion Sort: Step Through for (int p=1; p<a.length; p++) { Comparable tmp = a[p]; for (j=p; j>0 && tmp.compareTo(a[j-1])<0; j--) a[j] = a[j-1]; a[j]=tmp; } // p i : 0 a : 15 p (insert pth element into sorted array) 1 2 3 4 5 4 13 2 21 10 Insertion Sort: Step Through for (int p=1; p<a.length; p++) { Comparable tmp = a[p]; for (j=p; j>0 && tmp.compareTo(a[j-1])<0; j--) a[j] = a[j-1]; a[j]=tmp; } // p i : 0 p 1 a : 15 4 tmp=4 2 3 4 5 13 2 21 10 Insertion Sort: Step Through for (int p=1; p<a.length; p++) { Comparable tmp = a[p]; for (j=p; j>0 && tmp.compareTo(a[j-1])<0; j--) a[j] = a[j-1]; a[j]=tmp; } // p i : a : 0 15 p 1 j 4 tmp=4 2 3 4 5 tmp < a[j-1]! 13 2 21 10 Insertion Sort: Step Through for (int p=1; p<a.length; p++) { Comparable tmp = a[p]; for (j=p; j>0 && tmp.compareTo(a[j-1])<0; j--) a[j] = a[j-1]; a[j]=tmp; } // p i : a : 0 15 p 1 j 15 tmp=4 2 3 4 5 Copy a[j-1] down! 13 2 21 10 Insertion Sort: Step Through for (int p=1; p<a.length; p++) { Comparable tmp = a[p]; for (j=p; j>0 && tmp.compareTo(a[j-1])<0; j--) a[j] = a[j-1]; a[j]=tmp; } // p i : a : 0 j 15 p 1 2 3 4 5 j==0, exit inner loop. 15 tmp=4 13 2 21 10 Insertion Sort: Step Through for (int p=1; p<a.length; p++) { Comparable tmp = a[p]; for (j=p; j>0 && tmp.compareTo(a[j-1])<0; j--) a[j] = a[j-1]; a[j]=tmp; } // p i : a : 0 j 4 p 1 2 3 4 5 Copy tmp. 15 tmp=4 13 2 21 10 Insertion Sort: Step Through for (int p=1; p<a.length; p++) { Comparable tmp = a[p]; for (j=p; j>0 && tmp.compareTo(a[j-1])<0; j--) a[j] = a[j-1]; a[j]=tmp; } // p | sorted | unsorted i : 0 1 | 2 3 4 5 | a : 4 15 |13 2 21 10 | Insertion Sort: Step Through for (int p=1; p<a.length; p++) { Comparable tmp = a[p]; for (j=p; j>0 && tmp.compareTo(a[j-1])<0; j--) a[j] = a[j-1]; a[j]=tmp; } // p i : 0 1 a : 4 15 p (insert pth element into sorted array) 2 3 4 5 13 2 21 10 Insertion Sort: Step Through for (int p=1; p<a.length; p++) { Comparable tmp = a[p]; for (j=p; j>0 && tmp.compareTo(a[j-1])<0; j--) a[j] = a[j-1]; a[j]=tmp; } // p i : a : 0 4 1 15 tmp=13 p 2 j 13 3 4 5 tmp < a[j-1]! 2 21 10 Insertion Sort: Step Through for (int p=1; p<a.length; p++) { Comparable tmp = a[p]; for (j=p; j>0 && tmp.compareTo(a[j-1])<0; j--) a[j] = a[j-1]; a[j]=tmp; } // p i : a : 0 4 1 15 tmp=13 p 2 j 15 3 4 5 Copy a[j-1] down! 2 21 10 Insertion Sort: Step Through for (int p=1; p<a.length; p++) { Comparable tmp = a[p]; for (j=p; j>0 && tmp.compareTo(a[j-1])<0; j--) a[j] = a[j-1]; a[j]=tmp; } // p i : a : 0 4 1 j 15 tmp=13 p 2 3 4 5 tmp >= a[j-1], exit loop! 15 2 21 10 Insertion Sort: Step Through for (int p=1; p<a.length; p++) { Comparable tmp = a[p]; for (j=p; j>0 && tmp.compareTo(a[j-1])<0; j--) a[j] = a[j-1]; a[j]=tmp; } // p i : a : 0 4 1 j 13 tmp=13 p 2 3 4 5 Copy tmp! 15 2 21 10 Insertion Sort: Step Through for (int p=1; p<a.length; p++) { Comparable tmp = a[p]; for (j=p; j>0 && tmp.compareTo(a[j-1])<0; j--) a[j] = a[j-1]; a[j]=tmp; } // p i : a : | sorted | unsorted 0 1 2 | 3 4 5 | 4 13 15 | 2 21 10 | Insertion Sort: Step Through for (int p=1; p<a.length; p++) { Comparable tmp = a[p]; for (j=p; j>0 && tmp.compareTo(a[j-1])<0; j--) a[j] = a[j-1]; a[j]=tmp; } // p i : 0 1 2 p 3 4 5 Continue … a : 4 13 15 2 21 10 Insertion Sort: Analysis public static void insertionSort(Comparable a[]) { int j; for (int p=1; p<a.length; p++) { Comparable tmp = a[p]; for (j=p; j>0 && tmp.compareTo(a[j-1])<0; j--) a[j] = a[j-1]; a[j]=tmp; } // p } // insertionSort Count comparisons Assume a.length == n In general, for a given p==i the number of comparisons performed in the inner loop is i (from j=p downto j>0) p: 1 2 3 4 … i … (n-1) max #comparisons: 1 2 3 4 … i … (n-1) total number of comparisons ≤ 1 + 2 + 3 + … + (n-1) = (n-1)n/2 Selection Sort Pseudo-code Algorithm public static void selectionSort(Comparable a[]) { for (int p=0; p<a.length-1; p++) { Comparable min = a[p]; int minIndex = p; for (int j=p+1; j<a.length; j++) { if min.compareTo(a[j])>0 { minIndex = j; min = a[j]; } // new min found } // j swap(a,p,minIndex); } // p } // selectionSort Selection Sort: Step Through public static void selectionSort(Comparable a[]) { for (int p=0; p<a.length-1; p++) { Comparable min = a[p]; int minIndex = p; for (int j=p+1; j<a.length; j++) { if min.compareTo(a[j])>0 { minIndex = j; min = a[j]; } // new min found } // j swap(a,p,minIndex); } // p } // selectionSort Selection Sort Strategy: In each pass of the outer | | unsorted i : | 0 1 2 a : | 15 4 13 | loop, select smallest value in unsorted subarray (i.e., from pth element on). Swap smallest element with pth element. Increment p and repeat. 3 2 4 21 5 10 Selection Sort: Analysis public static void selectionSort(Comparable a[]) { for (int p=0; p<a.length-1; p++) { Comparable min = a[p]; int minIndex = p; for (int j=p+1; j<a.length; j++) { if min.compareTo(a[j])>0 { minIndex = j; min = a[j]; } // new min found } // j swap(a,p,minIndex); } // p } // selectionSort Count comparisons. Assume a.length == n In general, for a given p the number of comparisons performed in the inner loop is (from j=p+1 to j<a.length) = (n-p-1) p: 0 1 2 … i … (n-3)(n-2) max #comparisons: (n-1)(n-2)(n-3) … (n-i-1) … 2 1 total number of comparisons ≤ (n-1)+(n-2)+ … + 2 + 1 = (n-1)n/2 Bubble Sort Pseudo-code Algorithm public static void bubbleSort(Comparable a[]) { for (int p=a.length-1; p>0; p--) { for (int j=0; j<p; j++) if (a[j].compareTo(a[j+1])>0) swap(a,j,j+1); } // p } // bubbleSort Bubble Sort: Step Through public static void bubbleSort(Comparable for (int p=a.length-1; p>0; p--) { for (int j=0; j<p; j++) if (a[j].compareTo(a[j+1])>0) swap(a,j,j+1); } // p } // bubbleSort i : a : 0 15 1 4 a[]) { Bubble Sort Strategy: Outer loop starts with bottom of array (i.e. p=a.length-1). In each pass of outer loop, “bubble” largest element down by swapping adjacent elements (i.e., a[j] and a[j+1]) from the top whenever a[j] is larger. Decrement p |and repeat. unsorted | 2 3 4 5 | 13 2 21 10 | | Bubble Sort: Analysis public static void bubbleSort(Comparable a[]) { for (int p=a.length-1; p>0; p--) { for (int j=0; j<p; j++) if (a[j].compareTo(a[j+1])>0) swap(a,j,j+1); } // p } // bubbleSort Count comparisons. Assume a.length == n In general, for a given p==i the number of comparisons performed in the inner loop is i (from j=0 to j<p) p: (n-1) (n-2) (n-3) … i … 2 1 max #comparisons: (n-1) (n-2) (n-3) … i … 2 1 total number of comparisons ≤ (n-1)+(n-2) + … + 2 + 1 = (n-1)n/2 O(N log N) Sorting Algorithms HeapSort MergeSort QuickSort HeapSort • Strategy and Back-of-the-Envelope Analysis – Insert N elements into a Heap • Each insert takes O(log N) time • Inserting N elements takes O(N log N) time – Remove N elements from a Heap • Each delete takes O(log N) time • Removing N elements takes O(N log N) time MergeSort Pseudo-code Algorithm // Merge two sorted arrays into a single array public static Comparable[] merge (Comparable a[], Comparable b[]) { int i=0; int j=0; int k=0; while (i<a.length && j<b.length) { if (a[i]<b[j]) { c[k] = a[i]; // merge a-value i++; } // a < b else c[k] = b[j]; // merge b-value j++; } // b <= a k++; } // while // continued next slide } // mergeSort MergeSort Pseudo-code Algorithm if (i==a.length) // a-values exhausted, flush b while(j<b.length) { c[k] = b[j]; j++; k++; } // flush b-values else // b-values exhausted, flush a while(i<a.length) { c[k] = a[j]; i++; k++; } // flush a-values return c; } // mergeSort // c contains merged values MergeSort: Step Through • Start with two sorted sets of values a: 3 7 8 19 b: 2 5 6 10 c: 24 25 MergeSort: Step Through • Merge a: 3 7 8 19 b: _ 5 6 10 c: 2 24 25 MergeSort: Step Through • Merge a: _ 7 8 19 b: _ 5 6 10 c: 2 3 24 25 MergeSort: Step Through • Merge a: _ 7 8 19 b: _ _ 6 10 c: 2 3 5 24 25 MergeSort: Step Through • Merge a: _ 7 8 19 b: _ _ _ 10 c: 2 3 5 6 24 25 MergeSort: Step Through • Merge a: _ _ 8 19 b: _ _ _ 10 c: 2 3 5 6 24 7 25 MergeSort: Step Through • Merge a: _ _ _ 19 b: _ _ _ 10 c: 2 3 5 6 24 7 8 25 MergeSort: Step Through • Merge a: _ _ _ 19 b: _ _ _ _ c: 2 3 5 6 24 7 8 25 10 Exit first loop MergeSort: Step Through • Merge a: _ _ _ _ b: _ _ _ _ c: 2 3 5 6 24 7 8 25 10 Flush a-values 19 MergeSort: Step Through • Merge a: _ _ _ _ b: _ _ _ _ c: 2 3 5 6 _ 7 8 25 10 Flush a-values 19 24 MergeSort: Step Through • Merge a: _ _ _ _ b: _ _ _ _ c: 2 3 5 6 _ 7 8 _ 10 Flush a-values 19 24 25 MergeSort: Step Through • Merge a: _ _ _ _ b: _ _ _ _ c: 2 3 5 6 _ 7 8 _ 10 Return c-array 19 24 25 MergeSort: Text Example • Start with array of elements a: 5 9 1 0 12 15 7 8 11 13 16 24 10 4 3 2 MergeSort: Text Example • Merge 1-element lists 2-element list a: 5 9 1 0 12 15 7 8 11 13 16 24 10 4 3 2 b: 5 9 0 1 12 15 7 8 11 13 16 24 4 10 2 3 MergeSort: Text Example • Merge 2-element lists 4-element list b: 5 9 0 1 12 15 a: 0 1 5 9 7 7 8 11 13 16 24 8 12 15 11 13 16 24 4 10 2 2 4 10 Note that we move values from b to a in this pass. 3 3 MergeSort: Text Example • Merge 4-element lists 8-element list a: 0 1 5 9 7 8 12 15 11 13 16 24 b: 0 1 5 7 8 9 12 15 2 3 2 3 4 10 4 10 11 13 16 24 Note that we move values from a to b in this pass. MergeSort: Text Example • Merge 8-element lists 16-element list b: 0 1 5 7 8 9 12 15 2 a: 0 1 2 3 4 5 9 10 11 12 23 15 16 24 7 8 3 4 10 11 13 16 24 Note that we move values from b to a in this pass. QuickSort • See Weiss, §7.7 • Key: Partitioning, Figures 7.13 – 7.14 • Example: i: a: … 20 21 22 23 24 25 26 27 28 29 30 31 32 33 … … 19 24 36 9 7 16 20 31 26 17 19 18 23 14 … quickSort( a, 23, 31); QuickSort: Partitioning i: a: | | … 20 21 22|23 24 25 26 27 28 29 30 31|32 33 … … 19 24 36| 9 7 16 20 31 26 17 19 18|23 14 … | | quickSort( a, 23, 31 ); left = 23 right = 31 Assume CUTOFF=5 QuickSort: Partitioning i: a: | | … 20 21 22|23 24 25 26 27 28 29 30 31|32 33 … … 19 24 36| 9 7 16 20 19 26 17 18 31|23 14 … | | quickSort( a, 23, 31 ); left = 23 right = 31 pivot = 18 i=23, j=30 After call to median3 QuickSort: Partitioning i: a: | | … 20 21 22|23 24 25 26 27 28 29 30 31|32 33 … … 19 24 36| 9 7 16 20 19 26 17 18 31|23 14 … | i j | quickSort( a, 23, 31 ); left = 23 right = 31 pivot = 18 After statement 6 of Figure 7.14 QuickSort: Partitioning i: a: | | … 20 21 22|23 24 25 26 27 28 29 30 31|32 33 … … 19 24 36| 9 7 16 17 19 26 20 18 31|23 14 … | i j | quickSort( a, 23, 31 ); left = 23 right = 31 pivot = 18 After statement 8 of Figure 7.14 QuickSort: Partitioning i: a: | | … 20 21 22|23 24 25 26 27 28 29 30 31|32 33 … … 19 24 36| 9 7 16 17 19 26 20 18 31|23 14 … | j i | quickSort( a, 23, 31 ); left = 23 right = 31 pivot = 18 Just before statement 10 of Figure 7.14 QuickSort: Partitioning i: a: | | … 20 21 22|23 24 25 26 27 28 29 30 31|32 33 … … 19 24 36| 9 7 16 17 18 26 20 19 31|23 14 … | | quickSort( a, 23, 31 ); left = 23 right = 31 pivot = 18 After statement 10 of Figure 7.14 QuickSort: Analysis N elements in original array log N height Each level is created by partitioning O(N) time per pass Total time to create tree = time to perform QuickSort == O(N log N) Assuming tree is balanced assume good pivots are selected