Quick Sort, Shell Sort, Counting Sort,
Radix Sort AND Bucket Sort


Sorting algorithms contain interesting and important ideas for code optimization as well as algorithm design.


Last time we talked about Merge Sort
◦ Recursively calls:
MergeSort(1 st half of list)
MergeSort(2 nd half of list)
Then Merges results





Basically the partition works like this:
◦ Given an array of n values you must randomly pick an element in the array to partition by.
88 35 44 99 71 20 45 42 67 61
◦ Once you have picked this value, compare all of the rest of the elements to this value.
 If they are greater, put them to the “right” of the partition element.
 If they are less, put them to the “left” of the partition element.
35 44 20 42 45 88 61 99 67 71
Still need to be sorted
In the right spot
:D
Still need to be sorted
 So if we sort those 2 sides the whole array will be sorted.


Thus, similar to MergeSort, we can use a partition to break the sorting problem into two smaller sorting problems.
◦ QuickSort at a general level:
1) Partition the array with respect to a random element.
2) Sort the left part of the array, using Quick Sort.
3) Sort the right part of the array, using Quick Sort.


It should be clear that this algorithm will work
◦ But it may not be clear why it is faster than MergeSort.
 Like MergeSort it recursively solves 2 sub problems and requires linear additional work.
 BUT unlike MergeSort the sub problems are NOT guaranteed to be of equal size.
The reason that QuickSort is faster is that the partitioning step can actually be performed in place and very efficiently.
 This efficiency can more than make up for the lack of equal sized recursive calls.

8 3 6 9 2 4 7 5

Assume for now, that we partition based on the last element in the array, 5.

Start 2 counters: Low at array index 0 High at 2 nd to last index in the array

Advance the Low counter forward until a value greater than the pivot is encountered.

Advance the High counter backward until a value less than the pivot is encountered.
Continue to advance the counters as before.
both on the “wrong” side.

4 4 3 2 5 6 8 7 9 counter position to finish the partition.
Now as you can see our array is partitioned into a “left” and a
“right”.


◦ Wrong Choice:
 Just use the first element in the list
 If the input is random, this is acceptable.
 BUT, what if the list is already sorted or in reverse order?
 Then all the elements go into S1 or S2, consistently throughout recursive calls.
 So it would take O(n 2 ) to do nothing at all! (If presorted)
 EMBARRASSING!



A Safer Way
◦ Choose the pivot randomly
 Generally safe, since it’s unlikely the random pivot would consistently be a poor partition.
 Random number generation is generally expensive.
Median-of-Three Partitioning
◦ The best choice would be the median of the array.
 But that would be hard to calculate and slow.
 A good estimate is to pick 3 elements and use the median of those as the pivot.
 The rule of thumb: Pick the left, center, and right elements and take the median as the pivot.
8 1 4 9 6 3 5 2 7 0




Given an array of n elements, determine the kth smallest element.
◦ Clearly k must lie in between 1 and n inclusive
◦ The selection problem is different, but related to the sorting problem.

The idea is:
◦ Partition the array.
◦ There are 2 subarrays:
 One of size m, with the m smallest elements.
 The other of size n-m-1 .
 If k ≤ m, we know the k th smallest element is in the 1 st partition.
 If k == m+1, we know the k th smallest element IS the pivot.
 Otherwise, the k th smallest element is in the 2 nd partition.
35 44 20 42 45 88 61 99 67 71
Size m: if (k ≤ m) the k th smallest element is in here.
if (k==m+1)
We know the k th smalles is the pivot
Size n-m-1: if (k > m) the k th smallest element is in here.


Algorithm:
Quickselect(A, low, high, k):
1) m=Partition(A, low, high) // m is how many values are less than the partition element.
2) if k≤m, return Quickselect(low, low+m-1, k)
3) if k==m+1 return the pivot, A[low+m]
4) else return Quickselect(low+m+1, high, k-m-1)

So instead of doing 2 recursive calls, w only make one here.
◦ It turns out that on average Quickselect takes O(n) time, which is far better than it’s worst case performace of O(n 2 ) time.




Although this doesn’t have the fastest running time of all the sorting algorithms...
◦ It is fairly competitive with Quick and Merge sort for fairly decent sized data sets.

The basic idea:
◦ Instead of sorting all the elements at once, sort a small set of them
 Such as every 5 th element (you can use insertion sort)
 Then sort every 3 rd element, etc.
 Finally sort all the elements using insertion sort.

Rationale:
◦ A small insertion sort is quite efficient.
◦ A larger insertion sort can be efficient, if the elements are already
“close” to sorted order.
 By doing the smaller insertion sorts, the elements do get closer to in order before the larger insertions are done.

12 4 4 3 3 9 9 18 7 7 2 2 17 13 1 1 5 5 6 6
Sort every 5 th element:
4 17 13 18 12 6
4 1 3 5
Sort every 3 rd element:
2 6 9 12 7 18 17 13
1 2 3 4
Final a normal insertion sort:
5 6 7 9 12 13 17 18

Notice that by the time we do this last insertion sort, most elements don’t have a long way to go before being inserted.



Do we always do a 5,3,1 sort?
◦ No, in general shell sort will ALWAYS work as long as the last “pass” is a 1-sort.
What tends to work well is if each of the values in the increment sequence are in a geometric series.
◦ i.e. 1,2,4,8,16, etc.
◦ The initial passes will be really quick, O(n).
 It turns out, in practice a geometric ratio of 2.2 produces the best results.
◦ The actual average case analysis of this sort is too difficult to derive,
 But it can be shown with experimental results to indicate an average running time of O(n 1.25
).
 Dependent on the gap sequence.


In counting sort, we know that each of the values being sorted are in the range from 0 to m inclusive.
◦ We use this information to count the occurrences of each value.

Here is the algorithm for sorting an array a[0], …, a[n-1]:
1) Create an auxiliary C, indexed from 0 to m, and initialize each value to 0.
2) Run through the input array A, incrementing the number of occurrences of each value 0 through m by adding +1 to the appropriate index in C. (Thus C is a frequency array)
3) Run through the array C, a 2 nd time so that the value stored in each array slot represents the # of elements ≤ the index value in the original array A.
4) Now, run through the original input array A, and for each value in
A, use the auxiliary array C, to tell you the proper placement of that value in the sorted input, which be a new array B.
5) Copy the sorted array B into A.

Consider the input array to sort. There are values from 0 to 6
Index:
A:
0
3
1
6
2
4
3
1
4
3
5
4
6
1
First, create the frequency array C:
Index: 0 1 2
C: 0 2 0
3
2
4
3
5
0
6
1
Now we want to change C so that each array element stores the # of values less than or equal to the given index - 1:
Index:
C:
0
-1
1
1
2
1
3
3
4
6
5
6
6
7
Now that C is completed, we can start…
7
4

Original array:
Index: 0
A: 3
1
6
2
4
3
1
4
3
Frequency array:
Index: 0
C: -1
1
1
2
1
3
3
4
6
5
4
5
6
Now that C is completed,
We can start putting the elements in A in their correct spot in B:
6
1
6
7
7
4
Start with A[7], which contains 4.
Since C[ A[7]] = C[4] = 6, this means there are 6 elements ≤ 4, thus 4 should be place in index 6 of the output array, B[6]:
Index:
B:
0 1 2 3 4 5 6
4
7
Update C: We must decrement C[4] so that the next time we place a 4, it’s in a new location.
Index:
C:
0
-1
1
1
2
1
3
3
4
5
5
6
6
7




Note: Counting sort is a stable sort
◦ This means that ties in the original input stay in the same relative order after being sorted.
 For example, the last 4 in the input will be in array index 6 of the output, the second to last 4 in the input will be in array index 5 of the output, and the 4 in array index 3 of the input will be placed in index 4 of the output.
 So there was no unnecessary switching of equal values.

Another note:
◦ After getting the frequency array, why didn’t we just loop through each index in C and place each corresponding number in the array A directly?
 i.e. Since C[1]=1 originally, why not just place a 1 in A[0] and A[1] and move on…
 The reason is these numbers may be keys with associated data with them, and this approach would place the keys, but not all of the data.


◦ where n is the length of the input array and
◦ k is the length of the counting array.


The input to this sort must be non-negative integers all of a fixed length of digits.
◦ For example, numbers in the range100 – 999 or
1000 – 9999, have numbers of a fixed length of digits.

The sort works as follows:
1) Sort the values using a O(n) stable sort on the k th most significant digit.
2) Decrement k by 1.
3) Repeat step 1. (Unless k=0, they you’re done.

175
237
674
628
Unsorted
235
162
734
235
175
237
628
Sort digits
162
734
674
237
162
674
175
Sort tens
628
734
235
237
628
674
734
Sort hundreds
162
175
235


The running time of this sort should be O(nk)
◦ Since we do k stable sorts that each run O(n) time.
 Where k is the number of digits.
 (A stable sort is one where if two values being sorted, say v i and v then v j i are equal, and v i comes before v will STILL come before v j j in the unsorted list, in the sorted list.)

Depending on how many digits the numbers are, this sort can be more efficient than any
O(n log n) sort, depending on the range of values.


◦ Would it work the other way around, namely from most significant digit to least significant digit?

