Quick Sort
• Divide and Conquer approach
• It works by selecting a pivot element from the array and partitioning
the other elements into two subarrays
• according to whether they are less than or greater than the pivot
• The subarrays are then recursively sorted using the same process until
the entire array is sorted.
Example
Commentary
1. if len(arr) <= 1: - This line checks if the input array arr has only one or
zero elements. If so, the array is already sorted, so the function simply
returns the array. This serves as the base case for the recursive calls.
2. pivot = arr[len(arr) // 2] - This line selects a pivot element from the
input array. In this implementation, the pivot is chosen as the middle
element of the array.
3. left_half = [x for x in arr if x < pivot] - This line creates a new
subarray left_half that contains all the elements of arr that are less than
the pivot. This is done using a list comprehension that iterates over all
the elements of arr and adds them to left_half if they are less than pivot.
4. right_half = [x for x in arr if x > pivot] - This line creates a new
subarray right_half that contains all the elements of arr that are greater
than the pivot. This is done using a list comprehension that iterates over
all the elements of arr and adds them to right_half if they are greater
than pivot.
5. return quick_sort(left_half) + [pivot] + quick_sort(right_half) This line recursively sorts left_half and right_half using the quick_sort
function, and concatenates them with the pivot element in the middle.
This returns a new sorted array that is the result of combining the
sorted left_half, pivot, and sorted right_half
Implementation of QS
def quick_sort(arr):
if len(arr) <= 1:
return arr
pivot = arr[len(arr) // 2]
left_half = [x for x in arr if x < pivot]
right_half = [x for x in arr if x > pivot]
return quick_sort(left_half) + [pivot] + quick_sort(right_half)
Time Complexity of Implementation
1. if len(arr) <= 1: - This line has a time complexity of O(1), because it simply performs a comparison
on the length of the input array.
2. pivot = arr[len(arr) // 2] - This line has a time complexity of O(1), because it simply selects the
middle element of the input array.
3. left_half = [x for x in arr if x < pivot] - This line has a time complexity of O(n), because it iterates
over all n elements of the input array and performs a comparison on each element.
4. right_half = [x for x in arr if x > pivot] - This line has a time complexity of O(n), because it iterates
over all n elements of the input array and performs a comparison on each element.
5. return quick_sort(left_half) + [pivot] + quick_sort(right_half) - This line has a time complexity of
T(n) = 2T(n/2) + O(n), because it recursively calls the quick_sort function on two subarrays of size
n/2, and then concatenates the sorted subarrays with the pivot element in the middle. This leads to a
time complexity of O(n log n) on average, but O(n^2) in the worst case.
• Overall, the time complexity of Quick sort is dominated by the time complexity of line 5, which is
O(n log n) on average, but O(n^2) in the worst case. However, as I mentioned earlier, Quick sort is
generally faster than Merge sort in practice because it has a smaller constant factor and uses less
memory.
Time Complexity of QS
• The time complexity of Quick sort depends on the choice of the pivot
element.
• In the worst case, where the pivot is chosen poorly and always ends
up partitioning the array into two subarrays of size n-1 and 1
• The time complexity is O(n2)
• However, on average, if the pivot is chosen randomly, the time
complexity is O(n log n) - same as Merge sort
• Quick sort is generally faster than Merge sort in practice (because it
has a smaller constant factor and uses less memory)
What should be the Ω of QS ?
Why Ω(n log n) is Best Case(Lower Bound) ?
• Consider the partition step of Quick sort, where the array is divided into two
subarrays based on the pivot element
• In the best case, the pivot always divides the array into two subarrays of roughly
equal size
• In this case, each recursive call processes half the size of the original array, and the depth of
the recursion tree is O(log n)
• Therefore, the time complexity of the algorithm is O(n log n) in the best case.
• In the worst case, the pivot is always chosen as the largest or smallest element in
the array
•
•
•
•
Resulting in one subarray of size n-1 and another subarray of size 1
This leads to a recursion tree with a depth of n
Each level of the tree processes all n elements of the array
Therefore, the time complexity of the algorithm is O(n^2) in the worst case.
What about Theta ?
Counting Algorithm
Counting sort is an efficient non-comparative sorting algorithm that works by
counting the number of occurrences of each unique element in the input array and
then using that information to calculate the position of each element in the output
array;
1.Find the maximum element in the input array, and create an auxiliary array of size
max+1 to store the count of each element.
2.Traverse the input array and update the count of each element in the auxiliary
array
3.Compute the cumulative sum of the counts in the auxiliary array, so that each
count represents the number of elements less than or equal to that element
4.Traverse the input array in reverse order and place each element in its correct
position in the output array by looking up its count in the auxiliary array and using
that count to determine its position.
5.Repeat step 4 until all elements have been placed in their correct positions.
Example
1. Find the maximum element in the input array, which is 7. Create an auxiliary array of size 8 (max+1) initialized with all zeroes:
Auxiliary array: [0, 0, 0, 0, 0, 0, 0, 0]
2. Traverse the input array and update the count of each element in the auxiliary array:
Auxiliary array: [0, 2, 0, 2, 0, 2, 0, 1]
3. Compute the cumulative sum of the counts in the auxiliary array:
Auxiliary array: [0, 2, 2, 4, 4, 6, 6, 7]
4. Traverse the input array in reverse order and place each element in its correct position in the output array by looking up its count in
the auxiliary array and using that count to determine its position:
Input array: [3, 1, 5, 3, 7, 1, 5] Output array: [0, 0, 0, 0, 0, 0, 0, 0]
Place 5 at index 6 (count of 5 is 2): Input array: [3, 1, 5, 3, 7, 1, 0] Output array: [0, 0, 0, 0, 0, 0, 5, 0]
Place 7 at index 7 (count of 7 is 1): Input array: [3, 1, 5, 3, 0, 1, 0] Output array: [0, 0, 0, 0, 0, 0, 5, 7]
Place 3 at index 4 (count of 3 is 2): Input array: [3, 1, 5, 0, 0, 1, 0] Output array: [0, 0, 0, 3, 0, 0, 5, 7]
Place 5 at index 5 (count of 5 is 2): Input array: [3, 1, 0, 0, 0, 1, 0] Output array: [0, 0, 0, 3, 0, 5, 5, 7]
Continue… Final Output Array: [1, 1, 3, 3, 5, 5, 7]
Complexity Counting Sort Algorithm
• The time complexity of counting sort is O(n+k), where n is the size of the
input array and k is the range of the input values.
• The algorithm has two main steps: counting the occurrences of each input
value and then using the counts to place the values in their correct positions
in the output array.
• The first step takes O(n) time since we need to iterate through the entire
input array once. The second step takes O(k) time since we need to iterate
through the auxiliary array once (which has k elements).
• Therefore, the overall time complexity is O(n+k). This is a linear time
complexity, which makes counting sort a very efficient algorithm for sorting
integers with a small range. However, the space complexity of counting sort
is O(k), which can be a drawback if the range of input values is very large.
Big Oh of given Example
• For the given input array [3, 1, 5, 3, 7, 1, 5], the maximum value is 7 and the size of the
input array is 7. Therefore, k = 7 and n = 7.
• Step 1: Finding the maximum value in the input array takes O(n) time, which is O(7) in
this case.
• Step 2: Creating an auxiliary array of size max+1 takes O(k) time, which is O(7) in this
case.
• Step 3: Counting the occurrences of each element in the input array takes O(n) time,
which is O(7) in this case.
• Step 4: Computing the cumulative sum of the counts in the auxiliary array takes O(k)
time, which is O(7) in this case.
• Step 5: Traversing the input array in reverse order and placing each element in its correct
position in the output array takes O(n) time, which is O(7) in this case.
• Therefore, the overall time complexity of counting sort for this input array is O(n+k) =
O(7+7) = O(14).
Radix Sort
• Radix sort is a non-comparative integer sorting algorithm that sorts
elements by processing the digits of the numbers
• It works by sorting numbers digit by digit, from the least significant digit to
the most significant digit, using a stable sorting algorithm at each digit position
• Radix sort can be used to sort numbers in either ascending or descending order
Radix Sort Algorithm
• The algorithm starts by taking an input array of n integers
• Determining the maximum number of digits in the input array
• This is done by finding the maximum element in the input array and counting the
number of digits in it
• For example, if the maximum element in the input array is 947, the number of digits
is 3.
• Next, the algorithm proceeds to sort the elements in the input array by the
least significant digit (i.e., the ones digit)
• Using a stable sorting algorithm such as counting sort or bucket sort
• For each digit position (starting from the ones digit and moving left to right)
• The input array is partitioned into 10 buckets based on the value of the digit at that
position (i.e., bucket 0 contains all elements with a 0 in the current digit position,
bucket 1 contains all elements with a 1 in the current digit position, and so on).
• The elements in each bucket are then sorted recursively by the next most significant
digit until all digits have been processed.
Example (Redix)
• Input array [170, 45, 75, 90, 802, 24, 2, 66]
• Step 1: Determine the maximum number of digits - 802,
has 3 digits
Step 2: Perform counting sort on each digit position:
• Starting with the least significant digit position (i.e., the
rightmost digit), we perform counting sort on each digit
position, one by one. We iterate through the input array and
place each element into the appropriate bucket based on the
value of the digit at the current position
• For example-the least significant digit position, we have the
buckets:
Bucket 0: []
Bucket 1: [170]
Bucket 2: [802, 2]
Bucket 3: [24]
Bucket 4: [45]
Bucket 5: [75]
Bucket 6: [66]
Bucket 7: []
Bucket 8: [90]
Bucket 9: []
Cont…
• We then concatenate the buckets in order to
obtain the partially sorted array for the current
digit position:
• Partially sorted array (least significant digit
position): [170, 802, 2, 24, 45, 75, 66, 90]
• We then repeat this process for the next digit
position, and so on.
• For the second least significant digit position
(the tens digit), we have the following buckets:
Bucket 0: [002, 024]
Bucket 1: [045]
Bucket 2: [066]
Bucket 3: [075]
Bucket 4: []
Bucket 5: []
Bucket 6: []
Bucket 7: []
Bucket 8: [170, 802, 090]
Bucket 9: []
Cont…
• We again concatenate the buckets in order to
obtain the partially sorted array for the current
digit position:
• Partially sorted array (second least significant digit
position): [002, 024, 045, 066, 075, 170, 802, 090]
• We then repeat this process for the next digit
position (the hundreds digit), and so on.
• For the most significant digit position, we have the
following buckets:
• We concatenate the buckets in order to obtain the
fully sorted array:
Sorted array: [2, 24, 45, 66, 75, 90, 170, 802]
Bucket 0: []
Bucket 1: []
Bucket 2: []
Bucket 3: []
Bucket 4: []
Bucket 5: []
Bucket 6: [002, 024, 045, 066, 075, 090]
Bucket 7: []
Bucket 8: [170]
Bucket 9: [802]
Big Oh of Redix Example
• The time complexity of radix sort is O(d*(n+k)), where n is the number of
elements in the input array, k is the range of the input (i.e., the maximum
value minus the minimum value plus one), and d is the number of digits in
the maximum value.
• In this example, the maximum value is 802, which has three digits.
Therefore, d=3. The input array has 8 elements, and the range of the input is
802-2+1=801, so k=801.
• For each digit position, we perform counting sort on the input array, which
takes O(n+k) time. We do this d times (one for each digit position), so the
total time for the counting sort steps is O(d*(n+k)).
• Therefore, the time complexity of radix sort in this example is
O(3*(8+801)) = O(2409), which is linear in the size of the input array.