CS6045: Advanced Algorithms
Sorting Algorithms
Heap Data Structure
• A heap (nearly complete binary tree) can be
stored as an array A
–
–
–
–
–
Root of tree is A[1]
Parent of
Left child of A[i] = A[2i]
Right child of A[i] = A[2i + 1]
Computing is fast with binary representation
implementation
Heaps
• Max-heap property:
• Min-heap property:
Maintain the Heap Property
• Running
Time?
• O(log n)
Build a Heap
Correctness
• Loop invariant: At start of every iteration of
for loop, each node i+1, i+2, …, n is the
root of a max-heap
• Analysis running time?
• O(n log n)
• Tighter bound? O(n)
Heapsort
Analysis
•
•
•
•
•
BUILD-MAX-HEAP: O(n)
for loop: n – 1 times
Exchange elements: O(1)
MAX-HEAPIFY: O(log n)
Time:
– O(n log n)
Min-Heap Operations
2
2
4
6
8
10
11
13
7
4
9
6
8
3
12
2
10
3
13
12
7
6
3
9
8
11
10
7
4
13
12
9
11
Insert(S, x): O(height) = O(log n)
2
12
6
8
10 13
4
11
12
5
6
9
8
10 13
4
4
11
5
6
9
8
10 13
4
12
11
5
6
9
8
5
11
10 13
Extract-min(S):
return head, replace head key with the last, float down, O(log n)
12
9
Priority Queues
• Priority Queue
– Maintains a dynamic set S of elements
– Each element has a key --- an associated value
• Applications
– job scheduling on shared computer
– Dijkstra’s finding shortest paths in graphs
– Prim’s algorithm for minimum spanning tree
Priority Queues
• Operations supported by priority queue
– Insert(S, x) - inserts element with the pointer x
– Minimum/Maximum(S) - returns element with the minimum key
– Extract-Min/Max(S) - removes and returns minimum key
– Increase/Decrease-Key(S,x,k) – increases/decreases the value of
element x’s key to the new value k
Comparison Sorting
• The only operation that may be used to gain order
information about a sequence is comparison of pairs of
elements
• Insertion sort, merge sort, quicksort, heapsort
• Lower bound for comparison sorting?
Decision Tree Model
• Abstraction of any comparison sort
• Counting only comparisons
• Abstract everything else: such as control and data
movement
• How many leaves on the decision tree?
– >= n!
• What is the length of the longest path from root to leaf?
– Depend on the algorithm
Lower Bound for Comparison Sorting
• A lower bound on the heights of decision trees
in the lower bound on the running time of any
comparison sort algorithm
• (n log n)
– n! <= l <= 2h  2h >= n!
h >= lg(n!) >= lg (n/e)n //Stirlling’s approximation
= nlg(n/e) = nlgn – nlge
= (n log n)
Non-comparison Sorts
• Counting Sort
Analysis
• O(n + k)
• How big a k is practical?
Radix Sort
Correctness
• Induction on number of passes (i in pseudocode).
• Assume digits 1, 2, ……, i -1 are sorted.
• Show that a stable sort on digit i leaves digits 1, 2,
……, i sorted:
– If 2 digits in position i are different, ordering by
position i is correct, and positions 1, 2, …… , i - 1 are
irrelevant.
– If 2 digits in position i are equal, numbers are already in
the right order (by inductive hypothesis). The stable
sort on digit i leaves them in the right order.
Analysis
• O(n + k) per iteration
• d iterations
• O(d(n + k)) total
Bucket Sort
• Idea:
–
–
–
–
Divide [0,1) into n equal-sized buckets
Distribute the n input values into the buckets
Sort each bucket
Go through buckets in order, listing elements in
each on