Sell
100
IBM
$122
Sell
300
IBM
$120
Buy
500
IBM
$119
Buy
400
IBM
$118
Chapter 8: Priority Queues and Heaps
Nancy Amato
Parasol Lab, Dept. CSE, Texas A&M University
Acknowledgement: These slides are adapted from slides provided with Data Structures and
Algorithms in C++, Goodrich, Tamassia and Mount (Wiley 2004)
http://parasol.tamu.edu
Outline and Reading
• PriorityQueue ADT (§8.1)
• Total order relation (§8.1.1)
• Comparator ADT (§8.1.2)
• Sorting with a Priority Queue (§8.1.5)
• Implementing a PQ with a list (§8.2)
• Selection-sort and Insertion Sort (§8.2.2)
• Heaps (§8.3)
• Complete Binary Trees (§8.3.2)
• Implementing a PQ with a heap (§8.3.3)
• Heapsort (§8.3.5)
Priority Queues
Trees
2 4/8/2015 5:25
AM
Priority Queue ADT
• Additional methods
• A priority queue stores a
collection of items
• An item is a pair
(key, element)
• Main methods of the Priority
Queue ADT
• minKey()
returns, but does not remove,
the smallest key of an item
• minElement()
returns, but does not remove,
the element of an item with
smallest key
• size(), isEmpty()
• insertItem(k, o)
inserts an item with key k and • Applications:
element o
• Standby flyers
• removeMin()
• Auctions
removes the item with the
• Stock market
smallest key
Priority Queues
Trees
3 4/8/2015 5:25
AM
Comparator ADT
• A comparator encapsulates the action of comparing
two objects according to a given total order relation
• A generic priority queue uses a comparator as a
template argument, to define the comparison
function (<,=,>)
• The comparator is external to the keys being
compared. Thus, the same objects can be sorted in
different ways by using different comparators.
• When the priority queue needs to compare two
keys, it uses its comparator
Priority Queues
Trees
4 4/8/2015 5:25
AM
Using Comparators in C++
• A comparator class overloads
the “()” operator with a
comparison function.
• Example: Compare two points in
the plane lexicographically.
class LexCompare {
public:
int operator()(Point a, Point b) {
if
(a.x < b.x) return –1
else if (a.x > b.x) return +1
else if (a.y < b.y) return –1
else if (a.y > b.y) return +1
else return 0;
}
};
Priority Queues
Trees
5 4/8/2015 5:25
AM
Total Order Relation
• Keys in a priority
• Mathematical concept of
queue can be
total order relation 
arbitrary objects on
• Reflexive property:
which an order is
xx
defined
• Antisymmetric property:
• Two distinct items in
xyyxx=y
a priority queue can
• Transitive property:
have the same key
xyyzxz
Priority Queues
Trees
6 4/8/2015 5:25
AM
PQ-Sort:
Sorting with a Priority Queue
• We can use a priority queue
to sort a set of comparable
elements
•
•
Insert the elements one
by one with a series of
insertItem(e, e) operations
Remove the elements in
sorted order with a series
of removeMin() operations
• Running time depends on
the PQ implementation
Algorithm PQ-Sort(S, C)
Input sequence S, comparator
C for the elements of S
Output sequence S sorted in
increasing order according
to C
P  priority queue with
comparator C
while !S.isEmpty ()
e  S.remove (S.first())
P.insertItem(e, e)
while !P.isEmpty()
e  P.minElement()
P.removeMin()
S.insertLast(e)
Priority Queues
Trees
7 4/8/2015 5:25
AM
List-based Priority Queue
• Unsorted list implementation
• sorted list implementation
• Store the items of the priority queue in
a list-based sequence, in arbitrary
order
4
5
2
3
1
• Store the items of the priority
queue in a sequence, sorted by
key
1
2
3
4
5
• Performance:
• Performance:
• insertItem takes O(1) time since
we can insert the item at the
beginning or end of the sequence
• removeMin, minKey and
minElement take O(n) time since
we have to traverse the entire
sequence to find the smallest key
• insertItem takes O(n) time
since we have to find the
place where to insert the item
• removeMin, minKey and
minElement take O(1) time
since the smallest key is at
the beginning of the
sequence
Priority Queues
Trees
8 4/8/2015 5:25
AM
Selection-Sort
• Selection-sort is the variation of PQ-sort where the
priority queue is implemented with an unsorted
sequence
4
5
2
3
1
• Running time of Selection-sort:
• Inserting the elements into the priority queue with n
insertItem operations takes O(n) time
• Removing the elements in sorted order from the priority
queue with n removeMin operations takes time
proportional to
1 + 2 + …+ n
• Selection-sort runs in O(n2) time
Priority Queues
Trees
9 4/8/2015 5:25
AM
Exercise: Selection-Sort
• Selection-sort is the variation of PQ-sort where the
priority queue is implemented with an unsorted
sequence (first n insertItems, then n removeMins)
4
5
2
3
1
• Illustrate the performance of selection-sort on the
following input sequence:
• (22, 15, 36, 44, 10, 3, 9, 13, 29, 25)
Priority Queues
Trees
10 4/8/2015 5:25
AM
Insertion-Sort
•
Insertion-sort is the variation of PQ-sort where the priority
queue is implemented with a sorted sequence
1
2
3
4
Running time of Insertion-sort:
•
•
5
Inserting the elements into the priority queue with n insertItem
operations takes time proportional to
1 + 2 + …+ n
•
•
Removing the elements in sorted order from the priority queue
with a series of n removeMin operations takes O(n) time
Insertion-sort runs in O(n2) time
Priority Queues
Trees
11 4/8/2015 5:25
AM
Exercise: Insertion-Sort
• Insertion-sort is the variation of PQ-sort where the
priority queue is implemented with a sorted
sequence (first n insertItems, then n removeMins)
4
5
2
3
1
• Illustrate the performance of insertion-sort on the
following input sequence:
• (22, 15, 36, 44, 10, 3, 9, 13, 29, 25)
Priority Queues
Trees
12 4/8/2015 5:25
AM
In-place Insertion-sort
• Instead of using an
external data structure, we
can implement selectionsort and insertion-sort inplace (only O(1) extra
storage)
• A portion of the input
sequence itself serves as
the priority queue
• For in-place insertion-sort
5
4
2
3
1
5
4
2
3
1
4
5
2
3
1
2
4
5
3
1
2
3
4
5
1
2
3
4
5
2
3
• We keep sorted the initial
portion of the sequence
1
• We can use
swapElements instead of
modifying the sequence
1
Priority Queues
Trees
4
5
13 4/8/2015 5:25
AM
Heaps and Priority Queues
2
5
9
6
7
Priority Queues
Trees
14 4/8/2015 5:25
AM
What is a heap?
• A heap is a binary tree
storing keys at its internal
nodes and satisfying the
following properties:
• Heap-Order: for every internal
node v other than the root,
key(v)  key(parent(v))
• Complete Binary Tree: let h
be the height of the heap
• for i = 0, … , h - 1, there are 2i
nodes of depth i
• at depth h - 1, the internal
nodes are to the left of the
external nodes
• The last node of a heap
is the rightmost internal
node of depth h - 1
2
5
9
Priority Queues
Trees
6
7
last node
15 4/8/2015 5:25
AM
Height of a Heap
• Theorem: A heap storing n keys has height O(log n)
• Proof: (we apply the complete binary tree property)
• Let h be the height of a heap storing n keys
• Since there are 2i keys at depth i = 0, … , h - 2 and at least one key at
depth h - 1, we have n  1 + 2 + 4 + … + 2h-2 + 1
• Thus, n  2h-1 , i.e., h  log n + 1
depth keys
0
1
1
2
h-2
2h-2
h-1
1
Priority Queues
Trees
16 4/8/2015 5:25
AM
Exercise: Heaps
• Where may an item with the largest key be stored in
a heap?
• True or False:
• A pre-order traversal of a heap will list out its keys in sorted
order. Prove it is true or provide a counter example.
• Let H be a heap with 7 distinct elements (1,2,3,4,5,6,
and 7). Is it possible that a preorder traversal visits
the elements in sorted order? What about an inorder
traversal or a postorder traversal? In each case,
either show such a heap or prove that none exists.
Priority Queues
Trees
17 4/8/2015 5:25
AM
Heaps and Priority Queues
•
•
•
•
We can use a heap to implement a priority queue
We store a (key, element) item at each internal node
We keep track of the position of the last node
For simplicity, we show only the keys in the pictures
(2, Sue)
(5, Pat)
(9, Jeff)
(6, Mark)
(7, Anna)
Priority Queues
Trees
18 4/8/2015 5:25
AM
Insertion into a Heap
• Method insertItem of the
priority queue ADT
corresponds to the insertion of
a key k to the heap
• The insertion algorithm
consists of three steps
• Find the insertion node z (the
new last node)
• Store k at z and expand z into
an internal node
• Restore the heap-order
property (discussed next)
2
5
9
6
z
7
insertion node
2
5
9
Priority Queues
Trees
6
7
z
1
19 4/8/2015 5:25
AM
Upheap
• After the insertion of a new key k, the heap-order property may be
violated
• Algorithm upheap restores the heap-order property by swapping k
along an upward path from the insertion node
• Upheap terminates when the key k reaches the root or a node
whose parent has a key smaller than or equal to k
• Since a heap has height O(log n), upheap runs in O(log n) time
2
1
5
9
1
7
z
6
5
9
Priority Queues
Trees
2
7
z
6
20 4/8/2015 5:25
AM
Removal from a Heap
• Method removeMin of
the priority queue ADT
corresponds to the
removal of the root key
from the heap
• The removal algorithm
consists of three steps
• Replace the root key with
the key of the last node w
• Compress w and its
children into a leaf
• Restore the heap-order
property (discussed next)
2
5
6
w
9
7
last node
7
5
w
6
9
Priority Queues
Trees
21 4/8/2015 5:25
AM
Downheap
• After replacing the root key with the key k of the last node, the
heap-order property may be violated
• Algorithm downheap restores the heap-order property by swapping
key k along a downward path from the root
• Upheap terminates when key k reaches a leaf or a node whose
children have keys greater than or equal to k
• Since a heap has height O(log n), downheap runs in O(log n) time
7
5
w
5
6
9
7
w
6
9
Priority Queues
Trees
22 4/8/2015 5:25
AM
Updating the Last Node
• The insertion node can be found by traversing a path of O(log n)
nodes
• Go up until a left child or the root is reached
• If a left child is reached, go to the right child
• Go down left until a leaf is reached
• Similar algorithm for updating the last node after a removal
Priority Queues
Trees
23 4/8/2015 5:25
AM
Heap-Sort
• Consider a priority
queue with n items
implemented by means
of a heap
• the space used is O(n)
• methods insertItem and
removeMin take O(log n)
time
• methods size, isEmpty,
minKey, and minElement
take time O(1) time
• Using a heap-based
priority queue, we can
sort a sequence of n
elements in O(n log n)
time
• The resulting algorithm
is called heap-sort
• Heap-sort is much faster
than quadratic sorting
algorithms, such as
insertion-sort and
selection-sort
Priority Queues
Trees
24 4/8/2015 5:25
AM
Exercise: Heap-Sort
• Heap-sort is the variation of PQ-sort where the
priority queue is implemented with a heap (first n
insertItems, then n removeMins)
4
5
2
3
1
• Illustrate the performance of heap-sort on the
following input sequence:
• (22, 15, 36, 44, 10, 3, 9, 13, 29, 25)
Priority Queues
Trees
25 4/8/2015 5:25
AM
Vector-based Heap Implementation
• We can represent a heap with n
keys by means of a vector of
length n + 1
• For the node at rank i
• the left child is at rank 2i
• the right child is at rank 2i + 1
2
5
6
9
• Links between nodes are not
explicitly stored
• The leaves are not represented
• The cell of at rank 0 is not used
• Operation insertItem corresponds
0
to inserting at rank n + 1
• Operation removeMin corresponds
to removing at rank n
• Yields in-place heap-sort
Priority Queues
Trees
7
2
5
6
9
7
1
2
3
4
5
26 4/8/2015 5:25
AM
Priority Queue Sort Summary
• PQ-Sort consists of n insertions followed by n removeMin ops
Vectors
Insert
Remove PQ-Sort
Min
total
Insertion Sort (ordered
sequence)
O(n)
O(1)
O(n2)
Selection Sort
(unordered sequence)
O(1)
O(n)
O(n2)
Heap Sort (binary
heap, vector-based
implementation)
O(logn)
O(logn)
O(nlogn)
4/8/2015 5:25 AM
27
Merging Two Heaps
• We are given two two
heaps and a key k
• We create a new heap
with the root node
storing k and with the
two heaps as subtrees
• We perform downheap
to restore the heaporder property
3
8
2
5
4
6
7
3
8
2
5
4
6
2
3
8
Priority Queues
Trees
4
5
7
6
28 4/8/2015 5:25
AM
Bottom-up Heap Construction
• We can construct a heap
storing n given keys in
using a bottom-up
construction with log n
phases
• In phase i, pairs of heaps
with 2i -1 keys are
merged into heaps with
2i+1-1 keys
2i -1
2i -1
2i+1-1
Priority Queues
Trees
29 4/8/2015 5:25
AM
Example
16
15
4
25
16
12
6
5
15
4
7
23
11
12
6
Priority Queues
Trees
20
27
7
23
20
30 4/8/2015 5:25
AM
Example (cont’d)
25
16
5
15
4
15
16
11
12
6
4
25
5
27
9
23
6
12
11
Priority Queues
Trees
20
23
9
27
20
31 4/8/2015 5:25
AM
Example (cont’d)
7
8
15
16
4
25
5
6
12
11
23
9
4
5
25
20
6
15
16
27
7
8
12
11
Priority Queues
Trees
23
9
27
20
32 4/8/2015 5:25
AM
Example (end)
10
4
6
15
16
5
25
7
8
12
11
23
9
27
20
4
5
6
15
16
7
25
10
8
12
11
Priority Queues
Trees
23
9
27
20
33 4/8/2015 5:25
AM
Analysis
• We visualize the worst-case time of a downheap with a proxy path
that goes first right and then repeatedly goes left until the bottom of
the heap (this path may differ from the actual downheap path)
• Since each node is traversed by at most two proxy paths, the total
number of nodes of the proxy paths is O(n)
• Thus, bottom-up heap construction runs in O(n) time
• Bottom-up heap construction is faster than n successive insertions
and speeds up the first phase of heap-sort
Priority Queues
Trees
34 4/8/2015 5:25
AM
3 a
Locators
1 g
Priority Queues
Trees
4 e
35 4/8/2015 5:25
AM
Locators
• A locator identifies and tracks a (key,
element) item within a data structure
• A locator sticks with a specific item,
even if that element changes its
position in the data structure
• Intuitive notion:
• Application example:
• Orders to purchase and sell a
given stock are stored in two
priority queues (sell orders
and buy orders)
• the key of an order is the
price
• the element is the number of
shares
• claim check
• reservation number
• Methods of the locator ADT:
• key(): returns the key of the item
associated with the locator
• element(): returns the element of the
item associated with the locator
• When an order is placed, a
locator to it is returned
• Given a locator, an order can
be canceled or modified
Priority Queues
Trees
36 4/8/2015 5:25
AM
Locator-based Methods
• Locator-based priority queue
methods:
• insert(k, o): inserts the item
(k, o) and returns a locator
for it
• min(): returns the locator of
an item with smallest key
• remove(l): remove the item
with locator l
• replaceKey(l, k): replaces the
key of the item with locator l
• replaceElement(l, o):
replaces with o the element
of the item with locator l
• locators(): returns an iterator
over the locators of the items
in the priority queue
Locator-based dictionary
methods:
• insert(k, o): inserts the item (k,
o) and returns its locator
• find(k): if the dictionary
contains an item with key k,
returns its locator, else return a
special null locator
• remove(l): removes the item
with locator l and returns its
element
• locators(), replaceKey(l, k),
replaceElement(l, o)
Priority Queues
Trees
37 4/8/2015 5:25
AM
Possible Implementation
• The locator is an
object storing
• key
• element
• position (or rank) of
the item in the
underlying structure
6 d
3 a
9 b
• In turn, the position
(or array cell) stores
the locator
• Example:
• binary search tree
with locators
1 g
Priority Queues
Trees
4 e
8 c
38 4/8/2015 5:25
AM
Positions vs. Locators
• Position
• Locator
• represents a “place” in a data
structure
• related to other positions in
the data structure (e.g.,
previous/next or parent/child)
• often implemented as a
pointer to a node or the index
of an array cell
• Position-based ADTs (e.g.,
sequence and tree) are
fundamental data storage
schemes
• identifies and tracks a (key,
element) item
• unrelated to other locators in
the data structure
• often implemented as an
object storing the item and its
position in the underlying
structure
• Key-based ADTs (e.g., priority
queue and dictionary) can be
augmented with locator-based
methods
Priority Queues
Trees
39 4/8/2015 5:25
AM