Priority Queues
Binary and Binomial Heaps
Supports the following operations.
■
Insert element x.
■
Return min element.
■
Return and delete minimum element.
■
Decrease key of element x to k.
Applications.
These lecture slides are adapted
from CLRS, Chapters 6, 19.
■
Dijkstra’s shortest path algorithm.
■
Prim’s MST algorithm.
■
Event-driven simulation.
■
Huffman encoding.
■
Heapsort.
■
...
Princeton University • COS 423 • Theory of Algorithms • Spring 2002 • Kevin Wayne
2
Priority Queues in Action
Priority Queues
Heaps
Dijkstra’s Shortest Path Algorithm
PQinit()
for each v ∈ V
key(v) ← ∞
PQinsert(v)
key(s) ← 0
while (!PQisempty())
v = PQdelmin()
for each w ∈ Q s.t (v,w) ∈ E
if π(w) > π(v) + c(v,w)
PQdecrease(w, π(v) + c(v,w))
Operation
Linked List
Binary
Binomial
Fibonacci *
Relaxed
make-heap
1
1
1
1
1
insert
1
log N
log N
1
1
find-min
N
1
log N
1
1
delete-min
N
log N
log N
log N
log N
union
1
N
log N
1
1
decrease-key
1
log N
log N
1
1
delete
N
log N
log N
log N
log N
is-empty
1
1
1
1
1
Dijkstra/Prim
1 make-heap
|V| insert
|V| delete-min
|E| decrease-key
3
O(|V|2)
O(|E| log |V|)
O(|E| + |V| log |V|)
4
Binary Heap: Definition
Binary Heap: Properties
Binary heap.
■
■
Properties.
Almost complete binary tree.
– filled on all levels, except last, where filled from left to right
■
■
Min element is in root.
Heap with N elements has height = log2 N.
Min-heap ordered.
– every child greater than (or equal to) parent
06
06
14
78
91
83
14
45
18
47
77
81
84
78
53
99
N = 14
Height = 3
18
91
83
64
45
47
77
81
84
53
99
64
5
6
Binary Heaps: Array Implementation
Binary Heap: Insertion
Implementing binary heaps.
■
Insert element x into heap.
Use an array: no need for explicit parent or child pointers.
– Parent(i) = i/2
– Left(i)
= 2i
– Right(i)
= 2i + 1
■
■
Insert into next available slot.
Bubble up until it’s heap ordered.
– Peter principle: nodes rise to level of incompetence
06
06
1
14
45
2
3
14
78
18
47
53
4
5
6
7
83
91
81
8
9
10
77
84
99
64
11
12
13
14
78
83
7
45
47
18
91
81
77
84
53
99
64
42
next free slot
8
Binary Heap: Insertion
Binary Heap: Insertion
Insert element x into heap.
■
■
Insert element x into heap.
Insert into next available slot.
■
Bubble up until it’s heap ordered.
– Peter principle: nodes rise to level of incompetence
14
Bubble up until it’s heap ordered.
– Peter principle: nodes rise to level of incompetence
14
47
77
81
84
78
53
99
swap with parent
06
45
18
91
83
Insert into next available slot.
swap with parent
06
78
■
64
18
91
83
42
45
47
77
81
84
42
99
64
53
42
9
10
Binary Heap: Insertion
Binary Heap: Decrease Key
Insert element x into heap.
■
■
■
Decrease key of element x to k.
Insert into next available slot.
Bubble up until it’s heap ordered.
– Peter principle: nodes rise to level of incompetence
■
Bubble up until it’s heap ordered.
■
O(log N) operations.
O(log N) operations.
stop: heap ordered
06
14
78
83
81
14
42
47
18
91
06
77
84
78
45
99
64
83
53
11
42
47
18
91
81
77
84
45
99
64
53
12
Binary Heap: Delete Min
Binary Heap: Delete Min
Delete minimum element from heap.
■
■
Delete minimum element from heap.
Exchange root with rightmost leaf.
■
Bubble root down until it’s heap ordered.
– power struggle principle: better subordinate is promoted
■
Exchange root with rightmost leaf.
Bubble root down until it’s heap ordered.
– power struggle principle: better subordinate is promoted
06
53
14
78
18
91
83
14
42
47
77
81
84
78
45
99
64
18
91
83
53
42
47
77
81
84
45
99
64
06
13
14
Binary Heap: Delete Min
Binary Heap: Delete Min
Delete minimum element from heap.
■
■
Delete minimum element from heap.
Exchange root with rightmost leaf.
■
Bubble root down until it’s heap ordered.
– power struggle principle: better subordinate is promoted
14
83
81
53
47
77
84
78
45
99
exchange with right child
14
42
18
91
Bubble root down until it’s heap ordered.
– power struggle principle: better subordinate is promoted
exchange with left child
53
78
■
Exchange root with rightmost leaf.
83
64
15
42
47
18
91
81
77
84
45
99
64
16
Binary Heap: Delete Min
Binary Heap: Heapsort
Delete minimum element from heap.
■
■
■
Heapsort.
Exchange root with rightmost leaf.
Bubble root down until it’s heap ordered.
– power struggle principle: better subordinate is promoted
O(log N) operations.
18
Perform N delete-min operations.
■
O(N log N) sort.
■
No extra storage.
42
78
53
91
Insert N items into binary heap.
■
stop: heap ordered
14
83
■
47
77
81
84
45
99
64
17
18
Binary Heap: Union
Priority Queues
Heaps
Union.
■
■
■
Combine two binary heaps H1 and H2 into a single heap.
Operation
Linked List
Binary
Binomial
Fibonacci *
Relaxed
No easy solution.
– Ω(N) operations apparently required
make-heap
1
1
1
1
1
insert
1
log N
log N
1
1
find-min
N
1
log N
1
1
Can support fast union with fancier heaps.
delete-min
N
log N
log N
log N
log N
H1
H2
union
1
N
log N
1
1
14
11
decrease-key
1
log N
log N
1
1
delete
N
log N
log N
log N
log N
is-empty
1
1
1
1
1
78
41
18
91
81
53
77
84
62
99
64
19
20
Binomial Tree
Binomial Tree
Binomial tree.
■
Useful properties of order k binomial tree Bk.
Recursive definition:
Bk
B0
■
Number of nodes = 2k.
■
Height = k.
■
Degree of root = k.
■
Bk-1
Bk+1
Deleting root yields binomial
trees Bk-1, … , B0.
Bk-1
Bk
Proof.
■
B0
B1
B2
B3
B0
B4
B2
B1
B0
By induction on k.
B1
B2
B3
B4
21
22
Binomial Tree
Binomial Heap
A property useful for naming the data structure.
■
Bk has
k 
 
i 
Binomial heap. Vuillemin, 1978.
nodes at depth i.
■
Sequence of binomial trees that satisfy binomial heap property.
– each tree is min-heap ordered
– 0 or 1 binomial tree of order k
4
 =6
2
depth 0
depth 1
8
depth 2
depth 3
depth 4
45
B4
30
23
32
24
22
48
29
10
31
17
6
3
44
37
18
50
55
B4
23
B1
B0
24
Binomial Heap: Implementation
Binomial Heap: Properties
Implementation.
■
■
Properties of N-node binomial heap.
Represent trees using left-child, right sibling pointers.
– three links per node (parent, left, right)
■
■
Roots of trees connected with singly linked list.
– degrees of trees strictly decreasing from left to right
■
heap
29
10
6
3
44
37
18
17
Contains binomial tree Bi iff bi = 1 where bn⋅ b2b1b0 is binary
representation of N.
At most log2 N + 1 binomial trees.
Height ≤ log2 N.
18
37
8
48
50
10
31
17
3
44
37
30
23
32
24
22
48
31
18
N = 19
# trees = 3
height = 4
binary = 10011
10
45
50
29
6
t
gh
Ri
31
3
29
ft
Le
48
6
nt
re
a
P
■
Min key contained in root of B0, B1, . . . , Bk.
17
50
44
55
Binomial Heap
Leftist Power-of-2 Heap
B4
B1
B0
25
26
Binomial Heap: Union
Binomial Heap: Union
Create heap H that is union of heaps H’ and H’’.
■
■
"Mergeable heaps."
Easy if H’ and H’’ are each order k binomial trees.
– connect roots of H’ and H’’
– choose smaller key to be root of H
8
6
8
30
45
32
23
22
24
48
29
10
31
17
45
+
44
30
23
32
24
22
48
29
10
31
17
6
3
44
37
28
55
33
25
12
41
19 + 7 = 26
H’
7
50
50
55
15
18
+
1
1
1
1
0
0
1
1
0
0
1
1
1
1
1
0
1
0
H’’
27
28
Binomial Heap: Union
Binomial Heap: Union
12
18
8
30
23
22
48
29
10
31
17
6
3
44
37
15
45
+
32
24
7
18
8
45
28
33
23
32
24
22
48
10
31
17
3
44
37
12
50
55
30
29
6
+
7
33
25
12
50
25
41
15
18
28
55
41
29
7
3
12
37
18
30
3
25
28
15
7
33
25
37
7
3
12
37
18
25
41
8
30
23
22
48
29
10
31
17
6
3
44
37
15
45
+
55
32
24
7
18
8
45
33
23
32
24
22
48
10
31
17
3
44
37
12
50
28
30
29
6
25
+
41
55
15
7
33
25
12
50
28
41
12
12
18
18
31
18
32
3
28
15
7
33
25
7
37
3
12
37
18
3
25
28
41
8
30
23
22
48
29
10
31
17
32
24
3
44
37
7
18
8
45
28
+
18
15
7
33
25
30
23
32
24
22
48
29
10
31
17
6
3
44
37
3
12
37
18
15
7
33
25
18
12
50
28
41
6
8
45
30
23
32
24
22
48
29
10
31
17
44
28
50
15
7
33
25
3
12
37
18
41
55
33
34
Binomial Heap: Union
Binomial Heap: Delete Min
Create heap H that is union of heaps H’ and H’’.
Delete node with minimum key in binomial heap H.
Analogous to binary addition.
■
■
Running time. O(log N)
■
37
25
55
41
■
25
25
33
41
28
33
7
37
12
50
55
+
7
12
41
6
15
45
15
3
■
Find root x with min key in root list of H, and delete
H’ ← broken binomial trees
H ← Union(H’, H)
Proportional to number of trees in root lists ≤ 2( log2 N + 1).
Running time. O(log N)
8
19 + 7 = 26
+
1
1
1
1
0
0
1
1
0
0
1
1
1
1
1
0
1
0
45
30
23
32
24
22
48
29
10
31
17
3
6
44
37
18
H
50
55
35
36
Binomial Heap: Delete Min
Binomial Heap: Decrease Key
Delete node with minimum key in binomial heap H.
■
■
■
Decrease key of node x in binomial heap H.
Find root x with min key in root list of H, and delete
H’ ← broken binomial trees
■
Suppose x is in binomial tree Bk.
■
Bubble node x up the tree if x is too small.
H ← Union(H’, H)
Running time. O(log N)
Running time. O(log N)
■
6
8
45
30
23
32
24
22
48
29
10
31
17
44
18
H’
8
depth = 3
37
H
x
50
55
Proportional to depth of node x ≤ log2 N .
30
23
32
24
22
48
29
10
31
17
3
6
44
37
18
H
50
55
37
38
Binomial Heap: Delete
Binomial Heap: Insert
Delete node x in binomial heap H.
Insert a new node x into binomial heap H.
■
Decrease key of x to -∞.
■
■
Delete min.
■
Running time. O(log N)
H’ ← MakeHeap(x)
H ← Union(H’, H)
Running time. O(log N)
8
45
30
23
32
24
22
48
29
10
31
17
3
6
44
37
18
H
x
H’
50
55
39
40
Binomial Heap: Sequence of Inserts
Insert a new node x into binomial heap H.
If N = .......0, then only 1 steps.
■
■
If N = ......01, then only 2 steps.
■
If N = .....011, then only 3 steps.
■
If N = ....0111, then only 4 steps.
48
29
10
31
17
Priority Queues
3
6
44
37
x
Heaps
50
Inserting 1 item can take Ω(log N) time.
If N = 11...111, then log2 N steps.
■
But, inserting sequence of N items takes O(N) time!
■
■
■
(N/2)(1) + (N/4)(2) + (N/8)(3) + . . . ≤ 2N
Amortized analysis.
Basis for getting most operations
down to constant time.
N
∑ nn
n=1 2
= 2 − NN − N1−1
2
2
≤ 2
Operation
Linked List
Binary
Binomial
Fibonacci *
Relaxed
make-heap
1
1
1
1
1
insert
1
log N
log N
1
1
find-min
N
1
log N
1
1
delete-min
N
log N
log N
log N
log N
union
1
N
log N
1
1
decrease-key
1
log N
log N
1
1
delete
N
log N
log N
log N
log N
is-empty
1
1
1
1
1
just did this
41
42