Algorithms
Algorithms
F O U R T H
E D I T I O N
R OBERT S EDGEWICK | K EVIN W AYNE
2.4 P RIORITY Q UEUES
2.4 P RIORITY Q UEUES
‣ API and elementary implementations
‣ API and elementary implementations
‣ binary heaps
‣ binary heaps
‣ heapsort
Algorithms
‣ event-driven simulation
‣ heapsort
‣ event-driven simulation
R OBERT S EDGEWICK | K EVIN W AYNE
R OBERT S EDGEWICK | K EVIN W AYNE
http://algs4.cs.princeton.edu
http://algs4.cs.princeton.edu
Collections
Priority queue
A collection is a data types that store groups of items.
Collections. Insert and delete items. Which item to delete?
data type
key operations
data structure
stack
PUSH, POP
linked list, resizing array
queue
ENQUEUE, DEQUEUE
linked list, resizing array
Stack. Remove the item most recently added.
Queue. Remove the item least recently added.
priority queue
INSERT,
DELETE-MAX
Randomized queue. Remove a random item.
Priority queue. Remove the largest (or smallest) item.
binary heap
symbol table
PUT, GET, DELETE
BST, hash table
set
ADD, CONTAINS, DELETE
BST, hash table
operation argument
insert
insert
insert
remove max
insert
insert
insert
remove max
insert
insert
insert
remove max
“ Show me your code and conceal your data structures, and I shall
continue to be mystified. Show me your data structures, and I won't
usually need your code; it'll be obvious.” — Fred Brooks
return
value size
P
Q
E
Q
X
A
M
X
P
L
E
P
1
2
3
2
3
4
5
4
5
6
7
6
contents
(unordered)
P
P
P
P
P
P
P
P
P
P
P
E
Q
Q
E
E
E
E
E
E
E
E
M
conten
(ordere
E
X
X
X
M
M
M
M
A
A
A
A
A
A
A
P
M
P
P
P
L
L
L
E
E
3
P
P
E
E
E
A
A
A
A
A
A
A
4
A sequence of operations on a priority queue
Q
P
P
P
E
E
E
E
E
E
E
Q
X
P
M
M
M
L
E
E
Priority queue API
Priority queue applications
Requirement. Generic items are Comparable.
・Event-driven simulation.
・Numerical computation.
・Data compression.
・Graph searching.
・Number theory.
・Artificial intelligence.
・Statistics.
・Operating systems.
・Computer networks.
・Discrete optimization.
・Spam filtering.
Key must be Comparable
(bounded type parameter)
public class MaxPQ<Key extends Comparable<Key>>
create an empty priority queue
MaxPQ()
MaxPQ(Key[] a)
create a priority queue with given keys
insert a key into the priority queue
void insert(Key v)
return and remove the largest key
Key delMax()
boolean isEmpty()
is the priority queue empty?
Key max()
return the largest key
[ customers in a line, colliding particles ]
[ reducing roundoff error ]
[ Huffman codes ]
[ Dijkstra's algorithm, Prim's algorithm ]
[ sum of powers ]
[ A* search ]
[ online median in data stream ]
[ load balancing, interrupt handling ]
[ web cache ]
[ bin packing, scheduling ]
[ Bayesian spam filter ]
number of entries in the priority queue
int size()
Generalizes: stack, queue, randomized queue.
5
6
Priority queue client example
Priority queue client example
Challenge. Find the largest M items in a stream of N items.
Challenge. Find the largest M items in a stream of N items.
・Fraud detection:
・NSA monitoring:
・Fraud detection:
・NSA monitoring:
isolate $$ transactions.
flag most suspicious documents.
N huge, M large
Constraint. Not enough memory to store N items.
% more tinyBatch.txt
Turing
6/17/1990
vonNeumann 3/26/2002
Dijkstra
8/22/2007
vonNeumann 1/11/1999
Dijkstra
11/18/1995
Hoare
5/10/1993
vonNeumann 2/12/1994
Hoare
8/18/1992
Turing
1/11/2002
Thompson
2/27/2000
Turing
2/11/1991
Hoare
8/12/2003
vonNeumann 10/13/1993
Dijkstra
9/10/2000
Turing
10/12/1993
Hoare
2/10/2005
644.08
4121.85
2678.40
4409.74
837.42
3229.27
4732.35
4381.21
66.10
4747.08
2156.86
1025.70
2520.97
708.95
3532.36
4050.20
% java TopM
Thompson
vonNeumann
vonNeumann
Hoare
vonNeumann
isolate $$ transactions.
flag most suspicious documents.
N huge, M large
Constraint. Not enough memory to store N items.
5 < tinyBatch.txt
2/27/2000 4747.08
2/12/1994 4732.35
1/11/1999 4409.74
8/18/1992 4381.21
3/26/2002 4121.85
MinPQ<Transaction> pq = new MinPQ<Transaction>();
use a min-oriented pq
sort key
7
while (StdIn.hasNextLine())
{
String line = StdIn.readLine();
Transaction item = new Transaction(line);
pq.insert(item);
pq contains
if (pq.size() > M)
largest M items
pq.delMin();
}
Transaction data
type is Comparable
(ordered by $$)
8
Priority queue client example
Priority queue: unordered and ordered array implementation
Challenge. Find the largest M items in a stream of N items.
operation argument
implementation
sort
elementary PQ
binary heap
best in theory
time
insert
insert
insert
remove max
insert
insert
insert
remove max
insert
insert
insert
remove max
space
N log N
N
MN
M
N log M
M
N
M
return
value size
P
Q
E
1
2
3
2
3
4
5
4
5
6
7
6
Q
X
A
M
X
P
L
E
P
order of growth of finding the largest M in a stream of N items
contents
(unordered)
P
P
P
P
P
P
P
P
P
P
P
E
Q
Q
E
E
E
E
E
E
E
E
M
contents
(ordered)
P
P
E
E
E
A
A
A
A
A
A
A
E
X
X
X
M
M
M
M
A
A
A
A
A
A
A
P
M
P
P
P
L
L
L
E
E
Q
P
P
P
E
E
E
E
E
E
E
Q
X
P
M
M
M
L
E
E
X
P
P
P
M
L
L
X
P
P
M
M
P
P
P
P
A sequence of operations on a priority queue
9
Priority queue: unordered array implementation
10
Priority queue elementary implementations
Challenge. Implement all operations efficiently.
public class UnorderedArrayMaxPQ<Key extends Comparable<Key>>
{
private Key[] pq;
// pq[i] = ith element on pq
private int N;
// number of elements on pq
public UnorderedArrayMaxPQ(int capacity)
{ pq = (Key[]) new Comparable[capacity];
}
no generic
array creation
public boolean isEmpty()
{ return N == 0; }
public void insert(Key x)
{ pq[N++] = x; }
public Key delMax()
{
int max = 0;
for (int i = 1; i < N; i++)
if (less(max, i)) max = i;
exch(max, N-1);
should null out entry
return pq[--N];
to prevent loitering
}
less() and exch()
similar to sorting methods
(but don't pass pq[])
implementation
insert
del max
max
unordered array
1
N
N
ordered array
N
1
1
goal
log N
log N
log N
order of growth of running time for priority queue with N items
}
11
12
Complete binary tree
Binary tree. Empty or node with links to left and right binary trees.
Complete tree. Perfectly balanced, except for bottom level.
2.4 P RIORITY Q UEUES
‣ API and elementary implementations
‣ binary heaps
Algorithms
‣ heapsort
‣ event-driven simulation
complete tree with N = 16 nodes (height = 4)
R OBERT S EDGEWICK | K EVIN W AYNE
http://algs4.cs.princeton.edu
Property. Height of complete tree with N nodes is ⎣lg N⎦.
Pf. Height increases only when N is a power of 2.
14
A complete binary tree in nature
Binary heap representations
Binary heap. Array representation of a heap-ordered complete binary tree.
Heap-ordered binary tree.
・Keys in nodes.
・Parent's key no smaller than
children's keys.
i
a[i]
0
-
1
T
2
S
3
R
S
R
4
P
5
N
6
O
7
A
P
N
O
A
8
E
9 10 11
I H G
E
I
T
Array representation.
・
・Take nodes in level order.
・No explicit links needed!
Indices start at 1.
1
8 E
9 I
G
3 R
2 S
4 P
H
T
5 N
10 H
6 O
7 A
11 G
Heap representations
15
16
Binary heap properties
Binary heap demo
Proposition. Largest key is a[1], which is root of binary tree.
Insert. Add node at end, then swim it up.
Remove the maximum. Exchange root with node at end, then sink it down.
Proposition. Can use array indices to move through tree.
・Parent of node at k is at k/2.
・Children of node at k are at 2k and 2k+1.
heap ordered
i
a[i]
0
-
1
T
2
S
3
R
S
R
4
P
5
N
6
O
7
A
8
E
T
9 10 11
I H G
T
P
P
N
O
R
A
N
1
4 P
6 O
5 N
9 I
I
H
10 H
O
A
G
T
E
3 R
2 S
8 E
E
H
I
G
7 A
11 G
T
Heap representations
P
R
N
H
O
A
E
I
G
17
18
Binary heap demo
Promotion in a heap
Insert. Add node at end, then swim it up.
Scenario. Child's key becomes larger key than its parent's key.
Remove the maximum. Exchange root with node at end, then sink it down.
To eliminate the violation:
・Exchange key in child with key in parent.
・Repeat until heap order restored.
heap ordered
S
S
R
O
N
G
P
E
private void swim(int k)
{
while (k > 1 && less(k/2, k))
{
exch(k, k/2);
k = k/2;
}
parent of node at k is at k/2
}
A
H
I
R
P
5
N
E
I
T
H
O
A
violates heap order
(larger key than parent)
G
1
T
2
5 P
N
E
R
S
I
H
O
A
G
Bottom-up reheapify (swim)
S
R
O
N
P
G
A
E
I
H
Peter principle. Node promoted to level of incompetence.
19
20
Insertion in a heap
Demotion in a heap
Insert. Add node at end, then swim it up.
Scenario. Parent's key becomes smaller than one (or both) of its children's.
Cost. At most 1 + lg N compares.
insert
To eliminate
the
violation:
key
to remove
T
R
P
N
E
public void insert(Key x)
{
pq[++N] = x;
swim(N);
}
H
I
G
O
S
S
A
N
E
key to insert
・Exchange key in parent with key in larger child.
・Repeat until heap order restored.
R
P
I
G
T
R
P
N
E
H
I
G
O
A
N
E
I
G
T
R
S
N
E
S
add key to heap
violates heap order
S
swim up
P
I
G
why not smaller child?
remove the maximum
T
O
P
sink down
N
A
H
E
Heap operations
I
O
H
A
exchange key
with root
violates heap order
(smaller than a child)
private void sink(int k)
violates
{ H
heap order
children of node at k
while (2*k <= N)
are 2k and 2k+1
R
{
P
O int A j = 2*k;
remove
if node
(j < N && less(j, j+1)) j++;
T
from heap
if (!less(k, j)) break;
S
exch(k, j);
k R = j;
}
H
O
A
}
2
R
H
5 S
P
E
T
I
N
O
A
G
T
2
5
P
E
R
S
I
10
H
N
O
A
G
Top-down reheapify (sink)
G
Power struggle. Better subordinate promoted.
21
Delete the maximum in a heap
22
Binary heap: Java implementation
Delete max. Exchange root with node at end, then sink it down.
public class MaxPQ<Key extends Comparable<Key>>
{
private Key[] pq;
private int N;
Cost. At most 2 lg N compares.
insert
remove the maximum
T
R
P
N
public Key delMax()
E
I
{
Key max = pq[1];
exch(1, N--);
P
sink(1);
pq[N+1] = null;
N
return max;
E
I
}
H
G
O
S
A
N
E
key to insert
P
I
G
R
G
R
N
E
P
I
G
R
P
G
O
A
remove node
from heap
T
private void swim(int k)
private void sink(int k)
{
/* see previous code */
S
S
N
public boolean isEmpty()
{
return N == 0;
}
public void insert(Key key)
public Key delMax()
{
/* see previous code */
violates
heap order
S
add key to heap
violates heap order
S
public MaxPQ(int capacity)
{ pq = (Key[]) new Comparable[capacity+1];
A
exchange key
with root
H
T
I
O
H
prevent loitering
O
A
swim up
E
R
S
T
H
key to remove
T
O
N
A
H
E
H
I
PQ ops
}
heap helper functions
}
R
P
sink down
fixed capacity
(for simplicity)
}
O
private boolean less(int i, int j)
{
return pq[i].compareTo(pq[j]) < 0; }
private void exch(int i, int j)
{
Key t = pq[i]; pq[i] = pq[j]; pq[j] = t;
A
G
array helper functions
}
}
Heap operations
23
24
Priority queues implementation cost summary
Binary heap: practical improvements
Half-exchanges in sink and swim.
implementation
insert
del max
max
unordered array
1
N
N
ordered array
N
1
1
binary heap
log N
log N
1
・Reduces number of array accesses.
・Worth doing.
Z
order-of-growth of running time for priority queue with N items
T
L
B
1
\
X
25
26
Binary heap: practical improvements
Binary heap: practical improvements
Floyd's sink-to-bottom trick.
Multiway heaps.
・Sink key at root all the way to bottom.
・Swim key back up.
・Fewer compares; more exchanges.
・Worthwhile depending on cost of compare and exchange.
・Complete d-way tree.
・Parent's key no smaller than its children's keys.
・Swim takes log N compares; sink takes d log N compares.
・Sweet spot: d = 4.
1 compare per node
some extra compares and exchanges
d
R. W. Floyd
1978 Turing award
d
F
Z
X
Y
Y
N
T
G
O
J
L
P
W
I
K
A
B
D
K
E
1
E
X
H
F
R
S
V
C
L
M
Q
N
O
\
D
3-way heap
27
28
size in bytes of each heap element.
In this analysis, we restrict fanout to be a positive power of 2 and element size to be a power of 2. We also restrict
our heap configurations to those in which all of a parent’s children fit in a single cache block (where
). This
Binary
heap:
practical
limits the values
of that
we are
looking at;improvements
for a typical cache block size of 32 bytes, fanout is limited to 4 for 8 byte
heap elements, and fanout is limited to 8 for 4 byte heap elements. We also restrict our analysis to heap configurations
in which the bottom layer of the heap is completely full (where
=
).
Caching. Binary heap is not cache friendly.
Heaps are often used in discrete event simulations as a priority queue to store the events. In order to measure the
Cache-aligned
performance of heaps
operating as d-heap.
an event queue, we analyze our heaps in the hold model [25]. In the hold model,
Funnel heap.
Binary heap: practical improvements
Caching. Binary heap is not cache friendly.
・
It has been・
noticed previously that increasing a heap’s fanout can reduce the instruction count of its operations [27, Ex. 28 Pg.
158][12, Ex. 7-2
152].
・Pg.B-heap.
・…
0
block 0
1
2
3
4
block 1
0
1
2
block 2
3
4
5
6
block 3
7
8
9 10 11 12
Siblings
5 6
7 8 9 10 11 12
Figure 5: The layout of a -heap when four elements fit per cache line and the array is padded to cache-align the heap.
10
29
Priority queues implementation cost summary
30
Binary heap considerations
Underflow and overflow.
implementation
insert
del max
max
unordered array
1
N
N
ordered array
N
1
1
binary heap
log N
log N
1
d-ary heap
logd N
d logd N
1
Fibonacci
1
log N †
1
Brodal queue
1
log N
1
impossible
1
1
1
・Underflow: throw exception if deleting from empty PQ.
・Overflow: add no-arg constructor and use resizing array.
Minimum-oriented priority queue.
leads to log N
amortized time per op
(how to make worst case?)
・Replace less() with greater().
・Implement greater().
Other operations.
・Remove an arbitrary item.
・Change the priority of an item.
can implement efficiently with sink() and swim()
[ stay tuned for Prim/Dijkstra ]
Immutability of keys.
why impossible?
・Assumption: client does not change keys while they're on the PQ.
・Best practice: use immutable keys.
† amortized
order-of-growth of running time for priority queue with N items
31
32
Immutability: implementing in Java
Immutability: properties
Data type. Set of values and operations on those values.
Data type. Set of values and operations on those values.
Immutable data type. Can't change the data type value once created.
Immutable data type. Can't change the data type value once created.
public final class Vector {
private final int N;
private final double[] data;
Advantages.
can't override instance methods
・Simplifies debugging.
・Safer in presence of hostile code.
・Simplifies concurrent programming.
・Safe to use as key in priority queue or symbol table.
instance variables private and final
public Vector(double[] data) {
this.N = data.length;
this.data = new double[N];
for (int i = 0; i < N; i++)
this.data[i] = data[i];
}
defensive copy of mutable
instance variables
…
instance methods don't change
instance variables
Disadvantage. Must create new object for each data type value.
“ Classes should be immutable unless there's a very good reason
}
to make them mutable.… If a class cannot be made immutable,
you should still limit its mutability as much as possible. ”
Immutable. String, Integer, Double, Color, Vector, Transaction, Point2D.
— Joshua Bloch (Java architect)
Mutable. StringBuilder, Stack, Counter, Java array.
33
34
Sorting with a binary heap
Q. What is this sorting algorithm?
2.4 P RIORITY Q UEUES
‣ API and elementary implementations
‣ binary heaps
Algorithms
R OBERT S EDGEWICK | K EVIN W AYNE
http://algs4.cs.princeton.edu
‣ heapsort
public void sort(String[] a)
{
int N = a.length;
MaxPQ<String> pq = new MaxPQ<String>();
for (int i = 0; i < N; i++)
pq.insert(a[i]);
for (int i = N-1; i >= 0; i--)
a[i] = pq.delMax();
}
‣ event-driven simulation
Q. What are its properties?
A. N log N, extra array of length N, not stable.
Heapsort intuition. A heap is an array; do sort in place.
36
E
E
P
M
S
Heapsort
O
exch(1, 3)
sink(1, 2)
R
P
X
L
A
E
M
X tree.
T
S
S
M
R
binary
・EView input array as a complete
build a max-heap with all N keys.
・Heap construction: exch(1,
8)
exch(1, 2)
sink(2, 11)
P
S
remove
sink(1,
7) the maximum key.sink(1, 1)
・Sortdown: repeatedly
T
L
P
sink(1, 211)
2
O
8
9
XO
4 5
56
T E
8 10
9 11 10
3
R
S
E
X
7
1
2
3
4
S5
S
O
R
T
E
O
O
exch(1, 7)
sink(1, 6)
S
3
R
A X
7
6
S8
R
A
P L
L
LRE
APR
A
11
R
A
A X
LR
10
T
S
S
P
X
T
result (sorted)
A
LE
7
L
8
9
P
R
S
E
L 10
MO
5
T
M
P
9
L
10
1
P
S
O
R
T
1
2
3
4
5
E
X
A
6
8
2
OM
5
T
7
6
Heapsort: heap construction
E
O
O
X
X
P
L
A
R
T L
Aorder
R
array
in sorted
T
P
E
E
LE
A
A
A E
X
S M T S X T
M
3)
exch(1,exch(1,
3)
2)
sink(1,sink(1,
2)
R
R
O L
O
E
P M E P E E
M
9)
exch(1,exch(1,
9)
8)
sink(1,sink(1,
8)
S
A
L
MO
L M
L
11)
sink(2, sink(2,
11)
S
T
T
P
11)
sink(1, sink(1,
11)
X
T
P
M
R
T
P L
1
X
S
T
M
A
A
X
E
E
LE
M
S R T S X T
LE
P E
X
P
8
4
L
9
2
E
4
8
P
PO
X
T
9
10
6
E
3
R
X
7
3
11
5
10
6M
O
11
A
X
7
6
PO
E
7
P
M
E
R
R
T
L
E
R
L
A
E
X
T
E
L
A
A
A
S
R
A
X
T
L
O
E
X
SR
L
R
T
S
e
s
E
A
E
T O X P
O
E
L E
SM
R
A
E
X
T
A A
AA
P
P
exch(1, 7)
sink(1, 6)
exch(1, 4)
M E
sink(1, 3)
XS
E
L E
SM
R
S
R
OE ET EX
result (heap-ordered)
T
E
S
E M
A
RE
M
O
e
s
R
exch(1, 8)
sink(1, 7)
exch(1, 5)
O L
sink(1, 4)
SA
R
LL
PO
MM
E
exch(1, 9)
8)
exch(1,sink(1,
6)
M
sink(1, 5)
P
X
E
E
TP
A
X
X
S
X
R
LE
sink(1, 11)
exch(1, 10)
sink(1, 9)
S
L
S
X
L
L
P
OO
R
T
P
T
P
M
O
M
A
X
sink(4, 11)
O
T
P
E
E
P
M
e
s
S
O
M
A
R
X
P
E
exch(1, 11)T
sink(1, 10)
S
L
L
E
E
exch(1, 10)
sink(1, 9)
E
E
P
E
E
O
starting point (heap-ordered)
sink(2, 11)
S
M
M
R
T
3
E
A
E
L
P
M
starting point (arbitrary order)
A
10
11
X
SR TS XT
result (sorted)
result (sorted)
X
R
R
E
O M E O E E
A
E
E
L
M O
P
R
S
T
result (heap-ordered)
result (heap-ordered)
1 Heapsort:
2
3
4
5
6 sorting
7
8
9
10
11
constructing
and
down
Heapsort:
constructing
(left) and(left)
sorting
down (right)
a (right)
heap a heap
5
sink(5, 11)
1
A
E
L5 M
9
8
S
O
S
O
M
L
e
s
T
sortdown
O
1
2
E
MO
P
E
E
4
E
SR TS XT
R
O
E
L M
L
2
A
A
O
O
A L
E
E
A
A E
X
M
S
A R
O
S R T S X T
7)
exch(1,exch(1,
7)
6)
sink(1,sink(1,
6)
2)
exch(1,exch(1,
2)
1)
sink(1,sink(1,
1)
P
P
M L
M
R
L
S
L
R
O
E
A
A R
E
O M E O E E
M
X
X
L
R
P L
8)
exch(1,exch(1,
8)
7)
sink(1,sink(1,
7)
S
A
A
R
E 38
E
O
starting point (heap-ordered)
R
sink(3, 11)
A
M
P
X
heap construction
X
SR TS XT
R
L
P
11
S
E
S
exch(1, 11)
sink(1, 10)
O
P
M
P
PO
E
A
10
E
E
P
for (int k = N/2; k >= 1; k--)
T
sink(a, k, N);
E
E
7
S
First pass. Build heap using bottom-up method.
E
X
T
R
T
M
E
RL
O
P
PO
e
s
X
3
8
9
9
10
11
E
L
P
M
starting point (arbitrary order)
sink(4, 11)
11)
sink(3, sink(3,
11)
S
sortdown
S
P
E
sink(5, 11)
X
SR TS XT
E
4
MO
A
E
11
E
E
L M
L
7
X
heap construction
X
A
6
E
X
PO
E
R
3
11
T
E
O
37
A
A
6
SR TS XT
R
A E
4
8
O
A M
A
4)
exch(1,exch(1,
4)
3)
sink(1,sink(1,
3)
R
M
E3 E E
L
E
R
S
2
M1 A
exch(1,
1 exch(1,
2 5)3
4 5)
5
A R
S
O L
O
11
S
1
A4 L
A E5 M EO 6 O P O7 P P
S9 R T10S X11T
X
R8
R
X
E M E E X E
P
9
S
O L
O
10)
exch(1,exch(1,
10)
9)
sink(1,sink(1,
9)
R
8
P
P
A
7
P
O
X
T
L2 E
S
array in arbitrary order
E
exch(1,exch(1,
6)
6)
sink(1,sink(1,
5)
5)
SE
T
exch(1,
11)
1
2exch(1,
3
4
511)
6
S
X
E
T
T E X remaining
E Mlargest
M the
R
Repeatedly
delete
item.
M E P E E
P Sortdown.
M
10
S
L
X
T L
T
M
E
sorted result
sortdownsortdown
(in place)
X
we assume array entries are indexed 1 to N
M
R
Heap construction. Build max heap using bottom-up method.
A
L
E E
E
O
E
S M T O X
starting point
(heap-ordered)
starting
point (heap-ordered)
M
O
O
R
Heapsort
demo
X
P
O
X
T
T
sink(1,
10)
sink(1,
4)
sink(1,
10) P sorting
sink(1,
4)
constructing
L E
X (left)
T S and
L R A down
M O (right)
E E a heap
A E E L M
A X
11)
sink(4, sink(4,
11)
S
T
9
M
R
A
E
XO
AP
E
P M E P E E
M
7
XHeapsort:
A M P
L
X
T L
T
6
T
build max heap
(in place)
11
E
O
result (heap-ordered)
11)
sink(5, sink(5,
11)
L
S
R
P M LL P E R
EA
L
starting point
(arbitrary
order) order)
starting
E point (arbitrary
PM
M
TT
1
S
E
M
keys in arbitrary order
heap construction
heap construction
1
4
A
R
E
E
O
M
O
X
E
L
E
P
M
Heapsort demo
E
A
E
L
A in-place sort. O
R for
Basic
plan
X
T
S
R
exch(1, 9)
sink(1, 8)
sink(3, 11)
T
X
T
E
M
E
P
T O X P
X
40 (right) a
Heapsort: constructing (left) and sorting down
39
sink(3, 11)
O
exch(1, 9)
sink(1, 8)
S
X
exch(1, 3)
sink(1, 2)
R
P
E
E
A
E
Heapsort: sortdown
Heapsort: Java implementation
sortdown
heap construction
Second pass.
1
2
S
3
・
・Leave in array, instead of nulling out.
O
T
R
Remove the maximum, oneT at aE time.
A
X
4
8
5
9
10
6
7
L
sink(4, 11)
exch(1, 10)
sink(1, 9)
S
O
L
T
sink(3, 11)
exch(1, 9)
sink(1, 8)
S
L
T
sink(2, 11)
L
sink(1, 11)
X
L
R
M
1
S
2
E
L
T
E
4
P
8
X
R
E
E
O
result (heap-ordered)
Heapsort: constructing (left) and sorting down (right) a heap
9
3
5
S
10
private static void exch(Object[] a, int i, int j)
{ /* as before */ }
A
E
L
P
O
X
T
S
private static boolean less(Comparable[] a, int i, int j)
{ /* as before */ }
E
O
A
A
A
L
R
M
R
A
E
X
exch(1, 7)
sink(1, 6)
S
P
L
private static void sink(Comparable[] a, int k, int N)
{ /* as before */ }
P
O
X
E
E
T
S
M
T
S
exch(1, 2)
sink(1, 1)
P
M
R
T
M
A
R
E
L
R
O
E
E
A
E
X
exch(1, 8)
sink(1, 7)
S
X
P
O
L
T
S
but make static (and pass arguments)
E
A
E
P
X
T
S
O
exch(1, 3)
sink(1, 2)
R
O
M
T
M
A
R
E
E
P
E
M
L
R
P
X
A
E
X
T
E
M
O
M
L
E
A
R
O
E
E
P
M
A
X
exch(1, 4)
sink(1, 3)
S
P
O
X
T
S
R
P
R
E
M
A
X
E
L
E
A
R
P
O
X
T
S
S
L
E
M
E
A
R
public class Heap
{
public static void sort(Comparable[] a)
{
int N = a.length;
for (int k = N/2; k >= 1; k--)
sink(a, k, N);
while (N > 1)
{
exch(a, 1, N);
sink(a, 1, --N);
}
}
E
exch(1, 5)
sink(1, 4)
T
O
E
E
P
A
X
A
R
P
R
T
M
L
E
E
O
starting point (heap-ordered)
exch(1, 11)
sink(1, 10)
S
O
while (N > 1)
{
exch(a, 1, N--);
sink(a, 1, N);
}
M
M
L
S
P
11
E
L
P
M
starting point (arbitrary order)
sink(5, 11)
exch(1, 6)
sink(1, 5)
X
M
6
O
E
7
P
but convert from 1-based
indexing to 0-base indexing
}
11
X
T
result (sorted)
41
Heapsort: trace
42
Heapsort: mathematical analysis
N
k
initial values
0
1
S
2
O
3
R
a[i]
4 5 6
T E X
Proposition. Heap construction uses ≤ 2 N compares and ≤ N exchanges.
7
A
8
M
9 10 11
P L E
11
11
11
5
4
3
S
S
S
O
O
O
R
R
X
T
T
T
L
L
L
X
X
R
A
A
A
M
M
M
P
P
P
E
E
E
E
E
E
11
2
S
T
X
P
L
R
A
M
O
E
E
T
T
P
S
S
S
P
P
O
L
L
L
R
R
R
A
A
A
M
M
M
O
O
E
E
E
E
E
E
X
11
1
heap-ordered
10
1
X
X
T
9
8
7
1
1
1
S
R
P
P
P
O
R
E
E
O
O
M
L
L
L
E
E
E
A
A
A
M E T X
M S T X
R S T X
6
1
O
M
E
A
L
E
P
R
S
T
X
5
4
1
1
M
L
L
E
E
E
A
A
E
M
O
O
P
P
R
R
S
S
T
T
X
X
3
2
1
1
E
E
A
A
E
E
L
L
M
M
O
O
P
P
R
R
S
S
T
T
X
X
A
A
E
E
E
E
L
L
M
M
O
O
P
P
R
R
S
S
T
T
X
X
1
1
sorted result
max number of exchanges
to sink node
Pf sketch. [assume N = 2h+1 – 1]
3
2
2
1
0
1
0
0
1
0
0
1
0
0
0
binary heap of height h = 3
a tricky sum
(see COS 340)
h + 2(h
Heapsort trace (array contents just after each sink)
1) + 4(h
2) + 8(h
3) + . . . + 2h (0)
2h+1
= N
43
44
Heapsort: mathematical analysis
Introsort
Proposition. Heap construction uses ≤ 2 N compares and ≤ N exchanges.
Goal. As fast as quicksort in practice; N log N worst case, in place.
Proposition. Heapsort uses ≤ 2 N lg N compares and exchanges.
Introsort.
algorithm can be improved to ~ 1 N lg N
・Run quicksort.
・Cutoff to heapsort if stack depth exceeds 2 lg N.
・Cutoff to insertion sort for N = 16.
Significance. In-place sorting algorithm with N log N worst-case.
・Mergesort: no, linear extra space.
・Quicksort: no, quadratic time in worst case.
・Heapsort: yes!
in-place merge possible, not practical
N log N worst-case quicksort possible,
not practical
Bottom line. Heapsort is optimal for both time and space, but:
・Inner loop longer than quicksort’s.
・Makes poor use of cache.
・Not stable.
advanced tricks for improving
In the wild. C++ STL, Microsoft .NET Framework.
45
46
Sorting algorithms: summary
inplace?
selection
✔
insertion
✔
shell
stable?
✔
✔
merge
average
worst
remarks
½N2
½N2
½N2
N exchanges
N
¼N2
½N2
use for small N
or partially ordered
c N 3/2
tight code;
subquadratic
N log3 N
✔
timsort
best
✔
½ N lg N
?
N lg N
N lg N
N
N lg N
N lg N
quick
✔
N lg N
2 N ln N
½N2
3-way quick
✔
N
2 N ln N
½N2
heap
✔
?
✔
✔
N
2 N lg N
2 N lg N
N
N lg N
N lg N
2.4 P RIORITY Q UEUES
‣ API and elementary implementations
N log N guarantee;
‣ binary heaps
stable
improves mergesort
when preexisting order
Algorithms
N log N probabilistic guarantee;
fastest in practice
improves quicksort
when duplicate keys
R OBERT S EDGEWICK | K EVIN W AYNE
http://algs4.cs.princeton.edu
N log N guarantee;
in-place
holy sorting grail
47
‣ heapsort
‣ event-driven simulation
Molecular dynamics simulation of hard discs
Molecular dynamics simulation of hard discs
Goal. Simulate the motion of N moving particles that behave
Goal. Simulate the motion of N moving particles that behave
according to the laws of elastic collision.
according to the laws of elastic collision.
Hard disc model.
・Moving particles interact via elastic collisions with each other and walls.
・Each particle is a disc with known position, velocity, mass, and radius.
・No other forces.
temperature, pressure,
diffusion constant
motion of individual
atoms and molecules
Significance. Relates macroscopic observables to microscopic dynamics.
・Maxwell-Boltzmann: distribution of speeds as a function of temperature.
・Einstein: explain Brownian motion of pollen grains.
49
Warmup: bouncing balls
Warmup: bouncing balls
Time-driven simulation. N bouncing balls in the unit square.
public class BouncingBalls
{
public static void main(String[] args)
{
int N = Integer.parseInt(args[0]);
Ball[] balls = new Ball[N];
for (int i = 0; i < N; i++)
balls[i] = new Ball();
while(true)
{
StdDraw.clear();
for (int i = 0; i < N; i++)
{
balls[i].move(0.5);
balls[i].draw();
}
StdDraw.show(50);
}
main simulation loop
}
}
50
public class Ball
{
private double rx, ry;
private double vx, vy;
private final double radius;
public Ball(...)
{ /* initialize position and
% java BouncingBalls 100
// position
// velocity
// radius
velocity */
check for collision with walls
}
public void move(double dt)
{
if ((rx + vx*dt < radius) || (rx + vx*dt > 1.0 - radius)) { vx = -vx; }
if ((ry + vy*dt < radius) || (ry + vy*dt > 1.0 - radius)) { vy = -vy; }
rx = rx + vx*dt;
ry = ry + vy*dt;
}
public void draw()
{ StdDraw.filledCircle(rx, ry, radius); }
}
Missing. Check for balls colliding with each other.
・Physics problems: when? what effect?
・CS problems: which object does the check?
51
too many checks?
52
Time-driven simulation
Time-driven simulation
・Discretize time in quanta of size dt.
・Update the position of each particle after every dt units of time,
Main drawbacks.
・~ N / 2 overlap checks per time quantum.
・Simulation is too slow if dt is very small.
dt too small: excessive computation
・May miss collisions if dt is too large.
2
and check for overlaps.
・If overlap, roll back the clock to the time of the collision, update the
velocities of the colliding particles, and continue the simulation.
(if colliding particles fail to overlap when we are looking)
dt too large: may miss collisions
dt too small: excessive computation
dt too large: may miss collisions
t
t + dt
t + 2 dt
(collision detected)
t + Δt
(roll back clock)
Fundamental challenge for
time-driven simulation
53
54
Event-driven simulation
Particle-wall collision
Change state only when something happens.
Fundamental challenge for
time-drivenand
simulation
Collision prediction
resolution.
・Particle of radius s at position (rx, ry).
・Particle moving in unit box with velocity (vx, vy).
・Will it collide with a vertical wall? If so, when?
・Between collisions, particles move in straight-line trajectories.
・Focus only on times when collisions occur.
・Maintain PQ of collision events, prioritized by time.
・Remove the min = get next collision.
Collision prediction. Given position, velocity, and radius of a particle,
resolution (at time t + dt)
when will it collide next with a wall or another particle?
s
velocity after collision = ( − vx , vy)
position after collision = ( 1 − s , ry + vydt)
prediction (at time t)
dt ! time to hit wall
= distance/velocity
= (1 − s − rx )/vx
Collision resolution. If collision occurs, update colliding particle(s)
according to laws of elastic collisions.
vx
prediction (at time t)
particles hit unless one passes
intersection point before the other
arrives (see Exercise 3.6.X)
wall at
x=1
(rx , ry )
vy
1 − s − rx
Predicting and resolving a particle-wall collision
resolution (at time t + dt)
velocities of both particles
change after collision (see Exercise 3.6.X)
Predicting and resolving a particle-particle collision
55
56
Particle-particle collision prediction
Particle-particle collision prediction
Collision prediction.
Collision prediction.
・Particle i: radius s , position (rx , ry ), velocity (vx , vy ).
・Particle j: radius s , position (rx , ry ), velocity (vx , vy ).
・Will particles i and j collide? If so, when?
i
i
i
i
i
j
j
j
j
j
・Particle i: radius s , position (rx , ry ), velocity (vx , vy ).
・Particle j: radius s , position (rx , ry ), velocity (vx , vy ).
・Will particles i and j collide? If so, when?
i
i
i
i
i
j
j
j
j
j
(vxi', vyi')
⎧
⎪⎪ ∞
Δt = ⎨ ∞
⎪ Δv ⋅ Δr + d
⎪⎩ Δv ⋅ Δv
(vxj', vyj')
mi
si
(vxi , vyi )
(rxi , ryi)
if Δv ⋅ Δr ≥ 0
if d < 0
otherwise
(rxi', ryi')
d = (Δv ⋅ Δr)2 − (Δv ⋅ Δv) (Δr ⋅ Δr − σ 2 )
i
€
time = t + Δt
time = t
σ = σi + σ j
(vxj , vyj)
€
sj
j
57
Particle-particle collision resolution
€
€
Δv = (Δvx, Δvy) = (vxi − vx j , vyi − vy j )
Δr = (Δrx, Δry) = (rxi − rx j , ryi − ry j )
Δv€
⋅ Δv = (Δvx)2 + (Δvy)2
Δr ⋅ Δr = (Δrx)2 + (Δry)2
Δv ⋅ Δr = (Δvx)(Δrx) + (Δvy)(Δry)
€
€
€
Important note: This is physics, so we won’t be testing you on it!
58
Particle data type skeleton
Collision resolution. When two particles collide, how does velocity change?
= vxiiʹ′ + =Jx vx
/ mi i + Jx / mi
i
vyiʹ′ = vy
vyiiʹ′ + =Jy /vymi i + Jy / mi
vx jʹ′ = vx jʹ′ − =Jx vx
/ mj j− Jx / m j
vy ʹ′ = vy
vx ʹ′ − =Jy vx
/ m − Jy / m
public class Particle
{
private double rx, ry;
private double vx, vy;
private final double radius;
private final double mass;
private int count;
vx ʹ′
j
j
j j
Newton's second law
(momentum form)
//
//
//
//
//
position
velocity
radius
mass
number of collisions
public Particle(...) { }
j
public void move(double dt) { }
public void draw()
{ }
€
Jx =
€
public double timeToHit(Particle that) { }
public double timeToHitVerticalWall()
{ }
public double timeToHitHorizontalWall() { }
2 mi m j (Δv ⋅ Δr)
J Δrx
J Δry
, Jy =
, J =
σ
σ
σ(mi + m j )
public void bounceOff(Particle that)
public void bounceOffVerticalWall()
public void bounceOffHorizontalWall()
impulse due to normal force
(conservation of energy, conservation of momentum)
{ }
{ }
{ }
predict collision
with particle or wall
resolve collision
with particle or wall
}
€
Important note: This is physics, so we won’t be testing you on it!
59
60
Particle-particle collision and resolution implementation
public double timeToHit(Particle that)
{
if (this == that) return INFINITY;
double dx = that.rx - this.rx, dy = that.ry - this.ry;
double dvx = that.vx - this.vx; dvy = that.vy - this.vy;
double dvdr = dx*dvx + dy*dvy;
if( dvdr > 0) return INFINITY;
double dvdv = dvx*dvx + dvy*dvy;
double drdr = dx*dx + dy*dy;
double sigma = this.radius + that.radius;
double d = (dvdr*dvdr) - dvdv * (drdr - sigma*sigma);
if (d < 0) return INFINITY;
return -(dvdr + Math.sqrt(d)) / dvdv;
}
Collision system: event-driven simulation main loop
two particles on a collision course
Initialization.
・Fill PQ with all potential particle-wall collisions.
・Fill PQ with all potential particle-particle collisions.
no collision
third particle interferes: no collision
“potential” since collision may not happen if
some other collision intervenes
An invalidated event
public void bounceOff(Particle that)
{
double dx = that.rx - this.rx, dy = that.ry
double dvx = that.vx - this.vx, dvy = that.vy
double dvdr = dx*dvx + dy*dvy;
double dist = this.radius + that.radius;
double J = 2 * this.mass * that.mass * dvdr /
double Jx = J * dx / dist;
double Jy = J * dy / dist;
this.vx += Jx / this.mass;
this.vy += Jy / this.mass;
that.vx -= Jx / that.mass;
that.vy -= Jy / that.mass;
this.count++;
Important note: This is physics,
that.count++;
}
Main loop.
・Delete the impending event from PQ (min priority = t).
・If the event has been invalidated, ignore it.
・Advance all particles to time t, on a straight-line trajectory.
・Update the velocities of the colliding particle(s).
・Predict future particle-wall and particle-particle collisions involving the
- this.ry;
- this.vy;
((this.mass + that.mass) * dist);
colliding particle(s) and insert events onto PQ.
so we won’t be testing you on it!
61
Event data type
Collision system implementation: skeleton
Conventions.
・
・One particle null
・Both particles null
62
public class CollisionSystem
{
private MinPQ<Event> pq;
private double t = 0.0;
private Particle[] particles;
Neither particle null ⇒ particle-particle collision.
⇒ particle-wall collision.
⇒ redraw event.
// the priority queue
// simulation clock time
// the array of particles
public CollisionSystem(Particle[] particles) { }
private void predict(Particle a)
add to PQ all particle-wall and particleparticle collisions involving this particle
{
if (a == null) return;
for (int i = 0; i < N; i++)
{
double dt = a.timeToHit(particles[i]);
pq.insert(new Event(t + dt, a, particles[i]));
}
pq.insert(new Event(t + a.timeToHitVerticalWall() , a, null));
pq.insert(new Event(t + a.timeToHitHorizontalWall(), null, a));
private class Event implements Comparable<Event>
{
private double time;
// time of event
private Particle a, b;
// particles involved in event
private int countA, countB; // collision counts for a and b
public Event(double t, Particle a, Particle b) { }
create event
public int compareTo(Event that)
{
return this.time - that.time;
ordered by time
public boolean isValid()
{
}
}
}
invalid if
intervening
collision
private void redraw()
{ }
public void simulate() {
}
/* see next slide */
}
}
63
64
Collision system implementation: main event-driven simulation loop
public void simulate()
{
pq = new MinPQ<Event>();
for(int i = 0; i < N; i++) predict(particles[i]);
pq.insert(new Event(0, null, null));
while(!pq.isEmpty())
{
Event event = pq.delMin();
if(!event.isValid()) continue;
Particle a = event.a;
Particle b = event.b;
null
null
null
null
&&
&&
&&
&&
b
b
b
b
!=
==
!=
==
null)
null)
null)
null)
% java CollisionSystem 100
initialize PQ with
collision events and
redraw event
get next event
for(int i = 0; i < N; i++)
particles[i].move(event.time - t);
t = event.time;
if
(a !=
else if (a !=
else if (a ==
else if (a ==
Particle collision simulation example 1
a.bounceOff(b);
a.bounceOffVerticalWall()
b.bounceOffHorizontalWall();
redraw();
update positions
and time
process event
predict new events
based on changes
predict(a);
predict(b);
}
}
65
Particle collision simulation example 2
66
Particle collision simulation example 3
% java CollisionSystem < billiards.txt
% java CollisionSystem < brownian.txt
67
68
Particle collision simulation example 4
% java CollisionSystem < diffusion.txt
69