CS 332: Algorithms
Topological Sort
Minimum Spanning Trees
David Luebke
1
7/1/2016
Directed Acyclic Graphs
● A directed acyclic graph or DAG is a directed
graph with no directed cycles:
David Luebke
2
7/1/2016
DFS and DAGs
● Argue that a directed graph G is acyclic iff a
DFS of G yields no back edges:
■ Forward: if G is acyclic, will be no back edges
○ Trivial: a back edge implies a cycle
■ Backward: if no back edges, G is acyclic
○ Argue contrapositive: G has a cycle   a back edge
Let v be the vertex on the cycle first discovered, and u be the
predecessor of v on the cycle
 When v discovered, whole cycle is white
 Must visit everything reachable from v before returning from
DFS-Visit()
 So path from uv is yellowyellow, thus (u, v) is a back edge

David Luebke
3
7/1/2016
Topological Sort
● Topological sort of a DAG:
■ Linear ordering of all vertices in graph G such that
vertex u comes before vertex v if edge (u, v)  G
● Real-world example: getting dressed
David Luebke
4
7/1/2016
Getting Dressed
Underwear
Socks
Watch
Pants
Shoes
Shirt
Belt
Tie
Jacket
David Luebke
5
7/1/2016
Getting Dressed
Underwear
Socks
Watch
Pants
Shoes
Shirt
Belt
Tie
Jacket
Socks
Underwear
David Luebke
Pants
Shoes
Watch
6
Shirt
Belt
Tie
Jacket
7/1/2016
Topological Sort Algorithm
Topological-Sort()
{
Run DFS
When a vertex is finished, output it
Vertices are output in reverse
topological order
}
● Time: O(V+E)
● Correctness: Want to prove that
(u,v)  G  uf > vf
David Luebke
7
7/1/2016
Correctness of Topological Sort
● Claim: (u,v)  G  uf > vf
■ When (u,v) is explored, u is yellow
○ v = yellow  (u,v) is back edge. Contradiction (Why?)
○ v = white  v becomes descendent of u  vf < uf
(since must finish v before backtracking and finishing u)
○ v = black  v already finished  vf < uf
David Luebke
8
7/1/2016
Minimum Spanning Tree
● Problem: given a connected, undirected,
weighted graph:
6
4
5
9
14
2
10
15
3
David Luebke
8
9
7/1/2016
Minimum Spanning Tree
● Problem: given a connected, undirected,
weighted graph, find a spanning tree using
edges that minimize the total weight
6
4
5
9
14
2
10
15
3
David Luebke
8
10
7/1/2016
Minimum Spanning Tree
● Which edges form the minimum spanning tree
(MST) of the below graph?
A
6
4
5
H
9
B
14
C
2
10
G
E
3
David Luebke
F
15
D
8
11
7/1/2016
Minimum Spanning Tree
● Answer:
A
6
4
5
H
9
B
14
C
2
10
G
E
3
David Luebke
F
15
D
8
12
7/1/2016
Minimum Spanning Tree
● MSTs satisfy the optimal substructure property: an
optimal tree is composed of optimal subtrees
■ Let T be an MST of G with an edge (u,v) in the middle
■ Removing (u,v) partitions T into two trees T1 and T2
■ Claim: T1 is an MST of G1 = (V1,E1), and T2 is an MST of
G2 = (V2,E2)
(Do V1 and V2 share vertices? Why?)
■ Proof: w(T) = w(u,v) + w(T1) + w(T2)
(There can’t be a better tree than T1 or T2, or T would be
suboptimal)
David Luebke
13
7/1/2016
Minimum Spanning Tree
● Thm:
■ Let T be MST of G, and let A  T be subtree of T
■ Let (u,v) be min-weight edge connecting A to V-A
■ Then (u,v)  T
David Luebke
14
7/1/2016
Minimum Spanning Tree
● Thm:
■ Let T be MST of G, and let A  T be subtree of T
■ Let (u,v) be min-weight edge connecting A to V-A
■ Then (u,v)  T
● Proof: in book (see Thm 24.1)
David Luebke
15
7/1/2016
Prim’s Algorithm
MST-Prim(G, w, r)
Q = V[G];
for each u  Q
key[u] = ;
key[r] = 0;
p[r] = NULL;
while (Q not empty)
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
David Luebke
16
7/1/2016
Prim’s Algorithm
MST-Prim(G, w, r)
6
4
9
Q = V[G];
5
for each u  Q
14
2
key[u] = ;
10
15
key[r] = 0;
p[r] = NULL;
3
8
while (Q not empty)
Run on example graph
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
David Luebke
17
7/1/2016
Prim’s Algorithm

MST-Prim(G, w, r)
6
4
9
Q = V[G];
5


for each u  Q
14
2
key[u] = ;
10
15
key[r] = 0;


p[r] = NULL;
3
8

while (Q not empty)
Run on example graph
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
David Luebke
18


7/1/2016
Prim’s Algorithm

MST-Prim(G, w, r)
6
4
9
Q = V[G];
5


for each u  Q
14
2
key[u] = ;
10
15
key[r] = 0;
0
r

p[r] = NULL;
3
8

while (Q not empty)
Pick a start vertex r
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
David Luebke
19


7/1/2016
Prim’s Algorithm

MST-Prim(G, w, r)
6
4
9
Q = V[G];
5



for each u  Q
14
2
key[u] = ;
10
15
key[r] = 0;
0
u


p[r] = NULL;
3
8

while (Q not empty)
u = ExtractMin(Q); Red vertices have been removed from Q
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
David Luebke
20
7/1/2016
Prim’s Algorithm

MST-Prim(G, w, r)
6
4
9
Q = V[G];
5



for each u  Q
14
2
key[u] = ;
10
15
key[r] = 0;
0
u


p[r] = NULL;
3
8
3
while (Q not empty)
u = ExtractMin(Q); Red arrows indicate parent pointers
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
David Luebke
21
7/1/2016
Prim’s Algorithm

MST-Prim(G, w, r)
6
4
Q = V[G];
5
14

for each u  Q
14
2
key[u] = ;
10
key[r] = 0;
0
u

p[r] = NULL;
3
8
3
while (Q not empty)
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
David Luebke
22
9

15

7/1/2016
Prim’s Algorithm

MST-Prim(G, w, r)
6
4
Q = V[G];
5
14

for each u  Q
14
2
key[u] = ;
10
key[r] = 0;
0

p[r] = NULL;
3
8
3
while (Q not empty)
u
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
David Luebke
23
9

15

7/1/2016
Prim’s Algorithm

MST-Prim(G, w, r)
6
4
Q = V[G];
5
14

for each u  Q
14
2
key[u] = ;
10
key[r] = 0;
0
8
p[r] = NULL;
3
8
3
while (Q not empty)
u
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
David Luebke
24
9

15

7/1/2016
Prim’s Algorithm

MST-Prim(G, w, r)
6
4
Q = V[G];
5
10

for each u  Q
14
2
key[u] = ;
10
key[r] = 0;
0
8
p[r] = NULL;
3
8
3
while (Q not empty)
u
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
David Luebke
25
9

15

7/1/2016
Prim’s Algorithm

MST-Prim(G, w, r)
6
4
Q = V[G];
5
10

for each u  Q
14
2
key[u] = ;
10
key[r] = 0;
0
8
p[r] = NULL;
3
8
3
while (Q not empty)
u
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
David Luebke
26
9

15

7/1/2016
Prim’s Algorithm

MST-Prim(G, w, r)
6
4
Q = V[G];
5
10
2
for each u  Q
14
2
key[u] = ;
10
key[r] = 0;
0
8
p[r] = NULL;
3
8
3
while (Q not empty)
u
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
David Luebke
27
9

15

7/1/2016
Prim’s Algorithm

MST-Prim(G, w, r)
6
4
Q = V[G];
5
10
2
for each u  Q
14
2
key[u] = ;
10
key[r] = 0;
0
8
p[r] = NULL;
3
8
3
while (Q not empty)
u
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
David Luebke
28
9
15

15
7/1/2016
Prim’s Algorithm
u

MST-Prim(G, w, r)
6
4
9
Q = V[G];
5
10
2
for each u  Q
14
2
key[u] = ;
10
15
key[r] = 0;
0
8
p[r] = NULL;
3
8
3
while (Q not empty)
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
David Luebke
29

15
7/1/2016
Prim’s Algorithm
u

MST-Prim(G, w, r)
6
4
9
Q = V[G];
5
10
2
for each u  Q
14
2
key[u] = ;
10
15
key[r] = 0;
0
8
p[r] = NULL;
3
8
3
while (Q not empty)
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
David Luebke
30
9
15
7/1/2016
Prim’s Algorithm
u
4
MST-Prim(G, w, r)
6
4
9
Q = V[G];
5
10
2
for each u  Q
14
2
key[u] = ;
10
15
key[r] = 0;
0
8
p[r] = NULL;
3
8
3
while (Q not empty)
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
David Luebke
31
9
15
7/1/2016
Prim’s Algorithm
u
4
MST-Prim(G, w, r)
6
4
9
Q = V[G];
5
5
2
for each u  Q
14
2
key[u] = ;
10
15
key[r] = 0;
0
8
p[r] = NULL;
3
8
3
while (Q not empty)
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
David Luebke
32
9
15
7/1/2016
Prim’s Algorithm
u
4
MST-Prim(G, w, r)
6
4
Q = V[G];
5
5
2
for each u  Q
14
2
key[u] = ;
10
key[r] = 0;
0
8
p[r] = NULL;
3
8
3
while (Q not empty)
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
David Luebke
33
9
15
9
15
7/1/2016
Prim’s Algorithm
u
4
MST-Prim(G, w, r)
6
4
Q = V[G];
5
5
2
for each u  Q
14
2
key[u] = ;
10
key[r] = 0;
0
8
p[r] = NULL;
3
8
3
while (Q not empty)
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
David Luebke
34
9
15
9
15
7/1/2016
Prim’s Algorithm
4
MST-Prim(G, w, r)
6
4
Q = V[G];
5
5
2
for each u  Q
14
2
key[u] = ;
10
key[r] = 0;
0
8
p[r] = NULL;
3
8
3
while (Q not empty)
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
David Luebke
35
u
9
15
9
15
7/1/2016
Prim’s Algorithm
4
MST-Prim(G, w, r)
6
4
Q = V[G];
5
5
2
for each u  Q
14
2
key[u] = ;
10
key[r] = 0;
0
8
p[r] = NULL;
3
8
3
while (Q not empty)
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
David Luebke
36
9
9
u
15
15
7/1/2016
Prim’s Algorithm
MST-Prim(G, w, r)
Q = V[G];
for each u  Q
key[u] = ;
key[r] = 0;
What is the hidden cost
p[r] = NULL;
while (Q not empty)
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
David Luebke
37
in this code?
7/1/2016
Prim’s Algorithm
MST-Prim(G, w, r)
Q = V[G];
for each u  Q
key[u] = ;
key[r] = 0;
p[r] = NULL;
while (Q not empty)
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
DecreaseKey(v, w(u,v));
David Luebke
38
7/1/2016
Prim’s Algorithm
MST-Prim(G, w, r)
Q = V[G];
for each u  Q
key[u] = ; How often is ExtractMin() called?
key[r] = 0;
How
often
is
DecreaseKey()
called?
p[r] = NULL;
while (Q not empty)
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
DecreaseKey(v, w(u,v));
David Luebke
39
7/1/2016
Prim’s Algorithm
MST-Prim(G, w, r)
Q = V[G];
What will be the running time?
for each u  Q
key[u] = ;
A: Depends on queue
key[r] = 0;
binary heap: O(E lg V)
p[r] = NULL;
Fibonacci heap: O(V lg V + E)
while (Q not empty)
u = ExtractMin(Q);
for each v  Adj[u]
if (v  Q and w(u,v) < key[v])
p[v] = u;
key[v] = w(u,v);
David Luebke
40
7/1/2016