CSE 431/531: Analysis of Algorithms
Lectures 33-34: Minimum Spanning Trees
Lecturer: Shi Li
Department of Computer Science and Engineering
University at Buffalo
Spring 2016
MoWeFr 3:00-3:50pm
Knox 110
Outline
1
Minimum Spanning Tree
2
Kruskal’s Algorithm
3
Reverse-Kruskal’s Algorithm
4
Prim’s Algorithm
Spanning Tree
Def. Given a connected graph G = (V, E), a spanning tree
T = (V, F ) of G is a sub-graph of G that is a tree including all
vertices V .
b
a
c
d
e
i
h
g
f
b
a
c
d
i
h
e
g
f
Let T = (V, F ) be a subgraph of G = (V, E). The following
statements are equivalent:
T is a spanning tree of G;
T is acyclic and connected;
T is connected and has n − 1 edges;
T is acyclic and has n − 1 edges;
T is minimally connected: removal of any edge disconnects it;
T is maximally acyclic: addition of any edge creates a cycle;
T has a unique simple path between every pair of nodes.
Minimum Spanning Tree (MST) Problem
Input: Graph G = (V, E) and edge weights w : E → R
Output: the spanning tree T of G with the minimum total
weight
5
b
6
8
11
a
12
e
2
d
7
c
Recall: Steps for Designing Greedy Algorithms
Design a greedy strategy
Show that the greedy strategy is safe (that is, the strategy
does not destroy the optimality)
Show that the residual instance is also an instance for the
problem
Two classic greedy algorithms for MST
Kruskal’s Algorithm
Prim’s Algorithm
Outline
1
Minimum Spanning Tree
2
Kruskal’s Algorithm
3
Reverse-Kruskal’s Algorithm
4
Prim’s Algorithm
8
b
13
d
2
5
11
a
c
9
4
i
7
14
e
6
12
10
h
1
g
3
f
Q: Which edge can be safely included in the MST?
A: The edge with the smallest weight (lightest edge).
It Is Safe to Include the Lightest Edge
Take a spanning tree T
Assume the lightest edge e∗ is not in T
There is a unique path in T connecting u and v
Remove any edge e in the path to obtain another spanning
tree T 0
w(e∗ ) ≤ w(e) =⇒ w(T 0 ) ≤ w(T )
lightest edge e∗
v
u
Is the Residual Problem Still a MST Problem?
8
b
13
d
2
5
11
a
c
9
4
i
7
14
12
h
e
6
g∗
1
10
g
3
f
Residual problem: find the minimum spanning tree that
contains edge (g, h)
Contract the edge (g, h)
Residual problem: find the minimum spanning tree in the
contracted graph
Contraction of an Edge (u, v)
8
b
13
d
2
5
11
a
c
9
4
i
7
14
12
h
e
6
g∗
1
10
g
3
f
Remove u and v from the graph, and add a new vertex u∗
Remove all edges parallel to (u, v) from E
For every edge (u, w) ∈ E, w 6= v, change it to (u, u∗ )
For every edge (v, w) ∈ E, w 6= u, change it to (v, u∗ )
May create parallel edges! E.g. : two edges (i, g ∗ )
Greedy Algorithm
Repeat the following step until G contains only one vertex:
1
Choose the lightest edge e∗ , add e∗ to the spanning tree
2
Contract e∗ and let G be the contracted graph
What edges are removed due to contractions?
Edge (u, v) is removed if and only if there is a path
connecting u and v formed by the edges we contracted
Greedy Algorithm
Greedy-MST(G, w)
1
2
3
4
5
6
F = ∅;
sort edges in E in non-decreasing order of weights w
for each edge (u, v) in the order
if u and v are not connected by a path of edges in F
F = F ∪ {(u, v)};
return (V, F ).
Kruskal’s Algorithm: Efficient Implementation of
Greedy Algorithm
MST-Kruskal(G, w)
1
2
3
4
5
6
7
8
9
10
A←∅
S ← {{v} : v ∈ V }
sort the edges of E in non-decreasing order of weights w
for each edge (u, v) ∈ E in the order
Su ← the set in S containing u
Sv ← the set in S containing v
if Su 6= Sv
A ← A ∪ {(u, v)}
S ← S \ {Su } \ {Sv } ∪ {Su ∪ Sv }
return (V, A)
Kruskal’s Algorithm: Example
8
b
13
d
2
5
11
a
c
9
4
i
7
14
e
6
12
10
h
1
Sets: {a, b, c, i, f, g, h, d, e}
g
3
f
Running Time of Kruskal’s Algorithm
MST-Kruskal(G, w)
1
2
3
4
5
6
7
8
9
10
A←∅
S ← {{v} : v ∈ V }
sort the edges of E in non-decreasing order of weights w
for each edge (u, v) ∈ E in the order
Su ← the set in S containing u
Sv ← the set in S containing v
if Su 6= Sv
A ← A ∪ {(u, v)}
S ← S \ {Su } \ {Sv } ∪ {Su ∪ Sv }
return (V, A)
Use union-find data structure to support 2 , 5 , 6 , 7 , 9 .
Union-Find Data Structure
V : ground set
We need to maintain a partition of V and support following
operations:
Check if u and v are in the same set of the partition
Merge two sets in partition
U = {1, 2, 3, · · · , 16}
Partition:
{2, 3, 5, 9, 10, 12, 15}, {1, 7, 13, 16}, {4, 8, 11}, {6, 14}
10
1
2
12
8
7
3
15
4
16
9
6
11
13
5
par[i]: parent of i, (par[i] = nil if i is a root).
14
Union-Find Data Structure
10
1
2
12
8
7
3
15
4
16
9
6
11
14
13
5
How can we check if u and v are in the same set?
check if root(u) = root(v).
root(u): the root of the tree containing u
merge the trees with root r and r0 : par[r] ← r0 .
Union-Find Data Structure
root(v)
root(v)
1
1
2
3
4
if par[v] = nil then
return v
else
return root(par[v])
2
3
4
5
if par[v] = nil then
return v
else
par[v] ← root(par[v])
return par[v]
Problem: the tree might too deep.
Improvement: everything in the path directly points to the
root
Union-Find Data Structure
root(v)
1
2
3
4
5
if par[v] = nil then
return v
else
par[v] ← root(par[v])
return par[v]
10
5
1
2
12
8
7
3
15
4
16
9
13
6
11
14
Running Time of Kruskal’s Algorithm
MST-Kruskal(G, w)
1
2
3
4
5
6
7
8
9
10
A←∅
S ← {{v} : v ∈ V }
sort the edges of E in non-decreasing order of weights w
for each edge (u, v) ∈ E in the order
Su ← the set in S containing u
Sv ← the set in S containing v
if Su 6= Sv
A ← A ∪ {(u, v)}
S ← S \ {Su } \ {Sv } ∪ {Su ∪ Sv }
return (V, A)
2 , 5 , 6 , 7 , 9 takes time O(mα(n))
α(n) is very slow-growing: α(n) ≤ 4 for n ≤ 1080 .
Assumption Assume all edge weights are different.
Lemma An edge e ∈ E is not in the MST, if and only if there is
cycle C in G in which e is the heaviest edge.
8
b
13
d
2
5
11
a
c
9
4
i
7
14
e
6
12
10
h
1
g
3
f
(i, g) is not in the MST because of cycle (i, c, f, g)
(e, f ) is in the MST because no such cycle exists
Outline
1
Minimum Spanning Tree
2
Kruskal’s Algorithm
3
Reverse-Kruskal’s Algorithm
4
Prim’s Algorithm
Two Methods for Building a MST
1
Start from F ← ∅, and add edges to F one by one until we
obtain a spanning tree
2
Start from F ← E, and remove edges from F one by one
until we obtain a spanning tree
8
b
c
d
2
5
a
9
4
i
7
e
6
10
h
1
g
3
f
Q: Which edge can be safely excluded from the MST?
A: The heaviest non-bridge edge.
Def. A bridge is an edge whose removal disconnects the graph.
It Is Safe to Exclude the Heaviest Non-Bridge Edge
e∗ : heavist non-bridege edge
Let e∗ be the heaviest non-bridge edge
There is a path connecting two end points of e∗
All edges in the path are non-bridge paths
All edges in the path have weight at most w(e∗ )
e∗ is the heaviest edge of in a cycle
Thus, it is safe to exclude e∗ from the minimum spanning
tree, by Kruskal’s algorithm
Reverse Kruskal’s Algorithm
MST-Greedy(G, w)
1
2
3
4
5
6
F ←E
sort E in non-increasing order of weights
for every e in this order
if (V, F \ {e}) is connected then
F ← F \ {e}
return (V, F )
Reverse Kruskal’s Algorithm: Example
b
8
c
d
2
5
a
9
4
i
e
10
h
1
g
3
f
Outline
1
Minimum Spanning Tree
2
Kruskal’s Algorithm
3
Reverse-Kruskal’s Algorithm
4
Prim’s Algorithm
Design Greedy Strategy for MST
Recall the greedy strategy for Kruskal’s algorithm: choose
the edge with the smallest weight.
8
b
13
d
2
5
11
a
c
9
4
i
7
14
e
6
12
10
h
1
g
3
f
Greedy strategy for Prim’s algorithm: choose the lightest
edge incident to a.
It Is Safe to Choose the Lightest Edge Incident to a
lightest edge e∗ incident to a
C
a
Let T be a spanning tree
Consider all components obtained by removing a from T
Let e∗ be the lightest edge incident to a and e∗ connects a to
component C
Let e be the edge in T connecting a to C
T 0 = T \ e ∪ {e∗ } is a spanning tree with w(T 0 ) ≤ w(T )
Prim’s Algorithm: Example
8
b
13
d
2
5
11
a
c
9
4
i
7
14
e
6
12
10
h
1
g
3
f
Greedy Algorithm
Greedy-MST(G, w)
1
2
3
4
5
6
7
S ← {s}, where s is arbitrary vertex in V
F ←∅
while S 6= V
(u, v) ← lightest edge between S and V \ S,
assume u ∈ S and v ∈ V \ S
S ← S ∪ {v}
F ← F ∪ {(u, v)}
return (V, F )
Running time of naive implementation of the algorithm =
O(nm)
Prim’s Algorithm: Efficient Implementation of
Greedy Algorithm
For every v ∈ V \ S maintain
d(v) = minu∈S:(u,v)∈E w(u, v)
π(v) = arg minu∈S:(u,v)∈E w(u, v)
In every iteration
Pick u ∈ V \ S with the smallest d(u)
Add (π(u), u) to E
Add u to S, update d and π values.
Use a priority queue to support the operations
extract-min
update
Prim’s Algorithm
MST-Prim(G, w)
1
2
3
4
5
6
7
8
9
10
s ← arbitrary vertex of G
S ← ∅, d(s) ← 0 and d(v) ← ∞ for every v ∈ V \ {s}
while S 6= V do
u ← vertex in V \ S with the minimum d(u)
add u to S
for each v ∈ V \ S such that (u, v) ∈ E
if w(u, v) < d(v) then
d(v) ← w(u, v)
π(v) ← u
return (u, π(u))|u 6= s
Recap: Dijkstra’s Algorithm
Dijkstra(G, w, s)
1
2
3
4
5
6
7
8
9
10
S ← ∅, d(s) ← 0 and d(v) ← ∞ for every v ∈ V \ {s}
while S 6= V do
u ← vertex in V \ S with the minimum d(u)
add u to S
for each v ∈ V \ S such that (u, v) ∈ E
if d(u) + w(u, v) < d(v) then
d(v) ← d(u) + w(u, v)
π(v) ← u
return (u, π(u))|u 6= s
Prim’s Algorithm using Priority Queue
MST-Prim(G, w)
1
2
3
4
5
6
7
8
9
10
s ← arbitrary vertex of G
d(s) ← 0 and d(v) ← ∞ for every v ∈ V \ {s}
Q ← priority queue for V with d being key values
while Q is not empty do
u ←extract-min(Q)
for each v ∈ Q such that (u, v) ∈ E
if w(u, v) < d(v) then
d(v) ← w(u, v), update(Q, v)
π(v) ← u
return (u, π(u))|u 6= s
Running Time of Prim’s Algorithm Using Priority
Queue
O(n)× (time for extract-min) + O(m)× (time for update)
Priority-Queue extract-Min update
time
Heap
O(log n)
O(log n)
O(m log n)
Fibonacci Heap
O(log n)
O(1)
O(n log n + m)
Example
b
8
6
f
5
a
3
d
7
2
e
1
c
4
Assumption Assume all edge weights are different.
Lemma (u, v) is in MST, if and only if there exists a cut
(U, V \ U ), such that (u, v) is the lightest edge between U and
V \ U.
8
b
13
d
2
5
11
a
c
9
4
i
7
14
e
6
12
10
h
1
g
3
f
(c, f ) is in MST because of cut {a, b, c, i}, V \ {a, b, c, i}
(i, g) is not in MST because no such cut exists
“Evidence” for e ∈ MST or e ∈
/ MST
Assumption Assume all edge weights are different.
e ∈ MST ↔ there is a cut in which e is the lightest edge
e∈
/ MST ↔ there is a cycle in which e is the heaviest edge
Exactly one of the following is true:
there is a cut in which e is the lightest edge
there is a cycle in which e is the heaviest edge
Thus, the minimum spanning tree is unique with assumption.