22c:31 Algorithms
• Minimum-cost Spanning Tree (MST)
1
Minimum-Cost Spanning Tree
•
•
•
•
weighted connected undirected graph
spanning tree
cost of spanning tree is sum of edge costs
find spanning tree that has minimum cost
Example
8
1
7
2
2
3
4
9
4
10
12
5
6
6
14
7
3
8
• The graph has 10 edges.
• Spanning tree has only n - 1 = 7 edges.
• Need to either select 7 edges or discard 3.
We are interested in:
Finding a tree T that contains all the vertices
of a graph G
spanning tree
and has the least total weight over all
such trees
minimum-spanning tree (MST)
w(T ) 
 w((v, u))
( v ,u )T
The Crucial Fact about MST
e
V
1
min-weight
“bridge“
edge
Crucial Fact
V
2
The Crucial Fact about MST
The basis of all algorithms
Proposition: Let G = (V,E) be a weighted graph,
and let V1 and V2 be a partiotion of V (that is, V1
and V2 are two disjoint nonempty subsets and V =
V1 U V2 ).
Furthermore, let e be an edge with minimum weight
from among those with one vertex in V1 and the
other in V2.
There is a minimum spanning tree T that has e as
one of its edges.
Justification: If T doesn’t use e but use a to
connect V1 and V2, then T’ = T U { e} – { a } is
another spanning tree lighter than T.
a
e
V
1
min-weight
“bridge“
edge
Crucial Fact
V
2
Edge Selection Greedy Strategies
• Start with an n-vertex 0-edge forest.
• Consider edges in ascending order of cost. Select
edge if it does not form a cycle with already
selected edges.
 Kruskal’s method.
• Start with one vertex (a tree of a single vertex) and
grow it into an n-vertex tree by repeatedly adding
a vertex and an edge. When there is a choice, add
a least cost edge.
 Prim’s method.
Edge Rejection Greedy Strategies
• Start with the connected graph. Repeatedly find a
cycle and eliminate the highest cost edge on this
cycle. Stop when no cycles remain.
• Consider edges in descending order of cost.
Eliminate an edge provided this leaves behind a
connected graph.
Kruskal’s Method
8
1
7
2
2
3
4
9
4
10
5
12 6
6
14
7
3
1
3
5
7
8
2
4
6
8
• Start with a forest that has no edges.
• Consider edges in ascending order of cost.
• Edge (1,2) is considered first and added to the forest.
Kruskal’s Method
8
1
7
2
2
3
4
9
4
10
5
12 6
6
14
7
3
8
1
7
2
2
3
5
6
7
3
4
6
8
4
• Edge (7,8) is considered next and added.
• Edge (3,4) is considered next and added.
• Edge (5,6) is considered next and added.
• Edge (2,3) is considered next and added.
• Edge (1,3) is considered next and rejected because it
creates a cycle.
Kruskal’s Method
8
1
7
2
2
3
4
9
4
10
5
12 6
6
14
7
3
8
1
3
7
2
4
2
4
10
5
6
6
• Edge (2,4) is considered next and rejected because it
creates a cycle.
• Edge (3,5) is considered next and added.
• Edge (3,6) is considered next and rejected.
• Edge (5,7) is considered next and added.
14
7
3
8
Kruskal’s Method
8
1
7
2
2
3
4
9
4
10
5
12 6
6
14
7
3
8
1
3
7
2
2
4
4
10
5
6
6
• n - 1 edges have been selected and no cycle
formed.
• So we must have a spanning tree.
• Cost is 46.
• Minimum-cost spanning tree is unique when all
edge costs are different.
14
7
3
8
Prim’s Method
8
1
7
2
2
3
4
9
4
10
5
12 6
6
14
7
3
8
1
3
7
2
2
4
4
10
5
6
6
14
7
3
8
• Start with any vertex (a 1-vertex tree).
• Get a 2-vertex tree by adding a cheapest edge.
• Get a 3-vertex tree by adding a cheapest edge.
• Grow the tree one edge at a time until the tree has n - 1 edges
(and hence has all n vertices).
Greedy Minimum-Cost Spanning Tree Methods
• Can prove that all result in a minimum-cost
spanning tree. Let n = |V| and m = |E|.
• Kruskal’s uses union-find trees to run in
– O(m log m) time (for sorting edges)
• Prim’s method:
 O(n2) using adjacency matrix.
 O((n + m) log n) using adjacency list and a heap.
Pseudocode For Kruskal’s Method
Let G = (V, E), n = |V|.
Start with an empty set T of edges.
while (E is not empty && |T| != n-1)
{
Let (u,v) be a least-cost edge in E.
E = E - {(u,v)}. // delete edge from E
if ((u,v) does not create a cycle in T)
Add edge (u,v) to T.
}
if ( |T| == n-1) T is a mininum-cost spanning tree.
else G has no spanning tree.
Data Structures For Kruskal’s Method
Edge set E.
Operations are:
 Is E empty?
 Select and remove a least-cost edge.
Use a min heap of edges.
 Initialize. O(m) time.
 Remove and return m least-cost edges: O(m log m) time
We may sort E first and select edges one by one from
left to right. O(m log m) time
Data Structures For Kruskal’s Method
Set of selected edges T.
Operations are:
 Does T have n - 1 edges?
 Does the addition of an edge (u, v) to T result in a cycle?
 Add an edge to T.
Data Structures For Kruskal’s Method
Use an array linear list for the edges of T.
 Does T have n - 1 edges?
• Check size of linear list. O(1) time.
 Does the addition of an edge (u, v) to T result in a
cycle?
• Not easy.
 Add an edge to T.
• Add at right end of linear list. O(1) time.
Data Structures For Kruskal’s Method
Does the addition of an edge (u, v) to T result in a
cycle?
1
7
2
2
3
5
6
7
3
4
6
8
4
• Each component of T is a tree.
• When u and v are in the same component, the addition of the
edge (u,v) creates a cycle.
• When u and v are in the different components, the addition of
the edge (u,v) does not create a cycle.
Data Structures For Kruskal’s Method
1
7
2
2
3
5
6
7
3
4
6
8
4
• Each component of T is defined by the vertices in the
component.
• Represent each component as a set of vertices.
 {1, 2, 3, 4}, {5, 6}, {7, 8}
• Two vertices are in the same component iff they are in the
same set of vertices.
Data Structures For Kruskal’s Method
1
7
2
2
3
5
6
7
3
4
6
8
4
• When an edge (u, v) is added to T, the two
components that have vertices u and v combine to
become a single component.
• In our set representation of components, the set that
has vertex u and the set that has vertex v are united.
 {1, 2, 3, 4} U {5, 6} => {1, 2, 3, 4, 5, 6}
Data Structures For Kruskal’s Method
• Initially, T is empty.
1
3
5
7
2
4
6
8
• Initial sets are:
 {1} {2} {3} {4} {5} {6} {7} {8}
• Does the addition of an edge (u, v) to T result in a cycle? If
not, add edge to T. Each set has a name and let find(x) return
the name of the set containing vertex x.
s1 = find(u); s2 = find(v);
if (s1 != s2) union(s1, s2);
Data Structures For Kruskal’s Method
• Use FastUnionFind: each of union and find takes practically
constant time.
• Initialize.
 O(n) time.
• At most 2m finds and n-1 unions.
 Very close to O(n + m).
• Min heap operations to get edges in increasing order of cost
take O(m log m).
• Overall complexity of Kruskal’s method is O(n + m log m).
Prim‘s Algorithm
1. All vertices are marked as not visited
2. Any vertex v you like is chosen as starting vertex and
is marked as visited (define a cluster C)
3. The smallest-weighted edge e = (v,u), which connects
one vertex v inside the cluster C with another vertex u outside
of C, is chosen and is added to the MST and u is added into C.
4. The process is repeated until a spanning tree is formed
Prim’s MST Algorithm
MST-Prim(G,w,r)
; Graph G, weights w, root r
Q  V[G]
for each vertex u  Q do key[u]   ; infinite “distance”
key[r]  0
P[r]  NIL
while Q<>NIL do
u  Extract-Min(Q)
; remove closest node
; Update children of u so they have a parent and a min key val
; the key is the weight between node and parent
for each v Adj[u] do
if v Q & w(u,v)<key[v] then
P[v]  u
key[v]  w(u,v)
Prim’s algorithm: key[v] := min(key[v], w(u,v))
Dijkstra’s algorithm: key[v] := min(key[v], key[u]+w(u,v))
Prim’s Example
Example: Graph given earlier.
Q={ (e,0) (a,  ) (b,  ) (c,  ) (d,  ) (f,  ) (g,  ) (h,  ) }
inf
a
6
inf
4
inf
9
5
b
inf
c
14
d
2
inf
10
e
0/nil
f
3
h
8
inf
g
15
inf
Extract min, vertex e. Update neighbor if in Q and weight < key.
Prim’s Example
inf
a
6
14/e
4
inf
9
5
b
inf
c
14
d
2
inf
10
e
0/nil
f
3
h
8
inf
g
15
3/e
Q={ (a,  ) (b,14) (c,  ) (d,  ) (f,  ) (g,  ) (h,3) }
Extract min, vertex h. Update neighbor if in Q and weight < key
Prim’s Algorithm
inf
10/h
a
6
4
inf
9
5
b
inf
c
14
d
2
inf
10
e
0/nil
f
3
h
8
8/h
g
15
3/e
Q={ (a,  ) (b,10) (c,  ) (d,  ) (f,8) (g,  ) }
Extract min, vertex f. Update neighbor if in Q and weight < key
Prim’s Algorithm
inf
10/h
a
6
4
2/f
5
b
9
c
14
d
2
15/f
10
e
0/nil
f
3
h
8
inf
8/h
g
15
3/e
Q={ (a,  ) (b,10) (c, 2) (d,  ) (g,15) }
Extract min, vertex c. Update neighbor if in Q and weight < key
4/c
5/c
Prim’s Algorithm
a
6
4
2/f
5
b
9
c
14
d
2
15/f
10
e
0/nil
f
3
h
8
9/c
8/h
g
15
3/e
Q={ (a,4) (b,5) (d,9) (g,15) }
Extract min, vertex a. No keys are smaller than edges from a (4>2 on edge ac, 6>5 on edge ab) so nothing
done.
Q={ (b,5) (d,9) (g,15) }
Extract min, vertex b.
Same case, no keys are smaller than edges, so nothing is done.
Same for extracting d and g, and we are done.
Prim’s Algorithm
Get spanning tree by connecting nodes with their parents:
4/c
a
6
5/c
4
2/f
9
5
b
9/c
c
14
d
2
15/f
10
e
0/nil
f
3
h
3/e
8
8/h
g
15
Runtime for Prim’s Algorithm
MST-Prim(G,w,r)
; Graph G, weights w, root r
Q  V[G]
for each vertex u  Q do key[u]  
; infinite “distance”
key[r]  0
P[r]  NIL
while Q<>NIL do
u  Extract-Min(Q)
; remove closest node
; Update children of u so they have a parent and a min key val
; the key is the weight between node and parent
for each v Adj[u] do
if v Q & w(u,v)<key[v] then
P[v]  u
key[v]  w(u,v)
O(V)
O(V) if using a heap
O(lgV) if using a heap
O(E) over entire while(Q<>NIL) loop
O(lgV) to update if using a heap!
The inner loop takes O(E lg V) for the heap update inside the O(E) loop.
This is over all executions, so it is not multiplied by O(V) for the while loop
(this is included in the O(E) runtime through all edges.
The Extract-Min requires O(V lg V) time. O(lg V) for the Extract-Min and O(V) for the while loop.
Total runtime is then O(V lg V) + O(E lg V) which is O(E lg V) in a connected graph
(a connected graph will always have at least V-1 edges).