MEIO/UPC-UB : NETWORK FLOWS
Network Flows
UPCOPENCOURSEWARE number 34414
Topic 5:
Minimum Cost Spanning Tree
F.-Javier Heredia
This work is licensed under the Creative Commons AttributionNonCommercial-NoDerivs 3.0 Unported License. To view a copy of this
license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/ or send a
letter to Creative Commons, 444 Castro Street, Suite 900, Mountain View,
California, 94041, USA.
•
•
•
•
Definitions
Applications
Properties
Algorithms:
– Kruskal’s algorithm.
– Prim’s algorithm.
– Sollin’s algorithm.
• Exercise.
• Source material:
– R.K. Ahuja, Th.L. Magnanti, J. Orlin “Network Flows”, chap. 13.
– J. Orlin “Network Optimization” http://ocw.mit.edu/courses/sloanschool-of-management/15-082j-network-optimization-fall-2010/
F.-Javier Heredia http://gnom.upc.edu/heredia
MCST- 2
Minimum Cost Spanning Trees
MEIO/UPC-UB : NETWORK FLOWS
5.- Minimum Cost Spanning Trees
(Chap. 13 AMO)
• Undirected network G = (N, A).
– (i, j) is the same arc as (j, i).
– We associate with each arc (i, j) ∈ A a cost cij.
• A spanning tree T of G is a connected acyclic
subgraph that spans all the nodes.
A connected graph with n nodes and n – 1 arcs
is a spanning tree.
• The minimum cost spanning tree problem is
to find a spanning tree of minimum cost.
F.-Javier Heredia http://gnom.upc.edu/heredia
MCST- 3
Minimum Cost Spanning Trees
MEIO/UPC-UB : NETWORK FLOWS
Minimum Cost Spanning Tree Problem
35
1
2
25
40
3
10
20
15
4
8
15
17
30
5
6
21
11
F.-Javier Heredia http://gnom.upc.edu/heredia
7
MCST- 4
Minimum Cost Spanning Trees
MEIO/UPC-UB : NETWORK FLOWS
A Minimum Cost Spanning Tree Problem
35
1
2
25
40
3
10
20
15
4
8
15
17
30
5
6
21
11
F.-Javier Heredia http://gnom.upc.edu/heredia
7
MCST- 5
Minimum Cost Spanning Trees
MEIO/UPC-UB : NETWORK FLOWS
A Minimum Cost Spanning Tree
• Consider a communications company, such as
AT&T or GTE that needs to build a
communication network that connects n
different users. The cost of making a link
joining i and j is cij. What is the minimum cost
of connecting all of the users?
3
5
1
4
10
8
7
2
9
6
F.-Javier Heredia http://gnom.upc.edu/heredia
MCST- 6
Minimum Cost Spanning Trees
MEIO/UPC-UB : NETWORK FLOWS
Communications Systems
• Consider a system with a number of electronic
components. In order to make two pins i and j
of different components electrically equivalent,
one can connect i and j by a wire. How can we
connect n different pins in this way to make
them electrically equivalent to each other so as
to minimize the total wire length.
2
4
3
5
1
F.-Javier Heredia http://gnom.upc.edu/heredia
MCST- 7
Minimum Cost Spanning Trees
MEIO/UPC-UB : NETWORK FLOWS
Electronic Circuitry
• Consider the traveling salesman problem of
finding a minimum cost tour linking n cities.
• One way of formulating this problem is using
minimum spanning trees.
1
F.-Javier Heredia http://gnom.upc.edu/heredia
MCST- 8
Minimum Cost Spanning Trees
MEIO/UPC-UB : NETWORK FLOWS
The Traveling Salesman Problem
•
A collection of arcs is a tour if
1. There are two arcs incident to each node
2. The red arcs (those not incident to node 1) form a spanning tree
in G\1.
3. The graph T shown is a one-tree (a tree with one cycle)
•
Tour ≡ one-tree + all nodes with degree 2
1
F.-Javier Heredia http://gnom.upc.edu/heredia
MCST- 9
Minimum Cost Spanning Trees
MEIO/UPC-UB : NETWORK FLOWS
Representing the TSP problem
• Let T* be a spanning tree. We refer to the arcs
in T* as tree arcs and to those arcs not in T*
as nontree arcs.
7
8
11
6
1
4
5
2
3
9
F.-Javier Heredia http://gnom.upc.edu/heredia
10
MCST- 10
Minimum Cost Spanning Trees
MEIO/UPC-UB : NETWORK FLOWS
Definitions
Lemma: Let T be any spanning tree. Let a be any
nontree arc. Then adding a to T creates a unique cycle
C. Deleting any arc of C creates a spanning tree T’. (left
as exercise).
7
8
11
6
1
4
5
2
C
3
9
F.-Javier Heredia http://gnom.upc.edu/heredia
10
MCST- 11
Minimum Cost Spanning Trees
MEIO/UPC-UB : NETWORK FLOWS
Lemma
• Def: If (i,j) is a tree arc, then T*-(i,j) has two components,
say with node sets S and N-S. The arcs in (S,N-S)
constitute a cut. For example, suppose that (i,j) = (4, 5).
S
N-S
7
8
11
6
1
4
5
2
3
9
F.-Javier Heredia http://gnom.upc.edu/heredia
10
MCST- 12
Minimum Cost Spanning Trees
MEIO/UPC-UB : NETWORK FLOWS
Definitions
• Def. (Cut optimality conditions): For every tree arc (i, j) ∈
T* , cij ≤ ckl for every arc (k, l) contained in the cut formed
by deleting arc (i, j) from T*.
Example: Consider the cutset induced by deleting (4,5). The cut
optimality conditions are satisfied if every arc in the cut set has length
at least that of (4,5).
7
8
11
6
1
4
5
2
3
9
F.-Javier Heredia http://gnom.upc.edu/heredia
10
MCST- 13
Minimum Cost Spanning Trees
MEIO/UPC-UB : NETWORK FLOWS
Cut Optimality Conditions
• Theorem. A spanning tree T* is a minimum spanning tree if and only if
it satisfies the cut optimality conditions.
Proof:
⇒
– Suppose first that cij > ckl for some arc (k,l) in the cutset obtained from
deleting (i,j) from T*. Then T' = T*- (i,j) + (k,l) is a spanning tree with less
cost.
7
8
11
6
1
4
5
2
3
9
F.-Javier Heredia http://gnom.upc.edu/heredia
10
MCST- 14
Minimum Cost Spanning Trees
MEIO/UPC-UB : NETWORK FLOWS
Cut Optimality Theorem
⇐
– Conversely, suppose that T* satisfies the cut optimality conditions (c.o.c.). Let
T’ be an optimal spanning tree, T’≠ T*. Let (i,j) ∈ T*, (i,j) ∉ T’ . We claim that
(i,j) ∈ T’. Suppose that this were not true.
– Let P be the path from i to j in T’. Then P has at least one arc (k,l) in the cutset
induced by deleting (i,j) from T*.
– But then T’ – (k,l) + (i,j) is also a min cost tree as cij = ckl (⇐ c.o.c.).
Repeating the process for all the arcs in T* we show that T* is also optimal ■.
7
1
k
11
8
i
6l
j
2
3
9
F.-Javier Heredia http://gnom.upc.edu/heredia
10
MCST- 15 15
Minimum Cost Spanning Trees
MEIO/UPC-UB : NETWORK FLOWS
Cut Optimality Theorem
• Def.(Path optimality conditions) : For every nontree arc
(k, l) of G, cij ≤ ckl for every arc (i, j) contained in the path of
T* connecting nodes k and l.
Example. Every arc in the path from 11 to 6 in T* has
length at most that of (11,6)
7
8
6
11
1
4
5
2
3
9
F.-Javier Heredia http://gnom.upc.edu/heredia
10
MCST- 16
Minimum Cost Spanning Trees
MEIO/UPC-UB : NETWORK FLOWS
Path Optimality Conditions
• Theorem 13.2. (Path optimality theorem) : A spanning
tree T* is a minimum spanning tree if and only if it satisfies
the path optimality conditions.
Proof:
⇒ : Suppose first that T* is optimal and the path optimality conditions
are not satisfied. Suppose cij > ckl. Then T’ = T* - (i,j) + (k,l) has
lower cost than T*.
7
k
8
l
1
i
j
2
3
9
F.-Javier Heredia http://gnom.upc.edu/heredia
10 MCST- 17
Minimum Cost Spanning Trees
MEIO/UPC-UB : NETWORK FLOWS
Path Optimality Theorem
⇐ : Observation.
Let (k,l) be an arc in A-T*. Then (i,j) is on
the cycle contained in T* + (k,l) if and only if (k,l) is in the
cutset induced by T*-(i,j).
Thus, path optimality conditions are equivalent to cut
optimality conditions ■.
S(l,j)
S(k,i)
7
l
k
8
1
i
j
2
3
S(i,j)
9
F.-Javier Heredia http://gnom.upc.edu/heredia
10
MCST- 18
Minimum Cost Spanning Trees
MEIO/UPC-UB : NETWORK FLOWS
Path Optimality Theorem
begin
order the arcs a1, . . . , am in non-decreasing order of cost
T=ø
for i = 1 to m do
if T + ai does not have a cycle then T := T + ai
end
Correctness: at every stage k, the cost of the discarded
arc (i,j) is ≥ than the costs of any arc in the path P(i→j) on
[T*]k ⇒ path optimality conditions.
F.-Javier Heredia http://gnom.upc.edu/heredia
MCST- 19
Minimum Cost Spanning Trees
MEIO/UPC-UB : NETWORK FLOWS
Kruskal’s Greedy Algorithm
35
11
22
4
4
8
66
15
25
40
10
33
20
15
30
55
21
11
F.-Javier Heredia http://gnom.upc.edu/heredia
17
77
MCST- 20
Minimum Cost Spanning Trees
MEIO/UPC-UB : NETWORK FLOWS
The Greedy Algorithm in Action
35
35
11
22
4
4
8
66
15
15
25
25
40
40
10
10
33
20
20
15
15
30
30
55
21
11
17
17
21
11
F.-Javier Heredia http://gnom.upc.edu/heredia
77
MCST- 21
Minimum Cost Spanning Trees
MEIO/UPC-UB : NETWORK FLOWS
The Greedy Algorithm in Action
35
35
11
22
4
4
8
66
15
15
25
25
40
40
10
10
33
20
20
15
15
30
30
55
21
11
17
17
21
11
F.-Javier Heredia http://gnom.upc.edu/heredia
77
MCST- 22
Minimum Cost Spanning Trees
MEIO/UPC-UB : NETWORK FLOWS
The Greedy Algorithm in Action
35
35
11
22
4
4
8
66
15
15
25
25
40
40
10
10
33
20
20
15
15
30
30
55
21
11
17
17
21
11
F.-Javier Heredia http://gnom.upc.edu/heredia
77
MCST- 23
Minimum Cost Spanning Trees
MEIO/UPC-UB : NETWORK FLOWS
The Greedy Algorithm in Action
35
35
11
22
4
4
8
66
15
15
25
25
40
40
10
10
33
20
20
15
15
30
30
55
21
11
17
17
21
11
F.-Javier Heredia http://gnom.upc.edu/heredia
77
MCST- 24
Minimum Cost Spanning Trees
MEIO/UPC-UB : NETWORK FLOWS
The Greedy Algorithm in Action
35
35
11
22
4
4
8
66
15
15
25
25
40
40
10
10
33
20
20
15
15
30
30
55
21
11
17
17
21
11
F.-Javier Heredia http://gnom.upc.edu/heredia
77
MCST- 25
Minimum Cost Spanning Trees
MEIO/UPC-UB : NETWORK FLOWS
The Greedy Algorithm in Action
35
35
11
22
4
4
8
66
15
15
25
25
40
40
10
10
33
20
20
15
15
30
30
55
21
11
17
17
21
11
F.-Javier Heredia http://gnom.upc.edu/heredia
77
MCST- 26
Minimum Cost Spanning Trees
MEIO/UPC-UB : NETWORK FLOWS
The Greedy Algorithm in Action
35
35
11
22
4
4
8
66
15
15
25
25
40
40
10
10
33
20
20
15
15
30
30
55
21
11
17
17
21
11
F.-Javier Heredia http://gnom.upc.edu/heredia
77
MCST- 27
Minimum Cost Spanning Trees
MEIO/UPC-UB : NETWORK FLOWS
The Greedy Algorithm in Action
35
35
11
22
4
4
8
66
15
15
25
25
40
40
10
10
33
20
20
15
15
30
30
55
21
11
17
17
21
11
F.-Javier Heredia http://gnom.upc.edu/heredia
77
MCST- 28
Minimum Cost Spanning Trees
MEIO/UPC-UB : NETWORK FLOWS
The Greedy Algorithm in Action
35
35
11
22
4
4
8
66
15
15
25
25
40
40
10
10
33
20
20
15
15
30
30
55
21
11
17
17
21
11
F.-Javier Heredia http://gnom.upc.edu/heredia
77
MCST- 29
Minimum Cost Spanning Trees
MEIO/UPC-UB : NETWORK FLOWS
The Greedy Algorithm in Action
35
35
11
22
4
4
8
66
15
15
25
25
40
40
10
10
33
20
20
15
15
30
30
55
21
11
17
17
21
11
F.-Javier Heredia http://gnom.upc.edu/heredia
77
MCST- 30
Minimum Cost Spanning Trees
MEIO/UPC-UB : NETWORK FLOWS
The Greedy Algorithm in Action
35
35
11
22
4
4
8
66
15
15
25
25
40
40
10
10
33
20
20
15
15
30
30
55
21
11
17
17
21
11
F.-Javier Heredia http://gnom.upc.edu/heredia
77
MCST- 31
Minimum Cost Spanning Trees
MEIO/UPC-UB : NETWORK FLOWS
The Greedy Algorithm in Action
35
35
11
22
4
4
8
66
15
15
25
25
40
40
10
10
33
20
20
15
15
30
30
55
21
11
17
17
21
11
F.-Javier Heredia http://gnom.upc.edu/heredia
77
MCST- 32
Minimum Cost Spanning Trees
MEIO/UPC-UB : NETWORK FLOWS
The Greedy Algorithm in Action
• Step 1. Sort the arc costs.
O(m log n)
• Repeated Step: determine if T+ ai has a cycle?
When arc (j,k) is considered, we want to know if j and k
are in the same connected component C. If they are
not, they result in two components, C and D, being
merged.
• Next: a very simple way of keeping track of
components.
F.-Javier Heredia http://gnom.upc.edu/heredia
MCST- 33
Minimum Cost Spanning Trees
MEIO/UPC-UB : NETWORK FLOWS
Running Time Analysis
• Store each component C as set.
– Let First(C) = first node in the component of C.
• For each element j in a component C, let
– First(j) = First(C) = first node of C.
• Remark: adding (i,j) to a forest F creates a cycle
if and only if (i,j) are in the same component; i.e.,
– First(i) = First(j).
• When merging components C and D, put the
larger component first. If |C| > |D|,
– First(C∪D) := First(C).
F.-Javier Heredia http://gnom.upc.edu/heredia
MCST- 34
Minimum Cost Spanning Trees
MEIO/UPC-UB : NETWORK FLOWS
A simple, yet fairly efficient data structure
Node
1
2
3
4
5
6
7
First
1
2
3
4
5
4
7
35
35
11
10
10
22
66
15
15
25
25
40
40
4
4
8
33
20
20
15
15
30
30
55
21
11
17
17
21
11
77
root node
F.-Javier Heredia http://gnom.upc.edu/heredia
MCST- 35
Minimum Cost Spanning Trees
MEIO/UPC-UB : NETWORK FLOWS
The Greedy Algorithm with merging
Node
1
2
3
4
5
6
7
First
1
4
3
4
5
4
7
35
35
11
22
4
4
8
66
15
15
25
25
40
40
10
10
33
20
20
15
15
30
30
55
21
11
17
17
21
11
F.-Javier Heredia http://gnom.upc.edu/heredia
77
MCST- 36
Minimum Cost Spanning Trees
MEIO/UPC-UB : NETWORK FLOWS
The Greedy Algorithm with merging
Node
1
2
3
4
5
6
7
First
1
4
3
4
5
4
5
35
35
11
22
4
4
8
66
15
15
25
25
40
40
10
10
33
20
20
15
15
30
30
55
21
11
17
17
21
11
F.-Javier Heredia http://gnom.upc.edu/heredia
77
MCST- 37
Minimum Cost Spanning Trees
MEIO/UPC-UB : NETWORK FLOWS
The Greedy Algorithm with merging
Node
1
2
3
4
5
6
7
First
1
4
5
4
5
4
5
35
35
11
22
4
4
8
66
15
15
25
25
40
40
10
10
33
20
20
15
15
30
30
55
21
11
17
17
21
11
F.-Javier Heredia http://gnom.upc.edu/heredia
77
MCST- 38
Minimum Cost Spanning Trees
MEIO/UPC-UB : NETWORK FLOWS
The Greedy Algorithm with merging
Node
1
2
3
4
5
6
7
First
1
4
4
4
4
4
4
35
35
11
22
4
4
8
66
15
15
25
25
40
40
10
10
33
20
20
15
15
30
30
55
21
11
17
17
21
11
F.-Javier Heredia http://gnom.upc.edu/heredia
77
MCST- 39
Minimum Cost Spanning Trees
MEIO/UPC-UB : NETWORK FLOWS
The Greedy Algorithm with merging
Node
1
2
3
4
5
6
7
First
4
4
4
4
4
4
4
35
35
11
22
4
4
8
66
15
15
25
25
40
40
10
10
33
20
20
15
15
30
30
55
21
11
17
17
21
11
F.-Javier Heredia http://gnom.upc.edu/heredia
77
MCST- 40
Minimum Cost Spanning Trees
MEIO/UPC-UB : NETWORK FLOWS
The Greedy Algorithm with merging
• Time to determine if First(i) = First(j) for i,j:
– O(1) per arc. O(m) in total.
• Time to merge components C and D:
– Assume |C| ≥ |D|.
– O(1) per node of D (the smaller side)
– Each node i is on the smaller side of a merge at most log n times.
(Because the number of nodes in the component containing i at
least doubles when it is merged.)
– Total time spent merging: O(n log n).
• Total running time.
O(m + n log n + m log n )
F.-Javier Heredia http://gnom.upc.edu/heredia
MCST- 41
Minimum Cost Spanning Trees
MEIO/UPC-UB : NETWORK FLOWS
Analysis of the Running Time
begin
Exemple
S = {1}; T = ∅;
while S ≠ N do
begin
find the minimum cost arc (i,j) from S to N-S;
add j to S; add (i,j) to T;
end
end
Correctness: This algorithm builds a minimum cost
spanning tree from node 1. The algorithm guarantees that,
by construction, each arc added to T satisfy the cut
optimality conditions which are satisfied at the end.
F.-Javier Heredia http://gnom.upc.edu/heredia
MCST- 42
Minimum Cost Spanning Trees
MEIO/UPC-UB : NETWORK FLOWS
Prim’s Algorithm, a Dijkstra-like Algorithm
10
35
11
22
25
40
33
20
15
4
4
8
15
66
30
55
17
11
21
77
The minimum cost arc from yellow
nodes to green nodes can be found by
placing arc values in a priority queue.
F.-Javier Heredia http://gnom.upc.edu/heredia
MCST- 43
Minimum Cost Spanning Trees
MEIO/UPC-UB : NETWORK FLOWS
Prim’s Algorithm in Action
35
11
22
25
40
33
10
20
15
4
4
8
15
66
30
55
17
11
21
77
F.-Javier Heredia http://gnom.upc.edu/heredia
MCST- 44
Minimum Cost Spanning Trees
MEIO/UPC-UB : NETWORK FLOWS
Prim’s Algorithm in Action
35
11
22
25
40
33
10
20
15
8
4
4
15
66
30
55
17
11
21
77
F.-Javier Heredia http://gnom.upc.edu/heredia
MCST- 45
Minimum Cost Spanning Trees
MEIO/UPC-UB : NETWORK FLOWS
Prim’s Algorithm in Action
35
11
22
25
40
33
10
20
15
8
4
4
15
66
17
30
55
11
21
77
F.-Javier Heredia http://gnom.upc.edu/heredia
MCST- 46
Minimum Cost Spanning Trees
MEIO/UPC-UB : NETWORK FLOWS
Prim’s Algorithm in Action
35
11
22
25
40
10
20
8
4
4
15
17
30
11
33
15
66
55
21
77
F.-Javier Heredia http://gnom.upc.edu/heredia
MCST- 47
Minimum Cost Spanning Trees
MEIO/UPC-UB : NETWORK FLOWS
Prim’s Algorithm in Action
35
11
22
25
40
10
20
8
4
4
15
17
30
11
33
15
66
55
21
77
F.-Javier Heredia http://gnom.upc.edu/heredia
MCST- 48
Minimum Cost Spanning Trees
MEIO/UPC-UB : NETWORK FLOWS
Prim’s Algorithm in Action
35
11
22
25
40
10
20
8
4
4
15
17
30
11
33
15
66
55
21
77
F.-Javier Heredia http://gnom.upc.edu/heredia
MCST- 49
Minimum Cost Spanning Trees
MEIO/UPC-UB : NETWORK FLOWS
Prim’s Algorithm in Action
35
11
22
25
40
10
20
8
4
4
15
17
30
11
33
15
66
55
21
77
F.-Javier Heredia http://gnom.upc.edu/heredia
MCST- 50
Minimum Cost Spanning Trees
MEIO/UPC-UB : NETWORK FLOWS
Prim’s Algorithm in Action
35
22
11
25
40
10
20
8
4
4
15
17
30
11
33
15
66
55
21
77
F.-Javier Heredia http://gnom.upc.edu/heredia
MCST- 51
Minimum Cost Spanning Trees
MEIO/UPC-UB : NETWORK FLOWS
Prim’s Algorithm in Action
• n-1 iterations.
• At each iteration: select the minimum cost arc in
[S,N-S]→ at most m operations:
• Total running time of Prim’s algorithm: O(nm).
– Improved with heap implementations: O(m log n)
– Kruskal O(m + n log n + m log n ) > O(m logn)
F.-Javier Heredia http://gnom.upc.edu/heredia
MCST- 52
Minimum Cost Spanning Trees
MEIO/UPC-UB : NETWORK FLOWS
Running time of Prim’s Algorithm
begin
for each i ∈ N do Ni := {i};
T* := ∅
while |T*| < n - 1 do
begin
for each tree Nk of the forest do
find the least cost arc (ik,jk) from Nk to N\Nk ;
in case of ties, use a consistent tie breaking rule
let T* = T* ∪ {(i1, j1), (i2, j2), … , }
end
end
• Correctness: relies on the cut optimality conditions.
• Running time: at each iteration, the size of the smallest component at
least doubles, and so there are O(log n) iterations. A straightforward
implementation runs in O(m log n) time (AMO, Theorem 13.7) (same of
Prim’s alg.).
F.-Javier Heredia http://gnom.upc.edu/heredia
MCST- 53
Minimum Cost Spanning Trees
MEIO/UPC-UB : NETWORK FLOWS
Sollin’s Algorithm
10
35
11
22
25
40
20
4
4
8
15
66
30
17
21
33
15
55
11
77
Treat all nodes as singleton
components, and then select the min
cost arc leaving the component.
F.-Javier Heredia http://gnom.upc.edu/heredia
MCST- 54
Minimum Cost Spanning Trees
MEIO/UPC-UB : NETWORK FLOWS
Sollin’s Algorithm in Action
10
35
11
22
25
40
20
4
4
8
15
66
30
17
21
33
15
55
11
77
Find the min cost edge out of each
component
F.-Javier Heredia http://gnom.upc.edu/heredia
MCST- 55
Minimum Cost Spanning Trees
MEIO/UPC-UB : NETWORK FLOWS
Sollin’s Algorithm in Action
•
Consider the minimmum cost spanning tree problem
over the two graphs on figure 13.15, page 537 of AMO:
1. Select one of the graph and solve the MCSTP with the
Kruskal’s, Prim’s and Sollin’s algorithms.
2. Check, for some arcs of T*, the satisfaction of the Cut Optimality
Conditions.
3. Check, for some arcs of A-T* the satisfaction of the Path
Optimality Conditions.
4. Verify your solution with the help of Giden.
•
You must upload to the intranet a document (.doc or
.pdf) with the response to the above questions and the
file .gdn.
F.-Javier Heredia http://gnom.upc.edu/heredia
MCST- 56
Minimum Cost Spanning Trees
MEIO/UPC-UB : NETWORK FLOWS
Exercises