Minimum Spanning Trees Gallagher-Humblet-Spira (GHS) Algorithm 1 Weighted Graph G=(V,E), |V|=n, |E|=m 14 1 7 4 13 16 18 9 15 12 6 10 5 17 8 11 w (e ) 6 2 3 2 Spanning tree 14 1 7 4 13 16 18 9 15 12 6 10 5 17 8 11 2 3 Any tree T=(V,E’) (connected acyclic graph) spanning all the nodes of G 3 (MST) Minimum-weight spanning tree 14 1 7 4 13 16 18 9 15 12 6 10 5 8 11 17 2 3 A spanning tree s.t. the sum of its weights is minimized: For MST T : w (T ) w (e ) is minimized eT 4 Spanning tree fragment: 14 1 7 4 13 16 18 9 15 12 6 10 5 17 8 11 2 3 Any (connected) sub-tree of a MST 5 Minimum weight outgoing edge (MWOE) 14 1 7 4 13 16 18 9 15 12 6 10 5 17 8 11 2 3 An edge adjacent to the fragment with smallest weight and that does not create a cycle 6 Two important properties for building MST Property 1: The union of a fragment and any of its MWOE is a fragment of some MST (so called blue rule). Property 2: If the weights are distinct then the MST is unique 7 Property 1: The union of a fragment FT and any of its MWOE is a fragment of some MST. Proof: Distinguish two cases: 1. the MWOE belongs to T 2. the MWOE does not belong to T In both cases, we can prove the claim. 8 Case 1: e T Fragment F MST T e MWOE 9 Trivially, if e T then F {e } is a fragment Fragment F F {e } MST T e MWOE 10 Case 2: e T Fragment F MST T MWOE x e w (e ) w (x ) 11 If e T then add e to T and remove x Fragment F MST T x e w (e ) w (x ) 12 Fragment F Obtain T’ and since w (e ) w (x ) x e w(T' ) w(T) But w(T’) w(T), since T is an MST w(T’)=w(T), i.e., T’ is an MST 13 Fragment F F {e } e MST T’ thus F {e } is a fragment of T’ END OF PROOF 14 Property 2: If the weights are distinct then the MST is unique Proof: Basic Idea: Suppose there are two MSTs Then there is another MST of smaller weight contradiction! 15 Suppose there are two MSTs 16 Take the smallest weight edge not in intersection e 17 Cycle in RED MST e 18 Cycle in RED MST e e e’: any red edge not in BLUE MST ( since blue tree is acyclic) 19 Cycle in RED MST e e Since e is not in the intersection,w (e ) w (e ) (the weight of e is the smallest) 20 w (e ) w (e ) Cycle in RED MST e e Delete e and add e in RED MST we obtain a new tree with smaller weight contradiction! END OF PROOF 21 Prim’s Algorithm (sequential version) Start with a node as an initial fragment F Repeat Augment fragment F with a MWOE Until no other edge can be added to F 22 Fragment F 14 1 7 4 13 16 18 9 15 12 6 10 5 17 8 11 2 3 23 Fragment F 7 MWOE 14 1 4 13 16 18 9 15 12 6 10 5 17 8 11 2 3 24 Fragment F MWOE 7 4 14 1 13 16 18 9 15 12 6 5 10 8 11 2 3 25 Fragment F MWOE 14 1 7 4 13 16 18 9 15 12 6 10 5 17 8 11 2 3 26 Fragment F 14 1 7 4 13 16 18 9 15 12 6 10 5 17 8 11 2 3 27 Theorem: Proof: Prim’s algorithm gives an MST Use Property 1 repeatedly END OF PROOF 28 Prim’s algorithm (distributed version) • Works by repeatedly applying the blue rule to a single fragment, to yield the MST for G • Works with both asynchronous and synchronous nonanonymous, uniform models (and also with non-distinct weights) Algorithm (asynchronous high-level version): Let vertex r be the root as well as the first fragment REPEAT • • • • r broadcasts a message on the current fragment to search for the MWOE of the fragment (each vertex in the fragment searches for its local (i.e., adjacent) MWOE) convergecast (i.e., reverse broadcast towards r) the MWOE of the appended subfragment (i.e., the minimum of the local MWOEs of itself and its descendents) the MWOE of the fragment is then selected by r and added to the fragment, by sending an add-edge message on the appropriate path then, the root and nodes adjacent to that just entered in the fragment are notified the edge has been added UNTIL there is only one fragment left 29 Local description of asynchronous Prim Each processor stores: 1. The state of any of its incident edges, which can be either of {basic, branch, reject} 2. Its own state, which can be either {in, out} 3. Local MWOE 4. MWOE for each branch-down edge 5. Parent channel (route towards the root) 6. MWOE channel (route towards the MWOE of its appended subfragment) 30 Type of messages in asynchronous Prim 1. Search MWOE: coordination message initiated by the root 2. Test: check the status of a basic edge 3. Reject, Accept: response to Test 4. Report(weight): report to the parent node the MWOE of the appended subfragment 5. Add edge: say to the fragment node adjacent to the fragment’s MWOE to add it 6. Connect: perform the union of the found MWOE to the fragment (this changes the status of the corresponding end-node from out to in) 7. Connected: notify the root and adjacent nodes that connection has taken place Message complexity = O(n2) 31 Synchronous Prim It will work in O(n2) rounds…think to it by yourself… 32 Kruskal’s Algorithm (sequential version) Initially, each node is a fragment Repeat • Find the smallest MWOE e of all fragments • Merge the two fragments adjacent to e Until there is only one fragment left 33 Initially, every node is a fragment 14 1 7 4 13 16 18 9 15 12 6 10 5 17 8 11 2 3 34 Find the smallest MWOE 14 1 7 4 13 16 18 9 15 12 6 10 5 17 8 11 2 3 35 Merge the two fragments 14 1 7 4 13 16 18 9 15 12 6 10 5 17 8 11 2 3 36 Find the smallest MWOE 14 1 7 4 13 16 18 9 15 12 6 10 5 17 8 11 2 3 37 Merge the two fragments 14 1 7 4 13 16 18 9 15 12 6 10 5 17 8 11 2 3 38 Merge the two fragments 14 1 7 4 13 16 18 9 15 12 6 10 5 17 8 11 2 3 39 Resulting MST 14 1 7 4 13 16 18 9 15 12 6 10 5 17 8 11 2 3 40 Theorem: Kruskal’s algorithm gives an MST Proof: Use Property 1, and observe that no cycle is created. END OF PROOF 41 Synchronous GHS Algorithm Distributed version of Kruskal’s Algorithm • Works by repeatedly applying the blue rule to multiple fragments • Works with non-uniform models, distinct weights, synchronous start Initially, each node is a fragment Repeat in parallel: (Synchronous Phase) • Each fragment – coordinated by a fragment root node - finds its MWOE • Merge fragments adjacent to MWOE’s Until there is only one fragment left 42 Local description of syncr. GHS Each processor stores: 1. The state of any of its incident edges, which can be either of {basic, branch, reject} 2. Identity of its fragment (the weigth of a core edge – for single-node fragments, the proc. id ) 3. Local MWOE 4. MWOE for each branching-out edge 5. Parent channel (route towards the root) 6. MWOE channel (route towards the MWOE of its appended subfragment) 43 Type of messages 1. New fragment(identity): coordination message sent by the root at the end of a phase 2. Test(identity): for checking the status of a basic edge 3. Reject, Accept: response to Test 4. Report(weight): for reporting to the parent node the MWOE of the appended subfragment 5. Merge: sent by the root to the node incident to the MWOE to activate union of fragments 6. Connect(My Id): sent by the node incident to the MWOE to perform the union 44 Phase 0: Initially, every node is a fragment… 14 1 7 4 13 16 18 9 15 12 6 10 5 17 8 11 2 3 … and every node is a root of a fragment 45 Phase 1: Find the MWOE for each node 14 1 7 4 13 16 18 9 15 12 6 10 5 17 8 11 2 3 46 Phase 1: Merge the nodes and select a new root Root Root 14 1 13 4 18 9 15 12 6 10 5 17 Root 8 11 2 3 Root Symmetric MWOE Asymmetric MWOE The new root is adjacent to a symmetric MWOE 47 Rule for selecting a new root in a fragment Fragment 2 Fragment 1 root MWOE root Merging 2 fragments 48 Rule for selecting a new root in a fragment Merged Fragment root Higher ID Node on MWOE (non-anonymity) 49 Rule for selecting a new root in a fragment F2 root root F3 F1 F7 F4 root F5 root F6 root root root Merging more than 2 fragments 50 Rule for selecting a new root in a fragment Merged Fragment Root Higher ID Node on symmetric MWOE asymmetric 51 Remark: In merged fragments there is exactly one symmetric MWOE (n-1 edges vs n arrows) two zero F1 F2 F3 F4 F1 F2 F3 F4 F7 F5 F5 F6 Impossible Creates a fragment with no MWOE F6 F8 Impossible Creates a fragment with two MWOEs 52 After merging has taken place, the new root broadcasts New fragment(w(e)) to the new fragment, and afterwards a new phase starts w (e ) e is the symmetric MWOE of the merged fragments w (e ) e w (e ) w (e ) w (e ) w (e ) w (e ) w (e ) w (e ) w (e ) w (e ) w (e ) w (e ) w (e ) w (e ) w (e ) is the identity of the new fragment 53 In our example, at the end of phase 1 each fragment has its new identity. 4 1 Root7 1 4 4 Root 14 1 1 13 16 15 4 5 5 Root End of phase 1 10 5 12 5 9 17 2 18 2 11 2 6 8 3 2 2 2 2 Root 54 At the beginning of each new phase each node in fragment finds its MWOE MWOE 10 15 MWOE 7 3 25 12 4 MWOE 35 19 MWOE 22 55 To discover its own MWOE, each node sends a Test message containing its identity over its basic edge of min weight, until it receives an Accept test() 3 10 accept 15 test() 6 reject 56 Then it knows its local MWOE 10 MWOE 3 57 Then each node sends a Report with the MWOE of the appended subfragment to the root with convergecast (the global minimum survives in propagation) MWOE 10 10 3 3 15 3 3 4 MWOE 7 19 3 MWOE 7 25 22 12 35 19 MWOE 22 58 The root selects the minimum MWOE and sends along the appropriate path a Merge message, which will become a Connect message at the proper node 3 10 15 7 3 25 12 4 MWOE 35 19 22 59 Phase 2 of our example: After receiving the new identity, find again the MWOE for each fragment 14 1 7 4 13 16 18 9 15 12 6 10 5 17 8 11 2 3 60 Phase 2: Merge the fragments Root7 14 1 4 13 16 12 10 5 17 18 Root 9 15 6 8 11 2 3 61 At the end of phase 2 each fragment has its own unique identity. 7 Root7 7 7 15 13 9 9 10 5 12 9 End of phase 2 14 7 16 4 7 1 17 9 11 9 6 18 Root 8 3 2 9 9 9 9 62 Phase 3: Find the MWOE for each fragment 14 1 7 4 13 16 18 9 15 12 6 10 5 17 8 11 2 3 63 Phase 3: Merge the fragments 14 1 7 4 13 16 18 9 15 12 6 10 5 17 8 11 2 3 Root 64 Phase 3: New fragment 14 1 7 4 13 16 18 9 15 12 6 10 5 8 11 17 2 3 FINAL MST 65 Correctness • To guarantee correctness, phases must be syncronized • But at the beginning of a phase, each fragment can have a different number of nodes, and thus the MWOE selection is potentially asynchronous… • But each fragment can have at most n nodes, has height at most n-1, and each node has at most n-1 incident edges… • So, the MWOE selection requires at most 3n rounds, and the Merge message requires at most n rounds. • Then, the Connect message must be sent at round 4n+1 of a phase, and so at round 4n+2 a node knows whether it is a new root • Finally, the New fragment message will require at most n rounds. A fixed number of 5n+2 total rounds can be used to complete each phase (in some rounds nodes do nothing…)! 66 Complexity Phase Smallest Fragment size (#nodes) 0 1 1 2 2 4 i i 2 67 Algorithm Time Complexity Maximum possible fragment size 2i n Number of nodes Maximum # phases: i log2 n Total time = Phase time • #phases = O (n logn ) O (n ) logn 68 Algorithm Message Complexity Thr: Synchronous GHS requires O(m+n logn) msgs. Proof: We have the following messages: 1. Test-Reject msgs: at most 2 for each edge; 2. Each node sends/receives at most a single: New Fragment, Test-Accept, Report, Merge, Connect message for each phase. Since from previous lemma we have at most log n phases, the claim follows. END OF PROOF 69 Asynchronous Version of GHS Algorithm •Simulates the synchronous version •Works with uniform models, distinct weights, asynchronous start •Every fragment F has a level L(F)≥0: at the beginning, each node is a fragment of level 0 •Two types of merges: absorption and join 70 Local description of asyncr. GHS Like the synchronous, but: 1. Identity of a fragment is now given by an edge weight plus the level of the fragment; 2. A node has its own status, which can be either of {sleeping, finding, found} 71 Type of messages Like the synchronous, but now: 1. New fragment(weight,level,status): coordination message sent just after a merge 2. Test(weight,level): to test an edge 3. Connect(weight,level): to perform the union 72 Absorption Fragment Fragment F1 MWOE F2 If L(F1 ) L(F2 ) then F1 is absorbed by F2 (cost of merging proportional to F1 ) 73 The combined level is L(F ) L(F2 ) New fragment F1 MWOE F2 and a New fragment(weight,level,status) message is broadcasted to nodes of F1 by the node of F2 on which the merge took place 74 Join Fragment Fragment F1 MWOE F2 If L(F1 ) L(F2 ) and F1 and F2 have a symmetric MWOE, then F1 joins with F2 (cost of merging proportional to F1 F2 ) 75 The combined level is L( F ) L( F1,2 ) 1 New fragment F1 MWOE F2 and a New fragment(weight,L(F2)+1,finding) message is broadcasted to all nodes of F1 and F2, where weigth is that of the edge on which the merge took place 76 Remark: a joining requires that fragment levels are equal…how to control this? If L(F1)>L(F2), then F1 waits until L(F1)= L(F2) (this is obtained by letting F2 not replying to Test messages from F1 ) Fragment Fragment F1 Test F2 77 Algorithm Message Complexity Lemma: A fragment of level L contains at least 2L nodes. Proof: By induction. For L=0 is trivial. Assume it is true up to L=k-1, and let F be of level k. But then, either: 1. F was obtained by joining two fragments of level k-1, each containing at least 2k-1 nodes by inductive hypothesis F contains at least 2k-1 + 2k-1 = 2k nodes; 2. F was obtained after absorbing another fragment F’ of level<k apply recursively to F\F’, until case (1) applies. END OF PROOF 78 Algorithm Message Complexity (2) Thr: Asynchronous GHS requires O(m+n logn) msgs. Proof: We have the following messages: 1. Connect msgs: at most 2 for each edge; 2. Test-Reject msgs: at most 2 for each edge; 3. Each node sends/receives at most a single: New Fragment, Test-Accept, Merge, Report message each time the level of its fragment increases; and since from previous lemma each node can change at most log n levels, the claim follows. END OF PROOF 79 Homework Execute asynchronous GHS on the following graph: 18 2 3 4 2 4 1 17 11 9 7 8 8 24 14 5 10 6 6 1 9 7 12 assuming that system is pseudosynchronous: Start from 1 and 5, and messages sent from odd (resp., even) nodes are received after 1 (resp., 2) round(s) 80