prepared by Ilya Kolchinsky n generals, communicating through messengers some of the generals (up to m) might be traitors all loyal generals should decide on the same plan of action a small number of traitors cannot cause the loyal generals to adopt a bad plan every general sends v(i) ◦ every loyal general must obtain exactly same v(i)’s ◦ if the i-th general is loyal, the value that he sends must be used by all loyal generals as v(i) synchronous network (why?) for now: every general can directly communicate with any other general (fullyconnected network) reduction to the problem of a single general and multiple lieutenants ◦ why is it sufficient? A commanding general must send an order to his n-1 lieutenant generals such that interactive consistency holds in presence of at most m traitors: ◦ IC1: all loyal lieutenants obey the same order ◦ IC2: if the commanding general is loyal, then all loyal lieutenants obey the order he sends If the commander is loyal, IC2 is enough Examine a case of three generals, one traitor The only possible messages are “attack” and “retreat” Sounds simple? Commander attack retreat Lieutenant attack attack he said “retreat” he said “retreat” Lieutenant he said “attack” if ((X received from C) and (Y received from L)): execute X if ((X received from C) and (Y received from L)): execute X No solution with fewer than 3m+1 generals can cope with more than m traitors Proof: by contradiction and reduction to 3generals problem ◦ assume that a solution exists for 3m generals or fewer ◦ make each of the 3 generals simulate approximately one third of 3m generals ◦ since at most one general can be a traitor, there are at most m traitors among 3m generals ◦ reduction solves the problem of 3 generals, a contradiction. All generals execute the same algorithm ◦ sending messages ◦ receiving messages ◦ performing local computations Assumptions ◦ every message that is sent is delivered correctly ◦ the receiver of the message knows who sent it ◦ the absence of the message can be detected majority function, capable of selecting a majority value from a set of multiple values ◦ majorityv1 , v2 ,, vn v a default value to assume in case of a traitor which chooses not to send messages ◦ must be consistently returned by majority if no majority value exists OM(0): 1. the commander sends its value to every lieutenant 2. each lieutenant uses the value he receives from the commander, or the default value if no value is received OM(m), m > 0: 1. the commander sends its value to every lieutenant 2. for each i, lieutenant i sends the previously obtained value to n-2 other lieutenants using OM(m-1) 3. For each i, and each j≠i, let Vj be the value lieutenant i received from lieutenant j in previous step (using OM(m-1)). Lieutenant i uses the value majorityv1 ,, vn attack retreat attack retreat attack retreat retreat attack retreat attack retreat attack retreat attack retreat attack retreat attack retreat retreat attack attack attack retreat attack retreat attack retreat retreat attack retreat attack retreat attack retreat attack attack retreat attack retreat retreat attack attack m=2 2 0 1 C attack L L retreat attack attack retreat attack retreat attack retreat attack retreat retreat attack retreat attack retreat attack retreat attack attack retreat attack retreat retreat attack attack attack retreat attack retreat attack retreat retreat attack retreat attack retreat attack retreat attack attack retreat attack retreat retreat attack attack L attack attack L attack L L Why would this example fail if there were 3 traitors? Why is a single iteration insufficient and two are required? Lemma: For any m and k, OM(m) satisfies IC2 if there are more than 2k+m generals and at most k traitors Proof: by induction on m ◦ m=0: OM(0) works if the commander is loyal ◦ assume the claim holds for m-1,m>0 ◦ in step (2), each lieutenant i receives value v and applies OM(m-1) with n-1 generals n > 2k + m ⇒ n-1 > (2k + m) – 1 = 2k + (m-1) by induction hypothesis, each loyal lieutenant gets the value v ◦ there are at most k traitors and n-1 > 2k + (m-1)≥ 2k, majority of the lieutenants are loyal majorityv1 ,, vn v Theorem: For any m, Algorithm OM(m) satisfies conditions IC1 and IC2 if there are more than 3m generals and at most m traitors. Proof: by induction on m ◦ ◦ ◦ ◦ m=0: no traitors, hence OM(0) satisfies IC1 and IC2 assume the claim holds for m-1,m>0 if the commander is loyal, conditions hold by the Lemma if the commander is a traitor there are more than 3m-1 lieutenants, at most m-1 are traitors by induction hypothesis, OM(m-1) satisfies IC1 and IC2 any two loyal lieutenants get the same vector of values by determinism of majority function, any two loyal lieutenants obtain the same value v Number of messages? ◦ (n-1)(n-2)…(n-m-1) Time required? ◦ m+1 Introducing new assumptions: ◦ a loyal general's signature cannot be forged, and any alteration of the contents of his signed messages can be detected ◦ anyone can verify the authenticity of a general's signature The algorithm requires a choice function which receives a set of values and satisfies the following properties: ◦ if the set consists of a single element v, choice returns v ◦ if the set is empty, choice returns the default value 1. 2. 3. For each i: Vi The commander signs and sends his value to every lieutenant For each lieutenant i: A. If a message of the form v:0 received from the commander and no order has been received yet, then 1. Vi v 2. Send v:0:i to every other lieutenant 4. B. If a message of the form v : 0 : j1 : : jk and v Vi , then 1. Vi Vi v 2. if k < m, send the message v : 0 : j1 : : jk : i to every lieutenant other than j1 ,, jk When lieutenant i will receive no more messages, he obeys the order choiceVi C attack : 0 retreat : 0 attack : 0 V3={retreat,attack} V3={retreat} V3={} Order = CHOICE(attack,retreat) retreat : 0 : 3 L1 attack : 0 : 2 : 3 attack : 0 : 2 attack : 0 : 2 retreat : 0 : 3 : 2 retreat : 0 : 3 attack : 0 : 1 L2 V3={attack,retreat} V2={} V2={attack} L3 Theorem: For any m, Algorithm SM(m) solves the Byzantine Generals problem if there are at most m traitors. IC2: ◦ each loyal lieutenant receives commander’s order ◦ no additional order can be received by any loyal lieutenant ◦ hence, for each lieutenant the set Vi will consist of a single order, which he will obey by property of choice function IC1 follows from IC2 if the commander is loyal Assume the commander is a traitor Lemma (see proof below): if lieutenant i puts an order v into Vi , then lieutenant j must put an order v into V j . For each two loyal lieutenants i,j, the sets Vi , V j are the same Each two loyal lieutenants i,j obey the same order, which completes the proof Case 1: i receives the order v:0 in step 3a ◦ in this case, i sends it to j in step 3a(2), hence j receives it • Case 2: i receives the order v : 0 : j1 : : jk in step 3b, and j is one of the jr ◦ in this case, j must already have received the order v • Case 3: i receives the order v : 0 : j1 : : jk in step 3b, j is not one of the jr and k<m ◦ in this case, i sends the message v : 0 : j1 : : jk : i to j, so j must receive the order v • Case 4: i receives the order v : 0 : j1 : : jk in step 3b, j is not one of the jr and k=m ◦ since the commander is a traitor, at most m-1 of the lieutenants are traitors ◦ hence, at least one of the lieutenants j1 ,, jk is loyal ◦ this loyal lieutenant must have sent j the value v when he first received it, so j must therefore receive that value removing an assumption regarding the fullyconnected network what is the new upper bound on number of traitors? every processor knows the topology of the network every message includes the route through which it is supposed to be delivered a reliable processor validates the route ◦ drops a message if received from a wrong neighbor ◦ forwards a message to a neighbor only if it appears next to itself in the route Definition: t-connected graph is a graph in which there exist at least t disjoint paths between every pair of nodes Claim: if t >= 2m+1 and there are more than 3m processors, the problem is solvable Definition: Let a1 ,ar be the set of copies of the commander’s order received by the lieutenant i. Let U i be a set of lieutenants. A set U i is called a set of suspicious lieutenants determined by i if every message ai that did not pass through lieutenants in U i carries the same value. Algorithm Purifying a1 ,ar ; i 1. If a set U i of up to m suspicious lieutenants exists, then the purified value is the value of the messages that did not pass through U i . If no message is left, the default value is returned. 2. If there is no set U i of cardinality up to m, then the default value is returned. BG(0): BG(k), k > 0: 1. the commander sends its value to every lieutenant via 2m+1 disjoint paths 2. each lieutenant applies the purifying algorithm on the received values to find the value the commander intended to deliver 1. the commander sends its value to every lieutenant via 2m+1 disjoint paths 2. each lieutenant applies the purifying algorithm on the received values to find the value the commander intended to deliver 3. for each i, lieutenant i sends the previously obtained value to n-2 other lieutenants using BG(k-1) 4. For each i, and each j≠i, let Vj be the value lieutenant i received from lieutenant j in previous step (using BG(k-1)). Lieutenant i uses the value majority v1 ,, vn L4 retreat retreat attack retreat attack attack attack attack attack attack attack attack attack L1 C L3 attack attack attack attack attack attack attack retreat attack retreat attack L2 attack attack attack retreat attack attack Lemma: by use of Purifying algorithm every lieutenant can obtain the order sent by a loyal commander in presence of m traitors, if the connectivity of the network is at least 2m+1 Proof: ◦ let a1 ,, ar be the set of all the copies of the commander’s order that lieutenant i receives ◦ there are at most m faulty lieutenants, therefore at most m copies might be lost. This implies that the number of received copies, r, is at least m+1, a majority ◦ at least m+1 of the messages are relayed through routes which contain only reliable lieutenants ◦ hence, at least m+1 of the received copies carry the original order ◦ by definition of the Purifying Algorithm, only up to m independent copies can be eliminated, there are m+1 of independent copies of the correct value, hence they cannot be eliminated, which completes the proof. The rest of the proof is identical to the proof of correctness for OM(m), with an additional usage of the above lemma Solutions exist for Byzantine Generals Problem under various hypotheses These solutions are expensive ◦ in amount of time ◦ in amount of messages required The main reason is the requirement of achieving reliability in presence of arbitrary malfunctioning Assumptions can be changed and agreements can be relaxed L. Lamport, R. Shostak, M. Pease, “The Byzantine Generals Problem”, TPLS 4(3), 1982. Dolev, D. “The Byzantine generals strike again” J. Algorithms 3, 1 (Jan. 1982).