Set 10: Consensus with Byzantine Failures
CSCE 668
DISTRIBUTED ALGORITHMS AND
SYSTEMS
CSCE 668
Fall 2011
Prof. Jennifer Welch
1
Consensus with Byzantine Failures
2



How many processors total are needed to solve
consensus when f = 1 ?
Suppose n = 2. If p0 has input 0 and p1 has 1,
someone has to change, but not both. What if one
processor is faulty? How can the other one know?
Suppose n = 3. If p0 has input 0, p1 has input 1, and
p2 is faulty, then a tie-breaker is needed, but p2 can
act maliciously.
Set 10: Consensus with Byzantine Failures
CSCE 668
Processor Lower Bound for f = 1
3
Theorem (5.7): Any consensus algorithm for 1
Byzantine failure must have at least 4
processors.
Proof: Suppose in contradiction there is a
consensus algorithm A = (A,B,C) for 3
processors and 1 Byzantine failure.
p1
A
p0
B
p2
C
Set 10: Consensus with Byzantine Failures
CSCE 668
Specifying Faulty Behavior
4

Consider a ring of 6 nonfaulty processors running components of A like
this:
A
B
C
p1
p2
1
p0 1
0 p3
0


1
A
0
p5
p4
C
B
This execution  probably doesn't solve consensus (it doesn't have to).
But the processors do something -- this behavior is used to specify the
behavior of faulty processors in executions of A in the triangle.
Set 10: Consensus with Byzantine Failures
CSCE 668
Getting a Contradiction
5

Let 0 be this execution:
p1
act like p3
to p4 in 
p0
B
p1 and p2
must decide 0
0
0
0
p2
C
act like p0
to p5 in 
Set 10: Consensus with Byzantine Failures
CSCE 668
Getting a Contradiction
6

Let 1 be this execution:
p0 and p1
must decide 1
A
p1
B
act like p2
to p1 in 
1
p0
1
1
p2
act like p5
to p0 in 
Set 10: Consensus with Byzantine Failures
CSCE 668
The Contradiction
7

Let  be this execution:
What do p0 and
p2 decide?
act like p1
to p0 in 
A
p0
1
p1
?
act like p4
to p5 in 
0
p2
C
view of p0 in  = view of p0 in  = view of p0 in 1  p0 decides 1
view of p2 in  = view of p2 in  = view of p2 in 0  p2 decides 0
Set 10: Consensus with Byzantine Failures
CSCE 668
Views
8
β:
A
p1
p2
1
p1
?
1
0 p3
p0 1
0
act like p1
to p0 in 
p0
C
1
:
A
B
0
p2
0
p5
p4
C
B
act like p4
to p5 in 
C
A
1:
A
p0
p1
B
act like p2
to p1 in 
1
1
Set 10: Consensus with Byzantine Failures
1
p2
act like p5
to p0 in 
CSCE 668
Processor Lower Bound for Any f
9
Theorem: Any consensus algorithm for f Byzantine
failures must have at least 3f+1 processors.
Proof: Use a reduction to the 3:1 case.
 Suppose in contradiction there is an algorithm A for

f > 1 failures and n = 3f total processors.
Use A as a subroutine to construct an algorithm for 1
failure and 3 processors, a contradiction to theorem
just proved.
Set 10: Consensus with Byzantine Failures
CSCE 668
The Reduction
10


Partition the n ≤ 3f processors into three sets, P0, P1, and P2,
each of size at most f.
In the n = 3 case, let






p0 simulate P0
p1 simulate P1
p2 simulate P2
If one processor is faulty in the n = 3 system, then at most f
processors are faulty in the simulated system.
Thus the simulated system is correct.
Let the processors in the n = 3 system decide the same as
the simulated processors, and their decisions will also be
correct.
Set 10: Consensus with Byzantine Failures
CSCE 668
Exponential Tree Algorithm
11

This algorithm uses
f + 1 rounds (optimal)
 n = 3f + 1 processors (optimal)
 exponential size messages (sub-optimal)




Each processor keeps a tree data structure in its local
state
Values are filled in the tree during the f + 1 rounds
At the end, the values in the tree are used to calculate
the decision.
Set 10: Consensus with Byzantine Failures
CSCE 668
Local Tree Data Structure
12






Each tree node is labeled with a sequence of unique
processor indices.
Root's label is empty sequence ; root has level 0
Root has n children, labeled 0 through n - 1
Child node labeled i has n - 1 children, labeled i : 0
through i : n-1 (skipping i : i)
Node at level d labeled v has n - d children, labeled
v : 0 through v : n-1 (skipping any index appearing
in v)
Nodes at level f + 1 are leaves.
Set 10: Consensus with Byzantine Failures
CSCE 668
Example of Local Tree
13
The tree when n = 4 and f = 1 :
Set 10: Consensus with Byzantine Failures
CSCE 668
Filling in the Tree Nodes
14


Initially store your input in the root (level 0)
Round 1:
send level 0 of your tree to all
 store value x received from each pj in tree node labeled j
(level 1); use a default if necessary
 "pj told me that pj 's input was x"


Round 2:
send level 1 of your tree to all
 store value x received from each pj for each tree node k in
tree node labeled k : j (level 2); use a default if necessary
 "pj told me that pk told pj that pk's input was x"


Continue for f + 1 rounds
Set 10: Consensus with Byzantine Failures
CSCE 668
Calculating the Decision
15


In round f + 1, each processor uses the values in its
tree to compute its decision.
Recursively compute the "resolved" value for the root
of the tree, resolve(), based on the "resolved" values
for the other tree nodes:
value in tree node labeled  if it is a leaf
resolve() =
majority{resolve( ') : ' is a child of }
otherwise (use a default if tied)
Set 10: Consensus with Byzantine Failures
CSCE 668
Example of Resolving Values
16
The tree when n = 4 and f = 1 :
0
0
0
0
(assuming 0 is the default)
0
1
0
0
1
0
1
1
1
1
Set 10: Consensus with Byzantine Failures
1
1
CSCE 668
0
Resolved Values are Consistent
17
Lemma (5.9): If pi and pj are nonfaulty, then pi 's
resolved value for its tree node labeled ' j (what pj
tells pi for node ') equals what pj stores in its node
'.
'
'
'j
original
value = v
resolved
value = v
part of pi's tree
part of pj's tree
Set 10: Consensus with Byzantine Failures
CSCE 668
Resolved Values are Consistent
18
Proof Ideas:
 By induction on the height of the tree node.
 Uses inductive hypothesis to know that resolved
values for children of the tree node
corresponding to nonfaulty procs are
consistent.
 Uses fact that n > 3f and fact that each tree
node has at least n - f children to know that
majority of children are nonfaulty.
Set 10: Consensus with Byzantine Failures
CSCE 668
Validity
19




Suppose all inputs are v.
Nonfaulty proc. pi decides resolve(), which is the
majority among resolve(j), 0 ≤ j ≤ n-1, based on
pi 's tree.
Since resolved values are consistent, resolve(j) (at
pi) is value stored at the root of pj 's tree, which is pj 's
input value if pj is nonfaulty.
Since there are a majority of nonfaulty processors, pi
decides v.
Set 10: Consensus with Byzantine Failures
CSCE 668
Common Nodes
20

A tree node  is common if all nonfaulty procs.
compute the same value of resolve().


same resolved value
part of pi's tree
part of pj's tree
Set 10: Consensus with Byzantine Failures
CSCE 668
Common Frontiers
21

A tree node  has a common frontier if every path from
 to a leaf contains a common node.

pink means
common
Set 10: Consensus with Byzantine Failures
CSCE 668
Common Nodes and Frontiers
22
Lemma (5.10): If  has a common frontier, then  is
common.
Proof Ideas: By induction on height of . Uses fact
that resolve is defined using majority.
Set 10: Consensus with Byzantine Failures
CSCE 668
Agreement
23
The nodes on each path from a child of the root
to a leaf correspond to f + 1 different procs.
 Since there are at most f faulty processors, at
least one such node corresponds to a nonfaulty
processor.
 This node is common (by the lemma about the
consistency of resolved values).
 Thus the root has a common frontier.
 Thus the root is common (by preceding lemma).

Set 10: Consensus with Byzantine Failures
CSCE 668
Complexity
24
Exponential tree algorithm uses
 n > 3f processors
 f + 1 rounds
 exponential size messages:
 each
msg in round r contains
n(n-1)(n-2)…(n-(r-2)) values
 When r = f + 1, this is exponential if f is more than
constant relative to n
Set 10: Consensus with Byzantine Failures
CSCE 668
A Polynomial Algorithm
25




We can reduce the message size to polynomial with
a simple algorithm
The number of processors increases to
n > 4f
The number of rounds increases to
2(f + 1)
Uses f + 1 phases, each taking two rounds
Set 10: Consensus with Byzantine Failures
CSCE 668
Phase King Algorithm
26
Code for each processor pi:
pref := my input
first round of phase k, 1 ≤ k ≤ f+1:
send pref to all
receive prefs of others
let maj be value that occurs > n/2 times (0 if none)
let mult be number of times maj occurs
second round of phase k:
if i = k then send maj to all // I am the phase king
receive tie-breaker from pk (0 if none)
if mult > n/2 + f
then pref := maj
else pref := tie-breaker
if k = f + 1 then decide pref
Set 10: Consensus with Byzantine Failures
CSCE 668
Unanimous Phase Lemma
27
Lemma (5.12): If all nonfaulty processors prefer v at start of
phase k, then all do at end of phase k.
Proof:
 Each nonfaulty proc. receives at least n - f preferences for v in
first round of phase k
 Since n > 4f, it follows that n - f > n/2 + f
 So each nonfaulty proc. still prefers v.
Set 10: Consensus with Byzantine Failures
CSCE 668
Phase King Validity
28
Unanimous phase lemma implies validity:
 Suppose all procs have input v.
 Then at start of phase 1, all nf procs prefer v.
 So at end of phase 1, all nf procs prefer v.
 So at start of phase 2, all nf procs prefer v.
 So at end of phase 2, all nf procs prefer v.
 …
 At end of phase f + 1, all nf procs prefer v and
decide v.
Set 10: Consensus with Byzantine Failures
CSCE 668
Nonfaulty King Lemma
29
Lemma (5.13): If king of phase k is nonfaulty, then
all nonfaulty procs have same preference at end of
phase k.
Proof: Let pi and pj be nonfaulty.
Case 1: pi and pj both use pk 's tie-breaker. Since pk is
nonfaulty, they both have same preference.
Set 10: Consensus with Byzantine Failures
CSCE 668
Nonfaulty King Lemma
30
Case 2: pi uses its majority value v and pj uses king's
tie-breaker.
 Then pi receives more than n/2 + f preferences
for v
 So pk receives more than n/2 preferences for v
 So pk 's tie-breaker is v
Set 10: Consensus with Byzantine Failures
CSCE 668
Nonfaulty King Lemma
31
Case 3: pi and pj both use their own majority values.
 Suppose pi 's majority value is v
 Then pi receives more than n/2 + f preferences
for v
 So pj receives more than n/2 preferences for v
 So pj 's majority value is also v
Set 10: Consensus with Byzantine Failures
CSCE 668
Phase King Agreement
32
Use previous two lemmas to prove agreement:
 Since there are f + 1 phases, at least one has a
nonfaulty king.
 Nonfaulty King Lemma implies at the end of that
phase, all nonfaulty processors have same preference
 Unanimous Phase Lemma implies that from that phase
onward, all nonfaulty processors have same
preference
 Thus all nonfaulty decisions are same.
Set 10: Consensus with Byzantine Failures
CSCE 668
Complexities of Phase King
33



number of processors n > 4f
2(f + 1) rounds
O(n2f) messages, each of size log|V|
Set 10: Consensus with Byzantine Failures
CSCE 668