Performance Study of Byzantine Agreement
Protocol with Artificial Neural Network
by
Kok-Wah Lee
Faculty of Engineering & Technology, Multimedia University,
Jalan Ayer Keroh Lama, 75450 Bukit Beruang, Melaka, MALAYSIA.
Hong-Tat Ewe
Faculty of Information Technology, Multimedia University,
Jalan Multimedia, 63100 Cyberjaya, Selangor, MALAYSIA.
Corresponding Author :
Lee Kok Wah
Faculty of Engineering & Technology,
Multimedia University,
Jalan Ayer Keroh Lama,
75450 Melaka, MALAYSIA.
Tel : +60-(6)-252-3320
Fax : +60-(6)-231-6552
Email : kwlee@mmu.edu.my
Abstract :
Since 1982, numerous Byzantine Agreement Protocols (BAPs) have been developed to
solve arbitrary faults in the Byzantine Generals Problem (BGP). A novel BAP, using an artificial
neural network (ANN), was proposed by Wang and Kao. It requires message exchange rounds
similar to the traditional BAP and its suitability, in the context of network size, has not been
investigated. In the present study, we propose to adopt Nguyen-Widrow initialization in ANN
training, which modifies message communication and limits the message exchange rounds to
three rounds. This modified approach is referred to as BAP-ANN. The BAP-ANN performs
better than the traditional BAP, when the network size n is greater than nine. We also evaluate
the message exchange matrix (MEM) constructed during the message exchange stage. For a
fixed number of faulty nodes and remainder cases of (n mod 3), the study shows that the mean
epoch for ANN training decreases as the network size increases, which indicates better fault
tolerance.
Keywords:
Byzantine Agreement, Communication Protocol, Artificial Neural Networks, Cryptology, Fault
Tolerance.
2
1.
INTRODUCTION
With the emergence of the Internet, the consensus issue or Byzantine Generals
Problem (BGP) of the distributed system has become more significant in achieving fault
tolerance. The essential fault tolerance [3] [4] [22] necessary to reach common agreement
enables the importance of earlier investigations [14] [16] [20] [32]. Since the study of
Lamport et al. in 1982 [21], various Byzantine Agreement Protocols (BAPs) for fault
tolerance have been developed [10] [17] [27] [31] [39]. In their studies, numerous
assumptions were made to construct variations of the traditional BAP by Lamport et al. [21].
Wang and Kao [43] used a feedforward neural network (FFNN) with a backpropagation
learning algorithm (BPLA) to construct a BAP. The message communication mode of Wang
and Kao’s BAP is similar to the traditional BAP and its suitability, in the context of the
network size, has not been investigated.
The BAP approach [43] improved with artificial neural network (ANN) by Lee and
Ewe [24] has been termed, ANN-based BAP [23] [25]. Here, ANN-based BAP has been
renamed BAP with ANN (BAP-ANN). In BAP-ANN, we demonstrate a modified message
communication mode and Nguyen-Widrow initialization. Then, we evaluate the message
exchange matrix (MEM) constructed during the message exchange stage to investigate the
relationship between the network size and fault tolerance. From these studies, we infer that
the larger the network size, the better the fault tolerance.
The rest of the paper is organized as follows. Sections 2 and 3 discuss the issues of
BGP and BAP-ANN, respectively. In Sections 4 and 5, the MEM and its analysis are
explained, respectively. Conclusions are provided in Section 6.
3
2.
BYZANTINE GENERALS PROBLEM (BGP)
Internet activities have become very important and popular among users throughout
the world. This is due to the emergence of electronic data interchange (EDI), email, mobile
phone and electronic commerce. These Byzantine communications are subject to the
increasing trend of threat risks such as software errors and malicious human and computing
machine attacks. These failures are related to the fault tolerance and reliability issues of
distributed systems [3] [4] [22].
When a processor within a distributed network is infected with virus, worm or
spyware, it becomes a faulty processor and tries to confuse and attack the other processors by
sending out arbitrary messages. The distributed network is then said to become malicious and
suffers from Byzantine fault or universal fault. An ordered fault classification scheme is
shown in Fig. 1 [34] [38].
When a distributed network problem is caused by arbitrary message communication
in a malicious network, it is called Byzantine Generals Problem (BGP) [21]. The historical
basis of the term BGP is named after a martial situation in an ancient war [14] [20] [21],
where a group of Byzantine army generals surrounded an enemy city. The Byzantine
generals had to achieve a consensus on a common battle plan via the communication of
messengers only. The risk existed that one or more of the Byzantine generals might be
traitors and would try to confuse the other loyal Byzantine generals. The loyal Byzantine
generals had to find an algorithm to ensure that all the loyal Byzantine generals would obtain
the same agreement after the communication of the messengers among each other. The
success of solving BGP ensures a secure and trustable communication among the loyal
4
processors within a network. Due to the advancement in cryptology [13] [36] [42], the BGP
solution becomes possible and is commonly called Byzantine Agreement Protocol (BAP) [6]
[9] [15] [33] [37].
In analogy, this situation is the same as the security scenario we encounter in the
distributed systems and Internet nowadays. Both the loyal and faulty processors coexist
within a malicious network. It has been shown that for BGP of an unsigned message, the
Byzantine Agreement (BA) can be reached if, and only if, more than two-thirds of the
Byzantine generals are loyal [21].
In the context of a distributed computer network, an analogy between malicious
attack, software errors, and the situation of the Byzantine generals exists. For a distributed
network in a malicious environment, both the loyal and faulty processors exist. In this
Byzantine environment, the faulty processors can transmit arbitrary information to other
processors. Such arbitrary information is called Byzantine fault, and it is a fault class that can
be considered a universal fault set [12] [34] [38]. In a system model, the class of Byzantine
fault consists of every possible arbitrary or malicious fault such as timing fault, omission
fault and crash fault. A faulty processor can then confuse other loyal processors within a
distributed system.
A distributed system can be imagined as a set of interconnected processors through a
data communication network [21] [43]. A processor can be treated as a node, while a link
between any two processors can be treated as a path. The combination of these processors
and paths will collectively form a graph that symbolizes the distributed system. Therefore, in
the context of graphs, both loyal and faulty nodes exist. The malicious attacks and software
5
errors can cause the faulty node to behave in an arbitrary manner. In turn, the faulty node can
send a totally opposite message to all other nodes in the network. For the loyal node to
tolerate this Byzantine fault, all the loyal nodes have to arrive at a common value for the
BGP.
When applying BAP to get a common value, two interactive consistency conditions
(ICCs) must be fulfilled. They include:
ICC 1: All loyal nodes obey the same order.
ICC 2: If the commanding general is loyal, then every loyal lieutenant obeys the
order sent by the commanding general; or adopts the DEFAULT value if no
order is received or achieved.
These ICCs consider all types of malicious situations. “No order is received or
achieved” means no common agreement (BA) can be made from the commanding general’s
messages due to failed reception or due to lack of simple majority at any lieutenant node. For
this case, the DEFAULT value will be adopted.
The worst case scenario happens when the n-node communication network possesses
a maximum number of tolerable faulty nodes, m, as in Eqn. (1). The number of exchanged
messages to achieve the BA for traditional BAP, MsgTrad., is in (2). Meanwhile, the number
of rounds for message exchange is (m+1) [2] [16].
m =  (n-1)/3 
m
MsgTrad. =
(1)
(n  1)!
 (n  k  2)!
(2)
k 0
6
3.
BAP-ANN
Since 1982 [21], various BAPs have been proposed based on different assumptions
[20] [28] [37] that include considerations of network topology [28] [29] [35] [51], processor
fault, link fault [39] [40] [48] [49], node mobility [11] [44] [46] [47] and faulty node
detection [5] [26] [37] [45] [50]. The main purpose of these BAPs is to reduce the
computational load by specializing in the generalized model of traditional BAP [21]. Wang
and Kao [43] proposed an innovative BAP by using the artificial neural network [8] [19]
[41]. This concept has been further developed [23] [24] [25]. ANN is applied to every
processor to speed the process of BA achievement. The application of ANN shown in many
fields [1] [7] has three significant advantages over currently available BAPs. These
advantages include parallel processing capability of each node, reduction of memory space
requirement and the dynamic learning ability of the neural network.
The block diagram of BAP-ANN is depicted in Fig. 2. The operations of this BAP
and its implementations were described in detail in [24] [43]. From the information of [2]
[16] [51], its performance is improved by introducing a new scheme of message
communication mode as shown below. The formation of message exchange matrix (MEM)
[25] will be discussed in Section 4:
Message Communication Mode {
Round 1: One message is sent from the commander node to each lieutenant node.
Round 2: Each lieutenant node sends its received message to other lieutenant nodes.
Round 3: Each lieutenant node sends its (n-1) messages to other lieutenant nodes. }
7
This step reduces the round of message exchange to three rounds. Due to fewer
exchanged messages, the computation time and memory storage requirements are reduced.
Consequently, the time for fault tolerance and fault detection is decreased. The efficiency of
BAP-ANN is increased for the network size larger than nine (n ≥ 10). The illustration of this
improvement is depicted in Fig. 3.
The five stages of BAP-ANN include (i) initialization stage, (ii) message exchange
stage, (iii) training stage, (iv) application stage and (v) compromise stage. In the initialization
stage, a commander node determines which source message is delivered. Then, this node
broadcasts the source message to all the lieutenant nodes to start the message exchange stage.
At the end of the message exchange stage, MEM becomes the training data set for the
designed ANN.
After training the neural network, the ANN model can be applied for any BGP as
long as the condition in (1) is fulfilled. In this paper, we adopt an ANN model with the
architecture of two-layer feedforward neural network (FFNN) using a backpropagation
learning algorithm (BPLA). For the approach in [43], conventional BPLA was adopted. Here,
the modified BPLA applies the additional parameters of momentum, learning rate and slope
parameter. The advanced technique of initialization called Nguyen-Widrow initialization [30]
is also applied in our BPLA to help reduce early stage oscillation problems and gain faster
convergence in ANN training time. The ranges of these parameters are listed in Table 1. The
settings as explained in Table 1 include momentum = (0.2, 0.5, 0.9), learning rate = (0.5, 1.0)
and slope parameter = (2.0). For Nguyen-Widrow initialization, the settings are as follows:
Let:
p = number of input neurons Xi where i = 1, 2, 3, …, p
8
q = number of hidden neurons Yj where j = 1, 2, 3, …, q
r = number of output neurons Zk where k = 1, 2, 3, …, r
 = scale factor where  = 0.7(q)1/p
For each hidden neuron Yj:
Initialize the weights wij from input neuron Xi to hidden neuron Yj:
wij(old) = random number from -0.5 to +0.5
||wij(old)|| = [w1j(old)2 + w2j(old)2 + … + wij(old)2 + … + wnj(old)2 ]1/2
Re-initialize weights wij between input layer and hidden layer:
wij(new) =  * wij(old) / ||wij(old)||
Set bias bj on hidden neurons Yj for j = 1, 2, …, q:
bj = random number between - and +
For each output neuron Zk:
Initialize the wjk from hidden neuron Yj to output neuron Zk:
wjk = random number between -0.5 to +0.5
Set bias bk on output neurons Zk for k = 1, 2, …, r :
bk = random number between -0.5 to +0.5
9
For an n-processor distributed system, the number of input neurons, p, at the input
layer of ANN will be (n-1). There are three output neurons at the output layer indicating the
value of bit ‘0’, bit ‘1’ and the ‘DEFAULT’ value. In the hidden layer, the number of hidden
nodes, q, is initialized to be the ceiling of (p+r)/2 = (n+2)/2. Genetic algorithm can be used to
optimize the value of q. This method is called evolutionary neural networks.
The weights between any two layers of FFNN map every node to a node across the
layers. Hence, location of the faulty node is independent of its connected input neurons in the
ANN model of BAP-ANN. The crucial factor is the number of faulty processors or the
number of faulty nodes, m, within the distributed network system.
In the application stage, the weight matrix from the training stage is used for new
BAP application to reach the ICCs. This application stage is the same with the feed forward
process in the training stage. Received messages are fed to the input layer and the outputs are
collected from the output layer. These outputs are then used in the final stage, known as the
compromise stage, where the majority function is applied to achieve BA among all the loyal
nodes within the network.
4.
MESSAGE EXCHANGE MATRIX (MEM)
To train the neural network, a set of training data is needed. This training data is
obtained from the message exchange stage. It is then specially arranged into a form of (n1)*(n-1) matrix, which is called the message exchange matrix (MEM). In this section, we
discuss how MEM is constructed using the example of a 4-processor distributed system.
Figs. 4 and 6 show two kinds of faulty situations: faulty commander node and faulty
10
lieutenant node. From the loyal bit density data depicted in Figs. 5 and 7, we show that the
situation of faulty lieutenant node is worse than the situation of faulty commander node.
Hence the worse case of fault tolerance will occur when the tolerable faulty node is at its
maximum and the faulty node belongs to the lieutenant node.
We performed a mathematical analysis of MEM to explain the experimental
phenomena of a reduction in mean epoch for ANN training. These phenomena take place
when the network size increases with a fixed number of faulty nodes and (n mod 3), which is
the remainder of n/3. From the analysis, we can observe the suitability of BAP-ANN for
different sized networks. This is important for efficient BAP implementation so as to be able
to choose between the traditional BAP and BAP-ANN for different scenarios.
An n-processor distributed computer network is simulated as a graph of n-node of
fully connected network (FCN). In the first round, the commander node sends its order or
value to all the other (n-1) lieutenant nodes. At this point, each lieutenant node has one bit of
the message. In the second round, each lieutenant node sends its one bit of message to the
other (n-2) lieutenant nodes. Now, we have (n-1) bits of messages at every lieutenant node.
In the third round, which is also the final round, each lieutenant node delivers its (n-1) bits of
message to all the other (n-2) lieutenant nodes. Lastly, we have the total bits of message
(MsgANN) at all the (n-1) lieutenant nodes as in (3).
MsgANN = (n-1) * [ 1 + (n-2) + (n-1)(n-2) ]
= (n-1)3
(3)
11
Comparatively, the complexities of traditional BAP and BAP-ANN are O(nm+1) and
O(n3), respectively. As m is equal to or more than three, the efficiency of the BAP-ANN will
surpass the traditional BAP [25]. This phenomenon is shown in Fig. 3.
The message exchange phase of faulty commander node is shown in Figs. 4 and 5.
Since the worst case of FCN-4 is one faulty lieutenant node, we have the message exchange
phase of this example explained in detail in Figs. 6 and 7. Here, there are one commander
node C and three lieutenant nodes L1, L2 and L3. In the worst case, one of the three lieutenant
nodes is faulty. Hence, we have three faulty situations here, i.e. either L1, L2 or L3 is a
malicious node. Due to the symmetrical property of the ANN model, these three faulty
situations are equivalent to each other.
For example, let the lieutenant node L3 be the faulty node, sending out arbitrary
messages to confuse other loyal lieutenant nodes. Fig. 6a shows the first round of message
exchange stage, where the commander node delivers its order bit ‘1’ to the other three
lieutenant nodes. In Fig. 7a, it depicts the received message bit ‘1’ from the commander node
that is saved in the MEM of each lieutenant node.
In the second round, the loyal lieutenant nodes L1 and L2 send their saved value in the
first round to the other lieutenant nodes. On the other hand, the faulty node L3 will send bit
‘0’ to other lieutenant nodes to confuse the agreement. The message flow in the second round
is shown in Fig. 6b. Meanwhile Fig. 7b depicts the update of the MEM of each lieutenant
node after the second round of message exchange.
Fig. 6c shows the delivery of message string with a length of three bits from a
lieutenant node to the other lieutenant nodes. The exchange of message strings allows the
12
complete formation of MEM at each lieutenant node. Fig. 7c shows the final update of MEM.
MEM is now ready to be used as the training data set for the designed ANN at every node.
Among the MEM at each lieutenant node, it is only necessary to consider the MEM of the
loyal lieutenant node. In fact, we can neglect the MEM of the faulty lieutenant node. As long
as condition (1) is satisfied, consideration of MEM at the loyal nodes is sufficient to solve the
BGP. All the loyal lieutenant nodes will use the MEM formed to train the weight matrix and
the bias matrix of the designed ANN model. Once the ANN model is trained, it can be used
for future incoming BGP to achieve BA among all loyal nodes. As in this example, the
MEMs at the loyal lieutenant nodes, L1 and L2, are already sufficient to guarantee the
achievement of the BA.
MEM is a (n-1)*(n-1) matrix formed after the message exchange stage. This MEM is
used as the training data set during the training stage of BAP-ANN. After training the neural
network, the MEM is used as the testing data set in the application stage. For an n-processor
distributed network, a maximum of m faulty processors can be tolerated as in (1). Assume the
commander processor to be loyal, then there are m faulty lieutenant processors. Hence as we
take the distributed network as a graph of n nodes, we then have one commander node and
(n-1) lieutenant nodes. Among the (n-1) lieutenant nodes, m are faulty nodes. These (n-1)
lieutenant nodes are represented by (n-1) columns of the MEM with m columns representing
the faulty nodes.
The column of MEM represents the receiver nodes and the row of MEM represents
the sender node. Hence we have m rows of faulty message senders and m columns of faulty
message receivers. Due to the matrix property and the symmetrical property of ANN
architecture, the sequences of the numbers of faulty rows and faulty columns are independent
13
of BA achievement. As long as the number of faulty nodes is at a maximum of m nodes, the
BGP of the unsigned message can be solved with the BAP-ANN.
As the sequence of rows sent by the lieutenant nodes is independent from the neural
network training, we can gather the rows sent by the faulty nodes together. For example, let
us take the worst case of n = 4. For n = 4, the maximum value of m is 1. Therefore, either C,
L1, L2 or L3 is faulty. We consider the faulty L situation instead of faulty C situation since
faulty L is worse than faulty C. However, these three faulty cases (L1, L2 or L3) are
equivalent to each other. Hence, we can always assume that L1 is the faulty node. To make
the analysis easier, we arrange the MEM so that the rows with faulty messages are always at
the top of the MEM. In other words, the first m rows of the MEM are rows of messages sent
by the faulty lieutenant nodes. In addition, the first m columns from the left side of the MEM
are arranged to be columns with faulty messages. Therefore, we have an MEM as in Fig. 8
for analysis.
5.
ANALYSIS OF MESSAGE EXCHANGE MATRIX
In Fig. 8, we observe that MEM contains both loyal and faulty bits. Out of the (n-
1)*(n-1) bits, a total of (n-m-1)*(n-m-1) are loyal bits. To help analyze the MEM, we have
some denotations as follows:
n = number of nodes in the network
m = number of faulty nodes in the network
npr = total number of bits in the MEM = (n-1)2
14
nloy = number of bits of the loyal lieutenant nodes
nfau = number of bits of the faulty lieutenant nodes
vloy = number of row vectors sent by loyal lieutenant nodes
vfau = number of row vectors sent by faulty lieutenant nodes
vnpr = total number of row vectors
Assuming the commander node is loyal, we have m = number of faulty lieutenant
nodes. In addition, we can have some parameter relationships as follows in (4):
n = { 4, 5, 6, 7, …}
m = { m ≥ 0 & m ≤ floor((n-1)/3) }
floor(x) = a function to round the x value to nearest integer smaller than x.
npr = (n-1)2
nloy = (n-m-1)* (n-m-1) = (n-m-1)2
nfau = m*m + m*(n-m-1) + (n-m-1)*m = 2mn – 2m – m2
npr = nloy + nfau
vnpr = vloy + vfau
(4)
For an n-node distributed network, tolerance in the number of faulty nodes, m, ranges
from 0 to floor((n-1)/3). To analyze the effect of m on the MEM, we set the value of m to the
worst situation, where m = floor ((n-1)/3). For every critical value of m, there exist three
15
corresponding values of n where m = {1, 2, 3, 4, …}, we can have three cases of n as follows
in (5):
Case 1:
n = 3m + 1
(C1)
Case 2:
n = 3m + 2
(C2)
Case 3:
n = 3m + 3
(C3)
(5)
For the three above cases, we calculate some ratios to help the MEM analysis. These
ratios are (nloy/npr), (nfau/npr), (vloy/vnpr) and (vfau/vnpr). The computation results of the ratio
for every case are listed in Table 2. From the table, we notice the trend of the loyal bit
composition of a MEM as the size of the network increases. As we compute cases 1, 2 and 3,
we have the conclusions as follows in (6):
 nloy 
 nloy 
 nloy 

  
  

 npr  C1  npr  C 2  npr  C 3
 vloy

v
 npr

v
   loy


 C1  v npr

v
   loy


 C 2  v npr



C3
 nfau 
 nfau 
 nfau 

  
  

 npr  C1  npr  C 2  npr  C 3
 v fau 
v 
v 

   fau    fau 
v 




 npr  C1  v npr  C 2  v npr  C 3
(6)
From case 1 to case 3, the ratios of (nloy/npr) and (vloy/vnpr) increase, while the ratios
of (nfau/npr) and (vfau/vnpr) decrease. Consequently, as the network size, n, grows, case 2 and
16
case 3 will converge to case 1. For a larger network, the loyal bit density in a MEM is about
4/9 or 44.4% and the density of row vectors with loyal messages is about 2/3 or 66.7%.
In this paper, the BAP-ANN is studied by using an FCN model of an n-processor
distributed system for cases of n = 4, 7, 10, and n = 10, 11, 12. The performance evaluation
gives an analysis on the MEM formed after the message exchange stage and the required
mean epoch for neural network training. The MEM is used as both training and testing data
sets. The analysis implies that as the network size grows larger, a smaller number of epochs
will be required to train the neural network. For the case of fixed number of faulty node, this
is illustrated in Fig. 9 for n = 10, 11, 12. Meanwhile for the case of (n mod 3), it is shown in
Fig. 10. In what follows include the explanations of these phenomena.
As the number of node n increases, the number of bits in the MEM, which is (n1)*(n-1), increases in the order of n2. The pattern of bits in the MEM is classified into its
corresponding classes by using the ANN. A trained ANN performs its pattern classification
task by feeding the row vectors of the MEM into its input layer row by row.
In an MEM, all the loyal bit positions will share a common value. Therefore, there are
always a minimum of (n-m-1)2 bits out of a total of (n-1)2 sharing the same bit value.
Meanwhile, for faulty bit positions, all have randomly generated arguments. If the loyal bit
locations are all bit ‘1’, then the faulty bit locations can be either bit ‘0’ or bit ‘1’.
As the distributed network size, n, increases, the bit ‘0’ and bit ‘1’ in the faulty bit
position will be evenly distributed with a probability of occurrence at 0.5 for each. At the
same time, the loyal bit positions always carry the same bit value of either bit ‘0’ or bit ‘1’.
Hence, as n increases, the number of bit value with a loyal bit value will tend to be more. In
17
other words, the sample moves away from the worst case scenario, where all the faulty bit
positions carry a value different from the loyal bit value at the loyal bit positions. If n is big
enough and nloybit represents the number of bits carrying the loyal bit value, we have a ratio
as follows in (7):
(nloybit/npr) = [ nloy + floor(0.5*nfau) ] / npr
(7)
As n is sufficiently large, case 2 and case 3 will converge to case 1. Hence the
consideration of case 1 where (n = 3m + 1) will be sufficient to see the effect of increasing
the size of the distributed network on the bit composition of MEM. We will then generate a
simpler form for (7) as in (8) below:
(nloybit/npr)C1 = (nloy/npr)C1 + floor(0.5*nfau) / npr
= 4/9 + floor(0.5*nfau) / npr
(8)
Therefore, it becomes easier to recognize a pattern of bit composition for the message
exchange matrix as the value of n increases. Indirectly it means that it is easier to train the
neural network for faster detection of any Byzantine fault. Hence, we can finally deduce that
the mean epoch required to train the neural network reduces, as the distributed network
grows larger. This performance analysis has the implication to ease the computational
workload for fault tolerance within a malicious distributed computer network.
6.
CONCLUSIONS
Our MEM analysis proves that the loyal bit density of MEM increases with the size of
the distributed network. The increment of loyal bit density allows less mean epoch to train
18
the neural network. Hence, the BAP-ANN has an advantage over the traditional BAP as the
network size becomes bigger since BA can be achieved at a faster rate. This implies that the
secure communication within the distributed network can be improved by replacing
traditional BAP with BAP-ANN. Due to lesser-exchanged messages, our improved BAPANN achieves BA faster than the traditional BAP when n ≥ 10 as shown in Fig. 3. The
network size can thus be used as a criterion to determine appropriate BAP to be used for
different cases of BGP application.
The evaluation study indicates that for a fixed number of faulty nodes and (n mod 3),
which is the cases of n/3 remainder, the required mean epoch for neural network training
decreases as the network size n increases. These phenomena imply that larger networks are
safer and more tolerable to Byzantine fault. In the future, BAP-ANN may be used for
multipartite protocol or distributed cryptography [18] in view of the advancement of
distributed artificial intelligence (DAI).
7.
ACKNOWLEDGMENTS
This research was supported by Telekom Internal Research Funding, Malaysia under
the financial grants of EUCG02/2000/RF19 and PR/2001/0025. The authors would like to
thank anonymous reviewers, proofreaders and language editing service providers for helping
improve the readability of this article.
8.
REFERENCES
19
[1]
A.G. Adami, D.A.C. Barone, A speaker identification system using a model of artificial neural
networks for an elevator application, Information Sciences, vol. 138, no. 1-4 (2001) 1-5.
[2]
M.K. Aguilera, S. Toueg, A simple bivalency proof that t-resilient consensus requires t+1 rounds,
Information Processing Letters, vol. 71, no. 3-4 (1999) 155-158.
[3]
T. Anderson, P.A. Lee, Fault Tolerance Principles and Practice, Prentice Hall, Englewood Cliffs, NJ, USA,
1981.
[4]
A. Avizienis, Fault-tolerance systems, IEEE Transactions on Computers, vol. C-25, no. 12 (1976)
1304-1312.
[5]
B. Ayeb, A. Farhat, A flexible formal framework for masking/demasking faults, Information Sciences,
vol. 159, no. 1-2 (2004) 29-52.
[6]
O. Babaoglu, R. Drummond, Streets of Byzantium: Network architectures for fast reliable broadcasts,
IEEE Transactions on Software Engineering, vol. 11, no. 6 (1985) pp. 546-554.
[7]
Z. Boger, Experience in industrial plant model development using large-scale artificial neural
networks, Information Sciences, vol. 101, no. 3-4 (1997) 203-216.
[8]
G.R. Bolt, Fault Tolerance in Artificial Neural Networks, Ph.D. Thesis, York University, Ontario, Canada,
1992.
[9]
M. Castro, Practical Byzantine Fault Tolerance, Ph.D. Thesis, Massachusetts Institute of Technology,
Cambridge, MA, USA, 2001.
[10]
M. Castro, B. Liskov, Byzantine Fault Tolerance, USPTO Issued Patent, no. 6671821, filed on 21 Nov.
2000, granted on 30 Dec. 2003, pp. 1-30.
[11]
G.D. Crescenzo, R. Ge, G.R. Arce, Securing reliable server pooling in MANET against Byzantine
adversaries, IEEE Journal on Selected Areas in Communications, vol. 24, no. 2 (2006) 357-369.
[12]
F. Cristian, Understanding fault-tolerant distributed systems, Communications of the ACM, vol. 34, no.
2 (1991) 56-78.
[13]
W. Diffie, M.E. Hellman, New directions in cryptography, IEEE Transactions on Information Theory,
vol. IT-22, no. 6 (1976) 644-654.
[14]
D. Dolev, The Byzantine generals strike again, Journal of Algorithms, vol. 3, no. 1 (1982) 14-30.
[15]
P.N. Feldman, Optimal Algorithms for Byzantine Agreement, Ph.D. Thesis, Massachusetts Institute of
Technology, Cambridge, MA, USA, 1988.
[16]
M.J. Fischer, N. Lynch, A lower bound for the time to assure interactive consistency, Information
Processing Letters, vol. 14, no. 4 (1982) 183-186.
[17]
M. Fitzi, M. Hirt, Optimally efficient multi-valued Byzantine agreement, In: E. Ruppert (General
Chair), D. Malkhi (Program Chair), Proc. 25th ACM Symposium on Principles of Distributed
Computing, PODC ’06, Denver, Colorado, USA, 2006, pp. 163-168.
[18]
S. Goldwasser, New directions in cryptography: Twenty some years later (or Cryptography and complexity
theory: A match made in heaven), In: Proc. 38th IEEE Annual Symposium on Foundations of Computer
Science, SFCS 1997, Miami Beach, Florida, USA, 1997, pp. 314-324.
[19]
M.T. Hagan, H.B. Demuth, Neural Network Design, PWS Publishing Company, Boston, MA, USA, 1995.
[20]
L. Lamport, The weak Byzantine generals problem, Journal of ACM, vol. 30, no. 3 (1983) 668-676.
20
[21]
L. Lamport, R. Shostak, M. Pease, The Byzantine generals problem, ACM Transactions on
Programming Languages and Systems, vol. 4, no. 3 (1982) 382-401.
[22]
J.C. Laprie, Dependable computing and fault tolerance: Concepts and terminology, In: Proc. 15th IEEE
Int. Symposium on Fault-Tolerance Computing Systems, FTCS-15, vol. III, ANN Arbor, Michigan,
USA, 1985, pp. 2-11.
[23]
K.W. Lee, Artificial Neural Network Based Byzantine Agreement Protocol, Master’s Thesis, Multimedia
University, Bukit Beruang, Melaka, Malaysia, 2002.
[24]
K.W. Lee, H.T. Ewe, Artificial neural networks based algorithm for Byzantine generals problem, In: Proc.
MMU Int. Symposium on Information and Communications Technologies 2001, M2USIC 2001, Petaling
Jaya, Selangor, Malaysia, 2001, pp. 1.4(1-4).
[25]
K.W. Lee, H.T. Ewe, Artificial neural network based Byzantine agreement protocol, In: H. Leung (Ed.),
Proc. 6th IASTED Int. Conf. on Artificial Intelligence and Soft Computing, ASC 2002, Banff, Alberta,
Canada, 2002, pp. 368-373.
[26]
K.W. Lee, H.T. Ewe, Faulty node detection in the tripartite ANN based BAP, In: Proc. MMU Int.
Symposium on Information and Communications Technologies 2003, M2USIC 2003, Petaling Jaya,
Selangor, Malaysia, 2003, pp. 45-48.
[27]
J.P. Martin, L. Alvisi, Fast Byzantine consensus, IEEE Transactions on Dependable and Secure Computing,
vol. 3, no. 3 (2006) 202-215.
[28]
F.J. Meyer, D.K. Pradhan, Consensus with dual failure modes, IEEE Transactions on Parallel and
Distributed Systems, vol. 2, no. 2 (1991) 214-222.
[29]
M. Nesterenko, S. Tixeuil, Discovering network topology in the presence of Byzantine faults, In: Proc. 13th
Int. Colloquium on Structural Information and Communication Complexity, SIROCCO 2006, LNCS vol.
4056/2006, Chester, UK, 2006, pp. 212-226.
[30]
D. Nguyen, B. Widrow, Improving the learning speed of two-layer neural networks by choosing initial
values of adaptive weights, In: Proc. 1990 IEEE Int. Joint Conference on Neural Networks, IJCNN
1990, San Diego, CA, USA, 1990, pp. 21-26 (vol. 3).
[31]
V. Pathak, L. Iftode, Byzantine fault tolerant public key authentication in peer-to-peer systems,
Computer Networks, vol. 50, no. 4 (2006) 579-596.
[32]
M. Pease, R. Shostak, L. Lamport, Reaching agreement in the presence of faults, Journal of ACM, vol.
27, no. 2 (1980) 228-234.
[33]
R. Perlman, Network Layer Protocols with Byzantine Robustness, Ph.D. Thesis, Massachusetts
Institute of Technology, Cambridge, MA, USA, 1988.
[34]
A. Postma, Classes of Byzantine Fault-Tolerance Algorithms for Dependable Distributed Systems,
Ph.D. Thesis, Twente University, Netherlands, 1998.
[35]
D.V.S. Ravikant, V. Muthuramakrishnan, V. Srikanth, K. Srinathan, C. Pandu Rangan, On Byzantine
agreement over (2,3)-uniform hypergraphs, In: Proc. 18th Int. Conf. on Distributed Computing, DISC
2004, LNCS vol. 3274/2004, Amsterdam, The Netherlands, 2004, pp. 450-464.
[36]
B. Schneier, Applied Cryptography, 2nd Ed., John Wiley & Sons, New York, NY, USA, 1996.
[37]
K. Shin, P. Ramanathan, Diagnosis of processors with Byzantine faults in a distributed computing
system, In: Proc. 17th IEEE Int. Symposium on Fault-Tolerant Computing, FTCS-17, Pittsburgh,
Pennsylvania, USA, 1987, pp. 55-60.
21
[38]
D.P. Siewiorek, R.S. Swarz, Reliable Computer Systems: Design and Evaluation, 3rd Ed., A K Peters,
Natick, MA, USA, 1998.
[39]
H.S. Siu, Y.H. Chin, W.P. Yang, Byzantine agreement in the presence of mixed faults on processors
and links, IEEE Transactions on Parallel and Distributed Systems, vol. 9, no. 4 (1998) 335-345.
[40]
H.S. Siu, Y.H. Chin, W.P. Yang, Reaching strong consensus in the presence of mixed failure types,
Information Sciences, vol. 108, no. 1-4 (1998) 157-180.
[41]
L.A. Snider, Y.S. Yuen, The artificial neural networks based relay algorithm for distribution system high
impedance fault detection, In: Proc. 4th IEEE Int. Conf. on Advances in Power System Control, Operation
and Management, APSCOM-97, Hong Kong SAR, China, 1997, pp. 100-106.
[42]
W. Stallings, Cryptography and Network Security: Principles and Practice, 2nd Ed., Prentice Hall,
Upper Saddle River, NJ, USA, 1999.
[43]
S.C. Wang, S.H. Kao, A new approach for Byzantine agreement, In: Proc. 15th IEEE Int. Conf. on
Information Networking, ICOIN-15, Beppu City, Oita, Japan, 2001, pp. 518-524.
[44]
S.C. Wang, K.Q. Yan, Byzantine agreement under dual failure mobile network, Computer Standards &
Interfaces, vol. 28, no. 4 (2006) 475-492.
[45]
S.C. Wang, K.Q. Yan, Reaching fault diagnosis agreement on dual link failure mode, In: Proc. 7th
IEEE Int. Conf. on Parallel and Distributed Systems, ICPADS 2000, Iwate, Japan, 2000, pp. 291-298.
[46]
S.C. Wang, K.Q. Yan, H.C. Hsieh, The Byzantine agreement under mobile network, In: Proc. 2004
IEEE International Conference on Networking, Sensing & Control, ICNSC 2004, Taipei, Taiwan,
2004, pp. 46-51.
[47]
S.C. Wang, W.P. Yang, C.F. Cheng, Byzantine agreement on mobile ad-hoc network, In: Proc. 2004
IEEE Int. Conf. on Networking, Sensing & Control, ICNSC 2004, Taipei, Taiwan, 2004, pp. 52-57.
[48]
K.Q. Yan, C.H. Chin, S.C. Wang, Optimal agreement protocol in malicious faulty processors and
faulty links, IEEE Transactions on Knowledge and Data Engineering, vol. 4, no. 3 (1992) 266-280.
[49]
K.Q. Yan, S.C. Wang, Grouping Byzantine agreement, Computer Standards & Interfaces, vol. 28, no. 1
(2005) 75-92.
[50]
K.Q. Yan, S.C. Wang, Reaching fault diagnosis agreement on an unreliable general network,
Information Sciences, vol. 170, no. 2-4 (2005) 397-407.
[51]
K.Q. Yan, S.C. Wang, Y.H. Chin, Consensus under unreliable transmission, Information Processing
Letters, vol. 69, no. 5 (1999) 243-248.
22
9.
FIGURES
Byzantine Fault
Authenticated Byzantine Fault
Incorrect Computation Fault
Timing Fault
Omission Fault
Crash Fault
Fail-stop Fault
Fig. 1 An ordered fault classification
23
Initialization
Training
Compromise
Message
Exchange
Application
ANN
Fig. 2 Block diagram of BAP-ANN
24
Total number of exchanged messages, msg
10000
Traditional BAP
BAP-ANN
8000
6000
4000
2000
0
4
6
8
10
12
Number of processors, n
Fig. 3 Total number of exchanged messages (Msg) vs number of processors (n) for traditional
BAP and BAP-ANN
25
1
C
1
0
L2
L1
L3
Faulty Processor
Fig. 4a First round
C
1
1
L2
L1
1
1
0
L3
0
Faulty Processor
Fig. 4b Second round
C
110
110
L1
L2
110
110
110
L3
110
Fig. 4c Third round
Fig. 4 The FCN-4 message exchange stage with a faulty commander node
26
L1
L2
L3
1
1
0
Fig. 5a After first round of message exchange
L1
L2
1
1
0
1
1
0
L3
1
1
0
Fig. 5b After second round of message exchange
1
1
0
L1
1
1
0
1
1
0
1
1
0
L2
1
1
0
1
1
0
1
1
0
L3
1
1
0
1
1
0
Fig. 5c After third round of message exchange
Fig. 5 MEM at each lieutenant node during the FCN-4 message exchange stage with
a faulty commander node C
27
1
C
1
1
L2
L1
L3
Faulty Processor
Fig. 6a First round
C
1
1
L2
L1
1
1
0
L3
0
Faulty Processor
Fig. 6b Second round
C
110
110
L1
L2
110
110
000
L3
000
Fig. 6c Third round
Fig. 6 The FCN-4 message exchange stage with a faulty lieutenant node
28
L1
L2
L3
1
1
1
Fig. 7a After first round of message exchange
L1
L2
1
1
0
1
1
0
L3
1
1
1
Fig. 7b After second round of message exchange
1
1
0
L1
1
1
0
0
0
0
1
1
0
L2
1
1
0
0
0
0
1
1
0
L3
1
1
0
1
1
1
Fig. 7c After third round of message exchange
Fig. 7 MEM at each lieutenant node during the FCN-4 message exchange stage with
a faulty lieutenant node L3
29
(n-1) bits
m*m
m * (n-m-1)
(n-m-1) * m
(n-m-1) * (n-m-1)
m bits
(n-1) bits
(n-m-1) bits
Fig. 8 Message exchange matrix for an n-processor distributed system
30
1
n = 10
n = 11
n = 12
MSE
0.1
0.01
n = 10
0.001
n = 12
0
50
n = 11
100
150
200
250
Epoch
Fig. 9 Required epoch for n-node network of n = 10, 11 and 12 (where m = 3)
NB: MSE is the abbreviation of mean squared errors
31
1
n=4
n=7
n = 10
MSE
0.1
0.01
n = 10
n=7
n=4
0.001
0
1000
2000
3000
Epoch
Fig. 10 Required epoch for n-node network of n = 4, 7 and 10 (where n mod 3 = 1)
32
10.
TABLE
Table 1 Ranges of parameters for the BPLA
BPLA Parameter
Standard BPLA
Modified BPLA
Weight & bias
initialization
Random
Random / Nguyen-Widrow
Bias
Optional
Compulsory
Momentum, 
0.0
]0,1[ or 0 <  < 1
Adaptive learning rate, 
1.0
]0,1] or 0 <   1
Slope parameter
1.0
1
Convergence speed
Slower
Faster
33
Table 2 Various ratios for MEM analysis on the worst case of m
Case
C1 : n = 3m + 1
C2 : n = 3m + 2
C3 : n = 3m + 3
 nloy 


 npr 
4
9
 2m  1 


 3m  1 
 2m  2 


 3m  2 
 nfau 


 npr 
5
9
5m 2  2m
9m 2  6m  1
5m 2  4m
9m 2  12m  4
 vloy 


v 
 npr 
2
3
2m  1
3m  1
2m  2
3m  2
 v fau

v
 npr
1
3
m
3m  1
m
3m  2




34
2
2