Chapter 6
Basic Structure of Fuzzy Neural Networks
In this chapter we shall discuss the structure of fuzzy neural networks. We start
with general definitions of multifactorial functions. And we show that a fuzzy neuron can be formulated by means of standard multifactorial function. We also give
definitions of a fuzzy neural network based on fuzzy relationship and fuzzy neurons.
Finally, we describe a learning algorithm for a fuzzy neural network based on V and
A operations.
6.1
Definition of Fuzzy Neurons
Neural networks alone have demonstrated their ability to classify, recall, and associate information [l].In this chapter, we shall incorporate fuzziness to the networks.
The objective to include the fuzziness is to extend the capability of the neural networks to handle “vague” information than “crisp” information only. Previous work
has shown that fuzzy neural networks have achieved some level of success both fundamentally and practically [l-lo]. As indicated in reference [l],there are several
ways to classify fuzzy neural networks: (1) a fuzzy neuron with crisp signals used
to evaluate fuzzy weights, (2) a fuzzy neuron with fuzzy signals which is combined
with fuzzy weights, and (3) a fuzzy neuron described by fuzzy logic equations.
In this chapter, we shall discuss a fuzzy neural network where both inputs and
outputs can be either a crisp value or a fuzzy set. To do this we shall first introduce
multifactorial function [ll, 121. We have pointed out from Chapter 4 that one of
the basic functions of neurons is that the input to a neuron is synthesized first, then
activated, where the basic operators to be used as synthesizing are “ ” and “ . ”
denoted by (
. ) and called synthetic operators. However, there are divers styles
of synthetic operators such as (V,A), (V,.), (+, A), etc. More general synthetic
operators will be multifactorial functions, so we now briefly introduce the concept
of multifactorial functions.
In [0, lIm,a natural partial ordering “5” is defined as follows:
+
+,
( V X , Y E [O,
11”) (.5 Y * (Zj 5 yj,
j = 1 , 2 , . . . ,rn))
A multifactorial function is actually a projective mapping from an rn-ary space to a
© 2001 by CRC Press LLC
one-ary space, denoted by M,. On many occasions, it is possible to transform the
spaces into closed unit intervals:
that we call standard multifactorial functions. For what follows, we will focus our
discussion on standard multifactorial functions.
The standard multifactorial functions can be classified into two groups:
Group 1. Additive Standard Multifactorial (ASM) functions
The functions in this group satisfy the condition: b'(xl,x2,. . * , xm) E [0,l]",
which means that the synthesized value should not be greater than the largest of
component states and should not be less than the smallest of the component states.
The following is its normal definition.
is called an m-ary Additive
Definition 1 A mapping Mm : [0,1]" + [0,1]
Standard Multifactorial function (ASMm-func) if it satisfies the following axioms:
m
(m.2)
m
A xj 5 M m ( X ) 5 j=1
V xj;
j=1
(m.3) Mm(xl,x2,.. . , z m ) is a continuous function of each variable xj.
n
The set of all ASMm-func is denoted by M , = {MmlMm is a m - dimensional
ASM, - func}. Clearly, when m = 1, an ASMm-func is an identity mapping from
[0,1] to [0,1]. Also (m.2) implies M m ( a ,. . . , a ) = a.
Example 1 The follwing mappings are examples of ASMm-funcs from [0, lImto
[O, 11:
A :x
m
I--+
A ( X ) :=
A xj,
j=1
v
x HV ( X ):= v xj,
m
:
j=1
cx
:
C(X):= cwjxj,
m
+-+
j=1
© 2001 by CRC Press LLC
(6.3)
m
where wj E [0,1], and
C wj = 1.
j=1
m
where wj E [0,1], and
V
w j = 1.
j=1
where wj E [0,1] and
Cj”=1wj = 1.
where wj E [0,1], and
v wj = 1.
j=1
m
(6.11)
(6.12)
m
where p
> 0, wj E [0,1] and C
wj =
1.
j=1
Group 2. Non-additive Standard Multifactorial (NASM) functions
This group of functions does not satisfy axiom (m.2). That is, the synthesized
value can exceed the boundaries of axiom (m.2), i.e.,
© 2001 by CRC Press LLC
For example, a department is led and managed by three people; each of them has
a strong leading ability. But for some reason, they cannot work smoothly among
themselves. Hence, the collective leading ability (a multifactorial score) falls below
the individual’s, i.e.,
M3(z1,22,x3)5
v
xj,
j=1
where xj is the leading ability of the individual i, i = 1 , 2 , 3 , and M3(z1,22,~3)
is
the multifactorial leading ability indictor for the group of three.
On the other hand, it is possible for the three management people to work together
exceedingly well. This implies that the combined leadership score can be higher than
any one of the three individual’s, i.e.,
It has the same meaning in the Chinese old saying: “Three cobblers with their wits
combined can exceed Chulceh Liang, the master minded”.
Definition 2 A mapping M, : [0, 11, e [0,1] is called an m-ary Non-Additive
Standard Multifactorial function, denoted by NASM,-func, if it satisfies axioms
(m.l), (m.3),and the following axiom:
The set of all NASM,-funcs
Example 2 The mapping
NASM,-func:
is denoted by M A .
n
:
[0, 11,
+ [0,1] defined
as the following is a
m
(51,.
.*
,x,)
H r]:(~l,.
. . ,x,)
:=
r]: xj.
(6.14)
j=1
Next, we shall use the these definitions to define fuzzy neurons.
Definition 3
A fuzzy neuron is regarded as a mapping F N :
where M , E M,, 8 E [0,1] and cp is a mapping or an activation function, cp : R
! +
[0,1] with cp(u)= 0 when u 5 0; and !R is the field of all real numbers. And a neural
network formed by fuzzy neurons is called a fuzzy neural network.
Figure 1 illustrates the working mechanism of a fuzzy neuron.
© 2001 by CRC Press LLC
Figure 1 Illustration of a fuzzy neuron
Example 3
The following mappings from [O,lIn to [0,1] are all fuzzy neurons:
(6.16)
where wi E [0,1] and
Cy=lwi = 1.
(6.17)
where wi E [0,1] and
Vy=1 W i =
1
(6.18)
where wi E [0,1] and
Vy=l wi = 1.
(6.19)
where wi E [0,1] and
CyZlwi = 1.
(6.20)
where wi E [0,1] and
Vy=l wi 5 Vy=l xi.
(6.21)
where wi E [0,1] and
ATZl wi 5
xi .
Example 4 In Equation (6.17), if we let (V i)(wi= l ) ,8 = 0 and cp = id,where
id is an identity function, i.e., (V x ) ( i d ( x )= x ) , we have a special fuzzy neuron:
n
Y= Vxi,
i=l
© 2001 by CRC Press LLC
(6.22)
In the same way, from (6.21) we have another special fuzzy neuron:
n
y = Axa.
(6.23)
i= 1
6.2
Fuzzy Neural Networks
In this section, we shall discuss fuzzy neural networks. We first use the concept
of fuzzy relationship followed by the definition of fuzzy neurons. We also discuss a
learning algorithm for a fuzzy neural network.
6.2.1
Neural Network Representation of Fuzzy Relation Equations
We consider a typical kind of fuzzy relation equation:
XoR=B
(6.24)
is the matrix of
where X = ( x l , x 2 ,* . . , x n ) is the input vector, R = ( r i j ) n ,
coefficients, and B = ( b l , b 2 , . . . , b,) is the constant matrix. Commonly, the operator
‘(0”
can be defined as follows:
(6.25)
At first, it is easy to realize that the equation can be represented by a network
shown as Figure 2 , where the activation functions of the neurons f l , f 2 , . . . , f , are
all taken as identity functions and the threshold values are zero.
z 2 A network representing fuzzy relation equations
Equation (6.25) can be solved using a a fuzzy S learning algorithm descirbed in
Section 6.3. However, if the operator is not V and A, then Equation (6.24) is difficult
© 2001 by CRC Press LLC
to solve. In this way, we consider to re-structure the network as s hown Figure 3
where the activation function of the neuron f is also taken as an identity function
and the threshold value is zero. We can interpret the network as: given a group of
training samples:
(6.26)
{ ( ( r l j ,r2j, . . . ,r n j ) ,b j ) I j = 172, . . . ,m }
Figure 3 Another network representing fuzzy relation equations
In this way, we can solve the problem by
find the weight vectors (x1,x2,..-,xn).
using an adequate learning algorithm.
In Equation (6.25), if operator “A” is replaced by operator “.”
, i.e.:
n
V (xi
*
rij)
= bj, j = 1,2,*
* *
,m
(6.27)
i=l
then Equation (6.24) is a generalized fuzzy relation equation.
If synthetic operator (V, .) is replaced by (@, .), where “@” is so-called bounded
sum, i.e.,
(6.28)
then Equation (6.24) is “almost” the same as a usual system of linear equations.
Especially, if ‘@” is replaced by “+”, i.e.,
n
rijxi =
b j , j = 1 , 2 , .. . , m
(6.29)
i=l
then it is a system of linear equations. Of course, rij and bj must not be in [0,1] (it
has already exceeded the definition of fuzzy relation equations). In other words, a
system of linear equations can be also represented by a network.
6.2.2
A Fuzzy Neural Network Based on F N ( V ,A)
Obviously, there are different types of operations for neurons existing in a fuzzy
neural network. For example, from Definition 4 we have different ASM,-func mappings, Mm, that generate different results for a neuron. A commonly used fuzzy
© 2001 by CRC Press LLC
neural
erator
network
is to take A operator
on an input and weights followed by a V opon all inputs.
Here we consider that such a fuzzy neural network,
shown in
A) f rom Definition
4. The network
is known to have fuzzy
Figure 4 7 based on FN(V,
associative
memories
ability.
Wll
W12
X1-0
.-
x2-m
x,-m
tIzi!z!
WTl77l
A fuzzy neural network
Clearly,
based on the definition,
network
is as follows:
Yl
=
Y2
= (W12
. . .
m-
Y
Rewriting
Equation
(Wll
-
l
-
Y2
l
-
Ym
L2
-h
Figure 4
Yl
base on FN(V,
the relation
between
A)
input
A Xl)
v
(w21
A x2)
v
. . . v
(W,l
A x,)
A Xl)
v
(w22
A x2)
v
. . v
(w,2
A x,)
(Wlm
(6.30)
A 51)
V (W2m
as a matrix
A X2)
form,
V ’ ’ ’ V
(Wnm
Y = (yr, ~2,. . . , ym),
(6.31)
and
I 1
Wll
W12
W21
W22
”
. . .
Wnl
For given
(6.30)
we have
X = (51, ~2,. . . ,x,)
w=
of this
A 2,)
Y =xow,
where
and output
Wn2
“’
‘.’
Wlm
’
W2m
Wnm
a set of samples:
{(as, Wls
where a, = (a,~,a,~;~~,a,,),
a weight matrix W by means
= L2,. . s,P>,
(6.32)
b, = (bsr,bS:!,...,b,,),
s = 1,2;..,p,
we can obtain
of the following
system of fuzzy relation equations:
a weight matrix W by means of the following system of fuzzy relation equations:
al o W = br
a2 o W = b2
. ..
ap o W = b,
© 2001 by CRC Press LLC
.
If we collect a, and b,, respectively, we have
and
then Equation (6.33) can be expressed by a single matrix equation as follows:
AoW=B.
(6.34)
that is:
Equation (6.21) is a fuzzy relation equation and it is not difficult to solve. We
shall discuss next a fuzzy learning algorithm.
6.3
A Fuzzy 6 Learning Algorithm
We now briefly describe procedures for the fuzzy 6 learning algorithm [13].
Step 1 Randomize
wij
initial values w;'.
(i = 1 , 2 , . . . , n,j = 1,2, . . . ,m ).
w$3 = wi"j
'
Often we can assign ( w s = I ) , (V i , j ) .
Step 2
Collect a pair of sample (a,,b,). Let s = 1.
Step 3
Calculate the outputs incurred b y a,. Let k = 1,
v
n
b:j =
i=l
© 2001 by CRC Press LLC
(wij A
a s i ) , j = 1 , 2 , . . . , m.
(6.36)
Step 4
Adjust weights. Let
6s.l . = b sg
' . - b .s 3 , j = l , 2, . . . m.
7
Update weights, i.e., calculate ( k
Wij(k
where 0
< r] 5 1 is the
Step 5
+ 1) =
+ 1)th weights based on kth weights:
i
wij
( 4- r ] & j ,
wij ( t )A as2
E
(6.37)
learning rate.
Looping. Go to Step 3 until the following condition holds:
(V i j ) ( W i j ( k ) - Wij(k
where
> bsj
otherwise,
Wij@>,
+ 1) <:
> 0 is a small number for stopping the algorithm.
Step 6 Repeat a new input. Let s
=s
(6.38)
E),
Set k = k
+ 1.
+ 1 and go to Step 2 until s = p .
We give the following example with four inputs
Example 5
Given samples a,, b,, s = 1,2,3,4:
a1
= (0.3,0.4,0.5,0.6), b l = (0.6,0.4,0.5),
(0.7,0.2,1.0,0.1),
a3 = (0.4,0.3,0.9,0.8),
a4 = (0.2,0.1,0.2,0.3),
a2 =
SowehaveA=
and
When 0.5
[
0.3 0.4 0.5 0.6
0.7 0.2 1.0 0.1 I , B =
0.4 0.3 0.9 0.8
0.2 0.1 0.2 0.3
(0.7,0.7,0.7),
= (0.8,0.4,0.5),
= (0.3,0.3,0.3).
b2 =
b3
[ ]
b4
0.6 0.4 0.5
0.7 0.7 0.7
0.8 0.4 0.5
0.3 0.3 0.3
0.3 0.4 0.5 0.6
w11 w12 w13
0.6 0.4 0.5
0.7 0.2 1.0 0.1
w21 w22 w23
0.7 0.7 0.7
< r] 5 1 and E
= 0.0001,
(6.39)
at k = 80 we have stable W as follows:
1.0 1.0 1.0
1.0 1.0 1.0
0.7 0.4 0.5
1.0 0.4 0.5
We run several tests and find out that most values in W are the same, except
w33, and w43. The following table details the difference.
© 2001 by CRC Press LLC
~ 3 1 ,
Table 1 Results for Different Tests
10.5163 ~0.700001~0.500001~0.400001~
10.4181 ~0.700001~0.500001~0.420001~
I
6.4
,
I
I
The Convergence of Fuzzy 6 Learning Rule
In this section, we shall prove that fuzzy 6 learning is convergent
Theorem 1 Let { W ( k ) l k = 1 , 2 , . . .} be the weight matrix sequence in fuzzy 6
learning rule. Then W ( k ) must be convergent.
Proof From Expression (6.33), we have the following two cases:
Case 1: If w i j ( k ) A a,i > b,j, then
n
bbj =
V ( w i j ( k )A US^) > b s j .
i=l
Therefore
6,j =
As
> 0, we
b:j
- b,j
> 0.
know that
Case 2: If w i j ( k ) A u,i 5 b,j, then
WZj(k
+ 1) = W Z j ( k ) .
Hence, based on the two cases, we always have
which means that the sequence {W(k)}is a monotonous decrease sequence.
Besides, { W ( k ) } is bounded, because
0
c W(k) c I,
where 0 is a null matrix and I is an unit matrix. Clearly, {W(k)}must be
convergent.
Q.E.D.
© 2001 by CRC Press LLC
6.5
Conclusions
In this chapter, we introduced the basic structure of a fuzzy neuron and fuzzy
neural networks. First, we described what a fuzzy neuron is by means of multifactorial functions. Then the definition of fuzzy neural networks was given by using
fuzzy neurons. We also described a fuzzy 6 learning algorithm to solve the weights
of the F N ( V ,A) type of fuzzy neural network. An example was also given. At last,
we proved that the fuzzy 6 learning algorithm must be convergent.
© 2001 by CRC Press LLC
References
1. C. T . Lin and C. S. G. Lee, Neural Fuzzy Systems, Prentice-Hall, Englewood
Cliffs, 1996.
2. B. Kosko, Fuzzy cognitive, Journal of Man-machine Studies, Vol. 24, pp.
65-75, 1986.
3. B. Kosko, Neural Networks and Fuzzy Systems, Prentice-Hall, Englewood
Cliffs, 1990.
4. S. G. Raniuk and L. 0. Hall, Fuzznet: towards a fuzzy connectionist expert
system development tool, Proceedings of IJCNN-90- WASH-DC, pp. 483-486,
1989.
5. G. A. Carpenter, S. Grossberg, and D. B. Rosen, Fuzzy ART : Fast stable
learning and categorization of analog patterns by an adaptive resonance system, Neural Networks, Vol. 4, pp. 759-771, 1991.
6. G. A. Carpenter, S. Grossberg, and D. B. Rosen, Fuzzy ART: An adaptive
resonance algorithm for rapid stable classification of analog patterns, International Joint Conference on Neural Networks, IEEE Service Center, Piscataway,
N J , 1991.
7. G. A. Carpenter, S. Grossberg, and D. B. Rosen, A neural networks realization
of Fuzzy ART, Technical Report CAS/CNS-91-021, Boston University, Boston.
8. T . Yamakawa and A. Tomoda, A fuzzy neuron and its application to pattern
recognition, 3rd IFSA Congress, pp. 330-338, 1989.
9. T. Furura, A. Kokubu, and T . Sakamoto, NFS: Neuro-fuzzy inference system,
37th meeting of IPSJ 3J-4, pp. 1386-1387, 1988.
10. H. Takagi, Fusion technology of fuzzy theory and neural networks-survey and
future directions, Proceedings of the International Conference on Fuzzy Logic
and Neural Networks, Japan, 1990.
11. H. X. Li, Multifactorial functions in fuzzy sets theory, Fuzzy Sets a n d Systems,
Vol. 35, pp. 69-84, 1990.
12. H. X. Li, Multifactorial fuzzy sets and multifactorial degree of nearness, Fuzzy
Sets and Systems, Vol. 19, No. 3, pp. 291-298, 1986.
13. X. Li, Fuzzy Neural Networks and Its Applications, Guizhou Scientific and
Technologic Press, Guizhou, China, 1994.
© 2001 by CRC Press LLC