A New Fuzzy Neural Network with Trapezoidal Fuzzy Weights
Jee-Haeng Lee and Sung-Bae Cho
Computer Science Department, Yonsei University
134 Shinchon-dong, Sudaemoon-ku, Seoul 120-749, Korea
Phone: +82-2-361-2720 Fax: +82-2-365-2579
Email: [easygo, sbcho]@candy.yonsei.ac.kr
Key Word: fuzzy neural network, trapezoidal membership function, learning algorithm
Abstract
Fuzzy neural networks can be used to solve the
problems that have both fuzzy number and real number
information. This paper proposes a new learning
algorithm of fuzzy neural networks with trapezoidal
fuzzy weights. We construct the trapezoidal fuzzy
weights by composition of two triangles, and devise a
learning algorithm using the two triangular membership
functions. This model can learn a variety of fuzzy
numbers efficiently. The results of computer simulations
on numerical data show that the proposed fuzzy neural
networks produce better results than the conventional
methods.
1. Introduction
Neural networks have been widely applied to solve
various problems. However, conventional neural
networks can handle only numerical data represented by
real number vectors. Therefore, it is very difficult to
represent linguistic information from human experts and
ambiguous information with conventional neural
networks. To express linguistic and ambiguous
information, fuzzy set theory has been used in various
ways [1]. Since fuzzy neural networks can use fuzzy
numbers as well as real numbers, and represent linguistic
information better than standard neural networks, they
can be applied to many problems. Various approaches to
construct fuzzy neural networks have been proposed, but
there are few architectures and learning algorithms that
can be used as standard models [2].
This paper proposes a learning algorithm and
architecture of fuzzy neural networks which utilize
trapezoidal fuzzy numbers as weights. We construct a
trapezoid as composition of two triangles, and devise a
learning algorithm with the two triangular membership
functions effectively. The main reason of using new
weights is to solve the problems that have triangular and
trapezoidal fuzzy numbers together for inputs and
outputs. This paper shows new architecture and learning
algorithm of the fuzzy neural network. We show
simulation results that compares different fuzzy neural
networks for learning fuzzy numbers.
2. Fuzzy Neural Networks
2.1 Architecture
The input-output relation of a two-layer feedforward neural network can be formulated as follows.
This neural network has input, hidden, and output units.
The fuzzy weights and fuzzy biases are trapezoidal fuzzy
numbers, and the inputs and target outputs can be any
shape of fuzzy numbers [3,4]. Fig. 1 shows the
architecture of this fuzzified neural network.
Input
units
1
1
Output
units
1
i
j
k
nI
nH
nO
Hidden units
Fuzzy weights Wji
Fuzzy biases
Fuzzy weights Wkj
Fuzzy biases
Fig. 1. Architecture of fuzzy neural network.
Input units:
O pi  X pi ,
i  1,2,..., nI
Hidden units:
O pj  f ( Net pj ) ,
j  1,2,..., nH
Hence, we can write the cost function of (Xp, Tp) as
nI
Net pj   W ji O pi   j
i 1
eh   e ph
Output units:
h
O pk  f ( Net pk ) ,
k  1,2,..., nO
3.2 Derivation of Learning Algorithm
nH
Net pk  Wkj O pj   k
j 1
where Xpi is a fuzzy input, Wji and Wkj are trapezoidal
fuzzy weights, and j and k are trapezoidal fuzzy biases.
2.2 Fuzzy Numbers
The operations of fuzzy numbers can be defined by
the extension principle of Zadeh. Our approach is based
on the level sets of fuzzy numbers where arbitrary
number of h can be used. The fuzzy outputs are
numerically calculated by interval arithmetic for level
sets of fuzzy weights and fuzzy inputs [1]. For example,
Fig. 2 represents a fuzzy number denoted by 6 level sets
where h is 0.0, 0.2, …, 1.0.
In this section we derive a learning algorithm of the
neural network from the cost function described in the
previous section. The fuzzy weights and biases are
adjusted to minimize the value of cost function. Since the
adjustment should not distort the trapezoidal shape of
weights and biases, we do not change the h-level set of
the fuzzy weights independently [3,4]. In this paper, we
construct a trapezoid as composition of two symmetric
triangles (shown in Fig. 3). We can update each
trapezoidal fuzzy weights without destructing its shape
by the movement of the two triangles and the change of
their width. Then, let the trapezoidal fuzzy weights have
six parameters as follows.
L1 C1 U1 L2 C 2 U 2
Wkj  (wkj
, wkj , wkj , wkj , wkj , wkj )
Fig. 3 shows the parameters of trapezoidal fuzzy weights.
h
1
0
x
Fig. 2. Fuzzy numbers in fuzzy neural network.
3. Learning Algorithm
3.1 Cost Function
e pkh  e Lpkh  eUpkh
eUpkh  h 
([T pk ]hL  [O pk ]hL ) 2
2
U 2
([T pk ]U
h  [O pk ]h )
2
Then cost function of h level sets is
n0
e ph   e pkh .
k 1
0
Fig. 3. Trapezoidal fuzzy weights.
Let Tp be the nO dimensional fuzzy target output and
Opk be the actual output. Then, the cost function for the
kth output of level h is as follows.
e Lpkh  h 
Membership
Membership
1
We derive a rule of adjustment of the weights for
each parameter. According to the standard backpropagation algorithm, parameters are updated by the
following formula, and the other weights are updated in
the same way.
1
U1
U1
wU
kj (t  1)  wkj (t )  wkj (t )
L1
L1
L1
wkj
(t  1)  wkj
(t )  wkj
(t )
L1
wkj
(t )   
1
wU
kj (t )   
e ph
L1
wkj
e ph
1
wU
kj
L1
   wkj
(t  1)
1
   wU
kj (t  1)
where t is the index of number of iterations through
learning,  is positive real constant, and  is positive real
constant less than 1.0. Then the derivatives are written as
follows.
e ph

wkjL1


e ph
[Wkj ]hL1
e ph
[Wkj ]hL 2
[Wkj ]hL1



wkjL1
[Wkj ]hL 2

wkjL1
e ph
[Wkj ]hL1
e ph
[Wkj ]Uh 1
e ph
[Wkj ]Uh 2


1
wU
kj


e ph

wkjL1
[Wkj ]Uh 2
wkjL1
e ph
h
h
 
 (1  )
1
2 [Wkj ]U
2
h
[Wkj ]hL1
e ph
[Wkj ]hL 2
[Wkj ]hL1


1
wU
kj
[Wkj ]hL 2
e ph
[Wkj ]hL1
1
wU
kj
 (1 


e ph
[Wkj ]Uh 1
e ph
[Wkj ]Uh 2


If the inequalities are not satisfied after updated, the
modification the two weights are swaped.
[Wkj ]Uh 1
and
e ph
should hold following constraints.
2
wkjL1  wkjL 2 , wUkj 1  wU
for all 1  j  nH , 1  k  nO .
kj
[Wkj ]Uh 1
1
wU
kj
[Wkj ]Uh 2
1
wU
kj
e ph
h
h
)

1 2
2
[Wkj ]U
h
4. Simulation Results
Fuzzy neural networks can learn fuzzy numbers. We
construct various fuzzy numbers that have shapes of
triangle only, trapezoid only, and the mixture of both
triangle and trapezoid. Every experiment has done on the
same condition that learning rate is 0.5, momentum
constant is 0.5, 10 hidden nodes and 6-level sets, 0.0, 0.2,
…, 1.0. The neural networks have been learnedfor 3000
times to make a comparison with other models.
First experiment is to learn simple triangular fuzzy
numbers. Let us consider next fuzzy if-then rules :
If x is small then y is small.
If x is large then y is large.
where
h
L1
1 h
[Wkj ]hL1  wkj
 (1  )  wU
kj 
2
2
h
U1
L1 h
U1
[Wkj ]h  wkj   wkj  (1  )
2
2
Fig. 5(a) shows the membership functions of the fuzzy
numbers, and Fig. 5(b) shows actual outputs.
.
In this way, the amount of adjustment is calculated.
Finally, the weights can be updated as follows.
1
0.8
0.6
0.4
0.2
wkjL1 (t  1)  wkjL1 (t )  wkjL1 (t )
1
wU
kj (t
wkjC1 (t
 1) 
 1) 
1
wU
kj (t )

0
0
1
wU
kj (t )
1
wkjL1 (t  1)  wU
kj (t  1)
2
0.2
0.4
0.6
0.8
1
(a) Membership functions.
1
0.8
Since the two triangles in a trapezoidal fuzzy weight are
symmetric, wkjC1 (t  1) can be calculated easily.
wkjL 2 and wUkj 2 are modified in the same way. An
example for the adjustment of weights is illustrated in
Fig. 4. All the weights and biases of the fuzzy neural
network are modified in this way.
1
1
h
h
0
0
Fig. 4. Modification of weight.
When fuzzy neural network is updated, any weights
0.6
0.4
0.2
0
0
0.2
0.4
0.6
0.8
1
(b) Actual outputs.
Fig. 5. Learning triangular fuzzy numbers.
Next experiment is to learn simple triangular fuzzy
numbers. Let us consider next fuzzy if-then rules :
If x is medium small then y is medium small.
If x is medium large then y is medium large.
Fig. 6(a) shows membership functions of the fuzzy
numbers, and Fig. 6(b) shows actual outputs.
Final experiment is to learn the mixed fuzzy
numbers of triangle and trapzoid. Let us consider next
fuzzy if-then rules :
If x is medium small then y is medium small.
If x is medium large then y is medium large.
If x is middle then y is middle.
Fig. 7(a) shows the membership functions of the fuzzy
numbers, and Fig. 7(b) shows actual outputs.
1
0.8
0.6
Table 1 shows the mean-squared errors after 3000
iterations. For further information, refer to our recent
publication [5].
Shapes of fuzzy numbers
for inputs and outputs
Triangle
Trapezoid
Mixture
Triangle
0.00083
0.00762
0.00860
Trapezoid
0.00103
0.00866
0.00969
Proposed
0.00148
0.00656
0.00796
Table 1. Mean-squared errors for learning
various shapes of fuzzy numbers.
Types of
weight
0.4
0.2
5. Conclusions
0
0
0.2
0.4
0.6
0.8
1
(a) Membership functions.
1
0.8
0.6
0.4
0.2
0
0
0.2
0.4
0.6
0.8
1
(b) Actual outputs.
Fig. 6. Learning trapezoidal fuzzy numbers.
In this paper, we propose a new fuzzy neural
network with trapezoidal fuzzy weights which are made
by two triangular fuzzy weights, and derive a learning
algorithm for the neural network. With computer
simulations, our model has high fitting ability for more
complex fuzzy numbers.
Fuzzy neural network can learn both numerical and
linguistic values represented by fuzzy numbers, so it can
be applied to various problems where human knowledge
can be provided. However, there are models of fuzzy
neural networks regarded as standard models. A general
architecture and algorithms should be reported with
various experiments in many environments.
1
0.8
References
0.6
0.4
0.2
0
0
0.2
0.4
0.6
0.8
1
(a) Membership functions.
1
0.8
0.6
0.4
0.2
0
0
0.2
0.4
0.6
0.8
1
(b) Actual outputs.
Fig. 7. Learning mixed fuzzy numbers.
Regarding the modeling power, we cannot conclude that
the proposed fuzzy neural network is better than other
neural network with pure triangular and trapezoidal
fuzzy weights. However, with respect to the meansquared error, our model is better than other models.
[1] A. Kaufmann, M.M. Gupta, Introduction to Fuzzy
Arithmetic: Theory and Applications, Van Nostrand
Reinhold, New York, 1985.
[2] J.J. Buckley, and Y. Hayashi, “Fuzzy neural
networks: A survey,” Fuzzy Sets and Systems, vol.
66, pp. 1-13, 1994.
[3] H. Ishibuchi, M. Morioka, and I.B. Turksen,
“Learning by fuzzified neural networks,” Internat. J.
Approximate Reasoning, vol. 13, pp. 327-358, 1995.
[4] H. Ishibuchi, K. Kwon, and H. Tanaka, “A learning
algorithm of fuzzy neural networks with triangular
fuzzy weights,” Fuzzy Sets and Systems, vol. 71, pp.
277-293, 1995.
[5] K. H. Lee, and S. B. Cho, “A learning algorithm of
fuzzy neural networks with trapezoidal fuzzy
weights,” Proc. of Asian Fuzzy Systems Symposium,
Masan, Korea, June 1998.