First International ICSC Congress on Neuro Fuzzy Technologies
January 16-19, 2002; Capitolio de La Habana, Cuba
Claudio Moraga
Department of Computer Science
University of Dortmund
44221 Dortmund, Germany moraga@cs.uni-dortmund.de
The aim of the Tutorial is to give a two hours overview of the most relevant aspects in the area of fuzzy rule extraction supported by neural networks. Attendees to the Tutorial are expected to have a basic knowledge of feedforward neural networks and approximate reasoning in the frame of fuzzy logic.
(If requested, the tutorial may be extended (to a total of 4 hours) to include these introductory aspects).
Introduction:
The tutorial will discuss first the “when” and “why” of data-driven modeling and introduce some of the classical approaches, like the Wang-Mendel algorithm [WaM 92] or statistical regression.
Following this, the motivation for the neuro-fuzzy approach will be given [TMB 99].
Core: (The “how” of data-driven modeling)
The discussion of neuro-fuzzy rule extraction begins with an analysis of the ANFIS-System [Jan
92], [Jan 93], [JaS 95], [Jan 97] and of ANFIS-like systems ( e.g.
[GBB 92], [KTT 92], [YiO 92], [MoH
96]) including the case of using triangular fuzzy sets to represent the linguistic terms of the premises [God
97] Positive aspects and constraints for the rule extraction with ANFIS will be discussed. Possibilities of alleviating constraints by clustering or by an evolutionary design [MeM 96], [MFS 99] of the front end of
ANFIS will be considered. Special emphasis will be drawn upon the intepretability of the obtained rules
[ADT 97],[Rot 99]. ANFIS architectures with direct crisp outputs (Takagi-Sugeno, Tsukamoto) and fuzzy outputs (including possibly a defuzzification module) will be shown [Bra 93].
The second important approach to neuro-fuzzy rule extraction is based on the NARA-System
[TaH 91], [TSK 92] and NARA-like systems [HaM 96]. A comparative analysis will be done between
NARA and ANFIS as well as between NARA and neuro-fuzzy systems based on RFB-networks [JaS 93]
[BeB 97]. Refinements of NARA leading to splitting or chaining of rules will be presented [MoH 96]. A discussion on the interpretability of the extracted rules will close this chapter [Cra 96].
ANFIS and all ANFIS-like systems extract fuzzy if-then rules such that the premises are connected by a t-norm [Men 42], [ScS 83], [Web 83]. The t-norm being used is the product, because it is continuous and all over differentiable. It will be shown with simple examples, that there are real world problems that cannot conveniently be modelled by using t-norms to connect the premises [Mor 00a],
[MoT 01]. Compensating operators are required [Dom 82], [DuP 85], [KMP 96]. It will be shown that fuzzy if-then rules using the compensating operator symmetric sum [Sil 79] may be extracted by using feedforward neural networks with one hidden layer using the standard logistic activation function [Ben
98], [BCR 97]. It will further be shown that there is a whole class of S-shaped activation forms supporting a similar transformation [THM 99], [MoT 01]. Furthermore it will be shown [Mor 00b], [Mor 01] how to modify an ANFIS system to extract rules using the additive Gamma operator [ZiZ 80], [ZiZ 83] as well as weighted connectives [Dub 83], which are compensating operators.
The Tutorial will close with a discussion on achievements, limits and trends.

[ADT 95] Andrews R., Diederich J., Tickle A.B.: A survey and critique of techniques for extracting rules from trained artificial neural networks. Knowledge-based Systems 8 (6), 373-389, 1995
[Ben 98] Benítez J.M.: Extracción de Reglas Difusas con y de Redes Neuronales Artificiales.
Dissertation , Dept. Computer Science, University of Granada, Spain, 1998
Bersini, H., and Bontempi, G.: Now comes the time to defuzzify neuro-fuzzy models, Fuzzy Sets and
Syst ems, 90, 161-169, 1997
[BCR 97] Benítez J.M., Castro J.L., Requena I.: Are neural networks black boxes? IEEE Trans. on
Neural Networks 8 , 1156-1163, 1997
[Bra 93] Brahim K.: Neuro-Fuzzy Inferenz-Systeme. In: Fuzzy Logic. Theorie und Praxis (B. Reusch, Ed.), 176-
186, Springer, Heidelberg, 1993
[Cra 96] Craven M.: Extracting comprehensible models from trained neural networks. PhD Thesis,
University of Wisconsin, Madison, Wisconsin, USA, 1996
[Dom 82] Dombi J.: Basic concepts for a theory of evaluation: The aggregative operator. European Jr.
Operation Research 10 , 282-293, 1981
[Dub 83] Dubois, D.: Modèles mathématiques de l'imprécis et de l'incertain en vue d'applications aux techniques d'aide à la décision, Dissertation, Université Scientifique et Médicale de Grenoble, 1983
[DuP 85] Dubois D., Prade H.: A review of fuzzy set aggregation connectives. Information Sciences 36 ,
85-121, 1985
[GBB 92] Glorennec P.Y., Barret C., Brunet M.: Application of Neuro–Fuzzy Networks to identification and control of nonlinear dynamic systems. Proceedings Int. Conference on Information Processing and
Management of Uncertainty in Knowledge–based Systems (IPMU) , 507–510, Palma de Mallorca, 1992
[God 97] Godjevac J.: Neuro-fuzzy controllers. Design and Applications.
Presses Polytechniques et
Universitaires Romandes, Lausanne, Switzerland, 1997
[HaM 96] Han J., Moraga C.: Parametric Feedforward Network based Adaptive Fuzzy Modeling.
Proceedings Int. Symp. Intelligent Industrial Automation and Soft Computing , B–159–165, Reading UK,
ICSC Academic Press, 1996
[HeM 96] Heider R., Moraga C.: Evolutionary Synthesis of Neural Networks based on Graph Grammars.
Proceedings International Panel Conference on Soft and Intelligent Computing, 119–126 .
Budapest,
ISBN 963 420 510 0, 1996
[Jan 92] Jang J.S.R.: Fuzzy controller design without domain experts. Proc. 1 st IEEE Int’l Conference on
Fuzzy Systems , 289-296. IEEE-CS-Press, 1992
[Jan 93] Jang J.S.R.: ANFIS: Adaptive Network based Fuzzy Inference System. IEEE Transactions on
Systems, Man and Cybernetics 23 , (3), 665–685, 1993
[JaS 93]Jang J.S.R., Sun C.T.: Functional equivalence betwwen radial basis function networks and fuzzy inference systems. IEEE Trans. Neural Networks , 4, 156-163, 1993
[Jan 97] Jang J.S.R., (1997): ANFIS architecture. In: Neuro-fuzzy and Soft Computing (J.S. Jang, C.-T.
Sun, E. Mizutani, Eds.), Prentice Hall, New Jersey
[JaS 95] Jang J.S.R., Sun C.T.: Neuro–fuzzy Modeling and Control. Proceedings IEEE 83 , (3), 378–406,
1995
[KTT 92] Keller J.M., Tager R.R., Tahani H.: Neural Network implementation of fuzzy logic. Fuzzy Sets
and Systems 45 , (1), 1–12, 1992
[KMP 96] Klement P., Mesiar R., Pap E.: On the relationship of associative compensative operators to triangular norms and conorms. Int’l Jr. of Uncertainty, Fuzziness and Knowledge-based Systems 4 , (2),
129-144, 1996
[MFS 99] Mayer H.A., Fürlinger K., Strapetz M.: Extraction of compact rule sets from evolutionary designed artificial neural networks. Proc. 1999 Int`l Conference on Systems, Man, and Cybernetics.
I-
420-424, Tokyo, Japan. IEEE-CS-Press, 1999

[Men 42] Menger K.: Statistical Metrics. Proceedings National Academy of Sciences USA 28 , 535–537,
1942
[Mor 00a] Moraga, C.: Neuro-fuzzy modeling of compensating systems, Proc. Int'l Symposium on
Computational Intelligence , Kosice, Slovakia. In: Quo vadis, Computational Intelligence?
P. Sincak, J.
Vascak (Eds.), Physica Verlag, Heidelberg, 2000
[Mor 00b] Moraga C.: Neuro-Fuzzy Modeling with Standard Neural Networks. Proc. European
Symposiun on Intelligent Techniques ESIT-2000, (CD), Aachen, 2000
[Mor 01] Moraga C.: Neuro-fuzzy modeling between minimum and maximum. Proc. Workshop on
Computational Intelligence ; Theory and Applications. Press Technical University of Niš, Yugoslavia,
ISBN 86-801 35-36-4, 2001
[MoH 96] Moraga C., Han J.: Optimized neural nets for neuro–adaptive fuzzy modeling. Proc. Symp.
Qualitative Systems Modeling, Qualitative Fault Diagnosis, and Fuzzy Logic and Control, 50–55. (S-JEP
07759–94 Tempus), T.U. Budapest, 1996
[MoS 01] Moraga C., Salinas L.: Interpreting neural networks in the frame of the logic of Lukasiewicz.
Proc. Int’l Workconference on artificial and natural neural networks, IWANN 2001. Springer, Berlin,
2001
[MoT 00] Moraga, C.; Temme, K.-H.: S-neural networks are fuzzy models. Proc. Int'l Conf. on
Information Processing and Management of Uncertainty in Knowledge-based Systems , Madrid, Spain,
2000
[Rot 99] Rotshtein A.: Design and tuning of fuzzy rule-based systems for medical diagnosis. In: Fuzzy and Neuro-Fuzzy Systems in Medicine (H.-N. Teodorescu, A. Kandel, L.C. Jain, Eds.), 243-289. CRC
Press, Boca Raton, FLA, 1999
[Sas 97] Sasse, R.: Entwicklung eines wissensbasierten Systems zur Feststellung von Fehler-Ursachen-
Zusammenhängen in einem ausgewählten bereich eines Stahlproduzenten, Diplomarbeit, FB Informatik,
Universität Dortmund, 1997
[ScS 83] Schweizer B., Sklar A.: Probabilistic Metric Spaces . North Holland, Amsterdam, 1983
[Sil 79] Silvert W.: Symmetric Summation: A class of operations on fuzzy sets. IEEE Trans. SMC 9 , 659-
667, 1979
[TaH 91] Takagi H., Hagashi I.: NN-driven fuzzy reasoning. Int. Journal of Approximate Reasoning 5 ;
(3), 191–212, 1991
[TSK 92] Takagi H., Susuki N., Koda T., Kojima Y.: Neural Networks designed on approximate reasoning architecture and their applications. IEEE Trans. Neural Networks 3 , (5), 752–760, 1992
Temme, K.-H.; Heider, R.; Moraga, C.: Generalized neural networks for fuzzy modeling, Proc. 1999
Eusflat-Estylf Joint Conference , Palma de Mallorca, Spain, 469-472, 1999
[TMB 99] Tickle A.B., Maire F., Bologna G., Diederich J.: Extracting the knowledge embedded within trained artificial neural networks: defining the agenda. Proceedings International Symposium on Soft
Computing (SOCO'99) , 732-738, Genova, Italy. ICSC Academic Press, Canada, 1999
[WaM 92] Wang L.X., Mendel J.M.: Generating fuzzy rules learning from examples. IEEE Trans.
Systems, Man and Cybernetics , 22, 1414-1427, 1992
[Web 83] Weber, S.: A general concept of fuzzy connectives, negations and implications based on t– norms and t–conorms, Fuzzy Sets and Systems 11 , 115-134, 1983
[YiO 92] Yi H.J., Oh K.W.: Neural Network based Fuzzy Production Rule Generation and ist application to an Approximate Reasoning Approach. Proceedings 2nd. Int. Conference on Fuzzy Logic and Neural
Networks.
333–336, Iizuka, Japan, 1992
[ZiZ 80] Zimmermann, H.-J.; Zysno, P.: Latent connectives in human decision making, Fuzzy Sets and
Systems 4 37-51, 1980
[ZiZ 83] Zimmermann, H.-J.; Zysno, P.: Decisions and evaluations by hierarchical aggregation of information, Fuzzy Sets and Systems 10 243-266, 1983