Design of Support vector learning for Evolutionarybased fuzzy systems Chiou JuingShian1, Liu MingTang2 Department of Electrical Engineering, Southern Taiwan University of Technology, Tainan Hsien, Taiwan,R.O.C. E-mail:jschiou@mail.stut.edu.tw 1 Abstract. In this study we proposed an evolutionary genetic algorithm (GA) are discussed in which the membership function shapes and types and the fuzzy rule set, including the number of rules inside the rule set are evolved by GA to create a complete fuzzy controller. In addition, the genetic parameters of the evolutionary algorithm are adapted by using fuzzy system. Moreover, this investigation elucidates the feasibility of using SVMs to forecast genetic evolved process and then evolutionary genetic algorithm (GA) were employed to choose the parameters of a SVM model. Subsequently, examples of the simulation motor and active suspension system controlled result demonstrate that the propose method can get the faster evolution process, and the evolutionary fuzzy system can obtain the better performance. Keywords: FLC, genetic algorithms, optimal design, support vector machines (SVMs) 1 Introduction IN the design of fuzzy system, the definition of the membership functions, modeled by fuzzy sets and the rule sets are most important considerations; them also constitute the main difficulty [1,2]. In GA, there are two main reproduction operators, namely crossover and mutation. The choice of the probabilities of crossover and mutation is known to critically affect the behavior and the performance of GA, and a number of guidelines are existing in the literature for choosing them [3]. Instead of having fixed probabilities of crossover and mutation, a fuzzy-controlled crossover and mutation probabilities is presented in this paper. The basic concept is based on considering the useful knowledge and human expertise. For example, if the best fitness value of the individual does not change during several generations, the mutation rate should be increased to prevent the system premature convergence. When the fitness value of the individual grows up obviously and continuously, we should emphasize the crossover rate but the mutation rate should be reduced to keep the convergence of system. By using fuzzy system can assist GA to adjust the evolution parameters such as crossover rate and mutation rate. This paper presents evolutionary genetic algorithm by using genetic algorithm and FLC system are discussed in which the membership function shapes and types and the fuzzy rule set including the number of rules inside it are evolved using a GAs. In addition, crossover and mutation are two critical genetic parameters (operators). Moreover, we use fuzzy control to dynamic adjustment the crossover rate and mutation rate, even more using SVMs to forecast genetic evolved process and then evolutionary genetic algorithm (GA) to enhance the efficiency of evolution. 2 CHROMOSOME ENCODING METHODS GAs are search algorithms modeled after the mechanics of natural genetics. GAs can be used to obtain an approximate solution for single variable or multivariable optimal problems. While stochastic in nature, GAs perform a highly effective search of the problem hyperspace, efficiently directing the search to promising regions. So GAs work with a population of points rather than a single point. The analogy between biological evolution and a binary GA, it is required to encode values as individual chromosomes. The GA begins, like any other optimization algorithm, by defining the optimization variables, the fitness function, and the error amount. It ends like other optimization algorithms too, by testing for convergence. For the object is to optimize the number of fuzzy sets, which is the problem of membership function's distributing in fields, so fixing on membership function's shape is considered firstly. In order to properly represent the space distributing of fields, each membership function is determined by two values the start point x1 and the end point x2 . Theoretically, each fuzzy variable can have many fuzzy sets with its own membership function, but commonly used are three, five, seven, or nine fuzzy sets for each fuzzy variable, and the membership function can be linear or nonlinear. In this paper, we adopt six membership functions for the fuzzy set; they are left triangle, right triangle, triangle, Gaussian, and sigmoid functions, respectively. The membership function of each fuzzy set can be described as three parameter genes denoting the start point x1 , the end point x2 and the function type value respectively, the parameter gene in this paper is adopted real number coding, a total of six types of functions are used as the membership function gene candidates shown in Figure 1; each is represented by an integer from 0 to 6, zero is useless, one is Triangle, two is Left_triangle, three is Right_triangle, four is Gaussian, five is Sigmoid, six is Reverse_sigmoid. In order to have a homogeneous chromosome, integers are chosen to represent the start point x1 and the end point x2 instead of real values. Assume for the variable that its dynamic range is [0, 10] and that it has fuzzy sets. If the fuzzy membership functions are uniformly distributed over the range with half-way overlap as shown in Fig. 1, then the center point ci (i 1,...,7) of the ith membership function is located at ci x i1 xi 2 xi1 2 (1) Fig. 1. Six uniformly distributed membership functions. In this paper, the fuzzy numbers 1~7 on the fuzzy control rule table represent the linguistic values NL, NM, NS, ZE, PS, PM, and PL, respectively. The total length of the chromosome representing the system is (3×7)+(7×7)=70 and the system can be represented as S1 S 2 S 3 ...........S 21S 22 S 23 S 24 ...............S 68 S 69 S 70 where S1 S 2 S 3 ...........S 21 represents (start point, end point, type), S 22 to 3 3.1 S 70 represent fuzzy rule. EVOLUTIONARY GENETIC ALGORITHM ADAPTIVE GENETIC ALGORITHM Crossover and mutation are two critical operators; they facilitate an efficient search and guide the search into new regions. They can be varied during the running, usually starting out by running the GA with a relatively higher value for crossover and lower value for mutation, then tapering off the crossover value and increasing the mutation rate toward the end of the evolution. In order to make GA can explore the potential solutions in the searching field as soon as possible, the mutation rate and crossover rate should be duly adjusted during the evolution. Although there is no a rule for how to adjust these evolutionary parameters, but by using human experience can help us to achieve this purpose. For example, when the best fitness did not increasing after a few generations of the evolution, it means that the system could be stuck at a local minimum, so the system should probably concentrate on exploiting rather than exploring; in another word, the crossover rate should be decreased and the mutation rate should be increased. According to this kind of knowledge, we use fuzzy system to adjust the crossover and mutation rates. The best fitness (BF), the number of generations for unchanged best fitness (UN), and the variance of fitness (VF) are the input variables, and mutation rate (MR) and crossover rate (CR) are the output variables of the fuzzy system. The examples of the fuzzy control rules based on the linguistic description of the fuzzy implications are stated as If UN is high and VF is medium, then MR is high, and CR is low. For simplicity, each variable has three fuzzy sets: Low, Medium, and High. BF variable 0~4, VF variable 0~5, UN variable 0~10, MR variable 0.075~0.1, CR variable 0.65~0.9, so the fuzzy rule of CR and MR as shown in Table 1. Table 1 The fuzzy table of CR and MR (MR,CR) BF VF L M H L M H (L,H) (H,H) (H,H) (H,L) (H,L) (L,L) (L,H) (M,M) (M,M) (H,M) (H,M) (L,M) (L,H) (L,H) (L,H) (H,H) (M,H) (L,H) H M UN L For example, assume input [UN, BF, VF] = [5, 2, 2] for UN-VF, UN-VF inference, respectively. UN-BF inference:(M.M) If UN is 5 and BF is 2 then MR is 0.0875 and CR is 0775. UN-VF inference:(H,M) If UN is 5 and VF is 2 then MR is 0.0966 and CR is 0775. Finally, max MR, CR [0.0966,0775] . After the fuzzy inference, we can obtain the dynamic crossover and mutation rates which are used in the evolution. The flowchart of adaptive genetic algorithm is shown in Figure 2. Start Generate an Initial Population Decode chromosomes (BF,UN,VF) Optimize Fuzzy system Generate the Next Generation or Stop Yes No FLC Reproduction Support Vector Learning (CR) Crossover e R + _ k1 (MR) EVOLUTIONARY GENETIC ALGORITHM y FLC Mutation d dt k3 u Plant k2 FUZZY SYSTEM Fig. 2. Support vector learning for Evolutionary-based fuzzy systems 3.2 SUPPORT VECTOR MACHINES INTEGRATED GA GA can be applied in high dimensional, multi-modal, and complex problems. Hence, multi-modal to search the following question, namely calculate the increase of quantity. Therefore, we present a genetic fuzzy feature transformation method for support vector machines to do more accurate data (every gene variation) classification. Given data are first transformed into a high feature space by a genetic evolution, and then SVMs are used to map data into a higher feature space and then construct the hyperplane to make a final decision. Through frame of probability reference, each gene rate of change high to show system is it not convergence and is it is it evolve to continue to need to have. If the rate of change of gene equals zero one does not evolve again and asymptotically stable system. Support vector machines (SVMs) are a classification technique of machine learning based on statistical learning theory [10,12]. Considering a classification problem with two classes, SVMs are to construct an optimal hyper-plane that maximizes the margin between two classes. According to Vapnik statistical learning theory [10,11], the maximum of margin implies the extraordinary generalization capability and good performances of SVM classifiers [5,6,7]. So far, SVMs have already been successfully applied to many real fields. This paper method is to collect the best chromosome within every generation, and then the larger variation of gene to record in the chromosome, and determine the learning sample of SVM. The learning sample function is defined as follow: (2) S m _ 1 _ 1 , S m _ 1 _ 2 , S m _ 1 _ 3 ,..S m _ 1 _ 70 S m _ 2 _ 1 , S m _ 2 _ 2 , S m _ 2 _ 3 ,..S m _ 2 _ 70 Gm , S m _ l _ 1 , S m _ l _ 2 , S m _ l _ 3 ,..S m _ l _ 70 S LearningSample G m ( S m _ 1 ) - G m-1 ( S m _ 1 ) if S LearningSample 0 then Z m _ i 1 else Z m _ i -1. where Gm is the generation of evolution, S m _ l is the chromosome, S is the gene. Because the gene to larger variation represents chromosome can be optimized again, and then the opposite fixed of gene position form has already had good evolution that needn't be optimized again. It is in (2) that the chromosome best chromosome after classifying is represented. We can regard S m _ 1 as the learning sample and S m _ 2 as and train the dataset. Given a training dataset: X m S m _ 2 _ i , Z m _ i S m _ 2 _ i R n , Z m _ i 1,1, i 1, n (3) The optimal hyper-plane of X is defined as f ( S m _ 2 ) 0 , where f ( S m _ 2 ) sign w S m _ 2 _ i b , (4) In SVM literature, w is often referred to as the weight vector; b is called the bias. We choose the separating hyperplane w S m _ 2 _ i b 0 that is furthest away from the data points S m _ 2 _ i that is, that has maximal margin. Suppose S m _ 2 _ 1 and S m _ 2 _ 2 with Z m _ 1 1 and Z m _ 2 1 are positive and negative points closest to the hyperplane. For maximal separation, the hyperplane should be as far away as possible from each of them: w S m _ 2 _ 1 b 1 w S m _ 2 _ 2 b 1 w S m _ 2 _ 1 S m _ 2 _ 2 2 w 2 S m _ 2 _ 1 S m _ 2 _ 2 w w Maximising the margin is equivalent to maximizing 2 w , it turns out that the optimal separating hyperplane solving can be found as the solution to the equivalent optimization problem: (5) 1 2 w w ,b 2 subject to Z m _ i w S m _ 2 _ i b 1, min Then, constructing the optimal hyperplane is therefore a convex quadratic problem. This is the crucial property that allows generalization to the non-linear case. The Lagrange function differentiating it with regard to w, b and i 0 are introduced for each of the constraints (4) to get the following Lagrangian : Lw, b, (6) 1 2 n w i Z m _ i w S m _ 2 _ i b 1 2 i 1 Thus it possible to equivalently solve the dual optimization problem of maximizing (6), such that the gradient of L with respect to w and b vanishes, and requiring that i 0 , This is , (7) n L w , b , 0 , Z m _ i i 0 b i 1 n Lw, b, 0, w Z m _ i i S m _ 2 _ i w i 1 By substituting (7) into (6), the dual form of the optimization problem is derived. n M ( ) i Maximum i 1 n Subject to Z j 1 m_ j 1 n Z m _ i Z m _ j i j S m _ 2 _ i S m _ 2 _ j , 2 i , j 1 j 0; Q i 0, i 1,2,, n, To do this, we use the idea of SVM for crisp nonlinear regression. The basic idea is that a nonlinear regression function is achieved by simply preprocessing input : Rn F patterns S m _ 2 _ i by a map into some feature space F and then applying the standard ridge regression learning algorithm. Hence it suffices to know and use K S m _ 2 _ i , S m _ 2 _ j ( S m _ 2 _ i ), ( S m _ 2 _ j ) instead of () explicitly. Hence, we obtain the following dual optimization problem: n M ( ) i Maximum i 1 n Subject to Z j 1 m_ j 1 n Z m _ i Z m _ j i j K S m _ 2 _ i S m _ 2 _ j , 2 i , j 1 j 0; Q i 0, i 1,2,, n, (8) where Q is a positive constant and K S m _ 2 _ i S m _ 2 _ j is conventionally called a kernel function satisfying the Mercer theorem [10]. The kernel function that this paper uses is Gaussian function. The constant b is given by n b Z m _ i S m _ 2 _ i Z m _ j j S m _ 2 _ j j 1 Substituting (7) for (9) w in (4), we have f ( S m _ 2 ) Z m _ i i K S m _ 2 _ i S m b n (10) i 1 We can know separability of two subsets through checking whether the following inequalities Z m _ i wS m _ 2 _ i b 1; i 1,2,, n (11) A procedure to compute maximum margin for two subsets is described below [12]. Step 1. Solving the quadratic programming (8). Step 2. Determining the separating hyper-plane (10) according to (9). Step 3. Checking the separability between two subsets according to inequalities (11). Step 4. Let the margin be 0 if the two subsets are not separable. Step 5. Computing the maximum margin according to 1 /( w w) for the separable case where the vector w is determined by (7). 4 SIMULATION The active suspension control system of an automobile is currently of great interest, both academically and in the automobile industry worldwide. In 1999, Yi-Pin Kuo [9] uses GA-based fuzzy PI/PD controller is proposed for an automotive active suspension system (AASS). By using the merit of GA’s, the optimal decision-making rules for both types of controllers are constructed. The real-time simulation results demonstrate that the fusion of GA’s and fuzzy controller for an AASS can provide passengers much more ride comfort. In this study we proposed an intelligent fuzzy logic-based genetic algorithms by using genetic algorithm and FLC system are discussed in which the membership function shapes and types and the fuzzy rule set including the number of rules inside it are evolved using a GA’s. In addition, crossover and mutation are two critical genetic parameters (operators). We use fuzzy control to dynamic adjustment the crossover rate and mutation rate to enhance evolution velocity, and consider the comfortable that the passenger takes and automobile controlling at the same time. In order to evolve with higher speed, the system simulation uses the following support vector learning for Evolutionary-based fuzzy systems shown as Fig. 2. The input variables of the fuzzy controller are error e and error’s derivative e , and the output variable is control u . We simulation example uses the active suspension control system shown as Fig. 3. It consists of a quarter-car model, road profile model, and an active suspension controller. Without further explanation, it is assumed that the tire does not leave the ground and that z s and zu are measured from the static equilibrium position. In addition, the velocity of sprung mass zs and relative z s zu assumed to be measurable. velocity between unsprung and sprung mass are The dynamic equations are calculated as following: where ms z ks ( zu zs ) bs ( zu zs ) f a (12) mu zu k s ( zu zs ) bs ( zu zs ) f a kt ( zr zu ) (13) k s is constant of the spring, k t is spring constant of the tire, bs is damping, and zr is road surface interference. The active suspension control system is evaluated by the following aspects: passenger ride quality, suspension deflection, and road holding ability, as mentioned in the previous section from sensors: position and velocity Active Suspension Controller Force ms zs Sprung fa ks u bs mu zu Unsprung kt zr Road terrain Fig. 3. The active suspension system of a quarter-car To emulate the road terrain, we introduce the well-known step-like and pseudorandom road profiles in designing the controller. Mathematics model of road surface simulation as following S ( m) (14) CP sin( wct s i ) w ( c ) 2.5 2V where is the wave number (cycle/m); wc 7.7 (rad/s), the sprung mass natural frequency; V (m/s) is the vehicle speed; t s 0.01 s , sampling time of simulation; P 3.14 10 6 , representation of a poorer-than-average i 1 is the iteration step; (European) quality principal road; and C 10 , coefficient of S (m) which amplifies the disturbance to present the worse than normal road profile[9] . In order to consider the comfortableness that the passenger takes and automobile controlling at the same time, the two input variables to the fitness are defined as es1 ( R zs (0)) 2 , es2 ( R zs (1)) 2 ,..., esn ( R zs (n)) 2 (15) eu1 ( R zu (0)) 2 , eu2 ( R zu (1)) 2 ,..., eun ( R zu (n)) 2 (16) where R=0 is reference target, n is calculate the number of times, n F es i [ w (eui )] is error amount, the weight is w 10 . This page is control i 0 system minimum value of F by way of evolutionary fuzzy system. For more objective observation simulation result, the control method that we adopt the optimal linear feedback control law to study with this plan is compared, its control model is as follows: J (u ) where Q is symmetric and positive definite, corresponding to the control defined as x1 z s z u where (17) 1 T ( X QX u T Ru )dt 2 0 R is a weighting factor u , and X is the state vector and its entries are x 2 z s x3 z u z r x 4 z u x1 , x2 , x3 and x4 are the suspension deflection , sprung mass velocity, tire deflection, and tire velocity, respectively. For the quarter-car suspension system discussed in this section, the typical parameters for the suspension model are selected as mu 30kg, ms 250kg, k s 15000 N m k t 150000 N m , bs 1000 N .s m passive:Dashed line(--) , SVM for Evolutionary GA fuzzy:Dotted line(:) , Optimal:Dash-dotline(-.) Suspension Deflection(m) 0.05 0 Tire-Ground Contact (m) Suspension Acceleration(m/sec 2) -0.05 0 1 2 3 4 5 6 7 8 0 1 2 3 4 5 6 7 8 0 1 2 3 4 5 6 7 8 10 5 0 -5 0.02 0.01 0 -0.01 -0.02 Fig. 4. Time response to a rough road (passive: Dashed line; SVM for Evolutionary GA fuzzy: Dotted line; Optimal: Dash-dotline). (a) Suspension deflection. (b) Sprung mass acceleration. (c) The beating in the distance of the tire and ground. In this example, SVM for evolutionary Active suspension system was superior to passive suspension system and the optimal linear feedback control law of Active suspension system, the number value of different performance is arranged in table 2. Evolve the membership function appearing to see from Fig. 6. The fuzzy rules gene evolving finishing finally is: [6 5 3 7 5 1 5 1 7 3 2 1 2 6 2 7 6 5 7 7 7 2 5 6 7 6 4 7 3 2 2 1 1 1 2 2 7 3 6 2 7 6 6 6 3 4 1 1 3] Table 1. Active suspension system (SVM for Evolutionary GA fuzzy)gene code in the fuzzy table. e y e N P P P Z N N N Table 2. L M S E S M L N 6 6 3 4 1 1 3 M 2 7 3 6 2 7 6 N S Z 3 2 2 1 1 1 2 E P 2 5 6 7 6 4 7 S P 2 7 6 5 7 7 7 M P 1 7 3 2 1 2 6 L 6 5 3 7 5 1 5 The maximum overtake measure of the road surface response Suspension deflection (improvement %) Sprung mass acceleration (improvement %) The beating in the distance of the tire and ground (improvement %) L Passive Active (Opimal state feedback control) SVM for Evolutionary GA fuzzy 0.0498m 0.0433m ( 13.05% 0.0375m (0%) ) (23.1%) 4.847m/sec2 4.3563m/sec2 (0%) 10.12%) 56.2%) 0.0134m 0.0125m 0.0112m ( 16.42% (0%) (6.72%) ) ( Active 2.124m/sec2 ( Fig. 5. Fitness function of evolutionary Active suspension system. 1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 -1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1 Fig. 6. Membership function received after evolving. 3 CONCLUSION In the study of evolutionary-based fuzzy system have been discussed in which the membership function shapes and types and the fuzzy rule set, including the number of rules inside the rule set, are evolved using a GA. In addition, the crossover rate and mutation rate of GA are dynamic adjusted by fuzzy system. Moreover, this investigation elucidates the feasibility of using SVMs to forecast genetic evolved process and then evolutionary GA were employed to choose the parameters of a SVM model. The GA can help the fuzzy logic system to reach the optimization by using GA’s powerful searching ability, moreover, utilize the advantage of fuzzy logic can assist GA to increase the efficiency of evolution. The computer simulations demonstrate that the effectiveness for our proposed scheme. References 1. Jang, J. R., Self-Learning Fuzzy Controllers Based on Temporal Back Propagation, IEEE Transactions on Neural Networks, 3-5, 1992, pp.714-123. 2. Kropp, K. and Baitinger, U. G., Optimization of Fuzzy Logic Controller Inference Rules Using a Genetic Algorithms,.EUFIT'93-First Technologies, 1993, pp.1090-1096. 3. Grefenstette, J. J., Optimization of control parameters for genetic algorithms, IEEE Trans. Syst., Man, Cybern., 16, 1986, pp.122-128. 4. GUO, X., ZHOU, Y. and GONG, D., optimization of fuzzy sets of fuzzy control system based on hierarchical genetic algorithms, Proceedings of IEEE TENCON’02. 2002, pp. 1463-1466. 5. B.E. Boser, I.M. Guyon, V.N. Vapnik, A training algorithm for optimal margin classifiers, in: D.Guyon, Haussler (Eds.), Proceedings of the Fifth Annual Workshop on Computational Learning Theory, ACM Press, Pittsburgh, 1992, pp. 144– 152. 6. R. Schapire, Y. Freund, P. Bartlett, W. Sun Lee, Boosting the margin: a new explanation for theeffectiveness of voting methods, Ann. Statist. 26 (5) (1998) 1651– 1686. 7. J. Shawe-Taylor, N. Cristianini, On the generalization of soft margin algorithms, IEEE Trans. Inform.Theory 48 (10) (2002) 2721– 2735. 8. Xizhao Wang, Qiang He, Degang Chen, Daniel S. Yeung: A genetic algorithm for solving the inverse problem of support vector machines. Neurocomputing 68: 225-238 (2005) 9. Yi-Pin Kuo and Tzuu-Hseng S. Li, GA-Based Fuzzy PI/PD Controller for Automotive Active Suspension System, IEEE Transactions on Industrial Electronics, VOL. 46, NO. 6, December 1999 10. V.N. Vapnik, Statistical Learning Theory, Wiley, New York, ISBN 0-471-03003-1, 1998. 11. V.N. Vapnik, An overview of statistical learning theory, IEEE Trans. Neural Networks 10 (5) (1999)88– 999. 12. V.N. Vapnik, The Nature of Statistical Learning Theory, Springer, New York, ISBN 0-38798780-0,2000. 13. Xizhao Wang, Qiang He, Degang Chen, Daniel S. Yeung: A genetic algorithm for solving the inverse problem of support vector machines. Neurocomputing 68: 225-238 (2005). 14. Xi-Zhao Wang, Qiang He, De-Gang Chen, Daniel Yeung, A genetic algorithm for solving the inverse problem of support vector machines 15. Chiou, J.-S., and He Syu., “Applications of Fuzzy and Support Vector Machine to Speech Recognition,” Department of Electrical Engineering, Southern Taiwan University of Technology, Tainan Hsien, Taiwan, R.O.C.2006 International Symposium on Nano Science and Technology (ISNST), November 2006. 16. Macanobu Obika and Toru Yamamoto,” An Evolutionary Design of Robust PID Controllers,” Proceedings of the IEEE International Conference on Mechatronics & Automation Niagara Falls, Canada • July 2005 17. Ramirez, J., Yelamos, P., Gorriz, J.M. and Segura, J.C.: “SVM-based speech endpoint detection using contextual speech features”, Electron Lett., 2006, 42, (7), pp. 426-428