Research Journal of Applied Sciences, Engineering and Technology 4(20): 4012-4021, 2012 ISSN: 2040-7467 © Maxwell Scientific Organization, 2012 Submitted: December 20, 2011 Accepted: April 23, 2012 Published: October 15, 2012 Intelligent Self-developing and Self-adaptive Electric Load Forecaster based on Adaptive FNN+GA+GD Chinwang Lou and Mingchui Dong Faculty of Science and Technology, University of Macao, China Abstract: In this study, a novel electric load forecaster based on adaptive Fuzzy Neural Networks (FNN) and using Genetic Algorithm (GA) mixed with Gradient Descent (GD) is proposed to make it to posses the human learning ability. The proposed SDSA-FNN is firstly compared with various methods applied on function approximations. Moreover, it is applied on electric load forecasting application and verified on electric load data recorded on Macao power system. The simulation results reveal that the proposed methodology not only keeps the traditional objective function. Keywords: Fuzzy neural networks, smart grid, smart load forecaster, self-developing, self-adaptive INTRODUCTION In new advocated “smart grid” development, a lot of renewable energy sources, i.e., local Wind Farms (WF), Photovoltaic (PV) cells, battery storages, Electric Vehicles (EV) and huge various sensors are widely installed and distributed at customer side. This will make significant challenges to existing technologies of electric load forecasting because higher uncertainty, indefiniteness and variability would occur at demand side. Old techniques focusing on only forecasting accuracy are no longer adequate. New technologies are demanded to offer smart capabilities in handling the changing operation environment, i.e. self-learning, self-development, selfadaptability, intelligence, transparency and accuracy. Firstly, in our proposed SDSA-FNN (SelfDeveloping and Self-Adaptive FNN), algorithm will automatically determine optimal system structure in direction of achieving higher system accuracy. However, using more complex system structure will definitely reduce system error, but number of generated fuzzy rules will also increase, resulting in curse of dimensionality. A special technique in this study is used to avoid this problem. Secondly, when external operation environment is changed, traditional method is to re-train this whole system, resulting in completely different system structure. This new system might be no longer suitable for past operation conditions. This is not favorable for electric load forecasting application because electric load is characterized with repeated and periodic variation. However, in our novelty, this complete re-training is not necessary. System can be continuously refined such as to let past and new operation conditions are simultaneously suitable. This function is very important for application of electric load forecasting because electric load reveals periodic variation. The proposed methodology is finally applied and tested on power system of Macao which lies on the western side of the Pearl River Delta and faces the South China Sea. The simulation results reveal that proposed methodology not only keeps the traditional objective function, i.e., accuracy, but also enhance traditional forecaster with intelligence and smartness, resulting in a smart load forecaster. METHODOLOGY Interpretable fuzzy model for forecasting system: Our proposal is based on Takagi and Sugeno (1985) fuzzy model proposed in Takagi and Sugeno (1985), Wang and Mendel (1992) and Shi et al. (2002). T-S fuzzy model possess nice property to be able to approximate any realoccurring nonlinear relation to a certain degree of accuracy (Wang, 1992). Network structure of this MISO (Multi-Input SingleOutput) fuzzy model is shown in Fig. 1, which has following rule form. However, it has no difficulty to extend into MIMO (Multi-Input and Multi-Output) system. Rule i: If x1 is Ai1 and x2 is Ai2 and and xn is Ain then zi is yi with wi where, x : Input vector [x1, x2, xn]T zi : Output for ith fuzzy rule Aij : Fuzzy set for jth input variable xj in ith rule (i = 1, 2, m; j = 1, 2, n) yi : ith fuzzy rule weighting between normalization node and output node wi : ith fuzzy rule confidence between normalization node and output node Corresponding Author: Chinwang Lou, Faculty of Science and Technology, University of Macao, China 4012 Res. J. Appl. Sci. Eng. Technol., 4(20): 4012-4021, 2012 Fig. 1: Takagi-sugeno MISO fuzzy system m: Number of fuzzy rules n: Number of input variables rule due to activation of its antecedent by the observed input x: Assuming Gaussian-type Membership Function (MF) is adopted for fuzzy set of input variable due to its good approximation ability, infinite support, easy extraction of parameters and equivalency to RBFs neural networks (Jang and Sun, 1993; Koczy et al., 2000). Complete I/O relation of this fuzzy system can be formulated as below (Remark: I is node input, O is node output, subscript and superscript represent node and layer index, respectively): I i3 = Ai1 Ai 2 ... Ain , Oi3 = hi = I i3 where, i = 1, 2, m (m = M1´M2´ ´Mn). Layer 4-normalization nodes: Number of nodes in this layer is equal to that in layer 3. The firing strength calculated in layer 3 is normalized in this layer: Layer 1-input nodes: Each node in this layer, which represents an input variable to FNNs, directly transmits input vector x = [x1, x2, xn]T to next layer as described below: I i4 = hi m ∑h i i =1 Layer 5-output nodes: Node in this layer represents an output variable of the FNNs. The function of this node is to combine the incoming signals from layer 4 in order to produce an output response z to the input vector x: I 1j = x j , O1j = x j where, j = 1, 2, n. Layer 2-fuzzification nodes: Each node in layer 2 represents one linguistic label of one of the input variables. The output of this node specifies the degree to which an input xj belongs to kth fuzzy set of the associated input node j in layer 1: m z= ∑ yhw i =1 m ∑ i =1 I 2jk , Oi4 = I i4 ⎡ ( x j − a jk ) 2 ⎤ 2 = x j , O jk = A jk = exp ⎢ − ⎥ 2b 2jk ⎢⎣ ⎥⎦ where, ajk and bjk are center and width of this linguistic label respectively, j = 1, 2, .., n; k = 1, 2, Mj and Mj is number of fuzzy sets on jth input variable. Layer 3-pre-condition nodes: Each node in this layer represents one fuzzy logic rule and performs antecedent matching of this rule. The output of each node in this layer represents the firing strength of corresponding fuzzy i i i (1) hi wi Parameter determination: Parameter determination includes all ajk and bjk associated with center and width of jth fuzzy set of ith input variable in premise part, all yi and wi in consequent part (Fig. 1). The derivation of parameter optimization is presented below for the system structure as depicted in Fig. 1. Given P-paired multi-input and single output data samples (Xp, Tp) where,,, Xp = (xp1, xp2, xpn) where, p = 1, 2, P. Learning objective is to minimize cost function (2) by finding optimal parameters ajk, bjk, yi and wi. 4013 Res. J. Appl. Sci. Eng. Technol., 4(20): 4012-4021, 2012 Min E = P ∑E p=1 P = P (Tp − z p ) 2 p =1 2 ∑ b jk (t + 1) = b jk (t ) − η4 ⋅ ∂E / ∂b jk (t ) ⎡ m ∂ hi ⎢ ∑ wl y l hi wi yi ∑ ∂ b jk ⎢ l ∈A ∂E i =1 = − (Tp − zp) ⋅ ⎢ − 2 m ∂ bjk ⎛ m ⎞ ⎢ ∑ hi wi ⎜ ⎟ (2) ⎤ ⎥ ⎛ ∂ hl ⎞ ⎥ ⎜ ⎟ ⋅ ⎜ ∑ wl ⎟⎥ ⎝ l ∈Ak ∂ b jk ⎠ ⎥ hi wi ∑ ⎥ ⎝ j =1 ⎠ ⎦ k ⎢ ⎣ where, Tp : Targeted output at pth input-output sample; zp : System output for pth input sample; ∂ hi ⎡ ⎢ ∑ wl y l ∂ b l ∈ Ak jk ⎢ − = − (Tp − z p ) ⋅ m ⎢ w h ⎢ ∑ i i i =1 ⎣ Those parameters can be adjusted with following equations: yi(t+1) = yi(t)-01.ME/Myi(t) Ak : Set of all pre-condition nodes hl which have connections with this ajk/bjk; h1,h2,h3,h4 : Algorithm learning rates which are small positive numbers i C Update wi wi (t + 1) = wi (t ) − η2 . ∂ E / ∂ wi (t ) m ⎡ ⎤ ⎢ hi wi yi ⎥ ∑ ∂E y ⎥ = − (Tp − z p ). hi ⎢⎢ m i − i =m1 2 ∂ wi ⎛ ⎞ ⎥ ⎢ ∑ hi wi ⎜ ∑ hi wi ⎟ ⎥ ⎝ i =1 ⎠ ⎥⎦ ⎢⎣ i =1 yi − z p = − (Tp − z p ). hi m ∑ hi wi (4) i =1 Incremental evolving-construction of fuzzy system: Construction of proposed fuzzy system involves learning of premise structure and determination of parameters on fuzzy rule premise and consequent part. In existing literature, premise structure is determined either by offline or on-line algorithm. However, all techniques referred in online algorithm (Juang and Lin, 1998; Wu and Er, 2000; Angelov and Buswell, 2002; Kasabov and Song, 2002) and off-line algorithm suffer from the following disadvantages: C C Update ajk yi(t+1) = yi(t)-01.ME/Myi(t) ⎡ ∂ hl ⎢ ∑ wl y l ∂ a jk k ⎢ ⎢⎣ m ∑hw i =1 i i ⎤ ⎥ ⎥⎦ (3) ∑hw ∂E ⎢ l ∈A = − (Tp − zp). ⎢ ∂ a jk 2 where, hi wi ∂E = − (Tp − z p ). m ∂ yi i ⎤ ⎛ ∂ hl ⎞ ⎥⎥ ⎟⎟ ⋅ ⎜⎜ ∑ wl m ⎝ l ∈Ak ∂ b jk ⎠ ⎥ hi wi ∑ ⎥ i =1 ⎦ zp ⎡ ( x j − a jk ) ∂hl where, = hl ⋅ ⎢ 3 ∂b jk ⎢⎣ (b jk ) C Update yi i =1 i =1 (6) ⎤ ⎥ ⎛ ∂ hl ⎞ ⎥ ⎜ ⎟⎟ − m 2 . ⎜ ∑ wl ⎛ ⎞ ⎝ l ∈ Ak ∂ a jk ⎠ ⎥ ⎥ ⎜ ∑ hi wi ⎟ ⎝ i =1 ⎠ ⎥⎦ ∂ hl ⎡ ⎢ ∑ wl y l ∂ a zp l ∈ Ak jk . m = − (Tp − z p ). ⎢ m ⎢ hi wi ∑ ⎢ ∑ hi wi i =1 i =1 ⎣ C m ∑hw y i =1 i i i ⎤ ⎛ ∂ hl ⎞ ⎥⎥ ⎜ ⎟ ⋅ ⎜ ∑ wl ⎟ ⎝ l ∈ Ak ∂ a jk ⎠ ⎥ ⎥ ⎦ ⎡ ( x j − a jk ) ⎤ ∂ hl = hl ⋅ ⎢ where, ⎥ 2 ∂ a jk ⎣⎢ (b jk ) ⎥⎦ C (5) C C C Update bjk 4014 Fixed-grid partition algorithm suffers from exponential growth of fuzzy rules, resulting in curse of dimensionality. In addition, not all fuzzy sets are effective for inference. User-defined thresholds are required to control creation of new fuzzy set at input space. User-defined threshold is required to control maximum width of each fuzzy set. Smaller threshold will generate many fuzzy sets, but possibly result in over-fitting. Larger value can reduce number of fuzzy sets, but poor accuracy resulted. The thresholds are generally decided by trial-anderror process. Optimal value is not easy to determine. Most of methods are based on covering all input or input/output data by existing and new creation of partitions/clusters. Nevertheless, they have no relation with system accuracy. In other words, optimal number of fuzzy sets is not determined to achieve desired or maximized approximation accuracy. Res. J. Appl. Sci. Eng. Technol., 4(20): 4012-4021, 2012 Towards improving such disadvantages, in our proposed SDSA-FNN, there is no system existing at scratch. Initial fuzzy rule/sets which are created by first incoming training data vector are started with small width. When following input data vector is not covered locally, algorithm will automatically decide expansion of existing fuzzy sets or addition action in direction of achieving higher system accuracy. However, addition of new fuzzy set will definitely reduce system error but also increase number of fuzzy rules. In order to avoid this problem, i.e., curse of dimensionality, new created fuzzy rules will be tested to ascertain its novelty. If contribution is not significant, those fuzzy rules associated with new created fuzzy sets will be discarded in order to maintain compact and interpretable rule base. Contribution degree or significance of fuzzy rule is formulated below: By assuming wi = 1 in (1), it can be transformed into following compact matrix formulation: D = H2 + E q = [y1, y2, ..., ym]T0Un E = [g1, g2, …, gP]T By orthogonal decomposition, we have: H = PA (8) where, P = (p1, p2, …, pm)0UP×m is one orthogonal matrix; A 0Um×m is one upper triangular matrix. Substituting (8) into (7), we obtain: D = PA2+E = PG+E (9) Since P is an orthogonal matrix, sum of squares of D is given by: m DT D = ∑g i =1 2 i piT pi + E T E (10) If D is desired output after its mean has been removed, variance of D is given by: (7) where, D = [Y1, Y2 , …, YP]T0UP H = [h1, h2, …., hm] 0UP×m m m −1 D T D = m −1 ∑ gi2 piT pi + m −1 E T E i =1 Fig. 2: Flowchart of evolving-constructions of system structure 4015 (11) Res. J. Appl. Sci. Eng. Technol., 4(20): 4012-4021, 2012 As a result, error reduction ratio or significance of fuzzy rules due to pi is calculated by: erri = gi2 piT pi DT D , i = 1, 2, ..., m Hybrid of Genetic Algorithm (GA) and Gradient Descent (GD) based learning algorithm: After this evolving-construction of system structure is decided through learning from dataset, next step is to optimize parameters involved in this system, i.e. ajk, bjk, yi and wi which can be updated by Eq. (2) ~ (6). However, adopting hybrid of GA and GD based learning algorithm can speed up searching optimal minimum. (12) where, gi = piT D piT p The flowchart of evolving construction of model is shown in Fig. 2. , i = 1, 2, ..., m. In this way, system structure will gradually evolve to carry out forecasting action. Fuzzy sets on each fuzzy rule will automatically expand until new fuzzy set/rules are required indeed. In summary, advantage of our proposal can be described as follows: C System structure will gradually evolve, taking into account not only minimization of fuzzy rule generation, but also system accuracy. C Beside system accuracy, system will also adapt to cover all data in input space. C Number of fuzzy sets in each input variable can be different which has advantage than traditional methods with equal number of fuzzy sets on each input dimension. As a result, generated fuzzy rules can be minimized, resulting in higher computational efficiency because number of fuzzy rules which is equal to M1 ´ M2 ´ … ´ Mn (where,,, Mn is number of fuzzy subset for n-th input variable) can be reduced. C Each local fuzzy set’s width at input space is set small positive value initially. They can automatically expand to maximum value until creation of new fuzzy set is required indeed. C There is no a-prior user-defined threshold required in algorithm. Only desired system accuracy, i.e., e, is required for decision of expansion of existing fuzzy sets or creation of new one. Genetic Algorithm (GA) is a search algorithm based on the mechanics of natural selection. Its advantages in this study are remarkable as below: C GA is a global optimization algorithm with parallel searching capability. C Network parameters are firstly encoded into chromosomes. Since number of fuzzy sets/rules and associated parameters have been decided by our proposed evolving-construction of fuzzy system, the GA’s chromosome contains only the parameters to be optimized as shown in the following form: C Optimization process is guided by fitness function, which is represented by cost function (13). As a result, any complex calculations involving on partial differentiation Eq. (2) ~ (6) is not required during optimization process: (a) (b) Fig. 3: Proposed gradual evolution of fuzzy system 4016 Res. J. Appl. Sci. Eng. Technol., 4(20): 4012-4021, 2012 FFitness = C P (Tp − z p )2 p =1 2 ∑ C + FPenalty (13) C Penalty can be easily integrated into fitness function as shown in (13). Since MFs sought by GA would be very similar, we add one penalty term FPenalty in this fitness function. If similarity degree of resulted MFs which is calculated by the method reported, are too big, the respective penalty term will be added in fitness function. In this way, the generated MFs are not overlapped much. Following GA, GD algorithm as described in Eq. (2) ~ (6) is used for further searching an optimal solution. With initial starting point given by GA, convergence is much rapid and easy as observed in simulations. Incremental refinement of system structure and parameters adaptable to new operation environment: After evolving-constructions of this system structure as well as the parameters are properly settled down, this machine is ready for carrying out desired tasks. Nevertheless, training dataset cannot cover all possible operation conditions. Therefore, it is necessary to continuously refine and expand this data-driven evolving machine to suit for new operation conditions. This requires an update of system parameters and/or evolution of system structure. Traditionally, complete re-training of whole system is proceeded with a new dataset. Its disadvantages are low computational efficiency. Moreover, newly build-up system might be completely different structure and no longer suitable for old operation conditions. This will be not suitable for electric load forecasting application because electric load is characterized with periodic variations. On the other side, if old dataset is also accumulated in new training phase, computing cost will be very intensive. Extra computing memory is also required to store all old dataset. To tackle such a problem, a novel methodology, i.e., incremental refinement of system structure and update of necessary parameters is proposed as shown in Fig. 3. Its advantages are: C C C C Old operation conditions are still covered by this system’s structure. New operation conditions are covered by new evolving part added to existing structure. Least contribution of system parts (including fuzzy rules) is removed. Only new added substructure and associated parameters are required to update. There is no required computing memory to store all old dataset. It is fast to achieve a new appropriate system. Adaptable to new operation environment: C C New fuzzy set is discovered, resulting in new fuzzy rule and parts (a k, b k, h +1, y +1, w +1) added at old system structure. System continues to evolve by adding more fuzzy rules and parts (a k+1, b k+1, h +2, y +2, w +2) on old structure to suit for new operation environment. Since newly added nodes have no effect on old system structure, parameters in new evolving substructure of system can be settled down by (assuming that old system structure has m fuzzy rules and new evolving substructure results in m fuzzy rules): m′ m znew = ∑ y h w + ∑ y ′h ′w ′ i i i =1 m ∑ i =1 i hi wi + i i i = m+ 1 m′ ∑ i (14) hi′wi′ i = m+1 where, yi, hi, wi : Parameters at old system structure y , h , w : Parameters at new evolving substructure m : Number of fuzzy rules in old system structure m Number of new fuzzy rules in new evolving substructure znew : System output respective to new operation environment Tnew : Targeted output in new operation environment, which is shown in following formulas. C Update of new y: yi′ (t + 1) = yi′ (t ) − η1 ⋅ ∂ E / ∂ yi′ (t ) ∂E = − (Tnew − znew ) ⋅ ∂ yi′ hi′wi′ m ∑ hw i =1 C i i + m ∑ h ′w ′ i = m+ 1 i i Update of new w: wi′ (t + 1) = wi′ (t ) − η2 ⋅ ∂ E / ∂ wi′ (t ) ∂E yi′ − znew = − (Tnew − znew ) ⋅ hi′ m m ∂ wi′ ∑ hi wi + ∑ hi′wi′ i =1 C 4017 (15) Update of new a k: i = m+ 1 (16) Res. J. Appl. Sci. Eng. Technol., 4(20): 4012-4021, 2012 Fig. 4: Function diagram of proposed novel methodology Table 1: Performance comparsion in rmse Algorithm Number of rules/neurons DFNN [8] 6 GDFNN [17] 6 SOFNN [18] 5 ECSFS [12] 16 SDSA-FNN 12 Number of parameters 48 48 46 48 36 a ′jk (t + 1) = a ′jk (t ) − η3 ⋅ ∂ E / ∂ a ′jk (t ) ∂E = − (Tnew − z new ). ∂ a ′jk ∂ hl′ ⎡ ⎢ ∑ wl′ y l′ ∂ a ′ jk ⎢ l ∈Ak′ − m ⎢ m ⎢ ∑ hi wi + ∑ hi′wi′ i = m+1 ⎢ i =1 ⎢ ⎛ ∂ hl′ ⎞ ⎟⎟ ⎢ ⋅ ⎜⎜ ∑ wl ⎢⎣ ⎝ l ∈Ak′ ∂ a ′jk ⎠ where, C RMSE (testing) N.A. N.A. 0.0151 0.01292 0.01094 where, ⎡ ( xj − a ′jk ) ∂ hl′ = hl′ ⋅ ⎢ 3 ∂ b ′jk ⎢⎣ (b ′jk ) (17) ⎤ ⎥ ⎥ m m ⎥ hi wi + ∑ hi′wi′ ⎥ ∑ i =1 i = m+ 1 ⎥ ⎥ ⎥ ⎦⎥ z new ⎡ ( x j − a ′jk ) ⎤ ∂ hl′ = hl′ ⋅ ⎢ ⎥ 2 ∂ a ′jk ⎢⎣ (b ′jk ) ⎥⎦ where, A : a k, b k : h1 , h 2 , h 3 , h 4 : 2 ⎤ ⎥ ⎥⎦ Set of all pre-condition nodes h which have connections with this a k/b k Parameters at new evolving substructure Learning rates, which are small positive numbers Summarizing the aforementioned sections, this novel methodology is depicted in Fig. 4. Update of new b: SIMULATION RESULTS b ′jk (t + 1) = b ′jk (t ) − η4 ⋅ ∂ E / ∂b ′jk (t ) ∂E = − (Tnew − z new ) ⋅ ∂ b ′jk ∂ hl′ ⎡ ⎢ ∑ wl′ y l′ ∂ b ′ l ∈ A′ k jk ⎢ − m′ ⎢ m + h w h w ′ ′ ∑ ∑ ⎢ i i i i i = m+ 1 ⎢ i =1 ⎢⎛ ∂ hl′ ⎞ ⎟ ⎢ ⎜⎜ ∑ wl′ ∂ b ′jk ⎟⎠ ⎢⎣ ⎝ l ∈ Ak′ RMSE (training) 0.02830 0.02410 0.01570 0.01374 0.01149 (18) ⎤ ⎥ ⎥ ⋅ m m′ ⎥ hi wi + ∑ hi′wi′ ⎥ ∑ i =1 i = m+ 1 ⎥ ⎥ ⎥ ⎥⎦ znew The proposed methodology has been implemented in Matlab environment. Before we apply our method in application of electric load forecasting, simulation tests for different function approximations are firstly carried out and demonstrated below: A nonlinear dynamic system: We consider one nonlinear dynamic system which was used by ECSFS, DFNN, GDFNN and SOFNN as below. The task is to predict system output y(t+1) from system input vector of [y(t),y(t-1),u(t)]: 4018 Res. J. Appl. Sci. Eng. Technol., 4(20): 4012-4021, 2012 Table 2: Performance comparsion in NDEI Number of Methods rules/nodes/units Neural gas [24] 1000 RAN[25] 113 ESOM[26] 114 ESOM[26] 1000 EFuNN[27] 193 EFuNN[27] 1125 DENFIS[10] 58 DENFIS[10] 883 y (t + 1) = NDEI (testing) 0.062 0.373 0.320 0.044 0.401 0.094 0.276 0.042 y (t ) y (t − 1) y (t ) + 2.5 + u( t ) 1 + y (t ) 2 + y (t − 1) 2 Methods DENFIS with insertion[10] eTS[20] SAFIS[19] ECSFS[12] ECSFS[12] ECSFS[12] ECSFS[12] SDSA-FNN Number of rules /nodes/units 883 10 21 48 56 64 72 8 NDEI (testing) 0.0330 0.3310 0.3800 0.2570 0.2340 0.2140 0.2030 0.0849 (19) where, u(t) = sin (2Bt/25) y(0) = 0 and y(1) = 0. In this simulation, 200 data with tÎ[0, 200] are created to train and 200 data with tÎ[400, 600] are created to test. The performance is compared with ECFS, DFNN, GDFNN and SOFNN in RMSE as reported in Table 1. Mackey glass chaotic time series: Next, we consider a chaotic time series, i.e. Mackey Glass dataset, which has been used as a benchmark example to model and forecast performance comparison. This time series is created by using the MG timedelay differential defined as: Fig. 5: Distributions of fuzzy sets on each input dx (t ) 0.2 x (t − τ ) = − 01 . x(t ) dt 1 + x 10 (t − τ ) (20) where, x(0) = 1.2, x(t) = 0 for t<0 and t = 1.7. Out task is to predict x(t + 85) from the input vector of [x(t-18), x(t-12), x(x-6), x(t)]. Three thousand data with tÎ[201, 3200] are created to train and 500 data with tÎ[5001, 5500] are created for test. The performance index NDEI (nondimensional error index) is used to compare with DENFIS, eTS, SAFIS, ECSFS and several other methods as shown in Table 2. Load forecasting application: Finally, we test our proposed SDSA-FNN on application of electric load forecasting. Macao power system is used as a test bed. Its historical electric loads are collected from January to August of 2010.This SDSA-FNN in this application has five input variables and one output variable as below. Input variable Ld(t-1) : Electric load on dth day and (t-1)th time interval Ld(t-2) : Electric load on dth day and (t-2)th time interval Output variable Ld(t) : Electric load on dth day and tth time interval Ld(t-3) : Electric load on dth day and (t-3)th time interval Fig. 6: System’s structure with the sought fuzzy sets on input variables Ld-1(t) : Electric load on (d-1)th day and tth time interval Td : Average temperature value on dth day The training datasets are fed one by one for learning and evolving-construction of system structure. After this phase is completed, system parameters identification and optimization are followed accordingly. At the end, our approach finds out fuzzy set distribution on each input variable as demonstrated in Fig. 5. Consequently, eight fuzzy rules are generated which resulted in system structure as shown in Fig. 6. The system output and actual electric load are compared by using performance index MAPE (Mean Absolute Percentage Error).With the sought fuzzy rules, we obtain MAPE value as 1.7423% 4019 Res. J. Appl. Sci. Eng. Technol., 4(20): 4012-4021, 2012 Electric load (MW) 500 Real SDSA-FNN 450 400 350 300 250 0 20 40 60 80 100 Hours 120 140 160 180 Fig. 7: Seven-day load forecasting (Real: actual electric load data; SDSA-FNN: simulated data) Real SDSA-FNN 450 Fig. 11: New fuzzy sets/rules evolved and added on existing system in order to suit for new environment 400 750 350 300 250 0 20 40 60 80 100 Hours 120 140 160 180 Fig. 8: Seven-day load forecasting (Real: actual electric load data; NN: simulated data by traditional neural networks) 750 650 600 550 500 450 400 0 Real SDSA-FNN 700 Electric load (MW) Real SDSA-FNN 700 Electric load (MW) Electric load (MW) 500 20 40 60 Hours 80 100 120 Fig. 12: System performance after new parts added on existing system (Real: actual electric load data; SDSA-FNN: simulated data) 650 600 550 500 450 400 0 20 40 60 Hours 80 100 120 Fig. 9: System performance from system without any selfrefinement capability (Real: actual electric load data; SDSA-FNN: simulated data) Fig. 10: New evolved and added parts on existing system to suit for new operation environment for seven-day ahead electric load forecasting. Such a result is better with the frequently used forecasting tool From it, we can see that our approach can obtain better system accuracy than other methods for this simple function approximation. Our approach does not require any user-defined thresholds which are decided by trialand-error process. based on traditional neural networks which has MAPE value 1.8318% as shown in Fig. 7 and 8. However, our approach has advantage over traditional methods because it simultaneously achieves the objective of system accuracy, interpretability and transparency for users while traditional method results in a blackbox. To demonstrate our SDSA-FNN self-adaptive and self-learning capability in a new operation environment, Fig. 9 shows that existing system without any selfrefinement capability results in 3.1584% MAPE value. It is apparently not suitable for new operation environment. However, with our SDSA-FNN, it can learn and evolve new substructure in order to survive in this new environment as demonstrated in Fig. 10 and 11. With extra fuzzy sets on input variable Ld(t-3) and Tavg respectively, MAPE value is consequently reduced to 1.8865% as shown in Fig. 12. 4020 Res. J. Appl. Sci. Eng. Technol., 4(20): 4012-4021, 2012 The following main advantages can be summarized from simulation results C C C C B B B C The proposed methodology can automatically optimize system structure in direction of achieving higher system accuracy Unlike many existing algorithms, proposed methodology doesn’t require different user-defined thresholds to assist determination of system structure Complete re-training is not required for whole system so as to suit for new operation environment. Instead, the gradual re-refinement and expansion of existing system can keep its applicability in changing environment The apparent advantages of our approach are: Fast on achieving new applicable system Eliminate memorizing old datasets Applicable to both past and new operation environments This is favorable to application of electric load forecast because electric load profile characterizes repeated and periodic variable CONCLUSION The proposed SDSA-FNN is firstly compared with various methods applied on function approximations. The simulations reveal that it has better performances than other methods. Afterwards, it is applied on electric load forecasting application and verified on electric load data recorded on Macao power system. The simulation results reveal that the proposed methodology not only keeps the traditional objective function, i.e., forecasting accuracy, but also enhance this forecaster endorsing with human intelligence and smartness. In addition, complete retraining is not required for whole system so as to suit for new operation environment. Instead, the gradual rerefinement and expansion of existing system can keep its applicability in changing environment. The apparent advantages of our approach are: C C C Fast on achieving new applicable system Eliminate memorizing old datasets Applicable to both past and new operation environments REFERENCES Angelov, P. and R. Buswell, 2002. Identification of evolving fuzzy rule-based models. IEEE T. Fuzzy Syst., 10(5): 667-677. Jang, J.S. and C.T. Sun, 1993. Functional equivalence between radial basis function networks and fuzzy inference systems. IEEE T. Neural Netw., 4(1): 156159. Juang, C.F. and C.T. Lin, 1998. An on-line selfconstructing neural fuzzy inference network and its applications. IEEE T. Fuzzy Syst., 6(1): 12-32. Kasabov, N.K. and Q. Song, 2002. DENFIS: Dynamic evolving neural-fuzzy inference system and its application for time-series prediction. IEEE T. Fuzzy Syst., 10(2): 144-154. Koczy, L., D. Tikk and T. Gedeon, 2000. On functional equivalence of certain fuzzy controllers and RBF type approximation schemes. Int. J. Fuzzy Syst., 2(3): 164-175. Shi, Y., M. Mizumoto and P. Shi, 2002. Fuzzy if-then rule generation based on neural network and clustering algorithm techniques. Proc. IEEE Tencon, 02: 651-654. Wang, L., 1992. Fuzzy systems are universal approximators. Proceeding of 1st IEEE Conference Fuzzy System, San Diego, CA, pp: 1163-1169. Wang, L.X. and J.M. Mendel, 1992. Fuzzy basis functions, universal approximation and orthogonal least-squares learning. IEEE T. Neur. Network., 3: 807-814. Wu, S. and M.J. Er, 2000. Dynamic fuzzy neural networks-a novel approach to function approximation. IEEE T. Syst. Man Cy. B, 30(2): 358-364. 4021