Research Journal of Applied Sciences, Engineering and Technology 4(20): 4012-4021, 2012
ISSN: 2040-7467
© Maxwell Scientific Organization, 2012
Submitted: December 20, 2011
Accepted: April 23, 2012
Published: October 15, 2012
Intelligent Self-developing and Self-adaptive Electric Load Forecaster
based on Adaptive FNN+GA+GD
Chinwang Lou and Mingchui Dong
Faculty of Science and Technology, University of Macao, China
Abstract: In this study, a novel electric load forecaster based on adaptive Fuzzy Neural Networks (FNN) and
using Genetic Algorithm (GA) mixed with Gradient Descent (GD) is proposed to make it to posses the human
learning ability. The proposed SDSA-FNN is firstly compared with various methods applied on function
approximations. Moreover, it is applied on electric load forecasting application and verified on electric load
data recorded on Macao power system. The simulation results reveal that the proposed methodology not only
keeps the traditional objective function.
Keywords: Fuzzy neural networks, smart grid, smart load forecaster, self-developing, self-adaptive
INTRODUCTION
In new advocated “smart grid” development, a lot of
renewable energy sources, i.e., local Wind Farms (WF),
Photovoltaic (PV) cells, battery storages, Electric
Vehicles (EV) and huge various sensors are widely
installed and distributed at customer side. This will make
significant challenges to existing technologies of electric
load forecasting because higher uncertainty,
indefiniteness and variability would occur at demand side.
Old techniques focusing on only forecasting accuracy are
no longer adequate. New technologies are demanded to
offer smart capabilities in handling the changing operation
environment, i.e. self-learning, self-development, selfadaptability, intelligence, transparency and accuracy.
Firstly, in our proposed SDSA-FNN (SelfDeveloping and Self-Adaptive FNN), algorithm will
automatically determine optimal system structure in
direction of achieving higher system accuracy. However,
using more complex system structure will definitely
reduce system error, but number of generated fuzzy rules
will also increase, resulting in curse of dimensionality. A
special technique in this study is used to avoid this
problem.
Secondly, when external operation environment is
changed, traditional method is to re-train this whole
system, resulting in completely different system structure.
This new system might be no longer suitable for past
operation conditions. This is not favorable for electric
load forecasting application because electric load is
characterized with repeated and periodic variation.
However, in our novelty, this complete re-training is not
necessary. System can be continuously refined such as to
let past and new operation conditions are simultaneously
suitable. This function is very important for application of
electric load forecasting because electric load reveals
periodic variation.
The proposed methodology is finally applied and
tested on power system of Macao which lies on the
western side of the Pearl River Delta and faces the South
China Sea. The simulation results reveal that proposed
methodology not only keeps the traditional objective
function, i.e., accuracy, but also enhance traditional
forecaster with intelligence and smartness, resulting in a
smart load forecaster.
METHODOLOGY
Interpretable fuzzy model for forecasting system: Our
proposal is based on Takagi and Sugeno (1985) fuzzy
model proposed in Takagi and Sugeno (1985), Wang and
Mendel (1992) and Shi et al. (2002). T-S fuzzy model
possess nice property to be able to approximate any realoccurring nonlinear relation to a certain degree of
accuracy (Wang, 1992).
Network structure of this MISO (Multi-Input SingleOutput) fuzzy model is shown in Fig. 1, which has
following rule form. However, it has no difficulty to
extend into MIMO (Multi-Input and Multi-Output)
system.
Rule i: If x1 is Ai1 and x2 is Ai2 and and xn is Ain then zi
is yi with wi
where,
x : Input vector [x1, x2, xn]T
zi : Output for ith fuzzy rule
Aij : Fuzzy set for jth input variable xj in ith rule (i = 1, 2,
m; j = 1, 2, n)
yi : ith fuzzy rule weighting between normalization
node and output node
wi : ith fuzzy rule confidence between normalization
node and output node
Corresponding Author: Chinwang Lou, Faculty of Science and Technology, University of Macao, China
4012
Res. J. Appl. Sci. Eng. Technol., 4(20): 4012-4021, 2012
Fig. 1: Takagi-sugeno MISO fuzzy system
m: Number of fuzzy rules
n: Number of input variables
rule due to activation of its antecedent by the observed
input x:
Assuming Gaussian-type Membership Function (MF)
is adopted for fuzzy set of input variable due to its good
approximation ability, infinite support, easy extraction of
parameters and equivalency to RBFs neural networks
(Jang and Sun, 1993; Koczy et al., 2000). Complete I/O
relation of this fuzzy system can be formulated as below
(Remark: I is node input, O is node output, subscript and
superscript represent node and layer index, respectively):
I i3 = Ai1 Ai 2 ... Ain , Oi3 = hi = I i3
where, i = 1, 2, m (m = M1´M2´ ´Mn).
Layer 4-normalization nodes: Number of nodes in this
layer is equal to that in layer 3. The firing strength
calculated in layer 3 is normalized in this layer:
Layer 1-input nodes: Each node in this layer, which
represents an input variable to FNNs, directly transmits
input vector x = [x1, x2, xn]T to next layer as described
below:
I i4 =
hi
m
∑h
i
i =1
Layer 5-output nodes: Node in this layer represents an
output variable of the FNNs. The function of this node is
to combine the incoming signals from layer 4 in order to
produce an output response z to the input vector x:
I 1j = x j , O1j = x j
where, j = 1, 2, n.
Layer 2-fuzzification nodes: Each node in layer 2
represents one linguistic label of one of the input
variables. The output of this node specifies the degree to
which an input xj belongs to kth fuzzy set of the associated
input node j in layer 1:
m
z=
∑ yhw
i =1
m
∑
i =1
I 2jk
, Oi4 = I i4
⎡ ( x j − a jk ) 2 ⎤
2
= x j , O jk = A jk = exp ⎢ −
⎥
2b 2jk
⎢⎣
⎥⎦
where, ajk and bjk are center and width of this linguistic
label respectively, j = 1, 2, .., n; k = 1, 2, Mj and Mj is
number of fuzzy sets on jth input variable.
Layer 3-pre-condition nodes: Each node in this layer
represents one fuzzy logic rule and performs antecedent
matching of this rule. The output of each node in this
layer represents the firing strength of corresponding fuzzy
i i
i
(1)
hi wi
Parameter determination: Parameter determination
includes all ajk and bjk associated with center and width of
jth fuzzy set of ith input variable in premise part, all yi and
wi in consequent part (Fig. 1). The derivation of parameter
optimization is presented below for the system structure
as depicted in Fig. 1.
Given P-paired multi-input and single output data
samples (Xp, Tp) where,,, Xp = (xp1, xp2, xpn) where, p = 1,
2, P. Learning objective is to minimize cost function (2)
by finding optimal parameters ajk, bjk, yi and wi.
4013
Res. J. Appl. Sci. Eng. Technol., 4(20): 4012-4021, 2012
Min E =
P
∑E
p=1
P
=
P
(Tp − z p ) 2
p =1
2
∑
b jk (t + 1) = b jk (t ) − η4 ⋅ ∂E / ∂b jk (t )
⎡
m
∂ hi
⎢ ∑ wl y l
hi wi yi
∑
∂ b jk
⎢ l ∈A
∂E
i =1
= − (Tp − zp) ⋅ ⎢
−
2
m
∂ bjk
⎛ m
⎞
⎢ ∑ hi wi
⎜
⎟
(2)
⎤
⎥
⎛
∂ hl ⎞ ⎥
⎜
⎟
⋅ ⎜ ∑ wl
⎟⎥
⎝ l ∈Ak ∂ b jk ⎠ ⎥
hi wi
∑
⎥
⎝ j =1
⎠
⎦
k
⎢
⎣
where,
Tp : Targeted output at pth input-output sample;
zp : System output for pth input sample;
∂ hi
⎡
⎢ ∑ wl y l ∂ b
l ∈ Ak
jk
⎢
−
= − (Tp − z p ) ⋅
m
⎢
w
h
⎢ ∑ i i
i =1
⎣
Those parameters can be adjusted with following
equations:
yi(t+1) = yi(t)-01.ME/Myi(t)
Ak
: Set of all pre-condition nodes hl which have
connections with this ajk/bjk;
h1,h2,h3,h4 : Algorithm learning rates which are small
positive numbers
i
C Update wi
wi (t + 1) = wi (t ) − η2 . ∂ E / ∂ wi (t )
m
⎡
⎤
⎢
hi wi yi ⎥
∑
∂E
y
⎥
= − (Tp − z p ). hi ⎢⎢ m i − i =m1
2
∂ wi
⎛
⎞ ⎥
⎢ ∑ hi wi ⎜ ∑ hi wi ⎟ ⎥
⎝ i =1
⎠ ⎥⎦
⎢⎣ i =1
yi − z p
= − (Tp − z p ). hi m
∑ hi wi
(4)
i =1
Incremental evolving-construction of fuzzy system:
Construction of proposed fuzzy system involves learning
of premise structure and determination of parameters on
fuzzy rule premise and consequent part. In existing
literature, premise structure is determined either by offline or on-line algorithm.
However, all techniques referred in online algorithm
(Juang and Lin, 1998; Wu and Er, 2000; Angelov and
Buswell, 2002; Kasabov and Song, 2002) and off-line
algorithm suffer from the following disadvantages:
C
C Update ajk
yi(t+1) = yi(t)-01.ME/Myi(t)
⎡
∂ hl
⎢ ∑ wl y l
∂ a jk
k
⎢
⎢⎣
m
∑hw
i =1
i
i
⎤
⎥
⎥⎦
(3)
∑hw
∂E
⎢ l ∈A
= − (Tp − zp). ⎢
∂ a jk
2
where,
hi wi
∂E
= − (Tp − z p ). m
∂ yi
i
⎤
⎛
∂ hl ⎞ ⎥⎥
⎟⎟
⋅ ⎜⎜ ∑ wl
m
⎝ l ∈Ak ∂ b jk ⎠ ⎥
hi wi
∑
⎥
i =1
⎦
zp
⎡ ( x j − a jk )
∂hl
where,
= hl ⋅ ⎢
3
∂b jk
⎢⎣ (b jk )
C Update yi
i =1
i =1
(6)
⎤
⎥
⎛
∂ hl ⎞ ⎥
⎜
⎟⎟
− m
2 . ⎜ ∑ wl
⎛
⎞ ⎝ l ∈ Ak ∂ a jk ⎠ ⎥
⎥
⎜ ∑ hi wi ⎟
⎝ i =1
⎠
⎥⎦
∂ hl
⎡
⎢ ∑ wl y l ∂ a
zp
l ∈ Ak
jk
. m
= − (Tp − z p ). ⎢
m
⎢
hi wi
∑
⎢ ∑ hi wi
i =1
i =1
⎣
C
m
∑hw y
i =1
i
i
i
⎤
⎛
∂ hl ⎞ ⎥⎥
⎜
⎟
⋅ ⎜ ∑ wl
⎟
⎝ l ∈ Ak ∂ a jk ⎠ ⎥
⎥
⎦
⎡ ( x j − a jk ) ⎤
∂ hl
= hl ⋅ ⎢
where,
⎥
2
∂ a jk
⎣⎢ (b jk ) ⎥⎦
C
(5)
C
C
C Update bjk
4014
Fixed-grid
partition algorithm suffers from
exponential growth of fuzzy rules, resulting in curse
of dimensionality. In addition, not all fuzzy sets are
effective for inference.
User-defined thresholds are required to control
creation of new fuzzy set at input space.
User-defined threshold is required to control
maximum width of each fuzzy set. Smaller threshold
will generate many fuzzy sets, but possibly result in
over-fitting. Larger value can reduce number of fuzzy
sets, but poor accuracy resulted.
The thresholds are generally decided by trial-anderror process. Optimal value is not easy to determine.
Most of methods are based on covering all input or
input/output data by existing and new creation of
partitions/clusters. Nevertheless, they have no
relation with system accuracy. In other words,
optimal number of fuzzy sets is not determined to
achieve desired or maximized approximation
accuracy.
Res. J. Appl. Sci. Eng. Technol., 4(20): 4012-4021, 2012
Towards improving such disadvantages, in our
proposed SDSA-FNN, there is no system existing at
scratch. Initial fuzzy rule/sets which are created by first
incoming training data vector are started with small width.
When following input data vector is not covered locally,
algorithm will automatically decide expansion of existing
fuzzy sets or addition action in direction of achieving
higher system accuracy. However, addition of new fuzzy
set will definitely reduce system error but also increase
number of fuzzy rules. In order to avoid this problem, i.e.,
curse of dimensionality, new created fuzzy rules will be
tested to ascertain its novelty. If contribution is not
significant, those fuzzy rules associated with new created
fuzzy sets will be discarded in order to maintain compact
and interpretable rule base. Contribution degree or
significance of fuzzy rule is formulated below:
By assuming wi = 1 in (1), it can be transformed into
following compact matrix formulation:
D = H2 + E
q = [y1, y2, ..., ym]T0Un
E = [g1, g2, …, gP]T
By orthogonal decomposition, we have:
H = PA
(8)
where, P = (p1, p2, …, pm)0UP×m is one orthogonal matrix;
A 0Um×m is one upper triangular matrix.
Substituting (8) into (7), we obtain:
D = PA2+E = PG+E
(9)
Since P is an orthogonal matrix, sum of squares of D is
given by:
m
DT D =
∑g
i =1
2
i
piT pi + E T E
(10)
If D is desired output after its mean has been removed,
variance of D is given by:
(7)
where,
D = [Y1, Y2 , …, YP]T0UP
H = [h1, h2, …., hm] 0UP×m
m
m −1 D T D = m −1 ∑ gi2 piT pi + m −1 E T E
i =1
Fig. 2: Flowchart of evolving-constructions of system structure
4015
(11)
Res. J. Appl. Sci. Eng. Technol., 4(20): 4012-4021, 2012
As a result, error reduction ratio or significance of fuzzy
rules due to pi is calculated by:
erri =
gi2 piT pi
DT D
, i = 1, 2, ..., m
Hybrid of Genetic Algorithm (GA) and Gradient
Descent (GD) based learning algorithm: After this
evolving-construction of system structure is decided
through learning from dataset, next step is to optimize
parameters involved in this system, i.e. ajk, bjk, yi and wi
which can be updated by Eq. (2) ~ (6). However, adopting
hybrid of GA and GD based learning algorithm can speed
up searching optimal minimum.
(12)
where,
gi =
piT D
piT p
The flowchart of evolving construction of model is
shown in Fig. 2.
, i = 1, 2, ..., m.
In this way, system structure will gradually evolve to
carry out forecasting action. Fuzzy sets on each fuzzy rule
will automatically expand until new fuzzy set/rules are
required indeed. In summary, advantage of our proposal
can be described as follows:
C System structure will gradually evolve, taking into
account not only minimization of fuzzy rule generation,
but also system accuracy.
C Beside system accuracy, system will also adapt to cover
all data in input space.
C Number of fuzzy sets in each input variable can be
different which has advantage than traditional methods
with equal number of fuzzy sets on each input
dimension. As a result, generated fuzzy rules can be
minimized, resulting in higher computational efficiency
because number of fuzzy rules which is equal to M1 ´
M2 ´ … ´ Mn (where,,, Mn is number of fuzzy subset
for n-th input variable) can be reduced.
C Each local fuzzy set’s width at input space is set small
positive value initially. They can automatically expand
to maximum value until creation of new fuzzy set is
required indeed.
C There is no a-prior user-defined threshold required in
algorithm. Only desired system accuracy, i.e., e, is
required for decision of expansion of existing fuzzy sets
or creation of new one.
Genetic Algorithm (GA) is a search algorithm based on
the mechanics of natural selection. Its advantages in this
study are remarkable as below:
C GA is a global optimization algorithm with parallel
searching capability.
C Network parameters are firstly encoded into
chromosomes. Since number of fuzzy sets/rules and
associated parameters have been decided by our
proposed evolving-construction of fuzzy system, the
GA’s chromosome contains only the parameters to be
optimized as shown in the following form:
C Optimization process is guided by fitness function,
which is represented by cost function (13). As a result,
any complex calculations involving on partial
differentiation Eq. (2) ~ (6) is not required during
optimization process:
(a)
(b)
Fig. 3: Proposed gradual evolution of fuzzy system
4016
Res. J. Appl. Sci. Eng. Technol., 4(20): 4012-4021, 2012
FFitness =
C
P
(Tp − z p )2
p =1
2
∑
C
+ FPenalty
(13)
C
Penalty can be easily integrated into fitness function
as shown in (13). Since MFs sought by GA would be
very similar, we add one penalty term FPenalty in
this fitness function. If similarity degree of resulted
MFs which is calculated by the method reported, are
too big, the respective penalty term will be added in
fitness function. In this way, the generated MFs are
not overlapped much.
Following GA, GD algorithm as described in Eq. (2)
~ (6) is used for further searching an optimal solution.
With initial starting point given by GA, convergence is
much rapid and easy as observed in simulations.
Incremental refinement of system structure and
parameters adaptable to new operation environment:
After evolving-constructions of this system structure as
well as the parameters are properly settled down, this
machine is ready for carrying out desired tasks.
Nevertheless, training dataset cannot cover all possible
operation conditions. Therefore, it is necessary to
continuously refine and expand this data-driven evolving
machine to suit for new operation conditions. This
requires an update of system parameters and/or evolution
of system structure. Traditionally, complete re-training of
whole system is proceeded with a new dataset. Its
disadvantages are low computational efficiency.
Moreover, newly build-up system might be completely
different structure and no longer suitable for old operation
conditions. This will be not suitable for electric load
forecasting application because electric load is
characterized with periodic variations. On the other side,
if old dataset is also accumulated in new training phase,
computing cost will be very intensive. Extra computing
memory is also required to store all old dataset.
To tackle such a problem, a novel methodology, i.e.,
incremental refinement of system structure and update of
necessary parameters is proposed as shown in Fig. 3. Its
advantages are:
C
C
C
C
Old operation conditions are still covered by this
system’s structure.
New operation conditions are covered by new
evolving part added to existing structure.
Least contribution of system parts (including fuzzy
rules) is removed.
Only new added substructure and associated
parameters are required to update.
There is no required computing memory to store all
old dataset.
It is fast to achieve a new appropriate system.
Adaptable to new operation environment:
C
C
New fuzzy set is discovered, resulting in new fuzzy
rule and parts (a k, b k, h +1, y +1, w +1) added at old
system structure.
System continues to evolve by adding more fuzzy
rules and parts (a k+1, b k+1, h +2, y +2, w +2) on old
structure to suit for new operation environment.
Since newly added nodes have no effect on old
system structure, parameters in new evolving substructure
of system can be settled down by (assuming that old
system structure has m fuzzy rules and new evolving
substructure results in m fuzzy rules):
m′
m
znew =
∑ y h w + ∑ y ′h ′w ′
i i
i =1
m
∑
i =1
i
hi wi +
i i
i = m+ 1
m′
∑
i
(14)
hi′wi′
i = m+1
where,
yi, hi, wi : Parameters at old system structure
y , h , w : Parameters at new evolving substructure
m
: Number of fuzzy rules in old system structure
m
Number of new fuzzy rules in new evolving
substructure
znew
: System output respective to new operation
environment
Tnew
: Targeted output in new operation environment,
which is shown in following formulas.
C
Update of new y:
yi′ (t + 1) = yi′ (t ) − η1 ⋅ ∂ E / ∂ yi′ (t )
∂E
= − (Tnew − znew ) ⋅
∂ yi′
hi′wi′
m
∑ hw
i =1
C
i
i
+
m
∑ h ′w ′
i = m+ 1
i
i
Update of new w:
wi′ (t + 1) = wi′ (t ) − η2 ⋅ ∂ E / ∂ wi′ (t )
∂E
yi′ − znew
= − (Tnew − znew ) ⋅ hi′ m
m
∂ wi′
∑ hi wi + ∑ hi′wi′
i =1
C
4017
(15)
Update of new a k:
i = m+ 1
(16)
Res. J. Appl. Sci. Eng. Technol., 4(20): 4012-4021, 2012
Fig. 4: Function diagram of proposed novel methodology
Table 1: Performance comparsion in rmse
Algorithm
Number of rules/neurons
DFNN [8]
6
GDFNN [17]
6
SOFNN [18]
5
ECSFS [12]
16
SDSA-FNN
12
Number of parameters
48
48
46
48
36
a ′jk (t + 1) = a ′jk (t ) − η3 ⋅ ∂ E / ∂ a ′jk (t )
∂E
= − (Tnew − z new ).
∂ a ′jk
∂ hl′
⎡
⎢ ∑ wl′ y l′ ∂ a ′
jk
⎢ l ∈Ak′
−
m
⎢ m
⎢ ∑ hi wi + ∑ hi′wi′
i = m+1
⎢ i =1
⎢ ⎛
∂ hl′ ⎞
⎟⎟
⎢ ⋅ ⎜⎜ ∑ wl
⎢⎣ ⎝ l ∈Ak′ ∂ a ′jk ⎠
where,
C
RMSE (testing)
N.A.
N.A.
0.0151
0.01292
0.01094
where,
⎡ ( xj − a ′jk )
∂ hl′
= hl′ ⋅ ⎢
3
∂ b ′jk
⎢⎣ (b ′jk )
(17)
⎤
⎥
⎥
m
m
⎥
hi wi + ∑ hi′wi′ ⎥
∑
i =1
i = m+ 1
⎥
⎥
⎥
⎦⎥
z new
⎡ ( x j − a ′jk ) ⎤
∂ hl′
= hl′ ⋅ ⎢
⎥
2
∂ a ′jk
⎢⎣ (b ′jk ) ⎥⎦
where,
A
:
a k, b k
:
h1 , h 2 , h 3 , h 4
:
2
⎤
⎥
⎥⎦
Set of all pre-condition nodes
h which have connections with this a
k/b k
Parameters at new evolving
substructure
Learning rates, which are small
positive numbers
Summarizing the aforementioned sections, this novel
methodology is depicted in Fig. 4.
Update of new b:
SIMULATION RESULTS
b ′jk (t + 1) = b ′jk (t ) − η4 ⋅ ∂ E / ∂b ′jk (t )
∂E
= − (Tnew − z new ) ⋅
∂ b ′jk
∂ hl′
⎡
⎢ ∑ wl′ y l′ ∂ b ′
l ∈ A′ k
jk
⎢
−
m′
⎢ m
+
h
w
h
w
′
′
∑
∑
⎢
i i
i i
i = m+ 1
⎢ i =1
⎢⎛
∂ hl′ ⎞
⎟
⎢ ⎜⎜ ∑ wl′
∂
b ′jk ⎟⎠
⎢⎣ ⎝ l ∈ Ak′
RMSE (training)
0.02830
0.02410
0.01570
0.01374
0.01149
(18)
⎤
⎥
⎥
⋅
m
m′
⎥
hi wi + ∑ hi′wi′ ⎥
∑
i =1
i = m+ 1
⎥
⎥
⎥
⎥⎦
znew
The proposed methodology has been implemented in
Matlab environment. Before we apply our method in
application of electric load forecasting, simulation tests
for different function approximations are firstly carried
out and demonstrated below:
A nonlinear dynamic system: We consider one
nonlinear dynamic system which was used by ECSFS,
DFNN, GDFNN and SOFNN as below. The task is to
predict system output y(t+1) from system input vector of
[y(t),y(t-1),u(t)]:
4018
Res. J. Appl. Sci. Eng. Technol., 4(20): 4012-4021, 2012
Table 2: Performance comparsion in NDEI
Number of
Methods
rules/nodes/units
Neural gas [24]
1000
RAN[25]
113
ESOM[26]
114
ESOM[26]
1000
EFuNN[27]
193
EFuNN[27]
1125
DENFIS[10]
58
DENFIS[10]
883
y (t + 1) =
NDEI
(testing)
0.062
0.373
0.320
0.044
0.401
0.094
0.276
0.042
y (t ) y (t − 1) y (t ) + 2.5
+ u( t )
1 + y (t ) 2 + y (t − 1) 2
Methods
DENFIS with insertion[10]
eTS[20]
SAFIS[19]
ECSFS[12]
ECSFS[12]
ECSFS[12]
ECSFS[12]
SDSA-FNN
Number of rules
/nodes/units
883
10
21
48
56
64
72
8
NDEI
(testing)
0.0330
0.3310
0.3800
0.2570
0.2340
0.2140
0.2030
0.0849
(19)
where, u(t) = sin (2Bt/25) y(0) = 0 and y(1) = 0.
In this simulation, 200 data with tÎ[0, 200] are created
to train and 200 data with tÎ[400, 600] are created to test.
The performance is compared with ECFS, DFNN,
GDFNN and SOFNN in RMSE as reported in Table 1.
Mackey glass chaotic time series: Next, we consider a
chaotic time series, i.e. Mackey Glass dataset, which has
been used as a benchmark example to model and forecast
performance comparison.
This time series is created by using the MG timedelay differential defined as:
Fig. 5: Distributions of fuzzy sets on each input
dx (t )
0.2 x (t − τ )
=
− 01
. x(t )
dt
1 + x 10 (t − τ )
(20)
where, x(0) = 1.2, x(t) = 0 for t<0 and t = 1.7.
Out task is to predict x(t + 85) from the input vector
of [x(t-18), x(t-12), x(x-6), x(t)]. Three thousand data with
tÎ[201, 3200] are created to train and 500 data with
tÎ[5001, 5500] are created for test. The performance index
NDEI (nondimensional error index) is used to compare
with DENFIS, eTS, SAFIS, ECSFS and several other
methods as shown in Table 2.
Load forecasting application: Finally, we test our
proposed SDSA-FNN on application of electric load
forecasting. Macao power system is used as a test bed. Its
historical electric loads are collected from January to
August of 2010.This SDSA-FNN in this application has
five input variables and one output variable as below.
Input variable
Ld(t-1) : Electric load on dth
day and (t-1)th time interval
Ld(t-2) : Electric load on dth
day and (t-2)th time interval
Output variable
Ld(t) : Electric load on
dth day and tth time interval
Ld(t-3) : Electric load on dth
day and (t-3)th time interval
Fig. 6: System’s structure with the sought fuzzy sets on input
variables
Ld-1(t) : Electric load on (d-1)th
day and tth time interval
Td
: Average temperature value on dth day
The training datasets are fed one by one for learning
and evolving-construction of system structure. After this
phase is completed, system parameters identification and
optimization are followed accordingly. At the end, our
approach finds out fuzzy set distribution on each input
variable as demonstrated in Fig. 5. Consequently, eight
fuzzy rules are generated which resulted in system
structure as shown in Fig. 6. The system output and actual
electric load are compared by using performance index
MAPE (Mean Absolute Percentage Error).With the
sought fuzzy rules, we obtain MAPE value as 1.7423%
4019
Res. J. Appl. Sci. Eng. Technol., 4(20): 4012-4021, 2012
Electric load (MW)
500
Real
SDSA-FNN
450
400
350
300
250
0
20
40
60
80 100
Hours
120
140
160 180
Fig. 7: Seven-day load forecasting (Real: actual electric load
data; SDSA-FNN: simulated data)
Real
SDSA-FNN
450
Fig. 11: New fuzzy sets/rules evolved and added on existing
system in order to suit for new environment
400
750
350
300
250
0
20
40
60
80 100
Hours
120
140
160 180
Fig. 8: Seven-day load forecasting (Real: actual electric load
data; NN: simulated data by traditional neural networks)
750
650
600
550
500
450
400
0
Real
SDSA-FNN
700
Electric load (MW)
Real
SDSA-FNN
700
Electric load (MW)
Electric load (MW)
500
20
40
60
Hours
80
100
120
Fig. 12: System performance after new parts added on existing
system (Real: actual electric load data; SDSA-FNN:
simulated data)
650
600
550
500
450
400
0
20
40
60
Hours
80
100
120
Fig. 9: System performance from system without any selfrefinement capability (Real: actual electric load data;
SDSA-FNN: simulated data)
Fig. 10: New evolved and added parts on existing system to suit
for new operation environment
for seven-day ahead electric load forecasting. Such a
result is better with the frequently used forecasting tool
From it, we can see that our approach can obtain better
system accuracy than other methods for this simple
function approximation. Our approach does not require
any user-defined thresholds which are decided by trialand-error process. based on traditional neural networks
which has MAPE value 1.8318% as shown in Fig. 7 and
8. However, our approach has advantage over traditional
methods because it simultaneously achieves the objective
of system accuracy, interpretability and transparency for
users while traditional method results in a blackbox.
To demonstrate our SDSA-FNN self-adaptive and
self-learning capability in a new operation environment,
Fig. 9 shows that existing system without any selfrefinement capability results in 3.1584% MAPE value. It
is apparently not suitable for new operation environment.
However, with our SDSA-FNN, it can learn and evolve
new substructure in order to survive in this new
environment as demonstrated in Fig. 10 and 11. With
extra fuzzy sets on input variable Ld(t-3) and Tavg
respectively, MAPE value is consequently reduced to
1.8865% as shown in Fig. 12.
4020
Res. J. Appl. Sci. Eng. Technol., 4(20): 4012-4021, 2012
The following main advantages can be summarized from
simulation results
C
C
C
C
B
B
B
C
The proposed methodology can automatically
optimize system structure in direction of achieving
higher system accuracy
Unlike many existing algorithms, proposed
methodology doesn’t require different user-defined
thresholds to assist determination of system structure
Complete re-training is not required for whole system
so as to suit for new operation environment. Instead,
the gradual re-refinement and expansion of existing
system can keep its applicability in changing
environment
The apparent advantages of our approach are:
Fast on achieving new applicable system
Eliminate memorizing old datasets
Applicable to both past and new operation
environments
This is favorable to application of electric load
forecast because electric load profile characterizes
repeated and periodic variable
CONCLUSION
The proposed SDSA-FNN is firstly compared with
various methods applied on function approximations. The
simulations reveal that it has better performances than
other methods. Afterwards, it is applied on electric load
forecasting application and verified on electric load data
recorded on Macao power system. The simulation results
reveal that the proposed methodology not only keeps the
traditional objective function, i.e., forecasting accuracy,
but also enhance this forecaster endorsing with human
intelligence and smartness. In addition, complete retraining is not required for whole system so as to suit for
new operation environment. Instead, the gradual rerefinement and expansion of existing system can keep its
applicability in changing environment. The apparent
advantages of our approach are:
C
C
C
Fast on achieving new applicable system
Eliminate memorizing old datasets
Applicable to both past and new operation
environments
REFERENCES
Angelov, P. and R. Buswell, 2002. Identification of
evolving fuzzy rule-based models. IEEE T. Fuzzy
Syst., 10(5): 667-677.
Jang, J.S. and C.T. Sun, 1993. Functional equivalence
between radial basis function networks and fuzzy
inference systems. IEEE T. Neural Netw., 4(1): 156159.
Juang, C.F. and C.T. Lin, 1998. An on-line selfconstructing neural fuzzy inference network and its
applications. IEEE T. Fuzzy Syst., 6(1): 12-32.
Kasabov, N.K. and Q. Song, 2002. DENFIS: Dynamic
evolving neural-fuzzy inference system and its
application for time-series prediction. IEEE T. Fuzzy
Syst., 10(2): 144-154.
Koczy, L., D. Tikk and T. Gedeon, 2000. On functional
equivalence of certain fuzzy controllers and RBF
type approximation schemes. Int. J. Fuzzy Syst.,
2(3): 164-175.
Shi, Y., M. Mizumoto and P. Shi, 2002. Fuzzy if-then
rule generation based on neural network and
clustering algorithm techniques. Proc. IEEE Tencon,
02: 651-654.
Wang, L., 1992. Fuzzy systems are universal
approximators. Proceeding of 1st IEEE Conference
Fuzzy System, San Diego, CA, pp: 1163-1169.
Wang, L.X. and J.M. Mendel, 1992. Fuzzy basis
functions, universal approximation and orthogonal
least-squares learning. IEEE T. Neur. Network., 3:
807-814.
Wu, S. and M.J. Er, 2000. Dynamic fuzzy neural
networks-a novel approach to function
approximation. IEEE T. Syst. Man Cy. B, 30(2):
358-364.
4021