Confidential & No Circulation
5/29/2016
Dynamic Fuzzy Neural Networks :
Architectures, Algorithms, and Applications
Meng Joo Er
School of Electrical and Electronic Engineering
Nanyang Technological University
Email: emjer@ntu.edu.sg
1
Outline
 Introduction
 Dynamic Fuzzy Neural Networks
 Enhanced Dynamic Fuzzy Neural
Networks
 Generalized Dynamic Fuzzy
Neural Networks
 Applications
 Conclusions
2
1
Confidential & No Circulation
5/29/2016
Outline
 Introduction
 Dynamic Fuzzy Neural Networks
 Enhanced Dynamic Fuzzy Neural
Networks
 Generalized Dynamic Fuzzy
Neural Networks
 Applications
 Conclusions
3
Introduction
Artificial Intelligence
(http://www.eka-pr.com/)
Face Recognition
(https://www.theguardian.com)
IBM Watson
(http://spectrum.ieee.org)
Self-Driving Car
(http://www.hk01.com/)
AlphaGo
4
2
Confidential & No Circulation
5/29/2016
Introduction
Review of Intelligent System Theory
Fuzzy Systems
Neural Networks
Evolutionary Computation
Expert System
Data Mining
… …
Fuzzy system
Simpler, faster, and
more intelligent
+ Neural network
Fuzzy Neural Network
5
Introduction
Fuzzy System
Rule set: contains a number of fuzzy IF-THEN rules.
Label set: defines the membership functions of the
fuzzy sets used in the fuzzy rules.
Inference mechanism: performs the inference
operations on the rules.
Fuzzifier: transforms the crisp inputs into degrees of
match with linguistic values.
Defuzzifier: transforms the fuzzy results of the
inference into a crisp output.
6
3
Confidential & No Circulation
5/29/2016
Introduction
Fuzzy Set
Membership function (MF) is used to describe the
degree of belonging of an object
Triangular membership:
 | xm|
1 
 ( x)  

0
Gaussian membership:
if | x  m | 
 ( x )  exp[
otherwise
Trianglar MF
1
0.8
0.6
0.6
0.4
0.4
0.2
0
2
]
Gaussian MF
1
0.8
( x  c)2
0.2
m
0
5

10
15
20
25
0
0
c
2
4

6
8
10
7
Introduction
Fuzzy Rule
Fuzzy systems are essentially rule-based expert systems,
which consist of a set of linguistic rules in the form of
“IF-THEN”.
Fuzzy IF-THEN rules:
Ri : IF x1 is F1i and …and xr is Fri , THEN y1 is G1i and …and y s is Gsi
Takagi-Sugeno-Kang model:
Ri : IF x1 is F1i and...and xr is Fri , THEN y i   i0   1i x1  ir x r
Advantages of TSK model:
Computational efficiency
Works well with linear techniques
Works well with optimization and adaptive techniques
Guaranteed continuity of the output surface
Better suited to mathematical analysis
8
4
Confidential & No Circulation
5/29/2016
Introduction
Fuzzy Inference
Produce a crisp output by inference operations upon
fuzzy IF-THEN rules
Type I - Tsukamoto Fuzzy Models:
The overall output is the weighted average of each rule’s
crisp output induced by the rule’s firing strength wi :
u
y
y
i
 wi
i 1
u
w
i
i 1
9
Introduction
Fuzzy Inference
Type II - Mamdani Fuzzy Models:
The overall fuzzy output is derived by applying “maximum”
operation to the qualified fuzzy output
u
y

y
( wi )  wi
i 1
u

y
( wi )
i 1
10
5
Confidential & No Circulation
5/29/2016
Introduction
Fuzzy Inference
Type III - TSK Fuzzy Models:
The output of each rule is a linear combination of input
variables plus a constant term and the final crisp output is
the weighted average of each rule’s output
u
y( x ) 
y
i
 wi
i 1
where
u
w
y i   i0   1i x1  ir x r
i
i 1
y1   10   11 x1   12 x 2
y2   20   12 x1   22 x 2
y
w1 y1  w2 y2
w1  w2
11
Introduction
Neural Networks
A neural network consists of many neurons and can be
classified into feedforward (or non-recurrent) and
feedback (or recurrent) neural networks
A neuron:
Each neuron performs two functions: aggregation of its inputs from
other neurons or the external environment and generation of an output
from the aggregated inputs.
12
6
Confidential & No Circulation
5/29/2016
Introduction
Properties of Neural Networks
 An efficient method to approximate any nonlinear mapping
 Learning ability -- be trained from sample data
 Ability of generalization -- can respond correctly to any
inputs which are not learned in the sample data
 Can operate simultaneously on both quantitative and
qualitative data
 Naturally process many inputs and outputs and are readily
applicable to multivariable systems
 Highly parallel implementation
One of the most salient features of neural networks lies in
13
its learning capability from samples
Introduction
Learning Algorithms
 Supervised learning: Data and corresponding labels
are given
 Unsupervised learning: Only data are given, no labels
provided
 Semi-supervised learning: Some (if not all) labels are
provided
 Reinforcement learning: Only provides a scalar
performance measure without indicating the direction in
which the system could be improved
14
7
Confidential & No Circulation
5/29/2016
Introduction
RBF Neural Networks
Neural network formula:
u
y( X )  w0   wi Ri (|| X  Ci ||)
i 1
 Thin-plate-spline function
 Inverse multiquadric function
R( x )  x log( x )
R( x )  ( x 2   2 ) 1/ 2
2
 Gaussian function
 Multiquadric function
R( x )  ( x 2   2 )1/ 2
R( x )  exp( 
x2
2
)
15
Introduction
Comparison
Skills
Knowledge
acquisition
Fuzzy System
Neural Networks
Input
human expert
sample sets
Tools
interaction
quantitative and
qualitative
algorithm
Information
Uncertainty
Reasoning
Cognition
decision making
Mechanism
heuristic search
Speed
high
low
Fault-tolerance
Adaptation
Learning
Natural
language
Implementation
Flexibility
induction
explicit
high
quantitative
perception
parallel
computation
low
very high
adjusting
weights
implicit
low
16
8
Confidential & No Circulation
5/29/2016
Introduction
Fuzzy Neural Networks
Fuzzy System
Neural Networks
 Dependent on the specification
of good rules by human experts
 No easy way to predict the
output
 No formal framework for the
choice of various parameters
 Connecting weights invariably
shed no light on what the
network will do with the data
Fuzzy Neural Networks
Capture capabilities of both types of systems and overcome drawbacks
of each system
FNNs were used to:
 Tune fuzzy systems
 Extract fuzzy rules from given numerical examples
 Develop hybrid systems combining neural networks and fuzzy systems
17
Contents
 Introduction
 Dynamic Fuzzy Neural Networks
 Enhanced Dynamic Fuzzy Neural
Networks
 Generalized Dynamic Fuzzy
Neural Networks
 Applications
 Conclusions
18
9
Confidential & No Circulation
5/29/2016
Dynamic Fuzzy Neural Network
Motivation
 Lack of systematic design for membership functions
 Lack of adaptability for reasoning environment changes
 Structure identification is time-consuming
Dynamic Fuzzy
Neural Network
(D-FNN)
Extended
RBF NN
Pruning
technology
Hierarchical
learning
Adaptive structure
identification
Hierarchical on-line
self-organizing learning
Fast learning speed
19
Dynamic Fuzzy Neural Network
Architecture of D-FNN
…
…
MFru
……
…
……
xr
Layer 4
……
…
x1
Layer 3
……
Layer 2
……
Layer 1
Ru
Nu
Layer 5
w1
w
j
Y
wu
20
10
Confidential & No Circulation
5/29/2016
Dynamic Fuzzy Neural Network
Architecture of D-FNN
Layer 1
Layer 2
Layer 3
……
…
……
…
……
…
……
xr
……
…
x1
Layer 4
Ru
Nu
MFru
Layer 5
w1
wj
Y
wu
Layer 1: Each node in layer 1 represents an input
linguistic variable.
21
Dynamic Fuzzy Neural Network
Architecture of D-FNN
Layer 1
Layer 2
Layer 3
……
…
……
…
……
…
……
xr
……
…
x1
Layer 4
Ru
Nu
MFru
Layer 5
w1
wj
Y
wu
Layer 2: Each node in layer 2 represents a membership
function (MF) which is a Gaussian function given by:
 ij ( x i )  exp[
( x i  cij ) 2
 2j
]
i  1, 2r, j  1, 2u
22
11
Confidential & No Circulation
5/29/2016
Dynamic Fuzzy Neural Network
Architecture of D-FNN
Layer 1
Layer 2
……
…
……
…
MFru
……
…
……
xr
Layer 4
……
…
x1
Layer 3
Ru
Nu
Layer 5
w1
wj
Y
wu
Layer 3: Each node in layer 3 represents a possible IF-part of
fuzzy rules.
 X C 2 
j
j  1,2, , u

Output of jth RBF unit:  j  exp 
2


j


23
Dynamic Fuzzy Neural Network
Architecture of D-FNN
Layer 1
Layer 2
……
…
……
…
MFru
……
…
……
xr
Layer 4
……
…
x1
Layer 3
Ru
Nu
Layer 5
w1
wj
Y
wu
Layer 4: We refer to these nodes as N (normalized) nodes
j 
j
j  1,2,, u
u

k
24
k 1
12
Confidential & No Circulation
5/29/2016
Dynamic Fuzzy Neural Network
Architecture of D-FNN
Layer 1
Layer 2
……
…
……
…
MFru
……
…
……
xr
Layer 4
……
…
x1
Layer 3
Ru
Nu
Layer 5
w1
wj
Y
wu
Layer 5: Each node in this layer represents an output variable
which is the summation of incoming signals
u
y( X )   w k   k TSK model: wk   k 0   k1 x1     kr xr
k 1
25
Dynamic Fuzzy Neural Network
Learning Algorithm of the D-FNN
Rule
Generation
Allocation of RBF
Unit Parameters
Weight
Adjustment
Rule
Pruning
26
13
Confidential & No Circulation
5/29/2016
Dynamic Fuzzy Neural Network
Learning Algorithm of the D-FNN
Rule
Generation
Allocation of RBF
Unit Parameters
Weight
Adjustment
Error criterion: For observation
ei  ti  yi
 X i , ti 
 ke ?
Rule
Pruning
, define
An RBF unit should be
considered
Accommodation criterion:
d i  j   X i  Ci ,
j  1,2,  , u
If d min  arg min d i  j   k d , an RBF unit should be considered.
Hierarchical learning: Accommodation boundary is adjusted
dynamically
i
ke  max[emax   , emin ]
kd  max[dmax   i , dmin ]
Coarse learning to
fine learning
27
Dynamic Fuzzy Neural Network
Learning Algorithm of the D-FNN
Rule
Generation
Allocation of RBF
Unit Parameters
Weight
Adjustment
Rule
Pruning
Allocation of premise parameter :
Ci  X i
 i  k  dmin
 kd
Rule generation
Case 1:
|| ei ||  ke , dmin
Case 2:
|| ei ||  ke , dmin  kd
Do nothing or only update
consequent parameters
Case 3:
|| ei ||  ke , dmin  kd
Only adjust consequent
parameters
Case 4:
|| ei ||  ke , dmin  kd
For the kth nearest RBF unit:
Update the width of the nearest
RBF node and all the weights
 ik  kw   ik1
28
14
Confidential & No Circulation
5/29/2016
Dynamic Fuzzy Neural Network
Learning Algorithm of the D-FNN
Rule
Generation
Allocation of RBF
Unit Parameters
Weight
Adjustment
Rule
Pruning
Assume nth training pattern enters the D-FNN and all these n data
are memorized by the D-FNN in the input matrix P and the output
matrix T, respectively, upon which the weights are determined.
Give the output in the compact form as follows:
~
Y W
E || T  Y ||
where W is the weight matrix,  is the output matrix of N nodes.
~ ~
The objective is to find an optimal W * such that E T E is minimized.
This problem can be solved by the well-known Linear Least
Square (LLS) method by approximating:
W *  T (  T  ) 1  T
29
Dynamic Fuzzy Neural Network
Learning Algorithm of the D-FNN
Rule
Generation
Allocation of RBF
Unit Parameters
Weight
Adjustment
Rule
Pruning
Error Reduction Ratio (ERR) Method:
D  H  E
where D  T , H   T , and   W T .
T
QR decomposition:
H  QA
nv
where Q  (q1 , q2 , qv ) 
ERR:
( q iT D) 2
erri  T
qi qi D T D
The ERR offers a simple
and effective means of
seeking a subset of
significant regressors
iv
30
15
Confidential & No Circulation
5/29/2016
Dynamic Fuzzy Neural Network
Learning Algorithm of the D-FNN
Rule
Generation
Allocation of RBF
Unit Parameters
Weight
Adjustment
Rule
Pruning
Determination of RBF Units:
Define the ERR matrix as
  ( 1 ,  2 , u )  ( r 1)u
Significance of the ith rule is defined as
i 
If
 iT  i
r 1
Prespecified
threshold
 i  k err
31
then the ith rule is deleted.
Dynamic Fuzzy Neural Network
Flowchart of D-FNN Algorithm
Initialization
Generate a new rule
N
Generate first rule
?
Compute ERR
Compute distance and
find
Y
N
Compute actual output
error
?
Delete ith rule
N
?
Y
Y
N
Y
N
?
Adjust widths
Adjust consequents
parameters
Observations
completed?
Y
End
32
16
Confidential & No Circulation
5/29/2016
Outline
 Introduction
 Dynamic Fuzzy Neural Networks
 Enhanced Dynamic Fuzzy Neural
Networks
 Generalized Dynamic Fuzzy
Neural Networks
 Applications
 Conclusions
33
Enhanced Dynamic Fuzzy Neural Network
Motivation
Adaptive noise cancellation (ANC) using an adaptive filter:

Not measurable
nk 
Measurable
y k   xk   d k 

xk 
f 
Adaptive
filter
Measurable
d k 
Not measurable

d̂ k 

xˆ k   xk   d k   dˆ k 
D-FNN cannot be directly applied to ANC, because the
neuron generation depends on the systems error, which
cannot be defined for ANC.
34
17
Confidential & No Circulation
5/29/2016
Enhanced Dynamic Fuzzy Neural Network
Motivation
 The method
based on accommodation boundary and
system error could hardly be evaluated online
 Linear Least Square (LLS) method is not very effective in
processing signals online in a very noisy environment
Recursive Linear
Least Square (RLLS)
Estimator or Kalman
Filter method
Online self-organizing
mapping (SOM)
Enhanced
Dynamic
Fuzzy
Neural Network
35
Enhanced Dynamic Fuzzy Neural Network
ED-FNN Learning Algorithms
The ultimate objective is to implement the D-FNN learning
algorithm in real time
Input Space Partitioning: Online SOM is performed for every
training pattern (X(k), y(k))
Best Matching Neuron Center:
Update rule:
|| X (k )  C (k ) || min{|| X (k )  Ci (k ) ||}
b
j
C j ( k  1)  C j ( k )   ( k ) h ( k )[ X ( k )  C j ( k )]
bi
where k is the kth input pattern, a(k) is the adaptation coefficient, and hbi(k),
the neighborhood kernel centered on the winner unit, is given by
h ( k )  exp( 
bi
|| C  C j || 2
b
2 2 ( k )
)
36
18
Confidential & No Circulation
5/29/2016
Enhanced Dynamic Fuzzy Neural Network
ED-FNN Learning Algorithms
Adaptation of the Output Linear Weights: Recursive Linear
Least Square (RLLS) estimator is used to adjust the weights
RLLS:
T 1
S 0  (  i 1 i 1 )
Si  Si 1 
T
Si 1 i  i Si 1
1  Si 1
T
Wi  Wi 1  Si 1 i (Ti   i Wi 1 )
i  1,2,  , n
where Si is the error covariance matrix for the ith observation.
37
Enhanced Dynamic Fuzzy Neural Network
Flowchart of ED-FNN Algorithm
Determine parameters
of new rule
Start
Initialization
LLS to adjust weights
Generate first rule
N
SOM to update center
clustering
Compute distance and
find
?
Y
N
LLS to adjust
weights
First observation
completed?
Y
RLLS to adjust weights
Observations
completed?
Y
End
N
38
19
Confidential & No Circulation
5/29/2016
Outline
 Introduction
 Dynamic Fuzzy Neural Networks
 Enhanced Dynamic Fuzzy Neural
Networks
 Generalized Dynamic Fuzzy
Neural Networks
 Applications
 Conclusions
39
Generalized Dynamic Fuzzy Neural Networks
Motivation
Widths of Gaussian
MFs are the same
Do not coincide with the reality
The number of MFs is
irrespective of MF
distribution
Result in significant overlapping
and be opaque for users to
understand
The width of a new
Gaussian MF is only
determined by d minand k
If k and d min is too large, the
widths will be very large.
Several prespecified
parameters are selected
randomly
Not easy for users to implement
40
20
Confidential & No Circulation
5/29/2016
Generalized Dynamic Fuzzy Neural Networks
GD-FNN
Ellipsoidal Basis Function
(EBF)
On-line parameter allocation
Generalized Dynamic
Neural Network
Sensitivities analysis
Structure and parameters identification are performed
automatically and simultaneously without partitioning
input space and selecting initial parameters a priori
Fuzzy rules can be recruited or deleted dynamically
Fuzzy rules can be generated quickly without resorting to
the BP iteration learning
41
Generalized Dynamic Fuzzy Neural Networks
Architecture of GD-FNN
Layer 1
Layer 2
…
…
……
……
…
xr
……
…
x1
Layer 3
MFru
Layer 4
w1
wj
Y
wu
Ru
42
21
Confidential & No Circulation
5/29/2016
Generalized Dynamic Fuzzy Neural Networks
Equivalence of GD-FNN
Corollary 1: The proposed GD-FNN is functionally equivalent to
the TSK fuzzy system if the following conditions are true:
 The number of receptive field units of the GD-FNN is equal to the
number of fuzzy IF-THEN rules
 The output of each fuzzy IF-THEN rule is a linear combination of input
variables plus a constant term
 The membership functions within each rule are chosen as Gaussian
functions
 The T-norm operator used to compute each rule’s firing strength is
multiplication
 Both the GD-FNN and the TSK fuzzy inference system under
consideration use the same method, i.e. either weighted average or
weighted sum to derive their overall outputs
43
Generalized Dynamic Fuzzy Neural Networks
Learning Algorithm of the GD-FNN
Rule Generation
Propose criteria of
rule generation
Premise Parameter
Estimation
Allocate parameters
of new rules
Sensitivity Analysis
Analyze sensitivity of
input variables and fuzzy
rules and prune rules
Width Modification
Adjust width to obtain a
better local approximation
Consequent Parameter Determination
Determine the optimal
parameters ∗
44
22
Confidential & No Circulation
5/29/2016
Generalized Dynamic Fuzzy Neural Networks
Rule Generation
 System Errors
|| ek |||| tk  yk ||
If || ek ||  ke a new rule should be considered, where
1 k  n/3
 emax

ke   max[emax   k , emin ]
e
 min
n / 3  k  2n / 3
2n / 3  k  n
where e min is the desired accuracy of the output of the GDFNN, emax is the maximum error chosen, k is the learning
epoch, and   (0, 1) , called convergence constant is given by
 (
emin 3/ n
)
emax
45
Generalized Dynamic Fuzzy Neural Networks
Rule Generation
 1

 0 
0
 2
 1j



1
0
0 
 0
Definition (-Completeness of Fuzzy Rules)*:
For
2 any
1
2j
 

input in the operating range, there existsj atleast one
 0 
0
0
 strength) is

fuzzy rule so that the match degree (or firing

1 

0
 0
no less than .
 rj 2 

 -Completeness of Fuzzy Rules
Calculate Mahalanobis distance (M-distance)
md k  j  
X  C 
T
j
 1j X  C j 
Find
J  arg min (md k ( j ))
If
md k ,min  md k ( J )  k d
1 j u
d max  ln(1/  min )

kd   max[d max   k , d min ]

d min  ln(1/  max )
1 k  n/3
n / 3  k  2n / 3
2n / 3  k  n
which implies that the existing system is not satisfied with
-completeness and a new rule should be considered
*C. C. Lee, IEEE Trans. Syst, Man and Cybern, Vol. SMC 20, pp. 404-436, 1990
46
23
Confidential & No Circulation
5/29/2016
Generalized Dynamic Fuzzy Neural Networks
Premise Parameter Estimation
,
Definition (Semi-Closed Fuzzy Sets): Let
discourse of input . If each fuzzy set
1,2, ⋯ ,
be the universe of
,
∈ exp
is represented as the Gaussian function
for all
∈ , and satisfies the following boundary conditions
simultaneously:
exp
exp
exp
if |
if |
|
if |
|
|
and |
|
where is a tolerable small value, and the widths of Gaussian functions
are selected as
,|
|
/
,
1,2, ⋯ ,
where
and
are the two centers of neighboring MFs of th
membership function. Then, we call semi-closed fuzzy sets.
47
Generalized Dynamic Fuzzy Neural Networks
Premise Parameter Estimation
Theorem: The semi-closed fuzzy set
satisfy -completeness of fuzzy
rules, i.e., for all ∈ , there exists ∈ 1,2, ⋯ , , such that
.
Calculate E-distance:
Φ
where Φ ∈
,
,
,⋯,
,
set
,
,
1,2, ⋯ ,
2
and find
arg
If
min
, ,⋯,
can be completely represented by the existing fuzzy
without generating a new MF. Otherwise, a new
Gaussian MF is allocated with its width and center respectively being
determined by
max
,|
ln 1/
|
,
48
24
Confidential & No Circulation
5/29/2016
Generalized Dynamic Fuzzy Neural Networks
Sensitivity Analysis
 Error Reduction Ratio
Linear regression model:
D  H  E
where D  T , H   T , and   W T .
T
QR decomposition:
H  QA
nv
where Q  (q1 , q2 , qv ) 
ERR:
erri 
( q iT D) 2
q iT q i D T D
iv
49
Generalized Dynamic Fuzzy Neural Networks
Sensitivity Analysis
 Sensitivity of Fuzzy Rules
Define the ERR matrix
  ( 1 ,  2 , u )  ( r 1)u
Significance of the ith rule is defined as
i 
If
 iT  i
r 1
 i  k err
i  1,2,  , u
then the ith rule is deleted.
50
25
Confidential & No Circulation
5/29/2016
Generalized Dynamic Fuzzy Neural Networks
Sensitivity Analysis
 Sensitivity of Input Variables
Define
r 1
B j    j (k )
k 2
Bij 
j  1,2,  , u
errij
Bj
i  1,2,  , r
-- total ERR related to input variables in jth rule
-- ERR corresponding to ith input variable in jth rule
-- The significance of ith input variable in jth rule
If
is small, ith input variable is not sensitive to the system
output, and consequently, width of the hyperellipsoidal region in
ith input variable could be reduced without significant effect of
51
system performance.
Generalized Dynamic Fuzzy Neural Networks
Width Modification
In the case
and
, which implies the
,
significance of the rule is not good to accommodate all the
patterns, and the ellipsoidal region should be reduced.
Width modification:
 ijnew     ijold
where
1

1  k ( B  1/ r ) 2
 
w
ij

1

Bij  1/ r
Bij  1/ r
52
26
Confidential & No Circulation
5/29/2016
Generalized Dynamic Fuzzy Neural Networks
Consequent Parameter Determination
Give the output in the compact form as follows:
W  Y
Assume that the desired output is :
T  (t1 , t 2 t n )  n
Minimizing
||W   T ||2
∗
The optimal
can be determined by
W *  T ( T  ) 1 T
53
Generalized Dynamic Fuzzy Neural Networks
Flowchart of GD-FNN Algorithm
Initialization
Generate first rule
Determine parameters
of new rule
Compute M-distances
and find
,
Compute ERR
N
Compute actual output
error
N
?
Y
Delete ith rule
N
?
Y
Calculate
Adjust
widths
?
Y
N
?
Y
N
Generate a new rule
Observations
completed?
Y
End
Adjust
consequents
parameters
Do nothing or
adjust
consequent
parameter
Adjust
consequents
parameters
54
27
Confidential & No Circulation
5/29/2016
Outline
 Introduction
 Dynamic Fuzzy Neural Networks
 Enhanced Dynamic Fuzzy Neural
Networks
 Generalized Dynamic Fuzzy
Neural Networks
 Applications
 Conclusions
55
Applications
Application 1:
Mackey—Glass Time-Series Prediction
Time-series prediction is widely applied in economic and
business planning, inventory and production control,
weather forecasting, signal processing, control etc.
System model:
x (t  1)  (1  a) x (t ) 
where
0.1,
0.2, τ
17.
bx (t   )
1  x 10 (t   )
Prediction model:
x(t  p)  f [ x(t ), x(t  t ), x(t  2 t ), x(t  3t )]
6000 exemplar samples were generated between
0 and
0 for
0 and 0
6000 with initial conditions
1.2.
56
28
Confidential & No Circulation
5/29/2016
Applications
Application 1:
Mackey—Glass Time-Series Prediction
Mackey—Glass time-series prediction (from t = 118 to 617 and six-stepahead prediction)
Actual output error e(i)
30
Fuzzy rule generation
10
9
8
7
6
5
4
3
2
10
25
20
15
10
5
100
200
300
400
sample patterns
500
0 0
100
200
300
sample patterns
400
500
57
Applications
Application 1:
Mackey—Glass Time-Series Prediction
Generalization test of the D-FNN for Mackey—Glass time-series
prediction (from t = 642 to 1141 and six-step ahead prediction)
desired &
predicted output
1.4
Comparison between desired and
predicted outputs
1.2
0.02
1
0.01
0.8
0
0.6
-0.01
0.4
-0.02
0.2
500
600
700
800
Time t
900
prediction error
0.03
1000
-0.03
500
600
700
800
Time t
900
1000
58
29
Confidential & No Circulation
5/29/2016
Applications
Application 1:
Mackey—Glass Time-Series Prediction
Comparisons of structure and performance of different algorithms
(Training case: n = 500, p = 6)
Number of
Number of
RMSE of
RMSE of
rules
parameters
training
testing
D-FNN
10
100
0.0082
0.0083
ANFIS
16
104
0.0016
0.0015
OLS
35
211
0.0087
0.0089
RBF-AFS
21
210
0.0107
0.0128
Technique
59
Applications
Application 1:
Mackey—Glass Time-Series Prediction
Generalization comparison between the D-FNN and the ANFIS for p=85
Technique
Pattern
Number of
Number of
NDEI
length
rules
parameters
D-FNN
500
14
140
0.3375
ANFIS
500
16
104
0.036
Remark: It is shown that the performance of the D-FNN is
not so good as that of the ANFIS, even the D-FNN has more
adjustable parameters. The reason is that the ANFIS is
trained by iterative learning so that an overall optimal
solution can be achieved, whereas the D-FNN only acquires
a sub-optimal result.
60
30
Confidential & No Circulation
5/29/2016
Applications
Application 1:
Mackey—Glass Time-Series Prediction
Generalization comparison between D-FNN and RAN, RANEKF and MRAN for p=50
Technique
Pattern
Number of
Number of
NDEI
length
rules
parameters
D-FNN
2000
25
250
0.0544
RAN
5000
50
301
0.071
RANEKF
5000
56
337
0.056
M-RAN
5000
28
169
*
*The generalization error is measured by weighted prediction
error (WPE)
61
Applications
Application 2:
Prediction of Machine Health Condition
An online dynamic fuzzy neural network (OD-FNN) is
developed for prediction of machine health condition
prediction
Experimental platform PRONOSTIA
62
31
Confidential & No Circulation
5/29/2016
Applications
Application 2:
Prediction of Machine Health Condition
Prediction model:
y s k  r   f  ys k , ys k  r , ys k  2r , , ys k  nr ,
xs k , xs k  r , xs k  2r ,  , xs k  nr 
where and are the input and target signals respectively,
is the prediction step, and
1 is maximum lag.
500 data are applied to initialize the OD-FNN and 500 data
pairs are applied to test the OD-FNN performance for online
prediction.
63
Applications
Application 2:
Prediction of Machine Health Condition
Linear model online prediction results: (a) Prediction output with
error curves. (b) Accuracies of online training and prediction.
5
White noise
signal
64
32
Confidential & No Circulation
5/29/2016
Applications
Application 2:
Prediction of Machine Health Condition
Bearing MHC online prediction results: (a) Prediction output with
error curves. (b) Accuracies of online training and prediction.
65
Applications
Application 2:
Prediction of Machine Health Condition
Performance comparisons of all prediction methods
good tradeCaseAchieve aNN
types
off between prediction
and
LinearaccuracyOSBP-NN
cost
modelcomputational
OSELM-FNN
8.2471
1.3841
5.2681
1.6833
4.0210
0.6765
2.5951
1.6833
99.52
99.31
99.23
98.28
RRLS-NN
99.43
99.34
OD-FNN
99.31
99.18
Bearing
OSBP-NN
98.38
96.01
Best online prediction
OSELM-FNN
97.31
95.35
accuracy
RRLS-NN
97.34
95.89
OD-FNN
97.94
97.54
--online training accuracy
-- total computational time
66
--online prediction accuracy
33
Confidential & No Circulation
5/29/2016
Applications
Application 3:
Online Manipulator Control
Articulated two-link robot manipulator
67
Applications
Application 3:
Online Manipulator Control
Manipulator dynamics:


M ( q ) q  Q ( q, q )   d     1  2 
T
where

 1  [( I 1  m1l c1 2  I e  me l ce 2  me l12 )  2(me l1l ce cos  e ) cos q 2  2(me l1l ce sin  e ) sin q 2 ] q 1

 [( I e  me l ce )  (me l1l ce cos  e ) cos q 2  (me l1l ce sin  e ) sin q 2 ] q 2
2


 [(me l1l ce cos  e ) sin q 2  ( me l1l ce sin  e ) cos q 2 ] q1 q 2



 [(me l1l ce cos  e ) sin q 2  ( me l1l ce sin  e ) cos q 2 ](q1  q 2 ) q 2
  d1


 2  [( I e  me l ce 2 )  (me l1l ce cos  e ) cos q 2  (me l1l ce sin  e ) sin q 2 ] q 1  ( I e  me l ce 2 ) q 2

 [(me l1l ce cos  e ) sin q 2  ( me l1l ce sin  e ) cos q 2 ] q 1
 d2
2
68
34
Confidential & No Circulation
5/29/2016
Applications
Application 3:
Online Manipulator Control
Online adaptive fuzzy neural control system:
69
Applications
Application 3:
Online Manipulator Control
Performance comparison of GD-FNN with PD and (adaptive
fuzzy controller ) AFC controllers:
Steady State
Overshoot
Rise Time
Error (rad)
(%)
(sec)
Maximum
Control
Torque (Nm)
PD
q1
0.050
9.3
0.287
312.7
Kp =6000; Kv =100
q2
0.018
7.1
0.254
109.9
AFC
q1
0.022
0
0.237
425.9
Kp =25; Kv =35
q2
0.025
0
0.218
255.6
GD-FNN
q1
0.001
0
0.01
280.3
Kp =25; Kv =7
q2
0.001
0
0.01
110.0
70
35
Confidential & No Circulation
5/29/2016
Applications
Application 3:
Online Manipulator Control
Trajectory tracking results of adaptive online control
71
Applications
Application 3:
Online Manipulator Control
Convergence of tracking error
72
36
Confidential & No Circulation
5/29/2016
Outline
 Introduction
 Dynamic Fuzzy Neural Networks
 Enhanced Dynamic Fuzzy Neural
Networks
 Generalized Dynamic Fuzzy
Neural Networks
 Applications
 Conclusions
73
Conclusions and Recommendations
Conclusions
 A D-FNN based on extended Radial Basis Function
(RBF) neural network is introduced, which has the
following properties: online learning, hierarchical learning,
dynamic self-organizing structure, fast speed in learning,
and generalization in learning.
 An Enhanced Dynamic Fuzzy Neural Network (ED-FNN)
learning algorithm based on SOM and RLLS Estimator is
developed, which is suitable for real-time applications
and it outperforms other approaches
74
37
Confidential & No Circulation
5/29/2016
Conclusions and Recommendations
Conclusions
 A GD-FNN based on Ellipsoidal Basis Function (EBF) is
introduced, which can provide a simple and fast
approach to configure a fuzzy system so that
 some meaningful fuzzy rules could be acquired for
knowledge engineering, and
 the system could be used as a system modeling tool
for control engineering, pattern recognition, etc..
75
76
38