Paper study-
Saichon Jaiyen, Chidchanok Lursinsap, Suphakant Phimoltares
IEEE TRANSACTIONS ON NEURAL NETWORKS, VOL. 21, NO. 3, MARCH 2010
1
OUTLINE
 Introduction
 VEBF Neural Network
 Example for Training
 Experimental Results
2
OUTLINE
 Introduction
 VEBF Neural Network
 Example for Training
 Experimental Results
3
Introduction
 Most current training algorithms require both new incoming
data and those previously trained data together in order to
correctly learn the whole data set.
 This paper propose the very fast training algorithm to learn a
data set in only one pass.
 The structure of proposed neural network consists of three
layers but the structure is flexible and can be adjusted during
the training process.
4
OUTLINE
 Introduction
 VEBF Neural Network
 Example for Training
 Experimental Results
5
VEBF Neural Network
6
VEBF Neural Network
 VEBF : versatile elliptic basis function
 Outline of learning algorithm
 1. add a training data to the VEBF neural network
 2. create a new neuron or not


Create: set the parameters
It can join into a current node: update
 3.detect merge condition
7
VEBF Neural Network
 for each vector x = [x1,x2,…,xn]T in Rn and orthonormal basis
{u1,u2,…,un} for Rn
 xi = xTui
 the hyperellipsoidal equation unrotated and centered
at the origin is defined as
 Where ai is the width of the ith axis in hyperellipsoid.
 The simplification can be written as
 Define a new basis function as
8
VEBF Neural Network
 If the original axes of the hyperellipsoidal equation are
translated from the origin to the coordinates of c =
[c1,c2,…,cn]T . Consequently, the new coordinates of
vector x , denoted by x’ = [x’1,x’2,…,x’n]T , with respect
to the new axes can be written as
 The simplification can be written as
9
VEBF Neural Network
 The VEBF as
 Where {u1,u2,…,un} is the orthonormal basis, the constant
ai,i = 1,…,n, is the width of the ith axis in the
hyperellipsoid, and the center vector c = [c1,c2,…,cn]T
refers to the mean vector.
10
VEBF Neural Network
 Let X = {(xj,tj)| 1<= j <= N} be a finite set of N training
data, where xj ∈ Rn is a feature vector referred to as a
data vector and tj is the class label of the vector xj.
 We denote Ωk as a 5-tuple, (C(k),S(k),Nk,Ak,dk),
 C(k) = [c1,c2,…,cn]T is the center of the kth neuron
 S(k) is the covariance matrix of the kth neuron
 Nk is the total number of data related to the kth neuron
 Ak is the width vector of the kth neuron
 dk is the class label of the kth neuron
11
VEBF Neural Network
 Let cs be the index of this closest hidden neuron.

 If
> 0 , a new hidden neuron is allocated and
added into the network.
 If
< 0 , joint to the closest hidden neuron.
VEBF Neural Network
 Mean computation

 The recursive relation can be written as follows

 where
is the new mean vector,
, and
13
VEBF Neural Network
 Covariance matrix computation

 The recursive relation can be written as follows

 where
is the new covariance matrix,
, and


 The orthonormal axes vectors are computed by the
eigenvectors of the sorted eigenvalues of the covariance
matrix.
14
VEBF Neural Network - merge
 Let Ωx = (C(x),S(x),Nx,Ax,dx) and Ωy = (C(y),S(y),Ny,Ay,dy) be
any two hidden neurons x and y in a VEBF neural
network.
 merging criterion :
≤ 𝜃 , then these two
hidden neurons are merged into one new hidden neuron
Ωnew = (C(new),S(new),N new,A new,d new) .
 If merging criterion
 𝜃 is the threshold
15
VEBF Neural Network - merge
 The new parameters of this new hidden neuron can be
computed as follows:
 Where 𝜆𝑖 is the ith eigenvalue of the new covariance
matrix.
16
OUTLINE
 Introduction
 VEBF Neural Network
 Example for Training
 Experimental Results
17
Example for Training
 Suppose that X = {(5,16)T,0), (15,6)T,1) ,(10,18)T,0),
(5,6)T,1), (11,16)T,0)} is a set of training data in R2.
 Suppose that the training data in class 0 are illustrated by
“ + ” while the training data of class 1 is illustrated by
“ *.”
18
Example for Training
 1. The training data (5,16)T,0) is presented to the VEBF
neural network.
 class 0
 create a new neuron
19
Example for Training
 2. The training data (15,6)T,1) is fed to the VEBF neural
network.
 class 1
 create a new neuron
20
Example for Training
 3. The training data (10,18)T,0) is fed to the VEBF neural
network.
 class 0
 find the closest neuron
 detect the distance
 update the parameters
21
Example for Training
 3. The training data (10,18)T,0) is fed to the VEBF neural
network.
 class 0
 find the closest neuron
 detect the distance
 update the parameters
22
Example for Training
 4. The training data (5,6)T,1) is fed to the VEBF neural
network.
 class 1
 find the closest neuron
 detect the distance
 create a new neuron
23
Example for Training
 4. The training data (5,6)T,1) is fed to the VEBF neural
network.
 class 1
 find the closest neuron
 detect the distance
 create a new neuron
24
Example for Training
 5. The training data (11,16)T,0) is fed to the VEBF neural
network.
 class 0
 find the closest neuron
 detect the distance
 update the parameters
25
Example for Training
 5. The training data (11,16)T,0) is fed to the VEBF neural
network.
 class 0
 find the closest neuron
 detect the distance
 update the parameters
26
OUTLINE
 Introduction
 VEBF Neural Network
 Example for Training
 Experimental Results
27
Experimental Results
 In multiclass classification problem
 the results are compared with the conventional RBF neural
network with Gaussian RBF, multilayer perceptron (MLP).
 In two-class classification problem
 the results are also compared with the support vector
machine (SVM)
Experimental Results
 The data sets used to train and test are collected from
the University of California at Irvine (UCI) Repository of
the machine learning database.
29
Experimental Results
 Multiclass classification
Data set
# of attributes
# of classes
# of instances
Iris
4
3
150
30
Experimental Results
 Multiclass classification
Data set
# of attributes
# of classes
# of instances
E.coli
8
8
336
31
Experimental Results
 Multiclass classification
Data set
# of attributes
# of classes
# of instances
Yeast
8
10
1484
32
Experimental Results
 Multiclass classification
Data set
# of attributes
# of classes
# of instances
Image Segmentation
19
7
2310
33
Experimental Results
 Multiclass classification
Data set
# of attributes
# of classes
# of instances
Waveform
21
3
5000
34
Experimental Results
 Two-class classification
Data set
# of attributes
# of classes
# of instances
Heart
13
2
270
35
Experimental Results
 Two-class classification
Data set
# of attributes
# of classes
# of instances
Heart
13
2
270
36
Experimental Results
 Two-class classification
Data set
# of attributes
# of classes
# of instances
Spambase
57
2
4601
37
Experimental Results
 Two-class classification
Data set
# of attributes
# of classes
# of instances
Spambase
57
2
4601
38
Experimental Results
 Two-class classification
Data set
# of attributes
# of classes
# of instances
Sonar
60
2
208
39
Experimental Results
 Two-class classification
Data set
# of attributes
# of classes
# of instances
Sonar
60
2
208
40
Experimental Results
 Two-class classification
Data set
# of attributes
# of classes
# of instances
Liver
7
2
345
41
Experimental Results
 Two-class classification
Data set
# of attributes
# of classes
# of instances
Liver
7
2
345
42