International Journal of Application or Innovation in Engineering & Management (IJAIEM)
Web Site: www.ijaiem.org Email: editor@ijaiem.org, editorijaiem@gmail.com
Volume 2, Issue 12, December 2013
ISSN 2319 - 4847
Learning of alphabets using Kohonen's Self
organized featured map
Amit Goyal1 , Ankita Lakhanpal2 and Shaveta Goyal3
1,2
Department of Electronics and Communication Engineering, Maharishi Markandeshwar University,
Sadopur, Ambala, Haryana.
3
Department of Electronics and Communication Engineering, RIMT-IET, Mandi Gobindgarh, Punjab.
Abstract
Artificial neural networks are one of the widely used automated techniques and they yield high accuracy. Kohonen maps and
Counter Propagation Neural Networks are the most popular learning strategies based on Artificial Neural Networks. The Self
Organizing map has attracted an ever increasing amount of interest among researchers and practitioners in wide variety of fields.
The SOM has been analyzed extensively, a number of variants have been developed and, perhaps most notably, it has been applied
extensively within fields ranging from engineering sciences to medicine, biology, and economics. Kohonen Maps (or Self
Organizing Maps) are basically self-organizing systems which are capable to solve the unsupervised rather than the supervised
problems, while Counter-propagation Artificial Neural Networks are very similar to Kohonen maps, but an output layer is added
to the Kohonen layer in order to handle supervised modeling. In this paper, learning of alphabets with the help of Kohonen Self
Organizing Map is described. We use a self-organizing map for feature extraction of alphabets, thus the relative feature of them
are automatically defined based on arrangement of the competing units. Our proposed data structure and algorithm reduced the
SOM computational complexity by several order of magnitude.
Keywords: Artificial Neural Network, Self Organizing Maps, Unsupervised Learning, MATLAB
1. INTRODUCTION
Kohonen maps (or Self Organizing Maps, SOMs) are one of the most popular learning strategies among the several
Artificial Neural Networks algorithms proposed in literature [1]. They were developed in 1982 by Tuevo Kohonen, a
professor emeritus of the Academy of Finland. The earliest applications were mainly in the area of engineering. Later the
algorithm has become progressively more accepted as a standard data analysis method in a wide variety of fields that can
utilize unsupervised learning: clustering, visualization, data organization, characterization, and exploration. The
robustness of the SOM algorithm and its appealing visualization effects has also made it a prime candidate in several
large-scale information categorization and visualization projects. Self- Organizing Maps are aptly named, “SelfOrganizing” is because no supervision is required. Their uses are increasing related to several different tasks and
nowadays they can be considered as an important tool in multivariate statistics [2]. SOMs learn on their own through
unsupervised competitive learning. “Maps” is because they attempt to map their weights to conform to the given input
data. It consists of local data models located at the nodes of the low-dimensional map grid. The models are adapted via
competitive learning process. The nodes in a SOM network attempt to become like the inputs presented to them. In this
sense, this is how they learn. They can also be called “Feature Maps”, as in Self-Organizing Feature Maps. Retaining
principle 'features' of the input data is a fundamental principle of SOMs, and one of the things that makes them so
valuable. Specifically, the topological relationships between input data are preserved when mapped to a SOM network. .
The SOM is an unsupervised method for forming a representation of data [3, 4]. It consists of local data models located at
the nodes of the low-dimensional map grid. The models are adapted via competitive learning process.
There are two approaches for constructing the SOM for a new data set. One is to modify the SOM algorithm itself.
Another possibility is to convert the data into feature vectors so that the comparison and averaging can be done using
familiar Euclidean distance and arithmetic averaging. This is the approach taken in the current study, where the SOM is
constructed for alphabet recognition. In the current work, the SOM algorithm is very close to the alphabet recognization.
The only difference is that instead of single fixed dimensional feature vectors as input and models, the models associated
with the SOM nodes are vector sequences with variable lengths [5].
The algorithm for the Self Organizing Map neural network, which is used for alphabet learning, is elaborated in the
MATLAB environment.
2. METHODOLOGY
In this paper, two stages of development of the proposed alphabet learning system which include; data acquisition, feature
extraction, using Self Organizing neural network are presented. Experiments were performed with 18 alphabet samples
(English uppercase) which are created in the MATLAB with the desired dimensions. The results of the proposed system
showed better learning performance compared with similar work in literature.
Volume 2, Issue 12, December 2013
Page 283
International Journal of Application or Innovation in Engineering & Management (IJAIEM)
Web Site: www.ijaiem.org Email: editor@ijaiem.org, editorijaiem@gmail.com
Volume 2, Issue 12, December 2013
ISSN 2319 - 4847
2.1 Character Acquisition
The alphabets in this work are first created in the MATLAB.The dimensions of the characters are adjusted according to
requirement. The Self Organizing Map only learns the alphabets of specified dimensions. Alphabets considered were 18
upper case (A-R).
2.2 Feature Extraction
For extracting the feature, the Self Organizing neural network based feature extraction is used. The major advantage of
this approach stems from its robustness to small variation, ease of implementation and provides good learning rate. Self
Organizing neural network based feature extraction method provides good result even when certain pre-processing steps
like filtering, smoothing and slant removing are not considered.
Kohnen Self Organizing Map has a feed-forward structure with a single computational layer of neurons arranged in rows
and columns. Each neuron is fully connected to all the source units in the input layer.
Figure 1: Self Organizing Map Structure.
The aim is to learn a feature map from the spatially continuous input space, in which our input vectors live, to the low
dimensional spatially discrete output space, which is formed by arranging the computational neurons into a grid. This
mapping is performed by use of competitive learning, often called winner-takes-it-all strategy. The main steps of the
training procedure of SOM are presented on the flow chart given in Fig.2.
Figure 2: Flow-chart with representation of some of the important steps in the training process.
In training process of Kohnen self-organizing maps the vectors with N input variables are compared only with the
weights of neuron in the Kohnen layer [13].In each training step, samples are presented to the network, one at a time. For
each sample xi, the most similar neuron (i.e. the winning neuron) is selected on the basis of Euclidean distance [6].Once
the winning neuron c is found among the neurons in the Kohnen layer, the weights of both winning neuron and the
neighborhood neuron are adjusted using suitably selected learning rate η(t) and neighborhood function. The
neighborhood function is of rectangular and hexagonal shapes. The neighborhood function includes the learning rate
function η(t) which is a decreasing function of time and the function that dictates the form of the neighborhood function.
The form of the latter function also determines the rate of change around the winner unit. The neighborhood function can
be written as
hci(t) = α (t) . exp(- ||ri – rc||2 /2σ(t)2 )
(1)
in the case of the Gaussian neighborhood function around the winner neuron c.
(a)
(b)
Figure 3: Gaussian and Bubble neighborhood functions
Volume 2, Issue 12, December 2013
Page 284
International Journal of Application or Innovation in Engineering & Management (IJAIEM)
Web Site: www.ijaiem.org Email: editor@ijaiem.org, editorijaiem@gmail.com
Volume 2, Issue 12, December 2013
ISSN 2319 - 4847
A variety of neighborhood functions can be used. We can constrain the neighborhood function to be non-increasing
around the winner unit c. One choice for a neighborhood function is to use a Gaussian kernel around the winner neuron
as described above. This is computationally demanding as the exponential function has to be calculated, but can well be
approximated by the “bubble” neighborhood function. The bubble neighborhood function is a constant function in the
defined neighborhood of the winner neuron, that is, every neuron in the neighborhood is updated the same proportion of
the difference between the neuron and the presented sample vector. The bubble neighborhood function is a good
compromise between the computational cost and the approximation of the Gaussian. In the Figure 3 the two used forms of
the neighborhood function are illustrated. In the left, we see the form of a Gaussian (a), on the right the bubble function
(b).
The weights in Kohnen self organizing maps are updated as follows
wijnew = wijold + η(t) [xi - wijold ]
(2)
The learning rate is non-increasing function [10], which defines the intensity of the changes of the weights during the
training process. Some of the commonly used functions, which define the change of learning rate with time, are linear,
power, inverse functions. The intensity of correction of weights is determined by the shape of the neighborhood function.
The width of the neighborhood defined by the topological distance between the central neuron and farthest neuron
affected by the correction, during the training, also decreases.
2.3 Learning Algorithm of Self Organizing Map
Assume output nodes are connected in an array (usually 1 or 2 dimension) and the network is fully connected, means all
nodes in the input layer are connected to all nodes in the output layer. Use the competitive learning algorithm as follows.
1) Set learning rate η(t) and initialize weight matrix.
2) While stopping condition do step (3) to (7).
3) For each input vector x, do step (4) to (6).
4) For each j, compute the squared Euclidean error distance as follows
D (j) = ∑ (wij - xi )2 , where i = 1……n
j = 1……n
(3)
5) Now find index j, when D( j) is minimum.
6) For all the unit j, update weights as follows
wijnew = wijold + η(t) [xi - wijold]
7) Test the stopping condition.
3. RESULTS
The performance of the Kohnen Self Organizing Map Neural Network for learning of character has been tested for
various alphabets. More than 18 experiments were conducted by varying the values of the parameters namely training rate
coefficients and the number of neurons in the Kohonen layer of the Network. A general trend of increase in performance
with increase in samples was observed in the experiment. Followings images shows that how the Kohnen Self Organizing
Map learns the character “H”.
1
0.8
0.6
0.4
0.2
0
-0.2
-0.4
-0.6
-0.8
-1
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
Figure 4: Formation of alphabet H in MATLAB environment.
Volume 2, Issue 12, December 2013
Page 285
International Journal of Application or Innovation in Engineering & Management (IJAIEM)
Web Site: www.ijaiem.org Email: editor@ijaiem.org, editorijaiem@gmail.com
Volume 2, Issue 12, December 2013
ISSN 2319 - 4847
Input presentation number 5000
1
Neighbourhood size 2.3206
Learning rate 0.18206
0.8
0.6
0.4
0.2
0
-0.2
-0.4
-0.6
-0.8
-1
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
Figure 5: Learning of alphabet H in MATLAB environment.
In the above figures results of creating letter H in MATLAB environment is shown in Figure 4, whereas learning of
letter H is shown in Figure 5.When Self Organizing Map learns the letter H, the nodes of the grid matrix come closer at
the points where letter H is formed.
4. ANALYSIS
This paper presents our research efforts to develop efficient alphabet learning algorithm based on Kohonen's selforganizing feature map (SOM) algorithm. In the present work, the alphabets are created in MATLAB environment, then
their learning is performed using Kohnen Self Organizing map. In learning process nodes of grid matrix come closer at
the points where letter is formed and there is more space between the nodes of grid matrix where the alphabet is not
formed. Above results are shown for letter H. By using this technique one can learn any alphabet.
5. CONCLUSION
This work introduce a really easy however effective methodology based on Self Organizing Map to deal with the matter of
character learning. The result of this work showed better recognition performances in terms of accuracy and speed
compared with existing work in literature. The idea of learning characters using Self Organizing Map neural network has
proven its efficiency in terms of robustness. Future work can explore the necessity to integrate into an optimization
algorithm to further enhance the performance of the system.
REFERENCES:
[1]
[2]
[3]
[4]
[5]
T.kohnen, “Self Organization and Associative Memory”, Springer Verlag, Berlin, 1988.
R.Linsker, “Self-Organizing in a Perceptual”, IEEE Computer, Vol. 21, pp. 105-117, Mar. 1988.
Kohonen, “Self-Organized formation of topologically correct feature maps”, Biological Cybernetics, 43, 59-69, 1982.
Kohonen, “Self-Organizing maps”, Springer Verlag, 1995.
Somervou, P. & Kohonen, “Self-Organizing maps and learning vector quantization for feature sequences”, Neural
Processing Letters, 10, 151_159, 1999.
[6] Davide Ballabio & Mahdi Vasighi, “A MATLAB toolbox for Self Organizing Maps and supervised neural network
learning strategies”, Chemometrics and Intelligent Laboratory, 118, 24-32, 2012.
[7] Self-Organizing Map, 2nd ed. Berlin, Germany: Springer- Verlag, 1997.
[8] F. Bacao, V. Lobo, and M. Painho, “Self-organizing maps as substitutes for k-means clustering”, sIn Computational
Science – ICCS2005, Pt. 3, Lecture Notes in Computer Science, pages 209–217, 2005.
[9] T. Kohonen, “Median strings", Pattern Recognition Letters, vol. 3, pp. 309-313, 1985.
[10] T. Kohonen, “Self-Organizing Map of Symbol Strings”, Technical Report A 42, Helsinki University of Technology
laboratory of Computer and Information Science, 1996.
[11] J. Vesanto, J. Himberg, E. Alhoniemi, J. Parhankangas, “SOM Toolbox for Matlab 5”, Technical Report A57,
University of Technology, 2000.
[12] J. Zupan, M. Novic, I. Ruisánchez, Chemometrics and Intelligent Laboratory Systems 38, 1–23, 1997.
[13] D. Ballabio, M. Vasighi, V. Consonni, M. Kompany-Zareh, Chemometrics and Intelligent Laboratory Systems 105,
56-64, 2011.
[14] D. Polani, KohonenMaps, In: On the optimisation of self-organising maps by genetic algorithms, Elsevier,
Amsterdam, 1999.
Volume 2, Issue 12, December 2013
Page 286
International Journal of Application or Innovation in Engineering & Management (IJAIEM)
Web Site: www.ijaiem.org Email: editor@ijaiem.org, editorijaiem@gmail.com
Volume 2, Issue 12, December 2013
ISSN 2319 - 4847
[15] Carpenter G A, Grossberg S, “The ART of adaptive pattern recognition by a self-organizing neural network”, IEEE
Comput. 21: 77-88, 1988.
[16] Kohonen T, “The self-organizing map”, IEEE 78: 1464-1480, 1990.
[17] Anita, P. And Dayashankar S, “Handwritten English Character Recognition using Neural Network’’, International
Journal of Computer Science and Communication, 1(2): 141-144, 2010.
[18] Ahalt S C, Krishnamurthy A K, Chen P, Melton D E, “ Competitive learning algorithms for vector quantization”,
Neural Networks 3: 277-290, 1990.
[19] Carpenter G A, Grossberg S, “Self-organization of stable category recognition codes for analog input patterns”.Appl.
Opt.26:4919-4930, 1987
[20] Huang Z, Kuh A, “A combined self-organizing feature map and multilayer perceptron for isolated word recognition”,
IEEE recognition”, IEEE Trans. Signal Process. 40: 2651-2657, 1992.
AUTHORS
Amit Goyal received the B.Tech and M.Tech degrees in Electronics and Communication Engineering from
Punjab Technical University in 2002 and 2010, respectively. During 2002-2010, he worked as Asst. Prof. in
Electronics and Communication Department and taught various subjects like Neural Networks & Fuzzy
Logic, Wireless & Mobile Communication and Microprocessor & Interfacing etc. He is now Associate Prof.
in Maharishi Markandeshwar University, Sadopur, Ambala, Haryana.
Ankita Lakhanpal received the B.Tech degree in Electronics and Communication Engineering Engineering
from Punjab Technical University in 2011. She is now doing M.Tech from Maharishi Markandeshwar
University, Sadopur, Ambala, Haryana, under the guidance of Mr. Amit Goyal.
Shaveta Goyal received the B.Tech and M.Tech degrees in Electronics and Communication Engineering from
Punjab Technical University in 2002 and 2011, respectively. During 2002-2011, she worked as Asst. Prof. in
Electronics and Comsmunication Department and taught various subjects like Neural Networks, Digital Signal
Processing, Digital Circuit & Logic Design and Multimedia Communication etc. She is now Asstt. Prof. in RIMT-IET,
Mandi Gobindgarh, Punjab.
Volume 2, Issue 12, December 2013
Page 287