10.2. What is the self-organization in the network and what it
might be helpful for?
(translation by Bartosz Wilczyński; b.wilczynski.89@gmail.com)
Even this small piece of knowledge you have gained in the previous section, allows you to find out what sleforganizing networks are able to. They form - totally by their own, solely on the basis of observations of input
data - certain representation of the set of input signals into a set of output signals, whereby the curve meet
certain general criteria (I will discuss it in a moment), but in no way determined in advance by the creator of the
network or by its user. Structure and properties of projection which we need should arise spontaneously in a
coordinated self-learning process of all the elements of the network. Such spontaneous creation of needed
signals projection is called self-organization. From a certain point of view, it is just another form of self-learning,
but if you look at its effects (and you will see many of these, because the program used in this chapter gives you
all the interesting pictures), you will probably admit that we are dealing with a clearly higher level of adaptation
of the network, involving not only optimization of the parameters of each neuron separately, but - and this is just
a novelty - the coordination of activities of neurons, giving to calculations highly desirable effects of grouping
and collectivity.
Let us explain these concepts. The effect of grouping lies in the fact that the network in the process of
self-organization is trying to divide the input data, distinguishing among them certain classes of similarity. This
works so, that among the input objects (described by the input signals) such groups of them are detected
(completely automatically!), where you can place the signals which are similar to each other in their group and
which are clearly distinct from the signals assigned to other groups.
Such clustering of data is very useful in many applications, so a number of specialized mathematical
techniques allowing the analysis and the creation of such groups of them (mostly statistical) were developed.
Mentioned techniques are called in Polish "analiza skupień", and in the world literature they are better known
under the English name of the cluster analysis. They are quite willingly used in the economy - for example, to
detect which companies are similar to each other and can therefore pose a similar return prognosis of investment,
either in medicine - for the study which symptoms are indicatives of the different varieties of the same disease,
and which already indicate the presence of new - perhaps not known - disease.
Figure 10.5 Network grouping the input signals (components corresponding to the typical coding of
digital image) in the collections corresponding to different possible colors.
In Figure 10.5 you have an example of neural network which is able to group the input signals (color
image pixels coded in the usual manner using the three components of RGB) according to the criteria of
similarity of their colors. Figure 10.6 shows the effect of the network without self-organization process (on the
left) and after the process of self-organization (on the right). Figure 10.6 was created in such a way that in every
place where there is a topological layer neuron, was drawn a box filled with such color of the pixel input, which
makes this particular neuron able to become a "winner".
Figure 10.6. Activity projection of the network from Figure 10.5. On the left - before the process of
self-organization, right - after the process of self-organization.
Grouping of image pixels together, having a similar color, is not a task that could give any special benefits, so
you can see the figure 10.6 as a curiosity, and then you can shrug and say, "Okay, but what do we need that for?"
Well, look in Figure 10.7. This figure is showing what was in the topological layer of the self-organizing
network, which was told to compare data of different companies. At the entrance of the network various data
was given: type of business, capital owned, number of employees, balance data sheet for several recent quarters,
reflecting the profits gained or losses of the company etc. The network grouped these companies, and so placed
the parameters that each neuron had a habit to become the "winner" when the information about "his" company
was shown. Then - after waiting a year - those companies were examined. It turned out that some of them fell
down or they are threatened with bankruptcy - those whose neurons are indicated on Figure 10.7 in red. Others
on the contrary, proved to be affected by economic stagnation. Neurons, which are assigned to them in the
automatic process of self-organization are indicated in Figure 10.7 with the blue colour. And finally was a group
of companies that have developed in harmony - you guess they are marked on green.
Figure 10.7. The effect of clustering in the self-organizing neural network. The description in the text.
Such a very impressive examples of self-organization, leading to the ordering of certain data, you can find plenty
on the Internet - just type in Google the term "self-organizing networks" or better - a commonly used acronym
SOM (Self-Organizing Maps). By doing this very quickly you can find out that tasks of grouping the input data
described here into classes of similarity are needed and finds practical applications.
Neural networks engaged in the process of self-organization are therefore a very attractive tool to
perform the task of grouping the input data and the creation of classes of similarity among them. The
attractiveness of neural approach to the concerned problem is mainly that the self-organizing process can be
carried out here absolutely automatically and spontaneously, and the creator of the network does not need to give
any clues to it because the set of the necessary information is contained in the same input data (from which the
network itself will extract the observation that some are similar to each other, while others do not). In addition,
after the network is learnt how to cluster the input data, very useful systems arise - "winning" neurons, which
specialize in identifying specific classes of input signals, so they are detectors and can be used as their
indicators.
Having talked briefly about what is the effect of clustering in self-organizing neural networks I will tell you even
in a few words what I have meant by the collectivity of activity of the network.
Well, the networks in which the self-organization takes place are so organized that what a neuron
identifies, to a large extent depends also on what other neurons (located in its vicinity) identify. In this way,
community of neurons (or their collective) could process information in a fuller and richer way than each of the
individual neurons taken. See Figure 10.8.
Figure 10.8. The effect of recognition of geometric figures of self-organizing neural network. The description in
the text.
The figure shows the results of my research related to the recognition by the self-organizing neural network of
simple geometrical figures, shown to it as the images. In places where the network placed (totally by its own!)
the neurons indicating the different shapes of geometrical figures shown to it - there are pictures of the relevant
figures. Make with me a brief tour of this map, and you'll find that this distribution of neurons is certainly not
accidental!
We start - with red arrows, which I added to the figure - from the neuron, which becomes the "winner"
whenever a square is shown to the network. Not far from it there is the neuron indicating a pentagon, and a
little further - neuron signaling an octagon. Rightly: a pentagon is more similar to a square than an octagon!
Near the octagon the neuron recognising a circle was placed. From it short path leads to neuron signaling an
ellipse, which in turn is not far from the neuron which understands a semicircle. Semicircle has a clear affinity
for a smaller piece of the circle, called by us a sector, and this in turn is clearly similar to the shape of a
triangle...
As you can see, the effect of collective self-organized networks is very interesting. What is more, this is
the effect, which is a contribution to the broader considerations: Well system, or just the community of
appropriately linked and cooperative elements makes it possible to obtain new forms of behavior and new forms
of actions, much richer than might be expected taking into account each of the elements separately. For example,
every single insect is quite stupid and primitive animal, and yet the community of insects (bee colony, nest,
termite mound, etc.) are capable of deliberate, complex and undoubtedly intelligent actions. Once I describe my
experience that I myself have collected during the computer modeling of bee colony and then I will write about
it more - but not in this book.
Returning to the networks with self-organization, it is clear that they are quite convenient, useful and
frequently used tool, and also - perhaps primarily - are very interesting object of research. You will learn details
of the self-organization process ongoing in neural networks by studying precisely the material presented in this
chapter and - as usual - doing the experiments with the program especially prepared for this purpose. The key to
understanding and using of self-organized networks is the concept of neighborhood of neurons - and right now
we will consider it for a moment.