6.4. How to present action of nonlinear neurons?
(Translation by Joanna Jaworek-Korjakowska; jaworek@agh.edu.pl )
The results of nonlinear neurons and the neural networks which they built can be easily interpreted by drawing
the so called input signals space and indicating which values of these signals trigger positive neuron’s response
(sends signal +1), and which of them trigger negative response (it sends signal -1).
Let’s put our heads together and think how to interpret the data shown in the figure of input signals
space?
To begin with, each point inside the large square which is going to be created for you by the program
Example 06c, represents the set of two data (equivalent to the horizontal and vertical coordinate of the position
of a point), comprising the current input signals for the neural network. It can be imagined that, for example, the
analyzed network is the brain of the hypothetical animal equipped with two receptors – let’s say, the primitive
sense of sight and hearing. The more stronger the signal received by the sight is, the more changes the location
of the point to the right side. Also, the stronger the sounds is, the higher the location of the point in the figure is
(fig. 6.7).
It is light
and loud here
Sound
And there are the data
of the particular environment
of the hypothetical animal
It is loud
and dark here
(disco)
It is quiet
and dark here
(bedroom)
The subject of learning is to determine
the environment where the animal
feels well.
It is quiet
and light here
(beach)
Light
Fig. 6.7. The experimental world of the animal whose brain is the analyzed neuron
As it can be concluded, the bottom left corner of the square represents the conditions of silence and
darkness (bedroom), whereas the upper right corner refers to the maximum loud and light surrounding (the Main
Square at noon). The upper left corner may be associated with a disco (dark and loud), while the bottom right
corner – with the beach ( sunny and quiet). The attempts to imagine your own analogies for the other points
inside the square can be quite a challenge.
Each of the above-mentioned situations can be perceived by the modeled ‘animal’ in a positive way (the
point will then have intensive red color), or in a negative way (the color of the point will be blue – see fig. 6.8).
Now try to imagine that the ‘brain’ of our modeled animal consists of only one, but nonlinear, neuron.
Obviously, this neuron must have two inputs, because in the situation we are considering, the information is
provided by two receptors of ‘sight’ and ‘hearing”. For each of these receptors, the two weights must be
determined. For example, when the first weight is positive, there is an indication that your ‘animal’ is supposed
to like light surrounding, while by giving the negative value you ‘encourage’ your animal to hide in a dark
corner. The same procedure can be used with the second weight, thus you can determine if your animal prefers
peace and quiet, or it is an eager fan of Metallica concerts.
Rozdział 6, str. 1/4
Sound
Your animal
is unhappy
Your animal
is happy
Light
Fig. 6.8. Method for determining the "well-being" of the animal
This is the way to program the brain of your animal, which will then be tested, by providing at the input
various combinations of input signals (naturally, two signals). Each initial combination of signals can be treated
as an indication of a specified "environment" where the animal is experimentally placed. The proprieties of this
environment are symbolized by a point on the plane of the picture –the value of the first signal (lighting)
indicates the X-axis of this point, while the value of the second signal (sound) – its Y-axis. For such a point (that
is, for such set of input signals), the neuron makes its decision – in one case it will be +1 and in the other -1. In
other words, one type of environment will be accepted by animal, while the other will be rejected. Try to prepare
a map depicting the preferences of your neuron by coloring in red those points which were accepted by the
neuron (by sending at its output +1), and using blue to color the points which were ‘rejected ‘ by the neuron (-1
signal). In fact, there is no need to do it on your own – you can as well use my program Example 06c which will
perform all these tasks for you. To begin, you need to provide all the weights to define the preferences of your
neuron (fig. 6.9).
Parameters describing the
nonlinear neuron
Each point inside the square symbolizes the assembly of
two data (corresponding to the horizontal and vertical
position of the coordinates), which are the current input
signals for the neural network
Fig. 6.9. Setting of the neuron parameters in the program Example 06c
Rozdział 6, str. 2/4
Then you can freely check what value (+1 or -1) is generated by a neuron at its output at the definite
points on its input signals plane. The program is very useful – it not only performs calculations and draws the
above-mentioned map readily, but it also generates the required input signals for neurons and provide their
coordinates at the bottom of the window (on the right side – see fig. 6.10).
Fig. 6.10. Presentation (with color) the neuron response to a given input signal
As long as you feel like watching the position of each consecutively presented point, you can press the
button Show one random point and check the position of a new point on the screen (currently generated point is
already marked by projecting its coordinates on axes, so it can be easily distinguished from the points that were
previously generated). While checking the position of a new point, you may wonder why the neuron has
accepted or rejected it. It is easy, since the data concerning the presently discussed point are additionally
presented in the line below the picture. Many points will be needed to create a map which would show precisely
in which areas of the input signal planes the neuron’s response is positive, and in which the response is negative
– at that moment, you can press the button Show multiple points. It will switch the program into the continuous
mode in which the subsequent points are automatically generated and displayed on the "map" (you can always
switch back to the previous mode by aborting the process with the button Stop). After a while, the outline of the
area with the positive response of the neuron will appear, as well as the area where the neuron’s response is
negative (fig. 6.11).
Rozdział 6, str. 3/4
Fig. 6.11. A screenshot of the program Example 06c after generating many points
The border between these areas is – clearly – a straight line, therefore it is sometimes said that a single
neuron performs a linear discrimination of the input plane. It might sound wisely and enigmatically, but it
only refers to what you could see on the screen while conducting the experiments described here.
If you wish to see the map of the neuron’s response without putting much effort into generating the
points, then the program Example 06c also gives you the possibility to do so. In order to do that, you need to use
the Show the entire plane button (fig. 6.12). One of interesting experiences which I strongly encourage you to
do might be the change of nonlinear characteristic of neuron by the choice of the other one, available on the list
Transfer function type (in the group of options connected with neuron).
Fig. 6.12. A screenshot of the program Example 06c showing the response map of the neuron
Rozdział 6, str. 4/4