4.3. What is worth of noticing during further experiments?
(Translated by Krzysztof Kajdański; krzysiek.kajdanski@wp.pl)
I suggest that you spend some time playing with the program – I assure you it is worth it.
While entering different sets of data on input, you will discover fast that the examined neuron
works according to the quite simple rule. It just treats its weights as the model for the input
signal it wants to recognize. When there is given such a combination of signals that
corresponds to the weights of the neuron – it finds in it something familiar and reacts
enthusiastically – you receive high output signal then. When different signals are given – the
neuron shows indifference (low output signal) or even the aversion (negative output signal).
Well, that's its nature!
Careful examination will indicate that the behavior of the neuron depends only on the angle
between the vector of weight and the vector of the input signals. Have a look at it and use
the next program which will also state which flowers neuron likes and which dislikes – this
time the model of the ideal flower will be represented as the point (or vector) in so called
input space.
The term of the space (and connected with it the weight space) is quite important and quite
simple at the same time. It is important for you to focus your attention on this subject for a
while – it is not very complicated, though at first time it seems to be the secret knowledge.
When you set the preferences of a neuron – it means when you tell him how much it likes
fragrant flowers and independently colorful flowers – you in fact set the parameters of the, so
called, weight vector. You can draw two axes and on one (let's say – the horizontal) you can
mark the values of the first feature (fragrance) and on the other (the vertical for example) the
values of the second one (Fig. 4.8). If you want to mark the preferences of the neuron – you
can mark a values on the axes and then the point created by these coordinates will correspond
to the neuron's preferences. For instance, a neuron that 'values' only the fragrance of a flower
and color is rather indifferent to it – will be represented by the point located maximally to the
right (high value of the first coordinate) but on the horizontal axis of set (zero or low value of
the other coordinate). Apart from that, the neuron that you want to like puttyroots – flowers of
beautiful colors and weak, sometimes even unpleasant, smell – will be located high on the top
of the vertical axis (high value of color) but even on the left of this axis (acceptance to not
nice smell).
the values of the weights
for the color attribute
the point representing the neuron's features
'favorite' neuron's color
this vector also represents the neuron's features
’favorite’ neuron's smell
the values of the
weights
for the fragrance attribute
Fig. 4.8. The representation of the neuron's features as the point and the vector in
the attribute space
Identically you can treat any other object (flower) that you present the neuron to mark.
Its color will be on vertical axis and its fragrance on the horizontal one. If it is Lily-of-theValley it will be represented by the point located maximally to the right (it's hard to get
anything smelling nicer) but definitely low (color is not the advantage of this flower). If you
decide to show it a gillyflower it will be located high (it's generally nice coloured) but
definitely to the negative site of the horizontal axis (the smell of gillflower is hard to consider
as nice). In the bottom left part of the coordinate system you will find a sundew (it is a plant
that consumes insects – it lures flies with the appearance of rotting meat and similar smell),
and majestic roses will be found in the top right corner.
During consideration concerning neural networks it is convenient to mark objects (tested
flower and the images of the neurons of the ideal flowers) not only as points in the coordinate
system but as vectors. You will ackquire needed vectors joining the point with the beginning
of the coordinate system. That is exactly the way program Example 01b will work. It –
similary to the previous one – can be found in the Start1 menu. Playing with this program
notice the preceding facts:
 The value of the output depends mostly on the angle between the input vector
(representing the input signals) and the weight vector (the ideal object accepted by the
neuron). It is illustrated in the figure 4.9.
 if the angle between the input vector and the weight vector is small (both vectors are
located next to each other) – the value of neuron's output is positive and high
 if the angle between the input vector and the weight vector is big (both vectors create
angle greater than 900) – the value of neuron's output is negative and high
 if the angle between the input vector and the weight vector is close to 900 – the value
of neuron's output is low and neutral (near zero)
1
If you haven't installed the example programs yet – do it – the fun is about to start right now
 if the length of the input vector is far more smaller than the length of the weight
vector - the value of the neuron's output is neutral (near zero) independently on the
direction of the output vector;
All of the described characteristic features of the neuron's behavior you can test on your own
using the Example 01b program. Although the pictures produced by it, aren't so graphically
worked out, as the figure 4.9, they should be easy to understand. They would be the
convenient basis to collect your own experiments letting you to learn and remember what a
neuron actually does.
The work with the program is quite easy. The only thing you need to do is to click in the area
of the located on the left chart. First, click it with the right button to set the location of chosen
point corresponding to the neuron's weight factors (see Fig. 4.10). You will be shown the
point and its coordinates. Of course you can change it at any time, clicking again in the other
part of the chart or modifying the coordinates manually – the same way you did it in the
Example 01a program. Now click on the chart with the left button, to locate the position of
'the flower' and watch the answer of the program. If the neuron likes the flower (on the right
there is the value of the output signal so you can easily get to know what your neuron thinks
of the flower) then the appropriate point is marked on the chart with red (like a mountains on
a map – see Fig. 4.11). If the quotient is negative – point is marked with blue (like a seabed
on a map – see Fig. 4.12). When the reaction of the neuron is neutral – corresponding point is
marked with light blue (Fig4.13). After some time you will be able to imagine how the areas
corresponding to the decisions in the input space look like. To picture them easier you can try
to drag the mouse pointer over the chart with one of the buttons pressed to observe the results
changing.
the values of the
neuron's weights
for the color attribute
and the value of this
feature for a specific
flower
the point representing the neuron's features
'favorite' neuron's color
the point representing the flower's
features
the color of the flower
The angle determining the neuron's answer
'favorite' neuron's smell
the smell of the flower
the values of the neuron's weights
for the smell attribute
and the value of this feature for a specific flower
Fig. 4.9. Mutual position of the weights vector and the input signals vector – as the factor
determining the value of neuron's response
Here are the weights of
the neuron's inputs
located (click right
mouse button for fix it)
Here you can see
values of the
fixed weights.
The similar element you will see in many example
programs. If you hover on it and wait for a while you will
be able to read the guidelines.
Fig. 4.10. The window of the Example 01b program with the model object marked
We put flower here
(click left button)
Here you can see
its coordinates
Here the program
shows us the output
signal
Fig. 4.11. The presentation of the input vector location for which the output signal is
positive
Our 'flower'. The neuron
doesn't like it this time so it's
marked with blue color.
Fig. 4.12. The presentation of the input vector location for which the output signal is
negative
Fig. 4.13. The presentation of the input vector location for which the output signal is neutral