4.2. What can be expected from a single neuron?
(Translated by Krzysztof Kajdański; krzysiek.kajdanski@wp.pl)
As you probably remember – neuron gets input signals, multiplies them with factors named
weights (for every input set individually during the learning process) and then sums created in
that way ratio. Let us stop for a while on this simplified model. Although you know already
that in more complicated neurons these summed signal are converged to the output signal
with an appropriate function (generally non linear one), but believe me - in our first simplified
neuron it is not necessary. We will examine the behavior of this simple linear neuron. What
does the value of the output signal depends on?
It is easy to show that it depends on level of acceptance between the input signals on every
input and the values of the weights on these inputs. Although this rule matches ideally only
for the normalized input signals and weights (I'll specifically explain later how you should
understand it). However even without specific normalization the value of the output signal
can be treated as the measure of the similarity between assembly of the input signals and the
assembly of the corresponding weights.
You can say that a neuron has its own memory and stores there the representation of its
knowledge - the pattern of the input data, which it is sensitive for (in the form of the actual
values of the weights). If the input signals match the remembered pattern – the neuron
'recognizes' them as something familiar and answers to them with the strong output signal. If
there is no connection between the input signals and the pattern – output signal in near to zero
(no recognition). There is possible a total contradiction between the input signals and the
weights values. The linear neuron generates the negative output signal then – the stronger
contradiction between the neuron's image of the output signal and its real value, the stronger
output.
To begin with I encourage you to try to run the simple program named Example 01a.
You can start it and just perform a few experiments that program allows on. You can benefit
even more if you decide to change this program or even try to make it better. Then you will
learn much more about the network. After initialization of the Example 01a program you will
see the window shown on Fig. 4.2. In its top part there is the text explaining what actually we
are going to do.
The explanation of
the purpose of the
program. Just read
it.
Fig. 4.2. The window of the Example 01a program just after start
Blinking cursor signals that programs is waiting – for you! - to give it the weight of the
neuron's input which is connected to the fragrant value. You can enter it manually by typing
a number, or by clicking the arrows next to the field, or by pressing the up and down arrows
on the keyboard. After inserting the value for the fragrant feature, go on to the next field,
which corresponds to the flower's second feature – color.
Let us assume that you want your neuron to like colorful and fragrant flowers, with more
favor for color. After giving an appropriate answer the window can look like in the Fig. 4.3.
Described program, as every other you will use, gives you the possibility to change your
decision and choose another input data. Program is trying to update the results of its
calculations.
We change
the
preferences
of the neuron
Fig. 4.3. The initial stage of the user's communication with the Example 01a program.
After inputting the data concerning the neuron's features (in fact these are values of its
weights) there is time to check the way neuron works. You can input different sets of data in a
way shown in the 4.4 Fig. and then the program will calculate and answer what output signal
the neuron has generated and what it means. Remember that you can change the neuron's
preferences and the description of the flower at any time.
1
Let us show our
neuron some flower.
2
Push that button to see
the neuron's answer.
3
Neuron is kind of
glad. Who said that
only women like
flowers?
Fig. 4.4. The final stage of the user's communication with the Example 01a program.
Remark for the inquisitive readers: If you use mouse or arrow keys to input data you don't
have to – as you have probably noticed – click the Racalculate button every time you want to
see the result; calculations are made automatically. When you input the number from
keyboard you have to click it. That is because computer doesn't know if you have finished
entering the number or you are just in the middle and went to get yourself a tuna sandwich.
The next stage is to experience our neuron with unusual situation. The point of the experiment
shown in the Fig. 4.5 is to observe how the neuron reacts for object that differs from
remembered 'ideal' (colourful and fragrant flower). We showed it a flower full of colors but
with no fragrance at all. As you can see – it liked this flower as well!
The flower is kind of
weird...
I like it less, but it's
still nice!
Fig. 4.5. The program Example 01a in unusual situation.
Changing the parameters of the flower you can observe what the neuron will do in any other
circumstances. The examples of such experiments are shown in the Fig. 4.6. I tested the
behavior of the neuron when the flower has nice fragrance, but practically no color (it likes it
anyway), and then colorless and smelling badly (this one is not likeable at all).
This one is
colorless but at
least smells a bit.
'All right… But for
my birthday I want
to get nicer one'
This one is
colorless and
smells bad.
'I definitely don't like
this flower. Where
did you get it from?
Trashcan?'
Fig. 4.6. The example of the another experiment with the Example 01a program
And now let us test our neuron with a little more complicated task: let us check if it accepts a
flower that smells badly, if it is colorful enough.
Fig. 4.7. One another example of the experiment with the Example 01a program
As you can see, there is a plenty of ways to experiment. You can also try to change the
preferences of the neuron and see how it acts in different situations when it, for example, likes
when a flower smells bad (the weight corresponding to fragrance can be negative).