4.4. How to manage with the bigger amount of the inputs of the
neuron?
(Translated by Krzysztof Kajdański; krzysiek.kajdanski@wp.pl)
The examples above are nice and simple because they concern only one neuron with two
inputs. In real application of the neural networks you typically have to do with task where
many inputs are considered (because the solution of the problem you try to solve with the
neural network often depends on the huge amount of the input data) and there is a need to
apply many cooperating neurons to gain the level of compliance of the network appropriate
for the current problem.
That's the point in the neural networks where jokes ends and problems begin. You cannot
draw this (because nobody is able to imagine points in the input space having more than ten
dimensions!) and that's why everyone thinks that it must be very hard. However – it doesn't
have to be that way. I'll convince you in a minute. Just try out the program Example 01c (see
Fig. 4.14)
Fig. 4.14. The beginning of the conversation with the Example 01c program
The program asks to enter the number of the neuron's inputs. You can accept the default value
(5) and click Next. Then, as previously in the Example 01, you should input the weights,
which define the model of the signal, on which your neuron should react positively. Fill in the
column marked as w(i) (Fig. 4.15). Then you enter the values of the input signals in the x(i)
column and then program will calculate the output. Really simple, isn't it?
The
weights
The input signals
Fig. 4.15. The sequent part of the conversation with the Example 01c program
Don't worry that the program tells you something about some strength. You will get to know,
what you can use it for in a minute.
During experiments with this program you can easily notice, that even when we work in the
five dimensional space of weights and signals (that's a kind of space even Albert Einstein
didn't work with – he stopped on the four dimensional continuum) – we can easily foresee
what neuron will do. When while setting the coordinates of the input signals vector, you set
them to be similar to the corresponding weight vector coordinates – these values are
obligingly recollected by the program – you will get high value of the output signal. It is quite
obvious: similar values of weight vector and input vector means that these vector lay close to
each other and this – as you know from previous consideration – is a condition of getting the
high output signal. On the other hand, if the values of the coordinates of input signals vector
are set similarly to the values of the weight vector – but with the opposite sign - you will
make the input signal vector pointing another direction than the weight vector. Of course the
effect of this action would be the neuron generating strong negative output signal. Finally, if
you don't try to keep any accordance between the input signals and the weights – there is a
chance that your multi-dimensional neuron will remain haughtily indifferent towards the input
signals producing output signal with very small absolute value.
During experiments you will notice that high values of the output signals are returned in two
situations: when input signals correspond to the weights of the neuron (that's what we
expected) or just by entering huge input values, where weights are positive. The first way of
obtaining the high output is intelligent, sophisticated and elegant – well, whatever, since the
same (sometimes even better) effects we can get using the brute force and the method number
two. Therefore during entering the input signals you should try to make them of the same
strength (using the parameter given by the computer estimating the actual strength). Only then
you will be able to correctly interpret and compare your results. Similarly comparing the
behavior of two different neurons (with different weight values) for identical input signals –
they should tend to have the same value of memory trace strength – it is just the length of the
weight vector. In the network with the bigger amount of neurons the meaning of the strength
of the input signals radically decrease, when – stronger or weaker input signals – get to every
neuron and the difference of the signals from different neurons decide about the effect: these
better 'tuned' to the input signal and those worse 'tuned'. We will focus on that in a minute.
However when we consider only one neuron – dissimilar values of input signals can make
results harder to interpret – that's why we should agree on choosing the input values for
examined neuron in a way, when the sum of their squares is (estimating – high precision is
not needed here) between 5 and 35, for instance. Because program calculates the strength as a
square root from the sum of squares of the coordinates (that's the formula for the length of a
vector), let us agree that the strength of the signals should be between square root from 5 and
square root from 35, which is somewhere between 2 and 6.
Why should we choose these values? Because during my work with the program I found out,
that when entering small random integer values for five inputs of the neuron – you will get
(more or less accurately) values from this range. However if you prefer, you can choose any
other value and it is going to work, but remember to keep with it. The same limitation should
be applied to the values of weights (it will turn out to be useful, you'll see!). Thanks to it, it
will be easier to check, if the input signals 'match' the values.