7.4. How does the multilayer network constructed from nonlinear
elements work?
(translation by Piotr Czech, Piotr.Czech@polsl.pl)
Multi-layer networks which are constructed of nonlinear elements are currently the basic tool to
use neural networks in practical applications. There are a lot of ready programs which may be
successfully used when needed. I will tell you about the ready programs modelling various
kinds of networks (both commonly available – public domain and a lot of commercial ones)
some other time because what I want to present here are two simple programs which will make
it visible for you first how a multilayer network works (program Example 08a), and then (what
you will find in the next subchapter) I will show you the program Example 08b, where you will
find a method of teaching such a network.
If you analyse those two subchapters carefully and conduct the experiments with the use of offered
programs, you will be able to see yourself what this all backpropagation method is really about. I
really encourage you not to hurry and to read very carefully, think it all over and precisely in person
try out all what is going to be presented here, because backpropagation is the key and core of
techniques in neural networks. When you get to know precisely and understand this technique –
most issues connected with networks will be easy and obvious for you. If you do not, however, get
involved in this game which I offer to you here then you will not know exactly “what is going on”. I
am sure you can live without it, but is it really worth it?
Before you start the next programs – read a few additional remarks concerning how they
work which is not (unfortunately) as obvious as the functioning of the simple programs I showed
you before.
Program Example 08a shows the functioning of complex network built out of only four
neurons really precisely (it will be easier to follow them, see fig. 7.5) which form three layers –
input, which cannot be taught (at the top of the screen), output, where the signals will be copied on
the output of the network where they will be assessed and taught (at the bottom of the screen) and
the most important hidden layer, presented in the central part of the screen. Neurons and signals in
the network will be numbered with two numbers: number of the layer and following number of
neuron (or signal) in a layer.
And now a few words about the functioning of the program and instruction of use.
At the beginning the program asks you to give it the weight coefficient for all neurons in the
whole network (fig. 7.5).
Inputs (1,1) and (1,2)
introduce the external
signals to the network
Each neuron of hidden layer
receives 3 weights: for
neurons (1,1) and (1,2)
Hidden layer
Each neuron of output layer
receives 3 weights: for
neurons (2,1) and (2,2)
Output layer
Fig. 7.5. Setting the parameters and input signals for the network in program Example 08a
You should give exactly 12 of them, because – as I previously mentioned – there are 4 neurons
being taught here (two in the hidden layer and two in the output layer) and each of them has three
inputs (two from the neurons of the previous layer plus an additional input – also possessing the
weight coefficient – connected with “the threshold” which presents, in a given network, the
generator of the artificial, constant BIAS signal). When you work hard on that, think how nice
it is in typical networks, which possess hundreds of neurons and thousands of weight
coefficients, that you do not need to write in all the data personally but they are set on their
own – automatically – exactly during the teaching process. In the program described here, I
gave you the option of “writing in” all the parameters because I would really like you to have
the chance to check if the results which are provided by the network on the output of its
neurons are in accordance with those which you think should be right. In order to do it you
will be able to shape the input signals to the network freely and observe the signals both on
the inputs of neurons and on the outputs. For the lazy people there are of course the default
values… It is best to chose simple whole numbers as weight coefficients and input signals
because in this way you will easily check if the results which you achieve from the network are
in accordance with those which you calculated yourself and you will check if you really know
and understand what goes on in the network.
After giving the coefficients you must additionally give the values of both input signals
which the network will get on its inputs (fig. 7.5). Now is the moment when the program is
ready to work. You must have noticed how the structure of the network is presented (places
where particular neurons are placed and where particular signals will be displayed – see fig.
7.6.).
Places where the program
will display the calculated
signals
Fig. 7.6. Beginning of the network demonstration in the program Example 08a
If you click the button Calculate you start it and you will conduct the simulation process of
sending and processing the signals in a modelled network. First – in each following layer there
are calculations made and displayed (against a red background) of the sums of the values of
signals multiplied by the right weights of input signals (together with BIAS component). Next,
the calculated values of the output signals will be shown (“answers”) of particular neurons (also
initially against a red background), which were created after pushing the summed input signals
through transfer function (with the shape of a sigmoid). The roads of the signals being sent are
marked on the picture (fig. 7.7.) in the form of blue arrows.
Fig. 7.7. Visualisation of the direction of signals transmission in the initial section of the
network
Next the answers of neurons of the lower layer become the inputs for the neurons of the higher layer
and the whole process repeats itself (fig. 7.8).
If you press the button Calculate again, you can observe the movement of the signals through
following neurons from the input to the output of the analysed model of the network. You can
change the values on the inputs and observe the outputs – as many times as you want to check if
you really understand correctly what really goes on in the network. I advise you to spend some
time on detailed analysis and thinking over each number on the screen (you can even calculate it
alone on your calculator to be sure the numbers are matching those which you would expect on the
basis of your knowledge about neural networks) because only in this way will you know exactly
and understand precisely what the network really does.
Fig. 7.8. The last stage of the functioning of the program – all signals have been calculated