5.7. Can network be taught signal filtering?
(Translation by Piotr Czech, Piotr.Czech@polsl.pl)
Imagine that you have some signal with some noise put on it.
Telecommunication, automatic, electronics and mechatronics engineers torture themselves with
such signals every day, so the above situation is not strange.
If you ask one of those experts, what should be done to clear the signal from the noise, then with
feeling of superiority you will be instructed that you should use a filter.
If you tackle matter more precisely, you will get to know that a filter is a device (usually –
electronic) that allows to pass the useful signal , but stops the noise.
It works very good, and as the result of that fact we have efficiently working phones, radios, TVsets etc.
But each expert will confirm that it is possible to create a good filter only when the noise has
some property which is not present in the useful signal.
If something has this property – than it should be stopped by the filter as a noise.
If it does not have this property – than it is allowed to pass.
It is simple and effective!
However, to get the system work you have to know a lot about a noise, that disturbs your signal.
Without the needed knowledge you will not create a filter, because the filter does not posses the
data, which information should be kept and which should be stopped.
Unfortunately, it often happens that you do not know what the source of the noise disturbing your
signal is, and what its properties are.
If you send a probe rocket somewhere far away into space with the intention to gather signals
describing for example an unknown planetoid – then you do not know, what rubbish may come
and hang about with signals from the probe during its journey over millions of kilometres in
interplanetary space.
How to create a filter then?!
Yet, it is possible to separate a useful signal from unknown noise by using adaptive filtration
method.
In this case adaptive means teaching the signal receiving device the previously unknown rules of
separating the signals from noise. Particularly, the neural networks can be trained to filter and
select signal from noise.
To achieve this, samples of disturbed signal are treated as input signals for network, and “clean”
signals are used as output signals.
After some time the network will learn to select the undisturbed output signals from disturbed
ones and will be able to work as a filter.
At the beginning the
program will
automatically project
this window and will
ask for filling in the
indispensable
parameters needed to
create used signals and
to define the
configuration of the
network
Fig. 5.12. Disturbed signal which has to be filtered by neural network
Let’s consider a real example.
Let’s take a standard signal – for example a part of sinusoid signal – both a “clean” and a
disturbed one.
Fig. 5.13. Standard signal which has to be reconstructed by neural network
These signals – in the form of a file which allows to teach networks– can be created by Example
05 program.
This program will automatically produce a file with data called teaching_set, which can be used
to teach a simple network.
The window in which you can modify the required parameters to generate this file, appears when
we start the Example 05 program.
This window is shown in the figure 5.12.
Unfortunately, in this window the program demands from you to give the network size (Network
size), expected noise value (Noise level measure), frequency (Frequency) and number of
teaching steps (Number of teaching steps).
Fig. 5.14. Results of signal filtering after one step of teaching process
Looking at all these needed details, you can definitely become discouraged from the fun of
working with this program, because it looks mysteriously and vague, and in addition where would
you know these all necessary values from? It is not as bad as it seems, though.
Looking closely at the described window, you will find some information written in each of the
windows.
These values are selected and tested by me for the program to work well and show interesting
effects. Therefore if you do not have better ideas, then at the beginning you can use these default
settings and simply (without further ado) accept it by clicking the button Apply.
Of course in future, when you have the will and ability, each of these values can be changed at
will – at least just to observe “what happens, if”.
Fig. 5.15. Results of signal filtering after five steps of teaching process
The freedom, that the program gives you in this field, is very large.
The values that have been already introduced, can be erased any time (Cancel button), and you
can also immediately reconstruct again all the settings designed by me using Default button. As
the result the Example 05 program can really be liked, , despite the unfriendly impression you can
have at the beginning, and by using its operations, you can observe the work of the network,
which will learn to filter the signals by itself.
During the experiment the network will be correcting signal step by step, but every time you can
observe what the not-filtered signal (fig. 5.12) and the original signal, that is the signal without
noise (fig. 5.13) look like.
To achieve this aim, it is enough to mark proper option (reference or noisy) in Show signal group
in the range of the right margin displayed by the program on the screen.
Now, let’s take a look at how it works.
Fig. 5.16. Results of signal filtering after twenty steps of teaching process
The first results of filtration, after few steps, are not very promising (fig. 5.14 and 5.15), but the
constant teaching of the network (by repeated clicking the button Next) leads to the fact that
finally the well trained network learns to filter signals in a nearly perfect way (fig. 5.16).
The teaching process can proceed automatically or step by step.
It means that the program shows, at your request, all its mysteries or can pass many stages of
teaching automatically, during which the network makes its work perfect, and you can view only
the final effect at ease.
You decide about the teaching mode of the process, depending on your choice of Simulation
mode group(Auto or Manual).
Fig. 5.17. Estimation of effectiveness of signal filtration after twenty steps of teaching
At the beginning, during first experiments, it is worth to observe step by step how the network
teaching process proceeds, and if it is the case, one should choose Manual mode.
During next experiments it is possible to modify the number of steps (by using Menu>Configuration option on the upper edge of our program’s window) and try the longer teaching –
for example 50 or 100 steps using Auto mode.
The results are very informative, so it is worth to make an effort!
As you can see in the presented examples – the network really learns and improves its work in
such a way, that after some time (pretty short!), it quite effectively erases accidental disturbances
appearing in the original signal.
Filter effectiveness, which is generated as the result of using the network teaching process, can be
estimated by putting the picture of the signal before filtering on the process of the filtrated signal
(fig. 5.17).
Fig. 5.18. Results of filtration of a moved signal after one step of teaching
While experimenting with the program you will see that the network can really learn to filter the
signal and after some time performs this task quite well.
It is also easy to notice that the most difficult task is to teach the network to reproduce signal in a
place, where the values of the standard signal are small (particularly if they equal zero), therefore
in the Example 05 program it is possible to use two versions of network teaching – first with the
original sinusoid signal, and after that by sinusoid signal moved in such a way that the values
processed by the network could be only positive.
You choose the option with the window “choose case” (Choose case on the right side).
At the beginning, the results in the second case (definite in Choose case window as “with
displacement” – With shift) seem to be worse (see fig. 5.18), however persistent network
teaching, in this case, gives much better results than the mentioned before (fig. 5.19).
Fig. 5.19. Results of filtration of a moved signal after twenty steps of teaching
The observation of the network behaviour in both considered variants will help you in the future
to detect and analyse the causes of possible failures in the process of using the network to much
more complicated tasks.
I hope that the teaching process of the network filtering the signals was interesting for you and
helped you to understand how and in what way the network performs tasks during its work
improvement.
However, taking into account the entire attractiveness of tasks which were solved in this chapter
by us I have to confess that the linear networks are just ‘the kindergarten’ of the neural networks,
they are some kind of warm up.
These networks can only be single layer (if you want to know why – look at the end of chapter 3
of my book entitled Neural Networks, it is not possible to explain that without Maths, but I have
promised not to use it here), meanwhile the cerebral cortex is MULTILAYER...
So, the real adventure will really begin when you create and activate your first multilayer network
created from nonlinear neurons.
As you can guess yourself, it will happen quite soon – in the next chapter…