11.7. A few interesting examples
(Translation by Zbigniew Nagórny; nagorny@interia.pl )
I realize that the presented above description of the handling and functioning of the Example12b program could weary
you a little – therefore now the time for the award comes up. We will start to play together with this program you would
see what fine things it can do, and then you will be able by oneself to carry out further studies basing on practical hints
which I will present you now.
Here is shown the number
of the most similar pattern
(the risk of mistake for this
pattern is the greatest)
Here are pattern numbers
Here are shown measures of the
similarity of these patterns to the
currently entered character
Here are shown
miniatures of earlier
entered characters
Fig. 11.27. The way of information about the degree of the similarity of a new image to already remembered patterns
Let us analyse together the situation presented in Fig. 11.27. As seen in the upper screen part - I earlier already
entered patterns A, B, C and D. The currently entered pattern of the letter E turns out relatively weakly differentiated in
relation to the others. The measure of its similarity to the letter A pattern is equal to 22 (it is still a quite decent result)
but with the letter B, this pattern is related by the strength of 74 units and with the remaining patterns C and D, by the
strength of 56 units. At the right side of the window you will see (in the group box Most similar pattern(s)), how the
program warns that the entered pattern will be mistaken for the letter B pattern (no wonder...). Of course, it is your
decision but in my studies I would strongly reject such a pattern. It is possibly worth noting that taking the inversion,
i.e., the negative of this unlucky pattern, does not change the degree of its similarity to earlier considered patterns (Fig.
11.28).
Podrozdział 11.7, str. 1/5
Fig. 11.28. The inversion of the pattern shows exactly the same degree of similarity to earlier given patterns, as the
original pattern
Let us go back to technical details. This measure of the similarity of a new image to individual earlier defined
patterns, which is proposed by the program as the basic measure - is calculated in the program as so-called the scalar
product (Dot product, DP) of the vectors describing these signals in the input signal space. More precisely - the
absolute value of the scalar product is taken into consideration but you really need not reflect on that what it means. The
dot dispayed in the radio button named Dot product, (DP), which is seen in the centre of the screen, reminds of it. But
enough that you will click the second (located next to) radio button named Hamming distance (H) - the program will
switch to another way of the pattern similarity measure and it will calculate so-called the Hamming distance (Fig.
11.29).
Fig. 11.29. Signal similarity measures expressed by the Hamming distance
Thus, radio buttons: Dot product (DP) and Hamming distance (H) decide about that, which quantity - DP or
H – will be currently displayed under each pattern miniature.
Please notice, that in Fig. 11.29 values under individual patterns are different - now, this pattern which is the
most similar, shows the least distance to this image which you marked. But you know how to interpret this because the
black dot is currently displayed in the radio button Hamming distance (H) and not - as before, in the radio button Dot
product (DP) 1.
For what are these two possibilities in one program?
Well, depending of the circumstance, these two measures provide different information and can be useful for
two different objectives.
During the generation of new images there is better to rely on the observation (and the maximization) of the
scalar product. The Hamming distance could be misleading because it is sensitive to the use of the pattern inversion (see
Fig. 11.30 and compare it with Figs. 11.27., 11.28 and 11.29) what should not take place during the entering and
evaluation of new patterns.
1
The scalar product shows in what degree a new pattern repeats elements which were earlier present in accepted
patterns whereas the Hamming distance shows in how many points two images differ from each other.
Podrozdział 11.7, str. 2/5
Fig. 11.30. The value change of the Hamming distance with the character inversion
Whereas during the observation of the work of the network, when you will track its efforts tending towards
reminding of any pattern - the Hamming distance will be more useful. It will allow you to evaluate if the search process
is on a good way, if the network is approaching to or moving away from the correct pattern. It can be very exciting especially, when is possible to observe a very interesting phenomenon - of the escape of the network from the correct
pattern and “reminding” of a completely different information. See the example presented in Fig. 11.31.
In this box are shown miniatures of earlier entered
characters. Under each miniature is shown the
Hamming distance between a given pattern and the
noised image of ‘1’.
Here are shown pattern numbers
The noised pattern of ‘1’
Radio buttons for the
similarity measure choice
The possibility of comparing in boxes: Enlarged selected
input and Enlarged selected output – of miniatures (earlier
marked) of images and the similarity measure between them
(DP or H value between the two enlarged patterns)
(in this case – the Hamming distance is equal to 45).
The noised image of ‘1’ is the most
similar to the pattern number 1 (‘1’).
The Hamming distance between these
patterns is equal to 19.
Fig. 11.31. The process of the pattern search finished in failure, which is observed with the Hamming distance choice
This example corresponds to the situation of entering a disturbed stimulus into the network. In the original
signal number 1, 20 random changes were performed, therefore it became less similar to its own pattern and also
became very little similar to all the other patterns. This is confirmed by both the signal appearance and Hamming
distance values notified by the program.
From this moment, the dynamic process related to the working of feedbacks in the network starts. At first, the
network generates the image, which is dangerously approaching to the image 3, but afterwards it is clearly seen that the
network is “sucking in” by the attractor related to the pattern 9. Pay attention to numbers appearing under the pattern set
Podrozdział 11.7, str. 3/5
in both the box Input pattern(s) for teaching or recalling and the box Recalled entry pattern(s). They notify of
Hamming distances (and in Fig. 11.32 – of scalar product values) between the signal currently being in the network
output and all the considered patterns. This figure will help you to see which signals correspond to which distance sets.
Compare Hamming distance values between the noised pattern of ‘1’ and each pattern in both the box Input pattern(s)
for teaching orPrzyciski
recallingwyboru
and themiary
box Recalled entry pattern(s). Note that on the beginning, the Hamming distance
podobieństwa
between the correct version of the pattern number 1 and the signal shown in the network input is equal to 19. Other
patterns are safely far away - from 37 to 50 units. The program notifies of more distance units where the signal and
stimulus are little similar (0 and 1), and less, where a certain similarity, however, takes place, therefore everything is
sensible.
In this box are shown miniatures of earlier
entered characters. Under each miniature is
shown the scalar product value between a given
pattern and the noised image of ‘1’.
Here are shown pattern numbers
The noised pattern of ‘1’
Radio buttons for the
similarity measure choice
The possibility of comparing in boxes: Enlarged
selected input and Enlarged selected output – of
miniatures (earlier marked) of images and the
similarity measure between them (DP or H value
between the two enlarged patterns)
(in this case – the scalar product is equal to 6).
The noised image of ‘1’ is the most similar
to the pattern number 1 (‘1’).
The scalar product between these patterns
is equal to 58.
Fig. 11.32. The process of the pattern search with the choice of the scalar product measure, finished in failure
The experiment, which I described you, you can easily repeat on your computer. You can also obtain hundreds of
similar results with other patterns and other signals. It is worth to give oneself trouble over it because its execution, and
especially the result consideration, gives basis to very interesting conclusions. Perhaps just on this principle, new ideas
and unexpected associations arise in your brain too?
Experiments performed with the Hopfield network will show you that one task is not equal to another one.
Some patterns can be easily reconstructed, without any bigger problems, and others persistently spoil (see Fig. 11.33).
Therefore, in the next subsection I will tell you about such patterns, which can be well reconstructed (although perhaps
the advantage from it will be not very large).
Podrozdział 11.7, str. 4/5
Fig. 11.33. The success of the reconstruction of the letter C pattern and two forms of the reconstruction failure of the
letter A pattern with the neural network
Podrozdział 11.7, str. 5/5