THE USE OF ARTIFICIAL NEURAL
NETWORKS FOR DATA ANALYSIS
A.V. Gavrilov, V.M. Kangler
Novosibirsk state technical university, Novosibirsk, Russia, Email: avg@xeopsa.cs.nstu.ru
Abstract.
The principles of application of Hopfield neural network for Data Analysis is proposed in this
report. Here the artificial neural networks are used for realization of associative memory representing
information contents of a relational database (DB). This approach is used for solve the task of detection of
associative interrelations between of fields of Data Base and forecasting of values of its.
Introduction.
Critical competitive struggle, aspiration of the companies to receive profit from the investments in
information technologies, growth of number of the employees accepting the decisions, stimulated active
development of new area of computer science - detection of knowledge in databases (KDD - Knowledge
Discovery in Databases). Its basic assignment is automated search earlier unknown laws in databases
storing the information (for example, about activity of the companies), and use of the extracted knowledge
during acceptance of the decisions. With the help of methods KDD it is possible to reveal, for example,
structure of the consumers of the given goods to prevent frauds with credit cards or to predict change of a
situation in the financial market. The basic task of systems KDD is the realization of the analysis of the
data contained in databases, with the purpose of detection of the latent or unobvious laws. Allocate five
standard types of laws, which allow to reveal automatic intellectual methods of the analysis and
representation of the data: association, sequence, classification, clustering and forecasting. The means of
detection of knowledge in databases basically use five methods: an induction of associative rules; trees of
the decisions; K - nearest neighbours; neural networks; genetic algorithms. Their combination is
sometimes applied. The report is devoted to the decision of two tasks KDD, detection of associations and
forecasting, by artificial neural networks (ANNs).
The analysis of Data Base by Hopfield neural network.
Data Analysis in the given work is understood as a complex of the measures directed on
revealing of latent laws in databases. Use of the artificial neural networks for the decision of a task is
considered which within the framework of tasks of the automated intellectual analysis of the data can be
treated as follows.
1. The qualitative solving of a task of find of values of fields of DB, associated to other fields with
known values.
2. The solving of a task of forecasting of values of a set of fields of DB with the given values of other
set of fields.
Having system similar to associative memory, and also formal rules of carry of information
contents of an analyzed database in this memory, it is possible to solve this task. Thus is used the
association nature of such system and on values of some factor as "key" are taken is associative close
values of other factors. One of possible ways to realize of such associative memory is to construct the
distributed dynamic system or network of discrete elements with attractors as the typical patterns-images,
contained in DB [1]. Each such pattern will have the area of an attraction, and any entry condition
representing any allowable pattorn, is obliged to get in one of its areas of an attraction. With current of
time during evolution this initial structure is transformed to the closest of structures - attractors, stored in
memory, namely in that, it belonged to area of which attraction. Hence, submitting on an input as the
entry condition for such distributed system some structure, we shall carry out its automatic recognition,
which will be parallel. In a role of such distributed dynamic system it is offered to use artificial neural
network.
The decision, offered us, in a general view looks as follows. Artificial neural network is trained on
records of an analyzed relational database. During training neural network becomes gnoseological model
of the DB. The received thus model the user can use for forecasting and research of associative
connections latent in a database. As such model it is offered to use a network representing particular
realization of dynamic system with associative memory, proposed by Hopfield [2-4].
The primary preparation given for the decision of the put task consists in representation of set of
the factors as the contents of a relational database. The structural relations are thus set and the task is
reduced to definition of semantic affinity of contents of fields of the given structure. As binary artificial
neural network operate with binary vectors, the additional processing initial given is required with the
purpose of reduction to a kind represented by such vector.
The vector needed for artificial neural network turns out the concatination of binary subvectors
representing the fields of a relational database. Each field of a relational database can be referred to one
of two types: numerical or symbolical. The value of a symbolical field is brought in the dictionary,
appropriate each field. To raise efficiency of procedures of search of value in the dictionary and their
coding, contents of the dictionaries are ranging, the repeating values are excluded. The value of
numerical fields are represented differently. A range of value of each of numerical fields (to numerical
fields those concern also, which contain dates) is broken into intervals, and each single value is
represented by an interval, by which it belongs. The point on a numerical axis can be presented by a
degeneration interval. The replacement of numerical value with intervals provides large flexibility of
representation. Analogue of the dictionaries for numerical fields is the list of intervals. After the
dictionaries and sets of intervals were generated, they are exposed to coding. That is each value in the
dictionary or interval in a set the binary code representing subsequently this value or an interval in neural
network is put in conformity. The dimension of a code (length of a code) for each dictionary and set (in
other words for each field) is defined by capacity of set of various values in the dictionary or set, and also
way of coding. The various variants of coding can essentially have an effect for quality of work of
associative memory submitted by Hopfield neural network. The examples to that can be served by
experimental data received as a result of model modeling during preparation of the given work. The
formation of the dictionaries and their coding is a preparatory grade level of neural network. Training is
carried out according to a Hebb rule [5].
Each record of a database is represented as a binary vector and participates in formation of
weight factors of connections between formal neurons of a network. It is desirable to remove from training
repeating records of a database. As the repeated record of a vector in associative memory submitted by
neural network adverse has an effect for a power landscape of this network. Namely, the depth of a
power minimum appropriate to such vector is doubled. The deeper minimum usually has wider area of an
attraction, and it reduces areas of an attraction of the next minima and deforms the appropriate drawing
vectors. In a limiting case it can result that in system there will be only one very deep minimum of energy,
the area of which attraction grasps all possible spin configurations. At system the persuasive idea " is
formed as though ", that, certainly, it is extremely undesirable.
There are various versions of Hopfield networks with similar structure, but some differences are
in operation of them. Here are considered only binary condition neuron: + 1 and -1) Hopfield networks
with discrete functioning in time. Specifity uniting such network is asynchronous of operation of them.
That is as against synchronous functioning, with which the condition all neurons of a network is defined
simultaneously, the asynchronous functioning means in each particular moment of time an opportunity of
switching only one neuron. For modeling we deal with four algorithms of operation of Hopfield's neural
network:
1.
2.
3.
4.
The determined algorithm;
Stochastic algorithm;
The determined algorithm with annealing imitation;
Stochastic algorithm with annealing imitation.
The process of operation of neural network is not depended from mentioning algorithms is
multiiterative. Each of iterations includes two steps: a choice of the neuron-candidate, formation of a
condition of the chosen neuron-candidate. The difference of stochastic functioning from determined
consists in a method of a choice of the neuron-candidate. With stochastic - the candidate for switching is
neuron chosen casually with the help of the random-number generator. Thus, such situations are
possible, that during operation the condition of any neurons may be never analyzed. With the determined
operation neurons become the candidates for change of the condition by way of following numbers. In
algorithms with annealing imitation the change of a condition of the neuron-candidate carries probabilistic
character. If as a result of change of a condition on opposite the complete energy of a network will be
lowered, the condition of neuron varies on opposite. Differently neuron's condition varies on opposite with
the certain probability dependent on parameter, named in "temperature" of a network simulating a level of
thermal noise. On the first iteration of operation of the neural network the initial value of temperature
(maximal value) is established, then gradually through some of iterations the value of temperature
decreases. The iterative process stops, when temperature reaches final value (minimal value). If to
compare among themselves algorithms without and with annealing imitation, at the first set of advantages
before second (it is easier, require smaller amount of iterations, have greater accuracy - if the power
landscape near to a global minimum is not cut up by local minima), but if the power landscape contains
set of local minima, the algorithms with annealing imitation behave better (from the point of view of
criterion of quality of functioning). The confirmation of last statement can be served by the data of model
modeling. If to consider operation of system as a whole, it includes the following stages:
1.
2.
3.
4.
the formation of inquiry,
the preparation of an input vector of a network,
the cycles of functioning of the neural network,
the formation of a target vector.
Let's consider each stage.
The Formation of inquiry. At this stage the user chooses a part of fields DB and gives them some
values (using the dictionaries, appropriate to this fields, of values and sets of intervals). During functioning
the system will predict values of other fields (distinct from set by the user) similarly to associative
memory.
The preparation of an input vector of a neural network. At this stage the input vector is formed for
the neural network. It "is going" from subvectors, appropriate to the fields of Data Base. With formation of
a vector the zero categories subvectors are replaced on (-1). Those subvectors, whose value sets the
user, are marked as "frozen". The values of other subvectors are formed with use of the random-number
generator.
The cycles of operation of a neural network. This stage includes a number of cycles of operation
of a neural network. Before each cycle on the neural network moves an input vector generated at the
previous stage of operation. Each cycle of operation of neural network consists of repeated iterations,
thus each iteration in turn includes two above described stages (choice of the neuron-candidate,
formation of a condition of the chosen neuron-candidate). It is necessary to notice neurons that
appropriate "frozen" subvectors, cannot be chosen as the neuron-candidate, and, hence, they do not
change the condition during all functioning. On end of the first cycle received by a neural network a vector
and the value of energy of a network is remembered in the special buffer. At the end of each subsequent
cycle the value of energy of a network is compared with value stored in buffer. If the current value of
energy is less then in the buffer, a current condition of a network (the condition of a network represents a
vector in quality which component the condition neurons act) replaces in the buffer earlier stored vector.
The formation of a target vector. It is a formal stage, when a vector kept in the buffer is given out
by system as the vector - answer restored by associative memory by a fragment, given by the user.
Conclusions
To the basic advantages of the approach, proposed here, it is possible to relate the following.





Organic association of the decisions of such tasks as forecasting and detection of associations
within the framework of the uniform approach.
The training of neural network with the purpose of the subsequent modeling requires only two
continuous passes on a simulated database. While with the traditional approach, namely with
accumulation statistics about combinations of values contained in DB, the multiiterative complex
procedures of search are required.
The decision of a task in those situations, when the traditional approach appears statistically
insolvent. For example, when analyzed DB contains small number of similar records.
Universality of Hopfield network, used as a basis of construction of associative memory.
Simplicity of program realization of Hopfield model.
This approach was realized in the program of Data Base Analysis (AnalDB). There the SQLquery is the source of data for learning of the neural network in this program. This program is realized on
Delphi 3.0 and Cbuilder 3.0. It may be used for find of most probable values of any fields in context
determined by other fields, and for forecasting of fields of also unexisting record (if the records are the
patterns of process in time). This program is used for research of the charge of water in river Ob for last
104 years with the purpose of forecasting inflow of water and research of a demographic situation in one
district of Novosibirsk. This work is supported by the program "Universities of Russia - fundamental
researches" and grant of Russian Federation on a direction "Computer Science and cybernetics".
References
[1] Haken H. Synergetic computers for pattern recognition and associative memory. In Computational
Systems - Natural and Artificial. Ed. H. Haken. Springer - Verlag, Berlin, 1987.
[2] Hopfield J. J. Neural networks and physical systems with emergent collective computational abilities //
Proc. Nat. Acad. Sci. USA - 1982.
[3] Ackley D. H., Hinton G. E., Sejnowski T. J. A learning algorithm for Boltzmann machines // Cognit Sci.
- 1985.
[4] Vedenov A. A., Ezhov A. A., Kamchatnow A. M. A study of Hopfield model of associative memory.
Institute of atomic energy. - M.: IAE-4262/1. - 1986.
[5] Hebb D. O. The Organization of Behavior. - New York: Wiley, 1949