EEL 6876
Intelligent Diagnostics
Abduction using Neural Models
By Madan Bharadwaj
Instructor:
Dr. Avelino Gonzalez
1/21
Abdu ction u s ing Ne ura l Mod e ls
ABSTRACT
Addressing abductive reasoning effectively remains a formidable challenge to researchers world
over. Most of the work on Abduction has been done using a logical or a probabilistic approach.
In this paper, I review a few papers that discuss abduction in a Neural framework. The networks
proposed cover almost the complete range of abduction problems and provide elegant solutions,
which can be interpreted as similar to abduction performed by human beings.
PROLOGUE
With the rising prominence of Abduction as a vital form of reasoning in AI many researchers
have attempted to address abductive reasoning in a logical or a probabilistic framework. Some of
them have however pondered the use of Neural or connectionist models to deal with abductive
reasoning. Some of the first works using this approach dates back to late 80’s [3] when
‘connectionist’ ideas were floated and followed up with more developed neural models [4] to
address abduction. In the following sections we will see some of the more recent ideas which
have evolved and have been experimentally proved to provide a framework to use abduction for
real world problems.
It is rather surprising that abduction got recognized as a prime form of human reasoning only
recently when most of what we know can be thought of as abduced knowledge from real life
observations. To put that in an informal definition, Abduction is the process of evolving a
2/21
hypothesis from a set of observations. An elementary explanation of abduction can be found in
[5].
On the other hand Neural Networks are a class of algorithms that are taught to differentiate
between similar and dissimilar data using a training data set and then are used in the real world to
do the same on real data. Though one cannot immediately make a connection between the two,
they can be interpreted to be doing the same thing.
ABDUCTION & NEURAL NETWORKS: AN ANALOGY
Let us consider an example to illustrate the discussion floated in the previous section. A very
easily understood and widely used example for demonstrating the application of Neural
Networks is the ‘Character Recognition’ task. A neural network is popularly used to recognize
handwritten characters so that it can be fed automatically without having to perform manual
interpretation.
For example if we want our network to recognize A’s and B’s of the English alphabet, we train
the network with a collection of handwritten A’s and B’s and ‘teach’ it to recognize the two of
them as two different classes of data. Figure 1 shows a sample training set of 5 A’s and 5 B’s
(say collected from your rough book) that can be used to train a neural network for this task.
Once trained with the training data they will create classes in an implicit manner to represent A’s
and B’s. Now, they can be tested on real data from a real data source (an ‘A’ picked out from
your handwritten class notes) to elicit a response from it. The Neural Network has the inherent
3/21
capability to generalize from specific instances and hence will be able to recognize your ‘A’
once it sees it.
Figure 1: Handwritten Characters. A’s and B’s
Figure 2: After training the Neural Network classifies data into classes
Now returning to our original discussion, abduction can be seen as generalizing from a set of
observations to synthesize a hypothesis in order to explain the observations. In reality, we have a
collection of hypothesis and our abduction algorithm will be expected to pick one among them,
the one which best explains the observed data as it understands it. The observations can be
intuitively linked to the handwritten A’s and B’s and the collection of hypothesis can be
connected to the classes that the neural network creates to partition the data.
4/21
Through this simple explanation we can make an association in our minds between the properties
of abduction and the functioning of neural networks. With this introduction let’s probe into the
more concrete ideas that discuss implementing the concept.
There are two basic approaches to designing a neural network architecture.
1. Designing neural network architecture to reflect the problem dynamics
2. Designing an Energy function to represent the same
The first paper [1] we are going to discuss follows the first approach and portrays a simple
architecture that covers most domains of Abduction providing an elegant solution to our
problem. The second paper [2] proposes the latter approach and attempts to minimize an error
function to attain the same end.
“A UNIFIED MODEL FOR ABDUCTION-BASED REASONING” by B.AYEB, S.WANG AND J.GE
They call the network UNIFY and true to its name it is a unified model for all classes of
abduction problems as opposed to the other model which will only address linear and monotonic
abduction problems. The network is a simple 3-layered architecture built specifically to
implement abduction. The network hence bears a unique physical structure unlike most other
neural network models used for more multifarious purposes. The model works on the basis of
competition between cells (or nodes as it is more commonly addressed in neural network
literature) to encode the observations and maximize fitting hypotheses. The algorithm is
discussed incrementally, introducing the simple version first for the sake of simplicity and better
understanding and then introducing minor modifications to accommodate for other classes of
5/21
abduction problems. The modifications made are general and do not affect the class of abduction
treated before the modification.
ABDUCTION CLASSES
For any further discussion we need to understand the different types of abduction problems and
what they mean, to understand its ramifications on the network architecture. To define them
rather briefly, four major types of abduction problems are encountered in the literature. They are
1. Independent Abduction Problems
2. Monotonic Abduction Problems
3. Open Abduction Problems
4. Incompatible Abduction Problems
An abduction problem is defined as AP (H, O, R), where H denotes an elementary set of
Hypothesis, O denoted the set of observable facts and R denotes a mapping from H to O.
If each causal relationship can be expressed in the form h  o and where h  H & o  O . If one
and only one elementary hypothesis is needed to explain one observation, then the problem is
Independent.
If there is at least one causal relationship which can be expresses as  h  H  o meaning, o
needs to be derived from more than one hypothesis as a conjunction of two hypothesis, then the
problem is Monotonic.
6/21
For an observation uplet1 (OP, OA, OU), if we have OA  OU then the problem is Open. If
there is at least one instance from the observation set that is observed to be absent and one
instance that is explicitly stated as unknown then the problem is Open.
For an observation uplet1 (OP, OA, OU), if we have atleast one causal relationship which is
expressed as  h  H  , and  OA then the problem is Incompatible. If there is at least one
observation is observed to be absent but can be derived from the cardinal set of elementary
hypothesis which are synthesized from the other observations, then it belongs to the
Incompatible class of abduction problems.
The explanations provided above are inadequate to provide the clearest picture one can provide
towards the explanation of abduction classes. I urge the reader to refer pages 2-3 of [1] where it
is more clearly and elaborately explained with an example.
7/21
INITIAL NEURAL MODEL
The paper describes a simplified initial model to guide the reader through the process of building
the comprehensive network from its smaller and simpler blocks. A pictorial representation of the
idea is depicted in Figure 3.
Hypothesis Layer
Observation Layer
Inhibitory Weights
Excitatory Weights
Figure 3: The Initial Mode. Reprinted from [1]
The network comprises of two layers, one consisting of all the observations and the other
consisting of all the hypothesis. The two are connected by weights as described in the mapping
provided initially. Figure 4 illustrates the weights as described in the mapping rules
.
Figure 4: An illustration to show weights in axioms
8/21
The weights are intuitively the belief that we have over the presence of the hypothesis given the
observation. By altering the weights we can alter our belief or disbelief in our hypotheses. There
are two types of weights that are introduced in the architecture: Excitatory weights and Inhibitory
weights. The excitatory weights are present in between a hypothesis and an observation, whereas
an inhibitory weight can be seen in between two hypothesis. They serve to discredit opposing
hypothesis intending to weaken it and shove it off the competition process.
WORKING
The network is initialized to its startup setting where the values of the observation cells are
initialized to values based on the belief of the observation: a 0.5 for an observation cell would
mean a 50% belief that the observation was observed. The excitatory weights are initialized to
the values as given by the mapping rules and those that are not mentioned are set to zero. By this
setting the observation will not affect hypothesis that are not related to it. The initialization
inhibitory weights are not discussed and going by common sense they must be set to zero. The
hypothesis taken a initial value dictated by the equation…
where
and A is a small value, say 0.001.
9/21
After
initializing
the
network,
the
competition between the hypothesis begin.
For every iteration Stage II of the algorithm
described in Figure 5 is executed until the
termination condition is met or in neural
network terminology, ‘until the network
stabilizes’.
When the termination condition is met, one
can expect one or more hypothesis to have
won the competition and hence would
comprise the final set of elementary
hypothesis that can explain the observation
set fully.
Figure 5: Step by Step explanation of the algorithm. [1]
THE EQUATIONS
The unique structure of the network can be explained or its usefulness realized only when we
analyze the equations used to govern the dynamics of the network. Let us review the equations
used for this purpose.
There are four major equations that govern the activities of the network.
10/21
The hypothesis cell updating equation given by
,
Where EXi represents the Excitatory input to the hypothesis and the IHi represents the inhibitory
input the hypothesis cell. Note that the excitatory input is on top of the equation whereas the
inhibitory input is in the bottom. Hence more excitation will improve the value of the hypothesis
cell and more inhibition will lessen the value of the cell, which will translate into better or poorer
chances for the hypothesis to win the competition with the other hypothesis.
The Excitatory weight updating equation given by
Note that the weight is updated depending on the cell values of the concerned hypothesis and
observation and the present weight it holds.
The inhibitory weight equation given by
This equation is interesting since it calculates the inhibitory weight based on the differences
between the previous and present cell values of the two hypothesis cells that it connects. If one
value is negative and one is positive the weight will increase and hence weaken the other
11/21
hypothesis hence effectively implementing the hypothesis negation implemented in a logical
framework.
To summarize the working of the network one can say that the network structure ensures real
value inputs to hypothesis and observations and hence can portrays real life situations more
effectively than other approaches. The competition process may also be intuitively understood as
similar to the one that happens in our minds, where we weigh different theories to explain an
observation.
EXPERIMENTAL RESULTS
The authors have picked out a neat example to illustrate the concept. They have picked a simple
observation set and a mapping rule set to initialize the network and have come up with answers
that can be verified easily within a logical framework.
Figure 6: A graph showing the hypotheses cell values at various instances. Reprinted from [1]
Figure 6 shows a graph plotting the values of the different competing hypothesis at different time
intervals. One can immediately see how some hypothesis cells decay to zero and two others
12/21
survive till termination. Actually, one hypothesis comes out clearly as a winner and another
stabilizes to a value considerably greater than zero. The results can show how the process
debunks false or incomplete hypothesis and exalts correct or appropriate hypothesis. The other
immediate consequence is also that it gives real value outputs which can be construed as beliefs
in the resulting hypothesis. This would also be very much representative of the resuts one can
expect from real world abduction problems.
UNIFIED MODEL
In the last section the initial model we introduced provides the basis for the unified model that
we are going to present in this section. The reason for doing so is that the initial model cannot
explain the more complicated monotonic, open and incompatible abduction problems. The initial
model cannot resolve conjunction of hypothesis and hence we are introducing the unified model
that will address all problems including simple ones effectively.
The unified model consists of three layers in place of two of the initial model to help conjunction
of hypothesis. If an observation has to be explained by a conjunction of hypothesis only then the
intermediate layer comes into play. A node on the intermediate layer connected to the respective
hypothesis will act as a middle man to channel flow of information between the hypotheses and
the observation. Using slightly modified equations the unified model addresses the problem
effectively and elegantly. The nodes on the intermediate layer act like a composite jumbo
hypothesis representing the conjunction of all the associated hypotheses. This way the model can
be addressed like the initial model in basic framework with changes to accommodate the
intermediate layer implemented in the equations.
13/21
Figure 7: The Unified Model. Reprinted from [1]
Apart from this structural addition, there is one more major appendage implemented to ensure
maximum problem domain coverage. To cover the class of Incompatible abduction problems the
hypothesis cells that constitute incompatibility are connected by two connections. They act very
much similar to the inhibitory connections and in a way just add to the inhibition of these
hypotheses to avoid incompatibility in the final solution set.
Let us have a look at the equations that cover the network now. The hypothesis cell equations is
now given by
&
14/21
The ICi represents the connection weights of the Incompatibility connections. Since they inhibit
the hypothesis they are represented in the denominator. Negative influences of absent
observations are realized by using EX+ and EX- notations where, EX- represents the observations
are observed to be absent. The cell values of the observations cells for these observations will be
negative, hence inhibiting the hypothesis.
The incompatibility weights are updated according to the equation
where g1(x)=1 if x>0 or else 0 and g2(x) = 1 if x is a Exclusive hypothesis or in other words the
only hypothesis to observe a particular observation. The major benefit of the equation is it
eschews discriminating against exclusive hypothesis but inhibits all other hypothesis that are
connected, hence providing a way out of the incompatibility condition.
EXPERIMENTAL RESULTS
The algorithm was tested on various toy problems and graphs have been plotted to give the
reader an intuitive understanding of the results. Further the algorithm is tested on a problem
constructed from a real world murder case that went to trial by axiomizing the evidences and
15/21
constructing the possible outcomes of the case as candidate hypotheses. The example is very
interesting and shows that the network in very good light.
HOPFIELD AND ART NEURAL MODELS FOR ABDUCTION
The authors Goel and Ramanujam put forward a more conventional approach in [2] that has been
time tested and has well known properties. They propose two approaches, one using the Hopfield
model and other using ART2. They have also clearly stated the limitations of their study and
constraints using their approach.
The authors seem to prefer the use of the Energy function approach to the design of a neural
network, with analog internal operating parts. This they claim has inherent advantages over
digital circuitry. The idea is to develop a network that will minimize an Energy function and find
the minima of the function and hence finding the best hypothesis. To get to this end they have
performed some operations on the data set partitioning them into domains and using subnets of
the proposed neural architecture for each domain.
The paper starts by saying that the algorithm is being implemented for linear and monotonic
abduction problems. The first step of the process is to formulate the abduction task as an
constrained optimization problem. This they achieve by representing the observations and
hypotheses in data sets with certain easily deduced relationships between them. Then they
partition the problem into as many sets as possible, or in other words create a Power Set 3 for the
observation set and the hypotheses set and create a map between the two.
16/21
They then define the characteristics based on which the classifier will choose the hypotheses as
the best among its compatriots. They broadly define the characteristics as 1. Maximal Coverage
2. Maximum Belief and 3. Minimal Cardinality. They then take into account the possible
connection or associations that can be allowed between elementary hypotheses.
With this framework in hand they now approach the problem with a Hopfield neural model with
neurons having to minimize an energy function given by
where the W’s are the connection weights and the Vi and Vj are the outputs of the neurons. By
getting to minimize this function the network will determine choosing the best hypotheses.
The authors have also proposed an ART model for the same purpose, where they split the
domain into observations domain and hypotheses domain . By making use of local decisions
taken by the observation neurons and the hypotheses neurons and the information exchange
between the two domains a composite hypotheses set is synthesized.
I have made only a brief review of these models because they do not address all classes of
abduction problem and hence their significance is greatly truncated.
CRITIQUE: NEURAL MODELS FOR ABDUCTION
From this essentially incomplete review of literature and my other readings, I can see more and
more study is being done on developing models for abductive reasoning. Non-Monotonic
reasoning and reasoning under uncertainty are active research areas now. In my understanding
17/21
the framework for this kind of reasoning has to be in essence fuzzy. Since a large part of real
world explanations in the human reasoning process are not purely axiomized; where evidence for
a particular occurrence is not always 100% true but can be expressed with partial confidence and
since results of a hypotheses are evidence for some other hypotheses in real life, it is very much
necessary to maintain a fuzzy nature in the framework under which we can discuss abduction.
This property is inherently present in Neural Networks and hence can prove to be an effective
framework for abductive reasoning.
The downside of the approach is that the functioning of a neural network is still very much
abstract and has not been completely understood by researchers. This aspect promotes the
tendency not to invest heavily on techniques that can backfire at a later time and would force
researchers to redo and reprove all their work using a different approach. Hence the real credit
that Neural Networks may deserve is lost due to cynicism.
However recent studies like deriving a rule base from a neural network that explains the
functioning of a neural network [7] and other similar studies can help researchers shoo the fears
and embrace neural networks as a test bed for their research.
FUTURE AVENUES OF RESEARCH
The UNIFY paper [1] concedes that it still cannot solve the problems mooted by the cancellation
class of abduction problems discussed in [8]. The authors have promised more research on that
aspect and develop or modify their UNIFY to solve all the classes of Abduction problems.
18/21
Other Neural frameworks can be considered for the purpose. I would think ART provide a far
more elegant and flexible solution to the purpose than described in [2]. Studying that avenue of
thought would be an exciting experience. Genetic algorithms can also be touched upon, to see if
they can provide any basis for the problems.
SUMMARY
This paper reviewed a few interesting neural models proposed to solve abduction problems. The
UNIFY algorithm put forward by Ayeb et al stands out for their simplicity, applicability and
domain coverage. Other approaches using Hopfield networks and ART were also briefly
discussed. The concluding sections critiqued the idea as a whole, and provides some suggestions
and explanations behind the philosophy of using neural networks for abduction.
19/21
NOTATIONS
1
- An observation uplet (OP, OA,) is defined primarily for Multiple Observations which are
applicable only for the Open and Incompatible class of abduction problems. Here OP represents
the observations that are known to be present, OA represents the observations that is known to be
absent and OU represents the observations that are unknown.
2
- ART- Adaptive Resonance Theory
3
– A Power Set is a set containing all possible subsets of a set
REFERENCES:
[1]. B.Ayeb, S.Wang and J.Ge, “A Unified Model for Abduction-Based Reasoning” IEEE
Transaction on Systems, Man and Cybernetics – Part A: Systems and Humans, Vol 28,
No. 4, July 1998
[2]. A.K. Goel and J. Ramanujam, “A Neural Architecture for a Class of Abduction
Problems”, IEEE Transaction on Systems, Man and Cybernetics – Part B –
Cybernetics, Vol. 26, No. 6, December 1996
[3]. _____, “A Connectionist Model for Diagnostic Problem Solving: Part II”, IEEE
Transaction on Systems, Man and Cybernetics., Vol19, pp. 285-289, 1989
[4]. A. Goel, J. Ramanujam and P. Sadayappan, “Towards a ‘neural’ architecture of
abductive reasoning”, in Proc. 2nd Int. Conf. Neural Networks, 1988, pp. I-681-I-688.
[5]. D.Poole, A. Mackworth and R.Goebel, “Computational Intelligence: A Logical
Approach”, pp 319-343, Oxford University Press, 1998.
[6]. C. Christodoulou and M. Georgiopoulos, “Applications of Neural Networks in
Electromagnetics”, Boston: Artech House, 2001.
20/21
[7]. Castro, J.L.; Mantas, C.J.; Benitez, J.M., “Interpretation of artificial neural networks by
means of fuzzy rules”, IEEE Transactions on Neural Networks, Volume: 13 Issue: 1,
Jan. 2002. Page(s): 101 –116
[8]. T. Bylander, D. Allemang, M. C. Tanner, and J. R. Josephon, “The computational
complexity of abduction,” Artif. Intell., vol. 49, pp. 25–60, 1991.
21/21