From: AAAI Technical Report WS-94-01. Compilation copyright © 1994, AAAI (www.aaai.org). All rights reserved.
A Comparison of Model-Based and Incremental Case-Based
Approaches to Electronic Fault Diagnosis
Barry
Smyth
Hitachi Dublin Laboratory
16 Westland Row
Dublin 2
Ireland
barry@hdl.ie
P~idraig
Cunningham
Department of ComputerScience
Trinity College Dublin
College Green
Dublin 2
Ireland
Padraig.Cunningham@
cs.tcd.ie
is neededduring the diagnostic process. It is interesting to
note that backward-chaining systems have precisely this
Abstract
advantage in fault-diagnosis; the goal-directed reasoning
CBRseems well suited to fault diagnosis because
asks the user only for information that contributes to the
diagnostic episodes naturally form cases and muchof
hypothesis being examined.
expert competence seems to be based on reuse of old
In this paper we consider the problem of using CBRin
solutions. However,in manydiagnosis problems it is
electronic
fault diagnosis and discuss the re-engineering of
difficult to compile a complete case description in
an existing model-based,goal-directed diagnosis systemas a
advance, consequently the conventional one-shot case
case-based system. Fault diagnosis seems a good candidate
retrieval methodologywill not work. In this paper we
for CBRbecause it is clear that muchof humanexpertise in
introducea set of fault diagnosis problemsthat have this
characteristic and we describe a model-basedgoal-driven
fault diagnosis is experiencebased. Further, fault diagnosis
systemthat producesfocusedquestions that request extra
seems naturally case-based with each diagnostic episode
information required for diagnosis. The central
constituting a case. The problem that we have encountered
contributionin this paper is a description of a CBR
system
is centred on the cost of gatheringa useful case description.
that also has this characteristic of producing focused
In our problem domain there can be up to a hundred
questions in diagnosis. Wedescribe the information
symptomsthat potentially impact on the diagnosis and
theoretic mechanismthat allows the CBRsystem to do
there is a cost associated with gathering most of these. For
this and we present an evaluation of the CBRsystemand a
this reason the naive solution of collecting these readings to
comparisonof the two systems.
makea case is not practicable. This contrasts sharply with
the solution implemented in the existing model-based
Introduction
system (called NODAL)(Cunningham & Brady 1987),
(Cunningham 1988). This system uses goal-directed
The major attraction of CBR
is its cognitive plausibility. It
is clear that muchof humanexpert competenceis based on reasoning in the diagnosis and this has the advantage of
only querying the user for symptoms
that will contribute to
the reuse of past solutions in solving new problems.
the
diagnosis.
This
parsimony
dividend
of backwardHowever, one restriction
on the dominant CBR
chaining
systems
has
been
recognised
since
the
early days
methodologyis that it tends to be one-shot, requiring that
of MYCINand, in this paper, we will examine
the target case description be available in advanceand that
modifications to the CBRidea that can operate with the
the problem can be solved with a small number of
retrievals. This methodologycan be problematic for some same, or even less, information.
First we shall introduce the diagnostic task, describe the
diagnosis problems and in this paper we describe one such
existing model-based system and explain how the goalsituation. The problemin the situation that we describe is
directed reasoning dictates the number of symptoms
that all informationis not available in advanceand there is
requested fromthe user. This is covered in the next section.
a cost associated with getting information. Consequently,it
Next we examineother case-based diagnosis and explanation
is important that the amountof information requested is
systems and consider howthey tackle or avoid our problem.
minimised. What is needed is a CBRmethodologythat is
1,
incremental one that can indicate what extra information In the final section we describe our approachto incremental
CBRfor this particular diagnosis problem and compare it
with the existing model-based system (MBS).
1 Webelieve that there are two important waysthat a CBR
systemcan be incremental. CBRcan be incremental in that it
Goal-driven
diagnosis
builds its solutions using components
taken from several cases
(Smyth & Cunningham1992), (Redmond1990). Alternatively
It has long been recognised that goal-directed reasoning, or
it can be incrementalin the sense we mehere; that is that the backward rule-chaining,
has specific advantages in
target specification is composed
during the CBRprocess.
48
diagnosis. In circumstanceswherethere is a cost associated designed to detect these. Fault diagnosis of these SMPS
with determining the various symptomsassociated with the involves locating the faulty moduleand finding the faulty
fault or illness the goal-driven reasoning can focus the user componentin that module.
interaction and only request input that contributes to a
diagnosis. This advantage is not restricted to simple ruleFramefor Ol
based systems; model-basedsystems can adopt a goal-driven
reasoning strategy and enjoy the same advantage. NODAL
is such a systemfor electronic fault diagnosis.
$la(uo:
Fault diagnosis
supplies
in switching
NODAL
is a model-based system for fault diagnosis of
switching mode power supplies (SMPS). The system
implemented in KEEa hybrid expert systems development
environment. The original motivation for its development
was to produce a genetic diagnostic system for a class of
electrical
devices. NODAL
has a generic reasoning
mechanismand can be set up to work for a particular power
supply by encoding the model of that power-supply in the
system. The block diagram of one of the power-supplies
used in testing the system is shownin Figure 1.
/-- I s";l~
’
eoyr~
’
I~
O1
~
Mode:
Base:
Emitter:
Coll4mtor:
Module:
o±
mode power
!
- i I ~P"nffil~
Figure 1. The block diagramof the 24V/12V
power-supply.
The models in NODAL
have a hierarchical structure. The
top-level represents the blocks in the block diagram as
frames, the main information on the frames being
interconneetion information and also some information
about the characteristics of the blocks. These blocks are
interconnected by nodes and these nodes themselves are
represented as frames. These node frames carry information
used during the diagnosis. For more complex power°
supplies these blocks were further divided into sub-modules.
The detailed level of representation corresponds to
detailed information available in schematics of the SMPS
circuit. An exampleof the detail of the Local PowerSupply
moduleof the 24W12V
unit is shownin Figure 2 (a). The
components are represented as frames that carry
interconnectioninformationand details of the characteristics
of the components. Again, the interconnecting nodes are
also represented as frames. The frame for the Q1 transistor
is shownin Figure 2 (b).
NODAL
was designed for use in a repair shop so the
assumptionthat the circuit under examinationhas workedat
some stage reduces the numberof fault categories to be
considered. Componentfailure accounts for over 95%of
faults on SMPSthat have failed in operation so NODAL
is
49
l-
(a)
Unur
LPS-I
NODE-3
NOOE-2
LocoJPower
Supply
(b)
Figure 2. Thedetail of the LocalPowerSupplymodulein the
24V/12circuit and the framefor the QI transistor in that
circuit.
Since NODAL
is designed to operate in a repair shop the
first input in the diagnosis is the results from the test
equipment on which it was confirmed that the unit was
faulty. This input is shownin Figure 3. These function
tests are performedon the unit as a "olack box’, and measure
outputs associated with test inputs. These tests will number
between twenty and forty depending on the complexity of
the circuit. However,because the internals of the unit are
not.being examined, the amountof diagnostic information
that they carry is limited. Thetest results are processed by
the Function Test Rules (a shallow reasoning componentin
NODAL)
and a set of candidate faulty modulesis produced.
Voltages &
Function Signals Measured
Resistances
Test Restdts
O..a.o.
-
[ Rules I ~
P’I~ Diagnosis
,ol ID"
"
] ~_
Diagnosis
o
Figure 3. Datainput in NODAL
and the related reasoning
processes.
In order to further isolate the fault it is necessary to
perform some internal measurements on the unit. These
measurements are taken at the nodes mentioned already.
Measurements may involve estimating the goodness of a
signal, or measuring voltages and resistances.
This
information is stored in the frames during the diagnosis. A
typical circuit will have about 20 nodes at the modulelevel
and approaching 100 nodes altogether. Consequently there
is a large numberof measurementsthat can be taken during
the diagnosis. The advantageof the goal-driven diagnosis is
that it requests only measurementsthat contribute to its
current hypothesis. In a typical session only about 20%of
measurements are requested. This process will now be
described in moredetail.
Goal driven
reasoning
in NODAL
The output of the shallow reasoning process in NODAL
is
a candidate set" of faulty modules. The model-based
reasoning takes over at this stage with the objective of
detecting the faulty module and the faulty component
within that module. The behavioural component of the
model is expressed as rules and the goal driven reasoning
involves backwardchaining through these rules applied to
the modelof the unit under test. A typical rule from the
modulelevel reasoning is as follows:-
Whatis the
Whatis the
Whatis the
Whatis the
VOLTAGE
of
VOLTAGE
of
VOLTAGE
of
VOLTAGE
of
NODE-2?23.4
LPS-I? 18.79
NODE-3?18.12
NODE-9?0
It lookslike thefaultis in CR2
Even by NODAL’s
standards this dialogue is particularly
short as the first moduleto be examinedproves to be the
faulty one (for more example dialogues see (Cunningham
1988)). Nevertheless it serves to illustrate howthe goaldirected reasoning focuses the requesting of measurements
IF
from the operator. This brings our requirements on a CBR
(MODULE?
HAS A BADSIGNALONOUT-1 OR
system for the same task into focus. Since there is a cost
MODULE?
HASA BAD SIGNALONOUT-2)
ANDMODULE?
HAS A GOOD
SIGNALON IN-1
associated with determiningthe inputs to the diagnosis it is
ANDMODULE?
HASA GOODSIGNALON IN-2
important that the CBRsystem should not need to have all
THENTHESTATUSOF MODULE?
IS FAULTY
the inputs in advance. It should be able to direct the
The rule is expressed in the TellAndAskknowledge-base operator on what measurementsare important just as the
query language of KEEas follows:goal-driven system does.
IT-RULE-2-2
IF ((OWN.VALUETYPE?MODULE
TWO-TWO)
(OWN.VALUE
SIGNAL(GET.VALUE
?MODULE
OUT-1)?OS1)
Existing
(OWN.VALUE
SIGNAL(GET.VALUE
?MODULE
OUT-2)?OS2)
((EQUAL?OS1BAD) OR(EQUAL?OS2BAD))
(OWN,VALUE
SIGNAL(GET.VALUE
?MODULE
IN-l) ?1S1)
(EQUAL?1S1 GOOO)AND
(OWN.VALUE
SIGNAL(GET.VALUE
?MODULE
IN-2) ?1S2)
(EQUAL?1S2 GOOD))
THEN(A STATUSOF ?MOOULE
IS FAULTY)))
A typical rule from the componentlevel reasoning is:(NPN.B.E.DOD.RULE
(i~FIND’¢ (IN.CLASS?NPNSTD-NPN)
DONT.ASK)
(LISP (NOT(DIODE-BETWEEN-NOOES-P
’B-E-DIODE
(UNIT.NAME(GET.VALUE?NPN’BASE))
(UNIT.NAME(qGET.VALUE
?NPN’EMITTER))))))
THEN(OWN.VALUE
STATUS?NPNFAULTY)))
This rule checks to see if there is the correct diode
voltage drop betweenthe base and emitter of a transistor.
Usingthe modelof the circuit, the goal-directed reasoning
uses these heuristics to determine first the faulty module,
then the faulty component.
An example of a dialogue with NODAL
will illustrate
howthis worksin practice. The fault being analysed arises
from removing the zener diode (CR2) in the Local Power
Supply Module of the 24V/12V SMPS. This simulates
that diode blowing open-circuit. At the point when the
dialogue commencesthe system has already completed the
shallow reasoning based on the function test rules and has
established a candidate set of faulty modules. It then
proceeds to try and prove one of these to be faulty
(underlinedtext is input by the user):-
diagnostic
CBRsystems
Despite this difficulty that we identify in the use of CBRin
diagnosis there has been considerable interesting and
successful research in the area. Somesystems that are worth
highlighting are as follows:- The help desk application of Simoudisand Miller
(Simoudis & Miller 1991)
¯ CASEY:
a system for managingthe diagnosis of
cardiac disease (Koton1988)
¯ PROTOS:
a system for assisting in the diagnosis of
audiologydisorders. (Porter et al 1990)
¯ The GEhelp desk application of Kriegsmann&
Barletta (Kriegsmann&Barletta 1993)
In the remainder of this section we will examinethese
systems focusing on the aspects that are relevant to our
problem.
Simoudis & Miller’s
Help Desk System.
This help desk application is designed to assist product
support engineers in diagnosing customer problems with
the VMSoperating system. The system uses a two-phase
case retrieval process of surface feature-based retrieval
followed by model-basedvalidation. The surface features are
inexpensive to obtain and include hardware and software
system descriptions and data obtained from the core dump
associated with the system failure. The first phase in the
retrieval processreturns all cases fromthe case-basethat are
similar according to these surface criteria. The model-based
validation uses validation information from these cases to
Setupfor Test Vector1
direct further inquiry into the target case. This validation
process will determine whether the new problem is in fact
Whatis the SIGNALof NODE-2?
Good
similar to any of those retrieved fromthe case-base.
Whatis the SIGNALof NODE-3?
Bad
It appears that this strategy would not work in our
situation
because there is not an obvious set of surface
It looks like the fault is in the LOCAL-POWER-SUPPLY
features that wouldsignificantly reduce the case-base.
Switchingto considering~thecircuit at a component
level...
50
CASEY
In CASEY,
Koton integrates case-based and model-based
reasoning techniques to produce an expert system for
managingthe diagnosis of cardiac disease. Each case
represents a single patient diagnosis and is composed
of
both descriptive features and solution features. The
descriptive features correspondto the patient’s observed
symptoms
and test results, whereasthe solution features
describethe diagnosisandsuggestedtherapy.
Given some new patient description, CASEYwill
attempt to identify the causality underlyingthe observed
disorder andproposetherapy(the diagnosissolution) using
a three-step process. First, CASEY
will search for a case
that is similar to the current patient diagnosis context.
Second,it evaluates the significance of any differences
betweenthe target andbase case. This evaluationis carried
out using a set of evidenceprinciples and will reject a
matchif these principlessuggestthat certain featuresof the
base case cannotbe appliedto the target. Third, if noneof
these differencesrule out the applicabilityof the case, then
it adapts the solution of the base case to fit the target
context. CASEY
adapts a retrieved case using a set of
causalrepair strategies to modifythe nodesor links in the
casual explanationof the base case.
Thesystemoffers the operator a templateon whichthe
target specificationis to be entered.Theoperatoris free to
leave blanksin this templateas the retrieval mechanism
can
operate with incompleteinformation. The system uses an
inductively built decision tree to identify a group of
candidatecases with contextually similar features to the
target problem. For each candidate a score is computed
using nearest-neighbourmethods.The score reflects the
similarityof the candidateto the target. If the initial target
specification is too general or sparse to result in the
retrieval of a single best case or evena small subset of
candidates the system allows the operator to specify
additionalinformation
to further focusthe retrieval process.
Thedeterminationof whichadditional features to specifyis
left to the operator.
Incremental
CBR in NODALcBa
Ourobjective in this workis to explainhowthe desirable,
goal-directed behaviour of the NODAL
model-based
diagnosissystemmightbe transferred to an equivalentCBR
system, which we call NODALcnn.What we have
developed is a CBRsystem that can begin operation
withouta completetarget case descriptionand can generate
queriesthat will help it to homein on a solution. Wehave
implementedan information theoretic mechanismthat
PROTOS
identifies the maximally
discriminantdiagnosticfeatures of
PROTOS
is a classification and learning system for the retrievedcases. Thus,rather that expectingthe operator
operationin clinical audiology(Porter et al. ’90). PROTOSto chosethe next test to apply, the systemwill proposeone
automatically.In addition the test chosenshouldmaximise
implements a type of prototype based classification
whereby
surfacesfeatures of the case are usedto identify a the amountof newinformationgained, and henceensurethe
set of candidatecategories. Thesystemuses prototypical mostrapid route to the desireddiagnosis.
examplesof these categories in the remindingprocess. The
highest ranking of these remindingsis selected and the Case Representation & Retrieval
system initiates a dialogue with a humanexpert to Ourfirst objectivehas beento replace the first twostages
determinewhetherit is a correct diagnosis.If the diagnosis
diagnosis in the old system(see Figure3) with one CBR
is incorrect the systemattemptsto adjust its links between of
stage. Consequently
the cases features are the information
case features and categories in order to producea correct
used
in
the
diagnosis
at this point, that is the functiontest
diagnosis. The focus of the work on PROTOS
has been on results and the module
level signal information.Thecase
conceptacquisition in a weaktheory domainrather than on also contains a case nameand the identity of the faulty
the diagnosis itself. So, while PROTOS,
does conduct a
dialoguewith its operator, the motivationis different to moduleassociated with these symptoms(the solution).
case structure is shownin Figure 4. It is in the
what we have in mind. It is concerned with knowledge typical
natureof the diagnosistask that the easefeaturesare sparse.
acquisition while we want the dialogue to further the There is often a small numberof results to the function
diagnosisitself.
tests becauseoncea unit fails oneof the early tests it is not
possible to proceedwith subsequenttests. In addition, a
The GE Help Desk System
large portionof the signal features are not examined
as the
Kriegsmann& Barletta view CBRas having a numberof diagnosis quickly concentratesin a specific part of the
advantagesover moreconventionaltext-based or rule-based circuit.
help-desk systems (Kriegsmann& Barletta 1993). Their Case Name
Faulty
Module Name |
prototypesystemaimedat investigating the role of CBRin
Module
Function
(Solution)
Level Features
Test Results
providing help-desk functionality across a range of
computerhardware, software, and networkingproblems.
approx10 Io30
ayln’oz20 to 40
Theproblembeing addressedin this systemis similar to
~ ~
ours in that there is a cost associatedwith determiningthe
casefeatures.
Figure
4. Atypical casestructure in NODALo]
R.
51
Wecan nowconsider what happens during a typical run
of the CBRsystem. In this first scenario the unit fails one
function test, 3-POS-OUTPUT-VOLTAGE,
after which it
is not possible to continue with further function tests.
Cases matchingthese function test results are returned from
the case-base. In this example these are cases with the
signal features shownin Table 1. At this stage the unit
under test has not been probed for this signal information
so we want the system to ask some discriminating
questions - this was the particular strength of the old
NODALsystem.
Table1. This chart showsthe modulelevel portion of several
cases returnedduringthe diagnosis.
Nodes
G: ~3d
1 i lil
Case Name 23,456789012345678
Input-Filter-1 B
GB
Local-PS-1
GIC B
Driver&PS-1 GG
G
Control-Cct-1 G G GGBG
BBGG
G
Clock-Gen-1 GG
BGGG
G
Clock-Gen-2
GBGG
G
Clock-Gen-3 GG
GB
G
Driver&PS-2 GG
GB
B
Driver&PS-3 G G
G G GGGG
GBB
Xfmr-1
[G G GGGG
GBG
Xfmr-2
OG
GGGG
G B
Xfmr-3
GG
Ou~ut-Rect-1 G G GGGG
Output-Rect-2 O G! GGGG
G G
B:bad
1 1 1 1 1
Discriminating
Wecan view the decision tree as an information source
producing one of d messages from the set D. Let IDi I
represent the numberof cases with diagnosis Dr. Then the
expected information needed to generate the appropriate
message,for somecase, using the tree is:.
Io,I ,,,
I(ID,I
LID,I+...+ID,
IJ)
Consider
the root decisionnodeof the tree (see Figm¢5).
Assume
this nodetests the feature FEFand this feature has
possiblevaluesV--{VI ..... V.}. ThenV partitions (2 into
groupsof cases,GI ..... G.; whereGi contains those cases
that have value Vi for feature F.
G
F
nV
V
G1 ~’~--~
B~B
BG
GB
B
GB
..........
--]
n
Figure5. Theroot classification of the cases in C.
The solution we have adopted is to use information
theoretic criteria similar to those used in ID3 (Quinlan
1986) to determine the most discriminating feature to be
measuredat each stage in the narrowingdownof this subset
of cases.
Selecting
Dffi {DI,...,Da} the set of possible classes or diagnoses
(7 in Table1)
C={CI,...,Cc}the set of cases to classify
(14 in Table 1)
Fffi{Fb...,Ff} the set of descriptive features that will form
the nodes of the decision tree. (17 in Table 1)
Features
This selection of discriminating features amounts to
building a decision tree that will have leaf nodes
corresponding to the different diagnoses D and the set of
cases C will be located, or classified, on these nodes. It is
important that the tree is in some sense minimal so the
choice of whichfeature to test at any level of the tree is
critical. In ID3 this is done by selecting features based on
their information content or discriminatory power(Quinlan
1986). The process used in NODALcBR
is similar to that in
ID3 except that the semantics of the branching in the
decision tree is slightly different because of the large
number of unknowns in the case features.
A brief
explanation of howthe discrimination works is as follows:-
52
Let Gi contain IDij I cases with diagnosisDj, that is IDtj I
instances of class Dj. Then the expected information
required for the sub-tree of Gi is I(IDilI ..... IDidl). Wecall
obtain the expected information for the tree with F as root
by computingthe weighted average over all value branches
of F as follows:i
i
E(ID,
I+’"+ID.I)
Ddl)
° I(ID’I
E(F)-,-,
I,[D,I+...+I
.....ID’I)
n
(2)
Theweightof the ith branchis the proportionof casesin
C that belong to Gi. The information gained from using F,
or the discriminatorypowerof F, is:DP(F) =
I(ID,l ..... IO.l)- E(F)
So, at each stage in the reduction of the set of cases, the
most discriminating feature is selected using this criterion.
The user is requested to determinethe value of this feature
for the target case. Theretrieved cases that cannot matchon
this feature are removedfrom the relrieved set. This process
is repeated until the set reduces to one diagnosis or the
target case proves to be dissimilar to all the retrieved cases.
It is important to emphasisethat a discrimination tree for
the set of cases is not being produced, instead local
discriminations are determined at run-time. This technique
information that will confirmthat the cases match. The
importanceof this validationdependson the coverageof the
case-base.It is not requiredwhencoverageis good.
Whenwecomparedthe CBRsystem with the old system
on a sampleset of faults on the DC/DC
circuit we found
Evaluation
that is required only 83%of the user input that the MBR
In this section wewill present a comparisonof the MBR systemdid. This informationis plotted on a case by case
and CBRsystems and evaluate the strengths of the CBR basis in Figure6. Twoother smaller evaluationsare shown
system.Returningto the exampleintroducedearlier, wecan on Figure7 (a) and (b). In these situations the number
questions is reducedto 35%and 33%respectively. Froma
continue with the dialogue generatedby NODALcBa:situation whereour initial aspiration wasto producea CBR
Selectingfunction test failure cases: Retrieved14
systemthat wouldhave the informational parsimonyof a
goal-drivensystem wefind that the CBRsystemis better
> (INPUT-FILTER-1
LOCAL-PS-1
DRIVER&PS-1
CONTROL-CCT-1
CLOCK-GEN-1
CLOCK-GEN-2
CLOCK- than the old NODAL
system.
GEN-3DRIVER&PS-2
DRIVER&PS-3
XFMR-1XFMR-2
XFMR-30UTPUT-RECT-10UTPUT-RECT-1)
has provedremarkablysuccessful; indeedit results in less
questionsbeingaskedof the user that wasthe case with the
original NODAL
system.
Whatis
Whatis
Whatis
Whatis
Whatis
Whatis
O.K.....
the valuefor
the valuefor
the value for
the valuefor
the valuefor
the valuefor
Category 1
N2? G
N3? G
N10? G
N7? G
N8? G
N5? B
XFMR-2
i!iiii!iiiii~iii!iiiiiii~i~!i!!i!iiiii~iiiiii!iii
!
XFMR-1 ililili!!’:W:’:’:’:’:::’:’:’:’:::~:’:’:’,’:’::l
I |
XFMR-3
Selecting candidatemodules’ Retrieved2
(CLOCK-GEN-1CLOCK-GEN-2)
I;
Jl :
Output.Rectifle~- ! ! i ~ i "L:’.’.!!!~
" i ! Li[ i~!~!!ffff’l
Neg.t
Output-RectifierPoe-1
Validation:
The fault is in CLOCK-GENERATOR
if
N6is B or N6is G
ClockGenetarot-3
The 14 cases shownin Table 1 are returned and N2 is
foundto be the first mostdiscriminatingcriteria. After 6
questions the faulty moduleis discovered. This compares
with 7 questions in the model based reasoning of old
NODAL:Setupfor Test Vector1
Clod~Generator-2
,4aa
ClockGenerator-1
Driver-andPower-Supply-2
Control-Clrcult-1
I
Whatis
Whatis
Whatis
Whatis
Whatis
Whatis
Whatis
Drlvor-andPower-Sul0ply-3
the SIGNALof NODE-2?
Good
the SIGNALof NODE-3?
Good
the SIGNALof NODE-10?
G~d
the SIGNALof NODE-8?
Good
the SIGNALof NODE-7?
Good
the SIGNALof NODE-5?
Bad
the SIGNALof NODE-6?
Bad
I
Driver-and-~[i[
