From: AAAI Technical Report FS-95-04. Compilation copyright © 1995, AAAI (www.aaai.org). All rights reserved.
What Kind of Adaptation
do CBR Systems
Current Practice
Kathleen
Hanney,
Mark T. Keane
Padraig
Cunningham
Department of Computer Science
University of Dublin
Trinity College
Dublin 2, Ireland
Kathleen.Hanney, Mark.Keane,
Padraig.Cunningham@cs.tcd.ie
and
Abstract
This paper reviews a large number of CBRsystems
to determine when and what sort of adaptation is
currently used. Three taxonomies are proposed: an
adaptation-relevant taxonomyof CBRsystems, a taxonomyof the tasks performed by CBRsystems and
a taxonomyof adaptation knowledge. To the extent
that. the set of existing systemsreflects constraints
on what is feasible, this review showsinteresting dependencies betweendifferent system-types, the tasks
these systems achieve and the adaptation needed to
meet system goals. The CBRsystem designer may
find the partition of CBRsystems and the division of
adaptation knowledgesuggested by this paper useful.
Moreover,this paper mayhelp focus the initial stages
of systems developinent by suggesting (on the basis
of existing work)what types of adaptation knowledge
should be supported by a new system. In addition,
the paper provides a frameworkfor the preliminary
evaluation and comparisonof systems.
Introduction
The growth of CBRhas led to a great diversity in the
field. CBRsystems perform diverse tasks in a multiplicity of domains. Someof these systems contain all
of the characteristic processes of CBRsystems (e.g.,
retrieval, reuse, adaptation, learning), although many
employonly a subset of these processes. However, even
when these processes are present they come in many
different guises. In this paper, we examinethe diversity in adaptation. Our aim is to learn from current
practice, to review CBRsystems to determine when, in
practice, adaptation tends to be used. Such an analysis
is useful for manyreasons; for instance, it mayhelp system designers to predict adaptation requirements for
new systems or help constrain the comparison of adaptation methods in different systems. Westructure our
review with three taxonomies: a taxonomy of adaptation knowledge (see section 3), a taxonomy of CBR
systems (of relevance to adaptation) and a taxonomy
of task types (see section 2). To date, our review has
considered 53 CBRsystems in detail, using available
Need?:
A Review
of
Barry
Smyth
Hitachi Dublin Laboratory
University of Dublin
Trinity College
Dublin 2, Ireland
Barry.Smyth@hdl.ie
sources (11 of these systems were not reported in sufficient detail for our analysis). A list of reference sources
for the reviewed systems is given in the bibliography.
An adaptation-oriented
CBR systems
taxonomy
of
Adaptation is treated very differently in many CBR
systems. First, many systems avoid using adaptation
at all, so-called retrieval-only systems. Second, distinctions are often madebetween different types of adaptation, like derivational and transformational adaptation
(see (Carbonell 1983; 1986)). Third, most systems
domain-specific adaptation procedures but some also
use domain-independent strategies (e.g., step reordering in CHEF(Hammond1989)). Wereduce this diversity to just four dimensions to classify CBRsystems
with respect to adaptation:
¯ Presence or Absence of Adaptation - Systems clearly
divide into those that do or do not use adaptation. Wedefine adaptation as the modification of
case solutions, thus excluding systems that combine unmodified cases or case parts. For example,
ARCHIE2(Domeshek & Kolodner 1993) performs
no adaptation while CHEF(Hammond1989) does.
¯ Single or Multiple Cases - Solutions may be based
on single or multiple cases. Single-case systems base
their solutions on just one case, even if successive
cases are tested before a solution is reached. CASEY
(Koton 1989), for example uses a single past case.
Multiple-case systems base their solutions on more
than one case. In AIRQUAP(Lekkas & Avouris
1994), for example, the solution to the target problem is the mean value of the solutions to the most
similar past cases. Somemultiple case systems have
problem subparts that can be identified and separately modified to composea solution. For example,
the case-based design system, CADRE
(Dave et al.
1994), composes a design from several cases or case
parts.
CBR
&jster~
I
I
Single
Case
I
I
Multiple
Cases
I
I
I
NoAdaptation
NoAdaptation
Adaptalion
Adaptation
I
I
I I I
I
ktornio
Compound
Atomio
Coml)ound
IO) 2(0)5(1)At~i°Com~60)
8(0) Atoeio
11(0)
Comped
Manipulable
i
I
I
Interaction
MeInteraotke
3 (4)
4 (2)
Compound
Han~ulable
I
I
I
~,teraotion
~Interaotion
904)
iO(S)
Figure 1: Classifying Systems on Four Dimensions Relevant to Adaptation (N = 42)
erature. Setting aside this reporting bias, it is also
clear that some systems do not make practical sense.
For instance, we found no instances of systems using
adaptation with compoundsolutions that are not manipulated using adaptation knowledge (see 2 and 8);
clearly, if compoundsolutions are available and adaptation is being used then it makes practical sense to
manipulate the parts of these solutions using adaptation knowledge (see 3, 4, 9 and 10). Similarly,
does not make practical sense to use multiple cases,
without adaptation, to reach an atomic solution (see
11). Of the 42 systems reviewed the majority reasoned
from multiple cases (33 or 79%). The clustering
the multiple-case categories (9 and 12) indicates
important focus for CBRsystems. These categories
are sophisticated reasoners that merge solutions from
different problem-solving contexts and handle complex
interactions betweensolution goals. This initial classification shows that most published CBRsystems use
multiple cases, and divide evenly into sets that do and
do not use adaptation. To understand the basis for
this split and other aspects of the distribution we categorised these systems by the task they achieve.
Analysing
¯ Complexity of Case-solutions - Solutions may be
atomic-valued solutions that are primitive and undecomposable, compoundsolutions that are decomposable by processes other than adaptation (e.g.,
retrieval) or compound-manipulable solutions that
have parts which can be modified by adaptation
(Cunningham, Smyth, & Veale 1994). Solutions
AIRQUAP
for instance, are atomic numeric. CELIA
(Redmond 1992), has compound but not compoundmanipulable solutions since they are composed of
unmodified case snippets. PRODIGY
(Veloso 1992),
has compound-manipulable solutions since individual plan steps can be modified.
¯ Interactions within Case-solutions - Compoundmanipulable solutions can have solution-parts that are
independent or interacting (e.g., where the adaptation of one part of a solution requires further modification of other solution parts). As an example,
consider JULIA, (Hinrichs 1992), which detects and
fixes bad interactions between parts of meal designs.
Figure 1 combines these four dimensions into a hierarchy under which the reviewed systems are classified.
At the leaves of this tree the first number indicates
the number of the category and the number of systems found in this category is shown in brackets. It
is immediately clear that systems cluster into particular categories, even though all the categories are, in
principle, viable system-types. Why?
There are several possible reasons for this distribution. First, there is clearly a reporting bias in the literature. Manyof the simpler systems (e.g., 1, 5 and
7) are less likely to be reported in the research lit-
system-types
by task
Task classifications are not easy as there is no canonical set used by system builders. Wetried to achieve
some standardisation by adopting Clancey’s taxonomy
of tasks for expert systems (Clancey 1985). Only
three of Clancey’s task types are needed to categorise
CBP~systems: identify (basically recognition or classification),
predict, and design (subsuming configuration and planning). CBRsystems either do identification
alone (PROTOS(Bareiss 1989), yI-CBR
(Cunningham, Smyth, & Bonzano 1995), ARCIIIE2
(Domeshek & Kolodner 1993), ORCA(Bareiss &: Slatot 1993), ASK-SYSTEMS
(Ferguson et al. 1992),
CASCADE
(Simoudis 1992), CELIA (Redmond 1992))
or identification
and prediction (AIRQUAP(Lekkas
& Avouris 1994), JUDGE (Bain 1989), REBECAS
(Rougegrez-Loriette 1994)) or identification and design
(PPdAP~ (Kambhampati & IIendler
1992), CADSYN
(Maher & Zhang 1993), CASEY(Koton 1989), JULIA
(iiinrichs
1992), CIIEF (Hammond1989), CYCLOPS
(Navinchandra
1991), ROENTGEN(Berger 1994),
MEDIATOR(Simpson 1985), KRITIK (Goel 1989),
PRODIGY(Veloso 1992), COBRA(Slattery
1993),
CASPIAN(Price, Pegler, & Bell 1993), CADRE(Dave
et al. 1994), PLEXUS(Alterman 1988), DEJA
(Smyth & Cunningham 1992), PANDA(Roderman
& Tsatsoulis 1993), DDIS (Wang & IIoward 1991),
BOLERO(Lopez & Plaza 1993), TOLTEC(Tsatsoulis
& Kashyap 1993), ROUTER2(Goel & Callantine
1992), BOGART(Mostow 1989), APU (Bhansali
IIarandi 1993), PERSUADER.
(Sycara 1987), IIYPO
(Ashley 1991), CLAVIER(IIinkle
& Toomey 1994),
MBR-talk (Stanfill
& Waltz 1986), CABARET
(Skalak
& Rissland 1992), CABAssembler (Pu & Reschberger
42
[] cat12
go~
Ill)
carl0
I::1cat9
7O%
6O%
5O%
40~
[] cat7
care
Bcat5
20~
O%
J cat4
I
I
I
IdenUfg Identifu,Identifg, [] cat5
Predict Design
¯carl
tasks
Figure 2: Percentage of System-types used to achieve
different Tasks
1991), GREBE(Branting 1991), CADET(Sycara
Navinchandra 1991), SWALE
(Schank, Kass, &: Riesbeck 1994)). One system uses cases for identification
and control ACBARR
(Ram e~ al. 1992).
Figure 2 gives us some insight into the clustering
found in our system taxonomy. It shows strong dependencies between system-types and task-types. First,
identification tasks are the preserve of adaptation-free
systems (5, 6 and 12), whereas most prediction and
design tasks require adaptation (e.g., 1, 3, 4, 9 and
10). Second, prediction can be achieved using solutions that are either atomic or the simpler form of
compound-manipulable solution (1, 7 and 10). Third,
design tasks largely require adaptation on compoundmanipulable solutions (3, 4, 9 and 10), except when the
systems are design assistants (i.e., category 12 systems
like ARCHIE2).The distribution of systems performing different tasks suggests that certain system-types
are optimal for different tasks. (Clearly the complexity and completeness of domain knowledge is also an
important, but more difficult to characterise, consideration).
When types
of
adaptation
are used
knowledge
Our initial taxonomies show that CBRsystems using
adaptation are predominantly used when prediction
and design is required. In this section, we consider
the nature of the adaptation knowledge used by these
systems to determine whether there are any regularities in the use of adaptation knowledgeacross different system-categories and tasks. Several alternative
taxonomies of adaptation knowledge have been proposed (Collins 1989; Hinrichs 1992; Kolodner 1993;
Riesbeck & Schank 1992), but none of them meet
our specific needs. Kolodner’s taxonomy of adaptation methods includes substitution (used when a substitute already exists) and transformation (used
change a solution or solution element by inserting or
deleting a subpart). Various substitution methods are
distinguished according to the method used go find a
substitute.
Transformational methods can be either
model-guided or heuristically driven. For our purposes,
we need a taxonomy that is general enough to show
the commonalty between disparate systems but specific enough to be tell us something about adaptation.
Weidentify four main types of adaptation knowledge:
target-elaboration operators, role-substitution operators, subgoaling operators and goal-interaction operators:
* Target Elaboration Operators elaborate the target
problem description to facilitate adaptation, resulting in a better specified target. Elaboration can either add new information to or re-describe (or recategorise) somepart of the target. These operators
are clearly indispensable when aspects of the target are uncertain or underspecified.
In CYCLOPS
(Navinchandra 1991), for instance, similar cases can
signal problems or opportunities within a new design
resulting in the introduction of new criteria.
¯ Role Substitution Operators effect substitutions at
various levels of granularity. These operators can be
simple or complex. In simple role-substitution, a local substitution of parts of a solution (actions, values
or roles) is made without reference to dependencies
between these parts and other parts of the solution.
For example, PERSUADER
(Sycara 1987), adapts
numerical parameters using special-purpose critics.
Complexrole substitution can change several parts
of a solution at once, where such substitutions are
sensitive to causal or other dependencies between
solution parts. In CHEF,for example, the substitution of a new ingredient in a plan may require the
insertion of special preparation steps.
¯ Subgoaling operators handle the decomposition of
the adaptation task into smaller, simpler tasks. If a
solution requires several distinct adaptations, then
a subgoaling operator might sequence these adaptations.
DEJA VU (Smyth & Cunningham 1992),
delegates its adaptation tasks to adaptation specialists. Each specialist makes a local modification to a
retrieved case.
Goal interaction operators handle interactions between solution parts and handle both the detection
and repair of bad interactions that occur. In DEJA
VU, general adaptation strategies are used to handle bad interactions resulting from the local changes
made by adaptation specialists.
43
I00
go
80
70
target
elaboration
IWsimple
role
subsUlution
5O
4O
3O
20
10
0
i:i
::: R~omplex
role
::: substitution
: I Fl ~ubgo~ling
identi(!l,
predict
Dgoal
interaction
g
Figure 3: The Numberof Systems using different
Types of Adaptation Knowledge
Figure 4: The types of Adaptation Knowledgeused in
System Categories
To look at "task by adaptation-knowledge" dependencies we plotted the numberof prediction and design systems that used different types of adaptation
knowledge(see Figure 3). Several notable regularities
arise. The adaptation requirementsfor prediction are
quite simple; they merely involve target elaboration
and simple role substitution. However,in design systems the whole range of adaptation knowledgetends
to be employed;design systems use each type of adaptation knowledgeat least 75%of the time. This figure
shows that the main focus for adaptation knowledge
is in design tasks and that relatively simple adaptation knowledgecan be used in prediction (although assistants can circumventthis generalisation). No type
of adaptation knowledgeis required for identification
(see Figure 2). Amorefine-grained analysis is possible if welook at the "system-categoryby adaptationknowledge"dependencies. In Figure 4, we have a rough
ordering of adaptation systemsin terms of their complexity: 1 and 7 are the simplest with atomicsolutions,
4 and 3 are more complexin that they have compound
solutions (and use single cases), and 9 and 10 are the
most complex as they have compoundsolutions and
use multiple cases. Thefigure showsthat the requirement for adaptation knowledgeincreases as systems
increase in complexity: Simpler systems can get by
with target elaboration and role substitution, but more
complexsystems use diverse adaptation knowledge.It
is also notable that almost every system with compound,interacting solutions (3 and 9) requires the full
range of adaptation knowledge(irrespective of whether
single or multiple cases are being used).
In this paper wehave described three distinct taxonomiesto structure our review of adaptation in CBP~
systems. In general, the space of systems explored by
researchers is a goodindicator of practical constraints
on system construction. Acceptingthe restricted nature of our sample, it is clear that strong dependencies exist betweenthe adaptation knowledgeused, the
task to be achievedand the nature of the case solution.
Other moredetailed conclusions mayarise as moresystems are included in our on-goingreview, but several
arise here:
¯ identification tasks can be achievedwithout adaptation
¯ prediction tasks require more adaptation but minimal amounts are required
¯ design tasks have the heaviest adaptation requirementsin the amountand diversity of this knowledge
(except whenoverridden by the use of a case-based
assistant approach)
¯ systems using solutions with compound-interacting
parts need all forms of adaptation knowledge(irrespective of whethersolutions are based on single or
multiple cases)
Manymight think that these conclusions are obvious.
To the extent that they conformto an expert’s intuitions about the field, they will be successful if they
are obvious. However,it is useful to back such intuitions with an empirical analysis. Furthermore,the review suggests that systembuilders can expect different
adaptation requirementsfor specific task situations.
Conclusions
44
Appendix
What are the specific benefits and limitations of your
approach? Our analysis of CBRsystems can help system designers to predict requirements for new systems
or constrain the comparison of adaptation methods in
systems. Weshow that strong dependencies exist between the adaptation knowledge used, the task to be
achieved and the nature of the case solution. The major limitation of our approach is that it is valid only
to the extent that the set of existing systems reflects
constraints on what is feasible.
What hypotheses were explored? Our hypothesis
was that there were dependencies between different
system types, the tasks these systems perform and the
adaptation knowledge needed to achieve system goals.
What are the primary contributions
of your research? Our research mayprovide guidelines to system
builders to help them predict adaptation requirements.
In addition our work provides a useful framework for
the description and comparison of CBRsystems.
References
Alterman, R. 1988. Adaptive planning. Cognitive
Science 12:393-422.
Ashley, K. 1991. Reasoning with cases and hypotheticals in hypo. International
Journal Man-Machine
Studies 34:753-796.
Bain, W. 1989. Judge. In Riesbeck, C., and Schank,
R., eds., h~side Case-Based Reasoning. Northvale NJ
: Erlbaum. 93-140.
Bareiss, E., and Slator, B. 1993. The evolution of
a case-based computational approach to knowledge
representation, classification, and learning. In Nakumura, G.; Medin, D.; and Taraban, R., eds., Categorisation by Humansand Machines. NewYork: Academic Press.
Bareiss, E. 1989. Exemplar-Based Knowledge Acquisition: A Unified Approach to Concept Representation, Classification, and Learning. Boston: Academic
Press.
Berger, J. 1994. Roentgen: Radiation therapy and
case-based reasoning. In Proceedings of the lOth
Conferenceon Artificial Intelligence for Applications.
IEEE Computer Society Press.
Bhansali, S., and Harandi, M. 1993. Syntesis of unix
programs using derivational analogy. Machine Learning 10:7-55.
Branting, L. 1991. Exploiting the complementarity of
rules and precedents with reciprocity and fairness. In
Bareiss, E., ed., Proceedings of the Case-Based Reasoning Workshop. 39-50.
Carbonell, J. 1983. Learning by analogy: Formulating and generalizing plans from past experience.
In Michalski, R.; Carbonell, J.; and Mitchell, T.,
eds., Machine Learning: An Artificial h~telligence
Approach, volume 1. Springer-Verlag. 137-159.
45
Carbonell, J. 1986. Derivational analogy: A theory
of reconstructive problem solving and expertise acquisition plans from past experience. In Michalski, R.;
Carbonell, J.; and Mitchell, T., eds., MachineLearning: An Artificial Intelligence Approach, volume 2.
Morgan Kaufmann. 371-392.
Clancey, W. 1985. Heuristic classification. Artificial
Intelligence 27(3):289-350.
Collins, G. 1989. Plan creation. In Riesbeck, C.,
and Schank, R., eds., Inside Case-based Reasoning.
Northvale, NJ : Erlbaum. 249-289.
Cunningham, P.; Smyth, B.; and Bonzano, A. 1995.
An incremental case retrieval mechanismfor diagnosis. Technical Report TCD-CS-95-01, Department of
Computer Science Trinity College Dublin.
Cunningham, P.; Smyth, B.; and Veale, T. 1994.
On the limitations of memorybased reasoning. In
Keane, M.; Haton, J.-P.; and Manago, M., eds., Proceedings of the Second European Workshop on CaseBased Reasoning. 59-65.
Dave, B.; Schmitt, G.; Shen-Guan, S.; Bendel, L.;
Faltings, B.; Smith, I.; HuGK.; Bailey, S.; Ducret,
J.-M.; and Jent, K. 1994. Case-based spatial design
reasoning. In Keane, M.; Haton, J.-P.; and Manago,
M., eds., Proceedings of the Second European Workshop on Case-Based Reasoning. 115-123.
Domeshek, E., and Kolodner, J. 1993. Using the
points of large cases. AI EDAM
7(2):87-96.
Ferguson, W.; Bareiss, R.; Birbaum, L.; and Osgood,
R. 1992. Ask systems: An approach to the realization
of story-based teachers. The Journal of the Learning
Sciences 2(1):95-134.
Goel, A., and Callantine, T. 1992. An experiencebased approach to navigational route planning. In
Proceedings of the 1992 IEEE/RSJ International
Conference on Intelligent Robots and Systems. 705710.
Goel, A. 1989. Integration of Case-Based Reasoning and Model-Based Reasoning for Adaptive Design
Problem Solving. Ph.D. Dissertation, Department of
Computer and Information Science. The Ohio State
University.
Hammond, K. 1989. Case-Based Planning: Viewing Planning as a Memory Task. Boston: Academic
Press.
Hinkle, D., and Toomey, C. 1994. Clavier: Applying
case-based reasoning to composite part fabrication.
In Proceedings of the lOth Conference on Artificial
Intelligence for Applications.
Hinrichs, T. 1992. Problem Solving in Open Worlds:
A case study in design. Northvale, NJ:Erlbaum.
Kambhampati, S., and Hendler, J. 1992. Validationstructure-based theory of plan modification and reuse.
Artificial Intelligence 55:193-258.
Kolodner, J. 1993. Case-based Reasoning. Morgan
Kaufmann.
Koton, P. 1989. Using Experience in Learning and
Problem Solving. Ph.D. Dissertation, Department of
Computer Science MIT.
Lekkas, G., and Avouris, N. 1994. Case-based reasoning in environmental monitoring. Applied Artificial
intelligence 8:359-376.
Lopez, B., and Plaza, E. 1993. Case-based planning
for medical diagnosis. In Komorowski,J., and Ras, Z.,
eds., Methodologies for Intelligent Systems, Lecture
notes in artificial intelligence.
Maher, M., and Zhang, D. 1993. Cadsyn: A casebased design process model. AI EDAM7(2):97-110.
Mostow, J. 1989. Design by derivational analogy.
Artificial Intelligence 40:119-184.
Navinchandra, D. 1991. Exploration and Innovation
in Design, Towards a Computational Model. New
York Springer-Verlag.
Price, C.; Pegler, I.; and Bell, F. 1993. Case-based
reasoning in the melting pot. International Journal
of Applied Expert Systems 1(2):120-133.
Pu, P., and Reschberger, M. 1991. Assemblysequence
planning using case-based reasoning techniques. In
Gero, J., ed., Artificial Intelligence in Design. IEEE
Computer Society Press.
Ram, A.; Arkin, R.; Moorman, K.; and Clark, R.
1992. Case-based reactive navigation: A case-based
method for on-line selection and adaptation of reactive control parameters in autonomous robotic systems. Technical Report GIT-CC-92/57, School of Information and Computer Science, Georgia Institute
of Technology.
Redmond, M. 1992. Learning by Observing and Understanding Expert Problem Solving. Ph.D. Dissertation, School of Information and Computer Science,
Georgia Institute of Technology.
Riesbeck, C., and Schank, R. 1992. Inside Case-based
Reasoning. Lawrence Erlbaum Associates.
Roderman, S., and Tsatsoulis, C. 1993. A case-based
system to aid novice designers. A1 EDAM7(2):125133.
Rougegrez-Loriette, S. 1994. Prediction de Processus ?z partir de Comporlements observds: Le syst~me
REBECAS.Ph.D. Dissertation, Universit6 Paris 6.
Schank, R.; Kass, A.; and Riesbeck, C. 1994. Inside Case-based Explanation. Lawrence Erlbaum Associates.
Simoudis, E. 1992. Using case-based reasoning for
customer technical support. IEEE Expert 7(5):7-13.
Simpson, R. 1985. A computer model of case-based
reasoning in problem solving: An investigation in the
domain of dispute mediation. Technical Report GITICS-85/18, School of Information and Computer Science, Georgia Institute of Technology.
Skalak, D., and Rissland, E. 1992. Arguments and
cases: An inevitable intertwining. Artificial Intelligence and Law 1:3-44.
Slattery, S. 1993. Case-based reasoning, the derivational analogy approach. Technical report, Computer
Science Department, Trinity College Dublin. B.A.
Project.
Smyth, B., and Cunninghana, P. 1992. Deja vu: A
hierarchical case-based reasoning system for software
design. In Proceedings of the lOth European Conference on Artificial Intelligence.
Stanfill,
C., and Waltz, D. 1986. Toward memorybased reasoning.
Communications of the ACM
29(12):1213-1228.
Sycara, E., and Navinchandra, D. 1991. Influences:
A thematic abstraction for creative use of multiple
cases. In Bareiss, E., ed., Proceedings of the CaseBased Reasoning Workshop. 133-144.
Sycara, E. 1987. Resolving Adversarial Conflicts:
An Approach to Integrating Case-Based and Analytic
Methods. Ph.D. Dissertation, School of Information
and Computer Science, Georgia Institute of Technology.
Tsatsoulis, C., and Kashyap, R. 1993. Case-based reasoning and learning in manufacturing with the toltec
planner. IEEE Transactions on Systems, Man and
Cybernetics 23(4):1010-1022.
Veloso, M. 1992. Learning by Analogical Reasoning in
General Problem Solving. Ph.D. Dissertation, School
of Computer Science. Carnegie Mellon University.
~,¥ang, J., and Howard, H. 1991. A design-dependent
approach to integrated structural design. In Gero, J.,
ed., Artificial Intelligence in Design. Boston: Kluwer
Academic Publishers.