From: AAAI Technical Report SS-96-02. Compilation copyright © 1996, AAAI (www.aaai.org). All rights reserved.
Between Help and Engineering: constraints on user task automation
Joost Breuker
University of Amsterdam
Social Science Informatics(SWI)
Roeterstraat15
1018 WBAmsterdam,the Netherlands
breuker@swi.psy.uva,nl
Abstract
In this paper we describe two projects relevant
for automating user tasks. The first one, EUROHELPwas aimed at tools and methods for developing Intelligent Help Systems (IHS). Experiences and accomplishmentsin this project shed
light on understanding and monitoring task performance of users. The major bottleneck appears the lack of understanding of the content
of user tasks, calling for research in ontologies
rather than trying to cometo grips with the commonsense semantics of the workplace. A major part of the paper describes the CommonKADS
projects in which a methodologyand tools were
developed for building KBS. CommonKADS
has
a long (> 10 years) and extensive history, and
it is arguedthat it has survivedseveral shifts in
attention in problems about knowledgeacquisition (KA),becauseit has focusedon well defined,
coherent specification languages. The need for
such a core is signalled for current approaches
in KAwhich provide operational components
((configurations of) mechanisms)to domain
perts to develop small, but practical knowledge
based systems.
1
Introduction
In this paper we discuss problems and limitations of automating user tasks based upon experiences in two large
scale research and development projects: EUROHELP
and
KADS.First we will give a brief outline of the (relevant)
aspects of these projects. Thesecondpart of this paper focuses on the limitations and problemswe have identified in
these projects with respect to (supporting) the automation
of user tasks. Weassumethat the issue is not whether we
can provide automationto users in general. Technologyand
tools, that range fromhigh performancepowertoolsto intelligent workmateshave been developed (e.g. in KADS
and
Radboud Winkels
University of Amsterdam
ComputerScience and Law
Kloveniersburgwal72
1012 CZAmsterdam,the Netherlands
emailwinkels@lri.j ur.uva.nl
in EUROHELP)
to enable task automation. However, the
question nowis rather whetherwe are able to provide tools
to users whowill be able to automate their owntasks either as the side effect of their recurring activities (e.g. via
user modeling, plan recognition and learning) or by simple constructive activities whichdo not require software or
knowledgeengineering skills. In short, we want a nonprogrammersprogrammer’s assistant. The experiences in
EUROHELP
pertain in particular to understanding tasks of
users while they are performingtasks with conventional applications such as word-processorsor mailing systems. In
discussing KADS
-- a knowledge engineering methodology -- we will focus on a library of problemsolving methods, and their indexing.
2
EUROHELP
EUROHELP
was concerned with tools for the development
of Intelligent Help Systems (IHS). 1 These IHSs provided
both active and passive help to users of (conventional)
applications such as word processors, operating systems,
mailing systems etc. Passive help meansthat the user can
ask questions to the IHSabout the application; active help
meansthat the IHS’looks over the shoulder’ of the user, interprets her actions and offers spontaneous help. Reasons
for offering help are errors, inefficiencies and opportunities to expandthe skills of the user. Figure 1 summarizes
the architecture of a EUROHELP
IHS. The monitor consists of two submodules:a planner and a plan recognizer.
The plan recognizerinterprets the user’s performance in
termsof local goalsor subtasks,while the plannergenerates the correct/efficient waysto achieve such goals, given
the current state of the user’s competence(user model).The
diagnoseris called in whenthe user’s performance deviates fromsuch plans, and traces these in terms of lacking or
confused knowledge. Lacking knowledgeis not restricted
to the proper use of the commands
of the application, or
its current state, but also to its working.Whena diagnosis
1EUROHELP
was an Esprit project (P280, 1985- 1991).
involved120 personyearand delivered tools and a methodology
for IHSconstruction[Breuker,1990]
r - -A-PPIJOA-TI-ON-I
~ e.g. email system
L.
r
-
I
I
..J
EUROHELP-IHS
~
question
~
terpreter )
~ monitor
7
plan
~_~
e~nsU
Is~ta~d
~recognizer y ~applic!tion
_
i-//1
~diagn0ser
1
)
,~’
I
model
~e~e
L_
Figure 1: Architecture of an EUROHELP
IHS
is found,the discourseplannerprovidesan explanation
[Winkels,1992].Thequestioninterpreterallowsthe user
to specifyhis problem
or question
via theuseof anapplication dependentmenustructure [Pilkington, 1992]. Because
the questioninterpreteris coupledto the monitor,questions
are understood
in termsof the current state of performance
andof the application. This is in a nutshell whata EUROHELP
IHSconsists of (see also [Breukerand Wielinga,
1987; Winkels,1992]). A completedescription of EUROHELP
is available in [Breuker,1990].
EUROHELP
is based upondetailed empirical studies of
noviceusers of conventionalapplications[Sandberget al.,
1988]. Thesestudies showedthat, whenmonitoredby humanexperts, users hardlytake the initiative to ask questions. Mostinteractions are the result of expert interventions. Themajorreasonis that the user is often not aware
of problems
or alternativeroutes, or is not capableof specifying his problem.Note, that in these experimentsfull
communicationis possible betweenuser and expert. A
secondimportantissue is that the focusis on task performance.However,the plans (methods)for executingthese
tasks easily breakdown,
becausethey often "encapsulateinteractions" [Simmons,
1992]. Makingor correcting plans
often involves understandinghowthe application works.
The distinction (and mapping)between’task knowledge’
and’insight in the application’is explicit in the domain
representation (application model).Therefore,wehavetaken
the fact that a user shouldknowhowan applicationworks,
or at least whatthe worldit operateson consistsof, is as
importantas to knowhowto handleit. In anyeducational
philosophy,skill andinsight are mutuallysupportive(e.g.
[Anderson
et al., 1990]).
Athird, related issue is the fact that task decompositions
are highly dependenton the worldin whichthe task is to
be executed.This seemsalso a trivial observation,but it
leads to mostof the trivial problems
of noviceusers. If one
doesnot havea goodinsight of the (side) effects of automatedactions, planningcan only be performed’by analogy’
to knownworlds. However,this leads to problematicdifferences or mismatches
as Fig 2 shows,becausethe digital
(mail) worldis different fromthe physicalone.
3 KADS
~ is a comprehensivemethodologyfor deCommonKADS
veloping KBS.A CommonKADS
project consists of the
development
of a suite of dependentmodels(see Fig. 3).
Twomodels, the Organization modeland the Task model
describethe operationalcontext of the KBS(s)(the "workplace" [Yostet al., 1994]).Thecombination
of Taskmodel
andAgentmodelcontainsthe specificationof the task distribution betweenthe agents involved(machine,human).
The Communication
modelspecifies the required communications to support this cooperation(see [Breukerand
de Greef, 1993]).
2KADS,
initiated by SWI,Universityof Amsterdam,
wasdeveloped
in variousEspritprojects,between
1984and1994.It took
morethan200personyears.
Some
of the majorpartnerswere:Cap
Gemini,Siemens,IBM,STC,Touche
Ross,LloydsRegister,ENTEL,SICS,Free Universityof Brussels.KADS,
or someadapted
formis usedbymanyindustriesin Europeandthe USA.
send(letter)
prepare
enveppe
write
letter
I\
write
address
write
sender
send (e-mail)
put in
envelope
put in
mailbox
write
address
write
subject
write
message
type
eof
put
stamp
Figure 2: Sendingmail: plans (task decompositions,methods)for e-mall (Unix-mall) and snail
namic) roles) between basic reasoning steps (inferences)
are specified. Sucha networkis called an inference structure. (2) Control can be addedto an inference structure, resuiting in a task structure. A task structure is a(n instantiAgent
ated) problemsolving method(PSM),i.e. its major structure is a task decomposition,bottomingout in inferences.
Exper[.is.e
Commun!caU~n Wewill not describe here CML
in detail (see e.g.[Schreiber
et al., 1994]). Different from most other approachesin KA
(e.g. SBF[Yost et al., 1994], EXPECT
[Swartout and Gil,
model
1995; Gil and Paris, 1994], KREST
[Geldof and Slodzian,
1994]) CML
specifications are not directly operational for
several reasons such as efficiency (of. the Design model)
Figure 3: Suite of models in a CommonKADS
project
and independence of specific implementation platforms.
However,CMLcan be translated into its formal language
(ML)2, i.e. FOPLaugmentedwith reflection and various
Twoother modelsprovide specifications of the KBSitother logics, whichis executable by very inefficient provself: the Expertise model and the Design model. The Exing [van Harmelen and Balder, 1992]. The role of 2(ML)
pertise modelcontains a ’knowledgelevel’ specification of
is rather in validation and consistencycheckingthan for opthe required knowledgeto build a knowledgebased system,
erationality.
analogousto a data modelin software engineering. The ExExpertise models are not necessarily to be built from
pertise modelis the input for the construction of the Descratch, in a bottom up way. The CK-Library a prosign model,i.e. the specification of the systemarchitecture
vides reusable componentsfor constructing expertise modthat implementsthe knowledgespecified in the Expertise
els [Breuker and de Velde, 1994]. This does not meanthat
model. This two step system specification philosophysepathe CK-Library
is to be used exclusively in projects that folrates considerations of understanding and modelingthe relow a CommonKADS
methodology. Most of the compoquired problemsolving competence(’expertise’) from effinents to be foundin the CK-Library
are in fact inspired by or
cient and well structured implementations.
copied from other approaches, and in particular from fundaThe Expertise modelcan be viewedas a conceptual intermentalresearch in Artificial Intelligence (e.g. on diagnosis
face between the "workplace"and the machine. This means or planning methods). Moreover, it can be shownthat the
CK-Libraryalso supports other approaches[Valente et al.,
that it has an inherent ambiguity.Onthe one hand, the terms
to be used should be easily identifiable and understandable;
1993].
on the other hand these terms should not only be well deThe role of such a library is not so muchthe reuse of
these components,but rather the application of these modfined, but also easily maponto machinemechanisms.Most
eling fragments. In reuse, the primary effect is efficiency
research efforts in KADS
have therefore been devoted to
the design of a "Conceptual Modeling Language" (CML) by copying or reinstantiation. However,the problemsolving componentsare relatively simple objects: only in their
to express Expertise models[Breuker and Wielinga, 1989;
combinationsand their domainspecific instantiations may
Wielingaet al., 1992;Schreiberet al., 1993].
turn out to be very complex. The major role of the CKIn CML,like in most other approachesin knowledgeacLibrary is the guidance of the modelingprocess, i.e. the
quisition (KA), domainknowledgeis distinguished from
task knowledge.In task knowledgethere are two views. (1)
aThefull nameis CommonKADS
Expertise modelingLibrary,
A knowledgeflow view, in which the dependencies ((dyor CKEL.
HereI will simplyuse the term CK-Library
I mooo’
r/ 70o,
Organization
I
m°?’I
~
Task |
I m,oo,, /
DesiQn
3
knowledgeacquisition. The problemin knowledgeacquisition is that it is very difficult for a (novice)knowledge
engineer to identify the types of tasks and knowledge
in the data
documentation, interviews, etc. -- of some domain. A
knowledgeengineer mayneed to knowwhether it is a diagnostic or an assessment task whichhas to be automised,
what problem solving methodsmay be applicable to this
task, what assumptionson the nature and structure of the
domainknowledgeare implied by a particular method, etc.
Therefore, the CK-Libraryis not a simple store, but rather
a frameworkfor reuse, that should provide guidanceto its
user. This guidanceis not so muchin separate, linked documentation,but it is moreor less wired in by the structure
and the access of the CK-Library.This structure and access guides the process by constraining the possible routes
through the library. Only those componentsare accessible
from another, selected componentwhichare compatible. In
traversing the CK-Library,local selections are made, and
the selected componentspreserve the relations which are
implied by the structure of the CK-Library.The selections
are madeon the basis of data on knowledgein the application domainthat do or don’t matchthe assumptions-- features mthat are associated with a component.4
The major access to the CK-Library is by a typology
of problemtypes, such as diagnosis, planning, assessment
[Breuker, 1994]. The typology is not a taxonomy, but a
"suite", which expresses the dependencies between these
types as they occur in (sub)tasks (see Fig. 4). Thesetypes
are characterizedby their solutions (see Table I). It is beyondthe scope of this paper to explain in detail the nature
of these problem types. This suite exhibits a numberof
properties which makeit attractive for KA.The types are
mutually exclusive and appear to cover the full range of
problems/tasksas identified in the literature and in a decade
’KADS
experiences’. It is based upon the notion put forward by [Clancey, 1985] that problemsolving consists of
the construction of"a modelof a system-in-the-world". Althoughartificial problemsolvers hardly ever construct a
case modelin an explicit way, 5 this is indeed what is implied by understanding a problem. Whenthe modelis already given as generic knowledgein the knowledgebase,
this simplifies the construction to the selection and instantiation of the applicable knowledge.In that case, the task
contains problemsat the analytic spectrum of the suite.
Otherwise, the case modelhas to be made, constrained by
requirements, and we will characterize the task as a synthetic one (design, planning). Therefore, an importantproperty of the suite is that it guidesthe decomposition
of tasks.
For instance, any PSMhas two functional components: a
part whichgenerates solutions and a part whichevaluates
or tests these solutions. Thetest part invariably followsthe
suite at the analytic side. The suite not only enables the
recognition of this paradigmin real life tasks, but also enables the construction of new task decompositions(PSM).
Not only the access, but also the bottom of the CKLibrary is defined by a typology. The inferences, whichare
the terminal functions of PSMs,are typed according to the
operations they perform with respect to a KL-ONE
like ontology (see Table 2). A formalization of this typology can
be foundin [Aben, 1994]. It should be noted that other ontological commitments
result in other typologies. However,
this typology has the advancethat the commitments
are relatively small and widelyacceptedas a basis for representing
knowledge,so that the scope for application is large. The
types are well differentiated, and basedon a systematiccovering of "what can happen to a concept", as comparedto
typing like deduction, abduction and induction ("what can
happento a set of propositions"). Moreover,the terms refer
to reasoning operations which are not context dependent,
as e.g. the teleological terms that are used to characterize,
for instance, task or problems.This is a nice property for
reusable building blocks. (See also [Wielingaet al., 1992]
and [Aben, 1994] for alternatives.) Theseinferences have
a fixed meaningbut not a fixed grain size level. They can
be further decomposed,if necessary. Theyare standard or
canonical terms, rather than primitives or atoms.
This concern in KADS
with language, typology and formalization showsthat apparent approach to knowledgeacquisition is in providingconceptual, knowledgelevel tools,
which on one hand easily map onto real life data, and
which on the other hand are well defined to allow computational rendering. Theseanalytic tools are supplementedby
reusable models and modeling componentsfor specifying
the problemsolving knowledgefor an application.
4
Coming to terms
The problemtypology of the CK-Libraryclaims that the dependencies between kinds of (well defined) problems are
more or less universal. Ananalysis of PSMsfor diagnosis reveals that subtasks solve a particular kind of problem,
4 Animplemented
version of the CK-Library
is commercially whoseresult is used to solve problemsin another (next) subtask [Benjamins, 1995]. For instance, in the test parts of
available as part of the KADS-Tool
workbench
by ILOG(France);
a diagnosis-PSMcontain prediction and monitoring prob[Breukerandde Velde,1994]providesa descriptionof the philosophy, structure and ingredients of the CK-Lib.The CK-Library lems. The predictions and observations are a necessary
is not ’complete’. Not only new(arrangementsof) PSMmay
input for a monitoring problem/subtask, which identifies
developed,but there is hardly any supportfor domainknowledge conflicts, etc. Thesedependenciesare most explicit in all
modeling.Theemerginginterest in reusableontologiesmayfill
modelbased diagnosis PSM.In association based (heuristhis gapin the nearfuture.
5Anotableexceptionis ABEUs
’patient specific model’[Patil,
tic) methodsthey are less clear, because of the compiled
1988]
out nature of associations between symptomsand (classes
synthesis modification
behavioural view (system*environment)
I
I
planning
I
assessment
,
/_
l
rt
"modeling
assignment --~ prediction
IL___
I
I
I
analysis
design
7-
monitoring
* diagnosis
structural view (system)
Figure 4: A suite of problem types, dependenciesand views
solution
behavioural model
structure of elements
sequence of actions
distribution/assignments
state of system
discrepant states
faulty element
class/grade attribution
Table 1: Type of problemand corresponding solutions
operation type
generate concept
change concept
differentiate
manipulate structure
Inference
instantiate
classify
generalize
abstract
specify
select
assign-value
compute
compare
match
assemble
decompose
transform
l,,..I
I
J
_.l
L_
major type type of problem
synthesis
modeling
design
planning/reconstruction
modification assignment (scheduling, configuration)
analysis
prediction
monitoring
diagnosis
assessment
"7
I
I
l
I
I
71
II
4.1
input --+ output arguments
concept --+ instance
instance ---, concept
set of instances --, concept
concept --+ concept
concept -, concept
set ---+ concept/instance
attribute --+ attribute-value
structure ---+ attribute-value
value, value ~ difference value
structure, structure --+ difference
set ~ part-of structure
part-of structure --+ set
structure ~ structure
Table 2: Typologyof inferences [Wielingaet. al., 1992]
of) malfunctions. The identification of (sub)tasks characterized by a type of problem from the suite can be obscured by these shortcuts, but another important reason is
that the terms by which these problems/subtasks are described are highly context dependent. Diagnosis maymean
such diverse things as a malfunctioningcomponent,a class
of malfunctionsor a sequenceof events that caused a problem (accident), whichrefer to different problemtypes: diagnosis, assessment, respectively reconstruction (planning
[Simmons,1992]).
are not knowledgeintensive and the intelligence required
for performingthese tasks is the planning of automatedactions (leaf sub-tasks). The planning problemsinvolved are
not too complex, so that EUROHELP
could construct executable scripts instead of instructing the user whatto do if in
trouble. However,such a "butlering" modewas not recommendedfor the simple reason that it wouldonly hide the application, and learning to talk to the butler and to understand
his capabilities wouldbe as difficult as learning to use the
application itself. Wemayhypothesize that the scope and
size of effective automationof a user task is proportional
to its intensity of knowledgeuse. In knowledgeextensive
tasks, whichdo not havea strongly recurring routines, a set
of simple procedures maybe more effective than encapsulating complexroutines (of. the flexibility of knowledge
acquisition toolboxes with simple mechanisms).
A second suggestion from EUROHELP
concerns the feasibility of automateduser-task automation. Specifications
were written to combineplan recognition, student modeling
and machine learning techniques. Recognized plans were
stored in the student modeland routine plans wouldbe inducedfrom repeated patterns. Theseroutine, stereotypical
plans could be used both for user-tailored recognition and
help, and for limited butlering. Implementation,however,
was abandoned, because experiments showedthat 1) plan
recognition had a very limited scope, becausethe intentions
of users were not available, nor could they be assumedand
(2) sequences of actions and their variations were largely
’content’ dependent. For instance, understanding what the
user of a text editor is up to mayrequire understandingwriting competence(e.g. spelling, style, grammar6 ) and (some
of) the topics involved. This domaindependencefor understanding the tasks that users performwith machines(and of
course, also withoutmachines)suggeststhat at least the terminology(ontology) of the domainshould be available and
explicitly represented before automateduser task induction
becomesfeasible.
If in the literature there is a terminologicalconfusionon
what the terms meanand refer to (see [Breuker, 1994]),
the world of users the terminologyhas becomea major bottleneck in the access and deploymentof ’knowledgelevel’
tools for automatingproblemsolving tasks. Initially, i.e. at
the end of the 80-ies, mostof these tools incorporated some
particular PSMfor a particular kind of problemand with a
particular set of implied requirementson the nature of the
domain knowledge e.g. Generic Tasks [Chandrasekaran,
1988], Role Limiting Methods[Marcus, 1988]. Although
domainexpert (users) werequite capable to fill these shells,
it appearedthat the scope was too limited, i.e. only for a
few problem-type/domain combinations the implied PSM
appearedto be suitable. A moregeneral solution was found
in assemblingsystems rather than in filling readily available structures. PSMcan be built from combinations of
basic, reusable subtasks: inferences (KADS),or mechanisms (SBF). Tools for building artificial problemsolvers,
like K_REST,EXPECT
and SBFcontain some library of
mechanismswhich can be accessed and configured by some
task description. As it appears that the terms used in these
task descriptions are different across domains,basic mechanisms are added, imd the access interface grows to complex libraries by itself (e.g. [Yostet al., 1994]). This pragmatic approachfacilitates easy reuse, but lacks a structuring
frameworkfor systematic indexing. There is no wayto assess whether, or to what extent, the mechanismsare functionally equivalent or the terms are distinctive. In KADS
References
exactly the opposite happened. Mucheffort has been put
in the grounding of terms and structuring of the CK-Lib,
[Aben, 1993] M. Aben. Formallyspecifying re-usable knowlbut no reusable operationalizations are available. KADS
edge modelcomponents.KAJ,5:119-141, 1993.
fits within the software engineeringtradition that puts the
[Aben, 1994] M. Aben. Canonical functions: CommonKADS
emphasison accurate specification (languages) rather than
inferences. In Joost Breukerand Walter Vande Velde, edeasy programmingof problem solving tasks. However,the
itors, CommonKADS
Library for Expertise Modelling, pages
CommonKADS
CMLand related typoiogies could well be
89-120. IOS-Press/Ohmsha,Amsterdam/Tokyo,
1994.
used as unifying descriptors of tasks and mechanisms(cf.
[Andersonetal., 1990] J.R. Anderson,C.E Boyle, A. Corbett,
[Linster and Musen, 1992]). The problem typology proand M.Lewis.Cognitivemodeling
and intelligent tutoring. Arvides a reference frameworkfor task descriptions; the tytificial Intelligence,12:7--49,1990.
pologyof inferences gives a terminologyfor the functional
(and formal) description of mechanisms[Aben, 1993].
8Onemayobject then that text editing is a knowledge
intensive
task. Indeedit is, but in automation
onlythe informationmanageThe EUROHELP
experiences provide insights at the
mentparts havebeentaken out. Fuller automationwouldindeed
other end of the user task spectrum. The tasks of users
require text generationcompetence,
but the core of this compeof conventional applications are not typical problemsolvtence is alreadyautomated
in humans:spelling andstyle checking
ing tasks, but rather information management
tasks. They
are nowoften integratedfunctions.
[Benjamins, 1995] Richard Benjamins. A suite in Din. In The
KnowledgeEngineering ForumConference, pages -, 1995.
[Breuker and de Greet’, 1993] Joost Breuker and Paul de Greef.
Modelling system-user cooperation. In GuusSchreiber, Bob
Wielinga, and Joost Breuker, editors, KADS:a PrincipledApproach to KnowledgeEngineering, pages 47 - 70. Academic
Press, London, 1993.
[Breuker and de Velde, 1994] Joost
Breuker
and Walter Vande Velde, editors. The CommonKADSLibrary:
reusable componentsfor artificial problemsolving. IOS-Press,
Amsterdam,Tokyo,1994. to appearjune 1994.
[Breuker and Wielinga, 1987] J.A. Breuker and B.J. Wielinga.
Useof modelsin the interpretation of verbal data. In A. Kidd,
editor, Knowledgeacquisition for Expert Systems; a practical
handbook,pages 12 - 30. PlenumPress, NewYork, 1987.
[Breuker and Wielinga, 1989] J. A. Breuker and B. J. Wielinga.
Modeldriven knowledgeacquisition. In E Guida and G. Tasso,
editors, Topics in the Design of Expert Systems, pages 265296, Amsterdam,1989. North Holland.
[Breuker, 1990] J.A. Breuker, editor. EUROHELP:
developing
Intelligent Help Systems. EC, Copenhagen, DM,1990. 401
PP.
[Breuker, 1994] Joost Breuker. Componentsof problem solving.
In Luc Steels, GuusSchreiber, and Walter van de Velde, editors, A Future for KnowledgeAcquisition: proceedings of the
EKAW-94, EuropeanKnowledgeAcquisition Workshop, pages
118 - 136, Berlin, 1994. Springer Veflag.
[Chandrasekaran, 1988] B. Chandrasekaran. Generic Tasks as
building blocks for knowledgebased systems: the diagnosis
and routine design examples. The KnowledgeEngineering Review, 3(3): 183 - 210, 1988.
[Clancey,1985]W.J. Ciancey.Heuristic classification. Artificial
Intelligence, 27:289-350, 1985.
[Geldof and Slodzian, 1994] Sabine
Geldof and Aurelien Slodzian. Fromverification to modelling
guidelines. In Luc Steels, GuusSchreiber, and Walter Vande
Velde, editors, A Future for KnowledgeAcquisition: Proceedings of the 8th EuropeanWorkshopon KnowledgeAcquisition,
pages 226--243, Berlin, 1994. Springer.
[Gil and Pads, 1994] Y. Gil and C. Pads. Towards methodindependent knowledge acquisition. KnowledgeAcquisition,
6:163-178, 1994.
[Linster and Musen, 1992] M. Linster and M. A. Musen.Use of
gADSto create a conceptual model of the ONCOCIN
task.
KnowledgeAcquisition,4(1), 1992. Specialissue: ’The KADS
approach to knowledgeengineering’.
[Marcus, 1988] Sandra Marcus, editor. Automating Knowledge
Acquisition for Expert Systems. Kluwer, ReadingMA,1988.
[Patii, 1988]R.S. Patil. Artificial intelligence techniquesfor diagnostic reasoning in medicine. In H.E. Shobe and AAAI,
editors, Exploring Artificial Intelligence: Survey Talks from
the National Conferenceson Artificial Intelligence, pages 347379. MorganKaufmann,San Mateo, California, 1988.
[Pilkington, 1992] R. Pilkington. Intelligent Help: communicating with knowledgebased systems. Oxford University Press,
1992.
[Sandberg et al., 1988] J.A.C. Sandberg, J.A. Breuker,
and R. Winkels. Research on help-systems: Empirical study
and modelconstruction. In Y. Kodratoff, editor, Proceedings
of the EuropeanConferenceon Artificial Intelligence, ECAI88, pages 106 - 111, London,1988. Pitman.
[Schreiber et al., 1993] GuusSchreiber, BobWielinga, and Joost
Breuker. KADS:a Principled Approach to KnowledgeEngineering. AcademicPress, London, 1993.
[Schreiber et al., 1994] Guus Schreiber, Bob Wielinga, Hans
Akkermans,Walter Vande Veide, and Anjo Anjewierden. Cml:
the CommonKADS
conceptual modelling language. In Luc
Steels, GuusSchreiber, and Walter Vande Velde, editors, A Future for KnowledgeAcquisition: Proceedingsof the 8th European WorkshoponKnowledgeAcquisition, pages 1-25, Berlin,
1994. Springer.
[Simmons,1992] R.G. Simmons.The role of associational and
causal reasoning in problemsolving. Artificial Intelligence,
53(2-3): 159-207, 1992.
[Swartout and Gil, 1995] Bill Swartout and Yolanda Gii. EXPECT:explicit representations for flexible acquisition. In Proceedings of the Ninth KnowledgeAcquisition for Knowledge
Based Systems Workshop, KA W95, 1995.
[Valente etal., 1993] Andre Valente, Joust Breuker, and Bert
Bredeweg. Integrating modelling approaches in the Common
gADSlibrary. In A. Sloman, D. Hogg, G. Humphreys,Allan
Ramsay,and D. Partridge, editors, Prospectsfor Artificial Intelligence, Proceedingsof the AISB-93, pages 121 - 130. lOS
Press, Amsterdam,1993.
[van Harmelen and Balder, 1992] Frank van Harmelen and John
Balder. ML2:a formal language for KADSmodels. Knowledge Acquisition, 4:127-161, 1992.
[Wielingaet al., 1992] B.J. Wielinga, A. Th. Schreiber, andJ. A.
Breuker. KADS:A modelling approach to knowledge engineering. KnowledgeAcquisition, 4(1):5-54, 1992. Special issue "The gADSapproach to knowledge engineering".
[Winkels, 1992] R. Winkels. Explorations in Intelligent Tutoring
and Help. lOS-Press, Amsterdam,Tokyo, 1992.
[Yostetal., 1994] Gregg Yost, Georg Klinker, Marc Linster,
David Marques, and John McDermott. The SBF framework,
1991- 1994: from applications to workplaces. In Luc Steels,
GuusSchreiber, and Walter Vande Velde, editors, A Future
for KnowledgeAcquisition: Proceedings of the 8th European
Workshopon KnowledgeAcquisition, pages 318-340, Berlin,
1994. Springer.