From: AAAI Technical Report SS-93-07. Compilation copyright © 1993, AAAI (www.aaai.org). All rights reserved. AnIntelligent Interface to a DatabaseSystem* Alfredo Fernandez-Valmayor (*) Carlos Villarubia (**) Manuel Buenaga (*) Dep. Inform:~tica - Fac. Fisicas - UniversidadComplutensede Madrid 28040 Madrid, Spain. valmayor@fis.ucm.es - mbuenaga@dia.ucm.es (**) Dep. Informfitica - E.U. Inforrmttica - Universidadde Castilla-La Mancha 13004Ciudad Real, Spain. carlos@eui.uucp Abstract In this work, we describe the architecture of an intelligent interface that improvesthe effectiveness of full text retrieval methods through the semantic interpretation of user’s queries in natural language (NL). This interface comprises a user-expert module that integrates a dynamic model of human memory with a NL parser. This paper concentrates on the problem of the elaboration of index patterns out of specific cases or it/stances. The structure of the dynamic memoryof eases and parsing techniques are also discussed. IntrodUction. Manyresearchers have focused on the study of human memoryprocesses to find a modelto improve the way information is stored and retrieved from computers (Foltz, 1991). They believe that better understanding of how human memory indexes and searches information would facilitate the developmentof new and more effective methods of building databases. Our approach deviates from this. Wetry to use this better understanding of how human memoryworks, not to build newand moreefficient databases, but to build tools that will make possibly to use more effectively pre-existent ones. The basis for this improvementis the construction of an intelligent interface. An interface that integrates a model of humanmemorywith a natural language (NL) parser. In somedatabase systems (DBS)the user, if he is not an experienced one, can choose to interact with the * Acknowledgment: This work is supported in part by CICYTgrant TIC91-0217-C02-02: "Arquitectura para Interfaces en LenguajeNatural con Modeladode Usuario" and CICYTgrant TIC92-9958: "Metodologiade disefio de herramientas de desarrollo para sistemas expertos basados en casts" 45 computer through a humanintermediary, which helps him to find the information he wants. Interacting with a humanexpert has the additional advantage that the user does not need to learn any formal language; however, even with humanintermediaries information retrieval of textual databases may have a very poor performance. As an example, Blair and Maron(1985) report on a precision index of 80%and a recall index of less than 20%in a legal information retrieval system. Several researchers have discussed the possibility of integrating an expert module in a DBSto automate the role of the humanexpert (Borgman,Belkin, Croft, Lesk, & Landauer, 1988). This has been our approach. The interface we present here would eliminate in most cases the need for this human expert. In this interface a user-expert module improves the effectiveness of the retrieval proces s through the semantic interpretation of user’s naturallanguage query formulations. For that reason, NL processing capabilities are at the core of the system. In the user-expert moduleNLprocessing capabilities are founded on memorybased parsing techniques; an anticipated improvement of conceptual parsers 01iesbeck, 1986; Riesbeck & Schank, 1989). In this paper, we begin with an analysis of the process of information retrieval. This will be followed by an overview of our system and a discussion of the main issues motivated by the development of the prototype: the knowledge representation and knowledgeacquisition techniques used to modelexpertise; the structure and indexing of cases in the memorymodel and finally the memory based parsing techniques based on this model. The system we have had in mind when designing our prototype is a full-text retrieval system. A system designed to search a file of unstructured, or partially unstructured, textual information (FerngndezValmayor& Villarrubia, 1991). Information retrieval (IR) fromthis kind of systemis to someextent more complex thanretrieval fromstructureddatabases,since retrieval cues can be matchedby manywordsin the documents. Avariety of methodsexist to recover information from textual databases (Salton & McGilI, 1983). Booleanexpressions of terms used in the documents are the most simple approach. Other more sophisticated methodsassociate vectors of weighted terms with documents(Salton, 1989). In these later methods,documentsare represented as vectors with weighted components. User’s queries are also representedas vectors in the samespace, therefore distance betweenvectors representing queries, and documentscan be computed.The documentsretrieved wouldbe those with a distance less than a threshold quantityas specifiedbythe user. The effectiveness of a retrieval system can be evaluatedin termsof the so called recall andprecision indexes. Recall index measuresthe proportion of relevant documentsout of all relevant documentsin the database, while precision index measuresthe proportionof the recovereddocuments that are found to be relevantto the user’s needs.In practice, recall andprecisionindexestend to vary inversely;i.e., very specific queryformulationsretrieve fewnon-relevant items but also relatively few relevant ones. Query formulationscan be altered to reachthe desiredrecall and precision levels, usingrecall-enhancingdevices; e.g., term truncation, and precision-enhancing devices;e.g., termweighting(Salton, 1986). System overview The system we are working on is an interface to existing databases. Accordingly,in designing our system, we mainly deal with the the strategic elaborationandverification stages of the information retrieval process. In the first stage, the user can express his information needs in natural language. The user-expert modulewill translate these input queries into a network of conceptual dependency structures (CDs)that later it will use to generate optimizedretrieval cues andto makethe mapping into the specific querylanguageof the accesseddatabase. Oncethe user has queriedthe systemandretrieved the information, he can inform to the system on the appropriatenessof the retrieveddocuments. Thetwo maincomponentsof the user-expert module are a dynamicknowledgebase and a NLparser. We conceive this knowledgebase as a modelof human memorythat incorporates knowledge about the database’sdomainand aboutthe contextof individual users. The way in which the user-expert module lays betweenthe users and the DBSis depicted in the diagram. Fromthe diagram, wecan see that if the user is an expert he can interact directly with the DBS.If the user is not an expert, then he can express his questions in natural language.TheNLinterface parses the user querybuildinga conceptualdependency 46 structure (CD)that capturesthe meaning of the input. Parsing of user’s input is based on prior memory structures. In the knowledgebase, the long term memory has at its top level the dictionary used by the conceptual parser (Dyer, 1983; Birnbaum Selfridge, 1981). Theshort term memory,also in the knowledgebase, is a list of CDstructures under construction that represents the user-computer interaction duringthe session. TheCDstructures in the short term memory represent both the meaningof the input alreadyseen, andthe expectationsthat will guide the parser to translate next input and to optimizethe queries. Beforeit can be used, the systemneedsto be trained. Thedifference betweentraining and using the system is in the way humanscommunicatewith it. To be sure that initially the system makesthe correct connections betweenwordsand concepts, the system manager,or instructor, will give the system the appropriated information in the form of CD structures. Thesestructures will be incorporatedinto the long term memoryshaping up the initial dictionary and relations. At any time, if the system managerdetects that the systemis performingpoorly, he can give the system additional information to reinforce proper CDstructures and the connections betweenthem.In this way,the correct links in the long term memorystructure can be partially controlled. Knowledgerepresentation and knowledge acquisition. Thefirst issue whendesigninga knowledge base is to specify the formalism in which the part of the universe that is to be representedin the knowledge base is to be described. ConceptualDependency (CD) structures are the basic formalismwehaveselected to represent information in our model. The key idea behindCDtheory is that meaningcan be represented in a canonical, language-freemanner(Schank,1972). In our implementation a CDis composed of an atomicheadconceptanda list of attribute-value (or role-filler) pairs wherevaluesare againCDstructures. Complexevents, concepts, or objects, can be representedby meansof a networkof CDstructures, connectedbydifferent types of links, suchas, causal, temporalor intentional (Pazzani,1988). For example, we can connect a CD structure representingthe chemicalelementmercuryas a heavy metal used on someindustrial processes, with other CDstructure representingthe environmentalproblems causedby the uncontrolleddisposal of heavymetals. In this manner, the user-expert module can hypothesize that a search of information about mercury, must include a search of documentsabout contaminationcausedby heavymetals. In the knowledge base, general concepts(or patterns) are representedas a packed-CD structure (as opposed to a networkof CDstructures). These structures represent high order memory conceptualizations and USERS ] I N.L.INTERFACE KNOWLEDGE BASE MAPPINGFUNCTIONTO SPECIFIC QUERYLANGUAGES QUERY LANGUAGE DATA BASE SYSTEM are generally knownas MemoryOrganization Packets (MOP). MOPsare organized as a hierarchical and modifiable network that index individual cases or items of information (Schank, 1982; Kolodner1983). Other characteristic of our systemis that most of the knowledge in it must be acquired using its own mechanismsand not pre-stored or hand-tailored. That means that the system considers all external information as cases (instances with no variables) and that the system itself must generate using its own learning mechanismsall the patterns that index those casgs. Inductive learning is the basic mechanismthe system uses to create these new patterns. To learn new patterns, our prototype uses the empiricist algorithm making an input pattern more general or more specific, in response to feedback about howwell that pattern explains or classifies an instance. To makea pattern more specific, our system indexes under the same pattern (MOP-father),all the instances or cases that are compatible with this MOP-father. A new MOPis created from these events when someof them share one or morefeatures that are not present in the MOP father. This new MOPis indexed as a specialization of the MOP-father,and the events, used to create it, are indexed under it (Kolodner, 1983; Lebowitz, 1983). 47 To generalize its input, our program starts by replacing constants by variables (Fernandez-Valmayor & Fernandez Chamizo, 1992). SBL-vars (for Similarity Based Learning) are used to generalize input instances. These variables are created by the system when two CDstructures have a commonslot with values that are different, but with somepossible commoncharacteristics. For example: The two CDsbelow are two possible representations of the chemical elements lead and mercury. (Following the usual convention we represent CD’s head concepts in capital letters and attribute’s names in non-capital letters; PP stands for picture producer following Shank’s nomenclature and SBL-variables are written with namesstarting with char "/,"). (PP (PP type is-a INANIMATE-OBJECT CHEMICAL-ELEMENT name MERCURY) type is-a INANIMATE-OBJECT CHEMICAL-ELEMENT symbol group density Hg METAL HIGH symbol group density Pb METAL HIGH name LEAD) Using the generalization process described above our systemwill obtainthe followingMOP-pattern: (PP t y pe INANIMATE-OBJECT is-a symbol /,SYMBOL /~ IS-A METAL group density HIGH name /,NAME) In this pattern, the SBL-varLNAME can matchany valuein the slot ’name’,but the SBL-var LIS-Aonly will matchvalues in the slot ’is-a’ withany head conceptandattribute value pairs ’group-METAL’ and ’density-HIGH’. Thus,the systemcanuse this pattern to indexinformation aboutall highdensitymetals. Tocreatenewpatterns,our systemuses anothertype of variables; EBL-vars(for Explanation Based Learning).Thesevariables represent not merely conjunctive generalization of severalinstancesbutthe structural relation betweenthe componentsof a unique instance. For example: The CDbelow describesa possible episodeof water-contamination causedby the disposalof mercury waste (ACTION type actor object from located-in PTRANS CHEMICAL-WASTE-LTD. MERCURY FACTORY FORESTLAND manufacturer-of BATTERIES owned-by CHEMICAL-WASTE-LTD. ALMADEN to C4:IOJND located-in date 1-5-8g result CONTAMINATION type WATERCONTAMINATION agent MERCURY) TheEBL processwill obtainthe followingpattern(in this pattern EBL-vars are markedwith the symbol ,,?,,): (ACTION type PTRANS ?ACTOR actor object ,"K)6JECT FACTORY located-in f r om FORESTLAND manufacturer-of BATTERIES owned-by ?ACTOR located-in ALMADEN to date 1-5-89 result CONTAMINATION WATER-CONTAMINATION type .’?OBJECT) agent causes contaminationcomesfrom. Specific domain knowledge is includedin this processto deal with attributes (or roles) with special meaningin the domain. Finally,oursystemhasprocessesto changeits ’focus of attention’ or ’view point’ about a given information. In the examplesabove we have used to represent mercurythe CD: (PP type INANIMATE-OBJECT is-a CHEMICAL-ELEMENT symbol group density name MERCURY) Where the "focusof attention"is the CD-head concept ’PP’. Changing the "focus of attention" to ’MERCURY’, the systemobtains the following CD: (MERCURYname-of PP type NANIMATE-OBJECT is-a CHEMICAL-ELEMENT symbol Hg group METAL density HIGH) This CDrepresents the sameinformationthat the formerbut now,the focus is not a generic object (Picture Producer); the focus is MERCURY. This processof changingthe "focusof attention"of a CD hasthree important consequences: First, it makespossibly to matchCDsthat in other case will not be comparable; e.g., it makespossibly to match the CDabove ’with the CD(MERCURY <empty list of attribute-valuepairs>)that appearin the slots of the CDthat describes the watercontamination case andas a consequence to inheritthe attribute-value pairsof the former. Second,these new’viewpoints’ can be re-indexedin long termmemory makingnewentries at the concept dictionaryat the root of the memory. For example, changingthe ’focus of attention’ of the CDthat describesmercury as a PPwill createbetweenothers the followingnewentries: (MERCURY name-of Withthis patternthe systemcapturesthe idea that the actorof the action,that resultin watercontamination, is the ownerof the factoryfromwhichthe objectthat 48 Pig METAL HIGH PP) (Hg symbol-of CHEMICAL-ELEMENT) (METAL group-of CHEMICAL-ELEMENT) In this manner,the dictionary at the root of the memory will compriseall the termsusedin the CDs givento the system. Finally, whenit is combinedwith the process of variabilization, changingthe ’focus of attention’ resultsin patternsthatexpressdifferentabstractions or conceptualizations (although applied to the same information). For example, if in the watercontamination case we change the attention from ACTION to CONTAMINATION we obtain the followingCDfor the sameinformation: (CONTAMINATION result-of PTRANS ACTION t yp e CHEMICAL-WASTE-LTD. actor obiect MERCURY from FACTORY located-in FORESTLAND manufacturer-of BATTERIES owned-by CHEMICAL-WASTE-LTD. to located-in ALMADEN date 1-5-89 type WATER-CONTAMINATION agent MERCURY) WhenEBLand SBLprocesses are applied to this CD the followingpattern can be generated: (CONTAMINATION result-of ACTION t y pe date type agent PTRANS actor /,ACTOR object ?AGENT from ~,~ manufacturer-of BATTERIES to located-in ALMADEN /,DATE WATER-CONTAMINATION ?AGENT) This pattern expresses the abstraction that the contamination is causedby an agentthat is the object translated from places wherebatteries are madeto ground located in Almaden(Fernandez-Valmayor FernandezChamizo,1992) User-expert module implementation The NC prototype Thecurrent implementation of the user-expert module is based on NC, a prototype written in Allegro Common Lisp. This prototype was first developedto test memory organization and learning algorithms (Fernandez-Valmayor, 1990). In NC, learning algorithmsare used to create the abstractions that makethe discrimination network that indexes all input information. Our systemuses NCto index, by meansof a networkof patterns, the training cases and the casesthat result of translatinguser’s queriesinto CDsstructures. Our working hypothesis is that some basic but essential linguistic aspects of humanmemory can be modeledusing twosimple guide-lines: 49 ¯ Memoryis a schematic process, in which new informationis integrated into already established structureof patterns(Barlett, 1932). ¯ There is a small set of processes responsible for maintaining this structure. Memorystructure and organization Froma structural perspectivememory is divided into long term memoryand short term memory.The long termmemory is a sequenceof trees, andthe list of the roots of these trees constitutesits root. Asthe system incorporates newstructures to long-term memory, this list is reorganizedaddingto it the CDheadvalues that still are not in it. Theprocessof changingthe ’viewpoint’ also addsnewentries to memory’s root. Thenodesof the trees are the patterns that the system creates from input CDstructures. The links that connectthe trees’ nodeshavea label that expressesthe difference betweena child-nodeandits father anda frequencyvalue; this frequencyvalue is augmented each time the nodeis successfullyaccessedduring a searchoperation. Theleaves of the trees are the input CDstructures, these CDstructures can be the outcomeof the translation of the user’s input, or the CDstructuresby the systemmanagerwhentraining the system. Short term memory is the sequenceof CDstructures that result fromthe translation of user input during the session. At the end of the session the short term memoryis subsumedinto the long term memory. Unlike the short term memory,which experiences rapid and shifting changes, the long term memory, thoughupdatedat the end of each session, is a more stable representation of the domainand user’s peculiarities. Memory based parsing. The informationretrieval problemis deeply related with the problemof natural languageunderstanding. In the past years researchersin the IR field havebeen arguing that to build the next generation of informationretrieval systems, weneed systemsable to extract the semantic information from both documents anduser’s queries, and that the matchingof queries and documentsmustbe doneat this semantic level (VanRijsbergen,1987). At the sametimeover the past years researchersin the NLarea havedevelopeda variety of techniques for natural languageanalysis (Gazdar & Melish, 1989; Schank & Riesbeck, 1981; Riesbeck & Schank, 1989).Underlying these techniquesthere are different approaches to NLprocessing: declarative versus procedural, or semantic versus pure syntactically drivenparsing. However,all these approacheshavein common that they are basedon a search processplus a pattern matchinganalysis. Whatis different is the data structure they searchandthe kindof patternsthey try to match. Thus in memorybased parsing, understanding is consideredas the processof searching memoryfor related language structures (Schank, SHORTTERM MEORY LONG TERM MEMORY ~r=5¢[~ fr=6 I’~ fr=9 ~ ........ _~1~ I~ .~ .... frecue;~T--A~Lr of times the node is succesfully accessed gradedlist of expectations(alternative patterns) for the present input word. With this purpose, we define two functions, the ’weight’ function and the ’activation’ function. The parsing algorithm To develop a parsing algorithm based on our memory modelwe have to take into account three different but deeplyrelated issues: ¯ Howtraditional parsing algorithms behave and what are the principles uponthey operate. ¯ Howto infer from examples the structure of the language. ¯ Howto deal with the inherent ambiguity of natural language. 1980, 1982; Martin, 1989). Memorybased parsing can be also considered as a continuation of the work initially developedin conceptual analyzers and script appliers. In conceptual analysis, words are linked to conceptualizations in a dictionary. These conceptualizations are structures that represent the meaning of the word and they have associated procedural actions knownas ’demons’ or ’requests’. Conceptualizations not only have semantic meaning but also precise syntactic knowledge;for examplein the conceptualization associated with the word"say" (MTRANS actor object from X X X to X) First issue. It is claimed (Tomita, 1986) that the syntax of natural languagecan be realized as a context free grammar. Moreover, semantic grammarsare also context-free phrase structure grammars in which semantics and syntaxis is encoded in form of productions. Many general context-free parsing algorithms, able to handle arbitrary context-free grammars, have been formulated. Amongthem, the most well-knownis Earley’s algorithm. All practical algorithms have some similarity with Earley’s algorithm. General parsing algorithms are like Earley’s algorithm, in that they all construct wellformedsub-string tables (Tomita, 1986). At any point in its execution the parser can avoid to make redundant analysis by looking for phrases already recognized in its table of well-formed sub-strings. Whenall the phrases of a given type are stored in the we will have in the slot ’actor’ the demon:"expect to have already hear of the humanwho is performing this act", and in the slot TO’the demon:"expect that the humanreceiving the messagewill be the object of the preposition ’to’ " (Dyer, 1983). In our approach, we try to use a more integrated and less procedural control structure than the requestsstack structure used in McELI(Birnbaum& Selfridge, 1981) or the working memoryused in DYPAR (Dyer, 1983). In our system, the system’s dictionary is the root of the long term memory,the list of trees’ roots; and wordsin the input text are the primary cues to the information stored under them. The strength of associations betweenwords and the information stored under its corresponding entry are different and from them the parser builds the short term memoryas a 50 table there is no longerneedto use the grammar. We conceive our long term memory as a discrimination networkof patterns equivalent to a table of well-formed sub-strings.Thesepatternsplay a role comparableto that of the "items’ list" that constitute the table in Earley’s algorithm (Aho &Ullman, 1977). In our case, ’items’ are the patterns indexedundereach root’s memory entry, andthe long term memoryis used by the parsing algorithm in a waysimilar to that of ’traditional’ parsingalgorithms, that is weuse themto alternate top-downprediction with bottom-up production reduction (Pereira Shieber, 1987). Secondissue. The problemof learning a language froma training set of samplesentencesis the problem of grammaticalinference (Angluin& Smith, 1983). In our system initial learning happens whenthe instructor presents the systeman initial set of CD structures. ACDis a positive instanceto be learned. ACDis a tree structure (with labeled branches)that weconsideras equivalentto the derivationtree of the input sentence. Thus, the system must learn the languagefrom structural presentation, a technique used by pattern recognition researchers to aid grammatical inference. As we have seen above, parsing algorithms only need to use the grammarto build the well-formed sub-string table, so our problem is not to infer the grammar but to built the equivalent of the well-formedsub-string table. Thatis whatthe implementedlearning mechanisms do. Third issue. Usually natural languagegrammarsare ambiguous;a sentence can have multiple parses. In the case wherethere exists two or morepossible parses, it is not acceptablefor a practical natural languageparser to produceonly onearbitrary parse. All possible parses should be producedand stored somewhere for later disambiguation.Wheninferring a grammarwe have another source of ambiguity: the samelanguagemaybe generatedby several different grammars.Thus, the question ’which grammarwill the systemlearn?’ arise. Aswehaveseen whatour systemdoes is to build the equivalentof the parsinglist of itemsneededto parse the input sentence.So if there are differentunderlying grammarsin the training CDsinstances the wellformedsub-string table will correspondwith more than one grammar.Therefore, whensearching the memoryit will be founded expectations that correspondnot only to different parses in the same grammar but to parses in different grammars. Whatfollows is an outline of howthe parsing of an input proceeds: 1- Systeminitializes short term memory 2- It searchesfor the first wordin the input in the operational memory.It makesactive the patterns underit by putting themin the short term memory. 3- Theactivated patterns generateexpectationsabout wordsthat wouldbe seen next in the input. These 51 expectations will be tested against the patterns activated by the followingwordinput. Activepatterns in short term memory not compatibles with followingwordsin the input are discharged. 4- If it doesn’tfind the word,it leavesa holefor that word in the short term memoryand proceeds with next input word. Conclusions and future work. In this paper we have seen that a humanmemory model is not a good candidate to replace actual databasemodels. Its potential lies in its merit to modela humanexpert that helps the user to formulate better queries. Natural languageprocessingcapabilities are at the core of the systemdescribed here. Thefocus of our paper has been to develop further memorybased parsing techniques. If wecomparethe techniqueswe use in our system with previous work, wecan say that what wehave madeis to substitute the static dictionary (whichguidesthe parsing) and muchof its hand-tailored procedural knowledgeby a dynamic structureanda set of genericprocesses. Learningpatterns of well-formedsub-strings from positive exampleshas been the other focal point of our paper. The representation schemeand learning mechanisms describedhere can be consideredclose to those of Winston’slearning blocksworldconcepts,as they are describedin Michalski,Carbonell&Micheli (1983). TheCDnetworkused in our system and the semantic networkused by Winstonare similar, but in our systemthe networkof conceptsandrelations is morehomogeneous and it is processedin a less handcrafted way.Thegeneral process that in our system changes the ’point of view’ over a CDcan be consideredrelated to the negative-satellitelinks used by Wiston,but in our case this is not a particular information introduced by the user, but a general process that expands the learning power of the system. Other question that has been arisen is the need to comparethe representational powerof CDsstructures with logic formulas. External links betweenCDsand slot namescan be thought as two-argument predicates but in our system wehave also incorporated logic operators(’or’, ’and’,’not’) as links that are processed in a special mannerby functionslike the functionthat changesthe focusof attention. Textualanalysis of accurately retrieved information can be the next goal to improvethe context in which user questions and commentsmust be interpreted. Evaluation of text-analysis technologies has demonstrated that text-analysis techniques, incorporating natural language-processing,are more effective than traditional information-retrieval techniquesbasedon statistical classification when applicationsrequire structured representationsof the information present in texts (Lehnert & Sundheim, 1991) LanguageText. Cognitive Science vol 7, pp.l-40. Michalsky, R.S., Carbonell, J.G., Mitchell, T.M. (1983). MachineLearning:an Artificial Intelligence Approach. Morgan Kaufmann. Martin,C.E.(1989). Case-basod Parsing in Riesbeck, C.K. &Schank, R.C. Inside Case-BasedReasoning. Hillsdale, NJ. LawrenceEflbaum Associates. Pazzani, M.,(1988). Learning Causal Relationships: AnIntegration of Empirical and Explanation-Based learning Methods. PhD. Dissertation, Technical Report UCLA-A/-88-10. Pereira F.C.N. & Shieber S.M. (1987). Prolog and Natural-LanguageAnalysis. CSLIlecture notes; no.10 Riesbeck, C.K., (1986). FromConceptual analyzer Direct MemoryAccess Parsing: An Overview. In Advancesin Cognitive Science, Sharkey N.E. (Ed.). John Wiley & Sons. References Aho, A.V. & Uliman, J.D. (1977). Principles of Compiler Design. Addison Wesley. Angluin, D. &Smith, C.H. (1983). Inductive Inference: Theory and Methods, ComputingSurveys, Vol. 15,No.3, pp. 237-269. Barlett, F.C., (1932). Remembering.A Study in Experimental and Social Psychology. Cambridge University Press. (2a. reimpresi6n, 1977). Bimbaum,L. & Selfridge, M.,(1981). Conceptual analysis of natural language. En Schank, R.C. & Riesbcck C.K. ,(Ed.),1981. Inside Computer Understanding, LawrenceErlbaumAssociates, Inc., NewJersey. Borgman,C.L., Belkin, N.J., Croft, W.B., Lesk, M.E., Landauer, T.K. (1988). Retrieval systems for the information seeker: Canthe role of the intermexliaty be automated?Panel Session Human Factors in Computing Systems, CHI’ 88 Conference Proceedings (Washington, D.C.), ACM,NewYork, May 1988, 51-53. Riesbeck, C.K. & Schank, R.C., (1989). Inside CaseBased Reasoning. Hillsdale, NJ. Lawrence Erlbaum Associates. Salton, G. (1986) Anotherlook at automatic textrea’ieval systems. CommunicationsofACM,Vol 29, No.7, 648-656. Salton, G. (1989) Automatic Text Processing: The Transformation, Analysis, and Retrieval of Information by Computer. Reading, MA:AddisonWesley. Blair, D.C., & Maron,M.E. (1985). An evaluation retrieval effectivenessfor a full-text document retrieval system. Communications of ACM,28, 289-299. Dyer, M.,(1983). In Depth Understanding. MITPress. Fern~mdez-Valmayor, A. (1990). Diseflo de una Base de Conocimientos Dimimicay su Aplicacidn en un EntomoEducativo. Doctoral thesis. Universidad Complutense Madrid. Ferruindez-Valmayor, A. & Fern~dez Chamizo, C. (1992). Educationaland ResearchUtilization of Dynamic KnowledgeBase. Computers & Education, Vol.18, No. 1-3, pp. 51-61. Fern~dez-Valmayor,A. & Villarrubia, C. (1991). Propuesta de Sistema Unificado de Consulta de Bases de Datos documentalespara el Instituto de la Mediana y Pequefla EmpresaIndustrial, IMPI. Inform Ttcnico. Departamentode !nform~tica, Universidadde CastillaLa Mancha,Ciudad Real. Foltz, P.W. (1991). Models of HumanMemoryand ComputerInformation Retrieval: Similar Approaches to Similar Problems. ICS Tech Report 91-03. University of Colorado. Gazdar, G. & Mellish, C., (1989). Natural Language Processing in Lisp. Addison-Wesley. KolodnerJ. L.,(1983). MaintainingOrganization in Dynamic Long-Term Memory, Cognitive Science, Vol.7, pp.243-280. KoiodnerJ.L., (1983). "Reconstructive Memory: ComputerModel". Cognitive Science, Vol.7, pp.281328. l.,¢hnert, W. &Sundheim,B. (1991). A Performance Evaluation of Text-Analysis Technologies. A/ Magazine, Fall 1991, pp. 81-94. Salton, G., &McGill, M. J. (1983). Introduction to ModernInformation Retrieval. McGraw-Hill:New York. Schank, R.C., (1982). Dynamic Memory:A theory of remindingand learning in computersand people. CambridgeUniversity Press. Schank, R.C., (1980). Languageand Memory. Cognitive Science, 4, 243-284 Schank, R.C., (1972). "Conceptual Dependency: Theory of Natural LanguageUnderstanding", Cognitive Psychology. 1972, 552-631. Schank, R.C. & Riesbeck C.K.. Ed. (1981). Inside Computer Understanding. Lawrence Erlbaum Associates, Inc., NewJersey. Tomita, M. (1986). Efficient Parsing for Natural Language:A Fast algorithm for Practical Systems. Kluwer AcademicPublishers Van Rijsbergen, C.J. (1987). A new theoretical framework for information retrieval. SIGIR FORUM Vol. 21, No 1-2 Autum86/Winter 87 Lebowitz, M., (1983). Generalization from Natural 52