From: AAAI-83 Proceedings. Copyright ©1983, AAAI (www.aaai.org). All rights reserved.
RESEARCHER:
AN OVERVIEW1
Michael
Department
Computer
Science
Lebowitz
of Computer
Building,
New York,
University
NY 10027
(NAME:
<name-of-object
>
TYPE: unitnr or composite
if unitary
tar)
STRUCTURE!!: <shape-descri
<a list of rela f ion records>
if composite)
Abstract
this
paper
is a computer
system,
Described
in
RESEARCHER, beine: develoDed at Columbia that reads
natural languagk text-in th(? form of patent abstracts
and
creates.a permanent
long-term memory based on concepts
generaliz+
from
these
texts,
forming
an intelligent
mformatlon
s stem.
This pa er is intended
to give an
overview of RESEARCHER & e will describe briefly the
desi n
of
four
main
areas
dealt ’ with
in
the
w %ere a
RESEARCHER: 1) knowledge
representation
canonical
scheme for representing
physical
objects
has
2) memory-based
text processing,
3)
been developed,
generalization
and
generalization-based
as an E??g%
organization
that treats conce t formation
an a 4) generalization-based
of understanding,
part
question answering.
Figure 1:
Representation
Schema
ieces (composite).
The STRUCTURE
of two or more
eit % er a description
of the sha e of an
field contains
or a set of relation recor Bs, if it is
object, if it is unitar
S& e-descriptors
are
graphical
composite.
based
mostly
on visual
of o %jects
represe? t ations
Relation
records
generally
describe
binary
properties.
physical relations between parts of a complex object.
To date we have not explored shape descriptors
in great
ical
detail. We expect to have a system that uses prototy
shapes (in much the same way we use prototypical
o %‘ect
image- i ike
descriptions),
combining
declarative
and
as Kosslyn
and
representations
in much the same wa
Shwartz’s model [Kosslyn and Shwartz 1 71.
reater detail,
We have studied object relations in much
and have developed a canonical scheme for !I escribin
the
various ways that two objects can relate to each ot her.
This scheme is used to represent,
amono- other things, the
meanin s of words or phrases such as “above”,
“on top
that are used to describe physical
of” an cf “surrounding”
-2
relations.
Figure
2 shows the major
elements
used in relation
re resentation.
Various combinations
of values for the
fief ds shown in Figure 2 provide wide covera e of the
kinds of relations that objects can have with eat I! other.
1 Introduction
Natural language processin
and memory organization
are
logical components
of intel Pigent information
systems.
At
Columbia
developing
a computer
system
RESEAR6Hz&
%zt reads natural lan uage text in thg
form of patent abstracts
(disc drive pa F ents provide the
initial
domain)
and creates
a permanent
long-term
based on generalizations
that it makes from these
memor
texts. i n terms of task RESEARCHER
is similar to IPP
[Lebowitz
80; Lebowitz
8% a program
that read and
remembered
news stories.
e need o deal with complex
object representations
and descriptions
has introduced
a
whole new range of problems
not considered
for that
program.
2 Representation
Property
Theefirst problem to be worked on in any new domain in
AI 1s the design
of a scheme
to represent
relevant
concepts.. AI researchers
have not extensively
investi ated
Le %nert
representing
corn lex physical ob’ects (although
78; Kosslyn and 8 hwartz 771 an d others have a IIdressed
some of the issues we are concerned
with).
We have
system
with
the flavor
of
develo ed a frame-based
Schank 721, that deals
Ghan R‘s Conce tual Dependency
with ob’ects ins Pead of actions.
T 6 is scheme 1s described
and Lebowitz 821.
in detai i in [Wasserman
The basic frame-like structure
used to represent objects is
known +s a memette.
Memettes
are used as part of a
idiosyncratic
hlerarch!cal
set of prototypes
(generalized
descrtptlons
derived from specific instances j,. described
in
may be describing
a fairly
Section 4. A iven memette
ical disc drive) or a more
gene+
object ? e.g., a rotot
speclflc obJect \ a mo a el 19i;t-3 floppy disc drive).
In
somewhat
simp ified form,
the basic structure
of a
memette is shown in Figure 1.
The TYPE slot of a memette
indicates whether this is a
single indivisible structure
(unitary)
or a conglomeration
‘This research was supported
Advanced
Research
ProJects
N00039-82-C-0427.
Science
Columbia
Description
Word with
property
distance between objects
strength of contact
relative direction between
ob’ects
orien ta tio72 re r’
ative object orientation
e,nclosure
descri tion of full or
partia P enclosure
distance
contact
location
Figure 2:
Canonical
Relation
near
t,“b”o”vhe’ng
parallel
encircled
Fields
Certain
combinations
of relation
fields occur together
often enough that relations,
like objects and shapes, can
frequently
be
described
both
in
text
and
our
representations,
in terms ot DrototvDes.
The normal way
to-represent
an’ ob’ect is in therms 6f rotot pica1 relations
such as ON-TO+-OF
and SURRO?JND$$
that are in
t urn represented
canonically with the fields ih Figure 2.
scheme is used
As an exam le of how our representation
of a US patent
consider
E 2 1, taken from an abstract
about a computer disc drive.
in part by the Defense
Agency
under
contract
232
EX1
Combination
Enclosed
Disc
Filter Assembly
Drive
RESEARCIIER
is still
redictive
in nature.
However
since patents
are not Focused on events
as are news
redictions
of
IPP
(or
other
stories,
the action-based
e. . PBirnbaum
conce tual analyzers,
and Selfridge
811)
must g e extensively
modi s ied.
the predictions
used for understanding
in
S@fd~~ER
are based on the
h sical descriptions
built up, in much the same wa as I lY made
redictions
from events. The goal of RESEARCHER’s
un a erstanding
recess is to record in memory how a new object being
objects already known
8 escribed differs from generalized
(keeping
in mmd that
these
are idiosyncratic),
and
ultimately
to generalize new prototypes.
Processing
in RESEARCHER
concentrates
on words that
refer to physical
objects
in memory
and words that
describe
physical relations
between such objects.
Such
words are known as Memory Pointers (MPs) and Relation
Words . (RWs).
These
words guide RESEARCHER’s
gathered
rocessmg
m making use of any information
g ottom-up.
Conce tual analysis in this domain mvolves
careful processin
oP MP phrases (usually noun phrases:;;
identify
memet f es, modifications
to memettes,
repeated
mentions of memettes.
RWs are used to create
the relations between memettes described in section 2.
Particular
care in this domain has to be given to phrases
of the sort “X relation1 Y relation2 Z”. It is frequently
~;;~s~o,~ell
if relation2 relates Z to Y or X. So, m the
read/write
head above a disc connected
to a
cable”
structure
it is ‘not ap arent from the surface
whether
the disc or IThe read/write
head is connected
to
the cable.
Prepositional
phrase attachment
is a wellknown problem,
and is especial1
crucial in the patent
domain.
We have discovered tha f a set of heuristics that
maintains
a single
memette
in focus based
on the
memettes
and relations involved will solve most of these
problems
(although
in the lon run we ex ect to use, in
addition, a model of the device B eing descri E ed).
Figure
4 shows
the
output
from
RESEARCHER’s
processing of the initial part of EXI.
having
A combination
filter s stem for an enclosed disc
drive in which a brea %her filter is provided in a
central position in the disc drive cover and a
concentrically
air
filter
is
recirculatin
positioned aB out the breather filter.
A possible
Figure 3.
memette
structure
for this patent
in shown
in
(NAME. enclosed-disc-driveSTRUCTURE: ((SURROUNDS en~~sluh~~l~_T;lrP~~e~~~posite
TYPE: composite
(NAME: enclosure
STRUCTURE: ((ON-TOP-OF cover CaSe>))
(NAME:case TYPE: unitary
STRUCTURE: (box open-on-top))
(NAME: disc-drive
TYPE: composite
STRUCTURE: unknown)
(NAME: cover TYPE: c0m 0site
STRUCTURE: ((SURROUND~[centrally
(SURROUNDS[centrally
3
(NAME:
breather-filter
TYPE: unknown)
(NAME: recirculating-air-filter
Figure 3:
TYPE: unknown)
Representation
for EXl
The basic idea here is that we have a set of objects
relations
which can
related to each other b prototypical
be broken down into t l! eir canonical corn onen t s in order
2
to make low-level inferences,
when nee ed).
Note that
some of the information
shown in Figure 3 is not stated
explicitly in EXl.
For example, the ccqse is specified as a
unitary memette.; since virtually
nothing was said about
the enclosure, this information
was assumed by the reader
(from knowledge of prototypical
cases).
role
Four prototypical
relations
with their correspondin
ON- 5 OPfillers are used in this small) memette structure:
OF, SURROUNDS
and SURROUNDS[centrally]
(twice).
These define the relations
among the case, cover, and
various filters that are described in EXI.
Unitary
memettes
do not contain
any relation
records
under their STRUCTURE
nronertv:
instead. thev have a
tor. “Box bpenlon-to
” ‘was *given as
single shape-descri
1or of the case.
f his is not
a
the shape-descri
iece of information.
As yet there
articularly
func Pional
Ii as been no strong nee 2 to codify shape-descriptors.
3 Text
*(process-patent EXl)
Running RESEARCHER at 6:22:03 PM
Patent: EXl
(A COMBINATION FILTER SYSTEM FOR AN ENCLOSED DISC DRIVE 111
WHICH A BREATHER FILTER IS PROVIDED II A CEBiTRALPOSITIOB
IN THE DISC DRIVE COVER ABD A RECIRCULATING AIR FILTER IS
CONCEITRICALLY POSITIOHED ABOUT THE BREATHER FILTER *STOP*)
Processing:
Processing
RESEARCHER
begins its processing
of a patent
by
determining
from the text a conceptual
representation
of
the kind described in Section 2. In the ultimate version of
the program, this process will be strongly integrated
with
The conceptual
eneralization.
memory
search
and
erformed
b % ESEARCHER
is based on the
analysis
techniques desi ned for IPP.
memory- %ased unders %andin
This processing
involves a s op-down goal of reco nizing
structures
in memory integrated
with simple, bot 8om-up
syntactic techniques.
Natural1
since patents
are quite different
from news
hysical
stories, %‘0th because
they describe
complex
objects and because thev make considerable
use o P snecial
lane-uage,
the
precise
techniques
used
in
I%?%%RcHER
are distinct
from those used in IPP.
DISC-DRIVEI
crest of processing>
”
Figure 4:
In
233
the
output
RESEARCHER
trace
in
Figure
Processing
4,
we
can
EXI
see
how
RESEL4RCHER
identifies
the various objects mentioned
in the text as instances of eneral structures
described
in
such
DIS&DRIVE#
and
FILTER
)
%!$%%C L ER c:ates
new memettes
to re resent t e
specific instances
of these structures,
&ME&O for the
“filter
system”,
for example,
and records
how these
instances differ from the abstract prototypes.
Crucially
RESErmCHER
must do this without
being
specifically
provided
with examples
of these cpnce ts.
lnnE;‘dd, wh;; st;;;gyinstda;;es
ffFv;sa stream of mpu t!,. it
together
in
1ts
8 Memory, and notice similarities.
Generalization-Base
for two similar, slightly simplified
The representations
atents,
used . to . test the imtial version
of
disc drive
~i~~~e~CI!ER’s
generalization
module
are shown
m
enclosed-disc-drivel
I
/
enclosurel--------- disk-drivelon-top-of
\
II
II/
motor# I diskt 1 covert ----------> support-aembert
spindleX r/w-heads
The full
recessing
shown in ? igure 5.
of EXl
leads
to the
enclosed-disc-drive2
I
I
enclosure2---------------- disk-driveaon-top-of
\
I\
II
II/
\
disk# I cover# --------> bases /
motorP I
I surr \
spindle# r/w-head8
b-filtert ---> r-filterl
representation
Text Representation:
A list
of
Figure 6:
relations:
Sub'ect:
'SY&TEM'
DISC-DRIVEP
‘SYSTEM’
DISC-DRIVE*
'SYSTEM'
ElCLOSUBEt
COVERP
Relation:
R-PART
xk
’SYSTBB’
Ef-%T
R:PART
R-PART
R-SUBROUBDS
R-SUBROUBDS
LOCATIOB:CEBTER
RECIRCULATIBG-FILTERS
R-SUBROUBDS
BREATHER-FILTERS
COVERI
RECIRCULATIBC-FILTERI
DISC-DRIVEX
BREATHER-FILTEB#
RESEARCHER
Disc Drives
Clearly the two disc drives in Figure 6 have much in
common that can be the source of a new concept derived
through generalization
-- an enclosed disc drive.
created
by RESEARC%%?
7 shows
the
concept
generalization
mudtile.
BREATHER-FILTEBI
LOCATIOB:CEHTER
Figure 5:
Similar
Representation
enclosed-disc-drive#
/
I
encloauret-------- disk-driveXI
I
I
I
/
on-top-of
\
dj.sk# I cover# ---------->
<
motor# I
spindleP r/w-head+!
of EXI
The output in Figure 5 indicates that RESEARCHER
has
identified
the im ortant physical relations
mentioned
in
EXl.
(Actually, t he relations are among instances of the
abstract
memettes.) ‘It basically includes all the relations
shown earlier in Figure 3, our tar et representation
for
EXl plus part reh&onships.
Natura gily, the complete text
of EX1 describes many more relations.
>
(enclosed-disc-drive1and enclosed-disc-drive2
stored as variants of enclosed-disc-drive#)
Figure 7:
4 Generalization
Generalized
Enclosed
Disc Drive
The idea illustrated
in Figure 7 is that RESEARCHER
arts of two objects
that are similar,
and
finds the
In this
abstracts
t %em out into a generalized
concept.
example, the two devices contained similar disc drives and
enclosures. Each had a cover on top of some other ob’ect.
So these similarities
form the basis of a genera 1ized
enclosed
disc drive.
Only
the additional
parts
and
relations
of each instance
need be recorded
in memory
along with the generalization.
Adapting
Generalization-Based
Memory
for
use on
structural
descrintions
of the sort described
in Section
2 has proved to be a complex
and difficult
problem,
revolving around the assorted relations among the ob’ects
Here we will only present one o / the
in the descriptions.
major problems
and suggest the nature of the possible
solution.
genz;alizing
structural
The
central
two
in
matching
descriptions
is protbhleem recess
representations
(either of Pwo objects or of an object and
what
parts
and relations
a prototype),
determinin
correspond
as was pointe 2 out for simpler exam les in
Winston 7’i 1). Clearly, if we wish to determinemu;ttR tk:
in two drives are similar,. the
d isk mounts
compared with each other. Since, as mentione Y in Section
In order to store for later
uer
information
about the
patents
that
are read, Rl?SEdkCHER
makes use of
Generalization-Based
Memory.
This method, which was
831 and
is related
to
develo ed for IPP [Lebowitz
SchanR’s MOPS [&hank 821 involves storin
information
about given items in memory
in terms o5 generalized
prototypes.
The idea is to locate
the prototype
in
memory that best describes an exam le, and then store
only how the exam le varies from t i e rototype.
This
allows redundant
in Pormation
to be store s only once and
allows queries to be answered
in terms of descriptive
prototypes.
For Generalization-Based
Memory to be effective, it is not
adequate to sim ly make use of pre-specified
prototypes.
It is necessary
Por the system to create new prototypes
throu h a generalization
process.
This process involves
identi P ing similar objects and creating new concepts from
them r usmg a comparison
technique
of the sort used by
IPP [Lebowitz
831, and related
to traditional
“learning
from examples” programs, e.g., [Winston 721).
In
the
disc
drive
domain,.
typical
‘concepts
the
generalization
process
might
identify
as bein
useful
would
be floppy
disc drives
or double
side !I discs.
234
Acknowledgments
2, the central part of the description
of corn lex objects is
a sclt of relations,. we must associate the re Pations m one
object with those m the other.
The matching
process here is quite a difficult one. The
main problem
is that we are dealing with structured
objects,
and the parts. of very similar- objects
may be
So for
aggregated
differently
m various descriptions.
example, a read/write
he.ad might be described as a direct
atent,
but part of a
part of a disc drive,, in one
This makes
the
m anot lfier.
‘read/write
assembly
inherent similarity hard to identify.
At the moment, we deal with this “level problem”
with
simnle heuristics
that allow only a limited amount
of
during the comparison
process (to avoid
“le<el hopping”
ondence
amon
the need to consider every possible corres
levels) and a bit of combmatoric
force. ii owever, we fee9
that the ultimate
solution lies in more extensive
use of
Generalization-Based
Memory.
If a new object
can be
identified
as an instance of a eneralized
concept, *with
wB ich will be done with a
only a few minor differences
6
of the sort described
in
discrimination-net-based
searc
[Lebowitz 83]), then the levels of aggregation
will be set.
In effect, the existing concepts
create a canonical,
byi
dynamic,
framework
for describing
new objects.
addition, by using Generalization-Based
Memory, we need
between
compare
only a small number
of pifferences
objects, rather than complex descriptions.
uestion
hluch of the work described
here was carrind out b a
roup of Computer
Science
PhD students,
inclu d ing
Cecile Paris Tom Ellman and Laila
a renneth Wasserman,
Moussa, Master’s students
includin rd$rk
Lerner
and
including
Erik
undergraduates,
6
and
Galina
Comments
by Kathleen
Datskovsky.
a cKeown
on a
draft of this paper were greatly appreciated.
References
s
Birnbaum and Selfridge 811 Birnbaum
L. and
elfrid e M. Conceptual
analysis of natural language
In
R. C. !?cchank and C. K. Riesbeck
Ed Inside Compuier
Understandin
, Lawrence Erlbaum Associates, Hillsdale,
New Jersey, 1881, pp. 318 - 353.
[Dyer 821 D er, M. G. In-depth understanding:
A
computer mo clel of integrated
processin
for narrative
comprehension.
Technical Report 219, 9( ale University
Department
of Computer Science, 1982.
s
Foss:;nsa”pd Ehw+rtz 771 Kosslyn, S. M. and,,
A Simulation of Visual Imagery.
Co&itide
Science 1 (1977), 265 - 295.
Answering
The nresentation
of information
from a complex set of
data ‘in order to answer user
uestions is an -interesting
ointed out b
problem
in its own right (as B as been
including [Lehnert 77; %I cKeown 8217.
many researchers,
As part of RESEARCHER,
we have included a question
e
answering
module that concentrates
on taking advanta
effect!ve Fiy
of Generalization-Based
Memory
to more
convey information
to a user. Here we can only provide a
flavor of the approach we are taking.
langua e
RESEARCHER
accepts
questions
in natural
format.
It uses the same parser used to process texts f o
create a conceptual
representation
of the
uestion.
This
is much the same a preach as taken in BO‘k IS Dyer 821.
Also in similar fas I!ion to BORIS, we eventua fly expect
the question parsing process to identify actual structures
in memory, greatly simplifying the answering process.
a con;;ph$ai
develo ed
Once
RESEARCHER
has
representation
of a question, it searc Res memor
an answer using an approach similar to [Lehnert T 7 . That
is, a set of heuristics
is used to decide u on the I!ype of
question and what constitutes
a reasona %le answer. The
answer heuristics focus on using generalizations
that occur
to
quickly
convey
large
amqunts
of
in memory
articular mstances
information,
and then describing how
di e are also lookmg
may differ from the generalizations.
at how generalization-based
heuristics
might
aid. in
determining
what aspects of very complex representations
to try and convey to a questioner.
6 Conclusion
The development
of RESEARCHER
has led to interestin
results
in a number
of areas. Natural
language
tha &
involves corn lex physical objects is an excitin - topic, one
that can lea % to many interesting
applications. TV e believe
scheme
described
here,
the
that
the re resentation
a plication o P memory-based
parsing and Generalizationas well as generalization-based
question
Efased Memor
wil T all help lead to the successful development
answerin
of power Bul, robust, dynamic understanding
systems.
235
[Lebowitz 801 Lebowitz, M. Generalization
and
memory in an integrated
understandin
system.
Technical Report 186, Yale University 5 epartment
Computer
Science, 1980. PhD Thesis
&&rn~~~;,“,~~eLeb;$t~
- 40.
M. t“Geyralization
ogni ive
cience
of
from
7, 1 (1983), 1
[Lehnert 771 Lehnert
W. G. The Process o
. Lawrence &lbaum
Associates, Hi l-m
s ar/E:t%\
19 4 7.
Answerin
Jersey,
781 Lehnert, W.G. Representin
physical
memor
Technical Re ort 131 SIDepar Pment of Compu Per Scien>e,aiE78.
cKeown 821 McKeown, K. R. Generatin’
natural
TM
anguage text in response
to questions
about B atabase
structure.
Ph.D. Thesis, University of Pennsylvania,
1982.
Schank 721 Schank, R. C. “Conceptual
Dependency:
A theor
of natural lan uage understanding.”
Cognitzve
Psycho6gy
3, 4 (1972), ?32-- 631.
&hank 821 Schank, R. C. Dynamic
Memor
: A
!rheory o Remindin
and Learning
in Compu Pers and
People. c’ambridge
d niversity Press, New York, 1982.
Wasserman and Lebowitz 82 Wasserman
K. and
L ebowitz, M. Re resenting corn 1ex physical ob’ects in
Colum %ia University 8 epartment
of d omputer
s”~~%~*l982.
Winston 721 Winston
P. H. Learning structural
from examples,
In P. H. Wmston
Ed The
!l escriptions
Ps~ chology of Computer
Vzsion, McGraw-Hill:
New York,
19Y2.