LIBRARY
OF THE
MASSACHUSETTS INSTITUTE
OF TECHNOLOGY
^l&n-=teaSl
Hewoy
mss.
i;!si
JAN
ALFRED
P.
SLOAN SCHOOL OF MANAGEMENT
CONTROL STRUCTURE FOR A QUESTION-ANSERING SYSTEM
Ashok Malhotra
January 1974
#693-74
24
TECii
74
CONTROL STRUCTURE FOR A QUESTION-ANSERING SYSTEM
Ashok Malhotra
January 1974
#693-74
This work was supported by Honeywell Information Systems, Inc.
200 Smith Street, Waltham, Massachusetts 02154.
RECEIVED
JAN 28 1974
M.
I.
T.
LIURARIES
PAGE 2
INTRODUCTION
Despite my best efforts at explanation, my grandmother still
thinks
her,
of
the computer as a machine that answers questions.
this is
can do
intelligence far more
manifestation of
a
doing "sums";
than
"sums"!
It
question-answering
not surprising
is
therefore,
to me,
systems have as long
powerful
answers questions,
while everybody
for
a
To
few
that
history in artificial
intelligence as problem solving.
describes the
This paper
design
of a
proposed
answering system that will answer questions from
Ue
base.
and
the
strategies
corporate data
discuss the rationale behind the proposed system
will
design
assumptions.
a
question-
decisions
Further, we
developed for
that
will
follow
discuss
this system
from
the
initial
question-answering
the
basic principles
and the
underlying them.
EARLY QUESTION-ANSUERING SYSTEMS
flcCarthy
importance
(1)
of
seems
verbal
to have
learning
been the first
and
reasoning.
described the design for an "Advice Taker";
accept
natural
language input
and
be able
a
to realize
In
1958
the
he
machine that would
to
solve problems
PAGE 3
involving
deduction
attention to
much
of
number
a
contemporary
seminal contribution.
natural
the parse
sentences
it
subset of
prestructured tree
of
SYNTHEX
a
of
One
of
that
(4)
was
in
English.
It
format
a
an
data
questions
in a
data base
on a
able to
those
from
that
questions
in a
separate
of
the
answering
it.
was an ambitious system that contained the
childrens
encyclopedia
as a
data
An
base.
prepared listing the location of each "content word".
that
built
system
ingenious
thus
by
the declarative
of
to
was
inspired by
BASEBALL (3),
operated
base
discussed
(2),
answered
response
and was
to this
roots
it
with those
the input
how
surprising
is
and Phillips
system
a
it
trace its
can
the problems
statistics
creating
questions about
of
work
collaboration,
provided baseball
problems
issues and
A.I.
called
paper
This
had received earlier as data.
Chomsky's
limited
of basic
built
work,
matching
with
premises.
understanding
language
Chomsky's
given
on
text
index was
Sentences
could be relevant to the question were then retrieved using
this index and subjected to
an analysis procedure to screen
out
the irrelevant sentences.
The
analysis
considerations,
procedure
the system was incapable of
consolidate
was
however, and was
based
on
simple
syntactic
Further,
not very effective.
deduction and could not,
the information contained in
a number of
Since the grammar was independent of the program
it
therefore,
sentences.
could
easily
PAGE 4
be
extended but
the program
suffered from
knowledge and the underpinning of
SAD-SAM
and SIR
(5)
(B)
a
lack
of semantic
of the uorld.
a model
accepted information in the form
of
a restricted set of sentence types and created a semantic network
of
objects and relations.
questions.
Mas capable of
accepting and analysing
ownership information.
input formats
in
SIR
It was
but the
considerably more difficult
Each
This network was then used to answer
SAD-SAPl was resticted to familial
of these early
set,
relations while SIR
position
part,
and
relatively simple to increase the
semantic analysis
routines
were
to extend.
than one of the
systems attacked more
fol lowing problems.
a.
Creation and modification of
(restricted) natural
b.
c.
a world model
from
language input.
Answering questions from
a
data base.
Solution of simple problems using deduction on data base
e ements.
1
In addition,
understanding
some
most of
subset
of
difficult each of these problems
of
or
systems took
these
English!
is
Ue
on the
now
task
realize
and can admire the
of
how
ingenuity
these early systems that achieved encouraging results by more
less frontal
attacks.
PAGE 5
LATER SYSTEnS
It
surprising tinerefore, that later systems were more
is not
limited in their objectives
and tended to focus
of development
U.
natural
in
mainly
is
"transition network" grammar for
syntactic parser
very
is
are real,
simple,
if
practicability
and
he has
world situations.
geologists
lunar
system.
..
been able
"The prototype
was run twice a day for three days
..
(of
His data bases
to
demonstrate
were invited
.
the
systems in
the moon-rocks system)
and during this time the
...
to
During this demonstration
.
and
although its efficacy
powerful,
computer-based question-answering
of
(7)
Woods'
from the chemical analysis of the moon-rocks (8).
limited by its rudimentary semantic knowledge.
real
sentences
to analyze
a parser
He used this parser in a system to answer
it.
from the data base of an airline schedule book
questions
later,
a
languages and implemented
accordance with
great
there has been a
the other areas as well.
in
Hoods developed
A.
of
language
Ue shall describe only the natural
the above problems.
question-answering systems here although
deal
on only one
ask questions
80% of the
of the
questions
were asked and that fell within the scope of the data base
which
were parsed
and interpreted
correctly in
form
in
by Uinograd
(9)
exactly the
which they were asked ..."
A
for the
question and
world of
demonstrates
a
powerful
command system implemented
set of
children's
capabilities
of
blocks on
sentence
a
table-top
analysis
and
PAGE S
comprehension.
small
part
Further,
to
most
the world contributes
The simplicity of
the
impressi veness
the knowledge
of
procedure and this makes
it
In developing a novel
Carbonnel developed
of
in
the
is
no
however.
system,
the system
in
encoded as
difficult to extend.
approach to computer-aided instruction
the prototype
of a
knowledge-based
system
that was capable of answering questions in addition to asking and
evaluating them.
A
limited system based around knowledge of
the
geography of South
America
subset of English.
The system is undergoing further development.
is
operational
(10)
with a
limited
PAGE 7
A QUESTION-ANSWERING SYSTEH FOR nANAGEMENT
negligible
The
routine
than
documentation.
agreed
The reasons
computers
upon:
must be instructed
operate
they
impact of
computer-based systems
decision-making
in
at a
is
are also
behind this
are difficult
to
fairly
well
with and
Although
powerful,
with a
series of
specificity and
of
need
communicate
excrutiating detail.
level
on other
to
knoun
uell
too
elaborate conventions that the manager cannot tolerate.
To be useful,
deal
about the
Furthermore,
must
a
environment within
should not
it
be capable of
manager
which the
performing functions that
a question-answering
decided to allow
great
must "understand" a
manager operates.
programming
require procedural
are useful
and
to the
Starting with these premises we
his day-to-day work.
in
proposed
computer system
system as
questions that required
a
Ue
management aid.
retrieval
from a
data
base as well as the evaluation of functions of retrieved data.
restricted
class of
pre-programmed models could
the form of subroutines
but apart from
A
be executed in
was
these, no deduction
permi tted.
Starting
from these guidelines and
the system should be easy to
computer
the general
premise that
learn and use for managers with
experience we developed
a
specifications, many of which proved
more detailed
to be quite
no
set of system
different
from
earlier question-answering systems.
1. Si
nee
it
is
important that neophyte users should be able to
PAGE 8
learn to use the system easily and painlessly the system
should answer questions about itself.
unique
in
(He
believe this to be
question-answering systems.)
2.
The user should be protected from system errors and should not
3.
Attempts should be made
be expected to take any remedial action.
as far as possible.
to continue the dialog
If a
with the user
question cannot be answered the
system should attempt to say something relevant and
intelligent to convince the user that
request.
If a question
is
it
has understood his
ambiguous or unclear the system
should ask a question or make suitable assumptions (which
should be intimated to the user) and generate an answer.
4. In a real
situation
questions from
a
it
is not
possible to answer
data base containing only objects and events.
Aggregate data must be included.
For example,
sales
figures cannot be computed by checking and aggregating values
from the set of individual sales events.
The inclusion of
aggregates, however, gives rise to name ambiguities.
problem and its solution are described
(To our knowledge,
in a
This
later section.
no existing question answering system
incorporates aggregate data and the knowledge necessary to
answer questions about
5. The subset of natural
it.)
language that can be used to address the
system should possess adequate power and flexibility.
Similarly, response times to typical questions should be
PAGE 3
reasonable.
These
criteria have not been formalized because It
impossible to postulate satisfactory conditions a priori.
is
Ue propose to establish satisfaction along these dimensions
exper mental
i
ly.
Ue are convinced that
to have a great deal of
particular
Minsky
setting
it
makes
an
(11)
knowledge.
This
a
question-answering system uould need
knowledge about the business world,
the
was to be used in and about the data base.
excellent
case for
too can be considered
need
the
such
for
initial design
to be an
decision for the system.
Figure
is a
1.
Functionally,
the
schematic
system can
parser and the processor.
This
paper
a
a model
MAC.
and the
theory
case
of
grammar
of the world,
the design
is
answering
is
its
point
systems, such
parts,
the
rely
of
a model
the
semantic,
sentences.
and
of
The
the processor.
being implemented by Prof.
group at
U.
A.
Project
"case oriented" approach to
(See Fillmore
Celce-murcia
implementation.) Similarly, the processor
and this
of
Automatic Programming
It has a strongly
the analysis of English
two
data base
describes
parser has been designed and
nartin (12,13)
divided into
These two operating sub-systems
upon three knowledge bases:
scenario situation and
system.
diagram of the proposed
be
departure
as Uood8'(7,8),
is
from
and
(15)
the
(14)
for
for
an early
also knowledge-based
earlier
question-
its main strength.
PAGE 18
In addition,
more
the system
powerful
uii
I
considerably
operate on a data base
I
attempted with a question-answering
than anything
system to date.
The parser examines
parse
for
A
it.
the
parse
ly
identified
standard format.
attempts
response.
general
in
meaning of the
parse as
the
between
encoded
in a
input
and
the input request and generate an appropriate
Understanding the sentence therefore, takes place in a
sense within the parser and
much more specific sense
in a
the processor.
Ue shall say only
processor that
the
is
known
to the
the parser since
few words about
a
central
routine examines each word
is
and creates a
sentence
the
of
the
set of relations
is a
it
parts
processor uses
The
to fulfil
some sense,
in
sentence, but more accurately,
semantical
to the system
input
is,
in
paper.
to this
Unknown
is
The morphology
the input request and checks
world model.
it
words are
if
it
analyzed to
determine whether they are variants of known words or idioms.
recognized by the morphology routine a
If a word cannot be
message
printed out indicating the offending word and the user
is
is asked to retype his request.
Once
to
the complete sentence is
find the
sentence
main verb
accepted,
and to arrange
as cases of the
also
made of
the case-frame
of
the
the noun
phrases in
the
Initial prepositions that
main verb.
mark some of the noun phrases assist
the parser attempts
in
this process,
verb which
but use
is
determines the
PAGE 11
cases
contained
in
participate
world
the
Most of
in.
the sentence
If
his
request.
valid parse.
which
is
In some cases
the set
processor
isolates
the parser
prints
scenario knowledge to
of parses
cannot be
message
out a
word
be parsed,
or cannot
the
rephrase
to
more than one
processor
these parses are transmitted to the
All
If
ungrammat icai
and
words
the scenario world.
in
a message to the user and asks him
uses specialized
them.
this knowledge is
Specialized
model.
meanings and some idioms are contained
parser prints out
determine
meanings of the nouns which
can take and the
it
cases they can
the
eliminate some of
reduced to
stating that
one,
the
sentence is
the
ambi guous.
THE DATA BASE
This
sub-section
describes
language
a
knowledge.
Later
organization
on question-answering strategies
sub-sections
discuss
for
impact
the
encoding
of
this
and consider some
alternative organizations.
The knowledge representation language (which may be the same
programming language)
as the
objects
and
must allow
their properties.
It
the representation
must also
be able
property information of similar objects distinct.
it
is
base
convenient
if
the
in
which
a
to keep
addition,
language has facilities to query the data
and extract information from
HAPL (IB)
In
of
it.
manufacturing
Ue use a language called
corporation
called
The-
PAGE 12
Battery-Company can be represented sotneuhat as
(
fol lous:
(THE-BATTERY-COnPANY)
(THE-BATTERY-COnPANY)
(IS
(KIND CORPORATION))
(MANUFACTURE (THE-BATTERY-COHPANY)
(SELL (THE-BATTERY-COnPANY)
(KIND PRODUCT))
(KIND PRODUCT))
(EMPLOY (THE-BATTERY-COMPANY) EMPLOYEE))
(THE-BATTERY-COMPANY) MATERIAL))
(BUY
This
states
corporation.
buys
manufactures and sells
product
can
be
THE-BATTERY-COMPANY
that
It
a
into
this
a
kind
employees
of
and
The properties of the
kind of product.
inserted
is
employs
material,
definition
or
described
separately as:
((KIND PRODUCT)
(IS
(KIND PRODUCT) LEAD-BATTERY)
(COUNT (KIND PRODUCT) 5))
This states that the product
is
five types of
(lead-battery being used as a name).
could be described
further as
Massachusetts
but
law
Similarly,
a sub-chapter
this
may not
be
lead-batteries
the corporation
15 corporation
neccessary
in
and (KIND
CORPORATION) may be sufficient.
If
the
definition,
product
MAPL
ui
II
definition
keep
definitions that may exist
In
the same way,
in
it
is
imbedded
distinct
from
in
other
the
main
product
the system.
we can use MAPL to represent events.
The
statement "John met Mary at the corner." can be represented as;
PAGE 13
((MET)
(AGENT (NET) JOHN)
(AGENT
(HET)
MARY)
(LOCATION (MET)
(KIND CORNER))
(DETERHINER (KIND CORNER) DEFINITE))
flary can be
or
as a
represented as
kind of object
verb "met".
either another agent
depending on our
(as
above)
interpretation of the
PAGE 14
QUESTION-ANSUERING STRATEGIES
A semantic analysis of English questions is presented in the
questions
in
that
needed to
concentrates on what is questioned and about
it
arrive
The
at them.
meaning
determined
syntax
important to the analysis.
is
The analysis
questions
in part by
somewhat selfish in that
is
is,
it
mainly
analyzes
proposed question-answering
system can
Questions beginning with "why",
for example
that the
expect to encounter.
question
of the
the syntactic form and therefore
however,
,
of
rather than upon the syntactic form and the transformations
what
the
linguistic analysis
This differs from traditional
Appendix.
are not treated, as
the system does not
expect to be able
to
answer them.
UH-QUESTIONS
general, wh-questions
In
"which
Object
etc.)
"
ask
properties
(possibly
for
are
modified by
with "what",
(questions starting
the properties
questioned
tense words)
by
of objects
"be"
or
while event
events.
or
verbs
"have"
properties are
generally questioned by the main verb of the event.
After the
questioned
sentence
is
analyzed to
the property can be
or event exists in the knowledge
is
explicitly available.
determine the
retrieved directly
base and
if
if
property
the object
the property
value
At this point we should recognize that
the knowledge base may be organized by object or by event.
If
it
PAGE 15
is
organized by object then each
property
however,
one of
of
organization
be represented as a
I
as
its agent.
If
is also
of
the existence,
verb.
or "be",
possible in which information
is
be
A
stored
This allows a choice of retrieval paths and world or
both ways.
scenario knowledge can be used
least searching.
the
I
the knowledge base is organized by event objects will
represented as properties
dual
event ui
participants such
its
are questioned
Thus,
After
somewhat complex.
can be
leads
select the path that
to
to
finding the entity whose properties
the entity
is
located the value of the questioned property must be retrieved or
deduced.
This can be even more complex.
Different
question
strategy but the
general
finding
require
types
basic tasks of
Note
that
in all
finding the entity
requiring
will
that
embodies
some
level
wh-questions
in
different from answering them
not exist.
It may be
profit last year?" from
are
possible to
the data base
but
be our profit next year?" requires a predictive model
"Uhat
requires
answers
objective
restricted to the past and present tenses.
answering
and
remain
cases.
wh-questions
answer "Uhat was our
the above
of
of the questioned property
or deducing the value
more or less central
variants
of
subjective
the
future
in
the past
judgement.
tense
is
or present
Thus,
completely
tense
and
a search for suitable subjective models that may or may
PAGE 16
AGGREGATE DATA
Some properties,
itself,
gives rise
this
means little;
it
to
naming
a
be qualified by the parameters
has to
manufacturing
it
facility,
product, customer, salesman and time-period.
These may be
be aggregated.
to
by
"Sales",
problem.
over which
is
are
production figures
The aggregation can be made in several
best kept as aggregates.
ways and
costs and
such as
Some of
these parameters, or keys, as they are sometimes called, have
be
picked up from the various cases
of the parse,
to
others can be
or
filled in by default and still others may be assigned typical
commonsense
values.
TO
SUB-ENTITIES
there
fact,
In
default that operates
THEN
is
a
powerful, general
ATTRIBUTABLE
IF A PROPERTY IS
in English:
THE ABSENCE
OF
SUB-ENTITY SPECIFICATION
IMPLIES SUnriATION OVER THE SET OF SUB-ENTITIES, I.E. IT IMPLIES A
VALUE FOR THE ENTITY AS A UHOLE.
example,
if
there
sales for all products
Typical values
periods,
for
complete period
must
contain,
is
product then
on
the
data
usually be
unspecified.
therefore, a
in
will
list of key
assumed
a case grammer
because each
to be
the
last
item
variables which must be
In addition,
where the key variable
be contained in the sentence.
Time-
question.
Knowledge about each data
specified before the data can be retrieved.
contain information as to
the sum of
required.
depend
example, can
if
for
In a sales specification,
mention of a
is no
it
must
specifications
This is not too difficult
kind of information
has a
in
special
PAGE 17
assigned to
case
occur in the
time periods
For SKample,
it.
and customers
"during" case
usually
the
in
"recipient"
case.
Typical value information must also be contained in the knowledge
each data item but this
about
is
better organized by classes of
data items rather than for each item.
one of the dimensions of
In the case of "cost"
is
the reason for
variously named
transportation
single name
in
This gives rise to
expenditure.
or type of
costs such
expense" and
as "interest
As each cost category
cost".
the system
it
aggregation
is
"product
stored under a
becomes necessary to determine
the
unique name by operating upon the noun modifiers of the word cost
(or
expense)
and
what
can
transportation cost for lead"
done
the
our system
in
checking
if
context.
This
it
is
is
a
is "lead").
type of
named by
context
its
the context
by finding
a subset of
called
be
data item
(in "the
This
and then
one of the modifiers
continued recursively
till
is
or
no further
subset ting is possible.
Some sentences can be constructed so that the context serves
to determine the
retrieval.
unique name as
For example,
be assumed to
ask for the
well
as provide
a key for
its
"Show me the cost for all products" can
direct-manufacturing-cost as this
is
the only cost available that has "product" as a key.
Since
purposes an
fairly
different aggregations may
obvious strategy
disaggregated
be required for different
dictates that
state and
aggregated
data be
in
kept in a
response
to the
PAGE 18
is
stored
and
data
This requires knowledge of how the
needs of the question.
the keys
maintained for each data
look
to
for
the
in
sentence
to be
But maintaining data at one
item.
level
The aggregation of annual
of aggregation may not be sufficient.
sales for five products from five plants to five customers stored
This can mean a
by month requires 1508 probes of the data base!
long wait
at
console for
Thus,
user.
the
higher
levels
of
level
of
aggregation must be maintained as well.
system
The
should
aggregation required
"association
list"
is
the
highest
In our system,
has
time data
every
created
to be
This list contains the range over which aggregation
retrieved.
has
recognize
to
question and use the
the
of aggregation for which data is available.
level
an
able
be
to answer
to be performed for each key variable on the assumption that
the lowest
aggregation
whether
can be
level
of
aggregation
is performed,
will
however,
the
list
Before
used.
be
is
the
processed to check
data with a higher level of aggregation is available and
used.
If
the
so,
list
is
modified
and the
data-name
changed to a higher aggregate.
Uoods'
found
that
when
analysis of a rock, they wanted,
the ten most common elements.
distribution of sales
lunar
geologists
in fact,
asked
for the
its analysis for
only
Similarly, a manager may want the
by product whenever
he asks for
"sales".
Thus, particular questions may come to have iconic meanings above
and
beyond thier
literal meaning.
The
present design
of the
PAGE 19
system does not recognize iconic meanings.
the direct concerns of
not
are
and aggregation of data
It can be argued that retrieval
the question-answering system and
it
should restrict itself to extracting the key information from the
sentence and passing
knowledge
retrieval efficiently.
problem
and
hope
He
that
of the data
used
be
will
it
files to perform the
this decoupling
agree with
uses
system that
a data retrieval
on to
it
about the structure
the
of
future
in
implementat ions.
MODELS
The
use of
meaning
as
a
the word model,
replica.
relationships that
result,
Typically,
establish
stems from its
management,
in
models
the
dependencies
variables on independent,
or decision,
dependent variables are often figures
are
of
set
of
target,
or
a
health of the enterprise or the success of some portion of
the
model
used to assist
is
the
it
and
decision-making process by
in the
predicting the effects of changes
The
variables.
of merit that measure
in the
Independent variables on
the dependent variables.
Management
shapes
and
structures depending on the nature of the process they model
and
nature
the
of
models
the
can
take
a
decision-making
variety
they
of
support.
consider only one class of models, those that can be
by
a
set
described
can,
of
mathematical operations
in the next sub-section)
therefore, be encapsulated
(such
as
I4e
shall
represented
the functions
on specified items of data and
in a
subroutine.
(Other types of
PAGE 28
models,
and
in particular,
very important and very useful.
modified by the manager are
They have been excluded here
reasonable size.
Models
See,
order to limit the problem to a
in
however, Krumland (17).)
usually referred
are
specialized
those that can be calibrated,
Input data is not specified unless
neither
exceptional and
is
it
are the operations to be performed on
the question.
name in
to by
This information must,
it.
therefore, be available to the system as a property of the model.
Key
information relevant
however, be specified
A request
search
is
for
the question.
using
subroutine that
execution will,
key
the
calculates the
possible to
cause a
therefore,
After these have been located the data
input names.
retrieved
always
in
model
for
input data must,
retrieval of
to the
information
and
fed
Although
output value.
mathematical
specify models as
into
the
is
it
functions of
customary,
data only,
this is
therefore,
to specify models that use the output of other models
when the
as input.
Thus,
encounters
an input
evaluation routine
always convenient.
not
that
input retrieval
is a
to evaluate
model output
it.
It
is
routine for
it
models
calls the model
The structure of
correctly
specified models ensures that this recursion always terminates.
In
addition
knowledge about
also
a
an
argument list
model,
like knowledge
to
contain information as to where
found in the sentence.
Each model
and
a
subroutine,
about data items,
the
must
key variable values can be
must also have a
definition
PAGE 21
can
that
be used
calculated?".
questions
questions such
to answer
In fact,
as "How
is
profit
more than one definition is required
if
about various aspects of the model have to be answered
properly.
FUNCTIONS
required may be
the property
In some cases
a function
of
data such as percentage, distribution or average.
Unlike models,
functions
therefore,
can operate on
names of the inputs to the
data and,
variety of
a
the
the
function have to be specified in
sentence along with key specifications for their retrieval.
A
number of
are used
conventional devices
data items on which the function
is
to operate.
to specify the
For example,
the question
the
percentage of
will
name the data item and the subsetting characterict ic:
percentage of
distribution
a subset of
plant capacity
is required,
is
over all
utilized?".
the data item will
the key variable or subsetting
aggregated
an item is required,
Similarly,
"Uhat
if
a
be named along with
The data will have to
name.
if
be
the unspecified variables and retrieved as a
function of the specified characteristic.
In general,
names
occur
determination
in
data
names, key
different
of
the
of key specifications for
into account the function
did
parts
specifications and
to be executed.
sales increase over 1972?" and
sentence
subsetting
and
the
each argument must take
Consider,
"How
much
"How much did sales increase
PAGE 22
To Interpret and answer these two questions correctly
In 1972?".
understand that
the system must
function
to the
Typically, only one set of Key variables is
values.
the
the arguments
are both of the same type with different key variable
"increase"
others being picked up
specified,
The defaults, however,
by default.
depend on knowledge about the function.
the analysis
Thus,
data
has to
ask for functions
of questions that
depend heavily on
the function to
knowledge about
determine data name and key variable information.
been
Once this has
the data items can be retrieved and fed
done, however,
of
into
a subroutine that evaluates the function.
Functions are usually specified by name
exception
is "How
many" which
objects or events.
This
is,
in the
asks for the
question.
count of
however, an unusual
a set
An
of
function since it
does not operate on data.
DEFINITIONS
Consider
the questions "Uhat information
do you have about
product cost ?" and "Uhat do you know about product cost ?" After
product cost
possession
is
located directly and checked to be a subset of a
of "you"
(information) or
located by
a search from
among the possessions of "you" what does the system reply?
The problem
known
with "know"
is
similar, although
the
may be called knowledge and the search avoided.
what is required
is,
in
some sense,
the
definition of
object
In fact,
product
PAGE 23
This is implicit
cost.
in
the question but must be made explicit
for the system.
convention
examples
these
fact,
In
in English:
ITS MOST
IMPORTANT
being the principal
requires
general
more
a
IS ABOUT AN
QUESTION
REQUIRED.
1972?" asks for
OBJECT THEN
example,
For
value
the value of sales:
characteristic of data.
"What
dog?"
is a
the dominant characteristic of dogs in general, perhaps
that they are animals.
the
demonstrate
CHARACTERISTIC IS
were sales in
"Uhat
IF THE
this dog?", however,
"Uhat is
breed.
Carbonnel
Collins
and
in
these
represent
(10)
concepts" for each noun explicitly
asks
for
perhaps its
dominant characteristic of this particular dog:
"super
the data base.
YES-NO-QUESTIONS
As
discussed
restricted
truth of
stated
appendix,
the
in
to the present
and the past
propositions.
yes-no-questions,
if
tense inquire about the
Propositions
may
concern
the
properties of stated objects and the identity of actors and other
particulars
Yes-no-questions in the future tense
about events.
involve predictions
and judgements
about
the future
and
are,
therefore, excluded from the scope of this system.
In
a
data
base
organized
inquiring about the truth
answered
by selecting
by
nouns,
a
of stated properties
the appropriate nouns
properties against the stated
properties.
yes-no-question
of nouns can
be
and matching their
Ue find a
recursive
PAGE 24
matching
property
routine very
be
checked against those
useful
for this.
must be
It
mentioned in the questioned
remembered that the properties
in the data
must
base and not vice versa as
the data base may contain properties other than those questioned.
Event matching
Since events
possessor
is
somewhat more complex in such a data base.
stored
are
as
properties of
the
agent
or
located in the question and in
these nouns have to be
the data base and their properties searched for the event.
this is accomplished, or a set of possible events selected,
properties have to be matched against those
alternate organization
The
decision as
to
should
be
("be")
verb.
whether objects
stored under
the
the
in the
that
requires a
participate in
under
event and/or
their
question.
or verb,
by event,
Once
an
event
the existence
Direct retrieval for identity-questions and yes-no-
questions requires both facilities:
"Who robbed the bank?" property of "rob"
"Did Jack rob the bank?" property of Jack
If
properties
retrieval
is
are only
required for
indirect retrieval required
stored by
the
event then
second question
for events
in
an
indirect
much
like the
a noun-oriented
data
base.
There
events in
contents
seems to be an
the
world
but
essential duality between objects and
depending on
of the data base there
or vice versa.
Thus,
retrieval
the
question
and
the
may be more objects than events
through one
or the other
will
PAGE 25
lead
to
less
searching.
If
files is lees
of data
length
the
retrieval
important than the amount of searching required in the
process,
then a dual organization in which properties are stored
under both events
uith
the help
and objects and
In fact,
MAPL
is
selected,
path
search will
to minimize
Knowledge base,
of the
yield the best results.
organization
the retrieval
designed for the
dual
properties under both the
and automatically stores
object and the event.
IDENTITY QUESTIONS
Questions
different
start
that
yes-no-questions.
questions
is
Uhat
is
identity
the
properties stated
i
required
"Who"
while "which"
answer
as an
that
quite
more like
fact,
in
these
to
satisfies the
answering
the process of
yes-no-questions on a set of
a set of
questions ask
are
"which"
or
are,
object
of the
of satisfying the
objects that are capable
in general.
objects
"who"
the question and
in
can be likened to answering
candidate
with
wh-questions and
from other
for the
questions can
conditions
identity of
ask for
animate
either animate or
nam mate identities.
i
Answering routines for identity questions, therefore,
with
the selection of a
name for this set
in
is
set of candidate objects.
start
The generic
invariably specified as the main object noun
identity questions with "be" verbs that specify the properties
of the required object rather than the event
it
participated
in.
PAGE 2G
candidate set
The
is
the generic name.
agent
"Uho questions that
There
candidate set
is,
the
ask the identity of
any direct clues to the candidate
of an event do not give
set of objects.
the
the set of all objects that are "a kind of"
however,
the rule that the objects
in
given event and
to perform the
must be able
this can often be used to narrow down the set from the set of all
animate objects.
is
t
i
fairly small and so
through all of them is not very
search
a
me-consuming.
Once the candidate set
is
objects
In many data bases the set of animate
is
the selection process
established,
much like performing a yes-no-question matching on each
except
of each matching is
that the result
candidate or
"no".
The
reply
final
the identity of
the
can
be
the question
to
event
created from the set of identities that have been returned by the
individual
yes-no-questions.
the appropriate answer
is
If
this set
"None of
them"
is
set then
the null
for "which"
questions
and "No one" for "who" questions.
THE WORLD MODEL PROBLEn
Every data base and every question-answering system embodies
a
particular model of the world.
Further,
it
expects questions
about concepts and properties that are sensible
in
terms of
model.
if
the user has a
A severe problem can arise,
model of the world which
in significant ways.
is
therefore,
at variance with that of
the
this
system
For example, our data base contains direct
PAGE 27
manufacturing costs and overhead costs for activities that cannot
The overhead costs are
be directly attributed to manufacturing.
not
allocated to products, since we feel this
therefore break-even points
user
who likes to
have no meaning
think in terms
it
A
system.
in the
the system
if
merely
can't provide break-even data.
Ideally,
a
artificial, and
of break-even quantities will
ask for them and may not be able to proceed
says
is
the system should be able to realize that there
discrepancy between world models and since
model of the world
it
should explain
it
it
is
cannot change its
user to try
to the
to
influence his.
The
proposed
system
approach to this problem.
as break-even,
world.
that
Every
is
simple-minded
maintains a list of concepts,
such
models of
the
to variant
question asks for one of these concepts
responds by printing out
why the question
It
extremely
an
knows belong
it
time a
adopts
an explanation of
its world model
it
and
inappropriate.
IMPERATIVES
Besides
asking questions, the user
of the system should be
able to request services from the system by using commands.
allows the dialog
action
to be
much more
and as the actions possible
are the types of commands that
it.
The services provided
natural.
Commands ask
by the system are
may serve as meaningful
by the system
This
for
limited so
input
to
are limited to data
PAGE 28
retrieval, model execution and the provision of information about
itself.
Typical commands
the
to
therefore,
system,
are
as
fol lou:
"Show me the sales to Sears for 1973."
"Display the names of customers with outstandings
of over 85808."
These questions seem to ask for data retrieval.
"Compute the profit for '73."
"Calculate the return on investment last year."
These
questions seem to
ask for the
value of functions of
data or the execution of models.
The distinction
is specious,
base will determine
computed.
what can
The user will,
of the database and,
The structure of the data
however.
be retrieved
in general,
has to
and what
be
be unaware of the structure
therefore, his choice of verbs should not be
considered significant.
Other
verbs
have
semantic significance
and
will
require
special routines to process data and generate answers.
"Compare the distribution of sales
to Gulf and
Sears by unit."
"Contrast the sales for each quarter of this year and
last year.
"Sort the customers by outstandings."
CONCLUSION
Host of the
question-answering systems built
to date
have
PAGE 29
started
by devising data base
answering strategies
structures and defining question-
from scratch.
some extent,
To
this
was
necessary because each system addressed a particular kind of data
base and answered limited sets of question types using parsers of
very different styles and capability.
great deal about knowledge
from
these systems and
it
We have however,
representation and question
is
learnt a
analysis
desirable that future developers of
question-answering systems build upon this experience rather than
replicate
it.
Ue hope that our analysis,
limited as it is to our
situation and our goals, will be useful to future researchers.
PAGE 38
APPENDIX
ANALYSIS OF INTERROGATIVE FORHS IN ENGLISH
This appendix attempts to classify the types of questions and
commands possible
in English.
The classification is not primarily by
syntactic structure, since that
rather by semantic structure;
structure of the sentence
provides many clues
to
is
the concern of the parser,
uhat is asked and about uhat.
is still
but
The
rather important, houever, as it
its semantic function.
To the best of my
knowledge, such an analysis has not been attempted before. For reasons
of brevity presentation is restricted to the sentence types that are
expected to be encountered by the question-answering system.
English allows a wide variety of question types that can be
analyzed along a number of dimensions. Dost basically, questions can
be analyzed according to the answers they require: objective,
i.e.,
mathematical and logical functions of known data, and subjective,
i.e.,
requiring the formation of inferences and judgements beyond
those possible from the models and data contained
proposed system will not attempt
in the system.
to answer questions that
subjective answers. This does not mean
that it will
The
require
be unable to
answer questions that ask for reasons or opinions, but merely,
that
it
will be restricted to providing such answers that can be constructed
by sets of rules and data contained within the system.
Questions requiring objective answers can be further subdivided
according to the analysis required
to generate the answer and
the type
PAGE 31
of sentence construction used.
Uh-questions are so called because a word starting in "uh"
("how"
is an exception)
is used to ask
the question.
Such questions
Yes-no-questions, on the other hand,
ask for data or property values.
are distinguished by an auxilliary verb, such as "do" or "can",
preceding the noun group. These questions enquire about the truth of
stated propositions and expect either "yes" or "no" as an answer.
Sometimes, of course, the appropriate response
is "I
don't know".
UH-QUESTIONS
l.Also called adverb questions, these usually start with a
question adverb. Each adverb refers
to a
particular type of attribute. "Uhy" asks for reason or causality and
generally requires subjective answers.
"Uho broke the window ?"
Asking for the Identity of the
causal agent.
"Where did the accident happen
"Uhen did the accident happen
"What"
is special
in that
?"
?"
Asking for the location.
Asking for the time.
the attribute and the object must both be
specified for existence ("be" type) verbs.
"What is the hieght of the Empire State Building ?"
For other verbs, "what" asks the value of the object. Another case,
typically the agent, must also be specified.
"What did you see ?"
"What did Alex write ?"
PAGE 32
Sometimes the object can be assumed as "our" or filled
in from
context.
"What Here sales
in
1972 ?"
"Uhat" can also be used to ask for definitions.
"Uhat is a dog ?"
"Uhat is contribution margin 7"
"How"
is somewhat special
and comes in four forms:
"How many people attended the ball
?"
"How much beer will fill this glass ?"
"How tall
is
Maria
Asking for a count.
Asking for quantity.
Allowing the attribute
?"
to be named after
"How is profit defined
In the final
form,
main verb and,
7"
"How".
Asking for method.
the exact property questioned depends on the
in some cases,
even on its object.
Question adverbs can occur by themselves as a sentence provided the
context
2.
is known:
"Uhy ?",
"Uhere ?", "How ?".
Attribute values can also be questioned by an initial preposition
phrase with a question adverb
in
place of the noun.
"To whom did you entrust the keys 7"
"From whom did you get
the keys 7"
Asking for recepient.
Asking for source.
PAGE 33
?"
"Uith what did John hit him
Asking for instrument.
In some cases the preposition may be shifted to the end.
?"
"Uhat did John hit him with
IDENTITY-QUESTIONS
Questions starting with "who" or "which" are somewhat different from
other Uh-questions
in
that they
asl^
for the identity of objects
whose
properties satisfy certain criteria.
"Uho broke the window
?"
"Who is our largest customer
?"
"Uhich plants had costs over 10% of budget last year ?" The first of
these three questions requires a selection from among all animate
objects. This is the case with "who" questions with semantic verbs.
The second question requires a selection from the set of customers.
Specification of
a
be selected from
is
common noun that determines the set of objects to
typical of "who" questions with "be" verbs,
(questions of the type "Uho
is
Guru Haharaj Ji ?" are an exception.)
"Uhich" questions always specify a common noun which determines the
set of objects to be searched.
YES-NO-QUESTIONS
Yes-no-questions come
in a
variety of forms.
If
restricted to
the past and present tense they enquire whether objects have
stated properties and capabilities and whether stated events
PAGE 34
occurred. Another form
They ask only
if
is a
variation on Identity questions.
the set of objects that satisfy the criteria
is non-empty.
If
the future tense is also alloued, Yes-no-questions take
on a considerably more complex character and use modal
constructions to question stated events and courses of action
are plausible,
likely,
recommended, neccessary, etc. Future
tense yes-no-questions will not be discussed further in
this appendix.
The auxiliary verbs that signal yes-no-questions may be
semantical
ly empty.
If not,
they specify the following types
of questions:
Modals (could, uould, shall)
The potential or expectation
to act.
"Have" type verbs (has, had)
Possession or control over.
"Be" type verbs (is, are, were)
Existence.
The following are typical examples of yes-no-questions:
1.
Questions asking about the existence of
a property.
"Can monkeys sing ?"
"Is the car colored ?"
2.
Questions asking the truth of statements.
"Is contribution margin the difference between list price and
standard cost
?"
"Are monkeys green ?"
PAGE 35
3.
Questions asking whether attribute values satisfy certain relation.
"Is the car red ?"
"is profit this year greater than a million dollars ?"
"Is profit this year greater than last year ?"
4.
Questions asking whether
a command can be
"Can you stand on your head
carried out.
?"
"Can you show me the profit for 19B9 ?"
5.
Asking whether there
is a
non-empty set of objects with
given properties.
"Is there a house with three windows ?"
"Are there two cars are identical
?"
"Did any plants have costs greater than budget last year ?"
"Are there two cars that are identical ?"
Each of the above types of yes-no-questions can be formulated
In a different manner that specifies alternatives to be
selected
from:
"Do they have children or don't they ?"
"Can you show me the profit for 1969 or don't you have that data ?"
"Is his car green, black or red ?
Clearly,
these questions do not require "yes" or "no" as an answer
but rather the selected
al
ternativeCs) or a negative.
COnPOUND AND COMPLEX QUESTIONS
The above examples are all simple questions, but questions
PAGE 38
that occur as complex and compound sentences can be analyzed
similarly.
In complex questions, dependent clauses can be
used to specify time, place or other features of the entity
being questioned.
"Uhat uas Soars' profit
in the year
Ajax lumber wont bankrupt ?"
"Uhat car, did your brother say, he wanted us to tell Jane to buy
An important class of complex questions are those for which an
"if clause" specifies hypothetical conditions. Usually these are
parameters to a model; unmentioned parameters being picked up by
defaul
t.
"If sales went up by 10%, what would happen to profit ?"
"If we closed down warehouses 2 and 5 and fed their customers
from 3, how would this affect distribution cost ?"
Compound questions can usually be broken up into two or more
questions with the unrepeated conditions being common to them all.
"Uhat was the temperature
at 4 p.m.
?"
in
Phoenix on Saturday at 12 noon and
?'
PAGE 37
BIBLIOGRAPHY
(1)
McCarthy J., "Programs with Common Sense,"
Proc. Symposium on Mechanisation of Thought Processes,
H.M.S.O., London. 1959
(2)
Phillips, A. v., "A Question-Answering Routine,
Unpublished Masters Thesis,
Mathematics Department, M.I.T., Cambridge, Mass., 19B0
(3)
Green, Jr., B.
F.
,
et a!.,
"Baseball
Question Answerer," Proc. UJCC, Vol.
(4)
j
An Automatic
19,
1961
Simmons, R. P., et al., "Indexing and Dependency Logic
for Answering English Questions," American Documentation,
Vol. 15 1964
(5)
Lindsay, R.
K.
"Inferential Memory as the Basis of
,
Machines that understand Natural Language,"
in "Computers and Thought," Feigenbaum, E. A.,
Feldman,
(B)
Retrieval,"
(7)
eds.
J,
Raphael, B.
,
and
McGraw Hill, New York, 19G3
"A Computer Program for Semantic Information
,
in
Uoods, U. A.,
reference
(11)
pp 33-145
"Procedural Semantics for a
Question-Answering Machine.", Proceedings of the
FJCC 19G8
PAGE 38
Uoods. U. A., "An Experimental Parsing System For
(8)
Transition Netuiork Grammars."
in "Natural
Language
Processing", Randall Rust in Ed., Algorithmics Press
New York, N.Y. 1973
(9)
Uinograd
T.
"Understanding Natural Language,"
,
Academic Press, New York, 1972
(10)
Carbonell
J.
R.
(11)
(12)
and Collins A. H.
,
Artificial Intel
I
igence,
"
Proc.
,
"Natural Semantics in
IJCAI., 1973
"Semantic Information Processing,"
flinsky
f1.
I1,I.T.
Press, Cambridge Mass., 19G8, pp 1-38
,
ed.
,
Martin, U.A., "Translation of English Into HAPL using
Uinograd'
3
Syntax and a Semantic Case Grammer",
Automatic Programming
flemo 11
Project nac, M.I.T., April 17 1973
(13)
Martin, U.A., "Things That Really Matter
—
a Theory of Prepositions, Semantic Cases and Semantic
Type Checking,
Automatic Programming Memo 13
Project Mac, M.I.T, August 15 1973
(14)
Fillmore, C.
in Uni versa Is
T.
"The Case for Case,"
,
Linguistic Theory, Bach and Harms ed.
in
Holt, Rinehart and Uinston, 19G8
(15)
Celce-Murcia, M.
in
,
"Paradigms for Sentence Recognition,"
System Development Corporation final report
No. HRT-15892/7907
PAGE 39
(IG)
Mart
in,
U.
"riAPL 2,"
A.,
Automatic Programming demo 12
Project Mac,
(17)
Krumland
R.
M. I.T.,
B.
,
August
1
1973
"Concepts and Structures for a
Manager's Modelling System," Thesis Proposal,
Sloan School of Management, M.I.T., Oct 15 1973
H\
I
Lib-26-67
lUOU UUJ OCf 13
-J
,,/-73
||i||||||ii||i|ip|,.j,||T||fr|||"|
I
^DflD
3
0D3 TQb DM
'^O&O DD3 7°ih
3
31
6^1-7^
3
TQfiD DD3 7Tb
6=?^-7'^
3
TDflD
DD3
fl?7
?
3
TDflD
DD3 627
t
^'??'- 7^i
6^fo'-7<^
3
lOaO DD3
fl57
llill'!l'i'illi!i'i'"i'!''ir''i'lt'rr'i
fc^??-
3
TDflQ
DD3
fl2?
6?<?-7'-/
3 TOflO
3
DOS 7Tb
{
^Oflo'do'Vfl'^';"'^''