From: AAAI-80 Proceedings. Copyright © 1980, AAAI (www.aaai.org). All rights reserved.
HEARSAY-III:
A Domain-Independent Framework
for Ex
Robert Balzer
Lee Ermzrn
Philip London
Chuck Williams
USC/Information Sciences Institute*
Marina del Rey, CA 90291
Abstract
Hearsay-Ill is a conceptually simple extenslon of the basic ideas in
the Hearsay-II
speech-understanding
system [a].
That domaindependent expert system was, in turn, a product of a tradition of
increasingly sophisticated production-rule-based
expert systems.
The use of production systems to encapsulate expert knowledge
in manageable
and relatively
independent
chunks has been a
strong recurrent theme in Al. These systems have steadily grown
more sophisticated
in their pattern-match
and action languages,
and in their conflict-resolution
mechanisms [la].
In this paper,
we describe
the Hearsay-Ill
framework,
concentrating
on its
departures
from Hearsay-II.
scheduler,
the
provided
workspace,
structuring
called
of
blackboard
two major
To this heritage,
the
the
provided
incrementally
bhckboard
search,
via
a
building
-in
the structuring
Hearsay,
scheduling
interpretations
motivate
most of
- Through simple generalization,
the Hearsay
can be made domain independent.
- Scheduling
Hearsay
is itself so complex
blackboard-oriented
is needed
Our generalizations
the
approach
a task that
the
knowledge-based
to build adequate
schedulers.***
for
of
the main blackboard
consist of systematizing
activities:
The
structure
two-dimensional
hierarchical
that
of
and
mechanisms.
we bring two notions
our changes:
approach
advances
action of
KS.**
2. The Directions for Hearsay-III
1.The Heritage From Hearsay-II
Hearsay-II
the
which at each point chose for execution
one previously-matched
- aggregating
a composite
the
several interpretations
at one level
interpretation
at a higher level,
into
utterance:
- manipulating alternative
interpretations
(by creating
a placeholder for an unmade decision, indicating the
alternatives
of that decision, and ultimately replacing
the placeholder by a selected alternative),
and
- levels which contained different representations
(and
levels
of
abstraction)
of the
domain
(phones,
syllables, words, phrases, etc.).
- a location
utterance)
interpretation
Knowledge
agents
dimension (the time within the
positioned
each
which
within its level.
sources (KSs), relatively
which reacted
large
to blackboard
spoken
partial
The
production
changes
- criticizing
rules,
produced
complexity
dynamically
by other
both
of
the
created
domain blackboard,
encompassing
on the scheduling
level
partial
syllables
interpretations
into
higher-level
interpretations
a word)
(e.g.,
and
phrase
also
provided
a
they
the
to organizing
process,
Hearsay-II
competing
critics
same
interpretations
addition
or
activations
ones
writer;
from
competed
evaluators,
had
around
to
as
paradigm,
of
alternatives.
a
resources
expertise
necessitated
a
partial
in
generators
nodes
interpretations,
adopt the more neutral
for
among
is
a
the
These
include
the
regular,
to
react
to changes
The organization
of these
etc.) is left to the application
only
the
basic
mechanisms
(competence
knowledge)
search (performance
for
as the
with
the
knowledge).
creating
units
and
on
the
were
blackboard,
which
called hypotheses.
term unit.
Hearsay-Ill
aggregating
them,
represented
In Hearsay-Ill,
provides
i.e.,
complex
INTERLJSP), called A83.
‘This research wss supported by Defense Advrnced Research Projects Agency
contrect DAK15
72 C 0308
View.s and conclusions contained in this document ere
those of the authors and rho&l
not be interpreted em representing
the official
opinion or policy of DARPA, the U.S. Government, or eny other person or egency
connected with thorn.
AP3 has a pattern-matching
“A good discussion of scheduhng WI Hearsay-II
“‘This
notlon, in one form
owemple, [6], (21. and [III
108
another,
we
primitives
associating
hierarchically.
The blackboard
is implemented
in
purpose,
typed,
relational
database
system (built
in the form of
more
data
Thus domain KSs can be viewed
KSs controlling
In Hearsay-II,
level;
Thus,
provides
introducing
KSs.
react
KSs, which
as well.
priorities,
of
which
by
base
3. Blackboard Structure
the interpretation-building
allocate
This required
and
locations
move
records
and scheduling
systems.
handled
whose
KSs,
blackboard
Hearsay-Ill
scheduling
for exploring
alternative
conflicted if
Interpretations
activation
(with agendas,
building expert
on the basis
legal
overlapping
activity
also
interpretations.
and
word
lower-
contiguous
interpretation-building
method
i.e., handling search.
conflicting
three
lower-level
an edjacent
interpretations,
occupied
nearby
interpretation).
Within this aggregation-based
Hearsay-II
aggregating
producing
ones (e.g., predicting
of an exisiting
from several
is
blackboard
domain-dependent
KSs and in turn produced new changes.
The expertise
was thus
organized
around the activity
of building higher-level,
more
partial
interpretations.
scheduling
scheduling
separate,
were
proposed
them
a generalon top of
language;
this
csn be found in (51
is common
to 8 number
of others,
for
is used for retrieval
the
triggering
from the blackboard.
pattern
part of the definition
The
blackboard
somewhat
an
of the class tree
of
Unit
serve
other
some
been
to define
are
declared
is a system
from
informal
DomainUnit
by
the
For example,
for building formal
[l],
specifications
is ControlFragment,
Loop, Conditional,
is,
The
these
blackboards.
one
of
the
subclasses
of
Sequence,
The semantics
of the unit
classes other than Unit, DomainUnit, and SchedulingUnit
are left to
the application
Any
Demon, etc.
writer.
unit
may
interpretations.
serve
to
denote
choice
point in the problem-solving.
holder
for the interpretation
any other
unit, including
higher-level
units.
it represents;
alternatives of the choice.
they
The Choice
with
Choice
Set.
operation
When
on
a
alternative.
function
appropriate,
Choice
The
as a component
a Choice
Set
These may be explicit
may be implicit in a generator
Set,
a
KS
Selection
can
it
be
existing
done
in
with
the ability
to Select
another
alternative.
Contexts
selected
destructive,
retaining
are described
more in Section 5.
Hearsay-II’s
the
location
dimension
speech-understanding
Hearsay-III
blackboard.
dimension,
either
units
or
flexibility
of
the
time-within-utterance
is
The application
inherently
explicitly
(e.g.,
domain)
in the
as values
underlying
not
writer
associated
to have first-class
in KS triggering
patterns.
the
such a
structure
the
database
in
on
may create
with
relational
such constructs
imposed
interconnection
system
allows
of
to be used
execution:
created
Hearsay-II’s
basic
containing
the information
of the match.
sequencing
may be selected
needed
and saecuted,
is arbitrary
code, is run. The executing
blackboard
data
that
to execute
At some later
of
KS
triggered
the
KS in
i.e., the KS’s action,
it, which
usually
serves
interrupted
effect
for
the
execution
of any other
is an updated
of a KS execution
which
activations
of the same KS can pursue
retrieving
(potentially
private)
The
lndpendent
information
from
by
The
system,
select
problem
is:
the appropriate
given
the
activation
current
record
state
to execute
of
the
of
a
a context
if, for
Hearsay
109
that
in
KS
switches
and are
inherited
example,
in a wait state
poison
and
constraint
the
(e.g.,
KS discovers
whose
executing
arise
are
serve
to
context
to
diagnose
run
to
poison
domain
is poisoned
is unpoisoned.
allowed
from
attempting
a violated
execution
until the context
handlers,
specifically
for
Poisoned contexts
In addition, a KS can explicitly
units).
in
Special
poisoned
and correct
the
that gave rise to the poisoning.
A common application
alternative
of
it explicitly
as unsuitable
denoted
A KS activation
called
problems
next.
unless
of
(highest
ixecution
are made in that context
sometimes
conflicting
contexts,
scheduling
are
violation
is placed
blackboard.
and,
lines
context
the leaves.
aggregate
KSs,
context
with the context
alternative
in the most general
a condition called poisoned.
constraint.
the
same
of
matches.
in
and CONNIVER [9].
interacts
pursuit
its pattern
its changes
Contexts
KSs --
and is not
the same exploration
state
in the
that,
mechanism similar to those found
controlled
in which
down toward
as the
KS activation.
blackboard.
tree)
a method
For
independently.
by which KS triggering
A KS triggers
the
is indivisible; it runs to completion
building
an unmade
we still need
such as QA4 [lo]
languages
reasoning.
contexts,
to it the
a context
allows
occurs
initial focus for the activity of the execution.
Each KS execution
application
for
a means for representing
alternatives
mechanism
the
time, the activation
KS has available
features
added in Hearsay-Ill:
interpretations,
those
supports
Al programming
pattern of a KS is matched by a
of data on the blackboard, an activation record is
the environment
record
investigating
Hearsay-Ill
When the triggering
configuration
about alternative
The method
retains
mechanisms were
While Choice Sets provide
decision
4. Scheduling
Hearsay-Ill
To allow the
problem-solving
for
5. Context Mechanism
of
The
units.
status, for example,
allows
of activations
- The application
writer
provides
a base scheduler
procedure
that actually calls the primitive Eaecute
We intend
operation
for executing KS activations.
the base scheduler to be very simple; most of the
knowledge
about
scheduling
should
be
in the
scheduling KSs. For example, if the scheduling KSs
organize
the activation records
into a queue, the
base scheduler need consist simply of a loop that
removes the first element from the queue and calls
for its execution.
If the queue is ever empty, the
base scheduler simply terminates, marking the end of
system execution.
the
Select
a
the
a
the
units or
execute
with
into
are
associated
may
replacing
unit
manner, or it can be done in a new contezt,
irrevocable
a
Set is a place-
it can be dealt with as
its incorporation
Associated
alternative
competing
a Choice Set, represents
Such a unit, called
from which to select).
to use the Hearsay
triggering
collection
- When executed, scheduling KSs are expected to make
changes to the scheduling blackboard
to facilitate
organizing
the selection of activation records.
In
addition to triggering
on changes to the domain
blackboard, scheduling KSs can trigger on changes to
the scheduling blackboard, including the creation of
activation records.
The actions a scheduling KS may
take include associating information with activation
records (e.g., assigning priorities)
and creating new
about the
units that represent
meta-information
domain blackboard
(e.g., pointers
to the current
highest-rated
units on the domain blackboard).
The
scheduling
blackboard
is the
database
for
the
scheduling problem.
of programs
and it has subclasses
from
(i.e., a large
writer,
in the SAFE application,
specifications
execution
- Each activation record is a unit on the scheduling
blackboard.
The application writer supplies, as part
of the definition of each KS, code to be executed
when the triggering
pattern
is matched; this code
computes
a scheduling-blackboard
class (level)
in
which the activation record will be created.
by
immediate
application
KS
schemes
such schemes, several
The
SchedulingUnit;
the domain and scheduling
to his domain.
which
and
unit
of that class.
class Unit.
DomainUnit
of
schedultng
may be available
writer
generalized
The
separation
complex
as
Each unit is created
class.
is the general
are
subclasses
appropriate
Parallel,
have
of classes.
of
supplies
into an AP3 demon.
Hearsay-II
instance
writer
also an instance of all superclasses
subclasses
All
of
The
AP3 also has demons;
application
of a KS is turned
levels
as
inheritance,
classes
the
into a tree-structure
permanently
apex
which
interpretations
for the context
mechanism
lack good “locality”.
arises
First consider
when
the
exampfe
of
SAFE% Pfanning
represent
alternative
Phase,
which
uses
for
control
interpretations
Choice
Sets
to
knowledge
In
required
fragments.
applications
the case of the input sentence
“Send an acknowledgment
message on to the host.”
a Choice
Set served
sentence
include
existing
well.
being
structure;
identically
The
since
all
or
would
structure,
locality
be
toward
an
positioned
Set unit
will go.
An example
The
possible
occurrance
an
the message,
interpretations
of r triggers
existing
procedure
this
resides
(“After
r
at
same
the
place
to put the
alternatives.
Instead,
put them in brother
the
a
it to
demon
Since
structural
KSs
Set
these
contexts,
1.
the
there
representing
producing
alternative
earlier,
the blackboard
data structures
database.
In addition,
KS firing
because
must
also
KSs are
by activity
The
and all publicly
are represented
be
represented
in the
AP3 demons, and their
which
This
AP3
database
is similar
to those
is controlled
available
the relational
arguments
relational
in both assertion
capabilities
the application
are
for
These
modeling
directly
in
implementation
mechanisms
and
is implemented
INTERLISP
language
needed
constraints,
[12].
strong
Hearsay-Ill
Hearsay-Ill’s
initial implementation
AP3
of
considerations
Hearsay-Ill
has
tested
and
as described
SAFE system [l].
a system
system
for
program
producing
for
a
the
Hayes-Roth, F., and V. R. Lesser, “Focus of Attention in the
Hearsay-II System,” in Proc. 5th International Joint
Conference on Artificial Intefligence, pp. 27-35, Cambridge,
MA, 1977.
6.
Hayes-Roth,
small
demons
applications:
decoding
are currently
is the reimplementation
“jitterer”
language
which
so tHat a transformation
a
underway.
descriptions
chosen
by a user
used
as
transforms
Rulifson, J. F., R. J. Waldinger, and J. A, Derksen, “A Language
for Writing Problem-Solving Programs,”
in ZFIP 71,
pp. 201-205,
North-Holland, Amsterdam, 1972.
11.
Stefik, M., Planning with Constraints, Ph.D. thesis, Stanford
University, Computer Science Department, January 1980.
12.
Teitelman, W., lnterlisp Reference ManuaL, Xerox
Research Center, 1978.
13.
Waterman,
a
is applicable
143.
The Hearsay-Ill
separation
of
10.
fnference
architecture
competence
Reference
of expert=
is being
automatically
McDermott, D., and G. J. Sussman, The CONNIVER
Manual, MIT Al Laboratory, Memo 259a, 1974.
The
of the
is being used as the basis for
Finally, the system
9.
problem.
efforts
natural
Mann, W. C., and J. A. Moore, Computer as Author -- Results
and Prospects, USC/Information
Sciences Institute, Technical
Report RR-79-82,
1979.
of
effort.
above,
[8].
8.
an
efficiency
implementation
two
Hewitt, C. E., Description and Theoretical Analysis (Using
Schemata) of PLANNER: A Language for Proving Theorems
and Manipulating Models in a Robot, MIT Al Laboratory,
Technical Report TR-258, 1972.
the
design
B., and F. Hayes-Roth, Cognitive Prooesses in
The Rand Corporation, Technical
R-2366-ONR,
1979.
7.
is
being largely
However,
a cryptogram
Second, Hearsay
data structures
basis
on
as
contained
was almost trivial,
a substantial
major implementation
first of these,
the
been
problem
chosen
(e.g., contexts,
In fact,
conventions.
have forced
cryptarithmatic
Three
usage
was
already
typing).
a
set
AP3
it
because
to support
and
in AP3, which in turn
Conf.,
5.
Phning,
7. Implementation and Current Status
implemented
Balzer, R., N. Goldman, and D. Wile, “Informality in Program
Specifications,” IEEE Trans. Software Enp. SE-4, (21, March
Report
system
problem-solving
Fickas, S., “Automatic Goal-Directed
Program
Transformation,”
in 1st NationaL Artificial Intelligence
Palo Alto, CA, August 1980. (submitted)
domain.
The Hearsay-Ill
overall
4.
for each of
and retrieval.
available
the
toward
Erman, L. D., F. Hayes-Roth, V. R. Lesser, and D. R. Reddy,
“The Hearsay-II Speech-Understanding
System: Integrating
Knowledge to Resolve Uncertainty,” Computing Surveys 12,
(2), June 1980. (To appear)
in languages
typing
of
gives
go a long way
3.
is
in the database.
such as PLANNER [7], but also includes strong
typed
is to cause
database.
triggering
aspect
architecture
will undoubtably
and
issue.
Davis, R., Meta-Rules: Reasoning About Control, MIT Al
Laboratory, Al Memo 576, March 1980.
accessible
in the AP3 relational
any domain information
this
is a difficult
2.
so that each can be
6. Relational Database
As mentioned
simplifying
Hearsay-III
algorithms
larger
1978.
is
these
independently.
Hearsay-Ill
the
the
this out,
References
in
demon
as
would be embedded,
that
scheduling
seem to bear
(scheduling)
knowledge
with
(“The
the
level
that performance
expert
We wish to thank Jeff Barnett, Mark Fox, and Bill Mann for
their contributions
to the Hearsay-Ill design.
Neil Goldman has
provided excellent and responsive support of the AP3 relational
database
system.
Steve Fickas, Neil Goldman, Bill Mann, Jim
Moore, and Dave Wile have served as helpful and patient initial
users of the Hearsay-Ill system.
is the
to be embedded
y”).
Choice
interpretations
pursued
do
into which the sequence
no convenient
include
y”) and a sequence
procedure
interpretation
for
the imp passes
development
the
experience
Acknowledgments
input sentence,
“After receiving
the host.”
formulating
Pretiminary
process.
eventually
is lacking.
flexibility
developing
this
with
the Choice
the chosen interpretation
In some cases, however,
for
in sequence
alternatives
in the existing aggregate
can be placed where
interpretations
in parallel
in rapidly
now under
seem to indicate
to the imp and pass the
The possible
put
helps
for a solution.
seems to be a helpful one.
The
knowledge
from
p.erformance
110
D. A., and F. Hayes-Roth,
Palo Alto
Pattern-Directed
Systems, Academic Press, New York, 1978.