From: AAAI-87 Proceedings. Copyright ©1987, AAAI (www.aaai.org). All rights reserved.
perationality
for Ex
Richard
M.
Keller
,Rutgers University Department of Computer
New Brunswick, New Jersey 08903
Internet: Keller@Rutgers.Edu
an initial target
Abstrdct
enced a surge of interest
ing methods
called
et al., 19861.
In
ities
among
explanation-based
knowledge-intensive
to empirical
analysis
large
concept
numbers
particular,
although
the initial
given to an explanation-based
( i.e., it covers the correct
tion is in non-operational
the description
to improve
methods
perform
of a single
meth-
of similarin-
an in-depth,
training
example
- typically a positive instance.
That analysis involves first
explaining why the positive training instance is an example of the concept to be learned (the target concept), and
then generalizing
the explanation
in a principled
manner
so it is valid for a larger class of instances
than the original instance.
Finally, a description
of that larger class
of instances is extracted
from the generalized
explanation
structure.
The description
constitutes
a generalization
of
the original instance.
A seeming paradox of the explanation-based
paradigm
is that in order to produce its final description
of the target concept, the learning system must possess an initial
description of that same concept.
In fact, without “knowing” an initial description
of the target concept, it would be
impossible for the system to explain why the given training instance is an example
of the target concept.
So if
482
Machine Learning & Knowledge Acquisition
cannot
having
the task
target
system
the initial
descripare at
be used
effectively
There
a concept
methods
example
to learn
system,
descrip-
this
means
by the learner
is a significant
description
that gap by transforming,
analogy
the
Informally,
differ-
and being
able
system
is to
or operationalizing,
the
of an explanation-based
initial description.
As a concrete
ston et al’s
concept description
generally is correct
set of instances),
form.
task performance.
ence between
learn-
of training
with
existing knowledge that is unusable or impracticable
into
a form that is usable [Keller, 1983, Dietterich,
19861. Pn
[Mitchell
learning
analysis
is wrong
for
necessary
The way to untangle the paradox is to acknowledge
that learning can involve knowledge transformation,
as well
as knowledge acquisition.
Explanation-based
methods do
not acquire “new” knowledge, per se, but rather transform
has experi-
methods
a simple syntactic
and differences
stances,
in a class of analytic
explanation-based
contrast
ods, which perform
learning
is a prerequisite1
then why is learning
tion? What is there to be learned?
These questions
the very heart of the “explanation-based
paradox”.
narrow
the field of machine
description
methods,
in the first place ? What
to use it;
years,
concept
explanation-based
Operationality is the key property that distinguishes
the final description lecarned in an explanation-based
system from the initial concept description input to
the system. Yet most existing systems fail to define
operationality with necessary precision. In particular,
attempts to define operationality in terms of “efficient
instance recognition” tacitly incorporate several unrealistic, simplifying assumptions about the learner’s
performance task and the type of performance improvement desired. Over time, these assumptions are
likely to be violated, and the learning system’s effectiveness will deteriorate. We survey how operationality is defined and assessed in several explanationbased systems, and then present a more comprehensive definition of operationality. We also describe an
implemented system that incorporates our new definition and overcomes some of the limitations exhibited
by current operationality assessment schemes.
In recent
Science
for illustration,
consider
Win-
which uses explanation-based
a description
for CUP
[Winston
et al.,
19831. In this case, an initial CUP description
is given
to the system, expressed
in functional
terms:
“a CUP is
an open,
stable,
operational
mation
liftable
because
to enable
vessel.”
This
it does not contain
a vision
system
description
the necessary
(the performance
is noninforsystem)
to improve its performance
in recognizing
CUPS. In order
for the description
to be useful in improving
performance,
the learning
system
must transform
the functional
composed
tion into a structural description,
the vision system is designed to recognize:
light object with a handle,
pointing concavity.”
a flat bottom,
descrip-
of primitives
“a CUP is a
and an upward-
Although operationality
is the key property
that distinguishes the final description
learned in an explanationbased system from the initial target concept description.
most existing systems fail to define operationality
with necessary precision.
Current attempts
to define operationality
‘The initial target concept description may be represented in the
learning system explicitly (’
IX., in terms of declarative structures) or
implicitly (i.e., within the system’s procedures) [Keller, 1987b].
tacitly
incorporate
several
unrealistic,
tions that may not hold throughout
In particular,
ationality
many explanation-based
as an independent,
erty of concept
dynamic,
on the learner’s
of performance
understanding
sophisticated
properly
in dynamic,
In what
cisely
property
improvement
and analyze
learning
as a search through
Suppose
D1 in Figure
scription
provided
is a
of descriptions,
task
desired.
and the
A more
is necessary
systems that
real-world
environments.
we define
operationality
follows,
operprop-
operationality
performance
of operationality
explanation-based
treat
assump-
binary-valued
Actually,
continuous-valued
pendent
The role of operationality
with respect to explanationbased learning is clarified
by viewing explanation-based
systems
static,
descriptions.
of learning.
simplifying
the course
how operationality
detype
thorough
to construct
can function
more
is assessed
pre-
in sev-
eral explanation-based
systems.
Then we describe how the
MetaLEX
system [Keller, 1987b, Keller, 1987a] overcomes
some of the limitations
exhibited
by current operationality
assessment
schemes,
by incorporating
our new definition
of
operation&y.
explanation
a common
introduces
some terminology
framework
to serve as a basis for our discussion
of operationality.
We begin by distinguishing
its description.
drawn from
sionally
between
by a concept description,
Two
of instances.
the universe
considered
Figure
synonymous
1 illustrates
figure depicts
if they
these
the space
be described by a given
concept space represents
from instance
space,
space.
denote
over
are
concept.
space.
of the concepts
regions.
between
into
Note that there
a point in con-
description
D1 and D2 are two synonymous
space.
As
descriptions,
So when.there
exist
different
the same concept,
operationality
defines
preferring one description
over another.
ways to describe
the criterion
for
process
his formueffectively
The
through the concept space
(Dl) and the final descrip-
concept
(6).
ity
definition
describing
for
Explanation-based
a
of operationality
explanation-based
most
systems
commonly
cited
in
is the following:
Caakrena%Opera%iondi%y
efaa.: A concept description is operational if it can be used efficiently
to recognize
the figure de-
op-
is insufficient
learning.
the same
Below we review
both describing
concept C, which covers instances
I1,12,
and 13. However D2 is considered
operational,
whereas
D1 is not.
denote
the
space is partitioned
in concept
tion (D2)
can
At left,
We
D2 “concept
characterization
involves no searching
the initial description
In the center,
descriptions
The description
This
explanation-based
concepts
that
into
the space,
equates a concept with its description,
thereby masking the
issue of operationality.
Thus he effectively
characterizes
learning as a search through a concept space, not a concept
and
learning program.
Each point in
a unique set of instances
drawn
shown at right.
cept space and the points
the same
relationships.
operational
and non-operational
is a one-to-many
correspondence
illustrated,
which is a predicate
concept
descriptions
of all possible
picts the space of all possible
in concept
a concept
A concept represents
a subset of instances
A concept is denoted intensome universe.
Dl
we can
criterion.
between
[Mitchell, 19821. A crucial distinction
lation of the problem and ours is that Mitchell
learning
because
aud establishes
Then
D2 as a solution
termination
of transforming
de-
and D2
erationalization”
[Keller, 1987b, Keller, 19831.
Mitchell first described
learning
as a search
deac&ption
section
learned.
for traversing
as the search
space.
system,
node in a search,
as the means
and operationality
describing
This
description
as the starting
call the process
description
non-operational
to an explanation-based
is the final, operational
node,
the concept
1 is the initial,
instances
several systems
that
that operationality
on our retrospective
use explanation-based
methods.
defined
so the following
analysis
et
it denotes.
is instantiated
has not been
eral of these systems,
ins&cm
of the concept
how this definition
de95
explicitly
definitions
in
Note
in sevare based
and reconstruction.
nc%icm-S%ruc%use
s system,
system
the target
concept
is the set of drinking CUPS and the initial description
is a
functional description:
“a CUP is any open, stable, liftable
vessel.”
Instances,
tions of CUBS
s tructud
properties,
etc. Therefore,
in this system
terms,
from
however,
consist
the real world,
of physical
expressed
such as “flat” ) “handle”,
an operational
as a description
descrip-
in terms
of
“concave”,
CUP description
is defined
stated solely in structural
so it can be easily matched against instances.
2 [Mitchell
et ad., 19861: In LEXB, the target
concept is the set of USEFUL
problem solving moves to
apply during search.
The initial description
of USEFUL
given to LEX2 states that “USEFUL
diately or eventually
to a solution.”
moves lead immeInstances
consist of
calculus problem solving moves generated while solving
tual. problems.
TQ facilitate
matching against instances,
operational
description
Figure
1:
csncepe
acan
description
of USEFUL
is defined in LEXB as a
expressed in terms of the calculus features used
to describe instances
(e.g., “sin”, “3”, ‘“product”, etc.), or
in terms of features easily derivable from them (e.g., “trigfunction”,
“integer99 9 6‘po1ynomia199 9 etc.) S
Keller
483
.
o PRODIGY
a variety
including
[Minton,
of PRODIGY’s
problem
shop scheduling
domain,
formed
19861:
This
system
the “efficient
learns
of target
concepts
related to problem
solving,
the USEFUL
concept
learned by LEXB.
One
into
CLAMP,
finished
solving
goods
using
etc.
PRODIGY,
under
which
applying
SUCCESSFUL.
Instances
the machine-shop
operators
correspond
environment
description
stance
can learn
to different
is UNstates
of
the operators
are
recognition
description,
in the environment,
tural,
rect observability
requirement
“idle”,
assures
“busy”.
that
The
di-
UNSUCCESS-
proper
e GENESIS
[Mooney and DeJong,
19851: In one of
GENESIS’s
application
domains,
the target concept corresponds to WEALTH-ACQUISITION-SCENARIO,
and
including
a larger
An operational
number
of
of performance
napping,
from
in the three
systems
stances,
by the
described
GENESIS
is not
present in the in-
but rather in terms of abstract
schemata possessed
system.
A high-level
description
of WEALTH-
ACQUISITION-SCENARIO
recognition”
because
in GENESIS
parses
top-down
facilitates
the story
stories
“efficient
understanding
(i.e.,
instances)
instance
component
efficiently
in
fashion.
can be
description.
definition
time is the
oper-
The
including
cost,
way to eliminate
the
definition
there are arbitrarily
might be relevant to
elegance,
these
simplicity,
restrictive
of operationality
etc.
assumptions
is to redefine
that
system’s
inition
performance.
Table
of operationality.
operationality
There
in the revised
1 gives our revised
def-
are two requirements
on
definition:
usability and utd-
dty. The usability requirement
ensures that the description
can be used by the performance
system.
This means that
the description
must
be expressed
corresponds
to the original
duced in [Mostow, 19811.)
notion of operationality
The utility requirement
usability
the description
completes
purposes,
we can consider
a specific
subgoal
problem
attempts
“similar”
For our
(along
with
the current processing
state)
as a training
instance,
the
class of “similar” subgoals as SOAR’s target concept,
and
the generalized
production’s
conditions
as its operational
description.
An operational
description
of the target
con-
cept is one that can be used to efficiently recognize whether
a “similar” subgoal is true in a given processing
state. In
other words, in SOAR an operational
production
consists
solely of conditions
that can be easily evaluated
without
present in
conditions
initially
problem solving, including
that are evaluSOAR’s working memory, and conditions
ated using chunks acquired during problem solving.
The notion
ployed in these
of “efficient
instance
systems
is a suitable
defining operationality,
as a general-purpose
484
recognition”
emstarting
point for
but is in several ways inadequate
definition.
A basic problem is that
Machine learning & Knowledge Acquisition
or computable
by the system.
one step
Table
and in terms
of capabili-
chunking
Each time SOAR
by the system,
in terms
ties possessed
mechanism.
it in
terms of the performance
system that uses the learned concept description,
and in terms of the criteria for evaluating
e SOAR
[Rosenbloom
and Laird, 19861: In SOAR, a
form of explanation-based
learning is an integral part of its
solving activity for a specific subgoal, the system
to construct
a generalized
production
to achieve
subgoals without
resorting
to problem
solving.
for
than struc-
evaluating performance,
including
space efficiency.
A description that is operational
with respect to time efficiency
may not be operational
with respect
to space efficiency.
performance,
Unlike
description
rather
to use in evaluating
of natural language text describing
stories involving acquisition of wealth, such as stories involving inheritance,
kidetc.
be
ationality.
However, there are other types of ‘6efficiency9’
that may be just as appropriate
or more appropriate
for
Furthermore,
aside from efficiency,
large numbers of other criteria
that
arson,
gen-
we might
of novel cup designs
functional
SCENARIO
consists of any sequence of actions that culminate in an agent’s acquisition
of wealth.”
Instances
consist
above, an operational
description
for
stated in terms of the low-level features
instance
domain,
is functional,
from the abstract
measure
in-
use of a concept
Second, the “efficient instance
recognition”
used by most systems assumes that execution
FUL conditions
can be recognized
quickly and operator
application
can be avoided in a real-time
environment.
an initial (although
not explicitly
stated)
description
the target concept is that “a WEALTH-ACQUISITION-
of cups).
of generation
because
generated
uses,
descrip-
Although
in recognizing cups (e.g., if we want to
or in generating cups (e.g., if we are de-
signing new types
als and the machine-shop
performance.
instances”.
in the CUP
incor-
how the final
that the concept
one typical
are other
interested
either
drink a beverage)
the purposes
equipment
represents
there
must be phrased in
of the raw materi-
“temperature”,
assumes
For example,
e&ion.
tacitly
about
will be used to improve
the definition
applied.
An operational
description
terms of directly
observable
features
such as “shape”,
definition
assumptions
tion will be used to “recognize
like LATHE,
operators
in which
concept
First,
are trans-
for example,
these
recognition”
several restrictive
is a machine-
in which raw materials
POLISH,
conditions
domains
instance
porates
further:
(The
of data
usability
known
requirement
must
introtakes
not only
Operationallity Definition
1: Revisedl
Given:
QBA concept description
CBA performance
ayatem that makes use of the descrip-
tion to improve
l
Performance
of system
performance
objectives specifying
improvement
the type and extent
desired
operational
is considered
Tiren: the concept description
if it satisfies the following - two requirements:
1. usability:
formance
2. utility:
mance
the description
must
be usable
by the per-
system
when the description
system,
the system’s
prove in accordance
is used by the perforperformance
must im-
with the specified
objectives.
be usable,
description
system,
but also worth using.
In particular,
using the
must improve the behavior of the performance
as defined
by its performance
As an example
objectives.
of how this revised
or dynamic.
operationality
defi-
nition might be instantiated
for existing explanation-based
systems, consider once again Winston et al’s CUP domain.
In this domain, the performance
system might consist of
a mobile robot searching for cups in a room. The performance objectives
for the robot might be to improve the
speed with which
a CUP
it can recognize
description
correspond
able to easily
the properties
a structural
for instance,
an operational
With
For
it must
that
property
would
the robot
must
such
a description
progresses,
may become operational,
due to changes
in the performance
curate
assessment
assessment
system
is made.
and white camera.
tions which specify
a dynamic
do not.
namic
because
assessment
suppose
the notion
situation.
instead
systems
of operationalContinuing
we are learning
acquire
descriptions
with
operational
objective
al’s system and in PRODIGY
remains
the course of learning,
so these systems
the number
the system can generate.
Now the revised
definition correctly pinpoints
an operational
of cup designs
operationality
description
as
one expressed in terms of the functional design primitives
known by the system, instead of structural
primitives.
subsystem
[Utgoff9
generalization
ever, that the set of operational
cally adjust
to changes
As these b
the set of
is enlarged.
in LEX2
when the
be to increase
of the
or chunks,
cups in a design context.
In this case, the performance
system might consist of a design system containing
a library of functional
design primitives.
The performance
might
STABB
is dy-
in terms
respectively.
schemata
descriptions
term to the system’s
section
whereas
and SOAR
is defined
or chunks,
considered
the set of operational
about
system.
in the previous
in GENESIS
additional
camera
of operationality,
operation&y
ezistdng set of schemata
definition,
for the updated
surveyed
Assessment
if the
these same descriptions
operational
Some of the systems
others
of
with a black
However,
with a color camera,
as part
a performance
equipped
For this system,
any object
descripcolor attributes
should be considered
should be considered
perform
An ac-
on when the
consider
robot
for recognition.
non-operational
were replaced
is initially
and vice versa,
depends
For example,
of a mobile
that
environment.
of operationality
consisting
as “specific-
not be permitted
to fit the learning
the above example,
As learning
non-operational
be
used in the descrip-
description.
the revised
ity adjusts
by the robot,
properties
as well as “usable”,
evaluate
Therefore,
gravity”,
of object
cups.
can be de.
sensory systems.
Those properties
to structural
properties.
For a CUP descrip-
tion to be “utile”,
tion.
and retrieve
to be “usable”
be expressed in terms
tected by the robot’s
A dimension
characterizing
whether
eVn&bility:
operationahty
assessment
varies with time. Values: datdc
Similarly,
is augmented
19861 adds
language.
descriptions
a new
Note,
how-
in Winston
et
static throughout
cannot automati-
in the performance
environment.
e Granularity:
A dimension
characterizing
the assessment measure produced.
Values: binary or continuous.
Most
of the
sessment
systems
surveyed
of operationality:
operational”.
Rowever,
either
produce
a binary
“operational99
continuous-valued
as-
or “non-
assessment
has
In this section, we discuss how operationality
is assessed
in explanation-based
systems.
Thus we draw a distinction
distinct advantages
over binary assessment
because it allows the learning system to assess degrees of operational-
between how operationality
is defined and how it is evaluated in practice.
Conceptually,
each explanation-based
ity.
system
contains
which evaluates
an
operationality
a concept
sure of its operationality
three dimensions
which characterize
and produces
as output.
- variability,
Below,
granularity,
the operationality
duced by an assessment
procedure.
ous systems’ assessment
procedures
is given in Table 2. (The table
scribed in the previous
section,
system,
aa.9essment procedure,
description
which is described
a mea-
we describe
and certainty
measurements
A comparison
-
pro-
includes
the systems deas well as the MetaLEX
in the next
section.)
where
there
exist
several
synonymous,
(i.e., most effective) description.
Additionally,
continuousvalued assessment facilitates
attempts
to guide the search
through
concept
of progress
of vari-
along these dimensions
In situations
operational
descriptions
of the target
concept,
a metric
on operationality
enables the system to learn the “best”
PRODIGY
valued
description
through
is one
assessment.
sess how efficient
space by providing
a measure
the space.
system
that
In other
a given
features
words,
description
continuous-
PRODIGY
can
as-
is for the purposes
of recognition.
The system bases
a priori estimate
of the matching
this assessment
on an
costs associated
with
each
machine;shop
“observable”
feature
in the
ment. Given two synonymous,
the target concept, PRODIGY
erational
using the matching
Certrainty:
in the measurement
ment procedure.
cost estimates.
A dimension
produced
Values:
None of the systems
environ-
operational
descriptions
of
evaluates which is more opcharacterizing
confidence
by the operationality
unguaranteed
surveyed
can guarantee
racjr of their assessment measurements,
systems directly assess operationality.
assess-
or gumznleed.
the accu-
because none of the
In other words, the
Keller
485
systems do not actually
test whether
a given description
is “efficient to use for recognizing
instances’9.
Instead, the
systems
base their
assessments
being assessed is contained
of operational descriptions,
on whether
lem solving
the description
language
within a pre-defined
which is specified by the learn-
The problem
sumptions
mance
is that
upon
which
invalid.
system’s
the
the
designer
based
capabilities
change
discussed
an example
from
in the previous
guage consists
section,
of descriptions
solving
as more chunks
behavior
SOAR’s
are learned
correctly
deteriorate
and matching
costs
lancondi-
and thus
However,
are preSOAR’s
over time
increase.
(This type of difficulty has been documented
- with plans,
rather than chunks - for STRIPS-type
problem
solvers
involving any ar[Minton, 19851.) E ventually, descriptions
bitrary
chunked
condition
will no longer
be efficient
to use.
At this point, it may be necessary
to redefine operationality so that only the “most efficient”
chunks are permitted within operational
descriptions.
But SOAR lacks the
proper perspective
over its own problem
behavior to identify that the operational
tion has become
not automatically
so changing
solving/learning
language defini-
invalid over time. Moreover,
SOAR can“recompile”
a’new language
definition,
the definition
requires
human
MetaLEX
climbing
algorithm
from Table
intervention.
2, the MetaLEX
program
is
in a different equivalence
class with respect to these three
dimensions.
The next section discusses MetaLEX
and its
treatment
of operationality.
efficiency
The MetaLEX
program [Keller, 1987b,
successor to LEX2,
designed to explore
Keller, 1987a] is a
the concept oper-
ationalization
paradigm introduced
in [Keller, 19831. MetaLEX
approaches
essentially
the same learning
task as
LEXP, but solves it using a different technique.
In particMetaLEX
set of USEFUL
forward
486
problem
search.
operational
and LEX2
target
Both
solving
systems
concept
from the
out
operators
used
jectives
performance
and the other
require
SOLVER’s
that
problem
formance
learn
a description
moves
start
description
to execute
with
the
(“a USEFUL
Machine learning & Knowledge Acquisition
of the
during
same
initial
to search
the
on how MetaLEX
description.
guides
efficiency
establish
concept,
SOLVER
benchmark
calculus
nonprob-
per-
over the description
search.
inserts
that
SOLVER’s
problems.
ob-
improve
while also maintain-
for a description
MetaLEX
and observes
The
and effectiveness
a metric
the hill-climbing
one involving
description
efficiency
To assess operationality
FUL
system,
effectiveness.
an operational
The
measures
space that
involving
solving
ing its effectiveness.
of the USE-
description
behavior
During
into
on a set of
SOLVER’s
execu-
tion, the USEFUL
description
guides expansion
of problem solving moves:
any move recognized
as USEFUL
is
expanded, while other moves are pruned.
SOLVER’s
efficiency and effectiveness
are monitored
during the problem solving session.
If SOLVER’s
performance
meets the
Table
3: MetaLEX
Operationality
Definition
Given:
problem
solving
Performance
problem
Description
moves
of class of USEFUL
to expand
during
SOLVER,
system:
solving
Performance
ular, both
searches
of traversing
search
a forward
those problems
Then: the USEFUL
if it satisfies
1. usability:
SOLVER
2. utility:
search
system
objectives:
Improve
SOLVER’s
efficiency on a set of benchmark
calculus
by X%, while maintaining
its effectiveness
MetaL
V.
descrip-
of an operational
description
We do not describe the hill-
or the
Concept description:
As is evident
concept
space in this paper.
Instead,
we focus
assesses the operationality
of a concept
ified for the SOLVER
As
operational
production
target
for ac-
The definition of operationality
used by MetaLEX
is
given in Table 3. Note that there are two objectives
spec-
result
for clarification.
involving
will likely
as-
language
no longer
SOAR
In contrast,
to as-
The
state”),
descrip-
methods
is used as a means
description
in the direction
using a form of hill-climbing.
as the perfor-
over time.
tions that have already been chunked,
sumed efficient to use in recognition.
problem
the
In fact 9 this is inevitable
differ is in their
region. The explanation
the search space.
pos-
performance
is that the original language definition
identifies operational
descriptions.
Consider
language
original
the systems
Where
to a solution
an operational
space, with the initial target concept description
anchoring
the search in the non-operational
region and a training instance description
anchoring the search in the operational
descripfeatures
the lan-
of schemata
with using a “compiled”
sess operationality,
may become
composed
tion.
it into
tion using explanation-based
methods.
In a sense, LEX2
conducts a bidirectional
search of the concept description
recognition.
In a sense, the language is a “compiled” form
of the designer’s knowledge about the performance
system
guage includes any description
sessed by the system.
eventually
to transform
complishing
the transformation.
LEX2 transforms
the initial
ing system’s human designer.
This language includes only
terms that the designer decides can be evaluated efficiently
by the performance
system in order to accomplish
instance
[Keller, 1987b]. For LEX2, that language includes
tions expressed in terms of a variety of “syntactic”
of calculus problem solving states.
For GENESIS,
move leads
and attempt
run-time
problems
in solving
correctly
description
the following
is considered
operational
two sequirements:
the description
is usable
(i.e.,
evaluable)
by
and
using
the
description,
achieves an X% improvement
in effectiveness.
SOLVER’s
without
efficiency
a deterioration
established
performance
tion is considered
fails to attain
objectives,
operational.
the desired
levels
descrip-
this research
was provided
perform=ce
and DARPA
contract
the USEFUL
If SOLVER’s
of efficiency
In the terminology
of the previous section, MetaLEX’s
operationality
assessment
method can be characterized
as:
-
because
operationality
pends on the current state of SOLVER
performance
- because
degree of effectiveness
lems solved);
assessment
its performance
However,
MetaLEX’s
expensive
because
each
aLEX
time
a measure
seconds)
and the
of benchmark
assessment
SOLVER,
meets
prob-
assessment
costs
is required.
by estimating
system
1987a]
shop, University
perfortest.
uary 1987.
As such, operationality
for an explanation-based
system (or more generally,
for a knowledge
tion system) to “learn.” Yet current methods
operationality
tacitly
depend on simplifying
is
associated
with violated
assumptions
by defining operationality directly in terms of the performance
system. The
of operationality
in MetaLEX
may
provide
in-
sight into how to construct
general purpose explanationbased learning systems systems that are more sophisticated in their operationality
assessment
capabilities,
and
that function properly over time, and for tasks other than
“efficient
instance
Technical
recognition.
[Minton,
directing
this
encouragement
this paper.
research.
Discussions
ify the presentation
provided
Smadar
and helpful
with
of PRODIGY
formatting
Kedar-Cabelli
comments
Steve
and SOAR.
assistance.
Funding
June
Work-
1987.
#ML-TR-7.
CA, August
Improving
In
learning.
explanation-based
Corvallis,
19821
September
T. M. Mitchell.
1986.
Generalization
as search.
18(2):203-226,
March
Intelligence,
et aI., 19861 T.
M. Mitchell,
S. T. Kedar-Cabelli.
tion: a unifying view.
of
Proceedings of the
Compilation, Oregon State
Workshop on Knowledge
University,
1985.
the effectiveness
1982.
R. M. Keller,
and
Explanation-based
generaliza&earning, l(l), 1986.
Machine
[Mooney and DeJong,
19851 R. Mooney
and G. DeJong.
Learning
schemata
for natural
language
processing.
CA, August
[Mostow,
pages
IJCdI-8,
681-687,
19811 D. J. Mostow.
of Task Heuristics
thesis,
Mechanical Transformation
into Operational Procedures. PhD
Comp.Sci.Dept.,
[Rosenbloom
and Laird,
@MU,
19861
1981.
P. S. Rosenbloom
Mapping
explanation-based
Laird.
onto soar. In Proceedings AddI-86,
phia,
[Utgoff,
Los Angeles,
1985.
PA, August
[Winston
Philadel-
Machine Learning ofInductive
Academic,
et al., 19831
AAAI,
1986.
19861 P. E. Utgoff.
Bs’as. Kluwer
and J. E.
generalization
Hingham,
P. H. Winston,
and M. Lowry.
Learning
definitions,
In Proceeds’ngs AdAI-89,
D.C., August 1983.
MA,
T.
0.
physical
examples,
1986.
Binford,
B.
descriptions
and precedents.
pages 433-439,
Washington,
provided
on earlier
Minton
in context.
Selectively
generalizing
plans
In
Proceedings
IJCd I-9,
Los Angeles,
from functional
and to Jack
guidance in
1983.
Learning
Irvine,
19861 S. Minton.
Katz,
Thanks go to my thesis advisor, Tom Mitchell,
Mostow, both of whom provided invaluable
August
learning
&facAine
Report
problem-solving.
In Proceedings
transformafor assessing
performance
assumptions
that likely will be violated
as learning
progresses. The MetaLEX
program circumvents
the problems
handling
Concept
of Galifornia,
19851 S. Minton.
[Mitchell
system.
it means
D.C.,
conAAAI-
1987bj R. M. Keller.
The Role of Explicit Contextual Knowledge in Learning Concepts to Improve
PhD thesis, Rutgers
University,
JanPerformance.
Artificial
Operationality
is the key feature that distinguishes
the output concept description
from the input concept description
by re-expressing
In Proceedings
[Keller,
[Mitchell,
in an explanation-based
at the heart of what
R. M. Keller.
pages 596-599,
Met-
a system
recognition.
89,pages 182-186,Washington,
for
must be tested
in lieu of executing
#DCS83-51523
Learning at the knowl1(3):287-316,
1986.
19831 R. M. K e 11er. Learning
cepts for efficient
[Minton,
is also extremely
system
assessment
possible
by
whether
objectives.
method
the performance
these
mance whenever
is accomplished
and observing
the stated
operationality
reduces
yields
(in CPU
(in number
- because
executing
[Keller,
In Proc. 4th International
and
e “guaranteed
[Dietterich,
1986] T. G. Dietterich.
edge level. Machine Learning,
[Keller,
of the degree of efficiency
directly
assessment
deand the current
objectives;
@ “continuous”
grant
or effective-
ness, MetaLEX
can evaluate how far from operational
the
description
is, and can assess whether the operationalization search is headed in the right direction.
~9 “dynamic”
by NSF
#N00014-85-K-0116.
drafts
helped
of
clar-
Chun Liew
to support
Keller
487