From: AAAI Technical Report SS-95-07. Compilation copyright © 1995, AAAI (www.aaai.org). All rights reserved.
On Solving the Qualification Problem
Charles Elkan
Department of Computer Science and Engineering
University of California, San Diego
elkan@cs.ucsd.edu
ABSTRACT: The qualification problem may be the most
fundamental difficulty in formalizing common sense knowledge in general, and in formalizing knowledge about action
in particular. This position paper argues that the qualification
problem is intrinsicallycomputational as opposed to representational. This does not imply that explicit, formal, knowledge
has no role in the production of commonsensical behaviour,
but it does imply that the qualification problem cannot be
solved using currently available representation formalisms.
For a future representation formalism to help in solving the
qualification problem, it will be vital for it to possess a context
mechanism.
Artificial intelligence research has a history of wishful thinking. In the 1950s the hope was that the right heuristic search
algorithms, if only they could be found, would suffice to explain and duplicate human problem-solving skills.
Another hope arose after it became clear that large quantities of initial knowledge were necessary in problem-solving:
that this knowledge could be acquired by a natural language
understanding program which could be written after relatively
simple principles of syntax and semantics were elucidated.
Neither of these hopes has been realized, but wishful thinking springs eternal, and many researchers still hope that it is
possible somehow to obtain the benefits of possessing a large
mass of common sense knowledge without actually stating all
the knowledge formally.1
Lenat and Guha call attempts of this nature “free lunch tries”
[Lenat and Guha, 1990]. In the knowledge representation
community we are justifiably skeptical of the free lunch tries
of others, but we are still lured by a hope of our own. This is
the hope that the right nonmonotonic logic can, in and by itself,
solve dilemmas of common sense reasoning. The two bestknown examples of the inferences that nonmonotonic logics
are supposed to sanction concern a canary and a turkey:
“Tweety is a bird; therefore Tweety can fly.”
“Loading then shooting kills Fred; therefore loading,
then waiting, then shooting kills Fred.”
The desire for a formalism where these inferences are legitimate is the desire for a formalism where appropriate conclusions can be drawn even in the absence of full knowledge of
all relevant circumstances.
The qualification problem, as commonly understood, is
that in almost all common sense scenarios, it is impossible
to state all relevant circumstances. In other words, general
rules of common sense are always incomplete.2 However
many preconditions have been taken into account in the formalization of a rule, one can think of further preconditions
whose truth influences the truth of the consequent of the rule,
so the formalization should mention even more antecedents.
The first example of nonmonotonic reasoning above illustrates this phenomenon. The general rule that birds fly in
actuality has many unstated further antecedents, such as that
the birds in question are not penguins. The second example of nonmonotonic reasoning above is commonly viewed
as illustrating the frame problem, but the frame problem can
be viewed as a version of the qualification problem: here for
example, loading then shooting only kills Fred if many other
unstated assumptions are true.
In a nutshell, the frame problem is that one only wants to
state the conditions under which a property of the world does
not persist, leaving the conditions under which it does persist
unstated. We can distinguish weak and strong variants of the
qualification problem. The weak form is the case where the
unstated additional relevant antecedents could in principle
be made explicit, while the strong form is the case where
these antecedents could not plausibly be enumerated even in
principle. An instance of the frame problem is an instance of
the weak qualification problem if the conditions under which
world properties persist can in principle be enumerated; if
they cannot, one has an instance of the strong qualification
problem.
The conventional intuition behind the search for a nonmonotonic logic as a solution to the qualification problem is
that rules of common sense express “default” truths. According to this intuition, the rule that “if you turn the key, the car
starts”, which ignores further preconditions that the battery
should not be dead, the fuel pump should not be clogged, the
gas should contain no sugar, and so on, should be formalized
along the following lines:
1
“The fundamental issues of AI can only be solved with an orchestrated application of fuzzy logic, neural networks, genetic algorithms, and probabilistic reasoning.” [Vadiee and Jamshidi, 1994]
2
This general rule itself is also incomplete. Some rules of common sense are definitional and therefore can be specified completely.
no unstated precondition can defeat the rule. For any knowledge base including (1), if the knowledge base entails turn-key
and :dead-battery and :clogged-fuel and :sugar-in-gas then
it entails car-starts with certainty. The statement of rule (1)
excludes all potential qualifications except those stated explicitly. Other nonmonotonic logics with similar minimizing
semantics behave similarly. Yet the qualification problem is
precisely that the common sense rule behind (1) does have
unknown preconditions.5
Leaving open the question of the representational adequacy
of the default logic approach to solving the qualification problem, it is still apparent that the solution is computationally inadequate. The real issue from a practical perspective is not so
much how to enumerate the preconditions that are relevant to
a rule, but when to assume that they hold or do not hold. Consider reasoning with a knowledge base containing (1). Before
the default logic rule is used, the falsity of all defeating preturn-key & ˜dead-battery & ˜clogged-fuel
conditions must be checked one by one. This computational
& ˜sugar-in-gas -> car-starts
expense is precisely what a solution to the qualification problem should avoid. Certainly when a human turns an ignition
default-false: dead-battery, clogged-fuel, key and expects a car to start, he does not devote any time to
sugar-in-gas
enumerating reasons why the car may not start.6
Assertions of this type can be directly translated into assertions
If the computational dimension of the qualification probin default logic [Reiter, 1980]. The example above becomes
lem is the most fundamental, then perhaps explicit logical
knowledge has no place in the production of commonsensical
turn-key : :dead-battery :clogged-fuel :sugar-in-gas
behaviour? Several researchers, for example Brooks [1987],
(1)
have indeed reached this conclusion. Typically they advocar-starts
cate improvisation instead of logical reasoning about action.
There is no need to state explicitly that dead-battery is false
Moreover, there is wide agreement that the traditional view
by default, because the semantics of default logic makes it
of planning (dominant from the work on STRIPS [Fikes et
false automatically, if it can consistently be false. The same
al., 1972] to the development of TWEAK [Chapman, 1987])
applies to clogged-fuel and so on.3
is no longer tenable, because of intrinsic uncertainty in the
As a first approximation, we can identify three dimensions
environment and the need to react quickly to events.
of adequacy along which a candidate solution to a common
However, the search for some sort of logic to use in formalsense formalization problem can be judged [Elkan, 1991].
izing common sense knowledge and reasoning about action is
In summary, the solution is intuitively (philosophically) adstill a sensible enterprise. Even a weak ability to forecast the
equate if what is stated is clear, and clearly correct. The
consequences of actions before undertaking them can clearly
solution is representationally adequate if it is concise, modube adaptive. An improvising agent is doomed to choose aclar, and it can be combined without change with the solutions
tions whose optimality is only local, and in many domains,
to other common sense formalization problems. Finally, the
goals can only be achieved by choosing actions whose role
solution is computationally adequate if ecologically important
is indirect. Also, the biological evidence is that declarative,
problems can be solved efficiently using the solution.4
propositional knowledge does play a major part in directing
According to these criteria, the solution outlined above to
human behaviour. Neuroanatomical research has identified a
the problem of formalizing common sense knowledge about
distinct region of the brain that is responsible for acquiring
when a car starts is inadequate. It is inadequate on the intudeclarative memories of facts and events [Squire and Zolaitiveness dimension because all potential qualifications to the
5
basic rule “if you turn the key, the car starts” have been made
An unknown precondition is very different from a precondition
whose truth is unknown. It is of no help in solving the qualification
explicit. The semantics of default logic formally entails that
defaultrule129: turn-key -> car-starts
dead-battery cancels default129
clogged-fuel cancels default129
sugar-in-gas cancels default129
...
Specifying preconditions separately from a basic rule unfortunately involves reifying rules, which is technically difficult.
Even in a second-order language where quantification over
predicates as well as individuals is allowed, it is not possible
to assign a name such as default129 to an entire sentence,
and to write other sentences using the name. Another way
of stating default knowledge is therefore used in practice. In
this approach relevant preconditions are brought into the formalization of the common sense rule, and these preconditions
are stated to be typically false using a special operator of the
nonmonotonic logic. Informally, the following is written:
3
The fact that no axiom need be written to require that a
proposition-symbol be false by default implies that no propositionsymbol can be introduced that is not false by default. This is a significant apparent limitation on the expressiveness of default logic.
Recent work by Gelfond and Lifschitz [1991] and others has been
directed towards lifting this restriction, allowing the writer of a set
of axioms to choose for each atomic literal whether it should be
false by default or unknown by default (which is the semantics of
a newly introduced literal in standard monotonic logics). However
the resulting logics are technically no more expressive than standard
default logic.
4
“Behind all logic and its seeming sovereignty of movement, too,
there stand valuations or, more clearly, physiological demands for
the preservation of a certain type of life.” [Nietzsche, 1886]
problem to be able to choose whether a precondition should have
truth-value false or unknown by default. If the truth of a precondition
in the formalization of a common sense rule is set unknown by
default, then in the absence of further information (which is the
normal case) the rule will not be usable for drawing any inferences.
And the strong version of the qualification problem is that many
relevant preconditions cannot be stated at all in advance.
6
It is not enough to reply that the preconditions can be checked
once and for all, in a preprocessing phase. Each episode of trying
to start a car actually involves different instances of the propositions
dead-battery etc. If a well-accepted first-order variant of default
logic existed, it would have been used above, but one of the technical difficulties with default logic (which other nonmonotonic logics
share) is that it is difficult to extend the semantics from the propositional case to the full first-order case.
Morgan, 1991].
Given that it is useful to search for an appropriate logic for
formalizing common sense knowledge, what features should
this logic possess? The analysis above suggests that the logic
should possess a context mechanism. The preconditions that
may defeat a common sense rule are assumptions on which
the rule’s validity rests. The typical case is that all these assumptions do hold. This typical case should be handled computationally without enumerating all the assumptions, which
can be achieved by treating the collection of assumptions as
a unit. A collection of assumptions is a context.
The notion of context is intrinsically dynamic. One enters a context, one remains in a context, one changes context if the current context is untenable. Traditionally, logic
has had a static bias, but some logicians are developing dynamic formalisms where contexts play a central role [van
Benthem, 1991]. The motivation of their work is linguistic:
to model how a speech act changes the information state of a
hearer. What is suggested here is different in two ways, but
should lead to formalisms with similar technical features. The
two differences are first that environmental events, not speechacts, are what changes the information-state of an agent, and
second, that the state of an agent to be modelled is its “available” information rather than its entire long-term knowledge.7
Contexts are of course not a new idea in artificial intelligence. The idea of relying on assumptions that are left
unproven appears in [Doyle, 1983; Feldman and Rich, 1986]
and elsewhere. Truth maintenance systems have been called
context maintenance systems, and are widely used as a pragmatic solution to the qualification problem [de Kleer, 1986;
Petrie, Jr., 1989]. A context mechanism is an important recent addition to the CYC system [Guha and Lenat, 1992],
and there is recent work on formalizing contexts [Buvac and
Mason, 1993].
No doubt there are many further features that a logic for
formalizing everyday declarative knowledge should possess,
but a context mechanism permitting solutions to the qualification problem to be implemented is already a substantial
research topic.
References
[Bjork and Bjork, 1992] R. A. Bjork and E. L. Bjork. A
new theory of disuse and an old theory of stimulus fluctuation. In Alice F. Healy, Stephen M. Kosslyn, and
Richard M. Shiffrin, editors, From learning processes to
cognitive processes: Essays in honor of William K. Estes,
volume 2. Lawrence Erlbaum Associates, Hillsdale, New
Jersey, 1992.
[Brooks, 1987] Rodney A. Brooks. Planning is just a way of
avoiding figuring out what to do next. Technical Report
303, Artificial Intelligence Laboratory, MIT, September
1987.
7
One of the prominent differences between biological and artificial memory systems is that the short-term retrieval strength of
memories decays rapidly in humans and animals. A recent theory of
memory suggests that this phenomenon is adaptive: for an organism
whose reasoning resources are limited, it is advantageous for only a
subset of its long-term knowledge to be immediately available, with
recent experiences determining which knowledge is available [Bjork
and Bjork, 1992].
[Buvac and Mason, 1993] Sasa Buvac and Ian Mason.
Propositional logic of context. In Proceedings of the National Conference on Artificial Intelligence, Washington,
D.C., August 1993.
[Chapman, 1987] David Chapman. Planning for conjunctive
goals. Artificial Intelligence, 32:333–377, 1987.
[de Kleer, 1986] Johan de Kleer. An assumption-based TMS.
Artificial Intelligence, 28:127–162, 1986.
[Doyle, 1983] Jon Doyle. Some theories of reasoned assumptions: An essay in rational psychology. Technical Report 83-125, Department of Computer Science, CarnegieMellon University, May 1983.
[Elkan, 1991] Charles Elkan. Formalizing causation in firstorder logic: Lessons from an example. In Working Notes
of the AAAI Spring Symposium on Logical Formalizations
of Commonsense Reasoning, Stanford University, March
1991.
[Feldman and Rich, 1986] Yishai A. Feldman and Charles
Rich. Reasoning with simplifying assumptions: A methodology and example. In Proceedings of the National Conference on Artificial Intelligence, pages 2–7, August 1986.
[Fikes et al., 1972] Richard E. Fikes, Peter E. Hart, and
Nils J. Nilsson. Learning and executing generalized robot
plans. Artificial Intelligence, 3:251–288, 1972.
[Gelfond and Lifschitz, 1991] Michael Gelfond and Vladimir Lifschitz. Classical negation in logic programs and
disjunctive databases. New Generation Computing, 9(3–
4):365–85, 1991.
[Guha and Lenat, 1992] R. V. Guha and D. B. Lenat. Language, representation and contexts. Journal of Information
Processing, 15(3):340–349, 1992.
[Lenat and Guha, 1990] Douglas B. Lenat and Ramanathan V. Guha. Building Large Knowledge-Based Systems: Representation and Inference in the Cyc Project.
Addison-Wesley Publishing Company, 1990.
[Nietzsche, 1886] Friedrich Wilhelm Nietzsche. Beyond
Good and Evil; Prelude to a Philosophy of the Future.
1886. Translated, with commentary, by Walter Kaufmann.
Vintage Books, 1966.
[Petrie, Jr., 1989] Charles J. Petrie, Jr. Reason maintenance
in expert systems. Künstliche Intelligenz, (2):54–60, 1989.
[Reiter, 1980] Raymond Reiter. A logic for default reasoning.
Artificial Intelligence, 13:81–132, 1980.
[Squire and Zola-Morgan, 1991] Larry R. Squire and Stuart
Zola-Morgan. The medial temporal lobe memory system.
Science, 253:1380–1386, September 1991.
[Vadiee and Jamshidi, 1994] Nader Vadiee and Mohammad
Jamshidi. The promising future of fuzzy logic. IEEE
Expert, 9(4):36–38, August 1994.
[van Benthem, 1991] Johan van Benthem. Language in Action: Categories, Lambdas, and Dynamic Logic. NorthHolland, Amsterdam, 1991.