From: AAAI-02 Proceedings. Copyright © 2002, AAAI (www.aaai.org). All rights reserved.
Development and Deployment of a Disciple Agent
for Center of Gravity Analysis
Gheorghe Tecuci1,2, Mihai Boicu1, Dorin Marcu1, Bogdan Stanescu1, Cristina Boicu1,
Jerry Comello2, Antonio Lopez2, James Donlon1,2, William Cleckner2
1
Learning Agents Laboratory, Computer Science Department, MS 4A5, George Mason University, 4400 University Dr, Fairfax, VA 22030
{tecuci, mboicu, dmarcu, bstanesc, ccascava}@gmu.edu, http://lalab.gmu.edu
2
Center for Strategic Leadership, US Army War College, 650 Wright Ave, Carlisle Barracks, PA 17013
ComelloJ@awc.carlisle.army.mil, tlopez@xula.edu, james.donlon@us.army.mil, william.cleckner@csl.carlisle.army.mil
Abstract
This paper presents new significant advances in the Disciple
approach for building knowledge-based systems by subject
matter experts. It describes the innovative application of this
approach to the development of an agent for the analysis of
strategic centers of gravity in military conflicts. This
application has been deployed in several courses at the US
Army War College, and its use has been evaluated. The
presented results are those of a multi-faceted research and
development effort that synergistically integrates research in
Artificial Intelligence, Center of Gravity analysis, and
practical deployment of an agent into Education.
1 Introduction
The Learning Agents Laboratory of George Mason
University is developing a theory, methodology and a
family of tools, called the Disciple approach, for building
instructable knowledge-based systems or agents (Tecuci
1998). This effort directly addresses the knowledge
acquisition bottleneck which we consider to be one of the
major barriers in the development and maintenance of
Artificial Intelligence applications. The Disciple approach
relies on a powerful learning agent shell that can be trained
to solve problems by a subject matter expert, requiring only
limited assistance from a knowledge engineer. As an expert
system shell (Clancey 1984), the Disciple learning agent
shell includes a general problem solving engine that can be
reused for multiple applications. In addition, it includes a
multistrategy learning engine for building the knowledge
base through a mixed-initiative interaction with the subject
matter expert.
As the Disciple approach evolved, we have developed a
series of increasingly advanced learning agent shells
forming the Disciple family. The most recent family
member, Disciple-RKF, represents a significant
advancement over its predecessors: Disciple-WA (Tecuci
et al. 1999) and Disciple-COA (Tecuci et al. 2001). All
these three systems were developed as part of the “High
Copyright © 2002, American Association for Artificial Intelligence
(www.aaai.org). All rights reserved.
Performance Knowledge Bases” and “Rapid Knowledge
Formation” programs supported by DARPA and AFOSR
(Burke 1999). Both programs emphasized the use of
innovative challenge problems to focus and evaluate the
research and development efforts.
Disciple-RKF is the result of a multi-objective
collaboration between the Learning Agent Laboratory of
George Mason University and the Center for Strategic
Leadership of the US Army War College that
synergistically integrates research in Artificial Intelligence
(AI), with research in military Center of Gravity (COG)
analysis (Clausewitz 1832), and the practical use of agents
in education. The AI research objective is to develop the
technology that will enable subject matter experts who do
not have computer science or knowledge engineering
experience to develop instructable agents that incorporate
their problem solving expertise. The COG research
objective is to clarify and formalize the process of the
identification of the centers of gravity for enemy and
friendly forces at the strategic and operational levels of
war, and to enable the development of an intelligent
assistant for solving this complex problem. Finally, the
educational objective is to enhance the educational process
of senior military officers through the use of intelligent
agent technology. Each of these three objectives is
recognized as important and difficult in its own right. Our
experience with addressing them together in a synergistic
manner has resulted in faster progress in each of them.
Moreover, it offers a new perspective on how to combine
research in AI, with research in a specialized domain, and
with the development and deployment of prototype systems
in education and practice.
The paper presents the current status of this research and
development effort. The next section presents the COG
challenge problem. This is followed by an end-user
perspective on the developed Disciple-RKF/COG agent.
Section 4 presents an overview of the Disciple-RKF shell
and its use to build the Disciple-RKF/COG agent,
emphasizing its new capabilities with respect to the
previous Disciple shells. Then section 5 discusses the
deployment and evaluation of Disciple in two courses at the
US Army War College, “Case Studies in Center of Gravity
IAAI-02
853
Analysis,” and “Military Applications of Artificial
Intelligence.” The paper concludes with a summary of the
synergistic aspects of this collaborative work.
2 The Center of Gravity Problem
The military literature distinguishes between three levels of
conflicts: a strategic level focusing on winning wars, an
operational level focusing on winning campaigns, and a
tactical level focusing on winning battles. One of the most
difficult problems that senior military leaders face at the
strategic level is the determination and analysis of the
centers of gravity for friendly and opposing forces.
Originally introduced by Clausewitz in his classical work
“On War” (1832), the center of gravity is now understood
as representing “those characteristics, capabilities, or
localities from which a military force derives its freedom of
action, physical strength, or will to fight” (Department of
the Army 2001). A combatant should eliminate or influence
the enemy’s strategic center of gravity, while adequately
protecting its own (Giles and Galvin 1996).
Correctly identifying the centers of gravity of the
opposing forces is of highest importance in any conflict.
Therefore, in the education of strategic leaders at all the US
senior military service colleges, there is a great emphasis
on the center of gravity analysis (Strange 1996). COG
determination requires a wide range of background
knowledge, not only from the military domain, but also
from the economic, geographic, political, demographic,
historic, international, and other domains. In addition, the
situation, the adversaries involved, their goals, and their
capabilities can vary in important ways from one scenario
to another. When performing this analysis, experts rely on
their own professional experience and intuitions, without
following a rigorous approach. Recognizing these
difficulties, the Center for Strategic Leadership of the US
Army War College started in 1993 an effort to elicit and
formalize the knowledge of a number of experts in center
of gravity. This research resulted in a COG monograph
(Giles and Galvin 1996), which provided a basis for the
application of Disciple to this high value application
domain, and to the development of the Disciple-RKF/COG
instructable agent presented in the next section.
3 A Disciple Agent for COG Analysis
Disciple-RKF/COG is an agent used in the US Army War
College course titled “Case Studies in Center of Gravity
Analysis.” In this course Disciple-RKF/COG supports the
students to develop a center of gravity analysis report for a
war scenario.
First, a personal copy of Disciple-RKF/COG guides the
student to identify, study and describe the aspects of a
scenario (such as the 1945 US invasion of the island of
Okinawa) that are relevant for COG analysis. The studentagent interaction takes place as illustrated in Figure 1. The
left part of the window is a table of contents, whose
elements indicate various aspects of the scenario. When the
student selects one such aspect, Disciple asks specific
questions intended to acquire from the student a description
Figure 1: Scenario specification interface
854
IAAI-02
of that aspect, or to update a previously specified
description. All the answers are in natural language.
Taking the Okinawa_1945 scenario as our example,
Disciple starts by asking for a name and a description of the
scenario, and then asks for the opposing forces. Once the
student indicates Japan_1945 and US_1945 as opposing
forces, Disciple includes them in the table of contents,
together with their characteristics that the student needs to
specify (see the left hand side of Figure 1). Then, the
student may click on any of these aspects (e.g. “Industrial
capacity” under “Economic factors” of Japan_1945) and
the agent guides the student in specifying it. The student’s
specification may prompt additional questions from
Disciple, and a further expansion of the table of contents.
An orange, yellow, or white circle marks each title in the
table of contents, indicating respectively that all, some, or
none of the corresponding questions of Disciple have been
answered. The student is not required to answer all the
questions and Disciple can be asked, at any time, to
identify and test the strategic center of gravity candidates
for the current specification of the scenario.
The right hand side of Figure 2 shows some of the
solutions generated by Disciple for the Okinawa_1945
scenario. Each solution identifies an entity as a strategic
COG candidate and then indicates whether or not it can be
eliminated. In the case of Japan_1945, some of the
identified strategic center of gravity candidates are
Emperor Hirohito, Japanese Imperial General Staff, the
industrial capacity of Japan, and the military of Japan.
When a solution is selected in the right hand side of the
problem solving interface, its justification, at various levels
of abstractions, is displayed in the left hand side.
The left-hand side of Figure 2 shows the detailed
justification for the identification and testing of Emperor
Hirohito as a strategic COG candidate. Disciple uses the
task reduction paradigm to perform the top level problem
solving task: “Identify and test a strategic COG candidate
for the Okinawa_1945 scenario.” To perform this task,
Disciple asks itself a series of questions. The answer of
each question allows Disciple to reduce the current task to
a simpler one, until Disciple has enough information to first
identify a strategic COG candidate, and then to determine
whether it should be eliminated or not.
Emperor Hirohito is identified as a strategic COG
candidate for Japan_1945 in the Okinawa_1945 scenario
because, as the feudal god-king of Japan, he is its main
controlling element. After being identified as a candidate,
Emperor Hirohito is analyzed based on various elimination
tests, but he passes all of them. Because the people of Japan
see him as divine, and his will is actually their will,
Emperor Hirohito could impose them to accept the
unconditional surrender of Japan, which is the main
strategic goal of the US. As god-king of Japan and
commander in chief of the military, he can also impose his
will on the military of Japan. Also, as head of the
government, he can impose the government of Japan to
accept unconditional surrender. Being able to impose his
will on the Clausewitz’s trinity of power (people, military
and government), Emperor Hirohito is very likely to be the
strategic center of gravity of Japan in 1945.
Figure 2: The problem solving interface of Disciple-RKF/COG
IAAI-02
855
As another example, consider the industrial capacity of
Japan_1945, which is another source of strength, power
and resistance because it produces the war materiel and
transports of Japan. Disciple, however, eliminates this
strategic COG candidate, because the military and the
people of Japan_1945 are determined to fight to death and
not surrender even with diminished war materiel and
transports.
In the example scenario portrayed here, Disciple
eliminates all but two candidates for Japan -- Emperor
Hirohito and the Japanese Imperial General Staff -- and
suggests that the student should select one of them as the
strategic Center of Gravity of Japan in 1945. It is important
to point out that this example is only a representative
approach to the analysis of Japan’s center of gravity for the
Okinawa campaign. We recognize that subject matter
experts often differ in their judgments as to the
identification and analysis of center of gravity candidates
for any particular scenario. The important point for agent
development is that the Disciple agent can accommodate
the preferences of the expert who teaches it.
To our knowledge, this is the first time that an intelligent
agent for the strategic COG identification and testing has
been developed. More details about its specific use in the
COG and MAAI classes are presented in section 4.
4 Agent Development with Disciple-RKF
The Disciple-RKF/COG agent presented in the previous
section was developed by using the Disciple-RKF learning
agent shell, as will be described in this section. DiscipleRKF consists of an integrated set of knowledge acquisition,
learning and problem solving modules for a generic
knowledge base having two main components: an object
ontology that defines the concepts from a specific
application domain, and a set of task reduction rules
expressed with these concepts. Disciple-RKF represents a
significant evolution compared to the previous Disciple
shells. It implements more powerful knowledge
representation and reasoning mechanisms, and has an
improved interface that facilitates mixed-initiative
reasoning. Even more significantly, Disciple-RKF
incorporates new modules that allow a subject matter
expert to perform additional knowledge engineering tasks,
such as scenario specification, modeling of his problem
solving process, and task formalization.
In general, the process of developing a specific
knowledge-based agent with Disciple-RKF consists of two
major stages: 1) the development of the object ontology by
the knowledge engineer and the subject matter expert, and
2) the training of Disciple by the subject matter expert.
In the first development stage, a knowledge engineer
works with a subject matter expert to specify the type of
problems to be solved by the Disciple agent, to clarify how
these problems could be solved using Disciple’s task
reduction paradigm, and to develop an object ontology.
A fragment of the object ontology developed for the
COG domain is shown in Figure 3. The object ontology
856
IAAI-02
<object>
Scenario
Major_theater_
of_war_scenario
Agent
Force_goal
Small_scale_
contigency_scenario
Controlling_element
Strategic_COG_relevant_factor
Political_factor
Governing_body
Other_relevant_factor
Economic_factor
Geographic_factor
Established_governing_body
Ad_hoc_
governing_body
Resource_or_
infrastructure_element
Historic_factor
Demographic_factor
Military_factor
Civilization_factor
Psychosocial_factor
State_government
Group_
governing_body
Other_type_of_
governing_body
Other_political_
factor
International_
factor
Figure 3: A fragment of the COG object ontology
consists of hierarchical descriptions of objects and features,
represented as frames, according to the knowledge model
of the Open Knowledge Base Connectivity (OKBC)
protocol (Chaudhri et al. 1998). Disciple-RKF includes
several types of ontology browsers and editors that
facilitate the ontology development process. The careful
design and development of the object ontology is of utmost
importance because it is used by Disciple as its
generalization hierarchy for learning.
A new capability of Disciple-RKF is that ontology
development includes the definition of elicitation scripts for
objects and features. These scripts guide the expert to
define the instances that occur in a scenario (such as
Okinawa_1945 or Emperor Hirohito, as illustrated in
Section 3). Figure 4 shows the elicitation scripts associated
with the “Scenario” object. The top script specifies the
question to be asked by Disciple to elicit the name of the
scenario, how the user’s answer should be used to update
the ontology, what other scripts should be called after
updating the ontology, and even the appearance of the
Scenario
elicitation_scripts
Script: Elicit instances of Scenario
Control:
Question: “Provide a name for the scenario to be analyzed:”
Answer variable: <scenario-name>
Default value: new-scenario
Control type: single-line
Ontology action:
<scenario-name> instance-of Scenario
Script calls:
Elicit the superconcepts of the instance <scenario-name> of Scenario
Elicit the features of the instance <scenario-name> of Scenario
Script: Elicit the superconcepts of the instance <scenario-name> of Scenario
Control:
Question: “What kind of scenario is” <scenario-name> “?”
Answer variable: <scenario-type>
Default value: Major_theater_of_war_scenario
Control type: single-selection-list
Possible values: the elementary subconcepts of Scenario
Ontology action:
<scenario-name> instance-of <scenario-type>
Script: Elicit the features of the instance <scenario-name> of Scenario
Script calls:
Elicit the feature brief_description for <scenario-name>
Elicit the feature description for <scenario-name>
Elicit the feature has_as_opposing_force for <scenario-name>
Figure 4: Sample elicitation scripts
<object>
Scenario
Force
Force_goal
Strategic_COG_relevant_factor
resource_ or_
infrastructure_element
Strategic_ Operational_
goal
goal
Other_relevant_factor
Demographic_factor
Civilization_factor
Psychosocial_factor
Economic_factor
Historical_factor
Geographical_factor
International_factor
Object
ontology
Political_factor
Military_factor
Scenario specification
Modeling
Task and rule learning
Task and rule refinement
Problem solving
<object>
Object
ontology
Scenario
Force
Force_goal
Other_relevant_factor
Civilization_factor
Psychosocial_factor
Economic_factor
Historical_factor
Geographical_factor
insta nc e_ of
Scenarios
(elicited)
resource_ or_
infrastructure_element
Strategic_COG_relevant_factor
Strategic_ Operational_
goal
goal
Demographic_factor
International_factor
Political_factor
Military_factor
Sic ily_1 943
insta nce _of
B rita in_ 194 3
ha s a s
oppo sing
fo r ce
co m ponen t_
st ate
ins tance _of
instanc e _o f
An glo_a llie s_ 194 3
ins tanc e _of
Allie d_f or ce s_o per ation_H usky
type _ of_ ope ra tions
“ WW II Allie d inva sion
o f Sicily in 194 3”
insta nce _ of
US_19 4 3 ha s_a s_ indust r ial_f ac to r
insta nce _ of
U S_7 th _Ar m y_
(F o rc e_ 3 43)
insta nc e_o f
inst anc e_ of
B r _ 8 th _A rm y_
( F or ce _545 )
has _as_subg r oup
inst anc e_ of
has_as _subgr oup
has_a s_subgr o up W e ste rn _Nav al_T F
c o mpone nt_
stat e
has_ as_pr im a ry_
f or c e_ ele m ent
br ief _de scr ipt io n
has _as_s ubgr oup
has_a s_subgr o up
ha s_ a s_ subg r oup
Ea st e rn_N ava l_TF
instanc e _o f
US_9 th _Air _F o r ce
IF the task is
IF the task is
Identify the strategic COG candidateswithrespect to the
Identify thestrategic COG candidates withrespect to the
industrial civilization of a statewhich isa member of aforce
industrial civilization of a state which is amember of a force
IF the task is
IF the task is
Thestateis ?O2
The state is ?O2
Identify the strategic COG candidates withrespect to the
Identify thestrategicCOG candidates with respect tothe
Theforceis ?O1
The forceis ?O1
industrial civilization of a state which is amember of aforce
industrial civilization ofa state which isa member ofa force
IFsthe
is
IFstate
the task
explanationThe
explanationThe
tatetask
is ?O2
is ?Ois
2
Identify
the?O1
strate
Identify
strategic
COG candidateswith respect to the
?O2 has_as
_ind
ustrial_fac
tor gic
?O3COG candidates withrespect to the
?O2 has_as
_indus
?O3
The
force
is
The
forctrial_factor
e the
is ?O1
industrialra
ctor_of
ivilization
industrial civilization
?O3 is_a_major_gene
?O4
?O3 is_a_major_genera
tor_
of ?O4
IF state
the
task
isof a state which is amember of a force
IFsta
the
task
isof a statewhichisa member of aforce
explanationThe
explanationThe
is ?goo
O2ds_or_materiel
teis ?O2
?O4 ISstrategically_e
ssential_
?O4 ISstrategically_e
ssential_
aterie
l
Identif
y the
strategic COG candidateswith respect to the
Identify
thegoods_or_m
strategic COG
candida
teswithrespect to the
?O2 has_as_
indus
tria
?O2 has_as_industria
The
force
isl_fac
?O1tor ?O3
Theforceisl_factor
?O1 ?O3
indust
rial civilization
industrial civilization
?O3Upper
is_a_ma
jor_genera
tor_of
?O3Upis_
a_major_generato
?O4
Plausible
Bound
Con
IFdition
the
task?O4
isof a statewhichisa member ofa force
Plausible
per
Bo
und Condition
IFstate
ther_of
task
isof a state which is amember of aforce
pla
nation
nation
Thessential_
stateis ?O2
Theesse
is ?O2
?O4Force
ISex
strategic
ally_e
or_materie
l
?O4 Force
ISexpla
strategica
lly_
ntial_goods_or_materiel
?O1 IS
?O1 IS
Identify thegoods_
strategic
COG candidates
with respect to the
Identify
the strategic COG candidates withrespect to the
?O2 has_aThe
s_industrial_fac
?O2 has_as
_indus
trial_fac
force is ?O1tor ?O3
The
force
is ?O1tor ?O3
?O2 IS Force
?O2 IS Force
industrial
civilization
of a statewhichisa member of aforce
industrial
civilization
of a state which is amember of aforce
?O3Upper
is_a_major_genera
tor_of
?O4
?O3Uppe
is_a_m
ajor_ge
nerator_of
?O4
Plausible
Bound
Condition
Plausible
r Bou
nd Condition
IF?O3
the
is
IF?O3
task
is
expla
nation
expla
nation
has_as_indu
strial_fac
tor
has_as_industrial_fa
ctor
The
sta
tetask
is ?O2
The
sthe
tatetial_
is ?O2
?O4 Forc
IS
strategic
ally_e
ssential_
goods_
or_
materie
l
IS strateg
ally_e
ssen
goods_
or_materiel
?O1 IS
e has_as
?O1 ?O4
IS Forc
e haic
Identify
the?O1
strate
COG
candida
teswithrespect to the ?O3 IS
Identif
y is
the?O1
strate
?O2
_indus
trial_fac
tor gic
?O3
?O2
_ind
ustrial_fac
tor gic
?O3COG candidates withrespect to the
?O3 IS Industrial_fa
ctor The
Industrial_fa
ctors_as
force
is
The
force
?O2 IS Forc
e is_a_m
?O2 IS Forc
e is_a_m
industrial
civilization
indust
civilization
?O
3
ajor_
genera
tor_of
?O3
ajor_genera
tor_of
is_a_ma
jor_
nerato
r_of
?O4
jor_
ge
nerator_of
?O4
Pla
usge
ible
Upper
Bound
Condition
IF?O3
the
task?O4
isof a state which is amember of aforce is_a_ma
Pla
us
ible
Upper
Bound
Corial
IF
ndition
the
task?O4
isof a state which is amember of a force
expla
nation
xpla
nation
has_a
s_industrial_fac
tor
has_
as_industria
l_factor
Thees
state
isteriel
?O2 or_materiel
The?O3
state isteriel
?O
IS
strategic
ally_
sential_goods_
?O4
ISestra
tegic
ally_e
ssentia
l_?O2
goo
ds_
or_materie
l
?O4 IS Strate
ntial_goods_o
_ma
Strategically
ential_goods_o_ma
?O1gically
IS 4_esse
Force
?O1 Industrial_factor
IS _ess
Force
Identify
the strategic
COG candidates withrespect t?O4
o the IS
Identif
y the
strate
gic
COG candidate
s with respect to the
?O2 has_as
_indus
trial_fac
?O2 has_as_
indus
tria
?O3 IS
Industrial_fac
tor The
?O3 IS
force
is ?O1tor ?O3
The
force
isl_fac
?O1tor ?O3
?O2
ISjor_
Force
ISjor_genera
Force
industrial
civilization
of a state which is amember of aforce ?O2
indust
rial civilization
ofa statewhichisamember of a force
?O3
is_a_m
ajor_ge
nerator_of
?O4
?O3
istor_of
_a_ma
jor_genera
tor_of
?O4
is_a_ma
ge
nerator_of
?O4
is_a_ma
?O4
Pla
usCondition
ible
Upper
Bound
Condition
Plaus
ible
Uustrial_fac
pper
Bound
PlausibleLower
Bound
IF?O3
task
is
Plausible Lower
Bound
ndition
IF?O3
the
task
is
expla
nation
la
nattor
io
nCondition
has_a
s_industrial_fa
ctor
has_aCo
s_ind
The
sthe
tatetial_
isteriel
?O2
The
state
is riel
?O2
?O4Force
IS
strateg
ically_e
ssen
goods_or_materiel
?O4
ISexp
strategic
ally_e
ssential_
goods_or_materiel
IS Strate
esse
ntial_goods_o_ma
?O
4 Anglo_allies_
IS Stra
tegica
lly_es
sential_goo
ds
_o_
mate
?O1gically_
IS
?O1
IS
Force
?O1 ?O4
IS Anglo_allies
?O1
ISthe
1943
Identif
y is
the?O1
strate
ct to
Ide
ntify
tis
he?O1
strategic
COG candidateswithrespect to the
?O2ctor
has_as
_ind
ustrial_fac
tor gic
?O3COG candidates withrespe
?O2ctor
has_as
trial_factor
?O3
?O3 IS _1943
Industrial_fa
?O3 IS Ind
ustrial_fa
The
force
T_indus
he
force
2 IS Force
?O2 IS Force
?O2 IS US_1943?O
?O2 ISa forc
USe_1943
indust
civilization
ind
ustrial
civilization
?O3Upper
is_a_m
ajor_genera
tor_of
?O3
is_a_major_genera
tor_
of ?O4
is_ra_ma
jor_
generator_of
?O4
is_a_ma
jor_
ge
nerator_of
?O4
PlausibleLowe
Bound
Pla
usCondition
ible
Bound
Corial
IF
ndition
the
task?O4
isof a state which is amember ofha
Plausible
Lower
Bound
Pla
usible
Cond
ition
Upper
Bound
Condition
IF the
task
isof a statewhich isa member of aforce
xpla
nation
expla
nattor
ion
s_industrial_fac
tor
has_
as_
indus
tria
l_fac
has_as_
industrial_
factor
?O3
s_as_
industria
l_
facto
r ?O4
?O3
The?O3
state isteriel
The?O3
stateis ?O2
?O4
ISestra
tegic
ally_e
ssentia
l_?O2
goo
ds_or_materie
l
IS
strategic
ally_e
ssential_
aterie
l
IS Shas_a
trategically
sential_g
oods_o_ma
?O4
IS
Strate
gically
_esse
_o_materiel
?O1_1943
IS _es
Force
?O1s_19
IS43tria
Forc
e2ntial_goods
?O1In?O4
IS Anglo_allies
?O1Industrial_capacity
IStheA?O3
nglo_
allie
Identify
the
strategic
COG candida
tes with
to
Identify
thegoods_or_m
strategic COG
candida
teswithrespect to the
?O2ctor
has_as_
indus
tria
?O
has_as
_indus
trial_fac
?O3 IS
Industrial_fa
IS
Indus
l_fa
ctor
?O3 IS
dustrial_cap
acity_of_US_
1943
?O3respect
IS
_of_U
S_1943
The
force
isl_fac
?O1tor ?O3
The
force
?O1tor ?O3
IS Force
IS Forc
e3 is_a_major_genera
IF
the
task
is
IFisthe
task?O4
isof a state which is amember of aforce
?O2 IS US_1943?O2
?O2 of
ISaforce
US_1943?O2
indus
trial
civ
ilizat
ion
of
a
st
ate
whic
h
is
a
member
industrial
civilization
?O3
is_a_ma
jor_genera
tor_of ?O4
?O
tor_of
is_a_ma
jor_
ge
nerator_
of
?O4
is_a_ma
jor_ge
nerator_of
?O4
is_a_ma
jor_generator_of
?O4
is_a_major_generator_of
?O4
Plaus
ible
Uustrial_fac
pper
Bound
P
la
uss_industrial_factor
ible
Upper
Bound
Condition
Plausible
Lower
Bound
Co
ndition
Plausible
Lower
Bohas_a
und
exp
la
nattor
ionCondition
explanation
has_a
s_ind
?O3
?O3
Identif
y is
the?O2
strategic
COG candidates
spe
ct ?O4
to_indus
the
Iden
tify
the
strategic
COG
candidates
with respect to the
has
industrial_
factor
?O3
ha
s_as
ctorCondition
?O3
The
state
state
is
?O2
?O4
ISUS
strategic
ssential_
goods_
or_materie
l ?O4 with
?O
4_esse
ISUS
strategically_e
ssential_
goods_
or_m
aterie
l
?O
IS Strate
gically
_esse
ntial_goo
ds_o_ma
teriel
IStrial_fa
Strate
ally
ntial_goods_o
_ma
teriel
?O4 IS Wa
r__as_
materiel_
and_tran
sports
_of_
_194
3 ally_e
IS re
Wa
r_
mate
and_transports
_of_
_19
43 The
IS
Force
?O1gic
IS
Force
?O1
IS4 Anglo_allies_
1943
?O1
ISriel_
An
glo_allies
_1943
?O2ctor
has_as
_indus
trial_factor
?O2
has_as
_in
dus
trial_fac
tor ?O3of astate which is a member of aforce
?O3 ?O1
IS ity_of_US_
Ind
ustrial_fa
?O3
IS
Industrial_fac
indust
rial
a mem
ofa forc
e pac
indus
trial
ivilization
?O3 IS Industrial_capac
1943
?O3
ISberIndus
trial_ca
ity_of_US
_19
43 tor
The
force
iscivilization
?O1 ?O3of a statewhichis
The
force
isc?O1
?O2
IS Force
?O2
IS Force
?O2
IS US_1943
?O2
IS r_
US
_1943tor_of
?O3
is_a_major_genera
tor_is
of?O2
?O4
?O3Upper
istor_of
_a_m
ajor_gen
tor_of
?O4
is_a
_ma
jor_
ge
nerator_of
?O4
is_a_ma
jo
r_
genera
?O4
Thestate
Theera
stat
e is?O2
is_a_major_generato
r_Bound
of Pla
?O4
is_a_majo
genera
?O4
THEN
usible
Upper
Bound
Condition
THEN
Pla
us
ible
Bound
Condition
Plausible
Lower
Condition
Plausible
Lower
Bound
Condition
has_
as_
indus
tria
l_factor
?O3
has_a
s_industrial_fa
ctor
?O3
ex
planation
expla
nation
ha
s_as_
industria
facto
r ?O4
?O3
has
_as_
industrial_factor
?O3
IS
strategic
ssential_
goods_or_materie
l ?O4thaIS
?O4
IS
strategic
ally_e
ssen
goods_or_materiel
?O4
ISisl_
_esse
ntial_goods
?O4
IS is aStrate
esse
ntial_goods_o_ma
teriel
The_o_materiel
force is ?O1
The
forc
etial_
is
?O1
?O1gically
IS_of_
Forc
e
?O1gically_
IS COG
Force
?O4thaIS
orts
US
_194
3 ally_e
W?O1
ar_mate
l_and_transports_of_
US_
1943
Conclude
t?O3
an War_
econom
ic
tor
aStrate
strategic
Ctrial_fa
OG
t an economic
fact
or
strategic
?O1materiel_and_transp
ISfacAnglo_
allies
ISrie
Anglo_allies
?O2
has_
as_industrial_fac
tor ?O3 Conclude
?O2
has_as
_ind
ustrial_fac
tor ?O3
?O3
IS _1943
Indus
ctor
?O3apacity_of_US_
IS _1943
Industrial_fa
IS
Industrial_capacity
_of_U
S_
1943
?O3
IS Indust
rial_c
1943ctor
candidatefor a state?O2
whichISis aUS
member
ofa_ma
aforc
eForc
for a state?O2
whichISis aUS
mem
be?O
ris_
of2a_ma
a forc
eForce
ISjor_ge
e is_a_
ISjor_
_1943?O2
_1943
expla
nat
ion
tion
is_
nerator_of
?O4
ge
nerator_of
?O4
?O3
major_
ge
nerator_of ?O4 candidate
?O3
isexplana
_a_m
ajor_genera
tor_of ?O4
is_a_ma
jor_generator_of
?O4
THEN is ?O2is_a_major_generator_of
P
la?O4
us
ible
Upper
Bo
und
Condition
THEN
Pla
usCondition
ible
Upper
Bound
Condition
Plausible
Lower
Bo
und
Condition
Plausible
Lower
Bound
has_a
s_industrial_fac
tor
?O3
has_a
s_industrial_fac
tor
?O3
Thestate?O4
The
state
is
?
O2
has
_as_
industrial_
factor
?O3
has
_as_
in
dustrial_
factor
?O3
?O
2
has_as
_indus
trial_fac
tor
?O3
?O2
ha3s_a
s_ind
ustriateriel
l_f
actor
?O4
ISUS
strate
gically_essential_
goods_or_materiel
?O4
ISUS
stra
tegic
ally_e
ssentia
l_goo
ds_?O3
or_materiel
?O4
ISglo_a
ntial_goods_o
_materiel
?O4
IS
_ess
ential_goods_o_ma
IS
Wa
r_omic
mate
and_transports
_of_
_19
43
?O4 ISan Wa
r_materiel_
and_transports
_194
Conclude
t an econ
fact
or
is Strate
astrategic
C_esse
OG
Conclude
ec?O1
onomic
r is3aStrategically
strate
gic
COG
?O1
IS
Force
?O1
IS_of_
Force
?O1
ISriel_
An
lliesgically
_1943
ISfacto
Anglo_allies
The
forceistha
?O1
forceisthat
?O1
?O3
IS
Industrial_fac
?O
IS _1943
Industrial_fa
?O3 IS
Indust
rial_capacity_of_US
_1943
?O3 IS
Industrial_cap
acity_of_US_
1943
?O3 tor
is_a_major_generator_of ?O4 The
?O3ctor
is_a_major_generator_of ?O4
candidatefor a state?O2
which
membe
r ofa_ma
a forc
eForce
candidate for astate?O2
whichISisaUS
member
ofa_ma
aforce
ISjor_
ISjor_
Force
ISis aUS
_1943?O2
_1943?O2
is_
ge
nerator_of
?O4
is_
ge
ne
?O4_essential_goods_or_materiel
Theeconomic
?O3major_gene
The
eco
nomic
isis_a_
?O3major_generator_of
THENfactor isis_a_
Plausible
Bound
Condition
THstate
EN
Plaus
ible
Urator_of
pper
Bound
rator_of
?O4
?O4
?O
4Upper
IS strategic
ally_
es
sential_goods_
or_m
aterie
l factor
?O4
ISstrategically
Plausible
Lower
Bound
Condition
Pla_as_
usible
Lower
Bound
Co
ndition
has_a
s_industrial_fa
ctor
?O3
has_a
s_ind
ustrial_fac
torCondition
?O3
The state is ?O2
The
is ?O2
has
_as_
industrial_factor
?of_US
O3esse_194
has
industrial_
factor
?O3
?O4
IS is aStrate
teriel
?O
4 Anglo_allies_
ISisaStrate
gically
_esse
ntial_goods_o_ma
teriel
?O4 IS
Wa
r_materiel_
and_transports_
3
?O4 IS
Wa
r_materiel_
and_tran
sports
US_194
3
Conclude
t an economic
or
strategic
Conc
lude
t an econom
ic
strategic
COG
?O1gically_
IS COG
Forcential_goods_o_ma
?O1
IS_of_
Force
?O1
ISfact
Anglo_allies
_1943
?O1
ISfactor
1943
The
forceistha
?O1
The
forc
e istha
?O1
?O3apacity_of_US
IS
Industrial_fa
?O3
ISity_of_
Indu
strial_fa
?O3 IS
Indust
rial_c
_1943ctor
?O3 IS
Industrial_capac
US_
1943ctor
Pla
ibleUpper Bound Condition
Plaus
ible Upper Bound Condition
IS us
?O2
IS
candidate
forfac
a state
whic
hISis aUS
mem
ber?O2
ofa_ma
a forc
eForce
candidate
forfac
astate
whic
hISisaUS
membe
risof
ama
fo
rc
eForce
?O2
_1943
?O2
_1943
The
economic
tor is
?O3major_generator_of
The
economic
tor is
?O3
is_
jor_
generator_of ?O4
_a_
jor_
generator_of ?O4
THEN
THEN
is_a_
?O4
is_
a_major_generato
r_o
f ?O1
?O4
Plausible
Lowe
r Bound
Condition
Plaus
ible
Lower
Bound
Condition
?O
ISindust
Force
ISindus
Force
The state is ?O2
Thestateis ?O2
has1_as_
rial_factor ?O3 teriel
has_gically
as_
trial_factor ?O3
has
_as_
industrial_
?O3
has
_as
_indu
strial_
facto
rorts
?O3
?O4
ISisaSfactor
trategically
_ess
?O4
IS
_esse
o_materiel
?O4thaIS
Wa
r_materiel_
and_transports
USential_goods_o_ma
_1943
?O4 ISan War_
materiel_
and_transp
_of_
US_ntial_goods_
1943
Conclud
t an econom
ic
strate
gic
COG
Conclude
ec?O1
onom
ic
or
is Strate
astrategic
Ctria
OG
?O1
ISfactor
Anglo_allies
ISfactAnglo_allies
_19
?O
IS_of_
Force
?O2_of_U
IS43
Force
The
forcee
is ?O1
The
forceisthat
?O1
?O3 acity_of_US_
IS 2_1943
Indust
rial_fac
tor
?O3
IS
Indus
l_factor
?O3
IS
In
dustrial_cap
1943
?O3
IS
Industrial_capacity
S_
1943
candidate for astate?O2
which isaUS
mem
be3r of aforce
candidatefor a state?O2
which is aUS
member
of aforc
e_as
_194
_1943is_
has_a
s_industrial_fac
has
_indu
strial_factor
The economic
is ?O3IS
Theeconomic
factor is?O3IS
is_a_m
ajor_gene
rator_of tor
?O4 ?O3
a_ma
jor_ge
nerator_of
?O4 ?O3
TH
ENfactor
THEN
?O4
?O4
Pla_as_
usible
Lower
Bound
Co
ndition
Plausible
Lower
Bou
nd
Condition
The
state
is ?O2is_a_major_generator_of
The
stateis ?O2is_a_major_generator_of
has
industrial_
factor
?O3essential_goods_o_m
has
_as_
indus
trial_
fa
?O3
?O
ISsports
Industrial_fa
ctor
?O3
ISctor
Indus
trial_
?O4
ISis3aStrateg
ic
ally_
ateriel
?O4
IS
gic
ally
ntial_goods_o
_materiel
?O4
IS
Wa
r_materiel_
and_tran
_of_
US_194
3
IS
Wa
r_omic
mate
and_transports
_of_
USfactor
_19
43
Conc
lude
t an econom
ic
factor
strategic
COG
Conclude
t an econ
fact
or
is Strate
astrategic
C_esse
OG
?O1
IS
Anglo_allies_
1943
?O1
ISriel_
An
glo_allies
_1943
The
force
istha
?O1
The
force?O4
istha
?O1
?O3 IS Industrial_capac
ity_of_US_ge
1943
?O3 IS Industrial_capacity_o
f_U
S_1943
ajor_g
candidate
for astatewhic
hISisaUS
membe
ris_
ofa_major_
aforce nerator_of ?O4
candidateforfactor
a state
which
membe
ris
of_a_m
a forc
e enerator_of ?O4
_1943
?O2
ISis aUS
_1943
The
econo
mic
?O3
is
?O3major_ge
is_
a_major_generato
of
?O4
is_a_
nerator_of
?O4
TH
ENfactor is?O2
THEN
Plausible
B
ound
Plausible
Lower
Condition
?O
4Lower
ISr_l_
Strate
_essential_goods_o_materiel Theeconomic
?O4
IS Bound
Strategically_es
Thestateis ?O2
The state is ?O2
ha
s_as_
industria
factogically
rCondition
?O3
has
_as_
industrial_factor
?O3 sential_goods_o_materiel
?O4 ISan War_
materiel_
and_transp
orts
_of_
US_1943
?O4 IS
Wa
r_materie
l_
and_
_of_
US_1943
Conclude
econom
ic
or
is astrategic
COG
Conclude
t an economic
or
is atransports
strategic
COG
?O1
ISfactAnglo_allies
_1943
?O1
ISfact
Anglo_allies
_1943
The
forceisthat
?O1
The
forceistha
?O1
?O3 IS
Industrial_capacity
_of_U
S_1943
?O3 IS
Indust
rial_c
apacity_of_US_
1943
candidatefor a state?O2
whichIS
aUS_1
member
of Bound
aforceCondition
candidatefor a state?O2
whichIS
aUS
mem
be
forcde Condition
Pis
lausible
Lower
Pis
lausib
le
Lowe
Boun
943
_19
43r ofr a
Theeconomic
?O3major_generator_of
The economic
factor isis_a_
?O3ma
THENfactor isis_a_
THEN
?O4
jor_generator_of ?O4
Thestateis ?O2
The
state
is
?O2
?O1
IS
Anglo
_allies
_194
3
?O1
IS
Anglo_
allie
s_19
43
has_
as_indus
trial_factor ?O3
has
_as_industrial_
factor ?O3US_1943
?O4 IS
Wa
r_omic
mate
riel_or
and_transports
US_1943
?O4 IS
r_materiel_
and_transports
Conclude
t an econ
is astrategic_of_
COG
Conclude
t an Wa
economic
factor
is943
a strategic_of_
COG
The
forceistha
?O1
The
force istha
?O1
?O2
ISfact
USl_c
_1943
?O2Ind
IS
US_1
?O3 IS
Indus
tria
apacity_of_US_1943
?O3 IS
ustrial_capa
city_of_US_1943
candid
ateforfactor
a state
which
is_as_
amembe
r of a fac
forctor
e
candidate
forfactor
astate
wh
ichha
is
a mem
bertrial_fac
of a forctor
e
The
economic
isis_
?O3
The
economic
isis_a_
?O3
has
industrial_
s_as
_indus
THEN
THEN
a_ma
jor_generator_of
?O4 ?O3
major_generator_of
?O4 ?O3
The state?O4
is ?O2?O3War_ma
The state?O4
is ?O2?O3Wa
IS Indust
rial_capacity_of_US
_1943943
IS
In
dus
trial_ca
_of__of_
US_19
terie
l_a
s_of_US_1
r_materiel_a
nd_trans
US43
_1943
Conclude thaIS
t an economic
fact
ornd
is_transport
astrategic
COG
ConcludethaIS
t an econom
ic
factor
isapacity
straports
tegic
COG
The forceis ?O1
The force is ?O1
erator_of
_major_
generator_o
candidatefor a stateis_a_
whicmajor_gen
h is amem
ber of a forc?O4
e
candidate for astateis_a
whic
h isa membe
r of afforc?O4
e
The economic
tor is
The economic
factor is
?O3
THE
Nfac
TH
EN
?O4
IS?O3
War_materiel_and_transports_of_US_1943
The
state
is ?O2
The
sta
te?O4
is ?O2IS War_materiel_and_transports_of_US_1943
Conclude
that an economic factor isastrategicCOG
Conclude
The
force is ?O1
The
forceisthat
?O1an economicfactor is astrategic COG
THEN
THEN
candidate
forfactor
a state
ichisa member of a force
candidate
forfactor
a state
The
economic
iswh
?O3
The
economic
iswhic
?O3h is amember of aforce
Conclud
e tha
t an economic factor isa strategicCOG
Conclude
t an economic factor is a strategic COG
The
state is
?O2
The
stateis tha
?O2
candidate
candidate
The
force isfor
?O1astatewhich isa member of aforce
The
forceis for
?O1a state which is a memberof a force
state
is ?O2
Thestate
is?O2
The ecThe
onomic
factor
is ?O3
Theeconomic
factor
is?O3
Theforce is ?O1
Theforce is?O1
Theeconomic factor is ?O3
Theeconomicfactoris?O3
insta nc e_o f
“c ombine d and j oint o per ations”
No rt hwe st_ Af ric a_A ir _Fo r ce
General
tasks
and
rules
(learned)
Figure 5: The main phases of the agent training process
interface. The use of the elicitation scripts allows a
knowledge engineer to rapidly build customized interfaces
for Disciple agents, such as the one illustrated in Figure 1,
thus effectively transforming this software development
task into a knowledge engineering one.
The result of the first development stage is a customized
Disciple agent. This agent is trained to solve problems by a
subject matter expert, with very limited assistance from a
knowledge engineer, in the second major stage of agent
development. Figure 5 shows the main phases of the agent
training process, which starts with a knowledge base that
contains only a general object ontology (but no instances,
no problem solving tasks, and no task reduction rules), and
ends with a knowledge base that incorporates expert
problem solving knowledge.
During the Scenario specification phase, the Scenario
Specification module (which is a new module of DiscipleRKF) guides the expert in describing the objects that define
a specific strategic scenario (e.g. the US invasion of the
island of Okinawa in 1945). The expert does not work
directly with the object ontology in order to specify the
scenario. Instead, the expert-agent interaction takes place as
presented in section 3 and illustrated in Figure 1, this all
being directed by the execution of the elicitation scripts.
Experimental results show that the experts can easily
perform this task.
After the expert has specified the Okinawa_1945
scenario, he can start the Modeling of his COG reasoning
for this particular scenario. The expert uses the Modeling
module (which is another new module of Disciple-RKF) to
express his reasoning in English as a sequence of task
reduction steps like the ones illustrated in the left hand side
of Figure 2. An example of one task reduction step, defined
during modeling, is illustrated in the left hand side of
Figure 6. The top task is the current task that needs to be
reduced. The expert has to define a question that is relevant
to the reduction of this task, then answer the question, and
reduce the top task to a simpler one that incorporates the
information from the answer. Experimental results show
that this is the most challenging agent training activity for
the expert.
In the Task and rule learning phase, Disciple learns
general tasks and general rules from the task reduction
steps defined in the modeling phase. For instance, consider
the reduction step from the left hand side of Figure 6,
consisting of a task, a question, an answer, and a subtask.
Because all these expressions are in natural language, the
expert and the agent collaborate to translate them into the
formal logical expressions on the right hand side of Figure
6. First the natural language expression of each task is
structured into an abstract phrase called the task name,
which does not contain any instance, and several specific
phrases representing the task’s features. The formalization
is proposed by the agent and may be modified by the
expert. Next the expert and the agent collaborate to also
formalize the question and the answer from the left hand
side of Figure 6 into the explanation on the right hand side
of Figure 6. This explanation represents the best
approximation of the meaning of the question-answer pair
that can be formed with the elements of the object
ontology. In essence, the agent will use analogical
Natural Language
Test whether the Will_of_the_People_of_US_1945
can cause US_1945 to accept
US_giving_honorable_end_of_hostilities_to_Japan
Question:
Does the Will_of_the_People_of_US_1945 have the power to
cause the Military_of_US_1945 to accept
US_giving_honorable_end_of_hostilities_to_Japan?
Answer:
Yes, because US_1945 is a representative democracy and the
Will_of_the_People_of_US_1945 dictates the
Will_of_the_Military_of_US_1945
Test whether the Will_of_the_People_of_US_1945, that can
impose its will on the Military_of_US_1945,
can cause US_1945 to accept
US_giving_honorable_end_of_hostilities_to_Japan
Logic
Test whether the will of the people can cause a state to accept a goal
The will of the people is Will_of_the_People_of_US_1945
The state is US_1945
The goal is US_giving_honorable_end_of_hostilities_to_Japan
Explanation:
US_1945 has_as_governing_body Government_of_US_1945
Government_of_US_1945 IS Representative_democracy
US_1945 has_as_military_force Military_of_US_1945
Military_of_US_1945 has_as_will Will_of_the_Military_of_US_1945
Will_of_the_Military_of_US_1945 reflects Will_of_the_People_of_US_1945
Test whether the will of the people, that can impose its will on the
military, can cause a state to accept a goal
The will of the people is Will_of_the_People_of_US_1945
The military is Military_of_US_1945
The state is US_1945
The goal is US_giving_honorable_end_of_hostilities_to_Japan
Figure 6: Mixed-initiative language to logic translation
IAAI-02
857
Test whether the ?O1 can cause ?O2 to accept ?O3
Test whether the will of the people can cause a state to
accept a goal
The will of the people is ?O1
The state is ?O2
The goal is ?O3
Plausible Upper Bound Condition
?O1 is Strategic_cog_relevant_factor
?O2 is Agent
is Strategic_cog_relevant_factor
?O3 is Force_goal
Plausible Lower Bound Condition
?O1 is Will_of_the_People_of_US_1945
?O2 is US_1945
?O3 is US_giving_honorable_end_of_hostilities_to_Japan
Figure 7: Task learned from the example in Figure 6
reasoning and guidance from the expert to propose a set of
plausible explanation pieces from which the expert will
select the most appropriate ones (Tecuci et al. 2001).
Based on the formalizations from Figure 6 and the object
ontology from Figure 3, the Disciple agent learns the
general task shown in Figure 7 and the general rule shown
in Figure 8. Both the learned task and the learned rule have
an informal structure (that preserves the natural language of
the expert and is used in agent-user communication), and a
formal structure (that is used in the internal formal
reasoning of the agent).
Initially, when the agent has no rules and no tasks, the
expert teaches Disciple how to solve problems and Disciple
generates partially learned tasks and rules, as indicated
above. As Disciple learns from the expert, the interaction
between the expert and Disciple evolves from a teacherstudent interaction, toward an interaction where both
collaborate in solving a problem. During this mixedinitiative Problem Solving phase, Disciple learns not only
from the contributions of the expert, but also from its own
successful or unsuccessful problem solving attempts.
The learned formal rule in Figure 8 includes two
applicability conditions, a plausible upper bound (PUB)
condition, and a plausible lower bound (PLB) condition.
The PUB condition allows the rule to be applicable in
many analogous situations, but the result may not be
correct. The PLB condition allows the rule to be applicable
only in the situation from which the rule was learned. The
agent will apply this rule to solve new problems and the
feedback received from the expert will be used to further
refine the rule. In essence, the two conditions will converge
toward one another (usually through the specialization of
the PUB condition and the generalization of the PLB
condition), both approaching the exact applicability
condition of the rule. Rule refinement could lead to a
complex task reduction rule, with additional Except-When
conditions which should not be satisfied in order for the
rule to be applicable. Tasks are refined in a similar way.
It is important to stress that the expert does not deal
directly with the learned tasks and rules, but only with their
examples used in problem solving. Therefore, the complex
knowledge engineering operations of defining and
debugging problem solving rules are replaced in the
858
IAAI-02
Disciple approach with the much simpler operations of
defining and critiquing specific examples.
After the Disciple agent has been trained, it can be used
in the autonomous problem solving mode, to identify and
test the strategic COG candidates for a new scenario, as
was illustrated in Section 3.
5 Deployment and Evaluation
of Disciple-RKF/COG
The US Army War College regularly offers the courses
“Case Studies in Center of Gravity Analysis” and “Military
Applications of Artificial intelligence.” In the first course
(the COG course), the students use Disciple-RKF/COG as
an intelligent assistant that supports them to develop a
center of gravity analysis report for a war scenario, as
described in section 3. In the second course (the MAAI
course), the students act as subject matter experts that teach
personal Disciple-RKF agents their own reasoning in
IF: Test whether the ?O1 can cause ?O2 to accept ?O3
Question: Does the ?O1 have the power to cause the ?O4 to
accept ?O3?
Answer: Yes, because ?O2 is a representative democracy
and the ?O1 dictates the ?O5
THEN: Test whether the ?O1, that can impose its will on the
?O4, can cause ?O2 to accept ?O3
IF: Test whether the will of the people can cause a state to
accept a goal
The will of the people is ?O1
The state is ?O2
The goal is ?O3
Explanation
?O2 has_as_governing_body ?O6
?O6 IS Representative_democracy
?O2 has_as_military_force ?O4 has_as_will ?O5 reflects ?O1
Plausible Upper Bound Condition
?O1 is Will_of_agent
?O2 is Force
has_as_military_force ?O4
has_as_governing_body ?O6
?O3 is Force_goal
?O4 is Military_force
has_as_will ?O5
?O5 is Will_of_agent
reflects ?O1
?O6 is Representative_democracy
Plausible Lower Bound Condition
?O1 is Will_of_the_People_of_US_1945
?O2 is US_1945
has_as_military_force ?O4
has_as_governing_body ?O6
?O3 is US_giving_honorable_end_of_hostilities_to_Japan
?O4 is Military_of_US_1945
has_as_will ?O5
?O5 is Will_of_the_Military_of_US_1945
reflects ?O1
?O6 is Government_of_US_1945
THEN: Test whether the will of the people, that can impose
its will on the military, can cause a state to accept a goal
The will of the people is ?O1
The military is ?O4
The state is ?O2
The goal is ?O3
Figure 8: Rule learned from the example in Figure 6
Center of Gravity analysis, as described in section 4.
As briefly illustrated in section 3, Disciple-RKF/COG
guides the student to identify, study and describe the
relevant aspects of the opposing forces in a particular
scenario. Then Disciple identifies and tests the strategic
center of gravity candidates and generates a draft analysis
report that the student needs to finalize. The student must
examine Disciple’s reasoning, correct or complete it, or
even reject it and provide an alternative line of reasoning.
This is productive for several reasons. First the given agent
generates its proposed solutions by applying general
reasoning rules and heuristics learned previously from the
course’s instructor, to a new scenario described by the
student. Secondly, COG analysis is influenced by personal
experiences and subjective judgments, and the student
(who has unique military experience and biases) may have
a different interpretation of certain facts.
This requirement for the critical analysis of the solutions
generated by the agent is an important educational
component of military commanders that mimics military
practice. Commanders have to critically investigate several
courses of actions proposed by their staff and to make the
final decision on which one to use.
During the 2001 academic year, Disciple was
successfully used in both the Winter and Spring sessions of
the COG course. As a result of this initial success, the
USAWC decided to continue and expand the integration of
Disciple in this course for the next academic year and
beyond. At the end of the courses the students completed
detailed evaluation forms about Disciple and its modules,
addressing a wide range of issues, ranging from judging its
usefulness in achieving course’s objectives, to judging its
methodological approach to problem solving, and to
judging the ease of use and other aspects of various
modules. For instance, on a 5-point scale, from strongly
disagree to strongly agree, 7 out of 13 students agreed and
the other 6 s trongly agreed that the Scenario Specification
module should be used in future versions of the course.
Furthermore, 8 out of 11 students agreed, 1 strongly
agreed, and 2 were neutral that subject matter experts who
are not computer scientists can learn to express their
reasoning process using the task reduction paradigm. As
another example, 10 out of 13 students agreed, 1 strongly
agreed, 1 disagreed and 1 strongly disagreed that the
Scenario Specification tool is easy to use.
Several of the students that took the COG course in the
Winter 2001 session, together with additional students,
took the “Military Applications of Artificial Intelligence
Course” in the Spring 2001 session. In this course the
students were given a general overview of Artificial
Intelligence, as well as an introduction to Disciple-RKF.
These students used Disciple-RKF as subject matter
experts. The students were organized in five two-person
teams, with each team given the project to train a personal
Disciple-RKF agent shell according to its own reasoning in
COG identification for its historical scenario. All five
teams succeeded in developing working agents, with each
team addressing a unique scenario: 1) the capture of the
Leyte Island by the US forces in 1944; 2) the Inchon
landing during the Korean War in 1950; 3) the Falklands
war between Argentina and Britain in 1982; 4) the
stabilization mission in the Grenada Island in 1983; and 5)
the US invasion of Panama in December 1989.
In the last two 3-hour class sessions, all the five teams
participated in a controlled agent development experiment
that was videotaped in its entirety. Each team was again
provided with a copy of Disciple-RKF that contained the
generic object ontology from Figure 3, but no specific
instances, no tasks and no rules. They received a 7-page
report describing the Okinawa scenario, and were asked to
train their Disciple agent to identify center of gravity
candidates, based on that scenario. After each significant
phase of agent training and knowledge base development
(i.e. scenario specification, modeling, rule learning, and
rule refinement) a knowledge engineer reviewed their
work, and the team then made any necessary corrections
under the supervision of the knowledge engineer. Each
team succeeded in specifying the scenario and training the
agent, in a very short time, as indicated in Figure 9.
The top table in Figure 9 shows the size of the initial
object ontology. Each team interacted with Disciple to
populate this ontology with different instances and features
representing the Okinawa scenario. After that, each team
taught its Disciple agent to identify COG candidates for
this scenario. The bottom table in Figure 9 indicates both
the time spent by each team, and the number of knowledge
elements defined during this time. On average they defined
85.4 instances and 93.8 feature values in 1 hour and 6
minutes. The average number of rules per team was 18.8,
and the average time interval was 4 hours and 7 minutes.
Although obviously incomplete (both because of the use of
a single training scenario, and because of incomplete
training for that scenario), the knowledge bases were good
enough for identifying correct COG candidates not only for
the Okinawa (evaluation) scenario, but also for “new”
scenarios whose inputs were taken from the class projects.
At the end of this final experiment, the students
completed a detailed questionnaire, containing questions
about the main components of Disciple. One of the most
significant results was that 7 out of the 10 experts agreed, 1
expert strongly agreed and 2 experts were neutral with
respect to the statement: “I think that a subject matter
expert can use Disciple to build an agent, with limited
assistance from a knowledge engineer.” We consider this
experiment to be a very significant success. Indeed, to our
knowledge, this is the first time that subject matter experts
Initial KB
Generic Object Ontology
Concepts
Features
144
79
Okinawa Scenario
General Tasks and Rules
Teams Instances Feat Val
Time
Tasks Rules
Time
Team 1
94
103 1h 21min
18
17 3h 52min
Team 2
78
86
55min
23
22 4h 21min
Team 3
72
79
52min
22
19 4h 35min
Team 4
105
111 1h 23min
18
17 3h 58min
Team 5
78
90
59min
20
19 3h 46min
Average
85.4
93.8 1h 06min
20.2
18.8 4h 07min
Figure 9: Knowledge base development
during the final experiment
IAAI-02
859
have trained an agent their own problem solving expertise,
with very limited assistance from a knowledge engineer.
The deployment and evaluation of Disciple have also
revealed several limitations of this approach and have
provided numerous ideas for improvement. For instance,
while the subject matter expert has an increased role and
independence in agent development, the knowledge
engineer still has a critical role to play. He has to assure the
development of a fairly complete object ontology. He also
has to develop a generic modeling of the problem solving
process based on the task reduction paradigm. Even guided
by this generic modeling, and using natural language, the
subject matter expert has difficulties in expressing his
reasoning process. Therefore more work is needed to
develop methods for helping the expert in this task.
Several other research groups are addressing the problem
of direct knowledge acquisition from subject matter
experts, as part of the DARPA’s “Rapid Knowledge
Formation” program (Burke 1999). These groups, however,
are currently emphasizing the acquisition of textbook
knowledge, relying primarily on reusing knowledge from
existing knowledge repositories. In contrast, the emphasis
of our research is on acquiring expert’s problem solving
knowledge that is not normally represented in textbooks,
and relies primarily on teaching and learning.
6 Conclusions
This paper presented the current status of a multi-faceted
research and development effort that synergistically
integrates research in AI, research in COG analysis, and the
practical application to education.
The AI research in knowledge bases and agent
development by subject matter experts has benefited from
the COG domain that provided a complex challenge
problem. Identification and testing of strategic COG
candidates exemplifies expert problem solving that relies
on a wide range of domain knowledge, a significant part of
which is tacit. This research has also benefited from its
practical application to education. The COG and MAAI
courses allowed us to perform thorough experimentation
with real experts, resulting in the validation of our methods
and providing ideas for future improvements.
The research in COG analysis has benefited from the AI
research in that the agent development has helped clarify
and formalize the COG identification and testing process.
The COG reasoning models developed were validated in
the COG and MAAI classes, and are leading to a
significant revision and improvement of the COG
monograph of Giles and Galvin (1996).
Finally, the innovative application of the AI and COG
research to education through the use of the Disciple agent,
has had a significant impact on improving the COG and
MAAI courses. Done as a very successful experiment in
2001, it was made a regular part of the syllabi for 2002, to
be continued in the following years.
860
IAAI-02
Acknowledgments
This paper is dedicated to Mihai Draganescu who
supported the first research projects on the Disciple
approach at the Research Institute for Informatics and the
Romanian Academy. The research described in this paper
was sponsored by DARPA, AFRL, AFMC, USAF, under
agreement number F30602-00-2-0546, by AFOSR under
grant no. F49620-00-1-0072, and by the US Army War
College. Michael Bowman, Catalin Balan, Marcel
Barbulescu, Xianjun Hao and other members of the
LALAB contributed to the development of Disciple-RKF.
References
Burke, M. 1999. Rapid Knowledge Formation Program
Description, http://www.darpa.mil/ito/research/rkf/
Chaudhri, V. K.; Farquhar, A.; Fikes, R.; Park, P. D.; and
Rice, J. P. 1998. OKBC. A Programmatic Foundation for
Knowledge Base Interoperability. In Proceedings of the
Fifteenth National Conference on Artificial Intelligence,
600–607. Menlo Park, California: AAAI Press.
Clancey, W. J. 1984. NEOMYCIN: Reconfiguring a rulebased system with application to teaching. In Clancey W. J.
and Shortliffe, E. H. eds. Readings in Medical Artificial
Intelligence, 361-381. Reading, MA: Addison-Wesley.
Clausewitz, C.V. 1832. On War, translated and edited by
M. Howard and P. Paret. Princeton, NJ: Princeton
University Press, 1976.
Department of the Army 2001. Field Manual 3-0,
Operations. Washington, D.C.: U.S. Gov. Printing Office.
Giles, P.K., and Galvin, T.P. 1996. Center of Gravity:
Determination, Analysis and Application. CSL, U.S. Army
War College, PA: Carlisle Barracks.
Strange, J. 1996. Centers of Gravity & Critical
Vulnerabilities: Building on the Clausewitzian Foundation
So That We Can All Speak the Same Language. Quantico,
VA: Marine Corps University Foundation.
Tecuci, G. 1998. Building Intelligent Agents: An
Apprenticeship
Multistrategy
Learning
Theory,
Methodology, Tool and Case Studies. London, England:
Academic Press.
Tecuci, G.; Boicu, M.; Wright, K.; Lee, S. W.; Marcu, D.;
and Bowman, M. 1999. An Integrated Shell and
Methodology for Rapid Development of Knowledge-Based
Agents. In Proceedings of the Sixteenth National
Conference on Artificial Intelligence, 250-257. Menlo Park,
California: AAAI Press.
Tecuci G.; Boicu M.; Bowman M.; and Marcu D. with a
commentary by Burke M. 2001. An Innovative Application
from the DARPA Knowledge Bases Programs: Rapid
Development of a High Performance Knowledge Base for
Course of Action Critiquing. AI Magazine Vol. 22, No. 2:
43-61.