Agent-based Composition of
Behavior Models
Katia Sycara (PI)
Start date: 10/02/02
Gita Sukthankar
Anupriya Ankolekar
The Robotics Institute
Carnegie Mellon University
Talk Outline
•
•
•
•
•
•
•
Vision
Limitations of Current Models
Research Objectives
Research Approach
Expected Impact
Accomplishments
Deliverables
Fully automated, high fidelity Computer Generated
Forces have enormous value for military simulation
and training
•
High fidelity CGFs provide realistic adversaries and
team mates
•
•
•
•
Can be used for shipboard and embedded training
•
•
Utilize multi-agent architectures to go beyond current
limited behaviors to adaptive opponent/teammates with
human-like unpredictability
Can learn from experience
Embodying Team behaviors
Training can be conducted using standard computer
equipment (e.g. PCs)
Will be cost-effective and affordable
•
•
Automated CFGs reduce the training manning
requirements
Agent-oriented software engineering techniques promote
modularity and reuse
Limitations of Current Models
• Current CGF training models are limited and
inflexible
– They exhibit a small hard-coded set of behaviors
– They do not allow the coach to easily customise the training
experience
– They are hard to develop and troubleshoot
• Current human performance modeling techniques
–
–
–
–
Have not been successfully scaled to complex tasks
Have not been applied to modeling teams
Models are expensive to construct
Models do not allow reuse
Research Objectives
• Develop techniques that:
– Enable CGFs to increase range of behaviors to
incorporate smart human-like strategies and
adaptation
– Allow efficient reuse and composition of CGF
models
– Allow the development of models of adversaries
and team mates that are consistent with human
behavior modeling
– Reduce model construction time and cost
Research Approach
Integration of multi-agent architectures and software engineering
techniques to increase CGF sophistication and enable reuse
• Leverage our expertise in the development of intelligent agents to
increase the autonomy, range of behaviors and long-term strategic
level thinking of the CGFs
• Use knowledge bases of composable CGF plan fragments that
encapsulate particular behaviors
• Use libraries of reusable software components and connectors to
create executable code
• COTS game engines and state of the art animations provide a
realistic and affordable simulation platform deployable for
classroom, shipboard, and embedded training (PC’s with game
software)
• Demonstration Domain: Urban Warfare
What’s unique about our approach?
• The combination of semantically rich agent
representation and software engineering
development methodology
• The multi-agent architectural approach
enables modeling of team behaviors
• This approach will result in affordable,
coachable teams of realistic training forces
Functional Architecture
Trainer
Plan
Editor
Knowledge
Structures
Belief
Editor
In
te
rn
al
Reasoner
E
ve
nt
s
Reasoner
CGF Model
Trainee
Simulation
Environment
Armies Fight in Teams and so must
their Training Simulations
• Teamwork in Open Environments [Sycara et al.]
incorporates heterogeneous teams and dynamic
team formation
– Teams are not assumed to be fixed in size or team members
abilities
– Model accommodates dynamic role assignment according to
current situation and individual capability
– Model accommodates discovery and incorporation into the
team of new appropriate team members (adapts to the loss
of members)
– Teams can be formed/reformed dynamically during
execution in response to incoming/changing goals and
environment
– Negotiation of team goals and commitments
– Has been applied to Joint Mission Planning (Agent Storm)
Our approach enables reuse at
multiple levels
• Individual CGFs can be adapted for different
scenarios and domains
• Programmers reuse already developed CGF
behavior fragments to construct new CGFs
• Our multi-agent architecture (RETSINA) is a
proven model of software development that
has been reused across multiple domains
Composition
Composition of agents at task level
– SE language: an agent is a computational process (an
“smart” component). An agent can be viewed as a unit of
planning and execution
– Thus, composition of plan fragments and associated code
• Manage interdependencies between plan fragments by
matching preconditions, beliefs, commitments, constraints (at
reactive and cognitive levels)
• Manage interdependencies between code by matching inputs
and outputs
Promising approach from Software Engineering
– Use a library of adapters and connectors to manage
interdependencies and repair violated dependencies
between composed agents
Appropriate representation facilitates reuse and
composition of pre-existing plans
Knowledge base of
pre-developed plan fragments
CLEAR
AREA x
Team formed
Unexplored room
…
…
Abstract plan
fragments
Executable actions
communicated to
UT and executed
by CGF
CLEAR
ENTRY
GAIN
DOMINANT
POSITION
CLEAR
INTERIOR
OF x
Explored room
Defeated enemies
IF DOOR LOCKED
SHOOT BOLT
IF DOOR CLOSED
KICK DOOR
IF WIDE ENTRY
STRAFE ENTRY
HUG
WALL
…
Appropriate representation facilitates reuse and
composition of pre-existing plans
CLEAR
BUILDING
Clearing
Room
CLEAR
BUILDING
INTERIOR
Team formed
Unexplored room
CLEAR
ENTRY
DOOR LOCKED:
SHOOT BOLT
CLEAR
ROOM
GAIN
DOMINANT
POSITION
HUG
WALL
…
Abstract plan fragments
…
Executable actions that are communicated
to UT and executed by CGF
Explored room
Defeated enemies
CLEAR
ROOM
INTERIOR
…
Plan fragment
reuse and
composition in
similar new
situations
Appropriate representation facilitates reuse and
composition of pre-existing plans
Team formed
Unexplored cave
Clearing
Cave
CLEAR
ENTRY
WIDE ENTRY:
STRAFE ENTRY
CLEAR
CAVE
GAIN
DOMINANT
POSITION
HUG
WALL
…
Abstract plan fragments
…
Executable actions that are communicated
to UT and executed by CGF
Explored cave
Defeated enemies
CLEAR
CAVE
INTERIOR
…
Plan fragment
reuse and
composition in
similar new
situations
Realistic and Affordable Simulation
Environment: UnrealTournament (UT)
CGF
Agent
CGF
Agent
CGF
Agent
CGF
Agent
Gamebots TCP/IP Interface
Urban Scenario
UT Engine (C/C++)
We can embed CGFs into larger tactical
simulations
UT Game
Engine
OneSAF
To simulate larger
military entities,
behaviors, &
capabilities
To provide real-time
high quality graphics
and detailed local
behavior
Correlated
entities
Correlated
terrain
SAF Manager
Show entities
information
SAF
entities
Show existing
OTB simulation
Show network
information
Show Current
PDUs in OTB
Show UT entities
Advantages of our Approach
• Reuse
– knowledge base of plan fragments and beliefs supports
reuse in new situations
• Modularity
– agent-based architecture provides modularity of CGF plans
and behaviors
• Composition
– matching algorithms enable the matching of plan fragments
and behaviors so they can be composed to form more
intelligent adversaries and team mates, as situations warrant
• Verification
– our representation formalism can be used for formal modelchecking and verification of desirable properties of the
software, thus reducing development time
Expected Impact
If successful, our research will provide
Reprogrammable and Instructable CGF
teams which:
– Can be “Coached” by training instructor using a
simple GUI to provide trainee appropriate combat
experiences
– Exhibit realistic team behaviors
– Considerably reduce development time and cost
while increasing behavior realism
– Can be embedded in larger simulations (e.g.
OneSAF)
Accomplishments
• Developed initial Agent Representation Scheme
• Developed initial algorithm that matches current
situation to previously developed plan fragments for
reuse.
• Implemented initial teamwork scenario in Unreal
Tournament.
• Publications:
– Sycara, K. et al. “Integrating Agents into Human Teams”, In Salas
E. (ed.) Team Cognition, Erlbaum Publishers, 2003. In Press.
– Sycara K. et al. “Ontologies in Agent Architectures”, In S. Staab and
R. Studer (eds.) Handbook on Ontologies in Information Systems,
Springer 2003. In Press.
Hand Signal Behaviors
Cover Area
Listen
•
1
2
3
4
•
Wait
Hand signals are important for
team communication in urban
warfare since the enemy is often
in close proximity.
Extensions to Gamebots allow AI
control over these new behaviors.
http://www.millenniumsend.com/user/pender/articles/hands.html
Composition: L-Shaped Corridor and
Room Clearing
D
C
B
A
MCWP 3-35.3 Military Operations on Urbanized Terrain (MOUT)
Composition: L-Shaped Corridor and
Room Clearing
D
C
B
A
MCWP 3-35.3 Military Operations on Urbanized Terrain (MOUT)
Composition: L-Shaped Corridor and
Room Clearing
D
C
A
MCWP 3-35.3 Military Operations on Urbanized Terrain (MOUT)
Composition: L-Shaped Corridor and
Room Clearing
D
C
A
MCWP 3-35.3 Military Operations on Urbanized Terrain (MOUT)
Composition: L-Shaped Corridor and
Room Clearing
D
C
A
MCWP 3-35.3 Military Operations on Urbanized Terrain (MOUT)
Composition: L-Shaped Corridor and
Room Clearing
D
C
MCWP 3-35.3 Military Operations on Urbanized Terrain (MOUT)
Composition: L-Shaped Corridor and
Room Clearing
D
C
MCWP 3-35.3 Military Operations on Urbanized Terrain (MOUT)
Composition: L-Shaped Corridor and
Room Clearing
D
C
MCWP 3-35.3 Military Operations on Urbanized Terrain (MOUT)
Composition: L-Shaped Corridor and
Room Clearing
D
C
MCWP 3-35.3 Military Operations on Urbanized Terrain (MOUT)
Composition: L-Shaped Corridor and
Room Clearing
D
C
MCWP 3-35.3 Military Operations on Urbanized Terrain (MOUT)
Composition: L-Shaped Corridor and
Room Clearing
D
C
MCWP 3-35.3 Military Operations on Urbanized Terrain (MOUT)
Composition: L-Shaped Corridor and
Room Clearing
D
C
MCWP 3-35.3 Military Operations on Urbanized Terrain (MOUT)
Composition: L-Shaped Corridor and
Room Clearing
D
C
MCWP 3-35.3 Military Operations on Urbanized Terrain (MOUT)
Composition: L-Shaped Corridor and
Room Clearing
D
C
MCWP 3-35.3 Military Operations on Urbanized Terrain (MOUT)
Composition: L-Shaped Corridor and
Room Clearing
D
C
B
MCWP 3-35.3 Military Operations on Urbanized Terrain (MOUT)
Composition: L-shaped Corridor + Stacked 2Man Room Clearing
Milestones and Deliverables
4/30/03- 9/30/03
• Develop initial scenarios for CGF deployment
• Develop initial agent teamwork representation
• Implement the initial scenarios in Unreal Tournament
10/01/03-12/30/03
• Evaluate the resulting CGFs for realism
• Refine teamwork representation as a result
1/01/04 – 3/30/04
• Develop techniques for agent behavior reuse
• Continue development and testing of teamwork
schemes
• Implement them and test them in new situations
Milestones and Deliverables (2)
4/01/04- 6/30/04
• Evaluate the resulting CGFs from previous quarter for
realism and ease of development
• Develop and test mechanisms for agent behavior
composition
7/01/04 – 9/30/04
• Develop techniques for resolution of mismatches in
agent descriptions
• Develop techniques for propagation of constraints
across plans and agent beliefs
10/01/04 – 12/30/04
• Implement techniques from previous quarter in Unreal
Tournament and test in new situations
• Develop techniques for belief propagation across team
members
Milestones and Deliverables (3)
1/01/05- 3/30/05
• Develop indexing scheme for agent behaviors
• Develop techniques for dynamic retrieval of
agent behaviors and reuse
4/01/05 – 6/30/05
• Implement dynamic retrieval and reuse of agent
behaviors in new situations
• Design and implement coach’s GUI
7/01/05 – 9/30/05
• Test control of CGFs from coach’s GUI
• Demonstrate embedding of CGFs in OneSAF
Hand Signal Behaviors