Final Technical Report of Contract ONR N00014-02-1-0621
Entitled:
Mathematical Theory of Software Agent Behavior and Human Behavior
Models (HBMs) to Increase the Realism of Synthetic Agents
Submitted by
Barry G. Silverman, PI
Kevin O’Brien, Jason Cornwell
Michael Johns, Gnana Bharathy
Ackoff Center for Advancement of Systems Approaches (ACASA)
University of Pennsylvania, Towne 229c, Philadelphia, PA 19104-6315
www.acasa.upenn.edu
(215) 573-8368 barryg@seas.upenn.edu
to
Cognitive and Neural Sciences Division, Room 817
Office of Naval Research, 800 N. Quincy St, Arlington, VA 22217
Harold Hawkins, Program Director (HAWKINH@ONR.NAVY.MIL)
(703) 696-4323
July 2003
ABSTRACT
This report is our final deliverable and as such describes the efforts under the FY03 funding
to accomplish three main objectives: to assemble a more portable, embeddable, and maintainable
version of PMFserv; to participate in a testbed to demonstrate that PMFserv could be embedded in
a pre-existing gameworld to enhance the human-like behavior of the bots of that world (more
realistic); and to use that testbed to illustrate an effective interchange architecture and protocol.
We accomplished all three objectives. The first of these objectives was satisfied by
rewriting PMFserv’s various generations of Java modules in Python, and by reconfiguring its
modules in the process. Many design improvements were made including, among others, adding a
modular blackboard, distributing (affordance-based) perception handling, and creating a true PMF
server both for external game engines as well as editing environments and testing suites developed
in-house at the University of Pennsylvania.
We also accomplished the second of these objectives and the result is the ICT Unreal
Tournament Testbed with a recreation of the Mogadishu Black Hawk Down urban conflict
scenario. The player enters this virtual world as the leader of a small group of Rangers (Soarbots)
and encounters a number of Somali civilian males, females, militia, and suicide bombers, all of
whom are controlled by the PMFserv. In documenting the knowledge engineering and editing
process, this report also serves as a guide for authoring new scenarios in PMFserv.
Finally, we accomplished the third objective since, in this testbed, PMFserv runs as a COM
server that operates the minds and behaviors, while Unreal Tournament operates as a COM client
displaying the scenes and bodies. The client-side bodies interact with their server-side minds to
find out their reactions to events and to determine their next action choices. All this takes place
with reasonable runtime performance. This report also includes some of the pros and cons of this
interchange approach and how PMFserv is now available via an API in Windows.
Table of Contents
Summary of Accomplishments and Overview of Appendices
1) HSIAC Book Chapter 9 – Overview of PMFserv Evolution
2) Additions to the Performance Moderator Function (PMF) Anthology
3) Updated Technical Report on PMFserv
4) BRIMS Paper on Evaluating Affordances for Enhancing Affordability
5) ICT’s Mogadishu Testbed Scenario and Roles for PMFserv Bots: The 5 Ps
6) Integrating PMFserv Into Unreal Tournament: The Client-Server Architecture
7) The Authoring Modules of PMFserv and How They Support Knowledge
Engineering of Bots with PMFs and World Objects with Affordances
8) Model View Controller Design Pattern for Realtime Editing & Results Displays
Page
3
3
3
4
4
5
9
12
16
APPENDICES TO THIS REPORT
A) Initial Knowledge Engineering of PMFserv Bots – GSP tree branches and weights
plus World Object affordances (multi-page spreadsheet)
B) Authoring Screens of PMFserv – Knowledge Engineering (Agents, PMFs,
Affordances)
LINKED REPORTS THAT WERE ALSO PART OF THIS EFFORT
HSIAC Book Chapter 9 – Overview of PMFserv Evolution
Additions to the Performance Moderator Function (PMF) Anthology
Updated Technical Report on PMFserv
BRIMS Paper on Evaluating Affordances for Enhancing Affordability
OTHER MEDIA FILES ATTACHED
Mogadishu ICT-KE Spreadsheets
Videoclip of Crowd Milling, Looting, Taunting, Swarming, Fleeing
Videoclip of Militia Shooting from Behind Somali Women
Videoclip of Suicide Bomber
Videoclip of Civilians Appearing to Flee, but Picking Up Guns to become Combatants
Videoclip of Runtime Physiology Editing
Videoclip of Runtime GSP Tree Value Editing
20
22
Summary of Accomplishments and Overview of Appendices
This is a final technical report on the work performed under ONR contract N00014-02-10621 during the interval from May 2002 through the end of July 2003. We are grateful to the
Defense Modeling and Simulation Office (DMSO) for supporting this effort and for providing
guidance and direction, as well as for setting up our collaboration with the Institute for Creative
Technology (ICT) at USC. Specific individuals we interacted with throughout this research include
Harold Hawkins at ONR, Sue Numerich and Michael Young at DMSO, Josef Toth at IDA, John
Tyler from MITRE, and Michael Van Lent and Ryan McFadden from ICT. None of these
individuals are responsible for the statements or any errors in this final report.
Prior to the current project we performed 2 prior years of efforts (1) to study the field of
human behavior models, (2) to extract and assemble an anthology of almost 500 of the most wellvalidated Performance Moderator Functions (PMFs) from the literature, (3) to map the very best of
those PMFs into a common mathematical framework and from that into an integrated agent
cognition illustrating how PMFs on both physiology/stress and personality/culture/emotions
influence decision making, (4) to use that resulting cognitive framework for simulations of
situations where agent micro-decisions lead to emergence of macro-behaviors, and (5) to provide
several demonstrations of the value of PMFs for modeling terrorist leaders, crowds of protesters
(when does looting emerge), provocateurs and their impact (how can they tip the crowd mood), the
impact of individual behaviors like chanting on overall crowd behavior, and so on.
In the current contract we sought to consolidate all the previous accomplishments by (1)
rewriting the disparate software modules into an overall coherent framework (that we came to call
PMFserv) so it could be readily reused for new projects, (2) demonstrating how the PMFserv could
be integrated into other simulation environments such as JSAF, Unreal Tournament, etc. as well as
with complementary agent cognitive models (e.g., Soar, CogNet, etc.), and (3) illustrating the
above two objectives in terms of a recreation of the Mogadishu/Black Hawk Down (BHD)
scenario. In the end, we accomplished objective #1 by rewriting the PMFserv modules into Python,
although the culture/emotion/affordances module was sponsored outside of this effort. For
objective #2, our sponsor had us investigate many directions for integration and complementary
functionality demonstrations, and then narrowed the scope and asked us only to integrate into
Unreal Tournament (from Epic Games) in parallel with bots using alternative human behavior
models (i.e., SoarBots and AI-Implant Bots) to be assembled by other centers such as ICT and
University of Michigan. Finally, within that narrowed scope and working with the other
participants just mentioned we accomplished objective #3 as well.
In order to accomplish the three objectives just enumerated, we engaged in a number of
activities over the course of fifteen months. The major activities are described in what follows,
which in turn serves as an overview of each a previous paper/report that is linked to here, or it
serves as a summary of one of the appendices.
1. HSIAC Book Chapter 9 – Evolution of PMFserv – This is a chapter that the PI prepared for
an edited book. It provides a detailed accounting of what was discussed in the prior two
paragraphs and so is a nice overview of the history behind this current effort and of the main
thrusts of the current project period. The writeup is a little outdated having been completed in
April of 2003 though it won’t be published until 2004. The link to this chapter is:
http://www.seas.upenn.edu/~barryg/Ch9-final.pdf
2. Additions to the Performance Moderator Function (PMF) Anthology – In the planning for
the Mogadishu BHD Testbed scenario, several additional PMFs seemed relevant (e.g., Khatt
weed chewing effects, sleep and exertion PMFs, etc.). These were collected largely by a
doctoral student, Gnana Bharathy, not on the current contract, but who is researching under the
PI. In the end, the BHD scenario did not call for us to use most of these, however, since the
PMFserv physiology module includes a visual PMF reservoir editor, one could readily add
them or other PMFs to the current BHD testbed. See:
http://www.seas.upenn.edu/~barryg/PMF_Addendum1.doc
3. Updated Technical Report on PMFserv – In pulling together the disparate modules and
PMFs programmed in the prior years’ activities, we made a number of design improvements
that are documented in this appendix. Our prior experience taught us that we needed PMFserv
to be easier to maintain, update, and integrate with other environments. As a result, the most
significant changes were (1) to adopt a new language (Python instead of Java since Python is a
popular language for gluing together many modules and since it sits atop C), (2) to orient all
the PMFs/modules around a blackboard architecture so that the cognition is modular and
alternate PMFs can readily be dropped in to extend or replace current ones, and (3) to shift to a
new form of perception that likewise appeared to be more maintainable. The latter involved
adopting the affordances based architecture following Gibson’s ecological psychology. The
link to this tech report is:
http://www.seas.upenn.edu/~barryg/PMFserv_2003_revisions.doc
4. BRIMS Paper on Evaluating Affordances for Enhancing Affordability – We thought the
third change just mentioned was significant enough that it deserved to be highlighted at
BRIMS, and we wrote a paper comparing affordances to traditional AI approaches for
perception handling. This paper was published at BRIMS’03 and it also was a contender for
“recommended reading”, although it wasn’t ultimately selected by the BRIMS Program
Committee. Harold Hawkins did indicate this was a novel contribution within the field of
computer generated forces and wargaming, and he has encouraged us to submit a follow on
white paper on how we might conduct a more formal and thorough comparative assessment.
The link to the BRIMS paper is:
http://www.seas.upenn.edu/~barryg/BRIMS2003_AffordanceTheory.doc
5) ICT’s Mogadishu Testbed Scenario and Roles for PMFserv Bots: The 5 Ps
We were asked to investigate a scenario for demonstrating the capabilities of PMFserv as
well as for illustrating its potential for integration into other simulators. In general, scenarios are
like stories in that one invariably must define the components of and interactions between People,
Place, and Plot. Since gameplay is involved, a 4th P, that of Play is also included. Finally, since the
goal is a training game, one must also factor in the pedagogical or training objectives. Figure 1
illustrates how we step through these five Ps when defining a scenario in general, and for PMFserv
in particular. This figure and its elements are described in full detail in Appendix E.
Figure 1 – The Five “Ps” of Scenario Definition Efforts Using PMFserv
PEDAGOGICAL OBJECTIVES:
•Situations to Encounter
•Doctrine to Rehearse
•Skills to Master
•Behaviors to Learn
PLACE
•Select Location/Setting
•Insert Terrain/Buildings
•Add in Resources, Caches, etc
•Indicate Institutions
•Markup Affordances
PEOPLE
•Determine Types of Bot Groups/Individuals
•Calibrate Physiological/Stress PMFs
•Fill in Goal, Standard, Preference Trees/Wgts
•Identify Relationships & Political Identities
•Indicate Behavior/Action Potentials
PLOT vs. PLAY
•Define Levels & Missions(Level Objectives)
•Anticipate Player & Bot Behaviors
•Identify Bots to Summon Across Space & Time
•Allocate Training Objectives to Space/Time Script
•Insert Event/Engagement Triggers & Endpoints
Some of these details on the 5 Ps for the current implementation are in the following table. More
details are offered immediately this table.
Place:
- Bakara Market-area of Mogadishu, Somalia. A few
blocks around the location of the 1st helicopter crash
(Wolcott’s UH-60).
People:
- Populate Bakara Market-area, life in the Habr Gidr
Clan (civilian males, females, militia)
-
Player and up to 4 fellow Rangers (Soarbots) start a
few blocks away from the crash site
Plot:
- Reach helicopter crash site without taking casualties
- Stop crowds from looting downed helicopter
- Secure helicopter (destroy and move on)
- Crowds/Militia try to get player to accidentally shoot
a woman-shield
- At any time, if mistakenly harm a civilian, must face
emergence of civilians-turned-combatants and
appearance of one or more civilian suicide bombers
Play:
- Use your own tactics and ideas to manage the level
(create your own story)
- Replay challenging scenes until successful (try to
keep civilians from taunting, swarming, rioting,
looting while handling asymmetric threats and
avoiding unnecessary civilian deaths)
- Be given new life if fail to beat level or if make
mistakes
- Work out killing issues in virtual world, use good
doctrine in real world
Pedagogy:
- Practice the management of civilian crowds while
recognizing asymmetric threats
- Rehearse urban conflict doctrine and tactics against
real threats
- Learn to handle suicide bombers without getting
injured
ICT’s Mogadishu Testbed Scenario and Roles for PMFserv Bots -- There is one main level that
PMFserv bots must populate as the Somali civilians (males and females) and Somali militia
members. Also a terrorist bomber emerges. In the ICT scenario, the player starts as a Ranger in the
bottom right of the level surrounded by the Soarbots who are other rangers that receive commands
from him. The ICT final report describes those capabilities. Also, there are 4 militia that ICT
implemented patrolling the middle of the level as militia. In our version of this scenario, the
civilians summon these militia when they are needed.
Suicide
Bomber
(PMFserv)
Militiaman with
Female Shields
(PMFserv)
Civilian
Chopper Looters
(PMFserv)
Civilian Crowd
(PMFserv)
Helicopter
Crash Site
Militia Unit
W/ Leader
(AI Implant)
Start
The level has the player and his subordinates advance upon the Durant Crash Site, where they
encounter 2 groups of civilians, one gathered around the helicopter and the other looting aside it.
Because the chopper will not be capable of being destroyed, there must be some way for the player
to secure the area: possibly by just detonating a grenade after the area has been cleared of civilians,
effectively ending the level.
In our implementation of the Level, the player must encounter and disperse a crowd of Somali
civilians either looting the helicopter or observing the scene around the helicopter. In general,
these Somalis have grown up with violence and are not easily intimidated. The player is harassed
as he tries to dispel the crowd, and is forced to resort to violence.
http://www.acasa.upenn.edu/~barryg/unreal/demo1_small.avi
Also in our implementation, the player and his RangerBots (Soar) must encounter a crowd
of civilians with a Somali militia member shooting from behind them. The female bots have to
make a decision to act as shields or not for the militia member. If they do act as shields, the militia
member shoots at the player from behind the women. If the militia member dies, the women flee.
http://www.acasa.upenn.edu/~barryg/unreal/demo2_small.avi
If the Rangers or Player kill a civilian unprovoked, some of the Somali agents will search for a
way to revenge themselves on the Rangers. In this clip the Somalis appear to flee, when in fact
some of them are locating a weapon and intending to return fully armed and ready to engage.
http://www.acasa.upenn.edu/~barryg/unreal/demo3_small.avi
If the Rangers or Player kill a civilian, this may precipitate a second threat which is a
suicide bomber who appears as any other civilian male.
http://www.acasa.upenn.edu/~barryg/unreal/demo4_small.avi
None of these behaviors are programmed directly into our PMFserv Bots. Rather, the Goal
Standard and Preference Trees and weights are such that the chain of events described might
emerge. Whether these behaviors emerge or not depends on player and Ranger behavior as well as
on what the individual PMFserv Bots actually observe happening. The latter also extends to what
objects and perceptual types they notice in the world and how those project what they afford to the
Bot.
6) Integrating PMFserv Into Unreal Tournament: The Client-Server Architecture
In order to use PMFserv to operate the civilians and militia of the scenario, it is necessary
to provide an interchange architecture and to hookup PMFserv within this framework. The intent
of our effort was to demonstrate that PMFserv can be used to complement the behaviors of bots in
other enviromnments. That is, it was always a design goal for PMFserv that it would serve up
PMFs and behaviors that could become useful for third party M&S developers looking to scavenge
parts or to quickly add capabilities to their CGFs or SAFs that they currently lack. Thus, our agent
parts can operate as the brain, while their M&S environments generally will animate and display
the bodies, handle kinesthetics, avoid collisions, do expert naturalistic reasoning (one of our 5
coping styles), etc.
In the ICT testbed, the interchange between PMFserv and Unreal Tournament that most
satisfied our timetable and budget limits was the Microsoft COM interchange standard. Since
PMFserv is in Python which sits atop the C language and since UT runs in Windows for the
Testbed, it was relatively straightforward to adopt and implement the Component Object Model
(COM) specification and software from Microsoft. COM refers to both a specification and
implementation developed by Microsoft Corporation which provides a framework for integrating
components. COM defines an application programming interface (API) to allow for the creation of
components for use in integrating custom applications or to allow diverse components to interact.
However, COM is a low level service and in order to interact, components must adhere to a binary
structure specified by Microsoft. As long as components adhere to this binary structure,
components written in different languages can interoperate.
To use COM for our interchange required us to adopt a client-server approach (illustrated by
Figure 2) which required us to do the following:
 Create a COM server for PMFserv on the Python side that exposes itself via COM to any
application that is COM-aware. This made use of a pre-existing freeware DLL or Python
module for mapping between Python and Microsoft’s COM library.
 Create a Dynamic Linked Library (DLL) designed to work with Unreal Script that turned
Unreal into a COM client. This DLL was written in C++ and was inserted into Unreal as
“native code”.
This enabled Unreal Script to make direct calls to PMFserv functions and to send updates for
specific bots. At runtime, Unreal operates a process with the Unreal-COM client as a sub-process.
The PMFserv runs as a process on the same machine (currently) while the PMFserv COM Server
runs as third process under the control of the Windows COM facility. This COM server has two
threads, one ongoing thread that monitors client requests while the other thread is spawned when
client requests occur and lives until they are satisfied from the COM server side.
As Figure 2 and Table 1 show, there are essentially seven layers to this protocol – two for
UT, three for COM, and two for PMFserv. Thus if an event happens in UT, it must be sent through
all these layers for the relevant Bot in PMFserv to sense it and formulate a response. A similar path
must be traveled in the reverse order for the response to reach UT and be played out by the UT
game engine. The bots in PMFserv cannot directly call Unreal functions, but instead can poll the
PMFserv Services Layer to find out if anything has been updated since the last tick. Currently
PMFserv operates on the same machine as Unreal, however, the interchange makes it
straightforward to provide parallel processors, and by that to increase the number of bots in Unreal
without adversely affecting performance.
Figure 2 – Architecture Adopted for Interchange and Integration of PMFserv with Unreal
Tournament
Unreal (COM Client) PMFserv (COM Server)
Game Space
Custom Script:
Semantic Markups
And Procedure Library
SeenAs:
Foreign
Presence
Affords:
Protest,
Bomb
Helicopter
SeenAs:
Enemy
Sees Vehicle
Affords:
Attack,
Flee From
Sees
Crowd
SeenAs:
Friendly
Crowd
Options:
Excite,
Disperse
Crowd
Sees
SeenAs:
Rival Crowd
Affords:
Taunt,
Sees Attack
Microsft Common Object Model (COM Layers)
Hotel
PMFserv Bot
Standards Based Translation Layer
(Unreal Tournament
Infiltration
Engine and Services Layers)
Chosen action
Calculated
Emotions
PMFserv
Services
GSP Activations
•Scheduler
•Router
•Updater
•Etc.
Coping style
Stress Reservoir
Physiology
Reservoir
PMFserv Runtime
Editors (MVC)
Table 1 – Functionality Allocations Across the Runtime Interchange Protocol Using COM to
Connect PMFserv to Unreal Tournament
Unreal
Tournament
Environment
Custom
Unreal
Script
COM
Standard
Interface
PMFserv
Environment
& Services
Individual
PMFserv
Bots
•Art assets
•Weapons
•Animations
•Physics
•Sound Effects
•Standard
Behaviors
•Basic Bot AI
•Game Engine
•Camera &
Display
Services
•Overrides of
Unreal
Behaviors
•New AI &
Behaviors
•Semantic
Markup of
World
Objects &
Events
•COM client
in UT (C++)
•Event Data
Router
•Semantic
Labeling of
Events
•World Object
Affordances
•Bot Responses
Uploader
•Event Sensor
•Memory Unit
•Physiology/
Stress Module
•Personality,
Culture, &
Emotion
Module
•Decision
Module
•Response
Selector
•Microsoft’s
COM in
Windows
•COM
server in
Python
This seven layer protocol sounds potentially complex, yet it performed quite well in
practice and did not lead to latency of note in the responses of the bots. What follows is a summary
of the observed pros and cons of this approach.
PROS of the Interchange Architecture
 Uses a standardized software approach that’s widely available on all PCs
 Microsoft’s COM layer is straightforward, well documented, and rapid to implement
 Runtime performance was excellent – no noticeable latency between events and responses
CONS of the Interchange Architecture
 COM is a Microsoft artifact, and not a universal standard
 Limits portability to platforms using Windows
 COM approach doesn’t solve many interchange issues, but pushes most of the interchange
responsibility onto other layers
 Since there are no naming conventions or translation standards in general for human behavior
models, the resulting Custom Unreal Script was difficult to create and grew to about 1,000
lines of code, code that is not itself very reusable.
Due to time constraints, most of the Custom Unreal Script had to be dedicated to nuances of
this interchange environment and more specifically to this exact scenario. Given a few more such
interchanges one might observe some useful patterns and conventions might emerge that would
further help the field of human behavior model interchange. Certainly that is a worthy goal and a
trend that should be encouraged in the field as more M&S environments attempt to benefit from
existing and complementary types of human behavior models.
7) The Authoring Modules of PMFserv and How They Support Knowledge Engineering of
Bots with PMFs and World Objects with Affordances
Over the past three years, PMFserv has been used to create to construct and simulate the
people and objects of a number of scenarios, including crowd scenes (civil unrest, soccer
hooligans, urban conflict), the asymmetric threat, Black Hawk Down (Wolcott’s crash site), and a
car buying family. The earliest of these scenarios was an ambush at a military checkpoint by
terrorists hiding amongst civilians on a school bus -- inspired by the fourth situation depicted in
General Paul Gorman’s In Defense of Fombler’s Ford. The second scenario was a higher level
demonstration that simulated the target selection procedures of an Intifadah leader and the suicide
decisions of his follower. Shortly thereafter a number of scenarios were built that depicted
emergent crowd scenes. Each featured a crowd gathering to protest a social injustice. In one
series of scenarios this injustice was a roadblock that kept people from going to work. In several
others, protests outside of a prison by various agents (mediated by stress, emotion, and social
identity PMFs) lead to rioting and looting of nearby stores and the intimidation of police and
protestor’s alike.
Many of these scenarios featured similar characters: a group of protesters comprised of
both females and males, employed and unemployed, a police presence, a group of onlookers, and
an instigator or two trying to rouse the crowd. In various scenarios we evaluated the impact of
diverse PMFs (e.g., personal and cultural values, motivational need sets, behavior standards,
chanting, taunting, security doctrine/orders and behavior, etc.) on crowd outcome and illustrated
equilibrium being tipped. Throughout all these scenario efforts we documented a number of
lessons learned about the composability and engineering of the scenarios, reusability of the agent
PMFs and world objects, and the replication of predicted behaviors. For example as mentioned
earlier, the crowd literature indicates that looting tends to occur when young unemployed males
(who rarely join organized activities) take advantage of chaos and distracted security forces – and
in our simulations the protesting women and employed males tended to flee if a riot occurred,
while the unemployed males lurked on the fringes of the protest scenes and indeed dropped their
inhibitions and proceeded to loot after the tipping point was reached.
At no point was the outcome programmed into rules or equations – we only modeled
physiology PMFs and personal value weights in the GSP trees and cultural standards (again
weights on a hierarchy of behavior standards) – and individual agents made their own
(micro)decisions that lead to the emergent macro-behavior. Three of our papers about PMFserv
and these various scenarios were awarded Outstanding Paper (or Recommended Reading) at the
annual CGF conference over the past 3 years.
Throughout all of this effort we learned a fair amount about the needs for knowledge
engineering of bots and world objects. Figure 3 is an attempt to summarize that experience within
a four-step process we go through when authoring the bots and objects for a new scenario or
world. This four step process is used to amplify the 5 Ps of earlier Section 5. In Step One, we
repeat the “People” box of the 5 Ps. This is the starting point. From that, the knowledge engineer
identifies the number of bots of each class (e.g., civilian males, females, militia, etc.) and major
world objects they must interact with.
Figure 3 – Overview of the Knowledge Engineering Process for Authoring Bots and
World Objects with PMFserv
1. PEOPLE (Sketch)
•Determine Types of Bot Groups/Individuals
•Calibrate Physiological/Stress PMFs
•Fill in Goal, Standard, Preference Trees/Wgts
•Identify Relationships & Political Identities
•Indicate Behavior/Action Potentials
2. Initial Knowledge Engineering (Spreadsheets)
•One Sheet for Each Class of Bot
•One Column for Each Bot of that Class
•Rows are GSP Leaf Nodes
•Cells are Initial GSP Tree Weights (Bayes
Probabilities)
•One Perceptual Type (Ptype) Affordances Table for
Each Bot Class/World Object
•Multiple Sub-Tables for each Ptype Including:
•Actions one can take on this Ptype
•GSP Leafs Impacted by each action
•Action valences (succ/fail)
•Action affordances (probabilistic
intensities)
3. Use PMFserv Editing Environment
•Bot Create Screen
•Physiology Editor Screens
•Stress Threshold Editor
•GSPtree Editor Screens
•Emotion Module Tester
•Relationship Manager Screen
•Affordance/Ptype Editor
•Action Editor
4. Create External Procedure Library
•Program in Game (UTI) Script
•Author needed “action” functions
•Call proper animations
•Stub out semantic markup items
•List service layer support needs
The second step is the spreadsheet modeling phase. This results in a detailed first pass
through the GSP trees and affordances. We use a spreadsheet to organize the thinking behind this
effort. There have been a number of student projects that successfully used this approach to
produce term papers that cull references from the literature that support the various tree branches
and weights assigned to bots of a given class. Since spreadsheets are easily updated, they are a
convenient knowledge engineering tool during the early stages of research and revision. The
spreadsheets for the current effort are attached in Appendix E.
These spreadsheets help to bound the effort and provide the knowledge engineer with a
launching poing for the third step of the process. That step is to use the editors of the PMFserv
authoring environment to author each of the bots of that level including filling in their physiology
reservoirs, stress thresholds, GSP tree leafs and weights, and relationships. One also uses the
PMFserv editors to markup the major objects of the world with affordances – these are the actions
one can take on world objects and the valence and intensity impacts those actions afford to the
relevant leaf nodes of a given bot’s GSP trees. Since each world object might be perceived in
different ways at different times (e.g., as a flying helicopter, as a crashed helicopter) and in
different ways by different bots (e.g., crashed helicopter is lootable vs. to-be-protected), the
markups take some effort. However, this effort facilitates the affordance approach wherein the
objects of the world contain the perceptions that we might ascribe to them. To help with all these
steps PMFserv includes a number of editors including bot and object creation editor, physiology
editor, emotion module editor, decision editor, affordance editor, action editor, and others.
Appendix F contains a walkthrough of the major editors of the PMFserv and illustrates a slice of
the knowledge engineering of step 3 of the process. A few of the many items we added under
PMFserv bots include, but aren’t limited to:
-
Physiology - Fatigue, wounds, adrenaline, multiple wounds to die.
-
-
-
GSP/Emotion Trees – Goals, Standards, Preferences of members of the Habr Gidr subclan.
(capturing values about belonging to family/clan, devotion to cause, jealousy of America,
hatred of Rangers, impact of seeing Rangers as vulnerable, impact of seeing loved-one
wounded/killed, effect of mob-rule, fighting ferocity/willingness to die, acting as cover for
militia, switching to combatant, treating women as objects, etc.
Crowd actions should demonstrate a range of things like observing, curiosity/attraction to
noises/key sites, flocking, communicating, taunting, advancing, fleeing, swarming, panicking,
rioting, looting bodies and abandoned craft, and converting to combatants.
The trained militia were much smarter than the civilians-turned-combatants (CTCs). Militia
know how to shoot more accurately, avoid killing each other in cross-fire, use women for
cover, and so on.
Finally, in step four, one must establish the library of procedures in the native game
environment that will carry out the action choices of the PMFserv bots. PMFserv bots are mind,
and not body. Thus they need skins, bodies, physics, kinematics, animations, etc. provided for
them from the game engine’s existing bots. If a PMFserv bot decides to observe, flee, taunt, loot,
flock or swarm with the crowd, attack, die, etc. there must be game side code to handle these
actions.
The following are the Art Asset Suggestions we made in April 21, 2003 to support these
scenarios. None of these actually were finalized until after July 2003.
For a successful PMFserv demonstration the most important capability is the ability to
represent changes in the mental and physiological states of our agents in the 3D models they are
controlling. This translates into a variety of models, skins, and animation cycles for each agent
type. Since the point of this exercise is to demonstrate the capabilities of a group of cognitive
models and HBMs, the ability of the models to perform a wide variety of actions and assume a
wide variety of postures is more important than realistic terrain, vehicles, weapons, etc.
Every unreal bot should be able to accomplish those movements marked with a *, so there are
only a few extra animations.
 Standard walk cycle -- needed Custom Unreal Script to slow down the Somali bots (their
standard behavior has them running with supernatural speed across the screen)
 Crouching / Cowering in fear – use existing crouch position
 Waving / jumping with hands in the air
 Advancing threateningly/swarming – just have them walk forward toward Rangers
 Taking cover *-- only have the ability to flee
 Showing fatigue/gasping – use crouch and gasp
 Firing a gun *-- already exists
 Backing up *-- already exists
 Attacking with a melee weapon *-- we created a suicide bomber that blows up anything
nearby
 Getting shot *-- already exists
 Looting the helicopter – implemented as the following sequence: pausing, looking around
quickly to left and right, crouching, and standing back up.
 Militia members using another agent as a human shield—possible but there cannot by any
physical contact between the two characters (a limitation of UT)
Animations we wanted but which weren’t available from the animator until after the end of the
project period:
 Shouting
 Throwing a rock/stun grenade
 Taking cover (not just fleeing)
 Limping/Walking as if dazed
 Peering out from behind an object—doable, primarily AI driven
 Woman picking up a child and fleeing—not possible
 Militia member dragging a dead Ranger—possible, but unlikely
 Militia commander giving orders—OK
Instead, we found two Somali-looking civilian skins in Unreal Tournament’s public library that
had far more simplistic behaviors than these, and with which we could only create a scaled down
implementation of the crowd gestures and actions. These are the woman with a blue bourka and
the male with red shawl and white robe that appear in the video clips. In UT the bots we used
already existed with many of the low level behaviors including breathing, a celebratory animation
that looks a bit like break dancing, running, picking up a weapon, shooting, dying, and the like.
Many of these built-in behaviors had to be modified or overridden to slow them down and make
them fit our needs. These bots did not include navigation routines, walking, flocking, swarming,
attaching to crowds, taunting, and so on. They had no physiology in the sense of fatigue, noise
reactions, and so on. They had no emotions, coping styles, stress reactions, or decision making
functions. Much needed to be done to finalize the bots for the scenario vignettes and game called
for here. These changes all reside in the layer we refer to as Unreal Script. From the Somali bots
depicted below, we managed to cobble together and alter the break dancing and other animations
so in the end the visual behavior of the bots loosely approximates many of the desired animations
just listed.
8) Realtime Editing & Results Displays
Adopting a client-server architecture for embedding PMFserv (the mind-behavior server)
within a given game environment (the body client) helped us to engineer and effectively manage
the runtime interchange. One can carry this a step further and segment the code into a design
pattern known as the Model View Controller or MVC. MVC is a rather simple design pattern in
concept, though it turns out to be quite powerful in practice offering significant advantages both at
run time and during ongoing software maintenance efforts.
The traditional use of MVC is to separate the visual software from the model or engine, and
to also separate the user’s controls as well. This means all three types of software can be optimized
for performance and maintained without interfering with the other. The left side of Figure 5
illustrates a possible MVC pattern that the player might be experiencing. We do not know for sure
if Unreal is organized this way, but for sure the PMFserv is removed from any player viewing or
controls, as is are the other bots of the ICT testbed.
Figure 5 – Adopting the Model View Controller Design Pattern to Help Organize and
Maintain the Runtime Operation of PMFserv for Players as Well as Training Developers
Playing domain
View
•UTI
sees
View
updates
Player
uses
manipulates
Controls
•Move
•Shoot
•Cmd
Model
•UTI Engine
•PMFservBots
•SoarBots
•AII-Bots
Training domain
•Bot actions
•Group behav.
updates
sees
Trainer
manipulates
Controls
uses
•Spawn Bots
•Alter Bots
(PMFs, gate,
Reservoirs,
Weights)
In addition to this traditional user or player usage of MVC, we have adopted a
second tier implementation of the MVC pattern as shown on the right side of Figure 5. Having two
MVCs is probably consistent with the mind-body separation that exists for this testbed. At any
rate, the left side of Figure 5 corresponds to players controlling and viewing the physical world of
the gamespace, while the right side of Figure 5 correponds to developers and trainers who might
wish to view and control the mind and behavior of the bots.
Runtime visual editor and display windows have two primary advantages: they assist with
testing/debugging and realtime scenario changes. In terms of the latter, trainers often realize their
scenario isn’t quite right and they desire to have some way to alter behaviors of the bots at run time
or as the scenario is being played out. Our organization of PMFserv software follows this type of
implementation. In terms of the former, one way to verify that the PMFs are working properly is to
separately examine each agent and each PMF as the scenario unfolds. When assembling scenarios
and when engineering the bots, this kind activity is often helpful.
The following screen captures depict some of the MVC capability available to the
knowledge engineer and to the trainer. Also, this reports includes a number of videoclips that
show these capabilities dynamically. Many of the editor windows shown below also contain
controller spaces, so resetting any of a bot’s reservoir levels, tree weights, or nearly any other
parameter is permissible.
PMFServ Runtime Editing Video Clips:
http://www.acasa.upenn.edu/~barryg/unreal/demo5_small.avi
http://www.acasa.upenn.edu/~barryg/unreal/demo6_small.avi
PMFserv Screen shots in ICT’s Unreal Tournament demo
Before shooting starts:
 Somali woman observer’s Standards Tree is shown in upper right screen
 Somali woman in upper center screen sees her role as a shield and her affordances for
killing or submitting to the Male she is shielding the male if he makes such a request
 Also, notice the Somali woman views the Ranger as nonthreatening (upper left)
VIEW
CONTROLS
After player shoots civilian:
 Somali woman observer’s emotions are shown in upper right screen
 Also, notice the Somali woman now views the Ranger as “Murderous Ranger” in upper left
screen
After Somalis get their guns:
 Somali observers run home to retrieve their weapons
APPENDIX A
Initial Knowledge Engineering of PMFserv Bots – GSP tree branches and weights plus
World Object affordances
This appendix includes the spreadsheets that were used in the first pass at knowledge
engineering the classes of bots and world objects for the Mogadishu scenario.
NOTE: This file is included as an OLE object. Thus if you double click on the spreadsheet
on the next page you will be able to browse through (and in fact edit) the separate spreadsheets for
each of the male agents, female agents, militia agents, and world objects. Each of these
spreadsheets itself is multiple pages long.
APPENDIX B
Authoring Screens of PMFserv – Knowledge Engineering (Agents, PMFs, Affordances)
A civilian agent can be placed into any level in UnrealEd
Some of an agent’s physical properties can be edited within UnrealEd itself
Editing a new Somali bot and her GSP Trees
Editing the World Objects and their perceptual types available to various bots.
Editing the Affordances of Various actions