Hybrid Human-Agent Teams
for Simulation Based Training
1 / 53
Part 1 – The VR
Environment
2 / 53
Background
• The MRE (Mission Rehearsal Exercise):
Aims “…to create a virtual reality training
environment…” in which various scenarios
that may be encountered by military units on
peace-keeping missions can be played out.
3 / 54
4 / 54
Introduction
Methodology in the design of the system:
• Selecting state-of-the-art systems for
different needs of the simulation.
• Improving/modifying these systems where
needed.
• Developing specialized software for the
simulation (the actor agents, and their AI).
• Combining these different systems as
seamlessly as possible.
5 / 54
Tweaking
• Step-by-step improvements of the different
components of the system strive to create
a more believable and realistic experience
for the trainee.
• Every little step forward can take us a long
way towards better virtual reality.
6 / 54
System Components
• Video:
– Panoramic (150°), semi-circular, 2 2/3 meter
high screen encompasses the user.
– Environment and special effects rendered
using commercial Vega™ program.
– Characters are rendered using the
commercial PeopleShop™ program.
– Rendering is done in real-time.
7 / 54
System Components (2)
• Audio:
– Complex multi-channel audio system, with
speakers in front, sides, rear and overhead.
– Provides all background sound, with strong
spatial capabilities.
– High realism.
– Large efforts were made to synchronize
sound and events occurring on the screen.
8 / 54
Human Interface
• Input (speaking to characters)
– Speech recognition
– Identifies keywords
– No deeper understanding
– Plans for the future include merging natural
language technologies.
9 / 54
Human Interface (2)
• Output (Agents to humans)
– Pre-recorded statements
– Synthesized speech (in real-time)
– System can produce nuances in synthesized
speech, adds to believability.
– Lips are synchronized with generated speech!
10 / 54
Eliminating know-it-alls
• What would happen if the mother agent
could know everything you whispered to
your sergeant?
• What if your sergeant criticized your
actions based on facts you he had no way
of knowing?
11 / 54
Limiting Perception
• Filters were placed on the agents
perceptions, so that they only perceived
the parts of the simulation that a normal
person would.
– Don’t have eyes behind their back
– Can’t hear things far away, or with too much
background noise (helicopter flying over).
The sergeant will cup his ear when he can’t
hear you over the noise!!
12 / 54
The Script
• We can’t allow the trainee to do whatever
he/she wants, both because of limits in
technology, and because of the training
goals of the system.
• On the other hand, if every attempt to
deviate from the path is punished, the user
will soon get frustrated, and won’t actually
learn to find the right solution.
13 / 54
StoryNet
• The StoryNet model allows
several plot nodes, with
transitions between them.
• Within each node, there is
a relatively area of
‘freeplay’.
• Transitions between nodes
are caused by key actions
or events, in a way
transparent to the user.
14 / 54
15 / 54
The Director
• In the future, there are plans to implement
a Director agent, who will be in charge of
guiding events to the correct path, if the
student deviates too much.
16 / 54
Last but not least – the Cast
•
Three types of Agents/actors:
1. Scripted : have predetermined behavior
(movement/actions) which is triggered by
the controller of the simulation, or by the
actions of other agents.
17 / 54
The Cast (2)
2. AI : these include the main characters in the
simulation, with which the user has direct
contact.
These agents have a set of general, domain independent capabilities, operating over a
declarative representation of domain tasks.
The task are represented to the agents as a set
of steps (primitive actions, or other tasks), a set
of ordering constraints, and a set of causal links
that describe what is accomplished by each
step, and why it is needed (to achieve a
precondition for another step).
18 / 54
• This ‘knowledge’ about the world and
tasks in it, allow the AI agents to:
– generate a plan to fill their tasks,
– change their plans when the world state
changes,
– and maintain a dialog with humans and
teammates about the task.
19 / 54
The Cast (3)
3. AI agents with emotions:
– Are built on top of the regular AI model.
– Express emotion with gestures, and
nuances of speech (!!).
– Use their plan model to decide which
emotions to express.
– The emotional state influences the agent’s
beliefs, desires and intentions.
20 / 54
Expressing Emotions
• The emotional agents use their plan model to
decide which emotions to express.
21 / 54
Expressing Emotions (cont.)
• If the mother believes that the commander
plans to leave, and that that will make her
task of helping her child impossible, she
may express anger.
• If she believes the commander plans to
stay, she may express hope.
22 / 54
Emotions (2)
The emotional state influences the agent’s beliefs,
desires and intentions:
• The sergeant is under stress because he’s
responsible for the boy’s injury.
• He may seek to relieve the stress by
–
shifting the blame to others,
or by
–
•
forming an intention to help the boy get medical
help.
This choice will depend on other, adjustable,
character traits/states-of-mind.
23 / 54
Part 2 – Dialogue
24 / 53
Layout
•
•
•
•
Structure of AI model
Domain representation
Planning
Dialog actions
– Speech acts
– Grounding
– Stances
– Negotiation proceedings
– Future work
25 / 54
AI Model
• The agents use a general reasoning
algorithm to ‘understand’ and plan in a
declarative representation of team tasks.
• The algorithm is a domain-independent
non-linear STRIPS planer.
• The task representation is domain specific,
and encodes the given training scenario.
26 / 54
Task Representation
• Each task is represented by a set of steps,
which are either
– A primitive action – physical or sensory.
Or
– An abstract action – a sub-task (which in turn
is composed of a series of steps, etc.).
27 / 54
Task Representation (2)
• In addition, there may be ordering
constraints between the steps.
• There can be interdependencies between
steps, represented by causal links and
threat relations.
28 / 54
Task Interdependency
• Causal links – specify that one step’s
goals are another’s precondition.
• Threat relations – step A is a threat to step
B, if the completion of A causes the
removal of one of the preconditions of B.
29 / 54
Refinements - Responsibility and
Authority
• Task steps are associated with the team
members that are responsible for
performing them, and, optionally, with the
agent who has authority over that step
• The agent responsible for the step should
not perform it until authorization is given
by the teammate with that authority.
30 / 54
Planning
• When the team is given a (top-level) task,
each member recursively constructs a
complete detailed task model.
• Each member builds the model according
to his/her knowledge, which may be
incomplete or erroneous.
31 / 54
Re-planning
• The Agent’s perception module constantly
monitors the world, and sends messages to the
cognition module about changes in the world
state.
• Whenever changes occur, the agent rechecks
his full plan, and corrects it according to the
changes.
• For instance, if a goal in the plan has been
achieved by a teammate, there is no longer a
need to perform the appropriate task yourself.
32 / 54
Re-planning (cont.)
• Instead of creating the whole task plan
from scratch whenever a change occurs,
the agent uses it’s previous model as a
guideline, and so cuts down on the
amount of alternatives it calculates.
33 / 54
Agent’s Task Model
• At any given time, each agent holds a
complete plan that models the way in
which he believes the whole team can
accomplish the task.
• This plan allows the agent to answer
questions and reason about the task.
34 / 54
Courses Of Action (COA)
• In order to negotiate, there is a need to
support reasoning over several possible
COAs.
• Each COA is a high level task, as
described above, but is not automatically
marked as ‘intended’ by the agent just
because it achieves the goal.
• Instead, if the agent reasons that the COA
can achieve the goal, it is marked as
‘relevant’.
35 / 54
COAs (cont.)
• Since different COAs may be mutualy-exclusive,
threats and causal links between alternative
COAs must be ignored.
• On the other hand, the effects of a COA on other
tasks are taken into account when calculating
the utility of the that COA.
• A complex utility function is used to decide
between the alternative COAs.
36 / 54
Dialog
• The information state is the information
representing the complete current context of the
dialog.
• This model is more complex than the “dialog
state” of the standard plan-based approach.
• It contains all the necessary data relevant to the
current state of the dialog, and data needed to
decide on future dialog acts.
• The information state is updated via dialog acts,
according to dialog rules.
37 / 54
Speech Acts
• Core speech acts
– influence the topic under discussion
– Establish and remove commitments.
38 / 54
Speech Acts
•
•
•
•
•
Assert
Info-request
Suggest
Request
Order
These acts reflect social commitments of
the speakers, but do not directly affect the
BDI of the person addressed, since they
may be insincere.
39 / 54
Types Of Speech Acts
• Speech acts can describe three things:
• Action – consists of
– Agent : the person performing the action
– Event : the action being performed
– Patient : the object on which the action is being
performed.
• State – indicates the state of some aspect of an
object. (for instance, whether the door of the
house is open or closed).
• Question - about one of the above.
40 / 54
Grounding
• In order for an act to be considered as
accepted into the common ground of the
conversation, it needs to be acknowledged
by the agent to whom it was addressed.
• This is accomplished by grounding acts.
• Core speech acts are not seen as having
their full effects on the social state until
they are grounded.
41 / 54
Grounding Acts
•
•
•
•
•
•
•
•
Request-acknowledge
Acknowledge
Request-repair
Repair
Initiate
Continue
Display
Cancel
42 / 54
Stances
• The current state of team negotiation is
represented by a sequence of stances.
• Each stance represents the outward
representation of an agent’s view on the
issue.
43 / 54
Stances (2)
• Each stance is composed of
– The agent who holds the stance
– The action that the stance is about
– The attitude (stance) of the agent towards
the action.
– The audience to whom the agent is
presenting the stance.
– The reason for the stance
– The time at which the stance was made.
44 / 54
Stances (3)
• The actual attitude (stance) can be one of
the following (from most positive to most
negative):
– Committed (‫)התחייבות‬
– Endorsed (‫)מומלץ‬
– Mentioned
– Not mentioned
– Disparaged (‫שלא‬-‫)מומלץ‬
– Rejected (‫)דחיה‬
45 / 54
Negotiation
• The sequence of negotiation stances shows the
progression of the negotiation: progresses from
proposals of action towards commitments.
• Stances arise from core speech acts (for
example: if A requests B to do something, A’s
stance is committed to that action).
• There are also special negotiation acts: accept,
reject, counterproposal, and explanation (for or
against), each of which give rise to the
appropriate stance.
46 / 54
Initiating Negotiation
• An agent’s initiative characteristic
determines whether the agent is likely to
start a new negotiation.
• If another agent starts a negotiation with
an order or request, the agent addressed
is required to respond in some fashion.
47 / 54
Appropriate Responses
• The response to an action proposal is guided by
the following factors:
– The relevant party : the agent who is in charge of
the next step in performing the action (the authorizor,
the agent responsible for the action, or the addressee
himself)
– The dialog state : one of discussed, needsdiscussion or unmentioned.
– The plan state: how the action relates to the agents
plan (good/bad , intended/not-intended etc.)
48 / 54
Appropriate Responses (cont.)
Examples:
• The agent will reject the proposal if
plan-state is one of { bad, considered-bad,
unknown, conflict, goals-satisfied }
dialogue-state=needs-discussion.
• The agent will accept (reluctantly)! If
relevant-party=me
plan-state=considered-bad
dialogue-state=discussed
49 / 54
Appropriate Response (cont.)
• There are possibilities of overlap of
conditions, in which case several
responses are possible.
• The agent has to decide which responses
to choose, and in what order.
• The decision is based on practical
considerations (which is the best, or most
immediately relevant), and on social
relations between the proposing party and
the agent (superiority/rank).
50 / 54
Resolution
• Negotiation between teammates proceeds
until both agree to accept/reject, or until
one drops the contrary stance.
• It is also possible to “agree to disagree”,
assuming it is possible to proceed with
other actions despite the disagreement.
51 / 54
Team Action
• In order for an action to be performed, it must be
at least endorsed by the authorized agent, and
committed to by the responsible agent.
• If the authorized agent endorses the action in
front of the responsible agent who has
committed to the action, the responsible agent
will be expected to either execute the action (by
himself or with help) or explain why not.
52 / 54
Plans for the Future - Explaining
Responses
• In a training simulation, it is important that
the trainee understand why his proposals
or actions are incorrect or inappropriate.
• Work is being done on better methods of
explaining and justifying the responses of
the agents in to the trainee’s proposals.
53 / 54
References
•
Audio, Video and VR
– J. Rickel et al. Towards a New Generation of Virtual Humans for
Interactive Experiences .
– W. Swartout et al. Towards the Holodeck: Integrating Graphics, Sound, Character
and Story.
•
The Agent Models
– J. Rickel , W. L. Johnson Animated Agents for Procedural Training in Virtual
Reality: Perception, Cognition, and Motor Control.
– J. Rickel Extending Virtual Humans to Support Team Training in Virtual Reality.
– D. McAllester, D. Rosenblitt Systematic Nonlinear Planning.
•
The Negotiation Model
– S Larsson, D. Traum Information state and dialogue management in the TRINDI
Dialogue Move Engine Toolkit.
– D. Traum, Computational Theory of Grounding in Natural Language
Conversation.
•
More articles (and other related material) can be found at the web page of
The Institute for Creative Technologies
54 / 54