International Graduate Research Workshop 2013
on Adaptivity and Personalization in Informatics, March 23‐24, 2013
Edmonton, Alberta, Canada
Dr. Fuhua Lin
Outline
 Introduction
 Related Work
 The Roles of Agents
 Architectural Design
 QuizMASter
 Conclusions and Future work
Goal of this Tutorial
Discuss the basic concepts on
constructing virtual worlds with
multiagent systems (MAS).
Virtual Worlds
 Virtual Worlds are 3D graphical
environments that provides more
engaging and more immersive
user experience
 can provide
 more dimensions than physical
environments
 more social
 nuanced ways (in ways which
are able to show slight difference
that may be difficult to notice but
are fairly important ) for people.
Athabasca University’s Academic
Research Centre in Second Life
 AU's new ARC building was exactly replicated on the
AU island to assist with furniture placement and other
occupancy decisions. A video of this building is
accessible here:
 http://www.youtube.com/watch?v=pganB4_F7AQ
There are avatar only in this demo!
The avatars are 3D characters controlled by the
participants.
Multiagent Systems
 MAS --- encompasses distributed problem-solving
applications, such as network management, which
 do not typically involve Virtual Worlds
 has a focus on inter-agent Communication.
Coordination and Negotiation.
 COMP667 (Multiagent Systems) of Athabasca
University
Group Decision Making
 There are many settings where a
potentially large number of agents,
each with its own goals and
objectives, collectively interact so as
to produce a solution to some
problem.
 A solution that is produced under
these circumstances often reflects
the “tug-of-war” that led to it, with
each agent trying to pull the
solution in a direction that is
favorable to it.
Game Theory
 The field of game theory provides a
natural framework in which to talk
about what happens in such
situations, when a collection of
agents interacts strategically --- in
other words, with each trying to
optimize an individual objective
function.
Software agent 1
Software agent 2
Software agent 3
Software agent 4
NPCs
Software agent 5
Software agent 6
Software agent 7
Software agent 8
Avatars
Autonomous Agents
Russell and Norvig,
Autonomous Agents
in Virtual Worlds
 Using virtual worlds as a technique for
exploring agent behavior and agent
believability, or
 Using agents as a way of extending virtual
worlds into new application areas,
including
 Synthetic agents: intelligent actors
(Hayes-Roth et al. 1996), virtual actors,
virtual humans, (e.g. NPCs)
 Avatars (which are physical representation
of human users)
 In a 3D multi-user Web environments.
AGENTS
• Exist in an environment which they need to
•
•
•
•
•
perceive
Autonomously react to changes in that
environment
Proactively act to achieve goals
Interact with other agents
Behavior can be emergent (often coded in a
declarative language)
Belief-Desire-Intention (BDI) models a subset of
the human mind
 Why Autonomous?
The more autonomous, the more
convincing to the user and sustains the
feeling of presence in a virtual world!
Behavior Modeling:
Agents’ Level of Autonomy (LOA)
 Degree of autonomy of virtual environments depends on the
autonomy of their components, three level of autonomy
(LOA) (Thalmann, 2000):
 Guided: user-guided, like avatar.
 Programmed: could use omniscient approach, but inefficient.
 Autonomous:
 Have perception limitation
 Prediction about the world are always fallible
 The potential for reuse of agents in different virtual worlds
 The ability to distribute individual agents over separate processors
Is Autonomy useful and appropriate
for agents in virtual worlds?
 Omniscient agent
management soon runs
into combinatorial
problems when it must
track of what each agent
is supported to know
and perceive.
NPCs
 Realistic non-player characters (NPCs) are essential to
making virtual environments more real for players.
 This is true in video games where more believable NPCs
support the story narrative of a game, making them more
immersive, more convincing.
 It is also true in other areas where virtual worlds are used
such as education, increasing the effectiveness of those
environments.
PROBLEM
• Non-player Character (NPC) behaviors in 3D virtual
worlds are not sophisticated enough
• Impacts believability and ability to create complex
scenarios
• Domains
• Video games
• Education/Training virtual environments
SOLUTION
• Improved Artificial Intelligence
driving NPC behavior
• Machine Learning
• Natural Language Processing
• Machine Perception (watching the
user)
• Multi-agent Systems (MAS)
POSSIBLE APPROACHES
• Two approaches to combining an MAS with a game
engine
• A fully custom integrated solution
• A modular solution, integrating existing components
Gamebots
 A fully custom integrated solution
Virtual Singapora
Problem in using MAS in Virtual
Worlds
 Game AI is usually closely coupled with
other parts of the game code which
makes it hard to reuse or replace.
 Requires a large amount of investment
in time and resources and a high level
of expertise in Agent Oriented
Software Engineering (AOSE).
GOAL
• Create an integrated framework to simplify the process
of creating agents to control NPCs in Open
Wonderland
• Animation
• Movement
• Environmental percepts
• Synchronization
PAST RESEARCH
 Gamebots/Pogamut (2002 - “GameBots: A Flexible Test
Bed for Multiagent Team Research”)
 Commercial server (Unreal Engine)
 Open source client (using Unreal scripting language as
API)
 Other Projects
 Not Java-based (C, C#) or not open source
 OpenSim (C#) supports Second Life protocol
Agents’ Roles
AgentcontrolledNPC
 Research that has been done
with virtual agents and multiagent systems can be
leveraged to create more
realistic NPCs and purposeful
communication channels
among agents for
applications like game-style
educational activities.
Human
User’s Avatar
Our Approach
 Controlling NPCs with intelligent agents
through the creation of an interface
between a multi-agent system to a virtual
world engine.
Open Wonderland
 Open Wonderland is a toolkit for building 3D virtual
worlds (Kaplan, J., & Yankelovich, 2011).
 The architecture of the system, based entirely on open
standards, is highly modular and designed with a focus
on extensibility.
 Kaplan, J., & Yankelovich, N.: Open Wonderland: An
Extensible Virtual World Architecture. ;IEEE Internet
Computing(2011), 38-45
Underlying
Technologies
 Jason
 A platform for the development of multi-agent systems
Java-based open-source MAS used for creating BDI agents
that are based on the model of belief, desires, intentions.
 A Java-based interpreter for an extended version of
AgentSpeak. The core code is easily extendible making it
easy to add customizations at the agent and environment
level.
 AgentSpeak, a Prolog derivative that is well suited to
programming in AI, is the language used to control agents.
 AgentSpeak is only language that will be supported, at
least initially, when it comes to agent scripts run to control
NPCs in Open Wonderland.
 Declarative style --- offers a different way to script NPC
behavior than the more functional methods already
available in Open Wonderland (like JavaScript).
Underlying Technologies
 JADE
 Container model
 Agent Management System (AMS)
 Directory Facilitator (DF) to facilitate
the management of agents.
 CArtAGo –
 (Common ARTifact infrastructure for
AGents Open environments)
 Agents & Artifacts (A&A) model, using
workspaces, agents, and artifacts
(exposed resources).
MEMORY
Perception
A Client with TextChat Echo Agent
Proximity Detection
/* Initial goals */
!listen.
@msgHandler[atomic]
+proximity(FromUser,Enter,Distance) :
proximity(_,_,_)[source(percept)]
<- .print(FromUser);
text_chat_send_message(FromUser, "Proximity
Agent", "").
@listenHandler[atomic]
+!listen : true
<- proximity_listen;
text_chat_send_message("listening to
proximity messages", "JADE Controller", "");
cheer.
MOVEMENT AND ANIMATION
Interactions among Agents
Quiz Games in Classrooms
 In classrooms, teachers usually use quiz
games to create some interesting
activities.
 The purposes of Quiz Games for the
classes
 Good for reviewing and reinforcing
previously taught material
 Good for warming up or ending lesson on a
high
 A quiz encapsulates the basic unit of
conversation
Quiz Master
 Quiz master
 A TV game show, where a small
group of contestants compete
by answering questions
presented by the game show
host.
Host -
Pedagogical
agent
Purpose of the Project
• Build QuizMASter

a virtual world based
educational application
that mimics a quiz game
in classrooms.
 To build engaging, affectionate,
and effective pedagogical
agents which is the virtual host
in QuizMASter.
How?
Transformed Social Interaction –
TSI Theory (Bailenson et al. 2008)
 to describe the transformation of interaction in
mediated communication environments.
 Three principles of TSI
Principle 1: Self-Representation
 There is evidence in support of
the advantage of selfrepresentation.
 For example, Bailenson, et al.
found that the morphing of
faces of political candidates
with potential voters will
increase the affective bonding in
low information context
(Bailenson et al., 2006)
Principle 1: Self-Representation
 (Con’t)
the dramatic and subtle
changes to the
 Appearances
 Behaviors
 of the avatars chosen by
the users, such that
emotional bonding can be
established.
user
Implementation of the TSI
dimensions
Generic pedagogical agent:
Stephen
 Self-representation –
morphing of real faces
into the avatars
Transformed face of the
pedagogical agent that Oscar
will see in the virtual world
User 1: Oscar’s face
=
User 2: Steve’s face
40:60
In shape only
Transformed face of the
pedagogical agent that
Steve will see in the
virtual world
eVolver (http/l//www.evolver.com)
Principle 2: Sensory-Abilities:
 These transformations empower avatars
to complement human perceptual
abilities.
Agent-enhanced host avatar
 Agent technology’s roles:
 the sensory ability to process the
behaviors of contestants:


eye gazing or moving data
emotional status
 Detect students’ personal taint and
preferences to provide personal greeting,
etc.
 Learn about knowledge to make decision
 Provide advice/reminders to contestants.
 Express emotions to the contestants
Implementation of Sensory abilities
 Applied by recording and
observing the student avatar
behaviors, especially his/her gaze
direction.
 After all contestants enter the
studio, the Host Agent will greet
each and every student,
addressing his/her name with
eye-gaze directed to that student.
User monitor --- tracking of viewing angles of
participants
Contestant Name: Lin
Data: Lin's Look-Around behaviour during every
question session
Date: Fri Sep 23 22:01:48 MDT 2011
Question Number 1 Session -->Lin Did Not Looked
Around.
Question Number 2 Session -->Lin Did Not Looked
Around.
Question Number 3 Session -->Lin Did Not Looked
Around.
Question Number 5 Session -->Lin Did Not Looked
Around.
Principle 3: Situational Context
 Transformations
 Altering the spatial or temporal
structure of a conversation.


For example, the communication
between agents and students can be
optimally configured in terms of the
geographical setup of a conference
room.
For example, a class of 20 students can
sit directly in front of the virtual
instructor, and perceive the rest of the
students as sitting farther away.
Principle 3: Situational Context
(Con’t)
• Altering the flow of rendered time
in the communication session,
users can implement strategic
uses of rewind and fast forward
during a real-time interaction in
an attempt to increase
comprehension and efficiency.
• E.g. the point for providing hints
…
Scenario
 Participants
 Contestants
 The host
 The Audience
 Process of a QuizMASter session
 The host greets the contestants
 The audience will be cheering for and
looking at all the contestants
simultaneously.
 Questions are displayed.
 Contestants will answer the questions.
Scores will be kept.
• Another video showing the
introduction of the host and
the audience responses.
Demo
 http://www.youtube.com/watch?v=P0Gum-
RaMfo&feature=player_embedded
 http://www.youtube.com/watch?v=jPQLnZHewFg
 http://www.youtube.com/watch?v=r-d4FSqpoV0