V-Lab … An Intelligent Virtual
Laboratory for Autonomous
Agents – Towards Simulating
System of Systems
Mo Jamshidi, Ph.D., DEgr., Dr. H.C.
F-IEEE, F-ASME, F-AAAS, F-NYAS, F-HAE, F-TWAS
Regents Professor, Electrical and Computer Engr. Department
and
Director, Autonomous Control Engineering (ACE) Center
University of New Mexico, Albuquerque
Advisor, JPL (1992-93), NASA Headquarters (1996-2003)
Sr. Research Advisor, US AF Research Lab. (1984-90,2001-present)
Consultant, US DOE Office of Renewable Energy (2001-2003)
http://ace.unm.edu www.vlab.unm.edu pursue.unm.edu,
moj@wacong.org
OUTLINE
• Introduction to V-Lab ®
• Discrete-EVent simulation Specification (DEVS)
• Intelligent DEVS … I-DEVS
• Applications to Multi-agent Systems
• System of Systems Engineering
• Simulation and Experiment Movies
V-Lab®
is …
… a NASA Research Announcement (NRA),
Cross-Enterprise Grant with NASA Ames
Research Center and a joint effort between UNM
ACE Center, JPL, in cooperation with the Arizona
Integrate Modeling and Simulation Center (U
Arizona and Arizona State U) Dr. Edward Tunstel,
Jr. of JPL is one of the 3 Co-PIs on the grant.
V-Lab®’s potential role for NASA
Robotic Colonies – System of Systems Paradigm
MER
MER
MER
MER
Simulation Design
Qualities

Layered Architecture


Object Modularity


Breaks up the entire simulation into layers,
each with a distinct purpose and function
Functional components of the individual layers should
be broken up into distinct objects to allow for:
• Maintainability – Changing a single object without
modifying entire simulation.
• Reuse – Distinct objects can be reused in different
simulations
• Object Hierarchy – Provide a structure for the
composition and communication of objects
Our Solution – V-Lab®
V-Lab®

V-Lab® Environment Layers

Hardware Networking
• Networking foundation for intermachine communication

Middleware
• Software environment for interprocess communication
(CORBA, HLA, Sockets, etc.)

I-DEVS
• Library of tools for
computational intelligence in
DEVS formalism

V-Lab®
• Organizational structure for
DEVS objects.
• Management objects to control
time and message flow in multiagent systems
V-Lab®
I-DEVS
MIDDLEWARE
PHYSICAL NETWORK
DEVS



Provides C++ and Java
classes for object design
A-BC
A
Hierarchical
structure of objects
B-C
B
Two Types of Objects
 Coupled (A-BC, B-C)
C
Hierarchal tree
• Contains other objects

Atomic (A, B, C)
• Basic functionality
to be used by
coupled objects
A-BC
A
B-C
B
C
Coupling Relation
DEVS execution

Create Models
Determine imminent models

Imminent models send messages to output
ports
 Imminent models call Internal
Transition function
 Other models receive external messages
 Other models call External Transition
function
Repeat until no models can be imminent


IDEVS … Intelligent DEVS
DEVS
Discrete
Events
Coupled Models
Atomic
Models
Computational
Intelligence
Fuzzy-DEVS
Object: Implement various fuzzy functions and
operations within a DEVS Java environment.
Example – fuzzy inferencing
A DEVS model for a typical fuzzy rule:
IF x is A OR y is B THEN z is C
x
X
μ
A
Y
OR
y
X
μ
AMF
B
Y
X
μ
CMF
C
Y
z
Fuzzy-DEVS,
NEG
ZERO
Cont’d.
POS
a) Membership functions for fuzzy-DEVS
b) The fuzzy-DEVS controller
1
Theta
0.5
0
-0.5
c) The close-loop system
0
5
10
15
20
Time(sec)
25
30
35
d) The output of the system ()
Example of using fuzzy-DEVS to control an inverted pendulum
GA’s Basic Cycle
New Population
Old Population
Recombination
Evaluation
Crossover
Mutation
Selection
A binary chromosome
0 1 0 0 1 1 0 1 0 1 00
GA-DEVS, Cont’d.
GA-DEVS implementation inside V-Lab®
GA-DEVS, Cont’d. 2
A simulation window of GA-DEVS
Stochastic Learning
Automata




What is SLA?
SLA is a sequential machine.
Given a finite number of actions that can
be performed in a random environment,
when a specific action is taken place the
environment provides a random response
which is either favorable or unfavorable.
The objective in the design of the
automaton is to determine how the choice
of the action at any stage should be
guided by past actions and responses.
Stochastic Learning
Automata – Reinforcement
Learning
Stochastic
Learning
Automaton
Update
Action
Probabilities
s(n)
y(n)
G(.)
F(w(n),s(n))
SLA-DEVS
Stochastic Learning Automaton – A statistical
Learning approach to learning …
Schematic of SLA inside the DEVS Environment
Simulation, DEVS-SLA …
2 Agents
(a)
(c)
(b)
(d)
Simulation, Cont’d …
(e)
(f)
(g)
(h)
2 Agents
Neural Networks Multi-Layer
Perceptron
Output 1
Input
Output 2
Input Layer
Output Layer
Hidden Layers
The interest in neural networks comes from the networks’ ability
to mimic human brain as well as its ability to learn and respond.
NN-DEVS
Neural Networks – A neural-based approach
to learning … BPNN
A perceptron structure inside DEVS
NN-DEVS
Neural Networks – Logical connectives “AND”, “OR”,
and “XOR” are implemented in IDEVS
(a) AND
(b) OR
Number of training data errors versus epochs
(X, training epochs; Y, number of training
data errors)
(c) XOR
V-Lab



Uses the I-DEVS
structure for creating
objects
Organizes objects into 6
different categories
 SimEnv/SimMan
 Terrain Models
 Agent Models
 Physics Models
 Control Models
 Dynamic Models
Provides objects for the
time and message flow of
simulation
SimEnv
Terrain
Physics
Agents
Control
SimMan
Dynamic
Hierarchical Tree
SimMan and SimEnv

SimEnv





Highest leveled coupled
model
Instatiates all other
models
Houses all other models
Couples all other
models together
SimMan


Message liaison
providing indirection
between all other
models
Controls flow of time for
the simulation
SimEnv
Agent
Control
in/out
in/out
in
in
Physics
in/out
SimMan
in/out
Dynamic
out
in/out
Terrain
Coupling Relation
out
High-Level Models

Physics



Terrain


Model physical phenomena
Possibly a differential equation solver and set of
differential equations
Information about the layout of the environment
Dynamic

Analyze agent’s actuator state to make
environmental changes, e.g. agent’s position and
velocity.
High-Level Models

Control


Algorithms that control the actuators of an
agent model
Agent


Represent the agents in the simulation
Contain Sensor models
• Sensor models contain all information about
external world agent is aware of

Contain Actuator models
• Actuators contain state information that dynamic
models use
V-Lab® Cycle
Simulation execution is iteration through a 5 phase cycle

Phase 1


Phase 2


Control Algorithms update Agent Actuators
Phase 4


Update Agent Sensors
Phase 3


Check for termination conditions
Wait for all actuators to change state
Phase 5

Dynamic models update agents and environment
A V-Lab® THEME
EXAMPLE SIMULATION
SimEnv
Rover
Dynamics
Sensors
Terrain
Model
SimMan
Controller
Block Diagram of Theme Example
Plot
Model
A THEME EXAMPLE
SIMULATION – Rover Dynamics
1
 vr  vl  cos 
2
1
y   vr  vl  sin 
2
1
   vr  vl 
l
x
inw
||vv
rr
,max
xv
cos
syv
w
&


&


x
vl
vr
dx/dt=vcos
dy/dt=vcsin
d/dt=w
y

A THEME EXAMPLE
SIMULATION – Fuzzy Controller
a) Distance to objectiveb) Angle error
b) Angle error
c) Forward velocityd) Angular velocity.
d) Angular velocity.
Rule1: IF distance is ‘Near’, THEN forward velocity is ‘Slow’.
Rule2: IF distance is ‘Far’, THEN forward velocity is ‘Speedy’.
Rule3: IF angle is ‘Right’, THEN angular velocity is‘TurnRight’; forward velocity is ‘Slow’.
etc.
Pushing task:
G
B
fl<0
rB
rA
A
wl
fl=0
fl>0
Pushing
region fr>0
fr=0
wr
fr<0
Cooperative pushing task:
G
wl
wr
Pushing
regions
B
rB
Cooperative Pushing Task
“in”
“in”?object_reached
“in”?command
“in”?object_lost
“out”!goal_reached
“out”?object_touched
“in”?outside_pushing_region
“out”?object_reached_the_goal
“in”?everybody_ready
“out”
Simulation Results
GA-DEVS Controller (1)
y

 x

 y

 

Goal (xg, yg)
ir2
ir1
vl
w
θg
v
θ
l


1
(v r  vl ) cos 
2
1
(v r  vl ) sin 
2
1
(v r  v l )
l
 x 
cos  
0 
 y   v   sin    w  0
 


 

 
 0 
1
ir3
(x, y,θ)

vr
x
GA-DEVS Controller (4)
Goal
1
Goal
2
Goal
4
Goal
3
Goal
5
Goal
6
First simulation sample task using the GA control system, T=142 steps
GA-DEVS Controller (5)
Goal
1
Goal
2
Goal
5
Goal
3
Goal
6
Goal
4
Goal
7
Goal
8
Second simulation sample task using the GA control system, T=236 steps
GA-DEVS Controller (6)

Trajectory of the robot in a task
X
X
Trajectory of the robot for first task
Y
Trajectory of the robot for second task
Y
THEME EXAMPLE
SIMULATION - Sensors
Infra-red sensors. … Each sensor has a unique ID,X_s, Y_s and
theta_s, where X_s is the X-position, Y_s is the Y-position and
theta_s is the angle of each sensor relative to the rover in X-Y
plane.
GPS sensors … An atomic model simply simulates a position sensor, giving out
the current position of the rover
Compass Sensors … Respective atomic models simulate the sensing by
converting the angle  of the rover from radians to degrees.
A THEME EXAMPLE
SIMULATION – Distributed 2
Parts of a distributed simulation with GenDEVS and Sockets
Commercial ROVERS
Two Pioneer II Mobile rovers are
used to implement IDEVS-JAVA
(V-Lab®).
Movies will show demos at the end.
Experiment Results (DEVS-Fuzzy)
Experiment Results
(GA-DEVS)


Application: Pioneer 2 mobile
robot
Two experiments of path planning
 Goal searching task w/o obstacles. Robot
(10, 9, -π/2)  (4, 4, -π/2)returns to the
home point (10, 9, -π/2) after traveling to
the goal point.
Robot task of goal seeking (Home -> Goal -> Home)
System of Systems - Introduction


Changing Aerospace and Defense
Industry
Emphasis on “large-scale systems
integration”



Customers seeking solutions to
problems, not asking for specific vehicles
Mix of multiple systems capable of
independent operation but interact
with each other
Emerging System of Systems Context
EMERGING CONTEXT: SYSTEM OF
SYSTEMS

Meeting a need or set of
needs with a mix of
independently operating
systems


New and existing aircraft,
spacecraft, ground
equipment, other
independent systems
System of Systems
Examples




Coast Guard Deepwater
Program
FAA Air Traffic
Management
Army Future Combat
Systems
Robotic Colonies
System of Systems

SoS: A metasystem consisting of multiple autonomous
embedded complex systems that can be diverse in:








Technology
Context
Operation
Geography
Conceptual frame
An airplane is not SoS, an airport is a SoS.
A robot is not a SoS, but a robotic colony is a SoS
Significant challenges:

Determining the appropriate mix of independent systems

The operation of a SoS occurs in an uncertain environment

Interoperability
Keating, et al., 2003
System of Systems Definitions
SoS: No universally accepted definition
1. Operational & Management
independence+Geographical Dist. +
Emerging Behavior+Evolionary Dev.
(ML, Space)
2. Integration+InterOperability.+Optmiz. to enhance
battlefield scenarios (ML)
3. Large scale + distributed Systems
Leading to more complex systems
(Private Enterprize)
4. Within the context of warfighting
systems – Inter Op.+Com’d.
Synergy+Cont.+ Comp.+ Comm.
+Info. (C4I) & Intel. (ML)
Keating, et al., 2003
DISTINCTION BETWEEN SYSTEM
ENGINEERING AND SoSE
SoSE represents a necessary extension
and evolution of traditional system
engineering.





Greatly expanded SoS requirements
for tiered levels of discipline and rigor.
Centralized control structure vs. decentralized control structure
A typical individual system (well
defined end state, fixed budget, well
defined schedule, technical
baselines,homogeneous)
A typical System of Systems (not well
defined end state, periodic budget
variations, heterogeneous)
Nature of SoSE Engineering
Existing Complex Systems
Exclusive, Autonomous, Local
Transformation
Keating, et al., 2003
System of Systems
System of Systems
Integrated,
Aligned,
and
Interconnected,
Integrated
Mission,
Transforming
Global, Emergent Structure
Keating, et al., 2003
System of Systems
Engineering
The design, deployment,
operation, and
transformation of
metasystems that must
function as an integrated
complex system to
produce desirable results.
Keating, et. al 2003
Jamshidi, 2005
System of Systems
PROBLEM THEMES
1. Fragmented Perspectives
2. Lack of Rigorous Development
3. Lack of Theoretical Grounding
4.
IT Dominance
5. Limitations of trad. SE single
system focus
6. Whole Systems Analysis
Keating, et al., 2003
Modeling Issues
Modeling of Systems of Systems?
TOP
SoSE
LSS
TOP
BOT.
ss
SS-1
SS-2
SS-2
LSS
ss
SS-1
LSS
SS-1
…
SS-2
Traditional LSS Modeling
LSS
LSS
LSS
LSS
BOT.
LSS
SoSE Modeling Difficulty
EXAMPLES






Air Traffic Control
Personal Air Vehicles
Future Combat Systems
Internet
Intelligent Transport Systems
US Coast Guard Integrated
Deepwater System, etc., etc.
US COAST GUARD INTEGRATED
DEEPWATER SYSTEM

The United States Coast
Guard



Protect the public, the
environment, and U.S.
economic and security
interests in any maritime
region
International waters and
America's coasts, ports, and
inland waterways.
Missions





Maritime Security
Maritime Safety
Maritime Mobility
National Defense
Protection of Natural
Resources
US COAST GUARD INTEGRATED
DEEPWATER SYSTEM


An integrated approach to
upgrading existing assets
while transitioning to newer,
more capable platforms with
improved systems for
command, control,
communications, computers,
intelligence, surveillance, and
reconnaissance and innovative
logistics support.
Ensure compatibility and
interoperability of deepwater
asstes, while providing high
levels of operational
effectiveness.
Conclusions


V-Lab® is still at the beginning of its development
Three sub-groups are working:



Software: Responsible for the design and implementation of
V-LAB® (from Network to V-LAB®)
Hardware: Implementation of its tasks, features on
industrial or commercial rovers
Theory: Develop new theories to bridge the gap between
Soft Computing (FL, NN, GA, GP, SLA, etc.) and DEVS environment.
Future: Leverage V-Lab ® success
and 35 years of research on LSS for
SoSE applications is among future
directions.
Cover , Modeling and Simulation Magazine, 2003
Publications – to date
1. J. Burge, A. El-Osery, M. Jamshidi and M. Fathi “V-LAB : A Virtual
Laboratory for Distributed Robotic Modeling and Simulation,” Proc. IEEE
Int. Conference on Systems, Man and Cybernetics, Tucson, AZ,
October, 2001.
2. A. El-Osery, J. Burge, M. Jamshidi and M. Fathi “Stochastic Learning
Automaton for Learning Control of Robotic Agents, “Proc. IEEE Int.
Conference on Systems, Man and Cybernetics, Tucson, AZ, October,
2001.
3. A. El-Osery, J. Burge, A. Saha, M. Jamshidi, M. Fathi and M.
Akbarzadeh-T., “V-Lab – A Distributed Simulation and Modeling
Environment for Robotic Agents – Control Through Stochastic Learning
Automata,” to appear in IEEE Transactions on Systems, Man and
Cybernetics , Vol. 32, No. 6, pp. 795-804, December, 2002.
Publications – to date 2
4. A. El-Osery and M. Jamshidi, “Implementation of an SLA-Based Controller in a Virtual Laboratory,”
Proc. IEEE Int. Conference on Systems, Man and Cybernetics, Hammamet, Tunisia, October, 2002
5. M. Jamshidi, S. Sheikh-Bahaei, J. Kitzinger, P. Sridhar, S. Xia, Y. Wang, J. Liu, E. Tunstel, Jr , M.
Akbarzadeh, A. El-Osery, M. Fathi, X. Hu, and B. P. Zeigler, “A Distributed Intelligent Discrete-Event
Environment for Autonomous Agents Simulation,” Chapter 11, Applied Simulation, Kluwer Publishers,
Amsterdam, the Netherlands, 2003.
6. M. JAMSHIDI, S. Sheikh-Bahaei, J. Kitzinger, P. Sridhar, S. Beatty, S. Xia, Y. Wang, T. Song, U. Dole,
J. Liu, E. Tunstel, Jr., M. Akbarzadeh, P. Lino, A. El-Osery, M. fathi, X. Hu, and B. P. Zeigler,
“V-Lab© - A Distributed Intelligent Discrete-Event Environment for Autonomous Agents Simulation,”
Intelligent Automation and Soft Computing - Autosoft Journal, Vol. 9, No. 3, pp. 181-214 2003.
6. M. Jamshidi, S. Sheikh-Bahaei, J. Kitzinger, P. Sridhar, S. Xia, Y. Wang, J. Liu, E. Tunstel, Jr , M.
Akbarzadeh, A. El-Osery, T. Song, M. Fathi,“A Distributed Intelligent Discrete-Event Environment for
Autonomous Agents Simulation,” Chapter 11, Modeling and Simulation Magazine, (Cover article), M&S
International, Snn Diego, CA, 2003.
7. P. Sridhar and M. Jamshidi, “A Framework for Multi-agent Discrete Event Simulation: V-Lab®,” Proc.
IEEE SMC, the Hague, the Netherlands, Oct 10-12, 2004.
Publications – to date 3
8. S. Sheikh-Bahaei, M. Jamshidi,” Discrete Event Fuzzy Logic Control with Application to Sensor-Based
Intelligent Mobile Robot,” Proc. WAC 2004, Seville, Spain, June 28-July 1, 2004.
9. P. Sridhar and M. Jamshidi, “A Multi-agent Discrete Event Simulator: V-Lab®,” Proc. WAC 2004.
Seville, Spain, July 28-July 1, 2004.
10. S. Sheikh-Bahaei, M. Jamshidi, and P. Lino, “An Intelligent Discrete Event Approach to Modeling,
Simulation and Control of Autonomous Agents,” Intelligent Automation and Soft Computing - Autosoft
Journal, Vol. 10, No. 4, pp. 337-348, 2004.
THANK YOU!