Agent-based Simulation Platform Evaluation in
the Context of Human Behavior Modeling
Michal Laclavík, Štefan Dlugolinský, Martin Šeleng, Marcel Kvassay,
Bernhard Schneider, Holger Bracker,
Michał Wrzeszcz, Jacek Kitowski, Ladislav Hluchý
IKT Group - Institute of Informatics SAS
Dept. of Parallel and Distributed Computing
Research and Development Areas:
– Large-scale HPCN and Grid applications
– Intelligent and Knowledge oriented Technologies
URL: http://ikt.ui.sav.sk
Director & leader of PDC:
Dr. Dipl. Ing. Ladislav Hluchý
Experience from European projects:
– 6 project in FP6: EGEE II, K-Wf Grid, DEGREE
(coordinator), EGEE, int.eu.grid, MEDIGRID
– 4 projects in FP7:
Commius, Admire, EGEE III, Secricom
– 1 EDA project: EUSAS
Several National Projects (SPVV, VEGA, APVT)
IKT Group Focus:
–
–
–
–
–
Multi-Agent Systems
Information Processing
Semantic Web
Knowledge oriented Technologies
Parallel and Distributed
Information Processing
Solutions:
–
–
–
–
2 May 2011
AgentOWL: semantic web and FIPA agents
Ontea: Pattern-based Semantic Annotation
ACoMA: KM tool in Email
EMBET: Recommendation System
ITMAS 2011
2
EDA R&T JIP FP project
(for R&T Joint Investment Programme on Force Protection)
European Urban Simulation
for Asymmetric Scenarios
3
ITMAS 2011
2 May 2011
EUSAS Objectives
results
qualitative
evaluation
Serious
Game
updates
interactions
Training
Rules
Of
Engagement
Modelling
updates
models
Learning
results
Data
Farming
updates
interactions
quantitative
evaluation
Analysis
results
4
ITMAS 2011
2 May 2011
ABS Evaluation Approach
• Survey Literature
–
–
List of available simulation platforms is in Deliverable Appendix
Existing ABS evaluations were considered
• Evaluation Criteria/Features
–
12 features selected – list on the next slide
• Principle: Evaluation through implementation
– Exemplary Human Behavior scenario defined
• Civilians getting angry (throwing stones) or afraid (running to safety).
• Soldiers arresting Civilians if hit by stone
– Implemented in MASON, NetLogo and VBS2
2 May 2011
ITMAS 2011
5
Survey Literature
• Stupid Agent Model
– 16 features
– S.F. Railsback, S.L. Lytinen and S.K. Jackson Agent Based Simulation
Platforms: Review and Development Recommendations Simulation 8:9
(2005)
– NetLogo, MASON, Repast, Swarm and Java Swarm
– Later also others, like EcoLab
• Human Behavior Modeling
– We have chose 12 features
– Generic, but evaluated on Human Behavior Scenario
2 May 2011
ITMAS 2011
6
12 Features selected as evaluation criteria
1. Loading and Representing the Environment and the Scenario
2. Creating and Representing Agents
3. Behavior Implementation
4. Movement Implementation
5. Visualization
6. Parameterization
7. Model check-pointing
8. Analytical Tools
9. Logging
10. Performance
11. Standards
12. Development Environment
2 May 2011
ITMAS 2011
7
Exemplary Human Behavior Scenario
• Soldiers
– Catching civilians
if hit twice by stone
– If civilian is caught
(arrested), civilian will
disappear
– Soldiers are robotic
(no emotions)
• Civilians
– Driven by 2 emotions (fear and anger)
– When angry, trying to find stone, going to soldier and throws the stone
– When afraid, flying to safety area (yellow)
2 May 2011
ITMAS 2011
8
NETLOGO 4.1
• Advantages
– NetLogo can be invoked and controlled by
another program running on the JVM by
Controlling facility API (e.g. app which
automates series of model runs, embed
NetLogo models in a larger app)
– Simulation state (i.e. world) can be saved to
a CSV file and later loaded
– Java API for creating custom extensions to
NetLogo (commands, reporters)
– Models can be run without visualization
– Ability to load vector GIS data (points, lines,
and polygons - ESRI shapefiles), and raster
GIS data (grids) into NetLogo by GIS
extension
– Easy to draw graphs, create simulation
parameter controllers (sliders, buttons, etc.)
– Many useful tools like BehaviorSpace,
System Dynamics Modeler, HubNet, Logging
2 May 2011
ITMAS 2011
9
NETLOGO 4.1
• Disadvantages
– Slower than Mason, some parts of user code are interpreted at runtime
– The Controlling facility API is considered experimental. It is likely to
continue to change and grow.
– Support for creating 3D worlds is still in an experimental state. Only 2D
world is fully supported.
2 May 2011
ITMAS 2011
10
MASON
• Advantages
– Fast, overhead of simulation environment is minimal
– Java
– Models are completely independent from visualization
– Models may be checkpointed and recovered
– Agents are not forced to have a physical location,
which is good if we want to create agents
representing groups (meso and macro levels)
– Physical environment – any number of 2D or 3D layers
– Multiple Displays of simulation
– Time series Graphs, variable inspectors
– GIS data can be loaded
• Disadvantages
– NetLogo has better support for movement and
analysis of distances, objects etc. in physical
environment – this impacts development speed, but
gives flexibility
2 May 2011
ITMAS 2011
11
VBS2
• Integration Challenges
–
–
–
VBS is thread- and event-based. Our candidate ABS systems (MASON and NetLogo) are
step-based. Integration is not straightforward but feasible.
Changing the action in the middle of its execution may cause a jerking
animation. For example: while throwing a stone – the agent decides to
run to the safety area
Movement in VBS may be executed a bit differently from what was planned and simulated in
ABS: we need to use waypoints to minimize the discrepancy.
2 May 2011
ITMAS 2011
12
Evaluated Features
• Loading and Representing the Environment and the
Scenario
– MASON
• 2D, 3D, layered: IntGrid2D, Continuos2D
• GIS support: tested
– NetLogo
• 2D: two-dimensional grid of “patches”, 3D experimental
• easy import from bitmap
• GIS support: tested
• Creating and Representing Agents
– Soldier, Civilian, Stone
– MASON
• Represented by Java class (Steppable interface),
step(SimState state) method
• access to environment : SimState state
– NetLogo
• Turtles (dynamic), patches, links and the observer
2 May 2011
ITMAS 2011
13
Evaluated Features
• Behavior Implementation
– NetLogo
• turtle variable and the RUN command
– MASON
• step(SimState state)
• Movement Implementation
– NetLogo
• Direction and step
– MASON
• Go to X, Y
• Flocking, steering: implemented in demo
• Visualization
– MASON: strong separation of Model and Visualization
– NetLogo: possibility to switch off visualization, speed does not
change.
2 May 2011
ITMAS 2011
14
Evaluated Features
• Parameterization
– supported
• Model check-pointing
– Supported in both
– MASON: platform independent
• Analytical Tools
– MASON: improvement over the years
• Property inspectors
• Video, snapshot, streaming, charts
– NetLogo:
• Property inspectors
• snapshot, streaming, charts
• Logging
– MASON: log4j can be used
– NetLogo: using log4j integration
2 May 2011
ITMAS 2011
15
Evaluated Features: Performance
NetLogo
Number of Agents:
1 run
2 run
3 run
4 run
5 run
6 run
7 run
8 run
9 run
10 run
1000 steps (ms):
1 step (ms):
Mason
1000 steps 1000 steps
(ms)
(ms)
15
150
617
26760
483
27407
474
27804
476
27643
454
28200
470
27759
469
27704
453
27887
461
27230
476
27629
483
27602
0,48
27,60
10 steps
(ms)
1500
183323
180116
175386
189573
186249
180111
182526
182228
186770
181913
182820
18281,95
Number of Agents
NetLogo 1 step (ms)
MASON 1 step (ms)
MASON speed vs. NetLogo
2 May 2011
Number of Agents:
1 run
2 run
3 run
4 run
5 run
6 run
7 run
8 run
9 run
10 run
1000 steps (ms):
1 step (ms):
15
0,48
0,10
4,8 x
1000 steps 1000 steps 1000 steps 10 steps
(ms)
(ms)
(ms)
(ms)
15
150
1500
15000
98
516
21491
24716
109
814
23445
24842
99
724
19143
24292
96
479
22102
24778
90
775
23025
24807
91
570
22179
24858
87
485
20682
24908
99
658
22348
24849
130
436
20268
24497
114
439
20426
24883
101
590
21511 2474300
0,10
0,59
21,51
2474,30
150
1500
27,60 18281,95
0,59
21,51
46,8 x
849,9 x
ITMAS 2011
15000
2474,30
16
Evaluated Features
• Standards
– Agent Standards: FIPA – not relevant for simulation agents
– DIS and HLA standards
• Relevant but we did not test
• VBS2 will be integrated for training
– we plan to use the plug-in functionality in VBS2 and CORBA technology
• need to create a FOM - Federation Object Model
• Java based HLA:
– poRTIco
– Java port of CERTI
• Development Environment
– Both step based, easy debug, better then thread based MAS
– NetLogo:
• NetLogo IDE, debugging using variable inspectors
– MASON:
• Any Java IDE, standard Java debbuging
• We have used Eclipse
2 May 2011
ITMAS 2011
17
Conclusion
• Both are almost equal in many features
• NetLogo: better in physical movement support and some
analytical tools.
• MASON: much faster, supports strong separation of
visualization and behavior models, better support for 3D
environment, Java based - easier to integrate with other
systems.
Features
Language
Enviroment
GIS support
Movement
Stearing/Flocking Behaviour
Visualization
run with no visualization
Parametrization
Model check-pointing
Analytical Tools
Logging
Performance
2 May 2011
NetLogo
Logo, Java for simulation control
2D, 3D experimental
Yes
Heading angle + step
Not directly
2D, 2D as 3D
possible but not strictly separated
possible
Yes
Charts, Streamning, variable bars, snapshot
support using log4j
good for tens of agents
ITMAS 2011
MASON
Java
2D, 3D
Yes
just set(x,y)
Not directly
2D, 3D
separated behaviour and visualization models
possible
Yes, platform independent
Charts, Streamning, snapshot, video recording
not direct support but log4j can be used
good for thouslands of agents
18