Gala Roadmap Contribution
Name of lead contributor(s): Prof.A.G.Bruzzone , Dr.A.Tesei, NATO-CMRE
SIG: 3.4: Security, Safety and Crisis Management
General preamble on SIG 3.4
Within this SIG, stakeholders have to:
 deal with danger and high risk
 deal with situations hard to simulate in real life, or scenarios which are too complicated and/or
expensive to simulate as hands-on scenarios,
 train large numbers of people with different roles, competences and tasks, from operational to
management/strategic/tactical levels,
 train often their team leaders and officers to cooperate each others, with other institutions and
agencies, sometimes (such as within NATO) in an international, multi-lingual and multi-cultural
environment.
These general requirements very much affect the selection of upmost topics and there are
interesting examples in this sector as proposed in Figure 1. [34]
In reference to Crisis Management there are several existing studies, projects, tools and Information
Technologies (ITs) in support of Education & Training (E&T) based on SGs such as MACSIM for Crisis
Management: a research on the development of a serious game for crisis management composed by
different modules [24]; for instance in this case agents are used in order to support the background
decision making process.
The application of SGs for Training in emergency management is also of interest; this sometime
evolves in tactical decision games [25]. Training of the non-technical skills is often crucial to effective
management of emergency situations; this issue is currently receiving increasing emphasis in the
petrochemical sector. A case study was developed to address the explosion and fires at the Texaco
Refinery, Milford Haven, UK, which occurred in July 1994, with particular focus on the human factors
aspects of the event [36]. A key issue identified by the official report into this incident was the
importance of emergency management training. This research outlines a novel, low-fidelity training
intervention, the tactical decision game (TDG), which is designed to enhance the non-technical skills
(decision making, situation awareness, communication and co-ordination, teamwork and stress
management) required for emergency management. It is proposed that enhanced learning of these
non-technical skills, through experience and directed practice following repeated exposure to TDGs,
will lead to more efficient emergency management, particularly when dealing with hazardous
materials.
Another interesting work in crisis management starts from the concept that organizations
responsible for crisis management are already using SG technologies [26]; they were involved in
constructing crisis management systems (CMSs) to coordinate response to a crisis, provide decision
support during a crisis, and support activities prior to the crisis and after the crisis. If designed with
gaming in mind, those same CMSs could be easily used in a simulation mode to play a crisis
management game. Such a use of the system would also provide personnel with opportunities to
rehearse for real crises using the same tools they would have available to them in a real crisis. In this
research, authors provide some background for the use of simulation and gaming in crisis
management training, describe an architecture for simulation and gaming, and present a case study
to illustrate how virtual environments can be used for crisis management training.
In this thematic area, as tools and platforms, it is possible to cite the ICONS project, developed by
the University of Maryland [37]: it is a training organization that offers skills-based training programs
incorporating multi-player, real-time simulation exercises as a way for individuals, teams, and
organizations to enhance effectiveness. The training is conducted either face-to-face or through
customized web-based communication system, that is ICONSnet which provide a designed online
distributed role-play simulations.
Another examples is DREAD-Ed: a multiplayer, online serious game that proposes a technologybased teaching methodology to train people on decision making, effective communication, resource
and time management [38]. The DRAED-ED serious game provide a training programme to a broad
range of different target populations: members of the emergency forces, decision-makers in local
and central government and in industry, school teachers, school principals.
Within DREAD-ED’s virtual environment, learners from different backgrounds cooperate to simulate
group interactions and individual decision-making during an unexpected situation.
For Crisis Management a lot of Commercial Games exist as well, such as Darfur is Dying (Internet) An
online game by mtvU that simulates life in a Darfur refugee camp (see Error! Reference source not
found.) [39].
Another example is FloodSim: a flood prevention simulation/strategy game designed to inform the
people of the United Kingdom about the dangers of flooding as well as to help gather public opinion
on the problem that flooding presents to the UK [40]. The player takes control of the UK's flood
policies for three years and attempts to protect the people and the economy of the United Kingdom
from damage due to floods.
In addition, Food Force is another SG addressing humanitarian issues [41]. The UN's World Food
Programme designed this virtual world of food airdrops over crisis zones and trucks struggling up
difficult roads under rebel threat with emergency food supplies.
Another interesting game related to a very critical, long lasting, international crisis is Global Conflict:
Palestine is a 3D-adventure/rpg-game [26]; in this game you are given the role of a reporter in
Jerusalem, and have to write articles for your paper.
Among other commercial solutions for crisis management it is possible to mention:
 Advanced Disaster Management Simulator (ADMS) by ETC. Simulation [42]; this is a virtual
reality system used to immerse an incident commander into a simulated crisis management
situation. A combination of an immersive 3D environment and CM simulation that places the
incident commander (and various role players) within the scene.
 Incident CommanderTM by Breakaway Ltd: a 2D, map based, top down CM game that puts the
player in the role of an incident commander handling a wide variety of CM scenarios. The player
has to co-ordinate the numerous agencies to respond to the emerging crisis.
Based on this survey and SoA (State of the Art) analysis, it was possible to highlight the upmost
characteristics of this SIG:
• Wide range / High numbers of stakeholders
• SG adoption is generally welcome and already well assessed in many different applications
• There are very interested and active users, most related to Defense and Government Forces
and Agencies (e.g., MoDs, Police, Fire dept., airports, etc.)
In order to make this SoA study more sound, a selected number of significant games have been
taken as a reference in the fields of interest ; they are representative examples of the results of the
analysis of SoA/SG current status [1]:
- Cultural Awareness (Afghanistan Pre-deployment) [2]
- Boarders Ahoy! [3]
- OpenSea [4]
- Siqur [5]
- RescueSim [6]
- ADMS [7]
- Sibilla [21]
- Ramses [22]
- S4PT [23]
Fig.1 - Sample of Games for Decision Making and Crisis Management
Topic
High realism and high fidelity
Relevance/priority
(on a 1-5 scale)
SoA/ SG Current
Status description
(with references)
4 (high but not top, as if you exaggerate with realism and fidelity you risk to
make the game too difficult to play, too expensive, even boring)
High realism and high fidelity is a traditional topic in these fields of application
[8]. The main reason is the high risk correlated to the applications: non-realistic
game scenarios may mean not only uselessness but even dramatic
consequences, such as risk for life.
However, the need for a balance among reality, meaning and play spheres is
recognized [9][10] in order to make the game both useful and meaningful,
especially from training point of view.
Users in security and crisis management application fields are the most
demanding within the SIG on this topic. On security, in particular, SGs are
requested to train either staff or troops belonging to Armed Forces at
operational level, but also at tactical/strategic, high level [2,3,4]. These SGs
must achieve a high degree of realism on points where the training is focused.
If these points are uncertain when the architecture of the SG is designed, or if
they are subject to change as training is expanded, then it means that a high
degree of realism must be achieved more generally throughout many
elements of the SG. When carried to cover the widest possible training
objectives, this ultimately means very high-fidelity, including detailed physical
modeling, scalable multi-player capabilities, and often also refined 3D
graphical scene rendering and its specialized hardware striving for the ideal of
full immersion. Most of these requirements are shared by users interested in
crisis management, most of who, though, are not interested in specialized
hardware.
At the present, high realism and high fidelity are generally achieved through:
 sophisticated 3D graphics (such as VBS2-derived games such as Boarders
Ahoy!; Ramses; OpenSea Tactical Theatre Simulator; RescueSim, etc.);
 integration of refined mathematical/ physical models (events, sound, etc.)
and use of real data (previously collected) to give realistic inputs to a player
(OpenSea and RescueSim);
 involvement of real experts/actors on the field in the design phase
(OpenSea, ADMS, RescueSim);
 supply of dedicated hardware man-machine interfaces (e.g., ship bridge
consoles in OpenSea)
Identified user wishes to further increase realism&fidelity are:
 Better human perception, for example through:
Requirements
Opportunities
SoA Gaps (if any)


o
Man-machine interfaces equipped with more multi-media tools /
hardware and more physical devices/tools, either real or phantoms in
order to increase human perception and player’s immersion
o
More real data from sensors (radar, sonar, cameras, etc.)
More extensive use of non linear mechanisms and artificial intelligence
More realistic crowded scenarios, in particular, more realistic dynamic
modeling of crowds in terms of natural movements of single elements of a
crowd (ADMS represents a significant example of attempt to solve the
problem, but with not fully satisfactory results)
Topic
Interoperability and standardization
Relevance/priority 4
(on a 1-5 scale)
The relevance has been found to be high, but this requirement is not totally
general. In some cases mainly in security field, users ask for heavy use of
dedicated hardware, which cannot be always combined with interoperability.
Hence, in general, different games respond to different requirements (e.g.,
Afghanistan Pre-deployment course vs. OpenSea).
SoA/ SG Current
Standardization enables the accomplishment of various goals, such as technical
Status description performance, learning effectiveness, better performance in development,
(with references)
broadness of distribution with on-line/remote access of games, and improved
quality in educational environments. Standards and interoperability have
become an explicit goal in national and international initiatives of Serious
Games (SG) research, industry, and policy [11] because the advantages
associated with them are massive: they foster innovation and competition,
enhance diversity, give consumers choice, and can lead to unexpected benefits
over time [12]. In the technical context of serious games, interoperability
concerns the ability to transfer and make useful data and other information
across systems, applications, or components [13]. Up to now, there is little
research related to the interoperability between SGs and LMS platforms, most
of them use the SCORM (Sharable Content Object Reference Model)
specification to package and deploy web-based SGs and to send and receive
information from the LMS. Up to now, there is no standardized specification or
standard to integrate desktop games with an LMS platform [11]; however
there is a research trend related to the SCORM High Level Architecture (HLA),
which aims at integrating training simulation software with an LMS [14][15].
Requirements
Serious Games, unlike entertainment games, necessitates the implementation
of specific educational and technical requirements related to pedagogical
constructs, learners’ assessment and standardization.
For SG users interoperability corresponds to the ability to transfer and make
useful data and other information across systems of different nature,
applications, or components.
The application of standards and interoperability is seen to have the potential
to improve ease of development and use, while lowering overall cost.
Opportunities
Within GALA, MAN has addressed the role of standardization and the
interoperability between SG and Learning Management Systems (LMS).
A case study on the implementation of the “Cultural Awareness Training Afghanistan” game on the ILIAS platform (Integrated Learning, Information and
Work Cooperation System) of the “Carol I” National University of Defence
(MAN) in Romania has been carried out. The Cultural Awareness Training is
part of the Afghanistan Pre-Deployment Course developed by the Joint
Advanced Distributed Learning and Simulation Section, NATO Allied Command
Transformation. The ILIAS platform is an open source web-based learning
management system (LMS) – compliant with standards such as SCORM 1.2 and
2004, LOM, AICC, QTI, XML - that supports learning content management and
tools for collaboration, communication, evaluation and assessment. The main
aim of MAN’s work is to highlight the advantages and the restrictions of
implementing a SCORM compliant game into an LMS, in order to provide
practice-based guidance to game developers.
A very interesting example is provided by S4PT example were drones and
marine assets are simulated over a distributed real time HLA federation [23]
SoA Gaps (if any)
Up to now, there is little research related to the interoperability between SGs
and LMS platforms.
Up to now, there is no standardized specification or standard to integrate
desktop games with an LMS platform.
Serious Games and LMSs require further attention, in order to explore and
employ their full capacity within learning environments. Interoperability
initiatives, such as between SCORM compliant games and LMSs, although not
universally accepted, can have a significant impact on serious games
development.
Eventually, this and other interoperability standards should evolve to a level
that enables developers to create and integrate serious games more quickly
and easily and reduce needless duplication of effort.
Topic
Multiple player mode
Relevance/priority
(on a 1-5 scale)
SoA/ SG Current
Status description
(with references)
3
Requirements
Opportunities
SoA Gaps (if any)
This topic is particularly highlighted by stakeholders involved in crisis
management, and partially in security (mainly at strategic/tactical level [4]),
where a lot of actors of different kinds play an important role and must
cooperate [6] towards a common goal.
Training to cooperation
Possibility to have multiple players, possibly with different roles in the game
Possibility for the players to cooperate towards a common goal
Possibility to have players at different hierarchical levels (e.g., strategic role,
resource manager, operational role)
Possibility for the players to interact not (or not only) through the computer
(or the game platform in general), but directly between humans, for example
by talking.
Possibility to play through a network (e.g., a LAN, internet, etc.)
There are very interesting, refined and advanced games in the field that satisfy
most of present requirements. Significant examples are RescueSim and ADMS.
The possibility to play in a cooperative way can be provided either in a
classroom (RescueSim; Ramses [44]), or remotely (Reckon&Choose! [16]).
Multiplayer web game (Sibilla) demonstrated a potential for teaching classes
on communication and coordination in different contents such as Intelligence
and Homeland Security and Strategic Decision Making obtaining interesting
results [30]; the approach allows even to investigate the impact of different
cultural and educational background on team-working within cooperative and
competitive frameworks
One of the most interesting user wishes recorded during our surveys is the
possibility of interacting among players not (or not only) through the computer
(or the game platform in general), but directly. This is not available in present
games at our knowledge. Given the high technological potential (multi-media)
available nowadays, this gap might be filled in the future (possible idea, voice
recording, interpretation and translation into the game, while two players talk
together).
Topic
Feedback & Performance Assessment
Relevance/priority
(on a 1-5 scale)
SoA/ SG Current
Status description
(with references)
5
Requirements
Opportunities
Methodological assessment of performance is not very frequently reported in
literature in the case of security applications.
It usually comes from general approaches (see for example the Appendix on
possible structures for research in debriefing reported in [17]).
The most common approach to analyse the effectiveness of SGs in the security,
safety and crisis management fields is a simple test after the game session or a
debriefing session among stakeholders [18] or between students and tutor.
However the number of SGs is increasing, which have got as significant
features a direct, simultaneous feedback during the game session and a final
assessment of the player’s performances.
As abovementioned, interoperability makes assessment easier and remotely
available [11] and is highly recommended.
An interesting example is the module of Cultural awareness in Afghanistan
pre-deployment course [2]: a risk indicator appears every time the player
makes a decision which immediately indicates the correctness of that decision.
On the other hand, the final assessment is not detailed, just a quick summary
of what happened.
In other cases [5] feedback and assessment are achieved by measuring during
the game several parameters, such as needed time, number of tasks
completed, percentage of completeness, etc., and providing the player with a
more detailed assessment report at the end of the game. This implies logging
at least some of the player’s actions along the game.
A complete logging of occurred events is much less frequent (very good
examples are the logging capabilities provided by VBS2 platform).
Appropriate feedback during the play session (simultaneous feedback).
Simultaneous complete logging of player’s actions.
Adequate information level (“useful and effective” for most of users means:
not verbose, details on demand, simple but focused hints, etc.)
Hierarchy of information details:
- General: always available
- More detailed: available on demand (e.g., through pop-ups)
A general feedback and assessment at the end of the game, with percentage of
completion, measurement of learning progress with respect to the initial
knowledge of the player
Interoperability
Non-invasive measurement of physiological indicators
As an interesting, recent case study, related to capabilities of logging and
computerized debriefing is represented by the recent application of a logging
tool (NISIDA+) to MSTPA, the MultiStatic Tactical Planning Aid [20] developed
at CMRE with a view to providing both scientists and operational personnel a
powerful, configurable tool in order to assess the performance of (multistatic)
sonar sensor configurations. Through Nisida+ software package, developed at
CMRE in late 2012, the players can play back all their actions/decisions with
the aid of a specific graphical interface, and, hence, check and discuss their
performance with their tutor/trainer.
Another interesting opportunity is provided by crowdsourcing over the web as
some game potential demonstrate (i.e. Sibilla) [35]
SoA Gaps (if any)
Possibility to measure learning progress with respect to the initial knowledge
of the player
Physiological/neurological parameters to measure in a non-invasive way during
the game (through the application of specific sensors, dedicated hardware and
software systems): measurement of attention, stress, tiredness, brain activity,
etc. Equipment and devices exist but are rarely applied to SGs (of course also
due to cost issues)
2) Vision and Gap
For each identified topic in SIG it is proposed a vision for the year 2020; this supports the
identification of the gap between the state of the art and the vision. The analysis allow to highlight
research needs and steps (also including tools, methods, etc.) to achieve the vision.
Indeed for 2020 the SG technological developments are expected to improve performance, while
mobile solution and hardware will become more integrated in working/living process [27]; a major
shift will be due to the fact that users will have a much more positive attitude versus SG due to the
generation transition to the “Gamers”. [28]
Topic
Vision 2020
Gap: SoA - Vision
Research
Steps
Need
and
Possible
related
topics/trends/enablers
(indicate
conditions
and relationships)
High realism and high fidelity
SGs realism is not rarely a nightmare, so, while graphic continue to evolve,
the SG fidelity is often pretty low; this in Crisis management could lead to
negative training and it will require an evolution of serious games from
several point of view: in primis by introducing models supporting dynamic
interactive simulation instead than basic procedural training.
In addition Virtual Engine will improve performance over a wide range of
hardware platform improving immersive 3D environment realism
Emotional Engagement will become more challenging due to the evolution
of SG and the use over new conditions due to the mobile solution
Game Generation will emerge to have decision making position; this will
introduce people in the process with positive attitude versus SG and
experience in this framework; this will provide opportunities as well as
more challenging requirements for developers
In crisis management realism and fidelity will become more and more
demanding for SG respect actual requirements due to the more intense
use on training and education; this will require to introduce also in SG as in
Simulation effective Verification and Validation (V&V) issues and to
improve physical engines and model fidelity.
Necessity to move SG from basic procedural training to real dynamic
interactive simulation.
Necessity to include realistic behavioural models
User evolution will increase also the 3d immersive requirement up to
performance on light platforms (i.e., mobile) that are over passing actual
computational capabilities.
New uses modes and general evolution (i.e. users, computer games,
device) will require to reinforce emotional engagement
Necessity to a stronger cooperation among Simulation and SG
Communities on V&V, fidelity and modelling
Human Behaviour Models (HBM)
New Software Engine and Hardware
Transcultural Approach in SG design addressing user engagement and
problem framing will be fundamental for future developments [32].
Intelligent Agents will be useful to improve HBM and make more realistic
SG representing complex scenarios
Topic
Vision 2020
Interoperability and standardization
Interoperability will become a fundamental requirement to integrate SG
with among each other, with other systems and real equipment; High level
Architecture is currently the SoA in the area, while SGs have pretty limited
capability to interact in this world, the delay on Interoperability Topic is
due to immaturity of the developer world and in the weak connection
between SG developers and final users.
Crisis Management is stressing interoperability issues and it will become
very important an evolution in this sector.
Existing Standards will be subjected to review and restoring; this already
happen in 2010 with evolved HLA and it will happen again; therefore SGs
will need to move from standalone games to interoperable tools.
Gap: SoA - Vision
SGs have traditionally avoided the problem; the result is a balcanization of
SG engines with pretty good performance from graphic point of view and
very terrible results in term of fidelity.
Interoperability was not addressed with few exceptions until few years
ago; limited customer solutions were available just among the same
games and/or some basic hardware; there is a slight evolution in this area
from point of view of hardware integration, while interoperability among
different models and SG is still a major gap.
Research
Steps
Need
and Interoperability Standards
New SG Architecture
Possible
related
topics/trends/enablers
(indicate
conditions
and relationships)
SG developers evolution through experience in working on specific project
of the applicative domain (i.e. security, crisis Management)
A adoption of standards will lead to overpass several of these issues as
happen with (i.e. ST_VP, ST_VM and S4PT) [23][33].
The major driver from this point of view will be user requirements for
collective training and hardware platform evolution and their
interoperability (i.e. mobile technologies).
Due to the SG market growth and experience evolution this connection
will be drastically reinforced and the interoperability and standardization
will be addressed in next years; therefore this topic is still pretty
challenging technologically and in term of conceptual modelling [31];
Standardization will help in this direction.
Topic
Vision 2020
Gap: SoA - Vision
Research
Steps
Need
and
Possible
related
topics/trends/enablers
(indicate
conditions
and relationships)
Multiple player mode
Collective and Cooperative training will evolve even further in new context
and it will become common in education and training. In crisis
management this is a major requirement due to the cooperative
environment of this application area where many different actors interact
versus an aggressive agent (i.e. fire, crisis evolution etc)
Coordination and Harmony in team building it will become a requirement.
Users with attitude to interact in social networks will expect completely
new multiplayer approach emphasising mobility.
Deployable solution will become more popular and Simulation as a Service
(SaaS) will become a popular paradigm.
Most of existing SG could be divided between good graphic performance
and web games; due to engine evolution this gap will be more a more
reduced automatically.
Therefore a major challenge is to address Man Machine Interface (MMI)
for managing collective training: interaction among people in crisis
management are not easy to be covered by SG and even more in future
with more mobile solution that will need to use different lean MMI
SaaS is currently not really addressed by SG in crisis management and
need to be fully developed as is happening with new interesting initiatives
(i.e. DVx2) [33].
Collective Training
Simulation Standards
Intelligent Agents
MMI
Also in this case Transcultural Approach will be required to overpass these
current SG gaps connecting users and subject matter experts with SG
community [35]
Web Multiplayer Games will evolve to cover these aspects being more
interactive and common also in SG [21]
SaaS will require strong evolution in term of standards and capabilities in
order to address future requirements
Intelligent Agents will become sparring partners in future SG interactive in
multiplayer games and providing support for reproducing large complex
scenario with limited training audience
Topic
Vision 2020
Gap: SoA - Vision
Research
Steps
Need
and
Possible
related
topics/trends/enablers
(indicate
conditions
and relationships)
Feedback & Performance Assessment
New use paradigm will introduce concepts such as Crowdsourcing and
Self-Training in Crisis Management
In addition also traditional training sessions will require to evaluate
collective training among multiplayers
Self Tutoring based on Intelligent Agents and Virtual Humans will become
popular
Currently SG address to this topic is pretty limited and partial; it will be
required to develop a wide spectrum of methodologies and techniques to
support development of these capabilities
Intelligent Agents
Virtual Humans
Crowdsourcing Solutions
User Capability and Potential Evaluation
Team Building Metrics
A major requirement will be due to Intelligent Agents able to drive tutor
and partner in complex scenarios [24][33]
In this case Transcultural Approach will be important connecting SG
developers, modelling experts and educational and psychological
communities as is happening in developing new performance assessments
in crisis management for anti-terrorism with new games [29]
References
[1]
[2]
[3]
[4]
[5]
[6]
[7]
[8]
[9]
[10]
[11]
[12]
[13]
[14]
[15]
[16]
[17]
[18]
[19]
[20]
[21]
[22]
[23]
[24]
Tesei A., Barbieri A. and Kessel R., (2012)“Survey on Serious Games applied to security, safety
and crisis management: User requirements, present solutions and envisaged possible
improvements,” in Procs. of VS-Games2012 Int. Conf., Genoa, Oct. 2012.
http://adlunap.ro
www.vbsworlds.com
Kessel R. HPT2E Contextual Read Ahead. NURC Report 2012.
www.siqur.com
www.rescuesim.com
www.trainingfordisastermanagement.com
Caspian Learning, “Serious games in defence education,” UK MoD Report, Crown Copyright,
2008.
Harteveld C., R. Guimarães, I.S. Mayer, and R. Bidarra, “Balancing Play, Meaning and Reality:
The Design Philosophy of LEVEE PATROLLER,” Simulation & Gaming, 41(3) 316–340, 2010.
Harteveld C., “Making Sense of Virtual Risks. A Quasi-Experimental Investigation into GameBased Training,” PhD Thesis at TUD, Delft, Dec. 2012.
Ştefan A., I. Roceanu, D. Beligan, A. Tesei, A. Barbieri, “Standards in practice: a serious game
perspective,” in Procs. of the 9th Scientific Conf. eLearning and Software Education,
Bucharect, April 20013 (in press)
Palfrey J., U. Gasser, “Interop: The Promise and Perils of Highly Interconnected Systems.” New
York, Basic Books, 2012.
Stănescu I.A., Roceanu I., A. Ştefan, I. Martinez-Ortiz, “Principles of Serious Games
Interoperability,” in Procs. of the 6th International Conference on Virtual Learning, 2011.
Carron T., J.-C. Marty, J.-M. Heraud, ”Teaching with game-based learning management
systems: Exploring a pedagogical dungeon,” Simulation & Gaming, Vol. 39, pp. 353-378, 2008.
Moreno P. -Ger, D. Burgos, J. Torrente, ”Digital Games in eLearning Environments: Current
Uses and Emerging Trends,” Simulation & Gaming, Vol. 40, pp. 669-687, 2009.
http://l4s.fvaweb.eu
Crookall D., “Serious Games, Debriefing, and Simulation/Gaming as a Discipline,” Simulation &
Gaming, 41(6) 898 –920, 2010.
Christoff D., G. Eves, “Considerations for Serious Games use in safety culture improvement in
the resources industry,” Procs. of SimTecT Conference, 2008.
Roceanu I., D. Beligan, A. Tesei, “Measureing learning impact of serious games in military
training,” accepted for ITEC2013 Int. Conf., The International Forum for the Military Education,
Training and Simulation Sectors, Rome, May 2013.
Strode C., M. Oddone and M. Leonard, “Development of a real-time multistatic ASW
simulation facility at the NATO Undersea Research Centre,” NURC Internal Report NURC-FR2011-005, Sept. 2011.
Bruzzone A.G., Massei M., Tremori A. (2009) "Serious Games for Training and Education on
Defense against Terrorism" - NATO MSG-069 Symposium Use of M&S in: Support to
Operations, Irregular Warfare, Defence Against Terrorism and Coalition Tactical Force
Integration", Brussels, Belgium October 15, 16
Bruzzone A.G., Massei M., Solis A., Poggi S., Bartolucci C., Capponi L.D. (2013) "Serious Games
as enablers for Training and Education on Operations over Off-Shore Platforms and Ships",
Proceedings of Summersim2013, Toronto, July
Massei M., Tremori A. (2011) "Mobile Training Solutions Based on St_VP: A HLA Virtual
Simulation for Traning And Virtual Prototyping within Ports", Proceedings of SCM MEMTS
WAMS, IMDS, Sankt-Petersburg, Russia, June
Benjamins T.; Rothkrantz L. J. M., (2006), " MACSIM: Serious Gaming in Crisis Management via
Script-based Simulation", Cgames'2006: Proceedings Of The 9th International Conference On
Computer Games: Artificial Intelligence And Mobile Systems Pages: 225-232 Published:
2006.
[25] Crichton M. Flin R., (2001),”Training for emergency management: tactical decision games”,
Journal of Hazardous Materials, Volume 88, Issues 2–3, 14 December 2001, Pages 255-266
[26] Warren E. W., Jordan G. Stuart A., (2011), “Training and learning for crisis management using a
virtual simulation/gaming environment”, Cognition, Technology & Work Volume 13, Number 3
(2011), 163-173, DOI: 10.1007/s10111-011-0176-5
[27] Bruzzone A.G., Buck W,. Longo F., Sokolowski J., and Sottilare R. (Eds). Proceedings of DHSS:
The International Defense and Homeland Security Simulation Workshop 2012, Vienna, Austria,
September 19-21, DIPTEM Universita’ di Genova, 6-12.
[28] Raybourn, E.M. 2012. Beyond serious games: Transmedia for more effective training &
education.
[29] Tremori A., Baisini C., Enkvist T., Bruzzone A.G., Nyce J. M. (2012), "Intelligent Agents and
Serious Games for the development of Contextual Sensitivity", Proceedings of AHFE 2012, San
Francisco, US, July
[30] Bruzzone A.G., Tremori A., Cunha G. (2008) “Intelligence and Security as Framework for
Applying Serious Games”, Proceedings of SeriGamex 2008, November, Rome
[31] Fancello G., Tremori A. (2010) “Virtual Interoperable Port for Security & Safety Assessment”,
Proceedings of Sim & Sea, La Spezia, June
[32] Caroll J., Cameron D. (2009) "Learning to Respond: a crisis Management Simulation", Plenary
Speech at SIREN Workshop, Genoa, September
[33] Bruzzone A.G., Massei M., Tesei A., Poggi S., Agresta M., Bartolucci C. (2013) “MOIRE Virtual
Environment for Collective Training on Marine Domain”, NATO MSG-130, La Spezia, October
[34] Bruzzone A.G., Bocca E., Poggi S., Tremori A. (2012) “Strategic Decision Making Training
Through Serious Games”, Simulation Team Report for NATO DC, Genoa, June
[35] Bruzzone A.G., Cunha G., Elfrey P., Tremori A. (2009) "Simulation for education in resource
management in homeland security", Proceedings of SCSC, Istabul, July
[36] Crichton M., Flin R. (2001) "Training for Emergency Management: Tactical Decision Games",
Journal of Hazardous Materials, 88(2-3), 255-266
[37] www.icons.umd.edu
[38] Nigrelli M.R (2008) "Disaster READiness through EDucation: PR", EU Report, Rome, December
[39] www.gamesforchange.org/play/darfur-is-dying/
[40] http://playgen.com/play/floodsim
[41] www.wfp.org/videos/food-force-promo
[42] www.trainingfordisastermanagement.com/
[43] www.breakawaygames.com/serious-games/solutions/homeland/
[44] Bruzzone A.G., Massei M., Poggi S., Nicoletti L., Bartolucci C., Franzinetti G. (2013) “Serious
Games for Multi-user Crew Collaborative Procedural Training and Education”, Proceedings of
I/ITSEC2013, Orlando, December