Gala Roadmap Contribution
Name of lead contributor: Michela Mortara
SIG: 3.5 – Humanities&Heritage
Note: here I assumed 1=low importance, 5=high importance!
The reviewers have said that the roadmap should make clear the connections between the state-ofthe-art analysis and literature reviews to the vision and the goals of the project. They said the
roadmap should indicate the highest priority items, those which are key enablers but not top
priority, and those which could be dropped if resources are inadequate and the Commission agrees
to focus only on the most important items in the roadmap. To do this, proceed:
1) Identify the state of the art (SoA) for your SIG
List the main topics in your SG area and describe the SoA/ current status of SGs (giving sound
explanations, backed by appropriate literature references) in the Table. You can optionally identify
any gaps in the SoA – these may form the basis for research needs/challenges. The information can
be prepared by updating and properly structuring the text in your annual reports.
Topic
3D Acquisition of culturally significant artefacts
There is a substantial amount of interest from people working on the
digitisation of cultural heritage. An example of an important project in this
domain is Europeana, which aims to create a pan-European database of
digitised cultural heritage. Such projects present an interesting challenge, as
they allow other applications to tap into cultural heritage data and expose the
data in different ways.
We believe that there is much potential in integrating digital cultural heritage
data with serious games. This would valorise repositories of digitalised cultural
heritage data, of which it is likely that the numbers will increase in the future,
as more and more individual museums are working to open up their collections
digitally.
Relevance/priority 3
(on a 1-5 scale)
SoA/ SG Current
Status description Especially in archaeological, artistic and historical reconstructions, it may be
(with references)
required that some artefacts, buildings or sites to be faithfully represented in
the game environment. To this aim the real data of the subject have to be
acquired (3D scanning) to reconstruct its digital counterpart, rather than
manually design a model which only resembles the appearance of the real one
(details can be found in [CS08]).
Three-dimensional scanning devices are usually based on optical technology
(laser or structured light) and can be divided into active and passive devices.
The more frequently used in CH digitization are active optical devices. These
systems shoot some sort of light over the surface of the artefact and
reconstruct its geometry by checking how the light is reflected by the surface.
For small and medium scale objects, the location in 3D space of each point of
reflection is computed by geometric triangulation, known the light pattern
produced by the emitter (source and direction of propagation.
For large scale objects e.g., architectures, the time-of-flight approach is
appropriate. These systems can acquire regions ranging from a few meters to a
few hundred meters. The system works computing the time elapsed between
the emission of a point wise laser beam and the detection of the return beam
reflected by the surface. Passive optical acquisition methods are a cheaper but
lower-quality alternative, performing 3D reconstruction from a simple
sequence of high resolution digital photos of the artefact. Some of them have
been considerably improved, and show some interesting potential for a wider
diffusion. These approaches are based on the search of a small set of
correspondences (feature points seen from different viewpoints) between the
captured images, from which the 3D position of these points and the camera
parameters can be recovered. Starting from these few sparse points, a dense
depth-range map can be reconstructed from each image by interpolation and
applying stereo-matching techniques. The ARC 3D Web service
(http://www.arc3d.be/) developed within the Epoch Network of Excellence
(see related projects) allows users to upload their pictures and get the
corresponding aligned range maps.
Beside shape acquisition, a very important aspect in CH applications is colour
sampling. The majority of scanners implement low-quality colour sampling,
and only the apparent colour of the surface is taken into account, neglecting its
reflectance properties (which constitute the characterizing aspect of the
surface appearance). Much of research is devoted to new acquisition set-ups
and algorithms for capturing the colour properties with high-fidelity and
mapping it to the model surface [DCCCS10].
Requirements
The reviewed games in the field have shown that two major game genres have
been adopted into the field of Humanities and Heritage: puzzle/quiz games and
adventure/exploration games. Apart from standard puzzle games (e.g. National
Geographic Challenge, My Culture Quest) for which the 3D setting is
unnecessary, many of the recent adventure games are set in 3D. One
motivation for this point is to support situated cognition by offering a proper
environment/context where the learning process can take place [VE06]. The
player has to interact with objects in the environment to advance in the game
and this mechanism triggers motivation and active construction of knowledge
[FAGM11]. 3D Virtual Worlds make the experience more interesting and
compelling. However, it becomes crucial to attract the player attention on
objects with a role in the game: a well-designed environment will lead the
player trough the highlights letting him think he found them himself [CMAK11].
Believability is more important than realism: apart from the huge
computational load required, an excess of realism could be counterproductive,
while a credible environment is sufficient to guarantee the feeling of
immersion and generate a willing suspension of disbelief. Nonetheless, the
historical data must be accurate, and the main cultural artefacts must be
faithfully represented (e.g. in historical reconstructions). In virtual museum
applications, it might be desirable that the real objects are acquired to retain
the finest details.
Opportunities
creating an application interfacing with Europeana
SoA Gaps (if any)
The advantages of image-based reconstruction are quite evident. The only
hardware required is a simple good quality digital photographic camera, and
the scanning process requires just taking a reasonably large number of photos
all around the object. On the other hand, this approach still exhibits a lower
geometric precision than the assessed laser-based 3D scanning technologies.
From the point of view of robustness, speed and accuracy, the status of the
commercial scanning devices can be considered mature for most applications.
The major problem most often raised by practitioners in the CH field is their
excessive cost, especially when compared with the low budget which
characterizes most CH-related activities. Moreover, even if the acquisition
pipeline by laser scanning is currently semi-automatic, a lot of manual
intervention by experts is still needed to produce high quality models.
(Please duplicate this table for further topics in your SIG – we expect typically 5 to 10 tables per SIG)
Topic
Immersion with pervasive games
In pervasive games, which take place in real world environments, obtaining a
good immersion effect can be hard to achieve
Relevance/priority 3
(on a 1-5 scale)
SoA/ SG Current Lack of immersion may not be bad for pervasive serious games in museums.
Status description Brown & Cairns (2004) introduced three different levels of immersion:
(with references)
engagement, engrossment and total immersion. Engagement is about
overcoming game preference and willing to invest time, effort and attention.
Engrossment occurs when the gamer’s emotions are affected and the controls
become ‘invisible’. Finally, in a state of total immersion, the player experiences
a sense of presence, of being ‘in the game’.
Requirements
immersion in a museum app would derive the attention of the visitor from the
environment in which he is progressing, i.e. the museum, which is not what is
generally aimed for when designing a pervasive serious game. A balance
between immersion engagement, engrossment and concentration on the
museum environment would seem to be the sweet spot for which to aim in
pervasive serious games [C2013].
Opportunities
SoA Gaps (if any)
Immersion and entertainment seem to be very much related. This would leads
us to suggest that when designing pervasive serious games for cultural
heritage, it would be wise to focus on making sure that the games are doing
well in terms of immersion, in order to support learning. This may be a
challenge, especially for pervasive serious games. Indeed, the environment in
which the pervasive game is played is not controlled by the player, which can
lead to elements in the environment disturbing the player and breaking the
sense of immersion. In a busy museum, this can for example be the case when
multiple people are standing before an art work, obstructing the player’s
interaction with the exhibit.
Topic
Relevance/priority
(on a 1-5 scale)
SoA/ SG Current
Status description
(with references)
Smart management of digital cultural content
4
Virtual presentation of the Cultural Heritage is envisioned as one of the
emerging applications to largely benefit from the Web 2.0 paradigm of the
Internet use. As proved by the MOSAICA(www.mosaica-project.eu) project,
coupling cultural content with semantic web technologies not only allows users
to collaboratively collect, annotate and organize the content of interest, but
also to use this knowledge and apply their creativity for the benefit of all, thus
empowering communities to continue the Cultural Heritage into the digital
age.
Requirements
Opportunities
Indeed, web browsers are getting ready to bring the interactive 3D content
from the internet to our PCs. WebGL (http://www.khronos.org/webgl/) is a
web standard which does not need additional plug-ins and is currently starting
to be supported natively by web browsers. Based on WebGL, X3DOM
(http://www.x3dom.org) and XML3D (http://www.xml3d.org) aim to support
fully interactive 3D content as an integral part of mixed 2D/3D documents,
instead of adapting existing graphics technologies to the web. The declarative
approach they adopt allows 3D content to be easily processed by other
applications using common XML-based tools including web indexing,
transformations, and annotations. This strategy appears the most promising
one to seamlessly integrate 3D into the Semantic Web.
SoA Gaps (if any)
While the semantic web is able to manage text, images and videos, much has
still to be done with respect to the smart management of 3D content.
Topic
HCI for personalized CH experience
In-museum games have to cope with the waiting time of players not to create
queues or equivalently with the number of players who can be involved at the
same time. Usually exhibitions address a large number of players
contemporarily. A challenge here is to move from a mass-oriented approach
towards a personalized experience even within an application targeting a wide
audience contemporarily. The use of proper interaction devices may strongly
help in these directions.
Relevance/priority 5
(on a 1-5 scale)
SoA/ SG Current New HCI technologies offer the video gamer many ways, devices and sensors
Status description to feel immersed and a sheer number of interaction possibilities, replacing the
(with references)
traditional mouse/keyboard combination.
The use of Natural User Interfaces (NUI) has widely spread for the last decade
(see the GALA Workpackage 2 Deliverable D2.1 T2.4 Human-Computer
Interaction, Research Field Report, for a very detailed state of the art). Even if
Serious Games design goals are different from entertainment and leisure
games, they have taken benefit from these technological advances. Moreover,
concerning the cultural heritage field, it becomes even more meaningful: the
sensitive issue is to move from a current mass-oriented access to museums,
art galleries, historic places, and so on, to an approach centred at the
individual. This can come true with NUI, with which the player is taken into
account according to its specificity (interests, physiological, behavioural
and social features), and with which he will be the leader of his explorations
and initiatives.
Thus, two main fields that are relevant and efficient for cultural heritage
Serious Games have been identified: the Virtual Reality and Augmented
Reality. Therefore the corresponding HCI are often used in museum or
archaeological environments.
A non-expensive system named MARVINS (Mobile Augmented Reality Visible
Navigation System) is a mobile interactive interface between the user and the
Internet or other information access points. Head-mounted displays (e.g. Sony
Glasstron) interfaced with MARVINS provide the user with an AR experience
where he/she can selectively display any desired information allocated at some
remote server (e.g. a museum server). A tracking system is used to provide
orientation and navigation possibilities. The system renders the eye-screen
opaque for viewing displayed information, as well as transparent for viewing
with the naked eye, which can be adjusted accordingly.
Specific display techniques, like stereographic and multiscreen displays are
required in planetariums, art centres and domes, which enable an immersive
360 degree-vision of 3D worlds. The dome of the Society of Art of Montréal is
an example of such complex systems (http://www.sat.qc.ca/). Three different
set-ups are employed there: the Panoscope 360°, the Panodome and the
Cyclorama (see figure below). The advantage of these systems is that they
considerably simplify the production and presentation of panoramic and
immersive content.
The real-time VR Dome Theatre of the Foundation of Hellenic World located in
Athens is operated by 6 pairs of seamlessly blended projectors, each pair
providing a stereo image for one of the six tiles in which an hemispherical
surface of 13 Mt. in diameter is ideally subdivided [CAO09]. Each projector is
powered by one dedicated PC in a cluster providing parallel execution.
Interaction with the audience is possible through 2-axis joystick with analogue
values and 4 buttons with discrete values for each of the 132 seats of the
theatre.
Tangible User Interfaces (TUI) like touchscreen tablets and tables, have
become ubiquitous and therefore have also reached the gaming world and
provide RV and AR applications. Thanks to the touchscreen technologies which
are more intuitive, gameplay style has improved and enables players to live
experiences that would never be possible to create with former game input
devices like joysticks and buttons. For instance, the “Dessine-moi un
Mammouth” (Let’s draw a Mammoth!) game designed for the Gargas caves in
the French Pyrénées (http://www.numerigrottes-pyrenees.fr/p-parcoursnestploria_fr.htm) is an interactive AR application developed by [DCC07] and
studied in [DAR09]. It provides the gamer a funny experience of line drawings,
depicting prehistoric animals, directly onto a tracking board where real high
resolution photographs of the prehistoric engravements are displayed. The
same task made with a single mouse would have been far much painful.
Requirements
Opportunities
What will the future hold for the NUI? The answer will surely be found in the
field of Brain Computer Interfaces (BCI). Other new interaction devices could
include virtual retinal displays and active contact lens [BT05].
SoA Gaps (if any)
For playful interactive exhibitions and games, NUIs include also the Gesture
Based User Interface which can track an entire 3D body location, the head,
hand and eyes postures. The Microsoft Kinect is such an interactive system,
working with structured light depth cameras. Further research is going on to
develop new techniques for accurately tracking people and recognizing their
poses, even when they are occluding each other. No special markers on the
bodies are needed to track the people. Although current computer vision
research has achieved promising results in interpreting the pose and gestures
of a single person given multiple video sequences, interpreting the poses of
multiple people in a relatively dense group is still an open problem. The key
difficulties are the inter-person occlusion and limb ambiguities which hamper
the interpretation. At the moment there are only few interactive cultural
heritage games [AMLPPF10].
Mobile technologies for e-tourism, museum tours, and games like TidyCity,
need ways for interaction with mobile handheld devices using computer vision
technology, especially in context with augmented reality. That is, applications
where users look at the live image of the video camera on their mobile phone
and the scene that they see (i.e. the reality) is enriched by integrated 3dimensional virtual objects (i.e. an augmented reality). Even if virtual objects
are registered in 3D and updated in real-time, current interaction concepts are
often limited to pure 2D pointing and clicking on the device’s screen. However,
in order to explore the tremendous potential of this approach for areas such as
cultural heritage, entertainment, or tourism, users need to create, access,
modify, and annotate virtual objects and their relations to the real world by
manipulations in 3D. Relevant aspects are (1) how to track the state of a user in
3D on various levels (e.g. face, eyes, gaze using the front-facing camera, and
hand, fingers, gestures using the user-facing camera) in real-time in order to
create a more natural, intuitive, and powerful interaction; (2) how to analyse
the mobile phone’s video stream in order to accurately register virtual objects
with the real world, to prevent occlusion problems between, for example, the
user’s hand, real and virtual objects, and to enable seamless interaction with
real as well as virtual objects; (3) how can we use this information about the
environment and the state of the user in order to enable intuitive and
interactive creation, placement, and manipulation of the augmented reality in
the real, 3-dimensional world.
Topic
Tourism games
-develop games linked to existing geographic data and modelsRelevance/priority 3
(on a 1-5 scale)
SoA/ SG Current Most of the free Google Earth games are based on the geographic power that
Status description Google Earth brings to browsers and desktops: geography quizzes, flight and
(with references)
ship simulators, races, and commercial games like Grand Theft Auto.
Requirements
Opportunities
A big opportunity in tourism applications is the integration with Google Earth,
already the playing arena for lots of games.
SoA Gaps (if any)
We are aware of no serious game exploiting this detailed virtual version of the
real Earth for tourism and cultural applications.
Topic
Engagement in augmented visits with Location gaming
Relevance/priority 3
(on a 1-5 scale)
SoA/ SG Current
Status description The increased ubiquity of mobile phones has raised interest in how mobile
(with references)
devices and multimodal interfaces can be used to access shared spaces
populated by digital artefacts, while remaining grounded in the real world;
players are able to reference cultural artefacts in situ, allowing for the
superimposing of design elements onto images or objects in real world
locations. An example is the game A la recherche de l'empreinte perdue1
(seeking the lost footprint): equipped with a logbook and a smartphone,
1
http://www.oust-broceliande-vacances.com/broceliande-actualites/676-a-la-recherche-de-lempreinteperdue.html
players visit the Landes Lanvaux in Saint-Jean-Brévelay et Locminé looking for
clues with the help of residents. The journey is punctuated by 26 steps puzzles
leading the visitor to learn history and past events of the location, architectural
elements, anecdotes, recipes, and more. As an AR application, the player can
augment the experience superimposing ancient buildings over the real current
location.
Games on mobile devices have a great potential to engage museum visitors.
One popular type of game in this sense is “location-gaming.” The mechanics is
that players go places, do fast, simple tasks (like typing something into their
phone, or uploading a photo of something), and win a reward (the reward can
virtual “points,” or something tangible, like a free postcard or sandwich).
The leading platform for location-gaming is SCVNGR2, which recently reached
over 1 million users and has been funded with $15 million from Google and
other venture funds. SCVNGR is a free application for iPhone and Android that
uses the GPS in the smart phone to verify a user's location in the real world.
Requirements
Opportunities
Kellian Adams, the company’s museum education technologist said that
SCVNGR can work for museums and especially has potential for history,
connecting the past with modern locations within a quest that takes visitors to
various places. It also work well for any case where visitors have distances to
walk, such as in a zoo or botanical garden. Indeed, many cultural institutions
have already joined SCVNGR, among which the National Zoo, the Smithsonian,
the Children’s Museum of Indianapolis, the San Diego Zoo, the Minneapolis
Institute of Art, the Muhammed Ali Center, The Chicago Institute of Art, the
Museum of London.
SoA Gaps (if any)
At this stage, we found no SGs in Europe based on SCVNGR
Topic
Adoption of SGs in formal education
Especially in games for formal education, e.g. to be played at school the
adaptation of the difficulty level to the specific student is crucial to support the
teacher using learning games in the classroom.
Relevance/priority 5
(on a 1-5 scale)
SoA/ SG Current We found very few SGs in H&H that are used in formal education. One example
Status description is Playing History by Serious Games Interactive which represents a best
(with references)
practice from this point of view, because it is based on pedagogically sound
principles, follows the national curricula and provides support material for
teachers and students.
None of the reviewed games in the SIG has so far any adaptation mechanism,
except for a prototype developed by the 80DAYs project. At maximum, SGs
designed for use in school provide support material in the form of game
walkthrough, insights about the educational contents, suggestions about how
2
http://www.scvngr.com/
to structure the game sessions, test sheets, and a teacher portal to manage the
pupils’ work.
Requirements
The requirements for SGs to be used in formal education are
-compliance with established curricula
-adaptation to user profile
-flexibility of the game wrt topic and learning style
-multiplayer facilities (learning collaboratively or make competition)
-evaluation of learning in order to assess student performance (and for
adaptation)
Opportunities
SoA Gaps (if any)
-adaptation mechanisms to adapt difficulty level to the learner capabilities
-evaluation of the learner performance both for adaptation and for assessing
student performance
-evaluation of the learning impact of SGs per se, instead of or coupled with
traditional means
2) Vision and Gap
For each identified topic in your SIG develop a vision for the year 2020. Then determine the gap
between the state of the art and the vision. Identify the research needs and the steps (also including
tools, methods, etc.) to achieve the vision. You may complement the table with text drawn from
your annual reports.
Topic
3D Acquisition of culturally significant artefacts
Vision 2020
It will be very easy and quick to acquire high resolution 3D models of
significant artefacts. Museum operators will be able to do this operation
by themselves, without the need of external personnel and costly
operations
which
also
would
require
to
move
fragile
artistic/archaeological artefacts from the museum. No manual postprocessing will be necessary, and the museum operators could directly
upload the new content in a shared online repository for digital
preservation of the artefacts and large-scale access and fruition, including
SGs that use such models to engage people in a virtual/ augmented
museum visit.
Gap: SoA - Vision
At present, the outcome of 3D laser scanners still needs manual
intervention to achieve a polished surface mesh (technically speaking a
closed and manifold surface) that downstream applications can handle.
On the other side, image-based reconstruction is cheaper and easier but
yields low resolution results.
Research
Steps
Need
and New geometric tools are needed that use semantics, i.e. a formal
representation of prior knowledge about the object being scanned, to
optimize the reconstruction of the digital model. Such a priori knowledge
about the context should help in planning the steps of the scanning
session (i.e. the number, sequence and viewpoints of scans) and
overcome problems or correct errors such as holes in the digital model in
concave or reflective areas of the object.
Possible
related
topics/trends/enablers
(indicate
conditions
and relationships)
A minimum understanding of the process by the museum expert would be
always an enabler. The creation of new interdisciplinary curricula, e.g. in
digital humanities (this does not exist in Italy for example: it is left to the
students to achieve interdisciplinary expertise e.g. with master and PhD in
different sectors.)
Topic
Immersion with pervasive games
Vision 2020
It will be possible to immerse in an augmented experience everywhere
while remaining conscious about the reality around us.
Gap: SoA - Vision
Mobile AR systems like the MARVIN described above are still too invasive
so that the player still feels the presence of an interface between the real
and virtual reality. The total feeling of immersion is then difficult to
achieve. On the other hand, huge head-mounted displays also represent a
barrier to the feeling of the surrounding real space.
Conversely, such link to reality is maintained by mobile apps, but at the
expenses of very low immersion.
Research
Steps
Need
and Research efforts are needed in two directions: one on the HCI side, to
develop new lightweight AR devices. The other is more general and deals
with the design of a new paradigm and new tools to achieve immersion
through mobile devices like smartphones and tablets.
Possible
related Possibly related to psychology and BCI
topics/trends/enablers
(indicate
conditions
and relationships)
Topic
Vision 2020
Smart management of digital cultural content
It will be possible to manage in a smart and automatic way the digital cultural
content, for instance for using artefacts from a digital museum repository in a
game, or for increasing the level of natural interactions between characters and
objects inside a game, or for accessing related multimedia information while
remaining inside the game environment.
As an example, we can consider a virtual tourism game where the player faces
the 3D model of the main statue of the ‘‘fontana del Nettuno’’in Bologna.
A proper documentation of the geometric model would associate
the object with related multimedia sources, such as photos and videos of the
fountain, images of similar statues, etc. At component level it would allow the
user to appraise other images, videos or3D models sharing the same
decoration: the trident. See figure below:
Gap: SoA - Vision
This challenge is related to the
documentation of the lifecycle of 3D models, which suggests
interdisciplinary issues in connection with the semantic web
community. In fact, efficiently sharing, searching, and reusing
3D data also in networked environments has been proven to be
fundamental, together with the preservation of the annotation in
the digital shape workflow.
The annotation of a 3D object is central to the documentation. By 3D
annotation we mean the process by which a text-based piece of information is
linked/
associated to the object and its parts and stored for subsequent
uses. We speak about semantic annotation because (or when) the
text associated is meaningful in some context, and used for
understanding and storing information about the object, which
is not explicit or not contained in the geometric data.
Annotation may be related to the process/workflow itself that
was used to create or process the object and/or its parts : in [14]
two examples of how to document the 3D lifecycle are shown in
the contexts of product design and the acquisition and processing
of virtual human bodies. There are a number of open issues that
should be addressed to realise an ideal cataloguing and documentation of the lifecycle of 3D objects, among which coding of
the data provenance and version control, effective metadata
structures, interoperability, and object and part-based annotation.
In the general framework of multimedia, these themes are on the
research agenda of many initiatives(e.g., the recent [15] on data
provenance).
In the cultural heritage field, we may mention digital rights
management, updating and long-term preservation of 3D models
among the technical research challenges that should be addressed
to realise an ideal operational 3D digital archive [16]. However, a
big effort has still to be made on the documentation process itself:
currently available metadata for cultural artefacts are general
(e.g., ownership and copyright) and not referred to the geometry.
Research
and Steps
Need In this perspective, the complete documentation of the 3D data
and their acquisition and processing workflow appears crucial; on
the one hand, non-geometrical features, such as material, colour,
and preservation conditions, and environmental information (e.g.,
the exact location of a fountain in a square, the specific device and
the methodology with whom it has been acquired) should be
encoded into the models in order to enrich the cultural and
scientific value of the digital content, while guaranteeing the data
traceability and provenance; on the other hand, the annotation
process should be supported by smart tools that automatically
compute and encode geometrical properties (e.g., the size, the
kind of shape, the average diameter of a column) and structural
properties (e.g.,the number of handles of an amphora; the
number of columns of the façade of a temple).
Another aspect to take into consideration is that annotation should be able to
integrate different resources related to the same artifacts, i.e.,
text, 2D and 3D,and support both a semantic-based and geometry-based
classification.
In parallel, a standardisation issue on how to create and manage very large
digital repositories arises, which should involve the different countries . A first
attempt has been done by
Europeana [27], the European Digital Library, where the 3D
content initially lacked of a specific management. But the importance of such an issue became clear soon and the specific action
CARARE [28] has been started to enable access to 3D and
virtual reality content through Europeana.
Possible related Interoperability and semantics
topics/trends/ena
blers
(indicate
conditions
and
relationships)
Topic
HCI for personalized CH experience
Vision 2020
It will be possible to offer a personalized and unique experience to each
one of the many museum visitors contemporarily. The museum
environment has space and time limitations, and we can imagine of
overcoming such constraints by finding new ways of personalizing a CH
experience provided to a group of people contemporarily.
Gap: SoA - Vision
Up to know we are facing a trade-off between the level of interaction and
personalization and the number of visitors which can be involved at the
same time, varying from a massive experience which allows little
interaction and no personalization (e.g. in a dome) to a single-user
application.
Research
Steps
Need
and
User modelling, psychology and HCI
The Mixed realities (Virtual and Augmented) can be a compromise
between mass oriented exhibitions and personalized experience such as
the web 2.0 paradigm.
If VR applications particularly fit the requirements of huge exhibitions
which can be achieved inside domes and planetariums, AR ones are far
more adapted to a greatest number of museums purposes: AR is portable,
not expensive, encourages social collaboration and doesn’t break the
player from its reality.
AR future Serious Games will have to be designed, taking into account
human factors and ergonomics criteria: for instance, it could deal with
efficiency related to the weight of smartphones or tablets and what occurs
after an extended use of such devices.
Furthermore, AR SG design will have to focus on technological multi-modal
displays which can lead to the extra issue about getting the player
attention so that it can learn during his/her augmented visit and
distracting him/her from the physical visit.
To address the shared time and space issue which has been previously
mentioned, geolocalisation with GPS or QR codes can be used, combined
with MMORPG techniques. For instance, the Google Ingress android sci-fi
game (see figure below), by invitation only enables the player to use
his/her phone to interact with the game and at the same time with the
real world around him/her. To win, the player has also to collaborate with
hits/her real fellow neighbors and fight against real enemy players.
Ingress can almost be regarded like a AR serious game for CH because it
forces the player to go to museums and stand in front of sculptures or
historical buildings in order to improve his/her score.
Possible
related Immersion with pervasive games
topics/trends/enablers Smart management of digital content
(indicate
conditions
and relationships)
Topic
Vision 2020
Tourism games
A set of virtual tourism games linked to Google Earth will be available to
engage player in virtual visits of many places in the world, to learn
geography, architectural and natural heritage and raise cultural awareness
(including foreign languages, etiquette, beliefs, rituals, etc)
The social aspect may be included, including collaboration/competition
mechanisms between virtual and real visitors of the same locations (link to
pervasive gaming and location gaming)
In this vision, Google Earth (or another future system) will act as the
collector of different applications based on the geometric/geographic
model.
Gap: SoA - Vision
I think the gap is in the interoperability and in the business model:
different teams should share their own apps without charge for the player.
Ideally Google could sponsor this vision, provided there is a return of
investment (and in this case would be more on the engagement and fun
side rather than the educational one).
Municipalities, museums, places of interest, etc. could also sponsor these
apps.
Research
Steps
Need
and
Possible
related Pervasive games
topics/trends/enablers Location gaming
(indicate
conditions Interoperability and semantics
and relationships)
Topic
Vision 2020
Engagement in augmented visits with location gaming
The majority of cultural exhibitions will provide location gaming
applications to engage different profiles of users into the cultural content.
Gap: SoA - Vision
Research
Steps
The barrier to this vision is mostly related to a wider adoption of
educational games in public institutions, and in turn this issue is related to
the uncertainty about the learning impact of games and the development
costs.
Need
and Devise a formal methodology to assess the educational effectiveness of
sgs (related to different knowledge/skills/attitudes, related to different
levels in the Bloom’s taxonomy, and to retention over time)
Possible
related Pedagogy
topics/trends/enablers
(indicate
conditions
and relationships)
(please duplicate this table for further topics in your SIG as needed)
3) Solutions
This step is optional. If you can complete some steps please do so. If not that’s ok.
For each identified topic develop solutions to close the gap – in the short term, medium term, and
long term. Also indicate the priority of each solution: High, Enabler, or Low.
We will have a Roadmap session on at the Madrid meeting on Wednesday morning 9 – 12.30. We
will review the roadmap contributions and then have an interactive working session.
Topic
Short-term
solution
(indicate
time,
conditions,
relationships)
Mediumterm
solution
(indicate
time,
conditions,
relationships)
Priority
Long-term
solution
(indicate
time,
conditions,
relationships)
(please duplicate this table for further topics in your SIG as needed)
References
[DCCCS10] Dellepiane M., Callieri M., Corsini M., Cignoni P., Scopigno R. Improved color acquisition
and mapping on 3D models via flash-based photography. In: ACM Journal on Computing and Cultural
Heritage, vol. 2 (4) article n. 9. ACM, 2010.
[VE06] VAN ECK R.: Digital game-based learning: It’s not just the digital natives who are restless.
EDUCAUSE Review 41, 2(2006).
[FAGM11] FROSCHAUER J., ARENDS M., GOLDFARB D., MERKL D.: Towards an online multiplayer
serious game providing a joyful experience in learning art history. In Proceedings of VS-Games 2011
(2011), pp. 160–163.
[CMAK11] CHRISTOPOULOS D., MAVRIDIS P., ANDREADIS A., KARIGIANNIS J. N.: Using virtual
environments to tell the story: The battle of Thermopylae. In Proceedings of VS-Games 2011 (2011).
BROWN, E., & CAIRNS, P. (2004). A grounded investigation of game immersion. In CHI ’04 Extended
Abstracts on Human Factors in Computing Systems. New York, NY, USA: ACM., , 1297–1300.
[CAO09] Christopoulos, D., Apostolellis, P., Onasiadis, A. Educational Virtual Environments for Digital
Dome Display Systems with Audience Participation. Proceedings of the 13th Panhellenic Conference
in Informatics - Workshop in Education, pp. 265-275, Corfu, Greece, 2009
[DCC07] DEHAIS C., CHARVILLAT V., CONTER J.: Interactive augmentation of photographs depicting
prehistoric engravings. In Proceedings of EUROSIS (2007), pp. 74–80.
[DAR09] DJAOUTI D., ALVAREZ J., RAMPNOUX O., CHARVILLAT V., JESSEL J.: Serious games & cultural
heritage: A case study of prehistoric caves. In Proceedings of the 2009 15th International Conference
on Virtual Systems and Multimedia (VSMM ’09) (2009).
[BT05] British Telecom, 2005 BT Technology Timeline
www.btplc.com/Innovation/News/timeline/TechnologyTimeline.pdf.
(2006
till
2050),
2005,
[AMLPPF10] Eike Falk Anderson, Leigh McLoughlin, Fotis Liarokapis, Christopher Peters, Panagiotis
Petridis, Sara de Freitas. Developing serious games for cultural heritage: a state-of-the-art review.
Virtual Reality (2010) 14:255–275, DOI 10.1007/s10055-010-0177-3.
[CS08] Cignoni P., Scopigno R. Sampled 3D models for CH applications: an enabling medium or a
technological exercise?. In: ACM Journal on Computers and Cultural Heritage, vol. 1 (1) article n. 2.
ACM, 2008.
[C2013] Coenen T.,”MuseUs: case study of a pervasive cultural heritage serious game”, in ACM
Journal of Computing and Cultural Heritage, special issue on serious games for cultural heritage,
issue 6, n°2, 2013 (to appear).