SEEDIFF
PART B
llk
Collaborative Project
Seeing Differently
SEEDIFF
Small or medium-scale focused research project (STREP)
Work programme topics addressed:
1.
2.
3.
4.
Name of the coordinating person: Risto Ilmoniemi
List of participants:
Participant
No.
Participant organisation name
Part. Short name
Country
1 (Co-ordinator)
AALTO-KORKEAKOULUSAATIO
AALTO
Finland
2
AALBORG UNIVERSITET
Aalborg
Denmark
3
Valtion taidemuseo
Ateneum
Finland
4
Senseg Oy
Senseg
Finland
5
SVEUCILISTE U SPLITU (UNIVERSITY OF SPLIT)
UNIST
Croatia
6
FACULTY OF SCIENCE UNIVERSITY OF ZAGREB
UNIZG
Croatia
1
SEEDIFF
Abstract
The abstract should, at a glance, provide the reader with a clear understanding of the objectives of the proposal,
how they will be achieved, and their relevance to the Work Programme. This summary will be used as the short
description of the proposal in the evaluation process and in communications to the programme management
committees and other interested parties. It must therefore be short and precise and should not contain confidential
information. Please use plain typed text, avoiding formulae and other special characters. If the proposal is written
in a language other than English, please include an English version of the proposal abstract in part B.
There is a limit of 2,000 characters.
Note:
The Abstract is required in Form A1, not in Part B.
SEEDIFF is a ‘neuro-ICT–art’ project that will enhance human sensory experiences through developing and
applying a new generation of mobile “display” solutions to induce alternative channeling of stimuli that expand or
enhance senses. Two products, an affordable tactile display and an enriched 3-D spectral display, target three
distinct outcomes to enable “seeing” by the visually impaired (blind/partially-sighted), improved discernment for
the color blind , and augmented perception for norrmally sighted. Transforming visual content of natural and manmade scenes, including masterpieces of art, into perceivable tactile realm will result in innovative experiences and
a foundational step toward augmenting digital inclusion for those with visual dysfunction. The vision of the project
is enhanced experiences of cultural and civilizational achievements, such as visual arts; augmented communication;
and increased educational opportunities.
2
SEEDIFF
Table of Contents
1
SCIENTIFIC AND/OR TECHNICAL QUALITY, RELEVANT TO THE TOPICS ADDRESSED BY
THE CALL
5
1.1
Concept and objectives
1.1.1 Concept
1.1.2 S&T objectives
1.2
Progress beyond the state-of-the-art
1.2.1 State-of-the-art
1.2.2 Technical limitations of existing products, processes and/or services
1.2.3 Main innovations
1.2.4 Patent search results
8
8
10
10
10
1.3
S/T methodology and associated work plan
1.3.1 Interdependencies
1.3.2 Overall strategy of the work plan
1.3.3 Project planning
1.3.4 Work description
1.3.5 Project flow
11
15
15
15
16
4
2
5
5
6
IMPLEMENTATION
6
2.1
Management structure and procedures
2.1.1 Management capability of the co-ordinator
2.1.2 Management structure and decision-making structure
Figure 1. SEEDIFF management structure
Coordinator
Vice Coordinator
Steering Committee
Executive Committee
Work Package Leaders / Group Leaders
Administrative Contact Persons
2.1.3 Monitoring, reporting progress and documenting results
Project evaluation and other meetings
Resolution of conflicts
Risk assessment and management
2.1.4 Meeting agendas
6
6
7
7
7
8
8
8
9
9
10
10
10
11
11
2.2
Individual participants
2.2.1 Participant 1: Aalto
2.2.2 Participant 2: Aalborg
2.2.3 Participant 3: Ateneum
2.2.4 Participant 4: Senseg
2.2.5 Participant 5: University of Split (UNISP)
2.2.6 Participant 6: University of Zagreb (UNIZG)
12
12
13
14
15
16
18
2.3
Consortium as a whole
2.3.1 Consortium overview and role of the participants
2.3.2 Complementarity of participants
20
20
20
3
SEEDIFF
2.3.3
2.3.4
2.4
3
Industrial involvement and exploitation of the results
Subcontracting
Resources to be committed
2.4.1 Use of the resources
2.4.2 Equipment resources
2.4.3 Other major financial resources
20
20
20
20
21
21
IMPACT
22
3.1
Expected impacts listed in the work programme
3.1.1 Impact on the competitiveness of the proposers
3.1.2 Strategy for impact achievement
3.1.3 European dimension
3.1.4 Contribution to Community societal objectives
3.1.5 Other relevant European or National funded research
3.1.6 Influence of external factors
22
22
22
23
23
24
24
3.2
Dissemination and/or exploitation of project results, and management of intellectual property
3.2.1 Exploitation and dissemination plan for use of project results
3.2.2 Management of knowledge and intellectual property
24
24
25
4
ETHICAL ISSUES
27
4.1
Benefits of the present study and experiments
27
4.2
Risks of the proposed approach? [are there any?]
27
4.3
Approvals by ethical committees and informed consent
27
4.4
Data protection issues
27
Section 1 (20 pages, not including the tables ): Explain the concept of your project. What are the main ideas that
led you to propose this work? Describe in detail the S&T objectives. Show how they relate to the topics addressed
by the call. The objectives should be those achievable within the project, not through subsequent development.
They should be stated in a measurable and verifiable form, including through the milestones that will be indicated
under section 1.3 below.
4
SEEDIFF
1
Scientific and/or technical quality, relevant to the topics
addressed by the call
1.1
Concept and objectives
1.1.1
Concept
We intend to develop and apply new technologies and methods to expand human sensory experience, in
particular in blind, partially-sighted, and color-blind people. Specifically, our long-term vision includes neuroICT–art convergence and synergy that would allow: 1) the blind to interact with the visual world by means of
affordable new generation of haptic display technology that will set the foundation for enhancing their digital
inclusion; and 2) color discrimination in the color blind and enhanced color vision in those with intact color
vision by an innovative 3D-display technology that doubles the number of distinguishable chromatic bands of
the color spectrum (from bichromatic to tetrachromatic or from trichromatic to hexachromatic). The vision in
its full glory will require not only the technology but also understanding of the brain mechanisms of sensory
information representation, learning, and plasticicity.
---Fig. 1 to be prepared here--Fig. 1. Expanding senses. A) From visual/spatial to haptic. B) From trichromacy to hexachromacy.
Our intended breakthrough is threefold.
First, we plan to develop a novel, advanced, haptic-display platform and tools for mapping visual/spatial
information into tactile–auditory forms that will allow interactions, including haptic navigation in visuallybased scenes and maps. This allows turning the wealth of visual content of natural scenes, images or graphical
objects, including masterpieces of art, into tactile realm to be experienced by partially-sighted and blind
people.
Second, we plan develop methodology for producing and displaying hexachromatic images and video. This
can be achieved by dividing the light spectrum into 6 bands and presenting every other band to the left and
every other band to the right eye using standard 3D display technology. This will enable the color blind to
experience all standard colors (although not in the same we we are used to) and allow the sighted to expand
color vision from trichromacy to hexachromacy, giving new opportunities and challenges to artists,
researchers, and media professionals.
Third, by means of behavioral and neurophysiological studies of the new kinds of sensory expansion provided
by our new technological platforms, we plan to open new insights into the ability of the brain to adapt to new
kinds of sensory input and into its ability to form new kinds of representations of the external world. These
insights will contribute to the foundations of sensory expansion technology and will help us in integrating and
optimizing tactile and auditory coding of visual information and guide us in developing a breakthrough in
hexachromatic display technology.
The multiple complexities of our high-risk venture will be addressed by the project partners with expertise
ranging from information and communication technologies, physics, psychology and neuroscience to art
history and media technologies. The relevance of our project stems from the fact vision is the sense that most
directly defines our relationship with the surroundings. When awake with eyes open, we constantly maintain a
5
SEEDIFF
representation of space around us based on what we see. Approximately half of our cortex, with tens of
distinct visual areas, is devoted to vision. Unfortunately, our visual machinery may be compromised by
genetic defects, disease, aging or accidents. There are close to 300 million blind or partially sighted people in
the world; they lack direct access not only to everyday visual scenes and objects that aid and guide our
interactions with the environment, but also to abstract cultural and civilizational achievements such as visual
arts or even educational opportunities. In particular, digital exclusion of blind and partially sighted people is
still growing (Post-i2010: priorities for new strategy for European information society (2010-2015), European
Blind Union Position Paper1)
Although these collateral consequences provide major setbacks to the blind, there is hope that they may be
alleviated with proper technology and assistance. Such expectations are supported by structural and functional
neuroimaging studies that have demonstrated cortical plasticity and reorganization and confirmed the
involvement of most visual areas, including the primary visual cortex, in processing information from other
sensory modalities, thus supporting our approach of a multi-sensory input based restoration and/or expansion
of human visual perception. Consequently, our aim is to use our new technologies and ideas in combination
with advanced neurodynamic brain imaging methods to design and tune haptic–auditory stimuli and
demonstrate the concept that the blind can have an opportunity to get a glimpse of what people experience
when they observe visual scenes or pieces of art.
Although in this project we focus on making new mobile technologies and visual art available and affordable
to the blind, the ideas can be applied more generally and may also find wide applications for the sighted
people. We also aim at expanding the awareness of the majority of us to the condition of our fellow citizens
such as the blind, the deaf, and other people with sensory limitations. In three major art museums (Finnish
National Gallery Ateneum, Helsinki; Modern Gallery, Zagreb; Denmark???), we will organize exhibitions in
which tactile–auditory stands and prototype haptic–auditory displays will be placed next to the original
paintings and made available to all visitors. The new technologies may be used in daily life, as well as in
training and learning events to expand sensory perception or to compensate for a temporal loss or a profound
sensory deprivation by enhancing multisensory cortical interactions.
1.1.2
S&T objectives
Our objective is to expand human sensory experience by introducing and developing new haptic and color
display technologies for blind, partially-sighted, color-blind, and sighted people.
Our vision and aims can be described as follows:
New Display Technologies
1) SEEDIFF will develop affordable haptic-display platforms specifically aimed at allowing the blind to
better cope in a world built for those with intact vision. The emergence of such new haptic display
technology will present new opportunities beyond our goals that focus on unique representations of art
and a mobile-device user interface for the blind.
2) Methods to interpret and map visual scenes into tactile form will be developed. This complex mapping
will be achieved as a stepped approach: initially a designer is given tools to convert visual images into
tactile representation using manual processing; subsequently the process will be automated.
1
http://ec.europa.eu/information_society/eeurope/i2010/docs/post_i2010/additional_contributions/ebu.pdf
6
SEEDIFF
3) The expansion of visual experience in color blind as well as in people with intact color vision will be
achieved by widening the brain’s sense of color with 3D display technology that will present
differently colored images to each eye.
Seeing Differently
4) By developing the presentation of visual art to the blind, SEEDIFF will, while expanding the sensory–
artistic experience, also evolve a new art form, both for the blind and for the seeing population.
5) Human visual and tactile perception will be investigated via functional brain imaging methods aimed at
exploring the representations of visual objects and scenes including visual art.
6) The research will explore how the visual experience of a tactile display is altered by auditory guidance
and study the dynamics of cross-modal integration in a novel fashion.
Science of Art and Color
7) The science of color and art will be enhanced via SEEDIFF, as will our understanding and appreciation
of what color really is and how our color vision depends on the fact that we have trichromacy, i.e., that
each color is formed as a mixture of the activation of three types of light-sensing cells in the retina of
the eye. Our experimentation with hexachromacy is expected to introduce new aspects to color science
and strengthen non-linear relationship between stimulus intensity and brightness, called the Helmholtz–
Kohlrausch (HK) effect, in respect of the color blind.
Education and Public Awareness
8) The teaching of art, art history, and haptic–visual communication for the blind will be advanced via
SEEDIFF. The blind will be able to draw images with the haptic displays and communicate with other
blind people using haptic displays. In addition, the technology will also be used to enhance the human–
machine communication interface for the blind. Thus, the societal implications are wide, which will
result in high attention from the media.
The objectives are described in more detail in Section 1.3.3.3 (Work package descriptions). The measurable
outcomes are the deliverables.
7
SEEDIFF
1.2
Progress beyond the state-of-the-art
Describe the state-of-the-art in the area concerned, and the advance that the proposed project would bring about.
If applicable, refer to the results of any patent search you might have carried out.
SEEDIFF will develop novel tools for tactile/haptic visual-scene presentation to the blind. Our new haptic
display technology will revolutionize the user interface of mobile phones or tablets for the blind, allowing
them to browse the web in tactile form. Also, with this technology, it will become possible to present art and
other visual scenes to the blind via the internet, so as to bring these experiences to millions of people rather
than to only a handful. Furthermore, we will use standard 3D display technology to expand the sensation of
color, which is anticipated to lead to novel types of color content in movies and in photography, helping the
color blind as well people with full color vision to enrich their visual experience when watching TV or other
media.
1.2.1
State-of-the-art
State of the art of tactile displays for tablets and mobile phones. [Ville, please write]
State of the art of hexachromatic displays. [Risto will complete this]
Today, the visual world is available to the blind via reading and mobility aids, tactile and auditory sensory
substitution devices, and invasive neuroprosthetic approaches. Haptic displays for the blind are currently
expensive, clumsy, and designed for very limited use such as reading in Braille. The blind may also use textto-speech technology for content browsing. This is slow and cumbersome even in text-based scenes, but has
no power in describing visual images. Printed embossed maps are also limited to a certain scale, as well as
expensive and scarcely available. Therefore, inexpensive tactile displays with interactive map functions could
significantly enhance the quality of life of the partially sighted in allowing them to navigate more accurately
and provide more freedom for them to visit unfamiliar places. Although constantly becoming more advanced,
sensory substitution devices provide only very rough images of basic objects and scenes and don’t offer
additional information needed for appreciating the richness of our visual world, especially its more advanced
formats such as visual arts.
Art for the blind is currently limited to only a handful of small exhibitions always placed in museum corners
(as in Centre Pompidou, Paris). In most cases, curators of the museum guide blind visitors; the blind person is
a passive participant. Occasionally, visitors are given a small tactile printed diagram, which is not very useful;
the blind do not like to use it (Museum of Modern Art, New York City; National Gallery, London). In Tate
Modern permanent exhibitions in London there is a possibility to explore sculptures but not pictures. Other
galleries have 3D printouts of artefact replicas so that all can enjoy multisensory exploration via hands-on
experiences; however, this is impossible for a painting. Equally accessible methods have been developed by
UNIZG: tactile diagrams are placed on specially designed stands where the blind can take a piece of art in
"their hands" themselves and explore it while listening to an audio narrative (Box 1). By developing new
haptic mobile displays SEEDIFF will make visual art as environmental scenes equally accessible to sighted
and blind people.
Numerous approaches and potential solutions aimed at overcoming the difficulties that the blind constantly
face have been proposed, such as reading and mobility aids, more advanced sensory substitution devices
(SSDs) and invasive sensory restoration and neuroprosthetic approaches.
8
SEEDIFF
Sensory substitution refers to the transformation of the characteristics of one sensory modality into the stimuli
of another modality. Thus, in the case of blindness, SSDs represent a non-invasive rehabilitation approach
within which visual information is captured by an external device such as a video camera and communicated
to the blind via a machine–human interface in the form of typically auditory or tactile input. Specifically, the
first targeted modality for substituting vision was touch, due to the ease of transforming visual into tactile
signals as these are both characterized by 2-D spatial representations. Pioneering work in this field was done in
the 1970s by Paul Bach-y-Rita who devised a tactile display which mapped images from a video camera to a
vibrotactile device worn on the subject’s back. Later, smaller mobile tactile devices placed on the tongue and
forehead of blind individuals characterized by better spatial somatosensory resolution were developed. The
development of auditory-based devices was triggered by certain limitations of tactile SSDs, namely their price
and limited spatial resolution. The first auditory SSD device was The vOICe system developed by Meijer in
1992, after which more advanced and sophisticated devices were put forward [WHICH? Please describe].
Both auditory and tactile SSDs are potentially be very useful as they may allow the blind to learn how to
interpret and use the transmitted information in simple visual discrimination and recognition tasks as well as in
more demanding contexts that require constructing mental images of more complex environments.
Nevertheless, such devices have shown limited success among blind individuals in allowing them to “see
with the skin” (White et al., 1970) or to “see with the brain” (Bach-y-Rita et al., 2003) or to “see with the
sound” (Voice, Levy-Tzedek et al. 2012). The level of performance achieved through any of these SSDs
remains, however, inferior to any of the perceptual standards in that sensory substitution is slower and
less accurate, and inferior in terms of automaticity and effort. Noticeably, compared to visual perception,
the number of objects which can be jointly accessed or available through sensory substitution devices
seems to be much lower. Contrary to vision, where people have been shown to be able to track multiple
objects (e.g., Pylyshyn, 2004), users of sensory substitution devices have not been shown to be able to
track or identify multiple objects at once. Also, visual-to-tactile sensory substitution devices might never
reach the maximal resolution obtained [What is meant by this?]. SSD researchers justify such
observations by suboptimal resolution of SSD or training of participants. However, current knowledge
opens an obvious challenge: how can we make sense of the positive evidence collected within the
perceptual assumption, while acknowledging the limits and negative evidence that has just been observed.
We believe that when it comes to visuo–tactile conversion, however, that an alternative lies in
resemblance between the acquisition of reading skills and the integration of sensory substitution devices.
This comes from the fact that, in both cases, the progressive automatization of new identification skills
consists in building a second route, which presupposes the existence of a first sensory route and parallels
it. Additionally, when it comes to visual-tactile conversion, one should have on mind the existence of
crossmodal transfers such as the one which exists between the visual and tactile identification of shapes or
visual and tactile motion. [This paragraph (and the previous one) has to be clarified so that the evaluators,
who are often generalists and do not know this particular area of study, can understand the text on first
reading]
Restoration of sensory input to the visually impaired represents an alternative to sensory substitution
devices. There are four main types of approaches in sensory restoration, targeting the retina, optic nerve,
lateral geniculate nucleus (LGN), and the visual cortex. Devices based on these approaches have so far shown
some promising results, as experienced blind users can utilize visual phosphenes generated by some of these
devices to create meaningful visual percepts.
9
SEEDIFF
1.2.2
Technical limitations of existing products, processes and/or services
Generally, sensory substation devices have shown truly remarkable progress in recent years. However, at this
time they are still not able to provide detailed information needed for fully experiencing our visual world,
especially its more advanced formats. Furthermore, when using these devices the users are basically passive
and unable to use many features of normal visually-based exploration of our world. Therefore, although
providing the basic information, they leave the blind deprived of important contents of our visual cultural
heritage such as art and most content of the web. In the case of invasive sensory restoration and
neuroprosthetic approaches, there are still several major issues preventing these from becoming true clinical
solutions, primarily their invasiveness, costs, technical limitations and the inability to offer them to all
populations of the blind. Thus, these solutions still do not provide sight that resembles natural vision and are
even more limited in providing the blind access to our visual cultural heritage such as art.
1.2.3
Main innovations
1) The Jovicic DiTacta diagrams and stand with audio guides allow independent exploration by blind people of
visual scenes in an accessible way. The novelty is in representing visual information in a new format; the approach
involves formal analysis and is practical and didactic.
2) New haptic touch-screen technology providing a widely-accessible and inexpensive means for transforming
visual art and other visual images such as maps to haptic–auditory form on tablets and mobile phones.
3) New art form: haptic drawings for both the blind and the sighted, allowing also the blind to create haptic
“paintings”.
4) Hexachromatic display technology to expand the experience of color for both the color-blind and the normally
sighted.
1.2.4
Patent search results
[Ville, do you have anything ready for this? It is probably sufficient to limit this analysis to IPR regarding the
application for the blind]
10
SEEDIFF
1.3
S/T methodology and associated work plan
A detailed work plan should be presented, broken down into work packages* (WPs) which should follow the logical
phases of the implementation of the project, and include consortium management and assessment of progress
and results. (Please note that your overall approach to management will be described later, in section 2).
Please present your plans as follows:
i) Describe the overall strategy of the work plan.
ii) Show the timing of the different WPs and their components (Gantt chart or similar).
iii) Provide a detailed work description broken down into work packages:
- Work package list (please use table 1.3a);
- Deliverables list (please use table 1.3b);
- Description of each work package, and summary (please use table 1.3c)
- Summary effort table (please use table 1.3d)
- List of (please use table 1.3e)
iv) Provide a graphical presentation of the components showing their interdependencies (Pert diagram or similar)
Note:
The number of work packages used must be appropriate to the complexity of the work and the overall value of the
proposed project. The planning should be sufficiently detailed to justify the proposed effort and allow progress
monitoring by the Commission.
Any significant risks should be identified, and contingency plans described.
* A work package is a major sub-division of the proposed project with a verifiable end-point - normally a
deliverable or a milestone in the overall project.
SEEDIFF will develop a range of new technologies and techniques related to presenting visual art in a tactile–
auditory form. A new generation of haptic displays and software-controllable surfaces and sensations is
expected to make even visual art equally accessible to sighted and blind (Box). The S/T methodology will
include: 1) electrostatic technology for haptic display on mobile devices; 2) programming interface to
transform visual scenes to haptic–auditory multi-sensory format; 3) behavioural and functional brain imaging
studies to design and optimize the sensory inputs and explore uni- and multi-sensory perceptual pathways and
multi-sensory integration compensating for and/or extending human visual perception. This neuro-ICT–art
trans-disciplinary project will also include the analysis of paintings from the perspective of an art historian,
surveys, multi-media and web presentations, book preparation; it will showcase the new art and explore its
acceptance among sighted and blind, both adult and children within educational contexts.
The partners include four well-known universities, a major national art gallery, and a fast-growing SME.
UNIZG will analyse paintings to be presented to the blind, describe their composition, the elements of form,
lines and surfaces, the details in the pictures, and their colors in order to present them as multiple tactile
diagrams. The presentation of colors by textures will be developed, which in the electronic format will include
novel vibratory or other time-dependent effects. To help navigate the display and to convey our message, we
will develop audio enhancements and provide an audio guide that, in addition to explaining the content and
background of the painting, leads the blind person from one point to the next in the painting and from one
frame to another. Once developed, other partners in the proposal will test several potential tactile–auditory
11
SEEDIFF
combinations representing the visual input behaviourally and with functional brain-imaging studies with the
blind.
Box. Novel concepts and technologies under development by the partners related to
transformation of visual scenes to tactile–auditory multi-sensory format and
expended color vision
I. Novel tactile–auditory exhibition of visual art for blind and sighted invented by UNIZG (Fig.1).
Fig. 1. Left: Exhibition by UNIZG where a 5-display representation of Caravaggio’s painting
was presented to the blind in tactile–audio form. Right: One of the details of the painting; the
tactile pattern is overlaid on the corresponding visual image.
II. The haptic display technology based on a recent invention and under development by
Senseg (Fig. 2) will allow one to produce vivid sensations of texture and form on touch-screen
devices such as tablets or smart phones. New forms of tactile interaction will be developed to
represent the richness of visual information to partially sighted people on these displays.
Fig 2. Left: The principle of our haptic display. A transparent positively charged layer under the
surface of a touch screen gives rise to an electric field that makes the finger tip negative
(positive charges are pushed away). A force proportional to the electric field and the charge
displacement is created (proportional to V2). Right: It is possible to create vivid sensations of
shapes or textures on mobile displays.
III. For the color blind with dichromacy, we plan to expand color vision by a novel 3Ddisplay technology that can add spectral dimensions (as if there were 3 or more cone types in
the retina).
Fig 3. A pair of differently colored paintings (Kandinsky). When viewed stereoscopically (left
image to the left eye, right image to the right eye), new color sensations can be created. In
photography, every other wavelength band could be directed to the left image and the other
bands to the right, creating a hexachromatic display.12
SEEDIFF
Figure 4. Examples of paintings and their representations for haptic displays.
Senseg will develop its haptic technology to allow one to present tactile–auditory representations of art and
other visual scenes to the blind. It will provide software tools to enable other partners to experiment with the
13
SEEDIFF
company’s new technology and to design multisensory content for the blind. The software will allow a flexible
pace of exploration; the amount of detail or background information is selectable. The audio presentation can
be synchronized to the tactile exploration without the need to follow a strict order. This complex processing
will be done in steps allowing both the automated conversion of visual information into tactile realm, as well
as guided design of haptic design and new interaction elements. Aalto will experiment with the new color
display technology based on presenting different parts of the color spectrum to different eyes and will work on
software development. Together with UNIZG and UNISP, it will also perform MEG/EEG experiments with
tactile and multi-color visual stimuli, including visual mismatch (MMN) studies.
In collaboration, the university partners will conduct a range of behavioral experiments to study the effects of
different combinations of the simultaneously presented tactile and auditory information, in order to better
understand the coupling between the two sensory inputs in creating the emerging visual experience. These
results will be used for specifying the properties of cross-modal information used in the final product of this
project. They will use neurodynamic MEG/EEG methods to study multi-sensory modulations of human visual
processing in signal and source space evoked by visual, tactile, and auditory–tactile stimulation in subjects
with full vision in search of an analog of a retinotopic mapping of the visual field presented via a haptic
display in a simple object vs. face discrimination task. Partially sighted and blind subjects will be used in a
subset of studies to find evidence of auditory–haptic stimulation-related neuronal plasticity.
AALBORG [Correct?] will address the acceptance of the created art forms in educational settings, among
teachers and students of different ages, in an attempt to explore the possibility of using the developed products
in educating the blind, and/or applying the obtained technology in improving current educational aids for blind
children and adults.
Ateneum will contribute by focusing on cultural, art historical and practical aspects of the project. It will
participate in designing and/or evaluating multisensory presentations (a haptic display + audio) for the blind or
partially sighted and in designing and/or evaluating multi-colored images. The exhibition will be made
available to collaborator galleries in different parts of Europe to allow a large audience across EU to get in
contact with the project in its different phases. Providing masterpieces of the museum collection as objects for
collaborative research, and offering space and expertise in arranging its own exhibition and exhibitions in
other major museums, the gallery will form one of the hubs of activity in SEEDIFF. Led by Ateneum,
SEEDIFF will arrange an in-depth visitory survey that includes interviews and video recordings of the
behaviour of the blind and other visitors of the exhibition. Based on the ideas, methods, contents, and
observations of SEEDIFF, Ateneum, together with others, will develop educational and demonstration
material, produce a book, and design web pages to obtain publicity and to spread the results.
Aalborg, which has long-term experience of creative work in bridging science and art as well as in working
with minorities such as the blind, will act as a catalyst to research, develop and showcase the new art offered
by the state-of-the-art advancements resulting from SEEDIFF. This will include a lab-based/museum-based
Participatory Design strategy questioning the impact for the blind. From this will evolve new frameworks for
creative expression and related stimuli-enhanced experiences. The Noldus Observer behavioural analysis tool
will be used to examine responses, which will inform to advance the creation of the SEEDIFF state-of-the-art.
The phenomenon of blind artists will be researched along with prior related cross-modal works. Exhibitions in
the context of European Cultural Capital events will be arranged.
14
SEEDIFF
1.3.1
Interdependencies in SEEDIFF
1.3.2
Overall strategy of the work plan
We will develop two ICT platforms for the presentation of visual art and scenes in new haptic and color forms. We
will divide the necessary steps towards our goals into well-defined Work Packages (WPs) so that when the WP
tasks are done according to the schedule (see Gantt table), we will reach the goals during the project duration.
WP1 (see also Section 2) is meant for making sure that SEEDIFF proceeds according to the plan. WP2 and WP3
deal with the development of haptic and color displays, i.e., the tools needed for completing work in the other WPs.
WP4, based on previous work on fixed (usually glass) haptic frames and on our invention of hexachromatic
displays, deals with developing methodology to transform visual images to the new forms. In addition,
neurophysiological experiments in WP5 provide information regarding neural foundations of enhanced sensory
perception. WP6 is a bridge from the methodological development to societal applications of the new technology
and serves WP7 by developing dissemination and educational uses for the technology. The solutions developed in
the different WPs will be tested using behavioural studies that will also provide a bridge towards testing their neural
correlates in neurophysiological studies planned in WP5.
1.3.3
Project planning
15
SEEDIFF
1.3.4
Work description
1.3.4.1
Work package list
WP
#
Work package title
Type of
activity
Lead
partner
#
Lead
partner
Personmonths
Start
month
End
month
1
Management
MGT
1
Aalto
17
1
36
2
Haptic hardware and software
RTD
4
Senseg
71
1
36
3
Hexachromatic color technology
RTD
1
Aalto
43
1
36
4
Tactile–auditory representations of
visual scenes
RTD
5
UNIST?
65
1
36
5
Neural foundations of enhanced
sensory perception
RTD
6
UNIZG
59
1
36
6
Social and personal aspects
RTD
2
Aalborg
58
1
36
7
Dissemination
RTD
3
Ateneum
24
1
36
1.3.4.2
#
Deliverables list
Deliverable name
WP
Nature
Dissemination
level
Delivery date
(proj. month)
1.1
6-month internal report
1
R
CO
6
1.2
12-month internal report
1
R
CO
12
1.3
18-month mid-term report
1
R
CO
18
1.4
24-month internal report
1
R
CO
24
1.5
30-month internal report
1
R
CO
30
1.6
36-month final report
1
R
CO
36
2.1
2
16
SEEDIFF
#
Deliverable name
WP
2.2
2
3.1
3
3.2
3
4.1
4
4.2
4
5.1
5
5.2
5
6.1
6
6.2
6
7.1
7
7.2
7
Nature
Dissemination
level
Delivery date
(proj. month)
8.1
Web page
7
?
PU
6
8.2
Exhibition at Ateneum Art Museum
7
?
PU
9
8.3
Mid-term exploitation report
7
R
CO
18
8.4
Exhibition #2
7
?
PU
24
8.5
Exhibition #3
7
?
PU
30
8.6
Book
7
?
PU
36
17
SEEDIFF
Del. No.
Deliverable numbers in order of delivery dates. Please use the numbering convention <WP number>.<number of
deliverable within that WP>. For example, deliverable 4.2 would be the second deliverable from work package 4.
Nature
Please indicate the nature of the deliverable using one of the following codes:
R = Report, P = Prototype, D = Demonstrator, O = Other
Dissemination level
Please indicate the dissemination level using one of the following codes:
PU = Public
PP = Restricted to other programme participants (including the Commission Services).
RE = Restricted to a group specified by the consortium (including the Commission Services).
CO = Confidential, only for members of the consortium (including the Commission Services).
Delivery date
Measured in months from the project start date (month 1).
1.3.4.3
Work package descriptions
Work package no.
1
Work package title
Management
Activity type
MGT
Partic. No.
Partic. Short name
Person-months per participant
Starting date
Month 1
1
2
3
4
5
6
Aalto
Aalborg
Ateneum
Senseg
UNIST
UNIZG
18
-
-
-
-
1
18
SEEDIFF
Objectives
Ensuring that the objectives of SEEDIFF are reached.
Smooth running of the project.
Completion of all tasks with maximum efficiency.
Ensuring intellectual property.
Resolution of any conflicts.
Accounting and reporting to the Commission.
Description of work (possibly broken down into tasks) and role of partners
The work is mainly described in part 2.1. This covers in particular:
T1: Constitution of the Executive and Steering Committees
This will be initially done in connection when agreeing about the Consortium Agreement
T2: Day-to-day management, financial and administrative management
The day-to-day management will be done by the Coordinator and Work Package Leaders.
The Executive Committee and Coordinator will be in charge of the administrative and financial management.
T3: Reporting and assessments
The Coordinator will be in charge of collecting the reports given by each Participant and will prepare the reports,
together with the rest of the Executive Committee, in a form corresponding to templates provided by the
European Commission.
The Executive Committee will assess the progress of SEEDIFF in their monthly meetings and, more thoroughly in
the SEEDIFF project meetings.
T4: Protection of intellectual property
The Coordinator will oversee that any essential IPR is evaluated as to whether patenting is necessary. The
Consortium agreement will describe in detail how the IPR can be transferred from Participants to the industrial
Participants or to other parties. IPR issues will be covered also in WP “Dissemination”.
1
SEEDIFF
Deliverables (brief description) and month of delivery
1.1 6-month internal report, delivery month: 6
1.2 12-month internal report, delivery month: 12
1.3 18-month mid-term report, delivery month: 18
1.4 24-month internal report, delivery month: 24
1.5 30-month internal report, delivery month: 30
1.6 36-month final report, delivery month: 36
Work package no.
2
Work package title
Haptic hardware and software
Activity type
RTD
Partic. No.
Partic. Short name
Person-months per participant
Starting date
1
2
Aalto
Aalborg
6
2
Month 1
3
4
Ateneum Senseg
1
55
5
6
UNIST
UNIZG
6
1
Objectives
Using Senseg’s hardware technology, to develop a software platform for haptic displays for the blind.
2
SEEDIFF
Description of work (possibly broken down into tasks) and role of partners
T1: Building a touch screen HW platform that can be used as basis for developing tactile presentations from
visual scenes.
T2: Developing an approach and software tools for mapping visual scenes to haptics, reviewing possibilities of
includign interactive scenes.
T3: Building a user experience demonstration on the hardware platform that can be tested using behavioral
experiments with apporiate visually impaired target groups.
T4: Building a user experience demonstration on the hardware platform that includes tactile interaction with the
visual scenes to produce life-like experiences of the art scenes
Deliverables (brief description) and month of delivery
2.1
2.2
Work package no.
3
Work package title
Hexachromatic color technology
Activity type
RTD
Partic. No.
Partic. Short name
Person-months per participant
Starting date
Month 1
1
2
3
4
5
6
Aalto
Aalborg
Ateneum
Senseg
UNIST
UNIZG
35
3
1
-
1
8
2
SEEDIFF
Objectives
1) To build a platform for studies of hexachromatic color perception.
2) To develop the methodology of transforming visual scenes to hexachromatic descriptions. These descriptions
will be used as the basis of presenting hexachromatic images on standard 3D display devices such as 3D TVs.
3) Exploration of hexachromatic technology as a new aspect in visual art
Description of work (possibly broken down into tasks) and role of partners
T1: Study of color spectra in natural and man-made environments
- Environments to be studied: Gardens with flowers and colorful fruits; zoo with a number of animals, especially
birds and butterflys; supermarket with multiple man-made items; art museum with paintings.The spectra will be
used to determine the division of the visible part of the spectrum into 6 bands so that maximal information of the
original spectra is retained.
T2: Hexachromatic photography
Natural scenes (those mentioned above) will be photographed so that two complementary exposures (call them
“left” and “right”) of each scene are obtained with comb filters so that the left exposure is formed by light in oddnumbered bands of the visible spectrum while the right exposure is obtained by even-numbered bands.
T3: Hexachromatic “art”
- Pieces of visual art consisting of initially uniform patches of color (such as those by Piet Mondrian, Vasili
Kandinsky, Johannes Itten, or Joseph Albers) will be divided into left and right “component pictures” so that the
average picture is identical with the original but the two pictures have different colors in each patch so that the
information content of the image is increased.
T4: Testing perception of hexachromatic displays (Aalto)
- Hexachromatic photography and art will be tested behaviorally (sensitivity thresholds to different kinds of color
changes will be determined) and neurophysiologically (odd-ball paradigm: hexachromatic pictures and their
trichromatic counterparts (averages of left and right images) will be presented and differences in brain responses
will be studied)
Deliverables (brief description) and month of delivery
3.1
3.2
1
SEEDIFF
Work package no.
4
Work package title
Tactile–auditory representations of visual scenes
Activity type
RTD
Partic. No.
Partic. Short name
Person-months per participant
Starting date
Month 1
1
2
3
4
5
6
Aalto
Aalborg
Ateneum
Senseg
UNIST
UNIZG
5
1
2
1
24
32
Objectives
1) To develop the methodology of transforming visual scenes (including works of art) to haptic descriptions.
These descriptions will be used as the basis of building 2D tactile diagrams, or for producing input to haptic
display devices.
2
Error! Reference source not found.
Description of work (possibly broken down into tasks) and role of partners
T1: Determine the relevant tactile and auditory features that allow the translation of presented visual scenes into
an understandable tactile-auditory format
T2: Conduct behavioral experiments in order to test participants’ experience of developed tactile–auditory
representations and determine how well they translate the visual experience to the “viewer”: how well the viewers
understand the display, what emotional reactions it triggers, and how the artistic message is conveyed
T3: Conduct behavioral experiments in order to determine how blind and partially sighted subjects perceive
different types of visual art presented on tactile displays.
T4: Behavioral studies of the perception of visual art (UNIST, UNIZG)
- Interviews of blind people related to their perceptual experience when visual art is presented through tactile
diagrams
- Testing how auditory guidance alone can give rise to perception of visual art
- Experimentation with different ways of tactile presentation in order to find optimal forms of information
presentation
T2: Testing tactile presentation devices developed by SEEDIFF in educational settings (UNIST, Aalbort)
- Presenting SEEDIFF’s products to teachers, interviewing them and developing strategies for the use of the new
technology in education
T3: Testing perception of microelements (< 2–3 mm) and macroelements in tactile displays (UNIST)
- Evaluation of electrostatic tactile elements that optimize tactile spatial acuity for simple 2D forms
T5: Experiments on new forms of tactile art (Aalto)
- The idea is that blind and sighted people can use tactile devices to create tactile presentations
Deliverables (brief description) and month of delivery
4.1 A final tested tactile-auditory display, delivery month: 24
4.2 Book titled “Expanding human sensory experience”, delivery month: 36
2
Error! Reference source not found.
Work package no.
5
Work package title
Neural foundations of enhanced sensory perception
Activity type
RTD
Partic. No.
Partic. Short name
Person-months per participant
Starting date
Month 1
1
2
3
4
5
6
Aalto
Aalborg
Ateneum
Senseg
UNIST
UNIZG
12
1
-
1
6
43
Objectives
1) To identify whether there is a tactile–spatial analogue of retinotopic representation using MEG
2) To demonstrate that multi-object discrimination in complex visual scenes is enhanced by combined tactile and
auditory stimulation
3) To demonstrate multi-frame visual-to-tactile–auditory integration in coherent visual art perception
4) To develop and demonstrate an objective measure of hexachromatic perception based on MEG or EEG
Description of work (possibly broken down into tasks) and role of partners
T1: Search for tactile analogy of retinotopy (UNIZG, UNIST, Aalto)
Both blind and sighted subjects will be studied. Tactile objects are presented at different eccentricities from the
center of the screen while MEG is being measured.
UNIZG will select blind subjects and after behavioral studies in Croatia, design haptic stimuli for the Senseg
haptic display, and conduct MEG together with Aalto.
T2: d
T4: MEG/EEG recordings of brain activity elicited by hexachromatic displays (Aalto, UNIZG)
- Trichromatic and hexachromatic visual images will be presented to subjects in alternate order in such a way that
the average color remains the same but in the hexachromatic displays, the left and right eye receive different
colors. The hypothesis is that the brain reacts differently to these two kinds of stimuli. This will give us an
objective measure of the new kind of color perception so that we will not need to rely on verbal reports of the
subjects alone.
3
SEEDIFF
Deliverables (brief description) and month of delivery
5.1
5.2
2
SEEDIFF
Work package no.
6
Work package title
Social and personal aspects [Tony, change this completely if needed]
Activity type
RTD
Partic. No.
Partic. Short name
Person-months per participant
Starting date
Month 1
1
2
3
4
5
6
Aalto
Aalborg
Ateneum
Senseg
UNISP
UNIZG
2
43
1
2
6
4
Objectives
[Tony, please propose objectives and content to this WP.]
To determine the acceptance of novel tactile displays in educational settings.
Description of work (possibly broken down into tasks) and role of partners
T1:
T2: To test tactile presentation devices developed by SEEDIFF in educational settings (UNIST)
- Presenting SEEDIFF’s products to teachers, interviewing them and developing strategies for the use of the new
technology in education
T3:
-
Deliverables (brief description) and month of delivery
6.1 Educational workshop on tactile–auditory and hexachromatic presentations of art to adults and children
6.2
3
SEEDIFF
Work package no.
7
Work package title
Dissemination
Activity type
RTD
Partic. No.
Partic. Short name
Person-months per participant
Starting date
1
2
Aalto
Aalborg
4
6
Month 1
3
4
Ateneum Senseg
5
1
Objectives
To ensure internal and external dissemination of results.
To insure a good protection of intellectual property generated by the project.
To organise a workshop opened to a large industrial and research European community.
To estimate the potential markets for the produced devices.
To prepare the exploitation of the results.
2
5
6
UNIST
UNIZG
1
7
SEEDIFF
Description of work (possibly broken down into tasks) and role of partners
T1: Web page: creation, development, and maintenance (Aalto)
- The web page will be targeted to a wide audience: students, researchers, developers, investors, the
lay public as well as the media. Parts of it will be designed to be accessible also to blind people,
especially when they have access to the haptic technology developed in SEEDIFF.
T2: A continuous evaluation of patentability
T3: Exhibitions in 3 major art museums
- Present exhibitions in 3 major art museums
- Conduct behavioral experiments in order to collect feedback regarding visitors’ experience and
satisfaction. Analyse survey results and send feedback to haptic display developers on the basis of the
analysis
T4: Workshops and training events
- Near the end of the project, a series of local Workshops reporting the results of the project will
be arranged in Aalborg, Helsinki, Split, and Zagreb.
T6: Visitor surveys in art museums (Ateneum)
- Both blind and sighted visitors are asked either orally or by forms to describe their experience and
satisfaction
- Analysis of the survey results; feedback to haptic display developers on the basis of the analysis
T7: Collecting user feedback of the demo and performing basic psychophysiological testing of the
perceived tactile effects.
T4: Book
-
2
SEEDIFF
Deliverables (brief description) and month of delivery
7.1 Web page, delivery month: 6
7.2 Exhibition of haptic art at Ateneum Art Museum, delivery month: 9
7.3 Midterm exploitation and patentability report, delivery month: 18
7.4 Exhibition of haptic and color display technology in Aalborg, delivery month: 21
7.5 Exhibition presenting SEEDIFF advances at Ateneum, delivery month: 34
7.6 Workshop and course proceedings, delivery month: 36
7.7 Final exploitation report, delivery month: 36
7.8 Book titled “Expanding human sensory experience”, delivery month: 36
1.3.4.4
Staff efforts
A summary of the staff effort is useful for the evaluators. Please indicate in the table number of person months
over the whole duration of the planned work, for each work package by each participant.
Identify the work package leader for each WP by showing the relevant person month figure in bold.
Partic.
No.
Partic.
short
name
WP1
WP2
WP3
WP4
WP5
WP6
WP7
Total
person
months
15
6
30
5
8
2
4
70
1
43
6
56
1
5
12
1
Aalto
2
Aalborg
-
2
3
1
3
Ateneum
-
1
1
2
4
Senseg
-
55
-
1
1
2
1
60
5
UNIST
-
6
1
24
6
6
1
44
6
UNIZG
1
1
8
32
43
4
7
96
17
71
43
65
59
58
24
Total
3
-
SEEDIFF
1.3.4.5
Milestones list
Milestones are control points where decisions are needed with regard to the next stage of the project. For
example, a milestone may occur when a major result has been achieved, if its successful attainment is a required
for the next phase of work. Another example would be a point when the consortium must decide which of several
technologies to adopt for further development.
Milestone
no.
Milestone
name
WPs
involved
Expected
date
Means of
verification
1
Haptic display hardware
2
M9
Delivery to participants
2
Haptic display software
2
M12
Demonstration
3
Hexachromatic hardware
3
M6
Availability at Aalto
4
Hexachromatic software
3
M12
Demonstration
Expected date
Measured in months from the project start date (month 1).
Means of verification
Show how both the participants and the Commission can check that the milestone has been attained. Refer to
indicators if appropriate.
1.3.5
Project flow
1.3.5.1
Integration of efforts
1.3.5.2
Technical risks
Risk
Impact
Probability
Criticality
WP 2
R1: Hardware will not works as desired
Medium to Low
high
Response:
4
Medium
SEEDIFF
R2: xxx
Low
Medium
Low
Low
Medium
Medium to low
WP 6
R11: Full integrated software not ready for the
prototypes
Response: Acquisition will be done simultaneous with the MRI software and the MEG software.
A physical coil acting as a small phantom will be used for post-scaling of the images.
1.3.5.3
Management and financial risks
Risk
Impact
Probability
Criticality
R19: Default of Coordinator
High
Low
Medium
Low
Medium
Response: Replacement of the Coordinator by Selma Supek
R20: Default of a Participant
High
Response: Sharing of the work through the different Participants and subcontracting of the remaining work. It can
be also envisaged, depending on the Participant, to proceed to an open call in order to introduce a new
Participant for replacement.
R20: Conflict between Participants
Medium
Low
Low to medium
Response: a conflict resolution scheme is included in the consortium agreement.
R21: Costs underestimated
Medium
Response: search of in house or external additional funds.
5
Low
Low to medium
SEEDIFF
2
2.1
Implementation
Management structure and procedures
Describe the organisational structure and decision-making mechanisms of the project. Show how they are
matched to the complexity and scale of the project.
2.1.1
Management capability of the co-ordinator
The present Coordinator (Prof. Ilmoniemi) has documented ability to lead research groups and
consortia with determination and success. He has successfully coordinated two EU-funded projects, one
with 9 partners2 and the other with 13 partners3. In addition, he has led the Framework Programme project
participation as the head of the BioMag Laboratory4 at the Helsinki University Central Hospital and as the
CEO of the Nexstim Company5. He has also considerable experience on being evaluator of European
consortium proposals and ongoing projects6.
The Coordinator has extensive experience in demanding management tasks. He led the BioMag
Laboratory of the Helsinki University Central Hospital from its beginning in 1994 until 2003. He was the
Chairman of the Board of Nexstim Ltd. after he founded the medical technology company in 2000; he
was CEO in 2003–2005; the company has since grown to 30 people and continues to thrive under its new
leadership. The Coordinator became full permanent professor of Engineering Physics in 2006 and head of
the Laboratory of Biomedical Engineering at the Helsinki University of Technology in the beginning of
2007 and the Chair or the Department of Biomedical Engineering and Computational Science in 2008.
2
The European Commission, DG XII, Grant Contract PSS*1046, Joint European project “Imaging of language
functions in the brain”, 1999–2001 (BioMag Laboratory, Coordinator).
3
The European Commission, FP7-HEALTH-2007-A, Collaborative project “MEGMRI - Hybrid MEG-MRI Imaging
System", Contract No. 200859, 2008–2012 (BECS, Coordinator).
4
The European Commission, Joint European project “Objective evaluation of cognitive brain function and
dysfunction”, Contract N.o BMH4-CT96-0819 (DG XII–SSMA), 1996–1998 (BioMag Laboratory, Partner).
5
The European Commission, STREP project “Enough Sleep”, Contract No. 518189, 2005–2008 (Nexstim,
Partner).
6
Evaluation of R&D proposals in the field of Neurosciences under the program “Quality of Life and
Management of Living Resources” of the 5th Framework Program, European Commission, DG XII, Brussels,
January 17–20, 2000; in IST Future and Emerging Technologies FET calls, Brussels, May 22–26, 2000;
Helsinki, Sept. 11, 2000; Brussels, May 14–15, 2001; Helsinki, November 2001; Brussels, March 19–22, 2002;
Brussels, October 26–29, 2004; Helsinki Nov. 2004. Review of the progress of the ESPRIT project “Adaptive
Brain Interfaces” (European Commission, DG XII; project ABI, No. 28193). Ispra, Italy, October 29, 1999 and
Helsinki, May 16–17, 2000; of the European Commission IST project “Insight 2+”. Leuven, Belgium, December
2, 2002; of FP6 project PRESENCCIA, February 2007. [Flagship 2011–12 ETC..].
6
SEEDIFF
2.1.2
Management structure and decision-making structure
The highly interdisciplinary nature of and the wide scope of expertise, from art to neuroscience to ICT,
needed in SEEDIFF will demand close coordination between the participants and a clear management
plan. Project Management is considered a separate work package. A strong partnership will be reinforced
by a Consortium Agreement that will establish the rules on how the Project is regulated.
The main objective of Project Management is to make sure that all steps needed for the realization of our
goals will be made in time and within the frame of the budget.
Project Management is the task of the Coordinator, the Vice Coordinator, the Steering Committee, and
the Executive Committee.
Steering Committee
External consultants
European Commission
Coordinator
Project Officer
WP1
1
Executive Committee
WP 1
WP 2
WP 5
WP 6
WP 7
Figure 1. SEEDIFF management structure
Coordinator
The Coordinator (Prof. Ilmoniemi) will lead the project and have the ultimate responsibility for its
success. He will employ an experienced, knowledgeable Project Manager with project-management
skills to help perform daily project management, to coordinate project self-assessment, to organize
meetings, to organize the financial and technical bookkeeping, and to collect the deliverables.
The Coordinator will, after consulting the Executive and Steering Committees, direct resources to any
critical bottleneck that may appear. If necessary, resources may be redistributed among the Participants.
The Coordinator will also use every effort to guarantee that the results of the project will be fully
7
SEEDIFF
exploited by the Participant organizations. The Coordinator will encourage the licensing of new
technology to third parties or the formation of startup companies especially in cases where the present
Participant SME is not able or willing to utilize some of the results.
In the course of the project, the fundamental question to ask regularly is, “how does the present work of
each Participant contribute to the objective of the project”. The Coordinator, together with the Steering
and Executive Committees, will help revise the work plan whenever needed and to redirect efforts
correspondingly.
If the Coordinator is not able to perform his tasks, the Vice Coordinator will overtake his duties until the
situation returns to normal. If necessary, a new Coordinator may be selected by the SC by a majority vote
of at least 4/6.
Supported by the Executive and Steering Committees, the Coordinator oversees the progress of the
project, initiates action when problems arise, delays can be foreseen, or when new opportunities for
additional progress are discovered. The Coordinator is the link between the project and the Commission;
he is responsible for the overall success of the project.
Vice Coordinator
The Vice Coordinator (Prof. Supek) will complement the work of the Coordinator. In particular, if the
Coordinator is not present at a meeting that would otherwise be chaired by the Coordinator, she will
assume the role of the Coordinator and chair the meeting.
If necessary, a new Vice Coordinator may be selected by the SC by a majority vote of at least 4/6.
Steering Committee
The Steering Committee (SC) is composed of all Group Leaders (see Table 1), including the Coordinator
(Prof. Ilmoniemi) and Vice Coordinator (Prof. Supek). It is the ultimate decision-making body of the
project. The Steering Committee will be the body that makes any strategic decisions (budgetary,
scientific, etc.) for the project. If a Group Leader is unable to attend an SC meeting, he/she can assign a
representative with full voting power to such a meeting. It is also possible for to participate in decision
making at an SC meeting using Skype or other electronic means.
The Steering Committee will meet twice a year at the regular SEEDIFF project meetings. The decisionmaking process within the SC will be the result of a collegial discussion, followed by a vote in case no
unanimous decision can be made (one vote per those SC members who are present / absolute majority). In
the unlikely event when no majority can be reached, the Coordinator (or if he is absent, the Vice
Coordinator) will have the decisive vote. At least 4 members of the SC must be present for a decision to
be made.
Executive Committee
The Executive Committee (EC) will consist of three key members of different Participants, initially Risto
Ilmoniemi (Aalto, Coordinator), Ville Mäkinen (Senseg), and Selma Supek (UNIZG). The composition of
the EC can be changed by the SC by a majority vote of at least 4/6. The EC will hold monthly meetings
(actual, remote, or email). These will be organized and convened by the Project Manager. At the
discretion of the EC, the Project Manager will be present at the meetings, documenting the outcome of
each meeting.
EC will be in charge of reporting and the revision and updating of the work plan, on-going risk
assessment, and general self-assessment of the project. It prepares the agenda and material for each
Steering Committee meeting.
8
SEEDIFF
The work of the EC will begin already when a positive funding decision has been obtained. EC members
may then be asked to participate in the negotiations with the Commission services. The EC will prepare
the project already before the starting date, being in close contact with the other Participants.
The work of the EC continues also after the end of the project as long as there are open issues regarding
the project.
Work Package Leaders / Group Leaders
The Work Package Leaders (WPLs, see table below), who are also Group Leaders of the respective
Participants, are in charge of the implementation of the project work plan and reporting to the
Coordinator. The WPLs will be in frequent (at least every two months) contact with the Coordinator to
ensure that the objectives of the individual WPs are met in accordance with (i) milestones and (ii)
deliverables and that (iii) each Participant fulfils its commitment to the work package.
Participant
number
Participant
short name
Group leader/WP
Leader
WPs to
lead
1
Aalto
Risto Ilmoniemi
1, 4
2
Aalborg
Tony Brooks
6
3
Ateneum
Anja Olavinen
7
4
Senseg
Ville Mäkinen
3
5
UNIST
Ana Jeroncic
2?
6
UNIZG
Selma Supek
6?
Administrative Contact Persons
Regarding the administrative management, each Participant will nominate an Administrative Contact
Person in each institute/company in charge of communication all relevant finance and reporting matters
(finance distribution, audit certificates, human resource, legal aspects, etc.). This person should be distinct
from the group leader (see Table 1 for the names of group leaders) to assure that the information needed
will be readily available to the Coordinator.
The Project Manager serves also as Aalto’s Administrative Contact Person. His duties include the
collection of reports and other information from the participants and administrative work at Aalto,
including the sending of reports to the Commission. He works closely together with the Coordinator, with
an office next to that of the Coordinator’s.
9
SEEDIFF
2.1.3
Monitoring, reporting progress and documenting results
Project evaluation and other meetings
The project will be formally monitored and evaluated by the Commission. In addition, the Steering
Committee may ask external evaluators, commentators, consultants, and collaborators to contribute to the
assessment of the project. We have already asked Charles Schröder7 and Semir Zeki8 to act as an external
consultant. In addition, David Blankett, member of the British Parlament, House of Lords, blind himself,
is consulting the project from the perspective of both societal decision making and blindness.
The progress of SEEDIFF will be reported in progress reports (deliverables 1.1–1.6) as well as in
SEEDIFF project meetings. The main purpose of the meetings is to advance the progress of the project.
When decisions are made or project issues are formally discussed, one of the meeting participants (Project
Manager, when present) is selected to keep the minutes of the meeting.
The Steering Committee or the Executive Committee may invite additional people to their meetings if that
is considered advantageous for the project. In particular, if an EC member is not a member of the SC,
he/she will be invited to the SC meetings (but without voting rights).
A 2.5-day Kick-off Meeting will be arranged by Aalto during the first month of the project. The first half
day of the meeting will be devoted to the background and the general approach of the project. The second
day will consist of thorough exposition and discussion of the plans of each of the Participants. On the
third day, specific strategic issues and expected challenges or bottlenecks will be discussed and decisions
will be made on what measures should be taken to address these.
The Steering Committee will hold a Review of Progress Meeting twice a year. The SC may propose a
change in the Project Plan in light of the progress made, new discoveries or developments made
elsewhere. Such a proposal will be presented to the Project Officer appointed by the Commission. These
meetings may be combined with review meetings in presence of external experts appointed by the
Commission.
The Executive Committee will hold EC Monthly Meetings, usually by email or Skype, to discuss the
progress of the project.
Training events and small seminars are arranged throughout the project; unofficial project meetings can
be arranged between Participants in connection to these events.
Resolution of conflicts
All disputes or differences arising in connection with SEEDIFF that cannot be amicably settled within the
Steering Committee shall be finally settled through arbitration in Brussels under the Rules of the
International Chamber of Commerce (ICC). Arbitration shall be conducted in the English language. Three
Arbitrators (3) shall be selected. Each Party shall select one (1) arbitrator. Each Contractor or party shall
also notify the other Contractor or party/parties in writing of its selection and the two (2) arbitrators so
chosen shall select the third arbitrator. If either Contractor or party fails to appoint an arbitrator within
thirty (30) days after the Demand for Arbitration, then the International Chamber of Commerce shall
make the selection. If the two arbitrators selected by the parties cannot agree on the third arbitrator within
7
Prof. Schröder is pioneer in primate and human studies in cross-sensory integration. His has demonstated that
low-level sensory areas are involved in multisensory integration.
8
Prof. Zeki, author of “Vision of the Brain”, is a specialist in vision and in neuroesthetics.
10
SEEDIFF
thirty (30) days from the date the last of the two has been selected, then the International Chamber of
Commerce shall make the selection. In deciding matters, the arbitrators shall be bound by the terms and
conditions of the Consortium Agreement, and they will decide ex bon et aequo.
The arbitration award, if providing for damages, shall include interest from the date of any breach or other
violation of the Consortium Agreement.
The arbitration award shall be final and binding upon the Contractors or parties, not subject to appeal, and
honoured by the Contractors or parties without having resort to any court; however, if the award is not
carried out voluntarily and without delay, it shall be referred to and enforced by any court having
jurisdiction over the subject matter or any of the parties or their assets.
Each Contractor or party bears its own expenses incurred in utilizing arbitration and the fees for
arbitration shall be borne equally between the Contractors or parties.
Risk assessment and management
Tables in Section 1.5 [CHECK NUMBERING] describe the main risks forecasted in this project with
their severity and indication of contingency actions.
The Executive Committee is in charge of the risk management, according to the following principles.
1. Based on the risk-analysis tables, estimate any risk that may have increased or is about to realize.
2. Establish and implement counter-measures if appropriate.
3. Set up early measures to minimize foreseeable risks.
4. Follow-up of the effects of the measures that have been taken.
The risks include those in management, financial situation, technical challenges or setbacks, change of
Participants, changes in dissemination and/or exploitation possibilities.
2.1.4
Meeting agendas
The biannual SC meetings and monthly EC meetings will be conducted according to a standard agenda;
minutes of the meetings will be assembled in Deliverables 1.1–6.
2.1.4.1
Standard agenda for Steering Committee meetings
1. Opening of meeting
2. Review of Progress
3. Deliverables and milestones
4. Human resources issues
5. Financial issues
6. Monitoring of ethical and gender issues
7. Decisions regardging corrective actions, if needed
8. Other issues
9. Closing of meeting
11
SEEDIFF
Individual participants
2.1.5
Participant 1: Aalto
Main tasks: Coordination, hexachromatic color technology, neurophysiological studies.
Previous experience: Coordination of EU-funded and national projects, summer schools, conferences etc.; R&D
on 6D color technology (funded by Runar Bäckström Foundation, 2011–2012); extensive experience on
neurophysiological studies using MEG, EEG, and TMS.
The Department of Biomedical Engineering and Computational Science (BECS) at Aalto University School of
Science is a pioneer in several neuroimaging technologies. MEG development started at Aalto University (until
2010 Helsinki University of Technology) already in the 1970’s. The work led to the spin-off company Neuromag
(now Elekta Neuromag), the world leader in MEG devices. MEG research has been extensive both at Aalto and at
the Aalto-initiated joint research center BioMag. TMS development started at Aalto University in 1992 and
continued at BioMag and Nexstim Ltd. (founded by Ilmoniemi in 2000). Nexstim combined TMS with MRI and
EEG to a new imaging modality that can map cortical excitability and functional connectivity. NIRS development
started at Aalto University in 1998 and has led to low-noise transportable multichannel frequency-domain
instrumentation. Ongoing studies on adults and neonates include clinical research.
In addition to the front-line instrumentation development, BECS is very strong in computational technology,
including the development of methodology for brain imaging.
BECS has produced a number of innovative technologies also in… The idea of enhancing color vision by using
the now-common 3D display technology was …
Risto Ilmoniemi is Academy Professor of the Academy of Finland and professor of Engineering Physics at Aalto
University; prior to the Academy Professorship, he was Chair of the Department of Biomedical Engineering and
Computational Science (BECS). He headed the BioMag Laboratory of the Helsinki University Central Hospital in
1994–2003 and was Chairman and CEO of Nexstim in 2000–2003 and 2003–2005, respectively. At Aalto, he has
lectured “Structure and Operation of the Human Brain”, “Classical Electromagnetism” and “Functional Imaging
in Medicine”. He is author of about 200 peer-reviewed papers and 10 patents. His work to combine brain imaging
and stimulation technologies has been recognized by several awards9.
Otto Olavinen is . He will be the Project Manager and Aalto’s Administrative Contact Person.
Mika Pollari is an expert in image and signal processing. He was project manager in the FP7 MEGMRI project
(2008–2012) and coordinated BECS’s participation in the FP7 project IMPPACT. He will participate in image
and signal processing tasks and in project management tasks when needed.
Representative publications
P. T. Vesanen et al., “Hybrid ultra-low-field MRI and magnetoencephalography system based on a commercial whole-head
neuromagnetometer”, Magn. Reson. Med. DOI 10.1002/mrm.24413 (2012).
A. Susac, R. J. Ilmoniemi, D. Ranken, and S. Supek, “Sensory-memory-based change detection in face stimuli”, Transl.
Neurosci. 1, 286–291 (2011).
9
New Technology Foundation Innovation Prize 1997; Foundation for Medical Technology First Prize 1997; Innosuomi 2004
Prize by the President of Finland to Nexstim (Ilmoniemi, the founder of the company, was CEO); European IST Prize 2006
to Nexstim.
12
SEEDIFF
Participant 2: Aalborg
Main tasks:
Previous experience:
Aalborg University is …
Tony Brooks is …
NN is …
Representative publications [of Aalborg]
xx
xx
13
SEEDIFF
2.1.6
Participant 3: Ateneum
Main tasks: Leader of WP7 (Dissemination). 1) Organizing art exhibitions for the blind and partially sighted at
the Finnish National Gallery Ateneum. 2) Arranging in-depth visitor surveys, to research the experiences of
persons using tactile and haptic devices in the exhibition. 3) Editing a book with tentative title “The Wounded
Angel. Expanding human sensory experience”.
Previous experience: Ateneum has a long term and vast experience in organizing art exhibitions. When working
with visitor groups in the visual field of an art museum, the museum has been a pioneer in using stimulation of
other senses but vision: hearing, touching and smelling,. The Ateneum team has got plenty of experience in
working closely with the blind, e.g., producing the program of 12 art works in the Ateneum collection in detailed
tactile images, combined with an audio (Pictures to Listen); exhibiting sculptures that visitors can touch in the art
museum; and developing audio descriptions for 6 key art works in the collections of the museum.
As the National Gallery of Finland, Ateneum Art Museum houses the largest and most significant collection of art
in Finland. In producing exhibitions, Ateneum is working in an active collaboration with major European art
museums, such as the Picasso Museum in Paris and van Gogh Museum in Amsterdam. In 2000, Ateneum was
awarded as the most accessible museum for the disabled in Finland.
Anja Olavinen is an art historian and the Head of Education in the Finnish National Gallery Ateneum. She has
contributed in many ways in organizing art exhibitions in the museum, and has written and produced several
major educational and other publications, books and films. She's an expert of the entire work of Hugo Simberg,
and has written and edited the catalogue for a comprehensive Simberg exhibition held in Ateneum in 2000. She
will give her expertise in analysing art as well as in arranging exhibitions for the blind and partially sighted, and
will participate in the transformation of art works into haptic form together with Natasa Jovicic. She will also
arrange in-depth visitor surveys that include interviews and video recordings of the experiences of the blind and
other visitors in the exhibition.
Erica Othman is an art educator, working as the Educational Curator in Ateneum, and responsible for workshops
in the museum. She is an expert in teaching art to a great variety of visitor groups, and has created “Pictures to
Listen”, a program for blind and partially sighted. She will be organizing in-depth visitor surveys for the blind and
for sighted people at the haptic / 6D color exhibitions.
Anu Utriainen is a Producer in Ateneum, and responsible for events in the Ateneum Hall auditorium. She will be
the Adminstrative Contact Person in the SEEDIFF project.
Representative publications
Olavinen, Anja, Paloposki, Hanna-Leena: Hugo Simberg 1873 – 1917. Exhibition catalogue, Helsinki 2000.
Olavinen, Anja: Rikas elämä. Helene Schjerfbeck 1862–1946 [A Rich Life. Helene Schjerfbeck 1862–1946]. E-book,
Helsinki 2012.
Olavinen, Anja: Light and Shadow. The Life of Helene Schjerfbeck. A filmed biography of an artist, Helsinki 2012.
14
SEEDIFF
2.1.7
Participant 4: Senseg
Main tasks: (1) Providing and developing a platform where the touchscreen of a selected portable device is
turned into ‘feel screen’ by adding high fidelity tactile sensations with Senseg’s breakthrough electrostatic tactile
technology. (2) Examining and developing an experience demonstration software for making visual art tangible.
Previous experience: Senseg technology has the potential to revolutionize the user experience for touch-based
devices. Senseg tactile effects, or haptics, are created with electrostatic ‘vibration’ to convey the sense of physical
textures, edges and contours. Senseg’s technology enhances overall user experience / user interface by adding
another sense to the interaction with a touch device. Senseg breakthrough technology has received number of
international recognitions such as being the only ever Finnish based invention to be named among the top 50
inventions by the TIME Magazine. Senseg has also been awarded with eight granted patents for its technological
innovations. Senseg was founded 2006; it has its main offices in Espoo Finland, while the other main location is
Taipei, Taiwan. Senseg also has USA and Japan representation.
Senseg’s business focus has been in bringing the technology to consumer markets and despite great interest from
the academic and the visually impaired community towards Senseg technology, there have been resource
limitations to support these interests. Nonetheless, Senseg has co-operated with Finnish Federation of the Visually
Impaired and has also taken part in Haptic perception and interaction in mobile and multimodal contexts
(HAPIMM), which is a joint research project between University of Tampere, Tampere University of Technology
in Finland and Stanford University, USA.
Dr. Ville Mäkinen is a physicist, neuroscientist, and entrepreneur. He founded Senseg in June 2006. Having
created Senseg’s initial customer base in the field of medical technology, Dr. Mäkinen went on to develop the
innovations that underlie Senseg’s solution along with the growing of the company. Dr. Mäkinen led the team that
raised Senseg’s initial funding from Ambient Sound Investments, Avera – the Finnish National Seed Fund, and
the Finnish Funding Agency for Technology and Innovations. Dr. Mäkinen carried out his doctoral studies
(Approved with distinction) in Helsinki University of Technology in the area of signal processing, medical
physics. Dr. Mäkinen has 23 peer-reviewed articles in the field of brain research, an area that he keenly follows
despite pursuing an entrepreneurial career.
Dr. Jukka Linjama has 15 years of technology development experience, followed by 8 years at Nokia as a
haptics specialist and concept designer. He has contributed to a variety of scientific publications ranging from
acoustics to human-computer interaction and has 10 granted patents to his credit. Dr. Linjama devotes his time to
scientific, industrial and artistic projects. Since joining Senseg in 2007, he has been responsible for the humancentric development of Senseg’s offering, a unique haptic solution for touch interfaces.
Representative publications: Being a company, Senseg is better described by its exceptional media visibility
than by scientific publications. Senseg has been featured or awarded in for example the following media:
15
SEEDIFF
2.1.8
Participant 5: University of Split (UNISP)
Main tasks: 1) Implementation of behavioral studies on the perceptual experience of visual art. 2) Behavioral
studies when blind subjects use the technology developed by Senseg. 3) Data analysis in neuroimaging studies. 4)
Reaction-time studies linking behavioral and neurophysiological data. 5) Address acceptance of the created art
forms in educational settings.
Previous experience: Coordination of EU-funded and national projects, summer schools, conferences;
experience on behaviour, and neurophysiological studies using EEG and TMS; long-term research cooperation
with Croatian Association of Blind.
University of Split, established in 1974, is currently the fastest growing university in Croatia, particularly in
research activities in physics, medicine, and history and art. It has more than 25000 students at 13 faculties and 4
education centers. Its School of Medicine is one of the University strongest schools in education and research
activities. Since 2011, the School runs a successful medical studies in English, reflecting the University’s
international ambitions.
The School is the centre of excellence for evidence-based medicine, with the Croatian Branch of the Cochrane
Collaboration, Croatian Centre for Global Health, and Clinical Hospital Center as its integral parts. Its most
productive research groups are in the fields of cancer biology, applied physiology, genomics and
neurophysiology. The School has several neuroscience laboratories: Laboratory for Human and Experimental
Neurophysiology, Laboratory for Speech and Hearing Science, Laboratory for Physiology, Laboratory for Basic
Neuroscience, and Laboratory for Clinical Neuroscience with Sleep Lab.
Researchers at School are actively participating in numerous national and international projects and
collaborations, funded by, i.e., TEMPUS, FP7, LLP, IPA, COPE.
Ana Jeroncic is Associate professor and Vice Chair of Department of Research in Biomedicine and Health at
University of Split School of Medicine. Her research interests are in bio- and neuroinformatics. During doctoral
research, as a Marie Currie fellow, she used EEG and computational approaches to develop automatic detection of
sleep spindles in infants and adults. She explored topics from molecular to system level and acquired a range of
programming and computational skills.
Andreja Bubic is Associate Professor at the Faculty of Social Sciences and Humanities at the University of Split.
She received her PhD at the Max Planck University for Human Cognitive and Brain Sciences and the University
in Leipzig, where she used fMRI and EEG to study visual perception and the function of the premotor cortex.
After spending a fellowship in the Laboratory for Multisensory research in Jerusalem, Israel, and a postdoctoral
period in Martinos Center in Boston, USA, she now lectures in Split and does research in the field of psychology
and cognitive science.
Representative publications [of UNISP]
Rogic M*, Jeroncic A*, Bosnjak M, Sedlar A, Hren D, Deletis V. A visual object naming task standardized for the Croatian
language: A tool for research and clinical practice, Behavior Research Methods. 01/2013 Epub ahead of print: PMID:
23344740
Boraska V, Jeroncic A, et al.: Genome-wide meta-analysis of common variant differences between men and women, Human
Molecular Genetics. 21(21), 4805-4815, 2012.
Striem-Amit, E., Bubić, A., Amedi, A. (2012). Neurophysiological mechanisms underlying plastic changes and rehabilitation
following sensory loss. In M. Murray and M.Wallace (Eds.),The Neural Bases of Multisensory Processes. CRC Press, Taylor
and Francis Group.
16
SEEDIFF
Bubić, A., von Cramon, D.Y., Schubotz, R.I. (2011). Exploring the detection of associatively novel events using fMRI.
Human Brain Mapping, 32, 370-381.
Bubic, A., Striem-Amit, E., Amedi, A. (2010). Large-scale brain plasticity following blindness and the use of sensory
substitution devices. In J. Kaiser and M. Naumer (Eds.), Multisensory Object Perception in the Primate Brain. New York:
Springer.
Bubić, A., von Cramon, D.Y., Schubotz, R.I. (2010). Prediction, cognition and the brain. Frontiers in Human Neuroscience,
4, 25. doi:10.3389/fnhum.2010.00025.
Bubić, A., Bendixen, A., Schubotz, R.I., Jacobsen, T., Schröger, E. (2010). Differences in processing violations of sequential
and feature regularities as revealed by visual event-related brain potentials. Brain Research, 1317, 192-202.
17
SEEDIFF
Participant 6: University of Zagreb (UNIZG)
Main tasks: Leader of WP6 and participation in WP5 and WP7: 1) Tactile coding and auditory interpretation of
works of art; 2) Behavioral and neurophysiological studies on the tactile and tactile-auditory perception of the
blind and blindfolded participants; 3) Novel time series and pattern recognition analysis of the neurodynamics
multi-sensory integration in blind and sighted.
Previous experience: Coordination/participation in EU-funded and other international and national projects,
organization of conferences, summer schools, strong expertise in the human brain development and
neurophysiological studies using MEG (via collaborating laboratories: BioMag, Helsinki, MIND Research
Network,..), EEG, MRI/fMRI
The University of Zagreb, is the strongest Croatian university encompassing all fields of science and art,
boosting interdisciplinary as well as translational research, nurturing the culture of innovation and transfer of
knowledge. The UNIZG is strongly committed to building a modern and innovative university through
stimulating research excellence and facilitating the transfer of knowledge into the business sector. In addition to
its traditional strengths in elementary particle physics, nuclear, and solid state physics, Department of Physics
increasingly supports interdisciplinary research in new materials, nanotechnology, biophysics, and cognitive
neurodynamics.
Selma Supek is Assistant Professor of physics and biophysics at the Department of Physics, co-founder and codirector of the first UNIZAG international interdisciplinary postgraduate program in Language Communication
and Cognitive Neuroscience (2001–2004), founder of the Mind and Brain series at the InterUniversity Centre
(IUC), Dubrovnik, Croatia (www.brain.com.hr), and a member of the UNIZAG Coordination for Biomedical
Engineering. Since her doctoral thesis research at Los Alamos National Laboratory her research interest is related
to MEG studies of the human perception and cognition.
Natasa Jovicic is an art historian, director of the Holocaust Museum in Jasenovac, Croatia, with a master degree
in art and multicultural education (Columbia College Chicago). In 2009, she opened one of the first tactile
galleries in Europe within the Modern Gallery in Zagreb, based on her inovative tactile-auditory methodology.
Since then, she organized 12 exhibitions and presented her work in major museums (Ateneum, Georges
Pompidou, Tate Modern, Guggenheim, New York, Art Institute, Chicago).
Ana Susac is Senior Lecturer at the Department of Physics with strong research experience in EEG and MEG
studies of the neurodynamics of face processing and research interest in educational neuroscience including
behavioral and neurophysiological studies. She spent research periods at the BioMag Lab in Helsinki, MEG Lab
in Jena, and postdoctoral research at the University of Oxford.
Davor Horvatic is Assistant Professor at the Department of Physics. He obtained PhD in high energy physics and
acquired wide range of theoretical, statistical, and numerical skills. During last few years, he concentrated his
research in the field of complex systems and time series analysis, particularly related to the physiological signals.
He collaborates with H.E. Stanley’s group at Boston University.
Representative publications [of UNIZG]
Susac A, Ilmoniemi R, Supek S. Face activated neurodynamic cortical networks, Med. Biol. Eng. Comput. 49(5), 531-543,
2011.
Sušac, Ana; Ilmoniemi, Risto; Pihko, Elina; Nurminen, Jussi; Supek, Selma. Early dissociation of face and object processing:
A magnetoencephalographic study. Human Brain Mapping. 30 (3): 917-927, 2009.
Supek, S., Aine, C., Ranken D., Best E., Flynn E.R., Wood C.C.: Single vs paired visual stimulation: Superposition of early
neuromagnetic responses and retinotopy in extrastriate cortex in humans, Brain Research, 830: 43-55, 1999.
Aine, C.J., Supek, S., George, J.S., Ranken, D., Lewine, J.,
Sanders, J., Best, E., Tiee, W., Flynn, E.R., and Wood, C.C.:
18
SEEDIFF
Retinotopic organization of human visual cortex: Departures from the classical model. Cerebral Cortex, 6:354-361, 1996.
.D. Horvatic, H. E. Stanley, and B. Podobnik, "Detrended Cross-Correlation Analysis for Non-Stationary Time Series with
Periodic Trends," Europhys. Lett. (EPL) 94, 18007 (2011).
19
SEEDIFF
2.2
Consortium as a whole
Describe how the participants collectively constitute a consortium capable of achieving the project objectives,
and how they are suited and are committed to the tasks assigned to them. Show the complementarity between
participants. Explain how the composition of the consortium is well balanced in relation to the objectives of the
project. If appropriate, describe the industrial/commercial involvement to ensure exploitation of the results.
2.2.1
Consortium overview and role of the participants
2.2.2
Complementarity of participants
2.2.3
Industrial involvement and exploitation of the results
2.2.4
Subcontracting
Subcontracting: If any part of the work is to be subcontracted by the participant responsible for it, describe the
work involved and explain why a subcontract approach has been chosen for it.
2.3
Resources to be committed
In addition to the costs indicated on form A3 of the proposal, and the staff effort shown in section 1.3 above,
please identify any other major costs (e.g. equipment). Describe how the totality of the necessary resources will
be mobilised, including any resources that will complement the EC contribution. Show how the resources will be
integrated in a coherent way, and show how the overall financial plan for the project is adequate.
(Recommended length for Section 2.4 – two pages)
2.3.1
Use of the resources
20
SEEDIFF
2.3.2
Equipment resources
Aalto University will make available for the project professional equipment for photography as well as MEG and
EEG facilities for neurophysiological studies of visual, tactile, and auditory perception.
2.3.3
Other major financial resources
21
SEEDIFF
3
Impact
(Recommended length for the whole of Section 3 – ten pages)
3.1
Expected impacts listed in the work programme
Describe how your project will contribute towards the expected impacts listed in the work programme in relation
to the topic or topics in question. Mention the steps that will be needed bring about these impacts. Explain why
this contribution requires a European (rather than a national or local) approach. Indicate how account is taken of
other national or international research activities. Mention any assumptions and external factors that may
determine whether the impacts will be achieved.
3.1.1
Impact on the competitiveness of the proposers
3.1.1.1
Direct applications and market prospects
3.1.1.2
Potentially patentable ideas
3.1.1.3
Benefits and competitive advantages
3.1.1.4
Economic justification
3.1.2
Strategy for impact achievement
22
SEEDIFF
3.1.3
European dimension
3.1.3.1
European problem to be solved
3.1.3.2
Effects on transnational co-operation
3.1.3.3
Implementation and evolution of EU policies
3.1.3.4
Improvement of European social and economic cohesion
3.1.4
Contribution to Community societal objectives
3.1.4.1
Quality of life
potential in SEEDIFF outcomes where the realised displays are applied for adaptive uses across alternative
rehabilitation/therapeutic training situations and end-users. So this can be added as a future research potential.
The availability of haptic displays in museums and elsewhere will give the possibility for the blind to explore art
and other visual information independently, without the assistance of personnel.
3.1.4.2
Health and safety
3.1.4.3
Employment
23
SEEDIFF
3.1.4.4
Gender issues
SEEDIFF is gender neutral, in the sense that both men and women may equally participate in any of its
activities and application of the results.
SEEDIFF outputs will be gender neutral. Both female and male subjects and patients are studied.
The Coordinator and the Steering Committee will oversee and ensure that all Participants are aware of
harms caused by gender bias within the project or elsewhere.
At each meeting, the Steering Committee will make an assessment of the gender balance within the
project and, if needed, will implement actions in order to correct any unjustified changes.
3.1.5
Other relevant European or National funded research
3.1.6
Influence of external factors
3.2
Dissemination and/or exploitation of project results, and management of
intellectual property
Describe the measures you propose for the dissemination and/or exploitation of project results, and the
management of knowledge, of intellectual property, and of other innovation related activities arising from the
project.
3.2.1
Exploitation and dissemination plan for use of project results
3.2.1.1
c
3.2.1.2
Validation of the technology
24
SEEDIFF
3.2.1.3
Dissemination of results and technology transfer
A special task is devoted to the exploitation of the results. All the Participants of the consortium are
involved in this work package.
Exploitation plans given here will be modified according to the results of the market evaluation performed
during the first months of the contract in order to maximise the exploitation of the results.
Haptic systems
The exploitation for haptic systems will be naturally done by Senseg.
The present world market for haptic systems for the blind is….
This market will be explored by Senseg.
Our dissemination strategy relies on four aspects.
1) The publication of scientific breakthroughs in high impact journals, communications at international
conferences, etc.
2) A web-based information data base will be elaborated and maintained during the project containing
all relevant information required for the implementation of the Work Plan, at the pre-authorised various
levels of confidentiality: reference documents (contract, Consortium Agreement), meetings announcement
and minutes, planning advancement, financial data (financial distribution & reporting), publications and
reports under preparation/reviewing. For these reasons, a three-level confidentiality structure will be
established: Public information, Project Participants, and Evaluators (Executive Committee members,
finance, reviewing, etc.), all data being protected by firewalls. This platform will be accessible to all
members via the internet allowing real-time communication.
3) Exhibitions
4) Media
5) Finally, a workshop at the end of the project will constitute a major dissemination of results.
3.2.2
Management of knowledge and intellectual property
In this project, patent protection will be sought as new IPR emerges. The ownership of any inventions,
techniques, or methodologies will remain at the inventor organization, which is also responsible for
applying for patent. Regarding exploitation of patents with joint inventors, the distribution of ownership
will reflect the level of research contributions to the patented invention(s) of the different inventors and
their laboratories. This will be decided by mutual agreement under supervision of the SEEDIFF
Coordinator.
In addition, a basic training on quality and intellectual property management will be proposed during the
first phase of the programme to all the project Participants by a person responsible for patent.
An IPR management plan will be included in
the Consortium Agreement. Pre-Existing Knowledge to
25
SEEDIFF
be included/excluded in/from the project will be identified and documented in an annex to the Consortium
Agreement.
In particular, the Consortium Agreement will describe how the results of the project may be transferred
from the project or its Participants to the industrial Participant and other parties.
26
SEEDIFF
4
Ethical Issues
Describe any ethical issues that may arise in the project. In particular, you should explain the benefit and burden
of the experiments and the effects it may have on the research subject. Identify the countries where research will
be undertaken and which ethical committees and regulatory organisations will need to be approached during the
life of the project.
High ethical standards will be followed in all phases and aspects of the project.
Care will be taken to ensure that all human studies are safe for the volunteer subjects, whether blind,
partially sighted or sighted, and that the instruments will be designed, built, and used in conformity with
the regulations of European Union.
We strongly believe that the benefits of the present study will justify the cost and effort of the project.
4.1
Benefits of the present study and experiments
A large number of blind or visually impaired people have no or limited access to xxx. This project will
help in xxx and provide xxx to a large number of people.
For example, ….
4.2
Risks of the proposed approach? [are there any?]
.
4.3
Approvals by ethical committees and informed consent
All human studies within the project will be performed only after appropriate ethical approvals have been
obtained. The ethical committees and regulatory organizations in the Participant countries that will need
to be approached during the life of the project are listed below:
Finland: 1) Coordinating Ethics Committee, Hospital District of Helsinki and Uusimaa, 2) Research
Ethics Committee, Aalto University.
Denmark: ??
Croatia: Research Ethics Committee of University of Split School of Medicine. ?? from UniZG [Please
clarify]
Each subject (blind, partially sighted, or sighted) to be studied will be informed orally or in writing about
the purpose, execution, possible side effects or risks of the study and about the right to leave the study at
any time without any negative consequences.
Each subject who, after being explained orally or in writing about the study as explained above, wishes to
participate in the study will be asked to sign an informed consent form prior to the beginning of the study.
No subject will be studied without informed consent. Both the written information leaflet informing about
the study and the informed consent form will be the ones approved by the ethical committee in question.
4.4
Data protection issues
All subject or patient data will be coded after
determined from any of the data files. The data
the study session so that subject identity cannot be
files are labeled and the software identifies the subjects
27
SEEDIFF
and patients only by the code, which can be related to the individual only by the investigator in question.
ETHICAL ISSUES TABLE
YES
Informed Consent
 Does the proposal involve children?
 Does the proposal involve patients or persons not able to give
consent?
 Does the proposal involve adult healthy volunteers?
 Does the proposal involve Human Genetic Material?
 Does the proposal involve Human biological samples?
 Does the proposal involve Human data collection
Research on Human embryo/foetus
 Does the proposal involve Human Embryos?
 Does the proposal involve Human Foetal Tissue / Cells?
 Does the proposal involve Human Embryonic Stem Cells?
Privacy
 Does the proposal involve processing of genetic information or
personal data (e.g. health, sexual lifestyle, ethnicity, political
opinion, religious or philosophical conviction)?
 Does the proposal involve tracking the location or observation
of people?
Research on Animals
 Does the proposal involve research on animals?
 Are those animals transgenic small laboratory animals?
 Are those animals transgenic farm animals?
 Are those animals cloned farm animals?
 Are those animals nonhuman primates?
Research Involving Developing Countries
 Use of local resources (genetic, animal, plant etc)
 Benefit to local community (capacity building i.e. access to
healthcare, education etc)
Dual Use
 Research having direct military application
 Research having the potential for terrorist abuse
I CONFIRM THAT NONE OF THE ABOVE ISSUES APPLY TO MY
PROPOSAL
28
X
X
-
-
-
-
PAGE
SEEDIFF
Comments:- Museum showcase and positing an angle of building on prior work that can
sometimes get some add on value
In section 1.1.1 - you ask which art museums. Maybe this should be at least one in each
partner country and we have a strong cooperation with our regional art museums leadership
board who are keep to work with us and I can approach them to ask about a SEEDIFF
exhibition when appropriate. We also have links to other leading museums in Denmark should
a wider dissemination be required. I see no problems in this to showcase in Denmark.
Also, I feel it appropriate to say that I see this work in a way related to the Neuroesthetic
studies/activities being published (Zeki and others). I published (only a position paper) on
Neuroaesthetic Resonance as my previous research enquired to aesthetic resonance
(involving two EU projects and an i3 future probe (briefly outlined below inc URLs) - and in a
way I see SEEDIFF building on some of the embedded concepts from those projects) =
(1) CARESS - CREATING AESTHETICALLY RESONANT ENVIRONMENTS IN SOUND; The
purpose of CARESS was to create technological and educational tools that will motivate and
empower children to develop creativity, imagination and expression, through interactive
acoustic environments.
(2) CAREHERE - CREATING AESTHETICALLY RESONANT ENVIRONMENTS FOR THE
HANDICAPPED, ELDERLY AND REHABILITATION. The objective of our project is to
empower children and adults with special needs, the elderly in long term care and people
undergoing rehabilitation in hospital or at home, following for example stroke or brain injury. By
giving them access to affordable, appealing and readily usable state of the art technology for
the improvement of their physical and cognitive skills using feedback from acoustic and visual
stimuli. We are concerned with the (re-)development of physical and cognitive skills by
interaction with a responsive sound and visual environment: the improvement of motor control
through direct and immediate feedback through the aural and visual senses.
(3) - The i3.net future probe Twi-aysi = The World is as you see it. While CARESS successfully
motivated and empowered children to develop creativity, imagination and expression, through
interactive acoustic environments. The objective of Twi-aysi was to answer the question: Can
immersion in a visual environment hold similar potential for such children in terms of the
aesthetic resonance they might derive from movement within such a visual space? The
success of Twi-aysi led to the CAREHERE project (above).
29