Workpackage description (18 months)
Workpackage number
Participant id
Start date or starting event:
Objectives
The challenge for this WP is to develop computational methods that successfully implement
solutions to problems in MM-understanding. These solutions almost always contain large and/or
complex data, and require complex modeling. It will be closely linked to other WPs packages that
involve development of methodology. The network is concerned with a large number of modalities
of data, of problem type and application. However, an important strength of a network in this area
is that similar computational approaches can be used to solve problems across distinct data types
and application. Most of the methods rely on statistical modelling, and are implemented by
different Monte Carlo simulation approaches.
The first objective is the evaluation of different computational approaches to implementing MM understanding. This will allow the network to gain understanding in the most appropriate
computational tools to implement its research, and to see where the strengths of the network
members are.
The second objective is the development of new computational methods that successfully
implement the applications that are developed. The challenge of complex and large data means that
our proposed work will certainly require novel and powerful developments in this area.
The third objective is that the WP coordinates the network's activities in computational methods,
producing a coherent body of research that is of benefit across the other WPs, and is seen to
contribute substantially to research in the field more generally.
By 18 months, it is expected that the first objective will be met (see deliverables below), and that
means to achieve the third objective are in place. The second objective forms the main task of the
WP through the entire life of the network, and by Month 18 the work to achieve it will have started.
Description of work
Task 1. Current State of the Art. To meet the first objective, the initial task of the WP is to assess
the network's experience in computational methods, evaluate the needs that the network has and
will have, and to propose common themes of research that will benefit the network's activities. This
will primarily be the responsibility of the WP leader.
Task 2. Evaluation. Also to meet the first objective, evaluation of the effectiveness of different
computational approaches. This will be done by looking at how different approaches perform on
the same problem (measured in terms of "accuracy of solution" and, where appropriate,
computation time). Our aim is to have examples from all the media types, as well as multimedia.
Task 3. Improving Computation Time. This addresses objective 2. Many existing computational
methods are effective but too slow in applications where there is user interaction i.e. content based
image retrieval, where user feedback must be responded to in seconds. Although computer
performance continues to improve, great improvements in computing time can be made in other
ways. Improvements rely on efficient coding and use of computer resources, and empirical studies
to compare the speed of different approaches. Another approach is to propose new models that are
more efficient to implement than a current solution. Another important development that is
becoming cheaper and easier to implement is parallel computing. This lends itself well to many
MM-understanding tasks, since in many cases the data can be parsed into essentially independent
sections, which may be processed simultaneously (for example, SzTAKI has developed approaches
in image analysis).
Task 4. New Computational Approaches. This task also addresses objective 2. Given the
novelty of much of the network’s research, it is anticipated that this will be largest task of the WP.
The network's expertise is broad, and covers all the main areas of computational methods. We
expect that all the following fields will be explored:
4.1 Monte Carlo Markov Chain (MCMC) (both Bayesian and classical statistics). This is a
very powerful set of techniques that is often the only implementation method for many
complex model and data applications. The network has expertise in the use of MCMC in all
media types (video – TCD; image – TCD, INRIA, UTIA, SzTAKI; audio – Univ. Cam.).
Markov models (in this context HMMs and MRFs) are particularly suited to MCMC. Given
the ambitious nature of the network’s research, much of the work will have to use MCMC
methodology. It will require the extension of MCMC to new models and the development of
new diagnostic tools.
4.2 Optimisation (both deterministic and stochastic). Optimisation often works with MCMC (i.e.
simulated annealing), as well as being used for more traditional methods such as maximum
likelihood and the EM algorithm.
4.3 Support Vector Machines. A relatively new and powerful technique, coming from ideas of
statistical regression, that is suited to problems of classification. It has seen application in
audio signal classification (Univ. Cam.). Several groups will develop this methodology
further for their own applications in all media types. The application to MM problems will
require innovative new developments of this technique.
4.4 Sequential Monte Carlo methods (particle filtering, etc.). Several groups (TCD, Univ. Cam.,
ISTI-CNR) have experience in the use of particle filtering, another relatively new and
powerful set of Monte Carlo tools. This is well suited to problems that receive a stream of
data – audio and video in our context; ISTI-CNR are proposing application to source
separation in document analysis. It is another exciting research area where there are many
possibilities for the network to develop innovative new methods that have application to MM
understanding and to statistical modelling more generally.
4.5 Other Monte Carlo methods will be used wherever possible by the group (exact simulation
by importance sampling, rejection method, etc. although we anticipate that their use will be
limited by the complexity and size of the data that we must handle. Included here are wavelet
techniques.
Task 5. Coordination. This task addresses the third objective. This will primarily be the
responsibility of the WP leader. The first task will be the setting up of a WP webpage within the
network’s web space, containing summaries of members’ research, links to members’ webpages,
technical reports etc. Reports that summarize the current and proposed skills of network members
in computation will also be published. Further, the WP will actively consider workshops or
conferences in the area, if they are deemed useful and are not adequately covered by other meetings
in MM.
Deliverables
WP webpage. An effective and cost efficient method of coordinating and disseminating
information about the network's activities in this area, particularly as the issues addressed by this
WP will concern almost all network members. This will form a part of the network webspace. It
will be established by Month 6.
Software repository. A part of the webpage will be a database of software developed by network
members, and links to other useful free software. This will be established by Month 6.
Reports at Months 6 and 18 on the current state of the tasks (see Milestones for more details)
Conference. A good candidate for a network conference or workshop is one in computational
methods for MM understanding. The month 6 report will commit to evaluating the need for such a
workshop, in the context of addressing all 3 objectives (but primarily objective 3). While the WP
does not want to commit to such a workshop now, it is fully prepared to organize one if it
establishes the need. This would take place by Month 18.
Publications and conference presentations by network members of their research (most will be
delivered after Month 18).
Milestones
Month 6:
 Report on the current state of the art in computational methods for MM-understanding,
together with an assessment of the network's achievements in the area to date, and its current
and future needs as the network research develops. This report will address parts of
objective 1.
 Assessment of the need for a network conference in the area of computational methods.
 WP webpage set up, as described in Deliverables, including software repository.
Month 18: report on the evaluation of existing computational methods to problems in MM
understanding. Interim report on the work being conducted by researchers in the network on the
various different computational approaches, with preliminary results where available. This will
complete objective 1, as well as contributing to the other 2 objectives.
Group
Sztaki
INRIA Ariana
TCD-MM
TCD-ML
CNR-ISTI
UTIA
UNI Cam
Tasks
2, 3, 4.1
2, 3, 4.1, 4.2, 4.5
1, 2, 4.1, 4.4, 4.5, 5
MM
7
6
18
2, 4.1, 4.4
2, 3, 4.1, 4.2, 4.5
2, 4.1, 4.4
5
23
6