ELIS – Multimedia Lab
Multimedia Lab
Rik Van de Walle
Department of Electronics and Information Systems
Ghent University - IBBT
Gaston Crommenlaan 201 bus 8
B-9050 Ledeberg-Ghent, Belgium
t: +32 9 33 14914
f: +32 9 33 14896
m: +32 478 39 62 31
t secr: +32 9 33 14911
e: rik.vandewalle@ugent.be
URL: multimedialab.elis.ugent.be
ELIS – Multimedia Lab
Multimedia Lab - UGent
• History
– background in image and signal processing
– 1999: Multimedia Systems and Applications
new research area within Ghent University
– 2001: Multimedia Lab formally founded
new research group within ELIS department
– 2003: link/co-operation with IMEC
– 2004: partner of the IBBT
www.ibbt.be, Flemish Government
MMLab Overview
F2F MMSEM, Athens – December 2006
2/28
ELIS – Multimedia Lab
Multimedia Lab - UGent
• People
– 5 staff members
•
•
•
•
•
Rik Van de Walle, full professor
Peter De Neve, 10% lecturer
Erik Mannens, project management
Peter Lambert, senior researcher
Ellen Lammens, administrative management
– currently 25 researchers
• PhD students
• “others” (projects)
– about 15 master students per year
MMLab Overview
F2F MMSEM, Athens – December 2006
3/28
ELIS – Multimedia Lab
Multimedia Lab - UGent
• Teaching
– programs
• ba/ma Computer Science (Faculty of Engineering)
• ba/ma Electronics (Faculty of Engineering)
• ba/ma Informatics (Faculty of Sciences)
–
–
–
–
–
Multimedia Techniques (baC3 - baE3)
Development of MM Applications (maC1 - maE1)
Advanced MM Applications - project (maC2 - maE2)
Multimedia (baINFO2)
Internet Technology (baINFO3)
MMLab Overview
F2F MMSEM, Athens – December 2006
4/28
ELIS – Multimedia Lab
Research topics: data + metadata
opera: il barbiere di Siviglia
composer: Gioachino Rossini
director: Claudio Abbado
type: audio
format: MP3
opera: il barbiere di Siviglia
composer: Gioachino Rossini
libretto: Cesare Sterbini
type: web page
format: html
place: Sydney
building: opera house
architect: Jørn Utzon
type: still image
format: JPEG2000
MMLab Overview
F2F MMSEM, Athens – December 2006
5/28
ELIS – Multimedia Lab
Research topics: adaptation - optimization
multimedia
terminals
INTERACTION DEPENDING ON
mono/stereo audio, color depth, resolution, bandwidth, ...
network
MMLab Overview
F2F MMSEM, Athens – December 2006
6/28
ELIS – Multimedia Lab
Research topics (1)
• Advanced video applications
– development and exploitation
of scalable video compression techniques
– motion estimation and compensation
– ROI-based video compression (Region of Interest)
– adaptation of multimedia data
with respect to varying usage environments
– development of iDTV applications
(a/o based on MHP, the Multimedia Home Platform)
MMLab Overview
F2F MMSEM, Athens – December 2006
7/28
ELIS – Multimedia Lab
Research topics (2)
• Mobile multimedia applications
– reduction impact of network losses
on quality of multimedia presentations
(a/o adaptive coding/decoding
wrt varying network characteristics)
– rendering multimedia data
on mobile terminals with limited resources
– transparent handover of multimedia sessions
between different devices (session mobility)
– hardware/software co-design
of embedded multimedia systems
MMLab Overview
F2F MMSEM, Athens – December 2006
8/28
ELIS – Multimedia Lab
Research topics (3)
• Standardization
– MPEG-21
•
•
•
•
–
–
–
–
–
Digital Item Declaration
Digital Item Adaptation
Digital Item Processing
Development of Reference Software for various tools
MPEG-4 Scalable Video Coding (SVC)
MPEG-4 Advanced Video Coding (H.264/AVC)
JVT - Joint Video Team
VQEG - Video Quality Experts Group
IFTA - International Federation of Television Archives
MMLab Overview
F2F MMSEM, Athens – December 2006
9/28
ELIS – Multimedia Lab
Ex. 1: exploitation of scalability
MMLab Overview
F2F MMSEM, Athens – December 2006
10/28
ELIS – Multimedia Lab
Ex. 2: session mobility for mobile applications
student watching
a lesson on a settop box
tv/set-top
box
Media stream
student continuing
the session on a
PDA
Media stream
Servers
containing
online
classes
Session Transfer
Pocket
PC
Session Transfer
Media stream
finishing the elearning session
on a tablet PC
Tablet PC
MMLab Overview
F2F MMSEM, Athens – December 2006
11/28
ELIS – Multimedia Lab
Ex. 3: video adaptation for mobile applications
MMLab Overview
F2F MMSEM, Athens – December 2006
12/28
ELIS – Multimedia Lab
In short: transparent content adaptation
MMLab Overview
F2F MMSEM, Athens – December 2006
13/28
ELIS – Multimedia Lab
New Research Paths
related to MMSEM
MMLab Overview
F2F MMSEM, Athens – December 2006
14/28
ELIS – Multimedia Lab
Feature Extraction (Video)
• Use cases:
– Video surveillance
• Detection of moving foreground objects
– Video adaptation
• Change bitrate, framerate, resolution, coding format, …
– goal: reduction of network load, processing load,
adjustment to client capacities or user preferences
• Extraction of relevant features: ROI, semantics, shot
detection
MMLab Overview
F2F MMSEM, Athens – December 2006
15/28
ELIS – Multimedia Lab
Feature Extraction tool
• Low level extraction:
MMLab Overview
F2F MMSEM, Athens – December 2006
16/28
ELIS – Multimedia Lab
Integrated Video Surveillance System
• Combining expertise:
Raw Pixel
Data
MPEG-21
Digital Item
H.264
/AVC
Digital Item
Item
Descriptor
Descriptor
Item
Descriptor
Descriptor
Item
Component
Component
Component
features
features
MMLab Overview
F2F MMSEM, Athens – December 2006
17/28
ELIS – Multimedia Lab
Interoperable Integrated Video Surveillance System
• Issues
– Which features (high level <-> low level, automatic
<-> manual)
– How to describe them
– When to describe them (camera, video processing
module, network node, client)
– How to link them to video resource (storage,
distribution, presentation, …)
New Use Case?
MMLab Overview
F2F MMSEM, Athens – December 2006
18/28
ELIS – Multimedia Lab
New IBBT Projects
related to MMSEM
MMLab Overview
F2F MMSEM, Athens – December 2006
19/28
ELIS – Multimedia Lab
PeCMan
• The goal of the PeCMan project is to
research the major technological challenges
of a user-centric solution enabling uniform
storage and manipulation of data as well as
universal access to this personal data. A
successful solution will have premium
capabilities in the areas of security,
performance, usability and description of
the personal data
MMLab Overview
F2F MMSEM, Athens – December 2006
20/28
ELIS – Multimedia Lab
Personal Digital Photo Archives
--> need for effective, uniform annotation strategy
Manual annotation
Automatic annotation
+
detailed
fast
-
time consuming, mostly
not uniform
error prone, semantic gap
Overcome semantic gap?
•
•
•
•
semantic gap is
still very wide!
Extract the ”good” features -> which algorithm?, domain specific?
Combine which primitive features for scene understanding?
Machine learning -> many methods, training on different datasets, ...
Ontologies -> need to model the whole world? Content vs. Context
ELIS – Multimedia Lab
Feature extraction
- human vision system best tuned to color and texture perception
- imitate cells of the visual cortex
- texture features can provide semantic information
- texture helpful in object segmentation
MMLab Overview
F2F MMSEM, Athens – December 2006
22/28
ELIS – Multimedia Lab
Machine Learning
- supervised <-> unsupervised
- self-organizing map: processing principles of human (visual & auditory) cortex
- incremental learning: knowledge must be continuously updated to manage
non-static archives, relevance feedback
Feature extraction
Machine learning
Classification
Ontologies
Scene understanding
--> use both manual and automatic annotation with minimal user overhead
MMLab Overview
F2F MMSEM, Athens – December 2006
23/28
ELIS – Multimedia Lab
Lindo
MMLab Overview
F2F MMSEM, Athens – December 2006
24/28
ELIS – Multimedia Lab
FIPA / AVEQ
• investigate the processes related to engineering and
analysis of audiovisual material, based on the notion of
a virtual model of the product
• explore some aspects of reverse engineering, including
audiovisual-quantization, analysis and classification
• Stretching the idea of the virtual model, we may
develop a much more powerful analysis and
classification system when we succeed in reconstructing
the model – the background, the characters, the objects
that constitute the scene of the image – quantifying the
subjects per scene by a finite list of normalized and
relevant vectors and being able to express semantics,
history and behaviour in a single geospatial and
temporal reference
MMLab Overview
F2F MMSEM, Athens – December 2006
25/28
ELIS – Multimedia Lab
Industrial partners
• Main partners (projects & MPEG)
– Alcatel, France-Belgium
– Artec/Televic, Belgium
– Barco, Belgium
– Belgacom, Belgium
– CISCO/Scientific Atlanta, USA
– Concentra, Belgium
– Deutsche Telecom, Germany
– eTampere, Finland
– France Telecom, France
– IBM, USA
– Intel, USA
– LG Electronics, Korea
MMLab Overview
F2F MMSEM, Athens – December 2006
26/28
ELIS – Multimedia Lab
Industrial partners
• Main partners (projects & MPEG)
– Microsoft Research, USA
– Museon, Netherlands
– Philips, Netherlands
– Porthus, Belgium
– ScreenPeaks, Israel
– Siemens, Germany
– Sun Microsystems, USA
– Telindus, Belgium
– Thomson Multimedia, France
– T-Systems Nova GmbH, Germany
– VMMa, Belgium
– VRT, Belgium
MMLab Overview
F2F MMSEM, Athens – December 2006
27/28
ELIS – Multimedia Lab
Contact
Erik Mannens
Department of Electronics and Information Systems
Ghent University - IBBT
Gaston Crommenlaan 201 bus 8
B-9050 Ledeberg-Ghent, Belgium
t: +32 9 33 14992
f: +32 9 33 14896
m: +32 478 39 62 31
t secr: +32 9 33 14911
e: erik.mannens@ugent.be
URL: multimedialab.elis.ugent.be
MMLab Overview
F2F MMSEM, Athens – December 2006
28/28