The Digital Media Project
Date
2011/03/17
No.
1312/OCTV
Source L. Chiariglione, S. Matone
Title
Proposal in response to Core OCTV CfP
Proposal in response to Core OCTV CfP
1
Introduction
This document is a response to the Core OCTV CfP (dmp1309) and associated Requirements
(dmp1310) proposing technologies that would enable interoperability of independent
implementations of “Source”, “Server” and “Sink” depicted in Fig. 1.
Fig. 1 – Target systems of this proposal
2
System functionality
2.1 General
Please note that
1. In the following Content means any audiovisual sequence or a scene, i.e. a composition of media
2. Two out of three of these devices (e.g. Source and Client) may coalesce into a single device..
2.2 Source
The Source is a device (e.g. a computer equipped with appropriate peripherals, a camera with
processing capability and WiFi) by means of which a user can
1. Capture an audio-visual sequence
2. Stream it to the Server for storage or for distribution to Clients
3. Store it to a storage device (local or in the cloud)
4. Edit the audio-visual sequence
5. Upload the audio-visual sequence to the Server with
a. Description
b. Licences
6. Compose a scene of which audio-visual sequences captured in real time or stored are elements
7. Upload the scene to the Server
2.3 Sink
The Sink is a device (e.g. a computer equipped with appropriate peripherals) by means of which a
user can
1. Search for content on the Server
2. Select content on the Server
3. Receive streamed content from the Server possibly protected with a Conditional Access System
4. Interact with content on the Server via scene composition functionality and rich UI interface
2.4 Server
The Server is a device that
1. Lets a Source stream content for uploading or streaming to a Sink
2. Lets a Source upload a content
3. Lets a Sink receive content via streaming or download
4. Provides responses to content search by Sinks
5. Checks that a selection of a content for streaming is compatible with rights declared by Source
6. Adapts content to make it suitable to a Sink’s characteristics
7. Streams content to a Sink possibly protected with a Conditional Access System
3
System walkthroughs
3.1 Real-time streaming
Server provides a service to its subscribers whereby a subscriber can stream real-time video to any
other subscriber.
John sends a real-time video to Jane via Server
1. John uses a widget to
a. Authenticate with Server
b. Activate camera
c. Request to stream AV sequence to Jane’s Sink
2. Server requests Jane’s Sink to send environment description
3. Jane’s Sink sends environment description to Server
4. Camera streams AV sequence to Server
5. Server
a. Adapts video
b. Encrypts adapted video
c. Sends to Jane
i. Encrypted video
ii. Encrypted keys
6. Jane has a backgroud application on her Sink that
a. Decrypts keys
b. Decrypts video
c. Decodes video
d. Presents video
3.2 Creation and uploading of rich content
Server provides a service to its subscribers whereby a subscriber can upload content for distribution
to a plurality of subscribers.
Mary sends non-real-time video to a group of friends
1. Mary uses a widget to
a. Activate a camera
b. Store AV sequence to her Source
c. Edit AV sequence from her Source
d. Compose a scene of which AV sequence is an element on her Source
e. Create DI with metadata, template licence etc. on her Source
f. Authenticate with Server
g. Upload DI to Server
3.3 Push distribution of rich content
1. Mary requests Server to distribute video to Mary’s group of friends
2. Server requests the Sinks of Mary’s group of friends to send their environment descriptions
3. Server
a. Adapts video independently for each member of Mary’s group of friends
b. Sends the scene to Mary’s group of friends via parallel unicast sessions
4. Each member of Mary’s group of friends has a backgroud application on their Sinks that
a. Decodes scene
b. Presents scene
5. Each member of Mary’s group of friends interacts with scene
3.4 Pull distribution of rich content
Server provides content on demand service to its subscribers.
Mario watches content on demand
1. Mario uses a widget to
a. Authenticate with Server
b. Search for content on Server
2. Server displays choices
3. Mario selects Mary’s scene as content of interest
4. Server
a. Checks that selection is compatible with rights declared by Mary
b. Requests environment description of Mario’s Sink
5. Mario’s Sink sends environment description
6. Server
a. Adapts video to make it suitable to a Mario’s Sink’s characteristics
b. Streams scene
7. Mario
a. Watches video
b. Interacts with scene
4
Proposal
4.1 General proposal
The following technologies are proposed to satisfy the given functionality
Functionality
Video coding
Audio coding
Still pictures
2D graphic image
Text
Font
Composition
Metadata
Digital item
File format
Technology
AVC profile?
AAC
JPEG baseline
PNG
Unicode
OFF
LASeR Mini profile
MPEG-7 Simple Profile/TVA (which subset?)
DID
ISO Base Media FF (which part of it?)
MP4 FF
DI FF?
Conditional access system MSAF
Streaming
MPEG-2 TS, RTSP
Presentation
Qt
Widget
MPEG-U part 1
4.2 Technologies for real-time streaming
The following protocols and technologies are proposed to satisfy the given functionality
Device
Functionality
Protocol
Source Authenticate with Server Identify user
Authenticate user
Encode camera output
Deliver to End User
Deliver Content
Server Deliver to End user
Package content
Adapt Content
Adapta Content
Encrypt content
Process content
Encrypt keys
Process content
Stream content
Deliver Content
Deliver to End User
Deliver Content
Sink
Stream receiver
Deliver Content
Decrypt keys
Process content
Decrypt content
Process content
Decode content
Process content
Technology
Media framework
Media framework
IPMP components
IPMP components
RTSP
RTSP
IPMP components
IPMP components
Media framework
4.3 Technologies for push distribution
The following protocols and technologies are proposed to satisfy the given functionality
Device Functionality
Source Encode camera output
Edit AV sequence
Compose a scene
Create DI
Server
Sink
Protocol
Technology
Media framework
Media framework
LASeR Mini Profile
Create Content
DIDL
Describe Content MPEG-7 SMP/TVA
Create Licence
REL
Authenticate with Server
Identify user
Authenticate user
Upload DI
Store Content
Request to distribute video
Deliver Content
Adapts video for each destination
Media framework
RTSP
Deliver Content
Stream receiver
RTSP
Decode content
Media framework
4.4 Technologies for pull distribution
The following protocols and technologies are proposed to satisfy the given functionality
Device Functionality
Sink
Authenticate with Server
Server
Sink
Server
Sink
Search for content
Displays choices
Selects content of interest
Checks compatibility with rights
Deliver to End user
Adapt Content
Stream content
Stream receiver
Decode content
Protocol
Technology
Identify User
Authenticate User
Search Content
Verify Licence
Package content
Adapt Content
Deliver Content
Deliver Content
Process Content
REL
Media framework
RTSP
RTSP
Media framework
4.5 Proposal for Platform
The following functionality and technologies are proposed
Device
Server
Functionality
Environment
User management
Content management and storage
Data Persistence and Processing System
Data representation (Sink requests)
Technology
JEE + Spring
Spring security
Code to be developed ad hoc
Hibernate and DBMS (e.g. Postgres)
XML/HTML/CSS/LASeR
Web Services management
General security layer
Source/Sink Environment
Streaming
Media Framework
Rendering
REST API
Spring security
Linux/C++
GStreamer
GStreamer
Qt
Fig. 2 depicts the combination of proposed technologies
Fig. 2 – A possible architecture of Core OCTV Server