MULTIMEDIA SYSTEMS
NETWORKING
MULTIMEDIA SYSTEMS
IREK DEFEE
SOFTWARE
SERVER
NETWORK
CLIENT
MULTIMEDIA SYSTEM
MULTIMEDIA SYSTEMS
IREK DEFEE
• WE CONSIDER MULTIMEDIA
SYSTEMS BUILT-UP ON LARGE
SCALE: REACHING PRACTICALLY
EVERYBODY, LIKE TV, RADIO AND
TELEPHONE TODAY, THAT IS
THOUSANDS AND MILIONS OF USERS
• MULTIMEDIA MEANS BROADBAND
STREAMING AND INTERACTIVITY
MULTIMEDIA SYSTEMS
IREK DEFEE
MAIN PROBLEMS IN NETWORKING
• NETWORKS FOR MULTIMEDIA
REQUIRE:
- SUFFICIENT/CONTROLLED BANDWIDTH
- RESERVATION OF NETWORK
RESOURCES (WE CAN NOT LOSE DATA)
- STREAMING
- FAST INTERACTIVITY
DELIVERY CAN BE WIRED OR WIRELESS:
WIRED - USES PHYSICAL LINKS
WIRELESS – USES RADIO
MULTIMEDIA SYSTEMS
IREK DEFEE
• A MODEL OF INTERACTIVE SYSTEM
IS INTERNET
• A MODEL OF BROADBAND
STREAMING ARE TV, RADIO
• MULTIMEDIA SYSTEMS WOULD
BE SOMETHING LIKE INTEGRATION
OF THEM BOTH
• SOMETHING LIKE BECAUSE WE
CAN NOT PREDICT IN DETAIL WHAT
IT MIGHT BE
MULTIMEDIA SYSTEMS
IREK DEFEE
• COMMUNICATION SERVICE TYPES
- BROADCAST: SINGLE SOURCE,
MANY USERS – LIKE TV,
1-WAY DISTRIBUTION
- MULTICAST: LIKE BROADCAST BUT
USERS SIGN TO A SESSION, ONE
WAY CONNECTION
- RETRIEVAL, UNICAST –
ASYMMETRIC CONNECTION, USERS
RECEIVE MUCH MORE THAN SEND,
E.G. WWW
MULTIMEDIA SYSTEMS
IREK DEFEE
• COMMUNICATIVE SERVICE –
SYMMETRIC 2-WAY CONNECTION
LIKE TELEPHONE
IN MULTIMEDIA SYSTEMS WE ARE
MOSTLY INTERESTED IN THE
RETRIEVAL AND COMMUNICATIVE
SERVICES WITH HIGH BIT RATE
MULTIMEDIA SYSTEMS
IREK DEFEE
• STREAMING
STREAMING MEANS DATA ARE SEND
SYNCHRONOUSLY IN TIME AND WILL
REACH USER PRESERVING THEIR TIME
ORGANIZATION. NO DATA LOSS OR
DELAYS ARE ALLOWED.
EXAMPLE: TELEPHONE NETWORK
IN MULTIMEDIA STREAMING IS
ABSOLUTELY NECESSARY
MULTIMEDIA SYSTEMS
IREK DEFEE
• THIS IS VERY MUCH RELATED TO THE
PROBLEM OF NETWORK RESOURCE
MANAGEMENT AND
CONTROL
• THERE ARE TWO TYPES OF
NETWORKS
- CONNECTION-ORIENTED
- CONNECTIONLESS
- IN CONNECTION-ORIENTED
NETWORK
RESOURCES ARE ALLOCATED BEFORE
MULTIMEDIA SYSTEMS
IREK DEFEE
THE DATA TRANSFER IS STARTED.
IN CONNECTIONLESS NETWORK
RESERVATION IS NOT DONE: DATA
MAY BE TRANSFERRED WITHOUT ANY
GUARANTEE OF RESOURCE
ALLOCATION, OR WITH SOME
GUARANTEE ONLY.
MULTIMEDIA SYSTEMS
IREK DEFEE
• HOW STREAMING CAN BE ENSURED
IN NETWORKING?
THERE ARE THREE BASIC TYPES
OF NETWORKING
- CIRCUIT SWITCHING CONNECTION IS ’SWITCHED’
BEFORE SENDING DATA (”PHONE”)
- PACKET SWITCHING – DATA ARE
ORGANIZED IN PACKETS. EACH
PACKET CARRIES ADDRESSES
MULTIMEDIA SYSTEMS
IREK DEFEE
- CELL SWITCHING – BETWEEN
CIRCUIT AND PACKET SWITCHING
”PACKETS ARE SWITCHED”
(TECHNOLOGY CALLED ATM –
ASYNCHRONOUS TRANSFER MODE)
ALL THESE SYSTEMS HAVE VARIOUS
POSIIVE AND NEGATIVE ASPECTS
FOR MULTIMEDIA
MULTIMEDIA SYSTEMS
IREK DEFEE
IN CIRCUIT SWITCHING NETWORK
RESOURCES ARE FULLY RESERVED FOR
STREAM TRANSMISSION
IN PACKET SWITCHING PACKETS ARE
FLOWING IN LITTLE CONTROLLED WAY
MAKING STREAMING PROBLEMATIC
IN CIRCUIT SWITCHING INTERACTIVITY IS
SLOW (TIME FOR CONNECTION NEEDED)
IN PACKET SWITCHING INTERACTIVITY IS
FAST
MULTIMEDIA SYSTEMS
IREK DEFEE
• EXAMPLES:
- TELEPHONE SYSTEM – SWITCHING
(connection establishment) IS DONE IN
EXCHANGES, SWITCHBOARDS OR
SWITCHING CENTERS
- INTERNET - PACKETS ARE DIRECTED
IN ROUTERS WHICH READ THEIR
ADDRESSES AND DIRECT THEM
The difference is that only circuit switching
fully guarantees streaming but it is expensive
MULTIMEDIA SYSTEMS
IREK DEFEE
• ADVANTAGES OF CIRCUIT
SWITCHING:
- GUARANTEED DATA DELIVERY
- STREAMING AND TIMING OF DATA
NO PROBLEM
- DISADAVANTAGES:
- CIRCUIT ESTABLISHMENT (”call”)
TAKES TIME
- EXPENSIVE INFRASTRUCTURE
MULTIMEDIA SYSTEMS
IREK DEFEE
• PACKET SWITCHING
EXAMPLE: INTERNET
DATA ARE ORGANIZED IN PACKETS
PACKETS CARRY HEADERS WITH
SOURCE AND DESTINATION ADDRESS
PACKETS ARE ROUTED IN THE
NETWORK BASING ON THE
ADDRESSESS
MULTIMEDIA SYSTEMS
IREK DEFEE
• ADVANTAGES OF PACKET SWITCHING
- SIMPLICITY, EVEN A PC CAN BE ROUTER
- PACKETS CAN BE ROUTED
IMMEDIATELY
- PACKETS CAN BE SEND USING
DIFFERENT ROUTES, OR STORED
EASY ADAPTATION TO THE AMOUNT OF
TRAFFIC
- IDEAL FOR TIME NON-CRITICAL DATA
MULTIMEDIA SYSTEMS
IREK DEFEE
• DISADVANTAGES:
- NO GUARANTEE FOR
PACKET DELIVERY
- NO GUARANTEE FOR RESOURCES
NO GUARANTEE FOR PACKET
DELIVER ON-TIME, NO STREAMING
MULTIMEDIA SYSTEMS
IREK DEFEE
• EXAMPLE – WWW. WE GET INSTANT
RESPONSE FROM THE PS INTERNET
IF THE INTERNET WOULD BE CS, WE
WOULD NEED TO WAIT FOR
CONNECTION ESTABLISHMENT.
BUT THERE IS NO RESOURCE
RESERVATION FOR STREAMS IN PS
A SOLUTION FOR THIS PROBLEM
COULD BE INTELLIGENT PACKET
SWITCHING AND NETWORKING
MULTIMEDIA SYSTEMS
IREK DEFEE
• IN INTELLIGENT PS PACKETS HAVE
PRIORITIES ASSIGNED, MULTIMEDIA
PACKETS GET HIGH PRIORITY SO
THEY WILL NOT BE DISTURBED BY
OTHER PACKETS
IF THE NETWORK WOULD HAVE
ENOUGH CAPACITY, MULTIMEDIA
PACKETS WILL FLOW IN STREAMS
(if there is not enough capacity, no help)
MULTIMEDIA SYSTEMS
IREK DEFEE
• THUS PACKET SWITCHING CAN BE IN
PRINCIPLE BETTER BECAUSE OF
STREAMING AND FAST
INTERACTIVITY
• CURRENT NETWORKING IS THUS
EVOLVING IN THIS DIRECTION
(INTERNET IS BECOMING VERY
POWERFUL)
MULTIMEDIA SYSTEMS
IREK DEFEE
CELL SWITCHING OVERVIEW
• THIS TECHNOLOGY IS REALIZED IN
ONE PARTICULAR FORM CALLED
• ATM – ASYNCHRONOUS TRANSFER
• MODE
IT IS A COMBINATION OF CIRCUIT
SWITCHING AND PACKET SWITCHING
PACKETS HAVE CONSTANT, VERY
SHORT LENGTH –
48DATA+5HEADER=53BYTES/PACKET
MULTIMEDIA SYSTEMS
IREK DEFEE
•
ATM ENABLES COMBINATION OF
CIRCUIT AND PACKET SWITCHING
• BASIC CONCEPTS ARE VIRTUAL
CHANNEL AND VIRTUAL PATH
H
PAYL.
H
PAYL.
H
FLOW OF ATM CELLS
H – HEADER 5 BYTES
PAYLOAD 48 BYTES
MULTIMEDIA SYSTEMS
IREK DEFEE
PAYL.
H
• HEADERS ESTABLISH WHERE THE
PACKETS ARE TRANSMITTED
STRUCTURE OF ATM CELL HEADER:
GFC
VPI
4
8
VCI
16
GFC - GENERIC FLOW CONTROL
VPI - VIRTUAL PATH IDENTIFIER
VCI - VIRTUAL CIRCUIT IDENTIFIER
PT - PAYLOAD TYPE
CL - CALL LOSS PRIORITY
HEC – HEADER ERROR
CHECK
MULTIMEDIA
SYSTEMS
IREK DEFEE
PT CL HEC
3
1
8
bits
• FOR MULTIMEDIA DATA
ONE VP COULD BE ALLOCATED
AND SEVERAL VC FOR E.G.
VC=1 VIDEO
VP=5 VC=2 AUDIO
VC=3 TEXT
MULTIMEDIA SYSTEMS
IREK DEFEE
• IN THE ATM NETWORK THERE ARE
SWITCHES WHICH DIRECT PACKETS
ACCORDING TO THEIR VP AND VC
VP=1
VC=5
SWITCH 1
VP=6
VC=13
SWITCH 2
VP=4
VC=2
VP AND VC BETWEEN THE SWITCHES (THERE
CAN BE MANY OF THEM HAS TO BE ASSIGNED
TO ENABLE END-TO-END TRANSMISSION
MULTIMEDIA SYSTEMS
IREK DEFEE
• HOW THE ASSIGNMENT IS DONE?
IT CAN BE DONE BY HAND
OR IT CAN BE DONE
AUTOMATICALLY USING SPECIAL
SYSTEM FOR MAPPING END POINT
NUMBERS TO VP AND VC OF
INTERMEDIATE SWITCHES
MULTIMEDIA SYSTEMS
IREK DEFEE
• IN ATM THIS IS SOLVED IN A WAY
WHICH IS SIMILAR TO TELEPHONE
NETWORKS
IT IS POSSIBLE TO ESTABLISH
CONNECTION BETWEEN ANY
END-POINTS BECAUSE THEY HAVE
UNIQUE NUMBERS
MULTIMEDIA SYSTEMS
IREK DEFEE
• THE ATM CALLS CAN SPECIFY
BANDWIDTH AND OTHER
PARAMTERS
THERE ARE SEVERAL CLASSES OF
SERVICES
AAL – ATM ADAPTATION LAYER
AAL1- CONSTANT BITRATE WITH
TIMING
MULTIMEDIA SYSTEMS
IREK DEFEE
• AAL2 – VARIABLE BIT RATE WITH
TIMING
• AAL3/4- VARIABLE BIT RATE WITH
NO TIMING
• AAL5 VARIABLE BIT RATE WITH NO
TIMING, CONNECTIONLESS
• FOR THESE AAL’S IT IS SPECIFIED
HOW DIFFERENT PROTOCOLS ARE
MAPPED ON THEM
MULTIMEDIA SYSTEMS
IREK DEFEE
• FOR EXAMPLE MPEG-2 TRANSPORT
STREAM IS MAPPED INTO
2 X188 MPEG-2 PACKETS -> 8 ATM
CELLS
AAL2 IS IDEAL FOR MULTIMEDIA
BUT IT IS DIFFICULT TO IMPLEMENT
(TIMING)
LARGE ATM NETWORKS ONLY
RARELY ARE IMPLEMENTED
MULTIMEDIA SYSTEMS
IREK DEFEE
• BUT IN NEW THIRD GENERATION
WIRELESS NETWORKS
- WIRELESS LAN
- WIRELESS CELLULAR
ATM AND AAL2 (WIRELESS) IS
SPECIFIED AND IMPLEMENTED
THUS, THEY MAY OPERATE IN THE
ATM MODE
MULTIMEDIA SYSTEMS
IREK DEFEE
PACKET SWITCHING – IP INTERNET
IN STANDARD PACKET SWITCHING
THERE IS NO NETWORK RESOURCE
RESERVATION BEFORE THE
PACKETS ARE SEND.
THE BASIC METHOD OF
TRANSFERRING PACKETS ON THE
INTERNET IS THE
UDP – USER DATAGRAM PROTOCOL
MULTIMEDIA SYSTEMS
IREK DEFEE
IPv4 Header Structure
Ip version Header_length
Tos
Identification
TTL
Total length
Flags and fragmentation
Protocol
Header check sum
32 bit Source Ip address
32 bit Destination address
Options(if any)
Data
MULTIMEDIA SYSTEMS
IREK DEFEE
• UDP/IP PACKETS ARE ROUTED
ACCORDING TO THEIR ADRESSES,
ONCE SENT, THEY ARE
UNCONTROLLED
THIS CAN BE VERY UNRELIABLE
BUT ON RELIABLE AND HIGH
BANDWIDTH LINKS IT CAN BE
SUFFICIENT FOR MULTIMEDIA
MULTIMEDIA SYSTEMS
IREK DEFEE
• THE TCP/IP PROTOCOL IS USED TO
PROVIDE INFORMATION THAT
PACKETS REACHED THEIR
DESTINATION, THERE IS
CONFIRMATION SEND ABOUT THEIR
RECEPTION
• IF THERE IS NO CONFIRMATION,
PACKETS ARE RESEND
• TCP/IP IS GOOD FOR NON-TIME
CRITICAL DATA (EMAIL)
MULTIMEDIA SYSTEMS
IREK DEFEE
IP NETWORKING FOR
MULTIMEDIA
• FOR MULTIMEDIA DATA PACKET
SWITCHING INTERNET CAN BE VERY
BAD
• ON THE OTHER HAND INTERNET IS
CHEAP, AVAILABLE AND
UBIQUITOUS
MULTIMEDIA SYSTEMS
IREK DEFEE
FOR SOLVING PROBLEMS WITH
MULTIMEDIA DATA OVER IP PACKET
NETWORK, THERE ARE THREE
APPROACHES:
- INTRODUCING RESOURCE
RESERVATION, PROTOCOL CALLED
RSVP – RESOURCE RESERVATION
PROTOCOL AIMS TO DO IT
- INTRODUCE PACKET PRIORITY
LABELLING (DONE IN ROUTERS)
MULTIMEDIA SYSTEMS
IREK DEFEE
• BOTH WOULD REQUIRE
IMPLEMENTATION IN ROUTERS
AND ARE COMPLICATED
• THIRD APPROACH - SIMPLER
- TRYING TO FOLLOW HOW PACKETS
ARE FLOWING THROUGH THE
NETWORK AND SEND INFORMATION
TO THE SENDING SIDE
MULTIMEDIA SYSTEMS
IREK DEFEE
• REAL-TIME PROTOCOL RTP
IN THIS PROTOCOL EACH PACKET
GETS TIME STAMP AND NUMBER
WHEN IT IS SEND. ON THE RECEIVING
SIDE IT IS POSSIBLE TO DETECT IF
PACKETS ARE LOST AND WHAT ARE
THE PACKETS DELAYS. THECOMPLETE
PACKET LOOKS LIKE THIS
MULTIMEDIA SYSTEMS
IREK DEFEE
• THERE IS A PROTOCOL CALLED
RTCP – REAL TIME CONTROL
PROTOCOL WHICH COLLECTS
STATISTICAL INFORMATION ABOUT
PACKET DELAYS AND LOSS AND
SENDS IT TO THE SENDING SIDE
WHICH MAY REACT IN SOME WAY
MULTIMEDIA SYSTEMS
IREK DEFEE
• THE RTP/RTCP COMBINATION IS
SIMPLE AND REALISTIC FOR THE
CURRENT INTERNET:
- DOES NOT CHANGE ANYTHING
IN THE SYSTEM
- PROVIDES CONCISE REPORTING
- SENDING SIDE MAY ESTIMATE
QUALITY OF CONNECTION AND
REACT E.G. BY REDUCING BITRATE
MULTIMEDIA SYSTEMS
IREK DEFEE
RTP INTRODUCTION
provides end-to-end network transport functions
suitable for applications transmitting real-time data,
such as audio or video.
does not address resource reservation and does not
guarantee quality-of-service for real-time services.
independent of the underlying transport and network
layers.
RTP packet consists fixed RTP header, a possibly
empty list of contributing sources and the payload data
MULTIMEDIA SYSTEMS
IREK DEFEE
RTP header
V
P
X
CC
M
PTT
SEQUENCE NUMBER
TIMESTAMP
SYNCHRONIZATION SOURCE IDENTIFIER (SSRS)
CONTRIBUTING SOURCE IDENTIFIERS (CSRC)
...
MULTIMEDIA SYSTEMS
IREK DEFEE
RTP header
• Version (2 bits) identifies the version of RTP
• Padding (1 bit) packet contains one or more additional
padding octets
• Extension (1 bit) the fixed header must be followed by one
header extension
• CSRC Count (4 bits) contains the number of CSRC
identifiers that follow the fixed header.
V
P
X
CC
MULTIMEDIA SYSTEMS
IREK DEFEE
RTP header
• Marker (1 bit) the packet contains marker bits
• Payload Type (7 bits) identifies the format of the RTP payload
• Sequence number (16 bits) increments by one for each RTP
data packet sent, and may be used by the receiver to detect
packet loss and to restore packet sequence.
M
PT
MULTIMEDIA SYSTEMS
IREK DEFEE
SEQUENCE NUMBER
RTP header
• Timestamp (32 bits) reflects the sampling instant of the
first octet in the RTP data packet
• SSRC (32 bits) field identifies the synchronization source.
• CSRC list (0 to 15 items, 32 bits each) identifies
contributing sources for the payload. The number of
identifiers is given by the CC field.
TIMESTAMP
SYNCHRONIZATION SOURCE IDENTIFIER (SSRS)
CONTRIBUTING SOURCE IDENTIFIERS (CSRC)
...
MULTIMEDIA SYSTEMS
IREK DEFEE
RTP Applications
• Designed for multiparticipant multimedia
conferences
– Storage of continuos data
– Interactive distributed simulation
– Control and measurement applications
MULTIMEDIA SYSTEMS
IREK DEFEE
Implementation of RTP
• Typically integrated into application processing
• No direct coupling at the RTP level between
different media
– For example, audio and video are in different sessions
– Enables the choice of receiving only audio signal
• With UDP, RTP uses an even port number and the
corresponding RTCP stream uses the next higher
odd port number
MULTIMEDIA SYSTEMS
IREK DEFEE
RTP/UDP/IP
MULTIMEDIA SYSTEMS
IREK DEFEE
Application dependent protocol
• Complete implementation of RTP for a
particular application requires a profile and
a payload format (e.g., media encodings)
specification
MULTIMEDIA SYSTEMS
IREK DEFEE
Profile
• Defines
– Mapping of a set of payload formats to payload type
values
– Extensions into the RTP and RTCP headers
• RTP Profile for
– Audio and Video Conferences with Minimal Control
(IETF 1890)
– Source Authentication and Non-Repudiation of Audio
and Video Conferences (Internet Draft)
– Interactive media (proposed in journal paper)
MULTIMEDIA SYSTEMS
IREK DEFEE
Payload Types
• RFCs
– H.261 and H.263 video streams; Redundant Audio
Data; MPEG1/MPEG2 Video; Bundled MPEG; JPEGcompressed Video; Generic Forward Error Correction
– MPEG-4 IN PREPARATION
• Internet Drafts:
– MPEG-4 Streams; Interleaved Media; DV Format
Video; MPEG-2 AAC Streams; DTMF Digits,
Telephony Signals; Text Conversation; Real-Time
Pointers; DV Audio; MP3 Audio, Shared Multicast
Virtual Worlds (SMVW)
• Also proprietary streams can be used
MULTIMEDIA SYSTEMS
IREK DEFEE
RTCP
• RTP control protocol (RTCP)
• Periodic transmission of control packets to
all participants in session
MULTIMEDIA SYSTEMS
IREK DEFEE
Features of RTCP
• Feedback on the quality of data distribution
– Evaluate whether problems are local or global
– Transmission rate can be changed according to network
situation
• Persistent transport-level identifier for an RTP
source called the canonical name or CNAME
– Keep track of participants
– Associate multiple data streams from a given
participant in a set of related RTP sessions
MULTIMEDIA SYSTEMS
IREK DEFEE
Features of RTCP (2)
• Observe the number of participants
• Scaling of control information transmission rate in
order for RTP to suit also to a large number of
participants
– Maximum of 5 % of session bandwidth allocated to
RTCP
• Transport minimal session control information
– For example display participant identification in the
user interface
MULTIMEDIA SYSTEMS
IREK DEFEE
RTCP Packet Format
• SR: Sender report, for transmission and reception
statistics from participants that are active senders
• RR: Receiver report, for reception statistics from
participants that are not active senders
• SDES: Source description items
– describes participant using ASCII text
• BYE: Indicates end of participation
• APP: Application specific functions
MULTIMEDIA SYSTEMS
IREK DEFEE
Compound RTCP Packet
• Typically multiple RTCP packets are sent
together as a compound RTCP packet
• In the following format:
–
–
–
–
Encryption prefix if packet is encrypted
SR or RR
SDES
Other RTCP packet types
MULTIMEDIA SYSTEMS
IREK DEFEE
Sender report (SR)
• First section of SR
– Version, padding, reception report count,
packet type, and SSRC
– Similar to the beginning of RTP header
MULTIMEDIA SYSTEMS
IREK DEFEE
Second section of SR
• NTP timestamp: 64 bits
– Real time when report was sent
– Used in combination with timestamps returned in
reception reports to measure round-trip propagation to
those receivers
• RTP timestamp: 32 bits
– Same time as the NTP timestamp but with the same
random offset as the RTP timestamps in data packets
– Used by media- independent receivers to estimate the
nominal RTP clock frequency
MULTIMEDIA SYSTEMS
IREK DEFEE
Second section of SR (2)
• Sender's packet count: 32 bits
– Total number of RTP data packets transmitted
by the sender since starting transmission
• Sender's octet count: 32 bits
– Total number of payload octets transmitted in
RTP data packets by the sender since starting
transmission
– Used to estimate the average payload data rate
MULTIMEDIA SYSTEMS
IREK DEFEE
Third section of SR
• SSRC_n (source identifier): 32 bits
– SSRC identifier of the source to which the information
in this reception report block pertains
• Fraction lost: 8 bits
– Fraction of RTP data packets from source SSRC_n lost
since the previous SR or RR packet was sent
• Cumulative number of packets lost: 24 bits
– Total number of RTP data packets from source
SSRC_n that have been lost since the beginning of
reception
MULTIMEDIA SYSTEMS
IREK DEFEE
Third section of SR (2)
• Extended highest sequence number received: 32
bits
– Low 16 bits contain the highest sequence number
received in an RTP data packet from source SSRC_n
– Most significant 16 bits extend sequence number with
the corresponding count of sequence number cycles
• Interarrival jitter: 32 bits
– Estimate of the statistical variance of the RTP data
packet interarrival time
– J=J+(|D(i-1,i)|-J)/16
MULTIMEDIA SYSTEMS
IREK DEFEE
Third section of SR (3)
• Last SR timestamp (LSR): 32 bits
– NTP timestamp received as part of the most
recent RTCP sender report (SR) packet from
source SSRC_n
• Delay since last SR (DLSR): 32 bits
– Delay between receiving the last SR packet
from source SSRC_n and sending this reception
report block
MULTIMEDIA SYSTEMS
IREK DEFEE
Sender Report
• Profile may define extensions to SR
• Format of the receiver report (RR) packet is
the same as that of the SR packet except
sender information is omitted
MULTIMEDIA SYSTEMS
IREK DEFEE
Why RRs and SRs?
• Sender may modify its transmission rate based on
the feedback
• Receivers can determine whether problems are
local, regional or global
• Network managers may use profile-independent
monitors that receive only RTCP packets to
evaluate the performance of their networks for
multicast distribution
• Cumulative counts are used to make
measurements over both short and long time
periods, and to provide resilience against the loss
of a report
MULTIMEDIA SYSTEMS
IREK DEFEE
Why RRs and SRs? (2)
• Since timestamp is independent of the clock rate
for the data encoding, it is possible to implement
encoding- and profile-independent quality
monitors
• A third-party monitor can calculate the average
payload data rate and the average packet rate over
an interval without receiving the data
• Jitter measure may indicate congestion before it
leads to packet loss
– necessary to analyze a number of reports from one
receiver over time
or fromSYSTEMS
multiple receivers
MULTIMEDIA
IREK DEFEE
SDES: Source description RTCP
packet
• CNAME: Canonical end-point identifier SDES
item
– CNAME should be fixed for a given participant in
order to provide a binding across multiple media tools
• NAME: User name, EMAIL,PHONE,LOC:
Geographic user location, TOOL: Application or
tool name
• NOTE: Notice/status SDES item
– Intended for transient messages describing the current
state, e.g., "on the phone, can't talk"
• PRIV: Private extensions
SDES item
MULTIMEDIA SYSTEMS
IREK DEFEE
RTP Translators and Mixers
• Firewalls and low bandwidth connections
• Connects transport-level "clouds"
– Each cloud is defined by a common network and
transport protocol, multicast address or pair of unicast
addresses, and transport level destination port
• May add or removes encryption
• May change encoding of data or underlying
protocols
MULTIMEDIA SYSTEMS
IREK DEFEE
Translators
• Translator passes through the data streams from
different sources separately
• Forwards RTP packets with their SSRC identifier
intact
• Receivers cannot detect the presence of a
translator
• Replicator from multicast to unicast
• Application-level filter in firewalls
• May, for example, filter non-CNAME SDES
information if bandwidth is limited
MULTIMEDIA SYSTEMS
IREK DEFEE
Mixer
• Intermediate system that receives streams of RTP
packets from sources, combines them and then
forwards the combined stream
• Makes timing adjustments among the streams and
generates its own timing for the combined stream
• Packets marked with the mixer's own SSRC
identifier
• Generates its own reception reports for sources in
each cloud and sends them out only to the same
cloud
MULTIMEDIA SYSTEMS
IREK DEFEE
Pros and cons of mixers
• Output bandwidth is limited to that of one
source even when multiple sources are
active on the input side
• Receivers on the output side don't have any
control over which sources are passed
through or muted
• Receivers can't do inter-media
synchronization of the original streams
MULTIMEDIA SYSTEMS
IREK DEFEE
Loop detection
• Translators and mixers may create transmission
loops
• A loop of data packets to a multicast destination
can cause severe network flooding.
– All mixers and translators are required to implement a
loop detection algorithm
• SSRCs are kept globally unique for each RTP
session, they can be used to detect loops that may
be introduced by mixers or translator
MULTIMEDIA SYSTEMS
IREK DEFEE
RTSP (The Real Time Streaming Protocol)
MULTIMEDIA SYSTEMS
IREK DEFEE
RTSP introduction
• application level protocol for control over the delivery of
data with real-time properties.
• provides an extensible framework to enable controlled, on
demand delivery of real-time data, such as audio and
video.
• protocol provides means for choosing delivery channels
such as UPD, multicast UDP and TCP
MULTIMEDIA SYSTEMS
IREK DEFEE
RTSP introduction
• provides means for choosing delivery mechanisms based
on RTP
• does not depend on the mechanism used to carry
continuous media.
• similar in syntax and operation to HTTP
• control may occur on TCP connection while the data flow
via UDP.
MULTIMEDIA SYSTEMS
IREK DEFEE
RTSP request message
•
•
•
•
•
•
•
•
•
•
•
ANNOUNCE
DESCRIBE
GET_PARAMETER
OPTIONS
PAUSE
PLAY
RECORD
REDIRECT
SETUP
SET_PARAMETER
TEARDOWN
MULTIMEDIA SYSTEMS
IREK DEFEE
RTSP request message (describe)
• DESCRIBE method retrieves the description or a
presentation or media object identified request URL from
the server.
–
–
–
DESCRIBE rtsp://server.example.com/foo RTSP/1.0
Cseg: 312
Accept: application/sdp, application/rtsl,
application/mheg
• response
–
–
–
–
–
–
RTSP/1.0 200 OK
Cseg: 312
Date: 23 Jan 1999 15:35:06 GMT
Content-Type: application/sdp
Content-Length: 376
SDP part
MULTIMEDIA SYSTEMS
IREK DEFEE
RTSP request message (announce)
• two purposes:
• 1. When sent from client to server, ANNOUNCE posts the
description of a presentation or media object by the request
URL to a server.
• 2. When sent from server to client, ANNOUNCE updates the
session description in real-time.
• The whole presentation description should be sent, rather than just the
additional components.
MULTIMEDIA SYSTEMS
IREK DEFEE
RTSP request message (options)
• query the server about options it may or may not support
•
•
•
•
OPTIONS * RTSP/1.0
Cseq: 1
Require: implicit-play
Proxy-Require: gzipped-message
• Response could be
•
•
•
RTSP/1.0 200 OK
cseg: 1
Public: DESCRIBE, SETUP, TEARDOWN, PLAY, PAUSE
• or
•
•
•
RTSP/1.0 551 Option not supported
cseq:1
unsupported: implicit-play
MULTIMEDIA SYSTEMS
IREK DEFEE
RTSP request message (setup)
• specifies the transport mechanism to be used for the
streamed media
• change transport parameter
•
•
•
•
SETUP rtsp://example.com/foo RTSP/1.0
Cseg: 302
Session: 47112344
Transport: RTP/AVP;unicast;client_port=…
• If the change is not allowed response is
•
RTSP/1.0 455 Method Not Valid In This State
•
Cseq: 302
MULTIMEDIA SYSTEMS
IREK DEFEE
RTSP request message (play)
• tells the server to start sending data
• PLAY request may be queued
•
•
•
•
PLAY rtsp://example.com/foo RTSP/1.0
Cseg: 832
Session: 47112344
Range: npt=10-15
•
•
•
•
PLAY rtsp://example.com/foo RTSP/1.0
Cseg: 833
Session: 47112344
Range: npt=20-25
MULTIMEDIA SYSTEMS
IREK DEFEE
RTSP request message
• The PAUSE request causes the stream delivery to be
interrupted temporarily.
• The TEARDOWN request stops the stream delivery
freeing the resource associated with it.
• The REDIRECT request informs the client that it must
connect to another server location.
MULTIMEDIA SYSTEMS
IREK DEFEE
RTSP request message
• The GET_PARAMETER request retrieves the value of a
parameter of presentation or stream.
• The SET_PARAMETER requests to set the value of a
parameter for a presentation or stream.
• The RECORD request initiates recording a range of media
data according to the presentation description.
MULTIMEDIA SYSTEMS
IREK DEFEE
RTSP example
DESCRIBE
PLAY
200 OK
200 OK
SETUP (video)
PAUSE
200 OK
SETUP (audio)
200 OK
460 (error
message)
PAUSE
200 OK
MULTIMEDIA SYSTEMS
IREK DEFEE
HIGHER LEVEL PROTOCOLS
 DEVELOPED BY MMUSIC GROUP OF IETF
INCLUDE SDP AND SIP PROTOCOLS
IN ADDITION TO RTP/RTCP/RTSP
MULTIMEDIA SYSTEMS
IREK DEFEE
Introduction
.
• MMUSIC (Multiparty Multimedia Session
Control)
• Internet standard track protocols to support
Internet teleconferencing sessions.
MULTIMEDIA SYSTEMS
IREK DEFEE
SIP (Session Initiation Protocol)
• application-layer control protocol for creating,
modifying and terminating sessions with one or
more participants
• Internet multimedia conference, Internet telephone
calls and multimedia distribution.
MULTIMEDIA SYSTEMS
IREK DEFEE
SDP (Session Description Protocol)
• describing multimedia sessions
• session announcement, session invitation, and
other forms of multimedia session initiation.
MULTIMEDIA SYSTEMS
IREK DEFEE
SIP introduction
• supports name mapping and redirection services
• allows implementation of ISDN and Intelligent
Network telephony subscriber service.
• Internet telephony gateways that connect Public
Switched Telephone Network (PSTN) parties can
also use SIP to set up calls.
MULTIMEDIA SYSTEMS
IREK DEFEE
SIP introduction cont.
• an application-layer control protocol for creating,
modifying and terminating sessions with one or
more participants.
• invite both persons and ”robots”, such as a media
storage service.
• can be used to initiate sessions as well as invite
members to sessions that have been advertised and
established by other means.
MULTIMEDIA SYSTEMS
IREK DEFEE
SIP introduction
• does not offer conference control services such as
floor control or voting
• doesn’t prescribe how a conference is to be
managed
• can be used to introduce conference control
protocol
• does not reserve resources
• can convey to the invited system the information
necessary to do this.
MULTIMEDIA SYSTEMS
IREK DEFEE
SIP Addressing
• The SIP URL similar to a mailto or telnet URL i.e.
user@host
• User part is a user name or a telephone number
• the host part is either a domain name or a numeric
network address
• Sip:jorma.ollila@nokia.com
MULTIMEDIA SYSTEMS
IREK DEFEE
SIP entities
• SIP client is an application program that is able to
send SIP methods (invite).
• SIP server is an application that accepts SIP
methods and replies with status code.
• SIP proxy is an intermediary program that acts as
both a server and client for the purpose of making
requests on behalf of the other client
MULTIMEDIA SYSTEMS
IREK DEFEE
SIP entities
• SIP registrar is a server that accepts REGISTER
requests.
• SIP redirect server is a server that accepts a SIP
request, maps a address into zero or more new
addresses and returns these addresses to the client.
• Does not initiate or accepts call by itself.
• Location server is used by SIP request or proxy
server to obtain information about a callee’s
possible locations
MULTIMEDIA SYSTEMS
IREK DEFEE
SIP methods
• INVITE (call user)
• ACK (confirm
connection)
• CANCEL (cancel
operation)
• BYE (disconnect)
• REGISTER (sign up
with server)
• OPTIONS (capability
query)
MULTIMEDIA SYSTEMS
IREK DEFEE
SIP invite
• The INVITE method indicates that the user or
service is being invited to participate in a session.
• message body contains a description of the session
• type of media it is able to receive
• media it is willing to send
• parameters such as network destination.
MULTIMEDIA SYSTEMS
IREK DEFEE
SIP invite
•
•
•
•
•
•
•
•
•
INVITE sip:watson@boston.bell-tel.com SIP/2.0
Via: SIP/2.0/UDP kton.bell-tel.com
From: A. Bell <sip:a.g.bell@bell-tel.com>
To: T. Watson <sip:watson@boston.bell-tel.com>
Call-ID: 3298420296@kton.bell-tel.com
Cseq: 1 INVITE
Subject: Mr. Watson, come here.
Content-type: application/sdp
Content-length: …
•
•
•
•
•
•
(SDP)
v=0
o=bell 53655765 2353687637 IN IP4 128.3.4.5
s=Mr. Watson, come here.
c=IN IP4 kton.bell-tel.com
m=audio 3456 RTP/AVP 0 3 4 5
MULTIMEDIA SYSTEMS
IREK DEFEE
SIP ack
• confirms that the client has received a final
response to an INVITE request
• ACK is used only with INVITE request.
• ACK request may contain final session
descriptions to be used by callee.
• If the ACK message body is empty, the callee uses
the session description in the INVITE request.
MULTIMEDIA SYSTEMS
IREK DEFEE
SIP ack
• ACK sip:watson@boston.bell-tel.com SIP/2.0
•
Via: SIP/2.0/UDP kton.bell-tel.com
•
From: A. Bell <sip:a.g.bell@bell-tel.com>
•
To: T. Watson <sip:watson@boston.belltel.com>
•
Call-ID: 3298420296@kton.bell-tel.com
•
Cseq: 1 ACK
MULTIMEDIA SYSTEMS
IREK DEFEE
SIP cancel, bye and options
• cancels a pending request with the same header
field values, but doesn’t affect a completed
request.
• release the call
• may be issued by either caller or callee.
• find out the capabilities of the potential callee
• without ”ringing” the designated address
MULTIMEDIA SYSTEMS
IREK DEFEE
SIP register
• register the address listed in the To
header field with the SIP server.
• REGISTER sip:bell-tel.com SIP/2.0
•
Via: SIP/2.0/UDP saturn.bell-tel.com
•
From: sip:watson@bell-tel.com
•
To: sip:watson@.bell-tel.com>
•
Call-ID: 70710@saturn.bell-tel.com
•
Cseq: 1 Register
•
Contact: <sip:watson@saturn.belltel.com:3890;transport=udp>
•
Expires: 7200
MULTIMEDIA SYSTEMS
IREK DEFEE
SIP status messages (responses)
• 1xx:Informational - request received, continuing
to process the request
• 2xx:Success - the action was successfully
received, understood and
accepted
• 3xx:Redirection - further action needs to be taken
in order to complete the request.
• 4xx:Client error - request contains bad syntax or
cannot be fulfilled at this server.
• 5xx:Server error - the server failed to fulfill an
apparently valid request.
• 6xx:Global failure - the request cannot be fulfilled
at any server.
MULTIMEDIA SYSTEMS
IREK DEFEE
SIP status messages (responses)
•
•
•
•
100 Trying
180 Ringing
181 Call is being forwarded
182 Queued
•
•
•
•
403 Forbidden
404 Not Found ...
480 Temporarily not available
….
•
200 OK
•
•
•
•
•
•
300 Multiple choices
301 Moved permanently
302 Moved Temporarily
303 See other
304 Use proxy
380 Alternative Service
•
•
•
•
•
•
500 Internal server error
501 Not implemented
502 Bad gateway
503 Service unavailable
504 Gateway time-out
505 SIP version not supported
•
•
•
400 Bad request
401 Unauthorized
402 Payment required
•
•
•
•
600 Busy everywhere
603 Decline
604 Does not exist anywhere
605 Not acceptable
MULTIMEDIA SYSTEMS
IREK DEFEE
SIP status messages (responses)
•
•
•
•
•
•
•
SIP/2.0 180 Ringing
Via: SIP/2.0/UDP kton.bell-tel.com
From: A. Bell <sip:a.g.bell@belltel.com>
To: <sip:watson@boston.belltel.com>
Call-ID: 3298420296@kton.belltel.com
Cseq: 1 INVITE
Content-length:0
•
•
•
•
•
•
•
•
•
•
•
•
•
•
SIP/2.0 200 OK
Via: SIP/2.0/UDP kton.belltel.com
From: A. Bell
<sip:a.g.bell@bell-tel.com>
To: <sip:watson@boston.belltel.com>
Call-ID: 3298420296@kton.belltel.com
Cseq: 1 INVITE
Contact: sip:watson@boston.belltel.com
Content-type: application/sdp
Content-length: …
v=0
o=bell 53655765 2353687637 IN
IP4 128.3.4.5
s=Mr. Watson, come here.
c=IN IP4 kton.bell-tel.com
m=audio 3456 RTP/AVP 0 3 4 5
MULTIMEDIA SYSTEMS
IREK DEFEE
SIP operation in proxy mode
pulver.com
nortel.com
user@nortel.com
jeff.pulver
Location Server
pulver@von1
Proxy server
1. INVITE sip:jeff.pulver@pulver.com
2. INVITE sip:pulver@von1 SIP/2.03. SIP/2.0 200 ok
SIP/2.0
From: sip:user@nortel.com
From: sip:pulver@von1
From: sip:user@nortel.com
4. SIP/2.0 100 OK
5. ACK sip:jeff.pulver@pulver.com SIP/2.0
6. ACK sip:pulver@von1 SIP/2.0
From: sip:jeff.pulver@pulver.com From: sip:user@nortel.com
From: sip:user@nortel.com
MULTIMEDIA SYSTEMS
IREK DEFEE
SIP operation in redirect mode
pulver.com
Location Server
Jeff.pulver
nortel.com
Pulver@von1
user@nortel.com
Redirect Server
1. INVITE sip:jeff.pulver@pulver.com 2. SIP/2.0 320 Moved temporarily
From: sip:user@nortel.com
Contact: sip:pulver@von1.pulver.com
3.
6. ACK
ACK sip:jeff.pulver@pulverr.com
sip:pulver@von1.pulver.com
5.
SIP/2.0
200
OK
From:
4. INVITE sip:pulver@von1.pulver.com
From: sip:user@nortel.com
sip:user@nortel.com
To:
user@nortel.com
From: user@nortel.com
MULTIMEDIA SYSTEMS
IREK DEFEE
SIP and H.323
• SIP
• 60+ pages
• Firewall - friendly
• SIP address = email
address
• Personal mobility, IN
services
• H.323
• 200+ pages
• complex, multiple
protocols
• yet another address
• Not (yet)
MULTIMEDIA SYSTEMS
IREK DEFEE
SDP introduction
• describing multimedia sessions for the purposes of
session announcement, session invitation, and
other forms of multimedia session initiation
• advertise multimedia conference addresses and
conference tool-specific information necessary to
participation.
• does not incorporate a transport protocol, and is
intended to use with other protocols like SIP and
RTSP.
MULTIMEDIA SYSTEMS
IREK DEFEE
SDP includes:
• Session name and purpose
• Time(s) the session is active (start, stop, repeat
time and so on
• The media comprising the session (type, transport
protocol, format and so on)
• Information to receive those media (addresses,
ports, formats and so on)
• Bandwidth information
• Contact information
MULTIMEDIA SYSTEMS
IREK DEFEE
SDP example
–
–
–
–
–
–
–
–
–
–
–
–
–
–
–
v=0
o=mhadley 2890844526 2890842807 IN IP4 126.16.64.4
s=SDP seminar
i=A seminar on the session description protocol
u=http://www.cs.ucl.ac.uk/staff/M.Hadley/dsp.03.ps
e=mjh@isi.edu (Mark Hadley)
p=+44 171 380 7777
c=IN IP4 224.2.17.12/127
b=CT:128
t=2873397496 2873404696
a=recvonly
m=audio 49170 RTP/AVP 0
m=video 51372 RTP/AVP 31
m=application 32461 udp wb
a=orient:portrait
• V,O,S,T & M tags are required
MULTIMEDIA SYSTEMS
IREK DEFEE
APPLICATION TO
MULTIMEDIA – MPEG-4
EXAMPLE
 MPEG-4 SYSTEM FITS AND REQUIRES
SOMETHING LIKE THE RTP/RTCP/RTP
AND SDP/SIP
 IN MPEG-4 GENERIC NETWORKING SCHEME
IS PROVIDED, ABOVE PROTOCOLS CAN BE
ADAPTED TO IT
 THIS IS ONGOING WORK
MULTIMEDIA SYSTEMS
IREK DEFEE
Streaming Media Formats
MULTIMEDIA SYSTEMS
IREK DEFEE
MPEG-4 Systems Principle
Interactive Scene Description
Scene Description Stream
STREAMS
WHICH
CAN BE
TRANSPORTED
OVER NETWORK
Object Descriptor Stream
Visual Stream
Visual Stream
Visual Stream
Audio Stream
...
MULTIMEDIA SYSTEMS
IREK DEFEE
Example of MPEG-4 scene
MULTIMEDIA SYSTEMS
IREK DEFEE
Object-based compression and delivery
MULTIMEDIA SYSTEMS
IREK DEFEE
MPEG-4: An integrated Multimedia
System
Decoding
N
e
t
w
o
r
k
TransMux
FlexMux
Primitive
AV Objects
Composition and
Rendering
...
...
...
Elementary
Streams
Scene Description
Information
Object Descriptor
MULTIMEDIA SYSTEMS
IREK DEFEE
MPEG-4 Interactive
Scene
Display and
Local User
Interaction
MPEG-4
NETWORKING
Display and
User
Interaction
Interactive Audiovisual
Scene
Composition and Rendering
...
Scene
Description
Information
Object
Descriptor
Upchannel
Information
AV Objects
Data
Elementary Streams
SL
SL
SL
SL
SL
SL
...
Elementary Stream Interface
SL
(PES)
MPEG-2
TS
FlexMux
(RTP)
UDP
IP
AAL2
ATM
Sync
Layer
DMIF Application Interface
SL-Packetized Streams
FlexMux
Compression
Layer
FlexMux
MP4
DAB
Mux
Multiplexed Streams
MULTIMEDIA SYSTEMS
Transmission/Storage
IREK DEFEE Medium
...
Delivery
Layer
Example of MPEG-4 over IP
MULTIMEDIA SYSTEMS
IREK DEFEE
Carriage of MPEG-4 contents over IP
Networks
•
MPEG-4 is a standard designated for the representation and delivery of
multimedia information over a variety of transport protocols;
•
MPEG-4 includes:
– Interactive scene management;
– Visual & Audio representations;
– Functional systems (multiplexing, synchronization);
– Object descriptor framework;
• Transfer method for MPEG-4 over IP:
– In context with specific IP packets;
– Multiplexed in MPEG-4 Flexmux (carried in IP packets);
– Multiplexed in MPEG-2 TS (carried in IP packets);
MULTIMEDIA SYSTEMS
IREK DEFEE
HTTP Transport Protocol
•
It is a very simple way to stream media files;
•
The wire format is the same as the file-format storage;
•
Arbitrary MPEG-4 Intermedia files cannot be streamed directly using HTTP,
because of the “loose” in ordering the objects, and possible cross-references
within the media file;
•
Live streaming for Intermedia files can also be supported using proprietary
methods;
MULTIMEDIA SYSTEMS
IREK DEFEE
Media Control Protocols
•
In order to enable full streaming systems, a media control protocol needs to
be defined to support the following features:
1.
Seeking:
– Forward;
– Rewind;
– Skip.
2.
Bandwidth Scalability
3.
Live Streaming
MULTIMEDIA SYSTEMS
IREK DEFEE
MPEG-4 Systems
•
In MPEG-4 Systems, the transport of streams is divided in 4 layers:
– Compression Layer: includes elementary (raw) media streams (audio, video,
etc.).
– Synchronization Layer (SL): Adds a header to each unit of an elementary
stream, which includes time stamps, reference to a clock elementary stream, and
identification of key frames (RandomAccessPoint). This is similar to the task of
RTP in IP networks. However, SL does not contain payload type (like RTP), and
does not contain the Elementary Stream. In addition, an SL packet does not
contain an indication of its length, so it must be framed by a lower-level protocol
such as FlexMux or RTP.
– FlexMux Layer: Groups elementary streams according to common attributes, such
a QoS requirements. This is very simple multiplexing protocol, but also very low
overhead.
– TransxMux Layer: This is the actual transport protocol, such as RTP/UDP, MPEG2, etc. MPEG-4 does not define its own transport protocol, but assumes the
application relies on an existing transport protocol. The FlexMux Layer is optional,
but the Synchronization Layer is always present.
MULTIMEDIA SYSTEMS
IREK DEFEE
TCP/IP & SL-packet over UDP/IP
• TCP/IP:
– Not suitable for real-time transfers (high delays and causes jitter, it was
created for reliable transmission over timely delivery);
– Does not support multicast;
– Congestion control mechanism not suitable for AV media;
• SL-packet over UDP/IP:
– SL provides: Timing & Sequence numbering;
– UDP provides: Multiplexing, Length field, Checksum service;
– SL+UDP may be used like a transport protocol for AV media;
MULTIMEDIA SYSTEMS
IREK DEFEE
Problems of SL/UDP/IP
•
No other media stream can be synchronized with MPEG-4 data carried
directly over UDP;
•
The dynamic scene and session control concepts cannot be extended to non
MPEG-4 data;
•
No defined technique for synchronizing MPEG-4 streams from different
servers;
•
Streams from different servers may collide (their sources may become
unresolvable at destination) in a multicast session;
•
Mechanisms need to be defined to protect sensitive MPEG-4 data (RTP
supports FEC);
•
A feedback channel must be defined for quality control;
MULTIMEDIA SYSTEMS
IREK DEFEE
RTP & RTCP
•
RTP = Real-time Transport Protocol;
•
RTCP = Real-time Transport Control Protocol;
•
A session consists of an RTP/RTCP pair of channels
•
Usually works over UDP/IP (can work with other protocols also);
•
RTP supports:
–
–
–
–
–
–
Multicasting;
Payload type identification;
Time stamping;
Sequence numbering;
Delivery monitoring;
From UDP (Multiplexing, Length field, Checksum service);
MULTIMEDIA SYSTEMS
IREK DEFEE
RTP
•
RTP Problems:
– It does not support the timely delivery of data or any other QoS guarantees;
– It does not guarantee delivery, so packets may be delivered out of order or get lost
(no mechanism to recover from packet loss);
•
Time Stamp (TS):
– Place incoming packet in correct timing order
– Initial values are picked randomly and independently for each RTP stream;
– Increase in time indicated by each packet;
•
Sequence Number (SN):
– Detect packet loss;
– Increase by one for each packet;
•
For video frame that is split into multiple RTP packets: they share the same
TS but use different SN;
MULTIMEDIA SYSTEMS
IREK DEFEE
RTP Characteristics
•
RTP can map only one source stream onto RTP session:
– Multiplexing causes problems;
– For scale coding, each layer must have its own RTP session and multicast group;
•
Within each RTP session, source can change its data format over time;
•
It allows FEC (Forward Error Correction);
•
Synchronize across different media streams;
•
Provide feedback on the quality of data using packet counts;
•
Identify and keep track of participants;
MULTIMEDIA SYSTEMS
IREK DEFEE
MPEG-4 over RTP/UDP/IP
•
Easiest is to wrap the MPEG-4 SL packet in an RTP packet:
– High overhead: two full headers;
– RTCP may not provide enough control for the MPEG-4 stream;
•
Several types of MPEG-4 payloads are being defined by the IETF for different
ESs;
MULTIMEDIA SYSTEMS
IREK DEFEE
RTP ES & SL payload restrictions
•
RTP packetization is not media aware:
– No unique scheme can be defined, need a payload definition per payload type (not
practical);
– This may require the definition of new session and scene description mechanisms
to deal with all the flows;
•
Common restrictions:
– RTP timestamp corresponds to composition time stamp (CTS) of MPEG-4 stream;
– Packets should be sent in decoding order;
– Streams can be synchronized using RTCP;
•
Map SL stream onto RTP session:
– SL header is optional;
•
Reduced SL headers does not include:
– Sequence number (mapped to RTP header);
– Composition Time Stamp (mapped to RTP header);
MULTIMEDIA SYSTEMS
IREK DEFEE
Streams & Media Control in MPEG-4
•
Multiple Strems inMPEG-4:
– Port consuming:
• Each AV contains one or more streams;
• Each stream needs one RTP session;
– Potential solution:
• Selective bundling of ESs-FlexMux -> define a multiplexed MPEG-4 payload (may have to
be defined for multiple payload types);
• Generic RTP multiplexing for use with MPEG-4, under development by IETF.
•
MPEG-4 Media Control:
– Remote interactivity: add or delete a stream, etc.
– Media control channel allows renegotiating in time the available network and
processing resources;
– Must have an efficient signaling protocol that can handle such messages;
MULTIMEDIA SYSTEMS
IREK DEFEE
Media Control Framework
•
To allow a client and one or more servers to exchange types of control
messages and also allow for peer to peer exchange between two or more
clients, the framework requires several components:
– A description of the stored or live presentation;
– A set of protocols that can provide proper services for the back channel message
delivery;
– A set of protocols that can allocate resources for the involved hosts and networks;
MULTIMEDIA SYSTEMS
IREK DEFEE
Components of media control (1)
• Presentation Description:
– The client needs to refer to the description of a presentation that
expresses the temporal and static properties of the presentation itself;
– Must include information about the media, location of the media, etc.
– Should provide multiple description instances of the same presentation so
that the client can specify a given combination that fits its
needs/capabilities – the client is the orchestrator of the presentation and
the server is participating;
MULTIMEDIA SYSTEMS
IREK DEFEE
Components of media control (2)
• Client and Server State Representations:
– Out of band signaling is used: the data streams and the control
information are carried over separate channels using different protocols,
each best suited to their needs and modes of operation;
– As the media streams may be modified by the end user, the server needs
to a state if the streams status for each client it is serving;
– The client has to keep track of all the participating streams;
MULTIMEDIA SYSTEMS
IREK DEFEE
Components of media control (3)
• Basic Media Control Messages:
– A multimedia system should have access to control messages ranging
from remote VCR functions such as stop, play, fast forward and fast
reverse, to messages generated in response to user actions to modify the
presentation of a given object stream such as add or remove or alter, etc.
– The basic control functionality relates to presentation and stream set-up:
play, stop, pause, teardown and recording;
MULTIMEDIA SYSTEMS
IREK DEFEE
Real Time Streaming Protocol (RTSP)
•
•
•
•
It is an application level protocol that provides an extensible framework to
enable controlled delivery of real-time data, such as audio & video;
It can be transported over UDP, TCP and is designated to work with RTP and
HTTP;
Provide support for bidirectional communications (frame level timing for
remote video editing);
RTSP does:
– Control the transmission of media stream;
– Use out-of-band signaling;
•
RTSP does not:
– Define compression schemes;
– Define how AV is encapsulated;
– Define how to buffer;
MULTIMEDIA SYSTEMS
IREK DEFEE
MPEG-4 and RTSP
•
From DMIF’s perspective, RTSP is an application alongside MPEG-4
systems;
•
The RTSP client and server interact with the MPEG-4 systems;
•
The RTSP client and server control the streams through the DAI by an RTSPDMIF interface;
•
The interface is kept very simple by limiting it to field mapping between the
RTSP fields and the DAI primitive parameters;
•
The RTSP client server interactions are used to control the MPEG-4
elementary streams;
MULTIMEDIA SYSTEMS
IREK DEFEE
Session Initiation Protocol (SIP)
•
IETF Family of Session Protocols:
– Session Description Protocol (SDP);
– Session Announcement Protocol (SAP);
– Session Initiation Protocol (SIP);
•
SIP:
–
–
–
–
•
Two basic components: user agent & network server;
Independent of lower layer protocols;
Extensible to be application specific;
Gaining widespread use in IP telephony;
MPEG-4 and SIP:
– Unique ability to control different media types within a single session  Multiple
stream transmission in one network session;
– User Agent model fits in well with an MPEG-4 Client/Server model (point-to-point
communication);
MULTIMEDIA SYSTEMS
IREK DEFEE
Toolboxes for transmitting MPEG-4 over
internet
•
RTP  transport of audio/video/… data, quality-of-service feedback;
•
RTSP  very simple media control of streams;
•
SIP  inviting people, media servers to sessions – telephony and streaming
audio/video;
•
HTTP, SDP  retrieve media descriptions;
MULTIMEDIA SYSTEMS
IREK DEFEE
Issues
•
Encapsulation of MPEG-4 Sync layer packetized stream:
– IEC/ISO 14496-8, framework still in revision;
– Lots of issues still remain: time axis, buffer management, etc…
•
Interactivity between application and End User:
– Description of MPEG-4 content;
– Initialization of an MPEG-4 session;
•
SIP and IPC (Inter Process communication):
– How to describe the dynamic process of channel (stream) setup and release?
– What control information is necessary and how to transport it?
•
Transport and IP QoS:
– Must define a mapping mechanism among the different QoS mechanisms:
transport QoS (not available yet), network QoS, etc.
•
And more…e.g. DMIF
MULTIMEDIA SYSTEMS
IREK DEFEE
DMIF
• Delivery Multimedia Integration
Framework
• DMIF is networking part of the MPEG-4
standard
• DMIF specifies the Delivery Layer of
MPEG-4 (mostly in generic form)
MULTIMEDIA SYSTEMS
IREK DEFEE
MPEG-4 Standard Structure
Related to networking
Delivery
Layer
Delivery Aware, media unaware :
DMIF
FlexMux
FlexMux
Tool
Media and Delivery unaware :
Systems
DAI
Sync.
Layer
ESI
Media Aware, delivery unaware :
Audio and Visual Compression
Scene Description and
Object Descriptor Protocol
Compression
Layer
Composition
MULTIMEDIA SYSTEMS
IREK DEFEE
Why DMIF?
• many different delivery technologies, each with its
own peculiarities
• different APIs for different environments (Local
Files, Broadcast sources, Interactive servers
through a variety of transports)
• no consolidated solution for real time multimedia
streaming at certain QoS
• the introduction of QoS support in networks also
impacts on charging policies
MULTIMEDIA SYSTEMS
IREK DEFEE
DMIF goals
• Separate the delivery aspects from the multimedia
application issues
• Guarantee permanence of multimedia application
in presence of new delivery technologies
• Allow the concurrent usage of different delivery
technologies (e.g. broadcast + local retrieval)
• Allow resource logging to support flexible
charging policies
MULTIMEDIA SYSTEMS
IREK DEFEE
DMIF
• DMIF addresses the following scenarios:
Broadcast
source
– multicast/broadcast
Local
Storage
– local storage
Network
– remote interactive
MULTIMEDIA SYSTEMS
IREK DEFEE
The DMIF reference architecture
• supports the streaming of multimedia content
• avoids embedding delivery dependency in a
multimedia application
• allows the concurrent usage of multiple delivery
technologies
• enables a plug-in approach to add new modules
– to follow technical evolution in content streaming
– to allow ad-hoc delivery implementations (e.g. for
charging, admission control, Intelligent Networks …)
MULTIMEDIA SYSTEMS
IREK DEFEE
DMIF Filter
The DMIF reference architecture
Origin.
App
Origin. DMIF
for Broadcast
Target
DMIF
Target
Application
Origin. DMIF
for Local Files
Target
DMIF
Target
Application
Origin. DMIF
for Remote srv
DAI
Sig
map
Broadcast
source
Local
Storage
Network
DNI
DNI
MULTIMEDIA SYSTEMS
IREK DEFEE
Target
App
Target
DMIF
Sig
map
DAI
MPEG-4 synchronisation, mux & transport
(MPEG-4 own Sync Layer + Flexmux approach)
HTML Applic.
page Signal.
Video
Java
classOD BIU BIA JPG
Audio
GIF
ES Channels
SL
SL
SL
SL SL
SL
QoS
Manager
Synchronisation
Layer
DAI
SL
FlexMux Channels
Control
Plane
DMIF
FlexMux
Delivery Layer
FlexMux
TransMux Channels
RTP
HTTP
TCP
IP
ESI
MULTIMEDIA SYSTEMS
UDP
IREK DEFEE
DNI
RTCP
MPEG-4 synch., mux & transport (IETF
PROPOSAL, RTP CHANNEL MULTIPLEX)
Applic.
HTML Signal.
page
Video
Java
BIU
JPG
Audio
class
GIF
BIA
OD
ES Channels
Synchronisation and Protection
ESI
Generic MP4
ES (& SL-PDU) to
RTP mapping
DAI
DMIF
Delivery Layer
Control
Plane
DNI
TransMux Channels
HTTP
TCP
RTP/RTP Mux
MULTIMEDIAUDP
SYSTEMS
IREK DEFEE
IP
RTCP
Conclusions
•
MPEG-4 is here in a limited fashion;
•
We will see a marked growth in MPEG-4 products as chip sets become more
readily available;
•
Simple basic MPEG-4 service is not a problem;
•
There is a lot of ongoing research on how to deliver MPEG-4 content over IPbased networks;
MULTIMEDIA SYSTEMS
IREK DEFEE