Real-Time Protocols
RTP/RTCP/RTSP
Amit Hetawal
University of Delaware
CISC 856 -Fall 2005
Thanks to Professor Amer
Overview
•
•
•
•
•
•
History of streaming media
Streaming performance requirements
Protocol stack for multimedia services
Real-time transport protocol (RTP)
RTP control protocol (RTCP)
Real-time streaming protocol (RTSP)
Brief history of streaming media
Real-time multimedia streaming
• Real-time multimedia applications
– Video teleconferencing
– Internet Telephony (VoIP)
– Internet audio, video streaming
(A-PDUs)
Streaming performance requirements
– Sequencing
– to report PDU loss
– to report PDU reordering
– to perform out-of-order decoding
– Time stamping and Buffering
– for play out
– for jitter and delay calculation
– Payload type identification
– for media interpretation
– Error concealment –covers up errors from lost PDU by using
redundancy in most-adjacent-frame
– Quality of Service (QoS) feedback – from receiver to sender for
operation adjustment
– Rate control –sender reduces sending rate adaptively to network
congestion
Ideal Timing – no jitter
00.00.00
00.00.10
00.00.20
00.00.30
application
00.00.11
00.00.21
00.00.31
Send time
Play time
Reality – jitter
delay
00.00.00
00.00.10
00.00.20
00.00.11
00.00.30
00.00.21
00.00.25
00.00.40
00.00.35
00.00.37
Send time
00.00.41
00.00.47
00.00.51
Play time
Jitter
(contd.)
00.00.00
00.00.10
00.00.20
00.00.11
00.00.30
00.00.21
00.00.25
00.00.40
00.00.35
00.00.37
Send time
00.00.41
00.00.47
00.00.18
00.00.28
00.00.38
00.00.48
00.00.51
Play time
00.00.58
Jitter
(contd.)
Playback buffer
At time 00:00:18
At time 00:00:28
At time 00:00:38
How does Sequence number and Timestamp help ?
Audio silence example:
• Consider audio data
– What should the sender do during silence?
• Not send anything
– Why might this cause problems?
silence
• Receiver cannot distinguish between loss
and silence
Solution:
– After receiving no PDUs for a while, next PDU received at
the receiver will reflect a big jump in timestamp, but
have the correct next seq. no. Thus, receiver knows
what happened.
Streaming performance requirements
– Sequencing
– to report PDU loss
– to report PDU reordering
– to perform out-of-order decoding
– Time stamping and Buffering
– for play out
– for jitter and delay calculation
– Payload type identification
– for media interpretation
– Error concealment –covers up errors from lost PDU by using
redundancy in most-adjacent-frame
– Quality of Service (QoS) feedback – from receiver to sender for
operation adjustment
– Rate control –sender reduces sending rate adaptively to network
congestion
Support from transport layers
 TCP is not used because:
•
•
•
•
TCP does retransmissions  unbounded delays
No provision for time stamping
TCP does not support multicast
TCP congestion control (slow-start) unsuitable for real-time transport
RTP + UDP usually used for multimedia services
Protocol stack for multimedia services
RTSP
RTP
RTCP
TCP
(till now)
RTP: Introduction
•
Provides end-to-end transport functions for real-time
applications
– Supports different payload types
•
All RTP and RTCP PDUs are sent to same multicast group
(by all participants)
• All RTP PDUs sent to an even-numbered UDP port, 2p
• All RTCP PDUs sent to UDP port 2p+1
Transport
layer
•
•
Does NOT provide timely delivery or other QoS guarantees
– Relies on other protocols like RTCP and lower layers
Does NOT assume the underlying network is reliable and
delivers PDUs in sequence
– Uses sequence number
Application
RTP RTCP
UDP
IP
Data Link
Physical
RTP Session
 RTP session is sending and receiving of RTP data by a group
of participants
 For each participant, a session is a pair of transport
addresses used to communicate with the group
 If multiple media types are communicated by the group, the
transmission of each medium constitutes a session.
RTP Synchronization Source
 synchronization source - each source of RTP PDUs
 Identified by a unique,randomly chosen 32-bit ID (the SSRC)
 A host generating multiple streams within a single RTP must
use a different SSRC per stream
RTP Basics of Data Transmission
RTP PDUs
RTP PDU Header
Sampling instant of first data octet
• multiple PDUs can have same timestamp
• not necessarily monotonic
• used to synchronize different
Payload type
media streams
Incremented by one for
each RTP PDU:
• PDU loss detection
•Restore PDU sequence
Identifies synchronization source
Identifies contributing sources
(used by mixers)
Mixer
RTP mixer - an intermediate system that receives & combines
RTP PDUs of one or more RTP sessions into a new RTP PDU
• Stream may be transcoded, special effects may be performed.
• A mixer will typically have to define synchronization relationships between
streams.Thus…
 Sources that are mixed together become contributing sources (CSRC)
 Mixer itself appears as a new source having a new SSRC
Translator
• An intermediate system that…
 Connects two or more networks
 Multicasting through a firewall
 Modifies stream encoding, changing the stream’s timing
 Transparent to participants
 SSRC’s remain intact
end system 1
from ES1: SSRC=6
from ES2: SSRC=23
end system 2
from ES1: SSRC=6
from ES2: SSRC=23
transl.1
transl.2
authorized tunnel
firewall
from ES1: SSRC=6
from ES2: SSRC=23
RTP Control Protocol (RTCP)
 RTCP specifies report PDUs exchanged between sources and
destinations of multimedia information
 receiver reception report
 sender report
 source description report
 Reports contain statistics such as the number of RTP-PDUs sent,
number of RTP-PDUs lost, inter-arrival jitter
 Used by application to modify sender transmission rates and for
diagnostics purposes
RTCP message types
Typically, several RTCP PDUs of different types are transmitted
in a single UDP PDU
Sender/Receiver report PDUs
V
P
RC
PT=200/201  SR/RR
Length (16 bits)
SSRC of Sender
Header
NTP Timestamp, most significant word
NTP Timestamp, least significant word
RTP Timestamp
Sender’s PDU Count
Sender
Info
Sender’s Octet Count
SSRC_1 (SSRC of the 1st Source)
Fraction Lost
Cumulative Number of PDU Lost
Extended Highest sequence Number Received
Interarrival Jitter
Report
Block 1
Last SR (LSR)
Delay Since Last SR (DLSR)
SSRC_2 (SSRC of the 2nd Source)
……
Profile-Specific Extensions
Report
Block 2
Ethereal capture for RTP-PDU
Basic header
Ethereal capture for RTCP-PDU
header of SR report
sender info
receiver report block
SDES items
Synchronization of streams using RTCP
RTP audio
RTCP audio
RTP video
RTP video
Internetwork
•
Timestamps in RTP PDUs are tied to the individual video and audio sampling clocks
 timestamps are not tied to the wall-clock time, or each other!
• Each RTCP sender-report PDU contains (for most recently generated PDU in
associated RTP stream):
 The timestamp of RTP PDU
 The wall-clock time for when PDU was created
• Receivers can use this association to synchronize the playout of audio and video
RTCP bandwidth scaling
Problem
• What happens when there is
one sender and many receivers?
 RTCP reports scale linearly with
the number of participants and
would match or exceed the
amount of RTP data! More
overhead than useful data!
Solution
• RTCP attempts to limit its
traffic to 5% of the session
bandwidth to ensure it can
scale!
• RTCP gives 75% of this rate
to the receivers; and the
remaining 25% to the
sender.
Example
• Suppose one sender, sending
video at a rate of 2 Mbps. Then
RTCP attempts to limit its traffic
to 100 Kbps.
• The 75 kbps is equally shared
among receivers:
– With R receivers, each
receiver gets to send RTCP
traffic at 75/R kbps.
• Sender gets to send RTCP traffic
at 25 kbps.
Real-Time Streaming Protocol (RTSP)
•
•
•
•
•
Application layer protocol (default port 554)
Usually runs on RTP for stream & TCP for control
Provides the control channel
Uses out-of-band signaling
Usable for Live broadcasts / multicast
Also known as “Network remote control” for multi-media servers.
RTSP Overview
Web Server
web
browser
HTTP
presentation descriptor
Presentation
descriptor
media
player
Web Server/Media server
RTSP
pres. desc,streaming commands
RTP/RTCP
audio/video content
RTSP Methods
OPTIONS
CS
CS
determine capabilities of server/client
DESCRIBE
CS
get description of media stream
ANNOUNCE
CS
announce new session description
SETUP
CS
create media session
RECORD
CS
start media recording
PLAY
CS
start media delivery
PAUSE
CS
pause media delivery
REDIRECT
CS
redirection to another server
TEARDOWN
CS
immediate teardown
SET_PARAMETER
CS
change server/client parameter
GET_PARAMETER
CS
read server/client parameter
RTSP Session
Default port
554
RTSP
server
RTSP SETUP
RTSP OK
RTSP PLAY
RTSP OK
RTSP TEARDOWN
RTSP OK
TCP
RTSP
client
get UDP port
data
source
RTP VIDEO
RTP AUDIO
choose
UDP port
UDP
AV
subsystem
RTCP
media server
media player
Example:Media on demand (Unicast)
Media server A
audio.example.com
Media server V
video.example.com
Client C
Web server W
-holds the media descriptors
RTSP Message sequence
C -> W : GET/Twister.sdp HTTP/1.1
Host: www.example.com
Accept: application/sdp
W-> C : HTTP/1.0 200 OK
Content-Type: application/sdp
C-> A : SETUP rtsp://audio.example.com/twister/audio.en RTSP/1.0
Cseq:1
Transport : RTP/AVP/UDP;unicast;client_port=3056-3057
A-> C : RTSP/1.0 200 OK
Cseq:1
Session: 12345678
Transport : RTP/AVP/UDP;unicast;client_port=3056-3057
server_port=5000-5001
C->V : SETUP rtsp://video.example.com/twister/video.en RTSP/1.0
Cseq:1
Transport : RTP/AVP/UDP;unicast;client_port=3058-3059
A-> C : RTSP/1.0 200 OK
Cseq:1
Session: 23456789
Transport : RTP/AVP/UDP;unicast;client_port=3058-3059
server_port=5002-5003
W
V
C
A
RTSP Message sequence
(contd.)
C->V: PLAY rtsp://video.example.com/twister/video RTSP/1.0
Cseq: 2
Session: 23456789
W
V->C: RTSP/1.0 200 OK
Cseq: 2
Session: 23456789
RTP-Info: url=rtsp://video.example.com/twister/video;
seq=12312232;
C->A: PLAY rtsp://audio.example.com/twister/audio.en RTSP/1.0
Cseq: 2
Session: 12345678
A->C: RTSP/1.0 200 OK
Cseq: 2
Session: 12345678
RTP-Info: url=rtsp://audio.example.com/twister/audio.en;
seq=876655;
V
C
A
RTSP Message sequence
(contd.)
C->A: TEARDOWN rtsp://audio.example.com/twister/audio.en RTSP/1.0
Cseq: 3
Session: 12345678
W
A->C: RTSP/1.0 200 OK
Cseq: 3
V
C->V: TEARDOWN rtsp://video.example.com/twister/video RTSP/1.0
Cseq: 3
Session: 23456789
V->C: RTSP/1.0 200 OK
Cseq: 3
C
A
References
[1] B. A. Forouzan, “TCP/IP Protocol Suite”,
Third edition,
[2] H. Schulzrinne, S. Casner, R. Frederick and V.
Jacobson, "RTP: a transport protocol for real-time
applications", RFC 3550, July 2003.
[3] H. Schulzrinne, A. Rao and R. Lanphier, "Real Time
Streaming Protocol (RTSP)", RFC 2326, April 1998.
RTCP compound PDU
receiver
report
source 3
SDES
compound PDU
(single UDP datagram)
SSRC
receiver
report
source 2
RTCP PDU 2
SSRC
sender
report
SSRC
SR
SSRC
RTCP PDU 1
CNAME PHONE
Example
source 1 reports, there are 2 other sources
receiver
report
source 2
SSRC
sender
report
SSRC
SR
SSRC
RTCP PDU
receiver
report
source 3
RTCP processing in Translators
• SR sender information : Does not generate their own sender
information(most of the times), but forwards the SR PDUs
received from one side to other
• RR reception report blocks : Does not generate their own RR
reports (most of the times), but forwards RR reports received
from one side to another. SSRC are left intact
• SDES : Forwards without changing the SDES info. but may
filter non CNAME SDES, if bandwidth is limited
• BYE : Forwards BYE PDU unchanged. A translator about to
cease forwarding, send a BYE PDU to each connected nodes
RTCP processing in Mixers
• SR sender information : Generates its own SR info. Because
the characteristics of source stream is lost in the mix. The SR
info is sent in same direction as the mixed stream
• RR reception report blocks : Generates its own reports for
sources in each cloud and sends them only to same cloud
• SDES : Forwards without changing the SDES info. but may
filter non CNAME SDES, if bandwidth is limited
• BYE : Forwards BYE PDU unchanged. A mixer about to cease
forwarding, send a BYE PDU to each connected nodes
Source description PDUs
May contain:
–
–
–
–
–
–
–
–
a CNAME item (canonical identifier/name)
a NAME item (real user name)
an EMAIL item
a PHONE item
a LOC item (geographic location)
a TOOL item (application name)
a NOTE item (transient msg, e.g. for status)
a PRIV item (private extension)
CNAME=1
length
user and domain name
Value
1
2
3
4
5
6
7
8