Introduction
Topics in Multimedia
Computing and Networking
Ketan Mayer-Patel
CS 294-9 :: Fall 2003
The medium is the message.
• Marshall McLuhan (1967)
• Themes of the course:
– In multimedia communication, how the
message arrives is often as important as what
the message is.
– The two are entangled.
• How the message arrives: Networking
• What the message is: Multimedia Encoding
CS 294-9 :: Fall 2003
Characterizing media data
• Continuous media is a bit more accurate.
–
–
–
–
–
–
Periodic
Robust
Often large
Correlated
Endless?
Video, audio, stock ticker?, depth?
• Compare with web pages.
– Web pages are definitely multimedia, but don’t
fit the characteristics of continuous media.
CS 294-9 :: Fall 2003
Video Taxonomy
Compressed
Tape
Digital Betacam
DVCPro (D-7)
DV
DVCAM
Digital-8
D-9
DVCPro50
Uncompressed
Streaming
Digital
MPEG-1
D-1 (CCIR 601)
MPEG-2
D-2
MPEG-4
D-5
M-JPEG
H.261
H.263+
Real
Sorenson
Cinepak
Indeo
Video for Windows
CS 294-9 :: Fall 2003
Analog
Betacam
VHS
S-Video
Video-8
Hi-8
Betamax
Audio Taxonomy
Compressed
Uncompressed
Speech-based
General
LPC-10E
CELP
GSM
ADPCM
Dolby AC-3
MPEG-1 L3
MPEG-2 AAC
DTS
SDDS
THX
MiniDisc (ATRAC)
CS 294-9 :: Fall 2003
m-law
A-law
Linear PCM
Network Basics
Throughput = Amount of
data arriving per unit time.
Delay
Source
Dest
Loss
Interpacket
Arrival Time
Jitter = Variance of
interpacket arrival
times
CS 294-9 :: Fall 2003
Problems of Delay
• Interactive applications
– Video conferencing
• Human communication starts to break down at
around 500 msec delays.
– Feedback loops
• Haptic interfaces
• Quality of Service
– Best effort and end-to-end mechanisms can do
little to control delay.
CS 294-9 :: Fall 2003
Problems of Jitter
• Buffer management
– Buffers are used to smooth jitter, but how big
should the buffer be?
– Unexpected jitter can lead to starvation.
– Over-provisioning buffers creates delay.
• Jitter is hard to bound even with QoS.
• Media jitter vs. packet jitter
– Variable bitrate media (where number of
packets for “unit” of media varies) can make
this relationship hard to model.
CS 294-9 :: Fall 2003
Problems of Throughput
• Estimating throughput is problematic.
– Media coding may be tunable, but generally
changes in quality are perceptually annoying.
– Throughput will often vary faster than feedback
delay which makes estimation hard.
• Multicast makes this worse.
• Scalable encoding is one solution path.
• QoS mechanisms are another.
CS 294-9 :: Fall 2003
Problems of Loss
• Effect of loss is very media and encoding
specific.
– Loss of one packet in a video stream can result
in the loss of multiple frames.
– Other packets received can be rendered useless.
• Goodput vs. Throughput
• Retransmission often not feasible.
• Encoding may be designed to alleviate
packet loss effects.
• Forward Error Correction
CS 294-9 :: Fall 2003
Administrivia
• Who am I?
– kmp@cs.unc.edu
– On campus Mon. and Wed. (629 Soda)
• Who are you?
• Class web page
– http://inst.eecs.berkeley.edu/~cs294-9
– Schedule of topics and readings posted here.
• Need to sign up for presenting papers.
– More on this next week.
• Reading for next week.
– Jim Blinn’s article on NTSC
CS 294-9 :: Fall 2003
TCP/UDP Review
•
•
•
•
Transport protocols on top of IP
TCP: ordered, byte-stream, reliable
UDP: datagram, unreliable, unordered
Programming API: sockets
– Originally from UNIX, but available in
Windows as WinSock and Java
• Network byte-order is big endian .
CS 294-9 :: Fall 2003
UDP Packet Format
32 bits
Src Port #
Dest Port #
Length
Checksum
Data
CS 294-9 :: Fall 2003
UDP Review
• Connectionless
– No communication necessary other than the
datagram itself.
• Unreliable
– No guarantee or feedback whether data got ther
or not.
• Unordered
– Packets may arrive in any order
– Limit on delay: 1 minute (comes from IP)
CS 294-9 :: Fall 2003
TCP Packet Format
32 bits
Src Port #
Dest Port #
Sequence #
Acknowledgement #
Hdr UnLen. used
Flags
Window Size
Checksum
Urgent Ptr
Options
Data
CS 294-9 :: Fall 2003
TCP Lifecycle
• 3-way handshake
– Establishes connection.
• Data transfer
– Congestion controlled
– Reliable
– In order.
• FIN
– Connection teardown.
CS 294-9 :: Fall 2003
3-way handshake
• One side must be “listening” for a connection.
– Whoever is listening is “the server.”
• Client sends a SYN packet.
– SYN flag is set.
– Seq. # is randomly chosen starting byte number.
• Server sends back SYN-ACK packet.
– SYN flag is set, ACK flag is set.
– Seq. # is set and Ack # reflects client’s Seq. #
• Client sends back ACK
CS 294-9 :: Fall 2003
Data Transfer
• Full duplex connection.
– Both sides can send data simultaneously.
• Seq. # set to byte number of first byte.
• Ack # set to next byte expected.
• Receiver generates an ACK even if it doesn’t
have data to send back.
– Some implementations delay sending ACK to
optimize piggyback case.
– In general, implementations expected to generate
one ACK for each data packet received.
CS 294-9 :: Fall 2003
Reliability
• Sender maintains data until acknowledged.
– This is done at the OS. The application is free from this
responsibility.
• Sender estimates RTT from ACK’s.
• Timeout values dynamically calculated from RTT and
RTT variance.
• If timeout occurs, data retransmitted.
• Holes in the byte stream are possible.
– Receiver still generates ACK, but ACK # reflects next byte needed in seq.
– Multiple ACK’s used to retransmit quickly.
CS 294-9 :: Fall 2003
Transfer Example
Seq. 0
Seq. 10
Seq. 20
Seq. 30
Ack 10
Ack 10
Ack 10
Timeout
Seq. 10
Ack 40
CS 294-9 :: Fall 2003
Flow Control
Sender
Receiver
Application
Application
OS
TCP
Last ACK’d byte
First un-ACK’d byte
Recv Buffer
Send Buffer
Free Space
OS
TCP
Last un-ACK’d byte
Recv. Window
Recv. Window is advertised to
the sender in ACK.
CS 294-9 :: Fall 2003
Congestion Control
• Basic question: how fast to send?
– Too slow: underutilize network
– Too fast: no one gets anything useful through
– Not responsive: congestion collapse
• TCP congestion control is window-based.
–
–
–
–
TCP maintains a “window” of data in the network.
Successful transmissions allow window to increase.
Packet drops cause window to shrink.
Reliability feedback and congestion control coupled
together (may not have been the best idea).
CS 294-9 :: Fall 2003
Slow Start
• Slow Start
– When starting, don’t know what the window should
be, so probe for capacity.
– Initial window: 1 segment size
• A “segment” is a set number of bytes (i.e., packet size).
– For every sucessful ACK, increase window by 1
segement size.
– Doubles window size every RTT.
CS 294-9 :: Fall 2003
Slow Start Illustrated
Seq. 0
Ack 10
Seq. 10, 20
Ack 20, 30
Seq. 30, 40, 50, 60
Ack 40, 50, 60, 70
CS 294-9 :: Fall 2003
Congestion Avoidance
• Eventually slow-start results in a packet
drop. This establishes “ssthresh”.
• Ssthresh set at 1/2 of current window.
• Current window shutdown to 1 segment.
• Data retransmitted.
• Slow-start begins again.
• When window gets back to ssthresh, every
ACK results in window += 1/window.
• Additive increase, multiplicative decrease.
CS 294-9 :: Fall 2003
Window Size
AIMD Illustrated
ssthresh
ssthresh
ssthresh
Time
CS 294-9 :: Fall 2003
Misc. TCP Oddities
• Window is actually minimum of receiver
advertised window and congestion window.
• Duplicate ACK’s used to detect congestion
before actual timeout. (Fast retransmission)
• Special-case calculation of congestion
window after fast retransmission to allow
new data to be sent. (Fast recovery)
• Lots of tweaks and options (SACK, T/TCP)
CS 294-9 :: Fall 2003
Multimedia and TCP/UDP
• Streaming multimedia rarely uses TCP. Why?
– Retransmission is pointless
• By the time data is recovered, too late.
– AIMD congestion makes jitter and throughput changes
unbearable.
• So, why did we just review TCP?
– Everything else in the world is TCP.
– How non-TCP traffic affects TCP traffic is an important
question.
• In general, streaming media protocols built on
top of UDP.
CS 294-9 :: Fall 2003