Next Generation Streaming Technology
원 유집
yjwon@ece.hanyang.ac.kr
한양대학교 공과대학 전자전기 컴퓨터 공학부
한양대학교 전자전기컴퓨터 공학부
http://www.dmclab.hanyang.ac.kr
1
What is Streaming?
On-line Playback of multimedia data




Remote Playback vs. Local Playback
Bi-directional(ITV) vs. Uni-directional(VoD)
Unicast vs. Multicast
Streaming Requirement

Excessive Space Requirement


ATSC(19.2MBits/sec) movie of 110 minutes: 15 GBytes
Excessive Bandwidth Requirement

ATSC: about 19.2 Mbits/sec, MPEG1/MPEG4: approx. 300Kbits/sec,
MPEG2: about 10 Mbits/sec
한양대학교 전자전기컴퓨터 공학부
http://www.dmclab.hanyang.ac.kr
2
Markets in Multimedia Streaming

Worldwide growth of digital STBs(Feb. 8, 2001 MRG, Inc.)




> $11.5 B in annual sales in 2004, over 140 M units by 2004.
Growth of related digital services > $11 B (annually) by 2004.
in aggregated new revenues > $54 B by 2004.
service revenues






Electronic Program Guides (EPG)
Personal Video Recorders (PVRs)
Video-on-Demand (VOD)
Interactive TV (ITV)
Pay per View (PPV)
Companies

NDS, Sony, TiVo, Motorola, Microsoft, Pace, Sarnoff, DirecTV,
EchoStar, Hughes, Philips, Broadcom, Intel, National
Semiconductor, Liberate, OpenTV, nCube, AOLTV, WebTV,
Scientific Atlanta, Thomson, CacheVision, NBC, Wink, RespondTV
and many others.
한양대학교 전자전기컴퓨터 공학부
http://www.dmclab.hanyang.ac.kr
3
Components of Streaming: from Technology Aspect
System
File System, RTOS
Storage Subsystem
Authoring
L2: IMT-2000, Ethernet
MPEG-1/2/4
L3: IP
Sorenson Video
L4: RTP/RTCP/RTSP
H.323
Etc.
Network
한양대학교 전자전기컴퓨터 공학부
Compression
http://www.dmclab.hanyang.ac.kr
4
Components of Streaming: from Developer’s Aspect
Server
File System, RTOS,
Storage Subsystem,
Encoder, Network
Transport
Player, Network Transport,
Decoder, Presentation(BIFS,
SMIL, DMIF)
Client
한양대학교 전자전기컴퓨터 공학부
Encoder, Decoder
Presentation
Tool(BIFS, SMIL)
Authoring Tool
http://www.dmclab.hanyang.ac.kr
5
History of Multimedia Technology

Early 90’s: Local Playback of Multimedia Contents(OS issues)




Mid 90’s: Remote Playback of Multimedia Contents(NW Issues)





File System
Resource Allocation
Call Admission Control in single address space
RTP/RTCP/RTSP
Synchronization
Bandwidth Guarantee on Network
Smoothing
Late 90’s: Contents Management and Mobile Issues





Image Processing(Transcoding)
Pattern Matching(DB Search)
Authoring Tool
Multimedia in Mobile Environment
Heterogeneous Support(Scalable Encoding, Layered transmission)
한양대학교 전자전기컴퓨터 공학부
http://www.dmclab.hanyang.ac.kr
6
Market Trends
Technology
Industry
Early 1990’s
Emerging MM and VOD
Aggressive Investment
Mid 1990’s
Lack of Network BDW
Retreat
Late 1990’s
Advance of
High Speed Network
Lucrative Business
Opportunity
한양대학교 전자전기컴퓨터 공학부
http://www.dmclab.hanyang.ac.kr
7
Next Generation Streaming Support

Support for Heterogeneous Terminal


Support for Heterogeneous Network


Few tens of Mhz – Ghz terminal
Few tens of Kbits/sec – Several tens of Mbits/sec
Support for RASUM
Network
• QoS Guarantee
• Broadband
•Adaptive Bandwidth
Streaming Server
• Scalability
•Availability
•Reliability
Next Generation
Multimedia Streaming
Streaming Client
•Light Weight
•Heterogeneous Capability
•Adaptive Bandwidth
한양대학교 전자전기컴퓨터 공학부
http://www.dmclab.hanyang.ac.kr
8
Next Generation Streaming Environment
Notebook
Mobile Terminal
Internet
Internet
Server
Intranet
Mobile
Network
Streaming Server
한양대학교 전자전기컴퓨터 공학부
http://www.dmclab.hanyang.ac.kr
9
Distributed Scalable Streaming Server
Application
Server
High
Perpermance
computer
Switch
Switch
Mirrored
Servers
Storage
Backup
한양대학교 전자전기컴퓨터 공학부
Storage
Rack
http://www.dmclab.hanyang.ac.kr
10
Component Technology: Server
Multimedia System Software
Resource Allocation
QoS Management
Disk Scheduling
Buffer Management
Adaptive Streaming
Transcoding
Buffer
Management
Error Resilient Congestion Control
초 대용량 화일 시스템 구축
Massive Scale File System for Streaming
Serverless Network File System
File System for Multimedia Service
Distributed Scalable Clustering Technology
P-to-P Circuit Switch based interconnect
Light weight I/O
Distribution of Protocol Stack
한양대학교 전자전기컴퓨터 공학부
http://www.dmclab.hanyang.ac.kr
11
Multimedia Streaming: Server/Client

Streaming Server






Difficult to provide Bandwidth Guarantee
Bursty traffic
CPU scheduling: Legacy TS approach is not feasible.
File System: Legacy UFS does not fit.
How to configure the system to support +1 M users.
Client




Heterogeneous Terminal: PDA, Notebook, Desktop
Heterogeneous Network: T1, xDSL, Cable Modem, POT, IMT-2000
Wired/Wireless, Static/Dynamic Connection
Efficient codec to run on light weight processor( < 33 MHz)
한양대학교 전자전기컴퓨터 공학부
http://www.dmclab.hanyang.ac.kr
12
Issues







Adaptive stream QoS management technology for
heterogeneous network/client environment
Support multiple speed playback
TCP friendly congestion control mechanism for multimedia
streaming
Operating Systems Kernel optimized for streaming
Clustered File system technology optimized for multimedia
streaming operation
Light weight I/O technology which can handle hundreds of
terabytes data
Load Balancing
한양대학교 전자전기컴퓨터 공학부
http://www.dmclab.hanyang.ac.kr
13
Execution Environment

분산 실행환경
QoS Negotiation
Request for Stream
Sessions with Rate,
QoS, Criticality
비 사용자원
세션 i
실행환경
Sessions with
Thread, X, Memory
Parameter
시스템 자원 관리
Posix 운영체제
한양대학교 전자전기컴퓨터 공학부
자원
용량
CPU
Mem
Disk
운영체제
세션 j
최소 확보량
http://www.dmclab.hanyang.ac.kr
14
Serve Architecture
GUI
Web-Based Monitoring & Management
Stream Manager
Admission Control
bandwidth estimation
QoS Mapper
SL-Packetization
SRM
NIC
DISK
CPU
FlexMux
MM
Scheduler
Resource Monitor
Smart Micro Kernel
Session Manager
MPEG-4 File Manager
Linux Operating System (Kernel 2.2.x )
한양대학교 전자전기컴퓨터 공학부
http://www.dmclab.hanyang.ac.kr
15
System Components

Session
Server


NIC
Memory
Decoder
Display
resource allocation/scheduling for QoS guarantee
Component: CPU, Disk, Memory, NIC
QoS Mapper



Memory
NIC
Client
System Resource Manager


Disk
Map QoS metric to system resource metric
(25fps, 600*480, 1.5Mbps)  Memory(1.5Mbps) , CPU (5%)
Call Admission Control

request 요청시, QoS factor 가 변경시 ( ex, Fast forward, backward,.. ),
수락/거절 결정
한양대학교 전자전기컴퓨터 공학부
http://www.dmclab.hanyang.ac.kr
16
Client Architecture
Smart Client
MPEG-4 Player
UI/ Display: JMF for Portability
Codec: Native Code for Performance
RTP: Java/Native Code
Windows / Solaris / Linux
한양대학교 전자전기컴퓨터 공학부
http://www.dmclab.hanyang.ac.kr
17
Session Management Algorithm
Session Start
Session Termination
RTSP receives
request
RTSP receives tear
down request
QoS Mapper
No
Resource release
enough resource ?
Yes
QoS Negotiation
resource
reservation
Maximize QoS factor
of remaining users
Streaming start
한양대학교 전자전기컴퓨터 공학부
http://www.dmclab.hanyang.ac.kr
18
Session Management Algorithm
Playback Mode Change
RTSP receives 2x, 4x, -1x,2x,.. messages
QoS Mapper
Enough
resource ?
No
Yes
Send reject message
Resource reservation
Streaming


Thread per session/Thread per Resource
Issues: Synchronization between threads
한양대학교 전자전기컴퓨터 공학부
http://www.dmclab.hanyang.ac.kr
19
File System for Streaming

Typical Streaming Operation



Sequential Read
Occasional Fast-Forward, Fast-Backward, Pause
Characteristics of Streaming Operation


Bandwidth Guarantee
Minimize Delay variances
Playback
한양대학교 전자전기컴퓨터 공학부
Retrieval
http://www.dmclab.hanyang.ac.kr
20
File System for Streaming

Minimize Latency


Make the file structure flat  Reduce seek overhead
Minimize Delay Variation

File Structure should remain the same with the change in the file
size.
Is legacy UFS family OK? Probably Not!

UFS design philosophy



Handling wide variety of file size without loss of disk space
Optimized for random I/O
File System for Multimedia Streaming


Minorca(U. of Oslo, Norway), MMFS(SUNY Stony Brook, USA),
Presto(U. of Minnesota, USA), SMART(Hanyang U., Korea)
Tigershark(IBM Almaden), Tiger(Microsoft)
한양대학교 전자전기컴퓨터 공학부
http://www.dmclab.hanyang.ac.kr
21
Issues in Massive Scale Cluster Server Design?
How to Support +1 M concurrent session?
Single Gbit NIC supports 100 200Kbits/sec streams.
Server: Architectural O/S issues





Running entire stack on a general purpose SMP
No direct disk to NIC transfers
Bus based architecture
O/S may cause queues in wrong places
O/S supported I/O, IPC and synchronization are typically very
inefficient
한양대학교 전자전기컴퓨터 공학부
http://www.dmclab.hanyang.ac.kr
22
Server: Load Management Issues

Symmetric Architecture vs. Layered Architecture


Layered Architecture: Easier to manage, configure, engineer, but
performance implication is not clear
Load Distribution for Symmetric Architecture




Client Based Approach(Netscape Access)
Round Robin DNS(CISCO’s LocalDirector, Cisco’s Distributed
Director)
Dispatcher Based Approach(IBM Network Dispatcher)
Server Based Approach(Scalable Server WWW)
한양대학교 전자전기컴퓨터 공학부
http://www.dmclab.hanyang.ac.kr
23
Server: Content Management Issues


Large Facilities built as loosely connected clusters of servers
Significant overheads of contents support





NFS mounting  Bottleneck, single point of failure
Full Duplication  Consistency management overhead
Content Partitioning  Traffic distribution difficult and single point of
failure
File Cached in every server  Nonscalable
Content partitioning difficult to handle because


Shifting demand phenomenon
Heterogeneous servers
한양대학교 전자전기컴퓨터 공학부
http://www.dmclab.hanyang.ac.kr
24
Server: Layered Architecture


Number of Machines in each layer?
How to maintain state information across the layer?
IP
L5-7
IP
L5-7
IP
L5-7
TCP/
UDP
L5-7
L5-7
TCP/
UDP
한양대학교 전자전기컴퓨터 공학부
TCP/
UDP
TCP/
UDP
Server Farm
http://www.dmclab.hanyang.ac.kr
25
Multimedia Streaming: Network

QoS Guarantee





RTP/RTSP/RTCP
Diffserv(PHB)
Intserv(RSVP)
MPLS
Streaming Server
Adaptive Streaming



Scalable Encoding
TCP friendly congestion
control
Transcoding




Wired/Wireless Internet
Color  Black/White
Picture  Text
30 fps  3 fps
Mobile Multimedia


Smooth handoff
Error Resilience
한양대학교 전자전기컴퓨터 공학부
Clients
Internet does not guarantee
bandwidth!!!
http://www.dmclab.hanyang.ac.kr
26
Issues in Network Support for Streaming

QoS Guarantee




Stochastic vs. Deterministic Guarantee  QoS model
Diffserv: Router does not maintain the states of individual sessions.
Intserv: Router maintains the states of individual sessions. E.g.
RSVP
Determining the timely flow of information



Internet does not allow timely delivery of packet, but still mechanism
for detecting timely delivery of data is required.
RTP/RTCP, RTSP  The respective information is included in
packet header.
MPEG-4 over RTP


Which information is to be maintained?
In what format? (draft-gentric-avt-rtp-mpeg4-00.txt)
한양대학교 전자전기컴퓨터 공학부
http://www.dmclab.hanyang.ac.kr
27
Issues in Network Support for Streaming

Smoothing






Make the bursty video traffic smoother.
Server side smoothing(Transmission based on prior knowledge of
bandwidth requirement)
Client Side Smoothing(Introducing Buffer)
Better Congestion Control
Improve Loss/jitter situation.
More delay is introduced.
한양대학교 전자전기컴퓨터 공학부
http://www.dmclab.hanyang.ac.kr
28
Issues in Network Heterogeniety Support for
Streaming

Network environment gets more diverse.


Adaptive Streaming





T1, LAN, ADSL, Wireless LAN, 3 G mobile link
Media Transcoding
Source Driven vs. Receive Driven
Unicast vs. Multicast
Scalable Multimedia Model
Adjusting the rate



Adjust Frame rate  Drop frames.
Use hierarchically encoded streams.  Layered Transmission
For live broadcast, adjust the encoding rate(e.g. QCIF)
한양대학교 전자전기컴퓨터 공학부
http://www.dmclab.hanyang.ac.kr
29
Issues in Mobile Multimedia

Source of Difficulties




Wireless Link  Large Bit Error Rate(BER), Fading
Mobility  Packet Route Changes
Fluctuating Bandwidth
Mobility

hand-offs  Delay, Jitter
한양대학교 전자전기컴퓨터 공학부
http://www.dmclab.hanyang.ac.kr
30
Issues in Mobile Multimedia

Mobility
Advanced Reservation of Bandwidth
 Between Sender and Base Station
 Between Base Station and Terminal
 Routing Protocol: Anticipatory, Proactive
 utilization efficiency issue
 Transport Protocol: Layered, Adaptive
Multi Resolution Streaming
 Switch from MPEG to H. 263
 Multicast to Neighborhood station


한양대학교 전자전기컴퓨터 공학부
http://www.dmclab.hanyang.ac.kr
31
Presentation: SMIL

SMIL(Synchronized Multimedia Integration Language, W3C)

Feature





Describe the temporal behavior of the presentation.
Describe the layout of the presentation on a screen.
Associate hyperlinks with media object.
Objects: audio, video, animation, image, text, text stream, ref
Features

par
예)
<par>
<audio id=“a” begin=“6s” src=“audio.ra” />
<video id=“v” src=“video.rm” />
</par>

예)
6s
audio.ra
video.rm
seq
<seq>
<audio src=“audio1” />
<audio begin=“5s” src=“audio2” />
</seq>
한양대학교 전자전기컴퓨터 공학부
audio1
5s
audio2
http://www.dmclab.hanyang.ac.kr
32
Presentation: Structure of MPEG4
audiovisual objects
voice
hierarchically multiplexed
downstream control / data
sprite
hierarchically multiplexed
upstream control / data
2D background
audiovisual
presentation
y
3D objects
scene
coordinate
system
x
z
user events
video
compositor
projection
plane
audio
compositor
hypothetical viewer
speaker
한양대학교 전자전기컴퓨터 공학부
display
user input
http://www.dmclab.hanyang.ac.kr
33
Summary

Next Generation Multimedia Streaming Technology





Clustered Solution





Adaptive End to End Streaming Transport: Unicast vs. Multicast
Scalable Encoding
Presentation



High Speed Storage Interconnect
Content Partitioning
Load Management
Support for Heterogeniety


Massive Scale Support  Clustered Solution
Adaptive to Heterogeneous Network
Adaptive to Heterogeneous Terminal Capability
Presentation Technique
MPEG-4
SMIL
For further information, checkout
http://www.dmclab.hanyang.ac.kr/courseware/class/mmdbms/2000/index-mmdbms.htm
한양대학교 전자전기컴퓨터 공학부
http://www.dmclab.hanyang.ac.kr
34
Appendix
한양대학교 전자전기컴퓨터 공학부
http://www.dmclab.hanyang.ac.kr
35
Presentation: MPEG 4

MPEG-4


BIFS(Binary Format For Scenes): VRML like syntax
 Compressed binary format, Streaming, Animation, 2D
primitives, Enhanced audio, Facial animation
DMIF(Delivery Media Integration Framework)


Session Layer Level Support for Multimedia Presentation
Definition of transport and synchronization of stream
한양대학교 전자전기컴퓨터 공학부
http://www.dmclab.hanyang.ac.kr
36
Presentation: Hierarchical Structure of MPEG 4
VS0
Visual Session
VO0
Video Object
VOL0
Video Object Layer
Group of Video Object Plane
Video Object Plane
GOV0
VS1
VO1
VOL1
GOV1
VOP0 .....VOPn VOPn+1.....VOPm
한양대학교 전자전기컴퓨터 공학부
http://www.dmclab.hanyang.ac.kr
37