Changing Landscape of Multimedia
Applications
Today: Downlink Video
Broadcast
Tomorrow: Uplink Video
Transmission
Contemporary Video Coding Standards
• Motion-Compensated Predictive Coding (MPEG/H.26)
– High compression efficiency
– Rigid complexity partition between encoder (heavy) & decoder (light)
– High fragility to transmission losses
DFD
(Displaced Frame Difference)
+
• Image Coding (Motion JPEG)
– Low complexity
– High robustness to transmission losses
– Low compression efficiency
Rethinking Video Over Wireless
Challenges:
•
•
•
Low bandwidths  high compression efficiency
Limited handheld battery power  low end-device complexity
Lossy wireless medium  robustness to transmission losses
New Architecture: PRISM
(Power-efficient, Robust, hIgh-compression Syndrome-based Multimedia coding)
•
High compression efficiency
•
Flexible partition of complexity
between encoder & decoder
•
Inbuilt robustness to channel loss
•
Backward compatibility with existing
video standards
Puri & Ramchandran,
Allerton ’02
Heavy PRISM
Uplink Decoder
Light PRISM
Uplink Encoder
Heavy PRISM
Downlink Encoder
Trans-coding Proxy
Light PRISM
Downlink Decoder
Background: Distributed Source Coding
Source Coding with side-information (Slepian–Wolf, Wyner-Ziv)
X
Encoder
Decoder
^
X
•
X and Y are correlated sources
•
Y is available only at decoder
Y
• Exploit side-information Y at the decoder while encoding X
• No MSE performance loss over case when Y is available at
both encoder and decoder when innovations is Gaussian
• For the video coding case, X is the block to be coded and
the side-information Y consists of the previously decoded
blocks in the frame memory
Motion-Free Encoding?
Y1
X
...
Predictive
Encoder
Y1
X
...
Y1’
YM
… Motion Vector …
… Quantized …
DFD
PRISM
Encoder
Predictive
Decoder
Y1’
YM
?
...
YM’
...
X
YM’
PRISM
Decoder
X
• The encoder does not have access to Y1’, Y2’, etc
• Neither the encoder nor the decoder knows the correct sideinformation
• Can decoding work?
– Yes!
– A “modified” Wyner-Ziv paradigm is needed
(Ishwar, Prabhakaran, & Ramchandran ICIP ’03.)
PRISM
Y1’
Wyner-Ziv
Decoder
X
Wyner-Ziv
Encoder
bin index
Decoding failure
YT’
Wyner-Ziv
Decoder
YM ’
Wyner-Ziv
Decoder
Robustness Comparisons:
• Predictive Coding: channel errors
lead to prediction mismatch and drift
• PRISM: drift stopped if syndrome
code is “strong enough”:
Targeted noise ≥ Correlation Noise
+ Induced Channel Noise + Quant. Noise
X
Decoding
failure
• Need concept of “motion
compensation at decoder”!
• Need mechanism to detect
decoding failure
• In theory: joint typicality
(statistical consistency)
• In practice: use CRC
Standards-Compliant Auxiliary-Channel
Auxiliary-Channel
Auxiliary-Channel
Encoder
Coset
Index Wireless
Channel
Auxiliary-Channel
Decoder
Final
reconstruction
Xmain
X
^
Wireless
MPEG/H.26x X
Encoder
MPEG/H.26x Channel
bit-stream
MPEG/H.26X
Decoder
• Secondary description of video sent over auxiliary-channel.
• Need to find statistics of correlation noise Z = X – Xmain.
– Can leverage algorithm of Zhang, Regunathan and Rose (Asilomar ’99) to
develop recursive correlation estimation algorithm.
(Wang, Majumdar, Ramchandran, and Garudadri: PCS ’04.)
• Auxiliary channel allows drift correction without intra-refresh.
Results
•
Channel simulator provided by Qualcomm Inc. conforming to a CDMA 2000 1x
standard.
•
Performance comparison among 3 systems:
– H.263+ bitstream with 20% extra rate for FEC (RS codes)
– H.263+ bitstream with 20% extra rate for standard-compliant auxiliary channel
– PRISM
•
Standard-compliant auxiliary channel version outperforms H.263+FEC by 2.5-4
dB between error rates of 2-10%.
•
PRISM outperforms H.263+FEC by 6-8 dB between error rates of 2-10%.
H.263+ with FEC
H.263+ with Auxiliary Channel
Stefan, 352x240, 15fps, 2200 kbps, 8% error rate
PRISM
PRISM for Wireless Video Broadcast
Yb (“bad” side-information)
Rate = R
X
Decoder Bad
Xb
Decoder Good
Xg
Encoder
Rate = ∆R
Yg (“good” side-information)
• Broadcast source coding studied in information theory literature.
(Heegard & Berger, IT’85, Steinberg & Merhav IT’04)
• Lossy channel: need broadcast source-channel coding view.
– Can use PRISM constructions.
(Majumdar & Ramchandran, ICIP ’04)
• No need to deterministically track Yb and Yg at encoder.
• No need for multiple prediction loops  complexity savings.
• Multiple side-informations at each decoder  motion search at each decoder.
– Standards-compliant implementations possibly using the auxiliary channel
setup.
(Wang, Majumdar, & Ramchandran, ICASSP ’05)