An Introduction to H.264/AVC
and 3D Video Coding
Outline

Video Coding Concepts




H.264/AVC Introduction







inter prediction
intra prediction
transform & quantization
de-blocking filter
entropy coding
3D Video Coding



history
performance comparison
H.264/AVC Coding Tools


basic concept review
image coding structure
video coding structure
3D video format
multiview video coding
Summary with Q&A
Video Coding Concept
-basic concept review
-image coding structure
-video coding structure
The Scope of Image and Video
Coding Standardization

Only the Syntax and Decoder are standardized:
Images and Video
Needs for Video Compression


Without compression
 Visual telephony (e.g. CIF @ 15 frames/s):
 325 (pels) x 288 (lines) x 15 (farmes/s) x 1.5 bytes =
18.25 Mbit/s
 Digital TV (ITU-T 601 4:2:0 @30 frames/s):
 720 (pels) x 480 (lines) x 30 (farmes/s) x 1.5 bytes =
124.4 Mbit/s
 HDTV (e.g. 1280x720 pels 4:2:0 @ 60 frames/s):
Compression results in lower bit rates
 Lower transmission and storage cost
RGB vs. YCbCr [1/3]
RGB vs. YCbCr [2/3]
RGB vs. YCbCr [3/3]
Common YCbCr Formats
Subjective View
Block Based Coding [1/2]
Block Based Coding [2/2]
Group of Picture (GOP)
Video Coding Concept
-basic concept review
-image coding structure
-video coding structure
Image Coding Structure
Transform
Quantization
S:
0 1 2 3 4 5 6 7 (3 bits)
Quantization:
Quantization step-size Q=2: S/2
Quantization Levels (Q):
Inverse quantization (x2):
Quantization error:
0 0 1 1 2 2 3 3 (2 bits)
00224466
01010101
Quantization step-size Q=4: S/4
Quantization Levels (Q):
Inverse quantization (x4):
Quantization error:
0 0 0 0 1 1 1 1 (2 bits)
00004444
01230123
Effect of DCT + Quantization
Entropy coding
Video Coding Concept
-basic concept review
-image coding structure
-video coding structure
Temporal Redundancy [1/2]

The amount of data to be coded can be reduced significantly
Standard Video Encoder
Block Based Motion Compensation [1/2]
Algorithms for Motion Estimation

Full Search
 Guarantee
find the global minimum SAD
 high computational complexity

Fast Search
 Local
minimum SAD
 Low computational complexity
 Reduce candidate blocks
 Reduce matching pixels in candidate blocks
Diamond Search
Video coding structure
H.264/AVC Introduction
-History
-Performance comparison
History
Joint Video Team
MPEG-2 Has Hit A Wall
MPEG-4 in Comparison
H.26L Provides Focus
MPEG-4 “Adopts” H.264
State of the Art Standards

MPEG-2
 DVD,

DVT, since 1994
MPEG-4
 DVR,
Digital Still Camera, since 1999
 ~1.5x coding gain over MPEG-2 (ASP)

MPEG-4 part 10, AVC (H.264)
 Mobile
video, DVB-H, Blu-ray Disc and etc.
 2~3x coding gain over MPEG-2
AVC Profiles
coding tools and profiles
H.264/AVC Introduction
-History
-Performance comparison
Compare to Other Standard

Fair comparisons of H.26L(TML-8.0) versus H.263v3,MPEG-2,and
MPEG-4

TML-8.0 at half of the bit rate as MPEG-4 for the same visual fidelity
 Source from VCEG-N18.doc (Soptember,2001)

Objective evaluation

Average improvement of TML-8.0over MPEG-2 (VM-5) of 5.8 dB PSNR
(peak gain 7.2 dB) for equal bandwidths
 TML-8.0 average gain of 3.1 dB relative to H.263++ (High-Latency Profile)
for equal bandwidths (up to 5.2 dB)
 Gain of 2.2 dB over MPEG-4 (Advanced Simple Profile) for equal
bandwidths (max. 3.6 dB)
Test Sets

“Streaming” Test:

Four QCIF sequences coded at 10 Hz and 15 Hz (Foreman, Container, News,
Tempete)
 Four CIF sequences coded at 15 Hz and 30 Hz (Bus, Flower, Garden, Mobile
and Calendar, and Tempete)
 With B frame

“Real-Time Conversation” Test:

Four QCIF sequences encoded at 10Hz and 15Hz (Akiyo, Foreman, Mother
and Daughter, and Silent Voice)
 Four CIF sequences encoded at 15Hz and 30Hz (Carphone, Foreman, Paris,
and Sean)
 Without B frames
Objective evaluation [1/2]
Objective evaluation [2/2]
Subjective evaluation

Example: Sequence Mobile, frame 40
Perceptual Test of H.264/AVC High Profile
Objective Performance of H.264/AVC High Profile
Intra mode performance [1/2]

Average gain of H.264 to JPEG: 5.2 dB (luma)
 Average gain of H.264 to JPEG2000: 1.12 dB (luma)
 Average gain of Motion JPEG2000 to H.264: 1.42 dB (chroma)
 The smaller the bit rate, the higher the gain of H.264
Intra mode performance [2/2]
Intra mode performance [chroma]
Intra mode performance [FRExt]

a set of 8 photographic monochrome test images


with resolutions from 512x512 up to 2048x3072 samples
Average gain of H.264/AVC HP to JPEG2000: 0.5 dB

over the entire test image set and all bit-rates
JPEG2000 vs. H.264 Intra
H.264/AVC Coding Tools
-Inter prediction
-Intra prediction
-Transform and Quantization
-De-blocking Filter
-Entropy Coding
Basic Coding Structure
Standard Tools Comparison
Motion Compensation
Macro Block Partitions
Example – Frame 1
Example – Frame 2
Example – Residual [no MC]
Example – Residual [16x16]
Example – Residual [8x8]
Example – Residual [4x4]
Example of Variable Block Sizes

Large block means

Less bits for MVs
 More bits on residuals

Small block means

More bits for MVs
 Less bits on residuals
Summary

Key Features




50% bit rate saving against any other standards, by




Enhances motion compensation
Small blocks for transform coding
De-blocking filter
Better prediction
More computation
More memory
Video coding layer is still based on hybrid video coding
algorithm, buy with important differences