Ch. 6- H.264/AVC
Part I (pp.160~199)
Sheng-kai Lin
Shengkai Lin@CMLAB,
NTUCSIE 2004
1
Outline
• Introduction
• Baseline profile
–
–
–
–
–
–
Reference Picture Management
Slice
Inter-prediction
Intra-prediction
Deblocking filter
Transform and Quantisation
Shengkai Lin@CMLAB,
NTUCSIE 2004
2
H.264/AVC Introduction
(I)
• 4:2:0
• Define VCL and NAL
– VCL: Video Coding Layer
– NAL: Network abstraction
• Variable block size
• Multiple reference frame
• ¼ sample motion compensation
Shengkai Lin@CMLAB,
NTUCSIE 2004
3
H.264/AVC Introduction
(II)
• CAVLC
• CABAC
Shengkai Lin@CMLAB,
NTUCSIE 2004
4
Terms (I)
• Picture (frame)
– one or more slices
• Slice
– group of macroblock (1~ )
– minimal inter-dependency between
coded slices can limit the error
propagation
• Macroblock
Shengkai Lin@CMLAB,
NTUCSIE 2004
5
Terms (II)
• I slice
– I macroblock only
• P slice
– P, I macroblock
• B slice
– B,I macroblock
• SI slice
• SP slice
Shengkai Lin@CMLAB,
NTUCSIE 2004
6
Terms (III)
• I macroblock
– intra prediction from decoded samples
in the current slice
• for entire macroblock
• for each 4*4 block
• P macroblock
– inter prediction using past references
• 16*16, 16*8, 8*16, 8*8, 4*8, 8*4, 4*4
Shengkai Lin@CMLAB,
NTUCSIE 2004
7
Terms (IV)
• B macroblock
– inter prediction using past/ future
references
• list 0
– past reference frame buffer
• list 1
– future reference frame buffer
• RBSP (Raw Byte Sequence Payload)
– NAL payload
Shengkai Lin@CMLAB,
NTUCSIE 2004
8
H.264/AVC profiles
Shengkai Lin@CMLAB,
NTUCSIE 2004
9
H.264/AVC Encoder
Shengkai Lin@CMLAB,
NTUCSIE 2004
10
H.264/AVC Decoder
Shengkai Lin@CMLAB,
NTUCSIE 2004
11
Outline
• Introduction
• Baseline profile
–
–
–
–
–
–
Reference Picture Management
Slice
Inter-prediction
Intra-prediction
Deblocking filter
Transform and Quantisation
Shengkai Lin@CMLAB,
NTUCSIE 2004
12
Reference Picture
Management (I)
• Reference pictures are stored in
decoded picture buffer (DPB)
• short/long term reference picture
• A decoded frame may be marked as
– unused for reference
– short term picture
– long term picture
Shengkai Lin@CMLAB,
NTUCSIE 2004
13
Reference Picture
Management (II)
• “Sliding Window” memory management
– Keep #(long_term_pic+ short_term_pic)
– Remove short term picture if lack of space
• Adaptive memory control
– issued by encoder
– change the type of the ref frame
• IDR (Instantaneous Decoder Refresh)
– clear ref buffer
– I frame
Shengkai Lin@CMLAB,
NTUCSIE 2004
14
Outline
• Introduction
• Baseline profile
–
–
–
–
–
–
Reference Picture Management
Slice
Inter-prediction
Intra-prediction
Deblocking filter
Transform and Quantisation
Shengkai Lin@CMLAB,
NTUCSIE 2004
15
Redundant coded picture
• Send the duplicated part or all of a
coded picture
• Error resilience
Shengkai Lin@CMLAB,
NTUCSIE 2004
16
Arbitrary Slice Order
(ASO)
• The decoding order of the slices
could be arbitrary
• Application example
– reduce end-end transmission delay in RT
app
Shengkai Lin@CMLAB,
NTUCSIE 2004
17
Slice Group (I)
• former called “FMO”
– Flexible Macroblock Ordering
• a subset of the macroblocks and may
contain one or more slices
• Application example
– Error resilience
Shengkai Lin@CMLAB,
NTUCSIE 2004
18
Slice Group (II)
Shengkai Lin@CMLAB,
NTUCSIE 2004
19
Outline
• Introduction
• Baseline profile
–
–
–
–
–
–
Reference Picture Management
Slice
Inter-prediction
Intra-prediction
Deblocking filter
Transform and Quantisation
Shengkai Lin@CMLAB,
NTUCSIE 2004
20
Inter Prediction
• Variable block size
• ¼ pixel motion compensation
Shengkai Lin@CMLAB,
NTUCSIE 2004
21
Interpolation Samples (I)
1. Generate ½ samples by six tap FIR
•
•
b, h, m, s
FIR para :(1/32, -5/32, 5/8, 5/8, 5/32, 1/32)
2. Generate center ½ sample by FIR
•
j
3. Generate ¼ sample by linear
interpolation
Shengkai Lin@CMLAB,
NTUCSIE 2004
22
Interpolation Samples
(II)
Shengkai Lin@CMLAB,
NTUCSIE 2004
23
Motion vector prediction
(I)
• Encoding MV can cost a lot of bits
– especially small partition sizes
• Neighboring MV is highly correlated
– predictable
– encode the diff MVD and transmit it
Shengkai Lin@CMLAB,
NTUCSIE 2004
24
Motion vector prediction
(II)
• Partitions excluding 16 × 8 and 8 × 16,
MVp = (MVA+ MVB + MVC) /3
• For 16 × 8 partitions, MVp of the
upper 16 × 8 = MVB ;MVp for the
lower 16 × 8 = MVA
Shengkai Lin@CMLAB,
NTUCSIE 2004
25
Motion vector prediction
(III)
• For 8 × 16 partitions, MVp for the
left 8 × 16 = MVA ;MVp for the right
8 × 16 = MVC
• For skipped macroblocks, do as 16 ×
16 Inter mode
Shengkai Lin@CMLAB,
NTUCSIE 2004
26
Motion vector prediction
(IV)
Shengkai Lin@CMLAB,
NTUCSIE 2004
27
Outline
• Introduction
• Baseline profile
–
–
–
–
–
–
Reference Picture Management
Slice
Inter-prediction
Intra-prediction
Deblocking filter
Transform and Quantisation
Shengkai Lin@CMLAB,
NTUCSIE 2004
28
Intra Prediction
• For Luma samples
– 4*4 block
• 9 prediction modes
– 16*16 block
• 4 modes
– I_PCM
• transmit the encoded samples w/o pred. &
trans
Shengkai Lin@CMLAB,
NTUCSIE 2004
29
4*4 luma prediction
modes
Shengkai Lin@CMLAB,
NTUCSIE 2004
30
16*16 prediction modes
Shengkai Lin@CMLAB,
NTUCSIE 2004
31
8*8 Chroma Prediction
Modes
• Similar to 16*16 luma intraprediction except the different
numbering.
–
–
–
–
DC(mode 0)
horizontal (mode 1)
vertical(mode 2)
plane (mode 3).
Shengkai Lin@CMLAB,
NTUCSIE 2004
32
Signaling Intra Prediction
Modes (I)
• The choices of the mode should be signaled to the
decoder.
– might cost lots of bits.
• Intra modes for neighboring 4 × 4 blocks are
often correlated
– predict the parameters !
• The prediction mode for luma coded in Intra16×16 mode or chroma coded in Intra mode is
signalled in the macroblock header and the
following method is not used.
Shengkai Lin@CMLAB,
NTUCSIE 2004
33
Signaling Intra Prediction
Modes (II)
B
A
C
• If A and B are available, C = min (A,B)
• else if (neither A nor B are available) C = 2
(DC)
• else C = available (A,B)
Shengkai Lin@CMLAB,
NTUCSIE 2004
34
Signaling Intra Prediction
Modes (III)
• Flag prev_intra4×4_pred_mode
– 1: most probable prediction mode is used.
– 0: Flag rem_intra4×4_pred_mode
indicates the change of the mode
Shengkai Lin@CMLAB,
NTUCSIE 2004
35
Signaling Intra
Prediction Modes (IV)
• If rem_intra4×4_pred_mode <
most_probable_mode, the prediction
mode = rem intra4×4 pred mode
• else the prediction mode =
(rem_intra4×4_pred_mode+1)
Shengkai Lin@CMLAB,
NTUCSIE 2004
36
Outline
• Introduction
• Baseline profile
–
–
–
–
–
–
Reference Picture Management
Slice
Inter-prediction
Intra-prediction
Deblocking filter
Transform and Quantisation
Shengkai Lin@CMLAB,
NTUCSIE 2004
37
Deblocking filter
•
•
•
•
Filter 4 vertical/horizontal boundaries of luma
Filter 2 vertical/horizontal boundaries of chroma
Affect up to 3 samples on the either side.
The filter is stronger at places where there is
likely to be significant blocking distortion
– e.g.: such as the boundary of an intra coded macroblock
or a boundary between blocks that contain coded
coefficients.
Shengkai Lin@CMLAB,
NTUCSIE 2004
38
Outline
• Introduction
• Baseline profile
–
–
–
–
–
–
Reference Picture Management
Slice
Inter-prediction
Intra-prediction
Deblocking filter
Transform and Quantisation
Shengkai Lin@CMLAB,
NTUCSIE 2004
39
Transform and
Quantisation
• 3 transforms
– DCT-base transform for all 4*4 residual
block
– Hadamard transform for 4*4 luma DC
coefficient (in 16*16 intra)
– Hadamard transform for 2*2 chroma DC
coefficient
Shengkai Lin@CMLAB,
NTUCSIE 2004
40
DCT-base transform (I)
• Integer transform
• Might be zero mismatch
• The code part of the algo need add
and shift only
• Scale multiplication could be
integrated into quantiser.
• Inverse operation could be down by
16-bit arithmetic.
Shengkai Lin@CMLAB,
NTUCSIE 2004
41
DCT-base transform (II)
• a=1/2, b = (2/5)1/2, d = 1/2
Shengkai Lin@CMLAB,
NTUCSIE 2004
42
Quantisation
• Combine with the scale of the DCT
• Skip the rest details
Shengkai Lin@CMLAB,
NTUCSIE 2004
43
4*4 DC luma Transform
and Quantisatoion
• |ZD(i, j)| = (|YD(i, j)| MF(0,0) + 2f ) >> (qbits +1)
• sign (ZD(i, j)) = sign (YD(i, j))
Shengkai Lin@CMLAB,
NTUCSIE 2004
44
2*2 DC chroma Transform
and Quantisatoion
• |ZD(i, j)| = (|YD(i, j)| MF(0,0) + 2f ) >> (qbits +1)
• sign (ZD(i, j)) = sign (YD(i, j))
Shengkai Lin@CMLAB,
NTUCSIE 2004
45
• Thank you
• Next time, we’ll start at CAVLC
• Bye…
Shengkai Lin@CMLAB,
NTUCSIE 2004
46