MPEG-1 Video (Part 1)
Ketan Mayer-Patel
CS 294-9 :: Fall 2003
Encoding Techniques
•
•
•
•
•
•
Subsampling
Transform Coding
Run-length Encoding
Predictive Encoding
Entropy Encoding
Quantization
CS 294-9 :: Fall 2003
Bitstream Organization
Picture Data
Seq. Header
Width
Height
Frame Rate
Buffer Control
Seq. End Code
GOP Header
Time Code
Picture Header
Temporal Ref
Picture Type
Motion Vector Parameters
All headers begin with 23 zeroes followed by 9
bits that indicate header type. Encoding
process will never produce 23 zeroes.
CS 294-9 :: Fall 2003
Frame Types
• 3 Frame Types: I, P, B
I : All information for frame present.
P: Predictively encoded from previous I or P.
B: Predictively encoded from
previous I or P and next I or P.
I
B B P
B B P B B P B B P B B I
CS 294-9 :: Fall 2003
Frame Order
• Predictive relationships create an obvious
problem: B-frames depend on the future.
• Obvious solution: send the frames out of order.
I B B P
1 2 3 4
1 3 4 2
B B P B B P B B P B B I
5 6 7 8 9 10 11 12 13 14 15 16
6 7 5 9 10 8 12 13 11 15 16 14
CS 294-9 :: Fall 2003
Source Input
• Before we describe how I-frames are
encoded, we should describe our input.
• 3 planes of Y, U, V
– 8 bits per pixel.
– Y range [0,255].
– U and V range [-128,127]
• Planes are all of the same size.
• Pixels colocated between frames.
CS 294-9 :: Fall 2003
Chrominance Subsampling
•
•
•
•
•
•
First step: downsize chrominance.
4:2:0 (with chrominance samples centered)
Requires bilinear interpolation.
U and V biased by 128 to put in range [0,255]
Compression Ratio: 2:1
Wow, doing well already.
CS 294-9 :: Fall 2003
Subsampling In General
• Severe loss of data.
• Exploits imperceptibility of data loss.
– In this case: human not as sensitive to color.
• What if we were using images as input to
feature extractor?
– Depending on what was being extracted,
subsampling might not be such a good idea.
• Compression gain is directly related to
subsampling factors.
CS 294-9 :: Fall 2003
Macroblocks
• Y is cut into 8x8 tiled pixel regions.
• U and V cut into 8x8 tiled pixel regions.
• Macroblock defined as 4 Y tiles that form a
16x16 pixel region and associated U and V tiles.
• Macroblocks organized in row order fashion
from top to bottom.
• Compression gain: none.
CS 294-9 :: Fall 2003
Discrete Cosine Transform
• Each tile (aka block) in a macroblock is
transformed with a 2D DCT.
• DCT is an orthonormal, separable, frequency
basis much like a Fourier transform.
• 1-D case: 8 pixel values are transformed into
8 DCT coefficients.
• 2-D case: apply 1-D transform to all of the
rows and then apply 1-D transform to all of
the columns.
CS 294-9 :: Fall 2003
DCT Basis Functions
2x 1
* 0 )
16
2 2 x 0
7
2x 1
s
(
x
)
cos(
* )

16
x 0
7
2x 1
s
(
x
)
cos(
* 2 )

16
x 0
7
2x 1
s
(
x
)
cos(
* 3 )

16
x 0
7
2x 1
s
(
x
)
cos(
* 4 )

16
x 0
7
2x 1
s
(
x
)
cos(
* 5 )

16
x 0
7
2x 1
s
(
x
)
cos(
* 6 )

16
x 0
7
2x 1
s
(
x
)
cos(
* 7 )

16
x 0
1
s ( 0)
s (1)
s ( 2)
s (3)
s ( 4)
s (5)
s ( 6)
s (7 )
7
 s( x) cos(
CS 294-9 :: Fall 2003
S (0) DC
S (1)
S (2)
S (3)
S (4)
S (5)
S (6)
S (7 )
AC
DCT Properties
• 8-bit pixel values produces 12-bit signed
coefficient values.
• Fast algorithms exist for computation.
– 13 multiplies and 29 additions
– Fixed point integer math.
• Good perceptual properties.
– Losing higher freq. results in a bit of blurring.
– Ringing fairly minimal.
CS 294-9 :: Fall 2003
Transform Coding Properties
• No loss of data
– Except for numerical errors
• No compression either.
• Used to rearrange the data into a form to
make another coding technique more
effective.
CS 294-9 :: Fall 2003
Coefficient Quantization
• Each block is now 64 coefficients instead of 64
pixel values.
• Each coefficient quantized independently.
– Allows larger quantization factors to be used with higher
frequency coefficients.
• Quantization is controlled by two parameters:
– Quantization table.
• Set in picture header or system header.
• Two different tables, one each for intra and non-intra blocks.
– Quantization factor.
• Can be set on a per macroblock basis. Used to scale the table.
• Can take value from (2-62)
CS 294-9 :: Fall 2003
Quantization Properties
• Data loss relative to quantization step.
• Compression in two ways:
– Smaller range to represent.
• In our case 12 bit signed values turn into 9-bit
signed values.
– Creates runs of the same number.
– In our case: runs of zeroes.
CS 294-9 :: Fall 2003
Run Length Encoding
• High quantization step size for higher frequency
components results in lots of zero coefficients.
• Run Length Encoding provides better
representation.
– Convert 2D matrix into 1D ordering of coefficients.
– Reorganize as (run, value) pairs.
– Run specifies number of zeroes to insert in the
ordering before value appears.
– Special marker that indicates nothing left but zeroes.
CS 294-9 :: Fall 2003
Zig-Zag ordering
• In order to group as many of the zeroes
together, zig-zag ordering used.
CS 294-9 :: Fall 2003
RLE Properties
• Compression related to avg. size of run.
• No data loss.
CS 294-9 :: Fall 2003
DC Term Encoding
• At this stage, each block in our macroblock
is represented as a set of RLE’d DCT
coefficients.
• DC term is always coded even if it is zero.
– Coded as difference between last DC term and
current DC term.
– Blocks are ordered within a macroblock.
• Why code the difference?
– Avg. pixel value of one block is likely to be
correlated to nearby block.
CS 294-9 :: Fall 2003
DC Term Encoding Cont’d
• Now DC term is expressed as difference
from previous DC term (DC_DIFF)
• Encoded as two parts:
– Size of difference (i.e., log(DC_DIFF))
– Size number of bits that provides the value.
• Size is encoded as a Huffman code.
CS 294-9 :: Fall 2003
Differential Encoding
• Useful when values being encoded are well
correlated.
• Distribution of differences is expected to
not be uniform.
• No compression per se, but increases the
efficiency of entropy encoding techniques
(i.e., Huffman coding)
CS 294-9 :: Fall 2003
AC Term Encoding
• AC terms are given as (run,value) pairs.
• Encoded in one of two ways:
– Huffman code for (run, abs(value)) followed by single bit
for sign of value.
– Special Huffman code indicating ESCAPE, followed by 6
bits for run and either 8 or 16 bits for value.
• 6 bits for run simply encode 0 through 63
• First 8 bits of value put value at –128 to 127.
• If first 8 bits is -128, next 8 bits provide codes for –128 through –
255
• If first 8 bits is 0, next 8 bits provide codes for 128 through 255.
CS 294-9 :: Fall 2003
Entropy Coding
• Huffman codes are a form of entropy encoding.
• Relies on uneven distribution of values to be
encoded.
• Length of code associated with values inversely
related to weight in distribution.
– The more likely the value is to occur, the small the
code length relative to all the other codes.
• No data loss.
• Compression depends on distribution.
CS 294-9 :: Fall 2003
Stepping Back A Bit
Picture Header
Picture Data
Row Major Scan of Encoded Macroblocks
First Non-zero AC Coeff.
(variable bit length)
Luminance Blocks
U Block V Block
Last Non-zero AC Coeff.
(variable bit length)
Q Scale (5 bits)
Macroblock Type (1 or 2 bits)
Macroblock Address Increment (1-bit)
DC Bits (0-8 bits)
DC Size (2-7 bits)
CS 294-9 :: Fall 2003
EOB (2 bits)
Slices
• One last level of organization.
• Macroblocks grouped into slices.
– Typically, one row of macroblocks in one slice.
– Other groupings also possible.
• Slice starts with a slice header.
– Contains qscale. and indicates row in which slice starts.
• Decoder state is reset.
– DC predictors for Y, U, and V set to 1024.
– Prev. macroblock address set to address of first macroblock
in slice row (may not be first macroblock in slice).
CS 294-9 :: Fall 2003
I-Frame Review
• All macroblocks are intra-coded.
• Blocks DCT’d and quantized to produce
coefficients.
• DC terms encoded differentially.
• AC terms encoded with entropy codes
associated with (run,value) pairs.
– Escape code with fix length encoding for
seldom used possibilities.
• In general, compression ratio is 10:1 to 20:1
CS 294-9 :: Fall 2003