ELE488 – F 06
MPEG Project
This laboratory assignment is intended to introduce you to the fundamentals of video
coding and processing. Portions of this lab will require MATLAB. The required
laboratory files can be downloaded from the Blackboard course web site under the
Assignments tab. The executable files included and the sample video files are from the
companion web site to Yao Wang’s textbook, Video Processing and Communications.
The sample video files you will be using for this lab are uncompressed .yuv files, in
which each frame (there are 90 for each movie) is represented by three matrices
corresponding to the luminance component, Y, and two chrominance components, U (Cr)
and V (Cb). The components of each frame are then concatenated and the frames are
stored sequentially in one file.
The video file resolution for the luminance component uses the Common Intermediate
Format (CIF), which is 352 x 288 pixels. The chrominance components (U and V) are
subsampled with 4:2:0 (in which each 2 x 2 block of luminance values shares a single
value from the U and V components). Thus, the U and V components are both of size
176 x 144.
QCIF, which stands for Quarter CIF, is another popular video resolution format. Both
the width and height resolutions are half that of CIF, so its overall resolution is a
“quarter” that of CIF.
1. Generating QCIF video sequences
This portion of the lab will require MATLAB. You will use the input video
sequences football_cif.yuv and stefan_cif.yuv.
The first step is to load the input video sequences into MATLAB. You may use
the provided m-file yuv2mat.m which is prototyped as follows:
function [Y,U,V] = yuv2mat(filename, format)
where filename is the (string) name of the video sequence file, and format is a
string representing the resolution format of the sequence. For the input videos,
use ‘cif’ as format.
The function returns the Y, U, and V components of the video as 3-dimensional
matrices, so that Y(:, :, 1) is the luminance component of the first frame.
Write a MATLAB function cif2qcif.m:
function [YQ, UQ, VQ] = cif2qcif(Y, U, V)
which takes the Y, U, and V components of a video sequence in CIF format and
returns the components in QCIF format.
Once you have the QCIF components, save them out to file using the provided mfile mat2yuvfile.m which is prototyped as follows:
function mat2yuvfile(filename, Y, U, V)
IMPORTANT: Name the football QCIF video sequence football_qcif.yuv and the
Stefan video sequence stefan_qcif.yuv
2. Generate the DC frame video
In this section, you will be asked to compute the DC frame video of a video
sequence. The DC frame video is a very crude approximation of the original
video sequence and is sometimes used for fast video indexing/searching. The DC
frame video is computed by replacing each 8 x 8 sub-block of every frame with
its average value (or DC value) – hence the name. Therefore, the DC frame video
subsamples by 8 in both directions.
Write a function getDCVideo.m:
function DC = getDCVideo(X)
where X is a 3-dimensional matrix (e.g., the Y component of a video sequence).
The output matrix should be the DC frame video of that component.
Save the DC frame video out to a file with the following (using football video):
YDC = getDCVideo(Y);
UDC = getDCVideo(U);
VDC = getDCVideo(V);
mat2yuvfile(‘football_dc.yuv’, YDC, UDC, VDC);
Repeat for the Stefan video.
a) How would you obtain the DC frame video from the set of DCT coefficients?
3. Playing the uncompressed video sequences
Open the executable YUVviewer.exe. Play both video sequences in both CIF and
QCIF format as well as the DC frame video for both.
IMPORTANT: Make sure the Frame Size is set to the correct setting (CIF /
QCIF) before loading each of the videos. For the DC video, you will need to type
in the frame size manually.
4. Compressing the video sequences to MPEG
To encode the video sequences to MPEG, we will be using the executable
mpeg2encoder.exe. Open an MS-DOS console by typing “command” into the
Windows Run dialog box. Then change to your current directory by typing “cd
<dirname>”
mpeg2encoder takes two command line arguments – the parameter filename and
the output movie filename. The parameter file contains many important settings
for the encoder (such as bit rate, frame size, number of frames, etc.). A sample
parameter file for both the football_cif.yuv and stefan_cif.yuv videos are provided
in the params subdirectory.
To encode the two CIF video sequences to MPEG, type the following:
mpeg2encoder <parfile>.par <mpegfile>.mpg
For example, to encode the football CIF video to mpeg, type:
mpeg2encoder params\football_cif.par football_cif.mpg
When you compress, give each mpg video a descriptive filename.
a) Compress the CIF and QCIF versions of both videos to MPEG using their
corresponding parameter files. Save the videos as football_cif.mpg,
football_qcif.mpg, stefan_cif.mpg, and stefan_qcif.mpg.
b) Determine the resulting compression ratio for the encoded MPEG videos.
c) Modify the parameter files for the CIF videos to generate MPEG videos of
compression ratios of 10:1 and 60:1. Save the mpeg videos as
football_cif_10.mpg, football_cif_60.mpg, stefan_cif_10.mpg and
stefan_cif_60.mpg. Explain what you did to attain these ratios.
Hint: the bit rate is a very important parameter!
d) Also included in the params subdirectory are two parameter files for the DC
frame video sequences (football_dc.par and stefan_dc.par). Modify these files
to generate compression ratios of 10:1 and 60:1, as well as approximately the
same compression ratio you calculated in part (b). Save these videos as
football_dc_10.mpg, football_dc_60.mpg, football_dc.mpg,
stefan_dc_10.mpg, stefan_dc_60.mpg, and stefan_dc.mpg.
e) When you have finished encoding, watch all of the MPEG videos and
comment about the differences you notice between different compression
ratios, CIF vs. QCIF vs. DC, etc.
5. Decompress MPEG sequences back to YUV
To decode the MPEG videos, we will use the executables mpeg2decoder.exe and
YUVmixer.exe. This process will reconstruct an approximation of the original
uncompressed .yuv files from the MPEG compressed video. This reconstruction
will allow us to compare with the original videos and measure the distortion
introduced by the lossy MPEG compression.
To decompress one of the MPEG sequences, type the following:
mpeg2decoder –b <mpegfile>.mpg –o0 decoded\<file>_recon%d
YUVmixer decoded\<file>_recon 0 89
For example, to decode the CIF version of the football movie, type:
mpeg2decoder –b football_cif.mpg –o0 decoded\football_cif_recon%d
YUVmixer decoded\football_cif_recon 0 89
This will generate the reconstructed .yuv file football_cif_recon.yuv in the
decoded subdirectory.
Decompress the CIF and DC videos for all three compression ratios.
Once you have decompressed the video sequences, play them using
YUVviewer.exe as before and compare with the original video sequences.
6. PSNR Computations
Peak signal-to-noise ratio (PSNR) is frequently used as a measure of the distortion
introduced by MPEG compression.
To compute PSNR, we first need to calculate the mean squared error (MSE)
between the original video sequence and the reconstructed sequence. For video
sequences, MSE is simply the average squared pixel-by-pixel difference, which is
a measure of the noise power introduced.
Given the MSE, PSNR is defined as
PSNR = 10 log(v^2/MSE)
where the log is to base 10 and v is the maximum intensity value of the video
signal. In our case, we are working with 8-bit intensity values, so v = 255. Hence
the name PSNR, since this measures the maximum attainable SNR.
For color video, it is typically acceptable to compute PSNR using the luminance
component only.
Write a MATLAB function computePSNR:
function PSNR = computePSNR(Y1, Y2)
which takes as input the luminance component of two video sequences (e.g. the
original and reconstructed video sequences) and returns the PSNR.
The function should also compute and plot the PSNR versus frame number. Here
the PSNR for a given frame uses the MSE computed with respect to that frame
only.
IMPORTANT: Do not compute the overall PSNR as the average PSNR of the
frames. Explain why this is a bad idea.
a) Compute the PSNR between football_cif.yuv and football_cif_recon.yuv.
Load the files into MATLAB using the function yuv2mat as in Step 1.
Use the plot of PSNR versus frame number to guess the number of frames per
GOP (the actual GOP length is in the parameter file). Also, can you guess
which frames are the I-frames? Explain your reasoning.
b) You should have reconstructed video sequences at three different compression
ratios for the CIF and DC videos. When loading the DC videos, be sure to use
‘dc’ as the format when calling yuv2mat. Compute the PSNR and plot PSNR
versus compression ratio (label axes and title the plots accordingly). How
does the plot of PSNR versus compression ratio for the DC video compare
with that of the CIF video? Explain your results.
c) If you were to plot the PSNR versus compression ratio, as in part (b), for the
QCIF videos, where would you expect it to fall with respect to the CIF and
DC videos?
d) PSNR is a quantitative measure of the distortion introduced by MPEG
compression, but may not always be an accurate measure of the perceptual
distortion introduced. Use YUVviewer.exe to watch the reconstructed video
sequences and compare your opinion of the quality with the PSNR calculated.
Briefly comment.