REVISED
Homework Set 3 Solutions
EECS 455
Oct. 25, 2006
Revisions to solutions to problems 2, 6 and marked with
***
1. Let U be a continuous random variable with pdf pU(u). Consider an N-point quantizer for U with
levels x^ 1, ..., ^xN and thresholds a1, ..., aN whose MSE is denoted DU.
Let V = bU + c where b>0 and c are constants. Thus, V is a scaled and shifted version of U. From
random variable theory, we know that the pdf of V is related to that of U via
1
v-c
pV(v) = b pU( b )
That is the pdf of V is a scaled and shifted version of the pdf of U.
Now consider a scaled and shifted quantizer for V with levels b x^ +c, ..., b ^x +c and thresholds
1
b a1+c, ..., aN+c whose MSE is denoted DV.
N
(a) Sketch an example of pU(u) and the corresponding pV(v).
= 2U +1
U (u)
V(v)
1
1
2
2
2
2
2
(b) Show that σV = b σU , where σU and σ V denote the variances of U and V, respectively.
2
2
2
2
σV = E (V-E[V]) = E (bU+c - E[bU+c]) = E (bU+c-bE[U]-c) since E[bU+c]=bE[U]+c
= E (b(U-E[U]))
2
2
2
2
2
= b E (U-E[U]) = b σU
2
(c) Show that DV = b DU .
DV =
=
b ai+c
b ai+c
N
v-c
2 1
^x -c)2 p (v) dv =
(v
b
∑ ∫
i
V
∑ ∫ (v - b ^xi-c) b pU( b ) dv
i=1 b ai-1+c
i=1 b ai-1+c
N
ai
N
∑ ∫
i=1 ai-1
= b
N
2
2 1
(bu+c - b x^i-c) b pU(u) du
ai
∑ ∫
i=1 ai-1
v-c
with u = b , so v = bu+c
2 1
2
(u - ^xi) b pU(u) du = b DV
*
*
(d) Let DU,N, respectively DV,N, denote the least MSE of any quantizer with N levels for U,
respectively V. Show
*
2
*
DV,N ≤ b DU,N
Let ^x 1 , ..., ^xN and a1, ..., aN denote the levels and thresholds of an optimal N-point quantizer for U.
*
It has distortion DU = DU,N.
Consider the scaled and shifted quantizer for V with levels
b x^ +c, ..., b x^ +c and thresholds b
1
a1+c, ..., aN+c whose MSE is denoted DV.
N
2
From Part (c) we know DV = b DU .
We also know that since D*V,N is the least distortion of any N point quantizer for V, D*V,N ≤ DV.
Putting all these facts together we find
*
2
2
*
DV,N ≤ DV = b DU = b DU,N
1
(e) Show
*
2 *
DV,N ≥ b DU,N
1
c
We note that U = b V - b and we use the same approach as in Part (d) except b is replaced by 1/b
and c is replaced by -c/b.
Let ^x , ..., ^x and a , ..., a denote the levels and thresholds of an optimal N-point quantizer for V.
1
N
1
*
It has distortion DV = DV,N.
N
Consider the scaled and shifted quantizer for U with levels
(a1-c)/b, ..., (aN-c)/b whose MSE is denoted DU.
( x^ 1-c)/b, ..., (x^ N-c)/b and thresholds
From Part (c) we know DU = DV/b2 .
We also know that since D*U,N is the least distortion of any N point quantizer for U, D*U,N ≤ DU.
Putting all these facts together we find
*
DU,N
≤ DU = DU/b
Important Notes:
2
= D*U,N /b
2
*
2 *
1. We conclude from parts (c) and (d) that DV,N = b DU,N.
2. We also conclude that if we have a quantizer that is optimal for one pdf, then an optimal quantizer for
a scaled and shifted pdf is simply a scaled and shifted version of the original quantizer.
2. Problem 7.9, p. 371. Note that PX is just the variance of the random variables being quantized. You
will need to make use of the conclusions from Problem 1.
(1) We need to design an optimal 16-level uniform scalar quantizer for a Gaussian source with variance
σ2 = 10. The step size of this quantizer is σ = √
 10 times the step size 0.3352 from Table 7.1. The
other issue is the offset, i.e. the location of the levels. Since the Gaussian density is symmetric about 0
we'll make the levels and thresholds symmetric about 0. The result is
thresholds
-7.42 -6.36 -5.30 -4.24 -3.18 -2.12 -1.06 0 1.06 2.12 3.18 4.24 5.30 6.36 7.42
levels
-7.95 -6.89 -5.83 -4.77 -3.71 -2.65 -1.59 -0.53 0.53 1.59 2.65 3.71 4.77 5.83
***(2)
***(3)
6.89 7.95
The resulting distortion is σ2 = 10 times the distortion 0.01154 of Table 7.1, which is 0.1154.
With 8 levels, MSE = 10 × 0.03744 = 0.3744. The improvement in SNR is
0.3744
10 log10 0.154 = 5.88 dB
3. Problem 7.11, p. 372
(1) We need to find an optimal 16-level nonuniform scalar quantizer for a Gaussian with variance σ2 =
10, we take the levels and thresholds found in Table 7.2 and multiply them by σ = √
 10.
thresholds
-7.59 -5.83 -4.54 -3.48 -2.53 -1.65 -0.82 0 0.82 1.65 2.53 3.48 4.54 5.83 7.59
levels
-8.64 -6.54 -5.12 -3.97 -2.98 -2.08 -1.23 -0.41 0.41 1.23 2.08 2.98 3.97 5.12 6.54 8.64
(2) The resulting distortion is σ2=10 times the distortion 0.00947 of Table 7.2, namely, 0.0947.
(3) With 8 levels, MSE = 10 × 0.03454 = 0.3454. The improvement in SNR is
0.3454
10 log10 0.0947 = 5.62 dB
2
3/2
4. Show that for a Gaussian density, β = 2# 3 , where β is the term appearing in the high-resolution
formula for the distortion of an optimal nonuniform quantizer with N levels, when N is large, namely
D
1 1
β
12 N2
≅
(this is called the Panter-Dite formula)
Let the Gaussian density be p(u) =
1
β = 2
σ
u2
exp{- } with variance σ2 = 1. Then
2
 2"
√
1
∞
 ∞ 1/3
3
3
u2
1 
1



p (u) du = 1 ∫
exp{}
du
∫
6




(2")1/6
 -∞

 -∞

∞
 1
3
u2
1


=
2"3
exp{}
du
√


∫
2×3
 (2")1/6

2"3

√


-∞

3/2
notice that the integral equals one
because it is the integral of a Gaussian
= 2"3
5. Consider a Gaussian density with variance σ2 = 1
There were errors in the formulas given in class for ∆ and D: Here are the correct formulas:
For an optimum uniform scalar quantizer for a Gaussian source with variance 1,
∆ ≅ 4σ
ln N
√
 
4 ln N
4
R
and D ≅ σ 2 3 N2 = σ 2 3 (ln 2) 2R
2
N
(a) For an optimal uniform scalar quantizer for this pdf, complete the following table:
∆
R
1
2
3
4
5
from
Table 7.1
∆
MSE
MSE
from
∆2/12
from
Table 7.1
from
class
formula
MSE
from
class
formula
%
%
%
accuracy accuracy accuracy
of col 3 of col 5 of col 6
1.596
0.2123
0.3634
1.665
0.231
41.6%
4.3%
0.9957
0.0826
0.1188
1.177
0.116
30.5%
18.2%
36.4%
2.8%
0.586
0.0286
0.0374
0.721
0.043
23.6%
23.0%
15.7%
0.3352
0.0094
0.0115
0.416
0.014
18.9%
24.2%
25.1%
0.1881
0.0029
0.0035
0.233
0.005
15.5%
23.7%
29.3%
Comment on the accuracy of the formulas in columns 3, 5 and 6.
The accuracy of Column 3 improves as R increases. For R = 5, it is accurate to about 15%.
The Columns 5 & 6 do not show accuracy improving as R increases. The theory says the formulas
become accurate for large R. Apparently, R must be larger than 5.
(b) Complete the following table for the same situation:
R
1
2
3
4
5
∆
SNR in
SNR in
from
dB from dB from
Table 7.1 ∆2/12 Table 7.1
∆
from
class
formula
SNR in |col3-col4| |col5-col2| |col6-col4|
dB from
class
formula
1.596
6.7
4.4
1.665
6.4
2.3
4.3%
2.0
0.9957
10.8
9.3
1.177
9.4
1.6
18.2%
0.1
0.586
15.4
14.3
0.721
13.6
1.2
23.0%
0.6
0.3352
20.3
19.4
0.416
18.4
0.9
24.2%
1.0
0.1881
25.3
24.6
0.233
23.5
0.7
23.7%
1.1
Comment again on the accuracy of the formulas in columns 3, 5 and 6.
The accuracy of Column 3 improves as R increases. For R = 5, it is accurate to about .7dB.
The Columns 5 & 6 do not show accuracy improving as R increases. The theory says the formulas
becomes accurate for large R. At R = 5, the accuracy of column 6 for SNR is about 1 dB. Apparently,
R must be larger than 5 for greater accuracy.
3
(c) For an optimal nonuniform quantizer for the same pdf, complete the following table:
R
1
2
3
4
MSE
MSE
from
Table 7.2
from
class
formula
0.3634
0.679
0.1174
0.170
44.6%
0.03454
0.042
22.9%
0.009497
0.011
11.7%
We used the Panter-Dite formula D
col 3
accuracy
86.9%
≅
-2R
1 2
3/2
, with β = 2" 3 = 32.6.
σ β2
12
Comment on the accuracy of the formula in column 3.
The accuracy improves steadily as R increases. It's about 10% when R=4.
(d) Complete the following table for the same situation:
R
SNR in SNR in |col3-col2|
dB from dB from
Table 7.2
1
2
3
4
4.4
class
formula
1.7
2.7
9.3
7.7
1.6
14.6
13.7
0.9
20.2
19.7
0.5
Comment again on the accuracy of the formula in column 3.
Again, the accuracy improves steadily as R increases. It's about .5 dB when R=4.
6. (a) CD recordings are made with a uniform scalar quantizer with rate 16 bits/sample. Assuming that
music samples are Gaussian with variance σ2 and assuming that the level spacing ∆ is chosen optimally
for this source model, estimate the SNR of CD recordings.
σ2
4
R
As in Problem 5, Part (a), we estimate SNR using SNR = 10 log10 D , with D ≅ σ2 3 (ln 2) 2R .
2
The result is SNR = 84.6 dB .
(b) Suppose instead that optimal nonuniform scalar quantization were used with the same rate (16
bits/sample). Estimate how much larger the SNR would be than in Part (a).
σ2
-2R
1
, with
As in Problem 5, Part (a), we estimate SNR using SNR = 10 log10 D , with D ≅ 12 σ2 β 2
3/2
β = 2" 3 = 32.6.
The result is SNR = 92.0 dB .
(c) Suppose instead that optimal nonuniform scalar quantization were used with its rate chosen to give the
same SNR as in part (a). Estimate how much more music could be stored on a CD. Express your answer
in percent.
Approach 1: Solving for the rate that gives SNR = 84.6 dB with nonuniform scalar quantization gives R
= 14.7, which is 8.1% less than rate R = 16. Therefore, one can store 8.1% more music.
***
NOTE: Rounding up the rate to 15 is fine. This gives a 6.25% improvement instead of 8.1%.
Approach 2: At high rates SNR changes at 6 dB per bit, if R = 16 gives 92.0 dB and we wish to
reduce SNR to 84.6 dB, then the rate reduction is (92-84.6)/6 = 1.23. So we use R = 16 - 1.23 14.8,
which is 7.5% less than rate R = 16. Therefore, one can store 7.5% more music. The difference
between this and the previous answer is due to round off errors
4
7. In this problem you will use Matlab to perform uniform scalar quantization on an image. From the
Homework Auxiliary Files webpage, download the template Matlab script
uniformimagequant_template.m and the image peppers.tif. The Matlab script has a
couple of lines that you must complete in order that it will do uniform scalar quantization of the image
'peppers' at a variety of rates and display the results. (Two other images are supplied in case you want to
try the script on other images.)
(a) Complete the script and include a copy of it and a printout of the displayed figure in your homework .
(Among other things, the script requires one to compute MSE. Since we don't have a
probabilistic/statistical model (e.g. pdf) for the image, one needs to compute the empirical MSE between
the original and the quantized images. Specifically, one computes the sum of the squares of the pixel-bypixel differences between the original and quantized images, divided by the number of pixels.)
The printout and script are given below.
(b) Compare the MSE's computed by the script to the formula ∆2/12. (You can do this by hand, or
modify the script so that it computes MSE.)
R
6
5
4
3
2
1
MSE
1.3
5.3
21.5
86.2
309.2
1126.5
∆2/12
1.3
5.3
21.2
84.7
338.7
1354.7
We see that ∆2/12 is ver accurate until R = 2 or 1.
Answers to the following questions are somewhat subjective and will depend on how large are the images
on your screeen. It is recommended that you make them as large as possible by expanding the window as
much as possible.
(c) Estimate the size of the individual images on your screen.
2 1/4 inch by 2 1/4 inch
(d) What is the smallest rate at which you cannot see evidence of quantization? What is the corresponding
SNR?
R=5
(e) What is the smallest rate that you think would be acceptable, i.e. not too annoying, for posting on the
internet? What is the corresponding SNR?
R=3
(f) What is the smallest rate at which someone can clearly recognize what is in the image? What is the
corresponding SNR?
R=2
script:
%%% uniformimagequant.m
%%% a script to perform uniform scalar quantization of an image
x = double(imread('peppers.tif','tiff')) ;
R = [6 5 4 3 2 1 ] ;
% set of rates at which to quantize
%% compute variance
mean = sum(sum( x ) ) / prod(size(x))
variance = sum(sum( (x-mean).^2 )) / prod(size(x))
%% perform quantization and display the quantized image
for k = 1:length(R)
L = 2^R(k) ;
delta = 255/L ; %compute step size of uniform quantizer
y = ( floor(x/delta) + .5 ) * delta ; %compute quantized image
mse = sum(sum( (x-y).^2 )); %compute mse
5
mse = mse/prod(size(x)) ;
SNR = 10 * log10( variance /mse ) ; %compute SNR
subplot(3,2,k) ;
imagesc(y); %display image
axis image;
colormap gray;
xlabel(['R = ' num2str(R(k)) ', MSE = ' num2str(mse,3) ', SNR = ' num2str(SNR,3) 'dB'])
end
printout:
6
8. In this problem you will run a Matlab script that designs and runs nonuniform scalar quantization on an
image. From the Homework Auxiliary Files webpage, download the Matlab script
nonuniformimagequant.m and the image peppers.tif. Run the script.
(a) Include a printout of the displayed figure in your homework.
Printout is given below
The answers to parts (b)-(e) are somewhat subjective. and will depend somewhat on how large the images
are on your screeen. As before, it is recommended that you make them as large as possible by expanding
the window as much as possible.
(b) What is the smallest rate at which you cannot see evidence of quantization? What is the corresponding
SNR?
R=4
(c) What is the smallest rate that you think would be acceptable, i.e. not too annoying, for posting on the
internet? What is the corresponding SNR?
R=3
(d) What is the smallest rate at which someone can clearly recognize what is in the image? What is the
corresponding SNR?
R=2
(e) Comment on the improvement of nonuniform scalar quantization over uniform.
The SNR shows sizable improvements at rates less than 4.
Gains: .9 dB, 1.7 dB, 1.3 dB, 1.6 dB at rates 4, 3, 2, 1, respectively.
Qualitatively, we also see improvements at these rates
(f) For the quantizer with 16 levels, find the widths of the largest and smallest quantization cells and
their ratio.
The thresholds are
0 18.4 34.8 54.8 70.6 84.0 94.8 108.7 128.5 145.8 160.0 172.5 183.2 255
The largest cell is [183.2,255] which has width 71.8.
The narrowest cells are
[172.5,183.2] which has width 10.7.
The ratio is 6.7
This indicates that the quantizer has a lot of nonuniformity. Notice that there is another nearly equally
small cell, namley, [84.0,94.8] with width 10.8. It is no coincidence that this cell and the narrowest are
located where the histogram is largest.
Note: In practice, one would ordinarily design a scalar quantizer based on a training set of many images,
and then apply it to images that were not in the training set. For example, you might modify the script so
that it designs the quantizer based on the two images baboon and camerman and then run the quantizer
on peppers. It will be interesting to see how much this reduces SNR.
7