EQ2310 Digital Communications, HT21
Homework Assignment 2
Due: Monday, Jan. 10, 2022, 23:59
R. Thobaben
General Information
Background
This homework assignment is part of the continuous examination in the course EQ2310 Digital
Communications and will examine the course goals:
3. formulate a mathematical model that is applicable and relevant for a given problem in
the area
4. use a given or individually formulated mathematical model for solving a given technical
problem in the area and analyse the result and its reasonableness
5. compare different technologies in modern digital communication techniques, contrast
different technologies against one another and assess the suitability of individual technologies in different situations
Passing this assignment is a requirement for passing the course. For further information,
please check the Course PM.
Grading
The grade requirements for the homework assignment are specified as follows:
Grade
Requirement
C
The model is correctly formulated, the problem is correctly solved,
and the reasonableness is correctly analysed and verified. Few
insignificant mistakes are acceptable whereas too many mistakes will
lead to lower grades E or F.
E
The model is correctly formulated, the problem is correctly solved,
and the reasonableness is correctly analysed and verified under given
simplifying assumptions. Few insignificant mistakes are acceptable.
F
The model is not correct, the approach is not correct, or there are
severe mistakes that seriously affect the results.
Note:
• Before finalizing the grades, the examiner may call a few randomly sampled students to
a short interview in order to discuss the solutions.
• Solutions submitted after the submission deadline (see below) can only reach grade E
or F . Solutions submitted 24 hours after the deadline will be graded with F .
Submission
Solutions in PDF format (e.g., scans or photographs of handwritten solutions) are submitted
in Canvas, and the submission closes on Jan. 10, 2022, 23:59.
1
Plagiarism
This is an individual assignment, and the submitted solutions have to be prepared by the
students individually. Students are allowed to discuss general concepts that are relevant for
solving the problem with other students, and students are also allowed to solve related old
exam problems together. However, proof reading each others solutions or helping in solving
each others assignments is not allowed.
If plagiarism is detected, the case will be reported following KTH rules and policies.
Problem Formulation
In this problem, we consider a communication system that consists of a scalar quantizer (Q),
a source encoder, a channel encoder, a binary symmetric channel (BSC) with parameter ,
and the source and channel decoders at the receiver as illustrated in Figure 1. The goal is
to analyze some of the tradeoffs between system performance (e.g., rate and distortion) and
channel quality, and to suggest components for an implementation.
v
Q
s
Source
Encoder, Rs
b
Channel
Encoder, Rc
c
BSC
y
Channel
Decoder
b̂
Source
Decoder
v̂
Figure 1: Considered communication system.
We have the following system parameters:
• We have Ns source symbols v that are i.i.d. distributed with the probability density
function (PDF)
p(v) = exp(−v)I[0,∞] (v)
• The quantizer (Q) maps the Ns source symbols v into symbols s.
A scalar quantizer with k bit resolution is used that divides the interval [0, ∞] into 2k
disjoint quantization intervals
[a·i, a·(i + 1)], for i ∈ {0, . . . , 2k − 2},
Ii =
(1)
[a·i, ∞],
for i = 2k − 1.
That is, the 2k quantization intervalls are separated by 2k − 1 decision boundaries that
are equally spaced at a·i, and the last intervall contains the entire tail of the exponential
distribution. For every quantization interval we denote the reconstruction of the source
symbol as v̂i .
The symbols s ∈ {0, . . . , 2k − 1} at the output of the quantizer correspond to the labels
of the quantization intervals, and the encoding and decoding can be summarized as:
Encoding: s = Q(v) = i ∀v ∈ Ii and i ∈ {0, . . . , 2k − 1}
Decoding: v̂ = Q−1 (i) = v̂i
• The source encoder maps the Ns source symbols s into a length-Nb bit vector b using a
source code with rate Rs (in bit per source symbol on average).
• The channel encoder maps the Nb information bits b into Nc code bits c using a channel
code of rate Rc (in bit per channel use).
• Channel parameter (error probability) of the BSC is given by .
• The channel decoder returns a decoded information bit vector b̂.
• The source decoder returns directly the vector of reconstructed source values v̂.
2
Modeling
In a first step, you will use appropriate concepts and models from information theory to
identify the fundamental limits of the considered communication system. For simplicity,
we assume a = ln(2) in the following.
1. What is the highest theoretically possible number of information bits Nb that can be
reliably transmitted over a BSC with parameter for a fixed number of channel uses
Nc ?
Prepare a plot that shows the highest possible number of information bits Nb as a
function of the channel parameter in the range ∈ [0, 0.5] for Nc = 100 bits.
2. What is the smallest theoretically possible number of bits Nb that is required for representing Ns source symbols s for the source distribution and k-bit quantizer given above.
Justify your answer.
3. Now, we derive the expected distortion D(k) for the given source distribution and the
given quantizer as a function of the number of quantization bits per source symbol k:
(a) The optimal reconstruction value v̂i for the i-th quantization interval Ii , minimizing
the distortion di in Ii , is given by
Z
1
v̂i = E{v|i} =
v · p(v)dv.
Pr(i)
Ii
Show that for a = ln(2)
ln(2) · i + 1 − ln(2), i ∈ {0, . . . , 2k − 2}
v̂i =
ln(2) · i + 1,
i = 2k − 1.
(b) The expected distortion in the interval Ii for the reconstruction value v̂i is given
by
Z
1
2
di = E{(v − v̂i ) |i} =
(v − v̂i )2 · p(v)dv.
Pr(i)
Ii
Show that
di =
1 − 2 ln(2)2 , i ∈ {0, . . . , 2k − 2}
1,
i = 2k − 1.
Simplifying assumption for grade E for part 3(a) and 3(b)
Derive the corresponding results for a uniform distribution p(v) = I[0,1] (v) and a
k-bit scalar uniform quantizer with quantization intervals Ii = [2−k · i, 2−k · (i + 1)]
for i ∈ {0, . . . , 2k − 1}.
(c) Derive the average distortion
D(k) =
k −1
2X
di Pr(i),
i=0
and plot D(k) as a function of k.
What is the minimum distortion Dmin that we can achieve with this quantizer and
for which value of k is it reached (in good approximation)?
You can use your plot to answer this question.
3
4. Prepare a plot that shows the lowest possible distortion Dmin () as a function of the
channel parameter ∈ [0, 0.5] for different ratios Nc /Ns ∈ {2, 4, 6, 8, 10}.
Hint: For every find the maximum k that satisfies Hk (s) · Ns ≤ C()Nc , where Hk (s)
is the entropy of the quantization symbols s given k, and use D(k) derived above to
obtain the desired relation for Dmin ().
Describe and discuss the tradeoff between channel quality, number of source symbols,
and distortion (for a fixed number of channel uses) that you observe.
Design and Comparison
In this part, we will use the results from above as guidelines for the selection of system
components.
5. Propose two source codes for k = 3 and a = ln(2) with the following properties:
(A)
• Source Code A minimizes the rate Rs .
Does the proposed source code reach the entropy of the source?
(B)
• Source Code B consists of length-k codewords (i.e., Rs = k) and uses a Gray
mapping to ensure that codewords for neighbouring quantization intervalls only
differ in one bit position.
How much redundancy does the Source Code B keep?
6. In order to get some insights into the error resilience of the two source codes, generate
a short symbol sequence s (e.g., 4-5 symbols), encode it with the two source codes, and
investigate how bit errors affect the decoding of the two source codes (e.g., consider
error events with 1- or 2-bit errors and see what happens to the distortion).
Give a qualitative summary of your findings and give examples to support your conclusions.
Can you see that one source code is more robust than the other?
7. Now consider the channel codes in Table 1. For Source Code A and Source Code B
select one matching channel code each such that the overall rate is approximately 1
source symbol per 6 channel use (multiple choices are possible). For each of your source
and channel code pairs, discuss advantages and disadvantages (e.g., error performance,
complexity, robustness).
Rate
1/2
1/2
1/3
1/3
Table 1: Table of convolutional codes.
Generator (octal notation) Memory Free distance df ree
(5, 7)8
2
5
(133, 171)8
6
10
(5, 7, 7)8
2
8
(133, 145, 175)8
6
15
4