Information and Codes (Tutorial sheet)
Instructors: Mahdi Cheraghchi, Herbert Wiklicky
Autumn 2015, Imperial College London
09 November 2015
Exercise 1. Show that any generator matrix for an [n, k, d] binary code must have at least kd entries that
are equal to 1.
Solution. Since the rows of a generator matrix must be non-zero codewords, and any non-zero codeword
of a linear code must have at least d non-zero entries (because the zero vector must be a codeword), the
result follows.
Exercise 2. Show that for a fixed length n, the Hamming distance is a metric on the space of the words of
that length. Recall that a metric function d(x, y) satisfies the following conditions:
1. d(x, y) ≥ 0 (non-negativity).
2. d(x, y) = 0 if and only if x = y (identity of indiscernibles).
3. d(x, y) = d(y, x) (symmetry).
4. d(x, z) ≤ d(x, y) + d(y, z) (triangle inequality).
Solution. Observe that the Hamming distance of two vectors is the minimum number of symbol changes
required to transform one into the other. Using this, the first three conditions are trivial to verify. As for the
triange inequality
d(x, z) ≤ d(x, y) + d(y, z),
(1)
consider x = (x1 , . . . , xn ), y = (y1 , . . . , yn ), z = (z1 , . . . , zn ), and consider each position i of the vectors x, y
and z. If xi = zi , the corresponding position contributes 0 to the left-hand-side of equation (1). In this case,
either yi = xi = zi , thus contributing 0 to the right-hand-side as well, or yi 6= xi , zi , thus contributing 2 to
the right-hand-side. If xi 6= zi , so that the corresponding i contributes 1 to the left-hand-side of equation (1),
then yi must be different from at least one of xi and zi , thus contributing at least 1 to the right-hand-side.
Summing over all values of i we readily obtain the triangle inequality.
Exercise 3.
In this exercise, we will see that one can convert arbitrary codes into code with slightly
different parameters:
1. Let C be a binary (n, M, d) code with d odd. Then it can be converted into an (n + 1, M, d + 1) code.
2. Let C be an (n, M, d)q code (not necessarily binary). Then it can be converted into an (n − 1, M, d − 1)q
code.
Note: Other than the parameters of the code C, you should not assume anything else about the code.
Also your conversion should work for every n, k, d ≥ 1.
1
Solution.
1. The new code transforms each original codeword (c1 , . . . , cn ) into (c1 , . . . , cn , cn+1 ) where cn+1 is
defined as the parity of the bits; i.e., cn+1 := c1 + · · · + cn where addition is over GF (2). Consider two
codewords c = (c1 , . . . , cn ) and c0 = (c01 , . . . , c0n ) in the original code. If dH (c, c0 ) ≥ d + 1, then they are
different in at least d + 1 positions in the new code as well. Otherwise, dH (c, c0 ) = d, which is an odd
number. In this case, we consider the newly added bit of each codeword: cn+1 = c1 + · · · + cn and
c0n+1 = c01 + · · · + c0n . Observe that cn+1 + c0n+1 = (c1 + c01 ) + · · · + (cn + c0n ). Now, observe that for
each i, ci + c0i is 1 if ci 6= c0i , and is zero otherwise. Therefore, taking addition modulo 2,
cn+1 + c0n+1 = (c1 + c01 ) + · · · + (cn + c0n ) = d mod 2 = 1.
Therefore, cn+1 6= c0n+1 . We see the in the new code, the distance between transformed versions of c
and c0 becomes d + 1.
2. Simply remove the first coordinate of each codeword. We see that the Hamming distance between
each pair of codewords can only decrease by at most 1.
Exercise 4.
1. Let C be a binary code with minimum distance d. Prove that C can correct any combination of e1 errors
(i.e., bit flips) and e2 erasures provided that 2e1 + e2 + 1 ≤ d.
2. Let D be a decoder for C that guarantees to correct from any e errors as long as 2e + 1 ≤ d (but D does
not support possibility of erasures). Show that D can be turned into a decoder that can correct any
combination of erasures and errors as in the previous part.
Solution.
1. Consider a pattern of e2 ≤ d − 1 erasures, and the code obtained by deleting the erasure positions of
the code. The resulting code C 0 has a minimum distance at least d − e2 and thus can be corrected as
long as 2e1 ≤ (d − e2 ) − 1. Once C 0 has been error corrected, C can be erasure corrected since e2 ≤ d − 1.
2. Given a received word y containing e1 errors and e2 erasures such that 2e1 +e2 +1 ≤ d, we can proceed
as follows. Guess the erased positions to be all zeros (resulting in string a y 0 ), and call D(y 0 ). Then,
guess the erased positions to be all ones (resulting in a string y 00 ), and call D(y 00 ). Each call may return
a decoding or nothing. According to the previous part, we know that there is a unique codeword c
that is obtained from y by e1 errors and e2 erasures. Thus it suffices to show that at least one of the
two calls of D outputs such a codeword (note that for each output, we can simply check what the
number of erasures and errors would be with respect to that codeword). Let c0 be the bit string of
length e2 corresponding to the erased part of c. Observe that the Hamming distance between c0 and
either the all-zeros string or all-ones string must be at most e2 /2 (depending on whether c0 has more
zeros than ones or vice versa). As a result, the Hamming distance between c and either y 0 or y 00 must
at most e1 + e2 /2 < d/2. Since we know that D will output c whenever given something at distance
less than d/2 from c, we conclude that at least one of the calls to D must output the correct c.
2