Lecture 5:
HW1 due on Thursday. You may use anything established through
material in today’s class.
Will use the concept of equivalence of codes:
1. permutation of the positions of the codewords
2. for a fixed position, permute the symbols in that position with
a fixed permutation of the alphabet (different permutations are
allowed in different positions)
Example: {000, 111} ∼ {100, 011} ∼ {001, 110}.
Fact: if C and D are equivalent codes, then they have the same
n, M, d parameters. For example, the codes
Recall defn of Aq (n, d): the largest M such that there exists an
(n, M, d)q code.
Proposition: A2(5, 3) = 4.
Proof of Proposition: Recall C3 is a (5, 4, 3) binary code. So,
A2(5, 3) ≥ 4.
Suppose that C is a (5,5,3) binary code. We seek a contradiction.
By permuting symbols in each position, we may assume that 0 =
00000 ∈ C.
Thus, all other codewords must have weight 3,4 or 5.
If c = 11111 ∈ C, then for any codeword c0 ∈ C, c0 6= 0, d(c, c0) ≤
2, a contradiction. So, C can have words of weight only 3 or 4 (other
than 0).
It cannot have more than one word of weight 4, since the distance
between any two such words is 2.
1
Thus, C must have at least 3 codewords of weight 3: x, y, z. We
will show that this is impossible.
By permuting the codeword positions we may assume that x =
11100 ∈ C.
Claim: y and x have exactly one 1 in a common position (and
similarly, z and x have exactly one 1 in a common position).
Proof of claim: x and y must have at least one 1 in a common
position, since they both weight = 3.
Suppose they have two 1’s in common positions. By a permutation
of the first three positions, we may assume that
y = 011??
where one of the ?’s is a 1 and the other is a 0.
But with either choice, d(x, y) = 2, which is impossible.
And x and y cannot have three 1’s in common positions because
they are distinct words.
So, they must have only one 1 in common. This proves the claim.
.
So, y =???11 where exactly one of the ? is a 1.
Similarly, z =???11 where exactly one of the ? is a 1.
Thus, y and z must have a 0 in a common position. Thus,
d(y, z) = 2, a contradiction to d(C) = 3.
This proves the proposition. Two constructions that shorten or lengthen a code.
Defn: Let C be a code and 1 ≤ i ≤ n. The i-punctured code D
of C is the code obtained by deleting the i-bit in every codeword:
D = {x1 . . . xi−1xi+1 . . . xn : x ∈ C}
2
Example: The 2-repetition code is a punctured code of the 3repetition code.
Example: The code
D = {0000, 0101, 1010, 1111}
is the 3-punctured code of
C = C3 = {00000, 01110, 10110, 11011}
The puncturing map is f (x) := x1 . . . xi−1xi+1 . . . xn.
P
Defn: Let D be a binary code. Let p(x) = ni=1 xi mod 2.
C = {x p(x) : x ∈ D}
i.e., we add a parity bit to the end of each codeword in C. Then C
is obtained from D by adding an overall parity check.
Note that all elements of C have even weight.
Example:
C = {000000, 011101, 101101, 110110}
which is obtained from the code
D = C3 = {00000, 01110, 10110, 11011}
by adding an overall parity check.
Proposition (a combination of Theorem 2.7 and Corollary 2.8 in
the text). Let d, n ≥ 1.
1. Aq (n + 1, d + 1) ≤ Aq (n, d)
2. If d is odd, then A2(n + 1, d + 1) = A2(n, d).
Example: A2(6, 4) = A2(5, 3) = 4 and is achieved by the code
C = {000000, 011101, 101101, 110110}
3
Example: Part 2 above is false if d is even: from HW1 #7, we will
have A2(4, 2) = 23 = 8, but as above A2(5, 3) = 4.
Proof of Proposition:
1. Let C achieve Aq (n+1, d+1). Let x, y ∈ C s.t. d(x, y) = d+1.
Let i be a position s.t. xi 6= yi.
Let D be the i-punctured code of C.
Claim: D is an (n, |C|, d) code.
Proof: Clearly D has length n. Since d(C) ≥ 2, the puncturing
mapping from C to D is 1-1 and onto, and so |D| = |C|.
Since d(C) = d + 1, d(D) ≥ d. But d(x0, y 0) = d. So, d(D) = d.
It follows that Aq (n + 1, d + 1) ≤ Aq (n, d).
4
Lecture 6:
Next week, I will be away. You will have a guest lecturer for both
days. No office hours.
Will post HW2 by early next week, due on Thursday, Feb. 4.
Recall: Proposition (a combination of Theorem 2.7 and Corollary
2.8 in the text). Let d, n ≥ 1.
1. Aq (n + 1, d + 1) ≤ Aq (n, d)
2. If d is odd, then A2(n + 1, d + 1) = A2(n, d).
We already proved part 1.
2. Let d be odd and let D be a code that achieves A2(n, d).
Let C be the code obtained from D by adding an overall parity
check:
C = {x p(x) : x ∈ D}
Claim: C is an (n + 1, |D|, d + 1) code.
Proof: Clearly C has length n + 1.
The map f : D → C defined by f (x) = x p(x) is a bijection. So,
|C| = |D|.
If x, y ∈ D, x 6= y, then d(x, y) ≥ d.
If d(x, y) ≥ d + 1, then d(x p(x), y p(y)) ≥ d + 1.
If d(x, y) = d, then, since d is odd, d(x p(x), y p(y)) = d + 1.
In either case, d(x p(x), y p(y)) ≥ d+1, and this proves the claim.
It follows that
A2(n + 1, d + 1) ≥ |C| = |D| = A2(n, d)
and hence A2(n + 1, d + 1) = A2(n, d).
5
This proves the proposition. Given the difficulty of computing Aq (n, d) it is worth exploring
bounds on Aq (n, d).
Theorem (Singleton bound) Let C be an (n, M, d) code over an
alphabet of size q. Then
M ≤ q n−d+1
and so
Aq (n, d) ≤ q n−d+1
Singleton is a person’s name.
Proof: Consider the map: f : C → Fqn−d+1 defined by
f (x1 . . . xn) = x1 . . . xn−d+1
i.e., f deletes the last d − 1 positions.
We claim that f is 1-1. To see this, observe that if x, y ∈ C,
x 6= y, then since d(x, y) ≥ d, x and y must disagree in at least one
of their first n − d + 1 coordinates. Thus, f (x) 6= f (y).
Thus,
M = |C| ≤ |Fq |n−d+1 = q n−d+1
Equivalent forms of Singleton bound:
logq M ≤ n − d + 1
d ≤ n − logq M + 1
logq M + d ≤ n + 1
You can’t have your cake and eat it too.
Examples of Singleton bound:
6
1. Aq (n, n) ≤ q; in fact, we already showed that Aq (n, n) = q
(the repetition codes).
2. A2(n, 2) ≤ 2n−1; in fact, in HW1#7, you showed that A2(n, 2) =
2n−1.
Examples 1 and 2 are tight.
Following examples are not tight.
3. A2(n, n − 1) ≤ 22 = 4; in HW1#8, you showed that A2(n, n −
1) = 2 for n ≥ 4;
4. A2(5, 3) ≤ 23 = 8; in fact, we already showed that A2(5, 3) = 4;
so, in this case the Singleton bound is not tight.
Here is another upper bound.
Recall Hamming ball: given an alphabet Fq , positive integers n, r
and x ∈ Fqn,
Br (x) = {y ∈ Fqn : d(x, y) ≤ r}
Note that Br (x) depends on r, x, q, n but we suppress dependence
on q, n in the notation.
Example: q = 2, n = 3 : B1(000) = {000, 100, 010, 001}
B2(100) = {100, 000, 110, 101, 011, 001, 111}
For arbitrary q, n, Bn(x) = Fqn.
Defn: the volume of the Hamming ball is |Br (x)|.
Proposition:
r X
n
(q − 1)m
|Br (x)| =
m
m=0
In particular, |Br (x)| does not on x; so we can abbreviate Br (x) by
Br .
7
Proof:
|Br (x)| =
r
X
|{y ∈ Fqn : d(x, y) = m}|
m=0
and each word y ∈ Fqn s.t. d(x, y) = m is uniquely determined by
the m locations in which x and y differ and for each such location a
choice of q − 1 symbols. Special case: q = 2:
|Br (x)| =
r X
n
m=0
m
Theorem (Hamming Bound or sphere-packing bound): Let t ≥ 1.
Aq (n, 2t + 1) ≤ b Pt
m=0
qn
c
n
m
(q
−
1)
m
Proof: Let C be an (n, M, 2t + 1) code over Fq .
Then {Bt(c) : c ∈ C} are pairwise disjoint. Thus,
M · |Bt| = |C| · |Bt| = | ∪c∈C Bt(c)| ≤ |F n| = q n
Thus,
M ≤ Pt
m=0
qn
n
m
m (q − 1)
If C is a code that achieves Aq (n, 2t + 1), then we get the bound. Note: the Hamming bound applies only to odd d = 2t + 1, equivalently, t-error-correcting codes; however, there is way to deduce a
corresponding bound for even d: use Corollary 2.8.
Compare Hamming and Singleton:
The winner is the smaller upper bound.
1. A2(7, 3):
8
Hamming: A2(7, 3) ≤ 27/(1 + 7) = 16
Singleton: A2(7, 3) ≤ 27−3+1 = 32
So, Hamming beats Singleton! We will later see that A2(7, 3) = 16
2. A8(5, 3):
Hamming: A8(5, 3) ≤
85
85
=
1+5·7
36
5−3+1
3
Singleton: A8(5, 3) ≤ 8
=8
So, Singleton beats Hamming: 83 <
85
36
iff 1 <
82
36 ,
which is true.
Consider the special case: d = 3 (1-error-correcting (t = 1)) and
q = 2.
Hamming:
2n
A2(n, 3) ≤
1+n
Singleton:
2n
n−2
A2(n, 3) ≤ 2
=
4
In this case, Hamming wins for n ≥ 4; for n = 3, there is a tie; and
for n = 1, 2, it doesn’t make sense (as pointed out by a student in
class).
9