Lecture 17:
Recall: Defn: a linear code is a subspace of V (n, q).
Prop: Let C ⊆ V (n, q). TFAE:
1. C is a linear code
2. C is a subspace of V (n, q)
3. C is closed under vector addition and scalar mulitplication
4. C is closed under all linear combinations.
Recall: a generator matrix G for a linear code C is a matrix
whose rows form a basis for C.
For a [n, k]-code, G is k × n.
Defn: Generator matrix in standard form: G = [Ik | A], where
Ik is the k × k identity matrix and A is a k × (n − k) matrix.
Prop: For every linear code C, there is a linear code C 0 equivalent
to C by a permutation of codeword positions s.t. C 0 has a generator
matrix in standard form.
Proof: Let G be a generator matrix.
– Find RREF.
– Permute the columns of the leading entries so that Ik appears
in the left half of the matrix.
The encoder corresponding to a generator matrix in standard form
is a systematic encoder:
E(u) = u[Ik | A] = [u, uA].
So, the user message is transparent in the codeword. We are essentially encoding by appending “parities.” to the user message.
1
Example: for the 3-repetition code, G = [111] is a generator matrix in standard form.
E(u) = (x1, x2, x3) = (u, u, u).
Example: For C3, a basis is {01101, 10110} and so a generator
matrix is:
1 0 1 1 0
0 1 1 0 1
which is in standard form and
E(u1, u2) = (x1, x2, x3, x4, x5) = (u1, u2, u1 + u2, u1, u2)
x3 is a parity bit and x4, x5 are trivial parity bits.
Advantage 3 of linear code: easy to encode and to “uncode.”
Defn: Let u, v ∈ V (n, q). The inner product of u, v is
u·v =
n
X
uivi
i=1
(with arithmetic in GF (q)).
Also called scalar product or dot product.
Examples:
GF (3): (1, 2, 1) · (1, 2, 2) = 1 + 4 + 2 = 1
GF (4): (a, b, 1) · (a, a, b) = a2 + ba + b = b + 1 + b = 1
Basic properties of inner product:
1. (symmetry) u · v = v · u
2. (bilinearity) (αu + βw) · v = αu · v + βw · v
v · (αu + βw) = αv · u + βv · w
2
3. u · v = 0 for all v ∈ V (n, q) iff u = 0.
Proof: Parts 1 and 2 are immediate from the definition. For Part
3:
If: also immediate from the definition.
Only if: Let ei be the vector which is 1 in position i and 0 elsewhere
(the standard basis vectors).
Then ui = u · ei. So, u = 0. Defn: v, w are orthogonal if v · w = 0.
Examples:
GF (3) : (1, 2, 1) · (2, 2, 0) = 2 + 4 = 0.
GF (3) : (1, 2, 1) · (1, 2, 1) = 1 + 4 + 1 = 0.
Note: a nonzero vector can be orthogonal to itself!
Defn: The dual code of a linear code C in V (n, q) is:
C ⊥ := {v ∈ V (n, q) : v · w = 0 ∀ w ∈ C}
The dual code is the orthogonal complement of C, viewed as a
subspace of V (n, q).
Example: C = {000, 111}. Then
C ⊥ := {v ∈ V (3, 2) : v1 + v2 + v3 = 0} = {000, 110, 101, 011}
Example:
C3⊥ := {v ∈ V (3, 2) : v1+v3+v4 = 0, v2+v3+v5 = 0, v1+v2+v4+v5 = 0, }
But the last equation is redundant!
Prop: C ⊥ is a linear code in V (n, q).
Proof: By bilinearity of the scalar product, if u, v ∈ C ⊥, w ∈ C
and α, β ∈ GF (q), then
(αu + βv) · w = 0
3
and so αu + βv ∈ C ⊥. Prop: Let C = hSi. Then
C ⊥ = {v ∈ V (n, q) : v · s = 0 ∀ s ∈ S}
Proof: Clearly, C ⊥ is contained in the RHS.
Let v belong to the RHS. Let w ∈ C. Then w can be expressed
Pk
as a linear combination of elements of S: w = i=1 aisi. Then
v·w =v·(
k
X
aisi) =
i=1
k
X
aiv · si = 0.
i=1
So, v ∈ C ⊥. Corollary: Let G be a generator matrix for linear code C. Then
C ⊥ = {v ∈ V (n, q) : Gv T = 0}.
Proof: Gv T = 0 means that v is orthogonal to the rows of G.
C is the span of the rows of G.
Apply the previous Prop. Example: For repetition code,
C ⊥ := {v ∈ V (3, 2) : v1 + v2 + v3 = 0}
Example:
C3⊥ := {v ∈ V (3, 2) : v1 + v3 + v4 = 0, v2 + v3 + v5 = 0}
Theorem: Let C be a linear code in V (n, q). Then dim(C) +
dim(C ⊥) = n, i.e. if C is an [n, k] code then C ⊥ is an [n, n − k]
code.
4
Proof: We may assume that C has a generator matrix in standard
form G = [Ik | A]. By the Corollary above,
C ⊥ = {v ∈ V (n, q) : ∀i = 1, . . . , k, vi +
n
X
Ai,j vj = 0
j=k+1
So, for all choices of of vk+1, . . . , vn, there is a unique element of C ⊥:
namely, for each i = 1, . . . , k, define
vi = −
n
X
Ai,j vj
j=k+1
Moreover, this exhausts all of C ⊥.
Thus, |C ⊥| = q n−k and so dim(C ⊥) = n − k. 5
Lecture 18:
Midterm 2, Thursday, March 17 in class.
Exam covers material covered in class through March 3, focusing
on material covered in lecture since Feb. 2. This includes rings, fields,
vector spaces, linear codes, generator matrices, including generator
matrices in standard form.
In text, this includes chapters 3,4,5 and pp. 55-56.
Will post practice midterm by Saturday.
Extended office hours: TBA.
Recall: dim(C) + dim(C ⊥) = n.
Corollary: Let C be a linear code in V (n, q). Then (C ⊥)⊥ = C.
Proof: Clearly, (C ⊥)⊥ ⊇ C. And
dim((C ⊥)⊥) = n − dim(C ⊥) = n − (n − dim(C)) = dim(C).
Thus, (C ⊥)⊥ is a subspace that contains C and has the same dimension as C. Thus, they both have the same number of points, q dim(C),
and thus (C ⊥)⊥ = C.. Defn: A parity-check matrix for a linear code C is a generator
matrix H for the dual code C ⊥.
Note: If G is k × n, then H is (n − k) × n.
Prop: Let H be a parity check matrix for a linear code C. Then
v ∈ C iff Hv T = 0.
In other words v ∈ C iff v satisfies the checks corresponding to
each row of H.
Proof: Let v ∈ C. Since C = (C ⊥)⊥,
v∈C
6
iff v is orthogonal to all elements of C ⊥
iff v being orthogonal to all elements of a basis for C ⊥
iff Hv T = 0. .
Contrast between generator matrix G and parity check matrix H
for a linear code C:
– G is an interior description: the elements of C are the linear
combinations of rows of G, i.e., the codewords generated by rows of
G. Thinks of a parametrization of a plane.
– H is an exterior description: the elements of C are the vectors
that satisfy constraints defined by H: an element v of V (n, q) is a
codeword iff it satisfies R · v = 0 for every row R of H. Think of an
equation of a plane.
Think of a linear code C as a club. You can describe the members
of the club by saying:
either:
– those who are elected by a basis (the executive) of the club
or:
– those who satisfy requirements of the club: one for each row of
H.
By definition, a parity check matrix for C is a generator matrix
for C ⊥.
Since (C ⊥)⊥ = C, a generator matrix for C is a parity check
matrix for C ⊥.
Summary:

code
 C
C⊥
gen.matrix
G
H
7

p.c.matrix

H
G
How to find a parity check matrix, given a generator matrix:
Theorem: If G = [Ik | A] is a generator matrix, in standard form,
for a linear code C, then H = [−AT | In−k ] is a parity check matrix
for C.
We call the latter a parity check matrix in standard form.
Proof:
GH T = [Ik | A]
−A
In−k
= −A + A = 0.
Thus, all the rows of H, which are the columns of H T , are orthogonal
to all the rows of G. Thus, the rows of H are all in C ⊥.
The rows of an identity matrix are linearly independent and therefore the rows of H are linearly independent. And there are n − k
rows in H. Since dim(C ⊥) = n − k, the rows of H form a basis for
C ⊥. Note: this procedure also tells us how to find a basis for C ⊥:
namely, the rows of H.
Return to Examples above:
1. Repetition code:
G = [111] = [1 | 11]
−1
1 0
1 1 0
H=
|
=
−1
0 1
1 0 1
So, v ∈ C iff v1 + v2 = 0 and v1 + v3 = 0, equivalently v1 = v2 = v3
A basis for C ⊥: {110, 101}.
Compare with formula given earlier:
C ⊥ = {v ∈ V (3, 2) : v1 + v2 + v3 = 0}.
8
One basis element each for setting v2v3 = 10 or 01.
2. C3:
1 1 0
1 0
G=
|
=
1 0 1
0 1

 
 
−1 −1
1 0 0
1
H =  −1 0  |  0 1 0  =  1
0 −1
0 0 1
0
1 0
0 1
1 1 0
1 0 1

1 1 0 0
0 0 1 0
1 0 0 1
So, v ∈ C3 iff v1 + v2 + v3 = 0, v1 + v4 = 0 and v2 + v5 = 0,
equivalently v1 = v4, v2 = v5 and v3 = v1 + v2 (with v1 and v2 being
free variables)
A basis for C3⊥: {11100, 10010, 01001}.
Compare with earlier formula:
C3⊥ := {v ∈ V (3, 2) : v1 + v3 + v4 = 0, v2 + v3 + v5 = 0}
One basis element each for setting v3v4v5 = 100 or 010 or 001.
3. C = h210, 011i over GF (3). Then
2 1 0
G=
0 1 1
is a generator matrix for C. The RREF of G is a generator matrix
in standard form:
1 0 1
G=
= [I2 | A].
0 1 1
and the resulting parity check matrix is:
H = [−AT | I1] = −1 −1 1 = 2 2 1
How to find a generator matrix, given a parity check matrix:
9
– Given a parity check matrix H (an (n − k) × n matrix), find
RREF and permute columns (if necessary) so that the identity matrix
In−k occupies the last n − k columns.
H = [B | In−k ]
where B is an (n − k) × k matrix. Let
G = [Ik | − B T ]
a k × n matrix. Then
HGT = 0
Similar to the proof of the Theorem above, since H is a generator
matrix for C ⊥ and the rows of G are linearly independent, G is a
parity check matrix for C ⊥, i.e. a generator matrix for (C ⊥)⊥ = C.
How to compute d(C) from a parity check matrix:
Theorem: Let C be a linear code with parity check matrix H.
Then d(C) is the minimum size of a set of linearly dependent columns.
Example: C3;

1 1 1 0 0
H=1 0 0 1 0
0 1 0 0 1

T
d(C3) = 3 because no column is 0 , no two columns are identical
(equivalently, no two columns are linearly dependent, since code is
over GF (2)) and columns 2, 3 and 5 are linearly dependent: C 2 +
C 3 + C 5 = 0.
Proof of Theorem: Recall that d(C) = wt(C).
So we want to show that the minimum size of a set of linearly
dependent columns is wt(C).
10
Recall also that v ∈ C iff Hv T = 0.
Write H = [K 1K 2 . . . K n] where the K i are the columns. Then
Hv T = 0 iff
n
X
viC i = 0
i=1
So, for all v ∈ C, with v 6= 0, a set of exactly wt(v) columns is
linearly dependent.
Thus, the minimum size of a set of linearly dependent columns is
the minimum weight of a nonzero codeword, i.e., wt(C). Equivalently, any subset of m ≤ d(C) − 1 columns is linearly
independent.
The Hamming code is
C = {x : x1+x2+x3+x5 = 0, x1+x2+x4+x6 = 0, x1+x3+x4+x7 = 0}
Cluster picture: Clusters are
{x1, x2, x3, x5}, {x1, x2, x4, x6}, {x1, x3, x4, x7}
Each cluster has even parity.
Equivalently, the Hamming code is defined by the parity check
matrix
1 1 1 0 1 0 0
H= 1 1 0 1 0 1 0
1 0 1 1 0 0 1
11