Many Faces of Hamming Code
PARTHA PRATIM DEY
Department of Computer Science & Engineering
North South University
12 Kemal Ataturk Rd., Banani, Dhaka
Abstract:- In this paper, we present three alternative constructions of the (7,4,1) - Hamming code. This error
correcting code was developed by Richard W. Hamming in the 1950s when he was working in the Bell
Laboratories. It improved the practical applications of early computers by substantially improving their reliability.
But it is even more remarkable that many modern computers still use this or similar code to correct errors in main
memory. It is not an exaggeration to say that modern graphical computing, which requires large memories and
computers in critical control applications, which can not afford any significant probability of an erroneous result
would be impractical without this Hamming code. Besides that this code has made a penetration into the quantum
world too. It has currently been proposed by A. M. Steane [1] for quantum error correction.
Key-words:- Linear Code, Hamming Matrix, Encoder, Parity Check Matrix, Hadamard Matrix, Polynomial.
multiplication of message set W by matrix G is
called encoding.
1 Introduction
We begin with a couple of definitions.
Definition (1.1) Let V be
the
vector
dimension n over Z 2  {0,1} so
vector x has the shape
that
Let us consider an (n  k )  n matrix H of full row
n
2
space Z of
rank over Z 2 with HG t  0 . Since HG t  0 , we
a
typical
obtain HG t w t  0 for
any w W .
t
t
Hence H ( wG)  0 i.e. Hc  0 for all c  C . And
since H has full row rank, it can be shown
that Hc t  0 if and only if c  C . Thus H can be
used to identify code-words in C as follows:
C  {c  V | Hc t  0}.
The matrix H is called a parity check matrix.
x  ( x1 ,..., xn ), xi  Z 2 , i  1,..., n .
A [ n, s ] code C over Z 2 is a linear subspace of Z 2n of
dimension s .
s  n matrix G with elements
from Z 2 and of full row rank s is called a generator
matrix of a [ n, s ] code C if C is the linear subspace
spanned over Z 2 by the rows of G considered as
vectors in Z 2n .
Definition (1.2) A
Let W  Z 2k and V  Z 2n with k  n . We assume G to
be a k  n matrix over Z 2 of full row rank.
Then C  {c  V | c  wG for some w  W } is a
subspace of V of dimension k . Therefore, C is a
linear code of V with 2 k code-words, and G is a
generator matrix of C . Notice that, besides spanning
the code C , G also appends or encodes a k bit
message of W into a n bit code-word c of C when
the message w is multiplied by G . This is why, a
generator matrix is also known as encoder and the
Theorem (1.3) Given a generator matrix G we can
always construct a parity check matrix H , and vice
versa.
Proof. Suppose G is given. Find the null
space KerG  {v  V | Gv t  0} of G . Find a basis
of this null space. Let each element of the basis be a
row of the H . Since each row of H is in KerG , we
have GH t  0 . By taking transposition of this
equality, we get HG t  0 . Thus H is indeed a parity
check matrix.
We now prove the converse. Suppose H is given. We
find a basis for KerH  {v  V | Hv t  0} and let
each element of the basis be a row of G . Then the
sub-space spanned by the rows of G over Z 2 is
indeed KerH . Thus G is a generator matrix of the
code, given by parity check matrix H . ■
Next we consider a 3 7 matrix H , where the
columns
of
the
matrix H are
the
numbers 1,2,...,7 expressed in binary. The code C ,
having such a parity check matrix H , is called
a (7,4,1) -Hamming
code.
Actually
for
any r  2, a (r ,2 r  1) matrix H is called a Hamming
matrix if the columns of H contain the binary
equivalents of the integers from 1 to 2 r  1 , and the
corresponding code is called a Hamming code.
2
The
Hamming
Scheme
of
Information Transmission and Error
Correction
Richard W. Hamming is best known for his
pioneering work on error correcting codes for
computer systems. Before accepting a chair of
computer science at the Naval Postgraduate School at
Monterey, California, he spent thirty years from 1946
to 1976, working for the Bell Telephone
Laboratories. It was there where he came in contact
with primitive computers of that early era. Frustrated
by their lack of fundamental reliability, he puzzled
over the problem of how a computer might check and
correct its own results. This led him to devise
the (7,4,1) - code and other error correcting codes and
publish his results as “Error Detecting and Error
Correcting Codes” in the Bell System Technical
Journal, vol 29, pp. 147-160.These codes are a
special type of linear codes which are capable of not
only detecting errors, but also capable of correcting
them. In Hamming's scheme of error correction, a
symmetric binary channel is used i.e., information is
transmitted in the form of strings of 0 and 1 .
Moreover when a transmitter sends a signal 0 or 1 in
such a channel, associated with each signal there is a
constant
common
probability p for
incorrect
transmission due to noise in the channel. To improve
the accuracy of transmission in such a channel, a
message w  x1 ...x k  Z 2k is appended by n  k extra
signals x k 1 ...x n to form the codeword c  x1 ...x n .
Recall from our discussion in Section 1 that this
appending is carried out through multiplication by a
generator matrix G and the whole process is called
encoding. However, the coded message c  x1 ...x n is
then transmitted through some channel. If the
received vector r contains a corrupt bit, it is detected
and corrected. We discuss in next paragraph how in a
Hamming scheme, these error detection and
correction are accomplished.
Suppose c , a codeword of a (7,4,1) - Hamming code
is transmitted and we receive the vector r  Z 27 .
Then r  c  e for some error vector e  Z 27 that
contains one in the position where r and c differ and
zeros elsewhere. Note that Hr t  Hc t  He t  He t ,
so we can determine He t by computing Hr t . Since
error vector contains all zeros except a single
one, Hr t  He t will be one of the columns of H .
Let Hr t be the j th column of H . Then surely,
the j th bit of the received word r must have been
corrupted.
Finally, after correction of the received vector, a
decoding map D : Z 2n  Z 2k is applied on it to
retrieve the original message w  x1 ...x k .
3 Polynomial Construction
Let Z 2 [ x] be the integral domain of polynomials
over Z 2 .
Then f ( x)  x 7  1 is
an
irreducible
polynomial of degree 7 in Z 2 [ x] and the principal
ideal ( f ( x )) consists of all polynomials in Z 2 [ x] of
which f (x )
is
a
factor.
Moreover R  Z 2 [ x] /( f ( x)) ,
polynomials
in Z 2 [ x] of degree less than 7 , is a field. On the other
hand, g ( x)  x 3  x  1 is a primitive polynomial
in Z 2 [ x] of degree n  3 .
We let C be multiples
of g ( x)  x  x  1 in Z 2 [ x] of degree less than 7 .
This
is
a
vector
space
in R with
basis B 
3
{x 3  x  1, x 4  x 2  x, x 5  x 3  x 2 , x 6  x 4  x 3 }
. Each polynomial in the basis can be identified with
a 7 -dimensional vector. For example, x 3  x  1 can
be
rewritten
as
1 x 0  1 x  0  x 2  1 x 3  0  x 4  0  x 5  0  x 6 ,
and therefore can be identified with the 7 dimensional vector (1,1,0,1,0,0,0) . The same can be
B.
done
with
other
elements
of
Thus
H 2
H4  
H 2
H2 
 H 2 
and
B  {(1,1,0,1,0,0,0), (0,1,1,0,1,0,0), (0,0,1,1,0,1,0), (0,0,0,1,1,0,1)}  H 4 H 4 
H8  

can be looked as a basis in Z 27 . We will show that the
H 4  H 4 
are
normalized
Hadamard
matrices
of
code spanned by B is a (7,4,1) - Hamming code.
orders 4 and 8 respectively.
Let Ĝ be the matrix whose i th row is the i th vector in
We delete the first row and column from H 8 , change
B , from left i.e.,
all negative ones in H 8 to zeros, and call the resulting
1 1 0 1 0 0 0 
0

matrix A. Thus
1 1 0 1 0 0
Ĝ  
0 0 1 1 0 1 0 
0 1 0 1 0 1 0 


0
1
0
0
0
1
1
1 0 0 1 1 0 0 




Applying elementary row operations on Ĝ , after a
0 0 1 1 0 0 1 
couple of steps, we obtain the following matrix.


A  1 1 1 0 0 0 0 
0 1 0 0 1 0 1 
1 0 0 0 1 1 0 


0

1 0 0 0 1 1
1 0 0 0 0 1 1 

G
0 0 1 0 1 1 0 
0 0 1 0 1 1 1 




0 0 0 1 1 0 1 
Note that each row and column of A contains 3 ones.
If G is used as encoder, then the corresponding parity
It is because each row and column of H 8 except the
check matrix H is given by
first contains 4 ones. Hence, the dot product of any
1 0 1 1 1 0 0 
row or column of A with itself will be equal to 3 .


H  1 1 1 0 0 1 0 
Furthermore, in any pair of distinct rows
0 1 1 1 0 0 1 
of H 8 excluding the first, there are 4 positions in
Clearly H is a Hamming matrix.
which the rows differ, 2 positions in which the rows
both have a 1 , and 2 positions in which the rows both
have a  1 . Thus, in the corresponding pair of rows
4 Hadamard Matrix Construction
of A , there will be 1 position in which the rows both
An n  n matrix H is called a Hadamard matrix if the
have 1 , so the dot product of any two distinct rows
entries in H are all 1 or  1 , and HH t  nI for
of A will be equal to 1 . Therefore AAt  2 I  J ,
the n  n identity matrix I . A Hadamard matrix H is
where I is
the 7  7 identity
matrix,
and J is
said to be normalized if the first row and column
the 7  7 matrix of all ones. Since also JA  3J , then
of H contain only positive ones. The only normalized
we know A is the incidence matrix of Fano's plane
Hadamard matrices of orders one and two ( i.e., of
[2][3]. Singer [4] has proved that the rank of this
sizes 1 1 and 2  2) are
matrix is 4 over GF (2).
We will show that the code spanned by its rows is
H1  [1]
a (7,4,1) - Hamming code.
and
We take the first four independent rows of A to form
1 
1
H2  
.
1
1
Also,
0
1
Ĝ  
0

1
1 0 1 0 1 0

0 0 1 1 0 0
0 1 1 0 0 1

1 1 0 0 0 0
Applying elementary row operations on Ĝ , we
obtain:
1
0
G
0

0
0 0 0 0 1 1

1 0 0 1 0 1
.
0 1 0 1 1 0

0 0 1 1 1 1
As in Section 3, the parity check matrix for such
a G is
0 1 1 1 1 0 0 
H  1 0 1 1 0 1 0  ,
1 1 0 1 0 0 1 
which is clearly a Hamming matrix.
3 Venn Diagram Construction
This method uses three circles. When drawn as in the
Figure ( 4.1) , these circles split one another
into 7 regions numbered one through seven.
A 4 -bit message, such as m  x1 x 2 x3 x 4 , is encoded
into a 7 -bit code-word c as follows:
a. The bit x i is assigned to region i for each
i  {1,2,3,4} .
b. A bit, 0 or 1 , is assigned to each of the remaining
regions of the circles so that the total number
of 1' s in each circle is even.
c. The code-word c is then taken to be the 7 -bit
string x1 x2 x3 x4 x5 x6 x7 , where x i is the bit which
the i th region was assigned as in Fig 4.2 .
5
4
6
1
3
2
7
Fig 4.1
x6
x5
x3
x1 x 2
x7
Fig 4.2
Notice that condition b. above implies the following.
 x1  x3  x 4  x5  0

 x1  x 2  x 4  x 6  0
x  x  x  x  0
2
3
7
 1
The parity check matrix here is
1 0 1 1 1 0 0 
H  1 1 0 1 0 1 0  ,
1 1 1 0 0 0 1 
which is clearly a Hamming matrix.
References:
[1] A. M. Steane, Error Correcting Codes in Quantum
Theory, Physics Review Letters, 74, pp. 793-804,
1996.
[2] P. Dembowskii, Finite Geometries, New York:
Springer Verlag, 1968.
[3] H. L. Dorwart, The Geometry of Incidence,
Englewood Cliffs, N.J. : Prentice-Hall, 1966.
[4] J. A. Singer, Theorem in Finite Projective
Geometry and Some Applications to Number Theory,
Trans. Amer.Math. Soc., 43, pp. 377-385, 1938.