3.
CHAPTER 3 - LINEAR BLOCK CODES
In this chapter we introduce basic concepts of block codes. For ease of code synthesis and
implementation, we restrict our attention to a subclass of the class of all block codes, the linear block
codes. Because information in most current digital data communication and storage systems is coded
in the binary digits 0 and 1, we discuss only the linear block codes with symbols from the binary field
GF(2). It is straightforward to generalize the theory developed for the binary codes to codes with
symbols from a non-binary field.
First, linear block codes are defined and described in terms of generator and parity-check matrices.
The parity-check equations for a systematic code are derived.
Encoding of linear block codes is discussed. In Section 3.2 the concept of syndrome is introduced. The
use of syndrome for error detection and correction is discussed. In Sections 3.3 and 3.4 we define the
minimum distance of a block code and show that the random-error-detecting and random-errorcorrecting capabilities of a code are determined by its minimum distance. Probabilities of a decoding
error are discussed. In Section 3.5 the standard array and its application to the decoding of linear
block codes are presented. A general decoder based on the syndrome decoding scheme is given.
References [1] through [6] contain excellent treatments of linear block codes.
3.1.
3.1 Introduction to Linear Block Codes
We assume that the output of an information source is a sequence of the binary digits 0 and 1. In block
coding, this binary information sequence is segmented into message blocks of fixed length; each
message block, denoted by u, consists of k information digits. There are a total of 2k distinct messages.
The encoder, according to certain rules, transforms each input message u into a binary n-tuple v with
n > k. This binary n-tuple v is referred to as the code-word (or code vector) of the message u.
Therefore, corresponding to the 2k possible messages, there are 2k code-words. This set of 2k codewords is called a block code. For a block code to be useful, the 2k code-words must be distinct.
Therefore, there should be a one-to-one correspondence between a message u and its code-word v.
For a block code with 2k code-words and length n, unless it has a certain special structure, the encoding
apparatus will be prohibitively complex for large k and n, since it has to store the 2k code-words of
length n in a dictionary. Therefore, we restrict our attention to block codes that can be mechanized in a
practical manner.
A desirable structure for a block code to possess is linearity, which greatly reduces the encoding
complexity, as we shall see.
DEFINITION 3.1 A block code of length n and 2k code-words is called a linear (n, k) code if and only
if its 2k code-words form a k-dimensional subspace of the vector space of all the n-tuples over the field
GF(2).
In fact, a binary block code is linear if and only if the modulo-2 sum of two code-words is also a codeword. The block code given in Table 3.1 is a (7, 4) linear code. One can easily check that the sum of
any two code-words in this code is also a code-word.
Because an, k) linear code C is a k-dimensional subspace of the vector space V„ of all the binary ntuples, it is possible to [find k linearly independent code-words, go, gl, …, gk _ s. in C such that every
code-word av in C is a linear combination of these k code-words [ that is,
where = 0 or 1 for 0 < i < k. We arrange these k linearly independent code-words as the rows of a k x a
matrix as follows:
where gi = (grit), …, g1 -1) for 0 < i < k. If an = (uo, u s, …, u1_1) is the message to be encoded, the
corresponding code-word can be given as follows:
106728959.doc
petak, 12. veljače 2016
1
Clearly, the rows of C generate (or span) the (n, k) linear code C. For this reason, the matrix C is
called a generator matrix for C. Note that any k linearly independent code-words of an (n, k) linear
code can be used to form a generator matrix for the code. It follows from (3.3) that an (n, k) linear code
is completely specified by the k rows of a generator matrix G. Therefore, the encoder has only to store
the k rows of C and to form a linear combination of these k rows based on the input message as = (no,
0 1,, uk _ 1).
1. EXAMPLE 3.1 The (7, 4) linear code given in Table 3.1 has the following matrix as a
generator matrix
The (7, 4) linear code given in Table 3.1 has the following matrix as a generator matrix:
TABLE 3.1: Linear block code with k = 4 and n = 7.
Messages Code-words
If a = (1101) is the message to be encoded, its corresponding code-word, according to (3.3), will be
A desirable property for a linear block code to possess is the systematic structure of the code-words, as
shown in Figure 3.1, in which a code-word is divided into two parts, the message part and the
redundant checking part. The message part consists of k unaltered information (or message) digits, and
the redundant checking part consists of n < k parity-check digits, which are linear sums of the
information digits. A linear block code with this structure is referred to as a linear systematic block
code. The (7, 4) code given in Table 3.1 is a linear systematic block code: the rightmost four digits of
each code-word are identical to the corresponding information digits.
FIGURE 3.1: Systematic format of a code-word.
A linear systematic (n, k) code is completely specified by a k x /I matrix G of the following form:
where pi; = 0 or 1. Let ilk denote the k k identity matrix. Then, G = llkl, Let e. = (no, n •, u k._-1) be
die message to be encoded. The corresponding code-word is
It follows from (3.4) and (3.5) that the components of v are
and
for 0 < j < n - k. Equation (3,6a) shows that the rightmost k digits of a code-word v are identical to the
information digits Ito. /It, •, uk to be encoded, and (3.6b) shows that the leftmost 11 - k redundant
digits are linear sums of the information digits. The a k equations given by (3.6b) are called paritycheck equations of the code.
2. EXAMPLE 3.2 The matrix G, given in Example 3.1 is in systematic form
The matrix G, given in Example 3.1 is in systematic form. Let u = (u0, u1, u2, u3) be the message to be
encoded, and let v = (v0, v1, v2, v3, v4, v5, v6) be the corresponding code-word. Then,
By matrix multiplication, we obtain the following digits of the code-word v:
The code-word corresponding to the message (1 0 1 1) is (1 0 0 1 0 1 1).
There is another useful matrix associated with every linear block code. As stated in Chapter 2, for any
k x n matrix G with k linearly independent rows, there exists an (a < k) x a matrix i f, with a < k
linearly independent rows such that any vector in the row space of G is orthogonal to the rows of H,
and any vector that is orthogonal to the rows of t l is in the row space of G. Hence, we can describe the
(a, k) linear code C generated by G in an alternative way as follows: An n-tuple v is a code-word in the
code C generated by G if and only if v  HT = 0.
The code is said to be the null space of t i. This matrix d 1 is called a parity-check matrix of the code.
The 2"-k linear combinations of the rows of matrix H form an (a, a < k) linear code Cd. This code is
the null space of the (a, k) linear code C generated by matrix G (i.e., for any v E C and any w E Cd, v 
w = 0). Cd is called the dual code of C. Therefore, a parity-check matrix for a linear code C is a
2
106728959.doc
petak, 12. veljače 2016
generator matrix for its dual code Cd. If the generator matrix of an (a, k) linear code is in the
systematic form of (3.4), the parity-check matrix may take the following form:
where PT is the transpose of the matrix P. Let hj be the jth row of H. We can readily check that the
inner product of the i th row of G given by (3.4) and the jth row of H given by (3.7) is
for 0 < i < k and 0 < j < n - k. This implies that G = 0. Also, the n - k rows of t § are linearly
independent. Therefore, the H matrix of (3.7) is a parity-check matrix of the (a, k) linear code
generated by the matrix G of (3.4).
The parity-check equations given by (3.6b) also can be obtained from the parity-check matrix H of
(3.7). Let n u1, …, 1_1) be the message to be encoded. In systematic form the corresponding codeword will be
Using the fact that v = 0, we obtain
for 0 < j < a < k. Rearranging the equations of (3.8), we obtain the same parity-check equations of
(3.6b). Therefore, an (a, k) linear code is completely specified by its parity-check matrix.
3. EXAMPLE 3.3 Consider the generator matrix of the (7, 4) linear code
Consider the generator matrix of the (7, 4) linear code given in Example 3.1. The corresponding
parity-check matrix is
At this point we summarize the foregoing results: For any (a, k) linear block code C there exists a k x a
matrix C whose row space gives C. Furthermore, there exists an (a < k) x a matrix HI such that an atuple v is a code-word in C if and only if v PNiT = P. If C is of the form given by (3.4), then iliil may
take the form given by (3.7), and vice versa.
Based on the equations of (3.6a) and (3.6b), the encoding circuit for an (n, k) linear systematic code
can easily be implemented. The encoding circuit is shown in Figure 3.2, where -n- denotes a shiftregister stage (e.g., a flip-flop),  denotes a modulo-2 adder, and ---- denotes a connection if [Jo = 1,
and no connection if Pit = 0. The encoding operation is very simple. The message = (u p. //I.  °  ilk-1)
to be encoded is shifted into the message register and simultaneously into the channel.
As soon as the entire message has entered the message register the a < k parity-check digits are formed
at the outputs of the a < k modulo-2 adders. These parity-check digits are then serialized and shifted
into the channel. We see that the complexity of the encoding circuit is linearly proportional to the
block length of the code. The encoding circuit for the (7, 4) code given in Table 3.1 is shown in Figure
3.3, where the connection is based on the parity-check equations given in Example 3.2.
FIGURE 3.2: Encoding circuit for a linear systematic (n. k) code.
FIGURE 3.3: Encoding circuit for the (7, 4) systematic code given in Table 3.1.
3.2.
3.2 Syndrome and Error Detection
Consider an (11, k) linear code with generator matrix G and parity-check matrix Let v = (vo, vi, …,
v„_ t) be a code-word that was transmitted over a noisy channel. Let r = (r0 1-1, …, r„-t) be the
received vector at the output of the channel. Because of the channel noise, it may be different from -v.
The vector sum
is an n-tuple, where e, = 1 for 1-, v,, and ei = 0 for r, = v,. This n-tuple is called the error vector (or
error pattern), which simply displays the positions where the received vector it differ from the
transmitted code-word v. The 1’s in e are the transmission errors caused by the channel noise. It
follows from (3.9) that the received vector it is the vector sum of the transmitted code-word and the
error vector; that is,
106728959.doc
petak, 12. veljače 2016
3
Of course, the receiver does not know either w or e. On receiving r, the decoder must first determine
whether it contains transmission errors. If the presence of errors is detected, the decoder will either
take actions to locate the errors and correct them (FEC) or request a retransmission of v (ARQ).
When r is received, the decoder computes the following (77 - k)-tuple:
which is called the syndrome of r. Then, s = 0 if and only if it is a code-word, and s 0 0 if and only if it
is not a code-word. Therefore, when s 0 0, we know that it is not a code-word and the presence of
errors has been detected. When s = 0, it is a code-word, and the receiver accepts it as the transmitted
code-word. It is possible that the errors in certain error vectors are not detectable (i.e., it contains errors
but s = r  T = 0).
This happens when the error pattern a is identical to a nonzero code-word. In this event, a is the sum of
two code-words, which is a code-word, and consequently. ihlr = 0. Error patterns of this kind are
called undetectable error patterns. Because there are 2k – 1 nonzero code-words, there are 2k – 1
undetectable error patterns.
When an undetectable error pattern occurs, the decoder makes a decoding error. In a later section of
this chapter we derive the probability of an undetected error for a BSC and show that this error
probability can be made very small.
Based on (3.7) and (3.10), the syndrome digits are as follows:
If we examine the preceding equations carefully, we find that the syndrome s is simply the vector sum
of the received parity digits (r0, r1,, r„_k_i) and the parity-check digits recomputed from the received
information digits (r„-k, i'„_k+i, r„_1). Therefore, the syndrome can be formed by a circuit similar to
the encoding circuit. A general syndrome circuit is shown in Figure 3.4.
4. EXAMPLE 3.4 Consider the (7, 4) linear code whose parity-check matrix is given in Example
3.3
Consider the (7, 4) linear code whose parity-check matrix is given in Example 3.3. Let a = (1-0, r1, r2,
r5, r6) be the received vector. Then, the syndrome is given by
The syndrome digits are
The syndrome circuit for this code is shown in Figure 3.5. The syndrome 5 computed from the
received vector a depends only on the error pattern a and not on the transmitted code-word v. Because
Fr is the vector sum of v and a, it follows from (3.10) that
FIGURE 3.4: Syndrome circuit for a linear systematic (n, k) code.
FIGURE 3.5: Syndrome circuit for the (7, 4) code given in Table 3.1.
however, v  = 0. Consequently, we obtain the following relation between the syndrome and the error
pattern:
If the parity-check matrix is expressed in the systematic form as given by (3.7), multiplying out e HT
yields the following linear relationship between the syndrome digits and the error digits:
The syndrome digits are simply linear combinations of the error digits. Clearly, they provide
information about the error digits and therefore can be used for error correction.
At this point, one would think that any error correction scheme would be a method of solving the a k
linear equations of (3.13) for the error digits. Once the error pattern a was found, the vector FC e
would be taken as the actual transmitted code-word. Unfortunately, determining the true error vector e
is not a simple matter. This is because the - k linear equations of (3.13) do not have a unique solution
but have 2k solutions (this will be proved in Theorem 3.6).
In other words, there are 2k error patterns that result in the same syndrome, and the true error pattern e
is just one of them, Therefore, the decoder has to determine the true error vector from a set of 2 k
candidates. To minimize the probability of a decoding error, the most probable error pattern that
4
106728959.doc
petak, 12. veljače 2016
satisfies the equations of (3.13) is chosen as the true error vector. If the channel is a BSC, the most
probable error pattern is the one that has the smallest number of nonzero digits.
The notion of using syndrome for error correction may be clarified by an example.
5. EXAMPLE 3.5 Again, we consider the (7,4) code whose parity-check matrix is given in
Example 3.3
Again, we consider the (7,4) code whose parity-check matrix is given in Example 3.3. Let = (1001011)
be the transmitted code-word and r = (1001001) be the received vector. On receiving rr, the receiver
computes the syndrome:
Next, the receiver attempts to determine the true error vector c = (e0, et, e2, e3, es, e6), which yields
the given syndrome. It follows from (3.12) or (3.13) that the error digits are related to the syndrome
digits by the following linear equations:
There are 24 = 16 error patterns that satisfy the preceding equations, namely,
The error vector a = (0000010) has the smallest number of nonzero components. If the channel is a
BSC, e = (0000010) is the most probable error vector that satisfies the preceding equations. Taking e =
(0000010) as the true error vector, the receiver decodes the received vector r = (1001001) into the
following code-word:
We see that vl is the actual transmitted code-word. Hence, the receiver has performed a correct
decoding. Later we show that the (7, 4) linear code considered in this example is capable of correcting
any single error over a span of seven digits; that is, if a code-word is transmitted and if only one digit
is changed by the channel noise, the receiver will be able to determine the true error vector and to
perform a correct decoding.
More discussion on error correction based on syndrome is given in Section 3.5. Various methods of
determining the true error pattern from the n -k linear equations of (3.13) are presented in later chapters.
3.3.
3.3 The Minimum Distance of a Block Code
In this section we introduce an important parameter of a block code called the minimum distance. This
parameter determines the random-error-detecting and random-error-correcting capabilities of a code.
Let v = (vet, v1, …, v„_1) be a binary n-tuple. The Hamming weight (or simply weight) of v, denoted
by w(v), is defined as the number of nonzero components of v. For example, the Hamming weight of v
= (1001011) is 4. Let v and w be two n-tuples. The Hamming distance (or simply distance) between v
and w, denoted d(v, w), is defined as the number of places where they differ. For example, the
Hamming distance between v = (1001011) and w = (0100011) is 3; they differ in the zeroth, first, and
third places. The Hamming distance is a metric function that satisfies the triangle inequality. Let v. w,
and x be three n-tuples. Then,
(The proof of this inequality is left as a problem.) It follows from the definition of Hamming distance
and the definition of modulo-2 addition that the Hamming distance between two n-tuples v and w is
equal to the Hamming weight of the sum of v and w; that is.
For example, the Hamming distance between v = (1001011) and w = (1110010) is 4, and the weight of
v w = (0111001) is also 4.
Given a block code C, one can compute the Hamming distance between any two distinct code-words.
The minimum distance of C, denoted by 4117, is defined as.
If C is a linear block code, the sum of two code-words is also a code-word. It follows from (3.15) that
the Hamming distance between two code-words in C is equal to the Hamming weight of a third codeword in C. Then, it follows from (3.16) that
The parameter w,„i„ {w(tt) : E C, A = 0) is called the minimum weight of the linear code C.
Summarizing the preceding result, we have the following theorem.
106728959.doc
petak, 12. veljače 2016
5
THEOREM 3.1 The minimum distance of a linear block code is equal to the minimum weight of its
nonzero code-words and vice versa.
Therefore, for a linear block code, determining the minimum distance of the code is equivalent to
determining its minimum weight. The (7, 4) code given in Table 3.1 has minimum weight 3; thus, its
minimum distance is 3. Next, we prove a number of theorems that relate the weight structure of a
linear block code to its parity-check matrix.
THEOREM 302 Let C be an (n, k) linear code with parity-check matrix H. For each code-word of
Hamming weight 1, there exist /columns of H such that the vector sum of these 1 columns is equal to
the zero vector. Conversely, if there exist /columns of H whose vector sum is the zero vector, there
exists a code-word of Hamming weight I in C.
Proof We express the parity-check matrix in the following form:
where hi represents the ith column of H. Let nv = (vo, v1, …, v„_1) be a code-word of weight 1. Then,
v has 1 nonzero components. Let vi,, vi,, -  -, vi, be the /nonzero components of v, where 0 < it < i2
< …• < ii < n < 1. Then, vi, = vi, = …= vi, = 1. Because v is a code-word, we must have
This proves the first part of the theorem.
Now, suppose that hi,, hi,, …, hi, are 1 columns of HI such that
We form a binary n-tuple x = (A-0, °  °, xi-1) whose nonzero components are x1. 1, xi,, …• Th-e
Hamming weight of is 1. Consider the product
It follows from (3.18) that x  HT = 0. Thus, x is a code-word of weight /in C.
This proves the second part of the theorem.
The following two corollaries follow from Theorem 3.2.
COROLLARY 3.2.1 Let C be a linear block code with parity-check matrix H. If no d < 1 or fewer
columns of add to 0, the code has minimum weight at least d.
COROLLARY 3.2.2 Let C be a linear code with parity-check matrix i i. The minimum weight (or the
minimum distance) of C is equal to the smallest number of columns of H that sum to 0.
Consider the (7, 4) linear code given in Table 3.1. The parity-check matrix of this code is
We see that all columns of i are nonzero and that no two of them are alike. Therefore, no two or fewer
columns sum to 0. Hence, the minimum weight of this code is at least 3; however, the zeroth, second,
and sixth columns sum to 0. Thus, the minimum weight of the code is 3. From Table 3.1 we see that
the minimum weight of the code is indeed 3. It follows from Theorem 3.1 that the minimum distance is
3.
Corollaries 3.2.1 and 3.2.2 are generally used to determine the minimum distance or to establish a
lower bound on the minimum distance of a linear block code.
3.4.
3.4 Error-Detecting and Error-Correcting Capabilities of a Block Code
When a code-word v is transmitted over a noisy channel, an error pattern of l errors will result in a
received vector r that differs from the transmitted code-word v in l places [i.e., d(v, r) = 1]. If the
minimum distance of a block code C is dmin, any two distinct code-words of C differ in at least dmin
places. For this code C, no error pattern of dmin < 1 or fewer errors can change one code-word into
another. Therefore, any error pattern of d,11117 < 1 or fewer errors will result in a received vector r
that is not a code-word in C. When the receiver detects that the received vector is not a code-word of C,
we say that errors are detected.
Hence, a block code with minimum distance dr„,,, is capable of detecting all the error patterns of dmin
– 1 or fewer errors. However, it cannot detect all the error patterns of dmin errors because there exists at
least one pair of code-words that differ in dmin places, and there is an error pattern of d„„, errors that
6
106728959.doc
petak, 12. veljače 2016
will carry one into the other. The same argument applies to error patterns of more than d„„.„ errors.
For this reason we say that the random-error-detecting capability of a block code with minimum
distance dmin is dmin – 1.
Even though a block code with minimum distance dmin guarantees detection of all the error patterns of
dmin < 1 or fewer errors, it is also capable of detecting a large fraction of error patterns with dmin or
more errors. In fact, an (n, k) linear code is capable of detecting 2" - 2k error patterns of length n. This
can be shown as follows. Among the 2" – 1 possible nonzero error patterns, there are 2k – 1 error
patterns that are identical to the 2k – 1 nonzero code-words. if any of these 2k - error patterns occurs, it
alters the transmitted code-word v into another code-word w.
Thus, w is received, and its syndrome is zero. in this case, the decoder accepts w as the transmitted
code-word and thus performs an incorrect decoding. Therefore, there are 2k – 1 undetectable error
patterns. If an error pattern is not identical to a nonzero code-word, the received vector r will not be a
code-word and the syndrome will not be zero. In this case, an error will be detected. There are exactly
2" - 2k error patterns that are not identical to the code-words of an (a, k) linear code. These 2" - 2k error
patterns are detectable error patterns. For large a, 2k < 1 is, in general, much smaller than 2". Therefore,
only a small fraction of error patterns pass through the decoder without being detected.
Let C be an (a, k) linear code. Let A; be the number of code-words of weight i in C. The numbers Ao,
A1, …, A„ are called the weight distribution of C. If C is used only for error detection on a ISC, the
probability that the decoder will fail to detect the presence of errors can be computed from the weight
distribution of C. Let P„(E) denote the probability of an undetected error. Because an undetected error
occurs only when the error pattern is identical to a nonzero code-word of C,
where p is the transition probability of the BSC. If the minimum distance of C is dmin, then Al to
Admen _ I are zero.
Consider the (7, 4) code given in Table 3.1. The weight distribution of this code is Ao = 1, Al = A2 = 0,
A3 = 7, A4 = 7, As = A6 = 0, and A7 = 1. The probability of an undetected error is
If p = 1072, this probability is approximately 7 x 10-6. In other words, if 1 million code-words are
transmitted over a SSC with p = 10-2, on average seven erroneous code-words pass through the
decoder without being detected.
If a block code C with minimum distance dmin is used for random-error correction, one would like to
know how many errors the code is able to correct. The minimum distance dmin is either odd or even.
Let t be a positive integer such that
Next, we show that the code C is capable of correcting all the error patterns of t or fewer errors. Let 'v
and r be the transmitted code-word and the received vector, respectively. Let w be any other codeword in C. The Hamming distances among v, r, and w satisfy the triangle inequality:
Suppose that an error pattern of t' errors occurs during the transmission of v. Then, the received vector
r differs from v in t' places, and therefore d(v, r) = t'. Because v and w are code-words in C, we have
Combining (3.21) and (3.22) and using the fact that d(v, r) = t', we obtain the following inequality:
If t' < t,
The preceding inequality says that if an error pattern of t or fewer errors occurs, the received vector r is
closer (in Hamming distance) to the transmitted code-word v than to any other code-word w in C. For
a BSC, this means that the conditional probability P(r1v) is greater than the conditional probability
P(r1w) for w v. Based on the maximum likelihood decoding scheme, r is decoded into v, which is the
actual transmitted code-word. The result is a correct decoding, and thus errors are corrected.
In contrast, the code is not capable of correcting all the error patterns of 1 errors with 1 > t, for there is
at least one case in which an error pattern of 1 errors results in a received vector that is closer to an
incorrect code-word than to the transmitted code-word. To show this, let v and w be two code-words in
C such that
106728959.doc
petak, 12. veljače 2016
7
Let el and e2 be two error patterns that satisfy the following conditions:
i. el + e2 = v + w•
ii. el and e2 do not have nonzero components in common places.
Obviously, we have
Now, suppose that v is transmitted and is corrupted by the error pattern el. Then, the received vector is
The Hamming distance between v and B. is
The Hamming distance between w and r is
Now, suppose that the error pattern e1 contains more than r errors (i.e., w(e1) > r). Because 2t + 1 <
dmin < 2t 2, it follows from (3.23) that
Combining (3.24) and (3.25) and using the fact that w(e1) > t and w(e2) < t +1, we obtain the
following inequality:
This inequality says that there exists an error pattern of 1(1 > t) errors that results in a received vector
that is closer to an incorrect code-word than to the transmitted code-word. Based on the maximum
likelihood decoding scheme, an incorrect decoding would be performed.
In summary, a block code with minimum distance c/min guarantees correction of all the error patterns
of t = – 1(61„„„ – 1)/2J or fewer errors, where [(d„,„, – 1)/k] denotes the largest integer no greater than
(d,„„, – 1)/2. The parameter r = - 1)/2] is called the random-error-correcting capability of the code.
The code is referred to as a t-error-correcting code. The (7, 4) code given in Table 3.1 has minimum
distance 3 and thus r = 1. The code is capable of correcting any error pattern of single error over a
block of seven digits.
A block code with random-error-correcting capability t is usually capable of correcting many error
patterns of t -1- 1 or more errors. A t-error-corrects g (a, k) linear code is capable of correcting a total
of 2"-k error patterns, including those with t or fewer errors (this will be seen in the next section). If a
t-error-correcting block code is used strictly for error correction on a BSC with transition probability p,
the probability that the decoder commits an erroneous decoding is upper bounded by
In practice, a code is often used for correcting X or fewer errors and simultaneously detecting 1(1 > X)
or fewer errors. That is, when X or fewer errors occur, the code is capable of correcting them; when
more than X but fewer than /H- 1 errors occur; the code is capable of detecting their presence without
making a decoding error. For this purpose, the minimum distance d,17 of the code is at least X + 1 + 1
(left as a problem). Thus, a block code with ci„,i„ = 10 is capable of correcting three or fewer errors
and simultaneously detecting six or fewer errors.
So far, we have considered only the case in which the receiver makes a hard-decision for each received
symbol; however, a receiver may be designed to declare a symbol erased when it is received
ambiguously (or unreliably). In this case, the received sequence consists of zeros, ones, or erasures. A
code can be used to correct combinations of errors and erasures. A code with minimum distance d,„1„
is capable of correcting any pattern of v errors and e erasures provided the following condition
is satisfied. To see this, delete from all the code-words the e components where the receiver has
declared erasures. This deletion results in a shortened code of length - e. The minimum distance of this
shortened code is at least d„,„, - e > 2v + 1.
Hence, v errors can be corrected in the un-erased positions. As a result, the shortened code-word with e
components erased can be recovered. Finally, because (1177 17 > e +1, there is one and only one codeword in the original code that agrees with the unerased components. Consequently, the entire codeword can be recovered. This correction of combinations of errors and erasures is often used in practice.
From the preceding discussion we see that the random-error-detecting and random-error-correcting
capabilities of a block code are determined by the code's minimum distance. Clearly, for a given a and
8
106728959.doc
petak, 12. veljače 2016
k, one would like to construct a block code with as large a minimum distance as possible, in addition to
the implementation considerations.
Often an (n, k) linear block code with minimum distance d„,11, is denoted by (n, k, d„,111). The code
given by Table 3.1 is a (7, 4, 3) code.
3.5.
3.5 Standard Array and Syndrome Decoding
In this section we present a scheme for decoding linear block codes. Let C be an (n, k) linear code. Let
vi, v2, …, v2i, be the code-words of C. N0 matter which code-word is transmitted over a noisy channel,
the received vector r may be any of the 2" n-tuples over GF(2). Any decoding scheme used at the
receiver is a rule to partition the 2" possible received vectors into 2k disjoint subsets D1, D2, - …, D2k
such that the code-word vi is contained in the subset Di for 1 < i < 2k. Thus, each subset Di is one-toone correspondence to a code-word vi. If the received vector r is found in the subset D,, r is decoded
into vi. Decoding is correct if and only if the received vector r is in the subset Di that corresponds to
the code-word transmitted.
A method to partition the 2" possible received vectors into 2k disjoint subsets such that each subset
contains one and only one code-word is described here. The partition is based on the linear structure of
the code. First, we place the 2k code-words of C in a row with the all-zero code-word vi = (0, 0, …, 0)
as the first (leftmost) element. From the remaining 2" - 2k n-tuples, we choose an n-tuple 02 and place
it under the zero vector v1. Now, we form a second row by adding e2 to each code-word vi in the first
row and placing the sum e2 + vi under vi.
Having completed the second row, we choose an unused n-tuple e3 from the remaining n-tuples and
place it under v1. Then, we form a third row by adding 03 to each code-word 'if; in the first row and
placing e3 + vi under vi. We continue this process until we have used all the n-tuples. Then, we have
an array of rows and columns, as shown in Figure 3.6.
FIGURE 3.6: Standard array for an (n, k) linear code.
This array is called a standard array of the given linear code C.
It follows from the construction rule of a standard array that the sum of any two vectors in the same
row is a code-word in C. Next, we prove some important properties of a standard array.
THEOREM 3.3 N0 two n-tuples in the same row of a standard array are identical. Every n-tuple
appears in one and only one row.
Proof The first part of the theorem follows from the fact that all the code-words of C are distinct.
Suppose that two n-tuples in the lth row are identical, say + = a1 with i j. This means that a; = a1, which
is impossible.
Therefore, no two n-tuples in the same row are identical.
It follows from the construction rule of the standard array that every n-tuple appears at least once. Now,
suppose that an n-tuple appears in both the lth row and the ;nth row with 1 < m. Then this n-tuple must
be equal to e1 + vi for some i and equal to e„, + a; for some j. As a result, ei + ai = e,„ + a1. From this
equality we obtain e„, = e1 + + TO. Because ai and a1 are code-words in C, a; + ai is also a code-word
in C, say as. Then e„, = + as. This implies that the n-tuple e„, is in the lth row of the array, which
contradicts the construction rule of the array that e„, the first element of the 117th row, should be
unused in any previous row. Therefore, no n-tuple can appear in more than one row of the array. This
concludes the proof of the second part of the theorem.
From Theorem 3.3 we see that there are 2"/2k` = 2"-1c disjoint rows in the standard array, and that
each row consists of 2k distinct elements. The 2"-k rows are called the cosets of the code C, and the
first n-tuple e i of each coset is called a coset leader (or coset representative). The coset concept for a
subgroup was presented in Section 2.1. Any element in a coset can be used as its coset leader. This
does not change the elements of the coset; it simply permutes them.
106728959.doc
petak, 12. veljače 2016
9
6. EXAMPLE 3.6 Consider the (6, 3) linear code generated by the following matrix
Consider the (6, 3) linear code generated by the following matrix:
The standard array of this code is shown in Figure 3.7.
A standard array of an (n, k) linear code C consists of 2k disjoint columns. Each column consists of 2"k n-tuples, with the topmost one as a code-word in C. Let D1 denote the jth column of the standard
array. Then,
FIGURE 3.7: Standard array for a (6, 3) code.
where vj is a code-word of C, and e2, e3, …, e2,-k are the coset leaders. The 2k disjoint columns D1,
D2, …, D2/, can be used for decoding the code C as described earlier in this section. Suppose that the
code-word v j is transmitted over a noisy channel. From (3.27) we see that the received vector r is in
D1 if the error pattern caused by the channel is a coset leader. In this event, the received vector [r will
be decoded correctly into the transmitted code-word v.,: however, if the error pattern caused by the
channel is not a coset leader, an erroneous decoding will result. This can be seen as follows. The error
pattern x caused by the channel must be in some coset and under some nonzero code-word, say in the
lth coset and under the code-word v, = 0. Then, x - = er + v,, and the received vector is
The received vector r is thus in Ds and is decoded into vs, which is not the transmitted code-word. This
results in an erroneous decoding. Therefore, the decoding is correct if and only if the error pattern
caused by the channel is a coset leader. For this reason, the 2"-k coset leaders (including the zero
vector 0) are called the correctable error patterns. Summarizing the preceding results, we have the
following theorem:
THEOREM 3.4 Every (n, k) linear block code is capable of correcting 2"-k error patterns.
To minimize the probability of a decoding error, the error patterns that are most likely to occur for a
given channel should be chosen as the coset leaders. For a BSC, an error pattern of smaller weight is
more probable than an error pattern of larger weight. Therefore, when a standard array is formed, each
coset leader should be chosen to be a vector of least weight from the remaining available vectors. If
coset leaders are chosen in this manner, each coset leader has minimum weight in its coset. As a result,
the decoding based on the standard array is the minimum distance decoding (i.e., the maximum
likelihood decoding). To see this, let ir be the received vector. Suppose that r is found in the ith column
D, and lth coset of the standard array. Then, r is decoded into the code-word IT;. Because ir = ei v„ the
distance between ir and v, is
Now, consider the distance between r and any other code-word, say,
Because vi and v, are two different code-words, their vector sum, v, +, is a nonzero code-word, say vs.
Thus,
Because e1 and e1 + vs are in the same coset and since w (el) < w (el + vs), it follows from (3.28) and
(3.29) that
This says that the received vector is decoded into a closest code-word. Hence, if each coset leader is
chosen to have minimum weight in its coset, the decoding based on the standard array is the minimum
distance decoding, or MLD.
Let ai denote the number of coset leaders of weight i. The numbers no, al,, oe,, are called the weight
distribution of the coset leaders. Knowing these numbers, we can compute the probability of a
decoding error. Because a decoding error occurs if and only if the error pattern is not a coset leader,
the error probability for a BSC with transition probability p is
7. EXAMPLE 3.7 Consider the (6, 3) code given in Example 3.6
Consider the (6, 3) code given in Example 3.6. The standard array for this code is shown in Figure 3.7.
The weight distribution of the coset leaders is no = 1, al = 6, a2 = 1, and a3 = as = us = a6 = 0. Thus,
For p = 10-2, we have P(E) ti 1.37 x 10–3.
10
106728959.doc
petak, 12. veljače 2016
An (n, k) linear code is capable of detecting 2" - 2k error patterns, however, it is capable of correcting
only 2"-k error patterns. For large n, 2"-k is a small fraction of 2" - 2k. Therefore, the probability of a
decoding error is much higher than the probability of an undetected error.
THEOREM 3.5 For an (n, k) linear code C with minimum distance dmin, all the n-tuples of weight t =
L(dmin < 1)/2J or less can be used as coset leaders of a standard array of C. If all the n-tuples of weight t
or less are used as coset leaders, there is at least one n-tuple of weight t + 1 that cannot be used as a
coset leader.
Proof. Because the minimum distance of C is dmin, the minimum weight of C is also dmin. Let a and y
be two n-tuples of weight t or less. Clearly, the weight of a + y is
Suppose that a and y are in the same coset: then, a y must be a nonzero code-word in C. This is
impossible, because the weight of a + y is less than the minimum weight of C. Therefore, no two iituples of weight r or less can be in the same coset of C, and all the n-tuples of weight t or less can be
used as coset leaders.
Let v be a minimum weight code-word of C [i.e., w(v) = dmin]. Let a and y be two n-tuples that satisfy
the following two conditions:
i. + y = v,
ii. a and y do not have nonzero components in common places.
It follows from the definition that a and y must be in the same coset and
Suppose we choose y such that w(y) = r + 1. Because 2t + 1 < dmin < 2t + 2, we have w(x) = t or t + 1.
If x is used as a coset leader, then y cannot be a coset leader.
Theorem 3.5 reconfirms the fact that an (n, k) linear code with minimum distance dmin is capable of
correcting all the error patterns of [(dmin – 1)/2] or fewer errors, but it is not capable of correcting all
the error patterns of weight t + 1.
A standard array has an important property that can be used to simplify the decoding process. Let i i be
the parity-check matrix of the given (n, k) linear code C.
THEOREM 3.6 All the 2k n-tuples of a coset have the same syndrome. The syndromes for different
cosets are different.
Proof Consider the coset whose coset leader is el. A vector in this coset is the sum of el and some
code-word v, in C. The syndrome of this vector is
(since v, = 0). The preceding equality says that the syndrome of any vector in a coset is equal to the
syndrome of the coset leader. Therefore, all the vectors of a coset have the same syndrome.
Let e1 and le/ be the coset leaders of the jth and lth cosets, respectively, where j < 1. Suppose that the
syndromes of these two cosets are equal. Then,
This implies that e, + e1 is a code-word in C, say vi. Thus, e j + e1 = v;, and e1 = ej + vi. This implies
that e1 is in the jth coset, which contradicts the construction rule of a standard array that a coset leader
should be previously unused. Therefore, no two cosets have the same syndrome.
We recall that the syndrome of an n-tuple is an (n < k)-tuple, and there are 2"-k distinct (/- k)-tuples. It
follows from Theorem 3.6 that there is a one-to-one correspondence between a coset and an (11 - k)tuple syndrome; or there is a one-to-one correspondence between a coset leader (a correctable error
pattern) and a syndrome. Using this one-to-one correspondence relationship, we can form a decoding
table, which is much simpler to use than a standard array. The table consists of 2"-k coset leaders (the
correctable error patterns) and their corresponding syndromes. This table is either stored or wired in
the receiver. The decoding of a received vector consists of three steps:
1. Compute the syndrome of r, r  HT.
106728959.doc
petak, 12. veljače 2016
11
2. Locate the coset leader e1 whose syndrome is equal to r  HT. Then el is assumed to be the error
pattern caused by the channel.
3. Decode the received vector r into the code-word v* = r + el.
The described decoding scheme is called the syndrome decoding or table-lookup decoding. In principle,
table-lookup decoding can be applied to any (n, k) linear code. It results in minimum decoding delay
and minimum error probability; however, for large ri - k, the implementation of this decoding scheme
becomes impractical, and either a large storage or a complicated logic circuitry is needed. Several
practical decoding schemes that are variations of table-lookup decoding are discussed in subsequent
chapters. Each of these decoding schemes requires additional properties of a code other than the linear
structure.
8. EXAMPLE 3.8 Consider the (7, 4) linear code given in Table 3.1
Consider the (7, 4) linear code given in Table 3.1. The parity-check matrix, as given in Example 3.3, is
The code has 23 = 8 cosets, and therefore there are eight correctable error patterns (including the allzero vector). Because the minimum distance of the code is 3, it is capable of correcting all the error
patterns of weight 1 or 0. Hence, all the 7-tuples of weight 1 or 0 can be used as coset leaders. There
are = 8 such vectors. We see that for the (7, 4) linear code considered in this example, the number of
correctable error patterns guaranteed by the minimum distance is equal to the total number of
correctable error patterns. The correctable error patterns and their corresponding syndromes are given
in Table 3.2.
Suppose that the code-word v = (1001011) is transmitted, and r = (1001111) is received. For decoding
r, we compute the syndrome of IC:
TABLE 3.2: Decoding table for the (7, 4) linear code given in Table 3.1.
Syndrome
From Table 3.2 we find that (0 1 1) is the syndrome of the coset leader e = (0000100). Thus, (0000100)
is assumed to be the error pattern caused by the channel, and r is decoded into
which is the code-word transmitted. The decoding is correct, since the error pattern caused by the
channel is a coset leader.
Now, suppose that v = (0000000) is transmitted, and r = (1000100) is received. We see that two errors
have occurred during the transmission of v. The error pattern is not correctable and will cause a
decoding error. When r is received, the receiver computes the syndrome:
From the decoding table we find that the coset leader e = (0000010) corresponds to the syndrome s =
(111). As a result, r is decoded into the code-word
Because v* is not the code-word transmitted, a decoding error is committed.
Using Table 3.2, we see that the code is capable of correcting any single error over a block of seven
digits. When two or more errors occur, a decoding error will be committed.
The table-lookup decoding of an (n, k) linear code may be implemented as follows. The decoding table
is regarded as the truth table of n switching functions:
where so, s1, …• sn-k _1 are the syndrome digits, which are regarded as switching variables, and eo,
el, …, are the estimated error digits. When these n switching functions are derived and simplified, a
combinational logic circuit with the n < k syndrome digits as inputs and the estimated error digits as
outputs can be realized. The implementation of the syndrome circuit was discussed in Section 3.2. The
general decoder for an (11, k) linear code based on the table-lookup scheme is shown in Figure 3.8.
FIGURE 3.8: General decoder for a linear block code.
The cost of this decoder depends primarily on the complexity of the combinational logic circuit.
12
106728959.doc
petak, 12. veljače 2016
9. EXAMPLE 3.9 Again, we consider the (7, 4) code given in Table 3.1
Again, we consider the (7, 4) code given in Table 3.1. The syndrome circuit for this code is shown in
Figure 3.5. The decoding table is given by Table 3.2. From this table we form the truth table (Table
3.3).
TABLE 3.3: Truth table for the error digits of the correctable error patterns of the (7, 4) linear code
given in Table 3.1.
The switching expressions for the seven error digits are
where A denotes the logic-AND operation and s' denotes the logic-COMPLEMENT of s. These seven
switching expressions can be realized by seven 3-input AND gates. The complete circuit of the
decoder is shown in Figure 3.9.
FIGURE 3.9: Decoding circuit for the (7, 4) code given in Table 3.1.
3.6.
3.6 Probability of an Undetected Error for Linear Codes over a BSC
If an (n, k) linear code is used only for error detection over a BSC, the probability of an undetected
error 11), (E) can be computed from (3.19) if the weight distribution of the code is known. There exists
an interesting relationship between the weight distribution of a linear code and the weight distribution
of its dual code. This relationship often makes the computation of P„ (E) much easier. Let fAo,
A1,. …, An} be the weight distribution of an (a, k) linear code C, and let { 0, B1,, B„) be the weight
distribution of its dual code Cd. Now, we represent these two weight distributions in polynomial form
as follows:
Then, A(z) and B(z) are related by the following identity:
This identity is known as the MacWilliams identity [13]. The polynomials A(z) and B(z) are called the
weight enumerators for the (a, k) linear code C and its dual C,1. From the MacWilliams identity, we
see that if the weight distribution of the dual of a linear code is known, the weight distribution of the
code itself can be determined. As a result, this gives us more flexibility in computing the weight
distribution of a linear code.
Using the MacWilliams identity, we can compute the probability of an undetected error for an (a, k)
linear code from the weight distribution of its dual. First, we put the expression of (3.19) into the
following form:
Substituting z = p/(1 - p) in A(z) of (3.31) and using the fact that Ao = 1, we obtain the following
identity:
Combining (3.33) and (3.34), we have the following expression for the probability of an undetected
error:
From (3.35) and the MacWilliams identity of (3.32), we finally obtain the following expression for P„
(E):
where
Hence, there are two ways for computing the probability of an undetected error for a linear code; often,
one is easier than the other. If n - k is smaller than k, it is much easier to compute P„ (E) from (3.36);
otherwise, it is easier to use (3.35).
10. EXAMPLE 3.10 Consider the (7, 4) linear code given in Table 3.1
Consider the (7, 4) linear code given in Table 3.1. The dual of this code is generated by its paritycheck matrix,
(see Example 3.3). Taking the linear combinations of the rows of !, we obtain the following eight
vectors in the dual code:
106728959.doc
petak, 12. veljače 2016
13
Thus, the weight enumerator for the dual code is B(z) = 1 + 7z4. Using (3.36), we obtain the
probability of an undetected error for the (7, 4) linear code given in Table 3.1:
This probability was also computed in Section 3.4 using the weight distribution of the code itself.
Theoretically, we can compute the weight distribution of an (n, k) linear code by examining its 2k
code-words or by examining the 2"-k code-words of its dual and then applying the MacWilliams
identity; however, for large n, k, and n < k, the computation becomes practically impossible. Except for
some short linear codes and a few small classes of linear codes, the weight distributions for many
known linear codes are still unknown. Consequently, it is very difficult, if not impossible, to compute
their probability of an undetected error.
Although it is difficult to compute the probability of an undetected error for a specific (n, k) linear
code for large n and k, it is quite easy to derive an upper bound on the average probability of an
undetected error for the ensemble of all (n, k) linear systematic codes. As we showed earlier, an (n, k)
linear systematic code is completely specified by a matrix G of the form given by (3.4). The submatrix
consists of k(n < k) entries. Because each entry pi,/can be either a 0 or a 1, there are 2k(7-k) distinct
matrices G's of the form given by (3.4). Let F denote the ensemble of codes generated by these 2 k("-k)
matrices. Suppose that we choose a code randomly from F and use it for error detection. Let C1 be the
chosen code.
Then, the probability that C1 will be chosen is
Let A11 denote the number of code-words in C1 with weight i. It follows from (3.19) that the
probability of an undetected error for C1 is given by
The average probability of an undetected error for a linear code in F is defined as
where in denotes the number of codes in F. Substituting (3.37) and (3.38) into (3.39), we obtain
A nonzero n-tuple is contained in either exactly 2(k-1)(") codes in F or in none of the codes (left as a
problem). is Because there are 77 n-tuples of weight i, we have
Substituting (3.41) into (3.40), we obtain the following upper bound on the average probability of an
undetected error for an (n, k) linear systematic code:
Because [1 - (1 - p)11 ] < 1, it is clear that 1?„(E) < 2-°1-k).
The preceding result says that there exist (n, k) linear codes with the probability of an undetected error,
Pi,(E), upper bounded by 2-("-h). In other words, there exist (n k) linear codes with P,(E) decreasing
exponentially with the number of parity-check digits, n < k. Even for moderate 7i - k, these codes have
a very small probability of an undetected error. For example, let n - k = 30. There exist (n, k) linear
codes for which P„ (E) is upper bounded by 2-30 10-9. Many classes of linear codes have been
constructed for the past five decades; however, only a few small classes of linear codes have been
proved to have a P„ (E) that satisfies the upper bound 2-°1-k). It is still not known whether the other
known linear codes satisfy this upper bound.
Good codes for error detection and their applications in error control will be presented in later chapters.
An excellent treatment of error-detecting codes can be found in [12].
3.7.
3.7 Single-Parity-Check Codes, Repetition Codes, and Self-Dual Codes
A single-parity-check (SPC) code is a linear block code with a single parity check digit. Let n = (no,
01, uk_) be the message to be encoded. The single parity-check digit is given by
which is simply the modulo-2 sum of all the message digits. Adding this parity-check digit to each kdigit message results in a (k +1, k) linear block code. Each code-word is of the form
From (3.43), we readily see that p = 1 if the weight of message u is odd, and p = 0 if the weight of
message 1111 is even. Therefore, all the code-words of a SPC code have even weights, and the
14
106728959.doc
petak, 12. veljače 2016
minimum weight (or minimum distance) of the code is 2. The generator of the code in systematic form
is given by
From (3.44) we find that the parity-check matrix of the code is
Because all the code-words have even weights, a SPC code is also called an even-parity-check code.
SPC codes are often used for simple error detection. Any error pattern with an odd number of errors
will change a code-word into a received vector of odd weight that is not a code-word. Hence, the
syndrome of the received vector is not equal to zero. Consequently, all the error patterns of odd weight
are detectable.
A repetition code of length n is an (n, 1) linear block code that consists of only two code-words, the
all-zero code-word (0 0 …• 0) and the all-one code-word (1 1 …1). This code is obtained by simply
repeating a single message bit n times. The generator matrix of the code is
From (3.44) through (3.46), we readily see that the (n, 1) repetition code and the (n, n < 1) SPC code
are dual codes to each other.
SPC and repetition codes are often used as component codes for constructing long, powerful codes as
will be seen in later chapters.
A linear block code C that is equal to its dual code Cd is called a self-dual code. For a self-dual code,
the code length n must be even, and the dimension k of the code must be equal to n/2. Therefore, its
rate R is equal to z. Let G be a generator matrix of a self-dual code C. Then, G is also a generator
matrix of its dual code Cd and hence is a parity-check matrix of C. Consequently,
Suppose CI is M systematic form, =, 7 /2[. From (3.47), we can easily see that
Conversely, if a rate-4 (n, n/2) linear block code C satisfies the condition of (3.47) [or (3.48)], then it is
a self-dual code (the proof is left as a problem).
11. EXAMPLE 3.11 Consider the (8, 4) linear block code generated by the matrix
Consider the (8, 4) linear block code generated by the matrix
The code has a rate R It is easy to check that GT = Therefore, it is a self-dual code.
There are many good self-dual codes but the most well known self-dual code is the (24, 12) Cola,/code,
which will be discussed in Chapter 4.
106728959.doc
petak, 12. veljače 2016
15