Dr. Bradley J. Bazuin
Assistant Professor
Department of Electrical and Computer Engineering
College of Engineering and Applied Sciences

Chapter 8: Trellis and Graph Based Codes
8.1 The Structure of Convolutional Codes
8.2
8.3
Decoding of Convolutional Codes
Distance Properties of Binary Convolutional Codes
8.4
8.5
8.6
8.7
8.8
Punctured Convolutional Codes
Other Decoding Algorithms for Convolutional Codes
491
491
510
516
516
525
Practical Considerations in the Application of Convolutional
Codes 532
Nonbinary Dual-k Codes and Concatenated Codes 537
Maximum a Posteriori Decoding of Convolutional Codes—The
BCJR Algorithm 541
8.9
8.10
8.11
8.12
8.13
Turbo Codes and Iterative Decoding
Factor Graphs and the Sum-Product Algorithm
Low Density Parity Check Codes
548
558
568
Coding for Bandwidth-Constrained Channels—Trellis Coded
Modulation 571
Bibliographical Notes and References 589
Problems 590
ECE 6640
Notes and figures are based on or taken from materials in the course textbook:
J.G. Proakis and M.Salehi, Digital Communications, 5th ed., McGraw-Hill, 2008 .
2

• Our focus in Chapter 7 and Chapter 8 is on channel coding schemes with manageable decoding algorithms that are used to improve performance of communication systems over noisy channels.
• Chapter 7 is devoted to block codes whose construction is based on familiar algebraic structures such as groups, rings, and fields.
– Reed Solomon to be delayed until after convolutional codes.
• In Chapter 8 we will study coding schemes that are best represented in terms of graphs and trellises.
ECE 6640 3

• Block Codes
– In block codes one of the M = 2 k messages, each representing a binary sequence of length k, called the information sequence, is mapped to a binary sequence of length n, called the codeword, where n > k.
– The codeword is usually transmitted over the communication channel by sending a sequence of n binary symbols, for instance, by using BPSK or another symbol scheme.
• Convolutional Codes
– Convolutional codes are described in terms of finite-state machines. In these codes, at each time instance i , k information bits enter the encoder, causing n binary symbols generated at the encoder output and changing the state of the encoder.
ECE 6640 4

• Linear block codes were studied in detail in Chapter 7. These codes are mainly used with hard decision decoding that employs the built-in algebraic structure of the code based on the properties of finite fields.
• Hard decision decoding of these codes results in a binary symmetric channel model consisting of the binary modulator, the waveform channel, and the optimum binary detector.
• The decoder for these codes tries to find the codeword at the minimum
Hamming distance from the output of the BSC.
• The goal in designing good linear block codes is to find the code with highest minimum distance for a given n and k.
• Coding involves a k-tuple being translated into an n-tuple with hard decoding performing the inverse process. This is a memoryless operations.
ECE 6640
Notes and figures are based on or taken from materials in the course textbook:
J.G. Proakis and M.Salehi, Digital Communications, 5th ed., McGraw-Hill, 2008 .
5

• Another class of codes whose structure is more conveniently described in terms of trellises or graphs.
• We will see that for this family of codes, soft decision decoding is possible, and in some cases performance very close to channel capacity is achievable
• In convolutional codes, there is memory. Each k-tuple effects not just the codeword generated when the value is input, by the next K-1 codewords as well. The parameter K is called the constraint length.
– The encoder necessarily defines a previous “state” based on the past history and memory.
– The decoder defines the most likely “path” taken by the symbols
ECE 6640
J.G. Proakis and M.Salehi, Digital Communications, 5th ed., McGraw-Hill, 2008 .
6

• In general, the shift register consists of K (k-bit) stages and n linear algebraic function generators.
• The number of output bits for each k-bit input sequence is n bits. Consequently, the code rate is defined as R c
= k/n.
ECE 6640
Notes and figures are based on or taken from materials in the course textbook:
J.G. Proakis and M.Salehi, Digital Communications, 5th ed., McGraw-Hill, 2008 .
7

• Each message, m i
, may be a k-tuple. (or k could be a bit)
• K messages are in the encoder
• For each message input, an n-tuple is generated
• The code rate is k/n
• We will usually be working with k=1 and n=2 or 3
ECE 6640
Notes and figures are based on textbook: Bernard Sklar, “Digital Communications,
Fundamentals and Applications,” Prentice Hall PTR, Second Edition, 2001.
8

• Several methods are used for representing a convolutional encoder, the most popular being;
– the connection pictorial
– connection vectors or polynomials
– state diagrams
– tree diagrams
– trellis diagrams.
ECE 6640 9

• k=1, n=2, K-3
• Generator Polynomials
– G1 = 1 + X + X 2
– G2 = 1 + X 2
• To end a message, K-1 “zero” messages are transmitted.
This allows the encoder to be flushed.
– effective code rate is different than k/n … the actual rate would be
(2+k*m_length)/n*m_length
– a zero initialized and tailed encoder ….
ECE 6640
Notes and figures are based on textbook: Bernard Sklar, “Digital Communications,
Fundamentals and Applications,” Prentice Hall PTR, Second Edition, 2001.
10

• allow a single “1” to transition through the K stages
– 100
– 010
– 001
– 000
-> 11
-> 10
-> 11
-> 00
• If the input message where 1 0 1
– 1
– 0
– 1
– Bsum
11
11
10
00
10
11
00
11
00
00
10
10
11
11
– Bsum is the transmitted n-tuple sequence …. if a 2 zero tail follows
– The sequence/summation involves superpoition or linear addition.
• The impulse response of one k-tuple sums with the impulse responses of successive k-tuples!
ECE 6640 11

• As each k-tuple is input, an n-tuple is output
• This is a rate ½ encoding
• The “constraint” length is
K=3, the length of the k-tuple shift register.
• The effective code rate for m_length = 3 is: 3/10
ECE 6640
Notes and figures are based on textbook: Bernard Sklar, “Digital Communications,
Fundamentals and Applications,” Prentice Hall PTR, Second Edition, 2001.
12

• Generator Polynomials (also represented in octal)
– G1 = 1 + X + X 2  7 octal
– G2 = 1 + X 2  5 octal
• It is assumed to be a binary input.
• There are two generator polynomials, therefore n=2
– each polynomials generates one of the elements of the n-tuple output
• Polynomial Multiplication can be used to generate output sequences
• m(X)*g1(X) = (1 + X 2 )* (1 +X+ X 2) = 1 +X+X 3 + X 4
• m(X)*g2(X) = (1 + X 2 )* (1 + X 2) = 1 + X 4
• Output: (11 , 10 , 00, 10, 11) as before
ECE 6640 13

• Using the same encoding:
• Solid lines represent 0 inputs
• Dashed lines represent 1 inputs
• The n-tuple output is shown with the state transition
• It can be verified that 2 zeros always returns to the same
“steady state”
• Note: two previous k-tuples provide state, the new k-tuple drives transitions
ECE 6640
Notes and figures are based on textbook: Bernard Sklar, “Digital Communications,
Fundamentals and Applications,” Prentice Hall PTR, Second Edition, 2001.
14

• Input values define where to go next.
• Each set of branch outputs transmit complementary n–tuples
• States can be identified by repeated level operations
ECE 6640
Notes and figures are based on textbook: Bernard Sklar, “Digital Communications,
Fundamentals and Applications,” Prentice Hall PTR, Second Edition, 2001.
15

• Initial state, state development, continuous pattern observable, “fully engaged” after K-1 inputs, easily trace tail zeros to “initial state”.
ECE 6640
Notes and figures are based on textbook: Bernard Sklar, “Digital Communications,
Fundamentals and Applications,” Prentice Hall PTR, Second Edition, 2001.
16

• Start with the desired code structure, k, n, K.
– Determine the connection vectors or polynomials
– Define the “circuit schematic” or connection pictorial
• Pick the preferred diagrammatic display
– state diagrams
– tree diagrams
– trellis diagrams.
• For decoding, the trellis is likely the desired form …
ECE 6640 17

• Convolutional Code (3,1), rate 1/3, n=3, k=1, K=3
• Generator Polynomials
– G1 = 1
– G2 = 1 + D 2
– G3= 1 + D + D 2
ECE 6640 18

Example 8.1-1
• Generator Polynomials (also represented in octal)
– G1 = 1
– G2 = 1 + D 2
– G3= 1 + D + D 2
• It is assumed to be a binary input.
• There are three generator polynomials, therefore n=3
• Polynomial Multiplication can be used to generate output sequences u = (1 0 0 1 1 1)
• c1 = u(D)*g1(D) = (1 + D 3 + D 4 + D 5 )* (1 )
• c2 = u(D)*g2(D) = (1 + D 3 + D 4 + D 5 )* (1 + D 2 )
• c3 = u(D)*g3(D) = (1 + D 3 + D 4 + D 5 )* (1 +D+ D 2)
ECE 6640 19

Example 8.1-1
• EXAMPLE 8.1–1. Let the sequence u = (100111) be the input sequence to the convolutional encoder shown in
Figure 8.1–2.
ECE 6640
6 bits in plus 2 zero tail bits (flush memory)
8 x 3 = 24 bit output sequence 20

• The 3 bits shown next to each branch in the state diagram represent the output bits. A dotted line in the graph indicates that the input bit is a 1, while the solid line indicates that the input bit is a 0.
ECE 6640 21

ECE 6640
g
1 g
2 g
3





1
1
0
0
0
1



1 1 1

22

g
1 g
2 g
3





1
1
0
0
0
1



1 1 1

• The 3 bits shown next to each branch in the state diagram represent the output bits. A dotted line in the graph indicates that the input bit is a 1, while the solid line indicates that the input bit is a 0.
ECE 6640 23

• Consider the rate 2/3 convolutional encoder illustrated in
Figure 8.1–3. In this encoder, 2 bits at a time are shifted into it, and 3 output bits are generated.
• Structural Equivalence – a matrix structure
– requires “alternating” generator matrix and code output
ECE 6640 24

ECE 6640
25

ECE 6640
Note: the state is based on the 2 bits required (memory) when the next 2-bit input appears.
26

ECE 6640
27

ECE 6640
28

• Each approach can be readily implemented in hardware.
• Good codes have been found be computer searches for each value of the constraint length, K.
• The easy part, now for decoding.
ECE 6640 29

• As the codes have memory, we wish to use a decoder that achieves the minimum probability of error … using a condition called maximum likelihood.
• But first there is the Transfer Function of a Convolutional
Code
ECE 6640 30

• the textbook outlines a procedure for “tracing” from an
“input” to return to the “output”
– moving from state a back to state a!
• ignore trivial path (a to a)
• describe path transition as number of output “ones”
• determine state equations
ECE 6640 31

• The transfer function for the code is defined as
T(Z) = X e
/X a
.
• By solving the state equations given above, we obtain
ECE 6640 32

• The transfer function for this code indicates that there is a single path of Hamming distance d = 6 from the all-zero path that merges with the all-zero path at a given node.
– The transfer functions defines who many “ones” will be output based on 1 or more errors in decoding.
– If all 0’s are transmitted and a “state-error” occurs, there will be 6 ones transmitted before returning to the “base”/correct state and a cycle consisting of acba will have to be completed.
– Such a path is called a first event error and is used to bound the error probability of convolutional codes
• The transfer function T (Z) introduced above is similar to the weight enumeration function (WEF) A(Z) for block codes introduced in Chapter 7.
ECE 6640 33

• The transfer function can be used to provide more detailed information than just the distance of the various paths.
– Suppose we introduce a factor Y into all branch transitions caused by the input bit 1. Thus, as each branch is traversed, the cumulative exponent on
Y increases by 1 only if that branch transition is due to an input bit 1.
– Furthermore, we introduce a factor of J into each branch of the state diagram so that the exponent of J will serve as a counting variable to indicate the number of branches in any given path from node a to node e.
ECE 6640 34

• The transfer function can be used to provide more detailed information than just the distance of the various paths.
– Suppose we introduce a factor Y into all branch transitions caused by the input bit 1. Thus, as each branch is traversed, the cumulative exponent on
Y increases by 1 only if that branch transition is due to an input bit 1.
– Furthermore, we introduce a factor of J into each branch of the state diagram so that the exponent of J will serve as a counting variable to indicate the number of branches in any given path from node a to node e.
ECE 6640 35

• This form for the transfer functions gives the properties of all the paths in the convolutional code.
– That is, the first term in the expansion of T (Y, Z, J ) indicates that the distance d = 6 path is of length 3 and of the three information bits, one is a 1.
– The second and third terms in the expansion of T (Y, Z, J ) indicate that of the two d = 8 terms, one is of length 4 and the second has length 5.
ECE 6640 36

• As the codes have memory, we wish to use a decoder that achieves the minimum probability of error … using a condition called maximum likelihood.
ECE 6640 37

• There are various estimators for signals combined with random variables.
• In general we are interested in the maximum a-posteriori estimator of X for a given observation Y.
max x
P

X
 x
|
Y
 y

– this requires knowledge of the a-priori probability, so that
P

X
 x
|
Y
 y
  Y
 y
|
P
X

Y

 x y
 

X
 x

ECE 6640 38

• In some situations, we know
P

Y
 y
|
X
 but not the a-priori probabilities. x

• In these cases, we form a maximum likelihood estimate
For a maximum likelihood estimate, we perform max x
P

Y
 y
|
X
 x

ECE 6640 39

• A sequence or “chain” of subexperiments in which the outcome of a given subexperiment determines which subexperiment is performed next.
P
 s
0
, s
0
,  , s n
  s n
| s n
 1
  s n
| s n
 1
  s n
 1
| s n
 2

  
P
 s
1
| s
0
  
• If the output from the previous state in a trellis is known, the next state is only based on the previous state and the new input.
– the decoder can be computed one step at a time to determine the maximum likelihood path.
• Viterbi’s improvement on this concept.
ECE 6640
– In a Trellis, there is a repetition of states. If two paths arrive at the same state, only the path with the maximum likelihood must be maintained … the “other path” can no longer become the ML path!
40

• If all input message sequences are equally likely, the decoder that achieves the minimum probability of error is the one that compares the conditional probabilities of all possible paths against the received sequence.
P

Z | U

 max P

Z | all U
– where U are the possible message paths
U

• For a memoryless channel we can base the computation on the individual values of the observed path Z
P

Z
|
U

 i


 1
P

Z i
|
U i

 i
 

 1 j
 1
P z
 ji
| u ji

ECE 6640
– where Zi is the ith branch of the received sequence Z, zji is the jth code symbol of Zi and similarly for U and u.
41

• As the probability is a product of products, computation precision and the final magnitude is of concern.
• By taking the log of the products, a summation may be performed instead of multiplications.
– constants can easily be absorbed
– similar sets of magnitudes can be pre-computed and/or even scaled to more desirable values.
– the precision used for the values can vary as desired for the available bit precision (hard vs. soft values)
ECE 6640 42

• Our previous symbol determinations selected a detected symbol with no other considerations … a hard decision.
• The decision had computed metrics that were used to make the determination that were then discarded.
• What if the relative certainty of decision were maintained along with the decision.
– if one decision influenced another decision, hard decisions keep certainty from being used.
– maintaining a soft decision may allow overall higher decision accuracy when an interactions exists.
ECE 6640 43

• Bit error probability
– P(0|1)=P(1|0) = p
– P(1|1)=P(0|0) = 1-p
• Suppose Z and any possible message U differ in dm positions (related to the hamming distance). Then the ML probability for an L bit message becomes
P

Z
|
U

 p dm 

1  p

L
 dm
• taking the log log P

Z | U

 dm
 log
  
L
 dm

 log

1  p
 log
P

Z
|
U

  dm
 log


  p p




L
 log

1  p

ECE 6640 44

• The ML value for each possible U is then log P

Z | U

  dm
 log


  p p




L
 log

1  p

– The constant is identical for all possible U and can be pre-computed
– The log of the probability ratios is also a constant log P

Z | U

  dm

A

B
• Overall, we are looking for the possible sequence with a minimum Hamming distance.
– for hard decisions, we use the Hamming distance
– for soft decisions, we can use the “certainty values” shown in the previous figure!
ECE 6640 45

• The previous slide suggested that all possible U should be checked to determine the minimum value (maximum likelihood).
– If we compute the “metrics” for each U as they arrive, the trellis structure can reduce the number of computations that must be performed.
– For a 2^K-1 state trellis, only that number of possible U paths need to be considered.
• Each trellis state has two arriving states. If we compute path values for each one, only the smallest one needs to be maintained. The larger can never become smaller as more n-tuples arrive!
• Therefore, only 2^K-1 possible paths vs. 2^L possible paths for U must be considered!
ECE 6640 46

• Decoder Trellis with Hamming distances shown for each of the possible paths from “state to state”.
encoder trellis
ECE 6640
Notes and figures are based on textbook: Bernard Sklar, “Digital Communications,
Fundamentals and Applications,” Prentice Hall PTR, Second Edition, 2001.
47

• m: 1 1 0 1 1
• U: 11 01 01 00 01
• Z: 11 01 01 10 01
merging paths
ECE 6640
Notes and figures are based on textbook: Bernard Sklar, “Digital Communications,
Fundamentals and Applications,” Prentice Hall PTR, Second Edition, 2001.
48

• m: 1 1 0 1 1
• U: 11 01 01 00 01
• Z: 11 01 01 10 01
ECE 6640
Notes and figures are based on textbook: Bernard Sklar, “Digital Communications,
Fundamentals and Applications,” Prentice Hall PTR, Second Edition, 2001.
49

• Section 7.3.5.1, p. 406
Possible Connections
ECE 6640
Notes and figures are based on textbook: Bernard Sklar, “Digital Communications,
Fundamentals and Applications,” Prentice Hall PTR, Second Edition, 2001.
50

• State Metric Update based on new Branch Metric Values
– Hard coding uses bit difference measure
– Soft coding uses rms distances between actual and expected branch values
– The minimum path value is maintained after comparing incoming paths.
– Paths are eliminated that are not maintained.
• When all remaining paths use the same branch, update the output sequence
ECE 6640 51

• See doc comm.ViterbiDecoder
– matlab equaivalent to Dr. Bazuin’s trellis simulation structure
– Use:
– t = poly2trellis(K,[g1 g2 g3])
– t2 = poly2trellis([3],[7 7 5])
– t2.outputs
– t2.nextStates
ECE 6640 52

• Communication Objects.
– see ViterbiComm directory for demos
– TCM Modulation
• comm.PSKTCMModulator
• comm.RectangularQAMTCMModulator
• comm.GeneralQAMTCMModulator
– Convolutional Coding
• comm.ConvolutionalEncoder
• comm.ViterbiDecoder (Hard and Soft)
• comm.TurboEncoder – available from Matlab, no demo
ECE 6640 53

• Distance Properties
– If an all zero sequence is input and there is a bit error, how and how long will it take to return to an all zeros path?
– Find the “minimum free distance”
• The number of code bit errors required before returning
• Note that this is not time steps and not states moved through
• This determines the error correction capability t 
 d f
2
1 

ECE 6640
– Systematic and non systematic codes
• For linear block codes, any non-systematic code can be transformed into a systematic code (structure with I and data in columns)
• This is not true for convolutional codes. Convolutional codes focus on free distance, making them systematic would reduce the distance!
54

• A necessary and sufficient condition to have catastrophic error propagation is that the generator polynomials have a common factor.
– Catastrophic errors is when a finite number of code symbol errors can generate an infinite number of decoded data bit errors.
– See Sklar Section 7.4.3 and p. 414.
– See Proakis Section 8.1-4 and p. 508.
ECE 6640 55

• The following are equivalent to the transfer function discussion before.
• The focus is on finding the “minimum free distance”.
– In Proakis, we want d/dY(T(Y,Z)) at Y=1 and Z=“delta”
– Augmented transfer function T(Y,Z,J) where Z referes to the ones transmitted during the transfer from “a” to “e” and Y the “1” input bits.
• In Sklar, the variables used are N=Y and D=Z.
ECE 6640 56

• Split the state diagram to start at 00.. and end at 0..
– Show state transitions with the following notations
– D: code bit errors for a path
– L: one factor for every branch
– N: one factor for every branch taken due to a “1” input
• Define the state equations using the state diagram
– Determine the result with the smallest power of D and interpret
– See Figure 7.18 and page p. 412
– Proakis Figure 8.1-13 and page 503.
ECE 6640
Notes and figures are based on textbook: Bernard Sklar, “Digital Communications,
Fundamentals and Applications,” Prentice Hall PTR, Second Edition, 2001.
57

• Upper Bound of bit error probability
P
B
 dT

D , N
 dN
N  1 , D  2  p 

1  p

– for Figure 7.18 and Eq. 7.15 on p. 412 dT

D , N
 dN


1
 
1


2
D

2
5

D
D

1


N

N
2





1
D
2


D
2 
5

N
N

2


D
N
D


N


2
D 5

T

D , N


 2  D


D 5  N
1  2  D  N

D 5

1  2  D  N

2
P
B

D 5

1  2  D  N

2
N  1 , D  2  p 

1  p


1


2
 4

 p  p 

1  p


1  p

5


2
ECE 6640
Notes and figures are based on textbook: Bernard Sklar, “Digital Communications,
Fundamentals and Applications,” Prentice Hall PTR, Second Edition, 2001.
58

• For
• The bound becomes
E
C
N
0
 r 
E b
N
0
 k n

E b
N
0
P
B
 Q




5 
N
E
0 b




 exp


 5  E b
2  N
0




1



1  2  exp



2


E b
N
0






2
ECE 6640
Notes and figures are based on textbook: Bernard Sklar, “Digital Communications,
Fundamentals and Applications,” Prentice Hall PTR, Second Edition, 2001.
59

• From Eq. 6.19
G 


E b
N
0

 uncoded



E b
N
0

 coded
 
, for P b
 Same Value
• This is bounded by
– The 10 log base 10 of the code rate and the min. free distance
G  10  log
10
 r  d f

• Coding Gains are shown in Tables 7.2 and 7.3, p. 417
ECE 6640
Notes and figures are based on textbook: Bernard Sklar, “Digital Communications,
Fundamentals and Applications,” Prentice Hall PTR, Second Edition, 2001.
60

• The sequence error probability is bounded by
P e

T
Z
 
  y

 path p
 y
| 0
  
– in terms of the transfer function d
P e
 d


 d free a d
  d
– and for convolutional codes based on the Chap 7 derivation (7.2)
P b

1 k


T

Y
,
Z


Y
Y
 1 , Z
 
ECE 6640
– for soft decisions
– for hard decision
  exp


R c
  b

  2  p


1  p

61

ECE 6640
• Additional error bounds and computations
– Hard decision pairwise errors (p. 514-15)
P
2
 
 k d 

 d
 1

2


 d k 


 p k 

1  p
 n
 k
P
2

1
2





1 d

2 d




 p d
2 

1  p
 d
2
 k d 
 d 2


 d k 


 p k 

1  p
 n
 k
P e
 d


 d free a d

P
2
62

• The previous example used fractional “soft values”
– See the Viterbi example slides on line
• For digital processing hardware: use integer values and change the observed code to the maximum integer values
– For 0, 1 in an 8-level system use 0,7
– Compute distances as the rms value from the desired received code to the observed received code.
• Note that only 4 values need to be computed to define all branch metric values,
• Example: see Fig. 7.22 for (0,0) and (7,7) now computed distances from (0,7) and (7,0) and you have all 4!
– Apply computations and comparisons as done before.
ECE 6640
Notes and figures are based on textbook: Bernard Sklar, “Digital Communications,
Fundamentals and Applications,” Prentice Hall PTR, Second Edition, 2001.
63

Section 8.5:
• Sequential Decoding Algorithm, p. 525
• Stack Algorithm, p. 528
• Feedback Decoding, p. 529
ECE 6640 64

• Existed prior to Viterbi
• Generate a hypothesis about the transmitted sequence
– Compute metric between hypothesis and received signal
– If metric indicates reasonable agreement, go forward otherwise go backward, change the hypothesis and keep trying.
– Proakis p. 525 or Sklar section 7.5 and p. 422-425.
• Complexity
– Viterbi grows exponentially with constraint length
– Sequential is independent of the constraint length
• Can have buffer memory problems at low SNR (many trials)
ECE 6640
Notes and figures are based on textbook: Bernard Sklar, “Digital Communications,
Fundamentals and Applications,” Prentice Hall PTR, Second Edition, 2001.
65

• Use a look-ahead approach to determine the minimum
“future” hamming distance
– Look ahead length, L, has received code symbols forward in time
– Compare look-ahead paths for minimum hamming distance and take the tree branch that contains the minimum value ….
• Proakis p. 528 or Sklar Section 7.5.3 on p. 427-429
• Called a feedback decoder as detection decisions are required as feedback to compute the next set of code paths to search for a minimum.
ECE 6640
Notes and figures are based on textbook: Bernard Sklar, “Digital Communications,
Fundamentals and Applications,” Prentice Hall PTR, Second Edition, 2001.
66

• http://home.netcom.com/~chip.f/viterbi/tutorial.html
• http://www.eccpage.com/
• B. Sklar, "How I learned to love the trellis," in IEEE Signal Processing
Magazine, vol. 20, no. 3, pp. 87-102, May 2003.
– http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=1203212
ECE 6640 67

• Convolutional codes are widely used in many practical applications of communication system design.
– The choice of constraint length is dictated by the desired coding gain.
– Viterbi decoding is predominantly used for short constraint lengths
(K ≤ 10)
– Sequential decoding is used for long-constraint-length codes, where the complexity ofViterbi decoding becomes prohibitive.
ECE 6640 68

• Two important issues in the implementation of Viterbi decoding are
1. The effect of path memory truncation, which is a desirable feature that ensures a fixed decoding delay.
2. The degree of quantization of the input signal to the Viterbi decoder.
• As a rule of thumb, we stated that path memory truncation to about five constraint lengths has been found to result in negligible performance loss.
• In addition to path memory truncation, the computations were performed with eight-level (three bits) quantized input signals from the demodulator.
ECE 6640 69

ECE 6640
• Figure 8.6–2 illustrates the performance obtained by simulation for rate 1/2, constraint-lengths K = 3, 5, and 7 codes with memory path length of 32 bits.
• The broken curves are performance results obtained from the upper bound in the bit error rate given by Equation 8.2–
12.
• Note that the simulation results are close to the theoretical upper bounds, which indicate that the degradation due to path memory truncation and quantization of the input signal has a minor effect on performance (0.20–0.30 dB).
70

• ViterbiComm
– poly2trellis: define the convolutional code and trellis to be used
– istrellis: insuring that the trellis is valid and not catastrophic
– distpec: computes the free distance and the first N components of the weight and distance spectra of a linear convolutional code.
– comm.ConvolutionalEncoder
– quantiz - a quantization index and a quantized output value allowing either a hard or soft output value
– comm.ViterbiDecoder – either hard or soft decoding
– bercoding
ECE 6640
– Viterbi_Hard.m
– Viterbi_Soft.m
71

• Binary codes are particularly suitable for channels in which binary or quaternary PSK modulation and coherent demodulation is possible.
– However, there are many applications in which PSK modulation and coherent demodulation is not suitable or possible.
– In such cases, other modulation techniques, e.g., M-ary FSK, are employed in conjunction with noncoherent demodulation.
• Nonbinary codes are particularly matched to M-ary signals that are demodulated noncoherently.
• Just as Reed-Solomon Codes involved non-binary codes, there are dual-k codes that are easily decoded by means of the Viterbi algorithm using either soft-decision or hard-
ECE 6640 decision decoding.
72

ECE 6640
D free
=4
73

• In Section 7.13–2, we considered the concatenation of two
• block codes to form a long block code.
• Now that we have described convolutional codes, we broaden our viewpoint and consider the concatenation of a block code with a convolutional code or the concatenation of two convolutional codes.
ECE 6640 74

• In a conventional concatenated code, the outer code is usually chosen to be nonbinary, with each symbol selected from an alphabet of q = 2 k symbols.
– This code may be a block code, such as a Reed–Solomon code, or a convolutional code, such as a dual-k code.
• The inner code may be either binary or nonbinary, and either a block or a convolutional code.
– For example, a Reed–Solomon code may be selected as the outer code and a dual-k code may be selected as the inner code. In such a concatenation scheme, the number of symbols in the outer (Reed–
Solomon) code q equals 2 k , so that each symbol of the outer code maps into a k-bit symbol of the inner dual-k code.
– M-ary orthogonal signals may be used to transmit the symbols.
ECE 6640 75

• An alternate Decoding methodology based on MAP
• The BCJR algorithm, named after Bahl, Cocke, Jelinek, and Raviv Bahl et al. (1974), is a symbol-by-symbol maximum a posteriori decoding algorithm for convolutional codes.
• In this algorithm the decoder uses the MAP algorithm to decode each input symbol to the decoder rather than looking for the most likely input sequence.
ECE 6640 76

• From BCJR to turbo decoding: MAP algorithms made easier, Silvio
A. Abrantes, April 2004.
– http://paginas.fe.up.pt/~sam/textos/From%20BCJR%20to%20turbo.pdf
• Wikipedia: BCJR Algorithm
– https://en.wikipedia.org/wiki/BCJR_algorithm
• L. Bahl, J. Cocke, F. Jelinek and J. Raviv, "Optimal decoding of linear codes for minimizing symbol error rate (Corresp.)," in IEEE
Transactions on Information Theory, vol. 20, no. 2, pp. 284-287, Mar
1974.
– http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=1055186&tag=1
• Sichun WangEmail and François Patenaude, “A Systematic Approach to Modified BCJR MAP Algorithms for Convolutional Codes”,
EURASIP Journal on Advances in Signal Processing, 13 April 2006.
– http://asp.eurasipjournals.springeropen.com/articles/10.1155/ASP/2006/95360
ECE 6640 77

• The Algorithm uses maximum likelihood ratios to generate the MAP based on backward, current and forward recursive computations.
• It is an involved derivation that may best be described in the first reference provided.
– It includes numerical examples of the processing to be performed on an example trellis.
ECE 6640 78

• The construction and decoding of concatenated codes with interleaving, using convolutional codes.
• Parallel concatenated convolutional codes (PCCCs) with interleaving, also called turbo codes, were introduced by Berrou et al. (1993) and
Berrou and Glavieux (1996).
• A basic turbo encoder, shown in Figure 8.9–1, is a recursive systematic encoder (RSC or RSCC) that employs two recursive systematic convolutional encoders in parallel, where the second encoder is preceded by an interleaver.
ECE 6640 79

• We observe that the nominal rate at the output of the turbo encoder is Rc = 1/3.
• As in the case of concatenated block codes, the interleaver is usually selected to be a block pseudorandom interleaver that reorders the bits in the information sequence before feeding them to the second encoder.
• In effect, as will be shown later, the use of two recursive convolutional encoders in conjunction with the interleaver produces a code that contains very few codewords of low weight.
ECE 6640
– The use of the interleaver in conjunction with the two encoders results in codewords that have relatively few nearest neighbors.
That is, the codewords are relatively sparse.
80

• EXAMPLE 8.9–1.
– A (31, 27) RSC encoder is represented by g1 = (11001) and g2 =(10111) corresponding to g
1
(D) = 1+ D + D 4 g
2
(D) = 1+ D 2 + D 3 + D 4 .
– The encoder is given by the block diagram shown in Figure 8.9–2.
ECE 6640 81

• Turbo codes are two recursive systematic convolutional codes concatenated by an interleaver.
• Although the codes are linear and time-invariant, the operation of the interleaver, although linear, is not timeinvariant.
• The trellis of the resulting linear but time-varying finitestate machine has a huge number of states that makes maximum-likelihood decoding hopeless.
• Therefore the txt offers a “union bound” approach but refers readers to other papers.
ECE 6640 82

• A suboptimal iterative decoding algorithm, known as the turbo decoding algorithm, was proposed by Berrou et al. (1993) which achieves excellent performance very close to the theoretical bound predicted by Shannon.
• The turbo decoding algorithm is based on iterative usage of the Log-APP or the Max-Log-APP algorithm. (APP: a-posteriori probability) a BCJR simplification described on p. 546.
• A soft-input soft-output decoder is used that allows multiple iterations to be performed.
ECE 6640 83

ECE 6640
It is seen from these plots that three regions are distinguishable.
• For the low-SNR region where the error probability changes very slowly as a function of
Eb/N
0 and the number of iterations.
• For moderate SNRs the error probability drops rapidly with increasing Eb/N
0 and over many iterations Pb decreases consistently. This region is called the waterfall region or the turbo cliff region.
• Finally, for moderately large Eb/N
0 values, the code exhibits an error floor which is typically achieved with a few iterations.
• As discussed before, the error floor effect in turbo codes is due to their low minimum distance.
84

• The major drawback with decoding turbo codes with large interleavers is the decoding delay and the computational complexity inherent in the iterative decoding algorithm.
• In most data communication systems, however, the decoding delay is tolerable, and the additional computational complexity is usually justified by the significant coding gain that is achieved by the turbo code.
ECE 6640 85

• C. Berrou, A. Glavieux and P. Thitimajshima, "Near Shannon limit errorcorrecting coding and decoding: Turbo-codes. 1," Communications, 1993. ICC
'93 Geneva. Technical Program, Conference Record, IEEE International
Conference on, Geneva, 1993, pp. 1064-1070 vol.2.
– http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=397441&tag=1
• C. Berrou, P. Adde, E. Angui and S. Faudeil, "A low complexity soft-output
Viterbi decoder architecture," Communications, 1993. ICC '93 Geneva.
Technical Program, Conference Record, IEEE International Conference on,
Geneva, 1993, pp. 737-740 vol.2.
– http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=397371
• C. Berrou and A. Glavieux, "Near optimum error correcting coding and decoding: turbo-codes," in IEEE Transactions on Communications, vol. 44, no. 10, pp. 1261-1271, Oct 1996.
– http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=539767
ECE 6640 86

• In the treatment of block and convolutional codes, performance improvement was achieved by expanding the bandwidth of the transmitted signal by an amount equal to the reciprocal of the code rate. This situation applies to digital communication systems that are designed to operate in the power-limited region where R/W < 1.
• We consider the use of coded signals for bandwidth-constrained channels. For such channels, the digital communication system is designed to use bandwidth-efficient multilevel amplitude and phase modulation, such as PAM, PSK, DPSK, or QAM, and operates in the region where R/W > 1.
• This goal can be achieved by increasing the number of signals over the corresponding uncoded system to compensate for the redundancy introduced by the code … send more bits per symbol!
ECE 6640 87

• if the modulation is an integral part of the encoding process and is designed in conjunction with the code to increase the minimum
Euclidean distance between pairs of coded signals, the loss from the expansion of the signal set is easily overcome and a significant coding gain is achieved with relatively simple codes.
• The key to this integrated modulation and coding approach is to devise an effective method for mapping the coded bits into signal points such that the minimum Euclidean distance is maximized.
• Such a method was developed by Ungerboeck (1982), based on the principle of mapping by set partitioning
ECE 6640 88

EXAMPLE 8.12–1. AN 8-PSK SIGNAL
CONSTELLATION.
• Let us partition the eight-phase signal constellation shown in Figure 8.12–1 into subsets of increasing minimum Euclidean distance. In the eight-phase signal set, the signal points are located on a circle of radius sqrt(E) and have a minimum distance separation of
ECE 6640 89

• In the first partitioning, the eight points are subdivided into two subsets of four points each, such that the minimum distance between points increases to d1 = ssqrt(2E).
• In the second level of partitioning, each of the two subsets is subdivided into two subsets of two points, such that the minimum distance increases to d2 = 2 x sqrt(E). This results in four subsets of two points each.
• Finally, the last stage of partitioning leads to eight subsets, where each subset contains a single point. Note that each level of partitioning increases the minimum Euclidean distance between signal points.
ECE 6640 90

• A block of m information bits is separated into two groups of length k1 and k2 respectively.
• The k1 bits are encoded into n bits, while the k2 bits are left uncoded. Then, the n bits from the encoder are used to select one of the possible subsets in the partitioned signal set, while the k2 bits are used to select one of 2 k2 signal points in each subset. When k2 = 0, all m information bits are encoded.
ECE 6640 91

• The assignment of signal subsets to state transitions in the trellis is based on three heuristic rules
1. Use all subsets with equal frequency in the trellis.
2. Transitions originating from the same state or merging into the same state in the trellis are assigned subsets that are separated by the largest
Euclidean distance.
3. Parallel state transitions (when they occur) are assigned signal points separated by the largest
Euclidean distance. Parallel transitions in the trellis are characteristic of TCM that contains one or more uncoded information bits.
ECE 6640 92

ECE 6640
93

• G. Ungerboeck, "Channel coding with multilevel/phase signals," in IEEE Transactions on
Information Theory, vol. 28, no. 1, pp. 55-67, Jan 1982.
• G. Ungerboeck, "Trellis-coded modulation with redundant signal sets Part I: Introduction," in IEEE Communications Magazine, vol. 25, no. 2, pp. 5-11, February 1987.
• G. Ungerboeck, "Trellis-coded modulation with redundant signal sets Part II: State of the art," in IEEE Communications Magazine, vol. 25, no. 2, pp. 12-21, February 1987.
ECE 6640 94

• Additional performance values and possible configurations are provided.
• MATLAB has a range of TCM based tools for communication available.
• see ViterbiComm.zip file
• PSK-TCM and QAM-TCM demonstrations based on
MATLAB code are available.
ECE 6640 95

• K. Larsen, "Short convolutional codes with maximal free distance for rates
1/2, 1/3, and 1/4 (Corresp.)," in IEEE Transactions on Information Theory, vol. 19, no. 3, pp. 371-372, May 1973.
– http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=1055014
• E. Paaske, "Short binary convolutional codes with maximal free distance for rates 2/3 and 3/4 (Corresp.)," in IEEE Transactions on Information Theory, vol. 20, no. 5, pp. 683-689, Sep 1974.
– http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=1055264
• J. Conan, "The Weight Spectra of Some Short Low-Rate Convolutional
Codes," in IEEE Transactions on Communications, vol. 32, no. 9, pp. 1050-
1053, Sep 1984.
– http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=1096180
• Jinn-Ja Chang, Der-June Hwang and Mao-Chao Lin, "Some extended results on the search for good convolutional codes," in IEEE Transactions on
Information Theory, vol. 43, no. 5, pp. 1682-1697, Sep 1997.
– http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=623175
ECE 6640 96

• A Tutorial on Convolutional Coding with Viterbi Decoding by Chip Fleming of Spectrum Applications
– http://home.netcom.com/~chip.f/viterbi/tutorial.html
• Robert Morelos-Zaragoza, The Error Correcting Codes (ECC) Page
– http://www.eccpage.com/
• Matthew C. Valenti, Center for Identification Technology Research (CITeR),
West Virginia University Site
– http://www.csee.wvu.edu/~mvalenti/turbo.html
• Digital Communications I: Modulation and Coding Course, Period 3 – 2006,
Sorour Falahati, Lecture 13
ECE 6640 97