Channel coding
Abdelwaheb Marzouki
14/10/2025
Schedule of this tutorial
 Continuation of Monday lesson
 Group1: Block channel codinf in 3G and 4G cellular systems
• Including the decoding and the division circuit representation
 Group2: Convolutional and Turbo coding in 3G and 4 cellular systems
• Including the Viterbi decoder with explanatory example
 Rules to follow in your presentations
• The groups’ presentations have to cover at least the slides provided in the
present ppt file
• If diagrams and formulas are provided in this file, it is forbidden to copy others
on websites
• You can copy other diagrams and other formulas if they are complementary to
those present in this ppt file and provided that you indicate the references.
• The work must be done in groups and each member of the group must be able
to present any part of the work by himself.
• It is imperative that all group members remain facing the audience until the end
of the questions and answers session.
page 1
direction ou services
Directional channel models
Essential for the study of systems
incorporating smart antennas
 include the directional information
of the signals
• The dispersion of signal power in
time and the dispersion of power
in angle.
 Double-directional impulse
TX: N antennas
response: h(t,, ,  )
• : direction of arrival (DOA);
• : direction of departure (DOD);

- DOA and DOD vary slowly
- The phase varies quickly
page 2
direction ou services
N
2
1
dt
M
scatter 2
2
1
dr
2
1
2
3
1
RX: M antennas
3
Wideband MIMO channel
Frequency description

Rem: Wideband channel matrix in time domain
• 𝐇 𝝉 = σ𝒍=𝑳
𝒍=𝟏 𝐇𝐥 𝜹 𝜏 − 𝜏𝑙 ∶ 𝑵𝒙𝑴 𝒎𝒂𝒕𝒓𝒊𝒙 𝒇𝒖𝒏𝒄𝒕𝒊𝒐𝒏
• 𝐇𝑙 = ℎ 𝜏𝑙 , 𝜓𝑙 , 𝜃𝑙 𝐚𝑅𝑋 𝜃𝑙 𝐚 𝑇𝑋 𝜓𝑙 𝐻 : 𝑜𝑛𝑒 𝑟𝑎𝑛𝑘 𝑁𝑥𝑀 𝑚𝑎𝑡𝑟𝑖𝑥
• 𝐚𝑅𝑋 𝜃𝑙 and 𝐚 𝑇𝑋 𝜓𝑙 are called steering vectors
 Steering vectors for Uniform antenna array case
• 𝐚 𝑇𝑋,𝑙 = 1, 𝑒 −𝑗2𝜋Ψ𝑙 , … , 𝑒 −𝑗2𝜋(𝑁−1)Ψ𝑙
𝑇
• 𝐚𝑅𝑋,𝑙 = 1, 𝑒 −𝑗2𝜋Θ𝑙 , … , 𝑒 −𝑗2𝜋(𝑀−1)Θ𝑙
𝑇
; Ψ𝑙 = 𝑑𝑡 𝑠𝑖𝑛 𝜓𝑙 Τ𝜆
; Θ𝑙 = 𝑑𝑟 𝑠𝑖𝑛 𝜃𝑙 Τ𝜆
Wideband channel matrix in frequency domain
−𝑗2𝜋𝑓𝜏𝑙
• 𝐇 𝑓 = σ𝑙=𝐿
𝐇l ∶ 𝑁𝑥𝑀 𝑚𝑎𝑡𝑟𝑖𝑥 𝑓𝑢𝑛𝑐𝑡𝑖𝑜𝑛
𝑙=1 𝑒
 Narrowband channel matrix:
• all paths have the same time duration 𝜏1 = ⋯ = 𝜏𝐿

• 𝐇 𝑓 = 𝑒 −𝑗2𝜋𝑓𝜏1 σ𝑙 𝐇l = 𝐀𝑅𝑋 𝑫𝐀 𝑇𝑋 𝑯
• 𝐀𝑅𝑋 = 𝐚𝑅𝑋 𝜃1 … 𝐚𝑅𝑋 𝜃𝐿 ; 𝐀 𝑇𝑋 = 𝐚 𝑇𝑋 𝜓1
• 𝐃 = 𝑒 −𝑗2𝜋𝑓𝜏1 𝑑𝑖𝑎𝑔 ℎ 𝜏1 , 𝜓1 , 𝜃1 , … , ℎ 𝜏1 , 𝜓𝐿 , 𝜃𝐿
direction ou services
… 𝐚 𝑇𝑋 𝜓𝐿
Shape Factors For AOA First Order Statistics
Power profile P() is used for explain the 2D directional statistics of received signal
• P(): resulting power from  direction after demodulation and antenna filtering
• Assumes one receiving antenna (M=1)
 The shape factors are based on the complex Fourier coefficients

•

2𝜋
𝐹𝑛 = ‫׬‬0 𝑃(𝜃)𝑒 −𝑗𝑛𝜃 𝑑𝜃
Angular spread ∆=
𝐹1 2
1− 2
𝐹0
; 0 ≤ ∆≤ 1
• decreasing values of  ad indicate that multipath power is becoming more concentrated
about a single direction

Angular Constriction 𝛾 =
𝐹0 𝐹2 −𝐹1 2
𝐹0 2 −𝐹1
; 0≤𝛾≤1
• Increasing values of  indicate that multipath power is becoming more concentrated about
two directions

Azimuthal Direction of maximum fading 𝜃𝑚𝑎𝑥 = 1/2𝑎𝑟𝑔 𝐹0 𝐹2 − 𝐹1 2
.G.D. Durgin and T.S. Rappaport, "Three Parameters for Relating Small-Scale Temporal Fading to Multipath Angles-of-Arrival,"
in PIMRC '99, Osaka Japan, Sep 1999
page 4
direction ou services
Shape factors example

Two-Wave Model:
• 𝑃 𝜃 = 𝑃1 𝛿 𝜃 − 𝜃0 + 𝑃2 𝛿 𝜃 − 𝜃0 − 𝛼
• 𝜃0 is an arbitrary offset angle

𝐹𝑛 = ‫׬‬0 𝑃(𝜃)𝑒 −𝑗𝑛𝜃 𝑑𝜃 = 𝑃1 𝑒 −𝑗𝑛𝜃0 + 𝑃2 𝑒 −𝑗𝑛(𝜃0 +𝛼)
2𝜋
• 𝐹0 = 𝑃1 + 𝑃2 ; 𝐹1 = 𝑃1 𝑒 −𝑗𝜃0 + 𝑃2 𝑒 −𝑗(𝜃0 +𝛼) ;
𝐹2 = 𝑃1 𝑒 −𝑗2𝜃0 + 𝑃2 𝑒 −𝑗2(𝜃0 +𝛼)
 Show that:
• Λ=
2 𝑃1 𝑃2
𝛼
𝑠𝑖𝑛
𝑃1 +𝑃2
2
; 𝛾 = 1 ; 𝜃𝑚𝑎𝑥 = 𝜃0 +
𝛼+𝜋
2
• 𝛼 = 0 results in zero angular spread ∆= 0
• Λ = 1 occurs only when two multipath of
identical powers are separated by 𝜃0 = 𝜋
Graph show the case of P1 = P2
page 5
direction ou services
Narrow band MIMO channel
𝑦1 (𝑡)
𝑦2 (𝑡)
𝑥1 (𝑡)
𝑥2 (𝑡)
Scattering medium
H
𝑥𝑁𝑇 (𝑡)
𝑦𝑁𝑅 (𝑡)
Narrow band MIMO channel
• System model : y=Hx +n
• Fast Fading MIMO Channel: the channel is random
• Slow Fading MIMO Channel: if the channel is kown at TX and RX then it is determinist
• Receive Correlation matrix for a determinist channel: Ry,y =H.Rx,x.HH +Rn,n
• Rx,x is the covariance matrix of x and Rn,n is the covariance matrix of n
 Entropy of Gaussian complex sources : h(x)= Ex(log2(x) )
• For Gaussian sources: h(x) = log2(det(eRx,x))
• h(y) = log2(det(eRy,y))

page 6
direction ou services
Capacity of a narrow band MIMO channel
Mutual information of x and y with white noise and Gaussian source:
• I(x;y) = h(y)-h(y/x)= h(y)-h(n)
 Perfect Channel State Information (CSI) at the receiver : Rx,x is known at RX
 Mutual information with white noise and Gaussian source:
 I(x;y) =h(y)-h(y/x)=h(y)-h(n)= log2[det(H.Rx,x.HH +Rn,n)/ det(Rn,n)]


Case of a white noise n : 𝐑 𝐧,n = diag σ2n1 , … , σ2nN
R
=
1
𝐈
σ2n NR
• 𝐈 𝐱; 𝐲 = h 𝐱 − h 𝐲/𝐱 = h 𝐱 − h 𝐧 = log 2 det 𝐈NR +
1
𝐇𝐑 𝐱,𝐱 𝐇 H
2
σn
• Capacity: maximum of the mutual information
• Two cases:
- Channel unknown at the transmitter side: the mutual information is random
1
σn
– C = max 𝐸 log 2 det 𝐈NR + 2 𝐇𝐑 𝐱,𝐱 𝐇 H
𝐇
- Channel H known at the transmitter and at the receiver
• Optimise Rx,x in order to maximise I(x;y)
• Rx,x found by SVD decomposition of H + waterfilling algorithm
page 7
direction ou services
Capacity of the channel under known CSI (1)
 Problem inputs: H is known from TX and Rx,x is diagonal and has a rank k
• SVD decomposion of H : 𝐇 = 𝐔𝚺𝐕 H = 𝑼𝟏 𝚺𝟏 𝑽𝟏 𝑯
𝑺𝑽𝑫
𝒆𝒄𝒐𝒏𝒐𝒎𝒚 𝒔𝒊𝒛𝒆
• 𝐔1H 𝐔1 = 𝐔1H 𝐔 = 𝐈r and 𝐕1H 𝐕1 = 𝐕1H 𝐕 = 𝐈k where Ir is the (kxk) identity matrix
• 𝚺𝟏 has the diagonal form : 𝚺1 = 𝑑𝑖𝑎𝑔 𝜎1 , … , 𝜎𝑘
෥ = 𝚺𝟏 𝒙
෥+𝒏
෥ with 𝐲෤ = 𝐔𝟏 𝐇 𝐲 ; 𝐱 =
• Equivalent channel model (of y=Hx +n): 𝒚
෥ = 𝐔𝟏 𝐇 𝐧
𝐕𝟏 𝐱෤ ; 𝐧
෤ follows a normal distribution with: 𝐑 𝐱෤ ,෤𝐱 = diag 𝑃1 , … , 𝑃𝑘
 Source model: 𝐱
𝑖=𝑘
 Power constraints: σ𝑖=1 𝑃𝑖 ≤ 𝑃
𝑃𝑖 > 0
 The virtual source equality constraint is the same as the actual antenna power
෤ 𝐇 𝐱෤
equality constraint: σ𝑖=𝑘
=
ณ
𝐸 𝒙𝐇 𝒙
𝑖=1 𝑃𝑖 = 𝑡𝑟 𝐑 𝐱෤ ,෤𝐱 = 𝐸 𝐱
since 𝐕1H 𝐕1 =𝐈r
page 8
direction ou services
Capacity of the channel under known CSI (2)

2
2
Channel capacity: 𝐶 = σ𝑖=𝑘
𝑖=1 𝑙𝑜𝑔2 1 + 𝑃𝑖 𝜎𝑖 /𝜎𝑛
sbt: σ𝑖=𝑘
𝑖=1 𝑃𝑖 = 𝑃 𝑎𝑛𝑑 𝑃𝑖 > 0
•
To find Pi, we resolve the optimisation problem: max 𝑓 𝑠𝑏𝑡: 𝑃𝑖 ≥ 0
𝑃𝑖
2
2
• where: 𝑓 = σ𝑖=𝑘
− λ 𝑃 − σ𝑖=𝑘
𝑖=1 𝑙𝑜𝑔2 1 + 𝑃𝑖 𝜎𝑖 /𝜎𝑛
𝑖=1 𝑃𝑖
• 𝑃𝑖 =
1
𝜎𝑛 2
− 2
λ
𝜎𝑖
+
𝜎𝑛 2
𝑖=𝑘 1
, σ𝑖=1 − 2
λ
𝜎𝑖
+
= 𝑃, where: 𝑥 + = max(𝑥, 0)
• The Lagrange multiplyer  is found using the equality constraint: σ𝑖=𝑘
𝑖=1 𝑃𝑖 = 𝑃
•
1
=
λ
𝜎𝑛 2
𝑃+σ𝑖=𝑘
𝑖=1 𝜎2
𝑖
𝑘
• k is found using the inequality constraint : 𝑎𝑛𝑑 𝑃𝑖 > 0 , 𝑖 = 1, … , 𝑘
 To achieve the channel capacity the following conditions must be satisfied:
• The V1 matrix is used and precoder at the transmitter side
• The U1 matrix is is used as a filter at the receiver side
• The virtual channel powers are found using the water-fillin algorithm
• The source x is normally distribued
 The SVD based solution is an ideal case but is bandwidth consuming in FDD modes
page 9
direction ou services
MIMO capacity example: 4x2 channels
 Given 𝑯 =
𝑠
−𝑑
−𝑑
𝑠
−𝑠
𝑑
𝑑
𝑤ℎ𝑒𝑟𝑒 (𝑠, 𝑑) ∈ ℝ2+ 𝑎𝑛𝑑 𝑠 ≠ 𝑑 and assume 1Hz
−𝑠
bandwidth,
• Calculate is the capacity of this channel in the two following cases
• a) 𝑠 = 2, 𝑑 = 1, 𝜎𝑛 2 = 1W and P = 2 W
• b) 𝑠 = 2, 𝑑 = 1, 𝜎𝑛 2 = 1W and P = 0,1 W
 Answer:
• a) 𝐶 = 𝑙𝑜𝑔2 1 + 22 + 𝑙𝑜𝑔2 1 + 14/9 =C=5,87 bits/s/Hz
𝐿𝑜𝑔 2,8
• b) 𝐶 = 𝑙𝑜𝑔2 1 + 1.8 = 𝐿𝑜𝑔 2 = 1,48 bits/s/Hz
page 10
direction ou services
Zero-Forcing Receiver
 System model:
y =Hx +n
ෝ = 𝑾𝒚 + 𝒛 ; z=Wn
 Linear signal detection applies a weight matrix W on y : 𝒙
 Zero forcing method: W  WZF is the pseudo inverse of H
ෝ = 𝒙 + 𝒛𝑍𝐹 ; 𝒛𝑍𝐹 = 𝑯𝐻 𝑯 −1 𝑯𝐻 𝒏
• 𝑾𝑍𝐹 ≡ 𝑯† = 𝑯𝐻 𝑯 −1 𝑯𝐻 ; 𝒙
• If the the matrix H is invertible then 𝑾𝑍𝐹 = 𝑯−1
• If WZF is othogonal and n is white then then 𝒛𝑍𝐹 is also white: difficult to meet
• If H is ill-conditionned, the ZF receiver is instable  noise enhancement
 Noise power: 𝜎𝒛𝑍𝐹 2 = 𝐸 𝒛𝑍𝐹 2 = 𝐸 𝑇𝑟 𝒛𝑍𝐹 𝒛𝑍𝐹 𝐻
• 𝜎𝑧𝑍𝐹 2 = 𝜎𝑛 2 𝐸 𝑇𝑟 𝑯𝐻 𝑯 −1 𝑯𝐻 𝑯 𝑯𝐻 𝑯 −1
= 𝜎𝑛 2 𝐸 𝑇𝑟 𝑯𝐻 𝑯 −1
2
• 𝑯 = 𝑼𝜦𝑽𝐻  𝑯𝐻 𝑯 −1 = 𝑽𝜦−1 𝑽𝐻 𝑇𝑟 𝑯𝐻 𝑯 −1 = 𝑇𝑟 𝜦−1
•
2
𝜎𝑛 2
𝜎𝑛 2
2
𝜎𝑧𝑍𝐹 = σ𝑖 2 = σ𝑖 𝜎 where 𝜎𝑖 ≡ 𝜆𝑖 2
𝜆
𝑖
𝑖
i=Nt
 Achievable Sum Rate: IZF = σi=1 log 2 1 + γi,ZF
• SINR of the ith stream at the output of the ZF receiver: 𝜸𝑖,𝑍𝐹 =
direction ou services
𝑆𝑁𝑅
𝑯𝐻 𝑯 −1 𝑖,𝑖
Minimum Mean Square (MMSE) receiver
 Usually MMSE receiver is associated with interference cancellation
ෝ = 𝑾𝑀𝑆𝐸 𝒚 that minimises E 𝒙 − 𝒙
ෝ 2
• In MMSE we look for an estimator 𝒙
ෝ 2 = 𝑇𝑟 𝐸 𝒙 − 𝑾𝑀𝑆𝐸 𝒚 𝒙 − 𝑾𝑀𝑆𝐸 𝒚 𝐻
• E 𝒙−𝒙
 The minimum is obtained for: 𝑾𝑀𝑆𝐸 = Σ𝒙 𝑯𝐻 𝑯Σ𝒙 𝑯𝐻 + Σ𝑛 −1
• The MMSE requires the knowledge of the SNR at receiver side
• At low SNR, the MMSE becomes a matched filter: 𝑾𝑀𝑆𝐸 ≈ 𝑆𝑁𝑅. 𝑯𝐻
• At high SNR, MMSE becomes a ZF receiver: 𝑾𝑀𝑆𝐸 ≈ 𝑯𝑯𝐻 −1 𝑯𝐻
 Noise power sum: 𝜎𝒛𝑀𝑀𝑆𝐸 2 = 𝐸 𝒛𝑍𝑀𝑀𝑆𝐸 2 = 𝐸 𝑇𝑟 𝒛𝑀𝑀𝑆𝐸 𝒛𝑀𝑀𝑆𝐸 𝐻
•

𝑖=𝑁𝑡 𝜎𝑛 2 𝑆𝑁𝑅𝟐 𝜎𝑖
2
𝜎𝒛𝑀𝑀𝑆𝐸 = σ𝑖=1 𝑆𝑁𝑅𝜎 +1 2
𝑖
 There’s no noise enhancement with MMSE
Sum rate of MIMO MMSE receiver:
𝑖=𝑁
• 𝐼𝑍𝐹 = σ𝑖=1 𝑡 𝑙𝑜𝑔2 1 + 𝜸𝑖,𝑀𝑀𝑆𝐸
•
Where: 𝜸𝑖,𝑀𝑀𝑆𝐸 =
𝜎𝑛 2 𝑆𝑁𝑅 𝟐 𝜎𝑖 2
𝑆𝑁𝑅𝜎𝑖 2 +1 2
”Achievable Sum Rate of MIMO MMSE Receivers: A General Analytic Framework »
Matthew R. McKay†, Iain B. Collings∗, and Antonia M. Tulino, IEEE Transactions on Information Theory , Vol, 56, Jan 2010
page 12
direction ou services
Performances of the MMSE receiver
Comparison of the spectral efficiency of a MIMO system with optimal and MMSE receivers in i.i.d.
Rayleigh fading channels.Results are shown as a function of received Eb/N0 , for Nr = Nt = 3.
”Achievable Sum Rate of MIMO MMSE Receivers: A General Analytic Framework »
Matthew R. McKay†, Iain B. Collings∗, and Antonia M. Tulino, IEEE Transactions on Information Theory , Vol, 56, Jan 2010
page 13
direction ou services
Schedule of this tutorial
 Part 1: Linear Block Codes
 Part 2: Convolutional encoder
 Part 3: Polar codes
 Part 4: LDPC codes
page 14
direction ou services
Elements of Digital Communication System
page 15
direction ou services
Channel encoder and channel decoder
 A channel encoder protects the bits to be transmitted over a channel by
converting its input into an alternate sequence
• The ratio of the number of input bits to the channel encoder to the number of
output bits is called the code rate; 0 < R < 1.
 The channel decoder recovers from the channel output the input to the
channel encoder
 Coding techniques may be partitioned into automatic request-for-repeat
(ARQ) schemes and forward error- correction (FEC) schemes
• ARQ schemes detect whether or not the received word contains one or more
errors. If errors are detected, a request for retransmission is sent out from the
receiver back to the transmitter. The codes in this case are said to be errordetection codes
• In FEC schemes, the code is endowed with characteristics that permit error
correction. The codes are said to be error-correction codes, or sometimes
error-control codes.
page 16
direction ou services
Linear Block Codes
 In block coding, an information sequence is segmented
into message blocks of fixed length k
• There are 2k distinct messages
 Each length-k block u = (u0, u1, . . ., uk−1) is encoded into
a longer binary sequence v = (v0, v1, . . ., vn−1) of n binary
digits with n > k
 The output sequence v is called the codeword of the
message u
 The set of 2k codewords is said to form an (n,k) block
code
• R = k/n is called the code rate
page 17
direction ou services
Generator matrix
 A codeword v = (v0, . . ., vn−1) for message u = (u0,. . ., uk−1) given by:
𝑔0,0
𝒈𝟎
⋮
⋯ =
• v = u ·G. where: 𝑮 =
𝑔𝑘−1,0
𝒈𝒌−𝟏
⋯
⋱
⋯
𝑔0,𝑛−1
⋮
𝑔𝑘−1,𝑛−1
• The minimum distance, denoted by dmin, is defined as the smallest
distance between two different codewords in C
• Hamming distance between v and w is equal to the Hamming weight of
the vector sum of v and w: d(v,w) = w(v + w)
• The minimum Hamming distance is equal to the minimum weight
• The guaranteed error-detecting ability is e = dmin − 1.
page 18
direction ou services
Hamming sphere
 The Hamming sphere of radius t :
• 𝑆𝑐 = 𝑣; 𝑑 𝑣, 𝑐 ≤ 𝑡 = 𝑣; 𝑑 𝑣 − 𝑐, 0 ≤ 𝑡 = 𝑣 + 𝑐; 𝑣 ∈ 𝑆0
• If all Hamming spheres of radius t are disjoint, then we may correctly decode all
the words with up to t errors.
• If dmin is odd, we may choose t so that dmin = 2t + 1 : decoding spheres.
page 19
direction ou services
Standard array
 Standard array
page 20
direction ou services
Standard array example
 consider the (5, 2) code with
𝟏 𝟎 𝟏 𝟏 𝟏
𝟎 𝟏 𝟏 𝟎 𝟏
 The standard array is
 𝑮=
𝟎 𝟎 𝟎 𝟎 𝟎 𝟏 𝟎 𝟏 𝟏 𝟏 𝟎 𝟏 𝟏 𝟎 𝟏 𝟏 𝟏 𝟎 𝟏 𝟎
 𝟎 𝟎 𝟎 𝟎 𝟏
𝟏 𝟎 𝟏 𝟏 𝟎 𝟎 𝟏 𝟏 𝟎 𝟎 𝟏 𝟏 𝟎 𝟏 𝟏
 𝟎 𝟎 𝟎 𝟏 𝟎
𝟏 𝟎 𝟏 𝟎 𝟏 𝟎 𝟏 𝟏 𝟏 𝟏 𝟏 𝟏 𝟎 𝟎 𝟎
 𝟎 𝟎 𝟏 𝟎 𝟎
𝟏 𝟎 𝟎 𝟏 𝟏 𝟎 𝟏 𝟎 𝟎 𝟏 𝟏 𝟏 𝟏 𝟏 𝟎
 𝟎 𝟏 𝟎 𝟎 𝟎
𝟏 𝟏 𝟏 𝟏 𝟏 𝟎 𝟎 𝟏 𝟎 𝟏 𝟏 𝟎 𝟎 𝟏 𝟎
 𝟏 𝟎 𝟎 𝟎 𝟎
𝟎 𝟎 𝟏 𝟏 𝟏 𝟏 𝟏 𝟏 𝟎 𝟏 𝟎 𝟏 𝟎 𝟏 𝟎
 ------------------------------------------------------------------------------------------------ 𝟎 𝟎 𝟎 𝟏 𝟏
𝟏 𝟎 𝟏 𝟎 𝟎 𝟎 𝟏 𝟏 𝟏 𝟎 𝟏 𝟏 𝟎 𝟎 𝟏
 𝟎 𝟎 𝟏 𝟏 𝟎
𝟏 𝟎 𝟎 𝟎 𝟏 𝟎 𝟏 𝟎 𝟏 𝟏 𝟏 𝟏 𝟏 𝟎 𝟎

page 21
direction ou services
Linear systematic block code
 Systematic codeword
message part
redundunt check part
 Generator matrix of a systematic block code:
• 𝑮 = 𝑰𝒌
𝟏 ⋯ 𝟎 𝒑0,0
⋮
𝑷 = ⋮ ⋱ ⋮
𝟎 ⋯ 𝟏 𝒑𝑘−1,0
⋯
𝒑0,𝑛−𝒌−1
⋱
⋮
⋯ 𝒑𝑘−1,𝑛−𝑘−1
 Parity-Check Matrix:
 The null (or dual) space, denoted Cd, is a binary (n,n − k) linear
block code and is called the dual code of C.
• Let Bd be a basis of Cd and h0, h1, . . ., hn−k−1 be the n − k linearly
independent codewords in Bd then G·HT = O, where O is a k × (n − k)
zero matrix.
• A binary n-tuple c ∈ C is a codeword in C if and only if c ·HT = 0
• For a systematic code: 𝑯 = 𝑷𝑇 𝑰𝑛−𝒌
page 22
direction ou services
Example
 Consider the generator matrix of the (7,4) Hamming code with
1 0 0 0 1 1 0
0 1 0 0 0 1 1
 𝑮=
= 𝑰𝟒 𝑷
0 0 1 0 1 1 1
0 0 0 1 1 0 1
 Find the code vector for the message vector u= (1110), and check the validity of
generated word
1 0 1 1 1 0 0
𝑇
 Solution: c=u.G=(1110010) ; 𝑯 = 𝑷
𝑰𝟑 = 1 1 1 0 0 1 0
0 1 1 1 0 0 1
 𝒄𝑯𝑻 =
page 23
𝑻
𝑯𝒄𝑻 ; 𝑯𝒄𝑻 =
1
1
0
direction ou services
0 1 1
1 1 0
1 1 1
1 0
0 1
0 0
0
0
1
1
1
1
0
0
1
0
1+1
0
= 1+1+1+1 = 0
1+1
0
Hamming Codes
 Hamming code is a linear block code capable of correcting single errors
having a minimum distance dmin = 3
 H is chosen so that no row in HT is zero and the first n-p rows of HT form an
identity matrix and all the rows are distinct
• The following inequality should be satisfied: 2n-k -1≥n
• Hence, if k is known, the minimum size n for the code is uniquely determined
page 24
direction ou services
Some bounds
Hamming weight of v, denoted w(v), is defined as the number of nonzero components in
v.
 The smallest weight of the nonzero codewords in C, denoted wmin(C), is called the
minimum weight of C
• Singleton Bound: dmin(C) ≤n-k+ 1
• No dmin(C) − 1 or fewer errors can change a transmitted codeword into another
codeword in C. Therefore, all the error patterns with dmin(C) − 1 or fewer errors are
detectable by the channel decoder.


Probability of an undetected error:
• Hamming Bound: 𝑃𝑢 (𝐸) ≤ 2−(𝑛−𝑘)
• A code that satisfies this upper bound is said to be a good error-detection code.
page 25
direction ou services
Good error-detection code
 Hamming weight of v, denoted w(v), is defined as the number of nonzero
components in v.
 let Ai be the number of codewords in C with Hamming weight i
• the numbers A0,A1, . . .,An are called the weight distribution of C
• A0 = 1and A0+A1+ . . .+An =2k
• The smallest weight of the nonzero codewords in C, denoted wmin(C), is
called the minimum weight of C
 If C is used over a BSC with transition probability p
• There are Ai undetectable error patterns of weight i; each occurs with
probability pi(1 − p)n−i.
𝑖=𝑛
• Probability of an undetected error: 𝑃𝑢 (𝐸) = ෍ 𝐴𝑖 𝑝𝑖 (1 − 𝑝)𝑛−𝑖
𝑖=1
−(n−k )
• Hamming Bound: Pu ( E )  2
• A code that satisfies this upper bound is said to be a good error-detection
code.
page 26
direction ou services
Syndrome Table Decoding
The received vector r can be written as: r = c + e
 The syndrome is obtained by s = rHT = (c + e)HT = eHT
 Decoding is performed by computing the syndrome of the received vector, looking
up the corresponding error pattern, and subtracting the error pattern from the
received word.
 Example: Syndrome decoding table for the (7,4) Hamming code

page 27
Error pattern
Syndrome
0000000
000
1000000
100
0100000
010
0010000
001
0001000
110
0000100
011
0000010
111
0000001
101
direction ou services
Hamming Codes Decoding
 Let H= {h0; h1; . . .; hn} where hj is the jth column of H
 Calculating the syndrome corresponding to a given word pattern we
obtain
 h0T 
 T 
h1 
T

s j = e j H = ( 0,.., 0,1, 0,.., 0 )
= hTj




T
 hn −1 
Decoding algorithm:
1. Compute the syndrome s for the received word. If s = 0, the received code word is
correct .
2. Find the position j of the column of H that is the transposition of the syndrome.
3. Complement the jth bit in the received code word to obtain the corrected code
word.
page 28
direction ou services
Hamming Codes Decoding example
 Decode the received vector r = (010000000000000) using the (15,11) parity
check matrix.
1
0
H=
0

0
0 0 0 0 0 0 0 1 1 1 1 1 1 1
1 0 0 1 1 1 0 0 0 0 1 1 1 1

0 1 0 0 1 1 1 0 1 1 0 0 1 1

0 0 1 1 0 1 1 1 0 1 0 1 0 1
 Solution:
 j=2
The corrected received word is: c = (000000000000000)
 s = rHT =ejHT= hjT =(0100)T

page 29
direction ou services
Cyclic Codes
 A linear code C is said to be cyclic if the cyclic-shift of each codeword in C is
also a codeword in C: for all (c0, . . . , cn-1) ∈ C (cn, c0, . . . , cn−2) ∈ C.
 A codeword c = (c0, c1, . . ., cn−1) is represented by a polynomial over GF(2)
• c(X) = c0 + c1x + · · · + cn−1 xn−1.
• a nonzero code polynomial has degree at least n−k but not greater than n−1.
• There exists one and only one code polynomial of the following form:
• g(X) = 1+g1X + g2X2 + · · · + gn−k−1 Xn−k−1 + Xn−k
• The degree of g(X) is the number of parity-check bits of the code.
• Every code polynomial v(X) in C is divisible by g(X) and every polynomial of
degree n−1 or less that is divisible by g(X) is in C: v(X) = m(X)g(X),
• g(X): generator polynomial of the (n,k) cyclic code C
• If m = (m0,m1, . . .,mk−1) is the message to be encoded, then m(X) is the
message polynomial
page 30
direction ou services
Systematic representation of cyclic codes
 Let m = (m0,m1, . . .,mk−1) is the message to be encoded
 The encoding is achieved with a division circuit that divides the message
polynomial Xn−km(X) by its generator polynomial g(X) and takes the
remainder as the parity part of the codeword.
 The division circuit can be implemented with an (n − k)-stage shift-register
with feedback connections
1
g1
b1
g2
b2
2
gn-k-1
b3
…
bn-k-1
m0,. . ., mk−1
m k-1, ..., m 0 b n-k-1, ..., b 0
1
page 31
direction ou services
2
1
2
Syndrome Calculator of cyclic codes
 Received bits are fed from left into the (n − k) stages of the
feedback shift register
 The contents of the shift register contain the desired syndrome s
page 32
direction ou services
cyclic codes example (1)


Construct the shift register encoder for a cyclic code of length 7 generated by g(x)=x4 + x3 + x2 + 1, and
obtain the code word for message m =(010)
Solution:
• Contents of the shift register
Shift
Input
0
Register code words
0
0
0
0
1
0
0
0
0
0
2
1
1
0
1
0
3
0
0
1
1
1
1
1
0
2
0
1
message: 010
1
0101110
1
page 33
direction ou services
2
2
cyclic codes example (2)
First shift
0
0
0
0
message: 010
Second shift
1
0
1
0
message: 001
Third shift
0
1
1
1
message: 000
page 34
direction ou services
syndrome-computation for cyclic code
received code word: r(X)=m(X)g(X)+s(X)
• s(X) is a polynomial of degree n - k - 1 or less,
• r(X) = v(X) + e(X): an error in the received word is detected only when the syndrome
polynomial s(X) is nonzero
 Example: Hamming code with g(x)= x3 + x + 1 and a transmitted code word 1100101
with error at the fifth bit location
Shift
Input bits
Content of the

registers
v = (11000101)
0
x
000
r = (11100101)
1
1
100
2
0
010
3
1
101
4
0
100
5
1
110
6
1
111
7
1
001
1
received
bits
0
page 35
0
direction ou services
1
2
Cyclic redundancy check (CRC) code
 Exercice:
• Are the CRC codes cyclic?
• From the standard, give a descrition of the error correcting
capabilities of the following codes:
• CRC-12, CRC-16 and CRC-CCITT
page 36
direction ou services
Convolutional Codes
 A convolutional encoder processes incoming information bit stream
continuously, in a stream-oriented fashion
 History:
• Elias (1955): Introduction of the convolutional codes
• Wozencraft (1961): Sequential decoding
• Massey (1963): Majority logic decoding
• Viterbi (1967): ML decoding
• BCJR(1975): MAP decoding
• Berrou et al. (1993): Turbo codes
ELIAS P., “Error-free coding”, IEEE Transactions on Information Theory, p. 29–37,
September 1954.
Andrew J. Viterbi, "Error Bounds for Convolutional Codes and an Asymptotically Optimum Decoding Algorithm," IEEE
Transactions on Information Theory, Volume IT-13, pp. 260-269, April 1967.
page 37
direction ou services
Convolutional code representation

(𝑡)
(𝑡)
At every time t the encoder receives a block of k binary symbols 𝑢(𝑡) = 𝑢1 , . . , 𝑢𝑘
(𝑡)
(𝑡)
delivers a block of n binary symbols 𝑛(𝑡) = 𝑢1 , . . , 𝑢𝑛
and
using a set of shift registers
• Rate: R= k/n
• Constraint length m: the maximum number of bits in a single output stream that can be
affected by any input bit
• (k,n,m): representation of the convolutional code
page 38
direction ou services
Convolutional code example 1
Shift register presentation
Cell presentation

 Consider a (3,1,3) convolutional code

u=(11011….)
(𝟏) (𝟏) (𝟏) (𝟐) (𝟐) (𝟐)
 𝒙=
𝒙𝟏 𝒙𝟐 𝒙𝟑 𝒙𝟏 𝒙𝟐 𝒙𝟑 … .
 𝑮𝟏 =
𝟏𝟏𝟏 ; 𝑮𝟐 = 𝟎𝟏𝟏 ; 𝑮𝟑 = 𝟎𝟎𝟏
 𝒈𝟏 𝑿 = 𝟏 + 𝑿 + 𝑿𝟐 ; 𝒈𝟐 𝑿 = 𝟏 + 𝑿;
𝒈𝟑 𝑿 = 𝟏
 𝒙 = 𝐮𝑮∞ = 𝟏𝟏𝟏𝟏𝟎𝟎𝟎𝟏𝟎𝟏𝟏𝟎𝟏𝟎𝟎 …
page 39
direction ou services
111
𝐺∞ = 000
000
⋯
011
111
000
⋯
001 000
011 001
111 011
⋯ ⋯
000 000 ⋯
000 000 ⋯
001 000 ⋯
⋯ ⋯ ⋯
State Diagram of Convolutional Codes
A convolutional encoder can be considered as finite-state machine
 An encoder with n registers provides any one of 2n possible states
 There are 2𝑘 input edges are entering to each state and 2𝑘 edges leaving it.
 Each edge in the state diagram has a label of the form u/xxx,, where u is the input bit that
causes the state transition and xxx. . . is the corresponding output bits.
 Example: convolutional encoder with G(D)=[1+D 1+D2 1+D+D2 ]

0/000
S0 (00)
x(1) = u g(1) where g(1) =(1,1,0)
0/011
u
S0
1/111
0/101
x(2) = u g(2) where g(2) =(1,0,1)
S2
S1
1/100
x(3) = u g(3) where g(3) =(1,1,1)
1/010
0/110
S1 (10)
S2 (01)
S3
S3 (11)
1/001
page 40
direction ou services
Trellis Diagram
The trellis diagram is an expansion of state diagram by adding a time axis for time
information
• States are represented by nodes and are arranged vertically
• The horizontal axis represents time
 Example: g(1) (D) =1+D+D2 ; g(2) (D) =1+D2

0/00
1/11
S0=00
S1=10
S2=01
S3=11
0/00
0/00
0/00
0/00
1/11
1/11
0/11
1/10
0/10
S0
0/11
S0
0/10
S1
S2
1/10
1/01
1/11
S1
1/11
S2
0/10
0/11
1/10
0/10
1/01
0/11
0/11
S3
S3
t=0
t=1
t=2
1/10
1/01
0/11
t=3
1/10
The circuit
page 41
The state diagram
direction ou services
The trellis diagram
1/10
1/01
t=4
LTE Convolutional Encoder
 has constraint length 7 and is tail biting with coding rate 1/3 and octal
polynomials G0=133, G1=171 and G2=165.
Ref: 3GPP TS 36.212. "Multiplexing and channel coding." 3rd Generation Partnership Project;
page 42
direction ou services
Viterbi Decoding Algorithm
The Viterbi algorithm is a maximum likelihood decoding algorithm
• finds the path with largest metric through the trellis by comparing the metrics of all branch
paths entering each state with the corresponding received vector r iteratively
 Some definitions
• t (s’, s) = branch metric from state s’ at time t − 1 to state s at time t
• t−1(s’) = cumulative metric for the survivor state s’ at time t − 1; the sum of branch metrics
for the surviving path.
• t−1(s’,s)= tentative cumulative metric for the paths extending from state s’ at time t − 1 to
state s at time t; t−1(s’,s) = t−1(s’) + t (s’, s)
𝑡=𝑇
 Cumulative metric :  𝑇 = σ𝑡=1 𝑡


𝑗=𝑛
(𝑗)
(𝑗)
Branch metric for the BSC :𝑡 = σ𝑗=1 𝑑𝐻 𝑦𝑡 , 𝑥𝑡
𝑗=𝑛 2
 Branch metric for the BI−AWGNC :𝑡 = σ𝑗=1 𝑑𝐸
(𝑗) (𝑗)
𝑦𝑡 , 𝑥𝑡
𝑗=𝑛
direction ou services
(𝑗)
⊕ 𝑥𝑡
𝑗=𝑛
= σ𝑗=1
(𝑗) (𝑗)
• can be replaced by the correlation metric: 𝑡 ≡ σ𝑗=1 𝑦𝑡 𝑥𝑡
page 43
(𝑗)
𝑗=𝑛
= σ𝑗=1 𝑦𝑡
(𝑗)
(𝑗) 2
𝑦𝑡 − 𝑥𝑡
Add–Compare–Select Iteration
 Initialize. Set 0(S0 ) = 0 and 0(s’ ) = − for all s’  S0 .
 for t= 1 to T
1. Compute the possible branch metrics t (s’, s)
2. For each state s’ at time t − 1 and all possible states s at time t that
may be reached from s’, compute the tentative cumulative metrics
t−1(s’,s) = t−1(s’) + t (s’, s) for the paths extending from state s’ to
state s.
3. For each state s at time t, select and store the path possessing the
minimum among the metrics t−1(s’,s) . The cumulative metric for
state s will be t(s) = mins’ {t−1(s’,s) }
4. end
 Decision: Choose the trellis path with the best cumulative metric
• ties are decided arbitrarily
page 44
direction ou services
Viterbi decoding example 1
with g(1) (D) =1+D+D2 ;
g(2) (D) =1+D2 on the BSC; u = [1, 0, 0, 0]; x = [11, 10, 11, 00].
 y = [11, 11, 11, 00].
 Viterbi decoding example for the (1,2) code
Received :
11
S0
0/00
S1
1/11
11
2
0
11
0/00
1/11
4
2
0/10
S2
00
0/00
1/11
0/11
1/10
0/00
6
1
4
3
0/10
3
1
2
1/11 0/11
3
1/10 2
0/10
1
1/01
S3
1
t=0
t=1
t=2
0/11
1/01
3
0/11
2
1/10
t=3
1
1/10
4
4
1/01
4
3
t=4
Non-surviving paths are indicated by dashed segments and cumulative metrics are written in bold red near
merging branches. The ML path is the good one
page 45
direction ou services
Viterbi decoding example 2
with g(1) (D) =1+D+D2 ;
g(2) (D) =1+D2 on the BSC; u = [1, 0, 0, 0]; x = [11, 10, 11, 00].
 y = [00, 01, 00, 01].
 Viterbi decoding example for the (1,2) code
Received :
00
S0
0/00
S1
1/11
01
0
2
00
0/00
1/11
1
1
01
0/00
1/11
0/11
0/00
1
6
3
1/11 0/11
3
0/10
S2
4
2
3
1/01
S3
2
t=0
t=1
t=2
0/11
t=3
3
4
1/01
5
2
3
1/10
0/11
1/01
2
1/10 3
0/10
4
1/10
0/10
2
1/10
2t=4
Non-surviving paths are indicated by dashed segments and cumulative metrics are written in bold red near
merging branches. Any of the paths of distance 2 will do as the ML path
page 46
direction ou services
Viterbi decoding example 3
with g(1) (D) =1+D+D2 ; g(2) (D)
=1+D2 on the BI-AWGNC; u = [1, 0, 0, 0]; x = [11, 10, 11, 00].
 y = [−0.7 − 0.5, − 0.8 − 0.6, − 1.1 +0.4, + 0.9 +0.8].
 Viterbi decoding example for the (1,2) code
Received :
00
S0
01
00
3.8
-1.2
-2.6
2.1
1.2
0.2
0.7
3.4
S2
1.4
1.7
2.6
S3
1.0
2.5
2.4
S1
t=0
t=1
t=2
The cumulative metric of the ML path is 3.8.
page 47
01
direction ou services
1/-+
t=3
1/-+
t=4
Tail-Biting Convolutional Code
 Avoid the rate loss due to additional terminating bits
 Codewords can start in any state but each codeword
must end in the same state initial state
 Since both initial and terminating states are not known,
decoding is achieved by training examining each
initiating state apart
 For a (n; k; m) encoder, the m last input bits are inserted
at the beginning of the message in order to produce the
last state
• These L bits are discarded leaving us with the same input
message but with an encoder internal initial state that will
repeat itself at the end
page 48
direction ou services
Example: a tail-biting encoder matlab code
clear all;clc;
input_msg = randi([0 1],10,1);
g1= str2num(dec2base(bin2dec('1111001'),8))
g2= str2num(dec2base(bin2dec('1011011'),8))
trellStr = poly2trellis(7, [ g1 g2 ]);%802.16e
Standard
k = log2(trellStr.numInputSymbols);
m =log2(trellStr.numStates);
%% Collect tail bits
tail_bits = input_msg(end + 1 - k*m:end);
%% Obtain state bits
hEnc1 = comm.ConvolutionalEncoder;
hEnc1.TrellisStructure = trellStr;
hEnc1.TerminationMethod = 'Truncated';
hEnc1.FinalStateOutputPort = true;
[states, fin_state] = step(hEnc1, tail_bits);
%% Define main encoder
hEnc = comm.ConvolutionalEncoder;
hEnc.TrellisStructure = trellStr;
hEnc.TerminationMethod = 'Truncated';
hEnc.InitialStateInputPort = true;
coded_data = step(hEnc, input_msg, fin_state);
page 49
direction ou services

802.16e standard tail-biting convolutional
encoder.

input_msg = 0 01 1 1 0 1 1 01
k=1, m=6
tail_bits =101101
coded_data =10 10 10 10 11 01 11 11 11 01



K. Deergha Rao
Channel Coding Techniques for Wireless Communications
Concatenated codes with interleaver
(Turbo codes)
 Invented by Berrou et al. (1993)
 Showed performance close by 0.5 dB to the Shannon
capacity limit, at a bit error probability of 10-5
 Turbo code: two constituent convolution codes and one
interleaver connected in parallel (PCBC)
Information bits
x1
Encoder 1
Interleaver 
Parity bits 1
Modulator
x2
Parity bits 2
Encoder 2
x3
Ref: From BCJR to turbo decoding: MAP algorithms made easier © Silvio A. Abrantes* April 2004
page 50
direction ou services
Turbo decoder
 log-likelihood ratios (LLRs)
• 𝑳1 𝑢𝑘 = 𝒍𝒐𝒈
𝑷 𝑢𝑘 =0Τ𝒚(𝟏,𝟐)
𝑷 𝑢𝑘 =1Τ𝑦 (1,2)
=
𝑳𝒄𝒉 𝑢𝑘 +𝑳𝑒,1 𝒖 + 𝑳𝒊𝒏 𝒖
(𝟏,𝟐)
• 𝑳𝒄𝒉 𝑢𝑘 = 𝒍𝒐𝒈
• 𝑳𝒆,𝟏 𝑢𝑘 = 𝒍𝒐𝒈
• 𝑳1 𝑢𝑘 = 𝒍𝒐𝒈
𝒑 𝒚𝒌
ൗ𝑢𝑘 =0
(𝟏,𝟐)
𝒑 𝒚𝒌 ൗ𝑢𝑘 =1
(𝟏,𝟐)
𝒑 𝒚\k ൗ𝑢𝑘 =0
(𝟏,𝟐)
𝒑 𝒚\k ൗ𝑢𝑘 =1
𝑷 𝑢𝑘 =0
𝑷 𝑢𝑘 =1
 𝑳𝒆,𝟏 𝒖
:extrinsic information sent by
decoder 1 to decoder 2
 SISO: Soft Input Soft Output
• Example: BCJR algorithm
page 51
direction ou services
y(1)=x(1)+ n(1)
𝐮 = 𝑢1 , 𝑢2 , … , 𝑢𝐾
𝑳1 𝑢𝑘
SISO 1
y(2)=x(2)+ n(2)
𝑳𝟏𝟐 𝐮


𝑳𝟏𝟐 Π𝐮
y3=x(3)+ n(3)
SISO 2
𝑳𝟐𝟏 𝐮
1
𝑳𝟐𝟏 Π𝐮
𝑢ො 𝑘
Performance of the Turbo decoder

Ref: C. Berrou, A. Glavieux and P. Thitimajshima, “Near Shannon limit error-correcting coding and decoding:
Turbo codes”, Proc. Intern. Conf. Communications (ICC), Geneva, Switzerland, pp. 1064–1070, May 1993.
page 52
direction ou services