MSc Data Communications
By R. A. Carrasco
Professor in Mobile Communications
School of Electrical, Electronic and Computing Engineering
University of Newcastle-upon-Tyne
2006
Recommended Text Books
1. “Essentials of Error-Control Coding”,
Jorge Costineira, Patrick Guy Farrell
2. “Digital Communications”, John G.
Proakis, Fourth Edition
Goals Of A Digital
Communication System
• Deliver/Data from the source to the user in a:
•
FAST
•
INEXPENSIVE (Efficient)
•
RELIABLE WAY
Digital Modulation Schemes
Task: to compare different modulation schemes with different values of M
•
Choice of modulation scheme involve trading off
• Bandwidth
• Power
• Complexity
•
Define:
• Bandwidth
• Signal-to-noise ration
• Error probability
Examples: Memoryless Modulation (Waveforms are
chosen independently – each waveform depends only
on mi)
Source Symbols
0
Ts
0
1
1
0
1
0
1
1
0
A
t
a
-A
M=2
T=Ts
T
01
01
01
S1(t) = A, 0t<T
S2(t) = -A
10
01
b
t
Sinusoids with
4 different phases
M=4
T=2Ts
010
011
c
t
T
M=8
T = 3Ts
101
011001
d
010101
T
8 different
amplitude
levels
A crucial question is raised
what is the difference?
• If T is kept constant, the waveforms of scheme C requires less
bandwidth than those of 2, because the pulse duration is longer
• In the presence of noise, and if the same average signal power is
used, it is more difficult to distinguish among the waveforms of c.
AM/AM = Amplitude Modulation – Amplitude Modulation conversion
AM/PM = Amplitude Modulation – Phase Modulation conversion
Notice:
• Waveforms b have constant envelopes.
• This choice is good for nonlinear radio channels
Output
Envelope
A
B
Input
Envelope
A: Output envelop (AM/AM conversion)
B: Output phase shift (AM/PM conversion)
TRADE-OFF BETWEEN
BANDWIDTH AND POWER
•
In a Power-Limited Environment, use low values of M
•
In a Band-Limited Environment, use high values of M
What if both Bandwidth and Power are Limited ?
•
•
Expand Complexity
DEFINE:
BANDWIDTH
SIGNAL-TO-NOISE RATIO
ERROR PROBABILITY
Performance of Different Modulation
Schemes
DIGITAL MODULATION TRADEOFFS
SHANNON CAPACITY LIMIT FOR AWGN
C = W LOG (1 + S/N)
• S = Signal Power = e/T
• N = Noise Power = 2NoW
• W = Bandwidth
Define Bandwidth W
dB S(f)
Different bandwidth definitions of the power density spectrum
of (5.2). B1 is the half-power bandwidth: B2 is the equivalent
noise bandwidth: B3 is the null-to-null bandwidth: B4 is the
fractional power containment bandwidth at an arbitrary level:
B5 is the bounded power spectral density at a level of about
18dB. Notice that the depicted bandwidths are those around
the frequency f0.
0
-10
-20
-3
-2
-1
0
B1
B2
In general, W = /T Hz,
1
2
3
fT
• Half – power
•Equivalent – noise
B3
•Null – to – null
B4
•Fractional power containment
B5
•Bounded power spectral density
 depends on modulation scheme and on bandwidth definition
DEFINE SNR
log 2 M bits
Rs 
T
sec
M # of signals
1
 signalling rate
T
Rate at which the source outputs binary symbols
• Average Signal Power
• Signal-to-Noise Ratio

T
 log 2 M
 b
T
P
Average signal energy
b = average energy per bit
 b Rs
P
SNR 

N 0W N 0 W
Noise power spectral
density.
Bits/sec Hz
(Bandwidth
BPS/HZ Comparison
Digital Modulation Trade – Offs
(Comparison among different schemes)
Rs
w
16
SHANNON CAPACITY BOUND
4
2
LIMITED REGION
2
11
2
4
8
POWER
LIMITED REGION
0.125
4
6
8
16
64
2
2
4
16
32
0.25
0
Pb (e)  10 5
x
2
0.5
16
8
x
4
x
16
x
8
4
2
BANDWIDTH
16
8
4
8
-2
32
16
8
32
8
10
12
14
16
PAM (SSB)
(COHERENT) PSK
AM-PM
x
DCPSK
COHERENT FSK
INCOHERENT FSK
18
20
(dB)
b
N0
k=1
u1
Encoder for the (3, 1)
repetition code
x1= u1, x2=u1, x3=u3
x1
x2
x3
0
1
n=3
000
111
k=2
u2
x3
The whole word is defined by
u1
x2
n=3
x1
00
000
x1=u1, x2=u2
01
011
x3=u1+u2
10
101
11
110
Encoder for the (3,2) parity-check code
[1], pages 157 - 179
Hamming Code (7,4)
0000
0000 000
0001
0010
0001 011
0010 110
0011
0100
0011 101
0100 111
0101
0110
0101 100
0110 001
0111
1000
0111 010
1000 101
1001
1010
1001 110
1010 011
Block Encoders
1011
1100
1011 000
1100 010
Notice that only 16 of 128 sequences of length 7 are
used for transmission
1101
1110
1101 001
1110 100
1111
1111 111
u4
u3
u2
The codeword is defined by
u1
xi = ui, i = 1,2,3,4
x5 = u 1 + u 2 + u 3
x6 = u 2 + u 3 + u 4
x7 = u 1 + u 2 + u 4
Source
x7
x6
x5
x4
x3
x2
x1
(7, 4) Hamming Code
symbols
encoded
symbols
Convolutionally encoding the
sequence 101000...
Time
t1
Encoder
1 0 0
Output
x1 x2
x1
1
1
x2
t2
0 1 0
x1
1
1
Rate , K = 3
2
0
x2
t2
1 0 1
x1
x2
0
0
From B Sklar,
Digital Communiations.
Prentice-Hall, 1988
Convolutionally encoding the
sequence 101000...
t4
0
1 0
t5
0
1 0
t6
0
0 0
x1
x2
x1
x2
x1
x2
1
0
1
1
0
0
Output sequence: 11 10 00 10 11
(7, 5) Convolutional Encoder
c1
1
c1  d1  d 2  d 3
1
d3
d3
c2  d1  d 3
d1
d2
Input
d2
Data
d1
Constraint length K = 3
Code Rate = 1/2
Output
2
d1
d3
c2
Time
Interval
Input
Output
SW
Position
1
2
3
4
5
6
7
8
0
00
12
1
11
12
1
01
12
0
01
12
1
00
12
0
10
12
0
11
12
1
11
12
The Finite State Machine
a=00
1/10
Example message:
b=01
c=10
11
d=11
1/01
b
d
1/00
0/01
01
10
Input
0
1
1
0
1
0
0
1
0
1
Output
00
11
01
01
00
10
11
11
10
00
c
0/10
1/11
a
0/11
output bit
00
0/00
The 0 or 1
input bit
The coder in state a= 00
A 1 appearing at the input produces 11 at the output, the system moves to state b = 01
• If in state b, a 1 at the input produces
01 as the output bits. The system
then moves to state d (11).
• If a 0 appears at the input while the
system is in state b, the bit sequence
10 will appear at the output, and the
system will move to state c (10).
Tree Representation
00
00
a
11
b
00
10
11
00
01
11
10
00
11
01
0
01
10
00
11
1
11
10
Input data bits
10
00
01
11
11
01
00
01
01
10
10
1
2
3
c
d
a
b
c
Time
d
a
K=3
b
c
Rate=
d
a
b
c
d
4
Upward transition
0 bit
Downward transition
1 bit
1
2
Signal-flow Graph
1
D
11
D2
D
d
D
Xa
a
Xb
D2
D
Xd
D
b
Xc
c
D0 = 1
X a'  D 2 X c
X c  DX d  DX b
X d  DX d  DX b
X b  D2 X a  X c
D2
Xa’
a
Transfer Function T(D)
From equation 3
X d (1  D)  DX b
Therefore,
D
Xb
1 D
From equation 2
2 D 
D

Xb
2
D Xc
1 D 
T ( D) 

1  2D
Xa
Xb
2
D (1  D)
Xd 
D2
D2
D(1  D)
Xc 
X b  DX b 
Xb 
Xb
1 D
1 D
(1  D)
D
Xc 
Xb
1 D
From equation 1
D
1 D
D
D2 X a  X b  X c  X b 
Xb 
Xb 
Xb
1 D
1 D
1 D
1  2D
D 2X a 
Xb
1 D
1  2D
1  2D
Xa  2
Xb  2
Xb
D (1  D)
D  D3
 D3 
 D5 




1 D 
1 D 
D5







 1  2D   1  2D  1  2D

 

 D 2 (1  D)   1  D 


Transfer Function T(D)
Performing long division gives:
D 5  2 D 6  4 D 7  8D 8  
1  2D
D5
This gives the number of paths in the state
diagram with their corresponding
distances.
D5
 
D5  2
D 6  4
D 7  8
D8  
1  2 D 1 path
2 paths
4 paths
8 paths
D 5  2D 6
distance5
distance6
distance7
distance8
2D 6
2D 6  4D 7
In this case, the minimum distance of the
code is 5
4D 7
4 D 7  8D 8
8D 8
Block Encoders
by Professor R.A Carrasco
STATE
U
Ui
Ui-1
000
Ui-2
S1
00
111
001
S3
S2
10
01
100
X
110
011
s1=(00)
s2=(01)
s3=(10)
s4=(11)
010
S4
11
0
101
1
Source
Symbol
“State Diagram” of Code
u= (11011.....) corresponds to the paths s1 s3 s4 s2 s3 s4 through the state diagram and the output sequence is x=(111100010110100)
Tree Diagram
000
000
000
The path corresponding
to the input sequence
11011 is shown as an
example.
S1
S3
S1
S2
S3
110
010
100
0
S4
S1
101
000
1
001
011
S1
S2
S3
S3
100
001
010
100
111
011
110
111
100
001
011
111
111
011
111
000
S1
S2
S4
011
110 S3
010
101
S4
101
100
S4
Signal Flow Graph
Xa = S1
Xb = S3
Xc = S4
Xd = S2
D2
Xc
D
D
Xa
D3
Xb
D2
D2
Xd
D
Xa’
1) X b  D 3 X a  D 2 X d
2) X c  D 2 X c  DX b
3) X d  DX c  D 2 X b
Transfer Function T(D)
T ( D) 
X a ' DX d

Xa
Xa
X a
D
Xc 
Xb
1 D
D2
2D 2  D 4
2
Xd 
Xb  D Xb 
Xb
2
2
1 D
1 D
1 D2
Xb 
Xd
2
4
2D  D
From equation 1
Xb  D Xa  D Xd 
3
2
1 D2
2D  D
1  D 2  2 D 4  D 6 
3
Xd 

D
Xa

2
4
2D  D


2
4
Xd
1  D 2  2D 4  D 6

D 2D  D
DX a
3
2
4

Xb
2
4
6
1

D

2
D

D
T ( D) 
Xa
3
2
4
D 2D  D




D.D 3 2 D 2  D 4

1  D 2  2D 4  D 6
2D 6  D8
1  D 2  2D 4  D 6

Transfer Function of the State
Diagram
2 D 6  D 8  5 D10  5 D12  ...
1  D 2  2D 4  D 6 2D 6  D8
2 D 6  2 D 8  4 D10  2 D12
We have dfree = 6,
for error events:
D 8  4 D10  2 D12
D 8  D10  2 D12  D14
S1 to S3 to S2 to S1
and
S1 to S3 to S4 to S2 to S1
5 D10  0 D12  D14
5 D10  5 D12  10 D14  5D16
5D12  9 D14  5 D16
2D 6  D8
1  D 2  2D 4  D 6
 2
D6  
D 8  5
D10  5
D12  
2 paths
distance6
1 path
distance8
5 paths
distance10
5 paths
distance12
Trellis Diagram for Code (Periodic
from time 2 on)
00
000
000
000
000
000
000
111
111
111
111
111
111
001
001
001
001
States
10
110
011
01
110
011
100
110
011
110
011
Legend
Input 0
011
100
100
100
100
010
010
010
010
101
101
101
101
Input 1
11
0
1
2
3
4
5
6
Time
The minimum distance of the convolutional code l = N
dmin = dc (N), the column distance.
The free distance dfree of the convolutional code d f r ee lim d c l
Trellis Diagram for the computation
of dfree
00
0
0
0
0
0
0
1
1
1
1
3
States
10
2
2
2
01
2
1
11
0
1
2
2
2
2
2
1
1
1
1
1
1
1
2
2
2
2
3
4
5
Time
Trellis labels are the Hamming distances of encoder outputs and the
all-zero sequence.
Viterbi Algorithm
We want to compute
 k 1

  min  l ( l ) 
 l 0

{ 0 , 1 ,...,
k 1}
Functions whose arguments l can take on a finite
number of values
Simplest situation arises when T2, T1…….are “independent”
(The value taken on by each one of them does not influence
the other variables)
1
k 1
Then    min l ( l )
D
l 0
C
A
B
0
l
[1], pages 181 – 185
Viterbi Decoding of Convolutional Codes
P( y x )
received sequence
1.
Observe
is a maximum
transmitted symbols
P
(
y
|x
) P
(
y
|x
l)
l
(memoryless channel)
received
n0-tuple of
no-tuple
coded digits
2. We have, for a binary symmetric channel:
1-P
0
P
Tx
0
Rx
P
1
1
1-P
ln P( y l | x l )  d l ln
1 P
 n0 ln( 1  P)
P
Irrelevant
multiplicative
constant
Hamming distance
Between xl and yl
Irrelevant
additive
constant
Brute force approach:
• Compute all the values of the function and choose the
smallest one.
• We want a sequential algorithm
Viterbi Algorithm
What if 0, 1, … are not independent?
1
D
C
A B
Ely
Bishop
265
Los
Angeles
284
Las
182
Vegas
282
235
Cedar
City
224
Spanish
Forks
236
207
130
285
224
210
241
Denver
Durango
Blythe
Day 1
Grand
Junction
257
Salina
Page
228
0
Gallup
Williams
338
172
What is the shortest route from
Los Angeles to Denver?
215
Day 2
0 = A  1 = C
or
1 = D
0 = B  1 = D
Day 3
Day 4
Day 5
Viterbi Algorithm
l=0
l=1
2
l=2
1
2
1
3
4
1
2
1
1
4
1
1
2
1
2
1
0
4
4
(a)
1
(b)
(c)
2
4
3
4
3
2
2
3
6
2
4
4
1
5
2
l=3 4
l=5
l=4
4
0
6
(d)
l=5
1
l=2
3
1
1
1
1
2
2
2
l=4
0
2
2
0
l=1
2
l=3
3
3
(e)
1
5
Conclusion
dl
• We maximise P(y|x) by minimising 
, the Hamming distance
l
between the received sequence and coded sequence.
• Brute-Force Decoding
Compute all the distances between y and all the possible x’s.
Choose x that gives the minimum distance.
• Problems with Brute-Force Decoding
- Complexity
- Delay
• Viterbi algorithm solves the complexity problem (complexity
increases only linearly with sequence length)
• Truncated Viterbi algorithm also solves delay problem
Trellis-Coded Modulation
A.K.A
- Ungerboeck Codes
- Amplitude-Redundant Codes
- Modulation
How to increase transmission efficiency?
Reliability or Speed
• Band-limited environment
(Terrestrial radio communications)
• Use higher-order modulation schemes.
(8 PSK instead of 4 PSK)
Same BW, More bit/s per hz, more power
• Power-limited environment
(Satellite radio communications)
• Use coding: Less power, less bit/s per hz, BW
expanded
[2], pages 522-532
G. Ungerboeck, "Channel coding with multilevel/phase signals," IEEE Trans. Inform. Theory, vol. IT-28, pp. 55-67, 1982.
Construction of TCM
•
Constellation is divided into smaller constellations with
larger euclidean distances between constellation
points.
•
Construction of Trellis (Ungerboeck’s rules)
1.
2.
Parallel transitions are assigned members of the same partition
Adjacent transitions are assigned members of the next larger
transitions
Signals are used equally often
3.
Model for TCM
Memory Part
n
Select
Constellation
an
Select Signal
From
Constellation
Some examples of TCM schemes
Consider transmission of 2 bits/signal
We examine TCM schemes using 8-PSK
'
With uncoded 4-PSK we have
d min  2 '
We use the octogonary constellation
2
3
1
4
0
d’
5
6
7
' 
d'
2 sin

8
• The Free Distance of a convolutional code is the
Hamming distance between two encoded signals
• dfree is a measure of the separation among encoded
sequences: the larger dfree, the better the code (at least
for large enough SNR).
• Fact: To compute dfree for a linear convolutional code we
may consider the distances with respect to the all-zero
sequence.
Remerge
Split
(00)
(00)
An “error event”
A simple algorithm to compute dfree is
1) Compute dc(l) for l = 1,2,…
2) If the sequence giving dc(l) merges into the all-zero sequence, store its weight as dfree
First Key Point
Constellation size must be increased to get the same rate of information
• We gain
• We lose
d 2free
Minimum distance between sequences
2
d min
Minimum distance for uncoded transmission

'
Energy with coding
Energy without coding
Gain is

d 2free / 
2
d min
/'
Second Key Point
How to introduce the dependence among signals
xn  f an , an1 ,, an L 
Transmitted symbol
at time nT
We write
Source symbol
at time nT
Previous source
Symbols = “state” n
xn  f (an ,  n )
 n1  g (an ,  n )
(Describes output as a function
of input symbol + encoder state)
(Describes transitions between states)
TCM Example 1
Consider the 8-PSK TCM scheme, which involves the transmission of 2 bits/symbol using an
uncoded 4-PSK constellation and the coded 8-PSK constellation for the TCM scheme as shown
below

2
3
1
4
d min
0
0
5
6
Show that d min 
2 and
 
 0  2  ' sin  
8
7
'
TCM Example 1: Solution
 d min 


2

We have from the uncoded 4-PSK constellation sin 45   

2
 2
2
d min  2  sin 45   2 
We have from the coded 8-PSK constellation
20 
 '   '  2
2
2
20   ' '2  ' cos
Using sin 2

8

1
2 
1  cos

2
8 
20  2 ' (2) sin 2
 0  2  ' sin

8
 '  ' cos

4



 2 ' 1  cos 
4
4


8
 4 ' sin 2

8
TCM Scheme Based on 2-State
Trellis
0
0
d 2free
0
4
5

6
2
1
1
1

1

1
d
2

(0,2)  d 2 (0,1)  2  4 sin 2

8
 2.586
3
7
Hence, we get the coding gain

2.586
 1.293  1.1dB
2
a0
a1
I
4PSK
8PSK
Can this performance gain Trellis coded QPSK be
improved? The answer is yes by going to more
trellis states.
TCM Example 2
• Draw the set partition of 8-PSK with
maximum Euclidean distance between two
points.
• By how much is the distance between adjacent
signal points increased as a result of
partitioning?
2
d (0,2)
d (0,2)  2 ' 
2
'

d 2 (0,1)
2
d (0,1)  2  ' sin 
 4 sin
8
'
8
TCM Example 2: Solution
3
'
1
8
0
4
0

 0  2  ' sin
7
5
010
1
6
001
011
000
100
1  2 '  1.41  '
0
110
1
101
0
010
2  2  '
111
1
001
100
 0.765  '
011
000
111
101
(0, 4)
110
(2, 6)
(1, 5)
(3, 7)
TCM Scheme Based On A 4-State Trellis
Let us now use a TCM scheme with a more complex structure, in order to increase the coding gain.
Take a trellis with four states as shown below.
2
0
4
0426
0
0
And hence d free
2
6
2
1537
2
6 0
4
1
1


5
2640
7
3
3715

1 2
d (0,4)  d 2 (0,0)  d 2 (0,0)
'
'
2
1
1
 d 2 (0,4)  2  '
'
'

1
3
7
5
Calculating the Euclidian distance for each, one such path
s2 – s1 – s2, leaving and returning to s0, or s6 – s1 – s2
n
d E  min
s
s

0
n
n
sl  s0
n 2
l
n
1/ 2
 d 2free 


 '  4
  2
 
2 
2
 d min


  



TCM Code Worked Example
c4
a2
c3
c2
c1
a1
S1
16-QAM
8-PSK
Encoder
S2
Rate ½ 4-state Convolutional Code
Output
State Table for TCM Code
Inputs
a1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
a2
0
0
1
1
0
0
1
1
0
0
1
1
0
0
1
1
Initial State
S1
0
0
0
0
0
0
0
0
1
1
1
1
1
1
1
1
S2
0
0
0
0
1
1
1
1
0
0
0
0
1
1
1
1
Next State
S’1
0
0
0
0
1
1
1
1
0
0
0
0
1
1
1
1
S’2
0
1
0
1
0
1
0
1
1
0
1
0
1
0
1
0
Outputs
c1
0
0
0
0
1
1
1
1
0
0
0
0
1
1
1
1
c2
0
1
0
1
0
1
0
1
0
1
0
1
0
1
0
1
c3
0
0
1
1
0
0
1
1
0
0
1
1
0
0
1
1
Trellis Diagram of TCM Code
State
00
0
4
0426
0
10
6
4
6
6
0
1
01
3
3
7
7
1
11
2
OR
5
1537
2
  2 '    2 '   4 '

 

2
2
2640
d 2free  d 2 (0,6)  d 2 (0,6)
3715
5
d 2free
2
 d (0,4)   2    4 '


2
'
Coding Gain over Uncoded QPSK
Modulation

dmin =
2
2
d min
 2
Uncoded QPSK
 d 2free 


 '  4
   2 or 3 dB
Gain  = 
2 
2
 d min


  


TCM Problems
c4
a2
c3
a1
S1
S2
c2
c1
c4
a2
c3
c2
a1
S1
S2
S3
c1
16-QAM
8-PSK
Encoder
Output
16-QAM
8-PSK
Encoder
Output
c4
a2
c3
c2
a1
S1
S2
S3
S4
c1
16-QAM
8-PSK
Encoder
Output
The trellis-coded signal is formed as shown below, by encoding one bit using a rate
½ convolutional code with three additional information bits left uncoded. Perform the
set partitioning of a 32-QAM (cross) constellation and indicate the subsets in the
partition. By how much is the distance between the adjacent signal points increased as
a result of partitioning.
c1
a1
a2
a3
a4
c2
c3
c4
c5
TCM and Decoding
• Viterbi Algorithm is used with soft
decisions of the demodulator for maximum
likelihood estimation of the sequence
being transmitted
Turbo Encoding / Decoding
By R. A. Carrasco
Professor in Mobile Communications
School of Electrical, Electronic and Computing Engineering
University of Newcastle-upon-Tyne
[1], pages 209 – 215
http://en.wikipedia.org/wiki/Turbo_code
Introduction
• The Turbo Encoder
–
–
–
–
–
Overview
Component encoders and their construction
Tail bits
Interleaving
Puncturing
• The Turbo Decoder
– Overview
– Scaling
Introduction Cont’d
• Results
– AWGN results
• Performance
• Conclusions
Concatenated Coding and Turbo Codes
Outer encoder
Input
Systematic bits
Input
Inner encoder
data
data
encoder
Parity bits#1
channel
interleaver
Parity bits#2
Output
data
encoder
Outer decoder
Inner decoder
Serially concatenated codes
•
Parallel-concatenated (Turbo encoder)
Convolutional codes
Non-systematic convolutional codes (NSC)
– Have no fixed back paths;
– They act like a finite impulse response (FIR) digital filter;
– NSC codes do not lead themselves to parallel concatenation;
– At high SNR the BER performance of a classical NSC code is better than the
systematic convolutional codes of the same constraint length.
The Turbo Encoder
dk
Component
Encoder #1
s k = dk
pk1
Component
Encoder #2
pk2
Interleaver
dk-1
• Recursive systematic convolutional encoders in
parallel concatenation, separated by pseudo-random
interleaver
• Second systematic is interleaved version of first
systematic
– Interleaver process is known at the decoder, therefore this is
surplus to our needs
Component Encoders
• [7;5]8 RSC component
encoder
• 4 state trellis
representation
sk
dk
D
D
pk
• [23;33]8 RSC
component encoder
• 16 state trellis
representation
sk
dk
D
D
D
D
pk
Systematic convolutional codes
– Recursive Systematic Convolutional (RSC) codes can
be generated from NSC codes by connecting the output
of the encoder directly to the input;
– At low SNR the BER performance of an RSC code is
better than the NSC.
The operation of the Turbo encoder is as follow:
1.
The input data sequence is applied directly to encoder 1
and the interleaved version of the same input data
sequence is applied to encoder 2.
2.
The systematic bits (i.e. the original message bits) and
the two parity check bit streams (generated by the two
encoders) are multiplexed together to form the output of
the encoders.
Turbo code interleavers
The novelty of the parallel-concatenated turbo encoder lies in
the use of RSC codes and the introduction of an
interleaver between the two encoders;
•
The interleaver ensures that two permutations the same
input data are encoded to produce two different parity
sequences;
•
The effect of the interleaver is to tie together errors that are
easily made in one half of the turbo encoder to errors that
are exceptionally unlikely to occur in the other half;
•
This ensures robust performance in the event that the
channel characteristics are not known and is the reason
why turbo codes perform better than traditional codes.
Turbo code interleavers (Cont’d)
• The choice of interleaver is therefore the key to be
performance of a turbo coding system;
• Turbo code performance can be analysed in terms of the
Hamming distance between the code words;
• If the applied input sequence happens to terminate one of
the encoders, it is unlikely that, once interleaved, the
sequence will terminate the other leading to a large
hamming distance in at least one of the two encoders;
• A Pseudo-random interleaver is a good choice.
Interleaving
• Shannon states that large frame length
random codes can achieve channel
capacity
• By their very nature, random codes are
impossible to decode
• Pseudo-random interleavers make turbo
codes appear random while maintaining a
decodable structure
Interleaving cont’d
• Primary use
– To increase average codeword weight
– Altering bit position does not alter data-word weight
but can increase codeword weight
– Thus a low weight convolutional output from encoder
#1 does not mean a low-weight turbo output
DATAWORD
CODEWORD
CODEWORD
WEIGHT
01100
0011100001
4
01010
0011011001
5
10010
1101011100
6
Interleavers
• An interleaver takes a given sequence of symbols and permutes their positions,
arranging them in a different temporal order;
• The basis goal of an interleaver is to randomise the data sequence when used
against burst errors;
• In general, data interleavers can be classified into: block, convolutional, random
and linear interleavers;
• Block interleaver: data are first written in row format in a permutation matrix, and
then read in a column format;
• A pseudo – random interleaver is a variation of a block interleaver when data are
stored in a register at position that are deinterleaved randomly;
• Convolutional interleaver are characterised by a shift of the data, usually applied in
a fixed and cumulative way.
Example: Block interleaver
•
Data sequence: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
read out:
1
5
9
13
2
6
10
14
3
7
11
15
4
8
12
16
1 5 9 13 2 6 10 14 3 7 11 15 4 8 12 16
read in (interleave):
read out:
receive
Channel
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
read in (De-interleave):
transmit
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
Example: Pseudo - random interleaver
•
Data sequence: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
read out (by a random position
pattern):
1
6
11
15
2
5
9
13
4
7
12
14
3
8
16
10
1 6 11 15 2 5 9 13 4 7 12 14 3 8 16 10
read in (interleave):
read out: (by the inverse of random
position pattern)
receive
Channel
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
read in (De-interleave):
transmit
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
Convolutional interleaver
(N-1)/L
………….
(N-1)/L
Channel
………….
2L
2L
L
Continued…
Interleave: Input: …, x0, x1, x2, x3, x4, x5, x6, …
x0
x1
x-3
Output: …, x0, x-3, x-6, x-9, x4, x1, x-2, x-5, x8, x5,
x2, x-1 …
D
x2
x-6
x-2
D
D
x3
x-1
D
x-5
D
D
x-9
D
De-interleave:
x12
Channel
x8
D
receive
transmit
Corresponds to a delay of 4 symbols
in this example.
x9
Input: …, x0, x-3, x-6, x-9, x4, x1, x-2, x-5,
x8, x5, x2, x-1 …
D
D
x1
x5
D
D
x6
x0
x4
x2
D
x3
Output: …, x0, x1, x2, x3, x4, x5, x6, …
Puncturing
• High rate codes are usually generated by a
procedure known as puncturing;
• A change in the code rate to ½ could be achieved by
puncturing the two parity sequences prior to the
multiplexer. One bit might be deleted from code parity
output in turn, such that one parity bit remains for
each data bit.
Puncturing
• Used to reduce code rate
– Omits certain output bits according to a prearranged method
– Standard method reduces turbo codeword from
rate n/k = 1/3 to rate n/k = 1/2
s1,1 s1,2 s1,3 s1,4
p2,1 p2,2 p2,3 p2,4
puncturer
p1,1 p1,2 p1,3 p1,4
s1,1 p1,1 s1,2 p2,2 s1,3 p1,3 s1,4 p2,4
Tail Bits
• Tail bits are added to the dataword such that the first
component encoders codeword terminates at the all-zero state
– Look up table is most common method
Data bits
S0
=1
=0
S1
S2
S3
Tail
Turbo decoding
• A key component of iterative (Turbo) decoding is
the soft-in, soft-out (SISO) decoder;
Matched filter
S+H
8-level
3-bit
quantization
Combined soft decision
error control decoding
Soft decisions
Hard decisions
Binary quantization
Error
control
Matched filter
S+H
Hard decisions
[1], pages 239 -244
Soft decision
Digital 0
000
010
001
0
Digital 1
100
011
110
101
111
1
Soft decision
Hard decision vs. Soft decision
1.
Soft (multi-level) decisions;
2.
Hard (two-level) decisions;
Each soft decision contains not only information about the most likely
transmitted symbol
000 to 011 indicating a likely 0
100 to 111 indicating a likely 1
but also information about the confidence or likelihood which can be
placed on this decision.
The log-likelihood ratio
•
It is based on modulo-2 addition of the binary random
variable u, which is -1 for logic 0, and +1 for logic 1;
•
L(u) is the log-likelihood ratio for the binary random
variable and is defined as:
P(u  1)
L(u )  ln
P(u  1)
This is described as the ‘soft’ value of the binary random
variable u.
The sign of the value is the hard decision while the
magnitude represents the reliability of this decision;
The log-likelihood ratio
•
•
•
As L(u) increases towards +∞,the
probability that u=+1 also increases.
As L(u) increases towards - ∞, the
probability that u=-1 increases.
For the conditional log-likelihood ration
L(u/y) defined as:
P(u  1 / y )
L(u / y )  ln
P(u  1 / y )
The log-likelihood ratio
The information u is mapped to the encoded bits x. These
encoded bits are received by the decoder as y. All with the
time index k. From this the log-likelihood ration for the
system is:
P( xk  1 / yk )
L( xk / yk )  ln
P( xk  1 / yk )
From Bayes theorem, this is equivalent to:
L( xk / yk )  ln
P( yk / xk  1) P( xk  1)
P( yk / x k  1)
P( xk  1)
 ln
 ln
P( yk / xk  1) P( xk  1)
P( yk / xk  1)
P( xk  1)
The log-likelihood ratio
•
Assuming the ‘channel’ to be flat fading with Gaussian noise, the
Gaussian pdf, G(x):
G ( x) 
1
e
2

( xq )2
2 2
With q representing the mean and  2 representing the variance,
showing that:
P( y k | xk ) 
1

e
Eb
 y k  axk 2
N0
2N0

L( y k | xk )  ln
Eb
( yk  a ) 2
N0
P( y k / xk  1)
e
Eb
 ln Eb
4
ayk
P( y k / xk  1)
N0

( yk  a ) 2
e N0
The log-likelihood ratio
Eb
where N represent the signal to noise ratio per bit and a being the fading
0
amplitude (a = 1 for a non- fading Gaussian Channel).
From equation
L(u )  ln
P(u  1)
P(u  1)
P ( x, y )  P ( x | y ) P ( y )  P ( y | x ) P ( x )  P ( x | y ) 
P( y | x) P( x)
P( y )
 P y | x  1Px  1 


P
(
y
)
Px  1 | y 
  ln P( y | x  1) P( x  1)
L( x | y )  ln
 ln 
Px  1 | y 
P( y | x  1) P( x  1)
 P y | x  1P( x  1) 


P( y )


L( xk / yk )  ln
P( yk / x k  1)
P( xk  1)
 ln
 Lc ( y | x)  L( x)
P( y k / xk  1)
P( xk  1)
The log-likelihood ratio
The log likelihood ratio of xk depends on yk is:
L( xk / yk )  Lc ( yk )  L( xk )
where
Eb
Lc  4
a
N0
is the channel reliability.
Therefore L(xk/yk) is the weighted received
value.
Turbo Decoding: Scaling by Channel
Reliability
• Channel Reliability = 4*Eb/N0*Channel
Amplitude
– Channel Amplitude for AWGN = 1
– Channel Amplitude for Fading varies
Corrupted, Received
codeword
4*Eb/N0*A
Scaled, Corrupted,
Received codeword
Performance of Turbo Codes
100
10-1
10-2
Uncoded
10-3
10-4
Turbo codes
Shannon
limit
10-5
10-6
-4
-2
0
2
4
6
8
10
At a bit error rate of 10-5, the turbo code is less than 0.5 dB
from Shannon's theoretical limits.
Block diagram of Turbo Decoder
De-interleaver
Noise
Systematic
Decoder
Stage 1
Interleaver
Decoder
Stage 2
bits
De-interleaver
Noise parity-check bits ε1
Noise parity-check bits ε2
Hard-limiter
Block diagram of Turbo Decoder
Decoder bits
Turbo Decoder
Figure shows the basic structure of the turbo decoder. It operates on
noisy versions of the systematic bits and the two noisy version of the
parity bits in two decoding stages to produce an estimate of the original
message bits.
~
~
Set I 2 ( x)  0
∑
BCJR
I 2 ( x)
+
∑
~
I 1 ( x)
∑
I 2 ( x)
I
BCJR
I1 ( x)
u
ε1
Stage 1
u
Stage 2
ε2
D
+
∑
-
Hard
~
Limiter
x
Turbo Decoding
• The BCJR algorithm is a soft input –soft output decoding
algorithm with two recursions, one forward and the other
backward.
• At stage 1, the BCJR algorithm uses extrinsic information
I2(x) added to the input (u). At the output of the decoder
the ‘input’ is subtracted from the ‘output’ and only the
information generated by the 1st decoder is passed on I1(x).
For the first ‘run’ I2(x) is set to zero as there is no ‘prior’
information.
Turbo Decoding
• At stage 2, the BCJR algorithm uses extrinsic information
I1(x) added to the input (u). The input is then interleaved
so that the data sequence matches the previously
interleaved parity (ε2). The decoder output is then deinterleaved and the decoder’s ‘input’ is subtracted so that
only decoder 2’s information is passed on I2(x). After this
loop has repeated many times, the output of the 2nd
decoder is hard limited to form the output data.
Turbo Decoding
The first decoding stage use the BCJR Algorithm to produce a
soft estimate of systematic bit xJ, expressed as the loglikelihood ratio:
~
I1 ( x J )  ln(
P( x J  1 | u,  1 , I 2 ( x))
~
P( x J  0 | u,  1 , I 2 ( x))
)
J  1,2,3, 
where u is the set of noisy systematic bits, ε1 is the set of noisy
parity-check bits generated by encoder 1.
Turbo Decoding
I2(x) is the extrinsic information about the set of message
bits x derived from the second decoding stage and fed
back to the first stage.
K
I 1 ( x )   I1 ( x J )
J 1
The total log-likelihood ratio at the output of the first
decoding stage is therefore:
Turbo Decoding
The extrinsic information about the message bits derived from
~
~
the first decoding stage is:
I 1 ( x)  I1 ( x)  I 2 ( x)
~
where I 2 ( x ) is to be defined.
Other
information
Extrinsic
Soft-input
∑
Intrinsic
information
∑
Soft-output
information
Raw data
At the output of the SISO decoder, the ‘input’ is subtracted from the
‘output’ and only the reliability information generated by the decoder is
passed on as extrinsic information to the next decoder.
Turbo Decoding
~
The extrinsic information I 2 ( x ) fed back to the first
decoding stage is therefore:
~
~
I 2 ( x)  I 2 ( x)  I 1 ( x)
~
~
where I 1 ( x) is itself defined before and I 2 ( x ) is the loglikelihood ratio computed by the second storage.
~
I 2 ( x J )  log 2 (
P( x J  1 / u,  2 , I 1 ( x))
~
P( x J  0 / u,  2 , I 1 ( x))
)
J  1,2, 
Turbo Decoding
An estimate of the message bits x is computed by
hard-limiting the log-likelihood ratio I 2 ( x) at the output
of the second stage, as shown by:

x  sgn( I 2 (n))

we set x 2 ( x)  0 on the first iteration of the algorithm.
Turbo Decoding: Serial to Parallel
Conversion and Erasure Insertion
• Received, corrupted codeword is returned to
original three bit streams
• Erasures are replaced with a ‘null’
Serial to Parallel
s1,1 p1,1 s1,2 p2,2 s1,3 p1,3 s1,4 p2,4
s1,1 s1,2 s1,3 s1,4
p1,1
0
p1,3
0
0
p2,2
0
p2,4
Results over AWGN
[7;5] SOVA vs Log-MAP for AWGN, 512 data bits
1.00E+00
1.00E-01
BER
1.00E-02
7;5 punctured, AWGN, Log-MAP
1.00E-03
7;5 punctured, AWGN, SOVA
1.00E-04
1.00E-05
1.00E-06
0.5
1
1.5
2
snr
2.5
3
Questions
•
•
•
•
•
•
•
•
•
•
What is the MAP algorithm first of all? Who found it?
I have heard about the Viterbi algorithm and ML sequence estimation for decoding
coded sequences. What is the essential difference between these two methods?
But I haven’t heard about the MAP algorithm until recently (even though it was
discovered in 1974). Why?
What are SISO (Soft-Input-Soft-Output) algorithms first of all?
Well! I am quite comfortable with the basics of SISO algorithms. But tell me one thing.
Why should a decoder output soft values? I presume there is no need for it to do that.
How does the MAP algorithm work?
Well then! Explain MAP as an algorithm. (Some flow-charts or steps will do).
Are there any simplified versions of the MAP algorithm? (The standard one involves a
lot of multiplication and log business and requires a number of clock cycles to
execute.)
Is there any demo source code available for the MAP algorithm?
References
Problem 1
•
Let rc1=p/q1 and rc2=p/q2 be the codes rates of RSC encoders 1 and 2 in
the turbo encoder of figure 1. Determine the code rate of the turbo
code.
• The turbo encoder of figure 1 involves the use of two RSC encoders .
(i) Generalise this encoder to encompass a total of M interleavers .
(ii) construct the block diagram of the turbo decoder that exploits the M
sets of parity-check bits generated by such a generalization.
p

p
ENC
1
p
ENC
2
q1
q2
Figure 1
Problem 2
Consider the following generator matrices for rates ½
turbo codes:
(i)
4-state encoder: g (D) =
 1  D  D2 
1,

1  D2 

8-state encoder: g (D)=

1  D 2  D3 
1,
2
3
 1 D  D  D 
Construct the block diagram for each one of these RSC
encoders.
(ii) Construct the parity-check equation associated with each
encoder.
Problem 3
• Explain the principle of Non-systematic convolutional
codes (NSC) and Recursive systematic convolutional
codes (RSC) and make comparisons between the two
• Describe the operation of the turbo encoder
• Explain how important the interleaver process is to the
performance of a turbo coding system
Problem 4
• Describe the meaning of Hard decision and soft decision
for the turbo decoder process
• Discuss the log-likelihood ratio principle for turbo
decoding system
• Describe the iterative turbo decoding process
• Explain the operation of the soft-input-soft-output (SISO)
decoder
School of Electrical, Electronics and
Computer Engineering
Low Density Parity Check Codes: An
Overview
By R.A. Carrasco
Professor in Mobile Communications
University of Newcastle-upon-Tyne
[1], pages 277 – 287
http://en.wikipedia.org/wiki/LDPC
Outline
• Parity check codes
• What are LDPC Codes?
• Introduction and Background
• Message Passing Algorithm
• LDPC Decoding Process
– Sum-Product Algorithm
– Example of rate 1/3 LDPC (2,3) code
• Construction of LDPC codes
– Protograph Method
– Finite Geometries
– Combinatorial design
• Results of LDPC codes constructed using BIBD design
Parity Check Code
•
A binary parity check code is a block code: i.e., a collection of
binary vectors of fixed length n.
• The symbols in the code satisfy m parity check equations of the
form:
–
xa xb  xc  …  xz = 0
– where  means modulo 2 addition and
–
xa, xb, xc , … , xz
• are the code symbols in the equation.
•
Each codeword of length n can contain (n-m)=k information
digits and m check digits.
Example: Hamming Code with
n=7, k=4, and m=3
For a code word of the form c1, c2, c3, c4, c5, c6, c7, the equations are:
c1  c2  c3  c5 = 0
c1  c2  c4  c6 = 0
c1  c3  c4  c7 = 0
The parity check matrix for this code is then:
1 1 1 0 1 0 0
1 1 0 1 0 1 0
1 0 1 1 0 0 1
Note that c1 is contained in all three equations while c2 is contained in
only the first two equations.
What are Low Density Parity Check
Codes?
•The percentage of 1’s in the parity check matrix for a LDPC
code is low.
•A regular LDPC code has the property that:
–every code digit is contained in the same number of
equations,
–each equation contains the same number of code
symbols.
•An irregular LDPC code relaxes these conditions.
Equations for Simple LDPC Code with
n=12 and m=9
c3  c 6  c7  c8 = 0
c1  c2  c5  c12 = 0
c4  c9  c10  c11 = 0
c2  c6  c7  c10 = 0
c1  c3  c8  c11 = 0
c4  c5  c9  c12 = 0
c1  c 4  c5  c7 = 0
c6  c8  c11  c12= 0
c2  c3  c9  c10 = 0
The Parity Check Matrix for the
LDPC Code
c1 c2 c3 c4 c5 c6 c7 c8 c9c10c11c12
0
1
0
0
1
0
1
0
0
0
1
0
1
0
0
0
0
1
1
0
0
0
1
0
0
0
1
0
0
1
0
0
1
1
0
0
0
1
0
0
0
1
1
0
0
1
0
0
1
0
0
0
1
0
1
0
0
1
0
0
1
0
0
1
0
0
0
1
0
0
1
0
0
0
1
0
0
1
0
0
1
0
0
1
1
0
0
0
0
1
0
0
1
0
1
0
0
1
0
0
1
0
0
0
1
0
1
0
c3  c6  c7  c8 = 0
c1  c2  c5  c12 = 0
c4  c9  c10  c11 = 0
c2  c6  c7  c10 = 0
c1  c3  c8  c11 = 0
c4  c5  c9  c12 = 0
c1  c4  c5  c7 = 0
c6  c8  c11  c12= 0
c2  c3  c9  c10 = 0
Introduction
- LDPC codes were originally invented by Robert Gallager in the early
1960s but were largely ignored until they were rediscovered in the mid1990s by MacKay.
- Defined in terms of a parity check matrix that has a small number of nonzero entries in each column
- Randomly distributed non-zero entries
–Regular LDPC codes
–Irregular LDPC codes
- Sum and Product Algorithm used for decoding
•- Linear block code with sparse (small fraction of ones) parity-check matrix
•- Have natural representation in terms of bipartite graph
•- Simple and efficient iterative decoding
Introduction
Low Density Parity Check (LDPC) codes are a class of linear block codes
characterized by sparse parity check matrices (H).
Review of parity check matrices:
– For a (n,k) code, H is a (n-k ,n) matrix of ones and zeros.
– A codeword c is valid if cHT =s= 0
– Each row of H specifies a parity check equation. The code bits in positions where the row
is one must sum (modulo-2) to zero.
– In an LDPC code, only a few bits (~4 to 6) participate in each parity check equation.
– From parity check matrix we obtained Generator Matrix G which is used to generate
LDPC Codeword.
– G.HT = 0
– Parity check matrix is arranged in Systmatic from as H = [Im | P]
– Generator matrix G = [Ik | PT]
– Code can be expressed as c= x . G
Low Density Parity Check Codes
•
Representations Of LDPC Codes
parity check
matrix
1 0 1 0 0 1 1 0 1 0 0 1 
0 1 1 1 1 0 1 0 0 0 0 1 


0 1 0 1 0 1 0 1 1 1 0 0 


1
1
1
0
0
0
0
1
1
0
1
0


0 0 0 1 1 0 1 0 0 1 1 1 


1 0 0 0 1 1 0 1 0 1 1 0 
(Soft) Message passing:
Variable nodes communicate to
check nodes their reliability (loglikelihoods)
Check nodes decide which
variables are not reliable and
“suppress” their inputs
Number of edges in graph =
density of H
Sparse = small complexity
Parity Check Matrix to Tanner Graph
1 0 1 0 0 1
0 1 1 1 1 0

0 1 0 1 0 1

1 1 1 0 0 0
0 0 0 1 1 0

1 0 0 0 1 1
1 0 1 0 0 1

1 0 0 0 0 1
0 1 1 1 0 0

0 1 1 0 1 0
1 0 0 1 1 1

0 1 0 1 1 0 
LDPC Codes
• Bipartite graph with
connections defined
by matrix H
• r’: variable nodes
– corrupted codeword
• s: check nodes
– Syndrome, must be
all zero for the
decoder to claim no
error
• Given the syndromes and
the statistics of r’, the
LDPC decoder solves the
equation
r’HT=s
in an iterative manner
Construction of LDPC codes
• Random LDPC codes
– MacKay Construction
•Computer Generated random Construction
• Structured LDPC codes
– Well defined and structured code
– Algebraic and Combinatoric construction
– Encoding advantage over random LDPC codes
– Performs equally well as random codes
– Examples
•Vandermonde-matrix (Array codes)
• Finite Geometry
• Balance Incomplete block design
• Other Methods (e.g. Ramanujan Graphs)
Protograph Construction of LDPC codes by J.
Thorpe
•
A protograph can be any Tanner graph with a relatively small
number of nodes.
•
The protograph serves as a blueprint for constructing LDPC
codes of arbitrary size whose performance can be predicted
by analysing protograph.
J.Thrope, “Low-Density Parity Check codes constructed from Protograph,” IPN Progress report 42-154,
2003.
Protograph Construction (Continued)
LDPC Decoding (Message passing
Algorithm)
•Decoding is accomplished by passing messages along the lines of
the graph.
•The messages on the lines that connect to the ith variable node, ri,
are estimates of Pr[ri =1] (or some equivalent information).
•Each variable node is furnished an initial estimate of the probability
from the soft output of the channel.
•The variable node broadcasts this initial estimate to the check
nodes on the lines connected to that variable node.
•But each check node must make new estimates for the bits involved
in that parity equation and send these new estimates (on the lines)
back to the variable nodes.
Message passing Algorithm or SumProduct Algorithm
While(not equal to stop criteria)
{
- All variable nodes pass messages to
corresponding check nodes
- All check nodes pass messages to
corresponding variable nodes
}
Stop Criteria:
• Satisfying Equation r’HT=0
• Maximum number of iterations reached
LDPC Decoding Process
Check Node Processing
Variable Node Processing
Lm1n
z mn1
z mn 2
Lmn 2
Lmn1
Check Node
m
Lmn 3
z mn 3
Var Node
n
Lmn 4
z mn 4
Var Nodes
N(m)



Lmn  2 tanh   tanh z mn 2
 nN ( m ) \ n

1
z
L m2n
m1 n
zm
2n
zm3n
Lm n
3
zm
4n
Fn
Lm
Where Fn is the channel
reliability value
zmn  Fn 
zn  Fn 
4n
Check Nodes
M(n)
L
mn
mM ( n ) \ m
L
mM ( n )
mn
,
for hard
decision
Sum-Product Algorithm
Step #1 : Initialisation
LLR of the (soft) received signal yi , for AWGN
Lij = Ri = 4 y i
Eb
where j represents check and i variable node
No
Step # 2 Check to Variable Node
Extrinsic message from check node j to bit node i

L '

 1   tanh  i j  

2 
 i ' B j , i '  i


Ei j  ln 

L


i' j
 1



tanh


2 
 i ' B , i'  i


j

Where Bj represents set of column locations of the bits in the jth parity-check
equations and ln is natural logarithm
Sum-Product Algorithm (Continued)
Step #3 : Codeword test or Parity Check
Combined LLR is the sum of the extrinsic LLRs and the
original LLR calculated in step #1.
L i   Ei j  R i
jAi
Where Ai is the set of row locations of the parity-check equations
which check on the ith bit of the code
Hard decision is made for each bit
1, L i  0
zi  
0, L i  0
Sum-Product Algorithm (Continued)
Condition to stop further iterations:
If r =[r1, r2,…………. rn] is a valid code word then,
• It would satisfied H.rT = 0
• maximum number of iterations reached.
Step #4 : Variable to Check Node
Variable node i send LLR message to check node j
without using the information from check node j .
Lij 

j' Ai , j'  j
Ei j'  R i
Return to step # 2
Example:
Code word send is [ 001011 ] through AWGN channel with Eb/No
= 1.25 and Vector [-0.1 0.5 –0.8 1.0 –0.7 0.5] is received.
1
0
H
1

0
Parity Check Matrix
Step # 1
Iteration 1
R   0.5
1
0 1
2.5
0 1
1
1
0
0
1
0
0
1
 4.0
0
1 0 0
0 1 0
0 1 1
1
5.0
0 1


 R=



 3.5
4 yi
2.5
After Hard Decision we find that 2 bits are in errors so we need to
apply LDPC decoding to correct the errors.
Eb
No
Example (Continued)
Variable Nodes
Check Nodes
Variable Nodes
1
0

1

0
1 0 1 0 0 

1 1 0 1 0 
0 0 0 1 1 

0 1 1 01 
Check Nodes
Initialisation
0 5
0 0 
 0.5 2.5
 0 2.5  4 0  3.5 0 

Li,j  
 0.5 0
0 0  3.5 2.5 


0 4 5
0 2.5 
 0
Questions
c  c1 c2 c3 c4c5 c6
1. Suppose we have codeword c as follows:
where each ci is either 0 or 1 and codeword now has three paritycheck equations
c1  c2  c5  0
c1  c3  c6  0
c1  c2  c4  c6  0
a) Determine the parity check matrix H by using the above equation
b) Show the systematic form of H by applying Gauss Jordan
elimination
c) Determine Generator matrix G from H and prove G * HT = 0
d) Find out the dimension of the H, G
e) State whether the matrix is regular or irregular
Questions
2. The parity check matrix H of LDPC code is given below:
1 0 1 0 0 1
0 1 1 1 1 0

0 1 0 1 0 1
H
1 1 1 0 0 0
0 0 0 1 1 0

1 0 0 0 1 1
a)
b)
c)
d)
e)
f)
1 0 1 0 0 1
1 0 0 0 0 1 
0 1 1 1 0 0

0 1 1 0 1 0
1 0 0 1 1 1

0 1 0 1 1 0 
Determine the degree of rows and column
State whether the LDPC code is regular or irregular
Determine the rate of the LDPC code
Draw the tanner graph representation of this LDPC code.
What would be the code rate if we make rows equals to column
Write down the parity check equation of the LDPC code
Questions
3. Consider parity check matrix H generated in question 1,
a)
Determine message bits length k, parity bits length m, codeword length n
b)
Use the generator matrix G obtained in question 1 to generate all possible codewords c.
4.
a)
What is the difference between regular and irregular LDPC codes?
b)
What is the importance of cycles in parity check matrix?
c)
Identify the cycles of 4 in the following tanner graph.
Check
Nodes
Variable Nodes
Solutions
Question 1 a)
1 1 0 0 1 0 
H  1 0 1 0 0 1  (A)
1 1 0 1 0 1 
Desired diagonal
Question 1 b)
Applying Gaussian elimination first
1 1 0 0 1 0 
0 1 1 0 1 1 


1 1 0 1 0 1 
modulo-2 addition of R1
and R2 in equation (A)
1 1 0 0 1 0 
0 1 1 0 1 1 


0 0 0 1 1 1 
modulo-2 addition of R1
and R3 in equation (A)
This 1 needs to be
eliminated to achieve
the identity matrix I
1 1 0 0 1 0 
0 1 0 1 1 1 


0 0 1 0 1 1 
Swap C3 with C4 to
obtain the diagonal of 1’s
in the first 3 columns
Solutions
Now, we need an Identity matrix of 3 x 3 dimensions. As you can see the first 3 columns and
rows can become an identity matrix if we somehow eliminate 1 in the position (row =1 and
column =2). To do that we apply Jordan elimination to find the parity matrix in
systematic form, hence
Modulo- 2 addition of R2 into R1 gives
1 0 0 1 0 1 
Hsys  0 1 0 1 1 1 
0 0 1 0 1 1 
It is now in systematic form and represented as
H = [I | P]
Solutions
Generator matrix can be obtained by using H in the systematic form obtained in a)
G = [PT | I]
1 1 0 1 0 0 
G  0 1 1 0 1 0 
1 1 1 0 0 0 
To prove G * HT = 0
1
0
1 1 0 1 0 0  
0 1 1 0 1 0  * 0

 1
1 1 1 0 0 0  
0

1
0
1
0
1
1
1
0 
0 
0 0 0 
1  

  0 0 0 
0 
0 0 0 

1

1 
Solutions
d) Dimension of H is (3 × 6) and G is also (3 × 6)
e) Matrix is irregular because the number of 1’s in rows and columns are not equal e.g.
number of 1’s in 1st row is ‘3’ while 3rd row has ‘4’ 1’s. Similarly, number of 1’s in 1st
column is ‘3’ while in 2nd columns has ‘2’ 1’s.
Question 2
a) The parity check matrix H contains 6 ones in the each row and 3 ones in each column. The
degree of rows is the number of 1’s in the rows which is 6 in this case, similarly and the
degree of column is the number of 1’s in the column hence in this case 3.
b) Regular LDPC, because the number of ones in each row and column are the same
c) Rate = 1 – m / n = 1 – 6/12 = ½
Solutions
d) Tanner graph is obtained by connecting the check and variable nodes as follows
Solutions
e) If we make rows equals to columns then the code rate is 1. It means there is no redundancy
involved in the code and all bits are the information bits.
f) Parity check equations of the LDP code are
c1  c3  c6  c7  c9  c12  0
c2  c3  c4  c5  c7  c12  0
c2  c4  c6  c8  c9  c10  0
c1  c2  c3  c8  c9  c11  0
c4  c5  c7  c10  c11  c12  0
c1  c5  c6  c8  c10  c11  0
Solutions
Question 3
a)
message bits length k = 6-3 =3
parity bits length m = 3
codeword length n = 6
b) Since the information or message is 3 bits long therefore,
The information bits has 8 possibilities
as shown in the Table below
x0
x1
x2
0
0
0
0
0
1
0
1
0
0
1
1
1
0
0
1
0
1
1
1
0
1
1
1
Solutions
The codeword is generated by multiplying information bits with the generator matrix as
follows
c=xG
The Table below shows the code words generated by using G in question 1.
x
c
000
000000
001
111001
010
011010
011
100011
100
110100
101
001101
110
101110
111
010111
Solutions
Question 4
a) The Regular LDPC code has constant number of 1’s in the rows and columns of the
Parity check matrix.
The Irregular LDPC code has variable number of 1’s in the rows and columns of the
Parity check matrix.
b)
A cycle in a tanner graph is a sequence of connected vertices that start and end at the same
vertex in the graph, and other vertices participates only once in the cycle. The length of the
cycle is the number of edges it contains, and the girth of a graph is the size of the smallest
cycle. A good LDPC code should have a large girth so as to avoid short cycles in the tanner
Graph since they introduce an error floor. Avoiding short cycles have been proven to be more
effective in combating error floor in the LDPC code. Hence the design criteria of LDPC
code should be such that it removes most of the short cycles in the code.
Solutions
c) Cycles of length 4 are identified as follows
Check Nodes
Variable Nodes
Security in Mobile Systems
By Prof R A Carrasco
School of Electrical, Electronic and Computing Engineering
University of Newcastle-upon-Tyne
Security in Mobile Systems
• Air Interface Encryption
-Provides security to the Air Interface
-Mobile Station to Base Station
• End-to-end Encryption
-Provides security to the whole
communication path
-Mobile Station to Mobile Station
Air Interface Encryption Protocols
• Symmetric Key
-Use Challenge Response Protocols for
authentication and key agreement
• Asymmetric Key
-Use exchange and verification of
‘Certificates’ for authentication and key
agreement
Where it is used
Challenge Response Protocol
GSM
*Only authenticates the Mobile Station
*A3,A8,Algorithms are used
TETRA
*Authentication both Mobile Station and the
Network
*TA11, TA12, TA21, TA22 algorithm are used
3G
*Authentication and Key Agreement (AKA)
3G
• Advantages
- Simpler than Public key techniques
- Less processing power required in the hand set
• Disadvantages
- Network has to maintain a Database of secret
keys of all the Mobile stations supported by it
- The secret key is never changed in normal
operation
- Have to share secret keys with MS
Challenge-Response Protocol
Challenge-Response Protocol
1. MS sends its identity to BS
2. BS sends the receiver MS identity to AC
3. AC gets the corresponding key ‘k’ from database
4. AC generates a random number called a challenge
5. By hashing K and the challenge the AC computes a
signed response
6. It also generates a session authentication key by
hashing K and the challenge (difference hashing
function)
Challenge-response Protocol
7.AC sends the challenge, Response and session
key to BS
8.BS sends the challenges to the MS
9.MS computes the Response and the session
authentication key
10.MS sends the response to the BS
11. If the two Responses received by BS from AC
& MS are equal, the MS is authentic
12. Now MS and BS uses the session key to
encrypt the communication data between them
Challenge-Response Protocol
MS
BS
AC
Database
Identity (i)
K
challenge
Identity (i)
challenge
Response/ks
Response
K
Identity
key
I1
I2
K1
K2
Challenge response protocol in GSM
1)MS sends its IMSI (international mobile
subscriber identity) to the VLR
2)VLR sends the IMSI to AC via HLR
3)AC looks up in the database and gets the
authentication key “Ki” using IMSI
4)Authentication center generates RAND
5)It combines Ki with RAND to produce SRES
using A3 algorithm
6)It combines ki with RAND to produce kc using A8
algorithm
Challenge response protocol in GSM
7) The AC provides the HLR a set of RAND ,SRES ,kc
triplets
8) HLR sends one set to the VLR to authenticate the MS
9) VLR sends RAND to the MS
10) MS computes SRES and kc using ki and RAND
11) MS sends SRES to the VLR
12) VLR compares the two SRES ’s received from the HLR
and MS. If they are equal, the MS is authenticated
SRES=A3(ki,RAND) and kc=A8(ki,RAND)
TETRA Protocol
Protocol flow
1.MS sends its TMSI(TETRA Mobile subscriber Identity) to
BS (Normally a temporary Identity)
2) BS sends the TMSI to AC
3)AC looks up in the database and gets the Authentication
key ‘k’ using TMSI
4)AC generates a 80 bit RANDOM Seed (RS)
5)AC computes KS (session authentication key-128bits)
using K and RS
6)AC sends KS &RS to BS
7)BS generates a 80 bit random challenge called RAND1
and computes a 32 bit expected response called XRES1
TETRA Protocol
8.BS sends RAND1 and RS to the MS
9.MS computes KS using k & RS
10.Then MS computes RES1 using KS &
RAND1
11.MS sends RES1 to the BS
12.BS compares RES1 and XRES1.If they
are equal ,the MS is authenticated .
13.BS sends the results ‘R1’ of the
comparison to the MS
Authentication of the user
Protocol Flow
1)A Random number is chosen by AC called RS
1a)AC uses the K and RS to generate session key
(KS), using TA11 algorithm
2)AC sends the KS and RS to the base station.
3)BS generates a random number called RAND1.
4)BS computes expected response (XRES1) and
Derived Cypher key noting also TA12
Authentication of the user
5)BS sends RS and RAND1 to MS
6)MS using his own key (k) and RS
computes KS (session key) using TA11
also use TA12 computes RES1 and DCK1.
7)MS sends RES1 to BS.
8)BS computes XRES1 with RES1.
Comparison
• Challenge response Protocol
• Advantages
Simpler than Public key techniques
Less processing power required in the
hand set
• Disadvantages
Network has to maintain a Database of
secret keys of all the Mobile stations
supported by it
Comparison
• Public key Protocol
Advantages
• Network doesn’t has to share the secret keys
with MS
• Network doesn’t has to maintain a database of
secret keys of the Mobiles
Disadvantages
• Requires high processing power in the mobile
handsets to carry out the complex computations
in real time
Hybrid Protocol
• Combines the challenge-response
protocol with a Public key scheme.
• Here the AC also acts as the Certification
authority
• AC has the public key PAC and private
key SAC for a public key scheme.
• MS also has the public key pi and private
key si for a public key scheme.
End to End Encryption Requirements
• Authentication
• Key Management
• Encryption Synchronisation for multimedia
(e.g Video)
Secret key Methods
Advantages
• Less complex compared to public key
methods
• Less processing power required for
implementation
• Higher encryption rates than the public key
techniques
Disadvantages
• Difficult to manages keys
Public key Methods
Advantages
• Easy to manage keys
• Capable of providing Digital Signatures
Disadvantages
• More complex and time consuming
computations
• Not suitable for bulk encryption of user
data
Combined Secret-key and Public-key
Systems.
• Encryption and Decryption of User Data
(Private key technique)
• Session key Distribution (Public key
technique)
• Authentication (Public key technique)
Possible Implementation
•
•
•
•
Combined RSA and DES
Encryption and Decryption of Data (DES)
Session key distribution (RSA)
Authentication (RSA and MD5 Hash
Function)
• Combined Rabin’s Modular Square Root
(RMSR) and SAFER
One Way Hashing Functions
• A one way hash function, H(M), operates on an
arbitrary-length pre-image M and return a fixedlength value h
h=H(M) where h is of length m.
The pre-image should contain some kind of binary
representation of the length of the entire
message. This technique overcome a potential
security problem resulting from message with
different lengths possibly hashing to the same
value. (MD-Strengthening).
Characteristic of hash functions
• Given M. it is easy to compute h.
• Given h, it is hard to compute M such that
H(M)=h
• Given M, it is harder to find another
message, M’ such that H(M)=H(M’)
•
MD5 Hashing Algorithm
Variable Length
message
MD5
128 bit Message
Digest
Questions
1)
2)
3)
4)
5)
6)
7)
Describe the Channel Response Protocol for authentication and
key agreement.
Describe the Channel Response Protocol for GSM.
Describe the TETRA Protocol for authentication and key
agreement.
Describe the authentication for the user in mobile communications
and networking.
Describe the End to End encryption requirement, Secret Key
Methods, Public Key Methods and possible implementation.
Explain the principle of public/private key encryption. How can
such encryption schemes be used to authenticate a message and
check integrity.
Describe different types of data encryption standards.