Channel capacity and
Channel Coding
Channel Coding in Digital
Communication Systems
Channel Coding - Definition
• Information is transmitted through channels (eg. Wires, optical fibres and
even air)
• Error occurs in the transmitted signal due to the transmission in a nonideal channel. The signal power is also reduced.
• Noise exists in channels
• Noise signals corrupt the transmitted data
• Class of signal transformations designed to improve communication
performance by enabling the transmitted signals to better withstand
channel distortions such as noise, interference, and fading.
What is Coding?
• Coding is the conversion of information to another form for
some purpose.
• Source Coding : The purpose is lowering the redundancy in the
information. (e.g. ZIP, JPEG, MPEG2)
• Channel Coding : The purpose is to overcome channel noise.
Mathematical model of a channel
• Assume that our input to the channel is , and the output is
• Then the characteristics of the channel can be defined by its
conditional probability distribution
Information rate
• If the time rate at which source ‘s’ emits symbols is r(symbol, s) the
information rate ‘R’ of the source is given by
here,
=information rate
is entropy
is the rate at which symbols are generated.
and
Units:
•
is represented in average number of bits of information per second.it is
calculated as
=[ in symbols/sec]*[
in information bits/symbol]
• =information bits/second
Channel capacity and rate
• The maximum rate at which we can transmit a message with very low
probability of error and reconstruct the source message at the output is
called the capacity of the channel.
• Channel capacity is defined as the maximum possible value of the mutual
information
• We choose the best
to maximize
• For any rate
, we can transmit information with arbitrarily small
probability of error
Discrete memory less channels(DMC)
Definition Discrete channel :(X, p(y|x), Y)
A system consisting of an input alphabet X and output alphabet Y (finite sets)
and a probability transition matrix p(y|x) that expresses the probability of
observing the output symbol y given that we send the symbol x.
• The channel is said to be memoryless if the probability distribution of the output
depends only on the input at that time and is conditionally independent of previous
inputs or outputs.
• Definition “information” channel capacity of a discrete memoryless channel:
Bandwidth and channel capacity
• The transmission of data over the channel depends upon two parameters
, they are transmitted power and channel bandwidth.
• A communication channel cannot propagate a signal that contains a
frequency which is changing at a rate greater than bandwidth of the
channel.
• The greater the bandwidth available the higher the information carrying
capacity.
Shannon limit for channel capacity
• Definition:
The maximum data rate at which information can be transmitted over a given
communication channel, under given condition is referred to as the channel
capacity
C=Blog2(1+S/N) bps
• For a given level of noise, the data rate or capacity can be increased by
increasing either bandwidth or signal strength.
• If bandwidth B is increased , more noise is introduced to the system. Since
noise is assumed to be white noise
Thus as B increases, SNR
decreases.
Shannon limit for channel capacity
• Shannon’s theorem says that if
it is possible to transmit information without
any error and if noise present coding techniques are used to correct and detect
the errors.
• Noise less channel has infinite capacity i..e, if
then

then
• Infinite bandwidth channel has finite capacity. This is because as bandwidth
increases noise power also increases i.e.
• Due to this increase in noise power, signal to noise ratio decreases. Hence even
if  capacity does not approaches infinity.
Shannon limit for channel capacity
Channel Coding
• Channel encoding : The application of redundant symbols to correct data
errors.
• Modulation : Conversion of symbols to a waveform for transmission.
• Demodulation : Conversion of the waveform back to symbols, usually one
at a time.
• Decoding: Using the redundant symbols to correct errors.
Introduction: Channel Coding
Principle: Add redundancy to message sequence to minimize error rate
Transmitter
Source
Receiver
1 1 0 1
Channel
encoder
1 1 0 1
1 0 0 1 1 1 0 1
110 11 100
The channel encoder adds extra bits
(redundancy)to message bits. The encoded
signal is transmitted over the noisy channel
Sink
Channel
Channel
decoder
The channel decoder identifies the redundant
bits and uses them to detect and correct the
errors in the message if any.
ERROR CONTROL CODING
• The redundant bits in the message are called check bits. Errors can be detected and
corrected with the help of these bits.
• It is not possible to detect and correct all the errors in the message . Errors up to certain limit
can only be detected and corrected. There are two main methods used for error control coding
1) Forward error correction
The key idea of FEC is to transmit enough redundant data to allow receiver to recover from
errors all by itself. No sender retransmission required
2)
Error detection with retransmission (ARC technique)
• The check bits reduce the data rate through the channel.
• Tradeoffs between:
• Efficiency
• Reliability
• Encoding/Decoding complexity
Modulo 2
addition/XOR
0
0
0
0
1
1
1
0
1
1
1
0
Linear Block Codes
Key terms in error control coding
• Block length : The number of bits
the code.
after coding is called the block length of
• Code rate(r): the ratio of the message bits(k) and the encoder output bits “n” is
called code rate. Code rate is defined by
0<r<1
• Channel data rate: it is the bit rate at the output of the encoder. if the bit rate at
the input of the encoder is 𝑠 then the channel data rate will be 𝑜
𝑠
• Code efficiency: it is the ratio of message bits in a block to the transmitted bits
for that block by the encoder.
Efficiency=
Linear Block Codes: Example
Example: Find linear block code encoder G if code generator polynomial
g(x)=1+x+x3 for a (7, 4) code; n = total number of bits = 7, k = number of
information bits = 4, r = number of parity bits = n - k = 3
 x3 
p1  Re 
 1  x  110
3
1  x  x 
 x4 
p2  Re 
 x  x 2  011
3
1  x  x 


x5
p 3  Re 
 1  x  x 2  111 
3 
1  x  x 
 x6

p4  Re 
 1  x 2  101
3
1  x  x 
 xnk i1 
i
P  Remainder of 
, for i  1, 2, , k

 g ( x) 
1000 | 110 
0100 | 011
  [ I | P]
G
0010 | 111


0001
|
101


I is the identity matrix
P is the parity matrix
Linear Block Codes: Example

The Generator Polynomial can be used to determine the Generator
Matrix G that allows determination of parity bits for a given data bits of m
by multiplying as follows:
1000110
0100011
  [1011 | 100]
c  m.G  [1011] 
0010111


0001101


Data

Data Parity
Other combinations of m can be used to determine all other
possible code words
Block Codes: Linear Block Codes
Message
vector m
Generator
matrix G
Code
Vector C
Air
Parity
check
matrix HT
Code
Vector C
Transmitter
Receiver
Operations of the generator matrix and the parity check matrix

Consider a (7, 4) linear block code, given by G as
1000
 0100
G  
 0010

 0001
| 110 
| 011 

| 111 

| 101 
H
T
1011 | 100 
P  

 

1110
|
010
 
I
n

k

  0111 | 001 


T
For convenience, the code vector is expressed as
c  m | c p 
Where c p  mP is an (n-k)-bit parity check vector
Null
vector 0
Types of Errors
Single-bit error
Burst error
Parity Bit
Parity Bit Example
Even parity is more common
(11,7) Hamming Code Example
(11,7) Hamming Code Example
(11,7) Hamming Code Example
(11,7) Hamming Code Example
Cyclic block codes
• Cyclic codes are a subclass of linear block codes.
• Encoding and syndrome calculation are easily
performed using feedback shift-registers.
• Hence, relatively long block codes can be implemented with a
reasonable complexity.
• BCH and Reed-Solomon codes are cyclic codes.
Non-Systematic Cyclic code with Generator Polynomial
Example: Generating of (7,4) non-systematic cyclic code,
by generator polynomial G(p)=p3 +p+1
<- message
<- generator
<- encoded word
The same result obtained by Maple:
56
Non-Systematic Cyclic code with Generator Matrix
Non-Systematic Cyclic code with Generator Matrix
Systematic Cyclic code with Generator Polynomial
Systematic Cyclic code with Generator Matrix
https://electronicsdesk.com/cyclic-code.html
Division of the generated code by the generator polynomial
leaves no reminder
3
2
G ( p )  p  p  1

3
M
(
p
)

p
p

 p 7 4 M ( p )  p 6  p 4

n k
p 3  p 2  1  1
 p M ( p) / G ( p)  
 C ( p )

Q( p)
 n k
3
3
6
4
 p M ( p )  C( p )  p ( p  p )  1  p  p  1
Q( p )G ( p)  ( p  p  1)( p  p  1)  p  p  1
p3  p2  1
p3  p2  1 p6  p 4  1
p6  p5  p3
p5  p 4  p3  1
p 5  p 4 This
 p 2can be used for error
detection/correction as we inspect later
p  p 1
p3  p2  1
3
2
69