Coding Theory
Terms and Definition:
1. Data transmission
○ Process of sending digital or analog data over a communication medium to one or
more computing, network, communication or electronic devices.
○ It enables the transfer and communication of devices in a point-to-point,
point-to-multipoint and multipoint-to-multipoint environment.
■ A.k.a. digital transmission or digital communications.
2. Channel
○ Physical medium through which information is transmitted
○ Ex. Telephone lines, internet cables, fiber-optic lines and the atmosphere
3. Noise
○ Refers to interference caused by sunspots, lightning, meteor showers, poor typing,
and poor hearing among others
4. Noisy Channel
○ A communications channel in which the effects of random influences cannot be
dismissed
○ Communication channel that makes errors
■ Frames may be either damaged or lost completely.
5. Error-detecting codes
○ Codes that can detect errors, which they may or may not be able to correct ○
Concerned with improving the reliability of communication over noisy channels 6. Data
Compression
○ Efficient encoding of source information so that it takes as little space as possible
7. Cryptography
○ Involves security, privacy, or confidentiality of communication over an insecure
channel
8. Coding theory
○ Includes the study of error-detecting codes, data compression and cryptography ○
Goal: ​Improve the reliability of digital communication by devising methods that
enable the receiver to decide whether there have been errors during the
transmission, and if there are, to possibly recover the original message
9. Error detection
○ Identifying error made during the transmission of the message
10. Error correction
○ Recovering the original message despite the errors detected
11. Coding
○ A.k.a. Source coding, channel coding, or error control coding
○ Involves changing the message source (e.g. data terminal or the human voice) to a
suitable code for transmission through a channel
■ Ex. ASCII code
12. Source Encoder
○ Transforms the source output into a sequence of symbols
Every time information is transmitted on any channel, it may get corrupted by noise.
Goal of Technology:
● Construct an information system that results in:
○ Fast encoding of information
○ Easy transmission of encoded messages
○ Fast decoding of received messages
○ Correction of errors introduced in the channel, and
○ Maximum transfer of information per unit time
● The correction of errors is the primary goal in information technology.
Application of Coding Theory:
● Transmission of pictures from distant space
● Maintaining the quality of sound in CDs and telephone lines, computer networks, wireless
communication, and most recently, quantum error-correction
Famous Contributors to Coding Theory:
1. Claude Shannon
○ Credited for the idea of using redundancy to add reliability to information ■
People are conditioned against redundancy (e.x. Data Compression)
○ Shannon’s 1948 paper “A Mathematical Theory of Communication” marks the birth
of a new subject called ​information theory
■ Shannon established the theoretical foundation of the subject
2. Richard Hamming
○ One of the first to actually construct and implement error-correcting codes ○ His
research included work in the areas of coding theory, numerical methods, statistics,
and digital filtering
Examples of Error-Detecting Codes
1. Sending Telegrams
○ The end of the message would contain the number of words sent for the receiver
to know when a word got lost
2. Parity Check
○ Simplest form of error-detection
○ Parity
■ A single bitis appended to a bit string
● Odd parity or even parity
3. Repetition
○ Simplest possible error-correcting code
4. Hamming Distance
○ The number of positions in which strings x and y differ
Crossword Puzzle ​(page 148)
Across:
3 Detection
6 Cryptography
7 Shannon
8 Message
9 Repetition
10 Hamming
Down:
1 Compression
2 Parity
4 Correction
5 Channel
7 Source