Energy-Optimal Information Transmission
over
Wireless Channels
by
Cem Erin
Submitted to the Department of Mechanical Engineering
in partial fulfillment of the requirements for the degree of
Master of Science
at the
MASSACHUSETTS INSTITUTE OF TECHNOLOGY
January 2001
@ 2000 Massachusetts Institute of Technology.
All rights reserved.
Author ................
Depar ment of Mechanical Engineering
Vi
May 5, 2000
Certified by..............
Professor Haruhiko H. Asada
Ford Professor of Mechanical Engineering
Thesis Supervisor
A ccepted by .................................................
Professor Ain A. Sonin
Graduate Students
Chairman, Department Committee onMASSACHUSETTS
INSTITUTE
OF TECHNOLOGY
SEP 2 0 2000
LIBRARIES
BARKER
Energy-Optimal Information Transmission
over
Wireless Channels
by
Cem Erin
Submitted to the Department of Mechanical Engineering
on May 5, 2000, in partial fulfillment of the
requirements for the degree of
Master of Science
Abstract
Coding and modulation algorithms optimizing the energy efficiency in information
transmission are presented. Portable communication devices transmit nonuniformprobability messages using nonuniform-energy signal sequences. Hence, energy-optimal
information transmission can be achieved by representing higher-probability messages
by lower-energy signal sequences. In this thesis, first, for low data rate transmission,
this energy-optimal information transmission problem is formulated as a data compression problem and a memoryless data compression algorithm, namely Minimum
Energy Coding (ME Coding), optimizing the energy efficiency in information transmission over noiseless channels is presented. Then, extensions of ME Coding are
investigated for improving the energy efficiency at low data rates. Next, for high data
rate transmission, the energy-optimal information transmission problem is formulated as a modulation system design problem and an optimization algorithm, namely
joint code-constellation design (JCCD), is proposed as a method for designing optimal source coded modulation systems. Then, a sub-optimal modulation algorithm,
namely Minimum Energy Modulation (ME Modulation), is proposed as an information transmission technique that achieves energy efficient information transmission
at a reduced system complexity. Extensions of ME Modulation are investigated for
improving the energy efficiency at high data rates. Next, the issue of energy efficient
information transmission in the presence of channel noise is investigated. For powerlimited channels, low-rate energy-optimal information transmission is achieved via
the use of error correcting codes. For bandwidth-limited channels, high-rate energyoptimal information transmission is achieved via the use of coded modulation schemes
combined with multilevel and multidimensional signal constellations. In conclusion,
energy-optimal information transmission can be achieved through proper coding and
modulation.
Thesis Supervisor: Professor Haruhiko H. Asada
Title: Ford Professor of Mechanical Engineering
2
Contents
1
2
3
9
Introduction
1.1
Objective and Scope . . . . . .
. . . . . . . . . . . . . . . . . . . .
9
1.2
Background . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
10
1.3
Approach
. . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
11
1.4
Overview.
. . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . .
12
13
Problem Formulation
2.1
Signal Constellations . . . . . .
. . . . . . . . . . . . . . . . . . . .
13
2.2
Practical Signal Constellations .
. . . . . . . . . . . . . . . . . . . .
16
2.3
Average Energy per Message . .
. . . . . . . . . . . . . . . . . . . .
17
19
Energy-Optimal Data Compression
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
19
. . . . . . . . . . . . . . . . . . . . . . . .
21
Optimality Bound to ME Coding . . . . . . . . . . . . . . . . . . . .
23
3.1
M E Coding
3.2
Fixed-Length ME Coding
3.3
25
4 Extensions of ME Coding
4.1
Extension and Concatenation
. . . . . . . . . . . . . . . . . . . .
25
4.2
Mechanisms of Concatenation
. . . . . . . . . . . . . . . . . . . .
26
4.3
Concatenation Theory . . . . . . . . . . . . . . . . . . . . . . . . . .
29
5 Energy-Optimal Source Coded Modulation
34
5.1
Joint Code-Constellation Design (JCCD) . . . . . . . . . . . . . . . .
34
5.2
Optimization Procedure
. . . . . . . . . . . . . . . . . . . . . . . . .
35
3
6
5.3
JCCD Methodology Flow Chart . . . . . . . . . . . . . . . . . . . . .
37
5.4
Optimality Condition for 2D Constellations
39
. . . . . . . . . . . . . .
ME Modulation
41
6.1
ME Modulation Algorithm .............................
41
6.2
Optimality of ME Modulation . . . . . . . . . . . . . . . . . . . . . .
43
7 Extensions of ME Modulation
8
7.1
Extension and Concatenation
. . . . . . . . . . . . . . . . . . . . . .
46
7.2
Mechanisms of Concatenation . . . . . . . . . . . . . . . . . . . . . .
49
Energy-Efficient, Low-Rate, and Reliable Information Transmission
over Power-Limited Wireless Channels
54
8.1
Setting the Scene . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
54
8.1.1
The Channel
. . . . . .. . . . . . . . . . . . . . . . . . . . . .
55
8.1.2
The Decoder and Error Probability . . . . . . . . . . . . . . .
57
8.1.3
The Main Energy Efficient Error Control Problem . . . . . . .
60
8.2
9
46
Energy Efficient Block Codes
. . . . . . . . . . . . . . . . . . . . . .
8.2.1
Minimum Energy Hamming Codes
8.2.2
Minimum Energy Golay Codes
62
. . . . . . . . . . . . . . .
63
. . . . . . . . . . . . . . . . .
67
8.2.3
Minimum Energy Reed-Muller Codes . . . . . . . . . . . . . .
71
8.2.4
Implementation of Block Codes . . . . . . . . . . . . . . . . .
74
8.3
Minimum Energy Convolutional Codes . . . . . . . . . . . . . . . . .
75
8.4
Minimum Energy Burst-Error-Correcting Codes . . . . . . . . . . . .
76
Energy-Efficient, High-Rate, and Reliable Information Transmission
over Bandwidth-Limited Wireless Channels
78
9.1
Minimum Energy Multilevel Coded Modulation
. . . . . . . . . . . .
78
9.2
Minimum Energy Block-Coded Modulation . . . . . . . . . . . . . . .
79
9.3
Minimum Energy Trellis-Coded Modulation
. . . . . . . . . . . . . .
80
9.4
Minimum Energy Lattice Codes . . . . . . . . . . . . . . . . . . . . .
82
9.5
Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
83
4
10 Simulation Results
85
11 Conclusions
88
5
List of Figures
1-1
Portable Communication Device . . . . . . . . . . . . . . . . . . .
10
2-1
8-QAM Constellation . . . . . . . . . . . . . . . . . . . . . . . . .
15
2-2
Practical Signal Constellations . . . . . . . . . . . . . . . . . . . .
16
3-1
Lower Bound to ME Coding . . . . . . . . . . . . . . . . . . . . .
24
4-1
Direct Concatenation. (a) ME Coding, (b) Direct Concatenation.
26
4-2
ME Coding Example
4-3
Direct Concatenation Example . . . . . . . . . . . . .
28
4-4
Optimal Codebook for Direct Concatenation . . . . .
29
4-5
Probabilistic Optimization for Direct Concatenation .
30
4-6
Extension of ME Coding . . . . . . . . . . . . . . . .
31
4-7
Relations among different concatenation orders . . . .
33
5-1
JCCD Problem . . . . . . . . . . . . . . . . . . . . .
35
5-2
Optimization Procedure
. . . . . . . . . . . . . . . .
36
5-3
Design Methodology Flow Chart . . . . . . . . . . . .
38
6-1
16-QAM Constellation ..................
42
7-1
Direct Concatenation. (a) ME Modulation, (b) Direct Concatenation.
47
7-2
ME Modulation Example . . . . . . . . . . . . . . . .
49
7-3
Direct Concatenation Example . . . . . . . . . . . . .
50
7-4
Direct Concatenation Codebook . . . . . . . . . . . .
51
7-5
Optimal Codebook for Direct Concatenation . . . . .
52
. . . . . . . . . . . . . .
6
27
7-6
Probabilistic Optimization for Direct Concatenation .
53
7-7
Extension of ME Modulation
. . . . . . . . . . . . .
53
8-1
Block Diagram of a Communications System . . . . .
55
8-2
Discrete Memoryless Channel . . . . . . . . . . . . .
55
8-3
Binary Symmetric Channel . . . . . . . . . . . . . . .
56
8-4
Decoder .....
.........................
58
8-5
The Main Energy Efficient Error Correction Problem
61
8-6
Single Error Correcting Hamming Codes . . . . . . .
63
8-7
Single Error Corre cting and Double Error Detecting Hamming Codes
66
8-8
EPROM Encoder
74
de...........................
Logic Circuit Enco der . . . . . . . . . . . . . . . . . . . . . .
75
8-10 Convolutional Enc :)der . . . . . . . . . . . . . . . . . . . . . .
76
8-11 Interleaving . . . . . . . . . . . . . . . . . . . . . . . . . . . .
77
8-9
9-1
Block-Coded Modulation
80
9-2
Trellis Coded Modulation
81
9-3
Trellis Diagram . . . . . .
81
9-4
Coded Modulation System
83
10-1 Finger Ring Sensor . . . . . . . . . . . . . . . . . . . . . . . . . . . .
85
10-2 Ring Sensor Plethysmograph . . . . . . . . . . . . . . . . . . . . . . .
86
. . . . . . . . . .
87
10-3 Battery Energy: (1) ASCII Code, (2) ME Protocol
7
List of Tables
3.1
Minimum Codebook .......
6.1
16-QAM ME Modulation Example
8.1
ME Hamming Code ..................................
8
...........................
22
...................
43
64
Chapter 1
Introduction
1.1
Objective and Scope
Battery life optimization is a fundamental problem in the portable communication
device industry. In portable communication devices, the trend is towards smaller
and lighter devices offering increasingly demanding applications. This trend imposes
stringent constraints on the available energy resources thereby resulting in rather
impractical battery life times. Since the progress in battery technology is rather
slow and since frequent battery re-charging is rather undesirable from the consumers'
point of view, future market success of portable communication devices will heavily
depend on new information transmission technologies that will make efficient use of
the available energy resources.
An effort to develop new energy efficient information transmission technologies
requires understanding the operation of portable communication devices. Figure 1-1
shows a simplified block diagram of a portable communication device. In this figure, the two main components of a portable device are shown to be the information
source and the transmitter [1, 2].
The information source selects messages from a
set of possible messages to be delivered to the transmitter. The transmitter, which
usually includes an encoder and a modulator, generates a unique signal sequence for
each unique message. These signal sequences are then transmitted over the wireless
channel to a receiving device. In many portable communication devices, transmit9
Transmitter
Wireless Channel
Source
BtSkal.,
Conventional Approa
New Approach
Figure 1-1: Portable Communication Device
ted energy in the signal sequences forms a major fraction of the total device energy
consumption. Hence, the objective of this thesis is to develop new information transmission technologies that minimize the transmitted energy in portable communication
devices.
1.2
Background
Existing approaches to transmitted energy optimization can be grouped into two
classes: (i) adaptive techniques, and (ii) algorithmic techniques.
In the adaptive class, transmitted energy optimization is achieved by varying a
set of system parameters as a function of the system states. Some specific techniques include transmitter power level adaptation [3], error control adaptation [4], or
a combination of the two [5] for varying channel conditions.
In the algorithmic class, recognizing that the transmission of higher-energy signals consumes more battery energy compared to the transmission of lower-energy
signals, transmitted energy optimization is accomplished by designing modulators
that transmit lower energy signals more frequently. Figure 1-1 shows the algorithmic
approach, labeled as the conventional approach, as a mapping between bit sequences
and transmitted signal sequences. One such algorithmic technique first divides the
bit sequence produced by the encoder into nonuniform-length words according to a
prefix code thereby introducing a probability distribution on the prefix words, and
10
then maps the prefix words to the transmitted signals such that shorter words having
higher probabilities are assigned to lower energy signal sequences [6, 7].
Although
this approach promises an improvement in energy efficiency, the fact that the number of data bits transmitted per unit time becomes a random variable leads to system problems such as buffering and delay that may outweigh any possible improvement in energy efficiency. Another technique is to divide the signal constellation
into sub-constellations and provide a mapping between the bit sequences and signals
that makes more frequent use of inner sub-constellations [8-11]. This approach also
promises an improvement in energy efficiency; however, this improvement is achieved
with a significant increase in system complexity due to the complicated nature of the
bit sequence-to-signal mappings. Hence, a simple and practical coding and modulation algorithm that would optimize the transmitted energy with a minimum increase
in system complexity is still in demand.
1.3
Approach
This thesis introduces a new algorithmic approach to transmitted energy optimization
that does not result in system problems and does not require a significant increase
in system complexity. Furthermore, depending on the system, the new approach
promises significant energy efficiency improvements over the previously proposed approaches. Recognizing that the very basic function of a portable communication
device is to transmit unique messages using unique signal sequences and that different messages occur with different probabilities and different signal sequences are
characterized by different energies, energy consumption in information transmission is
optimized by mapping higher probability messages to lower energy signal sequences.
Based on this new idea, new coding and modulation algorithms optimizing the energy
efficiency in information transmission are presented.
11
1.4
Overview
The thesis starts with an introduction to signal constellations in Chapter 2. For low
data rate information transmission systems using On-Off Keying signal constellations,
an energy-optimal data compression algorithm, Minimum Energy Coding, is proposed
in Chapter 3. Chapter 4 investigates the extensions of this algorithm for improving
the energy efficiency. For high data rate information transmission systems using generalized multilevel signal constellations, Chapter 5 proposes joint code-constellation
design (JCCD) as an optimization algorithm that yields energy-optimal source coded
modulation systems. Since JCCD generally results in impractical signal constellations that require a significant increase in system complexity, a sub-optimal practical
source coded modulation algorithm is proposed in Chapter 6. Chapter 7 investigates
the extensions of this algorithm for improving the energy efficiency. For energy efficient information transmission over noisy power-limited channels, Chapter 8 proposes
the use of error correcting codes. For energy efficient information transmission over
noisy bandwidth-limited channels, Chapter 9 proposes the use of coded modulation
schemes. Chapter 10 demonstrates the attainable energy savings via the use of ME
Coding over the conventional ASCII code. The thesis closes with some conclusions
in Chapter 11.
12
Chapter 2
Problem Formulation
In wireless communications, information transmission is accomplished via the use of
signal sequences selected from a given signal constellation. In an effort to formulate
the energy efficient information transmission problem, this chapter first provides an
introduction to signal constellations and their accompanying performance metrics.
Then, practical signal constellations that are frequently used in wireless communications are presented. Finally, a new performance metric for evaluating the energy
performance of a portable communication device is introduced and the energy efficient
information transmission problem is formulated as a coding and modulation problem.
2.1
Signal Constellations
A signal constellation is a set of signals C = {c1 , c2 ,..., CM} that generates a Ddimensional space W(C) C L 2 [12]. The key parameters of finite signal constellations
include average error probability per signal, normalized rate, and peak signal energy.
Following is a list of definitions for these parameters.
Average Error Probability per Signal (Pr(error)). The average error probability per signal Pr(error)of a signal constellation using an optimal detector is
13
evaluated using the union bound estimate. The union bound estimate is given as
Pr(error)= E[Pr(errorjci)]
d2 .
Pr(errorci) ~ KminQ(
),
where the expectation is over the signals ci in the constellation,
(2.1)
Q
is the Gaussian
error function, dmin is the minimum distance between ci and cj, i # j, and Kmin is
the number of signals cj
$
ci that are at a minimum distance dmin from ci. The union
bound estimate is valid only (i) if the second nearest neighbors are at a significantly
greater distance than the first nearest neighbors and (ii) if there are not many nearest
neighbors. If any of these assumptions are violated, then further terms should be
included in the estimate. Equation (2.1) suggests that the average error probability
per signal of a signal constellation can be constrained by constraining the minimum
distance dmin of the constellation.
Normalized Rate (p). The normalized rate of a signal constellation with size
M and dimension D is defined to be
2
p
= -
D
log2 M
(b/2D),
(2.2)
which is equal to its spectral efficiency measured in (b/s)/Hz. Physically, spectral
efficiency measures how fast information is being transmitted. For example, if a QAM
signal constellation of dimension D = 2 has M = 8 sinusoidal signals each at a given
frequency and lasting 1 second, then transmission of each of the 8 signals results in a
transmission rate of
10928
= 3 bits/s/Hz. If a signal constellation has 16 signals and
all the other constellation parameters are kept the same, then the transmission rate
becomes log2 16 = 4 bits/s/Hz. Hence, the more the number of signals in the signal
constellation, the higher the transmission rate.
14
Q
2
-2
1
-1
2
-2
Figure 2-1: 8-QAM Constellation
Peak Signal Energy. The peak signal energy of a signal constellation is equal
to the energy of its maximum energy signal
Ep = max1ci
1
2
= max ej.
(2.3)
1
Since signals mostly take the form of electrical current or electrical voltage and
since energy of a signal is proportional to its current amplitude or voltage amplitude squared, ||c,112 represents the energy of the signal.
Figure 2-1 illustrates the above terminology on an 8-QAM signal constellation. In
this figure, it is seen that the signal constellation is D = 2 dimensional, where the 2
dimensions are generated by the in-phase (I) and quadrature (Q) components of the
harmonic signal (the constellation axis). The number of signals in the constellation
is M = 8. The minimum distance, dmin, of the signal constellation is dmin = v/2 as
seen by the Euclidean distance between the signal points. The normalized rate of the
signal constellation is p = (2/2)1og28 = 3 b/2D. The peak signal energy is 4 units as
seen by the squared Euclidean distance of the signal points on the coordinate axes
from the origin.
15
2.2
Practical Signal Constellations
In wireless communications, information transmission is accomplished via some information carrying signals selected from a practical signal constellation. Not all signal
constellations are practical. The practicality of a signal constellation is determined by
the ease with which the signals in the constellation are synthesized. The problem of
designing practical signal constellations has been investigated in the 1960s and early
1970s [6,13-22]. Some of the practical signal constellations that have been designed
until now are shown in Figure 2-2. In this figure, the in-phase axis, I, represents a
Q
PAM
OOK
0
00
*
0
*
S
PSK
0
000
QAM
Hexagonal
Multidimensional
Figure 2-2: Practical Signal Constellations
sine wave, and the quadrature axis,
Q, represents
a cosine wave. In On-Off Keying
Modulation (OOK), the two transmitted signals are the Off-signal, which is located
at the origin, and the On-signal, which is located on the positive I axis. In Pulse
Amplitude Modulation (PAM), the information is carried in the amplitude of the
in-phase signal. In PSK Modulation, however, the information carrying signal is a
16
combination of in-phase and quadrature components, has an amplitude that is fixed,
but is phase-shifted. In QAM Modulation, the signals are placed on a regular grid.
The transmitted signals are linear superpositions of integral multiples of in-phase and
quadrature signals. In hexagonal signal constellations, the signals are placed on the
corners and centers of hexagons. Finally, in multidimensional signal constellations,
dimensionality of the signal constellation is increased by extending the time or the
frequency axis.
2.3
Average Energy per Message
Formulation of the energy efficient information transmission problem is derived from
the new idea that energy efficiency can be optimized by representing higher probability
messages using lower energy signal sequences.
Based on this new idea, a natural
parameter for measuring the energy efficiency of a wireless device becomes average
transmitted energy per message or, in short, average energy per message B. The
average energy per message can be expressed as
q
E
Piei,
=
(2.4)
i=1
where P is the probability of the i-th message and ei is the energy of the i-th signal
sequence assigned to the i-th message given as
L
ej
=
z
|sijII 2 ,
(2.5)
j=1
where |sIj | 2 is the energy of the j-th signal in the i-th signal sequence. Hence, based
on the new performance metric being average energy per message, the energy-optimal
information transmission problem can be stated as follows.
Problem Statement.
Given a q-message source S with a known probability
distribution P, what is the optimal coding and modulation algorithm that minimizes
the average energy per message?
17
As an immediate response to this question, looking at equation (2.4), the optimal
coding and modulation algorithm would be one that
" Chooses the signal sequences of lower energy to minimize ej, and
" Assigns the higher probability messages to lower energy signal sequences to
minimize E91 Piej.
The next chapter investigates this problem for low rate information transmission
systems using On-Off Keying Modulation, where each high bit (i.e., a 1) results in an
On-signal and each low bit (i.e., a 0) results in an Off-signal.
18
Chapter 3
Energy-Optimal Data Compression
In an effort to achieve energy-optimal low-rate information transmission, this chapter
introduces Minimum Energy Coding combined with On-Off Keying Modulation as
the energy-optimal data compression algorithm. Next, the optimality of ME Coding
is proved. Then, a special practical case of ME Coding, fixed-length ME Coding, is
described. Finally, the parameters determining the optimal performance and a lower
bound on optimal performance are introduced.
3.1
ME Coding
In On-Off Keying Modulation systems, battery energy is consumed only when high
bits are transmitted, and virtually no energy is consumed when low bits are transmitted. Recognizing this fact, ME Coding is a data compression algorithm that aims to
optimize the energy efficiency in information transmission by minimizing the average
number of high bits used in coding the information source. ME Coding is generated
through two distinct steps: Codebook Optimality and Probabilistic Optimality. The
former is to determine a set of codewords, termed a codebook, that has fewest high
bits, and the latter is to assign codewords having less high bits to messages with
higher probability. The following two theorems underpin the theory of ME Coding.
19
Theorem 1: Probabilistic Optimality. Let S be a q-message i.i.d. source with
message probabilities
P = {P1
P2 > ...
Pq-1
q
P}.
(3.1)
Given a codebook of q codewords, each of which contains ni high bits, 1 < i < q, the
optimal code that minimizes the average number of high bits fi = D
niPi is given
by assigning the codewords to messages such that
n, :5 n2
<-
--- < nq-i < nq.
(3.2)
Proof. This theorem can be proved in the same way as the optimality of Huffman
Coding by replacing the codeword length by the number of high bits. See Appendix.
The major difference from Huffman Coding stems from the codebook used. Let W
be a codebook of q codewords arranged in the ascending number of high bits involved
in each codeword. The code C(W, S) assigns the q codewords in codebook W to q
messages in source S in such a manner that equations (3.1) and (3.2) determine the
minimum A for the given W. Hence, the minimum of ii varies depending on properties
of the codebook used. The remaining question is how to obtain a codebook that
provides the overall minimum of ft. Let WO = {wi, W 2 ,
... ,
the entire set of codewords that are usable for ME Coding; q
Wi, ---, Wqo-1, Wqa}
be
q. < +oo. We first
number each codeword in the ascending number of high bits involved in it, namely,
ni
n2 5
...
< ni < ... 5 no-1
nO,
where ni is the number of high bits in
codeword wi. Then, we generate the codebook used for ME Coding by taking the
first q codewords having the least high bits.
Definition: Minimum Codebook. Let the codewords of the whole codeword
set Wo
=
{w 1 , w 2 ,
... ,
w,
... , Wqo}
be numbered in the ascending number of high bits
involved in each codeword, n, 5 n 2 -<... < n
20
5 ... 5 nqo. A minimum codebook of
q codewords, Wmin, consists of the first q codewords of the whole codeword set W.
Wmin = {Wi, W2,
(3.3)
---, Wq-1, Wq} C Wo.
Theorem 2: Codebook Optimality. Let S be a q-message source and W be a
codebook of q-codewords taken from the usable codewords set Wo
=
{W1,
W 2,
...
, Wqo.-,
wqo},
q. < +oo; W C W0 . The necessary condition for a code C(W, S) to provide the
q
minimum average number of high bits ft =
Z
niPi is that the codebook W is a
minimum codebook of Wo.
Proof. See Appendix.
Combining Theorems 1 and 2, the following Corollary is easily obtained.
Corollary 1: ME Coding. Let S be a source and W. be the entire set of usable
codewords, q < q.
high bits f =
Z=
The optimal code C(W, S) that minimizes the average number of
1 niPi is given by
(i) Using the minimum codebook Wmin of W. for the codebook; W
=
Wmin C W, and
(ii) Assigning the q codewords of Wmin in the ascending order of number of high bits
to the q messages in the descending order of probabilities.
This optimal coding is referred to as ME Coding.
Consider a special case where Pi = P2 =
...
= Pq-i = Pq. The average number
of high bits in this case equals the total number of high bits in the codebook divided
by q. Therefore, the following Corollary holds.
Corollary 2. A minimum codebook contains the minimum number of high bits;
Min E_1 ni.
wo
3.2
Fixed-Length ME Coding
Of practical importance is ME Coding with fixed-length codewords. As shown by
Theorem 2 and Corollary 1, use of a minimum codebook is the necessary condition
for obtaining ME Coding. Thus, we must first generate a minimum codebook. For Lbit fixed-length codewords, Table 3.1 shows the entire set of usable codewords sorted
21
by the number of high bits; WO = {w 1 ,
... , Wq
0
}.
Note that the total number of usable
Table 3.1: Minimum Codebook
Codeword
W1
Number of
L
Codewords
W2 - WL+
WL+2 '....
L'
L)
Wq
(L)
(L
L1
Codeword
Pattern
Message
(L'
/
W2 L
L
NE-
m1
m2
...
codewords is qo = 2L. The first column has only
high bit, the second column has
L
=
L
(0
m]
=1 codeword with zero
L codewords with one high bit, and so on.
All the codewords are numbered from 1 to 2 L in the ascending order of the number
of high bits. The last codeword,
W 2 L,
consists of all high bits. Selecting the first q
codewords from this exhaustive list of 2 L codewords yields the minimum codebook
needed for ME Coding.
It is clear from Table 3.1 that, as the codeword length L becomes longer, the total
number of high bits involved in the first q codewords decreases. An extreme case
is unary coding, where L is long enough to express all q messages using codewords
with at most one high bit per codeword, e.g., 00010000. Since a longer codeword
takes a longer transmission time if the bit period
tb
is kept constant, transmission
rate decreases. Hence, this constitutes the trade-off between energy efficiency and
transmission rate.
22
3.3
Optimality Bound to ME Coding
In Sections 3.1 and 3.2, we proved the optimality of ME Coding and introduced
fixed-length ME Coding. Since a closed form solution for the optimal performance of
fixed-length ME Coding is not available, we derive the following optimality bound
k(Hk-A)
Bk
(3.4)
<1,
where Hk is the source entropy and Bk is the codebook capacity defined as
q
Bk = Zk-",
(3.5)
i=1
where k is an arbitrary constant greater than 1.
The derivation of this bound is
included in the Appendix.
Inequality (3.4) suggests that source entropy Hk and codebook capacity Bk provide
a lower bound on the optimal energy performance.
As the entropy Hk decreases
and the codebook capacity Bk increases, the bound to average number of high bits
becomes lower.
For a codebook consisting of q codewords of fixed-length L, the
maximum codebook capacity Max(Bk) can be obtained from Table 3.1.
Namely,
the minimum codebook consisting of the first q codewords in the table provides the
maximum codebook capacity given by
Max(Bk)=
1
L
()
I
(
L
where a and b are positive integers such that q =
1
=O
(
L
+ bkal
(3.6)
+ b. Note that any
exchange of the first q codewords with the other (2 - q) codewords decreases the value
of codebook capacity unless exchanging codewords within the (a+1)st column. Note
also that, as the codeword length L increases, maximum codebook capacity Max(Bk)
tends to increase, hence a longer codeword tends to lower the average number of high
23
n
x
-
Exact
Estimate
S
0.5-
0.5---------
---------
I
I
I
I
3
4
5
6
S2
-"
L
Figure 3-1: Lower Bound to ME Coding
bits and the energy consumption.
Figure 3-1 illustrates the compression performance of ME Coding in comparison with the lower bound in inequality (3.4). In this figure, the average number
of high bits per message is plotted against the codeword length for the ME Coding of two distinct sources for k = 10. The first source S1 is a 6-message source
having a probability distribution of P = {0.49, 0.29, 0.09, 0.07, 0.03, 0.03}, while the
second source S2 is again a 6-message source having a probability distribution of
P 2 = {0.35, 0.19,0.17,0.15,0.08, 0.06}. The entropies of the two sources are H = 1.71
bits and H2 = 2.13 bits, respectively. The exact ME Coding performance and the
estimated ME Coding performance provided by the bound are shown in Figure 3-1.
It is seen from both the exact performance and the estimate that higher entropy
sources result in higher average number of high bits per message compared to lower
entropy sources. Furthermore, it is also seen that as the ME Codebook codeword
length increases the average number of high bits per message decreases. Finally, it
is also seen that the lower bound provides a fairly good estimate of the exact ME
Coding performance. For example, for the low entropy source, at L = 3, the bound
is off by 14 %. In the next section, we will consider extensions of ME Coding for
improving the energy efficiency.
24
Chapter 4
Extensions of ME Coding
Chapter 3 introduced ME Coding as the optimal, memoryless data compression algorithm for energy-efficient information transmission. Given this optimal memoryless coding algorithm, this section takes a step further and introduces a technique,
termed concatenationor extensions of ME Coding, that improves the energy-efficiency
of memoryless ME Coding by using a simple memory mechanism.
4.1
Extension and Concatenation
Definition: Concatenation. Let S' be an i.i.d. source with known statistics, coded
using an L-bit fixed-length codebook W 1 . The p-th concatenation of S' and W' is a
data compression algorithm that encodes the p-th extension of the source alphabet,
SP, using a pL-bit fixed-length codebook, WP.
Concatenation uses memory. One drawback of using memory is that it increases
delay, which might be undesirable in some applications. Another issue is that concatenation becomes impractical as the concatenation order increases. Since the encoding
is in the form of a look-up table, as the concatenation order increases, the source
alphabet becomes very large resulting in impractical memory requirements and computational complexity.
A special case of concatenation is direct concatenation. As illustrated in Figure 41, a new codeword of length pL is created by directly concatenating the p codewords
25
used in the original coding to form a single codeword of length pL (p=2 in Figure 4-1).
Note that the bit stream of this elongated codeword is exactly the same as the original
Pi
P
iP
Jii
Pi Pi
(a)
(b)
Figure 4-1: Direct Concatenation. (a) ME Coding, (b) Direct Concatenation.
bit stream. Direct concatenation results in codewords having a total number of high
bits that is equal to the sum of the number of high bits in the original codewords.
These newly formed codewords are assigned to messages having probabilities that are
the product of the original message probabilities (source is i.i.d.). Higher-order direct
concatenation of fixed-length ME Coding is performed in a similar manner. We will
prove in Section 4.3 that direct concatenation results in energy efficiencies exactly
equal to that of ME Coding. Next, we will utilize direct concatenation to elucidate
the performance improvement mechanism in concatenation.
4.2
Mechanisms of Concatenation
Concatenation intertwines two separate mechanisms that would yield improved energy
efficiency: (i) variation in the number of high bits due to codeword elongation from L
to pL, (ii) probability distribution alteration due to source extension from S' to SP.
In this section, we will show, via an example, that direct concatenation, which results
in identical energy performance as the original non-concatenated ME Code, is usually
not optimal. Hence, energy efficiency improvement is possible through the sequential
steps of codebook and probabilistic optimization over direct concatenation.
Consider a 6-message source S' having the probability distribution PI = {45, 44, 5,4, 1, 1}
26
W0
Minimum
Codebook
Unused
Codewords
Codeword
W1
000
w2
001
w3
010
W4
10
w5
01
w6
10
w7
11 0
w8
111
4
1
1
4I
14
O4
5
1
2
j
Figure 4-2: ME Coding Example
in percentages and coded using 3-bit codewords according to the ME Coding algorithm. Figure 4-2 shows the resulting ME Code.
The first column lists the entire usable codewords arranged in the ascending number of high bits. The first q = 6 codewords form the minimum codebook, leaving two
codewords w7 and w8 unused. Each square block representing a codeword indicates
the number of high bits ni at the top left corner and the probability P of the message assigned to that codeword at the bottom right corner. The second order direct
concatenation of this ME Code can be generated by arranging the same ME Code in
a row and combining two codewords: one taken from the column and the other from
the row, as shown by the shaded block in Figure 4-2. Repeating this process for all
pairs of codewords results in the second order direct concatenation in Figure 4-3. For
higher order direct concatenation, we would have cubes and higher order geometrical
figures. In the following two steps, we investigate the codebook and probabilistic optimality of direct concatenation for this given example. A loss in optimality of either
of the two will warrant energy efficiency improvement over the original ME Code.
Step 1: Codebook Optimization. For a code C(W, S) to be optimal, the necessary
27
O
1
l1
1
19.80
2
19.36
2
1
2
2.20
2
2
2
2.25
1
1.80
1.76
2
0.45
3
2
3
[.4
2
2I3
2
0.45
0.45
2
1.76
3
2.20
0 44
3
0.44
3
4
0.25
2
0.20
3
0.05
3
0.05
3
4
2
3
3
3
4
'
0.20
0.16
3
0.04
4
0.01
4
0.05
3
4
4
3
0.44
0.44
11
2.25 11.801
19.80
3
0.05
0.04
0.04
2
3
3
3
3
4
4
4
0.04
0.01
4
0.01
4
5
5
0.01
5
4
5
4
5
E]
i
Figure 4-3: Direct Concatenation Example
condition is that the codebook W is a minimum codebook (Theorem 2). Hence, we
check if the direct concatenation codebook is a minimum codebook. The codebook
for direct concatenation is represented by the 6-by-6 matrix enclosed by the thick
line in Figure 4-3.
It is seen that direct concatenation uses codewords having a
maximum of 4 high bits at the bottom right corner of the thick line, while the set of
unused codewords contains codewords with {2, 2,3, 3} high bits indicated by the two
solid round-edged rectangles in Figure 4-4. Thus, the codebook is not optimal and the
unused codewords having less high bits should replace the ones with more high bits. In
Figure 4-4, messages originally assigned to 4 high bit codewords (i.e., P=0.01) move
to the newly available codewords having {2, 2,3, 3} high bits to achieve codebook
optimality. Note, however, that the global code optimality is not guaranteed until
the message-to-codeword assignment is optimized.
Step 2: Probabilistic Optimization. The optimal coding proven in Theorem 1 is
now applied to the minimum codebook in Figure 4-4. The assignments that don't
conform with Theorem 1 are swapped in Figure 4-5. For example, the message having a probability of 19.36% was originally assigned to a 2 high bit codeword in the
28
1
1
112
2
2
3
1
222
1
2
2
3
1
2
22
3(3 3
2
3
3
3
'4
2
3
3
3
',4
41
4
5
0.
.01
3
3
4
4
4
5
4:E4
5
5 15
3_4
14
3
3
4
4
'1 4
5
6
Figure 4-4: Optimal Codebook for Direct Concatenation
minimal codebook (w2 - W2 codeword), while a message having a probability of 1.80%
was assigned to a 1 high bit codeword (wi
-
w4 codeword). Thus, the two messages
are swapped to reduce the average number of high bits per message. Swapping is
continued until the probabilistic optimality condition in Theorem 1 is achieved. The
resulting code in Figure 4-6 is the optimal ME Code for second order concatenation. This two-step procedure elucidates the performance improvement mechanism
in concatenation.
4.3
Concatenation Theory
In this section, we present some theoretical results concerning analysis of concatenation mechanism. We first prove that direct concatenation yields an energy performance that is identical to the Memoryless ME Coding performance. Then, we show
that concatenation always improves memoryless ME Coding performance, given some
conditions on the codebook parameters. Next, using a counterexample, we show that
29
W1
W2
W3
1
2.2
2
9.3
2
2
2.20 1.76
3
1
2
2.
2
2
0.25 0.20
3
41
2
0.45
2
0.45
2
S 0.0
3
W6
W7
2
2
0.45
'-
119.80
2.25
F8
W5
12
1
W119.80
6
W4
.05
3
0.45 _000
3
0.01
3
0.
W8
0.44
4
3
3
0.05
4
3
4
3
2
2
2
1.76 0.20 0.1
3
3
4
4
4
5
3
3
0.05 0.04
4
4
4
5
3
0.01
3
3
4
4
4
5
4
4
i
z 5
5
D
3
0.44
3
0.44
3
0.05
0.04
0.04
0.04
Figure 4-5: Probabilistic Optimization for Direct Concatenation
this improvement is not monotonic with concatenation order. Finally, we extend the
ME Coding optimality bound to concatenations.
Theorem 3: Direct Concatenation. Let S' be a q-message i.i.d. source, W1
be a codebook, and C 1 (W 1 , Sl) be a code representing the q messages in S1 by the
q individual codewords in W 1 . Furthermore, let SP, WP, and CP(WP, SP) be the
source, the codebook, and the code associated with the p-th extension of the original
source and codebook. There exists at least one code, whose average number of high
bits per message FP/p is equal to that of the fixed-length ME Coding of the original,
non-extended case,
MEY
Vp ;> 2,
3 CP(WP, SP)
such that
-
p
= i
ME
(4.1)
Proof. See Appendix.
Theorem 4: Concatenation. Let S1 be a q-message i.i.d. source, W' be a
codebook, and C'(W 1 , Sl) be a code representing the q messages in S' by the q indi30
19.80
1
2
1
2
2.25
1
2
3
2
0.45
3
2
0.44
3
3
4
2
1.76
2
3
3
3
4
0.01
4
3
4
4
4
5
4
4
4
5
3
4
4
4
5
4
5
5
5
6
2
0.44
3
3
0.04
4
0.05
3
3
0.05 0.04
3
4
3
3
0.16
3
0.44
0.01
3
0.01
0.20
2
0.20
0.45 20.45 0.441(l'
0.25
1.76
2
0.45
2.20
2
1.80
2.20
19.36
2
1.80
19.80
2.25
0.05
0.05
3
0.04
0.04
0.01
Figure 4-6: Extension of ME Coding
vidual codewords in W 1 . Furthermore,let SP, Wp, and CP(WP, SP) be the source, the
codebook, and the code associated with the p-th extension of the original source and
codebook. There exists at least one code, whose average number of high bits per message iiP/p is less than that of the fixed-length ME Coding of the original, non-extended
case,
MY
Vp > 2,
3 CP(WP, SP)
such that
-
< i
(4.2)
if there exists at least one unused codeword in the originalfixed-length ME Codebook
(i.e., q < 2L) and the number of used codewords is not equal to 1 or L + 1.
Proof. See Appendix.
Definition: Geometric Mean Property (GMP). Let S be a q-message i.i.d.
source and W be an L-bit codebook. A code C(W, S) exhibits the geometric mean
property, if the following two conditions hold.
* If codewords i and j have equal number of high bits, ni = nj, then the messages
31
represented by those codewords should have equal probability Pi = P, V(i, j).
* If three codewords are such that the number of high bits in one of the codewords
is the arithmetic mean of that of the other two, ((ni + j) + (ni - j))/2 = ni,
then the probability of the message represented by the arithmetic mean codeword
must be the geometric mean of the probabilities of the messages represented by
the other two codewords, P(nj + j)P(ni - j) = P(ni)2 V(ij).
For example, the code comprising the source having the probability distribution
P = {0.216, 0.144, 0.144, 0.144, 0.096, 0.096,0.096, 0.064} and coded via 3-bit codewords having the set of number of high bits N = {0, 1, 1, 1, 2, 2, 2, 3} via ME Coding
exhibits GMP.
Theorem 5: Concatenation and GMP. Let S be a q-message i.i.d. source,
and W be an L-bit codebook, if the code does not exhibit GMP, and if q = 2 L, then
there exists at least one concatenation order 2p, where A 2P/2p is less than that of the
fixed-length ME Coding of the original, non-extended case, iiM.
Proof. See Appendix.
Remark 1. Average number of high bits per message for concatenation, i'PE/p,
is not a monotonically decreasing function of the concatenation order p. The following is a counterexample.
Let a source S' having the probability distribution
P1 = {0.75, 0.17, 0.04, 0.01, 0.01, 0.01, 0.01} and originally coded with L = 3 bit codewords, be extended and coded according to the ME Coding algorithm. The resulting
average numbers of high bits per message for the ME Codes for p = 1, 2, 3 and 4
are
ME/P
= 0.2800, 0.2671, 0.2584, and 0.2586, respectively. The average number
of high bits per message increases as concatenation order increases from 3 to 4 (i.e.,
0.2584 -+ 0.2586). Therefore, the average number of high bits per message of ME
Codes is not a monotonically decreasing function of the concatenation order.
This behavior is similar to that of Huffman Coding, for which counterexamples
have been found [23]. In ME Coding, Theorem 4 guarantees that, under the mild
conditions, concatenation of an arbitrary order decreases AilpE/p compared with the
original ME Coding
"'ME,
but it does not say that the average number of high bits per
32
message always decreases from p to p+1. For the above counterexample, however, it
can be shown by using Theorem 4 that
Treating the concatenation
ME/4.
'2ME
of order po =2 as an original ME Code, we can guarantee that iiE/Po is higher than
that of concatenation for p = 2 po, 3 po, ....
Figure 4-7 shows these relations among
the first six concatenations. Each directed arc indicates that hP/P decreases from
PO =- 2
2
P=1
4
3
5
6
*7
Figure 4-7: Relations among different concatenation orders
the origin to the destination in concatenation order p. Note that there is no directed
arc from p = 3 to p = 4, which agrees with the above counterexample.
Remark 2. Optimality bound for fixed-length ME Coding can be extended to
provide an optimality bound for concatenation.
As a source and a codebook are
extended according to concatenation, the following changes take place: (i) q -+ qP ,
(ii) L -+ pL, and (iii) H(S) -+ H(SP) = pH(S). Hence, making these substitutions
into the original optimality bound (3.4) gives the following bound for concatenation
(Hk- -)
k
P
S<1.
(4.3)
The next chapter investigates energy-optimal high-rate information transmission
via source coded modulation using practical signal constellations.
33
Chapter 5
Energy-Optimal Source Coded
Modulation
In an effort to achieve energy-optimal high-rate information transmission, this chapter
introduces the Joint Code-Constellation Design Problem. An optimization procedure
for designing energy-optimal source coded modulation systems is introduced and this
design procedure is elucidated.
5.1
Joint Code-Constellation Design (JCCD)
Joint Code-Constellation Design problem originates from the transmitted energy optimization problem that requires the minimization of average energy per message,
E =
E1Pjej. This expression suggests that minimization of average energy per
message can be decomposed into two problems:
" Energy-optimal constellation design problem: selection of the signal points that
will form the signal sequences, and
" Energy-optimal coding problem: mapping of the signal sequences to source
messages.
Figure 5-1 shows that the constellation design problem and the coding problem are
coupled problems and the coupled nature of these problems can be explained as fol34
Probabilities
Constellation
Design
Problem
Coding Problem
Signal Sequences
Figure 5-1: JCCD Problem
lows. The optimal signal constellation is determined by the mapping between the
source messages and the signal sequences. This is due to the fact that the mapping
between the source messages and the signal sequences induces a probability distribution on the signals in these sequences and these probabilities determine the optimal
design of the signal constellation. Hence, this explains the dependency of optimal
signal constellations on the coding. The optimal coding, on the other hand, is determined by the source message probabilities and the energies of the signal sequences.
Since the energies of the signal sequences are determined by the constellation from
which the signal sequences are chosen, optimal coding is affected by the signal constellation. Hence, this explains the dependency of optimal coding on the constellation.
Recognizing that the coding and constellation design problems are coupled problems,
we formulate these coupled problems as a constrained optimization problem. This
coupled problem of designing the coding and the constellation is referred to as the
JCCD problem.
5.2
Optimization Procedure
Solution of the JCCD problem requires that an optimization procedure be developed.
The objective of the JCCD problem is to minimize average energy per message, while
satisfying average error probability per signal, peak signal energy, and normalized
35
rate constraints. Hence, the optimization procedure can be stated as follows:
(5.1)
m n E,
implying a search over all signal constellations C subject to the constraints
d(ci, cj)
dmin
P
Pmin
V i 0j
(5.2)
(5.3)
(5.4)
iI2 <;Ep.
max||i
i
The first constraint on the minimum Euclidean distance of the constellation given
in (5.2) assures that the average error probability per signal is lower bounded. The
second constraint given in (5.3) is a constraint on normalized rate, which restricts the
dimensionality of the signal constellation. For QAM transmission, this constraint basically restricts the length L = D/2 of the signal sequences. Hence, this constraint will
not be a variable in the optimization routine and will only be used to determine the
constant parameter L at the outset of the optimization. Finally, the inequality (5.4)
is a constraint on peak energy due to transmitter hardware limitations.
Figure 5-2 illustrates this optimization procedure. In this figure, it is shown that
Ep
Message
F
c
Probabilities
sC2
dnin
C
Signal 1
Messages
7Ep
Signal 2
Transmitted Signal Sequences
Figure 5-2: Optimization Procedure
the source messages are mapped to the transmitted signal sequences (i.e., codewords)
36
of length L = 2 drawn from two identical QAM constellations. One such codeword is
illustrated by the nodes of the path that connects the two constellations. It is the objective of this algorithm to find the optimal locations of the signal points {c1 , c2 , c3 , c4 }
in this QAM constellation that will result in the code that has the minimum average
energy per message. In designing the signal constellation {ci, c2 , c3 , c4 }, the optimiza-
tion algorithms makes sure that (i) the signals are a minimum distance dmin apart
to satisfy the constraint on average error probability per signal and (ii) the signals
have energies that are less than Ep to satisfy the peak energy constraint. In designing
the code, once a signal constellation is selected, first all available 2-signal codewords
will be listed in increasing order of energy. Next, the first q codewords consuming
the least energy will be selected to form the codebook. Finally, these codewords in
increasing order of energy consumption are mapped to messages in decreasing order
of probabilities. The optimization algorithm will search the QAM space for the optimum code and constellation until the space is exhausted. The constellation space
search can be performed via discretization.
5.3
JCCD Methodology Flow Chart
The overall design methodology flow chart that incorporates the optimization procedure is shown in Figure 5-3. In this figure, it is shown that the first step in the
optimization procedure is to select an initial constellation and the length of signal
sequences according to (5.3). Next, the algorithm checks to see if this initial constellation design satisfies the constraints in inequalities (5.2) and (5.4). If the constraints
are not satisfied, then the algorithm checks to see if the maximum number of iterations has been exceeded; if it has, the algorithm simply terminates; if it hasn't, the
algorithm modifies the constellation and starts over. In the case that the constraints
are satisfied, the algorithm generates all L-signal sequences from this constellation,
picks the q lowest-energy signal sequences ordered in increasing levels of energy and
assigns these sequences to source messages in decreasing probabilities of occurrence.
Once this mapping is performed, the average energy per message for this mapping is
37
SInitial Constellation Design
Fail
Check Constraints
Not Fail
Design Source Coded Modulation
Modify Constellation
Increase
Check Average Energy
per Message
Decrease
Update Constellation
Not Exceeded
-Check Maximum Iterations
Exceeded
End
Figure 5-3: Design Methodology Flow Chart
38
calculated and stored in memory with the generating signal constellation if there are
no previous values in the memory. If the memory contains a previous average energy
per message and a signal constellation, the average energies of the two constellations
are compared and the constellation having a higher average energy is discarded. Then,
the algorithm checks to see if the optimization space is exhausted; if it has, then the
algorithm terminates; if it hasn't the constellation is modified and all the previous
steps are repeated in the search for a better constellation. Hence, at the end of the
optimization algorithm, the designer obtains the constellation that will result in the
source coded modulation system providing the optimal average energy per message
for the given source message probabilities.
5.4
Optimality Condition for 2D Constellations
In communication theory, it is well-known that zero-mean constellations are optimal
for equiprobable signal usage [12].
In this section, we extend this result to non-
equiprobable signal usage for the case where q < M(D/ 2 ).
Theorem 6: Optimality of Zero-Mean Constellations. Let C
=
{ci, c2 , ... ,
CM}
be a signal constellation containingM signals used with probabilitiesP = {P1, P-,..., P}
and each spaced dmin apart. The necessary conditionfor the optimality of the constellation C is given as
M
Z Pcj = 0.
(5.5)
where ci is the coordinate in the constellation space.
Proof. See Appendix.
Although the JCCD procedure results in energy-optimal source coded modulation
schemes, it has the drawback of requiring a significant increase in system complexity. This is due to the hardware difficulties in synthesizing the signals in the signal
constellation. To avoid hardware problems, while still achieving significant energy
savings, the next section introduces sub-optimal source coded modulation algorithms
39
that utilize practical signal constellations.
40
Chapter 6
ME Modulation
6.1
ME Modulation Algorithm
ME Modulation is a new source coded modulation algorithm that optimizes the transmitted energy for a given signal constellation. ME Modulation achieves this by (i)
choosing the minimum energy codewords from the given constellation, and (ii) mapping the minimum energy codewords in increasing order of energies to the source
messages in decreasing order of message probabilities. The following example demonstrates the ME Modulation algorithm.
Assume a transmission system using the 16-QAM constellation C = {c 1 ,..., c1 6 }
in Figure 6-1, where the signals are ordered in the increasing order of signal energy.
Signals having equal energies are in the same sub-constellations shown by the dashed
borderlines. For this 16-QAM constellation there are three sub-constellations: the
inner sub-constellation contains 4 signals ci-c4 with 2 units of energy, the middle
sub-constellation contains 8 signals c5 -c 12 with 10 units of energy, and the outer
sub-constellation contains 4 signals c13 -cI 6 with 18 units of energy. Assume that
the system transmits messages from a q = 200 message source S
=
{mi,
... ,
m 2 0 0}
having messages mi ordered in the decreasing order of message probabilities P =
{P 1 > ... > P 200 } as shown in the first column of Table 6.1. After analyzing the
signal constellation, the codeword length L = D/2 is determined from the minimum
normalized rate constraint p = (2/D)log 2 M
41
>
pmin. Assuming that the normalized
Q
C13
-- - - - - - -s
C5
6C12 1
C11
L 9-
C6
CI
C 01C70
C4_
C3
-
0
C8
C9
C10
C16
1
C14
-3
C
Figure 6-1: 16-QAM Constellation
rate constraint requires the use of codewords of length L = 2, the whole set of available
codewords Wo(C) = {(ci, cj)I1
i < 16, 1 < j < 16} containing 16 x 16 = 256
codewords is generated. This set is shown in the second column of Table 6.1. A closer
look at this set reveals that there are 16 codewords with 2 + 2 = 4 units of energy,
64 codewords of 2 + 10 = 12 units of energy, 96 codewords of 10 + 10 = 2 + 18 = 20
units of energy, 64 codewords of 10 + 18 = 28 units of energy, and 16 codewords of
18+18 = 36 units of energy. These codeword energies and the number of codewords
are shown in the third and fourth columns of Table 6.1, respectively. These codewords
are ordered in the increasing order of energy; and the first 200 codewords having
the least energy are chosen for the codebook. These 200 codewords, termed the
minimum codebook, are mapped to source messages in the decreasing order of message
probabilities as shown in the first two columns of Table 6.1. This mapping is called
the ME Modulation Code.
As demonstrated in the above example, ME Modulation algorithm requires the
prior selection of a signal constellation. This signal constellation can be any practical
QAM constellation [6,16]. Next, we prove that ME Modulation results in optimal
modulation codes for a given signal constellation.
42
Table 6.1: 16-QAM ME Modulation Example
Messages
Codeword Set
C.E. N.o.C.
16
4
{c 1 - c 4 } x {ci- c 4}
S- m 16
32
12
{c - c4 } x {c 5 - c12 }
M7- i 4
32
12
{c5 - c 12 } x {c 1 - c4 }
M 4 9 -- M 80
16
20
}
c
{c
}
x
c
{c
Mi81 - mn
16
13
4
8
16
20
{c 13 - c 16 } x {c 1 - c4 }
M87 - i10 2
64
20
{c - c 12 } x {c 5 - c 12 }
Mi103 - Mi176
32
28
}
c
{c 5 - c12 } x {c 13
M 200
M1- 7
16
32
28
c1 2 }
S{c 1 3 - c1 6 } x {c 5 16
36
c16 } x {c 13 - c1 6 }
S{c 13 -
6.2
Optimality of ME Modulation
ME Modulation is the optimal source coded modulation algorithm for a given signal
constellation. This optimality is determined by two distinct steps: Codebook Optimality and Probabilistic Optimality. The former is to determine a set of codewords or
signal sequences, termed a codebook, having the least energy, and the latter is to assign codewords having less energy to messages with higher probability. The following
two theorems underpin the theory of ME Modulation.
Theorem 7: Probabilistic Optimality. Let S be a source with message probabilities
P = {P1
--. > Pq-1
P2 >
Pq}.
(6.1)
Given a signal constellationC that generates a codebook of q codewords each of which
consumes ej units of energy, 1 < i < q, the optimal source coded modulation scheme
that minimizes the average energy per message E =
j is given by assigning
1 iej
the codewords to messages such that
el
e2
<
...
43
<
eq-1 <
eq.
(6.2)
Let W(C) be a codebook of q codewords generated by a given signal constellation
C = {c 1 , ..., cM}. Also let the codewords in W(C) be arranged in the order of ascend-
ing energy. The code (W(C), S) assigns the q codewords to q messages in source S
in such a manner that equations (6.1) and (6.2) determine the minimum E for the
given W(C). Hence, the minimum of E varies depending on the properties of the
codebook used. The remaining question is how to obtain a codebook that provides
the overall minimum of B. Let the constellation C generate the entire set of codewords W 0 (C) = {w 1 , w 2 ,
q
q. < +oo.
...
,
wi,
...
, WqoW-1,
Wq}
that are usable for ME Modulation;
We first number each codeword in the ascending order of energy,
namely, el 5 e 2 5 ... 5 ej < ... < eg.-1 ! eqo, where ej is the energy of codeword
wi. Then, we generate the codebook used for ME Modulation by taking the first q
codewords having the least energy.
Minimum Codebook.
Definition:
word set Wo(C) = {wi, w 2 ,
of energy, el 5 e2 5
...
...
, W,
K eq
...
Let the codewords of the whole code-
, Wqo}
be numbered in the ascending order
< ... < eq0 .
A minimum codebook of q code-
words, Wmin(C), consists of the first q codewords of the whole codeword set W 0 (C),
Wmin(C) = {Wi,W2,
---, Wq-1, Wq} C Wo(C).
Theorem 8: Codebook Optimality. Let S be a q-message source and C be a
signal constellation that generates a codebook W(C) of q-codewords taken from the usable codewords set W 0(C) = {W1,W2, ---,Wq-l,wqo}, q : qo < +oo; W(C) C W 0(C).
The necessary condition for a code (W(C), S) to provide the minimum average energy per message
E
= (j
Pej is that the codebook W(C) is a minimum codebook
of WO(C).
Proof. See Appendix.
Combining Theorems 7 and 8, the following corollary is easily obtained.
Corollary 3: ME Modulation. Let S be a q-message source, C be a finite signal constellation, and Wo(C) = {W1,
codewords, q
W2 ,
... , Wq,
...
, Wqo}
be the entire set of usable
q0 , derived from the signal constellation. The optimal code (W (C), S)
that minimizes the average energy per message E= (
involved in the codeword wi, is given by
44
eiPi, where ej is the energy
(i) Using the minimum codebook W.in(C) of W0 (C) for the codebook; W(C) =
Wmin(C) C W 0 (C), and
(ii) Assigning the q codewords of Wmin(C) in the ascending order of energy to the q
messages in the descending order of message probabilities.
This optimal mapping is referred to as ME Modulation.
Next, we investigate the extensions of ME Modulation for improving energy efficiency.
45
Chapter 7
Extensions of ME Modulation
Chapter 6 introduced ME Modulation as the energy-optimal source coded modulation
algorithm for a given signal constellation. Given this optimal modulation algorithm,
this section takes a step further and introduces extensions of ME Modulation that
improves the energy efficiency of non-extended ME Modulation. The chapter starts
with a formal definition of extensions of ME Modulation. Then, direct concatenation
is described as a special type of extension that achieves identical energy performance
as non-extended ME Modulation. Finally, an example elucidating the performance
improvement mechanism in concatenation is presented.
7.1
Extension and Concatenation
Definition: Concatenation. Let S' be a q-message i.i.d. source with known statistics and coded using an L-signal codebook W1 (C) generated from the constellation
C = {c1 , ..., cm}. The p-th concatenation of S' and W 1 (C) is a source coded modu-
lation algorithm that encodes the p-th extension of the message source, SP, using a
pL-signal codebook, WP(C).
Definition: Direct Concatenation. Direct concatenation is special type of
concatenation that encodes message words (i.e., coupled messages) using the concatenated codewords that comprise the codewords that were assigned to the messages
in these words in the original ME Modulation. This idea is clarified in Figure 7-1,
46
where direct concatenation is demonstrated for second order concatenation. In this
figure, a new codeword of length 6 signals is created by directly concatenating the 2
codewords used in the original ME Modulation to form a single codeword. Note that
(S ,l s i2, s i3) + (sj ,9SJ2, s j3)
(s i Sl
si2, s i3, si Psj2, sj3)
(s ,Isj 2, sj 3 ) + (S; , Si2, si 3)
(sj 1, Sj2, sj3, s
(a)
si2, S3
(b)
Figure 7-1: Direct Concatenation. (a) ME Modulation, (b) Direct Concatenation.
the signal sequence of this elongated codeword is exactly the same as the original
signal sequence. Direct concatenation results in codewords having a total energy that
is equal to the sum of the energies of the original codewords. These newly formed
codewords are assigned to words of messages that include the original messages assigned to the original codewords. Higher-order direct concatenation is performed in
a similar manner.
Theorem 9:
Direct Concatenation.
Let S1 be a q-message i.i.d. source,
W'(C) be a codebook having the set of energies E' = {ei, ..., eq}, and (W 1 (C), Si) be
a code representing the q messages in S' by the q codewords in W'(C). Furthermore,
let SP, WP(C), and (WP(C), SP) be the source, the codebook, and the code associated
with the p-th extension of the original source and codebook. There exists at least one
code, whose average energy per message RP/p is equal to that of the ME Modulation
of the original, non-extended case,
REy,
RP
Vp ;> 2,
- (WP(C), SP)
Proof. See Appendix.
47
such that
-=
p
E.
The average energy per message of direct concatenation is exactly the same as the
non-extended ME Modulation. This means that concatenation can provide at least
the same level of energy efficiency as non-extended ME Modulation and would provide
a better efficiency by modifying the direct concatenation case. Direct concatenation
does not guarantee codebook and probabilistic optimality introduced in Section 6.2.
A loss of optimality in either of the two will warrant a reduction in average energy per
message. Hence, to understand the energy efficiency improvement via concatenation,
the codebook optimality and probabilistic optimality should be examined for direct
concatenation.
Codebook optimality is decoupled from probabilistic optimality, since it is determined only from the signal constellation C, the codeword length, pL, and the number
of messages, qP, only. Direct concatenation uses a particular type of codewords restricted to the repeated pattern of the original codewords. Although there are MPL
usable codewords, the codebook of direct concatenation uses only this restricted set
of the whole usable codewords set. If there are some unused codewords having less energy, the codebook of direct concatenation is not a minimum codebook and reduction
in the average energy per message becomes possible for the direct concatenation.
Probabilistic optimality is determined by the mapping between the codewords and
source messages. In direct concatenation, the message-to-codeword assignment is predetermined, and is not necessarily an optimal one. Since the energy in a concatenated
codeword is the sum of the individual energies of the original codewords, while the
joint probability is the product of the individual probabilities, the ascending order of
energies may differ from the descending order of probabilities. Thus, the probabilistic
optimality must be re-investigated.
The following section uses an example to elucidate the performance improvement
mechanism in concatenation.
48
7.2
Mechanisms of Concatenation
Consider a source S' of q = 6 messages having the probability distribution P1 =
{65,21,8, 4,1, 1} in percentages. Let this source be transmitted using L = 2 signal
codewords drawn from a three signal constellation C
{c1 , c2 , c3 }. Let these three
=
signals have energies of 1, 2, and 3 units, respectively. Then, the set of available
codewords generated using this signal constellation will contain 9 codewords with the
following set of energies Ei = {2, 3, 3, 4, 4, 4, 5, 5, 6}. This available codeword set is
used to generate ME Modulation as shown in Figure 7-2. The first column lists the
WO
25 3
Codeword
218
w2 C
45
14
14
23
5
6
01 70.i
4
21
2
3
Minimum
Codebook
.
w2
cl
w3
C21,
1
w4
c 2 ,c
2
4
w5
c 1 ,C 3
4
w6
C3 ,C
4
,C 2
21
83
4
I
Unused
Codewords
W7
C2 ,C3
w8
C3,'C2
w9
C3, C36
5
5
Figure 7-2: ME Modulation Example
entire set of usable codewords arranged in the ascending order of energy. The first
q = 6 codewords form the minimum codebook, leaving the three codewords w7 , W8 ,
and w9 unused. The third column consisting of square blocks indicates the energy ei
at the top left corner and the probability P in percentages at the bottom right corner.
Note that the probabilities have been assigned in descending order from the top, wi,
49
to the bottom, W6 . Note also that the square blocks are grouped into rectangles in
terms of the energies, as shown by the thick lines in the figure.
The second order direct concatenation of this ME Modulation can be generated
by arranging the same ME Modulation in a row and combining two codewords: one
taken from the column and the other from the row, as shown in Figure 7-2. For
example, the shaded block in the figure is the concatenation of w2 from the column
and w5 from the row having e2 + e5 = 3 + 4 = 7 units of energy and the
probability of P2 x P5 = 0.21 x 0.01 = 0.0021. Repeating this process for all the
pairs of the column and row codewords yields the complete direct concatenation of
the original ME Modulation. As indicated by the thickest solid line in Figure 7-3, 36
codewords arranged in a 6-by-6 matrix are obtained. In the following two steps, we
investigate the codebook and probabilistic optimality of direct concatenation for this
given example.
4
5
5
6
4
13.65
5.20
W
5
6
6
7
7
7
1.64
0.84
0.z2
0.21
7
7
26
4.61
5
W
520
6
6
7
32
1j.
HJ
0.6
1.6
6
6
65
.65
7
7
0.08
0.0
8
7]
110.081
0.65
I
9
818
8
18
9
9
9
9
6
17
8
8
10
10
7
0.21
7
0.08
0.04
8
0.04
0.01
8
0.01
0.01
6
0.65
8
0.01
9
9
1
7
8
8
9
9
9
10
1
1
7
8
10
1
12
8
9
9
1
0
0
1
FE]LJ
Figure 7-3: Direct Concatenation Example
Step 1: Codebook Optimization. For a code (W(C), S) to be optimal, the
necessary condition is that the codebook W(C) is a minimum codebook. Hence, we
check if the direct concatenation codebook is a minimum codebook. Elimination of
50
the probabilities from Figure 7-3 results in Figure 7-4, which illustrates the set of
available codewords and the codebook for direct concatenation of ME Modulation.
5:5
4
6
6
6 j
5
51
6
6
6
6
7
7
7
7
7
7
6
7
7
8
8
8
6
77
8'88
7
7
8 1
8
8
6
7v
I
ff
9
7
7
00W
88
0.010.01
O
8
9
8
8
9
9
9
10
9
9
10
9
9
9
9
10
10
11
9
a
9n
10
10
11
9E9 1E lo 10l
11~ 11
Figure 7-4: Direct Concatenation Codebook
The codebook for direct concatenation is represented by the 6-by-6 matrix enclosed
by the thick dashed line. In this figure, it is seen that direct concatenation uses
codewords having a maximum energy of 8 units, while the set of unused codewords
contains codewords with energy 7 as marked by the two ellipses. Hence, the codebook
is not optimal, since the total energy of the codebook can be reduced. Thus, these
unused codewords having less energy should replace the ones with more energy to
reduce the average energy per message. This results in the minimum codebook shown
in Figure 7-5. As seen in this figure, messages originally assigned to 8 energy unit
codewords (i.e., P=0.01) move to the newly available codewords having 7 units of
energy. Note, however, that the global code optimality is not guaranteed until the
message-to-codeword assignment is optimized.
Step 2: Probabilistic Optimization.
The probabilistic optimality proven
in Theorem 7 is now applied to the minimum codebook. A code is not optimal,
51
5
5
4 5
5
6
16
6
6
7
17
7
7
7
7
7
8
8
6
66
6
700.
8
8
8
9
71
8
8
9
8
9
9
10
8
9
9
10
98
7
7
6 7
7
8
8
0
8
8
9
9
9
10
10
11
70
8
8
9
9
9
10
10
11
6
--
8P9
10
10 1 o0 E111
Figure 7-5: Optimal Codebook for Direct Concatenation
if there exists two message-to-codeword assignments (P, ei) and (Pj, ej), such that
Pi > Pj and ej > ej. Hence, the mappings exhibiting the non-optimality condition
(marked by ellipses in Figure 7-6) are swapped.
For example, the message having a
probability of 0.84 % was originally assigned to a codeword having 7 units of energy
in the minimal codebook (w2
-
W2
codeword), while a message having a probability
of 0.65 % was assigned to a codeword (w1
-
w4 codeword) having 6 units of energy.
Thus, the two messages are swapped in Figure 7-7 to reduce the average energy per
message. Swapping is continued until the non-optimality condition does not hold
between any pairs of mappings. The resulting optimal code, i.e., ME Modulation, is
shown in Figure 7-7.
52
5
5
13.65 5.20
5
6
6
5
5.20
6
6[
i6 7r
0.65
616
0.65
7
7
7
7
7
7
0.01
. 32 0.08 0.08
1.68
w
7
6
2.60
7
i
0.84 0.32
6
7
0.65
0. 1 0.08
8
8
(9.16)
8
7
(004
(0.04
0.21 10.08
7
.0
7
0.01
Figure 7-6: Probabilistic Optimization for Direct Concatenation
5
5
13.65 5.20
4.5
1
5
13.65
6
4.41
6
59
6
61
1.68
0.84
5.0
1.681
6
6
2.60
0.65
7
7
0.64 0.21
7
7
0.32
0.08
3100 -.
6
2.60
7
0.65
7
0.32
8
0.04
6
0.65
7
0.21
7
0.08
8
0.01
6
7
0.21
7
0.08
8
0.01
0.84
8
0.01
6
0.65
7 04 7
0.04
0.
7
0.21
7
0.08
00
8
0.01
7
0.16
Li.
0.04
7
Figure 7-7: Extension of ME Modulation
53
Chapter 8
Energy-Efficient, Low-Rate, and
Reliable Information Transmission
over Power-Limited Wireless
Channels
Previous chapters discussed the question of how to most efficiently encode source
information for transmission over a noiseless channel. In this chapter, the issue of
information transmission over a power-limited noisy channel is investigated, where
errors introduced by channel noise have to be corrected.
8.1
Setting the Scene
Figure 8-1 shows the major blocks of a communication system. The definitions that
follow will explain each block in this model of a communication system. Previous
chapters concentrated on the encoder and modulator block. However, this block
was investigated only in terms of data compression and source coded modulation
for noiseless channels. This chapter will first introduce the noisy channel, and then
consider the encoder and decoder blocks in terms of energy efficient error correction.
54
Noise
Source
Information
Source
|
Encoder
WChannel d
Decoder
f- (dstain
Modulatori
P
Hl) DemodulatorH
i
Figure 8-1: Block Diagram of a Communications System
8.1.1
The Channel
The communication channel is the medium through which the information bearing
signal is transmitted. This thesis considers information transmission over discrete
memoryless channels.
Definition: Discrete Memoryless Channel. A discrete memoryless channel
consists of an input alphabet, an output alphabet, and a set of channel transition
probabilities p(bj jai) satisfying
t
Zp(bjlai) = 1
j=1
for all i. Intuitively, we think of p(bj ai) as the probability that bj is received given
that ai was sent through the channel. This is illustrated in Figure 8-2.
ab
1
a2
b2
Channel
a)
a. 4p(bjI
b3
b.
as
bt
Transmitted
Messages
Received
Messages
Figure 8-2: Discrete Memoryless Channel
55
Furthermore, if a sequence of source messages aja 2 .. .a, represented as a codeword
w
=
w1 w 2 .. .wn is transmitted and a codeword d = did 2 ...d, is received, the channel
transition probability p(djw) is given by
n
71p(dilwi).
p(dlw) =
i=1
One of the most important discrete memoryless channels is the binary symmetric
channel, which has input and output alphabets {0, 1} and channel probabilities
p(1|0) =p(0j1)=p
and p(00)=P(1I1)= 1-p.
These characteristics of the binary symmetric channel are summarized in the channel
transition matrix given as
P(bla) = [P00.
P(110)
LP(011)
p(111),
[i
P
P
(8.1)
I - p
This channel is pictured in Figure 8-3. In this chapter, we focus on transmission over
I-P
0,
Transmitted
Messages
P
1190
Received
Messages
i-p
Figure 8-3: Binary Symmetric Channel
binary symmetric channels.
The binary symmetric channel is a noisy channel that causes errors in transmission
if p -$0. Let us first explain what we mean by single bit errorsor codeword errors. In
the binary symmetric channel, a bit error occurs if a 0 is sent and a 1 is received or
a 1 is sent and a 0 is received. Thus, the probability of a bit error, or the crossover
probability, is p. This can be seen in Figure 8-3. When a codeword w consisting of
56
the string ww
2 ..-wn
is sent through the channel and a codeword d consisting of the
string did 2 ...dn is received, we say that a codeword error has occurred if wfd, and
the incorrect d is not corrected. Next, we discuss the types of errors that may occur
when transmitting over a noisy channel.
Errors, in general, can be of four types: (i) isolated bit errors, (ii) burst errors,
(iii) erasures, and (iv) insertions. Isolated bit errors, as the name implies, occur when
the errors in transmission are independent. However, this can be unrealistic in certain
situations, where there is a strong correlation among the bit errors. In this case, we
speak of burst errors. For example, electrical interferences often last longer than the
time it takes to transmit a single bit, defects on magnetic tape or disk are usually
larger than the space required to store a single bit, and noise in wireless channels may
corrupt multiple bits at the same time. The third type of errors, namely, erasures,
are characterized by situations, where the decoder knows that a single bit is occurring
at a point, but cannot determine its value. Finally, insertions occur when extra bits
are inserted into the bit stream of transmitted data. Coding Theory aims to develop
codes correcting certain types of errors. Much work has been put into developing
structured codes correcting isolated bit errors. Hence, in this thesis we look into
codes correcting isolated bit errors.
8.1.2
The Decoder and Error Probability
Definition: Decoder. A decoder is a partial function
f
from the set of received
words to the set of codewords. The word partial refers to the fact that
f
may not
be defined for all received words. The intention is that if a word d is received, and if
f(d) is defined, then the decoder decides that f(d) is the codeword that was sent. If
f (d) is not the codeword that was sent, we say that a decoding error has been made.
By letting
Bi = f-1 (w) = {dlf (d) = w}
be the set of all outputs for which we decide that the correct input was w, we can
57
also think of a decoder as a collection {Bw, ,..., Bw,
}
of disjoint subsets of the set of
received words. This is illustrated in Figure 8-4. One important measure of decoder
w2 0.d
ow
2
2
.d4
-d4
Decoder
Channel
f(d)
Transmitted
Codewords
Received
Words
Decoded
Codewords
Figure 8-4: Decoder
performance is the error probability, hence we now define error probability.
Definition: Error Probability. For any decoder, the error probability given
that a codeword w is transmitted is given by
P(errorjwsent) =
E
p(dlw).
dof- 1 (w)
Hence, the errorprobabilityfor the whole code, pe, is given by
Pe = E
P(errorlw)p(w)= E
W
w
E
p(dJw)p(w)
djf-'(w)
A desirable decoder is one that offers a minimal error probability in the most
computationally practical way. In order to see how we can design a decoder that
minimizes the error probability, let us compute this probability by conditioning on
the output, rather than the input. If the output of the channel is d, then correct
decoding will be made iff f(d) was the actual input. Hence,
P(errord)= 1 - p(f (d)|d)
58
Averaging over all possible outputs, we have
Pe = EP(errord)p(d)= 1 d
p(f (d)d)p(d)
d
Now, this probability can be minimized by choosing a decision scheme that maximizes
the sum on the far right. But since each term in the sum is nonnegative, and since
the factors p(d) do not depend on the decoding algorithm, we can maximize the sum
by choosing f(d) so that p(f(d)Id) is as large as possible for all d.
Definition: Ideal Observer. For a given input distribution, any decoder
f for
which f(d) has the property that
p(f (d)Id) = max p(wld)
w
for all received words d, is called an ideal observer. In words, an ideal observer is one
for which f(d) is a codeword most likely to have been sent, given that d is received.
The ideal observer decoder has the advantage that it yields optimal probability
of error; however, it uses backward channel probabilities, which depend on the input
distribution. This suggests that the ideal observer will be computationally impractical. Thus, we seek ways to eliminate this dependency on the input distribution. We
can eliminate this dependency by assuming a uniform input distribution. This results
in the desirable feature that maximizing the backward channel probabilities become
equivalent to maximizing forward channel probabilities. This is extremely practical,
because it reduces the computation requirements. The undesirable feature is that the
error probability performance is not as good as the ideal observer.
Definition: Maximum Likelihood Decoder. Any decoder
f
for which f(d)
has the property that
p(dlf (d)) = max p(dlw)
w
for all received words d, is called a maximum likelihood decoder. In words, f(d) is a
transmitted codeword with the property that for no other codeword would it be more
59
likely that the output d was received.
Although the maximum likelihood decoding eliminates the dependence of decoder
on the input distribution, it still exhibits a coupling between the decoder and the
forward channel probabilities. Thus, we seek ways to eliminate this coupling. First,
let us define the Hamming Distance.
Definition:
Hamming Distance.
Let x and y be codewords of the same
length, over the same alphabet. The Hamming Distance d(x, y) between x and y is
the number of positions in which x and y differ. For instance, if x= 10110 and y=
10101, then d(x, y) = 2, since the two codewords differ only in positions 4 and 5.
We will now state without proof, a fundamental result in Coding Theory and
use this result in deriving energy optimal codes for wireless communications. The
fundamental result is that the maximum likelihood decoder reduces to minimum
distance decoding for binary symmetric channels with crossover probability p < 1/2.
This result is also true for binary erasure channels and some other channels of interest.
However, at this point we will only consider binary symmetric channels.
8.1.3
The Main Energy Efficient Error Control Problem
Before describing some coding schemes for energy efficient and reliable communication, we should describe the desirable characteristics of an efficient coding scheme and
elucidate the main problem of energy efficient error correction. The characteristics of
an efficient coding scheme can be summarized as
" minimizing the energy consumption in the RF transmitter,
" offering a desirable reliability performance in information transmission,
" requiring a computationally feasible decoding scheme,
" resulting in reasonable transmission rates.
In designing energy efficient codes for wireless communications, we will initially focus
on the first three requirements and will not concern ourselves with the fourth requirement. Our objective will be to design energy optimal codes, offering a given error
60
correction capability in a computationally feasible manner. Due to the computational
feasibility requirement, we will consider linear block codes.
The problem of energy efficient error correction is in fact a codebook design problem upon which ME Coding is performed. This problem is illustrated in Figure 8-5.
For a given codeword length the set of available codewords is shown by all the larger
0o0
000" 0
0 0 0 0
09 0
00o
00
00
0
0
0 0
0
o0**
0
*0 0 0
0 0
04 0
o
00 0
0 0
00
0
0
00
0 0 0 0
0
0
0
0
o
R-3
W 0
R-2
0
0 0
oR-1
0 0 0T
0
0
Rad i us t
0o
00
0
0 0
0
of
00Spheres
0
0
r a ns mi t t e d
Codeword
Used
Codewords
Hamming Distance
Unused Codewords
Figure 8-5: The Main Energy Efficient Error Correction Problem
filled and the smaller unfilled circles in this figure. The larger filled spheres are the
codewords used in the codebook and the smaller unfilled spheres are unused codewords. The problem is how to select the codewords such that all the codewords we
have selected are (i) at least 2t+ 1 units apart in terms of Hamming Distance, and (ii)
constitute the elements of a minimum codebook. Since we use linear block codes, the
first requirement is automatically determined by the type of code we use. The second
requirement is unfortunately not always satisfied by linear block codes. Satisfying the
second requirement involves the use of nonlinear codes which are computationally impractical. This illustrates the tradeoff between energy efficiency and computational
complexity. Since in our design computational feasibility is a hard constraint, we will
ignore the second requirement for the time being. Going back to the first requirement, we see that this requirement is necessary for t-error correction. This t-error
61
correction can be explained as follows.
Suppose a codeword w is transmitted as indicated in Figure 8-5 and suppose that
all the used codewords are at least 2t + 1 units apart, meaning that the spheres
of radius t around any used codewords will be non-intersecting. Thus, during the
transmission, if e errors occur in transmission, e < t, the received codeword (R-1)
will still be in the sphere of the sent codeword and can be decoded correctly as that
codeword. However, if the number of errors in transmission is e > t, then the received
codeword (R-2 or R-3) will not be in the sphere of radius t and will most probably
not be decoded correctly. Thus, any number of error less than or equal to t will be
corrected with this code of minimum distance 2t + 1. Once the codewords having
a minimum distance of 2t + 1 are selected, energy efficiency in transmission will be
optimized by performing ME Coding. That is the higher probability messages will be
mapped to codewords with less high bits. This way, we will achieve optimal energy
efficiency for a given reliability performance in a computationally feasible manner.
Theorem 10: Codebook Optimality for Error Correcting Codes. Let S
be a q-message source and W be a codebook of q-codewords taken from the usable
codewords set W. = {w 1 ,w 2,
... ,
wq-1,7wqa},
q
q. < +oo; W C Wo generated
by an error correcting code. The necessary condition for a code C(W, S) to provide
the minimum average number of high bits ii
(L
n1 P is that the codebook W is a
minimum codebook of W.
Proof. See Appendix.
Next, we introduce energy efficient linear codes for reliable communication.
8.2
Energy Efficient Block Codes
Many practical error-correcting codes used today are linear. The reasons why linear
codes are so popular can be summarized as follows.
" Evaluation of distance between codewords is much easier.
" Encoding is fast and requires little storage.
62
"
Determination of which errors are correctable/detectable is much easier.
" Probability of correct decoding is much easier to calculate.
" Very cunning decoding techniques exist for linear codes.
Due to these advantages we will investigate the use of linear codes for energy efficient
error recovery.
8.2.1
Minimum Energy Hamming Codes
Hamming Codes are perfect linear block codes offering single error correction capability. Hamming Codes provide this single error correction capability due to their
minimum distance of 3 bits. This is illustrated in Figure 8-6, where codewords 1 and
2 denote any two codewords in the Hamming codebook. Any single bit error in transHamming Distance 3
r
Codeword
1
t
Single Error
Correction
Codeword 2
Figure 8-6: Single Error Correcting Hamming Codes
mission of a codeword results in a received codeword, which is on the circumference
of the unit circles. Since this received codeword is still closest to the codeword that
is originally transmitted, any single error can be corrected as it is seen in Figure 8-6.
How do we combine ME Coding and Hamming Codes? Hamming Codes are basically usable codeword sets, Wos, for single error correction. Since Hamming Codes
impose no constraints on the mapping between the codeword set and the source messages, we can combine ME Coding with Hamming Codes just by regarding the given
Hamming Code as the usable codeword set WO in Theorem 2 and performing ME
63
Coding on this set. Rather than giving formal definitions and general descriptions of
the coding and decoding algorithms of ME Hamming Codes, we will give an example
that demonstrates these procedures. More detailed information on Hamming Codes
and extended Hamming Codes can be found in [24,25]
ME Hamming Code Example.
Let S be a q = 10 messages source with
probabilities given in Table 8.1. In order to generate a ME Hamming Code for this
S
m1
m2
M3
M4
M5
m6
m7
M8
Table 8.1: ME Hamming Code
Prob.
MEH
S
Prob.
MEH
0.19 0000000 mg
0.04 0001111
0.02 0110011
0.15 0010110 mio
0111100
0.13 0011001
0.12
0100101
1010101
0.11
0.10
0101010
1000011
1011010
1100110
0.08
0.06
1001100
1110000
1101001
1111111
source, we first construct the set of usable codewords: the Hamming Code.
Codebook Generation of ME Hamming Code. We construct this set of
codewords by choosing the longest Hamming Code (i.e., longest L) that satisfies the
transmission rate and error probability constraints and provides enough codewords
to code all q messages. The error probability constraints are checked by trial and
error. First, an error correcting code offering a certain amount of protection is used
and the error performance of the code is checked. If desirable reliability performances
are not achieved, then a different code offering more protection is used. We choose
the longest code because this maximizes the number of codewords with less high bits,
thereby optimizing the energy performance. Assuming that these constraints result
in the Hamming (7, 4) Code being the only feasible code in this example, we generate
the usable codeword set for this code and perform ME Coding. Hamming (7, 4)
Code is a single error correcting code that accepts 4-bit information sequences and
results in 7-bit codewords. The usable codeword set for the Hamming (7, 4) code is
64
generated using the parity check equations
c HT
(8.2)
0,
=
where c is each codeword in the codeword set W and HT is the transpose of the
parity check matrix H. The parity check matrix for the Hamming (7, 4) Code is
H=
0
0 0
1
0
1 1 0 0 1
1].
1
0 1
1
1
0 1
1
0
1
(8.3)
The resulting set of usable codewords, ordered in ascending number of high bits,
is given in the third and sixth columns in Table 8.1. From this Hamming Code,
we choose the minimum codebook consisting of the first 10 codewords, and assign
codewords having less high bits to more probable messages. The resulting code is the
ME Hamming Code (see Table 8.1).
Decoding of ME Hamming Codes. The decoding of Hamming Codes is as
follows. Assume that the codeword 0011101 containing a single bit error is received.
This erroneous codeword is multiplied by the parity check matrix of the Hamming
Code which results in the error vector e given as
e=HwT =
0 0 0
1
1
1
1
0 1
1
0 0
1
1
1
1
0
0
1
0
1
[0 0 1 1 1 0 I]T = [1 0 1]T.
This error vector represents the bit position of the error in binary coded decimal
form. That is there is an error occurring in the fifth binary digit, and this bit can be
complemented to correct this error: 0011101 -+ 0011001.
Extended Hamming Codes. We should also note that Hamming Codes can be
extended to provide double error detection by adding an overall parity check bit. This
overall parity check bit merely increases the minimum distance of Hamming Codes
to four bits, thereby enabling double error detection. Extended Hamming Codes are
65
illustrated in Figure 8-7, where we see that by achieving a minimum distance of 4,
extended Hamming Codes realize that a codeword containing double errors is not a
legal codeword and that a double error has occurred. Once the decoder detects a
Hamming Distance 4
Double Error
Detection
Codeword 2
Codeword I
Single Error
Correction
Figure 8-7: Single Error Correcting and Double Error Detecting Hamming Codes
double error, it can ask for a retransmission.
While constructing the ME Hamming Code, we realize the trade-off among the
three performance metrics: energy efficiency, reliability, and transmission rate. The
ME Hamming Code in Table 8.1 is using codewords having zero, three, and four
high bits. This code clearly provides a degraded energy performance compared to
ME Codes in Chapter 3, which would use zero, one, and two high bit codewords
only. Hence, this illustrates the trade-off between energy efficiency and reliability.
Furthermore, if the codeword length of the present ME Hamming Code is increased,
more three high bit codewords would become available, thereby improving the energy
efficiency of the current ME Hamming Code. However, this would degrade the transmission rate performance, re-introducing the trade-off between energy efficiency and
transmission rate. Hence, energy efficient transmission is a trade-off among energy
efficiency, reliability, and transmission rate.
66
8.2.2
Minimum Energy Golay Codes
We start explaining Golay Codes with extended Golay Code. Extended Golay Code
is used to correct three or fewer isolated bit errors. Extended Golay Code was in
fact used in the Voyager spacecraft program in the early 1980's and brought us the
close-up photographs of Jupiter and Saturn. Contrary to what we did in the previous
section, in this and the next sections we will give formal definitions and generalized
coding/decoding algorithms of ME Golay and Reed-Muller Codes.
Definition: Extended Golay Codes. Let B be the (12 x 12) matrix
Let G be the (12 x 24) matrix G = [I, B], where I is the (12 x 12) identity matrix.
The linear code C with generator matrix G is called the extended Golay Code and
will be denoted by C24. By inspection we see that BT = B; that is, B is a symmetric
matrix.
Characteristics of Extended Golay Codes and the Codebook Generation
Algorithm. Let us now summarize some important facts about the extended Golay
Code C24 with the generator matrix G = [I, B]:
1. C24 has length L = 24, dimension k = 12 and 212 = 4096 codewords.
67
2. A parity check matrix for C24 is the (24 x 12) matrix
check matrix for C24 is the (24 x 12) matrix H =
B
B
. Another parity
.
3. Another generator matrix for C24 is the (12 x 24) matrix [B, I].
4. The distance of C is 8.
5. C24 is a three-error-correcting code.
Thus, spanning the generator matrix using all codewords of length 12 bits, we can
obtain the (24, 12, 4) extended Golay Code. In other words, the Golay Code C24 can
be obtained by using the following matrix equation
C24 = uG
where u represents all binary strings of length 12 and G = [I, B] is the generator matrix with I is the identity matrix of size (12 x 12).
Thus, a binary string
u=001010000000would result in the codeword 001010000000,101101001100 through
simple matrix multiplication. Next, let us summarize the decoding algorithm for the
extended Golay Code.
Extended Golay Code Decoding Algorithm. Let oi be the word of length
12 with a 1 in the i-th position and Os elsewhere, e be the error vector, and bi be the
i-th row of B. Then, the decoding algorithm of extended Golay Code comprises the
following seven steps:
1. Compute the s = dH, where s is called the syndrome, d
codeword, and H is the parity check matrix given by H =
B
is the received
.
2. If wt(s) _< 3, then e = [s, 0], where wt(s) is the weight of the syndrome which
is equal to the number of is in the syndrome.
3. If wt(s + bi) < 2 for some row bi of B, then e = [s + bi, oi].
68
4. Compute the second syndrome sB.
5. If wt(sB)
3, then e = [0, sB].
6. If wt(sB + bi) < 2 for some row bi of B, then e = [oi, sB + bi].
This algorithm requires at most 26 weight calculations in the decoding procedure. Of
course, once e is determined, then no further steps in the algorithm need to be done.
The transmitted codeword can simply be found by w = e + d. Let us give some
examples illustrating this decoding procedure.
Suppose a codeword d=10111110111,010010010010 is received. The syndrome is
found to be
s = dH = 101111101111 + 001111101110 = 100000000001,
which has weight 2. Since wt(s) < 3, we find that
e = [s, 0] = 100000000001, 000000000000
and conclude that the transmitted word is
w = e + d = 001111101110, 010010010010.
As another example suppose a codeword d=001001001101,101000101000 is received. The syndrome for this word is
s = dH = 001001001101 + 111000000100 = 110001001001,
69
which has weight 5. Proceeding to step 3 of the decoding algorithm, we compute
s + bi = 000110001100
s+
b2 = 011111000010
s+
b3 = 101101011110
s+
b4 = 001001100100
s+
b5 = 000000010010.
Since wt(s + b5 ) < 2, we find that
e = [s + b5 , 05] = 000000010010, 000010000000
and conclude that
w = e + d = 001001011111, 101010101000
was the transmitted codeword.
ME Extended Golay Codes. The extended Golay Code coding and decoding
algorithms are used to derive a codebook enabling 3-error correction. The extended
Golay Code enables us to transmit 212 different codewords, thereby enabling to code
212 messages. The procedure for constructing the ME Extended Golay Code is described in the following three steps: (i) generate the Extended Golay Codebook, (ii)
choose the minimum codebook, and (iii) map the source messages to the minimum
codebook according to the ME Coding algorithm. Hence, we can perform ME Coding
for the available codewords in the extended Golay Code to achieve energy efficient
transmission. Furthermore, if we have much less than 212 messages, we can simply
encode extensions of the source alphabet using the extended Golay Code, which might
enable further energy savings.
Golay Codes. Another interesting 3-error correcting code can be obtained by
puncturing C24, that is removing a digit from every word in C24. The same digit must
70
be removed from each word. We shall remove the last digit.
Decoding of Golay Codes. Let B be the (12 x 11) matrix obtained from the
matrix B by deleting the last column. Let G be the (12 x 23) matrix G = [I12, B].
The linear code with generator matrix G is called the Golay Code and is denoted by
C23.
The Golay Code has length n = 23, dimension k = 12, and contains 212 = 4096
codewords. The Golay Code has minimum distance of 7 and will correct any 3 bit or
less errors.
The decoding algorithm for the Golay Code is based on the decoding algorithm
of the extended Golay Code. This algorithm can be summarized as follows
1. Form dO or dl, whichever has odd weight.
2. Decode di (i is 0 or 1) using the extended Golay Code decoding algorithm to
codeword win C24.
3. Remove the last digit from w.
ME Golay Codes. Golay Codes can be used for 3 or less error correction. These
codes can be mapped to source messages using the ME Coding algorithm to provide
energy efficient reliable transmission. The procedure for constructing the ME Golay
Code is the same as that of ME Hamming Codes and ME extended Golay Codes: (i)
generate the Golay Codebook, (ii) choose the minimum codebook, and (iii) map the
source messages to the minimum codebook according to the ME Coding algorithm.
In the following section, we will extend the Hamming and Golay Codes to t-error
correcting Reed-Muller Codes, where t is any positive number.
8.2.3
Minimum Energy Reed-Muller Codes
Definition: Reed-Muller Codes. The r-th order Reed-Muller Code of length 2'
is RM(r, m), 0 < r K m. A recursive definition of the Reed-Muller Code is such that
1. RM(0, m) = {00...0, 11...1}, RM(m, m) = K
7
2. RM(r, m) = {(x, x + y)Ix E RM(r, m - 1, y E RM(r - 1, m - 1)}, 0 < r < m.
71
Codebook Generation of Reed-Muller Codes. Rather than using this definition of the Reed-Muller Code, we will use a recursive construction of the generator
matrix of RM(r, m), which we will denote by G(r,m). For 0 < r < m, we define
G(r,m) by
G(r, n)
=
G(r, m - 1)
L 0
G(r, m - 1)
G(,r - 1, m - 1)j
For r = 0 we define
G(0, m) = [11...1]
and for r = m, we define
G(m,m) =G(m
- 1, m)
0 ...
01
Characteristics of Reed-Muller Codes. Let us summarize the properties of
an r-th order Reed-Muller Code RM(r, m).
" Length L = 2"
" Distance d = 2 "-r
* Dimension k = E'=
(')
" RM(r - 1, m) is contained in RM(r, m), r > 0
" Dual code RM(m - 1 - r,m), r < M.
By arbitrarily choosing the values for (r,m), we can construct a Reed-Muller code
that can correct up to (2"-' - 1)/2 errors.
72
Let us now give some examples demonstrating the recursive construction of the
Reed-Muller Code. The generator matrices for RM(0, 1) and R(1, 1) are
G(0, 1) = [11]
and
G(1, 1)
1]
01
=
Let m = 2, then the length is n = 22 = 4 and for r=1,2 we have
G(1, 2)= [(1 1)
L 0
G(1, 1)]
G(0,1)]
1,
G(2,2)=
G(112)
L0001
Substituting for G(0, 1) and G(1, 1), we get
1111
G(1, 2)=
01011
and
0101
G(2,2) =
0011
0011
0001
For m = 3, m = 23 = 8, we have
G(0,3) = [11111111], G(3,3) =
G(2,3)
0000000
G(1, 3) =
G(j2)
0
G(1, 2)1
G(2, 3) =
G(22)
0
G(0, 2)
Thus, substituting for G(0, 2), G(1, 2) and G(2, 2), we get
11111111
G(1,3) =
01010101
00110011
00001111
73
G(2,2)
G(1, 2)
Considering the first order Reed-Muller Code RM(1, m), we realize that the code
has a dimension
2'
- m - 1, a distance of 4, and length
2
*. Thus, the first order
Reed-Muller Code is an extended Hamming Code.
The decoding of Reed-Muller Codes is a simple operation. However, explanation
of the decoding operation requires some detailed background information. Hence, decoding of Reed-Muller Codes is not included here. A detailed treatment of the decoding operation in Reed-Muller Codes can be found in [25,26]. The use of Reed-Muller
Codes for energy efficient information transmission is similar to that of Hamming
and Golay Codes. Basically, Reed-Muller Code is used to the construct a codebook
of codewords that are a given distance apart, and ME Coding is performed on this
codebook to achieve energy efficient transmission.
8.2.4
Implementation of Block Codes
Block codes can be implemented via two approaches: (i) Look-up Table Implementation, and (ii) Logic Circuit Implementation.
The look-up table implementation is illustrated in Figure 8-8. The look-up table
Information
Sequence
EPROM
Wireless
Transmitter
Block
Code
Figure 8-8: EPROM Encoder
implementation consists of an EPROM implementation of a mapping between the
information sequences and the codewords of a given error correcting code.
This
approach is shown in Figure 8-8, where the information sequences are mapped to the
coded bits in the encoder according to the ME Coding algorithm and then sent to
the wireless transmitter.
The logic circuit implementation is shown in Figure 8-9. The logic circuit im74
0 Wireless Transmitter
Input u
UO
- -.
U2
-*
- ...
k-
Message Register
U0
U
POO
AO
U
...k-1,0
V0
POiI
UI
11
U k-1
''
k-1,j
UO
**' Rn-k-l
UI
P1,n-k-1
U k-1
...P-1,n-k-1
Vn-k-W1
VI
Wireless Transmitter
Figure 8-9: Logic Circuit Encoder
plementation is such that the information bit sequence to be encoded is shifted into
the message shift register and simultaneously into the wireless transmitter. As soon
as the entire message has entered the message register, n - k parity check digits are
formed at the output of n - k modulo-2 adders. These parity check digits are then
serialized and shifted into the wireless transmitter.
8.3
Minimum Energy Convolutional Codes
Convolutional codes have been the most popular error correcting codes due to their
simplicity of decoding.
Convolutional Codes are different from block codes since
the codewords are generated via a finite state machine that introduces dependencies
among multiple codewords. Convolutional codes can be combined with ME Coding
given that some modifications to general convolutional code generating algorithm
are performed. Hence, the algorithm for generating Minimum Energy Convolutional
Codes can be stated as follow. First, check the existence of an (n, k, m) convolutional
code. Next, obtain the connection vectors for that (n, k, m) convolutional code. Then,
list the inputs sequences of 2 k binary information sequences. Continue by padding
the information sequences with km zeros to form a block code equivalent of the convolutional encoder. Then, obtain the generator matrix for this block code equivalent
75
of this convolutional encoder. Next, obtain the finite state machine response (codewords) for the input information sequences. Finally, map the information sequences
producing less high bits to source messages with higher probability.
The convolutional encoder is shown in Figure 8-10. The encoder consists of k shift
-
2) _
X(
x(k)
d ,0 s 2,0
d e
cr.0
A
s
m
2.0
2.0
rb
yt(I)
F
s 0in
0r,
s
o hesht0rg,
an
h
-d)
III
III(2)
tyt
g(.}
~jg(.)
____ ____
___
___
____
___
___
ttyt
(n)
Figure 8-10: Convolutional Encoder
registers. The stage inputs to the shift registers are connected in a certain pattern to
n multi-input modulo-2 adders. Since modulo-2 addition is linear, the convolutional
encoder is called a linear encoder. A segment of k binary information bits is shifted
serially or in parallel into the shift registers, and the n-bit encoder output bits are
sampled and transmitted serially.
8.4
Minimum Energy Burst-Error-Correcting Codes
In wireless communications, transmission errors usually occur in bursts. Hence, in
this section we investigate the means of combining the existing burst error correcting
codes with ME Coding.
The most popular method of correcting burst errors in
interleaving. In this section, interleaving will be combined with ME Coding.
76
*
Transmission
li
I
n-k
k
Figure 8-11: Interleaving
The algorithm for combining interleaving with ME Coding can be summarized
as follows. First, an (n, k) block code is selected. Next, minimum energy coding is
performed on this block code. Then, information transmission is accomplished by
selecting A codewords at a time and forming an array code of A - by - n matrix
size. These array codewords are then transmitted column-by-column as shown in
Figure 8-11. This way the burst errors are distributed into the individual codewords,
and the error correction capability of the short codes take care of the bit errors that
are formed within them.
77
Chapter 9
Energy-Efficient, High-Rate, and
Reliable Information Transmission
over Bandwidth-Limited Wireless
Channels
9.1
Minimum Energy Multilevel Coded Modulation
As it was discussed in the previous chapters, information transmission over wireless
channels is accomplished by representing different messages via unique signal sequences. For information transmission over noiseless channels, as it was described in
Chapters 3-7, there was no restriction on the individual signals in a given unique signal
sequence. For information transmission over noisy power-limited channels, as it was
described in Chapter 8, the restriction on the individual signals in the unique signal
sequences is introduced via error correcting codes. For information transmission over
bandwidth-limited channels the restriction on signal sequences will be introduced by
coded modulation schemes. The following is a formal definition of coded modulation.
78
Definition: Coded Modulation. Coded modulation is a method of information
transmission via unique signal sequences selected from a multilevel signal constellation, where individual signals in the signal sequences exhibit dependencies introduced
by block codes, trellis codes, or lattice codes.
Existing coded modulation schemes, as will be introduced in the following sections,
offer sets of usable codewords generated from a given signal constellation. Hence,
coded modulation can be combined with ME Modulation through the sequential steps
of codebook optimality and probabilistic optimality. Thus, codebook optimality is
a necessary condition for the overall optimality of the coded modulation and it is
decoupled from the source message probabilities.
Theorem 11: Codebook Optimality for Coded Modulation. Let S be a
q-message source and C be a signal constellation that generates a codebook W(C) of
q-codewords taken from the usable codewords set W 0 (C)
=
{w 1,w 2 ,
---,
Wq 0 -1,wq
0
},
qo < +oo; W(C) c W 0 (C). The necessary condition for a code (W(C), S) to
q
provide the minimum average energy per message R =
Z
1 Pej is that the codebook
W(C) is a minimum codebook of W 0 (C).
Proof. See Appendix.
9.2
Minimum Energy Block-Coded Modulation
Block-Coded Modulation is a digital data transmission technique that introduces
dependencies among the signals in the transmission signal sequences via multiple
Block Codes as shown in Figure 9-1. In Figure 9-1, first the information sequence is
divided into information sub-sequences of length {ki, k2 , k 3 , k4 }. These information
sequences are fed into n-bit Block Codes having distances {di, d2 , d3 , d4 }. These Block
Codes produce n-bit codewords that form codeword arrays as shown in Figure 9-1.
Columns of these codeword arrays are mapped to signals from a multilevel signal
constellation and transmitted over the wireless channel.
Exemplary Block-Coded
Modulation schemes are given in [27-29].
Block-Coded Modulation can be combined with ME Modulation in the follow79
n,kj,dj)
Array Code
Signal Conte1ation
n,k2,d)Informafion
Sequence
-
Encoder
++ 4
+
, k3,d3-
n, k,d3)
(n, k4 ,d4)
Figure 9-1: Block-Coded Modulation
ing manner. The set of all available codeword arrays and the corresponding signal
sequences can be constructed. Each unique codeword array is mapped to a unique
signal sequence. The codeword arrays are listed in ascending order of energy consumption and mapped to the source messages in descending order of probabilities.
This algorithm is referred to as Minimum Energy Block-Coded Modulation.
9.3
Minimum Energy Trellis-Coded Modulation
Trellis-Coded Modulation is a digital data transmission technique that introduces
dependencies among the signals in the transmission signal sequences via convolutional
codes as shown in Figure 9-2 [30-32]. In this figure, the m incoming bits are divided
into two groups, one of length p and the other of length m - p. The first group of
p bits are fed into a convolutional encoder to produce coded bits that will pick a
80
Mn-P
M
mM
.n
Signal
Mapping
In
Select Signal
xn
Sn
from Subset
x
Convolutional
Encoder
Select Subset
Figure 9-2: Trellis Coded Modulation
subset of the signal constellation and the second group of length m -p is the uncoded
information bits that are directly mapped to signal points. The mapping between
the coded bits coming out of the convolutional encoder and the subsets of the whole
constellation is done via the Ungerboeck rules called mapping by set partitioning. A
comprehensive treatment of these schemes can be found in the references [31,32].
The operation of the convolutional encoder is described by a trellis diagram as
shown below. In the trellis diagram, the states represent the contents of the memory
Start State
End State
Allowable
Codeword
Paths
States
x+p
Figure 9-3: Trellis Diagram
81
(shift registers), the branches are labeled with the output bits of the convolutional
encoder. At each stage the output is a function of the encoder state and the input bits.
The output bits are mapped to subsets of the signal constellation via the well-known
Ungerboeck rules.
Signal sequences or codewords in a Trellis Code are represented by the path labels
through the trellis in Figure 9-3. Each signal sequence or codeword is determined by
the input information sequence and the initial state of the convolutional encoder (i.e.,
initial memory content of the convolutional encoder). Hence, codewords in a Trellis
Code are not produced by unique signal sequences and there are dependencies among
different codewords due to the initial memory content of the convolutional encoder.
This form of Trellis Codes is not suitable for combination with ME Modulation that
requires that unique signal sequences be determined solely by unique information
sequences. Hence, Trellis Codes have to be modified into Block Codes for combination
with ME Modulation.
There are four well-known techniques of converting Trellis
Codes to Block Codes. These techniques are direct truncation, zero-tail termination,
tail biting, and generalizedtail biting as explained in [33]. Using these modified Trellis
Codes, unique signal sequences can be constructed for unique information sequences
and signal sequences consuming less energy can be mapped to source messages of
higher probability. This algorithm is referred to as Minimum Energy Trellis-Coded
Modulation.
9.4
Minimum Energy Lattice Codes
Lattice Codes are frequently used for high rate information transmission [34,35]. Lattice Codes of different dimensions with their accompanying rules of construction are
contained in the comprehensive reference [36]. Some of the most famous lattices are
Z, A 2 , A 3 , D 4 , Dz, E 6 , E 7 , E 8 , K 12 ,
i 16 ,
and A 2 4. Recently, E8 lattice modems, Leech
Lattice modems, and Barness-Wall Lattice modems have appeared in the market.
The following example demonstrates how Lattice Codes can be combined with ME
Modulation.
82
Assume a Lattice Code that receives L information bits of binary data and encodes
it into eight values S1, S2, ..., S8 identifying it. The eight values are selected from pos-
itive, negative, and zero integer values. The method of selection of these eight values
defines a lattice code of dimension 8. The resulting set of 8-tuples (Si, S2, ..., S8) is
called the Lattice Code of dimension 8. In one such case the eight values S1, S2,
... ,
S8
are chosen such that they are either (i) all even or all odd integers or (ii) the total
of the integral values must be Omodulo4. These rules in fact describe the densest
known packings in 8 dimensions. The 8-tuples obeying these rules can be stored in
an EPROM as a look-up table and transmitted via a QAM system two coordinates
at a time in the form of 4 signals. Each of the two coordinates would modulate the
amplitudes of the sine and cosine waveforms to generate the QAM signals. The ME
Modulation for these 8-tuples can be formed as follows. First, the minimum codebook
of the Lattice Code must be generated according to the Lattice Code rules. Then,
the codewords in this codebook are ordered in ascending order of energy consumption
and mapped to source messages in descending order of probabilities. This algorithm
is referred to as Minimum Energy Lattice Codes.
9.5
Implementation
The coded modulation schemes described in this chapter can be implemented via the
QAM system
shown in Figure 9-4. In this figure, analog data (message) is converted
sin fc
Message
----0 ROM-1
SeA/uen ce
-- Mapper
-:Parallel
Sterial -Rgster
Serial
ROM-2
Parallel
Signal
Register -,Mapper
- A/D
LPF
+
ZA/
Z LPFC
Transmitter
0
Figure 9-4: Coded Modulation System
to digital data via an A/D converter. This digital data is shifted out of the A/D
converter in parallel and fed into the address lines of ROM-1 containing a mapping
83
between the digital representation of the messages and the codewords of the coded
modulation scheme. The L-bit codewords are shifted out in parallel and then in
turn converted to serial format via a parallel-to-serial converter. Then, a serial-toparallel converter converts groups of log2 M bits into parallel format. These log2M
bits select signal points from the constellation in ROM-2. The bit sequence-to-signal
point mapping is such that the memory contains a combination of certain coordinates
(X, Y) that represent the amplitudes of the in-phase and quadrature components
of the QAM signal. ROM-2 shifts these two amplitude values out in parallel and
generates an analog value for these amplitudes via a D/A converter. These values are
coupled with low pass filters that are in turn coupled to hardware modulator circuits.
The modulated waveforms are added to form the transmitted QAM waveform and
transmitted over the wireless channel.
84
Chapter 10
Simulation Results
This research developed coding and modulation algorithms that are applicable to
all energy critical applications. As an immediate application, however, ME Coding
is developed for patient monitoring systems in health-care applications. In patient
monitoring systems, effective technologies for remotely and continuously monitoring
the patients is in demand. To answer this demand, a wearable health monitoring
system in the form of a finger ring sensor is under development at the d'Arbeloff Laboratory for Information Systems and Technology at MIT [37]. The ring sensor shown
in Figure 10-1 senses the physiologic data of remotely located patients, transmits the
data wireless to a home computer that is connected to a doctor's office or a nursing
home through the Internet or some other health-care network. Since the ring sensor
Figure 10-1: Finger Ring Sensor
85
is designed to operate wireless for non-invasive monitoring, energy efficient operation
is a key design constraint. Hence, the achievable energy savings via ME Coding is
demonstrated by comparing the energy performance of the current ring prototype
using the ASCII Code with the future prototype that will use ME Coding.
Figure 10-2 shows some typical data obtained from the ring sensor. This data
Figure 10-2: Ring Sensor Plethysmograph
contains a rich variety of information from which a class of cardiac and circulatory
disorders can be detected.
In the current prototype of the ring sensor, this data
is transmitted wireless using the standard ASCII Code combined with on/off keying. ASCII Code encodes the digitized ring signal into eight-bit binary codewords
in binary coded decimal format. This scheme being unaware of the relative energy
consumptions of the high bits and the low bits results in inefficient operation in terms
of energy. Therefore, the attainable energy saving by replacing ASCII Code with ME
Coding is simulated .
The simulation is performed as follows. First, samples of long-duration ring sensor
data are collected from different patients, and a statistical analysis to obtain the
source message probabilities is performed. In this case, the source messages are the
data values sampled from the ring sensor plethysmograph. Then, these messages are
encoded using ME Coding. Next, a random set of data is collected from a random
86
patient and using ii = 1 Pini, the energy performance of ME Coding and ASCII
Code is simulated on this data set. Since the two codes use the same transmission
hardware and operating conditions, the power consumption coefficient Ct and the
bit period
tb
are the same for both codes.
The only difference results from the
two codes having different average number of high bits per message 'h. Figure 10-3
illustrates the results of one such simulation. This figure shows the battery energy
Figure 10-3: Battery Energy: (1) ASCII Code, (2) ME Protocol
left in two different ring sensors, one employing ASCII Code and the other employing
ME Coding, after transmitting the same amount of data. In this case, it is seen that
the performance improvement via ME Coding is almost 300%. This is due to the
fact that the ASCII Code represents the most frequent messages using large high bit
codewords. Hence, ME Coding, which will be employed in the future prototype, will
code these most frequent messages using less high bits codewords, thereby enabling
significant energy savings.
87
Chapter 11
Conclusions
This thesis investigated the problem of energy-optimal information transmission over
wireless channels. The major findings and conclusions of this investigation are as
follows.
Basic Idea.
Energy-optimal information transmissionis achieved by representinghigher-probability
messages via lower-energy signal sequences.
Energy-Optimal Data Compression.
" For low data rate transmission, ME Coding is proposed as a new memoryless
coding algorithm that provides energy optimal codes for sources with known
statistics.
* The conditions and parameters determining the optimality of ME Coding are
identified and a lower bound on optimal performance is derived.
" Concatenation is introduced as a technique that improves the memoryless performance via a simple memory mechanism.
" Energy efficiency improvement via concatenation is clearly explained and the
conditions of improvement are determined.
88
Energy-Optimal Source Coded Modulation.
" For high data rate transmission, ME Modulation is proposed as a new modulation algorithm that provides energy optimal source coded modulation schemes
for sources with known statistics.
" Extensions of ME Modulation are investigated for improving the energy efficiency and the energy efficiency improvement by extensions is clearly explained.
Energy-Optimal Information Transmission over Noisy Wireless Channels.
" For power-limited channels, energy-optimal information transmission is accomplished via the use of error correcting codes. These codes include block codes,
convolutional codes, and burst error correcting codes.
* For bandwidth-limited channels, energy-optimal information transmission is accomplished via the use of coded modulation techniques. These techniques include block coded modulation, trellis coded modulation, and lattice codes.
In conclusion, energy-optimal information transmission can be accomplished via
proper coding and modulation.
89
Bibliography
[1] C. E. Shannon. A mathematical theory of communication. Bell Systems Technical Journal,27:379-424 and 623-657, July and October 1948.
[2] C. E. Shannon. Communication in the Presence of Noise. Proceedings of the
IRE, 37:10-21, January 1948.
[3] J. M. Rulnick and N. Bambos. Mobile Power Management For Wireless Communication Networks. Wireless Networks, 3(1):3-14, 1997.
[4] P. Lettieri, C. Fragouli, and M. B. Srivastava. Low Power Error Control for
Wireless Links. In MobiCom 1997. Proceedings of the Third Annual ACM/IEEE
International Conference on Mobile Computing and Networking, pages 139-150,
1997.
[5] P. Agrawal, B. Narendran, J. Sienicki, and S. Yajnik. An Adaptive Power Control
and Coding Scheme For Mobile Radio Systems. In Proceedings of the 1996 IEEE
InternationalConference on Personal Wireless Communications,pages 283-288.
IEEE, Feb 19-21 1996.
[6] G. D. Forney, R. G. Gallager, G. R. Lang, F. M. Longstaff, and S. U. Qureshi.
Efficient Modulation for Band-Limited Channels.
IEEE Journal on Selected
Areas in Communications, 2(5):632-647, September 1984.
[7] F. R. Kschischang and S. Pasupathy. Optimum Nonuniform Signaling for Gaussian Channels. IEEE Transactions on Information Theory, 39(3):913-929, May
1993.
90
[8] G. D. Forney. Symbol Coding Apparatus, 1985. US Patent 4, 562, 426.
[9] G. R. Lang, G. D. Forney, S. Qureshi, F. M. Longstaff, and C. H. Lee. Signal Structures with Data Encoding/Decoding for QCM Modulations, 1985. US
Patent 4, 538, 284.
[10] R. G. Gallager. Source Coded Modulation System, 1986. US Patent 4, 586, 182.
[11] A. R. Calderbank and L. H. Ozarow. Nonequiprobable Signaling on the Gaussian
Channel. IEEE Transactions on Information Theory, 36(4):726-740, July 1990.
[12] G. D. Forney. Principles of Digital Communications. Unpublished Course Notes,
1999.
[13] C. C. Cahn. Performance of Digital Phase Modulation Communication Systems.
IRE Transactions on Communication Systems, CS-7:3-6, May 1959.
[14] C. C. Cahn. Combined Digital Phase and Amplitude Modulation Communication Systems. IRE Transactions on Communication Systems, CS-8:150-154,
1960.
[15] J. C. Hancock and R. W. Lucky. Performance of Combined Amplitude and Phase
Modulated Communication Systems. IRE Transactions on Communication Systems, CS-8:232-237, 1960.
[16] C. N. Campopiano and B. G. Glazer. A Coherent Digital Amplitude and Phase
Modulation Scheme. IRE Transactions on Communication Systems, CS-10:9095, 1962.
[17] J. M. Wozencraft and I. M. Jacobs. Principles and Communication Engineering.
Wiley, 1965.
[18] J. Salz, J. R. Sheehan, and D. J. Paris. Data Transmission by Combined AM
and PM. Bell System Technical Journal,50(7):2399-2419, 1971.
91
[19] K. Kawai, S. Shintani, and H. Yanagidaira. Optimum Combination of Amplitude
and Phase Modulation Scheme. IEEE International Conference on Communications, pages 29-6-29-11, 1972.
[20] C. M. Thomas. Amplitude Phase-Keying with M-ary Alphabets: A Technique
for Bandwidth Reduction. International Telemetering Conference Proceedings,
pages 289-300, October 1972.
[21] M. K. Simon and J. G. Smith. Hexagonal Multiple Phase-and-Amplitude-ShiftKeyed Signal Sets. IEEE Transactions on Communications, COM-21(10):11081115, October 1973.
[22] G. J. Foschini, R. D. Gitlin, and S. B. Weinstein.
Optimization of Two-
Dimensional Signal Constellations in the Presence of Gaussian Noise.
IEEE
Transactions on Communications, COM-22:28-38, 1974.
[23] P. M. Fenwick.
Huffman Code Efficiencies for Extensions of Sources.
IEEE
Transactions on Communications, 43(2-4):163-165, Feb-Apr 1995.
[24] R. W. Hamming. Coding and Information Theory. Prentice-Hall, 1986.
[25] J. Baylis. Error-CorrectingCodes: A Mathematical Introduction. Chapman and
Hall Mathematics, 1998.
[26] S. G. Wilson. Digital Modulation and Coding. Prentice Hall, 1996.
[27] H. Imai and S. Hirakawa. A new multilevel coding method using error-correcting
codes. IEEE Transactionson Information Theory, IT-23(3):371-377, 1977.
[28] S. I. Sayegh. A class of optimum block codes in signal space. IEEE Transactions
on Communications, Com-34(10):1043-1045, October 1986.
[29] R. M. Tanner. Method and apparatus for combining encoding and modulation.
Patent Number 4,882,733, November, 21 1989.
[30] G. Ungerboeck. Channel coding with multilevel/phase signals. IEEE Transactions on Information Theory, 28(1):55-67, January 1981.
92
[31] G. Ungerboeck. Trellis Coded Modulation with Redundant Signal Sets Part I:
Introduction. IEEE Communication Magazine, 25(2):5-11, February 1987.
[32] G. Ungerboeck. Trellis Coded Modulation with Redundant Signal Sets Part II:
State of the Art. IEEE Communication Magazine, 25(2):12-21, February 1987.
[33] H. H. Ma and J. K. Wolf. On tail biting convolutional codes. IEEE Transactions
on Communications, COM-34(2):104-111, February 1986.
[34] F. M. Longstaff and G. R. Lang. Data transmission with block coding. Patent
Number 4,630,288, December, 16 1986.
[35] F. M. Longstaff. Block coding into 24 coordinates and detection of transmitted
signals. Patent Number 4,733,401, March, 22 1988.
[36] J. H. Conway and N. J. A. Sloane.
Sphere Packings, Lattices and Groups.
Springer, third edition, 1999.
[37] B.-H. Yang, S. Rhee, and H. H. Asada. A Twenty-four Hour Tele-nursing System
using a Ring Sensor. In Proceedings of the 1998 IEEE InternationalConference
on Robotics and Automation, volume 1, pages 387-392. IEEE, 1998.
93
Appendix
Proof of Theorem 1
Let i and j be arbitrary integers such that 1 < i < j
q. From equations (3.1)
and (3.2),
Pi
Pj
and
ni<nj.
(11.1)
Consider the two terms involved in f = E 1 Pini,
Old:
Pini + Pjnj.
Interchanging the codewords for the i-th and j-th messages yields
New:
Pinj + Pjni.
Subtracting Old from New and using (11.1), the net change in the average number
of high bits due to this re-assignment becomes
New - Old = (Pi - Pj)(nj - ni) > 0.
Since the average number of high bits, h, does not decrease for any interchange of
codewords, this coding algorithm provides optimal codes for a given codebook with respect to energy consumption.
0
Proof of Theorem 2
Let ni < n 2 5
through
wq
...
:5 ni
5
n
be the number of high bits in the codewords w,
involved in the minimum codebook Wmin. The proof is given by showing
that replacing an arbitrary codeword wi involved in Wmin with an arbitrary codeword
wj not involved in Wmin does not decrease the minimum average number of high bits
A. Since 1 <i
q <j
gq and ni
ni+1 < ... < nq 5 nj, the following inequalities
94
hold
nj+j - ni
0,
ni+2-
Multiplying Pi, Pi+1 ,
... ,
nj~i+
---,
,
nq-nq-1
0, nj -
q > 0-
(11.2)
Pq by the above inequalities individually and summing them
yield
Pi(nj+ 1 - ni) + ... + Pq-(nq- nq-) + P(nj - nq)
0
or
Pini +
Adding Pin1
P i n, +
+
...
...
Pii+i+ ... + P-inq + Pqnj.
+ Pqflnq-1 + Pqnq
+ Pjinj-1 to both sides,
... Pi-ini- 1 + Pinj+...
+ Pq-ilnq1 + Pnq <
Pin+... + Pjini_ + Piji + ... + Pqirnq + Pqnj.
(11.3)
The left hand side gives ii for the minimum codebook, while the right hand side gives n
for a non-minimum codebook, where wi is replaced by wj. The former is always smaller
than or equal to the latter. Therefore, the codebook must be a minimum codebook in
order to minimize ii.
Derivation of ME Coding Optimality Bound
Consider the binomial bound given by
1=k-i1.
(k
+ 1 L i=1
k+1
95
(11.4)
Using this inequality, we define numbers Q, through Qq,
(11.5)
Qi=
L
K(k+1
k)
which sum to 1, E= 1 Qi = 1. These numbers may be regarded as a probability
distribution, hence, the following Gibbs inequality applies:
Q2.
q
< 0.
\Ijg,
(11.6)
Expanding the logarithmic term and dividing both sides by log 2 k yields the source
entropy separated from the rest of the above inequality,
q
1
q
(-P)
Pi1o9k
k
1
>Pi1og9k
(11.7)
Substituting (11.4) and (11.5) into (11.7) yields the lower bound to the average number of high bits, ft = E
Pni,
ft >
Hk - lOgkBk,
(11.8)
where Bk is a characteristic parameter of the codebook, referred to as codebook capacity, given by
q
Bk=
k.
(11.9)
i=1
Inequality (11.8) can also be expressed in an exponential form,
k(Hk -)
B
Bk
Hence, this completes the derivation.
96
<1.
(11.10)
Proof of Theorem 3
The average number of high bits for the ME Code is given by
ZE
l
!
Pini. Let
the p-th concatenation be performed via direct concatenation, where the number of
high bits involved in each new codeword is given by nil + ni +
2
probability by Pi P
...
2
...
+ ni,, and its
Pi,. Therefore, the average number of high bits for the p-th
direct concatenation is given by
q
S=
q
...
E
Pi2
EPi
i1=1
...
Pip (nil + ni2
+
..
+ nip)(11.11)
ip=1
Separating the terms including nil from others yields
IP
Pinil)(
=
i1=1
i2=1
Pi
i1=1
i2=1
...E
Pi2 --ip=1
Pi
(ni2 +
..
+ nip))
q
+ (ini)
E ... EP2
i=1
+ E
ip=1
q
q
=
Pi2 --- Pi)
...E
i2=1
.
Pi(ni
ip=1
2
+
... +nip)
Repeating this process for i 2 through ip yields
q
59p = p Eini
= pfjs.
(11.12)
i=1
Hence, RME =
HP/p.
This completes the proof.
Proof of Theorem 4
From Theorem 3 it follows that for any source S1 and any codeword length L, direct
concatenation results in codes having hP/p = iiME. Let nrmax be the maximum number
of high bits in N 1 ; nma ;> ni, 1 < i < q. Then, among all unused codewords, the
unused codeword with the minimum number of high bits must have either nma or
(nma + 1) high bits (This can be seen in Figure 4-2). When the p-th concatenation
is generated according to direct concatenation, the resulting codebook includes a
codeword with p x nma high bits, that is, the largest number of high bits. Since the
97
minimum number of high bits in unused codewords, umin, is either nmax or (nmax + 1),
this unused codeword has fewer high bits than that of the used codeword with p x nmax
high bits; umin < p x nmax. Note that, when q = L +1, nmax = 1 and Umin = 2, hence
this inequality does not hold for p = 2. Thus, for (q, L) pairs, where (i) q 0 1, (ii)
q 0 L + 1, and (iii) q <
2 L,
the codebook for direct concatenation is not minimum
in any concatenation order. Hence, the unused codewords having less high bits (nmax
or (nmax + 1)) can be incorporated into the codebook to reduce the total number of
high bits
ni + ... + nq-1+ (nmax + 1) < ni + ... + nqP-1+ pxnmax
(11.13)
Multiplying this equation through by the source message probabilities shows that
concatenation always improves the energy efficiency of the original unconcatenated
ME Code
ft = Pn + ... + PqplnqP1 + Pqp(nmax + 1)
< Pin +
...
+
Pq -1nqP
+ Pqp (pxnmax)
=
hdirect
=
PhME
This is true for any p, such that p > 2. Hence, this proves Theorem 4.
I
Proof of Theorem 5
We prove Theorem 5 by first showing that GMP does not hold if the second condition
does not hold, and then by showing that if the second condition does not hold, i will
decrease for some concatenation order 2p since Theorem 1 will not hold for direct
concatenation.
The first condition of GMP requires that if two codewords i and j have equal
number of high bits, ni = nj, then the messages represented by those codewords must
have equal probabilities P = Pj V(i, j). If this condition does not hold, that is if for
some ni and nj, ni = nj, and Pi 5 Pj, then P(ni + s)P(nj - s) 0 P(ni)2 for at least one
(i,j, s). This suggests that the first condition is a necessary condition for the second
98
condition and GMP does not hold whenever the second condition does not hold.
The second condition does not hold either if (i) P(ni - j) = aU, P(ni) = a + 6,
and P(ni +j) = a/(b]) or if (ii) P(ni - j) = ab, P(ni) = a -6,
and P(ni+j) = a/(bJ),
or if (iii) P(ni - j) = abi +6, P(ni) = a, and P(ni + j) = a/(bi), or if (iv) P(ni - j) =
ab' - 6, P(ni) = a, and P(n + j) = a/(bi), or if (v) P(nj - j) = abi, P(ni) = a, and
P(n +j) = a/(bJ) +6, or if (vi) P(nj -j)
= abi, P(ni) = a, and P(nj +j) = a/(bJ) -6.
We should prove that for all of these 6 cases, there will be a decrease in ii for some
2p. The proof is accomplished by showing that for direct concatenation, Theorem 1
is violated for some concatenation order 2p. Since the proofs for different cases are
similar, we include the first two cases only.
For the 2p-th direct concatenation of case 1, a message having a probability (a +
6)2P is represented by a codeword having pni + pni = 2pni high bits, while a message
having a probability of (ab)P+'(a/(bi))P-1 is represented by a codeword having (p +
1)(ni - j)+(p -1)(ni+j)
= 2pni -2j high bits. It is easily seen that 2pni > 2pni -2j.
Taking the ratio of the probabilities of the two messages, we see that the ratio is less
than 1 for some 2p
(abi)P+1(a/(bi))P-1
a
2
a
2
(a + 6)2P (a+6 P K +6 -6)K±6)
a+ J
2p-2
b2 < 1
'
(11.14)
since
2p-2
<< 1
a+6
(11.15)
for some large p. Thus, the two message-to-codeword assignments are swapped to
satisfy coding optimality of Theorem 1. Hence, if GMP does not hold through case
1, ii will decrease for some concatenation order 2p.
For the 2p-th direct concatenation of case 2, a message having a probability (a 6)2P is represented by a codeword having pni + pni = 2pni high bits, while a message
having a probability of (abi)P-1(a/(bi))P-1 is represented by a codeword having (p 1)(ni -j)+(p+1)(ni+j) = 2pni+2j high bits. It is easily seen that 2pni < 2pni+2j.
99
Taking the ratio of the probabilities of the two messages, we see that the ratio is less
than 1 for some p
a
(ab')P-1(a/(b'))P+1
(a -
2p-2
a-
a-
J)2p
a
2
b->1
1-6
since
2p-2
6)
a>>(11.17)
for some large p. Thus, the two message-to-codeword assignments are swapped to
satisfy coding optimality of Theorem 1. Hence, if GMP does not hold through case
2, i will decrease for some concatenation order 2p.
Hence, this completes the proof.
E
Proof of Theorem 6
We prove that non-zero mean constellations are not optimal, since translations of
these constellations providing zero-mean constellations reduce the average energy per
signal.
Suppose C' = {c', c',..., c'} is a constellation having signal probabilities P =
{P', P2,..., P, } resulting in a mean signal a given as
M
a=
Pi'c(,
(11.18)
i=1
where ci represents the coordinates of the i-th signal. Thus, the average energy of
this constellation can be written as
E[ic
112
]
= E [(c', c()].
(11.19)
Let C be a translate of the signal constellation C', where all the signal points in C'
are shifted according to the rule ci = c( - a. This translation will result in C being a
100
zero-mean constellation. Thus, substituting this expression into equation (11.19) and
simplifying we obtain the following result.
E [(c', c')] = E [(ci + a, ci + a)]
(11.20)
E [(ci, ci)] = E [IIcII12] + E [(ci, a)] + E [(a, ci)] + I1a1|2
(11.21)
E [(c', ci)] = E [IIcII 2] + 11a|12
(11.22)
since E [(ci, a)] and E [(a, ci)] are both equal to zero due to C being a zero-mean
constellation. Equation (11.22) shows that the average energy of a zero-mean constellation C is always smaller than the average energy of its translate non-zero mean
constellation. Thus, the necessary condition for the optimality of a non-equiprobable
constellation is for it to be zero-mean.
Proof of Theorem 7
We will prove Theorem 7 by showing that any interchange of codeword-to-message
assignment does not decrease the average energy per message. Let i and j be arbitrary
integers such that 1 < i < j 5 q. From equations (6.1) and (6.2),
Pi > Pj
and
Consider the two terms involved in i =
Old:
ej < ej.
Pej,
Pej + Pjej.
Interchanging the codewords for the i-th and j-th messages yields
New :
Pej + Pjei.
101
(11.23)
Subtracting Old from New and using (11.23), the net change in the average energy
per message due to this re-assignment becomes
New - Old = (Pi - Pj)(ej - ej) > 0.
Therefore, the average energy per message, fi, does not decrease for any interchange of
codewords. Hence, this coding algorithm provides optimal source coded modulation
schemes for a given signal constellation with respect to energy consumption.
L
Proof of Theorem 8
Let el 5 e 2 5
Wq
...
< ej 5
... < eq be the energy in the codewords w1 through
involved in the minimum codebook Wmin(C). The proof is given by showing that
replacing an arbitrary codeword wi involved in Wmin (C) with an arbitrary codeword
wj not involved in Wmin (C) does not decrease E. Since 1 < i < q < j <q 0
ej :5 ej+j
and
...: eq < ej, the following inequalities hold
e±i+
- ej > 0,
ei+2 - ei+ 1 > 0,
Multiplying P, Pi+,,
...
..,
eq -eq-1
0,
ej - eq > 0.
, Pq by the above inequalities individually and summing them
yield
Pj(ej+j - ej) + ... + Pq-(eq - eql) + Pq(ej - eq)
0
or
Pie + ... + P-ieq-i+ Pqeq 5 Pjej+1 + ... + Pq-ieq + Pej.
Adding Pie1 +
...
+ Pj1 ei- 1 to both sides,
P i e, + ... Pi_1ei-1 + Piei+... + Pq-ieq-_ + Pqeq <
Pie,+ ... + Pi-jej- 1 + Pe i+ +
102
...
+ Pqieq + Pej.
The left hand side gives the average energy per message for the minimum codebook,
while the right hand side provides the average energy per message for a non-minimum
codebook, where wi is replaced by wj. The former is always smaller than or equal to
the latter. Therefore, the codebook must be a minimum codebook in order to minimize R.
L
Proof of Theorem 9
The average energy per message for the ME Modulation is given by E9
- E5
= Pjej.
Let the p-th concatenation be performed via direct concatenation, where the energy
involved in each new codeword is given by ej, + ei +
... + ei,, and its probabil-
2
ity by PiPi
2
...
Pip. Therefore, the average energy per message for the p-th direct
concatenation is given by
q
P=
E
q
...
ii=1
Pi 2
EPi
...
Pip (eil + ei2 +
+ ei)
...
ip=1
Separating the terms including ej, from others yields
q
EP =
(EPi
1
.E...
ej,
=1
12=1
q
Pi2 .--Pi)
+
ip=i
q
q
i2=1
... E
Pi2
ip= 1
Pi
i1=1
.
Pip(ei 2 +
...
+ ei,)
q
=
(Piei)
i=1
... Pi2
H
i2=1
...Pi(ei2
+
ip=1
... + ei,))
Repeating this process for i2 through ip yields
q
EP = p
Pe= PMEE.
i=1
Simplifying gives
P
--- ME.
This completes the proof.
0
103
Proof of Theorem 10
Let ni < n2
through
wq
...
_
<
ni 5 ...
nq be the number of high bits in the codewords w,
involved in the minimum codebook Wmin of an error correcting code. The
proof is given by showing that replacing an arbitrary codeword wi involved in Wmin
with an arbitrary codeword wj not involved in Wmin does not decrease the minimum
average number of high bits ii. Since 1 < i < q < j
qO and ni 5 ni+1 < ... < nq
nj, the following inequalities hold
ni+1 - ni
0,
ni+ 2 - ni+>
Multiplying Pi, Pi+,,
...
0,
... ,
nq -nq-1
_ 0,
nj - nq
0.
(11.24)
, Pq by the above inequalities individually and summing them
yield
Pi(ni+1 - ni) + ... + Pq_1(nq - nq-_)+ Pq(nj -n)
0
or
Pini +
Adding Pin, +
+ Pq-inq_
...
...
+ Pqnfq 5
ini+1 + ... + Pqifnq + Pnj.
+ Piini- 1 to both sides,
Pin, + ... Pi-ini_1 + Pini+...+ Pq-in-li + Pnq <
Pin,+ ... +
i-ini_1 + Pini+1 + ... + P-inq + Pnj. (11.25)
The left hand side gives ii for the minimum codebook of the error correcting code,
while the right hand side gives i for a non-minimum codebook of the error correcting
code, where wi is replaced by wj. The former is always smaller than or equal to the
latter. Therefore, the codebook must be a minimum codebook to minimize i.
104
L
Proof of Theorem 11
Let el < e2 5
Wq
...
5 ej <
...
< eq be the energy in the codewords w, through
involved in the minimum codebook Wmin(C) of the coded modulation scheme.
The proof is given by showing that replacing an arbitrary codeword wi involved in
Wmin(C) with an arbitrary codeword wj not involved in Wmin(C) does not decrease
q <j :5q and ej 5 e1 i~ < ... < eq 5 ej, the following inequalities
E. Since 1< i
hold
e±i+
- ej
0,
ei+ 2 - ei+1 ! 0,
...,
eq - eq-1
>
0,
ej - eq> 0.
Multiplying Pi, P+1 , ..., Pq by the above inequalities individually and summing them
yield
Pi(ei+l - ei) +
...
+ Pq_1(eq - eq_1) + P(ej - eq) _> 0
or
Pje +
...
+ Pq-ieq_1 + Pqeq 5 Piei+l +
Adding Pie1 +
...
+ Pi_1ei_1 to both sides,
...
+ Pq-ieq + Pqej.
Pie1 + ... Pi- 1e-i1 + Piei+...+ Pq-ieq-1 + Pqeq
Pie,+ ... + Pj1 ej_ 1 + Piej+j + ... + Pqieq + Pej.
The left hand side gives the average energy per message for the minimum codebook of
the coded modulation scheme, while the right hand side provides the average energy
per message for a non-minimum codebook, where wi is replaced by wj. The former
is always smaller than or equal to the latter. Therefore, the codebook must be a
minimum codebook in order to minimize E.
105
0