LECTURE 20
Last time:
• Gaussian channels with feedback
• Upper bound to benefit of capacity
Lecture outline
• Multiple access channels
• Coding theorem
• Capacity region for Gaussian channels
Reading: Section 14.1-14.3.
Multiple access channels
Several users share the same medium
What is the right metric? Joint informa­
tion?
User 1 has rate R1 and user 2 has rate R2
How do we relate them to mutual informa­
tion?
Model: Yi = X1i + X2i + Ni
Liao and Ahlswede (independently, 1972)
R1 ≤ I(X1; Y |X2)
R2 ≤ I(X2; Y |X1)
R1 + R2 ≤ I((X1, X2); Y )
Coding theorem
mi: message sent by user i
� i: decoded message for user i
m
P e1 (P e2): probability that the decoded codeword for user 1 (2) is different from that
sent by user 1 (2) while the decoded codeword for user 2 (1) is the same as the one
sent by user 2 (1) (such errors will be de­
noted as errors of type 1 (2))
P e1,2: probability that the decoded codewords for both users 1 and 2 are different
from those sent by those users (such an
error will be denoted as error of type 3)
We begin by bounding the probability of er­
ror with an exponential argument and then
we explore the behavior of that argument
Coding theorem
We first consider errors of type 1 - results
for errors of type 2 can be derived analo­
gously
We denote P e1,m1,m2 the probability that
an error of type 1 occurs conditioned on
messages m1 and m2 being sent
Using the overbar to denote expectation
P e1,m1,m2
� � �
= y x1 x2 fX 1 (x1) fX 2 (x2) fY |X 1,X 2 (y|x1, x2)
P
��
�
�1 =
� 2 = m2|y, x1, x2
� m1) ∩ m
(m
���
dx2dx1dy.
Using the union bound, we obtain
P
��
� 1 �= m1) ∩ m
� 2 = m2|y, x1, x2
(m
⎧
⎨ �
⎩
�
P
m=
� m1
∀0 ≤ ρ ≤ 1.
��
�
���
≤
� 1 = m) ∩ m
� 2 = m2|y, x1, x2
(m
⎫ρ
���⎬
⎭
Coding theorem
Using arguments similar to those for the
single user strong coding theorem, we can
establish
∀ρ ∈ [0, 1], fX 1 (x1) and fX 2 (x2) probabil­
ity density functions for X1 and X2, respec­
tively, we have
P e1,m1,m2 ≤
�
exp −N
�
−ρR1 + E01
�
ρ, fX 1 (x1) , fX 2 (x2)
���
where we have defined,
�
�
1
E0 ρ, fX 1 (x1) , fX 2 (x2)
�� �
ln
��
x
y x2
=−
1
N
fX 2 (x2)
�
fX 1 (x) fY |X,X 2 y |x, x2
�
1
1+ρ
⎫
⎬
�1+ρ
dx
dx2dy
⎭
Coding theorem
It now suffices to determine the behavior
of the exponent to determine whether the
upper bound to error probability becomes
vanishingly small
The following lemma parallels the one for
the one-user case
If I (X 1; Y |X 2) > 0, then for all 1 ≥ ρ ≥ 0
we have
I (X 1; Y |X 2) ≥
E01
�
∂ 2N E01
∂N E01
�
�
ρ, fX 1 (x1) , fX 2 (x2)
∂ρ
(1)
�
ρ, fX 1 (x1) , fX 2 (x2) ≥ 0
�
ρ, fX 1 (x1) , fX 2 (x2)
∂ρ2
>0
(2)
�
≤0
(3)
�
�⎪
⎪
⎪
∂E01 ρ, fX 1 (x1) , fX 2 (x2) ⎪
⎪
I (X 1; Y |X 2)
⎪
⎪
=
.
⎪
⎪
⎪
∂ρ
N
⎪
⎪ρ=0
(4)
Coding theorem
Let q1,N , q2,N be a pair of probability den­
sity functions for the codewords of length
N of users 1 and 2
E01
�
�
1,N
2,N
ρ, q
,q
In order for
− ρR1 to be
strictly positive for
some ρ
in [0, 1],
it is
necessary and sufficient that
�
�
⎧
⎫⎪
⎪
1
1,N
2,N
⎨ ∂E ρ, q
⎬⎪
⎪
,
q
⎪
0
⎪
− R1 ⎪
> 0.
⎪
⎪
⎩
⎭
⎪
∂ρ
⎪
⎪ρ=0
We have that:
For all fX 1 (x1) , fX 2 (x2) probability density
functions for X1, X2, we have
I(X 1 ;Y |X 2 )
N
> R1 ≥ 0
⇒ ∃ρ ∈ [0, 1] s.t.
E1
0
�
�
ρ, fX 1 (x1) , fX 2 (x2) − R1ρ > 0
We can establish analogous results for er­
rors of type 2 and 3
Coding theorem
Let us define
Emin =
�
�
�
�
�
1
1,N
2,N
min maxρ E0 ρ, q
,q
− R1 ρ ,
�
�
�
�
2
maxρ E0 ρ, fX 1 (x1) , fX 2 (x2) − R2ρ ,
�
�
�
��
3
1,N
2,N
maxρ E0 ρ, q
,q
− (R1 + R2) ρ
where E02 and E03 is defined analogously to
E01. We may state the following theorem:
For allfX 1 (x1) , fX 2 (x2) probability density
functions for X1, X1, for any messages m1
and m2 of users 1 and 2, we have
P em1,m2 ≤ 3e−N Emin
and
I
��
�
�
X1 , X 2 ; Y
N
�
I X1; Y |X2
> R1 ≥ 0 and
N
�
�
I X2; Y |X1
> R2 ≥ 0 and
N
�
> R1 + R2 ≥ 0 ⇒ Emin > 0
Capacity region
Cover-Wyner region for two users
Capacity region
Consider AWGN Multiple-access channel,
2
user i has energy σX
1
Pentagon: dominant face corresponds to
�
1 ln
2
1+
2 +σ 2
σX
X2
1
2
σN
�
Interference cancellation at the corners:
�
�
1 ln
2
�
1+
�
1 ln
2
1+
2
σX
1
�
,1
2 ln 1 +
2 +σ 2
σX
N
2
2
σX
�
1
2
σN
�
�
,1
2 ln 1 +
2
σX
2
2
σN
2
σX
��
��
2
2
2
σX +σN
1
without interference cancellation:
�
�
1 ln
2
1+
2
σX
�
1
2 +σ 2
σX
N
2
�
,1
2 ln 1 +
Recall DS-CDMA example
2
σX
2
2
2
σX +σN
1
��
Capacity region
�
FDMA:
�
W1
2
ln 1 +
2
σX
1
2
W1 σN
�
�
, W22 ln 1 +
2
σX
2
2
W2 σN
for equal energies, equal W s desirable
TDMA: let α be the fraction of time that
user 1 transmits
�
�
α ln
2
1+
2
σX
1
2
ασN
�
�
α ln 1 +
, 1−
2
2
σX
��
2
2
(1−α)σN
for equal energies, α = 0.5 desirable
How do we achieve points on the dominant
face, that yields maximum sum rate?
First way: time share between the corners
Other way: rate splitting
��
Capacity region
Make one user (say user 1) into two virtual
users (virtual user 1 and virtual user 3) and
split energy between these two virtual users
Virtual user 1 rate:
�
1 ln
2
1+
2
ασX
1
2
2 +σ 2
σN +(1−α)σX
X2
1
User 2 rate:
�
1 ln
2
1+
�
2
σX
2
2
2
σN +(1−α)σX
1
Virtual user 3 rate:
�
1 ln
2
1+
2
(1−α)σX
2
σN
�
1
�
Capacity region
We have
⎛
⎞
2
ασX
1
1
⎝
⎠
ln
1 + 2
2
2
2
σN + (1 − α)σX + σX
1⎞
2
⎛
2
σX
1
⎝
2
⎠
+
ln
1 +
2
2
2
σN + (1 − α)σX
⎛
2
⎞
(1 − α)σX
1
⎝
1⎠
+
ln
1 +
2
2
σN
⎛
=
=
⎞
2
σX
1
⎝
1
⎝
2 ⎠
⎠
+
ln
1
+
ln
1 +
2
2
2
2
2
σX + σN
σN
2
⎛
⎞
2
2
σX + σX
1
⎝
2 ⎠
ln
1 +
1 2
2
σN
2
σX
1
⎞
1
⎛
Capacity region
If we have
�
R2
= 1
2
ln 1 +
2
σX
2
2
2
σN +(1−α)σX
1
�
,
then R1 is defined as
⎛
=
+
⎞
2
+ σX
1
⎝
2 ⎠
ln
1 +
− R2
2
2
σN
⎛
⎞
2
ασX
1
⎝
1
⎠
ln
1 +
2
2
2
2
σN + (1 − α)σX + σX
1
2
⎛
⎞
2
(1 − α)σX
1
⎝
1⎠
ln
1 +
2
2
σN
2
σX
1
One variable provides all the necessary de­
grees of freedom
Capacity region
In general, for µ users, the capacity region
is
�
i∈S Ri ≤ I((Xi )i∈S ; Y |(Xi)i�∈S ), ∀S ⊂ {1, . . . , µ}
We have 2µ − 1 pseudo-users are sufficient
to achieve any point on the multiple-access
dominant face
MIT OpenCourseWare
http://ocw.mit.edu
6.441 Information Theory
Spring 2010
For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.