306
IEEE TRANSACTIONS
ON INFORMATIONTHEOR~,VOL.IT-25,N0.
nonanticipatory channels,” Inform and Co&r., vol. 26, pp. 381391, 1971.
I261 A. I. Khinchine, Mathematical FounaWio~ of Information Theov.
New York: Dover, 1957.
(271 B. McMillan, “The basic theorems of information theory,”
-- Ann.
Math. Stat., vol. 24, pp. 196-219, 1953.
1281J. Moser, E. Phillips, and S. Varadhan, Ergodic Theory: A Seminar.
New York: Courant Inst. of Math. Sciences, 1975.
1291J. Nedoma, ‘The capacity of a discrete channel,” Trans. First
Prague Conf: Inform. Theory, pp. 143-181, Prague, 1957.
“On nonergodic channels,” Trans. Second Prague Conf:
[301 -,
Inform. Theory, pp. 363-395, 1960.
[311 D. L. Neuhoff, R. M. Gray, and L. D. Davisson, “Fixed rate
universal block source coding with a fidelity criterion,” IEEE
Trans. Inform. Theoty, vol. IT-21, pp. 511-528, 1975.
[321 D. S. Omstein, “An application of ergodic theory to probability
theory,” Ann. Prob., vol.. 1, pp. 43-58,1973.
Emodic Theory. Randomness.and Dvnamical Systems. New
1331 -.
Haven, CT’: Yale Univ. Press, 1974.
[341 J. C. Oxtoby, “Ergodic Sets,” Bull. Amer. Muth. Sot., vol. 58, pp.
116-136, 1952.
1351 K. R. Parthasarathy, “On the integral representation of the rate of
transmission of a stationary channel,” III. J. Math., vol. 2, pp.
299-305, 1961.
“Effective entropy rate and transmission of information
[361 through channels with additive random noise,” Sunk&, vol. A25,
pp. 75-84, 1963.
Probability Measures on Metric Spaces. New York:
[371 -,
Academic, 1968.
1381 E. Pfaffelhuber, “Channels with asymptotically decreasing memory and anticipation,” IEEE Trans. Inform Theory, vol. IT-17, pp.
379-385, 1971.
1391M. Pinsker, Information and Znformution Stabili@ of Random Vuri-
ubles und Processes. Moscow: Ird. Akad. Nauk. SSSR, 1960
(Translation: San Francisco, CA: Holden-Day, 1964).
Vu1 C. E. Shannon, “A mathematical theory of communication,” Bell
Lfvsf. Tech J., vol. 27, pp. 379-423, 623-656, 1948.
_1411
_ P. C. Shields, The Theory of Bernoulli Shifts. Chicago; IL: Univ.
of Chicago Press, 1973.
[42] V. Strassen, “The existence of probability measures with given
marginals,” Ann Math. Stat., vol. 36, pp. 423-439, 1965.
[43] K. Takano, “On the basic theorems of information theory,” Ann
Inst. Stut. Math., vol. 9, pp. 53-77, 1957.
144 I. P. Tsaregradsky, “On the capacity of a stationary channel with
finite memory,” Teor. Verqvutnosti i Primenen, vol. 3, pp. 84-96,
1958.
[45] K. Winkelbauer, “Channels with finite history,” in Truns. 2nd
Prague Con& Inform. Theoty, pp. 685-831, Prague, 1960.
[46] -,
‘On discrete information sources,” in Trans. 3rd Prague
ConjI Inform Theoty, pp. 765-830, Prague, 1964.
“On the coding theorem for decomposable discrete informa_1471
_ -,
tion channels: I,” Kybernetihxz,vol. 7, pp. 109-123, 1971.
[48] -,
“On the coding theorem for decomposable discrete information channels: II.” Kvbernetika. vol. 7. DD. 230-255, 1971.
“On the regul&ty condition for &composable communica[49] -,
tion channels,” Kybernetiku, vol. 7, pp. 314-327, 1971.
1501 J. Wolfowitz, “Strong converse of the coding theorem for the
general discrete finite-memory channel,” Inform and Contr., vol. 3,
pp. 89-93, 1960.
[511 J. Wolfowitz, Coding Theorems of Information Theory, 2nd ed.
Berlin: Springer-Verlag, 1964.
1521 Shen S. Yi, “The fundamental problem of stationary channel,” in
Trans. Third Prague Conf: Inform. Theory, pp. 637-639, Prague,
1962.
[531 D. L. Neuhoff and P. C. Shields, “Charnels with almost finite
memory,” to appear, IEEE Trans. Znform. Theory.
3,w.y
1979
A Coding Theorem for the Discrete
Memoryless Broadcast Channel
KATALIN
Abstmct-A coding theorem for the discrete memoryleas broadcast
channel is proved for tbe case where no common message is to be
transmitted. The theorem is a generalization of the results of Cover and
van der Meulen on this problem. Tbe result is tight for broadcast channels
having one deterministic component
I.
INTRODUCTION
DISCRETE
memoryless broadcast channel
(DMBC) as defined by Cover [ 1] is determined by a
pair of discrete memoryless channels with common input
alphabet ‘3. We denote by F and G the transition proba-
A
Manuscript received November 28, 1977; revised October 26, 1978.
This paper was presented at the IEEE Symposium on Information
Theory, Ithaca, NY, 1977.
The author is with the Mathematical Institute of the Hungarian
Academy of Sciences, H-1053 Budapest, Reahanoda u. 13-15, Hungary.
MARTON
bility matrices of these channels:
F={F(xly):yE?4,xE%},
G={G(zly):y~%,z~%}.
Here !X and 2E are the output alphabets, with cardinalities11%
IL IIWI~ll~ll < co. The DMBC corresponding to
the matrices F, G will be denoted by (F, G).
We assume that the conditional probabilities of receiving the sequencesx” E 5%”and z” E 2FTat the outputs of
the channels F and G, respectively, are given by
F”(x”lYn)=
laI F(x;lY;),
G”(z”l~“)=
iI!, G(z;lYi)
wherey”=y,y,***y,,.
In connection with the DMBC (F, G), we consider the
following coding problem. A sender has to transmit two
independent messagesover the channels F and G: one
messagefor receiver I observing the output of the channel
001%9448/79/0500-0306$00.75 01979 IEEE
MARTON: CODING THEOREMFOR MRMORYLESS
BROADCASTCHANNEL
F, and another for receiver II observing the output of the
channel G. The messages take their values in the sets
{1,2,.'. ,J} and {1,2;.. , K}. The sender uses a block
code of block length n. Given that the first messagehas
value j and the second message has value k, the sender
transmits a sequenceyjz E %“. The question is at which
rate pairs (n-i log J, n -’ log K) can this be done so that
both receivers can with high probability decode their
respective messagescorrectly. The asymptotic values of
these rate pairs consititute the capacity region of the
broadcast channel.
No computable formulas are known for the capacity
region of the DMBC, except for three special cases: 1) if
one of the component channels is “more capable” in the
sense of [2] than the other one: 2) if the DMBC (F, G) is
the product of the DMBC’s (F,, G,) and (F2, G2) (i.e.,
‘%=‘%IX~2, %=%IX%&
%=55,X&
F(x,,xz(y,,yz)=
307
is an (n,e)-code (e > 0) for the DMBC (F, G) if there exist
two disjoint families of decodingsets
{@+ l<jCJ},
(t$cfX”,
{S3,: l<k<K}
?PJ~C%~,6$n~~=CBknCB~=0, for ahj#j’,
k#k’) such that
The pair of numbers (n - ’ log J, n - ’ log K) is the rate pair
of the code. A pair of numbers is called achievable if, for
any fixed e > 0, it can be approximated by rate pairs of
(n,r)-codes. The capacity region of the channel is the set
of all achievable pairs.
We shall use the following notation. All random variables (r.v.‘s) in the paper are supposed to have finite
ranges. The symbols IV, U, and V will always denote r.v.‘s
F,(~,ly,F’,(x,l~,)~ ‘3 w~IY~,Y~)= Wz11~,)W21~,)~ for and Y, X, and Z will denote r.v.‘s with ranges 9, %, and
(Y~,Y~)E~, (xl,xJ~f% (.q,z2)~% G, is a degraded
version of F,, and F2 is a degraded version of G,; and 3) 2, respectively. We write (Y, X, Z) E 9 (F, G) if the condiif the DMBC is deterministic, i.e., F and G are (4 l)- tional distributions of X and Z given Y are defined by the
matrices. Case 1) has been solved recently in [3], generaliz- matrices F and G, respectively. (U, Y, X, Z) E ‘??(F, G) will
ing earlier results of [ 131,[4], [5], and [2]. (See also [6] for a mean that 1) (Y,X, Z) E 9 (F, G), and 2) both triples
(U, Y, X) and (U, Y, Z) are Markov chains.
related problem.) Case 2) is settled in [ 151.
We recall the following inner bound for the capacity
Although case 3) now seems almost trivial, it had been
region
of the DMBC.
an open problem for a long time. (The Blackwell channel
is a deterministic DMBC.) It was settled independently by
ThEorem 1 (Cover-van der Meulen-Hajek-Pursley):
Pinsker [7] and the author [9]. (See also [8] for a particular Let % denote the convex closure of the set
case; [9] contains only a heuristic proof.)
In the general case only an inner bound to the capacity
{(Rx,Rz): &, R, Z 0, Rx <I( WU/\X>, R, <I( WV
region is known. This can be obtained from the results of
AZ>, R,+R,<fin
{I(WAX), I(WA
van der Meulen [lo] and Cover [l l] (see [12]). The aim of
Z)}+I(U/\XIW)+I(V/\ZIW)
for
the present paper is to prove a better inner bound (Theosome ((U, V, W), Y, X, Z) E C?(F, G)
rem 2) to the capacity region of the DMBC. This bound is
such that U, V, and W are
proved by a random coding method which is a combinaindependent}.
tion of the coding techniques of Bergmans [ 131and Cover
and van der Meulen [lo], [l I], with the random coding Then any rate pair (Rx, R,)E% is achievable for the
technique used to prove source coding theorems in rate DMBC (F, G).
distortion theory.
Our goal is to prove the following generalization of this
Theorem 3 is an easy consequence of Theorem 2 and
describes the capacity region of the deterministic DMBC. theorem.
It is generalized by Theorem 4 to DMBC’s with one
Theorem 2: Let
deterministic component. Theorem 4 has also been proved
%={(R,,R,):
R,,R,>~, ~,<z(wu~x),
independently by Gelfand and Pinsker [17]. The converse
part of Theorem 4 is a special case of an outer bound for
‘R, <I( WV//Z), Rx+ R,
the general DMBC (Theorem 5), due to Korner and the
<rnin {Z(WAX),I(WAZ)}+Z(UAXIW)
author [ 161.
Theorems 3 and 4 show that Theorem 2 is more than a
+z(V/jZlW)-z(U//vlw)
formal generalization of Theorem 1. As a matter of fact,
forsome((U,V,W),Y,X,Z)EC?(F,G)}.
we shall show in Appendix II that Theorem 1 is not tight
for the Blackwell channel.
Then any rate pair (Rx, R,)E % is achievlbl5 for t8e
DMBC (F, G).
II.
?I
DEFINITIONS
AND RESULTS
Definition: For n = 1,2,. - . a set of codewordrsof length
(Yjk: I <j<J,
I <k<K}C%”
Remarks: 1) It is easy to see that ‘?i?,is convex. 2) In the
definition of ?R, no condition on the independence of the
r.v.‘s W, U, V is imposed. The set 3 consists of those rate
pairs in $R that correspond to independent r.v.‘s U, V, W.
3) Consider for a moment the subset $Jl+C ?ILconsisting of
308
IEEE TRANSACTIONS
ON INFORMATIONTHEORY,VOL. IT-~~,NO.~,MAY
rate pairs corresponding to W = const.:
the following outer bound for the capacity region of a
DMBC [16].
%,={(R,,R,):
R,,R,>Q,
Rx <Z(Ur\X),R,<Z(Vr\Z),
R,+R,<Z(UAX)+Z(V/jZ)-Z(U/yV)
forsome((U,V),Y,X,Z)E??(F,G)}.
Theorem 5 (Korner-Marton):
of the DMBC (F, G) satisfies
1979
The capacity region ?R
%rt{(R,.,R,): O<R,<Z(Y/\X),
O<R,<Z(V/\
Z), Rx +R, <Z(Yr\XjV)+Z(V/\Z)
for
(3)
We believe that the novelty of Theorem 2 is essentially in
some (V,Y,X,Z)E??(F,G)}.
establishing that any rate pair in 9& is achievable, Let us
give a heuristic reason why the rate pairs in 9+, should be Moreover V can be assumed to take at most 11%I] +2
achievable. Let (U”, Y”, Zn) denote the length n output of values.
the discrete memoryless correlated source with generic
Remarks: 1) A similar outer bound can be obtained for
variable (U, Y, Z). It can be shown by the method used in the rate region of codes all codewords of which have the
[ 141that, for some sequenceof positive numbers {S,} with same composition, say Q. Combining the bound for fixed
Q with that obtained by reversing the roles of F and G,
&-a
and
than letting Q vary over the distributions on %, the
V* is an r.v. such
lim n-i max {Z( V*AZ”):
n-+oo
bound (3) can be improved. 2) Theorem 5 can be proved
that n-‘Z(V*AU”)<&
and (V*,Y”,Z”)
is a
either by the method of “images of a set via two chanMarkov chain} = max { Z( V/\ Z) - Z( U/j V):
nels” used in [6], or by using only a single-letter technique
( V, Y, Z) is a Markov chain}.
for information quantities as in [ 181.(Such a technique is
It is easy to see from this that, for (U, Y,X, Z) E ‘??(F, G) also implicitly contained in the first method.) Here we
and large enough n, we can construct an r.v. VT such that give a proof of the second type, since it is simpler.
However, the method used in [6] would give a stronger
((U”, V,*), Y”,X”,Z”) EC?(F”, G”),
result: namely, that (1) holds also with q(e) (the region of
{Z(Vr\Z)-Z(U/\V):
,Ii%n-‘Z(V:jjZn)=max
the so-called e-achievablerates) instead of 9,. This means
that Theorem 4 holds with a strong converse.
((u,V),Y,X,Z)E~(F,G)}
Theorem 5 is proved in Appendix I.
and
In the proof of Theorem 2 we use the following notapiIn-‘z(v;AU”)=o
(2) tion:
i.e., Vj is asymptotically independent of U”. We also have
for all n. Therefore, if we had
Z( V,*/j Un) = 0 for all n, instead of (2) the achievability of
the pair R, = Z( U/\X), R, = max J Z( V/\ Z) - Z( U/\ V)]
(and hence of 9,,) would follow from the Cover-van der
Meulen theorem.
distribution of the r.v. X;
conditional distribution of the r.v. U
given the r.v. W;
P; and PGlw denote the nth memoryless extensions of
these distributions;
Theorem 3 (Pinsker-Marton): If (F, G) is a deterministic broadcast channel, i.e., if F and G are (0,l) matrices,
then the capacity region of (F, G) is
forafinitesetw,aEw,n=1,2..’
andw”=w,w,...w,,
EW,
n(a]w”) k {i: wi= a}lj; for an r.v. W with range
‘%‘, q>O andn=1,2;..,
n-'Z(U"AX")=Z(UAX)
%={(R,,R,): R,,R,>O, R,<H(X), R,<H(Z),
R, + R, < H(XZ) for some (Y, X, Z) E
~‘EG)}.
The direct part of this theorem follows from Theorem 2
by defining W= const, U=X, V= Z. The converse is
trivial. Pinsker extended this theorem to the case of multicomponent broadcast channels, all components of which
are deterministic.
px
P VW
tTw(q) P { w”E%P: In-‘n(uI w”)- P,(a)l<q,
all aEw};
for a pair of r.v.‘s (W,X) with range % X xx, for n, >O,
for a sequencewn E yw(n,) and for q2 >ni,
Tx(w”, Q) A {X” E tXn: In-‘n(ublw”x”)
P,(ub)l <nz, all (a,b)E % X %; n(ublw”x”)=
0 for P,(u,6)=0}.
Theorem 4 (Gelfand-Pinsker-Marton): If (F, G) is a
DMBC for which F is a (0,l) matrix then the capacity
region of (F, G) is
%={(R,,R,): ~GR,GH(X), OGR,GZ(VAZ),
R, +R, <H(XIV)+Z(V/r\Z)
for some
(K Y,XZ)@V’,G)}.
Moreover, this region remains unchanged if V is allowed
to take at most 11%]I +2 values.
The direct part of Theorem 4 follows from Theorem 2
with W = const, U= X. The converse is a special case of
III.
PROOFOFTHEOREM
2
It suffices to prove that the pair
Rj
Z(WU/\X)=Z(Wr\X)+Z(U//X~W),
R,~min{Z(W~X),Z(Wy\Z)}+Z(U~XIW)
+Z(Vr\ZIW)-Z(U/?\V(W)-R,
=z(v/\zIw)-z(u/jvIw)
-I~(wA~)-~(wAz)I+
309
MARTON: CODING THEOREMFOR MEMORYLESS
BROADCASTCHANNEL
is achievable for (( W, U, V), Y, X, Z) E 9 (F, G). We may
assumeR, >O, i.e.,
over the ensemble of the auxiliary random codes.
We have
Z(UAVlW)<Z(VAZlW)-lZ(WAX)-Z(WAZ)I+.
Fix the numbers E,6, n > 0. For n fixed, define’
Z= [ exp tntzt WAX) - a))]
(4)
J=[ev
(5)
(4Z(UAXIW)-6))]
L=[exp (n(Z(UAvlW)+6))]
(6)
K=[exp(n(Z(VAZIW)-IZ(WAX)-Z(WAZ)I+
-Z(U/\VIW)-2S))].
(7)
where
We then have
KL<exp(n(Z(VAZIW)-6))
03)
Furthermore, since ajk > aik, and, consequently,
ZKL < exp (n(Z( WV//Z) - 6)).
(9)
Ci3,y%1,\u{~,:
k’#k}
Using a random coding method, we shall define length
n codewords yiik (1 <i <I, 1 <j.< J, 1 <k <K) and decod- we have, as in (11)
ing sets eij c %‘, B3, c %” so that (1) holds with the index
j replaced by the pair of indices (ij) (and e replaced by
const e).
Select the length n sequences wi (1~ i <I) independently of each other and according to the distribution Pb.
If xjk = 1, then (wjuiivikr) l Y~,,(n/2)
and yijk E
Then, for fixed i, select the length n sequencesuii (1 <j <
5r(wiui,~vikr,n)
fcr
some
1.
Therefore,
recalling
the definiJ) and vik, (1 < k Q K, 1 <I < L) independently of each
tion
of
aij
and
aik,,
it
easily
follows
from
the
law
of large
other and according to the distributions P$ w(. IWJ and
numbers
and
from
the
relation
((
W,
U,
V),
Y,X,
Z) E
Ptf, w( * Iwi), respectively. The set
9 (F, G) that for large enough n,
{wi,uij,vi,: 1 <i<Z, 1 <j<J, 1 <k<K, 1 <I<L}
will be called an auxiliary random code.
For ij, k fixed, let xjk denote the indicator variable of
the event that the triple of sequences(wiuijvikr) belongs to
YwclV(7j/2) for at least one value of I. This value of I will
be denoted by Z(ijk). From the rule for selecting our
auxiliary random code, and from (6) it follows that, for
sufficiently large n,
and, similarly,
Furthermore, we have
Pr {xjk=O} <e.
(10)
If xjk = 1, let the codeword yijk be any sequence in
~Y(wiuijvik,Cijkj,n)and let yiik be arbitrary for xjk =O. Set
+ 2 T(x”,&iy)
Y#j
$i = ETX(WiUi~,
217)
< 2 7(x”&)+ 2 T(X”,&J
‘j/cl = TZtwiv&,>27)>
i’#i
aik = u 8iik,,
I
4ik = u
(15)
@Jikl= u
i
i, I
aik.
We define the decoding sets gij and ‘?Bkby
Qij = 8Jij\ U { 6&:
(i’j’)#(ij)}
%,=&,\u{&~,:
k’#k}.
For ij, k fixed, we shall estimate the expected value of
the quantities
1-Fn(6?ijJyijk)
and
Y#j
l-Gn($Bklyiik)
‘III (4)-(7) [M] denotesthe integer part of the number M.
(where 8$ = TX (We,27)), and
T(ZT ,‘;‘,4e)=7(zn,
u
i’,k’#k,I’
(16)
The rule for selecting the auxiliary code provides an
estimaJe of the condition+ expectations of the r.v.‘s
7(X’, &), 7(X”, giY), 7(Z”, ‘%3i’k,,,),
and 7(Zn, ‘$ik,l,), given
the values of the auxiliary codewords Wi, Uij, Vik,,
For X” E gij, Z” E aik, sufficiently small
T/,& ’ * * , &,.
310
~l33 ~SACIIONS
9, and large enough n, we have
and
n-‘[Z(Y”/\X”IV*)+Z(V+/\Z”)]
E{~(x",~i')IWi,l(ij,vik~,."li)ik~}
for
< exp [ - n(Z( WAX) - 6/2)],
E{ 7(x”,
i’#i
(17)
for j’#j
(18)
. * ,vi/d.}
&j')lwi,uij>2)ikl,.
< exp [ - n(Z( uAX( W) - 6/2)],
E{~(~“,~i’k’r)lWi,~ij,~ikDikl,.’* ,vih5}
for
<exp [ -4Z(WAZ)-6/2)],
ON INFORMATIONTHEORY,VOL. ‘r-25, NO. 3, MAY1979
<Z(YAX(V)+Z(VAZ)
(25)
provided that Y” is any r.v. with values in %“, X” and Z” are
the length n outputs of channels F and G, respectively, corresponding to the input Y”, and V*,Y” and (X”,Z”) form a
Markov chain.
It is clear that
n-‘I( Y”AX”l V) <n-‘I(
i’#i
<n-’ i: Z(yi/\Xi)
and any k’, I’ (19)
n-‘I( V*/\Z”)
for
E{&jk[
‘-F”(@ijlyijk)])
<e+Zexp [ -n(Z(WAX)-6/2)]
(21)
+Jexp [ -n(Z(U/\XIW)-a/2)].
Similarly, from (14), (16), (12), (19), and (20) we get
E{)(ijk[
exp [ -n(Z(WVy\Z)-S/2)]
+KLexp
(22)
[ -n(Z(Vy\ZIW)-a/2)].
E{2-F”(~ijIYijk)-Gn(~kIYijk)}<8r.
=n -’ 5 Z(&AZi)
(26)
i-l
where
&“=Zi+,.-*Z,,,
Xi-*&X,Xz..~Xi-,,
~=V*Xi-‘Z~,
1 < i < n. Moreover,
Z( PAZ”)--I(
V*/\X”)
=[Z(v*/\Z”)-Z(V*r\X,Z,“)]
+ [I( I’*AXIZ;) - Z( V*r\X,X,Z;)]
‘+..a +[Z(V*AX”-‘Zn)-Z(V*AX”)]
(27)
V*/\XilX’-‘ZF)].
Applying (27) to Y” instead of I’* yields
= $, [ Z( Yi/jZilX’-‘Zr)-I(
YiAXilX’-‘Z~)]~
(27’)
which is equivalent to
This implies the existence of an (n,Se)-code with rate
pair (Rx -26, R, - 26) for any e,6 > 0. The theorem is
proved.
We use (27) and (28) to overbound the left-hand side of (24):
n-‘[Z(Y”AX”IV*)+Z(V*AZ”)]
=n-‘[Z(Y”r\X”)+Z(V*AZ”)-Z(V*AX”)]
ACKNOWLEDGMENT
The author is indebted to J. Korner, who agreed to
publish Theorem 5 in this paper.
en-Ii*,
[I( YiAXilX-‘Z,“)+Z(Zi”AXiIX’-‘)
+I( V*r\ZilXi-‘Z/‘)
--I( V*r\XilXi-‘Zi”)]
(29)
=n -’ $, [ I( TAXiI vi) + Z(Xi-‘AZilZ~)
APPENDIX 1
+ Z( V*//Z,IX’-‘Z,“)]
Proof of Theorem 5
<n-’ i
Let the r.v. Y” be uniformly distributed over the codewords
{yjk: 1 <j <J, 1 <k <K} of an (n,c)-code for (F, G), and denote
by V* the r.v. taking the value k if Y”=y+ By Fano’s lemma
n-‘logJ=n-‘H(Y”]V*)<n-‘Z(Y”AX”IV*)+const.c,
n-‘logK=n-‘H(V*)<n-‘Z(V*/\Z”)+const.c;
where X” and 2” are the n-length outputs of the discrete
memoryless channels F and G, respectively, corresponding to the
input Y”. In order to prove (3), it suffices to show that for some
(V,Y,X,Z)EY(F,G)
we have
n-‘I( V*/\Zn)
Z( V*Xi-‘Zr/\Zi)
Z( Yn/\Zn) - I( Y”/\P)
Using (4), (5), (8), (9), and (10) it follows from (21) and
(22) that for all ij, k and for large enough n,
n-‘I( Y”AX’l
i
i=I
= $, [I( Pr\ZilXi-‘Z~)-Z(
l-G”(aklyijk)]}
<e+ZKL
<n-I
k’#k
and any I’. (20)
Substituting (13) and (15) into (1 l), taking mathematical expectation and using (17) and (18), we get
(25)
i-1
E{ T(Zn,~i~r)IWi,Uli,Vikl,’” ,vikL}
Gexp [ --n(Z(VAzIW)--~/2)],
Ynr\X’)
V*) <I( Y/\X)
(23)
<I( V//Z)
(24)
[Z(Y,AX,IVJ+Z(V,r\Z,)].
i-l
The third equality in (29) follows from
i- 1,2; * * ,n
(vi,F,Xi,Zi)E9(F,G),
(30)
which can be easily verified.
Equations (25), (26), and (29) together with (30) imply (23)(25) if we define
V=(Z,V,), Y= Y,, x=x,, z=z,,
where Z is an r.v. uniformly distributed over the set { 1,2; * * ,n}
and independent of ( V*, Y”,X”, Z”).
The fact that the region in (1) does not decrease if V is
allowed to take at most 11%II+2 values can be seen using [5,
lemma 31.
311
MARTON: CODING THEORBMFOR MBMORYLBSS
BROADCASTCHANNEL
hPENDIX
11
Let (F, G) be the Blackwell channel, i.e., % = {0,1,2}, 5%= 55
= (0, l}, and
F=(;p)
G=(i
p).
REFERENCES
Proposition:
max {Rx+&:
Equations (40) and (41) imply that iY(Z]XV) <H(Z] UV)=O,
which together with (32) means that we also have H(Z] V) =O.
Similarly H(X I U) = 0 and consequently Z( Uy\ V) > Z(XA Z) > 0,
which contradicts the assumption that U and V are independent.
[l]
(R,,R,)E&}<log3
=max {&+I$:
(R,,R,)E%}.
(31)
Proof From Theorem 3 it follows that the right-most side
of (31) equals
max {H(X,Z): (Y,X,Z)EC?(F,G)}=~~~~
and the maximum is achieved if and only if the joint distribution
of the pair (X,Z) is
Pr {X- 1, Z- l}=Pr {X= 1, Z=O}
=Pr {X=O,Z=l}=f.
(32)
Since in [12] size constraints are proved for the cardinality of
the auxiliary r.v.‘s W, U, V figuring in the definition of 4, it is
enough to prove that for ((U, V, W), Y,X, Z) E C?(F,G) such that
W, U, and V are independent, the inequality
I-h {Z(WAX),Z(WAZ)}+Z(UAXIW)
holds.
It can be easily seen that
+Z(VAZ]W)<log3
Z(UAXIW)+Z(VAZIW
<Z(Ur\XjVW)+Z(Vr\Z
‘IV
<Z( UV/\XZj W) <H(XZ ‘IW)
(33)
(34)
and it is obvious that
min { Z( WAX),Z( WAZ)} <I( 1;vr\XZ)
(35)
so the left-hand side of (33) can be upper-bounded by
Z( WAXZ) + fZ(XZ 1W) = H(XZ) < log 3.
(36)
Suppose that in (33) we have equality instead of strict inequality. Then the following conditions must be satisfied:
the pair (X, Z) has distribution (32)
(37)
Z(W/\X)=Z(W//Z)=Z(W/jXZ)
(38)
Z(U/jXlVW)+Z(VAZlW)
= Z( UVAXZl W) = H(XZI W) (39)
c.f., (36), (35), and (34). From (37) it follows that the distribution
of (X,Z) is indecomposable (see [19]). Therefore, (38) implies
that W is independent of (X,Z) (see [20]). From this it follows
that we can restrict attention to the case W=const. Then (39)
implies that
(X, Z) is a deterministic function of ( U, V)
(40)
Z(VAXIZ)=Z(UAZIXV)=O.
(41)
T. M. Cover, “Broadcast channels,” IEEE Trans. Inform Theory,
vol. IT-18, pp. 2-14, Jan. 1972.
[2] J. Kiimer and K. Marton, “Comparison of two noisy channels,”
Colloquia Mathematics Societatis J&s Bobai, 16, Topics in Information Theory, I. Csisz& and P. Elias, Eds. Amsterdam: NorthHolland, 1977, pp. 41 l-423.
131 A. El Gamal. “The canacitv of a class of broadcast channels,”
Dep. of Statistics, StandfordUniv., Stanford, CA, Tech. Rep. 24,
1978.
[4] R. G. Gallager, “Coding for degraded broadcast channels,” Problemy Peredachi Informatsii, vol. X, pp. 3-14, Sept. 1974, (in Russian).
[5] R. Ahlswede and J. Khmer, “Source coding with side information
and a converse for degraded broadcast channel,” IEEE Trans.
Inform. Theoty, vol. IT-II, pp. 629-37, Nov. 1975.
P31 J. Kiimer and K. Marton, “General broadcast channels with
degraded message sets,” IEi%E Trans. Inform Theory, vol. IT-23,
pp. 60-64, Jan. 1977.
(71 M. Pinsker, “The capacity region of noiselessbroadcast channels,”
Problemy Peredachi Informatsii, vol. XIV, pp. 28-34, 1978 (in
Russian).
PI S. Gelfand, “The capacity region of a broadcast channel,” Problevy PereaizchiInformatsii, vol. XIII, pp. 106-108, 1977 (in Russian).
[91 K. Marton, “The capacity region of deterministic broadcast
channels,” to appear in Trans. Int. Symp. on Information Theory,
Paris-Cachan, 1977.
WI E. van der Meulen, “Random coding theorems for the general
discrete memoryless broadcast channel,” IEEE Trans. Inform
Theory, vol. IT-21, pp. 180-190, Mar. 1975.
1111 T. Cover, “An achievable rate region for the broadcast channel,
IEEE Trans. Inform. Theory, vol. IT-21, pp. 399-404, July 1975.
WI B. E. Hajek and M. B. Pursley, “Evaluation of an achievable rate
region for the broadcast channel,” to appear ti IEEE Trans.
Inform. Theory.
u31 P. Bergmans, “Coding theorems for broadcast channels with degraded components,” IEEE Trans. Inform Theory, vol. IT-15, pp.
197-207, Mar. 1973.
[I41 J. Kiimer and K. Marton, “Images of a set via two channels and
their role in multi-user communication,” IEEE Trans. Inform
Theory, vol. IT-23, pp. 751-761, Nov. 1977.
1151 G. Poltirev, “The capacity region of parallel broadcast channels
with degraded components,” Problemy Pereaixhi Informatsii, vol.
XIII, pp. 23-35, 1977 (in Russian).
WI J. Kiimer and K. Marton, unpublished result.
1171 S. Gelfand and M. Pinsker, paper to appear in Problemy Peredachi
Informatsii.
WI I. Csiszar and J. Khmer, “Broadcast channels with confidential
messages,”IEEE Trans. Inform. Theory, vol. IT-24, pp. 339-348,
May 1978.
I191 P. G&s and J. Kiimer, “Common information is far less than
mutual information,” Problems Contr. Inform., vol. 2, pp. 149-162,
1973.
PO1 R. Ahlswede and J. Kiimer “On common information and related
characteristics of correlated information sources,” Preprint presented at the 7th Praaue Conf. Information Theorv.
<, Scot.
, 1974.