COMMUNICATION SCIENCES AND ENGINEERING
XII.
STATISTICAL COMMUNICATION THEORY
Prof. Y. W. Lee
Prof. A. G. Bose
Prof. D. J. Sakrison
Prof. M. Schetzen
Prof. H. L. Van Trees
V. R. Algazi
R. Alter
D. S. Arnstein
M. E. Austin
A.
1.
F.
M.
D.
M.
K.
G.
M.
E.
T.
A.
D.
J.
A.
R.
J.
W.
D.
Bauer
Bregstone
Bruce
Bush
Clemens
Gann
Goodman
Gray
G. Kincaid
J. Kramer
E. Nelsen
K. Omura
V. Oppenheim
B. Parente
S. Richters
S. Smith, Jr.
W. Steele
WORK COMPLETED
A STUDY OF THE PERFORMANCE OF LINEAR AND NONLINEAR SYSTEMS
This study has been completed by V. R. Algazi.
In August 1963 he submitted the
results to the Department of Electrical Engineering, M. I. T., as a thesis in partial fulfillment of the requirements for the degree of Doctor of Philosophy.
Y. W. Lee
2.
DESIGN AND ANALYSIS OF A DC TAPE RECORDING SYSTEM USING
TWO-STATE MODULATION
This study has been completed by D. E. Nelsen.
results to the Department of Electrical Engineering,
In August 1963 he submitted the
M. I. T., as a thesis in partial ful-
fillment of the requirements for the degree of Master of Science.
A. G.
3.
Bose
ANALOG MULTIPLIER BASED ON A TWO-STATE MODULATION SYSTEM
This study has been completed by C. E. Gray.
results to the Department of Electrical Engineering,
In August 1963 he submitted the
M. I. T.,
as a thesis in partial ful-
fillment of the requirements for the degree of Master of Science.
A. G.
4.
Bose
SYNCHRONOUS RECEIVER FOR DIGITAL MULTI-PHASE MODULATION
This study has been completed by L. M.
Goodman.
results to the Department of Electrical Engineering,
In August 1963 he submitted the
M. I. T., as a thesis in partial ful-
fillment of the requirements for the degree of Master of Science.
H. L.
Van Trees
This work was supported in part by the National Institutes of Health (Grant
MH-04737-03); and in part by the National Science Foundation (Grant G-16526); additional support was received under NASA Grant NsG-496.
QPR No. 71
165
(XII.
5.
STATISTICAL COMMUNICATION THEORY)
A MULTIPLEX
COMMUNICATION SYSTEM USING PSEUDO-NOISE
This study has been completed jointly by A. J.
Kramer and J.
CARRIERS
K. Omura.
In August
1963 they submitted the results to the Department of Electrical Engineering, M. I. T.,
as
a thesis in partial fulfillment of the requirements for the degree of Master of Science.
H. L.
B.
MEASUREMENT
Van Trees
OF THE KERNELS OF A NONLINEAR SYSTEM
OF FINITE ORDER
If a nonlinear system can be characterized by the first p kernels, then the system
can be represented in the form of a Volterra series of the pth order
p
y(t) = N [x(t)] =
~
n[x(t)],
(1)
n= 1
in which y(t) is the response of the nonlinear system for the input x(t) and
00
Xn[X(t)] =
00
...
h (T 1 ,
. .
.,
)
x(t-T )...
x(t-'
n)
dr
...
dTn.
(2)
In this report, we shall present an exact method by which the Volterra representation
of a nonlinear system that contains only a finite number of kernels can be determined
directly.
A first-order system is one in which the highest order kernel is h 1 (T 1).
Such a sys-
tem is linear and the first-order kernel is the impulse response of the linear system.
The technique that we shall present is one whereby each of the Volterra kernels of a p thorder system can be determined individually as a multidimensional impulse response.
To explain this technique, we first shall discuss the procedure for determining the Volterra kernels of second- and third-order systems.
The generalization of the procedure
for pth-order systems will then be given.
1.
x(t)
[
N2
Second-Order Systems
y(t)
The Volterra representation of a secondorder system is
y(t) = N 2 [x(t)] = nl[x(t)] + C2 [x(t) ] .
(3)
A schematic representation of such a system
Fig. XII-1.
QPR No. 71
shown in Fig. XII-l. From such a system,
representations
. Schematic is
Schematic representations
of a second-order system.
we can form a system with the output
166
STATISTICAL COMMUNICATION THEORY)
(XII.
Y1(t) =
[x(t)]
since, from Eq. 2,
and [-xt)] = -X 1 [x(t)]
and
Xc2 [-x(t)] = C2 [x(t)].
Thus the system formed as shown in Fig. XII-2a has the representation given by Eq. 4.
Also, the system of Fig. XII-2b has the output
(6)
Y2 (t)= x 2 [x(t)].
A method for measuring such an isolated second-order kernel has been given by
George.1 The method is based upon the observation that
g 2 (t) = 23C2 (x 1 X2 ) =
2
[xl+x 2] - 5
2
[x]-
(7)
C2 [x 2 ],'
in which, for convenience, we have dropped the argument t and we have defined
X2
(x
1
x2)
f
-00
f
h 2 (T 1 ' 2) x 1 (t-
1) x 2 (t-7
2
) d71d7 2 "
Equation 7 can be verified from Eq. 2 by direct substitution.
When this is carried out,
Eq. 7 follows as a result of the identity
a
N2
(a)
Fig. XII-2.
Connections for a second-order system for the output Yl(t).
Connections for a second-order system for the output y 2(t)
QPR No. 71
167
(XII. STATISTICAL COMMUNICATION THEORY)
2x 1 x 2 = (x 1 +x 2) 2 -
x2+x2.
A system with the output g 2 (t) is
x 2 (t) = uo(t-T 2), then
g 2 (t) = 2h
2 (t-T
1
,t-T
2
(9)
shown in Fig. XII-3.
Thus if x 1 (t) = u (t-T1) and
(10)
).
As shown in Fig. XII-4, this is the second-order impulse response
h 2 (71,
of the kernel
2 ) along a 450 line in the 71-7 2 plane.
x, (t)
Fig. XII-3.
Connections for the system JC2 for the output g (t).
2
(T 1 - T )
2
Fig. XII-4.
QPR No. 71
Line of a double-impulse response in the 71-72 plane
as given by Eq. 10. (Arrow points in the direction of
increasing t.)
168
(XII.
STATISTICAL COMMUNICATION THEORY)
We now note that we need not have suppressed the first-order kernel, since, from
Eqs. 3 and 7,
N 2 [x 1 +x 2 ] - N 2 [x 1 ] - N 2 [x 2 ] = K 2 [x +x ] - J
1
2
2
[xl]-
C2 [x 2 ] + x
1
[Xl+x 2 ] -
1 [x 1
1[x2
(11)
= g 2 (t)
and we have made use of the result that
1
[xl1+x
2
- '
1 [x 1 ]
- 3
1
[x 2 ] = 0.
(12)
Thus the system formed as shown in Fig. XII-5 also has the output g 2 (t).
The first-
order kernel can be obtained as the impulse response of the system of Fig. XII-2a.
x (t)
Fig. XII-5.
Connections for a
general secondorder system for
the output g (t).
2
S2 (t)
Alternatively,
from Eq. 3,
the impulse response of the second-order
system,
is
N 2 [u o (t)] = h 1 (t) + h 2 (t,t).
N 2,
(13)
Thus h 1 ( 1) can be determined from the impulse response of the second-order system
once h2(T 1 ,7 2 ) is known.
2.
Third-Order Systems
The Volterra representation of a third-order system is
y(t) = N 3 [x(t)] = E 1 [x(t)] + X 2 [x(t)] + C3 [x(t)].
(14)
Let us begin by considering the special case of a third-order system containing an
isolated third-order kernel, that is, we first shall consider the case for which
y(t) = N 3 [x(t)] =
QPR No. 71
C3 [x(t)].
(15)
169
(XII.
STATISTICAL COMMUNICATION THEORY)
A method for measuring such an isolated third-order kernel is obtained from the identity
g 3 (t) = 3!
=
3
(x X
2
1 X
3)
c3 [x 1 +x 2 +x 3 ] -
-
3
3
[x3+x 1] + '
3
[x 1 +x 2 ]-
X 3 [x 2 +x 3 ]
[x 1 ] + Jc3 [x 2
(16)
3[3],
in which we have defined
a
3
(x 1 X
2
3
) =
f0
f0
fc
-oO
-00o -o00
(17)
h3 (71,2, 3) xl(t-71) x2(t-72) x3(t-7 3 ) d71d7 2 d73.
Equation 16 can be verified by direct substitution in Eq. 2.
When this is carried out
Eq. 16 follows as a result of the identity
3! x 1x 2 x 3 = (x 1+x 2 +x 3) 3 -
(x+x
2
3+(x 2+x 3 3+(x 3 +x1 ) 3
A system with the output g 3 (t) is shown in Fig. XII-6.
uo(t-T2), and x 3 (t) = u (t-T
g 3 (t) = 3! h 3 (t-T ,t-T
+
x3
3
(18)
+x2+].
3 3
Thus if xl(t) = uo(t-T
1 ),
x 2 (t)
3 ), then
2 ,t-T
(19)
3 ).
This is the third-order impulse response of the kernel h 3 (T 1 ,7 2,7 3 ) along a line that
is at 450 with each of the coordinate axes in the 71 -2-73
space.
Any such 450 line in
the 71-7 2-73 space can be obtained by a proper choice of delays T , T , and T .
1
2
3
xl (t)
Fig. XII-6.
QPR No. 71
170
Connections for a
third-order system
for the output g 3 (t).
(XII. STATISTICAL COMMUNICATION
THEORY)
Now, if the third-order system of Fig. XII-6 also contains a first- and a secondorder kernel, then the output still will be g 3 (t), as given by Eq. 16.
To observe this, we
note that the output resulting from the presence of a first-order kernel is zero, since
3e 1 [x 1 +x 2 +x 3 ] -
Cl[Xl+X 2 - Jc 1 [X2X
3]
-
[x+Xl
+ C1 [X2 ] + J1[3] = 0.
+ '1[Xl]
(20)
Equation 20 follows as a result of the identity
(x 1 +x 2 +x 3 ) - [(x 1 +X 2 )+(x 2 +x 3 )+(x
3 +x 1 )]
+ [X 1+x2+x 3 ] = 0.
(21)
Also, the output resulting from the presence of a second-order kernel is zero, since
C2 [x1 +x2+x]3
-
c2 [x1+x2 - x 2 [x 2 +x 3] -
c2 [x 3 +xl] +
2
[x 1 1 + J
2 [x 2
+ J
2 [x 3 ]
= 0.
(22)
Equation 22 follows as a result of the identity
(x 1+x 2 +x 3)2 -
(x 1 +x 2 ) 2+(x
2 +x 3 ) 2+(x 3 +x 1 )2
+
x2+x2+x
= 0.
(23)
Thus the system shown in Fig. XII-6 has the output g 3 (t) as given by Eq. 16, even in the
case in which N 3 is a general third-order system.
The third-order Volterra kernel
can thus be obtained by means of the circuit of Fig. XII-6, as the third-order impulse
response given by Eq. 19.
One method of obtaining the first- and second-order kernels
is to subtract the third-order kernel from the third-order system, N 3 .
second-order system, N
2,
The result is a
whose kernels can be determined by the methods described
in the previous section.
3.
pth-Order Systems
The Volterra representation of a pth-order system, Np , is given by Eq. 1.
From
our previous discussion of second- and third-order systems, the approach is to form,
from the system Np, a system whose output is H p(X 1 ...
, Xp).
This was accomplished
for third-order systems as a result of the polynomial identities (18), (21), and (23).
In
fact, the form of the system of Fig. XII-6 is immediately apparent from identity (18).
The fact that the output of the system of Fig. XII-6 which is due to the second- and
third-order kernels is zero is a consequence of identities (21) and'(23).
p
The result for
-order systems follows from a generalization of these polynomial identities.
How-
ever, before discussing the general polynomial identity, we shall discuss the identity
that we consider for a fourth-order system.
4! x 1 x 2 x 3 x 4 = (x+..
+x)
4
-
x1
The identity for P = 4 is
x 2 +x 3 4+..
+
(x+x 2
4+ .
+. . .+x 4
(24)
QPR No. 71
171
(XII. STATISTICAL COMMUNICATION THEORY)
Each term in the second term of Eq. 24 is the sum of three different x's raised to the
fourth power.
There are
The second term, then, is the sum of all distinctly different such terms.
()
= 4 such terms.
Each term in the third term of Eq. 24 is the sum of two
different x's raised to the fourth power.
tinctly different such terms.
The third term, then, is the sum of all dis-
There are (4)= 6 such terms. Finally, each term in the
fourth term of Eq. 24 is an individual x raised to the fourth power.
then, is the sum of all distinctly different such terms.
The total number of terms in Eq. 24 is thus (4)
The fourth term,
There are (4)
+ (4+
+
= 4 such terms.
(4)= 24
-
1= 15.
We now can show that if we form, from a fourth-order system, a system according
to Eq. 24, the output will be 4! 3C4 (x 1x 2 x 3 x 4 ).
This is because the output resulting from
the first-, second-, and third-order kernels will be zero.
To show this, we need to
show that the right-hand side of Eq. 24 is zero if each term is raised to a power that is
less than four. That is, we must show that
(xl+. . .+x4 )n
-
+
(xl+2+x
3 n+..
(x+xn+...
-
+. .. +x
for n = 1,2,3.
= 0
(25)
This identity is seen by differentiating Eq. 24 with respect to xl.
3! x 2 x 3 x 4 = (x+. . +x 4 ) 3
-
(x 1 +x 2 +x3 )3+..
Compare Eq. 25 for n = 3 with Eq. 26.
.j
+
The result is
(x1+x 2 3+. ..
-
x.
(26)
The right-hand side of Eq. 26 is identical with
Eq. 25 except that terms that do not contain x 1 are missing. Thus as a function of xl ,
both equations are identical. However, the left-hand side of Eq. 26 is not a function of
x1.
Thus as a function of x 1 , Eq. 25 for n = 3 is a constant.
n = 3 is a constant as a function of x 2 , x 3 , and x 4 .
constant.
Similarly, Eq. 25 for
Equation 25 is thus, at most, a
The constant, however, is seen to be zero by considering the case for which
x1 = x 2 = x 3 = x 4 = 0.
Thus Eq. 25 is valid for n = 3.
To show that Eq. 25 is also valid
for n = 2, we differentiate Eq. 26 with respect to x 1 . We then obtain
0 = (x+.
. .+x 4 ) 2
-
(x 1 +x 2 +x 3 2+. ..
+
(x 1 +x
2
2+...
- x.
(27)
By an argument similar to that for n = 3, we establish the validity of Eq. 25 for n = 2.
Similarly, we can show that Eq. 25 is also valid for n = 1 by differentiating Eq. 27 with
respect to x 1 .
Thus, if we form, from a fourth-order system, a system according to
Eq. 24, the output resulting from the first-, second- and third-order kernels will be
zero and the output that is due to the fourth-order kernel will be
(28)
g 4 (t) = 4! ~C4 (x 1 X2 X4).
3
By choosing each x to be a unit impulse, the fourth-order kernel can be obtained as a
QPR No. 71
172
(XII.
STATISTICAL COMMUNICATION THEORY)
fourth-order impulse response. One method of obtaining the kernels of order less than
four is to subtract the fourth-order kernel from the fourth-order system. The result
is a third-order system, whose kernels can be determined by the methods described
previously.
Now consider the general case for a pth-order system. To obtain a system whose
output is
gp(t) = p!
p(X 1 . x p),
(29)
we form, from the pth-order system, a system according to the identity
p! x...
x
= (xl+..
+
+xp)P - [(Xl+. .. +xpl)P+...
+ (-l)p-l
..
]
+
[(x +. .. +xp_-2)P.
+. .. +xP]
.
(30)
Each term in the second term of Eq. 30 is the sum of (p-l) different x's raised to the
th
p power; thus the second term is the sum of all distinctly different such terms. There
are (pP 1 ) = p such terms. Each term in the third term of Eq. 30 is the sum of (p-2)
different x's raised to the p th power; thus the third term is the sum of all distinctly
(p)(p-1)
different such terms. There are
th 2
such terms. Similarly, for all of the
terms of Eq. 30, the last term is the p
term in which each term is an individual x
raised to the pth power; thus the pth term is the sum of all distinctly different such
terms. There are (P)= p such terms. The total number of terms, Mp, in Eq. 30 thus is
P
p
Mp =
(P) = 2P- 1.
(31)
n=l
The output of such a system will be as given by Eq. 29, since the output that is due to
the kernels of order less than p is zero. This follows from the result that
(x+.
+x )n
[(x +.
.
+x
+...] + .
.
+ (_1)Pl
xn+.
.
+x] = 0
for n = 1, ...
,p-1.
(32)
The validity of Eq. 32 can be established by differentiation of Eq. 30 in exactly the same
manner as for p = 4.
Thus, if we form, from a pth-order system, a system according to Eq. 30, the output
that is due to the first- through the (p-1)th-order kernels will be zero, and the output
that is due to the pth-order kernel will be given by Eq. 29. By choosing each input,
xn(t), to be a unit impulse, the pth-order kernel can be obtained as a pth-order impulse
response. Also, the p-dimensional kernel transform can be obtained experimentally by
choosing each input, xn(t), to be a sinusoid.
QPR No. 71
173
One method of obtaining the kernels of
(XII.
STATISTICAL COMMUNICATION THEORY)
order less than p is to subtract the pth-order kernel from the pth-order system. The
result is a (p-l)th-order system. In this manner, each kernel can be determined successively from the highest-order to the first-order kernel.
4. Suppression of Kernels
We have indicated that each kernel can be determined successively. However, if all
of the kernels of a pth-order system are to be determined, the measurements of the
lower-order kernels can be simplified by initially suppressing all of the even- or oddorder kernels.
This can be accomplished, since from Eq. 2
X2n[(t)] = X 2 n[-X(t)]
(33a)
X2n+l[x(t)] = -3 2n+ [-x(t)].
(33b)
and
Thus the system formed as shown in Fig. XII-7b contains no even-order kernels. Also,
the system formed as shown in Fig. XII-7a contains no odd-order kernels. Thus, for
example, to measure the second-order kernel of a third-order system, we need not subtract the third-order kernel from the system to form a second-order system. Rather,
the third- and first-order kernels can be suppressed by forming a system as depicted
in Fig. XII-7a. The second-order kernel then can be measured by forming, from
the resulting system, a system as shown in Fig. XII-5.
The techniques described in this report
are also useful in the synthesis of desired
N
P
S
x(t)
nonlinear systems.
e()
For example,
is
in the
suf-
N
design of square-law devices, it
(a)
ficient for many purposes to suppress oddorder kernels by means of the system of
Fig. XII-7a.
However, any set of kernels of
a pth-order system can be suppressed by the
1)
x(t)
-
(t
N
0
-
use of the systems that we have described.
5.
Some Measurement Techniques
Our discussion thus far would seem to
indicate that, to measure a p th-order
Fig. XII-7.
(a) Connections for a pthorder system to suppress
resulting
the response
from odd-order kernels.
(b) Connections for a pth_
order system to suppress
the response
resulting
from even-order kernels.
QPR No. 71
kernel, we would require 2P-1 identical pth
order systems. However, we could, instead,
perform 2P-1 measurements on one such
system and then add the outputs in the
manner indicated by Eq. 30. If this procedure is
174
carried out,
then some of the
(XII.
measurement will be redundant.
STATISTICAL COMMUNICATION THEORY)
For example, if each input, x (t), is an impulse, then
the response of the pth-order system for x 1 (t) is the same as for x 2 (t) except for a shift
in time.
Thus to obtain gp(t) as given by Eq. 29, only one measurement is needed to
determine the function that corresponds to the last term of Eq. 30.
separate measurements are required.
redundancies will arise.
Thus, at most, 2P-p
For certain relations among the delays, Tn, other
The minimum number of required measurements is
corresponds to the case in which all delays are equal so that gp(t) = p! h p(t, t, ...
The analysis that we have presented is exact for p th-order systems.
p, which
, t).
However, the
representation of a nonlinear system as a pth-order system may be adequate for only a
limited range of input amplitudes.
There are cases in which this range is sufficiently
small so that the analysis of the nonlinear system by means of very narrow pulses,
which approximate impulses, is not practical.
An example is certain electronic systems
that can be considered linear only for inputs of small amplitudes.
One technique that is
used in such cases is to obtain the step response of the system; the derivative of the
step response is the desired impulse response.
We shall now present an extension of
this technique to systems that can be considered to be of the pth order for a limited
range of input amplitudes.
If we form, from such a system, a system according to
Eq. 30, then the output that is due to the first- through the (p-l)th-order kernels will be
zero, and the output that is due to the pth-order kernel will be given by Eq. 29.
Further-
more, if each of the inputs is within a limited range, then the output that is due to the
kernels of order greater than p can be neglected.
cases is the step.
An input that can be used in such
Let
xn(t) = aul(t-Tn),
(34)
in which u _(t-Tn) is a unit step that starts at t = T n
t-T
S (t) = aPp!
f
-oo
t-T
1 ...
f
p h (7 i
7
1'
-0o
p
)
.
d71,
The output for such an input is
.
.,
dT .
pP
(35)
Consider the difference
s p(t+6) - sp(t) = aPpl
f
t-T
1
+6
1
.'
f
t-T +6
1
n
< 7
< t -T n +6.
n
h (71 ..
Tp) d7 ~ .. , d7T.
(36)
P
Let 6 be small so that we can consider h (71
t-T
p
....
T p7) essentially constant in the range
Then
s (t+6) - s (t)
Sp6
aP6 p
QPR No. 71
pl
h(t-T
1 , . . .,
t-T) = g (t).
p
p
175
(37)
(XII. STATISTICAL COMMUNICATION THEORY)
For p = 1, Eq. 37 reduces to
gl(t) =a
1s
(38)
jt)(8
which is the usual result for linear systems.
is restricted in amplitude,
Thus if the input to the nonlinear system
the p-dimensional impulse response,
gp(t), can be obtained
s p(t), by means of Eq. 37.
from the p-dimensional step response,
M.
Schetzen
References
1. D. A. George, Continuous Nonlinear Systems, Technical Report 355, Research
Laboratory of Electronics, M.I.T.,July 24, 1959, pp. 6-7.
A METHOD OF CONSTRUCTING FUNCTION GENERATORS
C.
In this report we shall describe a method for the construction of function generators.
The method is based upon the fact that the average value of a periodic train of pulses is
Consider the negative triangular wave,
the area of one pulse divided by the period.
ga(t), of amplitude E and period T shown in Fig. XII-8a. By adding a dc voltage, v, to
the triangular wave and half-wave rectifying the resultant, the wave, gb(t), of Fig. XII-8b
is obtained. The average value of this wave is independent of the period, T, and is equal
to v2/2E.
Thus the output of the system as shown in Fig. XII-9 is proportional to v2 t).
The bandwidth of this system is inversely proportional to the period, T, of the triangular
wave, ga(t). By this technique, a large class of functions can be generated by properly
shaping the pulses of the periodic wave.
Thus if the variation of the width of a pulse is
g9 (t)
Fig. XII-8.
(a)
(a) A negative triangular
wave. (b) The wave gb(t)
whose average value is
v2/2E.
9
b (t)i
I
QPR No. 71
\
I
,/\-
t
176
STATISTICAL COMMUNICATION THEORY)
(XII.
Fig. XII-9.
Square-law device.
g(x), in which x is the distance from the peak of the pulse, then the average value of the
v
wave depicted in Fig. XII-10 after half-wave rectification is equal to
Now let g(x) = f'(x).
-
g(x) dx.
Then the average value of the rectified periodic wave can be
1
expressed as 1f(v). For example, for
a square-law device we require that
2
f(x) = x , and thus we require g(x) = f'(x)
to be proportional to x.
For an n th-law
device, we require that f(x)= x n , and
thus, for such a device, g(x) must be
t
. The output of a
proportional to x
diode network for a triangular-wave
input is one source of such a periodic
Fig. XII-10.
Periodic wave pertinent to
general function generation.
By the addition of simple circuits,
many identical function generators can
be constructed from one periodic-wave oscillator. For example, an additional squarelaw device can be obtained from the system of Fig. XII-9 simply with two additional
rectifiers and a lowpass filter. Figure XII-11 is a schematic diagram of such a system.
The two square-law devices of Fig. XII-11 are essentially identical, since they utilize
the same periodic wave, ga(t), as the basis for their square-law character.
1T
i
FULL-WAVE
HALF- WAVE
RECTIFIER
i(t)
RECTIFIER
LOWPASS
FILTER
This result,
v '
OSCILLATOR
v2(t'
FULL-WAVE
hV td
k
a f) - I 2(tt HALF-WAVE
a
I
RECTIFIER
Fig. XII-11.
QPR No. 71
RECTIFIER
v(t)
2LOWPASS
FILTER
Two square-law devices using the same periodic-wave oscillator.
177
(XII.
STATISTICAL COMMUNICATION THEORY)
v1(t)
v2 (t)
ga (t)
FULL-WAVE
RECTIFIER
HALF - WAVE
RECTIFIER
FULL-WAVE
HALF-WAVE
RECTIFIER
RECTIFIER
FULL - WAVE
HALF - WAVE
RECTIFIER
RECTIFIER
that the function generators are essentially identical,
For example,
(t )
FILTER
A multiplier.
Fig. XII-12.
accuracy of the function generator,
2
-LOWPASS
can be used
to increase
the
since undesired nonlinearities can be eliminated.
a usual method of constructing a multiplier from square-law devices is
to make use of the relation 4v1
2 = (v 1 +v 2 )
- ( 1-V
2 ) 2.
duced if the square-law device contains a linear kernel.
However, an error is introSuch an error can be elimi-
nated by the system of Fig. XII-12, in which we have made use of the relation 2v
2 =
(v
2 - v -v 2 . The multiplier of Fig. XII-12 requires one more square-law device.
the additional square-law
However,
device is
obtained with just the addition of two
In a similar manner, the output that is due to other undesired kernels can
rectifiers.
be eliminated (see Section XII-B).
M. Schetzen
D.
OPTIMUM LAGUERRE FINITE-TERM EXPANSION OF FUNCTIONS
This report summarizes results obtained by Claude J.
R. Deal and presented as a
thesis to the Department of Electrical Engineering, M. I. T., September 1962, in partial
fulfillment of the requirements for the degree of Master of Science.
Under certain general conditions, a function f(t) can be expressed as
00
f(t) =
ann
(t),
(1)
n=l
in which {n(t)} is a complete set of orthonormal functions so that
0
f, 0(t
-oo
QPR No. 71
)
mIm~
(t) d
dt =
lIn(2)
m=n
o
m
n.
178
(XII.
STATISTICAL COMMUNICATION THEORY)
an upper limit of the number of terms that can be used in
In physical applications,
the series representation is determined by practical considerations.
The truncated
expansion thus differs from the actual function being expanded by an error, dN(t), which
is a function of N, the number of terms used in the expansion.
This error is
(3)
N(t)= f(t) - fN(t),
in which
N
fN(t) =
(4)
an n(t).
n=l
A measure of the size of this error is the integral-square error, E
N
oo
(5)
N(t) dt.
N= f
-00
The values of the coefficients, an, for which the integral-square error is a minimum
are given by
00oo
an
f
(6)
f(t) pn(t) dt.
-oo
An expression for the minimum integral-square error obtained by substituting
the
values of the coefficients as given by Eq. 6 in Eq. 5 is
N
co
EN
f
-oo
f 2 (t) dt-
n= 1
(7)
an2
The minimum integral-square error can always be made as small as desired by making
N, the number of terms used in the expansion, sufficiently large.
If, however, we set an
upper limit to N, the integral-square error, EN, given by Eq. 7 can be minimized by
stretching or compressing the time scale of the orthonormal set. Analytically, the time
scale of the orthonormal set can be stretched or compressed by replacing the variable
t by pt, in which p is
a positive constant.
orthonormal.
1.
n (pt),
The new set, i
then also is
1
Optimum Laguerre Expansion
The problem of determining the optimum value of the scale factor, p, has been
studied for the particular orthonormal set that is composed of the Laguerre functions.
1
A result of this study is a practical method by which we can locate all those values of
the scale factor, p, for which the integral-square error, as given by Eq. 7, has a local
minimum.
The set of Laguerre functions is orthonormal over the interval (0, oo).
QPR No. 71
179
Also, each
(XII.
STATISTICAL COMIVIUNICATION THEORY)
Laguerre function is realizable as the impulse response of a linear system. This set,
as well as others, with its network realizations has been discussed in detail by Lee. 2
The Laguerre set is obtained by forming an orthonormal set from the sequence of
functions, g (t),
t > 0
(pt)n ePt
in which p is the scale factor.
(8)
n = 0, 1, 2, 3, ...
t < 0
gn(t)
The first few terms and the general term of the
Laguerre functions are
1 0o(p,t) = v/2e-Pt
1 1 (p,t) = /p(2pt-1) e- p t
(9)
(2pt)
n(2pt)n-1
n(n-1)(2pt)n-2
n!
1! (n-1)
2!(n-2)!
n
n
n
. . .
+ (-1)n
e
The problem of interest is that of minimizing EN(p) which, from Eqs. 6 and 7, is
o
EN
=
f
N-1
f (t)dt-
-oo00
Ck(P)
(10)
k=0
and
00
Ck(p)
=
f
f(t)
k(p,t) dt.
(11)
0
The values of p for which EN(p) is stationary can be obtained by differentiating Eq. 10
with respect to p and making use of the identity
alk(P't)
k
ap
2p
k+
k- 1 (pt)
2p
1
ik+l
(
pt)
Then it is found after some algebra that EN(p) is stationary for those values of p for
which
CN(p) CN-1(p) = 0.
(13)
Thus the stationary points of EN(p) can be determined experimentally by measuring the
N th coefficient of the Laguerre expansion, CN-1(p), and the (N+l)st coefficient of the
Laguerre expansion, CN(p).
The stationary points of EN(p) are those values of p for
which either coefficient is zero. Information as to the nature of the stationary points
can be obtained by observing the sign of the coefficients.
Thus if the sign of the
product, CN(p) CN-1(p), changes at a root of Eq. 13, then the stationary point is either
QPR No. 71
180
(XII.
STATISTICAL COMMUNICATION THEORY)
a maximum or a minimum; if,
on the other hand, the sign does not change at a root,
then the stationary point is an inflection point.
Although the procedure derived does not yield the optimum value of p for which
EN(p) is an absolute minimum, we do obtain a set of values of which one is
optimum value.
the desired
The optimum value of p can be determined by measuring the integral-
square error at all the critical values of
integral-square error is
a minimum.
usually small, this method is
p and choosing that value for which the
Since the number of critical values of p is
a definite improvement over having no criterion with
which to select a value of p from its semi-infinite range.
We might expect that if a
function is to be expanded in a series containing only N terms, then, generally speaking,
the minimum of the integral-square error should occur for a value of p at which the
coefficient of the (N+1)
which this is not true.
th
term is
zero.
However,
examples have been obtained for
Therefore, in locating the value of p for which the integral-
square error is an absolute minimum, one should measure the integral-square error at
all values of p, as determined from Eq. 13, for which EN(p) has a relative minimum.
It should be noted that this technique of determining the optimum scale factor, p,
requires knowledge only of the product CN(p) CN-1(p) and does not require specific
knowledge of the function being expanded.
Thus, for example, this technique can be
used to determine the optimum scale factor in the measurement of correlation functions
by Laguerre series expansion.3
Some experimental work was performed and the results
were in agreement with those predicted.
1
M. Schetzen
References
1. C. J. R., Deal, Optimum Laguerre Series Expansion of Functions, S.M. Thesis,
Department of Electrical Engineering, M. I. T., September 1962.
2. Y. W. Lee, Statistical Theory of Communication (John Wiley and Sons, Inc., New
York, 1960).
3. M. Schetzen, Measurement of correlation functions, Quarterly Progress Report
No. 57, Research Laboratory of Electronics, M. I. T., April 15, 1960, pp. 93-98.
E.
DESIGN AND ANALYSIS OF A DC TAPE RECORDING SYSTEM USING TWOSTATE MODULATION
1.
Introduction
Since recording down to zero frequency cannot be achieved by direct means on con-
ventional tape recorders, a low-frequency signal must be converted to a high-frequency
form before it can be reproduced.
QPR No. 71
An investigation of the application of two-state
181
(XII.
STATISTICAL COMMUNICATION THEORY)
modulation to the recording of dc signals is almost completed1 and will be described in
this report.
For the purpose of this investigation we consider two-state modulation to be the
conversion of a slowly varying signal to a rectangular wave that is symmetric about
zero
and
has a duty
taneous
value
highest
frequencies
taneous
value
cycle of average
of the input.
The
present in
of the input
value
frequency
the
very
of this
input and may
signal.
nearly
Demodulation
wave
be
is
proportional
is
to the instan-
much higher
dependent
achieved by
upon
than the
the instan-
passing the two-
state signal through a lowpass filter that removes the higher frequency components of
the wave and leaves the desired output.
A specific modulation system, being developed by Bose,
investigation.
2
was used for much of the
Since the noise introduced by this modulation process is much less than
that introduced by the recording process, we assumed a noiseless modulation system.
Our objectives were to design and build a detecting system for restoring the two-state
wave from the recorder output so that noise in the demodulated output resulting from
the recorder noise would be minimized, and to determine limitations on the over-all
system performance imposed by this recorder noise.
2.
Basic Operation of the System
A block diagram of the system is
from the modulator is
recorded.
shown in Fig. XII-13.
The two-state signal f(t)
The recorder output r(t) consists of an alternating
positive-negative train of approximately Gaussian-shaped pulses as shown in Fig. XII-14.
INPUT
TWO - STATE
r( )
f (t)
DETECTOR
(RESTORES
TWO - STATE
MODULATOR
.() LOWPASS
FILTER
SIGNAL)
RECORDER
Fig. XII-13.
The dc recording system.
The significant noise introduced by the recorder is not additive but appears in two
forms: (a)
a multiplicative random amplitude modulation of the pulse train, and (b) a
random sideways displacement of the individual pulses of the output train.
is chiefly due to mechanical fluctuations in the tape drive mechanism.
This noise
It was observed
that appreciable amplitude variations occur over time intervals that are much longer
than the pulse width.
Regardless of the detecting system used for restoring the two-state wave from the
recorder output, a certain amount of jitter in the zero crossings of the restored wave
QPR No. 71
182
(XII.
RECORDERINPUT
STATISTICAL COMMUNICATION THEORY)
f(t)
Fig. XII-14. Recorder input and output
showing typical recorder
noise.
r (t)
RECORDEROUTPUT
UNDISTURBED
PULSE7
/
I7I
It
Therefore we desire a
g(t) will appear because of the variation in the pulses of r(t).
detecting system that restores the two-state signal so that jitter introduced into the
zero crossings of g(t) is minimized.
3.
Detector Construction
The detector for restoring the two-state signal from the recorder output was constructed by using a standard sequence of Digital Equipment Corporation modules. This
system would locate a pulse of r(t) by detecting the position of the zero slope point at
A bistable multivibrator would be triggered on or off, the state
the peak of that pulse.
depending on the polarity of that pulse. Thus the two-state wave is restored. Details of
construction are presented in the author's thesis.
3
A feature of pulse location by peak detection is that the determined position is
practically independent of the random amplitude-modulation noise from the recorder.
4.
Analysis of the Output Noise
A theoretical analysis was performed to relate the noise appearing in the demodu1
We
lated output to the zero-crossing jitter of g(t) introduced by the recorder noise.
consider g(t) to be a square wave (this is the case for zero input) which has its zeroSpecifically, a zero
crossings phase modulated by a continuous random signal x(t).
+ xQ
o a
s
kT
+ x-,
crossing ordinarily occurring at t = - -n the absence of jitter occurs at t=
as shown in Fig. XII-15.
The following restrictions are placed on x(t):
(a) x(t) must be wide-sense stationary, and
th
zero crossing occurs before
(b) max Ix(t) must be less than Tto ensure that the i
th
the jth when i < j.
Under these conditions the power density spectrum of g(t) for If
is given by
QPR No. 71
183
I < _-was
computed and
(XII.
STATISTICAL COMMUNICATION THEORY)
S (f)
n
2
S16A
1
2
T2
T
k=-
k os (f _2k+l)
2k+k
or alternatively by
Sn(f) -T
R
(0
) + 2
cos k7rT
(- 1 )k R
,
(2)
k= 1
where If <<1/max Ix(t) . Here, Sx(f) is the power density spectrum of x(t), Rx(T) is
the autocorrelation of x(t), T is the period of the unjittered square wave, and A is the
amplitude of g(t).
TWO - STATEWAVE
RESTORED
g (t)
F7
i
t
T
x(T)
T
kT
2
x(-
x( )
2
x (0)
)
2
x(T)
X(kTi
2
T
T
T
Fig. XII-15.
kT
2
2
Jittered square wave illustrating the process x(t).
Sn(f)
Fig. XII-16.
0
Output noise spectrum. (Power
within the shaded area lies out-
T
sx.q
QPR No. 71
side the signal band,
184
If
<f
.)
(XII.
STATISTICAL COMMUNICATION THEORY)
The two representations of S (f) are equivalent; the second is the Fourier series
for the first.
When power in x(t) is concentrated in low frequencies so that the shifted
spectra of Sx(f) do not overlap appreciably, the first form is more convenient than the
second; when the jitter on different zero crossings is relatively uncorrelated, most of
the Fourier coefficients become negligible and the Fourier series is
venient expression.
5.
the more con-
An example of S (f) is shown in Fig. XII-16.
Application of the Analysis to the Recording System
Referring to Fig. XII-16, we see that power in x(t) at frequencies below a =
does not produce amplitude noise in our signal band
f - fo, where 0 < fo
<
- f
That is,
power in the shaded portions of the spectra shown in Fig. XII-16 lies outside the signal
band.
This result implies that "wow" and "flutter" (terms commonly used to describe
effective tape speed variations resulting from both recording and reproducing) at frequencies below a will not contribute amplitude noise to the demodulated output.
In a typical recording application, T = 3000 cps, fo = 300 cps, and a = 2700 cps.
Therefore, wow and flutter below 2700 cps would not produce amplitude noise in the
final output.
There will, however, be time-base variations in the output, but for many
applications this effect can be tolerated.
This result indicates that significant improvement in noise characteristics is to be
obtained when a tape drive system designed particularly for the elimination of highfrequency flutter is used.
The spectrum of the output noise of the system constructed for this investigation
was measured.
The theoretical results given above were used to interpret
these
measurements.
For zero input and a 1-volt amplitude, 3-kc modulation carrier, a 290-Iv rms noise
voltage over a 0-300 cps bandwidth was measured.
For a 0.75-volt dc signal the corre-
sponding signal-to-noise ratio would be approximately 68 db.
D. E.
Nelsen
References
1. D. E. Nelsen, Design and Analysis of a D-C Tape Recording System Using TwoState Modulation, S. M. Thesis, Department of Electrical Engineering, M. I. T., August
1963.
2. A. G. Bose, A two-state modulation system, Quarterly Progress Report No. 66,
Research Laboratory of Electronics, M. I. T., July 15, 1962, pp. 187-189; Quarterly
Progress Report No. 67, pp. 115-119; Quarterly Progress Report No. 70, pp. 198-205;
paper presented at the 1963 Western Electronic Show and Convention, San Francisco,
California, August 20-23, 1963.
3.
D. E. Nelsen, op. cit., Appendix I.
QPR No. 71
185
(XII. STATISTICAL COMMUNICATION THEORY)
F.
EXPERIMENTAL INVESTIGATION OF THRESHOLD BEHAVIOR IN PHASELOCKED LOOPS
The phase-locked loop is
a nonlinear feedback system that can be represented by
the following mathematical model.
(See Fig. XII-17.)
mathematically analyzed in published data.
These
The phase-locked loop has been
models
include
both
linearized
models and various models that take into account the essentially nonlinear nature of the
phase-locked loop.1, 2
FILTER
E (t)
E*(t)
E*(t)
S2(t)
VOLTAGE CONTROLLED
OSCILLATOR
SYSTEM SIGNALS
2) + N(t)
Sl(t) =
2 A sin (wt +
S2(t)
=
2 cos (wt + 92)
E(t)
=
E(t)
= A sin (91- 2) +/2
E*(t)
S l(t) S2 (t)
f h(r)
N(t) cos (wt +
)
±2f(2w)
+
E (t-T) dr
-O
dG2
= K E* (t)
dt
v
Fig. XII-17.
Phase-locked loop.
We have experimentally measured the variance of the phase difference 0 = (61- 2)
of a laboratory model phase-locked loop to compare this measured variance with the
variances of 0 calculated by the various theoretical analyses.
3
The variance of 0 is plotted against the coherent noise-to-signal ratio in the noise
passband of the loop.
This coherent noise-to-signal ratio is NoK/4A 2 . Here, No/ 2 is
the height of the double-sided power density spectrum of the additive noise of the input
that is
N(t) in Fig. XII-17.
K is the open-loop gain of the phase-locked loop.
rms voltage of the input sinusoid.
The noise-to-signal ratio is then
QPR No. 71
_
__--
186
.
A is the
(XII.
STATISTICAL COMMUNICATION THEORY)
EXPERIMENTAL RESULTS
iF
3rd - ORDER
FUNCTIONAL SOLUTION
LINEAR TIMEVARIANT MODEL
Fig. XII-18.
Theoretical and experimental results.
LINEAR MODEL
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
NOISE - TO - SIGNAL RATIO,
0.8
0.9
1.0
NK
o
4A
2
The curves plotted in Fig. XII-18 are the results of theoretical analyses for the
1
linear model, the linear randomly time-variant model, the third-order functional
model, 2 and the experimental results. 3 All of these curves are for first-order loops.
It is noted that the experimental results agree closely with the linear model below
threshold and with the linear randomly time-variant model for the threshold region and
above.
The experimental and theoretical results have been discussed in detail.1-3
A. G. Gann
References
1. H. L. Van Trees, A lower bound on stability in phase-locked loops, Information
and Control 6, 195-212 (1963).
2. H. L. Van Trees, Functional Techniques for the Analysis of the Nonlinear
Behavior of Phase-Locked Loops, paper presented at the 1963 Western Electronic
Show and Convention, San Francisco, California, August 20-23, 1963.
3. A. G. Gann, Experimental Investigation of Threshold Behavior in Phase-Locked
Loops, S.M. Thesis, Department of Electrical Engineering, M. I. T., June 1963.
QPR No. 71
187
(XII.
G.
STATISTICAL COMMUNICATION THEORY)
MULTIPLEX COMMUNICATION SYSTEM USING PSEUDO-NOISE CARRIERS
This report summarizes the results of a study that has been submitted to the
Department of Electrical Engineering, M. I. T., as a joint thesis in partial fulfillment of
the requirements for the degree of Master of Science for J. K. Omura and for the simultaneous degrees of Master of Science and Electrical Engineer for A. J. Kramer.
1. Introduction
The aim of this research was to build an experimental two pseudo-noise carrier
system for studying some of the properties of its behavior.
The advantages of the wideband pseudo-noise communication system are well known.
A major disadvantage of the pseudo-noise carrier scheme is that it is relatively wasteful of channel bandwidth.
In order to compensate for the large bandwidth, several wideband carriers may be put in the same frequency band. If a conventional demodulation
technique is used, each carrier acts as an uncorrelated noise to every other carrier.
Van Trees1 has derived an efficient demodulation technique for an analog pseudo-noise
system.
The experimental system reported here was based on this efficient demodulation technique.
The results of this work indicate that the idea of the efficient demodulation technique
is basically sound and that an extension to more than two pseudo-noise carriers is
warranted.
Several important design considerations for efficient system operation have been
determined.
Figure XII-19 is a block diagram of the efficient one-message pseudo-noise carrier
x
m*(t)
F (s)
e(t)
OUTPUT
s (t)
vco
+
t
e (t)
=
t
f x (u) du
A sin [wt +
+
f
(u) du +
-00
x (t)
=
Pseudorandom Signal
m (t)
=
Modulation Signal
m* (t) =
QPR No. 71
Carrier
f
m*(u)du]
-OD
Output
VCO
=
Voltage- Controlled Oscillator
F(s)
=
Realizable Filter Transfer Function
Fig. XII-19.
Pseudo-Noise
t
t
= cos [wt +
f m(u) du]
-CD
-C0
s (t)
x (t)
Single-message pseudo-noise demodulator.
188
(XII. STATISTICAL COMMUNICATION THEORY)
SUBTRACTION
+
POINT
m (t)
I
(t)
+
t
t
cos [Wo t+jxl (u)du + m; (u)du
t
A-sin
t
t +fxl (u)du +fm;(u)du
-+
-- o
SUBTRACTION
POINT
A-sin [wt +fx
2 (u) du
-C
Fig. XII-20.
+ m (u)du
-C0
cos [
t + fx
-co
2 (u)du
+ fm
2
(u)du]
-CD
0
Two-message pseudo-noise demodulator.
demodulator, and Fig. XII-20 is a block diagram of the efficient two-message pseudonoise carrier demodulator.
Only a brief explanation of the results will be given in this report. More details on
the operation of the two pseudo-noise carrier system and the experimental results may
be found in the authors' thesis.
2
2. Discussion of Results
All data presented here were taken with no modulation. The transmitted pseudonoise carriers are 20 kc wide with center frequency at 70 kc.
Figure XII-21 shows the performance of a single standard phase-lock loop discriminator with a sine-wave carrier used. This performance serves as a standard of
comparison of the performance of the two-message pseudo-noise system.
In Fig. XII-22 we have the performance of a single-message pseudo-noise carrier
demodulator. This system has a wideband carrier. Two such wideband pseudo-noise
signals are demodulated by the efficient two-message demodulator of Fig. XII-20. Note
that this demodulator contains two single-message pseudo-noise demodulators with
interconnecting subtraction points. Figure XII-23 shows its performance. By comparing
its performance with the standard system and with the single-message pseudo-noise
system, we can determine how well the multiplex system works when we have the
QPR No. 71
189
15
10
20
25
30
SNR (db)
Fig. XII-21.
0
Performance of a standard system.
5
15
10
20
25
30
SNR (db)
Fig. XII-22.
QPR No.
71
Performance of a single-message pseudo-noise system.
190
(XII.
STATISTICAL COMMUNICATION THEORY)
So100
S2.1
Fig. XII-23.
Kc
S
o
Performance of a two-
message pseudo-noise
system.
1.I KC
600 CPS
120 CPS
105
10
15
20
25
30
SNR (db)
simultaneous transmittal of two pseudo-noise carriers.
The ordinate of Figs. XII-21-XII-23 is the rms noise output in millivolts.
The
abscissa is the power signal-to-noise ratio (SNR) at the input of the discriminator.
Additive white Gaussian noise was used. The parameter of the various curves is the
cutoff frequency of the lowpass filter at the outputs.
In all of the figures we can see a point at which the noise output of the demodulator
starts to increase rapidly. This increase is due to loss of lock with the incoming
signal when the noise at the input becomes too large.
In comparing Fig. XII-21 with Fig. XII-22, we see how much noise is generated by
slight inaccuracies of the demodulator in matching the pseudo-noise carrier at the
demodulator to the incoming pseudo-noise carrier. Figure XII-23 shows the noise that
is due to both incorrect pseudo-noise matching and interference of the two input signals
The interconnecting subtraction points eliminate most of the interfering pseudo-noise signal to each single-message demodulator.
with each other.
We observe that, when the signal-to-noise ratio is high, the two-message system
works almost as well as the one-message system. As one would expect, the threshold
in the two-message system is higher.
3.
Conclusion
Enough information was obtained to show that the basic idea for an efficient demodu-
lator of a multiplex pseudo-noise system is feasible.
The experimental data show
that good subtraction of the interfering pseudo-noise signal is very important.
QPR No. 71
191
In
(XII.
STATISTICAL COMMUNICATION THEORY)
the laboratory system the most important area of
improvement is better front-end
subtraction.
The experimental results indicate that an extension of the two pseudo-noise carrier
system is warranted.
This will not present any important problems that have not
already been encountered in the two-carrier system.
pseudo-noise system has been given.
A block diagram of an n-message
3
A. J.
Kramer, J.
K. Omura
References
1. H. L. Van Trees, An Efficient Demodulator for an Analog Pseudo-Noise Multiplex
System, Report 65G-3, Lincoln Laboratory, M. I. T., July 2, 1963.
2. A. J. Kramer and J. K. Omura, A Multiplex Communication System Using PseudoNoise Carriers, S. M. Thesis, Department of Electrical Engineering, M.I. T., August 19,
1963.
3.
H. L. Van Trees, op. cit.; see Fig. 1.
QPR No. 71
192