MITLibraries
Document Services
Room 14-0551
77 Massachusetts Avenue
Cambridge, MA 02139
Ph: 617.253.5668
Fax: 617.253.1690
Email: docs@mit.edu
http://libraries.mit.edu/docs
DISCLAIMER OF QUALITY
Due to the condition of the original material, there are unavoidable
flaws in this reproduction.
We have made every effort possible to
provide you with the best copy available. If you are dissatisfied with
this product and find it unusable, please contact Document Services as
soon as possible.
Thank you.
Pages are missing from the original document.
l
I
STATE VARIABLES,
THE FREDHOLM
AND OPTIMAL
THEORY
COMMUNICATIONS
by
ARTHUR BERNARD BAGGEROER
B. S. E. E., Purdue University
(1963)
S. M.,
Massachusetts
Institute of Technology
(1965)
E. E.,
Massachusetts
Institute
(1965)
SUBMITTED
of Technology
IN PARTIAL FULFILLMENT
REQUIREMENTS
FOR THE DEGREE
OF THE
OF
DOCTOR OF SCIENCE
at the
MASSACHUSETTS INSTITUTE OF TECHNOLOGY
January, 1968
ElectricalEngineeri~: Janu~~y 15, 1968 -
Signat~::a~~~~~~O~fCertified
by
Accepted by_~'
Chairman
!
v
Departmental
Thesis
Committee
Supervisor
.
on Graduate
_
Studies
2
STATE VARIABLES, THE FREDHOLM THEORY
AND
OPTIMAL COMMUNICATIONS
by
ARTHUR BERNARD BAGGEROER
Submitted to the Department of Electrical Engineering on
January 15, 1968 in partial fulfillment of the requirements
for
the degree of Doctor of Science.
ABSTRACT
This thesis is concerned with the use of state
variable techniques for solving Fredholm integral equations, and
the application of the resulting theory to several optimal communications problems.
The material may be divided into the
following areas;
(i)
the solution of homogeneous and nonhomogeneous Fredholm
integral equations;
(Li) optimal signal design for additive colored noise channels;
(iii) optimal smoothing and filtering with delay;
(iv) smoothing and filtering for nonlinear modulation systems;
(v) estimation theory for a distributed environment.
A method for solving Fredholm integral equations
of the second kind by state variable techniques is derived.
The
principal advantage of this method is that it leads to effective
computer algorithms for calculating numerical solutions.
The
only assumptions that are made are: (a) the kernel of the integral
equation is the covariance function of a random process; (b) this
random process is the output of a linear system having a white
noise input; (c) this linear system has a finite dimensional
state-variable
description of its input - output relationship.
Both the homogeneous and nonhomogeneous integral
equations are reduced to two linear first-vector
differential
equations plus an associated set of boundary conditions.
The
coefficients of these differential equations follow directly from
the matrices that describe the linear system.
In the case of the
homogeneous integral equation, the eigenvalues are found to be
the solutions to the transcendental
equation.
The eigenfunctions
also follow directly.
In addition, the Fredholm determinant function is
related to the transcendental
equation for the eigenvalues.
For
the nonhomogeneous equation, the vector differential equations
are identical to those that have appeared in the literature for
tz
3
optimal smoothing.
The methods for solving these equations are
discussed with particular consideration given to numerical
procedures.
In both types of equations
several analytical and
numerical examples are presented.
The results for the nonhomogeneous equation are
then applied to the problem of signal design for additive colored
noise channels.
Pontryagin's Principle is used to derive a
set of necessary conditions for the optimal signal when both its
energy and bandwith are constrained.
These conditions are
then used to devise a computer algorithm to effect the design.
Two numerical examples of the technique are presented.
The nonhomogeneous Fredholm results are applied
to deriving a structured approach to the optimal smoothing
problem.
By starting with the finite time Wiener'<Hopjtequati.on
we are able to find the estimator structure.
The smoother results
are then used to find the filter realizable with the delay. The
~erformance of both of these estimators are extensively discussed
and illustrated.
The methods for deriving the Fredholm theory results
are extended so as to be able to treat nonlinear modulation systems.
The smoother equations and an approximate realization of the
realizable filter are derived for these systems.
an approach to estimation on a distributed
Finally
medium is introduced.
The smoother structure when pure delay
and a simple case is
enters the observation method is derived
illustrated.
I
I
I
Thesis Supervisor:
Title
>
Harry L. Van Trees
Associate Professor
of Electrical
Engineering
4
ACKNOWLEDGEMENT
I wish to acknowledge and express
thanks to Professor
Harry L. Van Trees
of my doctoral studies.
my sincerest
for his patient supervision
His constant encouragement
provided a continual source of inspiration
and interest
for which I cannot be
too grateful.
I also wish to thank Professors
W. M. Siebert for their interest
W. B. Davenport and
as the readers
of my thesis
committee.
Several people have made valuable contributions
this thesis.
I have profited greatly from my association
fellow student Lewis D. Collins and Professor
Discussions
with Theodore
to
with my
Donald L. Snyder.
J. Cruise have clarified
several
issues
of the material.
Earlier
drafts of much of the material
Miss Camille Tortorici,
Mrs. Vera Conwicke.
while the final manuscript
Their efforts
The work was supported
Foundation,
a U. S. Navy Purchasing
Service Electronics
computations
Program.
were performed
Finally,
was done by
appreciated ..
in part by the National
Science
Contract and by the Joint
I am grateful for this support.
The
at the M. 1. T. Computation Center.
I thank my wife Carol for both her encour-age-
ment and her patience in the pursuit
7P
are sincerely
were typed by
of my graduate
education.
5
TABLE OF CONTENTS
ABSTRACT
2
ACKNOWLEDGEMENT
T ABLE OF CONTENTS
5
LI ST OF ILLUSTRATIONS
8
CHAPTER I
INTRODUCTION
10
CHAPTER II
STATE VARIABLE RANDOM PROCESSES
16
A.
Generation
ofState Variable
B.
Covariance
Functions
C.
The Derivation
Random Processes
for State Variable
of the Differential
Processes
for
Equations
16
22
26
't-
1
K (t,'r)f(t)d--r
~
-
b
CHAPTER III
HOMOGENEOUS FREDHOLM
INTEGRAL EQUATIONS
36
A.
The State Variable Solution to Homogeneous
Fredholm Integral Equations
36
B.
Examples
C.
The Fredholm
D.
Discussion of Results
Integral Equations
CHAPTERIV
of Eigenvalue
and Eigenfunction
Determinant
Determination
Function
for Homogeneous
49
59
Fredholm
NONHOMOGENEOUS FREDHOLM
INTEGRAL EQUATIONS
69
71
A.
Communication Model for the Detection
Signal in Colored Noise
of a Known
B.
The State Variable Approach to Nonhomogeneous
Fredholm Integral Equations
75
C.
Methods of Solving the Differential Equations
Nonhomogeneous Fredholm Equation
80
for the
72
6
D.
Examples
Fredholm
E.
Discussion
Equations
CHAPTER V
of Results
for Nonhomogeneous
92
Fredholm
OPTIMAL SIGNAL DESIGN FOR ADDITIVE
COLORED NOISE CHANNELS VIA STATE
VARIABLES
105
109
109
A.
Problem
B.
The Application
C.
The Signal Design Algor.ithm
135
D.
Examples of Optimal Signal Design
139
E.
Signal Design With a Hard Bandwidth Constraint
156
F.
Summary
160
CHAPTER VI
Statement
113
of the Minimum Principle
and Discussion
164
LINEAR SMOOTHING AND FILTERING
WITH DELAY
A.
The Optimal Linear
B.
Covariance
C.
Examples
D.
Filtering
E.
Performance
F.
Examples
G.
Discussion
CHAPTER VII
tt
of Solutions to the Nonhomogeneous
Equation
of Error
Smoother
166
for the Optimum Smoother
of the Optimal Smoother
175,1
Performance
194
With Delay
208
of the Filter
With Delay
of Performance
of the Filter
217
With Delay
223
and Conclusions
226
SMOOTHING AND FILTERING
MODULATION SYSTEMS
A.
State Variable
Modelling of Nonlinear
B.
Smoothing for Nonlinear
C.
Realizable
Filtering
FOR NONLINEAR
Modulation
Systems229
Modulation Systems
for Nonlinear
228
Modulation Systems
232
235
7
D.
Summary
245
CHAPTER VIII STATE VARIABLE ESTIMATION IN A
DISTRIBUTED ENVIRONMENT
A.
State Variable
Environment
Model for Observation
B.
State Variable
Estimation
C.
Discussion
of Estimation
in a Distributed
in the Presence
for Distributed
of Pure Delay
Systems
246
247
256
275
CHAPTER IX
SUMMARY
278
APPENDIX A
DERIVATION OF THE PROPERTIES OF
COVARIANCES FOR STATE VARIABLE
'RANDOM PROCESSES
283
A,PPENDIX B
COMPLEX RANDOM PROCESS GENERATION
286
REFERENCES
310
BIOGRAPHY
314
8
LIST OF ILLUSTRATIONS
2. 1
State Equation Model for Generating
2.2a
Covariance
2.2b
Spectrum for a Second Order System
28
3. 1
det (A(>-.)1 vs ,'\
58
3.2
~
3. 3a
d(A) vs. A Exact and Asymptotic
67
3.3b
In d()..)vs ,"), Exact and Asymptotic
68
4. 1
System Model for the Detection
in Colored Noise
74
4.2
d2 an? di
Function
vs , T
K (t,"r)
19
s.
Function for a Second Order System
n
=
27
1, 2, . . . , 6
vs. n for a First
60
of a Known Signal
Order Covariance
100
stt) = stntnnt)
4.3
g(t) and stt) foro;a Second Order
Function
sft) = sin(-rr-\:')
Covariance
102
4.4
g(t) and stt) for a Second Order
Function
sft) = sin(Bm)
Covariance
103
4.5
2
d and d2 vs , n for a Second Order
stt) = sin(nri'rl:)
FunctiorP
5. 1
Loci of Points for det(D(),
E
'>\»)
•
5.2
Performance
Candidates
5.3
Optimal stt) and g(t) for a First
=
Covariance
0 First
104
Order Spectrum
vs. Bandwidth for the Optimal Signal
First Order Spedfrum
Optimal stt) and g(t) for a First
B2 = 36
5.5
Loci of Points for det[D(').1l'X~
=
144
Order Spectrum
and
146
Order Spectrum
and
147
B2 = 9
5.4
142
0 Second Order Spectrum
152
9
5.6
Performance
Candidates
vs. Bandwidth for the Optimal Signal
Second Order Spectrum
5.7
O~timal stt) and g(t) for a Second Or-der- Spectrum
B = 13
6. 1
z.(t/T)
P =
6.2
£(t/T)
P =
154
and "2:.(t/t)for a First Order Process
1 Stationary Process
and Z1t/t) for a First
1) = LoG
199
Order Process
201
2s1 (~+
6.3
<[(tiT) and ~(t/t) for a First Order
Known Initial State
P = 0
6.4
~11 (tiT) (normalized) for a Second Order Process
Transmitted Signal o(1Xl
(t) T ~2dxl (t) I dt
6.5
153
Process
T
2:11 (til T) and z..(T IT) vs , C(1. (modulation
dx 1(t)/ dt) for a Second Order Process
203
with
206
of
207
coefficient
6.6
Covariance of Error for the Filter
for a First Order Process
With Delay
7.1
Diagram for Invariant
8. la
Distributed
System Observation
8. Ib
Space-time
Domain for the Propagation
B-1
Spectrum for a Second Order Complex Process
Poles are Closely Spaced
when
306
B-2
Spectrum for a Second Order Complex: Process
Poles are Widely Spaced
when
307
B-3
Spectrum for a Second Order Complex Process
a Mean Doppler Frequency Shift
with
308
B-4
Spectrum for a Second Order Complex Process
Nonsymmetric Pole Locations
with
309
225
Imbedding Argument
239
Model
254
Medium
255
10
CHAPTER I
INTRODUCTION
One of the more powerful techniques
of communication
problems
is the Fredholm
theory. 1, 2, 3, 4 Often, however,
used in the analysis
integral
equation
this theory is difficult to use because
solution methods are either too tedious for analytic procedures
awkward for convenient implementation
In recent years,
on a digital computer.
state variable
increasingly
useful,
especially
is primarily
due to their adaptability
several
integral
problems
equations.
important
the most significant
on linear filtering
to computational
b
9,10
approaches.
theory for solving
We shall then apply this theory to
methods have already provided solutions
problems
in communication
theory.
Undoubtably
of these is the original work of Kalman and Bucy
theory.
have used these techniques
processes.
have become
in optimal communications.
State variable
to several
techniques
in optimal control theory. 5, 6,7 This
In this thesis we shall deve lop a state variable
Fredholm
or too
8
Starting with this work,
many people
for the detection and estimation
of random
11
The major advantage that these techniques
that they lead to solutions
which are readily
implemented
digital computer.
Their essential
random processes
which are involved are represented
of differential
covariance
equations
functions.
rather
offer is
on a
aspect is that the systems
in terms
than by impulse, responses
Since the digital computer
for integrating
differential
equations,
of formulation
leads to convenient computational
or
and
is ideally suited
we can see how this type
methods of
solutions.
There is a second important
techniques.
application
theorists
methods.
problems,
Historically,
advantage of state variable
the concept of state
in optimal control theory.
Over the years,
have developed a vast literature
As a result,
found its first
pertaining
in using a state variable
we can expropriate
control
to state variable
approach to our
many of the methods that have been
developed in this area.
The application
theory to communications
integral
probably
equation,
is certainly
most familiar
The homogeneous
and e igenvalue s , is
11 S'mce t hiIS expansron
.
to see why we are interested
b
equation
in the context of a Karhunen- Loeve expansion
point in the analysis
eigenfunctions
integral
well known.
with its elgenfunctdons
of a ran d om process.
starting
of the Fredholm
and eigenvalues
of a particular
'
t h eory IS
problem,
in being able to determine
for this equation.
0f ten
t he
it is easy
the
Similarly,
the
12
nonhomogeneous equation is often encountered.
the optimal receiver
and its performance
signal in additive colored noise.
Its solution specifies
for detecting a known
3,4 Other applications
of it include
the solution to the finite time Wiener- Hopf equation and accuracy
bounds for parameter
estimation.
The major difficulty in using the Fredholm theory is
in obtaining solutions to these equations.
limited number of cases,
at best,
lies.
finding analytic solutions is difficult
while current numerical
of computer time.
With the exception of a
methods tend to use a large amount
This is where major contribution
of this thesis
For a wide class of Fredholm equations of interest
we shall
apply state variable
methods to the problem of finding solutions
to these equations.
Because of the inherent computational
that these techniques
offer,
advantages
we shall be able to devise a solution
algorithm that is both well suited and efficient for implementing
on a digital computer.
We shall find,
general interest
however,
that our solutions are of more
than for just solving these integral
We shall apply our results
to several
problems
equations.
in communications.
By coupling our method for solving the nonhomogeneous
equation with the Minimal Principle
shall be able to formulate
of optimal control theory,
and solve a signal design problem for
additive colored noise channels where bandwidth and energy
.
.
d 13,6
constr-aints are Impose .
b
we
13
We can also use the nonhomogeneous
derive the state variable
With the smoother
equation to
form of the optimal smoother.
14,15,16
equations we shall be able to find a structure
for the filter realizable
with a delay.
We shall also be able to
analyze the ner-for mance of both structures.
We shall also find that the technique we used in
solving the Fredholm
equations
may be used to solve new problems.
By extending these techniques, , we shall be able to find the
s moothe r equations for nonlinear
bor-r-owing results
R1 '"'lroximation
from control theory,
to the realizable
far this p-r-oblem,
modulation systems.
we shall also derive an
filter from the smoother
quite divorced
from the Fredholm
our approach.
In processing
enter the signals.
however,
array
process.
structure
delay factors
when pure delay enters
this leads to a significantly
in the thesis.
often
approach
our
approach for simplicity.
to develop and use a Fredholm
concepts for describing
theory approach;
longer derivation.
let us outline the sequence of
In Chapter 2 we shall introduce the
random processes
We shall also derive an important
b
data,
This is due only to
We use a variational
Before proceeding
material
theory.
is apparently
In this topic we shall use a variational
to derive the smoother
It is possible
structure
17
The final topic that we shall treat
observation
Again
by state variable
methods.
result that is used throughout
------------------------------_
_--------_._-
.•
14
the thesis.
In Chapter 3 we shall consider
homogeneous Fredholm
transcendental
of differential
the
2,4 In Chapter 4, we shall consider
equation solution.
We shall derive a set
Then we shall
solution methods which exist for solving this
set of equations.
In the remainder
results
a
and then determine
equations that specify its solution.
several
particular
We shall derive
We shall also show how to calculate
determinant.
the nonhomogeneous
present
equation.
equation for the eigenvalues,
the eigenfunctions.
Fredholm
integral
the solution of the
of Chapters
2-4.
of the thesis we shall apply the
In Chapter 5 we consider
optimal
signal design for detection in additive colored noise channels.
In Chapter 6, we present
an extensive
and filtering
In Chapter 7, we extend our results
with delay.
to treat smoothing and filtering
while in Chapter 8 we present
for nonlinear
two reasons.
illustrate
present
of linear smoothing
modulation systems,
an approach to estimation
when delays occur in the observation
We shall present
discussion
theory
method.
many examples.
We do this for
We shall work a number of analytic examples to
the use of the methods we derive.
a number of examples
We shall also
analyzed by numerical
In the course of the thesis we shall emphasize
of our methods.
methods.
the numerical
aspects
We feel this is where the major application, of
much of the material
lies.
Most problems
are too complex to be
.............., .......
--------------
---._-_
_------------------~------------------_.-
..
15
analyzed analytically,
is a very relevant
so finding effective numerical
problem.
We also want to indicate our notational
Generally,
vectors
scalars
conventions.
are lower case symbols which are not underscored;
are lower case symbols which are underscored;
are upper case symbols which are not underscored.
b
procedures
and matrices
-------------------------------
16
CHAPTER II
STATE VARIABLE RANDOM PROCESSES
In this chapter
and properties
need.
First,
generation
we shall introduce
of state variable
random processes
that we shall
we shall review the ideas of the description
of random processes
using state variable
we shall develop some of the properties
of these processes.
and
methods.
Then
of the second order moments
We shall also introduce
we use in many of our examples.
derivation
some of the concepts
Finally,
two processes
which
we shall present
which is common to many of the problems
a
that we
shall ana lyz e .
A.
Generation
of State Variable
Random Processes
In this section we shall briefly review some of the
concepts associated
with the description
processe s using state variable
establish
methods.
our notation conventions
detailed discussion
we refer
In the application
communication
relevant
information,
to
For a more
3 and 8.
of the state variable
methods to
the random processes
as being generated
that is excited by a white noise process.
of random
This is done principally
and terminology.
to references
theory problems,
are usually characterized
and generation
of interest
by a dynamical
Consequently,
which must somehow be provided,
system
the
is the
---------------
17
equations describing
the operation
of the dynamical
description
of the white noise excitation
density(ies)
or moments of the processes.
that we shall assume regarding
rather
system and a
than the probability
It is this point of view
the description
of our random
processes.
This is not a very restrictive
generate
a large class of processes
.can generate
rational
the important
spectra
assumption,
of interest.
class of stationary
quite conveniently
as we can
In particular,
processes
we
with
using constant parameter,
linear
dynamical systems.
The majority
generated
of the processes
that we shall discuss
by a system whose dynamics may be described
of a finite dimensional,
linear,
ordinary
differential
are
in terms
equation,
termed the state equation,
d~~t)
=
F(t)!(t)
+ G(t)~Jt), (linear state equation))
(2. 1)
where
!(t) is the state variable
vector
~(t) is the white excitation
(n x 1),
process
(m x
F(t) (n x n) and G(t) (n x m) are matrices
determine
(For notational
processes.
b
we shall work with continuous time
In the study of processes
this introduces
difficultie s; however,
that
the system dynamics.
simplicity,
dynamical system,
1),
generated
by a nonlinear
some attendant
mathematical
we shall not dis cuss them here.
10
) In general,
18
we shall assume that ~(t) is white, i. e.,
derivative
of an independent increment
have (assuming
it may be interpreted
process.
as the
Consequently,
we
zero mean)
(2. 2)
In order to describe
a state variable
equation completely,
the initial state of the system must be considered.
with representing
a random process
We shall assume
over the time interval
that the initial
mean random vector with a covariance
E [ x(T )xT(T )]
-
0 -
0
=
matrix
T
o
initial conditions,
:5 T :5
covariance
0
given by
(2. 3)
input signal and deter-
knowledge of ~(To) and U(T) for
x(t) for all t , With a random
-
random initial conditions
matrix
-
0
t is sufficient to determine
input and/or
state x(T ) is a zero
p .
In the case of a deterministic
ministic
We are concerned
we can determine
the
of the state vector,
K (t , T) = E[x(t)xT( T)] ,
x
for all t and
-
T
greater
than To.
only when the state representation
in Eq. 2. 1.
(2.4)
-
We should note that this is true
is linear
as we have assumed
I
,,-...
~
~
.....
......,
~I
~
<.9
z
~
0::
,..-A-,
~
~
0::
0
l-
o
w
>
W
.-ti
({)
~
~
t)
""'--'"
~
~
,;:,,,
..........
~
~
"
xl
0
W-l
>«
a:: z
WC.9
co({)
(()-
0
c
'-
x
~
~o
19
..........
z
-
0
I-
W
'--""
5
:c
.....
w
X
<.)
-
«
~
(,()
l~
W
L..---J
::31
'-"
......
.........
J-::31
..........
..,--,
,--
II
a
......
I
~
..........
xl
~
0
E
~I
'--y-J
w
II
..........
~
......'"
~I
""'--'"
~
...... ~
......
..........
~
(9
X
c
""'--'"
......
>'1
..........
""'--'"
w
z .....
W
<.9
0::
0
LL
-.J
W
0
0
~
Z
-
0
«
J-
:::>
w
C
W
J-
<t:
t(f)
..........
..-
""'--'"
::31
......
..........
""'--'"
LL
c
X
c:
""'--'"
""'--'"
......
---XI
......
----
0
z
:::>
~
a
W
z
0
..
~
~
0::
(f)
I-
<!
t-
w
W
a
:::::>
«
I-
-
a
Z
0
CD
w
t)
(f)
II
......
---~,
""'--'"
......
~
::31
""'--'"
....
""'--'"
..........
(9
+
.......
..........
~
xl
......
---u
....
"'C
1.L.
""'--'"
,-......
)(
.........
"0
..-I
C'3
bJ)
~
.r-f
20
Generally,
one does not observe the entire state vector
at the output of a dynamical
system,
~,
in many case s only the
first component of the vector is observed.
specify the relationship
state vector,
observation
between the observed
of the dynamic system.
random processes
Consequently,
relationship
random process
For the majority
that we shall consider,
is a linear,
and the
of the
we shall assume
that the
possibly time -varying,
i.e. , the observed
no memory transformation,
we must
random process
l(t) is given by
( observation
(If the observation
describing
(2.5)
is a linear transformation
and this transformation
state variables,
equation).
is representable
in terms
1. e. , there is an ordinary
the operation,
which involves memory,
of a system
differential
equation
we can reduce it to the previous
simply augmenting the state vector
of
and then redefining
case by
the matrix
C(t).)
In Fig. 2. 1 we have illustr ate d a block diagr am of the dynamic
system that generates
Finally,
the random processes
in a communications
noise is often present
shall consider
.,!"(t)
=
~(t)
of interest.
context,
in the actual observation.
additive white
Consequently,
signals of the form
+ w(t),
(2. 6)
where
~(t) is a process
generated
as previously
w(t) is a white noise process.
l,...'
we
discussed,
21
We assume that ~(t) has zero mean and a covariance
matrix given by
by
(2. 7)
where R(t) is a positive definite matrix.
Before proceeding
several
comments
It is often convenient to describe
in terms
do this,
of the system that generates
e. g. constant parameter
that may be generated
by a dynamical
descripti on, e. g. stationary
pr oces ses,
for x(T ), u(t) and w(t).
-
necessary
0
-
it is unnecessary
approach rather
In two of the chapters
systems
medium,
for the description
Finally,
state equation.
b
however,
Gaussian
for many of our
linear
approach.
linearly
by nonlinear
which
dynamical
through a distributed
in terms
Since the notation required
of these processes
we shall defer introducing
is peculiar
it until then.
we shall usually work with low pass waveforms.
In Appendix B we have introduced
variable.
of Gaussian
method cannot be described
of the generation
to the individual chapter
the assumption
which are generated
the observation
of a finite dimensional
or the Wiene r proce ss .
we shall analyze problems
or those which are observed
1. e.,
with a constant state
since we use a structured
than an unstructured
involve either processes
to the processes
We shall indicate whenever it is
-
to introduce this assumption;
derivations
On occasion we shall
refer
system
We have avoided introducing
statistics
the random processes
them.
systems
are in order.
the concept of a complex state
This concept allows us to analyze bandpass
waveforms
of
22
interest
with very little modification
to the low pass theory that we
shall develop.
B.
Covariance
Functions
for State Variable
Several of our derivations
Processes
concern the covariance
Ki(t, T) of a random process .l(t) which is generated
described
in the previous
section.
review how this covariance
describe
the system
can be related
for generating
The covariance
In this section
matrix
matrix
by the methods
we shall briefly
to the matrices
which
the random processe s.
of l(t) is defined to be
(2.8)
By using Eq. 2.5,
covariance
matrix
K (t , T)
l
=
K (t, T) is easily related
l
.
to the
of the state vector x(t),
T
C(t)K (t , T)C (T).
(2. 9)
~
In Appendix A, the following result
is shown:
e(t, T)K (T, T) for t ~ T,
K (t , T)
x
=
X
(2.10)
where e(t, T) is the transition
matrix
b
F(t), 1. e. ,
matrix for the system defined by the
23
d
dt e{t, T)
e( T, T)
Furthermore,
satisfies
=
=
(2.11a)
F{t)e(t, T),
(2. lIb)
1.
in Appendix A, it is shown that the matrix K (t, t)
x
the following differential
dt K (t, t)
x
d
=
F{t)K (t , t)
x
equation.
+ Kx (t, t)FT{t) + G{t)QGT{t),
t
(2.12a)
> To'
with the initial condition
K (T ,T )
x 0
0
=
(2.12b)
P .
Because
0
many of our examples
processes
and constant parameter
comments
regarding
matrices
describing
sequently,
their
systems
generation.
the generation
the transition
matrix
concern
stationary
we shall make some
Let us assume
of l{t) are constant.
that the
Con-
e{t, T) is given by the matrix
exponential
"(t
c ,T ) -e F(t-T) .
Furthermore,
let us assume that P
(2.13)
o
is chosen to be the steady
24
•
;'<
state solution P to Eq. 2.12a.
00
function only of At.
Therefore,
K (t , t + A t) is a
~
We have
e -FAt P
»
oo
At:S
0,
K (t , t+At =
(2.14)
-
x
At
2=
We shall use two particular
several
of our examples.
or one -pole , process.
The first
o.
stati onary proce sses in
is the first
The covariance
order Butterworth,
function for this proce ss is
(2.15)
The state equations
which describe
the generation
of this process
are
*One
can evaluate
shown that
Poo using transform
techniques.
It is easily
25
=
~~t)
+ u(t),
-kx(t)
t
>~,
(2. 16a)
= x(t),
y(t)
E [ u(t)u (T)]
(2. 16b)
= 2kS
o(t -T),
(2. 16c)
(2. 16d)
The matrices
involved are
F
=
-k
G
=
=
1
Q
c =
1
=
S
Po
We shall use this process
of the techniques
aspects
process
is generated
matrices
describing
F=
G=
Q
=
l:o _lJ
GJ
160
to illustrate
analytically
many
that we shall develop.
The second process
the numerical
a-e)
(2.17
2kS
we shall use in examples
of an analysis
using our techniques.
by a two dimensional
the process
generation
C =[ 1
p
o
= [4
0
to illustrate
This
state equation where the
are
0]
4:J
(2. IBa-e)
26
The process
l(t) is a stationary
process
whose covariance
function is
(2. 19a)
and whose spectrum
is
40
sy (w) =
(2. 19b)
We have illustrated
2.2b.
We have included this process
of the computational
to state matrices
aspects
any analysis
prohibitive
amount of time.
The Derivation
2. 2a and
to illustrate
some
We have also chosen
of y(t) has a peak in it
some interesting
aspects
We should also note that in all our
examples,
G..
of our techniques.
This will introduce
some of our examples.
in Figs.
principally
such that the spectrum
away from the origin.
~
these functions
involving this process
of the Differential
would require
Equations
a
for
K (t , T) f( T) d T
X.
-
Many of the problems
shall analyze involve the integral
in communication
operation
theory that we
27
Fig 2. 2a
If if
-; :i [TiT
Covariance Function for a
Second Order System
ILl
Ky(t. t+At} = ~ exp( -I6t\ }(3cos(3at}+sin(
iT
I~dr
h
1+
-t-t
+t
r ~ . ~I
~t
I++!--l.++-L-H+l-P-++:.+j.j.j"
Hi+H+Y++H·
i~ -l-
"~t
If
l~ •
I ~I
Hi·'
:r
0
IF
u
If
1+
., tr
!
~ 4
"a14i
Ht:
r
111:tt!it ·ftt:::ji·ntl+.1'f:ti
1,
t\
~tt1
, ~
..
-1
It
t
It
~
1m ~~
t
j
rJ
J
Il
ttl\
itt.
Lfl
f+
31Atl)}
28
..,
"
11"
9.
- ...• -c--v-t-t-t-t
;~.;..'+
t\t!f·
ttl
Fig.
2.2b
,;·i,!.
7.
'-",-
t
Spectrum for a
Second Order System
!
+.1 -
1
t
,+
tp
!
,
...
..
;
...
~T:
;·'!·t
.8
:i!.
.7
,- n
,-
~m
t
•5
t'
"
w
,,+,~ t
+
.
·~r~ trl:r;.H
.4
r~"'H
t"
r"
flit!
..
- +,
+i+
-+H··
+
- tj-
1"
It
T
,...
._-
,.
.....
...
t
"
!
·;-t
t
It:t
I","
tf 1--
:Ht
,·,t·
h
"i'f4-+
t \
+-
:;:'1;"
"1
±li'
t.,
.e.
lit!
rrr
f·
...+
',:1,6
• 1
C"
."
,9
;..f
29
K (t , T) f( T) dT ,
~(t)
Y..
In the study of Fredholm
the eigenfunction
integral
equations i(t) is either
1(t) in the homogeneous
.B:(t)in the nonhomogeneous
is the integral
(2. 20)
-
operation
case.
specified
operation.
In linear estimation
theory,
by the Wiener-Hopf
equation.
Solving these differential
equivalent to performing
Eq. 2.20.
the integral
operation
In many of our derivations,
ferential
equations
ferential
equations.
to convert
Let us now proceed
Eq. 2.9,
to
case, or it is the solution
this section we shall derive a set of differential
integral
related
equations
equations
specified
this
In
for this
is
by
we shall use these dif-
an integral
operator
with our derivation.
to a set of dif-
By using
we may write Eq. 2.-20 as
1.(t)
=
C(t)i(t),
(2.21)
where
Tf
W)
A
S
Kx(t, T)C T( Tl.!( T)dT,
T 0:5 t :5 T f"
(2.22)
To
We shall now determine
in terms
we have
of the function i(t).
a set of differential
Substituting
equations
Eq. 2•..l0in Eq. 2.22,
30
e(t,-r)K (-r, -r)CT (-r)f(-r)d-r
~(t)
x
-
Tf
S
+ Kx(t,t)
T
T
9 (T,t)C (T)f(T)dT,
Toost
(2. 23)
os Tf'
t
If we differentiate
Eq. 2. 23 with respect
to t , we obtain
t
=
5
Tf
S
+Kx(t, t)
89Ta~T,t) C T (T)f( T)dT , To os t os Tf'
(2. 24)
t
We have used Eq. 2. lIb and cancelled
first term
of the right-hand
two equal terms.
side of Eq. 2. 24 we substitute
In the
Eq.
2. lla,
and in the last term we use the fact that e T(-r, t) is the transition
matrix
for the adjoint equation of the matrix
a
at
T
e (-r,t)
=
-F
T
T
(t) e (-r,t).
F(t). 6
That is,
(2. 25)
pas
31
When we make these two substitutions
=
d~~t)
e (t ,or) KX(T, T)C T (T)f(
T)dT
-
F(t)
dKX(t,
+[
in Eq. 2. 24, we obtain
J
T
t)
-dt
- K (t, t)F
T
(t)
~
t
SfeT (T, t)C T (T)f(T)dT,
(2. 26)
By applying
Eq. 2. 12a, we obtain
T
+[ F{t)Kx(t, t) + G{t)Q GT{t)]
f
S
9T{T, tIC T{ T)f{T)dT,
t
(2. 27)
After
rearranging
terms
and using Eq. 2.23,
we finally
have
T
S
f
dS(t)
dt
=
F(t) S(t)
+ G(t)Q
T
G (t)
e T (T, t)C T (T)-!(T)dT,
t
(2. 28)
32
At this point we have derived
for ~(t);
however,
we see that an integral
Let us simply define this integral
functional
a differential
operation
operation
equation
still remains.
as a second linear
of iJt):
Tf
:'lIt) -
SeT (T, t)C T( T).!.(T)dT,
To
;S
t
Tf"
;S
(2.29)
t
Therefore,
we have
(2. 30)
It is now a simple matter
equation which !l(t) satisfies.
Differentiating
.
dt = - CT (t)f(t)
dn (t)
to derive
Se(
a second differential
Eq. 2.29 gives us
Tf
- F
T
(t)
T
T ,
T
t) C ( T) f (T) d T ,
t
(2.31)
where we have again used the adjoint relationship
Eq. 2. 25.
After substituting
given by
Eq. 2. 27, we have
(2.32)
33
We now want to derive
which Eqs.
two sets of boundary
2. 30 and 2. 32 satisfy.
shall consider,
and t = T
In all the applications
that we
the function f( T) is bounded at the end points,
Consequently,
r
conditions
by setting
t = T in Eq. 2.29,
f
t
=
To
we
obtain
(2.33)
The second boundary
we set t
=
condition
T , the first
o
follows directly
term is zero,
from Eq. 2.23.
while the second term
If
may be
written
Tf
5
=K (T ,T)
x
0
0
T
gT(T,t)C(T)f(T)dT,
(2. 34)
-
o
or
g(T )
-0
= Kxoo
(T
,T ):O,{T)
0
=
P ,,(T ).
Q...l.O
It is easy to see that the two boundar-y conditions
(2.35)
given by Eqs.
2.33
and 2. 35 are independent.
We may now summarize
We have derived two differential
the results
of our derivation.
equations:
(2. 30)
(repeated)
34
d~(t)
=
<it
T
-C(t) f(t) - F (t)!l.(t),
(2.32)
(repeated)
In addition,
we have the boundary conditions
(2.35)
(repeated)
(2.33)
(repeated)
The relation
to the original integral
operation
is given by
T
f
=
S(t)
C(t)~(t)
=
S
K (t ,T)f( T) d-r , T
X
t
:S
:S
0
Tf.
(2.36)
To
Notice that the only property
was its boundedne ss at the endpoints
excludes
considering
equations
functions may appear there.)
n linearly
independent
of i(t) which we required
of the interval.
of the first kind where singularity
Equations
2. 23 and 2. 35 each imply
boundary conditions.
Since the differential
equations are linear,
any solution that satisfies
conditions is unique.
Finally,
be reversed
the derivation
in order to obtain the functional
that is, we can integrate
the differential
the integral
equation.
differential
equations must be identical
functional operation
Consequently,
of Eq. 2. 22.
of a solution ~(t) that satisfies
(This
the boundary
of these equations
can
defined by Eq. 2. 22;
equations
the solution
rather
~(t) to the
to the result
This implies
than differen-
of the
that the existence
the boundary conditions
is both
·
35
necessary
and sufficient for the existence
of the solution to the
operation defined by Eq. 2. 22.
In this chapter we have developed the concepts that we
shall need.
In addition,
shall utilize in several
material
equations.
we have presented
subsequent
chapters.
a derivation
We shall now use this
to develop a theory for the solution of Fredholm
We then shall apply this theory to several
optimal communications.
which we
integral
problems
in
II
rr
36
f
I
[
j
CHAPTER
III
HOMOGENEOUS FREDHOLM INTEGRAL EQUATIONS
Homogeneous Fredholm
important
role in theoretical
tool, their most important
Loeve expansions
difficult aspects
of cases,
integral
communications.
use arises
of this the ory is that,
suited for determining
solutions
except in a limite d number
to these equations.
method to find a solution
efficient
and is especially
by computational
shall then work a number of examples
Finally,
As a theoretical
One of the more
it is very difficult to find solutions
technique that is both analytically
play an
in the theory of Karhunen-
of a random processes.
We shall now apply a state variable
to illustrate
methods.
well
We
the technique.
we shall show how our re sults can be us ed to find the
Fredholm
A.
equations
determinant
The State Variable
Integral
function.
Solution to Homogeneous Fredholm
Equations
The homogeneous
Fredholm
integral
equation is usually
written
T
f
5 Ky<t,
To
-
T)
1<T)dT = ~1<T) ,
(3. 1)
37
where the kernel K (t, T) is the covariance
'l..
random process
l.(t) which is generated
in the previous
chapter,
is the associated
vector
eigenvalue.
of a vector
by the methods
.1(t) is an eigenfunction
eigenfunction-scalar
expansion
matrix
described
solution,
and
A
(We should note that we are using a
eigenvalue
expansion.
18,3 In this
we have
00
-y(t)
=
L
=
i
(3. la)
y.m.(t),
l.!.l
1
where the generalized
Fourier
coefficient
is a scalar
given by
T
f
Yi
S
=
T
(3. lb)
l:T( -r)li( -r)d-r
o
v
This type of expansion
is necessary
corre la ted. )
if the components
The solution to this equation
There are an at most countable
solutions
number
exist to Eq. 3. 1 and there
of y(t) are
"'"
is an eigenvalue
of values
of A
are no solutions
>
problem.
0 for which
for A <
o.
If
K (t, T) is positive definite, then the solutions to Eq. 3. 1 have
S.
positive eigenvalues and form a complete orthonormal
set. However,
if K (t, T) is only non -negative
:y..
~ (t) with zero eigenvalues,
o
definite,
then there
exists
i .e. , they are orthogonal
solutions
to the kernel
:11'1'5'0'1....777TT
iT wumpnnw."
5'
"ws .
.
F' .. , '!ilz;rrrrr
7'
r:
17
38
Tf
S
KX(t, T)~O(T)dT
= 0,
To;5 t
;5
(3. 2)
Tf·
To
We shall consider
finding only those solutions
with positive
eigen-
values.
,If we index the eigenvalues
function by the subscript
K (t, T)
Y
and their
i, the integral
$.(
T) dT = ~. $ .( T) ,
1
1 1
T
associated
eigen-
Eq. 3.1 becomes
:5 :5T
o t r
(3. 3)
When we employ Eq. 2.9 we may write Eq. 3.3 as
=
C(t)
Let us now put Eq.
results
m .(t)
If in Eq.
0
:5
t
:5
T.
f
(3. 4)
3.4 we set
= -f(t) ,
the re sult is that the integral
(3. 5)
enclosed by parenthe ses is the function
~(t) as defined in Eq. 2.22. Consequently,
-
lXl
3.4 into such a form that we can employ the
of Section II-C.
Xl
Xo. m. (t), T
let us define ~ .(t) to be
-1
39
Tf
~i(t)
S
=
Kx(t, T)CT(T).t (T)dT , To
oS
t
oS
Tf;
(3. 6)
To
so that Eq. 3.4 becomes
C(t)g. (t)
-1
If we assume
=
A. m . (t},
(3. 7)
1.I.1
that A.. is positive,
1
positive definite,
if K (t,T) is
y
which is guaranteed
we can solve for the eigenfunction
of g .(t).
in terms
-1
This gives us
1
= x::-
(3. 8)
C(t) ~i (t) ,
1
If we examine Eq. 3.6,. we see that the integral
which is defined is of the same form as the operation
Section Il-C.
Consequently,
operation
considered
we can reduce it to a set of differential
equations with a two point boundary condition.
in Eq. 3.6 wi th f'{t) in Eq. 2.20.
Let us identify
in Eqs.
(3. 9)
.I.1
2.20
2.32 become
and 2.32,
1i(t)
Then, if we substitute
f(t) = m .(t)
-
in
we find that the differential
Eqs.
2.30
and
40
+ G(t)Q
ddt g. (t) = F(t) g. (t)
-1
-1
=
g .(t)
CT(t) CIt)
~.
.:.1.1
0
:s t :s Tf,
_ F T(t):!ll' (t), To:S t
-1
1
From Eqs.
GT(t)l1. (t), T
2.33 and 2.35,
the boundary
conditions
(3.10)
:s T .
(3. 11)
f
are
(3. 13a)
g.(T)=Pl1.(T).
-1
0
The desired
(3. 13b)
0:.1...1 0
is re lated to the solution ~i (t) by Eq. 3. 8.
eigenfunction
or
(3. 8)
(repeated)
The net result
transformed
the homogeneous
of differential
equations
the state equations
generate
of Eqs.
3. 4 - 3. 13 is that we have
Fredholm
integral
whose coefficients
and covariance
the random process
matrices
and eigenfunctions,
are directly
This is consistent
to
that are used to
to determine
the
let us make two observations.
Notice that we have a set of 2n differential
solve.
related
y(t).
Before we use the above results
eigenvalues
equation into a set
with previous
equations
methods for treating
to
41
stationary
ferential
processes.
In these methods,
equation to solve,
tor polynomial
where 2n is the degree
dif-
of the denomina-
of the spectrum.
Equation 3. 8 implies
with positive
one has a 2n-order
that all of the solutions
to Eq. 3. 1
A. are contained in the range space defined by C(t).
We should note that if C(t) is not onto for a set of t with nonzero
measure,
there
then K
Y..
(t,T) is not positive
may be solutions
with
definite.
In this situation
A. e qual to zero which are not
contained in this range space.
We shall now specify a general
solving these differential
eigenvalues
equations,
and eigenfunctions.
find a transcendental
the eigenvalues,
solution technique
which in turn specifies
With this technique
equation that specifies
the eigenfunctions
W(t:x.)
=
I
I
I
I
I
Given
follow directly.
W(t:A.) as
G(t)Q GT(t)
--------------~--------------I
I
I
I
I
I
I
so that in vector form Eqs.
the
we shall first
the eigenvalues.
Let us define the (2n x 2n) matrix
F(t)
for
3. 10 and 3. 11 become
(3. 14)
42
d
(3.15)
dt
Furthermore,
let us define the transition
matrix
associated
with
W(t:X-)by
(3.16)
w(T ,T
o
:X-)= I.
(We have emphasized
including
the
X- dependence
X- as an argument.
In terms
solution
(3. 17)
0
to Eq.
After employing
of W(t:X-) and w(t, T :X-)by
o
)
of this transition
matrix,
the most general
3. 14 is
the boundary
condition
specified
by Eq.
3.13,
we
have
[-~~-I -]
!l.(T
1
(3.19)
)
0)
To:::t
-r-
r
43
r
Let us now partition
'l!(t, T :x,) into four
n by n matrices
o
I
=
'1"!(t,T :x,)
0
I
ns,
Eq.
s.n(t, T 0 :x.)
--------------~--------------I
'1"!; (t, T :x,)
Rewriting
'1f;
I
I
I
'l!~~t, To :x,)
such that
3. 19 in terms
'1"! (t , T :x,)
.!l.!l
0
I
0
I
of these
(3.20)
partitions,
we have
(3.21)
Taking the lower partition,
the boundary
condition
given by Eq.
3. 12
requires
'll.(T ) = ['1"! t: (Tf, T :x'.)P
f
!l~
0 1
0
-1
This implies
(T , T :x..)].!l.(T ) = 0
.!l!l f
0 1
1
0
one of two consequencies.
n .(T )
..:J.1 0
which implies
+ '1"!
Either
= -o.
a trivial
(3.22)
(3. 23)
zero solution;
or,
I
det[w
t: (T , T :x'.)P
+"ill"
(T , T :x'.)]
.!l~ f
0 1 0
.!l!l. f
0 1
= o.
(3. 24)
44
ITthe latter
satisfies
is true,
Eq. 3. 15 has a nontrivial
the requisite
equivalence
equation,
boundary
of the se differential
this nontrivial
eigenvalue.
conditions.
solution which
Because
equations
of the functional
and the original
solution to Eq. 3.15 implies
That is, the eigenvalues
integral
that A..is an
1
of Eq. 3.1 are simply those
values of A..that satisfy the transcendental
equation specified
1
by
Eq. 3. 24.
Now that we have found an expression
we can show how the eigenfunctions
For convenience,
follow.
define A(A.)as
A (A.)= 'If (J T , T :A.)P + iIr
!l2.. f 0
0!l.!l
Consequently,
f
teristic"
with this root.
1
1
polynomial
(We have used the adjective
"charac-
A(A. )with those of the integral
i
Therefore,
of A(A..)
vector
0
in order to avoid confusing the eigenvalue
the matrix
factor
A.., A( A..)has a vanishing
) is the characteristic
~l
associated
0
the characteristic
has a root equal to zero and n.(T
(3.25)
( T , T :A.).
When A. is equal to an eigenvalue,
determinant.
for the eigenvalues,
to determine
equation,
n .(T )
.:.l.l
0
properties
1
of
Eq. 3. 1. )
to a multiplicative
we need to solve the linear homogene ous equation
A(A..)n.(T )
1 ~l
0
= o.
(3. 26)
Given 1'I.(T ) we can find the eigenfunctions
.:.l.l
upper partition
0
of Eq. 3.21 and Eq. 3.8.
This gives us
by using the
45
=
C~t) ['1fl=l:(t,T
"i
~~
:x'.)P
0
1
0
+'1fe (t,T
~.!l
:x'.)]n.(T
0
1
.:.Ll
),
(3. 27)
0
To~t~Tr
which is the desired
result.
Before proceeding
order roots of Eq. 3.24.
with nonzero
slope,
we should comment
In general,
about multiple-
the function det A(X,)vanishes
that is, near an eigenvalue
x'.,
1
(3.28)
where
c 1 is nonzero.
In the case of multiple - order eigenvalues,
the function det (A(x')) vanishe s tangentially;
value
x'. of order
1
that is,
near an eigen-
1.
(3. 29)
This implies
n.(T
.:.Ll
0
that there will be 1. linearly
independent
vectors
) satisfying
A(x'.)n .(T )
l.:.Ll
0
= -0,
(3. 30)
i. e . A (x'.) has rank n - 1..
--
1
If we examine
that we need to determine
'1f(t:T0 :x,) •
our solution we see that the only function
is the transition
For the important
matrix
case of kernels
of the output of a constant parameter
system,
of W(t :x'),
that are covariance
we can find an
s
46
analytic expression
for this transition
matrix in terms
of the
matrix exponential,
wet,
W(X.)(t-T0)
=
T :x,)
o
e
(3.31)
This matrix exponential
transform
methods.
may be conveniently
computed by Laplace
We have
(3.32)
where ;J:.:,.1 is the inverse
Laplace
operator.
(In the inversion
contour must be taken to the right of all pole locations
If one desires
a numerical
is the case for systems
method of calculation
evaluation
the
of [Is-W(X.)] -1. )
of this matrix exponential,
as
of order greate r than one, a pos sible
is to truncate
the series
expansion
L
00
eW(},.)t =
j
=
(3. 33)
0
In the case of time varying systems,
analytic method for determining
this transition
there is no general
matrix.
However,
we can still use our technique by evaluating this transition
matrix
by numerical
equation
methods,
~
by integrating
the differential
defining it.
We can now summarize
our results
for homogeneous
47
Fredholm
roots
equations.
The eigenvalues
of the transcendental
=
detA(A.)
1
A. are specified
1
by the
equation
(3.24)'
(repeated)
0
where A(A) is given by Eq. 3. 25.
A(A)
=
W
t
!l.2.
(T , T :A)P
f
0
0
The eigenfunctions
1t (t) =
1\..
~~
1
n .(T )
~l
0
1 ~l
--
satisfies
A(A.)n .(T
0
!l.!l.
(Tf, T :A)
0
(3.25)
(repeated)
are given by Eq. 3. 27
C,(t) [w t t(t, T :~ P
.
where
+W
) =
0
0
+
the orthogonality
w~nt
-~
(T , T :~] n. (T )
f
0
-Ii
0
(3.27)
(repeated)
relationship
o.
(3. 26)
-
(repeated)
(The multiplicative
normality
factor
requirement.)
may be determined
by applying the
The matrices
Wt t (t , T :A), we. (t, T :A),\]( t: (T , T :A), and W (T , T :A)
f
2.~
0
~11
0
" 2.
0
!l.!l. f
0
are partitions
associated
of the matrix il!{t,To :A) which is the transition
with the matrix
matrix
48
F(t)
W(t:A)
=
I
I
I
------------T--------------I
(3. 14)
I
CT(t)C(t)
-
A
(repeated)
I
I
I
I
These equations specify the eigenvalue s and eigenfunctions
for a
kernel K (t , T) which is the covariance matrix for the random
Y..
process Y..(t). This random process is generated at the output of
a linear
system that has a state variable
description
of its
dynamics and a white noise excitation.
To conclude this section we point out some advantages
that this technique offers.
We can solve Eq. 3. 1 in the vector
1.
those techniques
which rely upon spectral
vector case could cause some difficulty.
defined this problem
random processes
case.
factorization
For
methods the
(In some respects
away by our method of characterizing
of interest.
the
It should be pointed out that
depending upon the problem this method of characterization
jus t as fundamental
2.
3.
as the covariance
4.
independently
accurate
are chosen to generate
equations that must be solved follow directly.
One does not have to substitute
into the original integral
transcendental
any functions back
equation in order to determine
equation that determines
of the others,
the
the eigenvalues.
We can solve for each eigenvalue
solutions.
may be
method. )
Once the state matrices
:l(t) , the differential
we have
which is significant
and eigenfunction
in actually
obtaining
49
5.
We can study a certain
6.
Finally,
class of time varying
kernels.
the most important
technique is very well suited to numerical
to determine
numerical
analytic calculation
advantage is that the
methods.
This allows one
solutions easily for problems
in which an
is either difficult or not feasible.
The major disadvantage
is that the class of kernels
that we may study is limited to those that fit into our state variable
model.
However,
interest
in communications
B.
Examples
we emphasize that most of the processes
do fit within our model.
of Eigenvalue and Eigenfunction
In this section we shall illustrate
in the previous section.
examples.
First,
Determination
the method developed
To do this we shall consider
we shall do three examples
do this principally
to illustrate
previous section.
The processes
by a first order system.
several
analytically.
second-order
in these examples
are generated
In general these are the only systems
an example of the numerical
system.
We
the use of the formulae in the
which thi s type of analysis can be done in a reasonable
We shall then present
of
for
amount of time.
analysis
of a
It is this type of problem for which the
It allows one to obtain numerical
technique is most useful.
solutions very quickly with a digital computer.
Example.!. - Eigenvalues
and Eigenfunctions
The covariance
at t
=
0 is
for the Wiener Process
matrix of a Wiener process
which starts
50
=
K (t,T)
y
A state-variable
=
O.:5t,T.
representation
dx(t) ---ar-
y(t)
2
l1 min(t,T),
u (t) ,
of a system
(state equation)
(observation
l-'-X(t)
I
(3. 34)
which generates
y(t) is
(3. 35a)
I
equation),
(3. 35b)
where
E [ u(t)u( T)]
=
o(t- T),
(3. 35c)
(3. 35d)
(The Wiener process
starts
with a known initial
For convenience
indicated
in Fig.
let us identify the state matrices
=
0
c
G
=
=
1
Po
=
=
=T
1
and To
constant
required
as
(3. 36a - e)
0
Let us find the solution
Tf
=
)
2.1
F
Q
state by definition.
O. First,
parameter)
substitutions
to Eq.
3. 1 when we choose
we need the matrix
specified
by Eq.
3. 14.
of Eq. 3. 36 inEq.
W(x') (the system
After performing
3. 14 we have
is
the
51
I
I
I
I
0
-------r -------
=
W(~)
1
2
-1:X-
To find the eigenvalues
I
I
I
I
I
0
and eigenfunctions
transition
matrix
of W(~).
transition
matrix
'l'(t, 0 :~) is
=
I
I
I
I
I
~
~
~
sin( -l:.-.t)
:n:.
--------------~--------------..J:... sin ( -E-t)
T
I
I
I
I
(3. 38)
cos(~t)
I
I
~
We now simply apply the results
previous
we need to find the
If we apply Eq. 3.32 we find that the
cos(L t)
'l'(t, 0 :~)
(3. 37)
~
as summarized
at the end of the
section.
First
we substitute
Eqs.
t = T into Eq. 3.25 to find A(~).
3. 36e and 3. 38 evaluated
In order
for an eigenvalue
at
to
exist Eq. 3. 24 implies
det(A( X-.))
= cos( --L T) =
1
The distinct
solutions
.JXi
o.
to Eq. 3.39 are given by
(3. 39)
52
\
l
=
i =
(2~~TT)'"
0,
determining
(3.40)
1, 2, ...
The eigenfunctions
Mter
J
2
follow by substituting
the appropriate
Eq. 3. 38 in Eq. 3.27.
normalization
factor,
we have
(3.41)
Example ~ - Eigenvalues
Stationary
and Eigenfunctions
Proce ss
Let us now consider
K (t,T)
y
=
for ~ One Pole,
the kernel
of Eq. 3. 1 to be
Se-klt-TI.
This is the covariance
(3. 42)
of the output of a first
pole at -k and Po chosen such that the process
order
system
with a
is stationary.
The
this process
are given by Eqs.
2. 16
Since the kernel is stationary
only the difference
state equations that generate
and 2. 17.
between the upper and lower limits
Consequently,
we again set T
Proceeding
substituting
Eqs.
f
=T
as before,
3.44 in 3. 14.
of the integral
and T
are important.
= o.
the matrix W(~) follows by
53
-k
W(A.)
2k8
I
_______ JI
=
_
(3. 43)
I
I
I
1
-r
The transition
matrix
for W(A.) is
cos(kbt)
¥
- sinBkbt)!
sin(kbt)
I
=
'lr(t,o:A.)
k
I
------------------~----------------I
,(3.44)
I
1 sin(kbt)
b
-A.k
'I
cos(kbt)
+
sinWbt)
where
(3. 45)
By substituting
Eq. 3.46
which determines
det(A(A..))
1
in 3.24
=
-J[1o.
1
=
,8k]
sin(kb. T) + cos(kb. T)
1\..
1
1
=
°
(3. 46)
1
to compute the roots by hand,
put in a more convenient
1
we obtain an equation
our eigenvalues
In order
tan(kb. T)
and 3.25,
form.
Eq. 3.46
can be
This form is
2b.
1
b~ - 1
1
(3. 47)
54
Solving Eq. 3.46 for
A., in terms
1
gives us the expression
A
for the eigenvalues,
of the b.,
1
Applying Eq. 3.27 gives us the eigenfunctions.
They are of the
form
,. (t)
1
where
=
'Y. [ cos(kb. t)
1
1
'Y. is a normalizing
1
Example ~ - Eigenvalues
Stationary
+ ~u.
stationary
and Eigenfunctions
$
t :s T,
(3. 49)
for ~ One Pole,
Non-
Proce ss
of a constant
parameter
e. g. the Wiener process.
of this can be genera ted from the previous
setting P 0 equal to the mean-square
assume that we know the state at t
Po
0
1
factor.
The output process
necessarily
sin(kb. t)],
1
Instead of
power of the stationary
=
0 exactly;
is not
A second example
example.
= o.
In this case the covariance
system
process,
that is,
(3.50)
function becomes
55
S e -kt( e k-r - e -kT)
for t >
T J
>
t .
=
K (t,T)
Y
for
By substituting
to zero,
Eq. 3.44 in Eqs.
where,
+
(3. 51)
3. 24 and 3. 25 and setting
the equation for the eigenvalues
det(A(~i)) = cos(kbi T)
T
sin(kb T)
i
b.
P
o
equal
follows directly:
(3. 52)
= 0,
1
as before,
(3. 45)
or equivalently,
tan(kb. T)
1
From Eqs.
-b ..
(3. 53)
1
3.27 and 3.44,
$.1(t) =
where
=
1
'Y. is again a normalizing
Example
1
i -Eigenvalues
have the form
O:::t:ST,
'Y. sin( kb. t)
1
the eigenfunctions
(3. 54)
factor.
for ~ Two Pole,
Stationary
In this example we want to consider
the kernel is the covariance
Process
the analysis
of the output of a second-order
when
system.
56
In contrast
with the previous
a particular
examples,
however,
we shall consider
system and analyze it by using numerical
Obtaining analytic results
for systems
than one is straightforward,
whose dimension
but extremely
tedious.
that the kernel K (t,T) of Eq. 3.1 is the covariance
Y..
Eq .. 2.19 and illustrated
generating
by Fig.
methods.
2.2a.
Let us assume
function given by
The state equations
y(t) are specified by Eq. 2. 18. In addition,
Tf = 2 and T =
is greater
for
let us set
o.
First,
we need the matrix
W(x').
By substituting
Eq. 2.18 into Eq. 3. 14 we obtain
W(x') =
0
1
0
0
-10
-2
0
160
1
X,
--
0
0
10
0
0
-1
(3. 55)
2
In order to determine
transition
matrix
exponentiate
of W(x') evaluated
the matrix W(X,). T.
det] A( X,)],
at T
=
we need to find the
2, 1. e. we need to
To do this,
we used a straight-
forward approach by applying Eq. 3.33 and taking the first
in a nested fashion.
numerical
(For further
procedures,
discussion
parameter
matrix,
we performed
indicated by Eq. 3. 25 to find det] A(x')].
X, and repeating
function versus
x'.
our
see .Refe r ence 32.
Once we find this transition
operations
regarding
30 terms
this procedure,
The resulting
the
By varying the
we can plot this
curve is indicated
in Fig.
3. 1.
57
In this figure the zero crossings
eigenvalues.
This type of behavior for the function det(A(x')) is
typical of those that we have observed.
eigenvalues,
corresponding
function is well-behaved
to those with significant
as discussed
governed by the "tail" of Sy(w).
we
the amplitude of
the eigenvalues
are approaching
their
by Capon. 19 This behavior
is
(In Fig. 3.1 we have used arrow-
heads to indicate the location of the eigenvalues
Capon 's formulae.
the
difficult to compute A(x')
In this region the eigenvalues
behavior,
however,
Eve ntually,
become so small that it becomes
energy,
(nonperiodically).As
eigenvalues,
the os cillati on rapidly increase s ,
accurately.
In the region of the larger
and oscillating
approach the less significant
asymptotic
are the desired
As the eigenvalues
as indicated by
be come small, the comparison
is quite good.)
Since this state-variable
finding the significant
with an asymptotic
conveniently.
technique is well suited for
eigenvalue s , one could combine this method
method in order to find all of the eigenvalues
In all cases that we have studied,
we could account
for at least 95 percent (often as much as 99 percent)
of the total
energy
(3. 56)
by our method.
non-stationary
The asymptotic
kernels,
method,
or a comparable
could be used to calculate
one for
the eigenvalues
,
s
,..::......
I
:
.
t~
"~
_,'
.
'-1-1--
"
",
':
..
1\
'!
-
'-1-- ......- -
-+--
---
--1--1:-' ',.. -..
--1----
58
~
---j---*---
.---t::E-+
L
L
--
,
N
:.... ~ :.=f.=::,--
~9=: -::~~;=r:=':
-
--
=t=:::
±f~
,--+ '
- f-
I
('f'\
TTl
._- -"._
,-
,
..•
_
,-
....
- _,._,-,-,._.
......
.._,-,
...
....+,,-+;,+,',-4
..,..,+
..•. -+.++-+-"-+-HH-f"-+·-++++-+-HH-+-++++·I-+-t-1~-t-.-+-++-1-I-+-t-+-II-I·- - - - - -1--1.- --I--I--f- --1- --1-',..........
HH-:-+~-+++·t-+-HH--'-f·-+++-t-·t-+-HI-t-+-+++-t--I
4-+-+-+--+--i-+-+-~1-i1--J- -t-+--+-t--+-+--+-i-I--J-f---!---'-'f-I-t-+--+-+--f-4-+-·j-tf-..1+-·++"+··+-++-+-l-'+++-+-HI-4-+--+++++-+-HI-l-+--++-I-++-.f-t-.I-l'--+--+-+-I-++4-+-hHI-t--+-I--++++-+-HH-+-I-·j-HI-t-++-1wi ; \
I·
II
.
"
,
i:"
"
59
corresponding
the residual
energy not accounted for.
We can summarize
the behavior
this example by plotting in Fig.
T, the length of the interval
satisfy the monotonicity
of the eigenvalues
3. 2 the first six eigenvalue s against
=
(T
Tf - To).
requirement,
We see that the curves
20, 3
a~.1(T)
(3.57)
aT
In addition,
reflecting
the number of significant
the increase
eigenvalues
increases
computer time required
used the Fortran
to find the eigenvalues
been reprogrammed
sophisticated
T
We
(the method
for the IBM 360 as a gene ral
As indicated earlier,
algorithms
the
for this kernel.
language on the IBM 7094 computer
purpose routine.)
with
in the "2WT product".
We shall conclude this example by discussing
has recently
for
we did not employ any
for computing A(A.). The time required
to
compute the data for Fig. 3.2 in order to find the first eight
eigenvalues
(99.8 percent
of the energy) is approximately
in addition,
the eigenfunctions
20 seconds;
may be found with very little
additional computer time.
C.
The Fredholm
Determinant
In the application
munic ati on theory problems,
is often used,
Function
of the Fredholm
the Fredholm
determinant
e. g. in the design of receivers
their performance
for detecting Gaussian
This function is defined as
theory to comfunction
and the calculation
signals in Gaussian
of
noise.
60
,
ae
-/,'-+-4-t-I-t-4
&+1\-
!IIO,
~I- -;:: -
,
~l
...
-+-+--
t-
-
-~.
.-
- f-
=
"f--I-~~
~~c-t~--1
__
-r-
I-~-I-+--
1
-',i~=!=t=
j~t.~
1-+--'--1-->-
f-.Ct::::j.::
-t--H-fH-t::· i-
v:_J
q=
t=l=e/±i:-f=t=I=l=t=r--t=
t=t:=l---l
~~,:=
I-- f-I-l/j-I-I-:J
+-
""'+-+-+-+-·t-+-I---I-+_I
/I
,
E J~l:
::EEt::::~f="=t-e~
r-H-f-=f-:::"
--f-
.., II'fI--+-f-l'-1-f-+-f--r--+--
i=t, =t= 1=1='
1- .
-:1=1=
I-t-
I-I-
1+1-
....-1-
1-1--1-
i ..,
ttt±tillttttttti:t:tt=J
7~,':~+-+-+-+-f-+--f--I+
1-1- t=:.::t=t-
.-
...=t=,-_.-
~
-+.Ir
II
~
....~!II'-+,4+++-I-+-:I-Hl...f'-+-++-l--l-+-Y.HH-+-+-I-++-I-+-+j~r+-l--l-+-+-HH-+-+-I~++.+-4-l-:~_1--!--l--+--+-~I-+,
,j"...~-tI-~~__+__l_-t;._II'#·+-I-+_-I-'-H_+_+4...J,iJ
'4--l-+-+-Hf-I__+__l_-+--I--l--I-+_+-H_+-+-f-4I'1~~+-I-+-'I-:--HH-+-~
--!---l-4--f---j-l-f-i--f-l--+-4-+-1
-1-f-,+_+_+I--1!-r-""'I~-+~-f-~_I_+i--+~-_+-I--+-+--+-+-+_+-1
~F~4-H-f.'i\+-J14--f.-I-~+++--t-~H4-+-L
1-+++--t-+-HIH-t-+..,J++++--t-+-H4-+,/~-++++f--+-hH"-I--f.-++++--t-+-HI-I--+-+-+-+-+-+-1-1-1hl:'6
I~
( "~
I~
,
I",
,
j
I
.'
,
if
I'
'I
:iJ
I
"
/
!I
"-
=_ j--r=:= =;:~ __-
=r==
I--f-1:-t--1-1-1= -- .
f- -
f-I-'I-~
-- ;_I--~
J-!-!-,
1--1= "
1-
I--~:=
- -- --f-l-l-
,
=1=+=
l-I-l-
,::-l-l-
.-l=
±
•
I
Fig.
II
U'f--ffiH-l1,f+
·4-+--l-+-+-.~HH-4-++-iI'I+-I-+-+-~1_+-'+-f--+-+-l-+-+-HH-+_++-I--l-+-I-+-+-H.......i_
'I
-
An
=
n
;,
J
i':l
" If
1/
-'#-'J+.+++-+ -'r-+--H-I--1-+.
II
1/
[
,-++-HH++-H4.,-I-+-H--l--I--I-H-+--I--I-H-+--I--I-H4-1--l-H'--+-J_
I/
2..0
3.2
-.J-H-+-+l
-t-;f-~I,IYIH':;'+++--"VI+'-+--H4-+1
\+J-I.-Hr-+-+++-H-+-++-+-H-+-++-~H-+-++-~Hr-+-+++-H"-4-'I-
F
t='=t=t-=-,
- l-
, -,
I
[1'
. ,
f-l-f-
3.0
f=
~
·,1-
I-
=
VB.
T
1, 2, ... , 6
61
00
D
;r
(z)
IT
i = 1
=
(3. 57)
( l+zA.)
1
where the A. 'Is are the eigenvalues
of Eq. 3. 1.
1
In this section,
we shall show how the theory developed in Section 3. 1 of this c hapte r
can be used to find a closed form expression
expression
can be determined
either
calculate d by the same numerical
previous
for this function.
analytically,
procedures
This
it can be readily
employed in the
section.
To do this we make some observations
function det [A(.!.)] where
re garding the
z may be a complex variable.
z
(1.)
It is
easy to argue that det[ A(.!.)] is analytic in the finite plane.
z
(2. ) Because
our test for the eigenvalues
det[A(.!.)]
has zeroes
converges
to E(Eq.
z
only at
3.56).
!..
~l
(3.)
is necessary
and sufficient,
The sum of the eigenvalues
Given these observations,
that det] A(.!.)] has the infinite product form
one can show
21
z
00
det[A(~)]
=
Ao
TI
i
=
(I-Aiz)
(3. 58)
1
where
A
o
=
lim A('!')
z
(3. 59)
z ~ 0
Comparing this with Eq. 3. 57, we find that the Fredholm
function is given by
determinant
62
1
=
1
(3. 60)
-X-det[A(--)],
o
z
L e. we can evaluate D(z) by using the same function that we
employed for determining
a negative
the eigenvalues
of Eq. 3.1 except we use
argument.
The only issue that remains
evaluating the constant A ~ In a direct
o
has done this. 22
For completeness,
is a convenient
method for
proof of Eq. 3. 60 Collins
we shall include this part
of his derivation.
If we let z ~ 0
in Eq, 3.14,
z
G(t) Q GT(t)
F(t)
lim
W(t:.!..) becomes
W(t:.!.) =
z ~ 0
(3. 61)
z
o
Let us examine the differential
defining \If(t, T :.!..)when we substitute
oz
equation,
E q. 3. 61.
Eq. 3. 16
We see that
-a\t lim \If (; (t, T :.!..)= -F T(t) lim
z ~ O!l.~
0 z
z ~ 0
(3. 62)
Since the initial condition for this homogeneous
zero matrix,
lim
z~O
its solution is the zero matrix,
'1[ l: (t
21~
Similar ly we see that
1
z
,T : -)
0
=
0
equation is the
1. e.
(3. 63)
63
d
lim
z -+ 0
dt
However,
~
T
!L!l
(t , T :.!.) = -F (t)lim
w
z .-+ O.!l2l
0 Z
(t, T :.!.)
0 z
its initial condition is the identity matrix.
from the adjoint relationship
.
hm
Z -+
W
o!1!l
(3. 64)
Therefore,
we have
1
(T , T : -)= 9
f
0 Z
where 9(Tf, To) is the transition
-1 T
(T , T )
f
0
matrix
(3. 65)
of F(t).
Consequently,
from Eq. 3.25 we obtain
(3. 66)
Since a transition
determinant
matrix
cannot change sign, i. e.,
This implies
that A
o
is positive,
is nons ingular , its
it must always be positive.
and we can take its logarithm.
This yields 23
T
f
J
d~ Ln{det[ 9(t , T a)] )dt
=
To
5
Tf
To
1
T:d 9-
T
d9(t, T ) T
(t , Tal<
dt
a ) ] dt
=
64
Tf
-5
(3. 67)
Tr[ F(t)] dt
To
Therefore,
we finally
have
Tf
-5
A
=
o
Tr[ F(t)] dt
T
e
(3. 68)
0
and
Tf
)
=
~z)
e
Tr[F(t)]dt
To
1
det] A( - -)]
For many of our problems,
convenient
function to use;
det[i(
- A.)]
=
o
from which~z)
approaches
i
+ 00 or
decreasing
(1
1
+ -r-)'
the behavior
Before
(3. 70)
1
-co.
function
of A.it has the same behavior
regarding
TT
=
use the function
A..
can be quickly determined.
one as A.~
monotonically
D~(z) is not the most
instead, we shall
00
d(A.) D.
(3. 69)
z
For positive
The function
values
of A. it is a
of A., while for negative
as we have earlier
d(A.)
values
discussed
of det] A(A.)]•
proceeding,
we shall pause to mention
an
65
asymptotic
expression
for d(A) when y(t) is a stationary
and the time interval,
assumptions
or 2WT product
is large.
process
Under these
it is easy to show that
5
ClO
In d(>") ~
T
S (w)
y
ln( 1 +
X-
(3.71)
)dw
-C1C
This formula
allows use to determine
the asymptotic
behavior
of
d(X-)in our calculations.
Let us briefly
in the previous
lytically
determine
section.
d(A) for three
As before,
of the examples
we shall do two examples
and the third by numerical
ana-
methods.
Example ~ - d(A) for ~ Wiener Process
Since F
=
0, we have A
d(A) = det A( - A) = cosh( ~
o
= 1.
From Eq. 3.38 we have,
T).
(3. 72)
..J}\
Example.§.
- d( A) for ~ One Pole,
Stationary
Process
Using Eq. 2. 17 in Eq. 3. 63 we find
"
Ao
=e
To determine
+kT
det] A( -A)l
(3. 73)
J
we use Eq. 3.44.
Let us define
(3.74)
66
After some routine algebra,
d(A)
= e -kT[
we obtain
2
( f3 ~ 1 ) sinh( k BT)
+
=
cosh(k f3T)]
(3. 75)
Example!... - d(~) for ~ Two Pole
J
Stationary
Process
Let us find d(~) for the same stationary
in Example
4.
As in that example
kernel
considered
we shall use a numerical
analysis .
. From Eq. 3.68 we find
A
o
=e
4
(3. 76)
Next, we use the same computational
the eigenvalues;
however,
method that we used to determine
we must use a negative argument.
Fig. 3. 3a we have plotted the resulting
interval
with
lengths of 1, 2 and 3.
those derived by assuming
we have plotted the results
d(A.)with solid lines for time
As a means of comparing
the time interval
indicated
In
by Eq.
our results
is large,
3.71 with dotted lines.
We can see that for T:=: 2, we are very close to the asymptotic
results
indicated
by a stationary
Finally,
munic ati ons , e.g.,
process,
Eq. 3. 71.
analysis.
the function log (d(~))is often used in comas it related
to the realizable
Fig. 3. 3b we have plotted log( d(x)) vs ,
analysis,
large time interval
the s olid lines,
fitter
error.
In
A. as found by our exact
and the approximate
Again we have the close comparison
results
found using
for T :=:2.
...
67
0
z
?
~
N
::, '",
t:
10
~E!"::fj
.... ;~n :;.lig'
!JI
r
0
Fig.
":a>
3.3a
:;
:I
3:
::;'
d(A)
d(A)
VB. ~
VB.
x
~
exact (solid lines)
asymptotic (dotted lines)
oi
X
:II
III
III
III
T= 1,2,3
-e
-l
X
:ll
1'1
1'1
N
~
I\l
Z
n
:r
n
-e
o
r-
1"1
VI
Ci
>
VI
1"1
VI
:I
0
;0
oi
s
>
-e
::~:
.:':
'.".
-=+.::i=t:
0
: c::t:y
,
;
:-i-
!,:,
T',
it
+
I
:
n
o
o
/!l
><
Cl
o
o
x
n
i:
o
3:
"0
>
;<
Z
;
z
!1
.
z
o
:D
~o
o
?
3:
>
VI
VI
>
n
x
c
~
0',
>
68
o
III
>
\/I
/l1
\II
::s:
o
::tl
...
~
>
0(
..
"
'."
o
;
:
:
I'
,
.i !
i :
1'1
n
o
o
r'l
><
III
o
o
:lI
(l
o
I
"'a
>
:z
-<
Fig.
3.3b·
In[d(A)] vs , ~ exact (solid lines)
; In(d(i\)] vs.?t. asymptotic (dotted rIines )
T = 1, 2, 3
'b·'
.
j
,.::',
0',
69
(This plot also indicates
the behavior
for large A., whereas
Fig. 3. 3a
did not. )
D.
Discussion
of Results for Homogeneous
Fredholm
Integral
Equations
In this chapter
we have formulated
approach for solving homogeneous
we indicated earlier,
particularly
problems
Fredholm
a state variable
integral
the technique has several
from a computational
viewpoint.
we have a very general
As
advantages
Consequently,
where we need to evaluate the eigenvalues
functions directly,
equations.
in
and/or
eigen-
method available
which
allows us to make just such an evaluation with a minimum of effort.
Quite often we do not need these solutions
we require
an expression
be able to determine
involving them.
such expressions
the ory that we have developed,
Fredholm
determinant
regarding
Gaussian noise,
for example,
the desirability
determinant
comments
it enters
in two ways.
the threshold
First,
it appears
as the
of the likelihood receiver.
performance
bounds for this problem. 12
of estimating
method of
signals in
involved in the calculation
In an estimation
More
of
theory context if we consider
the parameters
are
function.
it is intimately
problem
several
of having a convenient
importantly,
2.
as we did with the
In the problem of detecting Gaussian
bias in calculating
we should
in a closed form by using the
with our discussion,
evaluating this Fredholm
1.
In many cases,
but
function.
Proceeding
in order
directly,
of a Gaussian
process
the
70
(or the system identification
Rao bound can be determined
Finally,
I
we can show that the Cramer
using this function.
the solution method we have developed is
important
in itself.
requiring
a determinant
important.
problem),
As we shall see in Chapter V, the concept of
to vanish for the existe nee of a solution is
In this chapter,
homogene ous differential
we shall find a similar
equations
set of
and boundary conditions
specify the solution to the optimal signal design problem
additive colored noise problems.
the vanishing
for
The method of solving these
equations is exactly analogous to the eigenvalue
requires
that
of a determinant
problem in that it
for a solution to exist.
Now that we have studied the solution of homogeneous
Fredholm
equations,
let us examine the nonhomogene ous equation.
71
CHAPTER
IV
NONHOMOGENEOUS FREDHOLM INTEGRAL EQUATIONS
Nonhomogeneous
considerable
interest
Fredholm
integral
in communication
theory.
we shall use the results
advantages
In this chapter
As before,
there
are some
to the methods introduced.
One of the more important
equation is the problem
detection
are of
of Chapter II to deve lop a state variable
method of solving these equations.
significant
equations
of determining
applications
the optimal receiver
of a known signal in the presence
Since we shall study this problem
of this integral
for the
of additive colored noise.
in detai 1 both in this chapter
and
in the next, we shall first pause briefly to review the communication
model for this problem.
this problem
This is not meant,
however,
is the only place where we can apply our methods.
Other applications
include the solution of finite time Wiener-Hopf
equations in order to find optimal estimators,
Chapter VI, and the calculation
estimation
to imply that
as we shall do in
of the Cramer-Rao
bound for the
of signal parameters.
After this brief review we shall pre sent our derivation.
The results
formulae
come quickly since we have derived
in Chapter II.
vector differential
The result
equations
equations have appeared
the required
of our derivation
and boundary
in the literature
conditions.
in another
is a pair of
Since these
context,
namely
72
the linear
smoother,
we have several
solution methods
We shall devote a section to introducing
these methods
available.
14,15,16
and comments
upon their applicability.
Finally,
we shall cons ider three
the se will be worked analytically
procedures
A.
examples.
Two of
while we shall re sort to numerical
for the third.
Communication
Model for the Detection
of a Known Signal
in Colored Noise
Let us briefly
the pr oblem of detecting
noise.
discussion
the model in Fig.
which on hypothesis
0, it transmits
1 transmits
~o(t) over the time interval
purposes
let us assume
The channel adds a vector
to the signal.
independent
We assume
components.
~(t) that is generated
Chapter II.
The first
according
~(t) that has a covariance
We have a
4. 1.
~ 1(t), while on hypothesis
To ;s t es T r
For
that ~ 1(t) is ~(t) while ~(t)
colored Gaussian
component
to the methods
is
noise process
of two
is a random process
we discussed
is a white Gaussian
as specified
we have the following detection
r(t)
model for
that this colored noise consists
The second component
on H 1
the communication
a known ve ctor signal in additive colored
We have illustrated
transmitter
-~ (t).
introduce
by Eq. 2.7.
in
process
Consequently,
problem
= ~(t) + X(t) + ~(t),
T ;s t ;s T
o
f
)
(4. 1)
73
Under these assumptions
that the optimal receiver
This realization
integrate
can be realized
is a correlation
product) the received
it is straightforward
to show
as indicated in Fig. 4.1
receiver.
We multiply (dot
signal E.(t) with a function ~(t), and then
over the observation
interval,
i. e. , the sufficient
statistic
for the decision device is
(4. 2)
=
The correlating
integral
signal is the solution to a nonhomogeneous
Fredholm
equation.
This integral
equation has the form
Tf
S
T
where
KX(t,T)~(T)dT
+
R(t)~(t)
= s It},
To:=;
t :=; Tf;
(4. 3)
o
K (t , or), the kernel of the equation,
Y..
'of the random process
generated
according
is the covariance
;r(t) , which we assumed
to the methods discussed
is
in
Chapter II;
~(t) is a known vector function,
the transmitted
signal;
74
~
.
b.O
.~
75
R(t) is the covariance
is assumed
weighting matrix
of ~(t),
which
definite i
to be positive
and
gJt) is the desired
solution,
the correlating
One can also find the performance
system.
It is again straightforward
usually termed
signal.
measure
for this
to show that this measure,
2
d , is given by
(4. 4)
where s(t) and [(t) are defined above.
and false alarm probabilities
Error
probabilities,
can all be determined
detection
in terms
of this
measure.
As we metnioned earlier
integral
we shall study the nonhomogeneous
Eq. 4.3 in the context of this detection
emphasize,
however,
that the techniques
that they do not need to be considered
problem.
We
developed are general
in this particular
in
context.
Let us now proceed to develop our solution method.
B.
The State Variable
Approach to Nonhomogeneous
Fredholm
Integral Equations
Let us make two remarks
in contrast
regarding
the solution.
First,
to the homogeneous equation that has an at most
countable number of solutions,
this equation has a unique solution
76
when R(t) is positive
solution in terms
homogeneous
level
(J"
definite.
Second,
of the eigenvalues
equation;
we may find a series
and eigenfunctions
for example,
in each component channel,
when there
of the
is equal white noise
we have
(4.5.)
(the general
vector
case requires
a simple,
but straightforward
modification) .
Let us now proceed
Eq. 4. 3.
(The first
with the derivation.
part of the derivation
L. D. Collins.24) By using Eq. 2.9,
We rewrite
is due to a sugge sti on by
we have
Tf
~(t)
=
R -1(t) [ ~(t) - C(t)
i
5
Kx(t, T)C T(T)g(T)dT],
To
T
o
=5t
:ST
(4.,6)
If in Eq. 4.6 we set
g:,(t)
= .!(t) ,
we have the result
(4. T)
that the integral
function .§.(t) as defined in Eq. 2. 22.
to be
enclosed
in parantheses
Consequently
J
is the
we define ;(t)
r
77
T
K (t, -r)C (-r)g(-r)d-r,
x
( '4. 8)
-
so that Eq. 4.3: becomes
~(t)
=
R -1(t) [~,<t) - C(t)~(t)] ,
For the class
Chapter
(4. 9)
of K (t, or) that we are considering,
l
II that the functional
defined by Eq.
as the solution to the following diffe rential
we have shown in
4.8 may be represented
equa ttons:
(4. 10) .
( 4. 11)
plus a set of boundary
conditions.
If we substitute
Eq. 4.9 in
Eq. 4.! 1» we obtain
d!l(t)
<ff:
Consequently,
equation
=
T
-1
T
C (t) R (t)C(t) s.(t) - F (t)21(t)
we have shown that the nonhomogeneous
can be reduced
and associated
boundary
(4.12)
Fredholm
to the following set of differential
conditions:
equations
78
(4.10)
(repeated)
dll(t)
ctt
.T
=C
(t)R
-1
(t)C(t)~(t)
-F
T
(t).!l(t) -
C T (t)
cr
~(t)
To ~ t ~ T
f
(4.12)
(repeated)
(4.13)
(4.14)
The desired
solution
is given by Eq .: 4.3 to be
(4. 9)
(repeated)
Quite often it will be convenient
as one differential
equation
in the form
~(t)
d
dt
to write Eq.''if ..TO and Eq. 4.~12
0
~(t)
= W(t)
2l(t)
.-
!].(t)
...
- ... _---
(41:.. 15,>
CT(t)R -1 (t)~(t)
T o ~ t es T ,
f
where we define w(t) to be
I
F(t)
A
w(1)=
J G(t)Q GT (t)
I
--------r------CT(t)R-1(t)C(t);
-FT(t)
(4.16)
79
We note that the coefficient
similar
to that which appeared
major difference
-X, did.
matrix
in the homogeneous
is that positive definite matrix
We also note from our discussion
determinant
of Eq. 4. 16, W(t) is
equation.
The
R(t) appears
whe re
of the Fredholm
that one wants to find the transition
matrix
of W(t)
in order to compute ~(X,).
The solution indicated by Eq. 4.9, has an appealing
interpretation
simplicity,
when one calculates
let us again assume
d
R(t)
2
=
as given by Eq. 4.4...
c I, <r
a scalar.
For
Substituting
Eq. 4.9 into Eq. 4.4 gives us
dt .
The first term is simply the pure white noise performance,
second term represents
the degradation,
of colored noise in the observation.
d2 = d 2 _ d2
w
g
T
f
2
=
w
d
j
T
(4. I 7)
d 2.
w
The
2
d , caused by the presence
g
Therefore,
we have
(4. 18a)
f sT<t~s<tl)dt ~ ~
(4. 18b)
0
Tf
2
d =
g
5 (.I?<tlC;tli<tl)
T
0
,
dt.
(4. 18c)
80
In the next chapter
performance
we shall consider
the problem
of maximizing
the
by choosing s(t} when it is subject to energy and band
width constraints.
C.
Methods of Solving the Differential
Nonhomogeneous
Fredholm
Equations
for the
Equation
In the last section we derived a pair of vector differential
equations that implicitly
specified the solution of Eq. 4. 3
As we shall see in Chapter VI, these equations
smoother,
or the state variable
formulation
appear in the optimal
of the unrealizable
:::t:
filter.
In this section we shall exploit the methods that have been
deve loped in the literature
for solving the smoothing equations
order to solve the differential
equation.
equations for the nonhomogeneous
Since these methods have evolved from the smoothing
the ory literature,
References
the material
in this section draws heavily upon
14 and 15.
Let us outline our approach.
methods,
in
each of which has a particular
We shall develop three
application.
method is useful for obtaining analytic solutions.
intermediate
result in the development
useful because it introduces
The first
The second is
of the third method.
some important
The third method is applicable
concepts
It is
and results.
to finding numer ica l
solutions
since the first two methods have some undesireable
aspects.
We should also note that the methods build upon one
~:~Thisis consistent with the estimator-subtractor
optimal receiver for detecting s(t). 3
realization
of the
81
another.
Consequently,
understand
one needs to read the entire
the development
of the last method.
Before proceeding,
the previous
let us summarize
section that we need and introduce
we shall require.
=
d
from
some notation that
equations
o
~(t)
err
the results
We want to solve the differential
~(t)
section to
, To;:: t ;::T
W(t)
f
!}.(t)
J (4. 1 5)
repeated)
where we have defined
F(t)
W(t)
G(t)QG T(t)
=
( 4. 16»)
(repeated)
and we have imposed the boundary conditions
(4. 13)
(repeated)
=
(4. 14)
.Q..
(repeated)
Furthermore,
the transition
let us introduce
matrix
d
dt ~(t,
T)
the following notation.
We define
of W(t) to be W(t, T), 1. e. ,
= W(t)'1r(t,
T),
(4. 19a)
82
'M"(t,
In addition,
submatrices
T)
=
I.
(4. 19b)
let us partition
this transition
matrix
into four
n x n
in the form
=
'M"(t,T)
(4.20)
'M"
(t , T)
!l'!l
Method 1
The basic appr oach of the first
superposition
we generate
of a particular
and a homogeneous
a convenient particular
the forcing term dependence.
let
S
-p
solution.
First,
solution in order to incorporate
Then we add a homogeneous
so as to satisfy the boundary conditions.
solution,
method is to use the
solution
In order to find a particular
(t) and n, (t) be the solution to Eq. 4. 15 with the
f:J
initial conditions
(4. 21)
Since we have specified
a complete
set of initial
conditions
we can
uniquely solve the equation
s
-p
d
dt
s
(t)
----
nP (t)
-p
=
W(t)
(t)
------
n P(t)
- - - ..
.-..
......
--
,T
0
;st.
CT(t)R-l(t)~(t)
(4.22)
83
In order
to match the boundary conditions,
particular
solution
a homogeneous
n
~h
solutions
(4.23)
o
(T ) is to be chosen.
0
Notice that the sum of these two
.
satisfies
of the form
T )
~(t,
where
solution
let us add to this
the initial boundary
condition
(Eq. 4. r3)
of 21 (To)' Therefore,
we want to chose 21h(To) such
li
that we satisfy the final boundary condition (Eq. 4. 14). To do this,
independent
let us rewrite
Eq. 4-k 23 in the form
[-.~!~~'-~~~J
=
n (T 0))
.:.!.h
(4.24)
. ~. (t , T )
21
0
where we define the matrice s
(4.25)
e (t,T
!l
Conse quently,
)
0
A
=~
t(t,T)P
21.2..
we have
0
0
+~
(t,T).
21!l
0
(4.26)
84
r~{t)J
ln
(4. 27)
(t)
To::: t :::T r:
Applying the final boundary
n
.:.Lb.
(T
Substituting
)
0
= - ell.-1 (Tf,
condition,
T. ) n
0 .:.Lp
this in Eq. 4.27
S(t)
-
=
that
)
(4. 28)
f
gives us the final result
-1
S (t) - ~t(t,
-p
(T
requires
To)C1? (T , T )11 (T ))
!l. f 0 -p f
~
for this method,
To::: t ::: T
(4. 29.)
f
8
(The matrix
C1?!l.~t,To)
can be shown to be nonsingular
Let us briefly
determine
4.26.
~t(t,
~
0
solutions
conditions
functions
into Eqs.
the differential
this method have constant
of the signal,
we need to
4. 1H, 4.25
~(t).)
and
We then
S (t) and rL(t) by solving Eq. 4.!l.5 with
-p
specified
Two comments
problems
First,
,,0
(This can be done independent
the initial
the method.
T ) and ~ (t, T ) as defined by Eqs.
find the particular
these
summarize
for all t.)
4.27
-'1J
by Eq. 4 .....
21. Finally,
and 4.28
are in order.
we substitute
to find s(t) and !l(t).
For a large
class
of
equations that one needs to solve using
c oe fficients.
Conseq uently,
well suited for finding ~(t) and !l(t) analytically.
the method is
We shall illustrate
the us e of this method in the next secti on with two examples.
85
We also observe that the differential
to solve are unstable.
W has eigenvalues
e. g. if the system parameters
with the positive
can (and does) encounter
equations
real parts.
numerical
are constant
one
solving these
[To' Tfl is long.
of finding an effective
we need
Consequently,
difficulty in numerically
when the time interval
the problem
equations
This leads us to
procedure
for solving
one equation.
In order
more methods.
to solve this problem
The first
concepts and results.
we shall introduce
two
of these will develop some important
The second shall use these concepts to develop
the solution method which has the desired
numerical
properties.
Method 2
The most difficult aspect of solving Eq. 4.15 is
satisfying
essential
the two point,
or mixed,
boundary conditions.
aspect of the second method is to introduce
function from which we can determine
identically
zero,
conditions
at t
backwards
over the inte rval.
s(T f).
With these conditions
a third
Since !)JTf) is always
this allows us to specify a complete
= Tr
The
set of boundary
we then integrate
Eq. 4.15
From Method 1, let us define
(4. 31)
.)1
II~Thenotation ~(t/t)
is consistent
with Chapter VI.
86
We shall find a matrix
satisfies.
differential
equation
that :E(t/t)
We have
dg} (t T )
~
'
0
j'
dt
Substituting
d:E(t t) Cb (t, T ) + :E(t/t)
dt
.!l
0
_
-
from Eqs.
dCb (t,T
"
dt
'
g} (t , T ) + :E(t/t)(C T(t)R-1(t)C(t)~(:(t,
"
(4.32)
4. 25, 4. 26, and 4. 19 , vre find
(t , T ) + G(t)Q GT (t) g} (t, T ) =
F(t)CI?(:
~
0
.!l
0
d~~/t)
)
0
0
_
.!l
0
0
(4.33)
Multiplying
by <I?-l(t,T ) andusingEq.
.!l
0
d:E(t/t)
dt
=
F(t)~(t/t)
4.3:1 yields
+ ~(t/t) F T(t)
(4.34)
The initial
condition
:E(T /T
o
)
0
=
follows from Eq s , 4.7.5 and 4.26
p
(~L .35)
0
Consequently,
we have the expected
the realizable
filter
matrix
differential
Ricatti
covariance
result
matrix
equation
that ~(t/t)
»
T ) -F 1t)<I? (t , T
~
is identical
since it satisfies
to
the same
and has the same initial
condition.
87
S
Let us define a function
s
-r
(t) =
S
-p
-r
(t)
(t) - <P t(t, T )Cl?-1(t,T )n (t)
~
0 11
0 ..:.l.p
= -p
S (t)
- :E(t/t)n (t)
~p
(4. 36)
We note that
(4. 37)
Now we shall find a differential
equation
for
S
-r
(t).
Differentiating
Eq. 4.36 and using Eq. 4. 34, yields
d € '(t)
d~
= .F(t)ip(t) + G(t)QG
(F(tl~(t/tl
T
(t)~p(t)
-
+ ~(t/tlF T(tl + G(tlQG T(t) - ~(t/t)
- E(t/tl(C T(tlR -l(tl C(tl.ip(tl
C T(t) R-1(t)c(tl~Ml)r,Jtl
- F T(tl :!lp(t)
_,cT (t) R -1 (t)~Jt»)
After cancelling
The initial
s
-r
and combining terms
condition
(T
) = 0
0
-
(4.38)
by using Eq.
4.36
we have
follows from Eqs: 4...36, 4.31 and 4. 22.
(4.40)
88
From Eq. 4.38 we have the expected r'e sultthat
satisfies
filter
a differential
estimator
application
equation of the same form as realizable
equation.
We should note that in this particular
of solving the nonhomogeneous
input ~(t) is deterministic
some received
rather
integral
than a random process,
the
~'
4. 34 and 4., 38 are the key to the second
We simply integrate
of an integral
operation
T
~(t)
+
n(t)
o
S
T , apply
f
Eq. 4. 15 backwards
time using the complete set of boundary conditions
in terms
=
them forward in time to t
Eq. 4. 31 to find '§JTf}, and then integrate
Expressed
equation,
signal r( t) .
Equations
method.
~ (t)
-r
=
at t
in
T.
f
we have
o
f
dt' ,
'If(t, t ')
t
(4.41 )
To:$t:$T
r
Let us examine this method for a moment.
approach was to convert
initial,
a two point boundary problem
or ftnal ,> value problem.
developed for this conversion
structure,
numer-icafly
into an
Since the -S.r(t) function that we
is the output of a realizable
it has many desirable
known regarding
The basic
properties.
effective procedures
filter
In particular,
for calculating
S
-r
(t)
.
However,
we will still have difficulty integrating
a lot is
89
Eq. 4. 41 backwards
integrated
system
in time since Eq. 4. 16 is also unstable
backwards.
For example,
W has eigenvalues
growing exponentials
endpoint.
with constant
in the left half plane.
as we integrate
when
parameter
These produce
Eq. 4. 16 backwards
from the
With our third method we shall eliminate
this undesirable
In this method we shall derive a result
which allows us
feature.
Method 3
to uncouple the equations
observe
for ~(t) and !1.(t). After we do this we shall
that the resulting
some desirable
need to derive
features
differential
from a computational
Eqs.
-r
~l=(t,
.2.
the difference
First,
we
of ~(t) and ~ (t).
-
-r
= ~ 2.t: ( t, T 0!l){- e- 1 ( Tf, T 0 )!1.P(Tf) + <b-:!l 1(t , T 0)!lP(t)}. =
1
T )<b- (t,
0!l
T ) {ll (t) - Cl2 (t , T )Cf?-l(T , T )ll (T )}
·0 -p
Il
0
21 f
0 -p f
.
~(t/t) ,,(t)
Consequently,
viewpoint.
4. 39 and 4.36
~(t) - ~ (t )
-
for ~(t) and !l(t) have
one key result.
Let us consider
Substituting
equations
=
(4. 42)
we have the result
;(t) - ~r(t)
!1.(t)= ~-l(t/t)
=
~(t/t)!1.(t),
(4.43)
(~(t) - ~r(t)).
(4.44)
90
If we substitute
separate
the differential
substituting
this into Eq. 4. lOwe
equations
can obtain
for ~ (t) and !).(t). We find
for !l.(t)
d ~\t) = F(t) ~(t) + G(t)Q G T(t):E-1(t/t)(~(t)
- ~r(t))
=
(F(t) + 'G(t)Q GT (t) :E-1 (tit) )~(t) - G(t)Q GT (t) :E-1 (tit) ~r(t),
(4. 45)
Similarly,
substituting
for ~(t) yields
(4.46)
We now note that by finding g (t) we can solve either
Eq. 4. 45
or Eq. 4.46
(or final)
-r
for.~(t)
or .!l(t) respectively.
The initial
condition for Eq. 4.45 is given by Eq. 4.37 while for Eq. 4.46
is given by Eq. 4. 14.
Either
function can be obtained from the
other by using Eq. 4.43 .
. Now let us examine the stability aspects
equations
qualitative.
when integrated
First,
backwards.
of these
Our discussion
we need to examine the coefficient
is es sentially
matrices
of
91
Eq. 4.45 and Eq. 4.46.
In Eq. 4.46,
this matrix
is
-( F(t) - ~(t/t) CT(t) R-l(t) C(t))T, which is the ne gative trans pose of
the coefficient
stable,
matrix of the realizable
filter.
so is Eq. 4.46 when integrated
For constant parameter
If the interval
syste ms , we can see heuristically
backwards
over large
is "long",
d ~(t/t) ~ 0
(4.47)
dt
over most of the interval.
if it is
backwards.
that Eq. 4.45 is also stable when integrated
time intervals.
Consequently,
If we assume equality,
i,e.,
=
~(t/t)
~oo'
we have
(4.48)
or,
both matrices
have the same eigenvalues;
is also stable when integrated
Consequently,
backwards
one can numerically
and obtain stable solutions.
therefore,
Eq.4.45
over the interval.
solve either Eq. 4.45 or Eq. 4.46
However,
v
if~(t/t)
A
is also available
should point out that Eq. 4.45 re quires its inversion
we
whereas
Eq. 4.46 does not.
Summary of Methods
In this secti on we have developed in considerable
methods which exist for solving the differential
detail
equations we derived
92
for the nonhomogeneous Fredholm
summarize
integral
these methods by discussing
equation.
their applicability.
If we have a constant parameter
an analytical
solution,
Let us now
system and want to find
method 1 is probably most useful since the
differential
equations have constant coefficients
problems.
However, if we want to obtain a numerical
especially
over a long time interval,
since methods 1 and 2 can create
numerically.
solution,
method 3 is probably the best,
some difficulties
We really never use method 2.
introducing
for a large class of
it is that it was an intermediate
when integrated
The essential
result
reason
in our derivation
of method 3.
Let us now apply the results
of this section to analyze
some examples of solving nonhomogeneous
Fredholm
equations with
our method developed in Section B of this chapter.
D.
Examples
of Solutions to the Nonhomogeneous
Fredholm
Equation
In this section we shall consider three examples
illustrate
the results
two examples
of the last two sections.
analytically
use method 1 as discussed
program
while we shall use numerical
in the last section.
used in the numerical
differential
Again, we shall work
procedures
In those examples that we work analytically,
for the third.
to
we shall
The computer
example integrated
the second
equation of method 3.
We shall present
in colored noise.
compute the d
2
the examples
After determining
2
(and d ) performance
g
in the context of detection
the solution g(t) , we shall
measures
discussed
at the end
for
93
of the previous
section.
Finally,
To = 0 and T = T and assume
f
signal,
in all three
examples
we shall set
that the forcing function,
or transmitted
s (t) is a pulsed sine wave with unit ener gy over this interval,
i,e. ,
s (t)
=
I2'
V
2
(4.49)
T Tsin
We also assume that R(t),
or the white noise level,
is a scalar
constant
=
R(t)
Example.!.
rr > 0
(4.50)
- g,(t) for ~ Wiener Process
Let us consider
kernel is the covariance
describe
a system
the parameters
the problem
of a Wiener process.
for generating
I
I
d
dt
=
T}(t)
___ L __
lJ. I
2'
a:-' I
0
First,
3. 35 and 3. 36
we substitute
into Eq. 4.15 in order to find
the equation that we need to solve.
0 I 1
Equations
this process.
of these equations
g(t)
of finding g(t) when the
We obtain
g(t)
0
--- - - ----- - - -.-.T}(
t)
lJ..,f'f T
0:
,O:St:ST
. (nnt)
sm
T
(4.51)
94
From Eq. 4.13
are
=
0,
(4. 52)
T)(T)
=
O.
(4. 53)
to the last section,
; (t) and T) (t).
p
p
o.
method
These are the solutions
p
. (n 1Tt
T
(t)
to Eq .. 4. 15 with
solutions
S p (0) =
(4. 54)
(J'{
nrr
n rrt
w'e define
'{
T) (t)
-T cOS(--rr-)
P
for this problem,
=
(n 1T )
T
2
2
to be
2
+ ~ .
(4.55)
(J
Next, we need to find the transition
After some straightforward
matrix
cosh([ 1:-]
(J
associated
calculation
we find
:
:
1
2 -2
1
2
w(t, 0) =
)
Sin -'-
------~2J~
'{2
1, we find particular
Doing this we obtain
;
where,
conditions
;(0)
Referring
T) (0) =
p
and Eq. 4. 14 the boundary
"2
t)
:
[~](J
with Eq. 4.15.
2
sinh([~] (J t)
I
---------------------~-------------------I
1
2
[ ~]
(J
"2
2
sin h([ L]
(J
1
2
I
I
I
t)
:
I
I
2
cosh([ L] t)
(J"
• (4.56)
95
We need to add a homogeneous
equation of the form
1
Eq. 4. 23
1
-2
2 2
[ !:-]
sinh( [ .l:.-] t)
2
(J"'
(J"'
=
(4. 57)
1
2 2
cosh([L]
11",(t) "
From Eqs.
4.57
t)
rr
and 4.54
we find that we must choose llh(O) to be
Eq. 4.28
(4.58)
Consequently,
we have
1
122
.§.(tl =
-l7J ~
2
sin(n;t~_(n;)[
~<r]
-2
sinh( [ 1::..- ]
(_l)n ---(j--"'1r-
<r'l
t)
2 2
cosh([ 1:....] T)
rr
(4.59)
Th~ solution g(t) is found by substituting
Eq. 4.59
into Eq. 4.9
96
1
(~1Ty)
g( t) = ~
2
2
+ (n; [ ~]
Sin(n;t)
2
-1
)
(_I)n.
1
2
'2
sinh([ L]
c. u
t)
CT
, O~t~
1
T)
CT
After some straightforward
calculate
(4.60)
2
2
cosh([.J:.-]
2
T.
but tedious
•.
LitiS
manipulation
we can also
~
d as given by Eq. 4. 18c
g
1
1+
.
~2
(niT)
T
2 2
T)
2 tanh([J:....]
CT
•
1
'Y
2
2
[1:-]
CT
Example ~ - g(t) f or ~ One Pole Stationary
l-..e \
(4. 61)
T
Spe ctrum
Let us consider the kernel to be the covariance function
.
2'. t!~ ,~~.",
("'1
for a one pole stationary process as described by Eqs. 2. 16 and 2. 17.
I,
From these equations
and Eq. 4. ls,the
differential
equations
that we
need to solve are
O~t::ST,
(4. 62)
97
subject to the boundary
conditions
specified
by Eqs. 4. 13 and 4. 14.
We shall now use the first
solution method which we discussed.
After some manipulation,
we find
T'Jp(t) =
-JIIT
1 (
. (nrrt)
rr,,2
k sm 'I'
- ~;
f)
(1 -
nrr
+T
cos
(nrrt)
T
ek>.t + ~; (1 + ~) e -kAt)
J
( 4. 64)
where
1
A
28J2
[1 + krr
=
( 4. 65)
J
( 4. 66)
In this example,
'1r(t, 0)
the transition
matrix,
'1r(t,
0) that we need is
=
1
2kArr e
kAt
1
-kAt
- 2kAO"e
('4. 67)
98
Therefore,
ll(t)
according
=ft~.
_1_2 (k
.
rJy
.L
to Eqs.
4. 29 and 4.30
n
sin (.ntrt
- FE!..
T ) + FE!..
T cos i,\fl. .1TTt.)- ~(
2T 1- '!')ekX.t
X.
2T (1 + '!')e
X. -kAt)
",;,
,
'
_ (( A+ 1)2 ekAt _ (A-1 )
4X.
4X.
2
e -kAt )
nh (0),
0:S t :S T
(6.
6
9)
I
where
2
(A-Il
-kAT)-1
4A
e
( 4. 70)
Finally,
using
Eq. 4.9 we find the solution
1 {~(n1T.)
g(t) = -..j{£
T rJy2
T
Ll,
g(t)
2 + k 2J. srn (ntrt)
T -
S n1T ((A+1)2 kAT
(A-1)2
(T AT .
2X. e
2A
e
-1
-kAT
)
(continued)
99
x (H
1) (_l)nekAt -e -k},,(T-t)
+(}_-1) (-l)ne -kxt -e -k>..(T-t) )}
o
:5 t :5
T
( 4. 71)
Ii
One can continue and evaluate the performance
computing d
rather
2
and d: according
to Eq. 4.18; however,
complex and is not too illustrative.
.
choice of parameters.
when k = 1,
0"
the presence
an
in Fig. 4. 2
For the case n = 1, we see that
of the colored noise degrade s our performance
approximately
however,
are presented
= 1, S = 1, and T = 2.
is
n for a particular
g
The results
the result
Instead of presenting
we shall plot d 2 and d2 against
analytic formula,
by
50 percent
from that of the white noise.
our performance
is within 2 percent
For n
==
8,
of the performance
for the white noise only performance.
We can easily see that this is what we would intuitively
expect.
TIlT
=
For n
1. 57.
=
1, the bandwidth of the signal is approximately
Consequently,
frequencies
most of the signal energy appears
where the spectrum
noise is significant
compared
a bandwidth of 8TI/T
=
12.56.
of the colored
component of the
to white noise level.
Therefore,
in the
For n
=
8 we have
most of'1.ts energy appears
"
where the white noise ~jt) is the dominant component of channel
observation
noise.
It should be apparent
simple examples
performance)
covariance
that an analytic
from the complexity
of these two
solution for g(t) (and the
is indeed a very difficult task when the kernel is the
of a process
Consequently,
generated
it is desirable
by higher
order syste m.
to have an efficient numerical
method.
rp.ID
-J--l-+-l-j-+-+--+--I-+-I-I-~-l-+-H-I--I-+-+-H-I-++-+-H-+-++-+-H-+++-J..c..;H-++-+-+-H-+-J--+1-++-+-+-~f-t=,=I=I=
-I---+-+-I--+--l--+---J-I---l-+---IH--I\~-+--t-l-l---+--+-+--I-+--l-+---IH--l------+-I-+--l---+--+--I-t--I-+-l~-+-+-I--t---l----+-+--I-t--I-+---I-I'- ,-
-':.:.i=-t=
-I--+--+-+-l--l--J---i-+--+-I
I,
----t--'
==l=r= -
'-,-1---;--
__:::"-!.:::..:...:.....I-I- --
-1---1-+-1-1
-+---+-+-t-t---I-+----!-I---l-+---I~-I
11-I- -1---1-+-+---11--1---+--+-+-+---1
"--+--+-+-+-I-+-+--l-+-+--l..-j-I---I--l--+++--HH--!--I-+-+-JH--I--++--I--I--"--!-+-J--+-If-~-I-I-
,-
=~D-
1-1-
1-1-'---
1-1--1-1--
,---
---1-.---, li-
----- ..-
-f---
-1--1---1-+-
-~:f---'--i---1--
1--'---
--
-----1---
----
=~-~--
-f-f--I-
-
,I---
1-
__
t=_~f-,
--I-
I
I
I
Fig. 4. 2
_
1.:...' __
01
1
Covariange Function s(t) = stntnnt)
liiiOiId
....
2_an_d_d~2~v...s_._n~f_o.r_a~F~ir...
...
s...t__
or_d_er_ ...
:::::::::::::::-,-H-,-H-,-
..'',-
~
1\~
..
Ht
H-,-H...l.-~~--:+-~L:~:-I=:~...l..[-...l..81--..:..I_-~=~I--ll.:..'-.:..-~I'1
8
101
Example ~ - ~
for ~ Two Pole Stationary
For the second-example,
approach to the analysis
we chooseK
0-
=
y
=
1 and T
Spectrum
we shall consider
a numerical
In partic
of a sec ond - orde r system.
ular ,
given by Eqs. 2.18 and 2.-19
-To find g(t) we determined ;(t) by using method 3
(t,T) tobe
2.
the covariance
in the last section.
In Figs.
the corresponding
4. 3 and 4. 4 we have drawn the signal s(t) and
solution g(t) for n
fre quency (n = 2) case,
significantly
filter,
we find that functionally
(n
=
>
=
8.
For the low-
s(t) and g(t) differ
-Here, we are approaching
solution.
while for the
8) case we find that s(t) and g(t) are nearly
We have summarized
in Fig. 4.5 by plotting the
n
2 and n
only near the end points of the inte rval,
high-frequency
identical.
=
d
2
the white noise,
the results
vs n behavior.
8 we are within 4 percent
or matched-
for this example
We see that for
of the white noise only performance.
Again we can see the effect of the colored noise upon the
detection pe rformance
.
For n
_at the peak of the spectrum
Consequently,
=
2, most of the signal energy appears
of the colored component
the performance
the signal energy is centered
is degraded
of the noise.
the most.
For n
=
8,
around f = 12.56 where the white noise
is dominant.
The computer
g(t) and to calculate
-IBM 7094 computer
time that we re quired to find a solution
its performance
time,
using the Fortran
The method has recently
purpose routine
is approximate ly 5-10 sec of
for the IBM 360.
language.
been rewritten
as a general
102
-.8
Fig. 4. 3
( )f
and s t
a Second Order
or.
(t) = stntrrt)
Covariance Function s
g(t)
103
Fig.
4. 4
g(t) and stt) for a Second Order
Covariance Function stt) = (4 t)
...
104
--
:
...
.-.
,-
: ..
-
.-
.
.1
-,--i--I---'--+-'
··,--I--'--I'T-[·--
1-':
1-
:
-
i
i __
-'-i---I-
- .. -- .. ----'--'-.,
I-,-f--,--
._I.:
I-i-
i-:
'_L.:.,_
-,--;
.,"-,_
i
T
,~
.--
,-
•.
:.
.. -;--i------:-+-1_1.::,--
,'-
...
r.,
, __
._
1-+-·,---
r":"
,---,
..
,
'-1-1--
i-+--
I·-i-I··-,·_,_
.. ': ,.._.-
'-li--,.-
1--
,-
I-:--'-r-r-I-I-
!.
'-I-'-T1
;.,--
''''1-
I
I··
I
;
,----I·
--
i---
·,----·1-'-1--'-'-,-,-·,-"-.
I-I---i-;-I-I--I-I--II
I
1--
';
,-••
;
!
j
--
1-..
Ii
:iI il
,·1_' __
1-
--"-1-'-'
:
!
-
Ii
__
'
'I-
-
i·I_'
__I_,_I'
__
:::=:~_::-~=::
..'.'
. _.
.J
!_.,_
,-
-- -,-,-·+-1
.
d2 and d2 VS. n for a Second Order
Covariange Function stt) = sintnrrt)
t
-
-j--
---'--1-,--
,--
__:_
i-i-
' __ , __ '_
,t--t--i---j--I-+--':--l-I-:~-::':I~::
i·-i--I-- i--I•
'-1--1--
,--
------
: I--i-
'1--'-"
..
•..
-
... -,---
l-+-+-'~--I--+-i
i---'
i
-I-
,I.
t._,.
Ii,
I
;--i-
..
'--rr
-IJ-!J __
LLI!_
.LLLL-I-J-f-l--+--i.
t
i
I
l--"
,
1--.-
1
!
5
,
C)
:
L,
--'.,
,.
1
--1-
.::=: JJII II 1=t~lt+--1
'
Itll
.01
i
.•...
--"---
-j---
'l-
...
i_l_
,-.
=: ~.'
3
1._
,
r
1
,_
1_'.
j
Fig. 4. 5
: ••"
-"--1-1-,--1-1--1
1-+··
1_' __
1---'--1--,-1-1-+--;-'--'-1"'-'-:-+__I
-"
.....-
.
~
I
.,.
.. ~·_r_--:--+_+_+_:---._+_;__i_-+-_+_+-H!_+··'_;_--i--t_.,._:__:_;_t--
-I-i·-I-.
-:
~.;
:: '.
__' __
,-i--i-----
,-
-- ,...-1--1--
1-
I
1-1-'--
I
".:
.
1_
-,--
'
......
I-i:-.
: __
.-----.
,I,
_! .. :
,Ii
___._.._:_
'--i--I'-'
-
..
i--,---'
,__,=,.;_
1--.1-1.----
,·-1----
,-,--i--
I
·.I=r~L~::,-, :.,:
'r-'
-i-
i--,-'---
--
:;-;.'--"---,
.._i:----ii-+--I---+-I-+'--:--I-+--i4---~-I~-j_+-i-~-"-;;I-+--,'-:-·---_T.,.+-".. ·--:-----;-:------+.....:...·,--+---:-·-'--+I-~-_:_!-_+·-...::4-1_;_-i-_i_'-~-'_+---+_i_-+-_+_+-t-;'_+
...
--11
..-
._-
'---'i--'---I----·-
,. -:-:':'::::::1=
.. --,----,------
1--
----
1-"+-1-1-:-·:-'1-
j-i-,-
I-T::r::::'-'T:'::::
'':::l.=::::::;==-'---'!-,--I-
--.-
j--
I-F=f:::.j=..--.
·'--1
·-i-t-~i··_·----i---
----_
i
,-
i-
-- '--,.-
-
---
...--i-::;
-r-:-::-~ ...t-'·'-j--'--I-
-,
'I--:~:'-
.--
•
.!-
-.
I-j
".~:
7
'i ,-
105
E.
Discussion
of Results for Non-Homogeneous
Equations
.
In this chapter we have formulated
Let us briefly
compare
Integral
a state variable
approach for the solution of non-homogeneous
equations.
Fredholm
Fredholm
integral
our approach to some of the
exi sting one s .
The approach of reducing
differential
equation certainly
an integral
25,26,27,11,3
is now new.
In one form or another
it is undoubtedly the most common procedure
to other differential
advantages
equation methods,
(many of which are shared
homogeneous
equation to a
used.
In comparison
our approach has several
with our solution method for the
equation).
1.
We can solve Eq. 4.3
2.
The differential
follow directly
when s(t) is a vector function.
equations that must be solved
once the state matrices
that describe
the generation
of the kerne 1 are chosen.
3.
We do not have to substitute
the original integral
any functions back into
equation in order to find a set of linear
equations that must be solved.
4.
We can study a wide class
of time varying
systems.
5.
The technique is well suited for finding numerical
solutions.
There are two major disadvantages.
1.
limite d.
important
The class of kerne Is that may be considered
However,
class
the technique is applicab le to a large and
of kernels
that appear in communications.
is
ilIt!llH--------------------------···--··,,---·-"-"-~--~----
--------
ti,"i'}.
106
2.
We cannot handle integral
w h en t he whiIte component
kin d , ~
For these equations
endpoints.
singularity
We precluded
that we have observed
these singularity
0f
functions
equations
.
.. 1dentic
. a "IIy zero. 3, 27
t h e norse
1S
appear at the interval
these in our derivation.
the limiting behavior
is to find the inverse
The inverse
T
In terms
kernel
to the differential
of the integral
kernel Q(T, u) is defined so that it satisfies
"
f
{R(t)
s (t-T) + K;l(t,
T)} Q( T, U)dT = I Ii( t-u)
o
( 4. 72)
of the inverse
T
g{t) =
kernel the solution g(t} is found to be
f
S
T
Q(t, u)~.<u)du,
approximate
o
the integral
[K] [Q]
=
(4.73)
To < t < Tf
One common nume r-ical method that uses this approach
g
a
equation
T
S
approaching
functions when the white noise is small.
equation approaches
second integral
We should note
of our solution
A second method that is in contrast
equation.
of the first
[Q]~
=I
operations
is to
4. 72 and 4. 73 by matrices
(4.74)
(4.75)
�---------------107
Let us briefly
compare
the computation
If we assume
using this approach to that of our approach.
sample the interval
dimensional.
at NI points)Equations
4.73
and 4.74
we find Q by s pe cifying the coefficients
NI.
the computations
The computation
to NI squared,
equations
that we
are NI
One can show that the number of computations
to find [Q] as given by 4 ..73 goes as (NI) cubed.
equations,
required
whereas
to implement
Eq. 4.3
required
proportional
for large NI, which is required
that
incr-e as es only linearly
the computations
again is linearly
If we assume
of our differential
we require
required
required
to NI.
is proportional
to solve our
The conclusion
for high accuracy,
with
is that
the differential
equation approach is superior.
Before we leave the topic of the inverse
point out an important
We can consider
specifies
kernel,
concept that we shall use in a later
that the non-homogeneous
a linear operation.
integral
In an explicit integral
we shall
chapter.
equation
representation,
-n.
this linear
operation
is given by Eq. 4. 72.
however ~ to specify g,(t) implicitly
It is completely
equivalent,
as the solution to our differential
equations.
-In the two following chapters
of this chapter.
In the next we shall apply them to the problem
designing optimal signals for detection
channe Is.
we shall apply the results
of
in additive colored noise
Our basic approach is to re gard the differential
equations we have developed as a dynamic system
final boundary conditions
When the problem
form we can apply the Maximal Principle
optimization.
13.6
with initial and
i~ expressed
of Pontryagin
in this
for the
------------~
..
-~-~--~_.
------------
108
In the subsequent
to solving Wiener-Hopf
We shall then proceed
and filtering
chapter
we shall present
equations by using the results
to deve lop a unified theory
with delay.
a new approach
of this chapter.
of linear
smoothing
109
CHAPTER
V
OPTIMAL SIGNALS DESIGN FOR ADDITIVE
COLORED NOISE CHANNELS VIA STATE VARIABLES
In the problem
of detecting
of additive colored noise,
formance
a known signal in the presence
the signal waveform
of the receiver.
affects the per-
For a given energy level,
signals result
in lower probabilities
Consequently,
by choosing the signal waveform
manner,
we may maximize
of error
the performance
If one does this optimization,
result
tend to have large bandwidths.
is stationary,
it places
optimization
this chapter.
For example,
noise spectrum.
This is the problem
width (defined later)
the detection
that
when the noise
band that is
Often the available
bandthi s
upon the bandwidth of the
we want to find the signal
performance
when the band-
is constrained.
Statement
We introduced
in the presence
the signals
which we want to consider
For a given energy level,
Problem
of the system.
the ref ore , in this case one must perform
waveform that optimizes
A.
in some optimal
however,
with some form of constraint
signal waveform.,
than do others.
the signal energy in a frequency
on the tail of the colored
width is restricted;
certain
the problem
of additive colored
of detecting
noise in Chapter
a known signal
IV.
Let us
in
110
briefly
review some of its aspects
we want to consider.
is illustrated
Depending upon the hypothesis,
sig.nal, s(t) or -s(t) is transmitted
In general,
is a zero mean Gaussian
w(t) plus an independent
over an additive scalar
we shall assume
process
that
colored
noise
of a white component
component y(t) with finite power.
the received
a known
that this colored
that consists
easy to show that the optimal receiver
ratio by correlating
problem
The model of the system that we want to study
in Fig. 4. 1.
noise channel.
for the optimization
computes
It is
the log-likelihood
signal with a second known function
g(t) .
This correlating
Fredholm
integral
equation,
signal may be determined
Eq.
by solving a
5. 1, of the second kind,
(5. 1)
where
s(t)
is the transmitted
g(t)
is the optimal correlating
No/2
is the power per unit bandwidth level of the
signa 1;
white noise (identified
signal;
as
CT
in the previous
chapter) ;
and
K (t , T)
Y
is the covariance
function of the colored
(finite power) component
is the observation
of the additive noise.
interval.
III
(We shall study only scalar
channels here.
However,
all the results
can be extended to vector channels with little or no difficulty.)
For this problem the appropriate
performance
measure
is given by
T
f
S
(5. 2)
stt) g(t) dt ,
To
Under the Gaussian
assumption
probabili ties of error,
assumption,
that we made,
false alarm
we can still interpret
we can determine
and detection.
If we relax this
2
d as the receiver
output
signal to noise ratio.
The choice of the signal s(t) is not completely
impose two constraints
upon it.
The first
free.
We
is an energy constraint,
or
2
(5.3)
s (-r)d-r=E.
Since Eq. 5.1 is linear,
it is easy to see that
dependent upon E.
Secondly,
multitude of ways.
Initially,
satisfy the constraint
2
d
is linearly
one can define bandwidth in a
we shall require
that our signal
112
00
SJ
=
2dw
ISs(wli
(5.4)
-00
I. e.,
we have used a mean square
the signal.
constraint
(We do not need a normalization
upon the derivative
factor because
of
of
Eq. 5. 3. )
We shall set up and solve the optimization
the constraint
of Eq. 5.4.
we can extend our results
After we have done this,
5.1 through
the performance
5.4.
measure
formulate
associated
approach
with Eq. 5.1.
important
find better
both Eqs.
issues.
signals.
the energy
and bandwidth
which one may want to consider
of the homogeneous
signal is the eigenfunction
satisfies
of
5.3 and 5.4.
the opti miz.ati on problem
and eigenvalues
in terms
d2 given by Eq. 5.2 where g(t) is related
imposed by Eqs.
The first
problem
~,
We want to find a signal s(t) that maximizes
to s(t) by Eq. 5. 1, and yet satisfies
constraints
using
we shall show how
to other types of constraints,
We can now state the optimization
Eqs.
problem
in terms
of the eigenfunctions
equation which may be
If one does this,
with the smallest
5.3 and 5.4'.
he finds that the optimal
eigenvalue
This approach
Unless Eq. 5.4 is satisfied
In addition,
is to
it neglects
neglects
which
two
with equality,
discontinuity
we can
effects
caused by turning the signal on and off at To and T respectively.
f
113
A second approach as proposed by Van Trees
of variations
while introducing
the constraints
Lagrange
.28The resulting
integral
converted to a set of differential
derived in Chapter IV.
multipliers
to incorporate
equation can then be
equations by using results
For the particular
the signal that we have initially
is to apply the calculus
form of constraints
used, 1. e. , Eqs.5.
the approach that we shall use is more ge neral.
However,
Many of the
that we shall develop can be extended to constraints
cannot be readily handled with the classical
up on
3 and 5.4, this is
undoubtedly the most direct method of the minimization.
results
we
calculus
which
of variations.
We assume that the colored component of the noise y(t) is a random
process
that is generated
this assumption
as we described
we recognize
that we can represent
integral Eq. 5.1 as a set of differential
the previous
chapter.
in Chapter II.
equations
Next we consider
of Pontryagin
consequently,
can be used to perform
the techniques
B.
two specific
in
with boundary
the Minimum Principle
'By using
solution to the problem,
examples
in order to illustrate
involved.
The Application
of the Minimum Principle
In this section
for the problem.
minimum principle
existence
as discussed
3
the optimization:
this approach we shall first find a general
then we shall consider
the linear
that this set of differential
equations can be viewed as a dynamic system
conditions and an input s(t);
Making
we shall develop a state variable
Using this formulation
to find the necessary
of an optimal signal.
formulation
we shall apply the
conditions
for the
We shall then exploit these conditions
114
to find an algorithm
for determining
Since there are several
the course
of our derivation,
subsections
important
issues
that arise
we shall divide this section into
1.
The State Variable
2.
The Minimum Principle
3.
The Reduction of the Order of the Equations
4.
The Transcendental
5.
The State Variable
In order
Formulation
Conditions
by 2n
and AB and
Solutions
Formulation
of the Proble m
to apply the Minimwn Principle
cost functionals,
a set of differential
and the Necessary
Equation for ~
the problem in terms
conditions,
of the Problem
of the Optimal Signal Candidates
The Asymptotic
formulate
in
as listed be low:
the Selection
1.
the optimal signal.
equations
of differential
and a control.
we need to
equations,
First,
we need to find
and boundary conditions
g(t), the solution of the Fredholm
integral
boundary
which relate
equation expressed
by
the solution of Eq. 5. 1 to the signal s(t).
We can do this by using the results
previous
chapter.
Reviewing the se results
derived in the
we have shown that
g(t), the solution to Eq. 5. 1 is given by Eq. 4. 9
g(t) =
The vector
Eqs.
N:
(5. 5)
(s(t) - CIt) ~.!t)),
function,
4.10 and 4.12
~(t), satisfies
the differential
equations,
115
(5. 6)
d.:l(t)
CT(t) N2 C(t)~(t)
--ar- =
- FT(t).:l(t) - CT(t) N
o
2
s(t)
(5.7)
0
To:St:STf
The boundary
conditions
which specify the solution
uniquely are
E qs. 4. 13 and 4. 14.
;(T )
-
0
=
(5.8)
P n(T )
o~
0
(5.9)
Consequently,
we have the desired
s(t) by solving
two vector
differential
result
that we can relate
equations
two point boundary value condition imposed
where we have a
upon them.
Let us now develop the cost functional
The performance
substitute
Eq.
measure
of our system
for the problem.
is given by Eq.
5.5 in Eq. 5. 2 and use Eq.
g(t) to
5.3,
5.2.
If we
we find that
2
N (S(T) - C(T) ;( T))dT
S(T)
o
T
=
2E
2
No - No
f
S
T
(5.10)
o
116
The first term in Eq. 5.10 is the performance
white noise present.
performance
noise.
when there
The second term represents
caused by the presence
As in Chapter IV,
of the colored
is just
the degradation
component
in
of the
let us define d2 to be
g
Tf
S
T
S(T)C(T)~(T)dT
o
and the function L(g(t),
L(~(t), s(t))
(5.11)
1
=
s(t)) to be
2
N s(t)C(t)~(t),
(5. 12a)
o
(5. 12b) .
Since the energy
E and the white noise level N /2 are constants,
o
is obvious that we can maximize
The state variable
variables
be related
ds(t)
dt
rather
Instead,
v(t) , to be the derivative
=
by ~inimizing
both the signal
cannot use s(t) as the control.
v(t)
2
formulation
by derivative
Since we are constraining
function,
d
requires
it
d 2.
g
that the system
than integral
operations.
and its derivative,
we
let us define the control
of the signal.
(5. 13)
117
Furthermore,
we require
(5. 14)
Eq. 5.14 is a logical requirement.
derivative
of the signal,
jump discontinuities
Since we are constraining
it is reasonable
to require
(implying singularities
the
that there be no
in v(t)) at the endpoints
of the interval.
We now have all the state equations
conditions
equations
that describe
the dynamics
are given by Eqs.
5.6,
5.7,
and boundary
of the system.
The state
and 5. 13.
(5. 6)
(repeated)
d~~t)
=
CT(t) N
2
2
C(t)~(t) - FT(t)!l(t) - CT(t)N
o
0
stt),
(5.7)
(repeated)
TostsTf
ds (t)
dt
=
+
We have (2n
given by Eqs.
£(T )
-
0
v( t) ,
(5. 13)
(repeated)
1) individual
5.8,
5.9,
= P oJ.11(T 0 )
equations.
The boundary
conditions
are
and 5.14.
(5. 8)
(repeated)
(5. 9)
(repeated)
(5. 14)
(repeated)
118
Notice that there
Consequently,
are (2n + 2) individual
boundary
conditions.
these conditions cannot be satisfied
for an arbitrary
v(t) .
In order
to introduce
the energy
we need to augment the state equations
and bandwidth constraints,
artificially
by adding the two
equations
dxE(t)
(5. 15)
dt
(5. 16)
(We have introduced
the factor
boundary
are
conditions
of 1/2 for a later
convenience.)
The
(5. 17)
(5. 18)
(5. 19)
It is easy to see that these differential
conditions
represent
the constraints
equations
described
and boundary
by Eqs. 5. 3 and 5. 4.
With these last re sults , vre have formulated
in a form where we can apply the Minimum Principle.
the pr oblem
119
2.
The Minimum Principle
In this section
Principle
two comments
is v(t) not s(t),
the system.
conditions
are in order.
First,
we shall not develop
on the Principle
i tse If.
Minimum
for optimality.
which is one of the components
Secondly,
to References
Conditions
we shall use Pontryagin's
to derive the necessary
proceeding,
ma terial
and the Necessary
Before
the control
function
of the state vector
for
much background
For further
information
we refer
6 and 13.
The Hamiltonian
poL( ~(t), s(t))
for this
system
is
+ p T·..s..(t).s.(t) + P.!l.T·(t)
•
2l.(t) + Ps (t) s(t)
2
Po -W- ~(t) C(t) ~(t)
o
+ p.£.T(t)(F(t)
~(t)
+ G(t)
Q G T(t) !1(t))
(5. 20)
':~Weshall drop the arguments
when there
is no specific
need for them.
120
'We have denoted the costate
the dynamics
E.(t).
of the system
The subscript
costates
dependent
(Eqs.
indicates
of the constraint
The system
of the state
5.6,
5.7 and 5.13) by the variable
state variable.
may be explicitly
has a fixed time interval
boundary
equations
conditions
constraints
The
timeand has
at both ends of the time interval.
Let ~(t), ~(t), and s(t) be the functions
differential
describing
are denoted by A.E(t) and x'B(t).
that we want to optimize
conditions
equation
the corresponding
equations
(non-autonomous),
boundary
vector
expressed
by Eqs.
given by Eqs.
of Eqs.
5.3 and 5.4,
5.8,
5.6,
that satisfy
5.7 and 5.13;
the
the
5.9 and 5. 14; and the
when the control
function is v(t).
,..
The Minimum Principle
necessary
that there
"
f\.
states:
In order
exist a constant
A.E(t), and x'B(t) (not all identically
assertions
a.
p
o
zero)
that v(t) by optimum,
and functions
1\
1\
1\
A
·..Pg(t)= V'~H,
2..
s
-~
*
(5. 21)
(5. 22)
n
",'
1\
P (t) =
s
"
x'E(t) =
",. /\,/\
=- H(~,
A
hold:
-.!l.
H
"
Pe (t) , P (t), P (t),
such that the following four
P (t) = V' H,
*
,.
it is
-en/as,
(5. 23)
-aa/as
(5. 24)
1\
1\
"
1\
I\.
Ito
A
"
,..
11,S ,xE' Xg Po' E.~' E..!l.'Ps' A.E,A.B,v , t)
121
As expected the energy and bandwidth constraint
constants
(therefore,
derivative
are
we shall drop the time dependence
Since we have no boundary
can minimize
costates
the Hamiltonian
upon the control
vs. the variable
notation. )
region,
we
v(t) by equating the
to zero.
(5.31)
Furthermore,
for this to be a minimum,
v
In general,
for v(t}.
=
,.. ~o
we require
equivalently,
from
that
5.31
(5.32)
v(t\
we can show that A.
B
> o.
Then we can solve Eq. 5.31
This yie lds
(5. 33)
Substitute
Eq. 5.33 in Eq. 5. 13.
Now w'e shall write the canonical
equations
expressed
5.7,
by Eqs , 5.6,
in an augmented vector
4n
+2
equations,
form.
5.13,
This yields
5.26,
5.27 and 5.28
a homogeneous
set of
122
(5. 25)
b.
for all
t in the interval
" ~ A
1\
H(~~, !l'~'
xE' xB'
as a function
"
A
[To' Tf] the function
"
P o· is a constant
d.
the costate
with p0
vector
2=
the derivative
t) .
d
0;
to the manifold
at each end of the interval.
what each of these
operations
indicated
assertions
by Eqs.
implies.
5.21
to
"
.Ps,(t) = -Po ~
1\
2
C
TAT"
(t) s(t) - F
T
2
"
(t)E.~(t) - C (t) N C(t)P.!l.(t))
o
0
To
.E'n(t)
...
rmrurmz e
1S
v;
conditions
Let us now examine
we find
v,
is perpendicular
defined by boundary
5.25,
~
I\.E' I\.B'
of the variable
c.
If we perform
~
E~' E.'!l'Ps'
=
Ps(t) =
-G(t)Q GT(t)~l.(t)
~p
0
2
'N
o
CT(t) ~ (t)
-
+
:5
:5
0
P-21(t)
T
:5
2
N
T j
o
t
(5. 26)
f
To:5t
F(t) Pn(t))
+ CT (t)
t
:5T
f}
(5.27)
- A.E(t)~(t), (5. 28)
:5
'
T"
fJ
(5. 29)
(5. 30)
~
-
<.u:ftl
I
I
0
0
0
0
CJ
:;:;8-a
~
CJ
2:
P:.i
:;:;<:W;I
1
~
--
<~
0
0
0
~
,~z °
80
:;:;-
8I
P:.i
~
0
-:Nk;°
~
123
:;:;<00
rrl
I
<:
.....
0
0
00
:;:;( 0..
+:i"
:;:;<0..1
0
-
<:1'
0
-s:=t
0
:;:;U
P:.i
:;:;-
a
CJ
8
-
8
+:i"
2:
-.wi
I
0
0
NlzO
0
~
I
-«
0
:;:;-
-
o
O
:Nlz
I
0
0..
:;:;-
<0.
':;:;-
to
<0.1
-~ -
0
U
-
Nlz °
:;:;-
8U
I
~
:;:;-
8-
8I
<0.1
CJ
P:.i
, ~O
,N
+i'
U
0
0.
80
0
I
0
0
::;::;-
0
<00
0
8
::;::;-
:;:;-
<1=1
rol~
.
rt'l
~
L()
CH
0
8
VI
~
VI
8
124
(From now on, we shall dr op the
A
notation and assume that we
refer to the optimal solution. )
If, in assertion
constant p
asymptotic
o
is identically
case.
(c) of the Minimum Principle,
zero,
then we have what we shall call an
We shall return
us for the interim
set p
equations are linear,
o
to this case later;
equal to unity.
this entails
however,
(d).
let
Since the costate
no loss of generality .
. Let us consider the boundary,or transversality,
implied by assertion
the
At the initial time,
conditions
these conditions
imply
(5..35)
and p (T ) is unspecified.
s
0
At the final,
or endpoint,
time we have
(5. 36)
and p n(T f) and .Ps(Tf) are unspecified.
are unspecified
constants
(A.
B
+
0).
We also have the boundary
5. 8, 5. 9 and 5. 14.
conditions given by Eqs.
total of 4n
2::
We also have that A.E and A.B
2 boundary conditions.
Therefore,
In addition,
we have a
we should notice
that we do not have to find the control v(t) in order to find s(t),
although we may easily deduce it from E q. 5. 33.
3.
The Reduction of the Order
of the Equations
by 2n
We are now in a position to show how the assertions
the Minimum Principle
optimal signal.
may be used to find the candidates
However,
before proceeding
of
for the
we shall derive a
125
result
that significantly
prove that,
simplifies
the solution
method.
We shall
in general,
_C(t)
~
=
P (t)
--!l
(5.37)
l
(5. 38)
This reduction
was suggested
by the variational
We point out though that our derivation
constraint
imposed
differential
upon the signal;
equations
approach
is independent
i,e.,
of Van Trees.
of the type of
it only depends
upon the
for ~(t) and !l(t).
Let us define two vectors
.§.1(t) and g 2(t) as
(5. 39)
(5.40)
If we differentiate
these
5. 26 and 5. 27 (with p
~ l(t)
..€ 2(t)
o
two equations
and substitute
Eqs.
5.6,
5.7,
equal to 1), we find
=
F(t) ~(t) + G(t) Q GT (t) !l(t) - G(t) Q GT (t)p s(t) + F(t) l?:o(t)
=
F(t)S1(t)
=
T
+G(t)QG (t)'s2(t),
CT (t) ~
C(t) ~(t) - F T (t) !l(t) - CT (t) ~
o
=
To===t===Tfl
s (t)
0
C(t) .s. 1(t) - F T (t) Eo 2(t) J T 0 ===
t ===
T r-
CT (t) ~
o
(5.41)
(5. 42)
126
The boundary conditions that the solution to these differential
equati ons satis fy may be found by using E qs. 5. 8, 5. 9, 5. 35 and
5.36.
They are
(5. 43)
(5.44)
Consequently,
differential
However,
Eqs.
5.41 to
5.44 specify two vector
linear
equation with a two point boundary value condition.
these equations are just those that specify the eigenvalues
and eigenfunctions
for the homogeneous
as shown in Chapter III,
Eqs.
in order to have a nontrivial
Fredholm
3.10 to 3.13.
integral
equation
We have shown that
solution to this problem,
we require
that
(5.45)
where A. is an eigenvalue
1
colored noise.
Clearly,
of a Karbunerr-Los ve expansion of the
the only solution is the trivial
proves the assertion
4.
of Eqs.
The Transcendental
since A. ::: o.
this is impossible
one, 1. e.,
1
Consequently,
~ 1(t) = ~ 2(t) = Q, which
5.37 and 5.38.
Equation for A and A and the Se lection of
E
B
the Optimal Signal Candidate s
In this subsection
to derive
a transcendental
we shall use the necessary
equation that must be satisfied
conditions
for an
127
optimal solution to exist.
The method and result
are very similar
to that which we used in Chapter III to find the eigenvalues
homogeneous
Fredholm
equation.
that this equation is in terms
we had but one before.
generate
The most important
Because
previous
subsection
Once we satisfy this equation,
of the linear
+
dependencies
2 equations.
we can
+
derive d in the
2 equation specified
F(t)
G(t)QG T(t)
.!l.(t)
CT(t) ~ C(t)
-F T(t)
0
.
0
-c T (t)--?N
0
~(t)
0
!l.( t)
0
=
s(t)
0
in
We have
~(t)
dt
is
for the optimum solution.
we can reduce the 4n
Eq. 5.34 to a set of 2n
d
distinction
A and A. , whereas
B
E
of two parameters,
a signal which is a candidate
of the
0
1
0
s(t)
- A.
B
4
-N C{t)
P (t)
s
-~
0
0
T
The boundary conditions
5. 14.
These conditions
be satisfied
ps(t)
(5. 46)
:St:STf"
are given by Eqs.
specify 2n + 2 boundary
5.8,
conditions
5.9,
and
that must
for an optimum to exist.
Since Eq. 5.46 is a homogeneous
not, in general,
have a nontrivial
may obtain a nontrivial
associated
0
0
solution,
solution.
linear
In order
equation,
to find where we
let us define the transition
with Eq. 5.46 to be X(t, To: A. , A. ).
E B
we may
matrix
We emphasize
the
128
dependence
of X upon ~E and A.
B
by including them as arguments.
Since Eq. 5.46 is linear,
to Eq. 5.46 in terms
conditions
specified
of this transition
by Eqs.
we can determine
matrix.
any solution
If we use the boundary
5.8 and 5. 14, we find that any solution
that satisfie s the initial conditions
may be w-ritten in the form
s(t)
P
't)( t)
,,( T )
0-
0
,,(T )
-
0
(5. 47)
o
s(t)
To::: t :::Tr
The final boundary condition requires
zero.
In order
transition
that .!l(T ) and s(T ) both be
f
f
to see what this implies,
matrix
let us partition
as follows (we drop the arguments
this
temporarily):
129
By substituting
partitions,
Eq. 5.38 in Eq.
the requirement
5.37,
we find that in terms
that !l.(T ) vanish,
f
of these
implies
(5.49)
Similarly,
we find that s(T f) being zero requires
o =
[X t(Tf,T
sE.-
+ Xs Ps (Tf,
We can write Eqs.
0
:A.E,A.B)P
0
+ Xs!l (Tf,T
0
:A.E,A.B)],,(T )
T :A.E, A.B)p (T )
0
s
0
5.49 and 5.50 more concisely
-
0
(5. 50)
in matrix-vector
form
,,( T )
-
o
=
(5.51)
0
130
or by defining the matrix
in Eq.
5.51 to be D(AE, AB), we have
T} (T )
0
(5. 52)
Equation
algebraic
equations.
a nontrivial
solution
to be identically
a set of n + 1 linear
5. 51 specifies
homogeneous
The only way that this set of equations
is for the determinant
zero.
Consequently,
of the matrix
can have
D(~,
the test for candidates
AB)
for the
optimal signal is to find those value s of AE and AB ( > 0) such that
(5. 53)
Once Eq. 5.53 is satisfied,
to Eq. 5.52 up to a multiplicative
p (T ), allows us to determine
S
we c an find a non-zero
constant.
Knowing !l(To) and
the candidate
signal(s),
SA
0
the particular
values
multiplicative
constant
constraint
of Eq.
of A and AB that satisfy
E
may be determined
solution
Eq. 5.53.
A
E'
B
(t), for
The
by applying the energy
5.3; i ,e. ,
(5. 54)
131
By using Eq. 5.33 we can determine
the bandwidth of the signal.
We have that
2
B =
~ Sf
T
2.
( d s~,
AB (T) )
dT
Tf
1
d-r =
S
EX.2
B
0
P
T
2
S
A. A. (T)dT
E' B
0
(5. 55)
In order
to satisfy
Eq. 5. 53" we require
that only the
rank of D(A.E"A.B)be less than or equal to n the dimension
!l.(t).
The case when this rank is less than
important
aspect of this optimization.
n presents
of g(t) and
an
For convenience,
let us
define
(5. 56)
n
D
specifies
the number
of linearly
independent
that we may obtain for the given values
solutions
necessary
in turn specify
conditions
"n
functions,
for optimality
We see that because
constraint
of any linear
of A.
the
and quadratic
of these functions
that have the
the necessary
Consequently,
than 1, we must consider
checking to see which candidate
A.B. These
v(t), that satisfy
of the linearity
same values of A.E and A.B also satisfy
is greater
and
to Eq. 5.52
given by the Minimum Principle.
combination
given by the Minimum Principle.
E
solutions
these linear
conditions
any time we find
combinations
is indeed optimum.
Of course,
"n
when
__ ------------------------=---=~-~=~=-~~=.-=o-=--=--=.=--==
__
=_~=
__
__=__
=---=---~-----~--~--
132
these candidates
i .e.,
are subject to the same constraints
the energy and bandwidth constraints
as any other;
given by Eqs . 5.54
and 5. 55.
5.
The Asymptotic
Solutions
An issue which we deferred
asymptotic
case when p
o
an useful in the analysis
equals zero.
algorithm
that satisfy
p
o
is zero the asymptotic
pathological
p
o
solutions.)
problem,
of the
We shall call the solutions
conditions
of the Minimum Principle
solutions.
(They are often called
In order to test for their existence,
equal to zero in Eq. 5.34 and examine the differential
P G(t) and E!l.(t).
form,
provide
it is worthwhile
with the discussion
of our design procedure.
the necessary
Since these solutions
of a particular
to examine them before proceeding
was the question of the
If we write these equations
we obtain the following homogeneous
when
we set
equations
for
in augmented vector
equation
I
-FT(t)
I
-C T(t) ~
I
d
C(t)
0
= --------t------~
dt
I
-G(t) Q GT(t) I
P (t)
-~
F(t)
(5.57)
The appropriate
boundary conditions
are specified
5.36.
From
Chapter III, Eq. 3. 13,
by Eqs.
5.35 and
133
(5. 58)
Let us define the transition
matrix
with Eq.
We note that <I?(t,T:-N /2) is related
<I?(t,T: -N /2).
o
'j
<I?(t,-r:-N
/.
o
T
with W(t:-N 0/2)
2) ='1' (T,t:-N
Any solution
In order
the bo mdar'y conditions,
to Eq.
5.57 to be
to the transition
0
matrix '1'(t, T: -Nd/2) . associated
<I?(t,T: -N /2).
o
associated
/ 2)
(5. 59)
0
5.57 may be found in terms
to find the solution
we partition
to Eq.
<I?(t,T:-N
o
of
5. 57 that satisfies
/2) into four
n x n
submatrices,
N
<I?(t:T:- ~)
If w'e incorporate
that the solution
=
(5. 60)
the boundary
to Eq.
condition
specified
by Eq.
5. 36
5.57 is
p (T )·
-!l. f
(5.61)
134
The initial condi ti on specified by Eq. 5. 35 require s
p (T )
-11
0
T
= '1'14U(Tf,
= - p U""-;S.
_P
1- (
No
T :- -2 )p (Tf)
0
-n
T )
0
= - P0'1'T~(T
,
.!l~ f
(5.62)
T :- N 2)P (Tf),
0
0n..
or
o=
(5. 63)
The only way that Eq. 5.36 can have a nontrivial
determinant
of the matrix
If we transpose
determinant
the matrix
value),
enclosed
we find that we have the test for an eigenvalue
for this determinant
1
where
expansion
>
0
associated
colored noise process.
Consequently,
There we showed that the only way
to vanish is for
A.. is an eigenvalue
1
in Eq. 5. 63 to vanish.
in Eq. 5. 64 (this doe s not change the
that we developed in Chapter Hl ,
A..
by brackets
solution is for the
with the Karhunen
Clearly,
the only solution is the trivial
Loeve
this is impossible.
one, i,e. ,
135
c.
The Signal Design Algorithm
In the previous
necessary
section we have derived the results
for solving the signal design problem.
we shall use these results
devise an algorithm
In this section
(and a fair amount of experience)
which when implemented
to
on a digital compute r
will find the optimal signals and their performance.
Ideally,
we want to be able to find the optimum signal
for given values of E and B.' Although this is certainly
possible,
it
is far more efficient to solve a p ar ttcular- problem where we let B
be a parameter
and then select the spe cific value in which we are
intere sted (the energy is normalized
to unity).
The reason
approach will become apparent when we consider
for this
some specific
examples.
The Minimum Principle
necessary
has provided us with a set of
condition from which we found a test (Eq. 5.53) for an
optimum signal.
The result
of this test is that we essentially
an eigenvalue problem in two dimensions.
The most difficult aspect
of the problem becomes finding the particular
that both satisfy this test and correspond
bandwidth.
The algorithm
method of approaching
values
of AE and AB
to a signal the desired
that we suggest here is simply a systematic
this aspect of the problem.
The algorithm
outline it.
have
has several
steps.
First,
Then in the next section we shall discuss
we shall
it in the context
of two examples.
a.
Find the loci points in the A , A plane that
E
B
satisfy Eq. 5. 53.
for calculating
This requires
transition
that we have an effective procedure
matrices.
In general,
one can simply
136
(5. 65)
If we substitute
differential
Eq. 5.65 in Eq. 5.34,
we find that the
equations for s(t) and ps(t) are
1
= - ~I\.B
s(t)
P (t)
s
(5. 66)
1
(5. 67)
The only solution to these equations that satisfies
the boundary
conditions specified by Eq. 5. 14 is
s(t) =
-J
~
2T
sin
(t-T.)
n1T T f - ~
(5. 68)
0
with
(5. 69)
Several comments
are in order.
these signals is easi lyshown to be nn/T.
assertion
(a) in our application
is non zero.
invariant,
We did not require
these solutions,
to transmit,
within our model.
(2.)
We have not violated
of the Minimum Principle
since p (t)
our system to be time
nor did we specify the dimension
Consequently,
practical
(3.)
(1 .) The bandwidth of
of the system.
which are probably the most
exist for all types of colored
noise that fit
s
137
numerically
transition
integrate
matrix.
generation
the differential
However,
equations
if the matrices
of the channel noise (F,G,Q,C)
use the matrix
that specify the
that describe
the
are constants,
then we can
i.e. ,
exponential,
(5. 70)
where Z(~,
A.B)is the coefficient
b.
matrix
For a particular
for !l(T 0) and p s(T 0)·
of Eq.
point on these
5.45.
loci,
solve Eq. 5. 52
Then use Eq. 5.47 to determine
the signals
sA. A. (t}, P A. A. (t) and ~A. A. (t).
E' B
s E' B
. E' B
c.
normalize
Since the performance
these signals
d.
normalized
Calculate
signals
e.
such that
is linearly
relate d to the energy,
sA.E,A.B(t) has unit energy.
the bandwidth and performance
as specified
by Eqs.
5.55 and 5. 11, respectively.
Repeat parts b, c, and d at appropriate
along these loci in the A. ,A. plane.
E B
(The interval
enough so that the bandwidth and performance
vary in a reasonably
continuous
of the
manner.)
intervals
should be small
as calculated
in part d
As we move along a
particular
locus in the. A. ,A. plane, plot the degradation
E B
2
bandwidth in a second plane, a d2, B plane.
~
g
f.
As mentioned
earlier,
we ne ed to pay spe cial
attention to the case when Eq. 5.53 has more than one solution.
situation
~,
corresponds
A.Bplane.
produced
to a crossing
of two or more loci in the
In this case find the solutions
2 2
in the d , B plane by linearly
g
and plot the locus
combining the different
This
138
signals.
Probably
the Fredholm
the most convenient
integral
g.
means of doing this is to use
equation technique
discussed
in Chapter
For a given value of bandwidth constraint
IV.
the
optimal signal is the one that corresponds
to the lowe st value of
d2.
g
while the others
This signal is the absolute minimum,
to relative
minima.
h.
in determining
We recommend
that one use his engineering
which loci on the ~,AB
The problem
studied,
true,
complex.
we have observed
would considerably
one chooses
largest
of determining
is being approached.
which loci in the AE, AB plane
the optimal solution is time consuming.
these loci can be rather
As we shall see,
In the two examples
a phenomenon
that we have
which, if it were generally
simplify this aspect
of the algorithm.
a value of AE and finds the solution
value of A , the signal that corresponds
B
B
and find the solution
in abs olute value.)
to this point is
a monotonic function of either
~
(Similarly,
of Eq. 5.53 with the largest
With this conjecture
If
of Eq. 5. 53 with the
globally optimum for the value of B that the signal has.
one can fix A
judgment
plane are on inte rest and in
deciding when the white noise performance
generate
corre spond
AE
the bandwidth then becomes
or AB. The only possible
difficulty that would be in finding the points corres ponding to loci
crossing.
We have not been able to prove that this phenomenon
is true in general.
However,
it seems
quite promising
evidence that we have and the analogy to the smallest
optimization
.
d .3
lmpose
procedure
in view of the
eigenvalue
when there is no bandwidth constraint
139
D.
Examples
of Optimal Signal Design
In this secti on we shall illustrate
shall consider
our technique.
We
the colored noise to have the one and two pole spectra
described
in Chapter II.
Example
!-Signal
Design with ~ One Pole Spectrum
The equations
describing
component of the observation
Since the process
the generation
of the colored
noise are given by Eqs.
is stationary,
let us also set To
=
2. 15 and 2.16.
=
0 and Tf
In order to set up the te st for the optimum signal,
need to find the coefficient
coefficients
d
dt
of the Eqs.
matrix
in Eq.
2. 15 and 2. 16 into Eq.
£(t)
-k
2kS
t](t)
2
No
k
0
0
0
2
-N
the
5.46 we obtain
0
£(t)
0
,,(t)
(5.71)
1
- A-
0
we
If we substitute
0
=
s(t)
5. 46.
T.
s(t)
B
4
ps(t)
-N
0
From Eqs.
5.8,
0
-A-
E
5.9 and 5. 14 the boundary
0
P (t)
conditions
s
are
£(0)
=
811(0),
(5. 72a)
t) (T)
=
0,
(5. 72b)
s(O)
=
s(T)
=
0
(5.72c)
140
Step (a) in our algorithm
requires
in A , A plane that satisfy Eq. 5.53.
E B
calculate
the transition
From this matrix
us to find the loci
To do this,
we need to
(4 x 4) of Eq. 5. 71 x{ T, 0 :AE, AB) .
matrix
we can compute D(AE, AB) (2 x 2) by using
Eq.5.51.
To illustrate
a computer
k, No/2,
to perform
S, and T.
k
=
1
=
1
=
=
1
this,
let us choose specific values and use
the required
calculations
for the parameters
Let us set
N·
0
2""
S
T
(5. 73 a-d)
2
The loci of points in the ~,
illustrated
in Fig.
A
B
plane that satisfy Eq. 5.53 are
5. 1.
In this figure only six loci have been plotted.
with lower values of A
the final solution.
B
for a given ~
(They correspond
width and performance
Other loci
exist but are not significant
to signals
in
with a large band-
very close to the "white noise only" limit.
We found these loci by first fixing AE and then locating the zeroes
of Eq. 5.53 as a function of AB·
According to Eq. 5.32,
only in region of the plane where A
B
example we did not observe
Consequently,
AE <
o.
we need to look for solutions
> o.
In this particular
any loci in the first
the only region of interest
quadrant
is where AB
>
of the plane.
0 and
141
The reas on for considering
should now be apparent.
We see that the loci asymptotically
approach those of the asymptotic
solutions.
a convenient place to start
E
near those specified
of the asymptotic
solutions
We have sketched their loci as given by
Eq. 5.69 with dotted lines.
values of A
the asymptotic
by Eq. 5. 69.
Therefore,
searching
for large
for the loci is
We should point out that the loci
solutions do not cause the determinant
by Eq. 5. 53 to vanish.
This is because
existence
we used the equations directly
Principle
rather
than the reduced
in determining
specified
their
derived from the Minimum
set of equations that were used to
derive Eq. 5.53.
Now, if we take the solutions
and determine
specified by these loci
the bandwidth and performance
signal according
of the corresponding
to steps (b) through (e), we produce
loci in a d2_B2 plane.
g
These are illustrated
in Fig.
a second set of
5.2.
identify the corresponding
loci by the numbers
addition we have indicated
the bandwidth and performance
asymptotic
We can
1 through 6).
In
of the
solutions by a large dot near the ide ntifying number
the loci aht approaches
because the entire
it.
(We can indicate
loci for the asymptotic
of
it by a single dot
solution corre sponds to
just one signal.)
In the AE - AB plane of Fig. 5. 1 as we move from left
to right on the solid lines,
Eq. 5. 53 to vanish,
those that cause the determinant
we generate
the solid lines in the direction
2
2
the loci in the d _B plane with
g
indicated by the arrowheads.
that they evolve from the dots corresponding
solutions.
of
We see
to the asymptotic
As can also be seen, the loci corresponding
to numbers
Fig.
5. 1
Loci of Points for
det[D(AE' "B)] = 0
First Order Spectrum
~...ilC.J..l.....l...l..l....l..U.JL.W..J....:...W.;~..1..!..!..W.~~...l.....i.J..I....i-J,,.l..;~~~~..l...W.J~...J.:l..U.~~.....l.-:.-J.....J......!..:....l...:..J....:..w...l...!...!..l.~...:....!..!..W..i..W..I..J.~;w''u~~~~~''''''''~~~..w.:..
...........
~....:...:..;,
........
~~~~
......
~
----------I-.l
I.
143
one and two are well behaved,
remaining
numbers
while those corresponding
have a rather
erratic
behavior.
As indicated by step (f) the other aspect
that needs to be considered
~
- A. plane.
B
is the crossings
At these crossings,
of the optimization
of the loci that occur in the
there are two signals with the
same values of A.E and A. that satisfy the necessary
B
If we find the degradation
to the
conditions.
and bandwidth of all possible
linear
combinations
of these signals (normalized to unit energy), we pr-oduce
plane
the loci in the d; - B1indicated by the dotted lines.
(There is no
relation between the dotted lines in each plane.)
that only some crossings
the absolute minimum,
the crossings.
corresponding
are candidates
for
we have not plotted the loci produced by all
Once we have generated
width constraint
1. e.,
are significant,
Since it is evident
region of interest,
for any particular
these loci over the band-
we can find the optimal signal
bandwidth constraint
value.
We
merely select the signal which produces
the absolute minimum 22 plane
or the lowest point in the d . - B / for the spe cified
degradation,
g
value of B.
At this point we pause to discuss
mentioned in the last section.
the two planes.
let us trace the loci in
Let us start at the top right of the graph with ~
large in absolute value.
downward,
To do this,
the phenomenon we
As we start to increase
2
2
the locus in the d - B plane eminates
and proceeds
g
A.E by moving
from point one
across
to the right.
The point on this locus where
the dotted line starts
corresponds
to the first
A.:E -
A.
B
plane.
this crossing
loci crossing
All the points on this dotted line correspond
point.
As we again move downward increasing
in the
to
A.E
I
t-
I
-1-
f-+
f-'-
--
1----1-
-r=i!
I
I
j=J
---L.
I--+-t-
I
--'--
I
--1= -~
t--
Fig.
5,2
Performance vs, Bandwidth
for the Optimal Signal Candidates
First Order Spectrum
10.
'00
1000.
145
and take the largest
A
B
solution,
we generate
the solid line.
This
continues until we reach another" dotted line which again corresponds
to a loci crossing.
wish.
We can continue in this manner
The important
patterns
as far as we
point is that we could ignore the complicated
corresponding
to remaining
values of A
B
that satisfied
our te st.
Let us illustrate
signals for two different
values of the bandwidth constraint,
2
the first ill~stration
of Fig.
the actual form of some of the optimal
we haveB
"
B.' In
2
2
If we examine the d g - B
equal to 9.
5. 3, we see that the optimal signal is a linear
combination
of the two solutions that are produced by the first crossing
one and two in the A
constraint
- A
E
is illustrated
a degradation
B
plane.
in Fig.
of 23.6 percent
of loci
The optimal signal for this
5. 3. We see that we can achieve
less than the white noise only
performance.
In addition,
the optimal receiver.
for this constraint
principally
we have drawn g(t), the corre lation signal for
The signal(s} exhibit no particular
value of B equal to 3 as it is composed
of the signals sin(iTt/T} and sin(2iTt/T}.
that because
of the possibility
of two different
are actually four signals that are optimal.
basically
symmetry
We should note
sign reversals
All of these,
there
however,
have the same wavesha pe.
In Fig.5.4
correlating
signal g(t} when we allow twice as much bandwidth,
L e. , B2 = 36.
to a crossing
we show the optimal signal s(t} and its
In this case,
in the A
width is sufficiently
E
the optimal signal does not correspond
- A plane,
B
as it did previously.
The band-
large enough so that we can attain a performance
146
,.
t
H
·
F
I
ih+
t
It
rmrr
Ii
r1Tf 1 tijl
H+·+
~.
r
i
;
.,.
;
J1
~
+
J
Optimal set) and get)
for a First Order Spectrum
and B2= 9
i:
-titt _'
,t-·
-It+-
, H-
2
Performance
J'
"fit
dg
=
. 265
.t
.1
.
;:1:1=·
t:
'4~ff
- ,·~1
~i
J.~
,
!$t~
±± It ·1
i-Hfl
f+H~
i
i
·1
1++ lli~
+j
J'J
..
L
• r ,
c
-r
-1_
4
1tri-l i~
J:HbJ
+
j
ftlf**~'
,
~:
$frJ
-rJ•
It-
+j
r~
,+H Il
-\1- L.
jH:t·
I·
+t'
• .1-
-+Jt- ,+
..l.
r
+F
...
-H+ -'-
m~·~i'
H
'jI
4:
-1+'
,--t--+
c.
"
,-
.r
It, :';l
I
~-I.
-tfr.
'.j
J
t- '
t
, ~ t-
''''
++
--+-+.-t
-1
I
f
J
'.
-r-t .
J-.'
,-I
J--l..4-
',Rt'± rt
I
J
"
±+",
1-:+
>H+
;:
.b]
,
,-i->
Iii
'-j
, ,+
.w.
. f+
~~
! ~ j~'i
,
,·t
+ "
t-
-l ~
-j'
t.L,'lf: . : .-t
,+
I
+
l.f
5.3
-:/~ t-
~
:++
••
t~-+
~i tj::rt Ii:
t
·1
r
!
,
i~
ttlPl"
1+
,+ ,
It- -_.
f
fIn1+
+1
,
~:
'+
Il
1~±l'
.1
:
,:++
- ~F
it It - t.
~j
~
: ,1Jf m~lttHH:tjlll:tn+l+++tlttl!
Ii
It:
!
:t! t±±.j:t'l-+-tt
J·,t1:l±ti-+
t ±
t
..
Fig.
t ..
:t
1~
:t
-ll- J-I
it-t'rt-H.-t.rtl~,1t
tit
H
"
tt!
"Hf:'
It+
)S
H-tt'lt-t-t-t'H-t
-t-
'H
I
II
+-t+t
I
II
I
i.
I'
·~.,. i. H-: t-i+"-
-1"'1
':PY'H+H -+- '
I 1-'
j-r'-' liTH'
f+ -'-tit
-t -Lt-'- 1T rTTTT
,
-I+f+
i
I,,-
=f-F
:HH-I.
, i+H
j
,
i~'
, ,-+-H-l-f-
iJ±t -~t-H-+
" ::+--:-I~H:i
, i$t
, -+-+--t"
H+r:
~h-;
l'
j~:.t;-:
~H-I'+"HT~
hH-r:- .
IT',
''t
·H··..·
147
Fig.
5.4
Optimal stt) and g(t)
for a ~irst Order Spectrum
and B = 36
Performance
i,
'+-l
m+
'
2
dg =. 0643
,H-+
--j~- -
-!+,
tH+ 11- l+-
, ++ :+
+t H- H- T f+t-,
, itT
+++1= f:~
E±
m '+~
t+ -
++
+ --t-+_ 11,+~# r
J
-T
ml:,mJ
-t--r
j
.
+
j
+--l-i-
~
-~t
'-i
I
-t-d-
+R+tit8
Rf=t=+IH-Tj---I+ -+:;
+,- --t+ It~: =im $=t'~fJ n
-+-
~-t
"1,
'+'
+t
i- -J
'
"
++t'
-~-t+~
-fm' :'
+
I
++1..1- ..
++
-
~+-i+
-H J-+
,
t~~~~-
it1-+
++tt
-] t-; ~
ITH
'-14'-+
' h-++J--H+l-t-t-i' ~
l++t
j-++~ _
-1++
t+-H--: '
q:iHH-l-
-1-: c++++
=t
l
.i
H
H-H
,,-;
->-
t
~+
1-1--
ii H-
L
i
i_<
I;:_-~
til--!-
-+ L+++
t++ .
-l++-++J.
lJ
148
only 6.4 percent below that of only the white noise being present.
The signal does display some symmetry
the correlating
in that s(t)
signal is almost identical
=
-s(T-t);
and
to s(t) indicating that we
(nearly) have a white noise type receiver.
We conclude this example by discussing
over a conventional
optimal signal.
transmit
signal that is attainable
We propose the following comparison.
The bandwidth that this signal consumes
We can find the performance
of these signals
of bandwidth,
to the asymptotic
sine wave.
when referenced
while 12 references
to
For this same amount'
beneath the dots in
of the pulsed
Index II reflects
the
to the pulsed sine wave performance
it to the white noise only pe rforrnance ,
since it reflects
12 is
'the total
of the system.
We can see that tIE performance
indicated by l
reasons
= (nTI/T)2 .
by referring
t with the performance
probably the index of most significance
improvement
Z
g op
We have done this in Table 1.
improvement
B
of the optimal signal -
(We can do this by looking directly
d2
g sm
sin(nTIt/T).
5.2 since these signals
solutions.
let us find the performance
Fig. 5. 2. ). we now compare
..j ZEIT
Z
d .
an
Let us
is n1T/T, 1. e.,
or from the large dots on Fig.
also correspond
d: opt.
by transmitting
a unit energy pulsed sine wave, 1. e.,
Fig. 4. Z
the improvement
Z
is hardly outstanding.
for this.
For the parameter
is just as significant
improvement
We can conjecture
values chosen,
two
the white noise
in effect as the colored component.
Also, the
one pole or first order spe ctrum is difficult to work with since the
_noise does not have much structure
signals c orre ctly.
to exploit by designing the
.
.
.....
0
U1
.....
0
0
U1
-J
N
-J
N
VJ
VJ
-J
VJ
0
......
0'
U1
N
.
-..0
0
.
o
0
ex>
-J
o
o
0
U1
VJ
o
o
~
-J
.....
N
VJ
U1
0
-..0
N
VJ
o
N
0
0
-..0
0
0
.~
.o
149
(Jq
,...
::s
til
~
H
o-S
~
0
0
~
......
N
N
0..
c+
0
'"0
til
,...
::s
"
N
0..
N
0..
0..
c+
0
'"0
(Jq
::s
0
0
N
0..
,...
til
.....
N
0..
II
(Jq
N
0..
.....H
c+
0
'"0
(Jq
,...
P>
,...
S
~
s
c+
0
'"0
til
0
ti
1-1)
CD
::s
o
P>
S
0
ti
1-1)
ti
CD
~
1-1)
0
::s
til
0
,...
ti
~......
(Jq
tn
P>
<:
CD
$j
::s
CD
,...
Ul
0..
CD
til
~
~......
0..
P>
::s
'"0
S
0
o
H
rt:r:j
tJj
~
150
Example ~ - Signal Design With ~ Two Pole Spectrum
In this example we shall consider
the signal design
problem when the colored component of the observation
noise has
a two pole spectrum.
reasons.
First,
We do this primarily
we want to develop a better
involved in implementing
for several
understanding
our algorithm.
of the problems
Secondly,
we want to
....
verify the phenomenom that we obser-ved with the one pole spectrum.
Thirdly,
the one pole spectrum
can give deceiving results
that our optimization
vements
considered
at times.
procedure
in the previous
Finally,
example
we want to demonstrate
can produce more significant
when working with a process
impro-
that has more "structure"
to
it.
We as sume that the colored component of the observation
noise is generated
spectral
by Eq. 2.18.
In addition,
we set the white noise
level to unity and assume that the interval
i.e ., T2 =·0, Tf = T = 2.
observation
interesting
Hence, the colore d component of the
noise dominates
This particular
because
'2
frequencies
for all frequencies
= 9.
This corresponds
We certainly
in frequencies
near the peak.
to allowing
where the spectrum
Yet, if the bandwidth is available
our previous
judgment that a linear combination
close to optimum,
the bandwidth
do not want to put all the signal energy
should be able to use it to our advantage.
from either
8 Hz.
is particularly
Suppose we constrain
which are lower than that frequency
has its peak.
conjecture
less than say.
colored noise spectrum
of its shape.
to be less than 3, or B
length is two,
we
Although one can
example,
or his engineering
of pulsed sine waves should be
it is not apparent that this is so.
151
Let us proceed with the steps in our algorithm.
first six loci in the A
E
Fig. 5.5.
- A
plane are illustrated
Again, we observe the asymptotic
negative values of A .
E
more complicated
First,
B
In two respects
and numbered in
behavior
for large
the loci exhibit a somewhat
behavior than those for the one pole spectrum.
they do not cross
simply pairwise.
Apparently,
with the adjacent locus;
i.e.,
third,
the second and fourth,
the third crosses
the second crosses
some regions they are extremely
Their behavior
etc.
Sec ondly, in
close together.
is very similar
others exhibit the same type of erratic
behavior.
there are regions where we must consider
combinations
of signals corresponding
B
plane.
Most importantly,
while the
As indicated by the
dotted lines,
- A
loci in the
to the one pole case.
We see that the first and second loci are well behaved,
~
they cross
the first and
In Fig. 5. 6we have plotted the corresponding
2
2
d - B plane.
g
The
to a loci crossing
the linear
in the
the maximum AB phenomenom
still occurs.
Let us now examine the shape of an optimal signal when we
constrain
the bandwidth such that the colored noise is dominant over
the allowed frequency rap-gee If we chose B
signal set) and its correlating
The signal is principally
=
13, the optimum
signal get) are illustrated
in Fig. 51.7.
composed of functions of the form sin(irt)
and sin(31T't). The degradation
approximately
2
realized
by this signal is .53 which is
15 percent below what one might expect using a non-
optimum approach of se lecting the pulsed sine wave with the best
performance
that still satisfies
the bandwidth constraint.
1
1
I
I
Fig.
5.6
Performance vs, Bandwidth
for the Optimal Signal Candidates
Second Order Spectrum
,
'OC>.
,I
1',
j..'
154
I.:::
Fig.
5.7
Optimal s(t) and g(t)
for a ~econd Order Spectrum
and B = 13
Performance
2
d = .53
g
00
00
N
-J
a-
U1
N
Vl
0
N
U1
-.J
0
.....
.....
a-
0
-o
N
N
N
a00
U1
-J
N
~
N
155
;:s
I-t
0..N
0...
N
0
to;
C'D
~
0
;:s
......
0
0
~
0.
N
en
~.
0..
N
-o
c-t-
"
~
0.
N
N
I-t
o-.s
0
0
......
~
0
(J'q
;:s
N~.
~.
en
P'
..-
;:s
(J'q
UJ.
C'D
<:
P'
~
c
S
S
c-t-
~.
0
"d
to;
0
H,
C'D
o
;:s
SP'
H,
to;
C'D
~.
;:s
0.1 /:
I-d
UJ.
H,
0
C'D
0..
;:s
0
en
0..
N
~.
en
to;
"d
P'
c
..-
I-d
0..
;:s
P'
o
0
S
"
......
(J'q
0
~
(J'q
en
~.
en
~.
;:s
(J'q
I-t
I-t
t:rj
1-3
~
OJ
t-t
156
In Table II we have s umrna r iz ed the performance
same way that we did in the first
example
the optimization
significantly.
example
has improved
This is probably
of this section.
our performance
due to the colored
in the
In this
more
noise being more
dominant in this example.
E.
Signal De sign With a Hard Bandwidth
In this section
we shall derive the differential
that specify the necessary
constrain
the absolute
Constraint
condition for optimality
value of the derivative
equations
when we
of the signal,
1. e. ,
we require
Iv(t)
As before,
I=
I
I
d~\t)
we constrain
s 2{-r)d-r:S
1
::5
Z
E
(5.74)
B,
the signal energy
by
(5. 75)
E
We can formulate
to that used previously.
the problem
The resulting
in a manner
Hamiltonian
very similar
is
T
s{t)C{t)~{t) + E.~ (t){F{t)~{t) + G{t)QG {t)21{t))
Po ~
o
(continue d)
157
(5. 76)
Taking the required
BH·
8I"
derivatives,
= -P g(t) = poe
T
we find
2
T
(t) No s(t) + F (t).P.s(t)
(5. 77)
~
an
=
-i>-!l.(t)
= G(t)Q GT (t)p c(t) - F(t)p (t)
- ~
-!l.
+ ~(t)s(t)
(5. 78)
(5. 79)
(5. 80)
The transversatility
conditions
imply
(5.81)
(5. 82)
158
We again have that A is a constant.
E
note that Eqs.
5.27,
5.35,
5.77,
5.75,
5.81,
5.34 respectively.
using the results
=
~(t)
Furthermore,
5.82 are identical
Consequently,
to Eqs.
we
5.26,
we can again show by
of Section B-3 that
(5.83a)
-p (t)
-ll
(5.83b)
Therefore,
Eq. 5.79 becomes
= -~
ps (t)
(assuming
Po equals unity)
( 5. 84)
C(t) ~(t) - ~s(t)
o
The major differe nee between the application
Minimum Principle
to this problem
in the minimization
of the Hamiltonian
v(t).
If ps(t) is non-zero,
of the
and the one in the text comes
as a function of the control
this minimization
implies
1
v(t)
=
This implies
"2
(5.85)
-E B sgn(ps(t))
that the optimal
signal has a constant
linear
slope of
+E 1/2B when p (t) is non-zero.
-
s
The differential
the necessary
dS(t)
dt
conditions
=
F(t) ~(t)
equations,
now non-linear,
that specify
are
+ G(t)Q
T
G (t)21(t)
(5. 86)
159
(5. 87)
1
ds(t)
<It
=
v(t)
=
-E
"2
(5.88)
B sgn(p s(t))
(5.89)
The boundary
conditions
are
(5.90)
(5.91)
(5.92)
If ps(t) is zero
over any region,
we cannot determine
v(t) by Eq.
5.85.
In such region
(5.93)
implies
s(t)
= - ~ 4N
E
C(t)~(t)
0
(5.94)
160
If we combine this equation with Eq. 5. 86, we find that the two
equations have the same form as those that specify the eigenfunction's
associated
with the colored noise.
This leads us to the following conjecture:
signal consists
of regions
of a constant
the optimal
slope of i.E l/2Band
regions
where the signal has the same functional form as the eigenfunction
the colored noise.
differential
F.
Finding a solution technique
of
for solving these
equations will be part of future research.
Summary and 'Discussion
We have presented
a state variable method for designing
optimal signals for detection in colored noise when there are energy
and bandwidth constraints.
The performance
by d2, which specified the loss of receiver
g
colored noise being present.
their associated
measure
We used the differential
as if they described
We then applied Ponttyagin's
conditions
conditions specified
determine
was given
performance
boundary condition that specified
and performance
necessary
measure
Minimal Principle
due to the
equations
the optimal receiver
a dynamic system.
to derive the
that the optimal signal must satisfy.
a characteristic
and
value problem.
These
We could
the optimal signal by solving this characteristic
value
problem.
We suggested
a computer
algorithm
for doing this.
By using this algorithm
we were able to analyze two examples
colored noise spectra.
In addition to finding the optimal signal and
its associated
performance,
intere sting features.
the algorithm
displayed
several
of
161
One may argue that we did not need to use the Minimum
Principle
to solve this problem
the 2n + 2 differential
usual calculus
equations
of variations
of the receiver.
since we could proceed directly
and boundary conditions via the
and the estimator-subtractor
The advantage of this formulation
when we change the type of constraints
we derived the differential
equations
I ds(t) I dt I :s B.
Minimal Principle,
apparent
In Section E
that specify the optimal signal
and a hard (bandwidth)
This problem
whereas
realization
becomes
upon the signal.
when we impose an energy constraint
constraint,
to
the calculus
is readily
solved using a
of variations
would require
much more effort.
We intend to continue studying the issue of hard constraints
by examining the design problem when we impose peak constraints
the signal itself and/or
its derivative.
aspect will be to find an efficient
resulting
differential
We expect that the important
computer
algorithm
to solve the
equations.
There are several
1.
on
other important
The most difficult aspect
issues.
of our method is to select
the signal that is globally optimum from all those that satisfy the
necessary
conditions
we have observed
optimum.
As we have discussed
a phenomenom which eliminates
We feel that there is sufficient
empirical
in the text,
this problem.
evidence to spend some
time trying to verify this theoretically.
2.
is a very general
The equivalence
result.
of ~(t) and -E.1") (t), and !l.(t) and E. S (t)
Because
of this generality
there should be a means of formulating
stochastic
minimum principle
it seems
these problems
that
with a
which would allow us to obtain the
162
final differential
been apparent
equations
However,
One can also pose the optimization
use a suboptimum
receiver.
For example,
to be a matched filter,
2'= ) j
g
problem
if we constrain
one wants to minimize
Tf Tf
d
this means has not
so far. 29
3.
receiver
directly.
when we
the
the functional
Tf
s (t)Ky(t, T)S (T)dT =
To To
.
5
(5.95)
S(t)C(t)~(t)dt
To
where we have defined
K (t , T)C(T)s(T)dT
(5.96)
x
The results
of Chapter II are now directly
applicable,
and we can
proceed in a manner analogous to the way we did in this chapter.
Similar
e. g.,
results
should exist for other types of suboptimal
those operating in a reverberation
4.
The algorithm
method for calculating
environment.
that we used requires
transition
matrices.
Consequently,
being reasonably
careful
we used a straightforward
this is certainly
an accurate
All the problems
we have examined to date have involved constant
systems.
receivers,
that
parameter
we could use the matrix
exponential.
we have not encountered
any difficulty when
series
no guarantee
sum approximati on.
that this series
By
However,
approach will always
163
work, for example as the dimension
of the system increase s , We
think that it is worthwhile to investigate
other methods to compare
their accuracy
method.
5.
to bandpass
possibility
carrier
and speed to the present
Finally,
the results
in this section can be extended
signal design by using the material
of a noise spectrum
introduces
that is not symmetrical
some interesting
advantage of this situation.
in Appendix B.
questions
The
about the
on how to best take
164
CHAPTER
VI
LINEAR SMOOTHING AND FILTERING WITH DELAY
In this chapter
we shall use the results
Chapter IV for solving nonhomogeneous
to develop a unified approach
delay.
to linear
In the smoothing problem
time inte rval [T
estimate
o
smoothing
integral
In a filtering
proble m we receive
time as a function of this endpoint.
endpoint time,
Tf' the estimate
Tf"
For the filter with delay,
a signal
which evolves in
of the state vector
of the state vector
T , the
f
S
T , is constantly
f
For the realizable
delay before we make our estimate.
Tf' the estimate
with
this signal over the
We then want to produce an estimate
g( Tf} vs .
equations
and filtering
L e. the endpoint time of the interval,
increasing.
in
vre receive a signal over a fixed
of the state vector that generated
continuously,
to find
Fredholm
We then want to find ~(t), T s t
0
,Tfl.
same fixed interval.
derived
filter
we want
right at the
we are allowed a
We want to find ~(Tf- A) vs.
A units prior
to the endpoint of
the inte rval.
Our approach to these problems
start with the Wiener-Hopf
response
implicitly
equation that specifies
for the optimal linear estimator.
find a set of differential
differential
3
We
the impulse
We then show how we can
equations that specify the optimal estimate
as part of their solution.
derive matrix
is straightforward.
equations
From these equations
which determine
we can
the covariance
165
of error.
Our approach to the problem of filtering
also straightforward.
In the solution to the optimal smoother,
simply allow the endpoint time of the observation
variable.
with delay is
We then derive a set of differential
function of this variable
endpoint time rather
a fixed obse rvation interval
we
inte rval to be a
equations
which are a
than the time within
as for the smoother.
The performance
is also derived in an analogous manner.
Several comments
First,
all the derivations
results,
are in order before proceeding.
in this chapter are original.
however,' have appeared in the liter ature.
referenced
where appropriate.
concerning
our methods is the approach.
deve loped concise ly and directly
They are
The most important
point to be made
The entire
starting
We shall require
the estimator
structure
regardless
of the statistics
approaches
to this problem are unstructured;
processes
the ory can be
from a few bas ic res ults .
Secondly, we shall employ a structured
topic.
Many of the
of the processes
approach to this
to be linear,
involved.
Existing
it is assumed
involved are Gaussian and then the estimator
is derived.
It is well known, however,
that the
structure
that both approaches
yield
the same estimator.
Thirdly,
we shall assume that the reader
with the well known results
variable
techniques,
Finally,
filtering
i.e. , the Kalman-Eucy
filter.
by using state
8,3
although these topics have been extensively
studied in the literature,
have appeared.
for realizable
is familiar
several
incorrect
derivations
We shall point out these errors
and results
and give the
166
correct
A.
results.
The Optimal Linear Smoother
In this section,
we shall derive a state variable
of the optimum linear smoother,
unrealizable
filter.
First,
the state variable
we shall establish
Let us assume that we generate
by the methods described
in Chapter II.
observe this process 'in the presence
over the interval
.!(t)
To
:5
t
:5
T
r
= -y(t) + ~(t) = C(t)
our model.
a random process
of an additive white noise w(t)
square error
of the system that generates
noise that has
R(t) O(t-T) as given by Eq. 2.7.
of the linear system
in estimating
we want to find a state-
that minimizes
consists
of two first-order
equati ons having a two-point boundary
Let us now proceed with the derivation
First,
optimal linear
w'e define the matrix
smoother
the mean-
each component of the state vector ~(t).
We shall find that this description
differential
the signal
(6. 1)
In the optimal smoothing problem
description
l(t)
Let us also ass ume that we
y(t) , ~(t) is an additive white observation
variable
of the
+ ~(t),
where ~(t) is the state vector
a covariance
equivalent
That is, we receive
~(t)
realization
to be h (t , T),
-0
impulse
or
vector
restrictions.
of these equations.
response
of the
167
1\
~(t)
=
This operator
t by observing
(6. 2)
h (t , T)r(T)dT,
~o
-
produces
an estimate
of the state vector,
rtr) over the entire interval
-
It is well known and can easily
methods that this impulse response
Hopf integral
[T
~(t) at time
,Tfl.
0
be shown by classical
satisfies
the following Wiener-
equation.
Tf
Kdr{t,
T)
=
S
~{t, v)Kr{v,
To
-
where Kdr(t, T) is the cross
signal and the received
of the received
For our application,
Therefore,
(6. 3)
T)dv,
covariance
signal,
of the desired
K (t , T) is the covariance
r
signal.
the desired
signal is the state vector,
~(t).
we have
htt , v)K (v, T)dv,
r
(6. 4)
~:~Althoughh (t , T) is a matrix and should be denoted by a capital letter
in our notaTi?>nconvention, we shall defer to the conventional notation.
TWe have assumed zero means for x(To)' v(t) and w(t). If the means
were non zero, we would need to add a bias term to Eq. 6.2.
168
with
(6.5)
The first
for h(t, T).
-c>
step in our derivation
In order
to do this,
we need to introduce
kernel Q (t , T) of K (t, T), the covariance
r
r
Let us now introduce
the nonhomogeneous
nonhomogeneous
integral
the inverse
of the received
some material
that we need.
Fredholm
is to solve this equation
signal.
from Chapter IV on
From Eq. 4.3
we can write the
equation as
(6. 6)
As we discussed
in Chapter IV, vre can consider
equation specifies
being the result
representation
a linear
operator
of this linear
upon ~(t), with the solution g,(t)
operation.
of this operation
that this integral
We define the integral
to be (Eq. 4. 73).
Tf
g{t) =
5
T
Operating
the integral
Q (t, T)s(T)dT,
r
(6. 7)
-
o
upon ~(t) with Qr(t, T) to find g(t) is equivalent
equation by means of our differential
It is easy to show that the inverse
kernel
satisfies
to solving
equation approach.
the following
169
integral
equation in two variables
Tf
S
(6.8)
Kr{t, v)Qr{v, T)dv = Io{t-T),
To
Let us multiply both sides of Eq. 6.4 by Qr(T,Z) and then
integrate
with respect
to T.
This yields
~
~
S
T
=
K (t,T)CT(T)Q (T,z)dT
x
r
-
-
S
T
o
.~
h(t, v)
0
SK r
T
(V,T)Q (T,z)dTdv
r
-
0
T
f
=
S
hJt,v)Io{v-z)dv
=~t,z),
To:St,
z:STr
(6.9)
To
We are not directly
of the optimum estimator.
interested
What we really
~(t), which is the output of the estimator.
substituting
in the impulse
response
want to find is the estimate
We can obtain this by
Eq. 6.9 into Eq. 6.2,
(6. 10)
~(
We should observe that the inverse kernel can be shown to be
i ,e. Qr(t, T) = QI{ T, t); therefore,
we can define it as
symmetric,
a pre - or post- multiplier operator.
170
Thus,
the optimum estimate
is the result
of two integral
operations.
We now want to show how we can reduce Eq.
6. 10 to two differential
equations
conditions.
with an associated
set of boundary
1\
estimate
x(t) is spe cifie d implicitly
by their
The
solution.
Let us define the term in parenthesis
in Eq. 6. 10 as
B'.r(T), so that we have
T
f
(6. 11)
Iir(Th=S Qr(T, z)!:.(z)dz,
-
Substituting
T o
this into Eq. 6. 9 gives us
"x(t)
Observe
(6. 12)
that Eqs.
6.10 and 6.11 are integral
encountered
in Chapters
can convert
each into two vector
associated
IV and II, respectively.
set of boundary
~(t) is replaced
Fredholm
by £(t).
differential
of the type
Consequently,
equations
we
with an
conditions.
From our previous
the nonhomogeneous
operations
From
discussion,
integral
g (T) is the solution to
-r
equation when the signal
Chapter IV, we have
(6. 13)
171
where ~(T)
is the solution to the differential
equations,
(Eqs.
4.10
and 4.12
(6. 14)
(6. 15)
The boundary' conditions
are (Eqs.
4.13 and 4. 14)
(6. 16)
(6. 17)
In Chapter II we found that the integral
by Eq. 6. 12 also has a differential
operation
equation representation.
given
If in
Eq. 2.22 we set
f(t) =
-
a
...9r
(t),
and then substitute
(6. 18) .
Eq. 6. 13 for [r(t),
found by solving the differential
"dt
dx(t)
=
"
F(t).!(t)
+ G(t)Q
we find that x(t) can be
equations
T
G (t)!l2(t),
To:5 t
:5
Tf
(6. 19)
172
(6. 20)
The boundary
conditions
for these equations
(Eqs. 2. 33 and 2. 35)
(6.21)
(6. 22)
Upon a first inspe ction it appears
four coupled vector
boundary
differential
conditions.
However,
equations
(Eqs.
with their
if we examine
we find that Il 1(t) and ~2(t} satisfy
Since both equations
that we need to solve
Eqs.
associated
6.15 and 6.20,
the same differential
have the same boundary
condition
6.17 and 6. 22), they must have identical
equation.
at t
solutions.
=
Tf
Therefore,
we have
(6. 23)
By replacing
Il j (t) and .!l2(t} by E,(t} in Eqs.
It.
6. 20, we see that i(t} and ~(t) satisfy
equations
conditions
(Eqs.
(Eqs.
6. 13, 6. 15, 6. 18 and
the same differential
6. 14 and 6. 19) and have the same boundary
6.16 and 6.21).
Therefore,
we must also have
173
A
= ~(t),
~(t)
Consequently,
equations
(6. 24)
we have shown that two of the four differential
are redundant.
We finally obtain the state variable
optimum linear
differential
smoother.
The optimum
representation
estimate
of the
~(t) satisfie s the
equati ons
"
dx(t)
Cit
=
"
T
F(t)~(t) + G(t)Q G (t):e.(t),
dp(t)
dt
=
T
-1
C (t) R (t)C(t) ~Jt)
(6 .25)
T
T
1
F (t):e.(t) - C (t) R - (t).r(t) ,
(6. 26)
where we impose the boundary
1\
x(T )
-
0
=
P
conditions
(6. 27)
p( T ),
0 -
0
(6. 28)
The smoother
known.
realization
It was first
specified
derived by Bryson
be assuming
Gaus sian statistics
to maximize
the a posteriori
When we compare
that specified
by Eqs.
6. 25 to 6. 28 is we 11
and Frazier
in reference
and then using a variational
probability
14
approach
of the state vector.
these equations
to those in Chapter
our solution to the nonhomogeneous
Fredholm
IV
integral
174
equation,
we observe that they are identical
in form.
The major
difference
is that our input is now a random
process
!:.(t), whereas
before we had a known signal ~(t).
The result
of this observation
is
that the solution methods developed in Chapter IV, Section C are also
applicable
(In fact,
to solving the above estimation
the methods presented
the literature
there
equations
for the smoother.
were originally
for solving these estimator
equations.
developed in
We have
expl.oited the above identity
of form to solve the equations
we derived
In order to make use of these methods,
in Chapter IV.
it
is obvious that one must identify !(t) with ~(t) and E.(t) with !l.(t).
Since we shall need the results
useful to relate
filter
the smoothing
structure.
derived
To do this we shall review
ships that we derived
suggested
structure
in the next section,
in Chapter IV.
above,
the variable
the realizabl~filter
estimate,
S
-r
it is
above to the realizable
some of the relation-
If we make the identity
(t) is easily
~ (t).
seen to correspond
to
It is well known, or it can be
-r
seen from Eq. 4.39 , that ~ (t) satisfies
the equation
-r
(6. 29)
where ~t/t)
d~(
satisfies
!-)
t
dt
=
the variance
F(t)~(~)
equation (Eq. 4.34).
+ FT(t)~(~)
-
~(i
)CT(t)R-l(t)C(t)~f)+G(t)QGT(t)
(6. 30)
~
t
roWeshall use the n,otation~(t/T)
and ~("T) inter-change ab ly, Both
symbols denote the covariance of error at time t (2: To) when we have
observed the signal r( T) over the interval [T ,T].
-
0
175
We have not demonstrated
matrix of the realizable
therefore,
directly that ~(t/t)
filter.
is indeed the covariance
It is straightforward
to do so,
we omit it.
The smoother
estimate
estimate ~ (t) in two ways.
Quite obviously,
-r
at the end of the interval
~(t) is re lated to the realizable
the estimate
correspond
as stated by Eq. 4.37
(6.31)
In addition we have the important
as expressed
relationship
throughout the interval
by Eq. 4.43
(6. 32)
We shall often exploit this relationship
in the remainder
of the
chapter.
There is one important
structures.
~(t/t),
In the realizable
this covariance
B.
filter,
was implicit in the filter
corresponding
Covariance
covariance
contrast
the covariance
structure.
~(t/Tf)
between the two
is not.
of error,
In the smoother,
the
Deriving an equation for
is the topic of our next se cti on.
of Error
for the Optimum Smoother
In this section we shall derive four matrix differential
equations,
each of which specifies
smoother.
Since this is a rather
briefly to outline our development.
the performance
long section,
of the optimal
we shall pause
176
First,
smoother
error
we shall derive differential
and the realizable
evaluate three expectations
and the noise processes
filter
of error,
error
~(t) and ~(t).
for both the
processes.
which involve these error
shall then derive the four differential
covariance
equations
We shall
processes
Using these expectations
equations
we
that specify the
~(t/T f)' of the optimal smoother.
We shall
point out and correct some errors that have appeared in the literature
14,15
on this topic.
Finally, we shall suggest an algorithm for
finding the steady state covariance
~oo'
by using some of the results
smoother
performance.
that we have derived for the
point for our analysis
-
and the realizable
consider the smoother
the differential
=
F(t)~(t)
r
process
for the
filter respectively.
First,
we
~(t) as generated
by Eq. 2. 1
error.
We note that the process
dx(t)
dt
is finding the
equations for e (t) and e (t), the error
optimal smoother
satisfies
of the realiz able filte r ,
Let us now proceed.
The starting
differential
of error
equation
+ G(t)~(t),
T
o
:S
t.
(2. 1)
(repeated)
We define the smoother
estimation
error
to be
(6. 33)
177
If we subtract
satisfies
Eq. 2.1 from Eq. 6.25,
the differential
we find that the error
equation
(6.34)
We can also w-rite Eq. 6.26 in terms
substituting
Eq. 6.1 into Eq. 6.26,
of the error,
~(t).
By
we obtain
(6.35)
We can also find the boundary conditions
and 6.35.
We still have at t
=
for Eqs.
6. 34
T
f
(6. 28)
(repeated)
To find the initial conditions,
€ (T
-0
However,
) - (-x( T ))
-00-0
=P
let us consider
initial state.
0
We have
p(T ).
-x(T ) is the a priori
-
Eq. 6.27.
error,
From our as sumptions
(6. 36)
e ,in the estimate
-0
of the
in Chapter II, this a priori
178
error
has zero mean and a covariance
!!(t) and ~t).
"e(T)
-
Consequently,
0
-e
-0
=P
of P
error.
(6. 37)
0
1\
e (t) = x (t) - x(t),
-r
If we substitute
-
of
6.36 as
p(T).
0-
We define this error
-r
and is independent
we may write Eq.
Let us now find a differential
filter
o
T
o
equation for the realizable
to be
-s t ,
(6. 38)
Eq. 6.1 into Eq. 6.29 and then subtract
2.1,
we
find
T
We ne ed to spe cify the initial
assumed
zero
"-
x (T )
-r
0
Therefore,
. a priori
a
priori
o
:::;t,
condition of Eq. 6. 39.
(6. 39)
Since we have
mean for x(T ), we have
-
0
= -O.
from Eq. 6.38 the realizable
(6. 40)
filter
error
at t
=
T
o
equals the
error
(6.41)
179
Finally,
filter
error.
we can re late the smoother
By using Eq. 6.32,
1\
1\
x(t) - x (t)
-
-r
=
1\
,...
-
to the realizable
we can write
(x(t) - x(t)) - (x (t) - x(t))
-
error
-r
-
=
(6.42)
(Since
= -0
E ,Eq.
= T o .)
(T')
€
-r
0
evaluated
at t
6.37 is obviously
Let us now summarize
a special
the important
case of Eq. 6.42
equations
that we
shall use in our analysis.
1.
augmented
Writing the smoothing
vector
form,
~(t)
d
dt
equations
6.34 and 6.35 in
we have
F(t)
=
CT (t) R- l(t) C(t)
G(t)!f(t)
=
CT(t) R -l(t)~(t)
G(t)u(t)
W(t)
(6.43)
180
where we can identify the coefficient
previously
by Eq. 4. 16.
matrix
The boundary
as it has been defined
conditions
for this equation
are
€(
-
T ) - €
0
-0
=
P p(T )
0-
(6.37)
0
(repe ate d)
(6.28)
(repeated)
where €
-0
is the a priori
2.
-i-
error
of the estimate
The realizable
§..r(t)
= (F(t)
filter
error
of the initial
is the solution to Eq. 6.39
- ~(t)t)CT(t)R-1(t)C(t))~r(t)
_~(})CT(t)R-1(t)W(t),
state.
- G(t)~(t)
To < t,
(6. 39)
(repe ate d)
where the initial conditi on is
(6.41)
(repeated)
and ~(tt ) is the covariance
3.
are related
of €
-r
(t).
The smoothing error
and the realizable
filter
error
by
(6. 42)
(repe ate d)
In order
performance
to derive the differential
of the smoother,
equations
we shall need to evaluate
for the
three
181
expectations.
We shall simply state the results
these expectations.
The interested
reader
of the evaluation
of
can follow the derivations
which follow these statements.
Expectation
.!.
(6.44)
Pr-oof;
By classical
arguments,
shown to be uncorrelated
given by Eqs.
the received
with the error
6.25 through 6.28,
e (t) for all t , Te [T
-
the random process
.res ult of a linear operation upon the received
it is also uncorrelated
Expectation
withs(t)
signal £(T) can be
signal;
,Tfl.
0
As
pt r) is the
consequently,
for all tv r e] To' Tfl.
.3
, te-=--(t)j'
A(t) = E
.§.~(t)
=
'1r(t,
Tf) --- ..--- - - - -)
To::: t :::Tf'
o
2(t)
(6. 45)
where '1r(t, Tf) is the transition
realiz able filte r transition
Proof:
matrix
for W(t) and $(T , t) is the
f
matrix.
In order to evaluate this expectation,
solution of Eq. 6.43 as an integral
operation.
we need to write the
From Eq. 6.31,
we
have
(6.46)
182
With Eq. 6.28 this provides
a complete
set of boundary
for Eq. 6.43 at the endpoint of the interval.
condition
Therefore,
we can
write
~(t)
=
Let us now substitute
A(t)
~(t,T)
---
_____
zero.
d-r ,
.-J
T
-1
C (T)R
(T)~(T)
t
-r
Since
of U(T) and wtr) for T>
--
Therefore,
t,
we have
e(t)l
-=-- •.
t
G( T)U( T)
this equation into the above expectation.
is identically
=E
The expectation
+
e (t) is independent
error
one of the terms
S
f
0
EJt)
the realizable
T
~r(Tf)
'1!(t, Tf) ---- -
= ~(t, Tf)
~';{t)
EJt) ..
(6. 48)
in this equation can be evaluated
b.y writing e
-r
(T )
f
as an inte gr a1 ope ration,
Tf
~r{Tf)
= ~(Tf' t)~r{t)
-
5
~(T f' T)[ G{T)u{T)
+ ~(~IC
T{T)R -l{T)~{T)]
t
(6. 49)
When we substitute
term vanishes
Eq. 6.49 in Eq.
6. 48, we find that the second
since e (t) is again independent
-r
of U(T) and W(T) for
-
-
d-r ,
183
T:::: t ,
Therefore,
we finally obtain the desired
E (t)
A(t)
-
=E
----
T
E
-r
(t)
=
result.
$(T , t):E(})
f
w(t,Tf)
E(t)
--------,
To:St:ST
o
r
(6. 45)
(repeated)
Expectation
i
(6. 50)
Let us substitute
B(t)
Eq. 6.44 into the above.
This yields
=E
G(t)~(t)
x
T
(6.51)
C(t) R-l(t) ~(t)
184
We can relate
~r(Tf)
second expectation.
to ~(t) and ~t),
and we can also evaluate
We find'
)e (T )
th(T , T o-r_
0__
f
't'
.
----------Tf
S
, +
-
-
,r_ .._
the
~(Tf
T0
+ :E( ~)e
T)[ G(T)U(T)
- -,.-::-
-
- '-
-
-
-
T(T)R-l(T)~(T)]
-
-
-
-
-
d-r
-
-- ---
o
x
T
+
S
f
~(t, T)
t
(6.52)
In Eq. 6.52 we note that e
-r
consequently,
evaluate
(T ) is independent
0
we can easily evaluate
--
the first term.
the second term we need to split the impulse
evaluated
the desired
at
T
=
t, the endpoint of the integration
result
In order to
since it is
region.
This yields
of Eq. 6.50.
With these expectations
equations
of u{t) and w(t);
for the covariance
us proceed by differentiating
of error
we can now derive the differential
of the optimal smoother.
the definition
of the covariance.
Let
We
185
want to evaluate
(6. 53)
Since the terms
consider
in the above are transposes
only one of them.
Substituting
of each other,
Eq.
6.34,
we shall
we have
(6.54)
where we have used Expectations
term we have the first differential
1 and 3.
Adding the transpose
equation for the smoother
covariance.
Differential
Equation
.!. for
:E(t/Tf)
:;'~
'We use the notation :E(t/Tf) to indicate the covariance of error at a
time t( :=:: To) where we have obse rved the signal r( T) over the inte rval
To;::::T;::::T
r
186
T o ~t~Tf'
There are two difficulties
with this expression.
term is difficult to evaluate.
over long time interval
*
First,
Secondly when integrated
it becomes
We shall now derive
for ~(t/T f) which eliminates
..
(6. 5 5)
the forcing
backwards
unstable.
a second differential
equation form
these difficulties.
If we solve Eq. 6.42 for E(t) we obtain
p(t)
=
~-1 ({-
Now we substitute
(6.56)
)(~Jt) - §..r(t)).
for .E(t) in Eq. 6. 34 (this is similar
to the approach
of Method 3 in Chapter IV) we find
(6. 57)
-G(t)~(t) ,
Let us now substitute
Eq. 6.57
2 and 3, we have a cancellation
in Eq.
6.53.
of terms,
If we us e Expectations
and we obtain the desired
result
*A relationship
of this form was first
1966 WESCON Proceedings.
derived
by Baggeroer
in the
187
Differential
Equation ~ for ~(t/Tf)
(6. 58)
Several comments
are in order here.
first derived by Rauch, Tung and Striebel
analysis
error
w'ays.
and then passing to a continuous
for the realizable
Obviously,
In addition,
estimate,
its inverse
it supplies the required
using a discrete
limit.I5
~(t/t) enters
appears
This formula was
time
The covariance
of
Eq. 6.58 in two
as part of the coefficient
boundary condition at t
=
terms.
Tf, for
(6.59)
Consequently,
Finally,
we can solve Eq. 6.58 backwards
we discus sed the stability
in Chapter IV.
there,
i. e.,
of equations
in time from t
Eq. 6. 58 will in general
for long time intervals
when done this way.
backwards
be stable.
from t
Tf.
with th is coefficient
We can reach the same conclusions
when integrated
=
=
here as we did
Tf' the solution to
Consequently,
'doe s not cause any numerical
obtaining solutions
difficulties
188
We can derive
~(t/Tf)
First,
a third differential
by using the covariance
we relate
equation for finding
rr (t/Tf)·
of p(t), which we denote by
this covariance
to ~(t/Tf).
We can rewrite
Eq. 6.42
in the form
(6. 60)
\/)
Post multiplying
Eq. 6.60 by its transpose,
value of the result
and using Expectation
taking the expected
1 gives us
(6.61)
By using an analysis
similar
to that used in deriving
form 2 for
~(t/T f)' we can find a dlffer-ent ia le quati.on for 1f(t/T
f)·
This
gives us our third form.
Differential
Equation ~ for ~(t/T f)
(6.62)
where
Tf
(t/T ) is related
f
is identically
zero,
to ~(t/Tf)
by Eq.
the boundary condition
6.61.
at t
=
Since E.(Tf)
Tf is obviously
189
(6. 63)
Again, some comments
integrating
discussed
backwards
earlier
already available,
are in order.
and the stability
form 2 since taking an inverse
In addition,
if ~(t/t)
to use this form rather
matrix
of
of the solution still apply as
for form 2 still apply.
it may be better
The same issues
is
than
at all points in the interval
is
not required.
The first three differential
equations that we have derived
are not well suited for finding analytic solutions
in the case of constant parameter
systems,
for ~(t/Tf).
the matrix
differential
equations have time varying coefficients.
We can eliminate
difficulty by considering
differential
a 2n x 2n matrix
than an n x n one as we have considered
need to c ombirie forms
before.
2 and 3 in an appropriate
One can show by straightforward
Eqs.
6.58,
6.61
Even
this
equation ra the r
To do this we
manner.
manipulation
of
and 6.62 that the following matrix diffe rentta l
equation is satisfied.
Differential
Equation!
for ~(t/T f)
G{t)QGT{t):
0
,
-----,--------
o
')
: (CT{t)R-1{t)C{t)
(6.64
a)
190
where W{t} is defined by Eq. 4.15
I
1
and P{..!...)
Tf
_ 1T{..!... ):~~{!.)
Tf
t
I
--- ---
.A
--1-- - -I
-~{!. }1T {...!.. } I
Tf
t
We can specify the boundary condition
---
{6.64b}
t
-)T{-}
Tf
1
I
at t
=
T by using Eqs.
f
6.59}
and 6.63)
Tf
~(rr)
T
f
P{T}
f
=
---
f
I
I
I
""i
(6. 65)
1
0
1
I
where ~{Tf/Tf} is the realizable
Consequently
the interval
0
0
filter
error
at Tf·
we can solve Eq. 6.64 a backwards
using this boundary
condition.
method 2 that we developed in Chapter
This is analogous
We have,
30
representation
to
IV.
ffwe extend the concept developed
find an integral
over
there,
for the solution to Eq.
it is easy to
6. 64
----------------------
191
T
~(-!)I
Tf I
0
I
---4--I
o I
G(T)QGr.r(T)
_______
0
I
o
0
,
,
1
T
I C (T)R
-1
wT(t,T)dT,
(T)C(T)
(,;
f
'.,"-
(6.66)
We should point out that solving for ~(t/T f) using this
for m doe s have certain
advantage s when the system parameters
constant.
the coefficient
matrix,
In this case,
an analytic advantage.
that we have a larger
deal.
Finally,
coefficient
problems
dimensional
solutions
matrix,
This is
we should remember
set of equations with which to
of ~(t/T f).
Since it involves W(t) as a
it introduce s virtually
the same stability
that we had for method 2 in Chapter IV.
This completes
forms for ~(t/Tf).
of each.
However,
exponential.
we observe that this form is not well suited for
finding numerical
our discussion
Before proceeding,
Form 1 was the starting
it was relatively
In addition,
of the differential
we shall discuss
equation
the merits
point of our derivation.
simple in appearance,
difficult to evaluate.
Forms
matrix W is a constant
which allows us to see the matrix
certainly
are
Although
the forcing term for it was
it introduced stability
2 and 3 were well suited for numerical
problems.
procedures,
since they
192
were stable when integrated
backwards
of the interval.
they were not well suited for analytic procedures
time varying coefficients.
methods,
with constant parameters.
matrix
was a constant for systems
It, however,
when used numerically
also introduced
stability
over long time intervals.
Several errors
have been made in the literature
11'
(tiT f) .14, 11£ we apply the results
the covariance
papers,
because they involved
Form 4 was well suited for analytic
since the coefficient
problems
However,
of p(t),
regarding
of these
we can quickly show that they imply
(6.67)
which is clearly
impossible
for a covariance.
We shall now correct
The differential
equation of 6.64,
the se errors.
14
was first derived by Bryson and Frazier.
erroneously
interpreted
lower right partition
-1T(t,T )
f
should be 1f(t/T
f
which,agrees
interpret
the partitions
partitions
are actually 1f(t/T ):z(t/t)
f
of P(t).
>;
results.
they
They state that the
we have seen that it is
as E [~(t)pT(t)].
However,
These
as we have indicated.
to find :z(t/T ),
f
their incorrect
of p(t) leaves their derivation
.
They also
From
zero.
for P(t/rf) however is :z(t/T )
f
we naively apply their results
covariance
However,
1, we know that this is identically
upper left partition
correct
f
with what Eq. 6.67 would indicate.
the off diagonal partitions
expectation
form 4, for ':z(t/T )
Their
Consequently,
if
we would obtain
assertions
rather
regarding
suspect.
the
193
Rauch, Tung and Striebe1 state that the error
with p(t) which is incorrect.
:E(t/T ) and
f
1\ (t/Tf)
I5
~(t) is correlated
They also have not related
correctly,
equation for :E(t/T ) is correct
f
as indicated
by Eq. 6.63.
as we have discussed
Their
previously
form 2.
Equation 6.58 also provides
between the ralizable
and unrealizable
the class ical Wiener approach.
the system parameters
is large.
If we are
errors
relationship
when calculated
This corresponds
are constant
somewhere
an interesting
to the case when
and the time interval
in the "middle"
using
[ To, Tfl
of the interval
d:E(~ )
f
dt
-
Denoting the realizable
error
To «t«
0,
(6. 68)
Tr
Wiener error
by :Eooand the unrealizable
by :E(t/oo), we obtain
This equation has an interesting
determine
however,
a linear
:E(t/oo)is easy to
implication.
since it is the unrealizable
covariance;
I
:E:o is usually difficult to determine.
matrix
are several
equation for finding :E00 (really
Eq. 6.69) provides
:E-~) .
ways of solving this linear equation,
Notice the linearity
of the equation eliminates
sol utions which arises
Since there
:Eoofollows directly.
the ambiguity
of
by setting the Rica tti equati on to steady state.
This completes
our discussion
of the performance
of
194
the optimal smoother.
C.
Examples
Let us now illustrate
it with several
examples.
of the Optimal Smoother Performance
In this last section we analyzed the performance,
covariance
of error,
shall illustrate
First,
of the optimal smoother.
this performance
analytically.
integrating
for first
We shall apply Eqs.
shall consider
the analysis
In this section we
by considering
we shall work two examples
or
several
examples.
order systems
6.64 and 6.65 to do this.
Then we
of a second order system by numerically
Eq. 6. 62 and then applying Eq. 6.61.
Example 6. 1 - Covariance
of Error
for ~ Wiener Process
In this first example we shall find the covariance
error
for a Wiener Process
additive white noise.
described
in Eqs.
that is received
in the presence
The system for generating
and 3.36.
3.35
of
of
this proce ss is
We repeat the system
matrices
here for convenience
F
=
0
C
G
=
=
1
P
Q
0
=
=
CT
(repeated)
and the observation
interval
and T
=
results
from previous
result,
we can find the solutions
f
T.
Fortunately,
(3.36)
0
1
In addition we assume that the spectral
noise is
~
height of the additive white
is [0, T], i.e.,
--
we have available
examples
T
0
0
many of the required
which concern the process.
rather
=
quickly.
As a
195
The first step is to find the matrix W(t) and its
associated
transition
the results
are indicated by Eqs.
substitute
t -
T
matrix
'1!(t, T).
We did this in Chapter IV and
4.51
and 4.56
.
(We need to
for t as the system has constant parameters.)
we need to find ~(T/T).
'If(t, T) as indicated
We do this by using the partitions
by Eq. 4.31.
Next
of
This yields
(6. 70)
Consequently,
Eq. 6.66 becomes
o
o
T
S
1
'¥(t, T)
t
,
0
T
I
'If (t, T)d
----i---o
I
I
I
o
T,
0< t < T.
2
-'=<r--
where 'If(t, T) is as defined by Eq. 4.56 as indicated
our the upper,
str aightforward
left partition
(6.71)
for ~(t/T),
manipulati on
above.
we find after some
Separating
196
(6.72)
Example
6.2 - Covariance
of Error
Let us now consider
process
generated
this process
matrices
are repeated
by Eqs.
F
=
-k
C
G
=
=
1
P
o
=
1
=
S
noise to be
(2.17)
(repeated)
of this representation.
that the process
generated
In addition,
(J"
available
4.62
(t-cr) for t.)
using Eq. 4.31 the realizable
covariance
is stationary,
interval
from previous
The matrix W(t) and its associated
'1i'(t, T) are given by Eqs.
Instead
examples.
matrix
(Again,
Next, we need to find ~(T IT).
of the initial
let
to be [0, T] .
transition
and 4.68, respectively.
filter
of
we chose the level of the
and the observation
Again, we have many results
an arbitrary
of
The state
2kS
a free variable.
need to substitute
The generation
2.16 and 2.17.
a slight variation
choosing Po =Psuch
observation
for a random
here for convenience
Q
it remain
the performance
by a system with a pole at -k.
is described
Let us consider
for ~ One Pole Process
covariance
state,
of error
By
~(T IT) for
P , is given by
o
we
197
(6.73a)
where
48
kN
~=
(6.73b)
o
For this system,
Eq. 6.66 becomes
T
I
~(T)
t
P(T)
=
T)
~(t,
-
-
o
T
S
~(t. T)
t
where ~(T/T)
-2k8
__ ~I
f
o
~
I
- - 1- I
I
-
-
0
OJ ~(t,
T)dT,
ostsT.
(6.74)
o , .!.
l
0"
is defined above by Eq.
6.73a.
Taking the upper left
{
portion for ~(t/T)
:E(~)
we find after some straight
= [ cosh [kA(T-t)]
~( ~)cosh[
+ sinh[~>"(T-t)]
k>..(T-t)) + (~
'i.)-
28)
forward
J
manipulation
x
strh [~>"(T-t)D
• 0:5 t :5 T.
(6.75)
198
In order
to illustrate
Case a - P
--
-
-0
= -S
this result
(Stationary
let us consider
process.
choices
of P .
o
Process)
Here we have chosen P
stationary
three
such that we are estimating
o
a
In this case
(6.76)
If we substitute
t
~(T)
_
T {(COSh[kA.(T-t)]
- ~( T)
Therefore,
respect
this equation into Eq. 6.75 , we can show that
+{-sinh[kA.(T-t)])(cosh
chos en k
=
Case b - p!
--
-
of error
to the midpoint of the interval.
quite easily.
-0
In Fig.
1,
CT
=
=
Le.
O<t<T.
(6. 77)
is symmetric
with
We can easily
=
1 and T
=
-----
T
~-;I;)
=Po = A.
2S
+
1
where we have
Filtering
----
In this case we have chosen the initial
that we do not gain any improvement
results,
2.
2S /A.+l (Steady State Realizable
--
show that they
the Wiener filtering
6.1 we have plotted Eq. 6.77
1, S
)
cosh] kA.T] + X sinh [kA.T]
we see that the covariance
approach the large time results,
[kA.t] +{- Sinh[kA.t])}
1
by realizable
Error)
covariance
filtering,
such
~
(6.78)
199
Fig. 6. 1
~(t/t) and Zt/T) for a
First Order Process
Pe = 1
I.
Stationary Process
.s
200
for all T.
Substituting
this equation
= E.A.+ s '()...
A.+
~(--!-)
T
)...2~1 {. ~(1+
{-)
I)
1
e
into Eq.
-2k(T-t)_
6. 7
we find
-
+ ~(l-{-)e -2k(T-t)}
• 0 ~ t ~ T.
(6. 79)
Therefore,
we see that the performance
move backwards
filter
error
behavior
from the endpoint.
as we could calculate
near the endpoint,
,
i ,e.,
--
performance
which the realizable
delay before
making
in detail
for the choice
Case c - P
--
-
- 0
.:.----
---
Let us consider
~( .:!:..
) =
T
near
filter
T, reflects
is the unrealizable
theory.
The
the gain in
(We shall develop
some
this concept
6.2 we have plotted
Eq.
6.79
figure.
State)
-----'-
the case when we know the initial
2 S sin h] k)...T]
A.c os h[-=-k"""'A.-=T::1-]-+-s=""in---:-h~[
k:r--:)...:-::T=-or-J
When we substitute
as we
could attain by allowing
in the previous
(Known Initial
a constant
from the classical
In Fig.
of parameters
= -0
This constant
its estimate.
in the next section.)
approaches
this into Eq.
6.75
we find
state
(6.80)
201
Fig.
6. 2
z(t/t) and 2.-(t/T) for a
First Order Process
Po
=
2S / (
'A +
1)
=
Uf
Lt'
11
e,
t
iF
-t-
1..
202
kxt] (cosh] kX.(T-t)] + ~ Sinh[kx.(T-t)])1
1
c osh( kAT) + ~ sin h] kAT]
28 (sinh[
A
=
~(4)
' 0 < t <T.
(6. 81)
In Fig.
5.3 we have plotted ~(t/T)
We can see that after.
information
regarding
our estimate.
interval
the initial
state
is virtually
quite long enough for this case.
Example
if the interval
~ - Covariance
analytically
dimensioned
~(tlool, the
useless
in making
of Error
in a reasonable
systems,
shall numerically
about all the systems
woeuse numerical
Eq.
this function we find ~(t/Tf)
6.62
In order
methods.
to determine
by using 6.61.
First
to study higher
To do this we
lI(t/T ).
f
though,
that we want to estimate
y(t) which is described
by Eqs.
covariance
function and a spectrum
and 2.19b,
respectively.
example,
that one can do
Given
let us
that we wish to study.
We shall assume
Instead
in middle
for ~ Two Pole Proces s
amount of time.
integrate
cons ide r the system
There is a plateau
is long enough. ')
We have analyzed
previous
it approaches
6 secs.where
(We should point out that 'we did not take the ,observation
of the interval
process
for the same choice of parameters.
of performing
let us consider
2.18
and 2.19
as illustrated
. It has a
in Figs.
the type of analysis
a variate
the stationary
2. 19a
done in the
of a pre - emphasis
probelm.
Since we have a two state
improve
our performance
by including
system,
let us see if we can
the second state,
or the
203
Fig.
,q
6.3
204
derivative
of the message,
in our transmitted
we then have that the output of the system
signal.
In this context,
is
(6.82)
where we desire
to estimate
fair comparison
to simply add the second state,
increase
the transmitted
xl (t) as our message.
power.
2
dX
(E[x I(t)
I
dt
+ 40
(t)
]
=
2
Q!2
=
since this would
Let us, therefore,
power to be fixed to its original level of 4.
4 Q!I
It would not be a
4
0 for a stationary
to see if it can improve
(6.83 )
differe nti able rand om pr oce s s) .
our performance.
the state matrices
DJ
R
=
length is 2.
for our system
02]
=
of Eq. 6. 83
We shall also assume
that the additive white noise level is I and the interval
In summary,
the
We have
we sha ll vary Q!I and Q!2 within the constraint
Therefore,
G
constrain
I
are
205
=
Q
(6.84 a-e)
160
with a 1 and aZ constrained
to be on the ellipse
(6.83)
(repeated)
In addition we choose the interval
T
f
length to be Z, or T
=
o
0 and
= T = Z.
If we proceed
with our analysis
by numerically
integrating
Eq. 6. 63 to find ~(t/T)
for various
value a 1 and a Z that satisfy
Eq. 6. 83 , we generate
the curves
of Fig.
~11(t/T)
(normalized),
the covariance
We can make several
curves.
Fig
Le.,
=
generate
time,
curves
= o.
Secondly,
we do not have a good physical
~ 11(T I T)(normalized),
interval,
this in Fig.
of the interval.
in time.
interpretation
the improvement
6.5.
the realizable
and ~11(T/ZJT),
only one of the states,
with respect
which are exactly inverted
We have summarized
as in
we have plotted the curves
of aZi if aZ is negative
Let us also consider
these
proces s, the curves
about the midpoint of the interval
0, or aZ
for positive values
regarding
a stationary
6. 1 unless the observed signal contains
a1
Here we have plotted
of the message.
observations
Although we are estimating
are not symmetrical
6.4.
the smoother
only
to a 1J we
At the present
for this observation.
in performance.
In ttis figure we have plotted
estimate
at the endpoint of the
performance
These points are very close to their
of the midpoint
asymptotic
206
Fig. 6.4
~11 (t / T) (normalized) for a
Second Order Process with
Trans mitted Signal
207
.to TtFFtnft$rt"um~nm.
rnn
~ffm1TIn:rt:'1~~TII,tfri'I+Ji\~IT}
':It'H';'IT,ITinTI:
~tit~~; FiX:;- tf:,
[T
'i
It
Ifill
I
r
rrfI[nrrUt[:im!HmT:ih~:
1 Ul:
~T,T~~"""""---~"""""----":"";;~-----':-"_':"';";';-------I
;gll,L
I;; ,I'~: ~.~. tl d ;, J 'T
Fig. 6.5
T
Z.ll(T/T)
and £."ll(-/T)
vs,
0( 2.. (modulation coefficient of dx 1(t) / dt )
[or a Second Order Process
,h-i+
.Cob
H'l-l.LI+H'Hl:tM
+++
p
+j
lrt
'j~+
Itt;
fF·
.36
00
+'u-Itr
tt·t
lInt
It±!
lIft
1+
!tIt
ft
~
+•.'.1
I+'H
~.
t-
-+
,1
Hi
-~
,f
t
208
limits,
~ll
00
the derivative
However,
oq.
and ~ll (t/
First,
we see that transmitting
in the signal does not improve the smoother
it can either
significantly.
performance.
degrade or improve the realiz able estimate
Choosing Q!2= . 2 improves
25 percent,
it by 50 percent.
This would indicate from this particular
that this type of preemphasis
with delay),
whereas
our performance
by approximately
(or filtering
some of
(over Q!2=0)
choosing Q!2= -. 25 degrades
proble m
is not useful when doing smoothing
while it can yield significant
improvement
for re alizab le filtering.
D.
Filtering
with Delay
The optimal smoother
tha t in the past and in the future,
particular
smoother
[To,Tfl.
increases,
point.
However,
structure
uses all the available
data, both
in making its estimate
one of the disadvantages
at a
of the optimal
is that it operates over a fixed time interval
Consequently,
as more data is accumulated,
we must resolve
Le.,
Tf
the smoothing equations if we are to use
this added data.
In contrast
to this,
the realizable
that evolves as the data is accumulated.
past data in making its estimate,
filter produces
It, however,
whereas,
an estimate
uses only
the smoother
make s use
of both past and future data for its estimate.
The filter realizable
of both the realizable
with delay combines the advantages
filter and the smoother.
delay before we are required
By allowing a fixed
to make our estimate,
we can find a
filter whose output evolves in time as the data is accumulated
yet it
is able to take advantage of a limited amount of future data in making
209
its estimate.
Let us discuss
the filter
assume that we have received
[To' Tfl.
t
=
We want to estimate
Tf - s , Le.,
variable
interval.
structure
the state vector
- ~ >.To)'
We
at the point·
where the independent
is T f' the endpoint of observation
We note that like the realizable
delay is a point estimator
in more detail.
the signal £(t) over the interval
find ~(Tf-~)(Tf
in our filter
structure
whereas
filter,
the filter with
the smoother
e sttmate s
the signal over an entire interval.
Our approach to finding the filter
forward.
*
We use the integral
method 2 of Chapter
then set t
=
Tf
then differentiate
variable
is straight-
specified
by solution
IV to find ~(t) for the optimal smoother.
~ in this integral
~(Tf - ~) in terms
for the desired
representation
structure
of the received
this integral
estimate
representation.
This gives us
data and the realizable
to find a set of differential
~(Tf - ~).
We
filter~
We
equations
We note that the independent
for these equations is Tf rather
than t, some internal
point
in a fixed interval.
Let us proceed
smoothing equations
with our derivation.
First
we write the
Eqs. 6. 25 and 6. 26 in an augmented
matrix
form,
*This
approach to the problem
1966 WESCON Proceedings. 1'.7
was first used by Baggeroer
in the
---
...
..-----
210
[
=
From
WIt)
~(t)
J
t
---------,
o
[~(~)J-
J
(6.85)
C T (t)R -l(t)r(t)
To:=:t:=:Tf
Eq. 4.41 the solution to these equations
in an integral
form is
(6. 86)
where
x
-r
(T ) is the realizable
f
interval
Tfo
Tf-Da>
To'
filter
Let us evaluate
r
----~(Tf-Da)J
-- - =
'l{(Tf-
output at the endpoint of the
Eq. 6. 86 at t
tRr(Tf)j
b., Tfl ~ - - - -
p(T - Da)
f
+
0
=
Tf - Da, with
rJ
~
ili(Tf-.6o, T) -;
S
T - Da
f
--
Q
(T)R IT)r(T)
(6.87)
This is the desired
time variable
integral
representation.
involved is Tf"
equation that Eq. 6.87 satisfies,
is T the endpoint of an increasing
f
internal
We note that the only
Let us now determine
time in a fixed interval.
a differential
where the independent
interval
rather
Differentiating
variable
than t , some
Eq.
6.87
]
~ - - - - d-r
we obtain
211
(6. 88)
It is a straightforward
exercise
to show
We also have
(6.90)
Let us substitute
these two equations
d~ r{Tf}/dTf from Eq. 6.29
.
plus the expression
We obtain
for
212
d~
~~T!=~lW(Tr-bo)
r ~(Tr-
'li"(Tr-bo, Tr)~_r~~rl
L
boJ
0
J
Sf 'li".(Tr-bo, TJ
Tf,-bo
Ie
; - - ~l- - - JdT
(T)R
\
o
~r(Tf)
- W(T ) - - - - -
f
+
+
o
---------
--------
-------
o
o
(6.91)
First,
we identify the term in the first bracket
from Eq. 6.85.
If we write out the term
find that it can be written
as
as
in the second bracket,
we
(T)r(TJ
213
I
Therefore,
w'e finally obtain the desired
differential
equation
d
dT
f
I
(6.92)
The only issue that remains
are the initial
conditions.
A
specify ~(T o) and p(T o} we must solve the smoothing
interval
[T
o
equations
to
over the
,T +~].
0
Let us now see if we can simplify
eliminating
In order
E,<Tf-~}'
From Eq.
our structure
by
6.42 we have
(6.93)
~~
This equation was first
Proce edings. Jb
obtained by Baggeroer
in the 1966 WESCON
�----------------_-:=.--.::._~==------~:;;;::;=:::;;;:::::::::::::::==~=======""
214
Substituting this into Eq. 6.92 we find
(6.94)
One of the difficulties
calculation
of the coefficient
'lEg
.2.!l (T f- .6., Tf)·
'lEg (Tf1'")
matrix
'lEsg(T -.6.,
f
.6., Tf) can be compute~ independent
filter structure.
~(Tf/Tf)
whi ch
lis already
For time varying
to compute this coefficient
Tf)~(Tf/Tf)
matrix
of T
available
systems
r
+
Therefore,
exercise
one
from the realizable
it may be more efficient
matrix by solving a differential
It is a straightforward
the coefficient
Eq. 6.94 is the
For constant paramete r system 'lEgg(T
f- A, T f) and
_
need only evaluate
for it.
in implementing
equation
using Eq. 6.89 to show that
satisfie s the equation
--=_---------------...:...---o----=-------====================~
215
(6.95)
The initial condition follows by setting T
together
with 6.95
using a discrete
=
To
+ c:
Eqs.
6.94
are the same as those derived by Meditch.
time approach.
upon the system,
f
31
We simply;Point out that depending
we may be able to compute the coefficient
matrix
more conveniently then by solving a matrix
differential
The most important
Eq. 6.94 is that it
aspect of implementing
appe ar s to be unstable.
To illustrate
Indeed,
if implemented
equation.
directly
it would be.
where the difficulty lies let us pause a moment in our
discussion.
Let us consider
the linear time invariant
a differential
system
for
with an impulse response
o
h(t)
equation repre sentation
<t<f:::..
J3
>
0
(6.96)
-
elsewhere
216
Certainly,
this system is stable under any realistic
criterion.
The
output of this system is given by
t
y(t) =
S
x(T)e13(t-T)dT.
t - ~
One can easily show that y(t) satisfies
the following differential
equation
(6·98)
Since
(3 is positive,
contradicts
this would indicate an unstable system which
what we had above.
trying to subtract
two responses
yie ld a stable net response.
practical
The difficulty lies in that we are
which are in general
This is not very feasible
unstable to
to do in a
se nse.
The consequence
of our discussion
is that Eq. 6.94
which has the same form as Eq. 6.98 is not suitable for
implementation.
We should manipulate
into an integral for-m
like
Eq. 6.
then realize
this representation
with a topped delay line.
our delay line will have a finite length of ~; consequently,
always realize
it as closely as desired
by decreasing
This ends our discus sion regarding
filter with delay.
Before proceeding
Note that
we can
the top spacing.
the structure
with our discussion
of the
of its
- ,----_--._-----------"'---'~--------_--.::~--.::====================:=217
performance
two comments
to find a differential
Eqs.
are in order.
First,
it is straightforward
equation for p(T f-A) as well as ~(Tf-A) from
6.92 and 6.93.
Second,
we emphasize
how quickly the filter
equations have been derive d from the smoother
structure
by using
our technique.
E.
Performance
of the Filter
of Delay
In this section we shall employ the techniques
deve loped to derive a matrix
the covariance
important
equation for ~(T f- A/T f) ,
for the filte r with de lay.
in two respects.
performance
filter
of error
differential
First,
it tells
This equation is
us how much we gain in
by allowing the delay and using the more complicated
structure.
Second, we can find how long the transient
are before we can attain steady state performance.
already derived
many of the required
results
our derivation
work with Eq. 6.58 and derive ~(Tf- A/T f) from it.
with our approach to deriving the filter
use Eq. 6.64 and separate
out the partition
also has an advantage in identifying
effects
Since we have
There are two ways in which we can proceed.
consistent
that we have
is short.
We can
Howeve r , to be
structure
we shall
for ~(Tf- A/T ).
f
some terms
This
in a form which is
easily to compute.
To proceed
representation
let us work with Eq. 6.64,
for the solution
have for Tf - A > To
of Eq. 6.66.
the integral
Setting t =T f - A we
218
T
T - D..
f
P( T
)
f
I
-D..
~ (f
I
)
I
1.
Tf
=
----T
T - D..
_ ~( __f)1T(
f
)
Tf
Tf
-1T(
T
-D..
f
Tf
I
I
T
):E( _f )
Tf
T
I
-1T(
f
I
=
- D..
)
T
f
I
G(T)QG T(T)
--
---
-
I
_..L. -
o
I
J
--
-
-
--
-
T
-1
C (T)R
(T)C(T)
T
'1r (T -D..,T)dT
f
•
(6. 99)
Let us now differentiate
to use Eqs.
this expres sion with re spect to T f"
6.89 , 6.90 and 6. 29.
F(Tf)~(Tf
TTl
)+~(Tf)F
f
f
Doing this we obtain
T(Tf)+G(Tf)QG
TTl
-~(-.i)G T(Tf)R -1( Tf)C(Tf)~(
Tf
T(T )
f
-l.)
T
f
I 0
I
I
I
I
+'1r(T -.6, Tf) - - - - - - - - - - - - - - - - - - - - - -:f
010
- -
We need
219
G(T -.6)Q GT(T -.6) )
r
+
r
--------1--o
To reduce
this equation,
0
I
-------
I T
-1
I C (Tr-.6)R
(T -.6)C(T -.6)
r
we ftr-st note that
r
220
T
f
:E(T )
0
f
(6.101)
o
0
After we combine
terms
with common factors
and use Eqs.
6.99
and 6. 10 1 , we have
T
f
- b.
dP,( -~)
f
dT
f
=
T - b.
f
W(Tf-~)P(-y-}
f
G(T _~}QGT(T
f
+
P(
f
- b.
T
T
)W (Tf-~)
f
o
I
the se cond term
I
IC
-
o
-~) I
-t----- ---"-----
+ ---------
Simplifying
f
T
T
(Tf-~)R
on the right,
-1
(Tf-~)C(Tf-.6)
we have finally
(6.102)
221
G(Tf-~)QG T(Tf-~) 1
0
I
-------~-+------------
+
I
IC
0.
T
-1
(Tf-~)R
(Tf-~)C(Tf-~)
I
(6.103)
We see that with the exception
is very similar
to the corresponding
To find the initial
form(s)
0+~).
+~
performance
we nee d to solve- one of the
equations
for ~(T
o
IT +~)
0
and
Doing this we find
To
:E( T +
~ )
0
T
0
P( T +LS)
the final equation
one for the smoother.
condition P( TJro
of the smoother
-rr (T o/T
of the added term
= ------r
0
T
0
Po1T(T+~)
0
1
I
I
I
T
0
- Tf( T +LS )P 0
0
---
--
---
- Tr(
T
T ~LS)
I
I
I
0
(6.104)
PAGES (8) MISSING FROM ORIGINAL
223
Let us n?w consider
Eqs.
the upper le ft partition
for :E(Tf- ~/T f) .
Using
6.99 and 6.61 with t = T - ~,. we find
f
(6.105)
To find the initial
to solve the smoother
condition ~ T /T +~) we again need
equations
o
over the interval
can again identify the coefficie.nt matrix
W (T f- .6/T f}that
gll
we had in the filter
evaluate this by one of two ways.
together
with Eq. 6.95 was first
One interesting
equation unless .6
=
0
[T
o
, T +.6].
0
Wgg (T - .6/T ):E(T /T }
f
structure.
f
f
f
We
+
As before we can
We finally note that Eq. 6. 105
derived
aspect
by Meditch in Reference
of Eq. 6.1<0 is that it is a linear
0 whereupon it becomes
equation for :E(Tf/Tf} as would be expected.
the matrix
Riccati
The second aspect is
31.
224
that it is unstable when integrated
a backwards
integration
forward.
We, therefore,
as we have done in previous
do this one will need to solve the smoother
variance
entire interval
of interest.
F.
of Per-for-mance of the Filter
Example
Let us illustrate
an example.
the results
To do this 'we numerically
The coefficient
suggest
problems.
To
equation over the
With De lay
of the previous
integrated
section with
Eq. 6.
matrix was evaluated by using the matrix
exponential.
The example that we shall study is a one pole process
with the initial covariance
is generated.
matrix chosen such that a stationary
The e quationsthat
are Eq. 2. 16 and Eq. 2. 17.
~(Tf- I:1/T ) that results
f
T - 1:1 for various
f
equation;
a delay.
error,
therefore,
from our numerical
covariance
0, which is
by solving the Ricatti
We have also indicated the lower limit on the covariance
as calculated
The curves
from the classical
length considered.
We have previously
The curves
no longer remain
as that of the realizable
we integrated
filter.
over the
Eq. 6.92 forward
indicated that this equation is unstable
in this direction.
more time constants,
The transient
limits very closely
Finally,
of
Wiener theory.
should be made.
approach the asymptotic
when integrated
several
=
we can quickly see how much we gain by allowing
behavior has about the same duration
in time.
of the process
tntegr ati on vs.
The top curve is for 1:1
as calculated
Several observations
interval
the generation
In Fig. 6. 6 we have plotted the
values of 1:1.
the re alizable filter
describe
process
If we plotted these curves
we could see this instability
over
entering.
constant as they start to gr ow exponentially.
225
..-,
Fig. 6.6
Covariance of Error for the
Filter with Delay, ~ {Tf-Il/Tf>
for r.a First Order Process
Stationary Process
.
.
~
I•
226
This oscillation
is even more pronounced
that we have studied.'
This is indicative
could expect if we integrated
describing
G.
the estimation
Discussion
in the second order
systems
of the type of behavior
we
the differential
structure
equation,
Eq. 6.92
directly.
and Conclusions
In this chapter
. smoothing and filtering
we have extensively
with delay.
discussed
linear
As we stated in the introduction,
we feel that our approach is a unified one in that everything
from the differential
equations
.The starting
time Wiener-Hopf
the smoothing differential
using our results
and IV.
equations
for the optimal smoother .
point for our development
equation.
We presented
equations
We then derived several
performance,
we discussed
and performance
results.
of filter
performance
examples
amounts
The smoother
and/or
in problems
of the smoother
with delay.
Both its structure
from the smoother
instability
extensively.
with delay be presenting
for various
II
for the smoother.
difficulty in implementing
a way to avoid this difficulty;
limited to the area of estimation
frequently
of error
possible
We suggested
equation by
forms for the differential
could be derived directly
we did not develop the suggestion
discussion
different
the filter
We also indicated
these results.
a method for deriving
the ory developed in Chapters
the covariance
After working several
was the finite
from this integral
for the Fredholm
specifying
follows
howeve r ,
We concluded the
an example
of its
of delay allowed.
filter
theory.
in radar/sonar
with delay are not specifically
For example,
the receiver
quite
has the optimal
227
. smoother
our results
as one of its components.
to realize
12.
Consequently,
this part of the receiver.
not the only application,
Reference
3
for further
discussion
we can apply
Certainly
we refer
to
this is
228
CHAPTER
VII
SMOOTHING AND FILTERING
NONLINEAR
In the previous
MODULATION
chapters
generation
of the random processes
by a linear
system
tation.
FOR
SYSTEMS
we have as sumed that the
of interest
with a finite dimensional
could be described
linear state represen-
In this chapter we shall extend our techniques
the problems
of smoothing and filtering
so as to treat
when the observation
method is a norr-Hnear function of the state of the system.
With this extension we can represent
schemes
and channels of current
interest
many modulation
in communications.
should point out though that this is not the most general
can be incorporated
still require
in the state variable
framework.
problem that
We shall
that our state equation be linear.
Let us outline our procedure.
our model in order to introduce
linearity.
We
We shall briefly
the notation required
review
for the non-
We shall then show how we can use our techniques
to
convert an integral
equation for the optimal smoothed estimate
pair of differential
equations for it.
problem by use of an approximation
smoothing equations
of equations
point.
described
which described
to a
We shall solve the filtering
technique for converting
over a fixed time interval
the realizable
the
to a pair
filter with a moving end-
229
A.
State Variable
Modelling of Non-Linear
In Chapter II we introduced
generation
of random processes
representation.
observed
~(t).
by systems
we shall require
be described
excitation
process
need to assume
process.
by a linear
=
F(t)~(t)
=
E [ x( T )x T (T )
0-
0
state
function of the state vector,
dynamics
of the
equation with a random
conditions.
that the state vector generated
T
E [ u (t) u (T)]
-
state
for the
method to allow the
that the internal
and random initial
Consequently,
dx(t)
ctt
with a linear
Let us modify our generation
As before,
where
the notation required
output, ,;y,(t)
, to be a non-linear
generation
Modulation Systems
However,
is a Gaussian
here we
random
we have
+
G(t) u(t), (linear
(7. 1)
state equation) ,
Q 0 (t - T),
=
(7. 2)
p •
(7. 3)
0
(t) is a white Gaussian
process,
with a "spectral
height" of
Q, and x(T ) is a Gauss ian random vector.
-
0
The non-linear
observe the state vector.
observed
process,
aspect
of this problem
We shall assume
is a continuous,
enters
in how we
that the output,
non-linear,
no memory
or
function
of the state vector,
~(t)
= ~(~(t), t),
(7. 4)
230
Let us also introduce
vector
for the gradient
of
.§.
with respect
to the state
~(t),
asl(x(t)
,t)
-
I
I
asl(x(t),t)I
-
I
a(x 1(t))
a(XZ(f))
-- --- - + - - - ,
C(x(t), t) ~
a
s (x( t, t)
- a(~(t))
~
I
a(x1 (t))
_- - - -
I
-1-
I
-I- - - - -,t
-
- -
-
I
I
I
::(x~(t))
'I
---
I
I
:
1
-1-.--
I
I
•
»:-
-I --
I
aSZ(~(t), t)
; as 1(x(t), t)
I -ar;:-r(-x""'n
(-;-;'t
I
I
- - __
0
I
I
I
I
I
- - - - -1- - - - - I - - - --I - - -- as (x(t), t) I
I
I as (x(t), t)
I
T
Therefore,
in the special
case
of linear
II a~
I
s ystems
s t
o
(t))
n
(7. 5)
C(~( t), t) is
.
.
independent
of ~(t) and may be identified
as the C(t) whic h we have
been using previously.
We can incorporate
operation
in our structure.
observation,
operation
which has a state
non-linear
vector
this is
no memory
operation.
FM modulation,
types of non-linear
If we can interpret
as the cascade
representation
to incorporate
modulation
certain
operation,
the memory
Probably
the modulation,
of linear
system
for its dynamics
we can simply
operation
or
(with memory)
then followed
augment
by a
the state
and then redefine
the most important
where we interpret~t
memory
the
application
as the cascade
of
of an
231
integrator
followed by a phase modulator.
The final aspect of our model is the received
Again, we shall assume
over the interval
£(t)
=
that the signal ~h
[ To' T fl is corrupted
the receiver
signal £(t).
observes
by added white noise,
~(~(t), t) + ~(t),
(7. 6)
where
(7. 7)
with R(t) positive
definite.
Here we must assume
that ~(t) is also
Gaussian.
Let us summarize
assumed
here and in Chapter
signal to be a nonlinear
have assumed
0
assumptions
II.
First,
we allowed the observed
function of the state vector.
that ~(t) and ~t) are Gauss ian random
x(T ) is a Gaussian
-
the difference s between what we have
random vector.
we
processe sand
In Chapter II, we made
regarding only their second order statistics.
can
We/consider the problems of smoothing and filtering
when the state equation is also non-linear.
a d liferent
Secondly,
approach.
density directly
In this approach,
where we incorporate
equation by using a time varying
first done in Ref.
However,
we maximize
the constraint
Lagrangian
and later in Ref.
we need to use
the a posteriori
of the state
multiplier.
This was
232
Finally,
modulation.
class
we have termed
This is simply a convenience.
of communication
systems
B.
of all the systems
Smoothing For Non-Linear
In this section
according
point for our derivation
by Van Trees
Modulation Systems
which implicitly
is an integral
section.
the optimal
The starting
equation for g(t) as derived
using an eigenfunction
From Eq. 5. liD in this reference,
estimate
generate
equations
signal is generated
of the previous
in Ref. 3
the
10
~(t) when the received
to the methods
to incorporate
we shall derive the differential
and their boundary conditions
smoothed estimate
using this formulation.
state vector
involved.
ope ration as a
One can model a large
and channels
One simply has a large augmented
dynamics
the observation
expansion
it is necessary
x(t) satisfy the following integral
approach.
that the optimal
equation
Tf
~(t)
=
S
T
K~(t, T)C T(~(T),
T)R -1 (T)(£(T) -~(~( T), T)d T,
To ='= t ='= T f
o
(7. 8)
We now observe
the integral
operation
only difference
whereas
that we have the same type of kernel
as we discussed
is that before
T
the kernel
A
now we have C (!.(T), T))R
if we want to use the results
-1
operated
of terms
we see that the derivation
The
upon CT(T)!.(T)
~
(T)(£(T)-~(!.(T), T)).
that we derived
must examine how this difference
When we do this,
in Chapter II, Section C.
for
Consequently,
in that section,
we
affects the derivation.
is unaffected.
Therefore,
233
. Eq.
in
2.22
and reduce
boundary
"
1\
C T (T)f.(T) by CT (~(T),
T)R -l( T)(!".(T)
-~(~(T),T)
we can replace
Eq.
7.8 to two differential
conditions.
optimal smoother
The differential
for this problem
equations
equations
with a set of
that describe
the
are
(7. 9)
=
d~\t)
-F T(t)E(t) - CT(~(t), t)R - l(tX;J t) -~(~(t), t)), To:S t .s Tf.
(7. 10)
The boundary
~(T
-
)
0
conditions
that are imposed
=
I
P p( T )
0=-
0
are the same.
We have
(7.11)
(7.12)
We shall make two comments
7.12.
First,
boundary
our derivation
conditions
approximations
of these differential
from Eq.
involved.
condition.
7.9 through
equations
There
we emphasize
Eqs.
and
are no
that the integral
It is usually
not sufficient;
its solution need not be unique.
Contrasting
corresponding
7. 8 is exact.
Second,
Eq. 7. 8 is only a necessary
consequently,
regarding
Eqs.
linear problem
7.9 through
7.12 to those derived
for the
in Chapter VI, Section A; we see that
the equation for ~(t) is the same,
while the nonlinear
aspects
of the
234
estimator
are introduced
in the equation for E.(t).
Now that we have derived the equations
consider
methods of solving them.
methods discussed
First,
let us run through the
for solving the parallel
set of linear equations
deve loped in section C of Chapte r IV.
superposition
principle;
we found a complete
therefore,
it is not applicable.
to realizable
= Tf by
at t
filter
In method 2
output.
introducing
If we
this we could also solve Eq. 7.9 and 7. 10 backwards
from the endpoint.
In the next section we shall derive
solution for the realizable
feasible.
Method 1 employed the
set of boundary conditions
a function that corresponded
could parallel
for ~(t) we must·
However,
this method.
filter,
so this technique
an approximate
is certainly
we shall still have instability
problems
with
The key to the third method was a linear relation
1\ (t) and p(t) as given by Eq. 4. 42.
between the functions "x(t) , x
-
Unfortunately,
time.
-r
-
we do not have such a relationship
Therefore,
only method 2 seems
Certainly,
at all promising.
there do exist other methods of solving non-
linear two point boundary value problems.
method of quasi-linearization.
are linearized
solution.
Then these linear equations
matrix
solution provides
equations
satisfactory
the problems,
1\
around some a priori
associated
convergence
however,
by use of
This new
around which the nonlinear
is repeated
until a
has been satisfied.
is generating
of ~(t) and E.(t). A reasonable
of the
are solved exactly,
This technique
criterion
is the
the estimation
estimate
with this system.
the next estimate
are linearized.
One technique
With this technique,
equations
the transition
at the current
the required
One of
a priori
estimate
choice might be to use the estimates
235
found by applying method 2, in effect combining this method with a
quasilinearization
approach.
We have seen how the realizable
solving the smoothing equations.
more interest
filter is important
This filter is certainly
than its use for this application.
in
of much
Let us now discuss
how we can use the smoothing equations to find an approximate
realization
c.
of the realizable
Realizable
Filtering
filter.
for Non-linear
Modulation Systems
In this section we shall present
approximate
derivation
realization
a derivation
of the optimum realizable
is a modified version
of that presented
for an
filter.
Our
by Detchmendy
17,33
and Shridar.
eliminate
'In particular,
the modifications
some of the troublesome
The fundamental
and the realizable
interval,
time variable
time variable
whereas
For the realizable
between the interval
involved.
in the realizable
interval
continually
estimator
In the
is the time within the fixed
filter the important
interval,
which
as the data are accumulated.
filter we want the estimate
at the end point of the
as a function of the length of the interval.
In order to make the transition
variables,
results.
is the end -point time of the observation
is not fixed but increases
observation
difference
in their
filter is the time variable
smoother the important
observation
issues
that we shall make
between the two time
we shall use the concept of "invariant
However,
before we do this,
particular
problem.
'We consider
imbedding".
let us motivate its us e for this
a sample function of E(t) near the end point
236
of the observation
that it vanishes
interval
at t = T
r
or near t
=
However,
at t = Tf - ~ T, we have from
Tf"
From E q. 7. 12 we have
Eq. 7. 10
(7. 13)
Now, we shall examine the same problem
functions from a slightly different
trajectories
same.
viewpoint.
for ~(t) and .e.(t) over the interval
way that these trajectories
shortened
with the same sample
interval.
However,
[ To' Tf- L\T].
solve a second problem
For this problem
the initial
the
We can
defined over this
conditions
are the
the endpoint time is now T - L\T, and .e.(Tf-L\T)
f
is equal to ~..!l. as defined in Eq. 7. 13 instead
We can produce the same trajectory
chosen non-zero
Let us consider
boundary
of being identically
by considering
zero.
an appropriately
condition for .e.(Tf-L\T).
This leads us to the following hypothetical
we imbed our smoothing problem
in a larger
class
que stion.
of problems
If
with
the boundary condition
(7. 14)
for Eq. 7. 10, how does the solution of Eq. 7.9 at T depend upon
f
changes in T and!l?
f
This question is answered
by the invariant
237
imbedding equation,
which is a partial
differential
equation for the
solution ofEq.
7.9 at Tf as a function of T and!l.
f
17
First, we shall sketch its derivation.
Let us consider
the solutions
to Eqs.
we impose the final bundary condition specified
shall denote these solutions
introduced
arguments
by Eq. 7. 14.
by ~(t, Tf,,,) and p(t, Tf<~l).
T f and!l. to emphasize
dependent upon these parameters.
assuming
7.9 and 7.10 when
Il to be an independent
We
We have
that these solutions
are
We also point out that we are
variable.
We have
A
= x(t)
)
(7.15)
(7.16)
We define
(7.17)
We note
(7.18)
For future convenience
let us also define the function
r by
238
(7. 19)
and the function
A by
=
(7. 20)
which is the desired
realizable
determine
a partial
differential
variables
Tf and .!l. •
Let us examine
illustrated
in Fig.
7.1.
filter
estimate
at T
r
We shall now
equation for ~(Tf' !]) in terms
the solutions
We have from
of the
~(t,Tf,T}) and E{t, Tf,T}) as
Eq.
7.10
AT=
t -- T f
A
1") -
A{~(Tf'
n ),n ' T f)AT = 11- A1")
(7.21)
239
t~
x(t, T r-
T -6T
f
'!V
T
f
T
t ~
E(t, T
vs , t
Tf-llT
,!l-)
vs , ~
Fig.
7. 1
Diagram for Invariant
Imbedding Argument
T
f
240
Now, we can interpret
second problem
[T
0'
!.(T - AT, T , 21) as being the solution
f
f
prod ucing the same trajectory
to a
ove r the interval
T f- AT] with the boundar-y condition !l - A.:l' i. e. ,
(7. 22)
We also have
(7. 23)
Combining Eqs.
7. 22 and 7. 23, we find
(7. 24)
We also can expand s(T ,,.,) in a two dimensional
f
Taylor
series.
Doing this,
i(Tf-AT,,.,-A!l)=~(Tf'.:l)
we obtain
-
ai(Tf'''')
aT
f
AT -
a ~ (T f'.:l)
8.:l
A.:l
(7. 25)
):<
We interpret
a~ (Tf,.:l) /a.:l as in Eq.
7. 5.
241
However,
as given by Eq. 7.21,
we have constrained
D.!l..to be
(7. 26)
Substituting
Eq. 7.26 in Eq. 7. 25, equating the result
dividing by D.T we obtain the desired
invariant
to Eq. 7. 24 and
imbedding equation
(7. 27)
This equation relates
the value of the solution to Eq. 7.9 at t
changes in Tf and 11, the end point time of the interval
boundary condition for Eq. 7. 10 at t
x(Tf) vs
Tf"
= Tr
by Eq. 7. 18 to find the realizable
equation that will generate
differential
solution would require
true;
however,
are interested
*We point
its solution
estimate
*
Let us see if we can find a set of ordinary
a partial
T to
f
and the
We now want to solve Eq. 7. 27 and evaluate
at !l = 0 as prescribed
=
the solution to Eq. 7. 27.
equation;
therefore,
in the solution
at !l.
=
This equation is
we would expect that its
an infinite set of equations.
let us try a finite order
differential
In general,
approximation.
this is
Since we
0, let us use a power series
out that we have made an expansion in terms of D.T. Since
there is currently some controversy regarding the significance of
the terms in the exqansion, we are certainly involved in this issue
with our approach.
0
242
expansion
in ll. We shall try a solution
of the form
(7. 28)
Eq. 7.28 implies
that we shall consider
explicitly
only terms
linear
in!l..
Let us substitute
Eqs.
our trial
solution into Eq. 7. 27.
7. 17 and 7. 18 and expanding terms
terms
to first
Using
order we obtain
2
of O( 1111 )
F(Tf:{~(Tf)
+
P"l(Tf)n]
+ G(Tf)QG
T(Tf)
+ terms
2
of O( 1111 )
(7. 29)
Now we combine terms
of the same order
in 11.
We find
243
- F{TflP 1 (T fl - G{Tf)Q GT{ Tfl)
n + terms
of O{
I~ 121 = 0
(7. 30)
.!l gives us a first order
Demanding a solution for arbitrary
approximation to the realizable
filter.
of each power of .!l must vanish.
"
d!.( T f)
A
dT f = F{Tfl:£{T fl
+ PI
{TflC
For arbitrary Il the coefficients
We find
T ~
-1
""
{:£{Tf}, Tfl R {Tfl (E.{Tfl - ~(:£{Tf), T fl)
(7. 31)
To complete the solution we need to specify some initial conditions
for Eqs.
7.32 and 7.23.
To do this we set T
f
=
To in Eq. 7.28.
We have
,n)
-~(T o..:.J.
I
=
.!l=Q
since we have assumed
1\
-x( T 0)
=
(7. 33)
0 )
'
zero a priori means.
We also have that
Eq. 7. 28 must satisfy the condition of Eq. 7. 11.
This implies
,
244
~;
P1{T o) = P 0
(This also requires
"
(7.34)
that the initial condition for the coefficients
in any
higher order expansi on must be zer o. )
Several comments
1.
order
are in order.
Although we have not made an issue about terms
AT in our expansion,
we have derived the same approximation
as found by Snyder who approximated
Fakker-Planck
If the observation
through 7. 34 are identical
This is certainly
easy to show that a first
the invariant
method is linear,
to those describing
to be expected.
order
Equations
7. 31
the Kalman-Bucy
For the linear
case,
it is
expansion yields an exact solution to
imbedding equation.
3.
therefore,
the solution to an a posteriori
equation. 10
2.
filter.
of
P l{Tf) is conditioned
upon the received
it cannot be computed a priori
also point out that we have no reason
P1{Tf) to a conditional
covariance
signal;
as in the linear case.
We
from this method to equate
matrix.
However,
in Snyder's
approach one can make this identification.
4.
approximations,
In general,
Finally,
if we want to consider higher
we should observe
the higher order terms
with the estimate
order
how P1{T ) couples to the estimate.
f
will couple both ways also,
and with the other terms.
In addition,
the number
of elements
in the higher order approximations
large,
on the order of (NF)n where NF is the state vector
~
dimension and n is the approximation
order.
1. e. ,
are going to be
245
D.
Summary
In this chapter
smoothing and realizable
We started
we have briefly
filtering
with an integral
outlined an approach
for non-linear
equation that specified
condition for the optimal smoothed estimate.
how some of the techniques
which we developed
could be extended to reduce this integral
linear differential
This reduction
differential
was an exact procedure;
equations
is equivalent
issue
for our problem;
of solving them.
realizable
in Chapter
II
equation to a pair of nonestimate.
solving the
specified
equations.
the smoother
we were still faced with the
One of the methods suggested
employed the
filter.
of solution for the realizable
imbedding.
derive a partial
in addition to the general
filter,
we introduced
differential
equation for the realizable
an approach
therefore,
desirability
the concept of
This concept used the smoother
This equation was difficult to implement;
structure
filter
to
estimate.
we introduced
which allowed us to find an appr oxima te solution which
could be implemented
structure
earlier
to solving the integral
however,
With this motivation
invariant
We then demonstrated
therefore,
equations
systems.
a necessary
equations for the optimal smoothed
These differential
structure
modulation
to
conveniently.
followed directly.
Using this approach,
the filter
246
CHAPTE R VIII
STATE VARIABLE ESTIMATION
IN A DISTRIBUTED ENVIRONMENT
In the application
munication problems,
example,
of state variable
the issue of delay arises
In these problems,
the delay enters
time of the signals between elements
difficulties
linear operation,
arise.
processes,
data processing.
of the finite propagation
the issue of delay into our
Although delay is cer tain Iy a
there does not exist a way of r ep reaenting
the methods that we introduced
For
in the array.
When we try to incorporate
several
in array
because
to com-
quite naturally.
since these methods are well adapted tovector
they appear to be well suited for problems
model,
techniques
in Chapter II.
it ~ith
If we examine the
iss ue of delay more close ly, we can see why this is true.
A delay operation
temporal
across
aspects,
an array,
involves spatial
as well as
whether it is caused by a wavefront propagating
or a signal passing through a delay,
We must recognize
the mode 1 introduced
or transmission.
that this type of opera tion is created
me chanism that is distributed
variable,
inherently
across
a spatial
by a
environment.
Since
in Chapter II involved only a single time
we should not expect that they would be able to handle this
situation where there is both a time and spatial variable.
24:l
In this chapter we shall extend our state variable
techniques
so that we can handle a certain class of problems
that involve a
distributed
arises
in array
environment.
This c las s of problems
data processing.
context of incorporating
are directly
We shall discuss our approach in the
a delay operation;
extendable to other distributed
medium may have several
and/or
lossy.
quite often
however,
the methods
environments
spatial coordinates,
where the
be nonhomogeneous
As we shall see, even the simple delay mechanism
causes a fair amount of difficulty.
Our approach here is also going to be different.
we worked with integral
them to differential
the differential
by a variational
In this chapter,
equations directly
enough general interest
A State Variable
we are going to derive
that the processes
the a posteriori
density
This does not mean that we cannot
methods,
theory to this problem.
by assuming
and then maximizing
approach.
extend our previous
A.
equations and used our methods to reduce
equations.
involved are Gaussian
Previously,
e. g., the Fredholm
integral
We have done this; however,
equation
it is not of
to develop it here for this single problem.
Model for Observation
in a Distributed
Environment
In this section we shall extend the concepts for the
generation
incorporate
of random processes
the distributed
of Chapter II so that we can
aspects of the delay operation.
is to find a set of state and observation
signals of the form
Our goal
equations to represent
248
m
~(t)
= a 0 ~Jt)
+
L. a
1~(t -
T i)
,
0:5
T
1 := T 2:= •.•
:=
T
m
1=1
(8. 1)
where s(t) is a signal process
previously
feature
discussed.
generated
by the methods we have
(We shall allow vector
does not require
any additional
observations;
modifications
Let us assume that we have generated
according to these methods.
observation
This implies
equation describing
this
to the theory)
a signal ~(t)
that we have a state and
its generation,
To < t (state equation»
~(t)
=
This also implies
ing the statistics
process
T o < t (observation
C(t)~(t),
equation).
(8.2)
(8. 3)
that we have made the following assumptions
regard-
of the initial state x(T ) and the driving noise
-
0
~(t),
(8. 4)
E[~(t);! T( T)]
For the purposes
=
Q 0(t-T)
of the derivation
assume that these are a Gaussian
Gaussian
random process
(8. 5)
in the next section,
random ve ctor
respectively.
we shall also
and a white
We emphasize
that the
249
result
is not dependent upon this assumption,
but more complex,derivation
since an alternate,
can be made without it by structuring
the filter to be linear.
As can easily be seen,
by a set of ordinary
differential
one independent variable.
variable
techniques,
as expressed
the generation
equations,
However,
equati on which describe s the dynamics
differential
variable
This operation
ordinary
of the delay operation
We need to introduce
of the system
differential
equation;
in terms
of
it is a partial
z as well as a temporal
z,
+
a~(t, z)
az
=
of propagation
(8. 6)
0
For this problem with pure delay,
we have assumed
a unity velocity
with no loss of generality.
It is easy to see how this equation describes
of the delay oper-ation.
general
an
which produces
cannot be described
equation with a spatial variable
a ~(t, z)
at
terms
with only
if we want to use state
in Eq. 8. 1 is not applicable.
a finite dimensional
or equations
the functional des cription
the de lay operation.
of ~(t) is described
Let us show how' we can represent
of this equation and a boundary
condition at z
= o.
the dynamics
S(t-T.) in
-
1
The
solution to Eq. 8.6 is
'1J(t, z)
-
= '1J
(t-z)
-0
(8. 7)
,,~
~I"
We should not confuse the vector
rna trices us ed.
w(t,z) with the earlier
transition
250
-wo
where
is an arbitrary
the boundary
function to be determined.
If we impose
condition
= ~(t),
-W{t,z)I
(B. B)
z = 0
we find
-W
o{t) =
r«. z) I
= ~(t).
=
z
(We could consider
If we evaluate
!(t,
(B.9)
0
that Eq. B. B describes
=
z) at z
the input to a delay line. )
Ti, or at the output of the delay line,
we
obtain
-W{t,z) I ..
-
Z
Consequently,
=
=
T.
1
-w
-0
(t-T.) = S{t-T.) .
1
we can rewrite
-
(B. 10)
1
Eq. B. 1 as
m
.l(t) = Q!os(t)
+
L
i
=
Q!i'\l((t,
T
i) )
condition
description
by Eq.
<
t )
system
B. 6 and whose input is described
of Eq. B. B.
(B. 11)
1
where -w{t, z) is the output of the dynamic
described
To
Thus,
we can eliminate
of the delay that appears
in Eq.
whose operation
by the boundary
the functional
B.l.
is
--_...---------~-~------=.;:==
251
We are now able to write a set of state and observation
equations
to represent
the effect of a delay in our observation.
We
have the state equations
dx(t}
dt
=
F(t)~(t)
+ G(t)~(t),
(8. 2)
(repeated)
T
o
::: t , O:s
z
(8. 6)
{repeated)
We impose
a boundary
!(t,O)
=
s(t}
=
condition
upon !(t,
z) at z
=
0,
C(t)~(t}
(8. 8)
( repeated)
upon !(t, z).
The observation
equation becomes
m
~(t) = O!oC(t)x(t)
+
L
i
=
O!iC(t-Ti)~(t-Ti)
=
1
m
aos
(t)
+
I:
i
=
I
As we can see,
the delay operation
important
Eqs.
enters
the equation describing
as a state
point to be made regarding
8.2 and 8.6.
(8. II)
ai~(t,Ti)
The state
of Eq.
equation.
the dynamics
However,
the difference
there
of
is an
between
8.2 at a particular
time can be
252
described
by a set of numbers
the state vector.
To describe
state of Eq. 8. 6, we must specify a set of functions
One cannot determine
of the variable
In dealing with distributed
media,
of the state vector to a state function is intrinsic
ide a of a system
the
to the
state.
The concept of the state of the delay operation
the pr oblem of the assumptions
initial state 'i"(T ,z),
-
z.
the state of this equation at a future time without
knowledge of these functions.
extension
the
0:5
0
Z
:5
that it is a zero mean Gaussian
leads us to
that must be made regarding
T
m • For the present
random
process
its
we shall assume
as a function of z with
a covariance
E['!r(T
-
0
, z)'!r(T , s)]
-
0
To complete
shall assume
o
= K (z,s)
0
our assumptions
:5 Z, S
regarding
that we observe .l(t) over the interval
presence
of an additive
received
signal !:.(t) is
!:.(t) = .x(t)
white Gaussian
noise.
;5
T
(8. 12a)
m
our model,
To :5t
;5
The refore,
+ ~(t),
where .l(t) is given by Eq. 8.9 and the covariance
we
Tf in the
our
(8. 13)
of w(t) is
(8. 14)
253
Before proceeding
derive the receiver
discuss
structure
to the next section whe re we shall
for estimating
how we could generalize
propagation
media.
we introduced
our model for other types of
The key to this is recognizing
to describe
the delay operation
equation which describes
the dynamics
It is, therefore,
interest.
appropriate
equation 8.6 as a generalization
At(t,z)
+
that the equati on
is actually
to consider
of our model.
media of
modifying
For example,
the
by the equation
a!(t, z)
az
+ B(t,z}!(t,z)
Az(t,z)
a state
of the distributed
dynamics of the medium may be described
a~(t, z)
-a-t-
x(t}, let us briefly
=
0, To<t,
0 < z
(8. 15)
where we impose the boundary condition of Eq. 8.9.
obviously a generalization
choosing the coefficient
homogeneous
and/or
of the delay mechanism
matri ces appropriate
Eq. 8.15 is
equation.
By
ly, one can mode I non-
lossy media.
We can also generalize
gains at each element.
Eq. 8.5 to include time varying
In this case Eq. 8. 11 becomes
m
y(t}
= a o(t)~(t)
+
> , a,
i
where
!(t, z)
=
(t)w(t, T,),
1
-
(8. 16)
1
1
is the solution to Eq. 8. 15.
We have illustrated
model, ~(t) is generated
our model in Fig.
by a system as described
8. 1.
In this
in Chapter II.
It
S
::s
OM
"0
- ===(
~
~I;::
-.::;
~
H
Q)
S
S
0
-S ~
~
-
9-S
0
Q)
H
$:l
OM
$:l
$:l
~"
~
~
U
--
.::;
0
ro
r-I
OM
$:l
Q)
H
0
OM
1tl
ro
c,
0
H
P-4
~
001
-~
XI
-l.-o-"'t
}oJ
C)
r-l
-
~
0
f:-i
b.O
0
Q)
r-I
Q)
"0
~
~
0
~
$:l'
0
OM
1tl
H
:>
Q)
CI.l
0
,.Q
r-I
Q)
0
"0
$:l
~
0
OM
H
~
ro
Q)
$:l
Q)
0
$:l
ro
r-I
b.O
OM
sa
254
~
~
--
r-I
Q)
0
"0
$:l
~
0
1tl
OM
H
:>
Q)
CI.l
0
,.Q
S
Q)
~
00
~
in
Q)
"0
~
::s
H
,.Q
OM
00
~
0
OM
ro
.
r-l
CO
b.O
~
OM
255
.
co
256
passes through a propagation,
may be described
by Eq. 8. 15.
a distributed,
The signal is then spatially
at various points in the medium and linearly
Eq. 8. 16.
The resulting
combined according to
We can generalize
by allowing different propagation
we would have a separate
sampled
signal then has white noise added to it
before being observed at the receiver.
even further
medium whose dynamics
paths.
medium description
the model
In this case,
for each path.
The model assumed allows a large degree of flexibility.
In some respects
assumptions
too much, since it requires
regarding
the medium,
us to make detailed
which in turn makes the receiver
quite detailed and difficult to implement.
However,
if one is faced
with a problem which does have this type of problem entering in
its observation
aspects
process,
of the problem.
one must somehow incorporate
the spatial
We feel that our approach is a reasonable
attempt at doing this.
B.
State Variable Estimation
in the Presence
of Pure Delay
In this section we shall derive the equations that specify
the optimal smoother when pure delay enters into the observation
method.
difference
We shall find that these equations are a set of differentialequations that specify the desired
estimate
implicitly
as
and observation
of
part of their solution.
We shall assume that the generation
the received
signal may be described
previous section.
by Eqs.
8.1 - 8.14 of the
Our approach to deriving the smoothe r structure
different than that used in Chapter VI.
We shall use a variational
approach to maximizing
density.
an a posteriori
In maximizing
this
is
257
density we shall need to introduce
the constraints
among the different
that since the delay operation
a state equation,
Lagrangian
variables.
is introduced
we shall be required
multipliers
to incorporate
We should also note
as a dynamic system
to estimate
with
its state also.
Let
us proceed with our derivation.
Since the processes
it is straightforward
involved were assumed
to show that the joint a posteriori
to be Gaussian,
density for
~(t) and !(t, z) has the form
where
k is a constant and the functional
J is given by
T
J(!.(T 0)' !!.(t) ,!(T 0' z)) =
iII
!. (T 0)
II
f
r: + i Sll~(t)
T
II Q -1 dt
o
+.!.2
(8.18)
*
IlxiiA
~
=
xAx
258
We note that x(t) is related
to x(T ) and u(t) by Eqs.
-
specified
by Eqs.
arguments
-
8.6 and 8.8,
0
-
uniquely determine
0
-
-
0
~(t) and ~t, z).
just as well minimize
-
0
However,
to perform
between the different
-
-
varying Lagnngian multiplier
functional
this minimization
x(t) to u(t) and x(T).
-
this density,
J as a function of the variables
porate the constraints
need to relate
The
,z) since these variables
It is easy to see that to maximize
'1r(T ,z).
-
and y(t) is given by Eq. 8.11.
for J are x(T ), v(t) and'1r(T
-
8. 2, '1I(t, z) is
J an identically
0
~(T 0)' !!(t) and
we need to incor-
variables.
First,
we
We can'ao this by using a timeA
.e.(t) for Eq.
zero term
we could
L
o
8.2.
We shall add to the
of the form
Tf
Lo
=
dx(t)
S
T
PT(t) (
at
(8.19)
- F(t)~(t) - G(t)u(t) ) dt
o
It will be useful to integrate
the first term
by parts.
Doing this,
we have
~f
J
T
( dpT(t)
-dt
~(t)
dt
(8. 20)
o
To incorporate
to introduce
)
+ .ET(t) F(t)~(t) + .ET(t)G(t)u(t)
a Lagrangian
each of the delay operations
multiplier
lJ.
S(t-T.) need
-
1
.(t, z) which is a function of
-1
259
both space and time.
operation
The state equation describing
is given by Eq. 8. 6.
L. which are all identically
Consequently,
we add to J in terms
zero
1
a~t, z)
at
(
+
a!{t, z) )
.
az
dt dz , 1
We shall also need to have this term integrated
with respect
the delay
to both variables
=
1, m,
by parts.
(8.21)
Doing this
t and z , we obtain
Tf
Li = ~
(.I?{t,
T/~(t,
Til - ~ T{t, 0) C{t)x{tl ) dt
T
·0
T.
1
+
S
(~T{Tf,Z)i]({Tf'Z)
- ~T{To'Zl!{To'ZrZ
To
al:!: T{t, z) )
8z
where we have substituted
!(t,z)dtdz,
i
=
C(t)x(t) for w(t, 0) according
We add Lo and Li as given by Eq. 8.20
and then we substitute
the expressions
Eq. 8.9 and Eq. 8. 3.
Doing this, we find,
I,m,
(8.22)
to Eq. 8.8.
and Eq.
8. 22,
for .l(t) and ~(t) as given by
260
T
f
J(x(T
),u(t), w(T
-0-
,z))
0
=
1
-2 Ilx(T
1+ 2
) II
-op-
1
-1 dt
+
m
iJ{T(T ,z)Q
-2
Q
T
5 J-("
1
m
II
TO
o
T
r Ilu(t)
j
o
0
(z,!:,)iJ{(T ,!:,)d!:, dz
0
-
0
0
+ PT(Tf)?£(T f) - !?(T
o)x(T 0) -
~
( d.E.~(t) x(t)+.E.T(t)F(t)?£(t)+.E.T(t)G(t)~(t))dt
To
T
m
i
=
f
S(
1
.t='f(t, Ti)~t,
Ti) - .t='f(t, O)C(t)?£(t))
dt
To
T.
1
So (~
z)'Ir(Tf, Z) - ~ ~(T
~(Tf'
-1
T
f
S
To
T.
\J (8l: at
1
-
-1
T
8~.
i (t z)
0
r
+
T
-~z
(t,z)
,z)w(T
0
-
,Z))
0
dz
)
.
!(t,
z)dt dz
(8. 23)
261
Let us now proceed
minimize
J.
First,
with our optimization
we define ~(T
optimum choice for minimizing
x{T )
0
=
1\
x{ T )
-
/It,.
E.{t)
~(T
-
Substituting
0
,z)
Eqs.
0
J.
-
0
,z) to be the
Now we expand
"
= -~(T
8.24
0
+e
J about these
+ e 0
0
,z)
(8. 24)
0 x (T )
-
0
(8.2S)
~(t)
+ eo~{T
-
through
0
,z)
+
~T(TolP~ 1 I) ~(Tol
(8.26)
into Eq.
8.26
T
e
-
to
i.e. , we let
optimal estimates,
-
~(t) and ~(T
),
-
procedure
uT(tlQ -ll)!:!(tl
o
1\
a.'lr{t,
1-
T.))
T
1
m
R -1(t)(-aoC{t)o~{t)
)
-
a.o'lr(t,T.))dt
1 -
i
'---'
=
1
we find
f
S
T
8.23,
1
X
+
262
!(
T
f
m
i
I=
J::y(t, Ti)O!(t,
1
Ti) - J::Y(t, O)C(t)O~(t))
dt
o
T.
1
J I~
~1
!(T ,
f
z)oW(T , z) - f1~(T
-
f
-1
,z)o'lf(T
0
-
J
,Z~dZ
0
o
+ termsof 0 (e)
2
(8. 27)
Now let us combine the common variations.
J(x(T
-0-
), u(t), w(T
-0
,z))
=
1\
J(x(T
We obtai n
A
), '"u(t), \]"[(T
,z))
-0--0
+
263
T
dp (t)
~T
\~
(t)Q!oC(t)
-
-dt
m
~
T
- P (t)F(t)
-
T
L""i
.
(t. O)C(t~c5X(t)
i=l
m
-d T (t)a. + ~ ~(t, T.~O~(t, T.) dt +
~(
L..
-
1
-1
1
1
i=l
T
m
T
m
m
J J
~_T(To'
s )QO(S, z)ds -
o
o
"~~T
L
-1
i
=
~ J J(
T.
!J:.{(Tf•
m
z)c5"\V(Tf•z)dz
Tf
-
o~(T
l(T.-Z)
1
,z)dz
-
T
T.
.~
1-1 T
o
+ terms
,Z)U
0
1
m 1
i~
0
T
a ~ :(t, z) a ~ . (t ,
;:
+ -~z
Z)) c5~(t, z)dt
0
o
of O(e 2),
(8. 28)
where
(8. 29)
We shall now make a series
variation
of the functional to vanish.
Lagrangian
coefficient
multiplier
First,
functions satisfy
of some of the variations
then argue that the coefficients
of arguments
we shall require
equations
vanishes
to cause the
of the remaining
that the
such that the
identically.
variations
e
We shall
must
dz
264
vanish because
First,
Lagrangian
it satisfy
"
/\
of !(T 0), u(t),
and
of the optimality
we shall impose
multiplier
restrictions
for the state Eq.
the differential
8.2.
!< To'
z).
upon E(t) , the
We shall require
that
equation
m
CT (t) (a~
(t)
+.
L .!.: 0)) ,
i(t ,
1
=
T 0 ==t .s T r:
1
(8. 30)
In addition,
we shall impose the boundary
condition
(8. 31)
Next, we shall restrict
delays
the Lagrangian
multipliers
for each of the
We shall require
T..
1
alJ..(t, z)
-1
at
alJ..(t,z)
+
-1
T. ,
1
az
i = 1, m
(8.32)
We note that these equations
lJ..(t,z) = lJ. ,(t-z),
-1
-0.
11
can be solved functionally,
T
0
==t ==T , 0< z <
f
where lJ. (t) is a function yet to be determined.
q-. '
T.,
1
i = 1, m
In addition,
(8.33)
we
1
shall impose both
shall require
temporal
and spatial
boundary
conditions.
We
265
=
~ .( T , z)
f
a.
-1
o :s z :s T.,
0,
(8. 34)
1
or from Eq. 8. 33
(8.35)
b.
~.(t,T.)
-1
1
=
T o :st:STf
a.d(t),
1-
(8.36)
)
or from Eq. 8.33
(8. 37)
We can interpret
each of these conditions
developed earlier.
specifies
a terminal
parallel
in terms
Eq. 8.34 is the parallel
state.
to the one imposed
If we substitute
Eq.
of results
to Eq.
8. 36 is a spatial
8.31 in that it
boundary
upon 'It(t, 0) as specified
Eqs.
8.30,
8.31,
8.32,
we
condition
by Eq. 8. 8.
8.34
and 8.36
in Eq. 8. 28
J(x(T ), u(t), 'It(T
-0--0
1\
,,1\
,z)) = J(x(T ),u(t), 'It(T
-0--0
T
E
((~T(To)p~l
,z) +
f
-I?(To))O.!(To) + I.(~T(t)Q-l_pT(t)G(t)o~(t)
o
(continued)
266
T
T
j
Jm
o
o
I
~T(To,!:.)Qo(!:.,Z)d!:.-
f1[(T ,z)U_1(Ti-Z)
o
i
c5w(To,z).
=1
(8. 38)
If we examine
they are the control
,..
the variations
parameters.
in the above we see that
Consequently,
we can argue that in
A
A
order for x( T ), u(t) and '1r(T ,z) to be optimum,
-
0
must vanish.
-
their
0
coefficients
We have then
1\
x(T ) = P
-
-
0
P (T )
0 -
(8.39)
0
(8. 40)
T
ill
i
2:-=
jJ..(T ,z)u
-1
0
-
l(T.-z)
1
1
=
m
Jo
A
Q (z,s)'1r(T
o
-
0
,s)ds,
(8. 41)
Eq. 8. 40 allows us to eliminate
equation for ~(t).
1\
~(t) in the differential
We obtain
(8. 42)
267
Eqs.
8.39 and 8.41 specify
state of the system.
Eq. 8.41 presents
integral
equation form.
integral
equation to a differential
conditions
a problem
upon the
since it is in an
Quite often we shall be able to reduce
those we have previously
that the initial
initial
discussed.
state estimate
a worst case situation.
equation by techniques
similar
It may be realistic
is spatially
white.
this
to
to assume
This is certainly
In thi s case we have
o
:5 Z, S :5
T
(8. 43)
m
for which E q. 8. 41 becomes
m
fJ.
.(T ,z)u
-1
0
-
l(T. -z)
This completes
by minimizing
1
K
-1
0
I\,
(z)'1r(T, z),
-
our derivation
the functional
for this structure.
=
J.
0
O:5z:5
T
m
of the estimator
Let us now summarize
(8. 44)
structure
our results
We have for To::=:t :5 T f (all the equations
are
repeated).
A
dx(t)
~t
=
F(t)~(t)
+ G(t)Q
GT(t)E(t)
(8. 42)
1\
a~(t, z)
at
=
O::=:Z:5T
m
(8. 6)
268
(8. 30)
ajJ..(t, z)
-1
o
at
:S
Z
:S
i
T.,
1
=
1,
m,
(8.32)
where
m
=
d(t)
-
The temporal
R-1(t)(r(t)
- a C1(t)x(t)
boundary
condition
-
P p( T )
0
CF'-
=
0
-
-
""
L Q!.iIt.(t,T.~
1-1
17
i = 1
(8. 29)
are
1\
x( T ),
-
(8.39)
0
m
i
I=
jJ..(T
-1
,z)u
0
-
I(T.-z)
1
/\
= K 0-1(z)'!r(T,
-
0
z),
O:S z :S T ,
m
(8.44)
1
(8. 31)
jJ. .(
-1
The spatial
T , z)
f
=
0,
boundary
o < z < T.,1
conditions
are
i
=
1,
ID.
(8. 34)
269
/'
=
!(t,O)
1'\
C(t)!.(t),
(8. 8)
J-L.(t,T.) = Q!.d(t),
-1
1
If we compare
variable
(8.36)
1-
these equations to those for the case of ordinary
equations,
we can see that these equations
state
are a logical
extension to them.
In the above representation,
we used the partial
A
differential
concerned
equations to specify
w(t, z) and the J.L .(t, z}.
-
-1
with a pure delay operations,
differential
we can solve these partial
equations and convert the above representation
of differential-difference
equations.
arbitrary
this representation
delay spacing,
due to the various
time intervals
In the general
involved.
which to work.
Let us illustrate
- Estimator
Structure
We shall consider
0
=
0
T
f
=
T > 4~T
T
1
=
~T
T
Z
=
Z~T
T
to a set
case of
can be rather
However,
equally spaced delays we can obtain a structure
Example!
When we are
complex
in the case of
which is easier
wi.th
this with an example.
for Two Equal Delays
the case when m
=
Z, and
(8.37 a-d)
270
This model would correspond
shall allow the system
to a three
element
for the generation
receiver.
We
of ~(t) to have an arbitrary
state description.
To convert
our receiver
to a differential-difference
equation form we need to solve the equations
We have done this in Eqs.
.t=i(t, z},
to determine
~(t) over the interval
r-
for !(t,
8.10 and 8.33.
[0, T].
z) and
First,
To do this,
we want
we need
1\
!(t, z).
We have
w
(t-z) =
-0
1\
!(t, z)
A
C(t-z)x(t-z)
-
0 est - z < T
=
A
!(O, z-t)
-2D.T<t-z<0
(8. 38)
The re for e, we obtain
d(t)
=
R
-1
,...
'"
1\
(t)[!:.(t)-Q!oC(t)~(t)-Q!IC(t-D.T)x(t-D.T)-Q!2W(O, 2D.T-t)],
D.T:::ts 2D.T,
~D.TstsT.
Now we need to find the functions
J..L
-0.
(t).
1
we have
From
Eqs.
(8.39)
8.35 and 8.37
271
J-L
o
T - DoTst sT,
=
(t)
1
1:0 (t)
z
t ST - DoT;
{::~{t+At),
-DoT s
Ol
T - 2D.TSt S T ,
=
{
- 2D.T:St
Q!l£!. (t+lDo T),
T - lDoT.
We are now able to write the estimation
,,/\
in terms
of ~(t), E(t) and 'l!(O, z}.
throughout
(8.40a)
The equation
(8.40b)
equations
solely
1\
for ~(t) is the same
the interval
A
d~(t)
T
A
-at = .F(t)!Jt)
By substituting
Eqs.
+ G(t)Q G (t)E(t),
o:s
(8. 41)
t:S T
8.39 and 8.40 for d(t) and fJ. .(t, 0) respectively,
we find for the different
-
-1
time intervals
+ (l2R -1 (t+2 A)[!:{t+2AT) -(loC{t+2AT)~{t+2AT)
-(l 1C{t+AT)~{t+AT) -azC{t)k6;) )
o
S
t
S
Dot
272
+(11 R
az
-1
[
1\
1\
/\]
(t+D.t) .!'(t+AT) -(10 C(t+AT)~(t+D.T) -(11 C(t)~(t) -azC(t-AT)~(t-D.T)
R -1 (t+ ?t..T)[ Eo( t+ Zt..T)- a0C( t+ Zt..T)g( t+ Zt..T)-
a1C(tMT)~
D.T
C T(t)
(aOR
-1(t)[ r(t)-
(11R -1(t+D.T)[£(t+D.T)
+
azR
<t<
(t+t..T)
-az
C(t )g( t) "
2D.T, (8. 42c)
ao C(t)~(t)-a1C(t-t..T)X(t-t..T)-azC(t~zt..T)x(t-
ZtIT)
-(1oC(t+D.T)~(t+D.T) -(11C(t)~(t) -azC(t)D.T}~(t-D.T)]
-1 (t+Zt..T)[.E<t+Zt..T)-a
0 C(t+
ze..T)~(t+Zt..T) .,a C(tMT)Q(t+t..T)-azC(t)~(t)
1
2D.T
<t <T -
2D.T
(8. 42d)
(cantin ued)
l)
273
(aOR
C T (t)
-1 (t)[ :r.(t) -aoC(t)~(t) -a1 C(t-b.T)~~ T
-em -a2C(t-
2t>.T)1t-2b.T)]
a1 R -1(t+b.T)[ £(t) - aoC(t+b.T)~(t+b.T) - a1C(t)~(t) - O!z C( t-b.T)~'<t-b.T)] )
T - 2.6.T
-F
(t)p(t)
-
The boundary conditions
p(O)
0-
p(T)
K-1(t)'lt(O,
o
-
(8. 42d)
-
T - .6.T
P
- .6.T.
T
dE(t)
-- dt =
<t <T
<t <
(8. 42e)
T.
are
1\
= x(O)
(8. 43)
-
=Q
(8. ~4)
t)
=
O2R -1 (2.6.T-t)
[
£,(2.6.T-t)
-02 C(2.6.T-t)x( ~ 2D.T-t)
o
A
-al C(.6.T-t)x(.6.T-t)
::= t ::= AT,
-02 'It( 0, t)]
(8. 45a)
,
274
(8. 45b)
AT == t == 2AT.
Eqs.
8.41 through 8.45 specify our receiver
understatement
structure.
It would be an
to say it is simply complex.
The major difficulty
encountered
i. e.,
equations both delayed and retarted,
~(t+AT) all can enter the same equation.
'"
is that 2£(t) enters
the
~(t), ~(t- AT), and
Unfortunately
the mathematical
theory for solving this type of equation has not been developed very
extensively,
if at all.
Therefore,
we shall suggest
two possible
approaches.
First,
orie can approximate
the de lay operation
by some
,-
finite Pade approximation.
The order would of course be dependent
on how large the delay is compared
process
receiver,
involved.
correlation
A second approach
and then designing
to
~(t-AT), X(t+AT) and ~(t+2AT), and the corresponding
the dimension
will impose a severe
solution algorithms
the
the receiver.
is to augment the state vector
function "E.(t-2AT), A
E.(t-AT), A
E.(t+AT) and "p(t+AT).
increases
time of the
We should point out that we are approximating
not the environment
include ~(t-2AT),
tothe
of the system
computational
this
involved quite quickly which
demand.
is one of the issues
Obviously,
Finding efficient
for our future research.
275
c.
Discussion
of Estimation
for Distributed
To conclude this chapter
comments on the status of estimation
Systems
we shall make some general
theory for distributed
As we have done in many parts
of this thesis we have
borrowed heavily from optimal control theory methods.
concerning the optimal control for distributed
advanced than it currently
relationship
is for estimation
systems
theory.
between the two we shall briefly
more pertinent
systems
Because
of the
in the area of optimal control theory
studied approximation
of articles
, In these articles
maximal (or minimal) principle
procedures
by
they developed a
for such systems.
They have also
by· the use of truncating
an
expansion series.
From the aspect of applications
to a particular
all their studie s have been conce rned with the heat,
equation.
if far more
mention some of the
was done in a series
35 38
Butkovskii and Lerner. -
orthonormal
The research
work that has been done in this are a.
The initial research
for distributed
syste m.s ..
This particular
all the studies
control literature.
systems
This is perhaps
which is of the parabolic
which have appeared
unfortunate
a dynamic system.
usually of the hyperbolic
type, and they are,
.
in almost
in the
since this equation,
type, is not representative
which commonly describe
to propagation
or diffusion,
equation has been investigated
of distributed
system,
of those
These equations
in particular"
are.
related
phenomena. 34, 39
Since Butkovskii's
in the area.
Most notable
programming
principles
.
1S
work several
Wang.
people have made studies
40 41
'
He has e mp loy.ed dynamic
to the control problem.
This gives a
276
functional equation which the optimal process
formulated
the concepts
distributed
systems.
of observability
In his examples
to expansion techniques
satisfie s , He has also
and controllability
(the heat equation) he resorts
in order to determine
the solution.
studies are probably the most concise formulation
which arise
in~distributed. systems
for
Wangts
of the concepts
which has appeared
to date.
Sakawa 42 has also studied the optimal control of a system
described
by the heat equation.
output relationship
He exploits
for such a system
Again expansion techniques
Other investigators
43-45
Murray.
the fact that the input-
can be analytically
determined.
are used to actually find a solution.
in this area include Goods on, Pie rre,
Recently,
the realizable
filter
structure
and
for the problem
that we studied in the last section was obtained by extending the
Kalman-Bucy
particular,
approach.
Many of the same issues
equation the results
spatial variables
We could derive the realizable
which further
filter
employed;
In array processing
however,
complicates
considered
the realizable
conjunction with the solution of the interval
disadvantage
In the classical
filter
the issue.
in Chapter VI.
here remains
whereas
theory,
estimation
error
.
equations.
approach is
performance
one does exist for the classical
however,
the system
to
is not usually
of the state variable
that a method to dete rrnine the mean square
not been determined,
equations.
we have seen that it is very useful in
A current
in
is a function of two
from the smoother
Whether we can do this for the problem
be seen.
enter,
delayed and advanced differential-difference
In addition the variance
essentially
46
is limited to be
has
theory.
277
stationary
and of infinite time.
techniques
must be used because
involved.
A state variable
computationally
transient
easier
In addition, numerical
of the non-rationality
approach to this problem
of the problem.
theory for this problem.
find a second,
in terms
obtain an approximate
concept tothe
correlations
We can employ these results
and eigenfunctions
of them.
structure.
'..
may be an appreciable
simpler
,Furthermore,
using this method.
to
If we actually
involved we can expand
By truncating
our receiver
analysis.
that we could derive a
this expansion we can
(This is similar
approach Of Butkovskti and Lerner.
derived in last section.
performance
This would allow
or modal, approach to this problem.
compute the eigenvalues
our receiver
may possibly be
how much we lose by using an asymptotic
We mentioned in the introduction
Fredholm
of the spectra
and at the same time allow one to study the
or time varying aspects
us to determine
integration
35- 38
structure
)A bank of
than that
we can calculate
the
in
278
CHAPTER IX
SUMMARY
We have presented
use of state variable
techniques
an extensive
for solving Fredholm
equations and the application
of the resulting
in optimal communications.
The material
developed the solution techniques
while the remaining
Chapters
the theory to solve various
In Chapter
random processes
were particularly
with systems
of the state vector Kx(t,'t).
a pair of differential
conditions.
2-4
equations,
aspects
of
theory problems.
the concepts of generating
described
by state variables.
in the properties
specified
We
of the covariance
By using these properties
able to reduce the linear operator
integral
in Chapters
5-8 exploited various
communication
ot the
theory to problems
for the integral
2 we introduced
interested
discussion
we were
by this covariance
equations with an associated
to
set of boundary
These equations were the key to many of our derivations.
In Chapter 3 we applied these concepts and results
solving homogeneous Fredholm
reduced the integral
integral
equations.
equation to a homogeneous
equations with imposed boundary conditions.
We
to
first
set of differential
The coefficients
for
279
these equations and the boundary conditions were determined
directly
in terms
of the kernel
of the state matrices
of the integral
matrix associated
that describe
equation.
We then used the transition
with these equations to find a transcendental
function whose roots specified the eigenvalues.
values,
matrix.
the etgenfuncti.ons follow directly
Finally,
the eigenvalues
the generation
Given these eigen-
from the same transition
we used the same transition
to find the Fredholm
matrix that specified
determinant.
As a result,
we had that the only function that needed to be calculated
in order
to find the eigenvalues,
determinant
eigenfunctions
function was the transition
In Chapter
matrix.
4 we again used the results
to reduce the nonhomogeneous
set of nonhomogeneous
conditions.
Fredholm
differential
The coefficients
differential
the generation
equations
integral
of Chapter 2
equation to a
equations with a set of boundary
of the differential
boundary conditions were again directly
which describe
and the Fredholm
equations
related
of the kernel.
and boundary conditions
and
to the state matrices
We noted that the
derived were the
same as those that specify the optimal smoother
structure.
Then
we exploited the methods that have been developed in the literature
for solving the smoother
integral
equation.
equations
so as to solve the nonhomogeneous
We were also careful to note the applicability
of each method that we introduced.
Chapter 5 considered
a problem
in optimal signal
280
design.
One of the more important
applications
integral
equation occurs in the communication
a known signal in additive colored noise.
Chapter 4 as describing
problem of detecting
We viewed the results
a dynamic system which related
ating signal in the optimal receiver
then used Pontryagin's
of the nonhomogeneous
Principle
to the transmitted
signal energy and its bandwidth are constrained.
necessary
conditions we devised an algorithm
signals and their resulting
performance
first and second order spectrums.
examples
1
the algorithm
interesting
equation results
We then found the differential
on a unified
with a delay.
equation.
structure
and the filter realizable
Chapter 2 so as to treat nonlinear
we reduced an integral
for 'the
equations for the realizable
from the smoothing equations.
methods for calculating
We used
from Chapter 4 and
We presented
the covariance
of error
of the
with delay.
In Chapter 7 we extended the results
results
features.
realizable
those from Chapter 2 to derive the state variable
smoother
when both the
to design optimal
Our starting point was the finite time Wiener-Hopf
several different
conditions
In the course of doing these
approach to optimal smoothing and filtering
with delay filter directly
We
By using the
Chapter 6 was an extensive presenation
smoother.
signal.
when the channel noise had
displayed several
the nonhomogeneous integral
the correl-
to derive a set of necessary
for the signal that optimizes the system performance
of
derived in
modulation systems.
equation that specifies
Using these
a necessary
281
condition for the smooth estimate
and boundary condition for it.
results
to a set of differential
Our derivation
was exact,
were equivalent to the original integral
introduced the concept of invariant
an approximate
realization
equations
so the
equation.
We then
imbedding in order to derive
of the realizable
filter from the nonlinear
smoothing equations.
In Chapter 8 we recognized
enters
our observation,
that when pure delay
we cannot describe
sional state equation.
Consequently,
it with a finite dimen-
we extended our concept of
state to include function states in a space-time
domain.
this concept we were able to derive the smoother
delayed observations.
media; however,
the case of pure delay was rather
In the course of our discussion
the structure
even in
summary
of the results.
we worked many examples
Although the methods were certainly
ally efficient when used properly,
we emphasized
to illustrate
analytic-
the numerical
of our methods since this is where we think their
application
for
complex.
This completes the general
aspects
structure
The methods we used were extendable to
other types of distributed
the methods derived.
With
major
lies.
We also indicated that the techniques
quite often powerful enough to be either
more complex problems.
that we used were
extended or applied to
These problems
suggest topics for further
282
research.
We shall list them by chapters:
2-4.
extending the results on solving the Fredholm integral
equations when the kernel is generated by a nonlinear
or dispersive system;
5
signal design for spread channels; solution algorithms
for problems when hard constraints are imposed;
6
effective implementation
7
solving the smoothing equations for nonlinear modulation
systems ;extension of invariant imbedding to treat
filters realizable with delay; finding the relationship
between the covariance of error and the invariant
imbedding terms;
8
evaluation of the smoother performance;
effective
solution procedures for the smoother; the use of a
modal or eigenfunction expansion for realizing the
filter structure.
In addition there
are many direct
of filters
applications
realizable
with delay;
of the theory we
have already developed.
Many of these are mentioned at the end
of each of the respective
chapters.
283
APPENDIX
A
DERIVATION OF THE PROPERTIES
OF
COVARIANCES FOR STATE VARIABLE
RANDOM PROCESSES
In this Appendix we shall sketch the derivation
properties
stated in Chapter II-B.
as discussed
7.
that !.(t) is generated
in Chapter II-A.
First,
t >
We assume
of the
we derive Eq. 2. 10.
The state at time
t is related
the input utt ') over the interval
Consider the case when
to the state
t> t ' >
7
at time
7
and
by
t
~(t)
=
!l(t,T)~(T)
+
J
!lIt, t')G(t')~(t')dt'
(A-I)
7
where 8(t, t ') is defined by Eq. 2. 11.
!.T (T), and take expectations
8(t,7)K
If we post-multiply
A-I by
we obtain
x
(7,T)
+
t
S
7
!lIt, t I)G(t I)E[ ~(t l)~ T( T)] dt '
(A-2)
284
However,
because
of the Markov nature
and x{ T) are independent
over the range of integration.
the second term of A-2 is zero,
K (t , T)
x
=
9{t, T)K
which is the first
X
of the state vector,
y,{tY)
Conse quentl.y,
and we have
(T, T),
t
>
part of the desired
second part of Eq. 2.10 is identical;
We now derive Eq. 2.12.
(A-3)
T
result.
The derivation
therefore,
of the
we omit it.
We proceed by differentiating
the definition of K (t , t)
x
d
T
K~(t,t)
By substituting
~(t)
T
Lr -at
~
(t)
the state equation,
dt K (t , t)
x
d
= E
=
+ ~(t)
dxT(t)
dt
J
(A-4)
we obtain
F(t)K (t , t) + K (t , t)FT(t)
x
x
(A-5)
Since the last two terms
only the sec ond te rm.
are transpose
s of each other,
The state at time
t in terms
we consider
of the initial
state x(T ) and the input u(tJ) for T < t ' < t is given by
-
0
0
t
~(t)
=
(lIt, To) ~ (To)
+
S
To
{lIt, t l)G(t ')~(t ')dt
1
(A-6)
285
Therefore,
we have
=
E [~(t)~ T(t}] GT(t}
{e(t, T }E[x(T
o
-
}uT(t}]
0-
+
t
J
9(t. t')G(t')E[
~(t
').'?(t)] dt'}G
(A-7)
T (t)
'To
that the first term is zero for t > T.
o
We assume
becomes upon performing
The second term
the expectation
t
E[!.(t)~T(t)]GT(t)
=
5
9(t,t')G(t')Q
o(t'-t)dt'GT(t)
(A-B)
To
The integral
interval.
is non-zero
at only the endpoint of the integration
We must assume that the limiting
is symmetrical;
the integration
therefore,
region.
form of the delta function
only one -half the "area"
IntegrationEq.
is included in
A-B thus yields
(A-9)
Substituting
desired
this term plus its transpose
into Eq. A-5 gives the
result
(A-9)
The initial condition Kx (T , T ) must be specified.
-
0
0
286
APPENDIX
COMPLEX
RANDOM PROCESS
All of the waveforms
signals.
In many applications,
considered,
B
GENERATION
considered
in the text were low pass
e. g. the signal
design problem
it is useful to be able to extend these concepts
case of bandpass
waveforms.
In this section
that we
to the
we shall show' how we
can do this by using the concept of a complex state variable.
The use of complex notation for representing
processe s and functions is we 11 known.
random proces s that has a spectrum
carrier
w we can represent
c
y(t)
=
Y (t) cos(w t)
=
Yc (t) - J Y s (t)
c
esc
For example,
narrow
band
if y(t) is a
which is narr ow band about a
it in the form
+Y
(t) sin(w t)
=
jw t
Re[y(t)e
c]
(B-la)
where
,."
y(t)
.
(B - 1b)
Y (t) and y (t) are low pass processes.
c
stationarity
s
assumptions,
Under the narrow
band and
one can show
( B-2a)
287
and
( B-2b)
Both components
have the same covariance,
is an odd function.
We can represent
complex process
and the cross
There are two important
yet) as sinusoidal
y(t) , termed
imaginary
parts
processes
that have identical
points to be made here.
modulation
of a low pass
the complex envelope.The real and
of yet), yc(t) and ys(t) respectively,
This provides
The use of complex state variables
consider
components.
notation is applicable,
However,
a key to our analysis.
is purely a notational
It is obvious that all the problems
may be solved by expanding the terms
imaginary
are random
auto c ovar iance s , and a very particular
form for their cross covariance.
convenience.
which one wants to
into their real and
for the problem
where the complex
this expansion is too general
and it leads to a needlessly
covariance
cumbersome
description
a formulation
of the
processe s involved.
If we are to describe
notation,
various
by complex
we want to have a convenient form for representing
covariances
enumerate
of the components.
them all individually,
using the higher dimensioned
notation for describing
are two processes;~,
Obviously,
the
if we have to
we have not gained anything over
representation.
random processes
the quadrature
y (t) of a narrow band process,
s
random processes
The complex
is applicable
components
when there
yc(t) and
which have the same auto
288
covariances
them.
and the particular
form for the cross
covariance
between
We now want to show that we can find a state representation
which generates
a complex random process
In addition we want to generalize
non-stationary
vector
Random Process
process
Generation
which satisfies
our concepts
Eq. B-2.
so as to include the
case.
with Complex Notation
In this section we shall develop the the ory needed to
describe
the generation
of complex random proces ses.
that we have the following state variable
description
Let us assume
of a linear
system
d~(t)
<:it =
i(t)
,...,."
,.",."
F(t)~(t) + G(t)~(t)
= C (t)~(t))
(linear
where all the coefficient
describe
statistics
the generation
(linear
state equation)
(B-3a)
J
obse rvation equation)
matrices
may be complex.
(B-3b)
In order to
we shall make two assumptions
of the driving noise ~(t) and the initial
-
regarding
the
state vector ~(T ).
-
0
With these two ass umptions we shall deve lop the entire theory.
Finally,
we shall demonstrate
that such representations
be used as a convenience to describe
band processes
First,
M
~(t).
the complex envelope
by showing that they yield results
Eqs. B-1 and B-2 in the stationary
let us consider
The complex covariance
can indeed
consistent
of narrow
with
case.
the white noise driving function
function for the proces s assuming
zero
289
mean is
(B-4)
where we have used the notation
(B-5)
i .e.,
the conjugate transpose.
covariance
in terms
K~t,T)
U
-
Let us expand this complex
of the quadrature
components.
+
= E[ (u (t)- j (t))(u T( T)
-e
u
-c
-s
Ku U (t,T)
-c-c
+ Ku
U
(t,T)
-s-s
j
u
(T))]
=
-s
+ jKu
u (t, T) - jK
u (t, T)
u-s-c
-c-s
1'/
Q O(t-T)
In order
that the covariance
representing
components
K
the covariances
#ttl
(B-6)
matrix
and cross
of ~(t), we shall require
u u
-c-c
be a convenient
method of
covariances
of the
that
1
fJ
(t,T) = K
(t , T) = '2 Re[ Q] O(t-T),
u u
-s-s
1
N
(t, T) = '2 Im ] Q] o(t- T).
(t,T) = -K
K
u u
u u
-c-s
-s-c
(B-7a)
(B-7b)
=
290
The covariance
matrices
for the two components
negative definite matrices,
and the cross
are identical
covariance
non-
matrix is a
,J
skew symmetric
matrix.
This implies
matrix with a non-negative
that Q(t) is a Hermitian
definite real part.
We also note that the conjugate operation
of the complex covariance
matrix,
in the definition
for under the above assumption
we have
(B-B)
Quite often, one does not have correlation
components
of u(t) (1. e. , E[ u (t) u T(t)] = 0) since any correlation
-
-c
-s
-
between the components
of the state vector
the coefficient
F(t) and G(t).
matrices
non-negative
definite symmetric
assumptions
we made,
u
c.
may be represented
In this case,
matrix.
Also,
is uncorrelated
Q(t) is a real
s.
for all i .
1
The next issue which we want to consider
whatever
In order that we be consistent
assumptions
is the initial
with the concept of state,
which we make regarding
the initial time To' should be satisfied
in
note that under the
with u
1
conditions.
between the
the state vector
at an arbitrary
at
time
t (t :::: To)·
First,
vector (we assume
covariance
we shall assume that x(T.) is a complex random
-
1
zero mean for simplicity).
matrix for this random vector
is
The complex
291
,.J,..,
P
o
I'J
= K,.,(T
,T ) =
x 0 0
E[x(T
-
+
)5{ (T )]
0-
0
=
(B-9)
The assumptions
covariance
which we shall make about the
of this random vector are
1
= "2 Re[
K
xx
-c-s
Consequently,
Hermitian
(To' T ,
0'
=
K
xx
-c-s
Po] )
11
with a non-negative
(B-I0a)
(B-I0b)
(T . , T. )
the complex covariance
matrix
,.I
matrix
of initial condition
definite real part.
is a
We also note
that under the above assumptions
(B- 11)
Let us now consider
the covariance
imply about
of the state vector ~(t) and the obse rved signal ~(t) .
Since we can relate
state vector,
what these assumptions
the covariance
we shall consider
of ~(t) directly
K (t,T) first.
x
to that of the
292
In the study of real state variable
random processes
one
,.,J
can determine
K",,(t,T) in terms
Q associated
function
with the covariance
N
K~(T ,T ) of the initial
x 0 0
operation.
,.,
,.,
~t,
state vector,
matrix
,.,
,..,
T) to Ki}{(t,t).
is replaced
The methods for determining
to find K~(t, t) as the solution
use the transition
operation
the linear
-
0
The
by a conjugate
K~t, T) are first
with the matrix
K~t, t) satisfies
~
x(T ).
w
of a linear differential
associated
the
noise
for complex state variable s are exactly paralle 1.
only change is that the transpose
transpose
of the excitation
N
u(t), and the covariance
The results
of the state equation matrices,
x
x
equation and then
F(t) to relate
matrix
differential
equation
~
dK'"'(t , t)
X
dt
~,
=
.IV
d
-
~
F(t)K~(t, t) + ~(t,
N+,'"
-
_I
'"'+
t) F (t)+ G(t) Q G (t)
(B-12)
,oJ
where the initial condition IW(T ,T ) is given as part of the system
x
description,
'"
~(t,
,.J
K~(t, T)
x
x
T)
0
0
is given by
=
T
where 9(t, T) is the complex transition
We also note for future reference
that
>
matrix
t ,
associated
(B-13a)
with F(t).
293
(B-13b)
We can readily
In order
operation
to do this,
Q is Hermitian;
initial conditions
therefore,
equation.
,.,
+
AI
+ F(t)K
,.,
transpose
x
I"'!
~+I"+
+
j'J
x
Since K''''(t,t) ~nd K-+(t,t)
x
(K"'(t, t) and K-(T,
x
x
the same
x
~
0
-
x
-
T ) is Hermitian
Consequently,
(B-14a)
(t , t) + G(t)Q G (t)
K"'(t, t) and K- (t , t) satisfy
~
they must be identical.
the conjugate
This yields
= .i'JK'"x+(t, t)F~ + (t)
linear differential
for all t ,
x
we simply perform
upon Eq. B-12.
dt
show that K-(t, t) is Hermitian
0
have the same
by assumption)
complex covariance
,
matrix
,J
K"'(t, t) is Hermitian
for all t , We can also show that
x
(B-14b)
for all t , In order to do this we note that this expectation
the linear
differential
satisfies
equation
(15)
Since Ql equals zero (Eq. B-8), the forcing term in this equation is
zero.
In addition,
the homogene ous solution is zero for all
t
294
(Eq. B-11).
This proves the assertion.
By using the above, we may prove that E[!(t)gr
zero for all t and
T.
( T)] equals
To do this we note that
'jJ
[,J ~T
:G(t,T)E ?£(T)?£(T)])
T
Since the expectations
zero,
the expectation
covariance
on the left equals zero for all
of the initial
vector
-
~(t), which is related
which we have made on
interest
directly
with the state
is the observed
to the state vector by Eq. B-3b.
simply state the properties
by the
0
we are not concerned
The vector
0
for all t :::T .
-
of a system.
t and 'T.
state vector ~(T ) are satisfied
of the state vector ~t)
Usually,
(B-16)
t ,
on the right side of the above equation are
We note here that the assumptions
the covariance
>
of the covariance
signal,
We can
1<- (t, T) since it is
Y...
related
directly
to the covariance
of the state vector.
This relation-
ship is given by
Kjt,T)
""
w+
= #C(t)K""'(t,T)C
(T)
x
Consequently,
it is clear that ~,J(t,
,./
Y
s.
(B-1?)
t) is Hermitian
and that
is zero.
We are now in a p osi ti on to derive
properties
regarding
the individual
components
some of the
of the observed
295
signal.
First,
let us prove that the covariances
~s(t) are identical.
We have
1
= 2'
,..,
(B-18a)
Re[ ~(t,T)]
1
~
2' Re[ Ki(t,
Consequently,
the covariances
in a similar
fashion.
of both components
covariance
We have
(B-18b)
T)]
half the real part of the complex covariance
The cross
of ~c(t) and
are equal to one-
matrix.
between components
may be found
296
E[;rc(t)~s( T)]
=
E { (~(t):i*(t))
ieT(T);
i*T(T) ) }
I
=
1
-"2
By using Eqs.
~
(B-19)
Im [K-(t, T)]
Y..
B-IB and B-19,
we have a convenient
method for finding the auto- and cross-covariance
imaginary
components
complex covariance
convenience
of the complex signal ~(t) in terms
s.
of working with just one covariance
the covariances
This is a major advantage
Stationary
This provides
function K-(t, T).
allows us to determine
of the real and
of the
the notational
matrix,
of the individual
yet it
components.
of our complex notation.
Random Processes
We now want to show that the assumptions
lead to results
which are consistent
developed for stationary
scalar
which we made
with those which have been
random processes.
By choosing
w
P
-0
tobethe
,...,
P
o
steady state solution to Eq. B-12,
=t-+oo
lim
Le.,
tV
K-(t, t),
~
(B-20)
297
,J
We can show that the covariance
.
Equivalently,
state vector
IV(At)
Y..
K,J(t, T) is stationary.
Y..
we could say that under this assumption
~(~),
For stationary
,.,
matrix
~(t) is a segment
random
=
processes
of stationary
for the initial
random
processes.
we shall use the notation
1'\/
KJJ(t,
Y..
(B-2l)
t +At),
i.e ., we shall use only one argument.
Since we have already
proven that the real
components
have the same covariance
is certainly
satisfied.
cross
(Eq.
covariance
Therefore,
(Eq. B-l8),
and imaginary
then Eq. B-2a
we need only to pr ove that the
is an odd function.
In general,
we have
B-l4),
Re[ IV
K,.J (At)] + j Im] "K,..,(At)]
Y..
Y..
(B-22a)
By equating the imaginary
parts
and substituting
Eq. B-l9,
we
obtain
-K
(-At),
(B -22b)
~sY..c
which for the scalar
conditions
spectrum
case is consistent
are sufficient
in the scalar
with Eq. B- Zb.
to show that Y..(t)has a real,
case.
These
positive
298
Summary
In this section
we introduced
complex random process
complex state variable
by exciting
description
discussed
in terms
description
were required
independent
C(t).
system
the second order
of a complex covariance
how we could determine
variable
a linear
a
having a
with a complex white noise.
then showed how we could describe
this process
the idea of generating
We
statistics
of
function and then we
this function from the state
of the system.
The only assumptions
to make were on U'(t) and ~(T ).
-
..
0
which we
Our results
of the form of the coefficient matrices
F(t),
were
G(t), and
Our methods were exactly paralle I to those for real state
variables,
and our results
developed for describing
notation.
to those which have been
narrow band processes
We shall now consider
of random processes
two examples
by complex
to illustrate
the type
which we may generate.
.!.
Example
The first
(scalar)
were similar
case.
example which we cons ide r is the firs t orde r
The equations
which describe
this system
are
I"OJ
d~(t)
=
-kx(t) + u(t)
(state equation) ,
(B-23)
(observation
(B-24)
and
I"OJ
y(t)
=
I"OJ
x(t)
The assumptions
regarding
t;(t) and "i(T ) are
o
equation) .
299
E[ u(t) u (T)1 = 2Re[ k] PO(t-T)
,..."
jI'tIJ
~:(
,..."
(B-25)
and
(B-26)
Since we have a scalar
addition,
process,
we have again assumed
. Eq. B-12,
zero
means.
"'"
First,
we shall find K (t, t).
x
which it satisfies
In
both P and P b must be real.
The differential
equation,
is
"'"
dK""'(t, t)
= _ kK''''''(t,
x
x
dt
t) - k':~K""'(t,t)
x
+
2Re[
k] P
(B-27)
The solution
to this equation
is
t
In order
>
T
(B-28)
o
"""
to find K (t,T), we need to find 9(t, T), the transition
x
for this system.
~(t, T)
=
This is easily
matrix
found to be
"'"
e -kl t-rr)
By substituting
Eqs . B-28 and B-29 into Eq.
"'"
(B-29)
B-13,
which is also K (t , T) for this partic ular example.
y
vre
"'"
find K"""(t,T)
x
Furthermore,
we
300
find the auto- and cross -covariance
applyting Eqs.
B-lB and B-l9
of the individual
components
by
respectively.
Let us now consider
the stationary
cas e in more detail.
In this case
Po
=
P.
(B-30)
If we perform
the indicated
substitutions,
we obtain
fOol
Pe
-k zxt
At ~
0
At:S
0
=
K~At)
x
This may be written
(B-3l)
as
(B-32)
By applying Eqs.
K
B-lB and B-l9
x x (L>.t) = Kx x (L>.t) = ~ e - Re[
c c
K
we find
x x
c s
k] IL>.t Icos(Im[ k] L>.t)
(B - 3 2a)
s s
(At)
The spectrum
=
I
P e - Re[ k] At I sin(Im[ k]~)
2
of the complex process
follows easily
(B-33b)
as
3.01
= __
S""( w)
x
(w-Im[
The spectra
in terms
2_R_e~[-=k]=.-P
__
----=k] ) 2 + Re[ k] 2
and cross
spectra
of xc(t) and xs(t) may easily be found
of the even and odd parts
From Eq. B-34,
(B-34)
of S:(w).
x
we see that in the stationary
case,
net effect of allowing a complex gain is that the spectrum
has a
fre quency shift equal to the imaginary
part of the gain.
band interpretation
this would correspond
about the carrier.
In general,
interpretation
suggests
In a narr ow
to a mean Doppler shift
we would not expect such a simple
of the effect of a complex state representation.
that we should consider
second order system
the
This
a second example where we have a
and two feedback gains.
Example ~
In this example we want to analyze
In the steady state,
it corresponds
that the pole locations
of one another.
analysis
with two poles.
Again, we shall analyze the stationary
informative
Note
-k1 and -k need not be complex conjugates
2
for the non- stationary
especially
to a system
a second order system.
case is straightforward,
The state and observation
case,
the
but is not
eq uations for this
system are
(B -3 5)
302
and
y(t)
=
[1:0]
(B -3 6)
The covariance
of the driving noise is
(B-37)
From the steady state solution to Eq. ,B-1Z, we have
(
jIm[
I
I
1
,
Re[k1
I
,...,
p = p
o
k1 kz ]
+kz]
J
----------1-------------------------------------jIm [k)~z]
-----
Re[
k 1+kz]
: (Re(k lkz)Re[ k 1+kz] +Im(k1 kz)Im(kl +kz))
I ----------------
I
Re[
1
k 1+kz]
(B-38)
,...
Therefore,
we have a stationary
process
In order to find the covariance
transition
matrix
Laplace transform
for the system.
techniques.
y(t) with power P.
matrix we need to find the
We do this by using matrix
We find.
303
(B-39)
If we substitute
Eq. B-1?,
.....
K .....
(~t)
Y
Eqs.
B-38 and B-39 in Eq. B-13 and then use
we obtain
=
(B-40)
We now want to determine
define two coefficients
the spectrum
S (w) from Eq. B-40.
y
Let us
for convenience:
Re[k ]
l
(B -41)
Re[k 1+kl]
k*+
l k1
•
(B-4l)
304
After a fair amount of manipulation,
sy (w) =
we can compute S (w).
y
zp
Re[AZ] Re[kz]
Z ,..,
z]
Re [k
,..,
z])
Z
z])
+ (w+Im[ k
We have plotted this function for various
,..,
,..,
kZ in Figure
particular
B-1 through B-4.
The values
figure are illustrated
of k
l
on the figures
i. e.,
the system
In Figures
of k 1 and
for a
Z
by the pole location
,..,
~as a pole at -k 1 and -k .
Z
B-1 and B-Z we illustrate
choosing the poles as complex conjugates
spectrum
(B-43)
.
values
,..,
and k
,..,
they produce,
)
+ Im[Az] (w+Im[k
that by simply
we can produce
either
very flat near w = 0 or is peaked with two
which is either
...,
symmetric
generate
lobes.
If one wanted to use real state variables
this spectrum,
In this case,
computation
a fourth order
system
would be required.
the complex notation has significantly
reduced
the
required.
Figure
B-3 illustrates
mean Doppler shifts.
complex system.
zero pattern
exists
is symmetric,
produces
For example,
about w = l/Z.
an interesting
Let us draw the pole-zero
If there
notation effectively
w= w .
c
to
"
a line w
in Figure
locations
= wc about
then in the stationary
a spectrum
observation
for the
which the pole-
case the complex
which is symmetric
B- 3, the pole pattern
We see that the spectrum
about
is symmetric
about
is symmetric
about
•
305
w
=
-1/2 als o.
Consequently,
we can use the complex notation to
introd uce mean Dopp ler shifts of narrow band proce sses .
Figure B -4 illustrates
are not symmetric
case.
about any axis.
that, we can obtain spectra
This is a relative ly important
In dealing with narrow band processes
frequency
about which the spectrum
processes
y (t) and y (t) are uncorrelated.
c
axis of symmetry
correlated.
that we can model narrow band processes
spectra
very conveniently
then the component
However,
(or if the choice of carrier
are definitely
if one can find a
is symmetric,
s
then the components
which
if there is no
is not at our disposal)
This example shows
with non-symmetric
with our complex state variable
notation.
306
Fig. B-1
Spectrum for a Second Order
Complex Process when Poles
are Closely Spaced
l'
i
j.
t
tt·ilt:
;
t- t ,
··1
1-+
'I
j
+
~.
-+
I;·
,.,.+
j
+ +-
It HHU
f
I++i
itt
t ~+
t
.r:
,1.t
~T"~-t""'j.j
-2
-.1.
o
'3
307
.r
Fig. B-2
Spectrum for a Second Order
Complex Process when Poles
are Widely Spaced
1-
1+
1$
t
-'-I.
3.
-z
.1..
o.
U)~
308
Fig. B-3
Spectrum for a Second Order
Complex Process with a Mean
Doppler Frequency Shift
- z.
_.......
0
1..
~ ---t-::1>~
1.
3
4
309
Fig. B-4
Spectrum for a Second Order
Complex Process with Nonsymmetrical Pole Locations
, "
"
"!
):,l~J
.....
1
It
tt+ '
rL + -t-'
1ft 1+
ffl.·
+
Jij
8=
j
j~l'
ltfl
Hif
fUll'~
w---- ......
310
REFERENCES
1.
R. Courant and D. Hilbert, Methods of Mathematical
Vol. I, Interscience
Publishers,
New :"york, 1953.
2.
W. V. Lovitt, Linear Integral
New York, 1924.
3.
H. L. Van Trees, Detection, Estimation and Modulation Theory,
Part I, John Wiley arid Sons, New York, 1968.
4.
C. W. Helstrom, Statistical
Pe rgamon, London, 1960.
5.
P. M. DeRusso, R. J, Roy, and C. M. Close, State Variables
for Engineers,
John Wiley and Sons, New York, 1965.
6.
M. Athans and P. L. Falb,
New York, 1966.
7.
L. Zadeh and C. Dosoer,
New York, 1963.
8.
R. E. Kalman and R. Bucy, "New Results in Linear Filtering
and Prediction Theory, II A. S. M. E. Journal of Basic Engineering,
Vol. 83, March 1961, 95-108.
9.
F. Schweppe, "Evaluation of Likelihood Functions for Gaussian
Signals," IEEE Transactions
on Information Theory, Vol. IT-II,
No.1, January 1965, pp. 61-70.
10.
D. Snyder, "The State Variable Approach to Continuous
February 1966.
Estimation,"
M. 1. T., Ph. D, Thesis,
Equations,
Physics,
McGraw-Hill,
Theory of Signal Detection,
Optimal Control,
Linear
McGraw- Hill,
System Theory,
McGraw- Hill,
11. W. Davenport and W .. Root, Random Signals and Noise,
McGraw-Hill,
New.Yqrk, 1958.
12. H. L. Van Trees,
Detection, Estimation,
and Modulation Theory,
Part II, John Wiley and Sons, (to appear).
13.
L. S. Pontryagin, V. G. Boltyanski, R. Gamkre1idze, and
E. Mishenko, The Mathematical Theory of Optimal Processes,
Interscience
Publishers,
New York, 1962.
311
14. A. E. Bryson and M. Frazier,
"Smoothing for Linear and
Nonlinear Dynamic Systems", Proceedings Optimum Systems
Conference,
Wright - Patterson Air Force Base, Sept. 1962.
15. H. Rauch, F. Tung, and C. Striebel," Maximum Likelihood
Estimates of Linear Dynamic Systems", AIAA Journal,
Vol. 3, No 8, August 1966.
16. A. Baggeroer," Maximum Aposteriori Interval
1966 WESCON Convention Record, Paper 7-3,
Estimation, "
August 1966.
17. D. 'M. Detchmendy and Sridar, "Sequential Estimation of
States and Par-ameter-s in Noisy Dynamical Systems, It A. S. M. E.
Journal of Basic Engineering,
June 1966, pp. 362-368.
of Vector- Valued
18. E. J. Kelley and W. L. Root, It A Representation
Random Processes,"
Group Report 55-21, (revised ),
April 22, 1960.
M. 1. T. Lincoln Laboratory,
19. J. Capon, "Asymptotic Eigenfunctions and Eigenvalues of a
Homogeneous Integral Equation," IRE Transactions
on
InformatioD.JTheory, Vol. IT-8, No.1,
Jan.L 1962, pp.
20. R. Y. Huang and R. A. Johnson, "Information Transmission
with Time Continuous Processes,"
IEEE Transactions
on
Information Theory, Vol. IT_-9, No.2,
April 1963, pp. 84-95.
21. E. T. Whitaker and G. N. Watson, A Course of Modern Analysis,
Cambridge University Press,
Cambridge, England, 1962.
22. L. D. Collins, "Evaluations of the Fredholm Determinant
for State Variable Covariance Functions,"
accepted for
publication in IEEE Proceedings,
Decenber 1967.
23. M. Athans and F. C. Schweppe, "Gradient Matrices and Matrix
Calculations,"
Technical Note 1965-53, M. 1. T. Lincoln
Laboratory,
1965.
24. L. D. Collins, "A Simplified Derivation of the Differential
Equation Solution to Fredholm Integral Equations of the Second
Kind," Internal Memo No. IM-LDC-19, Detection and
Estimation Theory Group, Resaerch Laboratory of Electronics,
M. 1. T., Cambridge, Mass., January 1967.
312
25. L. A. Zadeh and K. S. Miller,"Solution
to an Integral
Equation Occuring in the Theories of Prediction and Detection, "
IRE Transactions
in Information Theory, Vol. IT-2, No.3,
June 1956, pp.72-76.
26. D. Youla, "The Solution of a Homogeneous Wiener -Hopf
Integral Equation Occuring In the Expansion of Second Order
StationaryRandom Functions, ~' IRE Transactions
on
Information Theory, Vol. IT'-3, No.4, Sept. 1957, pp.187-l93.
27.
C. W. Helstrom,
"Solution of the Detection Integr-al. Equation for
Stationary Filtered White Noise, " IEEE Transactions
on
Information Theory, Vol. IT-II, No.3,
July 1965, pp. 335-339.
28. H. L. Van Trees,
Private
communication.
29. TH. ~c1. Kushner and F. C. Schweppe"." A. Maximum Principle
fer .,Stoc.ha.stic,Contr91.SY$tems,"
Journal of Mathematical
Analysis and Applications,
Vol. 8, pp.287-302,
1964.
30. M. Athans, "Solution of the Matrix Equation
X(t) = A(t)X(t) + X(t)B(t) + U(t)," Technical Note 1965-26,
M.1. T. Lincoln Laboratory,
June 1965
31. J. S. Meditch, "On Optimal Linear Smoothing Theory, "
Technical Report 67-105, Information Processing
and Control
System Laboratory,
Northwestern University,
March 1967.
32. A. Baggeroer, "Fortran Programs for a) Solving Fredholm
Integral Equations, b) Optimal Signal Design For Additive
Colored Noise Channels, c) Calculating Smoother and Filler
with Delay Covariances of Error,"
Internal Memo, Detection
and Estimation Theory Group, Research Laboratory for
Electronics,
M. 1. T. (to appear).
33. R. Bellman, R. Kalaba, and R. Sridhar, "Sensitivity Analysis
and Invariant Imbedding," Research Memorandum, No. 4039-PR,
March 1964.
.
34.
.
.
.-
P. R. Garabedian,
partial Difff.61r<miUal
E(1U1Rbi~AA'
John Wfl.ey
and Sons, New York, 1964
35. A. C. Butkovskii and A. Y. Lerner, "The Optimal Control of
Systems with Distributed Parameters,
" Automatika i Telemekhanika, Vol. 21, No.6,
June 1960, pp. 682-692
7
313
36. A. <e:: Butkovskii, "The Maximum Principle for Optimum
If Automatika
i
Systems with Distributed Parameters,
Telemekhanika"
VoL 22, No.9,
September 1961, pp.
1288-1301
in Systems With Distributed
37. A. Butkovskii, If Optimum Processes
Parameters,"
Automatika i Telemekhanika,
VoL 22, No.1
pp. 17-25, January, 196L
38. A. Butkovskii, "Some Approximate Methods For Solving
Problems of Optimal Control of Distributed Parameter
Systems,
Automatika i Telemeknanika,
VoL 22, No. 12, pp. 1565-1575,
December, 1961.
39. Courant and Hilbert, Methods of Mathematical
Interscience
Publishers,
New York, ·1962.
Physics,
Vol. II,
40. P. K. C.W.~ng, "Control of Distributed Parameter
Systems, "in
Advances in Control Systems, C. T. Leondes (Editor), Academic
Press,
New York, 1964.
41. P. K. C. Wang and F. Tung, "Optimum Control of Distributed.
Parameter
Systems," ASME Journal of Basic Engineering,
pp. 67-79, March 1964.
42. Y. Sakawa, "Solution of an Optimal Control Problem in a
Distributed Parameter
System," IEEE Transactions
on
Automatic Control, October 1964, pp. 420-~70.
of Systems with Distributed
43. R. Goodson, If GPtimalControl
Parameters,
"(JA.CC Proceedings,
June 1964, pp. 390-397.
44. P. Pierre,
"Minimum Mean Square Error Design of Distributed
Parameter
Control Systems,"
JACC Proceedings,
June 1964,
pp. 381-389.
45. M. Lasso, "The Modal Analysis and Synthesis of Linear
Distributed Control Systems,"
Ph. D. Thesis, Department of
Electrical Engineering,
M. 1. T. , Camridge, Massachusetts,
June 1965.
46. H. Kwakernaak, "Optimum Filtering in Linear Systems with
Time Delays, " IEEE Transactions
on Automatic Control,
VoL AC-12, No.2, April 1967, pp.169-l73.
If
7'
'T -- ". Emr
mrWT
I'TrneFTWrr
314
BIOGRA.PHY
Arthur
Massachusetts
Memorial
B. Baggeroer
on June 9, 1942.
High School,
was born in Weymouth,
He graduated
Wakefield,
Massachusetts.
the B. S. E. E. degree from Purdue University
and E. E. degrees
1968.
While at
Graduate
in 1963, the S. M.
M.1. T. he was a National Science Foundation
Engineering
Mitre Corporation
and the summers
Lincoln Laboratory.
Currently) he is an Instructor
at M. 1. T.
He spent the summers
of 1959 to 1965 working for the
of 1966 and 1967 at M.1. T. 's
He has presented
and the 1967 International
Conference
in the area of the application
papers
problems.
KappaNu,
Sigma Xi and IEEE.
He is a member
He is married
and currently
at the 1966 vyESCON
on Communication
of state variable
munication
two children,
He received
from M. 1. T. in 1965 and the Sc , D. degree in
Fellow from 1963 to 1966.
in Electrical
from Wakefield
techniques
Technology
to com-
of the Tau Beta Pi, Eta
to the former
Carol Coleman,
lives in Watertown,
has
Massachusetts.
Trrr'PWrmrrr? Tn