On The Application of Minimum Phase Space Volume Parameter Estimation
Dejan Djonin, Ljiljana Stanimirović
Institute Mihajlo Pupin, Volgina 15, Belgrade, YU
Abstract
In this paper, we consider the influence of high dimensional
noise process on the accuracy of signal parameter
estimation in low dimensional chaotic noise. Because of the
inherently deterministic nature of the chaotic signal, instead
of conventional probabilistic methods, a complexity
measure based on phase space volume (PSV) of the
reconstructed attractor is used to identify unknown system
parameters. It was shown that through minimization of
PSV a very effective system identification procedure could
be achieved. This procedure however relies upon the fact
that PSV of the chaotic process is negligible for embedding
dimensions higher than the true dimension of the chaotic
attractor, therefore any additional high dimensional noise
degrades the estimation accuracy. Monte Carlo simulations
are carried out to illustrate the efficiency of the minimum
PSV method for parameter estimation in the presence of
high dimensional noise. To reduce the optimization
complexity a kd-tree search algorithm was used which takes
only order Nlog(N) operations. Also, a modified
optimization procedure based on false nearest neighbor
(FNN) test is introduced to improve the robustness of the
estimation procedure.
Introduction
Nonlinear signal processing applications have recently drawn a
great deal of attention of the researchers. A considerable
number of very diverse but potentially awarding chaotic signal
processing applications have been published. Wide range of
signals such as radar signal [1], speech [2] and indoor multipath
signals [3] have been proved to be chaotic and inherently
deterministic. This deterministic nature of same real life signals
can be used to enhance conventional probabilistic signal
processing methods. Based on this determinism new methods
for classification, modeling, synchronization and control of
chaotic signals have been introduced [4],[5],[6].
On the other hand several efforts have been made to utilize
chaotic signals in signal synthesis for applications in
communications. Chaotic signals have proved useful for
cryptography, chaotic modulation [7],[8] and construction of
good error-correction codes [9]. All these applications
essentially depend on the ability to estimate certain parameters
of useful non-linear signal in chaotic or in purely random noise.
One important step toward the solution of that general problem
is maximum likelihood estimation of initial condition of a
chaotic discrete-time signal in noise [10],[11]. Several different
approaches for improving the quality of chaotic signal
contaminated with purely random noise have also been devised
[14],[15].
In this paper, a dynamic-based parameter estimation technique
introduced in [12],[13] called minimum phase space volume
(MPSV) is further investigated and modified. This method can
be applied on problems of estimating parameters of signals
embedded in chaotic noise. We consider here a much more
realistic situation where our signal of interest is corrupted with
the mixture of chaotic signal and purely random signal (high
dimensional noise). The basic idea of the original estimation
procedure is to exploit the finite dimensionality of chaotic
signal. Although it appears to be quite random in the time
domain, chaotic signal is attracted to stay in a finite
dimensional attractor manifold and hence has a finite “volume”
in phase space (PSV). Purely random signals however do not
have any regular behavior in finite low dimensional phase
space and hence its volume is expected to be relatively large. In
this context we can regard the conventional random signals as
very high dimensional chaotic signals.
The core of the MPSV algorithm is the minimization of the
PSV of the inversely filtered signal by adjusting its parameters.
It was shown that this procedure produces very efficient
estimates and that it was so robust that it did not require an
order determination procedure. However, for equalization
purposes [13] or some other real life estimation problems
MPSV algorithm should be tested in situations where chaotic
signal is contaminated with some high dimensional noise.
Monte Carlo simulations were carried out for various signal to
noise ratios. It was shown that minimum square error of the
parameter estimates quickly rises as the noise variance
approaches the attractor dimension. A threshold effect, noticed
in some other nonlinear estimation problems [10], occurs here
too.
To improve the accuracy of the estimates in presence of high
dimensional noise a modified method based on the false nearest
neighbor test (FNN) is used. FNN has proved to be very robust
method for embedding dimension evaluation of a chaotic signal
in the presence of noise.
The Minimum Phase Space Volume Parameter Estimation
According to the Takens embedding theorem [16] time delay
reconstruction can be used to reconstruct the dynamics of a
chaotic attractor. We can work in the “reconstructed” time
delay space and learn essentially as much as we could from the
“true” state space providing that our embedding dimension is
sufficient to unfold the attractor. The correct embedding
dimension of an attractor is chosen when points on the attractor
are near each other because of the dynamics and not by the
projection form higher dimension. This can be easily quantified
by using FNN test [4],[6].
Parameter estimation problem analyzed in this paper can be
formulated as
xt  s t  0   ct  nt
(2)
A practical approximation of PSV
N
J 1 u t ,    min u i  u j    u i  d 1  u j  d 1 (3)
j i
was used in [12],[13] to avoid complex computation needed by
the original PSV definition. Instead of using PSV as an
optimization function a modified function based on false
nearest neighbor test [6] can be used
J 2 u t ,  
d
uid  u j d
N

i 1
min
j i
u
 u j     u i  d 1  u j  d 1 
2
i
.
(4)
2
This criterion tells us if the nearest neighbors of the attractor
points as seen in dimension d are near or far in dimension d+1.
The search for nearest neighbors can be made feasible by using
kd-tree search algorithm which takes order N log(N) operations
to establish neighbor relationships among N points [17].
It should be noted that for chaotic dE dimensional signal ct,
J1d(ct) = 0 for d > dE. The same can be concluded for the
percentage of false nearest neighbors. FNN for a point i on the
attractor is declared whenever
u i  d  u j  d / min
i j
u
i
uj

2

   u i  d 1  u j  d 1

2
is greater then some fixed threshold. For illustration purposes
percentages of FNN for three different signals is shown in
Fig.1. It can be seen that random noise contamination of 0.1 of
the size of attractor only slightly degrades the accuracy of the
FNN percentage estimate.
The effectiveness of the MPSV and the FNN technique in the
presence of high dimensional noise was evaluated in the
problem of AR parameter estimation
p
xt   ai xt i  ct  nt
i 1
(6)
(5)
as was used in [12], [13], [7]. Minimum square error of the
parameter estimates was averaged over 100 trials. The
simulations were conducted for several signals to noise ratios
and number of points for both described optimization
procedures.
Simulation results will be shown in the full paper.
100
80
FNN (%)
ut  xt  st    st  0   st    ct  nt .
i 1
ct  ct 1 1  ct 1 
(1)
where st is the signal of interest, ct is chaotic signal and nt is
additive high dimensional noise, t = 1,…,N. The problem is to
identify parameter 0 by analyzing measured known signal xt.
This can be done by introducing optimization function in terms
of measured signal xt. Let dim(xt) denote minimal sufficient
embedding dimension of a certain signal xt. In the absence of
noise it was shown [12] that if dim(xt)<dim(st()) the correct
value of parameter  can be estimated by PSV minimization of
the inversely filtered signal
d
The chaotic signal ct is chosen to be logistic map
60
40
20
2
4
dE
6
8
10
Fig. 1 False Nearest Neighbor ratio in terms of embedding
dimension for Lorenz system (dots), Random Noise (solid line),
and chaotic signal corrupted with high dimensional noise
(dashed line). Noise signal was uniformly distributed signal in
the interval [-L,L] and populated with conventional computer
generated random numbers. L/RA= 0.1 was chosen, where RA is
the average radius of the attractor. N = 25000 points were used
for this calculation.
References
[1] S.Haykin, X.B.Li, "Detection of Signals in Chaos",
Proceedings of the IEEE, vol. 83, No.1, pp. 95-122, Jan.
1995.
[2] A. Kummar S.K. Mullick, “Attractor dimension, entropy
and modulation of speech time series”, Electron. Lett., vol.
26, no. 21, pp. 1524-1527, 1994.
[3] C.Tannous, R.Davies and A.Angus, “Strange attractor in
multipath propagation”, IEEE Trans. on Commun.,vol.
39,no. 5, pp. 629-631, 1991.
[4] H.D.I.Abarbanel, T.W.Frison, L.S.Tsimring, "Obtaining
Order in a World of Chaos.", IEEE Signal Processing
Magazine, vol. 15,.No. 3,pp. 49-66, May 1998.
[5] L.M.Pecora and T.L.Caroll, “Synchronization in chaotic
systems”, Phys. Rev. Lett. Vol. 64, no. 8, pp. 821-824,
1990.
[6] H.D.I. Abarbanel, “Analysis of Observed Chaotic Data”,
Springer – Verlag New York, 1995.
[7] H.Leung, J.Lam, "Design of Demodulator for the Chaotic
Modulation Communication System", IEEE Trans. on
Circuits & Systems I, vol. 44.,No. 3, March 1997.
[8] B.Chen, G.Wornell, “Analog Error-Correction Codes
Based on Chaotic Dynamical Systems”, IEEE Trans. On
Communications, vol. 46, No. 7, July 1998
[9] D.Đonin, D.Gaćeša, "Performances of Error-Correction
Codes Generated by Iterative Non-linear Mappings",
Proceedings of ISSSE'98, Pisa, Italy, Oct. 1998.
[10] S. Key, "Asymptotic Maximum Likelihood Estimator
Performance for Chaotic Signals in Noise", IEEE Trans.
on Signal Processing, vol. 43, No. 4, Apr. 1995.
[11] H.C. Papadopuolos, G.W.Wornell, “Statistical analysis and
spectral estimation techniques for one-dimensional chaotic
signals”, IEEE Trans. on Information Theory, Vol. 45, pp
312-317, Jan 1995.
[12]
H. Leung, X. Huang, Parameter Estimation in Chaotic
Noise, , IEEE Trans. on Signal Processing, vol. 44, No. 10,
October 1996.
[13] H.Leung, “System Identification Using Chaos With
Application to Equalization of a Chaotic Modulation
System”, IEEE Trans. on Circuits and Systems, vol. 45,
No. 3, March 1998.
[14] J.D.Farmer, J.J.Sidorowich, "Optimal Shadowing and
Noise Reduction", Physica D 47, pp. 373-392, 1991.
[15] C.Lee, D.B.Williams, "Generalized Iterative Methods for
Enhancing Contaminated Signals", IEEE Trans. on
Circuits and Systems I, Vol. 44, No. 6, June 1997.
[16] F.Takens, "Detecting Strange Attractor in Turbulence", in
Dynamical Systems and Turbulence, Warwick 1980.
[17] R.F.Sproull, “Fast kd tree search”, Algorithmica,6:579589, 1991.
- 30
- 35
- 40
- 45
- 50
- 55
0. 1
0. 2
0. 3
0. 4
0. 5
- 0. 18
- 0. 185
- 0. 19
- 0. 195
- 0. 2
- 0. 205
10
20
30
10
20
30
40
50
0. 97
0. 965
0. 96
0. 955
0. 95
0. 945
0. 94
0. 935
40
50