IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT, VOL. 45, NO 1, FEBRUARY 1996
12
Measurement of Noise (Cross-) Power
Spectra for Frequency-Domain
Identification Purposes: Large-Sample
Rik Pintelon, Member, IEEE, Patrick Guillaume, Member IEEE, and Johan Schoukens, Senior Member, IEEE
Abstract-Several frequency-domain estimators of parametric
transfer function models assume that the (cross-) power spectra
of the disturbing noise sources are known [l].This paper presents
a time-efficient method to measure these noise (cross-) power
spectra and studies its influence on the asymptotic properties of
the estimated model parameters.
anti-alias
filter
I. INTRODUCTION
HE basic experimental setup used in frequency-domain
identification of parametric transfer function models is
shown in Fig. 1. Since both the input and the output of
the device under test (DUT) are measured, the appropriate
stochastic description of the setup is an errors-in-variables
model [2], [3]
T
Fig, 1. Experimental setup used in frequency-domain system identification.
I
disturbing noise sources N X (f ) , NY (f )
where X ( f ) is the true (unknown) input spectrum H ( f , P )
the parametric transfer function model, P the vector of the
model parameters, and LPx (f),LPy ( f )the anti-alias filter
characteristics. N y ( f ) and N x ( f ) are zero mean, jointly
correlated, independent (over the frequency) complex random
variables (see Appendix A for a definition of complex noise)
which are related to the generator noise N,(f),
the process
and the measurement noise sources M x ( f ) ,
noise Np(f),
M Y ( f ) as
Several frequency-domain estimators of the model parameters
P , such as the generalized total least squares [4], bootstrapped
total least squares [5],the maximum likelihood [6] . . . (see
are known at the frequencies of interest. Equation (3) is called
a nonparametric noise model. Note that if LPx ( f )# LPy ( f )
then the measured output spectrum Y, (f ) , the variance & ( f )
and the covariance p ( f ) should be compensated by the ratio
LPx( f ) / L P y( f ) in the following way:
Ym(f) Ym(f)LPX(f)/LPY(f)
+
[l] for an overview), assume that the (co)variances of the
Manuscript received October 26, 1994; revised May 9, 1995. This work was
supported by the Belgian National Fund for Scientific Research, the Flemish
cpmmunity (Concerted Action IMMI), and the Belgian government (IUAP
50).
The authors are with Vrije Universiteit Brussel, Department ELEC, 1050
Brussels, Belgium.
Publisher Item Identifier S 0018-9456(96)00847-9.
P(f)
+
P(fWx(f)/LPy(f).
(4)
The ratio L P x ( f ) / L P y ( f )can easily be obtained via a
relative calibration of the acquisition channels. Note that
0018-9456/96$05.00 0 1996 IEEE
Authorized licensed use limited to: Rik Pintelon. Downloaded on December 1, 2008 at 09:02 from IEEE Xplore. Restrictions apply.
13
PINTELON et al.: MEASUREMENT OF NOISE (CROSS-) POWER SPECTRA
systems identified in a feedback loop also fit within the
stochastic framework (2) since it allows correlation between
the input and output disturbances.
In [2] and [ 3 ] ,the noise (cross-) power spectra are measured
by removing the excitation signal ( X ( f ) = 0). During this
noise measurement no information about the device under test
(DUT) is gathered. The total measurement time equals the
time necessary to measure the DUT ( X (f ) # 0) and the time
required for the noise analysis ( X ( f ) = 0). In this paper, it
will be shown that using periodic excitations ( X ( f ) # 0) it
is possible to measure simultaneously the DUT and the noise
(cross-) power spectra. Hence for a given measurement time
the new procedure will acquire more information about the
system (1) AND the noise model ( 3 ) than the former method.
As it will be explained in the paper the new method consists
of calculating the sample mean and the sample (co)variance of
correlated (not stochastically independent) experiments. While
the statistical properties of the sample mean and the sample
(c0)variance of INDEPENDENT experiments are well known [ 7 ] ,
no results for the DEPENDENT (correlated) case are available.
The contributions of the paper, hence, are
1) time-efficient measurement of noise (cross-) power spectra in a frequency-domain system identification context,
2) statistical analysis of the sample mean the sample
(co)variance of DEPENDENT experiments, and
3) study of the influence of the measured noise model on
the estimated model parameters.
Note that throughout the paper it will be implicitly assumed
that the disturbing noise is stationary.
11. MEASUREMENT
OF THE NOISEPOWER SPECTRA
B. New Approach
The new approach consists of a statistical analysis of M
periods of the input and output signals. The
discrete Fourier transform (DFT) of eaclh period is calculated,
resulting in M DEPENDENT samples
y?)(f) (i =
1, 2 , . . . , M ) of the input and output spectra X m ( f ) , Y m ( f )
at the frequencies of interest. The sample (co)variances are
used as estimates of the noise model (3)
CONSECUTIVE
~-k)(f),
.
M
a=1
. ( V $ ( f ) - m v ( f ) ) * (7)
where V and W equal X and/or Y , and with m v ( f ) (V =
X , Y ) the sample mean of respectively the input and the
output spectrum
DEPENDENT samples
It is reasonable to assume here that the
N $ ) ( f ) , N $ ) ( f ) (i = 1, 2 , . . . , M ) of the disturbing errors
N X ( f ) ,N y ( f ) are jointly mixing of order 4 (see Appendix A,
Definitions 3 and 4). Intuitively a mixing condition means that
the span of dependence is 'limited', for example, the class of
filtered white Gaussian noise sequencer; is mixing of infinite
order. The mixing condition on the frequency-domain noise
implies the same mixing condition on the time-domain noise
[8]. The sample mean (8) of the M DEPENDENT samples is
still an unbiased estimate of the true mean value V ( f )
E h "
=V(f)
(V = X,Y ) .
(9)
This is no longer true for the sample (co)variances (7)
A. Classical Approach
+
E{cwv(f)} = a w v ( f ) O(1/M)
(V, w = x,Y )
In the classical approach, the noise (cross-) power spectra
are measured when no excitation is applied to the DUT
( X ( f )= 0). The measurement procedure consists of acquiring
M time records of the input and output disturbances and
calculating the sample (co)variances of the corresponding
discrete Fourier spectra
'(10)
where O( 1/M) is a deterministic function that tends to zero as
l/M for M -+ cc (proof: see Appendix B). Under the mixing
condition of order 4 the sample (co)variances and the sample
mean converge in the mean square sense to their expected
value with a convergence rate of 1/J?i7
where V and W equal X and/or Y . Assuming that INDEPENDENT time records are gathered, the calculated noise
Fourier spectra N $ ) ( f ) , N $ ) ( f ) (i = 1, 2 , . . . ,M ) are
INDEPENDENT samples of the jointly correlated noise sources
N X (f ), N y ( f) . If in addition the fourth-order moments of the
noise exist, then it is well known that the sample (co)variances
(5) are unbiased estimates which converge ( M
cm)in the
mean square sense (see Appendix A, Definition 5) to their
expected value with a convergence rate of 1/d% [71
(proof for the sample mean: apply Theorem 1, Appendix C to
(8); proof for the sample (co)variances: first note that a linear
combination of mixing random variables, is mixing of the same
order [8] and next apply Theorem 2, corollary to Theorem 2
and Theorem 1 to (7)).
It is well known that the (co)variances of the sample means
of M INDEPENDENT samples equal the (c0)Yariances of one
sample divided by M [7]. This is not longer true if the samples
(i = 1, 2, . . . , M ) are correlated. Terms in addition to cwv ( f )
appear which do not vanish for M large
-+
( O m s ( l / a ) is a random variable which converges in the
mean square sense to zero as 1/A2 for M + 00).
(V,
w = x,Y )
Authorized licensed use limited to: Rik Pintelon. Downloaded on December 1, 2008 at 09:02 from IEEE Xplore. Restrictions apply.
(12)
IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT, VOL. 45, NO. 1, FEBRUARY 1996
14
with
then the series (18) limited to the K first terms
Here o g ; ) ( f ) is the (cross)-correlation between the noise
samples IV$)(~),
IV$)(~)(i = 1 , 2 , . . . , ~ )
is a consistent estimate of the series (13) limited to the K
first terns
K
(k)
k = 1, 2 , . . . , M - 1
o w v ( f )= E{N$+”(f)(N$)(f))*}
ab$)(f)
= E { N $ ) ( f ) ( N F + ” ( f ) ) * }i = 1, 2 , . . . ,M
-
k
(14)
(proof: see Appendix D). Note that o&$’ ( f ) = (o?; (f))’ .
The sample (cross)-correlations are used to estimate the terms
( k = 1, 2 , . . . , M - 2) in the series (13)
og;’(f)
~
(15)
(cL-,f“)(f) = ( $ $ ( f ) ) * , and V, W = X, Y ) . Note that no
&(M-1)
useful value of cWv
( f )can be calculated (only one sample
available). The calculation of the expected value and the mean
square convergence proof of the sample (cross)-correlations
(15) is done in exactly the same way as for the sample
(co)variances (7). It follows that
= &)(f)
+ 0(1/M)
IC
= 1, 2, ’ . . , M
2
(16)
-
and
CF$)(f)
=
(V,W = X , Y ) .If (19) is not fulfilled then a trade off should
be made between the bias and the variance of the truncated
series b w v , ~ ( (1
f ) 5 K < M-1). The following estimates of
the (co)variances of the sample means (12) are hence proposed
1
Cwv(f)= ~ ( C W V ( S ) b w v , ~ c ( f ) 1) I K < M - 1
(22)
with
+
\
M-k
(v$?k+z)
( f m v ( f ) )*
E{cG,)(f)}
k=l
.G$)(f) + o m s ( l / m )
k=1,2,...,M-2
(17)
(V,W = X , Y ) .Although c G $ ) ( f ) (IC fixed) is a consistent
estimate of o$$) ( f ) ,the variance of the series
(V,W = X , Y ) .In Section IV, it will be studied how many
significant terms in each series (20) can be identified.
Note that the results of this section also apply to the special
case where one analyzes the disturbing noise during the dead
time in between two consecutive bursts of the input and output
signals.
C. Alternatives
To avoid the problems related to the correlation of neighboring signal periods, one could think of using every second
or third period or of making repeated INDEPENDENT measurements. If the measurement time is not important then these are
indeed the easiest solutions. However, if the goal is to gather as
much information as possible concerning the device under test
in a given measurement time, then these easy solutions will
result in smaller signal-to-noise ratios. The total uncertainty
on the estimated model parameters is a combination of the
uncertainty on the sample means (8) and the uncertainty on
the sample (co)variances (22). It turns out that the first effect
(signal-to-noise ratio) is more important, and hence, the easy
solutions will result in larger parameter uncertainties compared
to the method proposed in Section.11.
EI.
c$F-~)(~)
cF$)(f) = 0
+
Q k >K
(19)
MEASURED
NOISE MODEL
ESTIMATED
MODELPARAMETERS
INFLUENCE OF THE
ON THE
does not decrease to zero as M tends to infinity. This can
easily be verified by noticing that the variance of each of the
Q last terms
( q = 2, 3 , . . . ,Q 1) in the series
(18) is 0(1/ q ) , independent of M . Hence, bwv ( f ) (18) is an
inconsistent (in the mean square sense) estimate of Pwv(f)
(13). More restrictive noise assumptions are necessary to
ensure consistency of the estimate (18). For example, if for
IC sufficiently large the (cross-correlation) of the experiments
is zero
I
Several frequency-domain estimators of the node1 parameters P , like the genralized total least squares (GTLS) [4],
the bootstrapped total least squares (BTLS) [5],the iterative
quadratic maximum likelihood (IQML) [l], and the maximum likelihood (ML) [2], [ 3 ] , require the knowledge of
the noise model (3). Their asymptotic (number of spectral
lines F t 00) properties (consistency, asymptotic efficiency,
asymptotic normality) are studied assuming that the noise
model is exactly known [l]. In this section, the influence of
the measured noise model on the estimated model parameters
is studied through the (equivalent) cost function L F ( . )of the
identification method.
Authorized licensed use limited to: Rik Pintelon. Downloaded on December 1, 2008 at 09:02 from IEEE Xplore. Restrictions apply.
PINTELON et al.: MEASUREMENT OF NOISE (CROSS-) POWER SPECTRA
15
A. Classical Approach
TABLE I
In the classical approach, the measured noise model ( 5 )
is stochastic independent of the measured input and output
spectra (1). Convergence in probability of the cost function
with the measured noise model ( 5 ) to the cost function with
the EXACT noise model (3) follows immediately:
P1imM-w LF(P, V“,
.wv(f))
= LF(P, Vm(f),
.wv(f))
(24)
where V , W equals X andlof Y (proof use the interchangeability of the probability limit and a continuous function,
and the fact that convergence in the mean square sense (6)
implies convergence in probability [9]). The convergence rate
is O p ( l / m ) ( O p ( l / m ) is a random variable which
converges in probability to zero as 1 / a for M -+ CO);
the proof is analogous to that in Appendix E. Hence, the
asymptotic ( F -+ CO) properties are still valid within an
0p(1/v‘Z) term.
B. New Approach
Here, the sample mean of the input and the output spectra
(8), and the corrected sample (co)variances (22) are fed to
the frequency-domain identification methods. The difficulty
in the analysis of the (equivalent) cost functions is that the
sample means are correlated with the sample (co)variances.
Nevertheless, if (19) is true, convergence in probability of the
cost function with the measured noise model (22) to the cost
function with the (co)variance model (12) with a convergence
rate of
can be shown
1/m
where V , W equals X and/or Y (proof see Appendix E).
The extra scaling by M is necessary to prevent that the cost
function blows up for increasing M . The Op(l/d%) term in
the cost function (25) shows up in the asymptotic ( F + CO)
properties of the estimators.
If (19) is not true, then the (co)variance model (22) is a
biased estimate of the (co)variance of the sample means (12).
This bias will introduce a bias in the GTLS, BTLS and ML
estimates of the model parameters P , which can be calculated
explicitly if the true model belongs to the model set H ( f , P )
[lo]: it is a function of the model, the selected frequency band
and the number of frequencies, and is proportional to a linear
combination of the following noise-to-signal ratio terms
.xf)
+ Pxx(f) .$(f> +
’
MlX(f)12
PYY(f) P ( f )
MlY(f)12
+ PYX(f)
’ MY(f)X*(f)
BOUNDS
ON THE
CORRELATION COEFFICIENT
d L,
0.5000
1
0.7071
2
0.8090
3
0.8660
4
0.9001
5
AS A
FUNCTION
OF K
Iv. STUDY OF THE CORRECTION TERMS(20)
Simulations with colored continuous-time noise indicate that
in most practical situations one correction term ( K = 1) in
each series (20) is sufficient, while only in some extreme cases
three terms ( K = 3) are required. In this section it will be
shown how to estimate the number K of relevant correction
terms in the series (20) for the case V = W .
Assuming that assumption (19) is true, the question arises
whether the expected value of the corrected variance (23)
is guaranteed to be positive. For M :sufficiently large (23)
(V = W ) can be written as
with @ ’ ( f )the correlation coefficient
@ ’ ( f )= real ( & ! ~ ) ) / c ~ v v ( f ) . (28)
Since 6$’ E [-1, 11, it is clear that (27) can become negative,
which is highly undesirable. What is the interpretation of this
phenomenon? Consider thereto the case K = 1. In Appendix
F it is shown for linearly filtered white noise that if assumption
(19) is true for K = 1, then IS$’I < 0.5, which guarantees the
non-negativeness of (27). Consequently, if IS$)I > 0.5 then,
the hypothesis (19) with K = 1 is false, (27) can be negative,
and K should be increased. Similar conclusions hold for the
case K > 1 (see Table I). As a result, the value of 6
;
) gives
a lower bound on K ; e.g., if I@’l E ((0.7071,0.80901, then
(27) should contain at least three correction terms ( K = 3).
Another important aspect in the estimation of the number
of relevant correction terms is the uncertainty of the estimated
variance of the sample mean (22) (V = W ) .For M sufficiently
large it can be written as
with d r ) (f) the sample correlation coefficient
*
(26)
From (26) it can be seen that the bias on the estimated
model parameters due to the bias on the noise model tends
to zero as 1/M. Since the noise on the sample mean tends to
zero as 1 / a ( l l ) , it can be concluded that the asymptotic
( M + CO) bias on the model parameters can be neglected.
d$)(f) = real (c$$(f))/cvv(f).
(30)
Exact calculation of the uncertainty bounds on the estimates
cvv and d$) is very difficult (the experiments are NOT
independent) and is out of the scope of this paper. One can
however get an idea of the uncertainty by looking to the results
Authorized licensed use limited to: Rik Pintelon. Downloaded on December 1, 2008 at 09:02 from IEEE Xplore. Restrictions apply.
16
IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT, VOL. 45, NO 1, FEBRUARY 1996
TABLE II
68% AND 95% CONFIDENCE BOUNoS ON THE SAMPLE VARIANCE CI/v
AND THE SAMPLE
CORRELATION
COEWICENT
d v ) E [-0.5. 0.31
FOR INDEPENDENT SAMPLES AS A FUNCTION
OF THE SAMPLE SIZE M
~
10
f03
k0.5
100
fO.l
f0.2
M.14
M.3
250
f0.05
33.1
33.09
33.17
1E3
f0.03
k0.05
fo.045
M.09
1E4
fO.O1
f0.02
fo.014
M.03
1E5
f0.003
k0.005
+_00.0044
H.009
i0.45
for INDEPENDENT experiments. While the uncertainty on the
sample variance cvv is proportional to C V V , it tums out that
the uncertainty on the sample correlation coefficient d:) is
almost independent of d:) in the range [-0.5, 0.51 [7]. This
means that a large number M of experiments are necessary
to obtain an acceptable uncertainty on d$), especially for
small values of the correlation coefficient (see Table 11). The
total uncertainty on the quantity between brackets in (29)
(1 + 2cf=’_,
d c ) ( f ) ) is bounded above by 2KAd$). It can
be used in combination with Table I1 to calculate an upper
bound on the number of relevant correction terms K by
imposing that 2 K A d t ’ << 1. For example, if the sample
size M equals 250, then no more than K = 5 correction
terms should be identified. The following correction strategy
is hence proposed. First calculate the correlation terms d$) ( f ) ,
and the lower (Klower)
and upper (Kupper)bounds on K (to
be deduced from Tables I and 11). Choose a value of K << M
in the interval [Klower,
Kupper]
such that
CVV(f)
20
(V = X , Y )
Fig. 2. Experimental setup.
systems. Since memory becomes cheaper the tendency is that
this number will still increase so that the correction discussed
in Section II-B will gain importance in the future.
The correlation between the M samples { X k )( f ), Y i ’ ( f );
‘I = 1, 2 , . . . , M } is only important IF the generator noise
N g ( f ) and/or process noise N , ( f ) sources (see Fig. 1) are
dominant over the usually white measurement noise sources
Mx(f) and M y ( f ) , and IF the transfer function H ( f ) of
the device under test exhibits large amplitude dynamics in
the frequency band of interest. This is typically the case for
vibrating mechanical structures.
The method described in Section 11-B assumes accurate synchronization between excitation and measurement. Otherwise,
due to a slight slip between the generation and acquisition
clock frequencies, the variance of the discrete Fourier spectra
will virtually be increased. This assumption is fulfilled for
all measurement devices where the generation and acquisition
clock frequencies are derived from the same reference clock,
like for example dynamic signal analyzers. Most network
analyzers and VXI-based measurement systems also allow
measurement according to that principle.
VI. EXPERIMENTAL
VERIFICATION
If such a value of K cannot be found, then no useful
correction can be made and M should be increased. Under normal circumstances the correlation between the samples { X k ) ( f ) Y
, i ) ( f ) :z = 1, 2 , . . . , M } is not that large
(lPwv(f)I << a w v ( f ) )so that (31) is practically always
fulfilled.
V. SOMEPRACTICALCONSIDERATIONS
Dozens of simulations with continuous-time noise indicate
that an autocorrelation length of the continuous-time noise of
1/5th and 4/5th of the signal period results in, respectively, a
10% and 30% correction (20) of the sample (co)variance (22).
Hence, several hundreds of signal periods (see Table 11) are
typically necessary to make sensible estimates of the correction
terms. Modern measurement devices already have the required
amount of memory to measure these large numbers of signal
periods; for example, 1E6 samples per channel in VXI-based
We have chosen to measure a lowly damped system and
to add generator noise to the excitation signal such that
var(Ng(f)) >> var ( M v ( f ) )(V = X , Y ) .The experimental
setup is shown in Fig. 2: the device under test is an aluminium
plate (52.5cm x 2.5 cm x 2 mm) which is excited in its center
by a mini shaker (B&K 4810). The excitation voltage v(t)
of the shaker is the sum of a multisine source (Wavetek 75)
and a random noise source (HP 3562A). The multisine has a
flat amplitude spectrum and consists of 58 components equally
distributed in the band [lo2 Hz, 267 Hz]. It is calculated using
the time-frequency-domain swapping algorithm [ 111, [ 121. The
applied force f ( t ) and the acceleration a ( t ) of the Al-plate
are measured at the excitation point by an impedance head
mounted on the shaker. Both signals are amplified and lowpass filtered (cutoff frequency of 300 Hz) before being fed to
the acquisition unit. M = 243 consecutive periods ( N = 256
samples per period, f s = 741.053 Hz) of the input and output
signals have been measured with a transient recorder (Nicolet
490).
Authorized licensed use limited to: Rik Pintelon. Downloaded on December 1, 2008 at 09:02 from IEEE Xplore. Restrictions apply.
17
PINTELON et al.: MEASUREMENT OF NOISE (CROSS-) POWER SPECTRA
50
3
=
n -10
5
c
50
-30
B -50
310 1 , , , ,
50
~~~~
100 150 200 250 300
-50 , , , ,
50 100 150 200 250 300
frequency (Hz)
frequency (Hz)
(a)
(b)
Fig. 3. Measured acceleration-to-force transfer function: (a) amplitude, and
(b) phase.
Fig. 5. Pole (x), zero (0)plots in the complex plane with the 68% confidence
ellipsoids (dotted lines): (a) estimates with uncorrected sample (co)variances
( K = O), and (b) estimates with corrected sample r(co)variances(22) ( K = 1).
TABLE I11
VALUE OF THE COST FUNCTION
WITH 95%
CONFIDENCE
BOUNDS
FOR TWOMODELCOMPLEX~IES
value cost function
model
-1
L
n = d = 2
n = d = 3
-
50
100
150 200 250
frequency (Hz)
(a)
300
1
50
L ' ' 1,
I
I
100 150 200 250
frequency (Hz)
"
" "
" "
uncorrected sample
corrected sample
theoretical value
cov. (22), K = 0
cov. ( 2 3 , K =: 1
(no model errors)
11.4 f 6.8
83.7 21
+
55
-
68.2 & 18
54
I
300
(b)
Fig. 4. Sample correlation coefficients (30) (solid lines) d!$)(f) and d$)!f)
of (a) the input errors, and the (b) output errors and the 95% uncertainty
bounds (dashed lines).
The mean values (8) and the sample (co)variances (22) are
estimated using the M = 243 measurements (samples) of the
input and output spectra. The measured frequency response
function m y ( f ) / m x ( f ) is shown in Fig. 2. Following the
strategy explained in Section IV it is found that only one
correction term ( K = 1) should be added in the calculation of
the sample (co)variances (22). Fig. 4 shows the corresponding
estimated sample correlation coefficients (30). Taking into
account the confidence bounds it follows that the correction
of the sample variance of the input and output errors is
significant respectively around 200 Hz and 150 Hz, which
corresponds exactly to the frequency bands of the device under
test with large amplitude dynamics. From Fig. 4 it can also
be seen that values of &'(f) and d $ ) ( f ) smaller than -0.5
are included within the 95% confidence bounds. Hence with
some probability the estimated sample variances (29) ( K = 1)
can become negative. This has been experimentally confirmed
by repeating the same experiment dozens of times. Adding
a second correction term does not help here: since for this
experiment Id@'(f)l << I d g ' ( f ) l (V = X , Y ) its only effect
is to double the uncertainty of the expression between brackets
in (29). For the experiments with negative sample variances
(29) ( K = 2 ) the only sound solution is to increase M to
1000 to more (see Table 11).
The parameters P of the rational transfer function model
H(s, P) = ~ ~ , , . i ~ s k / / C ~ = , , &
ares kidentified using a
maximum likelihood estimator [3], [6], [ 121. The identification
is done two times on the data set { m x ( f k ) ,m y ( f k ) :k =
1, 2, . . . , F } : once with the uncorrected sample (co)variances
{.xX'(f.k)/A4,.YY(f/c)/M, . Y x ( f k > / J M : k = 1, 2 , * * . , F } ,
and once with the corrected sample (c0)variances { C x x ( f k ) ,
C y y ( f k ) , Cyx(f/c): k = 1, 2 , . . . , F ; k = 1). The results
for the model ( n = d = 2 ) are shown in Fig. 5. It follows
that the estimates of the parameters (or poles and zeros) in
both cases are equal w&hin their uncertainty. The difference
between the corresponding uncertainties and the value of the
cost functions (see Table 111) is, however, significant. Using the
uncorrected sample (co)variances one would wrongly conclude
from Table 111 that the model order n = d = 2 is too large
(overmodeling' ), while the results with the corrected sample
(co)variances indicate that the model order n = d = 2 is
too low (undermodeling' ). No significant model errors can be
detected when the model complexity is increased to n = d = 3
(see Table 111).
VU. CONCLUSIONS
A time-efficient method to estimate (cross-) power spectra for frequency-domain identification purposes has been
presented. It consists of measuring M consecutive periods
of the input and output signals and requires a statistical
analysis of M DEPENDENT samples (experiments) of jointly
correlated complex random variables. For large sample values
(A4 >> 100) the classical formulae to calculate the sample
(co)variances must be corrected with terms which account
'Overmodeling means that the model not only fits the dynamics of the
system but also tries to follow the noise. A cost function value smaller than
its theoretical expected value indicates overmode ling.
*Undermodeling means that the model is unable to fit all the dynamics of
the system. A cost function value larger than its theoretical expected value is
a strong indication for undermodeling.
Authorized licensed use limited to: Rik Pintelon. Downloaded on December 1, 2008 at 09:02 from IEEE Xplore. Restrictions apply.
IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT, VOL. 45, NO. 1, FEBRUARY 1996
18
for the correlation between the experiments. These correction
terms are important if the device under test has large amplitude
dynamics and if the generator noise and/or the process noise
are dominant over the measurement noise. No sensible correction can be made for small sample values (A4< 100) unless
the interexperiment correlation is very high (absolute value
correlation coefficient close to 1). These conclusions can be
extended to any experiment where variance information about
the mean value is required and where the coloring of the noise
is important within the measured frequency band.
The influence of the noise model on the estimated model
parameters has been discussed for several estimators. Although
the correction of the sample (co)variances does not change
much the value of the estimated parameters, it may change
a lot the calculated uncertainty bounds, the value of the cost
function and the selected order qf the model.
where (*) denotes that the complex conjugate can be taken
or not at any place. Intuitively a mixing condition means
that the span of dependence is 'small'. Example: white
Gaussian noise passed through a stable linear time-invariant
filter satisfies (34) with Q = 00. For stationary noise
cum(x,(*I , (*)
, x k(*p1+ , ) is independent of U and (34)
reduces to
c o c o
co
k z = l ks=l
kp=l
P
n-m
+
Dejinition 1: z = x j y is a zero-mean complex random
variable if E { z } = 0, E { x 2 }= E{y2} and E{xy} = 0.
Property 1: Using definition 1 it can easily be verified
that the variance E{lx12} = 2E{z2} = 2E{y2} and that
E ( z 2 ) = 0.
Definition 2: V = [ ~ 1 2 2 is] ~a zero-mean complex noise
vector if z1 = x1
j y 1 and z2 = x2
jy2 are zeromean complex random variables and if E{xlxz} = E{yly2},
E{xlY2} = -E{y1z2}.
Property 2: Using Dejinition 2, it can easily be verified that
E { V V T } = 0 and that
+
1, a , . . . , & .
(35)
Definition 5: Convergence in the mean square sense ([ 131,
p. 56) is
1.i.m. x, = x
APPENDIXA
COMPLEXNOISE [61
=
U
lim E { ( x , - x)~}= 0.
n-00
(36)
APPENDIXB
EXPECTEDVALUESAMPLE(CO)VARIANCE
By definition of the mean value (33), and since the disturbances N $ ) ( f ) , N $ ) ( f ) have zero mean, it follows that
+
E { V V H }=
[% $1
+
with a? = E{/z1I2},ai = E{lz2I2}, p = E{zzzT} (complex covariance), superscript H the hermitian transposition
operator, and * the complex conjugate.
Definition 3 ([8],p. 19): The Qth-order cumulant
cum(Ic1, z 2 , . . . ,Z Q ) is the coefficient of j Q t l t z . . . t~
in the Taylor series expansion of Zn(E{exp(j E:='=z,k t k ) } )
about the origin tl = t 2 = . . . = tQ = 0 (another equivalent
definition can be found in [SI).
The cumulants up to order 3 are identical to the central
moments, for example,
cum (xk) = E { x k }
~ kz
):,
= cov (zk, xn).
(V,W = X , Y).
Since the noise samples N $ ) ( f ) , N $ ) ( f )
are jointly mixing of order 4, their joint second-order cumulants are absolutely summable (see Appendix A, Definition 4),
and hence the second term in (38) tends to zero as l/M for
M -+ Co.
APPENDIXC
THEOREMS
cum ( Z k , xi) = var (xk)
cum (
where V, W equal X and/or Y . Due to the multilinearity of
the cum(.) operator ([XI, p. 19) and using the relationship
V $ ) ( f )= V ( f ) N $ ) ( f ) ,(37) can be written as
(33)
Theorem 1: Take a complex random noise variable xk
which is mixing of order 2, and define a sequence of complex
numbers a k with ' d k E [l,001: l a k l < a < 00. Then the sum
This is no longer true for higher order (>3) cumulants [7].
Definition 4: A random noise sequence xk is mixing of
order Q if its cummulants up to order Q are absolutely
summable:
M
(39)
converges in the mean square sense to its expected value
Authorized licensed use limited to: Rik Pintelon. Downloaded on December 1, 2008 at 09:02 from IEEE Xplore. Restrictions apply.
19
PINTELON et al.: MEASUREMENT OF NOISE (CROSS-) POWER SPECTRA
with a convergence rate of 1/@
YM
(41)
= E { y M ) -k
where O m s ( l / a ) denotes a random variable that tends to
for M 4 CO.
zero as
Proof of Theorem 1: It is sufficient to show that the variance of YM tends to zero as 1/M for M ---f 00 (see Appendix
A, Definition 5). By definition of the variance (33), and due
to the multilinearity of the cum(.) operator ([8], p. 19), it
follows that
Each cumulant appearing in (44) 3s absolutely summable
since z k and z k are jointly mixing of order 4. Hence, each
term in (44) is absolutely summable since it consists of the
multiplication of absolutely summable cumulants3. It can be
concluded that
l/m
var ( Y M ) = cum ( Y M ,
=
YR)
--y2
u k u i cum
M2
l
M
M
(xk,
zt).
(42)
k=l n=l
The expression for the variance (42) can be bounded above by
U2
5 -C
M
(43)
with C a positive real number independent of M . The last
inequality in (43) is true since the second-order cumulants of
z k are absolutely summable (see Appendix A, Definition 4).
Theorem 2: If two complex random noise variables x k and
z k are jointly mixing of order 4, then their product yk = x k z k
is mixing of order 2.
Proof of Theorem 2: According to [15] (or 181, theorem
2.3.2, p. 21) the second-order cumulant of yk can be written
as a sum of products of cumulants of z k and xk
l
lim M
M+oo
M
F;Mlcum(yk, Y,)I
< CO.
(45)
k = l n=l
Corollary to Theorem 2: If Xk is mixing of order 4, then
= x i and yk = 1x1~1~
are mixing of order 2 (proof replace
Zk in theorem 2 by, respectively, xk and xi).
yk
APPENDIXD
(CO)VARIANCE
SAMPLEMEANS(9)
Using the definition of the covariance (33) and the multilinearity property of the cum (.) operator ([8], p. 19) it follows
that
where V , W equal X and/or Y . Due to the stationarity of the
noise the joint cumulant in (46) depends only on the difference
(2 - n) P I
Making the change of variables k = i - n and r = n, (46)
can be rewritten as
PROBABILITY
APPENDIXE
LIMITOF SOME COST FUNCTIONS
Formula (25) will be proven for the maximum likelihood
cost function. The proof for the generalized total least squares
(GTLS), bootstrapped total least squares (BTLS) and iterative
maximum likelihood (IQML) is analogous. Assuming that (19)
is true, the maximum likelihood cost function based on the
The series formed by the term-by-term multiplication of 2 absolutely
convergent series is absolutely convergent [ 141.
Authorized licensed use limited to: Rik Pintelon. Downloaded on December 1, 2008 at 09:02 from IEEE Xplore. Restrictions apply.
20
IEEE TRANSACTIONS ON INSTRUMENTATION AND MEASUREMENT, VOL. 45, NO. 1, FEBRUARY 1996
sample mean (8) of the input and output spectra is given by [6]
=-E
1
with n k a zero mean white noise sequence with variance
The correlation coefficient
of the process ~k is
02.
F
I m x ( f k ) N ( f k , PI-- m Y ( f k ) N f k l
F
k=l
P)I2 (49)
4 f k )
with N ( f , P ) and D ( f , P ) , respectively, the numerator and
denominator of the transfer function model H ( f , P ) , and
a
+ P X X , K ( f ) ) l N ( f , P)I2
+ ( O Y Y ( f )+ P Y Y , K ( f ) . ) l D ( f , m2
- area1 ( ( . Y X ( f ) + P Y X , K ( f ) ) D ( f , P ) N * ( f ,PI).
Maximizing S(l) with respect to the coefficients a1
gives the results shown in Table I.
a2,
. . . aK
m =(CXX(f)
(50)
Subtracting and adding the cost function (49) with the exact
noise model from the cost function with the measured noise
model (22) gives
where
Since the sample (co)variances ( M C X X( f ) M C y y ( f ) ,
M C y x ( f ) ) converge in the mean square sense to their
expected value with a Convergence rate of 1 / a this is
also true for s & ( f )
REFERENCES
R. Pintelon, P. Guillaume, Y. Rolain, J. Schoukens, and H. Van hamme,
“Parametric identificationof transfer functions in the frequency-domain,
a survey,” IEEE Trans. Automat. Contr., vol. 39, no. 11, pp. 2245-2260,
1994.
R. Pintelon and J. Schonkens, “Robust identification of transfer functions
in the s- and z-domains,” IEEE Trans. Instrum. Meas., ,vol. 39, no. 4,
pp. 565-573, 1990.
J. Schoukens and R. Pintelon, Identijkation of Linear Systems: A
Practical Guideline To Accurate Modeling. Oxford: Pergamon Press,
1991.
J. Swevers, B. De Moor, and H. Van Brussel, “Stepped sine system
identification errors-in-variables and the quotient singular value decomposition,” Mechanical Systems and Signal Processing, vol. 6, pp.
121-134, 1992.
H. Van hamme and R. Pintelon, “Application of the bootstrapped total
least squares @TLS) estimator in linear system identification,” Signal
Processing VI: Theories and Applications, J. Vandewalle et al., Ed.
Amsterdam: Elsevier, 1992, pp. 73 1-734.
R. Pintelon, P. Guillaume, Y. Rolain, and F. Verbeyst, “Identification
of linear systems captured in a feedback loop,” IEEE Trans. Instrum.
Meas., vol. 41, no. 6, pp. 747-754, 1992.
A. Stuart and J. K. Ord, Kendall’s Advanced Theory of Statistics, vol.
1. London: Charles Griffin, 1987.
D. R. Brillinger, Time Series: Data Analysis and Theory. New York:
McGraw-Hill, 1981.
E. Lukacs, Stochastic Convergence. New York Academic, 1975.
P. Guillaume, R. Pintelon, and J. Schoukens, “Robust parametric transfer function estimation using complex logarithmic frequency response
data,” IEEE Trans.Automat. Contr., vol. 40, no. 7, pp. 1180-1 190, 1995.
E. Van der Ouderaa, J. Schoukens, and J. Renneboog, “Peak factor minimization using a time-frequency-domain swapping algorithm,” ZEEE
Trans. Instricm. Meas., vol. 37, no. 2, pp. 207-212, 1988.
I. KollAr, Frequency-Domain System Ident@cation Toolbox for Use with
Matlab. Natick, M A The Mathworks, 1994.
A. H. Jazwinski, Stochastic Processes and Filtering Theory. London:
Academic, 1970.
T. J. I’a Bromwich, Introduction to the Theoly of InJnite Series. Loudon: MacMdlan & Co. Ltd., 1965.
V. P. Leonov and A. N. Shiryaev, “On a Method of Calculation of
Semi-Invariants,” Theory of Probability and its Applications, vol. IV,
no. 3, pp. 319-329, 1959.
(proof: apply the linearity property of the 1.i.m. operator [13]).
It follows that the second term in the right-hand side of (51)
is O p ( l / f i ) in probability (proof: convergence in the mean
square sense implies convergence in probability [9], p. 33,
theorem 2.2.2; and the probability limit and a continuous
function may be interchanged [9], p. 42, Theorem 2.3.3).
Rik Pintelon, (M’90) for a photograph and biography, see this issue, p.
11.
APPENDIXF
EXTREMEVALUES
OF THE CORRELATION COEFF’ICENT
Linearly filtered white noise satisfying (19) must be of the
form
Ek
=nk
+
K
amnkpm
m=l
(54)
Patrick Guillaume, (S’86-M’87) for a photograph and biography, see this
issue, p. 11.
Authorized licensed use limited to: Rik Pintelon. Downloaded on December 1, 2008 at 09:02 from IEEE Xplore. Restrictions apply.
PINTELON et al.: MEASUREMENT OF NOISE (CROSS-) POWER SPECTRA
Johan Schoukens(M’9CSM’92) was born in Belgium in 1957. He received the degree of engineer
in 1980, the degree of doctor in applied sciences in
1985 from the Vrije Universiteit Bmssel (VUB) and
Brussels, Belgium.
He is presently a Research Director of the National Fund for Scientific Research (NFWO) Brussels, Belgium, and part-time Lecturer at the VUB.
The prime factors of his research are in the field
of system identification for linear and non-linear
systeins.
Authorized licensed use limited to: Rik Pintelon. Downloaded on December 1, 2008 at 09:02 from IEEE Xplore. Restrictions apply.
21