Hindawi Publishing Corporation
Journal of Applied Mathematics
Volume 2013, Article ID 318786, 8 pages
http://dx.doi.org/10.1155/2013/318786
Research Article
Recursive Identification for Dynamic Linear Systems from
Noisy Input-Output Measurements
Dan Fan and Kueiming Lo
School of Software, Tsinghua University, Tsinghua National Laboratory for Information Science and Technology,
Beijing 100084, China
Correspondence should be addressed to Kueiming Lo; gluo@tsinghua.edu.cn
Received 7 March 2013; Accepted 13 April 2013
Academic Editor: Xiaojing Yang
Copyright © 2013 D. Fan and K. Lo. This is an open access article distributed under the Creative Commons Attribution License,
which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
Errors-in-variables (EIV) model is a kind of model with not only noisy output but also noisy input measurements, which can be
used for system modeling in many engineering applications. However, the identification for EIV model is much complicated due to
the input noises. This paper focuses on the adaptive identification problem of real-time EIV models. Some derivation errors in an
accuracy research of the popular Frisch scheme used for EIV identification have been pointed out in a recent study. To solve the same
modeling problem, a new algorithm is proposed in this paper. A Moving Average (MA) process is used as a substitute for the joint
impact of the mutually independent input and output noises, and then system parameters and the noise properties are estimated
in the view of the time domain and frequency domain separately. A recursive form of the first step calculation is constructed to
improve the calculation efficiency and online computation ability. Another advantage of the proposed algorithm is its applicableness
to different input processes situations. Numerical simulations are given to demonstrate the efficiency and robustness of the new
algorithm.
1. Introduction
In the field of engineering, modeling is an essential issue. In
most cases, the systems are modeled by stochastic models in
which the input signals are assumed to be measured exactly
and all the disturbed noises are added to the output signals;
that is, only the output measurements are noisy. These models
are called “errors-in-equation models.” However, there are
always signals beyond our control that also affect the input
of the systems; some of them cannot be included in the
output noises. Therefore, it is also necessary to consider
the modeling problem for those systems with noisy inputoutput measurements, especially when we concern the actual
physical laws of the process rather than the prediction of
the future behaviour [1]. This kind of model whose input
and output measurements are both containing noise is called
“errors-in-variables (EIV) model [2].”
The identification of EIV models has received a lot of
attention during the past decades. By far, EIV models have
been used in numerous applications, such as the modeling
problems in econometrics, computer vision, biomedicine,
chemical and image reconstruction, spectrum estimation,
speech analysis, noise cancelation, and digital communications [3–8].
In EIV models, the noise in input measurements cannot
be equivalent to the output error, which makes the identification of EIV models much more difficult. The identifiability
of EIV dynamic models was analyzed in [9, 10]. It is pointed
out that EIV dynamic models cannot be uniquely identified
from the second-order properties [9]. Thus specific prior
knowledge is needed to achieve the identifiability. Once the
identifiability is established, estimation algorithms can be
developed [10]. Owing to the noisy input measurements
in the EIV models, the standard least squares method for
errors-in-equation models cannot yield consistent estimates
anymore. To overcome this problem, a bias-compensated
least squares (BCLSs) principle was proposed in [4]. On
the basis of BCLS principle, various algorithms have been
2
developed, such as the Frisch scheme-based algorithms [7],
the KL algorithm [8], ECLS [9], BELS [10], and others in [11–
15].
Although there are such a number of approaches for
identifying different EIV models, the convergence of the
algorithms has always been a difficulty. Only a few literatures
have tried to solve this problem [12, 15, 16]. In [16], the accuracy of Frisch scheme for EIV identification was analyzed,
in which the estimates of the system parameters as well as
the noise variance were both proved asymptotically Gaussian
distributed by linearizing three primary equations in this
scheme. This conclusion can be perceived as the theoretical
support of the Frisch scheme-based algorithms. Based on
this work, continued extensions and real applications have
sprung up recently [17–20], which reaffirms the value of this
particular analysis result. However, the analysis in [16] needs
a condition that the estimates of the parameters are close to
their true values, which is not clear how to be guaranteed.
A counterexample that could not converge was present in
[21], and in addition, some derivation errors of [16] were
found and discussed at the same time. Furthermore, another
method was provided to identify the EIV model in [21].
But comparing to the model concerned in [16], due to the
difficulty of the identification problem, the one considered in
[21] was a simpler one with a stronger condition that the input
and output noise processes had the same variance, which has
been hampering its application in some degree.
The purpose of this paper is to consider how to avoid the
restrict condition in [21]; in other words, we are trying to
solve the same modeling problem as in [16], that is, to propose
an identification algorithm for the modeling of dynamic EIV
systems with independent input and output noises to estimate
both the unknown system parameters and the noise signals.
In order to achieve this purpose, we used a two-step method:
in Step 1, the original model is rewritten into another form
to get the system parameters in the time domain; in Step 2,
the noise variances are calculated in the frequency domain.
Moreover, the recursive form of the proposed method will be
presented to improve its operational efficiency and enhance
its online applicability.
The structure of the paper is as follows. In Section 2, the
concerned model is described in detail. The new identification algorithm is presented in Section 3. Some simulations
are given in Section 4 to illustrate the identification accuracy,
the convergence rate, and the antinoise performance. Finally,
conclusion remarks are given in Section 5.
Journal of Applied Mathematics
𝑢0 (𝑡)
Σ
𝑦0 (𝑡)
Dynamic
system
Σ
𝑢(𝑡)
̃ (𝑡)
𝑢
𝑦(𝑡)
̃ (𝑡)
𝑦
Figure 1: A basic errors-in-variables system.
research in this paper. The ARX(𝑛𝑎 , 𝑛𝑏 ) model is considered
here as
𝐴 (𝑧) 𝑦0 (𝑡) = 𝐵 (𝑧) 𝑢0 (𝑡) ,
(1)
where
𝐴 (𝑧) = 1 + 𝑎1 𝑧 + ⋅ ⋅ ⋅ + 𝑎𝑛𝑎 𝑧𝑛𝑎 ,
𝐵 (𝑧) = 𝑏1 𝑧 + ⋅ ⋅ ⋅ + 𝑏𝑛𝑏 𝑧𝑛𝑏 ,
(2)
are the polynomials in the backward shift operator 𝑧. The
{𝑎1 , 𝑎2 , . . . , 𝑎𝑛𝑎 , 𝑏1 , 𝑏2 , . . . , 𝑏𝑛𝑏 }are the unknown system parameters to be identified, while the measured variables 𝑢(𝑡) and
̃ Thus
𝑦(𝑡) are disturbed by the unknown noises 𝑢̃(𝑡) and 𝑦(𝑡).
the input and output measurements are
𝑢 (𝑡) = 𝑢0 (𝑡) + 𝑢̃ (𝑡) ,
𝑦 (𝑡) = 𝑦0 (𝑡) + 𝑦̃ (𝑡) .
(3)
After introducing the notations
𝑇
𝜃 = (𝑎1 , 𝑎2 , . . . , 𝑎𝑛𝑎 , 𝑏1 , 𝑏2 , . . . , 𝑏𝑛𝑏 ) ,
𝜑 (𝑡)
𝑇
= (−𝑦 (𝑡 − 1) , . . . , −𝑦 (𝑡 − 𝑛𝑎 ) , 𝑢 (𝑡 − 1) , . . . , 𝑢 (𝑡 − 𝑛𝑏 )) ,
𝜑̃ (𝑡)
𝑇
= (−𝑦̃ (𝑡 − 1) , . . . , −𝑦̃ (𝑡 − 𝑛𝑎 ) , 𝑢̃ (𝑡 − 1) , . . . , 𝑢̃ (𝑡 − 𝑛𝑏 )) ,
(4)
the EIV system can be described as the following model:
𝑦 (𝑡) = 𝜑 (𝑡)𝑇 𝜃 + 𝐴 (𝑧) 𝑦̃ (𝑡) − 𝐵 (𝑧) 𝑢̃ (𝑡) .
(5)
To ensure the identifiability, we list some assumptions first.
(A1) The EIV system is asymptotically stable, which means
that there is no zero of 𝐴(𝑧) inside the unit circle.
2. Problem Formulation
A basic dynamic EIV system is shown in Figure 1.
Unlike the normal errors-in-equation model, as mentioned before, the EIV model has noise in both input
measurements and output measurements. The immeasurable
true input and output processes 𝑢0 (𝑡) and 𝑦0 (𝑡) are linked by a
dynamic system, which can be a linear or a nonlinear system
in different applications. So far, most of the related studies
are focused on the linear systems, which is also the focus of
̃ are mutually independent
(A2) The noises 𝑢̃(𝑡) and 𝑦(𝑡)
and also independent of the true input and output
signals 𝑢0 (𝑡) and 𝑦0 (𝑡).
̃ are white noises with zero mean and
(A3) 𝑢̃(𝑡) and 𝑦(𝑡)
independent variances 𝜆 𝑢 and 𝜆 𝑦 .
The problem we need to solve is to estimate the system
parameter vector 𝜃 with the help of the measured regressor
vector 𝜑(𝑡). Furthermore, considering that a noise process
can be described by the mean and variance, to identify the
Journal of Applied Mathematics
3
zero-mean input and output noises is simplified to identify
the variances. Therefore, except for the system parameters, we
also want to estimate the output and input noise variances 𝜆 𝑦
and 𝜆 𝑢 . In the following section, we will give an algorithm
with two independent steps to fulfill the two aspects of the
estimate requirements.
For the new model (10), denote a new parameter vector 𝜃
and a new regressor vector 𝜑(𝑡) by
3. Identification Algorithms
and then the EIV system (5) can be rewritten as
As mentioned before, the identification for EIV system is
much more difficult because the input and output noises
are unknown. For the EIV system described in Section 2,
to overcome the influence of the input noise, we will use
another MA process {𝑤(𝑡)} as a substitute for the joint impact
of the mutually independent input and output noises, as
two mutually independent sequences of independent random
variables can be represented as an MA process which has
the same spectra with the two jointly sequences [22]. Then
the system can be modified as an ARMAX model, and what
need to do is changed to estimate the system parameters
of the new model and to determine the variances of the
input/output noises in terms of {𝑤(𝑡)}. Thus a two-step
recursive estimation algorithm can be constructed to identify
the system parameters 𝜃 and the noise variances 𝜆 𝑦 and 𝜆 𝑢 ,
respectively.
Step 1. for the time 𝑡, we used the obtained estimation of 𝑤(𝑡−
1) to estimate the parameters and get the current estimates
𝜃(𝑡) and 𝑤(𝑡).
Step 2. These results are utilized to calculate the estimates of
the noise variance 𝜆 𝑦 (𝑡) and 𝜆 𝑢 (𝑡).
In the following, we will give the algorithm followed by
proof.
Step 1 (estimation for the unkown system parameter 𝜃). For
convenience, denote the last two terms of (5) by V(𝑡), that is,
V (𝑡) = 𝐴 (𝑧) 𝑦̃ (𝑡) − 𝐵 (𝑧) 𝑢̃ (𝑡) ,
(6)
𝐸𝑦̃2 (𝑡) = 𝜆 𝑦 ,
𝐸𝑦̃2 (𝑡) = 𝜆 𝑢 .
(7)
(8)
where {𝑒(𝑡)} is white noise with
𝐸𝑒 (𝑡) = 0,
𝑛𝑐 = max {𝑛𝑎 , 𝑛𝑏 } .
(9)
It can be shown that we can find a pair of {𝑐𝑖 , 0 ≤ 𝑖 ≤ 𝑛𝑐 }
and 𝜆 𝑒 such that {𝑤(𝑡)} and {V(𝑡)} have the same spectra [22],
which means that {V(𝑡)} can be represented by {𝑤(𝑡)} in (8) as
𝑦 (𝑡) = 𝜑 (𝑡)𝑇 𝜃 + 𝑤 (𝑡) .
The {𝑐𝑖 , 0 ≤ 𝑖 ≤ 𝑛𝑐 } and 𝜆 𝑒 are intermediate variables.
𝑦 (𝑡) = 𝜑 (𝑡)𝑇 𝜃 + 𝑒 (𝑡) .
(12)
(13)
In this step, we will give a recursive algorithm to identify (13).
The covariance matrix of the regressor vector 𝜑(𝑡) and
output variables 𝑦(𝑡) is denoted by
𝑅𝜑 = 𝐸𝜑 (𝑖) 𝜑 (𝑖)𝑇 ,
(14)
𝑟𝜑𝑦 = 𝐸𝜑 (𝑖) 𝑦 (𝑖) .
For convenience, introduce
𝑡
̂ 𝜑 (𝑡) = ∑𝜑 (𝑖) 𝜑 (𝑖)𝑇 ,
𝑅
(15)
𝑖=1
𝑡
𝑟̂𝜑𝑦 (𝑡) = ∑𝜑 (𝑖) 𝑦 (𝑖) .
(16)
𝑖=1
Assume that the input {𝑢(𝑡)} is a stationary process; in the
̂ 𝜑 (𝑡)/𝑡 and 𝑟̂𝜑𝑦 (𝑡)/𝑡
calculation, we can use the algebra means 𝑅
instead of the mathematical expectations 𝑅𝜑 and 𝑟𝜑𝑦 in (14),
as by ergodicity, we have
̂ 𝜑 (𝑡)
𝑅
𝑡
𝑡→∞
󳨀󳨀󳨀󳨀→ 𝑅𝜑 ,
𝑟̂𝜑𝑦 (𝑡)
𝑡
𝑡→∞
󳨀󳨀󳨀󳨀→ 𝑟𝜑𝑦 .
(17)
Lemma 1 (Matrix Inversion Formula [23]). For the matrices
𝐴 ∈ 𝑅𝑛×𝑛 , 𝐶 ∈ 𝑅𝑛×1 , and 𝐷 ∈ 𝑅𝑛×1 , the inverse matrix of
𝐵 = 𝐴 + 𝐶𝐷𝑇 is
−1
= 𝐴−1 − 𝑎−1 𝐴−1 𝐶𝐷𝑇 𝐴−1 ,
(18)
Theorem 2. For system (13), under the assumptions (A1)–
(A3), the parameter vector 𝜃 can be estimated recursively as
follows with a large 𝑃(0) and arbitrary 𝜃(𝑡):
̂ (𝑡)
𝜃 (𝑡) = 𝜃 (𝑡 − 1) + 𝑎−1 (𝑡) 𝑃 (𝑡 − 1) 𝜑
̂ (𝑡)𝑇 𝜃 (𝑡 − 1)] ,
× [𝑦 (𝑡) −𝜑
2
𝐸𝑒 (𝑡) = 𝜆 𝑒 ,
𝑇
where 𝑎 = 1 + 𝐷𝑇 𝐴−1 𝐶.
Introduce an MA(𝑛𝑐 ) process
𝑤 (𝑡) = 𝑒 (𝑡) + 𝑐1 𝑒 (𝑡 − 1) + ⋅ ⋅ ⋅ + 𝑐𝑛𝑐 𝑒 (𝑡 − 𝑛𝑐 ) ,
(11)
𝜑 (𝑡) = (𝜑 (𝑡)𝑇 , 𝑒 (𝑡 − 1) , . . . , 𝑒 (𝑡 − 𝑛𝑐 )) ,
(𝐴 + 𝐶𝐷𝑇 )
̃ are mutually independent with
where 𝑢̃(𝑡) and 𝑦(𝑡)
𝐸𝑦̃ (𝑡) = 𝐸̃
𝑢 (𝑡) = 0,
𝑇
𝜃 = (𝜃𝑇 , 𝑐1 , 𝑐2 , . . . , 𝑐𝑛𝑐 ) ,
̂ (𝑡)𝑇 𝑃 (𝑡 − 1) ,
̂ (𝑡) 𝜑
𝑃 (𝑡) = 𝑃 (𝑡 − 1) − 𝑎−1 (𝑡) 𝑃 (𝑡 − 1) 𝜑
𝑇
̂ (𝑡) 𝜃 (𝑡) ,
𝜀 (𝑡) = 𝑦 (𝑡) − 𝜑
̂ (𝑡) = (𝜑 (𝑡)𝑇 , 𝜀 (𝑡 − 1) , . . . , 𝜀 (𝑡 − 𝑛 ))𝑇 ,
𝜑
𝑐
(19)
(10)
̂ 𝑇 𝑃(𝑡 − 1)𝜑(𝑡).
̂
where 𝑎(𝑡) = 1 + 𝜑(𝑡)
4
Journal of Applied Mathematics
Proof. Like the least squares methods, we use the covariance
matrix for help. By multiplying with 𝜑(𝑡) to the system model
(13), we have
similar way as for RPLR (recursive pseudolinear regression)
algorithm [22], that is, to form a substitute for 𝜑(𝑡) as
𝐸𝜑 (𝑡) 𝑦 (𝑡) − 𝐸𝜑 (𝑡) 𝜑 (𝑡)𝑇 𝜃 = 𝐸𝜑 (𝑡) 𝑒 (𝑡) ,
̂ (𝑡) = (𝜑 (𝑡)𝑇 , 𝜀 (𝑡 − 1) , . . . , 𝜀 (𝑡 − 𝑛 ))𝑇 ,
𝜑
𝑐
(20)
with assumptions (A2) and (A3) which can be rewritten as
𝑅𝜑 𝜃 = 𝑟𝜑𝑦 .
(21)
̂ 𝜑 (𝑡)/𝑡 and 𝑟̂𝜑𝑦 (𝑡)/𝑡 in (15) and
Replacing 𝑅𝜑 and 𝑟𝜑𝑦 with 𝑅
(16), respectively, as mentioned before, one has
̂ 𝜑 (𝑡) 𝜃 = 𝑟̂𝜑𝑦 (𝑡) .
𝑅
(22)
We note that (15) and (16) imply
̂ 𝜑 (𝑡) = 𝑅
̂ 𝜑 (𝑡 − 1) + 𝜑 (𝑡) 𝜑 (𝑡)𝑇 ,
𝑅
(23)
𝑟̂𝜑𝑦 (𝑡) = 𝑟̂𝜑𝑦 (𝑡 − 1) + 𝜑 (𝑡) 𝑦 (𝑡) .
̂ 𝜑 (𝑡) is reversible, 𝜃 is estimated as
Then on condition that 𝑅
̂ −1 (𝑡) 𝑟̂𝜑𝑦 (𝑡)
𝜃 (𝑡) = 𝑅
𝜑
̂ −1 (𝑡) (̂𝑟𝜑𝑦 (𝑡 − 1) + 𝜑 (𝑡) 𝑦 (𝑡))
=𝑅
𝜑
̂ −1 (𝑡) [(𝑅
̂ 𝜑 (𝑡) − 𝜑 (𝑡) 𝜑 (𝑡)𝑇 ) 𝜃 (𝑡 − 1) + 𝜑 (𝑡) 𝑦 (𝑡)]
= 𝑅
𝜑
̂ −1 (𝑡) 𝜑 (𝑡) (𝑦 (𝑡) − 𝜑 (𝑡)𝑇 𝜃 (𝑡 − 1)) .
= 𝜃 (𝑡 − 1) + 𝑅
𝜑
̂ 𝑇 𝜃(𝑖), 𝑖 ≥ 1. The proof of the
where 𝜀(𝑡 − 𝑖) = 𝑦(𝑖) − 𝜑(𝑖)
substitution’s correctness is omitted (see details in [22]).
̂ defined by (29) to replace the 𝜑(𝑡) in (26)
Then using 𝜑(𝑡)
and (28), we get Theorem 2 easily.
Theorem 2 gives a recursive algorithm to get the estimate
of 𝜃: At time 𝑡 − 1, we store only the finite-dimensional
̂
information {𝜃(𝑡−1), 𝑃(𝑡−1), 𝜑(𝑡−1)}.
At time 𝑡, it is updated
using (23), (27), and (29), which is done with a given fixed
amount of operations, making it a high operational efficiency
and suitable for online applications. Since 𝜃(𝑡) is obtained,
obviously 𝜃(𝑡) can be easily got by (11).
Next we go to estimate the noise properties, which will
be helpful in real applications such as the cascade system
modeling.
Step 2 (estimation for the noise variances 𝜆 𝑦 (𝑡) and 𝜆 𝑢 (𝑡)).
To estimate the noise variances 𝜆 𝑦 , 𝜆 𝑢 , we need to find the
relationship between {V(𝑡)} in (6) and {𝑤(𝑡)} in (8).
We know that the spectrum Φ𝑠 (𝜔) of a signal {𝑠(𝑡)} is the
Fourier transform of its covariance function 𝑅𝑠 (𝜏) as
∞
(24)
Using (23), we can calculate the vector 𝜃(𝑡). But we note
̂ 𝜑 (𝑡) at each recursive
that there is an inverse operation of 𝑅
step, which is a very time-consuming process. To avoid the
inversing, introduce
̂ −1 (𝑡) ,
𝑃 (𝑡) = 𝑅
𝜑
(25)
and apply the Matrix Inversion Formula in Lemma 1 to (24);
̂ 𝜑 (𝑡), 𝐶 = 𝐷𝑇 = 𝜑(𝑡), we have
taking 𝐴 = 𝑅
𝑃 (𝑡) = 𝑃 (𝑡 − 1) −
𝑃 (𝑡 − 1) 𝜑 (𝑡) 𝜑 (𝑡)𝑇 𝑃 (𝑡 − 1)
𝑇
1 + 𝜑 (𝑡) 𝑃 (𝑡 − 1) 𝜑 (𝑡)
.
(26)
𝑃 (𝑡 − 1) 𝜑 (𝑡)
1 + 𝜑 (𝑡)𝑇 𝑃 (𝑡 − 1) 𝜑 (𝑡)
.
(30)
𝜏=−∞
where
1 𝑁
∑𝑠 (𝑡) 𝑠 (𝑡 − 𝜏) .
𝑁→∞𝑁
𝑡=1
𝑅𝑠 (𝜏) = 𝐸𝑠 (𝑡) 𝑠 (𝑡 − 𝜏) = lim
(31)
Since {𝑤(𝑡)} and {V(𝑡)} have the same spectra, that is,
Φ𝑤 (𝜔) ≡ ΦV (𝜔) ,
(32)
𝑅V (𝜏) = 𝑅𝑤 (𝜏) .
Thus we can find the relationship between 𝜆 𝑦 , 𝜆 𝑢 , and 𝜆 𝑒 by
using the covariance functions 𝑅V (𝜏) and 𝑅𝑤 (𝜏).
At step 𝑡, an estimate of 𝑅𝑠 (𝜏) can be used as
𝑃 (𝑡 − 1) 𝜑 (𝑡)
1 + 𝜑 (𝑡)𝑇 𝑃 (𝑡 − 1) 𝜑 (𝑡)
(33)
(27)
Taking (27) into (23), we have
𝜃 (𝑡) = 𝜃 (𝑡 − 1) +
Φ𝑠 (𝜔) = ∑ 𝑅𝑠 (𝜏) 𝑒−𝑖𝜏𝜔 ,
this means that for all 𝜏 = 0, 1, 2, . . . , 𝑛𝑐 ,
Moreover, by (26) it is clear that
̂ −1 (𝑡) 𝜑 (𝑡) =
𝑅
𝜑
(29)
(28)
⋅ (𝑦 (𝑡) − 𝜑 (𝑡)𝑇 𝜃 (𝑡 − 1)) .
Noting that {𝑒(𝑡 − 1)} in (12) is unknown, 𝜑(𝑡) cannot
be constructed directly. This problem can be solved in the
𝑅𝑠𝜏 (𝑡) =
1 𝑡
∑ 𝑠 (𝑘) 𝑠 (𝑘 − 𝜏) .
𝑡 𝑘=1
(34)
𝜏
We switched to the notation 𝑅V𝜏 , 𝑅𝑤
rather than 𝑅V (𝜏), 𝑅𝑤 (𝜏)
to account for certain differences due to recursive step 𝑡.
Journal of Applied Mathematics
5
Introduce
Table 1: Estimation results of System 1.
𝑇
𝜃𝑎𝜏 (𝑡) = (𝑎𝜏 (𝑡) , 𝑎𝜏+1 (𝑡) , . . . , 𝑎𝜏+𝑛𝑎 (𝑡)) ,
𝑇
𝜃𝑏𝜏 (𝑡) = (𝑏𝜏+1 (𝑡) , 𝑏𝜏+2 (𝑡) , . . . , 𝑏𝜏+𝑛𝑏 (𝑡)) ,
𝑇
𝜃𝑐𝜏 (𝑡) = (𝑐𝜏 (𝑡) , 𝑐𝜏+1 (𝑡) , . . . , 𝑐𝜏+𝑛𝑐 (𝑡)) ,
(35)
𝑇
𝜑𝑦̃ (𝑡) = (−𝑦̃ (𝑡) , −𝑦̃ (𝑡 − 1) , . . . , −𝑦̃ (𝑡 − 𝑛𝑎 )) ,
Parameter
𝑎1
𝑎2
𝑏1
𝑏2
𝜆𝑦
𝜆𝑢
True value
−0.20
−0.15
0.30
−0.27
0.20
0.50
Estimation
−0.1975 ± 0.0025
−0.1483 ± 6.9910𝑒−4
0.2842 ± 2.3776𝑒−4
−0.2574 ± 7.5975𝑒−4
0.1922 ± 4.5330𝑒−4
0.5125 ± 0.0024
𝑇
𝑢 (𝑡 − 1) , . . . , 𝑢̃ (𝑡 − 𝑛𝑏 )) ,
𝜑𝑢̃ (𝑡) = (̃
𝑇
𝜑𝑒 (𝑡) = (𝑒 (𝑡) , 𝑒 (𝑡 − 1) , . . . , 𝑒 (𝑡 − 𝑛𝑐 )) ,
4. Simulation Examples
where
1,
𝑘 = 0,
{
{
𝑎𝑘 (𝑡) = {𝑎𝑘 (𝑡) , 0 < 𝑘 ≤ 𝑛𝑎 ,
{
𝑘 > 𝑛𝑎 or 𝑘 < 0,
{0,
𝑏𝑘 (𝑡) = {
𝑏𝑘 (𝑡) , 0 < 𝑘 ≤ 𝑛𝑏 ,
0,
𝑘 > 𝑛𝑏 or 𝑘 ≤ 0,
(36)
1,
𝑘 = 0,
{
{
𝑐𝑘 (𝑡) = {𝑐𝑘 (𝑡) , 0 < 𝑘 ≤ 𝑛𝑐 ,
{
𝑘 > 𝑛𝑐 or 𝑘 < 0,
{0,
are the transformation of the parameters in 𝜃(𝑡).
Then according to (A2), (A3), (6), (8), and (34), we have
𝜏
𝑅𝑤
(𝑡) =
=
𝜃 = (𝑎1 , 𝑎2 , 𝑏1 , 𝑏2 ) = (−0.2, −0.15, 0.3, −0.27)𝑇 .
𝑇
1
∑ 𝜃𝑐0 (𝑡) 𝜑𝑒 (𝑘) 𝜃0𝑐 (𝑡)𝑇 𝜑𝑒 (𝑘 − 𝜏)
𝑡 𝑘=1
(37)
𝜏
𝑇
1 𝑡
= ∑ 𝜃𝑐0 (𝑡) 𝜑𝑒 (𝑘) 𝜑𝑒 (𝑘)𝑇 𝜃𝑐 (𝑡)
𝑡 𝑘=1
=
Similarly we have
𝑇
𝑇
(38)
Then it is clear that for 𝜏 = 0, 1, 2, . . . , 𝑛𝑐 ,
𝜏
𝑅V𝜏 (𝑡) = 𝑅𝑤
(𝑡)
(39)
is a set of linear equations about the unknown 𝜆 𝑦 and 𝜆 𝑢 , in
which {𝜃𝜏𝑎 (𝑡), 𝜃𝑏𝜏 (𝑡), 𝜃𝑐𝜏 (𝑡)} is known at time 𝑡, and 𝜆 𝑒 can be
estimated by:
𝜆 𝑒 (𝑡) = 𝑅𝜀0 (𝑡) =
1 𝑡 2
∑ 𝜀 (𝑘)
𝑡 𝑘=1
System 1: 𝑦0 (𝑡) − 0.2𝑦0 (𝑡 − 1) − 0.15𝑦0 (𝑡 − 2)
= 0.3𝑢0 (𝑡 − 1) − 0.27𝑢0 (𝑡 − 2) .
(𝑡) 𝜆 𝑒 .
𝑅V𝜏 (𝑡) = 𝜃𝑎0 (𝑡) 𝜃𝑎𝜏 (𝑡) 𝜆 𝑦 + 𝜃𝑏0 (𝑡) 𝜃𝑏𝜏 (𝑡) 𝜆 𝑢 .
(41)
It is easy to get the system as follows, which is denoted by
System 1:
𝑡
𝑇
(𝑡) 𝜃𝑐𝜏
Case A. First we examine how well the algorithm works
for systems with Gaussian input. Consider an EIV dynamic
system with 𝑛𝑎 = 𝑛𝑏 = 2 and
𝑇
1 𝑡
∑ 𝑤 (𝑘) 𝑤 (𝑘 − 𝜏)
𝑡 𝑘=1
𝜃𝑐0
This section addresses some numerical evaluation of the
identification algorithm presented in this paper. Matlab 7.7
is used to do the simulations. To demonstrate its validness
to various EIV systems, we have chosen different signal
processes as the true input variables {𝑢0 (𝑡)} in each case:
in Case A, a zero-mean Gaussian process is used; in Case
B, a sawtooth signal is applied; in Case C, it is an ARMA
̃
process. The noise processes {̃
𝑢(𝑡)} and {𝑦(𝑡)}
in these cases
are mutually uncorrelated white noise signals with zeromean. The robustness of the algorithm is also tested, which
is shown in Case C.
(40)
by taking 𝜀(𝑡) in Theorem 2 as the estimate of 𝑒(𝑡).
Then the estimates 𝜆 𝑦 (𝑡) and 𝜆 𝑢 (𝑡) for the output/input
noise variances 𝜆 𝑦 and 𝜆 𝑢 can be calculated by (37)–(40).
(42)
Let the input signal {𝑢0 (𝑡)} be a zero-mean Gaussian
process whose variance equals 1. Let the noise signals {̃
𝑢(𝑡)}
̃
and {𝑦(𝑡)}
be mutually uncorrelated white noise signals with
𝜆 𝑦 = 0.2, 𝜆 𝑢 = 0.5, which means a strong noise environment
for the system.
The system is simulated for 𝑁 = 8000 steps. Calculation
results are listed in Table 1, where the calculation error is
defined by the standard deviation. Figures 2 and 3 show
the system parameter and the noise variances estimates
separately. Solid lines indicate the true values and dashed
lines denote the corresponding estimates. Noting that the
vertical coordinate scopes are very small in both figures, it
can be seen easily that the estimates are converging fast to the
true parameters.
Case B. Consider another system, System 2, with sawtooth
input:
System 2: 𝑦0 (𝑡) =
𝑧 − 2𝑧2
𝑢 (𝑡) ,
(1 − 0.9𝑧) (1 − 0.8𝑧) 0
(43)
6
Journal of Applied Mathematics
0.8
2
0.6
1.5
Parameter value
Parameter value
0.2
𝑎1 = −0.2
0
𝑎2 = −0.15
−0.2
𝑏2 = −0.27
−0.4
0
1000
2000
3000 4000 5000
Simulation step
6000
7000
8000
𝜆 𝑦 = 𝜆 𝑢 = 0.5
𝑎2 = 0.72
0
−0.5
−1
−1.5
𝑎1 = −1.7
𝑏2 = −2
−2.5
Figure 2: Estimation result of 𝜃 in System 1.
−3
0
500 1000 1500 2000 2500 3000 3500 4000 4500 5000
Simulation step
Figure 4: Estimates of System 2 by the new recursive method.
1
Case C. The proposed algorithm in this paper is valid not
only for EIV models with iid random input process but also
for those whose input signals are more general such as the
ARMA process. In order to examine it, we have considered
the following system:
𝜆 𝑢 = 0.5
Parameter value
0.5
−2
−0.6
−0.8
𝑏1 = 1
1
𝑏1 = 0.3
0.4
0.5
𝜆 𝑦 = 0.2
2 (1 − 0.5𝑧) 𝑧
𝑢 (𝑡) ,
1 + 0.2𝑧 − 0.48𝑧2 0
System 3: 𝑦0 (𝑡) =
where
0
𝑇
𝜃 = (𝑎1 , 𝑎2 , 𝑏1 , 𝑏2 ) = (0.2, −0.48, 2, −1)𝑇 .
−0.5
(46)
(47)
In this system, the input process {𝑢0 (𝑡)} is an ARMA
process:
0
1000
2000
3000 4000 5000
Simulation step
6000
7000
8000
𝜆 𝜉 = 1.
in which 𝑛𝑎 = 𝑛𝑏 = 𝑛𝑐 = 2, and
𝑇
(44)
(45)
The simulation results of the system parameters and the
noise variances are all displayed in Figure 4. The true values
and estimates are also denoted by solid lines, and dotted lines
respectively. We can see that the algorithm has an even better
performance for this kind of EIV system. All the estimates
converge to their corresponding true values consummately.
(49)
̃
The noise signals {̃
𝑢(𝑡)} and {𝑦(𝑡)}
are mutually uncorrelated
white noise signals with
𝜆 𝑦 = 0.2,
This is the counterexample which was presented in [21]
to show the unconvergency of the Frisch-based method
analyzed in [16]. We use our proposed algorithm to identify
this system under the same conditions; that is, the input
sawtooth signal’s amplitude equals 1 and its frequency is
10 Hz; the noises’ variances are
𝜆 𝑦 = 𝜆 𝑢 = 0.5.
(48)
with 𝜉(𝑡) being a zero-mean iid Gaussian process whose
variance is
Figure 3: Estimation result of 𝜆 𝑦 and 𝜆 𝑢 in System 1.
𝜃 = (𝑎1 , 𝑎2 , 𝑏1 , 𝑏2 ) = (−1.7, 0.72, 1, −2)𝑇 .
(1 − 5𝑧) 𝑢0 (𝑡) = (1 − 0.3𝑧) 𝜉 (𝑡)
𝜆 𝑢 = 0.5.
(50)
The estimation results are shown in Figures 5 and 6.
Similarly, the true values are marked by solid lines and the
estimates of parameters are marked by dotted lines. We can
see that for ARMA input process, the estimation can still
converge to the true parameters quickly.
In order to verify the robustness against noise, we further did several experiments with different signal-to-noise
ratios (SNRs) in System 3. The noise processes used in the
experiments are shown by the first three columns in Table 2.
Column 1 is the signal variables used to generate the input
signals; columns 2 and 3 are the variables of input noise and
output noise.
The average estimation error of System 3 (including the
system parameter estimation and the noises estimation) is
Journal of Applied Mathematics
7
3
Table 2: Comparison of the estimations for different SNRs.
Parameter value
2
𝑏1 = 2
1
𝑎1 = 0.2
0
𝑎2 = −0.48
−1
𝑏2 = −1
𝜆𝜉
1
1
1
1
1
1
1
𝜆𝑢
0.05
0.2
0.2
0.5
0.5
1
1
𝜆𝑦
0.05
0.2
0.5
0.2
0.5
1
2
Average error
0.0025
0.0039
0.0040
0.0049
0.0052
0.0086
0.0064
−2
−3
0
𝜆 𝑢 = 0.5
mutually independent input and output noises, this twostep algorithm can not only estimate the system parameter
vector as well as the noise variances with greater accuracy but
also reduce the computational complexity significantly due to
its recursive form. It has been shown by several simulation
results that the presented algorithm demonstrates, as shown
in the numerical simulations, a great accuracy, fast convergence speed, and good antinoise performance. Theoretical
analysis of the proposed algorithm and the identification of
some more complicated model such as EIV nonlinear models
will be considered in future work.
𝜆 𝑦 = 0.2
Acknowledgments
500 1000 1500 2000 2500 3000 3500 4000 4500 5000
Simulation step
Figure 5: Estimation result of 𝜃 in System 3 with ARMA input
process.
1
0.8
0.6
Parameter value
0.4
0.2
0
This work is supported by the funds NSFC61171121 and
NSFC60973049, the Science Foundation of Chinese Ministry
of Education, and China Mobile 2012.
−0.2
−0.4
−0.6
References
−0.8
−1
0
500 1000 1500 2000 2500 3000 3500 4000 4500 5000
Simulation step
Figure 6: Estimation of 𝜆 𝑦 and 𝜆 𝑢 in System 3 with ARMA input
process.
listed in column 4. It is clear that the performance of the
proposed algorithm is keeping good when the noises increase
(even when the noises are equal or larger than the input
signals).
5. Conclusions
This paper discussed the identification problem of dynamic
errors-in-variables (EIV) systems. EIV model is very useful
and has a wide engineering application range such as the
modeling of cascade system, and camera calibration. In
comparison with the usual errors-in-equation models, EIV
model has a more troublesome noise problem with the
input measurements being disturbed. Since several severe
errors in the previous analysis of the attractive Frisch scheme
identification approaches have been presented in the recent
studies, we developed an adaptive algorithm to solve the
same modeling problem. For the dynamic EIV model with
[1] T. Söderström, “Errors-in-variables methods in system identification,” Automatica, vol. 43, no. 6, pp. 939–958, 2007.
[2] R. J. Adcock, “Note on the method of least squares,” Analyst, vol.
4, pp. 183–184, 1877.
[3] L. Ng and V. Solo, “Errors-in-variables modeling in optical flow
estimation,” IEEE Transactions on Image Processing, vol. 10, no.
10, pp. 1528–1540, 2001.
[4] W. X. Zheng and C. B. Feng, “Unbiased parameter estimation
of linear systems in the presence of input and output noise,”
International Journal of Adaptive Control and Signal Processing,
vol. 3, no. 3, pp. 231–251, 1989.
[5] R. Diversi, “Bias-eliminating least-squares identification of
errors-in-variables models with mutually correlated noises,”
International Journal of Adaptive Control and Signal Processing,
2012.
[6] R. Diversi, R. Guidorzi, and U. Soverini, “Identification of
ARMAX models with noisy input and output,” World Congress,
vol. 18, no. 1, pp. 13121–13126, 2011.
[7] S. Beghelli, R. P. Guidorzi, and U. Soverini, “The Frisch scheme
in dynamic system identification,” Automatica, vol. 26, no. 1, pp.
171–176, 1990.
[8] K. V. Fernando and H. Nicholson, “Identification of linear
systems with input and output noise: the Koopmans-Levin
method,” IEE Proceedings D, vol. 132, no. 1, pp. 30–36, 1985.
[9] J. C. Agüero and G. C. Goodwin, “Identifiability of errors in
variables dynamic systems,” Automatica, vol. 44, no. 2, pp. 371–
382, 2008.
8
[10] W. X. Zheng, “A bias correction method for identification of
linear dynamic errors-in-variables models,” IEEE Transactions
on Automatic Control, vol. 47, no. 7, pp. 1142–1147, 2002.
[11] B. D. O. Anderson and M. Deistler, “Identifiability in dynamic
errors-in-variables models,” Journal of Time Series Analysis, vol.
5, no. 1, pp. 1–13, 1984.
[12] H.-F. Chen and J.-M. Yang, “Strongly consistent coefficient
estimate for errors-in-variables models,” Automatica, vol. 41, no.
6, pp. 1025–1033, 2005.
[13] M. Ekman, “Identification of linear systems with errors in variables using separable nonlinear least-squares,” in Proceedings of
the 16th Triennial World Congress of International Federation
of Automatic Control (IFAC ’05), pp. 815–820, Prague, Czech
Republic, July 2005.
[14] K. Mahata, “An improved bias-compensation approach for
errors-in-variables model identification,” Automatica, vol. 43,
no. 8, pp. 1339–1354, 2007.
[15] H. J. Palanthandalam-Madapusi, T. H. van Pelt, and D. S. Bernstein, “Parameter consistency and quadratically constrained
errors-in-variables least-squares identification,” International
Journal of Control, vol. 83, no. 4, pp. 862–877, 2010.
[16] T. Söderström, “Accuracy analysis of the Frisch scheme for
identifying errors-in-variables systems,” IEEE Transactions on
Automatic Control, vol. 52, no. 6, pp. 985–997, 2007.
[17] R. Diversi and R. Guidorzi, “A covariance-matching criterion in
the frisch scheme identification of MIMO EIV models,” System
Identification, vol. 16, no. 1, pp. 1647–1652, 2012.
[18] T. S. Söderström, “System identification for the errors-invariables problem,” Transactions of the Institute of Measurement
and Control, vol. 34, no. 7, pp. 780–792, 2012.
[19] T. Söderström, “Extending the Frisch scheme for errors-invariables identification to correlated output noise,” International
Journal of Adaptive Control and Signal Processing, vol. 22, no. 1,
pp. 55–73, 2008.
[20] M. Hong and T. Söderström, “Relations between biaseliminating least squares, the Frisch scheme and extended
compensated least squares methods for identifying errorsin-variables systems,” Automatica, vol. 45, no. 1, pp. 277–282,
2009.
[21] D. Fan and G. Luo, “Frisch Scheme identification for Errors-inVariables systems,” in Proceedings of the 9th IEEE International
Conference on Cognitive Informatics (ICCI ’10), pp. 794–799,
Beijing, China, July 2010.
[22] L. Ljung, System Identification-Theory for the User, Prentice Hall,
2nd edition, 1999.
[23] G. W. Stewart, Introduction to Matrix Computations, Academic
Press, New York, NY, USA, 1973.
Journal of Applied Mathematics
Advances in
Operations Research
Hindawi Publishing Corporation
http://www.hindawi.com
Volume 2014
Advances in
Decision Sciences
Hindawi Publishing Corporation
http://www.hindawi.com
Volume 2014
Mathematical Problems
in Engineering
Hindawi Publishing Corporation
http://www.hindawi.com
Volume 2014
Journal of
Algebra
Hindawi Publishing Corporation
http://www.hindawi.com
Probability and Statistics
Volume 2014
The Scientific
World Journal
Hindawi Publishing Corporation
http://www.hindawi.com
Hindawi Publishing Corporation
http://www.hindawi.com
Volume 2014
International Journal of
Differential Equations
Hindawi Publishing Corporation
http://www.hindawi.com
Volume 2014
Volume 2014
Submit your manuscripts at
http://www.hindawi.com
International Journal of
Advances in
Combinatorics
Hindawi Publishing Corporation
http://www.hindawi.com
Mathematical Physics
Hindawi Publishing Corporation
http://www.hindawi.com
Volume 2014
Journal of
Complex Analysis
Hindawi Publishing Corporation
http://www.hindawi.com
Volume 2014
International
Journal of
Mathematics and
Mathematical
Sciences
Journal of
Hindawi Publishing Corporation
http://www.hindawi.com
Stochastic Analysis
Abstract and
Applied Analysis
Hindawi Publishing Corporation
http://www.hindawi.com
Hindawi Publishing Corporation
http://www.hindawi.com
International Journal of
Mathematics
Volume 2014
Volume 2014
Discrete Dynamics in
Nature and Society
Volume 2014
Volume 2014
Journal of
Journal of
Discrete Mathematics
Journal of
Volume 2014
Hindawi Publishing Corporation
http://www.hindawi.com
Applied Mathematics
Journal of
Function Spaces
Hindawi Publishing Corporation
http://www.hindawi.com
Volume 2014
Hindawi Publishing Corporation
http://www.hindawi.com
Volume 2014
Hindawi Publishing Corporation
http://www.hindawi.com
Volume 2014
Optimization
Hindawi Publishing Corporation
http://www.hindawi.com
Volume 2014
Hindawi Publishing Corporation
http://www.hindawi.com
Volume 2014