On the Relationship between Minimum Norm and Linear
Prediction for Spatial Spectrum Estimation
Sverre Holm
Department of Informatics, University of Oslo
22 April 1998
Abstract
array. This gives a special form which consists of
finding the roots of a polynomial; the root-minimum
norm method. The method was generalized to an arbitrary array geometry in [3]. In its general form the
minimum norm method consists of finding the steering vector that lies in the noise subspace, where the
first element is unity, and where the length of the vector is minimized. It is an alternative to the MUSIC
method [4] in that this method finds the steering vector that is closest to the noise subspace.
The minimum norm method for spatial spectrum estimation is derived from the linear prediction method
in exactly the same way as the MUSIC method is
derived from the minimum variance method. The
derivation consists of replacing the correlation with
its noise subspace component and setting all noise
eigenvalues to unity. This makes it simpler to understand the methods and their properties. This relationship also brings out the meaning of setting the
first element to unity in the minimum norm method
— it corresponds to the predicted element in linear
prediction. There is also a parallel between properties: e.g. just as linear prediction has a lower detection threshold than minimum variance so does minimum norm compared to MUSIC. Thus the properties
of the subspace methods seem to be ’inherited’ from
the original non-subspace methods.
In this correspondence we will exploit the relationship between the minimum norm method and the
linear prediction method in order to give a derivation
of the method that fits into the framework of the spatial spectrum estimation presentation in sections 7.27.3 of [5]. We will also show how this explains some
of the properties of the method.
1 Introduction
2
The minimum norm method for spatial spectrum estimation is a subspace-based method that was first
presented completely by Kumaresan and Tufts [1].
Some of the ideas had already been given by Reddi
[2]. These first references both use a uniform linear
The derivation is based on [5] and uses the same terminology. Consider an array of M sensors at arbitrary locations with a spatial correlation matrix R,
~
and a steering vector from direction ζ~ which is e(ζ).
1
Theory
2.1 Minimum Variance and MUSIC
Another interesting link between the minimum
variance and MUSIC methods is found by considThe minimum variance method is derived from the
ering estimators that are nonlinear functions of the
consideration that the output energy of the beamcovariance matrix [6]:
former should be minimized while the response to
the actual direction given by ζ~ is unity. This gives
~ = e0 (ζ)R
~ −k e(ζ).
~
fk (ζ)
(5)
the following ouput power:
The minimum variance estimator is found by letting
~ = [e0 (ζ)R
~ −1 e(ζ)]
~ −1
P M V (ζ)
(1) k = 1:
~ = f1 (ζ)
~ −1 .
P M V (ζ)
(6)
where e0 means the conjugate transpose of e. In all
the subsequent subspace methods a model consisting When k grows the larger eigenvalues are more and
of Ns incoherent plane waves in additive white noise more neglected and in the limit the MUSIC estimate
is assumed. The spatial correlation matrix can then is obtained:
be divided into a rank Ns signal plus noise subspace
~ = lim [σ 2 k · fk (ζ)]
~ −1 .
and a rank M − Ns noise subspace:
P M U SIC (ζ)
(7)
k→∞
R=
Ns
X
i=1
λi vi vi0 +
M
X
λi vi vi0 .
(2) 2.2
Linear Prediction and Minimum Norm
i=Ns +1
The linear prediction (LP) method is a signal modeling approach where the output of sensor m0 is assumed to be a weighted linear combination of the
outputs of the other sensors, thus a model order of
M is assumed. The method is derived from the consideration that the output energy of the beamformer
should be minimized while the weight vector for el0
0
R = Vs Λs Vs + Vn Λn Vn
(3) ement m0 should be unity. This gives the following
ouput power:
where the subscripts stand for signal-plus-noise sub0 R−1 δ
δm
m0
0
space and noise subspace respectively.
~ =
P LP (ζ)
. (8)
0
−1
0
~
~
[e(ζ) R δm0 δm0 R−1 e(ζ)]
The MUSIC method prescribes that the spatial
spectrum estimate is found as the steering vector
which is closest to the noise subspace. Thus the norm Here δm0 is an M-element column vector with all
~ n V 0 e(ζ)
~ should be minimized. This can also zeros except for a 1 in position m0 . In order to sime0 (ζ)V
n
be interpreted as replacing the correlation matrix in plify the comparison, we will neglect the numerator,
(1) with the noise subspace component from (3), and because it is invariant with direction. For a linear
at the same time performing a whitening of the spa- array, the predicted element m0 should be selected
tial noise spectrum, i.e. setting all noise eigenvalues at one of the ends. Positions in the center of the
array usually give reduced resolution [7]. A causal
to unity. Thus the MUSIC estimate is:
model, as used in time series analysis, would preM U SIC ~
0 ~
0 ~ −1
P
(ζ) = [e (ζ)Vn Vn e(ζ)] .
(4) scribe m0 = M . For arbitrary geometries, ends can
It is assumed that the eigenvalues have been sorted,
i.e. that λ1 is the largest and λM is the smallest
eigenvalue. For a white background noise σ 2 = λi
for i = Ns+1 , . . . , M , where σ 2 is the noise power.
The spectral decomposition can be written in a more
compact form as:
2
Yu [8] develop a total least squares solution to the
linear prediction problem which is in fact the minimum norm solution [9]. However, the close parallel
between minimum variance and MUSIC on the one
hand, and linear prediction and minimum norm on
the other hand, has not been fully exploited before.
usually not be defined, so an element near the phase
center should be chosen.
Now we apply the same subspace idea as in the
MUSIC method, i.e. replace the correlation matrix in
the numerator of (8) with the noise subspace component from (3), and perform a whitening of the spatial
noise spectrum. This gives:
~ = [e(ζ)
~ 0 Vn V 0 δm δ0 Vn V 0 e(ζ)]
~ −1 . (9) 2.3
P M N (ζ)
0 m0
n
n
This equation gives a spatial spectrum estimate
which consists of finding the steering vector that lies
in the noise subspace, where the element m0 is unity
and the length of the vector is required to be minimum. Thus it is a generalization of the minimum
norm algorithm. The relationship with the linear prediction method also gives a basis for the selection of
the seemingly arbitrary value of m0 = 1 in the minimum norm method.
The relationship between linear prediction and
minimum norm can also be found by considering
linear prediction type estimators that are nonlinear
functions of the covariance matrix:
~ = e(ζ)
~ 0 R−k δm δ0 R−k e(ζ).
~
gk (ζ)
0 m0
A natural extension to the methods presented here is
to skip the prewhitening, i.e. let the noise eigenvalues remain in the expressions. This results in a variation of the MUSIC method called the eigenvector
(EV) method [7]:
~ = [e0 (ζ)V
~ n Λ−1 V 0 e(ζ)]
~ −1 .
P EV (ζ)
n
n
(10)
In the limit as k grows, the minimum norm estimate
is obtained:
~ = lim [σ
P M N (ζ)
k→∞
~ −1 .
· gk (ζ)]
(13)
This method has an advantage in scenarios with nonwhite background noise. It is also more robust to
incorrect estimation of the number of signals, Ns ,
than the MUSIC method. This can e.g. be seen from
the fact that the method degenerates to the minimum
variance method when Ns = 0, while the MUSIC
method will give a constant, independent of direction.
Based on the same principle one can develop a
method from the ideas of linear prediction and minimum norm which we call the LP-EV method or the
weighted minimum norm method.
The linear prediction estimator is found by letting
k = 1:
~ = g1 (ζ)
~ −1 .
P LP (ζ)
(11)
2k
Generalization to Eigenvector and LPEV methods
~ = [e(ζ)
~ 0 Vn Λ−1 V 0 δm δ0 Vn Λ−1 V 0 e(ζ)]
~ −1 .
P LP −EV (ζ)
n
n
0 m0
n
n
(12)
This method consists of finding the minimum
norm steering vector in the weighted noise subspace
when element m0 is constrained to be unity. The
weighting consists of normalizing each noise eigenvector according to its contribution, i.e. by the inverse of the eigenvalue. This method shares the same
properties as the eigenvector method with respect to
non-white backgrounds, it has reduced sensitivity to
The relationship given here between linear prediction and minimum norm is not a totally new one. In
[1], Kumaresan and Tufts use the similarity with the
autocorrelation method of linear prediction in their
derivation. In [3] the relationship with linear prediction is used to find a solution called the minimum
norm linear prediction vector. Further Rahman and
3
incorrect estimation of the number of signals, and
degenerates to the linear prediction method when
Ns = 0. This method was actually first developed
by Johnson [10], but its relationship to the minimum
norm method has not been noted before.
the linear prediction and minimum variance methods. As k goes to infinity, the minimum norm and
MUSIC methods are obtained. This parallel development has not been exploited before. We believe it
will benefit the presentation of these methods. It also
gives a rationale for the seemingly arbitrary choice of
forcing the first element to be unity in the minimum
norm method; this element is now seen to be equivalent to the predicted element in the linear prediction
method.
We have also observed that properties such as resolution, detection threshold, and variance seem to be
’inherited’ from the non-subspace methods, e.g. just
as linear prediction has a lower detection threshold
than the minimum variance method so does minimum norm have a lower threshold than MUSIC.
3 Comparison of Properties
The linear prediction method has improved resolution compared to minimum variance. It also has a
lower detection threshold. Its disadvantage is a larger
background fluctuation in non-signal regions [5].
Similarly, the minimum norm algorithm has a
smaller bias and a lower detection threshold than the
MUSIC method [11]. According to [12], these two
properties follow each other. In [13] it is shown that
for small samples, minimum norm has larger variance than MUSIC. It has also been shown that of all
the MUSIC-like algorithms, the original MUSIC algorithm has the smallest large-sample variance [14]
compared to e.g. minimum-norm.
Thus it is seen that to some extent comparison between LP and min-norm, and minimum variance and
MUSIC bring out almost parallel properties: Less
bias and lower detection threshold for linear prediction and minimum norm, and smaller variance for
minimum variance and MUSIC.
References
[1] R. Kumaresan and D. W. Tufts, “Estimating the
angle of arrival of multiple plane waves,” IEEE
Trans. Aeorospace and Electr. Syst, vol. AES19, pp. 134–139, Jan. 1983.
[2] S. S. Reddi, “Multiple source location - a digital approach,” IEEE Trans. Aeorospace and
Electr. Syst, vol. AES-15, pp. 95–105, Jan.
1979.
4 Conclusion
[3] F. Li, R. J. Vaccaro, and D. W. Tufts, “Minnorm linear prediction for arbitrary sensor arrays,” in IEEE Int. Conf. Acoust., Speech, Sign.
Proc., pp. 2613–2616, 1989.
The minimum norm method has been derived from
the linear prediction method for spatial spectrum estimation. The derivation is exactly parallel to the
derivation of MUSIC from the minimum variance
method. The derivation consists of replacing the
correlation with its noise subspace component and
setting all noise eigenvalues to unity. The parallel
derivations can also be seen by considering the methods as functions depending on R−k ; k = 1 gives
[4] R. Schmidt, “Multiple emitter location and
signal parameter estimation,” in Proc. of
the RADC Spectrum Estimation Workshop,
pp. 243–258, 1979.
4
MUSIC, MIN-NORM, and FINE,” IEEE
Trans. Signal Processing, vol. 42, pp. 1812–
1816, July 1994.
[5] D. H. Johnson and D. E. Dudgeon, Array Signal Processing. Englewood Cliffs, NJ: Prentice
Hall, 1993.
[6] C. Vaidyanathan and K. M. Buckley, “Perfor- [14] P. Stoica and A. Nehorai, “MUSIC, maximum
mance analysis of DOA estimation based on
likelihood, and Cramer-Rao bound: Further
nonlinear functions of covariance matrix,” Sigresults and comparisons,” IEEE Trans. Signal
nal Processing, vol. 50, pp. 5–16, 1996.
Processing, vol. 38, pp. 2140–2150, Dec. 1990.
[7] D. H. Johnson and S. DeGraaf, “Improving the
resolution of bearing in passive sonar arrays
by eigenvalue analysis,” IEEE Trans. Signal
Processing, vol. ASSP-30, pp. 638–647, Aug.
1982.
[8] M. A. Rahman and K.-B. Yu, “Total leastsquares approach for frequency estimation using linear prediction,” IEEE Trans. Signal Processing, vol. ASSP-35, pp. 1440–1454, Oct.
1987.
[9] E. M. Dowling and R. D. DeGroat, “The equivalence of the total least squares and minimum
norm methods,” IEEE Trans. Signal Processing, vol. 39, pp. 1891–1892, Aug. 1991.
[10] D. H. Johnson, “The application of spectral estimation methods to bearing estimation problems,” Proc. IEEE, vol. 70, pp. 1018–1028,
Sept. 1982.
[11] M. Kaveh and A. J. Barabell, “The statistical performance of the MUSIC and the
minimum-norm algorithms in resolving plane
waves in noise,” IEEE Trans. Signal Processing, vol. ASSP-34, pp. 331–341, Apr. 1986.
[12] W. Xu and M. Kaveh, “Comparative study of
the biases of MUSIC-like estimators,” Signal
Processing, vol. 50, pp. 39–55, 1996.
[13] X.-L. Xu and K. M. Buckley, “Bias and
variance of direction-of-arrival estimates from
5