Deterministic signals
Power spectral density -definitions
Power spectral density -properties
SGN 21006 Advanced Signal Processing:
Lecture 9 Spectrum estimation
Ioan Tabus
Department of Signal Processing
Tampere University of Technology
Finland
1 / 14
Deterministic signals
Power spectral density -definitions
Power spectral density -properties
Overview
I
I
I
Energy spectral density - for finite energy deterministic signals
Power spectral density - definitions for random signal
Power spectral density - properties
The slides follow Chapters 1 and 2 from P. Stoica, R. Moses, ”Spectral analysis of signals ”, available on-line at
http://user.it.uu.se/~ps/SAS-new.pdf
2 / 14
Deterministic signals
Power spectral density -definitions
Power spectral density -properties
Power spectral estimation
I
Goal: Given a finite number of values y1 , y2 , . . . , yN of a stationary signal, estimate the power over
narrow frequency bands.
I
Applications:
I the power over frequency represents a signature of the signal, by which it can be identified,
classified, compared.
I music decomposition over the semitones of the scale for automatic music transcription
I speech decomposition over bark or mel scales, for speech recognition
I monitoring the functioning of industrial machines or building and bridge structure
I characterization of physiological waves (EEG,EKG)
I estimate the direction of targets in underwater surveillance
I
First we introduce the definitions of spectral density for deterministic signals, then for random signals, and
then we state some properties of the spectral density.
I
The non-parametric and parametric estimation methods are discussed in next lectures.
3 / 14
Deterministic signals
Power spectral density -definitions
Power spectral density -properties
Deterministic signals
I
Given a discrete-time signal {yt }∞
t=−∞ having finite energy
∞
X
2
yt < ∞
t=−∞
then the Discrete-time Fourier transform (DTFT) can be defined as
Y (ω)
∞
X
=
y (t)e
−jωt
t=−∞
I
Here Y (ω) has a complex value and is the transform coefficient at frequency ω. The absolute value |Y (ω)|
is the amplitude and arg(Y (ω)) is the phase corresponding to transform coefficient Y (ω) at frequency ω.
The Parseval inequality holds for the DTFT:
∞
X
t=−∞
I
2
yt =
1
2π
Z π
2
|Y (ω)| dω
−π
The lefthand side (l.h.s) is the total energy of the signal, the r.h.s is the integral of the squared amplitude
over frequency. Hence the term |Y (ω)|2 has the significance of a density of energy over the frequency axis.
Energy spectral density definition
S(ω)
=
2
|Y (ω)|
and
identity the distribution of energy sums up to the total energy of the signal
P∞from Parseval
2
1 Rπ
t=−∞ yt = 2π −π S(ω)dω.
4 / 14
Deterministic signals
Power spectral density -definitions
Power spectral density -properties
Deterministic signals
I
Property: Define for all k the ”autocorrelation” of the deterministic signal (note the missing division by the
length of the signal, which is ∞)
∞
X
ρ(k) =
yt yt−k = ρ(−k)
t=−∞
Then
S(ω)
=
∞
X
ρ(k)e
−jωk
k=−∞
I
Proof:
S(ω)
=
∗
2
|Y (ω)| = Y (ω)Y (ω) =
∞
X
y (t)e
−jωt
∞
X
∞
X
y (t)y (τ )e
jω(τ −t)
(k=t−τ )
=
t=−∞ τ =−∞
=
∞
X
k=−∞
I
e
−jωk
y (τ )e
+jωτ
τ −∞
t=−∞
=
∞
X
∞
X
∞
X
y (t)y (t − k)e
−jωk
t=−∞ k=−∞
∞
X
t=−∞
y (t)y (t − k) =
∞
X
ρ(k)e
−jωk
k=−∞
The two equivalent definitions of S(ω), the first as S(ω) = |Y (ω)|2 and second
P∞
−jωk
S(ω) =
will be used for defining power spectral density for random signals.
k=−∞ ρ(k)e
5 / 14
Deterministic signals
Power spectral density -definitions
Power spectral density -properties
Power spectral density for random signals
I
Motivating the definition for the case of random signals:
I For different realizations of a random signal (e.g. generated by the same ARMA model using
different realizations of the noise) we would like to have similar evaluations of the distribution of
the power over frequencies. We need a deterministic function to characterize the content of power
over different frequencies.
I For a signal that is formed of a sum of k sinusoids (having frequencies ω1 , . . . , ωk , phases
φ1 , . . . , φk , and amplitudes α1 , . . . , αk ) plus white noise, the autocorrelation function is
obtained as the sum of the sinusoids having the same frequencies ω1 , . . . , ωk and amplitudes
α21 , . . . , α2k (but phases information φ1 , . . . , φk is lost). (See the lecture about spectrum
estimation for line models)
I Hence one can recover the powers of the signal over various frequencies from its autocorrelation
function. Taking the Fourier transform of the autocorrelation function, F [r (·)](ω), we obtain the
coefficients of the sinusoidal components over various frequencies.
6 / 14
Deterministic signals
Power spectral density -definitions
Power spectral density -properties
Power spectral density: Definition 1
I
We are given a discrete-time signal {yt }∞
t=−∞ which is a sequence of random variables with zero mean.
I
First power spectral density definition:
P(ω)
∞
X
=
r (k)e
−jωk
k=−∞
where r (k) is the auto covariance function r (k) = E [yt yt+k ].
I
The inverse Fourier transform will recover the correlation from the spectrum:
r (k)
I
=
1
2π
Z π
P(ω)e
jωk
dω
−π
The autocorrelation at lag 0 is
r (0)
=
2
E [yt ] =
1
2π
Z π
P(ω)dω
−π
This relation reinforces the power density interpretation: autocorrelation at lag 0 is the power of the signal,
which is equal to the integral of the power density over all frequencies.
I
One can interpret P(ω)dω as the infinitesimal power in the band (ω − dω
, ω + dω
)
2
2
7 / 14
Deterministic signals
Power spectral density -definitions
Power spectral density -properties
Power spectral density: Definition 2
I
The second definition generalizes to random variables the definition S(ω) = |Y (ω)|2 used for
deterministic signals of finite energy.
I
Define the finite Discrete Fourier transform of a sequence y1 , y1 , . . . , yN as
YN (ω)
N
X
=
yt e
−jωt
t=1
I
Second power spectral density definition is then
P(ω)
=
lim
N→∞
E
1
N
|YN (ω)|
2
or written directly as a function of the data
P(ω)
=
lim
N→∞
E

1
N
2 
N

X
−jωt yt e

t=1
8 / 14
Deterministic signals
Power spectral density -definitions
Power spectral density -properties
Power spectral density: Equivalence of the Definitions
I
I
Theorem: The Definitions 1 and 2 of the power spectral density are equivalent if the covariance function
1 PN
decays sufficiently rapidely, i.e if limN→∞ N
k=−N |k||r (k)| = 0.
Preliminary result (double summation formula) (Proof is left as an exercise) For any arbitrary function f
N X
N
X
f (t − s) =
t=1 s=1
I
Proof of the theorem We have
2
N
X
−jωt E yt e
=
t=1
=
N−1
X
(N − |τ |)f (τ )
τ =−N+1
E
N
X
yt e
−jωt
t=1
N X
N
X
N
X
ys e
jωs
=
s=1
r (t − s)e
N X
N
X
E [yt ys ]e
−jω(t−s)
t=1 s=1
−jω(t−s)
=
t=1 s=1
N−1
X
(N − |τ |)r (τ )e
−jωτ
τ =−N+1
where the last equality comes from the preliminary result on double sums. Now the second definition gives
 2 

N−1
N
 1 X
X
1
−jωt −jωτ
lim E
y
e
lim
(N − |τ |)r (τ )e
t
 = N→∞
N→∞  N N
t=1
τ =−N+1
=
∞
X
τ =−∞
r (τ )e
−jωτ
−
lim
N→∞
1
N−1
X
N τ =−N+1
|τ |r (τ )e
−jωτ
=
∞
X
r (τ )e
−jωτ
= P(ω)
τ =−∞
1 PN−1
since the term limN→∞ N
|τ |r (τ )e −jωτ is zero if r (k) decays rapidly.
τ =−N+1
9 / 14
Deterministic signals
Power spectral density -definitions
Power spectral density -properties
Properties of power spectral density
I
The power spectral density is real valued and nonnegative for all frequencies ω
P(ω) ≥ 0
∀ω
which can be seen from the second definition P(ω) = limN→∞ E
n
1
N
|YN (ω)|2
o
where the squared
value is always nonnegative.
I
For real valued signals yt the function P(ω) is even, i.e.,
P(ω) = P(−ω)
∀ω ∈ (−π, π)
Proof:
P(ω)
=
∞
X
r (k)e
k=−∞
=
r (0) + 2
−jωk
= r (0) +
∞
X
r (k)(e
k=1
∞
X
−jωk
+e
jωk
) = r (0) + 2
∞
X
r (k)cos(ωk)
k=1
r (k)cos(−ωk) = P(−ω)
k=1
10 / 14
Deterministic signals
Power spectral density -definitions
Power spectral density -properties
Properties of power spectral density
I
Theorem The transfer of power spectral density Pe (ω) of the input through an asymptotically stable linear
system having transfer function
H(z)
∞
X
=
hk z
−k
(1)
k=−∞
to the power spectral density Py (ω) of the output is given by
Py (ω)
=
2
|H(ω)| Pe (ω)
∀ω
where H(ω) = H(z) z=e jω .
y
e
t
P (ω)
e
t
H(q)
Py(ω)=|H(ω)|2Pe(ω)
11 / 14
Deterministic signals
Power spectral density -definitions
Power spectral density -properties
Properties of power spectral density
Proof of the power transfer theorem
I
The linear system output is the convolution of the input et with the impulse response hk :
yt
=
H(q)et =
∞
X
hk et−k
k=−∞
I
The autocorrelation of the output is
ry (k)
=
∞
X
Eyt yt−k = E
m=−∞
∞
X
=
∞
X
∞
X
hm et−m
∞
X
hs et−k−s =
s=−∞
∞
X
hm hs E [et−m et−k−s ]
m=−∞ s=−∞
hm hs re (k + s − m)
m=−∞ s=−∞
I
the power spectrum of the output is
Py (ω)
∞
X
=
ry (k)e
k=−∞

−jωk
∞
X
=
e
−jωk
∞
X
−jωm  

∞
X

=
H(ω)H (ω)Pe (ω) = |H(ω)| Pe (ω)
∗
hm e
jωs  
=
m=−∞
hm e
∞
X
hm hs re (k + s − m)
m=−∞ s=−∞
k=−∞

∞
X
s=−∞
∞
X

r (k + s − m)e
−jω(k+s−m) 
k+s−m=−∞
2
12 / 14
Deterministic signals
Power spectral density -definitions
Power spectral density -properties
Example of power spectral density
Example
I
I
I
1 e , where the polynomial A(z) has the following roots:
We consider an AR(6) process, yt = A(q)
t
z1,2 = 0.9e ±jπ/3 , z3,4 = 0.7e ±j3π/4 , z5,6 = 0.99e ±jπ/6 , and the white noise et has zero mean and
standard deviation σe = 1.
The power spectral density of the white noise is Pe (ω) = σe2 = 1.
The power spectral density of the random process yt obtained by passing white noise et through the linear
1 is
system with transfer function A(z)
Py (ω)
1
|A(e jω )|2
2
σe
short realizations of the white noise and of the random process yt are shown below
One realization of the white noise et
4
et
2
0
−2
−4
0
50
100
150
200
250
300
350
400
300
350
400
t ime t
One realization of AR process
20
10
yt
I
=
0
−10
−20
0
50
100
150
200
250
t ime t
13 / 14
Deterministic signals
Power spectral density -definitions
Power spectral density -properties
Example of power spectral density
the power spectral density Py (ω), and an estimate of the power spectral density obtained from a short
realization of the random process yt are shown below, first separated, then overlapped. The estimation
P̂y (ω) is our next topic.
True spectrum of AR process
15
4
10
3
5
2
logP ( ω)
yt
One realization of AR process
0
1
−5
0
−10
−1
−15
0
50
100
150
200
250
300
350
−2
400
0
0.5
1
t im e t
2
2.5
3
3.5
Estimated overlapped with true spectrum
4
3
3
logP̂ ( ω) , log P̂ ( ω)
4
2
1
0
−1
−2
−3
−4
1.5
fr e que nc y ω
Estimated spectrum from one realization
logP̂ ( ω)
I
2
1
0
−1
−2
−3
0
0.5
1
1.5
2
fr e que nc y ω
2.5
3
3.5
−4
0
0.5
1
1.5
2
2.5
3
3.5
fr e que nc y ω
14 / 14