16.322 Stochastic Estimation and Control, Fall 2004
Prof. Vander Velde
Lecture 12
Non-zero power at zero frequency
Non-zero power at non-zero frequency
If Rxx (τ ) includes a sinusoidal component corresponding to the component
x (t ) = A sin(ω0t + θ )
where θ is uniformly distributed over 2π , A is random, independent of θ , that
component will be
1
Rxx (τ ) = A2 cos ω0τ
2
∞
1
S xx (ω ) = ∫ A2 cos ω0τ e − jωτ dτ
2
−∞
∞
=
1 2 1 jω0τ
A ∫ ⎡⎣ e + e − jω0τ ⎤⎦ e − jωτ dτ
2 −∞ 2
=
1 2 1 ⎡ j(ω −ω0 )τ
− j ω +ω τ
+ e ( 0 ) ⎤ dτ
A ∫
e
⎦
2 −∞ 2 ⎣
∞
1
= π A2 ⎡⎣δ (ω − ω0 ) + δ (ω + ω0 )⎤⎦
2
Page 1 of 8
16.322 Stochastic Estimation and Control, Fall 2004
Prof. Vander Velde
Units of S xx
Mean squared value per unit frequency interval.
Usually:
∞
S xx (ω ) = ∫ Rxx (τ )e − jωτ dτ
∞
1
x =
2π
2
∞
∫S
xx
(ω )d ω
−∞
∞
=
∫S
xx
( f )df , where f =
−∞
ω
2π
q2
In this case, S xx ~
Hz
Next most common:
∞
1
S xx (ω ) =
Rxx (τ )e − jωτ dτ
∫
2π −∞
∞
x2 =
∫S
xx
(ω )d ω
−∞
In this case, S xx ~
q2
= q 2 sec
rad / sec
There is an alternate form of the power spectral density function.
Since S xx (ω ) is a measure of the power density of the harmonic components of
x(t ) , one should be able to get S xx (ω ) also from the Fourier Transform of x(t )
which is a direct decomposition of x (t ) into its infinitesimal harmonic
components.
This is true, and is the approach taken in the text. One difficulty is that the
Fourier Transform does not converge for members of stationary ensembles. The
mathematics are handled by a limiting process.
Page 2 of 8
16.322 Stochastic Estimation and Control, Fall 2004
Prof. Vander Velde
⎧ x (t ),
Define xT (t ) = ⎨
⎩ 0,
X T (ω ) =
∞
∫x
T
( −T < t < T )
elsewhere
(t )e − jωτ dτ
−∞
T
=
∫ x (t )e
− jωt
dt
−T
Then
S xx (ω ) = lim
T →∞
X T (ω )* X T (ω )
2T
2
X (ω )
= lim T
T →∞
2T
Notice that the operations of transforming and averaging and product are done
in opposite order here than if the transform of the autocorrelation function is
calculated.
If one has only a finite record of a single random function, and S xx (ω ) is to be so
calculated approximately under the ergodic hypothesis, it can be done either
way.
T −τ
2
Rxx (τ ) =
x (t ) x (t + τ )dτ
T − τ ∫0
τ max
S xx (ω ) = 2
∫
Rxx (τ ) cos ωτ dτ
0
x (ω ) =
T
∫ x (t )e
− jωt
dt
−T
X (ω )* X (ω )
S xx (ω ) =
2T
Standard deviation of S xx measured this way is approximately equal to mean.
Page 3 of 8
16.322 Stochastic Estimation and Control, Fall 2004
Prof. Vander Velde
The second approach is faster.
In fact, with the advent of the Fast Fourier Transform (initiated by Cooley and
Tukey), even if one wanted to calculate Rxx (τ ) from x (t ) , it is faster to transform
x (t ) to X (ω ) , form S xx (ω ) , and transform to get Rxx (τ ) than to integrate
x(t ) x (t + τ ) directly for all desired values of τ .
The Fast Fourier Transform is an amazingly efficient procedure for digital
calculation of finite Fourier Transforms.
References:
Full issue – IEEE Transactions on Audio and Electroacoustics, Vol. AU-15, No.2;
June 1967.
Tutorial article: Brighton, E.O. and Morrow, R.E.: The Fast Fourier Transform,
IEEE Spectrum; Dec. 1967.
Cross spectral density
In dealing with more than one random process, the cross power spectral density
arises naturally. For example, if
z (t ) = x (t ) + y (t )
where x(t ) and y (t ) are members of random ensembles, then we found before
that
Rzz (τ ) = Rxx (τ ) + Rxy (τ ) + R yx (τ ) + R yy (τ )
so that
S zz (ω ) =
∞
∫R
zz
(τ )e − jωτ dτ
−∞
= S xx (ω ) + S xy (ω ) + S yx (ω ) + S yy (ω )
where we have defined cross spectral density functions:
S xy (ω ) =
∞
∫R
xy
(τ )e − jωτ dτ
−∞
S yx (ω ) =
∞
∫R
yx
(τ )e − jωτ dτ
−∞
This is equivalent to the definition
Page 4 of 8
16.322 Stochastic Estimation and Control, Fall 2004
Prof. Vander Velde
S xy (ω ) = lim
T →∞
X T (ω )*YT (ω )
2T
YT (ω )* X T (ω )
T →∞
2T
We note from this that:
S yx (ω ) = S xy (ω )*
S yx (ω ) = lim
so that the sum of these two as they appear in S zz (ω ) is real.
Also note that S xy ( −ω ) = S xy (ω )* = S yx (ω ) .
Examples of Random Processes Analytically Defined
Example: Random step function
Amplitude an independent, random
Change points tn , Poisson-distributed with average density λ (points per
second)
P(k ) =
1
(λ τ
k!
)
k
e−λ τ
P(0) = e − λ τ
Rxx (τ ) = E [ x (t ) x (t + τ )]
= P(at least one change point in τ )a 2 + P (no change point in τ )a 2
(
)
= 1 − e−λ τ a 2 + e−λ τ a 2
= σ a2e − λ τ + a 2
Page 5 of 8
16.322 Stochastic Estimation and Control, Fall 2004
Prof. Vander Velde
S xx (ω ) =
∞
∫ ⎡⎣a
2
−∞
+ σ a2e − λ τ ⎤⎦ e − jωτ dτ
= 2π a 2δ (ω ) +
∞
0
2 − jωτ λτ
2 − jωτ − λτ
∫ σ a e e dτ + ∫ σ a e e dτ
−∞
= 2π a δ (ω ) +
2
0
σ
2
a
λ − jω
0
e
( λ − jω )τ
σ a2
e −( λ + jω )τ
+
−λ − jω
−∞
∞
0
⎡ 1
1 ⎤
= 2π a 2δ (ω ) + σ a2 ⎢
+
⎥
⎣ λ − jω λ + jω ⎦
2σ a2λ
2
= 2π a δ (ω ) + 2
λ + ω2
Example: A signal random process
Reference: Newton, G.C, L.A. Gould and J.F. Kaiser. Design of Linear Feedback
Controls. John Wiley, 1961. p.100. Papoulis calls this the semirandom
telegraph signal; p.288.
A signal takes the values plus and minus x0 only. It switches from one level
to the other at “event points” which are Poisson distributed in time with
constant average frequency λ . This is sometimes called a telegraph signal.
Page 6 of 8
16.322 Stochastic Estimation and Control, Fall 2004
Prof. Vander Velde
Average rate of occurrence of change points = λ points per second
Rxx (τ ) = E ⎡⎣ x (t ) x(t + τ ) ⎤⎦
= P(even number of change points in the interval τ ) x02
+ P(odd number of change points in the interval τ )( − x02 )
= x02
=
1
(λ τ
k =0,2,4,... k !
∑
)
k
e − λ τ − x02
1
k
( −1) ( λ τ
k =0,1,2,... k !
∑
2 −λ τ
0
=xe
∞
∑
( −λ τ )
)
k
1
(λ τ
k =1,3,5,... k !
∑
)
k
e−λ τ
e − λ τ x02
k
k!
k =0
2 −2 λ τ
0
=xe
You are generating higher harmonics in this case, as at each change point the
amplitude changes sign. In the previous example, changes in amplitude at
change point may be far smaller.
S xx (ω ) =
2(2)λ x02
ω 2 + ( 2λ )
2
Example: Binary function with an arbitrary amplitude distribution
The problem considers a binary function with a more general amplitude
distribution. The distribution of a is restricted to ±1 with equal probability.
That gives the pseudo-random binary code used by GPS.
Page 7 of 8
16.322 Stochastic Estimation and Control, Fall 2004
Prof. Vander Velde
t0 is uniformly distributed over (0, T )
Change points are periodic with period T
Amplitudes independent with a , a 2
Rxx (τ ) = E [ x (t ) x (t + τ )]
= P(1 or more change points in τ )a 2 + P (No change point in τ )a 2
τ ⎞
⎛
a 2 + ⎜1 − ⎟ a 2 , τ ≤ T
T
⎝ T⎠
⎧ 2⎛
τ ⎞ 2
⎪σ a ⎜ 1 − ⎟ + a , τ ≤ T
T⎠
Rxx (τ ) = ⎨ ⎝
⎪a 2,
τ >T
⎩
=
τ
Page 8 of 8