Chapter 7: Spectral Density 7-1 7-2 Introduction Relation of Spectral Density to the Fourier Transform Weiner-Khinchine Relationship 7-3 Properties of Spectral Density 7-4 Spectral Density and the Complex Frequency Plane 7-5 Mean-Square Values From Spectral Density 7-6 Relation of Spectral Density to the Autocorrelation Function 7-7 White Noise Noise Terminology: White Noise, Black Noise, Pink Noise Contour Integration – (Appendix I) 7-8 Cross-Spectral Density 7-9 Autocorrelation Function Estimate of Spectral Density 7-10 Periodogram Estimate of Spectral Density 7-11 Examples and Application of Spectral Density Concepts: Relation of Spectral Density to the Fourier Transform o Weiner-Khinchine Relationship Properties of Spectral Density Spectral Density and the Complex Frequency Plane Mean-Square Values From Spectral Density Relation of Spectral Density to the Autocorrelation Function White Noise, Black Noise, Pink Noise Contour Integration – (Appendix I) Cross-Spectral Density Autocorrelation Function Estimate of Spectral Density Periodogram Estimate of Spectral Density Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9. B.J. Bazuin, Spring 2015 1 of 22 ECE 3800 Chapter 7: Spectral Density Much of engineering employs frequency domain methods for signal and system analysis. Therefore., we are interested in frequency domain concepts and analysis related to probability and statistics! But first, let’s review Fourier Transforms … Y w yt exp iwt dt If y(t) is in volts, Y(w) is expressed in terms of Volts/(radians per second). Thus, the Y(w) represents the relative magnitude and phase of steady-state sinusoids that may be summed to produce the original signal. As such, it describes the amplitude density as a function of frequency. In order for the Fourier Transform to exists, two conditions must be met: yt dt 1) The integral of yt over all time exists, or 2) There are a finite number of discontinuities in y(t). For sample functions containing random variables resulting in an ensemble of sample functions, the individual Fourier Transforms may exist, but we cannot readily describe the Fourier Transform of the entire ensemble. What can we describe for all sample functions of an ensemble that contains time- or period- or frequency-based information? For WSS random processes, the autocorrelation function is time based and, for ergodic processes, describes all sample functions in the ensemble! In these cases the WienerKhinchine relations is valid that allows us to perform the following. We can define a power spectral density for the ensemble as: S XX w R XX R XX exp iw d If Rxx(t) is a volts-squared (V2) type term, so a Fourier transform would expressed in terms of V2/(radians per second) or V2/Hz. Thus, the SXX(w) represents the relative magnitude of sinusoidal components that are present in the auto-correlation. Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9. B.J. Bazuin, Spring 2015 2 of 22 ECE 3800 Note that Rxx(t) is symmetric in time so we could also computed the Power Spectral Density as S XX w R XX 2 R XX cosw d 0 The power spectral density of a real auto-corr3elation function has no phase! By the way, if we define the Power Spectral Density, we can define the inverse … S XX w R XX 1 R XX t 2 R XX 1 S XX w S XX w expiwt dw More on the Fourier Transform of a time domain signal … yt exp iwt dt Y w The “Power Spectral Density” of the time domain signal can be described as S YY w Y w conj Y w Y w 2 Proof: S YY w y t y t dt exp iw d Swapping the order of integration S YY w y t y t exp iw d dt S YY w yt Y w expiwt dt S YY w Y w y t expiwt dt S YY w Y w conj Y w Y w 2 Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9. B.J. Bazuin, Spring 2015 3 of 22 ECE 3800 Summary: How to form the power spectral density. For non-random signals: take the magnitude squared of the Fourier Transform Y w y t yt exp iwt dt S YY w Y w conj Y w Y w For ergodic, WSS random signals: 2 form the auro-correaltion and take the Fourier Transform R XX EX t X t or 1 T 2T XX lim T xt xt dt xt xt T S XX w R XX R exp iw d XX For non-ergodic or non-WSS random signals: Wiener-Khinchine relations is not valid. Why this is very important … the Fourier Transform of a “single instantiation” of a random process may be meaningless or even impossible to generate. But if the random process can be described in terms of the autocorrelation function (all ergodic, WSS processes), then the power spectral density can be defined. I can then know what the expected frequency spectrum output looks like and I can design a system to keep the required frequencies and filters out the unneeded frequencies (e.g. noise and interference). In communications, most of the transmitted waveform is random (changing information content). But, based on probability, I can still design appropriate transmitting electronics and receiving systems to send, receive, and detect the information! Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9. B.J. Bazuin, Spring 2015 4 of 22 ECE 3800 Relation of Spectral Density to the Autocorrelation Function For “the right” random processes, power spectral density is the Fourier Transform of the autocorrelation: S XX w R XX EX t X t exp iw d For an ergodic process, we can use time-based processing to arrive at an equivalent result … T 1 XX lim T 2T xt xt dt xt xt T 1 E X t X t XX lim T 2T T xt xt dt T T 1 XX E X t X t lim xt xt dt exp iw d T 2T T T 1 XX lim xt xt exp iw d dt 2 T T T T 1 XX lim xt xt exp iwt iwt d dt 2 T T T T 1 xt exp iwt xt exp iwt d dt XX lim T 2T T T 1 xt exp iwt xt exp iwt d dt XX lim T 2T T X X w If there exists 1 T 2T XX lim T xt exp iwt X w dt T 1 XX X w lim T 2T T xt exp i wt dt T XX X w X w X w 2 Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9. B.J. Bazuin, Spring 2015 5 of 22 ECE 3800 Properties of the Fourier Transform: X w x x exp iw d For x(t) purely real X w x x cosw i sinw d X w x x cosw d i x sinw d X w E X w i O X w x cosw d i x sinw d x cosw d E X w and O X w x sinw d Notice that: x cosw d x cos w d E X w E X w O X w x sinw d x sin w d O X w Therefore, the real part is symmetric and the imaginary part is anti-symmetric! Note also, X w conj X w X w * X(w) is conjugate symmetric about the zero axis. Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9. B.J. Bazuin, Spring 2015 6 of 22 ECE 3800 Relating this to a real autocorrelation function where R XX R XX R XX E X w i O X w R XX R XX cosw i sinw d R XX R XX t cos wt i sin wt dt R XX R XX t coswt dt i R XX t sinwt dt R XX E X w i O X w Since Rxx is symmetric, we must have that R XX R XX and E X w i O X w E X w i O X w For this to be true, i O X w i O X w , which can only occur if the odd portion of the Fourier transform is zero! O X w 0 . This provides information about the power spectral density, S XX w R XX E X w S XX w E X w S XX w 0 The power spectral density necessarily contains no phase information! This is the quick way; now let’s see how your text got to the same point … First, investigate the Fourier Transform and see if this makes sense … Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9. B.J. Bazuin, Spring 2015 7 of 22 ECE 3800 Relation of Spectral Density to the Fourier Transform Basic Fourier Transform Y w yt exp iwt dt where yt dt To make a function “Fourier Transformable” we can apply “time domain windows” to limit function to be transformed in either time or even magnitude. More typically, a windowed Spectrum Y w lim T T Windowt yt exp iwt dt T The most popular window is a rectangular window from –T to T. This is used to make time “finite” … an infinite time interval has yet to be completed! Other windows are possible too. A rectangle Function cuts the signal to a finite time period (that should be integrable). An exponential scaling can guarantee that a signal waveform goes to zero as time goes to infinity. For most applications, we are interested in the spectral magnitude and/or the spectral phase information, Y w or Y w . Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9. B.J. Bazuin, Spring 2015 8 of 22 ECE 3800 For the time window based on T, the time function can be described as y t , t T yT t t T 0, For this case, it can be shown that the integral of the magnitude square is also finite, or yT t dt 2 Based on the transform existing, Parseval’s theorem can provide some useful insight. Parseval’s theorem states that for two transformable functions with known transforms, the following holds: 1 f t g t dt 2 F w G w dw For the above time-bounded signal, the results for f t yt and g t y t are T yT t 2 T 1 dt 2 1 Y w Y w dw 2 Y w dw 2 As this appear to be the time integral of power, let make it the average power in the time interval by dividing by 2T: 1 2T T yT t 2 T 1 dt 4T Y w dw 2 The left hand is now related to the signal’s time averaged second moment, for which the time average and statistical mean are equivalent if y is ergodic. Using the limit, this may be more readily seen as: 1 T 2T lim T yT t dt yT t 2 T 2 1 T 4T lim Y w 2 dw Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9. B.J. Bazuin, Spring 2015 9 of 22 ECE 3800 Taking an expected value … T 1 1 2 2 2 E lim yT t dt Y lim E Y w dw T 2T T 4T T Y 1 lim T 2T 2 E yT t T 2 T Y2 Y2 1 2 1 dt 2 2 E Y w dw lim 2T T 2 E Y w dw lim 2T T Observing the right hand side, there appears to be a function related to frequency that describes the FFT of the time average or letting Y 2 Y 2 1 2 SYY w dw 2 E Y w where SYY w lim 2T T This function is also defined as the spectral density function (or power-spectral density) and is defined for both f and w as: 2 E Y w SYY w lim 2T T 2 EY f SYY f lim 2T T or The 2nd moment based on the spectral densities is defined, as: Y 2 1 2 SYY w dw and Y 2 SYY f df Note: The result is a power spectral density (in Watts/Hz), not a voltage spectrum as (in V/Hz) that you would normally compute for a Fourier transform. Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9. B.J. Bazuin, Spring 2015 10 of 22 ECE 3800 Exercise 7-2.2 A stationary random process has a two-sided spectral density given by S XX w a.) 24 w 2 16 Find the mean-square value of the process. X 1 2 1 S XX w dw 2 w 24 2 16 dw 24 2 3 arctan arctan w 24 1 dw arctan 2 2 16 16 w 16 2 X X2 2 1 3 3 3 2 2 b.) Find the mean square value of the process in the frequency band of +/- 1 Hz centered on the origin. 24 X 1Hz 2 2 2 X 1Hz 2 2 2 24 1 w dw arctan 2 2 16 4 2 w 16 1 3 2 2 arctan arctan 4 4 3 2 1.004 1.9173 This refers to the total signal power and the signal power in a defined part of the frequency band (that could be extracted or remain after perfect filtering). Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9. B.J. Bazuin, Spring 2015 11 of 22 ECE 3800 Properties of the Power Spectral Density The power spectral density as a function is always real, positive, (never negative as it is a magnitude) and an even function in w. As an even function, the PSD may be expected to have a polynomial form as: S XX w S 0 w 2n a 2n 2 w 2n 2 a 2n 4 w 2n 4 a 2 w 2 a0 w 2m b2m 2 w 2m 2 b2m 4 w 2m 4 b2 w 2 b0 where m>n. Notice the squared terms, any odd power would define an anti-symmetric element that, by definition and proof, can not exist! Finite property in frequency: The Power Spectral Density must also approach zero as w approached infinity …. Therefore, w 2n a 2n2 w 2n2 a 2 w 2 a0 w 2n 1 S S XX w lim S 0 2 m lim lim S 0 2m n 0 0 m 2 2 2 2 m w w w w b2 m 2 w w w b2 w b0 For m>n, the condition will be met. Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9. B.J. Bazuin, Spring 2015 12 of 22 ECE 3800 Example: Doing it the textbook way … form the PSD given X t A B cos2f 0 t where A, B, and f0 are constant and theta is a uniform R.V. from 0 to 2pi. Assuming a truncated sequence FX f T A B cos2f 0t exp j 2ft dt T FX f t rect A B cos2f 0 t exp j 2ft dt 2T Fourier Transform Property: The FT of a product is the convolution of the FTs. Therefore, B B FX f 2T sinc 2T f A f f f 0 exp j f f 0 exp j 2 2 FX f 2 AT sinc2T f BT sinc2T f f 0 exp j BT sinc2T f f 0 exp j Forming the magnitude squares F X f F X f FX f 2 4 A 2T 2 sin c2T f 2 B 2T 2 sin c2T f f 0 2 B 2T 2 sin c2T f f 0 2 fn E FX f 2 4A T 2 2 sin c2T f B 2T 2 sin c2T f f 0 2 2 B 2T 2 sin c2T f f 0 E fn 2 The expected value of the remaining phase is E fn 0 Therefore, 2 E FX f 4 A 2T 2 sin c2T f 2 B 2T 2 sin c2T f f 0 2 B 2T 2 sin c2T f f 0 2 Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9. B.J. Bazuin, Spring 2015 13 of 22 ECE 3800 Then for 2 EY f SYY f lim 2T T 1 S f lim T 2T 4 A 2T 2 sin c2T f 2 B 2T 2 sin c2T f f 2 0 B 2T 2 sin c2T f f 0 2 2 2 2 B 2T sin c2T f f 0 2 A 2T sin c2T f 4 S f lim T B 2 2 4 2T sin c2T f f 0 But this can be defined, in the limit as B2 B2 S f A f f f0 f f0 4 4 2 (what a pain in the ….) Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9. B.J. Bazuin, Spring 2015 14 of 22 ECE 3800 Alternately, using the autocorrelation function X t A B cos2f 0 t R XX E X t X t E A B cos2f 0 t A B cos2f 0 t A 2 AB cos2f 0 t AB cos2f 0 t R XX E B 2 cos2f 0 t cos2f 0 t R XX A 2 E AB cos2f 0 t AB cos2f 0 t E B 2 cos2f 0 t cos2f 0 t R XX A 2 B 2 E cos2f 0 t cos2f 0 t 1 1 R XX A 2 B 2 E cos2f 0 cos2f 0 2t 2 2 2 R XX A 2 B2 cos2f 0 2 Performing the Fourier transform 2 B2 S f R XX A cos2f 0 2 S f R XX A 2 f B2 B2 f f0 f f0 4 4 Which do you think is easier? One derivation attempts to provide a better “physical meaning” but may be confusing. The other definition may be harder to accept at face value. (a little math that produces the correct result) Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9. B.J. Bazuin, Spring 2015 15 of 22 ECE 3800 Another Example of a Discrete Spectral Density (p. 267) X t 5 10 sin2 6 t 1 8 cos2 12 t 2 where the phase angles are uniformly distributed R.V from 0 to 2π. With practice, we can see that 1 1 R XX A 2 B 2 E cos2 f1 cos2 f1 2t 21 2 2 1 1 C 2 E cos2 f 2 cos2 f 2 2t 2 2 2 2 which lead to R XX 25 100 64 cos2 6 cos2 12 2 2 And then taking the Fourier transform S XX f 25 f 100 1 1 1 64 1 f 6 f 6 f 12 f 12 2 2 2 2 2 2 S XX f 25 f 25 f 6 f 6 16 f 12 f 12 We also know from the before 1 X 2 2 S w dw S f df XX XX Therefore, the 2nd moment can be immediately computed as X2 25 f 25 f 6 f 6 16 f 12 f 12 df X 2 25 25 1 1 16 1 1 25 50 32 107 We can also see that X E 5 10 sin 2 6 t 1 8 cos2 12 t 2 5 So, 2 107 5 2 82 Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9. B.J. Bazuin, Spring 2015 16 of 22 ECE 3800 Another example: Determine the autocorrelation of the binary sequence, assuming p=0.5. xt pt t 0 k T A k k xt pt A k k t t 0 k T Determine the auto correlation of the discrete time sequence y t A k k t t 0 k T E y t y t E Ak t t 0 k T A j t t 0 j T j k R yy E y t y t E A j Ak t t 0 k T t t 0 j T k j 2 Ak t t 0 k T t t 0 k T k R yy E A j Ak t t 0 k T t t 0 j T k jj k R yy E A k k 2 E t t EA k j j k j 0 k T t t 0 k T Ak E t t 0 k T t t 0 j T T1 EA R yy E Ak 2 2 k 2 1 m T T m m0 T1 EA R yy E Ak E Ak 2 2 k 1 m T T m Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9. B.J. Bazuin, Spring 2015 17 of 22 ECE 3800 R yy A2 1 1 A2 m T T T m S yy f A2 1 1 1 m A2 f T T T m T From here, it can be shown that S xx f P f S yy f 2 m 1 1 2 S xx f P f A2 A2 2 f T T T m S xx f P f 2 A2 T P f 2 A2 T 2 m f T m This is a magnitude scaled version of the power spectral density of the pulse shape and numerous impulse responses with magnitudes shaped by the pulse at regular frequency intervals based on the signal periodicity. The result was picture in the textbook as … Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9. B.J. Bazuin, Spring 2015 18 of 22 ECE 3800 Binary Pulse Amplitude (PAM) signaling formats I did a series of similar definitions for the ECE4600 Communications course a few years ago … the results follow (a) Unipolar RZ & NRZ , (b) Polar RZ & NRZ , (c) Bipolar NRZ , (d) Split-phase Manchester, and (e) Polar quaternary NRZ. From: A. Bruce Carlson, P.B. Crilly, Communication Systems, 5th ed., McGraw-Hill, 2010. Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9. B.J. Bazuin, Spring 2015 19 of 22 ECE 3800 PAM Power Spectral Density: Polar NRZ The random signal can be describes as vt a k p Td k t Td k Tb rect T b E an 0, E an 2 2 1 , Tb 0 Td Tb and E a j ak 0, Rvv Evt vt 2 1 , Tb for j k Tb Tb S vv w E vt vt 2 Tb sinc 2 f Tb PAM Power Spectral Density: Arbitrary Pulse – similar to our textbook vt a k p Td t Td k D p D 1 , D 0 Td D and E an ma , E an a ma 2 S vv f a2 D P f 2 n0 n0 and Tb D, rb 2 m a D n S vv f a rb P f ma rb 2 2 2 1 2 P f Ra n exp j 2 f D D n a 2 ma 2 , Ra n 2 ma , S vv f k 2 2 1 D 2 n n P f D D Pn r n b 2 f n rb Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9. B.J. Bazuin, Spring 2015 20 of 22 ECE 3800 Power spectrum of Unipolar, binary RZ signal t p t rect Tb 2 E an f 1 sinc rect 2 rb t where P f 2 rb 2 rb A A2 2 , E an 2 2 2 A2 2 m ,n 0 a a 2 and Ra n 2 m 2 A , n0 a 4 2 2 f A2 A2 n sinc S vv f sinc f n rb 16 rb 2 2 rb 16 n Power spectrum of Unipolar, binary NRZ signal t f 1 pt rect rectrb t where P f sinc rb Tb rb E an A A2 2 , E an 2 2 2 A2 2 m ,n 0 a a 2 and Ra n 2 m 2 A , n0 a 4 2 f A2 A2 2 sinc S vv f sincn f n rb 4 rb 4 n rb But based in the sinc function equals 2 f A2 A2 sinc S vv f f 4 rb 4 r b Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9. B.J. Bazuin, Spring 2015 21 of 22 ECE 3800 Power spectrum of Polar, binary RZ signal (+/- A/2) t p t rect Tb 2 f 1 sinc rect 2 rb t where P f 2 rb 2 rb E an 0, E an 2 2 A 2 m 2 A2 , n 0 a 4 and Ra n a 4 2 ma 0, n 0 f A2 S vv f sinc 16 rb 2 rb 2 Power spectrum of Polar, binary NRZ signal (+/- A/2) t f 1 pt rect rectrb t where P f sinc rb Tb rb E an 0, E an 2 2 m 2 A2 , n 0 a 4 and Ra n a A 4 2 ma 0, n 0 2 f A2 S vv f sinc 4 rb rb 2 Why do we care? The bandwidth and spectral characteristics of the signals are very important. Issues include spectral capacity, filter selections, adjacent signal interference, etc. Notes and figures are based on or taken from materials in the course textbook: Probabilistic Methods of Signal and System Analysis (3rd ed.) by George R. Cooper and Clare D. McGillem; Oxford Press, 1999. ISBN: 0-19-512354-9. B.J. Bazuin, Spring 2015 22 of 22 ECE 3800