A Plea for Adaptive Data Analysis: Instantaneous Frequencies and Trends For Nonstationary Nonlinear Data Norden E. Huang Research Center for Adaptive Data Analysis National Central University Zhongli, Taiwan, China Hot Topic Conference, 2011 Instantaneous Frequencies and Trends For Nonstationary Nonlinear Data In search of frequency I found the trend and other information Prevailing Views on Instantaneous Frequency The term, Instantaneous Frequency, should be banished forever from the dictionary of the communication engineer. J. Shekel, 1953 The uncertainty principle makes the concept of an Instantaneous Frequency impossible. K. Gröchennig, 2001 How to define frequency? It seems to be trivial. But frequency is an important parameter for us to understand many physical phenomena. Definition of Frequency Given the period of a wave as T ; the frequency is defined as 1 T . Traditional Definition of Frequency • frequency = 1/period. • • • • • Definition too crude Only work for simple sinusoidal waves Does not apply to nonstationary processes Does not work for nonlinear processes Does not satisfy the need for wave equations The Idea and the need of Instantaneous Frequency According to the classic wave theory, the wave conservation law is based on a gradually changing φ(x,t) such that k , ; t k 0 . t Therefore, both wave number and frequency must have instantaneous values and differentiable. Instantaneous Frequency Velocity distance ; mean velocity time Newton v Frequency dx dt 1 ; mean frequency period HHT defines the phase function d dt So that both v and can appear in differential equations. Hilbert Transform : Definition For any x( t ) Lp , y( t ) 1 x( ) d , t then, x( t )and y( t ) form the analytic pairs: z( t ) x( t ) i y( t ) a( t ) e i ( t ) , where a( t ) x 2 y 2 1 / 2 and ( t ) tan 1 y( t ) . x( t ) The Traditional View of the Hilbert Transform for Data Analysis Traditional View a la Hahn (1995) : Data LOD Traditional View a la Hahn (1995) : Hilbert Traditional Approach a la Hahn (1995) : Phase Angle Traditional Approach a la Hahn (1995) : Phase Angle Details Traditional Approach a la Hahn (1995) : Frequency Why the traditional approach does not work? Hilbert Transform a cos + b : Data Hilbert Transform a cos + b : Phase Diagram Hilbert Transform a cos + b : Phase Angle Details Hilbert Transform a cos + b : Frequency The Empirical Mode Decomposition Method and Hilbert Spectral Analysis Sifting Empirical Mode Decomposition: Methodology : Test Data Empirical Mode Decomposition: Methodology : data and m1 Empirical Mode Decomposition: Methodology : data & h1 Empirical Mode Decomposition: Methodology : h1 & m2 Empirical Mode Decomposition: Methodology : h3 & m4 Empirical Mode Decomposition: Methodology : h4 & m5 Empirical Mode Decomposition Sifting : to get one IMF component x( t ) m 1 h1 , h1 m 2 h2 , ..... ..... hk 1 m k hk . hk c1 . The Stoppage Criteria The Cauchy type criterion: when SD is small than a preset value, where T SD h t 0 k 1 ( t ) hk ( t ) 2 T 2 h k 1 ( t ) t 0 Or, simply pre-determine the number of iterations. Effects of Sifting • To remove ridding waves • To reduce amplitude variations • The systematic study of the stoppage criteria leads to the conjecture connecting EMD and Fourier Expansion. Empirical Mode Decomposition: Methodology : IMF c1 Definition of the Intrinsic Mode Function (IMF): a necessary condition only! Any function having the same numbers of zero cros sin gs and extrema ,and also having symmetric envelopes defined by local max ima and min ima respectively is defined as an Intrinsic Mode Function ( IMF ). All IMF enjoys good Hilbert Transform : c( t ) a( t )e i ( t ) Empirical Mode Decomposition: Methodology : data, r1 and m1 Empirical Mode Decomposition Sifting : to get all the IMF components x( t ) c1 r1 , r1 c2 r2 , . . . rn 1 cn rn . x( t ) n c j1 j rn . Definition of Instantaneous Frequency The Fourier Transform of the Instrinsic Mode Funnction, c( t ), gives W ( ) i ( t ) a( t ) e dt t By Stationary phase approximation we have d ( t ) , dt This is defined as the Ins tan tan eous Frequency . An Example of Sifting & Time-Frequency Analysis Length Of Day Data LOD : IMF Orthogonality Check • Pair-wise % • Overall % • • • • • • • • • • • 0.0003 0.0001 0.0215 0.0117 0.0022 0.0031 0.0026 0.0083 0.0042 0.0369 0.0400 • 0.0452 LOD : Data & c12 LOD : Data & Sum c11-12 LOD : Data & sum c10-12 LOD : Data & c9 - 12 LOD : Data & c8 - 12 LOD : Detailed Data and Sum c8-c12 LOD : Data & c7 - 12 LOD : Detail Data and Sum IMF c7-c12 LOD : Difference Data – sum all IMFs Properties of EMD Basis The Adaptive Basis based on and derived from the data by the empirical method satisfy nearly all the traditional requirements for basis empirically and a posteriori: Complete Convergent Orthogonal Unique The combination of Hilbert Spectral Analysis and Empirical Mode Decomposition has been designated by NASA as HHT (HHT vs. FFT) Comparison between FFT and HHT 1. FFT : x( t ) aj e i jt . j 2. HHT : x( t ) a j ( t ) e j i j ( t )d . Comparisons: Fourier, Hilbert & Wavelet Speech Analysis Hello : Data Four comparsions D Traditional View a la Hahn (1995) : Hilbert Mean Annual Cycle & Envelope: 9 CEI Cases Mean Hilbert Spectrum : All CEs For For quantifying nonlinearity we need instantaneous frequency. How to define Nonlinearity? How to quantify it through data alone? The term, ‘Nonlinearity,’ has been loosely used, most of the time, simply as a fig leaf to cover our ignorance. Can we be more precise? How is nonlinearity defined? Based on Linear Algebra: nonlinearity is defined based on input vs. output. But in reality, such an approach is not practical: natural system are not clearly defined; inputs and out puts are hard to ascertain and quantify. Furthermore, without the governing equations, the order of nonlinearity is not known. In the autonomous systems the results could depend on initial conditions rather than the magnitude of the ‘inputs.’ The small parameter criteria could be misleading: sometimes, the smaller the parameter, the more nonlinear. Linear Systems Linear systems satisfy the properties of superposition and scaling. Given two valid inputs to a system H, x1 ( t ) and x2 ( t ) as well as their respective outputs y1 ( t ) H{ x1 ( t )} and y2 (t) = H{ x2 ( t )} then a linear system, H, must satisfy y1 ( t ) y1 ( t ) H{ x1 ( t ) x2 ( t )} for any scalar values α and β. How is nonlinearity defined? Based on Linear Algebra: nonlinearity is defined based on input vs. output. But in reality, such an approach is not practical: natural system are not clearly defined; inputs and out puts are hard to ascertain and quantify. Furthermore, without the governing equations, the order of nonlinearity is not known. In the autonomous systems the results could depend on initial conditions rather than the magnitude of the ‘inputs.’ The small parameter criteria could be misleading: sometimes, the smaller the parameter, the more nonlinear. How should nonlinearity be defined? The alternative is to define nonlinearity based on data characteristics: Intra-wave frequency modulation. Intra-wave frequency modulation is known as the harmonic distortion of the wave forms. But it could be better measured through the deviation of the instantaneous frequency from the mean frequency (based on the zero crossing period). Characteristics of Data from Nonlinear Processes d2 x 3 x x cos t 2 dt d2 x x 2 dt 1x 2 cos t Spring with position dependent cons tan t , int ra wave frequency mod ulation; therefore , we need ins tan tan eous frequency. Duffing Pendulum x d2x 2 x ( 1 x ) cos t . 2 dt Duffing Equation : Data Hilbert’s View on Nonlinear Data Intra-wave Frequency Modulation A simple mathematical model x( t ) cos t sin2t Duffing Type Wave Data: x = cos(wt+0.3 sin2wt) Duffing Type Wave Perturbation Expansion For 1 , we can have x( t ) cos t sin 2 t cos t cos sin 2 t sin t sin sin 2 t cos t sin t sin 2 t .... 1 cos t cos 3 t .... 2 2 This is very similar to the solution of Duffing equation . Duffing Type Wave Wavelet Spectrum Duffing Type Wave Hilbert Spectrum Degree of nonlinearity Let us consider a generalized intra-wave frequency modulation model as: d x( t ) cos( t sin t ) IF= 1 cos t dt IF IFz DN (Degree of Nolinearity ) should be IFz 2 1/ 2 . 2 Depending on the value of , we can have either a up-down symmetric or a asymmetric wave form. Degree of Nonlinearity • DN is determined by the combination of δη precisely with Hilbert Spectral Analysis. Either of them equals zero means linearity. • We can determine δ and η separately: – η can be determined from the instantaneous frequency modulations relative to the mean frequency. – δ can be determined from DN with known η. NB: from any IMF, the value of δη cannot be greater than 1. • The combination of δ and η gives us not only the Degree of Nonlinearity, but also some indications of the basic properties of the controlling Differential Equation, the Order of Nonlinearity. Stokes Models d2x 2 2 x x cos t with ; =0.1. 2 dt 25 Stokes I: is positive ranging from 0.1 to 0.375; beyond 0.375, there is no solution. Stokes II: is negative ranging from 0.1 to 0.391; beyond 0.391, there is no solution. Data and IFs : C1 Stokes Model c1: e=0.375; DN=0.2959 2 1.5 1 Frequency : Hz 0.5 0 -0.5 -1 -1.5 -2 IFq IFz Data IFq-IFz -2.5 -3 0 20 40 60 80 100 120 Time: Second 140 160 180 200 Summary Stokes I Summary Stokes I 0.35 C1 C2 CDN Combined Degree of Stationarity 0.3 0.25 0.2 0.15 0.1 0.05 0 0.1 0.15 0.2 0.25 0.3 Epsilon 0.35 0.4 0.45 Lorenz Model • Lorenz is highly nonlinear; it is the model equation that initiated chaotic studies. • Again it has three parameters. We decided to fix two and varying only one. • There is no small perturbation parameter. • We will present the results for ρ=28, the classic chaotic case. Phase Diagram for ro=28 Lorenz Phase : ro=28, sig=10, b=8/3 30 20 z 10 0 -10 -20 -30 20 10 50 40 0 30 20 -10 y 10 -20 0 x X-Component DN1=0.5147 CDN=0.5027 Data and IF Lorenz X : ro=28, sig=10, b=8/3 50 IFq IFz Data IFq-IFz 40 Frequency : Hz 30 20 10 0 -10 0 5 10 15 20 25 30 Time : second 35 40 45 50 Spectra data and IF 10 Spectra of Data and IF : Lorenz 28 2 Data Frequency 10 Spectral Density 10 10 10 10 10 1 0 -1 -2 -3 -4 -1 10 0 10 10 Frequency : Hz 1 10 2 Comparisons Fourier Wavelet Hilbert Basis a priori a priori Adaptive Frequency Integral transform: Global Integral transform: Regional Differentiation: Local Presentation Energy-frequency Energy-timefrequency Energy-timefrequency Nonlinear no no yes, quantifying Non-stationary no yes Yes, quantifying Uncertainty yes yes no Harmonics yes yes no How to define Trend ? Parametric or Non-parametric? Intrinsic vs. extrinsic approach? The State-of-the arts: Trend “One economist’s trend is another economist’s cycle” Watson : Engle, R. F. and Granger, C. W. J. 1991 Long-run Economic Relationships. Cambridge University Press. Philosophical Problem Anticipated 名不正則言不順 言不順則事不成 ——孔夫子 On Definition Without a proper definition, logic discourse would be impossible. Without logic discourse, nothing can be accomplished. Confucius Definition of the Trend Within the given data span, the trend is an intrinsically fitted monotonic function, or a function in which there can be at most one extremum. The trend should be an intrinsic and local property of the data; it is determined by the same mechanisms that generate the data. Being local, it has to associate with a local length scale, and be valid only within that length span, and be part of a full wave length. The method determining the trend should be intrinsic. Being intrinsic, the method for defining the trend has to be adaptive. All traditional trend determination methods are extrinsic. Algorithm for Trend • Trend should be defined neither parametrically nor non-parametrically. • It should be the residual obtained by removing cycles of all time scales from the data intrinsically. • Through EMD. Global Temperature Anomaly Annual Data from 1856 to 2003 Global Temperature Anomaly 1856 to 2003 IMF Mean of 10 Sifts : CC(1000, I) Mean IMF STD IMF Statistical Significance Test Data and Trend C6 Rate of Change Overall Trends : EMD and Linear Conclusion • With EMD, we can define the true instantaneous frequency and extract trend from any data. • We can also talk about nonlinearity quantitatively. • Among other applications, the degree of nonlinearity could be used to set an objective criterion in structural health monitoring and to quantify the degree of nonlinearity in natural phenomena; the trend could be used in financial as well as natural sciences. • These are all possible because of adaptive data analysis method. The Job of a Scientist The job of a scientist is to listen carefully to nature, not to tell nature how to behave. Richard Feynman To listen is to use adaptive method and let the data sing, and not to force the data to fit preconceived modes. All these results depends on adaptive approach. Thanks What This Means • EMD separates scales in physical space; it generates an extremely sparse representation for any given data. • Added noises help to make the decomposition more robust with uniform scale separations. • Instantaneous Frequency offers a total different view for nonlinear data: instantaneous frequency needs no harmonics and is unlimited by uncertainty principle. • Adaptive basis is indispensable for nonstationary and nonlinear data analysis • EMD establishes a new paradigm of data analysis Outstanding Mathematical Problems 1. Mathematic rigor on everything we do. (tighten the definitions of IMF,….) 2. Adaptive data analysis (no a priori basis methodology in general) 3. Prediction problem for nonstationary processes (end effects) 4. Optimization problem (the best stoppage criterion and the uniqueness of the decomposition) 5. Convergence problem (best spline implement, and 2-D) Conclusion Adaptive method is the only scientifically meaningful way to analyze nonlinear and nonstationary data. It is the only way to find out the underlying physical processes; therefore, it is indispensable in scientific research. EMD is adaptive; It is physical, direct, and simple. But, we have a lot of problems And need a lot of helps! Up Hill Does the road wind up-hill all the way? Yes, to the very end. Will the day’s journey take the whole long day? From morn to night, my friend. --- Christina Georgina Rossetti A less poetic paraphrase • There is no doubt that our road will be long and that our climb will be steep. …… • But, anything is possible. --- Barack Obama 18 Jan 2009, Lincoln Memorial History of HHT 1998: The Empirical Mode Decomposition Method and the Hilbert Spectrum for Non-stationary Time Series Analysis, Proc. Roy. Soc. London, A454, 903-995. The invention of the basic method of EMD, and Hilbert transform for determining the Instantaneous Frequency and energy. 1999: A New View of Nonlinear Water Waves – The Hilbert Spectrum, Ann. Rev. Fluid Mech. 31, 417-457. Introduction of the intermittence in decomposition. 2003: A confidence Limit for the Empirical mode decomposition and the Hilbert spectral analysis, Proc. of Roy. Soc. London, A459, 2317-2345. Establishment of a confidence limit without the ergodic assumption. 2004: A Study of the Characteristics of White Noise Using the Empirical Mode Decomposition Method, Proc. Roy. Soc. London, A460, 1179-1611. Defined statistical significance and predictability. Recent Developments in HHT 2007: On the trend, detrending, and variability of nonlinear and nonstationary time series. Proc. Natl. Acad. Sci., 104, 14,889-14,894. The correct adaptive trend determination method 2009: On Ensemble Empirical Mode Decomposition. Advances in Adaptive Data Analysis. Advances in Adaptive data Analysis, 1, 1-41 2009: On instantaneous Frequency. Advances in Adaptive Data Analysis , 2, 177-229. 2010: Multi-Dimensional Ensemble Empirical Mode Decomposition. Advances in Adaptive Data Analysis 3, 339372. 2011: Degree of Nonlinearity. Patent and Paper