What is Frequency?
Norden E. Huang
Research Center for Adaptive Data Analysis
Center for Dynamical Biomarkers and translational Medicine
National Central University
Zhongli, Taiwan, ROC
The 15th Wu Chien Shiung Science Camp, 2012
Definition of Frequency
Given the period of a wave as T ; the frequency is
defined as
 
1
T
.
T
This definition is easy for regular sine wave,
but not very practical for complicated
oscillations.
Impossible for complicated waves
Need Decomposition
Jean-Baptiste-Joseph Fourier
Fourier’s work is a great mathematical poem.
Lord Kelvin
1807 “On the Propagation of Heat in Solid Bodies”
1812 Grand Prize of Paris Institute
1817
Elected to Académie des Sciences
1822
Appointed as Secretary of Math Section
paper published
Ever since Fourier’s ground breaking
work, people always think of any
changes in terms of waves.
Fourier’s work is a great mathematical poem.
Lord Kelvin
Definitions of Fourier Type Frequency :
For any data from linear Processes
1. Fourier Analysis :
T
F (  )    x( t ) e
j
i jt
dt .
0
2. Wavelet Analysis
3. Wigner  Ville Analysis
Fourier Analysis
Fourier proved that the Fourier expansion in
terms of trigonometric function
x( t )   a j e
i j t
j
is complete, convergent, orthogonal and
unique. Therefore, every signal could be
think as combination of sinusoidal waves
each with constant amplitude and frequency.
Are those frequency meaningful?
Definition of Power Spectral Density
Since a signal with nonzero average power is not
square integrable, the Fourier transforms do not
exist in this case.
Fortunately, the Wiener-Khinchin Theroem
provides a simple alternative. The PSD is the
Fourier transform of the auto-correlation function,
R(τ), of the signal if the signal is treated as a widesense stationary random process:
S(  ) 



R(  )e 2 i d




S(  )d    2 ( t )
Fourier Spectrum
Surrogate Signal
I. Hello
The original data : Hello
The surrogate data : Hello
The Fourier Spectra : Hello
Surrogate Signal
II. delta function and white noise
Non-causality:
Event involves both past and future
Random and Delta Functions
Fourier Components : Random Function
Fourier Components : Delta Function
The Importance of Phase
Can we just use phase to
explore physical processes?
Yes, to some extent.
After more than 15 years of searching, I found
the key to nonlinear and nonstationary data
analysis is the proper definition for frequency.
Then:
In search of frequency, I found the methods to
quantify nonlinearity and determine the trend.
Instantaneous Frequencies and Trends
for Nonstationary Nonlinear Data
Hot Topic Conference
University of Minnesota,
Institute for Mathematics and Its Applications,
2011
How to define frequency?
It seems to be trivial.
But frequency is an important parameter for
us to understand many physical phenomena.
Traditional Definition of Frequency
• frequency = 1/period.
•
•
•
•
•
Definition too crude
Only work for simple sinusoidal waves
Does not apply to nonstationary processes
Does not work for nonlinear processes
Does not satisfy the need for wave equations
Instantaneous Frequency
Velocity 
distance
; mean velocity
time
Newton  v 
Frequency 
dx
dt
1
; mean frequency
period
HHT defines the phase function   
d
dt
So that both v and  can appear in differential equations.
Prevailing Views on
Instantaneous Frequency
The term, Instantaneous Frequency, should be banished
forever from the dictionary of the communication engineer.
J. Shekel, 1953
The uncertainty principle makes the concept of an
Instantaneous Frequency impossible.
K. Gröchennig, 2001
The Idea and the need of Instantaneous
Frequency
According to the classic wave theory, the wave
conservation law is based on a gradually changing φ(x,t)
such that
k   ,

  
;
t
k

    0 .
t
Therefore, both wave number and frequency must have
instantaneous values and differentiable.
This is the true definition of frequency.
Hilbert Transform : Definition
For any x( t )  Lp ,
y( t ) 
1



x(  )
d ,
t 
then, x( t )and y( t ) form the analytic pairs:
z( t )  x( t )  i y( t )  a( t ) e i  ( t ) ,
where
a( t )   x 2  y 2  1 / 2 and  ( t )  tan 1
y( t )
.
x( t )
The Traditional use of the Hilbert
Transform for Data Analysis
failed miserably
and gave IF a bad break.
Traditional View
a la Hahn (1995) : Data LOD
Traditional View
a la Hahn (1995) : Hilbert
Traditional Approach
a la Hahn (1995) : Phase Angle
Traditional Approach
a la Hahn (1995) : Phase Angle Details
Traditional Approach
a la Hahn (1995) : Frequency
Why the traditional approach
does not work?
Hilbert Transform a cos  + b :
Data
Hilbert Transform a cos  + b :
Phase Diagram
Hilbert Transform a cos  + b :
Phase Angle Details
Hilbert Transform a cos  + b :
Frequency
The Empirical Mode Decomposition
Method and Hilbert Spectral Analysis
Sifting
Empirical Mode Decomposition:
Methodology : Test Data
Empirical Mode Decomposition:
Methodology : data and m1
Empirical Mode Decomposition:
Methodology : data & h1
Empirical Mode Decomposition:
Methodology : h1 & m2
Empirical Mode Decomposition:
Methodology : h3 & m4
Empirical Mode Decomposition:
Methodology : h4 & m5
Empirical Mode Decomposition
Sifting : to get one IMF component
x( t )  m 1  h1 ,
h1  m 2  h2 ,
.....
.....
hk  1  m k  hk .

hk  c1
.
The Stoppage Criteria
The Cauchy type criterion: when SD is small than a preset value, where
T
SD 
h
t 0
k 1
( t )  hk ( t )
2
T
2
h
 k 1 ( t )
t 0
Or, simply pre-determine the number of iterations.
Empirical Mode Decomposition:
Methodology : IMF c1
Definition of the Intrinsic Mode Function
(IMF): a necessary condition only!
Any function having the same numbers of
zero  cros sin gs and extrema ,and also having
symmetric envelopes defined by local max ima
and min ima respectively is defined as an
Intrinsic Mode Function ( IMF ).
All IMF enjoys good Hilbert Transform :
  c( t )  a( t )e i  ( t )
Empirical Mode Decomposition:
Methodology : data, r1 and m1
Empirical Mode Decomposition
Sifting : to get all the IMF components
x( t )  c1  r1 ,
r1  c2  r2 ,
. . .
rn  1  cn  rn .
 x( t ) 
n
c
j 1
j
 rn .
An Example of Sifting
&
Time-Frequency Analysis
Length Of Day Data
LOD
:
IMF
Orthogonality Check
•
Pair-wise %
•
Overall %
•
•
•
•
•
•
•
•
•
•
•
0.0003
0.0001
0.0215
0.0117
0.0022
0.0031
0.0026
0.0083
0.0042
0.0369
0.0400
•
0.0452
LOD : Data & c12
LOD
: Data & Sum c11-12
LOD : Data & sum c10-12
LOD : Data & c9 - 12
LOD : Data & c8 - 12
LOD
: Detailed Data and Sum c8-c12
LOD : Data & c7 - 12
LOD
: Detail Data and Sum IMF c7-c12
LOD
: Difference Data – sum all IMFs
Traditional View
a la Hahn (1995) : Hilbert
Mean Annual Cycle & Envelope: 9 CEI
Cases
Uniqueness
It should be pointed out that in general
x( t )   a( t ) e
i j ( t )
j
is not unique. But the EMD determines the envelope
based on the data to make the expansion unique.
Phase Function computation no longer based on
Hilbert Transform, but through quadrature to
account for the full nonlinear wave form distortion
and other complications.
Properties of EMD Basis
The Adaptive Basis based on and derived from
the data by the empirical method satisfy nearly
all the traditional requirements for basis
empirically and a posteriori:
Complete
Convergent
Orthogonal
Unique
The combination of Hilbert Spectral Analysis and
Empirical Mode Decomposition has been
designated by NASA as
HHT
(HHT vs. FFT)
Comparison between FFT and HHT
1. FFT :
x( t )  
aj e
i jt
.
j
2. HHT :
x( t )    a j ( t ) e
j
i
  j (
t
)d
.
Comparisons:
Fourier, Hilbert & Wavelet
Speech Analysis
Hello : Data
Four comparsions D
The original data : Hello
Additionally
To quantify nonlinearity we also need
instantaneous frequency.
How to define Nonlinearity?
How to quantify it through data alone?
The term, ‘Nonlinearity,’ has been
loosely used, most of the time, simply
as a fig leaf to cover our ignorance.
Can we be more precise?
How is nonlinearity defined?
Based on Linear Algebra: nonlinearity is defined based
on input vs. output.
But in reality, such an approach is not practical: natural
system are not clearly defined; inputs and out puts are
hard to ascertain and quantify. Furthermore, without
the governing equations, the order of nonlinearity is not
known.
In the autonomous systems the results could depend
on initial conditions rather than the magnitude of the
‘inputs.’
The small parameter criteria could be misleading:
sometimes, the smaller the parameter, the more
nonlinear.
Linear Systems
Linear systems satisfy the properties of superposition
and scaling. Given two valid inputs to a system H,
x1 ( t ) and x2 ( t )
as well as their respective outputs
y1 ( t )  H{ x1 ( t )} and
y2 (t) = H{ x2 ( t )}
then a linear system, H, must satisfy
 y1 ( t )   y1 ( t )  H{  x1 ( t )   x2 ( t )}
for any scalar values α and β.
How is nonlinearity defined?
Based on Linear Algebra: nonlinearity is defined based
on input vs. output.
But in reality, such an approach is not practical: natural
system are not clearly defined; inputs and out puts are
hard to ascertain and quantify. Furthermore, without
the governing equations, the order of nonlinearity is not
known.
In the autonomous systems the results could depend
on initial conditions rather than the magnitude of the
‘inputs.’
The small parameter criteria could be misleading:
sometimes, the smaller the parameter, the more
nonlinear.
How should nonlinearity be
defined?
The alternative is to define nonlinearity based
on data characteristics: Intra-wave frequency
modulation.
Intra-wave frequency modulation is known as
the harmonic distortion of the wave forms. But
it could be better measured through the
deviation of the instantaneous frequency from
the mean frequency (based on the zero
crossing period).
Characteristics of Data from
Nonlinear Processes
d2 x
3

x


x
  cos  t
2
dt

d2 x
 x
2
dt
1 x 
2
  cos  t
 Spring with position dependent cons tan t ,
int ra  wave frequency mod ulation;
therefore , we need ins tan tan eous frequency .
Duffing Pendulum
x
d2x
2

x
(
1


x
)   cos  t .
2
dt
Duffing Equation : Data
Hilbert’s View on Nonlinear Data
Intra-wave Frequency Modulation
A simple mathematical model
x( t )  cos  t   sin 2 t 
Duffing Type Wave
Data: x = cos(wt+0.3 sin2wt)
Duffing Type Wave
Perturbation Expansion
For   1 , we can have
x( t )  cos   t   sin 2 t 
 cos  t cos   sin 2 t   sin  t sin   sin 2 t 
 cos  t   sin  t sin 2 t  ....



  1   cos  t  cos 3 t  ....
2
2

This is very similar to the solution of Duffing equation .
Duffing Type Wave
Wavelet Spectrum
Duffing Type Wave
Hilbert Spectrum
Duffing Type Wave
Marginal Spectra
Quantify nonlinearity
Degree of nonlinearity
Let us consider a generalized intra-wave frequency modulation model
as:
x( t )  cos(  t   sin t )  IF=
d
   1   cos  t 
dt
 IF  IFz 
DN (Degree of Nolinearity ) should be  

I
F
z


2
1/ 2


.
2
Depending on the value of  , we can have either a up-down symmetric
or a asymmetric wave form.
Degree of Nonlinearity
• DN is determined by the combination of δη precisely with
Hilbert Spectral Analysis. Either of them equals zero
means linearity.
• We can determine δ and η separately:
– η can be determined from the instantaneous
frequency modulations relative to the mean frequency.
– δ can be determined from DN with known η.
NB: from any IMF, the value of δη cannot be greater
than 1.
• The combination of δ and η gives us not only the Degree of
Nonlinearity, but also some indications of the basic
properties of the controlling Differential Equation, the Order
of Nonlinearity.
Stokes Models
d2x
2
2
 x   x   cos  t with  
;  =0.1.
2
dt
25
Stokes I:  is positive ranging from 0.1 to 0.375;
beyond 0.375, there is no solution.
Stokes II:  is negative ranging from 0.1 to 0.391;
beyond 0.391, there is no solution.
Data and IFs : C1
Stokes Model c1: e=0.375; DN=0.2959
2
1.5
1
Frequency : Hz
0.5
0
-0.5
-1
-1.5
IFq
IFz
Data
IFq-IFz
-2
-2.5
-3
0
20
40
60
T ime: Second
80
100
Duffing Models
d2x
2
3
 ax   x   cos  t with  
;  =0.1.
2
dt
25
Duffing I : a=1;  is positive ranging from 0.1 to any number;
there is always solution.
Duffing II : a=1;  is negative ranging from 0.1 to 0.230;
beyond 0.230 there is no solution.
Duffing O : a=-1;  is positive ranging from 0.1 to any number;
there is no solution, but the system would become chaotic.
Data and Ifs Details
Duffing I : e=0.500
3
2.5
2
Frequency : Hz
1.5
1
0.5
0
-0.5
-1
IFq
IFz
Data
IFq-IFz
-1.5
-2
60
65
70
75
80
Time: Second
85
90
95
100
Lorenz Model
• Lorenz is highly nonlinear; it is the model
equation that initiated chaotic studies.
• Again it has three parameters. We decided to fix
two and varying only one.
• There is no small perturbation parameter.
• We will present the results for ρ=28, the classic
chaotic case.
Phase Diagram for ro=28
Lorenz Phase : ro=28, sig=10, b=8/3
30
20
z
10
0
-10
-20
-30
20
10
50
40
0
30
20
-10
y
10
-20
0
x
X-Component
DN1=0.5147
CDN=0.5027
Data and IF
Lorenz X : ro=28, sig=10, b=8/3
50
IFq
IFz
Data
IFq-IFz
Frequency : Hz
40
30
20
10
0
-10
0
5
10
15
T ime : second
20
25
30
Lorenz Model
dx
   y  x
dt
dy
 x    z  y
dt
dz
 xy   z
dt
with  (Prandtl number)=10;  =8/3;
 (Rayleigh number) varying
Comparisons
Fourier
Wavelet
Hilbert
Basis
a priori
a priori
Adaptive
Frequency
Integral transform:
Global
Integral transform:
Regional
Differentiation:
Local
Presentation
Energy-frequency
Energy-timefrequency
Energy-timefrequency
Nonlinear
no
no
yes, quantifying
Non-stationary
no
yes
Yes, quantifying
Uncertainty
yes
yes
no
Harmonics
yes
yes
no
How to define Trend ?
Parametric or Non-parametric?
Intrinsic vs. extrinsic approach?
The State-of-the arts: Trend
“One economist’s trend is another
economist’s cycle”
Watson : Engle, R. F. and Granger, C. W. J. 1991 Long-run Economic
Relationships. Cambridge University Press.
Philosophical Problem Anticipated
名不正則言不順
言不順則事不成
——孔夫子
Definition of the Trend
Proceeding Royal Society of London, 1998
Proceedings of National Academy of Science, 2007
Within the given data span, the trend is an intrinsically
fitted monotonic function, or a function in which there
can be at most one extremum.
The trend should be an intrinsic and local property of the data; it is
determined by the same mechanisms that generate the data.
Being local, it has to associate with a local length scale, and be valid only
within that length span, and be part of a full wave length.
The method determining the trend should be intrinsic. Being intrinsic,
the method for defining the trend has to be adaptive.
All traditional trend determination methods are extrinsic.
Algorithm for Trend
• Trend should be defined neither
parametrically nor non-parametrically.
• It should be the residual obtained by
removing cycles of all time scales from the
data intrinsically.
• Through EMD.
Global Temperature Anomaly
Annual Data from 1856 to 2003
GSTA
IMF Mean of 10 Sifts : CC(1000, I)
Statistical Significance Test
IPCC Global Mean Temperature Trend
Comparison between non-linear rate with multi-rate of IPCC
Various Rates of Global Warming
0.025
0.02
Warming Rate: oC/Yr
0.015
0.01
0.005
0
-0.005
-0.01
-0.015
-0.02
1840
1860
1880
1900
1920
1940
Time : Year
1960
1980
2000
2020
Blue shadow and blue line are the warming rate of non-linear trend.
Magenta shadow and magenta line are the rate of combination of non-linear trend and AMO-like components.
Dashed lines are IPCC rates.
GSTA Summary
Summary GSTA DN Results : 5=CDN
0.5
SGQ
SGH
gq
gh
0.45
Degree of Nonlinearity
0.4
0.35
0.3
0.25
0.2
0.15
0.1
1
1.5
2
2.5
3
3.5
IMF Number : 5=CDN
4
4.5
5
Extend to N-dimension
Wave number in spatial data
I. Hurricane
The numerically simulated vertical velocity at 500 hPa level of Hurricane Rita of 2005. The warm color represents the upward
motion and the cold color downward motion. The unit is m/s.
The EEMD components of
the vertical velocity.
In each panel, the color
scales are different, the blue
lines corresponding to zeros.
The final components of hurricane Andrew.
Comparisons
Fourier
Wavelet
Hilbert
Basis
a priori
a priori
Adaptive
Frequency
Integral-Transform:
Global
Integral-Transform
Regional
Differentiation:
Local
Presentation
Energy-frequency
Energy-timefrequency
Energy-timefrequency
Nonlinear
no
no
Yes: quantify
Non-stationary
no
yes
Yes: quantify
Uncertainty
yes
yes
no
Harmonics
yes
yes
no
Scientific Activities
Collecting, analyzing, synthesizing, and
theorizing are the core of scientific activities.
Theory without data to prove is just hypothesis.
Therefore, data analysis is a key link in this
continuous loop.
Data Processing and Data Analysis
• Processing [proces < L. Processus < pp of Procedere =
Proceed: pro- forward + cedere, to go] : A particular
method of doing something.
– Data Processing >>>>
Mathematically meaningful parameters
• Analysis [Gr. ana, up, throughout + lysis, a loosing] : A
separating of any whole into its parts, especially with an
examination of the parts to find out their nature,
proportion, function, interrelationship etc.
– Data Analysis >>>>
Physical understandings
The Job of a Scientist
The job of a scientist is to listen carefully to
nature, not to tell nature how to behave.
Richard Feynman
To listen is to use adaptive methods and let the data sing, and
not to force the data to fit preconceived modes.
Thanks
Current Efforts and Applications
• Non-destructive Evaluation for Structural Health Monitoring
– (DOT, NSWC, DFRC/NASA, KSC/NASA Shuttle, THSR)
• Vibration, speech, and acoustic signal analyses
– (FBI, and DARPA)
• Earthquake Engineering
– (DOT)
• Bio-medical applications: dynamical biomarkers
– (Harvard, Johns Hopkins, NTU, VHT, …)
• Climate changes
– (NASA Goddard, NOAA, CCSP)
• Cosmological Gravity Wave
– (NASA Goddard)
• Financial market data analysis
– (NCU, AS)
• Theoretical foundations
– (Princeton University and Caltech)
Conclusions
• Yes, we can still think of any change in terms of waves,
but no the simple harmonic kind.
• With EMD, we can define the true frequency, quantify
nonlinearity and extract trend from any data.
• Among other applications, the degree of nonlinearity
could be used to set an objective criterion in health
monitoring and in natural phenomena; the trend could be
used in financial as well as natural sciences.
• These are all possible because of adaptive data analysis
method.
Data
Fourier Spectra : Stations #3,4,5,& 6
Phase Angle
Phase Angle Reference to Station # 1
Wave Fusion Station #5
The most precise way is
utilizing the phase to define
Instantaneous Frequency
Hilbert Spectrum : Station #1