Quantification of
Nonlinearity and Nonstionarity
Norden E. Huang
With collaboration of
Zhaohua Wu; Men-Tzung Lo; Wan-Hsin Hsieh;
Chung-Kang Peng; Xianyao Chen; Erdost Torun; K. K. Tung
IPAM, January 2013
The term, ‘Nonlinearity,’ has been
loosely used, most of the time, simply
as a fig leaf to cover our ignorance.
Can we measure it?
How is nonlinearity defined?
Based on Linear Algebra: nonlinearity is defined based
on input vs. output.
But in reality, such an approach is not practical: natural
system are not clearly defined; inputs and out puts are
hard to ascertain and quantify.
Nonlinear system is not always so compliant: in the
autonomous systems the results could depend on initial
conditions rather than the magnitude of the ‘inputs.’
There might not be that forthcoming small perturbation
parameter to guide us. Furthermore, the small
parameter criteria could be totally wrong: small
parameter is more nonlinear.
Linear Systems
Linear systems satisfy the properties of superposition
and scaling. Given two valid inputs
x 1 ( t ) and x 2 ( t )
as well as their respective outputs
y 1 ( t )  H { x 1 ( t )} and
y 2 (t) = H { x 2 ( t )}
then a linear system must satisfy
 y 1 ( t )   y 1 ( t )  H {  x 1 ( t )   x 2 ( t )}
for any scalar values α and β.
How is nonlinearity defined?
Based on Linear Algebra: nonlinearity is defined based
on input vs. output.
But in reality, such an approach is not practical: natural
system are not clearly defined; inputs and out puts are
hard to ascertain and quantify.
Nonlinear system is not always so compliant: in the
autonomous systems the results could depend on initial
conditions rather than the magnitude of the ‘inputs.’
There might not be that forthcoming small perturbation
parameter to guide us. Furthermore, the small
parameter criteria could be totally wrong: small
parameter is more nonlinear.
Nonlinearity Tests
• Based on input and outputs and probability distribution:
qualitative and incomplete (Bendat, 1990)
• Higher order spectral analysis, same as probability
distribution: qualitative and incomplete
• Nonparametric and parametric: Based on hypothesis
that the data from linear processes should have near
linear residue from a properly defined linear model
(ARMA, …), or based on specific model: Qualitative
How should nonlinearity be
defined?
The alternative is to define nonlinearity
based on data characteristics: Intra-wave
frequency modulation.
Intra-wave frequency modulation is the
deviation of the instantaneous frequency
from the mean frequency (based on the
zero crossing period).
Characteristics of Data from
Nonlinear Processes
d
2
dt

x
2
 x   x
d
2
dt

x
2
 x

3
  co s  t
1   x
2

  co s  t
S p rin g w ith p o sitio n d ep en d en t co n sta n t, w ill h a ve
in tra -w a ve freq u en cy m o d u l a t io n ;
th erefo re, w e n eed i n st a n ta n eo u s freq u en cy.
Nonlinear Pendulum : Asymmetric
2
d x
dt
2
 x ( 1   x )   cos  t .
3
Nonlinear Pendulum : Symmetric
2
d x
dt
2
 x (1   x
2
)   co s  t .
Duffing Equation : Data
Hilbert’s View on
Nonlinear Data
A simple mathematical model
x ( t )  cos   t   sin 2  t 
Duffing Type Wave
Data: x = cos(wt+0.3 sin2wt)
Duffing Type Wave
Perturbation Expansion
F or    1 , w e can h ave
x ( t )  cos   t   sin 2  t 
 cos  t cos   sin 2  t   sin  t sin   sin 2  t 
 cos  t   sin  t sin 2  t  ....
 


  1   cos  t 
cos 3  t  ....
2
2

T h is is very sim ilar to th e solu tion of D u ffin g equ atio n .
Duffing Type Wave
Wavelet Spectrum
Duffing Type Wave
Hilbert Spectrum
Duffing Type Wave
Marginal Spectra
The advantages of using HHT
• In Fourier representation based on linear and
stationary assumptions; intra-wave modulations
result in harmonic distortions with phase locked
non-physical harmonics residing in the higher
frequency ranges, where noise usually
dominates.
• In HHT representation based on instantaneous
frequency; intra-wave modulations result in the
broadening of fundamental frequency peak,
where signal strength is the strongest.
Define the degree of
nonlinearity
Based on HHT for intra-wave
frequency modulation
Characteristics of Data from
Nonlinear Processes
d
2
dt

x
2
 x   x
d
2
dt

x
2
 x

1 
  co s  t
1   x


  co s  t
S p rin g w ith p o sitio n d ep en d en t co n sta n t, w il l h a ve
in tra -w a ve freq u en cy m o d u la tio n ;
fo r even  , w e h a ve sym m etric w a ve fo rm ; fo r o d d  ,
w e h a ve a sym m etric w a v e fo rm .
Degree of nonlinearity
L et us consider a generalized intra-w ave frequency m odulation m odel
as:
x ( t )  cos(  t   sin   t ) 
IF =
d
dt
D N (D egree of N olinearity ) should be 
   1    cos   t 
 IF  IF z 


I
F
z


2
1/ 2


.
2
D epending on t he value of  , w e can have either an up-d ow n sym m etric
or an asym m etric w ave form .
T he relationship betw een  and  is com plicated except for infinitesim al  .
The influence of amplitude variations
Single component
To consider the local amplitude variations, the
definition of DN should also include the amplitude
information; therefore the definition for a single
component should be:
D N (D egree of N olin earity )  std
 IF  IF z 
az

 
IF z
az


The influence of amplitude variations
for signals with multiple components
To consider the case of signals with multiple components,
we should assign weight to each individual component
according to a normalized scheme:
x(t)=  c j ( t ) 
j
x (t)=  c j ( t )
2
2
j

n 
 C D N (C om bin ed D egree of N olin earity )    D N
j1 


j

2
cj 
 .
n
2 
c
 k
k1

Degree of Nonlinearity
• We can determine DN precisely with Hilbert
Spectral Analysis.
• We can also determine δ and η separately.
• η can be determined from the instantaneous
frequency modulations relative to the mean
frequency.
• δ can be determined from DN with η determined.
NB: from any IMF, the value of ηδ cannot be
greater than 1.
• The combination of δ and η gives us not only the
Degree of Nonlinearity, but also some indications of
the basic properties of the controlling Differential
Equation.
Calibration of the Degree of
Nonlinearity
Using various Nonlinear systems
Stokes Models
2
d x
dt
2
 x   x   cos  t w ith  
2
2
;  = 0.1.
25
S tokes I :  is positive ran gin g from 0.1 to 0.37 5;
beyon d 0.375, th ere is n o solu tion .
S tokes II :  is n egative ran gin g from 0.1 to 0.39 1 ;
beyon d 0.391, th ere is n o so lu tion .
Stokes I
  0 .3 7 5
Phase Diagram
Phase Diagram : Stokes Model I, e=0.375
1.5
1
0.5
Wave Elevation
0
-0.5
-1
-1.5
-2
-2.5
-3
-2
-1.5
-1
-0.5
0
0.5
Particle Velocity
1
1.5
2
IMFs
Stokes Model : IMFs
0
4000
8000
12000
0
500
1000
1500
2000
2500
Time: Second
3000
3500
4000
Data and IFs : C1
Stokes Model c1: e=0.375; DN=0.2959
2
1.5
1
Frequency : Hz
0.5
0
-0.5
-1
-1.5
-2
IFq
IFz
Data
IFq-IFz
-2.5
-3
0
20
40
60
80
100
120
Time: Second
140
160
180
200
Data and IFs : C2
Stokes Model c2: e=0.375; DN=0.1528
1.5
1
Frequency : Hz
0.5
0
-0.5
-1
-1.5
IFq
IFz
Data
IFq-IFz
-2
-2.5
0
20
40
60
80
100
120
Time: Second
140
160
180
200
Stokes II
  0 .3 9 1
Phase Diagram
Phase Diagram : Stokes Model II, e=0.391
2.5
2
Wave Elevation
1.5
1
0.5
0
-0.5
-1
-1.5
-1.5
-1
-0.5
0
Particle Velocity
0.5
1
1.5
Data and Ifs : C1
Stokes II : e=0.391
2.5
IFq
IFz
Data
IFq-IFz
2
Frequency : Hz
1.5
1
0.5
0
-0.5
-1
-1.5
0
20
40
60
80
100
120
Time: Second
140
160
180
200
Data and Ifs : C1 details
Stokes II : e=0.391
3
IFq
IFz
Data
IFq-IFz
2.5
2
Frequency : Hz
1.5
1
0.5
0
-0.5
-1
-1.5
-2
60
65
70
75
80
Time: Second
85
90
95
100
Data and Ifs : C2
Stokes II C2: e=0.391
2.5
IFq
IFz
Data
IFq-IFz
2
Frequency : Hz
1.5
1
0.5
0
-0.5
-1
-1.5
0
20
40
60
80
100
120
Time: Second
140
160
180
200
Combined Stokes I and II
Summary Stokes I and Stokes II
0.35
Combined Degree of Nonlinearity
0.3
0.25
s1c1
s1c2
S1CDN
s2c1
s2c2
S2CDN
0.2
0.15
0.1
0.05
0
0.1
0.15
0.2
0.25
0.3
Epsilon
0.35
0.4
0.45
Water Waves
Real Stokes waves
Comparison : Station #1
Data and IF : Station #1
DN=0.1607
Duffing Models
2
d x
dt
2
 a x   x   cos  t w ith  
3
2
;  = 0.1.
25
D u ffin g I : a= 1;  is positive ran gin g from 0.1 to an y n u m ber;
th ere is alw ays solu tion .
D u ffin g II : a= 1;  is n egativ e ran gin g from 0.1 to 0 .230;
beyon d 0.230 th ere is n o solu tion .
D u ffin g O : a= -1;  is positi ve ran gin g from 0.1 to an y n u m ber;
th ere is n o solu tion , b u t th e system w ou ld bec om e ch ao tic.
Duffing I
  0 .5 0 0
Phase Diagram
Phase Diagram : Duffing Model I, e=0.500
2
1.5
Wave Elevation
1
0.5
0
-0.5
-1
-1.5
-2
-2
-1
0
Particle Velocity
1
2
IMFs
IMF Duffing I, e=0.500
0
4000
8000
12000
16000
20000
0
500
1000
1500
2000
2500
Time: Data Point
3000
3500
4000
Data and IFs
Duffing I : e=0.500
2.5
IFq
IFz
Data
IFq-IFz
2
Frequency : Hz
1.5
1
0.5
0
-0.5
-1
-1.5
0
20
40
60
80
100
120
Time: Second
140
160
180
200
Data and Ifs Details
Duffing I : e=0.500
3
2.5
2
Frequency : Hz
1.5
1
0.5
0
-0.5
-1
IFq
IFz
Data
IFq-IFz
-1.5
-2
60
65
70
75
80
Time: Second
85
90
95
100
Summary Duffing I
Combined Degree of Stationarity
10
Duffing I Summary
1
d1c1
d1c2
d1c3
d1c4
D1CDN
10
10
10
0
-1
-2
-1
10
0
10
10
Epsilon
1
10
2
Duffing II
  0 .2 0 0
Summary Duffing II
Duffing II : e=0.200
2
IFq
IFz
Data
IFq-IFz
1.5
Frequency : Hz
1
0.5
0
-0.5
-1
-1.5
-2
0
20
40
60
80
100
120
Time: Second
140
160
180
200
Summary Duffing II
Summary Duffing Model II
Combined Degree of Stationarity
d2c1
d2c2
CDN
10
-1
-2
10 -1
10
Pertuebation Parameters : Epsilon
Duffing O : Original
  1 .0 0
Data and IFs
Duffing Model O : e=1.000
2
1.5
Frequency : Hz
1
0.5
0
-0.5
-1
IFq
IFz
Data
IFq-IFz
-1.5
-2
0
10
20
30
40
50
60
Time: Second
70
80
90
100
Data and Ifs : Details
Duffing Model O : e=1.000
3
IFq
IFz
Data
IFq-IFz
2.5
2
Frequency : Hz
1.5
1
0.5
0
-0.5
-1
-1.5
-2
60
65
70
75
80
Time: Second
85
90
95
100
Phase Diagram
Duffing Model O Phase : e=1.000
2
1.5
Wave Elevation
1
0.5
0
-0.5
-1
-1.5
-2
-1.5
-1
-0.5
0
Particle Velocity
0.5
1
1.5
IMFs
Duffing Model O IMFs : e=1.000
0
4000
8000
12000
16000
0
500
1000
1500
2000
2500
Time: Data Point
3000
3500
4000
Duffing 0 : Original
  0 .5 0
Phase : e=0.50
Duffing Model O Phase : e=0.500
2.5
2
1.5
Wave Elevation
1
0.5
0
-0.5
-1
-1.5
-2
-2.5
-1.5
-1
-0.5
0
Particle Velocity
0.5
1
1.5
IMF e=0.50
Duffing Model O IMF : e=0.500
0
4000
8000
12000
16000
0
500
1000
1500
2000
2500
Time: Data Point
3000
3500
4000
Data and Ifs : e=0.50
Duffing Model O : e=0.500
5
IFq
IFz
Data
IFq-IFz
4
Frequency : Hz
3
2
1
0
-1
-2
-3
0
20
40
60
80
100
120
Time: Second
140
160
180
200
Data and Ifs : details e=0.50
Duffing Model O : e=0.500
5
IFq
IFz
Data
IFq-IFz
4
Frequency : Hz
3
2
1
0
-1
-2
-3
60
65
70
75
80
Time: Second
85
90
95
100
Summary : Epsilon
Combined Degree of Stationarity
10
Duffing 0 Summary
0
d0c1
d0c2
d0c3
d0c4
D0CDN
10
10
-1
-2
-1
10
0
10
Epsilon
10
1
Summary All Duffing Models
10
2
d x
dt
Summary All Duffing Models
0
2
 x  x
3
  cos  t
 x  x
3
  co s  t
D0
D1
D2
2
Combined Degree of Stationarity
d x
dt
2
-1
10
2
d x
dt
10
2
 x  x
3
  co s  t
-2
-3
10 -2
10
10
-1
0
10
10
Pertuebation Parameters : Abs Epsilon
1
10
2
Lorenz Model
dx
dt
dy
dt
dz

 y  x
 x   z  y
 xy   z
dt
w ith  (P ra n d tl n u m b er)= 1 0 ;  = 8 /3 ;
 (R a yleig h n u m b er) va ryin g
Lorenz Model
• Lorenz is highly nonlinear; it is the model
equation that initiated chaotic studies.
• Again it has three parameters. We
decided to fix two and varying only one.
• There is no small perturbation parameter.
• We will present the results for ρ=28, the
classic chaotic case.
Phase Diagram for ro=28
Lorenz Phase : ro=28, sig=10, b=8/3
30
20
z
10
0
-10
-20
-30
20
10
50
40
0
30
20
-10
y
10
-20
0
x
X-Component
DN1=0.5147
CDN=0.5027
Data and IF
Lorenz X : ro=28, sig=10, b=8/3
50
IFq
IFz
Data
IFq-IFz
40
Frequency : Hz
30
20
10
0
-10
0
5
10
15
20
25
30
Time : second
35
40
45
50
Spectra data and IF
10
Spectra of Data and IF : Lorenz 28
2
Data
Frequency
10
Spectral Density
10
10
10
10
10
1
0
-1
-2
-3
-4
-1
10
0
10
10
Frequency : Hz
1
10
2
IMFs
Lorenz x IMF : ro=28, sig=10, b=8/3
0
2000
4000
6000
0
200
400
600
800
1000
1200
Time : second
1400
1600
1800
2000
Hilbert Spectrum
Lorenz x Hilbert : ro=28, sig=10, b=8/3
15
Frequency : Hz
10
5
0
0
1
2
3
4
5
6
Time : second
7
8
9
10
Degree of Nonstationarity
Quantify nonstationarity
Need to define the Degree Stationarity
• Traditionally, stationarity is taken for
granted; it is given; it is an article of faith.
• All the definitions of stationarity are too
restrictive and qualitative.
• Good definition need to be quantitative to
give a Degree of Stationarity
Definition : Strictly Stationary
F or a ran dom var iable x ( t ), if
2
 x( t ) 
,
 x( t )  m ,
an d th at
 x ( t 1 ), x ( t 2 ), ... x ( t n )  an d  x ( t 1   ), x ( t 2   ), ... x ( t n   ) 
h ave th e sam e joi n t distribu tion for all  .
Definition : Wide Sense Stationary
F or an y ran dom var iable x ( t ), if
2
 x( t ) 
,
 x( t )  m ,
an d th at
 x ( t 1 ), x ( t 2 )  an d  x ( t 1   ), x ( t 2   ) 
h ave th e sam e joi n t distribu tion for all  .
T h erefore ,
 x ( t1 ) x ( t2 )   C ( t1  t2 ) .
Definition : Statistically Stationary
• If the stationarity definitions are satisfied
with certain degree of averaging.
• All averaging involves a time scale. The
definition of this time scale is problematic.
Stationarity Tests
• To test stationarity or quantify non-stationarity,
we need a precise time-frequency analysis tool.
• In the past, Wigner-Ville distribution had been
used. But WV is Fourier based, which only
make sense under stationary assumption.
• We will use a more precise time-frequency
representation based on EMD and Hilbert
Spectral Analysis.
Degree of Stationarity
Huang et al (1998)
F o r a tim e  freq u en cy d istrib u tio n , H (  , t ),
n(  ) 
1
T
DS(  ) 
 H (  , t ) dt
;
t
1
T
T

0

H ( ,t )
1 

n(  ) 

2
dt .
Problems
• The instantaneous frequency used here includes
both intra-wave and inter-wave frequency
modulations: mixed nonlinearity with
nonstationarity.
• We have to define frequency here based on
whole wave period, ωz , to get only the interwave modulation.
• We have also to define the degree of nonstationarity in a time dependent way.
Tim-dependent Degree of
non-Stationarity: with a sliding window ΔT
 H (z ,t )  H (z ,t )
D S ( t ,  T )  std 
H (z ,t )

T
 H ( ,t )

z
= std 
 1
 H (  z , t )  T
  T
T


  T
Time-dependent Degree of
Non-linearity
For both nonstationary and
nonlinear processes
Time-dependent degree of nonlinearity
To consider the local frequency and amplitude
variations, the definition of DN should be timedependent as well. All values are defined within a
sliding window ΔT:
 IF ( t )  IF z ( t
D N (t ,  T )  std 

IF ( t )z

)
az
 
az

T
Application to Biomedical case
Heart Rate Variability : AF Patient
Conclusion
• With HHT, we can have a precisely defined
instantaneous frequency; therefore, we can also
define nonlinearity quantitatively.
• Nonlinearity should be a state of a system
dynamically rather than statistically.
• There are many applications for the degree of
nonlinearity in system integrity monitoring in
engineering, biomedical and natural phenomena.
Thanks