Translating Differential Equation Models Into
Kernel Methods for Data Analysis – Phase IV
Emphasis on Simulations
Presented to:
Drs. William Macready and Ashok Srivastava, NASA
CAMCOS team:
Shikha Naik, Afshin Tiraie,
Virginia Banh, Bao Fang,
Efrem Rensi, Lawrence Varela
Supervising faculty:
Drs. Igor Malyshev and Maria Cayco
Department of Mathematics
San Jose State University
May 2005
CAMCOS
Center for Applied Mathematics and Computer Science
SJSU
1
Table of Contents
Summary
Introduction
I.
Analysis
1. Fundamental solutions for the Fokker-Planck type equations with variable
coefficients
2. Generalized Kernel Method for parabolic PDEs.
3. Eigenfunction expansion for the 1-D Fokker-Planck equation.
4. Project background information and resources.
II.
Theoretical aspects of the Statistical Analysis performed in the project
1. Mean Square Error (MSE)
2. PDFs in use
3. Maximum Likelihood Inference.
III.
Simulation concepts, techniques and outcomes.
IV.
The “data generators” and statistical experiments
1. Regular double well Fokker-Planck equation case
2. General double well Fokker-Planck equation case
3. Log-Likelihood and the best fit parametric PDFs
V.
Kernel-based Solutions of the Fokker-Planck family of equations
VI.
Eigenfunction expansion – numerical experiments and analysis
VII.
The “Cloud Evolution” demo
Project extensions
References
2
Summary
This is the fourth and final phase of the NASA/CAMCOS project on Kernel Methods for
Differential Equation Models, initiated by Dr. William Macready at NASA Ames in the
fall of 2003.
This semester we investigated a set of four one-dimensional Fokker-Plank equations with
single and double well potentials and variable diffusion. Since the kernel method we
used required knowledge of the Green’s functions, we investigated simulation techniques
that would generate all necessary supporting functions. The kernel method itself has been
streamlined, and now it can be applied to an arbitrary one-dimensional parabolic problem
with, generally speaking, a non-symmetric and non-positive definite Green’s function or
kernel. The outline of the method and the proof of its convergence are provided in the
report.
Part of the project consisted of studying reconstruction techniques that can be applied to
(hypothetically) incomplete and noisy initial data which is later used in the generation of
the kernel-based approximations. Numerous simulations have been conducted under a
variety of noise and data loss conditions, using several interpolating options. Statistical
data was collected and analyzed, and recommendations were made.
A large number of MATLAB programs have been written (or modified from the previous
projects) to simulate the above mentioned procedures, to investigate the properties of the
approximate kernel solution(s) to the Fokker-Planck equations, and to perform other
tasks. The mathematical treatment, numerical experiments and software developments
are presented in this report.
In addition, a known eigenfunction expansion of the solution of the single well FokkerPlanck equation with constant diffusion has been independently derived and some
numerical investigations have been conducted.
Introduction
The goal of this project is to continue the investigation of the methods of translating
differential equation models into kernel functions. More specifically, the project will
explore simulation techniques for parabolic PDEs, modify and streamline the Kernel
Method in terms of its application to the general Fokker-Planck equations, and
investigate effects of noise and data loss on the construction of the kernel functions that
yield good approximation to the solutions of the given equations. The long-term goals of
the project include the development of techniques suitable for the models described by
more general multidimensional evolution partial differential equations and systems of
such equations.
Kernel methods represent a recent advance in machine learning algorithms, with
applications including the analysis of very large data sets. Central to the kernel method is
3
the kernel function, which is essentially a mathematical model that allows for the analysis
of regression and for density distribution estimates. It is therefore critical to choose a
kernel function that reflects the properties of the physical domain and the corresponding
differential equation models.
The current project serves as a fourth step toward the spectrum of physically motivated
models involving multi-dimensional domains and equations with variable coefficients,
noisy data, etc. These complex models are central to NASA’s long-term goals for this
project.
This semester we conducted an investigation of the kernel method for the general FokkerPlanck equation
w

2
(1)
 LFP w  [ ( D w)  2 ( D (2) w)]
t
x
x
(0.1)
with zero boundary conditions. Here the solution w( x, t ) is a probability density function
of the position (or velocity) of a particle, D (2) is a constant or variable diffusion
coefficient, and D(1) ( x)   f '( x) is a drift coefficient function, where f ( x) is a single
or double well potential of the field of external forces. The solution of the problem is
presumed to be known at a finite (sufficiently large) empirical set of points (sample)
which is given in the form:
{( x1 , y1 ),( x2 , y2 ),....,( xm , ym )}
(0.2)
Our goal is to find an approximation to the solution of (1) in the form of a linear
combination of pde-based kernels (Green’s functions measured at the sample):
m
w( x, t )   c j G( x, x j ; t ) ,
(0.3)
j 1
that fits the data (2), thus utilizing the techniques of the generic kernel method.
We also investigated effects of noisy and incomplete initial data on the construction of
the kernel functions.
We performed the following steps of investigation:

A new approach has been applied to achieve theoretical justification of the Kernel
Method for all cases of the one-dimensional Fokker-Planck equation with
constant and variable diffusion coefficient. We believe that this approach can be
applied to arbitrary parabolic PDE (sec. I-3).
4

Knowledge of the Green’s functions is essential for the Kernel Method
application to PDE models. But, since explicit formulae for the PDEs with
variable coefficients are seldom available, we pursued an additional line of
investigation that consisted of the simulation of the Dirac’s delta used as initial
condition for the parabolic PDE of interest (sections I-4, III).

Several MATLAB programs have been written to conduct a series of numerical
experiments for the method validation and property investigation purposes. These
programs, in different combinations depending on the task to be performed, have
been organized in sets for the user’s convenience (section V, CD).

We considered the possibility of noisy and “defective” data and designed equation
specific “data generators” to implement those effects and perform statistical
experiments and analysis to identify the best interpolation techniques. For the
Gaussian and pde-based kernel approximation cases we used the Kernel Method
mentioned above (section IV).

We investigated some numerical aspects of the eigenfunction expansions of the
solution of the standard one-dimensional Fokker-Planck equation in an infinite
domain (section VI).
I.
Analysis
1.
Fundamental solutions for the Fokker-Planck type equations with
variable coefficients
1.1.
Fokker-Planck equation case
a) Direct transformation
The Fokker-Planck equation
w
2w
w
 D 2  x
 w
t
x
x
(1.1.1)
is a particular case of the equation ([1], p. 93, #2.2):
w
2w
w
 a 2  bx
 (cx  d ) w
t
x
x
(1.1.2)
for the choice of a  D, b  d   , c  0 , where the following substitution of variables
5

D 2 t
D
e ,   [ 0 , ),  0 
,
2
2
z  xe t , z  (, )
(1.1.3)
(1.1.4)
reduces (1) to the heat (diffusion) equation with constant coefficients
u  2u

 z 2
(1.1.5)
e t w( x, t )  u ( z, ) .
(1.1.6)
where
The verification is quite straightforward but useful, since it shows exact relations between
operators that effect some calculations ahead:
w
2w
w
u  2u
 D 2  x
  w  De3 t [  2 ]  0
t
x
x
 z
(1.1.7)
Let us denote the operator on the left side of (7) as LFP , then the fundamental solution of
the Fokker-Planck equation at the initial moment t '  0 corresponding to the
instantaneous delta-source at a fixed point x ' , satisfies:
LFP   ( x  x ') (t )
(1.1.8)
If compared to (7), we obtain
De3 t [
u  2u

]   ( x  x ') (t ) ,
 z 2
(1.1.9)
where x and t need to be expressed in terms of z and  :
x  ze t , e t  (
 1/ 2
1

) , t
ln( ),    0 ,   0
0
2  0
(1.1.10)

u  2u 1  0 3/ 2
1

 2  ( )  ( z ( 0 )1/ 2  x ') ( ln( )),    0
 z
D 

2  0
(1.1.11)
Thus,
where x ' is fixed and is not a subject of the substitution performed above.
6
The fundamental solution of the heat operator (left side of (11)) is
H ( )  4z
K ( z , ) 
e
4
2
(1.1.12)
where H ( ) is the Heaviside function. Formally, the solution of the non-homogeneous
equation (11) can be written as a convolution of K ( z , ) with the term f on the right
([2], w  0 for t  0 and, respectively, u  0 for    0 ):
u ( z , )  f  K 

 


1
1

( 0 )3 / 2  ( 0  x ') ( ln( ))K ( z   ,   )d d


D  0  

2  0
(1.1.13)
Applying the substitution of variables
1


ln( )   ,
 e2 ,    0 e2 , d  2 0 e 2 d 
2  0
0
(1.1.14)
to the integrals in (13) and using the property of the delta-function (  ,     (0) ,
twice), we can simplify (13) as follows:
 
1
u ( z , )    e 3  ( e   x ') (  )2 0e 2 K ( z   ,   0e 2 )d d  
D 0 


1
  [  e  2 0 ( e   x ')K ( z   ,   0e 2 )d ] (  )d  
D 0 


1
 2 0 (  x ') K ( z   ,  0 )d  K ( z  x ',  0 )
D 
(1.1.15)
Thus the solution of (11) has the form
u( z, )  K ( z  x ',   0 ),    0 .
(1.1.16)
Remark. Another technique of obtaining the same formula for u ( z, ) will be given
in the case of a modified Fokker-Planck equation to expand the application base.
b) Inverse transformation.
7
Going back to the original problem and its solution we recall (3), (4) and (6) to obtain
w( x, t )  e t u( z, ) |z  z ( x,t ),  (t )  e t K ( z  x ',  0 ) |z  z ( x,t ),  (t ) .
(1.1.17)
Thus we need to see how the return to ( x, t ) will affect the function K . Since
D(e2 t  1) De2 t (1  e2 t )
  0 

2
2
(1.1.18)
we find
e t K ( z  x ',   0 ) |z  z ( x ,t ),   ( t ) 

 H (t )
2 D(1  e2 t )
exp(




t
2
e H (t )
( xe  x ')

exp  

De 2 t (1  e 2 t ) 
De 2 t (1  e 2 t )
4
 4

2
2


t
 ( x  e t x ') 2
2 D(1  e2 t )
)  P( x, t ; x ', 0)
(1.1.19)
where P( x, t ; x ', 0) is the fundamental solution of (1) ([3], p.100) at t '  0 .
Remark. It suffices to replace t with t  t ' in the formula (19) to reflect the shift of
the initial condition (  ( x  x ') in our case) from 0 moment to t ' . A straightforward
substitution of the time variable in (1) ( t  t '   , w( x, t )  w( x,   t ') v( x,  ) , etc.)
does not affect the equation and the derivation of (19). Therefore, the final formula for
the fundamental solution W ( x, t ; x ', t ') of (1) takes on the form:
W ( x, t ; x ', t ') 
1.2.
H (t  t ') 
2 D(1  e 2 (t t ') )
exp(
 ( x  e (t t ') x ')2
2 D(1  e 2 (t t ') )
)  P( x, t ; x ', t ')
(1.1.20)
Modified Fokker-Planck equation case
The modified FP equation has the form (see [4]):
w
2w  2 2

D 2 
x w w
t
x
4D
2
(1.1.21)
It corresponds to another particular case ([1], p.85) of
w
2w
 a 2  bx 2 w  cw
t
x
(1.1.22)
8
with the obvious choice of coefficients.
A substitution of variables similar to the one in part I
z  xe t ,  
D 2 t
e  A, ( A)
2
(1.1.23)
brings up again the equation (5) where
e
 ( t 

4D
x2 )
w( x, t ) u( z, ) .
(1.1.24)
The arbitrary constant A can be chosen to be 
D
2
which makes  0  0 for t  0
(compare to (3)).
In an effort to find a fundamental solution for (21) we shall apply a technique different
from the one used in part I. Consider the initial condition for (21) in the form
w( x, 0; x ', 0)   ( x  x ')
(1.1.25)
The substitution of (23) into (24) with t  0 implies:
u ( z, 0)  w( x, 0; x ', 0)e
since x  z at t  0 .


4D
x2
e


4D
x2
 ( ze
 t
 x ') |t 0  e


4D
z2
 ( z  x ')
(1.1.26)
Thus we obtain the following initial-value problem:
 2

z
u  2u
4D
 2  0, u ( z, 0)  e
 ( z  x ')
 z
(1.1.27)
And, since the distribution  ( z ) ( z  x ')   ( x ') ( z  x ') , we finally have


( x ')2
u  2u
 2  0, u ( z, 0)  e 4 D  ( z  x ')
 z
(1.1.28)


( x ')2
The fundamental solution of (28) is equal to K ( z  x ', ) times the constant e
,
where K is defined in (12). Therefore, following the reverse substitution of variables
(18)-(19), and using (24), the fundamental solution of (21) can be written in the form:
G ( x, t ; x ', 0)  [e


4D
( x ')2
K ( z  x ', ) |z  z ( x ,t ),  (t ) ] e
( t 

4D
x2 )
4D

9
e


4D
H (t )  e  t
( x ')2
2 D(1  e 2 t )

 e4D
exp(
 H (t )
( x 2  ( x ') 2 )
2 D(1  e 2 t )
 e2 t ( x  e  t x ') 2
2 De 2 t (1  e 2 t )
exp(
 ( x  e t x ') 2
2 D(1  e 2 t )
)e
)
( t 

4D
x2 )

(1.1.29)
Incidentally,
G( x, t; x ',0)  e

4D
( x2 ( x ')2 )
P( x, t; x ',0) ,
(1.1.30)
where P( x, t ; x ', 0) is the fundamental solution of (1) ([3], p.100) at t '  0 . Transition to
the case of t '  0 is identical to part I (see (20)), thus
G( x, t; x ', t ')  e

4D
( x2 ( x ')2 )
P( x, t; x ', t ')
(1.1.31)
Remark. It can be proved that G satisfies the modified FP equation in ( x, t ) and
the adjoint one in ( x ', t ') . It behaves like a delta-function when t  t '  0 . It also
decays to 0 with | x |  .
Remark. The connection between G for the modFP and P for the FP can be
established by other means ([4]), but the point here was to find G independently of the
fact whether those equations were connected or not.
1.3.
General Fokker-Planck equation (special case)
Let us consider equation ([1], p. 99, #6):
w
2w
w
 ( x 2  b) 2   x
 cw, b  0 ,
t
x
x
(1.1.32)
which can be classified as a “special case” of a general FP equation in one space variable.
The substitution of x  x( z ) defined by the one-to-one function
x
z ( x)  
0
d
 2  b
(1.1.33)
and
u ( z, t )
w( x( z ), t )
(1.1.34)
10
changes (32) into an equation with constant coefficients
u  2u

 cu
 z 2
(1.1.35)
whose fundamental solution (see also (12)) is given by
K c ( z , t )  ect K ( z , t )
(1.1.36)
The initial condition for (32) in the form
initial condition for (35) as follows:
w( x, 0)   ( x  x ') implies the corresponding
u ( z, 0)  w( x( z ), 0)   ( x( z )  x ')
(1.1.37)
According to [2], the solution of the problem (35), (37), extended by 0 for t  0 , satisfies
the non-homogeneous equation
u  2u

 cu   ( x( z )  x ') (t )
 z 2
(1.1.38)
and it can be written in the form
 
u ( z , )     ( ) ( x( )  x ')K c ( z   , t   )d d
(1.1.39)
0 
Using the substitution
  z ( ), d  z ' d 
1
 2  b
and the property of the delta-function
d , x( )  x( z ( ))  
  ( ) f (t   )d  f (t )
(1.1.40)
(1.1.41)
transforms (39) into
u ( z , ) 

  (  x ') K ( z  z( ), t )
c


1
 ( x ') 2  b
K c ( z  z ( x '), t )
1
 2  b
d 
(1.1.42)
11
By performing reverse substitution ( z  z ( x) , (33)), we obtain the fundamental solution
for (32) in the form:
H (t  t ')
W ( x, t ; x ', t ') 
 ( x ') 2  b
K c ( z ( x)  z ( x '), t  t ') ,
(1.1.43)
where
x
z ( x)  z ( x ')  
x'
d
 2  b
(1.1.44)
Remarks. 1) It can be verified that (43) satisfies (32) in ( x, t ) and its adjoint in
( x ', t ') . It behaves like delta-function when t  t '  0 , and since z ( x) 
 , it
x 
decays to 0 with | x |  . 2) All MATLAB simulations, including the Kernel Method
for this case, can be found in the “SpecFPdelta” directory on the enclosed CD.
1.4.
Heat equation with variable diffusion coefficient
Let us consider the equation
w
2w
 ax 2 2
t
x
(1.1.45)
with the delta initial condition
w( x, 0)   ( x  x '), x, x '  0
(1.1.46)
The substitution of the space variable in the form x  e z w( x, t )  u ( z ( x), t ) allows
reduction of the problem to the one with constant coefficients:
u
 2u
u
a 2 a
 0, u ( z, 0)   (e z  x ')
t
x
x
(1.1.47)
The fundamental solution of the operator in (47) can be found using Fourier transforms in
the form:
H (t )  ( z4aat )
U ( z, t ) 
e
4 at
2
(1.1.48)
It can be easily verified to satisfy the properties:
12

 U ( z, t )dz  1,

U ( z, t ) 
 ( z) .
t  0
(1.1.49)
Using (48), the solution of the problem (47) can be written in the form:
( z  ln x '  at )

H (t )
4 a
u ( z , t ; x ') 
e
.
x ' 4 at
2
(1.1.50)
By reversing the substitution of variables, the fundamental solution of the problem (45)(46) is:
(ln x  ln x '  at )

H (t )
4 a
E ( x, t ; x ') 
e
, x, x '  0 .
x ' 4 at
2
(1.1.51)
Remark. MATLAB simulations for this case, can be found in the “SP_Diff”
directory on the enclosed CD.
2.
Generalized Kernel Method for parabolic PDEs.
2.1.
Given a “data function” y ( x) . (Think of y ( x) as a continuous function – a
“spline” or other interpolation of the data {xi , yi } ). We presume it to be a solution to a
given PDE at some (unknown) time value t * . Then, using the integral representation of
the solution via Green’s function of the problem in the form
b
u ( x, t )   u0 ( )G ( x,  ; t )d
(1.2.1)
a
we, ideally, expect
b
u ( x, t*)  y ( x)   u0 ( )G ( x,  ; t*) d ,
(1.2.2)
a
which is an integral equation of the first kind for the unknown function u0 . Such
problem is a so-called ill posed one. We shall use some regularization technique later in
the process to guarantee stability of its (numerically obtained) solution.
Let’s introduce a generic solution of the given PDE in the form (1)
13
b
f ( x, t )   c( )G ( x,  ; t ) d
(1.2.3)
a
with a yet unknown continuous coefficient function c ( ) . We shall try to identify c ( )
from the requirement that f in (3) makes a good approximation to the given data
function y ( x) in (2) for some fixed value of t .
Thus, we reduce the solution of (2) to minimizing the following (least squares) functional
in the class of functions defined by (3):
b
R( f )   | f ( x)  y ( x) |2 dx  || f  y ||2L2 ,
(1.2.4)
a
Its minimum, using a Frechet differential, can be found from the equation:
R' ( f )[h]  2 f  y, h L2  0 ,
(1.2.5)
where h is from the same class as f , that is
b
h( x, t )   a ( )G ( x, t ; )d
(1.2.6)
a
Equation (5) can be now put in the form (we shall temporarily drop t
expressions for f and G for ease of notation):
from the
b
 f  y, h L2   f  y,  a ( )G ( x, )d  L2
a
b
b
a
a
  [ f ( x)  y ( x)][  a ( )G ( x, )d ] dx
, (which follows by Fubini’s
Theorem)
b
b
a
a
  a( )[  ( f ( x)  y ( x))G ( x, )dx] d
b
   ( f ( x)  y ( x))G ( x, )dx, a ( ) L2  0
(1.2.7)
a
for any a( )  C[a, b] . Since C[a, b] is dense in L2 [a, b] ,
14
b
 ( f ( x)  y( x))G( x, )dx  0
(1.2.8)
a
for all  (due to the continuity of the integrand), or in discrete form (a Riemann sum for
the uniform partitioning with x fixed):
m
 [ f ( x )  y( x )]G( x , )  0 .
i 1
i
i
(1.2.9)
i
Any information about f ( x) derived from (9) will be an approximation to that in (8).
Since  is arbitrary, we can replace it with any
(“sample”) to obtain a system of equations:
xj
from the same partitioning
m
 [ f ( x )  y( x )]G( x , x )  0
i 1
i
i
i
1
................................................
m
 [ f ( x )  y( x )]G( x , x
i 1
i
i
i
m
(1.2.10)
)0
Then, introducing matrix [G ] and vectors c, y, f
G( x1 , x1 ), G( x1 , x2 ),..., G( x1 , xm ) 
[G]  ................................................  ,
G( xm , x1 ), G( xm , x2 ),..., G( xm , xm ) 
(1.2.11)
 c( x1 ) 
 y ( x1 ) 
 f ( x1 ) 
c( x ) 
 y( x ) 
 f (x ) 
2 
c   2 , y   2 , f  
........ 
........ 
........ 






 c( xm ) 
 y ( xm ) 
 f ( xm ) 
(1.2.12)
we reduce (10) to the matrix form:
(f - y )T [G]  0 or
[GT ]f  [GT ]y
(1.2.13)
Now it is time to recall (3) for f . It allows us to express every component of the vector
f in the form
15
b
m
a
j 1
f ( xi )   c( )G ( xi ,  )d  x  c( x j )G ( xi , x j ) ,
that is,
f  x[G ]c .
(1.2.14)
(1.2.15)
And from (13), we get
[GT ](x[G]c)  [GT ]y
(1.2.16)
Let constant x be “absorbed” by the unknown vector of coefficients c thus producing
(15)-(16) in the form:
f  [G]c,
[GT ][G]c  [GT ]y
(1.2.17)
Finding c requires the inversion of the matrix [GT ][G] K which is symmetric and
positive definite, therefore invertible. Unfortunately it is also close to singular which
makes the problem (17) ill posed. To obtain an approximate solution of the matrix
equation (17) we shall consider an “epsilon-regularized” version of it first:
( I  K )cε  GT y
(1.2.18)
 f  [G ]c respectively, thus
 c and fε  [G ]c 
It can be proved that cε 
 0
 0
delivering an approximate solution to (17) (see sec. 3. below)
2.2.
A Different look at (17) – the “Kernel Method connection”
We start with (3) and obtain a discrete (vector) form of it (via discretization of the
integral and computing f ( x, t ) at prescribed sample set of points xi ):
m
f ( xi , t )   c( x j )G( xi , x j ; t )
(1.2.19)
j 1
Since the solution is presumed to be given at some time value as a vector y  { y1 ,... ym } ,
that is f (t*)  y , we need to find the vector of coefficients c  {ci } satisfying the
equation
[G ]c  y
(1.2.20)
Let us apply a nonzero matrix [GT ] to both sides of (20) to produce
16
[GT ][G]c
that is
Kc  [GT ]y
z,
Kc  z ,
(1.2.21)
(1.2.22)
where K is a symmetric and positive definite matrix, c is the vector to be found and
z is a transformed data vector. Any solution of (20) is also a solution of (22). But (22)
has a unique solution due to the properties of K . Therefore the same is true for (20).
Let us presume for a moment that we found an exact solution of (22). Thus the unique
vector c can be used in construction of f in the form
f  [G ]c
(which is simply equal to y )
(1.2.23)
The advantage of (22) lies in its connection to the generic kernel method. Since (22) is
basically an ill posed operator equation, its (numerical) solution requires regularization.
Let us denote the image of the approximate solution c under K as F and require that
it delivers a minimum to the regularized empirical functional as in [4]:
Re ( F ,  ) 
1 m
 [ zi  F ( xi )]2   || F ||2H
m i 1
(1.2.24)
where H is the K -based Reproducing Kernel Hilbert Space (RKHS).
Via a Frechet differential technique, retracing exactly the motions of the generic Kernel
Method, we can prove that unique minimizer for (24) can be found in the form
F  Kc
(1.2.25)
with the vector of coefficients being a solution of the equation
( I  K )c  z
(1.2.26)
which is nothing else but equation (18), thus providing connection between the approach
in 2.1. and the Kernel Method.
2.3.
Convergence issues.
a) Theorem. ([5], p. 219 “Inverse Operator”) Let A be a bounded linear operator in
Banach space N  N . If || A ||  1 then I  A is invertible, ( I  A)1 is bounded, and
1
.
(1.2.27)
|| ( I  A)1 || 
1 || A ||
As a result, the following can be established:
17
Corollary. Let A and B be bounded operators in a Banach space N  N with A
invertible. Then A  B is invertible if || B ||  || A1 ||1 , and
|| ( A  B)1 ||  (|| A1 ||1  || B ||)1 .
(1.2.28)
b) Consider equation (22) and its regularized (epsilon-perturbed) version (26) together:
Kc  z, ( I  K )c  z
(1.2.29)
We need to prove that c  c and f  [G]c  f  [G]c when   0 .
Let
A  K,  B   I, A  B   I  K .
(1.2.30)
A is invertible since K is symmetric and positive definite. Then, from the Corollary it
follows that we need to require   || B ||  || A1 ||1  || K 1 ||1 , or
 || K 1 ||  1
(1.2.31)
Given that condition (31) is met, from (28) we find:
|| ( I  K )1 || 
|| K 1 ||
,
1   || K 1 ||
(1.2.32)
that is, the inverse of  I  K is also a bounded operator.
c) Next, denote
K  A,
A   I  K , || A  A ||    0
(1.2.33)
that is, we have a sequence of bounded invertible operators A convergent to a bounded
invertible operator A . For their inverses (which we know exist) we find that
|| A1  A1 ||  || A1 AA1  A1 A A1 || 
|| A1 ( A  A ) A1 ||  || A1 ||  || ( A  A ) ||  || A1 || 
|| K 1 ||

0
1   || K 1 ||  0
Thus we proved that the sequence of inverses A1 is convergent to A1 .
 || A1 || 
18
d) Since c  A1z, c  A1z, f  Gc , f  Gc, z  GT y ,
(1.2.34)
this immediately leads to other inequalities:
|| c  c ||  || A1  A1 ||  || z ||   || A1 ||  || A 1 ||  || GT ||  || y || 
 || K 1 || 
|| K 1 ||
|| GT ||  || y ||
1   || K 1 ||
(1.2.35)
and
|| K 1 ||
|| f  f ||  || G ||  || c  c ||   || G ||  || K || 
|| GT ||  || y ||
1
1   || K ||
1
(1.2.36)
which imply the expected convergences and approximation error estimates.
e) It may be of interest to see how the convergence A1  A1 and other properties can
be established for the case of bounded linear operators in Em , that is, all the operators
are matrices.
In this case, the matrices I and K can be diagonalized:
1
K  P DP, I  P 1IP where D is a diagonal matrix of positive eigenvalues 1 ,..., m
of K (symmetry and positive definiteness of K  G T G ). Therefore equations in (29) can
be put in the form
P 1 DPc  z, P 1 ( I  D) Pc  z
(1.2.37)
and subsequently reduced to
Db  w, ( I  D)b  w, b  Pc, b  Pc , w  Pz
(1.2.38)
The diagonal matrix ( I  D ) is obviously invertible and
,0
1/   1 , 0, ...

 0,

1/   2 , 0, ...., 0
1

 w 
b  ( I  D ) w 
 D 1 w  b
 0
................................................


.........,1/   m 
 0, 0, ...
(1.2.39)
The rest follows automatically since P is invertible:
c  P 1b  P 1b  c, f  Gc  Gc  f .
(1.2.40)
19
f) The “C_epsilon_allcases” directory on the attached CD contains MATLAB programs
that demonstrate convergence of the Kernel Method by means of computing estimates
(35)-(36) and some other related norms for all cases covered by this report (single-well
FP both formula based and simulated, double-well FP, and 2 cases of general simulated
FP). We assembled, in the illustration, some figures and data for 2 cases: formula-based
“swFP” and simulated “dw_generalFP”.
formula-based “swFP” case
The figure above shows the behavior of the right side of the estimate (36) in time and
epsilon for 0  t  5 and 101    1030 .
The following table contains several time-slices of the data presented in a surface form
above:
epsilon
t_ind = 2
t_ind = 12
t_ind = 22
t_ind = 32
t_ind = 42
t_ind = 51
10^ -1
10^ -2
10^ -3
10^ -4
…
2.12E+16
2.09E+15
2.09E+14
2.09E+13
2.09E+12
2.09E+11
2.09E+10
2.09E+09
2.09E+08
2.09E+07
2.09E+06
2.09E+05
4.62E+18
4.58E+17
4.57E+16
4.57E+15
4.57E+14
4.57E+13
4.57E+12
4.57E+11
4.57E+10
4.57E+09
4.57E+08
4.57E+07
5.76E+18
5.73E+17
5.73E+16
5.73E+15
5.73E+14
5.73E+13
5.73E+12
5.73E+11
5.73E+10
5.73E+09
5.73E+08
5.73E+07
1.43E+19
1.43E+18
1.43E+17
1.43E+16
1.43E+15
1.43E+14
1.43E+13
1.43E+12
1.43E+11
1.43E+10
1.43E+09
1.43E+08
8.94E+18
8.92E+17
8.92E+16
8.92E+15
8.92E+14
8.92E+13
8.92E+12
8.92E+11
8.92E+10
8.92E+09
8.92E+08
8.92E+07
1.81E+19
1.80E+18
1.80E+17
1.80E+16
1.80E+15
1.80E+14
1.80E+13
1.80E+12
1.80E+11
1.80E+10
1.80E+09
1.80E+08
…
20
…
…
10^ -23
10^ -24
10^ -25
10^ -26
10^ -27
10^ -28
10^ -29
10^ -30
20885
2088.5
208.85
20.885
2.0885
0.20885
0.020885
0.0020885
0.00020885
2.09E-05
2.09E-06
2.09E-07
2.09E-08
2.09E-09
2.09E-10
2.09E-11
2.09E-12
2.09E-13
4.57E+06
4.57E+05
45730
4573
457.3
45.73
4.573
0.4573
0.04573
0.004573
0.0004573
4.57E-05
4.57E-06
4.57E-07
4.57E-08
4.57E-09
4.57E-10
4.57E-11
5.73E+06
5.73E+05
57302
5730.2
573.02
57.302
5.7302
0.57302
0.057302
0.0057302
0.00057302
5.73E-05
5.73E-06
5.73E-07
5.73E-08
5.73E-09
5.73E-10
5.73E-11
1.43E+07
1.43E+06
1.43E+05
14277
1427.7
142.77
14.277
1.4277
0.14277
0.014277
0.0014277
0.00014277
1.43E-05
1.43E-06
1.43E-07
1.43E-08
1.43E-09
1.43E-10
8.92E+06
8.92E+05
89159
8915.9
891.59
89.159
8.9159
0.89159
0.089159
0.0089159
0.00089159
8.92E-05
8.92E-06
8.92E-07
8.92E-08
8.92E-09
8.92E-10
8.92E-11
1.80E+07
1.80E+06
1.80E+05
18006
1800.6
180.06
18.006
1.8006
0.18006
0.018006
0.0018006
0.00018006
1.80E-05
1.80E-06
1.80E-07
1.80E-08
1.80E-09
1.80E-10
With epsilon small enough, || f  f || in (36) does not exceed 10 10 across all times.
simulated “dw_generalFP” case
Similarly, in the double-well general FP case we find behavior of the right side of the
estimate (36) in time and epsilon for 0  t  5 and 101    1030 .
and
21
epsilon
t_ind = 1
t_ind = 11
t_ind = 21
t_ind = 31
t_ind = 41
t_ind = 51
10^ -1
10^ -2
10^ -3
10^ -4
…
3.08E-01
3.04E-02
3.03E-03
3.03E-04
3.03E-05
3.03E-06
3.03E-07
3.03E-08
3.03E-09
3.03E-10
3.03E-11
3.03E-12
3.03E-13
3.03E-14
3.03E-15
3.03E-16
3.03E-17
3.03E-18
3.03E-19
3.03E-20
3.03E-21
3.03E-22
3.03E-23
3.03E-24
3.03E-25
3.03E-26
3.03E-27
3.03E-28
3.03E-29
3.03E-30
9.91E+16
9.81E+15
9.80E+14
9.80E+13
9.80E+12
9.80E+11
9.80E+10
9.80E+09
9.80E+08
9.80E+07
9.80E+06
9.80E+05
9.80E+04
9.80E+03
979.51
97.951
9.7951
0.97951
0.097951
0.0097951
0.00097951
9.80E-05
9.80E-06
9.80E-07
9.80E-08
9.80E-09
9.80E-10
9.80E-11
9.80E-12
9.80E-13
4.03E+17
4.00E+16
3.99E+15
3.99E+14
3.99E+13
3.99E+12
3.99E+11
3.99E+10
3.99E+09
3.99E+08
3.99E+07
3.99E+06
3.99E+05
3.99E+04
3994.4
399.44
39.944
3.9944
0.39944
0.039944
0.0039944
0.00039944
3.99E-05
3.99E-06
3.99E-07
3.99E-08
3.99E-09
3.99E-10
3.99E-11
3.99E-12
2.46E+18
2.44E+17
2.44E+16
2.44E+15
2.44E+14
2.44E+13
2.44E+12
2.44E+11
2.44E+10
2.44E+09
2.44E+08
2.44E+07
2.44E+06
2.44E+05
2.44E+04
2440.6
244.06
24.406
2.4406
0.24406
0.024406
0.0024406
0.00024406
2.44E-05
2.44E-06
2.44E-07
2.44E-08
2.44E-09
2.44E-10
2.44E-11
4.22E+18
4.19E+17
4.18E+16
4.18E+15
4.18E+14
4.18E+13
4.18E+12
4.18E+11
4.18E+10
4.18E+09
4.18E+08
4.18E+07
4.18E+06
4.18E+05
41845
4184.5
418.45
41.845
4.1845
0.41845
0.041845
0.0041845
0.00041845
4.18E-05
4.18E-06
4.18E-07
4.18E-08
4.18E-09
4.18E-10
4.18E-11
1.04E+18
1.03E+17
1.03E+16
1.03E+15
1.03E+14
1.03E+13
1.03E+12
1.03E+11
1.03E+10
1.03E+09
1.03E+08
1.03E+07
1.03E+06
1.03E+05
1.03E+04
1030.8
103.08
10.308
1.0308
0.10308
0.010308
0.0010308
0.00010308
1.03E-05
1.03E-06
1.03E-07
1.03E-08
1.03E-09
1.03E-10
1.03E-11
…
…
…
10^ -23
10^ -24
10^ -25
10^ -26
10^ -27
10^ -28
10^ -29
10^ -30
The figure below shows epsilon-behavior of the column t_ind = 11.
22
g) The estimates (35)-(36) use operator norms to show convergence in principle, but in
reality the processes shows a much better “convergence property” than the estimates.
The directories “dwFP_simulations”, “dwFPf0_S05”, “genFPdw200”, “genFPdw_f0”,
“genFPsw200”, “genFPsw_f0”, “swFP_simulations”, “swFPf0_S05” each contain
MATLAB programs that show convergence of corresponding KM to an “perfect surface”
at a specified time-slice. For example, in a double-well FP case at the time-slice with
t_ref = 4, we find that for   10 ( p 1) the quality of fit improves rapidly, and is very
reasonable even at   10 2 :
p = 1, mse = 4.5963e-005
p = 2, mse = 6.3379e-007
norm(y-f) = 0.0067796
norm(y-f) = 0.00079611
23
p = 3, mse = 6.7219e-009
p = 4, mse = 6.9516e-011
p = 5, mse = 7.3266e-013
p = 6, mse = 1.5989e-014
p = 7, mse = 1.2125e-015
p = 8, mse = 2.5155e-017
p = 9, mse = 2.7725e-019
p = 10, mse = 2.8006e-021
p = 11, mse = 2.8022e-023
norm(y-f) = 8.1987e-005
norm(y-f) = 8.3376e-006
norm(y-f) = 8.5595e-007
norm(y-f) = 1.2645e-007
norm(y-f) = 3.4821e-008
norm(y-f) = 5.0155e-009
norm(y-f) = 5.2654e-010
norm(y-f) = 5.292e-011
norm(y-f) = 5.2935e-012
All quoted cases demonstrate comparable quality of fit and fast convergence.
3.
Eigenfunction expansion for the 1-D Fokker-Planck equation derivation
Consider standard 1-D Fokker-Planck equation in the infinite domain with natural
boundary conditions ( w  0 at |  |  ):
w
2w

 D 2 
( w)  0
t


(1.3.1)
A substitution of variables
x  k , k 2 

x
, w( , t )  w( , t )
2D
k
u ( x, t )
(1.3.2)
reduces (1) to
u   2u



( xu )  0
2
t 2 x
x
(1.3.3)
Another change of functions
1
u ( x, t )  v( x, t ) exp(  x 2 )
2
(1.3.4)
leads to a modified equation:
v   2v

  ( x )v  0
t 2 x 2
(1.3.5)
where
24

 ( x)  ( x 2  1)
(1.3.6)
2
Now, applying the separation of variables routine to (5)
v( x, t )   ( x)et ,
(1.3.7)
we obtain the following eigenvalue problem:
 '' ( x 2  1)    ,  
2(   )
(1.3.8)

If we presume zero boundary conditions at infinity, then using [6], we find a complete
system of corresponding eigenvalues/eigenfunctions is given by:
n  2n  2, n  0,1, 2,...
 n ( x),  m ( x) L2 (  , ) 

 (2

e  x H n ( x) H m ( x)
2
n
n !  )1/ 2 (2 m m !  )1/ 2
0 m  n

1 m  n
n ( x)  (2n n!  )1/ 2 e x / 2 H n ( x)
2
dx
(1.3.9)
(1.3.10)
where H n ( x) are the Chebyshev/Hermite polynomials.
n ( x) is an orthonormal set of functions:
(1.3.11)
Therefore,
n   n ,
(1.3.12)
vn ( x, t )  n ( x)e  nt
(1.3.13)
and

v( x, t )   cnn ( x)e  nt
(1.3.14)
n 0
Thus, using (4), we found the eigenfunction expansion for the solution of (3), satisfying 0
boundary conditions at infinity:
25
u ( x, t )  e
1
 x2 
2
 c  ( x)e 
 nt
n
n 0
(1.3.15)
n
Now, if we impose the initial condition function in (3)
u ( x, 0)  u0 ( x) ,
(1.3.16)
the (Fourier) coefficients cn can be found as follows:
cn  u0 ( x)e
1 2
x
2
, n ( x) L2 

n
1/ 2
 (2 n!  ) u0 ( x) exp(

x2
x2
) exp(  ) H n ( x)dx
2
2
(1.3.17)
or

cn 
 (2 n!
n
 )1/ 2 u0 ( x) H n ( x)dx
(1.3.17’)

To find an expression for the fundamental solution (Green’s function) of (3) with 0 b.c.
we put
(1.3.18)
u0 ( x)   ( x  x ') .
And so,
cn  (2n n !  )1/ 2 H n ( x ')  exp(
x '2
)n ( x ')
2
(1.3.19)
Therefore, the eigenfunction expansion formula for the corresponding Green’s function
is:
G ( x, x '; t )  e
1
 ( x 2  x '2 ) 
2

n 0
n
( x ') n ( x)e nt
(1.3.20)
or
G ( x, x '; t )  e
1
 x2 
2
 (2
n 0
n
n !  ) 1/ 2 H n ( x ')  n ( x)e  nt ,
(1.3.20’)
or even
G ( x, x '; t )  e  x
2

 (2
n 0
n
n !  ) 1 H n ( x ') H n ( x)e  nt
(1.3.20’’)
depending on computational preferences.
The original Green’s function, W, from the original equation (1) can be calculated as
follows. Since

u ( x  k , t )  w( , t ), k 2 
,
(1.3.21)
2D
using (15) we can find
26
w( , t )  e
1
 ( k )2 
2
 c  (k )e 
 nt
n
n 0
(1.3.22)
n
and
w( ,0)  w0 ( )  e
1
 ( k )2 
2
 c  (k )
n 0
n
(1.3.23)
n
It can be easily verified that
 n (k ),  m (k ) L2 (  , ) 
e  ( k ) H n ( k  ) H m ( k  )

 (2

2
n !  )1/ 2 (2m m !  )1/ 2
n
0

 1
 k
mn
d
(1.3.24)
mn
Then, similarly to (17), we find
1
( k ) 2
1
cn   w0 ( )e 2
, n (k ) L2
k
(1.3.25)
If the initial condition for (1) is
w( , 0)   (   ')
(1.3.26)
then
cn  k exp(
(k ')2
)n (k ')
2
(1.3.27)
and
W ( ,  '; t )  ke
1
 (( k )2 ( k ')2 ) 
2

n 0
 ke  ( k )
2

 (2
n 0
n
n
(k ) n (k ')e  nt 
n !  ) 1 H n (k ') H n (k )e  nt
(1.3.28)
2. Remarks
1) A “shortcut” connection between G and W can be established as follows. We
start with the integral representation of the solution u in the form
27
u( x, t )   u0 ( x ')G( x, t; x ')dx '
(1.3.29)
R
and follow with the substitution (2) to obtain
w( , t )  u(k , t )   u0 ( x ')G(k , t; x ')dx ' .
R
Now let, x '  k ' . Then dx '  kd ', u0 (k ')  w0 ( ) . And we get,
w( , t )  k  w0 ( ')G(k , t; k ')d ' .
(1.3.30)
R
Thus,
W ( ,  '; t )  kG (k , k '; t )
(1.3.31)
as above (compare (20) and (28)).
2) Although the expansion of the type (28) is mentioned in quantum mechanics
literature ([7]), we offer here another (more transparent) derivation of it.
3. Validation.
Starting with a known series ([8], p.329, case m = 0)

n
 n! H
n 0
n
4
1
exp[
( xy   x 2   y 2 )], |  | 
2
1  4
2
1  4 2
1
( x)H n ( y ) 
we replace x, y,  with kx, ky, k 
1
1  4 2

1
1  e2 t
,
(1.3.32)

1
1
, 0    e t  (t  0) to find
2D
2
2
4
2e t
,

1  4 2 1  e2 t
(1.3.33)
4
1  4 2
2
2
2
[
k
xy


(
kx
)


(
ky
)

(kx) 2 ] 
2
1  4
4
k2
(4 xy  4 2 x 2  4 2 y 2  x 2  4 2 x 2 ] 
2
1  4


2 D(1  e
2 t
)
( x 2  y 2e2 t  2 xye t ) ,
(1.3.34)
28
and, finally
W ( x, y; t )  ke  ( kx )
2

 (2
n 0
ke ( kx )
2
n
n !  ) 1 H n (ky ) H n (kx)e  nt =
4
exp[
(k 2 xy   (kx) 2   (ky ) 2 )] 
2
2
1

4

1  4
1

4
1  4 2
2
2
2
exp(
[
k
xy


(
kx
)


(
ky
)

(kx) 2 ]) 
2
2 t
1  4
4
2 D(1  e )

2 D(1  e
2 t
exp(
)

2 D(1  e2 t )
exp(

2 D(1  e
2 t
)
 ( x  e  t y ) 2
2 D(1  e2 t )
( x 2  y 2 e 2 t  2 xye  t )) 
)  P ( x, t ; y ) .
(1.3.35)
That is, W in (28) is identical to the expression of the fundamental solution (Green’s
function) of the FP equation in [3].
4.
Project background information and resources.
4.1.
Equations
1) General Fokker-Planck equation
w

2
 LFP w  [ ( D (1) w)  2 ( D (2) w)]
t
x
x
(1.4.1)
D(1) ( x)   f '( x), D(2) ( x)  ax 2  b, a  0
(1.4.2)
w

2
 LFP w  [ ( f '( x) w)  2 ( D (2) ( x) w)]
t
x
x
(1.4.3)
or
Single-well parabolic potential
1
f ( x)   ( x   ) 2  b;
2
f '( x)   x  
xd
(1.4.4)
29
Asymmetric double-well parabolic potential
 fl , x  0
f ( x)  
 fr , x  0
(1.4.5)
1
fl ( x)   ( x  m1 ) 2  b1 , x  0
2
1
f r ( x)   ( x  m2 ) 2  b2 , x  0
2
(1.4.6)
where
f ( x) is continuous at 0 if constants b1 , b2 are chosen from the relation:
1
 (m12  m22 )  b2  b1
2
(1.4.7)
 ( x  m1 ), x  0
f '( x)  
 ( x  m2 ), x  0
(1.4.8)
Then
( f ' is discontinuous at 0).
The standard FP corresponds to the choice of constant D(2)  D :
w
2w 
 LFP w  D 2  ( D (1) w)
t
x
x
(1.4.9)
Further restrictions on coefficients (constant potential, therefore D (1)  0 ) reduces FP
to a diffusion equation:
w
2w
D 2
t
x
(1.4.10)
Remark. Depending on the choice of the potential f and diffusion coefficient D (2)
we shall have 4 cases to consider.
Special FP case requires special combination of coefficients:
w
2w
w
 ( x 2  b) 2   x
 cw, b  0
t
x
x
(1.4.11)
Special diffusion case (optional):
30
w
2w
 ax 2 2
t
x
4.2.
(1.4.12)
All problems will require initial condition
w( x,0)  w0 ( x)
(1.4.13)
and 0 boundary conditions at x   l or at infinity.
4.3.
(1.4.14)
Known and derived fundamental solutions
diffusion equation (10):
( x  x ')2

H (t   )
K ( x, t ; x ', ) 
e 4 D ( t  )
4 D (t   )
(1.4.15)
standard FP equation (9) with a single-well potential (4):
 H (t   )
P( x, t ; x ', ) 
2 D(1  e 2 (t  ) )
exp( 
 ( x  e  (t  ) x ') 2
2 D(1  e 2 ( t  ) )
)
(1.4.16)
3) special FP equation (11):
H (t  t ')
W ( x, t ; x ', t ') 
 ( x ') 2  b
K c ( z ( x)  z ( x '), t  t ')
(1.4.17)
where
x
z ( x)  z ( x ')  
x'
d
(1.4.18)
 2  b
and
K c ( z , t )  ect K ( z , t )
(1.4.19)
special diffusion case (optional):
(ln x  ln x '  at )

H (t )
4 a
E ( x, t ; x ') 
e
, x, x '  0
x ' 4 at
2
(1.4.20)
31
Remark. In all cases, we shall try a simulation technique to generate both known
fundamental solutions (for comparison/validation) and unknown ones (as the only source
of information)
4.4.
Integral representation via fundamental solution/Green’s function
The solution of all the problems outlined in 4.1-4.2 above can be expressed in the form:

w( x, t ) 
 w ( )G( x,  , t )d
0
(1.4.21)

(initial-value problem in infinite domain), or
a
w( x, t ) 
 w ( )G( x,  , t )d
0
(1.4.22)
a
(initial-value/boundary-value problem), where G stands for either fundamental solution
in case of (21), or Green’s function in case of (22).
These expressions make a “backbone” of the Kernel Method in application to PDEs.
Simulation of the delta-function and Green’s functions
Without entering the detail of the so-called generalized functions/distributions and their
spaces, we define here delta sequence as a “cap-shaped function” with the property:
lim   ( x) ( x)dx   (0) ,
 0

(1.4.23)
which means that  ( x)   ( x) in the distributional sense. Different examples of
 ( x) are available in the literature. We shall use the following ([2]):

2
), | x |  
C exp( 2
 ( x)  
  | x |2
0,
| x | 

(1.4.24)
where constant C is chosen so that
  ( x)dx  1,
x
n
(1.4.25)
Green’s functions for the problems in 4.1-4.2 can be found as solutions to corresponding
PDEs with  ( x  x ') as initial condition, thus representing a response of the system at
32
point ( x, t ) to instantaneous unit delta source (disturbance) applied at the point ( x ', 0) .
For the simulation purposes we shall use delta-sequence  ( x) (instead of delta) as the
initial condition and the pdepe Matlab solver.
II.
Theoretical aspects of the Statistical Analysis performed in the
project
1.
Mean Square Error (MSE)

Let Y and Y be the empirical distribution and the hypothesized pdf, respectively. Then,
the mean square error measures the mean of the summation of the differences at each

point between Y and Y .
n
(y
MSE =
i
 yi ) 2
1
n  k 1
where n is the number of points and k is the number of parameters that where estimated.
In our case, since n is very large, k and 1 can be ignored in the denominator.
We will perturb the original data by introducing noise or data reduction or both, then we
will reconstruct it. We will perform this process 1000 times for each case. Then, we will
analyze which type of distribution the MSEs
2.
Probability Density Functions in use
Here are the distributions of the mean square error data after experiment of
running 1000 times.
a.
The Inverse Gaussian Distribution
A continuous random variable X is said to have an Inverse Gaussian Distribution
if the pdf of X is:
f ( X ; , ) 




exp  2 ( x   )2 
3
2 x
 2 x

=0
where the parameter  is the mean.
where x > 0
otherwise
33
Figure 1-The Inverse Gaussian Function with Noise Introduced
b.
Log logistic
The variable x has a log logistic distribution with location parameter µ and scale
parameter  > 0 if ln x has a logistic distribution with parameters µ and  . The
logistic distribution has the density function:
(x  ) 
exp 

 

f ( X ; , ) 
when x  0
2

(x  ) 
 1  exp 

  

=0
otherwise
with location parameter µ and scale parameter  > 0, for all real x.
Figure 2-The Log-Logistic Function with Both Effects
c.
Log-normal distribution
34
Let a random variable X be normally distributed with mean  and variance  2 .
Then if we write X = ln Y, then Y is said to have a Log-Normal distribution.
Then the pdf for Y is
f ( y) 
1
y 2
 (ln y   ) 2
e
( 2 2 )
,y 0
(   (
2
f ( y )  0, elswhere.
For a Log-Normal distribution Y, E (Y )  e
2
))
and V (Y )  e 2    (e  1).
2
2
Figure3-Log Normal Distribution
d.
Birnbaum-Saunders distribution
The Birnbaum-Saunders distribution has the density function:
2

 x /    / x 
x
/



/
x
1


 when x >0
f (X ;  , ) 
exp 

2


2

2

x
2






 
= 0

otherwise
with scale parameter  > 0 and shape parameter  > 0, for x > 0. If x has a
Birnbaum-Saunders distribution with parameters  and  , then
1
x/ 
/x



has a standard normal distribution.
35
Figure4-The Birnbaum Saunders Distribution
3.
Log Likelihood Inference
Let specify a probability density or probability mass function for the observations
zi : g ( z )
In this expression θ represents one or more unknown parameters that govern the
distribution of Z.
Let Z has any distribution, then the likelihood function, L(θ; Z), is the product of
the probability density function g ( z ) evaluated at the n data points.
N
L( ; Z )   g  zi 
i 1
the probability of the observed data under the model g , and we can consider that
L( ; Z ) as a function of θ with Z be fixed.
n
N
i 1
i 1
We denote the logarithm of L   by l     l ( ; zi )   log g  zi  , and this
expression is called the log-likelihood, and each value l ( ; zi )  log g ( zi ) is
called a log-likelihood component.
III.
Simulation concepts, techniques and outcomes.
1.
Omega-epsilon Simulation
1.1 Discussion
36
The Green’s functions for the problems in 4.1-4.2 can be found as solutions to
corresponding PDEs with  ( x  x ') as its initial condition, thus representing a response
of the system at a point ( x, t ) to instantaneous unit delta source (disturbance) applied at
the point ( x ', 0) . As stated before, the set of equations for which we generated Green’s
functions were the Fokker-Planck family of equations. The Fokker-Planck equations are
defined with a drift D(1) ( x) and diffusion D (2) ( x) coefficients that influence the
predicted distribution of particles in an external field.

W   (1)
2
   D ( x)  2 D(2) ( x)  W
t  x
x

D(1) ( x)   f '( x), D(2) ( x)  1, D(2) ( x)  ax 2  b
In our experiments we simulated the Green’s functions with single and double well
potentials f(x) (potential of the field of external forces resisting the motion of particles)
whose profiles are illustrated below.
Single-well
Double-Well
In order to initiate our simulation we needed to approximate the initial source function
which is a Dirac’s delta. Since delta functions can not be used by MATLAB directly, we
replaced it with a sequence of functions,  ( x) , which is a smooth substitute which has a
spike at one point.
1.2 Experiment
We defined  ( x) as a “cap-shaped” sequence of functions, by the formulas below:

2
), | x |  
C exp( 2
 ( x)   
  | x |2
0,
| x | 

  ( x)dx  1,
x
n
37
It was especially important that we chose our constant C in such a way that guaranteed
our integration of  ( x) to produce a value of 1. This requirement of maintaining an
integration of 1 stems from our goal of maintaining probabilistic properties throughout
our simulations. The following code describes the implementation we used to define our
Omega-epsilon with the requirements stated above.
function u0 = swfp_deltaic(x)
% ic - initial cond. at t=0
%
global xp J
p = -5;
e = 10^p;
xJ = xp(J);
if abs(x-xJ)<e
omega_e = exp(-(e^2/(e^2-(x-xJ)^2)))/0.14715;
else
omega_e = 0;
end
u0 = omega_e;
Our Omega-epsilon has the following property lim  ( x) ( x)dx   (0) ,
 0  
meaning    in a distributional sense.
For simulation purposes we used delta sequences  ( x  x ') as our initial condition then
ran our pdepe MATLAB solver for every point x’.
In reference to (1.4.4), our single-well parabolic potential revealed a centrally located
area of low resistance.
Figure 1. Single-well parabolic potential
38
Figure 2 below illustrates the single-well Fokker-Planck responses to delta positions
close to either end of our x-domain (close to the boundaries). We observed a central area
of “attraction” as was expected from our single-well potential field presented in Figure 1.
Figure 2. Initial delta near left and right-most boundary conditions.
In reference to (1.4.6), our double-well parabolic potential revealed two general areas of
low resistance.
Figure 3. Double-well parabolic potential
Figure 4 below illustrates the double-well Fokker-Planck responses to delta positions
near the center and right-most boundary condition. We observed not one but two areas of
“attraction” (most probable location of the particles) as was expected from our doublewell potential field presented in figure 3.
39
Figure 4. Initial delta near the center and right-most boundary condition.
After close examination of the double-welled Fokker-Planck responses to the delta
positions injected near the right-most boundary condition we realized that our particles
were converging to the right well without chance of ever converging to the left well due
to the high resistance located in between these two wells.
For our generalized Fokker-Planck cases we observed identical PDF behaviors with
slight variations in the shape of our wells due to the variable diffusion coefficient
D(2) ( x)  ax2  b . Our generalized diffusion function caused the wells to spread and
widen. Figures 5 and 6 below illustrate the responses observed.
Figure 5. General single-well FP responses to delta
40
Figure 6. General double-well FP response to delta
2. Green’s Function Simulation
Discussion
After establishing the technique of simulating our Fokker-Planck responses to delta initial
conditions at various points, the next step involved extracting our kernels, or Green’s
functions from these simulations.
Experiment
At this point we generated FP responses for all four cases previously mentioned across
every interval within our x space and extracted time slices for all values of time. The
data received from these time slices generated our Green’s functions or kernels. The
following code illustrates this process of kernel extraction for our simulated single-well
case.
% simPkernel.m
%
function [w] = simP(x,t,g,J)
%
%generates simulated Kernel for s.w. FP
%
global g J xp
g = .5;
D = 1.0; % Set the value of D for use in the equation for the kernel
a1 = -40;
b1 = 40;
x1 = linspace(a1,b1,201);
a2 = -20; % NOTE: These values will be reassigned shortly to exactly
b2 = 20; % match values in x = [a1,b1]
a2index = 1 + (a2-a1)/(b1-a1)*(length(x1) - 1);
b2index = 1 + (b2-a1)/(b1-a1)*(length(x1) - 1);
a2 = x1(a2index);
b2 = x1(b2index);
display(['a1 = ', num2str(a1), ', b1 = ', num2str(b1),...
', a2 = ', num2str(a2), ', b2 = ', num2str(b2)]);
x2 = x1(a2index:b2index);
t0 = 0;
tf = 5;
t = linspace(t0,tf,51);
xp = x1;
41
for J = 1:length(xp)
J
v = swfp_delta(x1,t,g,J); %swfp_delta must be re-calibrated if number of points is
changed
W(:,J,:) = v';
end
save simPxxpt.mat W
Figure 7 below illustrates the kernels produced for various time intervals.
Figure 7. Simulated single-well kernels at different time slices.
After establishing this technique for generating kernels for the simulated single-well case
we applied this same method to our simulated double-well case. Figure 8 below
illustrates kernels produced for the simulated double-well case.
Figure 8. Simulated double-well kernels at different time slices.
We noticed a progressive separation within our simulated double-well kernels as time
progressed. This behavior is consistent with expected responses from double-welled
potentials.
42
In the final steps of our Green’s function simulations we applied the same techniques
used for the simulated single and double-well kernels to our generalized Fokker-Planck
cases and observed identical distribution behaviors. The shape of our kernels however,
reflected a flattening “cap shaped” figure influenced by the generalized diffusion
D(2) ( x)  ax2  b . Figures 9 and 10 illustrate the kernels produced for the generalized
single and double-well Fokker-Planck responses.
Figure 9. Simulated general single-well kernels at different time slices.
Figure 10. Simulated general double-well kernels at different time slices.
The kernels demonstrated above are generated in the following directories: swFPf0_S05,
dwFPf0_S05, genFPsw200, genFPdw_f0 as Matlab .mat files (simPxxpt.mat,
sim_dwFPkernel.mat, genswFPkernel200.mat, gendwFPkernel200.mat respectively) as
3-dimensional arrays, and they allow to observe the kernel surfaces for any t-slice
between 0 and 5.
IV.
The “data generators” and statistical experiments
43
1.
Regular double well Fokker-Planck equation case.
Generation of noisy and/or randomly reduced data.
We are using the following PDF function
1
1
1
5  4 ( x  7)2 2  4 (1 x )2 1  4 (6 x )2
e
 e
 e
8
8
U8
1
2 2
as our original data:
0.18
original data
0.16
0.14
0.12
0.1
0.08
0.06
0.04
0.02
0
0
50
100
150
200
250
To simulate data collected from the real world situation, we would “damage” the original
data by the introduction of noise and random data loss effect. Examples of noise and
data loss include misinterpreted data, measurement error, computation errors, and data
that become unacceptable due to various other factors.
Case (1) with only noise added. We keep our original data effected by random
variations at each data point as follows:
yn = y + (0.1*y).*randn(201,1); % introduction of noise
that is original initial data vector y is perturbed by 10% modulated normally distributed
random (separately generated for each data point) noise. Two curves yn (with noise,
red) and y (original, blue) are plotted in the figure below:
44
Case (2) with introduction of random data loss effect only. The data points retain
their positions but a portion of the data points is lost randomly (chosen by the following
part of the program “dwf0generator.m”, or a similar one depending on the equation
case):
for i = 1:length(x)
n = randn +.3;
if n > 0
xp = cat(1, xp, x(i));
yp = cat(1, yp, yn(i));
Ip = cat(1, Ip, i);
else
counter = counter+1;
end
end
If n > 0, we keep the point, otherwise we do not. Thus a new sample (with random
“holes” in it) is generated. We may effect the sensitivity (and percentage of points
remained) by varying the number (currently = .3) added to randn MATLAB function call.
The key factor is “randomness”. We do not want any human influence on the decision
of whether to keep certain points or not. Below is a graphical example of such randomly
“reduced” data.
45
0.18
"damaged" data
with random
data loss effect
0.16
0.14
0.12
0.1
L = 124
0.08
0.06
0.04
0.02
0
-0.02
-40
-30
-20
-10
0
10
20
30
40
Case (3) with both noise and random data loss effects added.
0.2
"damaged" data
with both effects
0.15
0.1
L = 114
0.05
0
-0.05
-40
-30
-20
-10
0
10
20
30
40
Interpolation of the “damaged” data.
We would interpolate the “damaged” data using three methods: cubic splines, Gaussian
kernels, and double well Fokker-Planck pde-based kernels.
Following are the examples of interpolated data (see dwf0generator.m in dwFPf0_S05
directory).
The following figure shows “damaged” data with cubic splines (Matlab generic) used
for interpolation:
46
0.2
original data
cubic splines
0.15
0.1
L = 114
0.05
0
-0.05
-40
-30
-20
-10
0
10
20
30
40
The next figure shows “damaged” data with pde-based kernels used for the
interpolation:
0.18
original data
pde based kernels
0.16
0.14
0.12
0.1
L = 114
0.08
0.06
0.04
0.02
0
-0.02
-40
-30
-20
-10
0
10
20
30
40
And, finally, the “damaged” data with Gaussian kernels used for the interpolation:
47
0.2
original data
Gaussian kernels
0.15
0.1
L = 114
0.05
0
-0.05
-40
-30
-20
-10
0
10
20
30
40
It should be noted that both kernel-based (pde and Gaussian) interpolation procedures are
performed here using the Kernel Method outlined in sec. I-2.
1000 runs – the MSE data collected
To ensure the accuracy of the MSEs, we ran each interpolation method for 1000 times
generating1000 MSEs. We graphed the 1000 MSE as a bar graph for each of the three
methods and three cases.
a) 1000 MSEs collected for Case (1) with cubic splines interpolation
(dwf0generator.m):
48
b) 1000 MSEs collected for Case (1) with Gaussian kernels interpolation
(dwf0generator.m):
c) 1000 MSEs collected for Case (1) with pde-based kernels interpolation
(dwf0generator.m):
d) 1000 MSEs collected for Case (2) with cubic splines interpolation
(dwf0generator.m):
49
e) 1000 MSEs collected for Case (2) with Gaussian kernels interpolation
(dwf0generator.m):
f) 1000 MSEs collected for Case (2) with pde-based kernels interpolation
(dwf0generator.m):
50
g) 1000 MSEs collected for Case (3) with cubic splines interpolation
(dwf0generator.m):
h) 1000 MSEs collected for Case (3) with Gaussian kernels interpolation
(dwf0generator.m):
51
i) 1000 MSEs collected for Case (3) with pde-based kernels interpolation
(dwf0generator.m):
2.
General double well Fokker-Planck equation case.
For the brevity of presentation we shall consider here only one case of the “damaged”
and interpolated data, the so-called Case (3) (noise plus data reduction), for the general
double well FP (see genFPdw_f0 directory):
j) 1000 MSEs collected for Case (3) with cubic splines interpolation
(gendwFP_f0generator.m):
52
k) 1000 MSEs collected for Case (3) with Gaussian kernels interpolation
(gendwFP_f0generator.m):
l) 1000 MSEs collected for Case (3) with pde-based kernels interpolation
(gendwFP_f0generator.m):
53
3.
Log-Likelihood and the best fit parametric PDFs
Probability Density Functions (PDFs)
After we collected and graphed the 1000 MSE for each case and each method, we
conducted statistical analysis. First we have to find among known PDF functions, those
that produce the best fit with the data sets under investigation. These standard PDFs can
later be used for further analysis. The following PDFs will be used:
Inverse Gaussian (, )
Log Normal (, )
Birnbaum Saunders (, )
Log Logistic (, )
As it was outlined in sec. II-3, the parametric PDF that fits observed data with the
largest log-likelihood ([10]) delivers the best approximation to the empirical data.
Therefore, we shall compare suitability of several parametric models (MATLAB,
Statistical Toolbox) by observing and comparing log-likelihood values generated by
“dfittool”. A greater log-likelihood means a closer match.
A.
Regular double well FP case
3.1.
Case (1) with all interpolation schemes.
a) Case (1) with cubic splines interpolation (dwf0generator.m):
54
PDF
Inverse Gaussian
Log Normal
Birnbaum
Saunders
11873.9
 = 1.07873e005
 = 5.37131e005
10957.2
 = -11.5265
 = 0.427781
10956
 = 9.84265e006
 = 0.437777
Plot
Log-Likelihood
Parameters
Case (1) with Gaussian kernels interpolation (dwf0generator.m):
PDF
Inverse Gaussian
Birnbaum
Saunders
Log Normal
12856
 = 3.07038e006
 = 6.32177e006
11936.4
 = 2.51997e006
 = 0.661644
11935.1
 = -12.8945
 = 0.631862
Plot
Log-Likelihood
Parameters
Case (1) with dw-fp pde-based kernels interpolation (dwf0generator.m):
PDF
Inverse Gaussian
Birnbaum
Saunders
Log Normal
12314.3
 = 5.99986e-
11395.6
 = 5.24623e-
11393.6
 = -12.1582
Plot
Log-Likelihood
Parameters
55
006
 = 1.94812e005
3.2.
006
 = 0.536041
 = 0.52.0043
Case (2) with all interpolation schemes.
d) Case (2) with cubic splines interpolation (dwf0generator.m):
PDF
Plot
Inverse Gaussian
Log Normal
Log Logistic
Log-Likelihood
Parameters
15394.6
 = 3.47268e007
 = 2.22912e008
14534.2
 = -16.4695
 = 1.67673
14522.2
 = -16.4896
 = 0.962886
Case (2) with Gaussian kernels interpolation (dwf0generator.m):
PDF
Plot
Inverse Gaussian
Log Logistic
Log Normal
Log-Likelihood
Parameters
15116.9
 = 3.47795e007
 = 3.88567e007
14318.3
 = -15.4416
 = 0.36216
14239.4
 = -15.3316
 = 0.721597
f) Case (2) with pde-based kernels interpolation (dwf0generator.m):
PDF
Plot
Inverse Gaussian
Log Normal
Log Logistic
56
Log-Likelihood
Parameters
3.3.
13417.8
 = 2.76503e005
 = 4.68352e008
12462
 = -14.9041
 = 2.78318
12452.9
 = -15.156
 = 1.59195
Case (3) with all interpolation schemes.
g) Case (3) with cubic splines interpolation (dwf0generator.m):
PDF
Plot
Inverse Gaussian
Log Logistic
Log Normal
Log-Likelihood
Parameters
11383
 = 1.37327e005
 = 2.95943e005
10477.9
 = -11.4118
 = 0.341431
10472.6
 = -11.398
 = 0.610765
Case (3) with Gaussian kernels interpolation (dwf0generator.m):
PDF
Plot
Inverse Gaussian
Log Logistic
Log Normal
Log-Likelihood
Parameters
11707
 = 9.23511e006
 = 8.18018e006
10799.1
 = -12.0926
 = 0.469961
10787.3
 = -12.0426
 = 0.849466
57
Case (3) with pde-based kernels interpolation (dwf0generator.m):
B.
PDF
Plot
Inverse Gaussian
Log Normal
Log Logistic
Log-Likelihood
Parameters
8717.19
 = 0.00029955
 = 2.91054e-005
7729.22
 = -9.62632
 = 1.61391
7719.57
 = -9.75443
 = 0.92087
General double well FP case
The following collection of tables describes the best fit parametric PDFs for one case of
the “damaged” and interpolated data (the so-called Case (3), noise plus data reduction),
for the general double well FP (see genFPdw_f0 directory):
Case (3) with cubic splines interpolation (gendwFP_f0generator.m):
PDF
Plot
Inverse Gaussian
Log Logistic
Log Normal
Log-Likelihood
Parameters
11670.1
 = 1.12009e005
 = 3.23783e005
10765.9
 = -11.5746
 = 0.300575
10758
 = -11.5594
 = 0.539551
k) Case (3) with Gaussian kernels interpolation (gendwFP_f0generator.m):
PDF
Plot
Inverse Gaussian
Log Logistic
Log Normal
58
Log-Likelihood
Parameters
12305.3
 = 5.11972e006
 = 6.54671e006
11398.3
 = -12.5499
 = 0.416615
11392.3
 = -12.5157
 = 0.744553
l) Case (3) with pde-based kernels interpolation (gendwFP_f0generator.m):
3.4.
PDF
Plot
Inverse Gaussian
Log Logistic
Log Normal
Log-Likelihood
Parameters
10230
 = 6.4081e-005
 = 1.10986e005
9263.37
 = -11.0916
 = 0.734337
9249.1
 = -10.9562
 = 1.33467
Analysis of the Outputs
Now we perform analysis by comparing Matlab generated means and variances for all
cases. In previous sections we demonstrated that Inverse Gaussian parametric PDF is
the best (maximum likelihood principle applied) among 20+ parametric models provided
by the Matlab Statistics Toolbox for all our cases above. Therefore we are presenting
below the information on data means and variances derived from that distribution. The
smaller the MSE, the better the quality of the interpolation, and a small variance
indicates that a particular interpolation method is “stable and predictable.”
Case (1) – with only introduction of noise (dwf0generator.m):
Interpolation
Method
PDF
(greatest
likelihood)
Cubic splines
Gaussian kernels
Inverse Gaussian
Inverse Gaussian
pde-based kernels
(dw-fp)
Inverse Gaussian
log
59
Mean
Variance
1.07873e-005
2.33697e-011
3.07038e-006
4.57866e-012
5.99986e-006
1.10868e-011
We found that Gaussian kernels provide the best interpolation method among the three.
It has the smallest mean MSE and smallest variance.
Case (2) – with only random data loss effect (dwf0generator.m):
Interpolation
Method
PDF
(greatest
likelihood)
Mean
Variance
Cubic splines
Gaussian kernels
Inverse Gaussian
Inverse Gaussian
pde-based kernels
(dw-fp)
Inverse Gaussian
3.47268e-007
1.87871e-012
3.47785e-007
1.08269e-013
2.76503e-005
4.51363e-007
log
Here we see that the mean of cubic splines is a little smaller than Gaussian kernels but
Gaussian kernels has a much smaller variance than cubic splines. Depending on which
parameter is more important to us, we can make either cubic splines or Gaussian kernels
as our interpolation method of choice.
c) Case (3) – with both noise and random data loss effect (dwf0generator.m):
Interpolation
Method
PDF
(greatest
likelihood)
Mean
Variance
Cubic splines
Gaussian kernels
Inverse Gaussian
Inverse Gaussian
pde-based kernels
(dw-fp)
Inverse Gaussian
1.37327e-005
8.75105e-011
9.23511e-006
9.6286e-011
2.9955e-004
9.23499e-007
log
Here we have a reverse situation, in which Gaussian kernels has a smaller mean than
cubic splines but cubic splines has a smaller variance. These two interpolation methods
are comparable.
The next table represents the case of the general double well FP.
Case (3) – with both noise and random data loss effect (gendwFP_f0generator.m):
Interpolation
Method
Cubic splines
Gaussian kernels
pde-based kernels
(general dw-fp)
60
PDF
(greatest
likelihood)
Mean
Variance
Inverse Gaussian
Inverse Gaussian
Inverse Gaussian
1.12009e-005
4.34013e-011
5.11972e-006
2.04982e-011
6.4081e-005
2.37094e-008
log
In this case, Gaussian kernels has the smallest mean and variance. Therefore it is the best
interpolation method here.
C.
General Single Well FP Equation Case:
The following tables describe the best fit parametric PDF for one case of the “damaged”
and interpolated data for the general single well:
Case 1: Noise Introduction MSE Data
By comparing the value of Log Likelihood, mean, and variance of each Distribution from
the following three tables when Noise introduced, we observe that the Gaussian Kernels
Method of the Inverse Gaussian Distribution is the best fit out of any other cases.
Case 1.1: Spline MSE data with noise introduction
Distribution
Inverse Gaussian
Log normal
Birnbaum Saunders
Plot
Log
Likelihoood
12214.8
11300
11296.7
Parameters
 = 9.28842e-006
 = 1.07132e-011
 = 9.28842e-006
 = 1.07132e-011
 =9.28773e-006
 =1.06434e-011
Comparing the Log Likelihood, mean, variance of Spline MSE data from the above three
distributions, we observe that the Log Likelihood for Inverse Gaussian Distribution is the
proper distribution to choose to fit our Spline mse histogram.
61
Case 1.2: PDE-Based Kernels MSE data with noise introduction
Distribution
Inverse Gaussian
Birnbaum Saunders
Log Normal
Plot
Log
Likelihoood
Parameters
12799.3
 = 4.09529e-006
 = 3.82547e-012
11880.3
 = 4.09544e-006
 = 3.75468e-012
11880.8
 = 4.09445e-006
 = 3.83061e-012
If we compare the Log Likelihood of PDE-Based MSE data for the above three
distributions, we observe the Log Likelihood for Inverse Gaussian Distribution is the best
one to fit our data.
Case 1.3: Gaussian Kernels MSE data with Noise introduction
Distribution
Inverse Gaussian
Birnbaum Saunders
Log Normal
Plot
62
Log
Likelihoood
Parameters
13287.8
 = 2.07952e-006
 = 1.74997e-012
12369.9


= 2.07912e-006
= 1.6675e-012
12372.1


= 2.08383e-006
= 1.75971e-012
Similarly, the Log Likelihood of Inverse Gaussian when both effects introduced by using
Gaussian Method to fix our data, it turns out that Inverse Gaussian is the best fit than any
other distribution.
Case 2: Data Loss Randomly MSE Data
However, when comparing the value of Log Likelihood, mean, and variance for PDEbased Kernels of Inverse Gaussian from the following three tables is the largest and
smallest respectably when Data Loss Effect introduced.
Case 2.1: Spline MSE data with Data loss effects
Distribution
Inverse Gaussian
Log normal
Birnbaum Saunders
Plot
Log
Likelihoood
17111.4
16293.2
Parameters
 = 1.02272e-007  = 8.00528e-008
 = 4.08286e-013
 = 5.47679e-013
15554.5
 = .000260932
 = 3.38342e-007
Similarly, the biggest and best Log Likelihood from the table above is the Inverse
Gaussian distribution to choose to fit our data.
Case 2.2: PDE-Based Kernels MSE data with Data loss effects
Distribution
Inverse Gaussian
Log Normal
Birnbaum Saunders
63
Plot
Log
Likelihoood
26833.7
25652.1
25232
Parameters
 = 3.88889e-008
 =7.37588e-010
 = 1.37752e-010
 = 1.30751e-015
 = 6.27058
 = 196.601
Case 2.3: Gaussian Kernels MSE data with Data loss effects
Distribution
Inverse Gaussian
Birnbaum Saunders
Log Normal
Plot
Log
Likelihoood
16541.8
14582.4
15705
Parameters
 = 8.59491e-008
 = 4.66119e-015
 =9.53102e-006
 =3.80953e-010
 =7.21728e-008
 = 2.35786e-015
If we compare the Log Likelihood of Spline MSE data for the above three distributions,
we observe that the Log Likelihood for Inverse Gaussian Distribution is the better one to
choose.
Case 3: Both Effects MSE Data
64
On the other hand, it turns out the Gaussian Kernels is the best fit when both effects
introduced because of comparing the value of Log likelihood, mean, and variance of each
distribution for each table below.
Case 3.1: Spline MSE data when both effects introduced
Distribution
Inverse Gaussian
Birnbaum Saunders
Log Normal
Plot
Log
Likelihoood
11587.6
10663.2
10679
Parameters





= 1.21327e-005
= 5.07316e-011
= 1.21426e-005
= 4.89606e-011

= 1.19823e-005
= 4.8081e-011
Again, if we compare the Log Likelihood of Spline MSE data for the above three
distributions, we observe that the Log Likelihood for Inverse Gaussian Distribution is the
best one to choose to fit our Spline MSE data.
Case 3.2: PDE-Based Kernels MSE data when both effects introduced
Distribution
Inverse Gaussian
Log Normal
Birnbaum Saunders
Plot
Log
Likelihoood
9478.96
8482.62
8454.25
65
Parameters
 = 0.000132523
 = 1.2847e-007

= 0.000101144
 = 8.08573e-008


= 0.000150998
= 5.93245e-008
If we compare the Log Likelihood of PDE-Based kernel MSE data for the above three
distributions, we observe that the Log Likelihood for Inverse Gaussian Distribution is the
best because it has the largest value.
Case 3.3: Gaussian Kernels MSE data when both effects introduced
Distribution
Inverse Gaussian
Birnbaum Saunders
Log Normal
Plot
Log
Likelihoood
12285.1
11349.6
11377
Parameters




 = 4.95279e-006
 = 1.73939e-011
= 5.14872e-006
= 1.97119e-011
= 5.17159e-006
= 1.77073e-011
Same situation occurs for Gaussian Kernels when both effects introduced.
V.
Kernel-based Solutions of the Fokker-Planck family of Equations
1.
Standard FP, single well case
1.1.
Description
We are solving the original Fokker-Planck equation with single-well parabolic potential
in the specified time/space domain of [-40,40] by [0,5]. The domain is uniformly
subdivided into a mesh of (x, t) points.
In part IV of the report we demonstrated procedures that allow noisy and incomplete
initial data can be “repaired” using several techniques. As a result, a new initial
condition function f0 has been generated for the subsequent use in the program,
swfp_f0.m. The boundary conditions are set to zero. This program plays only supporting
66
role. It is called inside of “sim_swFP_pdf_surfacef0.m” to produce a small one time-step
forward (size depends on the number of points in the partitioning of the time interval),
thus moving us into interior of the domain. This step is based upon observation that pdebased kernels are very good at interpolating the corresponding pde solution, but not
necessarily so for the initial condition of the problem.
Now, we are ready to generate the kernel surface for the single-well FP equation that, as
we know, describes evolution of the initial PDF.
It is done within
sim_swFP_pdf_surfacef0.m” program.
1.2.
Experiment
(a) In part III we simulated kernel for the single-well Fokker-Planck equation, that for a
specific time slice looks like the following figure:
We specify the larger domain, X1, over which the partial differential equation was solved.
Xi’s are determined by the interval [-40,40]. We picked the smaller subinterval of
[-40,40] i.e.[-20,20] to solve the least square problem. We recomputed the endpoints of
this subinterval and tweaked so as to be values that actually occur in array X1. Then we
create a single-well Fokker-Planck kernel matrix that is made up of the elements of
simulated kernel at a specific time in the subinterval[-20,20].
Our goal is to generate approximate kernel-based pdf surface and compare it with the
original surface produced by the Matlab pdepe solver. Using the above matrix we find
the specific pde kernel surface as a weighted sum of the kernels. Then we eliminate nontypical values (negative values attributable to Matlab handling of very small numbers) to
finally generate a proper probability density function. It is presented in the following
figure (see “swFPf0_S05” directory, program “sim_swFP_pdf_surfacef0.m”:
67
figure # PDF of approximating surface (swfp_pdf.mat)
For comparison we compute Mean Square Errors (MSEs) for the difference of the kernel
generated and swfp_f0.m generated surfaces at all (x, t) points.
The mse varies at every run of the “sim_swFP_pdf_surfacef0.m” program (due to
random effects involved in f0 construction). For the figure above the
average mse = 2.927810-012 a very good fit.
(b)The “sim_swFPconvergencef0.m” program was designed to demonstrate the
convergence of the Kernel Method we are using here. Below is an excerpt from that
program that illustrates the typical kernel procedure performed for several (decreasing)
values of the regularization parameter epsilon at one time-slice:
for p = pstart:pend
where pstart=1 and pend=11
e = 10^(-p+1);
M = (e*I+ P' * P); where P is the Fokker-Planck Kernel Matrix
c = M\(P' * y2);
f(p, :) = (P*c)';
end
Thus generated sequence of kernel functions f(p,x) are compared to the expected
(perfect) one made by pdepe solver. Only four initial kernel functions are presented in
the figure below, since they become visually indistinguishable from the perfect one,
although the MSEs are still computed for all cases for analysis:
68
p = 1, mse = 4.7896e-005
p = 3, mse = 6.6039e-009
p = 5, mse = 6.6701e-013
p = 7, mse = 6.6709e-017
p = 9, mse = 6.6709e-021
p = 11, mse = 6.6709e-025
p = 2, mse = 6.3138e-007
p = 4, mse = 6.6631e-011
p = 6, mse = 6.6709e-015
p = 8, mse = 6.6709e-019
p = 10, mse = 6.6709e-023
We can see that for a sufficiently small value of the regularization constant the Kernel
Method produces a high quality approximation to the solution of the standard single-well
Fokker-Planck Equation.
69
2.
Standard FP, double well case
2.1.
Description
As in 1.1., we are solving the original Fokker-Planck equation with double well parabolic
potential in the specified time/space domain of [-40,40] by [0,5]. The domain is
uniformly subdivided into a mesh of (x, t) points.
In part IV of the report we demonstrated procedures that allow noisy and incomplete
initial data can be “reconstructed” using several techniques. As a result, a new initial
condition function f0 has been generated for the subsequent use in the program,
dwfp_f0.m. The boundary conditions are set to zero. This program plays only
supporting role. It is called inside of “sim_dwFP_pdf_surfacef0.m” to produce a small
one time-step forward (size depends on the number of points in the partitioning of the
time interval), thus moving us into interior of the domain. This step is based upon
observation that pde-based kernels are very good at interpolating the corresponding pde
solution, but not necessarily so for the initial condition of the problem.
Now, we are ready to generate the kernel surface for the double well FP equation that, as
we know, describes evolution of the initial PDF. It is done within
“sim_dwFP_pdf_surfacef0.m” program.
2..2.
Experiment
(a) Let us recall that in part III we simulated kernel for the double-well Fokker-Planck
equation, that for a specific time slice looks like the following figure:
Similar to the single well case above we specify the larger domain, X1, over which the
partial differential equation was solved. Xi’s are determined by the interval [-40,40]. We
picked the smaller subinterval of [-40,40] i.e.[-20,20] to solve the least square problem.
70
We recomputed the endpoints of this subinterval and tweaked so as to be values that
actually occur in array X1. Then we create a double-well Fokker-Planck kernel matrix
that is made up of the elements of simulated kernel at a specific time in the subinterval[20,20].
Our goal is to generate approximate kernel-based pdf surface and compare it with the
original surface produced by the Matlab pdepe solver. Using the above matrix we find
the specific pde kernel surface as a weighted sum of the kernels. Then we eliminate nontypical values (negative values attributable to Matlab handling of very small numbers) to
finally generate a proper probability density function. It is presented in the following
figure (see “dwFPf0_S05” directory, program “sim_dwFP_pdf_surfacef0.m”:
figure # PDF of approximating surface (dwfp_pdf.mat)
For comparison we compute Mean Square Errors (MSEs) for the difference of the kernel
generated and dwfp_f0.m generated surfaces at all (x, t) points. The mse varies at every
run of the “sim_dwFP_pdf_surfacef0.m” program (due to random effects involved in f0
construction). For the figure above the average mse = 4.6205e-010 - a reasonably good
fit.
(b) The “sim_dwFPconvergencef0.m” program was designed to demonstrate the
convergence of the Kernel Method we are using here. Below is an excerpt from that
program that illustrates the typical kernel procedure performed for several (decreasing)
values of the regularization parameter epsilon at one time-slice:
for p = pstart:pend
where pstart=1 and pend=11
e = 10^(-p+1);
M = (e*I+ P' * P); where P is the Fokker-Planck Kernel Matrix
c = M\(P' * y2);
f(p, :) = (P*c)';
end
71
Thus generated sequence of kernel functions f(p,x) is compared to the expected (perfect)
one made by pdepe solver. Only four initial kernel functions are presented in the figure
below, since they become visually indistinguishable from the perfect one, although the
MSEs are still computed for all cases for analysis:
p = 1, mse = 4.1213e-005
p = 3, mse = 2.2258e-008
p = 5, mse = 2.278e-011
p = 7, mse = 8.738e-012
p = 9, mse = 7.4786e-012
p = 11, mse = 6.764e-012
p = 2, mse = 1.049e-006
p = 4, mse = 5.4999e-010
p = 6, mse = 1.0263e-011
p = 8, mse = 7.9939e-012
p = 10, mse = 7.087e-012
We can see that for a sufficiently small value of the regularization constant the Kernel
Method produces a high quality approximation to the solution of the standard double-well
Fokker-Planck Equation.
72
General FP, single-well case
3.1.
Description
As in 3.1., we are solving the General Fokker-Planck equation with single well parabolic
potential in the specified time/space domain of [-40,40] by [0,5]. The domain is
uniformly subdivided into a mesh of (x, t) points.
In part IV of the report we demonstrated procedures that allow noisy and incomplete
initial data can be “reconstructed” using several techniques. As a result, a new initial
condition function f0 has been generated for the subsequent use in the program,
genswfp_f0.m. The boundary conditions are set to zero. This program plays only
supporting role. It is called inside of “sim_genswFPextrapol_surface.m” to produce a
small one time-step forward (size depends on the number of points in the partitioning of
the time interval), thus moving us into interior of the domain. This step is based upon
observation that pde-based kernels are very good at interpolating the corresponding pde
solution, but not necessarily so for the initial condition of the problem.
Now, we are ready to generate the kernel surface for the general single well FP equation
that, as we know, describes evolution of the initial PDF. It is done within
“sim_genswFPextrapol_surface.m” program.
3.2.
Experiment
(a) Let us recall that in part III we simulated kernel for the single-well Fokker-Planck
equation, that for a specific time slice looks like the following figure:
Similar to the regular single well case above we specify the larger domain, X 1, over
which the partial differential equation was solved. Xi’s are determined by the interval
[-40,40]. We picked the smaller subinterval of [-40,40] i.e.[-20,20] to solve the least
square problem. We recomputed the endpoints of this subinterval and tweaked so as to be
values that actually occur in array X1. Then we create a general single-well Fokker73
Planck kernel matrix that is made up of the elements of simulated kernel at a specific
time in the subinterval[-20,20].
Our goal is to generate approximate kernel-based pdf surface and compare it with the
original surface produced by the Matlab pdepe solver. Using the above matrix we find
the specific pde kernel surface as a weighted sum of the kernels. Then we eliminate nontypical values (negative values attributable to Matlab handling of very small numbers) to
finally generate a proper probability density function. It is presented in the following
figure (see “genFPsw_f0!!” directory, program “sim_genswFPextrapol_surface.m”:
figure # PDF of approximating surface (genswFP_pdf.mat)
For comparison we compute Mean Square Errors (MSEs) for the difference of the kernel
generated and genswfp_f0.m generated surfaces at all (x, t) points. The mse varies at
every run of the “sim_genswFPextrapol_surface.m” program (due to random effects
involved in f0 construction). For the figure above the
average mse = 5.9234e-011- a reasonably good fit.
(b) The “sim_genswFPconvergence.m” program was designed to demonstrate the
convergence of the Kernel Method we are using here. Below is an excerpt from that
program that illustrates the typical kernel procedure performed for several (decreasing)
values of the regularization parameter epsilon at one time-slice:
for p = pstart:pend
e = 10^(-p+1);
M = (e*I+ P' * P);
c = M\(P' * y2);
f(p, :) = (P*c)';
end
where pstart=1 and pend=11
where P is the Fokker-Planck Kernel Matrix
74
Thus generated sequence of kernel functions f(p,x) is compared to the expected (perfect)
one made by pdepe solver. Only four initial kernel functions are presented in the figure
below, since they become visually indistinguishable from the perfect one, although the
MSEs are still computed for all cases for analysis:
p = 1, mse = 3.7259e-005
p = 3, mse = 5.0987e-009
p = 5, mse = 6.4482e-013
p = 7, mse = 7.2931e-015
p = 9, mse = 6.5783e-016
p = 11, mse = 8.3726e-017
p = 2, mse = 4.9223e-007
p = 4, mse = 5.1827e-011
p = 6, mse = 3.6038e-014
p = 8, mse = 2.1225e-015
p = 10, mse = 2.4589e-016
We can see that for a sufficiently small value of the regularization constant the Kernel
Method produces a high quality approximation to the solution of the general single-well
Fokker-Planck Equation.
75
General FP, double well case
4.1.
Description
As in 1.1.and 1.2, we are solving the General Fokker-Planck equation with double well
parabolic potential in the specified time/space domain of [-40,40] by [0,5]. The domain is
uniformly subdivided into a mesh of (x, t) points.
In part IV of the report we demonstrated procedures that allow noisy and incomplete
initial data can be “reconstructed” using several techniques. As a result, a new initial
condition function f0 has been generated for the subsequent use in the program,
gendwfp_f0.m. The boundary conditions are set to zero. This program plays only
supporting role. It is called inside of “sim_gendwFPextrapol_surface.m” to produce a
small one time-step forward (size depends on the number of points in the partitioning of
the time interval), thus moving us into interior of the domain. This step is based upon
observation that pde-based kernels are very good at interpolating the corresponding pde
solution, but not necessarily so for the initial condition of the problem.
Now, we are ready to generate the kernel surface for the general double well FP equation
that, as we know, describes evolution of the initial PDF. It is done within
“sim_gendwFPextrapol_surface.m” program.
4.2.
Experiment
(a) Let us recall that in part III we simulated kernel for the double-well Fokker-Planck
equation, that for a specific time slice looks like the following figure:
Similar to the single well case above we specify the larger domain, X1, over which the
partial differential equation was solved. Xi’s are determined by the interval
[-40,40]. We picked the smaller subinterval of [-40,40] i.e.[-20,20] to solve the least
square problem. We recomputed the endpoints of this subinterval and tweaked so as to be
76
values that actually occur in array X1. Then we create a general double-well FokkerPlanck kernel matrix that is made up of the elements of simulated kernel at a specific
time in the subinterval[-20,20].
Our goal is to generate approximate kernel-based pdf surface and compare it with the
original surface produced by the Matlab pdepe solver. Using the above matrix we find
the specific pde kernel surface as a weighted sum of the kernels. Then we eliminate nontypical values (negative values attributable to Matlab handling of very small numbers) to
finally generate a proper probability density function. It is presented in the following
figure (see “genFPdw_f0!!” directory, program “sim_gendwFPextrapol_surface.m”:
figure # PDF of approximating surface (gendwFP_pdf.mat)
For comparison we compute Mean Square Errors (MSEs) for the difference of the kernel
generated and gendwfp_f0.m generated surfaces at all (x, t) points. The mse varies at
every run of the “sim_gendwFPextrapol_surface.m” program (due to random effects
involved in f0 construction). For the figure above the
average mse = 3.6622e-012- a reasonably good fit.
(b) The “sim_gendwFPconvergence.m” program was designed to demonstrate the
convergence of the Kernel Method we are using here. Below is an excerpt from that
program that illustrates the typical kernel procedure performed for several (decreasing)
values of the regularization parameter epsilon at one time-slice:
for p = pstart:pend
e = 10^(-p+1);
M = (e*I+ P' * P);
c = M\(P' * y2);
f(p, :) = (P*c)';
end
where pstart=1 and pend=11
where P is the Fokker-Planck Kernel Matrix
77
Thus generated sequence of kernel functions f(p,x) is compared to the expected (perfect)
one made by pdepe solver. Only four initial kernel functions are presented in the figure
below, since they become visually indistinguishable from the perfect one, although the
MSEs are still computed for all cases for analysis:
p = 1, mse = 3.5865e-005
p = 3, mse = 4.9059e-009
p = 5, mse = 5.5442e-013
p = 7, mse = 4.4557e-015
p = 9, mse = 3.2343e-016
p = 11, mse = 7.1029e-017
p = 2, mse = 4.7513e-007
p = 4, mse = 4.9521e-011
p = 6, mse = 1.9504e-014
p = 8, mse = 1.2318e-015
p = 10, mse = 1.359e-016
We can see that for a sufficiently small value of the regularization constant the Kernel
Method produces a high quality approximation to the solution of the general double-well
Fokker-Planck Equation.
VI.
Eigenfunction expansion – numerical experiments and analysis
A. derivation
An eigenfunction solution to the standard 1-Dimensional Fokker-Planck equation (in the
infinite domain with natural boundary conditions) was derived using a series of
substitutions.
u    ( xu)  D 2u
(1)
2
t
x
x
78
See report pp. xx-xx for a more complete documentation, and note the substitution

k2 
.
2D
The initial condition of the 1-D F-P problem is u0 ( x)  u( x,0) , and solutions are
functions u ( x, t ) , where u is a probability density function (PDF) of matter located
along one dimension.
A more general form of the solution to this PDE, known as a Green’s function G ( x, t; x ') ,
has the advantage that given an initial condition u0 ( x ) , a particular solution u ( x, t ) can
be obtained by using the equation
u( x, t )   u0 ( x ')G ( x, t; x ')dx '
(2)
The Green’s function found via eigenfunction expansion is
W ( x, t; x ')  ke  kx 
2

 (2 n !
 ) 1 H n (kx) H n (kx )e  nt
n
n 0
(3)
An equivalent formulation of the Green’s function derived differently, is given by
1/ 2



P( x, t; x ')  
2 t 
 2 D(1  e ) 
exp( 
 ( x  e  t x)2
2 D(1  e 2 t )
)
(4)
These two formulations of the Green’s function were implemented in MATLAB, and
compared. Then, using one initial condition, eqn (2) was implemented with both
Green’s functions, and compared to the particular solution to the F-P eqn (1) found
directly by numerical means.
B. implementation
The 1-D Fokker-Planck problem is defined on the infinite domain ( x  ), but for our
purposes we set our domain to x  [20, 20] , and t  (0,5] . In MATLAB, x (thus x’ as
well) is represented as a linearly spaced vector. Each of the Green’s functions W(x,t;x’)
and P(x,t;x’) at one value of t is represented as an Lx  Lx matrix, where Lx , Lx
represent the lengths of the vectors x and x’ respectively. Of course, Lx  Lx since they
are identical vectors. The matrix W then, for example, is eqn (3) computed for every
combination of x and x’ at one time t. Matrix operations were used as much as possible
to maximize speed.
Note: Computation of the Green’s functions for t  0 is unstable.
79
Consider that at t = 0, u0 ( x )  u( x,0)   u0 ( x ')G ( x,0; x ')dx ' .
Thus, G ( x,0; x ')   ( x  x ') .
Dirac’s  -function is a theoretical construct, and attempting to represent it as a matrix of
finite numbers is impossible because it involves infinities. As a result, implementing any
formulation of a Green’s function is problematic for small values of t.
Green’s Function P
The implementation of P( x, t; x ') is fairly straightforward, and presents no computational
obstacles except for that noted above. The algorithm is included as MP.m, and is used as
an ideal to compare the W formulation with.
Green’s Function W
Implementing W ( x, t; x ') is more difficult than it initially appears. Obviously we cannot
compute an infinte number of terms, so the approximation must be
WN ( x, t; x ') 
k

e  kx 
2
H n (kx ) H n (kx)  nt
e
2n n !
n 0
N

(5)
where N is a suitable number of terms.
Computing the terms of the sequence poses a numerical problem. The 2n n ! denominator
becomes large for relatively small values of n, and so does the product of n-th degree
Hermite polynomials H n ( x ) H n ( x) .
Computed directly, both terms become
unmanagably large (Inf) for unacceptably small values of n (n > 150) . Thus, for larger
N, the sum is dominated by useless Inf (Infinity) and NaN (Not a Number, caused by Inf
H n ( x ) H n ( x)
/ Inf). However, the entire quotient
need not be very large. In fact, it
2n n !
must not because the series is theoretically convergent.
The solution to that numerical difficulty follows from the fact that it can be defined
recursively. Observe that for Hermite polynomials,
H 0 ( x)  1,
H1 ( x)  2 x,
H n1 ( x)  2 xH n ( x)  2nH n1 ( x)
By splitting up 2n n ! evenly we can define two formulas A and B such that for n  2k ,
80
An 1 ( x ) 
H n 1 ( x )
2 (1  3  n  1)
An ( x ) 
H n ( x)
2 (1  3  n  1)
An 1 ( x ) 
H n 1 ( x )
2 (1  3  n  1)
k 1
k
k
An ( x ) Bn ( x ') 
Clearly, for n  3 ,
Bn 1 ( x) 
H n 1 ( x)
2 (2  4  n  2)
k
Bn ( x) 
Bn 1 ( x) 
H n ( x)
2 (2  4  n)
k
H n 1 ( x)
2 (2  4  n)
k 1
H n ( x ) H n ( x)
2n n !
(6)
Subsituting into the formula for H n , we obtain the recursive relations
A0  1, A1  2 x
B0  1, B1  x
An  xAn 1  An 2
2
 n 1
Bn    xBn 1  
 Bn 2
n
 n 
 2 
 n 
An 1  
 xAn  
 An 1
 n 1
 n 1
Bn 1  xBn  Bn 1
provided that n is even!
Figure 1.
To see that the the stability of our desired product is not limitedby n, observe the plot of
An (20) Bn (20) vs n in figure 1.
81
Alternatively, we can include the e  nt in the recursive formulas as well, which may yield
an even better behaved solution. Then A and B are defined similarly to A and B above
except
H ( x)
H ( x)
An ( x )  k  kt n
B n ( x)  k  kt n
2 e (1  3  n  1)
2 e (2  4  n)
produce
An ( x ) B n ( x ') 
H n ( x ) H n ( x)  nt
e
2n n !
The recursion in that case is given by
B 0  1, B1  cx
A0  1, A1  2 x

An  c x An 1  An 2

 2 
 n 
An 1  
 x An  c 
 An 1
 n 1
 n 1
2
 n 1
B n    xB n 1  c 
 B n 2
n
 n 

B n 1  c xB n  B n 1

where c  e  t .
Now W can be expressed as
k  x2 N
WN ( x, t; x ') 
e  An ( x ) Bn ( x)e  nt

n 0
or
k  x2 N
WN ( x, t; x ') 
e  An ( x ) B n ( x)

n 0
(7)
(8)
Two MATLAB algorithms for W are included in this report. All of the analysis in the
following section was done using MW.m ( via eqn(7) ). In MW.m, A and B are
computed as a Lx  N and a Lx  N matrix. The e  nt term is created as a Lx  N
matrix, and multiplied elementwise with B , resulting in B . Then, the sum is computed
as the matrix product AT B . The two large matrices involved can pose memory problems.
On the computer used for the analysis, N = 10000, with 101 data points ( Lx  101 ) was
about the limit. Due the the following numerical issue however, using that many terms is
generally not only unnecessary, but ineffective.
e  nt  0 , for  nt  745 as far as MATLAB is concerned. (The smallest number
representable by double-precision floating point is 21074 , “subnormal” according to the
IEEE standard.)
82
1490
no longer improves the approximation, and
t
also indicates thast there must be an absolute minimum achievable error.
Thus, for our purposes (   0.5 ), N 
The other algorithm for W is included as MWalt.m, and based on eqn(8). It uses the
recursions A and B , so e  nt does not need to be computed separately. Also, the sum is
is cumulative instead of being incorporated into a matrix multiplication, eliminating the
memory problems associated with keeping large matrices. MWalt.m runs slow, since the
entire N-step recursion must be re-computed for each value of t. Also, it did not yield
significantly different results.
The quotient (6) is generally decreasing, so it makes sense that e  nt ultimately dictates
where the sum gets truncated, in both cases.
For t > 0,
N max (t )
1490
t
(9)
is the maximum effective number of terms, if double-precision floating point numbers
are used.
Obtaining a particular solution
The L2 inner product of a Green’s function with an initial condition to produce a
particular solution (2), is implemented as a loop over a range of t. For each ti , we
generate a Green’s function matrix G (P or W).
u( x, ti )   u0 ( x ')G( x, ti ; x ')dx '
can be expressed discretely as u( xi , t )   u0 ( xj )G( xi , t; xj )x , so it can be implemented
j
using the trapezoid method in MATLAB, or for faster computation, a matrix product
u( x, t )  Gu0x (with a slight scalar error). The included code uses the trapezoid
method.
The result is a Lt  Lx matrix. The algorithms are included as UP.m and UW.m.
83
C. analysis
For all of the following analysis, the domains x and x’ are both 101 linearly spaced points
(100 intervals) between from -20 to 20. Strictly speaking, t  (0,5] . For most analysis
however, a lower limit of t = 0.001 was sufficiently close to 0 to observe the trend.
Since the Green’s functions’ behavior was more interesting at low t, a logarithmic time
spacing was used for some analysis. W(n) or WN will be our notation for the Green’s
function W computed using n terms.
Figure 2. P
Figure 3. W(1000)
Figure 4.
W(1000)  P
Figures 2 and 3 are 3-D plots of Green’s functions P and W(1000) at t = 0.4. Notice that
on average, W and P look similar, but radically differ over a relatively small range. This
was found to be the general pattern of difference between W and P, more so at lower t
and n. Considering P to be the ideal case, we originally employed two types of error
measurement, which we presumed, would give us a better idea of the nature of the error
than one :
mse (mean squared error) gives an indication of the error on average
2
1
mse(n, t )  2  Wn ( xi , t; xj )  P( xi , t; xj ) 
(10)
Lx j i
sup error is the maximumum error over all compared points
84

sup( n, t )  max max Wn ( xi , t; xj )  P( xi , t; xj )
j
i

Experimentation yielded almost identical plots for sup error and mse, so only mse is
considered here. The error is plotted on a logarithmic scale.
Green’s Function Error over t for fixed N
Figure 5.
Figure 6.
When fixing values of n, we see that mse converges to a single t-dependent path.
Generally, for higher values of n, the error converges to the path earlier. Using criteria
described later, the apparent ideal path to which the errors converge is shown in figure 7.
Figure 7.
85
Green’s Function Error over N for fixed t
Figure 8.
Figure 9.
From figures 8 and 9 it is apparent that for each t there are thresholds for n, above which
mse does not change significantly. The circles indicate n values after which 10
consecutive sampled values of log10 (mse) change by a factor of less than 103 (or
maximum n checked). It is evident that a certain minimum error, depending on t, is to be
expected, no matter how many terms are used in computing W, but the number of terms
required to reach that critical point clearly falls short of the maximum N max (t ) established
in eqn(9). The same criteria that placed the circles in figures 8 and 9 determined the
points for figure 10. Figure 10 shows a comparison of the experimental maximum
effective N and the predicted N max (t ) (on a log scale) over t  [0.01,5] . The reason for
the almost constant difference is unclear, but could be revealed by further investigation.
86
Using a linear fit similar to that shown in figure 10, with 300 data points, a relation for
the
Figure 10.
experimentally determined maximum effective N, (which we shall call N best ) was found
to be
N best (t ) 102.151 t 0.851
(11)
142
 0.851
t
Something resembling an error estimate for can be derived from (3) and (5). Consider
the error RN ( x, t; x) , where W  WN  RN .
Then
RN  ke  kx 
2

 (2 n !
n
n  N 1
 ) 1 H n (kx) H n ( kx )e  nt
Expressed in terms of the orthonormal eigenfunctions  n ( x ) (Bem: reference to eqn 10
from eigenfunction handout),
87
RN  ke
1
 (( kx )2 ( kx  )2 ) 
2

N 1
RN 2  k 2e  (( kx )
( kx )
e
2
( kx  )2

2
( kx  )2 )

n
(kx ) n ( kx )e  nt


N 1 N 1
m
( kx ) m ( kx) n ( kx ) n ( kx )e  ( m  n ) t

RN 2 dx  k 2  e  ( m  n ) t m ( kx ) n ( kx )   m ( kx ) n ( kx )dx 
N 1 N 1

 k 2  e 2 nt n 2 ( kx )
N 1
( kx )
 e
2
2
( kx  )2

RN 2dxdx  k 2  e 2 nt   n 2 (kx )dx
N 1

 k 2  e 2 nt
N 1
 k2
e 2 ( N 1) t
1  e 2  t
Then
e( kx )
2
( kx )2
RN 2
L2
 2
k
e  ( N 1) t
1  e 2 t
(12)
88
Eqn(12) is interesting because the left side resembles the equation(10) for mse(n,t). It
also suggests an exponential relation between the product Nt and mse, which was
confirmed experimentally by data in figure 11.
Figure 11.
Finally, the particular solutions u ( x, t ) obtained via Green’s functions P and W are
compared in figures 12 and 13. The plots of mse vs t look similar to those of the Green’s
functions directly compared before, except that convergence to a minimal mse happens
earlier.
PDE Solution Error over t for fixed N
Figure 12.
Figure 13.
89
D. Conclusion
The investigation of the eigenfunction expansion formulation of the Green’s function for
(1) has yielded some discoveries of interest from a numerical analysis standpoint.
Having stated at the beginning that Green’s function computation is unstable for ultra low
values of t, it is no surprise that for t approaching zero, computation of the eigenfunction
expansion Green’s function (W) requires exponentially more terms, becoming
impractical; but for t > 0.1, the two Green’s functions are almost identical.
Thus, predicting the behavior of a system u ( x, t ) directly after the initial condition (0 < t
< 0.1) is problematic with the Green’s function methods and more so via the
eigenfunction expansion, but for mid to long term behavior, they produce equivalent
results, with an added advantage: solution of the Fokker-Planck PDE via Green’s
function method is faster, because solutions can be obtained explicitly. When we are not
concerned with events directly after init condtion, it is more convenient than numerical
methods such as finite differences.
VII
The “cloud evolution” demo
1. Jointly distributed random variables and marginal distributions.
In the previous sections we outlined methods suitable for the one-dimensional pdf
evolution controlled by one of the four Fokker-Planck equation cases (in discrete case pdf
is replaced by pmf - probability mass function). In the “cloud evolution” experiment that
follows below we shall treat x and y coordinates of the points, where objects of interest
are observed, as dependent random variables (dependency based on the geometry of the
cloud). Therefore the computed joint probability mass function f(x,y) is a product of a
marginal pmf, say f(y), by conditional f(x|y) ([9]):
f ( x, y )  f ( y ) f ( x | y )
Thus observing separately time-evolution of f(y) and f(x|y, described by the FP
equations of our choice, we will be able to reconstruct the 2-d pmf as a function of
(x,y,t).
2. Initial pdf - a spiral
For the “cloud evolution” demo we designed a 2-d set of objects (a cloud) clustered
around the Archimedes Spiral in polar coordinates (see f0pdf_generator.m in the
CloudEvolution directory):
90
Xtheta = 4*theta.*cos(theta);
Ytheta = 4*theta.*sin(theta);
Then, at every (out of 401) point of partitioning we randomly assign 1 to 5 objects to be
located at (or clustered at) that point, with total number of objects being computed as
well.
This allowed to generate 2-d PDF function (f_2d.mat) corresponding to the “cloud”:
F = zeros(length(x), length(y));
for k = 1:length(theta)
for i = 1:length(x)
for j = 1:length(y)
if
(x(i)-h < Xtheta(k)) & ( Xtheta(k) <= x(i)+h ) & ...
(y(j)-h < Ytheta(k)) & ( Ytheta(k)<= y(j)+h)
F(i,j) = F(i,j) + N(k);
% F(x,y)
end
end
end
end
f_2d = F/Ntotal; % 2-dimensional pdf
save f_2d.mat f_2d
3. Marginal PDFs (PMFs) (generation of f(y) and f(x|y)).
After f(y) (denoted f_y in the program) = sum(f_2d) (summation across all x's) was
found, the 2-d density allows calculation of the conditional pmf (denoted fx_y) at all
points of the mesh as follows:
91
for j = 1:length(y)
if f_y(j) == 0
for i = 1:length(x)
fx_y(i,j) = 0; % f of x given y; conditional
end
else
for i = 1:length(x)
fx_y(i,j) = f_2d(i,j)/f_y(j);
end
end
end
The results are saved and will be used as initial data in the next step:
xf0.mat fx_y
yf0.mat f_y
% 201x201 matrix of conditional pdf
% 1x201 vector of marginal y pdf
4. Propagation of f(y) by fp_f0.m – a single-well (regular) FP
The evolution of f(y) can be described by any of the four FP cases studied in the earlier
sections. For the illustration here we are using single-well (regular, D=1) Fokker-Planck
equation. Its solution is generated by fp_f0.m (or the kernel approximation of such
solution by corresponding program in swFPf0_S05 directory) and saved as twodimensional array 201x201 in (y,t) :
sw_Ypdf.mat sw_Ypdf
5. Similarly, propagation of f(x|y) for every fixed y in the partitioning is accomplished
in gendwfp_f0.m – written for a general double-well Fokker-Planck equation (or,
equivalently, found as a kernel approximation; genFPdw_f0 directory). The same
program was run 201 times for each y on the list with the corresponding initial condition
f(x|y), results were assembled and saved as three-dimensional 201x201x101 array in
(x,y,t) :
gendw_Xpdf.mat gendw_Xpdf
Using the last two PDF files, we reconstruct the 2-d pdf function for every time slice
(marked with either m in CloudMovie.m and below, or t_ref in cloud3.m):
for j = 1:length(y1)
z(j,:) = gendw_Xpdf(j,:,m)*sw_Ypdf(j,m);
end
Several slides of the “cloud evolution” PDFs for the time slices t_ref = 1, 2, 12, 24, 54,
94 are assembled below:
92
93
94
95
These figures clearly demonstrate a difference between single and double well cases. We
can observe that probability of finding objects has two points (locations) of concentration
in terms of x components and one in y. Resulting effect is a set of two areas of objects
concentration. There is also more pronounced diffusion (spread) of “matter” observed
along x-axis due to a variable diffusion coefficient in the general FP case, compared to
the regular one in y case.
We demonstrated here how a 1-d FP model can be applied to a 2-d pdf evolution
problem. Similarly, with larger amount of computation, it can be done in a 3-d case.
Project Extension
The logical extension of the project would be investigation of general 3-D Fokker-Planck
equations and simulation of the Green’s kernels for such equations for subsequent
application to the construction of the kernel approximations.
References
1. A.D. Polyanin, Handbook of Linear Partial Differential Equations for Engineers
and Scientists, Chapman&Hall/CRC, 2002.
2. V. S. Vladimirov, Equation of Mathematical Physics, Marcel Dekker, 1971.
3. H. Risken, The Fokker-Planck Equation, Springer, 1984.
4. Fall 2003, Spring 2004 CAMCOS reports.
5. D.H. Griffel, Applied Functional Analysis, Dover, 1985.
6. E. Kamke, Differentialgleichungen, Vol. 2, Leipzig, 1961.
7. C. Cohen-Tannoudji, et al., Quantum Mechanics, I, II, Wiley, 1977.
8. E.R. Hansen, A Table of Series and Products, Prentice-Hall, 1975.
9. J.L Devore, Probability and Statistics, Duxbury, 1999.
96
10. T. Hastie, et al., The Elements of Statistical Learning, Springer, 2001.
97