Computer Simulation of Stochastic Processes and
Visualization with Python
Ihor Lubashevsky
office 226A, p. 0242-37-2565
Email address: i-lubash@u-aizu.ac.jp
2010 Mathematics Subject Classification. Primary
Abstract. Recommended cites for Scientific Python
• Scipy Lecture Notes: http://www.scipy-lectures.org/
• Python Guide → Scenario Guide → Scientific Applications: start from:
http://docs.python-guide.org/en/latest/
Recommended books on Python
• John M. Stewart, Python for Scientists, Cambridge University Press,
Cambridge UK, 2014.
Contents
Lecture 1. What stochastic processes are:
1. Characteristic property of stochastic processes:
Individual unpredictability
2. Characteristic property of stochastic processes:
Statistical predictability
3. Axiomatic theory of probability. Simplified presentation
4. Self-consistency of the intuitive and axiomatic probabilities
5. Probability theory and stochastic processes
6. Comments
7. Exercise
8. Ergodic processes
5
7
9
10
10
10
11
Lecture 2. Random trajectories and their probability
1. Bayes’ formula for probability of complex events
2. Random trajectories: Construction
3. Markov stochastic processes
4. Dynamical random variables
5. Law of Large Numbers
Homework
13
13
14
16
17
19
20
Lecture 3. Fokker-Planck equations
1. Backward Fokker-Planck equation
2. Forward Fokker-Planck equation
3. Boundary conditions for the Fokker-Planck equations
Homework
21
22
23
25
27
Lecture 4. Forward Fokker-Planck Equation: Its applications
1. Systems with thermodynamic equilibrium: Detailed balance
2. Method of Moments
29
29
30
Numerical analysis of diffusion-type equations
3. Ordinary differential equations: initial value problem
4. Partial differential equations: initial value problem
Homework
37
37
37
38
Lecture 5. Backward Fokker-Planck Equation: Applications
1. First passage time problem: Example of one-dimensional stochastic
process
2. Escaping rate from a potential well
3. Probability of extreme events
39
iii
1
4
39
42
44
iv
CONTENTS
Lecture 6. Master equation
1. Chapman-Kolmogorov equation and master equation
2. Ergodic properties
3. Properties of the master equation
4. Kirchhoff method
5. System with detailed balance
6. Dynamical properties of master equation
7. One–Step Processes in Finite Systems
8. Poisson Process in Closed and Open Systems
Stochastic differential equations and noise-induced phase
transitions
1. Types of Stochastic Differential Equations
2. Relationship between different type stochastic differential equations
3. Transformation of Random Variables
4. Forms of the Fokker–Planck Equation
5. Equilibrium and Nonequilibrium Phase Transitions
6. The Verhulst Model of Third Order
7. The Genetic Model
8. Noise Induced Instability in the Geometric Brownian Motion
9. System Dynamics with Stagnation
10. Oscillator with Dynamical Traps
11. Dynamics with Traps in a Chain of Oscillators
12. Self–Freezing Model for Multilane Traffic
47
47
49
50
50
52
55
56
58
Lecture 7.
Bibliography
65
65
68
69
70
72
74
77
77
80
81
84
93
99
LECTURE 1
What stochastic processes are:
In 1827 Scottish botanist Robert Brown (1773–1858), while looking through
a microscope at minor particles ejected from pollen grains of the plant Clarkia
pulchella suspended in water, noted that these particles execute continuous jittery
motion without any visible reason. By repeating this experiment with very small
particles of inorganic materials he was able to rule out that the motion was liferelated.
50
40
y
30
20
Figure 1. Typical trajectory of Brownian motion {x(t), y(t)}.
Initially the Brownian particle was located at the
origin, the red circle
shows its terminal position.
10
0
10
20
10 0
10
x
20
30
40
50
1
2
1. WHAT STOCHASTIC PROCESSES ARE:
Figure 2. Drill-string vibrations as stochastic dynamics of the
field r(s, t) = {x(s, t), y(s, t)} describing the lateral deviation of
point s from the equilibrium vertical position.
Another example of stochastic processes is given in Fig. 2 showing vibrations
of a drill.
In mathematical terms, processes, we will consider, are described as time
variations—dynamics—of some variable x which can comprise N -components, i.e.,
x = {x1 , x2 , . . . , xN }. In other words, we may represent a random process as a motion trajectory of some point x in N -dimensional space RN . If time variations in the
variables x(t) are continuous, the process is called continuous; if these variations
are step-wise, the process is called discrete.
1. WHAT STOCHASTIC PROCESSES ARE:
3
Which process can be categorized as stochastic (random)?
process seeming random
1
random process
1.0
0
1
0.5
3
y2 (t)
y1 (t)
2
0.0
4
0.5
5
6
7
1.0
0
2
4
1
6
8 10 12
time, t
ensemble of 3 elements
14
16
0
2
4
0
2
4
6
8 10 12
time, t
ensemble of 3 elements
14
16
14
16
1.0
0
1
0.5
3
y2 (t)
y1 (t)
2
0.0
4
0.5
5
6
7
1.0
0
2
4
6
8 10
time, t
12
14
16
6
8 10
time, t
12
Figure 3. Two processes with complex dynamics. The process
shown in the left column only seems to be random, whereas the
process in the right column is realy random.
4
1. WHAT STOCHASTIC PROCESSES ARE:
1. Characteristic property of stochastic processes:
Individual unpredictability
For simplicity, let us consider an one-dimensional continuous process x(t) and
assume that we can control all the external factors which are able to influence it. So
it is quite natural to expect that if we reproduce once and once the same conditions,
the dynamics of the variable xi (t) will be also reproduced one-to-one at each trial
i. Nevertheless, as we saw, there are processes that violate this expectation.
25
20
random variable x
15
10
5
0
5
10
15
0
20
40
time
60
80
100
Figure 4. Example of processes affected by uncontrollable factors
in stable environment. Illustration of the individual unpredictability of stochastic (random) processes.
2. CHARACTERISTIC PROPERTY OF STOCHASTIC PROCESSES: STATISTICAL PREDICTABILITY
5
2. Characteristic property of stochastic processes:
Statistical predictability
Let us try to analyze the statistical properties, for example, of the ensemble of
terminal points {Xi },
def
Xi = xi (t)|t=T
for the previous collection of the trajectories {xi (t)}. It can be done using histograms:
30
random variable x
20
10
0
10
20
30
80
time
100
T
Figure 5. Illustration
of histogram construction.
Rather often random variables appear in this way—as a result of some random
process.
Figure 6 shows evolution of the histogram as the number of trajectories grows.
6
1. WHAT STOCHASTIC PROCESSES ARE:
ensemble of 100 elelements
0.06
0.04
frequency, p
frequency, p
0.05
0.03
0.02
0.01
40
0.045
0.040
0.035
0.030
0.025
0.020
0.015
0.010
0.005
0.000
30
20
10 0 10 20 30
random variable X
ensemble of 10000 elelements
40
frequency, p
frequency, p
0.00
40
30
20
10 0 10 20
random variable X
30
40
0.045
0.040
0.035
0.030
0.025
0.020
0.015
0.010
0.005
0.000
0.045
0.040
0.035
0.030
0.025
0.020
0.015
0.010
0.005
0.000
ensemble of 1000 elelements
40
30
40
30
20
10 0 10 20 30
random variable X
ensemble of 100000 elelements
20
10 0 10 20
random variable X
30
40
40
Figure 6. Histogram evolution.
Hypothesis of the limit description: Wrong! Let us try to introduce the
probability pi as the limit of the frequency νi (N ). Is it possible to say that
pi = lim νi (N )
N →∞
?
Cauchy’s definition of limit: For any arbitrary chosen small value there is such
number N that for any ensemble of N > N elements similar in properties the
inequality
νi (N ) − pi < .
holds. It is wrong, I just can construct such an ensemble just selecting elements
with special properties.
3. AXIOMATIC THEORY OF PROBABILITY. SIMPLIFIED PRESENTATION
7
3. Axiomatic theory of probability. Simplified presentation
To overcome the problem arising in the limit description of statistical ensembles Russian mathematician Andrey Kolmogorov (1903–1987) proposed to define
the probability of events in an axiomatic way. Namely, the following object are
introduced:
(1)
• S - the set of elementary events, the states of the analyzed system, all
the possible values taken by a given variable, etc.
• To each element e of the set S a non-negative value p(e) ≥ 0 is ascribed
such that
X
p(e) = 1 .
e∈S
(2)
The value p(e) is called the probability of elementary event e.
• For a set of elements A ⊂ S which may comprise many elements the
probability p(A) is introduced via the formula
X
def
p(A) =
p(e) , p(∅) = 0 .
e∈A
It is not enough to make the probability theory interesting. The given model is just
a trivial theory of set measure.
At the next step it is necessary to describe pairs of these elements (a, b), their
trios (a, b, c), etc. To do this let us consider the notion of conditional probability.
Let A and B be subsets of S, the conditional probability of A given B is defined as
the quotient of the probability of the joint of events A and B, and the probability
of B:
(3)
p(A|B) =
p(A ∩ B)
.
p(B)
The meaning of this definition is illustrated in the Venn diagram shown in Fig. 7
and is known as Bayes’ theorem
Figure 7. The Venn diagram
illustrating the meaning of the
conditional probability.
Now let us consider the Cartesian product of the sets S2 = S×S. Its elements
are the pairs (e1 , e2 ) where e1 , e2 ∈ S. The question is how the probability π(e1 , e2 )
of these pairs ascribed to them as the elements of the set S2 can be related to the
probability of the individual elements p(e). Let us introduce two sets of S2 :
[
[
U1 =
(e1 , e0 ) ,
U2 =
(e0 , e2 ) .
e0 ∈S
e0 ∈S
8
1. WHAT STOCHASTIC PROCESSES ARE:
The probability of complex events U1 and U2 admits interpretation of the probability of meeting e1 or e2 , respectively, when the other element can be any one from
the set S. In this case it is quite natural to set
π(U1 ) = p(e1 ) ,
π(U2 ) = p(e2 ) .
Obviously,
(e1 , e2 ) = U1
\
U2 .
Appealing to intuitive understanding the notion of independence we may say this
probability of meeting element e1 should not depend on the instantiation of the
other element of these pairs, so the equality
π(e1 , e2 )
p(e1 ) = π(U1 |U2 ) =
π(U2 )
has to be the case. Whence it follows immediately that
π(e1 , e2 ) = p(e1 )p(e2 ).
Finally the axiomatic theory of probability ascribes to the elements of the Cartesian
product
SM = {(e1 , e2 , . . . , eM )}
the probability specified by the expression
(4)
π(e1 , e2 , . . . , eM ) = p(e1 ) · p(e2 ) · · · p(eM ) .
Actually expression (4) is the mathematical formulation of the sentence about the
mutual independence the elementary events in the sequence {e1 , e2 , . . . , eM } or their
collection if the order is not essential.
4. SELF-CONSISTENCY OF THE INTUITIVE AND AXIOMATIC PROBABILITIES
9
4. Self-consistency of the intuitive and axiomatic probabilities
Let S = {xi } = {x1 , x2 , . . . , xM } be the set of values that a random variable
e can take with the probabilities {pi }. Then an N -tuple {e1 , e2 , . . . , eN } being the
point of the set SN may be regarded as an elementary instantiation of the ensemble
of N identical in properties, mutually independent random variables e ordered, for
example, according to the time of their appearance—some “weak” version of the
notion of stochastic process. Let P(n1 , n2 , . . . , nM ) denote the probability of the
fact that n1 random variables take the value x1 , n2 variables take the value x2 , and
so on. According to (4) the probability of one elementary state being of this form
is
Y
pni i = pn1 1 · pn2 2 · · · pnMM
i
The total number of such states is
N!
n1 !n2 ! . . . nM !
So the desired probability of the histogram form (n1 , n2 , . . . , nM ) is
Y pni
N!
i
(5)
P(n1 , n2 , . . . , nM ) =
pn1 1 · pn2 2 · · · pnMM = N !
.
n1 !n2 ! . . . nM !
n
i!
i
Let us analyze the behavior of this probability for finite M as N → ∞. To do this
we make use of Stirling’s formula for large values of n (Fig. 8)
n n
√
(6)
n! ∼
= 2πn
e
Figure 8. Factorial, gamma function, and Stirling’s formula [Wiki].
Then substituting (6) into (5) we get
"
N Y r
n #
√
N
1
pi e i
ln P = ln
2πN
e
2πni ni
i
"s
#
X
2πN
pi
Q
= ln
+ N ln N − N +
ni ln
+ ni
ni
i 2πni
i
"s
#
X
2πN
pi
Q
= ln
+ N ln N +
ni ln
ni
i 2πni
i
10
1. WHAT STOCHASTIC PROCESSES ARE:
Introducing the frequency of taking the value xi as
ni
νi =
N
the last equality can be rewritten as
#
"s
X
X
2π
(M − 1)
pi
Q
−
ln P = ln
ln N + N ln N +
N νi ln
−
ni ln N
2πν
2
νi
i
i
i
i
#
"s
X
pi
2π
Q
− (M − 1) ln N +
N νi ln
= ln
2πν
νi
i
i
i
Then within hight accuracy
ln P ≈ −N
X
νi ln
i
νi
pi
.
To find the maximum of the given function provided
X
νi = 1
i
let us use the method of Lagrange multipliers. Namely we consider the function
!
X
X
νi
+λ
νi − 1
F({νi }, λ) = −N
νi ln
pi
i
i
and find its extremum with respect to all the variables {νi } and λ. With respect
to νi it gives
νi
−N ln
− N + λ = 0 ⇒ νi = Cpi .
pi
P
Due to i pi = 1, the constant C = 1. Thereby this function attains its maximum
when νi = pi and the “thickness” of its neighborhood, i.e., individual fluctuations
√
|νi − pi | are located in the region whose thickness decreases with N as 1/ N .
5. Probability theory and stochastic processes
For stochastic processes the set of elementary events S is the set of possible
instantiations of individual trajectories {x(t)}t . In this case the probability of one
trajectory
P ({x(t)}t )
is the basic “brick” of the theory of stochastic (random) processes.
6. Comments
• There is a mathematical difficulty of introducing probability for continuous random trajectories.
• The validity of probability theory is not a question to mathematics but
physics.
7. Exercise
Probability theory is a deep mathematical discipline—just appealing to history.
So it cannot describe everything.
Propose an example of systems for which the notion of probability is not applicable. (Hint: the notion of ergodicity)
8. ERGODIC PROCESSES
11
8. Ergodic processes
12
random variable, e
10
8
6
4
2
0
2
4
0
20
40
60
80
time
Figure 9. Averaging over one trajectory.
A process is ergodic if
(7)
ti
T →∞ T
p(ei ) = lim
In slogan form for ergodic processes:
averaging over ensemble is equivalent to averaging over time.
There are non-ergodic processes.
100
LECTURE 2
Random trajectories and their probability
1. Bayes’ formula for probability of complex events
Let S be the set of elementary events and A be its subset—a complex event.
We want to calculate the probability P (A) of A but directly it is difficult because
of the set complexity. Bayes’ formula suggests a way how to do this via dividing
the complete set S of elementary events into the system of disjoint subsets {Hi }
called hypotheses or conditions, i.e.,
[
Hi ∩ Hi0 = ∅ for i 6= i0 and S =
Hi .
i
Noting that
A=
[
(A ∩ Hi )
i
and using Bayes’ theorem (Fig. 1) we can write
X
(8)
p(A) =
p(A|Hi ) p(Hi ) .
i
Bayes' theorem
Figure 1. Illustration of Bayes’ construction of the probability of
complex events
[Thomas Bayes /’beiz/ (1701–1761) English statistician and philosopher]
Likelihood analysis: If H is a hypothesis and E is an event than
p(H|E)p(E) = p(H ∩ E) = p(E|H)p(H)
=⇒
p(H|E) =
p(E|H)p(H)
.
p(E)
Bayesian inference:—another way of introducing probabilities—it provides a principled way of combining new evidence with prior beliefs through the application of
Bayes’ rule.
13
14
2. RANDOM TRAJECTORIES AND THEIR PROBABILITY
• H stands for any hypothesis whose probability may be affected by data
E—evidence. Often there are competing hypotheses, and the task is to
determine which is the most probable.
• P (H)—the prior probability—is the estimate of the probability of the
hypothesis H before the data E—the current evidence—is observed.
• P (E | H) is the probability of observing E given H. As a function of E
with H fixed, this is the likelihood —it indicates the compatibility of the
evidence with the given hypothesis. The likelihood function is a function
of the evidence E, while the posterior probability is a function of the
hypothesis H.
• P (E) is sometimes termed the marginal likelihood or “model evidence.”
This factor is the same for all possible hypotheses being considered (as is
evident from the fact that the hypothesis H does not appear anywhere in
the symbol, unlike for all the other factors), so this factor does not enter
into determining the relative probabilities of different hypotheses.
Bayes’ rule can also be written as follows:
P (H | E) =
P (E | H)
· P (H)
P (E)
where the factor PP(E|H)
(E) can be interpreted as the impact of E on the probability
of H.
2. Random trajectories: Construction
0
Figure 2. A random trajectory
of T steps in the space S.
Let us consider a set S = {e} of elementary events with the given probabilities
of realization {p0 (e)} and the collection of Cartesian produces {ST } (T is integer),
where each element is
ST = S × S × · · · × S .
|
{z
}
(T + 1) times
A point of the set ST is a random trajectory of T steps in the space S, in other
words, a random trajectory P(T ) is
(9)
P(T ) = (e0 , e1 , e2 , e3 , . . . , eT −1 , eT ) .
Here T may be regarded as (discrete) time. Random trajectories are also often
called random walks. The initial position e0 of the analyzed trajectories {P} may
be treated as given if p(e) = 1 for e = e0 and p(e) = 0 otherwise.
In order to construct the probability P(P) of realization of a trajectory P let
us make use of Bayes’ formula for calculating probability of complex events.
2. RANDOM TRAJECTORIES: CONSTRUCTION
15
At first, let us consider random trajectories with one step, i.e., pairs of points
(e0 , e1 ) ∈ S1 = S × S. It is necessary to understand how we can construct
the probability P{(e0 , e1 )}. Let us introduce a collection of sets—hypothesis or
conditions—H(e0 ) being subset of S1 :
[
def
(10)
H(e0 ) =
(e0 , e01 ) .
e01 ∈S
It is quite natural to define the probability of the complex events {H(e0 )} as
def
(11)
p[H(e0 )] = p0 (e0 ) .
Obviously
H(e0 ) ∩ (e0 , e1 ) = (e0 , e1 ) ,
H(e0 ) ∩ H(e00 ) = ∅ if e0 6= e00 ,
[
H(e00 ) = S1 .
e00 ∈S
So by virtue of Bayes’ formula (8) we can write
P{(e0 , e1 )} = P [e1 |e0 ] · p0 (e0 ) ,
(12)
where by definition
(13)
def
P [e1 |e0 ] = p[(e0 , e1 )|H(e0 )]
is the probability of finding the system at the state e1 if at the previous moment
of (discrete) time it was located at the state e0 . Appealing to physics one has to
specify the conditional probability P [e1 |e0 ]; in our description it is regarded as a
given value reflecting the particular properties of the system in question.
Now we are ready to repeat this method for constructing random trajectories
of 2 steps, they are trios of elements (e0 , e1 , e2 ). By analogy, let us introduce the
subsets H[(e0 , e1 )] of the set S2 as
[
def
(14)
H[(e0 , e1 )] =
(e0 , e1 , e02 ) .
e02 ∈S
Again, the probability of p[H[(e0 , e1 )] is defined as
(15)
def
p[H[(e0 , e1 )]] = P{(e0 , e1 )} .
It should be emphasized that the quantity on the left-hand side of expression (15)
is ascribed to the set S3 whereas one on the right-hand side has been defined for
the points of S3 . Again we have
H[(e0 , e1 )] ∩ (e0 , e1 , e2 ) = (e0 , e1 , e2 ) ,
H[(e0 , e1 )] ∩ H[(e00 , e01 )] = ∅ if e0 6= e00 or e1 6= e01 ,
[
H[(e00 , e01 )] = S2 .
(e00 ,e01 )∈S2
So again by virtue of Bayes’ formula (8) we can write
(16)
P{(e0 , e1 , e2 )} = P [e2 |(e0 , e1 )] · P{(e0 , e1 )} ,
16
2. RANDOM TRAJECTORIES AND THEIR PROBABILITY
where by definition
(17)
def
P [e2 |(e0 , e1 )] = p[(e0 , e1 , e2 )|H[(e0 , e1 )]]
is the probability of finding the system at the state e2 if at the previous two moments of time it was located at the states e0 and e1 , respectively. Again appealing
to physics the conditional probability P [e1 |(e0 , e1 )] should be specified; in our description it is also regarded as a given value reflecting the particular properties of
the system in question.
Combining equations (12) and (16) we write
(18)
P{(e0 , e1 , e2 )} = P [e2 |(e0 , e1 )] · P [e1 |e0 ] · p0 (e0 ) ,
which is the final result for the probability of random trajectories of 2 step.
The generalization to an arbitrary number of steps T is evident, it is
P{(e0 , e1 , e2 , . . . , eT −1 , eT )} = P [eT |(e0 , e1 , e2 , . . . , aT −1 )]
× P [eT −1 |(e0 , e1 , e2 , . . . , aT −2 )]
···
(19)
× P [e2 |(e0 , e1 )]
× P [e1 |e0 ]
× p0 (e0 )
This type of expression reflects the time stream, there is no effect of future on
present.
3. Markov stochastic processes
Markov chains, Markov (Markovian) stochastic processes, Markovian random
trajectories or random walks are synonyms. The main idea may be addressed to the
philosophic concept of presentism—only the present matters, exists, or have causal
power. With respect to the inanimate world—the physical reality—it is correct. If
for a given system
• the space S of the micro-level states (the set of elementary events) provides the complete description of the system properties and
• the system itself does not change the properties of the state space, then
(20)
P [eT |(e0 , e1 , e2 , . . . , aT −1 )] = P [eT |eT −1 ] ,
thus if the start point e0 is fixed then
(21)
P{(e0 ,e1 , e2 , . . . , eT −1 , eT )}
= P [eT |aT −1 ] · P [eT −1 |aT −2 ] · · · · P [e2 |e1 ] · P [e1 |e0 ]
Such random trajectories are called Markov (Markovian) random trajectories (Andrey Markov (1856–1922), Russian mathematician).
In slogan form, Markovian random processes have no memory; so for them
their history does not matter.
4. DYNAMICAL RANDOM VARIABLES
17
Splitting representation. Formula (21) enables us to split a random trajectory at any time step, e.g., at Ts and to represent the probability of its realization
as the product of the individual probabilities of its parts being independent of one
another. Namely,
(22)
P{(e0 ,e1 , e2 , . . . , eT −1 , eT )}
= P{(e0 , e1 , e2 , . . . , eTs )} × P{(eTs , eTs +1 , . . . , eT −1 , eT )}
This representation plays a significant role in the further constructions.
Comment: Animal foraging exemplifies non-Markovian stochastic processes
not meeting these conditions, within various interpretations, both of them. “True
self-avoiding random walks” is a mathematical example of non-Markovian random
walks.
4. Dynamical random variables
Let e ∈ S be a random variable whose properties are governed by a stochastic
process in the sense that the variable e is the terminal point of some random
trajectory initiated at a certain moment of time in the past. The problem is to
describe how the probabilistic properties of the variable e evolve with time T based
on the properties of the random trajectories.
In these terms finding the corresponding system E at the state e at time T is
not an elementary event; it is a complex event comprising all the trajectories that
initially started, e.g., at the point e0 have gotten the given state e at time T (in T
steps). This event can be represented as (Fig. 3)
[
(23)
ET → e =
(e0 , e1 , e2 , . . . , eT −1 , eT ) .
e1 ,e2 ,...,eT −1 ∈S
As a result, the probability of finding the system at the state e at time T is
X
(24)
G(e, T |e0 , T0 ) =
P{(e0 , e1 , e2 , . . . , eT −1 , eT )} .
e1 ,e2 ,...,eT −1 ∈S
random variable, e
15
10
5
0
5
10
0
20
40
time, T
60
80
100
Figure 3. Realization of dynamical random variable as a complex
event in the set of random trajectories
18
2. RANDOM TRAJECTORIES AND THEIR PROBABILITY
12
random variable, e
10
8
s
4
2
2
4
0
20
40
60
80
100
time
s
Figure 4. Illustration of the trajectory classification leading to
the Chapman-Kolmogorov equation.
Chapman-Kolmogorov equation. Let us use the classification of Markov
trajectories shown in Fig. 4. We fix some time moment Ts (step number) and due
to the splitting formula (22) write
X
G(e, T |e0 , T0 ) =
P{(e0 , e1 , e2 , . . . , eT −1 , eT )}
e1 ,e2 ,...,eT −1 ∈S

=
X
X

e Ts
e1 ,e2 ,...,eTs −1 ∈S
×
X
P{(e0 , e1 , e2 , . . . , eTs −1 , eTs )}

P{(eTs , eTs +1 , eTs +2 , . . . , eT −1 , eT )}
eTs +1 ,eTs +2 ,...,eT −1 ∈S
Whence the Chapman-Kolmogorov equation follows:
X
(25)
G(e, T |e0 , T0 ) =
G(e, T |es , Ts ) · G(es , Ts |e0 , T0 ) ,
e Ts
where Ts is an arbitrary chosen value from the interval T0 < Ts < T !
Figure 5 shows the diagram representing the Chapman-Kolmogorov equation.
Figure 5. The diagram of the Chapman-Kolmogorov equation.
Continuous homogeneous processes. For simplicity let us consider Markovian one-dimensional continuous space of elementary events S = R, where random variable x can take any real number, x ∈ R. Random trajectories {x(T )}
may be regarded a random walks of a walker which during one time step hops to a
5. LAW OF LARGE NUMBERS
19
neighboring point, which is describe by the conditional probability p(x|x0 ) of hoping
to the point x from the point x0 . This stochastic process is called homogeneous if
(26)
p(x|x0 ) = η(x − x0 ) .
In this case a random trajectory x(T ) can be represented as a sequence of random
mutually independent jumps of the walker
X
(27)
(x0 , x1 , x2 , . . . , xT −1 , xT ) =
ξi .
i=1,2,...,T
The probabilistic properties of all the steps are identical and described by the same
probability function η(ξ) meeting the equality
Z ∞
(28)
η(ξ) dξ = 1
−∞
and characterized by the moments
(29)
(30)
def
Z ∞
def
−∞
Z ∞
hξi =
ξ2 =
ξη(ξ) dξ ,
ξ 2 η(ξ) dξ .
−∞
The first moment is usually called the mean value of ξ whereas the second moment
is often represented as
2
x2i = hxi i + σ 2 ,
(31)
where
(32)
def
σ2 =
Z ∞
ξ − hξi
2
η(ξ) dξ .
−∞
The value of σ is usually called the statistical deviate or dispersion.
Now we can formulate the theorem called the Law of Large Numbers which
demonstrates a very tight relationship between Markovian stochastic processes and
the Gaussian distribution.
5. Law of Large Numbers
There are various forms of this law, let us consider its form for random walks
of type (27). The question is what what is the probability function (probability
distribution function) P (z) of the random variable
X
(33)
z=
ξi for T 1.
i=1,2,...,T
Let us introduce the notion of generating function:
Z ∞
def
(34)
g(ζ) = eiζz =
eizζ P (z) dz .
−∞
20
2. RANDOM TRAJECTORIES AND THEIR PROBABILITY
It meets the conditions justifying its name:
(35)
g|ζ=0 = 1 ,
(36)
dg
= i hξi ,
dζ ζ=0
(37)
d2 g
= − ξ2 .
dζ 2 ζ=0
We have the equality
gz (ζ) = eiζz =
(38)
Y
eiζξj = eiζξ
T
= [gξ (ζ)]T .
j
For large values of T small values of ζ contribute to (38) so
T
1
1 2 2
ξ ζ
≈ exp i hξi T ζ − σ 2 T ζ
[gξ (ζ)]T ≈ 1 + i hξi ζ −
2
2
The latter is the Fourier image of the Gaussian distribution, i.e., az T → ∞
1
[z − hζi T ]2
(39)
P (z) = √
exp −
2σ 2 T
2πσ 2 T
Law of Large Numbers and the Chapman-Kolmogorov equation. For
continuous multidimensional homogeneous processes {x(t)}, here x = {x1 , x2 , . . . , xM } ∈
RM ,
G(x, t|x0 , t0 ) = G(x − x0 , t − t0 ) .
(40)
In this case the Chapman-Kolmogorov equation (25)
Z
(41)
G(x, t) =
dxs G(x − xs , t − ts ) · G(xs , ts ) .
RM
Using the Fourier transform
def
Z
dxs eik·x G(x, t)
GF (k) =
RM
we convert (41) into
GF (k, t) = GF (k, t − ts ) · GF (k, ts ) ,
(42)
whence
(43)
1
G(k, t)F = exp i(v · k)t − ∆2 k 2 t
2
,
and
(44)
G(x, t) = √
M
1
2π∆2 t
[x − vt]2
· exp −
2∆2 t
.
Homework
(1) Study numerically the Law of Large Numbers
(2) Prove property (42).
(3) Calculating directly the Fourier transform prove that (44) has the Fourier
image (43) for one-dimensional processes, M = 1.
LECTURE 3
Fokker-Planck equations
Let us confine our consideration to continuous in time t and space x ∈ RM
stochastic processes x(t). The main idea is to reduce the integral ChapmanKolmogorov equation governing such processes to some differential equation. It
can be implemented if the intermediate time moment ts ∈ (t0 , t) tends to one of
the boundary points,
• ts → t − 0, it corresponds to the forward Fokker-Planck equation;
• ts → t0 + 0, it corresponds to the backward Fokker-Planck equation.
forward FPE
backward FPE
Figure 1. The diagrams illustrating the derivation of the forward
and backward Fokker-Planck equations.
We have to accept two additional assumptions besides to the fact that we
analyze Markov stochastic processes—random walks.
• Space-time locality of random walks. In slogan form it is: For a
short time a walker cannot go too far. In mathematical terms it is the
existence of the limits
Z
def
(45)
hδxi δxj i =
(xi − xi0 )(xj − xj0 )G(x, t|x0 , t0 ) dx < ∞
RM
(46)
def
Z
|hδxi i| =
(xi − xi0 )G(x, t|x0 , t0 ) dx < ∞ .
RM
In principle, inequality (46) is redundant because it stems from (45).
• Local homogeneity of random walks (diffusion region). The following two mean values treated as functions of the argument x0
hδxi δxj i (x0 , t0 )
hδxi i (x0 , t0 )
are smooth functions of this argument. So if we confine ourselves to some
small neighborhood of the point x0 , then inside it we may consider that
these quantities are certain constants.
21
22
3. FOKKER-PLANCK EQUATIONS
Remark: There are processes and media where one (or both) of the two assumptions does not hold. In this case we meet anomalous random walks. If the spacetime locality does not hold, we deal with superdiffusion, if the local homogeneity
does not hold, we have subdiffusion.
Let me remind you the key aspects of the Law of Large Numbers, Sec. 5,
page 19. Actually it is about the general type random walks meeting the two
conditions for small time scales inside a small region with approximate homogeneity.
Whence it actually follows that for small value of t − t0
hδxi δxj i ∝ (t − t0 )
hδxi i ∝ (t − t0 ) .
and finally we can introduce the coefficients characterizing local properties of such
random walks.
(47)
Z
(x0i − xi )(x0j − xj )
hδxi δxj i
def
= lim
G(x0 , t + δt|x, t) dx0
Dij (x, t) = lim
δt→+0
δt→+0
2δt
2δt
RM
(48)
hδxi i
Vi (x, t) = lim
= lim
δt→+0 δt
δt→+0
def
Z
(x0i − xi )
G(x0 , t + δt|x, t) dx0 .
δt
RM
The quantity Dij is called a diffusion coefficient (in 1D case), a diffusion matrix or
a matrix of diffusion coefficients (multidimensional case) or a diffusion tensor. The
quantity Vi is called the velocity of regular drift.
Remark: In this theory the quantities Dij (x, t) and Vi (x, t) are treated as
given ones. We cannot construct them appealing to the general properties of the
Chapman-Kolmogorov equation, they are determined by particular physical properties of random walks and the medium. In some sense the physics of analyzed
stochastic processes is hidden in the given coefficients.
The symmetry of diffusion tensor:
Dij (x, t) = Dji (x, t) .
1. Backward Fokker-Planck equation
Figure 2. The diagrams illustrating the derivation of the backward Fokker-Planck equation and the used parametrization of the
Chapman-Kolmogorov equation.
The backward Fokker-Planck equation is some operator acting on the second
pair of arguments, x0 , t0 of the Green function G(x, t|x0 , t0 ). I will remind you that
the Green function of random walks, by definition, is the probability (probability
density) of finding the walker at the point x at time t, provided initially, i.e., at
time t0 it was located at the point x0 . For this reason its name contains the
2. FORWARD FOKKER-PLANCK EQUATION
23
word “backward.” Figure 2 illustrates the Chapman-Kolmogorov equation in the
limit ts → t0 + 0. Using the shown parametrization we can write the ChapmanKolmogorov equation as
Z
(49)
G(x, t|x0 , t0 ) =
G(x, t|x0 + δx, t0 + δt)G(x0 + δx, t0 + δt|x0 , t0 ) dδx
RM
The first cofactor in the integrand of (49)
G(x, t|x0 + δx, t0 + δt)
smoothly changes for small values of δx (local homogeneity) and only small value of
δx are essential in (49) (space-time locality). The behavior of the second cofactor
G(x0 + δx, t0 + δt|x0 , t0 )
is just opposite. So using the Taylor series we can approximate the first cofactor
within the accuracy δt as
∂
G(x, t|x0 , t0 )
∂t0
X
1X
∂2
∂
δxi δxj
+
δxi
G(x, t|x0 , t0 ) +
G(x, t|x0 , t0 ) .
∂xi0
2 i,j
∂xi0 ∂xj0
i
(50) G(x, t|x0 + δx, t0 + δt) = G(x, t|x0 , t0 ) + δt
Substituting (50) into (49) and taking into account definitions (47), (48), as well
as the general normalization of the distribution (Green) function
Z
(51)
G(x0 + δx, t0 + δt|x0 , t0 ) dδx = 1
RM
we obtain the backward Fokker-Planck equation
(52)
−
X
∂2G
∂G
∂G X
=
Dij (x0 , t0 )
+
Vi (x0 , t0 )
∂t0
∂x
∂x
∂x
i0
j0
i0
ij
i
Here in the function of four arguments G(x, t|x0 , t0 ) the first pair of arguments play
the role of parameters.
The backward Fokker-Planck equation is subjected to the initial condition
(53)
G(x, t|x0 , t0 ) t=t0 = δ(x − x0 ) .
The corresponding boundary conditions will be discussed below.
2. Forward Fokker-Planck equation
The forward Fokker-Planck equation is some operator acting on the first pair
of arguments, x, t of the Green function G(x, t|x0 , t0 ). For this reason its name
contains the word “forward.” It is the most popular form and usually exactly the
forward Fokker-Planck equation is referred to as just the Fokker-Planck equation.
Figure 3 illustrates the Chapman-Kolmogorov equation in the limit ts → t − 0.
Using the shown parametrization we can write the Chapman-Kolmogorov equation
as
24
3. FOKKER-PLANCK EQUATIONS
Figure 3. The diagrams illustrating the derivation of the forward Fokker-Planck equation and the used parametrization of the
Chapman-Kolmogorov equation.
Z
(54)
G(x, t|x − δx, t − δt)G(x − δx, t − δt|x0 , t0 ) dδx
G(x, t|x0 , t0 ) =
RM
Now the first cofactor in the integrand of (54)
G(x, t|x − δx, t − δt)
strongly changes with δx, whereas the second one
G(x − δx, t − δt|x0 , t0 )
smoothly changes for small values of δx (local homogeneity) and only small value
of δx are essential in (54) (space-time locality).
Unfortunately, the following steps used above in deriving the backward FokkerPlanck equation in the given case cannot be reproduced because the integral arising
in this way
Z
not necessarily
G(x, t|x − δx, t − δt) dδx
=
1.
RM
I will use the trick proposed by Russian mathematician Lev Pontryagin (1908–1988)
to overcome this problem. Let us multiply equation (54) with an arbitrary chosen
smooth function φ(x) and then integrate it over x
Z
dx φ(x)G(x, t|x0 , t0 )
RM
ZZ
dx · dδx φ(x)G(x, t|x − δx, t − δt)G(x − δx, t − δt|x0 , t0 )
=
RM
ZZ
dx · dδx φ(x + δx)G(x + δx, t|x, t − δt)G(x, t − δt|x0 , t0 ) .
=
RM
where changing the notation of variables x → x + δx and x − δx → x has been
employed. Now we may expand the function φ(x + δx) in the Taylor series within
the accuracy δt
(55)
φ(x + δx) = φ(x) +
X
i
δxi
∂ 2 φ(x)
∂φ(x) 1 X
+
δxi δxj
.
∂xi
2 i,j
∂xi ∂xj
3. BOUNDARY CONDITIONS FOR THE FOKKER-PLANCK EQUATIONS
25
The substitution of this expression into the previous one and definitions (47), (48),
as well as the normalization (51) yield
Z
Z
dx φ(x)G(x, t|x0 , t0 ) =
dx φ(x)G(x, t − δt|x0 , t0 )
RM
RM
Z
dx
+
RM

X ∂φ(x)

i
∂xi
Vi (x, t)G(x, t|x0 , t0 ) +


Dij (x, t)G(x, t|x0 , t0 ) δt .

∂xi ∂xj
X ∂ 2 φ(x)
ij
within the accuracy of δt. Let us use the following trick:
Z
Z
Z
∂Φ
∂(ΦΨ)
∂Ψ
∂Ψ
dx
Ψ=
dx
−Φ
=−
dx Φ
∂xi
∂xi
∂xi
∂xi
RM
RM
RM
for arbitrary smooth functions Φ(x) and Ψ(x) such that at the boundary of integration region at least one of them is equal to zero. It enables us to rewrite the
previous equality as
Z
∂
dx φ(x) G(x, t|x0 , t0 )
∂t
RM


Z
 X ∂

X ∂2
=
dx φ(x) −
Vi (x, t)G(x, t|x0 , t0 ) +
Dij (x, t)G(x, t|x0 , t0 ) .


∂xi
∂xi ∂xj
RM
i
ij
Because the function φ(x) can be constructed in any way, we draw a conclusion
that the corresponding components of this integral must be equal, which yields the
forward Fokker-Planck equation




h
i
X
X
∂G
∂
∂
(56)
=
Dij (x, t)G − Vi (x, t)G

∂t
∂xi 
∂xj
i
j
In this case the second pair of arguments of the function G(x, t|x0 , t0 ) play the role
of parameters. As previously, the forward Fokker-Planck equation is subjected to
the initial condition having the same form
(57)
G(x, t|x0 , t0 ) t=t0 = δ(x − x0 ) .
3. Boundary conditions for the Fokker-Planck equations
The forward Fokker-Planck equation admits the interpretation as the conservation law of diffusing particle. Namely, let us introduce the diffusion flux J = {Ji }
by the formula
i
X ∂ h
(58)
Ji = −
Dij (x, t)G + Vi (x, t)G .
∂xj
j
In this term the Fokker-Planck equation (56) may be rewritten as
X ∂Ji
∂G
(59)
=−
.
∂t
∂xi
i
26
3. FOKKER-PLANCK EQUATIONS
outflux
control region
influx
Figure 4. Illustration of particle conservation law.
This equation has a form of conservation law illustrated in Fig. 4. If the concentration c(x, t) of particles distributed in space c(x, t) and they cannot be created
or annihilated, then their amount in a region Q is increased or decreased by their
flux J entering or leaving this region thought its boundary ∂Q only. It enables us
to write
Z
Z
I
Z X
d
∂c(x, t)
∂Ji
c(x, t) dx =
dx = − J(s) · ds = −
dx
dt
∂t
∂x
i
i
Q
Q
∂Q
Q
by virtue of the Gauss formula. Whence we get the conservation law in the form
of partial differential equation
X ∂Ji
∂c
=−
.
∂t
∂xi
i
So postulating that the vector J specified by (58) is the physical flux of walkers
that move in space Q and when they get the region boundary ∂Q this boundary is
able to affect their motion. Appealing to physical reasons we may formulate typical
situations at the boundary and the corresponding boundary conditions imposed on
the flux J and, thus, on the function G(x, t|x0 , t0 ).
3.1. Boundary conditions for the forward Fokker-Planck equation.
These conditions are imposed on the Green function G(x, t|x0 , t0 ) via the first pair
of its argument.
(60)
• Reflecting or impermeable boundary. Walkers just cannot cross the
boundary ∂Q so the normal component of the probability flux, or the flux
along the normal n = {ni } to the boundary at a given point, has to be
equal to zero:
i X
X
∂ h
J|∂Q · n = −
ni
Dij (x, t)G +
ni Vi (x, t)G = 0 for x ∈ ∂Q
∂xj
i
ij
• Absorbing boundary. After touching the boundary a walker is immediately trapped and never returns to the region. So their concentration
HOMEWORK
27
near the boundary ∂Q has to be much less than at the bulk points. In
other words we write
(61)
G|∂Q = 0 .
• Surface absorption. After getting the boundary ∂Q a walker is trapped
with “probability” σ and being trapped it never returns to the region. So
the normal component of the probability flux, or the flux along the normal
n = {ni } to the boundary at a given point, has to be equal to
i X
X
∂ h
Dij (x, t)G +
ni Vi (x, t)G = −σG for x ∈ ∂Q
(62) J|∂Q · n = −
ni
∂xj
i
ij
3.2. Boundary conditions for the backward Fokker-Planck equation.
These conditions are imposed on the Green function G(x, t|x0 , t0 ) via the second
pair of its argument.
Absorbing boundary. After touching the boundary a walker is immediately
trapped and never returns to the region, so if a walker was created at the boundary
it would be traps immediately. So
(63)
G|∂Q = 0 .
Reflecting or impermeable boundary requires a more sophisticated analysis.
Using the Chapman-Kolmogorov equation
Z
G(x, t|x0 , t0 ) = dxs G(x, t|xs , ts ) · G(xs , ts |x0 , t0 ) ,
Q
noting that its left-hand side does not depend on ts , and then differentiating this
equation we get
(64) Z
∂G(x, t|xs , ts )
∂G(xs , ts |x0 , t0 )
0 = dxs
· G(xs , ts |x0 , t0 ) + G(x, t|xs , ts ) ·
.
∂ts
∂ts
Q
Then using the forward and backward Fokker-Planck equations as well as the
boundary conditions for the forward Fokker-Planck equation it is possible to show
that at the reflecting boundary
X
∂G
= 0 for x0 ∈ ∂Q
(65)
ni Dij (x0 , t0 )
∂xj0
ij
Homework
• Prove the boundary condition (65).
Hint: make use of Gauss’s theorem
Z
X ∂Ai (x) I
X
dx
= ds
ns,i (xs )Ai (xs )
∂xi
i
i
Q
∂Q
where ns (xs ) is the unit vector normal to the boundary ∂Q at its given
point xs and directed outward of it.
• Using Python generate Mandelbrot set.
LECTURE 4
Forward Fokker-Planck Equation: Its applications
1. Systems with thermodynamic equilibrium: Detailed balance
As time goes on, the stationary distribution function
Gst (x))
←
G(x, t|x0 , t0 )
as t → ∞
obeys the equation following directly from the forward Fokker-Planck equation (56)



i
X ∂ X ∂ h
Dij (x)Gst − Vi (x)Gst
0=

∂xi 
∂xj
i
j
Let us accept an additional assumption that in the equilibrium there is no probability flux and set for any i
i
X ∂ h
(66)
0=
Dij (x)Gst − Vi (x)Gst .
∂xj
j
Condition (66) is called the detailed balance, the meaning of the use term will be
clarified in the following lecture devoted to the master equations.
So if spatial properties of the stationary probability function Gst (x) are described in terms of the “energy” relief U (r), i.e.,
Gst (x)) ∝ exp{−βU (x)}
where β = 1/T
and besides near the equilibrium the noise is linear, that is, the diffusion tensor Dij
does not depend on the random variable x, then the substitution of this expression
into (66) yields the equality
X
∂U (x)
(67)
Vi (x) = −
βDij
.
∂xj
j
The gradient
∂U (x)
∂xj
admits interpretation as some force acting on the walker, so in the general case we
can assume that this force causes the walker to migrate and write
X
∂U (x)
Vi = −
µij
∂xj
j
−
where µij is the particle mobility. Whence we have immediately the relationship
between the mobility and diffusion
(68)
T µij = Dij .
29
30
4. FORWARD FOKKER-PLANCK EQUATION: ITS APPLICATIONS
called the Einstein relation and is a certain gist the famous fluctuation-dissipation
theorem of equilibrium thermodynamics relating properties of random fluctuations
and dissipative characteristics to each other.
2. Method of Moments
2.1. Moments: Definition. Let us consider some random process {x(t)},
here x = {x1 , x2 , . . . xN } ∈ RN is a point of N -dimensional space. The function
(69)
G(y, t|x, t0 )
gives us the probability (probability density) of finding the walker at the point y
at time t provided initially (at time t0 ) it was located at the point x. By definition,
...nN
the moment M1n12n2...N
is the mean value
def
...nN
(70) M1n12n2...N
(x, t, t0 ) = h(δx1 )n1 (δx2 )n2 · · · (δxN )nN i
= E [(δx1 )n1 (δx2 )n2 · · · (δxN )nN ]
Z
dy (y1 − x1 )n1 (y2 − x2 )n2 · · · (yN − xN )nN G(y, t|x, t0 )
=
RN
being actually a function of the arguments x, t, and t0 ; here the notion δxi = yi −xi
has been used. If ni = 0 then the corresponding cofactor (yi − xi ) is absent because
(yi − xi )0 = 1 ,
so also the index i may be also omitted. For example, the first order moment
(71)
(1)
Mi (x, t, t0 ) = Mi1 (x, t, t0 ) = hδxi i
is merely the mean value of the walker displacement δxi along the axis xi from the
initial position x during the time interval (t − t0 ). The second order moment
(72)
(2)
Mij (x, t, t0 ) = Mi11
j (x, t, t0 ) = hδxi δxj i
together with the first one gives us the correlation coefficient of the walker displacement along the axes xi and xj :
D
E
(73)
Cij (x, t, t0 ) = (δxi − hδxi i) · (δxj − hδxj i) = hδxi δxj i − hδxi i hδxj i
or in the normalized form
(74)
cij (x, t, t0 ) = rh
hδxi δxj i − hδxi i hδxj i
i h
i ≤1
2
2
h(δxi )2 i − hδxi i · h(δxj )2 i − hδxj i
by virtue of the Cauchy-Schwarz inequality: for any two vectors {uk }M
k=1 and
{vk }M
k=1
M
X
k=1
2
uk vk
≤
M
X
k=1
u2k
M
X
vk2 .
k=1
The higher order moments characterize the deviation of the probability function
G(y, t|x, t0 ) from the Gaussian form. For example for 1D space R
(y − x − vt)2
1
g
G (y, t|x, 0) = √
exp −
.
4Dt
4πDt
2. METHOD OF MOMENTS
31
The second moment within the following normalization called skewness characterizes the asymmetry of the function G(y, t|x, t0 )
h
3 i
E δx − hδxi
(75)
γ1 = h
.
2 i3/2
E δx − hδxi
Negative Skew
Positive Skew
Figure 1. Illustration of the distribution function asymmetry
characterized by skewness.
The fourth moment called kurtosis characterizes the distribution “tails”
h
4 i
E δx − hδxi
(76)
γ2 = h
2 i2 .
E δx − hδxi
For the Gaussian distribution the kurtosis γ2 = 3.
Figure 2. Log-pdf for distributions with the difference γ2 − 3,
excess kurtosis of infinity (red); 2 (blue); 1, 1/2, 1/4, 1/8, and
1/16 (gray); and 0 (black).
32
4. FORWARD FOKKER-PLANCK EQUATION: ITS APPLICATIONS
2.2. Governing equations for moments. The case of unbounded space.
The distribution function (69) G(y, t|x, t0 ) obeys the forward Fokker-Planck equation (56)




h
i
X
X
∂G
∂
∂
=
Dij (y, t)G − Vi (y, t)G

∂t
∂yi 
∂yj
i
j
Integrating the left- and right-hand sides of this equation with some smooth function
φ(y) we get
Z
Z
∂
∂
∂G
Gφ(y)dy =
φ(y)dy =
hφi
∂t
∂t
∂t
N
N
R
R


Z X


h
i
X ∂
∂
=
Dij (y, t)G − Vi (y, t)G φ(y)dy

∂yi  j ∂yj
i
RN

!
Z X

i
X ∂ h
∂ 
=
φ(y)
Dij (y, t)G − Vi (y, t)G
dy

∂yi 
∂yj
i
j
RN
!
Z X
i
∂φ(y) X ∂ h
−
Dij (y, t)G − Vi (y, t)G dy
∂yi
∂yj
j
i
N
R
Z X
Z X
i X ∂φ(y)
∂ h
∂φ(y)
=−
Dij (y, t)G
dy +
Vi (y, t)G dy
∂y
∂y
∂yi
j
i
i
j
i
RN
=
RN
Z X
RN
ij
2
∂ φ(y)
Dij (y, t)Gdy +
∂yi ∂yj
Z X
RN
i
∂φ(y)
Vi (y, t)G dy
∂yi
It gives us the desired equation:


2

∂ hφi X X
∂ φ
∂φ
(77)
=
· Dij (y, t) +
· Vi (y, t)


∂t
∂yi ∂yj
∂yi
i
j
For φ(y) = δyk we obtain the equation governing the first order moment—the
mean displacement, namely, from (77)
∂ hδyk i
= hVk (y, t)i .
∂t
(78)
Here we have taken into account that
∂yk
def
= δki = 1, if i = k and 0, if i 6= k.
∂yi
For φ(y) = δyk · δyl we get
(79)
∂ hδyk · δyl i
= 2 hDkl (y, t)i + hδyk · Vl (y, t)i + hδyl · Vk (y, t)i
∂t
because
∂(yk · yl )
= yl δki + yk δli
∂yi
and
∂ 2 (yk · yl )
= δlj δki + δkj δli .
∂yi ∂yj
2. METHOD OF MOMENTS
33
Equations (78) and (79) should be subjected to the initial conditions
(80)
hδyk · δyl i |t=t0 = 0
hδyl i |t=t0 = 0 .
and
2.2.1. Some typical stochastic processes.
The Wiener process. Actually it is homogeneous random walks without bias,
i.e., for them Dij is constant and Vi = 0. Then by virtue of (78), (79), and (80)
(81)
hδyk · δyl i (x, t, t0 ) = 2Dkl (t − t0 )
and
hδyl i (x, t, t0 ) = 0 .
The Ornstein-Uhlenbeck process. It is 1D over-damped oscillator:
dy
= −ky + noise .
dt
For it D is constant and V = −ky = −k(x + δy). Whence
(82)
(83)
∂ hδyi
= −k hδyi − kx ,
∂t
or
∂ hyi
= −k hyi
∂t
that is the mean value hyi goes to zero as e−kt . The equation governing the second
order moment is
∂ (δy)2
= 2D − 2k hδyi x − 2k (δy)2 .
∂t
It leads to the steady-state solution
(84)
(δy)2 =
D
,.
k
light
Figure 3. Semiconductor with electron-hole plasma generated by
light absorption and the charged particle transport in crossed electric and magnetic fields.
2.3. Diffusion in weak magnetic fields. Let us consider a diffusion problem
of charged particles in a crossed electric E and magnetic field B. The field weakness
means that the length of free particle motion is a small spatial scale and the particle
motion can be described by the following coupled equations:
(85)
dx
= v,
dt
dv
m
e
m
= − v + eE + [v × B] + noise .
dt
τ
c
34
4. FORWARD FOKKER-PLANCK EQUATION: ITS APPLICATIONS
The (forward) Fokker-Planck equation for such random walks can be written
using Einstein’s notation as
i T ∂2P
∂P
∂(vi P )
∂ h
τ
eτ
(86) τ
= −τ
+
vi − eEi −
ijk · vj · Bk P +
,
∂t
∂xi
∂vi
m
mc
m ∂vi ∂vi
where T is the temperature (in energy units), Dv = T /m is the diffusion coefficient
in the velocity space, and the Levi-Civita symbol is


if (i, j, k) = (1, 2, 3), (2,3,1), or (3,1,2)
+1,
ijk = −1,
if (i, j, k) = (3, 2, 1), (1,3,2), or (2,1,3)


0,
if i = j, or j = k, or i = k
Figure 4 illustrates the Levi-Civita symbol (tensor).
2
3
+1
1
Figure 4. For the indices (i, j, k) in
ijk , the values 1, 2, 3 occurring in the
cyclic order (1,2,3) (yellow) correspond
to ijk = +1, while occurring in the reverse cyclic order (red) correspond to
ijk = −1, otherwise ijk = 0.
2
3
1
1
In the equilibrium when no magnetic field is applied to the system the FokkerPlanck equation reads
i T ∂2P
∂(vi P )
∂ h
τ
−τ
+
vi − eEi P +
= 0.
∂xi
∂vi
m
m ∂vi ∂vi
This equation has to possess and possesses the solution specified by the Gibbs
distribution (the general statement of the equilibrium thermodynamics)
1 mv 2
st
+ U (x)
.
(87)
P (x, v) = Const · exp −
T
2
where
eE(x) = −∇ · U (x) .
Actually this fact has been used in specifying the diffusion coefficient Dv for the
velocity space. It should be noted that within this complete description the system
does not admit the detailed balance because, for example, the steady-state diffusion
flux component along the axis xi
Qxi ∝ vi P st 6= 0 .
Let us define the diffusion flux at the spatial point x as
Z
def
J(x, t) =
(88)
dv vP (x, v, t) ≡ hviv C(x, t) ,
Rv
where the concentration of particles (walkers) at the point x at time t is
Z
def
(89)
C(x, t) =
dv P (x, v, t) .
Rv
2. METHOD OF MOMENTS
35
Integrating the left and right hand sides of equation (86) we get
∂Jl
∂[hvi vl i C]
τ
eτ
= −τ
− Jl + eEl C +
ljk Jj Bk C .
∂t
∂xi
m
mc
For the steady-state distribution (87) we have
T
st
hvi vj i = δij .
m
So for weak fields and smooth gradients and slow time variations from (90) we
obtain
(90)
(91)
τ
Jl = −
τ ∂
τ
eτ
(T C) + eEl C +
ljk Jj Bk .
m ∂xl
m
mc
Here we can introduce the diffusion coefficient D in the space and the particle
mobility µ as
def τ
def τ T
and
µ =
.
D =
m
m
such that again we have the Einstein relation D = µT , so within this reduced
description the detail balance has to hold. In the case of weak magnetic fields B
iterating (91) for small B we get the expression for the diffusion flux
e2 µ2
eµD
(92)
J = −∇ · (D C) + µeEC +
[E × B]C −
[ ∇ · (D C) × B] .
| c {z
| c
}
{z
}
the Hall effect
the photoelectromagnetic effect
If the electric field is absent and the temperature is constant then the diffusion
flux is
eD2
(93)
J = −D∇C +
[B × ∇]C .
Tc
This expression admits the interpretation in terms of non-symmetric diffusion matrix (tensor)
(94)
Dij = Ds δij + Da ijk Bk ,
which contradicts the interpretation of the diffusion flux based on the Fokker-Planck
equation.
Numerical analysis of diffusion-type equations
In this lecture we consider several examples of studying diffusion processes
based on solving the Fokker-Planck equation numerically (using Python). In mathematics such equations are classified as parabolic partial differential equations. However before this let us consider some characteristic examples of ordinary differential
equations.
3. Ordinary differential equations: initial value problem
The following is the general form of analyzed problem,
(95)
dy
= F (y, t)
dt
subject to the initial condtion y|t=t0 = y0 .
Pendulum (oscillator):
d2 x
= −ω 2 x
dt2
what is equivalent to
(96)
dx
=v
dt
dv
= −ω 2 v
dt
Lorenz model —a simplified description of turbulence with dynamical chaos
(97)
dx
= σ(y − x)
dt
dy
= ρx − y − xz
dt
dz
= xy − βz
dt
4. Partial differential equations: initial value problem
Diffusion equation being a simplified version of the Fokker-Planck equation,
(98)
∂c
∂2c
=D 2
∂t
∂x
37
38
NUMERICAL ANALYSIS OF DIFFUSION-TYPE EQUATIONS
Homework
• Study numerically the dynamics and phase portrait of the Van der Pol
oscillator (using odeint, Python)
d2 x
dx
− µ(1 − x2 )
+x=0
2
dt
dt
for several µ ∈ [0.01, 4.0]
• Solve diffusion equation numerically
∂c
∂2c
x ∈ [0, L], t ∈ [0, T ]
=
∂t
∂x2
for reflecting boundary conditions
∂c
=0
∂x x=0,L
and an initial condition similar to
(x − L/2)2
c(x, t) t=0 = exp
σ2
LECTURE 5
Backward Fokker-Planck Equation: Applications
1. First passage time problem: Example of one-dimensional stochastic
process
Let us consider a stochastic process x(t) (random walks) in the one-dimensional
space R+ = {x > 0}. Here the point x = 0 is the boundary of the region x > 0,
where the process develops. The dynamical properties of process are characterized
by the diffusion coefficient D and the regular drift speed V being constant quantities. The process is assumed to be initiated at the point x0 = y > 0 at time t0
and terminated when the variable x(t) gets the boundary x = 0. In other words,
it is assumed that the walker is trapped immediately and forever when it gets the
boundary point x = 0.
The first passage time problem poses the question about the probability density
F (y, t) that this walker gets the boundary x = 0 (for the first time) at time t.
The Green function G(x, t|y, t0 ) = G(x, y, t − t0 ) describes the probability density of finding the walker at the point x > 0 at time t, which implies that the
walker has not gotten the boundary up to the given moment of time t. Thereby the
probability G(y, t) that the walker has not gotten the boundary point x = 0 within
the time interval t := t − t0 (meaning t0 = 0) is determined by the integral
Z
G(x, y, t) dx
(99)
G(y, t) =
R+
The function G(y, t) may be called the probability of surviving.
The probability of surviving G(y, t) and the probability density of getting the
boundary x = 0 for the first time at the moment t are related by the expression
Zt
(100)
G(y, t) +
F (y, t0 ) dt0 = 1 .
0
In other words, the walker either survives within the time interval t or is trapped
by the boundary somewhen during this time. Thereby we have
(101)
F (y, t) = −
∂G(y, t)
∂t
being the basic relationship between the probabilistic characteristics of random
walks inside the region R+ and getting the region boundary.
Because we need to integrate the Green function G(x, y, t) over the first spatial
variable x it is quite natural make use of the backward Fokker-Planck equation for
39
40
5. BACKWARD FOKKER-PLANCK EQUATION: APPLICATIONS
G(x, y, t)
∂G(x, y, t)
∂G(x, y, t)
∂ 2 G(x, y, t)
+V
=D
.
2
∂t
∂y
∂y
(102)
where the variable x plays the role of parameter. Integrating Eq. (102) over x we
get
∂G(y, t)
∂ 2 G(y, t)
∂G(y, t)
+V
=D
.
∂t
∂y 2
∂y
Then finding the time derivative of the left- and right-hand sides of this equality
and taking into account identity (101) we obtain the equation
∂F (y, t)
∂F (y, t)
∂ 2 F (y, t)
+V
=D
∂t
∂y 2
∂y
(103)
governing the probability density F (y, t) of getting the boundary point for the first
time at the moment t.
In order to solve this equation and, thereby, to solve the first passage time
problem, we need to know the condition that should be imposed on the function
F (y, t) at the boundary point y = 0. Unfortunately it cannot be done directly
in a simple, clear way because the function F (y, t) exhibits a rather pathological
behavior at the boundary—if the random walks are initiated at the boundary point
y = 0, the walker will be trapped within infinitely short time interval, t = +0. So
we can to write only that
Zt
(104)
F (y, t0 )|y=0 dt0 = 1
for any t > 0.
0
It prompts us to employ the formalism of Laplace transforms, namely, instead of
the function F (y, t) let us deal with its Laplace transform
def
(105)
Z∞
FL (y, s) =
e−st F (y, t) dt0
for any s > 0.
0
In contrast to the origin F (y, t) the Laplace transform FL (y, s) exhibits a regular
behavior at the boundary point y = 0, namely, due to (104)
(106)
FL (y, s)|y=0 = 1
for any s > 0.
Now let us find the Laplace transform of the left- and right-hand sides of
Eq. (103). Because its right-hand side does not contain the time derivative we
can just replace in it the function F (y, t) by the Laplace transform FL (y, s). The
Laplace transform of the left-hand side is
Z∞
Z∞ −st
∂[e F (y, t)] ∂e−st
−st ∂F (y, t)
dt =
−
F (y, t) dt
e
∂t
∂t
∂t
0
0
= [e
−st
F (y, t)]t=∞
t=0 +
Z∞
0
se−st F (y, t) dt = sFL (y, s) .
1. FIRST PASSAGE TIME PROBLEM: EXAMPLE OF ONE-DIMENSIONAL STOCHASTIC PROCESS
41
Here we have taken into account that
e−st F (y, t)]t=∞ = 0
e−st F (y, t)]t=0 = 0
and
for any y > 0.
In particular, the latter equality means that if random walks are generated at some
internal point y > 0, the walker cannot get the boundary just after the process
initiation. As a result, the equation governing the Laplace transform FL (y, s) of
the reaching the boundary for the first time takes the form
(107)
sFL (y, s) = D
∂FL (y, s)
∂ 2 FL (y, s)
+V
∂y 2
∂y
subjected to the boundary condition (106).
Now we can analyze the particular features of the first passage time problem.
Eq. (107) subjected to the boundary condition (106) and the boundedness as y → ∞
admits the following solution
(108)
FL (y, s) = exp{−λy} ,
where
(109)
V
+
λ=
2D
r
s
V2
+ .
4D2
D
Let us consider particular cases.
No regular drift: V = 0. In this case expression (108) reads
r
s
y ,
(110)
FL (y, s) = exp −
D
According to the Table of Laplace transforms the Laplace transform of the function
√
a
√
exp{−a2 /4t} is exp{−a s}
2 πt3
So the desired expression for the first passage time probability is
y
y2
(111)
F (y, t) = √
exp −
.
4Dt
4πDt3
Regular drift toward the boundary: V < 0. The asymptotic behavior of
the Laplace transform as s → 0 (which corresponds to the limit t → ∞) is
n s o
(112)
FL (y, s) = exp − y ,
V
Noting that the Laplace transform of the Dirac δ-function
δ(t − T )
is
exp{−sT }
we may conclude that the walker will be trapped by the boundary with the interval
about t ∼ y/V . In particular,
hti = −
∂FL (s, y)
y
=
∂s
V
s=0
42
5. BACKWARD FOKKER-PLANCK EQUATION: APPLICATIONS
Regular drift inward the region: V > 0. The asymptotic behavior of the
Laplace transform as s → 0 (which corresponds to the limit t → ∞) is
V
(113)
FL (y, s) = exp − y ,
D
which means that there is finite probability G∞ that the walker will never touch
the boundary. Due to equality (100) it is
V
(114)
G∞ = 1 − exp − y .
D
2. Escaping rate from a potential well
Let us consider the system shown in Fig. 1—a particle affected by random
forces and located in a certain potential well. Dealing with an ensemble of particles
we can pose the question about the rate νbd at which particles escape from the
well—the probability density F(t) for such a particle to escape from the potential
well at a given time t provided initially, t = 0, it has been placed near the local
minimum (here x = 0). Namely
(115)
νbd = F(+0) ,
where the value +0 of the argument t means that we consider time scales exceeding
substantially the duration of all the transient processes during which the distribution of the particle inside the potential well attains locally quasi-equilibrium.
Figure 1. Escaping problem simulating the critical nucleus formation
The concept of potential well implies the barrier to be sufficiently high, Ωc 1,
therefore the particle can climb over it due to rare fluctuations lifting the particle
to points at the potential barrier where Ω(x) 1. If such an event does not lead to
escape, the particle will drift back to the neighborhood of the local minimum x = 0
whose thickness is specified by the inequality Ω(x) . 1. Thereby the subsequent
attempts of escaping may be considered as being mutually independent. After the
particle has climbed over the barrier the force −dΩ(x)/dx carries it away to distant
points, making the return impossible. So from this point of view we may refer
to the particle being inside the potential well or having escaped from it as two its
possible states without specifying the particular position. Therefore the probability
2. ESCAPING RATE FROM A POTENTIAL WELL
43
P(t − t0 ) that the particle remains inside the well at time t, if it has been placed
there at time t0 , obeys the equation:
P(t) = P(t − t0 )P(t0 )
for 0 < t0 < t .
We get the general expression for the function P(t)
t P(t) = exp −
,
τlife
where τlife is a certain constant specified by the particular properties of a potential well. The latter formula gives us immediately the general form of the escape
probability
1
t dP(t)
.
=
exp −
(116)
F(t) = −
dt
τlife
τlife
In order to find the lifetime τlife we will deal with the Laplace transform FL (s)
of the escape probability F(t)
Z∞
1
(117)
FL (s) := dt exp(−st)F(t) =
,
1 + sτlife
0
whence it follows that in the expansion of FL (s) with respect to s around the point
s=0
(118)
FL (s) = 1 − sτlife + . . .
the first order term directly contains the desired lifetime as the coefficient.
Following the standard approach we reduce the escaping problem to finding the
first passage time probability. In other words, we assume the particle never comes
back to the potential well if, after climbing the barrier, it reaches points where
Ωc − Ω(x) & 1 (Fig. 1). The particle may be withdrawn from the consideration
or, what is the same, it will be trapped when reaches for the first time any fixed
point x∗ in this region. The time it takes for the particle to reach the point x∗
after overcoming the barrier at the critical point xc is ignorable in comparison with
the characteristic waiting time for critical fluctuations. Thereby the function F(t)
specifies actually the probability of passing (reaching) the point x∗ for the first time
at the time moment t. This construction enables us to introduce a more detailed
relative function F(x, t) giving the probability for the particle initially placed at
the point 0 < x < x∗ to reach first the right boundary x∗ of the region under
consideration at the time moment t. The left boundary x = 0 is impermeable for
the particle. Then using the standard technique based on the backward FokkerPlanck equation we obtain the governing equation for the function FL (x, s)
(119)
sFL = D
∂Ω(x) ∂FL
∂ 2 FL
−µ
·
∂x2
∂x
∂x
subject to the boundary conditions
(120)
FL (0, s) = FL (x∗ , s) = 1 .
It directly follows that the first order term ϕ(x) in the expansion of the Laplace
transform FL (x, s) with respect to s
F(x, s) = 1 − sϕ(x) ,
44
5. BACKWARD FOKKER-PLANCK EQUATION: APPLICATIONS
obeys in turn the equation
D
(121)
∂ 2 ϕ(x)
∂x Ω(x) ∂x ϕ(x)
−µ
·
= −1
2
∂x
∂x
∂x
subject to the boundary conditions
∂ϕ(x)
=0
∂x x=0
(122)
and ϕ(x∗ ) = 0 .
The solution of the system (121) and (122) has the form
Z ∗
Z x0
00
1 x
0 Ω(x0 )/T
(123)
ϕ(x) =
dx e
dx00 e−Ω(x )/T
D x
0
and the value ϕ(0) gives us the desired lifetime:
(124)
τlife = ϕ(0) .
Here we have used the Einstein relation D = µT , where T is the temperature—the
measure of random force intensity.
Inside the potential well the function ϕ(x) takes practically a constant value
mainly contributed by the points x00 belonging to the well bottom, i.e. to the region
Ω(x00 ) . 1 and by the points x0 located near the top of the potential barrier, where
Ω(xc ) − Ω(x0 ) . 1. This feature leads us immediately to the approximation
(125)
τlife ≈
√
T 3/2
2π
D
"
∂ 2 Ω(x)
∂x2 x=xc
#− 1 2
−1
∂Ω(x)
eΩ(xc )/T ,
∂x x=0
which is the main result.
3. Probability of extreme events
Let us consider some one-dimensional random process {x(t)} originating from
the point x0 at time t = 0. The problem is to calculate the probability density
P(a, x0 , t) of that the maximal value
def
xmax = max
{x(t0 )} = a
0
0≤t ≤t
attained somewhere in the time interval [0, t] is equal to a given value a > x0 . This
stochastic process is assume to be a biased Wiener process governed by the forward
Fokker-Planck equation with constant coefficients
∂2G
∂G
∂G
=D 2 −v
.
∂t
∂t
∂x
We can write that the probability of the walker surviving during the time interval
[0, t] is
Za
(126)
Zt
dx G(x, x0 , t) = 1 −
P (a, x0 , t) =
−∞
0
dt0 F (a, x0 , t0 )
3. PROBABILITY OF EXTREME EVENTS
45
where F (a, x0 , t) is the probability density of getting the boundary a for the first
time at instant t. Besides, the desired probability density P(a, x0 , t) is given by the
expression
Zt
∂P (a, x0 , t)
∂
(127)
P(a, x0 , t) =
=−
dt0 F (a, x0 , t0 )
∂a
∂a
0
The Laplace transform of the functions entering equation (127) are related as
P(a, x0 , s) =
(128)
1 ∂F (a, x0 , s)
∂P (a, x0 , t)
=−
.
∂a
s
∂a
Equation for F (a, x0 , s) is
(129)
sF = D
dF
d2 F
+v
2
dx0
dx0
subject to the boundary conditions
F |x0 =a = 1 ,
(130)
F |x0 =−∞ = 0 .
The solution is
(
(131)
F (a, x0 , s) = exp −(a − x0 )
"r
s
v2
v
+
−
2
D 4D
2D
#)
.
LECTURE 6
Master equation
1. Chapman-Kolmogorov equation and master equation
The master equation is usually used for describing discrete random processes
{x(t)} when the variable x changes stepwise in time. Let us, at first, consider time
scales much shorter than all the characteristic scales Λ of the system dynamics. In
this case the probability G(x0 , t|x0 , t0 ) of the system remaining at the same state
x0 , where it was located initially, t = t0 , decreases with time as
(132)
G(x0 , t|x0 , t0 ) = exp{−ν(x0 , t0 ) · (t − t0 )} .
Here the coefficient ν(x0 , t0 ) can depend on the current state x0 and time t0 (note
that t − t0 Λ). Indeed, the probability G(x0 , t|x0 , t0 ) can be represented as
the probability that the walker did not leave the state x0 during the time interval
[t0 , t0 ], where t0 < t, and then remained at it during the time interval [t0 , t]. If this
stochastic process is Markov then
G(x0 , t|x0 , t0 ) = G(x0 , t|x0 , t0 ) · G(x0 , t0 |x0 , t0 ) .
Because the time moment t0 ∈ (t0 , t) may be chosen arbitrary only form (132)
admits this equality. Thereby within the first order in δt = t − t0
G(x0 , t|x0 , t0 ) = 1 − ν(x0 , t0 ) · δt .
It allows us to suppose that the probability of transition to the state x 6= x0 within
the time interval δt is proportional to δt, which can be written as
(133)
G(x, t|x0 , t0 ) = νx←x0 (t0 ) · δt ,
and correspondently
(134)
G(x0 , t|x0 , t0 ) = 1 −
X
νxs ←x0 (t0 ) · δt ,
xs : xs 6=x0
because the probability of two successive steps is proportional to (δt)2 and, thus, can
be ignored. The quantities νx←x0 are called transition rates. The two expression
can be combined with one
"
#
X
(135)
G(x, t|x0 , t0 ) = δxx0 + νx←x0 (t0 ) − δxx0
νxs ←x0 (t0 ) δt ,
xs
where the transition rate νx0 ←x0 (t0 ) can be introduced in any way, its particular
value does not matter. Expression (135) admits the diagram representation shown
in Fig. 1. Using the shown diagrams we immediately arrive at the forward and
backward master equations (Fig. 2). In mathematical terms they read
47
48
6. MASTER EQUATION
;
Figure 1. Diagram representation of the Green function
G(x, t|x0 , t0 ) for short time scales δt.
the forward master equation:
i
dGxx0 (t, t0 ) X h
=
νx←xs (t) · Gxs x0 (t, t0 ) − νxs ←x (t) · Gxx0 (t, t0 ) ,
dt
x
(136)
s
the backward master equation:
(137)
−
i
dGxx0 (t, t0 ) X h
=
Gxxs (t, t0 ) · νxs ←x0 (t0 ) − Gxx0 (t, t0 ) · νxs ←x0 (t0 ) .
dt0
x
s
forward
equation
backward
equation
Figure 2. Diagram representation of the forward and backward
master equation and the corresponding structure of the ChapmanKolmogorov equation.
2. ERGODIC PROPERTIES
49
2. Ergodic properties
Stochastic processes in the space with finite number of states is ergodic if there
is probability to get any state from another state. In this case the same state
will be visited many times and the system in some way reproduces statistically the
same scenario of motion. So if the transition rates do not depend on time the
stationary distribution has to emerge. In this case averaging over time is equivalent
to averaging over ensemble.
50
6. MASTER EQUATION
3. Properties of the master equation
If the state space of the stochastic variable is a discrete one, often considering
natural numbers within a finite range 0 ≤ n ≤ N , the master equation for the time
evolution of the probabilities p(n, t) is written as
X
dp(n, t)
=
{w(n, n0 )p(n0 , t) − w(n0 , n)p(n, t)} ,
dt
0
(138)
n 6=n
0
where w(n , n) ≥ 0 are rate constants for transitions from n to other n0 6= n.
Together with the initial probabilities p(n, t = 0) (n = 0, 1, 2, . . . , N ) and the
boundary conditions at n = 0 and n = N this set of equations governing the time
evolution of p(n, t) from the beginning at t = 0 to the long–time limit t → ∞ has
to be solved. The meaning of both terms is clear. The first (positive) term is the
inflow current to state n due to transitions from other states n0 , and the second
(negative) term is the outflow current due to opposite transitions from n to n0 .
Now let us define stationarity, sometimes called steady state, as a time independent distribution pst (n) by the condition dp(n, t)/dt|p=pst = 0. Therefore the
stationary master equation is given by
X
(139)
0=
w(n, n0 )pst (n0 ) − w(n0 , n)pst (n) .
n0 6=n
This equation states the obvious fact, that in the stationary or steady state regime
the sum of all transitions into any state n must be balanced by the sum of all
transitions from n into other states n0 . Based on the properties of the transition
rates per unit time the probabilities p(n, t) tend in the long–time limit to the
uniquely defined stationary distribution pst (n), for which in open systems a constant
probability flow is possible. This fundamental property of the master equation may
be stated as
(140)
lim p(n, t) = pst (n) .
t→∞
4. Kirchhoff method
The stationary solution can be found by an elegant method developed by Gustav Kirchhoff. The Markovian system under consideration is represented by a graph
G consisting of vertices and edges. Vertices correspond to system states, whereas
edges connect all vertices i and j for which at least one of the transition rates wij
and wji is nonzero. For nonzero transition rates wij , the graph G of the three–level
system is represented as
In the following we will assume that the graph G is connected. In principle, it
can also consist of unconnected parts. Then Kirchhoff’s method can be applied to
each part separately. An example is the already considered case with an isolated
state, where the problem, in fact, reduces to the solution of a two–level system
represented by two vertices connected with one edge. The only peculiarity is that
4. KIRCHHOFF METHOD
51
the stationary probabilities have to be normalized in such a way that the total
probability for the isolated subsystem is that given by the initial condition.
The stationary solution is represented by subgraphs of G, called maximal trees.
A maximal tree T {G} is a connected subgraph of G such that (i) all edges of T {G}
are edges of G, (ii) T {G} contains all vertices of G, and (iii) T {G} contains no
circuits (cyclic sequences of edges). It is easy to realize that one has to drop a
certain minimum number of edges of G to exclude circuits and obtain maximal
trees. Maximal trees of the graph G for the three–level system are
In order to construct the stationary solution of the master equation we need
directed maximal trees. For a given state i and tree T the directed tree Tj is
obtained from T by directing all its edges towards the vertex i. The superposition
of such trees makes up the general solution of the master equation. In case of
three–level system it is represented as
(141)
K1 =
+
+
K2 =
+
+
K3 =
+
+
Here the symbol
stands for the transition from the state j to
the state i with rate wij . We ascribe to each directed maximal tree Tj a weight
Ki {Tj } equal to the product of all transition rates wij corresponding to its edges
with the appropriate directions. The value Ki is defined as the sum of Ki {Tj }
running over all the maximal directed trees leading to state i
X
(142)
K1 =
Ki {Tj } .
all Tj
According to (141), for our three–level system we have
(143)
K1 = w12 w23 + w13 w32 + w12 w13 ,
(144)
K2 = w21 w13 + w23 w31 + w21 w23 ,
(145)
K3 = w31 w32 + w32 w21 + w31 w12 .
The Kirchhoff formula for the stationary probability distribution pst
i of an N –level
system, represented by a connected graph, is given by
(146)
Ki
pst
i = PN
j=1 Kj
.
52
6. MASTER EQUATION
5. System with detailed balance
Now we are discussing the question of equilibrium in a system without external
exchange. The condition of equilibrium in closed isolated systems is much stronger
than the former condition of stationarity (139). Here we demand as an additional
constraint a balance between each pair of states n and n0 separately. This so–called
detailed balance relation is written for the equilibrium distribution peq (n) as
(147)
0 = w(n, n0 )peq (n0 ) − w(n0 , n)peq (n) .
It always holds for one–step processes in one–dimensional systems with closed
boundaries further considered in our paper. Of course, each equilibrium state is
by definition also stationary. If the initial probability vector p(n, t = 0) is strongly
nonequilibrium, many probabilities p(n, t) change rapidly as soon as the evolution
starts (short–time regime), and then relax more slowly towards equilibrium (long–
time behavior). The final state called thermodynamic equilibrium is reached in the
limit t → ∞.
In the following we consider the three–level system thermodynamically. For
thermodynamic systems the detailed balance typically holds. It means that for an
arbitrary chosen pair of states i, j the stationary probability flux between them is
equal to zero
or in mathematical terms
(148)
eq
Jij := wji peq
i − wij pj = 0 .
Following Section 3 we use the symbol peq
i to designate the stationary distribution
function pst
for
the
systems
with
detailed
balance. As it is well known for such
i
a system, the energy Hi specifies the equilibrium distribution via the Boltzmann
formula
1
exp{−βHi } .
(149)
peq
i =
Z
Let us fix some state, for example, state 1. Then for any state i the detailed balance
reads
(150)
eq
wi1 peq
1 = w1i pi .
By virtue of (149) expression (150) is rewritten as
w1i
(151)
= exp{−β(H1 − Hi )} ,
wi1
thereby
(152)
Hi = H1 + kB T [ln w1i − ln wi1 ] .
Expression (152) actually enables us to construct the energy Hi using the given
transition rates wij . If H1 is known we can calculate any Hi . In particular, for
the three–level system this construction of the energies Hi is actually based on the
following maximal tree
5. SYSTEM WITH DETAILED BALANCE
53
So, on one hand, the energy Hi and correspondingly the distribution of a thermodynamical system can be constructed using one maximal tree only. On the
other hand, the Kirchhoff diagram technique deals with all the maximal trees being actually independent. In order to elucidate this seeming contradiction and to
propose an approach to constructing the energy H, without applying to a certain
fixed maximal tree, we consider the three–level system as example. The stationary
probability flux, for example, along the edge {12} can be calculated directly by
substituting Kirchhoff’s formula (146) into (148) within the replacement pst → peq ,
yielding
(153)
eq
eq
J12
= w21 peq
1 − w12 p2 = w21 w32 w13 − w12 w23 w31 = 0
The fluxes along {13} and {23} have the same value equal to zero. Expression (153)
provides the condition
(154)
w21 w32 w13 = w12 w23 w31
or using Kirchhoff diagrams the equation (154) is represented as
−
(155)
=
0
The following equalities stem immediately from (154)
(156)
(157)
K1 {Tj }
w12
w32 w13
=
=
,
K2 {Tj }
w21
w23 w31
K1 {Tj }
w13
w12 w23
k2 :=
=
=
.
K3 {Tj }
w31
w21 w32
k1 :=
Thereby for constructing the equilibrium distribution only one column in (141) may
be taken into account. The choice of any number of columns in (141) gives the same
result
k1
k2
1
, peq
, peq
.
(158)
peq
1 =
2 =
3 =
1 + k1 + k2
1 + k1 + k2
1 + k1 + k2
Now we demonstrated how to introduce the energy applying to the notion of
Kirchhoff’s diagrams. By definition, the energy of a state i within a tree Tj is
written as
(159)
Hi {Tj } = −kB T ln Ki {Tj } ,
for example, for state 1 expression (159) reads
(160)
H1 {T1 } = −kB T [ln w12 + ln w13 ] .
For a system with detailed balance the energy Hi {Tj } possesses the following property. The difference between energies of an arbitrary chosen pair of states {i, j}
within the same tree Tk is a constant
(161)
Hi {Tk } − Hj {Tk } = const .
54
6. MASTER EQUATION
By the way of example, we consider the following two trees directed to state 1 and,
similarly, two trees directed to state 2
(162)
Using expression (159) the corresponding energies are
H1 {T1 } = −kB T [ln w12 + ln w13 ] ,
H2 {T1 } = −kB T [ln w21 + ln w13 ] ,
H1 {T2 } = −kB T [ln w32 + ln w13 ] ,
H2 {T2 } = −kB T [ln w31 + ln w32 ] .
The differences between energies are written as
(163)
H1 {T1 } − H2 {T1 } = −kB T [ln w12 + ln w21 ] ,
(164)
H1 {T2 } − H2 {T2 } = −kB T [ln w32 + ln w13 + ln w31 + ln w32 ] .
Expressions (163) and (164) are equal to each other provided the condition (154)
is fulfilled and, thus,
(165)
ln(w32 w13 w21 ) = ln(w31 w23 w12 ) .
Now let us construct the energy Hi of a state i as
3
(166)
Hi =
1X
Hi {Tk } .
3
k=1
Then expression for Hi {Tk } can be rewritten as
(167)
Hi {Tk } = Hi + ∆H{Tk } .
In formula (167) the former term depends only on the state i. In the general case
the latter one should depend on the state i as well as given tree Tj . However due
to the detailed balance it turns out to depend only on the tree Tj . In fact, for a
fixed tree Tk we can write
(168)
∆Hi {Tk } − ∆Hj {Tk } =
3
3
X
1 X
0
0
Hi {Tj } −
Hj {Tj } = 0 .
= Hi {Tk } − Hj {Tk } −
3 0
0
j =1
j =1
By virtue of (161) using (167) expression (146) can be rewritten (β = 1/(kB T ))
(169)
exp{−βHi }
,
peq
i = P3
i=1 exp{−βHi }
which has the form of the Boltzmann distribution. We note that these speculations
hold also for multilevel systems with detailed balance.
6. DYNAMICAL PROPERTIES OF MASTER EQUATION
55
6. Dynamical properties of master equation
Using linear algebra we want to solve the master equation analytically by an
expansion in eigenfunctions. This method gives us a general solution of the time
dependent probability vector p(n, t) expressed by eigenvectors and eigenvalues. In
a first step we introduce the master equation, written as a set of coupled linear
differential equations (138), in a compact matrix form
d P(t)
= W P(t) ,
dt
(170)
with a probability vector P(t) = {p(n, t) | n = 0, . . . , N } and an undecomposable
asymmetric transition matrix W = {W (n, n0 ) | n, n0 = 0, . . . , N }. The elements of
the matrix are given by
X
w(m, n)
(171)
W (n, n0 ) = w(n, n0 ) − δn,n0
m6=n
and obey the following two properties
(172)
X
(173)
W (n, n0 ) ≥ 0
for n 6= n0 ,
W (n, n0 ) = 0
for each n0 .
n
As known from matrix theory, there are a number of consequences based on both
properties. Especially the transition matrix W has a single zero eigenvalue whose
eigenvector is the equilibrium probability distribution. In general, other eigenvalues
can be complex and they always have negative real part. In our special case where
the detailed balance (147) holds all eigenvalues are real, as discussed further on.
The solution P(t) of the master equation (170) with given initial vector P(0)
may be written formally as
(174)
P(t) = P(0) exp(W t) ,
P∞
(where exp(W t) = m=0 (W t)m /m!) but this does not help us to find P(t) explicitly.
The familiar method is to make W symmetric and thereby diagonalizable and
then to construct the solution as superposition of eigenvectors uλ related to (zero
or negative) eigenvalues λ in the form
X
(175)
P(t) =
cλ uλ eλ t .
λ
with up to now unknown coefficients cλ . Using the condition of detailed balance
(147) we transform the matrix W = {W (n, n0 )} to a new symmetric transition
f = {W
f (n, n0 )} with elements given by
matrix W
s
eq
0
def
0
0
f (n0 , n) .
f (n, n ) = W (n, n ) p (n ) = W
(176)
W
eq
p (n)
f have the same eigenvalues λi . Due to the symmetry
Both matrices W and W
f
of matrix W, all eigenvalues are real. They may be labeled in order of decreasing
algebraic values, so that λ0 = 0 and λi < 0 for 1 ≤ i ≤ N . Denoting the normalized
56
6. MASTER EQUATION
e i respectively, defined by the eigenvalue equations
eigenvectors by ui and u
X
(177)
W (n, n0 ) ui (n0 ) = λi ui (n)
;
W ui = λi ui
n0
(178)
X
f (n, n0 ) u
W
ei (n0 ) = λi u
ei (n)
;
fu
e i = λi u
ei
W
n0
p
and related by the transformation ui (n) = peq (n) u
ei (n) to each other, we are
ready to construct the time dependent solution of the fundamental master equation (170). According to superposition formula (175), where coefficients cλ are
calculated from the initial condition p(n, 0) at t = 0, the solution is then
#
" N
N
X
X
p
p(m, 0)
λi t
eq
,
u
ei (n) e
(179)
p(n, t) = p (n)
u
ei (m) p
peq (m)
m=0
i=0
or
(180)
p(n, t) =
N
X
i=0
ui (n) eλi t
" N
X
p(m, 0)
ui (m) eq
p (m)
m=0
#
.
This solution plays a very important role in the stochastic description of Markov
processes and can be found in different notations (e. g. as integral representation)
in many textbooks.
As time increases to infinity (t → ∞) only the term i = 0 in the solution
survives and the probabilities tend to equilibrium P(t) → Peq , written as
" N
#
N
X
X
p(m,
0)
(181)
p(n, t) = peq (n) +
ui (n) eλi t
ui (m) eq
.
p (m)
m=0
i=1
In the long–time limit all remaining modes cλ uλ eλ t decay exponentially. In the
short–time regime due to combinations of modes with different signs there is the
possibility of growing and subsequent shrinking of transient states as probability
current from initial distribution P(0) to equilibrium Peq via intermediates P(t) ?.
7. One–Step Processes in Finite Systems
We are speaking about a one–dimensional stochastic process if the state space is
characterized by one variable only. Often this discrete variable is a particle number
n ≥ 0 describing the amount of molecules in a box or the size of an aggregate.
In chemical physics such aggregation phenomena like formation and/or decay of
clusters are of great interest. Examples are the formation of a crystal or glass upon
cooling a liquid or the condensation of a droplet out of a supersaturated vapor. To
determine the relaxation dynamics of clusters of size n we take a particularly simple
Markov process with transitions between neighboring states n and n0 = n ± 1. This
situation is called a one–step process. In biophysics, if the variable n represents the
number of living individuals of a particular species, the one–step process is often
called birth–and–death process to investigate problems in population dynamics. .
Setting the transition rates w(n, n − 1) = w+ (n − 1), w(n, n + 1) = w− (n + 1),
and therefore also w(n + 1, n) = w+ (n), w(n − 1, n) = w− (n), see Fig. 3, now the
7. ONE–STEP PROCESSES IN FINITE SYSTEMS
57
forward master equation (138) reads
(182)
dp(n, t)
= w+ (n − 1) p(n − 1, t) + w− (n + 1) p(n + 1, t)
dt
− [w+ (n) + w− (n)] p(n, t) .
In general the forward and backward transition rates w+ (n), w− (n) are nonlinear
Figure 3. Illustration of a one–step process showing the up and
down or forward and backward transition probabilities between
neighboring states.
functions of the random variable n; the physical dimension of w± is one over time
(s−1 ). The master equation is always linear in the unknown probabilities p(n, t)
to be at state n at time t. It has to be completed by the boundary conditions.
The nonlinearity refers only to the transition coefficients. Further on we will pay
attention to particles as aggregates in a closed box or vehicular jams on a circular
road. Therefore in finite systems the range of the discrete variable n is bounded
between 0 and N (n = 0, 1, 2, . . . , N ).
The general one–step master equation (182) is valid for n = 1, 2, . . . , N − 1, but
meaningless at the boundaries n = 0 and n = N . Therefore we have to add two
boundary equations as closure conditions
dp(0, t)
= w− (1) p(1, t) − w+ (0) p(0, t) ,
dt
dp(N, t)
(184)
= w+ (N − 1) p(N − 1, t) − w− (N ) p(N, t) .
dt
To solve the set of equations we rewrite (182) as balance equation
(183)
dp(n, t)
= J(n + 1, t) − J(n, t)
dt
with probability current defined by
(185)
(186)
J(n, t) = w− (n) p(n, t) − w+ (n − 1) p(n − 1, t) .
In the stationary regime, remember (139), all flows (186) have to be independent of
n and therefore equal to a constant current of probability: J(n + 1) = J(n) = J. In
open systems the stationary solution is no longer unique, it depends on the current
J.
In finite systems with n = 0, 1, 2, . . . , N one finds a situation with zero flux
J = 0, which corresponds to steady state with a detailed balance relationship
58
6. MASTER EQUATION
similar to (147). Therefore the stationary distribution pst (n) fulfills the recurrence
relation
w+ (n − 1) st
(187)
pst (n) =
p (n − 1) .
w− (n)
By applying the iteration successively we get the relation
n
Y
w+ (m − 1)
st
st
(188)
p (n) = p (0)
,
w− (m)
m=1
which determines all probabilities pst (n) (n = 1, 2, . . . , N ) in terms of the first
unknown one pst (0). Taking into account the normalization condition
(189)
N
X
pst (n) = 1
n=0
or
pst (0) +
N
X
pst (n) = 1
n=1
the stationary probability distribution pst (n) in finite systems is finally written as

n
Y

w+ (m − 1)




w− (m)


m=1

n = 1, 2, . . . , N


N Y
k

X

w+ (m − 1)
 1+

w− (m)
(190)
pst (n) =
k=1 m=1





1


n=0.

N Y
k

X

w
(m
−
1)
+



 1+
w− (m)
m=1
k=1
It is often convenient to write the stationary solution (188) in the exponential form
(191)
pst (n) = pst (0) exp {−Φ(n)} ,
where, in analogy to physical systems, the function
n
X
w− (m)
(192)
Φ(n) =
ln
w+ (m − 1)
m=1
is called the potential. An example of a double–well potential Φ(n) and corresponding bistable stationary probability distribution is shown in Fig. 4. As we see, the
minimum of the potential corresponds to the probability maximum and vice versa.
8. Poisson Process in Closed and Open Systems
Up to now we have considered Markov processes in a more general framework
without defining the states of the system as well as the rates for the transitions
between these states precisely. The particular case, where the states are characterized by a single particle number n and the rates by a one–step backward transition
w− (n) only, is called decay process. A schematic realization of such stochastic process is shown in Fig. 5 illustrating dissolution or shrinkage of an bound state of n
members.
In a first step we present an example of traffic flow considered as Markov
process. We want to investigate the dissolution of a queue of cars standing in front
of traffic lights. When the lights switch to green, the first car starts to move. After
8. POISSON PROCESS IN CLOSED AND OPEN SYSTEMS
59
Figure 4. An example of double well potential (top) and the corresponding bistable probability distribution (bottom) depending
on the stochastic variable n.
n
n0
n0-1
n0-2
0
0
t
Figure 5. Sketch of a realization of a stochastic decay process
of Poisson type with shrinking particle number n starting from
n = n0 at t = 0.
a certain time interval (waiting time τ = const > 0) the next vehicle accelerates
to pass the stop line and so on. In our model we consider the decay of traffic
congestion without taking into account any influence of external factors like ramps
or intersections on driver’s behavior. The stochastic variable n(t) is the number of
cars which are bounded in the jam at time t.
When the initial jam size is finite, given by the value n(t = 0) = n0 , shown
in Fig. 5, the trajectory n(t) = n0 , n0 − 1, . . . , 2, 1, 0 consists of unit jumps at
random times. The jam starting with size n0 becomes smaller and smaller and
dissolves completely. In Fig. 6 we have shown three different stochastic trajectories
to illustrate car cluster dissolution. This one–step stochastic process is a death
process only, sometimes called Poisson process.
Defining p(n, t) as the probability to find a jam of size n at time t, the master
equation for the dissolution process reads
∂
p(n, t) = w− (n + 1)p(n + 1, t) − w− (n)p(n, t)
∂t
with the decay rate per unit time assumed as
(193)
(194)
w(n0 , n) = w(n − 1, n) ≡ w− (n) =
1
.
τ
60
6. MASTER EQUATION
Figure 6. Three different stochastic trajectories showing the dissolution of a car queue with the initial length (size) n0 = 50, i. e.,
the cluster size n vs dimensionless time t/τ . The theoretical average value is shown by smooth solid line.
In this approximation the experimentally known waiting time constant τ is a given
control parameter in our escape model. It is a reaction time of a driver, about 1.5
or 2 seconds, to escape from the jam when the road in front of his car becomes free.
Therefore the transition rate (194) is a constant w− = 1/τ independent of jam size
n.
For the described process of jam shrinkage (n0 ≥ n ≥ 0), starting with cluster
size n = n0 and ending with n = 0, we thus obtain the following master equation
including boundary conditions (compare (182) – (184))
(195)
(196)
(197)
∂
1
p(n0 , t) = − p(n0 , t) ,
∂t
τ
∂
1
p(n, t) = [p(n + 1, t) − p(n, t)] ,
∂t
τ
∂
1
p(0, t) = p(1, t)
∂t
τ
n0 − 1 ≥ n > 0 ,
and initial probability distribution p(n, t = 0) = δn,n0 . The delta–function means
that at the beginning the vehicular queue consists of exactly n0 cars.
In order to find the explicit expression of the probability distribution p(n, t)
we have to solve the set of equations (195) – (197). This can be done analytically
starting with the first equation, getting p(n0 , t) = exp(−t/τ ) as exponential decay
function, inserting the solution into the next equation for p(n0 − 1, t), solving it and
continue iteratively up to p(0, t). The general solution of the probability p(n, t) to
8. POISSON PROCESS IN CLOSED AND OPEN SYSTEMS
observe a car cluster of size n at time t is
(t/τ )n0 −n −t/τ
p(n, t) =
e
(198)
,
(n0 − n)!
(199)
p(0, t) = 1 −
61
0 < n ≤ n0 ,
nX
0 −1
(t/τ )m −t/τ
e
.
m!
m=0
As already mentioned (189), the
are always normalized to unity, which
Pnprobabilities
0
p(n, t) inserting (198, 199) to get one. The time
can be proven by summation n=0
evolution of the probability p(n, t) has been calculated from Eqs. (198) and (199)
for an initial queue length n0 = 50. The result is shown in Fig. 7 and compared to
numerical Monte Carlo simulation experiments.
The average or expectation value hni of the cluster size n is usually given by
(200)
hni(t) ≡
n0
X
n p(n, t) =
n=0
n0
X
n p(n, t)
n=1
and can be calculated using the known probabilities (198) to get the exact result
t
Q(n0 − 2, t)
τ
where Q(n, t) is an abbreviation called Poisson term
(201)
hni(t) = n0 Q(n0 − 1, t) −
(202)
Q(n, t) = e−t/τ
n
X
(t/τ )m
def
m=0
m!
.
The variance or second central moment hhnii(t) which measures the fluctuations is
given by
(203)
2
hhnii = h(n − hni) i = hn2 i − hni2
and can be also calculated as follows
2t
hhnii(t) = n0 n0 Q(n0 − 1, t) − Q(n0 − 2, t) (1 − Q(n0 − 1, t))
τ
2
t
t
+
(204)
Q(n0 − 3, t) − Q2 (n0 − 2, t) + Q(n0 − 2, t) .
τ
τ
In some approximation, where we set Q(n, t) (202) to one, the mean value (201)
reduces to a linearly decreasing function in time
(205)
hni(t) ≈ n0 − t/τ ,
whereas the variance (204) to a linearly increasing behavior
(206)
hhnii(t) ≈ t/τ .
In the case of linear mean value approximation (205) the time required, that the
jam dissolves totally, is given by
(207)
tend = n0 τ .
The exact result (203) and the linear approximation (205) for the mean value
depending on time are shown in Fig. 8 by solid and dashed lines, respectively. In
Fig. 9 we have shown the same plots for the variance (204) and its linearization
(206).
62
6. MASTER EQUATION
Figure 7. Probability distribution P (n, t) at 3 different times
(from the top to the bottom) t/τ = 5, 25, and 55 with the initial condition P (n, 0) = δn,50 . Solid lines – the analytical solution,
triangles – Monte Carlo results obtained by simulation of 5000
stochastic trajectories. Note that the pictures have different scales
along the probability axis.
8. POISSON PROCESS IN CLOSED AND OPEN SYSTEMS
63
Figure 8. The mean value hni of the cluster size depending on the
dimensionless time t/τ . The initial size of the cluster is n0 = 50.
The exact result is shown by thick solid line, the linear approximation – by a dashed line.
Figure 9. The variance hhnii depending on the dimensionless
time t/τ . The initial size of the cluster is n0 = 50. The exact
result is shown by thick solid line, the linear approximation – by a
dashed line.
Equations (205) and (206), however, do not describe the final stage of dissolution of any finite car cluster. In this case, taking the limit t → ∞ in the time
dependent results (198) and (199), we have
(208)
lim p(n, t) = δn,0 .
t→∞
If we do not consider the final stage of dissolution of a large cluster, i. e., if t
is remarkably smaller than tend (207), then the probability p(0, t) that the cluster
is completely dissolved is very small. This allows us to obtain correct results for
n > 0 by the following alternative method.
LECTURE 7
Stochastic differential equations and noise-induced
phase transitions
1. Types of Stochastic Differential Equations
An introduction to the rigorous description of stochastic differential equations
is given in many textbooks. Here we turn to a more informal constructions which
are actually justified by results presented in these textbooks.
For an ordinary dynamical system with, for example, a one–dimensional phase
space x ∈ R the governing equation is written as
dx
(209)
= f (x, t) ,
dt
where f (x, t) is a force acting on the system. From a naive point of view one might
just replace the force f (x, t) by the sum of the regular component f (x, t) and the
random Langevin source g(x, t)ξ(t) to describe a similar stochastic system rewriting
the governing equation as
dx
(210)
= f (x, t) + g(x, t)ξ(t) .
dt
Unfortunately, such a generalization is justified only in the case when the intensity
of Langevin sources does not depend on the phase variable, i. e. g(x, t) = g(t). To
understand this fault we consider the model, where
(211)
f (x, t) = 0 ,
g(x, t) = 1 + x ,
and 1 is a small parameter. Initially the system is assumed to be located at
the origin x(t = 0) = 0. Then, within the accuracy up to the first order in , the
solution of equation (210) can be obtained by iteration yielding
Zt
(212)
x(t) =
dt0 ξ(t0 ) + 0
Zt
dt0
0
Zt
0
dt00 ξ(t0 )ξ(t00 ) .
0
By virtue of
hξ(t)i = 0 and hξ(t)ξ(t0 )i = δ(t − t0 ) ,
the value of x(t) averaged over all the realizations of white noise x(t) is given by
the expression
Zt
(213)
hx(t)i = 0
dt0
Zt
0
dt00 δ(t0 − t00 ) .
0
Exactly such expressions are responsible for ambiguities in the formal use of differential equations for describing nonlinear stochastic systems. The matter is that the
point where the Dirac δ-function differs from zero lies at the boundary t0 = t00 of
65
66
7. STOCHASTIC DIFFERENTIAL EQUATIONS AND NOISE-INDUCED PHASE TRANSITIONS
Figure 1. Illustration of the fault met in describing stochastic
systems with nonlinear Langevin forces. See discussion preceding
expression (213).
the integration region (Fig. 1). In this case it is not clear which part of δ-function
should be ascribed to the integration region or, what is the same, which part is
outside it.
In order to overcome this problem, one can apply to the notion of infinitesimals,
i. e., the hyperreal numbers. Here we will confine our consideration to the physical
level of rigor. Dealing again with a standard dynamical system, its governing
equation (209) can be rewritten using infinitesimals as
(214)
dx = f (x, t)dt ,
where dx and dt are infinitesimally small increment of variable x and time step.
Roughly speaking, in the standard dynamical systems there is only one infinitesimally small variable, the time step dt. All the other infinitesimals are derived
from it by multiplying dt by some smooth function. From the standpoint of such a
microscopic level the distinction between systems with regular and stochastic dynamics is mainly due to stochastic ones possessing two or more really independent
infinitesimals, the time step dt and the infinitesimal moments {dW (t)} of the random Langevin forces. In the case under consideration there is only one moment of
random force, dW (t). Here time t is used at the symbol dW (t) in order to note the
relation of the random force moment to the current time. At different moments t, t0
of time the quantities dW (t) and dW (t0 ) are supposed to be mutually independent.
It is possible to think about dW (t) as the integral of white noise ξ(t)
t+dt
Z
(215)
dt0 ξ(t0 )
dW (t) =
t
which, however, is no more than a qualitative explanation of the properties ascribed
to the random force moment dW (t). Namely, they are
(216)
hdW (t)i = 0
and
[dW (t)]2 = dt .
1. TYPES OF STOCHASTIC DIFFERENTIAL EQUATIONS
67
Figure 2. Intermediate point xθ that determines the values of
forces acting on the system during the time interval (t, t + dt).
The latter equality
√ enables us to estimate the amplitude of the random force moment as dW (t) ∼ dt. In what follows it will be seen that a stochastic infinitesimal
quantity
affects the system dynamics only if it amplitude scales with the time step
√
as dt, whereas all the essential regular infinitesimals scales as dt. Therefore dealing with [dW (t)]2 we can ignore its random component and regard it as a regular
infinitesimal. In other words, the following relationship
(217)
[dW (t)]2 = dt
is adopted. In some sense it specifies the algebraic manipulations with infinitesimals
of stochastic systems.
Now we are ready to write down the governing equation for the given stochastic
system in infinitesimals. However, before doing this it is worthwhile to note that
the notion of infinitesimals enables one to pose a question about the point xθ at
which the force f (xθ , t) in equation (214) should be calculated. In particular, it is
naturally to set (Fig. 2)
(218)
xθ = x + θdx
where 0 ≤ θ ≤ 1 is a certain constant or a smooth function of x. For the standard
dynamical systems, however, this shift in equation (215) from the point x to point
xθ has no sense because it gives rise to addition terms on the left-hand side of (215)
having no effect on the system dynamics. Namely,
∂f
θdxdt ,
(219)
f (x, t)dt ⇒ f (xθ , t) ≈ f (x, t)dt +
∂x
and due to dx ∝ dt in the given case the last term scales as (dt)2 which is ignorable.
It is not so if the Langevin force undergoes the shift x → xθ as will be seen below.
Keeping in mind this possibility of dealing with the intermediate points {xθ }
we apply to the following equation in infinitesimals
(220a)
dx = fθ (xθ , t)dt + g(xθ , t)dW (t)
to describe the stochastic system with nonlinear Langevin force. This equation
actually inherits the structure of the formal equation (210) after its “integration”
from t to t + dt and the formal relationship (215) between the random infinitesimal
dW (t) and white noise ξ(t). In other words, the given stochastic system is characterized by the regular force fθ (x, t), the intensity g(x, t) of the random Langevin
force, and the parameter θ specifying the intermediate point xθ which determines
the magnitudes of these forces.
68
7. STOCHASTIC DIFFERENTIAL EQUATIONS AND NOISE-INDUCED PHASE TRANSITIONS
Table 1. Basic types of stochastic systems
θ
0
1/2
1
Name of stochastic process
Itô process
Stratonovich process
Hänggi–Klimontovich process
Concluding the construction of the given governing equation we note the following. First, the introduced basic parameter θ has no analogy in the standard
dynamical systems. Moreover it characterizes the physical properties of a given
system reflecting its features at the microscopic level. The subscript θ for the function fθ (x, t) is used to underline that it belongs to the triplet {f, g, θ}. Second,
in the general case fθ (x, t) is a vector and g(x,
√ t) as well as θ(x, t) is some tensor.
Third, as it follows from (220a), dx ∝ dW ∝ dt holds at the leading order in dt.
Therefore the difference
df
fθ (xθ , t)dt − fθ (x, t)dt ≈
θ(dt)3/2
dx
can be ignored because it scales as (dt)3/2 with time step dt and, therefore, is an
infinitesimal of higher order than dt. So the replacement
fθ (xθ , t) = fθ (x, t)
in equation (220a) will be adopted further. In this case equation (220a) reads
(220b)
dx = fθ (x, t)dt + g(xθ , t)dW (t) .
Fourth, special types of stochastic equations have individual names (see Table 1).
The reasons for this will be clear below.
2. Relationship between different type stochastic differential equations
As we have mentioned above, a stochastic system is described by the triplet
{fθ , g, θ} which aggregates the system properties on the microscopic level, and the
question concerning the specific value of θ should be corresponded to physics rather
than mathematics. For example, diffusion in solids is the Hänggi–Klimontovich
process whereas current induced by temperature gradient is the Ito process. Nevertheless, for a given system it is possible to make use of the description {fθ , g, θ},
where the value of parameter θ is chosen
for specific purposes. It is due to the
fact that the collection of triplets {fθ , g, θ} θ , where the form of the function
g(x, t) is fixed, gives the equivalent description of stochastic dynamics provided the
regular forces {fθ } meet some relationship. To show this we expand the function
g(xθ , t) = g(x+θdx, t) in the Taylor series with respect to the infinitesimal quantity
dx and cut it off at the first order term
∂g(x, t)
dx .
g(xθ , t) = g(x, t) + θ
∂x
As√
it follows from equation (220), at the leading order in dt, i. e. within the accuracy
of dt, the infinitesimals dx and dW are related by the expression dx = g(x, t)dW ,
so
θ ∂g 2 (x, t)
g(xθ , t) = g(x, t) +
dW .
2
∂x
3. TRANSFORMATION OF RANDOM VARIABLES
69
Substituting the latter expression into equation (220b) and taking into account
(217), it is reduced to the following Ito type stochastic governing equation
θ ∂g 2 (x, t)
(221)
dx = fθ (x, t) +
dt + g(x, t)dW (t) .
2
∂x
Therefore all the triplets {fθ , g, θ} with a given intensity g(x, t) of Langevin forces
actually describe the same stochastic system if the regular forces {fθ (x, t)} belong
to a family of functions such that
θ ∂g 2 (x, t)
,
2
∂x
where f0 (x, t) is the regular component of the Ito type triplet. A similar relationship
for the regular force of the Hänggi–Klimontovich type is written as
(222)
f0 (x, t) = fθ (x, t) +
(1 − θ) ∂g 2 (x, t)
.
2
∂x
As a closing remark to the present Section we note that the Ito representation
of a stochastic system has an advantage in describing the system dynamics as an
array of succeeding steps {dx} because the probability of the system transition from
the state x to the state x + dx is specified completely by its properties at the state x
only. The properties making the other two types of stochastic processes distinctive
will be discussed below.
(223)
f1 (x, t) = fθ (x, t) −
3. Transformation of Random Variables
Conversion from the random variable x to a new one y poses a question of
finding the governing equation for the random process y(t) ⇐ x(t) provided the
governing equation of the system dynamics described in terms of the variable x is
known, for example, it is equation (220). It is natural to consider this problem
within the assumption that the old and new random processes are characterized by
the same value of the parameter θ. Unfortunately, the standard rules valid for the
ordinary differential equations do not hold, in general, for stochastic systems. The
matter is that even for a smooth function x = ϕ(y) the expansion
1
ϕ(y + dy) = ϕ(y) + ϕ0 (y)dy + ϕ00 (y)(dy 2 ) + . . .
2
contains terms proportional to dt from two sources. The former is the second term
in this expansion due to the presence of the regular force. The latter is the third
term due to the contribution of the random force moment, namely, (dW )2 = dt.
To derive the transformation rule for the stochastic system under consideration
we make use of the following equalities
(224)
xθ = θϕ[yθ + (1 − θ)dy] + (1 − θ)ϕ[yθ − θdy]
1
= ϕ[yθ ] + θ(1 − θ)ϕ00 [yθ ](dy)2
2
and
(225)
dx = ϕ[yθ + (1 − θ)dy] − ϕ[yθ − θdy]
1
0
= ϕ [yθ ]dy +
− θ ϕ00 [yθ ](dy)2 ,
2
70
7. STOCHASTIC DIFFERENTIAL EQUATIONS AND NOISE-INDUCED PHASE TRANSITIONS
where yθ = y + θdy. In equation (220a) the argument xθ of the functions fθ (xθ , t)
and g(xθ , t) can be replaced
√ with another quantity deviating from xθ by an infinitesimal of order less than dt. Thus, by virtue of (225) the transformation of the
variables x = ϕ(y) is implemented, first, using the direct replacement
fθ (xθ , t), g(xθ , t) 7→ fθ [ϕ(yθ ), t], g[ϕ(yθ ), t] .
Second, the substitution of (224) into equation (220a) converts it to
1
0
(226)
ϕ [yθ ]dy +
− θ ϕ00 [yθ ](dy)2 = fθ [ϕ(yθ ), t]dt + g[ϕ(yθ ), t]dW (t) .
2
√
Whence it follows that at the leading order in dt, i.e. within the accuracy dt the
infinitesimals dy and dW are related as
dy =
g[ϕ(yθ ), t]
dW (t) .
ϕ0 [yθ ]
Taking into account rule (217), formula (226) leads us to the required governing
equation written for the random variable y related to x as x = ϕ(y)
1
1
ϕ00 [yθ ]
dy = 0
fθ [ϕ(yθ ), t] + θ −
g 2 [ϕ(yθ ), t] 0
dt
ϕ [yθ ]
2
(ϕ [yθ ])2
g[ϕ(yθ ), t]
+
(227)
dW (t) .
ϕ0 [yθ ]
It is the second term in the braces that makes the transformation of random variables distinct from one for ordinary dynamical systems. However, if the random
process is of the Stratonovich type, i.e. θ = 1/2, this term vanishes and the transformation of random variables takes the standard form
(228)
dy =
1
ϕ0 [y1/2 ]
f1/2 [ϕ(y1/2 ), t] dt +
g[ϕ(y1/2 ), t]
dW (t) .
ϕ0 [y1/2 ]
Exactly this property endows the Stratonovich representation of stochastic processes with advantage in dealing with a description based on various transformations of the system phase space.
4. Forms of the Fokker–Planck Equation
The relationship between the stochastic differential equations dealing with individual realizations of stochastic trajectories and the Fokker–Planck equation describing the dynamics of the distribution function P (x, t) is considered previously.
Here we will make use of the results presented there.
For the systems under consideration the diffusion coefficient is calculated as
follows
(dx)2
g 2 (x, t)
=
.
D(x, t) =
2 dt
2
Thus, for a stochastic system governed by the Ito type triplet {f0 , g, 0} the Fokker–
Planck equation takes the form
"
#
∂ 1 ∂ g2 P
∂P
(229)
=
− f0 P .
∂t
∂x 2 ∂x
4. FORMS OF THE FOKKER–PLANCK EQUATION
71
In order to write the corresponding equation for a system with a general triplet
{fθ , g, θ}, we can make use of the Fokker–Planck equation (229) after passing to
the equivalent Ito representation according to formula (222). In this way we get
"
#
∂P
∂ g 2θ ∂ g 2(1−θ) P
(230)
=
− fθ P .
∂t
∂x 2
∂x
In particular, for the Stratonovich processes, where θ = 1/2, the Fokker–Planck
equation is of the form
∂P
∂ g ∂ (gP )
(231)
=
− f1/2 P ,
∂t
∂x 2 ∂x
whereas for the Hänggi–Klimontovich processes it becomes
∂P
∂ g 2 ∂P
(232)
=
− f1 P .
∂t
∂x 2 ∂x
The Hänggi–Klimontovich representation of a stochastic dynamics is singled
out by the relation between the steady state distribution P st (x), the regular force
f (x), and the intensity g(x) of Langevin source. Namely, for an autonomous system,
i.e. when its properties does not depend on time, the steady state solution P st (x)
of equation (232) obeys the equality
g 2 (x) dP st
− f1 (x)P st = 0 .
2
dx
This equality is rather natural for unbounded one–dimensional systems, whereas
for multi–dimensional systems it also is the case when the detailed balance holds.
Direct integration of the latter equality gives us the expression for the steady state
distribution
Z x
1
2f1 (x0 ) 0
st
(233)
P (x) = exp
dx .
Z
g 2 (x0 )
Dealing with the initial triplet {fθ , g, θ} of the general type, formula (233) can be
rewritten as
Z x
1
2fθ (x0 ) 0
(234)
P st (x) =
exp
dx
g 2 (x0 )
Zg 2(1−θ) (x)
by virtue of (223). Here the coefficient Z is specified by the normalization condition
(235)
+∞
Z
P st (x) dx = 1 .
−∞
Naturally all these integrals should exist. Expression (233) will be used in the
analysis of the noised–induced phase transitions in the next section.
In the following two Sections we consider typical examples of the stochastic
systems undergoing nonequilibrium phase transition caused by nonlinear Langevin
sources.
The ability of noise to produce order under certain conditions, in particular,
to give rise to various cooperative phenomena and the corresponding phase transitions is now a well established fact. Such phase transitions manifest themselves
in the phase–space distribution changing its structure, for example, the number of
maxima.
72
7. STOCHASTIC DIFFERENTIAL EQUATIONS AND NOISE-INDUCED PHASE TRANSITIONS
5. Equilibrium and Nonequilibrium Phase Transitions
To elucidate the main features of the nonequilibrium phase transitions under
consideration let us, first, discuss briefly the basic notions of the equilibrium phase
transitions of second order. We make use of the Landau order parameter theory and
for the sake of simplicity consider a one-dimensional system with a nonconservative
scalar order parameter h(x, t). Its dynamics is governed by the equation
∂h
δH{h}
=−
+ gξ(t) ,
∂t
δh
where Γ is the kinetic coefficient, g is the amplitude of the random Langevin force
proportional to white noise ξ(t) whose averages meet the equalities
(236)
Γ
(237)
hξ(t)i = 0
and
hξ(t)ξ(t0 )i = δ(t − t0 ) ,
and the functional H{h} is the free energy typically written in the form
+∞
Z
1 2
2
` (∇h) + H(h)
H{h} =
dx
2
−∞
with the free energy density H(h) specified by the expression
1
1
α(T )h2 + βh4 .
2
4
Here, by definition, the symbol ∇ := ∂/∂x, the characteristic spatial scale `, the
coefficient β > 0, as well as the kinetic coefficient Γ > 0 and the noise amplitude g
are certain constant parameters of the given system, whereas the coefficient α(T )
depends on the temperature T . In the case of ` = 0, it changes the sign at the
critical temperature T = Tc . The following ansatz with some constant α0
H(h) =
α(T ) = α0
(T − Tc )
Tc
is usually adopted. In these terms the governing equation (236) becomes
∂h
= `2 ∇2 h − α(T )h − βh3 + gξ(t) .
∂t
For T > Tc the system possesses only one phase state matching the stable minimum of the free energy h(x) = 0 (Fig. 3). When the temperature drops below
the critical value, T < Tc , the free energy changes in form. Two new additional
extrema appear which are stable, whereas the previous one becomes unstable. As
a result, p
the homogeneous state decays, giving rise to composition of domains with
h± = ± |α|/β. This bifurcation is shown in Fig. 3 exhibiting also the resulting evolution of the probability function P (h) of the order parameter. It should be
pointed out that the width of the distribution P (h) is determined by the intensity g
of the Langevin forces. Within the frameworks of the given model the temperature
dependence of the coefficient α(T ) and the intensity g of the Langevin forces are
independent characteristics of the system properties. It is a rather formal statement
because on the microscopic level both of them are related to stochastic motion of
particles forming this system. Nevertheless, dealing with equilibrium phase transitions one can regard the random forces as just a source of disordering and the
appearance of new phases becomes pronounced when the difference between the
(238)
Γ
5. EQUILIBRIUM AND NONEQUILIBRIUM PHASE TRANSITIONS
73
Figure 3. Schematic illustration of the mechanism of the second order phase transitions in equilibrium systems. The upper
left frame shows the free energy density above and below the critical value of temperature, which is labeled with numbers 1 and
2, respectively. The resulting regular force is shown in the lower
left frame. The averaged order parameter vs. the temperature is
demonstrated in the upper right frame, whereas the lower right
frame depicts the distribution function of the order parameter
again above and below the phase transition.
new phase branches, (h+ − h− ), exceeds essentially the Langevin force intensity, g,
in magnitude.
In some sense the nonequilibrium phase transitions are distinguished from equilibrium ones by the fact that the phenomena causing them cannot be described
within models similar to (236) without substantial modification.
Below in this Section we will consider actually two types of nonequilibrium
phase transitions that are due to the creative action of noise. The former appears
in systems where the amplitude g(h) of the random Langevin forces depends essentially on the order parameter h, for example, tends to zero as h → 0. The
latter comes into being via anomalous behavior of the kinetic coefficient Γ(h) depending on the order parameter being now some vector h = {h1 , h2 , . . .}. Namely,
the phase space {h1 , h2 , . . .} of such systems contains a narrow layer, “low dimensional” domain called the dynamical trap region, where the kinetic coefficient Γ(h)
takes extremely large values. So, when the system enters this region its dynamics
is stagnated, which gives rise to long-lived states treated as new dynamical phases.
74
7. STOCHASTIC DIFFERENTIAL EQUATIONS AND NOISE-INDUCED PHASE TRANSITIONS
Figure 4. Illustration of the approach to detecting phase transitions in nonequilibrium systems what are caused by creative action
of noise.
In all these cases noise gives rise to a certain ordering in addition to random perturbations in the system motion. As a result, after the system having undergone
phase transition, the difference (h+ − h− ) between the mean values of the order
parameter h in the new phases and the width of local extrema of the distribution
function P (h) are of the same order in magnitude. In some sense they are due to
the same phenomena and, thus, cannot be analyzed independently of each other.
So, such transitions are typically detected by analyzing the distribution function
and fixing the transition between, e.g., the unimodal distribution to bimodal one
(Fig. 4) as a control parameter changes.
6. The Verhulst Model of Third Order
This model is related to one–dimensional systems with fading
√ dynamics near
the stationary point x = 0 or between two stable points x = ± λs (when λs > 0),
where the returning force contains linear and nonlinear component like this
(239)
dx
= λ(t)x − x3 ,
dt
with the coefficient λ(t) = λs + σξ(t) being not a constant but exhibiting random
fluctuations in time. The intensity of these fluctuations is quantified by the parameter σ. When the amplitude of these fluctuations exceeds some critical value,
the coefficient λ(t) changes the sign and during a short time the system behavior
becomes anomalous, for example, the point x = 0 becomes unstable. The Verhulst
model actually describes the competition between these random events of the system instability and the regular fading. The corresponding governing equation is of
the form
(240)
dx = λs x − x3 + σxθ dW (t)
6. THE VERHULST MODEL OF THIRD ORDER
75
and the parameter θ is assumed to be given in the interval 0 < θ < 1. Therefore
fθ (x) = λs x − x3 and g(x) = σx hold for this system. In what follows we will
analyze only the properties of the stationary distribution P st (x).
Formula (234) together with the normalization condition (235) immediately
gives us the desired function
2
2β−1
x
|x|
1
st
exp − 2
(for β > 0) ,
(241)
P (x) =
Γ(β)σ σ
σ
where the parameter β = β(λs , σ, θ) is
(242)
β=
λs
1
+θ−
σ2
2
and Γ(p) is the gamma function.
Let us consider two possible cases with λs > 0 or λs < 0 individually. When
λs > 0 and σ = 0 √
the system has one unstable stationary point x = 0 and two
stable ones x = ± λs according to (239). The random Langevin force of low
intensity, σ 1, can only disturb the system motion near these points. In the
given limit β > 12 holds and the distribution function is bimodal (Fig. 5). This
bimodality, however, is not due to the noise effect but is a result of the regular
force structure. As the noise intensity grows, the Langevin force destroys it, and for
0 < β < 21 the stationary distribution P st (x) becomes unimodal. For large values of
σ, corresponding to β < 0, the system undergoes collapse indicated formally by the
fact that function (241) has a nonintegrable singularity at x = 0. Accordingly, when
the system wandering in space gets the origin, it will never leave it. This “ordering”
is really due to the noise effect. In the given case the stationary distribution takes
the form of the Dirac δ-function
(243)
P st (x) = δ(x)
(for β > 0) .
Figure 5 visualizes this evolution of the stationary distribution as the intensity of the random Langevin forces increases. According to (241), the system is
characterized by the bimodal stationary distribution if
s
λs
(244)
σ < σc1 :=
.
(1 − θ)
√
For σ 1 its extrema are located near x ± λs . In the given case, the random
Langevin force affecting the system dynamics mainly disturbs its motion. We point
out that for the Hänggi–Klimontovich process, regarded here as the limit θ → 1 − 0,
it is the only possible behavior of the system. For the intermediate values of the
noise intensity σ, namely,
s
λs
(245)
σc1 < σ < σc2 :=
1
2 −θ
the Langevin force destroys the ordering by regular force but cannot order the
system motion itself. For the stochastic processes with θ ≥ 12 the critical value σc2
does not exist and the noise effect is purely destructive. It is so, in particular, for
the Stratonovich type processes. When θ < 21 and the noise intensity is rather high,
(246)
σ > σc2 ,
76
7. STOCHASTIC DIFFERENTIAL EQUATIONS AND NOISE-INDUCED PHASE TRANSITIONS
Figure 5. The stationary distribution function of the Ito type
Verhulst random process for different values of the noise intensity
σ. The upper left frame visualizes the distribution function for
σ < σc1 , the upper right frame shows it at the critical value σ =
σc1 , whereas the lower fragments exhibit the distribution function
in the case σc1 < σ < σc2 for two values of σ located near the
boundaries of this interval. In drawing the curves
the parameter
√
λs = 1 was used for which σc1 = 1 and σc2 = 2.
the Langevin force causes the system collapse at the origin. It means that after
some finite time the system will be inevitably trapped at the point x = 0.
If the parameter λs < 0, the Verhulst system without noise admits only one
stationary point x = 0 being stable. In this case a noise of low intensity cannot
destroy the system collapse at the origin and the stationary distribution function
is the Dirac δ-function. It is justified by the divergence of integral (235) for solution (241) with β < 0. The latter inequality is the case for small values of σ, as
it follows from expression (242). For a stochastic system with θ > 12 the Langevin
force, however, destroys this collapse when its intensity σ exceeds some critical
value
s
|λs |
∗
.
(247)
σ > σc :=
θ − 12
Here again the Langevin force plays a destructive role.
We note that the phenomena described by the given Verhulst model are mainly
the effects of the order breakdown proceeding via phase transitions caused by the
8. NOISE INDUCED INSTABILITY IN THE GEOMETRIC BROWNIAN MOTION
77
nonlinearity of the Langevin force. In the next section a system with the creative
role of nonlinear Langevin forces will be considered.
7. The Genetic Model
This model reflects characteristic features in the description of population genetics. By way of example we analyze it within a rather simplified formulation
dealing with the variable X determined inside the interval (0, 1) whose dynamics is
governed by the equation
dX
1
= − X + λ(t)X(1 − X) ,
dt
2
where the parameter λ(t) undergoes random fluctuations in time. For the sake of
simplicity we convert to the new variable x = (2X − 1) and assume the process
under consideration to be of the Stratonovich type. In this case the governing
equation in infinitesimals is written as
(248)
dx = −xdt + σ 1 − x2θ dW ,
where, as previously, the parameter σ characterizes the noise intensity. Besides, the
system is assumed to be localized initially at some point of the interval −1 < x < 1.
Here the regular force f1/2 (x) describes pure damping relaxation toward the origin,
x = 0, the unique stationary point being stable without the noise effect. The
intensity of the random Langevin forces, g(x) = σ(1 − x2 ), drops essentially at the
points x = ±1, which is responsible for the anomalous system behavior.
In this case formula (234) together with condition (235) gives
1
1
1
1
−1
st
,
(249)
P (x) = exp
K0
exp − 2
2σ 2
2σ 2 (1 − x2 )
σ 1 − x2
where K0 (. . .) is the modified Bessel function of the second kind of order 0. Analyzing directly this expression, we conclude that the distribution function P st (x)
is of the unimodal form when the noise intensity is rather low, namely, σ < 1. For
σ > 1, the Langevin force induces essential deviation of the system from the origin
and the distribution function become biomodal (Fig. 6). In particular, the function
P st (x) attains its maximum at the points
√
1 − σ2
xm = ±
.
σ
In the given system it is the nonlinear Langevin force that causes the formation of
new two phases, which is desired example of the constructive action of nonlinear
random forces.
8. Noise Induced Instability in the Geometric Brownian Motion
Both of the examples considered in the previous two Sections demonstrate
the fact that the system distribution can change its form depending on the noise
intensity when the Langevin force are essentially nonlinear. At first glance it could
be possible to have a think that the found transitions in the distribution form are no
more than some “artifacts” and can be eliminated by converting to the appropriate
78
7. STOCHASTIC DIFFERENTIAL EQUATIONS AND NOISE-INDUCED PHASE TRANSITIONS
Figure 6. The stationary distribution function of the
Stratonovich type Genetic random process for different values of the noise intensity σ.
variables y = ψ(x). Indeed, this transformation of variables leads to the conversion
of the probability density P (x) 7→ p(y), namely,
1
(250)
p(y) = 0
P (x) ,
ψ (x)
and the nonlinearity of the transformation y = ψ(x) can eliminate the distribution
bimodality.
The remaining part of the present section is the counterexample of this statement. It is provided by geometric random walks. The process is governed by the
following equation in infinitesimals
(251)
dx = βxdt + σxθ dW (t) ,
where the system motion is considered in the half–space x > 0 with the coefficient
β = ±1, and, as previously, the parameter σ quantifies the noise intensity. The
parameter θ of this process is assumed to be a given constant not equal to 1/2, i.e.
0 ≤ θ < 1/2 or 1/2 < θ ≤ 1. This model provides a rather simplified description of
the birth–death processes in many biological and ecological objects.
Let us convert to the new variable
(252)
x = exp(y) .
Then, by virtue of (227), equation (251) becomes
1
(253)
dy = β + θ −
σ 2 dt + σdW (t) .
2
Whence it follows that, if the Langevin source is rather weak, namely,
s
2
(254)
σ < σc :=
,
|2θ − 1|
8. NOISE INDUCED INSTABILITY IN THE GEOMETRIC BROWNIAN MOTION
79
Figure 7. The distribution function for geometric random walks
governed by equation (251) for different moments of time {ti } such
that σ 2 t1 = 0.1, σ 2 t2 = 1, σ 2 t3 = 2, and σ 2 t4 = 3. The curves
in figure are labeled with the corresponding numbers and the used
values ∆ = β/σ 2 + θ − 1/2 are shown also. The system initially
was located at x0 = 1.
then its presence cannot affect the system dynamics essentially. In other words, in
the y-space the system drifts either to −∞ or +∞, depending on the sign of the
parameter β. In the x-space it reflects in the system tending to the origin x or to
infinity with probability equal to unity as time goes on. When the noise intensity
increases the critical value, σ > σc , the system motion can change the direction.
It is the case when β = −1 and θ > 1/2 or β = 1 and θ < 1/2. This actually
demonstrates the instability of the point x = 0 caused by multiplicative noise.
Finalizing this Section, we present the distribution function p(y, t) and P (x, t)
written for the random variables y and x governed by equation (253). The latter
describes the standard diffusion process with constant drift, thereby,
( 2 )
(y − y0 ) − (β + (θ − 21 )σ 2 )t
1
p(y, t) = √
exp −
(255a)
,
2σ 2 t
2πσ 2 t
( 2 )
ln(x/x0 ) − β + θ − 12 σ 2 t
1
(255b)
P (x, t) = √
exp −
,
2σ 2 t
x 2πσ 2 t
where x0 = exp(y0 ) are the coordinates of the initial position of the particle and
relation (250) has been also taken into account. Figure 7 visualizes these functions
for different moments of time. It also should be noted that the geometric random
walks have rather anomalous properties. To demonstrate this fact, let us analyze
80
7. STOCHASTIC DIFFERENTIAL EQUATIONS AND NOISE-INDUCED PHASE TRANSITIONS
the time dependence of the moments of the variable x
Z∞
(256)
Mp (t) := xp P (x, t)dx for p = 1, 2, . . .
0
Substituting (255) into (256), a direct integration yields
p − 1 2
(257)
Mp (t) = exp β + θ +
σ pt .
2
So, when the effect of noise is ignorable, σ → 0, the system either goes to zero or
to infinity, depending on the sing of β. For the Langevin force of finite intensity,
σ > 0, there is always such an order p that
β
+
θ
p>1−2
σ2
and the moment Mp (t) diverges as t → ∞. In particular, even for θ = 0 and β = 0,
the first order moment M1 does not depend on time, whereas the second moment
M2 (t) goes to infinity as exp{σ 2 t}. In other words, even if the mean value of the
random variable x is fixed, its random fluctuations unboundedly grow in time.
9. System Dynamics with Stagnation
Now we pass to the consideration of another type phase transitions induced
by the action of noise, namely, phase transitions caused by dynamical traps. They
occur in systems where the kinetic coefficient Γ(h), see equation (236), depends
essentially on the system state h.
Originally the development of the dynamical trap notion or, more precisely,
the dynamical traps of the stagnation type was stimulated by a wide class of intricate cooperative phenomena found in the dynamics of various systems, e. g.,
vehicle ensembles moving on highways, fish and bird swarms, stock markets, etc.
The background of the models to be developed is the following. People as elements of a certain system cannot individually control all the governing parameters.
Therefore one chooses a few crucial parameters and mainly focuses attention on
them. When the equilibrium with respect to these crucial parameters is attained,
the human activity slows down, retarding in turn the system dynamics as a whole.
For example, in driving a car the control over the relative velocity is of prime importance in comparison with the correction of the headway distance. So, under
normal conditions a driver should first reduce the relative velocity between his car
and the car ahead and only then optimize the headway. In markets, the deviation
from the supply-demand equilibrium, reflected in price changes, also has to exhibit
faster time variations than, e.g., the production cost determined by technology
capabilities.
These speculations have led us to the concept of dynamical traps, i. e. a certain
“low” dimensional region (trap region) in the phase space where all the main kinetic
coefficients exhibit an anomalous behavior. As a result, all the time scales of the
system dynamics in the trap region become large in comparison with their values
outside it. The latter effect, in turn, causes long–lived states to appear in such a
system. In time patterns these states manifest themselves in a sequence of fragments
within which at least one of the phase variables remains approximately constant.
These fragments are continuously connected by sharp jumps of the given variable.
10. OSCILLATOR WITH DYNAMICAL TRAPS
81
Figure 8. Illustration of the dynamical trap effect. The left frame
depicts the phase space with the region of the dynamical traps
where the system motion is stagnated. The right frame shows the
time pattern for one of the phase state variables.
10. Oscillator with Dynamical Traps
In order to elucidate the mechanism of such nonequilibrium phase transitions,
the present Section analyzes a model derived from the damping harmonic oscillator
with the dynamical trap region being a narrow layer in the phase space, where the
particle velocity is equal to zero. Namely, the following system is under consideration
dx
(258)
= v,
dt
dv
σ
2
= −ω0 Ω(v) x +
(259)
v + 0 ξv (t) .
dt
ω0
Here x and v are the dynamical variables treated as the coordinate and velocity of
a certain particle, ω0 is the circular frequency of oscillations provided the system is
not affected by other factors, σ is the damping decrement, and the term 0 ξv (t) in
equation (259) is a random Langevin force of intensity 0 proportional to the white
noise ξv (t),
(260)
hξv (t)i = 0 ,
hξv (t)ξv (t0 )i = δ(t − t0 ) ,
with unit amplitude. At Ω(v) = 1 and σ = 0 the system of equations (258) to (259)
corresponds to the classical harmonic oscillator. Here we treat another case, where
the function Ω(v) describes the dynamical trap effect in the vicinity of v = 0. The
following simple ansatz
(261)
Ω(v) =
v 2 + 42 ϑ2t
,
v 2 + ϑ2t
is adopted, where the parameter ϑt characterizes the thickness of the trap region and
the parameter 4 ≤ 1 measures the trapping efficacy. When 4 = 1 the dynamical
trap effect is ignorable, for 4 = 0 it is most effective. It should be pointed out that
the governing equations (258) and (259) are written in terms of time derivatives
82
7. STOCHASTIC DIFFERENTIAL EQUATIONS AND NOISE-INDUCED PHASE TRANSITIONS
Figure 9. Characteristic structure of the phase space {x, v}. The
shadowed domain represents the trap region where the regular force
is depressed and the system motion is random. The regular force
depression is described by the factor Ω(v) illustrated in the left
window. The essence of the trap effect on the system dynamics
is shown in the right window. Outside the trap region the system
dynamics is mainly regular.
because in the given case the noise is additive and the result is independent of the
random process parameter θ.
The characteristic features of the given system are illustrated in Fig. 9. The
shadowed domain shows the trap region where the regular force, the former term
in Eq. (259), is depressed. The latter is described by the factor Ω(v) taking small
values in the trap region (for 4 1). Inside the trap region the system is mainly
governed by the random Langevin force. Outside the trap region it is approximately
harmonic.
In order to analyze the system dynamics, a dimensionless time t and the dynamical variables η and u are used. Namely, the time t is measured in units of
1/ω0 , i.e., t → t/ω0 and the units of the coordinate x and the velocity v are ϑt /ω0
and ϑt , respectively. So, by introducing the new variables
xω0
v
η=
and u =
,
ϑt
ϑt
the dynamical equations (258), (259) read (for the dimensionless time t)
dη
du
= u,
= −Ω[u] (η + σu) + ξ(t) ,
dt
dt
where the noise ξ(t) obeys conditions like equalities (260), the parameter is =
√
0 /( ω0 ϑt ), and the function Ω[u] is given by
(262)
Ω[u] =
u 2 + 42
.
u2 + 1
10. OSCILLATOR WITH DYNAMICAL TRAPS
83
Without noise, this system has only one stationary point {η = 0, u = 0} being
stable because it possesses a Lyapunov function
2
η2
u2
1 − 42
u + 42
(263)
H(η, u) =
.
+
+
ln
2
2
2
42
This Lyapunov function attains the absolute minimum at the point {η = 0, u = 0}
and obeys the inequality
dH(η, u)
= −σu2 < 0 for u 6= 0 .
dt
In particular, if σ = 0 and = 0, then function (263) is the first integral of the
system. In what follows, the values σ and will be treated as small parameters.
The dynamics of system (262) was analyzed numerically using a high order
stochastic Runge–Kutta method. The distribution function P(η, u) was calculated
numerically by finding the cumulative time during which the system is located
inside a given mesh on the (η, u)-plane for a path of a sufficiently long time of
motion, t ≈ 500000. The size of mesh was chosen to be about 1% of the dimension
characterizing the system location on the (η, u)-plane.
The evolution of the distribution function P(η, u) is shown in Fig. 10 in the
form of the level contours dividing the variation scale into ten equal parts. The
upper window corresponds to the case of 4 = 1 where the trap effect is absent
and the distribution function is unimodal. The third window illustrates the case
when the distribution function has the well pronounced bimodal shape shown also
in Fig. 11. Comparing the three upper windows in Fig. 10, it becomes evident that
there is certain relation Φc (4, σ, ) = 0 between the parameters 4, σ, and when
the system undergoes a second order phase transition, which manifests itself in the
change of the shape of the phase space density P(η, u) from unimodal to bimodal.
In particular, for σ = 0.1 and = 0.1 the critical value of the parameter 4 is
4c (σ, ) ≈ 0.5, as it is seen in the second window.
To understand the mechanism of the noise induced phase transition observed
numerically in the given system, consider a typical fragment of the system motion
through the trap region for 4 1 that is shown in Fig. 12. When it goes into the
trap region Qt , −ϑt v ϑt , the regular force Ω[u] (η + σu) containing the trap
factor Ω[u] and governing the regular motion becomes small. Hence, inside this
region the system dynamics becomes random due to the remaining weak Langevin
force ξ(t). However, the boundaries ∂+ Qt (where v ∼ ϑt ) and ∂− Qt (where v ∼
−ϑt ) are not identical in properties with respect to the system motion. At the
boundary ∂+ Qt the regular force leads the system inwards the trap region Qt ,
whereas at the boundary ∂− Qt it causes the system to leave the region Qt . Outside
the trap region Qt the regular force is dominant. Thereby, from the standpoint of
the system motion inside the region Qt , the boundary ∂+ Qt is “reflecting” whereas
the boundary ∂− Qt is “absorbing”.
As a result, the distribution of the residence time at different points of the region
Qt should be asymmetric, as schematically shown in Fig. 9(the right window). This
asymmetry is also seen in the distribution function P(η, u) obtained numerically.
Its maxima are located at the points with non-zero values of the velocity, which is
clearly visible in the lower window of Fig. 10. Therefore, during location inside the
trap region the mean velocity of the system must be positive and it tends to go away
from the origin. This effect gives rise to an increase in the “energy” H(η, u). Outside
(264)
84
7. STOCHASTIC DIFFERENTIAL EQUATIONS AND NOISE-INDUCED PHASE TRANSITIONS
Figure 10. Evolution of the distribution function P(η, u) (shown
by level contours) as the parameter 4 decreases. In numerical
calculations the values σ = 0.1 and = 0.1 were used. The lower
window depicts only one maximum of the distribution function.
the trap region the “energy” H(η, u) decreases according to expression (264). So,
when the former effect becomes sufficiently strong, i.e., the random force intensity
exceeds a certain critical value, > c (4, σ), the distribution function P(η, u)
becomes bimodal.
11. Dynamics with Traps in a Chain of Oscillators
The previous Section was devoted to the mechanism via which phase transitions
induced by dynamical traps arise. The present Section demonstrates the dynamical
traps, in fact, to give rise to cooperative phenomena that can be regarded as the
formation of new phases.
11. DYNAMICS WITH TRAPS IN A CHAIN OF OSCILLATORS
Figure 11. The form of the distribution function P(η, u) for the
parameters σ = 0.1, = 0.1, and 4 = 0.2.
Figure 12. A typical fragment of the system path going through
the trap region. The parameters σ = 0.1, = 0.1, and 4 = 0.01
were used in numerical simulations in order to make the trap effect
more pronounced.
85
86
7. STOCHASTIC DIFFERENTIAL EQUATIONS AND NOISE-INDUCED PHASE TRANSITIONS
Figure 13. The particle ensemble under consideration and the
structure of the phase space. The darkened region depicts the
points where the dynamical trap effect is pronounced. For the relationship between the variables xi , vi , hi , and ϑi see formulas (268)
and (269).
We analyze a one-dimensional ensemble of “lazy” particles. These particles
are characterized by their positions and velocities {xi , vi } as well as possess some
motives for active behavior. Particle i “wishes” to get the “optimal” middle position
between the nearest neighbors. Thus, one of the stimuli for it to accelerate or
decelerate is the difference ηi = xi − 21 (xi−1 +xi+1 ) provided its relative velocity ϑi =
vi − 21 (vi−1 + vi+1 ) with respect to the pair of the nearest neighbors is sufficiently
low. Otherwise, especially if particle i is currently located near the optimal position,
it has to eliminate the relative velocity ϑi , being the other stimulus for particle i
to change its state of motion. Since a particle cannot predict the dynamics of its
neighbors, it has to regard them as moving uniformly with the current velocities.
The acceleration dvi /dt is determined directly by both stimuli. The model to be
formulated combines both of these stimuli within a linear approximation (ηi + σϑi ),
where σ is the relative weight of the second stimulus.
When, however, the relative velocity ϑi attains sufficiently low values, the current situation for particle i cannot become worse, at least, rather fast. In this case
particle i “prefers” not to change the state of motion and to retard the correction
of its relative position. This assumption leads to the appearance of some common
cofactor Ω(ϑi ) in the corresponding governing equation like
dvi
∝ −Ω(ϑi ) ηi + σϑi .
dt
The cofactor Ω(ϑ) has to meet the inequality Ω(ϑ) 1 for ϑ ϑc and Ω(ϑ) ≈ 1
when ϑ ϑc , where ϑc is a critical value quantifying the particle “perception” of
speed. The inclusion of such a factor is the implementation of the dynamical trap
effect.
Now let us describe the model. The following linear chain of N point–like
particles is considered (Fig. 13). Each internal particle i 6= 1, N can freely move
along the x-axis interacting with the nearest neighbors, namely, particles i − 1
and i + 1 interact via ideal elastic springs with some quasi–viscous friction. The
11. DYNAMICS WITH TRAPS IN A CHAIN OF OSCILLATORS
87
dynamics of this particle ensemble is governed by the collection of coupled equations
dxi
(265)
= vi ,
dt
dvi
(266)
= −Ω(ϑi , hi )[ηi + σϑi + σ0 vi ] + ξi (t) .
dt
Here for i = 2, 3, . . . , N − 1 the variables ηi and ϑi to be called the symmetry
distortion and the distortion rate, respectively, are specified as
(267)
(268)
1
ηi = xi − (xi−1 + xi+1 ) ,
2
1
ϑi = vi − (vi−1 + vi+1 ) ,
2
the mean distance hi between the particles at the point xi , by definition, is
1
(xi+1 − xi−1 ) ,
2
and {ξi (t)} is the collection of mutually independent white noise sources of unit
amplitude, i.e.
(269)
hi =
(270)
ξi (t) = 0 ,
ξi (t)ξi0 (t0 ) = δii0 δ(t − t0 ) .
Besides, the parameter is the noise amplitude, σ is the viscous friction coefficient
of the springs, σ0 is a small parameter that can be treated as some viscous friction
related to the particle motion with respect to the given physical frame. It is introduced to prevent the system motion as a whole reaching infinitely high velocity.
The symbol h. . .i denotes averaging over all the noise realizations, δii0 and δ(t − t0 )
are the Kronecker symbol and the Dirac δ-function. The factor Ω(ϑi , hi ) is due to
the effect of dynamical traps and, following our previous ansatz, we write
(271)
Ω(ϑ, h) =
ϑ2 + 42 (h)
,
ϑ2 + 1
where the function 4(h) of the form
(272)
42 (h) = 42 + 1 − 42
h20
h2 + h20
is used. The parameter 4 ∈ [0.1] quantifies the dynamical trap influence and
the spatial scale h0 specifies the small distances within which the trap effect is
depressed, i.e. for h h0 its value is 4(h) ≈ 1, whereas for h h0 /4 it is
4(h) ≈ 4. If this parameter is 4 = 1, then the dynamical traps do not exist at
all. In the opposite case, 4 1, their influence is pronounced inside a certain
neighborhood of the h-axis (trap region) whose thickness is about unity (Fig. 17).
The temporal and spatial scales have been chosen so that the thickness of the trap
region is about unity, as well as the oscillation circular frequency is also equal to
unity outside the trap region. The terminal particles, i = 1 and i = N , are assumed
to be fixed, i.e.
(273)
x1 (t) = 0 ,
xN (t) = (N − 1)l ,
where l is the particle spacing in the homogeneous chain. The particles are treated
as mutually impermeable ones. Therefore, when the coordinate xi and xi+1 of an
internal particle pair become identical, an absolutely elastic collision is assumed
88
7. STOCHASTIC DIFFERENTIAL EQUATIONS AND NOISE-INDUCED PHASE TRANSITIONS
to happen, i.e. if xi (t) = xi+i (t) at a certain time t, then the timeless velocity
exchange
(274)
vi (t + 0) = vi+1 (t − 0) ,
vi+1 (t + 0) = vi (t − 0)
comes into being. Multiparticle collisions are ignored. The system of equations (??)
– (274) forms the model under consideration. We note again that the Langevin
sources enter this model linearly, so the governing equations admit the representation with time derivatives.
The stationary point xst
i = (i−1)l is stable with respect to small perturbations.
It stems from the linear stability analysis with respect to perturbations of the form
(275)
δxi (t) ∝ exp{γt + ikl(i − 1)} ,
where γ is the instability increment, k is the wave number, and the symbol i denotes
the imaginary unit. The boundary conditions (273) are fulfilled by assuming the
wave number k to take the values km = πm/[(N −1)l] for m = ±1, ±2, . . . , ±(N −2).
For large values of the particle number N the parameter k can be treated as a
continuous variable. Using the standard technique, the system of equations (265),
(266) for perturbation (275) leads us to the following relation between the instability
increment γ(k) and the wave number k:
kl 1
(276) γ = −Ω0 σ0 + σ sin2
2
2
s
kl kl 2
1
− Ω20 σ0 + σ sin2
.
+ i 2Ω0 sin2
2
2
2
Ansatz (271) has been used in deriving expression (276), enabling us to set Ω0 =
Ω(0, l) = 42 (l). Whence it follows that Re γ(k) > 0 for k > 0, so the homogeneous
state of the chain is stable with respect to infinitely small perturbations of the
particle arrangement.
The nonlinear dynamics of the given system has been analyzed numerically.
The integration of the stochastic differential equations (??), (266) was performed
using the E2 high order stochastic Runge–Kutta method ??. Particle collisions
were implemented analyzing a linear approximation of the system dynamics within
one elementary step of the numerical procedure and finding the time at which a
collision has happened. Then this step, treated as a complex one, was repeated.
The integration time step of 0.02 was used, the obtained results were checked to be
stable with respect to decreasing the integration time step. The ensemble of 1000
particles was studied in order to make the statistics sufficient and to avoid a strong
effect of the boundary conditions. The integration time T was chosen from 5000
to 8000 time units in order to make calculated distributions stable. At the initial
stage all the particles were distributed uniformly in space, whereas their velocities
were randomly and uniformly distributed within the unit interval. The results of
numerical simulation were used to evaluate the following partial distributions
N −M ZT
(277)
X
1
P(z) =
(N − 2M )(T − T0 )
i=M T
dt δ(z − zi (t)) ,
0
11. DYNAMICS WITH TRAPS IN A CHAIN OF OSCILLATORS
89
where the time dependence zi (t) describes the dynamics of one of the variables
ηi (t), ϑi (t), and vi (t) ascribed to particle i. Here z is a given point of the space
Rz describing the symmetry distortion η, the distortion rate ϑ, and the particle
velocity v, respectively. The variables {η, ϑ, v} enable one to represent the system
dynamics portrait within the space Rη × Rϑ × Rv or its subspace, N is the total
number of particles in the ensemble, and M is the number of particles located near
each of its boundaries. They are excluded from the consideration in order to weaken
a possible effect of the specific boundary conditions. The same is true for the lower
boundary of time integration T0 . Its value is chosen to eliminate the effect of the
specific initial conditions. The numerical implementation of the integration over
time in expression (277) was related to the direct summation of the obtained time
series. The partition of the corresponding space Rz was chosen such that the results
are practically independent of the cell size. The value of M was also chosen using
the stability of the result with respect to the double increase in M . The values of
M ∼ 50 and T0 ∼ 500–1000 were used.
Let us, first, discuss local properties of these ensembles. The term “local” means
that the corresponding state variable can take practically independent values when
the particle index i changes by one or two. The variable ηi (expression (267)) may
be regarded in such a manner. It describes the symmetry of particle arrangement
in space. When ηi = 0, particle i takes the middle position between the nearest
neighbors, particles i − 1 and i + 1. A nonzero value of ηi denotes its deviation from
this position, in other words, a local distortion of the ensemble symmetry. The
latter was the reason for the used name of the variables ηi as well as the variables
ϑi = dηi /dt.
Figure 14 shows the distribution of the variables η and ϑ depending on the
dissipation rate σ and the initial distance l between particles, i.e. their mean
density.
In the case of weak dissipation, the distribution functions of the symmetry distortion P(η) possesses two maxima, matching the effect described in the previous
Section. Noise makes the uniform particle distribution unstable and the particles
spend the main time in the vicinity of one or the other neighbors. It leads to the
bimodal distribution of the symmetry distortion η. After entering the region of
dynamical traps the particle motion is stagnated, whereas outside it particles move
relatively fast. This fact is reflected in the found distribution of the distortion rate
ϑ containing actually two components of different scales. The narrow component is
due to the particle motion inside the trap region. It should be practically independent of the mean distance between particles. By contrast, the wide one depends
remarkably on the particle density because it matches the fast motion of particles
outside the trap region and, thus, has to be affected by their relative dynamics.
Exactly this effect is demonstrated in Fig. 14 visualizing also the corresponding
properties of the particle paths.
In the case of strong dissipation, σ ≈ 1.0, the situation changes dramatically,
although the characteristic scales of the corresponding distributions turn out to be
of the same order in magnitude. Now the distribution function P(η) of the symmetry distortion has only one maximum at η = 0, however, its form is characterized by
two scales. In other words, it looks like a sum of two monoscale components. One
of them is sufficiently wide, its thickness is about the same value that is obtained
for the corresponding particle ensemble with weak dissipation. This component
90
7. STOCHASTIC DIFFERENTIAL EQUATIONS AND NOISE-INDUCED PHASE TRANSITIONS
Figure 14. The distribution functions of the symmetry distortion
η and the distortion rate ϑ for the 1000 particle ensemble with
low (l = 50, label 1) and high (l = 5, label 2) density and weak
(σ ≈ 0.1) and strong (σ ≈ 1.0) dissipation. The lower four windows
depict characteristic path fragments of duration of 1000 time units
formed by a single particle with index i = 500 on the phase plane
{η, ϑ} which was chosen due to its middle in the given ensemble.
The other parameters used are the noise amplitude = 0.1, the
trap effect measure 4 = 0.1, the small regularization friction coefficient σ0 = 0.01 and the regularization spatial scale h0 = 0.25. The
time interval within which the data were averaged changed from
2000 to 5000 in order to make the obtained distributions stable.
11. DYNAMICS WITH TRAPS IN A CHAIN OF OSCILLATORS
91
exhibits a remarkable dependence on the particle density, enabling us to relate it
to the particle motion outside the trap region. The other is characterized by an
extremely narrow and sharp form shown in detail in the inner window of Fig. 14 for
the dense particle ensemble. Its sharpness leads us to the assumption that “many–
particle” effects in such systems with dynamical traps cause the symmetrical state
to be singled out from the other possible states concerning the system properties.
By contrast, the distortion rate behaves similarly as in the previous case except for
some details. When the mean particle density is high (l = 5) the wide component of
the distortion rate distribution disappears and only the narrow one remains, with
the latter having a quasi-cusp form ∝ exp{−|ϑ|}. For the system with low density,
the peak of the distortion rate distribution splits into two small spikes.
These features can be explained by referring to the frames in Fig. 14, that exhibit typical path fragments formed by motion of a single particle on the {ηϑ}-plane.
Roughly speaking, now three motion types can be singled out: some stagnation inside a narrow neighborhood of the origin {η = 0 , ϑ = 0}, slow wandering inside the
trap region that, on average, follows a line with a finite positive slope, and the fast
motion outside the trap region. The fast motion fragments typically stem from an
arbitrary point of the low motion region and lead to certain neighborhood of the
origin. It seems that the systems with low density of particles have the possibility to go sufficiently far from the origin and during the fast motion come into the
stagnation region rarely. As a result, first, the distortion rate distribution function
is of a two scale form and contains two spikes on the peak. In the case of high
density, the fast motion is depressed substantially and the system migrates mainly
in the slow motion region entering the stagnation region many times. Thus, the
distortion rate distribution converts into a single-scale function and the symmetric
state occurs often, giving rise to a significant sharp component of the distortion
distribution located near the point η = 0.
Now let us discuss the nonlocal characteristics of the 1000-particle ensembles.
Figure 15 depicts the velocity distributions. As we see, it depends essentially on
both parameters, the mean particle density and the dissipation rate. When the
mean particle density is low and the dissipation is weak (l = 50 and σ ≈ 0.1),
the velocity distribution is practically of Gaussian form, however, its width has
extremely large values about 10. The tenfold increase of the particle density, l :
50 7→ 5, shrinks the velocity distribution to the same order and its scale gets values
similar to that of the distortion rate distribution in magnitude. However, in this case
the form of the velocity distribution is a monoscale function of the well pronounced
cusp form ∝ exp{−|v|}. In the case of strong dissipation (σ ≈ 1.0) the situation is
opposite. The system with low density (l = 50), as previously, is characterized by
an extremely wide velocity distribution, its width is about 10. However, now its
form deviates substantially from the Gaussian one. For the corresponding ensemble
with high density (l = 5) the velocity distribution is Gaussian with width about
1. The latter, nevertheless, is much larger than the same width in the absence of
dynamical traps.
These features of the velocity distribution characterizes the cooperative behavior of particles rather than their individual dynamics. In other words, there should
be strong correlations in the motion of not only neighboring particles but also distant ones. Therefore the velocity variations responsible for the formation of such
distributions describe in fact the motion of multiparticle clusters. To justify this,
92
7. STOCHASTIC DIFFERENTIAL EQUATIONS AND NOISE-INDUCED PHASE TRANSITIONS
Figure 15. The distribution functions of the particle velocities
and the characteristic time patterns formed by the velocity variations of the 500-th particle. Dynamics of the 1000-particle ensemble with low (l = 50, label 1) and high (l = 5, label 2) mean density
and weak (σ ≈ 0.1) and strong (σ ≈ 1.0) dissipation was implemented for the calculation time up to 8000 time units to make the
obtained distributions stable with respect to time increase. The
lower panels visualize the time patterns formed by 200 paths of
particle motion during 1000 time units and chosen in the middle of
the given ensemble. Here the curve thickness has been chosen so
that the difference in brightness can depict local variations in the
path spacing due to changes either in the particle density or in the
velocities of cooperative particle motion (in this way the different
long-lived states of the given particle ensemble become apparent).
The other parameters used are the noise amplitude = 0.1, the
trap effect measure 4 = 0.1, the small regularization friction coefficient σ0 = 0.01 and the regularization spatial scale h0 = 0.25.
12. SELF–FREEZING MODEL FOR MULTILANE TRAFFIC
93
we refer to the middle column windows in Fig. 15. They demonstrate some typical fragments of the time patterns formed by the velocities of individual particles.
When the mean particle density is low (l = 50), these patterns look like a sequence
of fragments {vα } inside which the particle velocity varies in the vicinity of some
level vα . The values {vα } are rather randomly distributed inside a certain region
of thickness V ∼ 10 in the vicinity of v = 0. The continuous transitions between
these fragments occur via sharp jumps. The typical duration of these fragments is
about T ∼ 100, which enables us to regard them as long-lived states because the
temporal scales of individual particle dynamics are about several units. Moreover,
these long-lived states can persist only if a group of many particles moves as a whole
because the characteristic distance L individually traveled by a particle involved in
such state is about L ∼ V T ∼ 1000 l.
The spatial structure of these cooperative states is visualized in Fig. 15 depicting also time patterns formed by paths {xi (t)} of 200 particles of duration about
1000 time units. These particles were chosen in the middle part of the 1000-particle
ensembles with low density. For high density ensembles such patterns also develop,
but are not so pronounced. As we see, a large number of different mesoscopic states
are formed in these systems. They differ from one another in size, the direction of
motion, the speed, the life time, etc. Moreover, the life time of such a state can be
much longer than the characteristic time interval during which particles forming it
currently will belong to this state individually. Besides, the patterns found could
be classified as hierarchical structures. Some relatively small domains formed by
cooperative motion of individual particles in their turn make up together larger superstructures. In other words, the observed long-lived cooperative states have their
“own” life independent, in some sense, of the individual particle dynamics. The
latter properties are the reason for regarding them as certain dynamical phases
arising in the systems under consideration due to the dynamical traps affecting
the individual particle motion. The term “dynamical” has been used to underline
that the complex cooperative motion of particles is responsible for these long-lived
states, without the continuous particle motion such states cannot exist.
12. Self–Freezing Model for Multilane Traffic
The present Section is devoted to a model for multilane congested traffic flow,
where the dynamical traps give rise to the continuum of long–lived states observed
in real traffic. When vehicles move on a multilane highway without changing the
lanes, they interact practically with the nearest neighbors ahead only. The more
frequently lane changing is performed, the more correlated traffic flow is on a multilane highway. Therefore, to characterize traffic flow on multilane highways, it is
reasonable to introduce an additional state variable, the order parameter η ?. In
this case the mean velocity v of multilane traffic flow is determined by both the
vehicle density ρ and the order parameter η, namely, v = ϑ(η, ρ).
However, to describe phase transitions in the cooperative vehicle motion, we
have to treat the multilane car interaction into more detail. The matter is that for a
car to change a lane, the local vehicle arrangement at the neighboring lanes should
be of a special form, otherwise, it will be frustrated for a certain time, as illustrated
in Fig. 16. For car 1 to be able to overtake car 2 the neighboring car 3 should
provide a room for this maneuver. In the opposite case the driver of car 1 has
to wait and the local car arrangement will not vary substantially. In other words,
94
7. STOCHASTIC DIFFERENTIAL EQUATIONS AND NOISE-INDUCED PHASE TRANSITIONS
Figure 16. Schematic illustration of the car arrangement at the
neighboring lane in the synchronized mode that enables overtaking
(a) or hinders it (b).
changes in the particular realizations of the local car arrangement can be frozen for
a certain time although the globally optimal car configuration is not attained at
the current time moment. Due to this self-freezing effect, the synchronized mode
can comprise a great amount of locally metastable states and correspond to certain
two-dimensional region in the ρq-plane rather than to a line q = ϑ(ρ)ρ. This feature
seems to be similar to that met in physical media with local order, for example,
in glasses where phase transitions are characterized by wide range of controlling
parameters (temperature, pressure, etc.) rather than their fixed values.
We should specify the evolution of the order parameter a to complete the
description of the long–lived state continuum of the cooperative car motion. Since
the order parameter a allows for microscopic details of the fundamental cluster
structure, its fluctuations will be treated as a random noise whose amplitude depend
on the vehicle density only. In contrast, the rate da/dt of time variations in the
order parameter a has to be affected substantially by the current value of the
order parameter η. In fact, as the order parameter η tends to the local optimum
value η0 (a, ρ) for the given a, the rate da/dt should be depressed because all the
drivers forming the fundamental cluster prefer to wait until a more comfortable car
configuration arises therefore inhibiting the evolution of the fundamental cluster
structure.
The dimensionless model describing these effects in the congested multilane
traffic takes the form
(278)
dη = −(η − a2 )dt + dWη (t) ,
(279)
da = −Ω0 $(η, a)adt + Ω0 $1/2 (η, a)dWa (t) ,
1/2
where the infinitesimal moments of random Langevin forces dWη and dWa are
mutually independent and the random process is of the Ito type. Finally, the
function $(η, a) may be specified as:
(η − a2 ) 2 /φ20 if η − a2 ≤ φ0
(280)
$(η, a) =
.
1
if η − a2 > φ0
12. SELF–FREEZING MODEL FOR MULTILANE TRAFFIC
95
Figure 17. The time pattern of the order parameters η and a if
the self-freezing effect is suppressed. The window (a) exhibits time
variations in the order parameter η in the case where the influence
of the order parameter a has been ignored at all. The window (b)
presents the dynamics of η affected by a.
Here η = a2 is the optimal value of the order parameter η attained for the
given value of the car ensemble arrangement, and a = 0 matches the totally optimal
structure of the car ensemble. When the car arrangement attains local optimum for
a fixed value of a, the drivers “do not know what to do” and the system dynamics
is stagnated. In other words, the curve η = a2 is the locus of dynamical traps and
factor (280) describes them, with φ0 being the threshold of the driver perception.
Let us now discuss the results obtained by simulating numerically the system
of Eqs. (278), (279) for Ω0 = 1, = 0.1, and φ0 = 0.5. First of all, Fig. 17
illustrates the evolution of the order parameters η and a when there is no selffreezing. In this case the order parameter a exhibits the standard random pattern of
Brownian movement inside a region of unit width. We see a collection of practically
independent spikes of unit width. A similar pattern (see Fig. 17(a)) is demonstrated
by the dynamics of the order parameter η provided the interaction of the parameters
96
7. STOCHASTIC DIFFERENTIAL EQUATIONS AND NOISE-INDUCED PHASE TRANSITIONS
Figure 18. The time pattern of the order parameters η and a
when the self-freezing effect is substantial.
η and a have been ignored, i.e., (η − a2 ) replaced by η. Naturally, in this case the
synchronized mode matches a line on the ρq-plane for 1. The dynamics of the
order parameter η, affected by the variable a but without the reciprocal influence,
is shown in Fig. 17(b). Again a collection of spikes can be seen, whose amplitude
as well as width has increased tenfold.
The dynamics changes dramatically for the full problem, see Fig. 18. The time
pattern takes a form corresponding to the long-lived state continuum. When the
point {a(t), η(t)} representing the current state of the synchronized mode wanders
on the aη-plane and reaches the curve η = a2 at any point, it will be trapped for a
certain time until it finally escapes from the trap due to the noise dWη (t). After
that the system again wanders in the aη-plane during a time interval about unity
before being trapped for the next time. Since the characteristic duration of the
trapping is much longer than unity, the pattern looks like a certain collection of
local metastable states of the synchronized mode. However, such prolonged stays
of the system are not metastable in the rigorous meaning and we preferred to call
them simply the long-lived states. Since each point of the curve η = a2 is a trap,
the long–lived states make up a certain continuum.
12. SELF–FREEZING MODEL FOR MULTILANE TRAFFIC
97
The problem of a car following a lead car driven with constant velocity is well
known. To derive the governing equations for the dynamics of the following car
a cost functional is constructed. This functional ranks the outcomes of different
driving strategies. Assuming rational–driver behavior, the existence of the Nash
equilibrium is proved.
Bibliography
99