Crispin Gardiner
Stochastic Methods
A Handbook for the Natural and Social Sciences
Fourth Edition
A
\
<£J Springer
Contents
1.
A Historical Introduction
1
1.1 Motivation
1
1.2 Some Historical Examples
2
1.2.1 Brownian Motion
2
1.2.2 Langevin's Equation
6
1.3 The Stock Market
8
1.3.1 Statistics of Returns
8
1.3.2 Financial Derivatives
9
1.3.3 The Black-Scholes Formula
10
1.3.4 Heavy Tailed Distributions
10
1.4 Birth-Death Processes
11
1.5 Noise in Electronic Systems
14
1.5.1 Shot Noise
14
1.5.2 Autocorrelation Functions and Spectra
18
1.5.3 Fourier Analysis of Fluctuating Functions: Stationary Systems 19
1.5.4 Johnson Noise and Nyquist's Theorem
20
2.
Probability Concepts
2.1 Events, and Sets of Events
2.2 Probabilities
2.2.1 Probability Axioms
2.2.2 The Meaning of P(A)
2.2.3 The Meaning of the Axioms
2.2.4 Random Variables
2.3 Joint and Conditional Probabilities: Independence
2.3.1 Joint Probabilities
2.3.2 Conditional Probabilities
2.3.3 Relationship Between Joint Probabilities of Different Orders
2.3.4 Independence
2.4 Mean Values and Probability Density
2.4.1 -Determination of Probability Density by Means
of Arbitrary Functions
2.4.2 Sets of Probability Zero
2.5 The Interpretation of Mean Values
2.5.1 Moments, Correlations, and Covariances
2.5.2 The Law of Large Numbers
2.6 Characteristic Function
2.7 Cumulant Generating Function: Correlation Functions and Cumulants
2.7.1 Example: Cumulant of Order 4: «X1X2X3X4))
2.7.2 Significance of Cumulants
2.8 Gaussian and Poissonian Probability Distributions
23
23
24
24
25
25
26
27
27
27
28
28
29
30
30
31
32
32
33
34
36
36
37
X
Contents
2.9
3.
2.8.1 The Gaussian Distribution
2.8.2 Central Limit Theorem
2.8.3 The Poisson Distribution
Limits of Sequences of Random Variables
2.9.1 Almost Certain Limit
2.9.2 Mean Square Limit (Limit in the Mean)
2.9.3 Stochastic Limit, or Limit in Probability
2.9.4 Limit in Distribution
2.9.5 Relationship Between Limits
Markov Processes
3.1 Stochastic Processes
3.1.1 Kinds of Stochastic Process
3.2 Markov Process
3.2.1 Consistency—the Chapman-Kolmogorov Equation
3.2.2 Discrete State Spaces
3.2.3 More General Measures...,
3.3 Continuity in Stochastic Processes
3.3.1 Mathematical Definition of a Continuous Markov Process ..
3.4 Differential Chapman-Kolmogorov Equation
3.4.1 Derivation of the Differential Chapman-Kolmogorov Equation
3.4.2 Status of the Differential Chapman-Kolmogorov Equation ..
3.5 Interpretation of Conditions and Results
3.5.1 Jump Processes: The Master Equation
3.5.2 Diffusion Processes—the Fokker-Planck Equation
3.5.3 Deterministic Processes—Liouville's Equation
3.5.4 General Processes
3.6 Equations for Time Development in Initial Time—Backward
Equations
3.7 Stationary and Homogeneous Markov Processes
3.7.1/ Ergodic Properties
3.7.2 Homogeneous Processes
3.7.3 Approach to a Stationary Process
3.7.4 Autocorrelation Function for Markov Processes
3.8 Examples of Markov Processes
3.8.1 The Wiener Process
3.8.2 The Random Walk in One Dimension
3.8.3 Poisson Process
3.8.4 The Ornstein-Uhlenbeck Process
3.8.5 Random Telegraph Process
37
38
39
40
40
41
41
41
41
42
42
42
43
44
44
45
45
46
47
48
51
51
51
52
54
55
55
56
57
60
60
63
65
65
68
71
72
75
Contents
4.
5.
The Ito Calculus and Stochastic Differential Equations
4.1 Motivation
4.2 Stochastic Integration
4.2.1 Definition of the Stochastic Integral
4.2.2 Ito Stochastic Integral
4.2.3 Example j£ W(t')dW(t')
4.2.4 The Stratonovich Integral
4.2.5 Nonanticipating Functions
4.2.6 Proof that dW{t)2 = dt and dW(t)2+N = 0
4.2.7 Properties of the Ito Stochastic Integral
4.3 Stochastic Differential Equations (SDE)
4.3.1 Ito Stochastic Differential Equation: Definition
4.3.2 Dependence on Initial Conditions and Parameters
4.3.3 Markov Property of the Solution of an Ito SDE
4.3.4 Change of Variables: Ito's Formula
4.3.5 Connection Between Fokker-Planck Equation and
Stochastic Differential Equation
4.3.6 Multivariable Systems
4.4 The Stratonovich Stochastic Integral
4.4.1 Definition of the Stratonovich Stochastic Integral
4.4.2 Stratonovich Stochastic Differential Equation
4.5 Some Examples and Solutions
4.5.1 Coefficients without x Dependence
4.5.2 Multiplicative Linear White Noise Process—Geometric
Brownian Motion
4.5.3 Complex Oscillator with Noisy Frequency
4.5.4 Ornstein-Uhlenbeck Process
4.5.5 Conversion from Cartesian to Polar Coordinates
4.5.6 Multivariate Ornstein-Uhlenbeck Process
4.5.7 The General Single Variable Linear Equation
4.5.8 Multivariable Linear Equations
4.5.9 Time-Dependent Ornstein-Uhlenbeck Process
The Fokker-Planck Equation
5.1 Probability Current and Boundary Conditions
5.1.1 Classification of Boundary Conditions
5.1.2 Boundary Conditions for the Backward FokkerPlanck Equation
5.2 Fokker-Planck Equation in One Dimension
5.2.1 Boundary Conditions in One Dimension
5.3 Stationary Solutions for Homogeneous Fokker-Planck Equations...
5.3.1 Examples of Stationary Solutions
5.4 Eigenfunction Methods for Homogeneous Processes
5.4.1 Eigenfunctions for Reflecting Boundaries
5.4.2 Eigenfunctions for Absorbing Boundaries
XI
77
77
79
79
81
81
82
82
83
85
88
89
91
92
92
93
95
96
96
96
99
99
100
101
103
104
105
108
110
Ill
113
114
116
116
117
118
120
122
124
124
126
XII
Contents
5.4.3 Examples
First Passage Times for Homogeneous Processes
5.5.1 Two Absorbing Barriers
5.5.2 One Absorbing Barrier
5.5.3 Application—Escape Over a Potential Barrier
5.5.4 Probability of Exit Through a Particular End of the Interval .
126
130
130
132
133
135
The Fokker-Planck Equation in Several Dimensions
6.1 Change of Variables
6.2 Stationary Solutions of Many Variable Fokker-Planck Equations . . .
6.2.1 Boundary Conditions
6.2.2 Potential Conditions
6.3 Detailed Balance
6.3.1 Definition of Detailed Balance
6.3.2 Detailed Balance for a Markov Process
6.3.3 Consequences of Detailed Balance for Stationary Mean,
Autocorrelation Function and Spectrum
6.3.4 Situations in Which Detailed Balance must be Generalised .
6.3.5 Implementation of Detailed Balance in the Differential
Chapman-Kolmogorov Equation
6.4 Examples of Detailed Balance in Fokker-Planck Equations
6.4.1 Kramers' Equation for Brownian Motion in a Potential
6.4.2 Deterministic Motion
6.4.3 Detailed Balance in Markovian Physical Systems
6.4.4 Ornstein-Uhlenbeck Process
6.4.5 The Onsager Relations
6.4.6 Significance of the Onsager Relations—FluctuationDissipation Theorem
6.5 Eigenfunction Methods in Many Variables
6.5.1 Relationship between Forward and Backward Eigenfunctions
6.5.2 Even Variables Only—Negativity of Eigenvalues.
6.5.3 A Variational Principle
6.5.4 Conditional Probability
6.5.5 Autocorrelation Matrix
6.5.6 Spectrum Matrix
6.6 First Exit Time from a Region (Homogeneous Processes)
6.6.1 Solutions of Mean Exit Time Problems
6.6.2 Distribution of Exit Points
138
138
140
140
141
142
142
143
Small Noise Approximations for Diffusion Processes
7.1 Comparison of Small Noise Expansions for Stochastic Differential
Equations and Fokker-Planck Equations
7.2 Small Noise Expansions for Stochastic Differential Equations
7.2.1 Validity of the Expansion
7.2.2 Stationary Solutions (Homogeneous Processes)
169
5.5
6.
7.
144
144
145
149
149
152
153
153
155
156
158
159
159
160
161
161
162
163
164
166
169
171
174
175
Contents
7.3
7.2.3
7.2.4
Small
7.3.1
7.3.2
7.3.3
Mean, Variance, and Time Correlation Function
Failure of Small Noise Perturbation Theories
Noise Expansion of the Fokker-Planck Equation
Equations for Moments and Autocorrelation Functions
Example
Asymptotic Method for Stationary Distribution
XIII
175
176
178
180
182
184
8.
The White Noise Limit
185
8.1 White Noise Process as a Limit of Nonwhite Process
185
8.1.1 Formulation of the Limit
186
8.1.2 Generalisations of the Method
189
8.2 Brownian Motion and the Smoluchowski Equation
192
8.2.1 Systematic Formulation in Terms of Operators and Projectors 194
8.2.2 Short-Time Behaviour
195
8.2.3 Boundary Conditions
197
8.2.4 Evaluation of Higher Order Corrections
198
8.3 Adiabatic Elimination of Fast Variables: The General Case
202
8.3.1 Example: Elimination of Short-Lived Chemical Intermediates202
8.3.2 Adiabatic Elimination in Haken's Model
206
8.3.3 Adiabatic Elimination of Fast Variables: A Nonlinear Case . 210
8.3.4 An Example with Arbitrary Nonlinear Coupling
214
9.
Beyond the White Noise Limit
9.1 Specification of the Problem
9.1.1 Eigenfunctions of L\
9.1.2 Projectors
9.2 Bloch's Perturbation Theory
9.2.1 Formalism for the Perturbation Theory
9.2.2 Application of Bloch's Perturbation Theory
9.2.3 Construction of the Conditional Probability
9.2.4 Stationary Solution P,(x, p)
9.2.5 Examples
9.2.6 Generalisation to a system driven by several Markov
Processes
9.3 Computation of Correlation Functions
9.3.1 Special Results for Ornstein-Uhlenbeck p(t)
9.3.2 Generalisation to Arbitrary Gaussian Inputs
9.4 The White Noise Limit
9.4.1 Relation of the White Noise Limit of <*(0£(0)> to the
Impulse Response Function
10. Levy Processes and Financial Applications
10.1 Stochastic Description of Stock Prices
10.2 The Brownian Motion Description of Financial Markets
10.2.1 Financial Assets
216
217
218
218
219
220
222
223
226
226
228
230
232
232
233
233
235
235
237
237
XIV
Contents
10.3
10.4
10.5
10.6
10.2.2 "Long" and "Short" Positions
10.2.3 Perfect Liquidity
10.2.4 The Black-Scholes Formula
10.2.5 Explicit Solution for the Option Price
10.2.6 Analysis of the Formula
10.2.7 The Risk-Neutral Formulation
10.2.8 Change of Measure and Girsanov's Theorem
Heavy Tails and Levy Processes
10.3.1 Levy Processes
10.3.2 Infinite Divisibility
10.3.3 The Poisson Process
10.3.4 The Compound Poisson Process
10.3.5 Levy Processes with Infinite Intensity
10.3.6 The Levy-Khinchin Formula
The Paretian Processes
10.4.1 Shapes of the Paretian Distributions
10.4.2 The Events of a Paretian Process
10.4.3 Stable Processes
10.4.4 Other Levy processes
Modelling the Empirical Behaviour of Financial Markets
10.5.1 Stylised Statistical Facts on Asset Returns
10.5.2 The Paretian Process Description
10.5.3 Implications for Realistic Models
10.5.4 Equivalent Martingale Measure
10.5.5 Hyperbolic Models
10.5.6 Choice of Models
Epilogue—the Crash of 2008
238
238
238
240
242
244
245
248
248
249
250
250
251
252
252
255
255
257
258
258
258
259
259
260
262
262
263
11. Master Equations and Jump Processes
264
11.1 Birth-Death Master Equations—One Variable
264
11.1.1 Stationary Solutions265
11.1.2 Example: Chemical Reaction X ^ A
267
11.1.3 A Chemical Bistable System
269
11.2 Approximation of Master Equations by Fokker-Planck Equations .. 273
11.2.1 Jump Process Approximation of a Diffusion Process
273
11.2.2 The Kramers-Moyal Expansion
275
11.2.3 Van Kampen's System Size Expansion
276
11.2.4 Kurtz's Theorem
280
11.2.5 Critical Fluctuations
281
11.3 Boundary Conditions for Birth-Death Processes
283
11.4 Mean First Passage Times
284
11.4.1 Probability of Absorption
286
11.4.2 Comparison with Fokker-Planck Equation
286
11.5 Birth-Death Systems with Many Variables
287
11.5.1 Stationary Solutions when Detailed Balance Holds
288
Contents
11.5.2 Stationary Solutions Without Detailed Balance
(Kirchoff's Solution)
11.5.3 System Size Expansion and Related Expansions
11.6 Some Examples
11.6.1 X + A^±2X
11.6.2 X^Y^A
k
XV
290
291
291
291
292
y
11.6.3 Prey-Predator System
11.6.4 Generating Function Equations
292
297
12. The Poisson Representation
12.1 Formulation of the Poisson Representation
12.2 Kinds of Poisson Representations
12.2.1 Real Poisson Representations
12.2.2 Complex Poisson Representations
12.2.3 The Positive Poisson Representation
12.3 Time Correlation Functions
12.3.1 Interpretation in Terms of Statistical Mechanics
12.3.2 Linearised Results
12.4 Trimolecular Reaction
12.4.1 Fokker-Planck Equation for Trimolecular Reaction
12.4.2 Third-Order Noise
12.4.3 Example of the Use of Third-Order Noise
12.5 Simulations Using the Positive Poisson representation
12.5.1 Analytic Treatment via the Deterministic Equation
12.5.2 Full Stochastic Case
12.5.3 Testing the Validity of Positive Poisson Simulations
12.6 Application of the Poisson Representation to Population Dynamics .
12.6.1 The Logistic Model
12.6.2 Poisson Representation Stochastic Differential Equation....
12.6.3 Environmental Noise
12.6.4 Extinction
13. Spatially Distributed Systems
13.1 Background
13.1.1 Functional Fokker-Planck Equations
13.2 Multivariate Master Equation Description
13.2.1 Continuum Form of Diffusion Master Equation
13.2.2 Combining Reactions and Diffusion
13.2.3 Poisson Representation Methods
13.3 Spatial and Temporal Correlation Structures
13.3.1 Reaction X^Y
301
301
305
305
306
309
313
314
318
318
319
323
324
326
326
329
331
332
332
333
333
334
336
336
338
~.... 339
341
345
346
347
347
h
13.3.2 Reactions B + X^C,
A + X—>2X
350
13.3.3 A Nonlinear Model with a Second-Order Phase Transition .. 355
XVI
Contents
13.4 Connection Between Local and Global Descriptions
13.4.1 Explicit Adiabatic Elimination of Inhomogeneous Modes ..
13.5 Phase-Space Master Equation
13.5.1 Treatment of Flow
13.5.2 Flow as a Birth-Death Process
13.5.3 Inclusion of Collisions—the Boltzmann Master Equation...
13.5.4 Collisions and Flow Together
359
360
362
362
363
366
369
14. Bistability, Metastability, and Escape Problems
372
14.1 Diffusion in a Double-Well Potential (One Variable)
372
14.1.1 Behaviour for D = 0
373
14.1.2 Behaviour if £>*is Very Small
373
14.1.3 Exit Time
•
375
14.1.4 Splitting Probability
375
14.1.5 Decay from an Unstable State
377
14.2 Equilibration of Populations in Each Well
378
14.2.1 Kramers' Method
378
14.2.2 Example: Reversible Denaturation of Chymotrypsinogen . . . 382
14.2.3 Bistability with Birth-Death Master Equations (One Variable) 384
14.3 Bistability in Multivariable Systems
386
14.3.1 Distribution of Exit Points
387
14.3.2 Asymptotic Analysis of Mean Exit Time
391
14.3.3 Kramers' Method in Several Dimensions
392
14.3.4 Example: Brownian Motion in a Double Potential
394
15. Simulation of Stochastic Differential Equations
15.1 The One Variable Taylor Expansion
15.1.1 Euler Methods
15.1.2 Higher Orders
15.1.3 Multiple Stochastic Integrals
15.1.4 The Euler Algorithm
15.1.5 Milstein Algorithm
15.2 The Meaning of Weak and Strong Convergence
15.3 Stability,
15.3.1 Consistency
15.4 Implicit and Semi-implicit Algorithms
15.5 Vector Stochastic Differential Equations
15.5.1 Formulae and Notation
15.5.2 Multiple Stochastic Integrals
15.5.3 The Vector Euler Algorithm
15.5.4 The Vector Milstein Algorithm
15.5.5 The Strong Vector Semi-implicit Algorithm
15.5.6 The Weak Vector Semi-implicit Algorithm
15.6 Higher Order Algorithms
15.7 Stochastic Partial Differential Equations
401
402
402
402
403
404
406
407
407
410
410
411
412
412
414
414
415
415
416
417
Contents
15.7.1 Fourier Transform Methods
15.7.2 The Interaction Picture Method
15.8 Software Resources
XVII
418
419
420
References
421
Bibliography
429
Author Index
434
Symbol Index
435
Subject Index
439