Fractional Calculus, Anomalous Diffusion, and Probability MURI Kickoff Meeting Brown University 7 December 2015 Mark M. Meerschaert Department of Statistics and Probability Michigan State University mcubed@stt.msu.edu http://www.stt.msu.edu/users/mcubed Fractional derivatives: An old idea gets new life Fractional derivatives dαf (x)/dxα for any α > 0 were invented by Leibniz soon after the more familiar integer derivatives. Some derivative formulas extended to the fractional case: dα λx α eλx e = λ dxα π dα [sin x] = sin x + α dxα 2 Γ(p + 1) dα p p−α x = x [ ] dxα Γ(p − α + 1) Probability and transforms If the random variable X has density f (x) so that P (a ≤ X ≤ b) = b a f (x)dx then f (x) has Fourier transform (FT) ∞ 1 1 − ikx + (ikx)2 + · · · f (x)dx fˆ(k) = 2! −∞ 1 = 1 − ikμ1 − k2μ2 + · · · 2 where the pth moment μp = ∞ −∞ xpf (x)dx Central limit theorem If μ1 = 0 and μ2 = 2 then fˆ(k) = 1 − k2 + · · · The IID sum Sn = X1 +· · ·+Xn has FT fˆ(k)n and the normalized √ sum Sn/ n has FT n √ 2 √ n fˆ(k/ n) = 1 − (k/ n) + · · · = 1− k2 n n + ··· → e−k ≡ ĝ(k) 2 as n → ∞. Inverting the Fourier transform reveals a Gaussian density 1 −x2/4 g(x) = √ e 4π Brownian motion If Xn represents a particle jump at time n then Sn = X1 +· · ·+Xn is the location of the particle at time n. Expanding the time scale by a factor of r > 0 and taking limits as r → ∞ shows that r −1/2S[rt] ⇒ At since fˆ(r −1/2k)[rt] = 1 − k2 r [rt] + ··· 2 → e−k t ≡ ĉ(k, t) for all t > 0. Inverting the FT shows that the density of the limiting Brownian motion process At is Gaussian c(x, t) = √ 2 1 e−x /(4t). 4πt 0.5 0.0 S 1.0 Simple random walk simulation 2 4 6 8 10 t R codes at www.stt.msu.edu/users/mcubed/Rcodes.zip −7 −5 S −3 −1 Longer time scale 0 20 40 60 t 80 100 0 5 S 10 15 Scaling limit: Brownian motion 0 200 400 600 800 1000 t Any finite variance jumps lead to the same limit process. The graph is a random fractal with dimension 2 − 1/2. The diffusion equation Taking Fourier transforms in the classical diffusion equation ∂ 2c(x, t) ∂c(x, t) = ∂t ∂x2 yields [since f (x) has FT (ik)fˆ(k)] d ĉ(k, t) = (ik)2ĉ(k, t) = −k2ĉ(k, t) dt whose solution 2 ĉ(k, t) = e−k t inverts to the same limit density for the Brownian motion At. For a cloud of diffusing particles, c(x, t) is the particle density. Properties of Brownian motion Since the density of t1/2A1 has FT √ 2 2 −( tk) −tk =e = ĉ(k, t) e d we see that At = t1/2A1 (same density). A cloud of diffusing particles spreads at the rate t1/2. 2 The plume tails off like e−x . Real plumes often spread faster, with a heavier tail. 0.20 0.10 0.00 density Classical diffusion profile −6 −4 −2 0 2 4 6 x1 Brownian motion Gaussian (Normal) density at time t = 1, 2, 3 showing square root spreading rate and fast tail decay. Heavy (power law) tails If P (X > x) ≈ x−α then f (x) ≈ αx−α−1 and some moments μk = ∞ −∞ xk f (x)dx do not exist. If 1 < α < 2 and μ1 = 0 then a typical X has FT fˆ(k) = 1 + (ik)α + · · · and n−1/α(X1 + · · · + Xn) has FT n −1/α n −1/α α fˆ(n k) = 1 + (n ik) + · · · n α (ik) = 1+ n + ··· α (ik) →e ≡ ĝ(k) as n → ∞. The inverse Fourier transform g(x) is called a stable density. This random walk is called a Lévy flight. Lévy motion If Sn = X1 + · · · + Xn is particle location at time n, then the scaling limit r −1/αS[rt] ⇒ At, since fˆ(r −1/αk)[rt] = 1 + (ik)α r [rt] + ··· α t(ik) →e ≡ ĉ(k, t). 1/α α α d Now At = t1/αA1, since t1/αA1 has FT e(ikt ) = et(ik) . The limit process At is called a stable Lévy motion. Lévy motion models super-diffusion, where a plume spreads like t1/α for α < 2, faster then Brownian motion. −2.0 −1.0 S 0.0 1.0 Heavy tail random walk simulation 2 4 6 8 10 t Random walk jumps P (X > x) = x−α with α = 1.5. −30 −50 S −10 0 Longer time scale 0 20 40 60 t Converging to an α-stable Lévy motion. 80 100 −200 −400 S 0 Scaling limit: Stable Lévy motion 0 200 400 600 800 1000 t Power law jumps persist in the limit process (non-locality). Graph is a random fractal of dimension 2 − 1/α. Fractional diffusion equation To solve the fractional diffusion equation ∂c(x, t) ∂ αc(x, t) = ∂t ∂xα take Fourier transforms to get d ĉ(k, t) = (ik)αĉ(k, t) dt and solve to obtain ĉ(k, t) = et(ik) α so the density of the Lévy motion solves this fractional PDE! In this case, c(x, t) falls off like x−α−1 as x → ∞. 0.00 0.10 c 0.20 0.30 Fractional diffusion profile −4 −2 0 2 4 x Stable α = 1.5 Lévy motion density at time t = 1, 2, 3 showing super-diffusive spreading rate, skewness, and power law tail. Eulerian view Fractional derivatives are nonlocal. jumps. Particles diffuse via long F 'x Dd DE (C ) Application to hydrology SOXPH GDWD B VWDEOH *DXVVLDQ 1RUPDOL]HG 0DVV 1RUPDOL]HG 0DVV Stable density with α = 1.1 captures early arrivals at the MADE experimental site. SOXPH GDWD B VWDEOH *DXVVLDQ E 6QDSVKRW GD\ D 6QDSVKRW GD\ /RQJLWXGLQDO 'LVWDQFH PHWHUV /RQJLWXGLQDO 'LVWDQFH PHWHUV ∂c(x, t) ∂c(x, t) ∂ αc(x, t) Governing equation: = −v +D ∂t ∂x ∂xα Particle tracking solution 0.04 0.02 0.00 Density 0.06 Random walk approximation at the MADE site, day 224 [M13]. −20 0 20 40 meters 60 80 100 Finite difference methods For α > 0 we have dαf (x)/dxα = limh→0 h−αΔαf (x) where Δαf (x) = ∞ wj f (x − (j − p)h), j=0 (−1)j Γ(α + 1) wj = j!Γ(α − j + 1) Use to construct explicit/implicit finite difference codes. These codes are mass-preserving since ∞ wj = 0. j=0 Use operator splitting for 2-d [MST06] or reaction term [BKM08]. Finite elements [Fix and Roop, 2004; Wang and Yang, 2013; D’Elia and Gunzburger, 2013] Finite volume method [Zhang et al., 2005] Spectral collocation method [Zayernouri and Karniadakis, 2014]. Numerical example (Is this problem well-posed?) Exact solution u(x, t) = e−tx3 and numerical approximation to ∂u(x, t) ∂ 1.8u(x, t) + q(x, t) = d(x) ∂t ∂x1.8 on 0 < x < 1 with u(x, 0) = x3, u(0, t) = 0, u(1, t) = e−t, q(x, t) = −(1 + x)e−tx3, d(x) = Γ(2.2)x2.8/6. 0.4 0.35 0.3 0.25 Exact CN 0.2 0.15 0.1 0.05 0 0 0.2 0.4 0.6 0.8 1 1.2 Isotropic fractional diffusion in R2 Particle density solves ∂ c(x, y, t) = Δα/2c(x, y, t) for some α ≤ 2. ∂t Here Δα/2f (x) has FT − k αfˆ(k), with Fourier symbol − k α y 1 0 −1 −1 0 x 1 Anisotropic diffusion (classical) ∂2 ∂2 ∂ Solution to c(x, y, t) = D1 2 c(x, y, t) + D2 2 c(x, y, t) ∂t ∂x ∂y Fourier symbol is −D1|k1|2 − D2|k2|2 (here D1 = 2, D2 = 1/2) y 1 0 −1 −1 0 x 1 Anisotropic anomalous diffusion: Case 1 ∂ ∂α ∂α Solution to c(x, y, t) + c(x, y, t) c(x, y, t) = α α ∂t ∂|x| ∂|y| Fourier symbol is −|k1|α − |k2|α (here α = 1.2) y 1 0 −1 −1 0 x 1 Anisotropic anomalous diffusion: Case 2 ∂ ∂α ∂α Solution to c(x, y, t) + α c(x, y, t) c(x, y, t) = α ∂t ∂x ∂y Fourier symbol is (ik1)α + (ik2)α (here α = 1.3) x2 1 0 −1 −1 0 x1 1 Anisotropic anomalous diffusion: Case 3 ∂ ∂β ∂α Solution to c(x, y, t) + β c(x, y, t) c(x, y, t) = ∂t ∂xα ∂y Fourier symbol is (ik1)α + (ik2)β with α = 1.8, β = 1.2. 2 x2 1 0 −1 −2 −2 −1 0 x1 1 2 Classical vs. anomalous diffusion The FT ĉ(k, t) = etψ(k) has Fourier symbol ψ(k). In the classical case, where α = 2, the symbols − k 2 = −|k1|2 − |k2|2 = (ik1)2 + (ik2)2 are all the same. In the fractional case, where 0 < α < 2, the symbols − k α = −|k1|α − |k2|α = (ik1)α + (ik2)α are all different. Only the first is isotropic. Strong anisotropy: α can depend on the coordinate. Nonlocal dispersion Fractional derivatives model fast spreading via long movements. ∂P ∂ 2P P ∂ 1.7P = C 1.7 + D 2 + rP 1 − ∂t ∂x ∂y K 20 y 10 0 −10 −20 −20 0 20 40 60 80 x Fractional diffusion jumps the barrier. Real invasive species data shows this behavior. 100 Anisotropic model for groundwater flow [ZBMLS06] Ground water flow: Heavier tail (smaller α) in the flow direction. 5 α2 =1.9 t=1 0 10-2 v2 = 0 10-3 10-4 -5 5 0.1 0.2 0.3 t=2 0 10-2 10-3 t=3 0 -5 v1 =10 10-4 -5 5 α1 =1.5 FFT RW 0 10 10-2 20 10-3 30 40 Distance from source 10-4 50 60 70 Compute solution via particle tracking, and inverse FFT. The general case: Nonlocal diffusion Scale-dependent random walks converge to a Markov process At on Rd whose probability density solves a Cauchy problem ∂ c(x, t) = ψ(−i∇, x)c(x, t), c(x, 0) = δ(x). ∂t The pseudo-differential operator [K11, BSW13] ψ(−i∇, x)f (x) = −v · ∇f (x) + ∇ · Q(x)∇f (x) + f (x − y) − f (x) + y · ∇f (x) φ(x, dy) generates a continuous convolution semigroup. Then c(x, t) has Fourier transform ĉ(k, t) = etψ(k,x) where the Fourier symbol ψ(k, x) = −v · (ik) + ik · Q(x)ik + −i k · y e − 1 − y · ik φ(x, dy). Open question: How to formulate on a bounded domain? Reflected stable process 50 0 −100 Y 150 Stable process (thin line) with negative jumps, and reflected stable (thick line). 0 200 400 600 t 800 1000 Fractional reflecting boundary condition Transition densities c(x, y, t) of the reflected stable process solve the forward equation [BKMSS16, Theorem 2.3] ∂c(x, y, t) ∂ αc(x, y, t) ; = α ∂t ∂(−y) ∂ α−1c(x, 0, t) = 0. α−1 ∂(−y) Without the reflecting boundary condition, the same fractional PDE governs the stable process on the entire real line. This is a no-flux boundary condition, just like the α = 2 case of Brownian motion. The backward equation is [BKMSS16, Theorem 2.1] ∂c(x, y, t) ∂c(0, y, t) ∂ αc(x, y, t) ; = = 0. α ∂t ∂x ∂x Forward process jumps toward boundary, backward process drifts. Numerical solutions to the forward equation Transition densities y → p(x, y, t) with index α = 1.8 and initial state x = 0, 1, 2, 4 (left to right) at times t = 0.5 (left), t = 1 (middle), and t = 2 (right). See [BKMSS16, Appendix] for Matlab code. 0.8 0.8 0.8 0.6 0.6 0.6 0.4 0.4 0.4 0.2 0.2 0.2 0 0 5 0 0 5 0 0 5 Plan of Work • Correctly define fractional derivatives on bounded domains • Impose appropriate (fractional) boundary conditions • Connect to Markov processes on bounded domain • Prove these problems are well-posed • Develop particle tracking solutions • Develop error bounds via random walk convergence Research Area 1: Theory of non-local operators Year 1: Develop checkable conditions for well-posedness of FPDEs based on current theory of Markov processes and pseudo-differential operators. Year 2: Derive appropriate fractional boundary conditions for reflected stable processes on an interval. Year 3: Extend the results of year two to bounded domains in d-dimensional space- Establish the mathematical theory on wellposedness for stochastic FPDEs for multi-dimensional systems. Year 4: Develop Markovian RW models for reflected processes in one spatial dimension, to provide a physical foundation. Year 5: Extend the Markovian RW models to d-dimensions. Research Area 2: Numerical solution of fractional PDEs Year 1: Using task from year 1 Area 1, develop particle tracking codes for FPDEs. Year 2: Extend particle tracking codes to handle BVP in one dimension. Year 3: Extend particle tracking codes further to handle BVP in d-dimensions. Year 4: Establish convergence results for Markovian RW models in one dimension, to provide error bounds for particle tracking codes. Year 5: Establish convergence results for Markovian RW models in d-dimensions, to provide error bounds for particle tracking codes. References 1. I.B. Aban, M.M. Meerschaert, and A.K. Panorska (2006) Parameter Estimation for the Truncated Pareto Distribution. Journal of the American Statistical Association: Theory and Methods. 101(473), 270–277. 2. Anderson, P. and M.M. Meerschaert (1998) Modeling river flows with heavy tails, Water Resources Res., 34, 2271–2280. 3. B. Baeumer, M. Kovács, and M.M. Meerschaert (2008) Numerical solutions for fractional reaction-diffusion equations. Comput. Math. Appl. 55, 2212-2226. 4. B. Baeumer, M. Kováks, M.M. Meerschaert, P. Straka, and R. Schilling (2015) Reflected spectrally negative stable processes and their governing equations. Trans. Amer. Math. Soc. 368(1), 227–248. 5. D.A. Benson, R. Schumer, M.M. Meerschaert and S.W. Wheatcraft (2001) Fractional dispersion, Lévy motions, and the MADE tracer tests. Transport in Porous Media 42, 211–240. 6. B. Böttcher, R.L. Schilling, and J. Wang (2013) Lévy-Type Processes: Construction, Approximation and Sample Path Properties. Springer, New York. 7. M. Chen and W. Deng, Fourth order accurate scheme for the space fractional diffusion equations. SIAM J. Numer. Anal. 52 No 3 (2014), 1418–1438 8. O. Defterli, M. DElia, Q. Du, M. Gunzburger, R. Lehoucq, and M.M. Meerschaert, Fractional Diffusion on Bounded Domains. Fractional Calculus and Applied Analysis, 18 (2015), No. 2, 342–360. 9. M. D’Elia and M. Gunzburger, The fractional Laplacian operator on bounded domains as a special case of the nonlocal diffusion operator. Comput. Math. Appl. 66 (2013), 1245–1260. 10. Q. Du, M. Gunzburger, R. B. Lehoucq, and K. Zhou, Analysis and approximation of nonlocal diffusion problems with volume constraints. SIAM Review 54 (2012), 667–696. 11. G.J. Fix, J.P. Roop, Least squares finite element solution of a fractional order two-point boundary value problem, J. Comp. Math. Appl. 48 (2004) 10171033. 12. V.N. Kolokoltsov (2011) Markov Processes, Semigroups, and Generators. De Gruyter, Berlin/Boston. 13. M.M. Meerschaert and H.-P. Scheffler (2001) Limit Distributions for Sums of Independent Random Vectors: Heavy Tails in Theory and Practice. Wiley, New York. 14. M.M. Meerschaert and H.P. Scheffler (2004) Limit theorems for continuous-time random walks with infinite mean waiting times. J. Appl. Probab. 41(3), 623–638. 15. M.M. Meerschaert and C. Tadjeran, Finite difference approximations for fractional advection-dispersion flow equations. J. Comput. Appl. Math. 172 (2004), 65–77. 16. M.M. Meerschaert and C. Tadjeran, Finite difference approximations for two-sided space-fractional partial differential equations, Appl. Numer. Math. 56 (2006), 80– 90. FORTRAN code at www.stt.msu.edu/users/mcubed 17. M.M. Meerschaert, H.P. Scheffler and C. Tadjeran, Finite difference methods for twodimensional fractional dispersion equation, Journal of Computational Physics, Vol. 211 (2006), No. 1, 249–261. FORTRAN code at www.stt.msu.edu/users/mcubed 18. M.M. Meerschaert and A. Sikorskii (2012) Stochastic Models for Fractional Calculus. De Gruyter Studies in Mathematics 43, De Gruyter, Berlin. 19. M.M. Meerschaert (2012) Fractional calculus, anomalous diffusion, and probability. Fractional Dynamics, R. Metzler and J. Klafter, Eds., World Scientific, Singapore, 265–284. 20. M.M. Meerschaert (2013) Mathematical Modeling. 4th Edition, Academic Press/Elsevier, New York. 21. I. Podlubny (1999) Fractional Differential Equations. Academic Press, San Diego. 22. G. Samorodnitsky and M. Taqqu (1994) Stable non-Gaussian Random Processes: Stochastic Models with Infinite Variance. Chapman and Hall, London. 23. R. Schumer, D. A. Benson, M.M. Meerschaert and S. W. Wheatcraft (2001) Eulerian derivation of the fractional advection-dispersion equation. Journal of Contaminant Hydrology 38, 69–88. 24. I.M. Sokolov and J. Klafter (2005) From diffusion to anomalous diffusion: a century after Einstein’s Brownian motion. Chaos 15, No. 2, 26–103. 25. C. Tadjeran, M.M. Meerschaert, and H.-P. Scheffler (2006) A second order accurate numerical approximation for the fractional diffusion equation. J. Comput. Phys. 213, 205–213. FORTRAN code at www.stt.msu.edu/users/mcubed 26. C. Tadjeran and M. Meerschaert (2007) A second order accurate numerical method for the two-dimensional fractional diffusion equation. J. Comput. Phys. 220, 813-823. FORTRAN code at www.stt.msu.edu/users/mcubed 27. H. Wang and D. Yang (2013) Wellposedness of Variable-Coefficient Conservative Fractional Elliptic Differential Equations. SIAM J. Numer. Anal. 51, No. 2, 1088–1107. 28. M. Zayernouri and G. Em Karniadakis (2014) Fractional Spectral Collocation Method. SIAM J. Sci. Comput. 36, No. 1, A40–A62. 29. X. Zhang, J.W. Crawford, L.K. Deeks, M.I. Shutler, A.G. Bengough, I.M. Young, A mass balance based numerical method for the fractional advection dispersion equation: theory and application, Water Resour. Res. 41 (2005) 1–10. 30. Y. Zhang, D.A. Benson, M.M. Meerschaert, E. M. LaBolle, and H.P. Scheffler (2006) Random walk approximation of fractional-order multiscaling anomalous diffusion. Physical Review E 74, 026706. 31. Y. Zhang, D.A. Benson, M.M. Meerschaert, H.P. Scheffler (2006) On using random walks to solve the space-fractional advection-dispersion equations. Journal of Statistical Physics 123, No. 1, 89–110. Difference quotients The derivative D1f (x) = limh→0 h−1Δf (x) where Δf (x) = f (x) − f (x − h). For positive integers α, Dαf (x) = limh→0 h−αΔαf (x) where Δ2f (x) = (f (x) − f (x − h)) − (f (x − h) − f (x − 2h)) = f (x) − 2f (x − h) + f (x − 2h), Δ3f (x) = f (x) − 3f (x − h) + 3f (x − 2h) − f (x − 3h) ... Δαf (x) = α α m=0 m (−1)mf (x − mh). Here α! α = m m!(α − m)! Fractional difference quotients For α > 0 define D αf (x) = limh→0 h−αΔαf (x) where Δαf (x) = ∞ α m=0 m (−1)mf (x − mh), Γ(α + 1) α = m m!Γ(α − m + 1) Since f (x − h) has FT e−ikhfˆ(k), and using the Binomial formula ∞ α (1 + z)α = z m for any complex |z| ≤ 1 m m=0 we see that Δαf (x) has FT ∞ α (−1)me−ikmhfˆ(k) = (1 − e−ikh)αfˆ(k) m=0 m and then the FT of h−αΔαf (x) is h−α(ikh)α α −ikh 1−e ikh fˆ(k) → (ik)αfˆ(k) as h → 0. Fractional derivative definitions Riemann-Liouville fractional integral: ∞ 1 α f (x − u)uα−1du. I f (x) = Γ(α) 0 Riemann-Liouville fractional derivative: ∞ 1 d f (x − y)y −αdy. Dαf (x) = D1I1−αf (x) = Γ(1 − α) dx 0 Caputo: Move the derivative inside ∞ 1 α 1−α 1 D f (x) = f (x − y)y −αdy. ∂ f (x) = I Γ(1 − α) 0 Use the Laplace transform f˜(s) = ∞ 0 ∞ 0 ∞ 0 e−sxf (x) dx to see that: e−sxDαf (x) dx = sαf˜(s) e−sx∂ αf (x) dx = sαf˜(s) − sα−1f (0). Fractional derivatives of exponentials If f (x) = eλx for some λ > 0, the Riemann-Liouville derivative ∞ 1 d eλ(x−y)y −αdy dx Γ(1 − α) 0 ∞ λx d e = e−λy y −αdy dx Γ(1 − α) 0 λx e d = λα−1Γ(1 − α) dx Γ(1 − α) d α−1 λx = λ e = λαeλx dx Dαf (x) = (Caputo agrees). Fractional derivatives of power laws If f (x) = xpI(x > 0) for p > 0, the Caputo derivative ∂ αf (x) = = = = = x 1 f (u)(x − u)−αdu Γ(1 − α) −∞ x 1 py p−1(x − y)−αdy Γ(1 − α) 0 x p y p−1(x − y)(1−α)−1dy Γ(1 − α) 0 Γ(p)Γ(1 − α) p+(1−α)−1 p x Γ(1 − α) Γ(p + 1 − α) pΓ(p) Γ(p + 1) p−α = x xp−α Γ(p + 1 − α) Γ(p − α + 1) (Riemann-Liouville agrees). Fractional derivative of Heaviside function If f (x) = I(x > 0) then f (x) ≡ 0 on x > 0 so ∂ αf (x) ≡ 0. But Dαf (x) = = = = = 1 d x f (u)(x − u)−αdu Γ(1 − α) dx −∞ x 1 d 1 (x − y)−αdy Γ(1 − α) dx 0 x 1 d u−αdu Γ(1 − α) dx 0 1−α 1 d x Γ(1 − α) dx 1 − α x−α = 0. Γ(1 − α) ∂xαf (x) = Dα x f (x) because of boundary term. Biological species growth and dispersion Fisher equation (reaction-diffusion) models growth and dispersal ∂ 2P ∂P ∂ 2P P = C 2 + D 2 + rP 1 − ∂t ∂x ∂y K 20 y 10 0 −10 −20 −20 0 20 40 60 x Note slow diffusion across the barrier gap. 80 100