GEOGG121: Methods
Differential Equations
Dr. Mathias (Mat) Disney
UCL Geography
Office: 113, Pearson Building
Tel: 7670 0592
Email: mdisney@ucl.geog.ac.uk
www.geog.ucl.ac.uk/~mdisney
Lecture outline
• Differential equations
– Introduction & importance
– Types of DE
• Examples
• Solving ODEs
– Analytical methods
• General solution, particular solutions
• Separation of variables, integrating factors, linear operators
– Numerical methods
• Euler, Runge-Kutta
• V. short intro to Monte Carlo (MC) methods
Reading material
• Textbooks
These are good UG textbooks that have WAY more detail than we need
–
–
–
Boas, M. L., 1985 (2nd ed) Mathematical Methods in the Physical Sciences, Wiley, 793pp.
Riley, K. F., M. Hobson & S. Bence (2006) Mathematical Methods for Physics &
Engineering, 3rd ed., CUP.
Croft, A., Davison, R. & Hargreaves, M. (1996) Engineering Mathematics, 2nd ed., Addison
Wesley.
• Methods, applications
–
–
–
Wainwright, J. and M. Mulligan (eds, 2004) Environmental Modelling: Finding Simplicity in
Complexity, J. Wiley and Sons, Chichester. Lots of examples particularly hydrology, soils,
veg, climate. Useful intro. ch 1 on models and methods
Campbell, G. S. and J. Norman (1998) An Introduction to Environmental Biophysics,
Springer NY, 2nd ed. Excellent on applications eg Beer’s Law, heat transport etc.
Monteith, J. L. and M. H. Unsworth (1990) Principles of Environmental Physics, Edward
Arnold. Small, but wide-ranging and superbly written.
• Links
–
–
http://www.math.ust.hk/~machas/differential-equations.pdf
http://www.physics.ohio-state.edu/~physedu/mapletutorial/tutorials/diff_eqs/intro.html
Introduction
• What is a differential equation?
– General 1st order DEs
dx
= f ( x, t )
dt
dy
= f ( y, x )
dx
– 1st case t is independent variable, x is dependent variable
– 2nd case, x is independent variable, y dependent
• Extremely important
– Equation relating rate of change of something (y) wrt to
something else (x)
– Any dynamic system (undergoing change) may be amenable to
description by differential equations
– Being able to formulate & solve is incredibly powerful
Examples
• Velocity
– Change of distance x with time t i.e. v =
• Acceleration
– Change of v with t i.e.
dx
dt
d2x
a= 2
dt
• Newton’s 2nd law
– Net force on a particle = rate of change of linear momentum (m
constant so…
d mv
dp
F=
=
dt
• Harmonic oscillator
( ) = m dv = ma
dt
dt
– Restoring force F on a system  displacement (-x) i.e. F = -kx
2
– So taking these two eqns we have
m
d x
= -kx
2
dt
Examples
• Radioactive decay of unstable nucleus
– Random, independent events, so for given sample of N atoms, no. of
decay events –dN in time dt  N
-dN
µN
dt
– So N(t) depends on No (initial N) and rate of decay
• Beer’s Law – attenuation of radiation
– For absorption only (no scattering), decreases in intensity (flux
density) of radiation at some distance x into medium, Φ(x) is
proportional to x
df ( x )
µ -f ( x )
dx
– Same form as above – will see leads to exponential decay
– Radiation in vegetation, clouds etc etc
Examples
• Compound Interest
– How does an investment S(t), change with time, given an annual
interest rate r compounded every time interval Δt, and annual deposit
amount k?
– Assuming deposit made after every time interval Δt
S ( t + Dt ) = S ( t ) + ( rDt ) S ( t ) + kDt
– So as Δt0
S ( t + Dt ) - S ( t ) dS
=
= rS + k
Dt
dt
Examples
• Population dynamics
– Logistic equation (Malthus, Verhulst, Lotka….)
– Rate of change of population P with t depends on Po, growth rate r
(birth rate – death rate) & max available population or ‘carrying
capacity’ K
æ Pö
dP
= rP ç1- ÷
è Kø
dt
– P << K, dP/dt  rP but as P increases (asymptotically) to K, dP/dt
goes to 0 (competition for resources – one in one out!)
– For constant K, if we set x = P/K then
d P
P æ P ö dx
= r ç1- ÷ ® = rx (1- x )
dt K
K è K ø dt
http://www.scholarpedia.org/article/Predator-prey_model#Lotka-Volterra_Model
Examples
• Population dynamics: II
– Lotka-Volterra (predator-prey) equations
– Same form, but now two populations x and y, with time –
dx
dy
= x ( a - by) , = -y ( c - dx )
dt
dt
– y is predator and yt+1 depends on yt AND prey population (x)
– x is prey, and xt+1 depends on xt AND y
– a, b, c, d – parameters describing relationship of y to x
• More generally can describe
– Competition – eg economic modelling
– Resources – reaction-diffusion equations
Examples
• Transport: momentum, heat, mass….
– Transport usually some constant (proportionality factor) x driving force
– Newton’s Law of viscosity for momentum transport
• Shear stress, τ, between fluid layers moving at different speeds - velocity gradient
perpendicular to flow, μ = coeff. of viscosity
– Fourier’s Law of heat transport
du
t =m
dz
• Heat flux density H in a material is proportional to (-) T gradient and area
perpendicular to gradient through which heat flowing, k = conductivity. In 1D case…
– Fick’s Law of diffusive transport
dT
H = -k
dz
• Flux density F’j of a diffusing substance with molecular diffusivity Dj across density
gradient dρj/dz (j is for different substances that diffuse through air)
dr j
Fj¢ = -D j
dz
See Campbell and Norman chapter 6
Types: analytical, non-analytical
• Analytical, closed form
– Exact solution e.g. in terms of elementary functions
such as ex, log x, sin x
• Non-analytical
– No simple solution in terms of basic functions
– Solution requires numerical methods (iterative) to
solve
– Provide an approximate solution, usually as infinite
series
Types: analytical, non-analytical
• Analytical example
– Exact solution e.g.
dx
= ax
dt
– Solve by integrating both sides
ò
dx
ln(x)+ c1 = at + c2
= a ò dt
x
– This is a GENERAL solution
x=e
at+(c2 -c1)
= eat e( 2
• Contains unknown constants
– We usually want a PARTICULAR solution
• Constants known
• Requires BOUNDARY conditions to be specified
c -c1)
Types: analytical, non-analytical
• Particular solution?
– BOUNDARY conditions e.g. set t = 0 to get c1, 2 i.e.
x(t = 0) = e( 2 1) = x0
– So x0 is the initial value and we have
c -c
x(t) = x0 eat
– Exponential model ALWAYS when dx/dt  x
• If a>0 == growth; if a < 0 == decay
• Population: a = growth rate i.e. (births-deaths)
• Beer’s Law: a = attenuation coeff. (amount x absorp. per
unit mass)
• Radioactive decay: a = decay rate
Types: analytical, non-analytical
• Analytical: population growth/decay example
dP
= (b - d)P
dt
P ( t ) = P0 e
(b-d )t
Log scale – obviously linear….
Types: ODEs, PDEs
• ODE (ordinary DE)
– Contains only ordinary derivatives
d 2 y dy
+ =x
2
dx dx
• PDE (partial DE)
– Contains partial derivatives – usually case when
depends on 2 or more independent variables
– E.g. wave equation: displacement u, as function of
time, t and position x
¶2 u 2 ¶2 u
=c
2
¶t
¶x 2
Types: Order
• ODE (ordinary DE)
– Contains only ordinary derivatives (no partials)
– Can be of different order
• Order of highest derivative
d2x
m 2 = -kx
dt
2nd
d 2 y dy
+ =x
2
dx dx
2nd
dx
5
= ( xt )
dt
1st
Types: Order -> Degree
• ODE (ordinary DE)
– Can further subdivide into different degree
• Degree (power) to which highest order derivative raised
æ dy ö
ç ÷ + y = sin t
è dt ø
3
1st order
3rd degree
dx 2
+ x =1
dt
1st order
1st degree
æ d 2 y ö æ dy ö3
ç 2 ÷ +ç ÷ = 3
è dx ø è dx ø
2
2nd order
2nd degree
Types: Linearity
• ODE (ordinary DE)
– Linear or non-linear?
• Linear if dependent variable and all its derivatives occur
only to the first power, otherwise, non-linear
• Product of terms with dependent variable == non-linear
• Functions sin, cos, exp, ln also non-linear
dy
sin x + y = x
dx
Linear
dy
y = x2
dx
Non-linear
y dy/dx
d2y 2
+y =0
2
dx
Non-linear
y2 term
dy
+ sin y = 0
dx
Non-linear
sin y term
Solving
• General solution
–
–
–
–
Often many solutions can satisfy a differential eqn
General solution includes all these e.g.
Verify that y = Cex is a solution of dy/dx = y, C is any constant
So
dy
= Ce x
dx
dy
– And for all values of x,
= y and eqn is satisfied for any C
dx
– C is arbitrary constant, vary it and get all possible solutions
– So in fact y = Cex is the general solution of dy/dx = y
Solving
• But for a particular solution
–
–
–
–
We must specify boundary conditions
Eg if at x = 0, we know y = 4 then from general solution
4 = Ce0 so C = 4 and
y = 4e x is the particular solution of dy/dx = y that satisfies the
condition that y(0) = 4
– Can be more than one constant in general solution
– For particular solution number of given independent conditions
MUST be same as number of constants
Types: analytical, non-analytical
• Analytical: Beer’s Law - attenuation
– k is extinction coefficient – absorptivity per unit depth, z (m-1)
df ( z)
= -kf ( z)
dz
f ( z) = f0 e-kz
– E.g. attenuation through atmosphere, where path length (z) 
1/cos(θsun), θsun is the solar zenith angle
– Take logs: ln f ( z ) = ln f0 - kz
– Plot z against ln(ϕ), slope is k, intercept is ϕ0 i.e. solar radiation
with no attenuation (top of atmos. – solar constant)
– [NB taking logs v powerful – always linearise if you can!]
Initial & boundary conditions
• One point conditions
– We saw y = Acos x + Bsin x as general solution of
– Need 2 conditions to get particular solution
d2y
+y=0
2
dx
• May be at a single point e.g. x = 0, y = 0 and dy/dx = 1
• So 0 = Acos0 + Bsin0 = A and solution becomes y = Bsin x
• Now apply second condition i.e. dy/dx = 1 when x = 0 so differentiate
dy
= B cos x
dx
1= Bcos0 = B
– Particular solution is then y = sin x
Solving: examples
• Verify that y = Acos x + Bsin x satisfies
d2y
+y=0
2
dx
d 2 x dx
• Verify that x = t + Alnt + B is a solution of t 2 + = 4t
dt
dt
– (2nd order, 1st degree, linear)
2
Initial & boundary conditions
• Two point conditions
2
d y
Again consider
+y=0
2
dx
Solution satisfying y = 0 when x = 0 AND y = 1 when x = 3π/2
So apply first condition to general solution y = Acos x + Bsin x
i.e. 0 = Acos0 + Bsin0 = A and solution is y = Bsin x
Applying second condition we see
3p
y =1 = Bsin
2
– And B = -1, so the particular solution is y = -sin x
–
–
–
–
–
– If solution required over interval a ≤ x ≤ b and conditions given
at both ends, these are boundary conditions (boundary value
problem)
– Solution subject to initial conditions = initial value problem
Separation of variables
• We have considered simple cases so far
dy
– Where
= f ( x ) and so y = ò f ( x ) dx
dx
• What about cases with ind. & dep. variables on RHS?
– E.g.
dy
= f ( x ) g ( y)
dx
• Important class of separable equations. Div by g(y) to
1 dy
solve
= f ( x)
g ( y) dx
– And then integrate both sides wrt x
1 dy
1
ò g ( y) dx dx = ò g ( y) = ò f ( x)dx
Separation of variables
• Equation is now separated & if we can integ. we have y
in terms of x- x
1
-x
– Eg dy = e where f ( x ) = e and g ( y ) =
y
dx
y
dy
– So multiply both sides by y to give y = e- x and then integrate
dx
both sides wrt x
y2
2
-x
-x
– i.e. ò ydy = ò e dx and so
= -e- x + C and y = -2e + 2C
2
– If we define D = 2C then y = ± D - 2e- x
Eg See Croft, Davison, Hargreaves section 18, or
http://www.cse.salford.ac.uk/profiles/gsmcdonald/H-Tutorials/ordinarydifferential-equations-separation-variables.pdf
http://en.wikipedia.org/wiki/Separation_of_variables
Using an integrating factor
dy
+ P ( x) y = Q ( x)
• For equations of form
dx
– Where P(x) and Q(x) are first order linear functions of x, we can
multiply by some (as yet unknown) function of x, μ(x)
dy
m ( x) + m ( x) P ( x) y = m ( x) Q ( x)
dx
d
m ( x ) y)
– But in such a way that LHS can be written as
(
dx
– And then d m x y = m x Q x
( ) ) ( ) ( )
(
dx
– Which is said to be exact, with μ(x) as the integrating factor
– Why is this useful?
Eg See Croft, Davison, Hargreaves section 18, or
http://www.cse.salford.ac.uk/profiles/gsmcdonald/H-Tutorials/ordinarydifferential-equations-integrating-factor.pdf
http://en.wikipedia.org/wiki/Integrating_factor
Using an integrating factor
• Because
d
m ( x ) y) = m ( x ) Q ( x ) it follows that m ( x ) y = ò m ( x ) Q ( x )dx
(
dx
• And if we can evaluate the integral, we can determine y
d
dy
• So as above, we want
(m y) = m + m Py
dx
dx
d
dy dm and so, from above
( m y) = m + y
dx
dx dx
dm
dy dm
dy
= mP
• m +
and
by
inspection
we
can
see
that
y = m + m Py
dx
dx dx
dx
dm
= ò P dx
• This is separable (hurrah!) i.e. ò
• Use product rule i.e.
m
http://en.wikipedia.org/wiki/Product_rule
http://www.cse.salford.ac.uk/profiles/gsmcdonald/H-Tutorials/ordinary-differentialequations-integrating-factor.pdf
Using an integrating factor
• And we see that ln m - ln K = ò P dx (-lnK is const. of integ.)
• And so
m
ln = ò P dx
K
• We can choose K = 1 (as we are multiplying all terms in equation
by integ. factor it is irrelevant), so
P( x ) dx
dy
ò
– Integrating factor for
+ P ( x ) y = Q ( x )is given by m ( x ) = e
dx
– And solution is given by m ( x ) y =
ò m ( x) Q ( x)dx
http://en.wikipedia.org/wiki/Product_rule
http://www.cse.salford.ac.uk/profiles/gsmcdonald/H-Tutorials/ordinary-differentialequations-integrating-factor.pdf
Using an integrating factor: example
dy y
+ =1
• Solve
dx x
dy
+ P ( x) y = Q ( x)
dx
1
and Q ( x ) =1
x
1
P( x ) dx
ò x dx ln x
ò
=e
=e =x
– Using the formula above m ( x ) = e
– From previous we see that P ( x ) =
– And we know the solution is given by m ( x ) y =
– So xy =
ò
x2
x dx = + C, as Q ( x ) =1
2
ò m ( x) Q ( x)dx
2nd order linear equations
d2y
dy
p
x
+
q
x
• Form ( ) dx 2 ( ) dx + r ( x ) y = f ( x )
– Where p(x), q(x), r(x) and f(x) are fns of x only
– This is inhomogeneous (dep on y)
– Related homogeneous form ignoring term independent of y
d2y
dy
p ( x) 2 + q ( x) + r ( x) y = 0
dx
dx
– Use shorthand L{y} when referring to general linear diff. eqn to
stand for all terms involving y or its derivatives. From above
– L { y} = f ( x ) for inhomogeneous general case
– And L { y} = 0 for general homogenous case
2
2
d
y
d
y
3
3
– Eg if
then
where
L
y
=
x
L { y} = 2 - 4y
- 4y = x
{ }
2
dx
dx
Linear operators
• When L{y} = f(x) is a linear differential equation, L is a
linear differential operator
– Any linear operator L carries out an operation on functions f1
and f2 as follows
1. L { f1 + f2 } = L { f1} + L { f2 }
2. L {af1 } = aL { f1} where a is a constant
3. L {af1 + bf2 } = aL { f1 } + bL { f1} where a, b are constants
d2y
dy
– Example: if L { y} = 2 + 3x - 2y show that L { y1 + y2 } =
dx
dx
– and L {ay} = aL { y}
L { y1 } + L { y2 }
Linear operators
• Note that L{y} = f(x) is a linear diff. eqn so L is a linear diff operator
• So
d2
d
– we see L { y1 + y2 } = 2 ( y1 + y2 ) + 3x
( y1 + y2 ) - 2 ( y1 + y2 )
dx
dx
2
d y1
dy1
d 2 y2
dy
– And rearrange:
= 2 + 3x
- 2y1 + 2 + 3x 2 - 2y2
dx
dx
dx
dx
– & because differentiation is a linear operator we can now see
L { y1 + y2 } = L { y1} + L { y2 }
d 2 ( ay)
d ( ay)
+ 3x
- 2 ( ay)
• For the second case L {ay} =
2
dx
dx
æ d2y
ö
ad 2 y
dy
dy
• So L {ay} =
+ 3ax - 2ay = a ç 2 + 3x - 2y ÷ = aL { y}
2
dx
dx
dx
è dx
ø
Partial differential equations
• DEs with two or more dependent variables
– Particularly important for motion (in 2 or 3D), where eg position
(x, y, z) varying with time t
• Key example of wave equation
– Eg in 1D where displacement u depends on time and position
2
– For speed c, satisfies ¶2u
2 ¶ u
=c
2
¶t
¶x 2
¶2 u
¶2 u
– Show u ( x,t ) = sin ( x + 2t ) is a solution of
=4 2
2
¶t
¶x
– Calculate partial derivatives of u(x, t) wrt to x, then t i.e.
¶u
¶u
= cos ( x + 2t )
= 2 cos ( x + 2t )
¶x
¶t
Partial differential equations
– Now 2nd partial derivatives of u(x, t) wrt to x, then t i.e.
¶2 u
¶2 u
= -sin ( x + 2t )
= -4sin ( x + 2t )
2
2
¶x
¶t
2
¶2 u
¶
éë-sin ( x + 2t )ùû = 4 u
– So now
=
-4sin
x
+
2t
=
4
(
)
¶t 2
¶x 2
– More generally we can express the periodic solutions as
(remembering trig identities)
–
u+ ( x, t ) = Ae (
i kx-wt )
and
u- ( x, t ) = Be (
i kx+wt )
– Where k is the wave vector (2π/λ); ω is the angular frequency (rads s-1) = 2π/T
for period T;
http://en.wikipedia.org/wiki/List_of_trigonometric_identities
http://www.physics.usu.edu/riffe/3750/Lecture%2018.pdf
http://en.wikipedia.org/wiki/Wave_vector
Partial differential equations
• In 3D?
– Just consider y and z also, so for q(x, y, z, t)
¶ q 2 æ¶ q ¶ q ¶ qö
=c ç 2 + 2 + 2÷
2
¶t
è ¶x ¶y ¶z ø
2
2
2
2
q+kxkykx ( x, y, z, t ) = Ae ( x
i k x+ky y+kz z-wt )
q-kxkykx ( x, y, z, t ) = Ae ( x
i k x+ky y+kz z+wt )
• Some v. important linear differential operators
– Del (gradient operator) Ñf ( x, y, z ) =
¶f
¶f
¶f
x̂ + ŷ + ẑ
¶x
¶y
¶z
2
2
2
¶
f
¶
f
¶
f
– Del squared (Laplacian) Ñ × Ñf ( x, y, z) = D ( x, y, z ) =
x̂
+
ŷ
+
ẑ
2
2
2
¶x
¶y
¶z
• Lead to eg Maxwell’s equations
http://www.physics.usu.edu/riffe/3750/Lecture%2018.pdf
Numerical approaches
• Euler’s Method
dy
= f ( x, y) with initial cond. y(x0) = y0
– Consider 1st order eqn
dx
– Find an approx. solution yn at equally spaced discrete values
(steps) of x, xn
dy
– Euler’s method == find gradient at x = x0 i.e.
= f ( x0 , y0 )
dx x=x0
– Tangent line approximation
True solution
y
y(x1)
Tangent approx.
y1
y0
0
x0
x1
http://en.wikipedia.org/wiki/Numerical_ordinary_differential_equations
x
Croft et al., p495
Numerical Recipes in C ch. 16, p710
http://apps.nrbook.com/c/index.html
Numerical approaches
• Euler’s Method
– True soln passes thru (x0, y0) with gradient f(x0, y0) at that point
– Straight line (y = mx + c) approx has eqn y = y0 + ( x - x0 ) f ( x0, y0 )
– This approximates true solution but only near (x0, y0), so only
extend it short dist. h along x axis to x = x1
– Here, y = y1 and y1 = y0 + ( x1 - x0 ) f ( x0, y0 )
– Since h = x1-x0 we see y1 = y0 + hf ( x0, y0 )
– Can then find y1, and we then know (x1, y1)…..rinse, repeat….
y(x1)
y1
y0
0
yi+1 = yi + hf ( xi , yi )
True solution
y
Tangent approx.
x
x
x
Generate series of values iteratively
Accuracy depends on h
Croft et al., p495
Numerical Recipes in C ch. 16, p710
http://apps.nrbook.com/c/index.html
Numerical approaches
• Euler’s Method: example
– Use Euler’s method with h = 0.25 to obtain numerical soln. of
dy
= -xy 2 with y(0) = 2, giving approx. values of y for 0 ≤ x ≤ 1
dx
– Need y1-4 over x1 = 0.25, x2 = 0.5, x3 = 0.75, x4 = 1.0 say, so
2
– yi+1 = yi + 0.25 -xi yi with x0 = 0 y0 = 2
– And
y1 = 2 - 0.25 ( 0) 2 2 = 2.000
(
y2
y3
y4
yi+1 = yi + hf ( xi , yi )
)
( )
= 2 - 0.25 ( 0.25) ( 2 ) = 1.750
= 1.750 - 0.25 ( 0.5) (1.750 ) = 1.367
= 1.367 - 0.25 ( 0.75) (1.367 ) = 1.017
2
2
2
NB There are more
accurate variants of
Euler’s method..
Exercise: this can be solved ANALYTICALLY via separation
of variables. What is the difference to the approx. solution?
Numerical approaches
• Runge-Kutta methods (4th order here….)
–
–
–
–
Family of methods for solving DEs (Euler methods are subset)
Iterative, starting from yi, no functions other than f(x,y) needed
No extra differentiation or additional starting values needed
BUT f(x, y) is evaluated several times for each step
dy
= f ( x, y) subject to y = y0 when x = x0, use
Euler
dx
h
yi+1 = yi + ( k1 + 2k2 + 2k3 + k4 )
6
– where
æ
æ
h
h ö
h
h ö
k1 = f ( xi , yi ), k2 = f ç xi + , yi + k1 ÷, k3 = f ç xi + , yi + k2 ÷, k4 = f ( xi + h, yi + hk3 )
è
è
2
2 ø
2
2 ø
– Solve
http://en.wikipedia.org/wiki/Numerical_ordinary_differential_equations
http://en.wikipedia.org/wiki/Runge%E2%80%93Kutta_methods
Croft et al., p502
Rile et al. p1026
Numerical Recipes in C ch. 16, p710
http://apps.nrbook.com/c/index.html
Numerical approaches
• Runge-Kutta example
– As before, but now use R-K with h = 0.25 to obtain numerical
soln. of dy = -xy 2 with y(0) = 2, giving approx. values of y for 0
≤ x ≤ 1 dx
– So for i = 0, first iteration requires
2
k1 = f ( x0 , y0 ) = - ( 0) ( 2) = 0
k2 = f ( 0.125, 2) = - ( 0.125) ( 2) = -0.5
2
æ
ö
0.25
2
k3 = f ç 0.125, 2 +
-0.5) ÷ = f ( 0.125,1.9375) = -0.125 (1.9375) = -0.4692
(
è
ø
2
k4 = f ( 0.25, 2 + 0.25 ( -0.4692)) = f ( 0.25,1.8827) = -0.25 (1.9375) = -0.8861
0.25
0 + 2 (-0.5) + 2 (-0.4692) + (-0.8861)) =1.8823
– And finally y1 = 2 +
(
6
2
– Repeat! c.f. 2 from Euler, and 1.8824 from analytical
Very brief intro to Monte Carlo
• Brute force method(s) for integration /
parameter estimation / sampling
– Powerful BUT essentially last resort as involves random
sampling of parameter space
– Time consuming – more samples gives better approximation
– Errors tend to reduce as 1/N1/2
• N = 100 -> error down by 10; N = 1000000 -> error down by 1000
– Fast computers can solve complex problems
• Applications:
– Numerical integration (eg radiative transfer eqn), Bayesian
inference, computational physics, sensitivity analysis etc etc
http://en.wikipedia.org/wiki/Monte_Carlo_method
http://en.wikipedia.org/wiki/Monte_Carlo_integration
Numerical Recipes in C ch. 7, p304
http://apps.nrbook.com/c/index.html
Basics: MC integration
•
•
Pick N random points in a multidimensional volume V, x1, x2, …. xN
MC integration approximates integral of function f over volume V as
ò f dV » V
f ±V
f2 - f
•
1 N
Where f º å f ( xi ) and
N i=1
•
+/- term is 1SD error – falls of as 1/N1/2
f
2
2
N
1 N 2
º å f ( xi )
N i=1
Choose random points in A
Integral is fraction of points
under curve x A
From
http://apps.nrbook.com/c/
index.html
Basics: MC integration
• Why not choose a grid? Error falls as N-1 (quadrature approach)
• BUT we need to choose grid spacing. For random we sample until
we have ‘good enough’ approximation
• Is there a middle ground? Pick points sort of at random BUT in
such a way as to fill space more quickly (avoid local clustering)?
• Yes – quasi-random sampling:
– Space filling: i.e. “maximally avoiding of each other”
Sobol method v pseudorandom: 1000 points
FROM: http://en.wikipedia.org/wiki/Lowdiscrepancy_sequence
Summary
• Differential equations
– Describe dynamic systems – wide range of examples, particularly
motion, population, decay (radiation – Beer’s Law, mass –
radioactivity)
• Types
–
–
–
–
Analytical, closed form solution, simple functions
Non-analytical: no simple solution, approximations?
ODEs, PDEs
Order: highest power of derivative
• Degree: power to which highest order derivative is raised
– Linear/non:
• Linear if dependent variable and all its derivatives occur only to the first
power, otherwise, non-linear
Summary
• Solving
– Analytical methods?
• Find general solution by integrating, leaves constants of integration
• To find a particular solution: need boundary conditions (initial, ….)
• Integrating factors, linear operators
– Numerical methods?
• Euler, Runge-Kutta – find approx. solution for discrete points
• Monte Carlo methods
– Very useful brute force numerical approach to integration, parameter
estimation, sampling
– If all else fails, guess…..
END
Example
• Radioactive decay
– Random, independent events, so for given sample of N atoms, no. of
decay events –dN in time dt  N so
dN
µ -N
dt
–
–
–
–
dN
= -l N
dt
Where λ is decay constant (analogous to Beer’s Law k) units 1/t
- lt
Solve as for Beer’s Law case so N t = N 0 e
i.e. N(t) depends on No (initial N) and rate of decay
λ often represented as 1/tau, where tau is time constant – mean
lifetime of decaying atoms
– Half life (t=T1/2) = time taken to decay to half initial N i.e. N0/2
– Express T1/2 in terms of tau
()
Example
• Radioactive decay
– EG: 14C has half-life of 5730 years & decay rate = 14 per minute per
gram of natural C
– How old is a sample with a decay rate of 4 per minute per gram?
– A: N/N0 = 4/14 = 0.286
– From prev., tau = T1/2/ln2 = 5730/ln2 = 8267 yrs
– So t = -tau x ln(N/N0) = 10356 yrs
Exercises
• General solution of
d2x
dx
3
+ 2x = 0 is given by x = Aet + Be2t
2
dt
dt
• Find particular solution satisfies x = 3 and dx/dt = 5 when t =0
• Resistor (R) capacitor (L) circuit (p458, Croft et al), with current
flow i(t) described by
di
iR + L = t, t ³ 0,i ( 0) = 0
dt
• Use integrating factor to find i(t)….approach: re-write as
di R
t
+ i=
dt L
L
Exercises
dy
= -xy 2 with y(x=0)=2 is
• Show that the analytical solution of
2
dx
y= 2
x +1
• Compare values from x = 0 to 1 with approx. solution obtained by
Euler’s method
•