Finite Difference Methods
18-20 September 2012
Thematic Outline
• Methods for computing temporal finite differences
• Methods for computing spatial finite differences
• Applications of grid staggering
• Impacts of truncation error that arises from the finite
difference approximations
Temporal Finite Differences
• Note: the temporal differencing methods to be
discussed apply to grid-point and spectral models.
• Two types of time differencing schemes…
– Explicit: all future time terms appear on one side of an equation
– Implicit: future time terms appear on both sides of an equation
– Hybrid explicit/implicit schemes also exist.
• Explicit differences can be solved directly; implicit
differences must be solved iteratively.
Explicit Temporal Differences
• Two methods…
– Single step (forward and centered differences)
– Multiple step (predictor-corrector methods)
• Single step example, centered difference:
f t 1  f t 1
 Ft
2t
f t 1  2tF t  f t 1
where f is some dependent field (u, v, T, etc.) and F is the forcing on that field
Explicit Temporal Differences
• Single step example, forward-in-time difference:
f t 1  f t
 Ft
t
f t 1  tF t  f t
• Let us consider these in the context of a real-world
example, a one-dimensional advection equation…
u
u
 u
t
x
Explicit Temporal Differences
• Centered difference:
t
t


u

u
u t 1  u t 1
t
x 1
x 1

 u 
2t
 2x 
t
t

 t 1
u

u
t 1
t
x 1
x 1
  u
u  u t 
x


• Forward-in-time difference:
t
t


u

u
u t 1  u t
t
x 1
x 1

 u 
t
 2x 
u
t 1
 u xt 1  u xt 1  t
  u
 u t 
 2x 
t
Note: in both cases,
we applied a secondorder centered finite
difference in space.
We’ll discuss this
method further
shortly.
Explicit Temporal Differences
• In predictor-corrector schemes, a prediction for the
dependent field at some future time is first made
using some finite difference approximation.
• From this, the forcing at that future time is obtained,
whether from finite difference or spectral methods.
• That forcing is then used, often with a forward-intime difference, to integrate to the next time.
Explicit Temporal Differences
• Example: Runge-Kutta type scheme…
t t
– Predict:
*
f 
F  ft
3
– With f*, obtain F*.
– Predict:
t *
t
f 
F f
2
– With fτ, obtain Fτ.
– Correct:

f t 1  tF   f t
• Predictor-corrector methods provide superior
accuracy at the expense of efficiency.
Explicit Temporal Differences
• Issue: acoustic and other fast-moving waves
– These waves require small time steps to maintain
computational stability in the advection terms (CFL)
– If not filtered, how to handle need for a small time step?
• Potential method: split-explicit schemes
–
–
–
–
Small ∆t for terms representing fast-moving waves
Large ∆t for terms representing slower-moving waves
Reduces number of computations on short time steps
Commonly used with current mesoscale model systems
Implicit Temporal Differences
• In explicit schemes, the forcing terms (spatial
derivatives) are evaluated only at the initial time t.
• In implicit schemes, these terms are evaluated at the
future time t+1 and the initial time t.
– In other words, they are valid at t+1/2.
– Thus, terms at t+1 appear on both sides of the equation
and the time derivative must be solved iteratively.
Implicit Temporal Differences
• Let us again consider the example of a onedimensional advection equation…
• Explicit, forward-in-time approximation:
t
t

 t
u

u
t 1
t
x 1
x 1
  u
u  u t 
 2x 
• Instead of representing the spatial derivative
exclusively at t=t, let us represent it as the average of
its values at the initial (t) and future (t+1) times…
Implicit Temporal Differences
• Spatial finite difference approximation:
u 1  u
u 
 


x 2  x t 1 x t 
• If we substitute and apply a second-order centered
spatial finite difference, this gives us equation (3.21),
t 1
t 1
t
t
t

 t
u

u
u

u
u

t
t 1
x 1
x 1
x 1
x 1

  u
u 

2  2x
2x 
Implicit Temporal Differences
• Advantage: implicit schemes are typically
unconditionally stable
– CFL criterion for advection terms is thus not a concern
– Enables us to use a longer time step to handle all waves
• Disadvantage: iterative nature requires more
computations per time step than explicit schemes
Longer time step savings < iterative computational expense
Implicit Temporal Differences
• To alleviate these issues, semi-implicit methods may
be used.
– Meteorological terms: handled explicitly
– Non-meteorological terms: handled implicitly
• Alleviating the need for a small time step to maintain linear
stability that arises with explicit methods.
• A common, long time step may be used for all terms
because implicit terms are stable for all time steps.
– Advantage over split-explicit schemes.
– Iterative computational expense is still present, however.
Spatial Differencing
• We must first establish our frame of reference: fixed
to the Earth or fixed to air parcels.
• Thus far, we have considered a frame of reference
tied to the Earth, an Eulerian reference frame.
– All terms are defined and computed on a fixed grid.
– Example: 1-D advection equation…
u
∆x
x-1
∆x
x
x+1
u x 1  u x 1
x 
x
2x
u x 1  u x 1
u
u


u
x
x
x
2x
u tx1  u tx1
u
x 
t
2t
Spatial Differencing
• But, as we’ve discussed before, the CFL criterion
applies to the advection terms in this framework.
• Can we utilize a reference frame that encompasses
the advection terms and eliminates this issue?
• Yes – if we use a reference frame tied to the motion
(i.e., fixed to air parcels)!
Spatial Differencing
• Such a reference frame is known as a Lagrangian
reference frame.
• Because Lagrangian methods follow air parcels, the
spatial and vertical resolutions of the model are flowdependent.
– Finer resolution is found in convergent regions.
– Coarser resolution is found in divergent regions.
– This is not ideal.
Spatial Differencing
• Alternative: semi-Lagrangian methods
– In a Lagrangian framework, only one set of parcels is
followed, but is followed over all time steps.
– In a semi-Lagrangian framework, a new set of parcels
(defined by the chosen grid) is followed at each time step.
• Parcels are followed utilizing parcel trajectories.
– Either forward or backward trajectories may be used.
– The spatial derivatives (advection terms) are implicitly
accounted for by the along-trajectory evolution.
Semi-Lagrangian Differencing
• Example: forward trajectories
– Evenly-spaced grid (black) is defined at t=t.
– Trajectories (red lines) are released from each grid point
and followed forward in time to t=t+1 (red dots).
– Non-advection terms in the primitive equations change
the values of atmospheric fields along the trajectories.
– The values of these fields at t=t+1 are then interpolated
back to a uniform grid and the process repeated.
x-1
x
x+1
Semi-Lagrangian Differencing
• Example: backward trajectories
– Evenly-spaced grid (black) is defined at t=t+1.
– Trajectories (red lines) are released from each grid point
and followed backward in time to t=t (red dots).
– Non-advection terms in the primitive equations change
the values of atmospheric fields along the trajectories.
• Need to interpolate from evenly-spaced grid valid at t=t to obtain
this forcing.
• But, no interpolation needed for t=t+1 – already on uniform grid!
x-1
x
x+1
Semi-Lagrangian Differencing
• Backward trajectory methods are preferred.
• Easier to interpolate from evenly-spaced grids to
non-gridded trajectory locations than the inverse.
• But, how can we represent the example on the
previous slide numerically (and not just graphically)?
Semi-Lagrangian Differencing
Imagine a two-dimensional grid, as below, at t=t+1. Our parcel
location at this time is given by (i,j).
(i,j)
Semi-Lagrangian Differencing
The backward trajectory from (i,j) is given by the red line. The
parcel locations at t=t-1 (x) and t=t (+) are noted in blue.
X
+
(i,j)
Semi-Lagrangian Differencing
~~
ˆ
ˆ
(
Let the location of X be (i , j ) and the location of + be i , j ) .
X
(iˆ, ˆj )
+
~~
(i , j )
(i,j)
Semi-Lagrangian Differencing
• Let the dependent variable in question be given by f
and the forcing by F.
• We want ft+1 and know ft-1 and ft by interpolation
between grid points at the previous times.
• F is computed locally using Eulerian finite difference
methods and then interpolated between grid points.
• The finite difference can then be explicit or implicit.
Semi-Lagrangian Differencing
• Explicit: recall our centered-in-time difference...
f
t 1
f
2t
t 1
F
t
• The values of ft+1, ft-1, and Ft are taken at their
locations along the trajectory, i.e.,
f i ,t j 1  f iˆt,ˆj1
2t
 F~it, ~j
Semi-Lagrangian Differencing
• Implicit: a generic example can be expressed as...
f
t 1
t t 1
t
 f  (F  F )
2
t
• As for the explicit case, the values of ft+1, ft, Ft+1, and
Ft are taken at their locations along the trajectory,
f
t 1
i, j
 f
t
~ ~
i ,j
t t 1
 ( Fi , j  F~it, ~j )
2
Semi-Lagrangian Differencing
• Advantages of semi-Lagrangian schemes...
– Loss of advection terms eliminates CFL criterion, enabling
the usage of a longer time step.
– Can be used for either grid point or spectral methods.
– Can also be used with semi-implicit differencing schemes.
– Minimizes non-linear instability (to be discussed soon).
• Disadvantage of semi-Lagrangian schemes…
– Not energy or mass conserving (a particular problem over
long model integrations)
Grid Staggering
• For some applications, it can be useful to define our
dependent variables on two slightly offset grids.
• This is known as grid staggering.
– Both horizontal and vertical grid staggering are possible.
– A typical staggering increment is ½ the grid interval.
• Which variables apply to each grid depends upon the
desired computational characteristics.
Grid Staggering
( u

)
x
simple 1-D example for u and θ
• (a): must use θj+1 and θj-1 (over 2∆x) to compute a centered
finite difference
• (b): can use θj+½ and θj-½ (over ∆x) to compute a centered
finite difference
• Staggering halves the effective grid increment and minimizes
the impacts of truncation error on the solution.
Grid Staggering
• There are many examples of staggered grids in
current NWP models.
• Arakawa C-grid: mass variables staggered from
kinematic variables; used in WRF-ARW model
Grid Staggering
• Other staggers…
– Arakawa B-grid: akin to C grid, but with mass and
kinematic variables staggered on grid centers and corners
rather than grid centers and sides
– Arakawa E-grid: akin to C grid, except rotated 45°
• These grids are used by the MM5/NMM-B and WRFNMM mesoscale models, respectively.
Grid Staggering
• Grid staggering increases effective resolution but
does not increase the number of grid points.
• Increase in effective resolution requires a smaller
time step to meet CFL criterion.
– Only if Eulerian differencing methods are used, however.
– Recall: as ∆x decreases, Courant number increases.
• The benefit from reduced truncation error outweighs
time step considerations, however.
Truncation Error
• Many of the primitive equations are differential
equations with partial derivative terms.
• If we utilize grid point methods, these terms must be
approximated with finite differences.
• These approximations introduce error, however,
depending upon their complexity.
• This error is known as truncation error.
Finite Difference Approximations
• First, however, it is useful to revisit how finite
difference approximations are obtained.
• Any function over an interval – i.e., a partial
derivative – can be approximated by a Taylor series.
• Generally, for a function in x,
( x  a)  f ( a)
f ( x)  f ( a )   (
)  R( N , x )
n
n!
x
n 1
N
n
n
Finite Difference Approximations
• Expanded to several terms, we obtain (3.24),
f (a ) ( x  a) 2  2 f (a) ( x  a)3  3 f (a)
f ( x)  f (a)  ( x  a)



2
3
x
2
x
6
x
( x  a) N  N f (a)
... 
 R( N , x)
N
N!
x
• Here, a is our point of interest and x is some point of
reference. R is a residual term representing the error.
Finite Difference Approximations
• This equation is used to obtain finite difference
approximations for the function f.
• The accuracy of each approximation is dependent
upon the highest power/order terms retained in the
equation.
– Retain higher order terms: increased accuracy.
– Keep only lower order terms: decreased accuracy.
Finite Difference Approximations
• Example: first order / two point, forward in space
– Let x = a+∆x and truncate 2nd order and higher terms…
f (a)
f (a  x)  f (a)  (a  x  a)
x
– Solve for the derivative…
f (a) f (a  x)  f (a)

x
x
Finite Difference Approximations
• Example: first order / two point, backward in space
– Let x = a-∆x and truncate 2nd order and higher terms…
f (a )
f (a  x)  f (a )  (a  x  a)
x
– Solve for the derivative…
f (a)
f (a  x)  f (a) f (a)  f (a  x)


x
x
x
• First order schemes are typically not used, however,
because they have large truncation errors.
Finite Difference Approximations
• Example: second order / three point, centered
– Let x = a+∆x and truncate 3nd order and higher terms…
f (a) (a  x  a) 2  2 f (a)
f (a  x)  f (a)  (a  x  a)

x
2
x 2
– Let x = a-∆x and truncate 3nd order and higher terms…
f (a) (a  x  a) 2  2 f (a)
f (a  x)  f (a)  (a  x  a)

x
2
x 2
Finite Difference Approximations
– Subtract the 2nd equation from the first…
f (a)
f (a  x)  f (a  x)  2x
x
– Solve for the first derivative…
f (a) f (a  x)  f (a  x)

x
2x
Finite Difference Approximations
• A five point / 4th order approximation is given by…
f (a) 1 2
1

[ ( f (a  x)  f (a  x))  ( f (a  2x)  f (a  2x))]
x
x 3
12
where the two closest points on each side of a are used
in the calculation of the derivative.
• Any of these – or even higher-order – expressions
may be used in numerical models to compute spatial
derivatives in physical / grid-point space.
Truncation Error
• Example: let f be a function for which the derivative
is exactly known and compare that result to the finite
difference approximations.
• As in the text, let f = A cos kx, where k = 2π/L and L is
the wavelength.
f
  kA sin kx
x
f f A cos k ( x  x)  A cos k ( x  x)


x x
2 x
Analytic
Centered Difference
Truncation Error
• The ratio of the approximation to the exact value
gives the truncation error, where a value of 1
indicates no error.
• As in the text, trig identities can be used to show:
f
A cos k ( x  x)  A cos k ( x  x)
sin kx
x 
2x

f
 kA sin kx
kx
x
• Since k = 2π/L, this ratio varies with ∆x and L.
Truncation Error
• How does this ratio vary with ∆x and L?
– If L is fixed (e.g., depicting some meteorological feature),
as ∆x -> 0, ∆x/L -> 0.
– As ∆x/L -> 0, k∆x -> 0.
– By identity, as k∆x -> 0, sin (k∆x) -> k∆x.
• Thus, for small ∆x (fine grid spacing), the ratio
approaches 1 and the truncation error is small (i.e.,
the wave is well-resolved).
Truncation Error
• If n = # of grid points, then L = n∆x.
Truncation Error
• As ∆x gets smaller, n gets larger (for fixed L).
– Ratio asymptotes to 1 as n increases.
Truncation Error
• As L gets smaller, n gets smaller (for fixed ∆x).
– Ratio asymptotes to 1 (0) for very large (small) waves.
Truncation Error
• Let’s consider a practical example. Let f = A cos kx
represent a ridge-trough pattern.
• If L = 1000 km, what ∆x is needed for n = 10 (i.e.,
fairly small truncation error)?
• Since L = n∆x, ∆x = L/n = 1000 km / 10 = 100 km.
• Similarly, if L = 50 km (e.g., mesohigh-wake low), we
need ∆x = 5 km for n = 10.
Truncation Error
• Thus, to reasonably “resolve” a wave of length L, you
need a horizontal grid spacing of at most L/10.
• Ideally, you want an even-finer grid spacing to better
resolve the wave and minimize truncation error.
• Similar arguments can be made in the inverse to
describe the wave(s) that can be resolved for a given
horizontal grid spacing.
Grid Spacing Versus Resolution
Grid spacing and resolution are NOT interchangeable!
• Resolution: defined by the wave that the model can
resolve for a given grid spacing.
• The resolution depends upon the grid spacing, yes,
but also upon the accuracy of the finite difference
approximation used (if using grid point methods).
– Better approximation -> less error -> finer resolution
Truncation Error
• Consider n=6…
X
X
f
x  0.96
f
4th order
x
f
 x  0 .8
f
2nd order
x
Truncation Error
• The ratio does not approach 0.96 for the 2nd order
scheme until beyond the limits of the chart.
• Thus, if simpler finite difference methods are
desired, many more grid points n (smaller ∆x) are/is
needed to achieve an acceptable truncation error!
Truncation Error
• With either 4th or 2nd order schemes, the error is still
unacceptably large at small scales (n<5). We’ll discuss
how we can deal with this later.
• Errors end up manifest in all fields that evolve
depending upon gradients…
– Geostrophic wind (pressure gradient)
– Advection terms (kinematic gradients)
– Vertical motion from continuity (kinematic gradients)
…and grow upscale and with increasing time!
Truncation Error: Take-Home Points
• Must consider the length of feature to be resolved.
• Must consider the # of grid points (or grid spacing)
needed to resolve that feature.
• Must consider the error inherent to the desired finite
difference approximation to be used.
• Aside: this is partially why spectral modeling evolved,
but that has its own unique issues that creep up…