Fakultät für Informatik
Technische Universität München
Simulation of Rain-Induced Floods on High Performance
Computers
Simulation regenbedingter Überschwemmungen auf
Hochleistungsrechnern
Master’s Thesis in Informatics
Nicolai Schaffroth
Supervisor
Advisor
Prof. Dr. Michael Bader
Roland Wittmann, M.Sc.
Submission Date:
15.09.2015
Declaration
I confirm that this master’s thesis is my own work and I have documented
all sources and material used.
Date
Signature
Abstract
Rain-induced floods are a serious concern in many places. Being able to
accurately simulate the behavior and effects of those floods can help plan
responses and take preventative measures. A common approach for the simulation of water flows is the use of numerical solvers for the shallow water
equations. Numerical solvers for non-trivial systems are typically computationally complex and therefore often too slow for the simulation of large
scenarios. Taking advantage of the capabilities of high performance computers can help significantly in that regard. In this thesis, the simulation
software SWE, originally created for tsunami modeling, was extended to include the features required for flood simulation, including models for rain
and friction. The accuracy of the resulting solver is validated on common
test scenarios and its computational performance measured and compared
to the FullSWOF solver. As an example for real-world use, a rain-induced
flood in Mecca, Saudi Arabia, is simulated.
Acknowledgments
I would like to thank my supervisor Prof. Michael Bader for offering me
the opportunity to work on this highly interesting topic and for his advice
and guidance during my research and work on this thesis, as well as the
interdisciplinary project I was able to complete under his supervision.
I am also very grateful to my advisor Roland Wittmann for his insight
into the issues that came up during my research, and for taking the time to
regularly discuss the progress and direction of my work.
Special thanks to Prof. Hans-Joachim Bungartz for his guidance throughout my studies at TUM and for inspiring my interest in numerical simulation.
Finally, I would like to thank my family for their continued support and
encouragement.
Contents
1 Introduction and Motivation
1
2 The Shallow Water Equations
2.1 Basic Model . . . . . . . . . .
2.2 Source Terms . . . . . . . . .
2.2.1 Bathymetry . . . . . .
2.2.2 Rain . . . . . . . . . .
2.2.3 Friction . . . . . . . .
2.3 Two-dimensional Extension .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
3
3
4
4
5
5
6
3 Implementation
3.1 SWE Code Base . . . . . . .
3.1.1 Discretization . . . .
3.1.2 CFL Condition . . .
3.1.3 Solvers . . . . . . . .
3.2 Additions and Modifications
3.2.1 Rain . . . . . . . . .
3.2.2 Friction . . . . . . .
3.2.3 Boundaries . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
9
9
10
11
11
15
15
16
17
.
.
.
.
.
.
.
.
.
.
21
21
22
24
26
26
28
31
32
32
38
.
.
.
.
.
.
.
.
4 Validation
4.1 SWASHES scenarios . . . . .
4.1.1 Subcritical Flow . . . .
4.1.2 Transcritical Flow with
4.2 MacDonald Type Scenarios .
4.2.1 Test Problem 1 . . . .
4.2.2 Test Problem 2 . . . .
4.3 Rain on Flat Plain . . . . . .
4.4 Dam Breaks . . . . . . . . . .
4.4.1 Diagonal Dam Break .
4.4.2 Radial Dam Break . .
. . . .
. . . .
Shock
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
5 Results
5.1 Performance . . . . . . . . .
5.1.1 Rain on Flat Plain .
5.1.2 Diagonal Dam Break
5.2 Application . . . . . . . . .
6 Conclusion
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
41
41
41
43
45
53
1
Introduction and Motivation
The simulation of water flows has a wide range of applications, such as flood
forecasting, dam break and tsunami simulations, creating models for pollutant transfer and soil erosion and predicting the risk of flooding when undertaking large-scale landscaping projects. A commonly used model for the
water flow in these kinds of scenarios are the shallow water equations, first
proposed by Barré de Saint-Venant [1]. Since no explicit solution for the
shallow water equations is known, numerical solvers are used for simulations
based on this model.
The main focus of this thesis is the simulation of rain-induced floods.
These can occur when prolonged rainfall exceeds the infiltration capacity of
the ground, causing water to accumulate and flow towards lower elevations.
In dry areas with naturally poor infiltration, rare but occasionally severe rain
can very quickly lead to extreme flooding, making predictive simulations an
invaluable tool for the prevention of catastrophic results. A commonly used
and freely available solver for the shallow water equations that is suitable for
flood simulation is FullSWOF (Full Shallow Water equations for Overland
Flow) [2]. While the FullSWOF software is flexible and accurate, it scales
poorly to high performance hardware and its computational performance for
larger simulations is suboptimal.
Real-world simulations often involve very large areas and long time scales,
requiring large amounts of computations. Therefore, computational efficiency
is an important concern in the implementation of the numerical solvers. Beyond the use of efficient algorithms and data structures, a strategy that
can be used to improve the computational performance is the simultaneous
computation of multiple independent update steps. This can be achieved
by splitting the domain across multiple processes, parallelizing independent
steps in the simulation process and using vector instructions provided by
modern CPUs to apply computations to multiple sets of data at once.
1
2
1. INTRODUCTION AND MOTIVATION
The work described in this thesis is based on the Shallow Water Equations
teaching code (SWE) [3], an existing code base for shallow water simulations,
including multiple solvers for the shallow water equations and designed for
parallelized execution on high performance computers. Originally created for
tsunami simulations, SWE lacks some important features required for the
simulation of rain and overland flow. The goal was, therefore, to extend the
SWE software for use in flood simulation, while retaining its computational
efficiency.
This thesis covers the basic model of the shallow water equations and
extensions required for flood simulation, the principles underlying the design
of the SWE software and the available solvers, as well as the additions that
were made over the course of this work. The accuracy of the simulation
achieved by the modified solver is validated on a series of test scenarios in
which it is compared to analytical solutions and the results computed by the
FullSWOF solver. Comparisons of the computational performance to the
FullSWOF solver are shown for two different benchmark scenarios. As an
example of real-world applications of the software, the results of a simulation
of a rain-induced flood in Mecca, Saudi Arabia, are shown.
2
The Shallow Water Equations
In this chapter, the shallow water equations are introduced as a model for
the flow of a fluid and their basic mathematical properties explained. Several
extensions required for the simulation of rain-induced floods are described
and added to the model.
2.1
Basic Model
The shallow water equations are a system of hyperbolic conservation laws
describing the flow of an incompressible fluid in situations where the vertical
velocity is negligible, which is the case for small-amplitude waves in a fluid
that is shallow relative to the wavelength. They are derived from depthintegrating the Navier–Stokes equations for cases where the horizontal scale
is much greater than the vertical scale. In one dimension, they take the form
∂h ∂hu
+
=0
∂t
∂x
(2.1)
∂hu ∂hu2 1 ∂h2
+
+ g
= 0,
∂t
∂x
2 ∂x
(2.2)
where g is the gravitational acceleration, h(x, t) is the depth of the fluid and
hu(x, t) is the horizontal momentum [4, ch. 13].
The equations describe conservation of mass (2.1) and momentum (2.2)
in an area in which the fluid is evenly distributed. Since the density of an
incompressible fluid remains constant, the mass of the fluid within a fixed
area can be reduced to its height h. Consequently, the momentum can be
reduced to hu. The conservation of momentum equation contains a term
describing acceleration by gravitational force due to height differences in
adjacent areas.
3
4
2. THE SHALLOW WATER EQUATIONS
Defining
"
q(x, t) =
h
hu
#
"
,
f (q) =
hu
2
hu + 21 gh2
#
,
the system of (2.1) and (2.2) can be written in the quasilinear form
∂q
∂q
+ f 0 (q)
= 0,
∂t
∂x
where the Jacobian matrix f 0 (q) is
"
#
0
1
f 0 (q) =
.
−u2 + gh 2u
The eigenvalues of f 0 (q) are
p
λ1 = u − gh,
and the corresponding eigenvectors
"
#
1
v1 =
,
√
u − gh
2.2
λ2 = u +
"
v2 =
p
gh
1
√
u + gh
(2.3)
(2.4)
(2.5)
(2.6)
#
.
(2.7)
Source Terms
When modeling realistic scenarios, the behavior described by the basic shallow water equations is usually insufficient, as many other factors influence
the flow of the fluid. This problem can be solved by adding source terms
corresponding to these factors to the right-hand side of the equations.
2.2.1
Bathymetry
The simplest source term, with great relevance for flood as well as tsunami
modeling results from the bathymetry. Since the ground is typically not flat
and even throughout the simulation area, height differences in the terrain can
lead to gravitational acceleration in addition to that caused by differences in
water height.
Using bathymetry data b(x), a source term describing this effect can be
added to the conservation of momentum equation (2.2), resulting in the onedimensional form of the equations typically used for tsunami modeling [5]:
∂b
∂hu ∂hu2 1 ∂h2
+
+ g
= −gh .
(2.8)
∂t
∂x
2 ∂x
∂x
An illustration of the variables occurring in this version of the shallow
water equations can be seen in Figure 2.1.
2.2. SOURCE TERMS
5
Figure 2.1: Vertical slice illustrating the variables in the one-dimensional shallow
water equations with bathymetry. Source: [6].
2.2.2
Rain
So far, the equations only model scenarios in which the fluid can enter and
exit an area exclusively across its boundaries. When simulating rain, additional mass is continuously added within the domain, necessitating the
addition of a source term to the conservation of mass equation (2.1):
∂h ∂hu
+
= r.
∂t
∂x
(2.9)
The function r(x, t) represents the rainfall intensity at a specific location
and time, in terms of the water height that would be added by rain per unit
of time in the absence of outflow [7].
2.2.3
Friction
Another important factor in the simulation of rain-induced floods, where the
water height is typically low compared to other applications such as tsunami
modeling, is bottom friction. Especially in realistic flood scenarios, the water often flows over relatively rough ground, such as farmland and overgrown
grass- and woodland, where friction forces can considerably dampen its momentum.
This can be modeled by adding another source term to the conservation
of momentum equation (2.8), based on a friction term Sf :
∂hu ∂hu2 1 ∂h2
∂b
+
+ g
= gh Sf −
.
∂t
∂x
2 ∂x
∂x
(2.10)
6
2. THE SHALLOW WATER EQUATIONS
Terrain
Manning coefficient n
Smooth asphalt / Cement
0.013
Rough asphalt / Concrete
0.016
Smooth earth
0.022
0.025
Gravel
Short grass / barren fields
0.030
High grass / row crops
0.035
0.040
Field crops / cleared forest
Dense brush / cultivated forest
0.100
Dense forest with undergrowth
0.150
Table 2.1: Values for Manning’s roughness coefficient on various types of terrain.
Source: [8, ch. 5].
The friction force is not constant and depends on the water height and
velocity within the area. Two commonly used friction models derived from
empirical considerations are Manning’s friction law
n2 (hu)|hu|
n2 u|u|
=
−
,
(2.11)
h4/3
h10/3
where n is Manning’s roughness coefficient, and Darcy-Weisbach’s friction
law
fD u|u|
fD (hu)|hu|
Sf = −
=−
,
(2.12)
8gh
8gh3
where fD is the Darcy friction factor [7]. Some common values for Manning’s
roughness coefficient are given in Table 2.1.
Sf = −
2.3
Two-dimensional Extension
So far, the shallow water equations were only described as a one-dimensional
model, with source terms for bathymetry, rain and friction resulting in the
system

∂h ∂hu


+
=r


∂t
∂x
(2.13)

∂hu ∂hu2 1 ∂h2
∂b


+
+ g
= gh Sf −
.

∂t
∂x
2 ∂x
∂x
This system can be extended to the second dimension by separating the
momentum into x and y components, hu(x, y, t) and hv(x, y, t). This changes
2.3. TWO-DIMENSIONAL EXTENSION
7
the conservation of mass equations (2.9) to
∂h ∂hu ∂hv
+
+
=r
∂t
∂x
∂y
(2.14)
and splits the conservation of momentum equation (2.10) into a pair of equations each describing the momentum in one cardinal direction:
∂b
∂hu ∂hu2 ∂huv 1 ∂h2
+
+
+ g
= gh Sf −
∂t
∂x
∂y
2 ∂x
∂x
(2.15)
∂hv ∂huv ∂hv 2 1 ∂h2
∂b
+
+
+ g
= gh Sf −
.
∂t
∂x
∂y
2 ∂y
∂y
The fact that the conservation of momentum equations (2.15) have the
same shape is beneficial when implementing a solver, as the flow can simply
be separated into the two components, rather than solving the entire system
(2.16) at once.
Combined, (2.14) and (2.15) form the two-dimensional system of shallow
water equations

∂h ∂hu ∂hv


+
+
=r


∂t
∂x
∂y





 ∂hu ∂hu2 ∂huv 1 ∂h2
∂b
+
+
+ g
= gh Sf −
(2.16)
∂t
∂x
∂y
2 ∂x
∂x





 ∂hv ∂huv ∂hv 2 1 ∂h2

∂b


+
+
+ g
= gh Sf −
,

∂t
∂x
∂y
2 ∂y
∂y
which can be used to model actual, realistic scenarios.
When considering the two-dimensional system without source terms,

∂h ∂hu ∂hv


+
+
=0



∂t
∂x
∂y




 ∂hu ∂hu2 ∂huv 1 ∂h2
+
+
+ g
=0
(2.17)

∂t
∂x
∂y
2 ∂x






∂hv ∂huv ∂hv 2 1 ∂h2


+
+
+ g
= 0,

∂t
∂x
∂y
2 ∂y
it can be written in the quasilinear form
∂q
∂q
∂q
+ f 0 (q)
+ g 0 (q)
= 0,
∂t
∂x
∂y
(2.18)
8
2. THE SHALLOW WATER EQUATIONS
using



h


q(x, y, t) =  hu  ,
hv


hu


f (q) =  hu2 + 12 gh2  ,
huv
with the Jacobian matrices


0
1 0


f 0 (q) =  −u2 + gh 2u 0  ,
−uv
v u
(2.19)

hv


g(q) = 
huv
,
1
2
2
hv + 2 gh

0
0 1


g 0 (q) =  −uv
v u  . (2.20)
−v 2 + gh 0 2v
The eigenvalues of f 0 (q) and g 0 (q) are
p
λx1 = u − gh,
λx2 = u,
p
λy2 = v,
λy1 = v − gh,
and the corresponding eigenvectors
 


0
1
√ 
 

vx2 =  0  ,
vx1 =  u − gh  ,
1
v




1
0




vy1 = 
vy2 =  −1  ,
u
,
√
v − gh
0

p
gh,
p
= v + gh
λx3 = u +
λy3

(2.21)

1
√ 

vx3 =  u + gh  ,
v

 (2.22)
1


vy3 = 
u
.
√
v + gh
3
Implementation
This chapter describes the implementation of numerical solvers for the shallow water equations in the SWE framework. The details of the additions
and modifications that were made to adapt the software for flood simulation
are explained, as well as some new models for specific boundary conditions
required by many test scenarios.
3.1
SWE Code Base
The Shallow Water Equations teaching code (SWE) [3] (available at https:
//github.com/TUM-I5/SWE) is an education-oriented code base for simulations based on the shallow water equations, originally oriented towards
tsunami simulation. It uses Finite Volume discretization, dividing the simulation area into a grid of rectangular cells and solving the Riemann problem
arising at the cell interfaces. The adaptive design of the code base allows
easy switching between different Riemann solver implementations.
Taking advantage of the parallelization and vectorization options on modern high performance computers was a central goal in the design of the SWE
code. The domain can be subdivided into multiple blocks of cells that are
processed in parallel, using MPI data exchange after completing the simulation of a timestep to allow waves to propagate across the block boundaries
(see Figure 3.1). Within each block, loop parallelization using OpenMP, as
well as vectorization [9] can be used to further speed up the computation of
the many independent cell updates.
SWE uses first order solvers with dynamic time stepping and operates on
single-precision floating point values to improve computational performance
and vectorization capabilities.
9
10
3. IMPLEMENTATION






 
Figure 3.1: Illustration of an SWE block. The white cells are ghost-layer cells
representing the block boundaries, the dark blue cells may be replicated in adjacent
blocks’ ghost layers. Source: [3].
3.1.1
Discretization
SWE uses a finite volume approach, with updates for the timestep tn → tn+1
based on the formula
∆t
A+ ∆Qni−0.5,j + A− ∆Qni+0.5,j
∆x
∆t
B + ∆Qni,j−0.5 + B − ∆Qni,j+0.5 ,
−
∆y
n
Qn+1
i,j = Qi,j −
(3.1)
where Qni,j = [hi,j , (hu)i,j , (hv)i,j ] is the vector of conserved quantities for the
cell (i, j) in the nth timestep, ∆x and ∆y are the dimensions of the grid cells
and A± ∆Qi∓0.5,j and B ± ∆Qi,j∓0.5 , the net updates, are the solutions of the
Riemann problem at the left, right, bottom and top edge of the cell (i, j),
respectively, following the concepts described in [10]. Specifically, the net
updates are computed by solving the Riemann problem
"
#
"
#
"
#
q2
0
q1
=
,
(3.2)
+ q22 1 2
∂
q2
(gq
q
)
−
+
gq
1
3
1
∂x
q1
2
x
t
x
using the quantity vector q = [h, hu, b], with the initial conditions
(
ql
if x < 0
q(x, 0) =
(3.3)
qr if x > 0
3.1. SWE CODE BASE
11
consisting of a jump between the values in the adjacent cells




hl
hr




ql =  (hu)l  and qr =  (hu)r  .
bl
br
3.1.2
(3.4)
CFL Condition
To maximize computational performance without compromising the quality
of the simulation, an upwind scheme [11] is used, where the length of the
timesteps is dynamically chosen throughout the simulation. To ensure convergence and stability, it needs to be ensured that the numerical domain of
dependence includes the true domain of dependence [12, ch. 3]. This is formalized in the Courant–Friedrichs–Lewy (CFL) condition [13] (translation:
[14]), which, in one dimension, takes the form
umax ∆t
≤ Cmax
∆x
⇔
∆t ≤
Cmax ∆x
.
umax
(3.5)
In practice, this can be imagined as setting an upper bound on the length
of a timestep, such that the movement of the water in one cell over the
duration of the timestep only affects the content of the neighboring cells and
not any cells beyond that. In [11], robustness is proven for Cmax ≤ 0.5. In
SWE, the CFL number is set to Cmax = 0.4, the same value that is used by
FullSWOF [2].
3.1.3
Solvers
Multiple Riemann solvers are implemented as part of SWE, which can be
chosen from at compile time. The solvers differ greatly in features and
complexity, and consequently also in the accuracy of their simulation and
their computational performance. The difference in the algorithmic complexity of the solvers also affects their compatibility with certain performance optimizations, most notably the auto-vectorization features of modern
R compilers.
Intel
All solvers considered here solve the Riemann problem (3.3) on the basis
of linearization followed by eigendecomposition of the first dimension of the
shallow water equations (3.2).
f-Wave
The f-Wave solver is a very basic Riemann solver based on the work of D.
Bale et al. [15]. It solves the homogeneous problem qt + f (q)x = 0 given by
12
3. IMPLEMENTATION
(3.2) by flux decomposition, directly including the bathymetry source term,
leading to
X
f (Qr ) − f (Ql ) − ∆xSb =
βp rp ,
(3.6)
where Sb is the discretized bathymetry source term, and rp are the eigenvectors of the linearized system (2.7).
A big drawback of this approach is that it does not preserve depth positivity, making it incapable of simulating wetting and drying of cells [12,
ch. 6]. The f-Wave solver is therefore forced to treat shorelines as walls to
avoid inundation and cannot be used for simulations on dry land, making it
unsuitable for flood simulation without major modifications.
Augmented Riemann
The Augmented Riemann solver, following the concepts in [16], uses a much
more complex approach. It augments the Riemann problem (3.3) by the
momentum flux φ(q) = hu2 + 12 gh2 and a component for the bathymetry b,
leading to the decomposition


Hr − Hl
 HU − HU  X


r
l
αp w p ,
(3.7)
=

 φ(Qr ) − φ(Ql ) 
Br − Bl
with the solutions

1
 s̆−

w1 =  − 2
 (s̆ )
0






 , w2 = 


0
0
1
1 −
(s̆ + s̆+ )
2






 , w3 = 


1
s̆+
(s̆+ )2
0



.

(3.8)
Here, s̆± are the Einfeldt speeds, defined as the minimum and the maximum of the eigenvalues of the shallow water equations (2.6) and the eigenvalues of the Roe averaged Jacobian
"
#
0
1
,
(3.9)
−û2 + g ĥ 2û
with
1
ĥ = (hr − hl ),
2
√
√
ul hl + ur hr
√
û = √
.
hl + hr
(3.10)
(3.11)
3.1. SWE CODE BASE
13
The implementation of the Augmented Riemann solver can be broken
down into four steps [9]:
1. Determine wet-dry state: The solver checks for dry cells and determines
whether the water level and momentum in adjacent wet cells is sufficient
to cause inundation.
2. Compute middle state and characteristic speeds: The water height h
and the wave speeds are computed under consideration of a middle state
determined from the wave structure of the nonlinear Riemann problem.
In case of mixed shock and rarefaction waves, Newton iteration is used
to solve the nonlinear system.
3. Compute wave decomposition: Using the previously computed wave
speeds, the system of equations (3.7) is solved.
4. Compute net updates: Depending on the wet-dry state, two of the three
components in (3.8) are used to compute the net updates.
With its sophisticated approach, the Augmented Riemann solver produces very accurate results for tsunami simulations, however, its high computational complexity makes it comparatively slow. The use of Newton iteration
introduces a loop with a data-dependent termination condition, preventing
auto-vectorization [17].
HLLE
The HLLE solver [18] is derived from A. Harten, P. Lax and B. van Leer’s
HLL solver [19], with improvements based on B. Einfeldt’s work on Riemann
solvers [20]. Its concept formed the basis for D. L. George’s Augmented
Riemann solver [12, ch. 3] and it contains many of the features present in
the previously described solver. However, some of the more computationally
complex parts in the Augmented Riemann solver follow a simpler design in
the HLLE solver, or are missing altogether, making it faster and allowing
R
auto-vectorization by Intel’s
compiler.
The HLLE solver computes wave composition based on the system


Hr − Hl

 X
αp w p ,
 HUr − HUl  =
φ(Qr ) − φ(Ql )
(3.12)
14
3. IMPLEMENTATION
similar to (3.7), but without the bathymetry component. The first and third
solution of the system are the eigenvectors




1
1




w1 =  s̆−  , w3 =  s̆+  ,
(3.13)
(s̆− )2
(s̆+ )2
with the corresponding eigenvalues
s1 = s̆− , s3 = s̆+ .
The second component can be chosen as
 
0
1
 
w2 =  0  and s2 = (s̆− + s̆+ ).
2
1
(3.14)
(3.15)
By adding the bathymetry source term to the results of the decomposition, instead of including it in the computation, a stationary discontinuity is
introduced into the solution. However, as demonstrated in [12, ch. 4], depth
positivity can still be preserved by bounding the components of the steady
state wave
#
"
# "
w01
bl − br
(3.16)
= 1
g(h
+
h
)(b
−
b
)
w02
l
r
l
r
2
in the ranges
w01
∈
s3
h∗
− s1 ∗ s3 − s1
, h
,
s3
s1
w02
∈ [ min(−hl (br − bl ), −hr (br − bl )),
g
(3.17)
(3.18)
max(−hl (br − bl ), −hr (br − bl ))],
where
(hu)l − (hu)r + s̆+ hr − s̆− hl
=
(3.19)
s̆+ − s̆−
is the HLLE middle height.
This allows the computation of the water height and wave speeds without having to compute a middle state. If, additionally, inundation is limited
to cases where the water height alone is sufficient, no dependency on middle state information beyond the HLLE middle height remains, allowing the
h∗
3.2. ADDITIONS AND MODIFICATIONS
15
removal of the computationally complex and not auto-vectorizable Newton
iteration. The implementation otherwise follows the same steps as the Augmented Riemann solver.
This results in a solver that is both sufficiently accurate and computationally efficient, making it an ideal choice as a basis for the implementation
of the flood simulation solver.
3.2
Additions and Modifications
To allow its use in flood simulation, the solver has to be extended to include
the rain and friction source terms. Further, additional boundary types were
needed to model features that are commonly used in test scenarios for overland flow, but were not relevant for tsunami simulation, such as rivers and
channels.
For the purpose of the computation models described in this section, let
h∗ and q ∗ = (hu)∗ represent water height and momentum resulting form the
net updates computed by the unmodified solver.
3.2.1
Rain
The rain source term only enters into the conservation of mass equation (2.9).
As it causes no problems for steady state and stability preservation, it can
be treated explicitly after the basic net updates are computed, similar to the
rain and infiltration model in the FullSWOF solver [2]. This results in the
modified height update
hn+1 = h∗ + r∆t.
(3.20)
Because rain can add water in locations that were previously dry or at
rest, computing it after the net updates can lead to water flow that is not
included in the wave speeds computed by the solver. To prevent this from
compromising stability when little or no other water movement is present,
a rain-dependent term has to be added to the CFL condition. Based on
a worst-case assumption of the entire weight of the added water leading
to unrestricted acceleration in a single cardinal direction, a limit can be
computed from the conservation of energy equation:
∆h
1
mu2 = mg
2
2
⇔
∆x
∆t
s
2
= gr∆t
⇔
∆t =
3
∆x2
.
gr
(3.21)
16
3. IMPLEMENTATION
This leads to an upper limit for the timestep of
s
2
3 (Cmax ∆x)
,
∆t ≤
gr
(3.22)
with the CFL number again chosen as Cmax = 0.4.
3.2.2
Friction
The friction source term enters into the conservation of momentum equation
(2.10), along with the source term resulting from the bathymetry. In this
case, the straight-forward explicit discretization leads to numerical instabilities and overestimates the velocity at wet-dry interfaces. Fortunately, there
are several other options for treating friction source terms.
A fully implicit method would produce the most accurate and stable
results, however the computational cost would be high. Further, due to
the non-linear form of the friction law, it cannot be generalized to the twodimensional model. It is therefore not a feasible approach for an efficient
solver.
In the apparent topography method [21, ch. 5], the topography is modified
to include the effects of the friction forces. The solver then computes the basic
net updates with a modified bathymetry term bmod = b − bf , with ∂x bf = Sf .
As noted in [7], this approach is also not entirely satisfactory, as it produces
a spurious peak at wet–dry fronts.
The best compromise between accuracy, stability and computational efficiency is a semi-implicit approach [22]. It allows the solver to first compute
the basic net updates and then applies the friction model by modifying the
momentum using the formula
q n+1 =
q∗
n
1 + gn2 ∆t hn (h|qn+1| )4/3
(3.23)
for Manning’s friction law and
q n+1 =
q∗
n
|
1 + fD ∆t 8hn|qhn+1
(3.24)
for the Darcy–Weisbach friction law. Note here that the value q ∗ represents
the total momentum within the cell after all other updates and not the net
updates computed by the solver.
For inclusion into the SWE solver, Manning’s friction law was chosen, due
to the greater availability of empirical values for the Manning coefficient n
and its widespread use for analysis and simulation in hydraulic engineering.
3.2. ADDITIONS AND MODIFICATIONS
3.2.3
17
Boundaries
At the edges of an SWE block, a ghost layer is used to simulate boundary conditions and connect multiple blocks when using MPI parallelization,
visualized in Figure 3.1. The version of SWE used for tsunami modeling
contained only the very basic boundary models Outflow, Wall, Connect and
Passive. Common test scenarios for flood simulation often involve rivers and
channels, requiring boundaries with imposed water heights and inflow rates,
which had to be added to SWE. The boundary type can be chosen separately for each edge of the block. The examples below show the values and
calculations for the left edge.
Outflow
The Outflow boundary effectively just adds an additional cell to the simulation area that acts like a copy of the outermost actual cell. It is used to
minimize boundary effects in scenarios focused on the behavior within the
domain. After each simulation step, the values in the boundary cells are set
to
h[0][j] = h[1][j],
hu[0][j] = hu[1][j],
hv[0][j] = hv[1][j].
Wall
This boundary models a solid, infinitely high wall along the edge of the block,
which reflects waves back into the domain. Its implementation is very similar
to the Outflow boundary, simply reversing the momentum perpendicular to
the edge:
h[0][j] = h[1][j],
hu[0][j] = −hu[1][j],
hv[0][j] = hv[1][j].
Connect
The Connect boundary is used to propagate changes between adjacent blocks
when the domain is split for MPI parallelization. Instead of locally computing
the contents of the boundary cells, their values are copied from the outermost
18
3. IMPLEMENTATION
cells within the grid of the adjacent block at the end of every timestep. This
allows the solver to compute the next timesteps as though the blocks were
actually connected, without having to access the data of the other task during
its computation.
Passive
Similar to the Connect boundary, this boundary type is intended for use at
the interface between separate blocks. However, instead of automatically
copying the cell content from the adjacent block, the ghost layer is made
accessible for other program components, which can then set the boundary
values according to their own requirements.
Imposed Height
The first boundary model that had to be newly implemented is the Imposed
Height boundary. It represents a scenario were the water height at the edge
remains constant under regular conditions, such as the outflow of a filled
channel. It requires a height parameter hb and a discharge parameter hub
which is used in case of super-critical flow. In the subcritical case, the values
in the boundary cells are set to
h[0][j] = hb ,
hu[0][j] = h[0][j]
p
hu[1][j]
√ p
−2 g
h[1][j] − h[0][j]
,
h[1][j]
hv[0][j] = 0.
In case of super-critical inflow, the values become
h[0][j] = hb ,
hu[0][j] = hub ,
hv[0][j] = 0
and
h[0][j] = h[1][j],
hu[0][j] = hu[1][j],
hv[0][j] = hv[1][j].
for super-critical outflow.
3.2. ADDITIONS AND MODIFICATIONS
19
Imposed Discharge
This boundary model represents scenarios in which the discharge at the
boundary remains constant under regular conditions. This can model inflow as well as outflow and is, for example, used for rivers flowing across the
domain. It requires a discharge parameter hub and a height parameter hb
which is used in case of super-critical flow. In case of subcritical flow, the
effective water height hs in the boundary cell is computed by using Newton
iteration to find the roots of the polynomial
p
p
hu[1][j]
− 2 gh[1][j] − |hub |,
(3.25)
f (hs ) = 2hs ghs + hs
h[1][j]
with
p
f (hs ) = 3 ghs +
0
p
hu[1][j]
− 2 gh[1][j] .
h[1][j]
(3.26)
Then, the boundary values in the subcritical case can be set to
h[0][j] = hs ,
hu[0][j] = hub ,
hv[0][j] = 0,
while in the super-critical case, they are
h[0][j] = hb ,
hu[0][j] = hub ,
hv[0][j] = 0.
Using Newton iteration here is less problematic than during the computation of the net updates, as the number of boundary cells is typically much
lower than the number of cells within the domain and the loop processing
them is not usually a target for vectorization.
4
Validation
To ensure that the solver is fit for practical use, validation is necessary to
check that it is working correctly and produces sufficiently accurate results.
In this chapter, a series of simple test cases are presented, that were used to
compare the results to known analytical solutions as well as those computed
by the FullSWOF solver [2]. To allow a fair comparison of the simulation
quality and computational performance between SWE and FullSWOF, the
latter was configured to also use a first order solver and single precision
floating point numbers.
4.1
SWASHES scenarios
The following scenarios are part of the SWASHES [23] collection of benchmark scenarios for which analytic solutions exist. They have a steady state
solution, which should remain stable and towards which the flow should converge for adequate starting conditions.
A steady state fulfills the condition
∂t h = ∂t q = 0,
(4.1)
where h is the water height and q = hu the momentum. With the onedimensional shallow water equations, this results in the system



q = rx + q0
2qr
1 q2

 ∂x b =
−
gh
∂
h
−
S
(h,
q)
−
,
x
f
gh h2
gh2
with q0 = q(t, x = 0) being the inflow rate at the left boundary.
21
(4.2)
22
4. VALIDATION
In the absence of rain and friction, (4.2) is reduced to

q = q0


1 q2

 ∂x b =
− gh ∂x h,
gh h2
(4.3)
which, in case of a regular solution, leads to the Bernoulli relation
q02
+ h(x) + b(x) = c,
2gh2 (x)
(4.4)
with c constant.
The original definition of the SWASHES scenarios is one-dimensional,
but they can be adapted for two-dimensional solvers by simply stretching
the domain to an arbitrary width and using outflow boundaries on the top
and bottom edges.
The scenarios considered here use a domain with a length of 25 m and a
bathymetry function with a bump defined as
(
0.2 − 0.05(x − 10)2 if 8 m < x < 12 m
b(x) =
(4.5)
0
else.
4.1.1
Subcritical Flow
In this scenario, the boundary conditions are chosen as an imposed discharge
of q = 4.42 m2 /s (inflow) upstream and an imposed height of h = 2 m downstream. The initial conditions are h + b = 2 m and q = 0 m2 /s.
This leads to a steady state with subcritical flow throughout the domain
and, using (4.4), an analytical solution for the water height given by
q02
q02
2
3
−
h
= 0, ∀x ∈ [0, 25],
(4.6)
h(x)
+
h(x) + b(x) −
r
2gh2r
2g
where hr = h(x = 25 m) is the water height at the right boundary.
Since the momentum of the water within the domain is initially zero, the
inflow from the left boundary at first results in an increased water height
(see Figure 4.1b). Once the momentum has spread through the domain and
moving water reaches the right boundary, it begins flowing out and the water
level drops. After some oscillation, the water settles into the steady state; a
constant flow at relatively even height, except for a small dip over the bump
in the bathymetry (see Figure 4.1f).
4.1. SWASHES SCENARIOS
23
(a) t = 0 s
(b) t = 6 s
(c) t = 12 s
(d) t = 18 s
(e) t = 24 s
(f) steady state
Figure 4.1: Bathymetry (b) and water surface (s) for the Subcritical Flow scenario.
Simulated with SWE on a 50 × 50 cell grid.
24
4.1.2
4. VALIDATION
Transcritical Flow with Shock
In the transcritical case, where the flow becomes torrential at the top of the
bump, the water height according to (4.4) is the solution of
q02
q02
3
2
h(x) + b(x) −
= 0, ∀x ∈ [0, 25],
(4.7)
−
h
−
b
h(x)
+
m
m
2gh2m
2g
where zm = maxx∈[0,25] z and hm is the corresponding water height.
Under specific conditions, the solution may contain a shock, with the flow
turning supercritical at the top of the bump, but becoming fluvial again after
a hydraulic jump. The water height is then given by resolving the system

2
q
q02

0
3
2

=0
h(x
)
+
b(x
)
−
−
h
−
b
h(x
)
+

shock
shock
m
m
shock

2gh2m
2g






q02
q02
3
2
(4.8)
h(xshock ) + b(xshock ) −
−
h
=0
h(x
)
+
r
shock

2gh2r
2g






1
g 2
1

2


−
q0
+
h1 − h22 = 0,
h1 h2
2
where h1 and h2 are the water heights upstream and downstream, respectively. The third equation in (4.8) is a Rankine–Hugoniot relation determining the location of the shock [24].
This is achieved in this scenario by choosing the boundary conditions
q = 0.18 m2 /s (upstream) and h = 0.33 m (downstream) and the initial
conditions h + b = 0.33 m and q = 0 m2 /s.
Because the water level is much lower relative to the height of the bump,
compared to the previous scenario, a significant drop in water height over
the bump can be observed even as the initial shock wave crosses the domain
(see Figure 4.2b). As the water level drops following the arrival of the flow at
the right boundary, the expected shock develops (Figure 4.2c and onwards).
The shock interrupts the upstream propagation of the fluctuations caused by
the outflow, leaving the water height on the left side of the domain constant
as the right side slowly resettles. In the steady state (see Figure 4.2f), the
water height can be seen dropping off sharply after the bump, where the flow
turns transcritical, and then evening out again towards the right.
4.1. SWASHES SCENARIOS
25
(a) t = 0 s
(b) t = 12 s
(c) t = 24 s
(d) t = 36 s
(e) t = 48 s
(f) steady state
Figure 4.2: Bathymetry (b) and water surface (s) for the Transcritical Flow with
Shock scenario. Simulated with SWE on a 250 × 250 cell grid.
26
4.2
4. VALIDATION
MacDonald Type Scenarios
MacDonald type scenarios [25] are another popular set of benchmark scenarios used for evaluation of open-channel solvers. They were originally constructed by determining the bed slope for a desired water depth and flow
rate. The scenarios are described by their bed slope function S0 (x), with
Z
b(x) = −
L
S0 (x) dx,
(4.9)
x
where L is the length of the domain.
To generate the input data for the simulation, the bed level b(x) is computed by composite trapezoidal quadrature, with subintervals corresponding
to the grid cells.
4.2.1
Test Problem 1
This scenario consists of a 1 km long, 10 m wide, rectangular channel with a
discharge of 20 m3 /s, which evaluates to an inflow rate of 2 m2 /s at the left
boundary, and an imposed height of 0.748409 m at the right boundary. The
scenario uses Manning’s friction model with n = 0.03. The bed slope is given
as
4/3
!
2
ĥ(x)
+
10
4
ĥ0 (x) + 0.36 S0 (x) = 1 −
10/3 ,
3
g ĥ(x)
10ĥ(x)
with
1/3
4
ĥ(x) =
g
1
1 + exp −16
2
x
1
−
1000 2
2 !!
and
2 !
1/3
2
x
1
x
1
4
−
exp −16
−
.
ĥ0 (x) = −
g
125 1000 2
1000 2
The function ĥ(x) is the expected solution for the water height in the
steady state. The steady state shows subcritical flow on both sides of the
domain.
When running the simulation, the water rushes down the channel as expected, slightly increasing in height where the slope is less steep. A small
pool forms at the opposite end of the channel, as water flows in from the
boundary, where a fixed height is imposed (see Figure 4.3b). After roughly
4.2. MACDONALD TYPE SCENARIOS
27
(a) t = 0 s
(b) t = 60 s
(c) t = 120 s
(d) t = 240 s
(e) t = 360 s
(f) steady state
Figure 4.3: Bathymetry (b) and water surface (s) for MacDonald Test Problem 1.
Simulated with SWE on a 1000 × 10 cell grid.
28
4. VALIDATION
Figure 4.4: Comparison of the water surface (s) in the steady state computed by
SWE and the analytical solution for MacDonald Test Problem 1.
360 s (Figure 4.3e), the flowing water reaches the right boundary and soon
after settles into the steady state (Figure 4.3f).
When comparing the result to the analytical solution (Figure 4.4), it
can be seen that SWE slightly overestimates the water height, however, the
difference is quite small and the overall shape of the water surface matches
the expected outcome.
4.2.2
Test Problem 2
This scenario also uses a 1 km long, 10 m wide, rectangular channel with
a discharge of 20 m3 /s, which evaluates to an inflow rate of 2 m2 /s at the
left boundary, but this time an imposed discharge boundary is used on the
right, with a discharge rate of 2 m2 /s and the height parameter set to the
analytically computed height of 0.864510 m. The Manning coefficient is set
to n = 0.02. The bed slope is given as
S0 (x) =
1−
4
g ĥ(x)3
!
4/3
2ĥ(x) + 10
ĥ0 (x) + 0.16 10/3 ,
10ĥ(x)
4.2. MACDONALD TYPE SCENARIOS
29
with
 1/3 
4


1−

 g
ĥ(x) = 
1/3

4



1−
g
1
x
1
tanh 3
−
for 0 ≤ x ≤ 500
3
1000 2
1
x
1
tanh 6
−
for 500 < x ≤ 1000
6
1000 2
and
 1/3

4
1
x
1
2


−
sech 3
−
for 0 ≤ x ≤ 500


g
1000
1000 2
ĥ0 (x) =
1/3


x
1
1
4

2

−
sech 6
−
for 500 < x ≤ 1000.
g
1000
1000 2
The function ĥ(x) is the expected solution for the water height in the
steady state. The flow in the steady state is subcritical at inflow and supercritical at outflow.
When running the simulation, a change in the shape of the wavefront
can be seen as the water flow reaches the steeper section of the channel
(Figure 4.5d), indicating the transition to transcritical flow. At around t =
360 s (Figure 4.5e), the flowing water reaches the right boundary, meeting
up with the pool that has, once again, formed there due to water flowing in
from the boundary.
Comparison with the analytical steady state solution shows good accuracy
(see Figure 4.6).
30
4. VALIDATION
(a) t = 0 s
(b) t = 60 s
(c) t = 120 s
(d) t = 240 s
(e) t = 360 s
(f) steady state
Figure 4.5: Bathymetry (b) and water surface (s) for MacDonald Test Problem 2.
Simulated with SWE on a 1000 × 10 cell grid.
4.3. RAIN ON FLAT PLAIN
(a) SWE
31
(b) analytical solution
Figure 4.6: Comparison of the water surface in the steady state computed by SWE
(s) and the analytical solution (s lef t, s right) for MacDonald Test Problem 2.
4.3
Rain on Flat Plain
This scenario serves as a simple test for the solvers capability to simulate
rain. It consists of a flat, even, square plain, 100 m × 100 m in size, with a
wall boundary surrounding the simulation area. For the entire duration of
the simulation, rain falls evenly at a constant rate of 50.4 mm/h. Friction
is modeled using Manning’s friction law with n = 0.035, although friction
should not play a role in theory, since no water movement across the terrain
should occur.
As expected, the water level rises linearly over the course of the simulation (see Figure 4.7a) and matches the analytically computed final height of
1.68 cm after 1200 s (Figure 4.7b).
32
4. VALIDATION
(a) water height over time
(b) final water level (t = 1200 s)
Figure 4.7: Water height (h) for the Rain on Flat Plain scenario. Simulated with
SWE on a 1000 × 1000 cell grid.
4.4
Dam Breaks
Dam breaks are commonly used as test scenarios for shallow water solvers
(see [26], [11]), as they produce fast-moving but predictable waves and, in
the case of dam breaks on dry ground, test the capability of the solver to
correctly simulate a moving wet-dry interface.
4.4.1
Diagonal Dam Break
This scenario consists of a flat, even, square plain, 100 m × 100 m in size,
with a wall boundary surrounding the simulation area. Initially, the terrain
is divided along the top-left to bottom-right diagonal and the upper-right
half is flooded up to a height of 1 m. The water is released at the start of
the simulation, simulating a breaking dam. The simulation uses Manning’s
friction law with n = 0.035.
This scenario features relatively high wave speeds and requires the simulation of shock as well as rarefaction waves. The wave patterns arising
along the top right to bottom left diagonal (slice orientation illustrated in
Figure 4.9) can be seen in the water height plot in Figure 4.10.
When comparing SWE’s solution for this scenario with those computed
by the FullSWOF solver, some differences can be observed. The first small
deviations can be seen in Figure 4.8c and Figure 4.8d (marked by circles),
where the water reflected from the left and bottom wall moves ahead of the
wavefront. In Figure 4.8g and Figure 4.8h, a more pronounced difference
4.4. DAM BREAKS
33
(a) t = 0 s (SWE)
(b) t = 0 s (FullSWOF)
(c) t = 12 s (SWE)
(d) t = 12 s (FullSWOF)
(e) t = 24 s (SWE)
(f) t = 24 s (FullSWOF)
Figure 4.8: 2D-Visualization of the water height (h) in the Diagonal Dam Break
scenario. Simulated with SWE (left) and FullSWOF (right) on a 1000 × 1000 cell
grid. Notable differences between the solutions are marked with gray circles.
34
4. VALIDATION
(g) t = 36 s (SWE)
(h) t = 36 s (FullSWOF)
(i) t = 48 s (SWE)
(j) t = 48 s (FullSWOF)
(k) t = 60 s (SWE)
(l) t = 60 s (FullSWOF)
Figure 4.8: 2D-Visualization of the water height (h) in the Diagonal Dam Break
scenario. Simulated with SWE (left) and FullSWOF (right) on a 1000 × 1000 cell
grid. Notable differences between the solutions are marked with gray circles.
4.4. DAM BREAKS
35
Figure 4.9: Orientation of the slice used for plotting the water height in the
Diagonal Dam Break scenario.
arises as the wavefront hits the bottom left corner. For a direct comparison
of the water heights along a diagonal slice, see Figure 4.11.
36
4. VALIDATION
(a) t = 0 s
(b) t = 12 s
(c) t = 24 s
(d) t = 36 s
(e) t = 48 s
(f) t = 60 s
Figure 4.10: Water height (h) plot for a diagonal slice (see Figure 4.9) of the
Diagonal Dam Break scenario. Simulated with SWE on a 1000 × 1000 cell grid.
4.4. DAM BREAKS
37
(a) t = 24 s
(b) t = 36 s
(c) t = 48 s
(d) t = 60 s
Figure 4.11: Comparison of water heights (h) for the Diagonal Dam Break scenario computed by SWE (red) and FullSWOF (blue). Simulated on a 1000 × 1000
cell grid.
38
4.4.2
4. VALIDATION
Radial Dam Break
In this scenario, a 3 m high cylinder of water with a radius of 100 m is released
at the center of a 1 km×1 km dry plain. This causes the water to spread very
rapidly in all directions across the plain, leading to high speeds and quickly
decreasing water heights.
Due to the high wave speeds at extremely low water heights, the solver
exhibits some unexpected (and unrealistic) behavior: The water is able to
spread much faster in the cardinal directions, while taking significantly longer
to flow along the diagonals. This is likely due to the fact that a simulation
step only computes updates for cardinally adjacent cells, while two simulation
steps are required for water to move one cell diagonally. While this difference
is usually negligible, it becomes significant when speeds are high, but a lot
of momentum is lost every timestep, due to water spreading into previously
empty cells.
When simulating the scenario without friction, the directional preference
is extremely pronounced, since speeds remain high for the entire duration of
the simulation. The water can be seen shooting outwards in cross pattern in
Figure 4.12b and onwards, with the diagonal expansion lagging far behind.
As the simulation continues, the center of the domain dries out completely
(see Figure 4.12e), as, without friction, there is nothing holding the water in
place.
Next, the scenario is simulated using Manning’s friction model with n =
0.035. The friction significantly slows down the expansion of the water, which
also lessens the directional preference. Again, the water begins moving in a
cross pattern (see Figure 4.13b), however, as the speeds are reduced, the
water moving diagonally slowly catches up.
Additionally, the center of the domain does not dry out within the 120 s
duration of the simulation, as the waters speed is quickly diminished by
friction, leading to the more realistic formation of a puddle (see Figure 4.13f),
rather than the water escaping towards infinity.
4.4. DAM BREAKS
39
(a) t = 0 s
(b) t = 12 s
(c) t = 24 s
(d) t = 36 s
(e) t = 60 s
(f) t = 120 s
Figure 4.12: 2D-Visualization of the water height (h) in the Radial Dam Break
scenario without friction. Simulated with SWE on a 320 × 320 cell grid.
40
4. VALIDATION
(a) t = 0 s
(b) t = 12 s
(c) t = 24 s
(d) t = 36 s
(e) t = 60 s
(f) t = 120 s
Figure 4.13: 2D-Visualization of the water height (h) in the Radial Dam Break
scenario with friction. Simulated with SWE on a 320 × 320 cell grid.
5
Results
This chapter shows the results of performance tests for the implementation of
the flood simulation solver, as well as an example of a real-world application.
5.1
Performance
As achieving good computational performance compared to other flood simulation software was an important goal, a comparison to the FullSWOF solver
was conducted for various cases of the Rain on Flat Plain and Diagonal Dam
Break scenarios. Simulations were run with different grid sizes (100 × 100,
200 × 200, 300 × 300 and 1000 × 1000 cells). The main measures for comparison are the total computation time, which is typically the most important
in practical applications, the number of timesteps computed, showing the
effects of the differences in the implementation of the solver on the efficiency
of the simulation, and the rate at which single cell updates are performed,
estimating the algorithmic efficiency of the implementations and execution
models.
5.1.1
Rain on Flat Plain
This scenario involves no water movement apart from rainfall in the expected solution. The number of iterations computed therefore reflects how
the solvers deal with timestep constraints due to rain.
As seen in Table 5.1, the FullSWOF solver computes significantly more
timesteps than SWE. This is due to the fact that in FullSWOF’s implementation of the CFL condition, the timestep is always restricted to at most 0.4
times the length of a grid cell, while SWE has no fixed restriction on the
length of a timestep and instead computes a limit based on the rainfall rate,
41
42
Solver configuration
SWE
FullSWOF
5. RESULTS
100 × 100
812
3000
Grid size
200 × 200 300 × 300 1000 × 1000
1623
6000
2435
9000
8119
30002
Table 5.1: Comparison of the number of timesteps computed for the Rain on Flat
Plain scenario. Simulated time: 1200 s.
Grid size
200 × 200 300 × 300 1000 × 1000
Solver configuration
100 × 100
SWE, single task
no vectorization
SWE, single task
with vectorization
SWE, 4 MPI tasks
no vectorization
SWE, 4 MPI tasks
with vectorization
FullSWOF
1.43 s
10.5 s
35.31 s
1363.76 s
0.91 s
6.54 s
21.65 s
852.57 s
0.39 s
2.66 s
8.92 s
343.45 s
0.24 s
1.70 s
5.54 s
213.75 s
8s
57 s
186 s
6921 s
Table 5.2: Comparison of total run times for the Rain on Flat Plain scenario.
Simulated time: 1200 s.
allowing much larger and therefore overall fewer timesteps. As a result, the
total computation time, seen in Table 5.2, differs greatly.
When comparing the element update rates (Table 5.3) between different
grid sizes, it can be seen that they initially increase with the size of the grid,
as the effects of computational overhead decrease, but drop again for large
grids, when the size of the L2 cache is exceeded, particularly for SWE.
The performance gain from MPI parallelization is nearly linear with the
number of tasks up to the number of cores available on the CPU. In these
tests, vectorization leads to an additional speed-up factor of about 1.6.
5.1. PERFORMANCE
43
Grid size
200 × 200 300 × 300 1000 × 1000
Solver configuration
100 × 100
SWE, single task
no vectorization
SWE, single task
with vectorization
SWE, 4 MPI tasks
no vectorization
SWE, 4 MPI tasks
with vectorization
FullSWOF
5.68
6.18
6.21
5.93
8.92
9.93
10.1
9.48
20.8
24.4
24.6
23.6
33.8
38.2
39.6
38.0
3.75
4.21
4.35
4.33
Table 5.3: Comparison of the simulation speed (in million element updates / s)
for the Rain on Flat Plain scenario. Simulated time: 1200 s.
5.1.2
Diagonal Dam Break
Here, the timesteps are constrained by the wave speeds and thus, the difference in the number of timesteps computed (Table 5.4) between SWE and
FullSWOF is much smaller.
As the simulation is overall slightly shorter than in the previous example,
the effects of computational overhead on small grids are more significant,
especially for FullSWOF, where the averaged element update rates, seen in
Table 5.6, are much lower as a consequence. This results in a continuous
increase in element update rates with increasing grid size for FullSWOF,
within the range of grid sizes tested here, while the values for SWE again
show the effects of cache usage.
Again, the gains from parallelization are roughly linear with the number
of tasks, within the limit of the available CPU cores.
Solver configuration
SWE
FullSWOF
100 × 100
Grid size
200 × 200 300 × 300 1000 × 1000
579
572
1204
1193
1839
1827
6518
6353
Table 5.4: Comparison of the number of timesteps computed for the Diagonal
Dam Break scenario. Simulated time: 60 s.
44
5. RESULTS
Solver configuration
100 × 100
SWE, single task
SWE, 4 MPI tasks
FullSWOF
0.96 s
0.26 s
3s
Grid size
200 × 200 300 × 300 1000 × 1000
7.39 s
1.98 s
16 s
25.07 s
6.85 s
48 s
1031.3 s
276.59 s
1583 s
Table 5.5: Comparison of total run times for the Diagonal Dam Break scenario.
Simulated time: 60 s.
Solver configuration
100 × 100
SWE, single task
SWE, 4 MPI tasks
FullSWOF
6.03
22.3
1.91
Grid size
200 × 200 300 × 300 1000 × 1000
6.52
24.3
2.98
6.60
24.2
3.43
6.35
23.6
4.01
Table 5.6: Comparison of the simulation speed (in million element updates / s)
for the Diagonal Dam Break scenario. Simulated time: 60 s.
5.2. APPLICATION
5.2
45
Application
As an example for a realistic application of the flood simulation software, the
results of severe rain in Mecca, Saudi Arabia, were simulated, using digitally
reconstructed topographic data [27].
Rainfall is rare in this region, but often quite severe when it occurs, making flooding a big concern for the local authorities. Figure 5.1 shows the
results of one such flood in 1941. Recent large-scale landscaping intended to
accommodate the increasing number of pilgrims to the holy sites, in combination with the naturally poor infiltration due to the dry and sandy ground,
further exacerbate the problem.
The 16 km × 9 km area was subdivided into 1270 blocks of 32 × 18 cells,
varying in size depending on the required level of detail in their respective
location, resulting in a total of 731520 cells. Constant rainfall of 55 mm/h
was simulated for roughly 5750 s across the entire terrain.
As expected, the simulation shows the rain water accumulating in valleys
and basins (clearly visible in Figure 5.2c and onwards). In the flatter locations outside of the mountainous terrain at the center of the simulation area,
the water rises more evenly, resulting in less flow across the terrain. This, in
combination with the lack of significant variation in the topography, allows
the use of much larger cells for the simulation in these areas.
Figure 5.3 shows a closer view of the valley containing the Masjid alH
arām,
the Grand Mosque, (see Figure 5.4 for its exact location) over the
.
course of the simulation. The mosque itself lies at a lower elevation than
most of the city, causing it to flood as water collects in the deepest parts of
the valley.
Figure 5.1: Mecca flooded after heavy rainfalls in 1941. Source: [27].
46
5. RESULTS
(a) t = 0 s
(b) t = 1150 s
Figure 5.2: Top-down view of the results of the Mecca simulation. Area subdivided into 1270 blocks. Coloring represents water height (h) on a logarithmic
scale.
5.2. APPLICATION
47
(c) t = 2300 s
(d) t = 3450 s
Figure 5.2: Top-down view of the results of the Mecca simulation. Area subdivided into 1270 blocks. Coloring represents water height (h) on a logarithmic
scale.
48
5. RESULTS
(e) t = 4600 s
(f) t = 5750 s
Figure 5.2: Top-down view of the results of the Mecca simulation. Area subdivided into 1270 blocks. Coloring represents water height (h) on a logarithmic
scale.
5.2. APPLICATION
49
(a) t = 0 s
(b) t = 1150 s
(c) t = 2300 s
Figure 5.3: Side view of the simulation results for the valley containing the Grand
Mosque (see Figure 5.4). Coloring represents water height (h) on a logarithmic
scale.
50
5. RESULTS
(d) t = 3450 s
(e) t = 4600 s
(f) t = 5750 s
Figure 5.3: Side view of the simulation results for the valley containing the Grand
Mosque (see Figure 5.4). Coloring represents water height (h) on a logarithmic
scale.
5.2. APPLICATION
51
Figure 5.4: Location of the Grand Mosque in the side view of the simulation
results. Pictured after 2300 s of rain.
The software used for this simulation is the result of the integration of the
modified SWE solver into current research code designed to simulate flood
scenarios based on data from Geographic Information System (GIS), based
on PeanoClaw [28].
6
Conclusion
In this thesis, a solver for the shallow water equations within the SWE framework was extended for the simulation of rain-induced floods with the aim of
creating a faster alternative to the FullSWOF solver. The HLLE solver was
chosen as the basis for this work, because of its computational efficiency and
suitability for parallelization and vectorization. Source terms for rain and
friction were added to the shallow water model underlying the solver. Explicit treatment of rain data was added and an addition made to the CFL
condition used to preserve numerical stability, allowing it to account for possible water flow resulting from the added rain water. A semi-implicit approach
is chosen for the treatment of friction, as it offers the best compromise between accuracy, stability and computational complexity. The friction model
is based on Manning’s friction law and can be used with different roughness
coefficients matching the ground material throughout the domain. For the
simulation of test scenarios involving rivers and channels, two new boundary
types for SWE’s simulation blocks were implemented, dealing with situations
in which the height or the discharge rate at the edges remain constant under
regular conditions.
Validation results for a number of common test scenarios, including steady
state flow scenarios from the SWASHES collection and MacDonald’s test
problems, as well as rain and dam break scenarios, show that the simulation
quality of the extended SWE software is comparable to that achieved by
the FullSWOF solver with an equal test setup and closely matches available
analytical solutions for most scenarios. The solver is computationally efficient
and scales well on high-performance computers through the use of MPI and
R compilers.
OpenMP parallelization as well as auto-vectorization on Intel
MPI parallelization leads to a nearly linear speedup with the number of tasks
when a sufficient number of processor cores is available and vectorization
gives a further boost to the execution speed on modern CPU architectures.
53
54
6. CONCLUSION
Overall this leads to a speed-up factor of nearly 10 compared to FullSWOF
on certain test setups, while producing comparably accurate results. On
systems with a greater degree of parallelism, the speed gain is likely even
more significant.
In future work, the performance for the simulation of real-world scenarios
can be further improved by using adaptively refined simulation grids. An
example of multiple grid resolutions used in one simulation can be seen in the
Mecca example (section 5.2), reducing the amount of computations needed
to simulate flat, open areas, while simulating at a high level of detail on
more diverse terrain. Allowing the grid resolution to automatically adapt
throughout the simulation would allow the solver to increase the level of detail
simulated in locations where actual change is occurring, while minimizing
the time spent on less interesting parts of the domain without the need to
manually predict the relevant areas and setup according block sizes.
Bibliography
[1] A. J. C. Barré de Saint-Venant, “Théorie du mouvement non-permanent
des eaux, avec application aux crues des rivières et à l’introduction
des marées dans leur lit,” Comptes Rendus de l’Académie des Sciences,
vol. 73, pp. 147–154, 1871.
[2] O. Delestre, F. Darboux, F. James, C. Lucas, C. Laguerre, and
S. Cordier, “FullSWOF: A Free Software Package for the Simulation of
Shallow Water Flows,” Research Report, Mapmo, université d’Orléans;
Institut National de la Recherche Agronomique, Jan. 2014.
[3] A. Breuer and M. Bader, “Teaching Parallel Programming Models on
a Shallow-Water Code,” in Proceedings of the 2012 11th International
Symposium on Parallel and Distributed Computing, ISPDC ’12, (Washington, DC, USA), pp. 301–308, IEEE Computer Society, 2012.
[4] R. J. LeVeque, Finite-Volume Methods for Hyperbolic Problems. Cambridge University Press, 2002.
[5] R. J. LeVeque, D. L. George, and M. J. Berger, “Tsunami Modelling with Adaptively Refined Finite Volume Methods,” Acta Numerica,
vol. 20, pp. 211–289, May 2011.
[6] M. Bader, “SWE – Anatomy of a Shallow Water Solver.” http://www5.
in.tum.de/SWE/lugano2013/swe_anatomy.pdf. Accessed: August 23,
2015.
[7] O. Delestre, S. Cordier, F. James, and F. Darboux, “Simulation of RainWater Overland-Flow,” in 12th International Conference on Hyperbolic
Problems (J.-G. L. E. Tadmor and A. Tzavaras, eds.), vol. 67 of Proceedings of Symposia in Applied Mathematics, (College Park, United States),
pp. 537–546, American Mathematical Society, June 2008.
[8] V. Te Chow, Open-Channel Hydraulics. McGraw-Hill civil engineering
series, McGraw-Hill, 1959.
57
58
BIBLIOGRAPHY
[9] M. Bader, A. Breuer, W. Hölzl, and S. Rettenberger, “Vectorization of
an Augmented Riemann Solver for the Shallow Water Equations,” in
Proceedings of the 2014 International Conference on High Performance
Computing and Simulation (W. W. Smari and V. Zeljkovic, eds.), HPCS
2014, pp. 193–201, IEEE, Aug. 2014.
[10] R. J. LeVeque, “Wave Propagation Algorithms for Multidimensional
Hyperbolic Systems,” J. Comput. Phys., vol. 131, pp. 327–353, Mar.
1997.
[11] C. Berthon and F. Foucher, “Efficient Well-Balanced Hydrostatic
Upwind Schemes for Shallow-Water Equations,” J. Comput. Phys.,
vol. 231, no. 15, pp. 4993–5015, 2012.
[12] D. L. George, Finite Volume Methods and Adaptive Refinement for
Tsunami Propagation and Inundation. PhD thesis, University of Washington, 2006.
[13] H. Lewy, K. Friedrichs, and R. Courant, “Über die partiellen Differenzengleichungen der mathematischen Physik,” Mathematische Annalen,
vol. 100, pp. 32–74, 1928.
[14] R. Courant, K. Friedrichs, and H. Lewy, “On the Partial Difference
Equations of Mathematical Physics,” IBM J. Res. Dev., vol. 11, pp. 215–
234, Mar. 1967.
[15] D. Bale, R. J. LeVeque, S. Mitran, and J. A. Rossmanith, “A WavePropagation Method for Conservation Laws and Balance Laws with Spatially Varying Flux Functions,” SIAM J. Sci. Comput., vol. 24, pp. 955–
978, 2002.
[16] D. L. George, “Augmented Riemann Solvers for the Shallow Water
Equations over Variable Topography with Steady States and Inundation,” J. Comput. Phys., vol. 227, pp. 3089–3113, Mar. 2008.
R
[17] Intel Corporation, A Guide to Vectorization with IntelC++
Compilers, 2012.
[18] N. Schaffroth, “A Vectorizable Riemann Solver for the Shallow Water
Equations,” Research Report, Technische Universität München, 2015.
[19] A. Harten, P. D. Lax, and B. van Leer, “On Upstream Differencing
and Godunov-Type Schemes for Hyperbolic Conservation Laws,” in
Upwind and High-Resolution Schemes (M. Hussaini, B. van Leer, and
J. Van Rosendale, eds.), pp. 53–79, Springer Berlin Heidelberg, 1997.
BIBLIOGRAPHY
59
[20] B. Einfeldt, “On Godunov-type Methods for Gas Dynamics,” SIAM J.
Numer. Anal., vol. 25, pp. 294–318, Apr. 1988.
[21] F. Bouchut, Nonlinear Stability of Finite Volume Methods for Hyperbolic
Conservation Laws and Well-Balanced Schemes for Sources. Frontiers
in Mathematics, Basel, Boston, Berlin: Birkhäuser, 2004.
[22] F. R. Fiedler and J. A. Ramirez, “A Numerical Method for Simulating
Discontinuous Shallow Flow over an Infiltrating Surface,” Int. J. Numer.
Meht. Fluids, vol. 32, no. 2, pp. 219–239, 2000.
[23] O. Delestre, C. Lucas, P.-A. Ksinant, F. Darboux, C. Laguerre, T. N. T.
Vo, F. James, and S. Cordier, “SWASHES: a compilation of Shallow
Water Analytic Solutions for Hydraulic and Environmental Studies,”
Int. J. Numer. Meth. Fluids, vol. 72, pp. 269–300, May 2013.
[24] S. Noelle, Y. Xing, and C.-W. Shu, “High-Order Well-Balanced Finite
Volume WENO Schemes for Shallow Water Equation with Moving Water,” J. Comput. Phys., vol. 226, no. 1, pp. 29 – 58, 2007.
[25] I. Macdonald, M. J. Baines, N. K. Nichols, and P. G. Samuels, “Analytic Benchmark Solutions for Open-Channel Flows,” J. Hydraul. Eng.,
vol. 123, no. 11, pp. 1041–1045, 1997.
[26] D. Ambrosi, “Approximation of Shallow Water Equations by Roe’s Riemann Solver,” Int. J. Numer. Meth. Fluids, vol. 20, pp. 157–168, 1995.
[27] A. K. Thabet, N. Smith, R. Wittmann, and J. Schneider, “A Visual
Framework for Digital Reconstruction of Topographic Maps,” Technical Report, King Abdullah University of Science and Technology, Sept.
2014.
[28] K. Unterweger, T. Weinzierl, D. I. Ketcheson, and A. Ahmadia,
“PeanoClaw – A Functionally-Decomposed Approach to Adaptive Mesh
Refinement with Local Time Stepping for Hyperbolic Conservation
Law Solvers,” Research Report TUM-I1332, Technische Universität
München, 2013.