A proposal for testing two-dimensional models
to use in the National Flood Insurance Program
Reinaldo Garcia, PhD
Presentation Outline
Motivation
Status quo
Why we need improvements
What are others doing
Suggested components for new model review
guidelines
Model performance and predictive capability
Depends on many factors, including:
• Model mathematical formulation
• Numerical method
• Appropriate use of boundary conditions
• Adequate configuration of grid or mesh
• Choice of roughness coefficients and time steps
• Data quality
• Expertise of the modeler to perform the simulation.
To ensure adequately testing the predictive capabilities of a 2D
flood model, testing guidelines should be based on
standardized benchmarks and specifications
Status Quo
FEMA requires that any hydrologic or hydraulic computer
model used in support of flood insurance studies must be
reviewed, tested and accepted by a government agency
responsible for the implementation of programs for flood
control and/or regulation of floodplains National Flood
Insurance Program (NFIP) regulations. Subparagraph 65.6.
FEMA relies on other agencies to review and test a model
and determine whether it is scientifically correct and
technically sound.
FEMA List: Numerical Models Meeting the Minimum
Requirement of NFIP Current Nationally Accepted Hydraulic
Models
Why we need improvements
FEMA does not establish a standard testing procedure to
determine if a model is scientifically or computationally
correct
No specifications or metrics to verify that the model can
provide adequate or accurate information to support NFIP
studies.
Inconsistent testing methods depending on the agency or
modeler doing the evaluation.
Federal agencies often have no budget or time test a
model that is not on the FEMA list.
Some government agencies also develop their own
models…
Status Quo
Is not open
Is not fully reliable
Hampers innovation
Does not serve well FEMA’s mission
What are others doing?
Components of sound model review guidelines
Model verification
Model validation
Standardized benchmarks
Metrics
Open
Model vs. Real World
Qualification
Fidelity of the equations
to reality
Validation
Determining
the degree to
which the
model is an
accurate
representation
of the real
world.
Calibration
Adjusting numerical
or physical
parameters in the
program to improve
comparison with
data
Verification
Determining that the program
accurately represent the
conceptual model
(program solves the equations
right)
Verification
Example of model verification
Comparison with Lynch & Gray 2-D analytical solutions
Example of model verification
Validation
“No one believes model results except the one who
performed the calculation,
...everyone believes observed or measured data except the
one who did the measurements”
P. J. Roache
Validation
UK Benchmarks
Provide a dataset against which 2D hydraulic modeling
packages can be evaluated
Provide evidence to ensure that such models used for
flood and coastal risk management are capable of
adequately predicting the variables upon which flood
risk management decisions are based.
UK Benchmarks
Flooding of a disconnected water body
Assess basic capability to simulate flooding of disconnected water bodies on
floodplains or coastal areas and wetting and drying of floodplains
Filling of flood plain depressions
Tests capability to predict inundation extent and final flood depth flow low
momentum flow over complex topographies. Also tests flooding of disconnected
water bodies and wetting and drying of floodplains. Predicts inundation extent,
with an emphasis on final distribution of water rather than peak levels.
Momentum conservation over a small
(0.25m high) obstruction
Tests capability to simulate flow at relatively low depths over an obstruction with
an adverse slope.
Speed of flood propagation over an
extended flood plain
Tests simulation of speed of propagation of flood wave and the prediction of
velocities at the leading edge of the advancing flood.
Valley flooding
Dam break
Tests simulation of major flood inundation at the valley scale.
Tests simulation of shocks and wake zones close to a failing dam. This could also
be applied to other examples like rapid opening of sluice gates, failures of
embankments, etc.
River to floodplain linking
Evaluates capability to simulate flood volume transfer between rivers and
floodplains using 1D and 2D model linking.
Rainfall and sewer surcharge
Tests capability to simulate shallow flows in urban areas with inputs from rainfall
and sewer surcharge.
Test relevance for specific applications
Application
Large scale flood risk mapping
Catchment Flood Management Plan
Flood Risk Assessment and detailed flood
mapping
Strategic Flood Risk Assessment
Flood hazard mapping
Contingency planning for real time flood
management
Reservoir flooding mapping
Test
1&2
1, 2 & 7
1, 2, 3 & 7
1, 2, 3, 4, 7 & 8
1, 2, 3, 4, 7 & 8
1, 2, 3, 4, 5, 7 & 8
1, 2, 3, 4, 5 & 6
Benchmark Examples
Flooding a disconnected water body
UK Benchmark Examples
Assess model’s ability to conserve momentum over an
obstruction in the topography.
Important when simulating flooding in urbanized
floodplains.
UK Benchmarks
Should be complemented with
verification
Americas Society of Civil Engineers
ASCE: Verification and Validation of 3D Free-Surface Flow
Models, 2008
Three main components:
– Analytical solutions
– Lab/flume experiments
– Field scale real-world applications
Provides a data set for 3D models only
Suggestions for Model Review Guidelines
Verification should precede validation
Validation based on set of benchmarks
Acceptable error tolerances (metrics)
Benchmarks based on intended use:
not all models are necessarily adequate for all projects
Standardized test and benchmark data sets should be
public and freely available
Implementing these model review guidelines would better
serve FEMA’s mission.
Thank you / Questions
rey@hydronia.com