Hydrologic Modeling in 2011
David R. Maidment
Center for Research in Water Resources
University of Texas at Austin
Leader of the CUAHSI
Hydrologic Information System Project
With acknowledgements to Rick Hooper, David Tarboton & Barbara Minsker
Hydrologic Modeling in 2011
• The charge and challenges
• Hydrologic information system – web
services
• Integrating models and data using
scientific workflows
• Hydrologic Observing System
Hydrologic Modeling in 2011
• The charge and challenges
• Hydrologic information system – web
services
• Integrating models and data using
scientific workflows
• Hydrologic Observing System
Workshop Charge
• What new technologies for observing,
simulating, and tele-communicating will emerge
over the next 5-10 years?
• how will they change the grand challenges for
modeling, what will those challenges be?
• Challenge for this session:
– How all the new devices/opportunities emerging in the
realm of “cyber-infrastructure”— including, perhaps
especially, visualization schemes — might change the
way models are developed and applied, including the
new kinds of scientific questions to be asked in
association with modeling.
Hydrologic Modeling
• We want to trace the
movement of water,
chemical and
biological constituents
through atmospheric,
surface and
subsurface water
• We want to do water,
mass and energy
balances
Hydrologic Information System
• A system is a connected
set of components e.g.
University of Texas System
• A web-based system is a
set of components
connected using the
internet
• A hydrologic information
system (HIS) is a webbased system linking
hydrologic databases, tools
and models
CUAHSI HIS partner institutions
USGS Water Watch System
A national hydrologic observing system already exists – CUAHSI adds to it
Real-time Water Quality Estimates
Estimated total nitrogen
cfs
mg/L
Stream discharge
CUAHSI Member Institutions
105 Universities as of May 2006
Challenges
• How to use test-beds to design real
WATERS Observatories?
• How to share data from the test-beds with
the whole community?
• How to include CUAHSI/CLEANER data
not collected in the test-beds?
• How to empower individual scientists?
• How to make use of petascale computing?
Hydrologic Modeling in 2011
• The charge and challenges
• Hydrologic information system – web
services
• Integrating models and data using
scientific workflows
• Hydrologic Observing System
CUAHSI Web Services
Your application
• Excel, ArcGIS, Matlab
• Fortran, C/C++, Visual Basic
• Hydrologic model
• …………….
Your operating system
• Windows, Unix, Linux, Mac
Web Application: Data Portal
Internet
CUAHSI
Web Services
Library
http://www.cuahsi.org/HIS/
Simple
Object
Access
Protocol
Water Data
Water quantity
and quality
Soil water
Meteorology
Remote sensing
Rainfall & Snow
Modeling
Water Data Web Sites
NWISWeb site output
# agency_cd Agency Code
# site_no USGS station number
# dv_dt
date of daily mean streamflow
# dv_va
daily mean streamflow value, in cubic-feet per-second
# dv_cd
daily mean streamflow value qualification code
#
# Sites in this file include:
# USGS 02087500 NEUSE RIVER NEAR CLAYTON, NC
#
agency_cd
site_no dv_dt
dv_va dv_cd
USGS 02087500
2003-09-01
1190
Time series of
USGS 02087500
2003-09-02
649
USGS 02087500
2003-09-03
525
streamflow at a
USGS 02087500
2003-09-04
486
gaging station
USGS 02087500
2003-09-05
733
USGS 02087500
2003-09-06
585
USGS 02087500
2003-09-07
485
USGS 02087500
2003-09-08
463
USGS 02087500
2003-09-09
673
USGS 02087500
2003-09-10
517
USGS 02087500
2003-09-11
454
CUAHSI Hydrologic Data Access System
http://river.sdsc.edu/HDAS
EPA
USGS
NCDC
NASA
NWS
Observatories
A common data window for accessing, viewing
and downloading hydrologic information
Observation Stations Map for the US
Ameriflux Towers (NASA & DOE)
NOAA Automated Surface
Observing System
USGS National Water Information System
NOAA Climate Reference Network
NWIS Station Observation Metadata
Describe what has been
measured at this station
Web Page Scraping
http://nwis.waterdata.usgs.gov/nwis/discharge?site_no=02087500&agency_cd=USGS&....
Programmatically construct a
URL string as produced by
manual use of the web page
Parse the resulting ASCII file
Data Sources
NASA
Storet
Extract
Ameriflux
CUAHSI
Unidata
NWIS
NCAR
Transform
CUAHSI Web Services
Excel
Visual Basic
C/C++
ArcGIS
Load
Matlab
Applications
http://www.cuahsi.org/his/
Fortran
Access
SAS
Some operational services
Core Web Methods
Method
Input
Output
GetSites
Obs Network
All station codes in network
GetSiteInfo
Station Code
Lat/long, station name
GetVariables
Obs Network or
data source
All variable codes
GetVariableInfo Variable code
Description of variable
GetValues
Station code or
lat/long point,
variable code,
begin date, end
date
A time series of values
GetChart
As for GetValue
A chart plotting the values
Operational Services
Service
Ameriflux Daymet MODIS NWIS NAM
HODM
Bear
Creek
GetSites
Yes
Yes
GetSiteInfo
Yes
GetVariables
Yes
Yes
Yes
GetVariableInfo Yes
GetValues
Yes
GetChart
Yes
Yes
Yes
Yes
Yes
Yes
Yes Yes Yes
Yes
XML Output from GetValues
NWIS
DayMet
MODIS
What is a Data Model
Lets see what Wikipedia says
• A data model is a model that describes in
an abstract way how data is represented
• Data models describe structured data for
storage in data management systems
such as relational databases.
• Early phases of many software
development projects emphasize the
design of a conceptual data model.
CUAHSI Point Hydrologic
Observations Data Model
• A relational database
stored in Access,
PostgreSQL,
SQL/Server, ….
• Stores observation
data made at points
• Consistent format for
storage of
observations from
many different
sources and of many
different types.
Streamflow
Precipitation
& Climate
Water Quality
Groundwater
levels
Soil
moisture
data
Flux tower
data
Hydrologic Observations Data Model (HODM)
Serving investigator data
• Several choices
– You build CUAHSI
compatible services Your implementation of
from your database CUAHSI services
– You copy data into the
HODM and use
CUAHSI services
Your database
– You copy your data to
an HODM and it is
served from SDSC
Standard
CUAHSI
services
HODM
Modeling Services
• Simulation models
can be packaged as
web services
• They can be queried
and provide
responses just like
data archives
• We have an
integrated network of
data sources and
models
A big challenge to integrate all the data streams!
Objective
•
Search multiple heterogeneous data sources simultaneously regardless of
semantic or structural differences between them
What we don’t want …..
NWIS
request
return
request
return
request
return
NAWQA
request
return
NAM-12
request
return
request
return
request
return
request
return
NARR
Michael Piasecki
Drexel University
Semantic Mediator
What we do want …..
request
request
NWIS
request
request
generic
request
request
request
NAWQA
Michael Piasecki
Drexel University
request
request
NARR
HODM
Hydrologic Modeling in 2011
• The charge and challenges
• Hydrologic information system – web
services
• Integrating models and data using
scientific workflows
• Hydrologic Observing System
Regional Storm Water
Modeling Program and Master
Plan for San Antonio
City of
San Antonio
San Antonio Regional Watershed Modeling System
“Bring the models
together”
Rainfall Data:
Rain gages
Nexrad
Floodplain
Management
Geospatial Data:
City, County
SARA, other
Modeling
System
Calibration Data:
Flows
Water Quality
Capital
Water quality
Improvement
planning
Planning
Integrated
Flood
Regional Water
Forecasting
Resources planning
GIS Preprocessors for Hydrologic Models
GIS
Database
Interface
GeoHMS
HEC-HMS
GeoRAS
HEC-RAS
GISGflow
Gflow
Connecting Arc Hydro and Hydrologic Models
Interface
data models
GIS
HMS
Geo
Database
Arc Hydro
data model
RAS
Gflow
Digital Rain Maps from National Weather Service
(03/04/2004)
FEMA 100-year flood plain map in
Bexar County
Regional Watershed Modeling System Case Study
Salado Creek
watershed
Components:
• Arc Hydro Geodatabase
for whole watershed
• HEC-HMS hydrology model
for whole watershed
• HEC-RAS hydraulic model
for Rosillo Creek
RC1 RU
01R CO
02R CO
03R CO
04R CO
05R CO
06R CO07R CO
Bexar County
08R CO
09R CO
Rosillo Creek
watershed
Arc Hydro and HEC-HMS
HEC-HMS
Hydrologic
Model
Arc Hydro
Schematic
Network
Calculates
Flows
Arc Hydro and HEC-RAS
Arc Hydro
Channel
Cross Sections
HEC-RAS
Hydraulic
Model
Calculates
Water Surface
Elevations
Flow Change Points
Models communicate with
one another through Arc Hydro
at designated points
Nexrad Map to Flood Map in
Arc 9 Model Builder
FLO
Flood map
as output
ODP
LAIN
MAP
Model for
flood flow
HMS
Nexrad rainfall map as input
Model
for flood
depth
Web-Accessible Regional
Watershed Modeling System
Complete storage of simulation
models and workflows in geodatabases
Hydrologic Modeling in 2011
• The charge and challenges
• Hydrologic information system – web
services
• Integrating models and data using
scientific workflows
• Hydrologic Observing System
CUAHSI Hydrologic Observing System
A multiscale web portal system for observing and interpreting hydrologic phenomena
by integrating data and models for any location or region in the United States
Multiscale information delivery
1:1,000,000 scale
North American Scale
(e.g. North American
Regional Reanalysis of climate)
1:500,000 scale
Continental US Scale
(coast to coast data
coverage, HIS-USA)
1:100,000 scale
Regional Scale
(e.g. Neuse basin)
1:24,000 scale
Watershed Scale
(e.g. Eno watershed )
Site scale
Site Scale
(experimental site level)
Point
Point Observation Scale
(gage, sampling location)
GeoTemporal Reference Frame
• A defined geospatial
coordinate system for
(x,y,z)
• A defined time
coordinate system
(UTC, Eastern
Standard Time, ….)
• A set of variables, V Variables, V
• Data values v(x,y,z,t)
Time, t
v – data values
Space
(x,y,z)
Data Cube
Series and Fields
Features
Series – ordered sequence of numbers
Time series – indexed by time
Frequency series – indexed by frequency
Point, line, area, volume
Discrete space representation
Surfaces
Fields – multidimensional arrays
Continuous space representation
Scalar fields – single value at each location
Vector fields – magnitude and direction
Random fields – probability distribution
North American Regional
Reanalysis of Climate
Precipitation
Evaporation
Variation during the day, July 2003
NetCDF format
mm / 3 hours
Continuous Space-Time Model –
NetCDF (Unidata)
Time, T
Coordinate
dimensions
{X}
D
Space, L
Variables, V
Variable dimensions
{Y}
Discrete Space-Time Data Model
ArcHydro
Time, TSDateTime
TSValue
Space, FeatureID
Variables, TSTypeID
Hydrologic Flux Coupler
Define the fluxes and flows associated with each hydrovolume
Evaporation
Precipitation
Streamflow
See Chapter 9 of
Status Report for Details
Groundwater recharge
ArcGIS ModelBuilder Application for Automated Water Balancing
Fields
Series
Geospatial
Water Resource Regions and HUC’s
NHDPlus for Region 17E
NHDPlus Reach Catchments
~ 3km2
Reach Attributes
• Slope
• Elevation
• Mean annual flow
– Corresponding velocity
• Drainage area
• % of upstream
drainage area in
different land uses
• Stream order
Ingestion of real-time streamflow
data
A national hydrologic observing system already exists – CUAHSI adds to it
Continental Water Dynamics Model
Hydrologic Information System
Hydrologic
Observing
System
Hydrologic
Modeling
System
Petascale Computing
• 2.6 million river reaches on a 1:100,000
scale map of continental US
• Solve continuity and momentum equations
once on each reach (~ 5.2 million
equations) takes ~ 200 parallel processors
• Pittsburgh Supercomputer Center has 3000
parallel processors
• It is within reach to simulate flows on all
reaches continuously through time with
data assimilation from gaging stations
Conclusions
• Web services support a web-based
hydrologic information system connnecting
data, tools and models
• Models can be configured as web services
• Scientific workflows automate the integration
of components
• A continental water dynamics model is
feasible