Time Series Plot from the Data Portal
CUAHSI Hydrologic Information
Systems and Web Services
By David R. Maidment
With support from many collaborators: Ilya Zaslavsky,
David Valentine, Reza Wahadj, Chaitan Baru , Praveen
Kumar, Michael Piasecki, Rick Hooper, Jon Duncan, David
Tarboton, Jeff Horsburgh, Venkat Lakshmi, Chunmaio
Zheng, Xu Liang, Yao Liang, Ken Reckhow, Upmanu Lall,
LeRoy Poff, Dennis Lettenmaier, Barbara Minsker, ……
And many graduate students and post-docs: Venkatesh
Merwade, Tim Whiteaker, Jon Goodall, Gil Strassberg, Ben
Ruddell, Luis Bermudez, Bora Beran, ……
Thanks to everyone for all their help!
HIS Goals
• Hydrologic Data Access System – better access to a large volume of high quality hydrologic data
• Support for Observatories – synthesizing hydrologic data for a region
• Advancement of Hydrologic Science – data modeling and advanced analysis
• Hydrologic Education – better data in the classroom, basin-focused teaching
HIS User Assessment (Chapter 4 in Status Report)
Which of the four HIS goals is most important to you?
Data
Access
Science
Observatory support
Education
HIS Goals
• Hydrologic Data Access System – better access to a large volume of high quality hydrologic data
• Support for Observatories – synthesizing hydrologic data for a region
• Advancement of Hydrologic Science – data modeling and advanced analysis
• Hydrologic Education – better data in the classroom, basin-focused teaching
Water Data
Water quantity and quality Soil water Rainfall & Snow
Remote sensing
Meteorology
Modeling
Water Data Web Sites
NWISWeb site output
# agency_cd Agency Code
# site_no USGS station number
# dv_dt date of daily mean streamflow
# dv_va daily mean streamflow value, in cubic-feet per-second
# dv_cd daily mean streamflow value qualification code
#
# Sites in this file include:
# USGS 02087500 NEUSE RIVER NEAR CLAYTON, NC
# agency_cd site_no dv_dt dv_va dv_cd
USGS 02087500 2003-09-01 1190
Time series of
USGS 02087500
USGS 02087500
USGS 02087500
2003-09-02
2003-09-03
2003-09-04
649
525
486 streamflow at a gaging station
USGS 02087500
USGS 02087500
USGS 02087500
USGS 02087500
2003-09-05
2003-09-06
2003-09-07
2003-09-08
733
585
485
463
USGS 02087500
USGS 02087500
USGS 02087500
2003-09-09
2003-09-10
2003-09-11
673
517
454
CUAHSI Hydrologic Data Access System http://river.sdsc.edu/HDAS
EPA
NCDC NASA
NWS
USGS
Observatory Data
A common data window for accessing, viewing and downloading hydrologic information
Observation Stations Map for the US
Ameriflux Towers (NASA & DOE) NOAA Automated Surface
Observing System
USGS National Water Information System NOAA Climate Reference Network
NWIS Station Observation Metadata
Describe what has been measured at this station
Web Page Scraping
http://nwis.waterdata.usgs.gov/nwis/discharge?site_no=02087500&agency_cd=USGS&....
Programmatically construct a
URL string as produced by manual use of the web page
Parse the resulting ASCII file
CUAHSI Web Services
Web Application: Data Portal
Your application
• Excel, ArcGIS, Matlab
• Fortran, C/C++, Visual Basic
• Hydrologic model
• …………….
Your operating system
• Windows, Unix, Linux, Mac
Internet S imple
O bject
A ccess
P rotocol
Web Services
Library
Series and Fields
Features
Point, line, area, volume
Discrete space representation
Series – ordered sequence of numbers
Time series – indexed by time
Frequency series – indexed by frequency
Surfaces Fields – multidimensional arrays
Continuous space representation Scalar fields – single value at each location
Vector fields – magnitude and direction
Random fields – probability distribution
North American Regional
Reanalysis of Climate
Precipitation Evaporation
NetCDF format
Variation during the day, July 2003 mm / 3 hours
Arc Hydro Time Series
Geospatial features associate with time series
HydroID
2906
Feature Class (HydroID)
Attribute Series Table (FeatureID)
Arc Hydro Time Series Object
TSDateTime
TSValue
Feature Class
(point, line, area)
FeatureID
TSType
TSType Table
NetCDF Data Model
Time Dimensions and
Coordinates
Value
Space (x,y,z)
Variable
Attributes
Data Sources
Storet
NASA
Ameriflux
Extract NCDC Unidata
NWIS NCAR
Transform
CUAHSI Web Services
Excel
ArcGIS
Load
Matlab
Applications http://www.cuahsi.org/his/
Access SAS
Fortran
Visual Basic
C/C++
Some operational services
Core Web Services
Service
GetSites
Input
Obs Network
GetSiteInfo
GetValues
GetChart
Station Code
GetVariables Obs Network or data source
GetVariableInfo Variable code
Station code or lat/long point, variable code, begin date, end date
As for GetValue
Output
All station codes in network
Lat/long, station name
All variable codes
Description of variable
A time series of values
A chart plotting the values
Operational Services
Service
GetSites
Ameriflux Daymet MODIS NWIS NAM
Yes
GetSiteInfo
GetChart
Yes
GetVariables Yes
GetVariableInfo Yes
GetValues Yes Yes Yes
Yes
Yes
Yes Yes
Yes
Yes
Yes
NCEP North
American
Forecast Model
12 Km grid for continental US
CUAHSI Web Services
http://www.cuahsi.org/his/webservices.html
Water OneFlow
• Like Geospatial OneStop, we need a “Water
OneFlow” – a common window for water data and models
Federal
State
Local
Academic
• Advancement of water science is critically dependent on integration of water information
Conclusions
• It would be good to define a collaboration between
CUAHSI and Unidata for web services that has a consistent vocabulary
• We in CUAHSI would defer to Unidata for definition of how to ingest real-time weather information as fields (netCDF with CF conventions)
• Try to define services that represent “ time histories ” of variables, past, present and future e.g. precipitation, evaporation, surface temp
