A Super-Regional Modeling Testbed
for Improving Forecasts of
Environmental Processes for the
U.S. Atlantic and Gulf of Mexico
Coasts
Cyberinfrastructure Status Update
March 29, 2011
Status Update : March 29, 2011
•
•
•
•
•
•
•
•
•
Testbed web site
SOS Server
SOS Parser
SLOSH Update
F-TDS Update
Unstructured Grid
IMEDS
CHPS
Matlab Toolbox
Super-Regional Testbed Collaboration
Site
Group-specific http://testbed.sura.org
information
Project-wide
access to data tools
Reports,
Documents
for Management Review
Create documents,
meetings, etc
One-stop for all Testbed Information and Functionality
• Ready for integration/hosting of modeling tools
• Project management tools
Access contro
for project
security
Public
Information
TDS Java SOS Service
• Addition of SOS service capabilities to THREDDS
server
Data Server
THREDDS
–Get capabilities
–Describe sensor
–Get observation
Catalog services
SOS
ISO
WCS
Service Offerings
getCapabilities
describeSensor
getObservation
Payload
XML
XML
XML
• Single and Multi-station Netcdf datafiles:
– EPA Netcdf datafiles
– NOAA NDBC Station Netcdf datafiles
• XML payload designed to be compatible with
existing SOS Parsers
WMS
Javascript SOS Client Parser
•
•
•
•
http://testbedwww.sura.org/sosParse/
IOOS DIF and SWE support
Cross browser support via jQuery.js
GetCapabilities and GetObservation
parsing
• HTML, CSV and JSON string outputs
• Aimed to work with TDS Java SOS plug-in
for time series point observations.
NDBC 12 hours Wave Height from buoy 4400
SLOSH Output Conversion:
Status Update
Justin R. Davis
University of Florida
March 29, 2011
Plan
• Phase I
– Work with Dr. Slinn and get his output converted
• Phase II
– Work with NHC and get native SLOSH output
converted
Note: With the exception of 1 Tecplot figure, all pictures are screen
captures taken of graphics “generated” by the SURA Testbed Sever.
• Phase I
– Dr. Slinn’s Output Format
•
•
•
•
ASCII Text
Initial (x,y,t,value)
Current (timestamp relative to NAP then spatial data)
Approach
– F90 Conversion Program
» Researcher familiarity with Fortran
– Converts to time series output (IMEDS/NetCDF/Tecplot)
– Converts to spatial output (NetCDF/Tecplot)
– Reads time series output from command line or file
» Currently using list of 238 stations compiled by UND
» Only use if within 5 km (variable) of grid cell center
Time series CDL (v3-9)
CF 1.3 Compliant
NetCDF-Java Tools UI
(webstart)
Time Series Output Comparisons
(Tecplot)
Spatial CDL (v3-9)
CF 1-3 and CDM Compliant
Spatial Output Plots (Godiva2)
OWI Wind
OWI Wind + Wave
OWI Wind + Wave + MF
Wind
Wind + Garrat Drag
• Phase I
– Remaining Issues
• Vertical datum not specified
• U, V rotation unclear
• Timestamp output is hardwired
• Phase II
– Native SLOSH Output Format
•
•
•
•
•
Rex File
Binary with Gif-like compression
Contains necessary data/equation (wind)
Lots of existing rex file data
Approach
– Using C Conversion Program
– Based on existing time series extraction tool (rexout)
– Converts to time series output (NetCDF)
» Same CDL as previously developed
– Implementing new NetCDF code into 3 files
» Header
» Time Series
» Sparial
– Implement as Style 4
– Current version: rexout-1-3-jrd-v1-2
• Phase II
– Tests
• Built test cases based on existing rexout cases
–
–
–
–
andrw.lft
audry.bpt
camil.bix
frederic.emo
• At the request of Jesse Feyen, compare NHC Ike
result with Dr. Slinn’s
– Get 80 xy observation stations (Arthur Taylor)
– Run rexout at these stations
–…
2008Ike_gl2.rex =>
2008Ike_gl2.nc
• Phase II
– Remaining Issues
• Need to address spatial output
• Vertical datum not specified
UF
– rexout reads several files in addition to data file
but none have datum, need to read another file.
• Needs additional metadata
– Version number
– Build/run dates
• Needs appropriate calling hooks in program
• Needs “SLOSH-approved” style (e.g. GNU ident)
• Must be compiled in 32-bit
– Status at NHC
• Calls/Emails with Arthur Taylor (developer of rex file) at NHC
• NHC has new code and is integrating
– Develops in Windows (MinGW – Minimalist GNU for Windows)
– Must build NetCDF/HDF from scratch (HDF binaries are not public domain - SZIP)
(done)
– Code setup for time series, needs some re-organization to handle spatial output)
Next Steps
• Get rexout to handle spatial output
• Iterate with NHC regarding source
integration
• Facilitate comparisons between Dr. Slinn’s
results and NHC Results
• Investigate other SLOSH domains
– Sabine Pass
– Extra-tropical Domain
The OPeNDAP Story

Local access to remote data...
which use well-known API’s like
netCDF are re-linked with the OPeNDAP client
library
 Data providers install OPeNDAP server
software (several are available) and data files
 Clients can access remote data via the API as
if the data files were local
 Clients
23
Server-side Analysis



In general server-side analysis is a
computation made by an OPeNDAP
server at the request of a client.
The specification of the computation is
transmitted to the server via the
OPeNDAP URL.
The data returned are the results (and
only the results) of the computation
24
F-TDS


The Ferret-THREDDS Data Server (FTDS) is an implementation of a general
purpose server-side analysis engine which
plugs into new or existing TDS
installations.
Where TDS is an OPeNDAP server
implemented in Java and available from
Unidata.
25
F-TDS Capabilities

F-TDS takes advantage of several characteristics of
Ferret.




New "virtual" data variables can be defined
Can build the metadata (netCDF header described by
dimensions, coordinate variables and the structure of data
variables) without performing any heavy calculations for
both data read from files and “virtual” data variables
Only performs calculations when the data are requested
Only calculates the minimal set needed to fulfil the current
request
26
ChesROMS: a practical
example

We are serving data from Chesapeake Bay
ROMS Community model




We’d like images of the water velocity model output.
Water velocity is produced in the model as separate
eastward and northward velocities on the U and V
elements of a single Arakawa computational grid.
Need to combine the eastward and northward
components to produce velocity. But to do this, the
values need to be on the same grid.
We do this with an F/TDS script. This produces an
OpenDAP accessible URL, which can be operated on
in standard tools such as Matlab. The eastward and
northward values are translated to a common “rho”
27
grid.
ChessROMS Eastward and
Northward ComputationalGrids
Full view and detail of grid: eastward grid in red, northward grid in green. These
are the original computation points on which the model outputs data.
28
The ChessROMS grids including
the “RHO” grid
Black diamonds are “Rho” grid points. Eastward points in red, northward
points in green. Eastward and northward velocity model output have been29
translated to the RHO grid by the F/TDS script.
Matlab code to plot the velocity
# Get the data from the F/TDS URL
url='http://testbedapps.sura.org/thredds/dodsC/estuarine_hypoxia/c
hesroms/vectors.nc';
nc=mDataset(url);
getVars(nc)
# grab the rotated u,v and grid points. The grid
# is the same for both UBARROT and VBARROT
u=nc{'UBARROT'}(1,:,:);
v=nc{'VBARROT'}(1,:,:);
g=nc{'VBARROT'}(1,:,:).grid
# Convert u and v to a complex for graphing
U=complex(u,v);
# plot the data
figure
pcolorjw(g.lon,g.lat,double(abs(U)));
arrows(g.lon,g.lat,U,.05,'black');
axeq
30
colorbar
The ChesROMS Water Velocity
ChessROMS Absolute Water
Velocity in Meters/second
ChessROMS Absolute Water
Velocity in Meters/second detail
with directional vectors
31
Plotting ChesROMS Water
Temperature



ChesROMS temperature grid seems to overlap
itself and is very difficult to plot.
There is a mask variable in the data set: we use
F/TDS to apply the mask to the temperature
data and produce a plotable data set.
We can not yet plot this in WMS and we don’t
know why, but the Ferret itself has a plotting
engine and the data looks reasonable plotted.
32
ChesROMS Water Temperature
plotted in Ferret
ChessROMS variable temp from
http://testbedapps.sura.org/threddsdev/
dodsC/estuarine_hypoxia/cbofs2/agg-
ChessROMS variable temp_masked
from
33
http://testbedapps.sura.org/threddsdev/d
IMEDS Target Diagrams
• Target Diagrams for error parameters
• Currently developing target diagram format
• Ease of integration to current IMEDS
version
Unstructured Grid Support
CHPS
MATLAB Interoperability Tools
Steps and Status
• Merging njtbx functionality with nctoolbox’s (Brian Schlining)
codebase (http://code.google.com/p/njtbx/)
• Creating methods for SURA Testbed use case (timeseries
extraction, geographic bounding boxes, search api, unstructured
grid support, comparisons/assessment…)
• Merging changes with nctoolbox open source development
on Google Code (http://code.google.com/p/nctoolbox/)
• Testing current state of the toolbox and distributing to others for
feedback (hopefully friendly!)
Whats Next?
• Refactor/other changes based on feedback
• Fill out discrete sampling geometry methods
• Fill out unstructured grid methods
• More testing with lots of datasets
• Leverage with java desktop client front end???