OGC Survey for DMAC ST
Luis Bermudez
Director of Interoperability Certification
lbermudez@opengeospatial.org
January 18th, 2011
Washington DC
1
!5 Responses
•
•
•
•
•
•
•
•
NASA JPL
IMOS
ASA
OBIS
EDAC
USGS
FWC
WHOI
•
•
•
•
•
•
•
NOAA / IOOS
DNR – MD
NOAA/NODC
BOEING
SCCOOS
NERACOOS
OBIS-USA
NASA Jet Propulsion Laboratory (JPL)
• Presenter Name : Michelle Gierarch
• The Jet Propulsion Laboratory is the lead U.S.
center for robotic exploration of the solar system,
and conducts major programs in space-based
Earth sciences.
• Are OGC Web Services a standard technology in
your organization? I
– We are trying to be compatible with OGC standards
such as WMS, WCS, and WFS, but they are not our
default "standard" protocol.
OGC Relation
• Are you a member (yes or no) - YES
• If so, which forums do you participate?
– YES .. But no specifics
OGC Standards in your organization
• OGC standards being used
– Lunar Mapping and Modeling Project (LMMP), uses
WMS/WCS/WFS
– WMS to provide rendered image(png and jpeg) from L2
and L3 data to our various tools.
– WCS ( this is on its way. We have services already that are
ready to use this protocol. )
– WFS to search and provide information about natural
events like hurricane…
• Planned OGC standards to be used in the next three
years:
– have plan to use OGC standards where applicable on all of
our web services.
Pros ans Cons of OGC standards
• PRO: - Basic profiles of various OGC standards
do insure a useful compatibility level between
applications
• PRO: - Use of OGC standards fosters
collaboration with other users
• CON: - Small community of OGC participants
leads to a proliferation of rapidly evolving
specifications of little use, sometimes even
overlapping ones
Title of Organization
• Presenter Name : Roger Proctor
• 1 sentence about the organization: Australian
Ocean Data Network / Integrated Marine
Observing System
• Are OGC Web Services a standard technology
in your organization ? Yes
OGC Relation
• Are you a member (yes or no) AODN/IMOS is
a member through the Australian Ocean Data
Centre Joint Facility (AODCJF)
• If so, which forums do you participate? Public?
OGC Standards in your organization
• OGC standards being used
– Here are some of the ways in which we use OGC web services:
– We harvest metadata via WCS, WMF, WFS & SOS getCapabilities documents (for IMOS
and AODN observations).
– We support CSW through Geonetwork for search access via the IMOS/AODN portal, and
potentially other clients.
– SOS is also used to publish data for the South Esk sensor web project.
– GeoServer is used to publish WMS/WFS (for use by the IMOS/AODN portal).
– some IMOS data is published via THREDDS, which implements WMS (ncWMS) and WCS.
– We will generate KML.
– We use SensorML for the South Esk project, and there is talk of a sensor repository that
will support SensorML.
• Planned OGC standards to be used in the next
three years:
– Non-specific, see answer to next question.
Non OGC standards
• Which non OGC standards are you using and
why ?
– We use or consider many standards such as OPeNDAP,
REST services, OAI-PMH, StarFL, ISO metadata standards
etc. Sometimes these overlap or compete with OGC
standards. We try to adopt the most widely used standards
(where possible). We are, nevertheless, supportive of the
OGC standards and would like to see these improved and
further adopted in our organisation where appropriate.
Challenges
• List of Challenges
– In general, the standards don't always fit our
needs. I don't think there's and easy solution as
making the standards more general isn't
necessarily useful either. The OGC standards are
also sometimes complicated and not consistently
implemented, which reduces interoperability.
ASA
• Presenter Name : Eoin Howlett
• 1 sentence about the organization:
Private consulting company specializing in data
management solutions for the ocean,
atmospheric and GIS community
• Are OGC Web Services a standard technology
in your organization ?
Yes
OGC Relation
• Are you a member (yes or no)
Yes
• If so, which forums do you participate?
Actively participated in NetCDF standards,
some participation in a metocean IE.
OGC Standards in your organization
• OGC standards being used
– WMS, WCS, WFS, SWE, KML, NetCDF
• Planned OGC standards to be used in the next
three years:
– WCS, WPS, WCPS, WMTS
– OGC Web Map Service – Proposed Animation
Service Extension
Non OGC standards
• Which non OGC standards are you using and
why ?
Custom JSON for responses of WMS
GetFeatureInfo, Custom NetCDF conventions,
Custom cataloging solutions, ESRI products
(Geodatabases, shapefiles),
Postgres/PostGIS, SpatialLite, and Oracle
spatial databases, ERDDAP no-standard
response formats.
Challenges
• Complexity and non-homogenous
specifications
• Client tools to support OGC services
• Performance (practicality of some standards)
• Import into ESRI products a hassle. We end
up coming up with a custom solution.
• Lack of testing suite to test developed OGC
services
WMS (with GetFeatureInfo)
WMS and SOS
NetCDF and WMS
OBIS
• Presenter Name : Edward Vanden Berghe
(Mark Fornwall)
• An on-line, open-access, globally-distributed
network of systematic, ecological, and
environmental information systems
• Are OGC Web Services a standard technology
in your organization ? Yes
OGC Relation
• Are you a member (yes or no) - no
• If so, which forums do you participate?
OGC Standards in your organization
• OBIS web site (www.obis.org) is built around
GeoServer, so has OGC services built-in.
• It provides data via WFS and WMS, and KML
Non OGC standards
• Which non OGC standards are you using and
why ?
– GCMD for metadata (also FGDC within OBIS USA);
– DiGIR for data exchange protocol; Darwin Core for
exchange format.
Challenges
• getCapabilities: We haven’t allowed direct
public access to the OGC capabilities of our
GeoServer yet, as this would 1/ make it
difficult for us to monitor use, and 2/ we're
afraid that users are going to swamp our
servers with data requests that are to broad;
so we wanted to build a layer between OGC
and the user, trying to log activity, and stop
overly broad requests.
Challenges
• Too many layers: One of the problems we
were facing while building the web site is that
we have too many layers - each taxon is a
layer, and there are several 1000s of those; so
we can't predefine them, we have to be able
to query the database on the fly. We worked
with Chris Holmes and his people at The Open
Plan Project to extend the functionality of
GeoServer. It has worked very well.
Earth Data Analysis Center
• Presenter Name : Karl Benedict
(kbene@edac.unm.edu)
• EDAC is an applied geospatial technologies
center affiliated with the University of New
Mexico
• Are OGC Web Services a standard technology
in your organization ? Yes
OGC Relation
• EDAC is a University member of OGC
• Thus far we have primarily lurked in the
various forums looking for indications of
where the standards are heading to facilitate
our project planning.
OGC Standards in your organization
• OGC standards being used
– WMS/Time-enabled WMS, WFS, WCS, CSW, KML
– Purpose
• Geospatial Clearinghouse (http://rgis.unm.edu)
• Core application infrastructure: data access & exchange, web mapping
• Content delivery to non-geo specialists
• Pros & Cons
– Pros: broad support in a wide variety of client platforms and
programming frameworks/libraries
– Cons: server and client implementations remain uneven in their
capabilities, stability, and performance.
• Planned OGC standards to be used in the next three years:
– WPS
Non OGC standards
• Which non OGC standards are you using and why ?
– OPeNDAP – as an alternative access model for data
published through the THREDDS server platform
– OAI-PMH – metadata harvesting with other systems for
repurposing of metadata also published via CSW
– X39.50 – additional catalog service protocol provided by
the GeoNetwork platform being used for catalog services
– HTTP (REST) – general web services “standard” for
functionality not directly supported by OGC and other
standards
– ISO19115 (and related standards: 19115-2, 19139) –
metadata content model and XML encoding
Challenges
• Slow uptake & understanding by end-user
community => services as infrastructure that endusers don’t necessarily know is there. Need for
tools and wrappers for services as the elements
that clients interact with.
• Incomplete/unstable implementation by some
data providers (e.g. incorrect representation of
time parameters in time-enabled WMS)
• Need for expanded parameterization within KML
for more efficient time support (i.e. adopt ISO
model used in time-enabled WMS)
WMS / Time-enabled WMS
• Dust Forecast Products
• Publication of hourly dust forecast products for
the greater Southwest region of the US in a
variety of formats
– Animated GIFs for direct linking in web pages
– Custom web mapping clients
– KML files for visualization in Google Earth
• Issues
– Limited direct client support for time-enabled WMS
requires developing custom conversion tools and
application interfaces
WCS
• Data exchange for distributed environmental
modeling
• Publication of meteorological forecast
products and low-resolution dust forecasts for
use in setting boundary conditions and
parameterizing high-resolution dust models
• Issues
– No significant issues. As this is an automated
system, client support was not a large issue.
WMS => KML
• Automated re-packaging data published as WMS into KML
for streamlined access and delivery to non-geospatial users
• Developed scripted converters from WMS capabilities XML
to KML wrapper for WMS map images
– XSLT for WMS -> KML
– Python script for time-enabled WMS - > KML
• Issues
– Temporal dimension in capabilities XML files published by some
services do not comply with OGC model
– XSLT method of conversion of time-enabled WMS -> KML
exposed some recursion limitations in traditional XML
processing tools
US Geological Survey
• Presenter Name : Rich Signell
• USGS Mission: “Science for a Changing World”
• Are OGC Web Services a standard technology
in your organization ? Yes.
OGC Relation
• The USGS is an OGC Member
• Architecture DWG (Arch DWG) Lead, Doug
Nebert, US Geological Survey (USGS)
• Catalog DWG (Cat DWG) Lead, Doug Nebert, US
Geological Survey (USGS)
• cf-netcdf-1.0.swg: Rich Signell
• Hydro Domain Working Group: Nate Booth,
Roland Viger
• Sponsors OGC Interoperability Program initiatives
OGC Standards in your organization
• OGC standards being used
– WMS , WCS, SOS, WPS, NetCDF encoding
• Planned OGC standards to be used in the next
three years:
– CF
Non OGC standards
• Which non OGC standards are you using and
why ? OpenDAP + CF Conventions , because
OGC standards cannot yet represent 4D model
output in a standard way.
• ESRI REST services
• OBIS
Challenges
• WMS: tiling standard: is it standard yet?
• WCS: not many clients, not sure how it’s
progressing. Can we progress to non-uniform
data?
• WPS: would love to have a hands-onworkshop, perhaps led by USGS CIDA (Nate
Booth, Dave Blodgett, Tom Kunicki)?
OGC WMS
• We use the Godiva2 web client to access
extended WMS (ncWMS) all the time as a
quick browse of model (and other gridded
data) results
OGC WPS, WCS and OPenDAP/CF
• USGS GeoData Portal (GDP):
http://cida.usgs.gov/climate/gdp/
• This is a very cool interactive climate app build
by USGS CIDA folks, using WPS, WCS,
OPeNDAP/CF, THREDDS
Florida Fish and Wildlife Conservation
Commission
• Presenter Name : Kathleen OKeife
• 1 sentence about the organization: Florida
Fish and Wildlife Conservation Commission, a
state agency.
• Are OGC Web Services a standard technology
in your organization ? Yes, we use several of
the standards.
OGC Relation
• Are you a member (yes or no) No.
• If so, which forums do you participate?
OGC Standards in your organization
• OGC standards being used
– WMS, WFS, KML, XML, REST
• Planned OGC standards to be used in the next
three years:
– Perhaps we will add GML
Non OGC standards
• Which non OGC standards are you using and
why ?
Do not know. Cannot answer for the whole
agency.
Challenges
• List of Challenges
From a GIS perspective, the standards are an
assistance.
Martha’s Vineyard Coastal Observatory
(Woods Hole Oceanographic Institution - WHOI)
• Presenter Name : Janet Fredericks
• MVCO provides real-time observations in support
of sponsored research. Data are freely and openly
available to the public. WHOI is a non-profit
research organization primarily funded by
NSF/ONR/NOAA/NASA and other federal funding,
as well as private grants.
• OGC Web Services are a standard technology in
our organization.
OGC Relation
• WHOI has recently become an associate
member (NPO)
• If so, which forums do you participate?
OGC Standards in your organization
• OGC standards being used (for example)
– MVCO – Sensor Web Enablement
– BCO-DMO - WMS
– Upper-Ocean Processes Group –
NetCDF/THREDDS
• Planned OGC standards to be used in the next
three years:
Non OGC standards
• Which non OGC standards are you using and
why ? W3C – OWS/RDF
For example: The SensorML description files
reference OWS ontologies (registered terms).
This enables data aggregation centers to map
our definitions to others (for example my
spiketest to your outlier or my qcflag-good
(value=0) to your good-flag (value=3)
Challenges
• List of Challenges – time-consuming and tedious
metadata
Most standards implementations are geared towards
discovery and transport of observations. Implementation
of data quality assessment capabilities tends to make
descriptions complex, creating a hurtle that no one seems
willing to take – we need to develop tools that automate
the generation of metadata in a standards-based (OGC)
framework and enable the discovery and assessment of
meta-data, along with observational data delivery. Why
aren’t we grasping this unprecedented opportunity to
provide a solid foundation in our evolving earth observing
systems?
Sensor Web Enablement (1.0)
SensorML for DescribeSensor
O&M for GetObservation
• Purpose: enable dynamic quality assessment of real-time
oceanographic data
• Noteworthy: http://q2o.whoi.edu/node/129
model for the description of how an observation came to be from
observed properties including: capabilities, characteristics and
provenance of sensor(s), deployment (s) and processing (process
descriptions and qc tests and results).
Next steps:
• Need to infuse/harmonize ISO 11915-2 and update to SWE 2.0
• what we call content-rich – other’s call complicated! Cultural
hurdle!
• Need to get someone (NDBC?) to ingest and utilize the capability
(The capability needs to be exercised.)
NOAA / IOOS
• Presenter Name : Derrick Snowden
• 1 sentence about the organization:
Interagency effort to develop a system of
systems for end to end management of ocean
information addressing data collection
through product delivery.
• Are OGC Web Services a standard technology
in your organization ? Emerging standard and
usage is increasing but not yet operational.
OGC Relation
• Are you a member (yes or no) Yes (NOAA is
the member)
• If so, which forums do you participate?
– SWE WG
– SensorML WG
OGC Standards in your organization
• OGC standards being used
– SOS primarily, also WMS/WCS
– SWE/O&M
– CF-netCDF, netCDF-4 Classic
• Planned OGC standards to be used in the next
three years:
– CS/W,
Non OGC standards
• Which non OGC standards are you using and
why?
– We use technologies based upon NetCDF but
which themselves are not OGC standards
– Global Telecommunications System, operational
mandate.
Challenges
• List of Challenges
– New and unproven technologies require extensive
experimentation and concept testing
– Lack of deployed systems to judge
– Complex and interwoven standards are often difficult
to interpret and understand, again, more pragmatic
experience and documentation. OGC Standards,
particularly those written in the new style, are not
easy to read and understand the big picture. How
are these useful to me today?
– Lack of clients, particularly those that allow OGC web
services to be used from within common tools.
OGC Standard (and versions)
• SOS 1.0 (GML, O&M 1.0, SensorML1.0.1, moving toward SWE
2.0, )
• Purpose (data type): Web Service providing access to, primarily, in
situ ocean data (moored buoys, gliders, shore stations, vertical
profile or sounding data)
• Noteworthy (cool things you have done with it)
• Issues: Many, primarily listed on challenges page. Difficulty in
implementation also inhibits testing for some of the other issues
that will likely pop up in the future.
– What are the throughput limitations (kbps) and are any inherent in the
service or completely related to implementation?
– How can we encourage interaction with instrument providers?
SensorML direct from the sensor or manufacturer etc?
– Need for lightweight profiles, but not too light.
Maryland Department of Natural
Resources
Maryland Department of Natural
Resources
• Presenter Name : Mark Trice
• 1 sentence about the organization:
– The Department of Natural Resources leads
Maryland in securing a sustainable future for our
environment, society, and economy by
preserving, protecting, restoring, and enhancing
the State’s natural resources.
• Are OGC Web Services a standard technology
in your organization ? NO
OGC Relation
• Are you a member (yes or no) -NO
• If so, which forums do you participate?
OGC Standards in your organization
• OGC standards being used
– None
• Planned OGC standards to be used in the next
three years:
– Maybe .. By making our data available through
the MARACOOS data assets map/portal
Non OGC standards
• We collect nearly 10M records of tidal water
quality data each year. Much of this data is a
deliverable to the EPA Chesapeake Bay
Program and the submission format is via a
web upload of monthly data sets in a EPA
format that dates back to the 1980’s. Some of
our newer data via automated data sondes is
made available through our website
www.eyesonthebay.net, but currently is not
available via services.
NOAA/NODC
• Presenter Name : Krisa M. Arzayus
• 1 sentence about the organization: The
Nation’s archive for oceanographic data
• Are OGC Web Services a standard technology
in your organization ?
Yes. Most of the software for data access that
are standard part of NODC operations like
THREDSS, Hyrax and now Geoportal
incorporate OGC web service standards.
OGC Relation
• Are you a member (yes or no)
• If so, which forums do you participate?
Yes, NOAA is a member of OGC (see
http://www.opengeospatial.org/ogc/members)
NODC follows most of the forums.
OGC Standards in your organization
• OGC standards being used
– WMS, WCS, CSW, KML and also Filter
Encoding(this is really part of other standards)
• Planned OGC standards to be used in the next
three years:
– NetCDF-CF and Sensor Observation Service (SOS)
? Also, DAP if it becomes OGC standard.
Non OGC standards
• Which non OGC standards are you using and why ?
Probably the most important standards that we use are http and ftp since I guess
users understand these the best. Most of the web service standards are built
on top of these standards. Example, OpenDAP. We use OpenDAP, since it is
designed for accessing scientific data – grids & sequences, which is what we
have.
As far as data format standards, we use NetCDF/HDF which are sort of becoming
de-facto standards for complex data. But NODC does not impose any
restriction on the originator format and so we probably have data in too many
standard formats. We don’t necessarily “use” the standards.
All the code lists and digital standards for Oceanographic data developed by
oceanographic community. I don’t know all of them.
We use the Open Archival Information System standard (OAIS) to model our
archive on.
Last, but not the least, FGDC and ISO standards for metadata representation.
Challenges
• List of Challenges
Rapid evolution of some of the OGC standards poses a
problem for adoption.
Standards for observational data is somewhat lacking
with OGC, at least until recently. OGC standards for
vector data are too complex for observational data
which are generally just point data.
OGC Standard (and versions)
• Purpose (data type)
• Noteworthy (cool things you have done with
it)
1) NODC Geoportal
2) Environmental Response Management Application Gulf
Response (to Deepwater Horizon)- it is a web-based GIS tool to
assist emergency responders and resource managers.
http://gomex.erma.noaa.gov/
• Issues
BOEING
• Presenter Name : Steve Uczekaj and Rick Blair
• 1 sentence about the organization: Boeing
• Are OGC Web Services a standard technology
in your organization ?
– YES
OGC Relation
• Are you a member (yes or no)
– Yes
• If so, which forums do you participate?
OGC Standards in your organization
• OGC standards being used
– SOS
– WMS
• Planned OGC standards to be used in the next
three years:
– WMTS ..
Non OGC standards
• Which non OGC standards are you using and
why ?
– ERDDAP
• Efficiency for Large Data Sets
• Data Aggregation from different sources
– Thredds
• Handles Large Data Sets
• Streaming protocol
• Wide spread use
Challenges
• List of Challenges
– Data Encoding Standards – Biggest one from My
perspective. No Standard way to encode the data
from SOS
– Need a way to define the interface to SOS service.
•
•
•
•
What Data Encoding
What semantics for specific operations
What semantics fo “standard” data fields
Etc.
– Need a ‘Streaming” type of data service suitable for
large data sets such as history, profiles, and
transects.
Southern California Coastal Ocean
Observing System (SCCOOS)
• Presenter Name : Julie Thomas (SCRIPPS)
• SCCOOS brings together coastal observations in
the Southern California Bight to provide
information necessary to address issues in
climate change, ecosystem preservation and
management, coastal water quality, maritime
operations, coastal hazards and national
security.
• Are OGC Web Services a standard technology in
your organization ? Depends on the data
OGC Relation
• Are you a member (yes or no) – YES? Via
UCSD
• If so, which forums do you participate?
– UCSD participate in various forums
OGC Standards in your organization
• OGC standards being used
– Those supported by THREDDS and NetCDF to
serve HFR as one of the DACs
Non OGC standards
• Which non OGC standards are you using and
why ?
– For Wave Data we used an NDBC XML file via
FTP.
Challenges
• No standard for NetCDF wave spectra.
ERDAP, NODC and CDIP have their own
formats.
NERACOOS
• Presenter Name : Eric Bridger
• Northeast regional IOOS component.
• Are OGC Web Services a standard technology
in your organization ? Yes. As part of the
IOOS and preceding IOOS we have been using
OGC standards for quite a few years.
OGC Relation
• Are you a member: Yes. Via Gulf of Maine
Research Institute.
• If so, which forums do you participate? SOS,
SWE, OWS, WMS
OGC Standards in your organization
• OGC standards being used
–
–
–
–
–
–
WMS
SOS
SWE 2.0
WFS
WCS
KML
• Planned OGC standards to be used in the next
three years:
– WMTS
– SOS 2.0
Non OGC standards
• Which non OGC standards are you using and
why ?
• THREDDS
• NetCDF
• EPA Exchange Network
Challenges
• List of Challenges
– Evolving changes to standards, keeping abreast
– Community agreements on which to utilize
– Lack of tools for non-technical implementors
OBIS-USA
• Presenter Name : OBIS-USA
• 1 sentence about the organization: OBIS-USA
aggregates marine biogeographic data (presence-absenceabundance and tracking) from US gov, university, museum and
non gov sources, and serves this data interoperably in
standard formats for applications.
• Are OGC Web Services a standard technology
in your organization ? Yes
OGC Relation
• Are you a member (yes or no) No
• If so, which forums do you participate?
OGC Standards in your organization
• OGC standards being used
– WMS (GeoServer-based)
• Planned OGC standards to be used in the next
three years:
– WFS
Non OGC standards
• Which non OGC standards are you using and
why ?
– FGDC/ISO
– GCMD
– Darwin Core
– ERDDAP (OpenDAP)
– DiGIR/Tapir
Challenges
• List of Challenges
– one challenge is simply staff time to implement and maintain
services
– a secondary challenge is server performance: some services
consume a lot of CPU to generate results; or consume a lot of
storage space for cache
• Less challenging:
– software cost is rarely a challenge because solutions are opensource
– given some configuration and management cost, source data
and OBIS-USA standards/processes make data compatible with
geospatial services such as WMS
WMS
• Purpose (data type) Serving Marine BioGeographic occurrence
data from many different datasets in a standard interoperable format
• Noteworthy (cool things you have done with
it) Providing data from 150 datasets to Federal data aggregators/portals
(Data.gov) without having to repeat 150 different configurations; a single
configuration in WMS was able to serve all OBIS-USA standard data
• Issues We are evaluating additional detail from OBIS-USA that we
would like to serve. We may need to implement WFS in addition to WMS;
we are uncertain until we do the detailed implementation if this will
satisfy all requirements.
Next Steps
• [LB] Summarize the challenges related to OGC
standards and action items related to those
challenges.
• [LB] Send it to the DMAC ST for review
• [LB] Follow up on action items.