The OPeNDAP/OGC Gateway
A NASA ACCESS Project integrated from two proposals:
-- The Development and Deployment of a CEOP Satellite
Data Server (Ken McDonald, GSFC)
-- Gateway for Interoperability of Atmosphere, Land, Ocean,
and Modeling Science Data (Liping Di, GMU)
Wenli Yang
Center for Spatial Information Science and Systems (CSISS)
School of Science
George Mason University
http://csiss.gmu.edu
OPeNDAP Developer’s Meeting, Feb. 21-23, 2007, Boulder, Colorado
Project Team
PI and Co-Is:
Ken McDonald (PI, NASA/GSFC), Yonsook Enloe (SGT
Inc.), Liping Di and Wenli Yang (GMU), Dan Holloway
(OPeNDAP), Ben Domenico (Unidata), Glenn Rutledge
(NOAA/NCDC).
Other members:
Chengfang Hu and Min Min (GMU), Sr. S/W Engr.(OPeNDAP)
Science advisors and user feedback:
Professor Toshio Koike (University of Tokyo), Dr. Mike
Bosilovich (NASA GSFC)
Project Goal
To address the interoperability of two data system
infrastructures widely used by different segments
of the Earth science research and applications
community, namely the Earth science community
which uses OPeNDAP and THREDDS protocols
and the geospatial community which uses OGC
protocols.
Specific Objectives
 To allow a user of a DAP client to discover and
access data provided through OGC servers.
 To leverage WCS rectification/reprojection and
interpolation operations with DAP access to
satellite products for the CEOP science
community.
 To allow a user of a OGC client to discover data
available through THREDDS servers.
CEOP Satellite Data Server
OPeNDAP/WCS Gateway Components
CEOP Satellite Data Server
OPeNDAP/WCS Gateway Components
 The original design was to develop the gateway components. The
gateway can then be installed with the OPeNDAP server to
link the server to a WCS server.
 With the development of server4, many of the components are
already included in the server. Thus, an independent gateway
is not needed.
 The CF-1.0 compliant netCDF format handler is embedded into the
WCS server. Server4 can always expect a valid CF-netCDF
from the WCS server.
 THREDDS catalog generator will be developed as a THREDDS
server at the front end and as a WCS 1.1 client, which sends
describeCoverage requests, at the back end.
 Currently, the catalog generator is implemented by making use of an
XML configuration file (the WCS server’s capabilities XML
file) without issuing requests to the WCS server.
OPeNDAP Server Implementation
Approach
WCS
DAP
OLFS
BES
Local cache
THREDDS
 OLFS interacts with local catalog to identify data source as WCS.
 OLFS instructs BES to set container type WCS; passes name, target, type to BES.
 BES sets container to WCS, uses name, target, type to interact with remote WCS.
 BES interns WCS response to local cache.
 BES uses handler (NetCDF, HDF, <type>) to process cached file to satisfy DAP request.
 Subsequent DAP requests operate against local cache until cache refresh signaled.
The Test Implementation
http://test.opendap.org:8080/opendap/data/wcs/Georectified-Grid/MYD11A2.A2004137.h10v05.004.2004147190109_EPSG.MODIS_Grid_8Day_1km_LST.nc.html
Corresponding WCS call
http://data.laits.gmu.edu/cgi-bin/ACCESS/wcs300?service=WCS&version=1.0.0&request=getCoverage&coverage=
/home/mmin/grid1/MYD11A2.A2004137.h10v05.004.2004147190109_EPSG.hdf:Grid:MODIS_Grid_8Day_1km_LST:LST_Day_1km&
crs=EPSG:4326&bbox=-100.8,38,-92.1,39.9&format=netCDF&width=300&height=200
The Test Implementation
http://test.opendap.org:8080/opendap/data/wcs/Georectified-Grid/MYD11A2.A2004137.h10v05.004.2004147190109_EPSG.MODIS_Grid_8Day_1km_LST.nc.html
DAS response
http://test.opendap.org:8080/opendap/data/wcs/contents.html
DDS response
WCS Coverage
f(x,y,z,t) = {T, RH, P,…}
Domain => Range
WCS Domain
WCS Range
DescribeCoverage Request
DescribeCoverage Response
GetCoverage Request Example
GetCoverage Request Example
GetCoverage Response Example
<?xml version="1.0" encoding="UTF-8"?>
<OperationResponse xmlns="http://www.opengis.net/ows/1.1"
xmlns:xlink="http://www.w3.org/1999/xlink"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.opengis.net/ows/1.1
../owsInputOutputData.xsd">
<ReferenceGroup>
<Abstract>Coverage created from GetCoverage operation
request to a WCS</Abstract>
<Reference xlink:href="coverage/image.tiff"
xlink:role="urn:ogc:def:role:WCS:1.1:coverage"/>
<Reference xlink:href="coverage/metadata.xml"
xlink:role="urn:ogc:def:role:WCS:1.1:metadata"/>
</ReferenceGroup>
</OperationResponse>
y
Last grid point
v
(x2,y2)
b
(u2,v2)
(a2,b2)
point Pin
P1=(deltU,0)
P2=(0,deltV)
(x1,y1)
(a1,b1)
a
p2
(x0,y0)
x
Figure 1 boundingBox specified
in CRS (a,b).
Figure 2 Source coverage and transformed
boundingBox in source coverage’s
baseCRS (x,y). Blue area shows
boundingBox being extended to the
nearest grid posts.
(u0,v0) p1
(u1,v1)
point Pout
First grid point
u
Figure 3 Output coverage and transformed boundingBox
and extended boundingBox in output coverage’s
baseCRS (u,v). Blue area shows the minimum
subset in source coverage.
The following steps describe one of (many?) approaches, called approach two, of a getCoverage request/response for a 2D grid, assuming
that the boundingBox CRS, source grid baseCRS, and output grid baseCRS are different:
1.
2.
3.
4.
5.
6.
A client specifies a boundingBox in CRS (a,b) and specifies the output grid’s origin (u0,v0) and offset p1 and p2 in the baseCRS of the
output grid (u,v) (this baseCRS is specified in wcs:GridCRS).
The boundingBox is transformed to the wcs:GridCRS of the source gird (green area in fig. 2) and the extent is extended to a nearest
minimum area (blue area in fig.2) that covers the boundingBox. The grid point values of the are subsetted.
The boundingBox is transformed to the wcs:GridCRS (u,v) of the output gird, whose grid point locations are determined by the origin
(u0,v0) and offset values p1 and p2 defined in the output part of the getCoverage request. The transformed boundingBox (green area in
fig. 3) is extended to a nearest grid points that completely include the transformed boundingBox. This output grid is shown in the grid
points constructed by the black lines. Note that this output grid may not necessary cover all areas of the minimum subset in the source
grid (the blue area in figure 2) as shown in figure 3, dependent on such factors as baseCRSs and offset values of source and output grid.
The minimum subset from the source grid (blue area in figure 2) is transformed to the output grid points.
In this method, some values in the minimum subset may not be used (e.g., point Pin in figure 3) while some output grid points may not
be available (e.g., point Pout in figure 3). Such issues can be avoid in approach one discussed in the previous chart.
In the output grid, the positions are defined at grid points, not grid cells. If the grid is interpreted as composed of grid cells, the grid
cells look like something as shown by the grid constructed by the dashed dark red lines. The position of the center of each such cell is
defined (or, cell position is defined at the cell center).
Last grid point
y
v
b
(a2,b2)
(a1,b1)
a
Figure 1 boundingBox specified
in CRS (a,b).
P1=(deltU,0)
P2=(0,deltV)
p2
(u0,v0) p1
First grid point
u
Figure 2 Output coverage extent (blue) and transformed
boundingBox (green) in output coverage’s
baseCRS (u,v). The output grid origin is at (u0,v0).
(x0,y0)
Figure 3 sourceCoverage, its baseCRS
(x,y) and origin (x0,y0). The
The green and blue areas are
correspondent to the
boundingBox and output grid.
The following steps describe one of (many?) approaches, called approach one, of a getCoverage request/response for a 2D grid, assuming
that the boundingBox CRS, source grid baseCRS, and output grid baseCRS are different:
1.
2.
3.
4.
x
A client specifies a boundingBox in CRS (a,b) and specifies the output grid’s origin (u0,v0) and offset p1 and p2 in the baseCRS of the
output grid (u,v) (this baseCRS is specified in wcs:GridCRS).
The boundingBox is transformed to the wcs:GridCRS of the output gird and the extent of the output grid is determined based on the
transformed boundingBox and the origin (u0,v0) and offsets p1 and p2, by extending the transformed boundingBox to the closest grid
points to completely include the boundingBox. The origin (u0,v0) and offsets p1 and p2 are defined in the output coverage’s
basedCRS, which is specified/included in the wcs:GridCRS. The transformed boundingBox is shown in green and the output grid
extent is shown in the blue area (grid points constructed by the black lines) in figure 2.
Values for the grid points in the output grid are derived by determining their positions in the source grid. In the source grid, the green
and blue areas show the areas of the boundingBox and the output grid if they would be transformed to the baseCRS of the source
coverage. These areas, however, usually need not to be transformed to the source coverage’s baseCRS. A server may chose to
transform the blue area so that only a minimum subset of the source grid needs to be read (note that the extent of the minimum subset
in the source grid is also dependent on interpolation method.).
In the output grid, the positions are defined at grid points, not grid cells. If the grid is interpreted as composed of grid cells, the grid
cells look like something as shown by the grid constructed by the dashed dark red lines. The position of the center of each such cell is
defined (or, cell position is defined at the cell center).
MODIS Data in Swath and Lat/Lon
Coordinates
ASTER Data in Swath and Lat/Lon
Coordinates
OGC Geoscience Gateway
CS/W protocol
WFS
Sub-gateway
Catalog Ingestor
WFS protocol
WCS
Semantic
Catalog
Sub-gateway
Data
Catalog
WCS protocol
Tools
Geoscience Protocols
THREDDS
Catalog
Geoscience
Servers
WCS-geoscience Gateway Prototype
in THREDDS
OGC CSW
The OGC Catalog Service for Web specifies the
interfaces, bindings, and a framework for defining
application profiles required to publish and access
digital catalogues for geospatial data and services.
OGC Catalog UML Model
Common Queryable Elements
OGC CSW Application Profiles
The ISO19115/19119 profile explains how catalogue
services based on the profile are organized and
implemented for the discovery and management of
geospatial data and service metadata which are compliant
with the ISO19115 and 19119 standards.
The ebRIM profile explains how services based on the
more general OASIS ebXML Registry Information
Model are organized and implemented.
Connecting THREDDS to CSW
The first of the following two approaches are adopted:
 Mapping THREDDS metadata to ISO metadata and
implementing a CSW server based on the ISO
profile.
 Implementing a CSW server for THREDDS metadata
based on ebRIM profile.
ISO19115 Metadata Information
MD_ReferenceSystem
<<Abstract>>
MD_SpatialRepresentation
(from Reference system information)
(from Spatial representation information)
+spatialRepresentationInfo
+referenceSystemInfo
MD_MetadataExtensionInformation
0..*
0..*
(from Metadata extension information)
+metadataExtensionInfo
0..*
DQ_DataQuality
(from Data quality information)
+dataQualityInfo
0..*
MD_MaintenanceInformation
+
+
+
MD_Distribution
+
(from Distribution information)
+
+distributionInfo+
+
0..1
+
+
+contentInfo
+
0..*
MD_Metadata
fileIdentifier [0..1] : CharacterString
language [0..1] : CharacterString
characterSet [0..1] : MD_CharacterSetCode = "utf8"
parentIdentifier [0..1] : CharacterString
hierarchyLevel [0..*] : MD_ScopeCode = "dataset"
hierarchyLevelName [0..*] : CharacterString
contact : CI_ResponsibleParty
dateStamp : Date
metadataStandardName [0..1] : CharacterString
metadataStandardVersion [0..1] : CharacterString
(from Maintenance information)
+metadataMaintenance
0..*
0..1
+resourceMaintenance
+identificationInfo
1..*
<<Abstract>>
MD_Identification
(from Identification information)
MD_ContentInformation
(from Content information)
+resourceConstraints
+metadataConstraints
0..*
0..*
MD_Constraints
+portrayalCatalogueInfo
(from Constraint information)
0..*
MD_PortrayalCatalogueReference
(from Portrayal catalogue information)
+applicationSchemaInfo
0..*
MD_ApplicationSchemaInformation
(from Application schema information)
Conditional statements:
language: documented if not defined by the encoding
standard
characterSet: documented if ISO 10646-1 not used
and not defined by the encoding standard
hierarchyLevel: documented if hierarchyLevel not
equal to "dataset"?
hierarchyLevelName: documented if hierarchyLevel
not equal to "dataset"?
MD_Metadata
MD_Usage
+ specificUsage : CharacterString
+ usageDateTime[0..1] : DateTime
+ userDeterminedLimitations[0..1] : CharacterString
+ userContactInfo [1..*] : CI_ResponsibleParty
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
(from Metadata entity set information)
MD_Constraints
(from Constraint inform ation)
+resourceConstraints
+resourceSpecificUsage
0..*
MD_BrowseGraphic
+ fileName : CharacterString
+ fileDescription[0..1] : CharacterString
+ fileType[0..1] : CharacterString
+graphicOverview
0..*
0..*
+identificationInfo
1..*
+
+
+
+
+
+
<<Abstract>>
MD_Identification
citation : CI_Citation
abstract : CharacterString
purpose [0..1] : CharacterString
credit [0..*] : CharacterString
status [0..*] : MD_ProgressCode
pointOfContact [0..*] : CI_ResponsibleParty
+descriptiveKeywords
0..*
+resourceFormat
MD_Format
0..*
MD_Keywords
+ keyword[1..*] : CharacterString
+ type [0..1] : MD_KeywordTypeCode
+ thesaurusName[0..1] : CI_Citation
(from Distribution inform ation)
<<CodeList>>
MD_KeywordTypeCode
+ discipline
+ place
+ stratum
+ temporal
+ theme
characterSet: documented if ISO
10646-1 is not used
{MD_Metadata.hierarchyLevelCode =
"dataset" implies count (geographicBox)
+ count (geographicDescription) >=1}
+resourceMaintenance
0..*
MD_MaintenanceInformation
(from Maintenance inform ation)
MD_ServiceIdentification=
+
+
+
+
+
+
+
+
+
+
MD_DataIdentification
spatialRepresentationType [0..*] : MD_SpatialRepresentationTypeCode
spatialResolution [0..*] : MD_Resolution
language [1..*] : CharacterString
characterSet [0..1] : MD_CharacterSetCode = "utf8"
topicCategory [1..*] : MD_TopicCategoryCode
geographicBox [0..*] : EX_GeographicBoundingBox
geographicDescription [0..*] : EX_GeographicDescription
environmentDescription [0..1] : CharacterString
extent [0..*] : EX_Extent
supplementalInformation [0..1] : CharacterString
<<CodeList>>
MD_TopicCategoryCode
farming
biota
boundaries
climatologyMeterologyAtmosphere
economy
elevation
environment
geoscientificInformation
health
imageryBaseMapsEarthCover
intelligenceMilitary
inlandWaters
location
oceans
planningCadastre
society
structure
transportation
utilitiesCommunications
<<CodeList>>
MD_CharacterSetCode
+ ucs2
+ ucs4
+ utf8
+ utf16
+ isoIec8859oneTo15
+ jis
+ shiftJIS
+ eucJP
<<CodeList>>
MD_ProgressCode
+ completed
+ historicalArchive
+ obsolete
+ onGoing
+ planned
+ required
+ underDevelopment
Scale
(from Units of Measure)
/Scale
<<CodeList>>
MD_SpatialRepresentationTypeCode
+ vector
+ grid
+ textTable
+ TIN
+ stereoModel
+ video
Where MD_Representative
Fraction.denominator =
1/Scale.measure And
Scale.targetUnits =
Scale.sourceUnits
<<DataType>>
MD_RepresentativeFraction
/+ denominator : Integer
<<Union>>
MD_Resolution
+ equivalentScale : MD_RepresentativeFraction
+ distance : Distance
Identification Information
THREDDS Catalog Information Model
THREDDS/CSW Mapping
The first step is to mapping the semantically equivalent metadata items
between the ISO19115 and the THREDDS information models.
THREDDS Catalog Ingestor
The ingestor is a THREDDS catalog server client at the front end. It
obtains information of data sets in the THREDDS catalog maps the
information to ISO metadata. At the back end, it writes to the CSW
database through JDBC. This will require that the ingestor have
write permission to the CSW database. It is also planned, if
resources are available, to implement the ingestor as a CSW client at
the back end. The client can register the THREDDS metadata to any
compliant CSW server through CSW protocol.
THREDDS Dataset Inventory Catalog
THREDDS Catalog Ingestor Tool Design
CSISS
Spatial
DB
THREDDS Data
Server(TDS)
Catalog.xml
THREDDS Dataset
Inventory Catalog
Parsing&
Mapping
DataGr
anule
BBOX
Organi
zation
…...
GMU CSW Search Interface
http://geobrain.laits.gmu.edu/csw/discovery/
GetRecord Request