Web Services Survey an Inventory
Background, Goals and Status
GCMD
OGC Cats
ECHO
OpenDAP Cat
ESIP Cat
Google
14th ESIP Federation Assembly Meeting
Renaissance Mayflower Hotel
Washington, DC
January 4-6, 2005
Presented by Rudolf Husar, Technology Infusion Workgroup, rhusar@me.wustl.edu
Background and Purpose
There is a rapidly growing array of distributed services for accessing, processing, visualizing,
cataloging, discovering or otherwise manipulating Earth Science information through
computer-computer interfaces.
The landscape is currently obscured by the lack of a suitable inventory and by the hype
associated with web services
Purposes of the WS Inventory/Survey
•
•
•
assess the size and characteristics of the computer-accessible (WS) resource pool
expose the main computer-computer WS linking protocols used and
get a rough assessment of the current WS usage environments.
Beneficiaries:
ESIP
Infusion DSWG workgroup
other DSWG workgroups, (e.g. Reuse and Standards)
It should also aid the creation and diffusion of ES knowledge for Achieving a Sustainable Planet (ESIP)
WS Inventory: Features and Approach
WS Inventory Survey Features
• Flexible to capture the varied WS technologies and their use environments
• Simple for easy participation
Approach to the WS Inventory Content Creation
•
•
•
The content of the WS Inventory collected from the community through a web-form.
Subsequent versions of this inventory could incorporate more specifics for service binding
The content could migrate to formal WS Discovery services such as GCMD, ECHO and
others.
Current task: Designing the web-form to be filled out by the participating members
Latest draft Web Service Inventory page on the Infusion WG website
http://www.sciencedatasystems.org/seeds/wg/infusion/Lists/Service%20Inventory/AllItems.aspx
Discussion thread for web-form design WS Invedntory Form Design by the WG
http://www.sciencedatasystems.org/seeds/wg/infusion/Lists/Web%20Services%20Discussion/AllItems.aspx
Select the node in the thread
Press Post Reply (remember to log in, so we know who you are).
WS Inventory as an Open, Living Resource?
•
Relationship to other Catalogs
–
–
(Social behavior)
Not to be rude (either you or I)
Not just coexisting (either you or I)
– Cooperate, be part of a value-chain
GCMD
OGC Cats
ECHO
OpenDAP Cat
ESIP Cat
Google
Cooperation of service catalog ‘vendors’ is achievable by
Opening the contents of one’s catalog to access by value-adding partners
Web services offer an simple, safe and agile technology for creating value chains
Catalog Linking Demo
A web service chaining technology demonstration
A template for linking GCMD, ECHO and other catalog chains?
Technology Infusion Website
Test client CATALOG, WashU
http://www.sciencedatasystems.org/seeds/wg/infusion/Lists/Service%20Inventory/AllItems.aspx
http://datafed2.seas.wustl.edu/dvoy_services/sds.aspx
Catalog data are collected through a form on SDS SharePoint server
Catalog content is maintained in a SharePoint database
SharePoint exposes data through SOAP web services
Client sends SOAP request to SherPoint server
Clients receives SOAP-wrapped catalog content
The XML data looks like this (SOAP envelope stripped away)
Client adds value, e.g. filter/transform/render add more metadata
SOAP
web services
WS Catalog/Survey Discussion Thread
Scope of Web Services Subgroup, 2005
WG Purpose: Find and demonstrate ways to infuse WS Technologies
1. Prepare an inventory of web services and their interfaces
–
–
–
•
New developments:
–
–
–
–
–
–
–
•
The WS landscape is changing rapidly. New services are emerging
We need a snapshot of the service landscape, particularly in the REASoN projects
But what ‘services’ do we include? SOAP, OPeNDAP, OGC, files?
We have prepared a Ws service survey form at the WG website, populated with test survey entries
At ESIP meeting, we presented the the idea of the WS survey to the combined WGs, Rob Raskin’s Data Discovery and Brian Wilson’s Web
services.
The response was not too favorable: do we need yet another catalog/survay? Whats wrong with GCMD, ECHO and other catalog/dicovery
efforts
In the ESIP working sessions we have explored the possibility of using the GCMD SERF registration for general service description and ECHO
as a formal WS registry.
Both GCMD and ECHO representatives recognized that lining the two catalog efforts was meaningful and some changes will be necessary in
each catalog to formalize the link
Fat is that GCMS has very few formal SOAP web services registered
Brian Wilson has volunteered to be the ESIP rep. for this effort
New ideas for WS status assessment
–
So, we still need some other low resistance approaches to gather the list of formal web services
•
–
–
–
•
•
•
At the ESIP discussion Jim Frew has suggested tha we send out an emal to the SEEDs community simply requesting URLs of the web description
documents, the WSDL from any one who has a service.
Brian Wilson suggested to do a Google search on WSDL..I did that an viola, there were at least a dozen Earth Science WSDLs
I am in favor of tese light-handed approaches to explore the currently functional WS landscape
•
•
Wilson - more documentation for the WSDL –
Next WG meeting Michael Burnett – UDDI -
•
Demonstrate web service linking
Several REASoNs have a goal of facilitating satellite data flow and processing
A narrow pilot project could demonstrate satellite data access/processing/analysis/delivery
Issues: Which REASoN projects? Data products? Services? Architecture(s)?
New developments:
–
–
At ESIP, Brian Wilsons WS WG cluster has set itself a goal to organize WS chaining demo projects
Our WS can actively participate in that prototyping exploration
1. Assess WS linking architectures for data access AND analysis
–
–
–
We don't quite know how to link distributed WSs into coherent science applications
An open dialog on the linking/architectural issues would speed up the group learning
Issues: e.g. Architecture without rigid architecture
•
James, I did want to get the SOAP stuff over early in my track, and then move on to Grid stuff. But if this doesn't fit in with your
plans, we can start with Grid in my track, then do the joint SOAP talk, and then return to Grid.
•
•
•
•
•
•
•
Tuesday
1:30-2 PM -- SOAP Protocol, Brian Wilson (joint with Data Protocol track)
2-2:30 PM -- Structuring Services with SOAP as Universal Glue, Brian Wilson (could be joint with Data Discovery, but not crucial)
2:30 - 3 PM -- Data Grid using SRB and Globus (Mike Smorul, Gary Jackson)
3-3:30 PM -- Break
3:30-5 PM -- More Grid Services stuff by Mike and Gary
5-5:30 PM -- Richard Troy on Grid Technologies
•
•
•
•
•
•
Note that I would still like to put Troy's talk on the first day, even though we have a half-hour less time than I thought. Is it
okay to go until 5:30
Wednesday
8:30-9 AM -- Orientation Talk, Goals of Track, Brian Wilson
9-9:30 AM -- Gathering an Inventory of Fed. Services, Rudy Husar
1st Discussion Period -- collaborating on services, service interop, experiences with using SOAP, etc.
Perhaps have a joint discussion with the Data Protocol track about inter-operability. James, should we try to agree on a time?
•
•
•
•
•
Grid Discussions (following above)
Lead off with half-hour talk by Paul Davis (or someone else?) -Capabilities Matrix for SRB & Globus,
How does one join the Federation Data Grid?
Ensuing discussion
•
•
Rest of Wed. & Thursday morning
Formulate recommendations for both Web & Grid Services use within the Federation.
•
I'm still working on filling in a more detailed but tentative schedule for Wed. and Thu. I'm going to try to come prepared with
some partially filled out comparison tables such as a Capabilities Matrix for SRB and Globus (single sign-on, data replication, job
submission, bulk data transfer, etc.) I will talk to Paul and company about this.
•
I suspect that we will have more *informal* presentations on Wed. to structure the discussions. Please feel free to suggest
particular discussions you would like to see happen or like to lead. If you want to make further presentations on Wed. to
provoke & facilitate discussion, let me know.
•
•
Every function can be looked at as a "service“
The key protocol(s) for such distributed services are SOAP and REST (one-line URL's).
•
If every data provider put up a SOAP service that took startTime, endTime, lat/lon box and
physicalVariableName(s) as inputs, and returned a list of unique object ID's (i.e., granule file ID's), then we
would almost be done. Then data discovery down to the inventory level would be reduced to "discovering" a list
or catalog of all of the relevant SOAP services for a particular scientific domain. A catalog that we are starting
to assemble via the survey form.
The SOAP service could also provide a translation table from generic variableNames (atmosphericTemperature)
to dataset-specific names, to make the discovery more generic.
•
•
I would like to inject these ideas into the Data Discovery track. Perhaps I can say a few words during the joint
half-hour, before or after Rudy's talk.
•
I could also be convinced that the effort to catalog the available services via a survey is not, per se, that
relevant to the Data Discovery topic.Perhaps Rudy will have to add a part to his talk that responds to the Data
Discovery problem. Alternatively or additionally, we could have a joint discussion period later in the day.
•
Rob, I'm already preparing too many talks so I'm not signing up for another joint talk. My joint SOAP protocol
talk will address how Data Discovery can be solved by using SOAP/REST services. If folks miss that talk, then I
can inject the idea in later discussions.