Resource wrappers, web
services, grid services
Jaspreet Singh
School of Computer Science
Cardiff University
BiodiversityWorld GRID Workshop
NeSC, Edinburgh – 30 June and 1 July 2005
Overview
•
•
•
•
•
•
•
•
BDWorld Project
Resource Wrappers
BDWorld Datatypes
BiodiversityWorld-GRID Interface (BGI)
Communications Layer
Web Services based BDW Wrappers
Globus Toolkit 4.0 & WSRF
Grid Services based BDW Wrappers
Further Work
BiodiversityWorld GRID Workshop
NeSC, Edinburgh – 30 June and 1 July 2005
BDWorld Project
• A flexible and extensible problem solving environment (PSE)
• Brings together heterogeneous, globally distributed, biodiversity related
resources and analytical tools
• The resources can be assembled into set of workflows to perform
complex scientific analysis
• Provides consistent mechanisms across various facets to achieve
interoperability
• The multi-layered architecture provides data related computation as part
of a separate layer
• Provides uniform interfaces for heterogeneous resources
• Provides a common resource access mechanism
• Standard data flow mechanisms that use well defined data objects
• Uses Triana Workflow System for assembling and executing workflows
BiodiversityWorld GRID Workshop
NeSC, Edinburgh – 30 June and 1 July 2005
BDWorld Wrappers
• A mechanism to provide consistent interface to resource users and a
standard resource access/invocation mechanism
• Insulate the core BDWorld System from heterogeneous resource
• Retains flexibility to use various operations supported by each resource
• Provides a generic access mechanism
• Wraps various kinds of resources and analytical tools
• Can be deployed in Web Services/Grid environment
• Provides interoperability mechanisms between client and heterogeneous
resources
• Operations on remote resources is invoked via the invokeOperation()
method implemented by all the wrappers
• Wrappers give consistent form to data retrieved from heterogeneous
resources by encapsulating them into BDWorld data types
BiodiversityWorld GRID Workshop
NeSC, Edinburgh – 30 June and 1 July 2005
InvokeOperation() method
• The parameters of invokeOperation()method
identify:
theResourceHandle: name of the resource to be
invoked
theOperationHandle: name of the operation to be
invoked on the remote resource/tool
theDataCollection is input data collection for the remote
resource
BiodiversityWorld GRID Workshop
NeSC, Edinburgh – 30 June and 1 July 2005
Triana Workflow System
BGI Communications Layer
Abstract Wrapper
Datatype Tools
Standard Operations
Configuration Tools
Data Transformation
Data Transformation Tools
Wrapper Logic
External Tools
BDW Wrapper
Remote Resource
BDW Wrapper Architecture
BiodiversityWorld GRID Workshop
NeSC, Edinburgh – 30 June and 1 July 2005
BDWorld Datatypes
• Encapsulate different types of data and sub-datatypes for
transporting data between end points
• Datatypes can be transformed into xml representations which
can be easily serialised
• Xml schema for datatypes conform to W3C standards
• Datatypes are flexible enough to encapsulate user defined
xml documents or data in string representation
• They also act as parsing tool to extract data from raw xml
• Higher level datatypes can hold several sub datatypes and
each sub datatype can hold several pieces of data
• They are extensible and support incorporation of new
datatypes.
BiodiversityWorld GRID Workshop
NeSC, Edinburgh – 30 June and 1 July 2005
BDWorld Datatypes
BiodiversityWorld GRID Workshop
NeSC, Edinburgh – 30 June and 1 July 2005
BDWorld-GRID Interface (BGI) Layer
• Provides standard mechanisms for invoking operations on heterogeneous
resources
• provides seamless access to BDW resources via resource wrappers
• Uses Xml/SOAP messaging system for invoking operations on resource
wrappers
• Act as an integrated mechanism for accessing all resource wrappers
• Isolates resources/resource wrapper implementation to enable use of web
services/grid technologies as part of a separate layer
• Isolates users from grid/web service complexities
• Helper Tool is provided to the user/triana for using the BGI layer
• Can provide multiple users/workflows with access to multiple resources at
a time
• Interoperable with other e-Science projects
BiodiversityWorld GRID Workshop
NeSC, Edinburgh – 30 June and 1 July 2005
BDW Web Services Architecture
• Web services is a mechanism of enabling distribute
computing based on open standards
• Wrappers are currently deployed in a web service
environment which can be accessed via the BGI
Layer at servers in Cardiff and Reading
• Web services enabled wrappers can be invoked via
the BGI Helper Tool
• Axis SOAP engine provides the WSDL that exposes
wrapper operations to outside world
• The MetadataAgent WS will provide access to MDR
via the BGI Layer
BiodiversityWorld GRID Workshop
NeSC, Edinburgh – 30 June and 1 July 2005
Triana Workflow Units
Unit A
Unit B
Unit C
MDR
Data Parsing Tools
BGI Helper Tool
Metadata Agent WS
BGI Communications Layer
WS Interface
WS Interface
WS Interface
BDW Wrapper
BDW Wrapper
BDW Wrapper
External Tool
Remote Resource
BDW Web Services Architecture
BiodiversityWorld GRID Workshop
NeSC, Edinburgh – 30 June and 1 July 2005
Java /non
java resource
Tomcat Servlet Container/
Axis Soap Engine
Drawbacks of web services
• Web services are not stateful
• WS provide mechanisms for invoking remote
operations but does not provide other
services such as resource management,
persistency, life cycle management,
notification etc.
• Each web service needs to be deployed
individually
BiodiversityWorld GRID Workshop
NeSC, Edinburgh – 30 June and 1 July 2005
GT4 Key Concepts
• Based on Open Grid Service Architecture (OGSA)
• OGSA defines common, standard and open
architecture for grid-based applications
• Standardises various services common to grid
applications (job management, resource monitoring
and discovery, resource management, security
services etc)
• Uses Web Services as underlying technology to
enable distributed computing
• But Web Services are not stateful
BiodiversityWorld GRID Workshop
NeSC, Edinburgh – 30 June and 1 July 2005
Web Service Resource and Web
Services Resource Framework (WSRF)
• WS-Resource construct expresses relationship between stateful
resources and Web Services. It is pairing of web service with a resource
• WSRF is a set of web services specifications which define rendering WSResource approach in terms of specific message exchanges and related
XML definitions
• It specifies how Web Services can be made stateful
• It defines conventions for managing state for applications to discover,
inspect and interact with stateful resources in standard and interoperable
way
• Using WSRF associations between a web service and one or more
stateful resources can be established
• The state of the WS-Resource can be accessed through a web service
interface by using endpoint references
• WSRF is the core of GT4
BiodiversityWorld GRID Workshop
NeSC, Edinburgh – 30 June and 1 July 2005
WSRF Specifications
• WS-ResourceProperties
• WS-ResourceLifetime
• WS-RenewableReferences
• WS-ServiceGroup
• WS-BaseFaults
Related Specifications
• WS-Notification
• WS-Addressing
BiodiversityWorld GRID Workshop
NeSC, Edinburgh – 30 June and 1 July 2005
OGSA
WSRF
Applications
requires
OGSA
WSRF
Web Services
specifies
Stateful
Web Services
extends
Web Service
Relation between OGSA, WSRF and GT4 (source: [1])
BiodiversityWorld GRID Workshop
NeSC, Edinburgh – 30 June and 1 July 2005
WSRF – An approach to
statefulness
• WSRF provides the mechanism to keep state
information by keeping the Web Service and state
information completely separate
• State information is stored in entity called resource
• A resource is be identified via its unique key
• When requiring stateful interaction, a web service
can be instructed to use a particular resource
• The resources can be stored in memory or on
secondary storage
BiodiversityWorld GRID Workshop
NeSC, Edinburgh – 30 June and 1 July 2005
WS-Resource
Resources
Instruct client to use Resource A
A
request
Client
Web Service
B
response
C
Client provides EPR corresponding
to Resource A
Web service loads resource A
Interaction of client with a stateful web service (source: [1])
BiodiversityWorld GRID Workshop
NeSC, Edinburgh – 30 June and 1 July 2005
Grid enabled BDW Resource
Wrappers
• Java WS Core component of GT4 being used
• Prototype implementation of resource wrappers deployed in
the grid environment
• Interaction of Grid enabled wrappers via the BGI Helper Tool
• Single integrated mechanism of invoking all resource
wrappers in the grid environment
• Grid services invoked via the dynamic loading of resource
wrappers
• Resource properties provide access to memory resident
BDW cache data
• Access to cache data via EPR from Triana/client without
invoking the resource behind again
BiodiversityWorld GRID Workshop
NeSC, Edinburgh – 30 June and 1 July 2005
Unit A
Triana Workflow Units
Unit B
Unit C
MDR
Data Parsing Tools
BGI Grid Helper Tool
Metadata Agent WS
BGI Communications Layer
BgiWrapperService
BgiWrapper
BgiWrapper
BDWCache
Remote Resource
BiodiversityWorld GRID Workshop
NeSC, Edinburgh – 30 June and 1 July 2005
BDWCache
BgiWrapper
BDWCache
GT4
Container
Further Work
• Move computation on the grid in addition to
resource access mechanism
• BDW Resource pool, automatic update of
resources
• Making resources persistent
• BDW Heartbeat monitor
• Security, Index Service, Monitoring and
Discovery System (MDS), OGSA-DAI for
BDW resources
BiodiversityWorld GRID Workshop
NeSC, Edinburgh – 30 June and 1 July 2005
References
[1] Globus Tutorial http://gdp.globus.org/gt4-tutorial/
[2] Global Grid Forum http://www.ggf.org/ggf_abt_overview.htm
[3]The WS-Resource Framework http://www.globus.org/wsrf/specs/ws-wsrf.pdf
BiodiversityWorld GRID Workshop
NeSC, Edinburgh – 30 June and 1 July 2005