OGSA-DAI
responding to a
changing world
UK Globus Week
7th April 2005
Neil Chue Hong
Project Manager, EPCC
N.ChueHong@epcc.ed.ac.uk
+44 131 650 5957
Overview
• What is OGSA-DAI?
• What was OGSA-DAI like?
• What is OGSA-DAI like now?
• The GT3 to GT4 transition
• What are the overheads?
• What’s the future like?
7th Apr 2005
UK Globus Week - www.ogsadai.org.uk
2
OGSA-DAI In One Slide
• Data access and integration.
• Expose heterogeneous data
resources to a grid through web
services.
• Interact with data resources:
–Queries and updates.
–Data transformation and
compression.
–Data delivery.
• Through a uniform documentoriented XML interface.
• Extensible framework for
efficiency and ease of use
–Activities
–Client Toolkit
7th Apr 2005
UK Globus Week - www.ogsadai.org.uk
3
OGSA-DAI As It Was
•
OGSI-compliant: Globus Toolkit 3 implementation.
– Release 1 used GT3 TP4
– Release 5 used GT3.2.1
•
DAI Service Group Registry:
–
•
Grid Data Service Factory:
–
•
Find a data resource.
Exposes a data resource.
Grid Data Service:
–
–
Allows interaction with a data resource.
GridDataTransport interface for third-party delivery.
DAISGR
7th Apr 2005
locates
GDSF
creates
Data
UK Globus Week - www.ogsadai.org.uk
GDS
4
GT3: base for Grid Services
• GT3:
– used Java Core, Security, GridFTP, utilities
– made it easy to develop stateful services quickly
• Issues:
– Early iterations (esp TPs) changed unexpectedly
– Later iterations much more stable, but reliant on particular Axis versions
(mostly Axis immaturity) and particular Jar versions
– this continues to happen in GT4
– Documentation was patchy
– how did you learn to build a Grid Service?
– were you a sys admin, developer, user or all three?
– “Hot” redeployment is nearly impossible
– needed for production service
– mostly Axis’ fault?
– We wanted to develop on Windows as well!
7th Apr 2005
UK Globus Week - www.ogsadai.org.uk
5
GT3 to GT4
• OGSI was deprecated:
– But we have to continue to support existing users.
– They may have made a significant investment.
• And was replaced by WSRF:
– So we need to be compatible.
– Use Globus Toolkit 4.
– provides us with the same (more) functionality
• And WS-I also comes along:
– And we are requested to support this also.
– Use OMII_1 or Apache Axis 1.2 / Tomcat.
• Are we the only ones having to do this?
7th Apr 2005
UK Globus Week - www.ogsadai.org.uk
6
A Problem with old OGSA-DAI
• Leveraged GT3 at all levels.
• Service implementations:
– Naturally!
• Core OGSA-DAI functionality:
– Useful XML <=> Java utility classes.
• OGSA-DAI activities:
– GridFTP and GridDataTransport third-party delivery.
• A lot of dependencies
– because we assumed (were told) a single middleware
7th Apr 2005
UK Globus Week - www.ogsadai.org.uk
7
Solutions
• Maintain three flavours of OGSA-DAI:
– +VEs:
– Leverage hosting framework utilities.
– -VEs:
– Three versions of the core functionality and documentation.
– Significant overlap between the flavours.
– Overheads of adding new core functionality.
• Raise the GT3 dependencies to the service level:
– Where they should be!
7th Apr 2005
UK Globus Week - www.ogsadai.org.uk
8
OGSA-DAI As It Now Is (1/2)
Client
Client Toolkit API
Client
OGSA-DAI Data Services
WSRF
WSRF
OGSI
WS-I
OGSI
WS-I
SOAP over HTTP
WSRF
Server
WS-I
XML
Presentation
Business
Logic
DAI Core
Relational
7th Apr 2005
OGSI
File
UK Globus Week - www.ogsadai.org.uk
Data
9
OGSA-DAI As It Now Is (2/2)
• Three specifications supported at the service-level:
– Yet sharing the same core OGSA-DAI functionality.
• DAI Core is independent of any web services toolkit.
– probably not independent of web services?
• But some current limitations:
– GridFTP or GridDataTransport activities in WS-I.
– Security in WS-I
– Access to WS-I or WSRF services via the Client Toolkit:
– A priority for OGSA-DAI Release 6 in April 2005.
• Provide all OGSI version functionality in WS-RF and WS-I releases
7th Apr 2005
UK Globus Week - www.ogsadai.org.uk
10
What changed?
• GT3->GT4:
–
–
–
–
–
–
rebuild against GT4
change namespaces
change schema
change some variable names, some methods
SDEs to ResourceProperties
…
• All fairly trivial stuff BUT
– breaks “message” compatibility with GT3
– need to check all tests
– complicates development
7th Apr 2005
UK Globus Week - www.ogsadai.org.uk
12
One-off Overheads
• 3-4 weeks of effort by 2 developers to move from GT3 to
GT4.
• Other developers moved onto next-phase design activities:
– No hanging around.
• BUT there are also…
7th Apr 2005
UK Globus Week - www.ogsadai.org.uk
13
Ongoing Overheads
• Multiply the following overheads:
– Service-level:
– Development.
– Testing.
– Documentation.
– Building and testing releases.
– User support.
• Testing is difficult for users (rather than developers)
– how do you bundle tests which require web services to be deployed
7th Apr 2005
UK Globus Week - www.ogsadai.org.uk
14
And Some More
• Possibly:
–
–
–
–
–
Provide inter-operability between flavours.
Bundle OGSA-DAI with OMII.
Bundle OGSA-DAI with GT4.
Support additional WS-I or WSRF implementations.
We are becoming OEM producers but have to repackage ourselves
• Clarification for our users.
• At the cost of:
– Performance enhancement.
– Developing functionality.
• We have lost probably 50 months of staff effort due to the multiplication of
support effort so far
7th Apr 2005
UK Globus Week - www.ogsadai.org.uk
15
Utility of a Client Toolkit
• Protect applications from:
– Manipulating XML.
– Changes in APIs.
– Changes in specifications.
• Provide stability to applications in troubled times.
• Support multiple specifications:
– Common interface to WS-I, WSRF and OGSI services.
– Encourage use of WS-I and WSRF – wean off OGSI.
• Choose suitable abstractions:
– Registry, Data Service
– Applications don’t care about WS-I, WSRF or OGSI.
– They just want to do their data stuff!
7th Apr 2005
UK Globus Week - www.ogsadai.org.uk
16
Supporting Other Specifications
• Other OGSA, data, web service specifications:
– Security:
– WS-Security, WS-SecureConversation, AuthZ
– Transactions:
– WS-AtomicTransaction, WS-Coordination
– …
• Importance of extensibility and abstraction:
– Define or select standard APIs for security, transactions, auditing,
accounting,…
– Allow users to plug in modules implementing their required specifications.
– Avoids specific specifications leaking into DAI core.
7th Apr 2005
UK Globus Week - www.ogsadai.org.uk
17
Is GT4 better?
• Is GT4 better?
– documentation is much improved
– installation is much improved
– but still quite difficult to get at a particular “part” if you’re running on
Windows
– some components have “beacon” tests, some don’t
– errors are improved
– but still difficult to locate the problem
– security has changed
– we might be able to do some dynamic reconfiguration!
– will be doing performance tests on OGSA-DAI R6 versus previous releases
– to be presented later in year
• Still the case that Google is often the easiest way of finding answers
– but we like gt4-friends!
7th Apr 2005
UK Globus Week - www.ogsadai.org.uk
18
What about the future?
• Existing userbase on GT3
• New userbase on OMII and GT4
• Getting people on to the NGS
• How long do we continue support for GT3
– what happens when support officially ends?
– how do we persuade people to migrate?
– how do we ensure this does not happen (often) again?
• What happened to the GTR?
• Long live the GDP!
7th Apr 2005
UK Globus Week - www.ogsadai.org.uk
19
Summary
• Supporting different interfaces is difficult:
•
•
•
– Multiple flavours are necessary as neither WS-I or WSRF
dominates (just now).
It is still not easy to develop services for split GT3/GT4/ANO
platforms
Client toolkits can protect users and assist deprecation
process.
Support for users will become a priority above development
of software
7th Apr 2005
UK Globus Week - www.ogsadai.org.uk
20
Geographically Distributed Team
EPCC Team, Edinburgh
NeSC, Edinburgh
NEReSC, Newcastle
ESNW, Manchester
IBM Development Team, Hursley IBM Dissemination Team
7th Apr 2005
UK Globus Week - www.ogsadai.org.uk
21