OGSA-DAI: Status and Future Plans
Neil Chue Hong
1
Status
 Project started 1st October 2003
– DAIT Funding for 2 years
– EPCC, NeSC, Newcastle, Manchester
 Industry Collaboration
– IBM engaged at start of 2004
– Oracle and Fujitsu on Programme Board
 People:
– 5 FTEs at EPCC, 5 FTEs at IBM
– 1 PDRA at each of NeSC, Newcastle, Manchester
 Releases:
– 6 monthly major releases, next is April
2
Project cycle
3. OGSA-DAI/T Implementation
2. DAIS
1. Exploitation
3
Milestones
WS-RF
Announced
R3.1: Technical preview of parts of R4
Stable Database
Stable
Services
Database
Specification
Servicesand
Specification and
Primer put forward
Primer
asput
a Proposed
forward as
Recommendation
a Proposed Recommendation
R5: Compliance with DAIS?, distributed query and
transactions, improved dependability and security,
coordinated contributor community
R6: Features depend on user priorities,
context and research
4
Mar '06 -
Feb '06 -
Jan '06 -
Dec '05 -
Nov '05 -
Oct '05 -
Sep '05 -
Aug '05 -
Jul '05 -
Jun '05 -
for the user community
May '05 -
Apr '05 -
Mar '05 -
Feb '05 -
Jan '05 -
Dec '05 -
Nov '04 -
Oct '04 -
Sep '04 -
Aug '04 -
Jul '04 -
R7.0 R7: Maintainable release
R6.0
R5.0
Jun '04 -
May '04 -
Apr '04 -
R4.0
Mar '04 -
Feb '04 -
Jan '04 -
R3.1
Release 4
 Built on top of Globus Toolkit 3.2
– No major change to perform doc schema
 Supported Client Toolkit library
– Easier development of applications
 Updated ActivityEngine
– Have addressed some memory and performance issues
 Additional DBMSs supported
– SQL Server, Postgres
 New GUI data browser
– contributed by FirstDIG Project
 Bulk load supported
– Allows distributed join scenarios using data browser
5
Release 4
 Secure Grid Data Transport implementation
 Documentation now in XHTML format
 Metadata registered with DAISGR
 Updated Exception Hierarchy
 Stored Procedure support
 DBMS Specific management operations
– Not DBMS transparent operations
 More file access prototypes
 User support, courses, training material
 Updated performance report
 Release date: end April / early May
6
DAIT Roadmap R5
R5 October 2004
– Possible alignment with WS-RF and DAIS Specs
• Assuming they settle in time
– At least a tech preview of OGSA-DAI and GT4
– Possible WS-I interface implementation?
– Distributed Relational Query Processing
• Looking to integrate the OGSA-DQP code
–
–
–
–
7
Improved dependability and security integration
Extended & integrated XML and relational facilities
Transaction participation
Coordinated OGSA-DAI contributor community
R6  R7
R6 April 2005
–
–
–
–
–
Integrated with GT4?
Functionality driven by user group (i.e. YOU)
New facilities depend on user priorities, context and research
OGSA-DAI components from contributor community
Increased data integration tools
R7 October 2005
– Maintainable release for the user community
8
OGSA-DAI and WS-RF
OGSA-DAI R4 will still be OGSI
OGSA-DAI R5 will probably have a WS-RF
interface
Currently investigating options for supporting
different interfaces in the future
– WS-RF, WS-I, legacy OGSI, …
OGSA-DAI codebase and architecture makes it
easier to port
– But still costly to support multiple versions
9
Issues
 Important facets that need to be taken into account:
– Metadata
• Data structure, access policies, data content, provenance, physical
properties, etc.
– Registries required for resource discovery/matching
• Allow dynamic binding to data sources based on provision of metadata
– Different modes of operation/delivery
• JDBC, ODBC, bulk data transfer, third-party data transfer, incremental
data transfer, delayed data transfer
– Security mechanisms
• Authentication, authorisation, accounting, audit, privacy/encryption
– Data transformation processes
• Reformatting, compression
– Facilitate transaction / workflow participation arrangements
10
Protecting Your Users
 Very rapidly moving field
– Technology changes
– Standard changes
– Middleware changes
 Need to ensure:
– Technology adopters investment in
OGSA-DAI is protected
– Shielded from future change
 Positives:
– Document interface helpful
– Client toolkit
11
DAIS Documents
OGSA Data Services Document
Grid Data Services Document
Relational
Realisation
XML
Realisation
Perform
Document
12
File
Realisation
Other
Specialisations
Supported Data Resources
Relational
XML
Other
MySQL

Xindice

DB2

eXist
?
Oracle

PostgreSQL


SQLServer
14
Files

Client Toolkit
Why? Nobody wants to write XML!
A programming API which makes writing
applications easier
– Now: Java
– Next: Perl, C, C#?
// Create a query
SQLQuery query = new SQLQuery(SQLQueryString);
ActivityRequest request = new ActivityRequest();
request.addActivity(query);
// Perform the query
Response response = gds.perform(request);
// Display the result
ResultSet rs = query.getResultSet();
displayResultSet(rs, 1);
15
Exploitation
We have delivered several releases of
OGSA-DAI There is much to be learned this
year from taking the application view
Tasks
– User Group Meetings
• Input to DAIS and DAI/T implementation
– Support to projects
• Skyserver, eDiaMoND, SIMDAT etc..
– Publications and presentations
• Globus World, GGF Workshops, All Hands etc
– Surveys and questionnaires
16
Requirements
What do you consider important?
–
–
–
–
–
–
–
–
18
Data access
Data integration
Application support
Service Extensibility
Performance
Security
Useability
User Support
Versioning
Which is more important?
–
–
–
–
–
–
–
–
–
–
OGSI / WS-RF /WS-I
DAIS specifications
Globus Tookit
Java
Java libraries
Databases
RowSet
XML Schema
Client Toolkit API
OGSA-DAI interfaces
How many versions should we support?
19
Platforms
Which is more important?
– Ease of installation and support
– Ability to access underlying platforms
Black box versus full extensibility
– How many people need to know whether its running on
Tomcat? On Axis?
20
Interfaces
Which is more important?
–
–
–
–
Continued support for OGSI
Port to WS-RF
Port to WS-I
Port to…
Web Services
– WSDL/SOAP/OGSA
– REST
– …
21
Data Integration
Which is more important?
–
–
–
–
–
–
–
–
22
Distributed Query Processing
Distributed Queries
Distributed Data Management
Federated databases
Virtual federation
Data virtualisation
Data provenance
Transactions
Languages
Which is more important?
Query languages
– SQL (92, 99, …), XPath, XQuery, RegExp, …
Programming languages
–
–
–
–
–
–
23
Java
C / C++
C#
Fortran
Perl / Python
…
Security
Which is more important?
– Transport Level Security
– Message Level Security
– Standards (GSI, GSS, Kerberos, WS-Security, …)
–
–
–
–
–
–
24
Scalability
Robustness
Performance
Definition
Management
Usability
Useability
Which is more important?
–
–
–
–
–
25
Client libraries for usability
Exemplar client framework
Additional Client Toolkit Activity implementation
Support for Client Toolkit version changes
Graphical components
User Support
Which is more important?
– Continue support for community
• via web site, mail list, announce, help, consultations
–
–
–
–
–
–
26
Training courses
Talks, demonstrations and presentations
User documentation
"How To" guides
Installation and configuration wizards
Different Installers and Uninstallers (e.g. WAR, …)
Requirements (again)
Which is more important?
– Data access
• SQL refactoring
• Interoperability of results
– Data integration
• Schema integration tools
• Data description tools
–
–
–
–
–
–
27
Application support
Service Extensibility
Performance
Security
Useability
User Support
Summary
Many paths which OGSA-DAI can take
– Which one to take
Future releases – need to be driven by your
requirements
– You need to tell us
It is a trade off
– No infinite development team
– Port versus support
– Functional and non-functional requirements
28