Databases and the Grid
OGSA-DAI
Architecture & Status
Malcolm Atkinson
OGSA-DAI Chief Architect
for
all members of the OGSA-DAI team
Director of National e-Science Centre
www.nesc.ac.uk
3rd September 2002
UK e-Science All Hands Meeting
Sheffield Hallam University
Overview
Database Task Force & GGF DAIS-WG
OGSA-DAI Project
Scope, Scale, Participants, Plans
Architecture
Status
Relationship with OGSA
Data Access & Integration
Central to e-Science
Collaboration




Shared Databases
Curated Knowledge
Accumulated Observations
Accumulated Simulations
Computation



Data mining
Input to models
Calibration of models
Presentation


Publication of results
Visualisation
UK DBTF
http://www.cs.man.ac.uk/grid-db
GGF
DAIS-WG
Membership
Malcolm Atkinson (NESC)
Vijay Dialani (Southampton Uni.)
OGSA-DAI
Norman Paton (Manchester Uni.)
Core Programme
Dave Pearson (Oracle UK)
Project
Tony Storey (IBM Hursley)
Paul Watson (Newcastle Uni.)
GGF DAIS WG
Chairs
Norman Paton (Manchester Uni.)
Leanne Guy (CERN)
Dave Pearson (Oracle UK)
Activity
BoF GGF4 Toronto
WG Meeting GGF5 Edinburgh
Workshops & Mail lists
Goals
Agree Standards for Database Access & Integration
Freely available reference implementations

OGSA-DAI one source & focus for discussions
OGSA-DAI Partners
IBM
USA
EPCC & NeSC
Glasgow
Newcastle
Belfast
Daresbury Lab
Manchester
Oxford
Cambridge
EPCC & NeSC
Oracle Hinxton
RAL
IBM UK
Cardiff
London
IBM Hursley
IBM USA
Southampton
Manchester e-SC
Newcastle e-SC
st February 2002
$5
million,
18
months,
started
1
Oracle
OGSA-DAI Scope
Definition and development of generic Grid data
services which provide access to and integration of
data held in databases, and the management of data
within a distributed environment.
Database
A stored, structured collection of data
Accessed using an API that takes account of the
structure of the data stored
Includes
Relational and object databases
XML repositories
Adequately described & managed collections of files
OGSA-DAI Plan
Two Phases
Phase 1: Started Feb 02 ends 30th September
Detailed Plan –








Requirements, Designs & Prototypes
6 Work Packages
Project Management (Oracle, EPCC)
Architecture (NeSC, DBTF)
XML Data Management (NeSC & EPCC)
Distributed Query Systems (Manchester & Newcastle)
Metadata & Registries (NeSC & EPCC)
Relational Databases (IBM UK)
Phase 2: 12 months


Structure and Objectives to be Refined in Major Review
GGF5 DAIS WG meeting a major input
OGSA-DAI Time Line
WS + GSI UK support ( > 100 downloads)
XML + OGSA Prototypes for Early Adopters
Design Documents & Demos for DAIS WG @ GGF5
XML + OGSA Prototype Available
RDB + GT2 / OGSA Prototypes Available
GGF6 WG Papers & Prototypes
Ship Alpha Release for GT3 Integration
Presentation & Beta @ GGF7
Productisation, RAMPS &
Extension
Feb ’02
May ’02
Phase 1 Starts
Jul ’02
Sep ’02
Dec ’02
Phase 2 Starts
Feb ’03
May ’03
Sep ’03
DAI Key Components
GridDataService
GDS
Access to data & DB operations
GridDataServiceFactory
GDSF Makes GDS
GridDataServiceRegistry GDSR Discovery of GDS(F) & Data
GridDataTransportVehicle GDTV Connects components + Moves Data
GridDataTransportDepot GDTD GDTV with persistence
OGSA Relationship
Class
GridService
GDS
Registry
NotificationConsumer
NotificationProducer
Mandatory
Optional
Normal
GDSF
Mandatory
Optional
Normal
GDSR
Mandatory
GDTS
Mandatory
Mandatory
Normal
GDTV
GDTD
Mandatory
Optional
Normal
DAI portType Usage
Class
GridDataService
GridDataTransport
GDS
Mandatory
Normal
GDSF
Optional
Normal
GDSR
Optional
GDTS
Optional
Mandatory
Optional
Mandatory
GDTV
GDTD
Factory
Mandatory
OGSA-DAI Architecture
GDSF
DSR
1
request
for
factory
client
OGSA-DAI Architecture
GDSF
DSR
1
request
for
factory
client
2
response
with
GDSFs
GSHs
OGSA-DAI Architecture
GDSF
3
script for 3
GDSs
DSR
1
request
for
factory
client
2
response
with
GDSFs
GSHs
OGSA-DAI Architecture
GDSF
4
creation of 3
GDSs
3
script for 3
GDSs
GDS1
DSR
1
request
for
factory
GDS2
client
2
response
with
GDSFs
GSHs
GDS3
OGSA-DAI Architecture
GDSF
4
creation of 3
GDSs
3
script for 3
GDSs
GDS1
DSR
1
request
for
factory
client
2
response
with
GDSFs
GSHs
5
response
with 3
GSHs
GDS2
GDS3
OGSA-DAI Architecture
GDSF
4
creation of 3
GDSs
3
script for 3
GDSs
GDS1
DSR
1
request
for
factory
client
5
response
with 3
GSHs
2
response
with
GDSFs
GSHs
GDS2
GDS3
6
scripts
requesting
DB actions
OGSA-DAI Architecture
GDSF
4
creation of 3
GDSs
3
script for 3
GDSs
GDS1
DSR
1
request
for
factory
client
5
response
with 3
GSHs
2
response
with
GDSFs
GSHs
GDS2
GDS3
6
scripts
requesting
DB actions
7
transfer data
batch to GDS2
stream to GDS3
OGSA-DAI Architecture
GDSF
4
creation of 3
GDSs
3
script for 3
GDSs
GDS1
7
transfer data
batch to GDS2
stream to GDS3
DSR
1
request
for
factory
client
5
response
with 3
GSHs
2
response
with
GDSFs
GSHs
GDS2
GDS3
6
scripts
requesting
DB actions
8
stream data
to GDS2
OGSA-DAI Architecture
GDSF
4
creation of 3
GDSs
3
script for 3
GDSs
GDS1
7
transfer data
batch to GDS2
stream to GDS3
DSR
1
request
for
factory
client
5
response
with 3
GSHs
2
response
with
GDSFs
GSHs
GDS2
GDS3
6
scripts
requesting
DB actions
9
transfer data
batch
to client
8
stream data
to GDS2
OGSA-DAI Architecture
GDSF
4
creation of 3
GDSs
3
script for 3
GDSs
GDS1
7
transfer data
batch to GDS2
stream to GDS3
DSR
1
request
for
factory
client
5
response
with 3
GSHs
2
response
with
GDSFs
GSHs
GDS3
6
scripts
requesting
DB actions
10
stream data
to specified
destination
GDS2
9
transfer data
batch
to client
8
stream data
to GDS2
Status
Teams & project coordination effective
Relationship with Early Adopters
with Grid Support Centre
with Globus
High rates of interaction
XML GDS & GDSF prototypes available
RDB demo available
Distributed Query demo available
Papers & designs presented at GGF5
www.nesc.ac.uk
Functional Scope &
Architecture for Phase 2
Drafts & Intensive discussions
OGSA-DAI & OGSA <((-:}
Description, e.g. portType Works Well
Expect to make extensive use of
Data Service Elements

Special to DBs: Static & Dynamic
Component Management
Notification
Grid-FTP
Accounting
Security:

Authentication, Authorisation & Privacy
Reliable invocation
…
OGSA-DAI Summary
On Schedule & Going Well
Contributions via DAIS-WG @ GGF5, 6, 7, …
Coordinating with GT3 Releases
Ending Phase 1 (Design Exploration)
Testing Architectural Design
Using OGSA
Working with Early Adopter Pilot Projects


AstroGrid & MyGrid and others
Many requests for access to the software
Releasing prototypes
Influence OGSA-DAI direction
Via DAIS-WG & as Prototype users