The Structure of D-Grid
Uwe Schwiegelshohn
D-Grid Corporation
TU Dortmund
October 13, 2008
History of D-Grid
2004
German Grid initiative by researchers with industrial participation
2005
Integration project and 5 (later 7) mainly academic communities
(phase 1)
2006
Grid hardware investments (7 core centers and community
additions)
2007
Communities with industrial participation, gap projects, service layer
projects (phase 2)
Grid hardware investments (cooperations: bwgrid, physics, climate
research)
2008
Successor of D-Grid integration project 2, D-Grid corporation
Grid hardware and software investments (network storage, grid
licenses, visualization) in preparation
2009
Communities with industrial leadership, gap projects, SLA and
knowledge layer in preparation

Cracow Grid Workshop 2008, Kraków, Poland
Structure of D-Grid
Uwe Schwiegelshohn
Funding of D-Grid
• D-Grid is mainly funded by the German Federal
Ministry of Research and Education (BMBF)
– Some contribution of some research institutions
– Co-payment of industrial contributors
• 22 projects (excluding phase 3)
– 117 partners
• Total funds exceeding 100 Mio €
– including phase 3 and all investments
• Funding period
– 2005 – 2011
– Funding period for each project is at most three years
• Funding after 2011
– No further funding of industrial communities is planned
– Continuous funding of academic communities on a discipline level
– Funding model of the infrastructure must be established.

Cracow Grid Workshop 2008, Kraków, Poland
Structure of D-Grid
Uwe Schwiegelshohn
Project Structure of D-Grid
Academic Community
Projects
Commercial Community
Projects
Service Layer
Projects
Gap - Projects
Integration Project

Cracow Grid Workshop 2008, Kraków, Poland
Structure of D-Grid
Uwe Schwiegelshohn
Integration Project (DGI 1 and DGI 2)
• Support
–
–
–
–
3 different middlewares
Globus 4.x, LCG/gLite, UNICORE
SAGA/GAT
only level 2 support, no direct user support
Helpdesk (only DGI 2)
Reference installation
concurrent operation of all middlewares
• Operation of D-Grid: Guidelines and central components
– Operation of VO services and basic user services
– Operation of resources
common to most
communities
• Security
– AAI/VO
– Firewalls
• Data and Information Management
– dCache, OGSA-DAI, IRODS
– Metadata handling (only DGI 2)
• Development of Basic Components
– Portal (GridSphere), Accounting, CRM, Monitoring (mainly DGI 2)
• Sustainability
development of a business model

Cracow Grid Workshop 2008, Kraków, Poland
Structure of D-Grid
Uwe Schwiegelshohn
D-Grid Infrastructure
Communities
Portal (GridSphere)
User-/
applicationinterfaces
GAT/SAGA API
D-Grid services
Nutzer
Scheduling and
workflow management
Grid services
Monitoring
UNICORE
LCG/gLite
Accounting and
billing
Special focus of DGI 2
Basic services
Security and
VO management
Globus toolset V4
Data management
Data access
D-Grid resources
Distributed
data archives
Data/
software
Network
infrastructure
Distributed
compute
resources

Cracow Grid Workshop 2008, Kraków, Poland
Structure of D-Grid
Uwe Schwiegelshohn
Use of Middleware Systems
• gLite/LCG
–
–
–
–
used exclusively by high energy physics
largest number of active users
large number of production jobs
largest European-wide network
• UNICORE
– used by two projects with industrial participation
• AeroGrid: aerospace industry
• BISGrid: heterogenous business information systems for SMEs,
orchestration of Grid workflow engines (BPEL)
• GT 4.x
– used by most projects (all other academic communities)
– largest number of potential but not yet active users (medical
community and humanities)
– large number of production jobs from a small number of users
(astrophysics)

Cracow Grid Workshop 2008, Kraków, Poland
Structure of D-Grid
Uwe Schwiegelshohn
Project Structure of D-Grid
Academic Community
Projects
Commercial Community
Projects
Service Layer
Projects
Gap - Projects
Integration Project

Cracow Grid Workshop 2008, Kraków, Poland
Structure of D-Grid
Uwe Schwiegelshohn
Gap - Projects
• Integration and Interoperability of VO-Technologies
– VOMS from gLite
– VOMRS (VO Membership Registration Service)
– Shibboleth
• Resource and Service Monitoring
– Grid-wide monitoring across different systems (MDS4, BDII/MDS2,
UNICORE CIS)
– VO-specific view for the communities
– Integration of external monitoring tools (Nagios)
• Potential new gap projects
– Short lived credential service
– Grid intrusion detection
– Scheduling interoperability

Cracow Grid Workshop 2008, Kraków, Poland
Structure of D-Grid
Uwe Schwiegelshohn
D-Grid Infrastructure
Communities
Portal (GridSphere)
User-/
applicationinterfaces
GAT/SAGA API
D-Grid services
Nutzer
Scheduling and
workflow management
Grid services
Monitoring
UNICORE
LCG/gLite
Accounting and
billing
Security and
VO management
Basic services
Globus toolset V4
Data management
Data access
D-Grid resources
Distributed
data archives
Data/
software
Network
infrastructure
Distributed
compute
resources

Cracow Grid Workshop 2008, Kraków, Poland
Structure of D-Grid
Uwe Schwiegelshohn
Project Structure of D-Grid
Academic Community
Projects
Commercial Community
Projects
Service Layer
Projects
Gap - Projects
Integration Project

Cracow Grid Workshop 2008, Kraków, Poland
Structure of D-Grid
Uwe Schwiegelshohn
Service Layer Projects
• Operational Information Systems and Orchestration
– Middleware enabling enterprises to switch between enterprisespecific Grid-enabled EAI and Grid utilization
– Integration of WS-BPEL into Grid technology
• Public Private Partnership Support
– Transition of enterprise and utility Grids to open service Grids or
SOKU
• Sustainable Grid Infrastructures
– Disseminating the knowledge of Grid technology
– Simplifying installation and maintenance of Grid middleware
• Potential new service layer projects
– Service Level Agreement Layer
– Knowledge Layer

Cracow Grid Workshop 2008, Kraków, Poland
Structure of D-Grid
Uwe Schwiegelshohn
Project Structure of D-Grid
Academic Community
Projects
Commercial Community
Projects
Service Layer
Projects
Gap - Projects
Integration Project

Cracow Grid Workshop 2008, Kraków, Poland
Structure of D-Grid
Uwe Schwiegelshohn
Academic Community Projects
• Climate Research
– evaluation of multi-model ensemble simulations
• High Energy Physics
– evaluation of CERN experiments (LHCb, Atlas, Alice, CMS)
• Astrophysics
– collaborative management of astrophysical resources including
telescopes and detectors
• Medical Sciences
– availability of heterogeneous data from basic research, clinical
research and patient caring
• Humanities
– collaborative editing, annotating, analyzing, and publishing of
specialist text sources
Are there similarities among these different communities?

Cracow Grid Workshop 2008, Kraków, Poland
Structure of D-Grid
Uwe Schwiegelshohn
Project Structure of D-Grid
Academic Community
Projects
Commercial Community
Projects
Service Layer
Projects
Gap - Projects
Integration Project

Cracow Grid Workshop 2008, Kraków, Poland
Structure of D-Grid
Uwe Schwiegelshohn
Commercial Community Projects
• Aerospace Engineering
• Financial Business Grid
• Automotive Industries
• Construction Industries
• Spatial Data Infrastructure Grid
• Engineering Applications
• Metereology for Power Generation
• Cooperative Developments using the Grid
Cooperation between industries, research centers and universities
Commercial use of Grid applications
Virtualization techniques
Security challenges

Cracow Grid Workshop 2008, Kraków, Poland
Structure of D-Grid
Uwe Schwiegelshohn
Organizational Structure of D-Grid
D – Grid Corporation
Representation
Service and
resource providers
Representation
Industrial users (SME)
Representation
Academic
communities

Cracow Grid Workshop 2008, Kraków, Poland
Structure of D-Grid
Uwe Schwiegelshohn
Present Tasks of D-Grid Corporation
• Coordination of D-Grid Projects
– Tracking of project progress
• Determination of problems in project schedules
• Discussion of approaches to handle these problems
– Initiating communication between projects
• Dissemination of project knowledge and expertise
• Circulation of public project information
– Addressing of problems relevant to several projects
• Collection of solution proposals from different projects
• Mediating in case of interaction problems between different
projects
• Representation of D-Grid
– Dissemination of information on D-Grid
– Representation on an international level

Cracow Grid Workshop 2008, Kraków, Poland
Structure of D-Grid
Uwe Schwiegelshohn
DGI and Communities
• The other D-Grid projects selected the content of DGI 2.
• Gap project proposals are openly discussed among the projects
before being submitted.
• DGI is planned evolve into a D-Grid Support Unit
– Service offering management process
– Funding model
• Membership fees of large research institutions
– Central components like security or operation
• Pay-for-a-service for different service packages
– Alternative components like support of a specific middleware
• Project funds
– Temporary services
• Service layers use a pay-for-a-service model (with project funds?)
• D-Grid is user-(community-)driven.
• Communities do not need to be exclusively German.
– International initiatives like high energy physics

Cracow Grid Workshop 2008, Kraków, Poland
Structure of D-Grid
Uwe Schwiegelshohn
Open Problems
• Number of users in most communities
– There are only few active users in most communities
• Education, coaching, user support
– Who takes care of individual users (not in a community)?
• Intermediate representation level
– How many representatives are needed?
– How are representatives selected?
– How representatives interact with each other?
• Legal problems
– Transferring services across states (and countries)?
– Private public cooperation
• Funding system
– Selection of services not belonging to the own organization
– Infrastructure investments

Cracow Grid Workshop 2008, Kraków, Poland
Structure of D-Grid
Uwe Schwiegelshohn