X#
Towards the CrossGrid
Architecture
Marian Bubak, Marek Garbacz,
Maciej Malawski, and Katarzyna Zając
Cracow Grid Workshop, November 5-6, 2001
X#
Outline
–
–
–
–
–
–
–
Introduction
CrossGrid Architecture Team
Layered architecture
Software integration
Standard operational procedures
Collaboration with DataGrid
Conclusion
Cracow Grid Workshop, November 5-6, 2001
X#
Introduction
–
–
–
–
–
Layered structure of the all X# applications
Reuse of SW from DataGrid and other # projects
Globus as the bottom layer of the middleware
Heterogeneous computer and storage systems
Distributed development and testing of SW
• 12 partners in applications
• 14 partners in middleware
• 15 partners in testbeds
Cracow Grid Workshop, November 5-6, 2001
X#
General rules
– Iterative improvement:
• development, testing on testbed, evaluation,
improvement
–
–
–
–
Modularity
Open source approach
SW well documented
Collaboration with other # projects
Cracow Grid Workshop, November 5-6, 2001
X#
Architecture Team
– Activity:
•
•
•
•
Merging of requirements from WP1, WP2, WP3
Specification of the X# architecture (protocols, APIs)
Establishing of standard operational procedures
Specification of the structure of deliverables
– Organization:
• Representatives of WPs (responsible for integration)
• Technical Architecture Team
Cracow Grid Workshop, November 5-6, 2001
X#
Layered Architecture
Interactive and Data Intensive Applications (WP1)
 Interactive simulation and visualisation of
a biomedical system
 Flooding crisis team support
 Distributed Data Analysis in High Energy
Physics
 Weather forecast and air pollution modelling
HLA
Datagrid
GriPhyN
...
Grid Visualisation
Kernel
Grid Application Programming
Environment (WP2)
Data Mining
 MPI code debugging and
verification
 Metrics and benchmarks
 Interactive and semiautomatic
performance evaluation tools
New CrossGrid Services (WP3)
Services
 Portals and roaming access
 Grid resource management
 Grid monitoring
 Optimisation of data access
Globus Middleware
Fabric Infrastructure (Testbed WP4)
Cracow Grid Workshop, November 5-6, 2001
X#
Architecture Components
WP3
3.1
Portal +
WP1
3.2
Roaming
Resource
Management
2.2
WP2
MPI
Verification
Applications
2.3
3.3
Benchmarks
Monitoring
2.4
3.4
Performance
analysis
Optimisation
of data
access
Cracow Grid Workshop, November 5-6, 2001
Connection with Datagrid
Data Management
Replica
Manager
Automatic data migration
& replication
Replica Catalog
interface
Replica
optimisation
CrossGrid
optimisation
of data access
Replica
Catalog
Cracow Grid Workshop, November 5-6, 2001
X#
Connection with Datagrid
Workload Management
Logging &
Bookkeping
Resource Broker
Matching job
requirements
against
available resources
RB Agent
High level
Strategy library
Crossgrid
Resource
Management
Job submission
Cracow Grid Workshop, November 5-6, 2001
X#
X#
Software Integration
– HLA - architecture for distributed simulations
– CCA - component-based approach
– Interoperability with DataGrid
Cracow Grid Workshop, November 5-6, 2001
X#
HLA -Overview
– consists of federates (simulations, simulation managers, data
collectors, passive viewers) that form federations
– is defined of three components:
• Federation rules
• proper interaction of simulations in a federation
• Interface specification
• defines Run-Time Infrastructure services
• Object Model Template
• common framework for HLA object documentation
Cracow Grid Workshop, November 5-6, 2001
X#
HLA - Important Features
• establishing a common high level simulation
architecture
• separates communication and simulation
• advanced time management
• flexible attribute and ownership mechanisms
• current implementation RTI 1.3 uses UDP multicast
communication
Cracow Grid Workshop, November 5-6, 2001
X#
Common Component Architecture
– Corba Component Model, Microsoft COM,
Java Beans
– CCA Forum addresses HPC community
– Scientific Interface Definition Language
• Components and ports
• Local and remote components
– CCA Toolkit (CCAT)
• framework based on Globus
Cracow Grid Workshop, November 5-6, 2001
X#
Software Integration - Summary
• Goal:establishing a common high level simulation
architecture on Grid for real-time simulations
• Possible solution:HLA implementation built within
component framework over GLOBUS architecture
Cracow Grid Workshop, November 5-6, 2001
X#
S/W Engineering Practices
– Developers’ common point of view
• Introduces work overhead
• Annoying and boring
– But the truth is
• Using strict operational procedures for
• Requirements traceability
• Configuration mgmt
• Testing
• ... helps to keep project activities on track
• We know where we are and what we do !
Cracow Grid Workshop, November 5-6, 2001
X#
Reqs. Traceability
– The 1st baseline
• Defined after the 1st quarter
• Base of project work !  Acceptance criteria
– Further modifications
• PRs/CRs mechanism - subsequent baselines
(versions).
– Identification
• All WPs deliverables –
• Refered reqs. baseline  what your work is based on?
Cracow Grid Workshop, November 5-6, 2001
X#
Integration &Testing
– WP1, WP2, WP3
• Unit tests
• Need for stubs for other WPs interfaces until they’re available
• WPs deliverables
– WP4 (International Testbed)
• Integration with DataGrid testbed
• Integration tests
• Hw/Sw components
• Software distribution/installation policy  what are we testing?
• Test plan
• Schedule
• Test logs, feedback distribution – what needs to be changed.
Cracow Grid Workshop, November 5-6, 2001
X#
Configuration Mgmt
– Central project repository
• Version control tool is to be choosen
• All project ‘official’ artifacts
• Source of code for WP4 to test
– Traceability of configuration items
• Maintained to upper level components
• E.g. design  requirements, code  design, etc.
Cracow Grid Workshop, November 5-6, 2001
X#
Conclusion
– We must start with an internal report which will
present
• Approaches to the definition of the architecture
• Coding guidelines
• Java conventions for naming, comments???
• Document publication rules
Cracow Grid Workshop, November 5-6, 2001