PUBLISHABLE EXECUTIVE SUMMARY
Enabling Grids for E-sciencE
Accomplishments during the first
reporting period
April 2004 to December 2004
Project Goals
The EGEE project involves partners from over 27 countries with the common
aim of developing an international computing grid infrastructure in Europe
which has access to major computing resources worldwide. This grid
infrastructure will be available to scientists around-the-clock and independent
of their geographic location, and is based on the EU Research Network
GÉANT. The goals of the EGEE project are:
* to build a consistent, robust and secure grid that attracts new computing
resources;
* to improve and maintain the middleware in order to deliver a reliable service
to users;
* to attract new users from science and industry and ensure training and
support for them.
Funded by the European Commission, the EGEE project consists of 12
partner federations, with 70 contractors and over 30 non-contracting
participants. Two pilot application domains have been selected for the project:
the Large Hadron Collider Computing Grid (LCG¹) supporting data-intensive
High Energy Physics (HEP) experiments and the Biomedical Grids,
addressing bioinformatics and healthcare data. The project comprises eleven
activities which are summarised below.
¹ The international grid project for particle physics. See http://lcg.web.cern.ch/LCG/
EGEE is a project funded by the European Commission - contract number INFSO-RI-508833
PUBLISHABLE EXECUTIVE SUMMARY
Project Management (NA1)
Prior to the project launch, the project management ensured submission of the
EU contract and Consortium Agreement and coordinated preparation of the
programme of work.
The main boards and committees established and coordinated by the project
management are: the Project Management Board (PMB) which deals with
managerial issues and assesses project progress (quarterly); the
Administrative Federation Meeting (AFM) which monitors, discusses and
solves administrative issues; the Project Executive Board (PEB) in which
activity leaders meet with the Project Director and Technical Director and
monitor technical progress of the project (weekly); the Collaboration Board
(CB) composed of representatives from all EGEE official partners (meet biannually at project conferences); the External Advisory Committee (EAC)
whose members attend the project conferences and provide feedback and
advice on the project’s progress to the Project Director.
During the first nine months of the project, the management has completed the
following tasks: instigated the writing of the execution plan documents from
each activity during the project start-up phase (specifies work breakdown,
individuals involved and deliverables); established the reporting procedures for
the quarterly reporting exercise in collaboration with the quality assurance
activity manager (includes timesheet reporting process, which was
successfully automated in the second quarter of the project); prepared the first
contract amendment, resolving inconsistencies and completing outstanding
sections; developed an open source licence for all software produced in the
project; participated in major technical conferences and public events worldwide on behalf of the project; established and trained the project office; and
managed and coordinated all project activities.
Dissemination and Outreach (NA2)
NA2 has designed and developed a consistent brand and style for all project
dissemination material and has produced a project folder, written a variety of
information sheets about key areas of the project and designed templates for
generic posters, technical posters, word documents (information sheets and
news releases) and presentations.
On the internet, the EGEE main portal gives users the opportunity to enter the
external site, technical site or training site. The external website targets a
broad audience and the NA2 partners have developed national language
versions using similar layouts. Journalists and the public can register on the
site to join the EGEE mailing list. A monthly EGEE newsletter is produced by
the CERN project office since September 2004.
To date there have been three press releases issued centrally by TERENA and
CERN and 24 issued in different languages (mainly translations of the centrally
issued release). EGEE has been featured in over 90 publications (online and
print) and promoted at over 100 different events worldwide.
Two EGEE conferences have been organised so far, one in Ireland (in April
2004) with 300 participants and one in The Netherlands (November 2004) with
401 participants. NA2, in collaboration with NA1 and NA4, also organised an
EGEE stand and live demonstrations at the IST 2004 event at The Hague,
November 2004.
EGEE is a project funded by the European Commission - contract number INFSO-RI-508833
PUBLISHABLE EXECUTIVE SUMMARY
User Training and Induction (NA3)
Over the reporting period, the EGEE training activity (NA3) has rapidly built up
capacity to ensure consistent delivery of quality training. Some 39 induction
and training events have been held, attended by over 1000 participants in
total. All of the federations have delivered training activities and the material
from all events is available via the NA3 web pages.
The GILDA testbed and GENIUS portal, developed by NA4, provide an
important resource for EGEE training which gives new users the opportunity to
access grid applications without waiting for certification. This allows a deeper
interaction with applications after becoming certified but before joining a Virtual
Organisation (VO). It also allows application developers to gain experience in
a grid environment.
Applications Identification and Support (NA4)
EGEE has already achieved its two-year target of having five disciplines
making use of the infrastructure. All four LHC HEP experiments (ALICE,
ATLAS, CMS and LHCB) have successfully used the EGEE infrastructure for
data challenges during 2004. There is a growing decentralized usage by
biomedical application developers, with about 20,000 jobs submitted
corresponding to approximately 30,000 hours CPU hours. Earth Sciences
applications, selected by the EGEE Generic Applications Advisory Panel
(EGAAP) in June 2004, are already active on the production infrastructure.
Astrophysics and computational chemistry are using the GILDA facility for
further testing.
EGEE support of LCG has helped ensure migration of HEP pilot applications,
through the creation of the HEP ARDA project. This project evaluates the new
EGEE middleware and has provided the experiment analysis prototype
systems to all 4 LHC experiments. Other HEP experiments, including DØ,
CDF and BaBAr, are also starting to make use of the EGEE infrastructure.
EGEE is a project funded by the European Commission - contract number INFSO-RI-508833
PUBLISHABLE EXECUTIVE SUMMARY
Policy and International Cooperation (NA5)
NA5 cooperates with national and international grid initiatives (US NSF), for a
(GGF) and other projects, and supports the e-Infrastructures and eInfrastructures Reflection group (eIRG) events. The purpose of the eIRG is to
facilitate dialogue between technical experts and policy makers in all
European countries.
In close collaboration with Project Management, NA5 has concentrated on:
cooperation between EGEE and the US future NSF Cyberinfrastructure
initiative; establishment of Russian and US grid federations in the EGEE
project management structure; preparation of a Memorandum of
Understanding (MoU) signed with South Korea to cover collaboration on grids
for physics and biology; planning the extension of the EGEE infrastructure to
the Baltic States and EGEE training events there in autumn 2004; and further
extensions to Latin American countries and other non-European areas.
This activity helped organise the e-Infrastructures workshops and meetings
and edited the eIRG White Paper, as well as revisions of this produced for the
eIRG meetings in April 2004 and November 2004. During the World
Conference on Information Technology, the activity supported the organisation
of a special forum dedicated to grid technology and services. NA5 has also
engaged with other EU grid projects, including the EU RI DEISA project, to
help develop a roadmap for the European Research Infrastructure.
Collaborative agreements have been made with SEE-GRID and GRIDCC, and
there has been consultation with related projects and initiatives such as
DILIGENT and the UK Open Middleware Infrastructure Institute (OMII).
In conjunction with the Project Office, NA5 organised and contributed to the
first Concertation meeting on e-Infrastructures in November 2004, which was
arranged in combination with the second EGEE conference. More than 240
people attended from over 15 European Research Infrastructure and Grid
Research projects.
EGEE is a project funded by the European Commission - contract number INFSO-RI-508833
PUBLISHABLE EXECUTIVE SUMMARY
Grid Support, Operations and Management (SA1)
Leveraging resources provided by LCG, SA1 has assembled and operates a
production grid with more than 90 resource centres providing in excess of
9000 processors, thereby greatly surpassing the two-year target of 50 sites. All
EGEE federations are actively using this grid and have resources available on
it at a level close to or exceeding project commitments (Annex 1 of contract).
There are more than 700 registered users of this grid. There are some 2000
CPU provided through non-EGEE sites in Taiwan, China, US, Japan, Canada,
Pakistan and India.
The establishment of the Regional Operations Centres (ROCs) in all the
regions is complete and they provide the first level support for deployment and
operations within their regions. The establishment of the Core Infrastructure
Centres (CICs) is well advanced, and the development plan is being
implemented. The four CICs currently take weekly turns to provide a (5x8)
service to monitor infrastructure problems and their resolution. The activity has
established tools to support this work, and written operational procedures that
are being implemented and further developed. This service started in
November 2004.
The processes for the creation and management of new Virtual Organisations
(VOs) have been established. The BioMed VO, Earth Resources VO, MAGICS
VO, EGEODE VO and Computational Chemistry VO are active alongside the
HEP VOs. In total there are more than 50 VOs deployed on the infrastructure,
including around 40 VOs of a regional nature.
Significant effort has been invested in the support of the LCG data challenges.
This is the first major use of a grid service of this size, providing valuable
feedback on middleware functionality, operations procedures, and debugging
and management tools. Peaks of over 8000 concurrent jobs have been
reached.
The pre-production service is active, using the LCG-2 middleware, with a few
gLite (the new EGEE middleware) components deployed on top of the basic
service. Technical discussions are taking place with other grid projects
(Grid3/Open Science Grid (OSG) and NorduGrid) to achieve interoperability
between the different infrastructures.
EGEE is a project funded by the European Commission - contract number INFSO-RI-508833
PUBLISHABLE EXECUTIVE SUMMARY
Network Resource Provision (SA2)
A document containing a comprehensive view of network services in GÉANT
and European NRENs was compiled during the summer, with the help of
TERENA which includes a detailed report about the implementation of network
services. The operational interface was drafted and then discussed at the
second EGEE conference. The updated version defines the major tasks and
outlines the main issues and solutions to them. Operational interface
components have been proposed, including Trouble Ticket Manager (TTM),
EGEE Network Operations centre (ENOC), Topology Database and SLA
database.
SA2 is due to produce a document "Open issues and model for network
services provisioning from GÉANT and the NRENs towards EGEE" early in
2005. A document with EGEE requirements was compiled. It focuses mainly
on two areas: Bandwidth Allocation and QoS, and Network Performance
Monitoring.
A document on terms and definitions used for SLR formulation was prepared,
discussed with EGEE members and published. It proposes the structure and
syntax of Service Requests. A study of service classes currently available in
GÉANT was also produced. It proposes new service classes, suitable for
serving the SLRs of EGEE applications.
Middleware Re-engineering and Integration (JRA1)
The architecture of the next generation grid middleware (gLite) being
developed by JRA1 has been documented. The gLite grid services follow a
Service Oriented Architecture, facilitating interoperability among grid services
and allowing easier compliance with upcoming standards such as WSRF. The
architecture of this set of services is not bound to specific implementations and
can be deployed and used independently.
The definition of the gLite grid service interfaces enables other grid
middleware implementations to offer the same interfaces allowing different
suppliers to interoperate and be interchanged. The interface design of the
gLite grid services has been completed and documented.
Existing services have been selected where possible from previous projects
such as AliEn, EU DataGrid and VDT. gLite was first installed on a distributed
testbed spanning CERN and Wisconsin University. Expert users from NA4
assessed the interfaces and semantics of the grid services of this prototype.
This testbed was not foreseen in the description of work but has proved
invaluable for gLite development.
gLite components passing integration and testing phases have been delivered
to SA1 for deployment on a pre-production service. Services developed within
gLite are: Workload Management Services; Data Management Services;
Information and Monitoring Services; Virtual Organisation Membership
Services (VOMS); Grid Access Services (GAS); and Package Manager (PM)
suites. Links to other projects have been established, including OGSA-DAI,
Unicore, and GEMSS. Contacts have been made with Platform Computing
and PPDG in the US, the University of Manchester (WSRF:Lite) and the
MammoGrid project. Existing contacts with DILIGENT and GridLab have been
intensified. Various activity members contribute to ongoing standardisation
efforts through GGF research and working groups.
EGEE is a project funded by the European Commission - contract number INFSO-RI-508833
PUBLISHABLE EXECUTIVE SUMMARY
Quality Assurance (JRA2)
The Quality Assurance Group (QAG), composed of quality representatives
from each activity, meets monthly. The Quality Plan, including the EGEE
Measurement Plan, has been delivered, and defines quality measures
applying throughout the project. It ensures standards and procedures are
established and followed throughout the project life cycle, defines quality
criteria by which the services can be evaluated as well as procedures to define
and monitor services (e.g. documentation, user support) within operations.
The main project procedures (document management and associated
templates, EU deliverables review, etc.) have been implemented on the JRA2
pages of the technical website. The main project metrics destined to evaluate
the project progress have been identified and the Measurement Plan
initialised.
Security (JRA3)
The middleware security group (MWSG), chaired by JRA3, is where activities
JRA1, JRA3, SA1, NA4, the Joint Security Group, Global Grid Forum (GGF),
Open Science Grid (OSG) and others meet to solve security issues
concerning the middleware design. Security requirements from all parts of the
project are prioritised by MWSG and handed over to JRA3 for implementation.
Together with the Project Office and NA5, this activity supports cross-project
collaboration aiming at a shared security infrastructure via the Joint Security
Policy Group (JSPG). Other achievements include: middleware security work
for the guiding documents Global Security Architecture and OGSA SEC
service initial recommendations for reengineering; supporting preparation of
security components for the JRA1 middleware (gLite) as part of software
maintenance and development; preparation of the document Security
operational procedures and incident handling, definition of a common grid
incident format in coordination with JSPG and OSG.
The activity’s security architect has been appointed area director of Security at
GGF12, facilitating future standardisation efforts. The activity also contributes
to integrating the Globus Dynamic Account System (DAS), and to coordinating
and collaborating with JRA1 security work (VOMS) and LCG security work
(VOMS admin). JRA3 is supporting LCG for legacy components LCAS and
LCMAPS and the grid-ftp server, and has completed the set-up of the PMA for
European Certification Authorities (CAs). Since then a recurring task has been
to chair the EUGridPMA and to expand the number of members.
EGEE is a project funded by the European Commission - contract number INFSO-RI-508833
PUBLISHABLE EXECUTIVE SUMMARY
Network Services Deployment (JRA4)
After an initial requirements gathering exercise, JRA4 initiated its prototype
implementation stage. In the network performance monitoring sub-activity
(NPM) a common interface for publishing and accessing network performance
information was agreed upon. This interface is based upon the NM-WG GGF
recommendation and involved GÉANT, participating NRENs and EGEE. An
early prototype not intended for inclusion in gLite was produced.
In the bandwidth allocation and reservation (BAR) on-demand sub-activity the
first high-level specification of the components and interfaces required to
request and access bandwidth services from the network providers has been
produced. A draft architecture was generated and the first BAR prototype is
being developed. A joint meeting at the second EGEE conference, with 55
participants, resulted in collaboration with JRA1 and JRA3. The Technical
Network Liaison Committee (TNLC) meeting at the same event strengthened
links with SA2. For the IPv6 task, contacts were established with JRA1 and
NA3, as well as with 6NET, to plan raising awareness. IPv6 training events
have been planned for EGEE partners to promote uptake.
EGEE is a project funded by the European Commission - contract number INFSO-RI-508833