Recommendation 1: NA1
Enabling Grids for E-sciencE
•
The reviewers endorse the proposed changes to the DoW for year 2 and
recommend that the pragmatic and flexible approach adopted so far be
continued as the EGI plans and corresponding transition challenges
become clearer
• The plans outlined during the review will be refined and
documented as an amendment to the DoW
– Done
• Task mapping to 6 EGI related proposals
– Subject of this meeting, will result in report (MNA1.5) “Status of
transition to a sustainable infrastructure”
EGEE-III INFSO-RI-222667
AA meeting Feb 2010
1
Recommendation 2: NA1/4/SA1
Enabling Grids for E-sciencE
•
The reviewers recommend that any opportunity to simplify the operating environment be
exploited during the final year in order to increase the dependability of the (very complex)
infrastructure and reach a very high level of stability before the hand-over to EGI. Candidates
for simplification include: fewer software versions, more stringent rules for adoption of new
software by sites, fewer VOs by regrouping and pruning.
•
NA4:
– Many unregistered VOs used for local testing/training/users
  VOs MUST have only local scope not across the whole grid
  Provide appropriate catchall VOs to avoid local VOs
– Fixing this within EGEE not possible, but can get process in place for EGI
 Force VOs to reregister each year to show they are active and details correct
  Make VO manager more responsible for the activity of the VO
– Immediately: 57 inactive VOs emailed. 46 responded. 6 can be closed
SA1: The oldest support client and server versions are now clearly listed bounding
the necessary support effort. Documented procedures are in place for:
– Oldest supported client/server version determination and policy
– Service retirement procedure
– Dealing with obsolete service instances
– High priority service upgrades
This will provider for a ‘younger’ infrastructure with reduced support burdens.
Work is still needed for services to provide their versions in the information
system to automate many of these procedures.
•
•
EGEE-III INFSO-RI-222667
AA meeting Feb 2010
2
Recommendation 3: NA1
Enabling Grids for E-sciencE
•
Propose to EGI_DS that the “virtuous circle” paradigm could be advantageously mapped to the
upcoming EGI/NGI era. This paradigm encompasses the full spectrum of activities which
contribute to a grid infrastructure, and in future these activities will be shared amongst a
number of actors (EGI, NGIs, SSCs, middleware developers) rather than coming under a single
project. Mapping the “virtuous circle” to all these actors would facilitate the identification of
requisite interfaces and processes essential for a smooth transition and uninterrupted
operation.
•
EGEE will pass this recommendation to EGI_DS during EGEE09 and as part of its input to
EGI_DS deliverable D5.4 (virtuous cycle figure included in document)
Figures from the EGI-Inspire proposal
EGEE-III INFSO-RI-222667
AA meeting Feb 2010
3
Recommendation 4: NA2
Enabling Grids for E-sciencE
• Include a strong message of continuity in the user awareness
campaign in the run-up to the EGI transition – for example, no
change to your VOs, no change to your CPU and storage
resources, application software, documentation etc.
• NA2 will ensure that a strong message of continuity runs through
the user awareness campaign during the transition to EGI.
• This message will be distributed via the existing EGEE channels.
– Directors Letters
– Quarterly Newsletters
– EGEE web portal eg FAQ section, EGI section
– Press releases to the trade press eg EGEE User Forum
– Articles in publications such as International Science Grid This
Week, HPCWire, Public Service Review Science & Technology,
eStrategies Projects, ICT Results and others.
– EGEE/EGI transition brochure (EGEE’09 & EGEE UF 10)
EGEE-III INFSO-RI-222667
AA meeting Feb 2010
4
Recommendation 5: NA1
Enabling Grids for E-sciencE
•
Review the resource figures highlighted in 3b below and, if necessary, take corrective
measures:
The reviewers received provisional cost and PM figures for the review period. Overall, the consumption of resources is on
track, with 55 per cent of the committed PM and 43 per cent of the budgeted costs being consumed in this period. In fact,
this demonstrates an economic efficiency, with additional PM being delivered within the budget. However, there is a
small number of individual partners where the reported figures differ markedly from a linear rate of consumption of
resources. Partner 9 has contributed only 21 per cent of its budgeted PM but consumed 45 per cent of its budgeted
costs. Partner 13 has contributed 73 per cent of its budgeted PM and consumed 60 per cent of its budgeted costs.
Partner 14 has contributed 47 per cent of its budgeted PM but consumed 84 per cent of its budgeted costs. Partner 15
has contributed 63 per cent of its budgeted PM but consumed only 37 per cent of its budgeted costs. Explanations for the
deviations in the cases of partners 13 and 14 were presented during the review meeting. However, the project should
continue to monitor these and the other cases in order to understand the reasons for these figures, with the overall aim
of ensuring that all contracted PMs are delivered by the end of the project. Further, the project should also follow up
cases (such as here in partner 15) where it appears that the unit PM cost is greatly lower than originally described in the
DoW. Where the explanation is that the work is being done by persons of a lower qualification, the project should make
sure that the performance of the work done and the deliverables are of due quality.
•
The EGEE project office will continue to monitor closely the costs and effort consumption by
all partners via the intermediate costs claims
– Meetings between the financial officer and the individual partners concerned were scheduled for
EGEE09
– Follow-up continuing via ICCs
EGEE-III INFSO-RI-222667
AA meeting Feb 2010
5
Recommendation 6: NA2
Enabling Grids for E-sciencE
•
Reach out to a wider community, beyond the early adopters of grid technology, encompassing relevant national and
European public and/or private research initiatives implementing the Strategic Research Agendas (SRAs) through the
European Technology Platforms (ETPs), Joint Technology Initiatives (JTIs) and European Institute of Innovation and
Technology (EIT). The project should take advantage of any opportunity that arises. In this respect, use cases and
application notes will be of particular interest.
•
High profile at several events in the grid calendar, some of which have attracted thousands of delegates from a wide
range of communities.
–
–
–
–
–
–
–
–
–
–
–
–
–
•
•
+ 350 events organised
or attended by regional
partners
Publicised links between EGEE-III and the RESERVOIR project, NESSI strategic project and part of the NESSI
architecture framework called NEXOF
–
•
GridFest, CERN
SC’08 and SC’09 in the US
“Cybercafe de grilles” at the Ville Européenne des Sciences in Paris
ICT 2008 in Lyon
Research Connections 09, Prague
International SuperComputing 09, Hamburg
eChallenges, Istanbul
British Science Association Science Communication Conference, London
NCeSS International Conference on e-Social Science, Cologne
HealthGrid Conference, Berlin
GEO European Projects Workshop, Istanbul
SciTech Europe, Brussels
eScience and Industry Workshop, Santiago, Chile etc etc
iSGTW and HPCwire articles, joint press release
JTI on GMES (Global Monitoring for the Environment and Security) now instead a public programme involving the EU,
ESA and Member States. (Target SWIFT newsletter and Enterprise & Industry Magazine)
Produced brochure on European e-Infrastructures and use cases with DANTE and DG-INFSO aimed at the European
Institute of Innovation and Technology Management Board and the Knowledge and Innovation Centres.
EGEE-III INFSO-RI-222667
AA meeting Feb 2010
6
Recommendation 7: NA2
Enabling Grids for E-sciencE
•
In year 2, conduct close-interviews (debriefings) with contacts already made within the
business outreach programme with a view to collecting useful information about market
barriers and opportunities. Where possible, information on the uptake of gLite based
commercial offerings should be collected during these interviews. Do not expend any
significant effort on finding additional contacts
•
Original Response:
–
•
•
Business Forum and Business Task Force will consolidate the organization of one-to-one site
visits to a compiled list of select business partners, in preference to the ones with significant
involvement from the EGEE Business Associates programme, current and past gLite Business
Applications and other formal relationships built through the Business Forum.
Efforts during the remaining period will be focused on interview preparations, organization and
execution of interviews. Activities and goals will be to prepare a template of required information
covering areas such as the EGEE program for business, market barriers and opportunities,
advices for the future, which will feed into the final deliverable reports of EGEE-III.
Actions
• Questions template prepared
• 15 Interviews conducted/collated
• Analysis underway
• To be included in Commercial Exploitation Plan
• Make up final Business Forum Newsletter
EGEE-III INFSO-RI-222667
AA meeting Feb 2010
7
Recommendation 8 - Trainers
Enabling Grids for E-sciencE
The activity should ensure during year 2 that as many trainers as possible become accredited,
particularly in countries/regions where the current numbers of trainers are low. The distribution of
accredited trainers (between induction, application development, administration, possibly others)
should be reviewed to ensure a suitable coverage of training needs.
•
Actions:
– Prioritised task :
 Increase trainers in large countries
 Ensure trainers in small countries
– Total number – 134 (30 since review)
– Registry developed which captures experience
– Contacted 15 NGIs (with no NA3 training activity)
 So far responses from 4
 Trainers provided for 3 NGIs
EGEE-III INFSO-RI-222667
AA meeting Feb 2010
8
Rec. #9
Enabling Grids for E-sciencE
• NA4:
• “Encourage new user communities to make use of the seed
resources that have been set up but do not add further
resources considering the timescale of the EGEE-III
project.…”
– NA4 plans to publicize better all of the support services offered by NA4
to the user community, including the availability of the seed resources
and help making use of them. We believe that the existing seed
resources are already sufficient and have no plans to increase the
computing resources available.
• Current users of seed resources:
–
–
–
–
VO: na4.vo.eu-egee.org (3/2009)
VO: eticsproject.eu (4/2009)
VO: climate-g.vo.eu-egee.org (8/2009)
VO: digitalmedia.vo.eu-egee.org (11/2009)
EGEE-III INFSO-RI-222667
AA meeting Feb 2010
9
Recommendation 10
Enabling Grids for E-sciencE
10. SA1: Review whether the currently chosen availability and
reliability metrics are the most representative of a grid
infrastructure like EGEE from the user perspective and evaluate
other solutions and ways of implementing them.
– For example, an average figure of 85 per cent was presented during the review,
but it is very improbable that any user will experience this. Rather, they are
likely to experience either a much higher or a much lower figure, due to the
distribution of site figures.
– Consider whether presenting the figures on a VO basis would give a better user
picture. Further, the differentiation between availability and reliability will be
invalid if sites in fact have significant planned unavailability (“scheduled
downtime”) but this is unreported. Check whether some of the bottom 20 per
cent of sites are inoperative and, if necessary, remove them from the
infrastructure.
EGEE-III INFSO-RI-222667
AA meeting Feb 2010
10
Recommendation 10
Enabling Grids for E-sciencE
• we will coach a VO, selected by NA4, to run a subset of the
existing infrastructure metrics using its credentials, and to add
its own set of VO-specific metrics
– Biomed Nagios server set up at LAL by Michel Jouvin
• Availability report for 10 most active Vos using ops probes
– Done since October:
http://gvdev.cern.ch/GRIDVIEW/downloads/Reports/200912/egee/
– Assessment: not very useful, availability very similar to ops; better
on demand, for VOs who wish to use the results and follow up with
sites
• Simplification of intervention procedures:
– Clarify/simplify the downtime declaration rules
 Implementation corrected, dissemination campaign
 Monthly follow up of reasons for unscheduled interventions
– Ranking done by reliability
EGEE-III INFSO-RI-222667
AA meeting Feb 2010
11
Recommendation 11
Enabling Grids for E-sciencE
11. SA1: Effectiveness and enforceability of SLAs should
be reviewed together with the cost/benefit of the whole
SLA approach in the very particular grid-resource
environment.
• site / ROC support
– New metrics defined: average assignment and solution time,
plus prototype implementation
• SLA included in site certification procedure
• Sites below 50% availability over 3 consecutive months
excluded from the infrastructure:
–
–
–
–
September: CN-BEIJING-PKU (AP) Taiwan-IPAS-LCG2 (AP)
October: none
November: none
December: PH-ASTI-BUHAWI (AP)
EGEE-III INFSO-RI-222667
AA meeting Feb 2010
12
New site/ROC support metrics
Enabling Grids for E-sciencE
EGEE-III INFSO-RI-222667
AA meeting Feb 2010
13
Recommendation 12
Enabling Grids for E-sciencE
• IPv6: complete the development, testing and
certification of IPv6 support.
• gLite IPv6 compliant components now available (not
services, only gLite stack part):
– LFC, DPM-SE, BDII (python), WMS/WMproxy. CREAM CE,
BLAH, LCG-UTILS, FTS
– Assement of IPv6 gLite source code compliance:
 Sept. 2007 67%, March 2009 80%, almost 90% January 2010
• Integration into EGEE validation process
– 2 dual-stack IPv4-IPv6 EGEE sites in the validation testbed
– Only IPv4 feature regression testing
EGEE-III INFSO-RI-222667
AA meeting Feb 2010
14
Recommendation 13
Enabling Grids for E-sciencE
• Continue to investigate the user community requirements for
advanced network services and provide a clear position regarding
the possibility of deploying solutions coming from GEANT-2's
AutoBAHN and other EC-funded developments, e.g. from
FEDERICA.
• The SA2 advanced network services team has made an
extensive study on advanced network services
available in Europe:
– AMPS, AutoBHAN, GLIF, Phosphorus
A review of advanced network services in USA has also
been made for Internet2, National Lambda Rail and
ESNET
This work is documented in the milestone MSA2.4 (due
to February).
EGEE-III INFSO-RI-222667
AA meeting Feb 2010
15
Recommendation 14
Enabling Grids for E-sciencE
• Extend participation in standardisation activities within OGF to the
Network Service Interface (NSI) Working Group. Closely monitor
relevant developments in telecommunications forums such as
IETF, ITU, and OIF.
• SA2 has only followed the Network Service Interface
Working Group (NSI-WG) and does not take an active
part in the work because it is very time consuming and
we had no manpower in reserve.
EGEE-III INFSO-RI-222667
AA meeting Feb 2010
16
JRA1 (and SA3): Recomm. 15
Enabling Grids for E-sciencE
•
“Appropriate software engineering processes, despite progress in this
area, have to be defined and enforced consistently and vigorously.
They should include adherence to appropriate documentation, code
commenting, interface documentation, and Intellectual Property
management standards.”
• Documentation
•
•
•
•
Developer ‘s Guide updated and advertised at All-Hands Meeting
MSA3.4.2 (“the release process”) completed
Document review requested at AHM
Many sessions on the process at AHM
• Product Team process
•
•
Process prepared for the EGI era
Migration from the previous model in progress
• Copyright and License
•
•
Situation bad, only ~10% of the ~5500 files are ok
Specific metrics defined to monitor the progress
• Release scheduling (see R#16)
EGEE-III INFSO-RI-222667
AA meeting Feb 2010
17
JRA1 (and SA3): Recomm. 16
Enabling Grids for E-sciencE
•
“The focus of activity in year 2 should be on producing a functionally
complete and stable version of gLite for at least one reference platform with a
significant reduction of outstanding change requests.”
•
Smarter change management:
• Purpose is to ensure most efficient use of resources
• Most important functional or stability issues are systematically identified
and fixed with priority
• Bug triage and cleanup
• Change classification and scheduling
• Several metrics defined to monitor the process
• Availability of a release schedule until the end of the project
Timetabling of completion of SL5 support
• Still missing WMS and FTS
• Scheduled before the end of the project
Slow progress on other platforms
• Lower-priority than other tasks
• Working on source packages
•
•
•
Create a community of contributors
EGEE-III INFSO-RI-222667
AA meeting Feb 2010
18
Recommendation #17
Enabling Grids for E-sciencE
•
Following the successful introduction of interworking with ARC, the gLite
interoperability should be extended to UNICORE. This is especially important in
view of the transition to EGI/NGI.
• Our initial response was:
• Find a backing scientific community
• Understand their requirements
• Merge these into the workplan where possible
• We have not identified requirements on the middleware
• Fusion (EUFORIA)
• Lattice QCD
• All appear to implement what's necessary at the application level
• Some completed gLite changes will help
• e.g. issuance of SAML assertions instead of X.509 ACs by
VOMS (already available)
• Fully addressed in EMI
EGEE-III INFSO-RI-222667
AA meeting Feb 2010
19
•
JRA1, SA3 (and SA1): Recomm.
18
“Take steps to remove the inconsistencies between the GGUS ticketing system and the
Enabling Grids for E-sciencE
Savannah bug tracking system. In particular, users should not have direct access into
Savannah: a clean process between GGUS and Savannah has to be defined. Use of
Savannah by JRA1 should be kept separate from the user tickets. The classification of
tickets (especially the critical state of the tickets) should be re-examined and a process
of reclassifying tickets within GGUS and Savannah should be put in place.”
• Strategy is to make all user-oriented metrics available
through the information in GGUS alone
•
Modify process so GGUS logs the beginning and end of any issue,
regardless of whether it is passed to another tracker
• GGUS the single entry point for users
– Savannah reserved for internal use
– Will be one of many project-specific trackers referenced by GGUS
• Status
– Necessary process changes have been documented and are already in
force
– A few technical changes, to be done with the GGUS team, still required
to “close the loop”
• Metrics defined to monitor the process
EGEE-III INFSO-RI-222667
AA meeting Feb 2010
20
Recommendation 19: TMB
Enabling Grids for E-sciencE
•
•
•
Take steps in year 2 to understand why MPI deployment is so limited,
given that several user communities have a latent requirement, and
take appropriate actions.
An MPI WG has been re-initiated by the Technical Management Board
and will work with the activities, user communities, middleware
developers and sites to ensure the deployment and support of MPI is
improved during the second year.
MPI-WG: Look at long-term issues needing software changes:
– Active session at EGEE’09 (REF) established the need for an MPI task force
– Better support for multi-threaded applications, controlling the placement of
processes, etc.
– MPI fixes from WG’s activity earlier in the year have moved out into production
 Release SL5(3.2.0 Bundle 03) & SL4(TBC): fix config and batch integration issues
•
MPI-TF: Look at short-term improvement of MPI in EGEE:
–
–
–
–
Work to identify common configuration problem and fixes
SAM tests enabled identifying ~85 sites publishing MPI tags  many failing
42 issues put in GGUS after F2F meeting, last week, 20 already solved/verify
Once 66% sites are OK (nearly there) tests will be made critical
 Ops teams solve in standard way using updated documentation and MPI group in GGUS.
EGEE-III INFSO-RI-222667
AA meeting Feb 2010
21