Grid Initiatives for e-Science virtual communities in
Europe and Latin America
WP3 – User Communities Support –
Achievements
Diego Scardaci (WP3 Manager)
(INFN Catania- Italy)
Final Project Review
Brussels, 11/10/2012
www.gisela-grid.eu
Outline
• Work Package Objectives
• Achievements
• Answers to Reviewers
• Conclusions
www.gisela-grid.eu
GISELA - Final Project Review - Brussels - 11/10/2012
2
WP3 Objective and Structure
• The main objective of WP3 was to provide specialised
support and training to all users of the GISELA eInfrastructure
• The GISELA Science Gateway has been a strategic
development of WP3 over Y2 to attract more users (e.g.
from Regional Communities) providing them with friendly
interface, hiding grid complexity
• WP3 Structure
• TWP3.1 – Coordination of the Activity (Led by INFN)
• TWP3.2 – VRC Support (Led by CLARA)
• TWP3.2 – Validation, Documentation and Training (Led by CIEMAT)
www.gisela-grid.eu
GISELA - Final Project Review - Brussels - 11/10/2012
3
TWP3.1 – Objectives & Achievements (1/3)
• Monitoring the work carried out by each VRC supported by
GISELA
‒ VRCs were continuously monitored during the whole project
‒ The GISELA Science Gateway (GSG) has been designed in a VRCdriven format to better evaluate the VRC work
• To evaluate requests from new applications interested in being
deployed on the GISELA production infrastructure
‒ A public survey to propose new applications has been available from the
first phase of projects (currently integrated on the VRC-driven GSG,
https://gisela-gw.ct.infn.it/integre-una-nueva-aplicacion-al-sg)
‒ Applications have been also identified through direct contacts with
CLARA and Latin American NRENs
www.gisela-grid.eu
GISELA - Final Project Review - Brussels - 11/10/2012
4
TWP3.1 – Objectives & Achievements (2/3)
• To setup and maintain a database with all sort of information
about the supported applications and VRCs
‒ Available at https://gisela-gw.ct.infn.it/application-database
• To keep track of the usage of the production infrastructure by
the supported VRCs as well as all scientific publications
generated by their members
‒ VRCs have been continuously monitored, in collaboration with CLARA
and NRENs, to identify and support applications
‒ All GISELA scientific publications are listed in the GISELA web-site
(http://www.gisela-grid.eu/index.php?option=com_content&view=article&id=43&Itemid=41)
‒ VRCs related publications have been listed in Deliverable D3.4
www.gisela-grid.eu
GISELA - Final Project Review - Brussels - 11/10/2012
5
TWP3.1 – Objectives & Achievements (3/3)
• To work in close collaboration with WP2 providing inputs to
dissemination pieces and websites
‒ The GSG layout has been designed in collaboration with WP2
‒ GSG contents management has been jointly done by WP2 and WP3
‒ WP3 provided WP2 with several inputs to prepare bulletins, short
articles, etc.
• To evaluate the technical and scientific impact of the supported
VRCs
‒ CLARA & NRENs have performed analysis of VRCs’ technical and
scientific impact (which led to the restyling of the GSG layout)
‒ We selected applications according to this analysis
www.gisela-grid.eu
GISELA - Final Project Review - Brussels - 11/10/2012
6
TWP3.2 – Objectives & Achievements (1/2)
• Identify potential new VRCs in Latin America working in close
collaboration with the CLARA partner. Group the applications’
users/developers into VRCs according to their commonalities.
Propose the creation of new VRCs
‒ A Task Force, made of WP3, CLARA and NRENs, identified key VRCs
and applications through both a dedicated survey and direct contacts
‒ As a consequence
 5 VRC-specific dedicated sections have been created in the GSG
 89 applications are supported out of which 59 are in production status
 16 applications are integrated in the GSG
www.gisela-grid.eu
GISELA - Final Project Review - Brussels - 11/10/2012
7
TWP3.2 – Objectives & Achievements (2/2)
• Work in close collaboration with WP4 and WP6 to install all
libraries, packages and/or services required by a particular VRC
in the CE nodes of the GISELA production infrastructure
‒ Almost all the gridified applications needed custom libraries and
packages that have been installed at the Resource Centres by WP4
‒ A new JSAGA adaptor has been created by WP6 to integrate OurGrid
resources in the GSG
‒ The adaptor for OurGrid will be integrated in the official JSAGA release
as a GISELA contribution
• Answer tickets related to the Applications/Users support and
issued by developers and users via the central support ticket
system
‒ WP3 answered all tickets related to the Applications/Users support (see
the WP4 presentation for ticket statistics)
www.gisela-grid.eu
GISELA - Final Project Review - Brussels - 11/10/2012
8
TWP3.3 – Objectives & Achievements (1/2)
• Coordinate the GISELA training activities, profiting from the
person-power previously trained with EELA and EELA-2
frameworks
‒ 14 training events (7 in Y2) have been organised and held during
GISELA, compared to the ≥ 6 pledged in the DoW
‒ Tutors have been selected from the list of persons trained in EELA and
EELA-2 projects
• Provide specific training to application developers
‒ Specific training events have been organised to port new applications on
the Grid Infrastructure
 5 in the whole duration of the project
 3 of them expressly devoted to the integration of applications into the GSG
during Y2
www.gisela-grid.eu
GISELA - Final Project Review - Brussels - 11/10/2012
9
TWP3.3 – Objectives & Achievements (2/2)
• Validate the Grid services deployed by WP4 and WP6
‒ Grid services deployed by WP4 and WP6 have been fully tested
• Create and maintain a repository of self-training materials
‒ The GISELA self-training materials repository contains 79 documents
(User guides, Installation guidelines, FAQs, etc.) available at:
http://documents.giselagrid.eu/collection/Project%20Training%20Material?ln=en
www.gisela-grid.eu
GISELA - Final Project Review - Brussels - 11/10/2012
10
GISELA Science Gateway
(http://gisela-gw.ct.infn.it)
• The GSG is a Web-based environment that allows users to fully exploit
the e-Infrastructure - computing (jobs) and storage (data) services - in
a very easy way, hiding the complexity of underlying Grid Middleware
Multi Languages
support
www.gisela-grid.eu
GISELA - Final Project Review - Brussels - 11/10/2012
11
GISELA Science Gateway
Main Functionalities (1/2)
• Multi-language support (as of today, Spanish and English)
• Authentication based on Identity Federations
• An open Identity Federation named GrIDP (http://gridp.ct.infn.it/) has
been created to serve all LA countries where an Identity Federation is
not yet operational
• Authorisation managed via LDAP registry
• Multi-middleware interoperability based on the SAGA OGF
standard (JSAGA API) allows to submit jobs and move data on
GISELA, using both gLite and OurGrid
• The GSG is compliant with the EGI VO Portal and Grid Security
Traceability & Logging Policies
• A new generation application registry allows users to directly
submit selected applications through the GISELA e-Infrastructure
• Training Material is available
www.gisela-grid.eu
GISELA - Final Project Review - Brussels - 11/10/2012
12
GISELA Science Gateway
Main Functionalities (2/2)
• VRC-driven and oriented: One specific section for each
supported VRC containing applications information
• GISELA Virtual Room: to organise virtual seminars or meetings
• Success Stories
• A section to propose new applications to be integrated in the
GSG
• Access through credentials provided by the most famous social
networks (e.g. Facebook) and a community page on Facebook
https://www.facebook.com/pages/GISELAScienceGatewayCommunity/347171765358049
www.gisela-grid.eu
GISELA - Final Project Review - Brussels - 11/10/2012
13
GISELA Science Gateway
A new concept of Application Database
• A new generation application registry allows users to directly
submit selected applications through the GISELA eInfrastructure
Direct links to applications run pages
www.gisela-grid.eu
GISELA - Final Project Review - Brussels - 11/10/2012
14
GISELA Science Gateway
Federated Authentication
•
•
Authentication based on username/password pair provided by an Identity Provider,
hiding digital certificates
Official Identity Federations exist in 5 GISELA countries and the GSG supports all of
them, on top of the “catchall” one and the eduGAIN inter-federation
Logo
www.gisela-grid.eu
Name
Country
Status
CAFe
Brazil
Integrated
COFRE
Chile
Integrated
IDEM-GARR
Italy
Integrated
RCTSaai
Portugal
Integrated
SIR
Spain
Integrated
GrIDP
«catch-all»
Integrated
eduGAIN
Inter-federation
Integrated
GISELA - Final Project Review - Brussels - 11/10/2012
15
Mexican Identity Federation
• GISELA fostered the creation of Identity Federations in LA and
selected Mexico as pilot country due to
– The high number of Mexican GSG users
– It is where the VO e-Infrastructure services will be run after GISELA
• In July 2012, a specific training secondment has been held in Catania
(Italy), in collaboration with the EPIKH Project, for a Mexican team
identified by CLARA, CUDI and UNAM
• By end of November 2012, the Mexican federation will comprise UNAM,
Universidad de Colima (UCOL), Dirección General de Institutos
Tecnológicos (DGIT) and CUDI for a total of about 500,000 students
and researchers!
www.gisela-grid.eu
GISELA - Final Project Review - Brussels - 11/10/2012
16
VRCs supported in the GSG
• WP3 concentrated its effort towards
– Creation of new VRCs sections in the GISELA SG
– Increase of the number of integrated applications
• At the end of the Project, the GSG includes 5 VRC sections with 16
applications
– Cultural Heritage (2)
– Earth Science (1)
– Industry (1)
– Life Sciences (7)
– Statistics and Mathematics (5)
• The LA Task Force defined a short-term plan to increase the number of
applications
– 2 applications have been integrated in the GSG after the end of GISELA
– 9 applications have been already selected to be integrated (an integration
training event will be held in Colombia by November)
www.gisela-grid.eu
GISELA - Final Project Review - Brussels - 11/10/2012
17
GISELA Science Gateway
Sustainability (1/2)
• CLARA and NRENs selected a group of LA people to support
the GISELA SG after the project end
• WP3 coached them thanks to a well-targeted training plan
• This group, called “Latin American Task Force - LA Task Force”
is currently able to manage all the aspects of the GISELA SG
–
–
–
–
–
–
Operational
Content management
Identity federations management
Integrate new applications on the GSG
Preparing GSG end users training materials
Organising GSG end users training events
The LA Task Force is operational!
www.gisela-grid.eu
GISELA - Final Project Review - Brussels - 11/10/2012
18
GISELA Science Gateway
Sustainability (2/2)
• INFN commitments
– Short-term
 The GSG operation is guaranteed by INFN-CATANIA until CLARA and
NRENs are ready to relocate it in Latin America
 INFN will continue to support the LA Task Force in the applications
integration process
– Long-term
 a MoU (currently in preparation) between INFN and CLARA will guarantee
the support and maintenance of the core GSG software
The creation of the LA Task Force and the INFN
commitment avoided any disruption on the GSG
service at the end of GISELA
www.gisela-grid.eu
GISELA - Final Project Review - Brussels - 11/10/2012
19
Training
• From the beginning of GISELA, the Training Plan (TP) has been
designed in close cooperation with CLARA TT in response to
VRC actual needs
• The outcomes of the TP contributed to the objective of longterm sustainability of the GISELA e-Infrastructure, as they
helped to create and consolidate a group of LA experts capable
of providing user support services and operating Grid Resource
Centers
• The TP also helped to consolidate a portfolio of applications
integrated in the GISELA Science Gateway
www.gisela-grid.eu
GISELA - Final Project Review - Brussels - 11/10/2012
20
Training Events organised during the project
Date
Event
Place
Special GISELA/EPIKH training support action: Adaptation and integration of
Catania - Italy
new applications into the GSG
Special GISELA/EPIKH training support action: Deploy an Identity Federation in
04th - 27th July 2012
Catania - Italy
Mexico
Latin America 2012 - Joint CHAIN/GISELA/EPIKH School for Application
18th - 26th June 2012
Mexico City - Mexico
Porting to Science Gateway
Latin America "special" 2012 - Joint CHAIN/GISELA School for Application
04th - 15th June 2012
Bogota - Colombia
Porting to Science Gateway
Latin America "special" 2012 - Joint CHAIN/GISELA School for Grid Site
Bucaramanga 28th May - 01st June 2012
Administrators
Colombia
04th - 27th July 2012
21st - 22nd May 2012
16th - 17th January 2012
CICESE Ensenada B.
C., Mexico
Grid Computing Seminar for end-users
Training of CEDIA System Administrators
Quito, Ecuador
Hermosillo, Sonora,
Mexico
13th - 17th June 2011
Grid Computing for Science & Technology Applications
10th December 2010
Joint CHAIN/GISELA/EPIKH Workshop
Valparaiso, Chile
29th November - 09th
December 2010
Joint CHAIN/GISELA/EPIKH School for Application Porting
Valparaiso, Chile
22nd - 26th November
2010
Joint GISELA/EPIKH School for Grid Site Administrators
Valparaiso, Chile
26th November 2010
Joint GISELA/EPIKH Workshop
Mexico City, Mexico
Joint GISELA/EPIKH School for Application Porting
Mexico City, Mexico
Joint GISELA/EPIKH School for Grid Site Administrators
Mexico City, Mexico
15th - 25th November
2010
th
08 - 12th November
2010
www.gisela-grid.eu
GISELA - Final Project Review - Brussels - 11/10/2012
21
Post-GISELA Training Events
Date
Event
Place
4th October 2012
Sesiones de Entrenamiento Básico para Usar
Aplicaciones en el Science Gateway (1st session)
GSG Virtual
Room
18th October 2012
Sesiones de Entrenamiento Básico para Usar
Aplicaciones en el Science Gateway (2nd session)
GSG Virtual
Room
1st November 2012
Sesiones de Entrenamiento Básico para Usar
Aplicaciones en el Science Gateway (3rd session)
GSG Virtual
Room
26th - 30th
November 2012
Tutorial for Applications Porting
3rd - 7th December
Tutorial for Resource Center Administrastion
2012
www.gisela-grid.eu
Bucamaranga Colombia
Buenos Aires Argentina
GISELA - Final Project Review - Brussels - 11/10/2012
22
Virtual Meetings (1/2)
• Dissemination and training activities promoted the usage of
the GISELA Science Gateway and were organised in
collaboration with WP2 and the CLARA TT
• All Virtual Meetings have been held using the virtual room
provided by INFN and integrated in the GSG
• Two kinds of WP3-related virtual meetings were organised
during the second year of the project
– Virtual Seminars: to teach developers and members of LA Task Force
how to integrate new applications in the GISELA Science Gateway
– Virtual Days: to attract potential new users and research communities
by presenting success stories and showing how e-Infrastructures and
Advanced Computing Services help to reach scientific goals in an
easier way
www.gisela-grid.eu
GISELA - Final Project Review - Brussels - 11/10/2012
23
Virtual Meetings (2/2)
www.gisela-grid.eu
GISELA - Final Project Review - Brussels - 11/10/2012
24
Quality Metrics
Quality metric
R1 Number of Active Users in the catch-all VO
(prod.vo.eu-eela.eu)
R2 Number of Active Users in the VRC-driven Science
Gateway
R3 Number of Applications in production status on the
e-Infrastructure
R4 Number of Applications integrated in VRC-driven
Science Gateways
R5 Number of specialized Science Gateway for VRC
D1 Number of supported VRCs
D2 Number of application’s groups from Latin America
D3 Number of training events
D4 Number of self-training material
D5 Number of papers published (WP3 related)
www.gisela-grid.eu
Current
status
Expected
outcome
108
100
174
100
59
40
16
15
5
6
45
14
79
21
2
≥3
≥ 36
≥6
≥ 40
≥ 20
GISELA - Final Project Review - Brussels - 11/10/2012
25
Reviewers Recommendations (1/2)
• Increase user pool size
‒ The new model based on the GSG allowed us to attract several new
users (174 as of today)
• Increase applications number
‒ 59 applications are in production status
• Establish a process for the user/partner feedback collection
‒ The new layout of the GSG, developed in collaboration with CLARA-TT
and WP2, derives from user opinions and needs
‒ The GISELA Science Gateway pages can be commented by users
• Support users through application development and deployment
‒ Virtual Seminars have been organised to teach developers and
members of the LA Task Force how to integrate new applications in the
GISELA Science Gateway
‒ The LA Task Force has been built and trained to continue this support
action after the project lifetime
www.gisela-grid.eu
GISELA - Final Project Review - Brussels - 11/10/2012
26
Reviewers Recommendations (2/2)
• Organisation of VRC-tailored tutorials, workshops, Gridification weeks
‒ Several training events have been organised to deploy new applications
on the GISELA SG during the second project year
‒ All of the applications integrated in the GSG during the training
events were proposed by the users through VRC-dedicated surveys
• Adapt current applications to the Science Gateway
‒ 16 applications have been integrated in the GSG
• Participate in the development of new services, as requested by the
VRCs (validate WP6 development)
‒ The JSAGA adaptor for OurGrid has been developed by WP6 and
tested by WP3
‒ The new adaptor has been integrated in the GSG and will be part of the
JSAGA release as a GISELA contribution
www.gisela-grid.eu
GISELA - Final Project Review - Brussels - 11/10/2012
27
WP3: e.o. 2012 planning
Latin American Task Force
August September
• ROC Handover
• Platform Testing
• SG Application Testing
• User Support (Tutorial)
• SG Community Marketing
October November
WP3
www.gisela-grid.eu
• ROC Operation
• New SG Application
• User Support (Tutorial)
• SG Community Marketing
December January
• ROC Operation
• New SG Application
• User Support
• End user training
• SG Community Marketing
GISELA - Final Project Review - Brussels - 11/10/2012
28
Conclusions (1/2)
• WP3 decided to follow a new strategy, based on the SG model,
to reach more users and thus contribute to the sustainability of
the e-Infrastructure
• WP3 built and deployed the VRC-driven GISELA Science
Gateway
– 174 users
– Users can submit jobs through a normal web browser
– Authentication based on Identity Federations: CAFe (Brazil), Cofre
(Chile), IDEM (Italy), RCTSaai (Portugal) and SIR (Spain) federations,
as well as eduGAIN inter-federation, currently integrated.
– The GSG is independent of the underlying middleware (SAGA OGF
Standard adopted)
– 5 VRC Specialised SGs currently available: Cultural Heritage, Earth
Sciences, Industry, Life Sciences and Statistics and Mathematics
– 16 Applications currently integrated
www.gisela-grid.eu
GISELA - Final Project Review - Brussels - 11/10/2012
29
Conclusions (2/2)
• WP3 promoted the creation of Identity Federations in Latin America in
collaboration with CLARA and NRENs
• Several training activities have been organised to create a LA Task
Force able to manage all the aspects of the GISELA SG
• Training material has been created and made available athttps://giselagw.ct.infn.it/documentacion-ayuda
• Strong dissemination activities have been organised in collaboration
with WP2 to promote the introduction of the Science Gateways
• Over the 2-year duration of GISELA, WP3 has been key to
– Supporting all kinds of users whether from large VRCs or from small
groups
– Anticipating needs to provide most helpful software tools, e.g. the GSG
– Taking care of the underlying Training Program tailored to all requests
www.gisela-grid.eu
GISELA - Final Project Review - Brussels - 11/10/2012
30