Università di Catania and INFN Catania - Italy
ALICE Collaboration and DataGrid Project
The GENIUS Grid Portal
Roberto Barbera(*)
(*)work
in collaboration with A. Falzone and A. Rodolico
Portals
& Portlets
2003, Edinburgh, 14.07.2003
CHEP 2000,
10.02.2000
Roberto Barbera
Università di Catania and INFN Catania - Italy
ALICE Collaboration and DataGrid Project
Outline
Introduction
Motivations
Current status
Real use made by the Experiments
Summary and conclusions
Portals & Portlets 2003, Edinburgh, 14.07.2003
Roberto
Roberto Barbera
Barbera
Università di Catania and INFN Catania - Italy
ALICE Collaboration and DataGrid Project
The Grid vision
Researchers perform their
activities regardless
geographical location,
interact with colleagues,
share and access data
The Grid: networked data
processing centres and
”middleware” software as
the “glue” of resources.
Scientific instruments and
experiments provide huge
amount of data
Roberto
Roberto Barbera
Barbera
Università di Catania and INFN Catania - Italy
ALICE Collaboration and DataGrid Project
The Grid metaphor
Mobile Access
G
R
I
D
Workstation
M
I
D
D
L
E
W
A
R
E
Supercomputer, PC-Cluster
Data-storage, Sensors, Experiments
Visualising
Internet, networks
Portals & Portlets 2003, Edinburgh, 14.07.2003
Roberto Barbera
Università di Catania and INFN Catania - Italy
ALICE Collaboration and DataGrid Project
DataGrid job submission work-flow
-in
it
UI
JDL
Replica
Catalogue
Input “sandbox”
DataSets info
id
-p
ro
xy
Output “sandbox”
SE &
St
at
u
s
gr
Expanded JDL
f
In
er
ok
Br
o
Job Submission
Service
Compute
Element
Publish
”
ox
db
an
Jo
b
“s
+
x”
bo
nd
Job Status
CE i
nfo
t
pu
ut
sa
t“
pu
In
Job Query
Job Submit Event
Globus RSL
Job Status
Logging &
Book-keeping
O
Resource
Broker
Author.
&Authen.
Information
Service
Storage
Element
Roberto Barbera
Università di Catania and INFN Catania - Italy
ALICE Collaboration and DataGrid Project
EDG m/w has been released but…
EDG software (Globus, UI, JDL, WP2, WP3, etc.) contains tens of
commands/switches which also have their own logical sequences.
Browsing Grid VO “directories” (users, RC’s, DB’s, etc.) requires LDAP
“speaking” and tomorrow could require SQL “speaking”.
“User gridification” is a tough task for a “rookie” ⇒ this does not fit with the
claim that “grids” are for everybody and that grid computing will be as easy
as surfing the Internet ?
Furthermore, all this holds for DataGrid. What will happen when other grids’
software (especially UI’s) will come up (PPDG, iVDGL, etc.) ? Will users
have to learn tens of “grid dialects” ?
Today “grid computing” is a rather complicated experience for unexperienced user only possible at selected machines (UI’s) ⇒ this does not
fit with the claim that one could do “grid computing” even from a PDA ?
Is there any way to set-up a “user-friendly” grid ?
Portals & Portlets 2003, Edinburgh, 14.07.2003
Roberto
Roberto Barbera
Barbera
Università di Catania and INFN Catania - Italy
ALICE Collaboration and DataGrid Project
A web portal: why and how ?
It can be accessed from everywhere and by
“everything” (desktop, laptop, PDA, WAP phone).
It can keep the same user interface to several
back-ends (grid “dialects” ⇔ command-line UI’s).
It must be redundantly “secure” at all levels: 1)
secure for web transactions, 2) secure for user
credentials, 3) secure for user authentication, 4)
secure at VO level.
All available grid services must be incorporated in
a logic way, just “one mouse click away”.
Its layout must be easily understandable and user
friendly.
Portals & Portlets 2003, Edinburgh, 14.07.2003
Roberto
Roberto Barbera
Barbera
Università di Catania and INFN Catania - Italy
ALICE Collaboration and DataGrid Project
A little bit of history
1998-1999
z
graphic interface to LSF (multi) clusters for ALICE
jobs
2000-2001
z
simple but effective web portal to submit ALICE
jobs to the grid using EnginFrame and GLOBUS
2002-now
z
official grant of the INFN Grid Project to
collaborate with NICE srl to integrate in a web
portal all services offered by the DataGrid middleware
Portals & Portlets 2003, Edinburgh, 14.07.2003
Roberto
Roberto Barbera
Barbera
Università di Catania and INFN Catania - Italy
ALICE Collaboration and DataGrid Project
GENIUS®
(Grid Enabled web eNvironment for
site Independent User job Submission)
[https://genius.ct.infn.it]
INFN/NICE collaboration
GENIUS web portal
Applications’
specific layer
DataGRID
architecture
ALICE
ATLAS
CMS
LHCb
Other apps
High level GRID middleware
GLOBUS
toolkit
Basic Services
OS & Net services
Portals & Portlets 2003, Edinburgh, 14.07.2003
Roberto Barbera
Università di Catania and INFN Catania - Italy
ALICE Collaboration and DataGrid Project
GENIUS:
how it works
https+java/xml+rfb
GENIUS
WEB Browser
EnginFrame
3-tier model
Local
WS
Apache
EDG
UI
EDG+GSI
the Grid
Roberto Barbera
Roberto Barbera
Università di Catania and INFN Catania - Italy
ALICE Collaboration and DataGrid Project
GENIUS show: the main page
Roberto Barbera
Università di Catania and INFN Catania - Italy
ALICE Collaboration and DataGrid Project
GENIUS show: OS authentication
Roberto Barbera
Università di Catania and INFN Catania - Italy
ALICE Collaboration and DataGrid Project
GENIUS show: file services
Roberto Barbera
Università di Catania and INFN Catania - Italy
ALICE Collaboration and DataGrid Project
GENIUS show: grid authentication
th
u
a
y
ox
r
P
y
M
!
n
io
t
a
c
i
t
en
Roberto Barbera
Università di Catania and INFN Catania - Italy
ALICE Collaboration and DataGrid Project
Grid authentication with MyProxy
EDG
UI
GENIUS
Server
(EDG UI)
WEB Browser
Local
WS
MyProxy
Server
grid-proxy-init
myproxy-init
ut
p
t
ou
s
d
i
gr
y
an
ce
i
v
er
el
d
te
g
yx
o
pr
my
exe
io
t
a
eg
n
cut
ion
the Grid
Roberto Barbera
Roberto Barbera
Università di Catania and INFN Catania - Italy
ALICE Collaboration and DataGrid Project
VO Selection
Roberto Barbera
Università di Catania and INFN Catania - Italy
ALICE Collaboration and DataGrid Project
RB Selection
Roberto Barbera
Università di Catania and INFN Catania - Italy
ALICE Collaboration and DataGrid Project
In collaboration with DATAMAT, Italy
Graphic job description (1/3)
Roberto
Roberto Barbera
Barbera
Università di Catania and INFN Catania - Italy
ALICE Collaboration and DataGrid Project
In collaboration with DATAMAT, Italy
Graphic job description (2/3)
Roberto
Roberto Barbera
Barbera
Università di Catania and INFN Catania - Italy
ALICE Collaboration and DataGrid Project
In collaboration with DATAMAT, Italy
Graphic job description (3/3)
Roberto
Roberto Barbera
Barbera
Università di Catania and INFN Catania - Italy
ALICE Collaboration and DataGrid Project
Single-jobs: CE selection
Roberto Barbera
Università di Catania and INFN Catania - Italy
ALICE Collaboration and DataGrid Project
Multi-jobs (1/7)
Roberto Barbera
Università di Catania and INFN Catania - Italy
ALICE Collaboration and DataGrid Project
Multi-jobs (2/7)
Roberto
Roberto Barbera
Barbera
Università di Catania and INFN Catania - Italy
ALICE Collaboration and DataGrid Project
Multi-jobs (3/7)
Roberto
Roberto Barbera
Barbera
Università di Catania and INFN Catania - Italy
ALICE Collaboration and DataGrid Project
Multi-jobs (4/7)
Roberto
Roberto Barbera
Barbera
Università di Catania and INFN Catania - Italy
ALICE Collaboration and DataGrid Project
Multi-jobs (5/7)
Roberto
Roberto Barbera
Barbera
Università di Catania and INFN Catania - Italy
ALICE Collaboration and DataGrid Project
Multi-jobs (6/7)
Roberto
Roberto Barbera
Barbera
Università di Catania and INFN Catania - Italy
ALICE Collaboration and DataGrid Project
Multi-jobs (7/7)
Roberto Barbera
Università di Catania and INFN Catania - Italy
ALICE Collaboration and DataGrid Project
GENIUS show: file publication
Roberto Barbera
Università di Catania and INFN Catania - Italy
ALICE Collaboration and DataGrid Project
GENIUS show: file publication
Roberto Barbera
Università di Catania and INFN Catania - Italy
ALICE Collaboration and DataGrid Project
GENIUS show: file publication
Roberto Barbera
Università di Catania and INFN Catania - Italy
ALICE Collaboration and DataGrid Project
GENIUS show: file publication
Roberto Barbera
Università di Catania and INFN Catania - Italy
ALICE Collaboration and DataGrid Project
GENIUS show: file publication
Roberto Barbera
Università di Catania and INFN Catania - Italy
ALICE Collaboration and DataGrid Project
GENIUS show: file browsing
Roberto Barbera
Università di Catania and INFN Catania - Italy
ALICE Collaboration and DataGrid Project
GENIUS show: file browsing
Roberto Barbera
Università di Catania and INFN Catania - Italy
ALICE Collaboration and DataGrid Project
GENIUS show: file browsing
Roberto Barbera
Università di Catania and INFN Catania - Italy
ALICE Collaboration and DataGrid Project
GENIUS show: file inspection
Roberto Barbera
Università di Catania and INFN Catania - Italy
ALICE Collaboration and DataGrid Project
GENIUS show: file replication
Roberto Barbera
Università di Catania and INFN Catania - Italy
ALICE Collaboration and DataGrid Project
GENIUS show: file replication
Roberto Barbera
Università di Catania and INFN Catania - Italy
ALICE Collaboration and DataGrid Project
GENIUS show: file replication
Roberto Barbera
Università di Catania and INFN Catania - Italy
ALICE Collaboration and DataGrid Project
GENIUS show: file downloading
Roberto Barbera
Università di Catania and INFN Catania - Italy
ALICE Collaboration and DataGrid Project
GENIUS show: file downloading
Roberto Barbera
Università di Catania and INFN Catania - Italy
ALICE Collaboration and DataGrid Project
GENIUS show: file downloading
Roberto Barbera
Università di Catania and INFN Catania - Italy
ALICE Collaboration and DataGrid Project
GENIUS show: file downloading
Roberto Barbera
Università di Catania and INFN Catania - Italy
ALICE Collaboration and DataGrid Project
GENIUS show: file downloading
Roberto Barbera
Università di Catania and INFN Catania - Italy
ALICE Collaboration and DataGrid Project
GENIUS show: file downloading
Roberto Barbera
Università di Catania and INFN Catania - Italy
ALICE Collaboration and DataGrid Project
Testbed browsing (1/2)
Roberto
Roberto Barbera
Barbera
Università di Catania and INFN Catania - Italy
ALICE Collaboration and DataGrid Project
Testbed browsing (2/2)
Roberto Barbera
Università di Catania and INFN Catania - Italy
ALICE Collaboration and DataGrid Project
GENIUS P2P services (new!)
Roberto Barbera
Università di Catania and INFN Catania - Italy
ALICE Collaboration and DataGrid Project
GENIUS and GridICE monitoring
Roberto Barbera
Università di Catania and INFN Catania - Italy
ALICE Collaboration and DataGrid Project
ALICE: interactive analysis
Roberto Barbera
Università di Catania and INFN Catania - Italy
ALICE Collaboration and DataGrid Project
(courtesy of L. Vaccarossa)
ATLAS: “GLUE” job submission
US
EU
Roberto
Roberto Barbera
Barbera
Università di Catania and INFN Catania - Italy
ALICE Collaboration and DataGrid Project
(courtesy of L. Vaccarossa)
ATLAS: interactive analysis
Roberto Barbera
Università di Catania and INFN Catania - Italy
ALICE Collaboration and DataGrid Project
(courtesy of M. Verlato)
CMS grid integration schema
Portals & Portlets 2003, Edinburgh, 14.07.2003
Roberto
Roberto Barbera
Barbera
Università di Catania and INFN Catania - Italy
ALICE Collaboration and DataGrid Project
(courtesy of M. Verlato)
CMS: GENIUS/Impala integration
Roberto
Roberto Barbera
Barbera
Università di Catania and INFN Catania - Italy
ALICE Collaboration and DataGrid Project
(courtesy of M. Verlato)
CMS: GENIUS/BOSS integration
Roberto
Roberto Barbera
Barbera
Università di Catania and INFN Catania - Italy
ALICE Collaboration and DataGrid Project
BaBar: GENIUS/AFS interface
Roberto
Roberto Barbera
Barbera
Università di Catania and INFN Catania - Italy
ALICE Collaboration and DataGrid Project
BaBar: multi-job submission
Roberto
Roberto Barbera
Barbera
Università di Catania and INFN Catania - Italy
ALICE Collaboration and DataGrid Project
BaBar: multi-job queue
Roberto
Roberto Barbera
Barbera
Università di Catania and INFN Catania - Italy
ALICE Collaboration and DataGrid Project
GENIUS show: Earth Ob. app.
Roberto Barbera
Università di Catania and INFN Catania - Italy
ALICE Collaboration and DataGrid Project
GENIUS show: Biomed. app.
Roberto Barbera
Università di Catania and INFN Catania - Italy
ALICE Collaboration and DataGrid Project
Present status
Current implementation of GENIUS includes more than 100 services.
Among them:
z secure web transactions, user authentication and authorization;
z grid authentication with MyProxy !
z remote interaction with the user’s and “grid” file system;
z interfaces for job submission/control, data management, VO servers, and
monitoring systems;
z several Resource Brokers and VO’s available;
z persistent (user level) book-keeping and spooler system;
z secure interactive analysis with TightVNC
z full support for “parallel” multi-jobs (“sequential” ones easy to
implement)
z web guided job description (in collaboration with DATAMAT)
z P2P services added with ChatEverywhere
z CVS available ! New and very easy installation procedure
Several installations of GENIUS in Italy and worldwide (Czech Rep., France,
Portugal, Russia, Spain, Taiwan, etc.)
GENIUS is the official Portal of the GriDis Project for dissemination
(http://web.datagrid.cnr.it/GriDis/jsp/index.jsp). Almost 80 people trained
during May-June
Roberto
Roberto Barbera
Barbera
Università di Catania and INFN Catania - Italy
ALICE Collaboration and DataGrid Project
Future devel. and activities
GENIUS is one of the grid portals officially included in the GGF Grid
Portal Manifesto and the GENIUS Team is involved into the GGF
GCE-RG working group.
PDA version available soon. New style sheets already defined
Possibility to create “skins” for different “look&feel”’s (in collaboration
with CNR)
Interface to EDG 2.0: work will start next week and continue during
July-August
Full integration with GriDis’ “grid on demand”
Integration of new specific VO services.
Graphic creation of new services
Integration of VOMS/VOX and DGAS tools for AAA
Close look at the technology of portlets as containers of grid services
(see how to match them with GENIUS services)
Integration of graphic job workflow creators. Discussions ongoing
between Catania, Datamat, FhG and CNR (first meeting in Rome on
July, 29)
Roberto Barbera
Università di Catania and INFN Catania - Italy
ALICE Collaboration and DataGrid Project
FAQ’s…and answers (1)
Q: I want to use GENIUS. Do I have to pay for it ?
A: No. GENIUS is “open source” and the underlying portal
framework EnginFrame is free for the education and research
communities.
Q: I want to use GENIUS. Do I need any software running on my
laptop ?
A: No client software needs to be installed apart from the web
browser. GENIUS can really be accessed from everywhere.
Q: Do I have to be afraid about cached password sent over the
web ?
A: Access passwords are securely “streamed” only when
needed and then destroyed. Only temporary sessions are
possible.
Q: Can new authentication methods implemented into GENIUS ?
A: Of course. Kerberos V is a good example. EnginFrame is
Roberto
already compliant with Kerberos authentication.
Roberto Barbera
Barbera
Università di Catania and INFN Catania - Italy
ALICE Collaboration and DataGrid Project
FAQ’s…and answers (2)
Q: I want to add a new VO to GENIUS and customize new
services for that VO. How can I do that ?
A: A new VO can be added to GENIUS in just minutes. New
VO specific services can be added just modifying only two
files: an XML file and a shell script.
Q: Can I use GENIUS to interface other m/w’s ?
A: Yes. Although GENIUS is currently based on the DataGrid
middleware(w/ and w/o GLUE extensions), it can be very
easily interfaced to others. A direct interface to the Globus
Toolkit already exists and another one to Condor is in
progress.
Q: How can I start downloading/using GENIUS ?
A: Go to the reference site https://genius.ct.infn.it, click on
“GENIUS CVS available” and follow the instructions.
Portals & Portlets 2003, Edinburgh, 14.07.2003
Roberto
Roberto Barbera
Barbera
Università di Catania and INFN Catania - Italy
ALICE Collaboration and DataGrid Project
Summary and conclusions
GENIUS, notwithstanding its “youth”, is “well on track”
and seems to have a recognized present and a
promising future
It tries to merge the concepts of “user portal” and
“science portal” and can easily evolve into a “problem
solving environment”
GENIUS is not only intended for a scientist day-by-day
use. It has expressly been included in the EU EGEE
Proposal as a dissemination and tutoring tool for non
expert users (in the last 2 months, during the Catania
and Rome EDG Tutorials, almost 80 people have been
trained to use it)
All GENIUS services (also those not shown here) can
be “seen” at the URL: http://alipc1.ct.infn.it/grid/genius/
gallery
Roberto
Roberto Barbera
Barbera