Web Access to the
Grid
using the
Grid Resource Broker
Giovanni Aloisio
Massimo Cafaro
Italo Epicoco
giovanni.aloisio@unile.it
massimo.cafaro@unile.it
italo.epicoco@unile.it
Center for Advanced
Computational Technologies
• Grid Portals
• The Grid Resource Broker (GRB)
• GRB Architecture
• GRB Services Overview
• GRB tools & technologies
• GRB Security
• GRB Services in depth
• GRB libraries
Grid Portals
• web sites providing specific contents
and related services to the scientific
community
• computational & data grids as back-end
• potentially managing a large number of
resources on behalf of the users
• User centric
• User friendly, ubiquitous web GUI
• Globus based, but:
• No need to know Globus
• No need to write Globus code
• No need to rewrite legacy code
• C APIs will be released soon under the
GNU Public License
GRB Architecture
rst Tier
cond Tier GRB Web Server MyProxy Server
GRB Libraries
GRB Web Services
Security
Info
Jobs
File/Data
GSI
MDS
GRAM
GridFTP
hird Tier
a user’s grid
• User’s profile management
• Access to Monitoring & Discovery
Services
• Job Submission
• Interactive
• Batch, with support for X-Windows
apps
• Pararameter Sweep
• Data-Flow
• Resource brokering
• Job tracking
• Grid Status
• High Performance File Transfers
• third-party
• parallel file transfer
• partial file transfer
• single file & directory transfer
tools
Globus Toolkit
• GSI
• MDS
• GRAM
• GridFTP
• C / Unix system
calls
• Globus Toolkit
v2.4
• HTML
• CGI
• MyProxy package
• TLS
• HTTP / HTTPS
• Ephemeral Cookies
• LDAP
• gSOAP
• GSI plugin for gSOAP
• MyProxy package used to store &
retrieve short-lived user’s credentials
• HTTPS used to send user’s proxy pwd
• Sessions established via ephemeral
cookies
• Cookies contain the following
information
• User’s login
• Timestamp
• Expiration date
• Message Authentication Code (MAC)
for the previous data
• GRB uses the keyed, non-malleable
MAC HMAC-SHA1
• it is computationally intractable
to generate a valid ciphertext
starting from a plaintext message related to a plaintext message
with a known ciphertext. This key property guarantees that no
adversary is able to generate a valid ciphertext without the
knowledge of both the server secret key and the plaintext,
independently of how many samples of valid plaintext and ciphertext
pairs the adversary owns
• Cookies are encrypted with TLS
• GRB uses GSI
Authorization
Web
yProxy, HTTPS
hemeral Cookies
GRB
GRB
Grid
Globus GSI
Authorization
User s Profile
Management
• Each user manages his/her grid by
adding, removing and editing:
• computational resources
• applications (work in progress)
• job submission sessions (work in
progress)
II
Access to MDS (GIIS)
GRB
GRIS
GRIS
GIIS
GRIS
GRIS
Allows queries
to arbitrary GIIS
Substring
search
Numeric
search
III
Access to MDS (GRIS)
GRIS
GRB
IV
Interactive
Job
Submit
• Interactive here means that the job
output is sent directly back to the client
browser
• Useful for little things such as listing the
user’s home directory, browsing the
contents of a file etc
• Support
for automatic staging
executable and/or input files
of
Executable
and/or input
staging
JobÕs output
V
Batch
Job
Submit
• Remote execution with support for
automatic staging of executable, input
and output files
• automatic
redirection of X-windows
display allows steering graphical apps
• simply
starting an xterm allows
writing,
editing,
compiling
and
debugging source code
VI Parameter Sweep
Jobs
• Allows
remote execution on a user
selected pool of machines of several
instances of the same executable, each
with a different input
• The
pool of machine can also be
automatically selected by the system
(brokering)
VII Data-Flow Jobs
• A simple Java applet allows drawing a
DAG whose vertices represent batch
jobs to be executed and whose edges
models precedence constraints
• The data-flow engine schedules the jobs
according to a topological sort of the
DAG using a depth-first-search traversal
VIII
Resource Brokering
QuickTime™ and a Graphics decompressor are needed to see this picture.
QuickTime™ and a Graphics decompressor are needed to see this picture.
GRIS
Query
G
R
B
GRIS
GIIS
GRIS
GRIS
s
e
c
r
u
o
s
e
R
Prog
Input
Job execution
Output
IX
Job Tracking
• Allows monitoring job status & related
file transfers for:
• batch jobs
• parameter sweep jobs
• data-flow jobs (work in progress)
X
Grid Status
• Allows verifying that Globus daemons
are up and running on the machines
belonging to the user’s grid
High Performance File
Transfers
• GRB library based on Globus GridFTP
control library
• File/directory transfer status is monitored
by GRB
• Supports third-party, parallel & partial file
transfers
• Very simple to use
• Proxy management
• Job submission
• File transfer
• Monitoring & Discovery Services
• Job status
lib_cookies
• cookie setup
• cookie encryption
• retrieving cookie information
lib_dataflow
• DAG management
• DAG topological sort
lib_myproxy
• proxy retrieval
• checking if a proxy is valid
lib_gsiftp
• Connection management
• Standard FTP commands
• Third-party transfers
• Parallel transfers using multiple streams
• Partial file transfers
• Support for directory transfers
experimental testbed
untry
Hostname
gridsurfer.unile.it
dev04.hepgrid.clrc.ac.
uk
clipper.lbl.gov
OS
linux
2.4.3
linux
2.2.16
solaris
2.7
TCP
buffer size
65535
65535
65535
experimental results I
Put one MB file test
BufSize 4096
BufSize 8192
BufSize 32768
BufSize 65536
From Italy to
UK
BufSize 16384
BufSize 4096
BufSize 8192
BufSize 32768
BufSize 65536
BufSize 16384
From Italy to
California
experimental results II
Put ten MB file test
BufSize 4096
BufSize 8192
BufSize 32768
BufSize 65536
From Italy to
UK
BufSize 16384
BufSize 4096
BufSize 8192
BufSize 32768
BufSize 65536
From Italy to
California
BufSize
experimental results
IIItransfer
Third-party
one, ten and one hundred MB files
Heterogeneous
Heterogeneousand
and
geographically
geographically
spread
spreadcomputing
computing
resources
resources
Computational
Computational Grid
Grid
Low
LowLevel
Level
Middleware
Middlewarefor
for
Grid
Gridmanagement
management
Globus
Globus Toolkit
Toolkit
Web
Webaccess
accesstotothe
the
grid:
grid:secure,
secure,easy,
easy,
transparent
transparent
Grid
Grid Resource
Resource
Broker
Broker
• Migration to GT3, OGSA & OGSI
• Better HCI
• GridLab GridSphere Portlet Framework
• support for customization
• logging
• additional support for job
scheduling/checking
• The Grid Resource Broker
• http://sara.unile.it/grb
• Giovanni Aloisio
• giovanni.aloisio@unile.it
• Massimo Cafaro
• massimo.cafaro@unile.it
• Italo Epicoco
• italo.epicoco@unile.it
G. Aloisio, M. Cafaro , C. Kesselman, R. Williams, “Web Access to
SuperComputing using the Grid”, IEEE Computing in Science and engineering,
Volume 3 Number 6 (2001), pp. 66-72
G. Aloisio, M. Cafaro, I. Epicoco, E. Blasi, “The Grid resource Broker, a
ubiquitous grid computing framework”, Journal of Scientific Programming,
Volume 10, Number 2 (2002), pp. 113-119, Special Issue on Grid Computing,
IOS Press, Amsterdam
G. Aloisio, M. Cafaro, “Web-based access to Grid using the Grid Resource
Broker”, Concurrency and Computation: Practice and Experience Journal,
Volume 14 Issue 13-15 (2002), pp. 1145-1160,
Special Issue on Grid
Computing Environments.
G. Aloisio, M. Cafaro, I. Epicoco, “Early experiences with the GridFTP protocol
using the GRB-GSIFTP library”, Future Generation Computer Systems journal,
Volume 18, Number 8 (2002), pp. 1053-1059, Special issue on Grid
Computing: Towards a New Computing Infrastructure
G. Aloisio, M. Cafaro, D. Lezzi, “The Desktop Grid Environment Enabler”,
Computing and Informatics, Volume 21, Number 4 (2002), pp. 333-345,
Special Issue on Grid Computing
G. Aloisio, M. Cafaro, P. Falabella, C. Kesselman, R. Williams, “Grid Computing
on the Web using the Globus Toolkit”, Proc. HPCN Europe 2000, Amsterdam,
Netherlands, Lecture Notes in Computer Science, Springer-Verlag, N. 1823,
pp. 32-40, 2000
G. Aloisio, M. Cafaro, E. Blasi, L. Depaolis, I. Epicoco, “The GRBLibrary: Grid
Programming with Globus in C”, Proc. HPCN Europe 2001, Amsterdam,
Netherlands, Lecture Notes in Computer Science, Springer-Verlag, N. 2110,
pp. 133-140, 2001