Globus Toolkit 4
By:
Jeton Bacaj, Joel Lathrop, Fugang
Wang & Andrew Younge
http://blackrose02.rit.edu/wiki/doku.php?id=grid:seminar1:gt4
Rochester Institute of Technology
10:33:24
Service Oriented Cyberinfrastructure Lab, http://blackrose02.rit.edu
1
Outline
• Introduction
• Components
–
–
–
–
Execution Management
Information Services
Data Management
Security
• Globus Implementation
–
–
–
–
Architectural overview
Globus and Rocks
Todo
Job submission
• Summary
10:33:24
Service Oriented Cyberinfrastructure Lab, http://blackrose02.rit.edu
2
Introduction to Globus Toolkit
• What is Globus?
– A “grid” toolkit
– The goal is, “Letting people share computing power, databases,
and other tools securely online across corporate, institutional,
and geographic boundaries without sacrificing local autonomy.”
• Modular Architecture
– Made of many components
– Not all components are inter-dependent
– Can install single, isolated components
• Globus Advantages
–
–
–
–
10:33:24
Scalable
Well documented API
Open source
Widely used
Service Oriented Cyberinfrastructure Lab, http://blackrose02.rit.edu
3
Introduction (cont)
• Components organized into classes
– Execution Management
• Initiation, monitoring, management, scheduling, and coordination
of remote computational tasks
– Information Services
• Monitoring, managing logs, archiving data, etc.
– Data Management
• Data location, transfer, and management
– Security
• Authentication, mapping Globus credentials with other forms of
credentials
– Common Runtime
• A collection of APIs allowing integration with different languages
and platforms
10:33:24
Service Oriented Cyberinfrastructure Lab, http://blackrose02.rit.edu
4
Execution Management
• Community Scheduler Framework
– Interface to resource schedulers
• Grid TeleControl Protocol
– WRSF-enabled service interface for control of remote
instruments
• Workspace Management
– Dynamic fostering of managing workspaces on hosts
• Grid Resource Allocation & Management
– “GRAM”; the bread ‘n butter of Globus Toolkit’s Execution
Management – deploying and monitoring jobs on a Grid
• Used to incorporate a wide variety of other grid and cluster services such as
Condor, PBS, LSF, SGE, BOINC and more!
10:33:24
Service Oriented Cyberinfrastructure Lab, http://blackrose02.rit.edu
5
Information Services
Some of the services that are included on Globus Toolkit:
• MDS
– Collects data about different resources about the grid
– By default, uses WSRF for polling operations
– WebMDS uses MDS that formats data in XML into HTML using XSLT
style sheets
• Indexing
– Services register with this component so others can utilize those
resources
– Caches resources of the registered services (speed obviously)
• Trigger
– What it sounds like - if there is a condition that’s met, an alert is raised
(notification via email for example)
– Monitors behavior - looks at the data with some preconfigured tests to
see if trigger conditions are met
10:33:24
Service Oriented Cyberinfrastructure Lab, http://blackrose02.rit.edu
6
Data Management and Replication
• Data Management
– GridFTP
• A Souped up version of FTP that’s secure, robust and
efficient
• Good for bulk data.
– Reliable File Tranfter (RFT)
• More reliable, job based file transfer using web services
• Data Replication
– Replica Location Service (RLS)
• Keeps track of location of the copies in the grid
– Data Replication Service (DRS)
• Higher level merge of RFT and RLS
10:33:24
Service Oriented Cyberinfrastructure Lab, http://blackrose02.rit.edu
7
Grid Security Infrastructure - GSI
• Conform to X.509 based PKI standard
• Certificate
– Used to identify and authenticate
– Structure
• (subjID, publicKey, CA’s ID, CA’s signature)
• Coded in X.509 format
– Host, service, user, proxy
• Mutual authentication
– Use SSL/TLS to exchange certificates
– Prove identity by signing a message with private key
10:33:24
Service Oriented Cyberinfrastructure Lab, http://blackrose02.rit.edu
8
Grid Security Infrastructure - GSI
• Confidential communication
– By default no encryption
– Easy to setup a shared key encryption
• Securing private key
– Pass phrase protected private key
• Proxy and certificate delegation
– Chain of trust
– RFC 3820
10:33:24
Service Oriented Cyberinfrastructure Lab, http://blackrose02.rit.edu
9
GSI - MyProxy
• MyProxy credential management service
Grid
Front End
User_home
CA
10:33:24
MyProxy
Server
Service Oriented Cyberinfrastructure Lab, http://blackrose02.rit.edu
10
GSI - MyProxy
• MyProxy credential management service
User_mobile
Grid
Front End
MyProxy
Server
10:33:24
Service Oriented Cyberinfrastructure Lab, http://blackrose02.rit.edu
11
MyProxy Commands
• MyProxy related commands in GT4
myproxy-init: store proxy credential
myproxy-store: store end-entity credential
myproxy-logon: retrieve a stored credential
myproxy-retrieve: retrieve end-entity credential
myproxy-destroy: remove credential
myproxy-info: query stored credentials
myproxy-change-pass-phrase:
change password encrypting private key
10:33:24
Service Oriented Cyberinfrastructure Lab, http://blackrose02.rit.edu
12
GSI - In Practice
• User certificate generation
User generates certificate request and private key
USER$ echo export GLOBUS_LOCATION=/usr/local/globus-4.0.6 >> ~/.bashrc
USER$ $GLOBUS_LOCATION/etc/globus-user-env.sh
USER$ grid-cert-request
Private key(passphrase protected):
~/.globus/userkey.pem
Certificate request:
~/.globus/usercert_request.pem
! Mail the certificate request to the administrator as prompted
10:33:24
Service Oriented Cyberinfrastructure Lab, http://blackrose02.rit.edu
13
GSI - In Practice
• User certificate generation
User generates certificate request and private key
CA signs the certificate
globus$ grid-ca-sign –in usercert-request.pem -out signed_USER.pem
globus$ sudo cp signed_USER.pem /home/USER/.globus/usercert.pem
10:33:24
Service Oriented Cyberinfrastructure Lab, http://blackrose02.rit.edu
14
GSI - In Practice
• User certificate generation
User generates certificate request and private key
CA signs the certificate
Add entry in gridmap file
/etc/grid-security/grid-mapfile
"/O=Grid/OU=GlobusTest/OU=simpleca-lilac03.rit.edu/OU=rit.edu
/CN=FIRST_NAME LAST_NAME" USER
10:33:24
Service Oriented Cyberinfrastructure Lab, http://blackrose02.rit.edu
15
Grid Architecture
Grid users
lilac03.rit.edu
Main Globus Server
globus-4.0.6
grid scheduler
Certificate Authority
lilac04.rit.edu
Rocks frontend
globus 4.0.3
Rocks queue manager
clover.rit.edu
VALinux
Rocks?
lilac02.rit.edu
compute-0-0
lilac01.rit.edu
compute-0-2
lilac05.rit.edu
compute-0-1
lilac08.rit.edu
compute-0-3
10:33:24
clover02.rit.edu
clover03.rit.edu
Condor/PBS/BOINC
clover04.rit.edu
Service Oriented Cyberinfrastructure Lab, http://blackrose02.rit.edu
...
??
...
16
Progress
• Main Globus installation
–
–
–
–
–
–
Used Ubuntu Linux 7.10 Server
Compiled and installed version 4.0.6 of Globus
Setup security using Simple-CA
Configured postgresql RFT Database
Started Globus container
Added test users and submitted test jobs
• Rocks Cluster
– Installed the frontend on lilac04.rit.edu
– Added Globus, Ganglia, and SGE roles to the frontend
– Installed clients on other lilac machines.
10:33:24
Service Oriented Cyberinfrastructure Lab, http://blackrose02.rit.edu
17
Todo List
• Link Rocks frontend to main Globus installation
– Configure the Certificates
– Setup and test GridFTP between machines
– Update /etc/grid-security/grid-mapfile
• Test the grid using by submitting a job to a
remote resource (Rocks)
• Setup independent CA behind a firewall using
MyProxy
– Would enhance security and simplify usage
10:33:24
Service Oriented Cyberinfrastructure Lab, http://blackrose02.rit.edu
18
Submitting a Job
• Submitting a simple job
xero@lilac03:~$ globusrun-ws -submit -c /bin/echo ROAR
Submitting job...Done.
Job ID: uuid:a07e6908-fbb8-11dc-884b-000874d2a480
Termination time: 03/28/2008 04:45 GMT
Current job state: Active
Current job state: CleanUp
Current job state: Done
Destroying job...Done.
• Job submission in Globus gets more complicated quickly
– Have to use large xml (RSL) files to specify job details
– Use a job_epr.xml file to monitor job status
– Make sure job results are retrieved
10:33:24
Service Oriented Cyberinfrastructure Lab, http://blackrose02.rit.edu
19
Summary
• The Globus Toolkit is a powerful middleware
application that allows one to build a multi-level grid
computing system that can span spatially and
administratively different computing systems.
• Setting up and managing a grid using Globus is a lot
of work!
• Globus can work with a wide variety of clusters,
queuing systems or other grid technologies in order
to provide the largest amount of resources
– System integration is unmatched by anything else
– Many tools already available for free
– Easy API to create custom tools for specific needs
10:33:24
Service Oriented Cyberinfrastructure Lab, http://blackrose02.rit.edu
20
Questions?
• Ask, and ye shall receive …
10:33:24
Service Oriented Cyberinfrastructure Lab, http://blackrose02.rit.edu
21
References
Foster, "A Globus Primer: Describing Globus Toolkit 4," 2005.
I. Foster, "Globus Toolkit 4: Software for Service-Oriented Systems," in Proc. Network And
Parallel Computing: IFIP International Conference, 2005, pp. 2-13.
C. K. Ian Foster, "Globus: a Metacomputing Infrastructure Toolkit," Iternational Journal of High
Performance Computing Applications, vol. 11, iss. 2, pp. 115-128, 1997.
L. P. N. M. C. K. I. F. M. D. Jennifer M. Schopf Ioan Raicu, "Monitoring and Discovery in a Web
Services Framework: Functionality and Performance of Globus Toolkit MDS4,"
Argonne National Laboratory2005.
W. Allcock, J. Bresnahan, R. Kettimuthu, and M. Link, "The Globus Striped GridFTP Framework
and Server," in Proc. SC '05: Proceedings of the 2005 ACM/IEEE conference on
Supercomputing, Washington, DC, USA, 2005, p. 54.
H. Kanemitsu and Y. Urano, "On the method for realizing globally and locally accessible
resource management for WS-GRAM," in Proc. Web Services, 2007. ICWS 2007.
IEEE International Conference on, 2007, pp. 1201-1204.
X. Zhang and J. M. Schopf, Performance Analysis of the Globus Toolkit Monitoring and
Discovery Service, MDS2, 2004.
I. N. Borja Sotomayor Lisa Childers, Globus Toolkit 4 Programming Java Services, Morgan
Kaufmann, 2006.
10:33:24
Service Oriented Cyberinfrastructure Lab, http://blackrose02.rit.edu
22