GRAM

Overview

Sudo entries

Scheduler Adapters

Configuring Data Staging

Job Monitoring
Globus Week 2005
Charles Bacon (bacon@mcs.anl.gov)
1
GRAM Overview


Intended for jobs where arbitrary
programs, stateful monitoring, credential
management, and file staging are
important
If the application is lightweight, with
modest input/output, may be a better
candidate for hosting directly as a WSRF
service
Globus Week 2005
Charles Bacon (bacon@mcs.anl.gov)
2
GRAM Prerequisites


A secure container
For staging jobs, access to an RFT service
and a GridFTP server
– Note that even stderr/stdout are considered
staging, so RFT and GridFTP are used in all
but the most basic jobs

sudo for running as other accounts

Can be integrated with PBS, LSF, Condor
Globus Week 2005
Charles Bacon (bacon@mcs.anl.gov)
3
Configuring GRAM sudo entries
globus ALL=(username1, username2)
NOPASSWD: /usr/local/globus4.0.0/libexec/globus-gridmap-and-execute
/usr/local/globus-4.0.0/libexec/globus-jobmanager-script.pl *
globus ALL=(username1, username2)
NOPASSWD: /usr/local/globus4.0.0/libexec/globus-gridmap-and-execute
/usr/local/globus-4.0.0/libexec/globusgram-local-proxy-tool *
Globus Week 2005
Charles Bacon (bacon@mcs.anl.gov)
4
globus-gridmap-and-execute


Performs a final safety check whether
GRAM should be allowed to act in the
current account
Will not run commands as root
Globus Week 2005
Charles Bacon (bacon@mcs.anl.gov)
5
Using Scheduler Adapters


At installation time, specify –enablewsgram-{condor,pbs,lsf}
Must have the client tools on your PATH
during the “make install” step
Globus Week 2005
Charles Bacon (bacon@mcs.anl.gov)
6
Scheduler Adapters Continued



The client will submit a job in an abstract
job-submission language
GRAM will translate that into a batch
scheduler submission script, which will be
submitted at the target user
The translation may involve site-specific
customization
– For instance, to use rsh instead of ssh in a
PBS job submission
Globus Week 2005
Charles Bacon (bacon@mcs.anl.gov)
7
Customizing Scheduler Adapters


The scheduler adapters are perl scripts in
lib/perl/Globus/GRAM/JobManager
May not need to edit directly, as GRAM
provides a setup script for some alterations
setup/globus/setup-globus-job-manager-pbs -remote-shell=rsh

There is a tutorial available in the WSGRAM developer documentation
– Dissects the LSF adapter
Globus Week 2005
Charles Bacon (bacon@mcs.anl.gov)
8
Data Staging

GRAM allows jobs to stage-in and stageout data

To perform this task it uses RFT

RFT in turn uses GridFTP servers

Simplest stage-in/stage-out example is
stdout/stderr
Globus Week 2005
Charles Bacon (bacon@mcs.anl.gov)
9
Data Staging Continued


By default, GRAM assumes there is an RFT
service on port 8080, and that there is a
GridFTP server on port 2811
To change the GridFTP location:
./setup-gram-service-common --gridftpserver="gsiftp://gridftp.host.org:1234"

To change the RFT port or host use
–staging-port and –staging-host
Globus Week 2005
Charles Bacon (bacon@mcs.anl.gov)
10
File System Mapping



The filesystem layout for your storage nodes may
differ from the layout exposed to compute nodes
Staging will fail until the mapping is specified
To establish the map, edit etc/gramservice/globus_gram_fs_map_config.xml
<mapping>
<jobPath>/pvfs/jobhome</jobPath>
<ftpPath>/pvfs/ftphome</ftpPath>
</mapping>
Globus Week 2005
Charles Bacon (bacon@mcs.anl.gov)
11
Compute Node Local Storage




Your cluster may have some local space
setup for temporary use during
computation
Set this value in etc/gram-serviceScheduler/jndi-config.xml under the
“scratchDirectory” parameter
Will be exposed to users as
$GLOBUS_SCRATCH_DIR
Defaults to $GLOBUS_USER_HOME
Globus Week 2005
Charles Bacon (bacon@mcs.anl.gov)
12
Alternate grid-mapfile


You may use a custom grid-mapfile for the
GRAM service, as you can for all services
This requires an update to the sudoers
entry
– Add –g /path/to/grid-mapfile
Globus Week 2005
Charles Bacon (bacon@mcs.anl.gov)
13
Job Resource Limit


GRAM limits its resource usage to 1000
simultaneous job resources by default
You may increase/decrease this limit as
appropriate
– setup-gram-service-common –max-job-limit=“150”
Globus Week 2005
Charles Bacon (bacon@mcs.anl.gov)
14
Job Monitoring


GRAM uses a Scheduler Event Generator
(SEG) to keep track of job status
Parses logfiles kept by the scheduler
– Allows for restartability if the container fails
– Higher performance than polling the scheduler

The location of the logfiles will be picked
up at installation time by looking for
variables like PBS_HOME
Globus Week 2005
Charles Bacon (bacon@mcs.anl.gov)
15
Scheduler Logfile Locations




Each scheduler also has a file under
etc/globus-<scheduler>.conf
This contains the log_file=/path/to/log
map for that scheduler
You may update this if your scheduler
keeps logs in a different location
The logs should be readable by the globus
user
Globus Week 2005
Charles Bacon (bacon@mcs.anl.gov)
16
Logging



WS GRAM creates log events for the
container. Depending on what you’d like
to debug, you may turn up the logging on
different pieces of the architecture
Handled by the container-log4j.properties
file in $GLOBUS_LOCATION
See the WS-GRAM Developer’s Guide to
see a list of the available class names
Globus Week 2005
Charles Bacon (bacon@mcs.anl.gov)
17
Submitting A Test Job

No staging:
– globusrun-ws –submit –c /bin/true
– echo $?

With staging
– globusrun-ws –submit –s –c /bin/echo “Hello”

From another host
– globusrun-ws –submit –F hostname –s –c
/bin/echo “Hello”
Globus Week 2005
Charles Bacon (bacon@mcs.anl.gov)
18