New Productivity Working Group
Architectural Area
Paul Foley
Platform Computing
March 2002
NPi Note Resource Description Language Requirements
1
Abstract
The New Productivity Initiative (NPi) architecture requires a Resource Description Language
(RDL) for many of the scenarios described in the NPi DRM Reference Model found on the NPi
Working Groups web site. The requirements of a Resource Description Language are discussed
and several products and standards are evaluated for their usability.
This document only presents the different RDLs that are popular in the grid communities does not
make any comments on their suitability to the Requirements. This document should be
considered a living document as groups in the GGF are continually working on RDL standards.
pfoley@platform.com
1
New Productivity Working Group
Architectural Area
Paul Foley
Platform Computing
March 2002
Table of Contents
1
Abstract .................................................................................................................................... 1
1
Introduction............................................................................................................................... 3
1.1
RDL Requirements of the NPi architecture....................................................................... 3
1.1.1
Functional Requirements........................................................................................... 3
1.1.2
RDL Characteristics ................................................................................................... 3
2
Globus RSL .............................................................................................................................. 3
3
W3C RDF ................................................................................................................................. 5
4
LSF Resource Requirements String ........................................................................................ 6
5
PBS .......................................................................................................................................... 8
6
Legion....................................................................................................................................... 8
7
Condor ClassAds ..................................................................................................................... 8
7.1
Papers and Slides ............................................................................................................. 9
7.2
ClassAd Distribution Downloads....................................................................................... 9
7.3
C++ Implementation.......................................................................................................... 9
7.4
Java Implementation ......................................................................................................... 9
7.5
Re: Globus RSL --> Condor's ClassAds ........................................................................... 9
8
Sun Grid Engine ..................................................................................................................... 12
9
Maui Scheduler ...................................................................................................................... 13
9.1
MauiME command file..................................................................................................... 13
9.1.1
MPI-over-Myrinet CMD file and Launch Script ........................................................ 13
9.1.2
MPI-over-ethernet CMD file and Launch Script....................................................... 14
9.1.3
Default CMD file....................................................................................................... 14
pfoley@platform.com
2
New Productivity Working Group
Architectural Area
1
Paul Foley
Platform Computing
March 2002
Introduction
1.1
RDL Requirements of the NPi architecture
1.1.1
Functional Requirements
•
Specifying a job's requirements keyword operator value plus logical operators (set, and,
or)
•
Communicating current resource utilization level keyword operator value, no logical
operators needed
•
The means by which resources are described or describe themselves keyword operator
value (plus addition of cost?)
•
Resource discovery including Keyword or keyword operator value tuple plus logical
operators
•
Data schema for logging (as described in Condor ClassAds)
Additional functions like: substitution (as found in Globus RSL), arithmetic operators (as found in
LSF), specification of order of preference (as found in LSF), specification of expected
usage/change of usage rate over time (as found in LSF)
1.1.2
2
RDL Characteristics
•
Hierarchical and nested (inheritance, and containment) Resource Definitions
•
Identifying individual entities and attributes of those entities
•
Grouping entities together with logical operators
•
Must be implement-able and map able to the current RDLs.
•
Extensible for new resource types and categories.
•
Validation – Syntactic validation, semantic validation
Globus RSL
Created to identify and manage demand via brokers and an information service across scientific
and technical grids. Support for parallel computations and co-allocation was paramount. Users
are highly technical.
Elementary form is parenthetical clauses (attribute operator value [value]) prefixed and connected
with additional operators as necessary. Additional operators include:
& for constraints
| for disjunctions
+ for multirequests
Supported attributes are: executable, arguments, environment, stdin, stdout, stderr,
resourceManagerContact, resourceManagerName. Unknown attributes are passed through and
may be handled by subsequent tools.
Examples from Globus tutorial slides:
& (count>=5) (count<=10)
pfoley@platform.com
3
New Productivity Working Group
Architectural Area
Paul Foley
Platform Computing
March 2002
(max_time=240)(memory>=64)
(executable=myprog)
‘Create 5-10 instances of myprog, each on a machine with at least 64 MB of memory that is
available to me for 4 hours’
& (executable=myprog)
( | (&(count=5)(memory>=64))
(&(count=10)(memory>=32)))
‘Create 5 instances of myprog on a machine that has at least 64 MB of memory, or 10 instances
on a machine with at least 32MB of memory’
+ (&(count=5)(memory>=64)
(executable=p1))
(&(network=atm)(executable=p2))
‘Execute 5 instances of p1 on a machine with at least 64MB of memory and execute p2 on a
machine with an ATM connection’
Common RSL attributes
rsl_substitution
GRAM attributes
stdin
stdout
stderr
queue
project
dryRun
maxMemory
minMemory
hostCount
arguments
count
directory
executable
environment
jobType
maxTime
maxWallTime
maxCpuTime
gramMyjob
Typical GRAM resource descriptions contain at least a few relations in a conjunction:
(* this is a comment *)
& (executable = a.out (* <-- that is an unquoted literal *))
(directory
= /home/nobody )
(arguments
= arg1 "arg 2")
(count = 1)
Substitutions can be used to make sure the same substring is used multiple times in
a resource description:
& (rsl_substitution
= (TOPDIR
"/home/nobody")
(DATADIR $(TOPDIR)"/data")
pfoley@platform.com
4
New Productivity Working Group
Architectural Area
Paul Foley
Platform Computing
March 2002
(EXECDIR $(TOPDIR)/bin) )
(executable = $(EXECDIR)/a.out
(* ^-- implicit concatenation *))
(directory
= $(TOPDIR) )
(arguments
= $(DATADIR)/file1
(* ^-- implicit concatenation *)
$(DATADIR) # /file2
(* ^-- explicit concatenation *)
*))
'$(FOO)'
(* <-- a quoted literal
(environment = (DATADIR $(DATADIR)))
(count = 1)
Performing all variable substitution and removing comments yields an equivalent RSL
string:
& (rsl_substitution
= (TOPDIR
"/home/nobody")
(DATADIR "/home/nobody/data")
(EXECDIR "/home/nobody/bin") )
(executable = "/home/nobody/bin/a.out" )
(directory
= "/home/nobody" )
(arguments
= "/home/nobody/data/file1"
"/home/nobody/data/file2"
"$(FOO)" )
(environment = (DATADIR "/home/nobody/data"))
(count = 1)
3
W3C RDF
Example of how W3C RDF format would be used to express DRM resources:
The hosts in cluster Able are HostA, HostB, HostC.
<rdf:RDF>
<rdf:Description about=”Able”>
<s:hosts>
<rdf:Bag>
<rdf:li resource=”HostA“/>
<rdf:li resource=”HostB“/>
<rdf:li resource=”HostC“/>
</rdf:Bag>
</s:hosts>
</rdf:Description>
pfoley@platform.com
5
New Productivity Working Group
Architectural Area
Paul Foley
Platform Computing
March 2002
</rdf:RDF>
[Paul Foley]
This shows the structure that I was trying to explain in our discussion last call (1Mar02). <hosts>
describes one level of a resource hierarchy, <bag> is a generic container within that layer and the
<rdf:li> (list items are entries within the generic container. I don’t agree with the final format since
it doesn’t allow an operator (e.g. memory > 1GB).
I think it would be useful to derive a set of worst-case but believable resource definition examples
against which to test out our evolving approaches. This needs to test the different dimensions of
use; e.g. a resource defining itself, a job defining resource requirements, a resource constraint as
captured in a rule/policy etc.
4
LSF Resource Requirements String
Describes the resources a job needs, used to match demand to supply i.e. select the host(s) for
job execution. Comprised of four sections:
Selection section – specifies the characteristics the job is seeking to match
select[selectstring]
Where selectstring can be created by combining resource names with logical and arithmetic
operators. Resources are load indices (dynamic, defined and built-in to LSF), static resources
(static, defined and built-in to LSF) and external load indices (dynamic, defined external to LSF).
Load indices
Index
Measures
status
r15s
host status string
run queue
processes
length
run queue
processes
length
run queue
processes
length
CPU
(per cent)
utilization
paging
pages in + pages out per
activity
second
logins
users
idle time
minutes
available swap megabytes
space
available
megabytes
memory
available
megabytes
space in
temporary file
system
( /tmp on
UNIX,
C:\temp on
Windows NT
for example)
disk I/O
kilobytes per
(shown by second
r1m
r15m
ut
pg
ls
it
swp
mem
tmp
io
pfoley@platform.com
Units
6
New Productivity Working Group
Architectural Area
Paul Foley
Platform Computing
March 2002
lsload -l)
external load
index
configured by
LSF
administrator
name
Static resources
Index
Measures
Units
Determined by
type
model
hname
cpuf
server
rexpri
ncpus
ndisks
maxmem
host type
host model
host name
CPU factor
host can run remote jobs
execution priority (UNIX only)
number of processors
number of local disks
maximum RAM memory available to
users
maximum available swap space
maximum available space in
temporary file system
string
string
string
relative
Boolean
processors
disks
megabytes
configuration
configuration
configuration
configuration
configuration
configuration
LIM
LIM
LIM
megabytes
megabytes
LIM
LIM
maxswp
maxtmp
nice(2) argument
Some resources can also reference the special values any and local.
select[(swp > 50 && type == MIPS) || (swp > 35 && type == ALPHA)]
select[((2*r15s + 3*r1m + r15m) / 6 < 1.0) && !fs && (cpuf > 4.0)]
select[mem>600 && mem<1000 && solaris]
select[ mem<=600 && solaris && cpuf>5]
Ordering section – specifies how resource match hits are to be sorted for selection
order[orderstring]
The order string allows you to specify criteria to select the best host on which to run the job. For
example, you can specify to sort so that those that are lightly loaded or that have large memory are
at the top of the list for selecting hosts.
order[tmp:r15s]
Resource Usage section – defines the expected resource usage of the job including the optional
specification of duration and decay
rusage[usagestring]
The resource usage string is used to specify resource reservation for jobs. Internal and external
load indices are used to define the initial reserved amount of the resource. Duration is the time
period within which the specified resources should be reserved. The decay value indicates how
the reserved amount should decrease over the duration.
rusage[mem=50:duration=100:decay=1]
This example indicates that 50MB memory should be reserved for the job. As the job runs, the
amount reserved will decrease at approximately 0.5 megabytes per minute until the 100 minutes
is up.
pfoley@platform.com
7
New Productivity Working Group
Architectural Area
Paul Foley
Platform Computing
March 2002
rusage[mem=25:swp=50]
This example sets the resource reservation for the job to 25MB of memory and 50MB of swap
space.
Job Spanning section – specifies the locality requirements of a parallel job
span[spanstring]
The span string indicates if a parallel batch job should span across multiple hosts.
span[hosts=1]
This indicates that all the processors allocated to this job must be on the same host.
span[ptile=4]
This indicates that only 2 processors on each host should be allocated to the job regardless
of how many processors the host possesses.
Complex resource requirements are specified with multiple sections:
select[ncpus>=4] span[hosts=1]
select[physmem>1500]rusage[physmem=1500:duration=30]
select[mem>1500]rusage[mem=1500:duration=30]
Commonly, no section name is given and the entire expression is treated as a selection string.
mem>1500
5
PBS
Seeking technical information
6
Legion
Contacted Marty Humphry (UVa)
7
Condor ClassAds
Classified Advertisements (classads) are the lingua franca of Condor. They are
used for describing jobs, workstations, and other resources. They are exchanged
by Condor processes to schedule jobs. They are logged to files for statistical and
debugging purposes. They are used to enquire about current state of the system.
A classad is a mapping from attribute names to expressions. In the simplest
cases, the expressions are simple constants (integer, floating point, or string). A
classad is thus a form of property list. Attribute expressions can also be more
complicated. There is a protocol for evaluating an attribute expression of a
classad vis a vis another ad. For example, the expression "other.size > 3" in
one ad evaluates to true if the other ad has an attribute named size and the
value of that attribute is (or evaluates to) an integer greater than three. Two
classads match if each ad has attribute requirements that evaluates to true in the
context of the other ad. Classad matching is used by the Condor central manager
to determine the compatibility of jobs and workstations where they may be run.
The classad library is available as a separate stand-alone package for use in
applications other than Condor.
pfoley@platform.com
8
New Productivity Working Group
Architectural Area
Paul Foley
Platform Computing
March 2002
If you use the ClassAd library, we encourage you to subscribe to the ClassAd
mailing list. This list is for discussion among all ClassAd users, and for
occasional announcements related to ClassAds. To subscribe, send an email to
"majordomo@cs.wisc.edu" with:
subscribe classad-users
in the body of your email. You will be required to confirm your registration. At
least one of the ClassAd developers reads this mailing list.
7.1 Papers and Slides
Matchmaking: Distributed Resource Management for High Throughput Computing Proceedings of
the Seventh IEEE International Symposium on High Performance Distributed Computing, July 2831, 1998, Chicago, IL.
A high-level overview of the ClassAd language and its use in Condor.
Matchmaking Frameworks for Distributed Resource Management Ph.d Dissertation of Rajesh
Raman, , October 2000
Overview of the ClassAd Language; PostScript slides of language features
The ClassAd Language Reference Manual. A formal specification of the syntax and semantics of
the ClassAd language (under construction).
7.2
ClassAd Distribution Downloads
7.3 C++ Implementation
A tutorial for programming the C++ implementation. The tutorial also includes an informal
description of the language.
Documentation of the C++ ClassAd Distribution.
Version 0.9.2 of the ClassAd library (12-Feb-2002)
(There is no need to download this version if you have Version 0.9.1. The new version just
simplifies the installation of ClassAds.)
7.4 Java Implementation
Documentation
classad.jar
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
7.5
Re: Globus RSL --> Condor's ClassAds
To: Massimo Sgaravatto - INFN Padova <massimo.sgaravatto@pd.infn.it>
Subject: Re: Globus RSL --> Condor's ClassAds
From: Stuart Martin <smartin@mcs.anl.gov>
Date: Fri, 14 Apr 2000 10:44:28 -0500
Cc: discuss@globus.org
pfoley@platform.com
9
New Productivity Working Group
Architectural Area
Paul Foley
Platform Computing
March 2002
In-Reply-To: <Pine.OSF.4.10.10004141342500.538-100000@ospd00.pd.infn.it>
Sender: owner-discuss@mcs.anl.gov
Massimo,
The Jobmanager only cares about a set of RSL attributes. Any attributes outside of the set are
ignored. The ones you used: requirements, log, notification ... are ignore.
There is some work going on to come up with a good way to allow scheduler specific attributes to
be passed down to the various scheduler scripts. But nothing concrete has been decided upon
yet.
If you need this to work in the near term, you could copy the technique used in the globus-scriptloadleveler-submit script. It looks for environment variables set in the (environment=xxx)
attribute. This will likely be very similar to the eventual implementation.
For example, in the RSL you could have
&
(executable=pippo.condor)
(stdout=out)
(stderr=err)
(count=1)
(label=TestGlobusCondor)
(arguments= 'primo' 'secondo')
(jobtype=condor)
(environment=(REQUIRMENTS='"machine ==
"lxde01.pd.infn.it"')(LOG=/home/sgaravat/log)(NOTIFICATION=always)(NOTIFY_USER=massim
o.sgaravatto@pd.infn.it))
Then modify the globus-scrpt-condor-submit to look for these variables and set the condor
arguments appropriatly.
-Stu
At 01:44 PM 4/14/00 +0200, Massimo Sgaravatto - INFN Padova wrote:
Hi !
I am trying to analyze how the RSL expressions are translated in the Condor's
ClassAds: I am submitting a simple job to a Globus machine that is a member of
a Condor pool, composed by ~ 200 machines (the INFN Condor Wan pool).
I use the command:
globusrun -r lxde01.pd.infn.it/jobmanager-condor -f condor.rsl
And this is condor.rsl:
&
(executable=pippo.condor)
(requirements='''machine == "lxde01.pd.infn.it"''')
(stdout=out)
pfoley@platform.com
10
New Productivity Working Group
Architectural Area
Paul Foley
Platform Computing
March 2002
(stderr=err)
(log=/home/sgaravat/log)
(jobtype=condor)
(count=1)
(label=TestGlobusCondor)
(arguments= 'primo' 'secondo')
(notification=always)
(notify_user=massimo.sgaravatto@pd.infn.it)
I edited <deploy>/libexec/globus-script-condor-submit, and deleted the line:
${rm} $condor_file
as suggested by Stu, so in /tmp a condor_script.description.* file is created:
this is the Condor submit description file.
This is my condor_script.description.* (created when I run globusrun):
####################
#
# description file for condor submision
#
####################
Universe
= standard
Executable = pippo.condor
Requirements = OpSys == "LINUX" && Arch == "INTEL" && FileSystemDomain != "xxx"
Arguments = primo secondo
Environment = GLOBUS_GRAM_MYJOB_CONTACT=URLxnexus://lxde01.pd.infn.it:20589/;X509_CERT_DIR=/etc/globus/share/certificates;GLOBUS_GRA
M_JOB_CONTACT=https://lxde01.pd.infn.it:20588/16475/955711556/;GLOBUS_DEPLOY_PAT
H=/etc/globus;GLOBUS_INSTALL_PATH=/usr/local.pd/globus;X509_USER_PROXY=/users/noi/
sgaravat/.globus/.gass_cache/globus_gass_cache_955711556;
Initialdir = /users/noi/sgaravat
Input = /dev/null
Output = /users/noi/sgaravat/.globus/.gass_cache/globus_gass_cache_955711557
Error = /users/noi/sgaravat/.globus/.gass_cache/globus_gass_cache_955711558
It seems that only the "common" RSL attributes are recognized, while the other
attributes (in this case: requirements, log, notification, notify_user)
are not passed unchanged to Condor.
Any ideas ?
Many thanks for your help!
Cheers, Massimo
pfoley@platform.com
11
New Productivity Working Group
Architectural Area
Paul Foley
Platform Computing
March 2002
\|///
\ ~ ~ //
(/ @ @ /)
-------oOOo-(_)-oOOo---------------------------------Massimo Sgaravatto
INFN Sezione di Padova
Via Marzolo, 8
35131 Padova - Italy
Tel: ++39 0498277029
Fax: ++39
0498277102
oooO
E-mail: massimo.sgaravatto@pd.infn.it
(
)
Oooo
Home page:
http://www.pd.infn.it/~sgaravat
--------\ (----(
)---------------------------------_)
) /
(_/
References:
Globus RSL --> Condor's ClassAds
From: Massimo Sgaravatto - INFN Padova <massimo.sgaravatto@pd.infn.it>
Prev by Date: Globus RSL --> Condor's ClassAds
Next by Date: globus-script-lsf-queue
Prev by thread: Globus RSL --> Condor's ClassAds
Next by thread: globus-script-lsf-queue
Index(es):
Date
Thread
8
Sun Grid Engine
pfoley@platform.com
12
New Productivity Working Group
Architectural Area
9
Paul Foley
Platform Computing
March 2002
Maui Scheduler
9.1
MauiME command file
Here are some sample MauiME command files.
NOTE FOR MPI JOBS: The executable specified in the "Exec" stanza is run
only on the "head node" that your job runs on. This executable should be a
script that invokes the appropriate mpirun that matches the mpicc you
compiled your program with, (same MPICH version and adapter type)!
9.1.1
MPI-over-Myrinet CMD file and Launch Script
This command file will run a 128 tasks, 1 task-per-node, on 128 nodes using
MPI-over-Myrinet driver.
# Initial working directory and wallclock time
IWD == "/tmp"
WCLimit == 3600
# Account stanza
Account == "WWGD?-1900-53X"
# Task geometry
Tasks == 128
Nodes == 128
TaskPerNode == 1
# Feature requests
Arch == x86
OS == Linux
# MPI Myrinet job
JobType == "mpi.ch_gm"
# Launch script (SEE NOTE BELOW)
Exec == "/usr/bin/runmpi_gm"
# NOTE: args, if any, are separated by whitespace ` '
Args == "/u/specialk/wank/specialks_mpi_myrinet_job"
# Output (Notice the envvar references!)
Output == "/tmp/$(MAUI_JOB_USER)-1x128-gm-$(MAUI_JOB_ID).out"
Error
== "/tmp/$(MAUI_JOB_USER)-1x128-gm-$(MAUI_JOB_ID).err"
Log
== "/tmp/$(MAUI_JOB_USER)-1x128-gm-$(MAUI_JOB_ID).log"
# Stdin tied to null
Input == "/dev/null"
pfoley@platform.com
13
New Productivity Working Group
Architectural Area
Paul Foley
Platform Computing
March 2002
The runmpi_gm launch script is installed by the RPM and tarball installations of
the Maui Scheduler. It is also in the CVS source tree in the mauisched/scripts/
directory.
9.1.2
MPI-over-ethernet CMD file and Launch Script
This will run a 128 tasks, 2 tasks-per-node, on 64 nodes using MPI-over-ethernet
driver.
# Initial working directory and wallclock time
IWD == "/tmp"
WCLimit == 360
# Account stanza
Account == "WWGD?-1900-53X"
# Task geometry
Tasks == 128
Nodes == 64
TaskPerNode == 2
# Feature requests
Arch == x86
OS == Linux
# MPI ethernet job
JobType == "mpi.ch_p4"
# Launch script (SEE NOTE BELOW)
Exec == "/usr/bin/runmpi_eth"
# NOTE: args, if any, are separated by whitespace ` '
Args == "/u/specialk/wank/specialks_mpi_ethernet_job foo bar"
# Output (Notice the envvar references!)
Output == "/tmp/$(MAUI_JOB_USER)-2x064-eth-$(MAUI_JOB_ID).out"
Error
== "/tmp/$(MAUI_JOB_USER)-2x064-eth-$(MAUI_JOB_ID).err"
Log
== "/tmp/$(MAUI_JOB_USER)-2x064-eth-$(MAUI_JOB_ID).log"
# Stdin tied to null
Input == "/dev/null"
The runmpi_eth launch script is installed by the RPM and tarball installations of
the Maui Scheduler. It is also in the CVS source tree in the mauisched/scripts/
directory.
9.1.3
Default CMD file
Use this JobType for non-MPI, custom parallel (or serial or embarassingly
parallel) jobs where you've concocted your own message-passing. This JobType
pfoley@platform.com
14
New Productivity Working Group
Architectural Area
Paul Foley
Platform Computing
March 2002
will run as many instances of the specified executable on each node and across
nodes as you specify (up to the limits of the cluster, of course).
START CONDITIONS: As you might imagine, each process of your parallel job
is started asynchronously, so you may need to implement a wait-for-all routine to
begin processing.
EXIT CONDITIONS: Your whole job will finish when one of the various
processes exits. Once the scheduler receives notice that one of the processes
has exited, it will stop the whole job! If this is not the desired behavior, then you
can change your code so that the various processes sleep until some condition is
met, at which point at least one of them can exit, stopping the job.
This CMD file specifies to cat the password file to an output file in /tmp across
128 tasks, 1 tasks-per-node, on 128 nodes. NOTE: there is a potential race
condition in getting output for all the nodes due to the EXIT CONDITIONS
section above. Although, in practice this will often run quickly enough that all
tasks will finish before one of them reports back that it has finished!(read the
EXIT CONDITIONS section above). Anyway, fixing this is left as an excercise for
the reader!
# start in /tmp 360 wallclock
IWD == "/tmp"
WCLimit == 360
# Account (this is necessary for billing)
Account == "STAFF-9998-S00"
Tasks == 128
Nodes == 128
TaskPerNode == 1
Arch == x86
OS == Linux
# no MPI
JobType == "Default"
# script (will run on all nodes specified)
Exec == "/bin/cat"
# NOTE: args separated by whitespace ` '
#Args == ""
# stdin
Input
== "/etc/passwd"
# stdout
Output == "/tmp/$(MAUI_JOB_USER)-default-$(MAUI_JOB_ID).out"
# stderr
Error
== "/tmp/$(MAUI_JOB_USER)-default-$(MAUI_JOB_ID).err"
pfoley@platform.com
15
New Productivity Working Group
Architectural Area
Log
Paul Foley
Platform Computing
March 2002
== "/tmp/$(MAUI_JOB_USER)-default-$(MAUI_JOB_ID).log"
Overview Package Class Tree Deprecated Index Help
PREV NEXT FRAMES
NO FRAMES
-------------------------------------------------------------------------------Serialized Form
-------------------------------------------------------------------------------Package unm.maui
Class unm.maui.MauiException implements Serializable
-------------------------------------------------------------------------------Package unm.maui.db
Class unm.maui.db.MauiDBException implements Serializable
-------------------------------------------------------------------------------Package unm.maui.misc
Class unm.maui.misc.AuthSecException implements Serializable
Class unm.maui.misc.JobChecker implements Serializable
Serialized Fields
prefix
java.lang.String prefix
Properties prefix.
Class unm.maui.misc.Logger implements Serializable
Serialized Fields
prefix
java.lang.String prefix
Properties prefix.
Class unm.maui.misc.SystemProfiler implements Serializable
Serialized Fields
prefix
java.lang.String prefix
Properties prefix.
-------------------------------------------------------------------------------Package unm.maui.res
Class unm.maui.res.BasicRes implements Serializable
Serialized Fields
submitter
pfoley@platform.com
16
New Productivity Working Group
Architectural Area
Paul Foley
Platform Computing
March 2002
java.lang.String submitter
User who created the reservation.
-------------------------------------------------------------------------------starttime
long starttime
Reservation starttime.
-------------------------------------------------------------------------------submittime
long submittime
Reservation submittime.
-------------------------------------------------------------------------------duration
long duration
Requested duration.
-------------------------------------------------------------------------------endtime
long endtime
Endtime.
-------------------------------------------------------------------------------state
java.lang.String state
Reservation state.
-------------------------------------------------------------------------------slotHash
java.util.Hashtable slotHash
Hashtable of NodeID -> SlotID[].
See Also:
Node, Slot
-------------------------------------------------------------------------------slotsCount
int slotsCount
Number of slots.
-------------------------------------------------------------------------------rmID
RMID rmID
Associated RMs.
--------------------------------------------------------------------------------
pfoley@platform.com
17
New Productivity Working Group
Architectural Area
Paul Foley
Platform Computing
March 2002
id
ReservationID id
ID.
-------------------------------------------------------------------------------finished
boolean finished
Class unm.maui.res.BasicSys implements Serializable
Serialized Fields
auth
java.lang.String auth
Authorization.
-------------------------------------------------------------------------------authType
java.lang.String authType
Authorization type.
See Also:
SysReservation.AUTH_USER, SysReservation.AUTH_GROUP,
SysReservation.AUTH_ACCOUNT
-------------------------------------------------------------------------------rmID
RMID rmID
RMID.
-------------------------------------------------------------------------------allowsOldFarts
boolean allowsOldFarts
Grandfathering-in.
-------------------------------------------------------------------------------recurring
boolean recurring
Is this a recurring container?
-------------------------------------------------------------------------------cronline
java.lang.String cronline
Cronline recurring interval.
-------------------------------------------------------------------------------limits
java.util.Hashtable limits
pfoley@platform.com
18
New Productivity Working Group
Architectural Area
Paul Foley
Platform Computing
March 2002
Hashtable of limits.
Class unm.maui.res.JobRes implements Serializable
Serialized Fields
job
Job job
Associated job.
Class unm.maui.res.ReservationException implements Serializable
Serialized Fields
id
int id
Problem identification.
-------------------------------------------------------------------------------resource
java.lang.String resource
Resource in question.
-------------------------------------------------------------------------------nodeID
NodeID nodeID
Node in question.
-------------------------------------------------------------------------------req
java.lang.String req
Requested feature (should be mismatch with avail).
-------------------------------------------------------------------------------avail
java.lang.String avail
Available feature (should be mismatch with req).
Class unm.maui.res.ReservationID implements Serializable
Serialized Fields
res
Reservation res
Associated reservation.
-------------------------------------------------------------------------------order
long order
Key order.
Class unm.maui.res.SysRes implements Serializable
pfoley@platform.com
19
New Productivity Working Group
Architectural Area
Paul Foley
Platform Computing
March 2002
Serialized Fields
authType
java.lang.String authType
Type of authorization.
-------------------------------------------------------------------------------auth
java.lang.String auth
Authorization string.
-------------------------------------------------------------------------------slotIDHash
java.util.Hashtable slotIDHash
Hashtable of SlotID -> SlotID. Used for quick Slot lookups.
See Also:
SlotID
-------------------------------------------------------------------------------limits
java.util.Hashtable limits
Hashtable of limits.
-------------------------------------------------------------------------------Package unm.maui.rm
Class unm.maui.rm.BasicJob implements Serializable
Serialized Fields
res
JobReservation res
Reservation.
-------------------------------------------------------------------------------state
java.lang.String state
Job state.
-------------------------------------------------------------------------------submittime
long submittime
Submission time (in epoch time).
-------------------------------------------------------------------------------jobID
JobID jobID
The JobID.
pfoley@platform.com
20
New Productivity Working Group
Architectural Area
Paul Foley
Platform Computing
March 2002
-------------------------------------------------------------------------------remoteID
JobID remoteID
The remoteID, may be null.
-------------------------------------------------------------------------------debitted
boolean debitted
Job has been debitted from system bank?
-------------------------------------------------------------------------------cmd
CMD cmd
Representation of the Job .cmd file.
-------------------------------------------------------------------------------qos
int qos
Quality of service value.
-------------------------------------------------------------------------------mesg
java.lang.String mesg
Info message.
Class unm.maui.rm.BasicNode implements Serializable
Serialized Fields
state
java.lang.String state
Node state.
-------------------------------------------------------------------------------features
java.util.Hashtable features
Space delimited list of features.
-------------------------------------------------------------------------------slots
Slot[] slots
Configured Slots.
-------------------------------------------------------------------------------slotIDHash
java.util.Hashtable slotIDHash
SlotID -> Slot.
pfoley@platform.com
21
New Productivity Working Group
Architectural Area
Paul Foley
Platform Computing
March 2002
See Also:
SlotID, Slot
-------------------------------------------------------------------------------nodeID
NodeID nodeID
NodeID.
-------------------------------------------------------------------------------parent
RM parent
Parent RM.
Class unm.maui.rm.BasicRM implements Serializable
Serialized Fields
scheduler
Scheduler scheduler
The scheduler object.
Reference to the parent scheduler.
-------------------------------------------------------------------------------rmID
RMID rmID
RMID.
Class unm.maui.rm.JobID implements Serializable
Serialized Fields
job
Job job
Associated job.
-------------------------------------------------------------------------------priority
long priority
Job priority.
Class unm.maui.rm.NodeID implements Serializable
Serialized Fields
node
Node node
Associated node.
Class unm.maui.rm.RMException implements Serializable
Class unm.maui.rm.RMID implements Serializable
Serialized Fields
pfoley@platform.com
22
New Productivity Working Group
Architectural Area
Paul Foley
Platform Computing
March 2002
rm
RM rm
Associated RM.
Class unm.maui.rm.Slot implements Serializable
Serialized Fields
id
SlotID id
SlotID.
-------------------------------------------------------------------------------parent
Node parent
Parent Node.
-------------------------------------------------------------------------------avail
boolean avail
Slot availability.
-------------------------------------------------------------------------------reservations
java.util.TreeMap reservations
-------------------------------------------------------------------------------rezes
Reservation[] rezes
Class unm.maui.rm.SlotID implements Serializable
Serialized Fields
slot
Slot slot
The Slot.
-------------------------------------------------------------------------------Package unm.maui.sched
Class unm.maui.sched.BackfillMod implements Serializable
Serialized Fields
scheduler
Scheduler scheduler
The scheduler.
-------------------------------------------------------------------------------prefix
java.lang.String prefix
pfoley@platform.com
23
New Productivity Working Group
Architectural Area
Paul Foley
Platform Computing
March 2002
Properties prefix.
Class unm.maui.sched.BasicSchedulable implements Serializable
Serialized Fields
duration
long duration
"Wallclock" time, (duration in seconds).
-------------------------------------------------------------------------------submitter
java.lang.String submitter
Name of user submitting schedulable.
-------------------------------------------------------------------------------user
java.lang.String user
Name of user owning schedulable.
-------------------------------------------------------------------------------group
java.lang.String group
Name of group owning schedulable.
-------------------------------------------------------------------------------account
java.lang.String account
Account name.
-------------------------------------------------------------------------------exceptions
java.util.Vector exceptions
List of exceptions.
-------------------------------------------------------------------------------reqSlots
int reqSlots
Total tasks (Slots) required.
-------------------------------------------------------------------------------reqNodesSlots
java.lang.String reqNodesSlots
Comma-separated string of NodeIDs and/or SlotIDs (requested nodes/slots).
-------------------------------------------------------------------------------geometry
java.lang.String geometry
pfoley@platform.com
24
New Productivity Working Group
Architectural Area
Paul Foley
Platform Computing
March 2002
Task geometry required.
If null, reqSlots is the determining factor.
-------------------------------------------------------------------------------reqFeatures
java.util.Hashtable reqFeatures
Mapping of required features. For example:
class == "foo" Arch == "R6000" Memory >= 64 ...
-------------------------------------------------------------------------------associate
java.io.Serializable associate
Associated Serializable object.
Class unm.maui.sched.EpilogEvent implements Serializable
Class unm.maui.sched.LoopEvent implements Serializable
Class unm.maui.sched.PrologEvent implements Serializable
Class unm.maui.sched.ReservationsMod implements Serializable
Serialized Fields
scheduler
Scheduler scheduler
The scheduler.
Class unm.maui.sched.Sched implements Serializable
Serialized Fields
reservations
BinaryTree reservations
Master Reservations (sorted by finishtime).
-------------------------------------------------------------------------------runHash
java.util.Hashtable runHash
Running Jobs hash (JobID -> Job).
-------------------------------------------------------------------------------holdHash
java.util.Hashtable holdHash
Held Jobs hash (JobID -> Job).
-------------------------------------------------------------------------------doneHash
java.util.Hashtable doneHash
Finished or cancelled Jobs hash (JobID -> Job).
--------------------------------------------------------------------------------
pfoley@platform.com
25
New Productivity Working Group
Architectural Area
Paul Foley
Platform Computing
March 2002
deferredHash
java.util.Hashtable deferredHash
Deferred Jobs hash (JobID -> Job).
-------------------------------------------------------------------------------submitQ
Queue submitQ
Job queue of JobID.
The queue is assumed to be in the correct order.
See Also:
Prioritizer
-------------------------------------------------------------------------------schedListeners
java.util.Vector schedListeners
SchedEvent modules.
-------------------------------------------------------------------------------manager
RM manager
Class unm.maui.sched.SchedEvent implements Serializable
Class unm.maui.sched.SchedException implements Serializable
-------------------------------------------------------------------------------Package unm.maui.server
Class unm.maui.server.CMD implements Serializable
Serialized Fields
reqnodes
int reqnodes
Requested nodes. LL-ism.
-------------------------------------------------------------------------------tasksPerNode
int tasksPerNode
Requested tasks-per-node. LL-ism.
-------------------------------------------------------------------------------totalTasks
int totalTasks
Total tasks. This should not be specified with #@node and #@tasks_per_node. This is both an
LL-ism and can be used by Maui.
--------------------------------------------------------------------------------
pfoley@platform.com
26
New Productivity Working Group
Architectural Area
Paul Foley
Platform Computing
March 2002
geometry
java.lang.String geometry
Maui Node/Slot geometry specification.
-------------------------------------------------------------------------------acct
java.lang.String acct
Job account.
-------------------------------------------------------------------------------user
java.lang.String user
The user.
-------------------------------------------------------------------------------uid
int uid
Posix UID.
-------------------------------------------------------------------------------group
java.lang.String group
The group.
-------------------------------------------------------------------------------gid
int gid
Posix GID.
-------------------------------------------------------------------------------jobtype
java.lang.String jobtype
JobType.
-------------------------------------------------------------------------------iwd
java.lang.String iwd
IWD.
-------------------------------------------------------------------------------duration
long duration
Job Duration.
-------------------------------------------------------------------------------qos
pfoley@platform.com
27
New Productivity Working Group
Architectural Area
Paul Foley
Platform Computing
March 2002
int qos
Requested QoS.
-------------------------------------------------------------------------------reqs
java.util.Hashtable reqs
Requirements string.
-------------------------------------------------------------------------------text
java.lang.String text
The entire CMD text.
-------------------------------------------------------------------------------envp
java.util.Vector envp
The entire environment.
Class unm.maui.server.RequestException implements Serializable
-------------------------------------------------------------------------------Package unm.maui.server.parser
Class unm.maui.server.parser.ParseException implements Serializable
Serialized Fields
specialConstructor
boolean specialConstructor
This variable determines which constructor was used to create this object and thereby affects the
semantics of the "getMessage" method (see below).
-------------------------------------------------------------------------------currentToken
Token currentToken
This is the last token that has been consumed successfully. If this object has been created due to
a parse error, the token followng this token will (therefore) be the first error token.
-------------------------------------------------------------------------------expectedTokenSequences
int[][] expectedTokenSequences
Each entry in this array is an array of integers. Each array of integers represents a sequence of
tokens (by their ordinal values) that is expected at this point of the parse.
-------------------------------------------------------------------------------tokenImage
java.lang.String[] tokenImage
pfoley@platform.com
28
New Productivity Working Group
Architectural Area
Paul Foley
Platform Computing
March 2002
This is a reference to the "tokenImage" array of the generated parser within which the parse error
occurred. This array is defined in the generated ...Constants interface.
-------------------------------------------------------------------------------eol
java.lang.String eol
The end of line string for this machine.
Class unm.maui.server.parser.TokenMgrError implements Serializable
Serialized Fields
errorCode
int errorCode
Indicates the reason why the exception is thrown. It will have one of the above 4 values.
-------------------------------------------------------------------------------Package unm.maui.wiki
Class unm.maui.wiki.Wiki implements Serializable
Serialized Fields
tcpPort
int tcpPort
TCP Port of Wiki Listener;
-------------------------------------------------------------------------------udpPort
int udpPort
UDP Port of Wiki Listener;
-------------------------------------------------------------------------------bufSize
int bufSize
Read/write buffer sizes.
-------------------------------------------------------------------------------wikiNodesHash
java.util.Hashtable wikiNodesHash
Hashtable of NodeID -> WikiNode.
-------------------------------------------------------------------------------wikiJobsHash
java.util.Hashtable wikiJobsHash
Hashtable of JobID -> Job.
-------------------------------------------------------------------------------hostHash
java.util.Hashtable hostHash
pfoley@platform.com
29
New Productivity Working Group
Architectural Area
Paul Foley
Platform Computing
March 2002
Hashtable of InetAddress -> hostname.
-------------------------------------------------------------------------------unknownHash
java.util.Hashtable unknownHash
Unknown jobs.
-------------------------------------------------------------------------------pollInterval
int pollInterval
Sleep interval in milliseconds
-------------------------------------------------------------------------------nodeTimeout
int nodeTimeout
Node timeout (in secs).
-------------------------------------------------------------------------------sockTimeout
int sockTimeout
Socket timeout (in secs).
-------------------------------------------------------------------------------prologLocal
java.lang.String prologLocal
Job prolog.
-------------------------------------------------------------------------------prologRemote
java.lang.String prologRemote
Job prolog.
-------------------------------------------------------------------------------epilogLocal
java.lang.String epilogLocal
Job epilog.
-------------------------------------------------------------------------------epilogRemote
java.lang.String epilogRemote
Job epilog.
-------------------------------------------------------------------------------stripFQDN
boolean stripFQDN
Strip FQDN.
pfoley@platform.com
30
New Productivity Working Group
Architectural Area
Paul Foley
Platform Computing
March 2002
-------------------------------------------------------------------------------multicast
boolean multicast
Do multicast to nodes?
-------------------------------------------------------------------------------multicastAddr
java.net.InetAddress multicastAddr
Multicast group.
-------------------------------------------------------------------------------multicastTTL
int multicastTTL
Multicast time to live.
-------------------------------------------------------------------------------udpReattempt
int udpReattempt
Reattempt sleep value.
-------------------------------------------------------------------------------udpBurst
int udpBurst
UDP busts.
-------------------------------------------------------------------------------prefix
java.lang.String prefix
Properties prefix.
-------------------------------------------------------------------------------purgeDelay
int purgeDelay
Seconds delay before purging a job from our tables.
This is needed for the delay between cancelling a job and having the remote node daemons
acknowlege it.
-------------------------------------------------------------------------------expectedResponseID
long expectedResponseID
-------------------------------------------------------------------------------responseHash
java.util.Hashtable responseHash
--------------------------------------------------------------------------------
pfoley@platform.com
31
New Productivity Working Group
Architectural Area
Paul Foley
Platform Computing
March 2002
respondedHash
java.util.Hashtable respondedHash
-------------------------------------------------------------------------------wikiNodes
WikiNode[] wikiNodes
Class unm.maui.wiki.WikiCMD implements Serializable
Serialized Fields
shell
java.lang.String shell
Shell.
-------------------------------------------------------------------------------env
java.lang.String env
Env.
-------------------------------------------------------------------------------exec
java.lang.String exec
Exec.
-------------------------------------------------------------------------------args
java.lang.String args
Args.
-------------------------------------------------------------------------------input
java.lang.String input
Input.
-------------------------------------------------------------------------------output
java.lang.String output
Output.
-------------------------------------------------------------------------------error
java.lang.String error
Error.
-------------------------------------------------------------------------------comment
java.lang.String comment
pfoley@platform.com
32
New Productivity Working Group
Architectural Area
Paul Foley
Platform Computing
March 2002
Comment.
-------------------------------------------------------------------------------log
java.lang.String log
Log.
-------------------------------------------------------------------------------message
java.lang.String message
Message.
-------------------------------------------------------------------------------prolog
java.lang.String prolog
ProLog.
-------------------------------------------------------------------------------epilog
java.lang.String epilog
EpiLog.
-------------------------------------------------------------------------------notify
java.lang.String notify
Notify.
-------------------------------------------------------------------------------upload
java.lang.String upload
Upload.
-------------------------------------------------------------------------------download
java.lang.String download
Download.
Class unm.maui.wiki.WikiJobUpdate implements Serializable
Serialized Fields
jobID
java.lang.String jobID
id.
-------------------------------------------------------------------------------user
java.lang.String user
pfoley@platform.com
33
New Productivity Working Group
Architectural Area
Paul Foley
Platform Computing
March 2002
user.
-------------------------------------------------------------------------------stdout
java.lang.String stdout
stdout.
-------------------------------------------------------------------------------stderr
java.lang.String stderr
stdout.
-------------------------------------------------------------------------------retcode
int retcode
Return code.
-------------------------------------------------------------------------------state
java.lang.String state
Job state.
Class unm.maui.wiki.WikiNode implements Serializable
Serialized Fields
updateTime
long updateTime
Epoch time node last updated (partial).
Class unm.maui.wiki.WikiNodeUpdate implements Serializable
Serialized Fields
state
java.lang.String state
Node state.
-------------------------------------------------------------------------------features
java.util.Hashtable features
Features hash.
Class unm.maui.wiki.WikiResponse implements Serializable
Serialized Fields
status
int status
response status.
zero is good news.
pfoley@platform.com
34
New Productivity Working Group
Architectural Area
Paul Foley
Platform Computing
March 2002
-------------------------------------------------------------------------------mesg
java.lang.String mesg
Message.
Class unm.maui.wiki.WikiUpdate implements Serializable
Serialized Fields
id
long id
The WikiUpdate ID.
This is automatically created.
-------------------------------------------------------------------------------responseID
long responseID
The response ID.
This is set to the id of the corresponding WikiComm (if so prompted).
-------------------------------------------------------------------------------ts
long ts
Timestamp in seconds.
-------------------------------------------------------------------------------host
java.lang.String host
The remote host.
-------------------------------------------------------------------------------Package unm.maui.wikid
Class unm.maui.wikid.AllocJob implements Serializable
Serialized Fields
type
java.lang.String type
-------------------------------------------------------------------------------uid
int uid
-------------------------------------------------------------------------------gid
int gid
-------------------------------------------------------------------------------numtasks
pfoley@platform.com
35
New Productivity Working Group
Architectural Area
Paul Foley
Platform Computing
March 2002
int numtasks
-------------------------------------------------------------------------------taskid
int taskid
-------------------------------------------------------------------------------user
java.lang.String user
Class unm.maui.wikid.CancelJob implements Serializable
Serialized Fields
jobID
java.lang.String jobID
The Job ID.
Class unm.maui.wikid.ShutdownNode implements Serializable
Class unm.maui.wikid.StartJob implements Serializable
Serialized Fields
jobID
java.lang.String jobID
The Job ID.
-------------------------------------------------------------------------------user
java.lang.String user
The user running the job.
-------------------------------------------------------------------------------type
java.lang.String type
Job type.
-------------------------------------------------------------------------------stdin
java.lang.String stdin
Standard input file.
-------------------------------------------------------------------------------stdout
java.lang.String stdout
Standard output file.
-------------------------------------------------------------------------------stderr
java.lang.String stderr
pfoley@platform.com
36
New Productivity Working Group
Architectural Area
Paul Foley
Platform Computing
March 2002
Standard error file.
-------------------------------------------------------------------------------log
java.lang.String log
Misc log file.
-------------------------------------------------------------------------------totalTasks
int totalTasks
Total number of tasks in the job.
-------------------------------------------------------------------------------tasks
java.lang.String tasks
Tasks allocation across all nodes of the job.
This is a whitespace separated list of the number of tasks per node. The node daemon should fit
the tasks into open slots on the node corresponding to localhost (found in the hosts variable).
Class unm.maui.wikid.Task implements Serializable
Serialized Fields
uid
int uid
UID to run as.
-------------------------------------------------------------------------------gid
int gid
GID to run as.
-------------------------------------------------------------------------------exec
java.lang.String exec
Executable.
-------------------------------------------------------------------------------args
java.lang.String[] args
Arguments.
-------------------------------------------------------------------------------envp
java.lang.String[] envp
Environment.
Class unm.maui.wikid.WikiComm implements Serializable
pfoley@platform.com
37
New Productivity Working Group
Architectural Area
Paul Foley
Platform Computing
March 2002
Serialized Fields
id
long id
The WikiComm ID.
This is automatically created.
-------------------------------------------------------------------------------ts
long ts
Timestamp in seconds.
-------------------------------------------------------------------------------hosts
java.lang.String hosts
The hosts we're sending this message to separated by whitespace.
The first host is the "master" for MPI-based jobs.
-------------------------------------------------------------------------------Package unm.util
Class unm.util.AssertionException implements Serializable
Class unm.util.BinaryTree implements Serializable
Serialized Fields
size
int size
The real size.
Class unm.util.Queue implements Serializable
Serialized Fields
tail
Queue.QueueNode tail
Tail of queue.
-------------------------------------------------------------------------------head
Queue.QueueNode head
Head of queue
-------------------------------------------------------------------------------count
int count
Class unm.util.Queue.QueueNode implements Serializable
Serialized Fields
next
pfoley@platform.com
38
New Productivity Working Group
Architectural Area
Paul Foley
Platform Computing
March 2002
Queue.QueueNode next
-------------------------------------------------------------------------------prev
Queue.QueueNode prev
-------------------------------------------------------------------------------key
Key key
Class unm.util.StringID implements Serializable
Serialized Fields
id
java.lang.String id
-------------------------------------------------------------------------------Overview Package Class Tree Deprecated Index Help
PREV NEXT FRAMES
NO FRAMES
--------------------------------------------------------------------------------
pfoley@platform.com
39