Monitoring and Discovery in
Globus Toolkit 3
Ben Clifford
USC/Information Sciences Institute
benc@isi.edu
The Traditional MDS2 architecture slide
R
R
R
R
?
R
dispersed users
?
network
R
R
R
R
R
R
R
?
?
R
R
R
R
R
R
R
R
VO-A

GRISes – represents a resource

GIISes – indexes multiple resources

LDAP protocols

OpenLDAP

Pull only
VO-B
2
MDS3

serviceData

Query mechanisms

XPath query support

Host status information

Index service

Hosting environment components
3
MDS3 is built on OGSI

Open Grid Services Infrastructure

Grid Service = Web Service + stuff

Key OGSI concept (as far as MDS is
concerned) is serviceData
4
serviceData

Every Grid Service can expose internal state as
serviceData elements
– An XML element of arbitrary complexity

Each service has a serviceData set
– The collection of serviceData Elements (SDEs)


Example: state of a host is exposed as an SDE by
GRAM.
Similar to MDS2 GRIS functionality, but in each
service (rather than once per host)
5
Retrieving service data

Pull – findServiceData operation (required)
– queryByServiceDataNames (required,
OGSI)
– queryByXPath (GT3)

Push – subscription/notification (optional)
– subscribeByServiceDataNames (optional,
OGSI)

Additional query types listed in well-known
SDEs: findServiceDataExtensibility and
subscribeExtensibility
6
By Name







queryByServiceDataNames,
subscribeByServiceDataNames
Parameter is list of SDE names
+ min/max frequency, sink, ttl for subscriptions
Returns entire SDEs (as return value or repeatedly
through subscription)
Conceptually very simple.
Query is guaranteed to be supported by any grid
service.
Disadvantage - If SDE large, the return value and
notifications are also large.
7
XPath query support

A W3C XML query language

Globus-defined query type
– Parameters are SDE name and XPath query
– Return value is result of evaluating XPath
query against specified SDE – a list of
matching elements.

Any service built on our OGSA core gets
support for this automatically.
– Based on Xalan XPath library.
8
XPath example

Query:
//Host[@Name=“dc-user.isi.edu]/ProcessorLoad

Result:
<ProcessorLoad Last1Min="00" Last5Min="00"
Last15Min="00“ />
9
Future query work

Other XML query languages?
– XQuery, XSLT (both based on XPath)

XPath subscription

Notification of partial SDE content?
– Don’t return entire SDE when only one
small part has changed
– Difficult problem for arbitrary XML data
when notification stream may be lossy.
10
Host status information

Similar to default MDS2 GRIS information
– Filesystems, CPU load, memory usage…

XML GLUE-like schema
– Represents clusters better than MDS2
schema

Single host providers included with GT3

Others working on Ganglia implementation
11
Host status example
<Cluster Name="pygar.isi.edu" UniqueID="pygar.isi.edu">
<SubCluster Name="pygar.isi.edu" UniqueID="pygar.isi.edu">
<Host Name="pygar.isi.edu" UniqueID="pygar.isi.edu">
<Processor
Vendor=" GenuineIntel" Model=" Intel(R) XEON(TM) CPU 2" Version="15.2.4"
ClockSpeed="2193" CacheL2="512"/>
<MainMemory VirtualSize="2047" RAMSize="1004" RAMAvailable="119"
VirtualAvailable="1716" />
<OperatingSystem Name="Linux" Release="2.4.7-10" Version="#1 Thu Sep 6 17:27:27 EDT
2001" />
<FileSystem Name="/“ Size="23510" AvailableSpace="650" Root="/" Type="unavailable"
ReadOnly="false“ />
<NetworkAdapter Name="eth0" IPAddress="128.9.72.46" InboundIP="True"
OutboundIP="True" MTU="1500"/>
<ProcessorLoad Last1Min="00" Last5Min="00" Last15Min="00“ />
</Host>
</SubCluster>
</Cluster>
12
Hosting Environment components
Pieces useful for writing your own information services:


Core OGSI support
Memory- or Xindice- based storage of serviceData
– Xindice provides persistent storage

Aggregator mechanism
– Receives notifications for other services and publishes
as service data.

Provider mechanism
– Run Java or unix-executable providers and publishes
output as service data.
13
Index Service

Take all of the components on the previous slide
and stick them all together.

Add a configuration mechanism.

Similar to MDS2 GIIS

Gathers serviceData from other grid services

Publishes collection through one service


Clients can query the index in the same way as
they can query any other service data.
Arbitrary XML data – no schema
14
Future collective level work


Various ideas for future collective level
services
Archive service
– Best way to store time-series of arbitrary
XML?
– Best way to query this archive?
– link to OGSA-DAI?
– Link to other archivers?
15
The End (unless there is time left over…)
16
Service Groups




A Service Group is a grid service that represents
some bag of other grid services
Contents of service group published as entry SDE
entry has a reference to the member service and
some arbitrary content data.
Examples:
– Registry
> Bag of services that have been registered
– ContainerRegistry
> Bag of services that are being run in the local hosting
environment
– Index service
> Bag of services that have their service data indexed.
17
Namespaces



All of the XML element names (for service
data, for query types) are qualified by XML
namespaces.
But I haven’t included them in these slides
for conciseness.
The Namespace Police will take me away.
18
Security


At the moment – ‘all or nothing’ for each
service
Investigating other ways to specify security
for this…
19