GT4 Experiences
Terry Harmer
ETF GT4 Evaluation
David McBride
LeSC
7 April 2005
Julie McCabe
BeSC
Globus Week @ NeSC
Anthony Stell
NeSC
1
GT3.9.2 Experiences
GT3.9.4 Experiences
GT3.9.5 (GT4 Beta 1) Experiences
GT4 Experiences
GT3.9.3 Experiences
7 April 2005
GT3.9.1 Experiences
Globus Week @ NeSC
2
GT3.9.5 Software
•
•
GT
–
–
–
–
–
GT
–
4.0 Common Runtime Components
Java WS Core
C WS Core
Python WS Core (contribution)
XIO
C Common Libraries
4.0 Security (GSI)
WS A&A
•
•
• Community Authorization Service
(CAS)
• Delegation Service
• Authorization Framework
• Message/Transport-level Security
– Credential Management
• MyProxy
• SimpleCA
– Utilities
GT
–
–
–
–
–
GT
–
4.0 Data Management
RFT
GridFTP
RLS
OGSA-DAI
Data Replication Service
4.0 Information Services
WS MDS (MDS4)
•
•
•
•
•
–
GT
–
–
Aggregator Framework
Index Service
Trigger Service
WebMDS
Pre-WS MDS (MDS2)
4.0 Execution Management
WS GRAM
Pre-WS GRAM
• GSI-OpenSSH
– Pre-WS Authentication &
Authorization
7 April 2005
Globus Week @ NeSC
3
What to Evaluate?
1. GRAM
David McBride @ LeSC
– Submitting, monitoring and managing jobs
on grid resources.
– SUN GridEngine
2. Security
Anthony Stell @ NeSC
3. GT3-GT4 Migration Terry Harmer
Julie McCabe @ BeSC
7 April 2005
Globus Week @ NeSC
4
GRAM
•
GT4 GRAM implementation has been substantially re-engineered to allow
for greater scalability both in terms of the number of jobs it can handle
at any given time and in terms of response time for any specific job.
•
•
A Scheduler Event Generator is responsible for continuously monitoring
the state of the cluster manager (eg by monitoring a logfile generated by
the cluster tools)
– SEG sends asynchronous state-change notifications when the state of
any job updates.
– This allows for a much faster response to job state changes
– Reduce the overhead involved in constantly polling the state of the
running system.
– Uses sudo which allows for secure, fine-grained privilege selection
rather than globus specific utilities.
A lightweight pre-compiled C application is used to drive the JobManager
modules directly - resulting in a much faster turnaround time and
substantially reduced resource requirements.
7 April 2005
Globus Week @ NeSC
5
What to Evaluate?
1. GRAM
David McBride @ LeSC
– Submitting, monitoring and managing jobs
on grid resources.
– SUN GridEngine
2. Security
Anthony Stell @ NeSC
3. GT3-GT4 Migration Terry Harmer
Julie McCabe @ BeSC
7 April 2005
Globus Week @ NeSC
6
Why did I choose
GT3-GT4 Migration?
• My experience is starting large-scale e-Science
projects with GT3 Alpha 1.
– All change…nothing is stable….. Déjà vu all over again
– We (BeSC) have no GT2 expertise or interest!
• I have a commercial interest
– Gridcast 300k Java GT3.0.2
– A prototype grid-based TV/Radio broadcasting infrastructure
developed with the BBC
• GT3.0.2 Test bed deployed since October 2003
– 2 Pilot user deployments within the BBC
• ie with real users doing real commercial things…
7 April 2005
Globus Week @ NeSC
7
Gridcast Test bed
Malvern
Belfast
Don’t know!
Processing
Clusters
BeSC
QinetiQ
IBM
BT Data Centre
Media Servers
Grid Service
Rack
JANET
Network
NGS
1 G b/s dedicated connection
1 Gb/s dedicated connection
BlueArc Server
Grid Service
Rack
Grid Service
Rack
London
BBC R&D
BBC NI
BlueArc Servers
Schedule
Server
Belfast
Presentation Suite
7 April 2005
Globus Week @ NeSC
8
Why did I choose
GT3-GT4 Migration?
And…
• I have a commercial interest
– Gridcast 300k Java GT3.0.2
– Genegrid 250k Java GT3.2
– Geddm
300k Java GT3.0.2
– RiskGrid
100k Java GT3.0.2
7 April 2005
Globus Week @ NeSC
9
Collection of
GT3.0.2 Services
BBC Northern
Ireland
Content
Services
Technical
Services
Control
Services
Schedule
Services
BBC Scotland
High
Bandwidth
IP Network
BBC Wales
Content
Services
Network
Management
Services
Schedule
Distribution
Services
Technical
Services
BBC Network
7 April 2005
Globus Week @ NeSC
10
A GT3.0.2 Service
Service
WSDL
Grid
Service
Java
Implementation
7 April 2005
Globus Week @ NeSC
11
A GT3.9.x Service
Service
WSDL
Web
Service
Implementation
Grid
Resource
Implementation
Resource
Home
Implementation
Implementation
7 April 2005
Globus Week @ NeSC
12
The Goal
Service
WSDL
Service
WSDL
Grid
Service
Java
Implementation
Web
Service
Implementation
Grid
Resource
Implementation
Resource
Home
Implementation
To convert WSDL and Service Implementation
–
–
–
–
7 April 2005
as systematically as possible
as quickly as possible
with as little effort as possible
with as little re-testing as possible
Globus Week @ NeSC
13
Migration of the WSDL
• Changes to the WSDL for GT3.9.x
– Include references to WS-.. Specifications
– Argument passing has changed and thus
service definitions need to change.
– Inclusion of the properties of the grid
resource.
7 April 2005
Globus Week @ NeSC
14
Changes to the WSDL for GT3.9.x
– Include references to WS-.. Specifications
<?xml version="1.0" encoding="UTF-8"?><definitions name="service-name"
targetNamespace="http://www.qub.ac.uk/gridcast/service-name"
xmlns="http://schemas.xmlsoap.org/wsdl/"
xmlns:tns="http://www.qub.ac.uk/gridcast/service-name"
xmlns:wsa="http://schemas.xmlsoap.org/ws/2004/03/addressing"
xmlns:wsdl="http://schemas.xmlsoap.org/wsdl/"
xmlns:wsrlw="http://docs.oasis-open.org/wsrf/2004/06/wsrf-WSResourceLifetime-1.2-draft-01.wsdl"
xmlns:wsrp="http://docs.oasis-open.org/wsrf/2004/06/wsrf-WSResourceProperties-1.2-draft-01.xsd"
xmlns:wsrpw="http://docs.oasisopen.org/wsrf/2004/06/wsrf-WS-ResourceProperties-1.2-draft-01.wsdl"
xmlns:wsdlpp="http://www.globus.org/namespaces/2004/10/WSDLPreprocessor"
xmlns:xsd="http://www.w3.org/2001/XMLSchema">
Not proved to be an issue
7 April 2005
Globus Week @ NeSC
15
Changes to the WSDL for GT3.9.x
– Argument passing has changed and thus service
definitions need to change.
– changes to WSDL and source
void refreshConfig (x provider, I index)
{
….
}
RefreshConfigRes refreshConfig (RefreshConfig param)
{
….
}
– Within BeSC we adopted this style for our GT3.x codebase and
so this change has not proven to be an issue for us.
– Annoying but actually more consistent and easier to manage.
7 April 2005
Globus Week @ NeSC
16
Changes to the WSDL for GT3.9.x
– Inclusion of the Resource Properties managed by the
grid service resource.
<xsd:element name=”myName” type=”xsd:string”/>
<xsd:element name=”index” type=”xsd:int”/
<xsd:element name=“FredResourceProperties">
<xsd:complexType>
<xsd:sequence>
<xsd:element ref="tns:myName“
minOccurs="1“maxOccurs="1"/>
<xsd:element ref="tns:index" minOccurs="1" maxOccurs="1"/>
</xsd:sequence>
</xsd:complexType>
</xsd:element>
• Too big a change…so we opted not to include resource properties.
7 April 2005
Globus Week @ NeSC
17
Migration of the WSDL
• Changes to the WSDL for GT3.9.x
– Include references to WS-.. Specifications √
– Argument passing has changed and thus
service definitions need to change. √
– Inclusion of the properties of the grid
resource. ×
We have a PERL script that does most of the work in the above.
7 April 2005
Globus Week @ NeSC
18
GT3.9.x Service Code
Service
WSDL
Service
WSDL
Grid
Service
Java
Implementation
•
•
•
Web
Service
Implementation
Resource
Implementation
Resource
Home
Implementation
Web service we (largely) auto-generate from the WSDL.
Resource implementation is the Grid Service Implementation
– We have not encountered that many significant changes here.
Resource Home Implementation is hand written.
–
Never more that 20 or so lines of Java
7 April 2005
Globus Week @ NeSC
19
The Web Service
• We use only to select the grid resource to
invoke….it acts as a dispatcher…
public DebugResponse debug(Debug msg)
{
return getResource().debug(msg);
}
• getResource is a method that selects the particular
grid service resource being addressed.
– Implementation supplied by globus
• debug is the grid service operation
7 April 2005
Globus Week @ NeSC
20
Implementation Structure
Base
Web
Service
Implementation
Web
Service
Implementation
Mostly original
GT3 service
7 April 2005
Base
Resource
Home
Implementation
Service
WSDL
Resource
Implementation
Generated
Resource
Home
Implementation
common
Globus Week @ NeSC
Hand written
21
Conversion issues
• GT3 Grid service may contain operations and libraries
that are no longer supported.
– Hasn’t proven to be that much of a problem.
• We have been lucky in that the new GT style is one that
we have followed.
– We packaged multiple arguments.
– We have a significant percentage of web service
code in our implementation.
• Gridcast code will be fully converted by the end of
April.
– The GT3.9.5 code is already in user deployments.
• BBCB BBC News 24 (White City/Media Centre London)
• BBC Network Deployment (BBCNI/BBCR&D/BT/BeSC)
7 April 2005
Globus Week @ NeSC
22
My Impressions….
• In comparison to GT3 releases …… GT3.9.x
releases
– Better planned, organised, documented
– Model has been consistent
– Releases are of a higher quality and contain
fewer bugs
7 April 2005
Globus Week @ NeSC
23
ETF Conclusions
• GT4 installation was straightforward with few
installation problems.
• GT4 installation documentation was of a high
quality, well structured and accurate.
• GT4 package descriptions were of a high
quality, well structured and accurate.
• GT4 simple services are relatively easy to
develop and well documented in a programmers
tutorial.
7 April 2005
Globus Week @ NeSC
24
ETF GT4 Conclusions
• GT4 service development requires the management of a
larger collection of files than GT3 but this not a
significant effort.
• GT4 conversion from GT3 required effort in changes to
service interfaces but required relatively few changes
in the service code in the examples considered.
GT4 services demonstrated significant improvements in
performance and reliability over their GT3 versions.
7 April 2005
Globus Week @ NeSC
25
However…….
• GT4 shortcoming currently is the lack of documentation
that will permit large-scale GT conversion, deployment
and GT4 development.
– Documentation appearing quickly!
• The creation of simple services is well documented but
the detail required for large-scale GT3 conversion or
new development is just starting to appear.
• The need for documentation and examples is recognised
by the Globus Alliance.
– This is also an opportunity for the UK?
7 April 2005
Globus Week @ NeSC
26