Legacy code deployment and execution in
GT3/GT4 Grids: experiences with
migrating GEMLCA from GT3 to GT4
Grid Execution Management for Legacy Code Architecture
Thierry Delaitre
Cavendish School of Computer Science
University of Westminster
T.Delaitre@wmin.ac.uk
www.cpc.wmin.ac.uk/GEMLCA
UK E-science OGSA Testbed
 test and evaluate OGSA/GT3 core
 evaluate performance, ease of deployment and use,
and interoperability
OGSA Testbed
Grid
University of Portsmouth
University of Westminster
University of Manchester
University of Reading
Daresbury, CCLRC
University of Southampton
SZTAKI (Hungary)
www.cpc.wmin.ac.uk/GEMLCA
The development team
Univ. of Westminster
MTA SZTAKI
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
1. Prof. Peter Kacsuk
2. Zoltan Farkas
3. Csaba Nemeth
4. Tamas Boczko
5. Jozsef Patvarczki
6. Gabor Hermann
7. Gergely Sipos
Prof. Peter Kacsuk
Prof. Stephen Winter
Dr. Thierry Delaitre
Dr. Gabor Terstyanszky
Ariel Goyeneche
Tamas Kiss
Noam Weingarten
Gabor Kecskemeti
Yonathan Zetuny
Szabolcs Illes
Peter Bulyaki
www.cpc.wmin.ac.uk/GEMLCA
Presentation Outline
1. GEMLCA objectives
2. Concept of GEMLCA
3. Experiences with migrating GEMLCA
4. GEMLCA client, the P-GRADE portal: DEMO
5. Conclusion – future work
www.cpc.wmin.ac.uk/GEMLCA
Legacy Applications
• Code from the past, maintained because it works
• Often supports business critical functions
• Not Grid enabled
What to do with legacy codes when utilising the Grid?
•
Bin them and implement Grid enabled
applications
•
Reengineer them
•
Port them onto the Grid with minimal
user effort
www.cpc.wmin.ac.uk/GEMLCA
GEMLCA – Grid Execution Management
for Legacy Code Architecture
Objectives
•
To deploy legacy code applications as
Grid services without reengineering the
original code and minimal user effort
•
To create complex Grid workflows
where components are legacy code
applications
•
To make these functions available from
a Grid Portal
www.cpc.wmin.ac.uk/GEMLCA
GEMLCA
GEMLCA
P-GRADE
Portal
Integration
Legacy applications in service-oriented Grids
Gemlca – PGrade portal approach: thin client
3rd generation Grids:
(OGSA: GT3, WSRF, gLite)
Web browser
Grid Site 1
P-GRADE
Portal Server
Desktop 1
Legacy applications
Web browser
Grid Site 2
Desktop N
Legacy applications
www.cpc.wmin.ac.uk/GEMLCA
Presentation Outline
1. GEMLCA objectives
2. Concept of GEMLCA
3. Experiences with migrating GEMLCA
4. GEMLCA client, the P-GRADE portal: DEMO
5. Conclusion – future work
www.cpc.wmin.ac.uk/GEMLCA
GEMLCA Concept
GEMLCA
Service
Client: to
apply
legacy code
Grid service
Legacy Code
Factory
Legacy Code
Process
Legacy
Code Job
Client: to
deploy
legacy code
Grid service
Grid Host
Environment
(GT3/GT4)
Compute
Servers
OGSA Container
MMJFS
Job Manager
(Condor/Fork)
Use the integrated
P-GRADE/GEMLCA
Portal for each
case
www.cpc.wmin.ac.uk/GEMLCA
Resource manager
deploys: a. LC, b.
GT3/4, c. GEMLCA
GEMLCA lifecycle – Job submission
Grid Service Client
Grid Service Client
Instance
GEMLCA Stubs
GEMLCA
Middleware
Client sign-on
GEMLCA Resource
List of LCGSs
GLCList
OGSA Grid
Middleware
Condor Cluster
Central Manager
Node
Grid
MapFile
Condor Daemon
LCGS Interfaces
GLCProcess
Local File
System
Admin Grid
Service Client
LCGSJob LCGSJob
OGSA Container
GEMLCA Stubs
GLCAdmin
MMJFS
GEMLCA File
Structure
Grid User
Proxy
Data
Fork/Condor
Job Manager
Grid FTP
www.cpc.wmin.ac.uk/GEMLCA
Node 01
Node 32
Job
Process
Job
Process
…..
GEMLCA lifecycle – Get results
Grid Service Client
Grid Service Client
Instance
GEMLCA Stubs
Client sign-on
List of LCGSs
LCGS Interfaces
LCGS status + Result
GEMLCA
Middleware
OGSA Grid
Middleware
Central Manager
Node
GEMLCA Resource
GLCList
Grid
MapFile
Local File
System
Admin Grid
Service Client
LCGSJob LCGSJob
OGSA Container
GEMLCA Stubs
GLCAdmin
MMJFS
GEMLCA File
Structure
Data
Condor Daemon
GLCProcess
Kill LCGS
Grid User
Proxy
Condor Cluster
Fork/Condor
Job Manager
Grid FTP
www.cpc.wmin.ac.uk/GEMLCA
Node 01
Node 32
Job
Process
Job
Process
…..
GEMLCA lifecycle – Deployments
Grid Service Client
Grid Service Client
Instance
GEMLCA Stubs
GEMLCA
Middleware
Client sign-on
GEMLCA Resource
List of LCGSs
GLCList
LCGS Interfaces
LCGS status + Result
OGSA Grid
Middleware
Central Manager
Node
Grid
MapFile
Local File
System
Admin Grid
Service Client
LCGSJob LCGSJob
OGSA Container
GEMLCA Stubs
GLCAdmin
MMJFS
Client sign-on
Grid User
Proxy
Data
Condor Daemon
GLCProcess
Kill LCGS
Legacy Code
Deployment
Condor Cluster
GEMLCA File
Structure
Fork/Condor
Job Manager
Grid FTP
www.cpc.wmin.ac.uk/GEMLCA
Node 01
Node 32
Job
Process
Job
Process
…..
Presentation Outline
1. GEMLCA objectives
2. Concept of GEMLCA
3. Experiences with migrating GEMLCA
4. GEMLCA client, the P-GRADE portal: DEMO
5. Conclusion – future work
www.cpc.wmin.ac.uk/GEMLCA
Migrating GEMLCA from GT3 to GT4:
What needs to be done ?
•
GEMLCA package for GT3 released on November 2004
•
Convert existing List, Process and Admin WS from OGSI to WSRF
www.cpc.wmin.ac.uk/GEMLCA
Experiences with migrating GEMLCA from GT3 to GT4
Advantages of GT4 over GT3 for GEMLCA
•
Better MMJFS performance, no UHE
•
Better performance when sending/receiving XML messages
•
GT4 documentation is better than in GT3 and continues to improve
How easy was it to convert from OGSI to WSRF ?
•
Straightforward to convert OGSI to WSRF class style
What were the difficulties from GT3 to GT4 ?
•
WSDL and complex types (3 slides)
•
Security (1 slide)
•
Gram API (1 slide)
www.cpc.wmin.ac.uk/GEMLCA
Experiences with migrating GEMLCA from GT3 to GT4
WSDL Interfaces
•
Native Axis java2wsdl not suitable for converting Java interfaces
to GT4 WSDL unless decorating is modified. WSDL for List and
Process WS done manually. WSDL for Admin WS generated
automatically from Java Interface once we added an XSLT
processing step after java2WSDL.
•
For the List and Process WS, we had to guess the expected WSDL
using a trial and error process. Original WSDL was inspired from
Borja’s tutorial (at the time of 3.9.3) and from the GWSDL.
•
We spent lots of time figuring out how to use complex types such
as passing multiple parameters or returning String []. Solution is
to use an Object (i.e. Beans). Objects also need to be used in 3.9.5
when returning String [] (was ok with 3.9.4)
www.cpc.wmin.ac.uk/GEMLCA
Experiences with migrating GEMLCA from GT3 to GT4
GLCList::getGLCList WS and complex types
WSDL generated manually for List WS
<xsd:element name=“getGLCListResponse”>
<xsd:complexType>
<xsd:sequence>
<xsd:element name=“legacyCodes” type=“xsd:string” minOccurs=“0” maxOccurs=“unbounded”/>
</xsd:sequence>
</xsd:complexType>
</xsd:element>
import uk.ac.wmin.cpc.gemlca.frontend.stubs.GLCList.GetGLCListRequest;
import uk.ac.wmin.cpc.gemlca.frontend.stubs.GLCList.GetGLCListResponse;
public class GLCList implements GLCListNamespaces {
public GetGLCListResponse getGLCList(GetGLCListRequest parameters)
www.cpc.wmin.ac.uk/GEMLCA
Experiences with migrating GEMLCA from GT3 to GT4
GLCAdmin::getPopulatedDtd WS and complex types
WSDL generated automatically for Admin WS
public interface GLCAdmin {
public String [] getPopulatedDtd(String LCName, boolean pUserLegacyCodes);
public boolean writeConfig(String [] EnvironmentDescription, boolean pUserLegacyCodes);
}
import uk.ac.wmin.cpc.gemlca.frontend.stubs.GLCAdmin.GetPopulatedDtd;
import uk.ac.wmin.cpc.gemlca.frontend.stubs.GLCAdmin.GetPopulatedDtdResponse;
public class GLCAdmin implements GLCAdminNamespaces {
public GetPopulatedDtdResponse getPopulatedDtd(GetPopulatedDtd Params) {
String LCName = Params.getIn0();
boolean pUserLegacyCode = Params.isIn1();
<element name=“getPopulatedDtd”>
<complexType>
<sequence>
<element name=“in0” type=“xsd:string”/>
<element name=“in1” type=“xsd:boolean”/>
</sequence>
</complexType>
</element>
www.cpc.wmin.ac.uk/GEMLCA
Experiences with migrating GEMLCA from GT3 to GT4
Security
•
•
Cannot do proxy delegation when client authorisation is set to
none. We experienced the GT3.2.0 to GT3.2.1 security
enhancement !
–
host/self auth not suitable
–
Using identity of service solution
Proxy delegation working first-time only after container start-up
–
There is a single instance of the stateless Web Service and hence we cannot use
SecurityManager.getManager().setServiceOwnerFromContext()
Iterator it=org.globus.gsi.jaas.JassSubject.getCurrentSubject().getPrivateCredential().iterator();
If(it.hasNext()) proxy = (GSSCredential) it.next();
job = new GramJob;
Job.setCredential(proxy);
www.cpc.wmin.ac.uk/GEMLCA
Experiences with migrating GEMLCA from GT3 to GT4
GRAM API
•
Tried to use low-level Gram API but we experienced some bugs
in the documentation which we fedback. Job status refresh was
not working at the beginning.
•
Ended up using GramJob API as in GT3.
•
Still cannot use notification call-backs when GramJob API is
embedded into a WS. (Currently using polling)
www.cpc.wmin.ac.uk/GEMLCA
What we’d like to see done in GT4
•
Proper detailed documentation (has improved but needs to continue)
•
GRAM call-back notification in a WS
•
Get Condor job manager working with Globus
–
•
•
PBS:
–
PBS not working even though it should work (PBS.pm uses SEG)
–
Decouple PBS master node and Globus Grid node on different machines (pbs tracejob ?).
Compilation issues on SLES9 AMD 64bits Opteron
–
•
Condor was not working properly for us in GT3 possibly due to NFS race conditions
(same reported by James Frey for PBS/LSF). Is it the same in GT4 ? What is the
recommended deployment scenario ? Can condor master and GT node installed on
different machines ? Is the condor’s SEG code reliable in GT4 ?
Tried using CVS but fait_accomplit fails
GT4 client not working on 64 bits OS using 32 or 64 bits JVM
Exception in thread "main" java.lang.NoSuchMethodError: org.apache.axis.description.TypeDesc.<init>(Ljava/lang/Class;Z)V
at org.apache.axis.message.addressing.EndpointReferenceType.<clinit>(EndpointReferenceType.java:333)
www.cpc.wmin.ac.uk/GEMLCA
Presentation Outline
1. GEMLCA objectives
2. Concept of GEMLCA
3. Server side
4. GEMLCA client – the P-GRADE portal: Live
DEMO
5. Conclusion – future work
www.cpc.wmin.ac.uk/GEMLCA
P-GRADE Grid Portal
PG-Portal functionalities
www.cpc.wmin.ac.uk/GEMLCA
Conclusions
•
GEMLCA enables the deployment of legacy code
applications as Grid services with minimal user effort.
•
GEMLCA is integrated with the P-GRADE portal to offer
user-friendly development and execution environment.
•
The integrated GEMLCA P-GRADE solution was
successfully demonstrated using the traffic simulator
and other applications
•
Gemlca package for GT3 released last November
•
Gemlca package for GT4 to be released in May 2005
www.cpc.wmin.ac.uk/GEMLCA
Thank you for your attention …
Any
questions?
Email:
gemlca-discuss@cpc.wmin.ac.uk
Websites:
www.cpc.wmin.ac.uk/gemlca
www.lpds.sztaki.hu/pgportal
www.cpc.wmin.ac.uk/GEMLCA