Enabling Grids for E-sciencE
Supporting legacy code
applications on EGEE VOs
by GEMLCA and
the P-GRADE portal
P. Kacsuk*, G. Sipos* and T. Kiss**
*MTA SZTAKI
** Univ. of Westminster
www.eu-egee.org
INFSO-RI-508833
Properties of the P-GRADE Porta
Enabling Grids for E-sciencE
• What is P-GRADE portal?
– General purpose, graphical, workflow-oriented Grid portal
– Supports the development and execution of workflow-based Grid
applications
– Components of the workflows can be
 Sequential jobs
 Parallel jobs (MPI, PVM)
 Legacy code (GEMLCA) services
– Easy porting of legacy and other applications to the Grid
– Enables the exploitation of two levels of parallelism
 intra-job
 inter-job
– Enables to control and observe the execution of the workflow
applications
– Enables fault-tolerant workflow execution
INFSO-RI-508833
Properties of the P-GRADE Porta
Enabling Grids for E-sciencE
• Enables application-specific extensions by new
portlets (E-GRID, Traffic simulation Grid, etc.)
• Hides the low-level Grid details
• Enables the access of every important feature of the
underlying Grid, e.g.:
– brokering service
– information service, etc.
• Can serve as a bridge between different Grids to
solve the Grid interoperability problem at the
workflow level:
– Enables easy porting of legacy and other applications
between various Grids (e.g. from LCG to gLite, from GT2 to
GT4, etc.)
– Hides middleware changes for the end-user
– Enables parallel execution of a workflow in several Grids
INFSO-RI-508833
Properties of the P-GRADE Porta
Enabling Grids for E-sciencE
• Enables application-specific extensions by new
portlets (E-GRID, Traffic simulation Grid, etc.)
• Hides the low-level Grid details
• Enables the access of every important feature of the
underlying Grid, e.g.:
– brokering service
– information service, etc.
• Can serve as a bridge between different Grids to
solve the Grid interoperability problem at the
workflow level:
– Enables easy porting of legacy and other applications
between various Grids (e.g. from LCG to gLite, from GT2 to
GT4, etc.)
– Hides middleware changes for the end-user
– Enables parallel execution of a workflow in several Grids
INFSO-RI-508833
P-GRADE portal in a nutshell
Enabling Grids for E-sciencE
Proxy management
Grid resources
management
Workflow creation
Job mapping to Grid
resources
Workflow management
and
execution visualization
INFSO-RI-508833
Two levels of parallelism by the
P-GRADE workflow
Enabling Grids for E-sciencE
•
Semantics of the workflow
enables two levels of
parallelism:
– Parallel execution
inside a workflow
node
– Parallel execution
among workflow
nodes
•
The P-GRADE Portal
workflow concept enables the
efficient parallelization of
complex problems
Multiple jobs can
run parallel
INFSO-RI-508833
The job can be a
parallel program
OGSA-DAI
Integration with P-GRADE
Grid Execution
Management for Legacy
Code Applications
Tamas Kukla and Tamas Kiss
Centre for Parallel Computing
University of Westminster
T.Kukla@student.westminster.ac.uk, kisst@wmin.ac.uk
www.cpc.wmin.ac.uk/GEMLCA
OGSA-DAI portlets
4 portlets implemented
1. Data Service Manager
2. Database Browser
3. Query Manager
4. Data Manipulation
www.cpc.wmin.ac.uk/GEMLCA
Data Service Manager portlet
Add/remove OGSADAI services
List
available
resources
of a
service
List
available
tables
Select
and
describe
tables
www.cpc.wmin.ac.uk/GEMLCA
Database Browser portlet
Select
service /
resource /
table
Display
the
selected
table or
XML
resource
Order by
the
selected
column
www.cpc.wmin.ac.uk/GEMLCA
Query Manager portlet
Display results on screen
Run
query on
selected
database
Display
results
See next
slide
www.cpc.wmin.ac.uk/GEMLCA
Query Manager portlet
Deliver results to file
Define
query
statement
Define
target
location
www.cpc.wmin.ac.uk/GEMLCA
Data can be
sliced up
and
compressed
before
delivery
Data Manipulation portlet
Manipulate data of selected database
Select
database
Define
update
statement
www.cpc.wmin.ac.uk/GEMLCA
Update
See next
slide
Data Manipulation portlet
Deliver results from file to database
Select
target
service /
resource
Define
target
table
Define
files to be
delivered
www.cpc.wmin.ac.uk/GEMLCA
Extracts
and
merges
A possible user scenario
•
The new OGSA-DAI portlet allows to browse an OGSA-DAI
database and deliver query results to a set of files.
•
These files can be used in normal or parameter study workflows as
input ports.
•
The workflow output can be converted and transferred back to the
database.
1.
2.
3.
4.
Select the data service and resource
Run a query and deliver results to a (set of) file(s)
Run the (parameter study) workflow on the data-sets
Transfer the results back into an OGSA-DAI database
www.cpc.wmin.ac.uk/GEMLCA
Step1: Select the OGSA-DAI service and
DataBase resource with the DataBrowser
User
List of resources
Storage resource
(local or remote)
Request
eXist XML
DataBase
resource
OGSA-DAI
service
Request
Extended PG-Portal
Workflow
Oracle DataBase
resource
DataBrowser
List of resources
www.cpc.wmin.ac.uk/GEMLCA
Step1: Select the OGSA-DAI service and
DataBase resource with the DataBrowser
Select service
and resource
www.cpc.wmin.ac.uk/GEMLCA
Step2: Run a query and deliver
results to a (set of) file(s)
User
Storage resource
(local or remote)
Query
OGSA-DAI
service
Results
Extended PG-Portal
Workflow
eXist XML
DataBase
resource
Query
Query
Oracle DataBase
resource
DataBrowser
Results
Results
www.cpc.wmin.ac.uk/GEMLCA
Step2: Run a query and deliver
results to a (set of) file(s)
Define
query
statement
Deliver results to a
set of files
www.cpc.wmin.ac.uk/GEMLCA
Step3: Run (parameter study) workflow
(this step does not use the OGSA-DAI portlets!)
User
Storage resource
(local or remote)
Input
data
Create
workflow
Workflow
output
OGSA-DAI
service
Extended PG-Portal
Workflow
eXist XML
DataBase
resource
Oracle DataBase
resource
DataBrowser
www.cpc.wmin.ac.uk/GEMLCA
Step4: Transfer results back into
an OGSA-DAI database
Storage resource
(local or remote)
User
Workflow
output
Confirmation
eXist XML
DataBase
resource
Request
OGSA-DAI
service
Workflow
output
Confirmation
Extended PG-Portal
Workflow
Workflow
output
Oracle DataBase
resource
DataBrowser
Confirmation
www.cpc.wmin.ac.uk/GEMLCA
Step4: Transfer results back into
an OGSA-DAI database
Define
target
database
Define
target
table
Define
files to be
transferred
and
converted
www.cpc.wmin.ac.uk/GEMLCA