Developing Scientific Applications
Using Standard Grid Middleware
Hiroshi Takemiya
Grid Technology Research Center
National Institute of Advanced Industrial Science and
Technology
National Institute of Advanced Industrial Science and Technology
Goal of the research
Examining and evaluating the effectiveness of grid
middleware by gridifying “real” applications
Computational Grid becomes feasible for running apps.
Many kinds of middleware has been provided
Several pioneering works succeeded in running apps.
Little information on how to “gridify” legacy apps.
How easily the application can be gridified?
How efficiently the application can be executed on the grid?
The information will be valuable for;
application programmers + middleware providers
Climate simulation, Molecular Simulation, Fluid simulation,
Virtual Screening, Astronomical Virtual Observatory …
Globus Toolkit, UNICORE, MPICH-G, Ninf-G …
Grid infrastructure
National Institute of Advanced Industrial Science and Technology
Weather Forecasting System
Predicting short- to middle-term weather change
Based on Barotropic S-model
Proposed by Dr. Tanaka
Legacy FORTRAN program
Simple and precise
Treating vertically averaged quantities
Solving shallow water equations
1989/1/30-2/12
2D simulation
150 sec for 100 days prediction/1 simulation
Succeeded in reproducing characteristic phenomenon
Distribution of jet streams
Blocking phenomenon of high atmospheric pressure
National Institute of Advanced Industrial Science and Technology
Ensemble Simulation
Keep high precision over long period
Taking a statistical ensemble mean
Introducing perturbation for each simulation
Requires100 ~ 1000 sample simulations
Gridifying the program enables quick response
Sample simulation １
perturbation
Time evolution (Leap Frog method)
perturbation
Sample simulation 2
Statistical result
Observational data
Time evolution (Leap Frog method)
…
perturbation
100 ~ 1000 simulations
Sample simulation N
perturbation
Time evolution (Leap Frog method)
National Institute of Advanced Industrial Science and Technology
Ninf-G Library at a glance
Reference implementation of GridRPC
Standardized at GGF Grid RPC WG
Providing RPC semantics on the grid
Suited for gridifying task parallel programs
Providing asynchronous RPC call as well
Built on top of Globus Toolkit
GRAM: invocation of server programs
Globus I/O, GASS: communication between C-S
MDS: managing stub information
GSI: authentication between C-S
4. connect
back
Client
1. interface
request
3. invoke
Executable
2. interface
reply
http://ninf.apgrid.org/
Globus I/O
GASS
IDL Compiler
generate
GRAM
MDS
GSI
Numerical
Library Server side
IDL
FILE
Client side
Remote Executable
fork
retrieve
Interface
Information
LDIF File
Assigning
independent
tasksIndustrial
on distributed
resources
National
Institute of Advanced
Science and Technology
Advantages of using Ninf-G library (1)
Construct grid applications easily
Hiding complicated mechanism of the grid
Based on familiar RPC semantics
Providing support tools for programming
ns_client_gen: client program compiler
ns_gen: stub generator
Client side
Server Server side
IDL
Server
Program
source
FILE
Client Program
Client
main(){
4. connect
back
Ninf-G
for(i
= 0; i < task_no; i++)
Client Library
Globus I/O
GASS
remote_call(){
IDL Compiler
Processing();
generate
grpc_call_async
3. invoke
Executable
(dest[i], “remote_call”,
2. interface args);
1. interface
reply
grpc_wait_all(dest);
}
request
Return;
GRAM
MDS
}
Server stub
Remote Executable
invoke
retrieve
Interface
Information
LDIF File
Requires
no detailed GSI
knowledge on Grid infrastructure
http://ninf.apgrid.org/
National Institute of Advanced Industrial Science and Technology
Advantages of using Ninf-G library (2)
Write once, run everywhere
Based on standard API
Many libraries (NetSolve, DIET, OmniRPC…) with
the same API
cf. MPI vs. NX library
Constructed on the most popular grid
middleware
Robust and flexible applications can be
constructed
Dynamic server program invocation
Recovering failures in establishing connections,
allocating tasks
National Institute of Advanced Industrial Science and Technology
Gridifying the Weather Forecasting System
Gridifying 2 parts of the program
Simulation part
Visualization part
Executed in a pipelined fashion
Accessing through the GridLib portal
Ninf-g
S-model Program
Reading Data
Grid
Lib
user
Solving Equations
Averaging results
Visualizing Results
Ninf-g
National Institute of Advanced Industrial Science and Technology
Ninf-g
Ninf-g
Ninf-g
Behavior of the System
server
grpc_call
client
GridLib
Sim.
Server
Globus-job-run
Grid
Lib
user
Gass-url-copy
National Institute of Advanced Industrial Science and Technology
Vis.
Server
Current Status of the System
Deployed on the ApGrid Testbed
Client program: cluster in AIST
Server program: 10 clusters in 9 institutes
Succeeded in giving a demonstration
CCGrid Conference: using 110 CPU’s
PRAGMA Workshop: using 183 CPU’s
Weather Simulation
Server Program
16 CPUs
TSUKUBA U, Japan
Visualization
Program
Weather Simulation
Server Program
64 CPUs
AIST，Japan
Weather Simulation
Server Program
16 CPUs
TITECH, Japan
APGrid Testbed
Weather Simulation
Server Program
16 CPUs
Weather Simulation
Client Program
TITECH, Japan
GridLib
Weather Simulation
Server Program
10 CPUs
156 CPUs
AIST，Japan
Osaka U, Japan
Weather Simulation
Server Program
16 CPUs
Doshisha U, Japan
Weather Simulation
Server Program
Weather Simulation
Server Program
Weather Simulation
Server Program
Weather Simulation
Server Program
15 CPUs
32 CPUs
16 CPUs
64 CPUs
HKU, HK
NCHC, Taiwan
KISTI, Korea
KU，Thailand
National Institute of Advanced Industrial Science and Technology
Lessons Learned (1)
How easily the application could be gridified?
Gridifying the app using Ninf-G was very easy!
13 days work, 300 lines modification
Most of the program modification can be performed on a single
computer.
Replacing a local call to remote one is straightforward.
Creating Server program is automated.
1. Specifying the interface of a server function.
2. Eliminating data dependence between C-S.
3. Inserting Ninf-G functions into the client program.
4. Creating server stubs
10 days work
On a single computer
3 days work
On a grid environment
National Institute of Advanced Industrial Science and Technology
Lessons Learned (2)
How was the performance of the application?
Attaining good performance is difficult
Initialization/termination cost are large
Modification of application, as well as middleware is needed
Large look-up cost of MDS
Long polling period of GRAM
Blocking of Ninf-G initialization/termination function
KISTI
Termination
Initialization
DU
AIST
0
200
400
600
800
1000
1200
1400
elapsed
time (sec)
National Institute of Advanced Industrial Science and Technology
Performance Result of Executing 200 Sample Simulations
Executing an optimized program
Bypassing MDS lookup
Modifying GRAM source to reducing polling period
Multi-threading application to avoid blocking
…
KISTI
DU
AIST
0
200
400
600
800
1000
1200
1400
elapsed time (sec)
National Institute of Advanced Industrial Science and Technology
Future Work
Middleware level
Improving Ninf-G based on knowledge gained
through the work
Ninf-G2 will be released in Nov. 2003
Designing task farming API on Ninf-G
difficult to implement efficient, robust, flexible
apps
Hiding scheduling, error recovery, multi-threading
mechanisms
Application level
Checking the scalability of middleware/apps.
Using more than 500 CPUs
National Institute of Advanced Industrial Science and Technology
National Institute of Advanced Industrial Science and Technology