Development
of
Parallel Coupler
The Fujin
Takashi Arakawa
arakawa@tokyo.rist.or.jp
Research Organization for Information
Science & Technology
General Meaning of Fujin
• A god of wind
• Inscribed in
Chinese/Japanese Kanji
characters
風
神
wind
god
Painted by Khorin Ogata
God of Wind
• Very popular name in Japan
– Used in various fields
Our Fujin
• Framework for meteorological model(s)
• Objectives
– parallel computing, domain nesting, coupling
– with minimal effort and minimal time
• Information hiding
– Modular and Layered Structure
– Fortran 90 employed
What can you do with the
Fujin?
• Parallelization
– Two Dimensional domain decomposition
• Domain Nesting
– Two way nesting
– No limitation for nesting depth
– Each coupled models can have different
nesting depth
• Coupling
– Multi-model coupling
Limitations of the Fujin
• Structured Grid based model only
– FEM, Spectral Model etc. are not
supported
• No implementation of interpolation
scheme for Nested Domains/Coupled
Models
– interpolation scheme is meteorological
issue
• not concerned problem for the Fujin
– Fujin provides only the simple interface
Structure of the Fujin
• 25 modules, 450 procedures
• Three layers
– Parallel package layer
• parallel library package using
MPI
– Data layer
• definition of data structure
• procedures for the data
– Interface layer
• Utility modules
– Error handling,logging,file
management
existing
model
Interface
Data Definition
Parallel
Package
Detail of the Interface Layer
• Coupler
– task control
– flux exchange
• Data Definition
Model Routines
Data
definition
– definition of the global
variables
Driver
Driver
Coupler
• Driver
Model Routines
– Initialization, Integration,
Result output
– Inserting procedures of
existing model
Grid Structure
(ni+1,nj+1)
(ni,nj)
j_end+1
j_end
j_start
j_start
(1,1)
(1,1)
i_start
i_start
i_end
i_end+1
• User is required to define the location of
each global variable in a initialization
subroutine
call InitData(pot_tmp,”potential temperature”, SCALAR_POINT)
Domain Nesting (1)
Time Integration Scheme
• Two way self-nesting
①
Domain 1
②
⑤
Domain 2
③ ④ ⑥ ⑦
Domain 3
Time
Domain Nesting (2)
Data Exchange Scheme
• Effective data
exchange scheme
①
1
3
②
2
1
2
3
4
③
④
4
①
Interpolation
①
Coupling
Task Control Strategy
• Each model is attached to one of these tasks
– Models in the same task are computed serially
– Models in the different tasks are computed
parallelly
Coupler
• Models in the same
task
 must have the same
number of domain
 Having a different
decomposition pattern
is allowed
Task 1
Model 1
Model 2
Task 2
Task 3
Model 3
Model 4
Coupling (2)
data exchange scheme
• Same as the scheme of domain nesting
• Type-D ← similar to MPCCI of GMD
• Difference from MPCCI is coupling strategy
C O
A C
C O
A C
C O
A C
C O
Connecting the Model
• Target model is CReSS
– non-hydrostatic model developed by
Nagoya Univ. and RIST
 over 85% for 64 PEs
 Characteristic of
CReSS, not the Fujin
100
parallel efficiency
• Parallel Efficiency
80
60
40
20
0
1
10
number of processors
100
Connecting the Model (2)
• Domain Nesting Test
– Three dimensional mountain wave
– Three domains
– Interpolation and boundary condition←Meso-NH
Future Plan
• Final goal
– To release the Fujin as free software
• Program structure is still complex
• Document