2D-PME method and REX-MS method
- Application of computational chemistry -
Masaaki Kawata
Grid Technology Research Center, AIST, JAPAN
m.kawata@aist.go.jp
National Institute of Advanced Industrial Science and Technology
Outline
Introduction
What is 2D-PME method ?
- how fast 2D-PME method is ! What is REX-MS method ?
- how efficient REX-MS method is ! Perspective
- combination of two methods -
National Institute of Advanced Industrial Science and Technology
Introduction
Task parallel calculation
ex. Parallel molecular dynamics or Monte
Carlo simulations.
Data parallel calculation
ex. optimum-pairing-search of drug
compounds (parameter survey)
National Institute of Advanced Industrial Science and Technology
Introduction(2)
Task parallel calculation
Given a conformation of target compound,
accurate estimation of physical quantities,
whose accuracy are comparable to
experiment, requires large amount of
computational resources.
⇒ parallel algorithm of molecular simulation
→ Parallel calculation of Coulomb interaction
＝2D-PME method
National Institute of Advanced Industrial Science and Technology
Introduction(3)
Data parallel calculation
Statistical interpretation should be included
in molecular simulation. But now we can’t do
statistical operation due to limited
computational resources.
Statistical operation requires extensive
survey of parameters space.
⇒ New strategy dealing with statistical
ensemble
→ REX-MS method
National Institute of Advanced Industrial Science and Technology
What is 2D-PME method ?
2D-PME method stands for Two-Dimensional
Particle Mesh Ewald method.
Fast and accurate method to calculate
Coulomb interaction in three-dimensional
systems with two-dimensional periodicity
(quasi-2D systems).
z
x
y
National Institute of Advanced Industrial Science and Technology
Why quasi-2D systems ?
Nano application
⇒surface of nano-structures
Bio application
⇒ membrane protein for
drug design
should be treated as
Quasi-2D system
National Institute of Advanced Industrial Science and Technology
What is quasi-2D system
H2O
H2O
Au
CH3(CH2)nS
z
y
x
(Left) Quasi two-dimensional simulation box, i.e., three-dimensional box with two-dimensional
periodicity in the (x, y) directions and with non-periodicity in the z direction. Original
particles are contained in the central box with lengths of the sides, Lx, Ly, and Lz in the x, y,
and z directions, respectively. (Right) Self-assembled monlayer membrane system. Images of
the simulation box are repeated in the (x,y) directions.
National Institute of Advanced Industrial Science and Technology
Coulomb interaction
7
kcal/mol
Coulomb interaction has longer tail
more than size of simulation box
→ Ewald method
qNa+ qCl−
6
5
r
4
3
2
More than 90 % of CPU time is
1
consumed for calculation of Coulomb 0
interactions
→ fast Ewald method
100
Acceleration of molecular simulations
⇒ Acceleration of Ewald method
National Institute of Advanced Industrial Science and Technology
200
300
400
Angstrom
500
2D Ewald method
1
Φ= ∑
2 h x ,h y
'
N
qi q j
N
∑∑ r − r
i =1 j =1
i
j
Φ kk ≠0
+ hx + h y
1
=
hx ×h y
Gx −1Gy −1 ∞
∑∑ ∑Q(t , t
t x =0 t y =0 t z =−∞
x
y
, t z ) × Ψ • Q(t x , t y , t z )
≈ Φr + Φkk ≠0 + Φkk =0 + Φs
α
Φ =−
π
s
Φ
k
k =0
N
[
N
∑q
2
i
i =1
= ∑ qi B Φkk =0 g , zi
]
i =1
(
N N
qi q j
1
'
Φ = ∑ ∑∑
erfc α rij + h x + h y
2 h x ,h y i =1 j =1 rij + h x + h y
r
)
National Institute of Advanced Industrial Science and Technology
2D-PME method
Φ
k
k ≠0
1
=
hx ×h y
Gx −1Gy −1 ∞
∑∑ ∑Q(t , t
t x =0 t y =0 t z =−∞
x
y
, t z ) × Ψ • Q(t x , t y , t z )
S (m ) =
N
∑q
j =1
Charge q
j
exp (− 2π i m ⋅ r j )
Contribution from each grid point: Ｑ
(k1,k2,k3)
S (m) ≅ C(m) ⋅ F(Q)(m)
National Institute of Advanced Industrial Science and Technology
Results(1)
System 1
System 2
System 3
National Institute of Advanced Industrial Science and Technology
Results(2)
CPU timea for a single-step MD calculation for a given accuracy, by using the
original method and the 2D-PME method.
System 1
System 2
System 3
Number of
charges
2928
2955
2817
Lx=Ly (Å)
32.15
22.29
46.35
Lz (Å)
32.15
66.87
15.45
Original Ewald
method (s)
1098.29
1455.58
1828.77
2D-PME
method(s)
2.38
6.52
2.86
Speedup
461.5
223.2
639.4
CPU time on Compaq Alpha Station XP1000 (Alpha21264 667MHz).
National Institute of Advanced Industrial Science and Technology
Flow chart of parallel MD calculations with
the 2D-PME method.
Update and redistribute r j and r& j
Calculate bonded interactions
r
r
Calculate Φ and F
together with van der Waals interactions
k
k
Calculate Φ k = 0and F k = 0
Construct u jfrom r ′j
Construct
M λ ( λ = x , y , z ) and their derivatives
Set Q ( t x , t y , t z )
Data transform as illustrated (forward)
forward
2D FFT and 1D Fourier integral
k
Calculate Φ k ≠ 0
~
~
Calculate Ψ ( m x , m y , m z ) ⋅ Q ( m x , m y , m z )
2D FFT and 1D Fourier integral
Data transform as illustrated (backward)
k
Calculate F k ≠ 0
Sum of all interactions
National Institute of Advanced Industrial Science and Technology
backward
Results (3)
Speedup factor of the parallel
MD calculations for the quasi-2D
systems by using the 2D-PME
method. The solid line is for
calculations with an SP switch
(300MB/sec BI-Direction) with
user space protocol, and the
dashed line is for calculations
with internet protocol. The
dotted line is the ideal speedup
factor, assuming an infinitely fast
network connecting the nodes.
60
Speedup factor
50
40
30
20
10
10
20
30
40
50
Number of processors
60
National Institute of Advanced Industrial Science and Technology
What is REX-MS method?
REX-MS stands for Replica Exchange
Molecular Simulation method.
Computationally efficient sampling method in
the phase space.
⇒ global optimization problem
suitable for grid environment
National Institute of Advanced Industrial Science and Technology
In REX-MS method
Replica１
Replica２
ReplicaN
1. Run N independent
simulation (N replicas) with
N different parameters,
respectively.
2. Exchange information
among replicas during the
simulation.
3. Search optimum solution
among N simulations.
Extended statistical ensemble
Sampling by using REX strategy is more efficient than
that by the sum of N independent simulations (not REX).
National Institute of Advanced Industrial Science and Technology
Two type of REX-MS methods
REX-MC method (by t.ikegami & h.takemiya)
Replica exchange Monte Carlo calculation
REX-MD method (by m.ito)
Replica exchange Molecular Dynamics toolkit
National Institute of Advanced Industrial Science and Technology
REX-MC (by t.ikegami)
Potential energy survey of molecules using
direct method (Combination of REX-MC with
ab-initio MO calculation）
⇒Superior to random walk survey of
complicated potential surface
National Institute of Advanced Industrial Science and Technology
Gridifying the program
Two levels of parallelization
Coarse grained: parallel montecarlo sampling
Fine grained: parallel ab-initio
energy calculation
Dynamic task scheduling,
machine reconfiguration
Bookkeeper
Reconfiguration request
Task scheduling for balancing Monitoring
Reconfiguration
load on a heterogeneous
computing resources
Machine scheduling for
reconfiguring machine sets on
the fly
Dynamic scheduling
REXMC client
Task allocation
Servers
meta-computing test bed
10 institutes/20 Supercomputers
ab initio calculation
National Institute of Advanced Industrial Science and Technology
HPC Challenge in SC2002
Metacomputing Test-bed
AIST
10 institutions (3 continentals) / 20 parallel computer (7 types)
High Performance Computing Center Stuttgart (HLRS),
Sandia National Laboratories (SNL),
Pittsburgh Supercomputing Center (PSC),
REXMC
Client Center (AIST),
REXMC Client
Grid Technology
Research
For C20 triplet
For C20 singlet
Manchester Computing Centre (MCC),
National Center for High Performance Computing (NCHC),
Japan Atomic Energy Research Institute (JAERI),
Korea Institute of Science and Technology Information (KISTI),
European Center of Parallelism in Barcelona (CEPBA/CIRI),
Bookkeeper
Finnish IT center for Science (CSC).
National Institute of Advanced Industrial Science and Technology
Potential Survey on the Metacomputing
testbed
Parallel execution of 1６（３２）
MC samples
Running over 150 hours with changing
machine configurations
Calculating 145 MC samples/hour using
860 CPU’s (at maximum)
Potential survey for C20 might be
completed in a month
(cf. > ~30 years on a single CPU)
Negligible Communication cost
< 1% of total time
Dynamic scheduling/configuration
mechanism is useful for
long time simulation
Simulation on the unstable environment
National Institute of Advanced Industrial Science and Technology
REMD toolkit (by m.ito)
A toolkit was developed to build a replica-exchange method
program suitable for solving the multiple-minima problem.
It is designed as an object-oriented framework to generate
variants of simulation programs by assembling the toolkit
components and force field programs.
The toolkit components provides the parallelization
mechanisms for various computational environments and the
sampling methods.
An arbitrary force field implementation can be plugged into
the toolkit to generate an executable program.
REMD toolkit
MPI
Serial
Grid
ＭＣ
Mindy(NAMD)
ＭＤ
Ｎｅｗ
Ｍｏｄｅｌ
National Institute of Advanced Industrial Science and Technology
Potential
Estimation of thermal distributions for molecular
structures is essential for elucidating biological
functionalities, e.g. a ligand-receptor binding.
Such estimation, however, is notoriously difficult
at low temperature because of a multiple-minima
problem.
Replica exchange method (REM) and replicaexchange molecular dynamics method (REMD) can
overcome the difficulty.
Each component of an REM/REMD algorithm, a
statistical ensemble, conformational sampling
method, and potential energy function has to be
customized to suit a particular biomolecular system.
Simulation programs are likely to adjust to various
parallelization environments.
An object-oriented framework can facilitate to
generate a variants of REM/REMD programs
suitable for various molecular systems and
different computational environments such as PC
clusters and Grid.
Component-based software development
Energy
Why do we need a software framework for
REM/REMD simulations?
National Institute of Advanced Industrial Science and Technology
構造座標
Efficiency of the REMD toolkit
The efficiency of the
generated program
(toolkit +
Mindy(NAMD)) was
examined by
estimzating the heat
capacity.
The error was found
to decay naturally
according to the
central limit theorem.
National Institute of Advanced Industrial Science and Technology
Perspective
Combination of 2D-PME method and REX-MS
method leads to fast, accurate and extensive
survey of the screening.
⇒higher throughput in drug design and in
material design of nano-structure.
Implementation of those methods on grid
environment brings new stage of the design
processes.
⇒ a promising application of grid
technology.
National Institute of Advanced Industrial Science and Technology
Reference
For more details about:
2D-PME, ask me or m.kawata@aist.go.jp
REX-MC, ask h.takemiya (here)
or mail to t-ikegami@aist.go.jp
REMD toolkit, mail to masakatsu-ito@aist.go.jp
National Institute of Advanced Industrial Science and Technology