Community grid infrastructures
for geosciences and materials
modelling
Martin Dove
University of Cambridge and
National Institute for Environmental eScience
NIEeS
http://www.niees.ac.uk
Hello and thank you: NIEeS
National Institute for Environmental eScience
Partnership between:
aiming to support the development of escience work
within the UK environmental science community
centre for national capability for escience within the UK
environmental sciences community
NIEeS
http://www.niees.ac.uk
Hello and thank you: NIEeS
National Institute for Environmental eScience
Stuart Ballard
Ian Frame
Gen-Tao Chiang
Therese Williams
NIEeS
http://www.niees.ac.uk
Hello and thank you: NIEeS
National Institute for Environmental eScience
Training events (eg Google
maps, XML from Fortran)
Source of expertise,
capability & information
Grid-enabling geoscience
and environmental sciences
applications
NIEeS
http://www.niees.ac.uk
New initiatives: eGY
We are one of the UK representatives for the
Electronic Geophysical Year 2007–8
“We can achieve a major step
forward in geoscience capability,
knowledge, and usage throughout
the world for the benefit of humanity
by accelerating the adoption of
modern and visionary practices for
managing and sharing data and
information.”
NIEeS
http://www.niees.ac.uk
Hello and thank you: eMinerals
eMinerals project
NERC-funded escience project involving Cambridge,
Bath, UCL, Royal Institution, CCLRC Daresbury, Birkbeck
Developing the concept of community grids to support
collaborative working in molecular-scale simulations
Emphasis on grid computing, data management,
information delivery and collaborative working
Core applications in understanding pollutants in the
environment at the atomic scale
NIEeS
http://www.niees.ac.uk
Hello and thank you: eMinerals
Toby White, Kat Austen, Andrew
Walker, Emilio Artacho, Peter
Murray-Rust (Cambridge)
Rik Tyer, Kerstin Kleese (CCLRC)
Steve Parker, Arnaud Marmier,
Corinne Arrouvel (Bath)
Ismael Bhana (Reading)
NIEeS
http://www.niees.ac.uk
Scales of geosciences and
environmental sciences
From global ...
… to molecular
NIEeS
http://www.niees.ac.uk
Our view of eScience
eScience refers to new science opportunities
that require distributed collaborations and are
enabled by emerging internet technologies.
These technologies include grid computing,
distributed data management and
collaborative tools.
Many tools are still in the process of rapid
development, and in some cases standards
are not yet established.
NIEeS
QuickTime™ and a
TIFF (Uncompressed) decompressor
are needed to see this picture.
QuickTime™ and a
TIFF (Uncompressed) decompressor
are needed to see this picture.
QuickTime™ and a
TIFF (Uncompressed) decompressor
are needed to see this picture.
http://www.niees.ac.uk
Our view of eScience
Computing grids
Data grids
Collaborative grids
NIEeS
http://www.niees.ac.uk
Implicit in the grid area
Use of wide-area compute grids, with
tools to handle authentication and the
job life cycle
Transfer of data into and out of the
grid environment, data sharing,
information delivery
Collaboration tools such as the
Access Grid for video conferencing;
data and information sharing
NIEeS
http://www.niees.ac.uk
Example: adsorption of dioxin molecules
on clay surfaces
+
Combinatorial study requiring grid computing,
data management and collaborative tools
NIEeS
http://www.niees.ac.uk
eScience: “Science beyond the lab
book”
Management of many tasks
Management of the resultant
data deluge
Sharing the information
content with collaborators
Stretching science beyond human limitations whilst
maintaining accuracy and accountability
NIEeS
http://www.niees.ac.uk
eMinerals science: Molecular-scale
environmental issues
Radioactive
waste disposal
Pollution: molecules and atoms
on mineral surfaces
Crystal dissolution and
weathering
Crystal growth and
scale inhibition
NIEeS
http://www.niees.ac.uk
eMinerals science: Molecular-scale
environmental issues
Radioactive
waste disposal
Pollution: molecules and atoms
on mineral surfaces
Crystal dissolution and
weathering
Crystal growth and
scale inhibition
NIEeS
http://www.niees.ac.uk
Using the “Virtual Organisation” model in
environmental science
A comprehensive assault on the issue of transport
of pollutants in the environment
Heavy metal poisonous waste
Toxic organic molecules
Nuclear waste encapsulation
NIEeS
http://www.niees.ac.uk
Collaborative science and the eMinerals
Virtual Organisation
Level of
theory
Natural organic matter
Sulphides
Oxides/hydroxides
Phosphates
Carbonates
Large empirical
models
Aluminosilicates
Linear-scaling quantum
mechanics
Clays, micas
Quantum Monte
Carlo
Organic molecules
Nitrates
Adsorbing
surface
Contaminant
NIEeS
http://www.niees.ac.uk
Collaborative science and the eMinerals
Virtual Organisation
Project team has
different simulation
skills
Level of
theory
Organic molecules
Nitrates
Natural organic matter
Sulphides
Oxides/hydroxides
Phosphates
Carbonates
Large empirical
models
Aluminosilicates
Linear-scaling quantum
mechanics
Clays, micas
Quantum Monte
Carlo
Project team has
different expertise in
the science
Together we pool
all essential skills and
Adsorbing
surface expertise
Contaminant
NIEeS
http://www.niees.ac.uk
So what does our Virtual Organisation
need?
Computing grid infrastructure for running large-scale
simulation studies
Easy-to-use tools for managing large-scale
combinatorial computational studies
Ability to share data between collaborators
Ability to extract and share information content
Grid-enabling of simulation codes
Means to communicate effectively (NOT email!)
NIEeS
http://www.niees.ac.uk
Components of the compute grid
High-throughput and highperformance clusters
Pools of individual machines
linked together by “Condor”
Authentication & authorisation, and job
submission, handled by “Globus”
NIEeS
http://www.niees.ac.uk
Parallel (HPC)
clusters
Access to external
facilities and grids Campus
grids
Data
vault
Compute
clusters
Data
vault
Condor
JobMgr
Globus
Internet
Cluster
JobMgr
Data
vault
Desktop
pools
Data
vault
Globus
Condor
JobMgr
Globus
Researcher
Application
server
Cluster
JobMgr
Globus
Data grid: the San Diego Storage
Resource Broker
Metadata
catalogue
Internet
Distributed file
management
NIEeS
Distributed data vaults
http://www.niees.ac.uk
SRB client tools
Unix Scommands (eg Sput, Sls,
Scd, Sget)
Web interface
GUI for MS Windows (InQ)
NIEeS
http://www.niees.ac.uk
Running jobs on our minigrid: Job
workflow
The scientist places input files and application
executables into the SRB
The scientist submits the job to a grid resource
The job downloads the files from the SRB
The calculations are performed
The job places all output files into the SRB
Metadata and core information are collected from
the output files
The scientist retrieves data files from the SRB and/or
core information from the metadata store
NIEeS
http://www.niees.ac.uk
Data
vault
Researcher
7. Researcher
interacts with
the metadata
database to
extract core
output values
Application
server
1. Upload data files
and application to
data vault
2. Submit job to
minigrid via MCS
5. Metadata is sent to
the application server
3. Data files and
application are
transferred to the
grid resource
6. Output files
are transferred to
the data vault
4. Job runs on
grid compute
resources
User interface: my_condor_submit tool
Executable
= ossia2004
pathToExe
= /home/rty.eminerals/OSSIA2004preferredMachineList = lv1.nwgrid.ac.uk-serial dl1.nw-grid.ac.uk-serial
jobType
= performance
numOfProcs
= 1
Output
= trans.out
Sdir
= /home/mdv.eminerals/RMCSdemo
Sget
= *
Sput
= *
GetEnvMetadata = true
RDesc
= Test sweep of temperature using ossia
RDatasetID
= 263
AgentX
= Temperature,trans.xml:/ParameterList[title='Initial
System']/Parameter[name='Temperature']
AgentX
= Energy,trans.xml:/PropertyList[last]/Property[title='Energy']
AgentX
= OrderParameter,trans.xml:/Module[last]/Property[title='Order
parameter']
AgentX
= HeatCapacity,trans.xml:/Module[last]/Property[title='Heat
capacity']
AgentX
=
Susceptibility,trans.xml:/Module[last]/Property[title='Susceptibility']
NIEeS
http://www.niees.ac.uk
User profile
Users do not want portals: portals are
for tools, not for the working
environment
QuickTime™ and a
TIFF (Uncompressed) decompressor
are needed to see this picture.
Users do not want their applications
pre-wrapped as services: they want to
have complete control over their
applications, e.g. to add capability
Users do not want a
provider/consumer model: that does
not provide the freedom they need
NIEeS
http://www.niees.ac.uk
Data sharing: the need for information
delivery tools
Classical
molecular
dynamics
methods
NIEeS
QuickTime™ and a
TIFF (Uncompressed) decompressor
are needed to see this picture.
QuickTime™ and a
TIFF (Uncompressed) decompressor
are needed to see this picture.
Quantum
mechanical
methods
http://www.niees.ac.uk
Collaborative grid: Data and information
sharing
NIEeS
http://www.niees.ac.uk
Data and information sharing: XML data
representation
<?xml version="1.0" encoding="UTF-8"?>
Chemical Markup Language
<cml convention="FoX_wcml-2.0" fileId="cis1.cml"
version="2.4" xmlns="http://www.xml-cml.org/schema">
<metadataList name="Metadata">
<metadata name="LeadProgramAuthor" content="Martin Dove"/>
<metadata name="Code name" content="ossia"/>
Capturing metadata
...
</metadataList>
<module title="Initial System" dictRef="emin:initialModule">
<parameterList>
<parameter dictRef="ossia:temperature" name="Temperature">
<scalar dataType="xsd:double” units="cmlUnits:eV">1.000000000000e-1</scalar>
</parameter>
<parameter dictRef="ossia:NumberOfSteps" name="Number of steps">
<scalar dataType="xsd:integer" units="units:countable">10000000</scalar>
</parameter>
...
Capturing initial parameters
</parameterList>
</module>
...
<module title="Finalization" dictRef="emin:finalModule">
<propertyList>
<property dictRef="ossia:Energy" title="Energy">
<scalar dataType="xsd:double" units="cmlUnits:eV">2.052516362912e-1</scalar>
</property>
Capturing computed properties
...
</propertyList>
</module>
</cml>
NIEeS
http://www.niees.ac.uk
What XML gives us
Simulation code output that is self-describing (no more
mere lists of numbers!)
XML files can be transformed to give user-centric and
information-centric representations of data, including
plotted data
XML files can have key information extracted easily,
essential for large combinatorial studies
XML enables automatic capture of metadata, and
metadata is essential for managing data
NIEeS
http://www.niees.ac.uk
XML → metadata
Our job submission tools
automatically harvest metadata
from our output XML files
We have developed a new set of
tools to access the metadata
database (“RCommands”)
We use metadata for locating data and datasets
created by our colleagues
We also use metadata for extracting core information
from data – useful for analysing combinatorial studies
NIEeS
http://www.niees.ac.uk
Collaborative grids
Classical
molecular
dynamics
methods
NIEeS
QuickTime™ and a
TIFF (Uncompressed) decompressor
are needed to see this picture.
QuickTime™ and a
TIFF (Uncompressed) decompressor
are needed to see this picture.
Quantum
mechanical
methods
http://www.niees.ac.uk
Researcher A
Data
vault
Upload XML data files to data vault
for sharing with collaborator
Project wiki
Instant
messaging
Annotate data
with metadata
Web 2.0
Access Grid
with JMAST
View information
content of data
files using ccViz
Using Rgem to share
simulation outputs
Application
server
Locate data
from metadata
Researcher B
Example: Compressibility of amorphous
silica
Density is not quite linear:
note that the gradient is
larger in the middle of the
plot than at either end.
Bulk modulus (BM):
dP
B  V
BM has minimum
dV around 2
GPa: compressibility = 1/B
has maximum

NIEeS
http://www.niees.ac.uk
Example: Compressibility of amorphous
silica
Molecular dynamics
simulations of pressuredependence of
amorphous silica
55
BKS 512
BKS 512 fit
BKS 4096A
BKS 4096A fit
BKS 4096B
BKS 4096B fit
TTAM 512
TTAM 512 fit
TTAM 4096A
TTAM 4096A fit
TTAM 4096 B
TTAM 4096 B fit
Volume (Å3/SiO2)
50
45
Volume curve shows
that silica gets softer
around 2 GPa
Negative derivative
defines the
compressibility
40
35
-5
-3
-1
1
3
5
Pressure (GPa)
NIEeS
http://www.niees.ac.uk
The message from this example
We had to run over 600 sets of
simulations and analyses ...
... each generating around 2 GB data
We used grid computing to run the
simulations using our tools, and our data
management system to extract and share
the key data
And part of this was run by a third-year
project student, Lucy
NIEeS
http://www.niees.ac.uk
Example: Dioxin molecules on layer
silicate neutral surfaces
+
Combinatorial study requiring grid computing,
data management and collaborative tools
NIEeS
http://www.niees.ac.uk
Range of molecule/substrate systems
Dioxins on pyrophyllite:
ΔE = 0.25 eV
Dioxins on quartz:
ΔE = 0.25 eV
Furans on pyrophyllite:
ΔE = 0.22 eV
Dioxins on calcite:
ΔE = 0.14 eV
Calculating how the binding energy
varies across the molecular series
from “all chlorine” to “no chlorine”
NIEeS
http://www.niees.ac.uk
Summary
The tools I have discussed can be used for many
different application domains within the geosciences
and environmental sciences
Emphasis on all core
areas: compute, data and
collaborative grids
Computing
grids
Data
grids
NIEeS
Collaborative
grids
http://www.niees.ac.uk