Production Cyberenvironment
for a
A Computational Chemistry Grid
PRAGMA13, NCSA
26 Sep 07
Sudhakar Pamidighantam
NCSA, University of Illinois at
Urbana-Champaign
sudhakar@ncsa.edu
National Center for Supercomputing Applications
Acknowledgements
National Center for Supercomputing Applications
Outline
• Historical Background Grid
Computational Chemistry
• Production Environments
• Current Status Web Services
• Usage (Grid and Science
Achievements)
• Brief Demo
• Future
National Center for Supercomputing Applications
Motivation
Software
- Reasonably Mature and easy to use to
address chemists questions of interest
Community of Users
- Need and capable of using the software
Some are non traditional computational
chemists
Resources
- Various in capacity and capability
National Center for Supercomputing Applications
Background
Qauntum Chemistry Remote Job Monitor
( Quantum Chemistry Workbench)
1998, NCSA
Chemviz
1999-2001, NSF (USA)
http://chemviz.ncsa.uiuc.edu
Technologies
Web Based Client Server Models
Visual Interfaces
Distributed computing (Condor)
National Center for Supercomputing Applications
GridChem
NCSA Alliance was commissioned 1998
Diverse HPC systems deployed
both at NCSA and Alliance Partner Sites
Batch schedulers different at sites
Policies favored different classes and modes of
use at different sites/HPC systems
National Center for Supercomputing Applications
Extended TeraGrid Facility
www.teragrid.org
National Center for Supercomputing Applications
NSF Petascale Road Map
• Track I Scheme Multi petaflop single site system to be
deployed by 2010
Several Consortia Competing (Now under review)
• Track 2 Sub petaflop systems
Several to be deployed until Track 1 is online
First one will be at TACC ( 450 TFlops) Available Fall 2007
( 50 000 Processors/Cores)
NCSA is deploying a 110 TFlops in April 2007
(10000 Processors/cores)
Second subpetaflops systems being reviewed
National Center for Supercomputing Applications
Grid and Gridlock
Alliance lead to Physical Grid
Grid lead to TeraGrid
Homogenous Grid with predefined fixed software and
system stack was planned (Teragrid) but it was difficult
to keep it homogenous
Local preferences and diversity leads to
heterogeneous grids now!
(Operating Systems, Schedulers, Policies, Software and Services)
Openness and standards that lead interoperability are
critical for successful services
National Center for Supercomputing Applications
Current Grid Status
Interfaces
Grid Hardware
Scientific
Applications
Middleware
National Center for Supercomputing Applications
User Community
Chemistry and Computational Biology
User Base
Sep 03 – Oct 04
NRAC
AAB
Small Allocations
------------------------------------------------------------#PIs 26
23
64
#SUs 5,953,100
1,374,100
640,000
National Center for Supercomputing Applications
National Center for Supercomputing Applications
Some User Issues Addressed by
the new Services
• New systems meant learning new commands
• Porting Codes
• Learning new job submissions and monitoring
protocols
• New proposals for time (time for new proposals)
• Computational modeling became more popular
and number of users increased (User
Management)
• Batch queues are longer / waiting increased
• Finding resources where to compute
complicated - probably multiple distributed sites
• Multiple proposals/allocations/logins
• Authentication and Data Security
• Data management
National Center for Supercomputing Applications
Computational Chemistry Grid
This is a Virtual Organization
Integrated Cyber Infrastructure for
Computational Chemistry
Integrates Applications, Middleware, HPC
resources, Scheduling and Data
management
National Center for Supercomputing Applications
Resources
System (Site)
Intel Cluster (OSC)
Procs
Avail
36
Total CPU
Hours/Year
Status
315,000 SMP and Cluster
nodes
HP Integrity
Superdome
(UKy)
33
290,000
IA32 Linux Cluster
(NCSA)
64
560,000 Allocated
Intel Cluster (LSU)
1024
IBM Power4 (TACC)
16
Teragrid (Multiple
Institutions)
2-10000
TB Replaced with
an SMP/
Cluster nodes
1,000,000 Allocated
140,000 Allocated
250,000 New Allocations
Expected
The initial Acesss Grid Testbed Nodes (38) and Condor SGI resources
(NCSA, 512 nodes) have been retired this year.
National Center for Supercomputing Applications
Other Resources
Extant HPC resources at various
Supercomputer Centers (Interoperable)
Optionally Other Grids and Hubs/local/personal
resources
These may require existing
allocations/Authorization
National Center for Supercomputing Applications
National Center for Supercomputing Applications
GridChem System
user
user
application
user
user
Portal Client
user
application
Grid Middleware
Proxy Server
Grid Services
Grid
http://www.nsf.gov/awardsearch/showAward.do?AwardNumber=0438312
National Center for Supercomputing Applications
Mass
Storage
Applications
• GridChem supports some apps already
– Gaussian, GAMESS, NWChem, Molpro, ADF,QMCPack,
Amber
• Schedule of integration of additional software
–
–
–
–
–
–
ACES-3
Crystal
Q-Chem
Wein2K
MCCCS Towhee
Others...
National Center for Supercomputing Applications
GridChem User Services
Allocation Request
https://www.gridchem.org/allocations/comm_form.php
National Center for Supercomputing Applications
GridChem User Services
Consulting Ticketing System
User View
National Center for Supercomputing Applications
GridChem User Services
Consulting Ticketing System
https://www.gridchem.org/consult/
Consultants View
National Center for Supercomputing Applications
Gridchem Middleware Service
(GMS)
National Center for Supercomputing Applications
GrdiChem Web Services
Quick Primer
Web Services is different from Web Page Systems or Web Servers:
There is no GUI
Web Services Share business logic, data & processes
through APIs with each other (not with user)
Web Services describe Standard way of interacting with “web based” applications
XML is used to tag the data,
SOAP is used to transfer the data,
WSDL is used for describing the services available and
UDDI is used for listing what services are available.
A client program connecting to a web service can read the WSDL
to determine what functions are available on the server. Any special
datatypes used are embedded in the WSDL file in the form
of XML Schema. Universal Description, Discovery, and Integration.
WSRF Standards Compliant.
National Center for Supercomputing Applications
GridChem Web Services
Client Objects  Database Interaction
Client
DTO (Data Transfer Object)
Serialize transfer through XML
DAO (Data Access Object)
How to get the DB objects
hb.xml (Hibernate Data Map)
describes obj/column data mapping
Business
Model
DTO
DAO
WS
Resources
Objects
Hibernate
hb.xml
Database
National Center for Supercomputing Applications
GridChem Data Models
Users
Projects
Resources
UserProjectResource
Users
userID
projectID
resourceID
loginName
SUsLocalUserUsed
Jobs
Resources
jobID
jobName
userID
projID
softID
cost
Resources
SoftwareResources
ComputeResources
resoruceID
Type
hostName
IPAddress
siteID
NetworkResources
StorageResources
National Center for Supercomputing Applications
Computational Chemistry Resource
National Center for Supercomputing Applications
GMS_WS Use Cases
http://www.gridchem.org:8668/space/GMS/usecase
•
•
•
•
•
•
Authentication
Job Submission
Resource Monitoring
Job Monitoring
File Retrieval
…
National Center for Supercomputing Applications
GridChem Web Services Operations
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
GetResourceProperty
SetTerminationTime
Destroy
Create
Login
LoadVO
RetrieveFiles
LoadFiles
DeleteFiles
LoadParentFiles
RefreshFiles
MakeDirectory
SubmitJob
SubmitMultipleJobs
PredictJobStartTime
KillJob
HideJob
UnhideJob
UnhideJobs
DeleteJob
FindJobs
GetJobStatus
RetrieveJobOutput
RetrieveNextDataBlock
StopFileAction
GetUserPreferences
PutUserPreferences
National Center for Supercomputing Applications
GMS_WS Authentication
http://www.gridchem.org:8668/space/GMS/usecase
GridChem Client
GMS
Contact GMS
Creates Session, Session RP and EPR
Sends EPR ( Like a Cookie, but more than that)
Login Request
(username:passwd)
Validates, Loads UserProjects
Sends acknowledgement
•
•
•
Retrieve UserProjects
(GetResourceProperty
Port Type [PT])
WSDL (Web Service Definition Language) is a language for describing how to interface
with XML-based services. It describes network services as a pair of endpoints operating
on messages with either document-oriented or procedure-oriented information.
The service interface is called the port type
WSDL FILE: <?xml version="1.0" encoding="UTF-8"?> <definitions name=“GMS"
targetNamespace=http://www.gridchem.org/gms " xmlns="http://schemas.xmlsoap.org/wsdl/" …
National Center for Supercomputing Applications
GMS_WS Authentication
http://www.gridchem.org:8668/space/GMS/usecase
GridChem Client
Selects project
LoadVO port type
(w. MAC address)
GMS
Verifies user/project/MACaddr
Load UserResources RP
Sends acknowledgement
Retrieve UserResources
[as userVO/ Profile]
(GetResourceProperty
port Type PT)
Validates, Loads UserProjects
Sends acknowledgement
National Center for Supercomputing Applications
GMS_WS Job Submission
GC Client
GMS
Create Job object
PredictJobStartTime PT
+ JobDTO
If decision OK,
SubmitJob PT
+ JobDTO
JobStart Prediction RP
Create Job object
API—Submit
Store Job Object
Send
Acknowledgement
PT = portType
RP = Resource Properties
DTO = Data Transfer Object
Need to check to make sure allocation-time is available.
Submission
CoGKit
GAT
“gsi-ssh”
Completion:
Email from
batch system
to GMS server
cron@GMS DB
National Center for Supercomputing Applications
GMS_WS Monitoring
GC Client
Request for Job,
Resource Status
Alloc. Balance
GMS
UserResource RP
Updated from DB
Resources/Kits/DB
cron@GMS server
cron@HPC Servers
Job Launcher Notifications
VO Admin email
parses email  DB
(status + cost)
Send info
Parse XML,
Display
PT = portType
RP = Resource Properties
DTO = Data Transfer Object
DB = Data Base
Discover Applications (Software
Resources)
Monitor System
Monitor Queues
National Center for Supercomputing Applications
GMS_WS Job Status
GC Client
Job Status
GMS
jobDTO.status
Resources/Kits/DB
Job
Launcher
Status
Update
Estimate Start
time
Scheduler
emails/
notifications
Notifications: Client, email, IM
National Center for Supercomputing Applications
GC Client
GMS_WS File Retrieval (MSS)
LoadFile PT
GetResourceProperty PT
FileDTO(?)
LoadFile PT
(project folder+job)
RetrieveFiles PT
(+file rel.path)
GetResourceProperty PT
GMS
MSS query
UserFiles RP +
FileDTO object
Resources/Kits/DB
Job Completion:
Send Output to MSS
Retrieve Root Dir.
Listing on MSS with
CoGKit or
GAT or
“gsi-ssh”
Validates project
folder owned by user.
Send new listing
API file request
Store locally
Create FileDTO
Load into UserData RP
Retrieve file:
CoGKit or
GAT or
“gsi-ssh”
PT = portType
RP = Resource Properties
DTO = Data Transfer Object
MSS = Mass Storage System
National Center for Supercomputing Applications
GMS_WS File Retrieval
GC Client
RetrieveJobOutput PT
(+JobDTO)
GMS
Resources/Kits/DB
Job Record from
DB.
Running: from Resource
Complete: from MSS
Create FileDTO (?)
Load into UserData RP
Retrieve file:
CoGKit or
GAT or
“gsiftp”
GetResourceProperty PT
PT = portType
RP = Resource Properties
DTO = Data Transfer Object
MSS = Mass Storage System
National Center for Supercomputing Applications
GridChem Web Services
WSRF (Web Services Resource Framework) Compliant
WSRF Specifications:
WS-ResourceProperties (WSRF-RP)
WS-ResourceLifetime (WSRF-RL)
WS-ServiceGroup (WSRF-SG)
WS-BaseFaults (WSRF-BF)
%ps -aux | grep ws
/usr/java/jdk1.5.0_05/bin/java \
Logging Configuration
-Dlog4j.configuration=container-log4j.properties \
Where to find Globus
-DGLOBUS_LOCATION=/usr/local/globus \
-Djava.endorsed.dirs=/usr/local/globus/endorsed \
-DGLOBUS_HOSTNAME=derrick.tacc.utexas.edu \
-DGLOBUS_TCP_PORT_RANGE=62500,64500 \
Where to get random seed
-Djava.security.egd=/dev/urandom \
for encryption key generation
-classpath /usr/local/globus/lib/bootstrap.jar:
/usr/local/globus/lib/cog-url.jar:
Classpath (required jars)
/usr/local/globus/lib/axis-url.jar
org.globus.bootstrap.Bootstrap org.globus.wsrf.container.ServiceContainer -nosec
National Center for Supercomputing Applications
GridChem Software Organization
Open Source Distribution
• CVS for GridChem
National Center for Supercomputing Applications
GMS_WS
• Package:
org.gridchem.service.gms
National Center for Supercomputing Applications
GMS_WS
+
National Center for Supercomputing Applications
Should these each be a separate package?
gms
client
dao
dto
exceptions
model
credential
file
file.task
job
job.task
notification
resource
user
persistence audit
query
synch gpir
test
GMS_WS
Classes for WSRF service implementation (PT)
Cmd line tests to mimic client requests
Data Access Obj – queries DB via persistent classes (hibernate)
Data Transfer Obj – (job,File,Hardware,Software,User) XML
How to handle errors (exceptions)
CCG Service business mode (how to interact)
Contains user’s credentials for job sub. file browsing,…
“Oversees correct” handling of user data (get/putfile).
Define Job & util & enumerations (SubmitTask, KillTask,…)
Autonomous notification via email, IM, textmesg.
CCGResource&Util, Synched by GPIR, abstract classes
NetworkRes., ComputeRes., SoftwareRes., StorageRes., VisualizationRes.
User (has attributes – Preference/Address)
DB operations (CRUD), OR Maps, pool mgmt,DB session,
Classes that communicate with other web services
Periodically update DB with GPIR info (GPIR calls)
JUnit service test (gms.properties): authen. VO retrieval,
Res.Query,Synch, Job Mgmt, File Mgmt, Notification
util
crypt
enumerators
gat
proxy
Contains utility and singleton classes for the service.
Encryption of login password
Mapping from GMS_WS enumeration classes DB
GAT util classes: GATContext & GAT Preferences generation
National Center for Supercomputing Applications
Classes deal with
CoGKit configuration.
GMS_WS external jars
• Testing
• For XML Parsing
• “Java” Document
Object Model
– Lightweight
– Reading/Writing XML
Docs
– Complements SAX
(parser) & DOM
– Uses Collections**
National Center for Supercomputing Applications
GridChem Resources Monitoring
http://portal.gridchem.org:8080/gridsphere/gridsphere?cid=home
National Center for Supercomputing Applications
GridChem Resources
New Computing Systems
System
Capacity (Cpus/Cores)
Capability
Mercury(NCSA)
1774
Small/Large Parallel Runs
Abe(NCSA)
9600
Massively Parallel Runs
DataStar(SDSC)
2368
SharedMemory Large
Runs
Bluegene/L(SDSC)
3456
Cluster Large Parallel
Runs
TeragridCluster(SDSC)
564
Small/Large Parallel Runs
BigRed(IU)
1024
SharedMemory
Small/Large Runs
BCX (UKy)
1360
Shared/Distributed
Memory small/Large
Parallel Runs
National Center for Supercomputing Applications
Application Software Resources
Currently Supported
Suite
Version
Location
Gaussian 03
C.02/D.01
Many Platforms
MolPro
2006.1
NCSA
NWChem
5.0/4.7
Many Platforms
Gamess
Jan 06
Many Platforms
Amber
8.0
Many Paltforms
QMCPack
2.0
NCSA
National Center for Supercomputing Applications
GridChem Software Resources
New Applications
Integration Underway
•
•
ADF
Amsterdam Density Functional Theory
Wien2K
Linearized Augemented Plain wave (DFT)
•
•
•
•
CPMD
QChem
Aces3
Gromacs
•
•
NAMD
Molecular Dynamics
DMol3
Periodic Molecular Systems ( Quantum
Chemistry)
Castep
Quantum Chemistry
MCCCS-Towhee Molecular Confirmation Sampling (Monte Carlo)
Crystal98/06
Crystal Optimizations (Quantum Chemistry)
….
•
•
•
•
Car Parinello Molecular Dynamics
Molecular Energetics (Quantum Chemistry)
Parallel Coupled Cluster Quantum Chemistry
Nano/Bio Simulations (Molecular Dynamics)
National Center for Supercomputing Applications
GridChem User Services
• Allocation
https://www.gridchem.org/allocations/index.shtml
Community and External Registration
Reviews, PI Registration and Access Creation
Community User Norms Established
• Consulting/User Services
https://www.gridchem.org/consult
Ticket tracking, Allocation Management
• Documentation, Training and Outreach
https://www.gridchem.org/doc_train/index.shtml
FAQ Extraction, Tutorials, Dissemination
Help is integrated into the GridChem client
National Center for Supercomputing Applications
Users and Usage
• 242 Users under 128 Projects
Include Academic PIs, two graduate
classes
And about 15 training users
More than a 442000 CPU Wallhours
since Jan 06
More than 10000 Jobs processed
National Center for Supercomputing Applications
Science Enabled
• Azide Reactions for Controlling Clean Silicon Surface
Chemistry: Benzylazide on Si(100)-2 x 1
Semyon Bocharov et al..
J. Am. Chem. Soc., 128 (29), 9300 -9301, 2006
• Chemistry of Diffusion Barrier Film Formation: Adsorption
and Dissociation of Tetrakis(dimethylamino)titanium on
Si(100)-2 × 1
Rodriguez-Reyes, J. C. F.; Teplyakov, A. V.
J. Phys. Chem. C.; 2007; 111(12); 4800-4808.
• Computational Studies of [2+2] and [4+2] Pericyclic
Reactions between Phosphinoboranes and Alkenes. Steric
and Electronic Effects in Identifying a Reactive
Phosphinoborane that Should Avoid Dimerization Thomas
M. Gilbert* and Steven M. Bachrach Organometallics, 26 (10),
2672 -2678, 2007.
National Center for Supercomputing Applications
Science Enabled
• Chemical Reactivity of the Biradicaloid (HO...ONO) Singlet
States of Peroxynitrous Acid. The Oxidation of
Hydrocarbons, Sulfides, and Selenides. Bach, R. D et al. J.
Am. Chem. Soc. 2005, 127, 3140-3155.
• The "Somersault" Mechanism for the P-450 Hydroxylation
of Hydrocarbons. The Intervention of Transient Inverted
Metastable Hydroperoxides. Bach, R. D.; Dmitrenko, O. J. Am.
Chem. Soc. 2006, 128(5), 1474-1488.
• The Effect of Carbonyl Substitution on the Strain Energy of
Small Ring Compounds and their Six-member Ring
Reference Compounds Bach, R. D.; Dmitrenko, O. J. Am.
Chem. Soc. 2006,128(14), 4598.
National Center for Supercomputing Applications
GridChem Client
Download Statistics
http://download.gridchem.org/usage/
National Center for Supercomputing Applications
Distribution of GridChem User Community
National Center for Supercomputing Applications
Job Distribution
Job Distribution by Time
140
300
120
250
100
Number of Jobs
350
200
150
100
80
60
40
50
20
Wall Clock x CPUs (~SUs)
Wall Clock Time x CPUs (~SUs)
National Center for Supercomputing Applications
50
48
46
44
42
40
38
36
34
32
30
28
26
24
22
20
18
16
14
12
8
10
6
4
0
2
2900
2700
2500
2300
2100
1900
1700
1500
1300
1100
900
700
500
300
0
100
Number of Jobs
Job Distribution by Time
System Wide Usage
HPC System
Usage (SUs)
Tungsten(NCSA)
5507
Copper(NCSA)
86484
CCGcluster(NCSA)
55709
Condor(NCSA)
30
SDX(UKy)
116143
CCGCluster(UKy)
.5
Longhorn(TACC)
54
CCGCluster(OSC)
62000
TGCluster(OSC)
36936
Cobalt(NCSA)
2485
Champion(TACC)
11
Mike4 (LSU)
14537
National Center for Supercomputing Applications
GridChem
Client Enhancements
• New Molecular Editor
JMolEditor (ANU) Integration
• VMD Is integrated
• Nanotube Generator (Tubegen) Will be
available
• Gamess Graphical User Interphase
National Center for Supercomputing Applications
Java Molecular Editor
• JMolEditor
Three Dimensional
Visual with Java 3D
Intuitive Molecule
Manipulation
Interactive Bond, Angle and Dihedral Settings
A Gaussian input generator Interface
National Center for Supercomputing Applications
Nanotube Generator:Tubegen
Cell Types
: Doren Research Group at the UniversityCrystal
of Delaware
Courtesy
Output Formats
National Center for Supercomputing Applications
GridChem Gamess GUI
National Center for Supercomputing Applications
GridChem Post Processing
• IR/Raman Spectra now accessible from G03, MolPro,
NWChem and Gamess Suites
VCD/ROA To be Included
National Center for Supercomputing Applications
GridChem Post Processing
• Normal Mode Viewing in 3D VRML
• Other Spectra With MO Integration
NMR
Electronic Spectra
National Center for Supercomputing Applications
GridChem Usability
Dynamic Information
National Center for Supercomputing Applications
GridChem Usability
• Information on Potential Start and
End Time for a given set of Job
parameters
• Automated Resource Selection
• Possible Job Migration In case of
dropped nodes or incomplete job
• Monitoring Multiple Jobs
• Automated Monitoring Job Output
National Center for Supercomputing Applications
GridChem Middleware Infrastructure
Implementation Currently underway
•
Implementation of GRMS resource management Service
http://www.gridlab.org/WorkPackages/wp-9
•
Moving toward Service based job submission eliminating
gateway interfaces
•
Infrastructure for multiple input files for single application
•
Infrastructure for multiple inputs in High Throughput
processing
•
Integrated workflow for multi scale coupled modeling
•
Meta-scheduling for High Throughput Processing
Match Making,
Round-robin scheduling, Preferred Host Set
usage
National Center for Supercomputing Applications
GridChem In New Collaborations
Resource Providers
• New Resource Providers
Open Science Grid Initially for Bio-related applications (open
source preferably)
• PRAGMA Partner sites
University of Hyderabad
• ORNL (Could be via TeraGrid)
• International Partners
KISTI, APAC, Daresbury Labs
National Center for Supercomputing Applications
Scientific Collaborations
• GridChem Extension to Molecular Sciences (Bio, Nano, Geo
and Materials Sciences) (NSF Proposal)
• Parameter Sweep for Potential Energy Hyper Surfaces
(Faculty Fellows, NCSA)
• Automated Parameterization of Force fields (NSF Proposal)
• Ab initio Molecular Dynamics (Faculty Fellows, NCSA)
• Education (CI-TEAM) (NSF Proposals)
• Multi-Scale Modeling (IACAT, UIUC)
National Center for Supercomputing Applications
Some New GridChem Infrastructure
•
•
•
•
•
•
Workflow Editors
Coupled Application Execution
Large Scale Computing
Metadata and Archiving
Rich Client Platform Refactorization
Intergrid Interactions
• Open Source Distribution
http://cvs.gridchem.org/cvs/
• Open Architecture and Implementation details
http://www.gridchem.org/wiki
National Center for Supercomputing Applications
Critical Gateways Issues
• Science Gateways compete with business as usual
for the end user research scientist
• No direct access to HPC systems may be possible
leading to apparent lack of control for users
• No “End to end solutions”
If part of the research needs require old ways
Gateways may be avoided
• Learning to use Gateways should provide substantial
added benefit –Cost/Benefit Issues for users
• Flexibility to integrate new applications as needed by
community quickly is critical to keep the user
community engaged
National Center for Supercomputing Applications
Authentication
National Center for Supercomputing Applications
Resource Status
National Center for Supercomputing Applications
Job Editor
National Center for Supercomputing Applications
Job Submission
National Center for Supercomputing Applications
Job Monitoring
National Center for Supercomputing Applications
Gradient Monitoring
National Center for Supercomputing Applications
Energy Monitoring
National Center for Supercomputing Applications
Post Processing
National Center for Supercomputing Applications
Visualization
Molecular Visualization
Electronic Properties
Spectra
Vibrational Modes
National Center for Supercomputing Applications
Molecular Visualization
Better molecule representations
(Ball and Stick/VDW/MS)
In Nanocad Molecular Editor
Third party visualizer integration
Chime/VMD
Export Possibilities to others interfaces
Deliver standard file formats
(XML,SDF,MSF,Smiles etc…)
National Center for Supercomputing Applications
Eigen Function Visualization
• Molecular Orbital/Fragment Orbital
• MO Density Visualization
• MO Density Properties
• Other functions
Radial distribution functions
National Center for Supercomputing Applications
Some example Visuals
Arginine Gamess/6-31G*
Total electronic density
2D - Slices
National Center for Supercomputing Applications
Electron Density in 3D
Interactive (VRML)
National Center for Supercomputing Applications
Orbital 2D Displays
N2 6-31g* Gamess
National Center for Supercomputing Applications
Orbital 3D
VRML
National Center for Supercomputing Applications
Spectra
• IR/Raman Vibrotational Spectra
• UV Visible Spectra
• Spectra to Normal Modes
• Spectra to Orbitals
National Center for Supercomputing Applications
Possible H-bonds network for P450cam
hydroperoxy intermediate
•
Nagano, S.; Poulos, T.L. J. Biol. Chem. 2005, 250, p.1668
Auclair, K.; Hu, Z.; Little, D. M.; Ortiz de Montellano, P. R.; Groves, J. T. J. Am.
Chem. Soc. 2002, 124, 6020.
GLY248 peptide
C
Suggested:
O
2.98Å
2.79Å
H
H
H
O
2.75Å
O
2.99Å
H
3.32Å
3.16Å
H
O
O H
3.07Å
H
N
THR252 accepts an H-bond
from the hydroperoxy (Fe(III)OOH that promotes the
second protonation on the distal
oxygen, leading to the O-O bond
cleavage
VAL253 peptide
O
Fe3+
THR252
CH3
National Center for Supercomputing Applications
The Somersault Isomerization of Model Cpd0
O
1.447Å
O
102.0
2.226Å
1.869Å
116.7
C
C
C
C
C
Fe
N
C
1.665Å
C
N
C
C
N
C
C
C
C
N
C
C
C
C
N
N
C
C
C
C
C
C
C
C
97.9
2.473Å
97.2
MIN
S
Robert Bach and
Olga Dmytrenko,
2006
E24.8 (24.3) 20.3 kcal/mol
E = 17.5 (17.8)
kcal/mol
TS
O
77.0
H
O
O
vi=101.5i cm-1
vi=93.7i cm-1
2.186Å
O
O
127.4
2.437Å
C
C
C
C
O
C
C
C
H
1.658Å
C
C
C
C
N
C
C
Fe
C
C
C
SH
2.487Å
S
C
N
Fe
N
C
C
N
C
SH
C
Fe
N
GS
Fe
C
C
C
2.408Å
C
C
C
C
S
O
C
N
C
C
177.9
C
C
C
C
1.662Å
O
97.8
EH-bonding = 17.0
kcal/mol
National Center for Supercomputing Applications
Energy Diagram for the Concerted Non-synchronous
Hydroxylation of Isobutane
(H3C)3C H
O
H
(H3C)3C
O
Fe
Energy diagram (kcal/mol)
for the oxidation of the
isobutane with ground
state, 24a
(GS-8 hydrogen bonded to
isobutane).
MIN-24b [model oxidant
MIN-10
(PorFe(SH)OHO)
hydrogen bonded
to isobutene] is not
necessarily
on the reaction pathway.
SH
H
(H3C)3C
O
H
O
HO
Fe
-4.0
MIN-24b
H
O
SH
Fe
TS-25
SH
TS-27
17.2
19.5
11.7
-19.2
MIN-26b
5.5
GS-24a
(H3C)3C
(H3C)3C
H
OH
O
Fe
HO
H
O
Fe
SH
(H3C)3C
-83.7
3.848Å
O
H H
O
Fe
SH
MIN-26a
SH
PRODUCT 28
(H3C)3C O
H
OH
Fe
National Center for Supercomputing Applications
SH
Somersault Mechanism
Summary for Isobutane Hydroxylation
H3C CH3
C
H3C CH3
H
H
C
H3C CH3
CH3
C
O
CH3
H
HO
H3C CH3
H
C
CH3
O
O
H
H
O
O
O
FeIV
FeIV
FeIV
FeIV
S
S
S
S
O
National Center for Supercomputing Applications
CH3
H
TetrakisDimethylAminoTitanium and its derivatives on Si(100)2x1 Surface: Diffusion Barrier Thinfilms on Silicon
Rodrigues-Reyes and Teplyakov
National Center for Supercomputing Applications
Benzylazide on Si(100)-2x1 Surface
Deposition of Aromatic Moieties on Silicon for Lateral Electron
Transfer
Bocharov et al..
National Center for Supercomputing Applications
[2+2] Cyclo Additions involving B=P Bonds
Gilbert and Bachrach
Ethyne Addition
Dimerization
Ethene Additions
National Center for Supercomputing Applications
Possible H-bonds network for P450cam
hydroperoxy intermediate
•
Nagano, S.; Poulos, T.L. J. Biol. Chem. 2005, 250, p.1668
Auclair, K.; Hu, Z.; Little, D. M.; Ortiz de Montellano, P. R.; Groves, J. T. J. Am.
Chem. Soc. 2002, 124, 6020.
GLY248 peptide
C
Suggested:
O
3.32Å
O H
O
2.98Å
THR252 accepts an H-bond
H
H
from the hydroperoxy (Fe(III)H
O
2.75Å
OOH that promotes the
O
H
H
2.99Å
second protonation on the distal
3.16Å
N
oxygen, leading to the O-O bond
H
VAL253 peptidecleavage
O
2.79Å
3.07Å
3+
Fe
THR252
CH3
National Center for Supercomputing Applications
The Somersault Isomerization of Model Cpd0
O
1.447Å
O
102.0
2.226Å
1.869Å
116.7
C
C
C
C
C
Fe
N
C
1.665Å
C
N
C
C
N
C
C
C
C
C
N
C
C
C
C
Fe
N
C
2.408Å
C
C
C
C
S
O
C
N
C
C
177.9
C
C
C
C
1.662Å
O
C
C
C
N
N
C
C
C
C
C
C
C
97.9
2.473Å
GS
97.2
MIN
S
E24.8 (24.3) 20.3 kcal/mol
E = 17.5 (17.8)
kcal/mol
TS
O
77.0
H
O
O
vi=101.5i cm-1
vi=93.7i cm-1
2.186Å
O
O
Fe
127.4
2.437Å
C
C
C
C
SH
C
C
O
1.658Å
C
C
C
C
N
C
C
Fe
C
C
C
SH
2.487Å
S
C
N
Fe
N
C
C
N
C
C
H
97.8
EH-bonding = 17.0
kcal/mol
Robert Bach and Olga Dmytrenko, 2006
National Center for Supercomputing Applications
Energy Diagram for the
Concerted Non-synchronous
Hydroxylation of Isobutane
(H3C)3C H
O
H
(H3C)3C
O
Fe
SH
H
(H3C)3C
O
H
O
HO
Fe
-4.0
MIN-24b
H
O
SH
Fe
TS-25
SH
TS-27
17.2
19.5
Energy diagram (kcal/mol) for
the oxidation of the isobutane
with ground state, 24a
(GS-8 hydrogen bonded to
isobutane).
MIN-24b [model oxidant
MIN-10 (PorFe(SH)OHO)
hydrogen bonded
to isobutene] is not necessarily
on the reaction pathway.
11.7
-19.2
MIN-26b
5.5
GS-24a
(H3C)3C
(H3C)3C
H
OH
O
Fe
HO
H
O
Fe
SH
(H3C)3C
-83.7
3.848Å
O
H H
O
Fe
SH
MIN-26a
SH
PRODUCT 28
(H3C)3C O
H
OH
Fe
National Center for Supercomputing Applications
SH
Somersault Mechanism
Summary for Isobutane Hydroxylation
H3C CH3
C
H3C CH3
H
H
C
H3C CH3
CH3
C
O
CH3
H
HO
H3C CH3
H
C
CH3
O
O
H
H
O
O
O
FeIV
FeIV
FeIV
FeIV
S
S
S
S
O
National Center for Supercomputing Applications
CH3
H
Unsymmetrical Mo(CO)4 Crown
Ethers
National Center for Supercomputing Applications
Dibenzaphosphepin based a,wbis(phosphorous)polyether chelated Mo(CO)4
National Center for Supercomputing Applications
Crystal Structures
CSD:XAPZAP
CSD:DEQDOS
cis-(6,6'-((1,1'-Binaphthyl)-2,2'diylbis(oxy))bis(dibenzo(d,f)(1,3,2)dioxaphosp
hepin))-tetracarbonyl-molybdenum(0)
C48 H28 Mo1 O10 P2
cis-Tetracarbonyl-(P,P'-(6-(2'-oxy-2-biphenyl)-3,6dioxa-hexanolato)-bis(dibenzo
(d,f)(1,3,2)dioxaphosphepine)-P,P')-molybdenum
C44 H32 Mo1 O12 P2
National Center for Supercomputing Applications
Starting Structure
National Center for Supercomputing Applications
Optimized Structure
National Center for Supercomputing Applications
Reference Structure for Comparison
7
8
National Center for Supercomputing Applications
Structural Comparisons
C-C Torsion Angles for the OCH2CH2O Fragments and for the Axially
Chiral Biaryl Groups
Atoms
C37-C42-C43-C48
C1-C6-C7-C12
C13-C22-C23-C32
C32-O-C33-C34
O-C33-C34-O
C33-C34-O-C35
C34-O-C35-C36
O-C35-C36-0
•
•
•
PCMODEL*
-49.9
45.4
75.6
-178.4
62.4
-80.6
174.6
66.2
UFF
-26.4
22.3
74.7
-140.8
-64.5
-118.9
118.9
56.0
Ab Initio
-43.0
-22.3
-85.9
159.7
-87.3
67.8
-153.4
64.0
Amber
-40.4
-72.8
-81.2
-171.2
-82.4
64.9
60.1
67.3
*Hariharasarma, et al. Organomet., 1232-1238, 2000.
Ab Initio=B3LYP/3-21G*
Amber9 ff03, GAFF, chloroform, 300K, median over 1ns MD
National Center for Supercomputing Applications
MD OCH2CH2O Structure
7
8
National Center for Supercomputing Applications
MD Biaryl Structure
National Center for Supercomputing Applications
1H
NMR Chemical Shift Comparison
For Aromatic Protons
Reference 32ppm (from TMS B3LYP/6-31g*)
Abinitio
Atom Exp.
Abinitio
5.6
H25
6.578
5.7
5.8
H26
6.737
5.9
5.9
H27
7.018
6.1
6.0
H28
7.623
6.5
Atom
H2
H3
H4
H5
Exp.
7.025
7.026
7.049
7.181
H8
H9
H10
H11
7.110
6.890
6.721
6.237
6.1
6.0
6.0
5.7
H14
H15
7.925
7.808
H17
H18
H19
H20
7.741
7.254
7.091
6.989
H30
H31
7.790
7.289
6.7
6.9
5.8
6.3
H38
H39
H40
H41
7.327
7.274
7.169
7.350
6.2
6.1
6.0
6.3
6.0
5.6
5.1
4.6
H44
H45
H46
H47
7.360
7.160
7.176
7.060
6.1
5.9
6.0
7.0
National Center for Supercomputing Applications
Third Year Plans
• Post Processing
Spectra and related entities
• New Application Support
Aces3, Dmol3, Vasp,…..
• Expansion of Resources
Teragrid, OSG, Pragma Systems and New
resources at Partner Sites
• Extension Plan
Two Proposals in review for Extension
National Center for Supercomputing Applications
Future Plans
• Preparations for Petaflop computing
High throughput massively parallel applications
• Complex workflows for integrating multiple
interdependent applications
Multiscale Computing
• Archiving and annotating data for future use
Open Data initiatives by NIH and NSF
National Center for Supercomputing Applications
Acknowledgments
•
•
•
•
•
•
•
•
•
Rion Dooley, TACC Middleware Infrastructure
Stelios Kyriacou, OSC Middleware Scripts
Chona Guiang, TACC Databases and Applications
Kent Milfeld, TACC Database Integration
Kailash Kotwani, NCSA, Applications and Middleware
Scott Brozell, OSC, Applications and Testing
Michael Sheetz, UKy, Application Interfaces
Vikram Gazula, UKy, Server Administration
Tom Roney, NCSA, Server and Database
Maintenance
National Center for Supercomputing Applications