Open Grid Computing
Environments: Advanced
Gateway Support Activities
RT Project Review
October 7th, 2010
Software
Description
OGCE Gadget
Container
Google Gadget/Open Social compatible
software for building Web-based user
interfaces.
XBaya
A visual user interface for composing,
launching and monitoring workflows
GFAC
An application factory service for wrapping
command-line tools as Web services
XRegistry
A service and workflow registry
Registry Gadget,
Experiment Builder
User interface for registering services and
creating online experiments with registered
workflows
Software Strategy
• We develop and package downloadable software
• Focus on gadget container and tools for running
science applications on grids and clouds.
• Provide a tool set that can be used in whole or in
part.
– If you just want GFac, then you can use it without
buying an entire framework.
• Outsource security, information services, data
and metadata, etc to other providers.
– MyProxy, TG IIS, Globus, Condor, XMC Cat, iRods, etc.
NSF SDCI Award: 2010-2013
• Indiana: Marlon Pierce (PI), Suresh Marru (CoPI), Raminder Singh, Rob Quick, Gregor von
Laszewski, Gerald Guo
• Purdue: Carol Song (Co-PI), David Braun, Lan
Zhao
• NCSA/UIUC: Sudhakar Pamidighantam (Co-PI),
Shaowen Wang, Yan Liu
• UTHSCSA: Emre Brookes (Co-PI), Borries
Demeler
Project Highlights
• Full Circle Development
– Directly fund both software developers and gateway
consumers.
• Directly supported (non-IU) gateways:
– UltraScan (UTHSCSA), GridChem (NCSA),
SimpleGrid/GISolve (UIUC), Purdue CCSM and
Environmental Data Gateways
– Among the most used TG gateways.
• Sustainability strategy: Apache Incubator for
workflow suite of tools
– XBaya, GFac, and supporting services.
UltraScan High Level Overview
User
Web Server
MySQL DB
US LIMS
UTHSCSA Jacinto
Terascale storage
GridControl
TeraGrid
TIGRE/Globus
Network
High Performance
Computing Clusters
UltraScan Middleware
• Immediate Goals: Use GFAC as a
replacement job submission
service.
– GRAM 2, 4, 5 independence
– Significant effort into GRAM5
testing on Ranger.
• Longer term goals
– Integrate with TG information
services to provide better job
scheduling.
• OGCE Resource Prediction Service
– Support UNICORE job
management.
Current Architecture
Gram5 Testing Steps
• Developed Testing harness to run different
UltraScan cases.
• Started with small number of jobs and increased
the concurrency later
• Watched job behavior of the job on resource and
monitored the gram log
– There were lot of issue which we found from
the logs and working with Globus team to fix
them
• Recorded all the job run data to create a google
gadget to create graph for different runs on
different resources.
TG Resources and Patterns
Version
Resource
Endpoint
GT 5.0.2
QueenBee
queenbee.loni-lsu.teragrid.org:2120/jobmanager-pbs
GT 5.0.2
Ranger
login5.ranger.tacc.teragrid.org:2120/jobmanager-sge
GT 5.0.2
Lonestar
gatekeeper.lonestar.tacc.teragrid.org:2120/jobmanagerlsf
Patterns:
Concurrent jobs
Batch Size
Total jobs
Job Status Pass : Fail
1
10
10
10:0
3
10
30
30:0
5
10
50
50:0
10
10
100
20:0
20
10
200
40:0
50
10
500
100:0
100
10
1000
200:0
200
5
1000
Not tested (Need allocation)
500
2
1000
Not tested (Need allocation)
Learning from this exercise on Ranger
• There were problems with Sun Grid Engine job manager
(sge.pm) not loading system environment and user
environment settings.
• You can't load modules using Gram parameters.
– On Ranger all the libraries are modules and user may want to load
some special module version for his job.
– Example is if you have different version of your code using different
modules.
• On Ranger user default shell is C shell and Gram runs using
bash shell. This is not a problem but users need to be
informed where to add user environment.
• You can look at detailed results on following
http://teragridforum.org/mediawiki/index.php?title=GRAM_5_Testing
GridChem Science Gateway
• A chemistry/material Science Gateway for running
computational chemistry codes, workflows, and parameter
sweeps.
• Integrates molecular science applications and tools for
community use.
• 400+ users heavily using TeraGrid. One of the consistent top5
TeraGrid Gateway users.
• Supports all popular Chemistry applications including
Gaussian, GAMESS, NWChem, QMCPack, Amber and MolPro,
CHARMM
• ParamChem is a follow-on project to develop workflows for
chemical parameter studies and provide the infrastructure to
execute them.
Cyberenvironments for Parameterization
Computational Reference Data Generation
GridChem , ParamChem
Integration
• Immediate Goals:
integrate XBaya workflow
composer with GridChem
user interface and
middleware.
– Target SC10
• Longer term goals:
provide part of GridChem
next generation
middleware; support
large scale ParamChem
workflows.
SimpleGrid (UIUC)
SimpleGrid is a Gateway teaching tool that is also
the core of GISolve
SimpleGrid Gadgets
Yan and Gerald’s work from our Oct 6th hacking session.
SimpleGrid, GISolve
• Short term goal: develop SimpleGrid Gadgets
deployable into gadget container.
– Must meet security requirements
– Support PHP development
– Support interactivity requirements
• Integrate YUI JavaScript libraries with Gadget
JavaScript.
• Longer term goals: investigate workflow, job
management tools. Apply to GISolve
Purdue CCSM and Data Portals
• Short terms goals: Develop CCSM and data
management gadgets and necessary backing
middleware.
– Support GWT development
– Interactivity and security requirements.
– Significant requirements overlap with SimpleGrid
• Longer term goals: Build gateways out of
gadgets hosted by multiple containers;
examine workflow and other tools.
Conclusion
• We focus initially on one component per
gateway.
– SimpleGrid, CCSM, Data Portal: gadgets
• Other gadget based gateways at UC
– GridChem: Xbaya
– UltraScan: GFac
• Goal is to establish an Apache-style
meritocracy for contributed code.
• Making distributed teams work: hacking
retreats, SourceForge, Jira, Google apps
Acknowledgments
• The following student interns have
contributed to the project over the last year
– Ye Fan
– Patanachai Tangchaisin
– Suresh Kumar Deivasigamani
– Roland Mai
– Sashikiran Challa
Backup Slides
OGCE Gateway Tool Adaption & Reuse
LEAD
LEAD
Experiment Builder, XRegistry Interface
GFac, XBaya,
XRegistry, FTR
Eventing System
GridChem
Xbaya, GC Middleware
GridChem
Resource
Discovery Service
TeraGrid
User Portal
GPIR, File
Browser
OGCE Team
Gadget Container,
GTLab, Javascript Cog,
XRegistry Interface,
Experiment Builder,
Axis2 Gfac, Axis2
Eventing System,
Resource Prediction
Service, Swarm
Ultrascan
OGCE
Re-engineer,
Generalize,
Build, Test and
Release
GFac, Eventing
System
OVP/
RST/
MIG
BioVLab
XBaya, GFac
ODI
Workflow Suite, Gadget Container
Bio Drug Screen
Swarm->GFac
EST Pipeline
Swarm->GFac
Future Grid
GFac, Xbaya, …
21
Putting It All Together
Software Strategy
• Focus on gadget container and tools for running
science applications on grids and clouds.
• Provide a tool set that can be used in whole or in
part.
– If you just want GFac, then you can use it without
buying an entire framework.
• Outsource security, information services, data
and metadata, etc to other providers.
– MyProxy, TG IIS, Globus, Condor, XMC Cat, iRods, etc.
Advanced Support Scenarios
• GridChem/ParamChem workflow support
• UltraScan Job Submission (GFAC)
• EST Pipeline
– Bioinformatics pipeline for managing mass job
submission.
More Information
• This is downloadable, packaged software.
– Apache Maven build system provides everything
you need to to build the gadget container,
gadgets, workflow composer, and backing
services.
– Get code by anonymous SVN checkout.
• Email: mpierce@cs.indiana.edu,
smarru@cs.indiana.edu, ogcediscuss@googlegroups.com
• OGCE Web Site: www.collab-ogce.org
• Blog/News Feed: http://collabogce.blogspot.com/
Acknowledgements and People
• Funding by TeraGrid GIG, RP and by OCI SDCI
• IU: Marlon Pierce, Suresh Marru, Raminder
Singh, Archit Kulshrestha, Zhenhua Guo
• TACC: Maytal Dahan, Rion Dooley
• SDSC: Nancy Wilkins-Diehr, Jeff Sale
• SDSU: Mary Thomas
GridChem Advanced Support
• IU Personal – Suresh Marru, Marlon Pierce,
Raminder Singh, Ye Fan
• Gateways Area Director – Nancy Wilkins-Diehr
• GridChem Team – Sudhakar Pamidigantam,
Rion Dooley, Vikram Gazula & Rest of
GridChem team.
Empirical ForceFields Parameterization
Need
Process
Lack of Accurate Force Fields Produce
Erroneous Property Estimation
Fig. 1. Errors (V) in electrostatic potential on a surface at 1.8 times van der Waals radii around N-methyl
propanamide for two models. (Left) Point charges; (right) charge, dipole, and quadrupole on C, N, and O; charge and
dipole on H. The errors are much reduced in the multipole approach
A. J. Stone Science 321, 787 -789 (2008)
Published by AAAS
Vanommeslaeghe et al.
J. Comp.Chem 2010, 31,
671-690
Cyberenvironments for Parameterization
Computational Reference Data Generation
Molecular Force Field Cyberenvironments
Parameter Initialization and optimization Workflow
Workflow
Manager
Parameter definitions
Optimization
Monitor
Model/Reference Data
Definition
Merit Function
Specification
Optimization
Methods Choice
Consistency Checker
Optmization Job
Launcher
Optimization
Incomplete?
Expert
Interface
Optimization Job
Completed?
Paramater testing Model
Successful Testing
Paramater Sensitivity
Analysis
Update Parameter Database
with new set
Notification of End
of Workflow
OGCE Gateway Tool Adaption & Reuse
LEAD
LEAD
Experiment Builder, XRegistry Interface
GFac, XBaya,
XRegistry, FTR
Eventing System
GridChem
XBaya
GridChem
Resource
Discovery Service
TeraGrid
User Portal
GPIR, File
Browser
OGCE Team
Gadget Container,
GTLab, Javascript Cog,
XRegistry Interface,
Experiment Builder,
Axis2 Gfac, Axis2
Eventing System,
Resource Prediction
Service
Ultrascan
OGCE
Re-engineer,
Generalize,
Build, Test and
Release
Gfac, Eventing
System
OVP/
RST/
MIG
BioVLab
XBaya, GFac
ODI
Workflow Suite
Bio Drug Screen
Workflow Suite
EST Pipeline
Workflow Suite
Future Grid
???
31
Cyberinfrastructure Layers
User
Interfaces
Gateway
Software
Web/Gadg
et
Container
Web/Gadge
t Interfaces
Application
Abstractions
Fault
Tolerance
Workflow
System
Auditing &
Reporting
Resource
Middleware
Cloud Interfaces
Compute
Resources
Computational
Clouds
Color Coding
Gateway
Abstraction
Interfaces
Web Enabled
Desktop
Applications
Monitoring
Registry
User
Management
Information
Services
Security
Provenance
& Metadata
Management
Grid Middleware
SSH & Resource
Managers
Computational
Grids
Local Resources
OGCE Gateway Components
Complimentary Gateway Components
Dependent resource provider components
LEAD/OGCE Layered Workflow
Architecture
Workflow
Interfaces (Design
& Definition)
XBaya GUI
(Composition,
Deploying, Steering
& Monitoring)
BPEL 2.0
Flex/Web
Composition
Gadget Interface for
Input Binding
Python
Scufl
Workflow
Specification
BPEL 1.0
Workflow
Execution & Control
Engines
Apache
ODE
GBPEL
Java Code
Pegasus DAG
Dynamic
Enactor
Condor
DAGMan
Jython Interpreter
Taverna
GFac Current & Future Features
Globus
Apache Axis2
Input
Handlers
Output
Handlers
Auditing
Registry Interface
Fault
Tolerance
Checkpoint
Support
Scheduling
Interface
Monitoring
Interface
Data Management Abstraction
Job Management
Abstraction
Campus
Resources
Amazon
Eucalyptus
Unicore
Condor
Color Coding
Existing Features
Planned/Requested Features
Gateway Computing Environments (GCE10)