Grappa
Grid Access Portal for Physics Applications
CHEP 2003
UCSD
March 24-28,2003
Daniel Engh (UC), Shava Smallen (IU),
Liang Fang (IU), Jerry Gieraltowski (ANL),
Rob Gardner (UC), Dennis Gannon (IU),
Randy Bramley (IU)
Grappa: A Web Portal
Browser
The Grappa portal is a
web portal designed to
provide to the user an
interface that
coordinates and
integrates the use of a
diverse set of web and
grid services
Command line
Grappa Portal
The “Grid”
GRAM Services
-compute sites
Web Services
GridFTP Services
-Athena Libraries
-packaged physics
apps
-data storage
-Magda
Catalogue
-Ganglia Monitor
-ATLAS web
services
Replica Locator
Service
Grappa History
• First Design proposals (2001)
• Implementation in XCAT Science Portal
(2002)
• Demos for ATLAS (2002)
– To U.S. ATLAS grid testbed sites
– To joint iVDGL/DataTAG WorldGrid testbed
• IST/SC 2002 demos (Nov 02)
• Jetspeed implementation (Mar 03)
Grappa Component Layers
•
•
•
•
user Xbook (notebook scripts)
– scripts customized for the user
(as simple or complex as
desired)
– conform to the xbooks API
– Athena xbook customized for
ATLAS job submission
– Many scripting languages
supported (Jython, Python,
Perl…)
Xbooks server
– A Jetspeed portlet
– manages requests from user
xbook
Jetspeed: Apache-Jakarta web
portlet framework
Tomcat server: provides the web
server
Browser
Command line
user Xbook
Portlets (incl. Xbooks Server)
Jetspeed
Tomcat Server
Grid Services
Athena Xbook
• HTML form interface
– Organizes input parameter
selection
– Menu selection of grid
services
• Jython scripts
– Python syntax
– Java toolkits
– Java CoG
• Implementation of Globus
Client tools
– XCAT technology available
• Toolkit to componentize
applications
• Implementation of DOECCA spec. as web services
• Currently not used (current
demo has only 1 element)
• Athena Xbook customized:
– Treat Athena (ATLAS) as
an external app (don’t
integrate it into portal)
– Prestage xbook/ CoG/
Athena libraries, input data
– Submit jobs via GRAM
– Transfer results to storage
site
– Monitor/cancel/resubmit
jobs.
• Submission parameters
managed by xbooks
manager
• GRAM IDs managed by
Athena xbook
Xbooks Architecture
• The user’s xbook
(notebook scripts)
interacts with the
xbooks server layer
– Contacts xbooks server,
registry, and manager
– Creates an instance of
the user’s xbook.
• See refs for xbook API
Xbooks Manager
• The xbook manager
dispatches requests
– finds the proper script
engine
– Launches script engine
• (e.g. JythonEngine)
– Records submission
parameters in xbook
repository
– Generates application
instance
• (e.g. job submission
requests)
Design Factors: Jetspeed
• Apache-Jakarta Jetspeed framework
– Open Source
– A large mainstream effort
• Hundreds of developers
• Solid support community for portlets
– Portlets
•
•
•
•
•
componentize the portal
interchangeable among projects
Proxy Manager could be same for LCG, Chem portals, etc.
GridFTP portlet
Xbooks server is a portlet, easily integrates user scripting with wide
availability of portlets
– World moving in direction of Grid/Web services
• We want to come along
• Don’t want to develop everything ourselves
• See refs for Jetspeed API
Design Factors: Java
• Jython chosen as the demo Athena xbook scripting
language
– Java toolkits provide lots of stuff
– Java CoG provides a java implementation of Globus client
tools within the xbook.
– Globus 3.0 & OGSA moves further in the java direction
– Java CoG part of Globus 3.0
• Python could be used as well.
– pyGlobus CoG available--provides interface to a local
installation of Globus.
– Swig to integrate C/C++ toolkits
– Python CoG also to be part of Globus 3.0..
Design Factors: Portals
• Portals vs. GUIs
– Not interchangeable approaches
– A portal can have an integrating function
• Integrate vs. Interface
– Functionality built into portal?
– Interface with functionality found external to portal (e.g. in web
services)?
– Integrate the use of interfaces (integrate the service clients)?
– Today, many believe a philosophy of distributing functionality as
grid components and services to achieve greater:
• Portability
• Scalability
• Reliability
– No single answer. Need to decide for your app what needs to be
integrated into your portal or notebook (xbook) and what needs to
be interfaced.
– Portal framework not the framework for all your components.
Status and Future
• Grappa Status
– Several demos to date
– Basic job submission and monitoring
– Now based on standard Jetspeed framework
• (XCATSP-use discontinued)
• Large jetspeed community
• Moving in direction of Grid Components
• Grappa Future
– Parameter tracking and workflow generation
• Chimera Virtual Data System
– Portlet development and integration
– Develop/wrap/access Physics-apps as Grid Components &
Services
– Interaction with other portal projects (eg. Ganga)
References
•
•
•
•
•
•
•
•
•
•
•
•
•
Grappa: grid.uchicago.edu/grappa/
GriPhyN: www.griphyn.org
Python CoG: www.itg.lbl.gov/gtg/projects/pyGlobus/
Jakarta Jetspeed: jakarta.apache.org/jetspeed
Xportlets webpage: www.extreme.indiana.edu/xportlets/project/index.shtml
ATLAS webpage: atlasexperiment.org
Java CoG: Gregor von Laszewski, Ian Foster, and Jarek Gawor. CoG Kits: A Bridge Between
Commodity Distributed Computing and High-Performance Grids. In ACM 2002 Java Grande
Conference, June 2000. pp3,22.
Jython: www.jython.org
XCATSP: Sriram Krishnan, Randall Bramley, Dennis Gannon, Madhusudhan Govindaraju,
Rahul Indurkar, Aleksander Slominski, Benjamin Temko, Jay Alameda, Richard Alkire,
Timothy Drews, and Eric Webb. The XCAT Science Portal. Proceedings of Supercomputing
2001. p4.
XCAT 2.0: Design and Implementation of Component based Web Services. Technical
Report 562, Indiana University Computer Science Department, June 2002. p22.
Chimera-VDS: Ian Foster, Jens Voekler, Michael Wilde, and Yong Zhao. Chimera: A Virtual
Data System for Representing, Querying, and Automating Data Derivation. 14th
International Conference on Scientific and Statistical Database Management, 2002. p22.
www-unix.griphyn.org/chimera/
XBOOKS: www.extreme.indiana.edu/xbooks
Extreme! Computing Lab www.extreme.indiana.edu
Glossary
• Component: A small binary object or program that performs a
specific function. The component is designed to easily operate
with other components and applications.
• DOE-CCA: DOE Common Component Architecture, a
specification for designing components.
• Chimera-VDS: Virtual Data System developed under GriPhyN,
at U.Chicago. A system for tracking file, parameter, and
executable dependencies used to construct workflows.
• CoG: Commodity Grid, user interfaces to Globus
– Java CoG, a java implementation of globus client tools
– Python (pyGlobus), interface to local globus tools
• Grappa: Grid Access Portal for Physics Applications, an
integration of scripting notebooks (or “xbooks”), an Xbooks
server portlet, Jetspeed, and Tomcat.
• GriPhyN: Grid Physics Network
Glossary
• Jetspeed: Apache portlet framework for designing web services
as interoperable components of a common framework
• Jython: Java+Python scripting language. Uses python syntax,
interprets python and java code
• Tomcat: Web Server
• OGSA: Open Grid Services Architecture (GLOBUS)
• Portal or Web Portal: a web site that provides services.
• Portlet: Similar to a Portal but reduced in scope and composed
as a web service component to interoperate with other portlets.
• RLS: Replica Locator Service (GLOBUS)
• SOAP: Simple Object Access Protocol, XML-based messaging
protocol used to encode the information in Web service request
and response messages before sending them over a network.
Glossary
• Xbooks: Jetspeed Portlet for managing user scripts (notebooks)
• XCAT: Ind.Univ. Extreme! Computing Lab’s Implementation of
CCA as web services
• XSOAP: I.U.E!C.L. Implementation of SOAP for XCAT
• XCATSP: I.U.E!C.L. Portal framework for XCAT (now replaced
with Jetspeed)
• Xportlets: I.U.E!.C.L. Jetspeed portlets.
• Workflow: Series of tasks needed to produce a result, this may
involve tracking software component dependencies, analysis
parameters, etc. See Chimera-VDS toolkit.
Appendix: Screenshots
• Screenshots of grappa portal showing:
–
–
–
–
Web access
Proxy management
GridFTP browsing
Xbooks pages
• Job submission
• Job monitoring
– Some more Portlets
– Adding portlets to portal.