Supercomputing, Visualization & e-Science
Manchester Computing
Grid-Enabled
Desktop Environments
…and other stuff
Stephen Pickles
Portals and Portlets Workshop
14-17 July, 2003
National e-Science Centre, Edinburgh
Outline
 A portal use-case
RealityGrid computational steering
 Grid Services in Perl – OGSI::Lite
"There's more than one way to do it!"
 Grid-Enabled Desktop Environments
The GRENADE project
2
Supercomputing, Visualization & e-Science
Supercomputing, Visualization & e-Science
Manchester Computing
A portal use-case
RealityGrid computational steering
RealityGrid
 EPSRC-funded e-Science pilot project (20 FTE x 3 years)
 Steering is central
– Computational steering
– On-line visualisation of large datasets
– Feedback-based performance control





Component-based and service-oriented
Grid-enabled (Globus and UNICORE)
Advanced Human-Computer Interfaces
Everything is distributed and collaborative
All in a materials science domain
– many "legacy" codes (Fortran, C, C++; MPI, OpenMP)
– 5 of which are now steerable
 Website: http://www.realitygrid.org
4
Supercomputing, Visualization & e-Science
Steering in the OGSA
OGSI-based middle tier
Steering
GS
bind
Simulation
Steering library
library
Steering
publish
Steering
Steering
library
client
find
connect
Client
Registry
data
transfer
publish
Steering library
bind
Steering
GS
5
Visualization
Visualization
Supercomputing, Visualization & e-Science
Grid services for
computational steering
Steering Grid Service (SGS)
 Part of OGSI middle tier between application and client (desktop client
or web portal)
 Provides public interface to "knobs" and "dials" of steerable application:
–
–
–
–
Monitor and steer parameters
Pause, resume and stop
Checkpoint and windback
Emit/consume "samples" for eg. visualization, re-setting boundary conditions
 Facilitates interoperability with other steering clients/portals
 Introduced without change to application source code
 Helps bootstrap direct inter-component connections for large data
transfers
Working prototype exists based on Perl implementation of OGSI
6
Supercomputing, Visualization & e-Science
RealityGrid portal
Already have desktop client tools
 Steering GUI
 Job control
– Originally Java/UNICORE GUI; Globus scripts today; GRENADE GUI's next?
Matthew Egbert (EPCC) starting development of portal (web client) now
Portlet frameworks make a lot of sense for us
 Hope to leverage existing portlets for proxy handling; resource
discovery; job control…
 …freeing us to concentrate on the novel issues raised by
computational steering:
– Parameter watch and steer; pause/resume/stop
– On-line, remote visualization
– Checkpoint management (next slide) inspired by GRASPARC (1993)
7
Supercomputing, Visualization & e-Science
A typical steered LB3D simulation
Cubic micellar phase,
high surfactant density
gradient.
Cubic micellar phase,
low surfactant density
gradient.
Initial condition:
Self-assembly
Random water/
starts.
surfactant mixture.
Rewind and
restart from
checkpoint.
8
Lamellar phase:
surfactant bilayers
between water layers.
Supercomputing, Visualization & e-Science
Supercomputing, Visualization & e-Science
Manchester Computing
Grid Services in Perl
"There's more than one way to do it."
OGSI::Lite
Perl OGSI hosting environment
 Near-complete OGSI implementation
 Written by Mark Mc Keown, Grid Support Centre,
University of Manchester
– in four months, part-time
 Lighter weight than Globus Toolkit version 3.0
– But slightly slower
 More portable than Java
 Runs on existing "Level 2 Grid"
10
Supercomputing, Visualization & e-Science
Perl for Grid services
Pluses
Minuses
 Powerful glue language
 Lightweight deployment
 Perl sucks
– Can even use pre-OGSA
middleware (eg. GT2) to bootstrap
OGSI container in user space on
the fly!
 Portable
– Will go places Java can't
 All prerequisites freely available
on CPAN
– Perl 5.6 or 5.8; SOAP::Lite; LWP;
XML::DOM; …
 Suit developers with significant
Perl code base
11
– Write once, read never
 Tooling support poor to
hopeless
– WSDL must be hand-crafted
 Lags mainstream Java Web
Service developments
 Scalability?
– In our approach, every Grid Service
Instance is a separate process
 Insert your pet gripe here
Supercomputing, Visualization & e-Science
OGSI::Lite - Status
Features:
 Container
To Do:
 ServiceGroup
– Can deploy/replace Grid services
without restarting container
– SSL version too
 Grid Service portType
– Inherit from base module
 Service Data
 Factories
– We can start factories too!
 Service Groups
 Notification
 Handle Resolver
12
– rules on interface and content
 removeByServiceDataName
 Plug other minor holes
 CGI version
 Compliance checking
 Does it really interoperate?
 Bug fixes
http://www.sve.man.ac.uk/Research/AtoZ/ILCT
Supercomputing, Visualization & e-Science
Supercomputing, Visualization & e-Science
Manchester Computing
Grid-Enabled
Desktop Environments
The GRENADE project
The Grid
The web of the future?
 Where is the Grid's killer app coming from?
 How can the Grid deliver its promised ubiquity within the
confines of the community that conceived it?
 Can we enlist the skills and imagination of a much wider
community to accelerate the development of the Grid?
 The DataGrid has been called "Napster for scientists". Is it
good enough for particle physicists but not friends and
family?
Get the geeks engaged!
14
Supercomputing, Visualization & e-Science
IMHO: User Interfaces to the Grid
 Portals are great, but…
– They break the symmetry between local and remote resources
• Why can't I use the same user interface to browse remote files as local ones?
– Can a third party extend the functionality of the portal?
• Lesson: make middle tier services accessible!
– A single portal provides uniformity of presentation, but a user must use
many portals…
• Where should persistent user context reside?
– Sometimes (eg. interactive visualization) a thin client won't do
• Or will it? Can TightVNC help deliver this through a web portal?
 Command line interfaces are powerful, but arcane
 Desktop metaphor has been neglected
15
Supercomputing, Visualization & e-Science
The Vision
 Robust, interoperable Grid toolkits ship with every PC
– We were promised ubiquity
– STOP PRESS: SUSE Linux now ships with GT2
 Grid functionality on every desktop
– accessibility, visibility, public awareness
 Ordinary people use Grids, not just e-Scientists
 Desktops consume and export Grid services
 Thriving open source Grid projects
– games, file sharing, code sharing, "friends and family grids"…
How do we get there from here?
16
Supercomputing, Visualization & e-Science
The Plan
GRENADE will explore the possibilities offered by tight integration of Grid
functionality into the user's desktop.
Two phase strategy:
1. Develop a prototype Grid Enabled Desktop (demonstrator) to pumpprime an on-going open source project,
2. Support the open source project and encourage emulation on other
platforms.
Target KDE in the first instance
 Shipped with most Linux distributions
 User base (Linux, Irix, AIX, Solaris,…) and portability
 Component-based architecture with XML descriptions
 Konqueror browser designed to integrate plug-in components
But will choose more portable Qt classes instead of KDE classes
wherever practical
17
Supercomputing, Visualization & e-Science
Curious observations
Early versions of Globus had
 poor separation between client and server bundles
 APIs limited to C and command line.
(These are "fixed" by better packaging, COG-kits and portals.)
GSI suggests having a single base for Grid activities where your private
key is stored. The obvious place is on your own PC or laptop (if
installing Globus was a trivial exercise and IPV4 wasn't fatally flawed).
Curiously, none of these presents an obstacle to GRENADE.
By demanding a complete Globus installation, GRENADE can
 use Globus API's and drill down into host's native API layers
 provide – and facilitate management of – Globus services.
18
Supercomputing, Visualization & e-Science
GRENADE Software Stack
GRENADE
QT
Globus
OS/Desktop
GRENADE GUI Apps
GRENADE Command
QT GUI Classes
Line Tools
GRENADE GRAM Job Controller
GRENADE Credential Manager
GRENADE Messaging Layer
GT2 Grid
Middleware
QT Non-Graphical
Classes
Message
Transport
OS
19
Supercomputing, Visualization & e-Science
GRENADE Stack
 Messaging layer
– Sends serialized objects over QDataStream (essentially a stream of bytes)
– Can be implemented over different transports, e.g. DCOP, SSL.
– KDE implementation will use DCOP.
 Credential Manager
– Looks after the user’s X509 certificates and keys
– Can generate GSI proxies
– Each credential has a simple name or alias identifying it, e.g. “ukes” or
“globus”
 GRAM Job Controller
– Looks after jobs launched via GRAM
– Each job has a simple name or alias identifying it, e.g. “rg-sim”
20
Supercomputing, Visualization & e-Science
GRENADE Command Line
 Adds value to underlying Grid middleware
– Names/aliases provide simple reference mechanism for both
credentials and jobs
– Can list, add to, remove credentials from the manager
– Can list known GRAM jobs, find out how they were started, etc.
– Can refer to GRAM jobs without using horrible URL
 Insulate the user from hard-to-use things, like:
– Proxy creation for non-standard location
– Inconsistent GT2 command line interface
 How does this fit with GCE shell??
21
Supercomputing, Visualization & e-Science
Deliverables
 Reference Implementation – GRENADE 1.0 (Q4 2003?)
–
–
–
–
–
–
Single-sign on
Job definition, submission and monitoring tools
MDS browser, pluggable in to Konqueror and job controller
Remote file-system browser (teach Konqueror to speak GridFTP)
Validated on Linux and Irix
Documentation
 White paper on Grid Enabled Desktop Environments
 Support open source project
– 1 day/week for 12 months
– Web site, mailing list and technical support
– Bug fixes, enhancements and new functionality as resources permit
 Release of GRENADE 1.1
 Final report
22
Supercomputing, Visualization & e-Science
Progress






GRENADE launched with workshop in December
About 2 months effort spent so far
Behind schedule
More comprehensive design than originally planned
Command line interface is new
Design is future-proofed against requirements for
– Drag-and-drop job submission
– Support for multiple credentials
 Coding has begun
23
Supercomputing, Visualization & e-Science
Possible extensions










xterm/gsi-ssh
Drag-and-drop job submission
Support for multiple credentials (singleton pattern in GRENADE 1.0)
GridFTP GUI, supporting 3rd party file transfers, remote-remote
Resource broker integrated with job submission GUI
GUI's for user management
File system synchronisation
Expose GRENADE services as OGSI services
GT3 integration
Port to Windows – messaging layer easy, Explorer integration harder,
Globus could be the show-stopper.
 Basis for P2P desktop Grids
 Your suggestion here!
26
Supercomputing, Visualization & e-Science
GRENADE: people and sponsors
Manchester Computing, e-Science Team:
 Stephen Pickles, Principal Investigator
 Jon MacLaren, Software Architect
Advanced Interfaces Group,
Dept of Computer Science:
 Steve Pettifer, Co-Investigator
 Cluster of Jameses, User Interfaces
SGI:
 Andrew Grant, Principal Contact
 Martyn Foster, Technical Contact
27
Supercomputing, Visualization & e-Science
Supercomputing, Visualization & e-Science
Manchester Computing
SVE @ Manchester Computing
World Leading Supercomputing
Service, Support and Research
Bringing Science and
Supercomputers Together
www.man.ac.uk/sve
sve@man.ac.uk