HotPage Portlet Development at
TACC
Eric Roberts
Texas Advanced Computing Center
TEXAS ADVANCED COMPUTING CENTER
Talk Outline
• Historical Overview
• Motivation
• Portlets + GridPort 2.3
• Current portlets
• Difficulties with Jetspeed
• Future Directions
2
History
• Current HotPage portals
– PACI, NPACI portals
– Interface written in Perl / CGI
– Accessed via Web Browser
• Perl CGI has limitations but we’ve still managed
to come quite far with it
– Telescience / BIRN portal
 Remote instrumentation
– GAMESS portals

Chemical modeling
3
Why Move To Portlet Technology?
• More Complex, but today’s grids require these
capabilities
• The need for a more powerful portal technology
– Perl HotPage lacks:
 Object-oriented framework
Object persistence / caching
– Portlets offer these plus:
 Customizable user interface


Pluggable, reusable components (portlets)
4
HotPage 3.0 alpha
• Implemented using Jetspeed and portlet
technologies
• Currently “wrapping” GridPort 2.3.1
– Use a thin CGI “wrapper” layer to communicate
between Jetspeed and GridPort
– Uses Globus Toolkit 2.2 / GSI security
• Leverages GridPort Information Repository
(GPIR) Query Web Services for informational
data
• Still need to use Perl / CGI to interact with
current GridPort 2.3 release
5
Why Jetspeed?
• Open Source
– It’s free
– Jakarta project has large growing developer community
• Portability
– Share portlet code
• Growing Acceptance
– Used by other organizations in the Grid portal community
 Indiana University (Alliance portal)
 University of Michigan (CHEF Project)
– Being used in various other projects
 DoD PET portal
 SciDAC DOE Fusion portal
 OGCE portal
6
Architecture: Portlets + GridPort 2.3
Tomcat 4.1.x
Web Server
GridPort 2.3 Services
Jetspeed
portlet
HTTP
portlet
Client
servlet
servlets
7
CGI Wrapper Layer
HTTP
Query WS
GridPort 2.3 Globus 2.2
Challenges Wrapping GridPort 2.3
• To track a users session in GridPort it sets cookies in the
browser. However, it is complicated to track users with
cookies using Jetspeed
• With help from Charles Severance from University of
Michigan we figured out how to leverage the Tomcat
session id to track users
• This involved minor modifications to GridPort in order to
accept the session id if it was available
• This way GridPort 2.3 accommodates both old and new
HotPage models
8
Jetspeed Login Servlet using GridPort 2.3
Client
Tomcat
Web Server
running
Jetspeed
tomcat session id
username
password
Logged In!
GridPort
Certificate
Repository
GridPort creates
session file
from session id
Verify username /
password with
Jetspeed user DB
Hypersonic SQL
Database
9
GridPort creates
proxy from
certificate w/
username &
password
GridPort creates
session file
from session id
Current Portlets
• Login/Logout Servlets
– GSI Authentication
• GridPort 2.3 wrapper portlets
–
–
–
–
File Listing, File Upload
Globusrun command execution
Pass results inside XML and apply XSLT
Also implemented using Velocity
• GPIR Informational Web Service portlets
–
–
–
–
System Monitor
Load, NodeMap, Jobs, MOTD, etc…
XML received from WS transformed with XSLT
Implemented as Velocity portlets
10
GridPort 2.3 Wrapper Portlets
TEXAS ADVANCED COMPUTING CENTER
Manage Files
• Upload files to users portal space
• List files
12
Command Execution (using Globusrun)
13
GPIR Portlets
• System Monitor
– Static, Load Status and Job and Queue information
– Includes Job Information for individual resources
• Machine Summary Suite
– Customizable machine menu
 Add/remove machines from menu using Jetspeed
parameter styles
– View NodeMap, Machine Information, Message of the
Day, Load and Job/Queue Information for each
machine
14
System Monitor
15
GPIR portlets
16
Currently Available
• Most recent build available at:
– http://hotpage.tacc.utexas.edu:2080/hotpage
• Capabilities:
– Anonymous Informational View
– Grid View
– Individual Resources View
– File Management
– Command Execution
17
Roadmap
• Full implementation of GridPort 2.3 functionality
– Batch Job Submit
– Job Status
– Etc.
• Integrate with GridPort 3.0
– Pure Java…no more Perl
• Administrative
– Account acquisition
• Other
– Integrate MyProxy, GridFTP and other portlets
18
Difficulties with Jetspeed
• Poor Documentation
• Lack of tools
– No IDE plug-ins for portlet testing like Sun ONE
Studio or WebSphere
• Slow Development
– Open Source projects, although free, can have long
development times
• Complex
– Steep learning curve for new portlet developers
• However, Jetspeed these challenges can be
overcome
19
Future Directions
• OGSA Compliance
– What does it mean to be an OGSI client
• Integration with GridPort 3.0
• J2EE Integration
• More customization capabilities
• Struts and JSF
20
GridPort 3.0 Plans
Tomislav Urban
Texas Advanced Computing Center
TEXAS ADVANCED COMPUTING CENTER
Current: GridPort 2.3
• Implemented in Perl
• “Services” are usual suspects:
–
–
–
–
–
Authentication/MyProxy
Job Submission
File Transfer
Information Services
Data Access (SRB)
• Approach
– Simple to install (Perl, little if any server side
configuration)
– Light-weight
• Current Portals…
22
NPACI HotPage
23
TACC HotPage
24
Telescience: Access to Instruments/Data
• Uses GridPort to Integrate
Telescience technologies with the
Grid
• Access to instruments
• Globus job control
• SRB data collections
•Migrating to BIRN
25
Fusion Grid Portal
• GT2.x
• GP2.x
• Java CoG
• SRB
• JetSpeed Based
Portal
• Will move to OGSA
(GT3) when ready.
Fusion Grid Portal
(Apache, Jetspeed)
Portlet
Components
EFIT
Portal
Web Services
Fusion
Monitor
System
Fusion Grid
Shell
GRID
Web
Services
GRID
S/W
MDS+
Viewer
Analysis/
Applications
EQDISK
Grid Portal
Catalogue
User
Data
MDS+
Collection/
Archive
DAS
Plasma
Shot
26
SRB/
MCAT
Current Status 2.x
• While moving on to GP3 design and
development, we are continuing to develop new
presentation technologies to work with GP2.x
(e.g. portlets, web services)
• 2.x development and support will continue
through 2004
• NPACKage
27
Motivation for next Release
TEXAS ADVANCED COMPUTING CENTER
Motivation for next release
• Requirements Evolution
• Technology Evolution
29
Requirements Evolution
• Larger user community
–
–
–
–
–
–
–
Campus Grid
SciDAC/DOE (Fusion Grid)
NPACI
DOD
State of Texas (TIGRE)
NMI Testbed
Collaborations
• Drives the need to support multiple VOs from a
single GridPort instance with improved
scalability and multi-portal management
30
Requirements Evolution
• Highly Distributed Users and Resources
• Drives the need to make GridPort services
accessible over Web/Grid services
• Intelligent Workflow
– Web Services based workflow
– Need the ability to automate typical use scenarios, job
submittal, file movement, visualization, etc.
• Remote Visualization
– Incremental Results
– Application Steering
31
Requirements Evolution
• Data
– Grid Meta Data
– The need for comprehensive gird meta-data including
VO-Resource-Site relationships, not just Job and
Load data
• Data Collection Access
– Integrate file and data collection access into GirdPort
– SRB integration supported under GP2.x will be kept
for GP3.0
32
Requirements Evolution
• Generalized Grid Computing Environments
(GCE)
• Application Profiling
– Need to enable the retention of run parameters
across multiple Job runs
• Advanced Scheduling
– Integration with one or more Grid Meta-schedulers
• More Sophisticated User Interfaces
– Expanded Client Types
 Portals, PDAs, Shells (Command Line Interface)
33
Technology Evolution
• Many technological changes since original
GridPort was conceived:
– OGSI/OGSA
 GT3.0
– XML
– Java
 J2EE
– Jetspeed/Portlets
34
Grid Middleware Role
• Seems to be a lot of work ongoing in the space between
low level grid software (e.g. Globus) and presentation
(Portlets and Portals)
• This suggests that there may be distinction between
“high” and “low” level grid services
• “High-level” middleware addresses:
– The need to abstract low-level services (e.g. Globus)
– Integration of various Grid services under a uniform API (SRB,
NWS, Globus, etc.)
– Service coordination
 Workflow
– User centric services
 Personalization (e.g. “My Jobs”, “My Applications”, etc.)
 Single sign-on
 Authorization
35
Approach…
TEXAS ADVANCED COMPUTING CENTER
Grid Middleware Architecture
• Two possible philosophies:
– Layer Approach
 Introduce a layer between low-level grid services and
presentation
 Handles most or all business logic
 Coordinates inter-service flow
 Provides common services and centralized persistence
 May cause dependencies
– Collection of services
 Each service talks directly to low-level grid services
 May need to provide its own configuration and persistence
mechanisms
 Each service remains independent and standalone
37
UI
• Expanded support for a
variety of UI clients
UI
– Thin Client
 Browser
– Think Client
 Desktop Application
(Client-Server)
– Command Line
 GCE Shell
– PDAs
– Functionality must be
available beyond the portlet
framework
Applications / Portals / Portlets
GridPort
(Service Aggregation,
Workflow)
OGSA Services
OGSI (GT3)
OS
Resources
38
Presentation
• Development of Portletbased portals and
applications
• This layer is reduced
largely to presentation
issues with GridPort or
other “high-level”
middleware driving most
application logic
• Some Portlets may talk
directly to low-level
services
UI
Applications / Portals / Portlets
GridPort
(Service Aggregation,
Workflow)
OGSA Services
OGSI (GT3)
OS
Resources
39
GridPort
• Workflow – interaction
between OGSA Service
components
• Component Approach
UI
Applications / Portals / Portlets
GridPort
(Service Aggregation,
Workflow)
– need robust OO design 
Java
• RDBMS for persistence
• Goal is to present the
Application/Portal
developer with a simple
unified API and built-in
Application Logic (e.g.
Workflow) to speed and
aid rapid development
OGSA Services
OGSI (GT3)
OS
Resources
40
Backend
• Distributed grid and web
services (OGSA)
• Will support
UI
Applications / Portals / Portlets
–
–
–
–
–
GridPort
(Service Aggregation,
Workflow)
OGSA Services
Security (GSI)
Scheduling (LSF)
Data Access (SRB)
Job Submission (GRAM)
Etc.
• Awaiting further definition
of required grid services
from GGF (OGSA-WG),
this will drive the
selection of which
services to aggregate
under GridPort
OGSI (GT3)
OS
Resources
41
J2EE
• GridPort 3.0 will be built on top of J2EE, leveraging the
significant benefits of the J2EE platform
–
–
–
–
–
–
–
Security
Transactionality
Distribution
Persistence
Scalability
Failover
Etc., etc.
• We are very interested in taking advantage of
forthcoming JBoss and WebSphere OGSI integrations
– “Write EJBs, expose Grid Services”
– Portlets then interact with EBJs from within or without J2EE Web
Container
– Would like to see JBoss Portlet implementation
42
GP3 Architecture
Web Container
Thin
Clients
Jetspeed
Portlets
Servlets
Web
Services
Grid
Services
JSPs
J2EE
Application
Server
OGSA
Services
http
Security
OGSI
Etc.
EJB Container - GridPort
Thick
Clients
RMI
Data
GridPort
Session Beans
Entity Beans
CMP
GPIR
43
Grid Services
• GridPort 3 services will be exposed as OGSI
compliant Grid Services for remote access
• They will also be available directly from within
the J2EE web container (or via RMI) in order to
ease scalability and performance concerns
– Use local direct calls where possible, Grid or Web
Service calls only where necessary
44
GPIR
• All data supporting portal development will be
cached in GPIR
• This data may come from a variety of sources,
or be produced internally
• GPIR is simply GridPort’s persistence
mechanism, though it too is exposed as a web
service
• GPIR focuses on multi VO Grid metadata
• Will be used to cache user sessions for
“statefullness” across multiple portal visits
45
Shift in Approach
• The use of J2EE, DBMS, and Java represent a
significant shift away from the light-weight approach
of the Perl-based GridPort 2.x
• Significant server-side resources now required
• Significant increase in complexity
• This is mitigated by the availability of GridPort
though a web/grid services interface
• Suggests two tiers of GridPort users
– User of an existing GP installation for a VO
– As currently, installation of one’s own GridPort
instance
• “Simple and practical”, remains the basic approach
46
Current Status 3.0
• Seeking to bring Software Engineering rigor to
GP3 development
• Development Methodology Established
– UML
– Rational Unified Process
– Building Development Environment
 Dev, QA, Publish/Production
• Currently in Requirements Gathering phase
47
Summary
• Seek to have pre-alpha prototype complete by
end of Fall 2003
• Will be using iterative development approach,
starting with core (existing GP2.x) services
implemented in the new architecture, then
moving on to new features
48
Summary
• Bottom line:
• GridPort seeks to aggregate core grid services
in the interest of presenting simple, powerful and
streamlined access to backend grid services and
higher-order workflow to UI developers (Portals,
Portlets, Shells, Applications, etc.)
49