Grid Portlets
Technology Update
Portals & Portlets 2006
Michael Russell <russell@man.poznan.pl>
Poznan Supercomputing and Networking Center
http://www.man.poznan.pl
Portals & Portlets - Monday 17 July 2006
Agenda
• Grid Portlets 1.3
• Introduction to Vine & Portlet Vine
• Future Plans
Portals & Portlets - Monday 17 July 2006
GridSphere’s Grid Portlets (1.3)
• Original goal of GridSphere project was to support Grid
portal development.
• However, the GridSphere portlet container itself contains
no support for Grid technologies.
• Instead, GridSphere Project has developed support for
Grid related functionality in a web application we call Grid
Portlets.
• Grid Portlets can also be used to develop applicationspecific Grid portal applications.
Portals & Portlets - Monday 17 July 2006
What Grid Portlets contains
• Grid Portlets offers a collection of portlet services that
provide a high-level API and model of the Grid.
• Grid Portlets contains a set of portlets that provide basic
Grid functionality.
• Our portlets were not meant as a complete solution for
Grid portals, but rather to showcase the Grid Portlet
Services API.
• Grid Portlets is distributed with support for GT2 services
(Mds2, GRAM, GridFTP, etc).
Portals & Portlets - Monday 17 July 2006
The Portlets
• Credential Manager Portlet
• File Browser Portlet
• Job Submission Portlet
• Resource Browser Portlet
• Resource Registry Portlet
Portals & Portlets - Monday 17 July 2006
Resource Registry Portlet
• Administrators can use the Resource Registry Portlet to
specify what resources to make available to portal
users.
Portals & Portlets - Monday 17 July 2006
Resource Browser Portlet
• Users can then see the resources that have been made
available with the Resource Browser Portlet.
• The Resource Browser Portlet provides a simple
collection of user interfaces for viewing resources.
• As we’ll see later, the Resource Browser Portlet also
supports the ability to display custom user interfaces
for viewing resources.
Portals & Portlets - Monday 17 July 2006
Resource Browser Portlet
• The Resource Browser Portlet provides a generic
framework for listing and viewing resources (and
eventually editing, deleting, searching, etc).
• The Resource Browser Portlet offers an environment for
viewing resource profiles
•
A resource profile defines a collection of user interfaces for
listing and viewing resources of a given type.
•
The Resource Browser Portlet will display the appropriate
interfaces given the set of resources and profiles deployed to
the portal.
Portals & Portlets - Monday 17 July 2006
Example profile: GridScreen
• Grid Portlets provides the ability to display custom HTML
and images in its genericre .
Portals & Portlets - Monday 17 July 2006
Credential Manager Portlet
• Users can retrieve credentials from a MyProxy
credential repository and enable their credentials for
“single sign-on” to computing resources at login time.
Portals & Portlets - Monday 17 July 2006
File Browser Portlet
• Users can browse files on remote computing resources
in a manner similar to how they might browse files on
their desktop. We have made it relatively simple to
create new directories, transfer and delete files all with
simple HTML interfaces.
Portals & Portlets - Monday 17 July 2006
Support for logical files
• Support for logical files in one browser interface! So,
instead of “registering” a physical file, they would
“upload” or “import” a file.
Portals & Portlets - Monday 17 July 2006
Job Submission Portlet
• Users define jobs using the job
submission portlet. The Job
Submission Portlet presents a
generic interface for specifying a
job.
• As we’ll see later, Grid Portlets
supports the ability to add
custom user interfaces for
specifying and monitoring jobs.
Portals & Portlets - Monday 17 July 2006
Selecting a job resource
• Moreover, the Job Submission Portlet naturally supports
wizards. The generic job user interface that come with
Grid Portlets is implemented as wizard.
Portals & Portlets - Monday 17 July 2006
Viewing job history & output
Portals & Portlets - Monday 17 July 2006
Action Component Model
• Our Grid Portlets extend from Action Component
Portlet.
• Action Component Portlet acts as a controller for resuable
JSP based components called Action Components.
• The UIs in Grid Portlets can be reused in other portlet web
applications.
• The Resource Browser Portlet and the Job Submission
Portlet allow developers to plugin in their own action
components for viewing resources and submitting jobs.
Portals & Portlets - Monday 17 July 2006
Important Services
• Credential Manager Service
• Credential Retrieval Service
• File Browser Service
• Job Submission Service
• Resource Registry Service
Portals & Portlets - Monday 17 July 2006
Example Grid Portlets 1.X Code
• Copying a set of files with the File Browser Service.
FileBrowser srcBrowser = fileBrowserService.createFileBrowser(user, srcHost);
FileLocation srcLocation = srcBrowser.createFileLocation(srcPath);
FileBrowser dstBrowser = fileBrowserService.createFileBrowser(user, dstHost);
FileLocation dstLocation = dstBrowser.createFileLocation(dstPath);
FileCopy copy = srcBrowser.copy(srcLocation, dstLocation);
copy.waitFor();
if (copy.getTaskStatus().equals(TaskStatus.FAILED)) {
throw new TaskException(copy.getTaskStatusMessage());
}
Portals & Portlets - Monday 17 July 2006
Support for other Grid infrastructures
• Support for additional Grid technologies can be added by
creating a Grid portlets project and implementing the
Grid portlet services API.
• Past projects include:
•
GT3 Portlets
•
GT4 Portlets
•
gLite Portlets
•
GridLab Portlets
Portals & Portlets - Monday 17 July 2006
Grid Portlets Usage
• In the last year since Grid Portlets 1.0 was released, the
Grid portlet service model has really started to take off.
• Many people began by using our Credential Manager
Service and started using our Job and File portlet services
as well.
• The GT4 Portlets project was created almost entirely by
contributors.
• Some people are using our Action Component model
without much advertisement on our part.
Portals & Portlets - Monday 17 July 2006
Grid Portlets 1.X Problems
• Problems & Goals
•
Enable developers to run and test Grid portlet services
outside of a servlet / portlet container
•
Add transparent support for multiple VOs (major
reengineering required)
•
Make it easier to plugin new security mechanisms
•
Address several internal limitations (persistence)
•
Improve the UI (support for AJAX, etc)
Portals & Portlets - Monday 17 July 2006
Moving To PSNC
• The Grid Portlets project moved to PSNC in February 2006.
• Inherited a great team with several years of experience in
Grid programming.
• Working on exciting projects!
•
•
•
HPC Europa (Job submission, Credential tools)
InteliGrid (Support for VOs, GAS)
OMII-Europe (interoperability, integration, testing!)
• Adding support for new technologies in OMII, etc...
•
•
•
•
EGEE (Glite Already have started, CREAM services next ;-)
Unicore (Already have done ;-)
JSDL (Already have done ;-)
OGSA-DAI
Portals & Portlets - Monday 17 July 2006
Vine & Portlet Vine
• Grid Portlets 1.X has been refactored into 2 new code
bases:
•
Vine - A base API that includes built-in support for security,
persistence and classes for modeling “resources” within
“domains”.
•
Portlet Vine - Builds on top of Vine to provide visual
support for Vine in JSR 168 compliant containers.
• Vines are configured and deployed by creating a
Vineyard. For example, a portlet vineyard makes it easy
to deploy portlet vines to a servlet & portlet container
environment.
Portals & Portlets - Monday 17 July 2006
Vine
• Vine is a modular, robust, extensible service oriented
application framework.
• Inherited & greatly improved upon the Resource / Task
model of Grid Portlets. Core Vine contains no Grid
code.
• Provides transparent support for multiple VOs (modeled
as “domains”), multiple security mechanisms
(authentication), much more.
• More stable, with shutdown hooks, improved
managment of database connections, etc.
• No dependencies on GridSphere. Can be deployed
standalone or in any application container.
Portals & Portlets - Monday 17 July 2006
Vine Projects
• Vine is developed and deployed from one or more Ant
build projects.
• Vine projects contain a set of build configurations and a
directory structure for adding source code, libraries,
configuration files, shell scripts, media files, etc.
• Unified model for configuration: same for standalone,
servlet, portlet, signed apps, etc.
• The main Vine project provides ant targets for creating
and upgrading Vine projects to support the latest Vine
build targets.
Portals & Portlets - Monday 17 July 2006
Some Additional Features
• Vine supports the ability to create the Vine database
dynamically and update database schema at startup time.
• Vine can be installed to run from a specific location on a
system for use among multiple users (i.e. from
$VINE_HOME).
• Vine can be deployed from Java Web Start with the help
of our Jar Signer project and will dynamically setup a
vine user environment, with tools for extracting
configuration files and other data from the jars deployed
with it.
Portals & Portlets - Monday 17 July 2006
Resource, the key concept
• Resource is the central concept in Vine.
• A resource is simply defined as anything that can be
utilized, i.e. a computer, a web service, a person, a book.
• A resource has a distinguished name composed of the
resource names of its parent resources and its own
resource name.
•
“/Domain=root/HostResource=rage1.man.poznan.pl/Service
Resource=ssh2”
• Resources have a set of attributes that describe or
parameterize the resource for use within an application.
Portals & Portlets - Monday 17 July 2006
Basic Idea…
Portals & Portlets - Monday 17 July 2006
Tasks modeled as resources
Portals & Portlets - Monday 17 July 2006
Domains define the application
• A domain is a resource that contains a set of resources
and “sub domains”.
• All Vine services operate within a given domain.
• All Vine applications have a “root” domain, and sub-
domains defined therein. Sub-domains of root can define
their own resources, visible only to that domain.
• Sub-domains can also define “domain rules” for including
or excluding resources from their parent domains
• Domains can be registered at startup time through an XML
based resource registry as well as at runtime...
Portals & Portlets - Monday 17 July 2006
The Root Domain
Portals & Portlets - Monday 17 July 2006
Domains configuration
• Domains defined in a
•
•
common directory, with
sub-domains defined in
sub-directories.
Note that domains can
still contain overlapping
sets of resources.
But this structure makes
it easy to add / remove
domain defintions.
Portals & Portlets - Monday 17 July 2006
Sub-domains
• Here is a simple example of a sub-domain definition for
PSNC.
• We are saying only include those resources that have
“PSNC” or “PolishGridCA” in their resource name.
• All resources in the parent domain (in this case, the “root”
domain) that satisfy these requirements will be included in
the PSNC domain.
Portals & Portlets - Monday 17 July 2006
Vine Services
• Vine contains several core services, most of which are
concerned with initializing a Vine application when it is first
invoked.
• The most important service is the Resource Manager.
• The Resource Manager is utilized by all other services in
Vine projects to manage resource information.
Portals & Portlets - Monday 17 July 2006
Resource Manager
• Resource Manager provides methods for listing,
creating, updating and saving resource (entries) in the
Vine database.
// Get resource manager service
ResourceManager resourceManager = (ResourcerManager)
serviceContext.createService(ResourceManager.KEY);
// Create a host resource
HostResourceProvider hostResource = (HostResourceProvider)
resourceManager.createResource(HostResource.KEY);
// Set the host name of this resource
hostResource.setHost(”rage1.man.poznan.pl”);
// Save the host resource
resourceManager.saveResource(hostResource);
Portals & Portlets - Monday 17 July 2006
So for example...
• We also might want to use the Resource Manager to list all
active file tasks submitted by a user (in a given domain).
// Get resource manager service
ResourceManager resourceManager = (ResourcerManager)
serviceContext.createService(ResourceManager.KEY);
// List all active file tasks
List fileTaskList = resourceManager.getResourcesByTypeQuery(FileTask.KEY,
“resource.StatusValue=’ACTIVE’”);
Portals & Portlets - Monday 17 July 2006
Vine Philosophy
• Functionality is added in layers, Grid Vine for example,
provides its own feature-rich API built upon Vine concepts.
• Functionality is added by defining new types of resources.
• Users then deploy which sets of resources they want to
support for a given application.
• Users create domains to define the problem space in
which an application operates.
• A domain defines all the resources an application has to
work with, and hence the overall capabilities the
application provides.
Portals & Portlets - Monday 17 July 2006
Vine Projects
• Some of the Vine projects that have been created:
•
Grid Vine - Models Grid concepts
•
Groupware Vine - Models organizations, persons, etc.
•
GT2 Vine - Implements various Grid resources and concepts
with GT2
•
SRB Vine - Implements “file resource” concepts from Grid
Vine
•
Unicore Vine - Implements “info resource” concepts from
Grid Vine
•
Certificate Manager - Provides GUI application for
requesting certificates and creating credentials for use in
particular domains.
Portals & Portlets - Monday 17 July 2006
Comparing the two APIs
Grid Vine (NEW)
FileBrowser srcBrowser = fileResourceManager.createFileBrowser(srcHost);
FileLocation srcLocation = srcBrowser.createFileLocation(srcPath);
FileBrowser dstBrowser = fileBrowserService.createFileBrowser(dstHost);
FileLocation dstLocation = dstBrowser.createFileLocation(dstPath);
FileCopy copy = srcBrowser.copy(srcLocation, dstLocation);
copy.waitFor();
if (copy.getTaskStatus().equals(TaskStatus.FAILED)) {
throw new TaskException(copy.getTaskStatusMessage());
}
Grid Portlets (OLD)
FileBrowser srcBrowser = fileBrowserService.createFileBrowser(user, srcHost);
FileLocation srcLocation = srcBrowser.createFileLocation(srcPath);
FileBrowser dstBrowser = fileBrowserService.createFileBrowser(user, dstHost);
FileLocation dstLocation = dstBrowser.createFileLocation(dstPath);
FileCopy copy = srcBrowser.copy(srcLocation, dstLocation);
copy.waitFor();
if (copy.getTaskStatus().equals(TaskStatus.FAILED)) {
throw new TaskException(copy.getTaskStatusMessage());
}
Portals & Portlets - Monday 17 July 2006
Portlet Vine
• Portlet Vine builds upon the Action Component Portlet
development model and includes 2 central portlets:
•
Resource Registry Portlet
•
•
Extended to provide support for defining the root and subdomains.
Resource Browser Portlet
•
Lists resources within the active domain.
Portals & Portlets - Monday 17 July 2006
Portlet Vine Projects
• The following projects have been created:
•
Grid Portlets 2.0
•
GT2 Portlets 2.0
•
GS Portlets
•
•
•
•
•
Credential Manager Portlet
File Browser Portlet
Job Submission Portlet
Defines GT2 resource profiles
Provides new “domain login portlet”
• More will come very soon!
•
•
•
GT4 Portlets 2.0
Unicore Portlets
EGEE Portlets
Portals & Portlets - Monday 17 July 2006
Domain Login Portlet...
• Packaged with the GS Portlets Project
• Enables users to select a domain (VO) to login to.
• But we can adapt this approach to other portlet
containers!
Portals & Portlets - Monday 17 July 2006
Certificate Manager Application
• Builds upon Vine to allow users to “activate” credentials
•
for a particular domain.
Each domain can have its own collection of certificate
authorities, certificate repositories (MyProxy), etc.
Portals & Portlets - Monday 17 July 2006
Future Plans
• Vine, Porlet Vine and related projects scheduled for
release in September 2006.
• This will include a Grid Portlets 2.0 release, based on the
Vine application framework.
• Grid Portlets 2.0 will include significantly enhanced UIs
•
•
•
Interactivity (AJAX)
Use or introduction of more sophisticated UI tools
Polished look & feel (borrow from Windows, Mac OS X, etc)
• Completely new model for representing applications in
store... prototype in Septemeber 2006, release end of
year (Grid Portlets 3.0?)
• Explore Vine in other portal frameworks, other language
environments (Ruby on Rails), etc.
Portals & Portlets - Monday 17 July 2006
Can I use Vine now?
• Vine IS being used in HPC-Europa, but not ready for
general release quite yet (need documentation!).
• Email me for instructions for how to access & use Vine
from our Subversion server:
•
svn co https://svn.gridsphere.org/vine/vine/trunk
vine
• Information will be announced before the end of August
describing the overall vision of Vine and documentation on
how to access and use from SVN.
• Vine may be released before Portlet Vine.
Portals & Portlets - Monday 17 July 2006