e-Science/e-Research Factoring
for the e-Framework
Matthew J. Dovey (JISC)
Jane Hunter (University of Queensland)
http://www.e-framework.org
Objectives






Review the common functionalities of the Research domain
Review the current set of services in the e-Framework to identify
overlaps
Produce a draft abstract definition including a scope and purpose
Initial mapping of associate specifications and standards
Where possible and appropriate, develop clusters of services that
will be recognisable to the Research communities to support them
in understanding and navigating the e-framework.
Identify priority areas focusing on
 'quick win demonstrators' - areas where reference models can be


rapidly developed to demonstrate to the Research community the
value of the reference model and e-Framework approach.
'pain points' - areas where the reference model approach might
help address recurring problems or issues, or move forward an
issue.
Raise any issues or conflicts encountered that need to be
addressed
http://www.e-framework.org
Learning/
Education
IT
Services
IT Services
Network
Management
Resource
Management
User Management
Admin
Group/VO
Management
Financial/Accounting
Identifier/Resolver
Management
System
Workflows
Resource
Search/Delivery
Management
Ontology Services
Management
Metadata
Services
Service
User
Management
Notification
Registry Services
Group/VO
Management
Personalisation
Management
Presentation/DeliveryIdentifier/ResolverSoftware
Management
System
Research/
Library/Information
Grid
Environment
eScience
Data/Metadata Capture
Remote Instrument
Provenance
VR/Immersive
Workflows
e-Lab notebook
Ontology Services
Visualisation
Metadata Services
Simulation/Modelling
Registry Services
IT
Research/
Library/Information
Services
Grid
Environment
Computational
IT Services
eScience
Data/Metadata Capture
Job Remote
Management
Network
Instrument
Provenance
Admin
Common
Financial/Accounting
Security
Authentication
Authorisation
Access Policies
Firewalls
Proxies
Certificates
Service Domains and Subdomains
Learning/
Education
VR/Immersive
e-Lab notebook
Visualisation
Simulation/Modelling
Computational
Job Management
Collaboration
AV Conferencing
Access Grid
Collaboration
Whiteboard
AV Conferencing
ChatAccess Grid
Whiteboard
Email
Chat
Presence
Email
Forum
Presence
Forum
Messaging
WikiMessaging
Wiki
Application
ApplicationSharing
Sharing
Common
Security
Authentication
Authorisation
Access Policies
Firewalls
Proxies
Certificates
Information Integration
Search/Delivery
Management
Service
Notification
Search Services
Management
Personalisation
Software
Presentation/Delivery
Management
Information Integration
Search Services
Publishing
Annotation
Publishing
Bibliography
Annotation
Publication
Bibliography
Documentation
Publication
Documentation
Digital
RightsRights
Management
Digital
Management
Licenses
Licenses
Languages
eLearning
Course Management
eLearning Activity Management
Curriculum
Course Management
Development
Activity Management
Learner Profile Mgt
Resource
CurriculumLearning
Management
Development
Assessment
Learner Profile Mgt
Learning Resource
Management
Assessment
Organisational
Languages
Project Management
Organisational
Personal Organisation
Project
Management Content Management
Personal
Personal
Organisation Information Management
Development
Digital
Repository
Content
Management
Personal
Data Management
Development
Information Management
Digital Repository
Data Management
Preservation
Preservation Metadata
Preservation
Preservation Metadata
http://www.e-framework.org
Overlaps and Crosswalk
Identifier
Management
Group/VO
Management
User Management
Job Management
Network
Management
Resource
Management
Software/Systems
Management
Service Management
Data Management
Information
Management
Digital Rights
Management
Metadata services
Ontology Services
Workflow Services
Provenance/Audit
trail Services
Search/Discovery
Services
Digital Repository
Services (Fedora,
DSpace, Greenstone,
ePrints)
Information
Integration Services
Presentation Services
Notification/Delivery
Services
Security Services
Collaboration
Services
Remote Instrument
Services
Visualization
Services
Simulation/Modelling
Services
Virtual/Immersive
Learning/ Research/ Library IT
Admin Common
Education Grid
Services
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
http://www.e-framework.org
eScience/eResearch Service Genres

eScience or eResearch Service Genres are those specifically
provided to enable researchers to capture, analyse, visualize
and model scientific data.
 We have distinguished these from "Grid" services which
manage jobs, network and compute power.






Remote Instrument
VR/Immersive
Visualisation
e-Lab notebook
Simulation/Modelling
Computational
http://www.e-framework.org
Collaboration Service Genres

Collaboration services are those services that enable groups of
physically distributed users to work together over networks.










AV Conferencing
Access Grid
Whiteboard
Chat
Email
Presence
Forum
Messaging
Wiki
Application Sharing
http://www.e-framework.org
Organisational Service Genres

"Organizational" services are those that
enable users to organize their projects,
activities and personal information
Project Management
 Personal Organisation
 Personal Development

http://www.e-framework.org
Grid/IT Service Genres

"Grid/IT" services are those low-level services that support the
management of a Grid environment or infrastructure.
They are typically invisible to end-users but available to and used
by systems administrators.









System Management
Service Management
Software Management
User Management
Group/VO Management
Identifier/Resolver
Network Management
Job Management
Resource Management
http://www.e-framework.org
Job Management Genre



Submit jobs to the Grid, log and monitor their status.
This is different to the testing phase of executable management.
Involves brokering and scheduling, potentially as part of a workflow.

Appropriate specifications include:







DRMAA
DAG
RSL
NPI
CI-apps
JSDL
etc.
http://www.e-framework.org
Job Management Expressions and Patterns
Define
A User will be able to define their computational needs and
computation
manage their allocations on multiple computational resources
resources required
Locate resources
Users will be able to see what resources are available, and see
what capabilities each offers.
Setup input
parameters for a
job
A user will be able to setup the input parameters for a job either
by uploading input parameter files or entering information into a
GUI for that job
Specify data
requirements
The users will be able to specify the temporary storage required
for a given run, including source/destination for streamed data.
The security precautions to be taken for this data, e.g. encryption,
can also be specified.
Submit a job
Submission of a job will be through easy to use interfaces
requiring no knowledge of the architecture behind it. A user may
submit it to any project-accessible resource, e.g. NGS or to the
test infrastructure
Specify steering
behaviour
A User can specify what parameters are to be monitored and what
parameters can be steered. Checkpointing behaviour must also be
specified.
Check status of
submitted job
A user will be able to check the status of a job submitted to NGS
or the test infrastructure
Select run
parameters for a
workflow
A user will be able to set up the input parameters for a workflow
either by uploading files for the workflow to access or by entering
the information into a GUI for that workflow.
Execute a
workflow
Execution of a workflow will be through easy to use interfaces
requiring no knowledge of the architecture behind it.
Provide metadata
The user must provide metadata concerning the simulation
Check workflow
status
A user will be able to check the status of a workflow.
Monitor state
Logging
resource usage vs. job details
Notification
Tell me when its finished or failed!
http://www.e-framework.org
Group and VO Management Genre





Defines membership of virtual organisations and their rights.
Support creating, reading, updating and deleting groups
Support creating, reading, updating and deleting data regarding membership of
groups
Support creating, reading, updating and deleting people information.
Group can index information about members, such as profiles viewable by peers,
instructors or supervisors.
Could also include items of profile information, including upload optional photo.
Profile items are viewable by others if permissions are set.

Appropriate specifications include:








IMS enterprise
IMS enterprise services (not yet available)
OKI group management OSID (not yet available)
CIM-users
VOMS
Etc.
http://www.e-framework.org
Group and VO Management Patterns and Expressions
Set up an group
Creates a new VO to establish the security boundary for an group. Access to the group's resources
should be restricted to this VO
Add user to an group
The PI will be able introduce new collaborators to the group.
Remove a user from an
group
The PI will be able expel a collaborator from the group.
Add a resource/ data to an
group
The PI will be able introduce new resources to the group.
Remove a resource/ data to
an group
The PI will be able remove resources from the group.
Authorize a user/ action pair
Allow or deny the given user to carry out the requested action
http://www.e-framework.org
Visualisation and Interactive Services Genre

Visualise data on Grid resources, either
using a remote or local tool. Data may be
output from a Grid job or even used for
steering the application.
http://www.e-framework.org
Visualisation and Interactive Services Expressions and Patterns
Data transformation
Search for
visualisation tools
A user will be able to select a visualisation tool based on agreed
selection criteria
Review
visualisation tools
A user will be able to review the metadata about a visualisation
service before selecting it.
Select visualisation On selecting a visualisation method, a user will be able to render
methods
and view results
Compare results
Users will want to be able to compare results from a job with
results from earlier runs, other executables, experimental results
etc. Comparisons could vary from side by side visualisation to
overlaying/ merging different visualisations.
Publish
visualisation
method
A new visualisation tool or service will be publishable for others
to use.
Select data to be
visualised
Setup user defined
filtering
The user specifies the filtering algorithms, and there should be
defaults
Setup user
rendering
Render data
View results
A user will be able to view results using selected visualisation
tools and data sets
Invoke
collaborative
visualisation
A user will be able to set up and commence collaborative
visualisation selecting a user or group to work with.
Perform
collabrative
visualisation
As the data is viewed and interacted with, all users will be able
to see the changes.
Steer visualisation
A user will be able to interact with a visualisation to steer a
procedure.
Generate Movies
A user will be able to generate movies from timeslice data using
a simple to use tool
Direct Manipulation
A user will be able to use different methods for direct
manipulation of visualisation, appropriate to different systems
Ability to create
visualisation
scenarios
A user is able to define visualisation scenarios
http://www.e-framework.org
Workflow Management Genre






Supports creation and management of multi-service transactions or aggregations of
choreographed services,
Users can create workflows from components which are executables with defined
states and input/ output requirements.
These can be published via a registry along with appropriate metadata for adaptation
and re-enactment by other group or general users.
Supports appropriate usage pattern of a collection of Web Services, in such a way
that the resulting composition describes how to achieve a particular business
process.
Supports the interaction pattern of a collection of Web Services; in this case, the
result is a description of the overall partner interactions.
Appropriate specifications include:












OKI Workflow OSID
Web Services Flow Language (WSFL)
Business Process Execution Language (BPEL)
SCUFL
Xlang
BPEL4WS
DPML
XPDL
GSFL
WSCL
OWL
etc.
http://www.e-framework.org
Workflow Management Expressions and Patterns
Composition/
development
Experts or users will be able to create a workflow by selecting
workflows and associating them with other workflows in a graph
like coordination structure. This process can be facilitated by a user
friendly GUI.
Edit an
workflow
Users will be able to edit a workflow using a GUI.
Test an
workflow
On selecting a workflow, published or not, a user will be able to test
it on the infrastructure provided
A 'PI' can create a registry for their group (i.e. restricted to group
Create a registry members only). Initially, their registry contains only published
workflows (copied from the master registry).Metadata is required.
User is able to upload a workflow to a group's registry, providing
Add a workflow they are a valid user with an e-Science certificate and have been
authorized to do so by the PI. Metadata is required.
Delete a
workflow
User is able to remove an workflow from their registry providing
they 'own' the workflow or have been authorised to do so by the PI.
Replace a
workflow
User is able to overwrite a workflow with a new one (= Add +
Delete). Metadata is required.
Version and
workflow
Users will be able to version their workflows and keep historic
information for later retrieval.
Search for a
workflow
Users will be able to search for workflows with selection criteria.
Review a
workflow
Users will be able to review metadata about a workflow before
selecting
On selecting a workflow, the user will be able to run it in the
Select and edit a
infrastructure provide. The user can also edit the workflow and will
workflow
be able to publish it once completed.
Test a workflow
Users will be able to select a workflow and test it on the
infrastructure provided
Publish a
workflow
Workflows will be publishable by users so that they may be
accessed by other users in or beyond their group. Publishing a
workflow will require the user to provide test data (?) and metadata
for that workflow for later retrieval. Accepted (peer review?)
workflows will be added to the master registry.
Add metadata to Users may add metadata to a workflow on publication or in their
a workflow
group's registry.
Setup security
for a workflow
On adding or publishing a workflow, a user will be able to set access
rights to that workflow, either by individuals, groups or user
attributes. Validity of user will be determined by this security and
also the security of individual executables within the workflow.
Review audit of A user may review who has accessed a workflow and the actions
workflow use
taken on it - reviewed, run or downloaded
Languages and
programming
flexibility
http://www.e-framework.org
Next Steps and Questions

Is this a useful process?
 Is this the right approach?
 Are there glaring deficiencies?
 Where do we go from here?
Filling the gaps
 Using the framework
 Building reference models

http://www.e-framework.org