Science Gateways and Clouds
Dr. Wibke Sudholt
CloudBroker GmbH
Technoparkstrasse 1, CH-8005 Zurich, Switzerland
Phone: +41 44 633 79 34
Email: wibke.sudholt@cloudbroker.com
Web: http://www.cloudbroker.com
SCI-BUS is supported by the FP7 Capacities Programme under contract no. RI-283481
Presentation Outline
• Cloud computing
• Science gateways
• CloudBroker Platform
• SCI-BUS project
• GaaS = Gateway as a Service
© CloudBroker GmbH
All rights reserved.
EGI Technical Forum 2012
20.09.2012 / 2
Cloud Computing
SCI-BUS is supported by the FP7 Capacities Programme under contract no. RI-283481
Cloud Definition
• Self service: Access to computer resources on demand,
without much initial investment in time, money or
expertise
• Pay per use: Only pay for what you actually use in
small steps, OpEx instead of CapEx, micropayments
• Scalability: Nearly unlimited elasticity, grow and shrink
the resource size upon request
⇒ Not only change in technology, but change in business
model
⇒ Interface between providers and users set at the right
place
© CloudBroker GmbH
All rights reserved.
EGI Technical Forum 2012
20.09.2012 / 4
Cloud Services
SaaS = Software as a Service
• Scientific / office / business / etc. applications
• Salesforce, Google Apps, etc.
PaaS = Platform as a Service
• Programming / deployment frameworks, distribution /
messaging / monitoring systems, databases, etc.
• Google App Engine, Windows Azure, etc.
IaaS = Infrastructure as a Service
• Virtual machines, storage, network, etc.
• Amazon EC2 and S3, Rackspace, etc.
© CloudBroker GmbH
All rights reserved.
EGI Technical Forum 2012
20.09.2012 / 5
Types of Clouds
Private Clouds
Hybrid Clouds
Public Clouds
• In-house or hosted
• Just for internal use
• Multi-tenant across
organization sites,
departments,
groups and users
• Focus on selfservice and
accountability
• Mixture of public
and private clouds
• Focus on scalability
and failover
• Offered by partner
organizations or
cloud providers
• Available to certain
external user
communities or to
everybody
• Multi-tenant across
organizations
• Focus on ondemand and payper-use
© CloudBroker GmbH
All rights reserved.
EGI Technical Forum 2012
20.09.2012 / 6
Science Gateways
SCI-BUS is supported by the FP7 Capacities Programme under contract no. RI-283481
Gateway Description
•
•
•
•
•
•
•
•
Portals to scientific applications for scientific end users, hiding the
infrastructure complexity in the background
Usually focus on a certain scientific domain, application area, geographic
region, user group, organization or similar
Front-ends might be web-based, client software, mobile apps, APIs, CLIs, etc.
Back-ends are typically distributed computing infrastructures (DCIs) such as
clusters, supercomputers, grids or clouds, but can also include sensors, etc.
Additional functionalities such as data handling, workbenches, workflows,
parameter sweeps, visualization, graphs, user management, community
interaction, social media, publications, reporting, etc.
Often have domain-specific functionalities such as naming, annotation and
display of items, metadata, item connections, access to content databases,
etc.
Example scientific domains include molecular science, proteomics, medical
imaging, astrophysics, seismology, rendering, etc.
Example providers include academic institutes, scientific communities,
research projects, NGIs, companies, etc.
© CloudBroker GmbH
All rights reserved.
EGI Technical Forum 2012
20.09.2012 / 8
© CloudBroker GmbH
All rights reserved.
Private
Public
Private
Public
Private
Public
Desktop Grids
(BOINC, Condor, etc.)
Institutional Grids
(Globus, UNICORE, etc.)
Clusters, Supercomputers
(SGE, PBS, LSF, etc.)
User Management, Community
Visualization, Reporting
Workflows, Parameter Sweeps
Data Handling, Metadata
Applications, Databases
Gateway Architecture
User
Science Gateway
Gateway Technology (Liferay, WS-PGRADE/gUSE, etc.)
DCIs
Clouds
SaaS
PaaS
IaaS
EGI Technical Forum 2012
20.09.2012 / 9
CloudBroker Platform
SCI-BUS is supported by the FP7 Capacities Programme under contract no. RI-283481
CloudBroker Platform
• Web-based application store for the deployment and
execution of scientific and technical software in the
cloud
• On demand, pay per use, browser / programmatic /
command-line access, cross-domain
• Uses infrastructure as a service (IaaS) from resource
providers
• Offers platform as a service (PaaS) for software vendors
and software as a service (SaaS) for end users
• Easy to use, speeds up time to market, no need for
own HPC infrastructure, OpEx instead of CapEx
© CloudBroker GmbH
All rights reserved.
EGI Technical Forum 2012
20.09.2012 / 11
Platform Architecture
End Users, Software Vendors, Resource Providers
CLI
Java Client Library
REST Web Service API
Chemistry
Applications
Biology
Applications
Health
Applications
Engineering
Applications
…
Applications
CloudBroker Platform
Amazon
Cloud
© CloudBroker GmbH
All rights reserved.
IBM
Cloud
OpenStack
Cloud
Eucalyptus
Cloud
CloudBroker Integration
Web
Browser
UI
User Tools
…
Cloud
EGI Technical Forum 2012
20.09.2012 / 12
Clouds and Applications
Cloud Resources
Application Software
•
•
•
Use your own resources and/or resources
offered by others
Configuration in platform needed for own
resources:
–
•
•
•
•
Amazon EC2 / S3
IBM SmartCloud Enterprise
© CloudBroker GmbH
All rights reserved.
Amazon EC2
IBM SmartCloud Enterprise
Software offered by CloudBroker:
–
•
Executables, deployments, prices
Auto-deployment with installation script:
–
–
•
Multi-threading and MPI possible
Configuration in platform needed for own
software:
–
OpenNebula
Resources offered by CloudBroker:
–
–
–
OpenStack (via EC2 API)
Eucalyptus / Walrus
Support in preparation:
–
•
Amazon EC2 / S3
IBM SmartCloud Enterprise / Nirvanix
Currently supported private clouds /
storages:
–
–
•
Instance types, regions, storages, prices
Currently supported public clouds / storages:
–
–
•
Use your own software on your own
resources and/or software offered by others
on resources offered by others
Any batch-oriented Linux command-line
application:
BLAST, AutoDock, Gromacs, X! Tandem,
OpenFOAM
Also already ported:
–
Rosetta, TransAT, Siesta, T-Coffee, FoldX, R
EGI Technical Forum 2012
20.09.2012 / 13
Platform Features
•
Computations:
•
– Job creation, submission and control
– Data file upload and download
– Automatic queuing, instance startup
and shutdown, data transfer (via cloud
storage) and fault tolerance
– Pipeline jobs, archives and tagging
•
•
– Registration (with user check in public
platform)
– User authentication with email and
password
– SSL transport-layer encryption
– Dashboard, activity logs
– User and organization management
– Different organization, user and
visibility levels
– Resource access and software license
limits
– Invoicing and billing, pre-payment or
monthly payment
– Documentation
Interfaces:
–
–
–
–
Web browser UI
REST web service API
Java client library
CLI
Marketplace:
– Platform usage surcharges as
percentage of resource and software
prices
– Free resources and software have zero
surcharges
© CloudBroker GmbH
All rights reserved.
Further features:
•
Platform flavors:
– Public installation under
https://platform.cloudbroker.com
– Also hosted and in-house installations
offered
EGI Technical Forum 2012
20.09.2012 / 14
Reference Examples
• Protein modeling with Rosetta (collaboration with
ETH Zurich and IBM)
• Computational fluid dynamics web UI for TransAT
(collaboration with ASCOMP and HSR)
• Platform node for KNIME (Konstanz Information
Miner) workflow framework
• Access to clouds in SCI-BUS (SCIentific gateway
Based User Support) EU FP7 project
• Further projects in industry, government and
academia
© CloudBroker GmbH
All rights reserved.
EGI Technical Forum 2012
20.09.2012 / 15
SCI-BUS Project
SCI-BUS is supported by the FP7 Capacities Programme under contract no. RI-283481
SCI-BUS Setup
•
•
•
•
EU FP7 project
http://www.sci-bus.eu
Running 10/2011-09/2014
Consortium:
– 15 partners from 10 countries (10
academic, 5 industrial)
– 6 subcontractors
– Additional associated partners (via
MoU)
• Goals:
– Provide researchers seamless
access to major computing, data
and networking infrastructures and
services
– Develop gateway technology
– Support community gateways
– Focus on scientific workflows
© CloudBroker GmbH
All rights reserved.
• Generic-purpose technologies:
– WS-PGRADE/gUSE from MTA
SZTAKI (see http://www.guse.hu
and presentation of Zoltan Farkas
in the morning)
– CloudBroker Platform
• Initial gateway communities:
– Systems biology, computational
chemistry, astrophysics,
heliophysics, seismology, medicine,
rendering, electronic document
handling, business process
optimization, SMEs, software
testing
• SCI-BUS booth at EGI Technical
Forum 2012:
– CloudBroker Platform demo at
15:40 today
EGI Technical Forum 2012
20.09.2012 / 17
SCI-BUS Architecture
Source: SCI-BUS
© CloudBroker GmbH
All rights reserved.
EGI Technical Forum 2012
20.09.2012 / 18
WS-PGRADE/gUSE
Source: http://www.guse.hu/?m=architecture&s=0
© CloudBroker GmbH
All rights reserved.
EGI Technical Forum 2012
20.09.2012 / 19
CloudBroker Platform
Integration in WS-PGRADE/gUSE
• Method:
– Use of the CloudBroker
Platform Java client library
• Existing features:
– CloudBroker DCI Bridge
backend plugin and
configuration interface
– CloudBroker security portlet
– Concrete workflow portlet
extension for CloudBroker jobs
• Upcoming features:
– Support for running any kind of
application via wrapper script
– Display of prices, costs and
invoices
– Different types of user mapping
Source: Zoltan Farkas, MTA SZTAKI
© CloudBroker GmbH
All rights reserved.
EGI Technical Forum 2012
20.09.2012 / 20
GaaS = Gateway as a Service
SCI-BUS is supported by the FP7 Capacities Programme under contract no. RI-283481
Gateway Sustainability
• Over time, science gateways need to become sustainable
• It does not make sense to offer a gateway if there is not enough
user interest in it
• There are more and more moves towards also metering and billing
academic services
• Many governments don’t have enough money for large CapEx
anymore and thus prefer stepwise OpEx
• Sponsors are hard to attract to science gateways
• Not always gateways can be outsourced or a spin-off company
generated from them
⇒ Base government funding to build, operate and support science
gateways is needed
⇒ Beyond that, apply the cloud business model to science gateways
© CloudBroker GmbH
All rights reserved.
EGI Technical Forum 2012
20.09.2012 / 22
Gateway Business Models
• Build science gateways from cloud building blocks (model used in
the CloudBroker Platform and SCI-BUS)
• Charge for not freely accessible resource consumption and
application usage (model used in the CloudBroker Platform and SCIBUS)
• Have a freemium model with extra charges for additional services
(model used in the CloudBroker Platform and SCI-BUS)
• Offer user subscriptions with different features
• Let commercial and other external users pay for gateway services
• Have a shareholder charge-back model
• Provide professional consulting, training and support
⇒ Develop your own business model
© CloudBroker GmbH
All rights reserved.
EGI Technical Forum 2012
20.09.2012 / 23
Thank you! – Questions?
Try out the CloudBroker Platform under
https://platform.cloudbroker.com
Visit SCI-BUS under
http://www.sci-bus.eu
SCI-BUS is supported by the FP7 Capacities Programme under contract no. RI-283481