OpenStack CE
Technology Review & Demo
Egan Ford
IBM Distinguished Engineer
egan@us.ibm.com
© 2012 IBM Corporation
PPT’s and Videos: http://xmission.com/~egan/cloud/
2
© 2012 IBM Corporation
Agenda
•
•
•
•
•
•
•
3
IBM SmartCloud and OpenStack
Cloud Taxonomy
Some OpenStack Public Use Cases
What is OpenStack
OpenStack Resources
IBM Resources/Solutions for OpenStack Available Today
OpenStack (Video) Demo
© 2012 IBM Corporation
Open architectures enable real innovation through interoperability
Evolve existing
infrastructure
to Cloud
Accelerate
adoption with
expert integrated
systems
Immediate access
to a managed
platform
Common Open Standards-based Cloud Management Services
4
© 2012 IBM Corporation
IBM embraces & invests in open source to foster innovation
Application
Servers
Service Oriented
Architecture
Cloud
Computing
Systems of
Interaction
Service
Orientation
June 1998: IBM enters into an
engineering agreement with The
Apache Group for development of the
open-source Apache HTTP server
software eventually becoming the
leader of the new Application Server
market
September 1999: IBM capitalizes
on an untapped market trend and
begins participating in the community
development of Linux with a $60M
annual investment
5
Social
Business
November 2001: IBM rallies 150 influential
vendors and the development community
around a new tools environment with a $40
Million software donation disrupting the
leadership of the software development
ecosystem
Open Cloud
Architecture
September 2012: IBM orchestrates the
launch of The OpenStack Foundation
boasting $10 million in funding and 5,600
members changing the dynamics of the
Cloud ecosystem
© 2012 IBM Corporation
An open cloud architecture is emerging…built on open technologies
OAuth
TOSCA
cloudfoundry.org
OSLC
Hardware
6
© 2012 IBM Corporation
The OpenStack Foundation – IBM is a driving force in it’s success
OpenStack is a global collaboration of developers & cloud
computing technologists working to produce an ubiquitous
Infrastructure as a Service (IaaS) open source cloud
computing platform for public & private clouds.
Software
Defined
Environmen
t
Platinum Sponsors
7
APR
2012
Exponential growth in 1+ YR
150
Contributors
2600 Individuals
Sep
2013
1021
Contributors
11,800+ Individuals
Gold Sponsors
© 2012 IBM Corporation
IBM is working to accelerate OpenStack Foundation success
LinkedData,
OSLC
TOSCA
OpenStack
Open Source
Reference
Cloud Standards
Customer Council
(CSCC)
Cloud Computing
Reference Architecture
(CCRA)
Because an open interoperable Cloud
is critical for flexible cloud deployment
and customer success…
12
IBM has 12 core contributors
2
IBM is #2 in contributions
90
IBMers have signed the CLA
IBM’s ecosystem approach to an
Open Cloud Architecture
370+
8
IBMers working on OpenStack – from formation of the
Foundation to Code Quality & New Function
© 2012 IBM Corporation
IBM contributions to OpenStack are wide-ranging, contributing to
OpenStack success and delivering real value
OpenStack Dashboard
Enables administrators and users to access & provision cloudbased resources through a self-service portal.
• Globalization and crowd-sourced translation integration (SOS)
• Cross hypervisor testing and validation (CCS)
OpenStack Compute
Provision and manage large networks of virtual machines
• Platform integration (CCS)
• High Availability enhancements (RES)
• Resource optimization (RES)
• Live upgrade contributions (LTC)
• Enablement for P & Z Systems, DB2 (CCS, SOS)
• ESXi support (SOS w/ VMWare)
• VM group enablement in scheduler (RES)
• CPU allocation for vCPUs (RES)
OpenStack Shared Services
Create petabytes of secure, reliable storage using standard HW
• Support for key emerging networking standards (RES)
• Quantum blueprints & migration from Nova (LTC)
• FibreChannel support (RES)
Libraries that provide image management, authentication &
security across all OpenStack projects
• Security & authentication enhancements (CCS, SOS)
• Image activation for OVF (CCS)
• Guest level metric collection (CCS)
• APIs: Enablement for key emerging standards (SOS)
• Membership services enhancements (CCS, RES)
• Glance: multiple image locations (CCS)
OpenStack Object Store
General OpenStack contributions
Create petabytes of secure, reliable storage using standard HW
•Block & object storage enablement for IBM capability (RES)
•Nova blueprints (LTC)
•Cinder local storage & local instance clone (CCS)
•Efficient clone image in Cinder SVC driver for cFlex (RES)
•Nova & Cinder storage blueprints (CCS, SOS)
•Storwise/SVC driver update – support iSCSI CHAP auth (SOS,
RES)
•Wsgi application interface enabling external web server (RES)
•Swift / Keystone interface for Keystone v3 API (RES)
IBM CONFIDENTIAL
•Drive IBM value-add capability from SCP (CCS)
•Community facing contributions – bug fixing, community
building & promotion (LTC, SOS)
•QA items (LTC)
Participation
Key
OpenStack Networking
CCS – Common Cloud Stack (STG & SWG)
LTC – Linux Technology Center (STG)
SOS – Standards & Open Source (SWG)
RES – Research
© 2011 IBM Corporation
IBM SmartCloud solutions are moving to an OpenStack-based
infrastructure layer, enabling smooth migration and upgrade
SmartCloud Orchestrator
Orchestrate Services across multiple environments and domains
Key
 Simple 3 tier structure, with increased Client Value at each tier
 Using open, common, standards based architecture
providing choice, flexibility, interoperability, portability
 Clean upgrade paths with progression to fully integrated and
factory optimized PureApplication System
 Significant customer benefits above and beyond base
OpenStack
SmartCloud Provisioning
SmartCloud Provisioning
Automate Optimized
Workloads
Automate Optimized
Workloads
SmartCloud Entry
SmartCloud Entry
Automate IT Delivery
Automate IT Delivery
Customer integrated
hardware
IBM CONFIDENTIAL
PureFlex System
Related Standards & Organizations
OSLC
CCRA
CIMI &
OVF
Common
Cloud Stack
TOSCA
Factory
Integrated
Bundle
Option
PureApp Server
Automate Optimized Workloads
PureApplication
System
© 2011 IBM Corporation
Cloud Taxonomy
11
Source: http://it20.info/2012/02/the-cloud-magic-rectangle-tm/
© 2012 IBM Corporation
Cloud Value Proposition and Positioning
Source: http://it20.info/2012/02/the-cloud-magic-rectangle-tm/
12
© 2012 IBM Corporation
How You (Provider) Build These Clouds
Source: http://it20.info/2012/02/the-cloud-magic-rectangle-tm/
13
© 2012 IBM Corporation
What You (Consumer) Get with These Clouds:
Source: http://it20.info/2012/02/the-cloud-magic-rectangle-tm/
14
© 2012 IBM Corporation
Policy-based Clouds and Design-for-fail Clouds are purpose
optimized Infrastructure Management solutions
Policy-based Clouds
•
•
•
•
•
•
15
Purpose optimized for longer-lived virtual
machines managed by Server
Administrator
Centralizes enterprise server
virtualization administration tasks
High degree of flexibility designed to
accommodate virtualization all workloads
Significant focus on managing availability
and QoS for long-lived workloads with
level of isolation
Characteristics derived from exploiting
enterprise class hardware
Legacy applications
Design-for-fail Clouds
• Purpose optimized for shorter-term virtual
machines managed via end-user or
automated process
• Decentralized control, embraces eventual
consistency, focus on making “good
enough” decisions
• High degree of standardization
• Significant focus on ensuring availability of
control plane
• Characteristics driven by software
• New applications
© 2012 IBM Corporation
Some OpenStack Public Use Cases
•
•
•
•
•
•
•
16
Internap
• http://www.internap.com/press-release/internap-announces-world%E2%80%99s-firstcommercially-available-openstack-cloud-compute-service/
Rackspace Cloud Servers, Powered by OpenStack
• http://www.rackspace.com/blog/rackspace-cloud-servers-powered-by-openstack-beta/
Deutsche Telekom
• http://www.telekom.com/media/media-kits/104982
AT&T
• http://arstechnica.com/business/news/2012/01/att-joins-openstack-as-it-launches-cloudfor-developers.ars
MercadoLibre
• http://openstack.org/user-stories/mercadolibre-inc/mercadolibre-s-bid-for-cloudautomation/
NeCTAR
• http://nectar.org.au/
San Diego Supercomputing Center
• http://openstack.org/user-stories/sdsc/
© 2012 IBM Corporation
OpenStack design tenets focus on delivering essential infrastructure
on an available, scalable, elastic control plane
Basic Design Tenets
1) Scalability and elasticity are our main goals
2) Any feature that limits our main goals must be optional
3) Everything should be asynchronous. If you can't do
something asynchronously, see #2
OpenStack Leadership's vision statement
“essential Infrastructure, support platform”
4) All required components must be horizontally scalable
5) Always use shared nothing architecture (SN) or sharding.
If you can't Share nothing/shard, see #2
6) Distribute everything. Especially logic. Move logic to
where state naturally exists.
7) Accept eventual consistency and use it where it is
appropriate.
8) Test everything. We require tests with submitted code.
(We will help you if you need it)
Sources:
http://www.openstack.org/downloads/openstack-compute-datasheet.pdf
http://wiki.openstack.org/BasicDesignTenets
17
© 2012 IBM Corporation
OpenStack
18
Source: http://ken.pepple.info/openstack/2012/09/25/openstack-folsom-architecture/
© 2012 IBM Corporation
OpenStack is comprised of seven core projects that form a complete
IaaS solution
IaaS
IaaS
Compute (Nova)
Storage (Cinder)
Network (Quantum)
Provision and manage
virtual resources
Dashboard (Horizon)
Self-service portal
Image (Glance)
Catalog and manage
server images
Identity (Keystone)
Unified authentication,
integrates with existing
systems
Object Storage (Swift)
petabytes of secure,
reliable object storage
19
Source: http://ken.pepple.info/openstack/2012/09/25/openstack-folsom-architecture/
© 2012 IBM Corporation
Compute delivers a fully featured, redundant, and scalable cloud
computing platform
Key Capabilities:
Architecture
•Manage virtualized server resources
• CPU/Memory/Disk/Network Interfaces
•API with rate limiting and authentication
•Distributed and asynchronous architecture
• Massively scalable and highly available system
•Live guest migration
• Move running guests between physical hosts
•Live VM management (Instance)
• Run, reboot, suspend, resize, terminate instances
•Security Groups
•Role Based Access Control (RBAC)
• Ensure security by user, role and project
•Projects & Quotas
•VNC Proxy through web browser
Sources:
http://ken.pepple.info/openstack/2012/09/25/openstack-folsom-architecture/
http://openstack.org/projects/compute/
20
© 2012 IBM Corporation
Compute management stack control plane is built on queue and
database
Key Capabilities:
• Responsible for providing communications hub and
managing data persistence
• RabbitMQ is default queue, MySQL DB
• Documented HA methods
• ZeroMQ implementation available to decentralize
queue
• Single “cell” (1 Queue, 1 Database) typically scales from
500 – 1000 physical machines
• Cells can be rolled up to support larger deployments
• Communications route through queue
• API requests are validated and placed on queue
• Workers listen to queues based on role or role +
hostname
• Responses are dispatched back through queue
21
© 2012 IBM Corporation
nova-compute manages individual hypervisors and compute nodes
Key Capabilities:
• Responsible for managing all interactions with individual
endpoints providing compute resource, e.g.
-- Attach iSCSI volume to phsyical host, map to guest as
additional HDD
• Implementations direct to native hypervisor APIs
– Avoids abstraction layers that bring least common
denomination support
– Enables easier exploitation of hypervisor
differentiators
• Service instance runs on every physical compute node,
helps to minimize failure domain
• Support for security groups that define firewall rules
• Support for
– KVM
– LXC
– VMware ESX/ESXi (4.1 update 1)
– Xen (XenServer 5.5, Xen Cloud Platform)
– Hyper V
22
© 2012 IBM Corporation
nova-scheduler allocates virtual resources to physical hardware
Key Capabilities:
• Determines which physical hardware to allocate to a
virtual resource
• Default scheduler uses a series of filters to reduce set of
applicable hosts and uses costing functions to provide
Weight
• Not a focus point for OpenStack
– Default implementation finds first fit
– Shorter the workload lifespan, less critical the
placement decision
• If default does not work, often deployers have specific
requirements and develop custom
23
© 2012 IBM Corporation
nova-api supports multiple API implementations and is the entry
point into the cloud
Key Capabilities:
• APIs supported
– OpenStack Compute API (REST-based)
– Similar to RackSpace APIs
– EC2 API (subset)
– Can be excluded
– Admin API (nova-manage)
• Robust extensions mechanism to add new capabilities
24
© 2012 IBM Corporation
Network automates management of networks and attachments
(network connectivity as a service)
Key Capabilities:
Architecture
25
•
Responsible for managing networks, ports, and
attachments on infrastructure for virtual resources
•
Create/delete tenant-specific L2 networks
•
L3 support (Floating IPs, DHCP, routing)
•
Moving to L4 and above in Grizzly
•
Attach / Detach host to network
•
Similar to dynamic VLAN support
•
Support for
• Open vSwitch
• OpenFlow (NEC & Floodlight controllers)
• Cisco Nexus
• Niciria
© 2012 IBM Corporation
Cinder manages block-based storage, enables persistent storage
Key Capabilities:
Architecture
26
•
Responsible for managing lifecycle of volumes and
exposing for attachment
•
Structure is a copy of Compute (Nova), sharing same
characteristics and structure in API server, scheduler,
etc.
•
Enables additional attached persistent block storage to
virtual machines
•
Support for booting virtual machines from nova-volume
backed storage
•
Allows multiple volumes to be attached per virtual
machine
•
Supports following
– ISCSI
– RADOS block devices (e.g. Ceph distributed file
system)
– Sheepdog
– Zadara
© 2012 IBM Corporation
Identity service offers unified, project-wide identity, token, service catalog,
and policy service designed to integrate with existing systems
Key Capabilities:
• Identity service provides auth credential validation and
data about Users, Tenants and Roles
• Token service validates and manages tokens used to
authenticate requests after initial credential verification
• Catalog service provides an endpoint registry used for
endpoint discovery.
• Policy service provides a rule-based authorization engine
and the associated rule management interface.
• Each service configured to serve data from pluggable
backend
– Key-Value, SQL, PAM, LDAP, PAM, Templates
• REST-based APIs
27
© 2012 IBM Corporation
Image service provides basic discovery, registration, and delivery
services for virtual disk images
Key Capabilities:
• Think Image Registry, not Image Repository
• REST-based APIs
• Query for information on public and private disk images
• Register new disk images
• Disk images can be stored in and delivered from a variety
of stores (e.g. SoNFS, Swift)
References
http://openstack.org/projects/image-service/
28
• Supported formats
– Raw
– Machine (a.k.a. AMI)
– VHD (Hyper-V)
– VDI (VirtualBox)
– qcow2 (Qemu/KVM)
– VMDK (VMWare)
– OVF (VMWare, others)
© 2012 IBM Corporation
Dashboard enables administrators and users to access and
provision cloud-based resources through a self-service portal
Key Capabilities:
• Thin wrapper over APIs, no local state
• Registration pattern for applications to hook into
• Ships with three central dashboards, a “User Dashboard”,
a “System Dashboard”, and a “Settings
• Out-of-the-box support for all core OpenStack projects
• Nova, Glace, Switch, Quantum
• Anyone can add a new component as a “first-class
citizen”.
• Follow design and style guide.
• Visual and interaction paradigms are maintained
throughout.
References
http://horizon.openstack.org/intro.html
29
• Console Access
© 2012 IBM Corporation
OpenStack Resources
•
•
•
•
•
•
•
•
•
30
Forums
• http://forums.openstack.org/
Wiki
• http://wiki.openstack.org/
Documentation
• http://docs.openstack.org/
Mailing Lists
• http://wiki.openstack.org/MailingLists
OpenStack Project Management
• https://launchpad.net/openstack
Blogs
• http://planet.openstack.org
Real-time chat room
• #openstack and #openstack-dev on irc://freenode.net (443 users currently logged in)
Rackspace Reference Architectures
• http://www.referencearchitecture.org/
Easy Install
• http://www.hastexo.com/resources/docs/installing-openstack-grizzly-20131-ubuntu-1204-precisepangolin
© 2012 IBM Corporation
IBM Resources/Solutions for OpenStack CE Available Today
•
developerWorks
• Google: openstack IBM developerworks
•
xCAT (FOSS) for 0-day deployment
• xCAT OpenStack Paper (CATStack)
• Automated qcow2 image creation for Glance
• HW control
• Bare-metal discovery and bring up
•Firmware, Base OS, etc…
•
IBM Intelligent Cluster Solutions (see Matt Ziegler's PPT)
• Preconfigured Switches
• Rack and stacked and ready to go
• Lab Services for 0-day
31
© 2012 IBM Corporation
IBM Resources/Solutions for OpenStack Available Today
•
All IBM System Software and Tools can coexist with OpenStack.
• Director, ASU, lflash, etc…
•
SoNAS for shared file (NFS, SMB)
•
XIV, v7000 for block storage (Cinder)
•
iDPX for scale-out Nova Compute and Swift
•
BNT switches for OpenFlow and Quantum
•
GPFS for iSCSI/block (Cinder) or file.
32
© 2012 IBM Corporation
OpenStack Demo Setup
Private Networks: eth0: 172.20.249/24 vm: 172.20.250/24
Control Nodes
172.20.249.10
Compute Nodes
172.20.249.11
compute
network
scheduler
volume
console
glance
api
compute
network
scheduler
volume
console
glance
api
os-essex0
os-essex1
10.0.9.10
VM
172.20.249.12
compute
network
VM
VM
10.0.9.11
172.20.249.13
172.20.249.X
compute
network
compute
network
os-essex3
os-essexX
VM
os-essex2
10.0.9.12
10.0.9.13
10.0.9.X
VM Firewall
HA Active/Passive
Scale Out
Public Networks: eth1: 10.0.9.0/25 vm: 10.0.9.128/25
33
© 2012 IBM Corporation
PPT’s and Videos: http://xmission.com/~egan/cloud/
34
© 2012 IBM Corporation