STUDY ABOUT OPENSTACK Documents explains about Openstack and its components Jai Krishna Pyneni jkp020@latech.edu About OpenStack OpenStack is a cloud operating system that controls large pools of compute, storage, and networking resources throughout a datacenter, through a dashboard that gives administrators control while empowering their users with required resources through a web interface. Openstack aims to provide open source cloud computing platform for public and private clouds everywhere. Predominantly acting as an infrastructure as a service (IaaS) platform, it is free and open-source software released under the terms of the Apache License Different Components of Openstack: 1. Compute (Nova) 2. Object Storage (Swift) 3. Block Storage (Cinder) 4. Networking (Neutron) 5. Dashboard (Horizon) 6. Identity Service (Keystone) 7. Image Service (Glance) 8. Telemetry (Ceilometer) 9. Orchestration (Heat) Compute (Nova): The OpenStack cloud operating system enables service providers to offer on-demand computing resources, by provisioning and managing large networks of virtual machines. Resources for compute are accessible via APIs for developers building cloud applications and via web interfaces for administrators and users. It is written in Python and uses many external libraries such as Eventlet (for concurrent programming), Kombu (for Advanced Message Queuing Protocol communication), and SQLAlchemy (for database access). OpenStack’s Compute is architected to provide flexibility as you design your cloud. The compute architecture is designed to scale horizontally on standard hardware with no proprietary hardware or software requirements and provide the ability to integrate with legacy systems and third-party technologies. It is designed to manage and automate pools of compute resources and can work with widely available virtualization technologies, as well as bare metal and high-performance computing (HPC) configurations. Features & Benefits of Compute Feature Manage virtualized commodity server resources CPU, memory, disk, and network interfaces Manage Local Area Networks (LAN) Flat, Flat DHCP, VLAN DHCP, IPv6 API with rate limiting and authentication Distributed and asynchronous architecture Virtual Machine (VM) image management Live VM management Run, reboot, suspend, resize, terminate instances Floating IP addresses Benefit Racks of commodity servers as pools of computing Improved utilization and automation of resources for greater cost efficiencies Programmatically allocate IPs and VLANs (for rapid provisioning of network capabilities and security features) Flexible networking models to suit needs of each application and/or user group Designed for automation and security (to make it easy for you to manage who has access to compute resources and prevent users from impacting each other with excessive API utilization) Massively scalable and highly available system (for increased assurance of system uptime) Easily store, import, share, and query images (to make it easy for you to spin up new standardized VMs) Increase productivity with lifecycle management (from a single user interface and using the APIs) Ability to assign (and re-assign) IP addresses to VMs Security Groups Flexibility to assign and control access to VM instances by creating separation between resource pools Role Based Access Control (RBAC) Ensure security by user, role and project Projects & Quotas Ability to allocate, track and limit resource utilization VNC Proxy through web browser Quick and easy CLI administration Store and Manage files programmatically via API Least privileged access design Automates resource management/provisioning Dashboard with fully integrated support for self-service provisioning VM Image Caching on compute nodes Learn more about the OpenStack Dashboard Root access separation for management & services Faster provisioning of VMs OpenStack Storage: Openstack uses two types of storages Object and Block storage for use with servers and applications Object Storage (Swift): Object Storage is a cost effective, scale-out storage. It provides a fully distributed, API-accessible storage platform that can be integrated directly into applications or used for backup, archiving and data retention. It's built for scale and optimized for durability, availability, and concurrency across the entire data set. Swift is ideal for storing unstructured data that can grow without bound. Object Storage Capabilities 1. OpenStack provides redundant, scalable object storage using clusters of standardized servers capable of storing petabytes of data 2. Object Storage is not a traditional file system, but rather a distributed storage system for static data such as virtual machine images, photo storage, email storage, backups and archives. Having no central "brain" or master point of control provides greater scalability, redundancy and durability. 3. Objects and files are written to multiple disk drives spread throughout servers in the data center, with the OpenStack software responsible for ensuring data replication and integrity across the cluster. 4. Storage clusters scale horizontally simply by adding new servers. Should a server or hard drive fail, OpenStack replicates its content from other active nodes to new locations in the cluster. Because OpenStack uses software logic to ensure data replication and distribution across different devices, inexpensive commodity hard drives and servers can be used in lieu of more expensive equipment. Block Storage (Cinder): Block Storage allows block of storages to be exposed and connected to compute instances for expanded storage, better performance and integration with enterprise storage platforms, such as NetApp, Nexenta and SolidFire. Cinder provides an infrastructure for managing volumes in OpenStack. It was originally a Nova component called nova-volume, but has become an independent project since the Folsom release. Block Storage Capabilities 1. OpenStack provides persistent block level storage devices for use with OpenStack compute instances. 2. The block storage system manages the creation, attaching and detaching of the block devices to servers. Block storage volumes are fully integrated into OpenStack Compute and the Dashboard allowing for cloud users to manage their own storage needs. 3. In addition to using simple Linux server storage, it has unified storage support for numerous storage platforms including Ceph, NetApp, Nexenta, SolidFire, and Zadara. 4. Block storage is appropriate for performance sensitive scenarios such as database storage, expandable file systems, or providing a server with access to raw block level storage. 5. Snapshot management provides powerful functionality for backing up data stored on block storage volumes. Snapshots can be restored or used to create a new block storage volume. Networking(Neuron): Neutron is an OpenStack project to provide "networking as a service" between interface devices (e.g., vNICs) managed by other Openstack services. OpenStack Networking ensures that network will not be the bottleneck or limiting factor in a cloud deployment and gives users real self-service, even over their network configurations. Neuron is a pluggable, scalable and API-driven system for managing networks and IP addresses. Networking Capabilities 1. OpenStack provides flexible networking models to suit the needs of different applications or user groups. Standard models include flat networks or VLANs for separation of servers and traffic. 2. OpenStack Networking manages IP addresses, allowing for dedicated static IPs or DHCP. Floating IPs allow traffic to be dynamically rerouted to any of your compute resources, which allows you to redirect traffic during maintenance or in the case of failure. 3. Users can create their own networks, control traffic and connect servers and devices to one or more networks. 4. The pluggable backend architecture lets users take advantage of commodity gear or advanced networking services from supported vendors. 5. Administrators can take advantage of software-defined networking (SDN) technology like OpenFlow to allow for high levels of multi-tenancy and massive scale. 6. OpenStack Networking has an extension framework allowing additional network services, such as intrusion detection systems (IDS), load balancing, firewalls and virtual private networks (VPN) to be deployed and managed. Dashboard (Horizon): Horizon is the canonical implementation of Openstack’s Dashboard, which provides a web based user interface to OpenStack services including Nova, Swift, Keystone, etc. The OpenStack dashboard provides administrators and users a graphical interface to access, provision and automate cloud-based resources. It is easy to plug in and expose third party products and services, such as billing, monitoring and additional management tools. Developers can automate access or build tools to manage their resources using the native OpenStack API. Dashboard Capabilities 1. The dashboard is an extensible web app that allows cloud administrators and users to control their compute, storage and networking resources. 2. As a cloud administrator, the dashboard provides an overall view of the size and state of your cloud. You can create users and projects, assign users to projects and set limits on the resources for those projects. 3. The dashboard provides users a self-service portal to provision their own resources within the limits set by administrators. OpenStack has several shared services those cover the three pillars of compute, storage and networking, making it easier to implement and operate your cloud. They are 1. 2. 3. 4. Identity Service (Keystone) Image Service (Glance) Telemetry Service (Ceilometer) Orchestration Service (Heat) Identity Service (Keystone): Keystone is the identity service used by OpenStack for authentication (authN) and high-level authorization (authZ). It currently supports token-based authN and user-service authorization. OpenStack Identity provides a central directory of users mapped to the OpenStack services they can access. It acts as a common authentication system across the cloud operating system and can integrate with existing backend directory services like LDAP (Lightweight Directory Access Protocol). . Users and third-party tools can programmatically determine which resources they can access. As an administrator, OpenStack Identity enables you to: 1. Configure centralized policies across users and systems. 2. Create users and tenants and define permissions for compute, storage and networking resources using role-based access control (RBAC) features. 3. Integrate with an existing directory like LDAP, allowing for a single source of identity authentication across the enterprise. As a user, OpenStack Identity enables you to: 1. Get a list of the services that you can access. 2. Make API requests or log into the web dashboard to create resources owned by your account. Image Service (Glance): The OpenStack Image Service provides discovery, registration and delivery services for disk and server images. The ability to copy or snapshot a server image and immediately store it away is a powerful capability of the OpenStack cloud operating system. The Image Service can store disk and server images in a variety of back-ends, including OpenStack Object Storage. A number of periodic processes run on the Image Service to support caching. Replication services ensures consistency and availability through the cluster. Other periodic processes include auditors, updaters, and reapers. Capabilities of the Image Service include: 1. Administrators can create base templates from which their users can start new compute instances 2. Users can choose from available images, or create their own from existing servers 3. Snapshots can also be stored in the Image Service so that virtual machines can be backed up quickly Telemetry Service (Ceilometer): The OpenStack Telemetry service aggregates usage and performance data across the services deployed in an OpenStack cloud. This powerful capability provides visibility and insight into the usage of the cloud across dozens of data points and allows cloud operators to view metrics globally or by individual deployed resources. The ceilometer aims to deliver a unique point of contact for billing systems to acquire all form of the measurements that a provider need for customer billing, across all current OpenStack core components Ceilometer aims to achieve the following 1. Provide efficient collection of metering data, in terms of CPU and network costs. 2. Allow deployers to integrate with the metering system directly or by replacing components. 3. Data may be collected by monitoring notifications sent from existing services or by polling the infrastructure. 4. Allow deployers to configure the type of data collected to meet their operating requirements. 5. The data collected by the metering system is made visible to some users through a REST API. Orchestration Service (Heat): OpenStack Orchestration is a template-driven engine that allows application developers to describe and automate the deployment of infrastructure. The flexible template language can specify compute, storage and networking configurations as well as detailed post-deployment activity to automate the full provisioning of infrastructure for services and applications. In Simple, OpenStack Heat program is to create a human- and machineaccessible service for managing the entire lifecycle of infrastructure and applications within OpenStack clouds. Apart from above components Openstack is developing to implement some other components like OpenStack Common Libraries (Oslo): Oslo is to bring together the python libraries containing code shared by OpenStack projects. The APIs provided by these libraries should be high quality, stable, consistent, documented and generally applicable. OpenStack Open Source Database (Trove): Trove is Database as a Service for OpenStack. It's designed to run entirely on OpenStack, with the goal of allowing users to quickly and easily utilize the features of a relational database without the burden of handling complex administrative tasks. Cloud users and database administrators can provision and manage multiple database instances as needed. Initially, the service will focus on providing resource isolation at high performance while automating complex administrative tasks including deployment, configuration, patching, backups, restores, and monitoring. Release History of different components of Openstack Release Release date Included Component code names name Austin 21 October 2010 Nova, Swift Bexar 3 February 2011 Nova, Glance, Swift Cactus 15 April 2011 Nova, Glance, Swift Diablo 22 September 2011 Nova, Glance, Swift Essex 5 April 2012 Nova, Glance, Swift, Horizon, Keystone Folsom 27 September 2012 Nova, Glance, Swift, Horizon, Keystone, Quantum, Cinder Grizzly 4 April 2013 Nova, Glance, Swift, Horizon, Keystone, Quantum, Cinder Havana 17 October 2013 Nova, Glance, Swift, Horizon, Keystone, Neutron, Cinder, Heat, Ceilometer Icehouse 17 April 2014 Nova, Glance, Swift, Horizon, Keystone, Neutron, Cinder, Heat, Ceilometer, Trove Architecture of Openstack Amazon Web Services compatibility OpenStack APIs are compatible with Amazon EC2 and Amazon S3 and thus client applications written for Amazon Web Services can be used with OpenStack with minimal porting effort. Use Cases of Openstack: 1. 2. 3. 4. 5. Service providers offering an IaaS compute platform or services higher up the stack IT departments acting as cloud service providers for business units and project teams Processing big data with tools like Hadoop Scaling compute up and down to meet demand for web resources and applications High-performance computing (HPC) environments processing diverse and intensive workloads Users: Different users of OpenStack are AT&T, HP cloud services, PayPal, Rackspace, Sony online gaming systems, Yahoo, Wikimedia labs, Intel, NASA etc... . Conceptual Architecture: Launching a virtual machine or instance involves many interactions among several services. The following diagram provides the conceptual architecture of a typical OpenStack environment. Refernces: http://docs.openstack.org/, https://wiki.openstack.org/wiki/Main_Page, http://en.wikipedia.org/wiki/OpenStack,