All you need to know about OpenStack Block Storage in less than an hour Dhruv Bhatnagar Nirendra Awasthi Agenda 2 1 What is Cinder 2 Architecture & Services 3 Concepts, Configuration and CLI 4 Service Interactions 5 Advanced Features 6 Cinder Drivers 7 Veritas HyperScale and Cinder 8 Q&A Copyright © 2017 Veritas Technologies. OpenStack Architecture 3 Copyright © 2017 Veritas Technologies. Different types of Storage in OpenStack Ephemeral • Non-persistent • Life Cycle coincides with an instance • Usually local FS/QCOW file Object • Typically ”cheap and deep” • Commonly SWIFT • Use cases: photos, mp4 etc 4 Copyright © 2017 Veritas Technologies. Shared FS • Example: NFS • Manilla Block • • • • Foundation for the other types Think raw disk Typically higher performance Cinder What is Cinder “…. short description of Cinder is that it virtualizes the management of block storage devices and provides end users with a self service API to request and consume those resources without requiring any knowledge of where their storage is actually deployed or on what type of device…” 5 Copyright © 2017 Veritas Technologies. What is Cinder used for…. • Enables user to manage their storage – Create volume – Create Snapshot – Backup – Attach/Detach • Cinder provides – commands and – API’s to interact with vendors’ storage backend – persistent storage to VM’s • Exposes vendors storage hardware to the cloud 6 Copyright © 2017 Veritas Technologies. Agenda 7 1 What is Cinder 2 Architecture & Services 3 Concepts, Configuration and CLI 4 Service Interactions 5 Advanced Features 6 Cinder Drivers 7 Veritas HyperScale and Cinder 8 Q&A Copyright © 2017 Veritas Technologies. Rest API Cinder Architecture Cinder Scheduler Cinder Backup Cinder API Messaging Bus (AMQP) Cinder Volume (Backend) 8 Copyright © 2017 Veritas Technologies. Cinder Volume (Backend) Cinder Volume (Backend) Agenda 9 1 What is Cinder 2 Architecture & Services 3 Concepts, Configuration and CLI 4 Service Interactions 5 Advanced Features 6 Cinder Drivers 7 Veritas HyperScale and Cinder 8 Q&A Copyright © 2017 Veritas Technologies. CLI Commands • List Cinder services – openstack volume service list 10 Copyright © 2017 Veritas Technologies. Add Cinder Backend (cinder.conf) • Enable Multi Backend Support by adding different sections for each cinder driver • “enabled_backends” parameter used to set which backends to enable • Any changes to cinder configuration needs cinder services to be restarted afterwards 11 Copyright © 2017 Veritas Technologies. CLI Commands • List Volume types – openstack volume type list 12 Copyright © 2017 Veritas Technologies. Volume Types • Created by Admin • After creating a new Volume Type – Update “volume_backend_name” key to hold the value of the Backend Name – Create a volume out of this new volume type to enable Vendor provided backend features 13 Copyright © 2017 Veritas Technologies. CLI Commands • Show Details of a volume type – openstack volume type show <Volume-Type> • List Volumes – openstack volume list 14 Copyright © 2017 Veritas Technologies. Volume status Status Description creating available reserved attaching detaching in-use maintenance deleting awaiting-transfer error error_deleting backing-up restoring-backup error_backing-up The volume is being created. The volume is ready to attach to an instance. The volume is reserved for attaching or shelved. The volume is attaching to an instance. The volume is detaching from an instance. The volume is attached to an instance. The volume is locked and being migrated. The volume is being deleted. The volume is awaiting for transfer. A volume creation error occurred. A volume deletion error occurred. The volume is being backed up. A backup is being restored to the volume. A backup error occurred. error_restoring A backup restoration error occurred. error_extending An error occurred while attempting to extend a volume. downloading uploading retyping extending The volume is downloading an image. The volume is being uploaded to an image. The volume is changing type to another volume type. The volume is being extended. 15 Copyright © 2017 Veritas Technologies. CLI Commands • Create Volumes – openstack volume create test-vol-1 --size 50 • openstack help volume create • Attach Cinder volume to a nova instance – openstack server add volume <instance-id> <vol-id> 16 Copyright © 2017 Veritas Technologies. Agenda 17 1 What is Cinder 2 Architecture & Services 3 Concepts, Configuration and CLI 4 Service Interactions 5 Advanced Features 6 Cinder Drivers 7 Veritas HyperScale and Cinder 8 Q&A Copyright © 2017 Veritas Technologies. Cinder Interactions (Creating a Volume) Cinder Client Cinder Scheduler Cinder API Messaging Bus (AMQP) Cinder Volume (HyperScale) 18 Copyright © 2017 Veritas Technologies. Cinder Volume (HyperScale) Cinder Volume (HyperScale) Cinder Interactions (Attach a volume) iSCSI is just an example. Several other protocols are supported as well Nova VM Cinder VM /dev/vda Storage Controller KVM iSCSI Initiator 19 Copyright © 2017 Veritas Technologies. iSCSI Target Volume API https://developer.openstack.org/api-ref/block-storage 20 Copyright © 2017 Veritas Technologies. Agenda 21 1 What is Cinder 2 Architecture & Services 3 Concepts, Configuration and CLI 4 Service Interactions 5 Advanced Features 6 Cinder Drivers 7 Veritas HyperScale and Cinder 8 Q&A Copyright © 2017 Veritas Technologies. Advanced features • Snapshot – A snapshot is a point-in-time copy of the data that a volume contains – A snapshot would live on the same storage backend as the active volume • Quota – Admins set this limit on volume, backup and snapshot capacity depending on policy settings • Volume transfer – Transfer a volume from one user to another user • Encryption – Encryption is done by Nova using dm-crypt which is a transparent disk encryption subsystem in Linux kernel • Backup – Full and incremental backups are supported – Supported providers are Swift, Ceph, NFS, GlusterFS, POSIX, Google Cloud and IBM TSM 22 Copyright © 2017 Veritas Technologies. Volume Migration • Move data between two backends with same volume type • Two backends can be located on same or different cinder nodes # cinder migrate [--force-host-copy [<True|False>]] [--lock-volume [<True|False>]] <volume> <host> 23 Copyright © 2017 Veritas Technologies. Volume Retype • Move between two backends with different volume types • Two backends can be located on same or different Cinder Volume node # cinder retype [--migration-policy <never|on-demand>] <volume> <volume-type> 24 Copyright © 2017 Veritas Technologies. Volume Groups • Concept introduced in Newton • Different volumes used by the same applications / workloads to be grouped and managed together • Operations supported – Create, delete, update, show and list groups 25 Copyright © 2017 Veritas Technologies. Glance Store and Create a VM 26 Copyright © 2017 Veritas Technologies. Glance Image Volume Cache • When Image is downloaded, a volume cache is created • Every create boot volume call will clone a volume instead of downloading the image again • Configuration: – image_volume_cache_enabled = True – image_volume_cache_max_size = SIZE_IN_GB – image_volume_cache_max_count = MAX_COUNT – And then Launch VMs via “boot from volume” option 27 Copyright © 2017 Veritas Technologies. Using Cinder as Backing Store for Glance • Backend should support fast cloning capabilities • Every “Create Volume from Image” call checks for availability of image in backend • If image is not present, it is downloaded and converted to raw • If image is present, image is cloned to new volume – Performance is improved 30 times when compared to downloading the same image 2nd time – Backends supporting de-duplication will perform even better 28 Copyright © 2017 Veritas Technologies. Agenda 29 1 What is Cinder 2 Architecture & Services 3 Concepts, Configuration and CLI 4 Service Interactions 5 Advanced Features 6 Cinder Drivers 7 Veritas HyperScale and Cinder 8 Q&A Copyright © 2017 Veritas Technologies. Cinder Drivers • What do they do? – Map requests from Cinder to external Storage Platform • LVM • Various other Storage Controllers • Distributed File Systems • Currently about 100 drivers in github – https://github.com/openstack/cinder/tree/master/cinder/volume/drivers – https://wiki.openstack.org/wiki/CinderSupportMatrix 30 Copyright © 2017 Veritas Technologies. Cinder Drivers ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● 31 Block Device Driver (local) Blockbridge (iSCSI) CloudByte (iSCSI) Coho (NFS) Datera (iSCSI) Dell Equallogic (iSCSI) Dell Storage Center (iSCSI/FC) Disco (disco) DotHill (iSCSI/FC) DRBD (DRBD/iSCSI) EMC VMAX (iSCSI/FC) EMC VNX (iSCSI/FC) EMC XtremIO (iSCSI/FC) EMC ScaleIO (scaleio) Fujitsu ETERNUS (iSCSI/FC) GlusterFS (GlusterFS) HGST (NFS) HPE 3PAR (iSCSI/FC) HPE LeftHand (iSCSI) HPE MSA (iSCSI/FC) Copyright © 2017 Veritas Technologies. ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● HPE XP (FC) Hitachi HBSD (iSCSI/FC) Hitachi HNAS (iSCSI/NFS) Huawei (iSCSI/FC) IBM DS8000 (FC) IBM Flashsystem (iSCSI/FC) IBM GPFS (GPFS) IBM Storwize SVC (iSCSI/FC) IBM XIV (iSCSI/FC) Infortrend (iSCSI/FC) Lenovo (iSCSI/FC) LVM (iSCSI) - Reference NetApp ONTAP (iSCSI/NFS/FC) NetApp E Series (iSCSI/FC) Nexenta (iSCSI/NFS) NFS (NFS) – Reference Nimble Storage (iSCSI) ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● Oracle Zfssa (iSCSI/NFS) Pure Storage (iSCSI/FC) ProphetStor (iSCSI/FC) Quobyte (quobyte) RBD (Ceph) - Reference Scality SOFS (scality) Sheepdog (sheepdog) SMBFS (SMB) SolidFire (iSCSI) Tegile (iSCSI/FC) Tintri (NFS) Violin (FC) VMware (VMDK) Virtuozzo Storage (NFS) Windows (SMB) X-IO technologies (iSCSI/FC) Veritas HyperScale Veritas Access Minimum driver features • Drivers must implement support for the core features: – Volume Create/Delete – Volume Attach/Detach – Snapshot Create/Delete – Create Volume from Snapshot – Copy Image to Volume – Copy Volume to Image – Clone Volume – Extend Volume 32 Copyright © 2017 Veritas Technologies. Commodity Storage Drawbacks / Limitations • HA • Scale • QoS • Data Dedup • Efficient Tiering 33 Copyright © 2017 Veritas Technologies. Vendor specific features • Different vendor backend enable different use cases • Backend are selected by cinder scheduler • Every backend is created while thinking of specific set of capabilities • The vendor specific features are thus exposed via – Custom volume types – Extensions 34 Copyright © 2017 Veritas Technologies. Provides ability to choose • Plugin architecture, multiple choices for backend storage • Don’t have to choose just one • Use the storage that fits your needs • Mix of proprietary and open-source choices 35 Copyright © 2017 Veritas Technologies. Agenda 36 1 What is Cinder 2 Architecture & Services 3 Concepts, Configuration and CLI 4 Service Interactions 5 Advanced Features 6 Cinder Drivers 7 Veritas HyperScale and Cinder 8 Q&A Copyright © 2017 Veritas Technologies. Integrating Storage Management with OpenStack 37 Copyright © 2017 Veritas Technologies. HyperScale: OpenStack Integration Compute Volume extension Scheduler Collector Volume Dashboard 41 Copyright © 2017 Veritas Technologies. HyperScale and Cinder • High Availability with DAS • Automated snapshot management • Predictable Performance • Hyper Scale • Zero Backup Window • Real Time Storage Insight • Storage Cost Saving 39 Copyright © 2017 Veritas Technologies. Key Takeaways • Manage Block Storage via self service APIs • Cinder provides CLI to create and manage OpenStack volumes • Cinder backends can be configured through cinder.conf config file • VM spawn time can be optimized using either image volume cache or cinder as glance store • Cinder volume drivers are pluggable interfaces to map Cinder requests to storage platform 40 Copyright © 2017 Veritas Technologies. Agenda 41 1 What is Cinder 2 Architecture & Services 3 Concepts, Configuration and CLI 4 Service Interactions 5 Advanced Features 6 Cinder Drivers 7 Veritas HyperScale and Cinder 8 Q&A Copyright © 2017 Veritas Technologies. Thank you! Nirendra Awasthi Dhruv Bhatnagar Copyright © 2017 Veritas Technologies. All rights reserved. Veritas and the Veritas Logo are trademarks or registered trademarks of Veritas Technologies or its affiliates in the U.S. and other countries. Other names may be trademarks of their respective owners. This document is provided for informational purposes only and is not intended as advertising. All warranties relating to the information in this document, either express or implied, are disclaimed to the maximum extent allowed by law. The information in this document is subject to change without notice.
