HITACHI’S OPENSTACK ACTIVITIES Steve Sonnenberg, HDS OpenSummit Hong Kong 2013 1 HITACHI: COMMITTED TO OPEN STACK June 2013: Hitachi joined the Red Hat OpenStack Cloud Infrastructure Partner Network August 2013: Officially announced participation and corporate sponsorship of OpenStack November 2013: Hitachi was approved as a Gold Member of the OpenStack Foundation OVERVIEW OF OPENSTACK VOLUME ACTIVITIES For Havana 3, HDS has submitted a Cinder volume driver for its HUS/AMS Midrange SAN (iSCSI) For IceHouse, additional drivers are in development now: HNAS iSCSI HNAS NFS Midrange HUS and AMS (Fibre Channel) Enterprise VSP and HUS-VM (Fibre Channel) OPENSTACK SUMMIT DEMO Remote connection Customer Portal OpenStack API VM VM Hypervisor HNAS UI (SMU) Enterprise Storage Mgr MidRange Storage Mgr NAS Storage Mgr Cinder Keystone Glance Nova Neutron Swift Control Path FC/iSCSI/ NFS Data Path HCP Virtual Cluster © Hitachi, Ltd. 2013. All rights reserved. 4 VOLUME / SNAPSHOT CATALOG © Hitachi, Ltd. 2013. All rights reserved. CATALOG OF BACKUPS © Hitachi, Ltd. 2013. All rights reserved. WHAT THIS MEANS TO OUR CUSTOMERS HDS brings some of the most powerful storage solutions to fortune 1000 companies A partial list of features include: Virtualization / De-duplication Short-long range Replication Power-savings and efficiency Dynamic provisioning / tiering Extreme performance Highest density and flash performance … These capabilities can now be leveraged by OpenStack adopters TYPICAL USE CASE Customer is using OpenStack for DevOps Large number of developers collaborating Each using the same image+environment Gold image + one or more appl. Volumes Need efficient cloning and storage utilization Solution Clone image and appl. volume using copy-onwrite to share blocks Dynamic provisioning doesn’t pre-allocate storage Enabling our deduplication engine squeezes out block-level duplicates that start to accumulate CINDER STORAGE 3 TB OF VOLUMES (Initially cloned, then salted) DEDUPE ENGINE OTHER OPENSTACK ACTIVITIES OpenStack Cloud Management Portal Web-based management UI Complex image/template management Task monitoring Shows operational progress Shows breakdown of sub-steps COMPLEX TEMPLATES A VM environment consists of OS image and environment Initially retrieved through glance Managed by hypervisor/libvirt using storage available to the compute server Application/Data volumes Not managed along with the ‘VM’ Ex. snapshot: de-attach, snap, re-attach Using APIs its not difficult to manage a VM and any/all of its volumes, uses include: Snapshot of a whole VM Backup of a whole VM Launch of a complex VM COMPLEX TEMPLATES Operations can be performed on all volumes associated with a running VM © Hitachi, Ltd. 2013. All rights reserved. BUILDING A TEMPLATE Template includes volumes to attach when creating the VM © Hitachi, Ltd. 2013. All rights reserved. LAUNCHED VM launch and volume attachment happen as a single operation © Hitachi, Ltd. 2013. All rights reserved. OTHER OPENSTACK ACTIVITIES Swift Object Server to HCP (Hitachi Content Platform) Gateway Planned IceHouse submission Leverages HDS world class object store Integrated Hadoop-based metadata search TYPICAL SWIFT ARCHITECTURE OPENSTACK Backup/image EASILY SCALE FOR PERFORMANCE OR CAPACITY Proxy, Account, Container aren’t modified Object server doesn’t manage local storage Proxy Server Account Server Container Server Object Server S3-compatible HCP Start with 4 Nodes and add nodes for increased capacity, object count, I/O or CPU SAN SAN Add storage for increased I/O or capacity (up to 80 petabytes) 64 billion objects No size limitations WHY USE HCP AS A SWIFT BACKEND? HCP is a 6th generation object store A partial list of features include: High Availability Data Protection, Metadata Protection, Index Protection, Multipath Proven scalability Full Object Life Cycle services Policy management Built-in Query Engine Autonomic Tech Refresh … Rather than use commodity servers and JBOD HCP builds on reliable/redundant components STRUCTURE OF AN OBJECT STORE Logical Representation A container for logically related objects Possibly a directory structure (or hierarchical naming) Objects (data or metadata) Metadata describing objects Physical Representation Typically data in stored in filesystems across multiple object servers HDS modified the object server to store objects using S3 HCP is a multi-protocol object store S3, NFS, CIFS, REST, SMTP A SWIFT EXAMPLE SWIFT ORGANIZES OBJECTS IN CONTAINERS • volumebackups: used to house the collection of volume backups • 4 volumes have been backed up (shown above) • The object names are hierarchical and consist of multiple objects • hcp: used for demonstration OBJECT DETAILS METADATA IS CARRIED USING HEADERS Application data stored as an object (below) HCP PREPARATION CREATED A ‘TENANT’ WITH A ‘SWIFT’ NAMESPACE S3 DATA IN HCP HCP HAS A GLOBAL HIERARCHICAL FILESYSTEM system-metadata custom-metadata LEVERAGING METADATA Used Cloudberry to add a ‘foo’ header HCP custom-metadata contains ‘foo’ (below) HCP SEARCH We can perform a variety of queries including leveraging a search-profile to enable field-level recognition THANK YOU 28