HITACHI’S
OPENSTACK
ACTIVITIES
Steve Sonnenberg, HDS
OpenSummit Hong Kong 2013
1
HITACHI: COMMITTED TO OPEN STACK

June 2013: Hitachi joined
the Red Hat OpenStack
Cloud Infrastructure
Partner Network

August 2013: Officially
announced participation
and corporate
sponsorship of
OpenStack

November 2013: Hitachi
was approved as a Gold
Member of the
OpenStack Foundation
OVERVIEW OF OPENSTACK VOLUME
ACTIVITIES

For Havana 3, HDS has submitted a Cinder
volume driver for its HUS/AMS Midrange SAN
(iSCSI)

For IceHouse, additional drivers are in
development now:
 HNAS iSCSI
 HNAS NFS
 Midrange HUS and AMS (Fibre Channel)
 Enterprise VSP and HUS-VM (Fibre Channel)
OPENSTACK SUMMIT DEMO
Remote
connection
Customer Portal
OpenStack
API
VM
VM
Hypervisor
HNAS UI
(SMU)
Enterprise
Storage Mgr
MidRange
Storage Mgr
NAS
Storage Mgr
Cinder
Keystone
Glance
Nova
Neutron
Swift
Control
Path
FC/iSCSI/
NFS
Data Path
HCP Virtual
Cluster
© Hitachi, Ltd. 2013. All rights reserved.
4
VOLUME / SNAPSHOT CATALOG
© Hitachi, Ltd. 2013. All rights reserved.
CATALOG OF BACKUPS
© Hitachi, Ltd. 2013. All rights reserved.
WHAT THIS MEANS TO OUR CUSTOMERS

HDS brings some of the most powerful storage
solutions to fortune 1000 companies

A partial list of features include:
 Virtualization / De-duplication
 Short-long range Replication
 Power-savings and efficiency
 Dynamic provisioning / tiering
 Extreme performance
 Highest density and flash performance
 …

These capabilities can now be leveraged by
OpenStack adopters
TYPICAL USE CASE

Customer is using OpenStack for DevOps
 Large number of developers collaborating
 Each using the same image+environment
 Gold image + one or more appl. Volumes
 Need efficient cloning and storage utilization

Solution
 Clone image and appl. volume using copy-onwrite to share blocks
 Dynamic provisioning doesn’t pre-allocate
storage
 Enabling our deduplication engine squeezes
out block-level duplicates that start to
accumulate
CINDER STORAGE
3 TB OF VOLUMES (Initially cloned, then salted)
DEDUPE ENGINE
OTHER OPENSTACK ACTIVITIES

OpenStack Cloud Management Portal
 Web-based management UI
 Complex image/template management
 Task monitoring
 Shows operational progress
 Shows breakdown of sub-steps
COMPLEX TEMPLATES

A VM environment consists of
 OS image and environment
 Initially retrieved through glance
 Managed by hypervisor/libvirt using storage
available to the compute server
 Application/Data volumes
 Not managed along with the ‘VM’
 Ex. snapshot: de-attach, snap, re-attach

Using APIs its not difficult to manage a VM and
any/all of its volumes, uses include:
 Snapshot of a whole VM
 Backup of a whole VM
 Launch of a complex VM
COMPLEX TEMPLATES
Operations can be performed on all volumes associated with a
running VM
© Hitachi, Ltd. 2013. All rights reserved.
BUILDING A TEMPLATE
Template includes volumes to attach when creating the VM
© Hitachi, Ltd. 2013. All rights reserved.
LAUNCHED
VM launch and volume attachment happen as a single operation
© Hitachi, Ltd. 2013. All rights reserved.
OTHER OPENSTACK ACTIVITIES

Swift Object Server to HCP (Hitachi Content
Platform) Gateway
 Planned IceHouse submission
 Leverages HDS world class object store
 Integrated Hadoop-based metadata search
TYPICAL SWIFT ARCHITECTURE
OPENSTACK
Backup/image
EASILY SCALE FOR PERFORMANCE OR
CAPACITY
Proxy, Account, Container aren’t modified
Object server doesn’t manage local storage
Proxy Server
Account Server
Container Server
Object Server
S3-compatible
HCP
Start with 4 Nodes and add nodes for increased capacity, object count, I/O or CPU
SAN
SAN
Add storage for increased I/O or
capacity (up to 80 petabytes)
64 billion
objects
No size
limitations
WHY USE HCP AS A SWIFT BACKEND?

HCP is a 6th generation object store

A partial list of features include:
 High Availability
 Data Protection, Metadata Protection,
Index Protection, Multipath
 Proven scalability
 Full Object Life Cycle services
 Policy management
 Built-in Query Engine
 Autonomic Tech Refresh
 …

Rather than use commodity servers and JBOD
 HCP builds on reliable/redundant components
STRUCTURE OF AN OBJECT STORE

Logical Representation
 A container for logically related objects
 Possibly a directory structure (or
hierarchical naming)
 Objects (data or metadata)
 Metadata describing objects

Physical Representation
 Typically data in stored in filesystems
across multiple object servers
 HDS modified the object server to store
objects using S3
 HCP is a multi-protocol object store
 S3, NFS, CIFS, REST, SMTP
A SWIFT EXAMPLE
SWIFT ORGANIZES OBJECTS IN CONTAINERS
• volumebackups: used to house the collection of volume backups
• 4 volumes have been backed up (shown above)
• The object names are hierarchical and consist of multiple objects
• hcp: used for demonstration
OBJECT DETAILS
METADATA IS CARRIED USING HEADERS
 Application data stored as an object (below)
HCP PREPARATION
CREATED A ‘TENANT’ WITH A ‘SWIFT’ NAMESPACE
S3 DATA IN HCP
HCP HAS A GLOBAL HIERARCHICAL FILESYSTEM
system-metadata
custom-metadata
LEVERAGING METADATA
 Used Cloudberry to add a ‘foo’ header
 HCP custom-metadata contains ‘foo’ (below)
HCP SEARCH
 We can perform a variety of queries including leveraging
a search-profile to enable field-level recognition
THANK YOU
28