CSE 598D Storage Systems, Spring 2007
Object Based Storage
Presented By: Kanishk Jain
Introduction

Object Based Storage



ANSI T10 Object-based Storage Devices
Standard
storage object: a logical collection of bytes
on a storage device, with well-known
methods for access, attributes describing
characteristics of the data, and security
policies that prevent unauthorized access.
“intelligent data layout”
Object Storage Interface

OSD model is simply a rearrangement of existing data management
functions

OSD is a level higher than block access but one level below file access
Background – NAS sharing



NAS being used to
share files among a
number of clients
The files themselves
may be stored on a
fast SAN
The file server is
used to intermediate
all requests and thus
becomes the
bottleneck !
Background – SAN sharing


The files themselves
are stored on a fast
SAN (e.g., iSCSI) to
which the clients are
also attached
While the file server
is removed as a
bottleneck, security
is a concern !
Object-based storage security
architecture


Metadata managers
grant capabilities to
clients; clients
present these
capabilities to the
devices on every I/O
to ensure security
Secure separation of
control and data
path !
Development of OSD

Most initial work on object storage
devices (OSD) was done at Parallel Data
Lab at CMU



Focused on developing underlying concepts
in two closely related areas: NASD and
Active Disks
Proposed as part of same project as NASD
Standardized by Storage Networking
Industry Association (SNIA) in 2004.
OSD v/s Active Disks




OSD standard only talks about the interface.
It does not assume anything about the
processing power at the disk.
OSD intelligence is software/firmware running
at the disk (no specifications for this)
Processing power of an OSD can be scaled to
meet the requirements of the functions an
active disk
File System – Application side
(User Component only)


The OSD has the intelligence to perform basic
data management functions such as space
allocation, free space management etc., those
functions are no longer part of the
application-side file system.
Thus the application side file system is
reduced to a manager : an abstraction layer
between user application and the OSD.

Only provides security and backward compatibility
File System - On the Device
(Storage Component)



Workload offered to OSDs may be quite
different from that of general-purpose file
systems
At the OSD level, objects typically have no
logical relationship, presenting a flat name
space
General-purpose file systems, which are
usually optimized for workloads exhibiting
relatively small variable-sized files, relatively
small hierarchical directories, and some
degree of locality are not effective in this case
Object based File System



Separation of metadata and data paths:
Separate metadata servers (MDS)
manage the directory hierarchy,
permissions and file to object mapping.
Distribution and replication of a file
across a sequence of objects on many
OSDs.
Example files systems: Lustre, Panasas,
Ceph
Some Optimizations in Ceph



Partitioning the directory tree: To efficiently balance load, the MDS
partition the directory tree across the cluster. A client guesses which
metadata server is responsible for a file, and contacts that server to
open the file. That MDS will forward the request to the correct MDS if
necessary. Distribution and replication of a file across a sequence of
objects on many OSDs.
Limit on object size and use of regions: Ceph limits objects to a
maximum size (e. g., 1MB), so files are a sequence of bytes broken into
chunks on the maximum object size boundary. Since only the MDS hold
the directory tree, OSDs do not have directory information to suggest
layout hints for file data. Instead, the OSDs organize objects into small
and large object regions, using small block sizes (e. g., 4KB or 8KB) for
small objects and large block sizes (e. g. 50–100% of the maximum
object size) for large objects.
Use of a specialized mapping algorithm: A file handle returned by
the metadata server describes which objects on which OSD contain the
file data. A special algorithm, RUSH maps a sequence index to the OSD
holding the object at that position in the sequence, distributing the
objects in a uniform way.
Possible Performance Results

OBFS outperforms Linux Ext2 and Ext3 by a factor of
two or three, and while OBFS is 1/25 the size of XFS,
it provides only slightly lower read performance and
10%–40% higher write performance
Possible Performance Results
(contd..)
Database Storage Management




Object attributes are also the key to giving storage
devices an awareness of how objects are being
accessed, so that it can use this information to
optimize disk layout specific to the application.
Database software often has very little detailed
information about the storage subsystem
Previous research took the view that a storage device
can provide relevant characteristics to applications
Device-specific information is known to the storage
subsystem, and thus it is better-equipped to manage
low-level storage tasks
Database Storage Management
(contd..)



Object attributes can contain information about the
expected behavior of an object such as expected
read/write ratio, access pattern (sequential vs.
random), or expected size, dimension, and content of
the object.
Using OSD, a DBMS can inform the storage
subsystem of the geometry of a relation, thereby
passing responsibility for low-level data layout to the
storage device.
The dependency between the metadata and storage
system/application is removed. This assists with data
sharing between different storage applications
OSD Objects and Attributes
Scalability
Scalability – what does that word really mean :

Capacity: number of bytes, number of objects, number of files, …etc.
OSD aggregation techniques will allow for hierarchical representations




of more complex objects that consist of larger numbers of smaller
objects.
Performance: Bandwidth, Transaction rate, Latency. OSD performance
management can be used in conjunction with OSD aggregation
techniques to more effectively scale each of these three performance
metrics and maintain required QoS levels on a per-object basis.
Connectivity: number of disks, hosts, arrays, …etc. Since the OSD
model requires self-managed devices and is transport agnostic the
number of OSDs and hosts can grow to the size limits of the transport
network.
Geographic: LAN, SAN, WAN, …etc. Again, since the OSD model is
transport agnostic and since there is a security model built into the
OSD architecture, the geographic scalability is not bounded.
Processing Power: OSD processing power can be scaled.
Other Advantages





Manageability: OSD management model relies on self-managed,
policy driven storage devices, that can be centrally managed
and locally administered (i.e. central policies, local execution).
Density: OSD on individual storage devices can optimize
densities by abstracting the physical characteristics of the
underlying storage medium
Cost: address issues such as $/MB, $/sqft, $/IOP, $/MB/sec,
TCO, …etc.
Adaptability: to changing applications. Can the OSD be
repurposed to different uses such as from a film editing station
to mail serving?
Capability: can add functionality for different applications. Can
additional functionality be added to an OSD to increase its
usefulness?
Other Advantages (contd..)





Availability: Fail-over capabilities between cooperating OSD
devices. 2-way failover versus N-way failover?
Reliability: Connection-integrity capabilities
Serviceability: Remote monitoring, remote servicing, hot-plug
capability, genocidal sparing. When an OSD dies and a new one
is put in it’s place, how does it get “rebuilt”? How automated is
the service process?
Interoperability: Supported by many OS vendors, file system
vendors, storage vendors, middleware vendors.
Power: decrease the power per unit volume by relying on the
policy-driven self management schemes to “power down”
objects (i.e. move them to disks and spin those disks down).
Cluster Computing


Traditionally 'divide-and-conquer' approach,
decomposing the problem to be solved into
thousands of independently executed tasks using a
problem's inherent data parallelism--identifying the
data partitions that comprise the individual task, then
distributing each task and corresponding partition to
the compute nodes for processing.
Data from a shared storage system is staged (copied)
to the compute nodes, processing is performed, and
results are de-staged from the nodes back to shared
storage when done. In many applications, the staging
setup time can be appreciable-up to several hours for
large clusters.
OSD for Cluster Computing



Object-based storage clustering is useful in
unlocking the full potential of these Linux
compute clusters.
Intrinsic ability to linearly scale in capacity
and performance to meet the demands of the
supercomputing applications.
High bandwidth parallel data access between
thousands of Linux cluster nodes and a
unified storage cluster over standard TCP/IP
networks.
Commercial Products
OSD Commands