ONLINE HIERARCHICAL
STORAGE MANAGER FOR
LINUX
Ottawa Linux Symposium 2009
Who am I ?

Greg Freemyer





CTO and Litigation Triage Solutions Specialist at The Norcross
Group
Using UNIX since 1983, and have an interest in storage systems
HP certified Master SAN Architect
Participate in several kernel mailing lists and try to help out on
kernelnewbies
I did not write any of OHSM, but I became an advisor to the
project after becoming aware of them via the
kernelnewbies mailing list
OHSM Team

The ohsm team was formed to compete in various university
competitions


The team comprises of currently six members



University of Pune, India
Project Mentor- Sandeep Kumar Sinha, NetApp, India
Students - Rishi Bhushan Agrawal, Rohit Kumar Sharma, Rohit Vashist, Sneha Hendre, Vineet
Agarwal
Competition results





Third Prize at Cognizance’ 09 held at Indian Institute of Technology, Roorkee.
Consolation prize at Techkriti’ 09 held at Indian Institute of Technology, Kanpur.
First Prize at INNOVATION 2k9 held at Cummins College of Engineering, Pune.
Second Prize at Jishin’ 09 held at Bharti Vidyapeeth College of Engineering for Women,
Pune.
Third Prize at Techno Dream’ 09 held at Maharashtra Academy of Engineering, Alandi.
Why OHSM ?

Extremely large cost ratio between low-cost consumer
grade storage to high-end SSD class storage
 Bare
SATA ~ $100/TB
$5,000/TB*



Bare Intel SSD ~
Enterprises need online relocation of data between
storage tiers
Enterprise Application Security Audits make moving
data from one logical location to another difficult and
expensive
No open source solution
* Based on $750 for a 160GB Intel SSD on July 9, 2009
What is OHSM ?
TRIGGER
File
System
Scanner
Relocation
Policy
Home==13
Home
Dest = 0
3
Tier 1
Tier 2
Tier 3
OHSM Goals ?







Leveraging Hybrid aggregates (volumes)
Policy based initial placement of files
Online relocation of data between tiers
Leverage user space, existing ABIs, and existing
kernel functionality
Minimal Patches to existing filesystem(s)
Kernel Module isolating OHSM kernel logic
Mainline kernel inclusion
Complexity and Challenges




Relocating open files
Keeping file system online
Ranged block allocation
Getting device mapping from device mapper
Status of project




Prototype implementation on ext2 with both kernel
and user space code
Design documents and full source available for ext2
prototype available for download
ext3 testing in progress
ext4 is in the architectural discussion stage
Prototype Architecture
High Level Architecture
Detailed Architecture
User Space Components
OHSM
User Interface
Device
Topology
Device Mapper
Library
OHSM
Administrator
XML
Parser
OHSM Kernel Driver
Tier Block
Range Map
Policies
Meta Data
File System
Scanner
File System
Sample policy file in XML
<DEVICES>
<DEV_TIER_INFO>
<NR_TIERS>3</NR_TIERS>
<NR_DEVICES>6</NR_DEVICES>
</DEV_TIER_INFO>
<DEV_TIER>
<TIER>1</TIER>
<DEVICE>/dev/md4</DEVICE>
<DEVICE>/dev/md5</DEVICE>
<TIER>2</TIER>
<DEVICE>/dev/md3</DEVICE>
<TIER>3</TIER>
<DEVICE>/dev/md1</DEVICE>
<DEVICE>/dev/md2</DEVICE>
<DEVICE>/dev/md6</DEVICE>
</DEV_TIER>
</DEVICES>
Policy File Generator : Allocation
Policy File Generator : Relocation
Policy File Generator : Tier Device
Configuration Files




/etc/ohsm/dtd_alloc.dtd
/etc/ohsm/dtd_reloc.dtd
/etc/ohsm/dtd_device.dtd
/etc/ohsm/ohsm.conf
User space / Kernel APIs

Enable/Disable OHSM
 ioctl
services
 Set Allocation and Relocation Policies
 Set Tier Device Mappings
 Enable/Disable Relocation Policy ( Based on
Relocation rule ID or Rule No)
 Triggering relocation
 Get Tier Status (No. of allocated/free blocks)
Ext4 implementation


New Architecture design for ext4 supporting perinode preallocation and per-CPU locality
preallocation
Ted Tso's Dec. 8, 2008 ext4 post 1 - 3 new ioctls:
 Blocks NOT to allocate from – global
 Preferred Block range to Allocate from - Inode
specific
 replace blocks with donor blocks - Inode specific
1 - http://markmail.org/message/qp7zjhhdzxum7rfn
Ext4 Implementation….



Akira Fujita Patches
 EXT4_IOC_ADD_GLOBAL_ALLOC_RULE 2 – rfc
 EXT4_IOC_ADD_INODE_ALLOC_RULE 2 – rfc
 EXT4_IOC_MOVE_EXT 3 - in 2.6.31-rc series
Open/Create Hooks
Home and Destination Tier, where to store?
2 - http://markmail.org/message/yigzopzx2gkdjyw3
3 - http://markmail.org/message/a43cgtefhhz45slc
What is the OHSM Project ?




Effort to build platform for HSM on Linux
Idea came when thinking of hybrid aggregates (SSD
+ Rotational)
Its hosted at code.google.com/p/fscops and
ohsm.sourceforge.net
Open to all
Future of OHSM ?







A free Open Source Hierarchical Storage Manager
for Linux framework
Ext4 support
Submission of patches to mainline
Allocation based on directories
Analytical Engine to trigger relocation
Support for extending FS when Logical Volume is
extended
Allocating inodes from optimum tier based on policy
Questions