Redefining Storage Economics
CORAID: Redefining Storage Economics
Confidential Analyst Presentation - January 2010
Coraid: Redefining Storage Economics
Technical Overview – February 2011
Legacy FC & iSCSI SANs ….not so simple.
Redefining Storage Economics
Cluster
Server
Server
Server
Virtualization OS
Server
Server
Server
Parallel File System
Server
Server
Multi-path IO Software
HBA
NPIV
Switch (FC)
(Required for Vmotion)
Storage Controller
 Difficult to configure
 Difficult to Scale
 Impossible to manage
 Vendor with the best interoperability test matrix wins
(EMC Matrix: 19,000 pages and counting. http://www.emc.com/collateral/elab/emc-support-matrices.pdf)
Server
Legacy Storage Architecture
Redefining Storage Economics
Legacy SAN
1:1 Mainframe to storage
Today’s SAN
Many:1 Servers to storage
Many:1 Virtual to physical servers
Scale-out Ethernet SAN
Redefining Storage Economics


Ethernet SAN
 Bare metal performance
 No controller bottleneck
Eliminates complex topologies
No more complex multi-pathing
 Ethernet simplicity
 Lightweight protocol wins!
SCSI
SCSI
iSCSI
Fibre Chan
TCP
ATA
AoE
IP
Ethernet
Ethernet
AoE
iSCSI
Storage Protocol Stacks
Fibre Chan
Fibre Channel
Coraid Ethernet SAN – Redefining Storage Economics
Redefining Storage Economics
3 Basic Building Blocks:

Commodity x86 low-cost hardware used to build scale-out, flexible storage servers (“EtherDrive” Arrays):







Ethernet (Layer-2) network is backplane: (No TCP/IP, No iSCSI, No FC)




Non-proprietary, Commodity X86 hardware
Supports SATA, SAS, SSD Drives (can be mixed in a single chassis)
Qty (6) 1Gb, or (4) 10Gb Ethernet Ports --- 230MB - 1800MB/ per sec, per single EtherDrive Array
$500 - $900 per 1 Terabyte
Multiple EtherDrive Models (2u, 3u, and 4u / 8, 16, 24, 36 disk slots)
Redundant Hot-swap power supplies & fans
Provides near Bare-metal performance
Jumbo Frame support (9K) only requirements for Ethernet switches
Market converging to Ethernet, 1Gb, 10Gb, 40Gb, 100Gb
AoE = ATA over Ethernet Storage Protocol: delivers high-performance, low cost alternative to iSCSI & FC SANs






Ethernet SAN driver (AoE) native in all LINUX kernels
Ethernet SAN HBAs (w/ AoE driver) for non-LINUX platforms - Standard Intel dual-ported PCIe, PCI-X cards, (1Gb, 10Gb)
Does not require TCP/IP like iSCSI or proprietary networks like Fibre Channel
AoE is a connection-less protocol that connects servers and storage directly across Layer-2 Ethernet
Does not require user configured multi-path IO (MPIO or NPIV) software
AoE is datagram-based, the bandwidth can scale with additional ports, unlike IP-based solutions


Ethernet SAN packets are automatically sent over every available network path between the server and storage (without MPIO
configuration)
iSCSI can be multi-path in the context that each conversation must take place on a single port thus limits max bandwidth of each
conversation to the speed of a single port
What is a Coraid Ethernet SAN?
Redefining Storage Economics
Scale-out Ethernet SAN
• Performance & Capacity scale
linearly with each added Coraid
storage array or HBA
Ethernet LAN
Non
LINUX
LINUX
Server
Server
HBA
EtherDrive driver
- Native in LINUX Kernel
Server
HBA
HBA
EtherDrive HBAs
- Dual-ported Ethernet Cards
1Gb or 10Gb
Ethernet
• Distributed, controller-less,
modular architecture scales from
a few Terabytes to multiPetabytes
SAN
Capacity
Storage
LUNs
Performance
Simplicity of Coraid Operating System - CorOS
Redefining Storage Economics

CorOS – Coraid Storage Operating System











Ethernet SAN presents itself as direct-attached disk




Compute side see’s HBA not as NIC but as local SCSI device
SCSI Commands issued from compute side directly to disks via Ethernet SAN
Automatically uses Multiple Paths across all ports to/from storage
Coraid EtherDrive Storage is true scale-out architecture




Parallel processing scale-out SAN operating system
SAN Fabric + RAID functionality inside a single EtherDrive Server
3 commands to set up Storage Server – configures in < 1 minute
All LUN’s use static naming conventions (LUN MetaData stored on disks)
MAC address tied to LUN address
Completely eliminates MPIO and Port Binding
Flood entire Network fabric with I/O and Throughput
Supports RAID 0, 1, 5, 6, 10, or JBOD with Global Hot Spares
Continuous RAIDShield – testing fixes bad blocks, reduces drive failure
RAID configuration stored on each disk allows disk portability
Controller-less based architecture
Ethernet SAN automatically broadcast all LUN’s to Compute side
Physical connection to SAN Fabric & Jumbo Frame support (9K) only requirements
Transmission Control built in


iSCSI uses Layer-4 TCP for transmission control
FC uses proprietary $$$ hardware
Blistering Performance of Ethernet SAN
Redefining Storage Economics

Proven performance equal to FC SAN at price points below iSCSI

See ESG Report which compares EMC, NetApp, DAS to Coraid

SR Line: 230MB per sec / per Coraid EtherDrive SR Server

SRX Line: 530MB – 1800MB per sec / per Coraid EtherDrive SRX Server

Customer HPC example:



Coraid Microsoft CIFS Driver – 920MB/sec


Using Microsoft 2003 / 2008
Automatic Multi-Pathing and Link Utilization






100TB Coraid + ZFS = 2Gb per sec, 125K IOPS
Less than $900 per TB
Multiple Ethernet Connections to the SAN will automatically be recognized and utilized
Transparent to Host – No configuration required
Host sees 1 LUN regardless of the number of paths utilized
Maximize Throughput
Automatic Failover
If Physical Connection established it will be used

Support LUN Masking, White Lists, VLAN
Cost-Effective Scale-out Storage: EtherDrive SR Series
Redefining Storage Economics
High-Performance Scale-out Storage - EtherDrive SRX Series
Redefining Storage Economics
Disk Drive Options
Redefining Storage Economics
EtherDrive SR4200, SR2421, SR1621, SR821
Drive Type
Capacity
RPM
SATA
500GB
7.2k
SATA
1TB
7.2k
SATA
600GB
10k
SATA
300GB
10k
SATA
2TB
5.4k
Drive Type
EtherDrive SRX3500 (2.5 drives)
Capacity
RPM
SSD
32GB
N/A
SSD
64GB
N/A
SAS
146GB
15k
SATA
300GB
10k
SATA
500GB
7.2k
SATA
600GB
10k
Disk Drive Options
Redefining Storage Economics
EtherDrive SRX4200, SRX3200, SRX2800
Drive Type
Capacity
RPM
SSD
32GB
N/A
SSD
64GB
N/A
SAS
146GB
15k
SAS
300GB
15k
SAS
600GB
15k
SATA
300GB
7.2k
SATA
500GB
10k
SATA
600GB
10k
SATA
1TB
7.2k
SATA
2TB
5.4k
EtherDrive Host Bus Adapter (HBAs)
Redefining Storage Economics
Supported HBA’s for EtherDrive
Redefining Storage Economics
Price-Performance Comparisons
Redefining Storage Economics
Technology
Network
HBA
$/TB
AoE
1Gb Ethernet (x2)
$199
$500
AoE
10Gb Ethernet (x2)
$899
$900
iSCSI
1Gb TCP/IP
$1,000
$1-5K
FC
2-4Gb Fibre Ch
$1-2K
$4-10K
Technology
Cap Ex
Op Ex
Perf
AoE
Low
Low
Best
iSCSI
Med
Med-High
Low
FC
High
High
Fast
Direct Connection Example


Host


Ethernet
Connection
EtherDrive
Storage
Ethernet Connection
SRxxx
SRXxxx

Linux OS can use existing NIC in host
Coraid HBA (Host Bus Adapter)
AoE Drivers on Host
Host Recognizes Available Storage on
CORAID
Device Identification


Host
Represented as a Local SCSI Disk
Linux Example

/dev/etherd/e10.10


Ethernet
Connection

Compare Traditional SAN Protocols

EtherDrive
Storage
SRxxx
SRXxxx
e<Shelf#.LUN#>
Persistent Identification
/dev/sdX

X = Arbitrary Letter

No persistent identifier

Requires additional
configurations for naming and
identification
Multiple Connection Example
Host
Ethernet
Connections
EtherDrive
Storage
SRxxx
SRXxxx
• SR & SRX – multiple Ethernet interfaces
• Supports multiple Ethernet connections
• CORAID AOE Driver automatically:
• Recognizes multiple paths
• Load Balance across all available paths
• Increase Throughput
• Failover
Multiple Host Access
File
Server
Host A



Ethernet
Switch

Multiple “Initiators” (Hosts) may
access Coraid storage
LUN – Segmentation of Raw Storage
Each device accesses assigned LUN
LUN Masking

EtherDrive
Storage
Host A
LUN
Server
LUN
SRxxx
SRXxx
x
Prevents Host A from even seeing the
Server’s LUN
EtherDrive SAN Manager Appliance
Redefining Storage Economics
Target Solutions
Redefining Storage Economics
Virtualization
Cloud Storage
EtherDrive’s totally lightweight SAN protocol provides
massive I/O for demanding VMware and Xen projects.
10Gb Ethernet SAN performance, petabyte scale-out architecture,
JBOD economics. Beat the big guys at their own game.
High Performance Computing
EtherDrive enables commodity economics for large-scale data
collection. No specialized storage expertise required.
Video/Network Surveillance
EtherDrive delivers linear parallel scaling of throughput and
capacity, without complex SAN topologies.
Other solutions in use today:
• NAS / File Storage
• Disk-to-Disk Backup
• Database and email
• Development Environments
• Storage Consolidation
• University Environments
Virtualization Environment
Redefining Storage Economics
•
•
•
•
•
Massive bandwidth to disks
Less head contention
More independent disk heads
Unlimited scalability (>60k nodes)
No need for MPIO or NPIV
High Performance Computing Environment
Redefining Storage Economics
Key Points
• Commodity scale-out architectures are vital to large-scale research and computing projects
• Coraid EtherDrive has been deployed in a variety of HPC projects, delivering massive capacity
and throughput at a fraction of the cost of legacy SAN systems
CORAID
VSX Appliance
EtherDrive VSX – SAN Virtualization Appliance
Redefining Storage Economics
Key Features and Benefits
Ethernet SAN
Virtualized
Servers

Ethernet SAN virtualization appliance,
provides volume management for EtherDrive
SAN storage

Aggregate pools of EtherDrive SR/SRX LUNs
together to form small or large virtual LUNs
up to 512TB
SRX
1 GbE/10GbE
Switch
SRX Physical LUN  Each VSX can manage up to 4080 virtual
LUNs
SRX

VSX can create up to 255 copy-on-write
snapshots per virtual LUN

Snapshots can be cloned by VSX and used
as read/write copies
SRX Physical LUN
SR
SRX Physical LUN  VSX can synchronously mirror logical
volumes to create a High Availability (HA)
storage system with true No Single Point of
Failure
LV Presented
By VSX

For disaster recovery scenarios the VSX
supports asynchronous remote replication

Supports 1Gb and 10Gb Ethernet
EtherDrive VSX – True No Single Point of Failure
Redefining Storage Economics
Synchronous Mirroring:
For applications that require high availability, data from one EtherDrive storage
appliance can be synchronously mirrored to another to protect against whole
array failures. Mirroring creates a simultaneous, exact duplicate version of a
physical volume (PV) to a local designated EtherDrive storage LUN target.
No Single Point of Failure Storage Architecture
Redefining Storage Economics
ESX 1
10
ESX 2
20
VSX
VSX
20
VSX
VSX
ESX 3
11
EtherDrive
Storage
SRX3200
EtherDrive
Storage
SRX3200
Synchronous Replication
Async-Replication for DR/COOP
Redefining Storage Economics
2 VSX
VSX
0 VSX
1EtherDrive
0 Storage
DR/COOP Site
1EtherDrive
1 Storage
IP WAN
Production Site
2 VSX
VSX
0 VSX
1EtherDrive
0 Storage
1EtherDrive
1 Storage
Virtualization Communications Flow
Redefining Storage Economics
ESX 1
NFS Server
ESX 2
1. VSX aggregates storage from SR units into (1) or
more Virtual Volume Group(s).
2. VSX can create LUNs of any size from this pool –
small partitions or large cross-chassis.
3. LUN growth is configured independent of the
initiator and Storage Appliance.
20
VSX
VSX
Volume
Group
10
EtherDrive
Storage
LUN 10.1
LUN 20.1
SR1621
11
EtherDrive
Storage
LUN 11.1
LUN 20.2
SR1621
12
LUN 12.1
EtherDrive
Storage
SR1621
LUN 20.3
Redefining Storage Economics
Redefining Storage Economics
Performance
Near-bare-metal, 2x faster than iSCSI
and 30% faster than FC, 5-8x price
performance over legacy storage
Simplicity
Ethernet SAN protocol eliminates
complex MPIO topologies and arcane
Fibre Channel management
Scalability
Just add more NICs and Disks.
Handles 1/10 GB Ethernet, scales to
multiple petabytes.
Virtualization Optimized
Flexible topology, lightweight
protocol, massive throughput/IOPS,
optimized disk layout
Storage Economics 101
Utility Pricing:
Starts $0.03/Gb/month
• $0.03 for SR-Series
(230 MB/sec, 2TB disk)
• $0.045 for SRX-Series
(500+ MB/sec, 1 TB disk)
• 2-year commit, 6-mo
prepay, includes onsite
“zero-hour” hot spares
Buy a Petabyte,
Get a Petabyte
Go Big: Enterprise
• Buy a PB of EtherDrive
SRX-Series
• Get a PB of SR-Series
• Total cost: $980k for 2 PB
• Multi-petabyte
• Enterprise software
license for CorOS -2 yrs
• All-you-can-eat
EtherDrive SRX & SR
• CORAID provides you
with BOM to procure
HW direct from
manufacturer
(Avg. cost <$500/TB)
Business implications:
• Amazon economics: <$0.05/GB/month
• Ethernet simplicity, 10 Gb performance
• One tier for all!
License Agreement
1200+ Customers
Redefining Storage Economics
“CORAID EtherDrive gives us a tremendous amount of flexibility in how we configure our storage — something
that is critical in the fast-changing world of next-generation sequencing.” -Dr. Elliott Margulies, NIH
VMWare References
Redefining Storage Economics
• Configuring Coraid EtherDrive SAN appliances and deploying with ESX/ESXi 3.5
and 4.x (Partner Support)
• http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1031322
• Vmware and Coraid Technology Alliance Partner
• http://www.coraid.com/pdf/app_notes/VMW_1110_Coraid_TAP.pdf
Questions?
Lauren Sands
lsands@coraid.com
617.455.7289
Dennis Sheehan
dennis@coraid.com
301.509.4328
http://www.coraid.com
CORAID CUSTOMER SUPPORT SERVICES
Redefining Storage Economics
I. All Coraid Hardware purchased new and in it’s original packaging includes a 36 month warranty.
II. CORAID offers two service contract offerings. ( Bolded text indicates 24 x 7 vs. eSupport. )
-
1.) eSupport:
S/W Update service.
24 x 7 access to the CORAID online support web site
email support to our US based support team.
Post email diagnostic RMA services ( Return to manufacturer. )
Hrs of coverage: From 8am to 5pm All US Time zones.
Days of week: M-F.
2.) 24 x 7
S/W Update service
24 x 7 access to the COAID online support web site
Email and telephone support to our support team located in Athens GA, USA
Post diagnostic RMA services & 1 day advanced RMA. Pre diagnostic
Hrs of coverage: 24 hrs a day
Days of week: M-Sun ( 7 days / week.)
There is an Optional / add on to 24x 7 service contract that has functional advantages inline with the Coraid SRX
architecture.
Zero Hrs / onsite hot spare . . This service includes an on site CORAID Etherdrive SRX chassis with all
interconnects.
The spare SRX Chassis is put onsite and remains in “stand by mode” ready to accept disk subsystems from active
SRX storage array.