Ned Pyle
Sr. PM, Windows Server
Hyper-V Clusters
SMB
Scale-Out File Server Clusters
Storage Spaces Virtualization and Resiliency
Shared JBOD
Storage
Windows Server 2012
Storage
Spaces
64-node
clusters
SMB3
Scale-Out
File Server
8,000 VMs
per Cluster
Offload
Data
Transfer
Dedup
Hyper-V
Replica
Virtual
Fibre
Channel
VM
Prioritization
ReFS
ClusterAware
Updating
iSCSI Target
Server
VM Storage
Migration
VHDX
SM API
SMI-S
Storage
Service
NFS 4.1
NTFS
Trim /
Unmap
CSV online
CHKDSK
http://www.emc.com/collateral/white-papers/h11383-vnxe-introduction-wp.pdf
http://www.netapp.com/us/media/tr-4172.pdf
http://images.apple.com/osx/preview/docs/OSX_Mavericks_Core_Technology_Overview.pdf
SQL Server
\\fs1\share
File Server Node A
\\fs1\share
File Server Node B
File Server Cluster
What is it?
Store Hyper-V files in shares over the SMB 3.0 protocol
Standalone and cluster
File storage used as cluster shared storage
Shared
Storage
What is it?
Store SQL database files in shares over the
SMB 3.0 protocol
Standalone and cluster
File storage used as cluster shared storage
Shared
Storage
Increases flexibility
Eases provisioning, management and migration
Leverages converged network
Reduces capital and operational expenses
Failover transparent to server application
1
Normal operation
Supports planned and unplanned failovers
2
Failover share - connections and handles lost,
temporary stall of IO
Hardware/Software Maintenance
Hardware/Software Failures
Load Rebalancing
3
Connections and handles auto-recovered
Application IO continues with no errors
Zero downtime – small IO delay during failover
Hyper-V
Resilient for both file and directory
operations
Requires
File Servers configured as Windows Failover Cluster
Windows Server 2012 on both the servers running the
application and file server cluster nodes
Shares enabled for “continuous availability”
(default configuration for clustered file shares)
Works for both classic file server clusters (cluster disks) and
scale-out file server clusters (CSV)
1
3
\\fs\share
\\fs\share
2
Targeted for server app storage
Example: Hyper-V and SQL Server
Increase available bandwidth by adding nodes
Leverages Cluster Shared Volumes (CSV)
Key capabilities
Active/Active file shares
Fault tolerance with zero downtime
Fast failure recovery
CHKDSK with zero downtime
Support for app consistent snapshots
Support for RDMA enabled networks
Optimization for server apps
Simple management
Advantages
Scalable, fast and efficient storage access
High throughput , low latency, minimal CPU
Load balancing, automatic failover and bandwidth aggregation
via SMB Multichannel
User
Kernel
Scenarios
High performance remote file access for application servers
Required hardware
Network w/
RDMA support
NIC
Throughput
HBA
Throughput
1Gb Ethernet
~0.1 GB/sec
3Gb SAS x4
~1.1 GB/sec
10Gb Ethernet
~1.1 GB/sec
6Gb SAS x4
~2.2 GB/sec
40Gb Ethernet
~4.5 GB/sec
4Gb FC
~0.4 GB/sec
32Gb InfiniBand (QDR)
~3.8 GB/sec
8Gb FC
~0.8 GB/sec
56Gb InfiniBand (FDR)
~6.5 GB/sec
16Gb FC
~1.5 GB/sec
Network w/
RDMA support
NTFS
SCSI
Full Throughput
Bandwidth aggregation with multiple NICs
Multiple CPUs cores engaged when NIC offers
Receive Side Scaling (RSS)
Automatic Failover
SMB Multichannel implements end-to-end
failure detection
Leverages NIC teaming (LBFO) if present, but
does not require it
Automatic Configuration
SMB detects and uses multiple paths
Requires least one of these
configs
Multiple network adapters
One or more NICs that support RSS
One or more NICs configured with teaming
One or more NICs that support RDMA
Sample Configurations
End-to-end encryption of SMB data in flight
Protects data
AES-CCM
SMB Signing updated
Prevents packet tampering
AES-CMAC
Both leverage AES-NI CPUs (Nehalem+) for
better performance
Client
Server
Note required, but highly recommended
No need for IPSec or WAN accelerators
Configured per share or server
Application workload over unsecured networks
Branch Offices over WAN networks
Another good reason to remove SMB 1
SMB Encryption
\\SOFS\Share1
\\SOFS\Share2
SMB
Share1
Share2
Share1
Share2
Storage Spaces
File Server 1
File Server 2
50% improvement for small IO workloads with SMB
over RDMA
Performance
Increased 8KB IOPs from ~300K IOPS to ~450K IOPS
per interface
Increased efficiency and density of hosting workloads
with small I/O’s such as OLTP database in a VM
Efficiency
Optimizes 40Gbps Ethernet and 56Gbps InfiniBand
Live Migration Times
70
60
Seconds
50
40
30
20
10
0
SMB leveraged for VMs to access
storage, distribution from VM library,
and live migration
Common
Infrastructure
Live Migration
Limit = 500 MB/s
Default
Limit = 100 MB/s
Desire to manage bandwidth of different
types of SMB communication
Configurable SMB bandwidth limits per
category
Control
Storage
No Limit
Default, VirtualMachine and
LiveMigration
VHDX
CSV1
(Metadata
Owner)
(Not Metadata
Owner)
CSV1
(Not Metadata
Owner)
File Server 1
SMB Server
Default Instance
SMB Client
CSV2
SMB Server
CSV Instance
SMB Client
SMB Client
SMB Client
SMB Server
CSV Instance
Hyper-V Host 2
SMB Server
Default Instance
Hyper-V Host 1
CSV2
(Metadata
Owner)
File Server 2
Shared SAS Storage
Scale-Out File Server
Hyper-V supports SMB version 3.0 only
WS2012 and later
A few 3rd parties have a subset of SMB 3 now
HVBPA will confirm
Active Directory is required
Continuously Available shares recommended
Virtual Machine Manager 2012 SP1 and 2012 R2 support Hyper-V over
SMB
File Server and Hyper-V must be separate servers
If using Failover Clusters, File Server and Hyper-V must be on separate
clusters
•
•
Lowest cost for shared storage
Shares not continuously available
Low cost for continuously available
shared storage
Limited scalability
(up to a few hundred disks)
•
•
Config
VHD
Child 1
Config
Disk
VHD
Share1
Share2
Disk
Disk
Child N
Disk
Config
Child 1
Config
Child N
VHD
Disk
VHD
Disk
Share1
Share2
Share1
Share2
Disk
A
Multi-node File Server
Dual-node File Server
Single-node File Server
Disk
Disk
B
Disk
Highest scalability
(up to thousands of disks)
Higher cost, but still lower than
connecting all Hyper-V hosts with FC
•
•
Child 1
Config
Disk
VHD
Share1
Disk
Config
Disk
Disk
VHD
Share2
Disk
Child N
Share3
Disk
C
Disk
Share4
Disk
Disk
All 1GbE
Mixed 1GbE/High Speed
Clients
Clients
All High Speed (10GbE/40GbE/56GbIB)
Clients
Clients
A
B
C
D
http://www.microsoft.com/enus/download/details.aspx?id=27284
http://connect.microsoft.com/site216
http://technet.microsoft.com/en-us/library/ff686200(v=WS.10).aspx
Windows File Server Team - http://blogs.technet.com/filecab
Jose Barreto - http://blogs.technet.com/josebda
What’s new in WS2012 R2 - http://technet.microsoft.com/en-us/library/dn250019.aspx
WS2012 R2 - http://blogs.technet.com/b/josebda/archive/2013/06/26/updated-links-onwindows-server-2012-r2-file-server-and-smb-3-0.aspx
WS2012 - http://blogs.technet.com/b/josebda/archive/2013/05/05/updated-links-onwindows-server-2012-file-server-and-smb-3-0.aspx
SOFS- http://technet.microsoft.com/en-us/library/hh831349.aspx
DFSR - http://technet.microsoft.com/en-us/library/dn281957.aspx
NFS - http://technet.microsoft.com/en-us/library/jj592688.aspx
iSCSI - http://technet.microsoft.com/en-us/library/dn305893.aspx
DAC - http://technet.microsoft.com/en-us/library/hh831717.aspx
Work Folders - http://technet.microsoft.com/en-us/library/dn265974.aspx
AD RMS - http://technet.microsoft.com/en-us/library/cc771234(v=ws.10).aspx
FCI - http://technet.microsoft.com/en-us/library/dd758761(v=ws.10).aspx
BranchCache - http://technet.microsoft.com/en-us/library/hh831696.aspx
Storage Spaces - http://technet.microsoft.com/en-us/library/hh831739.aspx
StorSimple - http://microsoft.com/storsimple
http://windowsservercatalog.com/results.aspx?&chtext=&cstext=&csttext=&chbtext=&bCa
tID=1573&cpID=0&avc=38&ava=0&avq=0&OR=1&PGS=25&ready=0
http://channel9.msdn.com/Events/TechEd/Australia/2013
http://www.microsoftvirtualacademy.com/
http://technet.microsoft.com/en-au/
http://msdn.microsoft.com/en-au/