VMAX Best Practices for
VMware Environments
David Robertson
vSpecialist
Itzik Reich
vSpecialist
© Copyright 2011 EMC Corporation. All rights reserved.
1
VMAX FA Flag Settings for vSphere
• SPC-2
• SCSI 3 (Optional) No effect on ESX anymore.
• Unique WWN (UWN)
• Common Serial Number (C)
• OS2007 (Optional)
Screenshot from within VirtualCenter using the VSI4
© Copyright 2011 EMC Corporation. All rights reserved.
2
Connectivity Considerations with VMAX
• VMware ESX Servers should have multiple physical
HBA
• VMware Servers should be connected to multiple
directors
– Directors 7 and 8 in single engine configurations
– Connections to different directors in different engines in
multiple engine configuration
• Connect each HBA to a minimum of two ports on
different directors
– Not a requirement but strongly recommended
– I/O intensive workload will benefit
– Servicing of the array is less impacting
© Copyright 2011 EMC Corporation. All rights reserved.
3
Connectivity Considerations with VMAX−
Cont.
Ideal
Minimum
Configuration
HBA1 HBA2
© Copyright 2011 EMC Corporation. All rights reserved.
HBA1 HBA2
VMware
vSphere
Servers
4
Connectivity Considerations with VMAX−
Cont.
HBA1 HBA2
© Copyright 2011 EMC Corporation. All rights reserved.
HBA1 HBA2
VMware
vSphere
Servers
5
Path Management with VMAX
• PowerPath/VE is strongly recommended for
vSphere environment
– PowerPath/VE is optimized for multi-controller
storage (i.e. VMAX)
• NMP policy is available with vSphere
– Use Round Robin policy for Symmetrix arrays
• esxcli nmp satp setdefaultpsp -P VMW_PSP_RR -s
VMW_SATP_SYMM
– We recommend setting the Round Robin IOPS
parameter to 1 (default is 1000)
– Use Fixed Policy for Gatekeepers
© Copyright 2011 EMC Corporation. All rights reserved.
6
Using the VSI to configure Multipathing
policy
© Copyright 2011 EMC Corporation. All rights reserved.
7
VMAX FA Configuration for SPC-2
• Turned on per Fibre Channel port or per initiator
• Do not activate in a live system if not previously
set
• Default from 5773 (DMX-4)
© Copyright 2011 EMC Corporation. All rights reserved.
8
Performance and Storage Layout –
vSphere with VMAX
• Use Virtual Provisioning
– Always provides optimal balance in VMware environments
– FAST VP
• Physical Disk Size and Protection
– Depends on the IO characteristics of the workload
• LUN Layout
– Avoid using same set of disks for applications with different
I/O characteristics
• Configuration for I/O intensive application data
– Follow best practices recommendations for a physical
server
© Copyright 2011 EMC Corporation. All rights reserved.
9
Partition Alignment (VMFS and Guest OS)
• Intel-based systems are misaligned due to metadata written by the BIOS
to handle LBA to CHS translation
– Partition Misalignment affects VMFS and Guest OS partitions
• Host-based Partition Utilities can be used to Align Partitions:
– For Linux and VMFS Alignment use fdisk
• Offset partition to 64 KB boundary
• Aligned VMFS partitions are now automatically created by vSphere Client
– For Windows Operating Systems (2003 in particular) use the diskpart
utility
• Create partition aligned on 64 KB boundary
• Use diskpar for versions of pre-Windows 2003 System at SP1
– Earlier versions of diskpart will show partitions as aligned, even if they are not
• For Metavolumes (MetaLUNs) only the base device needs to be aligned
© Copyright 2011 EMC Corporation. All rights reserved.
10
Effects of Partition Misalignment
• Symmetrix VMAX uses 64K track size
• In an Aligned System, the 64 KB write would be serviced by a single
drive
• File-system misalignment affects performance in two ways:
– Misalignment causes disk crossings: I/O broken across two drives
– Misalignment causes stripe crossings: I/O broken across stripe
elements
• Even if disk operations are buffered by cache, there is performance
impact
© Copyright 2011 EMC Corporation. All rights reserved.
11
VMAX Virtual Provisioning in VMware
vSphere Environments
• vSphere provides native thin provisioning
– Either one can be used
– Both features can be used simultaneously but doing so
increases risk
• VMAX Virtual Provisioning simplifies drive and DA
workload distribution
– Provides additional benefits besides optimizing storage
use
– Ensure enough paths and TDEVs to support the workload
• VMAX Virtual Provisioning provides additional
benefits
– Zero Reclaim and Rebalancing
© Copyright 2011 EMC Corporation. All rights reserved.
12
VMAX Virtual Provisioning for vSphere–
Performance Considerations
• RAID protection of data devices
– Balance between performance versus resiliency
• Fully allocate Virtually Provisioned devices if
– Applications sensitive to latency
– The risk of oversubscription is too high
• Optimum performance when IOs are aligned
– VMFS is aligned on 64 KB boundary (use vCenter)
– Virtual disks should be aligned
• Including boot volumes (Linux, pre-win2k8/windows7)
© Copyright 2011 EMC Corporation. All rights reserved.
13
VMAX Virtual Provisioning for vSphere–
Performance Considerations – Cont.
• Striped versus Concatenated thin metavolumes
– Depends on type of workload
• Small block versus large block
• Random versus sequential
• Reads versus writes
– Influenced by presence or absence of SRDF
– Concatenated thin metavolumes can be grown
• Frequently exploited feature in VMware environments
• Striped metas can be grown as of 5875 and SE 7.2
• Most VMware environments have small block
random read workload
© Copyright 2011 EMC Corporation. All rights reserved.
14
VMAX Zero Space Reclamation
• Reclaims thin pool storage by deallocating unnecessary track
groups
– Scans each track group and discards those containing all zeros
– Deallocated tracks are presented as all zeros by Symmetrix to host
Thin Pool
0000000000000000
100110010001111
• Primary use is post migration from “thick” to “thin”
– Migration performed using TimeFinder/Clone or Open Replicator for
Symmetrix
• Reclamation should be run prior to configuring any replication
relationships
– Thin devices in active TimeFinder or SRDF relationships will be
skipped
© Copyright 2011 EMC Corporation. All rights reserved.
15
VMAX Zero Space Reclamation – Cont.
• Very useful tool in VMware environments
– Relevant if customer upgraded to vSphere
VMware Thin
Zeroedthick (VMware Thick)
© Copyright 2011 EMC Corporation. All rights reserved.
Eagerzeroedthick
16
VMAX Virtual Provisioning
Automated Pool Rebalancing
• Rebalances allocated tracks across data devices
contained within thin pool
• Levels out imbalances caused by thin pool expansion
– Or unbinding thin devices from the thin pool
• Scheduled process that runs at given intervals
Thin Pool
– User defines imbalance as a percentage utilization difference
within the pool (user configurable 1% to 50%)
• In VMware environments, storage requirements can
increase rapidly
– Mass VM deployment (VDI, testing environments, etc…)
• Virtual environments are very dynamic
– Adding datastores, removing datastores
• Automated Pool Rebalancing maintains performance and
gives best TCO
© Copyright 2011 EMC Corporation. All rights reserved.
17
VMware vSphere and FAST VP
• FAST VP operates at the sub-LUN level
• Analyzes/moves data in groups of 10 extents (1 extent =
768k)
– Great value in vSphere environments
– RDMs, dedicated datastores
• Considerations are the same as that for physical servers
• Configure a single standard size of device on all tiers
– Same size TDEV’s recommended
• For optimal system performance use Optimizer in concert
with FAST VP
– Optimizer will balance load within a tier
• Utilize EMC Virtual Storage Integrator to identify the
Storage Type
© Copyright 2011 EMC Corporation. All rights reserved.
18
EMC FAST in Action
EMC Storage with an active ESX Cluster
VMware
VMware
VMware
VMware
All Fibre Channel
Disk Drives
Disk Resources are ~80% Busy
© Copyright 2011 EMC Corporation. All rights reserved.
19
EMC FAST in Action
Add Flash Drives and Apply FAST Policy
VMware
VMware
VMware
VMware
Tiered Storage
4% Flash Drives
96% FC Drives
68% Less Disk I/O Contention
2.5X Faster Disk Response Time
© Copyright 2011 EMC Corporation. All rights reserved.
20
VMware vStorage API for Array Integration (VAAI)
Available on VMAX with Enginuity 5875
Block Zero
Without API
 SCSI write – many blocks from host to
array
With API
 SCSI Write Same command – one
block from host to array and repeatedly
written
Benefits
 Reduce host I/O traffic
Without
© Copyright 2011 EMC Corporation. All rights reserved.
With
 Higher throughput for guest
applications
21
VMware vStorage API for Array Integration (VAAI)
Available on VMAX with Enginuity 5875
Full Copy
Without API
 SCSI Read – data from array to host
 SCSI Write – data from host to array
With API
 SCSI Extended Copy – data moved
within the array
Benefits
 Reduces time to migrate
Without
With
© Copyright 2011 EMC Corporation. All rights reserved.
 Reduces time to deploy
22
VMware vStorage API for Array Integration (VAAI)
Available on VMAX with Enginuity 5875
Hardware Assisted Locking
Without API
 Reserves the complete LUN
 Requires several SCSI commands
With API
 Works at a block level – no LUN lock
vmdk1
vmdk1
vmdk2
vmdk2
 SCSI Compare Swap command
Benefits
 Reduces SCSI reservation issues
Without
© Copyright 2011 EMC Corporation. All rights reserved.
With
 Allows more changes to metadata
23
Thank You!
• EMC Symmetrix VMAX Enginuity 5875: FAST VP, VAAI–Making The Best
Array even Better: http://itzikr.wordpress.com/2011/01/17/emc-symmetrixvmax-enginuity-5875-fast-vp-vaai-making-the-best-array-even-better/
• SRM Automated Failback Using EMC Symmetrix VMAX:
http://itzikr.wordpress.com/2011/01/10/srm-automatic-failback-using-emcsymmetrix-vmax/
• Increase Scalability for Virtual Desktops with EMC Symmetrix FAST VP
and VMware VAAI: http://www.emc.com/collateral/hardware/whitepapers/h8062-scalability-vmax-vaai-wp.pdf
• April 28-EMC Live Webcast: VNX Best Practices for VMware
Environments: http://info.emc.com/mk/get/DBM1069017709_raf_lp?reg_src=PA_Vmware
• Tuning NMP IOPS Parameter for Symmetrix Devices:
http://www.emc.com/collateral/hardware/white-papers/h8119-tuning-vmwaresymmetrix-wp.pdf
• Using EMC Symmetrix Storage in VMware vSphere Environments:
http://www.emc.com/collateral/hardware/solution-overview/h2529-vmwareesx-svr-w-symmetrix-wp-ldv.pdf
• White Paper: Using VMware vStorage APIs for Array Integration with
EMC Symmetrix VMAX: http://www.emc.com/collateral/hardware/whitepapers/h8115-vmware-vstorage-vmax-wp.pdf
© Copyright 2011 EMC Corporation. All rights reserved.
24
VMware Site Recovery Manager Failover/
Failback DEMO
© Copyright 2011 EMC Corporation. All rights reserved.
25
Storage Pool Management feature for VSI
DEMO
© Copyright 2011 EMC Corporation. All rights reserved.
26