The Host Server
VMware ESXi Configuration Guide
September 2016
The Data Infrastructure Software Company
Table of contents
Overview
Recent changes made to this document
VMware ESXi compatibility lists
The DataCore Server's settings
Running a DataCore Server in a Virtual Machine
The VMware ESXi Host's settings
VMware Path Selection Policies
Configuring the Round Robin Path Selection Policy
Configuring the Fixed Path Selection Policy
Configuring the Most Recently Used Path Selection Policy
Known issues
ESXi 6.x
ESXi 5.x
ESX 4.x
Appendix A
Preferred Server & Preferred Path settings
Appendix B
Configuring Disk Pools
Appendix C
Reclaiming storage
Appendix D
Moving from ‘Most Recently Used’ to either ‘Round Robin’ or ‘Fixed’ Path Selection Policy
Previous Changes
Page | 2
3
3
4
8
10
11
15
17
19
21
23
24
26
28
29
29
31
31
32
32
35
35
36
The Host Server – VMware ESXi Configuration Guide
Overview
This document provides VMware-specific settings when serving Virtual Disks from a DataCore
Server.
Fundamental VMware administration skills are assumed; including how to connect VMware
ESXi Hosts to storage array target ports (i.e. DataCore Server Front End ports) via either Fibre
Channel or iSCSI, along with the process of discovering, mounting and formatting disk devices in
general.
Recent changes made to this document
New information added since last update (August 2016)
Added
Known Issues
There has been a general re-organization of this section separating all issues into subsections determined by the
version of ESXi that the known issue refers to.
Known Issues - 6.x
Running Microsoft Cluster Services in a Virtual Machine on Virtual Disks that have more than one Front End
mapping to each DataCore Server may cause unexpected loss of access
This has now been fixed in https://kb.vmware.com/kb/2145663
VMware ESXi 6.0, Patch Release ESXi600-201608001 and was documented previously in VMware’s own internal
SR#15597438602.
Known Issues - 5.5
Running Microsoft Cluster Services in a Virtual Machine on Virtual Disks that have more than one Front End
mapping to each DataCore Server may cause unexpected loss of access
This affects ESX 5.5. This is documented in VMware’s own internal SR#15597438602. Please contact VMware
directly about this as DataCore is not aware of any fix for ESXi 5.5 at this time.
Updated
The VMware ESXi Host's settings – ISCSI Connections
The information that was previously in the 'Known Issues' section regarding connections from multiple NICs
sharing the same IQN has been moved to this section as it affects all versions of ESX and is not so much a 'Known
Issue' than a configuration requirement.
Known Issues – ESX 6.x
Unable to access filesystem for MSCS cluster nodes after vMotion
https://kb.vmware.com/kb/2144153
A fix is available (as well as a workaround) from the VMware knowledge base article above.
Previous changes made to this document
Please see page 36.
Page | 3
The Host Server – VMware ESXi Configuration Guide
VMware ESXi compatibility lists
Operating system versions
SANsymphony
9.0 PSP 4 Update 4
(1)
10.0 (all versions)
ESXi Version
With ALUA
Without ALUA
With ALUA
Without ALUA
3.x and earlier
Not Supported
Not Supported
Not Supported
Not Supported
4.0.x
Not Qualified
Not Qualified
Not Supported
Not Supported
4.1.x
Supported
Not Qualified
Not Supported
Not Supported
5.x
Supported
Not Qualified
Supported
Not Qualified
6.x
Supported
Not Qualified
Supported
Not Qualified
Regarding VMware’s own Hardware Compatibility List
Please see the official statement from DataCore here:
DataCore Software and VMware's Hardware Compatibility List (HCL)
http://datacore.custhelp.com/app/answers/detail/a_id/1131
Fibre Channel and iSCSI Connections
DataCore Software supports VMware ESXi Hosts using either Fibre Channel or iSCSI Connections
to SANsymphony Front-End (FE) Ports for any Virtual Disk type.
SCSI UNMAP
See the VStorage API for Array Integration (VAAI) compatibility table on page 7.
VSphere Metro Storage Clusters (vMSC)
vMSC is supported for all SANsymphony/VMware ESXi combinations that are listed as
'supported' in the VMware ESXi compatibility list above where Virtual Disks have been
formatted using VMFS5 file system. Virtual Disks whose file systems have been 'upgraded'
(from earlier versions of VMFS) to VMFS5 are not supported (even if the
SANsymphony/VMware ESXi combination is listed as 'supported' on the previous page).
1
SANsymphony-V 8.x and all versions of SANsymphony 9.x before PSP4 Update 4 are now ‘End of Life’.
Please see: End of Life Notifications http://datacore.custhelp.com/app/answers/detail/a_id/1329
Page | 4
The Host Server – VMware ESXi Configuration Guide
VMware ESXi compatibility lists
Operating system version compatibility notes
Supported
These VMware/SANsymphony combinations have been tested using all the host-specific
settings listed in this document against all Virtual Disk types. Mirrored and Dual Virtual Disks
have been tested for 'high availability' in all possible failure scenarios.
Not Qualified
These VMware/SANsymphony combinations have not been tested against any Mirrored or Dual
Virtual Disks types. DataCore therefore cannot guarantee 'high availability' in any failure
scenario (even if all host-specific settings listed in this document are followed) however, selfqualification may be possible. For more information on this please see:
http://datacore.custhelp.com/app/answers/detail/a_id/1506
Support for anyESXi versions that are considered ‘End of Life’ by VMware and are listed as 'Not
Qualified' can still be self-qualified but only if there is an agreed ‘support contract’ with
VMware. In this case though, DataCore Technical Support will not do root-cause analysis for
SANsymphony in the case of any future issues but will offer 'best effort' support to get Hosts
accessing any SANsymphony Virtual Disks.
Note: Non-mirrored Virtual Disks are always considered as supported even for 'Not Qualified'
combinations of VMware/SANsymphony.
Not Supported
These VMware/SANsymphony combinations have usually failed one or more of our 'high
availability' tests when using Mirrored or Dual Virtual Disks types; but also may simply be where
an Operating System's own requirements (or limitations) due to its age make it impractical to
test. Entries marked as 'Not Supported' can never be self-qualified. Mirrored or Dual Virtual
Disks types are configured at the end-user's own risk.
Note: Non-mirrored Virtual Disks are always considered as supported even for 'Not Supported'
combinations of VMware/SANsymphony.
End of Life VMware versions
Support for any VMware version that is considered ‘End of Life’ by VMware or has no active
development/Long Term Support can still be self-qualified but only if there is an agreed
‘support contract’ with VMware.
In this case, DataCore Technical Support will help the customer to get the Host Operating
system accessing Virtual Disks, but will not then do any root-cause analysis.
Page | 5
The Host Server – VMware ESXi Configuration Guide
VMware ESXi compatibility lists
VStorage API for Array Integration (VAAI)(1)
SANsymphony
9.0 PSP 4 Update 4
(2)
10.0 (all versions)
ESXi Version
VAAI
VAAI
3.x and earlier
N/A
N/A
Does not work
Does not work
5.x
Supported
Supported
6.x
Supported
Supported
4.x
(3)
Note: For more information on using SCSI UNMAP to reclaim storage from Disk Pools, please
refer to Appendix C: 'Reclaiming Storage' on page 32 for specific instructions.
VMware VVOL VASA API 2.0
SANsymphony
9.0 PSP 4 Update 4
10.0 PSP3 and earlier
10.0 PSP 4 and greater
ESXi Version
VASA support for VVOL
VASA support for VVOL
VASA support for VVOL
5.x and earlier
N/A
N/A
Not Supported
6.x
N/A
N/A
Supported
Note: this also includes vMotion over iSCSI. Also see the Getting Started with the VASA Provider
for SANsymphony section from the SANsymphony Help:
http://www.datacore.com/SSV-Webhelp/Getting_Started_with_VASA_Provider.htm
1
The following VAAI specific commands are supported by the DataCore Server:
Atomic Test & Set (ATS), Clone Blocks/Full Copy/XCOPY, Zero Blocks/Write Same and Block Delete/SCSI UNMAP
2
SCSI UNMAP supported was included in SANsymphony-V 9.0 PSP 4 Update 4. SANsymphony-V 8.x and all versions
of SANsymphony 9.x before PSP4 Update 4 are now ‘End of Life’. Please see: End of Life Notifications
http://datacore.custhelp.com/app/answers/detail/a_id/1329
3
For VMware ESXi 4.1 Hosts, VAAI must be disabled on the Host otherwise it will cause unexpected behaviour.
Please refer to VMware's own knowledgebase article: "Disabling the VAAI functionality in ESX/ESXi (1033665)"
http://kb.vmware.com/kb/1033665
Page | 6
The Host Server – VMware ESXi Configuration Guide
VMware ESXi compatibility lists
VMware ESXi Path Selection Policies (PSP)
VMware Path Selection Policy
Version
Most Recently Used (MRU)
Fixed
Round Robin (RR)
4.x
Supported without ALUA
Supported with ALUA
Supported with ALUA
5.x
Supported without ALUA
Supported with ALUA
Supported with ALUA
6.x
Supported without ALUA
Supported with ALUA
Supported with ALUA
Path Selection Policy compatibility notes
Round Robin (RR)
Any SANsymphony/ESXi version combinations that are listed as 'Not Supported' (refer to the
operating system compatibility notes on page 4) can only use non-mirrored Virtual Disks; so
failover configurations are not permitted. Using the RR for non-mirrored Virtual Disks however
is supported. Where the combination is listed as 'Not Qualified', and the Virtual Disk is
mirrored, RR is still 'Supported' but only when the ESXi version has been 'self-qualified'.
Most Recently Used (MRU) and Fixed
Any SANsymphony/ESXi version combinations that are listed as 'Not Supported' (refer to the
operating system compatibility notes on page 4) can only use non-mirrored Virtual Disks so the
table above is not-applicable. Where the combination is listed as 'Not Qualified', and the Virtual
Disk is mirrored, both PSP types are still 'Supported' but only when the ESXi version has been
'self-qualified'.
ESX 6.x
Fixed and RR Path Selection Policies have both tested by DataCore and are listed on VMware's
own Hardware Compatibility List (HCL). MRU is not on VMware's HCL for ESX 6.x and has not
been tested by DataCore Software so is considered 'not qualified'.
ESX 5.x and 4.x
Fixed and RR Path Selection Policies have both tested by DataCore but only the RR Path
Selection Policy (which is the more complex of the two) is listed on VMware's own Hardware
Compatibility List. Also note that MRU is not on VMware's HCL for ESX 5.x but has also not been
tested by DataCore Software so is considered 'not qualified'.
Page | 7
The Host Server – VMware ESXi Configuration Guide
The DataCore Server's settings
Also see:
Video: Configuring ESX Hosts in the DataCore Management Console
http://datacore.custhelp.com/app/answers/detail/a_id/1637
These are the Host-specific settings that need to be configured directly on the DataCore Server.
Operating system type
See the Registering Hosts section from the SANsymphony Help:
http://www.datacore.com/SSV-Webhelp/Hosts.htm
When registering the Host choose the 'VMware ESXi' menu option.
Port roles
Ports used for serving Virtual Disks to Hosts should only have the Front End (FE) role enabled.
Mixing other Port Role types may cause unexpected results as Ports that only have the FE role
enabled will be turned off when the DataCore Server software is stopped (even if the physical
server remains running). This helps to guarantee that any Hosts do not still try to access FE
Ports, for any reason, once the DataCore Software is stopped but where the DataCore Server
remains running. Any Port with the Mirror and/or Back End role enabled do not shut off when
the DataCore Server software is stopped but still remain active.
Multipathing support
The Multipathing Support option should be enabled so that Mirrored Virtual Disks or Dual
Virtual Disks can be served to Hosts from all available DataCore FE ports. Also see the
Multipathing Support section from the SANsymphony Help: http://www.datacore.com/SSVWebhelp/Hosts.htm
Non-mirrored Virtual Disks and Multipathing
Non-mirrored Virtual Disks can still be served to multiple Hosts and/or multiple Host Ports from
one or more DataCore Server FE Ports if required; in this case the Host can use its own
multipathing software to manage the multiple Host paths to the Single Virtual Disk as if it was a
Mirrored or Dual Virtual Disk.
Note: Hosts that have non-mirrored Virtual Disks served to them do not need Multipathing
Support enabled unless they have other Mirrored or Dual Virtual Disks served as well.
Page | 8
The Host Server – VMware ESXi Configuration Guide
The DataCore Server's settings
Asymmetrical Logical Unit Access (ALUA) support
The ALUA support option should be enabled if required and if Multipathing Support has been
also been enabled (see above). Please refer to the Operating system compatibility table on page
4 to see which combinations of VMware ESXi and SANsymphony support ALUA.
More information on Preferred Servers and Preferred Paths used by the ALUA function can be
found on in Appendix A on page 29.
Serving Virtual Disks to the Hosts for the first time
DataCore recommends that before serving Virtual Disks for the first time to a Host, that all
DataCore Front-End ports on all DataCore Servers are correctly discovered by the Host first.
Then, from within the SANsymphony Console, the Virtual Disk is marked Online, up to date and
that the storage sources have a host access status of Read/Write.
Virtual Disks LUNs and serving to more than one Host or Port
DataCore Virtual Disks always have their own unique Network Address Authority (NAA)
identifier that a Host can use to manage the same Virtual Disk being served to multiple Ports on
the same Host Server and the same Virtual Disk being served to multiple Hosts.
See the SCSI Standard Inquiry Data section from the online Help for more information on this:
http://www.datacore.com/SSV-Webhelp/Changing_Virtual_Disk_Settings.htm
While DataCore cannot guarantee that a disk device's NAA is used by a Host's operating system
to identify a disk device served to it over different paths generally we have found that it is. And
while there is sometimes a convention that all paths by the same disk device should always
using the same LUN 'number' to guarantees consistency for device identification, this may not
be technically true. Always refer to the Host Operating System vendor’s own documentation for
advice on this.
DataCore's Software does, however always try to create mappings between the Host's ports
and the DataCore Server's Front-end (FE) ports for a Virtual Disk using the same LUN number (5)
where it can. The software will first find the next available (lowest) LUN 'number' for the HostDataCore FE mapping combination being applied and will then try to apply that same LUN
number for all other mappings that are being attempted when the Virtual Disk is being served.
If any Host-DataCore FE port combination being requested at that moment is already using that
same LUN number (e.g. if a Host has other Virtual Disks served to it from previous) then the
software will find the next available LUN number and apply that to those specific HostDataCore FE mappings only.
5
The software will also try to match a LUN 'number' for all DataCore Server Mirror Port mappings of a Virtual Disk
too, although the Host does not 'see' these mirror mappings and so this does not technically need to be the same
as the Front End port mappings (or indeed as other Mirror Path mappings for the same Virtual Disk). Having Mirror
mappings using different LUNs has no functional impact on the Host or DataCore Server at all.
Page | 9
The Host Server – VMware ESXi Configuration Guide
The DataCore Server's settings
Running a DataCore Server in a Virtual
Machine
See the article Best Practice: Installing a DataCore Server within a Virtual Machine
http://datacore.custhelp.com/app/answers/detail/a_id/1155
Page | 10
The Host Server – VMware ESXi Configuration Guide
The VMware ESXi Host's settings
The following are the Host-specific settings that need to be configured directly on the Host
Server.
Note: Older versions (of VMware ESXi) may require different Host settings when compared to
newer versions. When a setting or configuration change is listed for one version but not another
then is only required for that specific version of VMware ESXi. If you have upgraded from an
older version and a specific setting is no longer documented for your newer version then assume
that no further changes are needed for those settings that are no longer listed in your newer
version and they should be left as they were.
ISCSI Connections
TCP Ports
Make sure TCP Port 3260 is opened for all iSCSI Communication to the DataCore Server.
See the 'TCP and UDP Ports' section from the SANsymphony Help:
http://www.datacore.com/SSV-Webhelp/windows_security_settings_disclosure.htm
ESX Hosts with IP addresses that share the same IQN connecting to the same DataCore Server
Front-end port is not supported (this also includes ESXi 'Port Binding').
The Front End port will only accept the ‘first’ connection from a given IQN that attempts to
login to it where a unique ISCSI Session ID (ISID) is created for that connection. All subsequent
connections that then come from a different NIC that happens to share the same IQN as the
first login, will causes a ISID conflict and will be rejected by the DataCore Server. After that, no
further iSCSI logins will be possible for this IQN. This may cause unexpected disconnects
between the Host and the DataCore Server for those connections.
It is important to note that if the ‘first’ successful connection gets disconnected for any reason
(e.g. by a ‘SCSI reset’), then one of the other NICs - sharing the same IQN - may re-attempt a
login and, if successful, will take the ‘session’ for itself. This will now block the previouslyconnected NIC from being able to re-connect and it will now remain disconnected.
See the following examples that illustrate this behavior:
Example 1 – The supported configuration
An ESX Host (ESX1) has two different Network Interfaces; each with its own IP address but each
with the same IQN:
Page | 11
The Host Server – VMware ESXi Configuration Guide
The VMware ESXi Host's settings
192.168.1.1
192.168.2.1
(iqn.esx1)
(iqn.esx1)
Two DataCore Servers each with two Front-end ports and their corresponding Network
Interfaces and IQNs:
192.168.1.100
192.168.2.100
192.168.1.101
192.168.2.101
(iqn.dcs1-1)
(iqn.dcs1-2)
(iqn.dcs2-1)
(iqn.dcs2-2)
Each Network Interface of ESX1 can connect to a separate Front-end Port on both DataCore
Servers:
(iqn.esx1)
(iqn.esx1)
(iqn.esx1)
(iqn.esx1)
192.168.1.1
192.168.1.2
192.168.2.1
192.168.2.2




ISCSI
ISCSI
ISCSI
ISCSI




192.168.1.101
192.168.1.102
192.168.2.101
192.168.2.102
(iqn.dcs1-1)
(iqn.dcs1-2)
(iqn.dcs2-1)
(iqn.dcs2-2)
There is no case in the above example where the ‘other’ Network Interface on ESX1 is also
trying to connect to the ‘other’ Front-end port on the same DataCore Server (i.e. there are no
multiple ISCSI session connections).
Example 2 – The unsupported configuration
Using the same type of configuration as example 1 above;
(iqn.esx1)
(iqn.esx1)
(iqn.esx1)
(iqn.esx1)
192.168.1.1
192.168.1.2
192.168.2.1
192.168.2.2




ISCSI
ISCSI
ISCSI
ISCSI




192.168.1.101
192.168.1.101
192.168.2.101
192.168.2.102
(iqn.dcs1-1)
(iqn.dcs1-1)
(iqn.dcs2-1)
(iqn.dcs2-2)
In this case, both of the Network Interfaces from ESX1 have been configured to connect to the
same Network Interface on DataCore Server 1 – in this case iqn.dcs1-1.
DataCore Server1 will accept only one of the connections and the other will be rejected; any
subsequent interruption over that iSCSI connection may then result in either of the two ESX
Network Interfaces being able to (re)connect to iqn.dcs1-1, forcing the other ESX connection to
be rejected. In other words, there is no guarantee that the ESX1 connection that was previously
logged into iqn.dcs1-1 will be able to reconnect if disconnected for any reason and if the ‘other’
Network Interface logs in before it. A solution in this case may be Teaming the NICs together as
the teamed connections will then only have a single IP address and be recognized by the
DataCore Server as a ‘single’ NIC.
Page | 12
The Host Server – VMware ESXi Configuration Guide
The VMware ESXi Host's settings
Also See the Important Notes section from the SANsymphony Help:
http://www.datacore.com/SSV-Webhelp/Configuring_iSCSI_Connections.htm
Page | 13
The Host Server – VMware ESXi Configuration Guide
The VMware ESXi Host's settings
Advanced Settings
Note: A reboot may not be needed if any of these settings are changed from a previous value
please check with VMware first.
ESX 6.x and 5.x
From within the ESXi Configuration Tab under Advanced Settings change and/or verify the
following values are set:
Disk.DiskMaxIOSize = 512
ESX 4.1.x
From within the ESXi Configuration Tab under Advanced Settings change and/or verify the
following values are set:
Disk.DiskMaxIOSize = 512
Disk.QFullSampleSize = 32
Disk.QFullThreshold = 8
Disk.UseLunReset = 1
Disk.UseDeviceReset = 0
SCSI.CRTimeoutDuringBoot = 10000
ESX 4.0.x
From within the ESXi Configuration Tab under Advanced Settings change and/or verify the
following values are set:
Disk.DiskMaxIOSize = 512
Disk.QFullSampleSize = 32
Disk.QFullThreshold = 8
Disk.UseLunReset = 1
Disk.UseDeviceReset = 0
SCSI.CRTimeoutDuringBoot = 1
SCSI.ConflictRetries = 200
Page | 14
The Host Server – VMware ESXi Configuration Guide
VMware Path Selection Policies
Which Path Selection Policies (PSP) are supported?
Please refer to VMware ESXi Path Selection Policies (PSP) compatibility list on page 6.
Which PSP does DataCore Software recommend?
DataCore does not recommend one particular policy over another; one user’s installation and
configuration of SANsymphony will be different to another's.
Note: Some PSPs are not supported by VMware themselves for certain types of Virtual Machine
Operating Systems. DataCore cannot take responsibility for these VMware-unsupported Virtual
Machine/PSP combinations should any issues occur.
See Multipathing policies in ESXi 5.x and ESX/ESXi 4.x (1011340)
http://kb.vmware.com/kb/1011340
Changing the PSP type on an already-served Virtual Disk
As long as the same Storage Array Type Plug-in (SATP) being used on the Host is the same for
the new PSP there is nothing that needs to be done on the DataCore Server.
If the current SATP is different to what the new PSP requires (for example, moving from 'Most
Recently Used' to 'Round Robin'), then DataCore recommend that you unserve the Virtual Disks
first, delete the old SATP value adding the new SATP and serve them back again.
Note: Changing the SATP type may also require that the ALUA option also be changed as well on
the Host, within the SANsymphony Console, from its current setting. In which case see the ‘After
changing the settings’ section of Changing multipath or ALUA support settings for hosts from
the SANsymphony Help: http://www.datacore.com/SSV-Webhelp/Hosts.htm
Using different PSPs for the same Virtual Disk on multiple Hosts
While this is technically possible, it is not supported and DataCore cannot guarantee the
behavior of the VMware ESXi Hosts in this case. Always use the same PSP for the same Virtual
Disk on all VMware ESXi Hosts that it is served to.
Page | 15
The Host Server – VMware ESXi Configuration Guide
VMware Path Selection Policies
Which Storage Array Type Plug-in (SATP) should I use?
Please refer to the following pages to determine which SATP and how to configure it for the
particular PSP you wish to use.
Note: Auto-detection by VMware ESXi to choose the correct Path Selection Policy and/or
Storage Array Plugin Type to use for a given Virtual Disk can be inconsistent in older versions
and VMware ESXi may, for example, default to Most Recently Used by default for any Virtual
Disk mapped to the Host regardless if the ALUA option had been enabled or not. This type of
mistake will cause unexpected results during any failover event.
It is, therefore, important to always verify manually, that both the correct PSP and SATP have
been selected. This can be done directly on the VMware vSphere client GUI or by running one of
the following two commands on the VMware ESXi console:
esxcli storage nmp device list | grep -C 3 DataCore
esxcli storage nmp device list | grep -C 3 0030d9
The first command lists all disk devices served to the VMware ESXi Host that have the
SANsymphony ‘DataCore’ SCSI Vendor string; the second command does the same thing but
uses DataCore’s unique NAA identifier. Both of these are part of any SANsymphony Virtual Disk’s
SCSI Standard Inquiry Data.
See the SCSI Standard Inquiry Data section from the SANsymphony Help:
http://www.datacore.com/SSV-Webhelp/Changing_Virtual_Disk_Settings.htm
The additional ‘-C’ switch for the ‘grep’ command will display an additional 3 lines of output
above and below the line that the string searched for appears on which should then include the
Path Selection Policy and the Storage Array Type Plugin. Increase this number to display more of
the Virtual Disk’s properties as required.
If using SANsymphony’s own ‘VMware vCenter Integration’, searching by the NAA identifier is
the only way to list the Virtual Disks on the command line.
Page | 16
The Host Server – VMware ESXi Configuration Guide
VMware Path Selection Policies
Configuring the Round Robin Path Selection Policy
Use the SATP 'VMW_SATP_ALUA' with the claim option 'tpgs_on'.
Round Robin can be configured using either the 'default' SATP type or by configuring a custom
SATP rule.
Using the default SATP type
It is possible to use VMware ESXi’s built-in, generic 'VMW_SATP_ALUA' rule
VMW_SATP_ALUA
system
tpgs_on
Any array with ALUA support
Using a custom SATP rule
To create a custom rule run the following command on the ESXi Host’s console
esxcli storage nmp satp rule add -V DataCore -M "Virtual Disk" -s VMW_SATP_ALUA
-c tpgs_on -P VMW_PSP_RR
Verify the custom rule has been set correctly
esxcli storage nmp satp rule list -s VMW_SATP_ALUA | grep DataCore
The response should look like something this(1)
VMW_SATP_ALUA
DataCore
Virtual Disk
user
tpgs_on
VMW_PSP_RR
This custom SATP rule can be used for all Virtual Disks from any DataCore Servers when using
the Round Robin PSP.
Note: Round Robin is only supported with the ALUA option enabled on the VMware Host from
within the DataCore Server's Console.
See Changing multipath or ALUA support settings for hosts from the SANsymphony Help:
http://www.datacore.com/SSV-Webhelp/Multipath_Support.htm
1
This example is taken from VMware ESXi version 5.5
Page | 17
The Host Server – VMware ESXi Configuration Guide
VMware Path Selection Policies
Which Preferred Server setting on the DataCore Server should I use with Round Robin?
DataCore recommends, when using the Round Robin PSP and configuring your Hosts for the
first time to set an explicit DataCore Server as the Preferred Server or leave the 'Auto select'
setting configured and not use the ‘All’ setting.
When the Host's Preferred Server setting is either 'Auto Select', or is using an explicitly named
DataCore Server, only the Host paths that are connected to either the first DataCore Server
listed in the Virtual Disk's properties (for 'Auto Select') or the named DataCore Server
respectively are set as ‘Active Optimized’. The Host's paths connected to the other DataCore
Server are set as 'Active Non-optimized'.
VMware Hosts will only send IO to 'Active Optimized' paths when there is a choice between
that or 'Active Non-optimized'.
Caution is therefore advised when using the ‘All’ setting as this allows the VMware Host to send
I/O to all paths on all DataCore Servers for any served Virtual Disk. While this may seem
preferable, in configurations where there are significant path distances between servers (e.g.
across remote sites) or where the speed of links between remote servers is significantly slower
than links between local servers on the same site, it is possible that longer I/O wait times are
encountered between paths to remote servers which will then cause additional delays for I/O
within the same request but sent via paths to local servers resulting in overall significant I/O
latency.
Testing is advised.
Please see Appendix A –Notes on Preferred Server and Preferred Path settings on page 29 for a
more details explanation when using the ‘All’ setting.
Also see the Preferred Servers section from the SANsymphony Help:
http://www.datacore.com/SSV-Webhelp/Port_Connections_and_Paths.htm
Page | 18
The Host Server – VMware ESXi Configuration Guide
VMware Path Selection Policies
Configuring the Fixed Path Selection Policy
Use the SATP 'VMW_SATP_ALUA' with the claim option 'tpgs_on'.
The Fixed PSP can be configured using either the 'default' SATP type or by configuring a custom
SATP rule.
Using the default SATP type
It is possible to use VMware ESXi’s built-in, generic 'VMW_SATP_ALUA' rule
VMW_SATP_ALUA
system
tpgs_on
Any array with ALUA support
Using a custom SATP rule
To create a custom rule run the following command on the ESXi Host’s console
esxcli storage nmp satp rule add -V DataCore -M "Virtual Disk" -s VMW_SATP_ALUA
-c tpgs_on -P VMW_PSP_FIXED
Verify the custom rule has been set correctly
esxcli storage nmp satp rule list -s VMW_SATP_ALUA | grep DataCore
The response should look like something this(1)
VMW_SATP_ALUA
DataCore
Virtual Disk
user
tpgs_on
VMW_PSP_FIXED
This custom SATP rule can be used for all Virtual Disks from any DataCore Servers when using
the Fixed PSP.
Note: Fixed PSP is only supported with the ALUA option enabled on the VMware Host from
within the DataCore Server's Console.
See Changing multipath or ALUA support settings for hosts from the SANsymphony Help:
http://www.datacore.com/SSV-Webhelp/Multipath_Support.htm
Which Preferred Server setting on the DataCore Server should I use with Fixed PSP?
1
This example is taken from VMware ESXi version 5.5
Page | 19
The Host Server – VMware ESXi Configuration Guide
VMware Path Selection Policies
Unlike Round Robin, where DataCore recommend (initially) to not use the 'All' Preferred Server
setting, when using the Fixed PSP the 'All' setting is mandatory and no other Preferred Server
setting is supported.
This is because the Fixed PSP always required an 'Active Optimized' path to failover or failback
to for it to work as expected.
Note: The Fixed PSP will not send IO to all 'Active Optimized' paths like the Round Robin PSP
does. The actual 'active' path used by the VMware Host that is using the Fixed PSP is configured
on the ESX Host directly and is not controlled by the DataCore Server. Please refer to VMware's
own documentation on how to configure the 'active' path when using the Fixed PSP.
Please see Appendix A – “Notes on Preferred Server and Preferred Path settings” on page 29 for
a more details explanation when using the ‘All’ setting.
Also see the Preferred Servers section from the SANsymphony Help:
http://www.datacore.com/SSV-Webhelp/Port_Connections_and_Paths.htm
Page | 20
The Host Server – VMware ESXi Configuration Guide
VMware Path Selection Policies
Configuring the Most Recently Used Path Selection Policy
Use the SATP 'VMW_SATP_DEFAULT_AA' with no claim option set.
The Fixed PSP can be configured using either the 'default' SATP type or by configuring a custom
SATP rule.
Using the default SATP type
It is possible to use VMware ESXi’s built-in, generic 'VMW_SATP_DEFAULT_AA' rule
VMW_SATP_DEFAULT_AA
fc
system
Fibre Channel Devices
Using a custom SATP rule
To create a custom rule run the following command on the ESXi Host’s console
esxcli storage nmp satp rule add -V DataCore -M "Virtual Disk" -s
VMW_SATP_DEFAULT_AA -P VMW_PSP_MRU
Verify the custom rule has been set correctly
esxcli storage nmp satp rule list -s VMW_SATP_DEFAULT_AA | grep DataCore
The response should look like something this(8)
VMW_SATP_DEFAULT_AA
DataCore
Virtual Disk
user
VMW_PSP_MRU
This custom SATP rule can be used for all Virtual Disks from any DataCore Servers when using
the Fixed PSP.
Note: Most Recently Used is only supported without the ALUA option enabled on the VMware
Host from within the DataCore Server's Console.
See Changing multipath or ALUA support settings for hosts from the SANsymphony Help:
http://www.datacore.com/SSV-Webhelp/Multipath_Support.htm
8
This example is taken from VMware ESXi version 5.5
Page | 21
The Host Server – VMware ESXi Configuration Guide
VMware Path Selection Policies
Which Preferred Server setting on the DataCore Server should I use with Most Recently Used?
Because the ALUA option is not supported when using the Most Recently Used PSP it must
never be enabled on the Host. The Preferred Server setting, which controls the ALUA state of a
given path to a Host from a DataCore Server will therefore be ignored by the Host.
Note: The actual 'active' path used by the VMware Host that is using the Most Recently Used
PSP is configured on the ESX Host directly and is not controlled by the DataCore Server. Please
refer to VMware's own documentation on how to configure the 'active' path when using the
Most Recently Used PSP.
Also see the Preferred Servers section from the SANsymphony Help:
http://www.datacore.com/SSV-Webhelp/Port_Connections_and_Paths.htm
Page | 22
The Host Server – VMware ESXi Configuration Guide
Known issues
The following is intended to make DataCore Software customers aware of any issues that may
affect performance, access or generally give unexpected results under certain conditions when
VMware ESXi is used with SANsymphony.
Some of the issues here have been found during DataCore’s own testing but many others are
issues reported by DataCore Software customers, where a specific problem had been identified
and then subsequently resolved.
DataCore cannot be held responsible for incorrect information regarding VMware products. No
assumption should be made that DataCore has direct communication with VMware regarding
the issues listed here and we always recommend that users contact the VMware directly to see
if there are any updates or fixes since they were reported to us.
For ‘Known issues’ for DataCore’s own Software products, please refer to the relevant DataCore
Software Component’s release notes.
Page | 23
The Host Server – VMware ESXi Configuration Guide
Known issues – ESX 6.x
ESXi 6.0.x
ESX Hosts with IP addresses that share the same IQN connecting to the same DataCore Server
Front-end port is not supported (this also includes ESXi 'Port Binding')
Please see the ISCSI Connections section on page 11
Running Microsoft Cluster Services in a Virtual Machine on Virtual Disks that have more than
one Front End mapping to each DataCore Server may cause unexpected loss of access
This has now been fixed in https://kb.vmware.com/kb/2145663 (VMware ESXi 6.0, Patch
Release ESXi600-201608001). This was documented in VMware’s own internal
SR#15597438602.
Storage PDL responses may not trigger path failover in vSphere 6.0
http://kb.vmware.com/kb/2144657.
This affects vSphere 6.0 and 6.0 Update 1. A fix is available (as well as a workaround) from the
VMware knowledge base article above.
Unable to access filesystem for MSCS cluster nodes after vMotion
https://kb.vmware.com/kb/2144153
A fix is available (as well as a workaround) from the VMware knowledge base article above.
The SCSI-3 Persistent Reserve tests fail for Windows 2012 Microsoft Clusters running in
VMware ESXi Virtual Machines
http://kb.vmware.com/kb/1037959
This is expected see 'additional notes' under the section 'VMware vSphere support for running
Microsoft clustered configurations' in the VMware knowledge base article above.
ESXi/ESX hosts with visibility to RDM LUNs being used by MSCS nodes with RDMs may take a
long time to start or during LUN rescan
http://kb.vmware.com/kb/1016106
VHBAs and other PCI devices may stop responding when using Interrupt Remapping
http://kb.vmware.com/kb/1030265
Under ‘heavy’ load the VMFS heartbeat may fail with false ‘ATS miscompare’ message
DataCore recommend disabling the VAAI ATS heartbeat setting.
Page | 24
The Host Server – VMware ESXi Configuration Guide
Known issues – ESX 6.x
Please see 'Enabling or Disabling VAAI ATS heartbeat': http://kb.vmware.com/kb/2113956.
A change in the VMFS heartbeat update method used by VAAI was released in vSphere ESXi 5.5
Update 2 and ESXi 6.0. Previously, the heartbeat involved normal' SCSI reads and writes'
whereas the new method offloads the validation step to the storage system (i.e. the DataCore
Serve) via VAAI ATS commands. DataCore do not require (or use) this ATS heartbeat setting and
are not aware of any problems caused by disabling it on the Host; but advice should always be
sought from VMware directly. Note: While the article linked above appears to be specifically
aimed at IBM storage, DataCore Virtual Disks will behave in the same way.
Significant numbers of Virtual Machines all running in the same Virtual Disk may result in
excessive ‘SCSI reservation requests’ leading to reservation conflicts between Hosts sharing
the Virtual Disk which may lead to increased I/O latency.
This only affects ESXi Hosts not using VAAI.
Reduce the number of running Virtual Machines on a single Virtual Disk and that ESX Hosts with
the ‘closest’ IO path to the DataCore Server all access the same, shared Virtual Disk as this will
also help to reduce the potential for excessive SCSI Reservation conflicts.
Also see: 'Analyzing SCSI Reservation conflicts on VMware Infrastructure 3.x, vSphere 4.x and
vSphere 5.x (1005009)' http://kb.vmware.com/kb/1005009
DataCore Software recommends using VAAI where the 'Atomic Test and Set (ATS) primitive' is
used instead as this is much better method for locking VMFS Datastores on Virtual Disks when
compared to the normal SCSI Reservation process.
Page | 25
The Host Server – VMware ESXi Configuration Guide
Known issues – ESX 5.x
ESXi 5.x
Includes 5.0.x, 5.1.x and 5.5.x
ESX Hosts with IP addresses that share the same IQN connecting to the same DataCore Server
Front-end port is not supported (this also includes ESXi 'Port Binding').
Please see the ISCSI Connections section on page 11
Running Microsoft Cluster Services in a Virtual Machine on Virtual Disks that have more than
one Front End mapping to each DataCore Server may cause unexpected loss of access
This affects vSphere 5.5.x and has been reported in VMware’s own internal SR#15597438602.
Please contact VMware for more information.
The SCSI-3 Persistent Reserve tests fail for Windows 2012 Microsoft Clusters running in
VMware ESXi Virtual Machines
http://kb.vmware.com/kb/1037959
This is expected see 'additional notes' under the section 'VMware vSphere support for running
Microsoft clustered configurations' in the VMware knowledge base article above.
ESXi/ESX hosts with visibility to RDM LUNs being used by MSCS nodes with RDMs may take a
long time to start or during LUN rescan
http://kb.vmware.com/kb/1016106
VHBAs and other PCI devices may stop responding when using Interrupt Remapping
http://kb.vmware.com/kb/1030265
Under ‘heavy’ load the VMFS heartbeat may fail with false ‘ATS miscompare’ message
DataCore recommend disabling the VAAI ATS heartbeat setting. Please see 'Enabling or
Disabling VAAI ATS heartbeat': http://kb.vmware.com/kb/2113956.
A change in the VMFS heartbeat update method used by VAAI was released in vSphere ESXi 5.5
Update 2 and ESXi 6.0. Previously, the heartbeat involved normal' SCSI reads and writes'
whereas the new method offloads the validation step to the storage system (i.e. the DataCore
Serve) via VAAI ATS commands. DataCore do not require (or use) this ATS heartbeat setting and
are not aware of any problems caused by disabling it on the Host; but advice should always be
sought from VMware directly. Note: While the article linked above appears to be specifically
aimed at IBM storage, DataCore Virtual Disks will behave in the same way.
Page | 26
The Host Server – VMware ESXi Configuration Guide
Known issues – ESX 5.x
Significant numbers of Virtual Machines all running in the same Virtual Disk may result in
excessive ‘SCSI reservation requests’ leading to reservation conflicts between Hosts sharing
the Virtual Disk which may lead to increased I/O latency.
This only affects ESXi Hosts not using VAAI.
Reduce the number of running Virtual Machines on a single Virtual Disk and that ESX Hosts with
the ‘closest’ IO path to the DataCore Server all access the same, shared Virtual Disk as this will
also help to reduce the potential for excessive SCSI Reservation conflicts. Also see Analyzing SCSI
Reservation conflicts on VMware Infrastructure 3.x, vSphere 4.x and vSphere 5.x (1005009)
http://kb.vmware.com/kb/1005009
DataCore Software recommends using VAAI where the 'Atomic Test and Set (ATS) primitive' is
used instead as this is much better method for locking VMFS Datastores on Virtual Disks when
compared to the normal SCSI Reservation process.
Page | 27
The Host Server – VMware ESXi Configuration Guide
Known issues – ESX 4.x
ESX 4.x
Includes 4.0.x, and 4.1.x
ESX Hosts with IP addresses that share the same IQN connecting to the same DataCore Server
Front-end port is not supported (this also includes ESXi 'Port Binding').
Please see the ISCSI Connections section on page 11
ESXi/ESX hosts with visibility to RDM LUNs being used by MSCS nodes with RDMs may take a
long time to start or during LUN rescan
http://kb.vmware.com/kb/1016106
VHBAs and other PCI devices may stop responding when using Interrupt Remapping
This affects vSphere 4.1.x. http://kb.vmware.com/kb/1030265
Significant numbers of Virtual Machines all running in the same Virtual Disk may result in
excessive ‘SCSI reservation requests’ leading to reservation conflicts between Hosts sharing
the Virtual Disk which may lead to increased I/O latency.
This only affects ESXi Hosts not using VAAI.
Reduce the number of running Virtual Machines on a single Virtual Disk and that ESX Hosts with
the ‘closest’ IO path to the DataCore Server all access the same, shared Virtual Disk as this will
also help to reduce the potential for excessive SCSI Reservation conflicts. Also see Analyzing SCSI
Reservation conflicts on VMware Infrastructure 3.x, vSphere 4.x and vSphere 5.x (1005009)
http://kb.vmware.com/kb/1005009
DataCore Software recommends using VAAI where the 'Atomic Test and Set (ATS) primitive' is
used instead as this is much better method for locking VMFS Datastores on Virtual Disks when
compared to the normal SCSI Reservation process.
Hosts do not support LUNs greater than 2-terabyte
http://kb.vmware.com/kb/3371739
ISCSI Patches required for ESXi 4.0 Hosts connected to DataCore Servers
Please make sure the following patches are applied:
VMware ESXi 4.0, Patch ESXi400-200906413-BG: http://kb.vmware.com/kb/1012232
VMware ESXi 4.0, Patch ESXi400-201003401-BG: http://kb.vmware.com/kb/1019492
Page | 28
The Host Server – VMware ESXi Configuration Guide
Appendix A
Preferred Server & Preferred Path settings
See the Preferred Servers and Preferred Paths sections from the SANsymphony Help:
http://www.datacore.com/SSV-Webhelp/Port_Connections_and_Paths.htm
Without ALUA enabled
If Hosts are registered without ALUA support, the Preferred Server and Preferred Path settings
will serve no function. All DataCore Servers and their respective Front End (FE) paths are
considered ‘equal’.
It is up to the Host’s own Operating System or Failover Software to determine which DataCore
Server is its preferred server.
With ALUA enabled
Setting the Preferred Server to ‘Auto’ (or an explicit DataCore Server), determines the DataCore
Server that is designated ‘Active Optimized’ for Host IO. The other DataCore Server is
designated ‘Active Non-Optimized’.
If for any reason the Storage Source on the preferred DataCore Server becomes unavailable,
and the Host Access for the Virtual Disk is set to Offline or Disabled, then the other DataCore
Server will be designated the ‘Active Optimized’ side. The Host will be notified by both
DataCore Servers that there has been an ALUA state change, forcing the Host to re-check the
ALUA state of both DataCore Servers and act accordingly.
If the Storage Source on the preferred DataCore Server becomes unavailable but the Host
Access for the Virtual Disk remains Read/Write, for example if only the Storage behind the
DataCore Server is unavailable but the FE and MR paths are all connected or if the Host
physically becomes disconnected from the preferred DataCore Server (e.g. Fibre Channel or
iSCSI Cable failure) then the ALUA state will not change for the remaining, ‘Active Nonoptimized’ side. However, in this case, the DataCore Server will not prevent access to the Host
nor will it change the way READ or WRITE IO is handled compared to the ‘Active Optimized’
side, but the Host will still register this DataCore Server’s Paths as ‘Active Non-Optimized’ which
may (or may not) affect how the Host behaves generally.
Page | 29
The Host Server – VMware ESXi Configuration Guide
Appendix A - Preferred Server & Preferred Path settings
In the case where the Preferred Server is set to ‘All’, then both DataCore Servers are designated
‘Active Optimized’ for Host IO.
All IO requests from a Host will use all Paths to all DataCore Servers equally, regardless of the
distance that the IO has to travel to the DataCore Server. For this reason, the ‘All’ setting is not
normally recommended. If a Host has to send a WRITE IO to a ‘remote’ DataCore Server (where
the IO Path is significantly distant compared to the other ‘local’ DataCore Server), then the
WAIT times accrued by having to send the IO not only across the SAN to the remote DataCore
Server, but for the remote DataCore Server to mirror back to the local DataCore Server and
then for the mirror write to be acknowledged from the local DataCore Server to the remote
DataCore Server and finally for the acknowledgement to be sent to the Host back across the
SAN, can be significant.
The benefits of being able to use all Paths to all DataCore Servers for all Virtual Disks are not
always clear cut. Testing is advised.
For Preferred Path settings it is stated in the SANsymphony Help:
A preferred front-end path setting can also be set manually for a particular virtual disk. In this
case, the manual setting for a virtual disk overrides the preferred path created by the preferred
server setting for the host.
So for example, if the Preferred Server is designated as DataCore Server A and the Preferred
Paths are designated as DataCore Server B, then DataCore Server B will be the ‘Active
Optimized’ Side not DataCore Server A.
In a two-node Server group there is usually nothing to be gained by making the Preferred Path
setting different to the Preferred Server setting and it may also cause confusion when trying to
diagnose path problems, or when redesigning your DataCore SAN with regard to Host IO Paths.
Where there are three or more Servers in a Server Group, and where one or more of these
DataCore Servers shares Mirror Paths between different DataCore Servers then setting the
Preferred Path makes more sense. So for example, DataCore Server A has two mirrored Virtual
Disks, one with DataCore Server B, and one with DataCore Server C and DataCore Server B also
has a mirrored Virtual Disk with DataCore Server C then using just the Preferred Server setting
to designate the ‘Active Optimized’ side for the Host’s Virtual Disks becomes more complicated.
In this case the Preferred Path setting can be used to override the Preferred Server setting for a
much more granular level of control.
Page | 30
The Host Server – VMware ESXi Configuration Guide
Appendix B
Configuring Disk Pools
See Creating Disk Pools and Adding Physical Disks from the SANsymphony Help:
http://www.datacore.com/SSV-Webhelp/About_Disk_Pools.htm
The smaller the SAU size, the larger the number of indexes are required, by the Disk Pool driver,
to keep track of the equivalent amount of allocated storage compared to a Disk Pool with a
larger SAU size; e.g. there are potentially four times as many indexes required in a Disk Pool
using a 32MB SAU size compared to one using 128MB – the default SAU size.
As SAUs are allocated for the very first time, the Disk Pool needs to update these indexes and
this may cause a slight delay for IO completion and might be noticeable on the Host. However
this will depend on a number of factors such as the speed of the physical disks, the number of
Hosts accessing the Disk Pool and their IO READ/WRITE patterns, the number of Virtual Disks in
the Disk Pool and their corresponding Storage Profiles.
Therefore, DataCore usually recommend using the default SAU size (128MB) as it is a good
compromise between physical storage allocation and IO overhead during the initial SAU
allocation index update. Should a smaller SAU size be preferred, the configuration should be
tested to make sure that a potential increased number of initial SAU allocations does not
impact the overall Host performance.
Page | 31
The Host Server – VMware ESXi Configuration Guide
Appendix C
Reclaiming storage
Using VMware’s ‘Block Delete/SCSI UNMAP’ VAAI primitive
As of SANsymphony 9.0 PSP4 there is support for the Block Delete/SCSI UNMAP VAAI primitive
which, when used in conjunction with either the VMware ESXi vmkfstools or esxcli command
(depending on the version of ESXi used), allows Hosts to trigger the Automatic Reclamation
function on served Virtual Disks:
For ESXi 5.0.x and 5.1.x Hosts:
Using vmkfstools to reclaim VMFS deleted blocks on thin-provisioned LUNs (2014849)
http://kb.vmware.com/kb/2014849
For ESXi 5.5.x Hosts:
vSphere 5.5 Command Line Documentation > vSphere Command-Line Interface
Documentation > vSphere Command-Line Interface Concepts and Examples > Managing Files
> Reclaiming Unused Storage Space
https://pubs.vmware.com/vsphere55/topic/com.vmware.vcli.examples.doc/cli_manage_files.5.6.html
For ESXi 6.x Hosts:
vSphere 6.0 Command Line Documentation > vSphere Command-Line Interface
Documentation > vSphere Command-Line Interface Concepts and Examples > Managing Files
> Reclaiming Unused Storage Space
https://pubs.vmware.com/vsphere60/topic/com.vmware.vcli.examples.doc/cli_manage_files.5.6.html
Or alternatively, see:
Using esxcli in vSphere 5.5 and 6.0 to reclaim VMFS deleted blocks on thin-provisioned LUNs
(2057513) http://kb.vmware.com/kb/2057513
Page | 32
The Host Server – VMware ESXi Configuration Guide
Appendix C – Reclaiming storage
Automatic Reclamation
Since SANsymphony 9.0 PSP4 the DataCore Server will, in the background, continuously scan
each Physical Disk in a Pool for any SAUs that contain all-zero data which are then reclaimed
back into the Disk Pool. The Automatic Reclamation process runs at a low priority, so as to not
potentially interfere with overall Disk Pool performance, and so will generally take longer to
scan a Physical Disk in a Pool compared to a Manual Reclamation request (see below).
No user intervention is required.
Manual Reclamation
For any VMware ESXi Hosts 4.1 or earlier or any VMware ESXi Hosts connected to DataCore
Servers running SANsymphony 9.0 PSP3 update2 or earlier (including all versions of
SANsymphony 8.x) a suggestion would be to create a new, ‘dummy’ virtual machine of an
appropriate size (If there is enough free space available in the VMFS) and then zero-fill it using
the vmkfstools ‘eagerzeroedthick’ option:
vmkfstools -c [size] -d eagerzeroedthick /vmfs/volumes/[mydummydir]/[mydummy.vmdk]
Or if using VMware’s own UI, format using the ‘Thick Provision Eager Zeroed’ option. Refer to
VMware’s own documentation on how to do this. Once the formatting has completed, delete
the dummy virtual machine. Then either wait for an Automatic Reclamation to take place or run
a Manual Reclamation.
See the Performing Reclamation section from the SANsymphony Help:
http://www.datacore.com/SSV-Webhelp/Reclaiming_Virtual_Disk_Space.htm
Note that it is also possible to script manual reclamation using the StartDcsVirtualDiskReclamation PowerShell Command.
Using Raw Device Mapped Virtual Disks
For Virtual Machines which directly access Virtual Disks via VMware ESXi’s ‘Raw Device
Mapping’ technology, it may be possible to zero-fill using the Virtual Machine’s own file system
using third party tools (i.e. sdelete for Microsoft Windows NTFS/FAT File systems or using the
dd command for UNIX/Linux File systems) without having to create a ‘dummy’ virtual machine.
Page | 33
The Host Server – VMware ESXi Configuration Guide
Appendix C – Reclaiming storage
How much storage will be reclaimed?
It is impossible to predict exactly how many Storage Allocation Units (SAUs) will be reclaimed.
For reclamation of an SAU to take place, it must contain only ‘all-zero’ block data over the
entire SAU else it will remain allocated and this is entirely dependent on how and where the
Host has written its data on the DataCore LUN.
For example, if the Host has written the data in such a way that every allocated SAU contains a
small amount of non-zero block data, even if the total amount of data is significantly less than
the total amount of SAU allocation, then no SAUs can be reclaimed.
It may be possible, in some cases, to use the Host Operating System’s own defragmentation
tools to force the non-zero block data to be moved to a more contiguous pattern, so leaving the
‘end’ of the DataCore LUN full of SAUs that no longer have non-zero data on them that can then
be reclaimed. However care should be taken that the act of defragmenting the data does not
itself cause more SAU allocation as the block data is moved around during the re-organization.
DataCore Software can offer no guarantees
Page | 34
The Host Server – VMware ESXi Configuration Guide
Appendix D
Moving from ‘Most Recently Used’ to either ‘Round
Robin’ or ‘Fixed’ Path Selection Policy
This will require Virtual Disks to be unserved from the Host and so planning for the appropriate
downtime will be needed if Virtual Machines need to be stopped. It may be possible to move
Virtual Machines over to different Hosts (using vMotion for example) before the Virtual Disks
are unserved, to allow changing the PSP for one Host at a time but please refer to VMware
directly regarding mixing different PSPs on different VMware ESXi Hosts for the same LUN
before attempting this in case this is not appropriate to your VMware configuration.
1. Unserve all Virtual Disks from the Host from within the SANsymphony Console.
2. At the VMware ESXi Host rescan all disk devices so that the DataCore Virtual Disks are
removed and then remove the Storage Array Type Claim Rule and Storage Array as
described on page 21.
3. From within the SANsymphony Console enable the ALUA option on the Host. See
Changing multipath or ALUA support settings for hosts from the SANsymphony Help:
http://www.datacore.com/SSV-Webhelp/Multipath_Support.htm
4. Re-serve all Virtual Disks to the Host from within the SANsymphony Console. Note you
may need to use the same LUN number and Initiator/Target paths as before.
5. At the VMware ESXi Host rescan to re-detect the Virtual Disks with ALUA enabled and
proceed to the appropriate page in this document for either Fixed or Round Robin Path
Selection Policy.
Note: To move from Fixed or Round Robin to Most Recently Used uses the same steps as above
but the ALUA option must be unchecked in Step #3.
Remember that no versions of VMware ESXi have been qualified with SANsymphony without the
ALUA option set, and so the Most Recently Used PSP is considered ‘unqualified’ by DataCore.
Please refer to page 4 regarding all unqualified configurations of VMware ESXi Hosts.
Page | 35
The Host Server – VMware ESXi Configuration Guide
Previous Changes
2016
August
Added
Known Issues
ESXi/ESX hosts with visibility to RDM LUNs being used by MSCS nodes with RDMs may take a long time to start or
during LUN rescan
Applies to ESX 6.x, 5.x and 4.x
Please see: http://kb.vmware.com/kb/1016106
July
Added
The DataCore Server's settings
Added link:
Video: Configuring ESX Hosts in the DataCore Management Console
http://datacore.custhelp.com/app/answers/detail/a_id/1637
Updated
This document has been reviewed for SANsymphony 10.0 PSP 5.
VMware ESXi compatibility lists
ESX 4.1 is now 'not supported' for SANsymphony 10.x – previously listed as 'unqualified'.
Because ESX 4.x is considered by VMware to be 'End of Availability' (https://kb.vmware.com/kb/2039567)
DataCore would not be able to get assistance from VMware if it were needed for any issues that were found
during 'Self Qualification'.
Known Issues
vMotion causing loss of access to filesystem for MSCS cluster nodes (2144153)
This was previously listed as "Running Microsoft Cluster Services in a Virtual Machine on Virtual Disks that have
more than one Front End mapping to each DataCore Server may cause unexpected loss of access". A
Knowledgebase article has now been released by VMware https://kb.vmware.com/kb/2144153
April
Updated
Known Issues - VMware 6.0
Storage PDL responses may not trigger path failover in vSphere 6.0
http://kb.vmware.com/kb/2144657.
Note: This affects both vSphere 6.0 and 6.0 U1 customers. A fix is available in 6.0 U2.
February
Updated
List of qualified VMware Versions - Qualification notes on VMware-specific functions
Removed references specific to 'End of Life' versions of SANsymphony-V – this includes all versions of
SANsymphony-V 8.x and any version of 9.x that are PSP 3 or earlier.
2015
December
Updated
List of qualified VMware Versions - Qualification notes on VMware-specific functions
Path Selection Policies and VMware ESX 6.x
Page | 36
The Host Server – VMware ESXi Configuration Guide
Previous changes
For ESX 6.x, Fixed and Round Robin Path Selection Policies are both tested and supported by DataCore and both
are also listed on VMware's own Hardware Compatibility List.
VSphere APIs for Storage Awareness (VASA)
For ESX 6.x, VASA is tested and supported by DataCore and is also listed in VMware's own Compatibility Guide.
VSphere APIs for Virtual Volumes (VVOL)
For ESX 6.x, VVOL is tested and supported by DataCore and is also listed in VMware's own Compatibility Guide.
November
Updated
SANsymphony-V 8.x and all versions of SANsymphony 9.x before PSP4 Update 4 are now ‘End of Life’. Please see:
End of Life Notifications http://datacore.custhelp.com/app/answers/detail/a_id/1329
October
Updated
Known Issues – VMware ESXi 5.x and 6.x
DataCore have been informed that there is now a ‘hotfix’ from VMware for the previously documented known
issue “Running Microsoft Cluster Services in a Virtual Machine on Virtual Disks that have more than one Front End
mapping to each DataCore Server may cause unexpected loss of access” (VMware’s own SR#15597438602).
Contact VMware for more information.
July
Added
List of qualified VMware ESXi Versions - Notes on qualification
This section has been updated and new information added regarding the definitions of all ‘qualified’, ‘unqualified’
and ‘not supported…’ labels. A new section on Linux distributions that are no longer in development has also been
added at the end of this section.
Known Issues
Moved some of the information from the Host Configuration section where problems can arise into the ‘Known
Issues’ section. The ISCSI Port Binding is no longer considered supported as even if configured to be used in
different subnets (as previously recommended) the sharing of IQNs for different iSCSI Initiators on the ESXi Hosts
cannot be avoided and this can lead to situations where different IP Addresses with the same IQN try to log into
the same DataCore FE Port and will not be able to. Please read the ‘Known Issues’ section for more detail.
May
Added
Known Issues – VMware ESXi 5.x and 6.x
An issue has been identified by VMware regarding Microsoft Clusters in Virtual Machines using SANsymphony-V
Virtual Disks served to more than one path on the same ESX host, which lead to unexpected loss of access.
Under ‘heavy’ load the new VMFS heartbeat process used by ESX 5.5. Update 2 and 6.x may fail with false ‘ATS
miscompare’ message.
Updated
Sections that apply to VMware ESXi 6.x have been explicitly labelled to avoid ambiguity.
April
Added
VMware ESXi Path Selection Policies (all)
Page | 37
The Host Server – VMware ESXi Configuration Guide
Previous changes
It has been observed that different versions of ESXi may or may not ‘auto configure’ the correct SATP claim rule for
Round Robin or Fixed Path Selection Policies when presented with a Virtual Disks from SANsymphony-V. Therefore
more explicit instructions on how to create a custom rules has been added.
Note: Existing SANsymphony-V installations probably do not need to worry about this new information as it does
not conflict with what was stated previously; but DataCore recommend that you review the section just to make
sure that your Virtual Disks are correctly configured.
Updated
List of qualified VMware ESXi Versions
Added VMware ESXi 6.x
Host Settings - VMware ESXi All versions
ISCSI Port binding
Clarified the statement regarding using same subnets as the VMKERNEL port.
Configuring VMware ESXi Path Selection Policies
General Notes – this section has been re-ordered. No new information has been added.
2014 and earlier
December
Added
DataCore Server Settings
Installing a DataCore Server ‘inside’ a VMware ESXi Virtual Machine
VMware ESXi Path Selection Policies
Which Path Selection Policy does DataCore Software Recommend?
Added some explanation on a frequently asked question based on the differences between Fixed and Round Robin
Path Selection Policies.
The ‘Preferred Server’ setting when using the … PSP
Added more detailed explanation regarding the SANsymphony-V ‘Preferred Server’ setting and how it applies to
each of the three supported Path Selection Policies (Round Robin, Fixed and MRU).
Updated
Appendix D - Moving from Most Recently Used to Round Robin or Fixed Path Selection Policies
Added more information about how to reduce the likelihood for downtime (by using vMotion).
November
Added
Known Issues
Most of the information was moved from the ‘Known Issues: Third Party Hardware and Software’ document:
http://datacore.custhelp.com/app/answers/detail/a_id/1277
Updated
List of qualified VMware ESXi versions
‘Not Supported’ has now been changed to mean explicitly ‘Not Supported for Mirrored or Dual Virtual Disks’.
Single Virtual Disks are now always considered supported.
Appendix B: Reclaiming Storage from Disk Pools
Page | 38
The Host Server – VMware ESXi Configuration Guide
Previous changes
For ESXi 5.5 Hosts, the command to reclaim VMFS deleted blocks has changed since earlier versions of ESXi 5.x. A
link to the appropriate VMware KB article for the later version of ESXi has therefore been added.
July
Updated
VMware ESXi Path Selection Policies – all types
The command to verify that a given SATP type had been set was incorrect for the later versions of VMware ESXi. It
was listed as:
esxcli nmp satp listrules -s [SATP_Type]
and should have been listed as:
esxcli storage nmp satp rule list -s [SATP_Type]
VMware ESXi Path Selection Policies – Fixed
Added clarifying notes at the start of this section, as the specific requirements for the Host Settings within the
SANsymphony-V Management Console, using the Fixed Path Selection Policy with VMware ESXi, contradict the
general statement (for all other Host Operating Systems) in the SANsymphony-V Release Notes regarding use the
‘All’ setting for the Preferred Server setting.
June
Updated
List of qualified VMware ESXi Versions
Updated to include SANsymphony-V 10.x
May
This document combines all of DataCore’s VMware information from older Technical Bulletins into a single
document including:
Technical Bulletin 5b: “VMware ESXi vSphere 4.0.x Hosts”.
Technical Bulletin 5c: “VMware ESXi vSphere 4.1.x Hosts”.
Technical Bulletin 5d: “VMware ESXi vSphere 5.x Hosts”.
Note: Technical Bulletin 5a: “VMware ESXi 2.x and 3.x Hosts” contains versions not supported with SANsymphonyV, so the information is not relevant to this document and has not been included.
Technical Bulletin 8: “Formatting Host’s File Systems on Virtual Disks created from Disk Pools”.
Technical Bulletin 11: “Disk Timeout Settings on Hosts”.
Technical Bulletin 16: “Reclaiming Space in Disk Pools”.
Added
Host Settings: VMware ESXi All Versions:
Notes on VMware iSCSI Port Binding
VMware ESXi Path Selection Policies:
‘Fixed AP’ is no longer included as this is not a supported Path Selection Policy with SANsymphony-V.
‘Fixed’ is supported (this was inconsistently documented across the different Technical Bulletins) but only with the
Preferred Server setting set to ‘All’.
‘Most Recently Used’ must only be used without the ALUA option set on the Host. However, no versions of
VMware ESXi, without the ALUA option set, have been qualified with SANsymphony-V, so this Path Selection Policy
is considered ‘unqualified’.
Page | 39
The Host Server – VMware ESXi Configuration Guide
Previous changes
Appendix A: This section gives more detail on the Preferred Server and Preferred Path settings with regard to how
it may affect a Host.
Appendix B: This section incorporates information regarding “Reclaiming Space in Disk Pools” (from Technical
Bulletin 16) that is specific to VMware Hosts.
Appendix C: This section adds additional information regarding “VMware’s vStorage APIs for Array Integration
(VAAI) with SANsymphony-V”.
Appendix D: This section adds more comprehensive steps for “Moving from Most Recently Used to Fixed or Round
Robin Path Selection Policy”.
Updated
DataCore Server Settings: VMware ESXi 4.0.x Hosts: Regarding Virtual Disk Names.
Host Settings: SCSI Reservation locking between VMware ESXi Hosts.
VMware ESXi Path Selection Policies: Previously the Preferred Server setting of ‘All’ was explicitly stated to not be
used within the SANsymphony-V Management Console. However, ‘Fixed’ requires that the Host’s Preferred Server
setting is set to ‘All’. ‘Round Robin’ may use the ‘All’ setting although caution is advised and more explanation is
provided in Appendix A why it may not be advisable.
An overall improvement of the explanations to most of the required Host Settings and DataCore Server Settings.
Technical Bulletin 5d: “VMware ESXi vSphere 5.x Hosts”
January 2014
Updated
The note on how to move from ‘Most Recently Used’, with the ALUA option not checked to ‘Fixed’/RR Path with
the ALUA option checked for a DataCore Disk with regard to SANsymphony-V 9.0 PSP3 and later versions.
December 2013
Added
VSphere ESXi 5.5 is qualified and no additional settings (from all previous 5.x versions) are needed. The SCSI
UNMAP primitive is supported from SANsymphony-V 9.0 PSP4.
Updated
DataCore Server configuration settings section (‘Virtual Disks mapped to more than one Host may need to use the
same LUN ‘number’ …’) for SANsymphony-V. Added a ‘warning’ note at the start of each Path Selection Policy
(PSP), cautioning the user, that a VM’s Operating System configuration may not be supported by VMware for a
particular PSP (i.e. of publication VMware state that MSCS VMs are not supported for Round Robin PSP).
April 2013
Removed
All references to SANmelody as this product is now ‘End of Life’ as of December 31, 2012
March 2013
Added
Use VMFS5 for VSphere Metro Storage Clusters (vMSC).
February 2013
Updated
The ‘General notes on path selection policies’. To allow for different behavior with the VMware vCenter
Integration function of SANsymphony-V.
Page | 40
The Host Server – VMware ESXi Configuration Guide
Previous changes
October 2012
Removed
All but one of the Advanced Settings; all other settings are no longer needed and can be ignored (there is no
requirement to reset or change the existing values for these other settings and they can be left as they are).
July 2012
Added
support for SANsymphony-V 9.x, no new technical information. Added extra steps to set the default path selection
policy to ‘Fixed’ instead of ‘MRU’ under the ‘Fixed/Round Robin path selection policy’ section. Added note under
‘General’ section that:
i.
VAAI is now supported - with SANsymphony-V 9.x and ESXi 5.x.
ii.
Strengthened warning that MRU is not supported with ALUA
June 2012
Added
Two new settings to be applied under the ‘General’ section for the Hosts (Disk.UseLunReset Disk.UseDeviceReset).
May 2012
Updated
The DataCore Server and Host minimum requirements.
Removed
All references to ‘End of Life’ versions that are no longer supported as of December 31 2011. Updated notes at the
start of ‘General notes for Path Selection Policies’. Updated copyright. Added note to ‘General notes on path
selection policies for ESXi 5.x’ on selecting the preferred path of Virtual Disk with multiple connections for
VMW_PSP_FIXED to the same DataCore Server.
December 2011
Initial publication of Technical Bulletin.
Technical Bulletin 5c: “VMware ESXi vSphere 4.1.x Hosts”
June 2013
Added
A ‘warning’ note at the start of each Path Selection Policy (PSP), cautioning the user, that a VM’s Operating System
configuration may not be supported by VMware for a particular PSP (i.e. of publication VMware state that MSCS
VMs are not supported for Round Robin PSP).
April 2013
Removed
All references to SANmelody as this is now ‘End of Life’ of December 31 2012. Updated the DataCore Server
Configuration Settings added Preferred Server notes.
July 2012
Added
Support for SANsymphony-V 9.x. No new settings required. Added notes under ‘General’ section that:
i.
VAAI is not supported with SANsymphony-V and ESXi 4.1.
ii.
Strengthened warning that MRU is not supported with ALUA
June 2012
Added
Two new settings to be applied under the ‘General’ section for the Hosts (Disk.UseLunReset Disk.UseDeviceReset).
Page | 41
The Host Server – VMware ESXi Configuration Guide
Previous changes
May 2012
Updated
The DataCore Server and Host minimum requirements. Removed all references to ‘End of Life’ SANsymphony and
SANmelody versions that are no longer supported as of December 31 2011. Added notes at the start of ‘General’
notes for Path Selection Policies. Updated copyright. Updated ‘Fixed AP and Round Robin Path Selection Policy’
with regard to ‘preferred paths’. Existing users should re-check their configurations and make any appropriate
changes as necessary.
November 2011
Updated
URL to VMware SAN Configuration guides changed.
October 2011
Removed
All references to ‘End of Life’ SANsymphony and SANmelody versions that are no longer supported as of July 31
2011. Moved known issues out of this Technical Bulletin and into the ‘Known Issues: Third Party
Software/Hardware with DataCore Servers’ document. Added MRU path policy. Added important note on how to
verify path selection policy in each case. For SANsymphony-V the first 12 characters of the Virtual Disk name no
longer needs to be unique.
February 2011
Added
Support for SANsymphony-V 8.x.
September 2010
Initial publication of Technical Bulletin.
Technical Bulletin 5b: “VMware ESXi vSphere 4.0.x Hosts”
June 2013
Added
A ‘warning’ note at the start of each Path Selection Policy (PSP), cautioning the user, that a VM’s Operating System
configuration may not be supported by VMware for a particular PSP (i.e. of publication VMware state that MSCS
VMs are not supported for Round Robin PSP).
April 2013
Removed
All references to SANmelody as this product is now ‘End of Life’ as of December 31, 2012
July 2012
Added
Support for SANsymphony-V 9.x. No new settings required. Corrected option for SCSI.CRTimeoutDuringBoot and
added back SCSI.ConflictRetries in ESXi(i) Host configuration settings - General.
June 2012
Added
Two new settings to be applied under the ‘General’ section for the Hosts (Disk.UseLunReset Disk.UseDeviceReset).
November 2011
Updated
URI to VMware SAN Configuration guides changed.
Page | 42
The Host Server – VMware ESXi Configuration Guide
Previous changes
October 2011
Removed
All references to ‘End of Life versions that are no longer supported as of July 31 2011. Moved all issues not specific
to configuring Hosts or DataCore Servers out of this Technical Bulletin and into the ‘Known Issues: Third Party
Software/Hardware with DataCore Servers’ document. Added important note on how to verify path selection
policy in each case. Changed requirement for Most Recently Used managed path policy – do not use the ‘ALUA’
option.
March 2011
Added
Support for SANsymphony-V 8.x
June 2010
Added
Support for 'Round-Robin' path selection policy with SANsymphony 7.0 PSP 3 Update 4 and SANmelody 3.0 PSP 3
update 4.
December 2009
Added
Support for 'Fixed Path' path selection policy with SANsymphony 7.0 PSP 3 and SANmelody 3.0 PSP 3. Previously
only MRU was supported
October 2009
Initial publication of Technical Bulletin
Page | 43
The Host Server – VMware ESXi Configuration Guide
COPYRIGHT
Copyright © 2016 by DataCore Software Corporation. All rights reserved.
DataCore, the DataCore logo and SANsymphony are trademarks of DataCore Software Corporation. Other DataCore product or service names
or logos referenced herein are trademarks of DataCore Software Corporation. All other products, services and company names mentioned
herein may be trademarks of their respective owners.
ALTHOUGH THE MATERIAL PRESENTED IN THIS DOCUMENT IS BELIEVED TO BE ACCURATE, IT IS PROVIDED “AS IS” AND USERS MUST TAKE ALL
RESPONSIBILITY FOR THE USE OR APPLICATION OF THE PRODUCTS DESCRIBED AND THE INFORMATION CONTAINED IN THIS DOCUMENT.
NEITHER DATACORE NOR ITS SUPPLIERS MAKE ANY EXPRESS OR IMPLIED REPRESENTATION, WARRANTY OR ENDORSEMENT REGARDING, AND
SHALL HAVE NO LIABILITY FOR, THE USE OR APPLICATION OF ANY DATACORE OR THIRD PARTY PRODUCTS OR THE OTHER INFORMATION
REFERRED TO IN THIS DOCUMENT. ALL SUCH WARRANTIES (INCLUDING ANY IMPLIED WARRANTIES OF MERCHANTABILITY, NONINFRINGEMENT, FITNESS FOR A PARTICULAR PURPOSE AND AGAINST HIDDEN DEFECTS) AND LIABILITY ARE HEREBY DISCLAIMED TO THE
FULLEST EXTENT PERMITTED BY LAW.
No part of this document may be copied, reproduced, translated or reduced to any electronic medium or machine-readable form without the
prior written consent of DataCore Software Corporation