Technical white paper Implementing Microsoft multisite clustering using HP 3PAR Peer Persistence Table of contents Introduction .................................................................................................................................................................................... 3 Terminology ................................................................................................................................................................................... 3 Features and benefits .................................................................................................................................................................. 3 Requirements ................................................................................................................................................................................. 4 Host persona 15 ........................................................................................................................................................................ 4 Verify each VV has same WWN on both arrays .................................................................................................................. 5 Verify VLUN status on both arrays ........................................................................................................................................ 5 Quorum Witness ........................................................................................................................................................................ 5 Planned switchover ...................................................................................................................................................................... 6 HP 3PAR Peer Persistence failures handling .......................................................................................................................... 6 Array-to-array communication failure ................................................................................................................................. 6 Single site to QW communication failure ............................................................................................................................. 6 Site 1 to QW and array-to-array communication failure.................................................................................................. 6 Site 2 to QW and array-to-array communication failure.................................................................................................. 6 Site 1 and site 2 both lose communications to QW ........................................................................................................... 7 Site 1 and site 2 to QW and array-to-array communication failure .............................................................................. 7 Multisite Windows Server Failover Cluster and HP 3PAR Peer Persistence considerations ......................................... 7 WSFC quorum model ............................................................................................................................................................... 7 Cluster validation test .............................................................................................................................................................. 8 Networking consideration ....................................................................................................................................................... 8 Tuning of cluster heartbeat settings .................................................................................................................................... 8 Microsoft MPIO DSM ................................................................................................................................................................. 8 HP 3PAR Quorum Witness hosting ....................................................................................................................................... 8 Remote Copy replication links ................................................................................................................................................ 8 HP 3PAR Peer Persistence maximums ................................................................................................................................ 9 Sample Peer Persistence configuration ................................................................................................................................... 9 Deploying Quorum Witness .................................................................................................................................................... 9 Requirements of Quorum Witness ........................................................................................................................................ 9 QW installation .......................................................................................................................................................................... 9 Creating Remote Copy targets ............................................................................................................................................. 10 Technical white paper | Implementing Microsoft multisite clustering using HP 3PAR Peer Persistence Peer Persistence configuration ............................................................................................................................................ 12 Troubleshooting Witness Quorum Communication issues ........................................................................................... 13 Creating Remote Copy groups ............................................................................................................................................. 14 Changing Group Policy ........................................................................................................................................................... 15 Exporting LUNs to hosts ........................................................................................................................................................ 17 Planned switchover example ............................................................................................................................................... 22 Recovering from automatic failover ................................................................................................................................... 27 For more information ................................................................................................................................................................. 34 Technical white paper | Implementing Microsoft multisite clustering using HP 3PAR Peer Persistence Introduction HP 3PAR StoreServ Peer Persistence software enables HP 3PAR StoreServ systems located in different data centers at metropolitan distances to act as peers to each other, presenting nearly continuous storage system access to hosts connected to them. This capability allows users to configure a high-availability solution between two sites or data centers where storage access, failover, and failback remain completely transparent to the hosts and applications running on those hosts. Compared to traditional cluster failover models where, upon failover, the applications must be restarted, Peer Persistence software allows hosts to remain online—serving their business applications even when they switch storage access from their original site to the storage array on the remote site, resulting in a much improved recovery time. This paper provides information about deploying a Microsoft® multisite cluster using Windows Server ® Failover Clustering (WSFC) across two data centers or sites with HP 3PAR StoreServ Peer Persistence. A Windows® multisite cluster configuration is designed to maintain data availability beyond a single physical or logical site failure. Terminology Throughout this paper, we identify the volume that is part of a primary Remote Copy group as the source volume. The replicated volume in the secondary Remote Copy group is called the target volume. Volumes created on an HP 3PAR StoreServ are called virtual volumes, or VVs. A VV exported to the host is known as a virtual logical unit number (VLUN). A zoning operation creates a logical connection in the Fibre Channel (FC) SAN between an FC host bus adapter (HBA) on a server and one on a storage system. Features and benefits Peer Persistence is a high-availability configuration between two data centers in which Windows hosts are set up in a multisite cluster configuration with access to storage arrays in both sites. Storage volumes created on one storage array are replicated to the other array using synchronous Remote Copy to make sure that the volumes are in sync at all times. Peer Persistence software takes advantage of the Asymmetric Logical Unit Access (ALUA) capability that allows paths to an SCSI device to be marked as having different characteristics. With ALUA, the same LUN can be exported from both arrays simultaneously, but only the paths to the side accepting write to the volume will be marked as active. The paths to the secondary side volume will be marked as standby preventing the host from performing any I/O using those paths. In the event of a non-disruptive transparent failover scenario, the standby paths are marked as active, and host traffic to the primary storage array is redirected to the secondary storage array without impact to the hosts. Figure 1 shows a Peer Persistence configuration. The Peer Persistence software supports bi-directional replication, which enables each site to perform as a disaster recovery center for the other. It enables users to move their applications from one site to another based on their business and performance needs without impact on the applications running on those hosts. An example would be workload mobility using Microsoft Hyper-V Live Migration in a Hyper-V deployment under Windows Server Failover Cluster. Peer Persistence provides virtual machine (VM) mobility across data centers that are used with Windows Server Failover Clustering to enable multisite storage high availability. In figure 1, certain VMs are being serviced by an HP 3PAR StoreServ Storage system on site 1 while other VMs are being serviced by another HP 3PAR StoreServ Storage system at site 2 located within a metropolitan distance. Hyper-V Live Migration allows customers to move VMs across sites. Peer Persistence software presents the VMs with the same virtual volume after it moves across data centers. In other words, movement of VMs across data centers becomes completely transparent to the applications those VMs are running. Peer Persistence achieves automatic transparent failover in the event of a complete array or data center failure with the help of the HP 3PAR Quorum Witness (QW) software. The HP 3PAR Quorum Witness software needs to be installed at a third site and will act as an independent witness should one of the arrays or data centers become unavailable. 3 Technical white paper | Implementing Microsoft multisite clustering using HP 3PAR Peer Persistence Figure 1. Peer Persistence setup with a Microsoft multisite failover cluster configuration Requirements • Firmware version on both HP 3PAR StoreServ arrays must be 3.2.1 or newer. • The sites must be set up in a Remote Copy 1-to-1 configuration with synchronous mode. • The round-trip latency between the two sites must be 2.6 milliseconds or less. • Quorum Witness software must be installed on a virtual machine at a third site. • The HP 3PAR StoreServ arrays communicate with Quorum Witness via the management port over TCP/IP. Quorum Witness needs to be reachable from both sites. • This non-disruptive failover configuration is supported with Windows Server 2008 R2, and Windows Server 2012 R2. • All source and target volumes exported from the two HP 3PAR StoreServ arrays must have the same volume Worldwide Name (WWN). • Hosts have to be connected to the HP 3PAR StoreServ Storage system using FC, iSCSI, or FCoE. • All associated hosts are connected to both arrays. • The Windows hosts that these volumes are exported to must be configured with host persona 15. It supports host capability ALUA configuration. • Both auto_failover and path_management polices are required to enable automatic transparent failover. For the latest supported configuration and requirement for HP 3PAR Peer Persistence in a vSphere environment, refer to the Single Point of Connectivity Knowledge (SPOCK). Host persona 15 Host personas are a set of behaviors that permit hosts connected to FC or iSCSI ports on the system to deviate from the default host behavior. By assigning a persona to a host, multiple host types that require distinct customized responses can share a single physical port on HP 3PAR StoreServ. For example, hosts running Windows, Linux®, and AIX operating systems can all connect to the same HP 3PAR StoreServ port. This simplifies connecting hosts to the system and reduces management costs related to complex host connections. In all versions since HP 3PAR OS 3.1.3, HP requires host persona 15 for Windows Server 2008, 2008 R2, 2012, or 2012 R2 hosts. Persona 15 supports ALUA path management and it must be used for host configurations that have Windows 2008 R2 and Windows 2012 R2 hosts connected to both the primary and the secondary arrays in an HP 3PAR Peer Persistence configuration. Existing Windows hosts defined with host persona other than host persona 15, such as persona 2, must be changed to persona 15. Refer to the HP 3PAR Windows Server 2012 and Windows 2008 implementation guide for more details about host creation and persona changes for those platforms. 4 Technical white paper | Implementing Microsoft multisite clustering using HP 3PAR Peer Persistence Verify each VV has same WWN on both arrays It is important that the source and the target volumes on the HP 3PAR StoreServ arrays have the same identity when both volumes are exported to the same Windows host. Hence, both volumes should have the same WWN. User can create volumes with the same WWN automatically using the new option in admitrcopyvv command. See the HP 3PAR StoreServ Remote Copy software user’s guide at HP Storage Library for instructions on how to use the admitrcopyvv command. The HP 3PAR Management Console (MC) also creates VVs with the same WWN when configuring Remote Copy groups. The replicated volume WWN can also be changed later if the volumes were created using the Management Console or any method other than the admitrcopyvv. See the HP 3PAR StoreServ Command Line Interface administrator’s manual at HP Storage Library or the HP 3PAR StoreServ Management Console online help for instructions on how to edit a VV’s WWN. Verify VLUN status on both arrays Make sure that volumes are exported to the host from both HP 3PAR StoreServ arrays. After the volumes are exported from both arrays, the showvlun command will show the state as “active” for VLUNs exported from the source side and “standby” for the VLUNs exported from the target side. Windows hosts will send I/O requests only to the VLUNs reported as active. Quorum Witness The HP 3PAR StoreServ Quorum Witness (QW) software enables transparent automatic failover between the two sites in a Microsoft multisite cluster configuration. The Quorum Witness software gets regularly updated with status information from the HP 3PAR StoreServ arrays. In the event of a failure of one of the HP 3PAR StoreServ arrays, the surviving HP 3PAR StoreServ array detects that Quorum Witness is not getting updated by the failed array and initiates a failover operation on any secondary groups associated with that failed array. Note QW details have to be set up at the target level. The automatic transparent failover will occur only for Remote Copy groups that use the target and have the “auto_failover” policy set. Failover is not automatic for Remote Copy groups between the same two HP 3PAR StoreServ systems that do not have the “auto_failover” policy set. The setting of path management is required as well. A volume group must have the path management policy set if it is to be valid for automatic and manual transparent failover. If these policies of “auto_failover” and path_management are set, then no automatic failover can occur if a disaster strikes. Quorum Witness is set up on a third site where events that may impact site 1 or site 2 cannot impact the Quorum Witness site at the same time. Additionally, QW connects to arrays on both sites using non-RC links. With the above two configuration characteristics (site and link independence), QW helps determine the following nature of failures: • RC link failure: QW can detect if the two arrays are alive but not communicating because of an RC link failure. QW would still be receiving updates from both of the arrays. • Array or site failure: QW can detect when one of the arrays or sites fails. QW would not be receiving updates from one of the arrays that has failed. The “Handling failures” section later in this paper covers the various failure conditions handled by Peer Persistence. The Quorum Witness software is packaged as an Open Virtualization Format (OVF) package for deployment in a vSphere environment or a Hyper-V environment. The Quorum Witness virtual machine should not be stored on allocated storage from either of the arrays that are part of the Peer Persistence configuration. If the volume is located on one of the arrays that is part of the Peer Persistence configuration, failure of the array will make the volume unavailable, which will result in the loss of access to the Quorum Witness virtual machine. Quorum Witness is a passive component of the configuration and the software itself does not provide any high availability. However, the virtual machine can be made highly available by using VMware® High Availability or Fault Tolerance, or Microsoft Failover Clustering for the Hyper-V based QW package. For the latest supported configuration and requirement for HP 3PAR Peer Persistence QW, refer to the Single Point of Connectivity Knowledge (SPOCK). HP 3PAR StoreServ systems communicate with Quorum Witness using the HP 3PAR StoreServ management interface. The network should be set up such that the QW server has network connectivity to the admin interface on both arrays. It is required to connect to the administration port from a minimum of two controller nodes to the network. However, if the array has more than two controllers, the best practice is to connect all controller nodes to the network. The Quorum Witness software uses port 8080 to communicate with the two HP 3PAR StoreServ systems in the quorum. Hence, firewall rules need to be changed to open port 8080 at the QW site. 5 Technical white paper | Implementing Microsoft multisite clustering using HP 3PAR Peer Persistence Planned switchover A switchover operation migrates the Remote Copy group role from source to target without impacting host I/O. This operation requires that the associated hosts are connected to the arrays on both sites. This is a planned operation and the Remote Copy groups need to already be started and synced for this command to work. This operation is useful when users are looking for workload mobility between the two sites. You can use the switchover command to reverse the replication, resulting in site 2 hosts being able to access the logical unit numbers (LUNs) locally from site 2 array. The system performs the following action as part of a switchover operation: • I/O from the hosts to the volumes in the source Remote Copy group is blocked and in-flight I/Os are allowed to drain. The Remote Copy group is stopped and snapshots are taken on the primary array. • The primary array target port group is changed to transition state and it sends a remote failover request to the secondary Remote Copy group. • The secondary array target port group is then changed to transition state and it takes a recovery point snapshot. • The secondary Remote Copy group changes state to become primary-reversed and makes the volumes read/write. • The primary-rev target port group is changed to active state and the array returns a failover complete message to the primary Remote Copy group. • The primary array target port group is changed to standby state and any blocked I/O is returned to the host with a sense error: NOT READY, LOGICAL UNIT NOT ACCESSIBLE, TARGET PORT IN STANDBY STATE. • The host will then perform SCSI inquiry requests to detect what target port groups have changed and which paths are now active, and I/O will now be serviced on the active path to the primary-reverse Remote Copy group. • All operations to this point should complete within 30 seconds to ensure that host I/O does not timeout. • The primary Remote Copy group will then send a remote recover request to the primary-reverse Remote Copy group. The primary-reverse Remote Copy group will perform a recover operation and will change the primary Remote Copy group state to secondary-reverse state. The Remote Copy group will then be started from primary-reverse to secondary-reverse. When the primary-reverse and secondary-reverse volumes are back in sync, the snapshots that were taken on both sides will be removed. The Remote Copy group will then undergo a reverse (-natural) operation. This operation will change the primary-reverse group to primary and secondary-reverse group to secondary. HP 3PAR Peer Persistence failures handling Peer Persistence with Quorum Witness can identify different types of failures and perform transparent failover automatically. HP 3PAR Peer Persistence will perform automatic transparent failover only for Remote Copy groups that have the auto_failover and path_management enabled. Array-to-array communication failure In the case of an array-to-array communication failure, HP 3PAR StoreServ will continue to send and receive communication to and from Quorum Witness. Automatic transparent failover does not occur and host I/Os continue to go to their primary volumes; however, replication of I/O across RC links will stop due to the failure. Single site to QW communication failure In the case where the arrays at either site 1 or site 2 lose connectivity with Quorum Witness, an automatic failover will not occur. Host I/O will continue and replication of I/O across RC links will continue normally. An automatic failover does not need to occur because the two HP 3PAR StoreServ arrays can still communicate with one another via the replication links they share. Site 1 to QW and array-to-array communication failure If the array in site 1 becomes isolated due to a dual network failure (communication with QW fails and the replication links to the array in site 2 fails simultaneously), Remote Copy groups that are in primary state on the array in site 1 will be transitioned to failsafe mode and will stop serving I/O to the ESXi hosts in site 1. The array in site 2 (which is still communicating with QW) will perform an automatic failover and host I/O will be redirected to the new primary volumes in site 2. Site 2 to QW and array-to-array communication failure This is the same as the previous case with the difference that site 2 will become isolated due to a dual network failure. 6 Technical white paper | Implementing Microsoft multisite clustering using HP 3PAR Peer Persistence Site 1 and site 2 both lose communications to QW In the case of a communication failure with QW from both sites, but where the Remote Copy links remain operational, both arrays will be aware of each other’s operational state. Automatic failover does not happen and host I/Os continue to the primary volumes; replication of I/O across RC links will continue as normal. Site 1 and site 2 to QW and array-to-array communication failure In the case where all network communication between the arrays and QW, and between both arrays fails, both arrays will be isolated as they can neither communicate with each other over the RC links nor with QW. In this case, Remote Copy groups that are primary will go into failsafe mode and stop serving I/O, and failover actions will not be performed. This will result in host I/O failure, and replication of I/O across RC links will stop due to the failure. Table 1. HP 3PAR Peer Persistence failures handling Replication stopped Automatic failover Host I/O impact Array-to-array Remote Copy links failure Yes No No Single site to QW network failure No No No Single site to QW network and array-to-array Remote Copy link failure Yes Yes No Both sites to QW network failure No No No Both sites to QW network and array-to-array Remote Copy link failure Yes No Yes Multisite Windows Server Failover Cluster and HP 3PAR Peer Persistence considerations The implementation of the Microsoft multisite clustering requires certain considerations for optimal deployment. In the next section, we will discuss a few recommendations that the user should consider when deploying HP 3PAR Peer Persistence. WSFC quorum model The quorum configuration in a failover cluster determines the number of failures that the cluster can tolerate. The quorum voting method is determined by the quorum mode configured for the WSFC. These are the quorum modes available to use in WSFC: • Node majority: More than one-half of the voting nodes in the cluster must vote affirmatively for the cluster to be healthy. • Node and file share majority: It is similar to node majority quorum mode, except that a remote file share is also configured as a voting witness, and connectivity from any node to that share is also counted as an affirmative vote. More than one-half of the possible votes must be affirmative for the cluster to be healthy. • Node and disk majority: It is similar to node majority quorum mode, except that a shared disk cluster resource is also designated as a voting witness, and connectivity from any node to that shared disk is also counted as an affirmative vote. More than one-half of the possible votes must be affirmative for the cluster to be healthy. • Disk only: A shared disk cluster resource is designated as a witness, and connectivity by any node to that shared disk is counted as an affirmative vote. For multisite failover clustering with an odd number of nodes, the node majority can be used as quorum mode. However, in this configuration, if the site with more nodes fails, manual intervention would be required to force the cluster to start at the other site, because of loss of the quorum. With an even number of nodes, the quorum mode of node and file share majority is a recommended option. A file share witness (FSW) is recommended since it gives the user the option to place it in a third site. Placing an FSW in a VM on a third site with the HP 3PAR QW VM is an effective option for more resiliency with such deployment scenarios. The file share witness should not reside on any node or VM in the cluster, and it should be visible to all nodes in the cluster. 7 Technical white paper | Implementing Microsoft multisite clustering using HP 3PAR Peer Persistence Cluster validation test The cluster validation test is integrated in Failover Cluster Manager to run a set of validation tests. The validation will test the configuration and provide an assessment of how well failover clustering can be supported in a given configuration. Microsoft requires a cluster validation report as the condition for providing Microsoft support to a given cluster configuration. Microsoft does acknowledge that in certain multisite cluster environments with SAN-based replication, the storage validation checks may fail or provide warnings. As a result, there are guidelines where the storage validation test can be skipped for the cluster validation tests. For more information, visit support.microsoft.com/kb/943984 and support.microsoft.com/kb/2914974. For the HP 3PAR Peer Persistence deployment in a multisite Windows cluster, the cluster validation must be completed with all the tests. This will allow the user to address any issues in the configuration that might impact the user cluster environment later in production. Any failure or warning in the report should be addressed and carefully reviewed. Networking consideration Prior to the release of Windows Server 2008, the options for deploying failover clusters in multisite configurations were limited. Multisite failover clusters were restricted to certain environments because of the requirement that all the cluster nodes be on the same subnet. Windows Server 2008 and Windows 2012 provide more flexibility in implementing multisite failover clusters. The requirement for stretched virtual local area networks (VLANs) to accommodate geographically dispersed hosts that are on different subnets is no longer needed. The failover cluster nodes can be deployed on different subnets across routed networks, and as a result eliminate the need to use VLANs. Tuning of cluster heartbeat settings In WSFC, the heartbeat is the process in which the nodes in the cluster can signal to one another that they are working. Since the release of Microsoft Windows 2008, it became possible for the user to tune the heartbeat settings in WSFC. As a result, there is more flexibility in certain deployments of multisite clusters in high-latency networks. The setting to tune depends whether the nodes are on the same subnet or VLAN or if it is a multi-subnet deployment. By default, the threshold that defines the number of heartbeats that are missed before a recovery task occurs is five heartbeats for nodes located in the same subnet or different subnets. For the delay parameters, specify the amount of time that the cluster network driver waits between sending cluster service heartbeats. Users should consider using the default values if possible and only change those values if there is the need to do so. For information about tuning heartbeat settings in a multisite failover cluster, see go.microsoft.com/fwlink/?LinkId=130588. Microsoft MPIO DSM HP 3PAR uses the in-box Microsoft MPIO DSM for multipathing. For information about the installation and configuration of the MPIO, refer to the HP 3PAR Windows Server 2012 and Windows Server 2008 Implementation Guide. When deploying HP 3PAR Peer Persistence for Windows multisite cluster, it is recommended to enable the path verification for the HP 3PAR disks on the Windows host (figure 17). HP 3PAR Quorum Witness hosting Given how critical QW is to the operation of HP 3PAR Peer Persistence, it is highly recommended that the QW VM be hosted outside the multisite cluster environment and should not be on the HP 3PAR arrays used in the HP 3PAR Peer Persistence deployment it arbitrates or a Windows host that is part of the multisite cluster. It is possible to use a single QW to support up to four HP 3PAR Peer Persistence deployments. This will give users the option for consolidation to save on resources and for ease of management as well. For the latest supported configuration and requirement for HP 3PAR Peer Persistence QW, refer to the Single Point of Connectivity Knowledge (SPOCK). Remote Copy replication links The current release of HP 3PAR Peer Persistence with 3.2.1 supports FC, FCoE, and iSCSI host connectivity. However, it is recommended that Remote Copy of Fibre Channel (RCFC) be used for the synchronous replication of data between the arrays in a Peer Persistence configuration. Use of RCFC will enable a high-bandwidth and low-latency connection for the replication of data between the arrays, as it uses a patented protocol for synchronous replication that only requires a single round trip across the replication links to replicate an I/O smaller than 80K in size. 8 Technical white paper | Implementing Microsoft multisite clustering using HP 3PAR Peer Persistence HP 3PAR Peer Persistence maximums It is highly recommended to consolidate VVs inside a few RC groups in an HP 3PAR Peer Persistence configuration. Such a configuration will make sure the host I/Os will be failed over within the timing window in case of a transparent failover. For the latest supported configuration, maximums, and requirement for HP 3PAR Peer Persistence, refer to the Single Point of Connectivity Knowledge (SPOCK). Sample Peer Persistence configuration This section gives the steps needed to set up a single Remote Copy group in Peer Persistence configuration. Refer to the HP 3PAR StoreServ Remote Copy software user’s guide for more details and the complete set of options available for the commands used in this section. In this example, we are deploying the HP 3PAR Peer Persistence between two sites: Kirkland and Redmond. The multisite Windows Server Failover Cluster consists of three hosts running Windows Server 2012 R2. Deploying Quorum Witness The Quorum Witness software provides the arbitrator functionality in the Peer Persistence setup. The detailed instructions for deploying Quorum Witness are listed in the HP 3PAR StoreServ Remote Copy software user’s guide at the HP Storage Library. Refer to the user’s guide for more detailed information. Requirements of Quorum Witness HP 3PAR StoreServ Quorum Witness deployment requires: • Microsoft Hyper-V Server 2012 R2 • 2 GB memory • Network interface with access to a network with the HP 3PAR StoreServ operating system • IP address (Static is recommended) QW installation Refer to the RC user guide for further information on how to import and deploy QW on the supported hypervisor. Note down the IP address displayed in the Summary tab after powering on the virtual machine. This IP address will be used when configuring Quorum Witness later. Figure 2. HP 3PAR Peer Persistence QW VM 9 Technical white paper | Implementing Microsoft multisite clustering using HP 3PAR Peer Persistence Creating Remote Copy targets This is a software configuration guide only and assumes that the right hardware requirements for setting up Remote Copy are met. This guide will not cover how to configure ports for Remote Copy, how to do cabling, how to perform zoning, and so on. This guide also assumes that you have met all requirements for Peer Persistence. Before you start replicating volumes from one array to another, you need to create Remote Copy targets on both arrays. Connect to both the arrays using the MC, then start with creating “New Configuration” in the Remote Copy Manager. • Select 1-1 configuration and select the two systems that will be used in the RC setup. Figure 3. HP 3PAR RC configuration—Target Configuration—Links • Select the ports that will be used for Remote Copy by dragging and dropping the links between the ports. RCFC ports are used in this example. • Click on finish as we would like to create groups later. 10 Technical white paper | Implementing Microsoft multisite clustering using HP 3PAR Peer Persistence Figure 4. HP 3PAR RC Links configuration Figure 5. HP 3PAR Remote Copy configuration 11 Technical white paper | Implementing Microsoft multisite clustering using HP 3PAR Peer Persistence Peer Persistence configuration Click on the Peer Persistence configuration tab after selecting the Remote Copy configuration navigation tree node. Use the IP address assigned to the QW during its configuration. Do not click on the “Start Witness” checkbox at time of creating the configuration. Press OK after specifying the IP address. Figure 6. HP 3PAR Peer Persistence configuration After a brief moment the Quorum Status will change to Not-started. Select both targets in the list and click on the Start button to start the Quorum Witness. Figure 7. HP 3PAR Peer Persistence configuration—Start The status should now change to Started. 12 Technical white paper | Implementing Microsoft multisite clustering using HP 3PAR Peer Persistence Figure 8. HP 3PAR Peer Persistence configuration—Status Redmond3PAR cli% showrcopy Remote Copy System Information Status: Started, Normal Target Information Name Kirkland3PAR ID Type Status Options 1 FC ready Policy 2FF70002AC018575 mirror_config Link Information Target Node Address Status Options Kirkland3PAR 0:2:3 20230002AC018575 Up Kirkland3PAR 1:2:4 21240002AC018575 Up receive 0:2:3 20230002AC018575 Up receive 1:2:4 21240002AC018575 Up Troubleshooting Witness Quorum Communication issues This operation needs to be done using CLI, so login to one of the HP 3PAR StoreServ arrays and perform the following: • Find the target name using showrcopy command. Redmond3PAR cli% showrcopy Remote Copy System Information Status: Started, Normal Target Information Name Kirkland3PAR ID Type Status Options 1 FC ready Policy 2FF70002AC018575 mirror_config Link Information Target Node Address Status Options Kirkland3PAR 0:2:3 20230002AC018575 Up 13 Technical white paper | Implementing Microsoft multisite clustering using HP 3PAR Peer Persistence Kirkland3PAR 1:2:4 21240002AC018575 Up receive 0:2:3 20230002AC018575 Up receive 1:2:4 21240002AC018575 Up Redmond3PAR cli% showrcopy -qw targets Remote Copy System Information Status: Started, Normal Target Information Name Kirkland3PAR ID Type Status Policy 1 FC ready QW-IP Q-Status Q-Status-Qual mirror_config 192.168.0.34 Started • Use the witness check command to verify the communication between Quorum Witness and management interface of the two arrays. A basic message is displayed if the verification is successful. Redmond3PAR cli% setrcopytarget witness check 192.168.0.34 Connectivity check passed Redmond3PAR cli% setrcopytarget witness check -remote 192.168.0.34 Kirkland3PAR Connectivity check passed • The following is an example if the Quorum Witness is not reachable: Redmond3PAR cli% setrcopytarget witness check 192.168.0.37 error: No route to Quorum Witness at 192.168.0.37 Creating Remote Copy groups Start the Create Remote Copy group wizard and select the source and target system. Next specify the group name and select synchronous mode. On the next screen, select the source volume you want to replicate and select the “Create new volume” on the target system. It’s preferred to create new volumes on the target, this way the volumes will have the same characteristics as the source volumes. If you manually created volume beforehand, then be sure that the volume size on target matches the volume size on the source and both volumes have the same WWN (Figure 10). Add all volumes in the group and click Finish. The group will start and initial synchronization will be started. More groups should be created. A typical configuration will have bi-directional HP 3PAR Remote Copy groups replication. Figure 9. HP 3PAR Peer Persistence RC group creation—VVs configuration 14 Technical white paper | Implementing Microsoft multisite clustering using HP 3PAR Peer Persistence Figure 10. HP 3PAR Peer Persistence RC group creation—VVs configuration To continue with the configuration and the setup of the RC group, the user would need to add the VVs into RC groups and use synchronous replication. Changing Group Policy It’s required to edit the replication group and enable “auto_failover” and “Path_management” policies to enable automatic transparent failover. Click on the Enable/Disable Groups button in the Peer Persistence tab. Select the group that you want to enable automatic failover and press OK ( Figure 11). Figure 11. HP 3PAR Peer Persistence Auto Failover 15 Technical white paper | Implementing Microsoft multisite clustering using HP 3PAR Peer Persistence This will enable the auto_failover and the path_management polices on the RC group. Use the showrcopy command to verify the auto failover and path management settings. Redmond3PAR cli% showrcopy Remote Copy System Information Status: Started, Normal Target Information Name Kirkland3PAR ID Type Status Options 1 FC ready Policy 2FF70002AC018575 mirror_config Link Information Target Node Address Status Options Kirkland3PAR 0:2:3 20230002AC018575 Up Kirkland3PAR 1:2:4 21240002AC018575 Up receive 0:2:3 20230002AC018575 Up receive 1:2:4 21240002AC018575 Up Group Information Name Target Status rc-gp-red-1 Kirkland3PAR Started auto_failover,path_management LocalVV ID Role Mode Primary Sync RemoteVV ID Options SyncStatus LastSyncTime clus1-red-csv-1 142 clus1-kir-csv-1 98 Synced NA clus1-red-csv-2 143 clus1-kir-csv-2 99 Synced NA Name Target Status rc-gp-red-2 Kirkland3PAR Started auto_failover,path_management LocalVV ID Role Mode Primary Sync RemoteVV ID Options SyncStatus LastSyncTime clus1-red-csv-3 144 clus1-kir-csv-3 101 Synced NA clus1-red-csv-4 145 clus1-kir-csv-4 102 Synced NA From the other array on the remote site, the same command would yield the following: Kirkland3PAR cli% showrcopy Remote Copy System Information Status: Started, Normal Target Information Name Redmond3PAR 16 ID Type Status Options 1 FC ready Policy 2FF70002AC018576 mirror_config Technical white paper | Implementing Microsoft multisite clustering using HP 3PAR Peer Persistence Link Information Target Node Address Status Options Redmond3PAR 0:2:3 20230002AC018576 Up Redmond3PAR 1:2:4 21240002AC018576 Up receive 0:2:3 20230002AC018576 Up receive 1:2:4 21240002AC018576 Up Group Information Name Target Status rc-gp-red-1.r99702 Redmond3PAR Started auto_failover,path_management LocalVV ID RemoteVV Role Mode Secondary Sync ID SyncStatus Options LastSyncTime clus1-kir-csv-1 98 clus1-red-csv-1 142 Synced NA clus1-kir-csv-2 99 clus1-red-csv-2 143 Synced NA Name Target Status rc-gp-red-2.r99702 Redmond3PAR Started auto_failover,path_management LocalVV ID RemoteVV Role Mode Secondary Sync ID SyncStatus Options LastSyncTime clus1-kir-csv-3 101 clus1-red-csv-3 144 Synced NA clus1-kir-csv-4 102 clus1-red-csv-4 145 Synced NA Exporting LUNs to hosts When creating Windows hosts they need to be created using Persona 15. For best practice add all hosts in the cluster into a host set and add all VVs in a Remote Copy group into a VV set. Figure 12. HP 3PAR Peer Persistence VV export to hosts in the Windows cluster 17 Technical white paper | Implementing Microsoft multisite clustering using HP 3PAR Peer Persistence Figure 13. HP 3PAR Peer Persistence RC groups Figure 14. Windows Server disk management view of the presented VVs 18 Technical white paper | Implementing Microsoft multisite clustering using HP 3PAR Peer Persistence Figure 15. Windows Server MPIO state for HP 3PAR Peer Persistence VVs Figure 16. Windows Server MPIO properties for HP 3PAR Peer Persistence VVs 19 Technical white paper | Implementing Microsoft multisite clustering using HP 3PAR Peer Persistence Figure 17. Enable path verification Figure 18. Windows cluster validation 20 Technical white paper | Implementing Microsoft multisite clustering using HP 3PAR Peer Persistence Figure 19. Configuring cluster disk resources Figure 20. Adding CSVs 21 Technical white paper | Implementing Microsoft multisite clustering using HP 3PAR Peer Persistence Figure 21. Using the CSVs to run VMs Planned switchover example The CLI command has to be used to initiate a planned failover/switchover. The command has to be issued on the array with the primary role. This section provides details about the workings of this operation. In this example the Remote Copy groups are primary on Redmond3PAR. Redmond3PAR cli% showrcopy Remote Copy System Information Status: Started, Normal Target Information Name Kirkland3PAR ID Type Status Options 1 FC ready Policy 2FF70002AC018575 mirror_config Link Information Target Node Address Status Options Kirkland3PAR 0:2:3 20230002AC018575 Up Kirkland3PAR 1:2:4 21240002AC018575 Up receive 0:2:3 20230002AC018575 Up receive 1:2:4 21240002AC018575 Up Group Information Name Target Status rc-gp-red-1 Kirkland3PAR Started auto_failover,path_management LocalVV 22 ID Role Mode Primary Sync RemoteVV ID SyncStatus Options LastSyncTime clus1-red-csv-1 142 clus1-kir-csv-1 98 Synced NA clus1-red-csv-2 143 clus1-kir-csv-2 99 Synced NA Technical white paper | Implementing Microsoft multisite clustering using HP 3PAR Peer Persistence Name Target Status rc-gp-red-2 Kirkland3PAR Started auto_failover,path_management LocalVV ID Role Mode Primary Sync RemoteVV ID Options SyncStatus LastSyncTime clus1-red-csv-3 144 clus1-kir-csv-3 101 Synced NA clus1-red-csv-4 145 clus1-kir-csv-4 102 Synced NA The Remote Copy groups are secondary on Kirkland3PAR. Kirkland3PAR cli% showrcopy Remote Copy System Information Status: Started, Normal Target Information Name Redmond3PAR ID Type Status Options 1 FC ready Policy 2FF70002AC018576 mirror_config Link Information Target Node Address Status Options Redmond3PAR 0:2:3 20230002AC018576 Up Redmond3PAR 1:2:4 21240002AC018576 Up receive 0:2:3 20230002AC018576 Up receive 1:2:4 21240002AC018576 Up Group Information Name Target Status rc-gp-red-1.r99702 Redmond3PAR Started auto_failover,path_management LocalVV ID RemoteVV Role Mode Secondary Sync ID SyncStatus Options LastSyncTime clus1-kir-csv-1 98 clus1-red-csv-1 142 Synced NA clus1-kir-csv-2 99 clus1-red-csv-2 143 Synced NA Name Target Status rc-gp-red-2.r99702 Redmond3PAR Started auto_failover,path_management LocalVV ID RemoteVV Role Mode Secondary Sync ID SyncStatus Options LastSyncTime clus1-kir-csv-3 101 clus1-red-csv-3 144 Synced NA clus1-kir-csv-4 102 clus1-red-csv-4 145 Synced NA 23 Technical white paper | Implementing Microsoft multisite clustering using HP 3PAR Peer Persistence Figure 22. The host path status before the switchover Figure 23. HP 3PAR Peer Persistence switchover using the HP 3PAR MC 24 Technical white paper | Implementing Microsoft multisite clustering using HP 3PAR Peer Persistence After the switchover command is executed successfully the primary role will move to Kirkland3PAR Redmond3PAR cli% showrcopy Remote Copy System Information Status: Started, Normal Target Information Name Kirkland3PAR ID Type Status Options 1 FC ready Policy 2FF70002AC018575 mirror_config Link Information Target Node Address Status Options Kirkland3PAR 0:2:3 20230002AC018575 Up Kirkland3PAR 1:2:4 21240002AC018575 Up receive 0:2:3 20230002AC018575 Up receive 1:2:4 21240002AC018575 Up Group Information Name Target Status rc-gp-red-1 Kirkland3PAR Started auto_failover,path_management LocalVV ID Role Mode Secondary Sync RemoteVV ID Options SyncStatus LastSyncTime clus1-red-csv-1 142 clus1-kir-csv-1 98 Synced NA clus1-red-csv-2 143 clus1-kir-csv-2 99 Synced NA Name Target Status rc-gp-red-2 Kirkland3PAR Started auto_failover,path_management LocalVV ID Role Mode Secondary Sync RemoteVV ID Options SyncStatus LastSyncTime clus1-red-csv-3 144 clus1-kir-csv-3 101 Synced NA clus1-red-csv-4 145 clus1-kir-csv-4 102 Synced NA Kirkland3PAR cli% showrcopy Remote Copy System Information Status: Started, Normal Target Information Name Redmond3PAR ID Type Status Options 1 FC ready Policy 2FF70002AC018576 mirror_config 25 Technical white paper | Implementing Microsoft multisite clustering using HP 3PAR Peer Persistence Link Information Target Node Address Status Options Redmond3PAR 0:2:3 20230002AC018576 Up Redmond3PAR 1:2:4 21240002AC018576 Up receive 0:2:3 20230002AC018576 Up receive 1:2:4 21240002AC018576 Up Group Information Name Target Status rc-gp-red-1.r99702 Redmond3PAR Started auto_failover,path_management LocalVV ID RemoteVV Mode Primary Sync ID SyncStatus Options LastSyncTime clus1-kir-csv-1 98 clus1-red-csv-1 142 Synced NA clus1-kir-csv-2 99 clus1-red-csv-2 143 Synced NA Name Target Status rc-gp-red-2.r99702 Redmond3PAR Started auto_failover,path_management LocalVV ID RemoteVV Role Mode Primary Sync ID SyncStatus Options LastSyncTime clus1-kir-csv-3 101 clus1-red-csv-3 144 Synced NA clus1-kir-csv-4 102 clus1-red-csv-4 145 Synced NA Figure 24. Host path status change after the switchover 26 Role Technical white paper | Implementing Microsoft multisite clustering using HP 3PAR Peer Persistence Figure 25. Host I/O load switch between the two arrays during a switchover Recovering from automatic failover If one array fails, then primary Remote Copy groups from the array that has “auto_failover” policy enabled will be failed over to the secondary array. Recovery process is needed to restart replication and reverse replication back in the original direction. Redmond3PAR cli% showrcopy Remote Copy System Information Status: Started, Normal Target Information Name Kirkland3PAR ID Type Status Options 1 FC ready Policy 2FF70002AC018575 mirror_config Link Information Target Node Address Status Options Kirkland3PAR 0:2:3 20230002AC018575 Up Kirkland3PAR 1:2:4 21240002AC018575 Up receive 0:2:3 20230002AC018575 Up receive 1:2:4 21240002AC018575 Up Group Information Name Target Status rc-gp-red-1 Kirkland3PAR Started auto_failover,path_management LocalVV ID Role Mode Primary Sync RemoteVV ID SyncStatus Options LastSyncTime clus1-red-csv-1 142 clus1-kir-csv-1 98 Synced NA clus1-red-csv-2 143 clus1-kir-csv-2 99 Synced NA 27 Technical white paper | Implementing Microsoft multisite clustering using HP 3PAR Peer Persistence Name Target Status rc-gp-red-2 Kirkland3PAR Started auto_failover,path_management LocalVV ID Role Mode Primary Sync RemoteVV ID Options SyncStatus LastSyncTime clus1-red-csv-3 144 clus1-kir-csv-3 101 Synced NA clus1-red-csv-4 145 clus1-kir-csv-4 102 Synced NA Kirkland3PAR cli% showrcopy Remote Copy System Information Status: Started, Normal Target Information Name Redmond3PAR ID Type Status Options 1 FC ready Policy 2FF70002AC018576 mirror_config Link Information Target Node Address Status Options Redmond3PAR 0:2:3 20230002AC018576 Up Redmond3PAR 1:2:4 21240002AC018576 Up receive 0:2:3 20230002AC018576 Up receive 1:2:4 21240002AC018576 Up Group Information Name Target Status rc-gp-red-1.r99702 Redmond3PAR Started auto_failover,path_management LocalVV ID RemoteVV Role Mode Secondary Sync ID SyncStatus Options LastSyncTime clus1-kir-csv-1 98 clus1-red-csv-1 142 Synced NA clus1-kir-csv-2 99 clus1-red-csv-2 143 Synced NA Name Target Status rc-gp-red-2.r99702 Redmond3PAR Started auto_failover,path_management LocalVV ID RemoteVV Role Mode Secondary Sync ID SyncStatus Options LastSyncTime clus1-kir-csv-3 101 clus1-red-csv-3 144 Synced NA clus1-kir-csv-4 102 clus1-red-csv-4 145 Synced NA After the Redmond3PAR become inaccessible, the Kirkland3PAR become the primary-rev. The RC links are down and the replication is stopped. Kirkland3PAR cli% showrcopy Remote Copy System Information Status: Started, Normal Target Information Name Redmond3PAR 28 ID Type Status Options 1 FC Policy failed 2FF70002AC018576 mirror_config Technical white paper | Implementing Microsoft multisite clustering using HP 3PAR Peer Persistence Link Information Target Node Address Status Options Redmond3PAR 0:2:3 20230002AC018576 Down Redmond3PAR 1:2:4 21240002AC018576 Down receive 0:2:3 20230002AC018576 Up receive 1:2:4 21240002AC018576 Up Group Information Name Target Status rc-gp-red-1.r99702 Redmond3PAR Stopped auto_failover,path_management Role LocalVV ID Mode Options RemoteVV Primary-Rev Sync ID SyncStatus LastSyncTime clus1-kir-csv-1 98 clus1-red-csv-1 142 Stopped 2014-10-15 15:11:16 PDT clus1-kir-csv-2 99 clus1-red-csv-2 143 Stopped 2014-10-15 15:11:16 PDT Name Target Status rc-gp-red-2.r99702 Redmond3PAR Stopped auto_failover,path_management Role LocalVV ID Mode Options RemoteVV Primary-Rev Sync ID SyncStatus LastSyncTime clus1-kir-csv-3 101 clus1-red-csv-3 144 Stopped 2014-10-15 15:11:16 PDT clus1-kir-csv-4 102 clus1-red-csv-4 145 Stopped 2014-10-15 15:11:16 PDT After the array is recovered and back online the showrcopy status will show Redmond3PAR cli% showrcopy Remote Copy System Information Status: Started, Normal Target Information Name Kirkland3PAR ID Type Status Options 1 FC ready Policy 2FF70002AC018575 mirror_config Link Information Target Node Address Status Options Kirkland3PAR 0:2:3 20230002AC018575 Up Kirkland3PAR 1:2:4 21240002AC018575 Up receive 0:2:3 20230002AC018575 Up receive 1:2:4 21240002AC018575 Up Group Information Name Target Status Role Mode rc-gp-red-1 Kirkland3PAR Failsafe Primary auto_failover,path_management LocalVV ID RemoteVV ID Options Sync SyncStatus LastSyncTime clus1-red-csv-1 142 clus1-kir-csv-1 98 Failsafe NA clus1-red-csv-2 143 clus1-kir-csv-2 99 Failsafe NA 29 Technical white paper | Implementing Microsoft multisite clustering using HP 3PAR Peer Persistence Name Target Status Role Mode rc-gp-red-2 Kirkland3PAR Failsafe Primary auto_failover,path_management LocalVV ID RemoteVV ID Options Sync SyncStatus LastSyncTime clus1-red-csv-3 144 clus1-kir-csv-3 101 Failsafe NA clus1-red-csv-4 145 clus1-kir-csv-4 102 Failsafe NA These steps should be used to recover the Remote Copy group after the array that had failed is back online. Select the group(s) that needs to be recovered under the Remote Copy configuration node. Click on the Recover button and press OK. Select Yes at the next prompt to continue with the recovery. Figure 26. HP 3PAR Peer Persistence recover The Restore button will be enabled after the recovery has completed. However, performing the restore operation might lead to the Windows hosts losing access to the LUNs for a short duration. Hence it’s recommended to use the CLI commands to complete the failback. Note It is also required that a manual device rescan is performed on each node in the Windows Server Failover Cluster in order to work around an issue with the MPIO DSM not detecting the change of status of the restored paths to the previously inaccessible array. Login to the array with the primary-rev role and issue the following command. Kirkland3PAR cli% setrcopygroup reverse -f -natural rc-gp-red-1.r99702 rc-gp-red2.r99702 setrcopygroup for reverse -natural rc-gp-red-1.r99702 rc-gp-red-2.r99702 reverse started with tasks: 4602 4603 30 Technical white paper | Implementing Microsoft multisite clustering using HP 3PAR Peer Persistence Now the two array would show the following; Kirkland3PAR cli% showrcopy Remote Copy System Information Status: Started, Normal Target Information Name Redmond3PAR ID Type Status Options 1 FC ready Policy 2FF70002AC018576 mirror_config Link Information Target Node Address Status Options Redmond3PAR 0:2:3 20230002AC018576 Up Redmond3PAR 1:2:4 21240002AC018576 Up receive 0:2:3 20230002AC018576 Up receive 1:2:4 21240002AC018576 Up Group Information Name Target Status rc-gp-red-1.r99702 Redmond3PAR Started auto_failover,path_management Role Mode Primary Sync LocalVV ID RemoteVV ID clus1-kir-csv-1 98 clus1-red-csv-1 142 Synced NA clus1-kir-csv-2 99 clus1-red-csv-2 143 Synced NA Name Target Status rc-gp-red-2.r99702 Redmond3PAR Started auto_failover,path_management RemoteVV SyncStatus Options Role Mode Primary Sync ID SyncStatus LastSyncTime Options LocalVV ID LastSyncTime clus1-kir-csv-3 101 clus1-red-csv-3 144 Synced NA clus1-kir-csv-4 102 clus1-red-csv-4 145 Synced NA Redmond3PAR cli% showrcopy Remote Copy System Information Status: Started, Normal Target Information Name Kirkland3PAR ID Type Status Options 1 FC ready Policy 2FF70002AC018575 mirror_config Link Information Target Node Address Status Options Kirkland3PAR 0:2:3 20230002AC018575 Up Kirkland3PAR 1:2:4 21240002AC018575 Up receive 0:2:3 20230002AC018575 Up receive 1:2:4 21240002AC018575 Up 31 Technical white paper | Implementing Microsoft multisite clustering using HP 3PAR Peer Persistence Group Information Name Target Status rc-gp-red-1 Kirkland3PAR Started auto_failover,path_management LocalVV ID Role Mode Secondary Sync RemoteVV ID Options SyncStatus LastSyncTime clus1-red-csv-1 142 clus1-kir-csv-1 98 Synced NA clus1-red-csv-2 143 clus1-kir-csv-2 99 Synced NA Name Target Status rc-gp-red-2 Kirkland3PAR Started auto_failover,path_management LocalVV ID Role Mode Secondary Sync RemoteVV ID Options SyncStatus LastSyncTime clus1-red-csv-3 144 clus1-kir-csv-3 101 Synced NA clus1-red-csv-4 145 clus1-kir-csv-4 102 Synced NA In order to restore the primary and secondary roles back to their state prior to the failure, a switchover is needed. Kirkland3PAR cli% setrcopygroup switchover rc-gp-red-1.r99702 rc-gp-red-2.r99702 This command will migrate the Remote Copy group(s) without impacting the host I/O. The direction of the indicated group(s) will be switched on both the local and the group's target system. Are you sure that you wish to perform this action? select q=quit y=yes n=no: y setrcopygroup for switchover rc-gp-red-1.r99702 rc-gp-red-2.r99702 switchover started with tasks: 4604 4605 This will bring the RC group back to their status prior to the failure. Kirkland3PAR cli% showrcopy Remote Copy System Information Status: Started, Normal Target Information Name Redmond3PAR ID Type Status Options 1 FC ready Policy 2FF70002AC018576 mirror_config Link Information Target Node Address Status Options Redmond3PAR 0:2:3 20230002AC018576 Up Redmond3PAR 1:2:4 21240002AC018576 Up 32 receive 0:2:3 20230002AC018576 Up receive 1:2:4 21240002AC018576 Up Technical white paper | Implementing Microsoft multisite clustering using HP 3PAR Peer Persistence Group Information Name Target Status rc-gp-red-1.r99702 Redmond3PAR Started auto_failover,path_management LocalVV ID RemoteVV Role Mode Secondary Sync ID SyncStatus Options LastSyncTime clus1-kir-csv-1 98 clus1-red-csv-1 142 Synced NA clus1-kir-csv-2 99 clus1-red-csv-2 143 Synced NA Name Target Status rc-gp-red-2.r99702 Redmond3PAR Started auto_failover,path_management LocalVV ID RemoteVV Role Mode Secondary Sync ID SyncStatus Options LastSyncTime clus1-kir-csv-3 101 clus1-red-csv-3 144 Synced NA clus1-kir-csv-4 102 clus1-red-csv-4 145 Synced NA Redmond3PAR cli% showrcopy Remote Copy System Information Status: Started, Normal Target Information Name Kirkland3PAR ID Type Status Options 1 FC ready Policy 2FF70002AC018575 mirror_config Link Information Target Node Address Status Options Kirkland3PAR 0:2:3 20230002AC018575 Up Kirkland3PAR 1:2:4 21240002AC018575 Up receive 0:2:3 20230002AC018575 Up receive 1:2:4 21240002AC018575 Up Group Information Name Target Status rc-gp-red-1 Kirkland3PAR Started auto_failover,path_management LocalVV ID Role Mode Primary Sync RemoteVV ID SyncStatus Options LastSyncTime clus1-red-csv-1 142 clus1-kir-csv-1 98 Synced NA clus1-red-csv-2 143 clus1-kir-csv-2 99 Synced NA Name Target Status rc-gp-red-2 Kirkland3PAR Started auto_failover,path_management LocalVV ID Role Mode Primary Sync RemoteVV ID SyncStatus Options LastSyncTime clus1-red-csv-3 144 clus1-kir-csv-3 101 Synced NA clus1-red-csv-4 145 clus1-kir-csv-4 102 Synced NA 33 Technical white paper | Implementing Microsoft multisite clustering using HP 3PAR Peer Persistence For more information HP Enterprise Storage Information Library, hp.com/Go/Storage/Docs HP 3PAR StoreServ Storage: hp.com/go/storeserv HP Support Center, HP 3PAR StoreServ Implementation Guide HP 3PAR StoreServ Storage best practice guide 4AA4-4524ENW HP 3PAR Remote Copy Software User Guide OS 3.2.1 Learn more at hp.com/solutions/activeanswers Sign up for updates hp.com/go/getupdated Share with colleagues Rate this document © Copyright 2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. The only warranties for HP products and services are set forth in the express warranty statements accompanying such products and services. Nothing herein should be construed as constituting an additional warranty. HP shall not be liable for technical or editorial errors or omissions contained herein. Microsoft, Windows, and Windows Server are trademarks of the Microsoft Group of companies. Linux is the registered trademark of Linus Torvalds in the U.S. and other countries. VMware is a registered trademark or trademark of VMware, Inc. in the United States and/or other jurisdictions. 4AA5-5894ENW, November 2014