EMC MISSIONCRITICAL BUSINESS
CONTINUITY FOR SAP
EMC VPLEX, EMC Symmetrix VMAX,
EMC VNX, VMware vSphere HA,
Brocade Networking, Oracle RAC,
SUSE Linux Enterprise
EMC Solutions Group
© Copyright 2012 EMC Corporation. All rights reserved.
1
Agenda
• Solution overview and architecture
• Solution components and configuration
– EMC VPLEX Metro
– VMware vSphere
– SAP system architecture
– Oracle database
– Brocade network
– EMC storage
• Testing and validation
• Summary and conclusion
© Copyright 2012 EMC Corporation. All rights reserved.
2
Mission-Critical Business Continuity for SAP
• Eliminate single points of failure at all layers in
environment
• Provide active/active data centers with nearzero RPOs and RTOs
Active/active data centers
Near-zero RTOs and RPOs
24/7 application availability
Fully automatic failure handling
and load balancing
Zero downtime maintenance
No single points of failure
Simplified deployment of Oracle
RAC on Extended Distance Clusters
Simplified high availability
management
Increased infrastructure utilization
© Copyright 2012 EMC Corporation. All rights reserved.
3
The Challenge; the Solution
The Challenge
SAP single points of failure
© Copyright 2012 EMC Corporation. All rights reserved.
The Solution
High availability and business continuity
4
Eliminating Single Points of Failure
© Copyright 2012 EMC Corporation. All rights reserved.
5
Solution Components
Mission-critical business continuity for SAP ERP is delivered by
combination of technologies from EMC, VMware, Oracle, SUSE,
and Brocade
 EMC VPLEX Metro
 EMC VPLEX Witness
 EMC Symmetrix VMAX and EMC VNX
 Oracle RAC on Extended Distance Clusters
 VMware vSphere
 VMware vSphere High Availability
 SUSE Linux Enterprise Server for SAP Applications,
with SUSE Linux Enterprise High Availability Extension
 SAP Enqueue Replication Server
 Brocade MLXe core routers
 Brocade DCX 8510 Backbones
© Copyright 2012 EMC Corporation. All rights reserved.
6
Solution Architecture
© Copyright 2012 EMC Corporation. All rights reserved.
7
Protection Layers
© Copyright 2012 EMC Corporation. All rights reserved.
8
VPLEX Metro – Introduction
Site A
VPLEX
Site B
Cross-Cluster Connect
• SAN-based storage
federation
• Active/active data centers
VPLEX
WITNESS
– ~100 km distance
• Workload rebalancing
• Near-zero RPO/RTO
• Data center migration
Site C
• VPLEX High Availability
Active AccessAnywhere Active
– VPLEX Witness
– VPLEX Cross-Cluster
Connect
© Copyright 2012 EMC Corporation. All rights reserved.
9
VPLEX Metro Configuration
VPLEX logical structures






© Copyright 2012 EMC Corporation. All rights reserved.
Consistency group
Virtual volume
Distributed device
Device
Extent
Storage volume
10
VMware Virtualization Components
 vSphere 5.0
 vMotion
 Storage vMotion
 VMware HA
 DRS (Distributed
© Copyright 2012 EMC Corporation. All rights reserved.
Resource Scheduler)
 EMC PowerPath/VE
 EMC Virtual Storage
Integrator (VSI)
11
VMware vSphere with VPLEX Metro
© Copyright 2012 EMC Corporation. All rights reserved.
12
VMware Stretched Cluster Configuration
vCenter
screenshots
© Copyright 2012 EMC Corporation. All rights reserved.
13
VMware HA and DRS Configuration
HA Restart Priority for SAP VMs
HA and DRS enabled for VMware stretched cluster
HA heartbeat datastores
DRS VM-VM affinity rule
© Copyright 2012 EMC Corporation. All rights reserved.
14
EMC Virtual Storage Integrator and VPLEX
EMC VSI tab in vCenter GUI
© Copyright 2012 EMC Corporation. All rights reserved.
15
SAP System Architecture
 SAP application software
– SAP Enhancement Package 4 for
SAP ERP 6.0 IDES
– SAP NetWeaver Application Server
for ABAP 7.01
– SAP Enqueue Replication Server
 Operating system
– SUSE Linux Enterprise Server
(SLES) for SAP Applications 11 SP1
– SUSE Linux Enterprise High
Availability Extension
 Virtualization
– SAP services on VMware virtual
machines
– Oracle RAC database on physical
servers
© Copyright 2012 EMC Corporation. All rights reserved.
16
SAP System Architecture – Design
Considerations
 Enqueue and message servers
decoupled from Central Instance and
implemented as services within
ASCS instance
 ERS installed as part of HA
architecture to provide zero
application lock loss
 Two dialog instances provide
redundant work processes such as
dialog, background, update, spool
 ASCS instance installed with virtual
hostname to decouple it from VM
hostname
 ERS instance installed with different
instance number to avoid confusion
when both ASCS and ERS are
under cluster control
© Copyright 2012 EMC Corporation. All rights reserved.
17
SAP System Architecture – Design
Considerations -Continued
 SAP update processes configured on
additional application server
instances
 ASCS, ERS, start, and dialog
instance profiles updated with ERS
configurations
 SAP shared file systems stored on
Oracle ACFS and mounted as NFS
shares on SAP VMs – presented
as highly available NFS resource
managed by Oracle Clusterware
 Storage for entire SAP environment
encapsulated, virtualized, distributed
across two sites, and made
available to SAP servers through
VPLEX Metro
© Copyright 2012 EMC Corporation. All rights reserved.
18
SUSE Linux Enterprise HAE Configuration
 SLES HAE protects enqueue and
message servers across two cluster
nodes built on VMware VMs
 VMware High Availability protects the
VMs
 Virtual IP address, master/slave, and
SAPInstance resource agents monitor
and control resource availability
 SAPInstance agent controls ASCS and
ERS instances – configured as
master/slave resource to ensure
ASCS and ERS are never started on
same node
 VMDK partition used as SBD STONITH
device – with multi-writer option
configured to enable write access by
multiple VMs
© Copyright 2012 EMC Corporation. All rights reserved.
19
Oracle Database Architecture
 Oracle components
– Oracle Database 11g Release 2 Enterprise Edition
– Oracle ASM
– Oracle ACFS
– Oracle Clusterware
 Single-instance database migrated to 4-node physical RAC cluster
on ASM

Oracle Extended RAC Over VPLEX
– Simplified management
– Hosts connect only to their local VPLEX cluster
– Hosts send I/O only once to the local cluster – dual writes not required
– No need to deploy Oracle voting disk and Clusterware on third site
– Eliminates costly host CPU cycles consumed by host-based mirroring
– Protect multiple databases and/or applications as a unit
© Copyright 2012 EMC Corporation. All rights reserved.
20
Oracle Database Configuration
 4 ACFS volumes mounted across RAC cluster
 TRANS, ASCS500, SAPMNT exported as NFS
shares to SAP servers
 Shared file systems presented as highly
available NFS resource managed by Oracle
Clusterware
 ASM disk groups configured to reflect
existing single-instance layout
© Copyright 2012 EMC Corporation. All rights reserved.
ACFS volume
Mount point
SAP_O_HOME
/oracle/VSE/112
SAPMNT
/sapmnt/VSE
USRSAPTRANS
/usr/sap/trans
ASCS00
/usr/sap/VSE/ASCS00
ASM disk group
No of
disks
Disk group
size GB)
Redundancy
OCR
5
40
Normal
EA_SAP_ACFS
4
64
External
EA_SAP_DATA
16
2,048
External
EA_SAP_REDO
4
64
External
EA_SAP_REDOM
4
64
External
EA_SAP_FRA
4
256
External
21
Brocade Network Infrastructure
IP network
SAN
© Copyright 2012 EMC Corporation. All rights reserved.
22
EMC Storage Layout
Site A – EMC Symmetrix VMAX
 Virtual Provisioning
© Copyright 2012 EMC Corporation. All rights reserved.
Site B – EMC VNX5700
• Traditional RAID groups and LUNs
23
Testing and Validation
Tests

SAP enqueue service process
failure

SAP ASCS instance virtual
machine failure

Oracle RAC node failure

Site failure

VPLEX cluster isolation
Expected behavior
Application continues
without interruption
(VPLEX cluster, ESXi server, network, RAC nodes)
© Copyright 2012 EMC Corporation. All rights reserved.
24
SAP Enqueue Service Process Failure
1
SAPInstance resource
agent detects/reports
failure.
2
Master/slave resource
agent promotes SAPASCS1
to master (which hosts
ASCS services).
3
Result
• Application continues without interruption
•
4
Master/slave resource
agent starts ERS on
SAPASCS2 when it rejoins
cluster.
Replicated lock table
restored.
No administrative intervention required
© Copyright 2012 EMC Corporation. All rights reserved.
25
SAP ASCS Instance VM Failure
Result
• Application continues without interruption
•
No administrative intervention required
© Copyright 2012 EMC Corporation. All rights reserved.
1
SAPASCS2 becomes
unavailable from vSphere
Client.
2
SAPInstance resource agent
detects/reports failure.
3
VMHA restarts failed VM on
surviving ESXi host.
4
Master/slave resource agent
promotes SAPASCS1 to
master (which hosts ASCS
services) and starts ERS on
SAPASCS2 when it rejoins
cluster.
5
Replicated lock table
restored.
26
Oracle RAC Node Failure
Result
• End user experiences longer
transaction response time
when DI work process
reconnects to other RAC
node.
1
RAC node goes offline – instance
VSE003 unavailable.
2
SAP instance work process
connects to another RAC node.
© Copyright 2012 EMC Corporation. All rights reserved.
•
Uncommitted transactions
rolled back at DB level to
guarantee data consistency;
end user receives system
error message and needs to
restart transaction.
•
No administrative
intervention required.
27
Environment Status Before Site Failure
Status
• All RAC nodes running.
© Copyright 2012 EMC Corporation. All rights reserved.
•
VPLEX clusters available
on both sites.
•
ESXi servers available on
both sites.
•
Site A and Site B SAP
virtual machines up.
28
Site Failure
1
VPLEX Witness overrides
consistency group detach
rule so VPLEX on Site B
remains available.
2
RAC nodes on Site B remain
available.
3
VMHA restarts SAPASCS1
and SAPDI1 on Site B.
4
SLE HAE detects failure of
SAPASCS1 and restarts ERS
when that node rejoins
cluster.
5
© Copyright 2012 EMC Corporation. All rights reserved.
End users on SAPDI1 lose
their sessions, but can log in
again when it restarts on Site
B. During restart, new users
are directed to SAPDI2.
29
VPLEX Cluster Isolation
© Copyright 2012 EMC Corporation. All rights reserved.
1
VPLEX Witness overrides
consistency group detach
rule so VPLEX on Site B
remains available.
2
RAC nodes on Site B
remain available.
3
RAC nodes on Site A are
ejected.
4
ESXi servers on Site A
remain available.
5
Virtual machines SAPASCS1
and SAPDI1 remain active
due to VPLEX Metro HA
Cross-Cluster Connect.
30
Testing and Validation
Tests

SAP enqueue service process
failure

SAP ASCS instance virtual
machine failure

Oracle RAC node failure

Site failure

VPLEX cluster isolation
Observed behavior
Application continues
without interruption
(VPLEX cluster, ESXi server, network, RAC nodes)
© Copyright 2012 EMC Corporation. All rights reserved.
31
Summary and Conclusion
Solution combines EMC, SAP, VMware, Oracle, SUSE, and Brocade
technologies to:
• Eliminate single points of failure at all layers in environment
• Provide active/active data centers with near-zero RPOs and RTOs
• Enable mission-critical business continuity for SAP applications
Active/active data centers
Near-zero RTOs and RPOs
24/7 application availability
Fully automatic failure handling
and load balancing
Zero downtime maintenance
No single points of failure
Simplified deployment of Oracle
RAC on Extended Distance Clusters
Simplified high availability
management
Increased infrastructure utilization
© Copyright 2012 EMC Corporation. All rights reserved.
32