Virtualizing Cisco Unified Communications Nutanix Best Practices Version 1.1 • June 2016 • BP-2006 Virtualizing Cisco Unified Communications Copyright Copyright 2016 Nutanix, Inc. Nutanix, Inc. 1740 Technology Drive, Suite 150 San Jose, CA 95110 All rights reserved. This product is protected by U.S. and international copyright and intellectual property laws. Nutanix is a trademark of Nutanix, Inc. in the United States and/or other jurisdictions. All other marks and names mentioned herein may be trademarks of their respective companies. Copyright | 2 Virtualizing Cisco Unified Communications Contents 1. Executive Summary................................................................................ 5 2. Introduction..............................................................................................6 2.1. Audience........................................................................................................................ 6 2.2. Purpose..........................................................................................................................6 3. Nutanix Enterprise Cloud Platform Overview.......................................7 3.1. Nutanix Acropolis Overview...........................................................................................7 3.2. Distributed Storage Fabric.............................................................................................8 3.3. App Mobility Fabric........................................................................................................8 3.4. AHV................................................................................................................................8 3.5. Nutanix Acropolis Architecture...................................................................................... 8 4. Cisco Unified Communications Overview.......................................... 12 4.1. Call Control: Cisco Unified Communications Manager................................................14 4.2. Voice Messaging: Cisco Unity Connection..................................................................15 4.3. Instant Messaging and Presence: CUCM IM and Presence....................................... 16 4.4. E911 Location Services: Cisco Emergency Responder.............................................. 17 4.5. License Management: Prime License Manager.......................................................... 19 5. Delivering High Performance and Highly Available Cisco UC.......... 21 6. Designing and Architecting Cisco UC on Nutanix............................. 23 6.1. Cisco Collaboration Sizing Tool.................................................................................. 23 7. Sample Deployments of Cisco UC on Nutanix...................................25 7.1. 1,000 User Deployment...............................................................................................25 7.2. 1,000 User Deployment VM Placement...................................................................... 26 7.3. 30,000 User Deployment.............................................................................................27 7.4. 30,000 User Deployment VM Placement.................................................................... 30 3 Virtualizing Cisco Unified Communications 8. Nutanix Best Practices for Cisco Unified Communications.............. 32 8.1. Nutanix Platform Guidance..........................................................................................32 8.2. Virtualization and Compute Configuration................................................................... 33 8.3. Storage Configuration..................................................................................................34 8.4. Networking................................................................................................................... 35 8.5. High Availability........................................................................................................... 36 8.6. Backup and Disaster Recovery................................................................................... 37 8.7. UC Application Installation...........................................................................................37 8.8. Cisco UC Virtualization Requirements........................................................................ 37 9. Conclusion............................................................................................. 40 Appendix......................................................................................................................... 41 Best Practices Checklist..................................................................................................... 41 References.......................................................................................................................... 42 Meet the Author.................................................................................................................. 43 About Nutanix......................................................................................................................43 List of Figures................................................................................................................44 List of Tables................................................................................................................. 45 4 Virtualizing Cisco Unified Communications 1. Executive Summary Nutanix delivers a highly resilient converged compute and storage platform that brings the benefits of web-scale infrastructure to organizations of all sizes. Designed for supporting virtualized environments including VMware vSphere, Microsoft Hyper-V, and AHV, Nutanix is the ideal infrastructure for running all types of virtual workloads, including real-time technologies such as Cisco Unified Communications (UC). UC administrators can easily provision the full Unified Communications suite on Nutanix, providing call control, voice messaging, E911, and Cisco Jabber with minimal datacenter footprint. Nutanix provides the functionality business and IT administrators need to deliver fast performance, easy scalability, simple management, fast provisioning, and high availability for Cisco UC environments, including: • Convergence of compute and storage into a single appliance. • A highly distributed software architecture that eliminates the bottlenecks and complexities found in traditional SAN and NAS storage platforms when scaling to a large number of Cisco UC workloads. • Data and I/O localization. Recently accessed VM data is stored locally on flash for fast access. • The Nutanix web-scale converged infrastructure provides full support for VAAI, allowing you to leverage all the latest advances from VMware and taking your solution to the next level. • Nondisruptive scaling. Autodiscovery and simple addition of new nodes to the Nutanix cluster enables a cost-effective pay-as-you-grow model. • Reduced storage requirements due to capacity-optimization technologies, such as compression and deduplication. • Data protection with Nutanix replication, VMCaliber snapshots, and disaster recovery that can be managed transparently in the Nutanix Prism web interface or API. • No-downtime, one-click upgrades of the Acropolis Operating System (AOS) storage software, firmware, and hypervisor of choice. This document shows the Nutanix recommended configuration for successfully implementing Cisco Unified Communications on Nutanix. The document also provides sizing guidance and scalability options. 1. Executive Summary | 5 Virtualizing Cisco Unified Communications 2. Introduction 2.1. Audience This best practice guide is intended for Nutanix and Cisco Unified Communications administrators and solutions architects who are responsible for planning, designing, and deploying UC infrastructures in either on-premises or “UC-as-a-Service” environments running on the Nutanix platform. Some portions of this document reference tools and websites that are necessary for the successful deployment of Cisco UC but that are only available to Cisco partners. As such, we assume a Cisco partner’s involvement in UC design and deployment. 2.2. Purpose This best practice guide acts as a reference when making design decisions for a Cisco Unified Communications deployment on the Nutanix infrastructure. The primary focus is on core Cisco Unified Communications applications. This document contains: • An overview of the Nutanix platform and the benefits of using web-scale infrastructure for Unified Communications. • An overview of the core Cisco Unified Communications components. • Sizing and placement guidelines for deploying UC VMs on Nutanix. • Best practices for Nutanix configuration for UC environments. Table 1: Document Version History Version Number Published Notes 1.0 January 2015 Original publication. 1.1 June 2016 Updated platform overview, processor specifications, and formatting. 2. Introduction | 6 Virtualizing Cisco Unified Communications 3. Nutanix Enterprise Cloud Platform Overview 3.1. Nutanix Acropolis Overview Nutanix delivers a hyperconverged infrastructure solution purpose-built for virtualization and cloud environments. This solution brings the performance and economic benefits of web-scale architecture to the enterprise through the Nutanix enterprise cloud platform, which is composed of two product families—Nutanix Acropolis and Nutanix Prism. Attributes of this solution include: • Storage and compute resources hyperconverged on x86 servers. • System intelligence located in software. • Data, metadata, and operations fully distributed across entire cluster of x86 servers. • Self-healing to tolerate and adjust to component failures. • API-based automation and rich analytics. Nutanix Acropolis can be broken down into three foundational components: the Distributed Storage Fabric (DSF), the App Mobility Fabric (AMF), and AHV. Prism provides one-click infrastructure management for virtual environments running on Acropolis. Acropolis is hypervisor agnostic, supporting two third-party hypervisors—ESXi and Hyper-V—in addition to the native Nutanix hypervisor, AHV. Figure 1: Nutanix Enterprise Cloud Platform 3. Nutanix Enterprise Cloud Platform Overview | 7 Virtualizing Cisco Unified Communications 3.2. Distributed Storage Fabric The Distributed Storage Fabric (DSF) delivers enterprise data storage as an on-demand service by employing a highly distributed software architecture. Nutanix eliminates the need for traditional SAN and NAS solutions while delivering a rich set of VM-centric software-defined services. Specifically, the DSF handles the data path of such features as snapshots, clones, high availability, disaster recovery, deduplication, compression, and erasure coding. The DSF operates via an interconnected network of Controller VMs (CVMs) that form a Nutanix cluster, and every node in the cluster has access to data from shared SSD, HDD, and cloud resources. The hypervisors and the DSF communicate using the industry-standard NFS, iSCSI, and SMB3 protocols. 3.3. App Mobility Fabric The App Mobility Fabric (AMF) is the Nutanix virtualization solution that allows apps to move across hypervisors. When virtual machines can move between hypervisors (for example, between VMware ESXi and AHV), administrators can host production and dev/test environments concurrently on different hypervisors and shift workloads between them as needed. AMF is implemented via a distributed, scale-out service that runs inside the CVM on every node within a Nutanix cluster. 3.4. AHV Nutanix ships with a hardened, enterprise-ready hypervisor based on proven open source technology. AHV is managed with the Prism interface, a robust REST API, and an interactive command-line interface called aCLI (Acropolis CLI). These tools combine to eliminate the management complexity typically associated with open source environments and allow out-ofthe-box virtualization on Nutanix—all without the licensing fees associated with other hypervisors. 3.5. Nutanix Acropolis Architecture Acropolis does not rely on traditional SAN or NAS storage or expensive storage network interconnects. It combines highly dense storage and server compute (CPU and RAM) into a single platform building block. Each building block is based on industry-standard Intel processor technology and delivers a unified, scale-out, shared-nothing architecture with no single points of failure. The Nutanix solution has no LUNs to manage, no RAID groups to configure, and no complicated storage multipathing to set up. All storage management is VM-centric, and the DSF optimizes I/O at the VM virtual disk level. There is one shared pool of storage that includes flash-based SSDs 3. Nutanix Enterprise Cloud Platform Overview | 8 Virtualizing Cisco Unified Communications for high performance and low-latency HDDs for affordable capacity. The file system automatically tiers data across different types of storage devices using intelligent data placement algorithms. These algorithms make sure that the most frequently used data is available in memory or in flash for the fastest possible performance. Figure 2: Information Life Cycle Management With the DSF, a CVM writes data to local flash memory for fast acknowledgment; the CVM also handles read operations locally for reduced latency and fast data delivery. The figure below shows an overview of the Nutanix architecture, including the hypervisor of your choice (AHV, ESXi, or Hyper-V), user VMs, the Nutanix storage CVM, and its local disk devices. Each CVM connects directly to the local storage controller and its associated disks. Using local storage controllers on each host localizes access to data through the DSF, thereby reducing storage I/O latency. The DSF replicates writes synchronously to at least one other Nutanix node in the system, distributing data throughout the cluster for resiliency and availability. Replication factor 2 (RF2) creates two identical data copies in the cluster, and replication factor 3 (RF3) creates three identical data copies. Having a local storage controller on each node ensures that storage performance as well as storage capacity increase linearly with node addition. 3. Nutanix Enterprise Cloud Platform Overview | 9 Virtualizing Cisco Unified Communications Figure 3: Overview of the Nutanix Architecture Local storage for each Nutanix node in the architecture appears to the hypervisor as one large pool of shared storage. This allows the DSF to support all key virtualization features. Data localization maintains performance and quality of service (QoS) on each host, minimizing the effect noisy VMs have on their neighbors’ performance. This functionality allows for large, mixed-workload clusters that are more efficient and more resilient to failure when compared to traditional architectures with standalone, shared, and dual-controller storage arrays. When VMs move from one hypervisor to another, such as during live migration and high availability, the now local CVM serves a newly migrated VM’s data. When reading old data (stored on the now remote CVM) the local CVM forwards the I/O request to the remote CVM. All write I/O occurs locally. The DSF detects that I/O is occurring from a different node and migrates the data to the local node in the background, allowing for all read I/O to now be served locally. The data only migrates when there have been enough reads and writes from the remote node to minimize network utilization. The next figure shows how data follows the VM as it moves between hypervisor nodes. 3. Nutanix Enterprise Cloud Platform Overview | 10 Virtualizing Cisco Unified Communications Figure 4: Data Locality and Live Migration 3. Nutanix Enterprise Cloud Platform Overview | 11 Virtualizing Cisco Unified Communications 4. Cisco Unified Communications Overview The Cisco Unified Communications (UC) suite provides collaboration tools that enable effective team communication regardless of device or location. These tools allow users to communicate in different forms, including voice, video, text, and even file sharing and screen sharing across PC, Mac, Android, and iPhone. Virtualization is now the primary method of deploying UC server applications supporting UC client interactions, replacing many physical servers that were required in the past. The Cisco Unified Communications suite consists of the following core virtual server application components: • Call control: Cisco Unified Communications Manager. • Voice messaging: Cisco Unity Connection. • Instant messaging and presence: Cisco Unified Communications Manager IM and Presence. • E911 location services: Cisco Emergency Responder. • License management: Cisco Prime License Manager. 4. Cisco Unified Communications Overview | 12 Virtualizing Cisco Unified Communications Figure 5: Cisco UC Overview The figure above shows the basic components of a UC deployment. This overview highlights virtual applications central to providing service to end users and clients. We’ve included required physical items such as the Cisco IOS voice gateway, laptops for Jabber clients, and IP phones to illustrate how the virtual server infrastructure fits into the overall UC system. Each UC virtual server application is implemented using a number of VMs. Additional UC components, such as Cisco Unified Contact Center for call center and help desk agents, Video Communications Server for telepresence, and WebEx, can be virtualized following similar methods, but we do not discuss them directly in this guide. 4. Cisco Unified Communications Overview | 13 Virtualizing Cisco Unified Communications 4.1. Call Control: Cisco Unified Communications Manager Cisco Unified Communications Manager (CUCM) is the central component of UC deployments responsible for routing all calls and acting as the registration server for IP endpoints. Multiple CUCM servers are deployed in a cluster to provide fault tolerance and scale. A typical CUCM cluster can contain up to eight call processing servers and can be spread across multiple geographic locations. In some cases this cluster can grow to contain up to 16 call processing servers. A single CUCM cluster can support a maximum of 40,000 IP endpoints. To scale beyond 40,000 endpoints, administrators can deploy multiple clusters to satisfy both scale and redundancy requirements. Figure 6: Geographically Distributed CUCM Cluster The figure above shows a geographically distributed cluster, with servers residing in New York and Boston forming a single CUCM cluster. This distributed CUCM cluster provides call control service to phones in New York, New Jersey, and Boston. CUCM, like many Cisco UC applications, is deployed using an Open Virtualization Archive (OVA) template and can be configured with different CPU, RAM, and disk sizes, as needed. The OVA file can be imported into vSphere and contains the definition of all attributes required to create the VM. CUCM servers that make up a cluster vary in size, depending on the number of phones they support. The user selects the server size in the VMware OVA deployment dropdown during vSphere import. Consult the Cisco DocWiki for a breakdown of the available CUCM OVA options. 4. Cisco Unified Communications Overview | 14 Virtualizing Cisco Unified Communications Performance of the CUCM server is heavily focused on CPU and RAM, as all call processing is performed in memory. During normal operation, storage is only utilized for writing out logs and call records. 4.2. Voice Messaging: Cisco Unity Connection The Cisco Unity Connection (CUC) application provides voice message and message waiting indicator (MWI) services, as well as basic auto attendant and IVR services. You can enable additional functionality to sync voice mailboxes to Microsoft Exchange inboxes. CUC servers are deployed in active/active pairs to form a cluster supporting up to 20,000 voice mailboxes. Multiple pairs (or clusters) can be joined via digital networking to scale up to 100,000 voice mailboxes. Figure 7: CUC Voicemail Clusters 4. Cisco Unified Communications Overview | 15 Virtualizing Cisco Unified Communications In the figure above, an active/active CUC cluster in New York provides voicemail service to all phones in New York and New Jersey. A separate active/active CUC cluster provides service to phones in Boston. CUC is heavily focused on processing power to perform voice transcoding and speech recognition. In addition, there are large storage requirements to read and write voice messages in real-time on both servers in a pair. More information about the CUC OVA sizes can be found in the Cisco DocWiki. 4.3. Instant Messaging and Presence: CUCM IM and Presence Instant Messaging and Presence (IM&P) provides service to Cisco Jabber endpoints on PC, Mac, Android, or iPhone. Chat messages and Jabber presence status are routed to Cisco Jabber clients through the IM&P server using the SIP and XMPP protocols. The on-hook status of phones is routed through CUCM to the IM&P server for use by the Jabber client contact list. IM&P servers are deployed in pairs that form a subcluster for high availability and load balancing. Up to three IM&P subclusters (six servers) can be deployed with each CUCM cluster. A subcluster can support up to 15,000 users—up to 45,000 in a single cluster when using the largest possible OVA. Details about the IM&P OVA sizes can be found in the Cisco DocWiki. 4. Cisco Unified Communications Overview | 16 Virtualizing Cisco Unified Communications Figure 8: IM&P Subclusters The figure above depicts a single CUCM cluster with two IM&P subclusters. The New York subcluster provides service to Jabber clients in New York and New Jersey, while the Boston subcluster is dedicated to Boston clients. The IM&P server primarily utilizes CPU and RAM for real-time routing of instant messaging. Storage utilization during normal operation focuses on writing logs. 4.4. E911 Location Services: Cisco Emergency Responder Cisco Emergency Responder (CER) is responsible for sending physical location data to the public-safety answering point (PSAP) when users place an emergency call. Local laws require Emergency Responder to provide E911 location services when IP endpoints make emergency 4. Cisco Unified Communications Overview | 17 Virtualizing Cisco Unified Communications calls. UC endpoints can exist on laptops, desktops, IP phones, and wireless handsets. As devices move from one location to another, the CER server is responsible for tracking the current location. During an emergency call the system invokes the CER server to send the calling device's current location to the PSAP operator. Two CER servers form a CER Server Group for high availability; multiple Server Groups form a CER Cluster. Information on the UC application OVA sizes for Emergency Responder can be found in the Cisco DocWiki. Figure 9: CER Server Group The figure above shows a sample CER Server Group split between New York and Boston. The CER Publisher server provides service to all phones, and the Subscriber only takes over if the Publisher is offline. 4. Cisco Unified Communications Overview | 18 Virtualizing Cisco Unified Communications Figure 10: E911 Call Example The figure above shows an example of an E911 call made from New York Phone C. CER identifies the location of the call and passes this information to the PSAP operator. CER primarily uses CPU and RAM to perform call routing and location lookups in real-time, as well as scheduled phone location discovery. Storage utilization consists of emergency call logging and system log files. 4.5. License Management: Prime License Manager Prime License Manager (PLM) provides a central point of control for uploading and managing licenses for Cisco UC components. Each customer license domain requires only one PLM node. A single PLM can manage and distribute licenses for multiple Cisco UC clusters. The resources required for the PLM server are minimal. 4. Cisco Unified Communications Overview | 19 Virtualizing Cisco Unified Communications Table 2: UC Cluster Sizes Product Cisco UC Cluster Size Resource Usage Profile CUCM VMs per cluster: 1–16 Heavy CPU and RAM use Users per cluster: 1–40,000 Light storage use Maximum clusters: unlimited IM&P VMs per IM&P subcluster: 1–2 Heavy CPU and RAM use Subclusters per CUCM cluster: 1–3 Light storage use Users per subcluster: 1–15,000 Users per cluster: 45,000 Maximum clusters: unlimited (1 per CUCM cluster) CUC VMs per cluster: 1–2 Heavy CPU and RAM use Users per cluster: 1–20,000 Large sized storage with medium use Maximum clusters: 10 CER VMs per cluster: 1–2 Heavy CPU and RAM use Users per cluster: 1–40,000 Light storage use Maximum clusters: 10 PLM 1 PLM VM per license domain Light CPU, RAM, and storage use 4. Cisco Unified Communications Overview | 20 Virtualizing Cisco Unified Communications 5. Delivering High Performance and Highly Available Cisco UC When using a virtualized Unified Communications infrastructure, selecting the optimal compute and storage architecture is critical for providing exceptional end user experience. The Nutanix enterprise cloud platform is a hyperconverged solution that combines storage and compute resources into a single, integrated appliance. It provides the manageability benefits of centralized storage while delivering high performance by keeping data local to server resources. IT managers can now deliver Unified Communications services with enterprise storage features without the cost and complexity of SAN and NAS systems. The Nutanix architecture is purpose-built for virtualization and supports popular technologies such as live VM migration, high availability (HA), distributed resource scheduling (DRS), and fault tolerance. Nutanix makes it simple to deploy and scale Cisco Unified Communications clusters that support complex collaboration feature sets. The scalability of the Nutanix cluster can accommodate UC clusters of any size. Entirely new UC components can be added easily to an existing Nutanix cluster without redesigning the underlying storage and compute infrastructure. In short, a UC deployment on Nutanix is flexible and scalable. With the Acropolis Distributed Storage Fabric (DSF), data is written to local flash memory for fast acknowledgment, while read operations are handled locally for reduced latency and fast delivery of UC data. Data is strategically written to other Nutanix nodes to provide automatic fault tolerance in case of hardware failure. In addition, the DSF includes advanced compression and deduplication technology to optimize storage utilization and capacity savings. The Nutanix solution is also hypervisor agnostic, allowing customers to use the hypervisor of their choice, including Microsoft Hyper-V, VMware ESXi, or AHV. Cisco Unified Communications is supported exclusively on VMware ESXi, so this document tailors all hypervisor-specific details to VMware. Nutanix is certified in the VMware hardware compatibility list, as well as the VMware Storage hardware compatibility list (HCL). These certifications allow administrators to rest easy knowing the platform is designed for interoperability with VMware features and is fully supported by both Nutanix and VMware. Nutanix delivers a highly available, easy-to-manage, fault-tolerant platform for Cisco Unified Communications applications. The fault tolerance and self-healing of the underlying DSF infrastructure allow critical UC services to be provided to end users without interruption. Moreover, Nutanix ease of installation and setup cuts down UC deployment times by slashing the interval needed to configure storage infrastructure. The Nutanix Prism Management interface simplifies infrastructure management by providing a transparent and user-friendly view of the 5. Delivering High Performance and Highly Available Cisco UC | 21 Virtualizing Cisco Unified Communications entire Nutanix cluster. VM-centric to the core, Nutanix is the ideal compute and storage platform for highly critical Cisco Unified Communications VMs. 5. Delivering High Performance and Highly Available Cisco UC | 22 Virtualizing Cisco Unified Communications 6. Designing and Architecting Cisco UC on Nutanix Unified Communications deployments are sized based on the user counts and device counts for each UC application. A simple example would be a 1,000 employee organization in which each user has one physical phone, one voice mailbox, and one Jabber client for Instant Messaging & Presence. In this instance, sizing input would be as follows: • CUCM: 2,000 Devices, 2,000 Lines The combination of 1,000 physical phones and 1,000 Jabber devices results in 2,000 devices that require call control service from the Communications Manager. It's important to identify the expected "busy hour call attempts" per phone to size appropriately. A phone that places two calls and receives one call during the busiest hour of the day has a BHCA of three, for example. Administrators can use historical phone usage reports to take the total call volume in the busiest hour and then divide by the number of users. Having real data is vital, as BHCA can vary widely between users and groups. Increased BHCA results in larger VM requirements or lower user-to-VM density. • IM&P: 1,000 Jabber Clients Each Cisco Jabber client connects to the IM & Presence server for XMPP messaging and presence information. IM&P sizing is determined by user count and the average size of the contact list—in this case, 1,000 users, assuming standard contact list sizes. • Cisco Unity Connection: 1,000 Voice Mailboxes 1,000 mailboxes are required to provide a voice mailbox for every user. Additional features, such as interactive IVRs or shared mailboxes, would increase the mailbox count. • Cisco Emergency Responder: 2,000 Tracked Devices Cisco Emergency Responder can track all physical phones via SNMP switch port discovery. Cisco Jabber clients are tracked via IP subnet or by user-provided locations. 6.1. Cisco Collaboration Sizing Tool Working with a Cisco partner, input the user count information into the Cisco Collaboration Sizing Tool. Additional details specific to UC features, such as computer telephony integration (CTI), shared lines, and other features, may be gathered to calculate size more accurately. The output of the Cisco Collaboration Sizing Tool determined the number and type of VMs recommended for the deployment. For this example, we use a 1:1 call processing redundancy model, meaning that each call processing server has a backup server capable of supporting full phone capacity during failure. 6. Designing and Architecting Cisco UC on Nutanix | 23 Virtualizing Cisco Unified Communications General guidelines for Cisco UC Sizing can be found in the Solution Reference Design Guide for Collaboration. For a more exact calculation that takes into account expected call volume, use the Cisco Collaboration Sizing Tool available to Cisco partners. Each UC application is listed with the number of VMs required and the OVA template size. For convenience, the table lists the actual size of the OVA in vCPUs, RAM, and storage. This information may change in future versions of Cisco UC products. To find the exact specifications for each product and each OVA size, refer to the DocWiki Virtualization page. The OVAs delivered by Cisco for the purposes of Unified Communications are fixed and cannot be modified, or UC application installation may fail. The purpose of the OVA is to ensure the best possible end user experience by enforcing rules regarding resource reservation, disk sizes, and oversubscription (these rules can be found in the Cisco Virtualization Sizing Guidelines page). An added benefit is that the OVA greatly simplifies creating VMs for Cisco UC. The following section shows two sample layouts of Unified Communications design on Nutanix: one for 1,000 users and the other for 30,000. 6. Designing and Architecting Cisco UC on Nutanix | 24 Virtualizing Cisco Unified Communications 7. Sample Deployments of Cisco UC on Nutanix 7.1. 1,000 User Deployment Arranging Cisco UC applications on the nodes in a Nutanix system can be accomplished in two steps. First, establish the number, type, and size of the VMs required for the solution using the Cisco Collaboration Sizing Tool, described above. This tool’s output provides the VMs per application, vCPU, RAM, storage requirements, and the number of supported users per application OVA. Note that the number of VMs is driven by both performance and redundancy requirements. As a result, the OVA size may not correspond exactly to the number of supported users. The table below contains sample output for 1,000 users. Table 3: 1,000 User Sizing Tool Output UC Product Product OVA CUCM 2,500 Users IM&P 1,000 Users CUC 1,000 Users CER 1,000 Users PLM Per VM Total # of VMs vCPUs RAM (GB) Storage (GB) vCPUs RAM (GB) Storage (GB) 2 1 4 80 2 8 160 2 1 2 80 2 4 160 2 2 4 160 4 8 320 2 1 4 80 2 8 160 1 1 4 50 1 4 50 11 32 850 Total 7. Sample Deployments of Cisco UC on Nutanix | 25 Virtualizing Cisco Unified Communications Using the sizing information in the table above, we can lay out the VMs on individual nodes in the Nutanix system. We've selected the 16-core Nutanix 3350 as a deployment example. This model allows ample room for the UC applications, using an N+1 hypervisor strategy in case of Nutanix node hardware failure. We only show three nodes below, but generally recommend four nodes in any configuration for higher availability. The fourth node also allows more non-UC workloads to be deployed in the configuration. Hardware Components Required and Rack Units: 1,000 Users This example uses just four rack units (RU), including all compute, storage, and networking infrastructure. Table 4: 1,000 User Hardware Requirements Component Quantity Rack Units Nutanix system 1 2 10 Gb top-of-rack switches 2 2 Total rack units 4 Nutanix Controller VM Configuration: 1,000 Users The Nutanix Controller VM (CVM) uses a default configuration with eight allocated vCPUs, four of which are reserved, and 16 GB RAM. VM Placement for Application-Level Availability: 1,000 Users VMs are placed on different Nutanix nodes to separate critical redundant services, such as the primary and secondary Communications Manager. Cisco UC does not support VMware DRS, so administrators must perform and maintain VM placement manually. 7.2. 1,000 User Deployment VM Placement The following diagram shows the 1:1 vCPU to physical core mapping Cisco requires, as well as the staggered VM placement. The Nutanix platform is capable of running the 1,000 user UC workload with ample room for N+1 redundancy. Utilize the spare capacity to run other VM workloads. Although Cisco requires that UC VM vCPUs not be oversubscribed, the Nutanix CVM only reserves four vCPUs out of the eight provisioned. Depending on the system load, it may be 7. Sample Deployments of Cisco UC on Nutanix | 26 Virtualizing Cisco Unified Communications possible to use the four unreserved vCPUs for non-Cisco UC workloads. When access to the DSF is not IOPS intensive, you can use these spare CPU cores. All free, unused resources are shaded. Figure 11: 1,000 User VM Placement with Cores Reserved and VMs Not Pinned 7.3. 30,000 User Deployment Planning and design for a larger number of users on Nutanix follows the same process as the small UC deployment. Provide the user count, BHCA, and applications required as input to the Cisco Collaboration Sizing Tool. A 30,000-user deployment would yield output similar to that in the following table. 7. Sample Deployments of Cisco UC on Nutanix | 27 Virtualizing Cisco Unified Communications Table 5: 30,000 User Sizing Tool Output UC Product Per VM Total # of VMs vCPUs RAM (GB) Storage (GB) vCPUs RAM (GB) Storage (GB) 11 2 6 110 22 66 1,210 4 4 8 2x 80 16 32 640 4 8 8 2x 300 32 32 2,400 30,000 Users 2 2 6 2x 80 4 12 320 PLM 1 1 4 50 1 4 50 75 146 4,620 Product OVA CUCM 7,500 Users IM&P 15,000 Users CUC 20,000 Users CER Total In this example we have chosen two blocks, containing seven 24-core nodes for maximum VM density. Other hardware models and core densities are available. N+1 hardware redundancy is achieved with an additional node to tolerate hardware failure. Critical UC application roles are separated manually between Nutanix nodes as required. Hardware Components Required and Rack Units: 30,000 Users This example uses just six rack units (RU), including all compute, storage, and networking infrastructure. Table 6: 30,000 User Hardware Requirements Component Quantity Rack Units Nutanix system 2 4 7. Sample Deployments of Cisco UC on Nutanix | 28 Virtualizing Cisco Unified Communications Component 10 Gb top-of-rack switches Quantity Rack Units 2 2 Total rack units 6 Nutanix Controller VM Configuration: 30,000 Users The Nutanix Controller VM (CVM) uses a default configuration with eight allocated vCPUs, four of which are reserved, and 16 GB RAM. VM Placement for Application-Level Availability: 30,000 Users VMs are placed on different Nutanix nodes to separate critical redundant services, such as the primary and secondary Communications Manager. Cisco UC does not support VMware DRS, so administrators must perform and maintain VM placement manually. Note that this Nutanix cluster can be started with a minimum of three Nutanix nodes, with additional nodes added as UC components are scaled up over time. 7. Sample Deployments of Cisco UC on Nutanix | 29 Virtualizing Cisco Unified Communications 7.4. 30,000 User Deployment VM Placement Figure 12: 30,000 User VM Placement: Block 1 7. Sample Deployments of Cisco UC on Nutanix | 30 Virtualizing Cisco Unified Communications Figure 13: 30,000 User VM Placement: Block 2 7. Sample Deployments of Cisco UC on Nutanix | 31 Virtualizing Cisco Unified Communications 8. Nutanix Best Practices for Cisco Unified Communications The following sections discuss Nutanix best practices for a Cisco Unified Communications deployment. Nutanix and Cisco UC best practices are summarized in the Best Practices Checklist included in the Appendix. 8.1. Nutanix Platform Guidance Nutanix has a wide range of node types, all of which vary in compute, storage capacity, and performance to suit almost any virtual workload. All Nutanix models meet the full processor requirements specified by Cisco UC. Therefore, any node type is a candidate for running Cisco UC. The exact model selected depends on both the disk space required by the combined UC applications, as well as the number of vCPUs required. The results from the Cisco Collaboration Sizing Tool and the specified OVA sizes drive this decision. Consult your Nutanix Systems Engineer to find the Nutanix node that’s the right fit for your enterprise. The NX-3460-G4 block shown in the figure below is an example of a typical configuration. The block has four nodes (A–D), and each node has the following specifications: • 2x Intel Ivy Bridge E5-2680v3, 24 cores, 2.5 GHz • Up to 512 GB RAM • 2x SSDs (480 GB–1.6 TB option) • 4x 1 TB SATA HDDs (2 TB option) • 2x 10 Gb SFP+ network interfaces (2 or 4 port add-on available) • 2x 1 Gb network interfaces • 1x out-of-band management interface (IPMI) • 4 nodes per 2RU form factor (shown) 8. Nutanix Best Practices for Cisco Unified Communications | 32 Virtualizing Cisco Unified Communications Figure 14: Nutanix System: Block of Four Nodes The NX-3000 can be mixed with other node types to form a single Nutanix cluster and thereby maximize performance and operational efficiency. This allows users to add and balance storage and compute capacity in a Nutanix cluster to suit their particular needs. 8.2. Virtualization and Compute Configuration Host and VM Sizing (vCPUs, RAM, Storage) Each UC VM is deployed using an OVA file downloaded from Cisco. These OVA files specify the CPU, RAM, and storage requirements of the UC application. Ensure that a 1:1 vCPU to physical core mapping is maintained for all Cisco UC VMs on the Nutanix cluster. Hyperthreading (HT) should be enabled, but CPU mapping should be done based on physical, and not logical, cores. Settings for HT sharing are not specified in the Cisco documentation, so this should be left at the default setting. CPU reservations are created as the OVA requires. RAM is allocated and reserved during OVA deployment. No additional configuration is required. The OVA template also dictates each UC application’s storage requirements, creating the correct number of disks at the appropriate size for each VM during deployment. VMware Feature Support Each Cisco UC application supports specific VMware features, such as DRS, HA, Site Recovery Manager (SRM), or Snapshots. Consult the Cisco DocWiki for complete and up-to-date 8. Nutanix Best Practices for Cisco Unified Communications | 33 Virtualizing Cisco Unified Communications information on supported VMware features with Cisco Unified Communications, as Cisco constantly updates this list. Not all Cisco UC applications support the same VMware features. 8.3. Storage Configuration A Nutanix cluster is created at each physical site. Cluster size ranges from at least three to, theoretically, an unlimited number. However, when using VMware, we recommend a 32-node maximum; this number aligns with VMware vSphere’s maximum cluster size of 32 ESXi hosts, while also maintaining a manageable fault domain. Additional nodes and clusters can be supported at each site as necessary. Advanced Nutanix features such as compression and deduplication can provide greatly improved storage capacity for workloads with shared, or easily compressible, data. Cisco Unified Communications VMs are unlikely to see these gains due to the nature of the data, so we do not recommend compression and deduplication in this context. These features can be enabled, however, if other workloads require them. Table 7: Nutanix Cluster and Storage Recommendations Cluster Size 3–32 nodes per cluster, one or more clusters per site Storage Pool Single storage pool with all SSD and HDD devices Container Single container with redundancy factor of 2 Used to host UC virtual disk files (VMDKs) Compression Not recommended for UC, but allowed Deduplication Not recommended for UC, but allowed VM Placement Stagger critical UC functions on multiple nodes to provide resiliency and spread workload Mixed Workloads Additional workloads can be virtualized on the nodes to maximize CPU and RAM utilization efficiency Most UC applications support coresidency with non-UC VMs. Follow Cisco Coresidency Guidelines 8. Nutanix Best Practices for Cisco Unified Communications | 34 Virtualizing Cisco Unified Communications 8.4. Networking Designed for true linear scaling, Nutanix recommends a leaf-spine network architecture. A leafspine architecture consists of two network tiers: an L2 leaf and an L3 spine based on 10 GbE or 40 GbE and nonblocking switches. This architecture maintains consistent performance without any throughput reduction due to a static maximum of three hops from any node in the network. The figure below shows a scale-out leaf-spine network architecture design. This architecture provides 20 Gb active throughput from each node to its L2 leaf, and scalable 80 Gb active throughput from each leaf to spine switch, providing scale from one Nutanix block to thousands without any impact to available bandwidth. Each node’s 10 GbE NIC is attached to a separate top-of-rack switch; the type of vSphere switch determines the load balancing and NIC teaming option. Nutanix recommends using “Route based on originating virtual port” with the vSphere Standard Switch and “Route based on physical NIC Load” with the vSphere Distributed Switch. No advanced switching configuration such as Link Aggregation Control Protocol (LACP), Cisco EtherChannel, or HP teaming is required for the Nutanix node. 8. Nutanix Best Practices for Cisco Unified Communications | 35 Virtualizing Cisco Unified Communications Figure 15: Nutanix Leaf-Spine Network Architecture Additional Nutanix networking best practices for VMware vSphere can be found in the Tech Note VMware vSphere Networking on Nutanix. Nutanix recommends always using the 10 Gbps network interfaces for the Nutanix Controller VM, hypervisor, and guest VM network connectivity. However, the 1 Gbps network infrastructure is supported for a maximum of eight Nutanix nodes and can be used if no 10 Gbps network infrastructure is available. 8.5. High Availability Cisco Unified Communications is designed for high availability at the application level. Every customer design includes additional VMs for fault tolerance and load balancing. Automatic VMware tools such as DRS and fault tolerance are not recommended for use with Cisco Unified Communications. Recovery from hardware failure should be handled instantaneously by 8. Nutanix Best Practices for Cisco Unified Communications | 36 Virtualizing Cisco Unified Communications geographic or local redundancy. Administrators can perform manual recovery to start failed VMs on new hardware if necessary. The DSF provides the ability to survive disk failure without impact to running VMs, as all data is written redundantly to peer nodes. Complete node failure would only interrupt service to VMs running on that node. VMs can be restarted on another node, or VMs can be restarted when the Nutanix node is running again. 8.6. Backup and Disaster Recovery Although Nutanix protection domains could be used for disaster recovery, the recommended best practice is to plan for disaster recovery at the Cisco UC application level with redundant sites and redundant UC VMs at those sites. The immediate failover should be handled at the application level within seconds to ensure zero interruption and downtime for UC endpoints. In case of hardware failure, the UC applications can be restarted manually on another Nutanix node if required to restore service. Backup and restore of a Cisco UC application is only supported through UC application-specific backup mechanisms. Cisco does not support any third-party backup mechanisms at this time. Loss of VM data is prevented by the DSF's synchronous replication on write technology, which defaults to a replication factor of two. 8.7. UC Application Installation Most Cisco UC applications are installed in a two-part process. First, the VM is created from the downloaded Cisco OVA file. Second, installation starts by mounting an ISO in the DVD drive of the newly created VM, and booting the VM. Location of this installation ISO is critical to installation performance. The UC installation ISO should be copied to the same Nutanix container as the UC application being installed to take advantage of data locality and shadow clones. This is especially important if multiple UC applications are being installed at the same time from the same ISO image. In the VMware vSphere Client select "Datastore ISO File" when connecting the DVD drive to the UC VM. 8.8. Cisco UC Virtualization Requirements Cisco specifies detailed requirements that must be met before a third party hardware or storage solution can be used to deploy Cisco Unified Communications. Nutanix complies with all requirements detailed on the Cisco DocWiki site. Because Cisco frequently updates the DocWiki site, this list is subject to change. The following list highlights the key areas of Nutanix compliance with Cisco UC requirements. 8. Nutanix Best Practices for Cisco Unified Communications | 37 Virtualizing Cisco Unified Communications CPU Oversubscription CPU oversubscription is not allowed when implementing virtualized Cisco UC. Plan for a 1:1 mapping of vCPUs to physical cores and deploy UC VMs using the Cisco supplied OVA. CPU reservations are created when the OVA is deployed. Hyperthreading should be enabled, but mapping of vCPUs must be performed based on the number of physical cores. Full Performance CPU Architecture UC-approved CPUs are specified by Cisco per CPU architecture and speed. Only approved processors on this list are supported for UC applications. All Nutanix models meet the “Full Performance” criteria. To find a full list of all Nutanix node specifications visit the Nutanix.com site and navigate to Hardware Platforms. Since processor models are upgraded frequently it is imperative to check the list of CPU architectures and speeds supported by Cisco, and compare that to the different models used in Nutanix nodes. Memory Oversubscription Memory must be assigned on a dedicated basis to Cisco UC VMs and cannot be oversubscribed. This is easy to ensure on Nutanix by deploying the Cisco provided OVA and planning VM layout accordingly. Memory reservation is performed when the OVA is deployed. Disk Oversubscription Thin provisioning can be used as long as disk space is available when a VM requires it. By design, Unified Communications servers eventually use all available disk space for logging and other functions. The Cisco recommended best practice is to use thick provisioning lazy zeroed to ensure smooth operation of critical UC VMs. This precaution is taken to avoid running out of disk space when a critical UC application requires it. Thin provisioning is an acceptable method when UC VMs are guaranteed to never run out of disk space during operation. NAS and NFS Latency NFS is a supported mechanism for storage as long as guest average latency (GAVG), a combination of disk and kernel latency, is less than 24 ms as reported in esxtop GAVG. More information about vSphere storage performance troubleshooting can be found in the VMware vSphere blog. Most importantly, any storage technology is supported as long as the capacity, performance, and latency requirements of the individual UC applications are met. Storage Performance (IOPS) Each Cisco UC application has an IOPS profile that can be found on the Cisco DocWiki site. Consult the OVA details for each UC application listed above. These performance numbers can be used to get a general idea of the load a UC application would place on the storage system. 8. Nutanix Best Practices for Cisco Unified Communications | 38 Virtualizing Cisco Unified Communications Use these numbers as a guideline for placing UC applications on Nutanix nodes to balance the workload over multiple Nutanix nodes. In general, most Cisco UC applications are not disk intensive. Note: CPU core requirements primarily drive UC VM placement, rather than storage or memory requirements. VMware HCL and VMware Storage HCL Cisco UC can be deployed on any third-party hardware and storage as long as the product meets all of the above requirements and is on the VMware Hardware Compatibility List (HCL). Nutanix hardware is certified by VMware and is on the VMware HCL and VMware Storage HCL. 8. Nutanix Best Practices for Cisco Unified Communications | 39 Virtualizing Cisco Unified Communications 9. Conclusion Cisco Unified Communications can be successfully deployed on the Nutanix platform, delivering high availability and true linear scalability. The Nutanix platform eliminates traditional SAN and NAS complexity while providing a highly resilient storage and compute infrastructure with a small datacenter footprint and full Cisco UC support. Cisco's adoption of virtualization for Unified Communications provides an amazing opportunity for UC administrators. The synergy between virtualized UC and Nutanix, an infrastructure platform purpose-built for virtualization, can be leveraged to build the best enterprise-class communications tools with the lowest administrative overhead. Nutanix is the optimal compute and storage platform for critical real-time UC applications, allowing Unified Communications cluster scaling as needed during a deployment, instead of building the entire infrastructure up front for only a small number of users. This flexibility makes Nutanix a perfect fit for on-premise, private-cloud, or “UC-as-a-Service” deployment models where the ability to scale over time is crucial. For Nutanix or UC on Nutanix questions please use our Nutanix Next online community: next.nutanix.com. 9. Conclusion | 40 Virtualizing Cisco Unified Communications Appendix Best Practices Checklist 1. General Suggestions a. Perform a current and future state analysis to identify UC cluster sizes and future growth. b. Spend time up front to architect a solution that meets both current and future needs. Use Nutanix scalability to grow as needed. c. Design to deliver consistent performance and reliability, as UC workloads need real-time highly available performance. 2. Nutanix Node and Platform Guidance a. The NX-3000 platform is a great fit in terms of performance and capacity for all UC workloads. Pick the platform that best suits all enterprise workloads. b. All Nutanix nodes currently meet Cisco UC processor specifications and can be used if other workloads require a specific platform. c. Mixing node types within a cluster is a viable option for running various workloads. 3. Compute and Memory a. Download and use the OVA for each Cisco UC Application. b. Perform 1:1 mapping between vCPUs and physical cores. c. Perform 1:1 RAM mapping. d. Follow the Supported VMware Features for Cisco UC DocWiki guide. 4. Storage and Nutanix Clustering a. Minimum Nutanix cluster size of 3 nodes. b. Maximum Nutanix cluster size of 32 nodes to limit fault domain and mirror VMware vSphere cluster maximum size. c. Each physical site requires a Nutanix cluster. d. Deploy all HDD and SSD in one storage pool. e. Deploy a single container with RF=2. f. Compression and deduplication are not recommended, but are allowed. g. Stagger critical UC applications to spread load and keep primary and secondary application components on different hardware. h. Use free processor space for other non-UC workloads to maximize system efficiency. 5. Networking a. Use a nonblocking leaf-spine 10 Gbps or 40 Gbps architecture for guaranteed throughput and low latency. Appendix | 41 Virtualizing Cisco Unified Communications b. Attach each Nutanix node to two separate top-of-rack switches. c. Use “Route based on originating virtual port” with the vSphere Standard Switch. d. Use “Route based on physical NIC load” with the vSphere Distributed Switch. e. LACP, Etherchannel, or MLAG are not required to the Nutanix nodes. f. Refer to the VMware vSphere Networking on Nutanix guide for complete details. 6. High Availability a. Use Cisco UC application failover and recovery concepts instead of VMware features like high availability and fault tolerance. 7. Backup and Disaster Recovery a. Use Cisco UC application-level backup and disaster recovery methods as the primary backup and DR solution. Full VM copies can be useful, but if you perform a full VM restore, you may need to rebuild database replication in the application. b. Acropolis DSF RF2 protects against disk and node failure. 8. UC Application Installation a. To take advantage of shadow clones, the Cisco UC installation ISO should be copied to the same Nutanix container (vSphere Datastore) as the target VM before VM installation. 9. Cisco UC Virtualization Requirements a. Use and refer to the Cisco DocWiki site often as these requirements are constantly changing. b. Do not oversubscribe CPU, memory, or disk. Use 1:1 mappings. Thick provisioning lazy zeroed is recommended but thin provisioning is allowed. c. Monitor CAVG latency in “esxtop” to ensure NFS command latency is within Cisco requirements. d. Use the IOPS measurements for each Cisco UC application in DocWiki to spread workloads effectively across the Nutanix cluster. References 1. 2. 3. 4. 5. 6. 7. 8. 9. CUCM OVA Sizes CUC OVA Sizes IM&P OVA Sizes CER OVA Sizes All OVA Sizes Cisco Virtualization Sizing Guidelines UC Virtualization Storage System Guidelines Supported VMware Features for Cisco UC Cisco Coresidency Guidelines Appendix | 42 Virtualizing Cisco Unified Communications 10. 11. 12. 13. 14. 15. 16. VMware vSphere Networking on Nutanix Cisco UC Virtualization Support At a Glance Cisco UC Approved Processors Nutanix Technical Solutions Storage Performance Requirements vSphere Storage Performance Troubleshooting Nutanix VMware HCL Meet the Author Jason Burns is an NPP-certified Staff Solutions Architect at Nutanix, Inc. and CCIE Collaboration #20707. He designs, tests, and documents virtual workloads on the Nutanix platform, creating solutions that solve critical business problems. Jason has designed and supported Unified Communications infrastructure in the enterprise for the past decade, deploying UC to connect hundreds of thousands of end-users. Outside of his day job, he has an unusual passion for certificates, security, and motorcycles. Follow Jason on Twitter @bbbburns. About Nutanix Nutanix makes infrastructure invisible, elevating IT to focus on the applications and services that power their business. The Nutanix enterprise cloud platform leverages web-scale engineering and consumer-grade design to natively converge compute, virtualization, and storage into a resilient, software-defined solution with rich machine intelligence. The result is predictable performance, cloud-like infrastructure consumption, robust security, and seamless application mobility for a broad range of enterprise applications. Learn more at www.nutanix.com or follow up on Twitter @nutanix. Appendix | 43 Virtualizing Cisco Unified Communications List of Figures Figure 1: Nutanix Enterprise Cloud Platform.....................................................................7 Figure 2: Information Life Cycle Management.................................................................. 9 Figure 3: Overview of the Nutanix Architecture...............................................................10 Figure 4: Data Locality and Live Migration......................................................................11 Figure 5: Cisco UC Overview..........................................................................................13 Figure 6: Geographically Distributed CUCM Cluster....................................................... 14 Figure 7: CUC Voicemail Clusters.................................................................................. 15 Figure 8: IM&P Subclusters.............................................................................................17 Figure 9: CER Server Group...........................................................................................18 Figure 10: E911 Call Example........................................................................................ 19 Figure 11: 1,000 User VM Placement with Cores Reserved and VMs Not Pinned..........27 Figure 12: 30,000 User VM Placement: Block 1............................................................. 30 Figure 13: 30,000 User VM Placement: Block 2............................................................. 31 Figure 14: Nutanix System: Block of Four Nodes........................................................... 33 Figure 15: Nutanix Leaf-Spine Network Architecture...................................................... 36 44 Virtualizing Cisco Unified Communications List of Tables Table 1: Document Version History.................................................................................. 6 Table 2: UC Cluster Sizes...............................................................................................20 Table 3: 1,000 User Sizing Tool Output..........................................................................25 Table 4: 1,000 User Hardware Requirements.................................................................26 Table 5: 30,000 User Sizing Tool Output........................................................................28 Table 6: 30,000 User Hardware Requirements...............................................................28 Table 7: Nutanix Cluster and Storage Recommendations.............................................. 34 45