Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 Last Updated: October 24, 2014 Building Architectures to Solve Business Problems 2 Cisco Validated Design About the Author ShivaKumar Shastri, Technical Marketing Engineer, Cisco Systems, Inc. Shivakumar Shastri is a Technical Marketing Engineer focusing on Unified Computing Service (UCS) performance and solutions engineering. Shivakumar has eighteen years of experience in multiple areas of IT infrastructure services. 3 About Cisco Validated Design (CVD) Program The CVD program consists of systems and solutions designed, tested, and documented to facilitate faster, more reliable, and more predictable customer deployments. For more information visit http://www.cisco.com/go/design-zone. ALL DESIGNS, SPECIFICATIONS, STATEMENTS, INFORMATION, AND RECOMMENDATIONS (COLLECTIVELY, “DESIGNS”) IN THIS MANUAL ARE PRESENTED “AS IS,” WITH ALL FAULTS. CISCO AND ITS SUPPLIERS DISCLAIM ALL WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE WARRANTY OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT OR ARISING FROM A COURSE OF DEALING, USAGE, OR TRADE PRACTICE. IN NO EVENT SHALL CISCO OR ITS SUPPLIERS BE LIABLE FOR ANY INDIRECT, SPECIAL, CONSEQUENTIAL, OR INCIDENTAL DAMAGES, INCLUDING, WITHOUT LIMITATION, LOST PROFITS OR LOSS OR DAMAGE TO DATA ARISING OUT OF THE USE OR INABILITY TO USE THE DESIGNS, EVEN IF CISCO OR ITS SUPPLIERS HAVE BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. THE DESIGNS ARE SUBJECT TO CHANGE WITHOUT NOTICE. USERS ARE SOLELY RESPONSIBLE FOR THEIR APPLICATION OF THE DESIGNS. THE DESIGNS DO NOT CONSTITUTE THE TECHNICAL OR OTHER PROFESSIONAL ADVICE OF CISCO, ITS SUPPLIERS OR PARTNERS. USERS SHOULD CONSULT THEIR OWN TECHNICAL ADVISORS BEFORE IMPLEMENTING THE DESIGNS. RESULTS MAY VARY DEPENDING ON FACTORS NOT TESTED BY CISCO. The Cisco implementation of TCP header compression is an adaptation of a program developed by the University of California, Berkeley (UCB) as part of UCB’s public domain version of the UNIX operating system. All rights reserved. Copyright © 1981, Regents of the University of California. Cisco and the Cisco logo are trademarks or registered trademarks of Cisco and/or its affiliates in the U.S. and other countries. To view a list of Cisco trademarks, go to this URL: http://www.cisco.com/go/trademarks. Third-party trademarks mentioned are the property of their respective owners. The use of the word partner does not imply a partnership relationship between Cisco and any other company. (1110R). Any Internet Protocol (IP) addresses and phone numbers used in this document are not intended to be actual addresses and phone numbers. Any examples, command display output, network topology diagrams, and other figures included in the document are shown for illustrative purposes only. Any use of actual IP addresses or phone numbers in illustrative content is unintentional and coincidental. Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 © 2014 Cisco Systems, Inc. All rights reserved. About Cisco Validated Design (CVD) Program 4 Acknowledgments Acknowledgments The author would like to acknowledge the following for their support and contribution to the design, validation and creation of this Cisco Validated Design (CVD): For support and contribution to the design, validation and creation of Cisco Validation Design (CVD), we would like to thank: • Mehul Bhatt—Cisco • Vijay Durairaj—Cisco • Jeff Fultz—Cisco • Muhammad Ashfaq—Cisco • Bathumalai Krishnan—Cisco • Brent Slone—EMC • Kevin Phillips—EMC About Cisco Validated Design (CVD) Program 5 Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 Introduction IT departments have embraced efficiencies such as hardware consolidation and agility brought about by virtualization and have looked to extend such efficiencies in an agnostic manner to platforms that are application ready. Since all workloads cannot or will not be virtualized on a hypervisor, it is also necessary to extend essential IaaS features of agility and measured self-services to other virtual and non-virtual environments. Capabilities that will allow for the easy introduction of such an application ready and platform independent approach will lead to a more cost-effective and inclusive IT-as-a-Service (ITaaS) Cloud. Cloud computing requires automation and self-service mechanisms that allow users to consume infrastructure without manual intervention for provisioning or configuration of pooled resources. The combination of standardization, workflow automation and self-service offered in a secure manner by UCS Director on a VSPEX platform, gives businesses the opportunity to offer IT-asa-service on shared platforms. This Cisco Validated Design (CVD) leverages capabilities of UCS Director to deploy a multi-tenant IaaS cloud platform on EMC VSPEX. Background IaaS is a Cloud service model where compute resources are delivered as a service rather than a product. Due to the nature of delivery and capabilities expected and provided, cloud computing offers a value proposition that is different from traditional enterprise IT environments. Virtual instances can be provisioned and terminated more quickly while sharing resources. The consumer can therefore expect to be billed only for resources used without incurring steep initial capital costs or hiring a dedicated IT department. For the provider, since the Cloud can reside in a remote location with a lower cost structure, a centralized model which can provide greater economies of scale is feasible. However, a standard implementation of an IaaS platform requires certain key features to be available. These features include self-service provisioning, a means of billing for services used and security to ensure appropriate access to data. Any shared platform, including Cloud, opens up access to key resources such as Infrastructure, Users and Applications. Ensuring the consistent and correct delivery of data on a shared platform comes with increased risk and complexity. System consolidation efforts have also accelerated the movement toward Corporate Headquarters: Cisco Systems, Inc., 170 West Tasman Drive, San Jose, CA 95134-1706 USA Copyright © 2014 Cisco Systems, Inc. All rights reserved. Objective and Benefits co-hosting on integrated platforms, and the likelihood of compromise is increased in a highly shared environment. This situation presents a need for enhanced security and an opportunity to create a framework and platform that instills trust. Many enterprises and IT service providers are developing cloud service offerings for public and private consumption. Regardless of whether the focus is on public or private cloud services, these efforts share several common objectives: • Cost-effective use of capital IT resources through co-hosting • Better service quality through virtualization features • Increased operational efficiency and agility through automation Enabling enterprises to migrate such environments to cloud architecture requires the capability to provide customer confidentiality while delivering the management and flexibility benefits of shared resources. Both private and public cloud providers must secure all customer data, communication and application environments from unauthorized access. Objective and Benefits This document illustrates the design and deployment steps required for implementing an IaaS solution using Cisco UCS Director (UCSD) 5.0 on EMC VSPEX platform consisting of UCS compute, Nexus switches and VNX 5400 storage array. The hypervisor used for virtual machines is VMware 5.5. The solution implemented as proposed provides for an enterprise Private Cloud (ePC) which can be hypervisor/OS agnostic and application ready. Standardized integration points between UCSD and other third-party tools for trouble-ticketing, notification and event monitoring functions lead to a cohesive and complete IaaS solution. Most Cisco UCS Director features covered in this Cisco Validated Document (CVD) are available in a platform agnostic manner. Features such as self-service portal, monitoring, chargeback for billing, orchestration/automation and Role-Based Access Control (RBAC) lead to benefits such as agility, efficiency and cost savings while providing necessary levels of security. Configuration details unique to this deployment are mentioned while VSPEX platform deployment procedure is with reference to an earlier CVD consisting of similar components. This end-to-end ePC solution takes full advantage of unified infrastructure components and Cisco UCS Director device support to provide provisioning, monitoring and management of the infrastructure by consumers. It is beyond the scope of this document to consider performance related details pertaining to the platform. Audience The reader of this document is expected to have the necessary training and background to install and configure VMware vSphere 5.5, EMC VNX 5400 series storage array, Cisco Nexus 5548UP and Nexus 1000v switches, Cisco UCS 5108 blade chassis with Cisco Unified Computing Systems Manager (UCSM). Both external and internal references are provided where applicable and it is recommended that the reader be familiar with these documents. Readers are also expected to be familiar with infrastructure and database security policies of customer installation. The intended audience of this document includes executives, partners, system architects and cloud administrators of IT environments who prefer to implement or use IaaS platform with Cisco UCS Director. Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 7 Key components Key components The Cisco solution for implementing an Iaas solution consists of the EMC VSPEX platform underneath the Cisco management suite, as is shown below: Figure 1 Block Diagram of Cisco Cloud Solution The Cisco Cloud solution integrates the best of Cisco’s hardware and management suite with EMC and VMware products. This accelerates the implementation and adoption process of the cloud infrastructure. The architecture provides sufficient flexibility to allow for customer choice while ensuring compatibility and support for the entire stack. The This cloud solution is applicable to customers who wish to preserve their investment and to those who want prefer to build out new infrastructures that are dedicated to a cloud. This The Cisco cloud solution takes provides the advantage of the a strong integration between Cisco and EMC VSPEX products/technologies with UCS Director. The key components of the Cisco cloud solution is, as shown in the following figure. Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 8 Key components Figure 2 Key Components of Cisco Cloud Solution Following Table 1 provides the datastore size for various EMC architectures, EMC VNX 5400, EMC VNX 5600, and EMC VNX 5800 shown in Figure 2. Table 1 Datastores for Different Scales Parameters 300VMs 600VMs 1000VMs Storage array VNX5400 VNX5600 VNX5800 Disk capacity and type 600 GB SAS 600 GB SAS 600 GB SAS Number of disks 110 220 360 RAID type RAID 5 groups RAID 5 groups RAID 5 groups Fast VP config 6 x 200 GB Flash Drives 10 x 200 GB Flash Drives 16 x 200 GB Flash Drives Hot spares 4 x 600 GB SAS 8 x 600 GB SAS 12 x 600 GB SAS 1 x 200 GB Flash 1 x 200 GB Flash 1 x 200 GB Flash Storage Virtualization NFS is a cluster file system that provides UDP based stateless storage protocol to access storage across multiple hosts over the network. It is preferable to deploy virtual machine files on shared storage to take advantage of VMware VMotion, VMware High Availability™ (HA), and VMware Distributed Resource Scheduler™ (DRS). This is considered a best practice for mission-critical deployments. Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 9 Cloud Overview & Considerations Data center virtualization and cloud management The UCS Director enables customized, self-service provisioning and life cycle management of cloud services that comply with established business policies. UCS Director provides a secure portal where authorized administrators, developers, and business users can request for new IT services and manage existing computer resources from predefined user-specific menus. It also enables administrators and architects to develop complex automation tasks within the workflow designer using pre-defined tasks from a library. VMware vSphere ESXi and VMware vCenter Server VMware vSphere ESXi is a virtualization platform for building cloud infrastructures. The vSphere enables you to confidently run your business-critical applications to meet demanding service level agreements (SLAs) at the lowest Total Cost of Ownership (TCO). This Cisco Cloud Solution gives the consumer an operational insight into virtual environment for improved availability, performance, and capacity utilization. EMC VNX EMC VNX® is a powerful and trusted storage array platform. This provides the highest level of performance, availability, and intelligence in th Cisco cloud environment. This supports Fiber-Channel (FC), iSCSI, FCoE and NFS/CIFS protocols. The VNX storage systems offers a broad array of functionality many features and tools such as Fully Automated Storage Tiering for Virtual Pools™ (FAST VP), enabling multiple storage service levels. All of the array components are capable of continued operation in case of hardware failure. The RAID disk configuration on the array provides protection against data loss due to individual disk failures and the available hot spare drives can be dynamically allocated to replace a failing disk. Cloud Overview & Considerations The Cisco Cloud computing is a model to enable convenient and on-demand access to a shared pool of configurable computing resources. The expectation is to provide rapid provision and release with minimal effort or interaction. The Cisco cloud model promotes availability and consists of characteristics that are deemed to be essential and categorized along with service and deployment models. Overview In keeping with the National Institute of Standards and Technology (NIST) model depicted below, this solution with UCS Director will be shown to provide the capability to provision processing, storage, network and other fundamental computing resources where the consumer can deploy and run arbitrary software including operating systems and applications. The consumer does not manage or control the underlying cloud infrastructure but has control over operating systems, storage, deployed applications within allocated resources. Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 10 Cloud Overview & Considerations Figure 3 Cisco Cloud Solution with UCS Director Cloud Model With respect to the above NIST definition, this solution leverages the functionality of Cisco UCS Director for implementing an Infrastructure-as-a-service (IaaS) for a Private Cloud to be deployed with all essential characteristics detailed. Essential Characteristics Elasticity This feature explains the ability of the platform to support dynamic provisioning and decommissioning based on the needs of the consumers. It ties into ‘capacity-on-demand’ and faster time to market. Elasticity requires seamless integration between the UCSD orchestration piece and the underlying VSPEX integrated stack to take full advantage of compute, network and storage resource scalability options. Broad Network Access Given the borderless nature of our networks and the number of devices used for access, this requirement translates to support for non-traditional end-points such as tablets and cell phones in a secure manner. UCS Director supports secure technologies such as TrustSec and include security related devices such as the ASA and VSG firewalls. Mobile and tablet access is provided by Android based CloudGenie application which interfaces with UCSD. CloudGenie access is not within purview of this CVD at this time. Measured Services An IaaS platform consists of pooled resources serving multiple workloads and tenants. Given the services model followed, end-users are expected to pay only for resources used. End-users could belong to different departments within an enterprise or come from entirely different business entities. Whether Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 11 Cloud Overview & Considerations internal to a company or across multiple companies, the platform, due to the shared nature, needs to incorporate a means to measure resource utilization for the purpose of billing. UCSD has chargeback/showback capabilities based on cost models that can be set by the cloud administrator/provider. Data generated from chargeback can then be integrated with a payment gateway (First Data). Internal to UCSD, there are also complimentary budget mechanisms tied to individual groups for resource management. On-Demand self-service provisioning and automation The customer needs to be able provision and manage their environment on a shared platform with the least amount of intervention and delay from the provider. Providing for this functionality requires the establishment of a self-service portal with necessary privileges. The portal should provide a catalog of items available for consumption over which the customer has access. It should also include automated means of deploying instances to contribute to overall agility. UCSD provides self-service portal capability after setting up a set of policies and mapping entities (groups & users) to resources on the VSPEX platform. Orchestration of workflows consisting of available and customizable tasks is enabled through a graphical designer. Platform Modularity Above essential features at the orchestration layer need to be supported throughout the integrated stack for correct and consistent execution. The VSPEX platform, with Cisco UCS compute, Nexus 5548UP switches and EMC VNX 5400 storage array, have flexibility built in at every layer to allow for elasticity within the Point of Delivery (PoD). Compute can scale to 160 hosts/blades within a single UCS domain with storage on the VNX5400 scaling to 250 disks of varying capacity and performance. The architecture calls for common infrastructure components and services such as Active Directory, DNS, DHCP, vCenter, Nexus 1000v VSM and UCS Director to be hosted external to the IaaS PoD to provide a centralized and uniform management structure. This model also allows for the addition of more integrated PoD’s for growth, if necessary, while preserving the cloud capabilities of UCSD. Data protection services Data protection is available through snapshots at the hypervisor layer. When enabled, this will allow individual user VM’s be backed up as required. UCSD 5.0 also has support for EMC Recovery Point. However, this feature has not been tested at this time. Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 12 Cloud Overview & Considerations Figure 4 Add an User Account Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 13 Cloud Overview & Considerations Figure 5 Enter User Account Details Integration Points The current setup consists of several components and their respective native tools leading to a myriad of integration points as depicted in Figure 6. Cisco UCS Dirctor has tight integration at the infrastructure layer with all underlying components within the VSPEX stack – UCS Manager and Unisphere. The Nexus 1000v VSM communicates with both vCenter and UCSD for distributed virtual switch functionality. Cisco UCS Director also has integration into VMware vCenter and the bare-metal agent (for PXE) to implement this platform. External to this setup, UCS Director provides standard north-bound API’s for integration with third-party ITSM tools for event monitoring, trouble-ticketing and billing. Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 14 Solution Architecture and Design Figure 6 Myriad of Integration Points Solution Architecture and Design Architecture The architecture for the Cisco cloud solution shown below uses two sets of hardware resources, as shown below: 1. Common Infrastructure services on redundant and self-contained hardware. 2. VSPEX PoD for IaaS workloads under the Cisco UCS Director management The common infrastructure services include active directory, DNS, DHCP, vCenter, UCS Director and Nexus 1000v virtual supervisor module (VSM). These components are considered core infrastructure as they provide necessary data-center wide services where the IaaS PoD resides. Since these services are integral to the deployment of IaaS, there is a need to adhere to best-practices in their design and implementation. This includes such features as high-availability, appropriate RAID setup and performance and scalability considerations given they may have to extend their services to multiple PoD’s. One other consideration is to not introduce dependencies between management tools and the hosts/platforms they manage. For example, installing vCenter on ESX. At a customers’ site, depending on whether this is a new data center, there may not be a need to build this infrastructure piece. In our setup, given the limited scope of one VSPEX PoD, this environment consists of a pair of UCS C-220 servers with internal disks. VMware has been used to clone the VM’s to serve as backups The IaaS VSPEX PoD consists of Cisco UCS blade and rack servers. Fiber channel based LUN’s from the EMC VNX5400 were provisioned for SAN booting these servers after creating a separate storage pool. The FC connections go directly to the fabric-interconnects (6248) from the servers. Data is designed to reside on a separate NFS pool (NFS-DS1), also provisioned from the same EMC VNX 5400 through the data mover. This NFS space and the corresponding mount-point is visible to all hosts with Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 15 Solution Architecture and Design hypervisor based user access control. At the network layer, three VLAN’s have been created – VM-Data (50), storage (60) and vMotion(70). The Cisco UCSD appliance was setup as a single node with bare metal agent connected over a separate VLAN(150) from the PXE VLAN (50). A highly-available and scalable multi-node UCSD setup is available if there is a need to scale across multiple data-centers. Figure 7 Solution Architecture and Design Tenant design User groups and accounts for the IaaS platform are created and managed from Cisco UCS Director. For this exercise, two groups with two users in each group were created. The user groups were mapped to resources through the virtual data centers (vDC) construct to constitute a multi-tenant setup. Each tenant had an administrator user and another end-user role. Catalog items were created and shared by the cloud administrator (admin) through appropriate access to the self-service portal after setting up required policies. Each tenant group was assigned a budget and resources within the PoD were assigned approximate values for active and in-active instances. The understanding is that instances use compute, network and storage resources and as such, capture the overall requirements of the customer while also simplifying cost estimation from the providers’ perspective. In the event there is a need for more granular/accurate cost estimation, UCS Director has provisions for specifying compute, network and storage costs as well. Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 16 Solution Architecture and Design Figure 8 Enter Tenant Details - Part 1 Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 17 Solution Architecture and Design Figure 9 Tenant Details - Part 2 The Cisco UCS Director uses Role-Based-Access-Control (RBAC) in according accordance to resource privileges to for users. Many standard roles are pre-defined and there is the a flexibility to add new users with customized access levels. The group admin role has the privilege to create end-user’susers within the group. Thus, the cloud admin needs has permission to only create a group admin for each tenant. Cloud Management Environment Sizing Minimum System Requirements for a Single-Node Setup The minimum system requirements depend upon how many Virtual Machines you plan to manage. Note For optimal performance, reserve additional CPU and memory resources. We recommend that you reserve the following resources in addition to the minimum system requirements listed in the tables below: CPU resources of more than or equal to 3000MHz, and additional memory of more than or equal to 4GB. Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 18 Solution Architecture and Design Up to 2000 VMs If you plan to manage up to 2,000 VMs, the Cisco UCS Director environment must meet at least the minimum system requirements in shown in the following table. Table 2 Minimum System Requirements for up to 2,000 VM Element Minimum Support Requirement vCPU 4 Memory 8 GB Hard Disk 100 GB Up to 5000 VMs If you plan to manage no more than 5,000 VMs, the Cisco UCS Director environment must meet at least the minimum system requirements and recommended configurations in shown in the following tables. Table 3 Minimum System Requirements for up to 5000 VMs Element Minimum Support Requirement vCPU 4 Memory 8 GB Hard Disk 100 GB Table 4 Minimum Recommended Configuration Service Recommende d Configuration File Location Parameter Broker 256 MB /opt/infra/broker/run.sh -Xms -Xmx client 512 MB /opt/infra/client/run.sh -Xms -Xmx controller 256 MB /opt/infra/controller/run.sh -Xms -Xmx eventmgr 512 MB /opt/infra/eventmgr/run.sh -Xms -Xmx idaccessm 512 MB gr /opt/infra/idaccessmgr/run.sh -Xms -Xmx inframgr 8 GB /opt/infra/inframgr/run.sh -Xms -Xmx Tomcat 1 GB /opt/infra/web_cloudmgr/apache JAVA_OPTS="$JAVA_OPTS -tomcat /bin/catalina.sh -Xmsm -Xmxm Table 5 Minimum Database Requirement Element Minimum Support Requirement thread_cache_size 100 max_connections 1000 innodb_lock_wait_timeout 100 query_cache_size 128 MB Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 19 Deployment of Cloud Services Element Minimum Support Requirement innodb_buffer_pool_size 4096 MB max_connect_errors 10000 connect_timeout 20 innodb_read_io_threads 64 innodb_write_io_threads 64 Deployment of Cloud Services This deployment will consider a single VSPEX stack and its management with UCSD in a non-redundant fashion. This is because UCSD is not in the data path and one instance can support multiple integrated stacks. To ensure best-practices, the UCSD instance is installed external to the managed VSPEX PoD on common infrastructure components consisting of a pair of Cisco UCS C-220 rack servers. For deployments that require greater scale and/or connectivity across Data Centers, a highly redundant setup of UCSD is available (multi-node setup). For more information on redundant and scalable setup, see the Cisco UCSD Guide: http://www.cisco.com/c/en/us/td/docs/unified_computing/ucs/ucs-director/vsphere-install-guide/5-0/b_ Installing_UCSDirector_on_vSphere_5_0.pdf The private cloud platform could reside in premises or provider space (hosted). As such, this deployment will be an enterprise Private Cloud (ePC) with characteristics deemed essential in the model defined by the National Institute of Standards and Technology (NIST). Common areas of monitoring, managing and boarding specific to ePC will also be shown through the Cisco UCS Director. The Cisco UCS Director uses a policy based model for managing resources that are assigned. Policies are a set of rules for the framework and how resources can be provisioned and accounted. Fox example, the setting up of a self-service portal requires establishing compute, network, storage and system policies and an application about cost model to leverage chargeback for billing purposes. Setting up of required policies to provide necessary functionality for an IaaS platform is covered in the following sections. Base Platform This document assumes that you have followed the procedure detailed in the link below to build the base VSPEX platform: http://www.cisco.com/c/en/us/td/docs/unified_computing/ucs/UCS_CVDs/ucs_vspex_vmw_55.html# wp170829 Network Availability Design option followed in this CVD is the NFS-Variant architecture which uses Fiber Channel (FC) datastores for SAN booting of hosts and a common filesystem on one NFS datastore for data (NFS-DS1). There are three VLAN’s, one each for VM-Data (50), Storage (60) and vMotion (70). Other changes to the VSPEX infrastructure detailed above include use of VMware 5.5 in place of VMware 5.1 and UCS Director 5.0 for providing IaaS Cloud functionality. Following is the high-level architecture diagram for all devices in this solution. Common infrastructure management services and applications such as Active Directory, DNS, DHCP, SMTP, NTP and applications including VMware vCenter, Cisco UCS Director with the bare-metal agent, Cisco Prime Network Services Controller (PNSC) and Cisco Nexus 1000v Virtual Supervisor Module (VSM) are hosted external to the EMC VSPEX PoD as shown below. Common Infrastructure components need to be highly redundant to ensure Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 20 Deployment of Cloud Services uninterrupted service as the applications residing in this space are shared and critical to the operation of the entire Data Center which could include multiple such converged stacks. The focus is on using a validated converged infrastructure (VSPEX) to provide resources for the cloud with IaaS features with Cisco UCS Director. High-Level Architecture Figure 10 Cisco UCS Director Infrastructure Abstraction-Single Phase Management The following section outlines prerequisites to install and setup a working instance of Cisco UCS Director. The intent is to leverage the automation features of Cisco UCS Director for correct and consistent cloud deployment. One of the constructs that is applicable and included in the catalog is “Application Container Template”, which will allow for definition and grouping of a three-tier application with a set of web, application and database instances and a gateway/firewall (VSG) for access control. The section below will conclude with a procedure to create an application container template. The understanding is that the same approach can be followed to deploy customized application containers with other supported devices/functionality as required. Cisco UCS Director Installation and Configuration You can download VMware ovf’s for Cisco UCS Director 5.0, Cisco UCS Director Baremetal Agent 5.0 and Cisco UCS Director 5.0 Workflows zip file from the following link: http://software.cisco.com/download/release.html?mdfid=286280023&flowid=71143&softwareid=2850 18084&release=5&relind=AVAILABLE&rellifecycle=&reltype=latest You will also need the bare-metal agent software installed for building bare-metal instances: Note 1. Log in to vSphere client, connect to vCenter 5.5 that is installed external to the VSPEX PoD on a common infrastructure. 2. Choose File, Deploy OVF Template, and choose Browse to downloaded the OVF files. First install the Cisco UCS Director OVF and then the bare-metal agent (BMA). Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 21 Deployment of Cloud Services Figure 11 Downloading OVF files 3. Choose the OVF file and click Open, then click Next. 4. Click Next on the OVFTemplateDetails page. 5. Read the terms End User License Agreement and Conditions, and click Accept, then click Next. 6. Provide an appropriate VM Name (say iaas-UCSD50) and choose the Infra.IAAS infrastructure cluster under IAAS_DC. Datacenter as the Inventory Location, and click Next. 7. Choose Infra.IAAS infrastructurecluster and click Next. 8. Choose the datastore location and click Next. 9. Click Next for the Disk Format page. 10. Choose Lab Network for destination network and click Next. Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 22 Deployment of Cloud Services Figure 12 Choosing Lab Network for Destination Network 11. Leave DHCP selected for now on the IP allocation page and click Next. 12. Click Finish. The import will begin and the progress of the import will be displayed on Next the screen. 13. Click Close. Initial UCS Director Setup Follow the procedure below to configure the Cisco UCS Director Virtual Machine on VMware: Note Upgrade the reserved resources for the newly created VM 1. Right-click on the UCSD VM icon and click Edit Settings. Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 23 Deployment of Cloud Services Figure 13 Setting up UCS Director 2. Choose the Resources tab. 3. Choose CPU, and change the Reservation to about 4000 MHz, then choose Memory, and change Reservation to over 4000MB. Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 24 Deployment of Cloud Services Figure 14 4. Resources Tab Click on Options, and choose VMware Tools, then click Synchronize guest time with host, then click OK to save the changes. Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 25 Deployment of Cloud Services Figure 15 Note Options Tab 5. Right-click the UCSD VM, and choose Power, Power on 6. Right-click the UCSD VM, choose Open Console to configure the Cisco UCS Director settings. First wait for the boot script to run and later you can configure a static IP. Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 26 Deployment of Cloud Services Figure 16 Configuring UCS Director Settings 7. For "do you want to configure static ip?" Enter “n” for no as we will configure the IP after boot. The booting process will take a few minutes. 8. After boot completes, the system has booted, you will see a login screen. (A DHCP assigned address might exist if you have DHCP running on this subnet.) Press Enter to select login. 9. Enter the login ID as "shell admin" and password as "change me" Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 27 Deployment of Cloud Services Figure 17 Login Page 10. At the Select prompt, enter 14 to Configure Network Interface. Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 28 Deployment of Cloud Services Figure 18 Configuring DHCP/ Static IP Address 11. For "Do you want to configure DHCP/Static IP?" Enter “S” for static. 12. Enter Eth0 for the interface you will configure. 13. Enter “y” for question if you want to configure Static IP for eth0. 14. Enter the IP, Netmask, Gateway and DNS server. Note The Cisco UCS Director server has dual interfaces with one leg (eth1) on a routable network (vlan 150) and another (eth0) on a private production vlan (50). Following is an output of option 15 to display network details: Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 29 Deployment of Cloud Services Figure 19 Summary Tab 15. Review the information and type “y” to continue 16. Press Enter to return to menu and input 1 to change the password 17. Press Enter to return to main menu. 18. Enter 28 to quit.Notice that the web URL is connect to https://<assigned IP>:443 19. For some browsers you may need to add the web URL to trusted sites to display correctly. Open the browser, and input the URL to UCSD. For I.E, click Tools, Internet Options, Security tab, Trusted Sites, Sites, and the address for your UCSD system. 20. Click Add and then click Close. 21. Press F5 to refresh the browser Configuring the Admin Account 1. Connect to the URL for your UCSD system using the IP address. 2. Log in as ”admin” with the password of "admin" and choose Login. Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 30 Deployment of Cloud Services Figure 20 Logging into Cisco UCS Director 3. Click OK to temporarily ignore the popup information message for login profile. 4. Click Administration on the menu bar and choose Users and Groups. 5. Choose Login Users tab, highlight admin, and choose Change Password to type a new password. 6. Click Save, and then OK. Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 31 Deployment of Cloud Services Figure 21 Change Password Message Box 7. After selecting the loging user as 'admin', click 'edit' and type user your email address. 8. Click Save, and then OK. 9. Goto Administration and choose, System and select Mail Setup tab. Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 32 Deployment of Cloud Services Figure 22 Mail Setup Tab 10. Enter the SMTP server ip address or hostname if you have working DNS 11. Enter the correct SMTP port (25 is default). 12. Enter the Outgoing Email sender address. 13. Enter the Server IP Address of the UCSD server 14. Click the Send Test Email box 15. Enter the Test Email Address 16. Click Save to validate that you get a “Successfully update mail the settings. 17. The system displays the “Test email Succeeded” message confirmation message. 18. Click OK Installing Licenses 1. Install the license by choosing Administration, License, then choose the License Keys tab to update license. Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 33 Deployment of Cloud Services Figure 23 Uploading License File 2. Click Browse and choose the license file that you received from Cisco and click Open to upload the file. 3. After uploading the file, click OK, and then Submit. 4. Choose the License Keys tab, click Refresh and validate you have a minimum of the base license. Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 34 Deployment of Cloud Services Figure 24 License Keys Tab Point of Delivery (PoD) A PoD is a collection of physical and virtual resources that can be managed together. We will create a site and a pod that will contain our VSPEX resources. 1. Add a site name by choosing Administration, Physical Accounts, then the Site Management tab. 2. Click Add to enter site name and contact name then click Submit. 3. Click OK to successfully add the message. 4. Choose Converged from the main menu and click Add. Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 35 Deployment of Cloud Services Figure 25 5. Converged Tab Enter the Pod Name, Site and choose VSPEX. Click Add. Adding EMC VNX Storage 1. Choose Administration and then Physical Accounts. 2. Click Add and pick the correct data center, category (storage) and type (VNX Unified). Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 36 Deployment of Cloud Services Figure 26 3. Creating EMC VNX Storage Account Click Submit to open a popup a screen with fields to be added. Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 37 Deployment of Cloud Services Figure 27 4. Editing an Account Enter the PoD name, and description for File (above) and Block accounts (below): Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 38 Deployment of Cloud Services Figure 28 Entering Account Details 5. Enter IP, login and password for VNX File control station and Block processors. 6. Choose HTTPS for the Transport Type. 7. Optionally add a description, Contact Email, Location and Service Provider. 8. Click Add, then OK. 9. Once the account has been added, select the newly added account from the list and choose Test Connection 10. Click Close. Adding Cisco Nexus Switches Follow the procedure below to add Cisco switches to the Pod. Repeat the steps for both type of Cisco Nexus 5000 switches. 1. Choose Administration and then Physical Accounts. Click the Manage Network Elements tab. Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 39 Deployment of Cloud Services Figure 29 Adding Network Element 2. Click Add Network Element. 3. Choose the pPod that you have created. 4. Choose Device Category as Cisco Nexus OS. 5. Enter the switch management IP address for the Device IP. 6. Choose SSH for the Protocol. 7. Enter 22 for the Port. 8. Enter administrator login ID. 9. Enter the switch admin password 10. Enter password again for Enable Password field. 11. Click Submit. Note It can take a few minutes to complete this operation ) 12. When the account has been added, repeat for the other switch. 13. Select the newly added switches and choose Test Connection and then click Close. Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 40 Deployment of Cloud Services Adding C-Series Rack Servers UCSM managed rack servers are discovered and imported when the UCSM account is added. Un-managed rack servers not connected to the fabric interconnects can also be added, monitored and managed by UCSD as follows: 1. Choose Administration and then Physical Accounts tab. 2. Choose the PoD, category and account type of Rack Server and click Submit: Figure 30 3. Adding Physical Account Enter required data for adding rack servers to be managed by Cisco UCS Director. Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 41 Deployment of Cloud Services Figure 31 Adding Rack Servers Add VMware Virtual Account The VCenter server needs to be added to our converged Pod in order for Cisco UCS Director to manage our VMware infrastructure. 1. Click Administration, Virtual Accounts and then Virtual Accounts tab. 2. Choose Add. Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 42 Deployment of Cloud Services Figure 32 Adding a Virtual Account 3. On the Add Cloud popup, choose the cloud type as VMware. 4. A second add screen will appear with VMware selected for cloud type. Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 43 Deployment of Cloud Services Figure 33 Adding VMware as a Cloud Type 5. Enter the cloud name. (Example: IAAS-Cloud) 6. Enter the vCenter server IP address. 7. Enter vcenter login and password for connectivity. 8. Leave the server access URL set to /sdk. 9. For Pod, input the pod you created (Example VSPEX-IAAS). 10. Click Add. 11. It can take a few minutes for Cisco UCS Director to complete the query of the Vcenter objects and the connection status to change to success. Highlight the account, and click Test Connectivity. 12. click Close. Create Local Users and Groups With Cisco UCS Director, you can use local accounts and/or Windows Active directory accounts. Here we will go through steps necessary to create a group and users within the group. You can use these for production or test purposes prior to roll out. 1. Choose Administration, Users and Groups and Authentication Preferences tab. Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 44 Deployment of Cloud Services Figure 34 Authentication Preferences Tab 2. Choose the authentication preference as LDAP First, failback to Local. 3. Choose the LDAP integration tab: Figure 35 LDAP Integration Tab 4. To get basic info, force a sync by choosing Request Manual LDAP Sync and click Submit, then OK 5. Click Search BaseDN and make selections on the popup to get a list to use for the correct BaseDN. Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 45 Deployment of Cloud Services Figure 36 Searching for Base DN 6. Click Select, Submit, then OK 7. To update records again, Click Request LDAP Sync, click Submit, then Ok 8. Click Refresh and this time it should be a success. Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 46 Deployment of Cloud Services Figure 37 Note Updated List of Records After System Refresh Local groups and users can also be added and managed. With the Cisco UCSD you can use either local accounts or link to LDAP/Windows Active directory accounts. Here we will go through the process of creating local groups and users. This process is optional depending on customer needs. If you already have a setup of Active Directory integration and you don’t need local accounts, you can skip this section. 1. Create a local group by choosing Administration, Users and Groups, and then Customer Organizations tab. 2. Click Add. Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 47 Deployment of Cloud Services Figure 38 3. Note Adding a Local Group Input the local group name, email address and click Add The User Role determines whether an account is specific to a group or not. Therefore, only accounts with privileges that can be limited to the group will be presented with the “User Group” field and a drop-down for it. 4. Click the Login Users tab and click Add to add a new user. Leave user type to default, and select local group created in previous step. Input login name, password, user email address and click Add then OK for adding a new user: Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 48 Deployment of Cloud Services Figure 39 Adding a New User Create Compute Policy 1. Click Polices, Virtual/hypervisor policies, Computing. Select the VMware Computing Policy tab and the default policy for editing: Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 49 Deployment of Cloud Services Figure 40 Choosing Compute Policy Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 50 Deployment of Cloud Services Figure 41 2. Editing Computing Policy Details Change the policy description and leave other values as defaults. Choose Resource Pool to point to the IaaS PoD with ESX type and version as required. Then click Submit and Save. Create Network Policy The Network Policy allows us to add a specific nic to our VM. We will customize the nic to use a certain VLAN ID 1. Choose Policies, Virtual/Hypervisor Policies, Network and then click VMware Network Policy tab: Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 51 Deployment of Cloud Services Figure 42 2. Creating New Policies Either use the existing default policy or create a new one as follows. Set the description field as VSPEX-IAAS-Cloud and click on the “+” sign next to VM Networks to add a network interface. Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 52 Deployment of Cloud Services Figure 43 3. VMware Network Policy Details Enter the NIC alias name(nic1) and Adapter Type of VMXNET3. In the Port Groups section, click the “+” to add a port-group and click on the Port Group Name. Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 53 Deployment of Cloud Services Figure 44 Adding a Port Group Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 54 Deployment of Cloud Services Figure 45 4. Selecting the Created Port Groups After making the selection, click on Select, Submit and OK. Create Storage Policy The storage policy allows us to select storage based on attributes as well as customize what actions the end user can perform for the VM creation. In this operation we will restrict the policy to use only the NFS datastore. 1. Click Policies, Virtual/hypervisor policies, and Storage. Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 55 Deployment of Cloud Services Figure 46 2. Choosing Storage Policy Choose the VMware Storage Policy tab and select the default policy for editing. Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 56 Deployment of Cloud Services Figure 47 3. Note VMware Storage Policy Tab Change the policy description to reflect our PoD and pick the NFS datastore provisioned for data (NFS-DS1). Click Next and then Submit and OK. There is a provision to choose other datastore as required. In this case, we will be using one NFS space for all data with individual SAN boot over Fiber Channel (FC) shown as datastore1 (x) below. Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 57 Deployment of Cloud Services Figure 48 Choosing Datastore Create System Policy The system policy provides for change to the VM when created via the portal on such variables as name generated, the template used, DNS setting and other system wide rules. 1. On the Policies menu, choose Virtual/Hypervisor Policies and then choose Service Delivery. Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 58 Deployment of Cloud Services Figure 49 2. Choosing Service Delivery Choose VMware System Policy tab and Add “Policy Name” and “Description” (optional). Change the “VM Name Template” to vspex-SR${SR_ID} without trailing spaces to avoid exceeding the 15 character Windows name limitation. Include “DNS domain” and IP of server followed by “VM Image Type” of Windows and Linux to allow for flexibility. Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 59 Deployment of Cloud Services Figure 50 3. Storage Policy - System Disk Policy Page - Part1 The next screenshot shows the rest of the “System Policy Information” page. Click Submit after entering details. Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 60 Deployment of Cloud Services Figure 51 Storage Policy - System Disk Policy Page - Part2 Create Virtual Data Center (vDC) A virtual data center (vDC) provides a construct that allows for the grouping of resources in an integrated stack and maps users and policies to allocated resources to accommodate tenant requirements. While an organization/department can manage multiple vDC’s, each vDC is a separate logical entity with specified approvers and quotas (if any). Creating Virtual Data Center (VDC) requires the following pre-requisite: Create End-User Policy -The End User Policy allows for setting of access to particular user operations within the vDC. There is the flexibility to set user access to be as restrictive or open as needed. User management and access includes such areas as VM Power Management, Resizing, Snapshots, VM deletes, Disk management, Network and Console Management. 1. Choose Policies, Virtual Hypervisor Policies and then Service Delivery. Click End-User Self-Service Policy tab after scrolling to the right to locate it. Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 61 Deployment of Cloud Services Figure 52 2. Choosing Service Delivery Tab Choose Add. In the Add End-User Policy pop-up, pick the correct account type (VMware) and click on Submit: Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 62 Deployment of Cloud Services Figure 53 3. Choosing Account Type for End User Policy In the End User Policy dialog box, provide the Policy Name (IAAS-POD VDC), Description (optional) and select required options as below: Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 63 Deployment of Cloud Services Figure 54 End User Policy Page - Part1 Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 64 Deployment of Cloud Services Figure 55 End User Policy Page - Part2 Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 65 Deployment of Cloud Services Figure 56 End User Policy Page - Part3 4. Click Submit. 1. Choose Policies, Virtual/Hypervisor Policies and then Virtual Data Centers. Click on vDC tab Creating VDC Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 66 Deployment of Cloud Services Figure 57 2. Choosing vDC Tab Select Add and pick VMware as the Account Type from the drop-down and Submit. Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 67 Deployment of Cloud Services Figure 58 3. Choosing VDC Account Type In the Add VDC pop-up, enter vDC_Grp-1 as the vDC name. Provide access to resources in this vDC to previously created group (group_1) by picking group_1 in the drop-down for the group field. Select the correct cloud name as well. Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 68 Deployment of Cloud Services Figure 59 4. Add vDC Page Choose the created System, Compute, Network and Storage Policies from respective drop-downs. Cost Model may be ignored, as default for now. Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 69 Deployment of Cloud Services Figure 60 5. Entering Information on Add vDC Page Choose the previously created End User Self-Service Policy (IAAS-POD VDC) and click on Add, then OK. Catalog Publishing You can see a catalog for self-provisioning VMs. The system administrator creates a catalog item, and defines its parameters such as cloud name, and group name to which the VM is bound. 1. Choose Policies > Catalogs > Add Pick Standard Catalog Type from the drop-down and click Submit. Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 70 Deployment of Cloud Services Figure 61 2. Note Creating Self Provisioning Catalog Following screenshot shows the information to be provided for a Redhat catalog item with access groups for selected groups. After entering the information, click Next and then Submit to add the item to the catalog. The new VM may be provisioned either with an ISO image as shown or by uploading a previously created image (FC_Gateway) as shown in the second screen below. Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 71 Deployment of Cloud Services Figure 62 Information for Redhat Catalog Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 72 Deployment of Cloud Services Figure 63 3. Information for Standard Catalog Items Confirm these catalog items are deployable on all hosts within the PoD by selecting the catalog item and then clicking on Deployability Assessment tab. Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 73 Deployment of Cloud Services Figure 64 4. Deployability Assessment Tab The output should show a status of “Yes” for Deployable column for the Host Node’s of interest: Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 74 Deployment of Cloud Services Figure 65 5. Deployment Assessment Section Following is a view of a self-service portal for ‘grp1-user1’ with a set of ‘standard’ and ‘advanced’ catalog items created using the above process. Opening the standard catalog folder presents the services available for the particular user. Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 75 Deployment of Cloud Services Figure 66 Opening a Standard Catalog in Cisco UCS Director Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 76 Deployment of Cloud Services Figure 67 6. Contents of Standard Catalog in Cisco UCS Director The “deployability assessment” option is shown when one of the catalog items is selected and the following screen provides the details on the ESXi hosts the selected catalog item may be deployed on for the user. Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 77 Deployment of Cloud Services Figure 68 Select the IAAS- RHEL Catalog Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 78 Deployment of Cloud Services Figure 69 Deployment Assessment of IAAS-Cloud Setting Quotas Resource limits at the group level and in units pertaining to either physical or virtual instances can be set as shown below. 1. Choose Administration > Users and Groups > Customer Organization tab. 2. Choose the group of interest and click on Edit Resource Limits. Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 79 Deployment of Cloud Services Figure 70 Customer Organizations Tab Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 80 Deployment of Cloud Services Figure 71 Customer Organizations Tab Adding PNSC Cisco Prime Network Services Controller (PNSC) is a virtual appliance used to manage instances of Virtual Secure Gateway (VSG) from a central location. VSG firewall appliances can be used by the application container construct within UCSD, thus providing east-west access control at the application level within a tenant on a multi-tenant platform. This is an optional step depending on the need for above functionality. 1. Choose Administration, Multi-Domain Managers and then click Add. Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 81 IaaS Features Figure 72 Adding Multi-Domain Manager Account 2. After providing an account name, choose a PNSC type from the drop-down. 3. Provide login and password for the PNSC management service. 4. Protocol of https with default port of 443 and click Submit. IaaS Features Elasticity Provisioning a stateless Cisco UCS blade server through workflow orchestration sets the stage for additional resources including compute, network and storage, required for true elasticity. This assumes hardware capacity is available for necessary expansion. Statelessness alludes to the capability to separate identity from the underlying hardware. Cisco UCS servers provide for this functionality where server identifiers such as the MAC, UUID, WWN, firmware and BIOS versions are stored as pools in UCS Manager, which is external to the server. Policies, which are rules that map resources (servers) to aforementioned pools, are then used to create a server with identity. This unique capability of Cisco UCS servers to be programmed with an identity allows for agile provisioning at the host level. However, persistence of statelessness is contingent upon booting the server from SAN. Hosts integrated into the Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 82 IaaS Features IaaS PoD through this stateless provisioning method can later be consumed using method documented in this CVD. This essential step precedes actual consumption and the user is referred to UCS Director VSPEX Management Guide for this purpose. A high-level list of tasks constituting the workflow to integrate a stateless server on VSPEX is as follows 1. Modify workflow priority (95) 2. Create UCS service profile (114) 3. Select UCS server (96) 4. Associate UCS service profile (97) 5. Power off UCS server (108) 6. Setup PXE boot (98) 7. Create VNX LUN (115) 8. Create VNX storage group (101) 9. Add VNX Host Initiator Entry (102) 10. Add VNX Host Initiator Entry (103) 11. Generic Configure SAN zoning (104) 12. Add hosts to VNX Storage Group (105) 13. Add VNX LUN to storage Group (106) 14. Modify UCS service profile boot policy (116) 15. Modify UCS Boot Policy LUN ID (107) 16. Reset UCS Server (109) 17. Monitor PXE Boot (99) 18. Modify UCS Service Profile Boot Policy (100) 19. Add VLAN to Service Profile (117) 20. Disassociate UCS Service Profile (110) 21. Wait for Specified Duration (111) 22. Associate UCS Service Profile (112) 23. Wait for specified duration (113) 24. Reset UCS Server (118) Security and Multitenancy In Cisco UCS Director, users get access privileges based on their roles (RBAC). The cloud administrator sets privileges based on available role templates and has the flexibility to create new roles or modify existing ones to suit the need. There is separation between users within the group and across groups as well. Preservation of user-space confidentiality through encryption and other means at multiple levels through use of access controls, virtual storage controllers, VLAN segmentation, firewall rules, and intrusion protection should be employed where possible. Data protection through continuous encryption of data in flight and at rest is essential for integrity. Cisco TrustSec SGT support by UCS Director and on most Cisco devices makes it easy to enable proper access control in a distributed manner for a scalable and secure platform. Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 83 IaaS Features Storage In this deployment, the need is for flexibility in resourcing the tenant at the virtual level while preventing unauthorized data access. To this end, boot LUNs are grouped in a separate Fiber Channel (FC) storage pool and shared by all hosts within the PoD. Data, also on the SAN through Network File System (NFS), is mapped from a common share. To make sure there is secure separation, user access controls at the hypervisor level (VMware) ensures users will not have unauthorized access to NFS space. Further access controls may be exercised through TrustSec (SGT) and VMware vShield if desired. System access controls at the time of creating NFS exports on VNX via Unisphere should list IPs of all target hosts for the “Root Hosts” and “Access Hosts” fields to allow complete access. NFS Security Settings Although generally regarded as a vulnerable file-sharing protocol, you can make NFS more secure by using the following configuration settings: 1. Defining read-only access for some (or all) hosts 2. Limiting root access to specific systems or subnets 3. Hiding export and mount information if a client does not have mount permissions for the file system corresponding to that entry In addition, if strong authentication is required, Secure NFS using Kerberos can be implemented.. All NFS exports are displayed by default. To hide NFS exports, you must change the value of the forceFullShowmount for mount facility parameter. Security on the VNX The EMC VNX 5400 storage array provides several layers of security including at the user access and logging and auditing levels. A Virtual Data Mover (VDM), which is a logical network abstraction on top of physical Data Movers, provides for additional network end-points to facilitate IP based separation for NFS mounts. Encryption The storage management server provides 256-bit (128-bit is also supported) symmetric encryption of all data passed between it and the client components that communicate with it, as listed in Ports used by Unisphere components on VNX for block on page 38 (Web browser, Secure CLI), as well as all data passed between storage management servers. The encryption is provided using SSL/TLS and uses the RSA encryption algorithm, which provides the same level of cryptographic strength as is employed in e-commerce. Encryption protects the transferred data from prying eyes-whether on the local LANs behind the corporate firewalls, or if the storage systems are being remotely managed over the Internet. Communication Security VLANs are logical networks that function independently of the physical network configuration. For example, VLANs enable you to put all of a department's computers on the same logical subnet, which can increase security and reduce network broadcast traffic.Configuring and Managing Networking on VNX provides additional information about Packet Reflect and VLANs as well as how to configure these features. For more information on security features of VNX Series of arrays, see Security Configuration Guide for VNX P/N 300-015-128 Rev 01 and P/N 300-013-510 Rev 03. Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 84 IaaS Features Figure 73 Cisco UCS Director Security Integration Chargeback Model and Metering The Chargeback module in UCSD gathers metering information at frequent intervals. This data can then be juxtaposed with cost-models to arrive at tenant costs and for reporting as well. Dashboard reports are also an offshoot of this module. The first step is to configure a budget policy to individual organizations. Configure Budget Policy 1. Select Administration > User and Groups > Customer Organizations. Then, select the group created (groups_1) and click Budget Policy. Enabling Budget Watch is required for monitoring resource usage for this group. The other two options allow for exceeding allocated budget and setting a budget. Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 85 IaaS Features Figure 74 Configuring Budget Policy Cost Model Within UCS Director, cost models can be created for each tenant. Costs for resources used in a vDC may be computed by the hour, month or year. Each tenant is typically created in a separate vDC to facilitate easy separation for billing purposes. • Standard cost model: This is a basic and linear cost model based on resource consumption over the allotted period. CPU, Memory and Disk Resources used and idle over the period and their respective cost structure is used to estimate cost. • Advanced cost model: This model is more customized and allows for greater granularity in choices and billing through the use of scripts. Such scripts that are tailored to customer needs have to be generated as they are not packaged with the system. The setup below considers a straight-line Standard cost model to illustrate functionality and setup. 1. Choose Policies > Virtual/Hypervisor Policies > Service Delivery and edit the default cost model. Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 86 IaaS Features Figure 75 2. Choosing Service Delivery We will select a standard cost model Type to illustrate chargeback with an initial setup cost of $50.00 (say). Here, the initial setup cost is assumed to include only costs pertaining to setting up the account. The VM cost needs to contain amortized fixed (CapEx) and variable costs (OpEx) for all under-lying system components that constitute a virtual instance – compute, network and storage. The capital expense component will be due to infrastructure – facilities and host platform. The variable operational expense portion could include such components as power and cooling, management and support costs. The approximate baseline used here to estimate chargeback is a unit active VM cost of $1.0 per hour and inactive VM cost of $0.10 per hour. The figures chosen are approximate and only used to illustrate method used and functionality on UCSD. The reader is referred to external whitepapers if there is a need for more accurate chargeback figures. The assumption is that the VM contains compute, network and storage. It is also possible to define units and costs for individual components for greater accuracy as shown in the second screen below: Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 87 IaaS Features Figure 76 Editing Cost Model Details Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 88 IaaS Features Figure 77 3. Editing Cost Model Integration with a payment gateway such as First Data is available for third-party billing. Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 89 IaaS Features Figure 78 4. Payment Gateway Page You need to setup a merchant account with First Data which provides the necessary secure certificate and password for authorizing payments through their gateway. The provided First Data certificate and password needs to be input in above form to setup payments to the provider for IaaS resources used. Policies and cost model presented above, along with quota’s set for tenants, come together while designing a self-service portal defined below. Dashboard The Dashboard provides a snapshot and trend of relevant data in easy to read graphs. It forms the basis of monitoring and provides a summary of the state of the entire enterprise on a single-pane. This functionality needs to be first enabled as follows: 1. Choose admin account on top right corner of the login screen and click Dashboard. Then, choose Enable Dashboard and Apply: Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 90 IaaS Features Figure 79 2. Accessing Dashboard Tab Choosing Physical > Compute and then highlighting the VSPEX-IAAS PoD and the Summary tab presents the following. A display of the list of available metrics is shown above the graphs when the arrow next to the wheel to the right of the screen (below CloudSense tab) is selected. Here we have a summary of compute related metrics. Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 91 IaaS Features Figure 80 3. Summary of Compute Related Metrics A snapshot of VM related metrics by selecting Virtual > Compute and then the PoD (IAAS-Cloud). If any of these metrics/graphs need to be on the main dashboard, it is just a matter of clicking on the down arrow to the right of each graph or summary and selecting Add to Dashboard. Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 92 IaaS Features Figure 81 4. Summary of VM Related Metrics Selecting Virtual > Storage and then IAAS-Cloud and then Summary tab. Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 93 IaaS Features Figure 82 5. Summary of Storage Capacity Virtual Network Metric snapshot. Choose Virtual > Network and then IAAS-Cloud and Summary tab. Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 94 Resource Monitoring Figure 83 IAAS-Cloud Summary Resource Monitoring The admin user has necessary privileges to monitor the entire Cloud or converged stack for a global view. Selecting each of the components (VMware, Compute, Network or Storage) below brings up comprehensive sets of metrics in tabbed displays for the component. Following is a sampling of metrics and views offered. 1. Choose Converged and then the site Cisco-IAAS for individual components and their status. Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 95 Resource Monitoring Figure 84 2. Monitoring Cloud Converged Stack Select VMware then click Topology tab, select Hostnode-VM Topology and choose the View Connectivity option. Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 96 Resource Monitoring Figure 85 Viewing VMware Topology Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 97 Resource Monitoring Figure 86 3. Viewing Host VM Mapping Further selecting admin in the Compute category brings up the following set of tabs with polled information for each compute component and other relevant data. Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 98 Resource Monitoring Figure 87 4. Summary of Compute Component Similar operation (selecting IAAS-VNX from Storage section) results in the following screen with tabs that present comprehensive data on the storage array. Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 99 Resource Monitoring Figure 88 Summary of Storage Array Data Self-Service Portal The Cisco UCS Director Self-Service Portal (Cloud Portal) for end-user provisioning, monitoring, and management is available out-of-the-box upon publishing a catalog, as described previously. A high-level view of dependencies and required setup is shown in the diagram below titled “Tenant Catalog for Self-Service Portal”. Through the Cloud Portal, one can perform permitted tasks on a pool of accessible resources using predefined policies. Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 100 Resource Monitoring Catalog Design and Implementation The Cloud Administrator creates a tenant group and adds users within the group as a prerequisite. Following this step, the tenant group is associated with cloud resources (vDC) and privileges assigned to users. Catalog items for self-service portal are then created and associated with tenant users. These steps are required prior to tenant user provisioning activities on the VSPEX Cloud platform with UCS Director. Tenant users generate a service request when one of the catalog items are selected for deployment with optional approvals prior to execution. Tenant Administrators and Operations personnel will then consume/release cloud resources as needed with chargeback tied to resource utilization. Figure 89 Tenant Catalog for Self-Service Portal Create a Service Request for a Catalog Entry 1. Click Policies > Catalogs and then click Add. Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 101 Resource Monitoring Figure 90 2. Note Adding a Catalog Provide a name and other input as shown. The catalog item is being associated with users in group_1. Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 102 Resource Monitoring Figure 91 3. Adding Catalogs Click Next and provide inputs for ‘Select’ as follows and click Next and Submit: Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 103 Resource Monitoring Figure 92 List of Catalog Entries This will create a service request in the catalog for an item of the type Advanced since we have chosen a prebuilt application container item for our catalog. A type of standard pertains to VMs. Note Typically, you have the chance to check “deployability” after you highlight the recently created catalog item by selecting the button called “deployability assessment”. This is not the case for advanced catalog items such as application containers. The service request process produces a provisioning workflow for VM creation that includes the following actions: • Budget validation (optional) • Dynamic resource allocation • Approvals (optional) • Provisioning • Life cycles setup and notification Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 104 Resource Monitoring Post provisioning, VM life cycle management actions are determined through RBAC set by the administrator. You can also view the entire list of virtual machines (VMs) provisioned using service requests under their group. All VMs that belong to a particular group are displayed. The available life cycle management actions are as follows: • VM power cycle management • Resizing a VM • Creating a VM disk • Deleting a VM disk • Adding a vNIC • Deleting a vNIC To view VM actions allowed for the user by the administrator on a VM, follow these steps: 1. Select Virtual Resources from the portal and then click on the VM tab. Select any VM provisioned in the Cloud and right-click to view a list of permitted operations. Figure 93 Select the VM Provisioned in the Cloud Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 105 Resource Monitoring 2. The operations listed above is a function of tasks permitted by the administrator at the time of creating the vDC and defined within the corresponding End-User Policy as shown before. Following is a view of service requests deployed on the particular vDC (vDC-Grp_1) by users in group-1. Selecting a request provides a view of the available options such as a create, cancel or rollback. Figure 94 3. Service Requests for the Created VMs The “Upload Files” tab provides details on files uploaded to common datastore. Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 106 Resource Monitoring Figure 95 Details of the Files in the Common Datastore Application Container Template An Application container is an abstraction that facilitates grouping of various components of a multi-tiered application into a single entity within a protected network. It consists of several instances, each performing a particular function (say) web, application or database in a three tiered application. The container can also include gateway, firewall and load-balancer appliances for additional functionality such as security, redundancy and bandwidth sharing. Once created, such containers can be assigned to users who can then deploy the same in a very quick and consistent manner for an application ready infrastructure. Steps documented in this section present a method with a simple Linux gateway (Fenced Virtual) with provisions to set access controls through IP tables. Additionally, greater security may be provided with the inclusion of a Virtual Security Gateway (VSG) firewall appliance as well. The VSG firewall resides within the container and provides east-west traffic access control with centralized management by Cisco Prime Network Services Controller (PNSC) which is installed on common infrastructure. All Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 107 Resource Monitoring components along with the technologies they use such as TrustSec are supported by UCS Director for a secure multi-tenant environment. It is beyond the scope of this CVD to go deeper into individual security related topics pertaining to the solution. Prerequisite The prerequisite for adding a policy is given as follows: 1. Choose Policies > Application Containers > Virtual Infrastructure Policies (tab), then click Add Policy. Figure 96 2. Adding Policy In the policy specification dialog, provide input as below. We select “Fenced Virtual” and later pick a basic Linux based gateway for the container. Alternatively, we can go without a gateway. Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 108 Resource Monitoring Figure 97 Adding Virtual Infrastructure Policy Figure 98 Adding Virtual Infrastructure Policy - Fencing Gateway 3. Select standard Linux gateway without load-balancer for a basic container without bandwidth sharing features. Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 109 Resource Monitoring Figure 99 Fencing Gateway - Selecting Gateway Policy Figure 100 Adding Virtual Infrastructure Policy - F5 Load Balancer Information Figure 101 Adding Virtual Infrastructure Policy - Summary 4. Click Submit to accept this virtual infrastructure policy to create an application container. Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 110 Resource Monitoring Application Container Template These steps describe creation of a fenced virtual application container template. 1. Select Policies > Application Containers and then click on Application Container Templates and Add Templates. Figure 102 Application Container Templates Page Figure 103 Providing Template Description 2. In the Next screen, select the previously created Virtual Infrastructure Policy. Figure 104 3. Choosing Virtual Infrastructure Policy In the Next screen, there is a choice to create multiple networks for the container template. After providing network details as below, click Submit: Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 111 Resource Monitoring Figure 105 4. Creating Networks Selecting OK and then clicking Add Entry provides access to add a VM with the required attributes to the container policy: Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 112 Resource Monitoring Figure 106 Add Entry to Virtual Machine Figure 107 Adding Application Container Template 5. Selecting Next takes you to the Application Container Template screen. Here, previously defined compute, network, storage and system policies along with the Cost Model are selected: Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 113 Resource Monitoring Figure 108 6. This next step provides options for privileges accorded to users for self-service provisioning: Figure 109 7. Do this only if you are including a VSG for East-West access control through a firewall appliance. Else, go to step 8. Click Next, if not including VSG firewall functionality. Figure 110 9. Choosing Option for Application Container Template This step of selecting container workflow is required only when VSG is included Note 8. Deployment Policies for Application Container Template Setting Up Workflows The final step displays the summary of all selections prior to confirmation. Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 114 Resource Monitoring Figure 111 Summary of Application Container Template Details Figure 112 Deployment Policies Summary Workflow Orchestration The Cisco UCS Director Orchestrator allows for automation of out-of-the-box tasks arranged as workflows using an intuitive graphical interface called the workflow designer. Both virtual and physical tasks can be included to design custom workflows. Triggers help initiate actions inside a workflow and the workflow itself may be executed by hand or through a trigger to kick the process off. A typical workflow consists of the following elements: • Workflow Designer (GUI interface) • Predefined Tasks for the supported component The simplest workflow consists of two connected tasks. A task represents a particular action or operation. The workflow determines the order in which your tasks are executed by Orchestrator. When constructing workflows, by dragging-and-dropping tasks, it is possible to route the output of one workflow into the input of another workflow. This connecting of multiple tasks is how complex workflows are created. Following is a workflow detailing steps needed to bring-up a locally booted rack server with either an ESXi or Linux image to add virtual capacity or provide a dedicated bare-metal host depending on the need. Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 115 Use Cases Figure 113 Choosing Workflow Designer Such an approach can be used to create workflows for consistent deployments using best practices. Other supported categories with tasks out-of-the-box include: • F5 Big IP Tasks • Redhat KVM • EMC V-Max, VPLEX & Isilon tasks • EMC Recover Point tasks • Hyper-V VM & host tasks Customized workflows and application container templates can be assigned to groups/users who have access to the self-service portal to provide benefits associated with such deployments. Use Cases Use cases are a well-known tool for expressing requirements at a high level. It provides a description of how groups of users and their resources may interact with one or more cloud computing systems to achieve specific goals. Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 116 Use Cases The following section presents descriptions of some actors, their goals and an idea of success and failure conditions with a view to clarify the interaction while meeting a subset of IaaS tasks defined by the NIST model. Table 6 cloud-subscriber-useActors Actor Name Description unidentified-user An entity in the Internet (human or script) that interacts with a cloud over the network and that has not been authenticated. cloud-subscriber An entity in the Internet (human or script) that interacts with a cloud over the network and that has not been authenticated. cloud-subscriber-user A user of a cloud-subscriber organization who will be consuming the cloud service provided by the cloud-provider as an end user. For example, an organization's email user who is using a SaaS email service the organization subscribes to would be a cloud-subscriber's user. cloud-subscriber-administrator An administrator type of user of a cloud-subscriber organization that performs (cloud) system related administration tasks for the cloud-subscriber organization. cloud-user A person who is authenticated to a cloud-provider but does not have a financial relationship with the cloud-provider. payment-broker A financial institution that can charge a cloud-subscriber for cloud services, either by checking or credit card. cloud-provider An organization providing network services and charging cloud-subscribers. A (public) cloud-provider provides services over the Internet. transport-agent A business organization that provides physical transport of storage media such as high-capacity hard drives. legal-representative A court, government investigator, or police. identity-provider An entity that is responsible for establishing and maintaining the digital identity associated with a person, organization, or (in some cases) a software program. [NSTIC]. Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 117 Use Cases Actor Name Description attribute-authority An entity that is responsible for creating and managing attributes (e.g., age, height) about digital identities, and for asserting facts about attribute values regarding an identity in response to requests. [NSTIC]. cloud-management-broker A service providing cloud management capabilities over and above those of the cloud-provider and/or across multiple cloud-providers. Service may be implemented as a commercial service apart from any cloud-provider, as cross-provider capabilities supplied by a cloud-provider or as cloud-subscriber-implemented management capabilities or tools. Account Services Figure 114 Viewing Login Users Open an account • Actors-unidentified-user(grp1-user1), cloud-subscriber(grp1-admin), payment-broker, cloud-provider (admin). • Goals: Cloud-provider opens a new account for an unidentified-user who then becomes a cloud-subscriber. • Assumptions: Service offered, cost and the payment mechanism is known and agreed upon and the user request is valid. Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 118 Use Cases • Success Scenario: The unidentified-user gets: – a unique name for the new account (grp1-user1) – optional: information about the unidentified-user's financials and – when the unidentified-user wants the account opened. (now) The cloud-provider verifies the unidentified-user's financial information. If the information is deemed valid by cloud-provider, the unidentified-user becomes a cloud-subscriber and the cloud-provider returns authentication information that the cloud-subscriber can subsequently use to access the service. • Observation: As “admin”, with “system admin” privileges, created a new user – “grp1-admin”, with “Group Admin” privileges for group_1. Logged back in as grp1-admin and ascertained access as provisioned. Grp1-admin could see and do only what was allowed by the “admin” user. Close an Account • Actors: Unidentified-user, cloud-subscriber, cloud-provider, payment-broker. • Goals: Close an existing account belonging to a group for a cloud-subscriber. • Success Scenario: The cloud-subscriber requests closing an account. • The cloud-provider: – performs the requested actions on the timetable requested; – deletes the cloud-subscriber's payment-broker information from the cloud-provider's systems; and – revokes the cloud-subscriber's authentication information. Now the cloud-subscriber is classified as an unidentified-user. • Observation: Proceeded to close (delete) grp2-user1 by ‘admin’. Tried logging in as grp2-user1 after deletion and was unsuccessful. Data categorized as ‘public’ was still available to the group admin account (grp2-admin) and hence recoverable if necessary. Terminate an Account • Actors: Unidentified-user, cloud-subscriber, cloud-provider. • Goals: Cloud-provider terminates a cloud-subscriber's account. • Assumptions: A cloud-provider determines that a cloud-subscriber's account should be terminated per the terms of the SLA. The issue of multiple accounts for a cloud-subscriber is not considered part of the scope of this use case, nor is the issue of retaining sufficient information to recognize an abusive cloud-subscriber trying to create a new account to continue the abuse. • Success Scenarios: (terminate, IaaS): Possible reasons for termination may be that the cloud-subscriber has violated acceptable usage guidelines (e.g., by storing illegal content, conducting cyber attacks, or misusing software licenses), or that the cloud-subscriber is no longer paying for service. The cloud-provider sends a notice to the cloud-subscriber explaining the termination event and any actions the cloud-subscriber may take to avoid it (e.g., paying overdue bills, deleting offending content) or to gracefully recover data. Optionally, the cloud-provider may freeze the cloud-subscriber's account pending resolution of the issues prompting the termination. The requested actions, charges the cloud-subscriber according to the terms of the service, notifies the cloud-subscriber that the account has been terminated, deletes the cloud-subscriber's payment information from the cloud-provider's system, and revokes the cloud-subscriber's identity credentials. At this point, the cloud-subscriber becomes an unidentified-user. Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 119 Use Cases • Observation: As ‘admin’, a password reset and not revealing the new password will lock the user out while retaining data and provide an opportunity for remediation. A permanent account delete has the effect of removing the user and associated data from the system and convert the user into an unidentified user. Data Services Copy data into the cloud • Actor: Cloud-subscriber, cloud-provider, transport-agent • Goals: Cloud-subscriber initiates a copy of data objects from the cloud-subscriber's system to a cloud-provider's system. Optionally, protect transferred objects from disclosure. • Assumptions: Assumes the Use Case "Open an Account" for cloud-subscriber on cloud-provider's system. The cloud-subscriber has modify access to a named data object container on the cloud-provider's system. • Success Scenario (cloud-subscriber-to-network copy, IaaS): The cloud-subscriber determines a local file for • Success Scenarios: (cloud-subscriber-to-network copy, IaaS): The cloud-subscriber determines a local file for copying to the cloud-provider's system. The cloud-subscriber issues a command to the cloud-provider's system to copy the object to a container on the cloud-provider's system. The command may perform both the object creation and the data transfer, or the data transfer may be performed with subsequent commands. The command specifies the location of the local file, the data encoding of the local file, and the name of the new object within the container. • Observation: There are two scenarios for this case. An ‘upload’ option for placing ova/zip/jar files for build purposes. A second method pertains to file/data transfer from a virtual instance. The upload option is strict with only certain types of files allowed for upload to ‘public’, ‘user’ or ‘group’ space’. Files uploaded to public space is available to all users in the group. Figure 115 Uploading File to Public Space Erase data in the cloud • Actors: Unidentified-user, cloud-subscriber, cloud-provider. • Goals: Erase a data object on behalf of a cloud-subscriber or unidentified-user. • Assumptions: One or more data objects already exist in a cloud-provider's system. A request to erase a data object includes the unique identifiers of the objects to delete.There is no redundant data storage by cloud-provider or redundant copies are deleted together. Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 120 Use Cases • Success: A cloud-subscriber sends a delete-objects request to the cloud-provider's system. At the requested deletion time, the system disables all new attempts to access the object. • Observation: A user with the privilege to delete can remove images and data from vm’s created. The deleted image becomes un-available for others in the group as well. Figure 116 Deleting Data File from the Cloud Identity Management User account provisioning • Actors: Cloud-subscriber, cloud-subscriber-administrator, cloud-provider • Goals: The cloud-subscriber requires to provision (create) user accounts for cloud-subscriber-users to access the cloud. Optimally, the cloud-subscriber requires the synchronization of enterprise system-wide user accounts from enterprise data center-based infrastructure to the cloud, as part of the necessary process to streamline and enforce identical enterprise security (i.e., authentication and access control policies) on cloud-subscriber-users accessing the cloud. Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 121 Use Cases • Assumption: The cloud-subscriber has well defined policies and capabilities for identity and access management for its enterprise IT applications and data objects. The cloud-subscriber has enterprise infrastructure to support the export of cloud-subscriber-user account identity and credential data. The cloud-subscriber can establish trusted connections to these cloud services. • Success: This scenario illustrates how a cloud-subscriber can provision accounts on the IaaS cloud. • Observation: User account provisioning allows for local and domain user creation (User Group -> Domain Users). Figure 117 Provisioning User Account User Authentication • Actors: Cloud-subscriber, cloud-subscriber-user, cloud-provider, identity-provider (optional) • Goals: The cloud-subscriber-user's should be able to authenticate themselves through a central LDAP/Active Directory system. • Assumption: The cloud-subscriber-user's account has been already provisioned in the cloud, see use case • Identity Management: User Account Provisioning. Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 122 Use Cases • Success: This scenario illustrates how a cloud-subscriber-user can authenticate against a cloud-based authentication service using the appropriate credentials to gain access to the cloud-based applications/services. • Observation: A combination of steps such as setting “Authentication Preferences”, “LDAP Integration” and a domain group account provides necessary mechanism. Figure 118 User Authentication Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 123 Use Cases Figure 119 Authentication Preferences Virtual Machine Life Cycle Services Provision Virtual Machine • Actors: cloud-subscriber, cloud-provider • Goals: The cloud-subscriber should have the capability to create VM images that meet its functions, performance and security requirements and launch them as VM instances to meets its IT support needs. • Assumption: The cloud-subscriber has an account with an IaaS cloud service that enables creation of Virtual Machine (VM) images and launching of new VM instances. The cloud-provider shall offer the following capabilities for VM Image creation to the cloud-subscriber: – A set of pre-defined VM images that meets a range of requirements (O/S version,CPU cores, memory, and security) – Tools to create a new VM image from scratch Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 124 Use Cases The cloud-provider shall support the following capabilities with respect to launching of a VM instance.Secure administration of the cloud-subscriber's VM instance through the ability to configure certain ports (e.g., opening of port 22 for enabling a SSH session. • Observation: A generic linux instance, vspex-SR79 was created from the self-service catalog. Provisioning succeeded after sufficient funds were made available for the group and a budget ceiling was removed. Figure 120 Service Request Page Manage/Reconfigure an existing virtual machine • Actors: Cloud-subscriber, cloud-provider • Goals: A cloud-subscriber stops, terminates, reboots, starts or otherwise manages the state of a virtual Instance • Assumptions: A suitable VM image (operating system executables and configuration data) exists. Possible formats include OVF. Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 125 Bill of Material • Success: A cloud-subscriber identifies a VM image to run. The cloud-provider provisions VM and performs the loading and boot-up cycle for the selected image for the requesting cloud-subscriber. Power-on, power-off and resizing of the VM. • Observation: The vspex-SR79 VM was powered-off from UCSD and memory and CPU resized prior to power-on. vCenter status was monitored and noted to reflect correct operation. Decommission a virtual machine • Actors: Cloud-subscriber, cloud-provider • Goals: The cloud-subscriber should have the capability to decommission VM resources that are no longer needed or do not meet functional, performance and security requirements and either reclaim such resources or relinquish to the provider. • Assumption: The cloud-subscriber has an account with an IaaS cloud service that enables decommissioning/removal of Virtual Machine (VM) images. • Success: The cloud-subscriber selects a specific Virtual Machine image supplied by the cloud-provider (O/S, CPU cores, memory, and security) be decommissioned to reclaim/relinquish associated resources. • Observation: A shutdown of the VM in question, while reducing active resource usage from a customer perspective, does not revert back resources for reuse by the provider. A VM delete option is preferred and sought. Bill of Material Table 7 Equipment Details Equipment Quantity Cisco UCS 4X blades in one chassis (5108), • B200 M3-Series blade servers with 128 GB RAM each • C-220 M3 Rack servers with 128 GB RAM each4X blades in one chassis (5108), 2xC-220 rack servers in PoD. 2xC-220 Infrastructure rack servers. Cisco Fabric Interconnect 6248 2 Cisco Nexus 5548UP Switches 2 EMC VNX 5400 storage array 1 EMC Unisphere storage management 1 VMware vSphere 5.5 ESXi hosts 6 VMware vCenter Server 5.5 1 Cisco UCS Director 5.0 1 Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 126 Conclusion Table 8 Component Specification Component Versions: Software Count Network Nexus 5548UP NX-OS-6.0(2)N1(2a) 2 Nexus 1000v 4.2(1)SV2(2.2) 2 Cisco UCS Fabric 2.2(2c)A 2 2.2(2c)C 2 Cisco UCS B200-M3 2.2(2c)B 4 VMware ESXi 5.5 build 1331820 X Cisco eNIC Driver 2.1.2.38 Cisco fNIC Driver 1.5.0.45 VMware vCenter 5.5 1 Services Cisco UCS Manager (UCSM) 2.2(2c) 1 Management Cisco UCS Director 5.0 build 50121 1 Cisco Prime Network Services Controller 3.0(2e) 1 Compute Interconnect 6248 Cisco UCS C220-M3 Storage EMC VNX 5400 05.33.000.5.052 1 Client 1.3.2.1.0051 Conclusion The IaaS platform discussed and deployed using the above procedure uses the common components of Cisco and EMC VSPEX Integrated Systems with compliments to address business requirements such as agility and cost with security. These functional requirements promote uniqueness and innovation in the integrated computing stack, augmenting the original EMC VSPEX architecture with support for essential IaaS services. The result is a framework for the easy and efficient consumption of resources, both within and external to the integrated platform in the form of an application ready IaaS. Such a setup is designed and built to appropriately address the diverse workloads, activities and business goals of any organization. This design and the validation discussed here describe the benefits of Cisco UCS Director and EMC VSPEX integrated stacks. Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 127 References References Cisco Virtualization solution for EMC VSPEX with VMware 5.5: http://www.cisco.com/c/en/us/td/docs/unified_computing/ucs/UCS_CVDs/ucs_vspex_vmw_55.html# wp170829 The NIST Definition of Cloud Computing, Peter Mlle and Timothy Grace: http://csrc.nist.gov/publications/subsists/800-145/SP800-145.pdf Cloud Computing Use Cases, National Institute of Standards and Technology (NIST): http://www.nist.gov/it/cloud/use-cases.cfm Cloud Computing Use Cases rev. 1.0, Cloud Standards Customer Council, 10/2011: http://www.cloudstandardscustomercouncil.org/use-cases/CloudComputingUseCases.pdf Cisco UCS Security: Target of Evaluation (Toe), 11/2012: https://www.commoncriteriaportal.org/files/files/st_vid10403-st.pdf Cisco Secure Enclave Data center Solution for EMC VSPEX: http://www.cisco.com/c/dam/en/us/TD/docs/unified_computing/us/Csco_ucscvd/ucs_vspex_sea.p df Cisco UCS Director Literature: http://www.cisco.com/en/US/products/ps13050 EMC Unified Storage and Multi-tenancy – Technology Concepts and Business Considerations: http://www.emc.com/collateral/hardware/white-papers/h8094-unified-storage-multivalent Cy-wp.pdf EMC Multi-tenant File Storage Solution: http://www.emc.com/collateral/white-papers/h12051-WP-multi-tenant-file-storage.pdf Cisco UCS Director VSPEX Management Guide, Rel 5.0: http://www.cisco.com/c/en/us/TD/docs/unified_computing/us/us-director/perspex-mgmt-guide/5-0/b_ Cisco_UCSD_VSPEX_MGT_GD_50.pdf Cisco UCS Director Administration Guide, Release 5.0: http://www.cisco.com/c/en/us/TD/docs/unified_computing/us/us-director/administration-guide/5-0/b_ Cisco_UCSD_Admin_Guide_50.pdf Cisco Systems Inc., White-paper “Managing Real Cost of On-Demand Enterprise Cloud Services with Charge-back Models”: Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 128 Addendum http://www.techdata.com/content/cloud/files/Cisco/Cloud_Services_Chargeback_Models_White_Pape r.pdf Cisco UCS Director Bare Metal Agent Installation and Configuration Guide, Release 5.0: http://www.cisco.com/c/en/us/TD/docs/unified_computing/us/us-director/ma-install-Config/5-0/b_ucs d_bma_install_config_guide_5_0.pdf PNSC Install and configuration: http://www.cisco.com/c/en/us/TD/docs/netting/virtual_network_mgmt_center/3-0/quick-start-guide/b_ 30_Quick_Start_Guide.pdf Addendum Nexus 1000V Configuration version 4.2(1)SV2(2.2) svs switch edition essential no feature telnet username admin password 5 $1$RX4EhYSt$sCzwSdghCLgUBVl7EX/x/1 role network-admin banner motd #Nexus 1000v Switch# ssh key rsa 2048 ip domain-lookup ip host IAAS_N1kV 10.29.150.149 hostname IAAS_N1kV errdisable recovery cause failed-port-state policy-map type qos jumbo vem 3 host id 72d7e9d5-76a9-e311-1111-010101010108 vem 4 host id 72d7e9d5-76a9-e311-1111-010101010106 vem 5 host id 72d7e9d5-76a9-e311-1111-010101010107 snmp-server user admin network-admin auth md5 0xa2cb98ffa3f2bc53380d54d63b6752db priv 0xa2cb98ffa3f2bc53380d54d63b6752db localizedkey vrf context management ip route 0.0.0.0/0 10.29.150.1 vlan 1,50,60,70,150 vlan 50 name ctrl_Net vlan 60 name Store_Net vlan 70 name vMotion_Net port-channel load-balance ethernet source-mac port-profile default max-ports 32 port-profile type ethernet Unused_Or_Quarantine_Uplink vmware port-group shutdown Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 129 Addendum description Port-group created for Nexus1000V internal usage. Do not use. state enabled port-profile type vethernet Unused_Or_Quarantine_Veth vmware port-group shutdown description Port-group created for Nexus1000V internal usage. Do not use. state enabled port-profile type ethernet mgmt_uplink vmware port-group switchport mode access switchport access vlan 50 channel-group auto mode on mac-pinning no shutdown system vlan 50 state enabled port-profile type ethernet storage_uplink vmware port-group switchport mode access switchport access vlan 60 mtu 9000 channel-group auto mode on mac-pinning no shutdown system vlan 60 state enabled port-profile type ethernet vmotion_uplink vmware port-group switchport mode access switchport access vlan 70 channel-group auto mode on mac-pinning no shutdown system vlan 70 state enabled port-profile type vethernet mgmt_nic capability l3control vmware port-group switchport mode access switchport access vlan 50 no shutdown system vlan 50 state enabled port-profile type vethernet storage_nic vmware port-group switchport mode access switchport access vlan 60 no shutdown state enabled port-profile type vethernet vmotion_nic vmware port-group switchport mode access switchport access vlan 70 no shutdown state enabled vdc IAAS_N1kV id limit-resource limit-resource limit-resource limit-resource limit-resource limit-resource 1 vlan minimum 16 maximum 2049 monitor-session minimum 0 maximum 2 vrf minimum 16 maximum 8192 port-channel minimum 0 maximum 768 u4route-mem minimum 1 maximum 1 u6route-mem minimum 1 maximum 1 interface port-channel1 inherit port-profile mgmt_uplink vem 3 Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 130 Addendum interface port-channel2 inherit port-profile storage_uplink vem 3 interface port-channel3 inherit port-profile vmotion_uplink vem 3 interface port-channel4 inherit port-profile vmotion_uplink vem 4 interface port-channel5 inherit port-profile storage_uplink vem 4 interface port-channel6 inherit port-profile mgmt_uplink vem 4 interface port-channel7 inherit port-profile vmotion_uplink vem 5 interface port-channel8 inherit port-profile storage_uplink vem 5 interface port-channel9 inherit port-profile mgmt_uplink vem 5 interface mgmt0 ip address 10.29.150.149/24 interface Vethernet1 inherit port-profile mgmt_nic description VMware VMkernel, vmk0 vmware dvport 32 dvswitch uuid "88 d6 01 50 5a 67 fc f6-ff b2 2f 14 c3 0f b2 5d" vmware vm mac 0025.B50D.2027 interface Vethernet2 inherit port-profile storage_nic description VMware VMkernel, vmk1 vmware dvport 64 dvswitch uuid "88 d6 01 50 5a 67 fc f6-ff b2 2f 14 c3 0f b2 5d" vmware vm mac 0050.5662.54B4 interface Vethernet3 inherit port-profile vmotion_nic description VMware VMkernel, vmk2 vmware dvport 100 dvswitch uuid "88 d6 01 50 5a 67 fc f6-ff b2 2f 14 c3 0f b2 5d" vmware vm mac 0050.5661.CD2D interface Vethernet4 inherit port-profile mgmt_nic description VMware VMkernel, vmk0 vmware dvport 33 dvswitch uuid "88 d6 01 50 5a 67 fc f6-ff b2 2f 14 c3 0f b2 5d" vmware vm mac 0025.B50D.2023 interface Vethernet5 inherit port-profile storage_nic description VMware VMkernel, vmk1 Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 131 Addendum vmware dvport 65 dvswitch uuid "88 d6 01 50 5a 67 fc f6-ff b2 2f 14 c3 0f b2 5d" vmware vm mac 0050.5666.D092 interface Vethernet6 inherit port-profile vmotion_nic description VMware VMkernel, vmk2 vmware dvport 101 dvswitch uuid "88 d6 01 50 5a 67 fc f6-ff b2 2f 14 c3 0f b2 5d" vmware vm mac 0050.5663.A929 interface Vethernet7 inherit port-profile mgmt_nic description VMware VMkernel, vmk0 vmware dvport 34 dvswitch uuid "88 d6 01 50 5a 67 fc f6-ff b2 2f 14 c3 0f b2 5d" vmware vm mac 0025.B50D.2025 interface Vethernet8 inherit port-profile storage_nic description VMware VMkernel, vmk1 vmware dvport 66 dvswitch uuid "88 d6 01 50 5a 67 fc f6-ff b2 2f 14 c3 0f b2 5d" vmware vm mac 0050.5663.AD91 interface Vethernet9 inherit port-profile vmotion_nic description VMware VMkernel, vmk2 vmware dvport 102 dvswitch uuid "88 d6 01 50 5a 67 fc f6-ff b2 2f 14 c3 0f b2 5d" vmware vm mac 0050.5665.1148 interface Vethernet10 inherit port-profile mgmt_nic description vspex-SR5, Network Adapter 1 vmware dvport 35 dvswitch uuid "88 d6 01 50 5a 67 fc f6-ff b2 2f 14 c3 0f b2 5d" vmware vm mac 0050.5681.79F6 interface Vethernet11 inherit port-profile mgmt_nic description vspex-SR7, Network Adapter 1 vmware dvport 36 dvswitch uuid "88 d6 01 50 5a 67 fc f6-ff b2 2f 14 c3 0f b2 5d" vmware vm mac 0050.5681.0CC9 interface Vethernet12 inherit port-profile mgmt_nic description test-gateway, Network Adapter 1 vmware dvport 37 dvswitch uuid "88 d6 01 50 5a 67 fc f6-ff b2 2f 14 c3 0f b2 5d" vmware vm mac 0050.5681.2431 interface Ethernet3/1 inherit port-profile mgmt_uplink interface Ethernet3/2 inherit port-profile mgmt_uplink interface Ethernet3/3 inherit port-profile storage_uplink interface Ethernet3/4 inherit port-profile storage_uplink interface Ethernet3/5 inherit port-profile vmotion_uplink interface Ethernet3/6 inherit port-profile vmotion_uplink interface Ethernet4/1 Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 132 Addendum inherit port-profile mgmt_uplink interface Ethernet4/2 inherit port-profile mgmt_uplink interface Ethernet4/3 inherit port-profile storage_uplink interface Ethernet4/4 inherit port-profile storage_uplink interface Ethernet4/5 inherit port-profile vmotion_uplink interface Ethernet4/6 inherit port-profile vmotion_uplink interface Ethernet5/1 inherit port-profile mgmt_uplink interface Ethernet5/2 inherit port-profile mgmt_uplink interface Ethernet5/3 inherit port-profile storage_uplink interface Ethernet5/4 inherit port-profile storage_uplink interface Ethernet5/5 inherit port-profile vmotion_uplink interface Ethernet5/6 inherit port-profile vmotion_uplink interface control0 ip address 10.10.50.15/24 line console boot kickstart bootflash:/nexus-1000v-kickstart.4.2.1.SV2.2.2.bin sup-1 boot system bootflash:/nexus-1000v.4.2.1.SV2.2.2.bin sup-1 boot kickstart bootflash:/nexus-1000v-kickstart.4.2.1.SV2.2.2.bin sup-2 boot system bootflash:/nexus-1000v.4.2.1.SV2.2.2.bin sup-2 svs-domain domain id 21 control vlan 1 packet vlan 1 svs mode L3 interface control0 svs connection vcenter protocol vmware-vim remote ip address 10.29.150.150 port 80 vmware dvs uuid "88 d6 01 50 5a 67 fc f6-ff b2 2f 14 c3 0f b2 5d" datacenter-name IAAS-DC admin user n1kUser max-ports 8192 connect vservice global type vsg tcp state-checks invalid-ack tcp state-checks seq-past-window no tcp state-checks window-variation no bypass asa-traffic vnm-policy-agent registration-ip 0.0.0.0 shared-secret ********** log-level Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 133 Addendum Nexus 5548UP Configuration version 6.0(2)N1(2) switchname sjc2-151-d20-n5ka feature cfs eth feature feature feature feature feature telnet distribute interface-vlan hsrp lacp vpc lldp username admin password 5 $1$Rw3QNHwc$fbNyKqKv/i74trVvd2/RX0 no password strength-check role network-admin banner motd #Nexus 5000 Switch # ip domain-lookup class-map type qos class-fcoe class-map type queuing class-fcoe match qos-group 1 class-map type queuing class-all-flood match qos-group 2 class-map type queuing class-ip-multicast match qos-group 2 class-map type network-qos class-fcoe match qos-group 1 class-map type network-qos class-all-flood match qos-group 2 class-map type network-qos class-ip-multicast match qos-group 2 policy-map type network-qos jumbo class type network-qos class-default mtu 9216 multicast-optimize system qos service-policy type qos input fcoe-default-in-policy service-policy type queuing input fcoe-default-in-policy service-policy type queuing output fcoe-default-out-policy service-policy type network-qos jumbo snmp-server user admin network-admin auth md5 0x0e44523313e99361c95947506cfcf98e priv 0x0e44523313e99361c95947506cfcf98e localizedkey vrf context management ip route 0.0.0.0/0 10.10.40.1 vlan 1 vlan 40 name Infra.Mgmt vlan 50 name Prod.Mgmt vlan 60 name NFS-Storage-traffic vlan 70 name vMotion-traffic vlan 150 route-map UCSC permit 10 vpc domain 101 role priority 1000 peer-keepalive destination 10.10.40.7 delay restore 150 port-profile default max-ports 512 Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 134 Addendum interface Vlan1 no shutdown interface Vlan40 description Infra.Mgmt-Network no shutdown no ip redirects ip address 10.10.40.2/24 hsrp version 2 hsrp 4 preempt delay minimum 180 priority 25 forwarding-threshold lower 0 upper 0 timers 1 3 ip 10.10.40.1 interface Vlan50 description Prod.Mgmt-Network no shutdown no ip redirects ip address 10.10.50.2/24 hsrp version 2 hsrp 1 preempt delay minimum 180 priority 200 ip 10.10.50.1 interface Vlan60 description NFS-Storage-Network no shutdown no ip redirects ip address 10.10.60.2/24 hsrp version 2 hsrp 3 preempt delay minimum 180 priority 200 timers 1 3 ip 10.10.60.1 interface Vlan70 description vMotion-traffic no shutdown no ip redirects ip address 10.10.70.2/24 hsrp version 2 hsrp 5 preempt delay minimum 180 priority 200 timers 1 3 ip 10.10.70.1 interface Vlan150 no shutdown no ip redirects ip address 10.29.150.254/24 interface port-channel1 switchport mode trunk switchport trunk allowed vlan 1,40,50,60,70 spanning-tree port type network speed 10000 vpc peer-link Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 135 Addendum interface port-channel17 description UCS-FabA-PC17 switchport mode trunk switchport trunk native vlan 50 switchport trunk allowed vlan 1,40,50,60,70 spanning-tree port type edge trunk vpc 17 interface port-channel18 description UCS-FabB-PC18 switchport mode trunk switchport trunk native vlan 50 switchport trunk allowed vlan 1,40,50,60,70 spanning-tree port type edge trunk vpc 18 interface port-channel23 description NFS-Storage-DM2 switchport access vlan 60 vpc 23 interface port-channel24 description NFS-Storage-DM3 switchport access vlan 60 vpc 24 interface Ethernet1/1 switchport mode trunk switchport trunk allowed vlan 1,40,50,60,70 channel-group 1 mode active interface Ethernet1/2 switchport mode trunk switchport trunk allowed vlan 1,40,50,60,70 channel-group 1 mode active interface Ethernet1/3 switchport access vlan 150 speed 1000 interface Ethernet1/4 switchport access vlan 150 speed 1000 interface Ethernet1/5 switchport access vlan 150 speed 1000 interface Ethernet1/6 switchport access vlan 150 speed 1000 interface Ethernet1/7 switchport access vlan 150 speed 1000 interface Ethernet1/8 switchport access vlan 150 speed 1000 interface Ethernet1/9 switchport access vlan 150 interface Ethernet1/10 Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 136 Addendum switchport access vlan 150 interface Ethernet1/11 description Connected-to-Jumphost.54 switchport access vlan 50 interface Ethernet1/12 switchport access vlan 150 interface Ethernet1/13 description Connected-to-Infra.Esx.18 switchport access vlan 50 interface Ethernet1/14 description Connected-to-Infra.Esx.19 switchport access vlan 50 interface Ethernet1/15 switchport access vlan 50 speed 1000 interface Ethernet1/16 switchport access vlan 50 speed 1000 interface Ethernet1/17 description UCS-FabA-Eth1/17 switchport mode trunk switchport trunk native vlan 50 switchport trunk allowed vlan 1,40,50,60,70 spanning-tree port type edge trunk channel-group 17 mode active interface Ethernet1/18 switchport mode trunk switchport trunk native vlan 50 switchport trunk allowed vlan 1,40,50,60,70 channel-group 18 mode active interface Ethernet1/19 interface Ethernet1/20 interface Ethernet1/21 interface Ethernet1/22 interface Ethernet1/23 switchport access vlan 60 channel-group 23 mode active interface Ethernet1/24 switchport access vlan 60 channel-group 24 mode active interface Ethernet1/25 interface Ethernet1/26 interface Ethernet1/27 interface Ethernet1/28 interface Ethernet1/29 Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 137 Addendum interface Ethernet1/30 interface Ethernet1/31 interface Ethernet1/32 interface Ethernet2/1 interface Ethernet2/2 interface Ethernet2/3 interface Ethernet2/4 interface Ethernet2/5 interface Ethernet2/6 interface Ethernet2/7 interface Ethernet2/8 interface Ethernet2/9 interface Ethernet2/10 interface Ethernet2/11 interface Ethernet2/12 interface Ethernet2/13 interface Ethernet2/14 interface Ethernet2/15 interface Ethernet2/16 interface mgmt0 ip address 10.29.150.160/24 line console line vty boot kickstart bootflash:/n5000-uk9-kickstart.6.0.2.N1.2.bin boot system bootflash:/n5000-uk9.6.0.2.N1.2.bin ip route 0.0.0.0/0 10.29.150.0/24 UCS Director 5.0 VSPEX Orchestration Task Library File Generated On: Thu Sep 25 11:46:42 UTC 2014, System Version: 5.0.0.0(50121) Copyright (C) 2009-2014 Cisco Systems Inc. All rights reserved. Service Container Tasks 1. Provision Container - Network 2. Provision Container - VM 3. Allocate Container VM Resources 4. Verify Container Resource Limits Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 138 Addendum 5. Setup Container Gateway 6. Setup Container F5 Load Balancer 7. Setup Container ASA Gateway 8. Send Container Email 9. De Provision Container - VM 10. De Provision Container - Network 11. Container VM Action 12. Re-Sync Container VMs 13. Allocate Additional Container VM Resources 14. Delete Container Cisco UCS Tasks 1. Select UCS Server 2. Create UCS Server Pool 3. Delete UCS Server Pool 4. Add Servers to UCS Server Pool 5. Delete Servers from UCS Server Pool 6. Associate UCS Service Profile Template 7. Reset UCS Server 8. Power On UCS Server 9. Power Off UCS Server 10. Create UCS Service Profile from Template 11. Create UCS Service Profile 12. Select UCS Service Profile 13. Modify UCS Service Profile Boot Policy 14. Delete UCS Service Profile 15. Associate UCS Service Profile 16. Disassociate UCS Server 17. Disassociate UCS Service Profile 18. Create UCS Boot Policy 19. Modify UCS Boot Policy LUN ID 20. Clone UCS Boot Policy 21. Modify UCS Boot Policy WWPN 22. Create VLAN Group 23. Delete UCS VLAN Group 24. Modify UCS VLAN/VLAN Group Org Permissions 25. Server Maintenance 26. Reacknowledge Server Slot Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 139 Addendum 27. Add VLAN 28. Add VLAN - RG 29. Delete UCS Boot Policy 30. Delete UCS VLAN 31. Add VLAN to Service Profile 32. Delete VLAN from Service Profile 33. Add iSCSI vNIC to Service Profile 34. Delete iSCSI vNIC from Service Profile 35. Add vNIC to UCS Service Profile 36. Delete vNIC from Service Profile 37. Create Service Profile iSCSI Boot Policy 38. Modify Service Profile Boot Policy to Boot From iSCSI 39. Delete VLAN from Service Profile vNIC 40. Add VLAN to vNIC Template 41. Delete VLAN from vNIC Template 42. Create UCS Organization 43. Delete UCS Organization 44. Rename UCS Service Profile 45. Manage UCS Servers 46. Unmanage UCS Servers 47. Verify UCS Server Management State 48. Disassociate UCS Service Profile Template 49. Clone UCS Service Profile Template 50. Delete UCS Service Profile Template 51. Clone UCS Service Profile 52. Add NTP Server to UCSM 53. Set Time Zone to UCSM 54. Delete NTP Server from UCSM 55. Add VLAN to Service Profile vNIC User and Group Tasks 1. Assign Service Profile to Group 2. Unassign Service Profile from Group 3. Add Group 4. Add User 5. Modify User 6. Modify User Password 7. Delete User Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 140 Addendum 8. Delete Group 9. Add User Access Profile 10. Modify User Access Profile 11. Delete User Access Profile 12. Assign vFiler to Group 13. Remove vFiler from Group 14. Assign Vserver to Group 15. Remove Vserver from Group 16. Assign CUIC VLAN to Group 17. Unassign CUIC VLAN from Group 18. Resource Limits to Group 19. Assign Volume Group to Group 20. Remove Volume Group from Group CIMC Tasks 1. Power On/Off CIMC Server 2. Select CIMC Boot Device 3. Configure Rack Server 4. Unconfigure Rack Server 1. Get Service Profile vNICs associated to VMware Hosts 2. Add HostNode to vFiler NFS Export 3. Register iSCSI Storage with Hostnode 4. Register Host with vCenter 5. VMware Host Power Action 6. Mount NFS Datastore 7. Add Hosts to DVSwitch 8. Remove Hosts from DVSwitch 9. Create Host Profile VMware Host Tasks 10. Apply Host Profile 11. Attach Host to Host Profile 12. Detach Host from Host Profile 13. Delete Host Profile 14. Collect Host Profile Inventory 15. Assign VMs from Resource Pool to VDC 16. Unregister Host from vCenter 17. Create Resource Pool Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 141 Addendum 18. Modify Resource Pool 19. Delete Resource Pool 20. VMware Remove Datastore from Host 21. Assign Resource Pool to Group 22. Assign Datastore to Group 23. Unassign Resource Pool from Group 24. Unassign Datastore from Group 25. Create Cluster PNSC Tasks 1. Deploy VSG 2. Undeploy Container VSG 3. Provision Container VSG Network 4. Deprovision Container VSG Network 5. Provision PNSC Policies 6. DeProvision PNSC Policies 7. Add Tenant 8. Delete Tenant 9. Add vDC 10. Delete vDC 11. Add App 12. Delete App 13. Add Tier 14. Delete Tier 15. Add Zone 16. Delete Zone 17. Add Zone Conditions 18. Delete Zone Conditions 19. Add ACL Policy 20. Delete ACL Policy 21. Add ACL Policy Rules 22. Delete ACL Policy Rules 23. Add ACL Policy Set 24. Delete ACL Policy Set 25. Add Compute Security Profile 26. Delete Compute Security Profile 27. Add Compute Firewall 28. Delete Compute Firewall Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 142 Addendum 29. Assign VSG To Compute Firewall 30. Unassign VSG From Compute Firewall 31. Bind Compute Security Profile To Port Profile 32. Unbind Compute Security Profile From Port Profile 33. Bind Compute Firewall To Nexus 1K 34. Unbind Compute Firewall From Nexus 1K General Tasks 1. Set the starting time for the next task 2. Get IP Address From Pool 3. Remove IPAddress Reservation 4. Send Email 5. Send Email Through Template 6. User Approval 7. SSH Command 8. Notify URL 9. Wait for Specified Duration 10. Modify Workflow Priority 11. Execute Cloupia Script 12. Wait For Service Requests 13. Generate VLAN from pool 14. Generate VXLAN from pool 15. Collect Inventory 16. Rollback Child Service Request 17. Budget Allocation 18. Guest Setup 19. Notification 20. Resource Limit 21. VMware Provision Inputs 22. Execute PowerShell Command 23. Reserve Capacity 24. Remove Capacity Reservation Procedural Tasks 1. Start Loop 2. End Loop 3. If Else 4. Conditional Task Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 143 Addendum Generic VM Tasks 1. VM Power Action 2. Select VM Task 3. Modify VM Life Cycle 4. VM SSH Command Business Process Tasks 1. Budget Watch 2. Resource Limit 1. VMware Resource Allocation 2. VMware VM Provision 3. Convert Image As VM 4. Create VM Snapshot 5. Save VM as Template 6. Clone VM as Image 7. Convert VM as Image 8. New VM Provision 9. VM Mount ISO As CD ROM VMware VM Tasks 10. OVF Import to VMware Cloud 11. Revert VM Snapshot 12. Mark/Unmark As Golden Snapshot 13. Delete VM Snapshot 14. Delete all VM Snapshots 15. Execute VIX Script 16. Resize VM Memory and CPU 17. Guest Setup 18. Resize VM Disk 19. VMware VM Resync 20. Create VM Disk 21. Delete VM Disk 22. Execute VM Command 23. File Explorer 24. Migrate VM 25. Resize VMware Generic Datastore 26. VM Configure VNC Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 144 Addendum 27. Delete VMware VM 28. Delete VMware Image 29. Assign VMs to VDC Network Services Tasks 1. Setup PXE Boot 2. Setup PXE Boot With BMA Selection 3. Setup Windows PXE Boot 4. Remove PXE Boot Setup 5. Monitor PXE Boot 6. DNS name to IP Resolver 7. IP Address to DNS name Resolver 1. Verify IPMI Connectivity 2. Power On/Off IPMI Server 3. Select Boot Device 1. Amazon VM Power Action 2. Create Amazon EC2 Volume 3. Create Amazon EC2 Volume from Snapshot 4. Attach Volume to EC2 Instance 5. Detach Volume 6. Delete Volume IPMI Tasks Amazon VM Tasks Network Services Tasks 1. Setup PXE Boot 2. Setup PXE Boot With BMA Selection 3. Setup Windows PXE Boot 4. Remove PXE Boot Setup 5. Monitor PXE Boot 6. DNS name to IP Resolver 7. IP Address to DNS name Resolver 1. Verify IPMI Connectivity 2. Power On/Off IPMI Server IPMI Tasks Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 145 Addendum 3. Select Boot Device 1. Amazon VM Power Action 2. Create Amazon EC2 Volume 3. Create Amazon EC2 Volume from Snapshot 4. Attach Volume to EC2 Instance 5. Detach Volume 6. Delete Volume 1. Provision Network 2. Switch Port Action 3. Configure SAN Zoning 4. Copy Running To Startup Configuration 5. Delete SAN Zone 6. Create VLAN 7. Delete VLAN 8. Create VSAN 9. Delete Network Element Amazon VM Tasks Cisco Network Tasks 10. Delete VSAN 11. Create Port Profile 12. Delete Port Profile 13. Update Port Profile 14. Create Port Channel 15. Delete Port Channel 16. Configure Trunk 17. Configure Access 18. Modify Service Policy 19. Update Trunk 20. Configure VPC Domain 21. Assign Port to Port Channel 22. Assign FC Port to VSAN 23. Delete Device Alias 24. Create Device FCAlias 25. Update Device FCAlias 26. Delete Device FCAlias 27. Create Device Alias Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 146 Addendum 28. Create ACL Entry 29. Delete ACL Entry 30. Add IP ACL Rule 31. Add MAC ACL Rule 32. Assign VLAN to Group 33. Unassign VLAN from Group 34. Configure QOS on Nexus 5K 35. Configure QOS on Nexus 9K 36. Delete N9K QOS Profile 37. Configure QOS on Nexus 1K 38. Create Static MACAddress 39. Remove Static MACAddress 40. 40.Assign Static MACAddress 41. UnAssign MACAddress Port 42. Configure MACAddress Table 43. Configure VTP 44. Create VXLAN 45. Update VXLAN 46. Remove VXLAN 47. Configure PVST 48. Configure Port License 49. Configure Port 50. Configure STP PORT 51. Configure MST INSTANCE 52. Configure MST 53. Assign VXLAN to PortProfile 54. Configure Feature 55. UnAssign VXLAN PortProfile 56. Encapsulate VXLAN PortProfile 57. Create N7K VDC 58. Remove N7K VDC 59. Update N7K VDC 60. Configure VPC PortChannel 61. Remove VPC PortChannel 62. Create VFC Interface 63. Associate VFC Interface 64. Allocate Port To VDC 65. Remove Port From VDC Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 147 Addendum 66. Associate VSAN to VLAN 67. UnAssociate VSAN from VLAN 68. Create SAN Zone 69. Create SAN Zone Set 70. Delete SAN Zone Set 71. Add SAN Zone to Zone Set 72. Add Member To SAN Zone 73. Remove Member From SAN Zone 74. Activate SAN Zone Set 75. Remove San Zone From Zone Set 76. Create SXP Connection Peer 77. Update SXP Connection Peer 78. Remove SXP Connection Peer 79. Create HSRP 80. Update HSRP 81. Remove HSRP 82. Create SVI 83. Remove SVI 84. Delete VFC Interface 85. Remove System VLAN from Port Profile 86. Remove System VLAN Undo Configuration 87. Generic Configure SAN Zoning 88. Create Private VLAN 89. Delete Private VLAN 90. Associate Private VLAN 91. Delete Associate Private VLAN 92. Configure Private VLAN Port 93. Remove Private VLAN Ports 94. Configure Private VLAN Port Profile 95. Execute Network Device CLI 96. Configure System Level HA Cisco Security Tasks 1. Create Security Context 2. Remove Security Context 3. Configure Sub Interface 4. Configure Context Interface 5. Configure Context ACL Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 148 Addendum 6. Configure Context NAT 7. Deploy ASAv OVF 8. TrustSec Refresh 9. Configure NAT 10. Configure License VMware Network Tasks 1. Add Network to VM 2. Create vSwitch 3. Delete vSwitch 4. Create Virtual Nic 5. Delete Virtual Nic 6. Create DVSwitch 7. Delete DVSwitch 8. Enable Discovery Protocol on DVSwitch 9. Enable/Disable vMotion on VMkernel Port 10. Create DVPortGroup 11. Delete DVPortGroup 12. Add Virtual Adapter 13. Generate VMware Generic PortGroup Identity 14. Add VMKernel Port On DVSwitch 15. Remove Virtual Adapters 16. Add Service Console PortGroup 17. Add PNIC to DVSwitch 18. Migrate vSwitch PNIC to DVSwitch 19. Migrate vSwitch VMkernal Port to DVSwitch 20. Migrate Default vSwitch to DVSwitch 21. Migrate Default vSwitch to DVSwitch By Mapping Policy 22. Create VMware Port Group 23. Create VMKernel Port Group 24. Remove VMware Networking 25. Modify VM Network 26. Add VM vNICs 27. Delete VM vNICs 28. Add PNIC to VSwitch 29. Assign Port Group to Group 30. UnAssign Port Group from Group 31. Assign DV Port Group to Group Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 149 Addendum 32. UnAssign DV Port Group from Group 33. Modify PortGroup 34. Update Network Policy VMware Storage Tasks 1.Rescan Storage Adapter EMC VNX Tasks 1. Create VNX Volume 2. Add VNX NFS Export 3. Modify VNX NFS Export 4. Create VNX File System 5. Delete VNX Volume 6. Delete VNX File System 7. Delete VNX NFS Export 8. Delete VNX Storage Pool 9. Extend VNX FileSystem 10. Create VNX Network Interface 11. Delete VNX Network Interface 12. Add VNX DNS Domain 13. Add VNX CIFS Server 14. Add VNX CIFS Share 15. Delete VNX DNS Domain 16. Delete VNX CIFS Share 17. Delete VNX CIFS Server 18. Create VNX RAID Group 19. Delete VNX RAID Group 20. Create VNX Block Storage Pool 21. Delete VNX Block Storage Pool 22. Expand VNX Block Storage Pool 23. Create VNX LUN 24. Delete VNX LUN 25. Add VNX LUN to Storage Group 26. Remove LUN from VNX Storage Group 27. Create VNX Storage Group 28. Delete VNX Storage Group 29. Add VNX Host Initiator Entry 30. Add Hosts to VNX Storage Group Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 150 Addendum 31. Create VNX Meta LUN 32. Create VNX Expand LUN 33. Remove Hosts from VNX Storage Group 34. Remove VNX Initiator 35. Associate VNX LUN as Datastore 36. VNX Storage Disk Allocator vDC Tasks 1. 1.Create vDC 2. Update Storage Policy 3. Update Hyper V Network Policy 4. Undo Update Storage Policy 5. Delete vDC Policy 6. Delete vDC 7. Modify Computing Policy 8. Modify Network Policy 9. Add Network To Network Policy 10. Remove Network From Network Policy Cisco Infrastructure as a Service (IaaS) for EMC VSPEX with UCS Director 5.0 151