Installing vThunder on VMware ESXi August, 2021 © 2021 A10 Networks, Inc.CONFIDENTIAL AND PROPRIETARY- ALL RIGHTS RESERVED. Information in this document is subject to change without notice. PATENT PROTECTION A10 Networks, Inc. products are protected by patents in the U.S. and elsewhere. The following website is provided to satisfy the virtual patent marking provisions of various jurisdictions including the virtual patent marking provisions of the America Invents Act. A10 Networks, Inc. products, including all Thunder Series products, are protected by one or more of U.S. patents and patents pending listed at: a10-virtual-patent-marking. TRADEMARKS A10 Networks, Inc. trademarks are listed at: a10-trademarks CONFIDENTIALITY This document contains confidential materials proprietary to A10 Networks, Inc.. This document and information and ideas herein may not be disclosed, copied, reproduced or distributed to anyone outside A10 Networks, Inc. without prior written consent of A10 Networks, Inc.. DISCLAIMER This document does not create any express or implied warranty about A10 Networks, Inc. or about its products or services, including but not limited to fitness for a particular use and non-infringement. A10 Networks, Inc. has made reasonable efforts to verify that the information contained herein is accurate, but A10 Networks, Inc. assumes no responsibility for its use. All information is provided "as-is." The product specifications and features described in this publication are based on the latest information available; however, specifications are subject to change without notice, and certain features may not be available upon initial product release. Contact A10 Networks, Inc. for current information regarding its products or services. A10 Networks, Inc. products and services are subject to A10 Networks, Inc. standard terms and conditions. ENVIRONMENTAL CONSIDERATIONS Some electronic components may possibly contain dangerous substances. For information on specific component types, please contact the manufacturer of that component. Always consult local authorities for regulations regarding proper disposal of electronic components in your area. FURTHER INFORMATION For additional information about A10 products, terms and conditions of delivery, and pricing, contact your nearest A10 Networks, Inc. location, which can be found by visiting www.a10networks.com. Table of Contents Chapter 1: Introduction to Installing vThunder on VMware ESXi 5 Minimum System Requirements 7 Recommended System Requirements 8 Global License Manager and Types of vThunder Licenses 9 Interfaces 11 Feature Support 13 Limitations 14 Chapter 2: Installing vThunder on VMware ESXi 16 Step 1. Downloading the vThunder Image 17 Step 2. Installing the vThunder Instance 17 Installing vThunder by Using vSphere Client 19 Installing vThunder by Using vCenter Server 22 Adding a New ESXi Hypervisor Host to vCenter 22 Deploying the OVF Template 22 Verifying Configuration of vThunder with Open VM Tools 30 VMware Properties Supported 31 Installing vThunder by Using Web Client 33 Installing vThunder by Using an ISO Image and vSphere Client 36 Step 3. Modifying the vSwitch Settings 42 Step 4. Accessing the vThunder Instance 43 Login Using the CLI 44 Login by Using the GUI 45 Chapter 3: Initial vThunder Configuration 48 Changing the Admin Password 49 Saving the Configuration Changes—Write Memory 49 Chapter 4: Configuring the Management Interface 50 Support for Non-dedicated Management Port Mode 52 Behavior from 2.7.2 Release Onwards 52 Behavior from 4.1.4-GR1-P5 Release Onwards 53 Configuring Non-dedicated Management Port Mode 54 3 Contents Installing vThunder on VMware ESXi Guidelines for Non-dedicated Management Port Mode 54 Adding Extra Ethernet Data Interfaces 55 Adding Extra Port Groups 56 Chapter 4: Advanced vThunder Configuration About Jumbo Frames 58 59 Enabling Jumbo Frames on the Host Side for ESXi 59 Enabling Jumbo Frames for vThunder 59 About Shared Polling Mode 60 Enabling Shared Polling Mode 62 Disabling Shared Polling Mode 63 Memory Support 64 vThunder Configuration on SLB or CGN 64 About SR-IOV and DirectPath I/O 67 Prerequisites for Running SR-IOV or DirectPath I/O 68 Limitations for Running SR-IOV or DirectPath I/O 68 Configuring SR-IOV 70 Configuring DirectPath I/O 71 Configuring vThunder for High Throughput 72 Additional Resources—Where to go from here? 73 4 Chapter 1: Introduction to Installing vThunder on VMware ESXi vThunder for VMware ESXi is a fully operational, software-only version of the ACOS Series Server Load Balancer (SLB), Application Delivery Controller (ADC), SSL Insight (SSLi), IPv6 migration device, CFW or a Carrier-Grade Networking (CGN) device. The maximum throughput of vThunder for VMware ESXi depends on the type of vThunder software license that was purchased and the VM configuration. vThunder is distributed in an ISO format and a non-ISO format (i.e., OVA) from A10 Support. You can install vThunder on a hardware platform running VMware ESXi 4.1 Update 2, VMware ESXi 5.0, VMware ESXi 5.5, VMware ESXi 6.0, VMware ESXi 6.5, VMware ESXi 6.7, or VMware ESXi 7.0 platforms. The product name for the ACOS virtual appliance changed from “SoftAX” to “vThunder” beginning with ACOS 2.7.1-P3 (SLB release) and ACOS 2.8.1 (IPv6 Migration release). This document uses the “vThunder” name, but some file names, directory paths, and screenshots may still refer to “SoftAX”. FIGURE 1-1 shows vThunder running on top of commodity servers (which are running the VMware ESXi hypervisor). 5 Installing vThunder on VMware ESXi Feedback Chapter 1: Introduction to Installing vThunder on VMware ESXi FIGURE 1-1: vThunder for VMware ESXi 6 Installing vThunder on VMware ESXi Feedback Chapter 1: Introduction to Installing vThunder on VMware ESXi Minimum System Requirements The minimum system requirement for configuring ACOS Series Server Load Balancer (SLB), Application Delivery Controller (ADC), SSL Insight (SSLi), IPv6 migration device, CFW or a Carrier-Grade Networking (CGN) device are as follows. The host on which vThunder is installed must meet the following minimal requirements: l 1 CPU (Intel VT-d enabled) l 16 GB disk space l 2 Ethernet ports (1 management interface and 1 data interface) NOTE: vThunder also supports configuring only one network adapter for all interfaces (both data and management). The vThunder instance must meet the following minimum requirements: l 1 vCPU NOTE: l For versions prior to 4.1.4, A10 Networks recommends manually configuring no system-poll-mode enable when changing the number of vCPUs on the VMware host to smaller than the original OVA image. Virtual memory ACOS versions are as follows: NOTE: o 2.7.x, 2.8.x: 2GB o 3.x: 8GB o 4.1.x, (Pre-4.1.4): 4GB o 4.1.4: 8GB o 5.x: 4GB vThunder requires at least 4 GB of virtual memory from version 4.1.4-GR1-P1 onwards. The exact memory requirement depends on features running on the system and data traffic. If memory 7 Installing vThunder on VMware ESXi Feedback Chapter 1: Introduction to Installing vThunder on VMware ESXi usage goes above 80 percent, then increase of existing memory is recommended. l Virtual disk image size requirements: o 10 GB for ACOS 2.7.x and earlier o 12 GB for ACOS 2.7.1-GR1-Px, 2.7.2-Px, and earlier o 16 GB for ACOS 3.x, 4.x, and later NOTE: If a user wants to upgrade vThunder to 4.1.x that is running on 2.7.1, then user must follow two- step process: 1. Upgrade vThunder to 2.7.2. It is recommended to upgrade to 2.7.2-P10 or higher version. 2. Upgrade to the desired 4.1.x release. If version prior to 2.7.2-P10 is being used, then upgrade using CLI. l ACOS software versions: NOTE: o For ADC features – ACOS Release 2.7.1 or later o For CGN features – ACOS Release 2.8.1 or later o For TPS features – ACOS Release 3.1.0 or later “1 Mgmt + 1 data interface” configuration is supported for TPS TAP mode only. l VMware ESXi 4.1 Update 2 client (required unless you plan to install using ovftool) l Separate port groups for each vThunder interface (see Adding Extra Port Groups), configured before you begin installing vThunder Recommended System Requirements The recommended system requirement for configuring ACOS Series Server Load Balancer (SLB), Application Delivery Controller (ADC), SSL Insight (SSLi), IPv6 migration device, CFW or a Carrier-Grade Networking (CGN) device are as follows: 8 Installing vThunder on VMware ESXi Feedback Chapter 1: Introduction to Installing vThunder on VMware ESXi l For better performance SSD is recommended over HDD for disk storage. l 3 Ethernet ports (1 management interface and 2 data interfaces). l Disable Hyper-Threading technology in the system BIOS. l Enable VT-d Virtualization technology in the system BIOS. l 4 or more vCPUs. l CPU pinning is required for optimal performance. That makes VM get a CPU time from only a specific CPU or a set of CPUs. l l Virtual disk image size: o 10 GB for ACOS 2.7.x and earlier o 12 GB for ACOS 2.7.1-GR1-Px, 2.7.2-Px, and earlier o 20 GB for ACOS 3.x, 4.x, and later ACOS software versions: o For ADC features – ACOS Release 4.1.4-P2 or later o For CGN features – ACOS Release 4.1.4-P2 or later o For TPS features – ACOS Release 3.2.2-P5 or later l VMware ESXi 4.1 Update 2 client (required unless you plan to install using ovftool). l Separate port groups for each vThunder interface (see Adding Extra Port Groups), configured before you begin installing vThunder. Global License Manager and Types of vThunder Licenses The GLM is the master licensing system for A10 Networks. The GLM is managed by A10 Networks and is the primary portal for license management for A10 products. The GLM provides a GUI where you can view and manage advanced licensing functions. Creating a GLM account is optional. You can use the ACOS CLI or GUI to license the ACOS devices. A GLM account enables you to perform advanced licensing functions and, where applicable, view and monitor device usage. The GLM portal is available at https://glm.a10networks.com. If you do not yet have a GLM account, contact sales@a10networks.com. vThunder requires a license. Without a license, the product cannot run production traffic, and the amount of bandwidth is only sufficient for testing network connectivity. After you 9 Installing vThunder on VMware ESXi Feedback Chapter 1: Introduction to Installing vThunder on VMware ESXi have downloaded and installed the vThunder software, you need a license before you can run live traffic. A10 Networks offers different types of licenses for your vThunder instance. vThunder supports the following licensing models: l Trial license — Create a trial license in the ACOS GUI. For more information, refer to Global License Manager User Guide. l Perpetual license — This licensing model is based on bandwidth. It is obtained by activation key license for your A10 virtual appliance, URL Classification License installation, and GLM account management. All licenses are generated and installed manually. For more information, refer to chapter Obtaining your Activation Key License in Global License Manager User Guide. l Pay As You Go (PAYG) license — This licensing model is subscription-based. There are two types of licensing models under PAYG licenses. Both these licensing models require that the vThunder instance has an Internet access to request the licenses from an A10 license server. The license models are as follows: o The Rental Billing Model (RBM) is designed for cloud service providers (CSPs) who offer Advanced Delivery Controller (ADC) services. This model enables such providers to bill their customers for a fixed amount of bandwidth, as well as adding surcharges for extra bandwidth consumed. o The Utility Billing Model (UBM) is based on actual data usage, in bytes, in which unlimited vThunder instances can be deployed and in which no bandwidth settings are required. For more information, refer to vThunder Pay-as-you-Go License. l Capacity Pool (FlexPool) license — This licensing model enables you to subscribe to a specific bandwidth pool in the Global License Manager (GLM) for a specific period of time, with an additional option of automatically renewing your license before the license expiry date. Unlike previous license models supported by A10 Networks, capacity pool (FlexPool) license is not node locked. You can configure multiple ACOS devices to share bandwidth from the common license pool. For more information, refer to Capacity Pool License User Guide. 10 Installing vThunder on VMware ESXi Feedback Chapter 1: Introduction to Installing vThunder on VMware ESXi NOTE: When a vThunder license has expired, vThunder functionality will continue, but at a reduced bandwidth. To view any of the above license type, it's features, and how to activate follow the following steps: 1. Sign In to Global License Manager via https://documentation.a10networks.com/signin.html page. 2. Enter your valid A10 Email, Password and then click Sign In tab. The A10 product documentation page is displayed. 3. On A10 Products page, go to Installation Guides for Form Factors section. Choose the product. 4. Click View tab. The Software Installation Guides page is displayed. (i.e. https://documentation.a10networks.com/Install/Software/A10_ACOS_Install/index.html). 5. Click View Licensing Guides option. The portal displays Licensing User Guide section. 6. Click Download PDF tab to open the appropriate Global License Manager guide. Interfaces When installing vThunder from an OVA file, three ports are automatically created (one management and two data ports). If required, you can add or remove data ports after the vThunder instance is deployed. The default ports are: l Management – Dedicated management interface l Ethernet 1 – Data interface l Ethernet 2 – Data interface To connect the vThunder to other devices, you must connect each vThunder interface to a separate port group on the virtual switch (vSwitch) on the VMware host. In a typical deployment, one of the data interfaces is connected to the server farm, and the other data interface is connected to the clients. However, one-arm deployment is also supported which requires one data port and one management port. You also can add additional data interfaces as needed. For more information refer to Adding Extra Ethernet Data Interfaces and Adding Extra Port Groups. 11 Installing vThunder on VMware ESXi Feedback Chapter 1: Introduction to Installing vThunder on VMware ESXi FIGURE 1-2 shows an example of vThunder interface connections. Each vThunder interface is connected to a separate port group on the VMware host’s vSwitch. Each of the port groups is connected to a separate physical interface (NIC). FIGURE 1-2: vThunder for VMware ESXi Interfaces vThunder also supports management connection to the command line interface (CLI) through the console in vSphere Client. The console is required for initial configuration. You can access the ACOS device on the Mgmt (Management), Ethernet 1 (Eth1), and Ethernet 2 (Eth2) interfaces after you configure IP addresses on them and connect them to a port group on a vSwitch. 12 Installing vThunder on VMware ESXi Feedback Chapter 1: Introduction to Installing vThunder on VMware ESXi Feature Support vThunder for VMware ESXi supports many of the same features as the Thunder Series hardware-based models, but the exact set of supported features varies based on whether vThunder is running an ADC (SLB) release, SSLi, or a CGN (IPv6 Migration) release. It supports minimum of 64GB memory that obtains 256K NAT IPs. The virtual Hard disk size in a vThunder can be expanded, even after the creation of the VM. The supported platforms are ESXI, KVM, Hyper-V, and AWS. To expand the virtual hard disk size follow the following steps: 1. Power off the VM. CAUTION: Before shutting down the VM, the user is required to take a back-up of the vThunder VM. 2. Navigate to the “Edit Settings” of the VM. The Virtual Hardware - Edit Setting window is displayed. FIGURE 1-3: Virtual Hardware - Edit Setting Window 3. Enter the size of the Virtual Hard disk. For example 20 GB. 4. Click Save tab to save the changes. 5. Power on the VM. 13 Installing vThunder on VMware ESXi Feedback Chapter 1: Introduction to Installing vThunder on VMware ESXi CAUTION: The size of the virtual disk can only be expanded but cannot be decreased. Refer to the vThunder Software for Virtual and Cloud Infrastructure Data Sheet for a complete summary of supported features. Limitations vThunder for VMware ESXi has the following limitations. l vThunder runs in non-promiscuous mode by default in order to achieve slight performance optimizations. However, the following limitations will apply in non-promiscuous mode: o VE interfaces can be bound to only 1 tagged or untagged physical interface. o VE MAC address assignment scheme changes are not supported. o The Virtualized Network Interface Card (VNIC) in the vSwitch to which the vThunder interface is attached may also need to be set to non-promiscuous mode for proper functioning. If these limitations are problematic, you may remove them by re-enabling promiscuous mode. A vThunder system that is running in non-promiscuous mode can be transitioned back to promiscuous mode with the following command: system promiscuous-mode NOTE: l When making the transition from promiscuous mode to non-promiscuous mode (or vice-versa), the vThunder instance must be reloaded. HA is supported in releases prior to ACOS 4.0. In-line HA for vThunder is supported in promiscuous mode. In ACOS 4.0 and later, HA is no longer supported. Redundancy can only be configured using VRRP-A. l LACP and Static trunk groups are not supported. l Port Mirror is not supported. 14 Installing vThunder on VMware ESXi Feedback Chapter 1: Introduction to Installing vThunder on VMware ESXi l l The maximum binding limitations are as follows: o For vTPS 3.2.x and 5.0.x, maximum vCPU is 48. o For ACOS 5.2.1-Px, maximum vCPU is 96. When the total CPU number is two for vThunder, the command show cpu displays the number as one control CPU and two data CPUs. vThunder-1#show cpu Time: Dec-22-2017, 14:08 1Sec 5Sec 10Sec 30Sec 60Sec -----------------------------------------------------------------------------Control1 11% 13% 20% 21% 36% Data1 0% 0% 0% 0% 0% Data2 0% 0% 0% 0% 0% A similar issue is seen when the total CPU number is one for vThunder. An output similar to the following is displayed: vThunder#show version | inc CPU Number of control CPUs is set to 1 Hardware: 1 CPUs(Stepping 1), Single 20G drive, Free storage is 12G vThunder#show cpu Time: Feb-27-2018, 07:58 1Sec 5Sec 10Sec 30Sec 60Sec -----------------------------------------------------------------------------Control1 5% 20% 12% 6% 5% Data1 3% 20% 11% 6% 5% 15 Chapter 2: Installing vThunder on VMware ESXi You can either install vThunder using the vSphere Client, vCenter server, or the Web client. You can either select an ISO image or an OVF image (OVA file) to install vThunder. Starting from ESXi 6.5, VMware does not support the vSphere Client. NOTE: You can also install vThunder using the ESXi CLI; see the VMware CLI documentation for the procedure. The work-flow is as follows: l Step 1. Downloading the vThunder Image l Step 2. Installing the vThunder Instance l Step 3. Modifying the vSwitch Settings l Step 4. Accessing the vThunder Instance 16 Installing vThunder on VMware ESXi Feedback Chapter 2: Installing vThunder on VMware ESXi Step 1. Downloading the vThunder Image You can download vThunder either as a trial software or a licensed software. To download the vThunder software (trial), log into your Global License Manager (GLM) account and see the following URL: https://glm.a10networks.com/downloads. To download the vThunder software (licensed), see the following URL: https://www.a10networks.com/support/axseries/software-downloads#vthunder. The A10 sales team should have set up a GLM account for you when you first purchase the product. If you do not yet have a GLM account, contact sales@a10networks.com. Step 2. Installing the vThunder Instance If you are installing ACOS version 4.1.4-P2 or later, you have the option of using VMware Tools. For VMware Tools, you must deploy the OVA image by using VMware vCenter and on ESXi version 6.5 or later. If you do not intend to use VMware Tools, you can install the vThunder image for ACOS 4.1.4-P2 or later by using the Web client. Starting with ACOS 5.2.1-P2 open-vm-tool is supported to install ACOS 5.2.1-P2 on ESXi 6.5 or later. NOTE: The vSphere client is not supported from ESXi 6.5 onwards. Installation of ACOS using VMware tools is not supported for versions earlier than 4.1.4-P2. You can use either an OVA image or an ISO image to install vThunder for ESXi. Also, for such ACOS images, earlier versions of ESXi is also supported. NOTE: All new OVA images have VM Tools (properties config while launching the VM) supported from ACOS 4.1.4-P2 or later version. The current OVA, support is as below: 17 Installing vThunder on VMware ESXi Feedback Chapter 2: Installing vThunder on VMware ESXi TABLE 2-1 : OVA Support ESXI Version vSphere Web Client vCenter6.0 or Higher 7.0 Supported Supported 6.7 Supported Supported 6.5 Supported Supported 6 Not Supported Supported 5.5 Not Supported Not Supported 5.1 Not Supported Not Supported To download a specific vThunder image, login to the support portal at https://www.a10networks.com/support and select a vThunder image from the SOFTWARE & DOCUMENTATION tab. After creating the VM, it might take some time for the VM to come up. This is expected behavior. NOTE: If you are installing vThunder on VMware ESXi 6.5 or later on Hewlett Packard- branded hardware and by using an ISO image, change the virtual disk type from VMware Para-virtual to LSI Logical Parallel. Else, the installation may fail with a hard disk error. Based on the ACOS version, you can choose any of the following installation methods to install vThunder on ESXi: l Installing vThunder by Using vSphere Client l Installing vThunder by Using vCenter Server l Installing vThunder by Using Web Client l Installing vThunder by Using an ISO Image and vSphere Client 18 Installing vThunder on VMware ESXi Feedback Chapter 2: Installing vThunder on VMware ESXi Installing vThunder by Using vSphere Client This section describes the process of installing a vThunder image on a vSphere client by using an OVA file. NOTE: vSphere Client is not supported from ESXi 6.5 onwards 1. Download or copy the vThunder OVA archive file into the virtual machine store folder. 2. Select File > Deploy OVF Template. 3. Click Browse and navigate to the vThunder OVA file, and then click Open. 4. Click Next. The OVF Template Details screen is displayed. FIGURE 2-2: OVF Template Details Screen 5. Click Next to view the End User License Agreement screen. 6. Review the license agreement, and if the terms are acceptable, click Accept. 7. Click Next to view the Name and Location screen. 8. If required, edit the default name of the vThunder template 9. Click Next. The Resource Pool screen is displayed. 10. Select the resource pool where you would like to deploy the template. 19 Installing vThunder on VMware ESXi Feedback Chapter 2: Installing vThunder on VMware ESXi FIGURE 2-3: Name and Location Screen NOTE: If a vThunder template is already installed using the default template name, you need to edit a new name for the new template to avoid a conflict. 11. Click Next. The Disk Format screen is displayed. 12. Select Thick provisioned format. This option provides better performance than Thin provisioned format. 13. The Network Mapping screen is displayed. 14. Map each vThunder network interface (Management, Ethernet 1, and Ethernet 2) to a separate port group in the Destination Networks column. 15. To map a network interface, select a vThunder interface in the Source Networks column, and then select the port group from the drop-down list in the Destination Networks column. For example, select source network “Management” and destination network “Mgmt”. 16. The actual names of the port groups may differ. Assign the names when you create them as a prerequisite for vThunder installation. 20 Installing vThunder on VMware ESXi Feedback Chapter 2: Installing vThunder on VMware ESXi FIGURE 2-4: Deploy OVF Template - Network Mapping 17. Click Next to proceed. The Ready To Complete screen is displayed. FIGURE 2-5: Ready to Complete Screen 21 Installing vThunder on VMware ESXi Feedback Chapter 2: Installing vThunder on VMware ESXi 18. Verify that all settings are correct, and click Finish. The vSphere Client deploys the new vThunder virtual machine. 19. Open vSphere Client, if not already open. 20. In the virtual machines inventory, select the vThunder virtual machine. 21. From the menu bar, select Inventory > Virtual Machine > Power > Power On. Installing vThunder by Using vCenter Server This section applies to installing ACOS 4.1.4-P2 or later on ESXi 6.5 or later with VMware Tools. If you are using an older version of the ESXi hypervisor, use a version of ACOS earlier than ACOS 4.1.4-P2. Note that earlier ACOS versions do not support VMware Tools. VMware Tools provide the option to configure important network properties during the VM boot-up process. NOTE: The VMware Tools properties are available with the OVA file. To utilize VMware Tools, download the vThunder image only as an OVA file. To configure VMware Tools properties, use the vCenter server to launch vThunder. Adding a New ESXi Hypervisor Host to vCenter 1. Create a new data center using any of the following options: l File > New > Data Center. l Right-click on the Server in the Navigator pane. Select Create a new Data Center. 2. Right-click on the Data Center DC1 in the Navigator pane. 3. Select Add a Host to add a new host to the Data Center. 4. Enter the IP address for the host and click OK. A new host is created. Deploying the OVF Template 1. Download or copy the vThunder OVA file into the vCenter server. 2. Deploy OVF Template on the new host. Launch the vThunder VM from vCenter using 22 Installing vThunder on VMware ESXi Feedback Chapter 2: Installing vThunder on VMware ESXi the following option: l Right-click on Host and select Deploy OVF Template. FIGURE 2-6: Deploy OVF Template from VM tab in Actions Pane 3. Click Browse and navigate to the vThunder OVA file, and then click Open in the Select an OVF template screen. 23 Installing vThunder on VMware ESXi Feedback Chapter 2: Installing vThunder on VMware ESXi FIGURE 2-7: Select ova image to deploy 4. Click Next to open the Select a name and folder screen. 24 Installing vThunder on VMware ESXi Feedback Chapter 2: Installing vThunder on VMware ESXi FIGURE 2-8: Select a name and folder 5. Enter the Virtual machine name and select a location for the virtual machine. 6. Click Next to open the Select a compute resource screen. 25 Installing vThunder on VMware ESXi Feedback Chapter 2: Installing vThunder on VMware ESXi FIGURE 2-9: Select a compute resource 7. Select the Host to deploy the VM. 8. Click Next to open the Review details screen. 9. Review the advanced configuration options. 10. Click Next to accept the advanced configuration options and to open the License agreements screen. 11. Review the license agreement, and if the terms are acceptable, click Accept. 12. Click Next to open the Select storage screen. 26 Installing vThunder on VMware ESXi Feedback Chapter 2: Installing vThunder on VMware ESXi FIGURE 2-10: Select storage 13. Select Thick provisioned format. This option provides better performance than the Thin provisioned format. 14. The Network Mapping screen is displayed. 15. Map each vThunder network interface (Management, Ethernet 1, and Ethernet 2) to a separate port group in the Destination Networks column. 16. To map a network interface, select a vThunder interface in the Source Networks column, and then select the port group from the drop-down list in the Destination Networks column. For example, select source network “Management” and destination network “Mgmt”. 17. The actual names of the port groups may differ. Assign the names when you create them as a prerequisite for vThunder installation. 18. Click Next to open the Customize Template options. For information on the supported 27 Installing vThunder on VMware ESXi Feedback Chapter 2: Installing vThunder on VMware ESXi parameters see VMware Properties Supported 19. Customize the IP allocation settings and update the network properties. FIGURE 2-11: Customize OVF Template - Network properties 20. Click Next to proceed. The Ready To Complete screen is displayed with details of all the configured network addresses and properties. 28 Installing vThunder on VMware ESXi Feedback Chapter 2: Installing vThunder on VMware ESXi FIGURE 2-12: Ready to complete 21. Verify that all settings are correct, and click Finish. The vCenter Server deploys the new vThunder virtual machine. 22. In the inventory of the virtual machines, select the vThunder virtual machine. FIGURE 2-13: Deployment Completed Screen 23. From the menu bar, select Inventory > Virtual Machine > Power > Power On. 29 Installing vThunder on VMware ESXi Feedback Chapter 2: Installing vThunder on VMware ESXi Verifying Configuration of vThunder with Open VM Tools To verify the vThunder configuration: 1. Log in to vCenter. 2. Open the vThunder CLI console by clicking the CLI icon on the Summary tab of vCenter. FIGURE 2-14: Open vThunder CLI Console 3. Open the command prompt for CLI. Check if the version and interfaces are configured according to the user-specified values in vThunder, using the following commands, the IP address of the management interface is configured on vThunder. vThunder(NOLICENSE)# show interfaces brief Port Link Dupl Speed Trunk Vlan Encap MAC IP Address IPs Flags Name -------------------------------------------------------------------mgmt Up Full 1000 N/A N/A N/A 000c.2997.b9df 10.10.10.10/24 1 1 Up Full 10000 none 1 N/A 000c.2997.b9e9 2.2.2.21/24 1 2 Up Full 10000 none 1 N/A 000c.2997.b9f3 3.3.3.31/24 1 30 Installing vThunder on VMware ESXi Feedback Chapter 2: Installing vThunder on VMware ESXi Global Throughput: 0 bits/sec (0 bytes/sec) Throughput: 0 bits/sec (0 bytes/sec) vThunder(NOLICENSE)# sh run !Current configuration: 99 bytes !Configuration last updated at 07:52:40 IST Wed Apr 21 2021 !Configuration last saved at 07:52:55 IST Wed Apr 21 2021 !64-bit Advanced Core OS (ACOS) version 5.2.1-p2, build 8 (Mar-102021,11:58) ! interface management ip address 10.10.10.10 255.255.255.0 ip default-gateway 20.20.20.20 interface ethernet 1 enable ip address 2.2.2.21 255.255.255.0 ! interface ethernet 2 enable ip address 3.3.3.31 255.255.255.0 ! ! ! VMware Properties Supported The following VMware Tools configuration parameters are supported for vThunder: 31 Installing vThunder on VMware ESXi Feedback Chapter 2: Installing vThunder on VMware ESXi TABLE 2-15 : VMware Tools Configuration Properties Configuration Properties Required/Optional Dependencies and Limitations Management Interface IP address Required Related properties to be configured: Management network mask Management IP allocation type properties Management subnet gateway IP address Required Management subnet gateway IP address is supported. Management IP allocation type (static/DHCP) Required Only static configuration is supported. Management network CIDR (Classless Inter-Domain Routing) Optional N/A Management subnet/network mask for the interface configuration Required N/A Data interface(s) IP address Required Related properties to be configured: Data subnet IP allocation type. Data network mask for the interface configuration. Data subnet IP allocation type (static/DHCP) Required Only static allocation is supported. Data network CIDR Optional N/A 32 Installing vThunder on VMware ESXi Feedback Chapter 2: Installing vThunder on VMware ESXi Configuration Properties Required/Optional Dependencies and Limitations Data network mask for the inter- Required face configuration N/A Network type (management/data) Optional N/A Labels for the interfaces Optional N/A Installing vThunder by Using Web Client You can install vThunder by using the web client. This method is suitable for all ESXi versions and all ACOS versions. However, VMware Tools is not supported for ACOS 4.1.4-P2 if you install by using the web client. Prior to running the installation, ensure that the appropriate vSwitches, port groups, and interfaces are created. In this example, three interfaces are created, out of which one is a management interface while the rest of the two interfaces are data interfaces. For the management interface, the Adapter type must be set to E1000. All data plane interfaces must be set to Adapter type VMXNET3. For the Network option beside each vNIC, select the network to which the vNIC is attached. Ensure Connect at Power On is checked for all the interfaces. NOTE: Setting up a vSwitch and port groups are beyond the scope of this document. Refer to the VMware documentation for more details. Perform the following steps: 1. Navigate to the host URL and launch the Web client. 2. Click Virtual Machines and then click Create/Register VM. 33 Installing vThunder on VMware ESXi Feedback Chapter 2: Installing vThunder on VMware ESXi FIGURE 2-16: Create/Register VM 3. In the New Virtual machine window, click Deploy a virtual machine from an OVF or OVA file. Click Next. 4. In the Select OVF and VMDK files window, enter the name of the virtual machine. 5. Click the designated area to select the file and then browse to the OVA image. Click Open. 6. After the file is displayed in the box, click Next. FIGURE 2-17: Select OVF and VMDK Files 7. In the Select Storage window, select an appropriate datastore and click Next. 34 Installing vThunder on VMware ESXi Feedback Chapter 2: Installing vThunder on VMware ESXi 8. In the license agreements window, scroll to the bottom of the license to click I Agree and then click Next. 9. In the Deployment options screen, complete the network mappings. Ensure Power on automatically is selected. Click Next. FIGURE 2-18: Deployment options 10. Skip the additional settings window by clicking Next. 11. In the Ready to complete window, review the VM properties and click Finish. Click Back to make any last-minute changes. The VM deployment takes some time. NOTE: We support UEFI booting on vThunder VMware ESXi from 5.0.0-P1 release onward. 12. After the VM is created, click the VM and then open the console. 35 Installing vThunder on VMware ESXi Feedback Chapter 2: Installing vThunder on VMware ESXi FIGURE 2-19: Open the Console 13. Log into vThunder with the default username admin and the default password a10. login as: admin Welcome to ACOS Using keyboard-interactive authentication. Password:*** type ? for help] Installing vThunder by Using an ISO Image and vSphere Client To install vThunder by using an ISO image and vSphere Client, perform the following steps: 1. Click on the ESX host (IP shown), then select the Configuration tab. 2. Navigate to Hardware > Storage and from the available datastores, right-click the required datastore to select Browse Datastore. The Datastore Browser window is displayed. 36 Installing vThunder on VMware ESXi Feedback Chapter 2: Installing vThunder on VMware ESXi FIGURE 2-20: Browse Datastore 3. In the Datastore Browser window, click the Upload icon and then click Upload File. FIGURE 2-21: Upload File to Datastore 37 Installing vThunder on VMware ESXi Feedback Chapter 2: Installing vThunder on VMware ESXi 4. Browse to the location where you have saved the vThunder ISO image and select the image. The vThunder ISO image is uploaded. 5. Close the Datastore Browser window. Proceed to install ACOS using vSphere Client. 6. In vSphere Client, to create a new vThunder virtual machine, perform any of the following steps: l Select Create a new virtual machine from the Getting Started page of the host. l Select File > New > Virtual Machine. l Press CTRL+N. FIGURE 2-22: Create a New VM 7. In the installation wizard, update the screens as follows: a. Configuration — Select Typical. b. Name and Location — Enter a name for the VM. If you have VMware vCenter installed, you are prompted to select a folder where the vThunder instance is deployed. 38 Installing vThunder on VMware ESXi Feedback Chapter 2: Installing vThunder on VMware ESXi c. Storage — Select the datastore on which you are going to install the vThunder instance. d. Guest Operating System —Select Other and the version as other 2.6x Linux (64-bit). e. Network — Under Create Network Connections, select the number of virtual network adapters to create. In FIGURE 2-23, three interfaces are created, out of which one is a management interface while the rest of the two interfaces are data interfaces. For the management interface, the Adapter type must be set to E1000. All data place interfaces must be set to Adapter type VMXNET3. For the Network option beside each vNIC, select the network to which the vNIC is attached. Ensure Connect at Power On is checked for all the interfaces. NOTE: Setting up a vSwitch and port groups are beyond the scope of this document. Refer to the VMware documentation for more details. 39 Installing vThunder on VMware ESXi Feedback Chapter 2: Installing vThunder on VMware ESXi FIGURE 2-23: Create Network Connections f. Create a Disk— Enter the virtual disk size and select Thick Provisioned Lazy Zeroed. g. Ready to Complete— To edit the settings further before creating the vThunder VM, check the Edit the virtual machine settings before completion checkbox and click Continue. 8. Under the Virtual Machine properties window, make the following edits: a. Under Hardware, select Memory and specify the size. Select CPUs and specify the number. 40 Installing vThunder on VMware ESXi Feedback Chapter 2: Installing vThunder on VMware ESXi FIGURE 2-24: Configure the vCPUs b. Select New CD/DVD (adding) and ensure Connect at power on is checked. Under Device Type, for Datastore ISO File, click Browse and select your vThunder ISO image. c. (Optional) Select New Floppy (adding) and click Remove. 9. Click Finish. 10. Power on the virtual machine and the system boots to the ISO image in the CD/DVD drive. NOTE: You must first change the BIOS boot setting to boot from CD/DVD drive and then change the boot setting to boot from the hard drive after the installation. 11. After the installation is complete, log in by using the following credentials: localhost login: install and Password: password 12. Type YesS at the prompt to verify the installation. 41 Installing vThunder on VMware ESXi Feedback Chapter 2: Installing vThunder on VMware ESXi FIGURE 2-25: Enter YesS 13. Log into vThunder with the default username admin and the default password a10. login as: admin Welcome to ACOS Using keyboard-interactive authentication. Password:*** [type ? for help] Step 3. Modifying the vSwitch Settings By default, VMware only allows packets that are addressed to a virtual machine (such as the vThunder) to be forwarded to the virtual switch (vSwitch) ports connected to that virtual machine. However, for proper operation, the vThunder must also be able to receive packets that are not addressed to it, such as packets addressed to load-balanced servers. NOTE: The procedure below only applies to VMware's vSwitch. If you are using a third-party virtual switch, such as the Cisco Nexus or Catalyst Series, this procedure may not be necessary. If the vThunder network interfaces are in a tagged VLAN, tagged VLAN mode also must be enabled on the vSwitch. By default, tagged VLAN support is disabled. 42 Installing vThunder on VMware ESXi Feedback Chapter 2: Installing vThunder on VMware ESXi 1. Open vSphere Client, if not already open. 2. In the virtual machines inventory, select the host machine on which the vThunder is installed. 3. Click the Configuration tab. 4. In the Hardware section, click Networking. 5. Click Properties next to the virtual machine to which the vThunder is connected. 6. Click the Port tab. 7. Select the interface. 8. Click Edit. 9. If the vThunder network interfaces are in a tagged VLAN, enter 4095 in the VLAN ID field to enable tagging. Otherwise, leave the VLAN ID set to None. NOTE: If you set enter 4095 in the VLAN ID field, both tagged and untagged packets with any VLAN ID is received by vThunder. If the field is set to None( 0 ) in the VLAN ID field, only untagged packets are received by vThunder. 10. Click OK. 11. Click Close to close the Properties tab. Step 4. Accessing the vThunder Instance Initial configuration of vThunder requires the console. Using the console, you can configure the IP addresses on the management and data interfaces. When you access vThunder by using the ESXi console, vThunder initially boots up with an IP address of 172.31.31.31/24. You can access the vThunder instance remotely by using the management interface, which is also the first interface assigned in VMware. You can access vThunder remotely by using either the CLI or the GUI. To access the vThunder instance by using the console, perform the following steps: 1. In the virtual machines inventory, select the vThunder virtual machine. 2. Click the Console tab or right-click and select Open Console. The Console window is displayed. 43 Installing vThunder on VMware ESXi Feedback Chapter 2: Installing vThunder on VMware ESXi 3. Click on the console window to activate keyboard support for the console window. NOTE: While keyboard support is active for a console window, you cannot interact with other windows. To escape the console, press Ctrl+Alt. 4. You are ready to make the initial configuration changes. See Initial vThunder Configuration. Use the following information to log into the vThunder virtual appliance with for the first time when using the CLI or GUI, as discussed in the next two sections. l Default management IP address — 172.31.31.31 /24 l Default admin username and password — admin, a10 l Default enable password required for configuration access — blank (Press Enter) Login Using the CLI 1. On a PC connected to a network that can access the vThunder management interface, open an SSH client. 2. SSH to the vThunder management IP address. 3. Log into vThunder with the default username admin and the default password a10. login as: admin Welcome to ACOS Using keyboard-interactive authentication. Password:*** [type ? for help] 4. Generally, if this the first time the SSH client has accessed the vThunder instance, the SSH client displays a security warning. Read the warning carefully, then acknowledge the warning to complete the connection. 5. Press Enter. The command prompt for the User EXEC level of the CLI is displayed: ACOS(NOLICENSE) > The User EXEC level allows you to enter a few basic commands, including some show 44 Installing vThunder on VMware ESXi Feedback Chapter 2: Installing vThunder on VMware ESXi commands as well as ping and traceroute NOTE: The vThunder prompt indicates that the vThunder instance is not licensed. 6. To access the Privileged EXEC level of the CLI and allow access to all configuration levels, enter the enable command. 7. At the Password: prompt, press Enter. The command prompt for the Privileged EXEC level of the CLI is displayed as follows: ACOS(NOLICENSE)# 8. To access the global configuration level, enter the configure command. The following command prompt is displayed: ACOS(config)(NOLICENSE)# 9. It is strongly suggested that a Privileged EXEC enable password be set up as follows: ACOS(config)#enable-password newpassword Login by Using the GUI Web access to the vThunder instance is supported on the Web browsers listed in TABLE 2-26. TABLE 2-26 : GUI Browser Support Browser Windows Linux MAC IE 10.0 and higher Supported N/A N/A Firefox 40.0.3 and higher Supported Supported N/A Safari 3.0 and higher Not Supported N/A Supported Chrome 45.0.2454.93 and higher Supported Supported Supported A screen resolution of at least 1024x768 is recommended. To access the vThunder instance by using the GUI, perform the following steps: 45 Installing vThunder on VMware ESXi Feedback Chapter 2: Installing vThunder on VMware ESXi 1. Open a supported web browser. 2. In the URL field, enter the IP address of the management interface of the vThunder instance. 3. If the browser displays a certificate warning, select the option to continue to the server (the ACOS device). NOTE: To prevent the certificate warning from appearing in the future, you can install a certificate signed by a Certificate Authority. A login page is displayed as shown in FIGURE 2-27. The name and appearance of the dialog depends on the browser you are using and the specific device which you are trying to access. FIGURE 2-27: Example GUI Login Dialog 4. Enter your default username admin and default password A10 and click Login. The Dashboard is displayed as shown in FIGURE 2-28, showing at-a-glance information for your vThunder instance. You can access this page again at any time while using the GUI by selecting Dashboard. Refer to the GUI online help for detailed information about this and all other GUI screens. 46 Installing vThunder on VMware ESXi Feedback Chapter 2: Installing vThunder on VMware ESXi FIGURE 2-28: Dashboard NOTE: GUI management sessions are not automatically terminated when you close the browser window. The session remains in effect until it times out. To immediately terminate a GUI session, click the Sign Out icon in the menu bar. 47 Chapter 3: Initial vThunder Configuration This chapter provides information about the initial vThunder configuration. The procedure for applying a license to a vThunder instance depends on the type of license that you have and is documented separately in the licensing guides. For more information, see Global License Manager and Types of vThunder Licenses. The following topics are covered: Changing the Admin Password 49 Saving the Configuration Changes—Write Memory 49 48 Installing vThunder on VMware ESXi Feedback Chapter 3: Initial vThunder Configuration Changing the Admin Password A10 Networks recommends that you change the admin password immediately for security. ACOS(config)#admin admin password newpassword ACOS(config-admin:admin)# The vThunder is now network accessible for configuration under the new IP address and admin password. NOTE: By default, Telnet access is disabled on all interfaces, including the management interface. SSH, HTTP, HTTPS, and SNMP access are enabled by default on the management interface only, and disabled by default on all data interfaces. Saving the Configuration Changes—Write Memory Configuration changes must be saved to system memory to take effect the next time the vThunder is powered on. Otherwise, the changes are lost if the vThunder virtual machine or its host machine are powered down. To write the current configuration to system memory, run the following command: ACOS(config)# write memory Building configuration... [OK] 49 Chapter 4: Configuring the Management Interface The following procedure discusses the assignment of an IP address to the management interface of the vThunder: 1. Configure the management interface IP address and default gateway. Starting with ACOS release 4.1.0, ACOS obtains an IP address for the management interface in the following order: a. If there is a management port IP configuration (either a static IP address or DHCP) in the active startup-config file, then ACOS either assigns the static IP to the vThunder management interface or attempts to get the IP address from the DHCP server. b. If there is no management port IP configuration (neither a static IP address nor DHCP), then vThunder attempts to get an IP address from an accessible DHCP server. c. If vThunder cannot obtain an IP address from a DHCP server, then the default static IP address of 172.31.31.31/24 is used. NOTE: The management interface is an out-of-band interface and should not be on the same subnet as any of the data interfaces. If the management interface and the data interfaces are not kept in separate IP subnets, some operations such as pinging may not perform as expected. In the following example, the IP address for the management interface is 192.168.2.228. None of the data interfaces should have an IP address of 192.168.2.x. ACOS(config)#interface management ACOS(config-if:management)#ip address 192.168.2.228 /24 ACOS(config-if:management)#ip default-gateway 192.168.2.1 2. Verify the interface IP address change: 50 Installing vThunder on VMware ESXi Feedback Chapter 4: Configuring the Management Interface ACOS(config-if:management)#show interface management GigabitEthernet 0 is up, line protocol is up. Hardware is GigabitEthernet, Address is xxxx.yyyy.zzzz Internet address is 192.168.2.228, Subnet mask is 255.255.255.0 ... 3. Optionally, configure the ACOS device to use the management interface as the source interface for automated management traffic generated by the ACOS device: ACOS(config-if:management)#ip control-apps-use-mgmt-port ACOS(config-if:management)#exit ACOS(config)# (For more information, see the “Management Interface as Source for Automated Management Traffic” chapter in the System Configuration and Administration Guide.) 51 Installing vThunder on VMware ESXi Feedback Chapter 4: Configuring the Management Interface Support for Non-dedicated Management Port Mode ACOS offers the ability of running vThunder for VMware in “non-dedicated management port mode”. In this mode, only one network adapter (VMXNET3 device driver) is used for all the interfaces (both data and management). This ability is in contrast with previous releases, in which the e1000 device driver was typically used as the driver for a dedicated management interface and a different driver was used for the data ports. In releases prior to 2.7.2-P4, it was typical for a regular vThunder for VMware instance to have drivers assigned to ports as shown in TABLE 4-1 below. The interfaces could have different drivers assigned to different interfaces. TABLE 4-1 : Drivers Assigned to Ports Management and data ports use different drivers All ports use VMXNET3 driver Eth1 – E1000 Eth1 – VMXNET3 Eth2 – VMXNET3 Eth2 – VMXNET3 Eth3 – VMXNET3 Eth3 – VMXNET3 When all interfaces use the VMXNET3 driver, there is non-dedicated management interface, and any random port can be used to provide management access. Non-dedicated management port mode can be helpful if you are running vThunder for VMware in an environment where it may not be possible to have a dedicated management port. NOTE: The non-dedicated management port mode is not supported in 4.1.4- GR1 release. It's support is added back in 4.1.4- GR1- P5 release with few updates. For more information see Behavior from 4.1.4-GR1-P5 Release Onwards section. Behavior from 2.7.2 Release Onwards Non-dedicated management port mode cannot be enabled or disabled through the CLI or GUI. Instead, the feature is enabled automatically by a new algorithm in the code. 52 Installing vThunder on VMware ESXi Feedback Chapter 4: Configuring the Management Interface This new algorithm runs a check whenever a new vThunder for VMware instance is booting. The algorithm checks for the presence of a dedicated management interface (“eth0”), and if it does not exist, then ACOS automatically enables the “non-dedicated management port mode”. As ACOS is performing this check during boot-up, the algorithm also checks the startup config file. If the startup config file is empty, then ACOS populates the config file with the configuration shown below. This config file defines the interface and allows it to receive an IP address from a DHCP server. Behavior from 4.1.4-GR1-P5 Release Onwards In non-dedicated management port mode, there is no interface dedicated for management traffic: l If vThunder is deployed with one vmxnet3 interface, vThunder operates in non-dedicated management mode by default. l If vThunder is deployed with more than one vmxnet3 interface, vThunder operates in dedicated management mode by default. The first interface is the management interface and the rest of the interfaces operate as data interfaces. The user can display and switch the mode using CLI commands as follows: 1. Enter the below mentioned CLI command to display the current management interface mode: #show system management-interface-mode Description Displays the current interface mode Syntax #show system management-interface-mode Mode EXEC Usage Usage Entering privileged EXEC mode displays the current mode on the interface. Types of mode: - Dedicated - Set the management interface in dedicated mode. - Non-Dedicated - Sets the management interface in non-dedicated mode. By default, the VM operates in a non-dedicated mode. 53 Installing vThunder on VMware ESXi Feedback Chapter 4: Configuring the Management Interface Example In the following example, the user enters privileged EXEC mode using the command. vThunder(NOLICENSE)#show system management-interfacemode Management interface is in dedicated mode 2. Enter the following CLI commands to switch the mode. l # system management-interface-mode dedicated – To switch to dedicated mode. l # system management-interface-mode non-dedicated – To switch to non-dedicated mode. Configuring Non-dedicated Management Port Mode The following is an example of a config file if the admin creates a vThunder instance with 3 interfaces. The number of interfaces in the config file can vary as needed. interface ethernet 1 enable ip address dhcp ! interface ethernet 2 enable ip address dhcp !interface ethernet 3 enable ip address dhcp ! enable-management service enable-management service enable-management service enable-management service ssh ethernet 1 to 3 http ethernet 1 to 3 https ethernet 1 to 3 snmp ethernet 1 to 3 Guidelines for Non-dedicated Management Port Mode l If a vThunder instance is running in “non-dedicated management port mode,” then a DHCP server should be set up for at least one of the interfaces to ensure that management access is possible. 54 Installing vThunder on VMware ESXi Feedback Chapter 4: Configuring the Management Interface l The auto-populated contents of the config file that is automatically created when the “non-dedicated management port mode” is enabled (i.e., the sample shown above) should not be deleted or modified, or this may cause the feature to stop working. l This feature applies to vThunder for VMware and does not apply to any other hypervisor flavors upon which vThunder can run. l This feature is supported in the following releases: ACOS 2.7.2-P4 through 2.7.2-P9, and ACOS 4.1.1 and later. Adding Extra Ethernet Data Interfaces The vThunder has two data interfaces by default. You can add more data interfaces as needed. Before adding an interface, see Adding Extra Port Groups. NOTE: vThunder does not support hot-swapping Ethernet ports. To add a new data port, you must stop the running instance, add the new port or delete an existing port, and then restart the vThunder instance. To add a data interface: 1. In the virtual machines inventory, select the vThunder virtual machine. 2. Click the Getting Started tab, if the page is not already displayed. 3. On the Getting Started page, select Edit virtual machines settings. The Virtual Machine Properties dialog is displayed. 4. Click Add. The Add Hardware dialog is displayed. 5. Select Ethernet Adapter and click Next. 6. In the Adapter Type section, select vmxnet3 from the Type drop-down list. If not available, manually add it first. NOTE: The type for data interfaces is “vmxnet3”, and the type for the management interface is “e1000”. 55 Installing vThunder on VMware ESXi Feedback Chapter 4: Configuring the Management Interface NOTE: To enable “non-dedicated management port mode”, make sure the management interface type is set to “vmxnet3” and not “e1000”. All interfaces should be set to the same driver/adapter (“vmxnet3”). See Support for Non-dedicated Management Port Mode for more information. 7. In the Network Connection section, select the vSwitch for the new vThunder interface, and click Next. 8. Review the configuration information to ensure it is correct, and then click Finish. The vThunder interface is added to the port group on the vSwitch. 9. Reboot the vThunder virtual machine by performing the following steps: a. In the virtual machines inventory, select the vThunder virtual machine. b. From the menu bar, select Inventory > Virtual Machine > Power > Reset. CAUTION: You must reboot the vThunder instance after adding/deleting an Ethernet port, or performance issues may occur. 10. To verify the new interfaces, log onto the vThunder instance using the CLI and enter the following command: show interface brief Compare the MAC addresses of the ACOS interfaces with the MAC addresses on the network interfaces configured in VMware for the vThunder. They should match. Adding Extra Port Groups vThunder requires a separate port group for each vThunder interface (Management, Ethernet 1, and Ethernet 2), configured before you begin vThunder installation. If the port groups are not already created in your ESXi, create them using the steps below. To add a port group to a vSwitch: 1. Start vSphere Client and log onto the VMware host system. 2. In the Inventory, select the host. 56 Installing vThunder on VMware ESXi Feedback Chapter 4: Configuring the Management Interface 3. Click the Configuration tab and select Networking. 4. In the right column, select Properties next to the virtual switch (vSwitch) name. 5. Click Add. 6. Select Virtual Machine as the connection type, and click Next. 7. Edit the name in the Network Label field. This is the name you will select in Step 2. Installing the vThunder Instance. 8. If your ESXi physical interface is not tagged, leave the VLAN ID set to 0. If your ESXi physical interface is tagged, set the VLAN ID to the VLAN tag number. 9. Click Next, then click Finish. 10. Repeat for each port group. The vThunder interfaces must be in separate port groups. 57 Chapter 4: Advanced vThunder Configuration This chapter provides details on how to configure specific advanced features for vThunder. The following topics are covered: About Jumbo Frames 59 About Shared Polling Mode 60 Memory Support 64 About SR-IOV and DirectPath I/O 67 Configuring vThunder for High Throughput 72 Additional Resources—Where to go from here? 73 58 Installing vThunder on VMware ESXi Feedback Chapter 4: Advanced vThunder Configuration About Jumbo Frames A jumbo frame is an Ethernet frame with a payload greater than the standard maximum transmission unit (MTU) of 1,500 bytes. This modification improves vThunder throughput and performance. Additional advantages of enabling jumbo frames include reduced interrupts and lower RAM utilization. For vThunder, jumbo frames are supported on ACOS 2.7.x, 2.8.x and 4.x, 5.x versions, and non-FTA platforms. The following is a list of limitations and requirements for running jumbo frames for the vThunder-Intel and ENA devices: l The vThunder instance must be running on top of an Intel 10Gb Ethernet Controller. l Jumbo frames are not supported on 1Gb NICs. l Supported jumbo frame packet types include: ICMP, UDP and TCP l vThunder can support jumbo frame packets up to a maximum size of 9216 bytes. l Memory assigned to the VM must be greater than 8 GB if using Jumbo Frames. Enabling Jumbo Frames on the Host Side for ESXi Before you enable Jumbo Frames on vThunder, see the documentation about Enabling Jumbo Frames at https://kb.vmware.com/s/article/1007654. Enabling Jumbo Frames for vThunder By default, Jumbo Frame support is disabled. Use the following appropriate CLI command to enable Jumbo Frame support on a vThunder data interface: l For ACOS version 2.7.x: enable-jumbo l For ACOS version 4.1.x: system-jumbo-global enable-jumbo Set the MTU size on the vThunder data interface to a value ranging from 1500 to 9216 bytes. The configured value must be larger than any jumbo packet expected to arrive on that data interface. The command is mtu bytes. You can enable jumbo support on a global basis. In this case, the MTU is not automatically changed on any interfaces, but you can increase the MTU on individual interfaces. 59 Installing vThunder on VMware ESXi Feedback Chapter 4: Advanced vThunder Configuration About Shared Polling Mode ACOS release 4.1.4-GR1-P1 and later only supports shared polling mode1 for deployments having a total number of CPUs less than four. From ACOS release 5.2.0 onwards, this support is also provided for deployments having a total number of CPUs greater than four. When shared polling mode is enabled, both I/O and data processing both are performed by all the vCPUs except the control CPU. If there is no I/O and data processing task in the queue, then the system automatically switches the CPU to idle mode to conserve CPU cycles. NOTE: This mode is only preferred when performance or latency is not the key criterion for the success and the user wants to maximize host CPU utilization due to multiple VMs running on it. 1This support is available on BareMetal and vThunder on KVM, ESXi, Hyper V, AWS, Azure, and OpenStack. 60 Installing vThunder on VMware ESXi Feedback Chapter 4: Advanced vThunder Configuration TABLE 4-2 : ACOS Modes and Selection Criteria Mode Behavior Criteria Additional Requirements Performance Polling Mode In polling mode, both I/O and Data threads continuously poll for the packet and process it. High performance + low latency required, combined with SRIOV. Configure CPU pinning with NUMA. High Performance Maximum utilization of CPU resources with some compromise on latency and performance. The host needs to share physical CPUs with multiple VMs. Lower CPU cycles consumed by the host. High Performance in specific cases. This mode always consumes 100% of the allotted CPU cycles. Note: System poll mode is default for more than 4 vCPUs. Shared Polling Mode NOTE: When the shared poll mode is enabled, I/O and data processing are both performed on all cores except the control CPU. The shared polling mode feature is supported for ACOS 5.2.0 and later versions. 61 Installing vThunder on VMware ESXi Feedback Chapter 4: Advanced vThunder Configuration Enabling Shared Polling Mode By default, shared polling mode is disabled. The following procedure has to be followed to enable Shared Polling mode: 1. Use the following CLI command from global config mode: vThunder(config)#system shared-poll-mode enable 2. Exit global config mode and reload the vThunder instance using the following command: vThunder(config)#exit vThunder#reload After vThunder finishes reloading, Shared Polling Mode will be enabled. 3. To verify Shared Polling Mode is enabled on the vThunder instance, check the output from the “show system shared-poll-mode” command. vThunder(config)# show system shared-poll-mode For example, A2# show system shared-poll-mode Shared poll mode is enabled A2# 4. CPU distribution can be viewed, with the “show cpu” command as shown below. From the output, it can be observed that no CPU does IO processing exclusively. For example, vThunder#show cpu Time: Mar-2-2019, 01:39 1Sec 5Sec 10Sec 30Sec 60Sec -----------------------------------------------------------------------------Control1 15% 15% 14% 18% 18% Data1 0% 0% 0% 0% 0% 62 Installing vThunder on VMware ESXi Feedback Chapter 4: Advanced vThunder Configuration Data2 0% 0% 0% 0% 0% Data3 0% 0% 0% 0% 0% Disabling Shared Polling Mode The following procedure is followed to disable Shared Polling mode: 1. Use the following command from global config mode to disable shared polling mode: For example: vThunder(config)#system shared-poll-mode disable 2. Exit global config mode and reload the vThunder instance using the following command: vThunder(config)#exit vThunder#reload After vThunder finishes reloading, Shared Polling Mode will be disabled. 3. CPU distribution can be viewed, when shared poll mode is disabled with the “show cpu” command as shown below. From the output, it can be observed that some CPUs are designated for IO processing. For example vThunder(config)#show cpu Time: Mar-2-2019, 01:37 1Sec 5Sec 10Sec 30Sec 60Sec -----------------------------------------------------------------------------Control1 20% 21% 21% 21% 21% Data1 0% 0% 0% 0% 0% Data2 0% 0% 0% 0% 0% I/O1 0% 0% 0% 0% 63 Installing vThunder on VMware ESXi Feedback Chapter 4: Advanced vThunder Configuration NOTE: For one vCPU, the control and data usage are shown separately, but both share the same vCPU. The actual usage of the CPU is cumulative of control and data usage. Memory Support vThunder devices support 128 GB memory and provision the resources to satisfy the high number of users and their throughput in a virtualized environment. Both NUMAs inside the compute host are used for provisioning the resources. Memory allocation is 64 GB from NUMA0 and 64 GB from NUMA1. This feature supports all platforms with 2 NUMA, 128 GB memory, and 35 virtual CPUs. NOTE: The memory allocation limits change according to available memory. vThunder Configuration on SLB or CGN To configure vThunder and validate 128 GB memory support, perform the following: 1. Configure the vThunder on SLB or CGN. For example Configure vThunder with SLB as: slb server s1 <Server-IP> port 80 tcp slb server s2 <Server-IP> port 80 tcp slb service-group sg1 tcp member s1 80 member s2 80 64 Installing vThunder on VMware ESXi Feedback Chapter 4: Advanced vThunder Configuration slb virtual-server Platform-vip <VIP> port 80 tcp source-nat auto service-group sg1 Configure vThunder with CGN as: interface ethernet {cli} enable ip address <Data1-IP> <net mask> ip nat inside interface ethernet {srv} enable ip address <Data2-IP> 2xx.xxx.xxx.0 ip nat outside class-list cgn_test <cli_subnet> lsn-lid 1 cgnv6 lsn inside source class-list cgn_test cgnv6 nat pool lsn-pool {pool} netmask /<net-mask> cgnv6 lsn-lid 1 source-nat-pool lsn-pool 2. Verify 128 GB memory support for each vThunder instance in terms of vCPUs and increased application resources such as fixed-NAT public IP addresses, private users count, etc, perform the following: 65 Installing vThunder on VMware ESXi Feedback Chapter 4: Advanced vThunder Configuration a. Launch the vThunder system with 128GB memory and 35 vCPUs ACOS image. b. Verify the limits using show system resource-usage and show cgvn6 resourceusage command. vThunder(NOLICENSE)#sh system resource-usage Resource Current Default Minimum Maximum ---------------------------------------------------------------------------l4-session-count 12582912 12582912 3145728 201326592 nat-pool-addr-count 10 10 10 15000 class-list-ipv6-addr-count 524288 524288 524288 1048576 class-list-ac-entry-count 65536 65536 65536 9216000 auth-portal-html-file-size 20 20 4 120 auth-portal-image-file-size 6 6 1 80 max-aflex-file-size 32 32 16 256 aflex-table-entry-count 102400 102400 102400 15728640 max-aflex-authz-collection-number 512 512 256 4096 radius-table-size 12000000 12000000 2000000 12000000 monitored-entity-count 32960 32960 32816 800288 authz-policy-number 128 128 32 2000 ram-cache-memory-limit 27648 27648 6912 27648 ipsec-sa-number 30000 30000 120 30000 cgn resource-usage vThunder#show cgn resource-usage Resource Current Default Minimum Maximum ------------------------------------------------------------------------lsn-nat-addr-count 2048 2048 2048 20000 66 Installing vThunder on VMware ESXi Feedback Chapter 4: Advanced vThunder Configuration fixed-nat-ip-addr-count 20480 20480 20480 512000 fixed-nat-inside-user-count 256000 256000 256000 8000000 radius-table-size 8000000 8000000 2000000 8000000 vThunder# c. Configure the maximum fixed-NAT IPs and inside users per the default limits and verify that they can be achieved. The default value is 30720k. d. Change the system resource for L4 sessions and reach the count. NOTE: The accumulative L4 session count should be lesser than the current value. Every value don't exceed the current configured value. e. Verify that the configured limits take effect only after reboot. NOTE: For some of the parameter update, reboot is not required. For example -auth-portal-html-file-size - auth-portal-image-file-size - max-aflex-file-size f. On reboot configure the Minimum - maximum number of fixed-NAT IPs and inside “User/RADIUS/IP-List” value between pre-defined range (Min-Max). g. Reboot or reload the system to view the updated value. About SR-IOV and DirectPath I/O Starting from the 4.1.2 P1 release, you can configure vThunder instances running on ESXi for Single Root I/O Virtualization (SR-IOV) or DirectPath I/O. SR-IOV enables a single supported NIC to be assigned as separate logical NICs for multiple vThunder instances. DirectPath I/O enables a supported NIC to be assigned exclusively to a single vThunder instance. Both SR-IOV and DirectPath I/O are recommended for running applications with very high packets and low latency requirements. Both of these features do not support some key virtualization functions. For more information on the limitations, refer to vmware.com/support/pubs. 67 Installing vThunder on VMware ESXi Feedback Chapter 4: Advanced vThunder Configuration For more information, refer to the following: l To understand the prerequisites for vThunder, refer to Prerequisites for Running SRIOV or DirectPath I/O. l To configure SR-IOV for a vThunder instance, refer to Configuring SR-IOV. l To configure DirectPath I/O for a vThunder instance, refer to Configuring DirectPath I/O. l For more information on SR-IOV and DirectPath I/O and how to configure these for ESXi, refer to https://kb.vmware.com/s/article/2038739. Prerequisites for Running SR-IOV or DirectPath I/O Ensure the following list of prerequisites are met for enabling SR-IOV or DirectPath I/O on vThunder: l The hardware platform supports Intel VT-d or IOMMU. l The NIC selected for either SR-IOV or DirectPath I/O belongs of one of the following types: o Intel 82599 10 GbE Controller o Intel Ethernet Converged Network Adapter X710 and XL710 (starting from ACOS 414) l The vThunder instance is configured with four or more CPUs. l The NIC and BIOS settings are enabled for either SR-IOV or DirectPath I/O. Refer to your platform and NIC documentation for more information. l For SR-IOV, the supported ESXi version is 5.1 or higher. For DirectPath I/O, the supported ESXi version is 4.0 or higher. Limitations for Running SR-IOV or DirectPath I/O The following are the list of limitations for running SR-IOV or DirecPath I/O: l For 82599 and X710, SR-IOV and DirectPath I/O for VMware ESXi is not supported in Interrupt mode. Configure Poll mode to support SR-IOV. XL710 supports both Poll mode 68 Installing vThunder on VMware ESXi Feedback Chapter 4: Advanced vThunder Configuration and Interrupt mode. l Tagged VLANs may not work if you configure SR-IOV for X710 and XL710. To resolve the issue, upgrade the ESXi host to 6.5 or newer and reboot the vThunder instance. Upgrade the ESXi host side i40e driver to version 2.0.6 or newer and reboot. Refer to https://my.vmware.com/web/vmware/details?downloadGroup=DT-ESXI60-INTELI40E-206&productId=491 and https://kb.vmware.com/s/article/2137853. Finally, remove the existing i40en driver and reboot the system by using the command esxcli software vib remove -n i40en. l For X710 and 82599, interfaces must be deleted in the reverse order of their addition. For example, in an example vThunder system, the following interfaces are available: eth1— 0000:06:00.0 eth2— 0000:06:01.0 eth3— 0000:06:02.0 eth4— 0000:06:03.0 If eth3 is added at first, followed by eth2 and eth1, the following order is expected: eth3— 0000:06:03.0 eth2— 0000:06:01.0 eth1— 0000:06:00.0 However, the interface order is auto-changed as follows: eth1— 0000:06:00.0 eth2— 0000:06:01.0 eth3— 0000:06:03.0 l Before importing a vThunder instance as an OVF template, remove the SR-IOV or Direct Passthrough interfaces from the vThunder instance. l For vThunder in ESXi host, a mixture of SR-IOV or Direct Passthrough and VMXNET3 data interfaces are not supported. As a workaround, delete the last interface and then add it again. l Promiscuous mode is not allowed in a VF. 69 Installing vThunder on VMware ESXi Feedback Chapter 4: Advanced vThunder Configuration l For X710 and 82599, the multicasts packets received by the ESXi Host NIC are dropped when SR-IOV is enabled for the two VFs created from one physical NIC. However, vThunder can send out the multicasts packets. l Jumbo Frames are not supported for the vThunder instance installed with the 82599 card and with DPDK and SR-IOV enabled. l VCS, VRRP, and IPv6 functions are not supported for the vThunder instance installed with the X710 card and SR-IOV enabled. RIP, OSPF, ISIS, and BGP routing protocols are not supported. However, unicast modes, such as VRRP-A unicast is supported. l Tagged VLAN traffic does not work for the vThunder instance configured with the 82599 SR-IOV interface. Configuring SR-IOV Configuring SR-IOV is a two-step process. First, you must define the virtual functions of the NIC by using the ESXi CLI. Next, you must add the virtual function to the vThunder instance. A virtual function can be mapped to only one vThunder instance. Before configuring SR-IOV, check that your system meets the prerequisites outlined in Prerequisites for Running SR-IOV or DirectPath I/O. Perform the following steps to configure SR-IOV: 1. Log into the ESXi shell and run the following command to get the current configuration of your vmnic: esxcli system module parameters list -m NIC_Driver_Module For example, for the i40e vmnic, the command is: esxcli system module parameters list -m i40e 2. Run the following command to define the maximum number of virtual functions for the vmnic: esxcli system module parameters set -m NIC_Driver_Module -p "max_vfs=n" For example, for the i40e vmnic, to enable two virtual functions each for the seventh and eighth vmnics, the command is as follows: esxcli system module parameters set -m i40e -p "max_vfs=0,0,0,0,0,0,2,2" 70 Installing vThunder on VMware ESXi Feedback Chapter 4: Advanced vThunder Configuration 3. Run the esxcli system module parameters list -m i40e command to check if the settings are correct. 4. Restart the ESXi host for the changes to take effect. 5. Select the vThunder instance in the vSphere client. Do not power on the VM. 6. Right-click the VM and select Edit Settings. The Virtual Machines Properties window is displayed. 7. In the Virtual Machines Properties window, select Add. The Add Hardware window is displayed. 8. In the Add Hardware window, select PCI Device and click Next. The Choose PCI device window is displayed. NOTE: Do not select Ethernet Adapter for adding a SR-IOV NIC. 9. In the Specify the physical PCI/PCIe Device to connect to drop-down menu, select the virtual function and click Next and then Finish. There are four virtual functions listed in the drop-down menu according to the configuration you specified in step 2. In the Virtual Machines Properties window, you see an addition under New PCI Device. 10. Click OK. 11. Power on the VM for the changes to take affect. Configuring DirectPath I/O Configuring DirectPath I/O is a two-step process. First you must activate the DirectPath I/O NIC in the ESXi host and then add the device to the vThunder instance. NOTE: Direct Passthrough is not supported with interrupt mode. Before configuring DirectPath I/O, check that your system meets the prerequisites outlined in Prerequisites for Running SR-IOV or DirectPath I/O. Perform the following steps to configure DirectPath I/O: 71 Installing vThunder on VMware ESXi Feedback Chapter 4: Advanced vThunder Configuration 1. Select the ESXi host from the vSphere client. 2. In the Configuration tab, click Hardware Advanced Settings. The Configuration page lists all available DirectPath I/O devices. A DirectPath I/O device with a green icon is enabled and active. A DirectPath I/O device with an orange icon is disabled. Reboot the host to enable the device. 3. Click Edit. 4. Select the NIC for DirectPath I/O and click OK. 5. Restart the ESXi host for the NIC to become active as a DirectPath I/O device. 6. Select the vThunder instance in the vSphere Client. Do not power on the VM. 7. Right-click the VM and select Edit Settings. The Virtual Machines Properties window is displayed. 8. In the Virtual Machines Properties window, select Add. The Add Hardware window is displayed. 9. In the Add Hardware window, select PCI Device and click Next. The Choose PCI device window is displayed. NOTE: Do not select Ethernet Adapter for adding a DirectPath I/O device. 10. In the Specify the physical PCI/PCIe Device to connect to drop-down menu, select the virtual function and click Next and then Finish. 11. In the Virtual Machines Properties window, you see an addition under New PCI Device. 12. Click OK. Power on the VM for the changes to take affect. Configuring vThunder for High Throughput vThunder supports 40G XL710 NIC cards that can be used to provide a throughput of about 100 Gbps. The following configuration must be supported for installing a minimum of four 40G XL710 NIC cards: 72 Installing vThunder on VMware ESXi Feedback Chapter 4: Advanced vThunder Configuration l A minimum of 16 vCPUS. l A minimum of 16 GB memory and 20 GB hard drive space l Set the interface type to PCI Passthrough. l Disable hyper-threading. Refer to your system manual for specific information to disable hyper-threading. l Enable CPU pinning and static allocation. The procedure is dependent on your operating system, refer to your operating system manual. l Configure ACOS in poll mode. l If the host is a dual-socket machine, it is recommended to pin the cores from both NUMA nodes equally. Additional Resources—Where to go from here? After you have logged into the vThunder GUI or CLI, you may be in need of assistance to configure the device. More information can be found in the latest ACOS Release Notes. This document has a list of new features, known issues, and other information to help get you started. It is also highly recommended to use the basic deployment instructions that appear in the System Configuration and Administration Guide. Feature information is available for ACOS products in the ACOS documents, which are available on the A10 Networks support site. Some relevant links included are: l vThunder data sheet: https://www.a10networks.com/sites/default/files/A10-DS-vThunder.pdf l A10 Networks documentation: https://documentation.a10networks.com/. 73