Converged Network Analyzer Monitoring Guide for the Converged Network Analyzer (CNA), Adaptive Path Controller-Enterprise (APC-E), Adaptive Path Controller-Internet (APC-I), and the CNA Server Products Version 4.1 14-601299 Issue 3 May 2008 Copyright 2008, Avaya Inc. All Rights Reserved This document contains information related to Converged Network Analyzer (CNA) software (as defined below) and Documentation (“Product”). “Documentation” means this document and Avaya’s information manuals in printed or electronic form containing operating instructions and performance specifications that Avaya or its suppliers generally make available to users of its products, and which Avaya delivers to End User with the Products. “End User” means any customer of Avaya or its authorized resellers, or any end user of the Product. See the Software and Documentation DVD/CD inserts for additional legal and licensing information. Notice Changes and corrections to the information in this document may be incorporated in future releases. Disclaimer Avaya, its affiliates or subsidiaries (“Avaya”) are not responsible for any modifications, additions or deletions to the original published version of the Documentation unless such modifications, additions or deletions were performed by Avaya. End User agrees to indemnify and hold harmless Avaya, Avaya's agents, servants, directors, officers, and employees against all claims, lawsuits, demands and judgments arising out of, or in connection with, subsequent modifications, additions or deletions to the Documentation to the extent made by the End User. Warranty Avaya provides a limited warranty on the Product. Refer to your customer sales agreement to establish the terms of the limited warranty. In addition, Avaya’s standard warranty language as well as information regarding support for the Product, while under warranty, is available through the following web site: http://www.avaya.com/support. License USE OR INSTALLATION OF THE PRODUCT INDICATES THE END USER’S ACCEPTANCE OF THE GENERAL LICENSE TERMS AVAILABLE ON THE AVAYA WEBSITE AT: http://www.avaya.com/ support (“GENERAL LICENSE TERMS”). DO NOT USE THE PRODUCT IF YOU DO NOT WISH TO BE BOUND BY THE GENERAL LICENSE TERMS. IN ADDITION TO THE GENERAL LICENSE TERMS, THE FOLLOWING LICENSE TERMS AND RESTRICTIONS WILL APPLY TO THE PRODUCT. Avaya grants End User a license within the scope of the license types described below. The applicable number of licenses and units of capacity for which the license is granted will be one (1), unless a different number of licenses or units of capacity is specified in the Documentation or other materials available to End User. “Designated Processor” means a single stand-alone computing device. “Server” means a Designated Processor that hosts a software application to be accessed by multiple users. “Software” means the computer programs in object code, originally licensed by Avaya and ultimately utilized by End User, whether as stand-alone products or pre-installed on Hardware. “Hardware” means the standard hardware products, originally sold by Avaya and ultimately utilized by End User. If your system is running in a TDM environment, the following license restriction applies: Designated System(s) License (DS). End User may install and use each copy of the Software on only one Designated Processor, unless a different number of Designated Processors is indicated in the Documentation or other materials available to End User. Avaya may require the Designated Processor(s) to be identified by type, serial number, feature key, location or other specific designation, or to be provided by End User to Avaya through electronic means established by Avaya specifically for this purpose. If your system is running in an IP environment, the following license restriction applies: Concurrent User License (CU). End User may install and use the Software on multiple Designated Processors or one or more Servers, so long as only the licensed number of Units are accessing and using the Software at any given time. A “Unit” means the unit on which Avaya, at its sole discretion, bases the pricing of its licenses and can be, without limitation, an agent, port or user, an e-mail or voice mail account in the name of a person or corporate function (e.g., webmaster or helpdesk), or a directory entry in the administrative database utilized by the Product that permits one user to interface with the Software. Units may be linked to a specific, identified Server. For all systems, the following license restriction applies: Shrinkwrap License (SR). With respect to Software that contains elements provided by third party suppliers, End User may install and use the Software in accordance with the terms and conditions of the “shrinkwrap” or “clickwrap” license accompanying the Software (“Shrinkwrap License”). The text of the Shrinkwrap License will be available from Avaya upon End User’s request (see “Copyright” below for more information). Copyright Except where expressly stated otherwise, the Product is protected by copyright and other laws respecting proprietary rights. Unauthorized reproduction, transfer, and or use can be a criminal, as well as a civil, offense under the applicable law. Certain Software programs or portions thereof included in the Product may contain software distributed under third party agreements (“Third Party Components”), which may contain terms that expand or limit rights to use certain portions of the Product (“Third Party Terms”). Information identifying Third Party Components and the Third Party Terms that apply to them is available on Avaya’s web site at http://support.avaya.com/ ThirdPartyLicense/. The disclaimers of warranties and limitations of liability set forth in the Third Party Terms do not affect any express warranty or limitation of liability that may be provided to you by Avaya pursuant to the license terms covering the Product contained in a separate written agreement between you and Avaya. To the extent there is a conflict between the General License Terms or your customer sales agreement and any Third Party Terms, the Third Party Terms shall prevail solely for such Third Party Components. Security and virus disclaimer End User's decision to acquire products from third parties is End User's sole responsibility, even if Avaya helps End User identify, evaluate or select them. Avaya is not responsible for, and will not be liable for, the quality or performance of such third party products or their suppliers. ALL INFORMATION IS BELIEVED TO BE CORRECT AT THE TIME OF PUBLICATION AND IS PROVIDED “AS IS”. AVAYA DISCLAIMS ALL WARRANTIES, EITHER EXPRESS OR IMPLIED, INCLUDING THE WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE AND FURTHERMORE, AVAYA MAKES NO REPRESENTATIONS OR WARRANTIES THAT THE STEPS RECOMMENDED WILL ELIMINATE SECURITY OR VIRUS THREATS TO END USER’ SYSTEMS. IN NO EVENT SHALL AVAYA BE LIABLE FOR ANY DAMAGES WHATSOEVER ARISING OUT OF OR IN CONNECTION WITH THE INFORMATION OR RECOMMENDED ACTIONS PROVIDED HEREIN, INCLUDING DIRECT, INDIRECT, CONSEQUENTIAL DAMAGES, LOSS OF BUSINESS PROFITS OR SPECIAL DAMAGES, EVEN IF AVAYA HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. Avaya does not warrant that this Product is immune from or will prevent unauthorized use of telecommunication services or facilities accessed through or connected to it. Avaya is not responsible for any damages or charges that result from either unauthorized uses or from incorrect installations of the security patches that are made available from time to time. Suspected security vulnerabilities with Avaya products should be reported to Avaya by sending mail to securityalerts@avaya.com. Trademarks All trademarks identified by ® and TM are registered trademarks or trademarks of Avaya Inc. All other trademarks are the property of their respective owners. Contents Chapter 1: Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 Chapter contents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 Intended audience . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Documentation library. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 7 Typographic Conventions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 System requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Web traffic optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . VPN optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 11 12 Supported browsers and operating systems . . . . . . . . . . . . . . . . . . . . 12 Chapter 2: Collecting Endpoint Addresses . . . . . . . . . . . . . . . . 13 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . EFC Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 13 Configuring the EFC Module . . . . . . . . . . CNA IP Addresses . . . . . . . . . . . . . . EFC Module Configuration . . . . . . . . . Access Lists . . . . . . . . . . . . . . . . . Active-Measurement Group Configuration Configuration Summary . . . . . . . . . . . . . . . . . 14 15 15 15 16 17 EFC and Application Traffic Reports . . . . . . . . . . . . . . . . . . . . . . . . . Ensuring that EFC Sees an Appropriate Traffic Set . . . . . . . . . . . . . . . 18 19 Chapter 3: Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 Overview of Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 Authentication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Local, RADIUS or TACACS+ . . . . . . . . . . . . . . . . . . . . . . . . . . . User Session Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 25 26 Enable Password . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 Remote Access . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Telnet . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Secure Shell . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 27 27 Secure Sockets Layer . . . . SSL Certificate/Key Files Management Web Server USTAT-SSL Module . . . Renewing Certificates . . 28 28 29 30 30 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Issue 3 May 2008 3 Contents Access Lists . . . . . . . Rule Order . . . . . . Wildcard Masks . . . Default Access Rules . . . . . . . . . . . . . . . . 31 31 31 33 Chapter 4: Server-Based Measurement . . . . . . . . . . . . . . . . . . 37 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 Setting up server-based measurements . Types of Measurement . . . . . . . . Measurement Frequency . . . . . . . Responding to Measurement Failures Active Measurement VIPs . . . . . . . . . . . . 38 38 42 43 44 Using Link-Groups. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 Named VIPs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 Using Agent-Based Measurement with Performance Groups . . . . . . . . . . . 46 Troubleshooting Agent-Based Measurement . . . . . . . . . . . . . . . . . . . . show active-measurements . . . . . . . . . . . . . . . . . . . . . . . . . . . . show statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46 47 50 Chapter 5: Configuring Agent-Based Measurement . . . . . . . . . . . 59 Chapter contents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59 About agent-based measurement . . . . . . . . . . . . . . . . . . . . . . . . . . About CNA agents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . About Chatter modules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59 60 61 Configuration overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62 Hives . . . . . . . . . . . . . . . . . . . . . . . . . About hives . . . . . . . . . . . . . . . . . . . Assigning Chatter modules to a hive. . . . . . Viewing the master key for a hive . . . . . . . Removing a Chatter module from a hive . . . . Defining down status for Chatter modules . . Synchronizing a Chatter module with its hive . Registering CNA agents with the hive . . . . . Removing a CNA agent from a hive . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63 63 64 65 66 66 67 67 69 Zones . . . . . . . . . . . . . . About zones . . . . . . . . Defining a zone hierarchy . Removing a zone . . . . . Renaming a zone . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69 69 71 73 73 4 CNA Monitoring Guide, Version 4.1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Contents Assigning CNA agents to zones . . . . . . . . . . . . . . . . . . . . . . . . . Removing a CNA agent from a zone . . . . . . . . . . . . . . . . . . . . . . . 74 75 Agent-based measurement tests . . . . . . About agent-based measurement tests Enabling a test . . . . . . . . . . . . . . Disabling a test . . . . . . . . . . . . . Disabling intra-zone tests . . . . . . . . Configuring a test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75 76 81 81 82 82 Agent-based measurement alarms . . . About alarms . . . . . . . . . . . . . Defining alarm destinations . . . . . Disabling absolute alarms globally. Disabling relative alarms globally . Configuring alarm sensitivity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84 84 85 86 87 87 Alarm templates . . . . . . . . . . . . . . . . About alarm templates . . . . . . . . . . Creating an alarm template . . . . . . . . Defining alarm destinations . . . . . . . . Defining test thresholds. . . . . . . . . . Disabling absolute alarms for a test type Disabling relative alarms for a test type . Assigning templates to zones or edges . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88 88 89 89 90 91 92 93 Merging and splitting nodes . . . . . . About merging and splitting nodes Merging nodes . . . . . . . . . . . . Splitting nodes . . . . . . . . . . . . Refreshing known nodes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94 94 95 95 96 Viewing the agent-based measurement configuration . . . . . . . . . . . . . . . 96 Chapter 6: Decision Policies and Application Models . . . . . . . . . . 97 Chapter contents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Decision policies. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Application models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97 97 98 . . . . . . . . . . . . . . . . . . . . . . . . . . . Command modes . . . . . . . . . . . . . . . . Engine command mode . . . . . . . . . . . Policy Configuration command mode . . . Link Policy Configuration command mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98 99 99 100 Issue 3 May 2008 5 Contents Configuration Tasks . . . . . . . . . . . . . . . . . . Configuring a decision policy. . . . . . . . . . . Configuring a specific link for a decision policy Examples . . . . . . . . . . . . . . . . . . . . . . Associating subnets with a policy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101 101 105 106 108 Details on Application Models . . Enterprise application model . Voice application model. . . . Streaming application model . Multimedia application model. Web application model . . . . Other application model. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110 111 114 115 118 120 121 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123 Index 6 CNA Monitoring Guide, Version 4.1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chapter 1: Introduction Chapter contents ● Intended audience ● Documentation library ● Typographic Conventions ● System requirements ● Supported browsers and operating systems Intended audience This document is intended for use by IP network administrators. You should have a detailed knowledge of how IP network work, including an in-depth familiarity with the Border Gateway Protocol (BGP). Documentation library You can download the following documents at http://support.avaya.com: Table 1: Documentation library Document Title Document No. CNA Installation Guide 14-300539 Content ● ● ● CNA Fundamentals Guide 14-601298 ● ● ● Hardware specifications Installing a CNA server Installing a CNA Agent Device Product overview Configuration fundamentals Configuring a new CNA server 1 of 2 Issue 3 May 2008 7 Introduction Table 1: Documentation library (continued) Document Title Document No. Content CNA Monitoring Guide 14-601299 Current path monitoring ● Collecting endpoint addresses ● Security ● Server-based measurement ● Agent-based measurement ● Decision policies and application models Adaptive Path Controller-Enterprise Guide 14-601303 ● ● ● Adaptive Path Controller-Internet Guide 14-602423 ● ● ● CNA Operator Guide 14-601304 ● ● ● CNA Appendices 14-601329 ● ● ● ● ● CNA Command Reference Guide 14-300540 CNA Release Notes Adaptive path control High availability WAN cost and load optimization Configuring your routers Polling routers VPN integration Web interface Server-based measurement reports Agent-based measurement reports Diagnostic tools Supported MIBs Upgrade procedure Migrating configuration files between hardware models API Describes all CLI commands for the CNA server. ● ● ● ● New features Supported hardware platforms, browsers, and operating systems Resolved issues Known issues 2 of 2 To set up user accounts for the web site’s Support area, call: ● 1 (877) 733-5511 in the United States ● 001 (978) 552-0444 outside of the United States Or you can send an e-mail message to support@avaya.com. 8 CNA Monitoring Guide, Version 4.1 Typographic Conventions Typographic Conventions Table 2 shows the typographical conventions that are used in this book: Table 2: Typographic conventions Convention Meaning Courier type The Courier type face is used to denote one of the following: ● Text as displayed on a monitor ● Command syntax examples. bold Bold text denotes command keywords (words that must be entered exactly as shown). italics Italics can mean any of the following: <> ● filenames ● book titles ● command argument names and values ● new terms not commonly understood Words enclosed in angle brackets should be replaced by user-specific data. In the following example, the words userid and password should be replaced by actual user ID and password values: Login: <userid> Password: <password> [] Square brackets are used to denote command line arguments that are optional. 1 of 2 Issue 3 May 2008 9 Introduction Table 2: Typographic conventions (continued) Convention Meaning {} Curly braces enclose a range of choices for a required argument. When used within square brackets, curly braces indicate that if you choose to set the optional argument indicated by the square brackets, then you must choose one of the choices enclosed in the curly brackets. In the example below, the curly brackets indicate that either in or out is always required for the ip access-group command: ip access-group <name> {in | out} In the following example, the version argument is optional for the snmp-serverhost command, but if it is set it must be either 1 or 2c: snmp-serverhost <host-id> [traps | inform] [version {1 | 2c}] | A vertical bar separates multiple choices among arguments. In the following example, the vertical bar indicates that the first argument of the clear subnet command can be either a subnet in standard, dotted-decimal format (1.2.3.4/n) or the keyword all: clear subnet {<subnet> | all} [routes-only] The second argument—the keyword routes-only—is enclosed in square brackets because it is optional. 2 of 2 System requirements The CNA Web client requires Sun’s Java Plug-in 1.5 with update 11. For information about installing the plug-in, see Java Version in Web Interface chapter of the CNA Reporting Guide. Other requirements will vary depending on whether you will be using the CNA system to optimize web or VPN traffic. 10 CNA Monitoring Guide, Version 4.1 System requirements Web traffic optimization If you are using the CNA system to optimize your Web site traffic, your network should meet the following requirements: ● Multi-homed connectivity to the Internet (at least two service providers) ● Full BGP feed from at least one of your service providers ● DNS implementation that provides the following: - address rotation (ability to round-robin destination addresses in reply to successive requests) - high availability, in which the DNS server conducts health checks and won’t return a destination if the target server is not in operation (GSLB—global server load balancer—is strongly recommended, though some form of direct-server return is a viable alternative. For information on getting packets through firewalls, see Active Measurement VIPs in Chapter 4: Server-Based Measurement of the CNA Monitoring Guide. For information on preventing health checks from skewing traffic measurements, see Chapter 6: Decision Policies and Application Models in the CNA Monitoring Guide. ● If Network Address Translation (NAT, as defined in RFC 2663) is in place between the CNA system and your service providers, you must ensure that remote-side responses to CNA measurement-traffic packets arrive at the USTAT’s VIP with original, or natural, addressing intact ● Edge routers must support BGP4 (as defined in RFC 1771 and RFC 1657) and Policy Based Routing (PBR) Issue 3 May 2008 11 Introduction VPN optimization If you are using the CNA system to optimize VPN traffic, your network should meet the following minimum requirements: ● Multi-homed connectivity in at least one direction between your headquarters and remote sites (either two Internet service providers, or one ISP and a private line, such as frame relay) ● Edge routers must support BGP4 (as defined in RFC 1771 and RFC 1657) and Policy Based Routing (PBR) or equivalent. Supported browsers and operating systems Table 3 shows the browsers and operating systems on which the CNA Web interface has been tested. Table 3: Supported Browsers, Operating Systems Operating System Browsers Microsoft Windows 2000 Internet Explorer 6.0 Microsoft Windows XP Internet Explorer 6.0, 7.0 Firefox 2.0 12 CNA Monitoring Guide, Version 4.1 Chapter 2: Collecting Endpoint Addresses Overview If your product includes the optional Endpoint Flow Collection module (EFC), you can obtain probe target addresses automatically, from a NetFlow, SFlow, or SPAN data stream sent to the CNA system from a router or switch. Note: Note: The functionality and commands in this chapter are displayed using NetFlow; however, the functionality and commands are applicable to NetFlow and SFlow. Use access lists to filter traffic so that data flows of interest can be isolated and diverted to an active-measurement group. Different lists can be established to isolate different types of traffic, which can be diverted to different groups. CLI commands allow you to further refine which addresses in the flows are to be used as probe targets. EFC Server The NetFlow and SFlow data are collected by the efcserver process whereas the SPAN data is collected by the monitord processes is sent to the CNA efcserver. The efcserver process maintains the data in a network table, which tracks data per subnet, and an active-measurements table, which tracks individual addresses that are sent to active-measurement groups to be used as probe targets. Data contained in these tables is made visible through the show efc command. You can clear the data in these tables using the clear efc active-measurement and clear efc prefix (deprecated) commands. Note: Note: The term netflow is used by Avaya to refer to both the Netflow process on a router and the CNA process that monitors the incoming data stream. The CNA process that monitors SPAN data is called monitor in the CNA CLI and flow-monitor in Medic reports. The Avaya EFC module requires version 5 or version 7 Netflow packets. Issue 3 May 2008 13 Collecting Endpoint Addresses Configuring the EFC Module On all supported IBM eServer xSeries devices, the EFC module is accessed through the ethernet port labeled ETH3, and optionally on the copper or fiber gigabit ethernet ports GE4 and GE5 on the x335 and labeled GE0 and GE1 on the x345. By default, the CNA system listens for Netflow data on ETH0 and SPAN data on ETH1 on CNA systems, although you can reverse these assignments with the CLI commands monitor interface and netflow interface. The steps needed to configure endpoint collection are as follows: ● configure IP addresses for the EFC fastethernet ports ● specify the UDP port that should be listened to for Netflow data and the router’s Netflow interface IP address ● configure access lists to filter the incoming streams for flows of interest ● configure individual active-measurement groups to accept flows and selectively turn acquired endpoints into probe targets ● enable the process Commands used to configure EFC are: efc enable efc filter efc bytes minimum efc bytes top efc exclude-subnet efc last-update-within efc source ip flow export module efc monitor enable monitor interface netflow enable netflow filter netflow interface netflow port netflow source resync efc filter 14 CNA Monitoring Guide, Version 4.1 Configuring the EFC Module CNA IP Addresses You configure an IP address for the EFC module the same way you configure addresses for your other CNA modules—with the interface command, while in config mode. If you will be receiving both Netflow and SPAN data simultaneously, configure an IP address for each of the EFC module’s two fastethernet ports: Avaya_ANS# config t Avaya_ANS(config)# interface FastEthernet 1/0 Avaya_ANS(config-if)# description “EFC Netflow data listener” Avaya_ANS(config-if)# ip address 172.16.1.173 255.255.255.0 Avaya_ANS(config-if)# end Avaya_ANS(config)# interface FastEthernet 1/1 Avaya_ANS(config-if)# description “EFC SPAN data listener” Avaya_ANS(config-if)# ip address 172.16.1.174 255.255.255.0 Avaya_ANS(config-if)# end EFC Module Configuration If you will be listening for Netflow data, enter config-efc mode and add the IP address of the router interface from which Netflow data will be received and the UDP port number to which the router is sending the Netflow packets: Avaya_ANS(config)# module efc Avaya_ANS(config-efc)# netflow source 192.168.1.15 Avaya_ANS(config-efc)# netflow port 9995 Avaya_ANS(config-efc)# end Note: Note: The port configured on the EFC module must match the port assignment configured on the router. All routers sending Netflow data to the CNA system should use the same UDP port number. You can specify multiple Netflow router IP addresses but only one Netflow UDP port in the CNA configuration. Other than monitor enable, there are no configuration commands needed on the EFC module to listen for SPAN data. Access Lists Use the ip access-list command to filter the data flow. You can define different lists with different criteria. Lists referenced from within an active-measurement group configuration block will control which endpoints from the flow end up as active-measurement probe targets. Issue 3 May 2008 15 Collecting Endpoint Addresses The following list, named filter-traffic, lets web traffic through but prevents telnet traffic from being used: Avaya_ANS(config)# ip access-list filter-traffic Avaya_ANS(config-acl)# remark “Let web traffic through” Avaya_ANS(config-acl)# permit tcp any any port 80 Avaya_ANS(config-acl)# permit tcp any port 80 any Avaya_ANS(config-acl)# remark “Drop telnet traffic” Avaya_ANS(config-acl)# deny tcp any port 23 any Avaya_ANS(config-acl)# deny tcp any any port 23 Avaya_ANS(config-acl)# end If you reference multiple lists within a single active-measurement group, the lists will be applied in the order in which they are specified in the active-measurement group configuration block. Note: If you modify an access list that has already been referenced as an EFC filter from within an active measurement group, you must update the list in the CNA efcserver process, with the resync efc filter command. Note: Active-Measurement Group Configuration You need to enable EFC collection within each active measurement group where you want endpoints to be used as probe targets, with the efc enable command. Use the efc filter command one or more times to identify the access lists you want to use for the active measurement group. You can also define which addresses from within the flow you want to be used as target addresses, with one or more of the following commands: ● efc bytes minimum—specifies a minimum amount of traffic volume a network must have ● efc bytes top—lets you pick prefixes by volume of traffic ● efc last-update-within—specifies a time interval within which a network must have been updated ● efc exclude-subnet—excludes a specific network by address ● efc source—selects either Netflow or SPAN data The access list used here, filter_traffic, was defined in the previous example. When you enter the active-measurement group command for an existing group, subsequently entered commands will be added to the existing group configuration block: Avaya_ANS(config)# module engine Avaya_ANS(config-engine)# active-measurement group efc_targets Avaya_ANS(config-engine-active)# efc enable Avaya_ANS(config-engine-active)# efc filter filter_traffic Avaya_ANS(config-engine-active)# efc source netflow 192.168.1.16 Avaya_ANS(config-engine-active)# efc bytes top 3 Avaya_ANS(config-engine-active)# end 16 CNA Monitoring Guide, Version 4.1 Configuring the EFC Module The efc enable command in the example enables EFC in the active-measurement group efc_targets, which should already have been configured as described in Chapter 5: Configuring Agent-Based Measurement. The efc filter command means the data stream from which the active-measurement group will obtain its target endpoints will be filtered by the access-list filter_traffic, defined in the previous example. The efc source command limits EFC targets for this active-measurement group to endpoints gathered from the Netflow stream originating on the router interface 192.168.1.16. The efc bytes top command means that from that stream, only the three most heavily trafficked prefixes will be probed. CNA also has the ability to define membership in efc-defined active-measurement groups based on BGP source-as. (This function requires that EFC is configured with the NetFlow option; no BGP source-as information is available to the EFC module via the SPAN/Monitor function.) There are two steps to configure this option: 1. Define an ip as-path access-list ip as-path access-list 101 permit 45000 2. Include that as-path access list in an efc filter within an active-measurement group. active-measurement group as-path efc enable efc filter 101 source Configuration Summary After adding the previously defined example commands, your configuration should contain the following configuration blocks: interface FastEthernet 1/0 description “EFC Netflow data listener” ip address 172.16.1.173 255.255.255.0 end ! interface FastEthernet 1/1 description “EFC SPAN data listener” ip address 172.16.1.174 255.255.255.0 end ! ... ip access-list filter-traffic remark “Let web traffic through” permit tcp any any port 80 Issue 3 May 2008 17 Collecting Endpoint Addresses permit tcp any port 80 any remark “Drop telnet traffic” deny tcp any port 23 any deny tcp any any port 23 end ! ... module engine ... active-measurement group efc_targets ... efc enable efc filter filter_traffic efc source netflow 192.168.1.16 efc bytes top 3 end end ! module efc netflow source 192.168.1.15 netflow port 9995 end netflow enable ! EFC and Application Traffic Reports Any CNA system with endpoint flow collection capability can be configured to produce the full range of application traffic reports. This functionality can be based on either NetFlow or SPAN/ Monitor data sources. The scope and significance of the reports is necessarily tied to the comprehensiveness of the observed traffic/traffic flow data. CNA can report on the total of all traffic that the EFC module sees. Thus, the significance of the new application traffic reporting is directly related to the scope and comprehensiveness of the traffic that is presented to the EFC module. For an effective EFC/dynamic measurement functionality, a minimal requirement is to enable the EFC module to build a representative “TopN” table of targets. Note that entirely successful measurement, performance optimization, and load optimization deployments are possible without presenting the complete set of production traffic to the EFC module. ANS5 does not change this aspect of functionality - existing deployments will continue to function as before. However, to take advantage of this new reporting capability, some sites may need to re-evaluate, and make changes to their EFC integration details. More details surrounding these issues are presented below. 18 CNA Monitoring Guide, Version 4.1 EFC and Application Traffic Reports Ensuring that EFC Sees an Appropriate Traffic Set The first issue to be assessed is the nature of the application traffic questions to be answered. There are both topological and directionality aspects to this issue. From a topological perspective, the important factor is that the traffic presented to EFC covers the desired traffic set. For example, this may be the set of all external traffic, or in other contexts, it may be the set of all traffic for a specific section of the network. In either case, it's important that the traffic flow information that the EFC module sees is aligned with the scope of the reporting interest. Note: CNA reports on the volume and nature of traffic flows it observes, but it does so in a way that is completely independent of the in/out directionality of traffic. For example, if only outbound traffic is presented to EFC, then all application traffic reports will represent outbound traffic. If the EFC module sees bi-directional traffic flow data, then all application traffic reports will be based on the sum of inbound and outbound traffic. Note: Ensuring that EFC Sees an Effectively Comprehensive Traffic Set The EFC module may be set up to consume two different types of data: observing actual traffic via SPAN/Monitor functionality or consuming NetFlow data feeds. The SPAN/Monitor approach presents several potential issues, including: ● Issues related to moving SPAN/Monitor traffic across a network ● Issues related to capacity constraints of forcing full-duplex traffic into a single transmit channel ● Issues related to consolidating multiple SPAN/Monitor traffic sources ● Issues related to traffic approaching “Line Rate” In general our experience is that the SPAN/Monitor approach to application traffic reporting faces increasing challenges as the size of the overall network, and volume of traffic, increase. A NetFlow-based approach mitigates several of these constraints: ● NetFlow data export traffic is much easier to move across a network ● NetFlow data export traffic is much smaller than actually traffic, and thus does not face the “Full Duplex Squeeze” issue ● NetFlow data export traffic can be effectively consolidated from multiple sources ● Aggregate NetFlow data export volumes are typically far less than typical line speed configurations Issue 3 May 2008 19 Collecting Endpoint Addresses NetFlow does face some complicating factors of its own, however. In general, the objective is to ensure the comprehensive collection of traffic from a specific “perimeter” of the network. Achieving this goal requires that several factors are considered: ● Per-Interface configuration ● Directionality of NetFlow Data ● Avoiding Double-Counting Data ● Cisco Catalyst-specific considerations NetFlow statistics are only generated on interfaces that have NetFlow configured (via the route-cache flow command, in most cases). Identifying the set of interfaces which should have NetFlow configured requires explicitly identifying both the specific “perimeter” for which traffic is to be reported, as well as the directionality of the desired traffic reporting. Within this scope, the nature of application traffic reporting makes it important to avoid having traffic flows “double counted” as viewed by the EFC module. One common topological pattern to look for here is direct connections between devices that are both NetFlow sources. In general, such “direct interconnection interfaces” should NOT be the source of NetFlow statistics. The Cisco Catalyst architecture presents another set of challenges. First, it is important to understand that there are two completely different paths that packets follow through these devices. In general, the first packet in each flow is initially presented to the Route Processor, which then defines the flow-forwarding pattern; and then future packets within the flow are presented directly to the switch fabric for forwarding. For our purposes, it is important to understand that the Route Processor and the switch fabric are independent sources of NetFlow data for the traffic that is presented to them. Thus, a comprehensive NetFlow data collection configuration requires that BOTH of these levels of the architecture have appropriate Flow Export configurations enabled. This task is further complicated by the fact that these devices can be running in two different configurations, each with its own specific configuration details: a “Native IOS” mode, or a “Split IOS/CatOS” mode. Please see appropriate Cisco documentation for details. Ensure that Observed Flows Contain All Necessary Details “Sample-based” Flow records will obscure information needed for Application Traffic Reporting. Reconciling Directionality for Application Traffic Reporting and Active Measurement The specific definition of appropriate active-measurement targets is configured via the application of access-lists (ACLs) to the overall “Top N” table that the EFC module constructs. Prior to the introduction of the application traffic reporting feature set, the task of appropriate ACL definitions was in some cases more convenient if EFC was presented uni-directional traffic flow data. (For example, only the interfaces on one side of the border routers would be configured as NetFlow sources.) In the event that a production configuration such as this is updated to change the nature of the flow data presented to the EFC module for application traffic reporting, it will be necessary to review and modify the ACLs that define active measurement groups. Please contact Avaya support if you have any questions about these details. 20 CNA Monitoring Guide, Version 4.1 EFC and Application Traffic Reports Identification of Applications CNA application traffic reporting bases “application” definitions on the observed TCP and UDP port information. There are three major classes of “Applications”: ● “Well known” applications, such as port 80/http, port 23/telnet; etc. ● User defined applications ● Dynamically observed application Well known applications all start with lowercase letters, contain no spaces, and are up to (32) characters long. User defined applications are configured within the EFC module via the combination of access list(s), and the user-defined-application name <application-name> command. User defined application names can be up to (32) characters long, and must start with an UPPERCASE letter. Dynamically observed applications are identified with the syntax “port-xxxx”, where “xxxx” corresponds to the observed TCP/UDP port number. Defining Custom Application Names Custom application names can be defined to identify traffic associated with any combination of IP address and port number - any set of traffic that can be identified with an ACL. To define a custom application name, start by defining one or more ACL to identify the traffic of interest. Once the (set of) ACLs have been defined, a custom application can be defined within module efc. Note that the custom application must have a name that starts with an UPPERCASE letter. Avaya_CNA# conf t Avaya_CNA (config)# ip access-list MyAccessList Avaya_CNA (config-acl)# Avaya_CNA (config-acl)# Avaya_CNA (config-acl)#end Avaya_CNA (config-acl)# remark “Access List for definition of custom application on tcp port 10042" Avaya_CNA (config-acl)# permit tcp 0.0.0.0 255.255.255.255 0.0.0.0 255.255.255.255 eq 10042 Avaya_CNA (config-acl)#end Avaya_CNA (config)# Avaya_CNA (config)# module efc Avaya_CNA (config-efc)# Avaya_CNA(config-efc)# user-defined-application name CustomApp efc-filter MyAccessList ? Issue 3 May 2008 21 Collecting Endpoint Addresses endpoint source|destination - Specify the endpoint <cr> Controlling the Recognition and Reporting of Applications There are two related commands within module efc for controlling the recognition of applications: application detection|no-detection exclude application|subnet|filter Sample: Avaya_CNA (config)# application detection? baseliner(config-efc)# application no-detection ? Note: Application Name Definition group Applications associated with a generic group to be detected name Specify application name to be detected discovered All applications discovered by CNA group Applications associated with a generic group to be detected name Specify application name to be detected predefined All applications predefined by CNA user-defined All user defined applications Note: The no application detection command does not instruct EFC to ignore any rules defined for a particular application. To accomplish that task, the user should explicitly configure the application no-detection command defined next. The persistence of the recognition of application endpoints can be controlled with this command from module engine: application age-out. Use the application priority command to ensure that the desired applications appear on the list for live trending. This is useful because of the system-imposed maximum of (100) live-trended 22 CNA Monitoring Guide, Version 4.1 EFC and Application Traffic Reports applications; when there are more applications than this number, then the excess will not be part of the trend-live processing. Please see the CNA Command Reference for details on any of these commands. Application Traffic-Based Measurement CNA adds the ability to define a group based on the applications observed in the EFC traffic. group <groupname> application <application name> This type of group definition can be referenced from within other groups in the configuration, such as report group and/or performance-group. Active-measurement-groups can also be defined based on the applications observed in the EFC traffic: Avaya_CNA# conf t Avaya_CNA (conf)# module engine Sample: Avaya_CNA (config-engine)$ active-measurement group efc application ? Application Name Definition discovered Measurements will be made to the endpoints of any flows observed that are in the discovered set of applications, based on just port numbers group Measurements towards endpoints of particular applications defined as a group name Measurements towards endpoints of a particular application predefined Measurements will be made to the endpoints of any flows observed that match predefined applications user-defined Endpoints for flows matching the user defined applications In either case, targets that are observed to have traffic matching the specified “Application” definitions will be included in the group measurement. Issue 3 May 2008 23 Collecting Endpoint Addresses 24 CNA Monitoring Guide, Version 4.1 Chapter 3: Security Overview of Security Security in the product is achieved through a variety of means, including: ● user authentication at login ● enable password ● secure shell (SSH) protocol for remote communications ● secure socket layer (SSL) protocol for protected web transactions ● access lists for protection of individual CNA interfaces This chapter of the Administrator Guide will describe how each of these security measures is implemented. Authentication Until you configure a user account, there is no local user defined in the CNA configuration. This allows you to initialize a new device through a terminal that is directly connected to the console port on the CNA system’s management module. However, until you have created a local user there is no remote access to the device. Once this user has been created (with the CNA command username), access without authentication is no longer permitted. Local, RADIUS or TACACS+ Authentication can be done locally by the CNA system itself, or by a RADIUS or TACACS+ server. You can specify the authentication methods with the aaa authentication login CLI command. Specify RADIUS servers with the radius-server host command. Specify TACACS+ servers with the tacacs-server host command. You can use these commands multiple times to specify more than one server of either type. If you want to use both RADIUS and TACACS+, the device will attempt the methods in the order specified. Issue 3 May 2008 25 Security If you specified RADIUS as the first method, for example, the CNA system will attempt to authenticate a login request through each of the RADIUS servers that you have identified with the radius-server host command. If none of these servers are able to authenticate the user, the CNA system will attempt to authenticate using the servers specified for the second method. If that fails, or you have not specified a choice in the configuration, the device will try local authentication. If that fails, the user attempting to log on will be denied access. TACACS+ Log In Type The CNA system will contact the server with the server’s TACACS_ASCII_LOGIN authentication-type. Authentication Server Order If you have specified RADIUS then TACACS+ as the order in which authentication servers are to be contacted, users logging in via telnet or ssh will have to re-enter a password if the RADIUS server fails to authenticate. If the order specified is TACACS+ then RADIUS and the TACACS+ server fails to authenticate, the user will not be prompted for a new password. The password entered the first time will be passed on to the RADIUS server. Users logging in through the web interface will not be prompted for a password a second time regardless of the authentication server order. User Session Control The CNA system limits concurrent web client sessions to 350, by writing session cookies to the CNA system’s flash memory or hard disk. See Maximum user sessions in Chapter 2: The Web interface of the Operator Guide. Use the show users command to display a list of user sessions currently open. CLI sessions will be identified as ttypN and web sessions will be identified as webttyN, where N is a number depicting order of log-in. You can force termination of a CLI session with the clear tty command. You cannot force termination of a web session; you must wait for it to time out. The CNA system supports a maximum of 60 CLI sessions. 26 CNA Monitoring Guide, Version 4.1 Enable Password Enable Password When a user successfully logs in with a valid username and password, a limited number of system-operations level commands are available through the Command Line Interface. In order to see or change the CNA configuration, however, the user must also enter the password that has been defined using the CNA enable password command. The CLI will prompt you for the password when you enter the enable command. You can return to log-in level access by entering the logout command. When you are using the management web interface, a dialog will ask for the enable password the first time you attempt to use a feature that requires it. Once you have entered the password, it will remain in memory as long as the browser continues to run. Remote Access You can log on to a configured CNA system from a remote location using either the telnet or secure shell protocol. You can use the management module’s IP address or the device’s host name if recognized by your DNS implementation. See the CNA Installation Guide for more information about enabling remote access for a new device. Telnet The telnet protocol is enabled by default on the CNA system. You can use this protocol to log on from a remote location once you have created a user account and configured an enable password. Secure Shell If you prefer a more secure channel of communications, you can enable the secure shell (ssh) protocol with the ip ssh on command. You can disable the telnet protocol with the no form of the ip telnet on CLI command. Issue 3 May 2008 27 Security Secure Sockets Layer The Secure Sockets Layer protocol (SSL) can be implemented in either or both of two ways: ● You can enable SSL on the management module to use a secure channel for communications between clients and the CNA web interface. ● If you have the optional SSL-enabled USTAT module, you can enable SSL for user traffic tests. SSL Certificate/Key Files Use the ssl add and ssl delete commands without a slot argument to provide or remove SSL certificate and key files for the management module web server. Use these commands with a slot argument when configuring USTAT-SSL modules to serve the user-traffic-test .gif file in response to https:// requests. To add a certificate and private key (and any Intermediate CAs that are required), you need to bundle into a single package the files that you have obtained from a Certificate Authority (CA), including certificate, key, and any chained intermediate certificates that may be provided. See the ssl add command in the CNA Command Reference Guide for information on bundling the certificate and key files, and file-format requirements. Each module requires its own set of certificate/key files, although they do not necessarily need to be unique. SSL certificates can now be associated with logical USTAT modules. This allows multiple, independent SSL certificates to share the single “slot” hardware while co-existing with a (possibly different) SSL certificate for the management module. The requirement is that the host name in the calling URL must exactly match the common name (CN) in the certificate that is returned. That is, if the URL on your web page contains the domain name www.webserver.com, then the certificate that is returned by the USTAT-SSL module must contain the name www.webserver.com. If you run a load balancer to distribute hits for a single DNS name through all of your USTAT-SSL modules, then you only need one set of certificate/key files. CAs require a fully qualified domain name, not an IP address, as the common name (CN) in the certificate. Unless you act as your own CA, each of your USTAT-SSL VIPs must be reachable through DNS, either through your load balancing mechanism or by way of an individual DNS entry. If you continue to encounter problems opening a trusted session, verify that you have installed all required certificates, including any Intermediate CAs that may have been issued when you obtained the certificate and key files. All files issued by your CA must be installed. 28 CNA Monitoring Guide, Version 4.1 Secure Sockets Layer Once you have installed certificate and key files, you can use the ssl export command to send backup copies to a remote location. Note: If you downgrade your CNA image to an earlier release, your certificate and key files will be lost unless you have backed them up prior to reloading the image. Note: Certificate Signing Request File Use the ssl generate-csr command to create a Certificate Signing Request (.csr) file, which you can then send to a CA in order to obtain the certificate and key files. When you execute the command, the CNA system will prompt you for the data the CA needs. Some of the fields may be optional, others required, and some fields may require a specific syntax. You should check with your CA before generating the .csr file. The ssl generate-csr command creates both the CSR and a new server.key file, which is installed automatically. If you are generating a CSR for a module that has already had SSL files installed, use the ssl delete command first to remove the old files. Otherwise, the module’s web server will not function properly. Management Web Server The CNA management web interface can be used in either open or secure mode. By default, the web server on the management module is disabled. You can enable it with either or both of the following commands, while in configuration mode (config): ● ip http server on—starts the web server on the management module ● ip http server ssl on—if the web server is already running, enables SSL or, if the web server is not running, starts the web server and enables SSL You can change web server access ports with the ip http server port and ip http server port ssl commands. You can allow both secure and open communications simultaneously, or you can force all requests through the secure channel. See the ip http server ssl on command in the CNA Command Reference Guide for details. Issue 3 May 2008 29 Security USTAT-SSL Module In order to serve web hits through a secure channel, you need to use the SSL-enabled version of the USTAT module, and you need to enable the SSL protocol on each module’s web server. While in configuration mode (config), enter USTAT configuration mode (config-ustat) and then enter the ip http server ssl on command, as shown by the following examples of command-line input (including CNA prompts): Avaya_ANS# config t Avaya_ANS(config)# module ustat 3 Avaya_ANS(config-ustat)# ip http server ssl on Avaya_ANS(config-ustat)# end Avaya_ANS(config)# Repeat this sequence for each USTAT-SSL module. Restarting multiple USTATs can be simplified by using a shorthand notation in which the commands are concatenated into a single one-line command: Avaya_ANS# config t mod ustat ISP-1 ip http server ssl on Avaya_ANS# config t mod ustat ISP-2 ip http server ssl on <...etc.> With SSL enabled, the USTAT-SSL module will accept either http:// or https:// requests. With SSL disabled, the USTAT-SSL module will accept only http:// requests. Renewing Certificates To renew SSL certificates on a module that is already configured, first export the existing certificates to a remote location with the ssl export command, then delete them from the module with the ssl delete command. Add your new certificates as described in SSL Certificate/Key Files, then restart the SSL server on the module, with the ip http server ssl on command. You do not have to stop the server if is already running. 30 CNA Monitoring Guide, Version 4.1 Access Lists Access Lists The CNA system’s CLI implementation allows you to create access lists and apply them selectively to regulate the type of traffic that is allowed or denied over an interface. This IP subnet security feature allows you to control subnet access at both the IP protocol level, and the TCP and UDP port levels. First, create the access list while in configuration (config) mode, using the ip access-list command. This will place the device into access-list configuration (config-acl) mode, which allows you to define the parameters of the list. Use the remark command to enter a line of descriptive text that will help you and others identify the intent of this list when you examine the configuration at some later date. Use the permit and deny commands any number of times to define the rules governing access to the interfaces to which the access list is applied. Each instance of a command defines a different rule, to be applied to a single port. You can define multiple access lists in the CNA configuration, but only one list in each direction (inbound and outbound) can be associated with an interface at any one time. Rule Order Order is important. If you specify a general rule first and then a more specific rule for the same port later, the general rule will override the more specific rule, which will be ignored. For example, if you deny access to a port for all hosts and then later permit access to that port for an individual host, the second, more specific rule will never be applied. The more general rule denying access to all hosts will prevail, because it was applied to the incoming packet first. The packet will never be tested against the second, more specific command. Conversely, if you enter a rule allowing access to a specific host first and then enter the rule denying access to all hosts, the host explicitly identified by the first rule will be allowed access to the interface. The second, more general rule will be applied to all packets not handled by the first rule. Wildcard Masks Because of the way the CNA system applies wildcard masks to IP addresses in an access-list, the following rules apply when defining permit and deny statements: ● You must zero out each bit in the IP address that is set to be ignored by the mask. ● Once a bit is set to be ignored (changed to 1) in the 32-bit binary representation of the mask, all bits to the right of that location must also be ones in the mask (and zeros in the address). Issue 3 May 2008 31 Security For example, if the address is 16.1.31.5 and you specify a mask of 0.0.255.255 with the intention of masking out the last two octets of the address, the first of the two rules requires that you specify the IP address and mask as: 16.1.0.0 0.0.255.255 If you specify a mask of 0.0.7.255, it gets more complicated. Here is the address, 16.1.31.5, in binary: 0001 0000 0000 0001 0001 1111 0000 0101 And here is the mask, 0.0.7.255: 0000 0000 0000 0000 0000 0111 1111 1111 Wherever there are ones in the mask, there must be zeros in the address: 0001 0000 0000 0000 0000 0001 0000 0000 0001 1000 0000 0111 0000 0000 1111 1111 The new binary representation translates to 16.1.24.0, so the address and mask must be specified in the access list rule as: 16.1.24.0 0.0.7.255 Additionally, because of the second rule, you cannot specify a mask such as 0.0.255.0. This defines a mask in which ones are followed by zeros: 0000 0000 0000 0000 1111 1111 0000 0000 Because the last (right-most) octet of the mask contains zeros after the third octet’s ones, it is in violation of the second rule. Nor would the mask 0.0.0.16 be allowed, since it also results in zeros to the right of a one: 0000 0000 0000 0000 0000 0000 0001 0000 In fact, the only valid mask values other than 255 are 127, 63, 31, 15, 7, 3 and 1, since they are the only values that result in an unbroken string of ones at the right-most end of the binary mask. Note: Note: The CNA wildcard mask rules differ from some router access-list rules, such as that defined by Cisco’s IOS, which do permit the intermingling of matching and ignored bits. 32 CNA Monitoring Guide, Version 4.1 Access Lists Default Access Rules The CNA system implements an implicitly defined access list on each interface, and it implicitly adds a final deny rule to the end of all access lists of any origin. This final rule is the equivalent of entering the following deny command in the configuration: deny ip 0.0.0.0 255.255.255.255 0.0.0.0 255.255.255.255 This denies any IP access to the interface through any port that has not been specifically allowed by a previous rule. This means that, when you apply an access list of your own to any interface, you must explicitly re-enable access to all of the ports required on that interface by the CNA system. Each of the modules that require an eth0 interface (all except the reporting module) requires its own unique set of permissions. Note: Note: The implicit deny command will not appear in the configuration. If you want a visual reminder that it is there, you can explicitly add the command to the end of your access lists. If you then later remove it, however, you should remember that it is still applied by the operating system by default—removing it from the configuration will not change that. Management Module If you apply an access list to inbound traffic over the management module’s eth0 interface, the implicit deny command that is automatically appended to the end of the list will prohibit access to any port not explicitly allowed by your access list. To re-enable the default permissions, add the following lines to your access list: remark "allows telnet access for any host" permit tcp 0.0.0.0 255.255.255.255 0.0.0.0 255.255.255.255 eq 23 remark "allows ssh access for any host" permit tcp 0.0.0.0 255.255.255.255 0.0.0.0 255.255.255.255 eq 22 remark "allows http access for any host" permit tcp 0.0.0.0 255.255.255.255 0.0.0.0 255.255.255.255 eq 80 remark "allows https (SSL) access for any host" permit tcp 0.0.0.0 255.255.255.255 0.0.0.0 255.255.255.255 eq 443 remark "allows TCP ACK packets to return from any host (for FTP, during restore)" permit tcp 0.0.0.0 255.255.255.255 0.0.0.0 255.255.255.255 gt 1023 established remark "allows CNA to be polled using SNMP from any host" permit udp 0.0.0.0 255.255.255.255 range 540 580 0.0.0.0 255.255.255.255 eq 161 remark "allows traffic from any NTP server" permit udp 0.0.0.0 255.255.255.255 0.0.0.0 255.255.255.255 eq 123 permit udp 0.0.0.0 255.255.255.255 eq 123 0.0.0.0 255.255.255.255 gt 1023 Issue 3 May 2008 33 Security remark "allows DNS queries to return from any server" permit tcp 0.0.0.0 255.255.255.255 eq 53 0.0.0.0 255.255.255.255 gt 1023 permit udp 0.0.0.0 255.255.255.255 eq 53 0.0.0.0 255.255.255.255 gt 1023 remark "allows icmp destination-unreachable messages to arrive, from any host" permit icmp 0.0.0.0 255.255.255.255 0.0.0.0 255.255.255.255 3 0 remark "allows icmp source-quench messages to arrive, from any host" permit icmp 0.0.0.0 255.255.255.255 0.0.0.0 255.255.255.255 4 0 remark "allows icmp time-exceeded messages to arrive, from any host" permit icmp 0.0.0.0 255.255.255.255 0.0.0.0 255.255.255.255 11 0 remark "allows icmp parameter-problem messages to arrive, from any host" permit icmp 0.0.0.0 255.255.255.255 0.0.0.0 255.255.255.255 12 0 remark "allows icmp echo messages to arrive, from any host pinging the interface" permit icmp 0.0.0.0 255.255.255.255 0.0.0.0 255.255.255.255 8 0 remark "allows icmp echo-reply messages to arrive, from any host pinged by CNA" permit icmp 0.0.0.0 255.255.255.255 0.0.0.0 255.255.255.255 0 0 remark "allows replies from any radius server 1645 (old) and 1812 (new) port" permit udp 0.0.0.0 255.255.255.255 eq 1645 0.0.0.0 255.255.255.255 gt 1023 permit udp 0.0.0.0 255.255.255.255 eq 1812 0.0.0.0 255.255.255.255 gt 1023 Engine Module If you apply an access list to inbound traffic over the engine module’s eth0 interface, the implicit deny command that is automatically appended to the end of the list will prohibit access to any port not explicitly allowed by your access list. To re-enable the default permissions, add the following lines to your access list: remark "allows TCP ACK packets to return from any host (required for BGP traffic)" permit tcp 0.0.0.0 255.255.255.255 0.0.0.0 255.255.255.255 gt 1023 established remark "allows traffic from an SNMP polling action to return" permit udp 0.0.0.0 255.255.255.255 eq 161 0.0.0.0 255.255.255.255 range 540 580 remark "allows an icmp destination-unreachable message to arrive, from any host" permit icmp 0.0.0.0 255.255.255.255 0.0.0.0 255.255.255.255 3 0 remark "allows an icmp source-quench message to arrive, from any host" permit icmp 0.0.0.0 255.255.255.255 0.0.0.0 255.255.255.255 4 0 remark "allows an icmp time-exceeded message to arrive, from any host" permit icmp 0.0.0.0 255.255.255.255 0.0.0.0 255.255.255.255 11 0 remark "allows an icmp parameter-problem message to arrive, from any host" permit icmp 0.0.0.0 255.255.255.255 0.0.0.0 255.255.255.255 12 0 34 CNA Monitoring Guide, Version 4.1 Access Lists remark "allows icmp echo messages to arrive, from any host pinging the interface" permit icmp 0.0.0.0 255.255.255.255 0.0.0.0 255.255.255.255 8 0 remark "allows icmp echo-reply messages to arrive, from any host pinged by CNA" permit icmp 0.0.0.0 255.255.255.255 0.0.0.0 255.255.255.255 0 0 USTAT Module If you apply an access list to inbound traffic over a USTAT or USTAT-SSL module’s eth0 interface, the implicit deny command that is automatically appended to the end of the list will prohibit access to any port not explicitly allowed by your access list. To re-enable the default permissions, add the following lines to your access list: remark "allows access from any host for a GRE Tunnel IP type 47 " permit 47 0.0.0.0 255.255.255.255 0.0.0.0 255.255.255.255 remark "allows TCP ACK packets to return, from any host (required by active probes)" permit tcp 0.0.0.0 255.255.255.255 0.0.0.0 255.255.255.255 gt 1023 established remark "allows http access for any host (required for user traffic tests)" permit tcp 0.0.0.0 255.255.255.255 0.0.0.0 255.255.255.255 eq 80 remark "allows https access for any host (required for USTAT-SSL user traffic tests" permit tcp 0.0.0.0 255.255.255.255 0.0.0.0 255.255.255.255 eq 443 remark "allows icmp destination-unreachable messages to arrive, from any host" permit icmp 0.0.0.0 255.255.255.255 0.0.0.0 255.255.255.255 3 0 remark "allows icmp source-quench messages to arrive, from any host" permit icmp 0.0.0.0 255.255.255.255 0.0.0.0 255.255.255.255 4 0 remark "allows icmp time-exceeded messages to arrive, from any host" permit icmp 0.0.0.0 255.255.255.255 0.0.0.0 255.255.255.255 11 0 remark "allows icmp parameter-problem messages to arrive, from any host" permit icmp 0.0.0.0 255.255.255.255 0.0.0.0 255.255.255.255 12 0 remark "allows icmp echo messages to arrive, from any host pinging the interface" permit icmp 0.0.0.0 255.255.255.255 0.0.0.0 255.255.255.255 8 0 remark "allows icmp echo-reply messages to arrive, from any host pinged by CNA" permit icmp 0.0.0.0 255.255.255.255 0.0.0.0 255.255.255.255 0 0 Example Configuration of an Access List A new access list is required whenever you change a default port assignment. If, for example, you change the web server port from its default of 80 (or 443 for SSL), you must enable the new port with an access list and apply that list to the management module’s eth0 interface: # assign new web server port ip http server port 7000 Issue 3 May 2008 35 Security # create new access list to permit access to the new port from any host ip access-list my_new_access_list permit tcp 0.0.0.0. 255.255.255.255 0.0.0.0 255.255.255.255 eq 7000 end # apply new access list to eth0 interface on management module interface FastEthernet 0/0 ip access-group my_new_access_list in end The problem with this example is the implicit deny command that the CNA system appends to all access lists by default. At the same time that your new access list has allowed use of port 7000, it has also denied use of all other ports on the interface. Whenever you add an access list to an interface, you should re-enable default permissions which permit traffic required by the CNA system. You should create your new access list as follows: #create new access list ip access-list my_new_access_list # copy desired permit/deny commands from the Management Module example, but substitute # the following for the permit command that enables http on port 80 permit tcp 0.0.0.0 255.255.255.255 0.0.0.0 255.255.255.255 eq 7000 end The permit command takes the IP address of the source of the incoming traffic as the first address argument and the address of the module interface—the packet destination—as the second address argument. Specifying these addresses as 0.0.0.0 255.255.255.255 means that any host, regardless of IP address, can send traffic to the designated port across any interface that is configured on the module. As an additional security measure, you could use specific IP addresses. You can add a different permit command specifying as the source argument the IP address of each host you want to allow into the CNA system, and specifying as the destination argument the address you assigned to the management module’s eth0 port. This will limit access to the web server to only those hosts for which there is a permit command, and they will only have access to a single specified address. If you had assigned address 1.2.3.4 to the management module’s fastethernet 0/0 interface (the eth0 interface), any of the permit commands in any of these access list examples could specify the actual address for the packet destination as in the following permit rule: permit tcp 0.0.0.0. 255.255.255.255 1.2.3.4 0.0.0.0 eq 7000 36 CNA Monitoring Guide, Version 4.1 Chapter 4: Server-Based Measurement Overview Server-based measurement is one of two methods that the product measures the performance of your network’s traffic flow. Since the objective of both methods (server-based measurement with the single pixel GIF is the other) is to gather as many measurements as possible for subnets of interest, you can use either of the two, or both simultaneously. An agent-based measurement is made through a TCP session, an ICMP ping, or execution of a traceroute type utility initiated by the CNA system at regular intervals. The CNA system measures the time required to complete a TCP handshake or for the ICMP echo request to be answered, or, in the case of a traceroute probe, measures the time it takes for the traceroute packet to return from the last hop it was able to reach. Single pixel GIF measurement is a passive test that is conducted as a consequence of a web client downloading a page on your web site. The principal advantages of the single pixel GIF are its passive nature and its broad coverage of the Internet. Web surfers—your website clients—initiate the test whenever they download a page that contains a test-related URL. Data is collected for the subnet in which the web surfer’s IP address is located. Over time, the CNA measurement database will grow to cover virtually the entire spectrum of subnets representing people who have an interest in your web site. The principal advantages of the active probing mechanism are that it is predictable and it is configured to meet your needs. You choose in advance which subnets will be measured, and CNA will send probes on a regular, predetermined schedule. You are guaranteed measurements for each designated subnet. In addition, you can begin collecting measurements as soon as your CNA configuration is in place, without having to first make changes to your web pages. If you are using an CNA system to manage a VPN exclusively, then agent-based measurements are your sole source of measurements. If you are using your CNA system to optimize traffic between your web site and end users somewhere in the Internet cloud, or between your local subnet users and external web sites, then you should have an understanding of both the server-based measurement and agent-based measurement processes. See Chapter 5: Configuring Agent-Based Measurement. Issue 3 May 2008 37 Server-Based Measurement Setting up server-based measurements You need to set up your USTAT modules as described in Chapter 3: Configuring a New CNA Server Device of the CNA Fundamentals Guide, but instead of (or in addition to) configuring your Web site as described in Chapter 5: Configuring Agent-Based Measurement use the active-measurement group and related CLI commands to define agent-based measurement, either singly or in groups. Alternately, you can obtain measurement target addresses automatically from the optional Endpoint Collector (EFC) module. See Chapter 2: Collecting Endpoint Addresses. Targets obtained from the EFC module will be added to active measurement groups with the efc enable command. With agent-based measurement, you specify target IP addresses, type of measurement, and a schedule for contacting them. If you specify targets as part of an active-measurement group, you can specify parameters once for all members in the group. You can temporarily disable measuring of members of a group without removing them from the CNA configuration. Agent-based measurement traffic utilizes the same subnet infrastructure—GRE tunnels between the CNA system and your edge routers, if they exist, or your policy based routing (PBR) rules—which you have configured for all other CNA traffic, as described in Chapter 2: Configuring Your Routers in the Adaptive Path Controller-Internet Guide. Types of Measurement Measurements can be configured as one of three types: ● tcp—the CNA system will initiate and immediately close a TCP session with the target; the HRTT that is recorded in the CNA measurement database will be based on the time it takes for the CNA system and the target host to complete the TCP session handshake ● icmp—the CNA system will send an ICMP ping to the target; the HRTT that is recorded in the database will be based on the time it takes for the ICMP response to be returned ● traceprobe—for addresses that you know to be unreachable by TCP or ICMP probes but which you still need to measure, the CNA system will execute a traceroute and calculate the HRTT based on the time it takes for the last reachable host to respond; the measurement will not be recorded in the database unless all measurements through each of your USTATs converge on the same host. For information on setting convergence, see traceprobe convergence-rule command in the CNA Command Reference Guide. 38 CNA Monitoring Guide, Version 4.1 Setting up server-based measurements Of these three types, the tcp measurement produces more data but is most likely to encounter access roadblocks. Because there are several distinct handshakes involved in opening and closing a TCP session and each handshake sends a new set of packets through the Internet, a single tcp measurement produces several measurements to the CNA database. Use tcp measurement when endpoints are reachable from your subnet and can respond to a TCP session request. The icmp type of measurement is more likely to get through firewalls but it only contributes a single measurement to the database. Use icmp measurement when you think the endpoint will be reachable but will not respond to a TCP session request. Also use ICMP measurements when the target has limitations on the number of open TCP sessions that can be supported. For example, some VPN devices restrict the number of TCP sessions that can be opened in a given time interval. If your desired measuring frequency is greater than this restriction, then you should use ICMP measurement. The traceprobe type is a method of last resort, for those cases where you cannot penetrate a firewall or access policy but still need to target the address in order to obtain measurements. In this case, the measurements will terminate before reaching the target, and the end-to-end delay measurements may be less complete than those obtained by the other two measurement types. This is done over UDP (at a port starting at 33434) toward the end target specified in the configuration, and measures the ICMP response from the last reachable node. Note: Note: This technique has the potential to set off intrusion detection systems - you can use commands like traceprobe retries-per-hop, traceprobe max-failing-hops, and traceprobe hops to reduce the number of packets sent. The traceprobe measurement system has been designed to maximize measurement accuracy, minimize impact on system resources, and try to minimize ineffective measurement 'overspray'. Traceprobe starts by attempting to PING (ICMP ECHO) the actual endpoint target. If the target responds to the ICMP request, then traceprobe will continue to use ICMP to measure that target. If the endpoint target does NOT respond to the ICMP ECHO request, then a traceroute sequence is initiated in attempt to determine a responsive convergence point. Once a convergence point is identified, then measurements to that convergence point are made via ICMP ECHO requests. Note that if a convergence point stops responding on all links simultaneously, CNA will not interpret that as necessarily a connectivity problem to the actual endpoint. (if an intermediate router reboots, dynamic routing will try to find another path.) Rather, CNA will attempt to identify a new convergence point for the target. If no convergence point can be identified, measurements will cease, and total silence will eventually cause any performance routes to be withdrawn. By default, this process will NOT automatically generate route withdrawal or an outage signal. Traceprobe default parameters should be appropriate for most production environments. If necessary, traceprobe behavior can be customized with the following commands: threshold host-disappeared threshold proxy-disappeared Issue 3 May 2008 39 Server-Based Measurement threshold host-link-fail threshold proxy-link-fail refresh-time proxy-disappeared refresh-time host-disappeared refresh-time reacquire refresh-time acquire refresh-time verify For VPN deployments where the end-points are known to be reachable, such as when a central office is probing a branch office, use either TCP or ICMP probes as appropriate for your subnet policies. For unusual VPN deployments where the end-points are not reachable, use the traceprobe type. For deployments where you have roving VPN users whose addresses are collected through the EFC module and you expect to be able to reach them via the telnet protocol or the ping utility once their tunnels are up, use either tcp or icmp probes. If you are using the EFC module to obtain addresses of external web surfers and you expect a large percentage of them to be behind firewalls, use the traceprobe type. If you are optimizing traffic from your local users to external web sites and have manually entered addresses of known web sites, or if you obtain packets through an EFC filter set to listen for traffic going to a remote port 80 destination, use tcp or icmp probes. Active-measurement groups can now include the optional tos byte <> command. This command allows the ToS byte on outbound measurement traffic to carry the specified ToS value. An example configuration: active-measurement group tos-marked-measurements type icmp rate 1 per-second efc enable tos byte 16 efc bytes top 5 end In this example, all outbound measurements for this active-measurement group will include with a ToS byte value of (16). It is important to understand that the configuration for ToS byte settings encompasses the entire ToS byte. This byte has been defined in a number of different ways over time, with different definitions of the fields within this byte. RFC 791 and RFC 795, back in September 1981, defined this byte as follows: Type of Service: 8 bits 0 1 2 3 4 5 6 7 +-----+-----+-----+-----+-----+-----+-----+-----+ | PRECEDENCE | D | T | R | 0 | 0 | +-----+-----+-----+-----+-----+-----+-----+-----+ Bits 0-2: Precedence. 40 CNA Monitoring Guide, Version 4.1 Setting up server-based measurements Bit 3: Bits 4: Bits 5: Bit 6-7: 0 = Normal Delay, 1 = Low Delay. 0 = Normal Throughput, 1 = High Throughput. 0 = Normal Reliability, 1 = High Reliability. Reserved for Future Use. Precedence 111 110 101 100 011 010 001 000 - Network Control Internetwork Control CRITIC/ECP Flash Override Flash Immediate Priority Routine The three bits at positions (3), (4), and (5) were at this stage collectively known as “ToS bits”. RFC 1349 (July 1992) proposed an alternate interpretation, which involved expanding the “Type of Service” field from (3) single bit fields, to a single, (4) bit ToS field; but left the “Precedence” bits intact: 0 1 2 3 4 5 6 7 +-----+-----+-----+-----+-----+-----+-----+-----+ | PRECEDENCE | TOS | MBZ | +-----+-----+-----+-----+-----+-----+-----+-----+ 1000 -minimize delay 0100 -maximize throughput 0010 -maximize reliability 0001 -minimize monetary cost 0000 -normal service RFC 2474 (December 1998) redefines the entire ToS byte into two fields: a (6) bit “DiffServe Code Point” (DSCP), and two bits “unused”: 0 1 2 3 4 5 6 7 +-----+-----+-----+-----+-----+-----+-----+-----+ | DiffServe Code Point (DSCP) | Unused | +-----+-----+-----+-----+-----+-----+-----+-----+ DSCP: differentiated services codepoint CU: currently unused When configuring the CNA ToS parameter, it is important to remember that in this context, the entire ToS byte is included (values from 0 to 255). 0 1 2 3 4 5 6 7 +-----+-----+-----+-----+-----+-----+-----+-----+ | 128 | 64 | 32 | 16 | 8 | 4 | 2 | 1 | +-----+-----+-----+-----+-----+-----+-----+-----+ Issue 3 May 2008 41 Server-Based Measurement For example, the following access list definition: ip access-list netflow-tos-filter permit tcp 0.0.0.0 255.255.255.255 0.0.0.0 255.255.255.255 tos 16 end translates to 'only bit [3] is set', which corresponds with ToS “Minimize Delay” setting. Contact Data All traceprobe packets include a contact string, which you can set to include any information you choose, using the contact command. Measurement Frequency For each USTAT module, the CNA system can effectively send probes to and receive data back from as many as 250 ICMP targets per second and 225 tcp/traceprobes per second. If multiple links are being probed by a single USTAT module, then keep in mind that its capacity will be shared over those links. However, you may want to probe some targets more often than others. While there is no precise formula for calculating how you should distribute your active probes, in general your aggregate measurement traffic should correlate to the actual traffic in a meaningful way. In other words, you should probe high-traffic prefixes at a higher frequency than low-traffic prefixes. If you want to disable all active probing, then set active-measurement max-probes-per-second to zero. This parameter is really a maximum average rate, and sets a limit on the probing rate only if the running configuration calls for a rate that exceeds max-probes-per-second. If it does, then each target’s probing period will be extended on a pro-rated basis to reduce the total probing rate to max-probes-per-second. It is possible that the actual probing rate may exceed max-probes-per-second during momentary spikes. The default value is 200. Be aware, however, that the number of measurements the engine module can handle per second is finite; measurements in excess of that number will be dropped. See Measurements in Chapter 3: Adaptive Path Control of the Adaptive Path Controller Guide for more information about the system’s capacity to measure traffic—including both active probes and passive user traffic tests. When a probe is first enabled, the CNA system will pick the exact starting time at random during the time interval between the moment you enable the probe and the end of the first interval. That is, if you enable probes for ten different targets exactly at noon and specify an interval of once every hour, each of the ten probes will be sent out at random times during the hour between noon and 1 p.m. This is done to spread out the probe times among all of your targets in order to avoid bursts in probing activity. 42 CNA Monitoring Guide, Version 4.1 Setting up server-based measurements When using the EFC module as a source for active measurement targets (see Configuring the EFC Module in Chapter 2: Collecting Endpoint Addresses), it is important to note that the measurement frequency is shared by all the targets generated by the EFC. For example, if you have configured an active-measurement group to probe the top 250 web servers (as determined by the EFC), and you wish to probe each web server once a second, then you should set the probing rate to be 250 probes per second. Responding to Measurement Failures Use the probe-backoff command to determine how aggressively the CNA system should send active probes of type TCP or ICMP to a target in the face of successive probe failures. Note: Note: The active-measurement avoid command will reduce but not eliminate probe contacts. A traceprobe is intended to locate a suitable target in place of addresses that you know in advance cannot be reached. Such proxy addresses are identified by sending a traceroute packet to the unreachable address. The last host reached before the traceroute packet fails is the host that will be probed by the CNA probing algorithm for a specified period of time; then another traceroute packet will be sent to this host, but with the TTL now set to expire upon reaching it. The resulting ICMP time exceeded message will confirm to CNA that the target is still valid. This second phase will be repeated periodically. You can prevent contact with a specified address during the CNA probing portion of the cycle by using the active-measurement avoid command, but there is no way to configure the traceroute process to avoid an intermediate hop. The trigger value set by the probe-backoff command is not the same as the timeout value that is set when the probe is configured. For each probe, CNA will wait for a response for the amount of time specified by the probe’s timeout value (see the target command) before considering that individual probe to have failed. Backoff data is maintained by the CNA am-scheduler process for each probe target’s resolved IP address (one target may resolve to multiple addresses) for each USTAT. The next time a probe is scheduled for the target, probing activity will take place normally, including the address that did not respond previously. This cycle will continue for the length of time specified by the trigger argument of the probe-backoff command. If successive probes continue to fail over the same link for the amount of time specified by the trigger argument, probing of the target over that link will be placed in a wait state for the amount of time specified by the wait argument of the probe-backoff command. An outage event will be reported to the CNA decision-maker process whenever a probe attempt times out, except that these reports will be suppressed if no measurements have been received from a successful probe of the subnet during the past hour. This suppression prevents the am-measurer process from inundating the decision-maker process with reports for a subnet that may not exist (an address may have been misconfigured, for example) or is no longer in use. Issue 3 May 2008 43 Server-Based Measurement Active Measurement VIPs In order to receive randomly generated traffic from unknown users anywhere in the Internet (see Chapter 5: Configuring Agent-Based Measurement), your USTAT VIPs must have IP addresses that are generally unrestricted. Active probes, on the other hand, require that the USTATs be able to receive TCP or ICMP reply packets. If you don’t want to or can’t change your subnet policies to allow the USTAT VIPs to receive the TCP or ICMP reply packets required by an active probe, you can create a special purpose VIP for each USTAT. If, for example, you load balance by direct server return, your subnet may drop traffic other than SYN packets; or, you may have configured your firewall to accept only traffic that has been initiated on the outside. Firewalls are sometimes configured to accept only TCP port 80/SYN packets. The active measurement VIP allows you to configure your subnet to allow these active-measurement VIPs to be used for outgoing, locally initiated traffic only. See the vip command in the CNA Command Reference Guide. The CNA system will always use the active measurement VIP, if configured, for active probes. If there is no active-measurement VIP configured, the CNA system will use the standard VIP. Using Link-Groups You can restrict active probes to specified links by associating probes with a link-group. This allows you to probe your remote site at one rate over a private link and a different rate over other links that traverse the Internet, for example. Use the link-group command to define a link-group. Use the use link-group command to associate a link-group with an active measurement group. Use the active-measurement use-link-group command to associate a link-group with probes that are not defined within an active measurement group. Misconfigurations, such as incorrectly spelled or undefined link-groups, will be ignored as much as possible by the active-measurement process, and valid probes will be sent even though errors exist in the link-group configurations. However, such inaccuracies will be logged and displayed in the show statistics am-scheduler command output and you can determine where probes are and are not being sent by reading the show active-measurement command output. 44 CNA Monitoring Guide, Version 4.1 Named VIPs Named VIPs If you create active measurement VIP addresses with the vip command, you can selectively change the IP addresses used to send probes according to different active-measurement groups. For example, if you have two USTATs and want to send probes to one set of targets through one set of VIP addresses and to a different set of targets through a separate set of VIP addresses, you could configure named VIPs as follows: module ustat A vip name one vip name two end module ustat B vip name one vip name two end 192.168.10.1 192.168.20.1 192.168.10.2 192.168.20.2 Assuming you already have two active measurement groups configured, designate different named VIPs for each as follows: module engine active-measurement group one use named-vip one end active-measurement group two use named-vip two end end . . . Probes configured in group one will be sent from addresses 192.168.10.1 and 192.168.10.2, and probes configured in group two will be sent from addresses 192.168.20.1 and 192.168.20.2. If you configure an active measurement group to use a named vip, but a particular ustat does not have a vip of the specified name, then that group's targets will not probe over the USTAT with the missing name. Issue 3 May 2008 45 Server-Based Measurement Using Agent-Based Measurement with Performance Groups Agent-Based Measurement can guarantee that there will be sufficient measurements in the CNA database for specific prefixes of interest to you, such as: ● endpoints of your IPsec VPN—see Chapter 4: VPN Integration in the Adaptive Path Controller-Internet Guide. ● your customer/client locations—target individual interfaces, or, optionally, use one probe to gather data for multiple prefixes with a performance group Since performance groups (see the performance-group command) aggregate measurements for all prefixes in the group, data gathered by your probe target will be applied to all group-member prefixes. To use a single probe to gather data for multiple prefixes, include the target of the active probe as one of the prefixes in the group. All prefixes in the group should be located close to each other in the subnet topology, so that routes between your subnet and any one of the group will approximate routes to any of the prefixes in the group. This can be useful if you want to be sure your CNA measurement database contains data for a subnet that contains a destination important to you but that you don’t want to target directly. If the subnet is geographically close to another subnet in which you have already targeted a probe, the data you gather for one can be applied to both by means of a performance group. Troubleshooting Agent-Based Measurement Agent-based measurement can fail for a number of reasons, some of them beyond the control of the CNA system. You should be sure your targets can be reached through your edge routers. Check to be sure that all required static routes are defined on the routers, tunnels between the USTATs and edge routers are configured properly, and your PBR rules on the edge routers permit traffic out of the USTATs and back in to the appropriate VIP addresses. You can use the CNA CLI commands traceroute and tcpdump to discover where your probes are going. The CNA implementation of the ping command can tell you if your targets can be reached. Use the USTAT module implementation of these commands when debugging measurement problems. In addition, two show commands, show active-measurements and show statistics, provide detailed information about your active probe configurations. 46 CNA Monitoring Guide, Version 4.1 Troubleshooting Agent-Based Measurement A series of closely spaced back-to-back DNS lookup failures can cause the CNA am-scheduler process to fail to carry out its primary mission of sending out probes. If this happens, the CNA log will contain a series of info-level messages. Use the command debug am-scheduler to see them. You will also need to track down and fix the reason your DNS server is slow to respond, or else the condition will reoccur. If you are certain you are not sending too many DNS-dependent probes too fast, you should verify that the host names in your CNA configuration are valid and that your DNS server is properly configured. You may have to restart your DNS server if it is running an early version of BIND that is prone to memory leaks. Output of the show statistics am-scheduler command includes a table of DNS average lookup times plus an estimate of how many DNS lookups per second will be required to accommodate your configured probing rates. show active-measurements The show active-measurements command displays a table of active probes that meet your specified filter criteria. Columns in the table are: ● P—Current state of probing ● S—Failure suppression ● Group—Active measurement group name (n/a means the probe was configured as a standalone probe, using the target command rather than the active-measurement group commands) ● Target—IP address of the probe target, as configured, or its hostname. Addresses marked “(EFC)” indicates that these are derived from Endpoint Flow Collection. ● Address—DNS-resolved address of the probe ● Type—Configured probe type (either tcp or icmp) ● USTAT—Slot number or subsystem and name of USTAT to which this row of the table applies, in the form <name> <slot> (USTATs which are still configured to use slot numbers instead of names will repeat the slot number as the name entry; if a USTAT module is defined in the configuration but is not associated with a slot or subsystem, the slot entry will be shown in the output as <unk>) ● Success—Number of probes that were successful ● Errors—Summary of what went wrong (error codes are defined in the key at the top of the table) Output is displayed as follows: ssf-neat-5008# show active-measurements range-address 4.2.0.0/16 Thu Dec 11 14:04:59 PST 2003 Summary Rows Total Timing Out Net Unrch Host Unrch Wait Recovery Issue 3 May 2008 47 Server-Based Measurement Shown Found 21 21 12 12 0 0 0 0 0 0 0 0 Measurement Results 'unk' means no mapped slot number Probing: normal = ., timing out = t, net unreachable = n, host unreachable = h, wait = w, recovery = r Failure Suppression: no failures = ., recent failure = f, suppressed = s, unmeasured = u Errors: DNS = d, Queue Drop = q, Probe Timeout = t, Network Unreachable = n, Host Unreachable = h, Port Unreachable = p, Other = o PS ts .. .. .. tu tu .. Group top-n-e top-n-e top-n-e top-n-e top-n-e top-n-e top-n-e Target (EFC) (EFC) (EFC) (EFC) (EFC) (EFC) (EFC) Address 4.2.143.75 4.2.143.75 4.2.143.75 4.2.163.162 4.2.167.130 4.2.167.130 4.2.170.194 Type icmp icmp icmp icmp icmp icmp icmp Ustat (Slot) 4 (4) 5 (5) 6 (6) 6 (6) 4 (4) 5 (5) 6 (6) Success 0 203 198 106 0 0 220 Errors 203t 0 5t 105t 203t 203t 0 This display summarizes the results of two stand-alone active probes configured using the active-measurement target command, on an CNA system that has four USTATs (modules 4, 5 and 6). P and S columns show normal probing status for four rows and three rows are currently timing out; and outages are not being reported to the decision-maker process for first one (s indicates suppressed). Possible values for the P column are: ● . (period)—probing activity is normal ● n—the CNA system has received a subnet unreachable ICMP packet in response to a probe; probing will continue until the subnet has been unreachable for the length of time specified by the trigger argument of the probe-backoff command ● h—the CNA system has received a host unreachable ICMP packet in response to a probe; probing will continue until the host has been unreachable for the length of time specified by the trigger argument of the probe-backoff command ● t—probing continues, although no response has been received from a probe for the amount of time specified by the timeout argument in the probe configuration; probing will continue until the timeout period specified by the trigger argument of the probe-backoff command ● w—wait-state triggered by probe-backoff; no probes will be sent for the period of time specified by the wait argument of the probe-backoff command ● r—recovery, which means that the system is now attempting to find a convergence point because of probe failures to a target. 48 CNA Monitoring Guide, Version 4.1 Troubleshooting Agent-Based Measurement Possible values for the O column are: ● . (period)—outages are being reported to the decision-maker process ● f—probe failure ● u—unmeasured; the target has never been successfully probed and no probe failures will be reported to the decision-maker process. ● s—the CNA system has not received a measurement from this subnet for at least the last hour, so the outage report is being suppressed rather than inundating the decision-maker process with reports for possibly non-existent prefixes Target and Address values are the same for each, since there was no DNS resolution of targets (as there would have been had the targets been specified by host name). Over time, these values can give you insight about how hosts are configured in DNS. Each time the target resolves to a different address, there will be a new entry in the table. By comparing measurement counts, you can discover how often the host name resolves to a specific IP address. The Module column lists USTATs by name and slot number. On an IBM xSeries platform running CNA, there is no relevant slot or subsystem information, regardless of the number of USTAT modules configured. The first table compares the number of lines that match the search criteria specified when you launched the command to the number that are actually in the current output. For example, if you execute the show active-measurements command with no search parameters, the Found row will report the total number of probe reports currently in memory. If used with the traceprobe parameter, it will show the last reachable hosts for a given target. If the last reachable hosts are not the same for all ustats, then the measurements are not reported to the database. If you are not seeing measurements for a traceprobe target, you can use this command to help explain this. However, the command output will never exceed 5,000 lines, so the Shown line will report that number of probes. If the number of probes found exceeds the number of lines shown, you should refine your search query in order to see the additional probe reports. The first two lines in the second table show the results of the first configured probe, which targets the address 4.2.143.75 (one line for each of the four USTATs). In 203 attempts (total of success and errors), the probe succeeded on two of the USTATs and failed on the other one. The error codes indicate that all probes timed out on the first USTAT and five on the second. Use the command clear active-measurements to clear the data tables and counters displayed by the show active-measurements command. This command also resets the Probing status to '.', the Outage Reporting status to 's', clears the decision-maker process’s knowledge of the data contained in the display, and, for traceprobe targets, forces a traceroute. Issue 3 May 2008 49 Server-Based Measurement show statistics More detailed information is available from the show statistics command executed with either the am-measurer or am-scheduler process name as the argument. You can clear the tables displayed by this command at any time so that you can start at zero when tracking new probes, with the clear statistics command. Begin with the am-scheduler output, then once you’ve identified a trouble spot by executing show statistics am-scheduler, you can focus on the specific USTAT with the am-measurer data show statistics am-scheduler If you execute the command as show statistics am-scheduler, the output will display information in several distinct blocks. The first block of display data gives you a high level summary of the active measurement system. The remaining sections show various active measurement statistics, such as probe rates attempted and completed, failures, DNS behavior. ssf-neat-5008# show stat am-scheduler Thu Dec 11 13:58:31 PST 2003 Current State: RUNNING Start Time : 1071113800423 ms, 2003/12/10 19:36:40.423, Reconfig Time: 1071176828121 ms, 2003/12/11 13:07:08.121, Clear Time : 1071113800443 ms, 2003/12/10 19:36:40.443, Last EFC Resp: 1071179911191 ms, 2003/12/11 13:58:31.191, Debug Mode : off Configuration Summary Total Number of Targets : 84 Number of TCP Targets : 34 Number of ICMP Targets : 50 Number of Traceprobe Targets : 0 Number of Unknown Targets : 0 Configured Total Probes Per Second : 510.766667 Configured Max Probes Per Second : 1000.0 Multiplying rates by scale factor : 1.0 Expected DNS Lookups Per Second : 0.0 Configuration Errors Group Target Type Error Normal Events (per minute) name total-count 1-min 5-min 60-min Tot Req Att 32030931 30214 30177 29246 Tot Req Sent 32030931 30214 30177 29246 TCP Req Att 74471 68 67 68 TCP Req Sent 74471 68 67 68 ICMP Req Att 31956460 30146 30109 29178 ICMP Req Sent 31956460 30146 30109 29178 Trace Req Att 0 0 0 0 Use Trace Cache 0 0 0 0 Trace Req Sent 0 0 0 0 Results Rcvd 32019819 30155 30179 29234 Result Recorded 32019826 30155 30179 29234 50 CNA Monitoring Guide, Version 4.1 18h21m50s940ms ago 51m23s242ms ago 18h21m50s920ms ago 172ms ago peak 31681 31681 78 78 31609 31609 0 0 0 31218 31218 peak age 0d00:06:41 0d00:06:41 0d05:23:12 0d05:23:12 0d00:06:41 0d00:06:41 0d18:21:50 0d18:21:50 0d18:21:50 0d00:06:52 0d00:06:52 Troubleshooting Agent-Based Measurement Trace Meas Sent 0 0 0 0 0 0d18:21:50 Trace No Conv 0 0 0 0 0 0d18:21:50 Bifurcated Prov 0 0 0 0 0 0d18:21:50 EFC Successes 128106 118 117 117 123 0d00:07:27 Fail Rpts Rcvd 17339161 16313 16313 15284 18798 0d14:22:04 Fail Rpts Sent 397196 446 447 437 989 0d14:22:10 Fail Rpts Supp 16941965 15867 15866 14847 17974 0d14:35:28 Error Events (per minute) name total-count 1-min 5-min 60-min peak peak age EFC Failures 0 0 0 0 0 0d18:21:50 EFC Timeouts 7 0 0 0 5 0d05:48:15 DNS No Host 0 0 0 0 0 0d18:21:50 DNS Timeouts 0 0 0 0 0 0d18:21:50 DNS Error 0 0 0 0 0 0d18:21:50 Unk Resolver 0 0 0 0 0 0d18:21:50 Req Connect Err 0 0 0 0 0 0d18:21:50 Runt Req Err 0 0 0 0 0 0d18:21:50 Req Send Err 0 0 0 0 0 0d18:21:50 Result Rcv Err 0 0 0 0 0 0d18:21:50 Times (in microseconds) name 1-min 5-min 60-min peak peak age DNS Time 0 0 0 0 0d18:21:50 EFC Time 2907 4113 4114 27408 0d05:47:40 Other name 1-min 5-min 60-min peak peak age Trace Cache% 0 0 0 0 0d18:21:50 Disabled Groups Group Group Group rs-web-clients Duplicate Targets Group Target Type Rate Port netscreen-targets 67.21.230.64 icmp 12/m n/a rw-home 67.21.230.64 icmp 4/m n/a Configured Target Diagnostics group target type address top-n-efc (EFC) icmp 12.2.41.18 VIP Definitions Vip Ustat (Address) (default) 4 (64.254.162.201), 5 (64.254.162.202), 6 (64.254.162.203) ● Start time is the last time the process was started, ● reconfig time is the last time it was reconfigured, and ● clear time shows the last time the clear statistics command was executed for the am-scheduler process. ● Last EFC Resp shows how long, in microseconds, it has been since the active measurement system has heard from the efc process. ● Configured Total Probes Per Second in the show command output will be the value entered into the configuration, not the scaled-back actual number of probes. ● Configured Max Probes Per Second value is the value which has been set in your configuration by the active-measurement max-probes-per-second command. Issue 3 May 2008 51 Server-Based Measurement ● Multiplying rates by scale factor shows. ● Expected DNS Lookups Per Second value is a calculated estimate of how many DNS lookups are required per second to support your configured active probing rate. The next block “Normal Events (per minute)” shows rolling averages of per-minute rates of probes sent. The 1-min column shows average probes per minute during the previous 60 seconds; the 5-min column shows average per-minute rate during the previous five minutes, and the 60-min column shows average per-minute rate during the previous hour. The rows show totals for the various types of probes attempted and sent (e.g., the number of TCP and ICMP probes attempted and sent). ● Req Att values (probe requests attempted) are incremented when the scheduled time for the probe arrives ● Req Snt values (probe requests sent) are incremented when the am-scheduler process hands the probe off to the am-measurer process. Normally, these values should be identical to the Rq Att values for a particular category. ● TCP Att and TCP Snt are the number of TCP probes attempted and sent respectively. ● ICMP Att and ICMP Snt are the number of ICMP probes attempted and sent respectively. ● Trace Att and Trace Snt are the number of traceprobes attempted and sent respectively. ● Trc Cache is the number of traceprobe results pulled from cached values rather than measured from the subnet. ● Trace Meas is the number of traceprobe measurements am-scheduler sends to the decision maker process. ● Trace No Conv is the number of traceprobes that failed to converge. ● Bifurcated Prov is the number of bifurcated provider subnets that the systems has detected. A bifurcated provider is one whose subnet has portions that are internally isolated from each other within their subnet, but still reachable from different external routes. The requests sent values should be matched by requests received values in the show statistics am-measurer output. Possible causes for a mismatch between sent and received values are: DNS lookup of target’s host name failed, and an internal error. The average DNS lookup times should be short enough to handle the number of lookups expected. ● EFC Successes (internal statistic) ● Fail Rpts Rcvd shows the number of probe failures seen by the USTAT. ● Fail Rpts Sent shows the number of probe failures reported to the decision-maker process by the USTAT. ● Fail Rpts Supp shows the number of suppressed probe failures. This occurs when failures are detected across all links for a particular probe. 52 CNA Monitoring Guide, Version 4.1 Troubleshooting Agent-Based Measurement The block titled “Error Events (per minute)” has the following: ● EFC Failures (internal statistic) ● EFC Timeouts (internal statistic) ● DNS No Host indicates that the DNS server was not able to resolve a host name. ● DNS Timeouts shows the number of DNS server timeouts. ● DNS Error indicates some other DNS-related error different from the previous No Host and Timeouts errors (e.g., bind is not running). ● Unk Resolver (internal statistic) ● Req Connect Err indicates an internal communications error between the am-scheduler and am-measurer processes. This error can occur if the measurer process is not yet running when the process sends it a packet, such as during a reboot, or after a Medic restart of the am-scheduler process, or after a configuration change has triggered a restart of the am-measurer process. ● Runt Req Err this is packet that is below the minimum length for its type, a condition that should be extremely rare, and can be caused by a hardware problem. ● Req Send Err indicates a problem with communication between the am-scheduler and am-measurer processes (see Req Connect Err). ● Result Rcv Err indicates a problem with communication between the am-scheduler and am-measurer processes (see Req Connect Err). The block titled “Times (in microseconds)” shows the following: ● DNS Time shows the average time, in microseconds, used to resolve DNS names. ● EFC Time shows the average time, in microseconds, used to resolve DNS names. Miscellaneous items: ● Trc Cache% shows the percentage of traceprobe measurements reported out of cache rather than measured in the subnet. ● Disabled Groups show active measurement groups that have been disabled in the configuration (see the disable command) ● Duplicate Targets are those that have been duplicated in the configuration (same target definition in two separate groups, or one in a group and another not in a group). Two probes to the same IP address are not necessarily considered duplicate targets—DNS may have resolved a host name to the same IP address of another target. Addresses that differ only in syntax (1.2.3.4 and 001.002.003.004, for example) will not be flagged as duplicates. If more than one probe is configured with the same host name, the probes will be considered duplicates. This table is informational only. Probes will be sent as configured, even if duplicated. If the duplication is not desirable, you will need to correct it in the configuration yourself ● VIP Definitions section shows, for each VIP name, the IP address that each ustat has for that name. Issue 3 May 2008 53 Server-Based Measurement If you have configured a greater number of probes per second than allowed by the probe-backoff command, the CNA system will scale back the number of probes to conform with the limit. See Measurement Frequency earlier in this chapter for more information. See the active-measurement group command in the CNA Command Reference Guide for more information about configuring target parameters for probes that are defined in an active measurement group.) show statistics am-measurer If you execute show statistics with the am-measurer process as the argument, information similar to the am-scheduler output will be displayed, divided into individual groups per USTAT. In addition to rolling average probe rates, the display will contain some CNA-internal statistics, including time spent in internal queues, time spent polling, and thread pool size, as well as the times since the last restart, reconfiguration, or clearing of the am-measurer process. The display also contains a list of error counters per USTAT. The following display shows part of the output for a single USTAT; if you use the all keyword, the output will contain similar tables for each USTAT: ssf-neat-5008# show stat am-measurer Thu Dec 11 14:01:41 PST 2003 module 4: OK Restart Time : 1071018678368 ms, 2003/12/09 17:11:18.368, 1d20h50m22s624ms ago Clear Time : 1071018678389 ms, 2003/12/09 17:11:18.389, 1d20h50m22s603ms ago Debug Mode : off Normal Events (per minute) name total-count 1-min 5-min 60-min peak peak age Req Received 26147178 10058 10040 9661 29859 1d05:24:06 Valid Requests 26147178 10058 10040 9661 29859 1d05:24:06 TCP Requests 60801 10 22 22 45 0d05:26:55 ICMP Requests 26086377 10048 10018 9638 29850 1d05:24:08 TracePrb Req 0 0 0 0 0 1d20:50:22 Attempts 26146867 10060 10035 9660 29971 1d05:24:12 TCP Probes 51245 8 18 18 37 1d05:25:16 ICMP Probes 11354039 4375 4367 4368 13546 1d04:23:50 TracePngs 0 0 0 0 0 1d20:50:22 TracePng In Prg 0 0 0 0 0 1d20:50:22 TraceRts 0 0 0 0 0 1d20:50:22 TraceRt In Prg 0 0 0 0 0 1d20:50:22 Tot TracePrb 0 0 0 0 0 1d20:50:22 Error Events (per minute) name total-count 1-min 5-min 60-min peak peak age Interrupted Rcv 0 0 0 0 0 1d20:50:22 Bad Send Addr 0 0 0 0 0 1d20:50:22 Runt Req Packet 0 0 0 0 0 1d20:50:22 Bad Target Type 0 0 0 0 0 1d20:50:22 Job Queue Full 0 0 0 0 0 1d20:50:22 Queue Timeouts 0 0 0 0 0 1d20:50:22 Bad Result Type 0 0 0 0 0 1d20:50:22 Bad Result Code 0 0 0 0 0 1d20:50:22 TCP Timeout 5394 0 2 2 5 0d14:56:30 54 CNA Monitoring Guide, Version 4.1 Troubleshooting Agent-Based Measurement TCP Probe Err 4161 1 1 1 4 0d06:00:49 ICMP Timeout 14139853 5469 5436 5067 16282 1d05:24:42 ICMP Probe Err 592175 207 210 202 694 1d03:58:56 TracePrb T-Out 0 0 0 0 0 1d20:50:22 TracePrb Errors 0 0 0 0 0 1d20:50:22 RTT Range Err 0 0 0 0 0 1d20:50:22 Response Errors 0 0 0 0 0 1d20:50:22 Times (in microseconds) name 1-min 5-min 60-min peak peak age Queue Time 0 19 3 1402 0d21:38:11 Run Time 172 173 168 351 1d00:34:01 TCP Netwk Time 68765 57891 61821 481343 0d00:40:37 ICMP Netwk Time 166726 172476 173563 393642 1d04:52:54 TracePng Time 0 0 0 0 1d20:50:22 TraceRt Time 0 0 0 0 1d20:50:22 TracePrb Time 0 0 0 0 1d20:50:22 Other name 1-min 5-min 60-min peak peak age TCP Pool 0 0 0 4 1d02:13:05 ICMP Pool 285 285 266 852 1d05:24:44 TraceRt Pool 0 0 0 0 1d20:50:22 TracePng Pool 0 0 0 0 1d20:50:22 TCP Q Size 0 0 0 0 1d20:50:22 ICMP Q Size 0 0 0 0 1d20:50:22 TraceRt Q Size 0 0 0 0 1d20:50:22 TracePng Q Size 0 0 0 0 1d20:50:22 Error Counts Error Name Count Since Update Network is unreachable 6192 0d00:00:35 Connection timed out 14145247 0d00:00:01 Connection refused 17133 0d00:00:08 No route to host 230024 0d00:00:02 Traceroutes Underway From VIP To Target ssf-neat-5008# show active-measurements range-address 4.2.0.0/16 Thu Dec 11 14:04:59 PST 2003 Summary Rows Total Timing Out Net Unrch Host Unrch Wait Recovery Shown 21 12 0 0 0 0 Found 21 12 0 0 0 0 Measurement Results 'unk' means no mapped slot number Probing: normal = ., timing out = t, net unreachable = n, host unreachable = h, wait = w, recovery = r Failure Suppression: no failures = ., recent failure = f, suppressed = s, unmeasured = u Errors: DNS = d, Queue Drop = q, Probe Timeout = t, Network Unreachable = n, Host Unreachable = h, Port Unreachable = p, Other = o PS Group Target Address Type Ustat (Slot) Success Errors tu top-n-e (EFC) 4.2.143.75 icmp 4 (4) 0 203t tu top-n-e (EFC) 4.2.143.75 icmp 5 (5) 0 201t tu top-n-e (EFC) 4.2.143.75 icmp 6 (6) 0 204t,1o tu top-n-e (EFC) 4.2.143.76 icmp 4 (4) 0 202t tu top-n-e (EFC) 4.2.143.76 icmp 5 (5) 0 204t Issue 3 May 2008 55 Server-Based Measurement ● Restart Time values refer to the most recent restart of the am-measurer process. ● Clear Time values refer to the most recent execution of the clear active-measurements command. The “Normal Events (per minute)” block shows rolling averages of per-minute rates of probes sent. The 1-min column shows average probes per minute during the previous 60 seconds; the 5-min column shows average per-minute rate during the previous five minutes, and the 60-min column shows average per-minute rate during the previous hour. ● Req Rcvd and Valid Req values represent packet arrivals from the am-scheduler process. Under normal circumstances they should be identical, and equal to the sum of TCP and ICMP requests. Arriving packets are transferred to a job queue to await processing. ● The Attempts values refer to the number of packets removed from the queue and processed, regardless of outcome. ● The TCP Probes and ICMP Probes values refer to probes that were successfully sent and for which reply packets were received. Differences between TCP Req and TCP Probes, and between ICMP Req and ICMP Probes, are indicators that your active probes are not finding their targets, or targets are not replying, or the reply packets are not getting back to the CNA system. ● TracePngs is the number of trace-ping operations per minute. ● TracePng In Prg represents the number of trace-ping requests dropped because a traceroute to that target was already underway. ● Tracerts is number of traceroute operations per minute ● TraceRt In Prg is number of traceroute requests dropped because a traceroute to that target was already underway. ● Tot TracePrb is number of traceprobe measurements per minute, including both traceroute and trace-ping. The block that shows “Error Events (per minute)” has the following items: Interrupted Receives shows the number of times a probe request from the am-scheduler process was interrupted. Bad Send Addresses represents an internal addressing error. Runt Request Packets counts the number request packets received from the am-scheduler process that were shorter than the minimum size for the type of packet. This should be an extremely rare event, possibly caused by a hardware problem. Bad Target Type means the probe was not configured for either TCP or ICMP. This is a fail-safe counter. Since the CNA CLI will not allow you to enter a probe with any other type, this counter should never increment. 56 CNA Monitoring Guide, Version 4.1 Troubleshooting Agent-Based Measurement Job Queue Full and Queue Timeouts counters refer to an internal queue of probes received from the am-scheduler process and awaiting processing by the am-measurer process. High numbers in either of these counters means your probes are timing out—taking too long to complete, which causes the am-measurer process to cancel the request and move on to the next. Invalid Result Target Type and Bad Result Code represent internal errors. Under normal circumstances, you should not see values for these items. TCP Timeout shows the number of TCP-type probes that were sent but for which the targets did not respond. TCP Probe Err counts the number of TCP protocol errors encountered. ICMP Timeout shows the number of ICMP-type probes that were sent but for which the targets did not respond. ICMP Probe Err counts the number ICMP protocol errors encountered. TracePrb T-out shows the number of traceroute-type probes that were sent but for which the targets did not respond. TracePrb Err counts the number traceroute errors encountered. RTT Range Err counts the number of times a measure round trip time yielded incorrect results (e.g., a negative number, or greater than the timeout value). This could indicate a subnet time problem. The next table, “Times (in microseconds)”, in the show command output shows rolling averages of such things as time spent in the job queue and size of the job queue and thread pool: The information in this table is primarily of value when investigating CNA-related questions rather than active-probe configuration problems. The last section, “Traceroutes Underway”, shows which traceroutes the am-measurer process is currently running. Issue 3 May 2008 57 Server-Based Measurement 58 CNA Monitoring Guide, Version 4.1 Chapter 5: Configuring Agent-Based Measurement Chapter contents This chapter contains the following sections: ● About agent-based measurement ● Configuration overview ● Hives ● Zones ● Agent-based measurement tests ● Agent-based measurement alarms ● Alarm templates ● Merging and splitting nodes ● Viewing the agent-based measurement configuration About agent-based measurement Use CNA’s agent-based measurement to monitor and test subnet performance between CNA test agents. Test agents are embedded in Avaya Media and Security Gateways, IP Phones, switches from Extreme Networks, and in CNA servers. You can also install small, stand-alone devices, called CNA Agent Devices, in your subnet to serve as CNA test agents. For more information on CNA agents, see About CNA agents. The CNA server’s Chatter software module schedules tests between test agents, collects, and analyzes, and reports the results, and dynamically generates a map of the subnet topology. The map shows discovered nodes and links that are used by test packets. For more information on Chatter modules, see About Chatter modules. The CNA servers and CNA agents in your subnet make up an agent-based measurement subnet, called a hive. You can organize CNA agents into a hierarchy of zones (and sub-zones) to control the distribution of tests and analysis of the results. You can arrange the hierarchy of zones to help track long-distance link performance, or functionally to track communications between regions or departments, or both. Tests are performed between every pair of zones, and between every pair of sub-zones within a zone. For more information on hives and zones, see Hives and Zones. Issue 3 May 2008 59 Configuring Agent-Based Measurement Alarm notifications can be sent to e-mail addresses, system logs, or SNMP traps for external alarm monitoring systems. For information on alarm notifications, see Agent-based measurement alarms. About CNA agents CNA agents are subnet devices running software that performs synthetic subnet tests in response to requests from a CNA server’s Chatter module. CNA agents send test packets to perform these tests and then forward the test results to a Chatter module for analysis and reporting. Each CNA agent can run tests with another CNA agent or with a specified IP address. Two types of devices can serve as CNA agents: ● Avaya products and Avaya-partner products such as Security Gateways (SG), Media Gateways, Extreme switches, and IP phones that have embedded CNA agent software. ● CNA Agent Devices, which are small, stand-alone devices that run CNA agent software. You can buy these devices from Avaya and install them in your subnet to provide agent-based measurement data. The CNA server itself has embedded CNA agent software. Figure 1 shows a basic CNA implementation with one CNA server and multiple CNA agents. Figure 1: CNA agents in CNA system 60 CNA Monitoring Guide, Version 4.1 About agent-based measurement CNA agents in Avaya security gateways CNA agents in security gateways provide the unique ability to measure subnet performance inside Virtual Private Network (VPN) tunnels between secure gateways. Systems without this capability can measure only the overall performance between the tunnel endpoints. With this capability, performance can be measured over every link within the tunnel. CNA agents in Avaya media gateways CNA agents in media gateways provide the ability to measure end-to-end service to the edge of the Public Service Telephone Network (PSTN), or at points where code changes are required for interworking between high-speed (LAN) and low-speed (WAN) links. About Chatter modules The CNA server’s Chatter software module performs the following functions: ● Schedules tests between CNA agents. ● Performs alarm notification. ● Analyzes, reports, and stores test results. ● Shares the following information with other CNA Chatter modules in your subnet: - Configuration information for agent-based measurement: ● Security keys ● CNA agents ● Authorized users ● Chatter module configurations. Any configuration change that you make to a Chatter module is immediately communicated to other Chatter modules in the hive. - Test results. Each Chatter module shares its test results with other Chatter modules in your subnet to generate a unified report of the subnet status. Thus, you see the same agent-based measurement data regardless of the CNA server that you log in to. ● Provides management capabilities for the agent-based measurement system, also called a hive. For more information on hives, see Hives. Issue 3 May 2008 61 Configuring Agent-Based Measurement Configuration overview Table 4 provides an overview of the configuration process for agent-based measurement. You can use it as a reference to guide you through the process. Table 4: Overview: Configuring agent-based measurement Step Task See Section 1 Add each CNA server to the hive. Assigning Chatter modules to a hive 2 Configure endpoints to register with the hive. Registering CNA agents with the hive 3 Create the zone hierarchy. Defining a zone hierarchy 4 Assign CNA agents to zones. Assigning CNA agents to zones 5 Specify tests to run and configure test parameters. Enabling a test Configuring a test 6 Define alarm notification addresses. Defining alarm destinations 7 Configure alarm templates. Creating an alarm template Defining alarm destinations Defining test thresholds Assigning templates to zones or edges 62 CNA Monitoring Guide, Version 4.1 Hives Hives This section contains the following topics: ● About hives ● Assigning Chatter modules to a hive ● Viewing the master key for a hive ● Removing a Chatter module from a hive ● Defining down status for Chatter modules ● Synchronizing a Chatter module with its hive ● Registering CNA agents with the hive ● Removing a CNA agent from a hive About hives An agent-based measurement system consisting of one or more CNA agents and one or more CNA servers is called a hive. The CNA servers in the hive: ● Schedule tests between CNA agents. ● Perform agent-based measurement testing, analysis, and alarm notification. ● Analyze, report, and store test results. ● Share the following information for agent-based measurement: - Security keys - CNA agents - Authorized users - Chatter module configurations. Any configuration change that you make to a Chatter module is immediately communicated to other Chatter modules in the hive. ● Share test results. Each Chatter module shares its test results with other Chatter modules in your subnet to generate a unified report of the subnet status. Thus, you see the same agent-based measurement data regardless of the CNA server that you log in to. ● Provide management capabilities for the hive. ● Dynamically generate a map of the subnet topology by using traceroute tests. The map shows discovered nodes and links that are used by test packets. ● Beginning with Version 4.0, the hive can provide data to the supernode. For more information regarding the supernode, see the CNA Operator Guide. Issue 3 May 2008 63 Configuring Agent-Based Measurement More than one hive can monitor your network; however, CNA agents and CNA servers can belong to only one hive. Hives cannot share CNA agents and CNA servers. Fault tolerance for CNA servers If your agent-based measurement system (hive) contains more than one CNA server, the Chatter modules can assume the responsibilities of any other Chatter module if it were to fail. The hive continues to function when CNA failures or subnet problems occur. The remaining Chatter modules continue to run the tests administered (within system capacity, and using all agents that are still accessible). The test injection rate is scaled down proportionally to the number of Chatter modules lost from the hive. The ability of the system to continue running the tests that you set up before the failure is limited by the resources available on the remaining Chatter modules. If subnet connectivity is reestablished, or the CNA server problem resolved, the lost Chatter modules automatically rejoin the hive. When Chatter modules are lost from a hive, certain administrative user commands are not available (for example, rekeying the security keys, adding new agents, and so forth.). This restriction is to prevent potential problems when the lost Chatter modules rejoin the hive. CNA agent information is shared among all Chatter modules in the hive, so every Chatter module knows about all CNA agents in the system and therefore can use any agent in the system to perform tests. Note: Note: If you replace a CNA server with another CNA server, hive membership is not transferred to the new server when you load the configuration from the backup configuration. To assign the new CNA server to a hive, you must execute the chatter join-hive command. This command assigns the server to the appropriate hive. For more information on this procedure, see Assigning Chatter modules to a hive. Assigning Chatter modules to a hive To assign a CNA server’s Chatter module to a hive: 1. Access Privileged command mode on the CNA server that you want to add to the hive. To access this mode, enter the following command: enable. 64 CNA Monitoring Guide, Version 4.1 Hives 2. Enter the following command: chatter join-hive ip-address <ip> master-key <key> where <ip> is the IP address of a remote Chatter module that is currently participating in the hive. <key> is the master key of the hive. For information on how to obtain the master key of a hive, see Viewing the master key for a hive. Note: Note: You cannot add a Chatter module when one of the existing Chatter modules is unreachable or unresponsive, or when another Chatter module is still being added to or removed from the hive. 3. About an hour after adding a Chatter module to the hive, once the system has stabilized, access privileged command mode, and enter the following command: chatter assign This command clears and recalculates the associations of cells and edges to Chatter modules. Note: Note: If this command changes the association of a cell or edge from one Chatter module to another, previously trended data for the cell or edge is lost. New data for the cell or edge is maintained by the Chatter module that the cell or edge is now associated with. For more information on these commands, see chatter join-hive and chatter assign in the CNA Command Reference Guide. Viewing the master key for a hive To assign a Chatter module to a hive, you need the master key for the hive. The master key is used for secure and authenticated communication between all Chatter modules in the hive. To view the master key for a Chatter module: 1. Access privileged command mode on a CNA server that currently participates in the hive. To access this mode, enter the following command: enable. 2. Enter the following command: show chatter master-key For more information on this command, see show chatter master-key in the CNA Command Reference Guide. Issue 3 May 2008 65 Configuring Agent-Based Measurement Removing a Chatter module from a hive To remove a CNA server’s Chatter module from a hive: 1. Access privileged command mode on the CNA server that you want to remove. To access this mode, enter the following command: enable. 2. Enter the following command: chatter leave-hive For more information on this command, see chatter leave-hive in the CNA Command Reference Guide. Defining down status for Chatter modules The Chatter modules in a hive send heartbeat messages between themselves to ensure that all Chatter modules are functional and performing the tests assigned to them. Heartbeat messages are sent every 60 seconds. If a specific number of heartbeats are missed from a Chatter module, its status changes to down, and its tasks are redistributed between the remaining Chatter modules in the hive. By default, 10 missed heartbeats define down status for Chatter modules, but you can change this setting to any number from 1 to 255. To change the number of missed heartbeats that define down status for Chatter modules: 1. Access Chatter Configuration command mode. To access this mode, enter the following commands: a. enable (Privileged mode) b. configure terminal (Configuration mode) c. module chatter (Chatter Configuration mode) 2. Enter the following command: chatter missed-heartbeats <value> where <value> is number of missed heartbeats that defines down status for Chatter modules. For more information on this command, see chatter missed-heartbeats in the CNA Command Reference Guide. Note: Note: For the Chatter modules to recognize each other’s heartbeat messages, all CNA servers in the hive must run the same version of CNA software. If the CNA servers run different versions of software, they will not recognize each other’s heartbeat messages, and their status changes to down. 66 CNA Monitoring Guide, Version 4.1 Hives Synchronizing a Chatter module with its hive Chatter modules in a hive automatically synchronize their data periodically. However, you may sometimes want to force one Chatter module to synchronize its data with that of other Chatter modules in its hive. To force a Chatter module to synchronize its data with other Chatter modules in its hive: 1. Access Privileged command mode. To access this mode, enter the following command: enable. 2. Enter the following command: resync chatter chapld For more information on this command, see resync chatter chapld in the CNA Command Reference Guide. Registering CNA agents with the hive CNA agents in your subnet need to register with the hive so that the CNA servers in the hive can: ● Schedule tests to the agent. ● Analyze the test data. ● Share the test results so that they can include the results in their reports. Agent-based measurement reports reflect data from the entire subnet, not just local CNA agents. To ensure security of communication between CNA agents and CNA servers, the registration process: ● Authenticates the identity of both the CNA agent and CNA server ● Exchanges private keys that are used encrypt and decrypt all subsequent messages between the CNA servers and CNA agents. Maximum CNA agents supported The CNA system supports a maximum of 1000 CNA agents. A maximum of 700 endpoints can be IP phones. However, if a system contains more than 64 total CNA agents, the CNA Agents embedded in IP Phones are not used to run tests. Avaya recommends that you configure CNA agents in the following order: 1. Add all Chatter modules to the hive. See Assigning Chatter modules to a hive. 2. Register CNA agents in security gateways, media gateways, and Avaya partner products such as Extreme switches, as well as any standalone CNA Agent Devices. Issue 3 May 2008 67 Configuring Agent-Based Measurement 3. Register Avaya IP phones. Configure the IP phone DHCP servers so that each IP phone knows the addresses of at least one CNA server. Not all phones need to register with CNA. Avaya recommends that you register at least two CNA agents on each LAN segment of interest, but no more than two of these CNA agents can be IP phones. Reboot the appropriate phones so that they reregister with the DHCP servers, and then with CNA. CNA server’s embedded agent software When you add a CNA server to a hive, the server’s embedded agent software automatically registers with the server and requires no manual configuration. Like any other type of CNA agent, the CNA server’s embedded CNA agent can be assigned to any zone in your zone hierarchy. Agents that are embedded in CNA servers are useful for measuring subnet performance between subnet operations centers or similar hubs where CNA servers are usually placed. Avaya Security Gateways’ embedded CNA agents CNA agents in Avaya Security Gateways provide the unique ability to measure subnet performance inside Virtual Private Network (VPN) tunnels between secure gateways. Systems without this capability can measure only the overall performance of the tunnel. With this capability, as long as Avaya Security Gateways terminate the VPN on both sides of the tunnel, you can measure performance over every link in the tunnel, even if the links are located in a Service Provider's subnet. Note: Note: Only Avaya Security Gateways running VPN Manager 3.5 or later support CNA agent functionality. If your security gateways are running VPN Manager 3.5, contact Avaya support for instructions. To register the CNA agent in a secure gateway, perform the following tasks: 1. Define the IP address to be used for registration. 2. Propagate the IP address for registration to all security gateways in the domain. Registering other types of CNA agents For information on how to install and configure stand-alone CNA Agent Devices, see the CNA Installation Guide. For information on how to configure other types of CNA agents to register with a hive, see the documentation for the specific product. 68 CNA Monitoring Guide, Version 4.1 Zones Removing a CNA agent from a hive Once a CNA agent registers with a hive, it is not removed from the hive's database unless you do so manually. This functionality prevents the hive from losing track of agents that may be temporarily unavailable. If you are sure that you no longer want to use a specific CNA agent, manually remove it. To manually remove a CNA agent: 1. Access privileged command mode. To access this mode, enter the following command: enable. 2. Enter the following command: chatter remove-cna-agent ip-address <ip> where <ip> is the IP address of the specific CNA agent to remove. For more information on this command, see chatter remove-cna-agent in the CNA Command Reference Guide. Zones This section contains the following topics: ● About zones ● Defining a zone hierarchy ● Removing a zone ● Renaming a zone ● Assigning CNA agents to zones ● Removing a CNA agent from a zone About zones Purpose of zones To organize a hive, you define a hierarchy of zones, which can be based on your organization’s structure, functional elements, geographic areas, or any combination of these characteristics. Issue 3 May 2008 69 Configuring Agent-Based Measurement The zone hierarchy determines how agent-based measurement tests are scheduled and test results are displayed. The purpose of the zone hierarchy is to identify places where end users need services so that these services can be measured and analyzed realistically on an end-to-end basis. So when you define zones, do so in the way that is most meaningful for your organization. Table 5 describes how to organize zones based on how you want to measure subnet performance. Table 5: Zone organization for measurement of subnet performance To measure subnet performance... Organize zones by... Over Wide Area Networks (WANs) Geography Between internal organizations Organizational function Both over WANs and between internal organizations Geography at one level and organizational function at a different level CNA measures end-to-end subnet performance at each level of the zone hierarchy for each zone that contains subzones or CNA agents. To measure end-to-end performance, CNA runs tests between every combination of the zone’s subzones. One test target is randomly chosen from each of the two subzones participating in the test. For more information on how these tests are scheduled, see Agent-based measurement tests. You must plan your zone hierarchy at the same time that you plan deployment of CNA agents. The CNA agents must be in the correct physical location to measure subnet performance to that location. Zone configuration guidelines The following guidelines and restrictions apply to your zone hierarchy: ● Zones can either be further categorized into subzones or can contain specific CNA agents. Zones cannot contain both subzones and CNA agents. ● CNA agents cannot be assigned to the first level (parent or root) zone. ● The zone hierarchy must have at least two levels. ● The system supports a maximum of 500 zones (at all levels). ● Zone names must be unique within the hive. ● Zone names should describe the zones geographically, functionally, or any other way that makes the data meaningful. 70 CNA Monitoring Guide, Version 4.1 Zones Defining a zone hierarchy Zones contain subzones, which in turn contain either lower level subzones or CNA agents. Zone names at each level are separated by a dot. When you enter a fully qualified zone name, the higher level zones are automatically created, if not already created. Procedure To create a zone or subzone: 1. Access Chatter Configuration command mode. To access this mode, enter the following commands: a. enable (Privileged mode) b. configure terminal (Configuration mode) c. module chatter (Chatter Configuration mode) 2. Enter the following command: zone <name> where <name> is a fully qualified name of the zone or subzone that you are creating. The name consists of a zone and multiple subzones that are separated by a period (x.y.z format). Restrictions on the name are: - The name cannot be an integer. - The name must start with a letter (upper or lower case) and be followed by a letter (upper or lower case), digit, dash or underscore. - All zone and subzone names must be unique. For more information on this command, see zone in the CNA Command Reference Guide. Example Zone AV contains two subzones: ● APAC ● US Each of these zones contains further subzones. For example: ● APAC contains subzones: - Beijing - Malaysia - Sydney Issue 3 May 2008 71 Configuring Agent-Based Measurement ● US contains subzones: - Milpitas - Boston (Boston contains two further subzones: Waltham and Concord). Leaf zones (the lowest level subzones) contain CNA agents. Figure 2 shows this zone hierarchy. Figure 2: Zone hierarchy example The following configuration commands create this zone hierarchy: hostname (config-chatter)# zone AV.APAC.Beijing define hostname (config-chatter-zone)# cna-agent 192.168.1.3 hostname (config-chatter-zone)# cna-agent 192.168.1.2 hostname (config-chatter-zone)# end hostname (config-chatter)# zone AV.APAC.Malaysia hostname (config-chatter)# zone AV.APAC.Sydney define hostname (config-chatter-zone)# cna-agent 192.168.2.2 hostname (config-chatter-zone)# cna-agent 192.168.2.3 hostname (config-chatter-zone)# end hostname (config-chatter)# zone AV.US.Milpitas define hostname (config-chatter-zone)# cna-agent 192.168.5.2 hostname (config-chatter-zone)# cna-agent 192.168.5.3 72 CNA Monitoring Guide, Version 4.1 Zones hostname (config-chatter-zone)# cna-agent 192.168.5.4 hostname (config-chatter-zone)# end hostname (config-chatter)# zone AV.US.Boston.Waltham define hostname (config-chatter-zone)# cna-agent 192.168.3.3 hostname (config-chatter-zone)# cna-agent 192.168.3.4 hostname (config-chatter-zone)# end hostname (config-chatter)# zone AV.US.Boston.Concord hostname (config-chatter-zone)# cna-agent 192.168.4.3 hostname (config-chatter-zone)# cna-agent 192.168.4.2 hostname (config-chatter-zone)# end Removing a zone To remove a zone or subzone: 1. Access Chatter Configuration command mode. To access this mode, enter the following commands: a. enable (Privileged mode) b. configure terminal (Configuration mode) c. module chatter (Chatter Configuration mode) 2. Enter the following command: no zone <name> where <name> is a fully qualified name of the zone or subzone that you are removing. For more information on this command, see zone in the CNA Command Reference Guide. Renaming a zone You can rename a zone; however, this functionality is designed primarily to correct an error in a specific zone name, not to change the hierarchy of a zone. The zone name that you enter must be a single zone name, not a fully qualified name. To change the hierarchy of a zone, you must delete it and then recreate it with the new hierarchy. Issue 3 May 2008 73 Configuring Agent-Based Measurement Procedure To rename a zone or subzone: 1. Access Privileged mode. To access this mode, enter the following command: enable. 2. Enter the following command: rename zone <orig-name> <new-name> where <orig-name> is the specific, existing name of the zone or subzone that you are renaming. Enter just the name, not the fully qualified name (x.y.z). <new-name> is the new name for the zone or subzone. Enter just the name, not the fully qualified name (x.y.z). Restrictions on the name are: - The name cannot be an integer. - The name must start with a letter (upper or lower case) and be followed by a letter (upper or lower case), digit, dash or underscore. - All zone and subzone names must be unique. For more information on this command, see rename zone in the CNA Command Reference Guide. Assigning CNA agents to zones The CNA system supports a maximum of 1000 CNA agents (at all levels). Procedure To assign a CNA agent to a zone: 1. Access Chatter Zone Configuration mode. To access this mode, enter the following commands: a. enable (Privileged mode) b. configure terminal (Configuration mode) c. module chatter (Chatter Configuration mode) d. zone <name> define (Chatter Zone Configuration mode) 2. Enter the following command: cna-agent <ip-address> where <ip-address> is the IP address of the CNA agent that you are adding. For more information on this command, see cna-agent in the CNA Command Reference Guide. 74 CNA Monitoring Guide, Version 4.1 Agent-based measurement tests Removing a CNA agent from a zone To remove a CNA agent from a zone: 1. Access Chatter Zone Configuration mode. To access this mode, enter the following commands: a. enable (Privileged mode) b. configure terminal (Configuration mode) c. module chatter (Chatter Configuration mode) d. zone <name> define (Chatter Zone Configuration mode) 2. Enter the following command: no cna-agent <ip-address> where <ip-address> is the IP address of the CNA agent that you are removing. For more information on this command, see cna-agent in the CNA Command Reference Guide. Agent-based measurement tests This section contains the following topics: ● About agent-based measurement tests ● Enabling a test ● Enabling a test ● Disabling a test ● Disabling intra-zone tests ● Configuring a test Issue 3 May 2008 75 Configuring Agent-Based Measurement About agent-based measurement tests Four test types CNA provides four tests for agent-based measurement: ● Real Time Protocol (RTP) is used for media streams such as voice, audio, and video that have a time-dependent relationship between packets. The RTP test echoes a media stream between two endpoints. An RTP test measures the following variables: - Delay—How long (in milliseconds one-way) it takes the UDP packet stream to reach its destination. - Jitter—Unevenness in packet delay (in milliseconds). - Loss—Percentage of packets lost in transit. - Inter Arrival Time—Time between receipt of successive test results from a specific subzone combination (cell). For example, if the test rate is 60 tests per minute, and the zone hierarchy has 60 cells, each cell should report one test result per minute. IAT should be 60 seconds. A longer IAT may indicate a subnet problem or an unresponsive CNA server or agent. ● Ping or echo-check is used to establish connectivity and measure basic connection characteristics between two endpoints. A Ping test measures the following variables: - RTT (Round Trip Time) in milliseconds (ICMP both directions). - Loss—Percentage of packets lost in transit. - Inter Arrival Time—Time between receipt of successive test results from a specific subzone combination (cell). ● Traceroute collects routing information by identifying each node in succession until the destination is reached. A traceroute test measures the following variables: - Hop RTT—Round Trip Time (RTT) in milliseconds for each hop along the route (UDP outbound and ICMP inbound). - Inter Arrival Time—Time between receipt of successive test results from a specific subzone combination (cell). ● Transmission Control Protocol (TCP) Connect is used for media streams that do not have a time dependent relationship, but that must be transmitted accurately, typically data streams. A TCP test measures the following variables: - Cerror—Connection success or failure. - Delay—How long (in milliseconds) the connection takes to complete. - Inter Arrival Time—Time between receipt of successive test results from a specific subzone combination (cell). 76 CNA Monitoring Guide, Version 4.1 Agent-based measurement tests Test targets Each test can run between two CNA agents, or between one CNA agent and an IP address. By default, tests run between CNA agents. ● When tests run between CNA agents, CNA randomly chooses one CNA agent from each of the two subzones participating in the test. If the subzone itself does not contain CNA agents, a CNA agent is chosen from one of the subzone’s lower level subzones that does contain CNA agents. Tests are run for each zone that contains subzones. To test a zone, CNA runs tests between every combination of the zone’s subzones. Each subzone combination is called a cell. For more information on cells, see Cell definition. Tests are also run between CNA agents in the same subzone. These tests are called intra-zone tests and have the same subzone for their source and destination. By default intra-zone testing is enabled, but you can disable it. For more information on intra-zone testing and how to disable it, see Disabling intra-zone tests. For an example of all test targets in a specific zone hierarchy, see Example: test targets. ● When tests run between a CNA agent and an IP address, CNA randomly chooses a CNA agent from the subzone participating in the test to send test packets to the target IP address that you specify. Tests are run for each zone that contains subzones. To test between a zone and specific IP address, CNA runs tests from each of the zone’s subzones to the target IP address. Each subzone-target IP address combination is called a cell. For more information on cells, see Cell definition. For information on how to set an IP address as a test target, see Configuring a test. Use the target command to set the IP address that you want to test. ! Important: Important: When testing to one IP address, be careful not to overload the device or the subnet containing it. If the test rate is too high, the volume of test packets being sent to the single device may exceed that device's ability to process IP packets. Running tests to a single IP address should be used only for diagnostic purposes and only for short periods of time. An empty zone (no subzones or CNA agents) does not participate in any tests. A zone must have CNA agents assigned to its subzones to participate in tests. All communication is encrypted to protect from outside attacks. Issue 3 May 2008 77 Configuring Agent-Based Measurement Cell definition A cell is a pair of targets between which CNA schedules tests: ● When CNA tests between two CNA agents, a cell is a pair of subzones that have a common parent zone. To determine the real-time subnet performance of the parent zone, CNA runs tests for each of the zone’s cells. ● When CNA tests between a CNA agent and an IP address, a cell is one subzone and one IP address. To determine the real-time subnet performance between a parent zone and specific IP address, CNA runs tests for each cell. Example: Cells for testing between CNA agents Figure 3 shows a sample zone hierarchy for root zone AV. This root zone has two subzones: APAC and US. Table 6 shows a matrix of zone APAC’s three subzones and the nine cells that they comprise. To test APAC’s subnet performance, CNA runs tests for each cell shown in the table except for the cells that have Malaysia for a source or destination. Malaysia contains no CNA agents, so it does not participate in any tests. Figure 3: Zone hierarchy example 78 CNA Monitoring Guide, Version 4.1 Agent-based measurement tests Table 6: Test cells for APAC zone Beijing Malaysia Sydney Beijing Cell 1 Beijing to Beijing (intra-zone test) Cell 2 Beijing to Malaysia Cell 3 Beijing to Sydney Malaysia Cell 4 Malaysia to Beijing Cell 5 Malaysia to Malaysia (intra-zone test) Cell 6 Malaysia to Sydney Sydney Cell 7 Sydney to Beijing Cell 8 Sydney to Malaysia Cell 9 Sydney to Sydney (intra-zone test) Example: Cells for testing between a CNA agent and IP address Table 7 shows a matrix of zone APAC’s three subzones and a target IP address. To test subnet performance between zone APAC and the target IP address, CNA runs tests for each cell shown in the table except for the cell 2, which has Malaysia for a source. Malaysia contains no CNA agents, so it does not participate in any tests. Table 7: Test cells for APAC zone to 192.168.5.5 192.168.5.5 Beijing Cell 1 Beijing to 192.168.5.5 Malaysia Cell 2 Malaysia to 192.168.5.5 Sydney Cell 3 Sydney to 192.168.5.5 Example: test targets Zone hierarchy Figure 3 shows a sample zone hierarchy for root zone AV. This root zone has two subzones: APAC and US. ● APAC contains subzones: - Beijing (contains CNA agents) - Malaysia (contains no CNA agents or subzones) - Sydney (contains CNA agents) Issue 3 May 2008 79 Configuring Agent-Based Measurement ● US contains subzones: - Boston (contains two subzones: Waltham and Concord) - Milpitas (contains CNA agents) Test targets For zone AV, tests are run between the following subzones. ● APAC → APAC (intra-zone measurement) ● US → US (intra-zone measurement) ● APAC → US ● US → APAC Tests are run at the test rate that is set for the test type. To set the test rate, use the rate command in Configuring a test. Each time the test is run, CNA chooses a different pair of subzones for the test. From each of the two subzones participating in the test, CNA randomly chooses one CNA agent to perform the test. For zone APAC, tests are run between the following subzones. To conduct the test, CNA randomly chooses one CNA agent from each of the two subzones participating in the test. ● Beijing → Sydney ● Sydney → Beijing ● Beijing → Beijing (intra-zone measurement) ● Sydney → Sydney (intra-zone measurement) Note: The following tests are not run for zone APAC because Malaysia does not contain any CNA agents: Note: - Malaysia → Sydney - Sydney → Malaysia - Beijing → Malaysia - Malaysia → Beijing - Malaysia → Malaysia (intra-zone measurement) For zone US, tests are run between the following subzones. To conduct the test, CNA randomly chooses one CNA agent from each of the two subzones participating in the test. Since Boston has no CNA agents assigned to it, a CNA agent is randomly chosen from one of its subzones, Concord or Waltham. ● Boston → Milpitas ● Milpitas → Boston ● Boston → Boston (intra-zone measurement) ● Milpitas → Milpitas (intra-zone measurement) 80 CNA Monitoring Guide, Version 4.1 Agent-based measurement tests For zone Boston, tests are run between the following subzones. To conduct the test, CNA randomly chooses one CNA agent from each of the two subzones participating in the test. ● Waltham → Concord ● Concord → Waltham ● Waltham → Waltham (intra-zone measurement) ● Concord → Concord (intra-zone measurement) Enabling a test To enable an agent-based measurement test: 1. Access Chatter Active Measurement Configuration mode for the test that you want to enable. To access this mode, enter the following commands: a. enable (Privileged mode) b. configure terminal (Configuration mode) c. module chatter (Chatter Configuration mode) d. active-measurement name {rtp | ping | tcp | traceroute} (Chatter Active Measurement Configuration mode) 2. Enter the following command: enable For more information on this command, see enable in the CNA Command Reference Guide. Disabling a test To disable an agent-based measurement test: 1. Access Chatter Active Measurement Configuration mode for the test that you want to disable. To access this mode, enter the following commands: a. enable (Privileged mode) b. configure terminal (Configuration mode) c. module chatter (Chatter Configuration mode) d. active-measurement name {rtp | ping | tcp | traceroute} (Chatter Active Measurement Configuration mode) 2. Enter the following command: no enable For more information on this command, see enable in the CNA Command Reference Guide. Issue 3 May 2008 81 Configuring Agent-Based Measurement Disabling intra-zone tests Intra-zone tests are tests run between two CNA agents in the same zone or subzone. These tests have the same zone or subzone for their source and destination. By default, intra-zone testing is enabled. To disable intra-zone testing: 1. Access Chatter Configuration command mode. To access this mode, enter the following commands: a. enable (Privileged mode) b. configure terminal (Configuration mode) c. module chatter (Chatter Configuration mode) 2. Enter the following command: measurement intra-zone disable Configuring a test To configure a specific agent-based measurement test: 1. Access Chatter Active Measurement Configuration mode for the test that you want to configure. To access this mode, enter the following commands: a. enable (Privileged mode) b. configure terminal (Configuration mode) c. module chatter (Chatter Configuration mode) d. active-measurement name {rtp | ping | tcp | traceroute} (Chatter Active Measurement Configuration mode) 82 CNA Monitoring Guide, Version 4.1 Agent-based measurement tests 2. Use the following commands to configure the test: Table 8: Agent-based measurement test commands Command Purpose rate (Chatter module only) Defines the test frequency for the test over the entire agent-based measurement system. For example, if you set a rate of one per minute for RTP tests, CNA injects one RTP test per minute into the subnet. Applicable for all test types. repeat Defines the number of times that the test is conducted. Applicable for all test types. Enter 0 for infinite repeat. topological-analysis disable Disables or enables topological analysis for the test. The default setting is enabled. Topological analysis issues a traceroute between the CNA agents to discover intermediate hops and devices. This information makes it possible for CNA to identify potential hops where subnet disruption is present. Applicable for all test types. number-of-packets Defines the number of packets that are sent between test targets for each instance of the test. Applicable for RTP, Ping, and Traceroute tests. interval-between-packets Defines the interval between packets that are sent between test targets. Applicable for RTP, Ping, and Traceroute tests. packet-length Defines the payload length in bytes for test packets that are sent between test targets. Applicable only for Ping and RTP tests. timeout (Chatter module only) Defines the time (in milliseconds) at which the test times out if no response is received from the remote device. Applicable for all test types. tos byte (Chatter module only) Defines the type-of-service (TOS) byte to use when performing the test. Use this command to mark agent-based measurement traffic. Define a TOS byte for agent-based measurement traffic if you want CNA to measure the performance of one (or more) specific TOS markings. Applicable for RTP, Ping, and Traceroute tests. max-ttl Defines the maximum TTL (in hops) for traceroute tests. Applicable only for Traceroute tests. 1 of 2 Issue 3 May 2008 83 Configuring Agent-Based Measurement Table 8: Agent-based measurement test commands (continued) Command Purpose port (Chatter module only) Defines the port to which the TCP test is directed. The default setting is port 80. Applicable only for TCP tests. target (Chatter module only) Configures the test to be conducted from CNA agents to one specific IP address. If a target is not specified (the normal mode of operation), tests are performed between randomly selected pairs of CNA agents. Applicable for all test types. 2 of 2 Agent-based measurement alarms This section contains the following topics: ● About alarms ● Defining alarm destinations ● Disabling absolute alarms globally ● Disabling relative alarms globally ● Configuring alarm sensitivity About alarms Alarm types CNA provides two types of alarms for agent-based measurement: ● Absolute alarms are generated when a specific test measurement exceeds the threshold that you configure. ● Relative alarms are generated when the most recent results for a specific test show an abrupt increase or decrease. The thresholds for determining an “abrupt increase or decrease” are self-adjusting thresholds based on a statistical technique. Note that if the measurement remains near or outside the threshold, the threshold adjusts to reflect this changed norm. Once the threshold self-adjusts, the alarm clears even though the measurement itself has not changed. The steady state becomes the threshold value. This value can be higher or lower than the configured threshold. 84 CNA Monitoring Guide, Version 4.1 Agent-based measurement alarms To configure the alarm threshold values, you create alarm templates and assign them alarm threshold values, as well as alarm destinations, and other alarm settings. You then assign the templates to specific zones or edges. For more information on alarm templates, see Alarm templates. Alarm destinations Alarms are sent to the destinations that you configure. To configure alarm destinations: 1. First, you define all destinations that CNA can use for alarm notifications. You can define: - Four e-mail addresses - Four Syslog servers - Four hosts to receive SNMP traps and informs. For information on how to define alarm destinations, see Defining alarm destinations. 2. Once you have defined the alarm destinations, you create alarm templates and assign them specific alarm destinations (one or more of those defined in Step 1), as well as alarm threshold values, and other alarm settings. 3. Lastly, you assign a template to specific zones or edges. For information on how to configure alarm templates and assign them to specific zones or edges, see Alarm templates. Defining alarm destinations To define alarm destinations for agent-based measurement: 1. Access Chatter Configuration command mode. To access this mode, enter the following commands: a. enable (Privileged mode) b. configure terminal (Configuration mode) c. module chatter (Chatter Configuration mode) Issue 3 May 2008 85 Configuring Agent-Based Measurement 2. Use the following commands to define the destinations that CNA can use for alarm notifications: Table 9: Alarm destination commands Command Purpose smtp-server (Chatter module only) Defines up to four e-mail addresses that can receive alarm notifications for agent-based measurement. snmp-server Defines up to four hosts that can receive SNMP traps or informs for agent-based measurement. logging-server Defines up to four Syslog servers that can receive alarm notifications for agent-based measurement. Disabling absolute alarms globally By default, absolute alarms are globally enabled. If you want to disable them, you can disable them either globally, or you can disable them for specific tests in specific alarm templates. The following procedure disables absolute alarms globally. For information on how to disable absolute alarms for a specific test in a specific alarm template, see Disabling absolute alarms for a test type. To disable absolute alarms globally: 1. Access Chatter Configuration command mode. To access this mode, enter the following commands: a. enable (Privileged mode) b. configure terminal (Configuration mode) c. module chatter (Chatter Configuration mode) 2. Enter the following command: alarm-absolute disable For more information on this command, see alarm-absolute disable in the CNA Command Reference Guide. 86 CNA Monitoring Guide, Version 4.1 Agent-based measurement alarms Disabling relative alarms globally By default, relative alarms are globally enabled. If you want to disable them, you can disable them either globally, or you can disable them for specific tests in specific alarm templates. The following procedure disables relative alarms globally. For information on how to disable relative alarms for a specific test in a specific alarm template, see Disabling relative alarms for a test type. To disable relative alarms globally: 1. Access Chatter Configuration command mode. To access this mode, enter the following commands: a. enable (Privileged mode) b. configure terminal (Configuration mode) c. module chatter (Chatter Configuration mode) 2. Enter the following command: alarm-relative disable For more information on this command, see alarm-relative disable in the CNA Command Reference Guide. Configuring alarm sensitivity You can control CNA’s sensitivity to alarm conditions by setting the number of times that a threshold value must be exceeded for an alarm to be generated. Note that you must balance the test rate with alarm sensitivity to meet your needs. For example, if the test rate is once per minute, and the zone hierarchy has 60 cells, all cells are tested once an hour. If you set the system to generate an alarm when the threshold is exceeded 10 times, the system will take 10 hours to generate an alarm for a cell. To prevent a few spurious points from setting off an alarm, Avaya recommends that you set the system to generate an alarm when the threshold is exceeded no less than five times. Generally an alarm should indicate a persistent condition. Procedure To set the number of times that the threshold must be exceeded for an alarm to be generated: 1. Access Chatter Configuration command mode. To access this mode, enter the following commands: a. enable (Privileged mode) b. configure terminal (Configuration mode) c. module chatter (Chatter Configuration mode) Issue 3 May 2008 87 Configuring Agent-Based Measurement 2. Enter the following command: alarm-threshold-exceeded <count> where <count> is the number of times the threshold must be exceeded for an alarm to be generated. Note: To prevent a few spurious points from setting off an alarm, Avaya recommends that you set the system to generate an alarm when the threshold is exceeded no less than five times. Generally an alarm should indicate a persistent condition. Note: For more information on this command, see alarm-threshold-exceeded in the CNA Command Reference Guide. Alarm templates This section contains the following topics: ● About alarm templates ● Creating an alarm template ● Defining alarm destinations ● Defining test thresholds ● Disabling absolute alarms for a test type ● Disabling relative alarms for a test type ● Assigning templates to zones or edges About alarm templates An alarm template specifies the following parameters and actions for all links and all zones assigned to it: ● Alarm notification destinations: - Up to four SNMP trap locations for communication with external alarm monitoring systems. - Up to four e-mail addresses for communication with specific personnel. - Up to four Syslog servers for communication with systems that require this protocol. For information on how to define the alarm destinations for an alarm template, see Defining alarm destinations. 88 CNA Monitoring Guide, Version 4.1 Alarm templates ● Alarm parameters: - Thresholds for absolute alarms. You can configure a threshold for each test variable in each test (RTP, Ping, TCP, and Traceroute). See Defining test thresholds. - Enable or disable absolute alarms for each test or individual test variables. See Disabling absolute alarms for a test type. - Enable or disable relative alarms for each test or individual test variables. Disabling relative alarms for a test type. Note: Note: Alarm parameters are specified separately for each test variable. A single alarm template can have any number of links and any number of zones assigned to it. When an alarm condition occurs on any zone or link that is assigned the template, a notification is sent to all alarm destinations that are defined for the template. Creating an alarm template To create an alarm template: 1. Access Chatter Configuration command mode. To access this mode, enter the following commands: a. enable (Privileged mode) b. configure terminal (Configuration mode) c. module chatter (Chatter Configuration mode) 2. Enter the following command: alarm name <name> where <name> is the name of the alarm template that you are creating. Each alarm template must have a unique name. For more information on this command, see alarm name in the CNA Command Reference Guide. Defining alarm destinations When an alarm condition occurs on any zone or link that is assigned the template, a notification is sent to all alarm destinations that are defined for the template. Issue 3 May 2008 89 Configuring Agent-Based Measurement Procedure To define alarm destinations for an alarm template: 1. Access Chatter Alarm Configuration command mode. To access this mode, enter the following commands: a. enable (Privileged mode) b. configure terminal (Configuration mode) c. module chatter (Chatter Configuration mode) d. alarm name <name> (Chatter Alarm Configuration command mode) 2. Use the commands in Table 10 to define alarm destinations for the template. Table 10: Alarm template: alarm destination commands Command Purpose smtp to (Chatter module only) Specifies the e-mail addresses to which alarm notifications for this alarm template are sent. snmp host Specifies the hosts to which SNMP traps for this alarm template are sent. logging host Specifies the Syslog servers to which Syslog messages for this alarm template are sent. Defining test thresholds To define test thresholds for an alarm template: 1. Access Chatter Alarm Configuration command mode. To access this mode, enter the following commands: a. enable (Privileged mode) b. configure terminal (Configuration mode) c. module chatter (Chatter Configuration mode) d. alarm name <name> (Chatter Alarm Configuration command mode) 90 CNA Monitoring Guide, Version 4.1 Alarm templates 2. Use the commands in Table 11 to define thresholds for specific tests. Table 11: Alarm template: test threshold commands Command Purpose threshold rtp Defines the alarm thresholds for RTP tests on any zone or edge that is assigned this alarm template. Thresholds can be defined for: ● Delay ● Jitter ● Packet loss ● Inter arrival time threshold ping Defines the alarm thresholds for Ping tests on any zone or edge that is assigned this alarm template. Thresholds can be defined for: ● Round trip time (RTT) ● Packet loss ● Inter arrival time threshold tcp Defines the alarm thresholds for TCP tests on any zone or edge that is assigned this alarm template. Thresholds can be defined for: ● Delay ● Cerror ● Inter arrival time threshold traceroute Defines the alarm thresholds for Traceroute tests on any zone or edge that is assigned this alarm template. Thresholds can be defined for: ● Hop round trip time (RTT) ● Inter arrival time Disabling absolute alarms for a test type This procedure disables absolute alarms for specific tests in a specific alarm template. You can also disable absolute alarms globally. For information on how to disable absolute alarms globally, see Disabling absolute alarms globally. Issue 3 May 2008 91 Configuring Agent-Based Measurement Procedure To disable absolute alarms for a test type: 1. Access Chatter Alarm Configuration command mode. To access this mode, enter the following commands: a. enable (Privileged mode) b. configure terminal (Configuration mode) c. module chatter (Chatter Configuration mode) d. alarm name <name> (Chatter Alarm Configuration command mode) 2. Use the commands in Table 12 to disable absolute alarms for a test type. You can disable absolute alarms for specific test variables or all variables. Table 12: Alarm template: disable absolute alarm commands Command Purpose alarm-absolute disable rtp Disables absolute alarms for RTP tests on any zone or edge that is assigned this alarm template. You can control generation of absolute alarms for each variable individually or all variables collectively. alarm-absolute disable ping Disables absolute alarms for Ping tests on any zone or edge that is assigned this alarm template. You can control generation of absolute alarms for each variable individually or all variables collectively. alarm-absolute disable traceroute Disables absolute alarms for Traceroute tests on any zone or edge that is assigned this alarm template. You can control generation of absolute alarms for each variable individually or all variables collectively. alarm-absolute disable tcp Disables absolute alarms for TCP tests on any zone or edge that is assigned this alarm template. You can control generation of absolute alarms for each variable individually or all variables collectively. Disabling relative alarms for a test type This procedure disables relative alarms for specific tests in a specific alarm template. You can also disable relative alarms globally. For information on how to disable relative alarms globally, see Disabling relative alarms globally. 92 CNA Monitoring Guide, Version 4.1 Alarm templates Procedure To disable relative alarms for a test type: 1. Access Chatter Alarm Configuration command mode. To access this mode, enter the following commands: a. enable (Privileged mode) b. configure terminal (Configuration mode) c. module chatter (Chatter Configuration mode) d. alarm name <name> (Chatter Alarm Configuration command mode) 2. Use the commands in Table 13 to disable relative alarms for a test type. You can disable relative alarms for specific test variables or all variables. Table 13: Alarm template: disable relative alarm commands Command Purpose alarm-relative disable rtp Disables relative alarms for RTP tests on any zone or edge that is assigned this alarm template. You can control generation of relative alarms for each variable individually or all variables collectively. alarm-relative disable ping Disables relative alarms for Ping tests on any zone or edge that is assigned this alarm template. You can control generation of relative alarms for each variable individually or all variables collectively. alarm-relative disable traceroute Disables relative alarms for Traceroute tests on any zone or edge that is assigned this alarm template. You can control generation of relative alarms for each variable individually or all variables collectively. alarm-relative disable tcp Disables relative alarms for TCP tests on any zone or edge that is assigned this alarm template. You can control generation of relative alarms for each variable individually or all variables collectively. Assigning templates to zones or edges To assign alarm templates to zones or edges: 1. Access Chatter Alarm Configuration command mode. To access this mode, enter the following commands: a. enable (Privileged mode) Issue 3 May 2008 93 Configuring Agent-Based Measurement b. configure terminal (Configuration mode) c. module chatter (Chatter Configuration mode) d. alarm name <name> (Chatter Alarm Configuration command mode) 2. Use the commands in Table 14 to assign alarm templates to zones or edges. Table 14: Alarm template: assign zone or edge commands Command Purpose set-alarm edge Assigns an alarm template to a Chatter edge. Each edge can have one alarm template assigned. set-alarm zone Assigns an alarm template to a Chatter zone. Each zone can have one alarm template assigned. Merging and splitting nodes This section contains the following topics: ● About merging and splitting nodes ● Merging nodes ● Splitting nodes ● Refreshing known nodes About merging and splitting nodes CNA uses results from traceroute tests to automatically map the nodes and links between CNA agents. When generating this topology map, CNA identifies and merges IP addresses that belong to a single node. However, sometimes this automatic merge functionality can fail for various reasons. If a node has multiple IP addresses (typically a router, which always has multiple interfaces with different IP addresses), CNA may sometimes split the node into multiple nodes. Or, if CNA erroneously identifies multiple IP addresses as belonging to the same device, CNA may merge the nodes into a single node. If you notice this type of error when you view the topology map, you can manually merge intermediate hops into a single node, or you can manually split a router node into intermediate hops. The following sections describe how to perform these tasks. 94 CNA Monitoring Guide, Version 4.1 Merging and splitting nodes Merging nodes To manually merge intermediate hops into a node: 1. Access Privileged command mode. To access this mode, enter the following command: enable. 2. Enter the following command: chatter node add <ip-list> where <ip-list> is a list of IP addresses (hops) that you want merged to a router node. Separate each IP addresses by one space. For more information on this command, see chatter node add in the CNA Command Reference Guide. Add to CLI Guide: use one space to separate multiple IP addresses. Splitting nodes ! Important: Important: Split nodes that were originally merged by CNA may be remerged if CNA determines that the IP addresses are the same device. The chatter node split command does not exclude the split node from Chatter’s automatic merge functionality. To manually split a router node in to intermediate hops 1. Access Privileged command mode. To access this mode, enter the following command: enable. 2. Enter the following command: chatter node split <ip-address> where <ip-address> is the IP address of the node that you want split. For more information on this command, see chatter node split in the CNA Command Reference Guide. Issue 3 May 2008 95 Configuring Agent-Based Measurement Refreshing known nodes To refresh the list of nodes that are known to CNA: 1. Access Privileged command mode. To access this mode, enter the following command: enable. 2. Enter the following command: resync chatter node-alias Use this command if node names that you enter by using the report prefix-alias (deprecated) command are not automatically transferred to agent-based measurement displays and reports. For more information on this command, see resync chatter node-alias in the CNA Command Reference Guide. Viewing the agent-based measurement configuration To view a CNA server’s agent-based measurement configuration: 1. Access Privileged command mode. To access this mode, enter the following command: enable. 2. Enter the following command: show statistics chatter [admin | alarms | cells | classes | destinations | edges | load | tasks | zones] For more information on this command, see show statistics chatter in the CNA Command Reference Guide. 96 CNA Monitoring Guide, Version 4.1 Chapter contents Chapter 6: Decision Policies and Application Models Chapter contents This chapter contains the following sections: ● Overview ● Command modes ● Configuration Tasks ● Details on Application Models Overview Decision policies Decision policies make it possible for you: ● To give preferential treatment to traffic to subnets of special interest ● To divert traffic to subnets of limited or no interest from premium links For example, you can configure decision policies to: ● Give certain subnets higher priority in the CNA measurement and scheduler queues. By prioritizing traffic to a specific subnet, route assertion for the subnet can be expedited. ● Designate different winner-set-width values for individual subnets, allowing you to control which links are accessible to individual subnets ● Designate different load policies for subnets ● Optimize your VPN traffic Decision policies are configured on the APC (adaptive path controller) engine module. Issue 3 May 2008 97 Decision Policies and Application Models Application models Application models make it possible for you to fine-tune a decision policy according to the particular applications on your subnet. CNA provides six preconfigured application models, but you can easily modify them to suit your needs. CNA uses the application model that you select to assess the relative impact of different types of subnet impairment (latency, loss, and jitter) and act on it in a manner consistent with real-world applications. The preconfigured application models are: ● enterprise - optimizes time-critical TCP transactions on your subnet, such as transactional web ● multimedia - optimizes real-time applications on your subnet; example: video conferences ● streaming - optimizes streaming applications on your subnet; example: video streaming ● voice - optimizes voice over IP traffic on your subnet ● web - optimizes web transactions on your subnet ● other - can be used to vary the optimization parameters for non-web applications on your subnet when you are using only TCP for your measurements For example, if you have a set of subnets that you want to optimize for video conferences, you can associate these subnets with the multimedia application model. This model considers factors that influence multimedia applications (for example, delay, jitter, loss) Command modes To perform the tasks in this chapter, you need access to the following command modes: ● Engine command mode ● Policy Configuration command mode ● Link Policy Configuration command mode The following sections list the commands for accessing these command modes. 98 CNA Monitoring Guide, Version 4.1 Command modes Engine command mode To access Engine command mode, enter the following commands at the default user-level prompt (hostname> prompt): Step Command Resulting Command mode Command prompt 1 enable System operations mode hostname# 2 configure terminal Configuration mode hostname (config)# 3 module engine Adaptive Path Control (APC) Engine mode hostname (config-engine)# Policy Configuration command mode To access Policy Configuration command mode, enter the following commands at the default user-level prompt (hostname> prompt): Step Command Resulting Command mode Command prompt 1 enable System operations mode hostname# 2 configure terminal Configuration mode hostname (config)# 3 module engine Adaptive Path Control (APC) Engine mode hostname (config-engine)# 4 decision-policy <name> Policy Configuration mode hostname (config-engine-policy) Issue 3 May 2008 99 Decision Policies and Application Models Link Policy Configuration command mode To access Link Policy Configuration command mode, enter the following commands at the default user-level prompt (hostname> prompt): Step Command Resulting Command mode Command prompt 1 enable System operations mode hostname# 2 configure terminal Configuration hostname (config)# 3 module engine Adaptive Path Control (APC) Engine hostname (config-engine)# 4 decision-policy <name> Policy Configuration hostname (config-engine-policy) 5 link <name> Link Configuration hostname (config-engine-policy-link)# ! Important: Important: When you access Link Policy Configuration mode, the commands that you enter apply to the link only for those subnets associated with the current decision policy. 100 CNA Monitoring Guide, Version 4.1 Configuration Tasks Configuration Tasks To implement a decision policy, first define the policy, and then associate individual subnets with the policy. This sections contains procedures for the following tasks: Configuring a decision policy Configuring a specific link for a decision policy Associating subnets with a policy Configuring a decision policy To configure a decision policy: 1. Access APC Engine command mode. For information on how to access this command mode, see Engine command mode. 2. Enter the following command to create a decision policy or access an existing one: decision-policy <name> where <name> is the new policy name or existing policy name. 3. Use the following commands to configure the decision policy: Table 15: Decision policy commands Command Purpose application Associates an application with the decision policy. ● ● ● ● ● ● application-model enterprise application-model multimedia application-model other application-model streaming application-model voice application-model web Configures the application model. Options are: ● enterprise ● multimedia ● other ● streaming ● voice ● web For more information on application models, see Details on Application Models. To assign a model to the decision policy, use the set-application-model command. 1 of 4 Issue 3 May 2008 101 Decision Policies and Application Models Table 15: Decision policy commands (continued) Command Purpose as-path Configures the values used by the CNA system to construct a synthetic AS-path for a route not obtained from BGP. auto-create subnets Enables and configures automatic creation of subnets. bgp-community Assigns a BGP community value to all routes selected route assertions. The only keyword that you can use in Policy Configuration mode is all. damped-mode Enables for disables damped mode. When damped mode is enabled, CNA selects a new winner for a subnet only if the current winner is no longer in the winner set. decision-policy-assignment Sets the length of time that an automatically assigned decision policy remains in effect. If no measurements are received from any subnets that are associated with this policy for this length of time, then the automatically assigned decision policy expires. flap-control Controls how often CNA sends updates to an edge router for subnets associated with the policy. ignore-measurements Prevents the CNA decision-maker from recording measurements for specified subnets in the CNA database. This command is designed to prevent health checks (Web hits that you initiate to verify the availability of the CNA system’s USTAT Web server) from skewing CNA-generated reports. link Accesses Link Policy Configuration mode. ! Important: load evaluation-interval Important: When you access Link Policy Configuration mode, the commands that you enter apply to the link only for those subnets associated with the current decision policy. Sets a required time interval between load samples used for load optimization. For more information on load optimization, see Chapter 14, WAN Cost and Load Optimization. 2 of 4 102 CNA Monitoring Guide, Version 4.1 Configuration Tasks Table 15: Decision policy commands (continued) Command Purpose load use-threshold-table Sets usage thresholds and direction of data flow for load optimization. For more information on load optimization, see Chapter 14, WAN Cost and Load Optimization. min-required-measure Sets the minimum number of traffic test measurements required before CNA determines a best path for a subnet. outage-detection active-probe Sets the number of consecutive active-probe failures required for CNA to detect an outage for a subnet. outage-detection brownout Configures the parameters that CNA uses to detect subnet brownout. A subnet brownout occurs when application performance is significantly reduced for end users due to subnet problems such as congestion, packet loss, link or equipment failures, or configuration errors. outage-detection silence Sets the time between measurements that is required for CNA to detect an outage for a subnet. outage-detection total-silence Sets the parameters that CNA uses to detect that a subnet is totally silent—no measurements recorded for any of its links for a specific period of time. persistent-hosts Allow probe failures from hosts to mark a subnet not-rescued prefer-in-winner-set bgp Enables consideration of BGP status when CNA selects a performance route from the winner set. When this setting is enabled, the winning link must be at least as good as the BGP-selected route. perfer-in-winner-set most-reliable Enables consideration of a link reliability when CNA selects a performance route from the winner set. When this setting is enabled, the winning link must have the best reliability score, which is determined by the frequency that a link moves out of the winner set (for example, a link which rarely moves out of the winner set will tend to have better reliability). 3 of 4 Issue 3 May 2008 103 Decision Policies and Application Models Table 15: Decision policy commands (continued) Command Purpose priority Assigns a priority to the decision policy. When more than one policy is associated with a subnet, the policy with a higher priority value is used. The default value is zero (0). If two policies have equal priority values, CNA chooses the policy based on alphabetical order (except for the global decision policy, which always has the lowest priority). For example, gnomon is chosen before vole). priority-subnet Prioritizes all subnets that are associated with the decision policy. When subnets are prioritized, they are processed by CNA decision-maker before non-priority subnets. reliability half-life Sets the half-life parameter, which influences link reliability. reliability tolerance Sets the tolerance parameter, which influences link reliability. route-assert-filter Configures the conditions under which the CNA decision maker asserts routes known to BGP. set-application-model Sets which application model CNA uses when making decisions for the decision policy. Options are: ● enterprise ● other ● streaming ● voice ● multimedia ● web If you do not assign an application model to the policy, then the globally set model is used. To set a global application model, enter this command while in Engine command mode. If no application model is set globally, then the default application model, web, is used. winner-set-width Sets the acceptable range for preferred-route performance scores. To be in the winner-set (the set of service provider links from which the best performing route will be selected), a route’s performance score must be within this range. 4 of 4 For examples of the commands described in this section, see Examples. 104 CNA Monitoring Guide, Version 4.1 Configuration Tasks Note: Most of these commands are available both globally and for a specific decision policy. When applied globally, they affect all decisions for all subnets. When applied to a specific policy, the parameters are applied only to those subnets associated with the policy, and the policy values override the globally set parameters. Note: Any parameter not explicitly set in the decision policy assumes the globally set value for that parameter, and any link-specific setting not explicitly set in the decision policy assumes the value that was set for that link at the global level. Configuring a specific link for a decision policy From Policy Configuration mode, you can access Link Policy Configuration mode (confide-engine-policy-link), from which you can set any link parameter. For more information on how to access Link Policy Configuration mode, see Link Policy Configuration command mode. ! Important: Important: When you access Link Policy Configuration mode, the commands that you enter apply to the designated link only for those subnets associated with the current decision policy. To configure a specific link: 1. Access Link Policy Configuration command mode. For information on how to access this command mode, see Link Policy Configuration command mode 2. Use the following commands to configure the link: Table 16: Link policy commands Command Purpose bgp-community assert if-winner <list> Sets BGP community values for this link in this policy: ● assert if-winner—values are associated with routes only when the specified link is the winner as determined by the CNA decision-maker process ● assert if-in-winner-set—values are associated with routes only when the specified link is in the winner-set ● assert if-not-in-winner-set—values are associated with routes when the specified link is not in the winner set bgp-community assert if-in-winner-set <list> bgp-community assert if-not-in-winner-set <list> outbound-bgp-community Use bgp-community command 1 of 2 Issue 3 May 2008 105 Decision Policies and Application Models Table 16: Link policy commands (continued) Command Purpose bgp-alternate-next-hop BGP Alternate Next Hop penalty Assigns a numerical penalty to the link. CNA uses the penalty value to calculate the performance metric for evaluating the link against other links. winner-set-priority Sets link priority in the winner set. The APC engine uses this priority to choose the best link in the winner set. 2 of 2 For examples of the commands described in this section, see Examples. Examples Table 17: Examples: Configuring Decision Policies and Links Action Command Create a decision policy named policy_1 Avaya_ANS (config-engine)# decision-policy policy_1 Set a narrow winner-set width for all subnets associated with policy_1 Avaya_ANS (config-engine-policy)# winner-set-width 3 5 The winner-set-width value is set to 3, and the extended width is set to 5 (used when load-optimization is enabled and the winner set defined by the winner-set-width value is empty). Prioritize any subnet associated with policy_1. Decisions involving these subnets are processed by CNA decision-maker before non-priority subnets. Avaya_ANS (config-engine-policy)# priority-subnet 1 of 2 106 CNA Monitoring Guide, Version 4.1 Configuration Tasks Table 17: Examples: Configuring Decision Policies and Links (continued) Action Command Accesses Link Configuration mode for link_1. Avaya_ANS (config-engine-policy)# link link_1 For any subnet associated with policy_1, this command specifies that link_1 be chosen when it is in the winner set. Avaya_ANS (config-engine-policy-link)# winner-set-priority prefer Accesses Link Configuration mode for link_2. Avaya_ANS (config-engine-policy)# link link_2 For any subnet associated with policy_1, this command specifies that link_2 be avoided. If link_2 is in the winner set, it will be the CNA system’s last choice. Avaya_ANS (config-engine-policy-link)# winner-set-priority avoid Exits Link Configuration mode. Avaya_ANS (config-engine-policy-link)# end Exits Policy Configuration mode. Avaya_ANS (config-engine-policy-link)# end 2 of 2 After adding the above commands, a show running-config command shows the following block of commands in the CNA configuration: decision-policy policy_1 winner-set-width 3 5 priority-subnet link link_1 winner-set-priority prefer end link link_2 winner-set-priority avoid end end Issue 3 May 2008 107 Decision Policies and Application Models Associating subnets with a policy Once you have configured a decision policy, you can associate subnets with it. With the exception of automatically generated subnets, policies are not inherited by a more specific subnet from a more general subnet. For example, to apply a policy to both the 172.42.0.0/16 and the 172.42.1.0/24 subnets, you must explicitly associate each subnet with the policy, as shown in Examples. Automatically generated subnets, on the other hand, inherit the policy of the more general subnet it is a part of, unless it is overridden by a dynamic decision policy. For more information on auto-created subnets, see auto-create-subnets. To associate subnets with a decision policy: 1. Access Engine command mode. For information on how to access this command mode, see Engine command mode. 2. Enter the following command to access an existing decision policy and associate a subnet with it: set-decision-policy <name> {subnet <address> | group <group-name> | active-measurement-group <am-group-name>} where <name> is the policy name <address> is the IP address of the subnet. Enter the IP address in the format of a.b.c.d/n. Such user-defined subnets—a more specific subnet representing one of your VPN endpoints, for example—are added to the CNA database and are indicated by the link code in snapshot reports and show prefix (deprecated) command output. 3. Repeat Step 2 for each subnet that you want associated with the decision policy. Examples The following examples associate two subnets with the policy_1 decision policy: Avaya_ANS (config-engine)# set-decision-policy policy_1 subnet 172.42.0.0/16 Avaya_ANS (config-engine)# set-decision-policy policy_1 subnet 172.42.1.0/24 Creating a group for subnets You can create a group of subnets so that you can use the group name for all future references to the subnets. Subnet groups save you from having to manually enter multiple commands for multiple subnet addresses. 108 CNA Monitoring Guide, Version 4.1 Configuration Tasks Use the following commands to create a group and add subnets: Table 18: Subnet group commands Command Purpose group Creates a group, or accesses an existing one. subnet Adds a subnet to a group For example, if the subnets in the previous examples belong to your VPN in New York, you could define a group as follows: Avaya_ANS Avaya_ANS Avaya_ANS Avaya_ANS (config)# group (config-group)# (config-group)# (config-group)# vpnNY subnet 172.42.0.0/16 subnet 172.42.1.0/24 end These commands create a group named vpnNY and add two subnets. Now you can use the group name in place of the subnet addresses. Dynamic decision policies Dynamic decision policies are policies that the CNA system automatically assigns to subnets. Instead of manually assigning a decision policy to particular subnet (as in the previous examples), you can give CNA general guidelines for making these assignments. For example, if you configure an active measurement group to probe endpoints involved in VoIP traffic that the EFC module is tasked to find, you can configure the CNA system to use a decision policy that will optimize VoIP traffic for the endpoint’s entire subnet (not just the endpoint found by EFC). The policy is considered to be dynamic because the subnet is automatically associated with the policy when the endpoint is first measured. This eliminates the need for you to specifically configure which prefixes require a specific policy. Decision policies are automatically assigned to subnets in the following circumstances: ● Active measurements: When CNA attempts to measure an endpoint in an active measurement group, then its subnet is associated with the decision policy that you assigned to the active measurement group (by using the set-decision-policy command). If you did not assign a decision policy to the group, then the default, global decision policy is used. ● Single-pixel .gif measurements: If auto-creation of subnets is enabled, the decision policy that is assigned to a newly detected endpoint (using the set-decision-policy command) is assigned to its subnet. If no decision policy is assigned, then the default, global policy is assigned to the subnet. When auto-creation of subnets is enabled, every time CNA detects a new endpoint, it creates a more-specific subnet (if one does not already exist). Issue 3 May 2008 109 Decision Policies and Application Models Decision policy priority Decision policies can be assigned to a subnet in the following ways: ● manually assigned to a subnet (see Configuring a decision policy) ● through a default, global decision policy, which is set with the CLI ● dynamically, by the CNA system. For information on dynamic decision policies, see Dynamic decision policies. When more than one policy is associated with a subnet, the policy that has the higher priority value is used. The default value is zero (0), but you can change this value by using the priority command. If two policies have equal priority values, CNA chooses the policy based on alphabetical order (except for the global decision policy, which always has the lowest priority). For example, gnomon is chosen before vole). Details on Application Models CNA provides six preconfigured application models: ● Enterprise. For more information on this model, see Enterprise application model. ● Voice. For more information on this model, see Voice application model. ● Streaming. For more information on this model, see Streaming application model. ● Multimedia. For more information on this model, see Multimedia application model. ● Web. For more information on this model, see Web application model. ● Other. For more information on this model, see Other application model. Each has internal parameters that you can adjust according to the needs of your applications. In many cases, however, the application models provided do not require any changes and can be used with the default values. Table 19 shows how the type of application influences your choice of application model when the measurement techniques you are using are traceprobe and ICMP. Note: Note: If you are using only TCP for your measurements, then you must use the Web or Other application model. The method that you choose to use likely depends largely on what is convenient for your subnet infrastructure. Traceprobe and ICMP allow for greater accuracy of jitter and loss measurements than is possible with TCP. When using traceprobe and ICMP, the overall performance of the CNA system may show a slight to substantial increase, depending on the type of application in your particular subnet. 110 CNA Monitoring Guide, Version 4.1 Details on Application Models Note: If you use an application model that considers jitter, you should consider enabling auto-creation of subnets in the decision policy. By auto creating subnets, you minimize potential misinterpretation of differences in latency to different points in the same subnet as jitter. In many production contexts, this may involve the automatic creation of subnets up to a /32 degree of specificity. Note: Table 19: Examples of applications and appropriate models for traceprobe and ICMP measurements Application Type Enterprise Model Multimedia Model Real-Time Terminal host gaming Video conferencing Transactional Web services Data Feed Bulk Data Streaming Model Voice Model Voice over IP Streaming audio Streaming video Email Peer-Peer File transfer Enterprise application model Use the enterprise model to optimize time-critical TCP transactions on your subnet, such as transactional Web, file transfer, and gaming (see table Table 19 for examples of which application model is recommended for a particular application type). When using the enterprise application model, you must use either the traceprobe or ICMP measurement mode. If you can use only TCP, then you should use the Web application model because the enterprise model requires measurements of the loss characteristics in your subnet to a higher degree of precision than is possible with TCP. The following parameters control the behavior of the enterprise application model: ● application-delay - represents the calculated average delay (in milliseconds) of a transaction and takes into account an average of three TCP turns for a typical transaction. The metric-contribution affects the weighting of this delay in CNA’s calculations. Its value determines the mid-point of a curve that tells CNA how strictly it should regard application delay in its decisions. A higher number for this value places greater importance on delay (an application is considered to be more affected by a delay). Table 20 shows how the weighting of application-delay affects the performance calculation, from best (*****) to worst (*). Issue 3 May 2008 111 Decision Policies and Application Models Table 20: Application delay affects application performance Application delay (ms) Decision Metric Application Performance 0 - 225 0 - 100 ***** 226 - 500 101 - 200 **** 501 - 750 201 - 300 *** 751 - 1500 301 - 500 ** > 1500 501 - 100 * The first column contains the application delay (in milliseconds). It correlates with the decision metric, which is used by the decision-maker process to evaluate the performance of a route for a particular subnet. The default values for application delay are shown, but you can adjust this table when needed. The range of the decision metric is from 0 (best performance) to 1000 (worst performance). You can change the distribution of this range with metric-contribution. Its default value of 1500ms means that when an application is experiencing 1500ms of delay; the decision metric is 500. For the enterprise model, when the decision metric exceeds 300, then performance for that subnet-link is considered unacceptable. Table 21 shows what happens when you change the metric-contribution to 750ms: the decision metric is affected considerably more by application delay because the metric-contribution sets a delay of 750ms as the mid-point of the decision metric (500), as opposed to the default 1500ms. Table 21: Application Delay: metric-contribution = 750ms Application delay (ms) Decision Metric Application Performance 0 - 125 0 - 100 ***** 126 - 250 101 - 200 **** 251 - 375 201 - 300 *** 376 - 750 301 - 500 ** > 750 501 - 100 * Application Delay is derived from two measurement values: Loss and Round Trip Time (RTT). Table 22 shows how the loss percentage corresponds with application delay, and Table 23 shows the effects of Round Trip Time (RTT). Together, loss and RTT determine the computed application delay and share approximately equal weight. 112 CNA Monitoring Guide, Version 4.1 Details on Application Models Table 22: Loss Affects Application Delay Loss (%) Application delay (ms) Application Performance 0 - 0.8 0 - 225 ***** 0.8 - 3.0 226 - 500 **** 3.0 - 6.0 501 - 750 *** 6.0 - 12.0 751 - 1500 ** > 12.0 > 1500 * Table 23: Loss Affects Application Delay Round Trip Time (ms) Application delay (ms) Application Performance 0 - 40 0 - 225 ***** 41 - 150 226 - 500 **** 151 - 300 501 - 750 *** 301 - 500 751 - 1500 ** > 500 > 1500 * ● delay - determines how the moving average of delay is calculated. The half-life parameter controls the smoothness of the moving average. A high value results in smaller changes in the calculated average delay because the delay contribution of past measurements is greater than when half-life is set low. ● loss - controls the smoothing factor of measured loss. A high value results in a smaller contribution of past loss events; conversely, a low value allows past loss events to have a greater influence on the decision-maker’s calculations. ● metric - this is an internal parameter, do not change it. ● short-term-loss - measures the observed loss in the last window-width packets received. Also uses a threshold to compare against the number of losses in the window to determine if there is loss. Below threshold short-term-loss is 0, above threshold it equals num losses / window-width. Issue 3 May 2008 113 Decision Policies and Application Models Voice application model Use the voice application model to optimize voice traffic on your subnet. When using the voice application model, you must use either the traceprobe or ICMP measurement modes. If you can employ only TCP, then you should use the web or other application model because the voice model requires measurements of the loss and jitter characteristics in your subnet to a higher degree of precision than is possible with TCP. The following parameters control the behavior of the voice application model: ● delay - determines how the moving average of delay is calculated. The half-life parameter controls the smoothness of the moving average. A high value results in smaller changes in the calculated average delay because the delay contribution of past measurements are greater than when half-life is set low. The metric-contribution parameter affects the weighting of this delay in the decision-maker’s performance calculations. Its value determines the mid-point of a curve that tells the decision-maker how strictly it should regard application delay in its decisions. A higher number for this value places greater importance on delay (for example, an application is considered to be impacted more by a delay). It works in the same way as explained in Enterprise application model for application-delay. ● jitter min-buffer - sets the amount of buffer delay introduced by a client, in milliseconds ● loss half-life - controls the smoothing factor of measured loss. A high value results in a smaller contribution of past loss events; conversely, a low value allows past loss events to have a greater influence on the decision-maker’s calculations. ● loss-metric-contribution - affects the weighting of the loss percentage in the decision-maker’s performance calculations. Its value determines the mid-point of a curve that tells the decision-maker process how strictly it should regard this loss in its decisions. A higher number for this value places greater importance on loss. It works in the same way as explained in Enterprise application model for application-delay. ● metric - this is an internal parameter, do not change it. ● short-term-loss - measures the observed loss in the last window-width packets received. Also uses a threshold to compare against the number of losses in the window to determine if there is loss. Below threshold short-term-loss is 0, above threshold it equals num losses / window-width. For the application voice model, application performance is derived from delay, jitter, and loss measurements. Loss measurements consist of both a long term loss average and a short term loss average, and the decision-maker uses the larger of the two along with the delay and jitter measurements for its ultimate routing decision (the resulting value is called the decision metric). Table 24 shows how the computed delay correlates with the decision metric and application performance, from best (*****) to worst (*), while Table 25 shows how the measured loss percentage correlates with the decision metric and application performance. 114 CNA Monitoring Guide, Version 4.1 Details on Application Models Table 24: Round Trip Time Affects Application Performance Delay (ms) Decision Metric Application Performance 0 - 125 0 - 100 ***** 126 - 250 101 - 200 **** 251 - 400 201 - 300 *** 401 - 800 301 - 500 ** > 800 501 - 100 * Table 25: Loss Affects Application Performance Loss (%) Decision Metric Application Performance 0 - 1.5 0 - 100 ***** 1.6 - 3.0 101 - 200 **** 3.1 - 5.0 201 - 300 *** 5.1 - 10 301 - 500 ** > 10 501 - 100 * For the voice application model, delay is calculated from round trip time (RTT) and jitter (as opposed to the enterprise model where RTT and loss percentage are the factors for delay). The CNA system’s default values for RTT and loss are shown, but you can adjust the metric-contribution of each when needed, in the same way as explained in Enterprise application model for application-delay. Streaming application model Use this model to optimize streaming audio and video applications on your subnet. See Table 19 for examples of which application model is recommended for a particular application type. When using the streaming application model, you must use either the traceprobe or ICMP measurement modes. If you can employ only TCP, then you should use the web application model because the streaming model requires measurements of the loss and jitter characteristics in your subnet to a higher degree of precision than is possible with TCP. Issue 3 May 2008 115 Decision Policies and Application Models The following parameters control the behavior of the streaming application model: ● delay - determines how the moving average of delay is calculated. The half-life parameter controls the smoothness of the moving average. A high value results in smaller changes in the calculated average delay because the delay contribution of past measurements is greater than when half-life is set low. The metric-contribution parameter affects the weighting of this delay in the decision-maker’s performance calculations. Its value determines the mid-point of a curve that tells the decision-maker how strictly it should regard application delay in its decisions. A higher number for this value places greater importance on delay for example, an application is considered to be impacted more by a delay). It works in the same way as explained in Enterprise application model for application-delay. ● jitter min-buffer - sets the amount of buffer delay introduced by a client application, in milliseconds ● loss half-life - controls the smoothing factor of measured loss. A high value results in a smaller contribution of past loss events; conversely, a low value allows past loss events to have a greater influence on the decision-maker’s calculations. ● loss metric-contribution - affects the weighting of the loss percentage in the decision-maker’s performance calculations. Its value determines the mid-point of a curve that tells the decision-maker process how strictly it should regard this loss in its decisions. A higher number for this value places greater importance on loss for example, an application is considered to be affected more by packet loss). It works in the same way as explained in Enterprise application model for application-delay. ● metric - this is an internal parameter, do not change it. ● short-term-loss - measures the observed loss in the last window-width packets received. Also uses a threshold to compare against the number of losses in the window to determine if there is loss. Below threshold short-term-loss is 0, above threshold it equals num losses / window-width. For the application streaming model, application performance is derived from delay, jitter, and loss measurements. Loss measurements consist of both a long term loss average and a short term loss average, and the decision-maker uses the larger of the two along with the delay and jitter measurements for its ultimate routing decision (the resulting value is called the decision metric). Table 26 shows how the computed delay correlates with the decision metric and application performance, from best (*****) to worst (*), while Table 27 shows how the measured loss percentage correlates with the decision metric and application performance. 116 CNA Monitoring Guide, Version 4.1 Details on Application Models Table 26: Round Trip Time Affects Application Performance Delay (ms) Decision Metric Application Performance 0 - 125 0 - 100 ***** 126 - 250 101 - 200 **** 251 - 400 201 - 300 *** 401 - 800 301 - 500 ** > 800 501 - 100 * Table 27: Loss Affects Application Performance Loss (%) Decision Metric Application Performance 0 - 1.5 0 - 100 ***** 1.6 - 3.0 101 - 200 **** 3.1 - 5.0 201 - 300 *** 5.1 - 10 301 - 500 ** > 10 501 - 100 * For the streaming application model, delay is calculated from round trip time (RTT) and jitter (as opposed to the enterprise model where RTT and loss percentage are the factors for delay). The CNA system’s default values for RTT and loss are shown, but you can adjust the metric-contribution of each when needed, in the same way as explained in Enterprise application model for application-delay. Issue 3 May 2008 117 Decision Policies and Application Models Multimedia application model Use this model to optimize real-time applications on your subnet. Examples include voice over IP, streaming audio, and video conferencing applications. See Table 19 for examples of which application model is recommended for a particular application type. When using the multimedia application model, you must use either the traceprobe or ICMP measurement modes. If you can employ only TCP, then you should use the web application model because the multimedia model requires measurements of the loss and jitter characteristics in your subnet to a higher degree of precision than is possible with TCP. The following parameters control the behavior of the multimedia application model: ● delay - determines how the moving average of delay is calculated. The half-life parameter controls the smoothness of the moving average. A high value results in smaller changes in the calculated average delay because the delay contribution of past measurements is greater than when half-life is set low. The metric-contribution parameter affects the weighting of this delay in the decision-maker’s performance calculations. Its value determines the mid-point of a curve that tells the decision-maker how strictly it should regard application delay in its decisions. A higher number for this value places greater importance on delay (for example, an application is considered to be affected more by a delay). It works in the same way as explained in Enterprise application model for application-delay. ● jitter min-buffer - sets the amount of buffer delay introduced by a client application (for example, voice over IP, video conferences), in milliseconds. ● loss half-life - controls the smoothing factor of measured loss. A high value results in a smaller contribution of past loss events; conversely, a low value allows past loss events to have a greater influence on the decision-maker’s calculations. ● loss metric - contribution affects the weighting of the loss percentage in the decision-maker’s performance calculations. Its value determines the mid-point of a curve that tells the decision-maker process how strictly it should regard this loss in its decisions. A higher number for this value places greater importance on loss (for example, an application is considered to be affected more by packet loss). It works in the same way as explained in Enterprise application model for application-delay. ● metric - this is an internal parameter, do not change it. ● short-term-loss - measures the observed loss in the last window-width packets received. Also uses a threshold to compare against the number of losses in the window to determine if there is loss. Below threshold short-term-loss is 0, above threshold it equals num losses / window-width. For the application multimedia model, application performance is derived from delay, jitter, and loss measurements. Loss measurements consist of both a long term loss average and a short term loss average, and the decision-maker uses the larger of the two along with the delay and jitter measurements for its ultimate routing decision (the resulting value is called the decision metric). Table 31 shows how the computed delay correlates with the decision metric and 118 CNA Monitoring Guide, Version 4.1 Details on Application Models application performance, from best (*****) to worst (*), while Table 29 shows how the measured loss percentage correlates with the decision metric and application performance. For the multimedia application model, delay is calculated from round trip time (RTT) and jitter (as opposed to the enterprise model where RTT and loss percentage are the factors for delay). The CNA system’s default values for RTT and loss are shown, but you can adjust the metric-contribution of each when needed, in the same way as explained in Enterprise application model for application-delay. Table 28: Round Trip Time Affects Application Performance Delay (ms) Decision Metric Application Performance 0 - 125 0 - 100 ***** 126 - 250 101 - 200 **** 251 - 400 201 - 300 *** 401 - 800 301 - 500 ** > 800 501 - 100 * Table 29: Loss Affects Application Performance Loss (%) Decision Metric Application Performance 0 - 1.5 0 - 100 ***** 1.6 - 3.0 101 - 200 **** 3.1 - 5.0 201 - 300 *** 5.1 - 10 301 - 500 ** > 10 501 - 100 * Issue 3 May 2008 119 Decision Policies and Application Models Web application model Use the Web application model to optimize Web transactions on your subnet. If you use only TCP type probes, then you must use this model (or the other model, below). The following parameters control the behavior of the Web application model: ● delay - determines how the moving average of delay is calculated. The half-life parameter controls the smoothness of the delay moving average. A high value results in smaller changes in the calculated average delay because the delay contribution of past measurements is greater than when half-life is set low. This parameter does not affect decision-maker’s calculations, and is used only by the reporting module. ● hrtt - determines how handshake round trip time influences decision-maker’s calculations. The metric-contribution affects the weighting of the measured hrtt. Its value determines the mid-point of a curve that tells decision-maker how strictly it should regard measured hrtt in its decisions. A higher number for this value places greater importance on it for example, an application is considered to be affected more by longer handshake round trip times). ● loss decent-rate - this is an internal process, do not change it. ● loss half-life - controls the smoothing factor of measured loss. A high value results in a smaller contribution of past loss events. This parameter does not affect decision-maker’s calculations, and is used only by the reporting module. ● loss-metric-contribution - affects the weighting of the loss percentage in the decision-maker’s performance calculations. Its value determines the mid-point of a curve that tells the decision-maker process how strictly it should regard this loss in its decisions. A higher number for this value places greater importance on loss. It works in the same way as explained in Enterprise application model for application-delay ● short-term-loss - measures the observed loss in the last window-width packets received. Also uses a threshold to compare against the number of losses in the window to determine if there is loss. Below threshold short-term-loss is 0, above threshold it equals num losses / window-width. For the Web application model, application performance is derived from handshake round trip time (HRTT). Table 30 shows how HRTT correlates with the decision metric and application performance, from best (*****) to worst (*). The CNA system’s default values for HRTT are shown, but you can adjust the metric-contribution of HRTT when needed, in the same way as explained in the Enterprise application model for application-delay above. 120 CNA Monitoring Guide, Version 4.1 Details on Application Models Table 30: Round Trip Time Affects Application Performance Handshake Round Trip Time (ms) Decision Metric Application Performance 0 - 150 0 - 100 ***** 151 - 300 101 - 200 **** 301 - 500 201 - 300 *** 501 - 915 301 - 500 ** > 915 501 - 100 * Table 31: Round Trip Time Affects Application Performance Handshake Round Trip Time (ms) Decision Metric Application Performance 0 - 150 0 - 100 ***** 151 - 300 101 - 200 **** 301 - 500 201 - 300 *** 501 - 915 301 - 500 ** > 915 501 - 100 * Other application model Use the Other application model in cases where you do not want to use the Web model (you might, for example, wish to modify it for some special application needs, but you still want to keep the Web model for normal Web traffic), but where you are forced to use TCP type measurement. The following parameters control the behavior of the other application model: ● delay - determines how the moving average of delay is calculated. The half-life parameter controls the smoothness of the delay moving average. A high value results in smaller changes in the calculated average delay because the delay contribution of past measurements is greater than when half-life is set low. This parameter does not affect decision-maker’s calculations, and is used only by the reporting module. Issue 3 May 2008 121 Decision Policies and Application Models ● hrtt - determines how handshake round trip time influences decision-maker’s calculations. The metric-contribution affects the weighting of the measured hrtt. Its value determines the mid-point of a curve that tells decision-maker how strictly it should regard measured hrtt in its decisions. A higher number for this value places greater importance on it (for example, an application is considered to be affected more by longer handshake round trip times). ● loss half-life - controls the smoothing factor of measured loss. A high value results in a smaller contribution of past loss events. This parameter does not affect decision-maker’s calculations, and is used only by the reporting module. ● short-term-loss - measures the observed loss in the last window-width packets received. Also uses a threshold to compare against the number of losses in the window to determine if there is loss. Below threshold short-term-loss is 0, above threshold it equals num losses / window-width. For the other application model, application performance is derived from handshake round trip time (HRTT). Table 32 shows how HRTT correlates with the decision metric and application performance, from best (*****) to worst (*). The CNA system’s default values for HRTT are shown, but you can adjust the metric-contribution of HRTT when needed, in the same way as explained in the enterprise model for application-delay above. Table 32: Round Trip Time Affects Application Performance Handshake Round Trip Time (ms) Decision Metric Application Performance 0 - 150 0 - 100 ***** 151 - 300 101 - 200 **** 301 - 500 201 - 300 *** 501 - 915 301 - 500 ** > 915 501 - 100 * 122 CNA Monitoring Guide, Version 4.1 Index Index A aaa authentication login . . . . . . active probe . . . . . . . . . . . . active-measurement group . . . . . active-measurement name . . . . . active-measurement target . . . . . address rotation . . . . . . . . . . agent-based measurement active-measurement name . . . alarm name . . . . . . . . . . alarm-absolute disable . . . . . alarm-relative disable . . . . . . alarm-threshold-exceeded . . . . chatter assign . . . . . . . . . chatter join-hive . . . . . . . . chatter leave-hive . . . . . . . . chatter missed-heartbeats . . . . chatter node add . . . . . . . . chatter node split . . . . . . . . chatter remove-cna-agent . . . . cna-agent . . . . . . . . . . . measurement intra-zone disable . no cna-agent . . . . . . . . . . no zone . . . . . . . . . . . . resync chatter chapld . . . . . . resync chatter node-alias . . . . show chatter master-key . . . . show statistic chatter . . . . . . zone . . . . . . . . . . . . . . alarm name . . . . . . . . . . . . alarm-absolute disable . . . . . . . alarm-relative disable . . . . . . . alarm-threshold-exceeded . . . . . am-measurer . . . . . . . . . . . am-scheduler . . . . . . . . . . . application-model other . . . . . . application-model streaming . . . . application-model voice . . . . . . application-model web . . . . . . . C . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 . . 38 . 38, 47 . . 81 . . 47 . . .11 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81 . 89 . 86 . 87 . 88 . 65 . 65 . 66 . 66 . 95 . 95 . 69 . 74 . 82 . 75 . 73 . 67 . 96 . 65 . 96 . 71 . 89 . 86 . 87 . 88 . 50 . 50 . 121 . 115 . 114 . 120 B bgp . . . . . . . . . . . . . . . . . . . . . . . . .11 BGP4 . . . . . . . . . . . . . . . . . . . . . . 11, 12 chatter assign . . . . . . . . . . . . . . chatter join-hive . . . . . . . . . . . . . chatter leave-hive. . . . . . . . . . . . . chatter missed-heartbeats . . . . . . . . . chatter node add . . . . . . . . . . . . . chatter node split . . . . . . . . . . . . . chatter remove-cna-agent . . . . . . . . . clear active-measurements . . . . . . . . clear epc active-measurement endpoint . . clear epc prefix . . . . . . . . . . . . . . clear statistics . . . . . . . . . . . . . . cna-agent . . . . . . . . . . . . . . . . commands aaa authentication login . . . . . . . . active-measurement group . . . . . . . active-measurement target . . . . . . . clear active-measurements . . . . . . clear epc active-measurement endpoint . clear epc prefix . . . . . . . . . . . . clear statistics . . . . . . . . . . . . . enable . . . . . . . . . . . . . . . . epc bytes minimum . . . . . . . . . . epc bytes top . . . . . . . . . . . . . epc enable . . . . . . . . . . . . . . epc exclude-prefix . . . . . . . . . . . epc filter . . . . . . . . . . . . . . . epc last-update-within . . . . . . . . . epc source . . . . . . . . . . . . . . ip http server ssl on . . . . . . . . . . ip ssh on . . . . . . . . . . . . . . . ip telnet on . . . . . . . . . . . . . . radius-server host . . . . . . . . . . . resync epc filter . . . . . . . . . . . . show active-measurements . . . . . . show epc . . . . . . . . . . . . . . . show statistics . . . . . . . . . . . . tacacs-server host . . . . . . . . . . . username . . . . . . . . . . . . . . . vip active-measurement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65 65 66 66 95 95 69 49 13 13 50 74 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 47 47 49 13 13 50 27 16 16 16 16 16 16 16 29 27 27 25 16 46 13 46 25 25 44 D direct-server return . . . . . . . . . . . . . . . . . 11 DNS . . . . . . . . . . . . . . . . . . . . . . . . 11 Issue 3 May 2008 123 Index network address translator . . . . . . . . . . . . . 11 no cna-agent . . . . . . . . . . . . . . . . . . . . 75 no zone . . . . . . . . . . . . . . . . . . . . . . 73 E enable . . . . . . . EPC . . . . . . . . epc bytes minimum . epc bytes top . . . . epc enable . . . . . epc exclude-prefix . . epc filter . . . . . . . epc last-update-within epc source . . . . . epcserver . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 13 16 16 16 16 16 16 16 13 P PBR . . . . . . . . plug-in . . . . . . . policy based routing probe types icmp . . . . . . tcp . . . . . . . traceprobe . . . . . . . . . . . . . . . . . 11, 12 . . . . . . . . . . . . . . . . 10 . . . . . . . . . . . . . . 11, 12 . . . . . . . . . . . . . . . . 38 . . . . . . . . . . . . . . . . 38 . . . . . . . . . . . . . . . . 38 F R flow-monitor . . . . . . . . . . . . . . . . . . . . 13 full BGP feed . . . . . . . . . . . . . . . . . . . .11 radius-server host . . . resync chatter chapld . . resync chatter node-alias resync epc filter . . . . round-robin. . . . . . . G . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 67 96 16 11 secure shell . . . . . . . show active-measurements show chatter master-key . show epc . . . . . . . . show statistics . . . . . . show statistics chatter . . SPAN . . . . . . . . . . ssh . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 46 65 13 46 96 13 27 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 38 27 38 GSLB . . . . . . . . . . . . . . . . . . . . . . . .11 S H high availability . . . . . . . . . . . . . . . . . . .11 I icmp . . . . . . . ip http server ssl on ip ssh on . . . . . ip telnet on . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 29 27 27 T J Java . . . . . . . . . . . . . . . . . . . . . . . 10 tacacs-server host tcp . . . . . . . . telnet . . . . . . traceprobe . . . . . . . . . . . . . . . . . . . . L link group . . . . . . . . . . . . . . . . . . . . . 44 U username . . . . . . . . . . . . . . . . . . . . . 25 M measurement intra-zone disable . . . . . . . . . . 82 monitor . . . . . . . . . . . . . . . . . . . . . . 13 multi-homed . . . . . . . . . . . . . . . . . . . 11, 12 V N Z NAT. . . . . . . . . . . . . . . . . . . . . . . . .11 NetFlow . . . . . . . . . . . . . . . . . . . . . . 13 netflowd . . . . . . . . . . . . . . . . . . . . . . 13 zone . . . . . . . . . . . . . . . . . . . . . . . . 71 124 CNA Monitoring Guide, Version 4.1 vip active-measurement . . . . . . . . . . . . . . . 44