Quest Management Xtension Operations Manager 2007 Edition Cisco Monitoring Extension Guide Unleash the Power of System Center Operations Manager 2007 for Complex, Heterogeneous Environments Cisco Solutions Guide © 2009 Quest Software, Inc. ALL RIGHTS RESERVED. This guide contains proprietary information protected by copyright. The software described in this guide is furnished under a software license or nondisclosure agreement. This software may be used or copied only in accordance with the terms of the applicable agreement. No part of this guide may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying and recording for any purpose other than the purchaser's personal use without the written permission of Quest Software, Inc. If you have any questions regarding your potential use of this material, contact: Quest Software World Headquarters LEGAL Dept 5 Polaris Way Aliso Viejo, CA 92656 USA www.quest.com email: legal@quest.com Refer to our Web site for regional and international office information. TRADEMARKS Quest, Quest Software, the Quest Software logo, Aelita, Benchmark Factory, Big Brother, DataFactory, DeployDirector, ERDisk, Fastlane, Final, Foglight, Funnel Web, I/Watch, Imceda, InLook, InTrust, IT Dad, JClass, JProbe, LeccoTech, LiveReorg, NBSpool, NetBase, PerformaSure, PL/Vision, Quest Central, RAPS, SharePlex, Sitraka, SmartAlarm, Speed Change Manager, Speed Coefficient, Spotlight, SQL Firewall, SQL Impact, SQL LiteSpeed, SQL Navigator, SQLab, SQLab Tuner, SQLab Xpert, SQLGuardian, SQLProtector, SQL Watch, Stat, Stat!, Toad, T.O.A.D., Tag and Follow, Vintela, Virtual DBA, and XRT are trademarks and registered trademarks of Quest Software, Inc. in the United States of America and other countries. All other trademarks and registered trademarks used in this guide are property of their respective owners. Disclaimer The information in this document is provided in connection with Quest products. No license, express or implied, by estoppel or otherwise, to any intellectual property right is granted by this document or in connection with the sale of Quest products. EXCEPT AS SET FORTH IN QUEST'S TERMS AND CONDITIONS AS SPECIFIED IN THE LICENSE AGREEMENT FOR THIS PRODUCT, QUEST ASSUMES NO LIABILITY WHATSOEVER AND DISCLAIMS ANY EXPRESS, IMPLIED OR STATUTORY WARRANTY RELATING TO ITS PRODUCTS INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTY OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, OR NON-INFRINGEMENT. IN NO EVENT SHALL QUEST BE LIABLE FOR ANY DIRECT, INDIRECT, CONSEQUENTIAL, PUNITIVE, SPECIAL OR INCIDENTAL DAMAGES (INCLUDING, WITHOUT LIMITATION, DAMAGES FOR LOSS OF PROFITS, BUSINESS INTERRUPTION OR LOSS OF INFORMATION) ARISING OUT OF THE USE OR INABILITY TO USE THIS DOCUMENT, EVEN IF QUEST HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. Quest makes no representations or warranties with respect to the accuracy or completeness of the contents of this document and reserves the right to make changes to specifications and product descriptions at any time without notice. Quest does not make any commitment to update the information contained in this document. Quest® Management Xtension - Operations Manager 2007 Edition Cisco Monitoring Extension Guide Version: 7.0.0.11 Quest QMX Base Framework Version: 4.0.0.519 Updated – May 2010 Contents About this Guide .................................................................................................................................. 7 What’s New ....................................................................................................... 7 Upgrades for Current Users .............................................................................. 7 Conventions ...................................................................................................... 8 About Quest System Center Solutions Group ......................................................... 8 About Quest Software, Inc. ................................................................................. 8 Contacting Quest Software, Inc. ........................................................................... 9 Join the Community ........................................................................................ 9 Introducing QMX - Operations Manager 2007 ........................................................................................... 9 What is a Quest Extension? ................................................................................. 9 What is a Quest Agentless Solution? ..................................................................... 9 Quest Agentless Solutions Are Open Source .......................................................... 9 How Agentless Extensions Work ......................................................................... 10 Agent-Based Solutions ...................................................................................... 10 About Quest’s Cisco Monitoring Extension...............................................................................................11 Cisco SNMP (WATCHDOG job) Portion of the Cisco Extension ................................. 11 Cisco SNMP (TRAP job) Portion of the Cisco Extension .......................................... 12 Cisco CLI (PERFORMANCE job) Portion of Cisco Extension ..................................... 13 Installing the Cisco Extension ...............................................................................................................13 Customizing the Cisco SNMP (Trap) Portion of the Cisco Extension ............................................................15 Starting the Configuration Tool .......................................................................... 15 __AppNotes Tab............................................................................................... 17 Specifying Hosts to Monitor ............................................................................... 18 Cloning Hosts ............................................................................................... 19 __Hosts Tab .................................................................................................... 19 Select Checkbox ........................................................................................... 20 Disable Checkbox .......................................................................................... 20 DNS Name Boxes .......................................................................................... 20 Login Prompt Boxes ....................................................................................... 20 UserId Boxes ................................................................................................ 20 Password Prompt Boxes ................................................................................. 20 Login Password Boxes.................................................................................... 21 Port Boxes ................................................................................................... 21 Secure Connection Checkbox .......................................................................... 21 Enable Password Boxes .................................................................................. 21 Cisco Solutions Guide Trap, Read and Write Community Boxes .......................................................... 21 NetFlow Checkbox ......................................................................................... 21 _GlobalVariables Tab ........................................................................................ 23 Clone/Remove Checkbox ................................................................................ 23 Name/Value Pairs .......................................................................................... 23 AlertFilters Tab ................................................................................................ 26 Clone/Remove Checkbox ................................................................................ 26 Disable Checkbox .......................................................................................... 26 Alert Filter Box .............................................................................................. 27 Notes Box .................................................................................................... 27 CriticalInterfacesList Tab ................................................................................... 28 Clone/Remove Checkbox ................................................................................ 28 Disable Checkbox .......................................................................................... 28 Interface Text Box......................................................................................... 28 NetFlow Tab .................................................................................................... 30 OIDGets Tab.................................................................................................... 31 Select Checkbox ........................................................................................... 32 Disable Checkbox .......................................................................................... 32 Object Box ................................................................................................... 33 Counter Box ................................................................................................. 34 Comparator and Alert Threshold Checkboxes .................................................... 34 Graph Checkbox ........................................................................................... 35 Use Delta Checkbox....................................................................................... 35 Aggregate Drop-down Box ............................................................................. 35 Auto Resolve Checkbox .................................................................................. 35 Mib Name Box .............................................................................................. 35 SNMP Name Box ........................................................................................... 35 SNMP OID Box .............................................................................................. 35 PerformanceMetrics Tab .................................................................................... 37 Disable Checkbox .......................................................................................... 37 Name Box .................................................................................................... 37 Metric Category Box ...................................................................................... 38 Metric Name Box ........................................................................................... 38 Comparator and Alert Threshold Checkboxes .................................................... 38 Graph Checkbox ........................................................................................... 38 Use Delta Checkbox....................................................................................... 38 Page 4 Cisco Solutions Guide Aggregate Drop-down Box ............................................................................. 39 Auto Resolve Checkbox .................................................................................. 39 TableGets Tab ................................................................................................. 40 Select Checkbox ........................................................................................... 42 Disable Checkbox .......................................................................................... 42 Object Box ................................................................................................... 42 Counter Box ................................................................................................. 43 Comparator and Alert Threshold Checkboxes .................................................... 43 Graph Checkbox ........................................................................................... 43 Use Delta Checkbox....................................................................................... 43 Aggregate Drop-down Box ............................................................................. 43 Auto Resolve Checkbox .................................................................................. 44 Mib Name Box .............................................................................................. 44 Table Name Box ............................................................................................ 44 SNMP Name Box ........................................................................................... 44 Table Row Unique Identifier Box ..................................................................... 44 _VirtualAgentJobs Tab ...................................................................................... 46 Job Name ..................................................................................................... 46 Cisco Extension Options .......................................................................................................................48 Message Filter DB Option .................................................................................. 48 Autodiscovery Option ........................................................................................ 49 Clone VA Option ............................................................................................... 56 Refresh Option ................................................................................................. 56 Start All Option ................................................................................................ 56 Stop All Option ................................................................................................ 56 Version Option ................................................................................................. 56 Using the Trap Editor to Modify an Alert .................................................................................................57 Finding the Record that Matches the Alert ........................................................ 59 SNMP Trap Sample ........................................................................................... 60 Cisco Device Options ...........................................................................................................................61 Ping Option ..................................................................................................... 61 Nslookup Option .............................................................................................. 62 Trace Route Option .......................................................................................... 62 Test Connection Option ..................................................................................... 62 Telnet or SSH Option ........................................................................................ 62 Run Diagnostics Option ..................................................................................... 62 Show Log Option .............................................................................................. 62 Page 5 Cisco Solutions Guide Clear Log Option .............................................................................................. 62 Start or Stop Option ......................................................................................... 63 Cisco Trap and Performance Screen Shots ..............................................................................................64 Alert View ..........................................................................................................................................64 Performance View ............................................................................................ 65 Diagram View .................................................................................................. 66 Generic Reporting View ..................................................................................... 66 Troubleshooting Installation and Operation Problems ...............................................................................67 Check Software Requirements ........................................................................... 67 Basic Troubleshooting Tips ................................................................................ 68 Windows SNMP and Cisco SNMP Setup Examples ................................................. 69 Performance Issues .......................................................................................... 71 If You Call Support ........................................................................................... 71 Page 6 Cisco Solutions Guide About this Guide This guide is intended for Windows and Cisco system administrators who will be installing Quest® Management Xtensions - Operations Manager 2007 Edition (QMX Operations Manager 2007) Cisco Monitoring Extension. By following the instructions presented in this guide, a system administrator will be able to manage all Alerts and Performance data produced for Cisco devices in the Microsoft System Center Operations Manager 2007 Console. What’s New Please be sure to read the tab “AppNotes” for the Cisco solution inside the QMX GUI/Config Tool. This latest iteration of the Cisco Monitoring Extension: Added mixed-mode Test Connection (test CLI, SNMP, or both). Added new Auto-Discovery methods; Nmap for speed. CLI for CLI only runs, SNMP for SNMP only runs. Quest suggests you install Nmap (freeware) from Inscure.org, or CNET, etc. And used Nmap for Auto-Discovery for the Cisco solutions. Added support for SAN device bandwidth measurements (for Cisco SAN running either SANOS or NXOS) that previously existed for other Cisco OS’s. Netflow ‘top-talkers’ support was added. The ability to choose the method of data extraction has changed. The default is mixed mode SNMP/CLI. Optionally you can select CLI ONLY or SNMP ONLY (in which case you would use CLI or SNMP Auto-Discovery only, if using AutoDiscovery at all). For THIS release running SNMP only means forgoing Netflow “top-talkers” (as well as “3750 stacked switch member failures and ASA primary failures). These issues will be resolved in a future release wherein running CLI only or SNMP only will produce the exact same results. CLI ONLY has one caveat for this release – the “WATCHDOG” job that keeps the State View in sync uses SNMP (if you don’t care about the State View, this is not an issue for CLI ONLY users). This will be selectable in a future release. For the time being, Quest suggests you run in the “default SNMP/CLI” mode (which is selectable from the GUI/Config Tool). All “sub-folders” that existed OOTB in prior releases have been removed. The Cisco folder now stands alone; this at the request of numerous customers that need to separate their devices by customer, Vlan, geography,and so on, rather than by device OS. Auto-resolution of alerts as well as delta and aggregate processing are now part of the solution. Upgrades for Current Users Please read “What’s New” that precedes. If you have an older version of the Quest Base Framework or Cisco Monitoring Extension, you need to upgrade them to the latest versions if you require ACE Load Balancer support or you desire mixed-mode Auto-Discovery support. Page 7 Cisco Solutions Guide You must upgrade the Quest Base Framework to 4.0.1 in order to get full common routine functionality out of the updated Cisco solution. If you are running the QMX Large Scale SNMP Solution against your Cisco devices – that needs to be upgraded as well. To verify your version of the Quest Base Framework 1. In the Configuration Tool’s Help Menu, choose the Base Framework Version option. 2. Compare your version with the latest Base Framework version at: http://www.quest.com/quest-management-xtensions-operationsmanager/release-information.aspx. Note that 4.0.1.? is newer than any 4.0.0.? release. 3. Upgrade the Quest Base Framework if you do not have the latest version. To verify your version of the Cisco Monitoring Extension 1. In the Configuration Tool, right-click Cisco in the navigation tree to open the context menu. 2. Select the Version option to see what version of the Cisco Monitoring Extension you have. This option opens the Version_History.txt file. The only version you see (or the one at the top) is what is currently installed on your system (any version information below that is for historical purposes only). Note: If you do not see the Version option, you have an old version of the Quest Base Framework and Cisco solution and need to upgrade it. Conventions Quest Software, Inc. products support a number of different implementations of UNIXlike operating systems. The term “non-Windows” throughout this document is used to encompass all QMX solutions that do not relate specifically to Windows monitoring (this therefore includes Cisco). (See http://managementextensions.org/monitorextensions.jspa for a list of all platforms supported by QMX Operations Manager 2007.) About Quest System Center Solutions Group With a comprehensive set of solutions that extend the powerful capabilities of the Microsoft System Center family to heterogeneous environments, Quest Software enables IT professionals to leverage System Center as the single, end-to-end platform for managing physical and virtual IT environments across data centers, desktops and devices. For more information on Quest’s System Center Solutions group, please visit: http://www.quest.com/system-center/ About Quest Software, Inc. Now more than ever, organizations need to work smart and improve efficiency. Quest Software creates and supports smart systems management products—helping our customers solve everyday IT challenges faster and easier. Visit www.quest.com for more information. Page 8 Cisco Solutions Guide Contacting Quest Software, Inc. Phone 949.754.8000 (United States and Canada) Email info@quest.com Mail Quest Software, Inc. World Headquarters 5 Polaris Way Aliso Viejo, CA 92656 USA Web site www.quest.com SupportLink www.quest.com/support Email to support@quest.com Please refer to our Web site for regional and international office information. Join the Community Get the latest downloads, How To’s, and FAQs; or, post questions to and join discussions with the Quest Management Xtensions Team and other community members at: http://www.management-extensions.org/index.jspa Introducing QMX - Operations Manager 2007 QMX - Operations Manager 2007 is the Quest Software, Inc. solution that extends the power of Microsoft System Center Operations Manager 2007 to non-Windows systems, devices, and applications using agentless extensions. What is a Quest Extension? Extensions are virtual agents that extend the functionality of Operations Manager 2007 to non-Windows resources. An Extension is made up of VBScript(s) and/or JScript(s) that call the Quest Base Framework APIs in order to perform the desired actions, such as SSH’ing to a device, issuing a command, receiving SNMP Traps, sending SNMP Gets, talking to the Microsoft MCF, and so forth. What is a Quest Agentless Solution? An “agentless” solution means that no software resides on the target platform like agent-based solutions. Thus, to install them, you are not required to distribute any software to your client machines. And, because Quest’s “agentless” solutions are 100% Windows code and scripts, with basic Windows-based VBScript or Jscript experience, you can easily modify or extend them. Quest Agentless Solutions Are Open Source About 95% of the Quest solutions are written using Microsoft’s JScript. The other 5% use VBScript. This means you can modify or extend the solutions to add specific requirements that may not be met “out-of-the-box”. Page 9 Cisco Solutions Guide How Agentless Extensions Work Think of these solutions as a programmed human being that follows orders like a robot. Extensions (that is, the script code) can login to a system, type commands, read and analyze responses, and take actions. They have built-in processes to create WMI Events and Alerts, or create performance records. The rest of the job is up to Operations Manager 2007. Extensions leverage a common software component called the Quest Base Framework. The Quest Base Framework is code that offers services such as fault-tolerance, scheduling, task management, and so forth to all of the Extensions regardless of the hardware platform (that is, 32-bit or 64-bit). Extensions rely on the services and functions provided by this software component. Think of it as the SDK or Application Programming Interface (API) that allows the solutions to issue commands, read responses, and so forth. The Quest solutions support SNMP version 1 and version 2: GET, SET, and the receipt of Traps (SNMP V1) and Notification-Types (SNMP V2). At a minimum, all Quest SNMP solutions can: poll intermittently to ensure there is connectivity between the computer where the Quest solution resides receive inbound SNMP Traps and convert them into WMI format which allows QMX – Operations Manager 2007 to present the information as Operations Manager 2007 Alerts create Alerts and/or Performance records from user-specified OIDs QMX – Operations Manager 2007 is unique in that it does not require MIB compiles, MOFs, namespace instantiation for the device type, application, and so forth, or user-built scripts to “tear apart” inbound Traps. SNMP Extensions take care of those types of solutions much like the Telnet/SSH Extensions. Many QMX – Operations Manager 2007 SNMP solutions also handle specific SNMP GETs. They can all create Operations Manager 2007 alerts and/or performance records. The type of solution Quest builds depends on the capability of the device, application, operating system, solution requirements, and so forth. You determine what information you need to obtain and the best way to get it: Telnet/SSH or SNMP, or a combination of the two, as is the case with Cisco. Quest also provides some WEB-services or API-based solutions. The Quest F5 solution is based on WEB-services (not Telnet/SSH or SNMP) that provide in-depth information about the F5 devices. The Quest Remedy Connector is an example of an API-based solution. QMX – Operations Manager 2007 can “speak” to almost any device, application, operating system, and so forth because it speaks a “languages” they all understand. The only possible exception is a hardened system without a Quest-supported GUI interface. Agent-Based Solutions Quest also provides two agent-based solutions. AS400 Extension for MOM/Operations Manager 2007 MVS/ZOS Extension for MOM/Operations Manager 2007 Page 10 Cisco Solutions Guide Unix and Linux solutions (as of Base Framework 4.0.1.?) for ConfigMgr only (as well as agentless for the same OSs – you choose). About Quest’s Cisco Monitoring Extension Quest currently has a single Cisco Monitoring Extension for both the Command Line Interface (CLI) and SNMP APIs. That is, Telnet or SSH and/or SNMP, whichever your Cisco devices support (meaning either one or both). Selecting SNMP ONLY at run time has some (minor) limitations – see the AppNotes tab in the Cisco solution. Some sites told us that they own certain Cisco equipment that other companies manage for them – but they want to know ‘how the devices are performing’. Since the management companies frequently deny CLI (Telnet and/or SSH) access, SNMP ONLY is an alternative. Quest recommends you run in ‘mixed-mode’ CLI/SNMP (both at the same time) if it’s at all pratical on your site. The Cisco Monitoring Extension is available from the Quest Community Forum at: http://www.management-extensions.org/index.jspa (Go to the SCOM~Downloads link and look in the Network section to find the Cisco Monitoring Extension – or just click CTRL+f and type “Cisco” in the ‘find box’.) QMX – Operations Manager 2007 extends the Microsoft Windows Management Instrumentation (WMI) so that your Cisco devices appear to Operations Manager 2007 as if they are Windows devices. The QMX – Operations Manager 2007 methods and properties and “device” instantiations (that is, information that represents the target Cisco devices) are all part of WMI. When you start WMI, its properties and methods (and Quest’s properties and methods) are all immediately available. QMX – Operations Manager 2007 is not a parallel solution with its own console. You can write event, alert, and performance rules exactly like you write them for Windows devices! You can use standard Microsoft Reporting to create reports for Cisco devices the same way you do for Windows device. Cisco makes descriptions for most variables available by means of their MIBs and QMX stores those descriptions and variable references in the Access database that comes along with the Cisco solution download. Because QMX – Operations Manager 2007 translates inbound Traps into Operations Manager 2007 Alert format, including the descriptive text of the varbinds, descriptions for most variables appear in the Operations Manager 2007 Alert. Since the Extension is mostly JScript (and some VBScript) it is easily extensible by anyone that knows basic Windows Script Host (WSH) scripting. You never put anything on the Cisco device. Any changes you make to the Extension script are immediately available; no software distribution is required like with agent-based solutions where each modification requires you to upload the changes. This also means the solutions are very versatile when it comes to custom work. There is no “going back to the factory” for “next year’s rollout” with the Quest “agentless” solutions. Cisco SNMP (WATCHDOG job) Portion of the Cisco Extension The Cisco SNMP (WATCHDOG) portion of the Cisco Monitoring Extension polls to ensure connectivity between the computer on which the Cisco Monitoring Extension is installed Page 11 Cisco Solutions Guide and the target device (the Cisco Router, Switch, Firewall, Load Balancer, and so forth). If the ongoing connectivity test fails, it creates an Operations Manager 2007 Alert to inform you of the situation so you can pro-actively take corrective action rather than waiting for a telephone call about the problem. For instance a problem might occur at 2:00 a.m. that goes unnoticed until the first user arrives at work at 7:00 a.m., a 5-hour troubleshooting window that has been lost. The Cisco Monitoring Extension can spot and correct a problem before it affects even a single user. Cisco SNMP (TRAP job) Portion of the Cisco Extension Quest has one database that works for all Cisco equipment. By default, Operations Manager 2007 converts all Traps from all Cisco devices to alerts using a single database (either Access or SQL) created by Quest specifically for the Cisco device types. It has every potential Trap from every potential Cisco device type (IOS/CATOS/NXOS/etc. level is not a consideration). If a Trap never occurs, no Operations Manager 2007 Alert is created. For example, perhaps you have PIX firewalls today. All the ASA firewall Trap information stored in the database is useless because you do not have a device that can produce that Trap. All you have done is use up a little bit of disk space. And, if you switch from PIX to ASA, the Cisco Monitoring Extension will continue to work without interruption. The Cisco Monitoring Extension database is built for “one size fits all” for all Cisco device types and equipment – everything from a 16xx desktop router (2 ports) to a 65xx Core Router/Switch, such as NXOS with x ports – just count up all the devices and multiply! Traps (and Notifications) are the critical events Cisco believes you need to be notified about. Cisco Monitoring Extension simply relays the information to you by means of Operations Manager 2007 Alerts. SNMP Traps are converted to WMI Events, WMI Events cause the associated QMX Alerting rule to run. The QMX Alerting rule simply creates an Alert for every QMX WMI based event (which obviously includes the QMX Cisco solution). SNMP Gets for specific OIDS (stand-alone or within a MIB table) are handled generically. The Cisco Monitoring Extension ships with disabled samples of each under the OIDGets and TableGets tabs. You may enable them or add your own “SNMP MIB requests for OIDs/Tables” and specify the conditions for your solution. For example you may define what returned condition causes an alert, if any; performance records you want to produce; or, whether you want to process the information based on Deltas and/or Aggregates. Finally, you can auto-resolve threshold alerts. For example, if the CPU utilization goes over 90% and creates an alert, you can set the Cisco Monitoring Extension to auto-resolve the alert for you. There is no standard concept within the SNMP TRAPs design for “resolution TRAPs”. For example, there is no “overheated device TRAP” along with an “overheated device is no longer in danger” TRAP. The logic to handle this for SNMP TRAPs would be very cumbersome. But, TRAPs can have knowledge base “text” to inform the end user of the condition. Keep in mind that since the Extensions are open-source scripts, you can modify them for your environment. Page 12 Cisco Solutions Guide The Cisco SNMP portion of the Cisco Monitoring Extension writes the selected performance data to Operations Manager SQL databases in the standard format. This allows you to utilize standard Operations Manager 2007 reporting for your operation. For example, you can obtain data about your Cisco network-based operational status in real time (Performance Views) or “after-the-fact” (Reporting). The Cisco SNMP portion of the Cisco Monitoring Extension allows you to obtain any available OID and, optionally, create Alerts and/or write Performance data. Cisco CLI (PERFORMANCE job) Portion of Cisco Extension The Cisco CLI (Performance) portion of the Cisco Monitoring Extension allows you to configure Performance thresholds (via the PerformanceMetrics tab) for specific Cisco metrics, giving you the flexibility you need to control the thresholds that make sense for your operations. For example, 70% CPU might be critical for you while 95% is the critical threshold for another customer. The metrics are fixed as delilvered out-of-the-box. You cannot just “add one” without adding the code to process it! (Well, you can, but you won’t get any results). You can opt to use the defaults which alert and create performance data for all devices and interfaces; or you can customize the Cisco Monitoring Extension to alert and report only on those devices or interfaces that are of concern to your operation (the CriticalInterfaceList tab). The Cisco PerformanceMetrics tab metrics can be obtained by CLI or SNMP (whether CLI and/or SNMP is used is controlled by a selectable setting in the __GlobalVariables tab (see variable g_strProtocol). The Cisco (Performance) portion of the Cisco Monitoring Extension writes the selected performance data to the Operations Manager SQL database in the standard format. This allows you to utilize standard Operations Manager 2007 reporting for your operation. For example, you can obtain data about your Cisco network-based operational status in real time (Performance Views) or “after-the-fact” (Reporting). The Cisco (Performance) portion of the Cisco Monitoring Extension allows you to obtain CPU, Memory, Interface throughput, and Interface errors. For QMX/OpsMgr performance reasons, keep in mind that a CPU performance record is ‘one record’ per run cycle, whereas an ‘inbound errors’ performance record would be one record per interface per device (that’s a lot more than ‘one record’ as with the CPU metric). In other words, it’s a balancing act if you have limited CPU/Memory on the QMX box (you can, of course, always get a ‘bigger box’ if you find that you require more metrics than the current box can handle) or, spread the load over multiple QMX boxes. Installing the Cisco Extension To install the Cisco Monitoring Extension 1. Go to the Community Forum Web site at: http://www.managementextensions.org/index.jspa 2. Click SCOM~DOWNLOADS. Page 13 Cisco Solutions Guide 3. Scroll down to the Network section and click Cisco or use CTRL+f and enter “Cisco” in the ‘Find box’. 4. Do the 4 steps in the order listed Click Save to download the Extension installation file to a directory on the file system of the Windows server where you installed the Quest Base Framework or just click Run 5. Double-click the .msi file to start the Extension installation setup wizard. The Wizard will lead you through the process of installing the Cisco Monitoring Extension. Once you install the Extension you are ready to configure it. Page 14 Cisco Solutions Guide Customizing the Cisco SNMP (Trap) Portion of the Cisco Extension Once you have installed the Extensions, you can customize them using the Configuration Tool. Starting the Configuration Tool To start the Configuration Tool 1. Please note that the examples in this document assume Base Framework 4.0.1.? (? = 33 was released on March 18th, 2011).From the Start menu, navigate to All Programs | Quest Software | WMI Providers | non-Windows | QMX Administration Console. Page 15 Cisco Solutions Guide Note: When you mouse-over any blue text or buttons, “flyover” help displays for your convenience. Use Ctrl-Z to turn Tooltips off. 2. In the Configuration Tool, navigate to SCOM | MOM. 3. Select the Cisco node. Page 16 Cisco Solutions Guide __AppNotes Tab Quest recommends that you read the notes on that page before doing any customization of the other tabs that appear; they are: __Hosts _GlobalVariables AlertFilter CriticalInterfaceList NetFlow OIDGets PerformanceMetrics TableGets __VirtualAgentJobs Page 17 Cisco Solutions Guide Specifying Hosts to Monitor Specifying a host to monitor is your first step in configuring an Extension. The __Host tab displays the list of hosts (systems and/or devices) that Cisco Monitoring Extension is capable of monitoring. When you first install Cisco Monitoring Extension, by default it lists sample Host entries with a default status of “Disabled”. Quest provides these samples for you to use as a starting place. Clone the ones that most closely match your environment requirements. (Look in the Notes field for some hints!) If you do not need these entries, Quest recommends that you leave them disabled for possible future cloning needs. Page 18 Cisco Solutions Guide To add hosts, clone a sample host and modify it to your requirements. Cloning Hosts To clone a host To add a host, clone the default host and modify it to your specifications. 1. Select one or more hosts to clone. Note: Use the Select (or Clone/Remove) checkbox in conjunction with the CLONE or REMOVE buttons to either create a new (cloned) entry or delete (remove) an existing entry. 2. Click CLONE. 3. Click SAVE. Once you have cloned a host entry you are ready to customize the fields on the __Hosts tab. __Hosts Tab The values in the fields on the __Hosts tab allow the Extension to connect to the host (system, device, or application) you want to monitor. The __Hosts tab has the following fields: Select (checkbox) Disable (checkbox) DNS name Login Prompt UserID Password Prompt Login Password Port Secure Connection (checkbox) Page 19 Cisco Solutions Guide Enable Password1 Trap Community Read Community Write Community NetFlow (checkbox) The following describes the fields in the __Hosts tab. Select Checkbox Use the Select checkbox in conjunction with the CLONE or REMOVE buttons to either create a new (cloned) entry or delete (remove) an existing entry. Disable Checkbox Use the Disable checkbox to prevent the auto-start function from starting the Extensions against the selected devices. Note that any disabled device entry appears on the navigation tree with a pound sign (#) in front of the DNS name. DNS Name Boxes Each Cisco device you want to monitor requires that you supply the DNS Name, not the FQDN or IP address. If you do not have DNS available, for immediate availability, make up a name and put it in the Windows /etc/hosts file; otherwise, you can add it by means of DNS management to your Forwardees list. Login Prompt Boxes Enter a Login Prompt2 so the Cisco Monitoring Extension knows what to look for on the screen once the Telnet/SSH connection is started. UserId Boxes Enter the UserId so the Cisco Monitoring Extension can respond to the Login Prompt. Note: To encrypt the UserId or Password, see Encrypting UserIDs and Password. Password Prompt Boxes Enter the Password Prompt so the Cisco Monitoring Extension knows what to look for on the screen once the Telnet/SSH connection is started. 1 The “Enable” password is only required for PIX and ASA devices or a device where standard commands (pager, terminal, show, etc.) are restricted (and thus won’t work in ‘non-enabled’ mode). Some Cisco devices are set up without a Login Prompt because the device responds immediately to a Password Prompt. In that case, leave “ssword” as the Login Prompt and “NOUSERID” in the UserId box. The Cisco device will send the Password Prompt twice, but it will still result in a successful login. The default Login Prompt is “partial text” of the Login (or username) Prompt since some devices use initial capitals and others do not. Log in manually to any device following this logic: Telnet or SSH to the DNS name, look for the Login Prompt text (which might be “ssword”) on the screen, respond with the login UserId, look for the Password Prompt on the screen, and respond with the Login Password. You are logged in. 2 Page 20 Cisco Solutions Guide Login Password Boxes Enter the Login Password so the Cisco Monitoring Extension can respond to the Password Prompt. Port Boxes Enter the port number you want to use to connect to the non-Windows host system. Typically Telnet uses port 23; while SSH uses port 22. Secure Connection Checkbox Unselect the Secure Connection checkbox to use Telnet; select the Secure Connection checkbox to use SSH. Enable Password Boxes Enter the Enable Password to enable the Cisco Monitoring Extension to run commands that are sometimes “locked down” on certain accounts (something as simple as “show int”). Note: The Enable Password3 must be encrypted. No clear text is allowed in this field! (See Encrypting UserIDs and Password.) Trap, Read and Write Community Boxes To receive the device Traps and SNMP Get data that will be converted to Operations Manager 2007 Alerts, the Cisco Monitoring Extension requires the Trap Community name and the Read Community name (if the NetFlow checkbox is set, the Write Community name is also required). The SNMP community names are case sensitive and must match the names contained in the Windows SNMP Service’s Security tab4. If you opt to use Community Name restrictions in the Cisco device configuration on the device itself and in the Configuration Tool tabs, it is best to ask your Cisco Administrator for the proper case sensitive names. You must set up the Cisco device to THROW Traps to the IP Address(es) of the Window system(s) device(s) where the Cisco Monitoring Extension is installed. You must also set up the Windows SNMP Service (where QMX is running) to accept SNMP GETs and allow SNMP Sets. NetFlow Checkbox Select the NetFlow Checkbox if a) you want NetFlow data from the Cisco device and b) the Cisco device supports NetFlow CLI “top-talkers”. Be sure to check the NetFlow tab if you select the checkbox as the tab contains the threshold Value that indicates the minimum number of packets transmitted that will cause a “top-talkers” alert. The NetFlow packet counter that is compared to “Value” resets to zero (0) at the start of each poll cycle. The solution ships with a “sample file name” so you can see the “expected format” for an encrypted file. It is file: C:\Program Files\eXc Software\WMI Providers\nonWindows\Virtual Agent Library\MOM\CiscoPerf\ enablepassword.txt. Note that “file:” precedes the actual location of the encrypted file. “file:” signals the solution that it has received an encrypted file and must decrypt the contents before proceeding. 3 4 See Windows SNMP and Cisco SNMP Setup Examples. Page 21 Cisco Solutions Guide The poll cycle runs every uint32RunEveryXSeconds number of seconds (you will find this setting in the__VirtualAgentJobs tab to the right of the PERFORMANCE job name. The default is 600 seconds (10 minutes). If you make any changes to any fields in the tab, click SAVE, then stop and restart the Extensions for the changes to take effect. Naming Schema If you are prompted for a “password” or “Password” when you issue a “Telnet” or “SSH” command, set the Login Prompt to “ssword”. That is, leave off the first letter. This is to accommodate the fact that some devices use Initial Capitals and others do not. If you are prompted for anything else, set the Login Prompt to whatever word it prompted you for, less the first character. For example, if you open a DOS command shell and enter you “Telnet CiscoRouter” (assuming you have a device named CiscoRouter) and it returns “login” or “Login”, enter “ogin” as the Login Prompt. If the Telnet command returns “Username” or “username”, enter “sername” as the Login Prompt. Then, set the proper UserId and supply that name, pre-pended with “file:C:\” (assuming “C:\” is where it is located) in the UserID field. Encrypting UserIDs and Password Quest Software recommends that you encrypt the user IDs and passwords for use with the Cisco Monitoring Extension whether for non-Windows systems or devices or for interacting with Operations Manager 2007. To encrypt user IDs or passwords 1. From the Configuration Tool Tools menu, select Encryption. 2. Enter the Text to encrypt and specify a directory path and filename in which to store the encrypted text and click OK. 3. Click the small light blue button to the right of the file name to specify an encrypted text file instead of clear text. 4. Open the encrypted file. 5. Modify the descriptive text in the Notes field as desired. Page 22 Cisco Solutions Guide _GlobalVariables Tab The _GlobalVariables tab contains entries that allow you to alter the default operating environment. The GlobalVariable tab has the following fields: Clone/Remove Name Value Clone/Remove Checkbox Use the Clone/Remove checkbox in conjunction with the CLONE or REMOVE buttons to either create a new (cloned) entry or delete (remove) an existing entry. To create a new entry or delete an entry 1. Select the entry(s). 2. Click CLONE or REMOVE. Name/Value Pairs The Name box specifies the name of the global variable. Note: You use this variable in the Virtual Agent job logic. g_boolNoValuesInAlerts Select the Value box to enable this variable. Note: it suppresses the portion of alert text that would thwart the value of the Operations Manager 2007 “Repeat Count” field. If you do not enable this variable, it uses the full alert text. This may not be desirable if the same alert repeats frequently but with ever changing values. For example, CPU is 95.4%, CPU is 97.3%, CPU is 98.1%. Quest recommends running with this checkbox set. Page 23 Cisco Solutions Guide g_boolShowCiscoCommandOutput Select the Value box to produce additional diagnostic output in the DebugLog. If you encounter an issue, it is likely that support will ask for the DebugLog. If some of the required data is not there, you may be asked to set this checkbox and reproduce the problem. Quest recommends running with this checkbox set. g_strSNMPVersion Set the Value to 2 to use version 2 for SNMP GETs. Note: It automatically falls back to version 1 if version 2 Gets fail. 8 g_strSNMPGetSetPort Set the Value to use port 161, the industry standard, unless your site does not use port 161 for SNMP Gets/Sets. Otherwise, set this value to any valid UDP port number. Note: This is not the TRAP PORT which is always port 162. g_strWatchdogCLIOrSNMP This value defaults to SNMP. In a future release you will be able to set it to CLI if you want to verify device connectivity via connect/login vs. SNMP GET. If you set it to anything other the SNMP (for THIS release) it is ignored and SNMP is used anyway. See g_strProtocol below regarding the effect on the State View. g_strProtocol The default is SNMP/CLI. This means you allow both CLI and SNMP access to the device. This will get you the most comprehensive results. SNMP or CLI (only) are for sites where only one type of access is allowed. For THIS release running SNMP only means forgoing Netflow “top-talkers” as well as “3750 stacked switch member failures and ASA primary failures. These issues will be resolved in a future release wherein running CLI only or SNMP only will produce the exact same results. CLI ONLY has one caveat for this release – the “WATCHDOG” job that keeps the State View in sync uses SNMP (if you don’t care about the State View). This is not an issue for CLI ONLY users. This will be selectable in a future release. For the time being, Quest suggests you run in the “default SNMP/CLI” mode g_boolSNMPUseSQL Normally this box should not be checkmarked, and thus g_strSNMPSQLServerName and g_strSNMPSQLDataBaseName should be left blank). Select the Value box to enable the use of an SQL database (discussed in the downloadable QMX Installation doc (note: it works, but using SQL instead of Access is no longer supported). Quest therefore recommends using 32 or 64-bit Access (and thus the shipped .mdb file). g_strSNMPSQLServerName Supply the name of the Server that houses the SQL database (see preceding note). g_strSNMPSQLDataBaseName Supply the name of the DataBase that houses the SQL database (see preceding note). Page 24 Cisco Solutions Guide If you make any changes to any fields in the tab, click SAVE, then stop and restart the Extensions for the changes to take effect. Page 25 Cisco Solutions Guide AlertFilters Tab The Alert Filters tab contains entries that you can use to prevent Alerts from showing up in the Operations console. The AlertFilters tab has the following fields: Clone/Remove Disable Alert Filter Notes Clone/Remove Checkbox Use the Clone/Remove checkbox in conjunction with the CLONE or REMOVE buttons to either create a new (cloned) entry or delete (remove) an existing entry. To create a new entry or delete an entry 1. Select the entry(s). 2. Click CLONE or REMOVE. Note: If this doesn’t appear to work, click SAVE first, then click CLONE or REMOVE. Disable Checkbox Use the Disable checkbox to “ignore” the contents (string or regular expression) in the Alert Filter. For example, if you want to report on every device or interface, select the Disable check box for all the entries. This will eliminate all devices from the exclusion list because they would all be disabled. Note: When you limit device or interface alerting in this manner, you do not limit performance reporting since that is under control of the Graph checkbox that appears in the PerformanceMetrics tab. (See Graph Checkbox.) Page 26 Cisco Solutions Guide Alert Filter Box Enter Alert Filters to suppress the generation of an alert. Alert Filters contain text strings or regular expressions. If the Cisco (Performance) portion of the Cisco Monitoring Extension detects that an Alert Filter matches the alert description, it suppresses an alert from appearing in the Operations Console. The sample alert, “tty trap signifies”, is enabled intentionally. This is because the Cisco extension generates this alert when if it uses the CLI protocol. By default it runs every 600 seconds, which can be annoying. The “tty trap signifies” Alert Filter suppresses the alert. “Null0” is enabled because it appears at the back of TableGet tab tables and is not actually an “entry” in the table of interest – it’s an internal “end marker” – no need to display that meaningless marker repeatedly as an alert. Note: If you have a large Cisco environment with only a few interfaces per device that you want to alert on and create performance data for, see the CriticalInterfacesList Tab. Notes Box The Notes are a freeform area that you can use at your discretion. If you make any changes to any fields in the tab, click SAVE, then stop and restart the Extensions for the changes to take effect. Page 27 Cisco Solutions Guide CriticalInterfacesList Tab The CriticalInterfaceList tab has the following fields: Clone/Remove Disable Interface Text (match means alert/write performance records) Notes Clone/Remove Checkbox Use the Clone/Remove checkbox in conjunction with the CLONE or REMOVE buttons to either create a new (cloned) entry or delete (remove) an existing entry. To create a new entry or delete an entry 1. Select the entry(s). 2. Click CLONE or REMOVE. Note: If this does not appear to work, click SAVE first, then click CLONE or REMOVE. Disable Checkbox Use the Disable checkbox to “ignore” the contents (string or regular expression) in the Interface Text box. If you want to report on every device or interface, select the Disable check box for all the entries. This will eliminate all devices from the exclusion list because they would all be disabled. This allows for backward compatibility for existing customers that are upgrading this Extension. Note: If you limit device or interface alerting in this manner, you do not limit performance reporting since that is under control of the Graph checkbox that appears in the PerformanceMetrics Tab. Interface Text Box Enter the string or regular expression you want the Cisco CLI (Performance) portion of the Cisco Monitoring Extension to match in the Interface Text box. This allows you to Page 28 Cisco Solutions Guide limit the devices or interfaces included for alerting and performance data. For example, you may want to create alerts or performance metrics only on uplinks or on critical workstations. Note: If you enable even a single entry (by unselecting the Disable checkbox) you defeat the default of reporting ALL devices and ALL interfaces. You must include every device/interface you want to monitor if you want to use the CriticalInterfacesList Tab even if you want to monitor two interfaces on one Switch and ALL the interfaces on another switch. You might think that if the device is not in the list it should have all its interfaces reported. The problem with that logic is that the Interface Text does not necessarily have to contain the device name. For example, if all your Switches use interface GigabitEthernet2/0 for an uplink there is no need to include the device name. Limiting the devices or interfaces included for alerting and performance data can save a tremendous amount of I/O to the Operations Manager 2007 databases. For example, if you opt for 2 interfaces per 48 ports (interface switches), you can reduce the I/O by approximately 96%. (See the PerformanceMetrics Tab for additional information regarding performance metrics). If you make any changes to any fields in the tab, click SAVE, then stop and restart the Extensions for the changes to take effect. Page 29 Cisco Solutions Guide NetFlow Tab To process NetFlow information you must enable the PERFORMANCE Run checkbox in the __VirtualAgentJobs tab g_intNetFlowPacketsThreshold This value defaults to 10000 (packets). Select the NetFlow Checkbox in the __Hosts tab if a) you want NetFlow data from the Cisco device and b) the Cisco device support NetFlow CLI “top-talkers”. Be sure to set a reasonable threshold Value (for your site) that indicates the minimum number of packets transmitted that will cause a “top-talkers” alert. The NetFlow packet counter that is compared to “Value” resets to zero (0) at the start of each poll cycle. The poll cycle runs every uint32RunEveryXSeconds number of seconds (you will find this setting in the__VirtualAgentJobs tab to the right of the PERFORMANCE job name). The default is 600 seconds (10 minutes). Page 30 Cisco Solutions Guide OIDGets Tab The OIDGets tab contains entries that you can use to create alerts and/or write performance records based on any (non-Table-included5) OIDs from any MIB referenced by Cisco in the Operations Console (see Cisco mibs sub-directory). To process OIDGets information you must enable the PERFORMANCE Run checkbox in the __VirtualAgentJobs tab. Note: Unselect it if you do not need this functionality. The OIDGets tab has the following fields: Select Disable Object Counter Comparator Alert Threshold Graph Use Delta Aggregate Auto Resolve Alert Mib Name SNMP Name SNMP OID See TableGets tab (later in this document) for a discussion of how to get OIDs that are included in MIB tables 5 Page 31 Cisco Solutions Guide Select Checkbox Use the Clone/Remove checkbox in conjunction with the CLONE or REMOVE buttons to either create a new (cloned) entry or delete (remove) an existing entry. To create a new entry or delete an entry 1. Select the entry(s). 2. Click CLONE or REMOVE. Note: If this doesn’t appear to work, click SAVE first, then click CLONE or REMOVE. Disable Checkbox Use the Disable checkbox to “ignore” the contents of the entire row in the OIDGets. The Cisco Extension ships with two disabled sample rows, one for Inbound Packets and one for Outbound Packets. Set the Comparator and Alert Threshold pair to “GE” and ”0” respectively on one or the other or both to see how this functionality works. You will see an alert in the Operations Console after you SAVE and restart the Extension. You should have a MIB Browser/Walker available. Quest recommends that you download the free Personal Edition from http://www.ireasoning.com/ because it downloads and installs with the RFC1213-MIB already loaded. This is the MIB that Quest uses in the sample disabled entries you see in the OIDGets tab contents: Page 32 Cisco Solutions Guide Object Box Use the Object box to specify the Object Name within the MIB that contains the OID of interest. Example of how to use iReasoning MIB browser to find required OID settings Try this: 1. Open iReasoning’s MIB Browser. 2. Open RFC1213-MIB. 3. Scroll down to open “snmp”: Page 33 Cisco Solutions Guide Counter Box Use the Counter Box to identify the name of the metric you see in the Operations Console when you graph the performance metrics for your SNMP devices. This is a freeform area for you to identify the “type” of information you want. For example, the sample “Inbound Packets” Counter refers to the “snmp” OID named “snmpInPkts” (see SNMP Name Box for more information). At run time, the script variable, g_ArrayOfSNMPMetric in the PerformanceMetrics.js script gets populated with the values defined in this field. Comparator and Alert Threshold Checkboxes Use the Comparator and Alert Threshold checkboxes in conjunction with each other to control whether or not you want Operations Manager 2007 to create an Alert. The Comparator symbols: EQ – “equal to” GE – “greater than or equal to” GT – “greater than” LE – “less than or equal to” LT – “less than” NE – “not equal” Operations Manager 2007 will never create an Alert when the Comparator is LT and the Alert Threshold is 0, indicating “less than zero”, because threshold values are always Page 34 Cisco Solutions Guide zero or positive. This is the proper way to disable Operations Manager 2007 Alerting for a particular metric. Graph Checkbox Use the Graph checkbox to create a performance graph and report data. The data goes into the Operations Manager 2007 database (the data warehouse) and is used for Operations Manager 2007 Performance and Reporting services. Note: Setting the disable checkmark is equivalent to setting the Comparator to “LT” (less than), the Alert Threshold to 0 (zero) and unselecting the Graph checkbox. Use Delta Checkbox Use the Use Delta checkbox to indicate you that want the “performance metric” value to be the difference (delta) between the current value and the previously collected value. Be aware that the delta is determined in either direction (positive or negative) between the current and prior metric. If the checkbox is unselected, the current “performance metric” value is used without any consideration as to the delta value. Aggregate Drop-down Box Use the Aggregate Drop-down Box to select the number of occurrences of the threshold value you want to “happen” before it creates an alert. This has no effect on the creation of performance records. Auto Resolve Checkbox Use the Auto Resolve Alert checkbox to indicate you want an existing alert to autoresolve when a threshold value is no longer exceeded. If the checkbox is unselected then the previously created alert remains visible in the Operations Console view(s). Mib Name Box Use the Mib Name Box to define the exact name of the MIB that contains the metric. For example, the RFC1213-MIB contains the “snmp” OID named “snmpInPkts” (see SNMP Name Box). The MIB must be in the Cisco mibs sub-directory and the name must match the MIB Name in the MIB itself which might differ from the actual MIB Name itself. Find the MIB name by opening the MIB and looking for a line similar to this: RFC1213-MIB DEFINITIONS ::= BEGIN. You can extract the MIB Name from this line. SNMP Name Box Use the SNMP Name Box to reference the variable in the mib file defining the name of the specific field from which you intend to do the SNMP Get. You can view the available mibs in the mibs subdirectory. For example, snmpInPkts is in the Object named “snmp” in the MIB named RFC1213-MIB”. Note: You can easily see this structure using the iReasoning MIB Browser, as shown in the previous screen shot. SNMP OID Box Use the SNMP OID Box to reference the variable in the SNMP MIB file related to the SNMP Name. For example, the OID for snmpInPkts (which is subordinate to “snmp” in the “RFC1213-MIB”) is .1.3.6.1.2.1.11.1.0. Note: Do not forget the first period (“.”) in the OID string you specify. Page 35 Cisco Solutions Guide If you use iReasoning’s MIB browser, click the “snmpInPkts” entry in the “OID:” display area to see that the OID for snmpInPkts is indeed .1.3.6.1.2.1.11.0, as shown in the previous screen shot. If you make any changes to any fields in the tab, click SAVE, then stop and restart the Extensions for the changes to take effect. Page 36 Cisco Solutions Guide PerformanceMetrics Tab To process PerformanceMetrics information you must enable the PERFORMANCE Run checkbox in the __VirtualAgentJobs tab. Note: Unselect it if you do not need this functionality. The PerformanceMetrics Tab has the following fields: Disable Name Metric Category Metric Name Comparator Alert Threshold Graph Disable Checkbox Use the Disable checkbox to prevent Operations Manager 2007 from creating the Alerts or Performance records it uses for Performance and Reporting services. If the checkbox is not selected, it allows the creation of Operations Manager 2007 Alerts or Performance records. Note: It does not guarantee that Operations Manager 2007 will produce an Alert or Performance record. Cisco Monitoring Extension ships with reasonable defaults. If you enable every entry (as shown), the amount of data it produces could overwhelm the database. And, depending on your CPU speed, monitoring intervals, I/O capability, and so forth, the amount of I/O could be burdensome. Name Box The Name of the Cisco tuning parameter. Do not modify the text in the Name box, Cisco defines these names and the scripts depend on them. Page 37 Cisco Solutions Guide Metric Category Box The Metric Category groups common performance metrics within MOM. At run time, the g_ArrayOfMetricCategory script variable in the All.js job script is populated with the values defined in this field. This is free form text. To regroup certain performance metrics, you can modify them. However, Quest does not recommend that you not modify the text in the Metric Category box. Metric Name Box The Metric Name is the name you see in MOM when you graph the performance metrics for your Cisco device. At run time, the g_ArrayOfMetric script variable in the All.js job script is populated with the values defined in this field. Quest does not recommend that you not modify the text in the Metric name box. Comparator and Alert Threshold Checkboxes Use the Comparator and Alert Threshold checkboxes in conjunction with each other to control whether or not you want Operations Manager 2007 to create an Alert. The Comparator symbols: EQ – “equal to” GE – “greater than or equal to” GT – “greater than” LE – “less than or equal to” LT – “less than” NE – “not equal” Operations Manager 2007 will never create an Alert when the Comparator is LT and the Alert Threshold is 0, indicating “less than zero”, because threshold values are always zero or positive. This is the proper way to disable Operations Manager 2007 Alerting for a particular metric. Graph Checkbox Use the Graph checkbox to create a performance graph and report data. The data goes into the Operations Manager 2007 database (the data warehouse) and is used for Operations Manager 2007 Performance and Reporting services. Note: Setting the disable checkmark is equivalent to setting the Comparator to “LT” (less than), the Alert Threshold to 0 (zero) and unselecting the Graph checkbox. Use Delta Checkbox Use the Use Delta checkbox to indicate you that want the “performance metric” value to be the difference (delta) between the current value and the previously collected value. Be aware that the delta is determined in either direction (positive or negative) between the current and prior metric. If the checkbox is unselected, the current “performance metric” value is used without any consideration as to the delta value. Page 38 Cisco Solutions Guide Aggregate Drop-down Box Use the Aggregate Drop-down Box to select the number of occurrences of the threshold value you want to “happen” before it creates an alert. This has no effect on the creation of performance records. Auto Resolve Checkbox Use the Auto Resolve Alert checkbox to indicate you want an existing alert to autoresolve when a threshold value is no longer exceeded. If the checkbox is unselected then the previously created alert remains visible in the Operations Console view(s). Page 39 Cisco Solutions Guide TableGets Tab The TableGets tab contains entries that you can use to create alerts and/or write performance records based on any Table-included OIDs from any MIB referenced by Cisco (see Cisco mibs sub-directory) in the Operations Console. To process TableGets information you must enable the PERFORMANCE Run checkbox in the __VirtualAgentJobs tab Note: Unselect it if you do not need this functionality. Page 40 Cisco Solutions Guide Example of how to use iReasoning MIB browser to find required “Table” settings The TableGets tab has the following fields: Select Disable Object Counter Comparator Alert Threshold Graph Use Delta Page 41 Cisco Solutions Guide Aggregate Auto Resolve Alert Mib Name Table Name SNMP Name Table Row Unique Identifier Select Checkbox Use the Clone/Remove checkbox in conjunction with the CLONE or REMOVE buttons to either create a new (cloned) entry or delete (remove) an existing entry. To create a new entry or delete an entry 3. Select the entry(s). 4. Click CLONE or REMOVE. Note: If this doesn’t appear to work, click SAVE first, then click CLONE or REMOVE. Disable Checkbox Use the Disable checkbox to “ignore” the contents of the entire row in the TableGets. The Cisco Monitoring Extension ships with four disabled sample rows: one for Inbound Discards, one for Inbound Errors, one for Outbound Errors, and one for Outbound Discards. Set the Comparator and Alert Threshold pair to “GE” and “0” respectively on any row to see how this functionality works. You will see alerts6 in the Operations Console after you do a SAVE and restart the Extension. You should have a MIB Browser/Walker available. Quest recommends that you download the free Personal Edition from http://www.ireasoning.com/ because it downloads and installs with the RFC1213-MIB already loaded. This is the MIB that Quest uses in the sample disabled entries you see in the TableGets tab contents. Object Box Use the Object Box to specify the Object Name within the MIB that contains the TABLE of interest. Try this: 1. Open iReasoning’s MIB Browser. 2. Open RFC1213-MIB. 3. Scroll down and navigate to ifTable | ifEntry (as shown in the previous screen shot). Since you may be monitoring a great number of interfaces – be careful what you enable with GE/0 as there will be an alert for every interface. 6 Page 42 Cisco Solutions Guide Counter Box Use the Counter Box to identify the name of the metric you see in the Operations Console when you graph the performance metrics for your SNMP devices. This is a freeform area for you to identify the “type” of information you want. For example, “Inbound Discards” refers to the “ifIndex/ifInDiscards” Table named “ifTable”. (See SNMP Name Box). Comparator and Alert Threshold Checkboxes Use the Comparator and Alert Threshold checkboxes in conjunction with each other to control whether or not you want Operations Manager 2007 to create an Alert. The Comparator symbols: EQ – “equal to” GE – “greater than or equal to” GT – “greater than” LE – “less than or equal to” LT – “less than” NE – “not equal” Operations Manager 2007 will never create an Alert when the Comparator is LT and the Alert Threshold is 0, indicating “less than zero”, because threshold values are always zero or positive. This is the proper way to disable Operations Manager 2007 Alerting for a particular metric. Graph Checkbox Use the Graph checkbox to create a performance graph and report data. The data goes into the Operations Manager 2007 database (the data warehouse) and is used for Operations Manager 2007 Performance and Reporting services. Note: Setting the disable checkmark is equivalent to setting the Comparator to “LT” (less than), the Alert Threshold to 0 (zero) and unselecting the Graph checkbox. Use Delta Checkbox Use the Use Delta checkbox to indicate you that want the “performance metric” value to be the difference (delta) between the current value and the previously collected value. Be aware that the delta is determined in either direction (positive or negative) between the current and prior metric. If the checkbox is unselected, the current “performance metric” value is used without any consideration as to the delta value. Aggregate Drop-down Box Use the Aggregate Drop-down Box to select the number of occurrences of the threshold value you want to “happen” before it creates an alert. This has no effect on the creation of performance records. Page 43 Cisco Solutions Guide Auto Resolve Checkbox Use the Auto Resolve Alert checkbox to indicate you want an existing alert to autoresolve when a threshold value is no longer exceeded. If the checkbox is unselected then the previously created alert remains visible in the Operations Console view(s). Mib Name Box Use the Mib Name Box to define the exact name of the MIB that contains the metric. For example, RFC1213-MIB contains the Table named “ifTable” (see SNMP Name Box and Table Row Unique Identifier Box). The MIB must be in the Cisco mibs sub-directory and the name must match the MIB Name in the MIB itself which might differ from the actual MIB Name itself. To see the MIB name, open the MIB and look for a line similar to this sample: RFC1213-MIB DEFINITIONS ::= BEGIN. You can extract the MIB Name from this line. Table Name Box Use the Table Name Box to identify the exact name of the specific table within the MIB that contains the SNMP OIDs on which you intend to do the SNMP Gets. If the metric is an oid within an SNMP table, this name field must contain the exact name of the table. For example, the RFC1213-MIB contains the Interfaces table named “ifTable”. Be careful when enabling “TableGets” because there is an entry for every interface which could easily run into hundreds or thousands of interfaces depending on the size of your network and the devices you have set the Cisco Monitoring Extension to monitor. Use a SNMP MIB browser program to identify the SNMP table names you would like to collect. SNMP Name Box Use the SNMP Name box to define a specific field that is subordinate to the Table Name on which you intend to do the SNMP Gets. For example, “ifInDiscards” is in the “ifTable” in the MIB named RFC1213-MIB. Note: You can easily see this structure using the iReasoning MIB Browser, as shown in the previous screen shot. Table Row Unique Identifier Box Use the Table Row Unique Identifier box to identify the row within the table containing the name of the SNMP OID. Specify the SNMP Name here, not the SNMP OID. This is similar to a key on a database table. Use a SNMP MIB browser program to identify the Table Row Unique Identifier. For example, the OID for ifInDiscards (which is subordinate to “ifIndex” which is subordinate to “ifTable” in the “RFC1213-MIB”) is calculated for you and obtained repeatedly (once per interface). Note: You can easily see this structure using the iReasoning MIB Browser, as shown in the previous screen shot. Open “ifTable” and then “ifIndex” under the “interfaces” section to reveal all the potential fields (each with unique OIDs) that can be processed. Page 44 Cisco Solutions Guide If you make any changes to any fields in the tab, click SAVE, then stop and restart the Extensions for the changes to take effect. Page 45 Cisco Solutions Guide _VirtualAgentJobs Tab The __VirtualAgentJobs tab is primarily used by developers who want to do in-depth Extension customization. Quest support also uses the settings on this page to assist you in debugging. Note: Use caution when altering the default values on the __VirtualAgentJobs tab. The only settings the average user should modify are the Run checkbox and unit32RunEveryXSeconds polling time, as described below. Job Name A Job is an executable script that carries out the functions of the Extension. Next to each Job Name is a Run checkbox and a Unit32RunEveryXSeconds box. The OOTB jobs supported are: PERFORMANCE: Run this job to gather up performance data. It will gather up data as specified in the NetFlow, OIDGets, Performance Metrics, and TableGets tabs. TRAPS: Run this job to take TRAPS inbound from the Cisco device and turn them into alerts. WATCHDOG: Run this job to ensure Cisco device connectivity (between it and the QMX box) and to keep the State View updated (when it runs; default: every 307 seconds). Run Column Checkbox Use the Run checkbox to indicate whether you want a Job to run or not. Unselect the Run check box to prevent this Job from running. Page 46 Cisco Solutions Guide Uint32RunEveryXSeconds Column Box Enter the number of seconds you want to occur between Performance Statistics gathering sessions in the uint32RunEveryXSeconds box. WARNING: Never reset a factory setting of “0” for uint32RunEveryXSeconds. This setting means the solution will be running all the time (which is necessary if shipped this way from the factory; e.g. ‘tailing’ on a log must never stop!). Resetting this value from 0 to any other value will cause unpredictable results (in the case of a MonitorLog Job (which tails) – it would result in potentially lost log records. Each solution is unique – but a factory setting of time zero should never be reset! Note: Do not set this number to be less than the time it actually takes the Extension to finish running. A 1605 Router can finish in about 20 seconds; however, a fully loaded 6500 Core Switch can take up to 20 minutes (or 1200 seconds). If you set it for a longer time and then check the CiscoPerf LOGS sub-directory, it will show you the actual Start and Stop time for each run. This might help you determine the approximate minimum time to set. If you set it too low, the Extension will be cancelled in mid-stream and restarted. Uint32RunEveryXSeconds expect this Extension to finish prior to starting the next one. If it is not done, it cancels the current copy and starts a new one. Note: Quest cautions you against specifying an extremely low run time, such as 60 or 120 seconds, as CPU or Memory utilization could become extremely high with little benefit achieved. 600 is the default setting; 300 is practical, given the number of devices to monitor as well as the CPU and Memory available on the box where the Cisco Monitoring Extension is running. If you make any changes, click SAVE, then stop and restart the Extensions for the changes to take effect. Note: Quest recommends that you use the default settings that you get when you download the solution. Changing these values could cause unexpected and undesirable results. Page 47 Cisco Solutions Guide Cisco Extension Options Right-click on any Cisco Extension sub-folder to open the context menu: Message Filter DB Option Select the Message Filter DB option to start the Trap Editor (See Using the Trap Editor to Modify an (Access DataBase) Alert for more information.) Note: The Message Filter DB option only appears for SNMP solutions and only works if you are using Access. If you opted to convert the TRAP DataBase (.mdb) to SQL (or were forced to because you are running 64-bit) – you can still edit the “Trap Database” – but you must do so using SQL directly (so you will get an error or nothing when you try to open the .mdb this way when you’ve converted it!). Page 48 Cisco Solutions Guide Autodiscovery Option Even if you read about AutoDiscovery in the downloadable QMX Installation Guide, read here – because Cisco has some caveats from the standard discovery. Select the Autodiscovery option to avoid manually populating each required __Hosts entry. The Quest Extensions require configuration information about the Cisco devices you want to monitor. The Auto Discovery program simplifies the configuration of the Cisco Monitoring Extension. There are 3 ways to discover devices for Cisco. They are via CLI, SNMP, or Nmap. Nmap is a free tool and requires that you download it (from insecure.org, or CNET, etc.) and put the location of the download in the “path” for the QMX box (so QMX will find it). For past users, the CLI and SNMP remain “mostly the same” – although the format is quite different (and you now must supply the ‘enable password’ file (although it is only used for PIX and ASA devices (or those that require being ‘enabled’ to execute certain commands our scripts run – which are mostly available to ‘casual users’ (unless you have locked down your environment somewhat)). For the non-PIX and non-ASA you can supply a name of (for instance) “file:C:\NotRequired”. The file doesn’t even have to exist, because the solution will never attempt to read it. The default as delivered from the factory for Tab/Item: GlobalVariables~g_strProtocol is SNMP/CLI. This is the most comprehensive and fool-proof way to run the solution. But it requires that you have both CLI (Telnet and/or SSH) and SNMP available. Some customers do not. For them, running in CLI (only) or SNMP (only) is an option. Read the AppNotes tab in the solution for the most up to date information regarding this (and other information of interest). Running AutoDiscovery in CLI (only) mode will not fill in the SNMP information accurately (since it’s assumed you won’t be using SNMP or intend to fill it in yourself – it will guess at a Community of “public”). The assumption if you do this is that you intend to reset Tab/Item: GlobalVariables~g_strProtocol to “CLI”. Running AutoDiscovery in SNMP (only) mode will not fill in the CLI information accurately (it will indeed even set a port of 0 to draw attention to this fact) (since it’s assumed you won’t be using CLI or intend to fill it in yourself). The assumption if you do this is that you intend to reset Tab/Item: GlobalVariables~g_strProtocol to “SNMP”. Running AutoDiscovery in Nmap mode will include both CLI (Telnet and/or SSH) information as well as SNMP information. However, whereas the CLI method will actually log in to your Cisco devices, Nmap will not. It will detect the device, and attempt to detect if a Telnet or SSH Server exists. If it can’t detect it – it will “guess” at Telnet (usually meaning the device is quite old, so Telnet is more likely than SSH). You should always use Test Connection before running the populated __Hosts entries! The upside of Nmap is it’s speed compared to CLI. The downside is the lack of checking. However, since most accounts have service ids/passwords (which are required even for the CLI discovery) this should present no issue. The Nmap discovery will just “take your word for it” that you have supplied the right userid and password and SNMP Community Name(s) and will populate detected Cisco device __Hosts entries using that information. Quest has determined that in most cases running with Nmap is appropriate due to the speed advantage over CLI. This would not necessarily be the case if the network had a lot of Page 49 Cisco Solutions Guide ‘older devices’ where Telnet would be ‘assumed’ when in reality those devices were SSH! This is based on available device ‘fingerprinting’ with Nmap. If it doesn’t recognize the device “for sure” it will make a “best guess”, but, normally, when there’s a “best guess” response, there is no indication of whether or not a Telnet or SSH server (ports 22/23 by default) are open (thus, QMX makes it’s “guess” of Telnet). We suggest trying it on a small number of devices by using the “Range” method of discovery. Page 50 Cisco Solutions Guide The screen image below depicts what the Auto-Discovery screen looks like when you first bring it up (note that it is set to do Nmap discovery): Page 51 Cisco Solutions Guide If you remove the Nmap checkmark, you will be able to select either SNMP or CLI methods for discovery: Page 52 Cisco Solutions Guide Page 53 Cisco Solutions Guide Refer to the three (3) preceding screen shots when reading the discussion points that follow. Follow these steps in order: RUN NMAP? 1) Determine if you will be using CLI/SNMP together to monitor you devices. If so, proceed to the next step (after insuring you have Nmap installed and in your QMX system’s ‘path’). If not, proceed to SNMP ONLY or CLI ONLY below step 18 and the text that follows. 2) Click on Range or Subnet. Information request boxes will appear to the right, fill them in as appropriate for your selection. 3) Leave the Output Hosts XML file name alone (unless you are just ‘testing’ and don’t really want to populate the __Hosts tab for the Cisco solution! 4) Leave the Merge checkbox on. Always. 5) Leave the Ping Timeout set at 500. If your run seems to miss devices that are ‘pingable’ you may have a really slow network. You can up the Ping Timeout and try again (rarely happens). 6) Leave Device is Pingable set (unless you are positive that your devices are not pingable (which would be unusual for most production accounts). 7) Leave the Nmap checkbox set (because you decided in step 1 that you were going to use CLI/SNMP together, you have Nmap installed, you have it in your system’s ‘path’ (so QMX can locate it without further directives), and you need to populate both CLI and SNMP fields. NOTE: Use SNMP Discovery and Use CLI Discovery are always greyed out when the Nmap checkbox is set. 8) Supply the Default Enable ENCRYPTED Password. This means you must have an enable password that is the same on all the devices you will discover (you can change the results ‘after the run’ if necessary) – just like having a service userid/password combination. NOTE: Enable passwords are so critical that we require they be encrypted. You’ll find the encryption tool in the QMX Administrator’s Console under “Tools”. You will ‘name the file’ that contains the encrypted enable password like this (example) in the box: “file:C:\encryptedfilename”. The “file:” tells the software that this field is RSA 1024 bit encrypted and to decrypt it at run time (again, it’s only necessary (typically) for PIX and ASA devices – but you MUST supply a “file:” name here (valid or not!). NOTE: you can use the elipses “...” to the right to enter the encryption tool! 9) The SNMP Timeout works just like the PING Timeout. The default setting should work in most cases. Decreasing it could mean missed devices during the discovery. Increasing it could slow your run down with no gain (unless your network is really slow! in which case you’ll end up having to increase it because you’ll notice a lot of missed devices (unlikely)). 10) Leave Create Device EVEN IF SNMP GET FAILS off. Always. 11) Fill in the proper SNMP Read, Write, and TRAP Community names. NOTE: Only devices that you’ll run QMX NetMon “Top-Talkers” on require a Write Community name. You can supply (example) “ANYTHING” as the Write Community name if you are going to use our NetMon feature (refer to the AppNotes for more information about this). These names are all CAsE SensITiVe! 12) Leave the SNMP OID To Query alone unless told by support to change it (will mean you have opened an incident) 13) Leave the Expected Response alone unless told by support to change it (will mean you have opened an incident). 14) The Login Timeout works just like the other timeouts. Leave it alone (most login prompts after “connect” should easily appear within 5 seconds!). Page 54 Cisco Solutions Guide 15) Change the SSH and Telnet ports only if you don’t use the standard defaults on your account. 16) Supply the userid to log into the device (plain text or encrypted) ... you can use the elipses “...” to the right to enter the encryption tool! 17) Supply the password to log into the device (plain text or encrypted) ... you can use the elipses “...” to the right to enter the encryption tool! 18) Leave the CLI command to issue and expected response alone unless you know that a different command/response will work for your device instead! NOW CLICK THE PROCESS BUTTON! It may take a bit before the Status starts to show up (time for initial locate of online devices to process). When the run is done, click OK and please read the output file that pops up on your screen. There might be some interesting information you require (if the run didn’t go as expected). Normally the ‘longer lines’ of information are what you are looking for. The short ones are normally just “tracking info”. NOW right click on the Cisco folder and select Refresh. Your newly populated devices will appear! Run a Test Connection on each to insure each device is in ‘working order’. Once satisfied you can start the device solutions up. DO NOT use Start All – because it will start them all at once vs. Using auto-start (they will be started with a 10-15 second gap, or start them manually (leaving a 10-15 second gap between each start to “load balance” a bit)). SNMP ONLY 1) Uncheck Nmap and set the Radio Button for Use SNMP Discovery. Return to Step 1 above at “RUN NMAP?” 2) Set the appropriate fields (read through steps 1 through 18 above skipping those steps that don’t apply (the fields NOT to be set will be greyed out!). 3) Follow the instructions after step 18. CLI ONLY 4) Uncheck Nmap and set the Radio Button for Use CLI Discovery. Return to Step 1 above at “RUN NMAP?” 5) Set the appropriate fields (read through steps 1 through 18 above skipping those steps that don’t apply (the fields NOT to be set will be greyed out!). 6) Follow the instructions after step 18. Page 55 Cisco Solutions Guide Clone VA Option Select the Clone VA option to create a sub-folder identical to any Cisco folder you choose. The “tabs” used in the Cisco Monitoring Extension are “cloned” to the sub-folder so that you can apply different criteria per folder. For example, you might prefer to split up your test, development, and production IOS devices because you want to set different threshold values for each. Using the Clone VA option, you can put one to n devices in the cloned sub-folder. Refresh Option Select the Refresh option to access the latest __Hosts information. Use this option after Autodiscovery. Start All Option Select the Start All option to start all the non-disabled Hosts. This is reasonable for trials and testing; however, it is not a best practice for production or large trial environments because it starts all the Hosts immediately. This could potentially cause a large strain on CPU and/or Memory resources. Use the auto-start facility that ships with the product to start and restart Hosts since it “stages the start ups so they don’t occur simultaneously, as it does when you use the Start All option. Stop All Option Select the Stop All option to stop all the non-disabled Hosts. However, if you do this and auto-start is enabled, the Hosts will start back up shortly! To prevent this, disable the auto-start facility and then select Stop All. Version Option Select the Version option to see what version of the Cisco Monitoring Extension you have. This option opens the Version_History.txt file. The only version you see (or the one at the top) is what is currently installed on your system (any version information below that is historical). Page 56 Cisco Solutions Guide Using the Trap Editor to Modify an (Access DataBase) Alert Quest’s Extensions are ready for use “as is”. However, you may want to change some of the default values during the testing phase or your rollout to production. Use the Trap Editor to configure Alert severity and text of the Alerts. To open the Trap Editor 1. Right click Cisco and select Message Filter DB7. The Trap DB/Editor opens. To use the Trap Editor 1. Select Generate MOM Alert? to produce an event or alert. Unselect this option to prevent one or more specific Traps from appearing as Operations Manager 2007 Alerts8. 2. Set the Alert Severity9 for one or more Trap. If you have converted the .mdb to SQL, go into the converted SQL database and do the equivalent work there. 7 8 a specific alert may be of no consequence to your particular account requirements The Operations Manager 2007 Alert Severity is pre-set; however, any particular Trap (Operations Manager 2007 Alert) may be more or less important to your account than the preset severity. 9 Page 57 Cisco Solutions Guide MOM 2005 Operations Manager 2007 10 = Success 10 20 = Information 10 30 = Warning 30 40 = Error 30 50 = Critical 50 60 = Security 50 70 = Service unavailable 50 Note: Operations Manager 2007 accepts all seven MOM 2005 Severities, but groups them differently in the associated Alerts display. 3. Edit the content of the alerttemplate box to change the text of an alert10 or rearrange the variables, and so forth. The text displayed in the alerttemplate box is the text that you see in the Alert Description field when this Trap arrives. You can add, change, or remove any text you desire. 4. Arrow #4 points to the oid field of the Trap Editor. When a Trap arrives from an SNMP device, its “key” is a world-wide unique OID. If the text of the alert in the alerttemplate box does not give you enough information about the error, you can use this Trap’s OID identification to get more information. For example, CISCO has an “OID lookup” that gives you more details about most OIDs. You must contact Cisco directly for that information. Another great tool for OID lookups is at: http://support.ipmonitor.com/mibs_byoidtree.aspx. Or, you can do a Google search for the OID number. 5. Arrow #5 points to two distinct areas of the Trap Editor. The first pointer (going left to right) exists because this specific Trap has some sub-OIDs that provide variable information along with the basic fixed text that you see in alerttemplate box. When a variable exists, its contents appears where the other (upward) arrow points. In this case, it is pointing to “%2”. The variable that arrives with the Trap is placed in this position of the text prior to generating the Operations Manager 2007 Alert. Note that the variables are numbered from 1 to x. This number appears at both arrow points associated with Arrow #5. This is how the variables table and the Operations Manager 2007 Alert text are “tied together”. At the time of download every11 SNMP Trap Extension has an Access Data Base named MessageFilterDB.mdb. You can opt to convert this to SQL after you download and install it. This database contains all the relevant Trap information for the device type. The Your account may have a set procedure to perform when a particular Trap arrives. You can modify the text that displays in the Operations Manager 2007 Alert to inform the users. Note: if you make a change in the alert template and it does not take effect, either stop and restart the Extension to propagate the changes or make the changes in the script named VirtualAgent.js. Final overrides can occur in that script and for some solutions, such as BlackBerry, it is necessary that they do. 10 11 Except the SNMP Generic Solution. Page 58 Cisco Solutions Guide MessageFilterDB.mdb in the Cisco Extension directory contains all the Traps that all Cisco devices can issue; thus, there is no requirement for a MIB for the available SNMP device solutions. Quest has pre-processed the MIBs for you and placed the relevant information into the MessageFilterDB.mdb. Finding the Record that Matches the Alert To find the record that matches the Alert you see in the Operations Manager 2007 Console Alert View, open the Find and Replace dialog. Use the “Record:” number at the bottom left of the screen to move from record to record (that is, Trap to Trap). Or, enter either an OID or some of the text of the alert you are interested in finding in the Find and Replace dialog’s Find What box. Note: Use caution when searching for a trap using text from the alert; sometimes text is repeated in different Traps. Use the OID number whenever possible, to avoid confusion. Page 59 Cisco Solutions Guide SNMP Trap Sample The following is a sample of an inbound SNMP Trap converted to an Operations Manager 2007 Alert: This a sample of a Performance Graph for the Cisco Router CPU: Page 60 Cisco Solutions Guide Cisco Device Options Right-click a specific device within the Cisco Extension to open the context menu: Ping Option Select Ping to test whether this device is reachable across an IP network on this device using the DNS name. Page 61 Cisco Solutions Guide Nslookup Option Select Nslookup to find various details relating to DNS including the IP addresses of this device using the DNS name. Trace Route Option Select Trace route to determine the route taken by packets across an IP network using this device’s DNS name. Test Connection Option Select Test Connection to execute either a CLI login test and/or an SNMP Get command to verify that the basic device entry information in the __Hosts tab is set up correctly. Select the proper radio button and click Run Test. You need to run both the CLI and SNMP Test Connections if you are running the Cisco Monitoring Extension as delivered without disabling any of the Run checkboxes in the __VirtualAgentJobs tab. Note: Run this command before running Start to run the Extensions against a specific device to ensure connectivity exists. Typical errors you might receive are: o DNS server is not available (put the entry in /etc/hosts) o wrong login prompts are set up o wrong responses are set up o wrong ports and/or Secure Connection checkbox is set inappropriately for the target device o SNMP Get failed Note: This option does not test enablepassword.txt (See Enable Password ) which is checked only when you select Start. Telnet or SSH Option Select Telnet or SSH (only one or the other will show up in the context menu depending on what is appropriate for your solution). Run Diagnostics Option Select Run Diagnostics to produce basic troubleshooting information to send to Quest support. When it completes the diagnostics testing, the Windows Script Host dialog opens, showing the path to a file containing the results. Typically it is located in the C:\Program Files\eXc Software\WMI Providers\nonWindows\Diagnostics folder. Show Log Option Select Show Log to display the Debug log produced by the Extension. If this entry is greyed out, there is no data in the log. Clear Log Option Select Clear Log to empty the contents of the Debug log produced by the Extension. If this entry is greyed out there is no data in the log. Typically, you use this option to minimize the amount of data in the log when you are debugging an issue. For example, Page 62 Cisco Solutions Guide you might Stop the device (Extensions), click Clear Log, click Start to start the device (and Extensions), wait for the problem to occur, click Stop to stop the Extension, and click Show Log. Start or Stop Option Select Start or Stop (only one or the other displays in the context menu depending on if the device is running or not. Select Start to start a specific device (and its Extensions); select Stop to stop a specific device (and its Extensions). Note: If you see both Start and Stop in the context menu at the same time, the device is in mid-stream. Wait about 2 seconds and right-click the device again. You should only see Start or Stop, not both. If you attempt to do a Start or Stop when both options appear, a warning displays. Wait about two seconds and try again. Page 63 Cisco Solutions Guide Cisco Trap and Performance Screen Shots Alert View Page 64 Cisco Solutions Guide Performance View Page 65 Cisco Solutions Guide Diagram View Generic Reporting View Page 66 Cisco Solutions Guide Troubleshooting Installation and Operation Problems Before you call Quest support, try some of the following troubleshooting steps. Check Software Requirements 1) SNMP Service Set the Windows SNMP Service’s Security tab appropriately for your environment. Be aware that Group Policy Object (GPO) may have this tab locked down or may reset changes you make after you make them. Note: You must restart this SNMP Service after making any changes to the Security tab contents. 2) SNMP Trap Service Set the properties to “automatic” and restart the service. Note: The default value for the SNMP Trap Service properties are set to “manual”. 3) Microsoft WMI SNMP Provider Check the Add/Remove Windows Components and check the options selected in the Management and Monitoring Tools. Page 67 Cisco Solutions Guide 4) Microsoft Access Runtime Unless you opt to convert the .mdb that is shipped with the Cisco Monitoring Extension to SQL, you must install the Microsoft Access Runtime. Basic Troubleshooting Tips Check the following: 1) Not having the WATCHDOG job checkbox set on the __VirtualAgentJobs tab causes the State View to always indicate there is an availability issue. 2) UDP Ports 161 and 162 must be open. 3) Access lists must allow traffic between the Cisco device and the QMX box (e.g. if your Config specifies “snmp-server community blah RW 20” --- make sure the “access list” for “20” allows the required traffic! (if you do not see anything after the “blah R” or “blah RO” or “blah RW” – then there isn’t an access list associated with Reads or Writes --- don’t forget to check TRAPS as well). 4) You don’t get any TRAPs converted to alerts as you expect. Check that you have the Cisco device set up properly to throw ALL TRAPS and to throw them to the QMX box(es). Here’s an example of a proper setup for a Cisco 1605 router: Let’s discuss the parmameters: The Community Name for READING and WRITING (Gets and Sets) is all lower case “public”. We require both READ and WRITE (RW) on the community statement (you can drop the WRITE and use RO instead of RW if you are not going to use NetFlow). The location is optional. We don’t use it. All the “enable traps blah” came from a single “conf t” command which was “snmpserver enable traps”. Some customers make the mistake of picking specific TRAPS (we want them all – we’ll handle them on “our side”) – or set up to throw TRAPS but don’t enable any of them! The final three statements tell what hosts (yes, you can do multiple) to throw SNMP TRAPS at … and what the TRAP community name is. In this case (for ease of lab testing – we used “public” for Reads, Writes, and TRAPs). We could easily have used different names for each. Page 68 Cisco Solutions Guide You don’t get any TRAPs converted to alerts as you expect. Check that you have the WMI SNMP Provider installed (Add/Remove Windows Components~Management & Monitoring). If you don’t – install it. If you do – remove it, reboot, and re-install it (sometimes Windows patches disable it and it must be reinstalled although it already is!). 5) Community names are case sensitive and must match in all three locations: the Configuration Tool, the Windows SNMP Services Security Tab (if you opt to restrict SNMP traffic flow by Community Name), and in the Cisco device itself. 6) NIC drivers (such as, Broadcom) must be at the latest level, especially if you employ SSH for any connections going through the NIC card. 7) Read the downloadable installation documentation. Follow the directions from top to bottom; do not skip any steps, do not read ahead or jump around. Windows SNMP and Cisco SNMP Setup Examples12 The following is an example from a running system. You may set your system up differently. For example, perhaps you restrict SNMP TRAP senders, use GPO, employ CATOS rather than IOS, or have different TRAPS enabled. (Note: This example assumes that all SNMP Community names are “public”.). If you are using NetFlow – be sure to set up a Write Community as well (with a minimum of Read Write privileges). SNMP Community names must be in sync in the following: 1. In the device. Note: You set this up using the “conf t” setup command in enabled mode. Traps must be thrown to the “only” QMX box or to the “primary” and “backup” QMX boxes if you are running with QMX failover. The above example is for a ‘primary’ at 100, ‘backup’ at 101, and ‘another computer at 110’ (meaning you can run QMX alongside other solutions). 2. In the Windows SNMP Services. Ask your Windows Administrator and Cisco Administrator for the details required for your environment. The SNMP Services Security tab was allowed to default, which allows TRAPs from any device; CATOS uses “set” commands you will not see in the examples because the examples are based on IOS. 12 Page 69 Cisco Solutions Guide Note: You must have the SNMP TRAP Service set to automatic, ensure it is started, and that the SNMP Services Security tab contents are set up correctly. 3. In the Configuration Tool: Page 70 Cisco Solutions Guide Performance Issues Quest recommends that you upgrade your Base Framework to the latest version available. If you plan to connect to more than 50 Cisco devices please refer to the section of the same name in the downloadable OpsMgr 2007 Installation Guide. If You Call Support Supply Quest level 2 or level 3 support engineers with the following: 1. Provide the output from the Run Diagnostics option which produces basic trouble-shooting information. (See Cisco Device Options.) 2. Use NetMon (preferably 3.3 or greater; downloadable directly from the web – ie. search google for ‘download netmon 3.3’) to trace network traffic. (Quest recommends this tool over WireShark.) 3. Use a MIB walker to display MIB information. Quest recommends that you download the free Personal Edition Mib Walker from http://www.ireasoning.com/ 4. A schematic of your network layout especially where firewalls or intrusion detection equipment is located in relationship to the Operations Manager 2007 device, where the Quest Extensions are installed so that they can determine if there is a firewall between the System Center Operations Manager 2007 and the equipment you are trying to monitor. Page 71 Cisco Solutions Guide 5. If you require the assistance of Quest level 2 or level 3 support engineering, it is always best to have Visual Studio Pro installed in order to “walk the code” by means of the VS Debugger. Page 72