June 5, 2013 [CUCM TRACE SETUP AND COLLECTION] Setup & Collecting Traces from Call Manager CallManager 5.x and above uses the Cisco CallManager Serviceability page to enable traces and the Real Time Monitoring Tool (RTMT) to retrieve the traces from the server. 1. On a computer that has IP connectivity to the CallManager server, go to the address http://<ip address>:8443/ccmservice/ or choose "CallManager Serviceability" from the pull-down menu on the Administration page as shown. You need to log on with the CCMAdministrator username and password. sdl for 9.x and later sdi needed for 8.6.x and lower 2. Once you are on the Serviceability page, choose Trace > Configuration from the top menu. CUCM Trace Setup & Collection Page 1 June 5, 2013 [CUCM TRACE SETUP AND COLLECTION] 3. On the Trace Configuration page that loads, choose the server and the service for which you want to enable traces. In this example, we chose the Cisco CallManager Service on the server 14.80.99.210. When the options for the chosen service appear, click the check box Trace On. Ensure that you enable the Detailed trace level because this provides the maximum amount of data to the TAC. When the options are set properly, scroll to the bottom of the screen and click the Save button. CUCM Trace Setup & Collection Page 2 June 5, 2013 [CUCM TRACE SETUP AND COLLECTION] From CallManager 6.x, there is an additional option called Service Group. Choose the CM services from the Service Group drop-down list and select the CallManager which is needed for trace configuration. 4. (optional) Cisco TAC can request that SDL traces be gathered as well for the CallManager, CTIManager, or other services. Follow these steps to enable SDL traces for the Cisco CallManager service. a. From the Cisco CallManager Traces page shown here (also known as CallManager SDI Traces), choose SDL Configuration from the Related Links pull-down menu and click Go: b. Follow the same procedure used for CallManager traces to enable SDL traces as shown: CUCM Trace Setup & Collection Page 3 June 5, 2013 [CUCM TRACE SETUP AND COLLECTION] 5. Perform the action that reproduces the problem you that you have. The detailed troubleshooting information is logged on the CallManager server. 6. Now that you have recreated the problem with traces enabled, you need to gather the trace files off the CallManager server. Since Cisco CallManager 5.x and above runs on a Linux operating system, The RTMT must be used to gather the traces files. In order to install the RTMT, follow these instructions: a. From the CallManager Administration page, choose Application > Plugins from the top menu. b. A search window appears. Click Find with the search field empty to show all possible plugins. Find the RTMT from the list for your specific operating system and click the Download link on the right to save the file to your local computer. CUCM Trace Setup & Collection Page 4 June 5, 2013 [CUCM TRACE SETUP AND COLLECTION] c. When the download finishes, run the installer and step through the on screen instructions. 7. When the installation has completed, run the RTMT application. Specify the CallManager server from which you collect traces, as well as the CCMAdministrator username and password as shown: CUCM Trace Setup & Collection Page 5 June 5, 2013 [CUCM TRACE SETUP AND COLLECTION] 8. Choose CM-Default from the list: CUCM Trace Setup & Collection Page 6 June 5, 2013 [CUCM TRACE SETUP AND COLLECTION] 9. Choose Tools > Trace & Log Central. CUCM Trace Setup & Collection Page 7 June 5, 2013 [CUCM TRACE SETUP AND COLLECTION] 10. Double-click Collect Files.Choose the services and servers from which you want to gather trace files and then click Next: 11. A window pops up that requests the time and date range from which to collect the traces. Try to be as specific as you can here and use either absolute or relative time ranges as shown. The tool also allows you to specify the location on your local computer to which to store the files. You can change the location if you click the Browse button. When you send files to TAC, it is also generally a good idea to choose the Zip Files option to reduce the number of files that needs to be sent. After you set the options, choose Finish. CUCM Trace Setup & Collection Page 8 June 5, 2013 [CUCM TRACE SETUP AND COLLECTION] CUCM Trace Setup & Collection Page 9 June 5, 2013 [CUCM TRACE SETUP AND COLLECTION] 12. RTMT then saves the files to your local computer. The status is shown in the main window: Note: If you reproduce a problem, make sure to select the file for the timeframe when you reproduced it. Look at the modification date and the timestamps in the file. The best way to collect the right traces is to reproduce a problem, quickly locate the most recent file, and copy it from Cisco CallManager. Files are overwritten after some period of time. In order to find the current file that is logged, click View > Refresh on the menu bar and look at the dates and times on the files. You can view and configure the location, size, and lifespan of the trace files, as shown in the preceding diagram. Traces can be CPU intensive for the Cisco CallManager server. It is a good practice to turn off traces after you have collected them. Follow the same procedure used to enable the traces, but uncheck the "Trace On" settings and save. Tips to Troubleshoot Know where your devices are registered. Each Cisco CallManager log traces files locally. If a phone or gateway is registered to a particular CallManager, then the call is processed on that CallManager if the call is initiated there. You must capture traces on that Cisco CallManager in order to debug a problem. A common mistake is to have devices register on a Subscriber, but capture CUCM Trace Setup & Collection Page 10 June 5, 2013 [CUCM TRACE SETUP AND COLLECTION] traces on the Publisher. These trace files are nearly empty (and most definitely do not have the call in them). Another common problem is to have Device 1 registered to Cisco CallManager 1 and Device 2 registered to Cisco CallManager 2. If Device1 calls Device 2, the call trace is in Cisco CallManager 1; if Device 2 calls Device1, the trace is in Cisco CallManager 2. If you troubleshoot a two-way calling issue, you need both traces from both Cisco CallManagers in order to get the full picture. Include DNs (phone numbers) or IP addresses (if gateways) for all devices involved in the problem. This enables the Technical Support engineer to quickly locate the phones and other devices involved in the problem calls. Know the approximate time of the problem in the traces. Multiple calls are probably made, so it helps to know the approximate time of the call that Cisco Technical Support needs to focus on. CUCM Trace Setup & Collection Page 11