Troubleshoot a Sprint NIC Off−line Issue with ICM Document ID: 23342 Contents Introduction Before You Begin Conventions Prerequisites Components Used Sprint/Cisco Relationship Identify the Problem Solution Related Information Introduction This document guides you through the steps necessary to determine the root cause of messages in the Cisco Intelligent Contact Management (ICM) Sprint Network Interface Controller (NIC) process indicating that one or more SPRCOMM links are CLOSED (off−line). When ICM loses signaling from the Sprint network, the Sprint NIC process on the ICM CallRouter and Event Viewer on an Administration Workstation (AW) display messages similar to this: 06:37:41 SPRCOMM Link 2 to SCP IRVINE 91/1:11:3 CLOSED. 06:38:44 X25Receiver − Unexpected error on link 2 PVC 4(Error 0 hex=0). Re−Establish 06:38:45 PVCEntry − reset_channel failed on link 2 PVC 4 06:38:45 PVCEntry − reset_channel failed on link 2 PVC 1 06:38:45 X25 link (2) PVC (1) reset failed. Detaching. In virtually all cases, this is caused by conditions that are occurring outside of Cisco ICM. Use the steps outlined below to arrive at the root cause. Before You Begin Conventions For more information on document conventions, see the Cisco Technical Tips Conventions. Prerequisites Readers of this document should be knowledgeable of the following: • Cisco ICM • Sprint Network N00−number (example: 700, 800, 900) call routing • Microsoft Windows NT Components Used The information in this document is based on the software and hardware versions below. • Cisco ICM version 4.6.2 and later The information presented in this document was created from devices in a specific lab environment. All of the devices used in this document started with a cleared (default) configuration. If you are working in a live network, ensure that you understand the potential impact of any command before using it. Sprint/Cisco Relationship Sprint's Intelligent Network Service Delivery facility allows customer premise based equipment to participate in the Sprint Network N00−number (example: 700, 800, 900) call routing. A set of Service Control Points (SCPs) in the Sprint network provide the communications function between the Sprint network and customer premise equipment. Customer premise equipment is termed by Sprint as an External Routing Processor, or SiteRP, and is actually Cisco ICM. The SCP is an end node in the Sprint network, and it is responsible for processing N00−number call inquiry requests received from telephone switches throughout the Sprint network. The SiteRP node is an end node located at a customer site to which the SCP redirects inquiry requests. As stated previously, ICM assumes the role of the SiteRP. The SiteRP interface in Cisco ICM is implemented as a Microsoft Windows NT process, known as the Sprint NIC, running on the ICM Central Controller. Cisco ICM receives call inquiries from, and returns inquiry responses (labels) to the Sprint network through the Sprint NIC. There are five SCPs in the Sprint network. In a duplexed ICM environment, each ICM NIC connects to the Sprint network through five 56−kbps point−to−point communication links, one to each SCP, provisioned on the Sprint FiberNet network. The five communications links are five DS0 channels derived from a dedicated T1.5 circuit. Two T1.5 circuits are provisioned on Sprint FiberNet to connect the duplexed ICM to the five Sprint SCPs. The Sprint FiberNet circuit terminating equipment for each side of a duplexed ICM consists of a channel bank−like device called TP7. The terminating equipment is provided by Sprint. Each ICM Sprint NIC contains three Eicon Technology Dual−Port Network Adapter/PC (DPNA) cards. The two ports on a DPNA card are designated as port 1 and port 2, where port 1 is the port closest to the top edge of the card and port 2 is the port closest to the PC connector edge of the card. Five of the six DPNA ports are used to connect to the SCPs. The remaining DPNA port is not used and is disabled. In a simplexed ICM configuration, five DPNA cards are required if the Sprint NIC is connected to the SCPs through redundant links. Cisco supplies five 9−foot cables, each of which connects from a DPNA port to the Sprint circuit terminating equipment using a V.35 interface. The cable has a male DB−26 connector to the DPNA card and a standard male 34−pin V.35 connector to the Sprint circuit terminating device. The communications links are routed to the SCPs in the Sprint network. The physical network interface for the duplexed ICM configuration is shown in Figure 1. Figure 1: Physical Network Interface for a Duplexed ICM Configuration Identify the Problem This problem is noticed when one or more Sprint SCP's are flagged by Cisco ICM as being CLOSED. Below is an excerpt from the sprx.log files on the CallRouter, where x represents Sprint NIC a or b. These log files can be viewed on the CallRouter by looking at the sprx process window or by dumping the sprx.log file using the Dumplog Utiliity. 06:37:41 06:38:44 06:38:45 06:38:45 06:38:45 SPRCOMM Link 2 to SCP IRVINE 91/1:11:3 CLOSED. X25Receiver − Unexpected error on link 2 PVC 4(Error 0 hex=0). Re−Establish PVCEntry − reset_channel failed on link 2 PVC 4 PVCEntry − reset_channel failed on link 2 PVC 1 X25 link (2) PVC (1) reset failed. Detaching. The above code indicates that the Cisco ICM CallRouter is not receiving the necessary signaling from Sprint network facilities. Note: These events also display in the Monitor ICM application. Solution Proceed with the following steps on the Cisco ICM CallRouter you see evidence of a network problem with the Sprint links. These are the quickest steps in determining why the Sprint network is not communicating with ICM. 1. Stop ICM services on the CallRouter where the Sprint NIC processes are experiencing difficulties connecting to the Sprint network. 2. Stop Eicon services on the CallRouter where the Sprint NIC processes are experiencing difficulties connecting to the Sprint network. 3. From a command prompt, run the following: C:\EICON\WSNT>eccard start /E 1 EiconCard Management Utility Copyright (C) Eicon Technology Corporation 1993−1997. All Rights Reserved. EiconCard 1: S94 PCB version 800287.3 EiconCard 1 initialized Activating port(s) ... ECCARD: ERROR #PD342. Port 1. Network status [a0− 0]. Link level is not ready. Port 2. Network connection established. Command partially completed. Error(s) occurred. −−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− According to Eicon Networks Connections for Windows NT Problem Determination−Physical Layer support page, this means that the network has an error. The related excerpt from that page follows: d) [A0−00] Link Level is not ready. This message indicates that the EiconCard is working, and the physical layer is working, but that Layer 2 did not activate correctly. This is most likely a problem with the network or the modem, and needs to be sorted out in conjunction with the PTT. 4. In order to verify that the Eicon card and drivers are installed and operating correctly, run the following from a command prompt: C:\EICON\WSNT>eccard status EiconCard Management Utility Copyright (C) Eicon Technology Corporation 1993−1997. All Rights Reserved. EiconCard Configuration : Card Type ID Status PCB IO Port IRQ Address −−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− 1 S94 20003 LOADED 800287.3 FC60 12 n/a 2 S94 20005 LOADED 800287.3 FC70 20 n/a EiconCard Port mapping : Card Port Port name Protocol −−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− 1 01 PORT1 X25 02 PORT2 X25 2 03 PORT3 X25 04 PORT4 X25 EiconCard Memory status : Card Type Free Total −−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−−− 1 S94 1796 K 2048 K 2 S94 1796 K 2048 K Command completed successfully. Because the Eicon card status command completed successfully, and based on the information from Eicon, you can determine that this issue is caused by Sprint network equipment or configurations which are external to Cisco ICM. In this example, Sprint needs to determine why the X.25 Layer 2 protocol is failing on the given links. 5. When done, restart the Eicon and ICM services. 6. If the results point to a Sprint network problem, open a trouble ticket with Sprint. Otherwise, contact Cisco for further assistance. Related Information • How to Use the Dumplog Utility • Eicon Networks • Technical Support − Cisco Systems Contacts & Feedback | Help | Site Map © 2014 − 2015 Cisco Systems, Inc. All rights reserved. Terms & Conditions | Privacy Statement | Cookie Policy | Trademarks of Cisco Systems, Inc. Updated: Sep 20, 2005 Document ID: 23342