Troubleshooting Guide
for Cisco Unified Customer Voice Portal
7.0(2)
January 2011
Americas Headquarters
Cisco Systems, Inc.
170 West Tasman Drive
San Jose, CA 95134-1706
USA
http://www.cisco.com
Tel: 408 526-4000
800 553-NETS (6387)
Fax: 408 527-0833
THE SPECIFICATIONS AND INFORMATION REGARDING THE PRODUCTS IN THIS MANUAL ARE SUBJECT TO CHANGE WITHOUT NOTICE.
ALL STATEMENTS, INFORMATION, AND RECOMMENDATIONS IN THIS MANUAL ARE BELIEVED TO BE ACCURATE BUT ARE PRESENTED
WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED. USERS MUST TAKE FULL RESPONSIBILITY FOR THEIR APPLICATION OF
ANY PRODUCTS.
THE SOFTWARE LICENSE AND LIMITED WARRANTY FOR THE ACCOMPANYING PRODUCT ARE SET FORTH IN THE INFORMATION PACKET
THAT SHIPPED WITH THE PRODUCT AND ARE INCORPORATED HEREIN BY THIS REFERENCE. IF YOU ARE UNABLE TO LOCATE THE
SOFTWARE LICENSE OR LIMITED WARRANTY, CONTACT YOUR CISCO REPRESENTATIVE FOR A COPY.
The Cisco implementation of TCP header compression is an adaptation of a program developed by the University of California, Berkeley (UCB) as
part of UCBs public domain version of the UNIX operating system. All rights reserved. Copyright 1981, Regents of the University of California.
NOTWITHSTANDING ANY OTHER WARRANTY HEREIN, ALL DOCUMENT FILES AND SOFTWARE OF THESE SUPPLIERS ARE PROVIDED
"AS IS" WITH ALL FAULTS. CISCO AND THE ABOVE-NAMED SUPPLIERS DISCLAIM ALL WARRANTIES, EXPRESSED OR IMPLIED, INCLUDING,
WITHOUT LIMITATION, THOSE OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT OR ARISING
FROM A COURSE OF DEALING, USAGE, OR TRADE PRACTICE.
IN NO EVENT SHALL CISCO OR ITS SUPPLIERS BE LIABLE FOR ANY INDIRECT, SPECIAL, CONSEQUENTIAL, OR INCIDENTAL DAMAGES,
INCLUDING, WITHOUT LIMITATION, LOST PROFITS OR LOSS OR DAMAGE TO DATA ARISING OUT OF THE USE OR INABILITY TO USE
THIS MANUAL, EVEN IF CISCO OR ITS SUPPLIERS HAVE BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.
Cisco and the Cisco Logo are trademarks of Cisco Systems, Inc. and/or its affiliates in the U.S. and other countries. A listing of Cisco's trademarks
can be found at http://www.cisco.com/go/trademarks. Third party trademarks mentioned are the property of their respective owners. The use of
the word partner does not imply a partnership relationship between Cisco and any other company. (1005R)
Any Internet Protocol (IP) addresses used in this document are not intended to be actual addresses. Any examples, command display output, and
figures included in the document are shown for illustrative purposes only. Any use of actual IP addresses in illustrative content is unintentional and
coincidental.
Copyright 2011 Cisco Systems, Inc. All rights reserved.
Table of Contents
Preface ...........................................................................................................................................................1
Purpose .....................................................................................................................................................1
Audience ....................................................................................................................................................1
Organization ..............................................................................................................................................1
Related Documentation .............................................................................................................................2
Conventions................................................................................................................................................3
Obtaining Documentation and Submitting a Service Request...................................................................4
Documentation Feedback...........................................................................................................................4
1. Troubleshooting Overview...........................................................................................................................5
Problem Solving Process...........................................................................................................................5
Guidelines to Assist in Isolating Problems.............................................................................................5
Problem Solving Tasks...............................................................................................................................6
Troubleshooting Checklist.........................................................................................................................11
2. Symptoms and Solutions..........................................................................................................................13
Overview...................................................................................................................................................13
Call Server................................................................................................................................................13
VXML Server............................................................................................................................................38
Reporting Server......................................................................................................................................54
Audio........................................................................................................................................................60
Transfer and Connect...............................................................................................................................77
Unified CM................................................................................................................................................91
Unified ICME............................................................................................................................................91
Content Services Switch..........................................................................................................................98
General System Issues............................................................................................................................98
H.323 Troubleshooting............................................................................................................................105
SIP Troubleshooting................................................................................................................................109
SNMP Troubleshooting...........................................................................................................................117
Installation Troubleshooting....................................................................................................................118
Gateway Troubleshooting.......................................................................................................................124
Cisco Unified Presence Server..............................................................................................................128
Operations Console (OAMP)..................................................................................................................129
Cisco Security Agent (CSA)...................................................................................................................130
Support Tools..........................................................................................................................................131
Backup and Restore...............................................................................................................................131
Appendix A. Error Messages......................................................................................................................135
Infrastructure Error Messages................................................................................................................135
IVR Error Messages...............................................................................................................................141
VMS Error Messages.............................................................................................................................145
Reporting Server Error Messages..........................................................................................................149
ICM Error Messages...............................................................................................................................151
SIP Error Messages...............................................................................................................................154
Appendix B. Event and Cause Codes.........................................................................................................157
IVR Service Error Codes........................................................................................................................157
Unified ICME Error Codes......................................................................................................................159
SIP Codes..............................................................................................................................................161
Appendix C. Troubleshooting Radvision Errors...........................................................................................163
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
i
Appendix D. Accessing and Interpreting SNMP Data.................................................................................165
Appendix E. Internal Interfaces Description................................................................................................169
Appendix F. Trace Bits Description..............................................................................................................171
Appendix G. Debugging..............................................................................................................................175
SIP IOS Debugging................................................................................................................................175
SIP IOS Troubleshooting Commands................................................................................................176
H.323 IOS Debugging............................................................................................................................176
Index ...........................................................................................................................................................179
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
ii
Preface
Purpose
This manual describes how to troubleshoot common issues for the Cisco Unified Customer
Voice Portal (Unified CVP) product. The document also explains the tools and techniques that
can assist in the troubleshooting process.
Audience
This document is intended for Contact Center managers, Unified CVP system managers, VoIP
technical experts, and IVR Service application developers.
Organization
The document is divided into the following chapters.
Chapter
Description
Chapter 1, Troubleshooting
Overview
Describes various methods of approaching troubleshooting tasks.
Chapter 2, Symptoms and
Solutions
Describes particular issues, any associated error messages, the cause of the issue, as
well as the solution.
Appendix A, Error Messages
Provides information on Unified CVP error messages.
Appendix B, Event and Cause
Codes
Provides information on Unified CVP error codes.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
1
Preface
Related Documentation
Chapter
Description
Appendix C, Troubleshooting
Radvision Issues
Describes how to isolate Unified CVP errors from those of Radvision. This appendix
aims at distinguishing these errors, along with an explanation of each error code,
and the appropriate action to take before involving Radvision TAC.
Appendix D, Accessing and
Interpreting SNMP Data
Provides information on SNMP severity levels and their descriptions.
Appendix E, Internal Interfaces
Description
Summarizes the major interfaces between the Unified CVP product components and
other Cisco products.
Appendix F, Trace Bits
Description
Lists the common trace bits, their corresponding values and the description for each
Unified CVP subsystem type. Each value is unique within each Unified CVP
subsystem.
Appendix G, Debugging
Describes SIP troubleshooting for some problems that may occur, and describes
some useful command line commands for H.323 IOS troubleshooting.
Related Documentation
Note: Planning your Unified CVP solution is an important part of the process in setting up
Unified CVP. Cisco recommends that you read the Cisco Unified Customer Voice Portal Release
7.x Solution Reference Network Design (SRND) guide before configuring your Unified CVP
solution. With Unified 7.x, the Planning Guide for Cisco Unified Customer Voice Portal has
been incorporated into the SRND guide.
Unified CVP provides the following documentation:
• Cisco Security Agent Installation/Deployment for Cisco Unified Customer Voice Portal
provides installation instructions and information about Cisco Security Agent for the Unified
CVP deployment.We strongly urge you to read this document in its entirety.
• Cisco Unified Customer Voice Portal Release 7.x Solution Reference Network Design (SRND)
provides design considerations and guidelines for deploying contact center voice response
solutions based on Cisco Unified Customer Voice Portal (Unified CVP) 7.x releases.
• Configuration and Administration Guide for Cisco Unified Customer Voice Portal describes
how to set up, run, and administer the Cisco Unified CVP product, including associated
configuration.
• Element Specifications for Cisco Unified CVP VXML Server and Unified Call Studio describes
the settings, element data, exit states, and configuration options for Elements.
• Installation and Upgrade Guide for Cisco Unified Customer Voice Portal describes how to
install Unified CVP software, perform initial configuration, and upgrade.
• Operations Console Online Help for Cisco Unified Customer Voice Portal describes how to
use the Operations Console to configure Unified CVP solution components.
• Port Utilization Guide for Cisco Unified Customer Voice Portal describes the ports used in
a Unified CVP deployment.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
2
Preface
Conventions
• Programming Guide for Cisco Unified CVP VXML Server and Unified Call Studio describes
how to build components that run on the Cisco Unified VXML Server.
• Reporting Guide for Cisco Unified Customer Voice Portal describes the Reporting Server,
including how to configure and manage it, and discusses the hosted database.
• Say It Smart Specifications for Cisco Unified CVP VXML Server and Unified Call Studio
describes in detail the functionality and configuration options for all Say It Smart plugins
included with the software.
• User Guide for Cisco Unified CVP VXML Server and Unified Call Studio describes the
functionality of Call Studio including creating projects, using the Call Studio environment,
and deploying applications to the Unified CVP VXML Server.
For additional information about Unified ICME, see the Cisco web site Cisco web site (http:/
/www.cisco.com/en/US/products/sw/custcosw/ps1001/tsd_products_support_series_home.html)
listing Unified ICME documentation.
Conventions
This manual uses the following conventions:
Convention
Description
boldface font
Boldface font is used to indicate commands, such as user entries,
keys, buttons, and folder and submenu names. For example:
• Choose Edit > Find.
• Click Finish.
italic font
Italic font is used to indicate the following:
• To introduce a new term. Example: A skill group is a
collection of agents who share similar skills.
• For emphasis. Example: Do not use the numerical naming
convention.
• A syntax value that the user must replace. Example: IF
(condition, true-value, false-value)
• A book title. Example: See the Cisco CRS Installation Guide.
window font
Window font, such as Courier, is used for the following:
• Text as it appears in code or that the window displays.
Example: <html><title>Cisco Systems,Inc. </
title></html>
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
3
Preface
Obtaining Documentation and Submitting a Service Request
Convention
Description
< >
Angle brackets are used to indicate the following:
• For arguments where the context does not allow italic, such
as ASCII output.
• A character string that the user enters but that does not appear
on the window such as a password.
Obtaining Documentation and Submitting a Service Request
For information on obtaining documentation, submitting a service request, and gathering
additional information, see the monthly What's New in Cisco Product Documentation, which
also lists all new and revised Cisco technical documentation, at:
http://www.cisco.com/en/US/docs/general/whatsnew/whatsnew.html
Subscribe to the What's New in Cisco Product Documentation as a Really Simple Syndication
(RSS) feed and set content to be delivered directly to your desktop using a reader application.
The RSS feeds are a free service and Cisco currently supports RSS version 2.0.
Documentation Feedback
You can provide comments about this document by sending email to the following address:
mailto:ccbu_docfeedback@cisco.com
We appreciate your comments.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
4
Chapter 1
Troubleshooting Overview
When troubleshooting a telephony or IP network environment, it is important to define the
specific symptoms, identify all potential problems that could be causing the symptoms, and
then systematically eliminate each potential problem from most likely to least likely until the
symptoms disappear.
This chapter contains the following topics:
• Problem Solving Process, page 5
• Problem Solving Tasks, page 6
• Troubleshooting Checklist, page 11
Problem Solving Process
The following steps provide some guidelines to assist in isolating a particular issue.
Guidelines to Assist in Isolating Problems
Step 1
Analyze the problem and create a clear problem statement. Define symptoms and potential
causes.
Step 2
Gather the facts that you need to help isolate possible causes.
Step 3
Consider possible causes based on the facts that you gathered.
• Was anything recently added, removed, or modified?
• Is it a reproducible event?
• Does it occur at a particular time of day, or day of week?
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
5
Chapter 1: Troubleshooting Overview
Problem Solving Tasks
• Have there been any changes made to the domain, network, or security policies?
Step 4
Create an action plan. Begin with the most likely problem and devise a plan in which you
manipulate only one variable at a time.
Step 5
Implement the action plan, performing each step while testing to see whether the symptom
disappears.
Step 6
Analyze the results to determine whether the problem has been resolved. If it has, the process
is complete.
Step 7
If the problem has not been resolved, create an action plan based on the next most probable
cause on your list, or contact the Cisco Technical Assistance Center (TAC), or your Cisco
Partner.
Note: Only change one variable at a time. If that does not resolve the issue, undo that change
and move on to the next step of your plan.
Problem Solving Tasks
In addition to the process discussed above, there are other areas to consider if a problem arises
within the Unified CVP environment. The information below provides some additional
information to assist in isolating or resolving a particular issue.
• Verify that the software versions comply with the Hardware and System Software Specification
for Unified Customer Voice Portal document. A version incompatibility might not necessarily
be the cause of the issue, but can be investigated and considered as a possibility. This document
is available at:
http://www.Cisco.com/en/US/products/sw/custcosw/ps1006/
prod_technical_reference_list.html
• Review the Unified CVP logs for possible error or warning events. Use the Support Tools
utility since it has the ability to retrieve and process trace logs from most components, and
to set or reset trace levels on those components. To access the Support Tools, open the
Operations Console, and click Support Tools from the Tools menu. The Configuration and
Administration Guide for Cisco Unified Customer Voice Portal and Cisco Support Tools
User Guide for Cisco Unified Software provide additional information about Support Tools.
Note: Users can execute some UNIX equivalent commands, such as more, grep, diff, tail,
tee on Unified CVP servers during troubleshooting. These commands are part of the Support
Tools installation. The PATH environment variable allows these commands to be accessible
via command line.
Unified CVP captures two different types of logs: events and errors. When a Unified CVP
component generates an event, it is logged in the Unified CVP log. When a log statement is
written to the log file and the level is at ERROR or above, it is also written to error log file.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
6
Chapter 1: Troubleshooting Overview
Problem Solving Tasks
Note: This file is created on demand. If there are no ERROR level (or above) logs, you do
not see this file.
Below are the locations of the log files for Unified CVP. In the listings below, CVP_HOME
is the location that was selected for the installation of your Unified CVP software, such as
C:\Cisco\CVP.
– CVP_HOME\logs\
Call Server and/or Reporting Server logs.
– CVP_HOME\logs\OAMP\
Operations Console logs.
– CVP_HOME\logs\VXML\
VXML Server logs.
– CVP_HOME\logs\SNMP\
SNMP Agent logs.
– CVP_HOME\logs\ORM\
Unified CVP Resource Manager logs.
Note: The Operations Console Resource Manager (ORM) is a service that is installed
during the installation of the Operations Console. When installing the Operations Console,
the two Windows services that are installed are Unified CVP OpsConsoleServer and
Unified CVP Resource Manager. The Resource Manager resides on each Unified CVP
server and it allows the Operations Console to communicate with that server and perform
functions on that server, such as starting, stopping, or monitoring.
Virtually all of the Unified CVP logs start a new log file at 12:00AM every night, with the
date as part of the filename. The VXML Server logs do not necessarily start a new file at
12:00AM. The format of the date is yyyy-mm-dd. These logs also start a new log file when
they reach the predefined size limit of 10 MB and have a number as part of the filename
extension. The number indicates which log it was for that day. When the log directory size
reaches the maximum number of megabytes (20,000MB) for disk storage for log files, the
old files are purged as necessary. Both the Log File Size and Log Directory Size are
configurable in the device Infrastructure tab of the Operations Console.
• Check that the network is functioning properly. For the initial connectivity tests, the following
command line tools may be helpful:
– ping - tests whether a host is reachable across an IP network. The ping command offers
several parameters or flags that can be set to produce various outputs. To view all of the
available parameters, type ping /? at the command prompt.
C:\> ping 10.86.132.150
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
7
Chapter 1: Troubleshooting Overview
Problem Solving Tasks
Pinging 10.86.132.150 with 32 bytes of data:
Reply from 10.86.132.150: bytes=32 time=5ms TTL=122
Reply from 10.86.132.150: bytes=32 time=4ms TTL=122
Reply from 10.86.132.150: bytes=32 time=4ms TTL=122
Reply from 10.86.132.150: bytes=32 time=4ms TTL=122
Ping statistics for 10.86.132.150: Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milliseconds: Minimum = 4ms, Maximum = 5ms, Average
= 4ms
C:\>
– traceroute - determines the packet route that is taken to the target host across an IP network.
The command in a Windows environment is: tracert. The tracert command offers several
parameters or flags that can be set to produce various outputs. To view all of the available
parameters, type tracert /? at the command prompt.
C:\> tracert 10.1.1.6
Tracing route to 10.1.1.6 over a maximum of 30 hops
--------------------------------------------------1 2 ms 3 ms 2 ms 10.1.2.1
2 25 ms 83 ms 88 ms 192.168.11.1
3 25 ms 79 ms 93 ms 10.1.1.6
Trace complete.
– netstat - lists the active incoming and outgoing network connections on a system. The
netstat command offers several parameters or flags that can be set to produce various
outputs. To view all of the available parameters, type netstat /? at the command prompt.
C:\>netstat -a -n
Active Connections
Proto Local Address Foreign Address State
TCP 0.0.0.0:42 0.0.0.0:0 LISTENING
TCP 0.0.0.0:88 0.0.0.0:0 LISTENING
TCP 0.0.0.0:135 0.0.0.0:0 LISTENING
TCP 0.0.0.0:389 0.0.0.0:0 LISTENING
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
8
Chapter 1: Troubleshooting Overview
Problem Solving Tasks
TCP 0.0.0.0:445 0.0.0.0:0 LISTENING
TCP 0.0.0.0:593 0.0.0.0:0 LISTENING
TCP 0.0.0.0:1038 0.0.0.0:0 LISTENING
TCP 0.0.0.0:1041 0.0.0.0:0 LISTENING
TCP 0.0.0.0:1048 0.0.0.0:0 LISTENING
TCP 0.0.0.0:1723 0.0.0.0:0 LISTENING
TCP 0.0.0.0:3268 0.0.0.0:0 LISTENING
TCP 10.99.99.1:53 0.0.0.0:0 LISTENING
TCP 10.99.99.1:139 0.0.0.0:0 LISTENING
TCP 10.99.99.1:389 10.99.99.1:1092 ESTABLISHED
TCP 10.99.99.1:1092 10.99.99.1:389 ESTABLISHED
TCP 10.99.99.1:3604 10.99.99.1:135 TIME_WAIT
TCP 10.99.99.1:3605 10.99.99.1:1077 TIME_WAIT
UDP 0.0.0.0:135 *:*
UDP 0.0.0.0:445 *:*
UDP 0.0.0.0:1087 *:*
UDP 10.99.99.1:53 *:*
UDP 10.99.99.1:137 *:*
UDP 10.99.99.1:138 *:*
• Using the Windows Services control panel, verify that all appropriate services are running
for that server. To view the Service control panel, go to Start and select Run. In the Run
window, type services.msc and click OK.
• Check the Application and System logs in the Windows Event Viewer for warning or error
events. To start the Event Viewer, click Start and select Run. In the Run window, type
eventvwr.exe and click OK. This launches the Event Viewer console, which displays the
three logging options (Application, Security, and System).
• A device may be in some state other than Up, such as Down, or Not Reachable. Check the
Operations Console to verify the status of a particular device. To check the status:
– From the Operations Console, select System > Control Center.
– Click the Device Pool tab in the left frame of the page.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
9
Chapter 1: Troubleshooting Overview
Problem Solving Tasks
– In the right frame of the page, click the General tab if it is not already selected.
– Locate the device you are interested in and check the status of that device in the Status
column.
• Check the call statistics from the Operations Console to try to determine any abnormalities.
The reported values from the various components might provide some information that will
assist you in isolating the issue.
Note: The Operations Console online help describes the device status fields in detail. From
the console, choose System > Control Center and then select the Device Pool tab and choose
a device pool. You can also view a particular type of device by selecting the Device Type
tab and choosing a device type. The status of some devices such as, IOS devices, Unified
CM, ICM servers, and SIP proxy servers display as N/A (Not Applicable) because the
Operations Console does not monitor these device types.
You can access call statistics either by:
– Choosing System > Control Center, selecting a Call Server, clicking Statistics icon in
the toolbar, and then selecting the various tabs (Unified ICME, SIP, IVR, or Infrastructure).
– Choosing Device Management > CVP Call Server, selecting a Call Server, clicking the
Statistics icon in the toolbar, and then selecting the various tabs (Unified ICME, SIP,
IVR, or Infrastructure).
The examples above list the procedure for viewing the statistics for a Call Server. The statistics
for other devices, such as the Reporting Server, VXML Server, and so forth can be viewed
in the same manner.
Below are some of the initial areas to check in the Infrastructure tab for abnormal values or
thresholds.
– Check for licensing issues in the Infrastructure tab. The Licensing Statistics column is
broken up into Realtime, Interval, and Aggregate sections. Check the value being reported
for Port Licenses Available against the value of Maximum Port Licenses Used.
– Check the level of the Current Port Licenses State. This reports on the threshold level of
port license usage. There are four levels: safe, warning, critical and failure. An administrator
may set the required percentage of port licenses in use needed to reach a given threshold
level, with the exception of the failure level which is reached when the number of ports
checked out is equal to the total number of licenses ports.
– Check the Uptime statistic, as well as the memory and thread pool statistics for possible
issues. If the Uptime statistic is low, then that indicates that the component was recently
unavailable for some period of time. High memory or thread pool statistics may indicate
a low available resource issue.
Below are some of the initial areas to check in the SIP tab for abnormal values or thresholds.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
10
Chapter 1: Troubleshooting Overview
Troubleshooting Checklist
– In the Interval Statistics section, check the value of the Failed SIP Transfers (Pre-Dialog).
This is the total number of failed transfers on the first Unified CVP transfer in the last
statistics aggregation interval. A SIP dialog is established after the first Unified CVP
transfer is completed. The metric does not include rejections due to the SIP service being
out of service. The metric includes failed transfers that were made after a label was returned
from the Unified ICME in a CONNECT message.
– Check the value of the Failed SIP Transfers (Post-Dialog). The total number of failed
transfers on second and subsequent Unified CVP transfers in the last statistics aggregation
interval. After SIP dialog is established, Unified CVP performs SIP transfers using
re-INVITE messages. The metric does not include rejections of SIP REFER transfers. The
metric includes failed transfers that were made after a label was returned from the Unified
ICME in a CONNECT message.
Below are some of the initial areas to check in the IVR tab for abnormal values or thresholds.
– Check the Active Calls statistic. This is the number of active calls being serviced by the
IVR service currently. Refresh the page to determine if the calls are increasing.
– Check the Average, Maximum, and Minimum Call Latency statistics. These report the
amount of time in milliseconds it took for the IVR Service to complete the processing of
a New Call Request or a Request Instruction Request during the time interval. Long
durations may indicate a network or load problem.
Also, verify that your network environment has been correctly configured. The table below
shows the speed and duplex settings for Unified CVP.
Ethernet Switch Speed
Capability
Server/Gateway NIC Speed Switch Port Speed/Duplex Server/Gateway NIC
Capability
Setting
Speed/Duplex Setting
1000 Mb
1000 Mb
Auto / Auto
Auto / Auto
1000 Mb
100 Mb
100 Mb / Full
100 Mb / Full
100 Mb
100 Mb
100 Mb / Full
100 Mb / Full
100 Mb
1000 Mb
100 Mb / Full
100 Mb / Full
If you have exhausted all the common causes and actions including those outlined in this
document or others that you have identified in your environment, contact the Cisco Technical
Assistance Center (TAC) or your Cisco Partner.
Troubleshooting Checklist
Complete this checklist to assist in isolating the issue, or to provide information to your support
partner or Cisco Technical Support.
1. What is the version of Unified CVP that is currently running? Include any Hotfix or Service
Release (SR) information.
2. Is this a new installation or an upgrade?
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
11
Chapter 1: Troubleshooting Overview
Troubleshooting Checklist
3. If this is an upgrade, what version was previously installed?
4. When did the problem occur?
5. What are the observed symptoms, and the conditions under which these symptoms occur?
6. Was anything changed or updated in hardware, software, or network components prior to
the first occurrence of the observed symptoms?
7. Describe the related call flow. Some examples include: Public Switched Telephone Network
(PSTN) originated or IP Phone originated. If it is a transfer, is it Unified CVP dequeue or
agent originated? Is the call inbound ACD, inbound IVR Service treatment, inbound queue
treatment, outpulse transfer, or agent transfer?
8. Is the problem reproducible?
9. What Unified CVP deployment model used for deployment?
10. What is the call transfer method used?
11. Are you able to capture a screen shot of the error or failure? If Yes, save it to a file and
attach to a case.
Network Topology
Complete this checklist to assist in isolating the issue, or to provide information to Cisco
Technical Support.
1. Has auto-negotiate been disabled on all PCs, routers, and switch ports?
Note: Duplex/speed mismatch between a device and its corresponding port on the switch
is the single most common problem for network latency.
2. Is a network topology diagram available?
3. Which type of IP Gateway is being used in this Unified CVP solution?
4. Save the output from a show tech, show stack, and a show Gatekeeper endpoint from
the Gateway.
5. Which type of Gatekeeper is being used in this Unified CVP solution?
6. On which server are the recorded media files located, and what is the path to those files?
7. Are Content Service Switches (CSS) used for failover?
8. Collect and provide versions of IOS, Digital Signal Processor (DSP) , applications, and
Engineering Special (ES)/patch levels in the environment.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
12
Chapter 2
Symptoms and Solutions
Overview
This chapter contains problem descriptions and steps to diagnose and resolve each problem.
Problem descriptions are grouped by Unified CVP component.
Call Server
This section contains problems associated with the Call Server.
Device Becomes Corrupted
Symptom:
A device has become corrupted. The user receives a specific error message that indicates what
device has been corrupted.
Message:
"Device with IP Address: {1} and Hostname: {2} is in an inconsistent state. Please Save and
Deploy the properties of this Server again to fix the problem. If problem persists, please contact
your administrator for more details."
"Device with IP Address: {1} and Hostname: {2} is not ready at this time. Please retry the
operation at a later time."
"Device with IP Address: {1} and Hostname: {2} is in an inconsistent state and cannot be
properly deleted at this time. Please retry the operation later or try deleting again for forceful
delete."
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
13
Chapter 2: Symptoms and Solutions
Call Server
Cause:
The most likely cause is that the Unified CVP software was reinstalled.
Action:
The following steps outline the process to resolve the issue of software being reinstalled on a
server.
1. Identify the affected devices from the Control Center of the Operations Console. The
affected devices are:
- devices with the same IP addresses, and
- devices associated to the devices identified with the same IP address.
2. Go to the Edit page of each device and click Save&Deploy.
3. Restart all affected devices from Control Center in the Operations Console.
If the procedure above does not resolve the issue, then you must clean up the device by
performing the following steps.
Device Clean Up
1. Delete the device from the Operations Console.
2. Reinstall the software on that device.
3. Recreate the device from the Operations Console.
Each Operations Console Server will be identified by a unique identifier. The identifier is
generated at the time of creating a first device through that Operations Console Server. Once
generated, the identifier will then be stored in the Operations Console Server database and in
the Resource Manager Configuration file on every device that includes the Operations Console
Server. The purpose of this unique identifier is to prevent management of a device from multiple
Operations Consoles.
The unique identifier gets imported whenever the import functionality of an Operations Console
database is invoked. Importing a database which was exported from a different Operations
Console Server is not a supported feature unless the Operations Console is newly installed or
has never been used before.
If a user uses an Operations Console Server that is already managing at least one device to
import a database (from a different Operations Console Server or an empty database which was
never used to create a device) the following side effects will occur:
User may not be able to manage devices which were managed by the same Operations Console
server prior to the import. This is because the unique identifiers will now be different between
the Operations Console Server and the previously managed device. User may not be able to
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
14
Chapter 2: Symptoms and Solutions
Call Server
create an IOS device such as Gatekeeper(s), Gateway(s), and CSS(s) from the Operations
Console.
An Operations Console Server that is managing at least one device must not import an empty
database that was previously exported from any Operations Console Server. This will cause a
mismatch in the unique identifiers used since the Operations Console database and the Operations
Console Server have a unique identifier and the empty database has no identifier. If a customer
requires replacing the existing database with an empty database, the recommended way is to
reinstall the Operations Console Server software. If this step is not followed, the customer can
no longer manage devices that were previously managed by the same Operations Console Server.
SSL Recovery
Symptom:
The Operations Console is unable to communicate with the Operations Console Resource
Manager (ORM) service on a device.
Message:
Call Server device with IP Address: <x.x.x.x> and Hostname: <server1> operation failed. Device
could not be reached because of a failure in negotiating security certificates. Please refer to
documentation for configuring security before trying the operation again.
Cause:
If the user checks the secure communications checkbox in Operations Console device
configuration without first configuring security certificates as outlined in Chapter 6 of the
Configuration and Administration Guide for Cisco Unified Customer Voice Portal, the Operations
Console will not be able to communicate with the Operations Console Resource Manager on
the device.
Action:
The solution is to edit the orm_jmx.properties in the conf directory of the device. Change the
property com.sun.management.jmxremote.ssl to false to put the device back into
non-secure mode. Next, uncheck the checkbox in the Operations Console and restart the
Operations Console Resource Manager service on the device. This will put both the Operations
Console and the device Resource Manager back into a non-secure mode. The user can then
configure security certificates before checking the secure communications checkbox.
Note: Changes in security settings require that the Operations Console service and the Resource
Manager service be restarted.
Resource Manager Service Fails to Start
Symptom:
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
15
Chapter 2: Symptoms and Solutions
Call Server
Since the default log setting is ERROR for the Resource Manager Wrapper, the log file should
provide exception details of the failure. The wrapper log file does indicate there is a failure in
Resource Manager service startup, but the details of the event are not logged.
Message:
None.
Cause:
The wrapper log was set to the wrong level.
Action:
The change is unable to be made from Support Tools, so the change requires manual steps to
change this setting. The log file to look for the exception is
%CVP_HOME%\logs\orm_wrapper.log.
1. Stop the 'Cisco CVP Resource Manager' service.
2. Edit %CVP_HOME%\conf\wrapper.conf file to change the property
'wrapper.logfile.loglevel' value from 'ERROR' to 'INFO'.
3. Save and close the file.
4. Start the 'Cisco CVP Resource Manager' service.
If the Resource Manager service fails to start, the ORM logs (orm_wrapper.log) collected from
Support Tools will show the detailed information of the event.
Call Server Is Not Reachable
Symptom:
After a forced delete of the Reporting Server, the Call Server state did not change from the
Down state to the Partial or Up state.
Message:
Opsconsole Control Center: Call Server status is down.
Cause:
The Call Server messageAdapter.properties file is corrupt.
Action:
1. From the command line, run %CVP_HOME%\bin\tac\reimage.bat on the Call Server.
2. Restart the Operations Console Resource Manager (ORM) service.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
16
Chapter 2: Symptoms and Solutions
Call Server
3. Log in to the Operations Console, and choose Device Management > CVP Call Server.
4. Click Save & Deploy to save the changes and apply them to the Call Server.
Note: All co-located devices—including VXML Server, Reporting Server, and Video
Media Server—must be deployed after reimage.bat is run. You must repeat these steps
for each co-located device.
Note: Optionally, you can reinstall the CVP device.
Call Server Cannot Communicate with Co-located Reporting Server
Symptom:
The environment:
Call Server 1 (primary) & Reporting Server 1.
Call Server 2 (secondary)
In the above deployment, when the Operations Console Resource Manager (ORM) on Call
Server 1 is down, the ORM on Call Server 2 is unable to communicate with the Reporting
Server.
Message:
None.
Cause:
This is not a supported model for deployment.
Action:
A co-located Reporting Server can never scale to support multiple Call Servers and must not
be configured in that manner.
Call Server or VoiceXML Service Cannot Be Restarted
Symptom:
After applying license files, the Call Server service or the VoiceXML service are unable to be
restarted. Both services can be stopped, but receive a warning message when trying to restart
them.
Message:
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
17
Chapter 2: Symptoms and Solutions
Call Server
Windows could not start the Unified CVP Call Server/VXML Server on Local Computer. For
more information, review the System Event Log. If this is a non-Microsoft service, contact the
service vendor, and refer to service-specific error code 0.
Cause:
The problem is due to the side effect of the first version (1.0) of the MS06-040 patch (KB921883):
Action:
Apply the new version of the Microsoft security patch KB921883.
See Also
Refer to the Microsoft Security Bulletin KB921883 that is available on the Microsoft web site.
Go to: http://support.microsoft.com/
Connection with PIM Dropped, Call Server Going Out of Service
Symptom:
The TCP connection between ICM VRU PG and CVP CallServer ICM subsystem was dropped.
• Calls are dropped during load conditions.
• Wireshark can not capture H.323 messages when a call processing load is running on the
Call Server.
Message:
The following errors are written to the CVP error log file when either the connection between
the VRU PIM and CVP drops for an unknown reason, or the VRU PIM is intentionally shut
down for maintenance:
• LOGMSG_ICM_SS_PIM_SHUTDOWN: Lost socket connection to VRU PIM. Transitioning
to partial service.
• LOGMSG_ICM_SS_STATE: Waiting for VRU PIM Connection.
Cause:
The TCP Chimney feature is an offload engine that was introduced in Windows Server 2003
Scalable networking Pack.
Action:
Disable the TCP Chimney feature.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
18
Chapter 2: Symptoms and Solutions
Call Server
1. Open Registry Editor, and locate
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters
2. Set the EnableTCPChimney parameter to 0 .
3. Reboot the machine (required).
H.323 Service Cannot Access Call Server
Symptom:
Some HTTP requests from the Unified CVP Voice Browser to the Call Server are being blocked.
Message:
Blocked by port blocking rule VoiceBrowser.exe Prevent IRC communication.
Blocked by port blocking rule tomcat5.exe Prevent IRC communication.
Cause:
VirusScan is blocking ports in the range of 6666-6669, which are used in IRC communications.
The application does not have to be listening on these ports for it to be blocked. If the application
is assigned a client port in that range, the attempt to connect to a server using that port is blocked
by VirusScan on the server machine. The connect attempt fails.
Action:
Check the VirusScan logs for error messages. access the logs,VirusScan Console -> Task ->
View Log You might find entries like these: 10/10/2006 3:15:01 AM Blocked by port blocking
rule VoiceBrowser.exe Prevent IRC communication 10.86.129.138 10/10/2006 8:49:01 PM
Blocked by port blocking rule tomcat5.exe Prevent IRC communication 10.86.129.211.
To view VirusScan logs
1. Go to Start > Programs > Network Associates > VirusScan Console.
2. Click on the Task menu and select View Log.
You might find entries such as:
Blocked by port blocking rule VoiceBrowser.exe Prevent IRC communication.
Blocked by port blocking rule tomcat5.exe Prevent IRC communication.
INFO: Retrying communication with Call Server localhost:8000
INFO: Retrying communication with Call Server localhost:8000
Voice Browser to Call Server localhost:8000 connection unsuccessful.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
19
Chapter 2: Symptoms and Solutions
Call Server
Voice Browser cannot access any Call Servers and is no longer accepting calls.
Voice Browser <VB machine name here> is out of service.
Exclude the Unified CVP processes from port blocking when using McAfee VirusScan on a
system running Unified CVP Call Server. If a different anti-virus product is used, do the
equivalent exclusions for the port blocking rules for that product.
Instructions for McAfee VirusScan on Unified CVP Call Server.
1. Open the VirusScan Console.
2. Click Access Protection.
3. Click Properties under the Task menu.
4. Click the Port Blocking tab.
5. Check both Prevent IRC Communication rules.
6. Click the first Prevent IRC Communication rule and click Edit.
7. Enter VoiceBrowser.exe,tomcat5.exe to the Excluded Process field.
8. Click OK.
9. Click the second Prevent IRC Communication rule and click Edit.
10. Enter VoiceBrowser.exe,tomcat5.exe to the Excluded Process field.
11. Click OK.
Resetting System Clock Stops IVR Service Requests
Symptom:
Resetting the system clock on the Call Server causes the IVR Service to stop functioning.
Message:
None
Cause:
Changing the time of the system clock on the Unified CVP Call Server causes the IVR Service
to stop accepting calls.
Action:
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
20
Chapter 2: Symptoms and Solutions
Call Server
Do not reset the Windows system clock on a machine running Unified CVP. Resetting the
Windows system clock is not supported on a Call Server.
IVR Service CALL_RESULT Returns Error Code Other than 0
Symptom:
The IVR Service sends a CALL_RESULT error code other than a zero (0) to the Call Server
log.
Message:
Below is an example of one of the messages:
4765560: 10.86.129.211: Aug 18 2007 06:09:59:484 -0400:
%TEMP:CVP7.0_IVR-7-TRACE_CALL: {Thrd=http-8000-3} VBServlet:service: Request
from 10.86.129.20: {CALL_ID=979679B9-2DD811DB-B72D0014-6944B762,
MSG_TYPE=CALL_RESULT, CALL_SEQ_NUM=2, ERROR_CODE=32}
Cause:
The problem is due to an error with the Run Script Request from Unified ICME.
Action:
Check the Error Code that was received in the Event Codes and Reasons appendix of this
document, and take the appropriate action. For additional information and a complete list of the
cause codes, refer to the Cause Codes and Debug Values section of the Cisco IOS Voice
Troubleshooting and Monitoring Guide.
Call Server Is Unreachable
Symptom:
The Call Server is unreachable.
Message:
UNREACHABLE
Cause:
A configured component is unreachable in any one of the following three conditions:
• Operations Console is unable to connect to the Operations Console Resource Manager that
is co-located with the Call Server. For example, the Operations Console Resource Manager
is down.
• The Operations Console Resource Manager on the Call Server has not received a state event
from the controller of that component.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
21
Chapter 2: Symptoms and Solutions
Call Server
• The Operations Console Resource Manager is unable to connect to the Call Server so it is
not receiving state events from the central controller. For example, the Call Server is down.
Action:
In this example, all three services (Unified ICME, IVR, and SIP) have been configured. The
central controller reports that IVR Service and SIP are IN_SERVICE, but it does not report the
status of the Unified ICME Service to the Operations Console for some reason. When this
occurs, Operations Console reports the status of Unified ICME as UNREACHABLE. Operations
Console aggregates the status of the various components of the device to arrive at device status.
It sees that two of the components (IVR Service and SIP) are in SERVICE, but Unified ICME
is UNREACHABLE. Operations Console shows the status of the Call Server as
UNREACHABLE even though IVR Service and SIP are actually in service.
Check the network environment for connectivity, as well as the actual state of the server.
Received ERROR_SCRIPT_NAME = 6 on IVR Service
Symptom:
The ERROR_SCRIPT_NAME = 6 error code is received on the IVR Service.
Message:
ERROR_SCRIPT_NAME = 6
Cause:
The VRU Script Name data that is passed from the Unified ICME to the IVR Service does not
contain the expected components. These components are micro-application name, media file
name, and media file type.
Action:
In the Unified ICME Network VRU Script List tool, locate the appropriate script and resolve
the configuration error.
See Also
For more information, refer to the Micro-Application configuration section of the Unified CVP
Configuration and Administration Guide.
Invalid Path or File Name
Symptom:
Note: This is a Call Server issue that is found within the Video Media Server (VMS), which is
a part of the Call Server.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
22
Chapter 2: Symptoms and Solutions
Call Server
Sometimes, a file is read from the VMS movies directory with a name that does not validate to
the standard accepted character set. These files are marked with the "File could not be
read" status.
Message:
File could not properly be read from the file system.
Cause 1:
An invalid status for a media file stating that the "file could not be read", generally
implies that the file has an invalid path or file name.
Action 1:
Verify and correct the path or file name.
Cause 2:
An invalid status for a video media file (that is a 3gp or an XML file) stating that the "file
could not be properly read", generally implies that the file has an invalid file name or
presents within an invalid path.
Action 2:
Verify and correct the path or file name.
The file names can contain alphanumeric characters, underscore, spaces, and the following
special characters: !$&().,'-;@[]^_`{}~+=. File names cannot lead with a period (.) or a space.
Only path names with alphabetical, numeric, spaces, and !$&().,'-;@[]^_`{}~+= characters are
permitted; path names cannot lead or end with a period (.) or a space.
“Dialogue Failure Event” Error in the Call Server Log Files
Symptom:
Callers hear critical error media and “Dialogue Failure Event” errors appear in the Call Server
log files.
Message:
None.
Cause:
If a call reaches the end of ICM script processing without being queued or released, the Unified
ICME sends a “Dialogue Failure Event” message to the Call Server. The Call Server, in turn,
sends an error message to the H.323 Service.
Action:
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
23
Chapter 2: Symptoms and Solutions
Call Server
The author of the script must ensure that each path in the script ends with one of the following
Script Editor nodes: Release, Label, Skill Group, or Queue to Skill Group.
Call Server “Variable Data Is Invalid” Error
Symptom:
The Call Server is reporting an error.
Message:
Variable Data is Invalid.
Cause:
The ICM script might truncate values with leading zeros or zeros after decimals or rounds
values.
Action:
Put quotes around numbers in a Script Editor Set node so it is processed as a string. This is
especially important if:
• Leading zeros are present (example: dates).
• Trailing zeros are present after a decimal point (example: currency).
• The number is very large (example: a number normally expressed through exponential
notation).
Unable Add IOS Device Because Device Is Already Managed
Symptom:
When attempting to add a device to the Operations Console, an error message is received.
Message:
Gateway device with IP Address: <ip address> and Hostname: <hostname> cannot be created.
The device is already managed by another Operations Console.
Cause:
This occurred because a database was imported from a different Operations Console. Each
Operations Console Server is identified by a unique identifier. The identifier is generated at the
time of creating a first device through that Operations Console Server. Once generated, the
identifier will then be stored in the Operations Console Server database and in the Resource
Manager Configuration file in every device that includes the Operations Console Server. The
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
24
Chapter 2: Symptoms and Solutions
Call Server
purpose of this unique identifier is to prevent multiple Operations Console Servers from
configuring the same device.
The side effects of importing a database to an Operations Console Server that is managing at
least one device, if the database to be imported is an exported database from a different Operations
Console Server or is an empty database which was never used to create a device are:
1. User may not be able to manage devices which were managed by the same Operations
Console server prior to the import.
2. User may not be able to create an IOS device such as Gatekeeper(s), Gateway(s), and
CSS(s) from the Operations Console.
Importing a database which was exported from a different Operations Console Server is not a
supported feature. In this case the import may succeed but will have various side effects including
the above two side effects.
Action:
Importing a database which was exported from a different Operations Console Server is not a
supported feature. The solution is to never import an empty database to an Operations Console
Server, if the server is managing at least one device prior to import. If customers require replacing
the existing database with an empty database, the customer is to reinstall the Operations Console
Server software. Doing so, the customer can no longer manage devices that were previously
managed by the same Operations Console Server.
VRU Application Error in Call Server Log
Symptom:
The call might not go through the Call Server, so a prompt is not played. An error or warning
message might appear in the Call Server log.
Message:
VRU APPLICATION ERROR: Assigning the tag 7 to the non-existing ECC variable
“user.microapp.error_code”.
Cause:
The ECC Variable is not configured on the Unified ICME and/or NAM software, or the defined
length is not the same on both devices.
Action:
Add the identical ECC variable definition to the Unified ICME and/or NAM.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
25
Chapter 2: Symptoms and Solutions
Call Server
ECC Variable Contents Truncated after Passing through Call Server
Symptom:
The contents of an ECC Variable configured in Script Editor are truncated after passing through
the Call Server. For example, the Text-To-Speech (TTS) text that is spoken to the caller is not
the complete text that was configured. Another example of this behavior is where there are
media fetch failures and the URL to the media file is only a subset of the expected URL.
Message:
None.
Cause:
The cause of this problem may be that the length of the ECC Variable value that was set in
Script Editor is longer than the maximum length of the ECC Variable configured at setup time.
Action:
The solution is to make the maximum length of the ECC Variables longer. This is done using
the Unified ICME Configuration Manager. If you do alter the maximum length of ECC Variables,
you need to restart the Call Server after making the change.
Note: In a NAM/ICM environment, the length needs to be identical on all NAM and Unified
ICME environments or the variable will not pass.
"Have You Configured Your Call Server?" Error
Symptom:
The Call Server is running, but the an error message is logged in the log file.
Message:
Have you configured your Call Server?
Cause:
The Call Server was started from the Services Control Panel, but it was not configured from
the Operations Console.
Action:
Configure the Call Server from the Operations Console. After completing the configuration,
restart the Call Server from the Services Control Panel.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
26
Chapter 2: Symptoms and Solutions
Call Server
Erratic Behavior When Everything Is Configured Properly
Symptom:
General erratic behavior when everything is configured properly. Sometimes TTS plays,
sometimes it does not, or the call stops when the VXML Server is running in script node in
Unified ICME. Other behaviors include calls succeed in Unified ICME, but no TTS plays, or
the caller hears silence, but no errors are reported in the log. Unavailable TTS errors are reported.
Message:
None.
Cause:
If everything appears to be configured properly, and behavior is generally erratic, check the
configuration of your Ethernet link.
Action:
Do the following:
• Check the Gateway log files.
• Check the VXML Server application logs.
• Verify the network settings. The chart below shows the speed and duplex settings for Unified
CVP.
Table 1: Speed and Duplex Settings
Ethernet Switch Speed
Capability
Server/Gateway NIC Speed Switch Port Speed/Duplex Server/Gateway NIC
Capability
Setting
Speed/Duplex Setting
1000 Mb
1000 Mb
Auto / Auto
Auto / Auto
1000 Mb
100 Mb
100 Mb / Full
100 Mb / Full
100 Mb
100 Mb
100 Mb / Full
100 Mb / Full
100 Mb
1000 Mb
100 Mb / Full
100 Mb / Full
IVR Subsystem Goes into Disabled State When Call Server Service Starts
Symptom:
When the Call Server service is started, the IVR subsystem goes into a Disabled state.
Message:
Cause 1:
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
27
Chapter 2: Symptoms and Solutions
Call Server
The Unified CVP license in use is a non-expiring license (no expiration date).
Action 1:
Use a Unified CVP license that has an expiration date
Cause 2:
If the Unified CVP license in use has an expiration date, the Unified CVP Call Server service
need to be recycled.
Action 2:
Recycle the Unified CVP Call Server Service. After recycling, check to see if the IVR subsystem
is not in a Disabled state. If it is disabled, the process needs to be repeated a few times.
IVR Subsystem Returns a 500 Rejection on VXML Fetch from Gateway
Symptom:
IVR Subsystem returns a 500 rejection on VXML fetch from the gateway as seen in the gateway
or CVP logs.
Message:
Cause:
Incorrect setting of parameter "Max Length of DNIS" on ICM service configuration page of
Operations Console.
Action:
Set parameter to the same length as the DNIS part of the label returned by ICM.
Full Video Call Disconnected When Routed Through IVP
Symptom:
Full video call is disconnected when it is routed through IVP.
Message:
Cause:
Access number DN prefixes are not added to the iCONTACT application and/or the iCONTACT
xml configuration.
Action:
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
28
Chapter 2: Symptoms and Solutions
Call Server
See Chapter 17 in the Configuration and Administration Guide for Cisco Unified Customer
Voice Portalfor information on modifying the Access Number in the iCONTACT xml file.
SIP Code 488 or 603 Returned on Full Video Call Rejection
Symptom:
SIP code 488 or 603 is returned on a full video call rejection from IVP. A Fast Busy is heard.
Message:
Cause:
MCU or MVP/EMP unit on IVP is not registering or is in service.
Action:
Follow the steps in the Radvision documentation to place the MCU or MVP in service.
Full Video Warm Transfer with Queuing Not Working
Symptom:
SIP trunk on outgoing call to VRU label is not getting routed to IVP.
Message:
Cause:
Action:
Set the SIP trunk on the route pattern for the ICME VRU label to point to IVP instead of CVP.
CVP Warm Transfer With Queuing Get Disconnected
Symptom:
CVP warm transfer with queuing gets disconnected after a few seconds when the transfer is
complete. CUCM logs and/or sniffer trace show a reinvitation collision scenario. This only
occurs with IP originated callers, not TDM callers.
Message:
Cause:
This problem is visible when CVP and CUPS are using proxy and UDP on the outgoing direction,
and the CUCM SIP trunk is using TCP on the outgoing direction.
Action:
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
29
Chapter 2: Symptoms and Solutions
Call Server
Set UDP on the CUCM SIP trunk security profile or set CVP and CUPS to use TCP.
Hold Movie not Visible to Caller on Hold
Symptom:
Note: This tip applies to full video only.
The caller does not see a hold movie when agent places the caller on hold.
Message:
Cause 1:
Mis-configuration of ECC variables that control the hold movie to be played to caller.
Action 1:
Make sure the user.microapp.media_server and user.cvpmovies_bg_media ECC variable are
configured correctly via the "Set Variable" node in the ICM script: for example,
user.microapp.media_server = "rtsp://10.86.129.233" user.cvpmovies_bg_media="app/hold.3gp"
Cause 2:
The default hold movie is not configured in the
/home/rv/ivp/applications/iCONTACT/conf/iCONTACTConfiguration.xml file.
Action 2:
Verify if a default hold movie is configured in the
/home/rv/ivp/applications/iCONTACT/conf/iCONTACTConfiguration.xml file as follows:
<leg-configuration>
<type>agent</type>
<bg-media>rtsp://<IP addr of VMS>/en-us/app/hold.3gp</bg-media>
</leg-configuration>
Specify the RTSP URL (as shown above) or as a file:// URL (if file stored locally on IVP
box). (e.g. file:///home/rv/movies/local/hold.3gp). 1
Full Video Calls Drop upon Dialing the DNIS
Symptom:
1)
A Linux server in which the IVP software from Radvision is loaded is refered to as an IVP system. IVP systems are used as part of the Unified CVP Release
7.x solution.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
30
Chapter 2: Symptoms and Solutions
Call Server
Upon dialing the Dialed Number Identification Service (DNIS), full video calls are immediately
dropped.
Message:
none
Cause 1:
The connection between Unified CVP and the Interactive Video Platform (IVP) system from
Radvision (used as part of the Unified CVP solution) may not be established.
Action 1:
Verify the following requirements:
• The Call Server status is functioning (up) in the OAMP control center.
• A valid connection exists between Unified CVP and IVP via Diag Servlet. To verify, click
on Dump IVR subsystem and look for the following ContactClient statements:
ContactClientManager: video enabled: true
ContactClient count: 1
ContactClient [192.168.150.170]:9981, connectionId:iCONTACT-Connection-1
• Look for following log message in CVP log: 2472: 192.168.150.173: Sep 24 2007
11:35:25.500 -0400: %CVP_7_0_IVR-1-VIDEO_CONNECTION_UP: Connection to
iContact server [192.168.150.170]:9981 is UP [id:3100]
Cause 2:
The access numbers and/or prefix numbers have not been configured for iCONTACT in IVP
Manager and/or iCONTACT configuration file.
Action 2:
1. Configure the access numbers and/or prefix numbers for iCONTACT using IVP Manager.
2. Configure the same access numbers and/or prefix numbers in the iCONTACT configuration
file.
3. Restart IVP after making these changes.
Undecipherable Message Is Displayed When the VMS Subsystem Is Down
Symptom:
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
31
Chapter 2: Symptoms and Solutions
Call Server
When playing a video to a caller, the DarwinStreamingServer.exe is terminated. The video stops
playing and an error message is recorded in the VMS Error Log.
Message:
340: 10.30.30.97: Oct 25 2007 16:59:46.267 -0400:
%CVP_7_0_VMS-3-VMS_SUBSYSTEM_STREAMING_SERVER_DOWN: The Streaming
Server State: UNKNOWN. [id:11512]
Cause:
Note: There are three internal states outputted by the VMS Darwin Streaming Server keep-alive
thread which help drive the overall state of the VMS. These are internal states, not necessarily
representative of the state of the VMS as a whole.
The appropriate error message for an agent to see when VMS detects Darwin is down is "VMS
Subsystem is down". You may see a different error message if the streaming server poller
is in between cycles when the VMS Subsystem goes down. This error message is accurate as
it is the output from the keep-alive thread, which tracks the VMS Subsystem through three
states: IN_SERVICE, PARTIAL_SERVICE and UNKNOWN. These are not the states that the
VMS Subsystem uses.
Action:
To see the appropriate error message, wait for one full polling cycle (default: 30 sec) to ensure
that the proper error message appears.
Connection between Unified CVP and IVP cannot be established when setting up TLS
Symptom:
Note: This tip applies to full video only.
When the Transport Layer Security (TLS) is being set up, the connection cannot be established
between Unified CVP and IVP
Message:
SSL session handshake failed (is the server SSL enabled?).
Log messages constantly show connection being made and then broken.
Cause 1:
The TLS setting is enabled on the Call Server but not in iContact. Enable TLS setting on Call
Server and IVP are not in sync.
Action 1:
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
32
Chapter 2: Symptoms and Solutions
Call Server
Verify that both Call Server and iContact are configured for TLS. Call Server configuration is
enabled via OAMP. iContact configuration is via iContactConfiguration.xml. Both Call Server
and IVP must be restarted if their settings for enabling/disabling TLS are changed.
Cause 2:
The required procedures to import/export keys between Call Server and IVP has not completed
or is not accurate.
Action 2:
Verify that the proper keys are imported and exported. Refer to the TLS configuration
documentation for procedural Call Server/IVP information.
Only One Call Server Is Receiving Calls
Symptom:
Note: This tip applies to full video only.
When multiple Call Servers are connected to a single iCONTACT, only one of the Call Servers
is receiving calls.
Message:
None
Cause:
iCONTACT configuration may not be set to distribute calls to Call Servers via round-robin
mechanism.
Action:
Open the iCONTACT configuration file and verify that the notification type is set to round
robin i.e <notification-type>Round-robin</notification-type>.
Calls Are Disconnecting
Symptom:
There are a number of abnormally disconnected calls.
Message:
Calls are disconnecting.
Cause:
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
33
Chapter 2: Symptoms and Solutions
Call Server
The Radvision IVP Linux interface, as well as Radvision MVP/MCU, does not autonegotiate
interface link speed/duplex mode properly.
Action:
On the Radvision IVP Linux server, use the ethtool utility to manually set the current interface
link speed (MB) and duplex mode.
1. At the command prompt type, ethtool -s eth0 speed 100 duplex full
2. To permanently change the interface link speed/duplex mode settings (that is, to survive
a reboot), in the /etc/sysconfig/network-scripts directory add the following line to the
ifcfg-eth0 file:
ETHTOOL_OPTS="speed 100 duplex full autoneg off "
Non-English Video Text Overlay is Garbled in Full Video
Symptom:
Message:
Cause 1:
The "user.micropp.charset" ECC variable is not set correctly for the target language.
Action 1:
Verify that the correct value is set in the "user.microapp.charset" ECC variable for the target
language in the ICM script. The supported values values are specified in the Configuration and
Administration Guide for Cisco Unified Customer Voice Portal.
Cause 2:
The Radvision MCU is not configured for the target language.
Action 2:
Upgrade the Radvision MCU to the target language so that the proper fonts are loaded. If the
MCU is already configured for the target language, restart the MCU and EMP
Movie Media of Video Micro-Application Cannot be Interrupted
Symptom:
Note: This tip applies to full video only.
Movie media of video micro-app cannot be interrupted by a DTMF barge-in or when the agent
becomes available.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
34
Chapter 2: Symptoms and Solutions
Call Server
Message:
None.
Cause:
This can happen if the micro-application movie media only has an audio-stream and does not
have any video stream.
Action:
In the Network VRU script micro-app configuration, change the Show/Display option in the
Configuration to A for audio-only.
Media Poller timing issues
Symptom:
Metadata files are modified from within the administrative web UI or from within the file system
while the media poller is polling in the background.
Message:
none
Cause:
The media poller feature is used to automatically add new video resources from the file system,
but can result in some rare timing issues.
The media poller begins its polling and cycles through the c:\foo directory. At this time, if an
administrator adds and saves a new file c:\foo\bar.xml using the administrative web UI, the
administrator (and possibly agents) see bar.xml displaying in their web UI. At this point the
media poller cycle ends as c:\foo\bar.xml was not present when that directory was polled. This
is because c:\foo\bar.xml is considered to be deleted by the media poller and is removed from
memory. Subsequent refreshes of the administrative or agent web UIs show c:\foo\bar.xml as
missing. The following media poller cycle picks up c:\foo\bar.xml more permanently.
Action:
This is a subtle, timing issue. In the end, the media poller performs its function and gets back
to synchronizing with the file system after another polling cycle. The VMS media poller process
will never delete or modify existing files on the system. It will only poll for their existence and
current status.
Invalid Thumbnail Images on VMS Server
Symptom:
Invalid images in the VMS movie library causing unintended consequences on the VMS system.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
35
Chapter 2: Symptoms and Solutions
Call Server
Message:
none
Cause:
none
Action:
Be sure to adhere to the following guidelines for thumbnail images on the VMS system. Valid
thumbnail image pixel size is 120 x 80 (width x height).Thumbnail images must have the same
name as the XML file, with the exception of the file extension. Supported file extensions are:
.jpg, .gif, and .png.
Note: Large images use more network bandwidth and will impact Video Media Server
performance when they are displayed to agents. Images that are larger or smaller than120 x 80
will be resized for display only. The content on the image itself does not change on the file
system.
CALL Server Cannot Connect to IVP Server
Symptom:
Note: This tip applies to full video only.
Call Server cannot get a connection to the IVP server.
Message:
The following is seen in the iCONTACT log: 2007-11-16 18:41:09,320 (iContactStartupHandler)
- Can not start iContact application server. The following is seen in the Call Server log
CVP_7_0_Infrastructure-3-SOCKET_SETUP: Failed getting socket for host: 10.86.129.104
port: 9981 Exception: java.net.ConnectException: Connection refused: connect
Cause:
The most likely cause is that the iCONTACT XML configuration file has an XML syntax error.
Action:
Fix any XML syntax errors in the iCONTACT configuration file and restart the IVP server
using the IVP Manager.
VMS Administrator Replaces Video Content While Agent is Previewing
Symptom:
Note: An administrator should be aware that deleting/replacing/moving video files while calls
are being served can result in unpredictable behavior. VMS is limited in that it cannot control
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
36
Chapter 2: Symptoms and Solutions
Call Server
how the Darwin Streaming Server or the IVP server will react to file changes. However, the
results are very minor. At worst, the currently playing video stops, which is not fatal to the
system.
VMS Administrator replaces a .3gp video file with new content, but retains the same filename
while at the same time a VMS Agent is Previewing the initial video. The video displaying in
the VMS Agent QuickTime window freezes.
Message:
none
Cause:
none
Action:
VMS Agent must close the QuickTime video window, reselect the video, and then Preview.
The new video content will now be available.
VMS Administrator Replaces Video Content While Agent is Pushing Initial Video to Caller
Symptom:
Note: An administrator should be aware that deleting/replacing/moving video files while calls
are being served can result in unpredictable behavior. VMS is limited in that it cannot control
how the Darwin Streaming Server or the IVP server will react to file changes. However, the
results are very minor. At worst, the currently playing video stops, which is not fatal to the
system.
VMS Administrator replaces a .3gp video file w\new content, but retains the same filename
while a VMS Agent is Pushing the initial video to a Caller. The video displaying in the Callers
window stops.
Message:
none
Cause:
none
Action:
VMS Agent must reselect the video and then Push it to the Caller. The new video content will
display to the Caller.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
37
Chapter 2: Symptoms and Solutions
VXML Server
Unable to Process Playback Request
Symptom:
An agent may see this error in the Cisco Unified CVP Video Desktop if the agent attempts a
playback request after a call has ended. However, if the agent sees this during an active call, it
means that the Video Desktop is still trying to send requests for a previous call.
Message:
Unable to process playback request. The call has been lost.
Cause:
Action:
To be able to send playback requests for the current call:
1. Close the Video Desktop browser window.
2. Click the Video Desktop button in CTIOS to open the Video Desktop with the new call
information.
3. Playback requests should now work correctly.
VXML Server
Ended Session Request Error
Symptom:
This error message appears when an HTTP request initiated by the root document is made by
the browser for a session that the VXML Server has already ended. The way the VXML Server
knows what session the request is asking for is by looking at the cookie included in the request
for the session’s ID. A request from the root document indicates either a VXML error occurred
or the caller hung up. In a typical call, all requests made for a call session come in series and
so there is never a request that comes after the VXML Server believes the call has ended. This
error message would appear if a request telling the VXML Server to end the session is followed
by a new request initiated by the root document referencing the same cookie.
Message:
SERVER ERROR: There has been an error with the http session: a request was made due to an
event caught in the root document that refers to a session that has already ended.
Cause 1:
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
38
Chapter 2: Symptoms and Solutions
VXML Server
This is a very rare situation as normal operation would not cause any further requests to be made
after the request that ended the session (it is expected that the session-ending request is the last
one). Note that for this error to appear, the new request must come from the root document, in
many cases indicating an error encountered by the H.323 Service. There could be several rare
situations where this could occur. The first begins with the end of call event activated by the
final request to VXML Server taking a long time to run. This would cause the browser to make
a new request timing out the session. This next request would have to be made before the initial
request returned, but after the end of call event completed, causing the VXML Server to end
the session before the new request was received. In order for this situation to occur, the timing
would have to be very precise.
Action 1:
For the first situation, the developer must ensure that their end of call events do not have the
possibility of taking longer to execute than the H.323 Service’s fetch timeout VoiceXML
property. Another possible solution if the end of call event is a Java class would be to execute
the class in a separately spawned thread. The problem with this would be that the spawned
thread should not attempt to access content in the session due to the possibility that it could be
invalidated while the thread was executing.
Cause 2:
Another situation that could cause this would be some network overhead or slowdown causing
the final response for a call to fail to arrive at the browser, precipitating a time out to occur.
Since VXML Server functioned without problem and ended the session, this new request would
cause the error to occur. Note that some Call Servers and operating systems under high HTTP
load can queue HTTP requests and potentially even lose requests if the queue is too large. The
administrator should investigate this queue if their system is expected to support high load (for
example, on Apache Tomcat, this is known as acceptcount).
Action 2:
For the second situation, it may be an indication of excessive network, operating system, or
Call Server load. For the network, perform diagnostics to ensure proper operation. For the
operating system, check the CPU and memory utilization to ensure they are not maxed out. And
for the Call Server, check the configuration to ensure the HTTP request queue is not too small
(for example, on Apache Tomcat, the acceptcount setting can be configured in the conf/server.xml
configuration file). Finally, the administrator may decide to direct less load on the system.
The administrator may also choose to ignore these issues if they occur rarely since there are no
bad consequences for the call session that encounters this error.
Severity: Due to the fact that this error message appears only in very rare situations, as well as
the fact that even if it does occur, the caller does not encounter any issues with their call, this
error message is benign.
Ending Session Request Error
Symptom:
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
39
Chapter 2: Symptoms and Solutions
VXML Server
This error message appears when an HTTP request initiated by the root document is made by
the browser for a session that VXML Server has started the process of ending but that process
is not complete. This is similar to the previous error, but where in that situation the request was
made after the session had completed, this situation involves the request occurring while the
VXML Server was ending the session but before it completed.
Message:
SERVER ERROR: An HTTP request was made due to an event caught in the root document
to a session that has been slated for invalidation.
Cause:
One very rare situation that could cause this error to occur under standard call volume involves
a perfect timing of events. This error would not be produced if the extra request arrives while
an end of call event was executing. However if it did take a long time to execute, prompting
the browser to time out the request by having the root document make a new request, and that
request came at exactly the time the end of call event completed but before VXML Server was
able to finish the session ending process, this error would occur. Since this process normally
occurs rapidly, the chances that this would occur during this period is very small.
This error would also be encountered if the network, operating system, or Call Server on which
VXML Server is installed is sufficiently overloaded such that either HTTP requests are being
queued up too much, HTTP requests are being dropped or the Call Server is so overloaded that
the normally trivial process of ending a session takes a non-trivial amount of time.
Severity: The severity for this error is very low because there is no effect on the caller since
this occurs after the call is complete. Additionally, the situations in which this can occur during
normal operation are extremely rare.
Action:
For the first situation, the developer must ensure that their end of call events do not have the
possibility of taking longer to execute than the H.323 Service’s fetch timeout VoiceXML
property. Another possible solution if the end of call event is a Java class would be to execute
the class in a separately spawned thread. As long as the end of call class completed its execution
before the session invalidation delay specified in the conf/global_conf.xml of the installation
directory, there would be no risk of the class referring to data within the session after it was
invalidated.
For the second situation, it may be an indication of excessive network, operating system, or
Call Server load. For the network, perform diagnostics to ensure proper operation. For the
operating system, check the CPU and memory utilization to ensure they are not maxed out. And
for the Call Server, check the configuration to ensure the HTTP request queue is not too small
(for example, on Apache Tomcat, the acceptcount setting can be configured in the conf/server.xml
configuration file). Finally, the administrator may decide to direct less load on the system.
The administrator may also choose to ignore these issues if they occur rarely since there are no
bad consequences for the call session that encounters this error.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
40
Chapter 2: Symptoms and Solutions
VXML Server
Session Does Not Contain Call Information
Symptom:
This is a rare error message that is displayed after end session request error. In this situation,
VXML Server has completed its process of ending a session, however the Call Server has not
completed its process for ending the session when a new HTTP request is received referencing
that session. This would yield a legitimate session from the Call Server’s standpoint but an
invalid session by VXML Server’s standpoint.
Message:
SERVER ERROR: The session is not associated with a Server Configuration.
Cause:
The situations where this would be encountered are identical to those listed in the previous error
condition except the timing of the second request would occur after the standard end of call
process taken by VXML Server completed but before the Call Server actually removed the
session. The timing for this situation is actually tighter than the previous situation and therefore
would occur even more rarely. While there is a chance that this could occur under standard call
volume, it would more likely occur only under extreme load to the network, operating system,
or network.
Severity: The severity for this error is very low because there is no effect on the caller since
this occurs after the call is complete. Additionally, the situations in which this can occur during
normal operation are extremely rare.
Action:
For the first situation, the developer must ensure that their end of call events do not have the
possibility of taking longer to execute than the H.323 Service’s fetch timeout VoiceXML
property. Another possible solution, if the end of call event is a Java class would be to execute
the class in a separately spawned thread. As long as the end of call class completed its execution
before the session invalidation delay specified in the conf/global_conf.xml of the installation
directory, there would be no risk of the class referring to data within the session after it was
invalidated.
For the second situation, it may be an indication of excessive network, operating system, or
Call Server load. For the network, perform diagnostics to ensure proper operation. For the
operating system, check the CPU and memory utilization to ensure they are not maxed out. And
for the Call Server, check the configuration to ensure the HTTP request queue is not too small
(for example, on Apache Tomcat, the acceptcount setting can be configured in the conf/server.xml
configuration file). Finally, the administrator may decide to direct less load on the system.
The administrator may also choose to ignore these issues if they occur rarely since there is no
bad consequences for the call session that encounters this error.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
41
Chapter 2: Symptoms and Solutions
VXML Server
Rapid Multiple Sequential Applications Warning
Symptom:
This warning message appears when an HTTP request that is not initiated by the root document
is made by the browser for a session that VXML Server has started the process of ending, and
a built-in error-correcting mechanism within the VXML Server has attempted to resolve the
issue by ignoring the cookie hence issuing a warning rather than an error. While the scenario
is similar to previous errors, this is a warning, not an error since the call itself was handled
without the caller encountering any error messages. The warning message is logged to inform
the administrator of a situation that could indicate high call volume, which could cause other
more severe issues.
Message:
SERVER ERROR: Warning: An HTTP request was made to a session that has been slated for
invalidation. It is treated as if not associated with a session.
Cause:
The likeliest situation that would cause this warning message to appear involves a setup where
a VXML Server application is configured to return to Unified ICME, that after some processing
promptly visits a second application within the same call. Due to the fact that these two
applications are visited in the same call, the cookie created in the first application and stored
by the browser is referenced in the first HTTP submit to the second application. This is not
normally an issue because the first application’s session ends before the next application’s first
submit is made, especially if the ICM script performed time-consuming tasks before visiting
the second VXML Server application. However under high volume, the process for ending a
session may not take a trivial amount of time. Additionally, if the ICM script completes in a
very short period of time, basically a shorter amount of time than it takes the Call Server to end
a session, the second application’s first request would arrive before the first application’s session
ended and with the cookie referenced in the header, would make the VXML Server believe that
the new request was made for the first application. VXML Server has a mechanism to recover
from this exact situation. It waits for a certain period of time for the first application’s session
to end and when it does, continues with the call to the second application normally. It logs this
warning as a note to the administrator that the volume may be reaching its upper limits.
Severity: Very low since no call was adversely affected (any problem was averted by the
error-correcting mechanism in the VXML Server).
Action:
If this warning appears very rarely, the administrator can safely ignore these warnings. If they
appear more frequently, this would be an indication of high volume and the administrator should
attempt to reduce the call volume to the machine.
Should it be found that the ICM script executed in between visits to VXML Server applications
runs too quickly, a workaround to use would be to put a slight pause in this script to allow the
first application’s session to end. Performing load tests on the system yields the optimal pause
length.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
42
Chapter 2: Symptoms and Solutions
VXML Server
Session Loss During a Call Error
Symptom:
The bad_url error message is generally displayed when VXML Server receives an HTTP request
it does not expect or understand. This occurs when a request is made to an application that does
not exist (or the application name is misspelled or has case-sensitive differences). The error
message includes the URL that caused the error, including any HTTP GET arguments. The
message, however, does not report any HTTP POST arguments due to their potential size, so
when the URL “/CVP/Server” is displayed, this indicates that there were either only HTTP
POST arguments passed to the URL or no arguments passed at all. The initial URL for an
application as well as the URL requesting the root document includes GET arguments (that is
“/CVP/Server?application=MyApp”), however all subsequent requests made by the VXML
Server use HTTP POST arguments to the /CVP/Server URL. Therefore, this error message is
most likely due to an HTTP submit made for a standard request within a call that has somehow
lost its session or the cookie that references the session.
This error would likely occur within a call and so the caller would hear an error message.
Message:
SERVER ERROR: bad_url: /CVP/Server
Cause:
There are several situations where this can be seen:
1. An H.323 Service or person accesses VXML Server via an incorrectly configured URL
“/CVP/Server” without passing any arguments. Since the VXML Server does not know
what to do, it returns a VoiceXML page with an error message and logs the error.
2. Some network device involving either the H.323 Service or some entity in the network
such as a load balancer, loses or removes the cookie associated with the call session. In
an environment in which a load balancer distributes call load across multiple VXML Server
instances, it is necessary for the load balancer to be “sticky”. This means that once a call
is sent to a server, all subsequent requests for the same session must be sent to the same
server. If the load balancer fails to do so, it sends a request with a cookie referring to an
unrecognizable session to a server prompting it to return an error VoiceXML page and
record this exact error. A similar situation occurs if some other network device acts to
remove cookies from HTTP requests or modifies them for its own purposes; these would
become unrecognizable by the VXML Server.
3. An issue with developer-specified code. If the developer uses an element that manually
invalidates the call session, but that element contains VoiceXML that requests a new page,
this error would occur because once the session is invalidated, any further requests would
refer to an unavailable session. In another situation, if the developer creates a hot event
that overrides an existing event caught in the VXML Server root document, this error
would be logged if the event occurs after the call session ended either as a result of a
hangup or other event.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
43
Chapter 2: Symptoms and Solutions
VXML Server
4. Some process that handles HTTP requests causes a request to be delayed so that it arrives
out of order with other requests to the same session. This would only cause the error to
occur if the request that was delayed was followed by a request by the H.323 Service to
end the session. When the delayed request finally arrives, the session it referred to would
have ended by then and the error would be logged. Note that if this delayed request was
initiated from the root document, one of the previous errors listed in this appendix would
occur. This error message would occur only if the delayed request was for a standard page
within the application. This unusual situation would most likely occur when the system
is experiencing high load. Additionally, if the H.323 Service and VXML Server were
connected via the Internet as opposed to via a LAN, there could be a greater chance that
an HTTP request could be delayed.
Severity:
Situation 1 is most likely due to an error by the administrator or developer and should never
occur in a production system. Situation 2 should also be discovered during the testing and
integration stage and should not occur in production, however there still exists a possibility with
a bug in a load balancer or a glitch in the network setup that would cause this in a live system.
Situation 3 should also be managed with proper testing that would focus on when the VoiceXML
event is thrown. The last situation is the most severe as it could occur in a production system
that is overloaded.
Note: Since this error would most likely fall within a call, the caller would hear an error message
and most likely their call would be terminated. As a result, this error does have consequences
with the caller and should be considered more severe.
Action:
For the first two situations, ensuring the system is properly tested before moving to production
is essential. For the third situation, there should be good justification for defining a hot event
that overrides default functionality. Eliminating this hot event may resolve the issue. For the
final situation, it may be an indication of excessive network, operating system, or Call Server
load. For the network, perform diagnostics to ensure proper operation. For the operating system,
check the CPU and memory utilization to ensure they are not maxed out. And for the Call Server,
check the configuration to ensure the HTTP request queue is not too small (for example, on
Apache Tomcat, the acceptcount setting can be configured in the conf/server.xml configuration
file). Finally, the administrator may decide to direct less load on the system.
Additionally, a preferred setup would involve keeping the H.323 Service on a local network
with the VXML Server machine. That way, all communication would be as fast as possible,
without any pitfalls with using the Internet as the communications medium.
Session Ending within an Element Error
Symptom:
This error is displayed when during the process of producing a response to a browser request,
the session is invalidated. This is slightly different than the previous situations. In those situations,
the session was already invalidated when the browser makes the request. Here, the session is
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
44
Chapter 2: Symptoms and Solutions
VXML Server
invalidated while VXML Server is in the process of handling a request. This occurs within a
call, precipitating an error message to be played to the caller.
Message:
SERVER ERROR: An error occurred because the HttpSession has been invalidated. This could
have been caused by an action element whose execution time exceeded the VoiceXML gateway's
time limit for VoiceXML page requests.
Cause:
There is one specific situation where this can occur under standard load. An element that takes
a long time to complete, more time than the H.323 Service’s fetch timeout property (typically
around 30 seconds) would hold up the response while it worked. The browser would time out
the request, causing a new request to be made to end the session with an error. When the element
finally completes, any subsequent work done by the VXML Server would encounter a null
session and log this error.
Under correct conditions, elements should execute very quickly. Likely causes for delays would
be when the element is accessing external resources such as databases or, less likely, the element
is performing a very complex calculation. While the error message references an action element
(which is usually the preferred element to make calculations or access databases), this can occur
within any kind of element, both standard and configurable.
Another situation where this could occur would be if the system is under very high call volume.
In this case, the operations of the system may cause normally rapid operations to slow down to
non-trivial lengths of time. This, coupled with an element that performs a slow operation, could
cause this error to occur.
Severity: The severity of this error is high since the caller would encounter an error message
during the call and could be an indication that the system is overloaded.
Action:
When designing an element that has the potential for taking a long time, the developer may
wish to implement mechanisms for handling a slowdown such as implementing internal timeouts,
or using backup systems if the primary systems fail. If the application designer knows that an
element can take a long time to run, they can consider increasing the fetch timeout VoiceXML
property just for that element, write the element to run in a separate thread so the call can continue
without delay, or provide audio to play while the caller is waiting.
The administrator may also wish to analyze the system to see if there is excessive load on it and
take measures to reduce the load if so.
Empty Error Message
Symptom:
This error message is displayed with no message at all other than saying there is a server error.
Message:
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
45
Chapter 2: Symptoms and Solutions
VXML Server
SERVER ERROR:
Cause:
This error message appears when a Java exception that itself does not define an error message
occurs. Most of the time, this is a Java NullPointerException, which defines no error message.
Note that all VXML Server errors contain descriptive error messages and only very few Java
exceptions have no message.
Note that if this exception occurred within a custom Java class written by a developer, the Java
stack trace of the exception is displayed in the error log. This stack trace provides the developer
information on where this error occurred. If there is no stack trace, this could be an indication
of a bug in the VXML Server, though it is possible that an incorrectly configured element or
Java class could cause some object to be null that should not be, causing the error to occur within
VXML Server code.
Severity: This message should be considered severe because a NullPointerException only
occurs when Java objects that should have a value do not.
Action:
If this error occurs in a custom Java class, the developer should debug it to find out the issue
by using the stack trace. If the error occurs without a stack trace, under standard call volume,
the developer may first wish to debug the problem to see if there is any custom code that could
be setting up data incorrectly. If the options are exhausted, this would be a situation where Cisco
customer support should be contacted.
VXML Server Displays Information of an Old Version
Symptom:
When starting the Call Server, the VXML Server Call Services displays information of an old
version.
Message:
Information displayed about an older version.
Cause:
A prior version than 4.0 of the VXML Server software is still installed.
Action:
If a prior version of VXML Server software was installed on the same Call Server, it is likely
that the old expanded Web application still resides in the Call Server. Either delete the old
expanded directory or use the Call Server console to decommission the old Web application
and load the new Web application archive (WAR) file.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
46
Chapter 2: Symptoms and Solutions
VXML Server
VXML Server Console Displays No Information
Symptom:
When starting the Call Server, the console displays no information on VXML Server Call
Services at all.
Message:
No message displayed
Cause:
This is most likely caused by not having the VXML Server Call Services Web application
archive file, audium.war, in the correct directory.
Action:
This could have been due to not installing VXML Server Call Services or choosing the wrong
directory in the VXML Server installer for the Call Server’s Web application archive directory.
VXML Server Sample Application Displays Page Not Found
Symptom:
When running the Hello World sample application to test if VXML Server Call Services is
running, the page could not be found.
Message:
Page not found.
Cause:
This URL would be valid only if the Web browser were running on the same machine as Call
Services.
Action:
The URL to be entered to obtain the Hello World application is:
http://localhost:PORT/Audium/Server?audium_application= HelloWorld
This URL would be valid only if the Web browser were running on the same machine as Call
Services. If it is a different machine, the appropriate IP address should be used instead of
“localhost”. Additionally, the port on which the Call Server is listening must be used. Use the
netstat command to determine the port number.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
47
Chapter 2: Symptoms and Solutions
VXML Server
VXML Server Sample Application Displays Technical Difficulties
Symptom:
When running the Hello World sample application to test if the VXML Server Call Services is
running, a VoiceXML page appears indicating that there are technical difficulties.
Message:
VoiceXML pages indicate that there are technical difficulties.
Cause:
Several possibilities exist for this error as outlined below.
Action:
There are several possibilities that exist for this error:
• The application name was not typed in exactly as it appears in the URL shown in the previous
option. All application names do not have spaces
• There is no HelloWorld application deployed on Call Services. Check the
INSTALLATION_PATH\CallServices\applications folder. If it does not contain a folder
named HelloWorld, this application has to be newly deployed from Call Studio.
• When the Call Server was launched, Call Services encountered an error. See the
troubleshooting options in this section for resolutions.
Excessive Unavailable ASR Errors in VXML Server Log
Symptom:
Excessive unavailable ASR errors in VXML Server application error log.
Message:
SERVER ERROR:A VXML exception occurred:error.com.Cisco.media.resource.unavailable.asr
Cause:
The default MRCP timeout value of 3 seconds needs to be increased.
Action:
The default MRCP timeout values need to be increased to 5 seconds on the gateway as follows:
mrcp client timeout connect 5mrcp client timeout message 5
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
48
Chapter 2: Symptoms and Solutions
VXML Server
VXML Server Loggers Not Functioning
Symptom:
When deploying a new application on a VXML Server, you receive an error message.
Message:
There was a problem instantiating the logger class
com.Cisco.CVP.vxml.datafeed.DatafeedLogger.
Cause:
The VXML Server and the Call Server are not communicating.
Action:
The application cannot be deployed successfully until the VXML Server and Call Server are
communicating. Check the network connectivity between the two servers. Traceroute and Ping
are some of the initial network testing utilities that are available to verify the network integrity.
Be sure to test the network connectivity in both directions. Also verify that the network cards
are set to the correct settings. These settings are provided in the General Systems Issues section
of this document.
VXML Server Is Unreachable
Symptom:
The VXML Server shows as Unreachable.
Message:
UNREACHABLE
Cause:
An UNREACHABLE status is reported when the Operations Console Resource Manager does
not get a valid number for active calls after running status script.
Action:
The Operations Console bypasses the Operations Console Resource Manager on the Call Server
to which the VoiceXML service is connected and directly talks to the Operations Console
Resource Manager that is co-located with the VXML Server. The reason for this is that the
status obtained through the central controller on the Call Server is incorrect if no calls have
been placed through the VXML server. It reports and displays the status as STOPPED. The
status of VXML server is obtained in a different manner than that for Reporting Server.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
49
Chapter 2: Symptoms and Solutions
VXML Server
VXML Server Does Not Start After Patch
Symptom:
Patch installer and Explorer errors display during the application of the CVP 7.0.2 patch.
Message:
"Unable to delete directory: C:\Cisco\CVP\VXMLServer\Tomcat\webapps\CVP". Please try
to delete this directory manually before restarting the system.
Cause:
The directory was not deleted after upgrade.
Action:
• Delete the %CVP_HOME\VXMLServer\Tomcat\Webapps\CVP directory.
• Restart the VXML Server.
VXML Server Fails After CVP 7.0 Upgrade
Symptom:
The VXML Server application fails following an upgrade to CVP 7.0.
Message:
None.
Cause:
Customer audio files that reside in the .war expansion directory
(CVP_HOME/VXMLServer/Tomcat/webapps) are not preserved upon upgrade to CVP 7.0.
Action:
Redeploy custom audio files to the VXML Server, but not to the .war expansion directory
because the contents of this directory are not preserved at upgrade time.
VoiceXML Scripts Fail
Symptom:
VoiceXML scripts fail with a CALL_ERROR error message.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
50
Chapter 2: Symptoms and Solutions
VXML Server
Message:
CALL_ERROR; http://10.8.7.70:8080/CVP/Server?application=HelloWorld at line 5: Undeclared
ECMAScript variable audium_vxmlLog
Cause:
The Cisco Gateway is configured with the 'no http client cookie' statement.
Action:
Enable the HTTP client to send and receive cookies on the Cisco Gateway when using VoiceXML
applications by entering the following command:
http client cookie
This command is enabled by default. You do not need to enter it unless you have previously
disabled cookie support by using the 'no http client cookie' command. Use the 'show
running-config' command to verify the HTTP configuration information. If the defaults are
used, HTTP configuration commands are not shown. If changes have been made to the HTTP
defaults, the configuration is shown. For example, the following output shows the HTTP client
cookie parameter configured to a non-default value:
no http client cookie
See Also
For more information on the 'HTTP Client Cookie' feature, see the Cisco IOS Tcl IVR and
VoiceXML Application Guide.
Call Start Element Deleted From VXML Server
Symptom:
Call Start element is deleted from the VXML Server application.
Message:
None.
Cause:
Usually user error.
Action:
Add the new Page Entry element to the call flow, then right-click on it and choose Start Of
Call -> Yes. This element is automatically used as the first element - start of call. You can then
manually rename it to Call Start.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
51
Chapter 2: Symptoms and Solutions
VXML Server
VoiceXML Application Fails to Access Another Resource
Symptom:
VoiceXML application fails to access another resource, for example, to play a media .wav fail,
and the IOS log contains SERVER ERROR: A VoiceXML exception occurred:
error.badfetch.http.-n, where n is a negative error code.
Message:
SERVER ERROR: A VoiceXML exception occurred: error.badfetch.http.-n
Cause:
error.badfetch.http.<errorcode> has a positive error code when a connection to the HTTP server
is successful, for example; 200 (OK), 400 (Bad Request), or 404 (Not Found). When a gateway
encounters an internal error, including server connection failure, it uses the following negative
error codes to differentiate the error from a server response code:
-1 Client error
-2 Timeout error
-3 Message decoding error
-4 Message encoding error
-5 Message transmission error
-6 Client write queue full
-7 Connection error
-8 Client unsupported
-9 Client socket receiving error
Action:
Make sure the resource you are attempting to access is running. For example, if the VoiceXML
application fails to access a .wav file from a media server, make sure that the media server's
World Wide Web Server service is running.
Also make sure that the path to the .wav file is correct.
Error 500: Error Initializing Velocity
Symptom:
In trying to access the link listed below, an error message is received.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
52
Chapter 2: Symptoms and Solutions
VXML Server
http://<server name>:7000/CVP/Licensing
Message:
Error 500: Error initializing Velocity: java.lang.Exception: Unable to configure
AvalonLogSystem: java.security.AccessControlException: Access denied (java.io.FilePermission
/opt/WebSphere/AppServer/profiles/AppSrv01 read)
Cause:
This is due to the Use Java 2 security to restrict application access to local resources being
enabled. When enabled, it prevents the VXML Server to run.
Action:
Modify the setting via the WebSphere Application Server (WAS) Integrated Solutions Console.
Accessing the Integrated Solutions Console
1. From a web browser enter the following link:http://<server name>:9060/ibm/console
(http://<server name>:9060/ibm/console)
2. Click the Security link in the left frame after you log in.
3. Click Secure administration, applications, and infrastructure.
4. In the Java 2 security section uncheck Use Java 2 security to restrict application access
to local resources.
5. Click Apply.
Keytool Error: Failed to Establish Chain from Reply
Symptom:
Receiving the error message.
Message:
"keytool error: Failed to establish chain from reply"
Cause:
This indicates that the keystore was not set up properly and keytool cannot find the a path from
your certificate down to a root certificate through the intermediate certificate.
Action:
Check the configuration of the keystore.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
53
Chapter 2: Symptoms and Solutions
Reporting Server
VXML Server Statistics Not Available Error Message
Symptom:
An error message from the Operations Console indicates that VXML Server statistics are not
available.
Message:
VXML Server statistics are not available.
Cause:
VXML Server requires at least one Studio application script that enables Datafeed Logger to
report statistics. This error occurs when Datafeed Logger has been removed or is missing in
either of the following conditions:
• The HelloWorld script that is installed and deployed by the installer does not contain the
Datafeed Logger.
• The Studio script(s) developed prior to Release 4.x have been upgraded and does not contain
the Datafeed Logger.
Action:
Add the Datafeed logger to the script(s).
Reporting Server
Reporting Server Association Error
Symptom:
A warning popup is produced with an error message and the associated Call Servers changes
are reverted.
Message:
Associate Call Servers can only be changed when deploying to device.
Cause:
When a user creates a new Reporting Server they are required to associate a Call Server. If
neither device is created yet the user must perform a Save, instead of a Save & Deploy. If the
user decides to change, or add to the list of associated Call Servers and perform a Save, a warning
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
54
Chapter 2: Symptoms and Solutions
Reporting Server
popup is produced with the message "Associate Call Servers can only be changed when deploying
to device." and the associated Call Servers changes are reverted.
Action:
In order to proceed past this warning the user has two options:
1. Install the Unified CVP software and perform a Save & Deploy.
2. Delete the Reporting Server device, read it with the changes to the associated Call Server
list and Save.
No Informix Services Found on Reporting Server
Symptom:
After installing the Reporting Server, the Informix Services with the Unified CVP database
name is not found.
Message:
None.
Cause:
The .bat files were not run.
Action:
none
Starting Informix Database Causes Client Locale Not Set Error
Symptom:
When starting Informix from the command prompt generates an error message.
Message:
CLIENT_LOCALE is not set. The database cannot be accessed.
Cause:
The database was tried to be started from the command line instead of the Services control
panel.
Action:
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
55
Chapter 2: Symptoms and Solutions
Reporting Server
Verify that the database has stopped. Go to the Services Control Panel and start the Informix
service with the appended database name.
Reporting Server Not Receiving Messages From Message Bus
Symptom:
Reporting server does not receive the messages from the message bus. There is no indication
of an error in the error log.
Message:
None.
Cause 1:
Check if there are other Reporting Servers that are listening on the same port.
Action 1:
Open a command prompt and type: netstat -a
Stop the offending Reporting Server that is listening on the same port. Once that server is
stopped, restart the Reporting Server that was not receiving the messages.
Cause 2:
Another possibility is that the Reporting Server was added from the Operations Console after
the Call Server was started.
Action 2:
Restart the Call Server and the Reporting Server.
Reports Are Slow
Symptom:
The reports are slow to generate.
Message:
None.
Cause:
The database may be very large, or report filters may be too wide.
Action:
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
56
Chapter 2: Symptoms and Solutions
Reporting Server
Contact your database administrator. Request that the database undergo a backup followed by
an off-hours purge of the database.
Queries Timing Out Under Load
Symptom:
Queries are timing out under load.
Message:
No message displays.
Cause:
This is typical response when the database is under heavy load.
Action:
Consider querying the database when the database is not under heavy load.
You might also try to reduce contention with active processes using the following statements:
• SET LOCK MODE WAIT 30
the WAIT <seconds> option causes the session to suspend the process until the lock releases
or until the waiting period ends.
• SET ISOLATION DIRTY READ
this option instructs the session not to place locks on the data when it is attempting to read,
and potentially read data ignoring existing locks.
Reporting Server Cannot Connect to Database
Symptom:
The Reporting Server is unable to connect to the database.
Message:
None.
Cause:
cvp_db user password has expired.
Action:
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
57
Chapter 2: Symptoms and Solutions
Reporting Server
Have the Administrator reset the database password.
Reporting Server Goes into Partial Service Mode
Symptom:
The Reporting Server shows that it is in a partial service mode.
Message:
Partial Service.
Cause:
The Informix database services are not running.
Action:
Go to Start > Administrative Tools > Computer Management.
Click Services and Applications.
Click Services.
Check that the following services are started:
• Informix Dynamic Server Message Service
• Informix IDS - cvp_db_<hostname>
Start services if they are not running.
Reporting Server Cannot Initialize JDBC Connection
Symptom:
The Reporting Server is unable to initialize the Java Database Connectivity (JDBC) connection.
Message:
RPT-3-REPORTING_INTERNAL_ERROR: Can't initialize the JDBC Connection
java.sql.SQLException: com.informix.asf.IfxASFException: Attempt to connect to database
server (cvp_db_<dbserver>) failed.
Cause:
The database is disconnected and is unable to reconnect possibly due to a bad password.
Action:
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
58
Chapter 2: Symptoms and Solutions
Reporting Server
Check the database password from the Operations Console. From the Device Management menu
select CVP Reporting Server. Select the Reporting Server, then select Database
Administration -> Change User Passwords. Click Database User and change the password.
Reporting Server Is Unreachable
Symptom:
The Reporting Server is Unreachable.
Message:
UNREACHABLE
Cause:
Reporting server is UNREACHABLE in any one of the following three conditions:
• Operations Console is unable to connect to the Operations Console Resource Manager that
is co-located with the Call Server. For example, the Operations Console Resource Manager
is down.
• The Operations Console Resource Manager on the Call Server has not received a state event
from the central controller for the Reporting Service.
• The Operations Console Resource Manager is unable to connect to the Call Server and it is
not receiving state events from the central controller. For example, the Call Server is down.
The state of the Reporting Server is obtained from the Operations Console Resource Manager
that is co-located with the Call Server to which the Reporting Server is connected. If Call Server1
and Reporting Server1 are two different machines and both have Operations Console Resource
Managers running on them, Operations Console talks to the Operations Console Resource
Manager on Call Server1 to get status of the Reporting Server because that is where the central
controller reports the status of the Reporting Server.
Action:
Configure the Reporting Server if the Operations Console Resource Manager on the Reporting
Server is running, but Operations Console shows the status as unreachable.
Errors During Reporting Server Database Purge
Symptom:
If you attempt to run a database purge on the reporting server when another purge is already in
progress, or if the reporting server is down, the following messages appear in the log.
Message:
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
59
Chapter 2: Symptoms and Solutions
Audio
createClientSocket -- Couldn't get I/O streams for the socket.
Error creating the client socket.
Error sending a Start message to Reporting Server.
Error sending an End message to Reporting Server.
Cause:
The reporting server could not open a socket connection to itself, either because it was down
or a purge process was in progress.
Action:
No action necessary. This condition and the associated messages do not prevent the back-end
task from completing the purge process.
Audio
Incoming Audio-Only Calls to IVP Are Not Rejected
Symptom:
Incoming audio-only calls to IVP are not rejected.
Message:
None.
Cause:
Audio-only or basic video calls are routed to IVP. The ivp.xml configuration file on the IVP
machine is not set to reject audio-only calls.
Action:
Add <general rejectIncomingAudioOnlyCalls="enable" /> to /home/rv/ivp/conf/ivp.xml, and
then restart the IVP machine.
Note: Enabling this parameter prohibits audio-only agents from performing warm consultative
transfer back into IVP. Audio-only calls must be routed through the CVP Call Server.
Caller Hears Nothing but No Errors Logged
Symptom:
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
60
Chapter 2: Symptoms and Solutions
Audio
The caller does not hear any voice audio, and Unified ICME and Unified CVP logs do not
indicate any problems.
Message:
None.
Cause:
There may be a problem with explicitly appending a .wav extension to the Network VRU Script
Name.
Action:
In Configure ICM, remove the .wav extension from the Network VRU Script name. For example,
if the name is PM,welcome.wav, change it to PM,welcome.
Caller Does Not Hear Prompt or Expected Event Does Not Occur
Symptom:
When a call is made, the caller does not hear a prompt or an expected event (such as a transfer)
does not take place.
Message:
None.
Cause 1:
There has been a misconfiguration on the Unified ICME.
Action 1:
Do the following:
1. In Configure ICM, under Network VRU Scripts, note the Network VRU for the Network
VRU script in the ICM Run VRU Script node.
2. In Configure ICM, under Labels for the Network VRU that is running the ICM script,
click Set Target, click Network VRU and select the Network VRU noted in Step 1.
Note: When using VRU Types 2 or 5, try running Call Tracer.
Cause 2:
An error has occurred in Unified CVP.
Action 2:
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
61
Chapter 2: Symptoms and Solutions
Audio
Do the following:
1. Use Monitor mode in the ICM Script Editor to see if the call is following the expected
path (that is, it does not travel through an X branch).
2. Check the Call Server logs to see what the error or warning messages have been generated.
Caller Does not Hear Multicast Music on Hold
Symptom:
The system is configured to use multicast MOH when callers are placed on hold, but Caller
does not hear multiocast MOH when placed on hold and the call is going through a SIP trunk
. Additionally, the Unified Communications Manager Realtime Monitor indicates that mMOH
is active.
Message:
None
Cause:
N/A
Action:
Add the global command: ccm-manager music-on-hold on the Gateway.
One-Way Audio on Voice Gateway
Symptom:
One-way audio on a AS5350 or AS5400 voice gateway after an IP transfer.
Message:
None.
Cause 1:
One-way audio after a remote H.323 device sends the VoIP-Gateway a HOLD or TRANSFER
message.
Action 1:
Disable the voice-fastpath command on the Gateway, using the syntax: no voice-fastpath
disable.
Note: This command is enabled by default and the syntax is hidden.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
62
Chapter 2: Symptoms and Solutions
Audio
Cause 2:
The Gateway and Cisco Unified Communications Manager are in different subnets and no IP
route defined in the Gateway.
Action 2:
Ping from the Gateway to the Unified CM and also from the Unified CM to the Gateway. If
there is a problem with pinging, add an IP route for the different subnet in the Gateway.
For example, specify the command ip route 0.0.0.0 0.0.0.0 10.86.137.1 where
10.86.137.1 is the default routing gateway for the Gateway. This command enables routing to
all subnets.
After entering this command, try a ping session again. If that is successful then you should be
able to listen to the two-way voice.
GetSpeech Micro-Application With Datatype "Currency" Does Not Work with Negative Currencies
Symptom:
The GetSpeech Micro-Application with datatype set to "currency" does not work with negative
currencies.
Message:
None.
Cause:
The ASR-TTS server is Nuance 8.5(ASR). This version of Nuance does not support negative
currencies for the VXML Server built-in "currency" grammar.
Action:
Write your own grammar for dealing with currency or contact Nuance support.
A Prompt File From Media Server Did Not Run
Symptom:
A prompt file from Media Server did not run. Other symptoms of this issue may include:
• There is an error message in the Unified CVP logs stating a problem with invalid encoding.
• There is an error message in the Unified CVP logs stating the audio file has a bad header.
• There is an error message in the Unified CVP logs stating that the payload type in audio file
does not match the payload.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
63
Chapter 2: Symptoms and Solutions
Audio
Message:
Invalid encoding
ERROR: Audio file had a bad header.
ERROR: Payload type in audio file does not match the payload.
Cause 1:
Codec mismatch between the Voice Browser configured codec and Audio file codec.
Action 1:
Complete the following steps:
1. Convert the prompt file codec to the H.323 Service configured codec (g711Alaw64K or
g711Ulaw64K).
2. Verify that the codec is using the sCodec command from VBAdmin and check the prompt
file properties to verify that codec matches the VB configured codec.
Cause 2:
The audio file is not a valid .wav file.
Action 2:
Re-record or convert the audio file in standard .wav format.
Long Pauses between Prompts
Symptom:
Caller experiences long pauses between prompts.
Message:
None.
Cause:
There are a couple of possible causes for this issue.
1. There is a WAIT node in the ICM script.
2. There is delay in the network or one of the components in the solution.
Action:
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
64
Chapter 2: Symptoms and Solutions
Audio
Below are the possible solutions to the causes mentioned above.
1. WAIT nodes are not supported with Unified CVP. Check for WAIT nodes in the ICM
script and remove them.
2. The point of latency within the network must be isolated and resolved.
Caller Hears System Error Message
Symptom:
A caller hears a system error message.
Message:
The caller hears the default audio message that indicates a system error has occurred.
Cause:
When a caller hears a system error message, this usually indicates an ICM scripting or
configuration error.
Action:
Complete the following steps:
1. If the caller is experiencing the error before hearing any other prompts, in Unified ICME
software make sure the dialed number is associated with a call type and the call type is
associated with a script.
2. In the ICM script, be certain that all external paths out of Run External Nodes have some
default action (transfer to default LABEL, Queue to Skill Group, and so forth) .
3. In the ICM script, make sure that the last node in the script at the logical end of the call
terminates gracefully, such as with a RELEASE or LABEL. Otherwise, the system error
is played.
4. Unified CVP might have timed out waiting for a response from Unified ICME software.
This is typically caused by an Application Gateway data base dip that is taking longer than
the Unified CVP Call Server timeout settings.
Prompts Not Played Intermittently, Random Prompts, Random ICM Scripts
Symptom:
The callers are encountering problems such as prompts only being played intermittently or are
not being played at all. Other issues include random prompts, and random ICM scripts.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
65
Chapter 2: Symptoms and Solutions
Audio
Message:
None.
Cause:
Media file caching is not configured correctly on the gateway.
Action:
If media caching is not enabled on the gateway, the gateway is forced to load every prompt for
each call. This can sometimes create a situation that overloads the gateway and can cause prompt
play failures. For procedures on how to configure gateway caching, refer to the Configuration
and Administration Guide for Cisco Unified Customer Voice Portal.
No Ring Tone When Transfer to Unified CCE Agent
Symptom:
No ring tone is heard when transferred to Unified Contact Center Enterprise agent.
Message:
None
Cause:
Cisco Unified CM has an incorrectly configured service parameter.
Action:
In CCMAdmin > Service > ServiceParameters > Unified CM, check the value of H323
parameter Send H225 User Info Message. It must be set to User Info for Ring Back Tone.
Garbled Prompts or Voice
Symptom:
Garbled prompts or voice
Message:
N/A
Cause 1:
There is a PIX firewall between the gateway and Unified CVP or between gateway and Unified
CM.
Action 1:
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
66
Chapter 2: Symptoms and Solutions
Audio
PIX firewalls can sometimes cause issues where the prompts or voice communications become
garbled. In an attempt to determine if the firewall is the issue, try to temporarily circumvent the
firewall to see if that resolves the issue. If it does appear to be a firewall issue, then review PIX
Firewall documentation for additional information.
Cause 2:
Another possibility is that there is a latency issue in the network that is causing IP phone jitter
or bad quality prompts.
Action 2:
The network should be analyzed to determine if jitter conditions are present. If jitter is detected,
then the IP packets are getting delayed somewhere on the network.
Caller Hears Prompt then Dead Air, Fast Busy, or Pause, then Disconnected
Symptom:
Caller hears prompt and then dead air, or fast busy, or a pause, and is then disconnected.
Message:
None
Cause:
If this is Comprehensive or VRU-Only, and the last ICM script node executed was a PM (Play
Media), a dummy dial-peer and translation rule may be missing from the gateway config.
Action:
Play Media VXML contains a "dummy" transfer to 987654 that is meant to fail. However, it is
possible that the gateway contains a dial-peer that tries to send a call to 987654. Add the following
lines exactly as they appear below to the gateway to prevent that dial-peer match from occurring.
Note: 987654 is the actual number you type.
voice translation-rule 1
rule 1 /987654/ //
!
voice translation-profile block
translate called 1
!
dial-peer voice 987654 voip
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
67
Chapter 2: Symptoms and Solutions
Audio
translation-profile incoming block
incoming called-number 987654
Apply the translation-profile incoming block to all VoIP dial-peers
One-way Voice after Agent Transfer
Symptom:
When a phone call is established from an IP station through a Cisco IOS voice gateway or router,
and the agent transfers the call, only one of the parties receives audio (one-way communication).
Message:
None.
Cause:
No IP connectivity between the phone and the gateway.
Action:
To summarize the more common ones:
• Check Basic IP Routing: Basic IP reachability should always be checked first. As RTP
streams are connectionless (transported over UDP), traffic may travel successfully in one
direction, but get lost in the opposite direction.
• Enable IP Routing: Make sure your router has IP routing enabled, that is, does not have the
global configuration command "no IP routing".
To enable IP routing, simply type the following global configuration command in your Cisco
IOS gateway:
gateway(config)# conf t
gateway(config)# ip routing
• Cut-through Two Way Audio Early: In some cases it is necessary to establish a two-way
audio path as soon as the RTP channel is opened. In other words, before the connect message
is received. To achieve this, type the voice rtp send-recv global configuration command:
gateway(config)# conf t
gateway(config)# voice rtp send-recv
• Disable voice-fastpath: The Cisco IOS command voice-fastpath enable is a hidden global
configuration command for the AS5350 and AS5400, which is enabled by default. To disable
it, use:
gateway(config)# conf t
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
68
Chapter 2: Symptoms and Solutions
Audio
gateway(config)#no voice-fastpath enable.
See Also
For a more comprehensive list of possible solutions, go to: http://www.Cisco.com/en/US/tech/
tk652/tk698/technologies_tech_note09186a008009484b.shtml
Micro-Application May Not Always Create Correct VoiceXML
Symptom:
The execution of Play Data (PD) Micro-Application may not always create VoiceXML as per
the current VoiceXML and SSML built-in grammar types. This may lead to variations on
expected spoken results when using TTS.
Message:
The TTS server bases its rendering of speech interpretation based on how we output these values,
and it may not produce the exact output we are predicting or desire.
Cause:
Configuration not in alignment with VoiceXML standards.
Action:
Change the say-as element and the interpret-as attribute to align them to the following
standards in order to hear the desired output:
Voice Extensible Markup Language (VoiceXML) Version 2.0 (Specifically Appendix P): http:/
/www.w3.org/TR/voicexml20/#dmlABuiltins
Speech Synthesis Markup Language (SSML) 1.0:http://www.w3.org/TR/speech-synthesis/
Speech Recognition Grammar Specification (SRGS) 1.0: http://www.w3.org/TR/
speech-grammar/
Semantic Interpretation for Speech Recognition (SISR) - W3C Working Draft 1 April 2003:
http://www.w3.org/TR/2003/WD-semantic-interpretation-20030401/
Large Number of No Matches for Speech Recognition
Symptom:
There are a large number of no matches for speech recognition.
Message:
None
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
69
Chapter 2: Symptoms and Solutions
Audio
Cause:
Speech recognition confidence threshold is set too high.
Action:
When the Speech recognition server recognizes voice input, it determines a confidence score
that signifies how confident the server is that it correctly matched caller input. This confidence
score is anywhere from 0 to 1.0 where 1.0 is 100% confidence that the grammar match is correct.
By default, Unified CVP tells the Speech recognition server to throw an invalid entry error for
all confidences below 0.7 or 70% for "GetDigits" and "Menu" and 0.5 or 50% for "GetSpeech".
For some applications, this value is too high and results in false invalid entry errors.
Consult the ASR vendor to determine if the default confidence thresholds used by Unified CVP
(0.7 for GetDigits and Menu, 0.5 for GetSpeech) is incorrect for your applications. If the default
values are indeed incorrect and need to be changed, the Unified CVP templates need to be
adjusted. Contact your support contact for instructions on template modification.
Same Prompt Repeats Multiple Times
Symptom:
Same Prompt Repeats Multiple Times
Message:
None
Cause:
When using ASR/TTS, this symptom is caused when the gateway cannot connect to the ASR
or TTS server.
Action:
In Operations Console > Call Server > IVR tab there is a field called Number of tries for
ASR/TTS server. The default is zero. If the gateway cannot connect to the ASR or TTS server,
the value configured determines the number of times it tries to connect. When that happens, the
prompt repeats on each subsequent try. Also, on the gateway the customer can specify a backup
ASR/TTS server (look for “ip host asr-en-us-backup” or “tts-en-us-backup” in the gateway
‘show run’ config). If specified, the gateway tries the backup server it uses the value configured
to determine the number of times it tries to connect. The end result is a condition where the
caller hears the prompt repeat if the value is greater than zero. The solution is to resolve the
connection to the ASR/TTS server.
Cannot Deploy Newly Added Speech Server
Symptom:
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
70
Chapter 2: Symptoms and Solutions
Audio
An error messages is received after the Unified CVP Operations Console Server, and the
Operations Console Resource Manager on a Nuance Speech Server was installed, and added
the Speech Server in Operations Console and saved configuration.
Message:
"<devicetype> device with IP Address: <ip> and Hostname: <host> operation failed. Device
could not be reached. The device may have multiple NIC cards enabled. To resolve the issue
please keep only one NIC card with required IP address enabled. In addition to the NIC cards
change also restart the Resource Manager service on the device."
Cause:
In some instances, when starting the RMI Server, the RMI server remote stubs contain the IP
address of the local host (127.0.0.1) instead of the real IP address. Since the 'Resource Manager
service' on each machine uses JMX (RMI Server), in the situation where remote stubs containing
local host (127.0.0.1) the Operations Console Server cannot communicate with that specific
'Resource Manager' service. It is because the Operations Console Server uses the real IP address
to communicate with JMX (RMI) Server in the Resource Manager Service where as remote
stubs contain local host ip address, so the connection fails.
Action:
As indicated in the error message that is displayed in the user interface (UI), perform the steps
below:
1. Check the server where Resource Manager service is running to determine if it has multiple
network interface cards (NIC) enabled. If so, disable all of the NIC cards, except the one
that has the required IP address for Unified CVP communication.
2. Open the Services control panel, and locate the Unified CVP Resource Manager service.
3. Stop the service.
4. After the service has stopped, start the service.
SpeechWorks Does Not Work with Unified CVP
Symptom:
ScanSoft OSSServer.cfg, and Baseline.xml files require changes.
Message:
The default configuration for SpeechWorks Media Server (SWMS) 3.19, OpenSpeech Recognizer
(OSR) 3.09, RealSpeak 4.0.10 from ScanSoft requires changes to work with Unified CVP.
Cause:
Default configuration still in place.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
71
Chapter 2: Symptoms and Solutions
Audio
Action:
The default configuration for SpeechWorks Media Server (SWMS) 3.19, OpenSpeech Recognizer
(OSR) 3.09, RealSpeak 4.0.10 from ScanSoft requires changes to work with Unified CVP. The
following parameters need to be set:
OSSServer.cfg:
The OSSServer.cfg file is for MRCP server and webserver application configuration. On a
Scansoft Server it can be found at the following path location:
C:\Program Files\SpeechWorks\MediaServer\server\config
The Name, Type and Value entries should be configured as shown below. Values should be
changed, lines un-commented or entries added in order to have the OSSServer.cfg file reflect
these entries. Be careful to search for each one first because most exist already and just need
changes to the "Value".
server.session.ossrec.useWidecharStringGrammers VXIInteger 0
server.transport.port VXIInteger 554
server.transport.dtmfPayloadType VXIInteger 101
server.session.ossweb.machine VXIString <server_name>
server.session.mrcpdefaults.nbest_list_length VXIString 1
server.session.mrcpdefaults.no-input-timeout VXIString 10000
server.session.mrcpdefaults.completetimeout VXIString 1000
server.session.mrcpdefaults.incompletetimeout VXIString 1000
server.session.mrcpdefaults.fetch-timeout VXIString 5000
server.transport.dtmfTriggerLeading VXIInteger 0
server.transport.dtmfTrailingEdgeTimeout VXIInteger 2000
## All DiagTag values should be set to "0" to disable debug and improve
performance.
server.log.diagTag.2000 VXIInteger 0
server.log.diagTag.2001 VXIInteger 0
server.log.diagTag.3000 VXIInteger 0
server.log.diagTag.3001 VXIInteger 0
server.log.diagTag.6000 VXIInteger 0
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
72
Chapter 2: Symptoms and Solutions
Audio
server.log.diagTag.6001 VXIInteger 0
server.log.diagTag.8000 VXIInteger 0
server.log.diagTag.9000 VXIInteger 0
server.log.diagTag.10000 VXIInteger 0
server.log.diagTag.10001 VXIInteger 0
server.log.diagTag.79999 VXIInteger 0
Baseline.xml:
In the Baseline.xml file, there is a parameter called, swirec_extra_nbest_keys that specifies
which ScanSoft keys to return. The default setting of this parameter:
<!-- Add a ScanSoft grammar key to the XML result. -->
<param name="swirec_extra_nbest_keys">
<value>SWI_meaning</value>
<value>SWI_literal</value>
<value>SWI_grammarName</value>
</param>
Below are the results of the default code:
Code: <result><interpretation conf="1"><text mode="voice">reebar</text>
<instance grammar="mygrammar"><SWI_ literal>reebar</SWI_literal>
<SWI_grammarName>mygrammar</SWI_grammarName>
<SWI_meaning>{SWI_literal:reebar}</SWI_meaning>
</instance></interpretation></result>
The following are changes that need to be made to the Baseline.xml file to in order to remove
grammar keys in the XML result. Below are the required changes to the Program
Files\SpeechWorks\OpenSpeech Recognizer\config\Baseline.xml.
1. Open the Baseline.xml file.
2. Change the swirec_extra_nbest_keys parameter from:
<!-- Add a ScanSoft grammar key to the XML result. -->
param name="swirec_extra_nbest_keys">
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
73
Chapter 2: Symptoms and Solutions
Audio
<value>SWI_meaning</value>
<value>SWI_literal</value>
<value>SWI_grammarName</value>
</param>
to:
<!-- Add a ScanSoft grammar key to the XML result. -->
param name="swirec_extra_nbest_keys">
<value></value>
</param>
3. Save the Baseline.xml file.
4. Restart the server.
5. With the changes in place, the XML result will now be:
Code:
<result><interpretation conf="1"><text mode="voice">reebar</text><instance
grammar="mygrammar"></instance></interpretation></result>
Hearing English System Critical Message When Locale Not Set to en-us
Symptom:
The caller is hearing English system critical media when the locale is set to something other
than en-us.
Message:
None.
Cause:
The language specific prompts were not copied to the H.323 Service.
Action:
You need to copy the language specific prompts to the H.323 Service.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
74
Chapter 2: Symptoms and Solutions
Audio
Speech and DTMF Keystroke Recognition Take Long Time to Complete
Symptom:
Any speech and DTMF keystroke recognition seem to take a long time to complete, and initial
words or keystrokes are missed.
Message:
None.
Cause:
Using M, GD or GS Micro-Applications with input_type set to 'B' (both).
Action:
Virus scanning can severely degrade recognition speed if you have it running in your recognition
servers. Check with your recognition service provider for their input on how to configure virus
scanning for maximum protection and minimum performance impact.
Speech Recognition Does Not Recognize Wide Character String Grammars
Symptom:
Scansoft Speech Recognition Server does not recognize wide character string grammars.
Message:
None.
Cause:
ScanSoft server not configured for Wide Character String Grammars.
Action:
Set the following:
In the \ScanSoft\SpeechWorks OpenSpeech Media Server\Server\config directory, modify the
OSSserver.cfg file to add
server.session.ossrec.useWidecharStringGrammars VXIInteger 0
In the OSSserver.cfg, set:
server.transport.dtmfPayloadType VXIInteger 101
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
75
Chapter 2: Symptoms and Solutions
Audio
Large Number of Invalid Entries for Speech Recognition
Symptom:
Large number of invalid entries for speech recognition
Message:
None.
Cause:
Unified CM has an incorrectly configured service parameter.
Action:
When the Speech recognition server recognizes voice input, it determines a confidence score
that signifies how confident the server is that it correctly matched caller input. This confidence
score is anywhere from 0 to 1.0 where 1.0 is 100% confidence that the grammar match is correct.
By default, Customer Voice Portal tells the Speech recognition server to throw an invalid entry
error for all confidences below 0.7 or 70% for GetDigits and Menu and 0.5 or 50% for GetSpeech.
For some applications, this value is too high and results in false invalid entry errors.
Consult the ASR vender to determine if the default confidence thresholds used by Customer
Voice Portal (0.7 for GetDigits and Menu, 0.5 for GetSpeech) is incorrect for your applications.
If the default values are indeed incorrect and need to be changed, the Customer Voice Portal
templates needs to be adjusted. Contact Cisco TAC for instructions on template modification.
DTMF Barge-in Not Detected
Symptom:
Start of speech not flagged in Scansoft configuration file causing DTMF barge-in not to be
detected.
Message:
None.
Cause:
Change made to Scansoft OSSserver.cfg file under ScanSoft\SpeechWorks OpenSpeech Media
Server\Server\config.
Action:
Set the following: server.session.ossrec.mrcp.startOfSpeechOnDTMF VXIInteger
1
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
76
Chapter 2: Symptoms and Solutions
Transfer and Connect
Clicking Noise Heard During Scansoft TTS on IP Phones
Symptom:
A clicking noise is heard during Scansoft TTS on IP phones.
Message:
None.
Cause:
Change made to Scansoft OSSserver.cfg file under ScanSoft\SpeechWorks OpenSpeech Media
Server\Server\config.
Action:
Set the following:
server.transport.audio.playerPacketDataSize VXIInteger 160
Transfer and Connect
Agent Does Consult Transfer to CVP and Completes Consult While Connected to IVR
Symptom:
The agent does a consult transfer to CVP and then completes the consult while still connected
to the IVR, instead of waiting until finally being connected to the agent.
Message:
Cause:
To avoid this issue, do a single step transfer instead of a consult transfer to get the caller
transfered to the CVP IVR. In the event you still want to use a consult transfer into CVP for
some reason (for example, an agent desktop is customized for only one flavor of transfer), you
can create a separate SIP trunk that has MTP enabled. This SIP trunk will service the route
patterns destined for CVP. The DNIS you are using is different than the warm transfer DNIS,
so there should be no problem using separate SIP trunks.
It happens for SIP only (H323 is fine). Future enhancement to the VXML gateway will support
reinvitations on the IVR leg, in which case MTP will not be required on this particular call flow.
Action:
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
77
Chapter 2: Symptoms and Solutions
Transfer and Connect
The set up includes adding an additional SIP trunk with MTP, allowing MTP to be allocated
only on the warm transfer legs. 1) Create the secondary SIP trunk, call it "Outbound SIP
trunk-Warm Xfers" to differentiate it with the incoming SIP trunk that handles calls from CUPS
or CVP. 2) Apply MTP and point it to the CVP IP address. 3) Create the Route Pattern for the
VRU label and set the outbound sip trunk.
Agent Available and Transfer Does Not Happen
Symptom:
When an agent becomes available during prompt-play in a Queue to Skill Group script, the
transfer never happens.
Message:
None.
Cause:
The Interruptible checkbox in a Network VRU Script configuration is not checked. (This is the
default setting). The Unified ICME software does not interrupt a VRU script to perform a transfer
if this feature is not enabled.
Action:
Select the Interruptible checkbox on all Network VRU script nodes that might be run while a
caller is waiting in queue for an agent.
Problem with IP Transfer
Symptom:
Problem with IP Transfer.
Message:
None.
Cause 1:
Unified CM is not registered with the Gatekeeper because the Gatekeeper information in Unified
CM is incorrect.
Action 1:
Change the Gatekeeper host name used during configuration to an IP address.
Cause 2:
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
78
Chapter 2: Symptoms and Solutions
Transfer and Connect
Unified CM is not registered with the Gatekeeper because the Gatekeeper information in Unified
CM is incorrect.
Action 2:
Do the following:
1. In ccmadmin, select Device > Gatekeeper.
2. Add the Gatekeeper Device Information and select gateway as the Terminal Type.
3. Click Update, then Reset Gatekeeper.
4. Finally click, Reset to apply the changes.
Cause 3:
Unified CM in not registered with Gatekeeper because the Reset and Restart buttons was not
clicked after the Gatekeeper information was updated.
Action 3:
Do the following:
1. In Operations Console, select Device > Gatekeeper.
2. Add the Gatekeeper Device Information and select gateway as the Terminal Type.
3. Click Update, then Reset Gatekeeper.
4. Click Reset to apply the changes.
Cause 4:
Unified CM in not registered with Gatekeeper because a Gatekeeper registration port is not
explicitly configured in Gatekeeper.
Action 4:
Do the following:
1. Check the Gatekeeper configuration to verify that port 1719 is explicitly defined for the
local Gatekeeper zone. For example: zone local gk1 Cisco.com 10.86.137.146
1719.
2. Verify the Unified CM registration in Gatekeeper by entering the show Gatekeeper
endpoints command in the Gatekeeper CLI.
Cause 5:
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
79
Chapter 2: Symptoms and Solutions
Transfer and Connect
If the Unified CM is an egress Gateway, the Voice Browser is not defined as a Gateway in the
Unified CM configuration.
Action 5:
If the H.323 Service displays the message: ERROR: IP Transfer - Destination endpoint at IP
Address <number1> refused connection for phone number <number2> with reason code
Unreachable Destination (where: <number1> is the Unified CM IP address and <number2> is
the label where you want the call to be transferred), do the following:1.2.
1. In Operations Console on the Unified CM machine by selecting Device > Gateway.
2. Click Find and verify that there is an entry and is configured.
Cause 6:
The call is getting routed elsewhere.
Action 6:
If the H.323 Service displays the message: ERROR: IP Transfer - Destination endpoint at IP
Address <number1> refused connection for phone number <number2> with reason code
Unreachable Destination (where <number1> is not the Unified CM IP address that’s expected
and <number2> is the label where you want the call to be transferred), then there is a problem
with Gatekeeper routing. From the Gatekeeper CLI, enter show Gatekeeper
gw-type-prefix and examine the Gatekeeper routing table.
Agent Cannot Transfer Calls to Unified CVP or Another Agent
Symptom:
Agent cannot transfer calls back to the Unified CVP or to another agent.
Message:
None.
Cause:
The Unified CM SingleStepTransferViaRedirect setting is incorrect. Other symptoms: The
OPCTest list indicates that the initial call has not cleared; the Agent Desktop displays the initial
call.
Action:
In the Unified CM, modify the SingleStepTransferViaRedirect setting to 1.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
80
Chapter 2: Symptoms and Solutions
Transfer and Connect
Agent Cannot Perform a Consult Transfer and Conference to a Second Agent
Symptom:
An agent cannot perform a consult transfer and conference to a second agent if
NetworkTransferEnabled is set to 1.
Message:
None.
Cause:
The Unified ICME router cannot distinguish between an agent pressing the consult button vs.
the blind transfer button. Therefore, for Unified ICME, the routing behavior is the same whether
the agent types "1234" or presses consult or blind transfer.
Action:
Uncheck the Network Transfer Preferred box on the Unified CM PIM in PG Explorer.
• Set NetworkTransferEnabled to 1 if the agent is available, or to 0 if the agent is not available.
• If the agent is not available, they still need a Translation Route to VRU node to make a call
queue in the type 2 VRU.
When a Call Is IP Transferred, the Caller Is Disconnected
Symptom:
When a call is IP Transferred, the caller is disconnected.
Message:
None.
Cause 1:
Gateway and Gatekeeper cannot find each other on the network.
Action 1:
Make sure that the Gateway can ping the Gatekeeper and vice versa:
ping <gk ip address> from Gateway
ping <gw ip address> from Gatekeeper
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
81
Chapter 2: Symptoms and Solutions
Transfer and Connect
Make sure that Gateway and Gatekeeper can resolve each other’s names to IP address.
Cause 2:
Configuration on the Gateway might be incorrect.
Action 2:
Verify that the Gateway configuration is correct.
Turn on debug in the Gateway with the following commands:
debug h225 asn1
debug cch323 ras
term mon
If the Gateway is configured properly, it should send a RAS message to register with Gatekeeper.
Note: Turn off debug in the Gateway using the following command: no debug all
A Call Placed from an Analog Phone on an FXS Port Fails
Symptom:
A call placed from an analog phone on an FXS port does not appear to trigger the new_call.vxml
application.
Message:
None.
Cause:
There is no application session command defined on the POTS dial-peer for the telephone’s
voice port.
Note: This command is necessary to enable phones on FXS ports to run a VXML Server script.
Action:
On the POTS dial-peer, model your configuration after the following:
dial-peer voice 1000 pots
application session
destination-pattern 12016881000
port 1/0/0
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
82
Chapter 2: Symptoms and Solutions
Transfer and Connect
Calls Dropped When a Caller Tries to Invoke Supplementary Services
Symptom:
Calls are dropped when a caller tries to invoke supplementary services like Call Hold, consult
or conference.
Message:
ERROR: Supplementary service invoked by Caller. This feature is not supported.
Cause:
Caller is trying to invoke supplementary services like Call Hold, consult or conference. This is
an unsupported feature.
Action:
On Unified CM (with supported Unified CVP version), turn on MTP (Media Termination Point).
Call is Disconnected during IP Transfer and the Gatekeeper Is Configured
Symptom:
The call is disconnected during IP transfer and the Gatekeeper is configured.
Message:
None.
Cause:
The configured H.323 Gatekeeper is not running or Gatekeeper is configured incorrect.
Action:
Follow these steps:
1. Make sure the H.323 Gatekeeper is running on the IP address configured in the Unified
CVP H.323 Service. Verify that the correct IP address (and not host name) of H.323
Gatekeeper is configured by using the sGK command from Voice Browser Admin.
2. Verify that the IP network connectivity between the Unified CVP H.323 Service and H.323
Gatekeeper.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
83
Chapter 2: Symptoms and Solutions
Transfer and Connect
Agent Desktop Gets a "Call Failed Due to Network Interruption" Error
Symptom:
Agent desktop gets an error message stating, “call failed due to network interruption.” Unified
CM trace log contains the message, “Calls are dropped in H225 with a CV=41 which shows a
‘temporary failure’.”
Message:
Call failed due to network interruption.
Cause:
IP Security policies on your H.323 Service or Call Server machines can prevent connections,
thereby causing calls to fail.
Action:
View the current IP Security policy on the machine on which the Call Server is installed:
1. Select Start--> Run--> mmc-Console.
2. Click Add/Remove Snap-in.
3. Click Add.
4. ClickIP Security Policy Management.
5. Click Finish and OK.
6. In the left pane, click IP Security Policies on Local Machine.
7. Either disable the policy or edit the policy to allow the subnet from which calls are trying to
connect.
Call Back For IVR Service Treatment or Subsequent Network Transfer Fails
Symptom:
Taking a call back for IVR Service treatment or a subsequent network transfer does not work.
Message:
None.
Cause 1:
The VRU PIM that connects to the Unified CVP is not set to Type 5.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
84
Chapter 2: Symptoms and Solutions
Transfer and Connect
Action 1:
In Unified ICME Configuration Manager, Network VRU Explorer, change the Type to 5.
Cause 2:
The ICM script does not include a Set node setting the Call.NetworkTransferEnabled variable
to 1.
Action 2:
In any script that might perform VRU Network Transfer or Takeback for IVR Service treatment,
add a Set node at the beginning of the script setting the
Call.NetworkTransferEnabled variable to 1.
Call Disconnected As Soon As Agent Answers
Symptom:
Call disconnected as soon as agent answers.
Message:
N/A
Cause:
There is a codec incompatibility between the phone and the codec capabilities that Unified CVP
is sending.
Action:
Do the following:
1. Find the Region in which the phone resides. In CCMAdmin, check the Device Pool the
phone is in and the Region the Device Pool is in.
2. In VBAdmin, do a ‘ShowLocationsBasedCAC’.
–
If set to onGateway, look at the device pool (and subsequently Region) for the
originating gateway.
–
If set to offGateway, look at the device pool (and subsequently Region) for the Unified
CVP H.323 Service.
3. You now have two regions: the phone region and the originating region. In CCMAdmin
> System > Region, check what codec is to be used between those two regions. Both
caller and agent need to be able to speak that codec. If they cannot, a transcoding resource
needs to be allocated on Unified CM.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
85
Chapter 2: Symptoms and Solutions
Transfer and Connect
4. If the originating endpoint is an IOS gateway, most probably the codec is incorrectly
configured. IOS gateways can speak both g711 and g729.
On the gateway:
–
conf t
–
voice class codec 1
codec preference 1 g711ulaw
codec preference 2 g729r8
Note: For outside of North America, it is typically g711alaw.
–
Assign that voice-class codec to the Unified CVP voip dial-peer
5. If the originating endpoint is an IP phone (or a call initiated from Cisco Unified Outbound
Agent), codec renegotiation is not possible. Either the device region needs to be changed
or a transcoding resource needs to be allocated in Unified CM.
Call Immediately Disconnected and Caller Hears Nothing
Symptom:
Caller is immediately disconnected and does not hear anything.
Message:
None.
Cause:
There is a gateway misconfiguration on the IVR Service leg of the call.
Action:
If the call is clearly getting to the ICM script, but failing on the ‘Send to VRU’ node or on the
first Run External node if there is no explicit ‘Send to VRU’ node, then the gateway configuration
should be checked.
Agent Goes Ready, Queued Caller Never Delivered to Agent
Symptom:
Agent is ready but caller in queue never delivered to agent
Message:
None
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
86
Chapter 2: Symptoms and Solutions
Transfer and Connect
Cause 1:
The Interruptible box on the queuing Network VRU script is not checked.
Action 1:
In the Unified ICME Network VRU Script Explorer, look at the Network VRU script that is
queuing the caller. It must be marked Interruptible for the Unified ICME software to be able to
interrupt the script and deliver the caller to the available agent.
Cause 2:
The agent Device Target does not have a label associated with the Unified CVP routing client.
Action 2:
In ICM Device Target explorer, look at the labels for the agent device target. The label must
be associated with the Unified CVP routing client. If is not associated with a Unified CVP
routing client, then one must be added.
Cause 3:
The NetworkTransferEnabled flag has not been set to 1 in the ICM script.
Action 3:
At the beginning of the first Unified ICME caller entry script, and on every post-route script
that may be invoked later in the call, the NetworkTransferEnabled variable must be set to the
value of 1.
Cause 4:
The Gatekeeper routing is not configured properly and the Unified CVP H.323 Service cannot
find the correct Unified CM to which to send the agent transfer.
Action 4:
When the H.323 Service gets a device target label from the Unified ICME, it must look up that
label on the Gatekeeper in order to know to what IP address to send the call. Configure the
Gatekeeper with the correct routing information to resolve this issue.
Post-Route Transfers Do Not Work
Symptom:
Post-route transfers do not work.
Message:
None.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
87
Chapter 2: Symptoms and Solutions
Transfer and Connect
Cause:
There is an Unified ICME configuration error.
Action:
• Make sure the following are defined for the post-route number the agent is transferring to:
– Dialed number plan
– dialed number
– call type
– script
• Verify that post-routing is enabled on the Unified CVP Peripheral Gateway. Open the PG
Explorer utility and check under the Peripheral tab.
• Make sure NetworkTransferEnabled variable is set to 1 in the ICM script that occurred
before the post routing script. It is important to note that the NetworkTransferEnabled variable
applies to future transfers, so it must be set in the script before the post-route. If the agent is
going to do more post-routes, it must also be set at the beginning of the current post-route
script.
• The transfer label that is returned in the post-route script must be defined in the Unified CVP
routing client.
No Ringback on Blind Transfer
Symptom:
Caller does not hear ringing once an agent performs a blind transfer call to a second agent. At
that point, the caller only hears silence.
Message:
None
Cause:
Caller is transferred to ICM script and hears a prompt. The caller is transferred to an agent and
that agent performs a Single Step Transfer to another agent. At that point, the first agent is out
of the call, and the caller does not hear any ringing.
Action:
In CCMAdmin under Service > ServiceParameters, select your server and the Unified CM
Service. Then under H323 Parameters, set "Send H225 UserInfo Message" to "H225 Info for
Call Progress Tone".
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
88
Chapter 2: Symptoms and Solutions
Transfer and Connect
RONA and Router Requery Not Working as Expected
Symptom:
Ring On No Answer (RONA) and router requery are not working as expected.
Message:
None.
Cause:
RONA timers are not set correctly.
Action:
Check the RONA timers and reconfigure times. There are three system timers that affect RONA
behavior:
• Unified CVP - In VBAdmin, SetRNATimeout and ShowRNATimout. H.323 Service must
be restarted for a setting change to take effect.
• Agent Desktop settings Ring no Answer Time.
• Unified CM Forward No Answer Timer
Call Disconnected as Soon as Agent Answers
Symptom:
As soon as the agent answers, the call drops.
Message:
None
Cause:
There is a codec incompatibility between the phone and the codec capabilities that Unified CVP
is sending.
Action:
Do the following:
1. Find the Region in which the phone resides. In CCMAdmin, check the Device Pool the
phone is in and the Region the Device Pool is in.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
89
Chapter 2: Symptoms and Solutions
Transfer and Connect
2. In VBAdmin, do a ‘ShowLocationsBasedCAC’.
–
If set to on, look at the device pool and Region for the originating gateway.
–
If set to off, look at the device pool and Region for the Unified CVP H.323 Service.
3. There are two regions: the phone region and the originating region. In CCMAdmin
>System > Region, check what codec is being used between those two regions. Both
caller and agent need to be using the same codec. If are not, then a transcoding resource
needs to be allocated on Unified CM.
4. If the originating endpoint is an IOS gateway, the codec may be incorrectly configured.
IOS gateways can be configured for both g711 and g729.
On the gateway:
–
type: conf t
–
type: voice class codec 1 codec preference 1 g711ulaw codec
preference 2 g729r8
–
Assign that voice-class codec to the Unified CVP voip dial-peer
5. If the originating endpoint is an IP phone or a call initiated from Cisco Unified Outbound
Agent, then codec renegotiation is not possible. Either the device region needs to be
changed or a transcoding resource needs to be allocated in Unified CM.
DTMF Tones not Recognized
Symptom:
Dual-Tone Multi-Frequency (DTMF) tones are not recognized.
Message:
None
Cause 1:
If using ASR, and the caller is coming from the PSTN, the gateway dial-peers may be incorrectly
configured.
Action 1:
Possible Solution: In Unified CVP, there are two VoIP dial-peers involved in the call: One to
go from the gateway to the Unified CVP H.323 Service and one to go from the Unified CVP
H.323 Service to the IVR gateway.
Each of these dial-peers must have the following attributes to be recognizabled correctly by
DTMF:
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
90
Chapter 2: Symptoms and Solutions
Unified CM
• dial-peer voice 800 voip
• destination-pattern 8.........
• session target ipv4:10.64.228.100
• dtmf-relay rtp-nte h245-signal h245-alphanumeric
• codec g711ulaw no vad
Cause 2:
If using Scansoft ASR, the DTMF payload type is incorrectly configured in Scansoft.
Action 2:
In Scansoft config file C:\Program Files\ScanSoft\SpeechWorks OpenSpeech Media
Server\Server\config\OssServer.cfg, change the following value from 96 to 101:
server.transport.dtmfPayloadType VXIInteger 101
Unified CM
ASP Error Occurs on Unified CM
Symptom:
An .asp error occurs on the Unified CM and it does not function normally.
Message:
None.
Cause:
If the Call Server is installed on the same machine as the Unified CM, and you uninstall the
Call Server, some files the Unified CM needs might be deleted.
Action:
Reinstall Unified CM.
Unified ICME
Unified ICME Lookup Requests Show 0
Symptom:
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
91
Chapter 2: Symptoms and Solutions
Unified ICME
When reviewing the Unified ICME Lookup Requests in the Realtime Statistics area of the
Operations Console, the value typically reports zero requests.
Message:
ICM Lookup Requests = 0
Cause:
This request is too transient, unlike that of some of the other statistics, such as Active Calls. In
a 60 second call the ICM Lookup Request may only last a few hundred milliseconds or less.
The probability of catching a real time snapshot of this event is extremely low. As a result, the
reported value for that statistic will typically be zero.
Action:
No action required. The value will show zero most of the time for the Realtime Statistic of ICM
Lookup Requests.
Unified ICME Timeout Occurs and No Error Found
Symptom:
In a NAM/CICM environment, an Unified ICME timeout occurs and there is no indication of
an error in the Call Server logs.
Message:
None.
Cause:
If the time to retrieve the first script request (or connect) takes longer than the NAM’s remote
ICM timeouts, then default routing occurs.
Action:
Locate and correct any network or script delay, or increase the NAM remote ICM timeouts.
Call Receives Default Route Label From Unified ICME
Symptom:
A new call arriving at the Customer Voice Portal receives a default route label from the Unified
ICME. You do not see an Call Server error for this type of ICM timeout.
Message:
None.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
92
Chapter 2: Symptoms and Solutions
Unified ICME
Cause:
The Unified ICME is taking a long time to handle new call requests.
Action:
Ensure there are no wait nodes in the ICM script prior to the Run VRU Script node. For more
information, see the Cisco ICM Software Scripting and Media Routing Guide.
ICM Script Fails on Run VRU Script Node
Symptom:
The ICM script fails on a Run VRU Script node and there is no indication of a failure in the
Call Server log.
Message:
None.
Cause:
The Dialed Number, Network VRU Script or Network VRU might be associated with the wrong
Customer.
Action:
Use Configure ICM to ensure that the:
• Dialed Number and Network VRU Script are associated with the correct Customer.
• Customer is associated with the correct Network VRU.
Running Out of ECC Variable Space
Symptom:
Running out of ECC variable space.
Message:
None.
Cause:
IVR Service scripts are complex and require many ECC variables.
Action:
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
93
Chapter 2: Symptoms and Solutions
Unified ICME
Some amount of ECC variable space can be freed up by doing the following:
• As of Unified ICME 6.0 SR1, there are a number of Unified CVP ECC variables that are no
longer needed. You can save about 170 bytes by deleting them from your Unified ICME
configuration. They are:
– user.num.steps.executed
– user.session.handled
– user.task.id
– user.connect.script.config
– user.connect.script.name
– user.connection.type
– user.last.redirected.address
In addition, if you have Unified CVP ECC variables which you are not using and have set
them to a length of 1, they can now be eliminated as well, as of Unified ICME 6.0 SR1. That
may save a few more bytes.
• You may want to switch to VXML Server scripting. All of the ECC-heavy IVR Service
speech-enabled scripting that would normally require lots of ECC variables can now be
confined to the VXML Server.
VRU Script Returns Incorrect Result
Symptom:
When prompted, the caller enters digits, but the script returns incorrect results.
Message:
None.
Cause:
One of the most likely possibilities is that this is a scripting issue. For example, setting
user.microapp.play_data = 167.8900, but when the audio is played, customer hears "one hundred
sixty seven point eight nine". The "zero zero" from the value of 167.8900 is missing. This is
because of missing double quote signs to make the passed data "167.8900" as a string. Without
these double quote signs around the value, it is treated as a number instead of a string.
Action:
Add the double quote signs around the Micro-Application value.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
94
Chapter 2: Symptoms and Solutions
Unified ICME
Incorrect Number for Play Data Micro-applications
Symptom:
If you enter an integer value that has a max value of 0x7fffffff (2147483647) in the ICM script
it does not play the required prompt.
Message:
33: 10.77.60.174: Apr 20 2007 16:50:43.062 +0530:
%CVP_4_0_IVR-3-MICROAPP_FORMAT_ERROR: [VARIABLE_DATA(13)]
CALLGUID=0EB8F53B-10000112-69665ACE-0A4D3CAE: [VARIABLE_DATA(13)] A
number value is expected in '-1e+006' {VRUScriptName: PD,Number,8 ConfigParam: 9,Y }
[id:3016]"
Cause:
Large numbers must be used as strings (enclosed in quotes) in ICM scripts used by Play Data
Micro-applications.
Action:
If you are using integers that are larger than nine digits, enclose the value in quotation marks,
so it will be treated as a string.
Caller Is Restarted at Beginning of ICM Script
Symptom:
Caller experiences a situation where they are restarted at beginning of an ICM script.
Message:
None.
Cause:
The Call Restart feature is enabled in the Unified CVP H.323 Service.
Action:
There is an optional feature which can be enabled in the Unified CVP H.323 Service that restarts
the caller at the beginning of the ICM script in lieu of disconnecting the caller. Normally, without
this optional feature enabled, when some critical system error occurs, the caller hears the system
error message and is disconnected. This feature is enabled by doing SetNewCallOnly on in
VBAdmin. Do a ShowNewCallOnly in VBAdmin to see if the feature is enabled.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
95
Chapter 2: Symptoms and Solutions
Unified ICME
Numeric data Is not Playing in the Expected Format
Symptom:
Numeric data is not playing in the expected format.
Message:
None.
Cause:
The ICM script might be:
• Truncating values with leading 0’s or 0’s after decimals.
• Rounding values.
Action:
Put quotes around numbers in a Script Editor Set node so it is processed as a string. This is
especially important if:
• Leading 0’s are present (example: dates).
• Trailing 0’s are present after a decimal point (example: currency).
• The number is very large (example: a number normally expressed through exponential
notation).
Unified ICME Service Control Only Displays H.323 Service After Core Software Install
Symptom:
After rebooting the Call Server, the Unified CVP H.323 Service, and Unified CVP Resource
Manager should all start automatically. The issue is that only the H.323 Service is listed in the
ICM Service Control.
Message:
None
Cause:
The Unified ICME service control is for the legacy purposes that the Unified CVP H.323 Service
carries.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
96
Chapter 2: Symptoms and Solutions
Unified ICME
Action:
The user is required to check the All checkbox in the lower left corner of the Unified ICME
Service Control panel in order to see the Unified CVP Call Server and Unified CVP Resource
Manager services. All three are not displayed by default for the Unified ICME Service Control
list. Unified CVP Services should be started and stopped using the Operations Console.
Error Code 31, 32, 33 Returned to ICM Script
Symptom:
CALL_RESULT returns an error code 31, 32, or 33 to the ICM script.
Message:
Error code 31, 32, or 33 received.
Cause:
If using ASR/TTS, the gateway is unable to connect to the ASR or the TTS server.
Action:
To resolve this issue, the connection to the server must be restored.
Error Code 9 Returned to ICM Script
Symptom:
ICM script receives an error code 9.
Message:
Error code 9 received.
Cause:
The gateway or Unified CVP H.323 Service cannot connect to the media server or the prompt
does not exist.
Action:
To resolve this issue, the connection to the server must be restored.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
97
Chapter 2: Symptoms and Solutions
Content Services Switch
Content Services Switch
Content Services Switch Shows Call Server Out of Service but IVR Service Is up
Symptom:
Content Services Switch (CSS) shows Call Server out of service but IVR Service is up.
Message:
None
Cause:
The keepalive retry period is set too low.
Action:
The CSS polls the Unified CVP Call Server with a special Unified CVP script that is manually
placed on the CSS. The keep alive retry period must be greater than the retry period specified
in the script. Make sure the CSS keep alive retry period is set to 6. Also do a show script
<script name> on the CSS to verify that the script exists.
service vxml1
keepalive maxfailure 1
keepalive retryperiod 6
ip address 10.86.129.22
keepalive type script <script name> "10.86.129.31"
active
General System Issues
Intermittent Timeout Issues
Symptom:
ICM timeouts in the Unified CVP Call Server log.
Message:
N/A
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
98
Chapter 2: Symptoms and Solutions
General System Issues
Cause:
This may be due to your network settings not being set properly.
Action:
The chart below shows the speed and duplex settings for Unified CVP.
Ethernet Switch Speed
Capability
Server/Gateway NIC Speed Switch Port Speed/Duplex Server/Gateway NIC
Capability
Setting
Speed/Duplex Setting
1000 Mb
1000 Mb
Auto / Auto
Auto / Auto
1000 Mb
100 Mb
100 Mb / Full
100 Mb / Full
100 Mb
100 Mb
100 Mb / Full
100 Mb / Full
100 Mb
1000 Mb
100 Mb / Full
100 Mb / Full
Windows OS Application Log Fills up
Symptom:
Windows Operating System Application log fills up, and clearing the log only resolves the issue
temporarily.
Message:
None
Cause 1:
Event log settings are not set to roll over.
Action 1:
Change the Event setting so that the Application Log rolls over by doing the following:
1. Select Start > Programs > Administrative Tools > Event Viewer.
2. Right-click Application Log and select Properties from the menu.
3. Select the Overwrite events as needed option.
4. Click Apply and OK.
Cause 2:
Some unnecessary protocols might have been installed. For example, NetBEUI has been known
to fill the event log.
Action 2:
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
99
Chapter 2: Symptoms and Solutions
General System Issues
Uninstall protocols that are not necessary in this application.
Error Message: Operation Failed. Device Could Not Be Reached
Symptom:
Message:
"<DeviceType> device with IP Address: <IP> and Hostname: <hostname> operation failed.
Device could not be reached"
Cause:
Action:
Error When Adding a Dial-Peer
Symptom:
When adding a dial-peer, an error message is issued and the dial-peer is not added.
Message:
Could not add peer.
Cause:
This message implies that the amount of memory required to add the dial-peer is not available.
Action:
Use the show memory command to verify the available free memory. If the available free
memory is > 0, then the logging buffer is taking up all available memory. In this case, adjust
the amount of memory used by the log buffer by changing the value of logging buffered
<numeric value>. The recommended value for this setting is between 10 - 20% of the total
Gateway memory.
"Java is Not a Recognized Program" Error
Symptom:
"Java is not a recognized program" error message is displayed when trying to run the .bat files
in the Unified CVP Server/admin directory ( or the .bat files in the application specific admin
directory) after a Windows upgrade.
Message:
Java is not a recognized program
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
100
Chapter 2: Symptoms and Solutions
General System Issues
Cause:
The environment variable containing the java install path was overwritten during the upgrade.
Action:
Possible Solution 1: (Reboot the machine.)
VXML Server requires the use of an environment variable that is created by the installer. In
order for this environment variable to take effect, the machine must be rebooted. If the machine
was not rebooted by the installer, reboot the machine manually. Before rebooting, close all
running application and save any unsaved work.
Possible Solution 2: (Reset the Java Path system variable.)
1. From Start, select Programs > Administrative Tools > Computer Management.
2. From the Computer Management menu bar, select Action > Properties.
3. Select Advanced Tab > Environment Variables Settings.
4. In the System Variables Path, click Edit and add the path to where the java.exe file is installed.
"Password Does Not Meet the Operating System Security Requirements" Error
Symptom:
If an attempt is made to change an existing user password before the domain password policy's
minimum password age is reached, the user receives an error.
Message:
Password does not meet the operating system security requirements.
Cause:
This condition is caused by attempting to change an existing user password before the domain
password policy minimum password age has been reached.
Action:
The user password should not be changed until the minimum password age specified in the
password policy for the domain has been exceeded.
Password Lost or Forgotten
Symptom:
Password has been lost or forgotten.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
101
Chapter 2: Symptoms and Solutions
General System Issues
Message:
None.
Cause:
For some reason, a password has been lost or forgotten.
Action:
Contact your Cisco Partner or Cisco Technical Support for assistance.
Server IP Address Has Been Changed
Symptom:
The IP address of a Unified CVP server has been changed and is now listed as Not Reachable.
Message:
Not Reachable
Cause:
The IP address of a Unified CVP server has been changed locally instead of from the Operations
Console.
Action:
If you have already changed the IP address of the server, you may be able to delete it from the
Operations Console, but the properties files on the server will not be deleted. This is due to the
Operations Console not being able to access the server using original IP address. Adding the
server with the new IP address to the Operations Console will add new property entries, but
will not clean up the original property entries. The original property files will need to be cleaned
up using a manual process. Contact your Cisco Partner or Cisco Technical Support for assistance
when removing property files.
Below is the correct procedure for changing server IP addresses.
1. Create the new device from the Operations Console. Use the original device as template.
From the menu bar of the device, select the "Use As Template" option. Assign the new
IP address to the new device and click Save. Do not select Save and Deploy since you
have not changed the physical server to the new IP address.
2. Delete the device from the Operations Console before changing the IP address of the
server.
3. Configure the new IP address on the local server.
4. From the Operations Console, select Save and Deploy of the new device.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
102
Chapter 2: Symptoms and Solutions
General System Issues
5. Restart the server.
Unable to Open Command Prompt Window or Other Applications
Symptom:
It fails when trying to open a command prompt window, or some other application.
Message:
None.
Cause:
There are too many Windows threads in use on the system.
Action:
If the total number of calls configured in Call Server and VBAdmin is too high, the system can
run out if threads.
• 1. Open the Task Manager on the Call Server.
2. Click the Processes tab.
3. Click the View menu.
4. Click the Select Columns option.
5. Check the Thread Count box.
6. Click the Threads heading to sort by decreasing thread count. If the thread count starts
approaching 2000, then this is most likely the problem.
In VBAdmin, the ShowMaxTotalCalls should never exceed 400.
When Hyperthreading Is Enabled, Unified CVP Performance Is Degraded
Symptom:
When hyperthreading is enabled, the Unified CVP performance is degraded.
Message:
None.
Cause:
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
103
Chapter 2: Symptoms and Solutions
General System Issues
When hyperthreading is enabled in Windows Operating System on the Call Server, the number
of concurrent calls handled decreases by approximately ten percent.
Action:
Disable hyperthreading.
Users Cannot Login
Symptom:
Users, such as: Informix, cvp_dbadmin, and cvp_dbuser are unable to log in, even when using
their assigned password.
Message:
Login failed message
Cause:
The password for this user has expired. This can occur if the user has not logged in for the
number of days specified by the MaximumPasswordAge security policy, system calendar has
been set forward, or security policy has been modified.
Action:
Log in as administrator and change the password for the user.
Errors Reported by Various Unified CVP Components
Symptom:
Errors reported by various Unified CVP components. ICM script fails calls from Play Media
node, and many calls abnormally disconnect.
Message:
None.
Cause:
Running Symantec virus scan software with a heavy call load (greater than 2 calls per second)
can cause these types of errors.
Action:
If your call load is heavier than two calls per second, disable Symantec virus scan software and
use McAfee 8.0i virus scan software instead.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
104
Chapter 2: Symptoms and Solutions
H.323 Troubleshooting
Other Areas to Check
Symptom:
General failures.
Message:
None.
Cause:
Software version incompatibilities, hotfixes not installed or installed incorrectly, firewalls,
network slowness.
Action:
• Verify that the your software versions comply with the Unified CVP compatibility matrix in
the Hardware and System Software Specification for the Cisco Unified Customer Voice
Portal.
• Determine if there are any firewalls in the solution. Firewalls can introduce network latencies
and misrouted messages if it is not configured correctly.
H.323 Troubleshooting
H.323 Service Process Restarts Unexpectedly
Symptom:
H.323 Service process restarts unexpectedly
Message:
None
Cause 1:
An internal memory error occurred in the H.323 Service.
Action 1:
Search the H.323 Service logs for an occurrence of the word EXCEPTION around the time of
the failure. If found, this indicates a memory corruption issue. Contact your support provider
for additional information.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
105
Chapter 2: Symptoms and Solutions
H.323 Troubleshooting
Cause 2:
Another possibility is the user accidentally hung the H.323 Service console window by
highlighting text on the screen or using the scroll bar to view the contents of the console window.
Action 2:
Check the H.323 Service logs for an occurrence of the phrase H.323 Service will be restarted
in 1800 seconds. If no EXCEPTION errors are found around that time, it indicates that the user
inadvertently hung the H.323 Service process.
H.323 Service Out of Service
Symptom:
The H.323 service is out of service. Although it is out of service, the H.323 Service allows calls
in progress to complete, but does not accept new calls.
Message:
None.
Cause 1:
H.323 Service is not registered to a Gatekeeper.
Action 1:
In VBAdmin, verify the Gatekeeper IP address by typing showGatekeeper.
On the Gatekeeper, type show Gatekeeper endpoints
Cause 2:
There are no in-service Call Servers. The Call Server may take itself out of service when Unified
ICME response times become too long.
Action 2:
Within the Operations Console, go to Device Management > Unified CM. Go to Engine >
Engine Configuration. Change New Call Throughput Upper Threshold and Call Event
Throughput Upper Threshold values to 30000.
Cause 3:
A user has administratively taken the H.323 Service out of service.
Action 3:
A user can administratively take the Call Server out of service by selecting Go Out of Service
from AppAdmin > Engine. To resolve this issue, start the service.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
106
Chapter 2: Symptoms and Solutions
H.323 Troubleshooting
Cause 4:
The H.323 Service has gone out of service due to an internal processing error.
Action 4:
Consult your support provider for additional information regarding this type of event.
H.323 Service Log Indicates Wrong Number of Licensed IVR Service Ports
Symptom:
The H.323 Service log file indicates that the maxIVRPorts = 50, but the CVP.license file has
150 ports enabled.
Message:
H323CallMgr::getRegistrySettings: maxIVRPorts = 50
Cause:
It appears that the remaining ports are not being recognized. However, the Unified CVP License
file applies to the Call Server when using the SIP Protocol. The Unified CVP H.323 Service is
limited by the MaxTotalCalls registry value, so there is no correlation between the Unified CVP
VoiceBrowser MaxTotalCalls and the CVP.license file.
Action:
The maxTotalCalls registry value is based on the capacity for the Unified CVP H.323 Service
and should not be changed. In this case, you need to provision more than one Unified CVP
H.323 Service.
H.323 Service Must be Registered with Gatekeeper to Perform IP Transfers Error
Symptom:
Error message in the H.323 Service logs states “ERROR: H.323 Service must be registered with
Gatekeeper to perform IP transfers” during a call.
Message:
H.323 Service must be registered with Gatekeeper to perform IP transfers.
Cause:
The Unified CVP Call Server is not successfully registered with the H.323 Gatekeeper.
Action:
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
107
Chapter 2: Symptoms and Solutions
H.323 Troubleshooting
Set the Gatekeeper IP address from the VBAdmin using setGK <Gatekeeper IP address>.
Restart the Unified CVP Call Server after changing the H.323 Gatekeeper.
Unexpected Event for the Gatekeeper Registrations Error
Symptom:
Error Message in the H.323 Service Logs States “Unexpected event for the Gatekeeper
registration. Please make sure H.323 Gatekeeper is up and running.”
Message:
Unexpected event for the Gatekeeper registration. Please make sure H.323 Gatekeeper is up
and running.
Cause:
The configured H.323 Gatekeeper is not running or Gatekeeper is not configured correctly.
Action:
Follow these steps:
• Make sure the H.323 Gatekeeper is running on the IP address configured in the Unified CVP
Call Server. Verify that the correct IP address of H.323 Gatekeeper is configured.
• Verify that the IP network connectivity between the Unified CVP Call Server and H.323
Gatekeeper.
VBadmin Command Fails
Symptom:
An attempt to run VBAdmin from the command window fails. The browser disappears, the
window does not remain open long enough to accept a setGK command, and the following error
message appears.
Message:
"trace: PIMcom:Init - Failure connection to pipe \\.customer\voicebrowser\VBcmdpipe "
Cause:
Service Pack 1 or later of Windows 2003 is not installed on the system.
Action:
Install Windows 2003 Service Pack 1 or later.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
108
Chapter 2: Symptoms and Solutions
SIP Troubleshooting
SIP Troubleshooting
SIP to SIP Calls Do Not Play Ring Ringback Tone
Symptom:
The ringback tone is not heard when a SIP to SIP call is placed.
Message:
None.
Cause:
In high load situations, the playback of ringtone for many calls might over utilize the CPU
processing of the gateway.
Action:
In this case, it is necessary to use load balancing or configure the gateways to optimally distribute
media termination of calls.
The following are some facts regarding ringtone:
• Local ringback in SIP is performed with a 180 response process before the call is established.
• After the call has been established, ringtone is played back to the caller using an intermediate
transfer mechanism. This is accomplished by a transfer to a ringtone playback voip dialpeer
on the Gateway. Once the agent answers the call, the caller is then reinvited to the agent. At
that point, the ringtone dialog is dropped.
• Troubleshooting ringtone issues consists of configuring the dial-peer to match an incoming
voip call to the dialed number (DN) that is configured in the SIP Service for Ringtone. For
additional information for SIP configuration, refer to the Configuration and Administration
Guide for Cisco Unified Customer Voice Portal Guide.
• Ringtone playback is not the same as music on hold (MOH) on the Unified CM. For example,
when a caller is put on hold at Unified CM phone they hear music. The ringtone playback is
separate from this feature.
• Necessary items for ringtone playback:
– Ringback.wav on the Gateway.
– Ringtone service and TCL configured on the Gateway.
– Dial-peer associated with ringtone service.
– Static routes to route calls from Unified CVP to the Gateway.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
109
Chapter 2: Symptoms and Solutions
SIP Troubleshooting
Assigning DN Auto Answer Settings for Shared Lines
Symptom:
The Auto Answer DN setting is not compatible with Cisco TelePresence on shared lines.
Message:
None.
Cause:
Unified CVP calls routed through a shared SIP trunk line causes problems when using Cisco.
Action:
Do not enable the Auto Answer DN setting for shared Directory Numbers (DNs) when using
Cisco TelePresence.
SIP Calls Experience Long Delay or Fast Busy
Symptom:
During the making of a SIP call, the caller experiences a long delay or hears a fast busy condition.
Message:
None.
Cause:
If the calls go through, but there is a long delay or a fast busy, it could be the result of a Domain
Name Server (DNS) problem in the proxy.
Action:
Verify that the correct nameserver and domain information was configured in the /etc/resolv.conf
file. If DNS is not being used, then remove any entries in that file.
SIP Calls Receives 500 Internal Server Error "Routing Failed" Event
Symptom:
A SIP call fails and a 500 Internal Server Error "Routing Failed" event is received.
Message:
500 Internal Server Error "Routing Failed".
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
110
Chapter 2: Symptoms and Solutions
SIP Troubleshooting
Cause:
A SIP Trunk is not configured to receive or send calls to the IP Address from the source of the
SIP INVITE event.
Action:
The solution is to configure a SIP Trunk to the IP address of the sending endpoint, CUPS,
Gateway, or Unified CVP.
SIP Call Receives 489 Bad Event Message
Symptom:
A 489 Bad Event message is received from a Unified CM or Proxy Server when a SIP call is
made.
Message:
489 Bad Event
Cause:
Logs indicate 489 Bad Event messages are being received from either the Unified CM or the
Proxy Server. This event is used to indicate that the server did not understand the event package
that was specified in the Event header field. The problem is that Key Press Markup Language
(KPML) is trying to be used, and is currently not supported in Unified CVP.
Action:
The solution is to only use the inband Dual-Tone Multi-Frequency (DTMF) method setting
(RFC 2833) on the SIP Trunk that is associated with the Unified CM or the SIP Proxy Server.
This is done by selecting the last checkbox on the SIP Trunk configuration page of the Unified
CM Administration page.
Back to Back User Agent (B2BUA) Sending 503 Server Not Available
Symptom:
The Back-To-Back User Agent (B2BUA) is sending a 503 Server Not Available response before
the call is even established.
Message:
503 Server Not Available
Cause 1:
This is most likely caused by the SIP service not running. The ICM VRU PIM needs to connect
to the ICM Subsystem for it to go into service. When that happens, SIP can then go into service.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
111
Chapter 2: Symptoms and Solutions
SIP Troubleshooting
If the ICM VRU PIM connection is lost, then the SIP Service goes into partial service and rejects
calls.
Action 1:
Enable the ICM Subsystem to allow the SIP Subsystem to go into service.
Cause 2:
This is a license depletion issue, which typically equates to a high call rate. When all the licenses
are in use, the service goes into a partial state and rejects calls. When the active call number
drops and licenses are available again, then the SIP Subsystem will start accepting calls.
Action 2:
Monitor the call statistics from the Operations Console to determine if additional licenses are
needed.
REJECTED WITH 401 - Unauthorized Reason: Q.850;cause=57
Symptom:
SIP call failure.
Message:
%CVP_4_0_SIP-3-SIP_CALL_ERROR: GUID:24FCDB65-129111DB-82A20014-6944B792
- LEGID: 115289260464070 - OUT - DsSipInvitation <sip:1106@10.86.129.214;transport=tcp>;tag=11296519 REJECTED WITH 401 - Unauthorized
Reason: Q.850;cause=57
Cause:
Unified CM SIP Trunk Security Profile has authentication turned on.
Action:
Clear the digest and other authentication check boxes on the Unified CM SIP Trunk Security
Profile associated with the SIP Trunk that is being used with Unified CVP B2BUA.
MTP Resource Allocated Even Though MTP is Disabled
Symptom:
Media Transfer Protocol (MTP) Resource is allocated for each call coming into SIP trunk even
though MTP disabled is selected on the trunk.
Message:
None.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
112
Chapter 2: Symptoms and Solutions
SIP Troubleshooting
Cause:
Failure to negotiate DTMF method occurs for the call, according to the DTMF method supported
by the endpoints involved in the call.
Action:
Set the endpoints of the call to negotiate the same DTMF method, that is RFC2833 for Unified
CVP calls.
Note: Out of Band (OOB) Key Press Markup Language (KPML) is not supported for the Unified
CVP 4.0 and 4.1 releases.
SIP Call Fail on SIP Trunk on Unified CM to Back to Back User Agent
Symptom:
SIP calls cannot be made out on the SIP Trunk on the Unified CM to the Back to Back User
Agent (B2BUA).
Message:
None.
Cause:
The route pattern is not defined, in use, or has the wrong wildcards configured. For example, a
pattern of 7xxx routes any calls with a DN beginning with a 7 and has 3 more digits after it.
Using something like 7* does not work.
Action:
Correct the route pattern on the Unified CM.
SIP Call Is Cleared From Unified CVP with Code 127
Symptom:
SIP call is cleared from Unified CVP with code 127.
Message:
None.
Cause:
Call was detected as a phantom or zombie call, which may have happened as a result of a network
down scenario.
Action:
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
113
Chapter 2: Symptoms and Solutions
SIP Troubleshooting
Check the server network connectivity.
Critical Error Audio Is Not Heard on an Abnormal Disconnect SIP Call
Symptom:
A critical error wav file is not heard on an abnormal disconnect call. Instead the caller hears a
fast busy condition, or the call just drops.
Message:
None.
Cause:
When using REFER transfers, including the REFER used to play back critical error.wav for
abnormal disconnects, an outbound voip dial peer is necessary on the ingress gateway. This
outbound dial peer is needed because when the REFER message comes into gateway from
Unified CVP Call Server, it needs to match an outbound dial peer in order for the call to succeed,
or else a 503 rejection occurs if no dial peers match the REFER-TO header URI. Dial peer
destination targets must match the labels in the REFER-TO SIP URI, that is <error
DN>@<sip-server> and other labels that may be used in the Unified ICME routing label.
Action:
Configure an outbound voip dial peer on the ingress gateway. For example:
dial-peer voice 1050 voip
destination-pattern 9.......
voice-class codec 1
session protocol sipv2
session target sip-server
session transport tcp
dtmf-relay rtp-nte h245-signal h245-alphanumeric
no vad
SIP Calls Survivability Script Failing
Symptom:
Survivability script does not seem to work.
Message:
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
114
Chapter 2: Symptoms and Solutions
SIP Troubleshooting
None.
Cause:
SIP calls from the IOS gateway are getting REFER to the ERROR DN, and critical error wav
file is heard (or not heard). SIP calls originating from gateway do not contain "--CVP" in the
remote party id header display name of the INVITE.
Action:
The pots dial peer must match the incoming called number and be associated with the
survivability service. For example:
dial-peer voice 1000 pots
service cvp-survivability
incoming called-number 8T
direct-inward-dial
482 Loop Detected Error from CUPS SIP Proxy
Symptom:
482 loop detected error is received from the CUPS SIP Proxy. Ringing is heard by caller, but
the call is never answered.
Message:
482 Loop Detected
Cause:
When using SRV with a CUPS farm for failover, the SIP URL from Unified CVP contains the
SRV domain name. Proxy loops that call back to itself because it does not know about itself as
the SRV domain.
Action:
Under Service Parameters section in CUPS admin, add the domain name in the "SRV cluster
name" editbox.
SIP Call Fails with 404 Not Found Error Message
Symptom:
Sending call to Unified CM with a domain name like an SRV cluster name fails with 404 not
found rejections.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
115
Chapter 2: Symptoms and Solutions
SIP Troubleshooting
Message:
404 Not Found.
Cause:
The cluster fully qualified domain name (FQDN) is missing.
Action:
Go to System -> Enterprise Parameter -> Cluster Fully Qualified Domain Name on the Unified
CM and add the Cluster FQDN.
H.323 Gateway/SIP Call with IVR Service is Terminated with Reason Code: Q.850;Cause=38
Symptom:
Any failure of the bootstrap VXML Server fetches to Call Server causes the SIP IVR Service
leg of call to be terminated by the recovery handoff TCL scripts on VXML Server Gateway
with a Q850 code of 38.
The same error also occurs on the H.323 gateway; however, the failure occurs less often.
Message:
Q.850;Cause=38
Cause:
If the Unified CVP loses network connectivity, the VXML Server Gateway is not able to get
information from the IVR Service, and as a result a code 38 rejection is generated in the Gateway
logs.
Action:
Be sure that the network interface configuration is as follows:
ip route-cache same-interface
ip route-cache cef
ip route-cache
ip mroute-cache
no cdp enable
If specified, remove the following line from the network interface:
keepalive 1800
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
116
Chapter 2: Symptoms and Solutions
SNMP Troubleshooting
SIP Call Drops When Mobile Agent's Phone (Remote CTI Port) is Busy
Symptom:
If you are using Mobile Agent with the CUPS SIP Proxy and CUPS is configured with multiple
static routes for the same Destination Pattern to Unified CM nodes, calls may be dropped when
the Mobile Agent's phone rings busy.
Message:
Multiple 180 Ringing messages and the 403 Forbidden message are sent to CVP. ICM GED
125 Dialog Failures occur between CVP and ICM.
Cause:
By default, CUPS attempts to reach the Mobile Agent's phone using each of the configured
static routes.
Action:
In the CUPS Service Parameter Configuration Window, add 403 to the "4xx Contact Failover
Exception List".
SNMP Troubleshooting
SNMP Notification: authenticationFailure
Symptom:
The SNMP log shows frequent connect and disconnect events from the Reporting Server.
Message:
Received SNMPv2c Trap:
Community: public
From: <ip address>
sysUpTime.0 = 00:09:54.02
snmpTrapOID.0 = authenticationFailure
Cause:
Missing SNMP Community Strings.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
117
Chapter 2: Symptoms and Solutions
Installation Troubleshooting
Action:
You need to manually import the previously configured community strings from the windows
SNMP Service on that machine. Complete the following steps.
Importing Previously Configured SNMP Community Strings (windows)
1. To see a list of the previously configured Windows SNMP V1 community strings:
–
a. Open Windows Services viewer.
–
b. Right-click SNMP Service and select Properties.
–
c. Select the Security tab.
This tab lists the accepted V1 community strings and the access granted for each
string.
It also lists the hosts from which SNMP packets are accepted.
Note: The accepted hosts apply to all community strings, whereas the Operations
Console gives you more granularity, allowing you to specify accepted hosts on a
community string basis.
2. Configure these community string from the Operations Console.
–
a. Open the Operations Console and select SNMP | V1/V2C | Community String.
–
b. For each community string discovered above that has not already been configured
in the Operations Console, add it via Add New.
c. Enter the community string exactly as it was listed in the step above.
d. Choose V1 as the version.
e. For windows community strings with permission other than Read Only, select
Read Write in the Operations Console.
f. Be sure to select the device that the community string was reported.
Installation Troubleshooting
Increase Size of the Virtual Memory Paging File
Symptom:
Windows Virtual Memory is too low. Event Viewer (System log) shows that system is running
out of virtual memory.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
118
Chapter 2: Symptoms and Solutions
Installation Troubleshooting
Message:
Application popup: Windows - Virtual Memory Minimum Too Low: Your system is low on
virtual memory.
Cause:
Windows is increasing the size of your virtual memory paging file. During this process, memory
requests for some applications may be denied.
Action:
To change the size of the virtual memory paging file, perform the following steps.
1. From the Windows Start menu, click Start --> Settings --> Control Panel.
2. Double-click System.
3. On the Advanced tab under Performance, click Settings.
4. On the Advanced tab under Virtual memory, click Change.
5. Under Paging file size for the selected drive, click Custom size and type a new paging
file size in megabytes in the Initial size (MB) and Maximum size (MB) boxes, and then
click Set.
Note: The Initial and Maximum size should be set to the same value.
InstallShield Engine Cannot Be Installed
Symptom:
During Unified CVP installation, the following error message appears:
Message:
“The InstallShield Engine(iKernel.exe) could not be installed. Ikernel.exe could not be copied
to C:\Program Files\Common Files\InstallShield\Engine\6 \Intel 32.”
Cause:
Multiple instances of the Install Program are being run.
Note: InstallShield (the install program for the Unified CVP) allows only one ikernel.exe to
be running at a time.
Action:
Stop all install programs and run the Unified CVP installation program.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
119
Chapter 2: Symptoms and Solutions
Installation Troubleshooting
File in Use Message When Uninstalling
Symptom:
While trying to uninstall Unified CVP, a dialog box displays with the message a file is in use
and the software cannot be uninstalled.
Message:
File is in use and the software cannot be uninstalled.
Cause:
H.323 Service, and dumplog use the files msgs2.dll, Icrmsgs.dll, cat2.dll, and Icrcat.dll.
Sometimes the filter files appl2.flt, custvis2.flt, object2.flt, and primpos2.flt. If you try to uninstall
while one of the other processes is using these files, the Uninstall procedure is not able to remove
them.
Action:
Before running Uninstall, verify that:
• All Unified CVP services are stopped.
• The Unified ICME Service Control Panel is closed.
• The dumplog utility is not running.
• All file accessing programs (such as Microsoft Word or Windows Explorer) are closed.
Error Uninstalling WebSphere Application Server
Symptom:
You encounter an error condition when you attempt to uninstall IBM WebSphere Application
Server.
Message:
Several possible.
Cause:
Several possible.
Action:
See the Unified CVP Installation log for details and instructions.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
120
Chapter 2: Symptoms and Solutions
Installation Troubleshooting
See Also
IBM WebSphere Application Server documentation and support website.
Password Issues
Symptom:
Passwords must meet complexity requirements.
Message:
N/A
Cause:
This security setting determines whether passwords must meet complexity requirements.
Action:
If this policy is enabled, passwords must meet the following minimum requirements:
• Not contain all or part of the user's account name.
• Be at least twelve characters in length.
• Contain characters from three of the following four categories:
– English uppercase characters (A through Z)
– English lowercase characters (a through z)
– Base 10 digits (0 through 9)
– Non-alphabetic characters (for example, !, $, #, %)
• Complexity requirements are enforced when passwords are changed or created.
Reporting Password Error During Installation
Symptom:
When you specify the Reporting password during Reporting installation, you encounter the
following message:
Message:
Error 2245 while running: changing default password for user Informix.
Cause:
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
121
Chapter 2: Symptoms and Solutions
Installation Troubleshooting
The password security policy on the machine on which you are installing Reporting conflicts
with Unified CVP Reporting password requirements. Most likely, the machine password security
policy has a minimum password age greater than 0 or the minimum password length is greater
than 13 characters.
Action:
On the machine on which you are installing Reporting, abort the installation process. Open the
local Security Policy control panel (Control Panel > Administrative Tools > Local
SecurityPolicy) and examine the Password Policy (Security Settings > Account Policies >
PasswordPolicy). If the password policy on your machine conflicts with Unified CVP Reporting
password requirements, make the appropriate adjustments. Restart the installation process.
Install Radvision Package in Linux
Symptom:
MySQL service will not start.
Message:
Starting MySQL../etc/init.d/mysql: line 151: kill: (15147) - No such
process [FAILED]
Cause:
The problem is caused by "selinux" or Security Enhanced Linux which is commonly installed
with linux.
Action:
Disable it by editing /etc/selinux/config and changing SELINUX=enforcing to disabled. Then
reboot.
Dialog Box During Reporting Uninstallation
Symptom:
If you are removing the Unified CVP Reporting component, the following message displays
when component removal is complete:
Message:
Could not delete product files and/or directories.
Cause:
This dialog box is a present in this release solely due to a Informix product limitation, and
doesn't indicate a problem with removing Reporting.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
122
Chapter 2: Symptoms and Solutions
Installation Troubleshooting
Action:
No action necessary.
Install with Non-US English Region/Locale Settings
Symptom:
ICM subsystem not starting.
Message:
Cause:
CVP software was installed using non-US English region and locale settings for OAMP and
Call Server.
Action:
Reinstall the system with US English locale.
Call Server CPU spikes to 90%-100%
Symptom:
During the installation process, the Call Server CPU shows high utilization adn spikes to
90%-100%.
Message:
Error logs on the outprocess service indicating terminateConnection:Transport Down every 5-6
seconds.
Cause:
This occurs if there is an outprocess service (for example a VXML Server, REP Server or ORM)
which does not have the correct version of cvp-common.jar. This can happen, if the CallServer
or any other component is uninstalled and the CVP_HOME folder is not deleted after the
uninstall. Re-installing the component, does not delete the cvp.war file or overwrite it.
Action:
Ensure all components have the correct versions installed and the cvp-common.jar file is
consistient on all components. Re-installing the correct build on all services is the right way to
correct this problem.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
123
Chapter 2: Symptoms and Solutions
Gateway Troubleshooting
Gateway Troubleshooting
Ringing Stops on IP Phone Calling TDM
Symptom:
Ringback tone is clipped when an outbound call is placed from TNP IP phones to a TDM
endpoint. For IP originated calls whose destination is a TDM endpoint, and the call goes through
Unified CVP, this shows up when the call originates from TNP phones (7971, 7970, 7961,
7941). Customer experience will be something like that given in the call flow example below:
• Dial 1234 from a TNP phone (7941, 7961, 7970, 7971)
• Call goes out a SIP trunk to Unified Cisco Voice Portal
• Unified CVP sends call to a Voice Gateway (for example, AS5400) to receive IVR treatment
• Caller hears a prompt, and so forth
• Call then gets transferred to a destination on the TDM side (for example, outbound call to
endpoint with extension 5678)
• On the IP phone side, user hears the first ring but then nothing while the destination end point
continues to ring
Message:
None.
Cause:
The appropriate settings were not made on the Gateway that is placing the outbound call.
Action:
Apply the following setting on the POTS dial-peer on the Gateway that is placing the outbound
call to the TDM destination:
GW(config-dial-peer)#progress_ind alert strip
Gateway Cannot Connect to ASR/TTS Server
Symptom:
Gateway cannot connect to ASR/TTS server.
Message:
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
124
Chapter 2: Symptoms and Solutions
Gateway Troubleshooting
None.
Cause:
There is a connectivity problem between the gateway and the ASR/TTS server or there are not
enough ASR/TTS licenses available.
Action:
Check that the Gateway has the correct IP addresses for the ASR and TTS servers, as well as
the ASR and TTS backup servers.
Run a series of ping tests to each of the IP addresses of the ASR and TTS servers and backup
server. Verify that there is network connectivity to each of the servers.
Check that the ASR and TTS services are running.
Check that the IVR Service configuration is correct.
Pressing Digit 9 Causes No-Match
Symptom:
In DTMF-only adapter, pressing digit 9 while using options causes no-match after 10 seconds.
Message:
None.
Cause:
This delay is caused by the following link grammar in the root document for the DTMF-only
adapter.
<link event="nomatch"> <grammar mode="dtmf"
type="application/grammar+regex">99999</grammar>
</link>
Since the options are created with modal=false, the link grammar remains active in the Gateway
DTMF-browser. On pressing 9, the Gateway starts to match this grammar and waits for the next
input instead of returning a 'no-match' immediately.
Action:
When writing a VXML Server script for a DTMF-only adapter that uses the Get Digits, Forms,
or n_Option_Menu elements, follow this procedure to avoid delay in processing the digit 9
when the script is executed.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
125
Chapter 2: Symptoms and Solutions
Gateway Troubleshooting
1. In the VXML Server script, insert a hotlink to an inline grammar. When configuring the
hotlink’s grammar, the values of the digits must not match: –The value of any digit that
is expected in response to the Get Digits element. –The value of an option number used
in any n_Option_Menu element.
2. If the inline grammar in Step 1 contains more than one digit, set the interdigittimeout in
the root document properties to 5 seconds. For example:
interdigittimeout 5s
Examples:
If the VXML Server script uses 3_Option_Menu, where 1, 2, and 9 are valid options, then the
inline grammar can have the value 8, but cannot have the value 1, 2, or 9.
Similarly, if the VXML Server script uses 9_Option_Menu, where digits 1 through 9 inclusive
are valid options, then the inline grammar can have the value 99, and the interdigittimeout must
be set to 5s.
If you are using a GetDigits element and have set the follow range: min=2, max=4, then the
link grammar can be 99999, and you must set the interdigittimeout to 5 seconds.
If the script contains an Option Menu as well as GetDigits elements, you only need to set the
link grammar once.
Receiving Bootstrap Warning Error
Symptom:
When adding the bootstrap service to the gateway you get bootstrap warnings.
Message:
config-app)# service bootstrap flash:bootstrap.tcl
(config-app-param)# paramspace english language en
(config-app-param)# paramspace english index 0
(config-app-param)# paramspace english location flash
(config-app-param)# param CVPcallserver-primary 10.3.3.14
Warning: parameter CVPcallserver-primary has not been registered under
bootstrap namespace
(config-app-param)# paramspace english prefix en
(config-app-param)# param CVPcallserver-port 8000
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
126
Chapter 2: Symptoms and Solutions
Gateway Troubleshooting
Warning: parameter CVPcallserver-port has not been registered under
bootstrap namespace
(config-app-param)# param CVPcallserver-backup 10.3.3.14
Warning: parameter CVPcallserver-backup has not been registered under
bootstrap namespace
Cause:
Verify that the service is configured correctly.
Action:
When this error is received, it is usually a benign warning event. Verify that the service has
been configured correctly. To view the configuration, type the following command:
router># show run
The output should still show that it has been configured, as the example below shows.
service bootstrap140 flash:bootstrap.tcl
param CVPcallserver-backup 10.3.3.14
paramspace english language en
paramspace english index 0
paramspace english location flash
paramspace english prefix en
param CVPcallserver-primary 10.3.3.14
param CVP-port 8000
Switch Transfer Is Not Going to the Gateway Defined in the Gatekeeper Zone Prefix Command
Symptom:
Switch Transfer is not going to the Gateway defined in the Gatekeeper zone prefix command.
Message:
None.
Cause:
The IP address of the outbound Gateway, which is registered with a DNS name, is used in the
zone prefix.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
127
Chapter 2: Symptoms and Solutions
Cisco Unified Presence Server
Action:
Use the same name of the outbound Gateway in zone prefix. To verify the outbound Gatekeeper
priority list, do the following:
1. Enter the show Gatekeeper gw-type-prefix command in the Gatekeeper CLI.
2. Enter the show Gatekeeper end command in the Gatekeeper CLI and verify that H323
ID used by egress gateway/endpoint to register with Gatekeeper is used in the Zone prefix
configuration of Gatekeeper.
Cisco Unified Presence Server
Installation Keeps Failing in Network Connectivity Validation Page
Symptom:
During the installation of the Cisco Unified Presence Server, the installation fails at the Network
Connectivity Validation screen.
Message:
None
Cause 1:
Wrong hostname is configured in the server.
Action 1:
Make sure the CUPS hostname is configured correctly and it matches the name specified in the
Call Server page of the CM Publisher.
Cause 2:
Wrong CM Publisher hostname or password.
Action 2:
Make sure the CM Publisher's hostname and password are entered correctly. If you have DNS
server setup, make sure the CM Publisher's hostname is resolvable.
The Install Complains About Hardware Check Failure
Symptom:
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
128
Chapter 2: Symptoms and Solutions
Operations Console (OAMP)
During the installation of the Cisco Unified Presence Server, the installation complains about
a hardware check failure.
Message:
None
Cause:
Using non-compatible MCS server hardware.
Action:
Verify that you are using a supported server.
Call Fails When Transferred to Agent
Symptom:
A call is transferred to an agent, and the agent is reserved, but call transfer fails.
Message:
None.
Cause:
Cisco Unified Presence Server (CUPS) trunk on the Unified CM is not configured.
Action:
Configure the CUPS trunk on your Unified CM. Refer to the Configuration and Administration
Guide for Cisco Unified Customer Voice Portal for configuration information.
Operations Console (OAMP)
OAMP Functionality Is Limited
Symptom:
User is able to log in to the Operations Console, but cannot navigate the drop-down menus.
Drop-down menus do not respond to mouse clicks or key stroke combinations.
Message:
No message displays.
Cause:
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
129
Chapter 2: Symptoms and Solutions
Cisco Security Agent (CSA)
Javascript is disabled in Internet Explorer.
Action:
Enable JavaScript in Internet Explorer.
Cisco Security Agent (CSA)
CSA Agent is Disabled; You Cannot Re-enable It
Symptom:
CSA agent is disabled on the majority of boxes and cannot be re-enabled in the usual ways.
Message:
An Application Event error is recorded as follows:
Event Type: Information
Event Source: CSAgent
Event Category: Local Event Mgr
Event ID: 256
Description: The Agent Rule file hash has been modified. Shims will be disabled, and NO
security policies will be enforced.
Cause:
This happens when the new standalone agent is installed over the current one.
Action:
1. Navigate to C:\Program Files\Cisco Systems\CSAgent\cfg\
2. Delete the agent.state file
3. Restart the agent
Note: If this problem persists, uninstall the agent and re-install the agent again.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
130
Chapter 2: Symptoms and Solutions
Support Tools
Support Tools
Unable to Login to Support Tools
Symptom:
Attempts to login to the Support Tools fails.
Message:
Login failed.
Cause:
The Support Tools server was not installed in a domain.
Action:
Log into the Support Tools using the following format:
.\<username>
Backup and Restore
Reporting Server Purging Daily
Symptom:
Purge events are occurring on a daily basis on the Reporting Server.
Message:
None.
Cause:
The retention times might be set too high.
Action:
Back up the database, then change the retention times.
To configure Reporting Database purge settings:
1. From the Operations Console, choose Device Management > CVP Reporting Server.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
131
Chapter 2: Symptoms and Solutions
Backup and Restore
2. In the Reporting Server window that opens, click Edit.
3. Select the Database Administration menu in the toolbar, then select Data Purge.
4. Change the data retention time for each category of data.
5. Enter your Database Administrator Password and click Save & Deploy.
For a complete list and description of ECC variables, see the Configuration and Administration
Guide for Cisco Unified Customer Voice Portal.
Changing Backup Server Requires Restart
Symptom:
Changing backup server requires a restart.
Message:
None.
Cause:
Changing primary or backup Call Server updates the Call Server messageAdapter.properties
file and informs the user to restart all the affected devices.
Action:
Restart all of the affected devices.
"Retrying ASR Server. Last tried BACKUP asr-en-US-backup. primary tried:0, backup tried:1" Error
Symptom:
Missing parameters in the nuance-resources.txt file.
Message:
Retrying ASR Server. Last tried BACKUP asr-en-US-backup. primary tried:0, backup tried:1
Cause:
Missing parameters in the nuance-resources.txt file.
Action:
If you are using Nuance speech recognition to recognize DTMF digits, be sure the following
parameters are in your nuance-resources.txt file on the recognition server machine:
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
132
Chapter 2: Symptoms and Solutions
Backup and Restore
dtmf.SuppressNotifications=FALSE
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
133
Chapter 2: Symptoms and Solutions
Backup and Restore
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
134
Appendix A
Error Messages
This appendix provides detailed information on error messages.
Infrastructure Error Messages
Table 2: Infrastructure Error Messages
Title
Description
Cause
Resolution
EMERGENCY
STATE_MANAGER_
STARTUP_FAILURE
StateManager did not start.
The specific cause will be The Call Server must be
noted in the log message. restarted to resolve this issue.
STARTUP
start up failed.
The specific cause will be The Call Server must be
noted in the log message. restarted to resolve this issue.
START
No Services have been started. The Call Server has not
been setup through the
Operations Console.
SHUTDOWN
Unable to shutdown the Web
Application Server (WAS).
Login to the Operations
Console, and create a Call
Server with the IP address of
this server.
The specific cause for the Before attempting to restart
shutdown will be noted in the server, manually shut it
the log message.
down with the command:
CVP_HOME\bin\runcvp
stopservice call
If this fails, you must force
the process to stop with
Windows Task Manager.
Once it has been stopped,
then restart the server.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
135
Error Messages
Infrastructure Error Messages
Title
Description
Cause
Resolution
PORT_THRESHOLD
The Port License Checkout
Threshold is at the threshold
level.
The volume of calls on the
Call Server is approaching
the maximum capacity
allowed in the license.
When the call volume
decreases, a message stating
that the threshold has
returned to the safe level will
be logged. If the system is
often running at or near
licensed capacity you may
wish to contact Cisco or your
Cisco Partner to discuss the
increasing of your system
capacity.
ALERT
PORT_THRESHOLD
The Port License Checkout
The volume of calls on the No steps need to be taken.
Threshold has returned to the Call Server is no longer
safe level.
near the maximum
capacity.
SNMP_CONNECTION_
FAILURE
Could not bind serviceability There is a non-Unified
manager to <port>. Retrying in CVP process running
<time> minutes.
which is bound to the
port(s) listed.
(AIX) SNMP_
INIT_FAILURE
Could not start SNMP Agent. The specific cause will be Resolve the reason for start
noted in the log message. up failure, and then restart the
SNMP Agent. To restart the
SNMP Agent, type:
If this message is consistently
being logged, the processes
bound to the listed port(s)
should be stopped, and then
restart Unified CVP.
CVP_HOME\bin\snmp-wrapper.
sh start
at a command prompt.
Note: CVP_HOME is the
location where Unified CVP
was installed.
(AIX) SNMP_
FORWARDER_INIT
_FAILURE
Could not initialize SNMP
Forwarder.
The specific cause will be The Unified CVP application
noted in the log message. that logged this error must be
restarted to enable the SNMP
services.
CRITICAL
CAH_STARTUP _FAILURE The Common Admin Handler The server has been
will not run properly. The
misconfigured. The service
underlying service is absent. noted in the log message
failed to start.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
136
Try restarting the server. If
this message appears during
the restart, then the server
must be removed from the
Operations Console. Once it
has been removed, you can
Error Messages
Infrastructure Error Messages
Title
Description
Cause
Resolution
recreate it from the
Operations Console.
HEARTBEATS_ STOPPED Heartbeats from <service ID>
stopped. Unilaterally setting
state to PARTIAL_SERVICE.
Sending update.
LICENSE_ EXPIRATION
The service noted in the log
message has failed or lost
connectivity to the Call
Server.
If the noted service has been
manually shutdown, then this
message is normal and
expected. If it was not
manually shutdown, then
check if it has failed. If it
failed, attempt to restart it.
Verify that the server on
which the noted service
should be running has a valid
network connection to the
Call Server. Once everything
is running and connected you
will see the following
message: "STATE_
MANAGEMENT". This
message notes that the
"Heartbeats from <service
ID> resumed" where the
service ID's match.
The <license type> license for The license on the system
<licensed utility> has expired, has expired and the system
gracefully shutting down the will shutdown gracefully.
system.
The system will not start with
an expired license. Contact
Cisco or your Cisco Partner
to obtain a new license for
your server.
ERROR
LOGMSG_SET_
PROP_TYPE_ERROR
Set property operation failed
for the specified Property.
This occurs when Property Restart CallServer, OAMP,
Type is not a String.
and ORM. Reset the property
through OAMP only.
LOGMSG_SOCKET_
SETUP
Errors were encountered during Network issues could cause Ensure all servers are in the
socket setup.
problems while creating same domain. If this message
sockets.
occurs repeatedly, restart the
relevant server (that is,
callserver/vxml/reporting/iContact
application).
START
Could not start <service id>
due to: {reason}
SHUTDOWN
{shutdown type} shutdown of Either a forced or graceful
[{subsystem id}] failed due to: shutdown (as noted) of the
{reason}
subsystem failed due to the
reason noted.
The subsystem noted in the Restart the Call Server.
log message failed to start
up due to the reason noted.
If the Call Server successfully
shuts down, then this
message can be safely
ignored. If the Call Server
failed to shutdown (as
reported by Windows
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
137
Error Messages
Infrastructure Error Messages
Title
Description
Cause
Resolution
Services) you must stop it
manually with the command:
CVP_HOME\bin\runcvp
stopservice call.
If this fails, you must force
the process to stop with
Windows Task Manager.
SECURITY
Could not configure secure
connection in Tomcat's
server.xml.
During the installation,
modification of the
server.xml file failed.
Key size of {size} illegal, Java During setup of the security
security policies allow
system, local security
maximum keysize of {size}. policies caused key
creation to fail.
(AIX) SERVER_SETUP
SNMP Agent Server setup
failed because {reason}.
Make sure to completely
uninstall Unified CVP,
reboot, and then re-install.
First, make sure your system
is running a valid copy of
Windows which is setup with
the correct locality. Next,
make sure you do not have a
limited security version of the
Java Runtime Environment
(JRE) on your system that is
overriding the one installed
with Unified CVP. Finally, if
this message was seen during
install, you must completely
uninstall Unified CVP. Once
that has been completed,
re-install Unified CVP.
Otherwise Unified CVP
needs to be restarted.
The SNMP service failed Restart the SNMP service. At
to start due to the reason a command prompt, type:
noted in the log message.
CVP_HOME\bin\snmp-wrapper.
sh restart
CVP_HOME is the location
where Unified CVP was
installed.
WARN
DUPLICATE_ RETRIEVAL Second or more attempt at
getting the
CommonAdminHandler
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
138
Although operations will If the Operations Console
not be affected, the system operations on this server are
has been misconfigured. failing, then remove the
server from the Operations
Console. Once removed,
reconfigure it from the
Operations Console.
Error Messages
Infrastructure Error Messages
Title
Description
Cause
Resolution
INVALID_MSG
Message processing aborted in
({service}) due to Error:
{error}. Message was:
{message details}.
This may be caused by
connecting together
incompatible versions of
Unified CVP.
If you have recently upgraded
some Unified CVP
components, make sure all
necessary components have
been upgraded. Please send
this error message in it's
entirety to Cisco Technical
Support, or your Cisco
Support Partner.
IO_ERROR
Properties may not have been There was a problem
Make sure all properties files
{loaded/saved} due to an IO accessing a properties file in the CVP_HOME\conf
Exception.
on the disk.
directory are read/write
enabled, and are not currently
opened in an editor.
LICENSING
{license type} license already The system was started
expired on {date}.
with an expired license.
Please contact Cisco or your
Cisco Partner for a new
license.
SNMP_CONNECTION_
FAILURE
Connection to the SNMP
Subagent on port {port(s)}
failed. Retrying next port in
{time} minutes.
In Windows, the SNMP
service can be started from
the services control panel.
Select the Cisco CVP SNMP
Management service, and
click Start.
If this message is seen
consistently, the local
SNMP service may not be
running.
In an AIX environment, the
SNMP service can be started
by typing:
CVP_HOME\bin\snmp-wrapper.
sh start
at a command prompt.
NOTICE
SUBSYSTEM_NOT
_SHUTDOWN
Normal: Subsystem
[{subsystem id}] is in
[{operation state}] state.
During a graceful
shutdown, the externally
connected subsystem noted
in the log message that it
was still running. This is
normal.
RECEIVED_STATE _MSG StateManager: Subsystem
The subsystem noted has
[{subsystem id}] reported
moved to a different state
change to state [{state}] due to for the reason noted.
[{reason}].
If you wish to have all
services associated with the
Call Server that was just
shutdown, as noted in these
log messages, this can be
accomplished via the
Operations Console Control
Center.
These messages are expected
during the operation of
Unified CVP, especially
during start up and shutdown.
If subsystems are frequently
going into
PARTIAL_SERIVE due to
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
139
Error Messages
Infrastructure Error Messages
Title
Description
Cause
Resolution
"dependency state unknown"
this may indicate an
over-used link between the
server and the Unified ICME
VRU PG.
STARTUP
MessageBus Started.
Expected during Call
Server start up.
No action required.
SHUTDOWN
MessageBus Stopped.
Expected during Call
Server shutdown.
No action required.
SERVLET_STARTUP
Servlet Startup Error: {error} This Web Application
No action required.
Server (WAS) initialization
error is benign and
automatically corrected.
START
All Subsystems have been
started.
Expected during Call
Server start up.
No action required.
BAD_LOG_
MESSAGE_XML
The CVPLogMessages.xml file
was corrupt or could not be
found. Default log message and
settings will be used.
CVPLogMessages.xml file
is either missing from
CVP_HOME\conf or has
been incorrectly modified.
This will not affect the
operation of the server, but
default logging will take
the place of any customized
logging that may have been
configured.
Restore
CVPLogMessages.xml file
with a correct version from
the Operations Console.
[AUDIT]
The system has started.
Expected during Call
Server start up.
No action required.
[AUDIT]
The system has completely
shutdown.
Expected during Call
Server shutdown.
No action required.
[VERSION]
{product} version {version
Version information
number}({version type}) built expected during start up.
on {date}.
No action required.
CAH_STARTUP
The {subsystem id} subsystem Expected during start up.
will start the Common Admin
Handler.
No action required.
STATE_MANAGEMENT
Heartbeats from {subsystem
id} resumed. Current state:
{last known state}. Reason:
{reason}. Requesting current
state of {subsystem id}...
[AUDIT]
Log Level now set to [{level}]. The log level of the
No action required.
subsystem which logs this
message has been set to the
level noted.
INFO
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
140
This message will be seen No action required.
when a subsystem that lost
it's connection to the Call
Server has reconnected.
Error Messages
IVR Error Messages
Title
Description
Cause
Resolution
(AIX) SERVER_SETUP
SNMP Agent Server setup on Expected during start up of No action required.
port {port}.
the SNMP service on AIX.
[AUDIT]
Maximum log file size: {size} Informational log messages No action required.
MB. Maximum logging
regarding the current (as of
directory size: {size} MB.
log time) logging file and
directory size, expected at
least at the beginning of
every log file.
[AUDIT]
"{subsystem id}" is at Log
Informational message
Use the Support Tools from
Level {level} with trace mask: regarding the current (as of the Operations Console to
{mask}
log time) log level and
modify log levels.
trace mask of the
subsystem noted.
STATS
{subsystem id/service} {stat
type} statistics: {comma
delineated statistics}
SNMP_CONNECTION
SNMP service has successfully Expected after start up.
connected to the local SNMP
Agent.
No action required.
Description
Resolution
Information message with Use the Operations Console
the current (as of log time) to view statistics on demand.
statistics for the
subsystem/service noted.
IVR Error Messages
Table 3: IVR Error Messages
Title
Cause
ALERT
IVR_VIDEO_
CONNECTION_UP
Socket connection between
CVP (IVR subsystem) and
Radvision IVP (iContact
Application) is established
successfully.
Occurs when socket
Not applicable.
connection between IVR
and iContact is established.
IVR_VIDEO_
CONNECTION_DOWN
Socket connection between
CVP (IVR subsystem) and
Radvision IVP (iContact
Application) is down.
Occurs when socket
connection between IVR
and iContact is broken.
By default, IVR will
reattempt to connect to
iContact every two seconds.
If connection stills fails after
a few attempts, ensure that
IVP is up and running using
the IVP Manager.
In IVP Manager, ensure that
the iCONTACT application
is loaded and activated. If the
iCONTACT application is
okay, ensure both sides are
configured to use the same
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
141
Error Messages
IVR Error Messages
Title
Description
Cause
Resolution
protocol (that is, either both
sides use TCP or both sides
use TLS).
If both sides are using the
same protocol, check IVP
box with netstats to see if port
9981 (normal iCONTACT
port) is opened. If it is
opened, check the
iCONTACT log for errors of
not being able to establish a
listen server socket. In the
worse case scenario, users
may be required to perform a
hard reboot of IVP.
IVR_VIDEO_ALERT
Read the specific error
messages. If it says that the
IVR service timed-out
waiting for a connect
response from iCONTACT,
then the cause of this error
occurs after the TCP
connection is already
established. This
connection request is an
iCONTACT api call after
the TCP connection is
already established. This
should not happen unless
something is wrong in
iCONTACT.
If the error says that the IVR
service timed-out waiting for
a connect request response
from iCONTACT, then:
Occurs if someone
manually tampers with or
moves the file. This error
may also occur if file is not
in Java properties file
format. It is unlikely that
anyone will make changes
to this file unless they are
providing writing to the
Call Server VPD (Video
Play Data) localization
API.
Ensure the custom grammar
properties file
(customGrammar.properties)
is located under
<CVP_HOME>\conf\customgrammar.
IVR_CANNOT_
This error occurs if someone is If a custom localizer class
INSTANTIATE_CUSTOM_ developing a custom VPD
is being developed, the
LOCALIZER_CLASS
(Video Play Media)
possible causes are:
localization class.
1. Ensure the name of the
custom localizer class
in customgrammar
IVR_CUSTOM_
LOCALIZER_FILE_
MISSING_OR_
UNREADABLE
Errors are encountered that
prevent video call processing.
Action must be taken
immediately.
This error may occur if
someone is developing a
custom Video Play Media
(VPD) localization class.
Custom Grammar properties
file may have been moved or
removed, corrupted or not
defined correctly.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
142
1. Check the IVP box. If
necessary, restart the
IVP box using the IVP
Manager.
2. If the error persists,
reboot the IVP
machine.
Ensure that any changes to
the file are compliant with
Java properties file format.
Restart CallServer after this
issue is fixed.
Error Messages
IVR Error Messages
Title
Description
Cause
Resolution
1. The custom localizer
class name may not
match the actual class
name.
2. The custom localizer
class may not be in
the class path of the
Call server.
3. The custom localizer
class may not be
implementing the
Localizer interface.
properties matches the
actual class name.
2. Place the custom
localizer class or jar file
in the classpath of the
Call Server as specified
in the VPD localization
located in the
Configuration and
Administration Guide
for Cisco Unified
Customer Voice Portal
guide.
3. Ensure that the class
implements the
Localizer interface.
Restart Call Server after
any of the above issues
are fixed.
IVR_LOCALIZER_
The VPD custom localizer
INITIALIZATION_FAILED class returns 'false' from it's
init( ) method.
This error usually only
occurs if someone is
developing a custom VPD
(Video Play Media)
localization class. It is
caused when the custom
localizer class returns a
boolean 'false" from it's
init( ) method.
IVR_API_ALERT
Investigate the custom
Fix problem and restart Call
VPD localizer class to
Server if it is necessary to
determine why this error is reload the class.
occurring. Potential
problems could be that
some needed resource
cannot be found or there is
an internal error.
These errors are only logged
by VPD localization classes.
They are provided as a hook
for custom localization classes
to log errors that require
immediate attention.
1. Check the custom localizer
class code to see why it's
returning 'false' from it's init(
) method. 2. Fix problem and
restart CallServer to reload
the class.
CRITICAL
IVR_API_CRITICAL
These errors are only logged
by VPD localization classes.
They are provided as a hook
for custom localization classes
to log errors that may not
require immediate attention but
are critical to it's function.
Investigate the custom
Fix problem and restart Call
VPD localizer class to
Server if it is necessary to
determine why this error is reload the class.
occurring. Potential
problems could be that
some needed resource
cannot be found or there is
an internal error.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
143
Error Messages
IVR Error Messages
Title
Description
Cause
Resolution
ERROR
IVR_VIDEO_ERROR
Errors were encountered during There are many possible
call processing.
call processing errors. The
log message has the details
of the specific error.
Examples include media
not found, receiving error
status codes from
iCONTACT, no response
from iCONTACT and so
on.
Resolution depends on the
specific error logged. For
example, if the error is media
not found, then ensure that
the movies are located where
they should be. If the error is
caused by an error status code
from iCONTACT, then
investigate iCONTACT logs
to see possible causes.
IVR_VIDEO_CALL_
TIMED_OUT
Error encountered when IVR A request to ICM from the Ensure that ICM PIM is up
times out waiting for
ICM subsystem has taken and running and ICM/IVR
instructions from ICM.
longer than expected.
subsystems are in full service.
Investigate possible network
issues between CVP and ICM
that may have caused the
delay.
IVR_VIDEOSERVLET_
ERROR
Error logged by the video
servlet component.
Most likely cause is that a
request was received by the
video servlet for action on
a call session that does not
exist.
IVR_VIDEOSERVLET_
INVALID_REQUEST
Error logged by the video
servlet component when an
invalid request is made.
The video servlet received Not applicable.
a request that is not valid.
For instance, this error is
logged if a request is made
to fast forward or rewind a
live stream video which is
not possible.
Not applicable. The call
session may have ended
before the request was
received.
IVR_NO_AUDIO_LIST_ The VPD custom localizer did
OR_TEXT_FROM_LOCALIZE not return localized text nor an
audio file list to the Call Server
for a VPD.
No audio list or text output Fix problem and restart Call
received from localizer for Server if it is necessary to
a particular data type for a reload the class.
specific data.
IVR_API_ERROR
Investigate the custom
Fix problem and restart Call
VPD localizer class.
Server if it is necessary to
Potential problems include reload the class.
that the custom localizer
class could not return
localized text for a VPD or
could not generate the list
of audio files for a VPD.
These errors are only logged
by VPD localization classes.
They are provided as a hook
for custom localization classes
to log errors during call
processing.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
144
Error Messages
VMS Error Messages
VMS Error Messages
Table 4: VMS Error Messages
Title
Description
Cause
Resolution
EMERGENCY
VMS_SUBSYSTEM_INIT_ Unable to instantiate the media The internal media library Restart the Call Server
MEDIA_LIBRARY_INVALID library.
was not initialized in
service. Check that the
memory.
cvp-vms-backend.jar is not
corrupted.
VMS_SUBSYSTEM_INIT_ Unable to create the search
CREATE_INDEX_FAILURE index.
The search index was
unable to be created.
VMS_SUBSYSTEM_
The CVP_HOME environment The subsystem could not
PROPERTY_CVP_HOME_ variable value is missing or
retrieve the value of the
FAILURE
invalid.
CVP_HOME environment
variable.
VMS_SUBSYSTEM_
PROPERTY_FAILURE
If VMS does not recover
gracefully from this issue, go
to the VMS index directory
on the file system and delete
all files within this directory.
Determine if the environment
variable CVP_HOME exists
and its value is equivalent to
the root CVP directory.
Restart the Call Server
service.
The defined VMS property is The listed property does not Verify the property exists
missing or invalid.
exist, has no value, or has within vms.properties and is
an invalid value.
associated with a valid value.
ALERT
VMS_SUBSYSTEM_INIT_ The schema file property is
SCHEMA_FILE_FAILURE missing or invalid.
The VMS schema file is
missing or invalid.
Check in the VMS schema
directory for the presence of
the XML schema file.
Determine if the contents of
the schema file are valid
XML.
VMS_SUBSYSTEM_INIT_ Unable to start the media
START_POLLER_FAILURE poller.
The media poller was
unable to be started.
Check that the user-selected
VMS movies directory is
present on the file system.
Restart the Call Server
service.
VMS_SUBSYSTEM_INIT_ Unable to create the media
SETUP_POLLER_FAILURE poller.
The media poller was
Check that the user-selected
unable to be initialized in VMS movies directory is
memory.
present on the file system.
Restart the Call Server
service.
VMS_SUBSYSTEM_INIT_ Unable to create the media
The media poller could not Check that the user-selected
SETUP_POLLER_FACTORY_ poller because metadata factory be initialized.
VMS movies directory is
FAILURE
creation failed.
present on the file system.
Check that the VMS XML
schema exists on the file
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
145
Error Messages
VMS Error Messages
Title
Description
Cause
Resolution
system and is a valid XML
schema. Restart the Call
Server service.
VMS_SUBSYSTEM_INIT_ Unable to create the login
SETUP_LOGIN_MANAGER_ manager.
FAILURE
The login manager could
not be initialized.
Check that the
cvp-vms-backend.jar is
present and not corrupted.
Make sure that port 2100 is
not being used by another
application.
VMS_SUBSYSTEM_INIT_ An unexpected error occurred The subsystem failed to
Restart the Call Server
UNEXPECTED_
during VMS subsystem
start due to an unexpected service.
EXCEPTION
initialization.
error.
VMS_SUBSYSTEM_INIT_ Unable to start the media poller The media poller manager Check for the existence and
POLLER_MANAGER_
manager.
failed to initialize.
validity of the VMS XML
FAILURE
schema on the file system.
Check in the logs that the
media poller was started
successfully. Check that the
user-selected VMS movies
directory is present. Restart
the Call Server service.
VMS_SUBSYSTEM_
UNSUPPORTED
VMS version is currently is
unsupported.
An unsupported device
Verify that all CVP
attempted to interface with components that are
VMS subsystem.
communicating with VMS
are of the supported version.
VMS_MEDIA_POLLER_
MANAGER_RESTART_
FAILURE
Unable to restart the media
poller after X number of
attempts.
The media poller failed to Restart the Call Server
stop appropriately.
service.
VMS_MEDIA_POLLER_
MANAGER_DEAD
The media poller is dead.
The media poller manager Check the logs if the media
detected that the media
poller manager restarted the
poller is dead.
media poller successfully
after this message. Restart the
Call Server service if needed.
VMS_SUBSYSTEM_
PROPERTY_MEDIA_
HOME_FAILURE
The media home directory is
invalid.
The media home property Verify that
could not be retrieved.
VMS.darwin.home,
VMS.darwin.app, and
VMS.darwin.config are
present with valid values in
the vms.properties file.
Restart the Call Server
service.
VMS_SUBSYSTEM_
PROPERTY_DARWIN_
HOME_FAILURE
Unable to retrieve the Darwin The Darwin media
media directory location.
directory property could
not be retrieved.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
146
Verify that
VMS.darwin.home,
VMS.darwin.app, and
VMS.darwin.config are
present with valid values in
the vms.properties file.
Error Messages
VMS Error Messages
Title
Description
Cause
Resolution
Verify that the CVP Darwin
Streaming Server directory is
present. Verify that the
Darwin Streaming Server
streamingserver.xml contains
an XML element pointing to
the VMS movies directory.
Restart the Call Server
service.
VMS_SUBSYSTEM_
LOCAL_IP_FAILURE
The local IP address could not The local IP address could Verify that the machine has
be determined.
not be determined.
a valid network interface with
an assigned IP address.
Re-add the VMS device via
OAMP. Restart the Call
Server service.
VMS_SUBSYSTEM_
LOCAL_IP_INVALID
The local <IP address>is
invalid.
The local IP address is not Verify that the OAMP
valid.
configured IP address is a
valid IP address or that the
local IP address on local
network interfaces are valid.
Restart the Call Server
service.
ERROR
VMS_SUBSYSTEM_INIT_ Unable to start the Darwin
START_DARWIN_FAILURE Streaming Server.
Not applicable.
Not applicable.
VMS_SUBSYSTEM_INIT_ Darwin Streaming Server has Not applicable.
DARWIN_PARTIAL_
started, but is in partial service.
SERVICE
Not applicable.
VMS_SUBSYSTEM_
RELOAD_MEDIA_
POLLER_FAILURE
The media poller could not be The media poller failed to
reloaded.
reload during a VMS
administration reload
process.
Verify the VMS movies
directory is present. Verify
the VMS XML schema is
present and valid. Restart the
Call Server service.
VMS_SUBSYSTEM_
RELOAD_MEDIA_
POLLER_MANAGER_
FAILURE
The media poller manager
could not be reloaded.
Verify the VMS movies
directory is present. Verify
the VMS XML schema is
present and valid. Restart the
Call Server service.
VMS_MEDIA_POLLER_
PROCESS_FOLDER_
FAILURE
Failed to create folder media: Not applicable.
<folder name>.
VMS_MEDIA_POLLER_
PROCESS_MEDIA_
WITH_XML_READ_
FAILURE
Failed to read media: <xml file The associated XML
Verify the XML file is
name>.
metadata file could not be present on the file system and
read from the file system. validates to the schema.
Verify the VMS movies
directory is present. Verify
The media poller manager
failed to reload during a
VMS administration reload
process.
Not applicable.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
147
Error Messages
VMS Error Messages
Title
Description
Cause
Resolution
the VMS XML schema file
is present and valid. Restart
the Call Server service.
VMS_MEDIA_POLLER_ Failed to create generic media. Not applicable.
PROCESS_MEDIA_ONLY_
FAILURE
Not applicable.
VMS_MEDIA_POLLER_
PROCESS_XML_ONLY_
READ_FAILURE
Failed to read media.
Not applicable.
Not applicable.
VMS_SUBSYSTEM_
STREAMING_SERVER_
DOWN
The Streaming ServerState is Not applicable.
down.
Not applicable.
VMS_ACTION_
EXCEPTION_CAUGHT
VMS caught an unhandled
exception: <exception stack
trace>.
Verify there are no errors in
the Call Server Tomcat logs.
Check for exception details
in the CVP VMS logs.
Restart the Call Server
service.
An unhandled exception
was caught via a web
action.
VMS_ERROR_INVALID_ Got an invalid value for media The media home property Verify that VMS.darwin.app,
MEDIA_HOME
home from the subsystem
is invalid or missing.
VMS.darwin.config, and
VMS.darwin.home are
present in vms.properties
with appropriate values.
Verify CVP Darwin
Streaming Server directory is
present. Verify DSS
streamingserver.xml contains
valid reference to the
user-selected movies
directory. Restart the Call
Server service.
VMS_ERROR_COULD_
NOT_PARSE_MEDIA_
HOME
Could not parse the value for The Darwin Streaming
Check for the Darwin
media home.
Server config file is invalid. Streaming Server
streamingserver.xml
configuration file presence.
Verify the XML config file
is properly formatted XML.
Verify DSS
streamingserver.xml contains
valid reference to the
user-selected movies
directory. Restart the Call
Server service.
VMS_ERROR_COULD_ Could not parse the value for The Darwin Streaming
Check for the Darwin
NOT_PARSE_RTSP_PORT the rtsp port.
Server config file is invalid. Streaming Server
streamingserver.xml
configuration file presence.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
148
Error Messages
Reporting Server Error Messages
Title
Description
Cause
Resolution
Verify the XML config file
is properly formatted XML.
Verify DSS
streamingserver.xml contains
valid reference to the RTSP
port. Restart the Call Server
service.
VMS_ERROR_NO_
RESOURCES_PARAM
Could not find entry in
web.xml for
The resources parameter in Verify the web.xml is present
the Tomcat web
in the VMS web deployment
deployment descriptor for in the WEB-INF directory.
VMS is missing or invalid.
VMS_MISC_EXCEPTION An exception was caught:
An exception was caught
method - <method name> type by VMS.
- <type>, message - <message>
Turn on
TRACE_HANDLED_EXCEPTION
tracing and troubleshoot the
exception. Restart the Call
Server service.
Reporting Server Error Messages
Table 5: Reporting Server Error Messages
Emergency
Description
Cause
Resolution
ALERT
Alert
Unable to start subsystem: {ss} The cause will be noted in The configuration must be
the log message.
fixed and call server must be
restarted.
Alert
Reporting Server is not
licensed
The license file is wrong
Check if the license file
presences are correct.
Connect to Cisco.
Alert
License file is invalid or does The license file is wrong
not exist
Check if the license file
presences are correct.
Connect to Cisco.
Alert
Alert
Alert
Alert
Alert
Alert
Alert
Alert
Alert
Alert
Alert
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
149
Error Messages
Reporting Server Error Messages
Emergency
Description
Cause
Resolution
Alert
Alert
Alert
Alert
ERRORS
Error
Unable to stop: {message}
Error
Message adapter admin object Message adapter manager Check the infrastructures
cannot be found
can't find the message
adapter admin object by SS
name
Error
Unable to get Subsystem state: Any problem while trying Check the infrastructures
{error_msg}
to get SS state
Error
Can't fetch Subsystem Name
from {msg}
Problem while trying to
Check the infrastructures
fetch SS name from tabular
message
Error
Can't fetch Subsystem Type
from {msg}
Problem while trying to
Check the infrastructures
fetch SS type from tabular
message
Error
Can't fetch Message Bus Name Problem while trying to
Check the infrastructures
from {msg}
fetch Message Bus Name
from tabular message
Error
No Call Descriptor found for Can't find the call
{msg}
descriptor for message
N/A
Error
No message handler for {msg} Can't find the handler for
this tabular message type
N/A
Error
The
The Thread has been
ReportingMessageProcessor: interrupted.
{this} has been interrupted.
Restart the reporting server
service.
Error
Error creating the client socket. Error creating the client
socket with
ReportingServer.
Check if ReportingServer
service runs and do netstat,
to see if the server socket
opened for port 5503.
Error
Error sending a Start message Error sending a Start
to Reporting Server.
message to Reporting
Server.
Check if ReportingServer
service runs and do netstat,
to see if the server socket
opened for port 5503.
Error
Error sending an End message Error sending an End
to Reporting Server.
message to Reporting
Server.
Check if ReportingServer
service runs and do netstat,
to see if the server socket
opened for port 5503.
Error
Error closing the client socket. Error closing the client
socket.
N/A
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
150
The cause will be noted in N/A
the log message.
Error Messages
ICM Error Messages
Emergency
Description
Cause
Resolution
Error
Error
Error
Error
Error
Error
Error
Error
ICM Error Messages
Table 6: ICM Error Messages
Title
Description
Comment
Resolution
Critical
LOGMSG_ICM_SS_MSGBUS_SHUTDOWN Message Bus
Connection is
Down.
ICM subsystem received a Restart the CVP Call Server.
plug-in down message
from the message bus.
Message bus may have
shutdown due to an internal
error.
LOGMSG_ICM_SS_PIM_SHUTDOWN Lost socket
connection to
VRU PIM.
Transitioning to
partial service.
ICM subsystem lost
Check VRU PIM logs for
connection with the VRU reason of disconnect.
PIM. PIM may have been
shutdown on purpose or a
failure occurred in PIM.
Secondary PIM should
re-connect.
Alert
LOGMSG_ICM_SS_STATE
VRU PIM
connection is
down.
Transitioned to
partial service.
ICM subsystem lost
Check VRU PIM logs for
connection with the VRU reason of disconnect.
PIM. PIM may have been
shutdown on purpose or a
failure occurred in PIM.
Secondary PIM should
re-connect.
LOGMSG_ICM_SS_STATE
Waiting for
VRU PIM
connection.
ICM subsystem ready to
accept connection from
VRU PIM.
LOGMSG_ICM_SS_STATE
ICMSubsystem An error was encountered Check ICM SS configuration
init() failed
in initializing the ICM
and restart Call Server
subsystem. Most likely due
to invalid configuration.
Make sure VRU PIM is
configured to connect to call
server on the same port.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
151
Error Messages
ICM Error Messages
Title
Description
Comment
Resolution
LOGMSG_ICM_SS_STATE
Transitioning to A critical thread
No manual intervention
partial service. deadlocked or terminated required. ICM SS will
Reason =
unexpectedly.
transition to partial service and
restart the critical thread.
LOGMSG_ICM_SS_STATE
Transitioned to Graceful shutdown
partial service requested.
due to graceful
shutdown
request.
Verify that graceful shutdown
was intentional.
LOGMSG_ICM_SS_STATE
Transitioned to Graceful shutdown
stopping due to requested.
graceful
shutdown
request.
Verify that graceful shutdown
was intentional.
LOGMSG_ICM_SS_STATE
Transitioned to Graceful shutdown
stopped due to requested.
graceful
shutdown
request
Verify that graceful shutdown
was intentional.
LOGMSG_ICM_SS_STATE
Transitioned to Forced shutdown
stopping due to requested.
forced shutdown
request.
Verify that forced shutdown
was intentional.
LOGMSG_ICM_SS_STATE
Transitioned to Forced shutdown
stopped due to requested.
forced shutdown
request
Verify that forced shutdown
was intentional.
Error
LOGMSG_ICM_SS_UNSUPPORTED_MSG %s Received
Unsupported
Message,
Type=%s
Unknown/unhandled
message received from
ICM.
CVP should be connected to
a supported ICM version.
Check the CVP BOM for
supported versions.
LOGMSG_ICM_SS_HEARTBEAT_FAILURE Message Bus
Critical thread Message
Listener missed Bus Listener either dead
a heartbeat.
locked or was terminated
either unexpectedly or due
to a normal shutdown
request.
No manual intervention
required. ICM SS will
transition to partial service and
restart the critical thread.
LOGMSG_ICM_SS_HEARTBEAT_FAILURE Message Bus
Publisher
missed a
heartbeat.
No manual intervention
required. ICM SS will
transition to partial service and
restart the critical thread.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
152
Critical thread Message
Bus Publisher either dead
locked or was terminated
either unexpectedly or due
to a normal shutdown
request.
Error Messages
ICM Error Messages
Title
Description
Comment
Resolution
LOGMSG_ICM_SS_HEARTBEAT_FAILURE PIM Listener
missed a
heartbeat.
Critical thread PIM
Listener either dead locked
or was terminated either
unexpectedly or due to a
normal shutdown request.
No manual intervention
required. ICM SS will
transition to partial service and
restart the critical thread.
LOGMSG_ICM_SS_HEARTBEAT_FAILURE PIM Publisher
missed a
heartbeat.
Critical thread PIM
Publisher either dead
locked or was terminated
either unexpectedly or due
to a normal shutdown
request.
No manual intervention
required. ICM SS will
transition to partial service and
restart the critical thread.
LOGMSG_ICM_SS_HEARTBEAT_FAILURE Admin Monitor Critical thread Message
missed a
Bus Listener either dead
heartbeat.
locked or was terminated
either unexpectedly or due
to a normal shutdown
request.
No manual intervention
required. ICM SS will
transition to partial service and
restart the critical thread.
LOGMSG_ICM_SS_HEARTBEAT_FAILURE Garbage
Collector
missed a
heartbeat
Critical thread Garbage
Collector either dead
locked or was terminated
either unexpectedly or due
to a normal shutdown
request.
No manual intervention
required. ICM SS will
transition to partial service and
restart the critical thread.
LOGMSG_ICM_SS_EXCEPTION
An unexpected exception Forward logs to TAC/DE for
occurred in the subsystem. analysis.
Stack trace
LOGMSG_ICM_SS_MSGLAYER_ERROR %s Unable to
publish, receive
or process
message,
Message=%s
There was an error
publishing a message to the
ICM PIM plug-in via the
messaging layer
In most cases, these errors are
normal since a connection
may be terminating while
messages are still being
published for example. Restart
call server. Forward logs to
TAC/DE if errors are
occurring while all
connections are up and calls
should be processed normally.
ICM_CALL_ERROR
ICM subsystem has run out
of trunks. In most cases,
this is due ICM SS not
getting the proper
CALL_STATE_EVENT
from SIP/IVR/VXML
subsystems to release the
trunk it occupies when
sending a NEW_CALL or
REQUEST_INSTRUCTION
message to ICM
This usually requires a
DEBUG trace with Call trace
level turned on in ICM
subsystem to debug this issue
to understand the call event
flow. In a non-productions
system, use diagnostic servlet
to dump ICM calls to
determine which calls are still
active in the system. Capture
a screen shot of this to track
some of the call GUIDs in the
*** NO
TRUNKS ARE
AVAILABLE
***
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
153
Error Messages
SIP Error Messages
Title
Description
Comment
Resolution
log. Depending on the call
flow, also turn on Call Debug
tracing for SIP/IVR/VXML
subsystems to track the same
call end to end to determine
what’s happening in terms of
event flow in the other
subsystems for that call.
ICM_CALL_ERROR
Removing call:
<callid> due to
exception.
Exception:
<exception
message>
ICM subsystem ran out of This is an internal error.
trunks or ICM subsystem Forward logs to TAC/DE for
threw an exception
analysis.
processing a call state
event or run script result
for a call and needs to
remove the call from the
call table.
ICM_CALL_ERROR
Current state of
ICM service is
<current state>.
Rejecting new
ICM label
lookup request
<msg dump>
ICM subsystem is not In
Service and will reject the
ICM label lookup request
or a New Call request until
it’s back In Service.
This message will most likely
appear due to a timing issue
where ICM SS is no longer
connected to a VRU PIM and
while it transitions into partial
service and the state bubbles
up to the dependent services
such as SIP/IVR/VXML,
some new calls and label
lookup requests from VXML
service may still get through
and ICM SS will reject these
requests. Make sure VRU PIM
is configured to connect to call
server on the same port. Check
VRU PIM logs for reason of
disconnect.
SIP Error Messages
Table 7: SIP Error Messages
Title
Description
Comment
SIP_INTERNAL_ERROR
Failed to verify outbound
proxy hostname with DNS
Server: + {PROXY NAME}
Could not resolve
Change the hostname on CVP
Outbound Proxy with DNS. or else configure your DNS
server to resolve the
hostname.
SIP_INTERNAL_ERROR
No local static routes
configured.
Non-proxy mode is
Add local routes. OAMP
selected but no local routes should not allow this to
are configured on CVP.
happen anyway.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
154
Resolution
Error Messages
SIP Error Messages
Title
Description
Comment
Resolution
SIP_INTERNAL_ERROR
B2BUA is not configured with B2BUA cannot route to the Add a route to the Error DN
a route for making CVP
Error DN.
in OAMP.
ERROR [+{ERROR DN
HERE}+"] playback calls.
Please add this route."
SIP_INTERNAL_ERROR
B2BUA is not configured with B2BUA cannot route to the Add a route to the Ringtone
a route for making CVP
Ringtone DN.
DN in OAMP.
RINGTONE [+{RINGTONE
DN HERE}+"] playback calls.
Please add this route."
SIP_INTERNAL_ERROR
B2BUA is not configured with B2BUA cannot route to the Add a route for this particular
a route for making calls to
returned label.
label.
[+{ICM LABEL HERE}+"].
Please add this route."
SIP_CALL_ERROR
{THE CALLGUID HERE} + No route configured for
": Destination URL is null,
label.
cannot make the transfer."
Add a route for this particular
label.
SIP_CALL_ERROR
{THE CALLGUID HERE} +
": Aborting XFER. The call to
the RINGTONE dial peer is
exceeding 20 msecs. Check
ringtone gateway(s), OR the
thread pool for B2BUA is
getting backlogged. Check
STATS_RT_RUNNING_POOL_THREADS
or increase thread pool size."
Confirm the routing in CUPS
Proxy server is correct for
Ringtone calls.
SIP_CALL_ERROR
{THE CALLGUID HERE} + No route for REFER label Add a route for this particular
": Destination URL is null,
is found.
label.
cannot make the REFER."
SIP_CALL_ERROR
{THE CALLGUID HERE} + REFER operation was not
": Refer failed with "+sipcode successfully performed.
+"-"+
DsSipResponseCode.getReasonPhrase(sipcode)+".
May be a problem with DN
Routes or Gateway
Dail-Peers."
Check that the endpoint for
the transfer is configured on
the gateway or call manager.
Perhaps an incoming or
outgoing dial-peer is needed.
SIP_CALL_ERROR
{THE CALLGUID HERE} + Dialog Failure on the ICM
" - DIALOGUE_FAILURE
script was encountered and
from ICM Router sends 500 B2BUA is ending the call.
rejection to call."
The ICM script returned a
dialog failure. If it is a
timeout, confirm that the call
is able to be sent to the
VXML gateway in timely
manner. Confirm that the
Microapp is fetching
correctly with the IVR
service.
Thread to perform
Ringtone during the
transfer has waited 20
seconds and timed out.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
155
Error Messages
SIP Error Messages
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
156
Appendix B
Event and Cause Codes
Event codes and cause codes provide information on the status of a particular operation within
the Unified CVP environment. H.323 and SIP standard cause codes reflect the nature of the
associated issue. For each of the issues, an ITU-T Q.850 release cause code is also assigned and
maps the new standard categories with the Q.850 release cause code and description that is
assigned to each category.
IVR Service Error Codes
Table 8: IVR Service Error Codes
Error Code Name
Error Code Error Code Description
Value
ERROR_NONE
0
No error.
ERROR_HANGUP
1
N/A
ERROR_NETWORK
2
Failure of an IP network connection.
ERROR_SYSTEM
3
Unexpected failure of an Unified CVP component.
ERROR_SCRIPT_TYPE
4
N/A
5
Micro-Application name passed from the Unified ICME
to the Call Server did not exist on the Call Server.
6
VRU Script Name data passed from the Unified ICME
to the Call Server does not contain the expected
components (Micro-Application name, media file name,
media file type, uniqueness value).
7
Data passed from the Unified ICME to the Call Server
is not consistent with what the micro-application
requires for processing.
8
An ECC variable was set to a value the Application
Server did not recognize. ECC variable definitions must
be the same in Unified ICME and Unified CVP.
ERROR_UNKNOWN_UAPP
ERROR_SCRIPT_NAME
ERROR_CONFIG_PARAM
ERROR_MISCONFIG_ECC
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
157
Event and Cause Codes
IVR Service Error Codes
Error Code Name
ERROR_MEDIA_FILE
Error Code Error Code Description
Value
9
Media file name passed from the Unified ICME to the
Call Server did not exist on the Media Server.
10
N/A
11
The VoiceXML Interpreter did not recognize a
VoiceXML format passed from the Call Server.
12
The VoiceXML Interpreter did not recognize a
VoiceXML element passed from the Call Server.
13
The variable data passed was not valid for the script
type being processed.
ERROR_NO_VAR_DATA
14
N/A
ERROR_FORMAT
15
N/A
16
Caller was unsuccessful in entering digits during each
of the tries allowed by the Micro-Application.
17
Caller did not enter digits in response to the prompt for
each of the tries allowed by the Micro-Application.
18
Unable to perform video-related request due to resource
limitations.
19
Unable to start the recording or some type of recording
failure.
20
N/A
21
Caller did not enter digits in response to the prompt in
the time allowed by the Micro-Application.
ERROR_CALLED_HANGUP
22
N/A
ERROR_NO_ANSWER
23
No answer
ERROR_BUSY
24
Busy
ERROR_GEN_TRANSFER
25
General transfer error
ERROR_INVALID_EXTN
26
Invalid extension
ERROR_HANGUP_FORCED
27
Called party hung up
ERROR_AFTER_TRANS_ESTAB
28
Error after transfer established
30
The VoiceXML Interpreter did not recognize the locale
passed from the Call Server.
31
TTS service was unable to perform a request due to
resource limitations.
32
TTS service was unable to perform a request due to
resource limitations. However, it may also indicate that
an unsupported locale was specified.
33
N/A
40
Returned if the Call Server is unavailable (shutdown,
network connection disabled, and so forth)
ERROR_SEMANTIC
ERROR_VXML_FORMAT
ERROR_VXML_ELEMENT
ERROR_VARIABLE_DATA
ERROR_MAX_INVALID
ERROR_MAX_NO_ENTRY
MEDIA_RESOURCE_VIDEO
RECORDING_FAILED
ERROR_DATA_RANGE
ERROR_TIMED_OUT
ERROR_UNSUPPORTED_LANGUAGE
ERROR_MEDIA_RESOURCE_ASR
ERROR_MEDIA_RESOURCE_TTS
ERROR_GENERAL_ASR_TTS
ERROR_CVP_SYSTEM_UNAVAILABLE
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
158
Event and Cause Codes
Unified ICME Error Codes
Error Code Name
Error Code Error Code Description
Value
ERROR_CVP_APP_ERROR
ERROR_CVP_APP_HANGUP
ERROR_CVP_APP_SUSPENDED
41
Returned if some VXML Server-specific error occurs
(For example, java exception).
42
Returned if a hangup element is used, instead of
subdialog return element, to send a Cisco application.
43
Returned if VXML Server application is suspended.
44
Returned when an emergency error occurs (for example,
an application is called that has not been loaded in the
VXML Server application).
45
Returned when the VXML Server encounters a bad
fetch situation. This code is returned when either a wav
file or an external grammar file is not found.
ERROR_CVP_NO_SESSION_ERROR
ERROR_CVP_BAD_FETCH
Unified ICME Error Codes
Table 9: Unified ICME Error Codes
Error Code Name
Error Code Value
Error Code Description
NO_ERROR
0
No error occurred.
INVALID_VERSION
1
The version number requested by the
Peripheral Gateway is not supported by
the VRU.
SESSION_ALREADY_ACTIVE
2
The VRU already has an active
communication session with a
Peripheral Gateway.
VRU_OFFLINE
3
The VRU is unavailable.
SESSION_NOT_ACTIVE
4
No session is currently active.
INVALID_DIALED_NUMBER
5
The dialed number specified is not
known to the Unified ICME.
EVENTS_NOT_SUPPORTED
6
The VRU does not support the Event
Data Feed.
POLLING_NOT_SUPPORTED
7
(Obsolete)
ROUTING_NOT_SUPPORTED
8
The VRU does not support the Call
Routing Interface.
TIME_SYNCH_NOT_SUPPORTED
9
The VRU does not support the Time
Synchronization Interface.
TIMEOUT
10
A time-out has occurred.
PG_OFFLINE
11
The Peripheral Gateway is off-line.
REQUEST_REFUSED
12
The request was refused due to a
temporary condition.
ROUTING_NOT_AVAILABLE
13
The Unified ICME routing service is
not available.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
159
Event and Cause Codes
Unified ICME Error Codes
Error Code Name
Error Code Value
Error Code Description
ROUTE_NOT_ACCEPTED
14
The VRU did not accept the supplied
route.
UNSPECIFIED_FAILURE
15
An unspecified error occurred.
INVALID_INVOKEID
16
An invalid InvokeID has been specified
in a result or response message.
SERVICE_CTRL_NOT_SUPPORTED
17
A request was made to initialize the
Service Control Interface to a VRU that
does not support the interface.
NO_SCRIPT
18
An ICM script has not been scheduled
for the call type which the VRU is
seeking instruction.
CALL_VARIABLE1
19
The VRU could not process a request
due to invalid value in CallVariable1.
CALL_VARIABLE2
20
The VRU could not process a request
due to invalid value in CallVariable2.
CALL_VARIABLE3
21
The VRU could not process a request
due to invalid value in CallVariable3.
CALL_VARIABLE4
22
The VRU could not process a request
due to invalid value in CallVariable4.
CALL_VARIABLE5
23
The VRU could not process a request
due to invalid value in CallVariable5.
CALL_VARIABLE6
24
The VRU could not process a request
due to invalid value in CallVariable6.
CALL_VARIABLE7
25
The VRU could not process a request
due to invalid value in CallVariable7.
CALL_VARIABLE8
26
The VRU could not process a request
due to invalid value in CallVariable8.
CALL_VARIABLE9
27
The VRU could not process a request
due to invalid value in CallVariable9.
CALL_VARIABLE10
28
The VRU could not process a request
due to invalid value in CallVariable10.
INVALID_SCRIPT
29
The Script ID specified in the
RUN_SCRIPT_REQ message was
invalid on the VRU.
INVALID_CALLID
30
The CallID specified in a Service
Control request message not valid on
the VRU.
DUPLICATE_DIALOGUEID
31
The dialogue identifier specified in
either a NEW_CALL or
REQUEST_INSTRUCTION message
was not unique.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
160
Event and Cause Codes
SIP Codes
Error Code Name
Error Code Value
Error Code Description
INVALID_MESSAGE
32
This error code may be sent in response
to any message received with missing
required floating fields.
INVALID_DIALOGUEID
33
The DialogueID specified in a request
message is no longer valid.
OPERATION_CANCELLED
34
The specified operation was
successfully cancelled by a cancel
request.
OPERATION_NOT_CANCELLED
35
The specified operation could not be
cancelled by a cancel request.
SIP Codes
SIP Codes:
SIP codes are available on the Cisco web site. To view the codes, go to the following link:
http://www.cisco.com/en/US/docs/ios/12_2t/12_2t11/feature/guide/ftmap.html ( http://
www.cisco.com/en/US/docs/ios/12_2t/12_2t11/feature/guide/ftmap.html)
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
161
Event and Cause Codes
SIP Codes
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
162
Appendix C
Troubleshooting Radvision Errors
This appendix lists issues that would require Radvision support personnel to examine prior to
involving Cisco TAC.
Potential Radvision issues are as follows:
• User Interface issues with IVP componets (for example, IVP Manager, MCU Admin GUI)
• Alarms and Events in IVP Manager for various Radvision components
• High CPU usage on IVP components
• Recording anomalies or unexpected behaviors in recorded files
• Error-notification messages in CVP logs
<?xml version="1.0" encoding="UTF-8" ?>
<icontact-xml-api version="2.0">
<notification connection-id="iCONTACT-Connection-2">
<error-notification session-id="000E7FF13F0D07903040196543882926"
error-code="GoFailed">
<error-value/>
</error-notification>
</notification>
</icontact-xml-api>
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
163
Troubleshooting Radvision Errors
– After an error-notification is sent by iCONTACT to CVP, iCONTACT will terminate the
call.
• Error-response messages seen in CVP logs with certain error codes If the error-code is one
of internal-timeout, general-error, then this is most likely a Radvision internal
issue. For example:
<?xml version="1.0" encoding="UTF-8" ?>
<icontact-xml-api version="2.0">
<response connection-id="iCONTACT-Connection-2" request-id="5"
is-final="true">
<error-response>
<error-code code="internal-timeout"/>
<error-value/>
</error-response>
</response>
</icontact-xml-api>
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
164
Appendix D
Accessing and Interpreting SNMP Data
The Unified CVP SNMP agent allows customers and partners to integrate with their existing
SNMP Network Management System (NMS) to provide instantaneous feedback on the health
of their Unified CVP system. The Call Server, VXML Server, and Reporting Server can be
configured to send SNMP traps and statistics to any standard SNMP Network Management
System. You can configure a link to the administration web page for an SNMP monitoring tool
and then access it by selecting SNMP Monitor from the Tools menu of the Operations Console.
To add a URL link to a tool:
1. Choose Tools > Configure from the main Operations Console menu bar.
The Configure Tools window opens, listing the current URL configured for each tool
listed on the Tools menu.
2. In the Enter New URL text box for the tool you want to configure, enter the URL for
each tool to launch.
The web page indicated by this URL is launched when you select the tool from the Tools
menu.
3. Click Save to save the URLs.
Once you have configured the administration web page for your NMS, you can launch the
SNMP monitoring tool from the Operations Console. The procedure below describes how to
launch the tool.
To launch an SNMP monitoring tool:
1. Choose Tools > SNMP Monitor from the main Operations Console menu.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
165
Accessing and Interpreting SNMP Data
The table below describes the available severity levels and their descriptions for Unified CVP
events. The severity level for an event can be set by editing the CVPLogMessages.xml file on
the server that generated that particular event.
Severity
Name
Description
0
Emergency
System or service unusable.
Examples:
• Service repeatedly fails to start up
• System ran out of disk space while disk space
is essential for this system to operate
• Application requires root privileges to run but
does not have them.
1
Alert
Action must be taken immediately.
Examples:
• Application is about to run out of license.
• Application is about to run out of disk space.
• Too many unauthorized access attempts detected.
• Denial of service attack is detected.
2
Critical
Critical condition. Similar to alert, but not
necessarily requiring an immediate action.
Examples:
• Received an invalid authentication request.
• Service crashed due to an error that could not be
handled, like an out of memory condition,
(provided it has a watchdog process to restart it,
it does not necessarily require immediate action).
• Unexpected code error that could not be handled.
3
Error
An error condition, which does not necessarily
impact the ability of the service to continue to
function.
Examples:
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
166
Accessing and Interpreting SNMP Data
Severity
Name
Description
• Problem parsing/processing a particular request
which does not prevent the application from
handling other request.
• Unexpected, but handled, code exception.
4
Warning
A warning about some condition, which is not
necessarily an error.
Examples:
• Lost network connection to some resource.
• Timed out waiting for a response.
5
Notice
Notifications about interesting system-level
conditions, which are not error conditions.
Examples:
• Configuration was updated (not audit level
information).
• Process has halted.
• Process is shutting down gracefully on request.
6
Informational
Informational messages are distinguished from
notification in that they provide information for
internal flows of the application or per-request
information instead of system-wide notifications.
Informational messages are used for
troubleshooting by users who are familiar with the
basic flows of the application.
Examples:
• Request received.
• Request was parsed successfully.
• Request being processed.
• Response sent back.
• Acknowledgment received.
• Detailed audit information.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
167
Accessing and Interpreting SNMP Data
Severity
Name
Description
7
Debug
Debugging messages are similar to informational
messages, but provide more detail and they require
the user to have better knowledge of system
internals. These messages are typically reserved
for very advanced users or Cisco technical support.
Examples:
• Complete details for a request packet.
• Internal state machine state changes.
• Internal profiling statistics.
• Internal events.
See Also
For additional information, refer to the Configuration and Administration Guide for Unified
Customer Voice Portal.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
168
Appendix E
Internal Interfaces Description
The following table summarizes the major interfaces between the Unified CVP product
components and other Cisco products.
Table 10: Major Internal Interfaces
Interface
Interface Characteristics
Between This Component... ...and This Component
Interface Type
Selection Determined By
Gateway (Ingress or Egress) Call Server
SIP
SIP Proxy Server or dial-peer
configuration
H.323 Service
H.323
Gateway dial peers or
Gatekeeper, based on H.323
Service availability and
proximity
Gateway
Gatekeeper
H.323
Gateway configuration
H.323 Service
IVR Service
HTTP (URL with VXML
response). H.323 Service
client, IVR Service server.
H.323 Service configuration
Media Server
HTTP
URL is given in VXML
generated by the IVR Service
from information specified in
Unified ICME.
Unified CM/IP Telephone
Gateway
Unified CM/IP Telephone
Normal web access (DNS,
distributors, and so forth) is
used to resolve it.
Note: The H.323 to Media
Server interface exists ONLY
in legacy H.323 with Media
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
169
Internal Interfaces Description
Interface
Interface Characteristics
Termination call flow
models.
IVR Service
Unified ICME
ICM VRU Messaging
CVP configuration
Gateway
VXML Server
HTTP (URL with VXML
response)
Unified ICME script
configuration (though this
does not apply for VXML
Server (standalone))
Unified ICMH
Unified ICME
Proprietary
Customer identified by
calling information; Unified
ICMH configuration
determines Unified ICME
Unified ICME
Customer database
SQL Gateway
Unified ICME Configuration
IVR Service
Unified ICME Configuration
Reporting Server
Informix DB
JDBC
Operations Console
VXML Server
Call Server
Proprietary
Operations Console,
VoiceXML script, and
Unified ICME script
configuration
VMS
Call Server
HTTPS
MediaServer variable as
configured in the ICM script
or Call Server IVR tab.
VMS
IVP
RTSP/RTP
MediaServer variable as
configured in the ICM script
or Call Server IVR tab.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
170
Appendix F
Trace Bits Description
This appendix lists the common trace bits, their corresponding values and the description for
each Unified CVP subsystem type. Each value is unique within each Unified CVP subsystem.
Type
Trace Bits
Value
Description
Globally Reserved
Trace Settings
GLOBAL RESERVED 3
0x00000200
Reserved for future use
GLOBAL RESERVED 4
0x00000400
GLOBAL RESERVED 5
0x00000800
GLOBAL RESERVED 6
0x00001000
GLOBAL RESERVED 7
0x00002000
GLOBAL RESERVED 8
0x00004000
GLOBAL RESERVED 9
0x00008000
Message Layer Trace MSGLAYER_MESSAGE
Level Settings
MSGLAYER_METHOD
0x0001000000L Trace mask for message values
0x0002000000L Trace mask for method values
MSGLAYER_HANDLED_EXCEPTION 0x0004000000L Trace mask for exception values
MSGLAYER_PARAM
0x0008000000L Trace mask for param values
MSGLAYER_GLOBAL_EVENT 0x0010000000L Trace value from the old messaging layer
code
MSGLAYER_EXTERNAL_EVENT 0x0020000000L Trace value for external event
MSGLAYER_STATIC_FIELD
0x0040000000L Trace value for static field
MSGLAYER_EXTERNAL_STATE 0x0080000000L Trace value for external state
MSGLAYER_INTERNAL_STATE 0x0100000000L Trace value for internal state
MSGLAYER_CODE_BRANCH 0x0200000000L Trace value for code branch
MSGLAYER_CODE_MARKER 0x0400000000L Trace value for code marker
MSGLAYER_CLASS_DUMP
0x0800000000L Trace value for class dump
MSGLAYER_LOCAL_DUMP
0x1000000000L Trace value for local dump
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
171
Trace Bits Description
Type
ORM Trace Level
Settings
Subsystem Video
Media Trace Level
Settings
OAMP Trace Level
Settings
Trace Bits
Value
Description
INPROCESS_MESSAGE
0x00100000L
Trace value for external event
INPROCESS_METHOD
0x00200000L
Trace value for static field
INPROCESS_HANDLED_EXCEPTION 0x00400000L
Trace value for external state
INPROCESS_PARAM
0x00800000L
Trace value for internal state
OUTPROCESS_MESSAGE
0x01000000L
Trace value for message traces
OUTPROCESS_METHOD
0x02000000L
Trace value for method input/output traces
OUTPROCESS_HANDLED_EXCEPTION 0x04000000L
Trace value for message handling
exceptions
OUTPROCESS_PARAM
0x08000000L
Trace value for message parameters
TRACE_ORM_INFRA
0x00010000
Traces related to ORM startup, utilities,
infrastructure
TRACE_ORM_FT
0x00020000
Traces related to File Transfer (both FTP
and HTTP)
TRACE_ORM_IOS
0x00040000
Traces related to a IOS low level code
logging
TRACE_ORM PC
0x00080000
Traces related to CVP services
RACE_ORM_PC_MSGBUS
0x00100000
Traces related to CVP Service messages
sent across through messagebus
TRACE_ORM_PC_MSGBUS_HEARTBEA 0x00200000
Traces related to CVP Services heartbeat
TRACE_ORM_PC_CONTROLCENTER_STATS 0x00400000
Traces related to CVP Service Control
Center Stats
Subsystem Init
0x00010000L
VMS subsystem initialization trace
Subsystem Shutdown
0x00020000L
VMS subsystem shutdown trace
Media Poller
0x00040000L
Media poller operations
Media
0x00080000L
Media modification operations
Search
0x00100000L
Search operations
Metadata Factory
0x00200000L
Metadata factory operations
VMS Web
0x01000000L
Outputs debug information from the VMS
web code
Subsystem Updates
0x00800000L
Outputs debug information from the VMS
Subsystem (other than startup/shutdown),
such as state machine changes.
Streaming Server Poller
0x00400000L
Outputs debug information for the
streaming server keep-alive thread (a
thread that polls Darwin to make sure it's
up and running).
MISC_CONFIGURATION
0x10000000L
Misc logging
DEVICE_CONFIGURATION
0x01000000L
Device logging
ST_CONFIGURATION
0x02000000L
ST logging
BULK_OPERATIONS
0x00100000L
Bulk operations
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
172
Trace Bits Description
Type
Infrastructure Trace
Level Settings
Trace Bits
Value
Description
BULK_EXCEPTION_STACKTRACE 0x00200000L
Bulk exception stack traces
DATABASE_MODIFY
0x00010000L
DB update, insert, delete logging
DATABASE_SELECT
0x00020000L
DB select logging
DATABASE_PO_INFO
0x00040000L
Persistent Object ID and class details
MANAGEMENT
0x00001000L
MGMT logging
LOAD_SUBSYSTEM
0x00010000
Used when loading subsystems
TRACE_THREAD
0x00020000
All Infrastructure thread operations
TRACE_MSG
0x00040000
Information about messages sent and
received
TRACE_MSG_DETAIL
0x00080000
Detailed information about messages sent
and received
TRACE_MESSAGE_HANDLING 0x00100000
Information about the handling of the
messages
TRACE_TIMER
0x00200000
Information about when a timer expires
TRACE_STATE
0x00400000
Information about state changes and events
TRACE_SECURITY
0x00800000
Information about all security operations
TRACE_LICENSING
0x01000000
Information about all licensing operations
TRACE_STARTUP
0x02000000
Information about detailed startup
information
TRACE_SHUTDOWN
0x04000000
Detailed shutdown information
TRACE_STATS
0x08000000
Detailed information about the Stats
Manager operations
TRACE_SNMP
0x10000000
SNMP forwarder logging statistics
0x00010000
Traces related to a JDBC processing
0x00040000
Traces related to a CallRegistry processing
0x00800000
Traces related to load processing
0x00000001
Traces related to a call/call processing
TRACE_METHOD
0x00000002
Traces when entering/exiting a method
TRACE_PARAM
0x00000004
Traces for any parameters (not just method
arguments)
TRACE_LOW_LEVEL
0x00000008
Traces for bits and bytes
TRACE_CLASSDUMP
0x00000010
All information available about a class
TRACE_HEARTBEAT
0x00000020
Traces related to heartbeats, heartbeat
threads, or heartbeats sent/received
Reporting Service
TRACE_JDBC
Trace Level Settings TRACE_CALL_REG
TRACE_BATCH
Common Trace Bits TRACE_CALL
TRACE_HANDLED_EXCEPTION 0x00000040
Traces related to the exception and how it
was handled
TRACE_OOOQUEUE
Traces related to OoOQueue tracing
0x00000080
TRACE_GARBAGE_COLLECTOR 0x00000100
Traces related to GarbageCollector tracing
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
173
Trace Bits Description
Type
Trace Bits
Value
Description
TRACE_MESSAGE
0x00000200
General debug details of
incoming/outgoing messages
TRACE_DEBUG_STATS
0x00000400
Enables printing of debug statistics
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
174
Appendix G
Debugging
This appendix discusses various debugging facilities that are available to help you diagnose
Unified CVP problems.
SIP IOS Debugging
The Session Initiation Protocol (SIP), is an application-layer control (signaling) protocol for
creating, modifying, and terminating sessions with one or more participants. These sessions
include Internet telephone calls, multimedia distribution, and multimedia conferences. SIP is a
simple, ASCII-based protocol that uses requests and responses to establish communication
among the various components in the network and to ultimately establish a conference between
two or more endpoints.
Users in a SIP network are identified by unique SIP addresses. A SIP address is similar to an
e-mail address and is in the format of sip:userID@gateway.com. The user ID can be either a
user name or an E.164 address. Users register with a registrar server using their assigned SIP
addresses. The registrar server provides this information to the location server upon request.
When a user initiates a call, a SIP request is sent to a SIP server (either a proxy or a redirect
server). The request includes the address of the caller (in the From header field) and the address
of the intended called party (in the To header field).
Over time, a SIP end user might move between end systems. The location of the end user can
be dynamically registered with the SIP server. The location server can use one or more protocols
(including finger, rwhois, and LDAP) to locate the end user. Because the end user can be logged
in at more than one station and because the location server can sometimes have inaccurate
information, it might return more than one address for the end user. If the request is coming
through a SIP proxy server, the proxy server tries each of the returned addresses until it locates
the end user. If the request is coming through a SIP redirect server, the redirect server forwards
all the addresses to the caller in the Contact header field of the invitation response.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
175
Debugging
H.323 IOS Debugging
SIP IOS Troubleshooting Commands
The table below lists several debug commands that are useful for troubleshooting problems with
SIP. Use the no form of the command to disable the debug output.
For example, no debug ccsip all
Debug Command
Description
debug ccsip all
Displays all Session Initiation Protocol (SIP)-related
debugging output.
debug ccsip calls
Displays all Session Initiation Protocol (SIP) Service
Provider Interface (SPI) call tracing information.
debug ccsip error
Shows all Session Initiation Protocol (SIP) Service
Provider Interface (SPI) errors.
debug ccsip events
Enables tracing of events that are specific to the service
provider interface (SPI).
debug ccsip info
Enable tracing of general information for the service
provider interface (SPI).
debug ccsip media
Enables tracing of the Session Limitation Protocol (SIP)
call media streams.
debug ccsip messages
Displays all Session Initiation Protocol (SIP) Service
Provider Interface (SPI) message tracing information.
debug ccsip preauth
Enables the diagnostic reporting of authentication,
authorization, and accounting (AAA) preauthentication
for Session Initiation Protocol (SIP) calls.
debug ccsip states
Displays all Session Initiation Protocol (SIP) Service
Provider Interface (SPI) state tracing information.
show sip status
Displays the Session Initiation Protocol (SIP) user agent
listener status.
show sip statistics
Displays the Session Initiation Protocol (SIP) user agent
statistics.
For additional information on these and other Cisco IOS commands, refer to the Cisco IOS
Debug Command Reference (http://www.cisco.com/en/US/products/sw/iosswrel/ps5187/
products_command_reference_book09186a008017cf4d.html)
Note: This section assumes that you possess basic proficiency in the use of Cisco IOS software
commands.
H.323 IOS Debugging
The Cisco Internet Operating System (IOS) is very complex, with many commands and
configuration possibilities. Many of these have no direct relationship to Unified CVP. There
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
176
Debugging
H.323 IOS Debugging
are some basic IOS configuration settings and tools that may be helpful. Below are some of the
more common and useful commands.
Debug Command
Description
show run
Allows the user with EXEC privileges to view
the running configuration. To gain EXEC
privileges, use the enable EXEC command.
For additional information on the enable EXEC
command, refer to the Cisco IOS Command
Reference document.
show ver
Allows the user to obtain the current IOS
version and specific hardware information.
show proc cpu hist
The command displays an ASCII graphical
form of the total CPU usage on the device over
a period of time.
show Gatekeeper endpoints
Displays the status of all the registered
endpoints for a Gatekeeper.
show proc cpu sorted
Displays detailed CPU utilization statistics
(CPU use per process) that are sorted by
percentage.
show memory statistics
Displays a view of the memory information.
copy tftp flash
Run this command to copy a file from a tftp
server into flash.
copy flash tftp
Run this command to copy files from flash to
a tftp server.
show Gatekeeper gw-type-prefix
Use this command to check the priority of the
gateways registered with the Gatekeeper. Check
to see if any are marked as "out-of-resources."
show standby
Displays which Gatekeeper is primary and
which is standby.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
177
Debugging
H.323 IOS Debugging
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
178
Index
Cannot connect....36
Index
IVR error messages....141
media poller....35
audio problems
troubleshooting....60
movie media....34
Overview
Call Server
Troubleshooting overview....5
device state....16
Radvision Errors....163
troubleshooting....13
Reporting Server
Cisco Unified Presence Server
troubleshooting....128
Content Services Switch
troubleshooting....98
Debugging....175
troubleshooting....54
Reporting Server Error Messages....149
SIP
SIP IOS Debugging....175
troubleshooting....109
H.323 IOS....175
SIP Error Message....154
SIP IOS....175
SNMP
device status....10
accessing and intrepreting SNMP data....165
DNIS....31
troubleshooting....117
dropping calls
full video....31
Error Messages
infrastructure....135
Support Tools
troubleshooting....131
Symptoms and Solutions
Troubleshooting Solutions....13
IVR....135
TLS....32
VMS....135
Trace bits....171
Event and Cause Codes....157
H.323
Troubleshooting
Audio Problems....60
H.323 IOS Debugging....176
Backup problems....131
troubleshooting....105
Call Server....13
hold movie....30
checklist....11
ICM Error Messages....151
Cisco Unified Presence Server....128
Infrastructure error messages....135
Connect problems....77
Installation
Content Service Switch....98
troubleshooting....118
Gateway ....124
internal interfaces....169
General system problems....98
invalid images....35
H.323 ....105
IVP Server
Reporting Server....54
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
Index 179
Index
Restore problems....131
SIP ....109
SNMP....117
Support Tools ....131
Transfer problems....77
Unified CM....91
Unified ICME....91
VXML Server....38
Unified CM
troubleshooting....91
Unified ICME
troubleshooting....91
video content
while agent previewing....37
while agent pushing....37
video termination....32
VMS error messages....145
VXML Server
troubleshooting....38
Troubleshooting Guide for Cisco Unified Customer Voice Portal 7.0(2)
Index 180