Troubleshooting Guide
for Cisco Unified Customer Voice Portal
4.1(1)
November 2007
Americas Headquarters
Cisco Systems, Inc.
170 West Tasman Drive
San Jose, CA 95134-1706
USA
http://www.cisco.com
Tel: 408 526-4000
800 553-NETS (6387)
Fax: 408 527-0833
THE SPECIFICATIONS AND INFORMATION REGARDING THE PRODUCTS IN THIS MANUAL ARE SUBJECT TO CHANGE WITHOUT NOTICE.
ALL STATEMENTS, INFORMATION, AND RECOMMENDATIONS IN THIS MANUAL ARE BELIEVED TO BE ACCURATE BUT ARE PRESENTED
WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED. USERS MUST TAKE FULL RESPONSIBILITY FOR THEIR APPLICATION OF
ANY PRODUCTS.
THE SOFTWARE LICENSE AND LIMITED WARRANTY FOR THE ACCOMPANYING PRODUCT ARE SET FORTH IN THE INFORMATION PACKET
THAT SHIPPED WITH THE PRODUCT AND ARE INCORPORATED HEREIN BY THIS REFERENCE. IF YOU ARE UNABLE TO LOCATE THE
SOFTWARE LICENSE OR LIMITED WARRANTY, CONTACT YOUR CISCO REPRESENTATIVE FOR A COPY.
The Cisco implementation of TCP header compression is an adaptation of a program developed by the University of California, Berkeley (UCB) as
part of UCBs public domain version of the UNIX operating system. All rights reserved. Copyright © 1981, Regents of the University of California.
NOTWITHSTANDING ANY OTHER WARRANTY HEREIN, ALL DOCUMENT FILES AND SOFTWARE OF THESE SUPPLIERS ARE PROVIDED
"AS IS" WITH ALL FAULTS. CISCO AND THE ABOVE-NAMED SUPPLIERS DISCLAIM ALL WARRANTIES, EXPRESSED OR IMPLIED, INCLUDING,
WITHOUT LIMITATION, THOSE OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT OR ARISING
FROM A COURSE OF DEALING, USAGE, OR TRADE PRACTICE.
IN NO EVENT SHALL CISCO OR ITS SUPPLIERS BE LIABLE FOR ANY INDIRECT, SPECIAL, CONSEQUENTIAL, OR INCIDENTAL DAMAGES,
INCLUDING, WITHOUT LIMITATION, LOST PROFITS OR LOSS OR DAMAGE TO DATA ARISING OUT OF THE USE OR INABILITY TO USE
THIS MANUAL, EVEN IF CISCO OR ITS SUPPLIERS HAVE BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.
CCVP, the Cisco logo, and Welcome to the Human Network are trademarks of Cisco Systems, Inc.; Changing the Way We Work, Live, Play, and
Learn is a service mark of Cisco Systems, Inc.; and Access Registrar, Aironet, BPX, Catalyst, CCDA, CCDP, CCIE, CCIP, CCNA, CCNP, CCSP,
Cisco, the Cisco Certified Internetwork Expert logo, Cisco IOS, Cisco Press, Cisco Systems, Cisco Systems Capital, the Cisco Systems logo, Cisco
Unity, Enterprise/Solver, EtherChannel, EtherFast, EtherSwitch, Fast Step, Follow Me Browsing, FormShare, GigaDrive, HomeLink, Internet Quotient,
IOS, iPhone, IP/TV, iQ Expertise, the iQ logo, iQ Net Readiness Scorecard, iQuick Study, LightStream, Linksys, MeetingPlace, MGX, Networkers,
Networking Academy, Network Registrar, PIX, ProConnect, ScriptShare, SMARTnet, StackWise, The Fastest Way to Increase Your Internet Quotient,
and TransPath are registered trademarks of Cisco Systems, Inc. and/or its affiliates in the United States and certain other countries.
All other trademarks mentioned in this document or Website are the property of their respective owners. The use of the word partner does not imply
a partnership relationship between Cisco and any other company. (0710R)
Any Internet Protocol (IP) addresses used in this document are not intended to be actual addresses. Any examples, command display output, and
figures included in the document are shown for illustrative purposes only. Any use of actual IP addresses in illustrative content is unintentional and
coincidental.
Copyright © 2007 Cisco Systems, Inc. All rights reserved.
Table of Contents
Preface ...........................................................................................................................................................1
Purpose .....................................................................................................................................................1
Audience ....................................................................................................................................................1
Organization ..............................................................................................................................................1
Related Documentation .............................................................................................................................2
Conventions................................................................................................................................................3
Obtaining Documentation, Obtaining Support, and Security Guidelines....................................................4
1. Troubleshooting Overview...........................................................................................................................5
Problem Solving Process...........................................................................................................................5
...............................................................................................................................................................5
Problem Solving Tasks...............................................................................................................................6
Troubleshooting Checklist.........................................................................................................................12
2. Symptoms and Solutions..........................................................................................................................15
Call Server................................................................................................................................................15
Device Becomes Corrupted.................................................................................................................15
SSL Recovery......................................................................................................................................17
Resource Manager Service Fails to Start............................................................................................17
Call Server Cannot Communicate With Co-located Reporting Server................................................18
Call Server or VoiceXML Service Cannot Be Restarted .....................................................................19
H.323 Service Cannot Access Call Server..........................................................................................19
Resetting System Clock Stops IVR Service Requests........................................................................21
IVR Service CALL_RESULT Returns Error Code Other Than 0..........................................................21
Call Server is Unreachable..................................................................................................................22
Received ERROR_SCRIPT_NAME = 6 on IVR Service.....................................................................23
Received ERROR_MEDIA_FILE = 9 on IVR Service..........................................................................23
“Dialogue Failure Event” Error in the Call Server Log Files.................................................................24
Call Server “Variable Data is Invalid” Error..........................................................................................24
Unable Add IOS Device Because Device is Already Managed...........................................................25
VRU Application Error in Call Server Log............................................................................................26
ECC Variable Contents Truncated After Passing Through Call Server................................................26
"Have You Configured Your Call Server?" Error...................................................................................27
Erratic Behavior When Everything is Configured Properly..................................................................27
IVR Subsystem Goes Into Disabled State When CallServer Service Starts.......................................28
VXML Server............................................................................................................................................28
Ended Session Request Error.............................................................................................................28
Ending Session Request Error............................................................................................................30
Session Does Not Contain Call Information........................................................................................31
Rapid Multiple Sequential Applications Warning.................................................................................32
Session Loss During a Call Error.........................................................................................................33
Session Ending Within an Element Error.............................................................................................35
Empty Error Message..........................................................................................................................36
VXML Server Displays Information of an Old Version..........................................................................36
VXML Server Console Displays No Information..................................................................................37
VXML Server Sample Application Displays Page Not Found..............................................................37
VXML Server Sample Application Displays Technical Difficulties........................................................38
Excessive Unavailable ASR Errors in VXML Server Log.....................................................................38
VXML Server Loggers Not Functioning...............................................................................................39
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
i
VXML Server is Unreachable..............................................................................................................39
VoiceXML Scripts Fail..........................................................................................................................40
Call Start Element Deleted From VXML Server...................................................................................40
VoiceXML Application Fails to Access Another Resource...................................................................41
Error 500: Error Initializing Velocity......................................................................................................42
Keytool Error: Failed to Establish Chain From Reply...........................................................................42
VXML Server Statistics Not Available Error Message..........................................................................43
Reporting Server......................................................................................................................................43
Reporting Server Association Error.....................................................................................................43
No Informix Services Found On Reporting Server..............................................................................44
Starting Informix Database Causes Client Locale Not Set Error.........................................................44
Reporting Server Not Receiving Messages From Message Bus.........................................................45
Reports are Slow.................................................................................................................................45
Reporting Server Cannot Connect to Database..................................................................................46
Reporting Server Goes Into Partial Service Mode...............................................................................46
Reporting Server Cannot Initialize JDBC Connection.........................................................................47
Reporting Server is Unreachable.........................................................................................................47
Errors During Reporting Server Database Purge................................................................................48
Audio........................................................................................................................................................48
Caller Hears Nothing but No Errors Logged........................................................................................48
Caller Does Not Hear Prompt or Expected Event Does Not Occur.....................................................49
One-Way Audio on Voice Gateway......................................................................................................50
GetSpeech Micro-Application With Datatype "Currency" Does Not Work With Negative Currencies...50
A Prompt File From Media Server Did Not Run...................................................................................51
Long Pauses Between Prompts...........................................................................................................52
Caller Hears System Error Message...................................................................................................52
Prompts Not Played Intermittently, Random Prompts, Random ICM Scripts......................................53
No Ring Tone When Transfer to Unified Contact Center Enterprise Agent..........................................53
Garbled Prompts or Voice....................................................................................................................54
Caller Hears Prompt then Dead Air, Fast Busy, or Pause, then Disconnected....................................54
One-way Voice After Agent Transfer....................................................................................................55
Micro-Application May Not Always Create Correct VoiceXML ............................................................56
Large Number of No Matches for Speech Recognition.......................................................................57
Same Prompt Repeats Multiple Times................................................................................................58
Cannot Deploy Newly Added Speech Server......................................................................................58
SpeechWorks Does Not Work With Unified CVP.................................................................................59
Hearing English System Critical Message When Locale Not Set to en-us..........................................62
Speech and DTMF Keystroke Recognition Take Long Time to Complete...........................................62
Speech Recognition Does Not Recognize Wide Character String Grammars....................................63
Large Number of Invalid Entries for Speech Recognition....................................................................63
DTMF Barge-in Not Detected..............................................................................................................64
Clicking Noise Heard During Scansoft TTS on IP Phones..................................................................64
Transfer and Connect...............................................................................................................................65
Agent Available and Transfer Does Not Happen..................................................................................65
Problem with IP Transfer......................................................................................................................65
Agent Cannot Transfer Calls to Unified CVP or Another Agent...........................................................67
Agent Cannot Perform a Consult Transfer and Conference to a Second Agent..................................67
When a Call is IP Transferred, the Caller is Disconnected...................................................................68
A Call Placed From an Analog Phone on an FXS Port Fails...............................................................69
Calls Dropped When a Caller Tries to Invoke Supplementary Services..............................................69
Call is Disconnected During IP Transfer and the Gatekeeper is Configured........................................70
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
ii
Agent Desktop Gets a "Call Failed Due to Network Interruption" Error...............................................70
Call Back For IVR Service Treatment or Subsequent Network Transfer Fails......................................71
Call Disconnected As Soon As Agent Answers...................................................................................72
Call Immediately Disconnected and Caller Hears Nothing..................................................................73
Agent Goes Ready, Queued Caller Never Delivered to Agent.............................................................73
Post-Route Transfers Do Not Work......................................................................................................74
No Ringback On Blind Transfer............................................................................................................75
RONA and Router Requery Not Working as Expected........................................................................75
Call Disconnected as Soon as Agent Answers....................................................................................76
DTMF Tones not Recognized..............................................................................................................77
Unified CM................................................................................................................................................78
Blind Transfer Does Not Work with Type 10 VRU if Unified CM Configuration Changes Not Made ....78
ASP Error Occurs on Unified CM .......................................................................................................79
Unified ICME............................................................................................................................................79
ICM Lookup Requests Show 0............................................................................................................79
ICM Timeout Occurs and No Error Found...........................................................................................80
Call Receives Default Route Label From Unified ICME.......................................................................80
ICM Script Fails on Run VRU Script Node...........................................................................................80
Running Out of ECC Variable Space...................................................................................................81
VRU Script Returns Incorrect Result...................................................................................................82
Caller is Restarted at Beginning of ICM Script....................................................................................82
Numeric data is not Playing in the Expected Format...........................................................................83
ICM Service Control Only Displays H.323 Service After Core Software Install...................................83
Error Code 31, 32, 33 Returned to ICM Script....................................................................................84
Error Code 9 Returned to ICM Script..................................................................................................84
Content Services Switch..........................................................................................................................85
Content Services Switch Shows Call Server Out of Service but IVR Service is up.............................85
General System Issues............................................................................................................................85
Intermittent Timeout Issues.................................................................................................................85
Windows OS Application Log Fills up..................................................................................................86
Error Message: Operation Failed. Device Could Not Be Reached.......................................................87
"Java is Not a Recognized Program" Error..........................................................................................87
"Password Does Not Meet the Operating System Security Requirements" Error................................88
Password Lost or Forgotten.................................................................................................................88
Server IP Address Has Been Changed...............................................................................................88
Unable to Open Command Prompt Window or Other Applications.....................................................89
When Hyperthreading is Enabled, Unified CVP Performance is Degraded.........................................90
Users Cannot Log In............................................................................................................................90
Errors Reported by Various Unified CVP Components........................................................................91
Other Areas to Check..........................................................................................................................91
H.323 Troubleshooting..............................................................................................................................92
H.323 Service Process Restarts Unexpectedly...................................................................................92
H.323 Service Out of Service..............................................................................................................92
H.323 Service Log Indicates Wrong Number of Licensed IVR Service Ports......................................93
"H.323 Service Must be Registered With Gatekeeper to Perform IP Transfers” Error.........................94
“Unexpected Event for the Gatekeeper Registration” Error.................................................................94
VBadmin Command Fails....................................................................................................................95
SIP Troubleshooting..................................................................................................................................95
SIP to SIP Calls Do Not Play Ring Ringback Tone..............................................................................95
SIP Calls Experience Long Delay or Fast Busy...................................................................................96
SIP Calls Receives 500 Internal Server Error "Routing Failed" Event.................................................97
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
iii
SIP Call Receives 489 Bad Event Message........................................................................................97
Back to Back User Agent (B2BUA) Sending 503 Server Not Available...............................................98
REJECTED WITH 401 - Unauthorized Reason: Q.850;cause=57 .....................................................98
MTP Resource Allocated Even Though MTP is Disabled....................................................................99
SIP Call Fail On SIP Trunk On Unified CM to Back to Back User Agent..............................................99
SIP Call is Cleared From Unified CVP With Code 127......................................................................100
Critical Error Audio is Not Heard on an Abnormal Disconnect SIP Call............................................100
SIP Calls Survivability Script Failing..................................................................................................101
482 Loop Detected Error From CUPS SIP Proxy..............................................................................101
SIP Call Fails With 404 Not Found Error Message............................................................................102
SIP Call With IVR Service is Terminated With Reason Code: Q.850;Cause=38...............................102
SNMP Troubleshooting...........................................................................................................................103
SNMP Notification: authenticationFailure...........................................................................................103
Installation Troubleshooting....................................................................................................................104
InstallShield Engine Cannot Be Installed...........................................................................................104
File in Use Message When Uninstalling............................................................................................105
Error Uninstalling WebSphere Application Server.............................................................................105
Password Issues................................................................................................................................106
"HTTPServer Installed Before WebSphere" Installer Message.........................................................106
Reporting Password Error During Installation....................................................................................107
Dialog Box During Reporting Uninstallation.......................................................................................107
Call Server CPU spikes to 90%-100%...............................................................................................108
Gateway Troubleshooting.......................................................................................................................108
Ringing Stops on IP Phone Calling TDM...........................................................................................108
Gateway Cannot Connect to ASR/TTS Server..................................................................................109
Pressing Digit 9 Causes No-Match....................................................................................................110
Receiving Bootstrap Warning Error ..................................................................................................111
Switch Transfer is Not Going to the Gateway Defined in the Gatekeeper Zone Prefix Command.....112
Cisco Unified Presence Server..............................................................................................................113
Installation Keeps Failing in Network Connectivity Validation Page...................................................113
The Install Complains About Hardware Check Failure......................................................................113
Call Fails When Transferred to Agent.................................................................................................114
Support Tools..........................................................................................................................................114
Unable to Login to Support Tools.......................................................................................................114
Backup And Restore...............................................................................................................................115
Reporting Server Purging Daily.........................................................................................................115
Changing Backup Server Requires Restart.......................................................................................115
"Retrying ASR Server. Last tried BACKUP asr-en-US-backup. primary tried:0, backup tried:1" Error.116
Appendix A. Event and Cause Codes.........................................................................................................117
IVR Service Error Codes........................................................................................................................117
ICM Error Codes.....................................................................................................................................119
Infrastructure Error Messages................................................................................................................121
SIP Codes..............................................................................................................................................126
Appendix B. Accessing and Interpreting SNMP Data.................................................................................129
Appendix C. Debugging..............................................................................................................................133
SIP IOS Debugging................................................................................................................................133
SIP IOS Troubleshooting Commands................................................................................................134
H.323 IOS Debugging............................................................................................................................134
Index ...........................................................................................................................................................137
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
iv
Preface
Purpose
This manual describes how to troubleshoot common issues for the Cisco Unified Customer
Voice Portal (Unified CVP) product. The document also explains the tools and techniques that
can assist in the troubleshooting process.
Audience
This document is intended for Contact Center managers, Unified CVP system managers, VoIP
technical experts, and IVR Service application developers.
Note: Be aware that some Cisco product and Unified CVP component names have changed.
Refer to the Glossary in the Planning Guide for Cisco Unified Customer Voice Portal for
information about product and component names, acronyms, and their descriptions.
Organization
The document is divided into the following chapters.
Chapter
Description
Chapter 1, Troubleshooting
Overview
Describes various methods of approaching troubleshooting tasks.
Chapter 2, Symptoms and
Solutions
Describes particular issues, any associated error messages, the cause of the issue, as
well as the solution.
Appendix A, Event Codes and
Reason Codes
Table that shows how error codes that are received from the H.323 Services are
mapped to Unified ICME event IDs and cause codes.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
1
Preface
Related Documentation
Chapter
Description
Appendix B, Accessing and
Interpreting SNMP Data
Provides information on SNMP severity levels and their descriptions.
Appendix C, Debugging
Describes SIP troubleshooting for some problems that may occur, and describes
some useful command line commands for H.323 IOS troubleshooting.
Related Documentation
Unified CVP provides the following documentation:
• Cisco Security Agent Installation/Deployment for Cisco Unified Customer Voice Portal
provides installation instructions and information about Cisco Security Agent for the Unified
CVP deployment. We strongly urge you to read this document in its entirety.
• Configuration and Administration Guide for Cisco Unified Customer Voice Portal describes
how to set up, run, and administer the Cisco Unified CVP product, including associated
configuration.
• Element Specifications for Cisco Unified CVP VXML Server and Cisco Unified Call Studio
describes the settings, element data, exit states, and configuration options for Elements.
• Installation and Upgrade Guide for Cisco Unified Customer Voice Portal describes how to
install Unified CVP software, perform initial configuration, and upgrade.
• Operations Console Online Help for Cisco Unified Customer Voice Portal describes how to
use the Operations Console to configure Unified CVP solution components.
• Planning Guide for Cisco Unified Customer Voice Portal provides a product overview and
describes how to plan for a Unified CVP deployment.
• Port Utilization Guide for Cisco Unified Customer Voice Portal describes the ports used in
a Unified CVP deployment.
• Programming Guide for Cisco Unified CVP VXML Server and Cisco Unified Call Studio
describes how to build components that run on the Cisco Unified CVP VXML Server.
• Reporting Guide for Cisco Unified Customer Voice Portal describes the Reporting Server,
including how to configure and manage it, and discusses the hosted database.
• Say It Smart Specifications for Cisco Unified CVP VXML Server and Cisco Unified Call
Studio describes in detail the functionality and configuration options for all Say It Smart
plugins included with the software.
• Troubleshooting Guide for Cisco Unified Customer Voice Portal describes how to isolate
and solve problems in the Unified CVP solution.
• User Guide for Cisco Unified CVP VXML Server and Cisco Unified Call Studio describes
the functionality of Cisco Unified Call Studio including creating projects, using the Cisco
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
2
Preface
Conventions
Unified Call Studio environment, and deploying applications to the Cisco Unified CVP
VXML Server.
For additional information about Unified ICME, see the Cisco web site http://www.cisco.com/
en/US/products/sw/custcosw/ps1001/tsd_products_support_series_home.html listing Unified
ICME documentation.
Conventions
This manual uses the following conventions:
Convention
Description
boldface font
Boldface font is used to indicate commands,
such as user entries, keys, buttons, and folder
and submenu names. For example:
• Choose Edit > Find.
• Click Finish.
italic font
Italic font is used to indicate the following:
• To introduce a new term. Example: A skill
group is a collection of agents who share
similar skills.
• For emphasis. Example: Do not use the
numerical naming convention.
• A syntax value that the user must replace.
Example: IF (condition, true-value,
false-value)
• A book title. Example: See the Cisco CRS
Installation Guide.
window font
Window font, such as Courier, is used for the
following:
• Text as it appears in code or that the window
displays. Example: <html><title>Cisco
Systems,Inc. </title></html>
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
3
Preface
Obtaining Documentation, Obtaining Support, and Security Guidelines
Convention
Description
< >
Angle brackets are used to indicate the
following:
• For arguments where the context does not
allow italic, such as ASCII output.
• A character string that the user enters but
that does not appear on the window such as
a password.
Obtaining Documentation, Obtaining Support, and Security Guidelines
For information on obtaining documentation, obtaining support, providing documentation
feedback, security guidelines, and also recommended aliases and general Cisco documents, see
the monthly What's New in Cisco Product Documentation, which also lists all new and revised
Cisco technical documentation, at:
http://www.cisco.com/en/US/docs/general/whatsnew/whatsnew.html
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
4
Chapter 1
Troubleshooting Overview
When troubleshooting a telephony or IP network environment, it is important to define the
specific symptoms, identify all potential problems that could be causing the symptoms, and
then systematically eliminate each potential problem from most likely to least likely until the
symptoms disappear.
This section contains the following topics:
• Problem Solving Process, page 5
• Problem Solving Tasks, page 6
• Troubleshooting Checklist, page 12
Problem Solving Process
The following steps provide some guidelines to assist in isolating a particular issue.
Step 1
Analyze the problem and create a clear problem statement. Define symptoms and potential
causes.
Step 2
Gather the facts that you need to help isolate possible causes.
Step 3
Consider possible causes based on the facts that you gathered.
• Was anything recently added, removed, or modified?
• Is it a reproducible event?
• Does it occur at a particular time of day, or day of week?
• Have there been any changes made to the domain, network, or security policies?
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
5
Chapter 1: - Troubleshooting Overview
Problem Solving Tasks
Step 4
Create an action plan. Begin with the most likely problem and devise a plan in which you
manipulate only one variable at a time.
Step 5
Implement the action plan, performing each step while testing to see whether the symptom
disappears.
Step 6
Analyze the results to determine whether the problem has been resolved. If it has, the process
is complete.
Step 7
If the problem has not been resolved, create an action plan based on the next most probable
cause on your list, or contact the Cisco Technical Assistance Center (TAC), or your Cisco
Partner.
Note: Only change only one variable at a time. If that does not resolve the issue, undo that
change and move on to the next step of your plan.
Problem Solving Tasks
In addition to the process discussed above, there are other areas to consider if a problem arises
within the Unified CVP environment. The information below provides some additional
information to assist in isolating or resolving a particular issue.
• Verify that the software versions comply with the Unified CVP Hardware and System Software
Specifications (Bill of Materials) document. A version incompatibility might not necessarily
be the cause of the issue, but can be investigated and considered as a possibility. This document
is available at:
http://www.Cisco.com/en/US/products/sw/custcosw/ps1006/
prod_technical_reference_list.html
• Review the Unified CVP logs for possible error or warning events. Use the Support Tools
utility to view logs, since it has the ability to retrieve and process trace logs from most
components, and has the ability to set or reset trace levels on those components. To access
the Support Tools, open the Operations Console, and click Support Tools from the Tools
menu. For additional information on Support Tools, refer to the Configuration and
Administration Guide for Cisco Unified Customer Voice Portal.
Note: Users can now execute some UNIX equivalent commands, such as more, grep, diff,
tail, tee on Unified CVP 4.0(2) servers during troubleshooting. These commands are part of
the Support Tools installation. For Unified CVP 4.0(2) and later releases, the PATH
environment variable is updated to allow these commands to be accessible via command
line.
Unified CVP captures two different types of logs: events and errors. When a Unified CVP
component generates an event, it is logged in the Unified CVP log. When a log statement is
written to the log file and the level is at ERROR or above, it is also written to error log file.
Note: This file is created on demand. If there are no ERROR level (or above) logs, you do
not see this file.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
6
Chapter 1: Troubleshooting Overview
Problem Solving Tasks
Below are the locations of the log files for Unified CVP. In the listings below, CVP_HOME
is the location that was selected for the installation of your Unified CVP software, such as
C:\Cisco\CVP.
– CVP_HOME\logs\
Call Server and/or Reporting Server logs.
– CVP_HOME\logs\OAMP\
Operations Console logs.
– CVP_HOME\logs\VXML\
VXML Server logs.
– CVP_HOME\logs\SNMP\
SNMP Agent logs.
– CVP_HOME\logs\ORM\
Unified CVP Resource Manager logs.
Note: The Operations Console Resource Manager (ORM) is a service that is installed
during the installation of the Operations Console. When installing the Operations Console,
the two Windows services that are installed are CVP OpsConsoleServer and CVP Resource
Manager. The Resource Manager resides on each Unified CVP server and it allows the
Operations Console to communicate with that server and perform functions on that server,
such as starting, stop, or monitoring.
Virtually all of the Unified CVP logs start a new log file at 12:00AM every night, with the
date as part of the filename. The VXML Server logs do not necessarily start a new file at
12:00AM. The format of the date is yyyy-mm-dd. These logs also start a new log file when
they reach the predefined size limit of 5 MB and have a number as part of the filename
extension. The number indicates which log it was for that day. And when the entire logs
directory reaches a predefined size of 500 files, old files are purged as necessary. Both the
Log File Size and Log Dir Size are configurable in the device Infrastructure tab of the
Operations Console.
• Check that the network is functioning properly. For the initial connectivity tests, the following
command line tools may be helpful:
– ping - tests whether a host is reachable across an IP network. The ping command offers
several parameters or flags that can be set to produce various outputs. To view all of the
available parameters, type ping /? at the command prompt.
C:\> ping 10.86.132.150
Pinging 10.86.132.150 with 32 bytes of data:
Reply from 10.86.132.150: bytes=32 time=5ms TTL=122
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
7
Chapter 1: - Troubleshooting Overview
Problem Solving Tasks
Reply from 10.86.132.150: bytes=32 time=4ms TTL=122
Reply from 10.86.132.150: bytes=32 time=4ms TTL=122
Reply from 10.86.132.150: bytes=32 time=4ms TTL=122
Ping statistics for 10.86.132.150: Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milliseconds: Minimum = 4ms, Maximum = 5ms, Average
= 4ms
C:\>
– traceroute - determines the packet route that is taken to the target host across an IP network.
The command in a Windows environment is: tracert. The tracert command offers several
parameters or flags that can be set to produce various outputs. To view all of the available
parameters, type tracert /? at the command prompt.
C:\> tracert 10.1.1.6
Tracing route to 10.1.1.6 over a maximum of 30 hops
--------------------------------------------------1 2 ms 3 ms 2 ms 10.1.2.1
2 25 ms 83 ms 88 ms 192.168.11.1
3 25 ms 79 ms 93 ms 10.1.1.6
Trace complete.
– netstat - lists the active incoming and outgoing network connections on a system. The
netstat command offers several parameters or flags that can be set to produce various
outputs. To view all of the available parameters, type netstat /? at the command prompt.
C:\>netstat -a -n
Active Connections
Proto Local Address Foreign Address State
TCP 0.0.0.0:42 0.0.0.0:0 LISTENING
TCP 0.0.0.0:88 0.0.0.0:0 LISTENING
TCP 0.0.0.0:135 0.0.0.0:0 LISTENING
TCP 0.0.0.0:389 0.0.0.0:0 LISTENING
TCP 0.0.0.0:445 0.0.0.0:0 LISTENING
TCP 0.0.0.0:593 0.0.0.0:0 LISTENING
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
8
Chapter 1: Troubleshooting Overview
Problem Solving Tasks
TCP 0.0.0.0:1038 0.0.0.0:0 LISTENING
TCP 0.0.0.0:1041 0.0.0.0:0 LISTENING
TCP 0.0.0.0:1048 0.0.0.0:0 LISTENING
TCP 0.0.0.0:1723 0.0.0.0:0 LISTENING
TCP 0.0.0.0:3268 0.0.0.0:0 LISTENING
TCP 10.99.99.1:53 0.0.0.0:0 LISTENING
TCP 10.99.99.1:139 0.0.0.0:0 LISTENING
TCP 10.99.99.1:389 10.99.99.1:1092 ESTABLISHED
TCP 10.99.99.1:1092 10.99.99.1:389 ESTABLISHED
TCP 10.99.99.1:3604 10.99.99.1:135 TIME_WAIT
TCP 10.99.99.1:3605 10.99.99.1:1077 TIME_WAIT
UDP 0.0.0.0:135 *:*
UDP 0.0.0.0:445 *:*
UDP 0.0.0.0:1087 *:*
UDP 10.99.99.1:53 *:*
UDP 10.99.99.1:137 *:*
UDP 10.99.99.1:138 *:*
• Using the Windows Services control panel, verify that all appropriate services are running
for that server. To view the Service control panel, go to Start and select Run. In the Run
window, type services.msc and click OK.
• Check the Application and System logs in the Windows Event Viewer for warning or error
events. To start the Event Viewer, click Start and select Run. In the Run window, type
eventvwr.exe and click OK. This launches the Event Viewer console, which displays the
three logging options (Application, Security, and System).
• A device may be in some state other than Up, such as Down, or Not Reachable. Check the
Operations Console to verify the status of a particular device. To check the status:
– From the Operations Console, select System > Control Center.
– Click the Device Pool tab in the left frame of the page.
– In the right frame of the page, click the General tab if it is not already selected.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
9
Chapter 1: - Troubleshooting Overview
Problem Solving Tasks
– Locate the device you are interested in and check the status of that device in the Status
column.
• Check the call statistics from the Operations Console to try to determine any abnormalities.
The reported values from the various components might provide some information that will
assist you in isolating the issue.
You can access call statistics either by:
– Choosing System > Control Center, selecting a Call Server, clicking Statistics icon in
the toolbar, and then selecting the various tabs (ICM, SIP, IVR, or Infrastructure).
– Choosing Device Management > CVP Call Server, selecting a Call Server, clicking the
Statistics icon in the toolbar, and then selecting the various tabs (ICM, SIP, IVR, or
Infrastructure).
The examples above list the procedure for viewing the statistics for a Call Server. The statistics
for other devices, such as the Reporting Server, VXML Server, etc. can be viewed in the
same manner.
Below are some of the initial areas to check in the Infrastructure tab for abnormal values or
thresholds.
– Check for licensing issues in the Infrastructure tab. The Licensing Statistics column is
broken up in to a Realtime, Interval, and Aggregate sections. Check the value being reported
for Port Licenses Available against the value of Maximum Port Licenses Used.
– Check the level of the Current Port Licenses State. This reports on the threshold level of
port license usage. There are four levels: safe, warning, critical and failure. An administrator
may set the required percentage of port licenses in use needed to reach a given threshold
level, with the exception of the failure level which is reached when the number of ports
checked out is equal to the total number of licenses ports.
– Check the Uptime statistic, as well as the memory and thread pool statistics for possible
issues. If the Uptime statistic is low, then that indicates that the component was recently
unavailable for some period of time. High memory or thread pool statistics may indicate
a low available resource issue.
Below are some of the initial areas to check in the SIP tab for abnormal values or thresholds.
– In the Interval Statistics section, check the value of the Failed SIP Transfers (Pre-Dialog).
This is the total number of failed transfers on the first Unified CVP transfer in the last
statistics aggregation interval. A SIP dialog is established after the first Unified CVP
transfer is completed. The metric does not include rejections due to the SIP service being
out of service. The metric includes failed transfers that were made after a label was returned
from the ICM in a CONNECT message.
– Check the value of the Failed SIP Transfers (Post-Dialog). The total number of failed
transfers on second and subsequent Unified CVP transfers in the last statistics aggregation
interval. After SIP dialog is established, Unified CVP performs SIP transfers using
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
10
Chapter 1: Troubleshooting Overview
Problem Solving Tasks
re-INVITE messages. The metric does not include rejections of SIP REFER transfers. The
metric includes failed transfers that were made after a label was returned from the ICM
in a CONNECT message.
Below are some of the initial areas to check in the IVR tab for abnormal values or thresholds.
– Check the Active Calls statistic. This is the number of active calls being serviced by the
IVR service currently. Refresh the page to determine if the calls are increasing.
– Check the Average, Maximum, and Minimum Call Latency statistics. These report the
amount of time in milliseconds it took for the IVR Service to complete the processing of
a New Call Request or a Request Instruction Request during the time interval. Long
durations may indicate a network or load problem.
Also, verify that your network environment has been correctly configured. The chart below
shows the speed and duplex settings for Unified CVP.
Ethernet Switch Speed
Capability
Server/Gateway NIC Speed Switch Port Speed/Duplex Server/Gateway NIC
Capability
Setting
Speed/Duplex Setting
1000 Mb
1000 Mb
Auto / Auto
Auto / Auto
1000 Mb
100 Mb
100 Mb / Full
100 Mb / Full
100 Mb
100 Mb
100 Mb / Full
100 Mb / Full
100 Mb
1000 Mb
100 Mb / Full
100 Mb / Full
If you have exhausted all the common causes and actions including those outlined in this
document or others that you have identified in your environment, then contact the Cisco Technical
Assistance Center (TAC), or your Cisco Partner.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
11
Chapter 1: - Troubleshooting Overview
Troubleshooting Checklist
Troubleshooting Checklist
Complete this checklist to assist in isolating the issue, or to provide information to your support
partner or Cisco Technical Support.
1. What is the version of Unified CVP that is currently running? Include any Hotfix or Service
Release (SR) information.
2. Is this a new installation or an upgrade?
3. If this is an upgrade, what version was previously installed?
4. When did the problem occur?
5. What are the observed symptoms, and the conditions under which these symptoms occur?
6. Was anything changed or updated in hardware, software, or network components prior to
the first occurrence of the observed symptoms?
7. Describe the related call flow? Some examples include: Public Switched Telephone
Network (PSTN) originated or IP Phone originated? If it is a transfer, is it Unified CVP
dequeue or agent originated? Is the call inbound ACD, inbound IVR Service treatment,
inbound queue treatment, outpulse transfer, or agent transfer?
8. Is the problem reproducible?
9. What Unified CVP deployment model used for deployment?
10. What is the call transfer method used?
11. Are you able to capture a screenshot of the error or failure? If Yes, save it to a file and
attach to a case.
Network Topology
Complete this checklist to assist in isolating the issue, or to provide information to Cisco
Technical Support.
1. Has the auto-negotiate feature been disabled on all PCs, routers, and switch ports?
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
12
Chapter 1: Troubleshooting Overview
Troubleshooting Checklist
Note: Duplex/speed mismatch between a device and its corresponding port on the switch
is the single most common problem for network latency. Check the network settings in
the Other Considerations section in chapter 1 of this document.
2. Is a network topology diagram available?
3. Which type of IP Gateway is being used in this Unified CVP solution?
4. Save the output from a show tech, show stack, and a show gatekeeper endpoint from
the Gateway.
5. Which type of Gatekeeper is being used in this Unified CVP solution?
6. On which server are the recorded media files located,and what is the path to those files?
7. Are Content Service Switches (CSS) used for failover?
8. Collect and provide versions of IOS, Digital Signal Processor (DSP) , applications, and
end system (ES)/patch levels in the environment.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
13
Chapter 1: - Troubleshooting Overview
Troubleshooting Checklist
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
14
Chapter 2
Troubleshooting
Symptoms and Solutions
This chapter contains problem descriptions and steps to diagnose and resolve each problem.
Problem descriptions are grouped by Unified CVP component.
Call Server
Device Becomes Corrupted
Symptom:
A device has become corrupted. The user receives a specific error message that indicates what
device has been corrupted.
Message:
"Device with IP Address: {1} and Hostname: {2} is in an inconsistent state. Please Save and
Deploy the properties of this Server again to fix the problem. If problem persists, please contact
your administrator for more details."
"Device with IP Address: {1} and Hostname: {2} is not ready at this time. Please retry the
operation at a later time."
"Device with IP Address: {1} and Hostname: {2} is in an inconsistent state and cannot be
properly deleted at this time. Please retry the operation later or try deleting again for forceful
delete."
Cause:
The most likely cause is that the Unified CVP software was reinstalled.
Action:
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
15
Chapter 2 Troubleshooting - Symptoms and Solutions
Call Server
The following steps outline the process to resolve the issue of software being reinstalled on a
server.
1. Identify the affected devices from the Control Center of the Operations Console. The
affected devices are:
- devices with the same IP addresses, and
- devices associated to the devices identified with the same IP address.
2. Go to the Edit page of each device and click Save&Deploy.
3. Restart all affected devices from Control Center in the Operations Console.
If the procedure above does not resolve the issue, then you must clean up the device by
performing the following steps.
Device Clean Up
1. Delete the device from the Operations Console.
2. Reinstall the software on that device.
3. Recreate the device from the Operations Console.
Each Operations Console Server will be identified by a unique identifier. The identifier is
generated at the time of creating a first device through that Operations Console Server. Once
generated, the identifier will then be stored in the Operations Console Server database and in
the Resource Manager Configuration file on every device that includes the Operations Console
Server. The purpose of this unique identifier is to prevent management of a device from multiple
Operations Consoles.
The unique identifier gets imported whenever the import functionality of an Operations Console
database is invoked. Importing a database which was exported from a different Operations
Console Server is not a supported feature unless the Operations Console is newly installed or
has never been used before.
If a user uses an Operations Console Server that is already managing at least one device to
import a database (from a different Operations Console Server or a empty database which was
never used to create a device) the following side effects will occur:
User may not be able to manage devices which were managed by the same Operations Console
server prior to the import. This is because the unique identifiers will now be different between
the Operations Console Server and the previously managed device. User may not be able to
create an IOS device such as Gatekeeper(s), Gateway(s), and CSS(s) from the Operations
Console.
An Operations Console Server that is managing at least one device must not import an empty
database that was previously exported from any Operations Console Server. This will cause a
mismatch in the unique identifiers used since the Operations Console database and the Operations
Console Server have a unique identifier and the empty database has no identifier. If a customer
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
16
Chapter 2Troubleshooting Symptoms and Solutions
Call Server
requires replacing the existing database with an empty database, the recommended way is to
reinstall the Operations Console Server software. If this step is not followed, the customer can
no longer manage devices that are previously managed by the same Operations Console Server.
SSL Recovery
Symptom:
The Operations Console is unable to communicate with the Operations Console Resource
Manager (ORM) service on a device.
Message:
Call Server device with IP Address: <x.x.x.x> and Hostname: <server1> operation failed. Device
could not be reached because of a failure in negotiating security certificates. Please refer to
documentation for configuring security before trying the operation again.
Cause:
If the user checks the secure communications checkbox in Operations Console device
configuration without first configuring security certificates as outlined in Chapter 6 of the
Configuration and Administration Guide for Cisco Unified Customer Voice Portal, the Operations
Console will not be able to communicate with the Operations Console Resource Manager on
the device.
Action:
The solution is to edit the orm_jmx.properties in the conf directory of the device. Change the
property com.sun.management.jmxremote.ssl to false to put the device back into
non-secure mode. Next, uncheck the checkbox in Operations Console and restart the Operations
Console Resource Manager service on the device. This will put both Operations Console and
the device Operations Console Resource Manager back into a non-secure mode. The user then
can configure security certificates before checking the secure communications checkbox.
Note: Changes in security settings require that the ops console service and the resource mgr
services be restarted.
Resource Manager Service Fails to Start
Symptom:
Since the default log setting is ERROR for the Resource Manager Wrapper, the log file should
provide exception details of the failure. The wrapper log file does indicate there is a failure in
Resource Manager service startup, but the details of the event are not logged.
Message:
None.
Cause:
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
17
Chapter 2 Troubleshooting - Symptoms and Solutions
Call Server
The wrapper log was set to the wrong level.
Action:
The change is unable to be made from Support Tools, so the change requires manual steps to
change this setting. M, then save and close the file. Start the 'Resource Manager' service again.
If the Resource Manager service start problem persists, now the ORM logs collected from
Support Tools may show the exact exception. The log file to look for the exception is
%CVP_HOME%\logs\orm_wrapper.log.
1. Stop the 'Cisco CVP Resource Manager' service.
2. Edit %CVP_HOME%\conf\wrapper.conf file to change the property
'wrapper.logfile.loglevel' value from 'ERROR' to 'INFO'.
3. Save and close the file.
4. Start the 'Cisco CVP Resource Manager' service.
If the Resource Manager service fails to start, the ORM logs (orm_wrapper.log) collected from
Support Tools will show the detailed information of the event.
Call Server Cannot Communicate With Co-located Reporting Server
Symptom:
The environment:
Call Server 1 (primary) & Reporting Server 1.
Call Server 2 (secondary)
In the above deployment, when the Operations Console Resource Manager (ORM) on Call
Server 1 is down, the ORM on Call Server 2 is unable to communicate with the Reporting
Server.
Message:
None.
Cause:
This is not a supported model for deployment.
Action:
A co-located Reporting Server can never scale to support multiple Call Servers and must not
be configured in that manner.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
18
Chapter 2Troubleshooting Symptoms and Solutions
Call Server
Call Server or VoiceXML Service Cannot Be Restarted
Symptom:
After applying license files, the Call Server service or the VoiceXML service are unable to be
restarted. Both services can be stopped, but receive a warning message when trying to restart
them.
Message:
Windows could not start the Unified CVP Call Server/VXML Server on Local Computer. For
more information, review the System Event Log. If this is a non-Microsoft service, contact the
service vendor, and refer to service-specific error code 0.
Cause:
The problem is due to the side effect of the first version (1.0) of the MS06-040 patch (KB921883):
Action:
Apply the new version of the Microsoft security patch KB921883.
See Also
Refer to the Microsoft Security Bulletin KB921883 that is available on the Microsoft web site.
Go to: http://support.microsoft.com/
H.323 Service Cannot Access Call Server
Symptom:
Some HTTP requests from the Unified CVP Voice Browser to the Call Server are being blocked.
Message:
Blocked by port blocking rule VoiceBrowser.exe Prevent IRC communication.
Blocked by port blocking rule tomcat5.exe Prevent IRC communication.
Cause:
VirusScan is blocking ports in the range of 6666-6669, which are used in IRC communications.
The application does not have to be listening on these ports for it to be blocked. If the application
is assigned a client port in that range, the attempt to connect to a server using that port is blocked
by VirusScan on the server machine. The connect attempt fails.
Action:
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
19
Chapter 2 Troubleshooting - Symptoms and Solutions
Call Server
Check the VirusScan logs for error messages. access the logs,VirusScan Console -> Task ->
View Log You might find entries like these: 10/10/2006 3:15:01 AM Blocked by port blocking
rule VoiceBrowser.exe Prevent IRC communication 10.86.129.138 10/10/2006 8:49:01 PM
Blocked by port blocking rule tomcat5.exe Prevent IRC communication 10.86.129.211.
To view VirusScan logs
1. Go to Start > Programs > Network Associates > VirusScan Console.
2. Click on the Task menu and select View Log.
You might find entries such as:
Blocked by port blocking rule VoiceBrowser.exe Prevent IRC communication.
Blocked by port blocking rule tomcat5.exe Prevent IRC communication.
INFO: Retrying communication with call server localhost:8000
INFO: Retrying communication with call server localhost:8000
Voice Browser to Call Server localhost:8000 connection unsuccessful.
Voice Browser cannot access any Call Servers and is no longer accepting calls.
Voice Browser <VB machine name here> is out of service.
Exclude the Unified CVP processes from port blocking when using McAfee VirusScan on a
system running Unified CVP Call Server. If a different anti-virus product is used, do the
equivalent exclusions for the port blocking rules for that product.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
20
Chapter 2Troubleshooting Symptoms and Solutions
Call Server
Instructions for McAfee VirusScan on Unified CVP Call Server.
1. Open the VirusScan Console.
2. Click Access Protection.
3. Click Properties under the Task menu.
4. Click the Port Blocking tab.
5. Check both Prevent IRC Communication rules.
6. Click the first Prevent IRC Communication rule and click Edit.
7. Enter VoiceBrowser.exe,tomcat5.exe to the Excluded Process field.
8. Click OK.
9. Click the second Prevent IRC Communication rule and click Edit.
10. Enter VoiceBrowser.exe,tomcat5.exe to the Excluded Process field.
11. Click OK.
Resetting System Clock Stops IVR Service Requests
Symptom:
Resetting the system clock on the Call Server causes the IVR Service to stop functioning.
Message:
None
Cause:
Changing the time of the system clock on the Unified CVP Call Server causes the IVR Service
to stop accepting calls.
Action:
Do not reset the Windows system clock on a machine running Unified CVP. Resetting the
Windows system clock is not supported on a Call Server.
IVR Service CALL_RESULT Returns Error Code Other Than 0
Symptom:
The IVR Service sends a CALL_RESULT error code other than a zero (0) to the Call Server
log.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
21
Chapter 2 Troubleshooting - Symptoms and Solutions
Call Server
Message:
Below is an example of one of the messages:
4765560: 10.86.129.211: Aug 18 2006 06:09:59:484 -0400:
%TEMP:CVP4.0_IVR-7-TRACE_CALL: {Thrd=http-8000-3} VBServlet:service: Request
from 10.86.129.20: {CALL_ID=979679B9-2DD811DB-B72D0014-6944B762,
MSG_TYPE=CALL_RESULT, CALL_SEQ_NUM=2, ERROR_CODE=32}
Cause:
The problem is due to an error with the Run Script Request from ICM.
Action:
Check the Error Code that was received in the Event Codes and Reasons appendix of this
document, and take the appropriate action. For additional information and a complete list of the
cause codes, refer to the Cause Codes and Debug Values section of the Cisco IOS Voice
Troubleshooting and Monitoring Guide.
Call Server is Unreachable
Symptom:
The Call Server is unreachable.
Message:
UNREACHABLE
Cause:
A configured component is unreachable in any one of the following three conditions:
• Operations Console is unable to connect to the Operations Console Resource Manager that
is co-located with the Call Server. For example, the Operations Console Resource Manager
is down).
• The Operations Console Resource Manager on the Call Server has not received a state event
from the controller of that component.
• The Operations Console Resource Manager is unable to connect to the Call Server so it is
not receiving state events from central controller. For example, the Call Server is down.
Action:
In this example, all three services (ICM, IVR, and SIP) have been configured. The central
controller reports that IVR Service and SIP are IN_SERVICE, but it does not report the status
of the ICM Service to the Operations Console for some reason. When this occurs, Operations
Console reports the status of ICM as UNREACHABLE. Operations Console aggregates the
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
22
Chapter 2Troubleshooting Symptoms and Solutions
Call Server
status of the various components of the device to arrive at device status. It sees that two of the
components (IVR Service and SIP) are in SERVICE, but ICM is UNREACHABLE. Operations
Console shows the status of the call server as UNREACHABLE even though IVR Service and
SIP are actually in service.
Check the network environment for connectivity, as well as the actual state of the server.
Received ERROR_SCRIPT_NAME = 6 on IVR Service
Symptom:
The ERROR_SCRIPT_NAME = 6 error code is received on the IVR Service.
Message:
ERROR_SCRIPT_NAME = 6
Cause:
The VRU Script Name data that is passed from the ICM to the IVR Service does not contain
the expected components. These components are micro-application name, media file name, and
media file type.
Action:
In the ICM Network VRU Script List tool, locate the appropriate script and resolve the
configuration error.
See Also
For more information, refer to the Micro-Application configuration section of the Unified CVP
4.1(1) Configuration and Administration Guide.
Received ERROR_MEDIA_FILE = 9 on IVR Service
Symptom:
The ERROR_MEDIA_FILE = 9 error code is received on the IVR Service.
Message:
ERROR_MEDIA_FILE = 9
Cause:
The media file name that is passed from the ICM to the Call Server did not exist on the Media
Server. Another possibility is that the Media Server cannot be reached or the Media Server
configuration contains a misconfigured IP address.
Action:
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
23
Chapter 2 Troubleshooting - Symptoms and Solutions
Call Server
There are several possible actions to resolve this issue:
• Using the ICM Network VRU Script List tool, locate the appropriate script and resolve the
Media File name.
• On the Gateway, type show run to determine if the Media Server IP address is correct.
• Type ping <Media Server IP address> to determine if the server is reachable.
• Verify that the HTTP Server (IIS or Tomcat) is running on the Media Server.
• Check the CPU on the Media Server to verify it is not running close to 100%. If a high CPU
utilization is noticed, then this would indicate that too many requests are reaching the server
and the customer needs additional Media Servers to compensate.
“Dialogue Failure Event” Error in the Call Server Log Files
Symptom:
Callers hear critical error media and “Dialogue Failure Event” errors appear in the Call Server
log files.
Message:
None.
Cause:
If a call reaches the end of ICM script processing without being queued or released, the ICM
sends a “Dialogue Failure Event” message to the Call Server. The Call Server, in turn, sends
an error message to the H.323 Service.
Action:
The author of the script must ensure that each path in the script ends with one of the following
Script Editor nodes: Release, Label, Skill Group, or Queue to Skill Group.
Call Server “Variable Data is Invalid” Error
Symptom:
The Call Server is reporting an error.
Message:
Variable Data is Invalid.
Cause:
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
24
Chapter 2Troubleshooting Symptoms and Solutions
Call Server
The ICM script might truncate values with leading zeros or zeros after decimals or rounds
values.
Action:
Put quotes around numbers in a Script Editor Set node so it is processed as a string. This is
especially important if:
• Leading zeros are present (example: dates).
• Trailing zeros are present after a decimal point (example: currency).
• The number is very large (example: a number normally expressed through exponential
notation).
Unable Add IOS Device Because Device is Already Managed
Symptom:
When attempting to add a device to the Operations Console, an error message is received.
Message:
Gateway device with IP Address: <ip address> and Hostname: <hostname> cannot be created.
The device is already managed by another Operations Console.
Cause:
This occurred because a database was imported from a different Operations Console. Each
Operations Console Server is identified by a unique identifier. The identifier is generated at the
time of creating a first device through that Operations Console Server. Once generated, the
identifier will then be stored in the Operations Console Server database and in the Resource
Manager Configuration file in every device that includes the Operations Console Server. The
purpose of this unique identifier is to prevent multiple Operations Console Servers from
configuring the same device.
The side effects of importing database to an Operations Console Server that is managing at least
one device, if the database to be imported is an exported database from a different Operations
Console Server or is an empty database which was never used to create a device are: 1. 2.
1. User may not be able to manage devices which were managed by the same OPS Console
server prior to the import.
2. User may not be able to create an IOS device such as Gatekeeper(s), Gateway(s), and
CSS(s) from the OPS Console.
Importing a database which was exported from a different OPS Console Server is not a supported
feature. In this case the import may succeed but will have various side effects including above
two side effects.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
25
Chapter 2 Troubleshooting - Symptoms and Solutions
Call Server
Action:
Importing a database which was exported from a different OPS Console Server is not a supported
feature. The solution is to never import an empty database to an Operations Console Server, if
the server is managing at least one device prior to import. If customers require replacing the
existing database with an empty database, the customer is to reinstall the Operations Console
Server software. Doing so, the customer can no longer manage devices that are previously
managed by the same Operations Console Server.
VRU Application Error in Call Server Log
Symptom:
The call might not go through the Call Server, so a prompt is not played. An error or warning
message might appear in the Call Server log.
Message:
VRU APPLICATION ERROR: Assigning the tag 7 to the non-existing ECC variable
“user.microapp.error_code”.
Cause:
The ECC Variable is not configured on the ICM and/or NAM software, or the defined length
is not the same on both devices.
Action:
Add the identical ECC variable definition to the ICM and/or NAM.
ECC Variable Contents Truncated After Passing Through Call Server
Symptom:
The contents of an ECC Variable configured in Script Editor are truncated after passing through
the Call Server. For example, the Text-To-Speech (TTS) text that is spoken to the caller is not
the complete text that was configured. Another example of this behavior is where there are
media fetch failures and the URL to the media file is only a subset of the expected URL.
Message:
None.
Cause:
The cause of this problem may be that the length of the ECC Variable value that was set in
Script Editor is longer than the maximum length of the ECC Variable configured at setup time.
Action:
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
26
Chapter 2Troubleshooting Symptoms and Solutions
Call Server
The solution is to make the maximum length of the ECC Variables longer. This is done using
the ICM Configuration Manager. If you do alter the maximum length of ECC Variables, you
need to restart the Call Server after making the change.
Note: In a NAM/ICM environment, the length needs to be identical on all NAM and ICM or
the variable does not pass.
"Have You Configured Your Call Server?" Error
Symptom:
The Call Server is running, but the an error message is logged in the log file.
Message:
Have you configured your Call Server?
Cause:
The Call Server was started from the Services Control Panel, but it was not configured from
the Operations Console.
Action:
Configure the Call Server from the Operations Console. After completing the configuration,
restart the Call Server from the Services Control Panel.
Erratic Behavior When Everything is Configured Properly
Symptom:
Generally erratic behavior when everything is configured properly. Sometimes TTS plays,
sometimes is does not, or calls stop the VXML Server run script node in ICM. Other behavior
include calls succeed in ICM but no TTS plays, or the caller hears silence, and no errors are
reported in the log. Unavailable TTS errors are reported.
Message:
None.
Cause:
If everything appears to be configured properly, and behavior is generally erratic, check the
configuration of your Ethernet link.
Action:
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
27
Chapter 2 Troubleshooting - Symptoms and Solutions
VXML Server
Do the following:
• Check the Gateway log files.
• Check the VXML Server application logs.
• Verify the network settings. The chart below shows the speed and duplex settings for Unified
CVP.
Ethernet Switch Speed
Capability
Server/Gateway NIC Speed Switch Port Speed/Duplex Server/Gateway NIC
Capability
Setting
Speed/Duplex Setting
1000 Mb
1000 Mb
Auto / Auto
Auto / Auto
1000 Mb
100 Mb
100 Mb / Full
100 Mb / Full
100 Mb
100 Mb
100 Mb / Full
100 Mb / Full
100 Mb
1000 Mb
100 Mb / Full
100 Mb / Full
IVR Subsystem Goes Into Disabled State When CallServer Service Starts
Symptom:
When the CallServer service is started, the IVR subsystem goes into a Disabled state.
Message:
Cause 1:
The CVP license in use is a non-expiring license (no expiration date).
Action 1:
Use a CVP license that has an expiration date
Cause 2:
If the CVP license in use has an expiration date, the CVP CallServer service need to be recycled.
Action 2:
Recycle the CVP CallServer Service. After recycling, check to see if the IVR subsystem is not
in a Disabled state. If it is disabled, the process needs to be repeated a few times.
VXML Server
Ended Session Request Error
Symptom:
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
28
Chapter 2Troubleshooting Symptoms and Solutions
VXML Server
This error message appears when an HTTP request initiated by the root document is made by
the browser for a session that the VXML Server has already ended. The way the VXML Server
knows what session the request is asking for is by looking at the cookie included in the request
for the session’s ID. A request from the root document indicates either a VXML error occurred
or the caller hung up. In a typical call, all requests made for a call session comes in series and
so there is never a request that comes after the VXML Server believes the call has ended. This
error message would appear if a request telling the VXML Server to end the session is followed
by a new request initiated by the root document referencing the same cookie.
Message:
SERVER ERROR: There has been an error with the http session: a request was made due to an
event caught in the root document that refers to a session that has already ended.
Cause 1:
This is a very rare situation as normal operation would not cause any further requests to be made
after the request that ended the session (it is expected that the session-ending request is the last
one). Note that for this error to appear, the new request must come from the root document, in
many cases indicating an error encountered by the H.323 Service. There could be several rare
situations where this could occur. The first begins with the end of call event activated by the
final request to VXML Server taking a long time to run. This would cause the browser to make
a new request timing out the session. This next request would have to be made before the initial
request returned, but after the end of call event completed, causing the VXML Server to end
the session before the new request was received. In order for this situation to occur, the timing
would have to be very precise.
Action 1:
For the first situation, the developer must ensure that their end of call events do not have the
possibility of taking longer to execute than the H.323 Service’s fetchtimeout VoiceXML property.
Another possible solution if the end of call event is a Java class would be to execute the class
in a separately spawned thread. The problem with this would be that the spawned thread should
not attempt to access content in the session due to the possibility that it could be invalidated
while the thread was executing.
Cause 2:
Another situation that could cause this would be some network overhead or slowdown causing
the final response for a call to fail to arrive at the browser, precipitating a time out to occur.
Since VXML Server functioned without problem and ended the session, this new request would
cause the error to occur. Note that some Call Servers and operating systems under high HTTP
load can queue HTTP requests and potentially even lose requests if the queue is too large. The
administrator should investigate this queue if their system is expected to support high load (for
example, on Apache Tomcat, this is known as acceptcount).
Action 2:
For the second situation, it may be an indication of excessive network, operating system, or
Call Server load. For the network, perform diagnostics to ensure proper operation. For the
operating system, check the CPU and memory utilization to ensure they are not maxed out. And
for the Call Server, check the configuration to ensure the HTTP request queue is not too small
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
29
Chapter 2 Troubleshooting - Symptoms and Solutions
VXML Server
(for example, on Apache Tomcat, the acceptcount setting can be configured in the conf/server.xml
configuration file). Finally, the administrator may decide to direct less load on the system.
The administrator may also choose to ignore these issues if they occur rarely since there is no
bad consequences for the call session that encounters this error.
Severity: Due to the fact that this error message appears only in very rare situations, as well as
the fact that even if it does occur, the caller does not encounter any issues with their call, this
error message is benign.
Ending Session Request Error
Symptom:
This error message appears when an HTTP request initiated by the root document is made by
the browser for a session that VXML Server has started the process of ending but that process
is not complete. This is similar to the previous error, but where in that situation the request was
made after the session had completed, this situation involves the request occurring while the
VXML Server was ending the session but before it completed.
Message:
SERVER ERROR: An HTTP request was made due to an event caught in the root document
to a session that has been slated for invalidation.
Cause:
One very rare situation that could cause this error to occur under standard call volume involves
a perfect timing of events. This error would not be produced if the extra request arrives while
an end of call event was executing. However if it did take a long time to execute, prompting
the browser to time out the request by having the root document make a new request, and that
request came at exactly the time the end of call event completed but before VXML Server was
able to finish the session ending process, this error would occur. Since this process normally
occurs rapidly, the chances that this would occur during this period is very small.
This error would also be encountered if the network, operating system, or Call Server on which
VXML Server is installed is sufficiently overloaded such that either HTTP requests are being
queued up too much, HTTP requests are being dropped or the Call Server is so overloaded that
the normally trivial process of ending a session takes a non-trivial amount of time.
Severity: The severity for this error is very low because there is no effect on the caller since
this occurs after the call is complete. Additionally, the situations in which this can occur during
normal operation are extremely rare.
Action:
For the first situation, the developer must ensure that their end of call events do not have the
possibility of taking longer to execute than the H.323 Service’s fetchtimeout VoiceXML property.
Another possible solution if the end of call event is a Java class would be to execute the class
in a separately spawned thread. As long as the end of call class completed its execution before
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
30
Chapter 2Troubleshooting Symptoms and Solutions
VXML Server
the session invalidation delay specified in the conf/global_conf.xml of the installation directory,
there would be no risk of the class referring to data within the session after it was invalidated.
For the second situation, it may be an indication of excessive network, operating system, or
Call Server load. For the network, perform diagnostics to ensure proper operation. For the
operating system, check the CPU and memory utilization to ensure they are not maxed out. And
for the Call Server, check the configuration to ensure the HTTP request queue is not too small
(for example, on Apache Tomcat, the acceptcount setting can be configured in the conf/server.xml
configuration file). Finally, the administrator may decide to direct less load on the system.
The administrator may also choose to ignore these issues if they occur rarely since there is no
bad consequences for the call session that encounters this error.
Session Does Not Contain Call Information
Symptom:
This is a rare error message that is displayed after end session request error. In this situation,
VXML Server has completed its process of ending a session, however the Call Server has not
completed its process for ending the session when a new HTTP request is received referencing
that session. This would yield a legitimate session from the Call Server’s standpoint but an
invalid session by VXML Server’s standpoint.
Message:
SERVER ERROR: The session is not associated with a Server Configuration.
Cause:
The situations where this would be encountered are identical to those listed in the previous error
condition except the timing of the second request would occur after the standard end of call
process taken by VXML Server completed but before the Call Server actually removed the
session. The timing for this situation is actually tighter than the previous situation and therefore
would occur even rarer. While there is a chance that this could occur under standard call volume,
it would more likely occur only under extreme load to the network, operating system, or network.
Severity The severity for this error is very low because there is no effect on the caller since this
occurs after the call is complete. Additionally, the situations in which this can occur during
normal operation are extremely rare.
Action:
For the first situation, the developer must ensure that their end of call events do not have the
possibility of taking longer to execute than the H.323 Service’s fetchtimeout VoiceXML property.
Another possible solution if the end of call event is a Java class would be to execute the class
in a separately spawned thread. As long as the end of call class completed its execution before
the session invalidation delay specified in the conf/global_conf.xml of the installation directory,
there would be no risk of the class referring to data within the session after it was invalidated.
For the second situation, it may be an indication of excessive network, operating system, or
Call Server load. For the network, perform diagnostics to ensure proper operation. For the
operating system, check the CPU and memory utilization to ensure they are not maxed out. And
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
31
Chapter 2 Troubleshooting - Symptoms and Solutions
VXML Server
for the Call Server, check the configuration to ensure the HTTP request queue is not too small
(for example, on Apache Tomcat, the acceptcount setting can be configured in the conf/server.xml
configuration file). Finally, the administrator may decide to direct less load on the system.
The administrator may also choose to ignore these issues if they occur rarely since there is no
bad consequences for the call session that encounters this error.
Rapid Multiple Sequential Applications Warning
Symptom:
This warning message appears when an HTTP request that is not initiated by the root document
is made by the browser for a session that VXML Server has started the process of ending, and
a built-in error-correcting mechanism within the VXML Server has attempted to resolve the
issue by ignoring the cookie hence issuing a warning rather than an error. While the scenario
is similar to previous errors, this is a warning, not an error since the call itself was handled
without the caller encountering any error messages. The warning message is logged to inform
the administrator of a situation that could indicate high call volume, which could cause other
more severe issues.
Message:
SERVER ERROR: Warning: An HTTP request was made to a session that has been slated for
invalidation. It is treated as if not associated with a session.
Cause:
The likeliest situation that would cause this warning message to appear involves a setup where
a VXML Server application is configured to return to ICM, that after some processing promptly
visits a second application within the same call. Due to the fact that these two applications are
visited in the same call, the cookie created in the first application and stored by the browser is
referenced in the first HTTP submit to the second application. This is not normally an issue
because the first application’s session ends before the next application’s first submit is made,
especially if the ICM script performed time-consuming tasks before visiting the second VXML
Server application. However under high volume, the process for ending a session may not take
a trivial amount of time. Additionally, if the ICM script completes in a very short period of
time, basically a shorter amount of time than it takes the Call Server to end a session, the second
application’s first request would arrive before the first application’s session ended and with the
cookie referenced in the header, would make the VXML Server believe that the new request
was made for the first application. VXML Server has a mechanism to recover from this exact
situation. It waits for a certain period of time for the first application’s session to end and when
it does, continues with the call to the second application normally. It logs this warning as a note
to the administrator that the volume may be reaching its upper limits.
Severity: Very low since no call was adversely affected (any problem was averted by the
error-correcting mechanism in the VXML Server).
Action:
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
32
Chapter 2Troubleshooting Symptoms and Solutions
VXML Server
If this warning appears very rarely, the administrator can safely ignore these warnings. If they
appear more frequently, this would be an indication of high volume and the administrator should
attempt to reduce the call volume to the machine.
Should it be found that the ICM script executed in between visits to VXML Server applications
runs too quickly, a workaround to use would be to put a slight pause in this script to allow the
first application’s session to end. Performing load tests on the system yields the optimal pause
length.
Session Loss During a Call Error
Symptom:
The bad_url error message is generally displayed when VXML Server receives an HTTP request
it does not expect or understand. This occurs when a request is made to an application that does
not exist (or the application name is misspelled or has case-sensitive differences). The error
message includes the URL that caused the error, including any HTTP GET arguments. The
message, however, does not report any HTTP POST arguments due to their potential size, so
when the URL “/CVP/Server” is displayed, this indicates that there were either only HTTP
POST arguments passed to the URL or no arguments passed at all. The initial URL for an
application as well as the URL requesting the root document includes GET arguments (i.e.
“/CVP/Server?application=MyApp”), however all subsequent requests made by the VXML
Server use HTTP POST arguments to the /CVP/Server URL. Therefore, this error message is
most likely due to an HTTP submit made for a standard request within a call that has somehow
lost its session or the cookie that references the session.
This error would likely occur within a call and so the caller would hear an error message.
Message:
SERVER ERROR: bad_url: /CVP/Server
Cause:
There are several situations where this can be seen:
1. A H.323 Service or person accesses VXML Server via an incorrectly configured URL
“/CVP/Server” without passing any arguments. Since the VXML Server does not know
what to do, it returns a VoiceXML page with an error message and log the error.
2. Some network device involving either the H.323 Service or some entity in the network
such as a load balancer, loses or removes the cookie associated with the call session. In
an environment in which a load balancer distributes call load across multiple VXML Server
instances, it is necessary for the load balancer to be “sticky”. This means that once a call
is sent to a server, all subsequent requests for the same session must be sent to the same
server. If the load balancer fails to do so, it sends a request with a cookie referring to an
unrecognizable session to a server prompting it to return an error VoiceXML page and
record this exact error. A similar situation occurs if some other network device acts to
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
33
Chapter 2 Troubleshooting - Symptoms and Solutions
VXML Server
remove cookies from HTTP requests or modifies them for its own purposes, these would
become unrecognizable by the VXML Server.
3. An issue with developer-specified code. If the developer uses an element that manually
invalidates the call session, but that element contains VoiceXML that requests a new page,
this error would occur because once the session is invalidated, any further requests would
refer to an unavailable session. In another situation, if the developer creates a hot event
that overrides an existing event caught in the VXML Server root document, this error
would be logged if the event occurs after the call session ended either as a result of a
hangup or other event.
4. Some process that handles HTTP requests causes a request to be delayed so that it arrives
out of order with other requests to the same session. This would only cause the error to
occur if the request that was delayed was followed by a request by the H.323 Service to
end the session. When the delayed request finally arrives, the session it referred to would
have ended by then and the error would be logged. Note that if this delayed request was
initiated from the root document, one of the previous errors listed in this appendix would
occur. This error message would occur only if the delayed request was for a standard page
within the application. This unusual situation would most likely occur when the system
is experiencing high load. Additionally, if the H.323 Service and VXML Server were
connected via the Internet as opposed to via a LAN, there could be a greater chance that
an HTTP request could be delayed.
Severity Situation 1 is most likely due to an error by the administrator or developer and should
never occur in a production system. Situation 2 should also be discovered during the testing
and integration stage and should not occur in production, however there still exists a possibility
with a bug in a load balancer or a glitch in the network setup that would cause this in a live
system. Situation 3 should also be managed with proper testing that would focus on when the
VoiceXML event is thrown. The last situation is the most severe as it could occur in a production
system that is overloaded.
Note: Since this error would most likely fall within a call, the caller would hear an error message
and most likely their call would be terminated. As a result, this error does have consequences
with the caller and should be considered more severe.
Action:
For the first two situations, ensuring the system is properly tested before moving to production
is essential. For the third situation, there should be good justification for defining a hot event
that overrides default functionality. Eliminating this hot event may resolve the issue. For the
final situation, it may be an indication of excessive network, operating system, or Call Server
load. For the network, perform diagnostics to ensure proper operation. For the operating system,
check the CPU and memory utilization to ensure they are not maxed out. And for the Call Server,
check the configuration to ensure the HTTP request queue is not too small (for example, on
Apache Tomcat, the acceptcount setting can be configured in the conf/server.xml configuration
file). Finally, the administrator may decide to direct less load on the system.
Additionally, a preferred setup would involve keeping the H.323 Service on a local network
with the VXML Server machine. That way, all communication would be as fast as possible,
without any pitfalls with using the Internet as the communications medium.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
34
Chapter 2Troubleshooting Symptoms and Solutions
VXML Server
Session Ending Within an Element Error
Symptom:
This error is displayed when during the process of producing a response to a browser request,
the session is invalidated. This is slightly different than the previous situations. In those situations,
the session was already invalidated when the browser makes the request. Here, the session is
invalidated while VXML Server is in the process of handling a request. This occurs within a
call, precipitating an error message to be played to the caller.
Message:
SERVER ERROR: An error occurred because the HttpSession has been invalidated. This could
have been caused by an action element whose execution time exceeded the VoiceXML gateway's
time limit for VoiceXML page requests.
Cause:
There is one specific situation where this can occur under standard load. An element that takes
a long time to complete, more time than the H.323 Service’s fetchtimeout property (typically
around 30 seconds) would hold up the response while it worked. The browser would time out
the request, causing a new request to be made to end the session with an error. When the element
finally completes, any subsequent work done by the VXML Server would encounter a null
session and log this error.
Under correct conditions, elements should execute very quickly. Likely causes for delays would
be when the element is accessing external resources such as databases or, less likely, the element
is performing a very complex calculation. While the error message references an action element
(which is usually the preferred element to make calculations or access databases), this can occur
within any kind of element, both standard and configurable.
Another situation where this could occur would be if the system is under very high call volume.
In this case, the operations of the system may cause normally rapid operations to slow down to
non-trivial lengths of time. This, coupled with an element that performs a slow operation, could
cause this error to occur.
Severity: The severity of this error is high since the caller would encounter an error message
during the call and could be an indication that the system is overloaded.
Action:
When designing an element that has the potential for taking a long time, the developer may
wish to implement mechanisms for handling a slowdown such as implementing internal timeouts,
or using backup systems if the primary systems fail. If the application designer knows that an
element can take a long time to run, they can consider increasing the fetchtimeout VoiceXML
property just for that element, write the element to run in a separate thread so the call can continue
without delay, or provide audio to play while the caller is waiting.
The administrator may also wish to analyze the system to see if there is excessive load on it and
take measures to reduce the load if so.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
35
Chapter 2 Troubleshooting - Symptoms and Solutions
VXML Server
Empty Error Message
Symptom:
This error message is displayed with no message at all other than saying there is a server error.
Message:
SERVER ERROR:
Cause:
This error message appears when a Java exception that itself does not define an error message
occurs. Most of the time, this is a Java NullPointerException, which defines no error message.
Note that all VXML Server errors contain descriptive error messages and only very few Java
exceptions have no message.
Note that if this exception occurred within a custom Java class written by a developer, the Java
stack trace of the exception is displayed in the error log. This stack trace provides the developer
information on where this error occurred. If there is no stack trace, this could be an indication
of a bug in the VXML Server, though it is possible that an incorrectly configured element or
Java class could cause some object to be null that should not be, causing the error to occur within
VXML Server code.
Severity: This message should be considered severe because a NullPointerException only
occurs when Java objects that should have a value do not.
Action:
If this error occurs in a custom Java class, the developer should debug it to find out the issue
by using the stack trace. If the error occurs without a stack trace, under standard call volume,
the developer may first wish to debug the problem to see if there is any custom code that could
be setting up data incorrectly. If the options are exhausted, this would be a situation where Cisco
customer support should be contacted.
VXML Server Displays Information of an Old Version
Symptom:
When starting the Call Server, the VXML Server Call Services displays information of an old
version.
Message:
Information displayed about an older version.
Cause:
A prior version than 4.0 of the VXML Server software is still installed.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
36
Chapter 2Troubleshooting Symptoms and Solutions
VXML Server
Action:
If a prior version of VXML Server software was installed on the same Call Server, it is likely
that the old expanded Web application still resides in the Call Server. Either delete the old
expanded directory or use the Call Server console to decommission the old Web application
and load the new Web application archive (WAR) file.
VXML Server Console Displays No Information
Symptom:
When starting the Call Server, the console displays no information on VXML Server Call
Services at all.
Message:
No message displayed
Cause:
This is most likely caused by not having the VXML Server Call Services Web application
archive file, audium.war, in the correct directory.
Action:
This could have been due to not installing VXML Server Call Services or choosing the wrong
directory in the VXML Server installer for the Call Server’s Web application archive directory.
VXML Server Sample Application Displays Page Not Found
Symptom:
When running the Hello World sample application to test if VXML Server Call Services is
running, the page could not be found.
Message:
Page not found.
Cause:
This URL would be valid only if the Web browser were running on the same machine as Call
Services.
Action:
The URL to be entered to obtain the Hello World application is:
http://localhost:PORT/Audium/Server?audium_application= HelloWorld
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
37
Chapter 2 Troubleshooting - Symptoms and Solutions
VXML Server
This URL would be valid only if the Web browser were running on the same machine as Call
Services. If it is a different machine, the appropriate IP address should be used instead of
“localhost”. Additionally, the port on which the Call Server is listening must be used. Use the
netstat command to determine the port number.
VXML Server Sample Application Displays Technical Difficulties
Symptom:
When running the Hello World sample application to test if the VXML Server Call Services is
running, a VoiceXML page appears indicating that there are technical difficulties.
Message:
VoiceXML pages indicate that there are technical difficulties.
Cause:
Several possibilities exist for this error as outlined below.
Action:
There are several possibilities that exist for this error:
• The application name was not typed in exactly as it appears in the URL shown in the previous
option. All application names do not have spaces
• There is no HelloWorld application deployed on Call Services. Check the
INSTALLATION_PATH\CallServices\applications folder. If it does not contain a folder
named HelloWorld, this application has to be newly deployed from Call Studio.
• When the Call Server was launched, Call Services encountered an error. See the
troubleshooting options in this section for resolutions.
Excessive Unavailable ASR Errors in VXML Server Log
Symptom:
Excessive unavailable ASR errors in VXML Server application error log.
Message:
SERVER ERROR:A VXML exception occurred:error.com.Cisco.media.resource.unavailable.asr
Cause:
The default MRCP timeout value of 3 seconds needs to be increased.
Action:
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
38
Chapter 2Troubleshooting Symptoms and Solutions
VXML Server
The default MRCP timeout values need to be increased to 5 seconds on the gateway as follows:
mrcp client timeout connect 5mrcp client timeout message 5
VXML Server Loggers Not Functioning
Symptom:
When deploying a new application on a VXML Server results with an error message if the
VXML Server is not connected to a Call Server.
Message:
There was a problem instantiating the logger class
com.Cisco.CVP.vxml.datafeed.DatafeedLogger.
Cause:
The VXML Server and the Call Server are not communicating.
Action:
The application cannot be deployed successfully until the VXML Server and Call Server are
communicating. Check the network connectivity between the two servers. Traceroute and Ping
are some of the initial network testing utilities that are available to verity the network integrity.
Be sure to test the network connectivity in both directions. Also verify that the network cards
are set to the correct settings. These settings are provided in the General Systems Issues section
of this document.
VXML Server is Unreachable
Symptom:
The VXML Server shows as Unreachable.
Message:
UNREACHABLE
Cause:
An UNREACHABLE status is reported when the Operations Console Resource Manager does
not get a valid number for active calls after running status script.
Action:
The Operations Console bypasses the Operations Console Resource Manager on the Call Server
to which the VoiceXML service is connected and directly talks to the Operations Console
Resource Manager that is co-located with the VXML Server. The reason for this is that the
status obtained through the central controller on the Call Server is incorrect if no calls have
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
39
Chapter 2 Troubleshooting - Symptoms and Solutions
VXML Server
been placed through the VXML server. It reports and display the status as STOPPED. The status
of VXML server is obtained in a different manner than that for Reporting Server.
VoiceXML Scripts Fail
Symptom:
VoiceXML scripts fail with a CALL_ERROR error message.
Message:
CALL_ERROR; http://10.8.7.70:8080/CVP/Server?application=HelloWorld at line 5: Undeclared
ECMAScript variable audium_vxmlLog
Cause:
The Cisco Gateway is configured with the 'no http client cookie' statement.
Action:
Enable the HTTP client to send and receive cookies on the Cisco Gateway when using VoiceXML
applications by entering the following command:
http client cookie
This command is enabled by default. You do not need to enter it unless you have previously
disabled cookie support by using the 'no http client cookie' command. Use the 'show
running-config' command to verify the HTTP configuration information. If the defaults are
used, HTTP configuration commands are not shown. If changes have been made to the HTTP
defaults, the configuration is shown. For example, the following output shows the HTTP client
cookie parameter configured to a non-default value:
no http client cookie
See Also
For more information on the 'HTTP Client Cookie' feature, see the Cisco IOS Tcl IVR and
VoiceXML Application Guide.
Call Start Element Deleted From VXML Server
Symptom:
Call Start element is deleted from the VXML Server application.
Message:
None.
Cause:
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
40
Chapter 2Troubleshooting Symptoms and Solutions
VXML Server
Usually user error.
Action:
Add the new Page Entry element to the call flow, then right-click on it and choose Start Of
Call -> Yes. This element is automatically used as the first element - start of call. You can then
manually rename it to Call Start.
VoiceXML Application Fails to Access Another Resource
Symptom:
VoiceXML application fails to access another resource, for example, to play a media .wav fail,
and the IOS log contains SERVER ERROR: A VoiceXML exception occurred:
error.badfetch.http.-n, where n is a negative error code.
Message:
SERVER ERROR: A VoiceXML exception occurred: error.badfetch.http.-n
Cause:
error.badfetch.http.<errorcode> has a positive error code when a connection to the HTTP server
is successful, for example; 200 (OK), 400 (Bad Request), or 404 (Not Found). When a gateway
encounters an internal error, including server connection failure, it uses the following negative
error codes to differentiate the error from a server response code:
-1 Client error
-2 Timeout error
-3 Message decoding error
-4 Message encoding error
-5 Message transmission error
-6 Client write queue full
-7 Connection error
-8 Client unsupported
-9 Client socket receiving error
Action:
Make sure the resource you are attempting to access is running. For example, if the VoiceXML
application fails to access a .wav file from a media server, make sure that the media server's
World Wide Web Server service is running.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
41
Chapter 2 Troubleshooting - Symptoms and Solutions
VXML Server
Also make sure that the path to the .wav file is correct.
Error 500: Error Initializing Velocity
Symptom:
In trying to access the link listed below, an error message is received.
http://<server name>:7000/CVP/Licensing
Message:
Error 500: Error initializing Velocity: java.lang.Exception: Unable to configure
AvalonLogSystem: java.security.AccessControlException: Access denied (java.io.FilePermission
/opt/WebSphere/AppServer/profiles/AppSrv01 read)
Cause:
This is due to the Use Java 2 security to restrict application access to local resources being
enabled. When enabled, it prevents the VXML Server to run.
Action:
Modify the setting via the WebSphere Application Server (WAS) Integrated Solutions Console.
Accessing the Integrated Solutions Console
1. From a web browser enter the following link:http://<server name>:9060/ibm/console
(http://<server name>:9060/ibm/console)
2. Click the Security link in the left frame after you login.
3. Click Secure administration, applications, and infrastructure.
4. In the Java 2 security section uncheck Use Java 2 security to restrict application access
to local resources.
5. Click Apply.
Keytool Error: Failed to Establish Chain From Reply
Symptom:
Receiving the error message.
Message:
"keytool error: Failed to establish chain from reply"
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
42
Chapter 2Troubleshooting Symptoms and Solutions
Reporting Server
Cause:
This indicates that the keystore was not set up properly and keytool cannot find the a path from
your certificate down to a root certificate through the intermediate certificate.
Action:
Check the configuration of the keystore.
VXML Server Statistics Not Available Error Message
Symptom:
Receiving an error from the Operations Console that the VXML Server statistics are not available.
Message:
VXML Server statistics are not available.
Cause:
The typical cause of this condition is that the application is not running.
Action:
Deploy a Call Studio script that contains an ICMRequest label. An alternative approach would
be to enable the data feed logger in Call Studio and deploy a Call Studio script that does not
contain an ICMRequest.
Reporting Server
Reporting Server Association Error
Symptom:
A warning popup is produced with an error message and the associated Call Servers changes
are reverted.
Message:
Associate Call Servers can only be changed when deploying to device.
Cause:
When a user creates a new Reporting Server they are required to associate a Call Server. If
neither device is created yet the user must perform a Save, instead of a Save & Deploy. If the
user decides to change, or add to the list of associated Call Servers and perform a Save a warning
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
43
Chapter 2 Troubleshooting - Symptoms and Solutions
Reporting Server
popup is produced with the message "Associate call servers can only be changed when deploying
to device." and the associated Call Servers changes are reverted.
Action:
In order to proceed past this warning the user has two options:
1. Install the CVP software and perform a Save & Deploy
2. Delete the Reporting Server device, readd it with the changes to the associated Call Server
list and Save.
No Informix Services Found On Reporting Server
Symptom:
After installing the Reporting Server, the Informix Services with the Unified CVP database
name are not found.
Message:
None.
Cause:
The .bat files were not run.
Action:
Starting Informix Database Causes Client Locale Not Set Error
Symptom:
When starting Informix from the command prompt generates an error message.
Message:
CLIENT_LOCALE is not set. The database cannot be accessed.
Cause:
The database was tried to be started from the command line instead of the Services control
panel.
Action:
Verify that the database has stopped. Go to the Services Control Panel and start the Informix
service with the appended database name.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
44
Chapter 2Troubleshooting Symptoms and Solutions
Reporting Server
Reporting Server Not Receiving Messages From Message Bus
Symptom:
Reporting server does not receive the messages from the message bus. There is no indication
of an error in the error log.
Message:
None.
Cause 1:
Check if there are other Reporting Servers that are listening on the same port.
Action 1:
Open a command prompt and type: netstat -a
Stop the offending Reporting Server that is listening on the same port. Once that server is
stopped, restart the Reporting Server that was not receiving the messages.
Cause 2:
Another possibility is that the Reporting Server was added from the Operations Console after
the Call Server was started.
Action 2:
Restart the Call Server and the Reporting Server.
Reports are Slow
Symptom:
The reports are slow to generate.
Message:
None.
Cause:
The database may be very large, or report filters may be too wide.
Action:
Contact your database administrator. Request that the database be backup the database, and
then an off hours purge to be scheduled.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
45
Chapter 2 Troubleshooting - Symptoms and Solutions
Reporting Server
Reporting Server Cannot Connect to Database
Symptom:
The Reporting Server is unable to connect to the database.
Message:
None.
Cause:
cvp_db user password has expired.
Action:
Have the Administrator reset the database password.
Reporting Server Goes Into Partial Service Mode
Symptom:
The Reporting Server shows that it is in a partial service mode.
Message:
Partial Service.
Cause:
The Informix database services are not running.
Action:
Go to Start > Administrative Tools > Computer Management.
Click Services and Applications.
Click Services.
Check that the following services are started:
• Informix Dynamic Server Message Service
• Informix IDS - cvp_db_<hostname>
Start services if they are not running.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
46
Chapter 2Troubleshooting Symptoms and Solutions
Reporting Server
Reporting Server Cannot Initialize JDBC Connection
Symptom:
The Reporting Server is unable to initialize the Java Database Connectivity (JDBC) connection.
Message:
RPT-3-REPORTING_INTERNAL_ERROR: Can't initialize the JDBC Connection
java.sql.SQLException: com.informix.asf.IfxASFException: Attempt to connect to database
server (cvp_db_<dbserver>) failed.
Cause:
The database is disconnected and is unable to reconnect possibly due to a bad password.
Action:
Check the database password from the Operations Console. From the Device Management menu
select CVP Reporting Server. Select the Reporting Server, then select Database
Administration -> Change User Passwords. Click Database User and change the password.
Reporting Server is Unreachable
Symptom:
The Reporting Server is Unreachable
Message:
UNREACHABLE
Cause:
Reporting server is UNREACHABLE in any one of the following three conditions:
• Operations Console is unable to connect to the Operations Console Resource Manager that
is co-located with the Call Server. For example, the Operations Console Resource Manager
is down.
• The Operations Console Resource Manager on the Call Server has not received a state event
from the central controller for the Reporting Service.
• The Operations Console Resource Manager is unable to connect to the Call Server and it is
not receiving state events from the central controller. For example, the Call Server is down.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
47
Chapter 2 Troubleshooting - Symptoms and Solutions
Audio
The state of the Reporting Server is obtained from the Operations Console Resource Manager
that is co-located with the Call Server to which the Reporting Server is connected. If Call Server1
and Reporting Server1 are two different machines and both have Operations Console Resource
Managers running on them, Operations Console talks to the Operations Console Resource
Manager on Call Server1 to get status of the Reporting Server because that is where the central
controller reports the status of the Reporting Server.
Action:
Configure the Reporting Server if the Operations Console Resource Manager on the Reporting
Server is running, but Operations Console shows the status as unreachable.
Errors During Reporting Server Database Purge
Symptom:
If you attempt to run a database purge on the reporting server when another purge is already in
progress, or if the reporting server is down, the following messages appear in the log.
Message:
createClientSocket -- Couldn't get I/O streams for the socket.
Error creating the client socket.
Error sending a Start message to Reporting Server.
Error sending an End message to Reporting Server.
Cause:
The reporting server could not open a socket connection to itself, either because it was down
or a purge process was in progress.
Action:
No action necessary. This condition and the associated messages do not prevent the backend
task from completing the purge process.
Audio
Caller Hears Nothing but No Errors Logged
Symptom:
The caller does not hear any voice audio, and Unified ICME and Unified CVP logs do not
indicate any problems.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
48
Chapter 2Troubleshooting Symptoms and Solutions
Audio
Message:
None.
Cause:
There may be a problem with explicitly appending a .wav extension to the Network VRU Script
Name.
Action:
In Configure ICM, remove the .wav extension from the Network VRU Script name. For example,
if the name is PM,welcome.wav, change it to PM,welcome.
Caller Does Not Hear Prompt or Expected Event Does Not Occur
Symptom:
When a call is made, the caller does not hear a prompt or an expected event (such as a transfer)
does not take place.
Message:
None.
Cause 1:
There has been a misconfiguration on the ICM.
Action 1:
Do the following:
1. In Configure ICM, under Network VRU Scripts, note the Network VRU for the Network
VRU script in the ICM Run VRU Script node.
2. In Configure ICM, under Labels for the Network VRU that is running the ICM Script,
click Set Target, click Network VRU and select the Network VRU noted in Step 1.
Note: When using VRU Types 2 or 5, try running Call Tracer.
Cause 2:
An error has occurred in Unified CVP.
Action 2:
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
49
Chapter 2 Troubleshooting - Symptoms and Solutions
Audio
Do the following:
1. Use Monitor mode in the ICM Script Editor to see if the call is following the expected
path (that is, it does not travel through an X branch).
2. Check the Call Server logs to see what the error or warning messages have been generated.
One-Way Audio on Voice Gateway
Symptom:
One-way audio on a AS5350 or AS5400 voice gateway after an IP transfer.
Message:
None.
Cause 1:
One-way audio after a remote H.323 device sends the VoIP-Gateway a HOLD or TRANSFER
message.
Action 1:
Disable the voice-fastpath command on the Gateway, using the syntax: no voice-fastpath
disable.
Note: This command is enabled by default and the syntax is hidden.
Cause 2:
The Gateway and Cisco Unified Communications Manager are in different subnets and no IP
route defined in the Gateway.
Action 2:
Ping from the Gateway to the Unified CM and also from the Unified CM to the Gateway. If
there is a problem with pinging, add an IP route for the different subnet in the Gateway.
For example, specify the command ip route 0.0.0.0 0.0.0.0 10.86.137.1 where
10.86.137.1 is the default routing gateway for the Gateway. This command enables routing to
all subnets.
After entering this command, try a ping session again. If that is successful then you should be
able to listen to the two-way voice.
GetSpeech Micro-Application With Datatype "Currency" Does Not Work With Negative Currencies
Symptom:
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
50
Chapter 2Troubleshooting Symptoms and Solutions
Audio
The GetSpeech Micro-Application with datatype set to "currency" does not work with negative
currencies.
Message:
None.
Cause:
The ASR-TTS server is Nuance 8.5(ASR). This version of Nuance does not support negative
currencies for the VXML Server built-in "currency" grammar.
Action:
Write your own grammar for dealing with currency or contact Nuance support.
A Prompt File From Media Server Did Not Run
Symptom:
A prompt file from Media Server did not run. Other symptoms of this issue may include:
• There is an error message in the Unified CVP logs stating a problem with invalid encoding.
• There is an error message in the Unified CVP logs stating the audio file has a bad header.
• There is an error message in the Unified CVP logs stating that the payload type in audio file
does not match the payload.
Message:
Invalid encoding.
ERROR: Audio file had a bad header.
ERROR: Payload type in audio file does not match the payload.
Cause 1:
Codec mismatch between the Voice Browser configured codec and Audio file codec.
Action 1:
Follow these steps:1.. 2.
1. Convert the prompt file codec to the H.323 Service configured codec (g711Alaw64K or
g711Ulaw64K).
2. Verify that the codec is using the sCodec command from VBAdmin and check the prompt
file properties to verify that codec matches the VB configured codec.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
51
Chapter 2 Troubleshooting - Symptoms and Solutions
Audio
Cause 2:
The audio file is not a valid .wav file.
Action 2:
Re-record or convert the audio file in standard .wav format.
Long Pauses Between Prompts
Symptom:
Caller experiences long pauses between prompts.
Message:
None.
Cause:
There are a couple of possible causes for this issue.
1. There is a WAIT node in the ICM script.
2. There is delay in the network or one of the components in the solution.
Action:
Below are the possible solutions to the causes mentioned above.
1. WAIT nodes are not supported with Unified CVP. Check for WAIT nodes in the ICM
script and remove them.
2. The point of latency within the network must be isolated and resolved.
Caller Hears System Error Message
Symptom:
A caller hears a system error message.
Message:
The caller hears the default audio message that indicates a system error has occurred.
Cause:
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
52
Chapter 2Troubleshooting Symptoms and Solutions
Audio
When a caller hears a system error message, this usually indicates an ICM scripting or
configuration error.
Action:
Do the following:
1. If the caller is experiencing the error before hearing any other prompts, in ICM software
make sure the dialed number is associated with a call type and the call type is associated
with a script.
2. In the ICM script, be certain that all external paths out of Run External Nodes have some
default action (transfer to default LABEL, Queue to Skill Group, and so forth) .
3. In the ICM script, make sure that the last node in the script at the logical end of the call
terminates gracefully, such as with a RELEASE or LABEL. Otherwise, the system error
is played.
4. Unified CVP might have timed out waiting for a response from ICM software. This is
typically caused by an Application Gateway data base dip that is taking longer than the
Unified CVP Call Server timeout settings.
Prompts Not Played Intermittently, Random Prompts, Random ICM Scripts
Symptom:
The callers are encountering problems such as prompts only being played intermittently, are
not being played at all. Other issues include random prompts, and random ICM scripts.
Message:
None.
Cause:
Media file caching is not configured correctly on the gateway.
Action:
If media caching is not enabled on the gateway, then that forces the gateway to load every
prompt for every call. This can sometimes create a situation that overloads the gateway and can
cause prompt play failures. For procedures on how to configure gateway caching, refer to the
Configuration and Administration Guide for Cisco Unified Customer Voice Portal.
No Ring Tone When Transfer to Unified Contact Center Enterprise Agent
Symptom:
No ring tone is heard when transferred to Unified Contact Center Enterprise agent.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
53
Chapter 2 Troubleshooting - Symptoms and Solutions
Audio
Message:
None
Cause:
Cisco Unified CM has an incorrectly configured service parameter.
Action:
In CCMAdmin > Service > ServiceParameters > Unified CM, check the value of H323
parameter Send H225 User Info Message. It must be set to User Info for Ring Back Tone.
Garbled Prompts or Voice
Symptom:
Garbled prompts or voice
Message:
N/A
Cause 1:
There is a PIX firewall between the gateway and Unified CVP or between gateway and Unified
CM.
Action 1:
PIX firewalls can sometimes cause issues where the prompts or voice communications become
garbled. In an attempt to determine if the firewall is the issue, try to temporarily circumvent the
firewall to see if that resolves the issue. If it does appear to be a firewall issue, then review PIX
Firewall documentation for additional information.
Cause 2:
Another possibility is that there is a latency issue in the network that is causing IP phone jitter
or bad quality prompts.
Action 2:
The network should be analyzed to determine if jitter conditions are present. If jitter is detected,
then the IP packets are getting delayed somewhere on the network.
Caller Hears Prompt then Dead Air, Fast Busy, or Pause, then Disconnected
Symptom:
Caller hears prompt and then dead air, or fast busy, or a pause, and is then disconnected.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
54
Chapter 2Troubleshooting Symptoms and Solutions
Audio
Message:
None
Cause:
If this is Comprehensive or VRU-Only, and the last ICM script node executed was a PM (Play
Media) , a dummy dial-peer and translation rule may be missing from the gateway config.
Action:
Play Media VXML contains a "dummy" transfer to 987654 that is meant to fail. However, it is
possible that the gateway contains a dial-peer that tries to send a call to 987654. Add the following
lines exactly as they appear below to the gateway to prevent that dial-peer match from occurring.
Note: 987654 is the actual number you type.
voice translation-rule 1
rule 1 /987654/ //
!
voice translation-profile block
translate called 1
!
dial-peer voice 987654 voip
translation-profile incoming block
incoming called-number 987654
Apply the translation-profile incoming block to all VoIP dial-peers
One-way Voice After Agent Transfer
Symptom:
When a phone call is established from an IP station through a Cisco IOS voice gateway or router,
and the agent transfers the call, only one of the parties receives audio (one-way communication).
Message:
None.
Cause:
No IP connectivity between the phone and the gateway.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
55
Chapter 2 Troubleshooting - Symptoms and Solutions
Audio
Action:
The more common solutions are listed below.
To summarize the more common ones:
• Check Basic IP Routing: Basic IP reachability should always be checked first. As RTP
streams are connectionless (transported over UDP), traffic may travel successfully in one
direction, but get lost in the opposite direction.
• Enable IP Routing: Make sure your router has IP routing enabled, that is, does not have the
global configuration command "no IP routing".
To enable IP routing, simply type the following global configuration command in your Cisco
IOS gateway:
gateway(config)# conf t
gateway(config)# ip routing
• Cut-through Two Way Audio Early: In some cases it is necessary to establish a two-way
audio path as soon as the RTP channel is opened. In other words, before the connect message
is received. To achieve this, type the voice rtp send-recv global configuration command:
gateway(config)# conf t
gateway(config)# voice rtp send-recv
• Disable voice-fastpath: The Cisco IOS command voice-fastpath enable is a hidden global
configuration command for the AS5350 and AS5400, which is enabled by default. To disable
it, use:
gateway(config)# conf t
gateway(config)#no voice-fastpath enable.
See Also
For a more comprehensive list of possible solutions, go to: http://www.Cisco.com/en/US/tech/
tk652/tk698/technologies_tech_note09186a008009484b.shtml
Micro-Application May Not Always Create Correct VoiceXML
Symptom:
The execution of Play Data (PD) Micro-Application may not always create VoiceXML as per
the current VoiceXML and SSML built-in grammar types. This may lead to variations on
expected spoken results when using TTS.
Message:
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
56
Chapter 2Troubleshooting Symptoms and Solutions
Audio
The TTS server bases its rendering of speech interpretation based on how we output these values,
and it may not produce the exact output we are predicting or desire.
Cause:
Configuration not in alignment with VoiceXML standards.
Action:
Change the say-as element and the interpret-as attribute to align them to the following
standards in order to hear the desired output:
Voice Extensible Markup Language (VoiceXML) Version 2.0 (Specifically Appendix P): http:/
/www.w3.org/TR/voicexml20/#dmlABuiltins
Speech Synthesis Markup Language (SSML) 1.0:http://www.w3.org/TR/speech-synthesis/
Speech Recognition Grammar Specification (SRGS) 1.0: http://www.w3.org/TR/
speech-grammar/
Semantic Interpretation for Speech Recognition (SISR) - W3C Working Draft 1 April 2003:
http://www.w3.org/TR/2003/WD-semantic-interpretation-20030401/
Large Number of No Matches for Speech Recognition
Symptom:
There are a large number of no matches for speech recognition
Message:
None
Cause:
Speech recognition confidence threshold is set too high.
Action:
When the Speech recognition server recognizes voice input, it determines a confidence score
that signifies how confident the server is that it correctly matched caller input. This confidence
score is anywhere from 0 to 1.0 where 1.0 is 100% confidence that the grammar match is correct.
By default, Unified CVP tells the Speech recognition server to throw an invalid entry error for
all confidences below 0.7 or 70% for "GetDigits" and "Menu" and 0.5 or 50% for "GetSpeech".
For some applications, this value is too high and results in false invalid entry errors.
Consult the ASR vendor to determine if the default confidence thresholds used by Unified CVP
(0.7 for GetDigits and Menu, 0.5 for GetSpeech) is incorrect for your applications. If the default
values are indeed incorrect and need to be changed, the Unified CVP templates need to be
adjusted. Contact your support contact for instructions on template modification.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
57
Chapter 2 Troubleshooting - Symptoms and Solutions
Audio
Same Prompt Repeats Multiple Times
Symptom:
Same Prompt Repeats Multiple Times
Message:
None
Cause:
When using ASR/TTS, this symptom is caused when the gateway cannot connect to the ASR
or TTS server.
Action:
In Operations Console > Call Server > IVR tab there is a field called Number of tries for
ASR/TTS server. The default is zero. If the gateway cannot connect to the ASR or TTS server,
the value configured determines the number of times it tries to connect. When that happens, the
prompt repeats on each subsequent try. Also, on the gateway the customer can specify a backup
ASR/TTS server (look for “ip host asr-en-us-backup” or “tts-en-us-backup” in the gateway
‘show run’ config). If specified, the gateway tries the backup server it uses the value configured
to determine the number of times it tries to connect. The end result is a condition where the
caller hears the prompt repeat if the value is greater than zero. The solution is to resolve the
connection to the ASR/TTS server.
Cannot Deploy Newly Added Speech Server
Symptom:
An error messages is received after the Unified CVP Operations Console Server, and the
Operations Console Resource Manager on a Nuance Speech Server was installed, and added
the Speech Server in Operations Console and saved configuration.
Message:
"<devicetype> device with IP Address: <ip> and Hostname: <host> operation failed. Device
could not be reached. The device may have multiple NIC cards enabled. To resolve the issue
please keep only one NIC card with required IP address enabled. In addition to the NIC cards
change also restart the Resource Manager service on the device."
Cause:
In some instances, when starting the RMI Server, the RMI server remote stubs contain the IP
address of the local host (127.0.0.1) instead of the real IP address. Since the 'Resource Manager
service' on each machine uses JMX (RMI Server), in the situation where remote stubs containing
local host (127.0.0.1) the Operations Console Server cannot communicate with that specific
'Resource Manager' service. It is because the Operations Console Server uses the real IP address
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
58
Chapter 2Troubleshooting Symptoms and Solutions
Audio
to communicate with JMX (RMI) Server in the Resource Manager Service where as remote
stubs contain local host ip address, so the connection fails.
Action:
As indicated in the error message that is displayed in the user interface (UI), perform the steps
below:
1. Check whether the server where Resource Manager service is running to determine if it
has multiple network interface cards (NIC) enabled. If so, disable all of the NIC cards,
except the one that has the required IP address for Unified CVP communication.
2. Open the Services control panel, and locate the the Unified CVP Resource Manager service.
3. Stop the service.
4. After the service has stopped, start the service.
SpeechWorks Does Not Work With Unified CVP
Symptom:
ScanSoft OSSServer.cfg, and Baseline.xml files require changes.
Message:
The default configuration for SpeechWorks Media Server (SWMS) 3.19, OpenSpeech Recognizer
(OSR) 3.09, RealSpeak 4.0.10 from ScanSoft requires changes to work with Unified CVP.
Cause:
Default configuration still in place.
Action:
The default configuration for SpeechWorks Media Server (SWMS) 3.19, OpenSpeech Recognizer
(OSR) 3.09, RealSpeak 4.0.10 from ScanSoft requires changes to work with Unified CVP. The
following parameters need to be set as follows:
OSSServer.cfg:
The OSSServer.cfg file is for MRCP server and webserver application configuration. On a
Scansoft Server it can be found at the following path location:
C:\Program Files\SpeechWorks\MediaServer\server\config
The following Name, Type and Value entries be configured as shown below. Values should be
changed, lines un-commented or entries added in order to have the OSSServer.cfg file reflect
these entries. Be careful to search for each one first because most exist already and just need
changes to the "Value".
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
59
Chapter 2 Troubleshooting - Symptoms and Solutions
Audio
server.session.ossrec.useWidecharStringGrammers VXIInteger 0
server.transport.port VXIInteger 554
server.transport.dtmfPayloadType VXIInteger 101
server.session.ossweb.machine VXIString <server_name>
server.session.mrcpdefaults.nbest_list_length VXIString 1
server.session.mrcpdefaults.no-input-timeout VXIString 10000
server.session.mrcpdefaults.completetimeout VXIString 1000
server.session.mrcpdefaults.incompletetimeout VXIString 1000
server.session.mrcpdefaults.fetch-timeout VXIString 5000
server.transport.dtmfTriggerLeading VXIInteger 0
server.transport.dtmfTrailingEdgeTimeout VXIInteger 2000
## All DiagTag values should be set to "0" to disable debug and improve
performance.
server.log.diagTag.2000 VXIInteger 0
server.log.diagTag.2001 VXIInteger 0
server.log.diagTag.3000 VXIInteger 0
server.log.diagTag.3001 VXIInteger 0
server.log.diagTag.6000 VXIInteger 0
server.log.diagTag.6001 VXIInteger 0
server.log.diagTag.8000 VXIInteger 0
server.log.diagTag.9000 VXIInteger 0
server.log.diagTag.10000 VXIInteger 0
server.log.diagTag.10001 VXIInteger 0
server.log.diagTag.79999 VXIInteger 0
Baseline.xml:
In the Baseline.xml file, there is a parameter called, swirec_extra_nbest_keys that specifies
which ScanSoft keys to return. The default setting of this parameter:
<!-- Add a ScanSoft grammar key to the XML result. -->
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
60
Chapter 2Troubleshooting Symptoms and Solutions
Audio
<param name="swirec_extra_nbest_keys">
<value>SWI_meaning</value>
<value>SWI_literal</value>
<value>SWI_grammarName</value>
</param>
Below are the results of the default code:
Code: <result><interpretation conf="1"><text mode="voice">reebar</text>
<instance grammar="mygrammar"><SWI_ literal>reebar</SWI_literal>
<SWI_grammarName>mygrammar</SWI_grammarName>
<SWI_meaning>{SWI_literal:reebar}</SWI_meaning>
</instance></interpretation></result>
The following are changes that need to be made to the Baseline.xml file to in order to remove
grammar keys in the XML result. Below are the required changes to the Program
Files\SpeechWorks\OpenSpeech Recognizer\config\Baseline.xml.
1. Open the Baseline.xml file.
2. Change the swirec_extra_nbest_keys parameter from:
<!-- Add a ScanSoft grammar key to the XML result. -->
param name="swirec_extra_nbest_keys">
<value>SWI_meaning</value>
<value>SWI_literal</value>
<value>SWI_grammarName</value>
</param>
to:
<!-- Add a ScanSoft grammar key to the XML result. -->
param name="swirec_extra_nbest_keys">
<value></value>
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
61
Chapter 2 Troubleshooting - Symptoms and Solutions
Audio
</param>
3. Save the Baseline.xml file.
4. Restart the server.
5. With the changes in place, the XML result will now be:
Code:
<result><interpretation conf="1"><text mode="voice">reebar</text><instance
grammar="mygrammar"></ins tance></interpretation></result>
Hearing English System Critical Message When Locale Not Set to en-us
Symptom:
The caller is hearing English system critical media when the locale is set to something other
than en-us.
Message:
None.
Cause:
The language specific prompts were not copied to the H.323 Service.
Action:
You need to copy the language specific prompts to the H.323 Service.
Speech and DTMF Keystroke Recognition Take Long Time to Complete
Symptom:
Any speech and DTMF keystroke recognition seem to take a long time to complete, and initial
words or keystrokes are missed.
Message:
None.
Cause:
Using M, GD or GS Micro-Applications with input_type set to 'B' (both).
Action:
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
62
Chapter 2Troubleshooting Symptoms and Solutions
Audio
Virus scanning can severely degrade recognition speed if you have it running in your recognition
servers. Check with your recognition service provider for their input on how to configure virus
scanning for maximum protection and minimum performance impact.
Speech Recognition Does Not Recognize Wide Character String Grammars
Symptom:
Scansoft Speech Recognition Server does not recognize wide character string grammars.
Message:
None.
Cause:
ScanSoft server not configured for Wide Character String Grammars.
Action:
Set the following:
In the \ScanSoft\SpeechWorks OpenSpeech Media Server\Server\config directory, modify the
OSSserver.cfg file to add
server.session.ossrec.useWidecharStringGrammars VXIInteger 0
In the OSSserver.cfg, set:
server.transport.dtmfPayloadType VXIInteger 101
Large Number of Invalid Entries for Speech Recognition
Symptom:
Large number of invalid entries for speech recognition
Message:
None.
Cause:
Unified CM has an incorrectly configured service parameter.
Action:
When the Speech recognition server recognizes voice input, it determines a confidence score
that signifies how confident the server is that it correctly matched caller input. This confidence
score is anywhere from 0 to 1.0 where 1.0 is 100% confidence that the grammar match is correct.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
63
Chapter 2 Troubleshooting - Symptoms and Solutions
Audio
By default, Customer Voice Portal tells the Speech recognition server to throw an invalid entry
error for all confidences below 0.7 or 70% for GetDigits and Menu and 0.5 or 50% for GetSpeech.
For some applications, this value is too high and results in false invalid entry errors.
Consult the ASR vender to determine if the default confidence thresholds used by Customer
Voice Portal (0.7 for GetDigits and Menu, 0.5 for GetSpeech) is incorrect for your applications.
If the default values are indeed incorrect and need to be changed, the Customer Voice Portal
templates needs to be adjusted. Contact Cisco TAC for instructions on template modification.
DTMF Barge-in Not Detected
Symptom:
Start of speech not flagged in Scansoft configuration file causing DTMF barge-in not to be
detected.
Message:
None.
Cause:
Change made to Scansoft OSSserver.cfg file under ScanSoft\SpeechWorks OpenSpeech Media
Server\Server\config.
Action:
Set the following: server.session.ossrec.mrcp.startOfSpeechOnDTMF VXIInteger
1
Clicking Noise Heard During Scansoft TTS on IP Phones
Symptom:
A clicking noise is heard during Scansoft TTS on IP phones.
Message:
None.
Cause:
Change made to Scansoft OSSserver.cfg file under ScanSoft\SpeechWorks OpenSpeech Media
Server\Server\config.
Action:
Set the following:
server.transport.audio.playerPacketDataSize VXIInteger 160
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
64
Chapter 2Troubleshooting Symptoms and Solutions
Transfer and Connect
Transfer and Connect
Agent Available and Transfer Does Not Happen
Symptom:
When an agent becomes available during prompt-play in a Queue to Skill Group script, the
transfer never happens.
Message:
None.
Cause:
The Interruptible checkbox in a Network VRU Script configuration is not checked. (This is the
default setting). The ICM software does not interrupt a VRU script to perform a transfer if this
feature is not enabled.
Action:
Select the Interruptible checkbox on all Network VRU script nodes that might be run while a
caller is waiting in queue for an agent.
Problem with IP Transfer
Symptom:
Problem with IP Transfer.
Message:
None.
Cause 1:
Unified CM is not registered with the Gatekeeper because the Gatekeeper information in Unified
CM is incorrect.
Action 1:
Change the Gatekeeper host name used during configuration to an IP address.
Cause 2:
Unified CM is not registered with the Gatekeeper because the Gatekeeper information in Unified
CM is incorrect.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
65
Chapter 2 Troubleshooting - Symptoms and Solutions
Transfer and Connect
Action 2:
Do the following:
1. In ccmadmin, select Device > Gatekeeper.
2. Add the Gatekeeper Device Information and select gateway as the Terminal Type.
3. Click Update, then Reset Gatekeeper.
4. Finally click, Reset to apply the changes.
Cause 3:
Unified CM in not registered with Gatekeeper because the Reset and Restart buttons was not
clicked after the Gatekeeper information was updated.
Action 3:
Do the following:
1. In Operations Console, select Device > Gatekeeper.
2. Add the Gatekeeper Device Information and select gateway as the Terminal Type.
3. Click Update, then Reset Gatekeeper.
4. Click Reset to apply the changes.
Cause 4:
Unified CM in not registered with Gatekeeper because a Gatekeeper registration port is not
explicitly configured in Gatekeeper.
Action 4:
Do the following:
1. Check the Gatekeeper configuration to verify that port 1719 is explicitly defined for the
local Gatekeeper zone. For example: zone local gk1 Cisco.com 10.86.137.146
1719.
2. Verify the Unified CM registration in Gatekeeper by entering the show gatekeeper
endpoints command in the Gatekeeper CLI.
Cause 5:
If the Unified CM is an egress Gateway, the Voice Browser is not defined as a Gateway in the
Unified CM configuration.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
66
Chapter 2Troubleshooting Symptoms and Solutions
Transfer and Connect
Action 5:
If the H.323 Service displays the message: ERROR: IP Transfer - Destination endpoint at IP
Address <number1> refused connection for phone number <number2> with reason code
Unreachable Destination (where: <number1> is the Unified CM IP address and <number2> is
the label where you want the call to be transferred), do the following:1.2.
1. In Operations Console on the Unified CM machine by selecting Device > Gateway.
2. Click Find and verify that there is an entry and is configured.
Cause 6:
The call is getting routed elsewhere.
Action 6:
If the H.323 Service displays the message: ERROR: IP Transfer - Destination endpoint at IP
Address <number1> refused connection for phone number <number2> with reason code
Unreachable Destination (where <number1> is not the Unified CM IP address that’s expected
and <number2> is the label where you want the call to be transferred), then there is a problem
with Gatekeeper routing. From the Gatekeeper CLI, enter show gatekeeper
gw-type-prefix and examine the Gatekeeper routing table.
Agent Cannot Transfer Calls to Unified CVP or Another Agent
Symptom:
Agent cannot transfer calls back to the Unified CVP or to another agent.
Message:
None.
Cause:
The Unified CM SingleStepTransferViaRedirect setting is incorrect. Other symptoms: The
OPCTest list indicates that the initial call has not cleared; the Agent Desktop displays the initial
call.
Action:
In the Unified CM, modify the SingleStepTransferViaRedirect setting to 1.
Agent Cannot Perform a Consult Transfer and Conference to a Second Agent
Symptom:
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
67
Chapter 2 Troubleshooting - Symptoms and Solutions
Transfer and Connect
An agent cannot perform a consult transfer and conference to a second agent if
NetworkTransferEnabled is set to 1.
Message:
None.
Cause:
The ICM router cannot distinguish between an agent pressing the consult button vs. the blind
transfer button. Therefore, for ICM, the routing behavior is the same whether the agent types
"1234" or presses consult or blind transfer.
Action:
Uncheck the Network Transfer Preferred box on the Unified CM PIM in PG Explorer.
• Set NetworkTransferEnabled to 1 if the agent is available, or to 0 if the agent is not available.
• If the agent is not available, they still need a Translation Route to VRU node to make a call
queue in the type 2 VRU.
When a Call is IP Transferred, the Caller is Disconnected
Symptom:
When a call is IP Transferred, the caller is disconnected.
Message:
None.
Cause 1:
Gateway and Gatekeeper cannot find each other on the network.
Action 1:
Make sure that the Gateway can ping the Gatekeeper and vice versa:
ping <gk ip address> from Gateway
ping <gw ip address> from Gatekeeper
Make sure that Gateway and Gatekeeper can resolve each other’s names to IP address.
Cause 2:
Configuration on the Gateway might be incorrect.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
68
Chapter 2Troubleshooting Symptoms and Solutions
Transfer and Connect
Action 2:
Verify that the Gateway configuration is correct.
Turn on debug in the Gateway with the following commands:
debug h225 asn1
debug cch323 ras
term mon
If the Gateway is configured properly, it should send a RAS message to register with Gatekeeper.
Note: Turn off debug in the Gateway using the following command: no debug all
A Call Placed From an Analog Phone on an FXS Port Fails
Symptom:
A call placed from an analog phone on an FXS port does not appear to trigger the new_call.vxml
application.
Message:
None.
Cause:
There is no application session command defined on the POTS dial-peer for the telephone’s
voice port.
Note: This command is necessary to enable phones on FXS ports to run a VXML Server script.
Action:
On the POTS dial-peer, model your configuration after the following:
dial-peer voice 1000 pots
application session
destination-pattern 12016881000
port 1/0/0
Calls Dropped When a Caller Tries to Invoke Supplementary Services
Symptom:
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
69
Chapter 2 Troubleshooting - Symptoms and Solutions
Transfer and Connect
Calls are dropped when a caller tries to invoke supplementary services like Call Hold, consult
or conference.
Message:
ERROR: Supplementary service invoked by Caller. This feature is not supported.
Cause:
Caller is trying to invoke supplementary services like Call Hold, consult or conference. This is
an unsupported feature.
Action:
On Unified CM 4.0 (with supported Unified CVP version), turn on MTP (Media Termination
Point).
Call is Disconnected During IP Transfer and the Gatekeeper is Configured
Symptom:
The call is disconnected during IP transfer and the Gatekeeper is configured.
Message:
None.
Cause:
The configured H.323 Gatekeeper is not running or Gatekeeper is configured incorrect.
Action:
Follow these steps:
1. Make sure the H.323 Gatekeeper is running on the IP address configured in the Unified
CVP H.323 Service. Verify that the correct IP address (and not host name) of H.323
Gatekeeper is configured by using the sGK command from Voice Browser Admin.
2. Verify that the IP network connectivity between the Unified CVP H.323 Service and H.323
Gatekeeper.
Agent Desktop Gets a "Call Failed Due to Network Interruption" Error
Symptom:
Agent desktop gets an error message stating, “call failed due to network interruption.” Unified
CM trace log contains the message, “Calls are dropped in H225 with a CV=41 which shows a
‘temporary failure’.”
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
70
Chapter 2Troubleshooting Symptoms and Solutions
Transfer and Connect
Message:
Call failed due to network interruption.
Cause:
IPSecurity policies on your H.323 Service or Call Server machines can prevent connections,
thereby causing calls to fail.
Action:
View the current IPSecurity policy on the machine on which the Call Server is installed:
Select Start--> Run--> mmc-Console.
Click Add/Remove Snap-in.
Click Add.
ClickIP Security Policy Management.
Click Finish and OK.
In the left pane, click IP Security Policies on Local Machine.
Either disable the policy or edit the policy to allow the subnet from which calls are trying to
connect.
Call Back For IVR Service Treatment or Subsequent Network Transfer Fails
Symptom:
Taking a call back for IVR Service treatment or a subsequent network transfer does not work.
Message:
None.
Cause 1:
The VRU PIM that connects to the Unified CVP is not set to Type 5.
Action 1:
In ICM Configuration Manager, Network VRU Explorer, change the Type to 5.
Cause 2:
The ICM script does not include a Set node setting the Call.NetworkTransferEnabled variable
to 1.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
71
Chapter 2 Troubleshooting - Symptoms and Solutions
Transfer and Connect
Action 2:
In any script that might perform VRU Network Transfer or Takeback for IVR Service treatment,
add a Set node at the beginning of the script setting the
Call.NetworkTransferEnabled variable to 1.
Call Disconnected As Soon As Agent Answers
Symptom:
Call disconnected as soon as agent answers
Message:
N/A
Cause:
There is a codec incompatibility between the phone and the codec capabilities that Unified CVP
is sending.
Action:
Do the following:
1. Find the Region in which the phone resides. In CCMAdmin, check the Device Pool the
phone is in and the Region the Device Pool is in.
2. In VBAdmin, do a ‘ShowLocationsBasedCAC’.
–
If set to onGateway, look at the device pool (and subsequently Region) for the
originating gateway.
–
If set to offGateway, look at the device pool (and subsequently Region) for the Unified
CVP H.323 Service.
3. You now have two regions: the phone region and the originating region. In CCMAdmin
> System > Region, check what codec is to be used between those two regions. Both
caller and agent need to be able to speak that codec. If they cannot, a transcoding resource
needs to be allocated on Unified CM.
4. 4. If the originating endpoint is an IOS gateway, most probably the codec is incorrectly
configured. IOS gateways can speak both g711 and g729.
On the gateway:
–
conf t
–
voice class codec 1
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
72
Chapter 2Troubleshooting Symptoms and Solutions
Transfer and Connect
codec preference 1 g711ulaw
codec preference 2 g729r8
Note: For outside of North America, it is typically g711alaw.
–
Assign that voice-class codec to the Unified CVP voip dial-peer
5. If the originating endpoint is an IP phone (or a call initiated from Cisco Unified Outbound
Agent), codec renegotiation is not possible. Either the device region needs to be changed
or a transcoding resource needs to be allocated in Unified CM.
Call Immediately Disconnected and Caller Hears Nothing
Symptom:
Caller is immediately disconnected and does not hear anything.
Message:
None.
Cause:
There is a gateway misconfiguration on the IVR Service leg of the call.
Action:
If the call is clearly getting to the ICM script, but failing on the ‘Send to VRU’ node or on the
first Run External node if there is no explicit ‘Send to VRU’ node, then the gateway configuration
should be checked.
Agent Goes Ready, Queued Caller Never Delivered to Agent
Symptom:
Agent goes ready but caller in queue never delivered to agent
Message:
None
Cause 1:
The Interruptible box on the queuing Network VRU script is not checked.
Action 1:
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
73
Chapter 2 Troubleshooting - Symptoms and Solutions
Transfer and Connect
In ICM Network VRU Script Explorer, look at the Network VRU script that is queuing the
caller. It must be marked Interruptible in order for the ICM to be able to interrupt the script and
deliver the caller to the available agent.
Cause 2:
The agent Device Target does not have a label associated with the Unified CVP routing client.
Action 2:
In ICM Device Target explorer, look at the labels for the agent device target. The label must
be associated with the Unified CVP routing client. If is not associated with a Unified CVP
routing client, then one must be added.
Cause 3:
The NetworkTransferEnabled flag has not been set to 1 in the ICM script.
Action 3:
At the beginning of the first ICM caller entry script, and on every post-route script that may be
invoked later in the call, the NetworkTransferEnabled variable must be set to the value of 1.
Cause 4:
The gatekeeper routing is not configured properly and the Unified CVP H.323 Service cannot
find the correct Unified CM to which to send the agent transfer.
Action 4:
When the H.323 Service gets a device target label from the ICM, it must look up that label on
the Gatekeeper in order to know to what IP address to send the call. Configure the Gatekeeper
with the correct routing information to resolve this issue.
Post-Route Transfers Do Not Work
Symptom:
Post-route transfers do not work.
Message:
None.
Cause:
There is an ICM configuration error.
Action:
• Make sure the following are defined for the post-route number the agent is transferring to:
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
74
Chapter 2Troubleshooting Symptoms and Solutions
Transfer and Connect
– Dialed number plan
– dialed number
– call type
– script
• Verify that post-routing is enabled on the Unified CVP Peripheral Gateway. Open the PG
Explorer utility and check under the Peripheral tab.
• Make sure NetworkTransferEnabled variable is set to 1 in the ICM script that occurred
before the post routing script. It is important to note that the NetworkTransferEnabled variable
applies to future transfers, so it must be set in the script before the post-route. If the agent is
going to do more post-routes, it must also be set at the beginning of the current post-route
script.
• The transfer label that is returned in the post-route script must be defined in the Unified CVP
routing client.
No Ringback On Blind Transfer
Symptom:
Caller does not hear ringing once an agent performs a blind transfer call to a second agent. At
that point, the caller only hears silence.
Message:
None
Cause:
Caller is transferred to ICM Script and hears a prompt. The caller is transferred to an agent and
that agent performs a Single Step Transfer to another agent. At that point, the first agent is out
of the call, and the caller does not hear any ringing.
Action:
In CCMAdmin under under Service > ServiceParameters, select your server and the Unified
CM Service. Then under H323 Parameters, set "Send H225 UserInfo Message" to "H225 Info
for Call Progress Tone".
RONA and Router Requery Not Working as Expected
Symptom:
Ring On No Answer (RONA) and router requery are not working as expected.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
75
Chapter 2 Troubleshooting - Symptoms and Solutions
Transfer and Connect
Message:
None.
Cause:
RONA timers are not set correctly.
Action:
Check the RONA timers and reconfigure times. There are three system timers that affect RONA
behavior:
• Unified CVP - In VBAdmin, SetRNATimeout and ShowRNATimout. H.323 Service must
be restarted for a setting change to take effect.
• Agent Desktop settings Ring no Answer Time.
• Unified CM Forward No Answer Timer
Call Disconnected as Soon as Agent Answers
Symptom:
As soon as the agent answers, the call drops.
Message:
None
Cause:
There is a codec incompatibility between the phone and the codec capabilities that Unified CVP
is sending.
Action:
Do the following:
1. Find the Region in which the phone resides. In CCMAdmin, check the Device Pool the
phone is in and the Region the Device Pool is in.
2. In VBAdmin, do a ‘ShowLocationsBasedCAC’.
–
If set to on, look at the device pool and Region for the originating gateway.
–
If set to off, look at the device pool and Region for the Unified CVP H.323 Service.
3. There are two regions: the phone region and the originating region. In CCMAdmin
>System > Region, check what codec is being used between those two regions. Both
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
76
Chapter 2Troubleshooting Symptoms and Solutions
Transfer and Connect
caller and agent need to be using the same codec. If are not, then a transcoding resource
needs to be allocated on Unified CM.
4. If the originating endpoint is an IOS gateway, the codec may be incorrectly configured.
IOS gateways can be configured for both g711 and g729.
On the gateway:
–
type: conf t
–
type: voice class codec 1 codec preference 1 g711ulaw codec
preference 2 g729r8
–
Assign that voice-class codec to the Unified CVP voip dial-peer
5. If the originating endpoint is an IP phone or a call initiated from Cisco Unified Outbound
Agent, then codec renegotiation is not possible. Either the device region needs to be
changed or a transcoding resource needs to be allocated in Unified CM.
DTMF Tones not Recognized
Symptom:
Dual-Tone Multi-Frequency (DTMF) tones are not recognized.
Message:
None
Cause 1:
If using ASR, and the caller is coming from the PSTN, the gateway dial-peers may be incorrectly
configured.
Action 1:
Possible Solution: In Unified CVP, there are two VoIP dial-peers involved in the call: One to
go from the gateway to the Unified CVP H.323 Service and one to go from the Unified CVP
H.323 Service to the IVR gateway.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
77
Chapter 2 Troubleshooting - Symptoms and Solutions
Unified CM
Each of these dial-peers must have the following attributes to make DTMF be recognized
correctly:
• dial-peer voice 800 voip
• destination-pattern 8.........
• session target ipv4:10.64.228.100
• dtmf-relay rtp-nte h245-signal h245-alphanumeric
• codec g711ulaw no vad
Cause 2:
If using Scansoft ASR, the DTMF payload type is incorrectly configured in Scansoft.
Action 2:
In Scansoft config file C:\Program Files\ScanSoft\SpeechWorks OpenSpeech Media
Server\Server\config\OssServer.cfg, change the following value from 96 to 101:
server.transport.dtmfPayloadType VXIInteger 101
Unified CM
Blind Transfer Does Not Work with Type 10 VRU if Unified CM Configuration Changes Not Made
Symptom:
Blind transfer, involving Unified CCE, does not work with Unified CVP as a Type 10 VRU.
Message:
None
Cause:
Changes in configuration for Unified CM 5.0 are required.
Action:
Make the following configuration changes to Unified CM 5.0:
1. Uncheck the "Wait for Far End H.245 Terminal Capability Set" parameter in CCMAdmin
for the CVP gateway device.
2. Change the value of the CCM H323 Service Parameter called "Send H225 User Info
Message" to the value "H225 Info for Call Progress".
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
78
Chapter 2Troubleshooting Symptoms and Solutions
Unified ICME
ASP Error Occurs on Unified CM
Symptom:
An .asp error occurs on the Unified CM and it does not function normally.
Message:
None.
Cause:
If you installed a Call Server on the same machine as the Unified CM, and then you uninstalled
the Call Server, some files the Unified CM needs might have been deleted.
Action:
Reinstall Unified CM.
Unified ICME
ICM Lookup Requests Show 0
Symptom:
When reviewing the ICM Lookup Requests in the Realtime Statistics area of the Operations
Console, the value typically reports zero requests.
Message:
ICM Lookup Requests = 0
Cause:
This request is too transient, unlike that of some of the other statistics, such as Active Calls. In
a 60 second call the ICM Lookup Request may only last a few hundred milliseconds or less.
The probability of catching a real time snapshot of this event is extremely low. As a result, the
reported value for that statistic will typically be zero.
Action:
No action required. The value will show zero most of the time for the Realtime Statistic of ICM
Lookup Requests.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
79
Chapter 2 Troubleshooting - Symptoms and Solutions
Unified ICME
ICM Timeout Occurs and No Error Found
Symptom:
In a NAM/CICM environment, an ICM timeout occurs and there is no indication of an error in
the Call Server logs.
Message:
None.
Cause:
If the time to retrieve the first script request (or connect) takes longer than the NAM’s remote
ICM timeouts, then default routing occurs.
Action:
Locate and correct any network or script delay, or increase the NAM remote ICM timeouts.
Call Receives Default Route Label From Unified ICME
Symptom:
A new call arriving at the Customer Voice Portal receives a default route label from the ICM.
You do not see an Call Server error for this type of ICM timeout.
Message:
None.
Cause:
The ICM is taking a long time to handle new call requests.
Action:
Ensure there are no wait nodes in the ICM script prior to the Run VRU Script node. For more
information, see the Cisco ICM Software Scripting and Media Routing Guide.
ICM Script Fails on Run VRU Script Node
Symptom:
The ICM script fails on a Run VRU Script node and there is no indication of a failure in the
Call Server log.
Message:
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
80
Chapter 2Troubleshooting Symptoms and Solutions
Unified ICME
None.
Cause:
The Dialed Number, Network VRU Script or Network VRU might be associated with the wrong
Customer.
Action:
Use Configure ICM to ensure that the:
• Dialed Number and Network VRU Script are associated with the correct Customer.
• Customer is associated with the correct Network VRU.
Running Out of ECC Variable Space
Symptom:
Running out of ECC variable space.
Message:
None.
Cause:
IVR Service scripts are complex and require many ECC variables.
Action:
Some amount of ECC variable space can be freed up by doing the following:
• As of ICM 6.0 SR1, there are a number of Unified CVP ECC variables that are no longer
needed. You can save about 170 bytes by deleting them from your ICM configuration. They
are:
– user.num.steps.executed
– user.session.handled
– user.task.id
– user.connect.script.config
– user.connect.script.name
– user.connection.type
– user.last.redirected.address
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
81
Chapter 2 Troubleshooting - Symptoms and Solutions
Unified ICME
In addition, if you have Unified CVP ECC variables which you are not using and have set
them to a length of 1, they can now be eliminated as well, as of ICM 6.0 SR1. That may save
a few more bytes.
• You may want to switch to VXML Server scripting. All of the ECC-heavy IVR Service
speech-enabled scripting that would normally require lots of ECC variables can now be
confined to the VXML Server.
VRU Script Returns Incorrect Result
Symptom:
When prompted, the caller enters digits, but the script returns incorrect results.
Message:
None.
Cause:
One of the most likely possibilities is that this is a scripting issue. For example, setting
user.microapp.play_data = 167.8900, but when the audio is played, customer hears "one hundred
sixty seven point eight nine". The "zero zero" from the value of 167.8900 are missing. This is
because of missing double quote signs to make the passed data "167.8900" as a string. Without
these double quote signs around the value, it is treated as a number instead of a string.
Action:
Add the double quote signs around the Micro-Application value.
Caller is Restarted at Beginning of ICM Script
Symptom:
Caller experiences a situation where they are restarted at beginning of an ICM script.
Message:
None.
Cause:
The Call Restart feature is enabled in the Unified CVP H.323 Service.
Action:
There is an optional feature which can be enabled in the Unified CVP H.323 Service that restarts
the caller at the beginning of the ICM script in lieu of disconnecting the caller. Normally, without
this optional feature enabled, when some critical system error occurs, the caller hears the system
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
82
Chapter 2Troubleshooting Symptoms and Solutions
Unified ICME
error message and is disconnected. This feature is enabled by doing SetNewCallOnly on in
VBAdmin. Do a ShowNewCallOnly in VBAdmin to see if the feature is enabled.
Numeric data is not Playing in the Expected Format
Symptom:
Numeric data is not playing in the expected format.
Message:
None.
Cause:
The ICM script might be:
• Truncating values with leading 0’s or 0’s after decimals.
• Rounding values.
Action:
Put quotes around numbers in a Script Editor Set node so it is processed as a string. This is
especially important if:
• Leading 0’s are present (example: dates).
• Trailing 0’s are present after a decimal point (example: currency).
• The number is very large (example: a number normally expressed through exponential
notation).
ICM Service Control Only Displays H.323 Service After Core Software Install
Symptom:
After rebooting the Call Server, the Unified CVP H.323 Service, and Unified CVP Resource
Manager should all start automatically. The issue is that only the H.323 Service is listed in the
ICM Service Control.
Message:
None
Cause:
The ICM service control is for the legacy purposes that the Unified CVP H.323 Service carries.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
83
Chapter 2 Troubleshooting - Symptoms and Solutions
Unified ICME
Action:
The user is required to check the All checkbox in the lower left corner of the ICM Service
Control panel in order to see the Unified CVP Call Server and Unified CVP Resource Manager
services. All three are not displayed by default for the ICM Service Control list. Unified CVP
Services should be started and stopped using the Operations Console.
Error Code 31, 32, 33 Returned to ICM Script
Symptom:
CALL_RESULT returns an error code 31, 32, or 33 to the ICM Script.
Message:
Error code 31, 32, or 33 received.
Cause:
If using ASR/TTS, the gateway is unable to connect to the ASR or the TTS server.
Action:
To resolve this issue, the connection to the server must be restored.
Error Code 9 Returned to ICM Script
Symptom:
ICM script receives an error code 9.
Message:
Error code 9 received.
Cause:
The gateway or Unified CVP H.323 Service cannot connect to the media server or the prompt
does not exist.
Action:
To resolve this issue, the connection to the server must be restored.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
84
Chapter 2Troubleshooting Symptoms and Solutions
Content Services Switch
Content Services Switch
Content Services Switch Shows Call Server Out of Service but IVR Service is up
Symptom:
Content Services Switch (CSS) shows Call Server out of service but IVR Service is up.
Message:
None
Cause:
The keepalive retry period is set too low.
Action:
The CSS polls the Unified CVP Call Server with a special Unified CVP script that is manually
placed on the CSS. The keepalive retry period must be greater than the retry period specified
in the script. Make sure the CSS keepalive retry period is set to 6. Also do a show script
<script name> on the CSS to verify that the script exists.
service vxml1
keepalive maxfailure 1
keepalive retryperiod 6
ip address 10.86.129.22
keepalive type script <script name> "10.86.129.31"
active
General System Issues
Intermittent Timeout Issues
Symptom:
ICM timeouts in the CVP Call Server log.
Message:
N/A
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
85
Chapter 2 Troubleshooting - Symptoms and Solutions
General System Issues
Cause:
This may be due to your network settings not being set properly.
Action:
The chart below shows the speed and duplex settings for Unified CVP.
Ethernet Switch Speed
Capability
Server/Gateway NIC Speed Switch Port Speed/Duplex Server/Gateway NIC
Capability
Setting
Speed/Duplex Setting
1000 Mb
1000 Mb
Auto / Auto
Auto / Auto
1000 Mb
100 Mb
100 Mb / Full
100 Mb / Full
100 Mb
100 Mb
100 Mb / Full
100 Mb / Full
100 Mb
1000 Mb
100 Mb / Full
100 Mb / Full
Windows OS Application Log Fills up
Symptom:
Windows Operating System Application log fills up, and clearing the log only resolves the issue
temporarily.
Message:
None
Cause 1:
Event log settings are not set to roll over.
Action 1:
Change the Event setting so that the Application Log rolls over by doing the following:
1. Select Start > Programs > Administrative Tools > Event Viewer.
2. Right-click Application Log and select Properties from the menu.
3. Select the Overwrite events as needed option.
4. Click Apply and OK.
Cause 2:
Some unnecessary protocols might have been installed. For example, NetBEUI has been known
to fill the event log.
Action 2:
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
86
Chapter 2Troubleshooting Symptoms and Solutions
General System Issues
Uninstall protocols that are not necessary in this application.
Error Message: Operation Failed. Device Could Not Be Reached
Symptom:
Message:
"<DeviceType> device with IP Address: <IP> and Hostname: <hostname> operation failed.
Device could not be reached"
Cause:
Action:
"Java is Not a Recognized Program" Error
Symptom:
"Java is not a recognized program" error message is displayed when trying to run the .bat files
in the Unified CVP Server/admin directory ( or the .bat files in the application specific admin
directory) after a Windows upgrade.
Message:
Java is not a recognized program
Cause:
The environment variable containing the java install path was overwritten during the upgrade.
Action:
Possible Solution 1: (Reboot the machine.)
VXML Server requires the use of an environment variable that is created by the installer. In
order for this environment variable to take effect, the machine must be rebooted. If the machine
was not rebooted by the installer, reboot the machine manually. Before rebooting, close all
running application and save any unsaved work.
Possible Solution 2: (Reset the Java Path system variable.)
a. From Start, select Programs > Administrative Tools > Computer Management.
b. From the Computer Management menu bar, select Action > Properties.
c. Select Advanced Tab > Environment Variables Settings.
d. In the System Variables Path, click Edit and add the path to where the java.exe file is installed.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
87
Chapter 2 Troubleshooting - Symptoms and Solutions
General System Issues
"Password Does Not Meet the Operating System Security Requirements" Error
Symptom:
If an attempt is made to change an existing user password before the domain password policy's
minimum password age is reached, the user receives an error.
Message:
Password does not meet the operating system security requirements.
Cause:
This condition is caused by attempting to change an existing user password before the domain
password policy minimum password age has been reached.
Action:
The user password should not be changed until the minimum password age specified in the
password policy for the domain has been exceeded.
Password Lost or Forgotten
Symptom:
Password has been lost or forgotten
Message:
None.
Cause:
For some reason, a password has been lost or forgotten.
Action:
Contact your Cisco Partner or Cisco Technical Support for assistance.
Server IP Address Has Been Changed
Symptom:
The IP address of a Unified CVP server has been changed and is now listed as Not Reachable.
Message:
Not Reachable
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
88
Chapter 2Troubleshooting Symptoms and Solutions
General System Issues
Cause:
The IP address of a Unified CVP server has been changed locally instead of from the Operations
Console.
Action:
If you have already changed the IP address of the server, you may be able to delete it from the
Operations Console, but the properties files on the server will not be deleted. This is due to the
Operations Console not being able to access the server using original IP address. Adding the
server with the new IP address to the Operations Console will add new property entries, but
will not clean up the original property entries. The original property files will need to be cleaned
up using a manual process. Contact your Cisco Partner or Cisco Technical Support for assistance
when removing property files.
Below is the correct procedure for changing server IP addressed.
1. Create the new device from the Operations Console. Use the original device as template.
From the menu bar of the device, select the "Use As Template" option. Assign the new
IP address to the new device and click Save. Do not select Save and Deploy since you
have not changed the physical server to the new IP address.
2. Delete the device from the Operations Console before changing the IP address of the
server.
3. Configure the new IP address on the local server.
4. From the Operations Console, select Save and Deploy of the new device.
5. Restart the server.
Unable to Open Command Prompt Window or Other Applications
Symptom:
It fails when trying to open a command prompt window, or some other application.
Message:
None.
Cause:
There are too many Windows threads in use on the system.
Action:
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
89
Chapter 2 Troubleshooting - Symptoms and Solutions
General System Issues
If the total number of calls configured in Call Server and VBAdmin is too high, the system can
run out if threads.
• Open the Task Manager on the Call Server.
Click the Processes tab.
Click the View menu.
Click the Select Columns option.
Check the Thread Count box.
Click the Threads heading to sort by decreasing thread count. If the thread count starts
approaching 2000, then this is most likely the problem.
• In VBAdmin, the ShowMaxTotalCalls should never exceed 400.
When Hyperthreading is Enabled, Unified CVP Performance is Degraded
Symptom:
When hyperthreading is enabled, the Unified CVP performance is degraded.
Message:
None.
Cause:
When hyperthreading is enable in Windows Operating System on the Call Server, the number
of concurrent calls handled decreases by approximately ten percent.
Action:
Disable hyperthreading.
Users Cannot Log In
Symptom:
User such as Informix, cvp_dbadmin, and cvp_dbuser are unable to log in, even when using
their assigned password.
Message:
Login failed message
Cause:
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
90
Chapter 2Troubleshooting Symptoms and Solutions
General System Issues
The password for this user has expired. This can occur if the user has not logged in in the number
of days specified by the MaximumPasswordAge security policy, if the system calendar has been
set forward, or if the security policy has been modified.
Action:
Log in as an administrator and change the password for the user.
Errors Reported by Various Unified CVP Components
Symptom:
Errors reported by various Unified CVP components. ICM script fails calls from Play Media
node, and many calls abnormally disconnect.
Message:
None.
Cause:
Running Symantec virus scan software with a heavy call load (greater than 2 calls per second)
can cause these types of errors.
Action:
Disable Symantec virus scan software and use McAfee 8.0i virus scan software instead, if your
call load is heavier than 2 calls per second.
Other Areas to Check
Symptom:
General failures.
Message:
None.
Cause:
Software version incompatibilities, hotfixes not installed or installed incorrectly, firewalls,
network slowness.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
91
Chapter 2 Troubleshooting - Symptoms and Solutions
H.323 Troubleshooting
Action:
• Verify that the your software versions comply with the Customer Voice Portal compatibility
matrix in the Customer Voice Portal BOM.
• Determine if there are any firewalls in the solution. Firewalls can introduce network latencies
and misrouted messages if it is not configured correctly.
H.323 Troubleshooting
H.323 Service Process Restarts Unexpectedly
Symptom:
H.323 Service process restarts unexpectedly
Message:
None
Cause 1:
An internal memory error occurred in the H.323 Service.
Action 1:
Search the H.323 Service logs for an occurrence of the word EXCEPTION around the time of
the failure. If found, this indicates a memory corruption issue. Contact your support provider
for additional information.
Cause 2:
Another possibility is that the user accidentally hung the H.323 Service console window by
highlighting text on the screen or using the scroll bar to view the contents of the console window.
Action 2:
Check the H.323 Service logs for an occurrence of the phrase H.323 Service will be restarted
in 1800 seconds. If no EXCEPTION errors are found around that time, it indicates that the user
inadvertently hung the H.323 Service process.
H.323 Service Out of Service
Symptom:
The H.323 service is out of service. Although it is out of service, the H.323 Service allows calls
in progress to complete, but does not accept new calls.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
92
Chapter 2Troubleshooting Symptoms and Solutions
H.323 Troubleshooting
Message:
None.
Cause 1:
H.323 Service is not registered to a gatekeeper.
Action 1:
In VBAdmin, verify the Gatekeeper IP address by typing showGatekeeper.
On the Gatekeeper, type show gatekeeper endpoints
Cause 2:
There are no in-service Call Servers. The Call Server may take itself out of service when ICM
response times get too long.
Action 2:
Within the Operations Console, go to Device Management > Unified CM. Go to Engine >
Engine Configuration. Change New Call Throughput Upper Threshold and Call Event
Throughput Upper Threshold values to 30000.
Cause 3:
A user has administratively taken the H.323 Service out of service.
Action 3:
A user can administratively take the Call Server out of service by selecting Go Out of Service
from AppAdmin > Engine. To resolve this issue, start the service.
Cause 4:
The H.323 Service has gone out of service due to an internal processing error.
Action 4:
Consult your support provider for additional information regarding this type of event.
H.323 Service Log Indicates Wrong Number of Licensed IVR Service Ports
Symptom:
The H.323 Service log file indicates that the maxIVRPorts = 50, but the CVP.license file has
150 ports enabled.
Message:
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
93
Chapter 2 Troubleshooting - Symptoms and Solutions
H.323 Troubleshooting
H323CallMgr::getRegistrySettings: maxIVRPorts = 50
Cause:
It appears that the remaining ports are not being recognized. However, the Unified CVP License
file applies to the Call Server when using the SIP Protocol. The Unified CVP H.323 Service is
limited by the MaxTotalCalls registry value, so there is no correlation between the Unified CVP
VoiceBrowser MaxTotalCalls and the CVP.license file.
Action:
The maxTotalCalls registry value is based on the capacity for the Unified CVP H.323 Service
and should not be changed. In this case, you need to provision more than one Unified CVP
H.323 Service.
"H.323 Service Must be Registered With Gatekeeper to Perform IP Transfers” Error
Symptom:
Error message in the H.323 Service logs states “ERROR: H.323 Service must be registered with
Gatekeeper to perform IP transfers” during a call.
Message:
H.323 Service must be registered with Gatekeeper to perform IP transfers.
Cause:
The Unified CVP Call Server is not successfully registered with the H.323 Gatekeeper.
Action:
Set the Gatekeeper IP address from the VBAdmin using setGK <Gatekeeper IP address>.
Restart the Unified CVP Call Server after changing the H.323 gatekeeper.
“Unexpected Event for the Gatekeeper Registration” Error
Symptom:
Error Message in the H.323 Service Logs States “Unexpected event for the gatekeeper
registration. Please make sure H.323 Gatekeeper is up and running.”
Message:
Unexpected event for the gatekeeper registration. Please make sure H.323 Gatekeeper is up and
running.
Cause:
The configured H.323 Gatekeeper is not running or Gatekeeper is not configured correctly.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
94
Chapter 2Troubleshooting Symptoms and Solutions
SIP Troubleshooting
Action:
Follow these steps:
• Make sure the H.323 Gatekeeper is running on the IP address configured in the Unified CVP
Call Server. Verify that the correct IP address of H.323 Gatekeeper is configured.
• Verify that the IP network connectivity between the Unified CVP Call Server and H.323
Gatekeeper.
VBadmin Command Fails
Symptom:
An attempt to run VBAdmin from the command window fails. The browser disappears, the
window does not remain open long enough to accept a setGK command, and the following error
message appears.
Message:
"trace: PIMcom:Init - Failure connection to pipe \\.customer\voicebrowser\VBcmdpipe "
Cause:
Service Pack 1 or later of Windows 2003 is not installed on the system.
Action:
Install Windows 2003 Service Pack 1 or later.
SIP Troubleshooting
SIP to SIP Calls Do Not Play Ring Ringback Tone
Symptom:
The ringback tone is not heard when a SIP to SIP call is placed.
Message:
None.
Cause:
In high load situations, the playback of ringtone for many calls might over utilize the CPU
processing of the gateway.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
95
Chapter 2 Troubleshooting - Symptoms and Solutions
SIP Troubleshooting
Action:
In this case, it is necessary to use load balancing or configure the gateways to optimally distribute
media termination of calls.
The following are some facts regarding ringtone:
• Local ringback in SIP is performed with a 180 response process before the call is established.
• After the call has been established, ringtone is played back to the caller using an intermediate
transfer mechanism. This is accomplished by a transfer to a ringtone playback voip dialpeer
on the Gateway. Once the agent answers the call, the caller is then reinvited to the agent. At
that point, the ringtone dialog is dropped.
• Troubleshooting ringtone issues consists of configuring the dial-peer to match an incoming
voip call to the dialed number (DN) that is configured in the SIP Service for Ringtone. For
additional information for SIP configuration, refer to the Configuration and Administration
Guide for Cisco Unified Customer Voice Portal Guide.
• Ringtone playback is not the same as music on hold (MOH) on the Unified CM. For example,
when a caller is put on hold at Unified CM phone they hear music. The ringtone playback is
separate from this feature.
• Necessary items for ringtone playback:
– Ringback.wav on the Gateway.
– Ringtone service and TCL configured on the Gateway.
– Dial-peer associated with ringtone service.
– Static routes to route calls from Unified CVP to the Gateway.
SIP Calls Experience Long Delay or Fast Busy
Symptom:
During the making of a SIP call, the caller experiences a long delay or hears a fast busy condition.
Message:
None.
Cause:
If the calls go through, but there is a long delay or a fast busy, it could be the result of a Domain
Name Server (DNS) problem in the proxy.
Action:
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
96
Chapter 2Troubleshooting Symptoms and Solutions
SIP Troubleshooting
Verify that the correct nameserver and domain information was configured in the /etc/resolv.conf
file. If DNS is not being used, then remove any entries in that file.
SIP Calls Receives 500 Internal Server Error "Routing Failed" Event
Symptom:
A SIP call fails and a 500 Internal Server Error "Routing Failed" event is received.
Message:
500 Internal Server Error "Routing Failed".
Cause:
A SIP Trunk is not configured to receive or send calls to the IP Address from the source of the
SIP INVITE event.
Action:
The solution is to configure a SIP Trunk to the IP address of the sending endpoint, CUPS,
Gateway, or Unified CVP.
SIP Call Receives 489 Bad Event Message
Symptom:
A 489 Bad Event message is received from a Unified CM or Proxy Server when a SIP call is
made.
Message:
489 Bad Event
Cause:
Logs indicate 489 Bad Event messages are being received from either the Unified CM or the
Proxy Server. This event is used to indicate that the server did not understand the event package
that was specified in the Event header field. The problem is that Key Press Markup Language
(KPML) is trying to be used, and is currently not supported in Unified CVP.
Action:
The solution is to only use the inband Dual-Tone Multi-Frequency (DTMF) method setting
(RFC 2833) on the SIP Trunk that is associated with the Unified CM or the SIP Proxy Server.
This is done by selecting the last checkbox on the SIP Trunk configuration page of the Unified
CM Administration page.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
97
Chapter 2 Troubleshooting - Symptoms and Solutions
SIP Troubleshooting
Back to Back User Agent (B2BUA) Sending 503 Server Not Available
Symptom:
The Back-To-Back User Agent (B2BUA) is sending a 503 Server Not Available response before
the call is even established.
Message:
503 Server Not Available
Cause 1:
This is most likely caused by the SIP service not being in service. The ICM VRU PIM needs
to connect to the ICM Service for it to go into service. When that happens, SIP can then go into
service. If the ICM VRU PIM connection is lost, then the SIP Service goes into partial service
and rejects calls.
Action 1:
The solution is to enable the ICM Service to allow the SIP service to go into service.
Cause 2:
There is a license depletion issue. For license depletion, this typically equates to a high call rate.
When all the licenses are in use, the service goes into a partial state and rejects calls. When the
active call number drops and licenses are available again, then the SIP Service will start accepting
calls.
Action 2:
Monitor the call statistics from the Operations Console to determine if additional licenses are
needed.
REJECTED WITH 401 - Unauthorized Reason: Q.850;cause=57
Symptom:
SIP call failure.
Message:
%CVP_4_0_SIP-3-SIP_CALL_ERROR: GUID:24FCDB65-129111DB-82A20014-6944B792
- LEGID: 115289260464070 - OUT - DsSipInvitation <sip:1106@10.86.129.214;transport=tcp>;tag=11296519 REJECTED WITH 401 - Unauthorized
Reason: Q.850;cause=57
Cause:
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
98
Chapter 2Troubleshooting Symptoms and Solutions
SIP Troubleshooting
Unified CM SIP Trunk Security Profile has authentication turned on.
Action:
Uncheck digest and other authentication check boxes on the Unified CM SIP Trunk Security
Profile associated with the SIP Trunk that is being used with Unified CVP B2BUA.
MTP Resource Allocated Even Though MTP is Disabled
Symptom:
Media Transfer Protocol (MTP) Resource is allocated for each call coming into SIP trunk even
though MTP disabled is checked off on the trunk.
Message:
None.
Cause:
Failure to negotiate DTMF method occurs for the call, according to the DTMF method supported
by the endpoints involved in the call.
Action:
Set the endpoints of the call to negotiate the same DTMF method, i.e. RFC2833 for Unified
CVP calls.
Note: Out of Band (OOB) Key Press Markup Language (KPML) is not supported for the Unified
CVP 4.0 and 4.1 releases.
SIP Call Fail On SIP Trunk On Unified CM to Back to Back User Agent
Symptom:
SIP calls cannot be made out on the SIP Trunk on the Unified CM to the Back to Back User
Agent (B2BUA).
Message:
None.
Cause:
This occurs because the route pattern is not defined, in use, or has the wrong wildcards
configured. For example, a pattern of 7xxx routes any calls with a DN beginning with a 7 and
has 3 more digits after it. Using something like 7* does not work.
Action:
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
99
Chapter 2 Troubleshooting - Symptoms and Solutions
SIP Troubleshooting
The solution to this problem is to correct the route pattern on the Unified CM.
SIP Call is Cleared From Unified CVP With Code 127
Symptom:
SIP call is cleared from Unified CVP with code 127.
Message:
None.
Cause:
Call was detected as a phantom or zombie call, which may have happened as a result of a network
down scenario.
Action:
Check the server network connectivity.
Critical Error Audio is Not Heard on an Abnormal Disconnect SIP Call
Symptom:
A critical error wav file is not heard on an abnormal disconnect call. Instead the caller hears a
fast busy condition, or the call just drops.
Message:
None.
Cause:
When using REFER transfers, including the REFER used to play back critical error.wav for
abnormal disconnects, an outbound voip dial peer is necessary on the ingress gateway. This
outbound dial peer is needed because when the REFER message comes into gateway from
Unified CVP call server, it needs to match an outbound dial peer in order for the call to succeed,
or else a 503 rejection occurs if no dial peers match the REFER-TO header URI. Dial peer
destination targets must match the labels in the REFER-TO SIP URI, i.e. <error
DN>@<sip-server> and other labels that may be used in the ICM routing label.
Action:
Configure an outbound voip dial peer on the ingress gateway. For example:
dial-peer voice 1050 voip
destination-pattern 9.......
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
100
Chapter 2Troubleshooting Symptoms and Solutions
SIP Troubleshooting
voice-class codec 1
session protocol sipv2
session target sip-server
session transport tcp
dtmf-relay rtp-nte h245-signal h245-alphanumeric
no vad
SIP Calls Survivability Script Failing
Symptom:
Survivability script does not seem to work.
Message:
None.
Cause:
SIP calls from the IOS gateway are getting REFER to the ERROR DN, and critical error wav
file is heard (or not heard). SIP calls originating from gateway do not contain "--CVP" in the
remote party id header display name of the INVITE.
Action:
The pots dial peer must match the incoming called number and be associated with the
survivability service. For example:
dial-peer voice 1000 pots
service cvp-survivability
incoming called-number 8T
direct-inward-dial
482 Loop Detected Error From CUPS SIP Proxy
Symptom:
482 loop detected error is received from the CUPS SIP Proxy. Ringing is heard by caller, but
the call is never answered.
Message:
482 Loop Detected
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
101
Chapter 2 Troubleshooting - Symptoms and Solutions
SIP Troubleshooting
Cause:
When using SRV with a CUPS farm for failover, SIP URL from Unified CVP has the SRV
domain name. Proxy loops that call back to itself because it does not know about itself as the
SRV domain.
Action:
Under Service Parameters section in CUPS admin, add the domain name in the "SRV cluster
name" editbox.
SIP Call Fails With 404 Not Found Error Message
Symptom:
Sending call to Unified CM with a domain name like an SRV cluster name fails with 404 not
found rejections.
Message:
404 Not Found.
Cause:
The cluster fully qualified domain name (FQDN) is missing.
Action:
Go to System -> Enterprise Parameter -> Cluster Fully Qualified Domain Name on the Unified
CM and add the Cluster FQDN.
SIP Call With IVR Service is Terminated With Reason Code: Q.850;Cause=38
Symptom:
Any failure of the bootstrap VXML Server fetches to Call Server causes the SIP IVR Service
leg of call to be terminated by the recovery handoff TCL scripts on VXML Server Gateway
with a Q850 code of 38.
Message:
Q.850;Cause=38
Cause:
If the Unified CVP loses network connectivity, the VXML Server Gateway is not able to get
information from the IVR Service, and as a result a code 38 rejection is generated in the Gateway
logs.
Action:
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
102
Chapter 2Troubleshooting Symptoms and Solutions
SNMP Troubleshooting
remove the following lines from the network interface config in IOS:
no ip route-cache cef
no ip route-cache
no ip mroute-cache
keepalive 1800
no cdp enable
SNMP Troubleshooting
SNMP Notification: authenticationFailure
Symptom:
The SNMP log shows frequent connect and disconnect events from the Reporting Server.
Message:
Received SNMPv2c Trap:
Community: public
From: <ip address>
sysUpTime.0 = 00:09:54.02
snmpTrapOID.0 = authenticationFailure
Cause:
Missing SNMP Community Strings.
Action:
You need to manually import the previously configured community strings from the windows
SNMP Service on that machine. The procedure below explains the process.
Importing Previously Configured SNMP Community Strings (windows)
1. To see a list of the previously configured Windows SNMP V1 community strings:
–
Open Windows Services viewer.
–
Right-click SNMP Service and select Properties.
–
Select the Security tab.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
103
Chapter 2 Troubleshooting - Symptoms and Solutions
Installation Troubleshooting
This tab lists the accepted V1 community strings and the access granted for each
string.
It also lists the hosts from which SNMP packets are accepted.
Note: The accepted hosts apply to all community strings, whereas the Operations
Console gives you more granularity, allowing you to specify accepted hosts on a
community string basis.
2. Configure these community string from the Operations Console.
–
Open the Operations Console and select SNMP | V1/V2C | Community String.
–
For each community string discovered above that has not already been configured
in the Operations Console, add it via Add New.
Enter the community string exactly as it was listed in the step above.
Choose V1 as the version.
For windows community strings with permission other than Read Only, select Read
Write in the Operations Console.
Be sure to select the device that the community string was reported.
Installation Troubleshooting
InstallShield Engine Cannot Be Installed
Symptom:
During Unified CVP installation, the following error message appears:
Message:
“The InstallShield Engine(iKernel.exe) could not be installed. Ikernel.exe could not be copied
to C:\Program Files\Common Files\InstallShield\Engine\6 \Intel 32.”
Cause:
Multiple instances of the Install Program are being run.
Note: InstallShield (the install program for the Unified CVP) allows only one ikernel.exe to
be running at a time.
Action:
Stop all install programs and run the Unified CVP installation program.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
104
Chapter 2Troubleshooting Symptoms and Solutions
Installation Troubleshooting
File in Use Message When Uninstalling
Symptom:
While trying to uninstall Unified CVP, a dialog box appears saying that a file is in use and the
software cannot be uninstalled.
Message:
File is in use and the software cannot be uninstalled.
Cause:
H.323 Service, and dumplog use the files msgs2.dll, Icrmsgs.dll, cat2.dll,Icrcat.dll, and sometimes
the filter files appl2.flt, custvis2.flt, object2.flt, and primpos2.flt. If you try to uninstall while
one of the other processes is using these files, the Uninstall procedure is not able to remove
them.
Action:
Before running Uninstall, verify that:
• All Unified CVP services are stopped.
• The ICM Service Control Panel is closed.
• The dumplog utility is not running.
• All file accessing programs (such as Microsoft Word or Windows Explorer) are closed.
Error Uninstalling WebSphere Application Server
Symptom:
You encounter an error condition when you attempt to uninstall IBM WebSphere Application
Server
Message:
Several possible.
Cause:
Several possible.
Action:
See the CVP Installation log for details and instructions.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
105
Chapter 2 Troubleshooting - Symptoms and Solutions
Installation Troubleshooting
See Also
IBM WebSphere Application Server documentation and support website.
Password Issues
Symptom:
Passwords must meet complexity requirements.
Message:
N/A
Cause:
This security setting determines whether passwords must meet complexity requirements.
Action:
If this policy is enabled, passwords must meet the following minimum requirements:
• Not contain all or part of the user's account name.
• Be at least twelve characters in length.
• Contain characters from three of the following four categories:
– English uppercase characters (A through Z)
– English lowercase characters (a through z)
– Base 10 digits (0 through 9)
– Non-alphabetic characters (for example, !, $, #, %)
• Complexity requirements are enforced when passwords are changed or created.
"HTTPServer Installed Before WebSphere" Installer Message
Symptom:
The following message appears when you attempt to select WebSphere Application Server on
the Unified CVP Install program Application Server Select screen:
Message:
It appears that HTTPServer was installed before WebSphere. HTTPServer MUST be installed
after WebSphere. You will not be allowed to contine with the installation unless you choose
Tomcat.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
106
Chapter 2Troubleshooting Symptoms and Solutions
Installation Troubleshooting
Cause:
IBM HTTPServer was installed before WebSphere Application Server. WebSphere Application
Server must be installed before IBM HTTPServer.
Action:
Do the following:
• Cancel out of the install program
• Uninstall IBM HTTPServer
• Reinstall IBM HTTPServer
• Run the install program again
Reporting Password Error During Installation
Symptom:
When you specify the Reporting password during Reporting installation, you encounter the
following message:
Message:
Error 2245 while running: changing default password for user Informix
Cause:
The password security policy on the machine on which you are installing Reporting conflicts
with Unified CVP Reporting password requirements. Most likely, the machine password security
policy has a minimum password age greater than 0 or the minimum password length is greater
than 13 characters.
Action:
On the machine on which you are installing Reporting, abort the installation process. Open the
local Security Policy control panel (Control Panel > Administrative Tools > Local
SecurityPolicy) and examine the Password Policy (Security Settings > Account Policies >
PasswordPolicy). If the password policy on your machine conflicts with Unified CVP Reporting
password requirements, make the appropriate adjustments. Restart the installation process.
Dialog Box During Reporting Uninstallation
Symptom:
If you are removing the Unified CVP Reporting component, the following dialog box appears
when component removal is complete:
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
107
Chapter 2 Troubleshooting - Symptoms and Solutions
Gateway Troubleshooting
Message:
Could not delete product files and/or directories. Check documentation for more details.
Cause:
This dialog box is a present in this release solely due to a Informix product limitation, and
doesn't indicate a problem with removing Reporting.
Action:
No action necessary.
Call Server CPU spikes to 90%-100%
Symptom:
During the installation process, the Call Server CPU shows high utilization adn spikes to
90%-100%.
Message:
Error logs on the outprocess service indicating terminateConnection:Transport Down every 5-6
seconds.
Cause:
This occurs if there is an outprocess service (for example a VXML Server, REP Server or ORM)
which does not have the correct version of cvp-common.jar. This can happen, if the CallServer
or any other component is uninstalled and the CVP_HOME folder is not deleted after the
uninstall. Re-installing the component, does not delete the cvp.war file or overwrite it.
Action:
Ensure all components have the correct versions installed and the cvp-common.jar file is
consistient on all components. Re-installing the correct build on all services is the right way to
correct this problem.
Gateway Troubleshooting
Ringing Stops on IP Phone Calling TDM
Symptom:
Ringback tone is clipped when an outbound call is placed from TNP IP phones to a TDM
endpoint. For IP originated calls whose destination is a TDM endpoint, and the call goes through
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
108
Chapter 2Troubleshooting Symptoms and Solutions
Gateway Troubleshooting
Unified CVP, this shows up when the call originates from TNP phones (7971, 7970, 7961,
7941). Customer experience will be something like that given in the call flow example below:
• Dial 1234 from a TNP phone (7941, 7961, 7970, 7971)
• Call goes out a SIP trunk to Unified Cisco Voice Portal
• Unified CVP sends call to a Voice Gateway (for example, AS5400) to receive IVR treatment
• Caller hears a prompt, and so forth
• Call then gets transferred to a destination on the TDM side (for example, outbound call to
endpoint with extension 5678)
• On the IP phone side, user hears the first ring but then nothing while the destination end point
continues to ring
Message:
None.
Cause:
Appropriate setting not made to Gateway that is placing outbound call.
Action:
The following setting should be applied on the POTS dial-peer on the Gateway that is placing
the outbound call to the TDM destination:
GW(config-dial-peer)#progress_ind alert strip
Gateway Cannot Connect to ASR/TTS Server
Symptom:
Gateway cannot connect to ASR/TTS server.
Message:
None.
Cause:
There is a connectivity problem between the gateway and the ASR/TTS server or there are not
enough ASR/TTS licenses available.
Action:
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
109
Chapter 2 Troubleshooting - Symptoms and Solutions
Gateway Troubleshooting
Check that the Gateway has the correct IP addresses for the ASR and TTS servers, as well as
the ASR and TTS backup servers.
Run a series of ping tests to each of the IP addresses of the ASR and TTS servers and backup
server. Verify that there is network connectivity to each of the servers.
Check that the ASR and TTS services are running.
Check that the IVR Service configuration is correct.
Pressing Digit 9 Causes No-Match
Symptom:
In DTMF-only adapter, pressing digit 9 while using options causes no-match after 10 seconds.
Message:
None.
Cause:
This delay is caused by the following link grammar in the root document for the DTMF-only
adapter.
<link event="nomatch"> <grammar mode="dtmf"
type="application/grammar+regex">99999</grammar>
</link>
Since the options are created with modal=false, the link grammar remains active in the Gateway
DTMF-browser. On pressing 9, the Gateway starts to match this grammar and waits for the next
input instead of returning a 'no-match' immediately.
Action:
When writing a VXML Server script for a DTMF-only adapter that uses the Get Digits, Forms,
or n_Option_Menu elements, follow this procedure to avoid delay in processing the digit 9
when the script is executed.
1. In the VXML Server script, insert a hotlink to an inline grammar. When configuring the
hotlink’s grammar, the values of the digits must not match: –The value of any digit that
is expected in response to the Get Digits element. –The value of an option number used
in any n_Option_Menu element.
2. If the inline grammar in Step 1 contains more than one digit, set the interdigittimeout in
the root document properties to 5 seconds. For example:
interdigittimeout 5s
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
110
Chapter 2Troubleshooting Symptoms and Solutions
Gateway Troubleshooting
Examples:
If the VXML Server script uses 3_Option_Menu, where 1, 2, and 9 are valid options, then the
inline grammar can have the value 8, but cannot have the value 1, 2, or 9.
Similarly, if the VXML Server script uses 9_Option_Menu, where digits 1 through 9 inclusive
are valid options, then the inline grammar can have the value 99, and the interdigittimeout must
be set to 5s.
If you are using a GetDigits element and have set the follow range: min=2, max=4, then the
link grammar can be 99999, and you must set the interdigittimeout to 5 seconds.
If the script contains an Option Menu as well as GetDigits elements, you only need to set the
link grammar once.
Receiving Bootstrap Warning Error
Symptom:
When adding the bootstrap service to the gateway you get bootstrap warnings.
Message:
config-app)# service bootstrap flash:bootstrap.tcl
(config-app-param)# paramspace english language en
(config-app-param)# paramspace english index 0
(config-app-param)# paramspace english location flash
(config-app-param)# param CVPcallserver-primary 10.3.3.14
Warning: parameter CVPcallserver-primary has not been registered under
bootstrap namespace
(config-app-param)# paramspace english prefix en
(config-app-param)# param CVPcallserver-port 8000
Warning: parameter CVPcallserver-port has not been registered under
bootstrap namespace
(config-app-param)# param CVPcallserver-backup 10.3.3.14
Warning: parameter CVPcallserver-backup has not been registered under
bootstrap namespace
Cause:
Verify that the service is configured correctly.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
111
Chapter 2 Troubleshooting - Symptoms and Solutions
Gateway Troubleshooting
Action:
When this is received, it is usually a benign warning event. Verify that the service has been
configured correctly. To view the configuration, type the following command:
router># show run
The output should still show that it has been configured, as the example below shows.
service bootstrap140 flash:bootstrap.tcl
param CVPcallserver-backup 10.3.3.14
paramspace english language en
paramspace english index 0
paramspace english location flash
paramspace english prefix en
param CVPcallserver-primary 10.3.3.14
param CVP-port 8000
Switch Transfer is Not Going to the Gateway Defined in the Gatekeeper Zone Prefix Command
Symptom:
Switch Transfer is not going to the Gateway defined in the Gatekeeper zone prefix command.
Message:
None.
Cause:
The IP address of the outbound Gateway, which is registered with a DNS name, is used in the
zone prefix.
Action:
Use the same name of the outbound Gateway in zone prefix.To verify the outbound Gatekeeper
priority list, do the following:
1. Enter the show gatekeeper gw-type-prefix command in the Gatekeeper CLI.
2. Enter the show gatekeeper end command in the gatekeeper CLI and verify that H323
ID used by egress gateway/endpoint to register with gatekeeper is used in the Zone prefix
configuration of gatekeeper.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
112
Chapter 2Troubleshooting Symptoms and Solutions
Cisco Unified Presence Server
Cisco Unified Presence Server
Installation Keeps Failing in Network Connectivity Validation Page
Symptom:
During the installation of the Cisco Unified Presence Server, the installation fails at the Network
Connectivity Validation screen.
Message:
None
Cause 1:
Wrong hostname is configured in the server.
Action 1:
Make sure the CUPS hostname is configured correctly and it matches the name specified in the
Call Server page of the CM Publisher.
Cause 2:
Wrong CM Publisher hostname or password.
Action 2:
Make sure the CM Publisher's hostname and password are entered correctly. If you have DNS
server setup, make sure the CM Publisher's hostname is resolvable.
The Install Complains About Hardware Check Failure
Symptom:
During the installation of the Cisco Unified Presence Server, the installation complains about
a hardware check failure.
Message:
None
Cause:
Using non-compatible MCS server hardware.
Action:
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
113
Chapter 2 Troubleshooting - Symptoms and Solutions
Support Tools
Verify that you are using a supported server.
Call Fails When Transferred to Agent
Symptom:
A call is transferred to an agent, and the agent is reserved, but call transfer fails.
Message:
None.
Cause:
Cisco Unified Presence Server (CUPS) trunk on the Unified CM is not configured.
Action:
Configure the CUPS trunk on your Unified CM. Refer to the Configuration and Administration
Guide for Cisco Unified Customer Voice Portal for configuration information.
Support Tools
Unable to Login to Support Tools
Symptom:
Attempts to login to the Support Tools fails.
Message:
Login failed.
Cause:
The Support Tools server was not installed in a domain.
Action:
To login to the Support Tools in this case requires that the user login with the following format:
.\<username>
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
114
Chapter 2Troubleshooting Symptoms and Solutions
Backup And Restore
Backup And Restore
Reporting Server Purging Daily
Symptom:
You are seeing purge events on a daily basis on the Reporting Server
Message:
None.
Cause:
The retention times are possibly set too high.
Action:
Backup the database, then change the retention times.
To configure Reporting Database purge settings:
1. From the Operations Console, choose Device Management > CVP Reporting Server.
2. In the Reporting Server window that opens, click Edit.
3. Select the Database Administration menu in the toolbar, then select Data Purge.
4. Change the data retention time for each category of data.
5. Enter your Database Administrator Password and click Save & Deploy.
For a complete list and description of ECC variables, see the Configuration and Administration
Guide for Cisco Unified Customer Voice Portal.
Changing Backup Server Requires Restart
Symptom:
Changing backup server requires a restart.
Message:
None.
Cause:
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
115
Chapter 2 Troubleshooting - Symptoms and Solutions
Backup And Restore
Changing primary or backup Call Server updates the Call Server messageAdapter.properties
file and informs the user to restart all the affected devices.
Action:
Restart all of the affected devices.
"Retrying ASR Server. Last tried BACKUP asr-en-US-backup. primary tried:0, backup tried:1" Error
Symptom:
Missing parameters in the nuance-resources.txt file.
Message:
Retrying ASR Server. Last tried BACKUP asr-en-US-backup. primary tried:0, backup tried:1
Cause:
Missing parameters in the nuance-resources.txt file.
Action:
If you are using Nuance speech recognition to recognize DTMF digits, be sure the following
parameters are in your nuance-resources.txt file on the recognition server machine:
dtmf.SuppressNotifications=FALSE
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
116
Appendix A
Event and Cause Codes
Event codes and cause codes provide information on the status of a particular operation within
the Unified CVP environment. H.323 and SIP standard cause codes reflect the nature of the
associated issue. For each of the issues, an ITU-T Q.850 release cause code is also assigned and
maps the new standard categories with the Q.850 release cause code and description that is
assigned to each category.
IVR Service Error Codes
Table 1: IVR Service Error Codes
Error Code Name
Error Code Error Code Description
Value
ERROR_NONE
0
No error.
ERROR_HANGUP
1
N/A
ERROR_NETWORK
2
Failure of an IP network connection.
ERROR_SYSTEM
3
Unexpected failure of an Unified CVP component.
ERROR_SCRIPT_TYPE
4
N/A
5
Micro-Application name passed from the Unified ICME
to the Call Server did not exist on the Call Server.
6
VRU Script Name data passed from the ICM to the Call
Server does not contain the expected components
(Micro-Application name, media file name, media file
type, uniqueness value).
7
Data passed from the Unified ICME to the Call Server
is not consistent with what the micro-application
requires for processing.
8
An ECC variable was set to a value the Application
Server did not recognize. ECC variable definitions must
be the same in Unified ICME and Unified CVP.
ERROR_UNKNOWN_UAPP
ERROR_SCRIPT_NAME
ERROR_CONFIG_PARAM
ERROR_MISCONFIG_ECC
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
117
Event and Cause Codes
IVR Service Error Codes
Error Code Name
ERROR_MEDIA_FILE
Error Code Error Code Description
Value
9
Media file name passed from the Unified ICME to the
Call Server did not exist on the Media Server.
10
N/A
11
The VoiceXML Interpreter did not recognize a
VoicdeXML format passed from the Call Server.
12
The VoiceXML Interpreter did not recognize a
VoiceXML element passed from the Call Server.
13
The variable data passed was not valid for the script
type being processed.
ERROR_NO_VAR_DATA
14
N/A
ERROR_FORMAT
15
N/A
16
Caller was unsuccessful in entering digits during each
of the tries allowed by the Micro-Application.
17
Caller did not enter digits in response to the prompt for
each of the tries allowed by the Micro-Application.
20
N/A
21
Caller did not enter digits in response to the prompt in
the time allowed by the Micro-Application.
ERROR_CALLED_HANGUP
22
N/A
ERROR_NO_ANSWER
23
N/A
ERROR_BUSY
24
N/A
ERROR_GEN_TRANSFER
25
N/A
ERROR_INVALID_EXTN
26
N/A
ERROR_HANGUP_FORCED
27
N/A
ERROR_AFTER_TRANS_ESTAB
28
N/A
30
The VoiceXML Interpreter did not recognize the locale
passed from the Call Server.
31
TTS service was unable to perform a request due to
resource limitations.
32
TTS service was unable to perform a request due to
resource limitations. However, it may also indicate that
an unsupported locale was specified.
33
N/A
40
Returned if the Call Server is unavailable (shutdown,
network connection disabled, etc.)
41
Returned if some VXML Server-specific error occurs
(For example, java exception).
42
Returned if a hangup element is used, instead of
subdialog return element, to send a Cisco application.
43
Returned if VXML Server application is suspended.
ERROR_SEMANTIC
ERROR_VXML_FORMAT
ERROR_VXML_ELEMENT
ERROR_VARIABLE_DATA
ERROR_MAX_INVALID
ERROR_MAX_NO_ENTRY
ERROR_DATA_RANGE
ERROR_TIMED_OUT
ERROR_UNSUPPORTED_LANGUAGE
ERROR_MEDIA_RESOURCE_ASR
ERROR_MEDIA_RESOURCE_TTS
ERROR_GENERAL_ASR_TTS
ERROR_CVP_SYSTEM_UNAVAILABLE
ERROR_CVP_APP_ERROR
ERROR_CVP_APP_HANGUP
ERROR_CVP_APP_SUSPENDED
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
118
Event and Cause Codes
ICM Error Codes
Error Code Name
Error Code Error Code Description
Value
ERROR_CVP_NO_SESSION_ERROR
44
Returned when an emergency error occurs (for example,
an application is called that has not been loaded in the
VXML Server application).
45
Returned when the VXML Server encounters a bad
fetch situation. This code is returned when either a wav
file or an external grammar file is not found.
ERROR_CVP_BAD_FETCH
ICM Error Codes
Table 2: ICM Error Codes
Error Code Name
Error Code Value
Error Code Description
NO_ERROR
0
No error occurred.
INVALID_VERSION
1
The version number requested by the
Peripheral Gateway is not supported by
the VRU.
SESSION_ALREADY_ACTIVE
2
The VRU already has an active
communication session with a
Peripheral Gateway.
VRU_OFFLINE
3
The VRU is unavailable.
SESSION_NOT_ACTIVE
4
No session is currently active.
INVALID_DIALED_NUMBER
5
The dialed number specified is not
known to the ICM.
EVENTS_NOT_SUPPORTED
6
The VRU does not support the Event
Data Feed.
POLLING_NOT_SUPPORTED
7
(Obsolete)
ROUTING_NOT_SUPPORTED
8
The VRU does not support the Call
Routing Interface.
TIME_SYNCH_NOT_SUPPORTED
9
The VRU does not support the Time
Synchronization Interface.
TIMEOUT
10
A time-out has occurred.
PG_OFFLINE
11
The Peripheral Gateway is off-line.
REQUEST_REFUSED
12
The request was refused due to a
temporary condition.
ROUTING_NOT_AVAILABLE
13
The ICM routing service is not
available.
ROUTE_NOT_ACCEPTED
14
The VRU did not accept the supplied
route.
UNSPECIFIED_FAILURE
15
An unspecified error occurred.
INVALID_INVOKEID
16
An invalid InvokeID has been specified
in a result or response message.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
119
Event and Cause Codes
ICM Error Codes
Error Code Name
Error Code Value
Error Code Description
SERVICE_CTRL_NOT_SUPPORTED
17
A request was made to initialize the
Service Control Interface to a VRU that
does not support the interface.
NO_SCRIPT
18
An ICM Script has not been scheduled
for the call type which the VRU is
seeking instruction.
CALL_VARIABLE1
19
The VRU could not process a request
due to invalid value in CallVariable1.
CALL_VARIABLE2
20
The VRU could not process a request
due to invalid value in CallVariable2.
CALL_VARIABLE3
21
The VRU could not process a request
due to invalid value in CallVariable3.
CALL_VARIABLE4
22
The VRU could not process a request
due to invalid value in CallVariable4.
CALL_VARIABLE5
23
The VRU could not process a request
due to invalid value in CallVariable5.
CALL_VARIABLE6
24
The VRU could not process a request
due to invalid value in CallVariable6.
CALL_VARIABLE7
25
The VRU could not process a request
due to invalid value in CallVariable7.
CALL_VARIABLE8
26
The VRU could not process a request
due to invalid value in CallVariable8.
CALL_VARIABLE9
27
The VRU could not process a request
due to invalid value in CallVariable9.
CALL_VARIABLE10
28
The VRU could not process a request
due to invalid value in CallVariable10.
INVALID_SCRIPT
29
The Script ID specified in the
RUN_SCRIPT_REQ message was
invalid on the VRU.
INVALID_CALLID
30
The CallID specified in a Service
Control request message not valid on
the VRU.
DUPLICATE_DIALOGUEID
31
The dialogue identifier specified in
either a NEW_CALL or
REQUEST_INSTRUCTION message
was not unique.
INVALID_MESSAGE
32
This error code may be sent in response
to any message received with missing
required floating fields.
INVALID_DIALOGUEID
33
The DialogueID specified in a request
message is no longer valid.
OPERATION_CANCELLED
34
The specified operation was
successfully cancelled by a cancel
request.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
120
Event and Cause Codes
Infrastructure Error Messages
Error Code Name
Error Code Value
Error Code Description
OPERATION_NOT_CANCELLED
35
The specified operation could not be
cancelled by a cancel request.
Infrastructure Error Messages
Table 3: Infrastructure Error Messages
Title
Description
Cause
Resolution
EMERGENCY
STATE_MANAGER_
STARTUP_FAILURE
StateManager did not start.
The specific cause will be The Call Server must be
noted in the log message. restarted to resolve this issue.
STARTUP
start up failed.
The specific cause will be The Call Server must be
noted in the log message. restarted to resolve this issue.
START
No Services have been started. The Call Server has not
been setup through the
Operations Console.
SHUTDOWN
Unable to shutdown the Web
Application Server (WAS).
Login to the Operations
Console, and create a Call
Server with the IP address of
this server.
The specific cause for the Before attempting to restart
shutdown will be noted in the server, manually shut it
the log message.
down with the command:
CVP_HOME\bin\runcvp
stopservice call
If this fails, you must force
the process to stop with
Windows Task Manager.
Once it has been stopped,
then restart the server.
PORT_THRESHOLD
The Port License Checkout
Threshold is at the threshold
level.
The volume of calls on the
call server is approaching
the maximum capacity
allowed in the license.
When the call volume
decreases, a message stating
that the threshold has
returned to the safe level will
be logged. If the system is
often running at or near
licensed capacity you may
wish to contact Cisco or your
Cisco Partner to discuss the
increasing of your system
capacity.
ALERT
PORT_THRESHOLD
The Port License Checkout
The volume of calls on the No steps need to be taken.
Threshold has returned to the Call Server is no longer
safe level.
near the maximum
capacity.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
121
Event and Cause Codes
Infrastructure Error Messages
Title
Description
Cause
Resolution
SNMP_CONNECTION_
FAILURE
Could not bind serviceability There is a non-Unified
manager to <port>. Retrying in CVP process running
<time> minutes.
which is bound to the
port(s) listed.
(AIX) SNMP_
INIT_FAILURE
Could not start SNMP Agent. The specific cause will be Resolve the reason for start
noted in the log message. up failure, and then restart the
SNMP Agent. To restart the
SNMP Agent, type:
If this message is consistently
being logged, the processes
bound to the listed port(s)
should be stopped, and then
restart Unified CVP.
CVP_HOME\bin\snmp-wrapper.sh
start
at a command prompt.
Note: CVP_HOME is the
location where Unified CVP
was installed.
(AIX) SNMP_
FORWARDER_INIT
_FAILURE
Could not initialize SNMP
Forwarder.
The specific cause will be The Unified CVP application
noted in the log message. that logged this error must be
restarted to enable the SNMP
services.
CRITICAL
CAH_STARTUP _FAILURE The Common Admin Handler The server has been
will not run properly. The
misconfigured. The service
underlying service is absent. noted in the log message
failed to start.
Try restarting the server. If
this message appears during
the restart, then the server
must be removed from the
Operations Console. Once it
has been removed, you can
recreate it from the
Operations Console.
HEARTBEATS_ STOPPED Heartbeats from <service ID>
stopped. Unilaterally setting
state to PARTIAL_SERVICE.
Sending update.
If the noted service has been
manually shutdown, then this
message is normal and
expected. If it was not
manually shutdown, then
check if it has failed. If it
failed, attempt to restart it.
Verify that the server on
which the noted service
should be running has a valid
network connection to the
Call Server. Once everything
is running and connected you
will see the following
message: "STATE_
MANAGEMENT". This
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
122
The service noted in the log
message has failed or lost
connectivity to the Call
Server.
Event and Cause Codes
Infrastructure Error Messages
Title
Description
Cause
Resolution
message notes that the
"Heartbeats from <service
ID> resumed" where the
service ID's match.
LICENSE_ EXPIRATION
The <license type> license for The license on the system
<licensed utility> has expired, has expired and the system
gracefully shutting down the will shutdown gracefully.
system.
The system will not start with
an expired license. Contact
Cisco or your Cisco Partner
to obtain a new license for
your server.
ERROR
START
Could not start <service id>
due to: {reason}
The subsystem noted in the Restart the Call Server.
log message failed to start
up due to the reason noted.
SHUTDOWN
{shutdown type} shutdown of Either a forced or graceful
[{subsystem id}] failed due to: shutdown (as noted) of the
{reason}
subsystem failed due to the
reason noted.
If the Call Server successfully
shuts down, then this
message can be safely
ignored. If the Call Server
failed to shutdown (as
reported by Windows
Services) you must stop it
manually with the command:
CVP_HOME\bin\runcvp
stopservice call.
If this fails, you must force
the process to stop with
Windows Task Manager.
SECURITY
Could not configure secure
connection in Tomcat's
server.xml.
During the installation,
modification of the
server.xml file failed.
SECURITY
Key size of {size} illegal, Java During setup of the security
security policies allow
system, local security
maximum keysize of {size}. policies caused key
creation to fail.
Make sure to completely
uninstall Unified CVP,
reboot, and then re-install.
First, make sure your system
is running a valid copy of
Windows which is setup with
the correct locality. Next,
make sure you do not have a
limited security version of the
Java Runtime Environment
(JRE) on your system that is
overriding the one installed
with Unified CVP. Finally, if
this message was seen during
install, you must completely
unistall Unified CVP. Once
that has been completed,
re-install Unified CVP.
Otherwise Unified CVP
needs to be restarted.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
123
Event and Cause Codes
Infrastructure Error Messages
Title
Description
Cause
Resolution
(AIX) SERVER_SETUP
SNMPAgent Server setup
failed because {reason}.
The SNMP service failed Restart the SNMP service. At
to start due to the reason a command prompt, type:
noted in the log message.
CVP_HOME\bin\snmp-wrapper.sh
restart
CVP_HOME is the location
where Unified CVP was
installed.
WARN
DUPLICATE_ RETRIEVAL Second or more attempt at
getting the
CommonAdminHandler
Although operations will If the Operations Console
not be affected, the system operations on this server are
has been misconfigured. failing, then remove the
server from the Operations
Console. Once removed,
reconfigure it from the
Operations Console.
INVALID_MSG
Message processing aborted in
({service}) due to Error:
{error}. Message was:
{message details}.
This may be caused by
connecting together
incompatible versions of
Unified CVP.
IO_ERROR
Properties may not have been There was a problem
Make sure all properties files
{loaded/saved} due to an IO accessing a properties file in the CVP_HOME\conf
Exception.
on the disk.
directory are read/write
enabled, and are not currently
opened in an editor.
LICENSING
{license type} license already The system was started
expired on {date}.
with an expired license.
Please contact Cisco or your
Cisco Partner for a new
license.
SNMP_CONNECTION_
FAILURE
Connection to the SNMP
Subagent on port {port(s)}
failed. Retrying next port in
{time} minutes.
In Windows, the SNMP
service can be started from
the services control panel.
Select the Cisco CVP SNMP
Management service, and
click Start.
If this message is seen
consistently, the local
SNMP service may not be
running.
If you have recently upgraded
some Unified CVP
components, make sure all
necessary components have
been upgraded. Please send
this error message in it's
entirety to Cisco Technical
Support, or your Cisco
Support Partner.
In an AIX environment, the
SNMP service can be started
by typing:
CVP_HOME\bin\snmp-wrapper.sh
start
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
124
Event and Cause Codes
Infrastructure Error Messages
Title
Description
Cause
Resolution
at a command prompt.
NOTICE
SUBSYSTEM_NOT
_SHUTDOWN
Normal: Subsystem
[{subsystem id}] is in
[{operation state}] state.
During a graceful
shutdown, the externally
connected subsystem noted
in the log message that it
was still running. This is
normal.
If you wish to have all
services associated with the
Call Server that was just
shutdown, as noted in these
log messages, this can be
accomplished via the
Operations Console Control
Center.
RECEIVED_STATE _MSG StateManager: Subsystem
The subsystem noted has
[{subsystem id}] reported
transitioned into a different
change to state [{state}] due to state for the reason noted.
[{reason}].
These messages are expected
during the operation of
Unified CVP, especially
during start up and shutdown.
If subsystems are frequently
going into
PARTIAL_SERIVE due to
"dependency state unknown"
this may indicate an
over-used link between the
server and the Unified ICME
VRU PG.
STARTUP
MessageBus Started.
Expected during Call
Server start up.
No action required.
SHUTDOWN
MessageBus Stopped.
Expected during Call
Server shutdown.
No action required.
SERVLET_STARTUP
Servlet Startup Error: {error} This Web Application
No action required.
Server (WAS) initialization
error is benign and
automatically corrected.
START
All Subsystems have been
started.
Expected during Call
Server start up.
No action required.
BAD_LOG_
MESSAGE_XML
The CVPLogMessages.xml file
was corrupt or could not be
found. Default log message and
settings wll be used.
CVPLogMessages.xml file
is either missing from
CVP_HOME\conf or has
been incorrectly modified.
This will not affect the
operation of the server, but
default logging will take
the place of any customized
logging that may have been
configured.
Restore
CVPLogMessages.xml file
with a correct version from
the Operations Console.
The system has started.
Expected during Call
Server start up.
No action required.
INFO
[AUDIT]
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
125
Event and Cause Codes
SIP Codes
Title
Description
Cause
Resolution
[AUDIT]
The system has completely
shutdown.
Expected during Call
Server shutdown.
No action required.
[VERSION]
{product} version {version
Versioning information
number}({version type}) built expected during start up.
on {date}.
No action required.
CAH_STARTUP
The {subsystem id} subsystem Expected during start up.
will start the Common Admin
Handler.
No action required.
STATE_MANAGEMENT
Heartbeats from {subsystem
id} resumed. Current state:
{last known state}. Reason:
{reason}. Requesting current
state of {subsystem id}...
[AUDIT]
Log Level now set to [{level}]. The log level of the
No action required.
subsystem which logs this
message has been set to the
level noted.
(AIX) SERVER_SETUP
SNMP Agent Server setup on Expected during start up of No action required.
port {port}.
the SNMP service on AIX.
[AUDIT]
Maximum log file size: {size} Informational log messages No action required.
MB. Maximum logging
regarding the current (as of
directory size: {size} MB.
log time) logging file and
directory size, expected at
least at the beginning of
every log file.
[AUDIT]
"{subsystem id}" is at Log
Informational message
Use the Support Tools from
Level {level} with trace mask: regarding the current (as of the Operations Console to
{mask}
log time) log level and
modify log levels.
trace mask of the
subsystem noted.
STATS
{subsystem id/service} {stat
type} statistics: {comma
delineated statistics}
SNMP_CONNECTION
SNMP service has successfully Expected after start up.
connected to the local SNMP
Agent.
This message will be seen No action required.
when a subsystem that lost
it's connection to the Call
Server has reconnected.
Information mesasge with Use the Operations Console
the current (as of log time) to view statistics on demand.
statistics for the
subsystem/service noted.
No action required.
SIP Codes
SIP Codes:
SIP codes are available on the Cisco web site. To view the codes, go to the following link:
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
126
Event and Cause Codes
SIP Codes
http://www.cisco.com/univercd/cc/td/doc/product/software/ios122/122newft/122limit/122x/
122xb/122xb_2/ftmap.htm
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
127
Event and Cause Codes
SIP Codes
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
128
Appendix B
Accessing and Interpreting SNMP Data
The Unified CVP SNMP agent allows customers and partners to integrate with their existing
SNMP Network Management System (NMS) to provide instantaneous feedback on the health
of their Unified CVP system. The Call Server, VXML Server, and Reporting Server can be
configured to send SNMP traps and statistics to any standard SNMP Network Management
System. You can configure a link to the administration web page for an SNMP monitoring tool
and then access it by selecting SNMP Monitor from the Tools menu of the Operations Console.
To add a URL link to a tool:
1. Choose Tools > Configure from the main Operations Console menu bar.
The Configure Tools window opens, listing the current URL configured for each tool
listed on the Tools menu.
2. In the Enter New URL text box for the tool you want to configure, enter the URL for each
tool to launch.
The web page indicated by this URL is launched when you select the tool from the Tools
menu.
3. Click Save to save the URLs.
Once you have configured the administration web page for your NMS, you can launch the
SNMP monitoring tool from the Operations Console. The procedure below describes how to
launch the tool.
To launch an SNMP monitoring tool:
1. Choose Tools > SNMP Monitor from the main Operations Console menu.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
129
Accessing and Interpreting SNMP Data
The table below describes the available severity levels and their descriptions for Unified CVP
events. The severity level for an event can be set by editing the CVPLogMessages.xml file on
the server that generated that particular event.
Severity
Name
Description
0
Emergency
System or service unusable.
Examples:
• Service repeatedly fails to start up
• System ran out of disk space while disk space
is essential for this system to operate
• Application requires root privileges to run but
does not have them.
1
Alert
Action must be taken immediately.
Examples:
• Application is about to run out of license.
• Application is about to run out of disk space.
• Too many unauthorized access attempts detected.
• Denial of service attack is detected.
2
Critical
Critical condition. Similar to alert, but not
necessarily requiring an immediate action.
Examples:
• Received an invalid authentication request.
• Service crashed due to an error that could not be
handled, like an out of memory condition,
(provided it has a watchdog process to restart it,
it does not necessarily require immediate action).
• Unexpected code error that could not be handled.
3
Error
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
130
An error condition, which does not necessarily
impact the ability of the service to continue to
function.
Accessing and Interpreting SNMP Data
Severity
Name
Description
Examples:
• Problem parsing/processing a particular request
which does not prevent the application from
handling other request.
• Unexpected, but handled, code exception.
4
Warning
A warning about some condition, which is not
necessarily an error.
Examples:
• Lost network connection to some resource.
• Timed out waiting for a response.
5
Notice
Notifications about interesting system-level
conditions, which are not error conditions.
Examples:
• Configuration was updated (not audit level
information).
• Process has halted.
• Process is shutting down gracefully on request.
6
Informational
Informational messages are distinguished from
notification in that they provide information for
internal flows of the application or per-request
information instead of system-wide notifications.
Informational messages are used for
troubleshooting by users who are familiar with the
basic flows of the application.
Examples:
• Request received.
• Request was parsed successfully.
• Request being processed.
• Response sent back.
• Acknowledgment received.
• Detailed audit information.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
131
Accessing and Interpreting SNMP Data
Severity
Name
Description
7
Debug
Debugging messages are similar to informational
messages, but provide more detail and they require
the user to have better knowledge of system
internals. These messages are typically reserved
for very advanced users or Cisco technical support.
Examples:
• Complete details for a request packet.
• Internal state machine state changes.
• Internal profiling statistics.
• Internal events.
See Also
For additional information, refer to the Configuration and Administration Guide for Unified
Customer Voice Portal.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
132
Appendix C
Debugging
This appendix discusses various debugging facilities that are available to help you diagnosed
Unified CVP problems.
SIP IOS Debugging
The Session Initiation Protocol (SIP), is an application-layer control (signaling) protocol for
creating, modifying, and terminating sessions with one or more participants. These sessions
include Internet telephone calls, multimedia distribution, and multimedia conferences. SIP is a
simple, ASCII-based protocol that uses requests and responses to establish communication
among the various components in the network and to ultimately establish a conference between
two or more endpoints.
Users in a SIP network are identified by unique SIP addresses. A SIP address is similar to an
e-mail address and is in the format of sip:userID@gateway.com. The user ID can be either a
user name or an E.164 address. Users register with a registrar server using their assigned SIP
addresses. The registrar server provides this information to the location server upon request.
When a user initiates a call, a SIP request is sent to a SIP server (either a proxy or a redirect
server). The request includes the address of the caller (in the From header field) and the address
of the intended called party (in the To header field).
Over time, a SIP end user might move between end systems. The location of the end user can
be dynamically registered with the SIP server. The location server can use one or more protocols
(including finger, rwhois, and LDAP) to locate the end user. Because the end user can be logged
in at more than one station and because the location server can sometimes have inaccurate
information, it might return more than one address for the end user. If the request is coming
through a SIP proxy server, the proxy server tries each of the returned addresses until it locates
the end user. If the request is coming through a SIP redirect server, the redirect server forwards
all the addresses to the caller in the Contact header field of the invitation response.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
133
Debugging
H.323 IOS Debugging
SIP IOS Troubleshooting Commands
The table below lists several debug commands that are useful for troubleshooting problems with
SIP. Use the no form of the command to disable the debug output.
For example, no debug ccsip all
Debug Command
Description
debug ccsip all
Displays all Session Initiation Protocol (SIP)-related
debugging output.
debug ccsip calls
Displays all Session Initiation Protocol (SIP) Service
Provider Interface (SPI) call tracing information.
debug ccsip error
Shows all Session Initiation Protocol (SIP) Service
Provider Interface (SPI) errors.
debug ccsip events
Enables tracing of events that are specific to the service
provider interface (SPI).
debug ccsip info
Enable tracing of general information for the service
provider interface (SPI).
debug ccsip media
Enables tracing of the Session Limitation Protocol (SIP)
call media streams.
debug ccsip messages
Displays all Session Initiation Protocol (SIP) Service
Provider Interface (SPI) message tracing information.
debug ccsip preauth
Enables the diagnostic reporting of authentication,
authorization, and accounting (AAA) preauthentication
for Session Initiation Protocol (SIP) calls.
debug ccsip states
Displays all Session Initiation Protocol (SIP) Service
Provider Interface (SPI) state tracing information.
show sip status
Displays the Session Initiation Protocol (SIP) user agent
listener status.
show sip statistics
Displays the Session Initiation Protocol (SIP) user agent
statistics.
For additional information on these and other Cisco IOS commands, refer to the Cisco IOS
Debug Command Reference (http://www.cisco.com/en/US/products/sw/iosswrel/ps5187/
products_command_reference_book09186a008017cf4d.html)
Note: This section assumes that you possess basic proficiency in the use of Cisco IOS software
commands.
H.323 IOS Debugging
The Cisco Internet Operating System (IOS) is very complex, with many commands and
configuration possibilities. Many of these however have no direct relationship to Unified CVP.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
134
Debugging
H.323 IOS Debugging
There are some basic IOS configuration settings and tools that may be helpful. Below are some
of the more common and useful commands.
Debug Command
Description
show run
Allows the user with EXEC privileges to view
the running configuration. To gain EXEC
privileges, use the enable EXEC command.
For additional information on the enable EXEC
command, refer to the Cisco IOS Command
Reference document.
show ver
Allows the user to obtain the current IOS
version and specific hardware information.
show proc cpu hist
The command displays an ASCII graphical
form of the total CPU usage on the device over
a period of time.
show gatekeeper endpoints
Displays the status of all the registered
endpoints for a gatekeeper.
show proc cpu sorted
Displays detailed CPU utilization statistics
(CPU use per process) that are sorted by
percentage.
show memory statistics
Displays a view of the memory information.
copy tftp flash
Run this command to copy a file from a tftp
server into flash.
copy flash tftp
Run this command to copy files from flash to
a tftp server.
show gatekeeper gw-type-prefix
Use this command to check the priority of the
gateways registered with the gatekeeper. Check
to see if any are marked as "out-of-resources."
show standby
Displays which gatekeeper is primary and
which is standby.
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
135
Debugging
H.323 IOS Debugging
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
136
Index
Troubleshooting H.323
Index
H.323 problems....92
Troubleshooting Reporting Server
Accessing SNMP data
SNMP....129
Event Codes
Cause Codes....117
H.323 Debugging
H.323 IOS....134
Installation
Symptoms....104
Interpreting SNMP data
SNMP NMS....129
Overview
Troubleshooting overview....5
SIP Debugging
Reporting Server....43
Troubleshooting Restores
Restore problems....115
Troubleshooting SIP
SIP problems....95
Troubleshooting Support Tools
Support Tools problems....114
Troubleshooting Transfer
Transfer problems....65
Troubleshooting Unified ICME
Unified ICME problems....79
Troubleshooting VXML
VXML Server....28
SIP IOS....133
Symptoms and Solutions
Troubleshooting Solutions....15
Troubleshooting Audio
Audio Problems....48
Troubleshooting Backups
Backup problems....115
Troubleshooting Call Server
Call Server....15
Troubleshooting Checklist
Checklist....12
Troubleshooting Connect
Connect problems....65
Troubleshooting CUPS
CUPS problems....113
Troubleshooting Gateway
Gateway problems....108
Troubleshooting general system
General system problems....85
Troubleshooting Guide for Cisco Unified Customer Voice Portal 4.1(1)
Index 137