IPTT
IP Telephony
Troubleshooting
Student Guide
Version 4.0
Copyright © 2004, Cisco Systems, Inc.
IP Telephony Troubleshooting (IPTT) v4.0
i
Copyright
2004, Cisco Systems, Inc. All rights reserved.
Cisco Systems has more than 200 offices in the following countries and regions. Addresses, phone numbers, and fax
numbers are listed on the Cisco Web site at www.cisco.com/go/offices.
Argentina • Australia • Austria • Belgium • Brazil • Bulgaria • Canada • Chile • China PRC • Colombia • Costa Rica
Croatia • Cyprus • Czech Republic • Denmark • Dubai, UAE • Finland • France • Germany • Greece
Hong Kong SAR • Hungary • India • Indonesia • Ireland • Israel • Italy • Japan • Korea • Luxembourg • Malaysia
Mexico • The Netherlands • New Zealand • Norway • Peru • Philippines • Poland • Portugal • Puerto Rico • Romania
Russia • Saudi Arabia • Scotland • Singapore • Slovakia • Slovenia • South Africa • Spain • Sweden • Switzerland
Taiwan • Thailand • Turkey Ukraine • United Kingdom • United States • Venezuela • Vietnam • Zimbabwe
Copyright 2004 Cisco Systems, Inc. All rights reserved. CCIP, CCSP, the Cisco Arrow logo, the Cisco
Powered Network mark, Cisco Unity, Follow Me Browsing, FormShare, and StackWise are trademarks of
Cisco Systems, Inc.; Changing the Way We Work, Live, Play, and Learn, and iQuick Study are service marks of
Cisco Systems, Inc.; and Aironet, ASIST, BPX, Catalyst, CCDA, CCDP, CCIE, CCNA, CCNP, Cisco, the Cisco
Certified Internetwork Expert logo, Cisco IOS, the Cisco IOS logo, Cisco Press, Cisco Systems, Cisco Systems
Capital, the Cisco Systems logo, Empowering the Internet Generation, Enterprise/Solver, EtherChannel,
EtherSwitch, Fast Step, GigaStack, Internet Quotient, IOS, IP/TV, iQ Expertise, iQ logo, the iQ Net Readiness
Scorecard, LightStream, Linksys, MGX, MICA, the Networkers logo, Networking Academy, Network Registrar,
Packet, PIX, Post-Routing, Pre-Routing, RateMUX, Registrar, ScriptShare, SlideCast, SMARTnet, StrataView Plus,
Stratm, SwitchProbe, TeleRouter, The Fastest Way to Increase Your Internet Quotient, TransPath, and VCO are
registered trademarks of Cisco Systems, Inc. and/or its affiliates in the United States and certain other countries.
All other trademarks mentioned in this document or Website are the property of their respective owners. The use of
the word partner does not imply a partnership relationship between Cisco and any other company. (0402R)
Copyright © 2004, Cisco Systems, Inc.
IP Telephony Troubleshooting (IPTT) v4.0
ii
Table of Contents
Volume 1
Course Introduction
Overview
Course Objectives
Course Outline
IPTT Cluster
Cisco Certification Track
Learner Skills and Knowledge
Learner Responsibilities
General Administration
Course Flow Diagram
Icons and Symbols
Sources of Information
Learner Introductions
Applying Troubleshooting Methods
Overview
Module Objectives
Module Outline
Introducing IP Telephony Troubleshooting
Overview
Relevance
Objectives
Learner Skills and Knowledge
Outline
Voice Networks
Cisco CallManager
Cisco Unity
Network Infrastructure
Voice Clients
Summary
References
Quiz
Quiz Answer Key
Troubleshooting IP Telephony Networks
Overview
Relevance
Objectives
Learner Skills and Knowledge
Outline
Network Failure Preparation
Systematic Troubleshooting Method
Example: Network Problem
Define the Problem
Gather Facts
Consider Possibilities
Create Action Plan
Implement Action Plan
Observe Results
Restart the Problem-Solving Process
Document Results
Summary
References
Quiz
Quiz Answer Key
1
1
2
6
7
8
10
11
12
13
14
15
16
1-1
1-1
1-1
1-2
1-3
1-3
1-3
1-3
1-3
1-4
1-5
1-6
1-7
1-8
1-9
1-10
1-10
1-11
1-12
1-13
1-13
1-13
1-14
1-14
1-14
1-15
1-16
1-18
1-19
1-21
1-23
1-25
1-27
1-28
1-29
1-30
1-31
1-31
1-32
1-33
Troubleshooting Cisco CallManager, Network Signaling, and Dial Plan
Overview
Module Objectives
Module Outline
2-1
2-2
2-2
Troubleshooting Cisco CallManager Signaling Architecture
2-3
Overview
Relevance
Objectives
Learner Skills and Knowledge
Outline
Cisco CallManager Signaling Overview
H.323 Protocol Architecture
MGCP Protocol Functionality
Session Initiation Protocol Architecture
ISDN and Q.931
Station Definition and Call Flow
Call Preservation
Cisco CallManager Trace and Trace Collection Tool
Summary
References
Quiz
Lesson Review Answer Key
2-3
2-3
2-4
2-4
2-4
2-5
2-6
2-12
2-19
2-25
2-28
2-32
2-34
2-55
2-55
2-56
2-57
Troubleshooting Cisco CallManager Dial Plan, Call Routing, and Media Resources
Overview
Relevance
Objectives
Learner Skills and Knowledge
Outline
Cisco CallManager Overview
Partitions and Calling Search Spaces
Troubleshooting Route Patterns
Translation pattern issues
Dial plan analysis
Distributed Call-Processing Model
Centralized Call-Processing Model
Troubleshooting Media Resources
Summary
References
Quiz
Quiz Answer Key
ii
Cisco IP Telephony Troubleshooting (IPTT) v4.0
2-59
2-59
2-59
2-60
2-60
2-60
2-61
2-73
2-76
2-84
2-84
2-86
2-91
2-93
2-100
2-100
2-101
2-102
Troubleshooting Cisco CallManager CTI Manager, JTAPI, and TSP
Overview
Relevance
Objectives
Learner Skills and Knowledge
Outline
Computer Telephony Integration
CTI Route Points and CTI Ports
Cisco CallManager JTAPI and TSP
Troubleshooting Cisco CallManager CTI Manager
Troubleshooting Cisco CallManager JTAPI and TSP
Summary
References
Quiz
Quiz Answer Key
2-1
2-103
2-103
2-103
2-103
2-103
2-104
2-105
2-116
2-117
2-118
2-119
2-123
2-123
2-124
2-125
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Cisco CallManager Upgrades
2-127
Overview
Relevance
Objectives
Learner Skills and Knowledge
Outline
Cisco CallManager Upgrades
Upgrade Restrictions and Procedures
Cisco CallManager Upgrade Assistant Utility
Issues That Can Arise from Upgrades
Resolving Upgrade Issues
Summary
References
Quiz
Quiz Answer Key
2-127
2-127
2-127
2-127
2-128
2-129
2-131
2-136
2-138
2-141
2-142
2-142
2-143
2-144
Cisco AVVID Troubleshooting Tools
3-1
Overview
Module Objectives
Module Outline
3-1
3-2
3-2
Applying Cisco CallManager and Operating System Troubleshooting Tools
Overview
Relevance
Objectives
Learner Skills and Knowledge
Outline
Cisco CallManager Component Versions
Administrative Serviceability Tool
Alarm Configuration
Real-Time Monitoring Tool
Microsoft Windows 2000 Event Viewer
Microsoft Performance Monitor
Accounts and Passwords
Summary
References
Quiz
Quiz Answer Key
3-3
3-3
3-4
3-4
3-4
3-5
3-7
3-10
3-13
3-18
3-20
3-21
3-23
3-23
3-24
3-25
Using Database Tools
3-27
Overview
Relevance
Objectives
Learner Skills and Knowledge
Outline
Introduction to Microsoft SQL Server 2000
Microsoft SQL Enterprise Manager
Recreating Subscriber Connections
Microsoft SQL Query Analyzer
Microsoft SQL Command-Line Utilities
Cisco AVVID Directory Service Utilities
Summary
References
Quiz
Quiz Answer Key
Copyright
2004, Cisco Systems, Inc.
3-3
3-27
3-27
3-27
3-27
3-28
3-29
3-31
3-34
3-36
3-37
3-39
3-46
3-46
3-47
3-48
Cisco IP Telephony Troubleshooting (IPTT) v4.0
iii
Troubleshooting Using Other Tools
Overview
Relevance
Objectives
Learner Skills and Knowledge
Outline
Cisco CallManager Q.931 Translator
Q.931/H.225 Translator X
Cisco CallManager DBLHelper
Network (Protocol) Analyzers
Cisco Dialed Number Analyzer Tool
Summary
References
Quiz
Lesson Review Answer Key
iv
Cisco IP Telephony Troubleshooting (IPTT) v4.0
3-49
3-49
3-49
3-49
3-49
3-50
3-51
3-54
3-57
3-60
3-61
3-72
3-72
3-73
3-74
Copyright © 2004, Cisco Systems, Inc.
Table of Contents
Volume 2
Troubleshooting Network Infrastructure
Overview
Module Objectives
Module Outline
Troubleshooting Data Network Infrastructure
Overview
Relevance
Objectives
Learner Skills and Knowledge
Outline
Impact of Data Network
Switching Configuration
Auxiliary VLANs
Routing Configuration
Multicast Routing
Summary
References
Quiz
Quiz Answer Key
Troubleshooting Gateways
4-1
4-1
4-2
4-2
4-3
4-3
4-3
4-3
4-4
4-4
4-5
4-6
4-13
4-17
4-27
4-30
4-30
4-31
4-32
4-33
Overview
Relevance
Objectives
Learner Skills and Knowledge
Outline
Troubleshooting Common Gateway Issues
Troubleshooting Common Digital Voice Connections
Signaling Protocol Configuration
Dial Peers and Digit Analysis
Router Call Flow and Call Control
Survival Remote Site Telephony and Monitoring
Summary
References
Quiz
Lesson Review Answer Key
4-33
4-33
4-33
4-33
4-34
4-35
4-56
4-64
4-67
4-75
4-85
4-105
4-106
4-107
4-108
Troubleshooting Gatekeepers and Cisco SIP Trunks
4-109
Overview
Relevance
Objectives
Learner Skills and Knowledge
Outline
Gatekeeper Overview
Troubleshooting Gatekeepers with Gateways and Cisco CallManager
Troubleshooting Gatekeeper Endpoints
Gatekeeper and Cisco CallManager Call Processing
Troubleshooting Cisco SIP Trunks
Summary
References
Quiz
Quiz Answer Key
Troubleshooting Voice Quality Issues
Overview
Module Objectives
Module Outline
4-109
4-109
4-109
4-109
4-110
4-111
4-120
4-122
4-127
4-129
4-142
4-143
4-144
4-147
5-1
5-1
5-2
5-2
Resolving Voice QoS Issues with Cisco QoS Tools
Overview
Relevance
Objectives
Learner Skills and Knowledge
Outline
Quality of Service Defined
Converged Networks
Converged Networks: Quality Issues
Lack of Bandwidth
End-to-End Delay
Example: Effects of Delay
Packet Loss
QoS Requirements
QoS Policy
QoS for Converged Networks
Example: Traffic Classification
Example: Defining QoS Policies
LAN QoS Considerations
Summary
References
Quiz
Quiz Answer Key
5-3
5-3
5-3
5-3
5-4
5-5
5-6
5-8
5-10
5-12
5-13
5-16
5-18
5-21
5-23
5-24
5-25
5-26
5-28
5-29
5-30
5-31
Troubleshooting Voice over IP Quality Problems
5-33
Overview
Relevance
Objectives
Learner Skills and Knowledge
Outline
Identifying and Isolating Voice Quality Problems
Example
Quality Report Tool for IP Phones
Troubleshooting Layer 2 Voice Quality Problems
Troubleshooting Layer 3 Voice Quality Problems
Additional Considerations
Summary
References
Quiz
Quiz Answer Key
5-33
5-33
5-33
5-33
5-34
5-35
5-35
5-41
5-45
5-48
5-51
5-56
5-56
5-57
5-58
Troubleshooting Echo
Overview
Relevance
Objectives
Learner Skills and Knowledge
Outline
Defining Echo in an IP Telephony Network
Sources and Types of Echo
Defining the Echo Canceller
Measuring Echo in an IP Telephony Network
Adjusting Echo in an IP Telephony Network
Summary
References
Quiz
Quiz Answer Key
ii
5-3
Cisco IP Telephony Troubleshooting (IPTT) v4.0
5-59
5-59
5-59
5-59
5-59
5-60
5-61
5-64
5-67
5-69
5-77
5-79
5-79
5-80
5-81
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Cisco Unity
6-1
Overview
Module Objectives
Module Outline
6-1
6-1
6-2
Troubleshooting Common Cisco Unity Configuration, Integration,
and Operation Issues
Overview
Relevance
Objectives
Learner Skills and Knowledge
Outline
Troubleshooting Steps
Event Notification Utility
Cisco Unity Administration Tools
Cisco Unity Troubleshooting Tools
Call Transfer Problems
Port Problems
System Programming Problems
Error Messages and Disaster Recovery
Message Problems
Message Waiting Indicators
Summary
References
Quiz
Quiz Answer Key
6-3
6-3
6-4
6-4
6-5
6-6
6-8
6-10
6-15
6-18
6-21
6-24
6-25
6-28
6-32
6-41
6-42
6-43
6-45
Escalating Cisco TAC Service Requests
7-1
Overview
Module Objectives
Module Outline
7-1
7-1
7-2
Using the Cisco TAC
7-3
Overview
Relevance
Objectives
Learner Skills and Knowledge
Outline
Cisco TAC Overview
Cisco TAC Home Page
Bug Toolkit
Saved Searches
Bug Groups
E-Mail Notification Rules
My Stuff Tab
Escalation Process
Severity Levels
Summary
References
Quiz
Quiz Answer Key
Copyright
2004, Cisco Systems, Inc.
6-3
7-3
7-3
7-3
7-3
7-4
7-5
7-6
7-10
7-14
7-15
7-15
7-15
7-16
7-17
7-18
7-18
7-19
7-20
Cisco IP Telephony Troubleshooting (IPTT) v4.0
iii
Cisco TAC Service Requests and Telephone Service Providers
Overview
Relevance
Objectives
Learner Skills and Knowledge
Outline
Opening a Service Request On Line
Cisco TAC Telephone Numbers
Cisco TAC Service Request Querying Options
Contacting a Cisco TAC Center
Escalating a Case
Keeping a Case Open
Documenting the Resolution
Escalation Summary
Summary
References
Quiz
Quiz Answer Key
iv
Cisco IP Telephony Troubleshooting (IPTT) v4.0
7-21
7-21
7-21
7-21
7-22
7-22
7-23
7-24
7-25
7-26
7-28
7-29
7-30
7-31
7-33
7-33
7-34
7-35
Copyright © 2004, Cisco Systems, Inc.
IPTT
Course Introduction
Overview
Cisco IP Telephony Troubleshooting (IPTT) v4.0 equips systems engineers with the knowledge
and skills required to troubleshoot enterprise Cisco CallManager, Cisco Unity, and IP network
deployments. IPTT is one of several courses in a curriculum that addresses design and planning
practices and provides practical experience in configuring, deploying, and troubleshooting
Cisco Architecture for Voice, Video and Integrated Data (AVVID) solutions.
This five-day course presents the technical issues of troubleshooting IP telephony networks.
Both sales and technical personnel will benefit from learning the Cisco methodology for
implementing IP telephony networks.
Major topics covered include troubleshooting components connected to both the LAN and the
WAN, including aspects of Cisco CallManager, Cisco Unity, and Cisco AVVID; the IP
telephony troubleshooting cluster; and IP telephony specialization.
Course Objectives
This topic lists the course objectives.
Course Objectives
Upon completing this course, you will be able to:
• Troubleshoot any part of the Cisco AVVID model
• Describe the steps associated with an effective approach to
troubleshooting problems
• Identify correct and incorrect flows associated with the following
protocols:
– Skinny Station Protocol
– H.323
– Q.931
– Media Gateway Control Protocol
– SIP
– IP Phone registration (DHCP and TFTP)
• Identify and resolve problems associated with database
synchronization and data exchange between the Microsoft SQL
publisher and its subscribers
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—5
Upon completing this course, you will be able to meet these objectives:
Troubleshoot any part of the Cisco AVVID model
Describe the steps associated with an effective approach to troubleshooting problems
Identify correct and incorrect flows associated with the following protocols:
—
Skinny Station Protocol
—
H.323
—
Q.931
—
Media Gateway Control Protocol (MGCP)
—
Session initiation protocol (SIP)
—
Phone registration DHCP and TFTP
Identify and resolve problems associated with database synchronization and data exchange
between the Microsoft SQL publisher and its subscribers
2
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Course Objectives (Cont.)
Upon completing this course, you will be able to:
• Identify inconsistencies and potential problem areas when given a set of
requirements and an associated dial plan
• State when each of the following Cisco CallManager tools and service aids should
be used and describe the expected output of each:
– System diagnostic interface or signal distribution layer traces or both
– Microsoft Performance monitor counters
– Administrative Serviceability Tool (AST)
– Event Viewer
– Q.931 Translator/Translator X/XLog
– Protocol Analyzer/Ethereal
– Administration Utility
– Quality Report Tool (QRT)
• Use the Cisco CallManager 4.0 Trace tool to verify related issues
• Use the output from these tools and service aids to identify the source of IP
telephony-related problems when given a problem description
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—6
Identify inconsistencies and potential problem areas when given a set of requirements and
an associated dial plan
State when each of the following Cisco CallManager tools and service aids should be used
and describe the expected output of each:
—
System diagnostic interface or signal distribution layer traces or both
—
Microsoft Performance Monitor counters (This performance monitor is a tool that
resides under the Administrator Tools web page on the server.)
—
Administrative Serviceability Tool (AST)
—
Event Viewer
—
Q.931 Translator/Q.931 Translator X/Triple Combo
—
Protocol analyzer/Ethereal
—
Administration utility
—
Quality Report Tool (QRT)
Use the Cisco CallManager Trace Collection tool to verify protocols such as Skinny Station
Protocol, H.323, Q.931, MGCP, SIP, DCHP, and TFTP
Use the output from these tools and service aids to identify the source of IP telephonyrelated problems when given a problem description
Copyright © 2004, Cisco Systems, Inc.
Course Introduction
3
Course Objectives (Cont.)
Upon completing this course, you will be able to:
• Identify and troubleshoot media resource issues in Cisco
CallManager
• Identify and troubleshoot Cisco CallManager CTI Manager
and TSP-related issues
• Identify the source of IP telephony-related problems when
given a problem description and the output from Call Detail
Record or Call Management Record files
• List router and switch configuration parameters that can
affect IP telephony connectivity or voice quality
• Run and interpret the output of router and switch
serviceability-related commands, such as show or debug
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—7
Identify and troubleshoot media resources, CTI Manager and TSP related issues
Identify the source of IP telephony-related problems when given a problem description and
the output from Call Detail Record (CDR) or Call Management Record (CMR) files
List router and switch configuration parameters that can affect IP telephony connectivity or
voice quality
Run and interpret the output of router and switch serviceability-related commands, such as
show or debug
4
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Course Objectives (Cont.)
Upon completing this course, you will be
able to:
• Analyze bandwidth utilization when given an
existing Cisco CallManager installation and
identify the appropriate bandwidth management
technology—such as advanced queuing,
compression, or packet prioritization—that could
be used to eliminate voice quality issues
• Use the appropriate tools and service aids to
identify the source of the problem when given a
problem description associated with a Cisco Unity
voice-mail feature
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—8
Analyze bandwidth utilization when given an existing Cisco CallManager installation and
identify the appropriate bandwidth management technology—such as advanced queuing,
compression, or packet prioritization—that could be used to eliminate voice quality issues
Use the appropriate tools and service aids to identify the source of the problem when given
a problem description associated with a Cisco Unity voice-mail feature
Copyright © 2004, Cisco Systems, Inc.
Course Introduction
5
Course Outline
The outline lists the modules included in this course.
Course Outline
• Applying Troubleshooting Methods
• Troubleshooting Cisco CallManager, Network
Signaling, and Dial Plan
• Cisco AVVID Troubleshooting Tools
• Troubleshooting Network Infrastructure
• Troubleshooting Voice Quality Issues
• Troubleshooting Cisco Unity
• Escalating Cisco TAC Service Requests
© 2004 Cisco Systems, Inc. All rights reserved.
6
Cisco IP Telephony Troubleshooting (IPTT) v4.0
IPTT v4.0—4
Copyright © 2004, Cisco Systems, Inc.
IPTT Cluster
This figure shows a high-level overview of a Cisco AVVID network that you should be able to
build and troubleshoot by the end of this course.
IPTT Cluster
RNO
PRI
PC
Desktop
PC
Desktop
PSTN
Publisher
Subscriber
PC
Desktop
FXS
Cisco
Unity
PC
Desktop
PC
Desktop
FR
Console
Cisco
SoftPhone 2
x1002
Cisco
SoftPhoneRNO
x6001
FXO
Cisco
SoftPhone 1
x1001
Console
7960 x1010 on Cisco
SoftPhone 1
PC
Desktop
7960 x6110
on Cisco
SoftPhone 1
Voice
Modem
x1401
PC
Desktop
PC
Desktop
Cisco
SoftPhoneHQ
x1501
PC
Desktop
DFW-HQ
DFW
Cisco
SoftPhoneBR
x1501
DFWBRANCH
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—9
This course will teach you how to troubleshoot Cisco CallManager and other IP telephony
components in a LAN and WAN environment, including how to accomplish these tasks:
Troubleshoot Cisco CallManager basic operations
Troubleshoot potential dial plan configurations
Use Microsoft Performance Monitor with specific counters to aid in troubleshooting
Use the Event Viewer for troubleshooting
Use traces to view potential problems
Use Bug Tracker
Use Q.931 traces
Use debug and show commands on Cisco IOS software devices
Troubleshoot quality of service (QoS) methods
Troubleshoot a Cisco Unity 3.0 voice-mail system
Copyright © 2004, Cisco Systems, Inc.
Course Introduction
7
Cisco Certification Track
This topic lists the certification requirements of this course.
CCIE Voice
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—10
Cisco requires the IPTT course for specialization as an IP Telephony Operations Specialist.
Cisco CCNA® certification is required for the IP Telephony Operations Specialist.
Note
Cisco CCNP® certification supersedes the CCNA requirement.
The required exams are as follows:
—
Cisco QoS
—
Cisco IP Telephony Troubleshooting
The required courses are as follows:
8
—
Implementing Cisco Quality of Service (QOS)
—
Cisco IP Telephony (CIPT)
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
IP Telephony Specialization
Cisco IP Telephony Design Specialist
Cisco IP Telephony Operations Specialist
IP Telephony Design Specialist Prerequisites
IP Telephony Operations Specialist Prerequisites
Valid CCDA Certification
Valid CCNA Certification
IP Telephony Support Specialist Exams and Recommended
ALERT
Training (Register for Exams)
IP Telephony Operations Specialist Exams and Recommended
ALERT
Training (Register for Exams)
Exam
Recommended Training
Exam
Recommended Training
9E0-xxx IPTD
IP Telephony Design (IPTD)
9E0-421 IPTT
IP Telephony Troubleshooting (IPTT)
Implementing Cisco Quality of Service (QOS)
642-642 QOS.
Implementing Cisco Quality of Service (QOS)
642-642 QOS.
Cisco IP Telephony Support Specialist
IP Telephony Support Specialist Prerequisites
Valid CCNP Certification
IP Telephony Support Specialist Exams and Recommended
ALERT
Training (Register for Exams)
Exam
Recommended Training
9E0-402 CIPT
Cisco IP Telephony (CIPT)
9E0-423 CVOICE
642-642 QOS.
© 2004 Cisco Systems, Inc. All rights reserved.
Note
Cisco Voice over IP (CVOICE)
Implementing Cisco Quality of Service (QOS)
IPTT v4.0—11
For more information regarding the Cisco IP Telephony Specialization program, go to
http://www.cisco.com/en/US/learning/le3/le2/le41/le79/learning_certification_type_home.htm
Copyright © 2004, Cisco Systems, Inc.
Course Introduction
9
Learner Skills and Knowledge
This topic lists the course prerequisites.
Prerequisite Learner Skills
and Knowledge
CIPT or Cisco
AVVID Bootcamp
QOS
CVOICE
ICND
Cisco IP Telephony
Troubleshooting
BCMSN
CUSE
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—12
To benefit fully from this course, you must have these prerequisite skills and knowledge:
Cisco IP Telephony (CIPT) or Cisco AVVID Bootcamp
Implementing Cisco Quality of Service (QOS)
Cisco Voice over IP (CVOICE)
Interconnecting Cisco Network Devices (ICND)
Building Cisco Multilayer Switched Networks (BCMSN)
Cisco Unity System Engineer (CUSE)
10
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Learner Responsibilities
This topic discusses the responsibilities of the learners.
Learner Responsibilities
• Complete
prerequisites
• Introduce
yourself
• Ask questions
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—13
To take full advantage of the information presented in this course, you must have completed the
prerequisite requirements.
In class, you are expected to participate in all lesson exercises and assessments.
In addition, you are encouraged to ask any questions relevant to the course materials.
If you have pertinent information or questions concerning future Cisco product releases and
product features, please discuss these topics during breaks or after class. The instructor will
answer your questions or direct you to an appropriate information source.
Copyright © 2004, Cisco Systems, Inc.
Course Introduction
11
General Administration
This topic lists the administrative issues for the course.
General Administration
Class-Related
• Sign-in sheet
• Course Materials
• Length and times
• Attire
Facilities-Related
• Site emergency
procedures
• Rest rooms
• Telephones/faxes
• Break and lunchroom
locations
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—14
The instructor will discuss the administrative issues noted here so you know exactly what to
expect from the class.
Sign-in process
Starting and anticipated ending times of each class day
Class breaks and lunch facilities
Appropriate attire during class
Materials you can expect to receive during class
What to do in the event of an emergency
Location of the rest rooms
How to send and receive telephone and fax messages
12
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Course Flow Diagram
This topic covers the suggested flow of the course materials.
Course Flow Diagram
A
M
Day 1
Day 2
Day 3
Day 4
Day 5
Course
Introduction
Troubleshooting
CallManager,
Network Signaling,
and Dial Plan
Cisco AVVID
Troubleshooting
Tools
Troubleshooting
Network
Infrastructure
Troubleshooting
Cisco Unity
Troubleshooting
Voice
Quality Issues
Escalating
TAC Service
Requests
Applying
Troubleshooting
Methods
Lunch
P
M
Troubleshooting
CallManager,
Network
Signaling, and
Dial Plan
© 2004 Cisco Systems, Inc. All rights reserved.
Cisco AVVID
Troubleshooting
Tools
Troubleshooting
Network
Infrastructure
IPTT v4.0—15
The schedule reflects the recommended structure for this course. This structure allows enough
time for the instructor to present the course information and for you to work through the
laboratory exercises. The exact timing of the subject materials and labs depends on the pace of
your specific class.
Copyright © 2004, Cisco Systems, Inc.
Course Introduction
13
Icons and Symbols
This topic shows the Cisco icons and symbols used in this course.
Cisco Icons and Symbols
PC
Network
Cloud
File
Server
Web
Server
Database
Web
Browser
Switches
VoiceEnabled
Router
IP Phone
Phones
Cisco
CallManager
Multiswitch
Device
WAN
Softswitch
Bridge
© 2004 Cisco Systems, Inc. All rights reserved.
14
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Router
IPTT v4.0—16
Copyright © 2004, Cisco Systems, Inc.
Sources of Information
This topic shows supplemental resources.
Sources of Information
Cisco IP telephony solution and network design guides
• http://www.cisco.com/univercd/cc/td/doc/product/voice/ip_tele/i
ndex.htm
Cisco Unity Forum
• http://www.answermonkey.com
Cisco Unity documentation
• http://www.cisco.com/univercd/cc/td/doc/product/voice/c_unity/
index.htm
Cisco Press
• http://www.ciscopress.com/
• Networking Technology > Convergence/Voice/IP Telephony
• See various Cisco Press books
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—17
Most of the information presented in this course can be found on Cisco.com or on CD-ROM.
These supporting materials are available in HTML format, and as manuals in portable data
format (PDF) and release notes.
Copyright © 2004, Cisco Systems, Inc.
Course Introduction
15
Learner Introductions
This is the point in the course where you introduce yourself.
Learner Introductions
• Your name
• Your
company
• Skills and
knowledge
• Brief history
• Objective
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—18
Prepare to share the following information:
Your name
Your company
If you have most or all of the prerequisite skills
A profile of your experience
What you would like to learn from this course
16
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Module 1
Applying Troubleshooting
Methods
Overview
A systematic troubleshooting model is necessary to provide consistent network services and
minimize service interruptions. This module will teach you how to prepare a systematic
troubleshooting method and how to create and implement an action plan. You will learn how to
investigate and define problems, use various tools and techniques to gather facts, assemble an
effective troubleshooting plan, and observe and document the solutions.
Module Objectives
Upon completing this module, you will be able to apply effective troubleshooting methods to
resolve issues in complex IP telephony networks.
Module Objectives
• Explain the complexities of IP telephony
troubleshooting
• Troubleshoot common IP telephony network
problems using a Cisco recommended
methodology
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—1-2
Module Outline
The outline lists the components of this module.
Module Outline
• Overview of IP Telephony Troubleshooting
• Troubleshooting IP Telephony Networks
© 2004 Cisco Systems, Inc. All rights reserved.
1-2
Cisco IP Telephony Troubleshooting (IPTT) v4.0
IPTT v4.0—1-3
Copyright © 2004, Cisco Systems, Inc.
Introducing IP Telephony
Troubleshooting
Overview
With the complexity of Cisco Systems IP telephony networks, a good administrator must have
a solid understanding of how to troubleshoot the various areas of voice networks and existing
networks. This lesson provides a brief overview of the many areas of troubleshooting found in
Cisco voice networks.
Relevance
Before you begin troubleshooting any network, you must recognize the broad areas that can
malfunction. For example, a failure of a manager Cisco IP Phone to connect to Cisco
CallManager does not necessarily mean that the Cisco CallManager server itself is
malfunctioning. This lesson helps identify the major areas of the Cisco IP telephony network
that can enable you to isolate problem areas quickly during troubleshooting.
Objectives
Upon completing this lesson, you will be able to:
Identify voice network troubleshooting issues
Identify general Cisco CallManager troubleshooting issues
Identify general Cisco Unity troubleshooting issues
Identify general network infrastructure troubleshooting issues
Identify general complexities of troubleshooting voice networks/clients
Learner Skills and Knowledge
To benefit fully from this lesson, you must have these prerequisite skills and knowledge:
Fundamental understanding of Cisco voice networks
Outline
The outline lists the topics included in this lesson.
Outline
• Overview
• Cisco Voice Networks
• Cisco CallManager
• Cisco Unity
• Network Infrastructure
• Voice Clients
• Summary
• Quiz
© 2004 Cisco Systems, Inc. All rights reserved.
1-4
Cisco IP Telephony Troubleshooting (IPTT) v4.0
IPTT v4.0—1-3
Copyright © 2004, Cisco Systems, Inc.
Voice Networks
This topic discusses the breadth of Cisco voice networks.
Voice Networks
RNO
PRI
PC
Desktop
PC
Desktop
PSTN
Publisher
Subscriber
PC
Desktop
FXS
Cisco
Unity
PC
Desktop
PC
Desktop
FR
Console
Cisco Soft
Phone 2
x1002
Cisco Soft
Phone RNO
x6001
FXO
Cisco Soft
Phone 1
x1001
Console
7960 x1010 on Cisco
Soft Phone 1
PC
Desktop
7960 x6110
on Cisco
Soft Phone 1
Voice
Modem
x1401
PC
Desktop
PC
Desktop
DFW
© 2004 Cisco Systems, Inc. All rights reserved.
Cisco Soft
Phone HQ
x1501
DFW-HQ
PC
Desktop
Cisco Soft
Phone BR
x1501
DFWBRANCH
IPTT v4.0—1-4
Cisco IP telephony networks can be extremely complex. Voice networks not only introduce
their own set of troubleshooting areas, but they also rely on the existing network infrastructure.
Because of this reliance, a good IP telephony network administrator must be adept not only in
troubleshooting application layer components, such as Cisco CallManager or Cisco Unity, but
also in common foundation troubleshooting issues.
The four common areas for troubleshooting Cisco voice networks are as follows:
Cisco CallManager
Cisco Unity
Network infrastructure
Voice clients
Depending on the complexity of the voice network and the services that you would like to add,
you may introduce additional troubleshooting areas. For example, if you want to add
AutoAttendant services on a separate server to the voice network using CallManager 3.3.x up
to 4.0, you could install a Cisco Customer Response Applications (CRA) server. This server
can introduce its own potential problems on the network, along with its own areas of
troubleshooting.
Copyright © 2004, Cisco Systems, Inc.
Applying Troubleshooting Methods
1-5
Cisco CallManager
This topic provides a brief overview of Cisco CallManager.
Cisco CallManager
RNO
PRI
PC
Desktop
PC
Desktop
PSTN
Publisher
Subscriber
PC
Desktop
FXS
Cisco
Unity
PC
Desktop
PC
Desktop
FR
Console
Cisco Soft
Phone 2
x1002
Cisco Soft
Phone RNO
x6001
FXO
Cisco Soft
Phone 1
x1001
Console
7960 x1010 on Cisco
Soft Phone 1
PC
Desktop
7960 x6110
on Cisco
Soft Phone 1
Voice
Modem
x1401
PC
Desktop
Cisco Soft
Phone HQ
x1501
PC
Desktop
DFW-HQ
DFW
© 2004 Cisco Systems, Inc. All rights reserved.
PC
Desktop
Cisco Soft
Phone BR
x1501
DFWBRANCH
IPTT v4.0—1-5
Cisco CallManager provides the core voice functionality in the IP telephony network. Cisco
CallManager is the software-based, call-processing component that provides the same
functionality as legacy PBX systems. When troubleshooting common Voice over IP (VoIP)
issues, such as one-way voice, no dial tone, or reorder tones, Cisco CallManager is typically the
first place to look for problems. Because of the large amount of complex configurations stored
in Cisco CallManager, you can usually attribute most of the common voice issues to a
configuration problem.
To assist in troubleshooting, Cisco CallManager has many built-in diagnostic tools. The Route
Plan Report tool generates a flow chart of the entire dial plan, which can be useful when
troubleshooting improperly routed calls or reorder tones. However, you can perform most indepth troubleshooting through the Trace utility. The Trace utility is capable of logging detailed
call processing and digit analysis. Traces can be useful in nearly any troubleshooting situation
that requires intense study of the call processing performed by Cisco CallManager.
Additional tools are available and provided by your service provider (xLog, enhanced Q.931
Translator, and more).
1-6
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Cisco Unity
This topic provides a brief overview of Cisco Unity.
Cisco Unity
RNO
PRI
PC
Desktop
PC
Desktop
PSTN
Publisher
Subscriber
PC
Desktop
FXS
Cisco
Unity
PC
Desktop
PC
Desktop
FR
Console
Cisco Soft
Phone 2
x1002
DFW
© 2004 Cisco Systems, Inc. All rights reserved.
Cisco Soft
Phone RNO
x6001
FXO
Cisco Soft
Phone 1
x1001
Console
7960 x1010 on Cisco
Soft Phone 1
PC
Desktop
7960 x6110
on Cisco
Soft Phone 1
Voice
Modem
x1401
PC
Desktop
PC
Desktop
Cisco Soft
Phone HQ
x1501
DFW-HQ
PC
Desktop
Cisco Soft
Phone BR
x1501
DFWBRANCH
IPTT v4.0—1-6
Nearly every corporate voice network that requires IP telephony services will also have a need
for voice-mail services. Cisco Unity provides unified messaging services, such as voice mail, email, and fax, to your IP telephony network. Cisco Unity can also be a common
troubleshooting source because of its advanced configuration. When troubleshooting issues,
such as Message Waiting Indicators (MWIs), voice-mail audio levels, or automatic call
transfers, you usually examine the Cisco Unity server first.
Because Cisco Unity integrates tightly with Cisco CallManager, many of the troubleshooting
areas result from a configuration error between the two devices. Because of this integration,
troubleshooting Cisco Unity problems may turn into troubleshooting Cisco CallManager. The
initial configuration and communication between Cisco CallManager and Cisco Unity is
critical, because that process sets the stage for the remainder of your IP telephony
communications troubleshooting.
Copyright © 2004, Cisco Systems, Inc.
Applying Troubleshooting Methods
1-7
Network Infrastructure
This topic provides a brief overview of the IP telephony network infrastructure.
Network Infrastructure
RNO
PRI
PC
Desktop
PC
Desktop
PSTN
Publisher
Subscriber
PC
Desktop
FXS
Cisco
Unity
PC
Desktop
PC
Desktop
FR
Console
Cisco Soft
Phone 2
x1002
Cisco Soft
Phone RNO
x6001
FXO
Cisco Soft
Phone 1
x1001
Console
7960 x1010 on Cisco
Soft Phone 1
PC
Desktop
7960 x6110
on Cisco
Soft Phone 1
Voice
Modem
x1401
PC
Desktop
Cisco Soft
Phone HQ
x1501
PC
Desktop
DFW-HQ
DFW
© 2004 Cisco Systems, Inc. All rights reserved.
PC
Desktop
Cisco Soft
Phone BR
x1501
DFWBRANCH
IPTT v4.0—1-7
The IP telephony network infrastructure establishes the foundation for your entire voice
network. The infrastructure can be a common source of troubleshooting because it supports not
only the new voice features and traffic patterns, but also the existing data network. The merging
of the voice network and the data network has many benefits, but also carries considerable risk
because any existing data network issues will affect the voice network as well. For example, if
your routing protocol requires 60 seconds to converge and a primary link is lost, both the data
and voice network are without service for 60 seconds. In most environments, this kind of
failure is unacceptable.
Because the voice network relies completely on the existing network foundation, you must
understand how to troubleshoot all current network issues in a timely manner. You may need to
upgrade many of your routers and switches to support new voice functionality, such as inline
power, auxiliary VLANs, and voice interface cards (VICs). You must also have a thorough
understanding of the operation of these new features and equipment and of the troubleshooting
methods to use during a malfunction.
1-8
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Voice Clients
This topic provides a brief overview of IP telephony clients.
Cisco IP Telephony Clients
RNO
PRI
PC
Desktop
PC
Desktop
PSTN
Publisher
Subscriber
PC
Desktop
FXS
Cisco
Unity
PC
Desktop
PC
Desktop
FR
Console
Cisco Soft
Phone 2
x1002
Cisco Soft
Phone RNO
x6001
FXO
Cisco Soft
Phone 1
x1001
Console
7960 x1010 on Cisco
Soft Phone 1
PC
Desktop
7960 x6110
on Cisco
Soft Phone 1
Voice
Modem
x1401
PC
Desktop
PC
Desktop
Cisco Soft
Phone HQ
x1501
PC
Desktop
DFW-HQ
DFW
Cisco Soft
Phone BR
x1501
DFWBRANCH
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—1-8
In some instances, voice clients can cause voice network malfunctions. Voice clients can
include, but are not limited to the following:
Cisco IP Phones, including IP Communicator
Cisco SoftPhone
Cisco CallManager Attendant console or Cisco WebAttendant, or both
Analog devices, such as fax machines or legacy telephones
Either the end user or the network administrator can incorrectly configure or connect these
devices to the network. You may also find certain devices to be incompatible with your Cisco
IP telephony network. In any event, you should consider client devices when performing any
troubleshooting on the voice network.
Note
The remainder of the IPTT course does not focus on troubleshooting voice client issues;
these issues are typically simpler in nature, and an administrator can usually solve these
problems by a physical check of the device.
Copyright © 2004, Cisco Systems, Inc.
Applying Troubleshooting Methods
1-9
Summary
This topic summarizes the key points discussed in this lesson.
Summary
• There are four common areas for troubleshooting
Cisco voice networks: Cisco CallManager, Cisco Unity,
network infrastructure, and voice clients.
• Cisco CallManager is typically the first place to look for
problems when troubleshooting common VoIP issues.
• Cisco Unity is a common troubleshooting source
because of its advanced configuration.
• The network infrastructure can be a common source of
troubleshooting because it supports both the voice
network and the data network.
• Incorrect configuration of voice clients can be a
common troubleshooting source.
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—1-9
References
For additional information, refer to this resource:
Internetworking Troubleshooting Handbook, Second Edition:
http://www.cisco.com/univercd/cc/td/doc/cisintwk/itg_v1/tr1901.htm
1-10
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Quiz
Use the practice items here to review what you have learned in this lesson. The correct answers
are found in the Quiz Answer Key.
Q1)
A 900 MHz cordless telephone is considered a voice client in a Cisco IP telephony
network.
A)
B)
Q2)
If you discovered that your Cisco IP Phone was not able to obtain inline power, which
area would you check first?
A)
B)
C)
D)
Q3)
Windows 2000 Network Monitor
Cisco CallManager
Cisco Unity
Cisco switch configuration
Users complain that their message waiting light does not turn off after they have
checked their voice mail. Which of these would you check first?
A)
B)
C)
D)
Q4)
true
false
Windows 2000 Network Monitor
Cisco CallManager
Cisco Unity
Cisco router debug commands
You would like to perform a detailed trace of the call processing that occurs on your
voice network. Which platform best provides this capability?
A)
B)
C)
D)
Windows 2000 Network Monitor
Cisco CallManager
Cisco Unity
Cisco router debug commands
Copyright © 2004, Cisco Systems, Inc.
Applying Troubleshooting Methods
1-11
Quiz Answer Key
1-12
Q1)
A
Q2)
D
Q3)
C
Q4)
B
Q5)
B
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting IP Telephony
Networks
Overview
Preparing a systematic troubleshooting method requires the identification of essential tools and
procedures for troubleshooting IP telephony and data network systems. This lesson will teach
you basic procedures for identifying network system problems. You will learn to use these
methods to assist you in troubleshooting nearly every type of network issue that you may
encounter.
Relevance
To effectively troubleshoot your environment, you must systematically define the problem,
gather the pertinent data, and consider the potential causes. It is important to develop a standard
troubleshooting procedure within your organization. This lesson serves as an example template
on how to troubleshoot a problem.
Objectives
Upon completing this lesson, you will be able to:
List considerations in preparing for a network outage
Describe the importance of a systematic troubleshooting method
Apply recommended methods to clearly define a voice network problem
Apply recommended methods to gather facts as part of a recommended troubleshooting
methodology
Construct a list of possible problem causes, based on gathered facts, to troubleshoot voice
network problems
Formulate an action plan, based on a list of possible causes, to troubleshoot voice network
problems
Employ an action plan to troubleshoot voice network problems
Analyze the results of a troubleshooting action plan and determine whether the problem is
resolved
Formulate a new action plan, based on a list of possible causes and previous results, to
troubleshoot voice network problems if a previous action plan does not solve the problem
Learner Skills and Knowledge
To benefit fully from this lesson, you must have these prerequisite skills and knowledge:
Basic understanding of Cisco voice networks
Outline
The outline lists the topics included in this lesson.
Outline
• Overview
• Network Failure Preparation
• Systematic Troubleshooting Method
• Define the Problem
• Gather Facts
• Consider Possibilities
• Create Action Plan
• Implement Action Plan
• Observe Results
• Restart the Problem-Solving Process
• Document Results
• Summary
• Quiz
© 2004 Cisco Systems, Inc. All rights reserved.
1-14
Cisco IP Telephony Troubleshooting (IPTT) v4.0
IPTT v4.0—1-3
Copyright © 2004, Cisco Systems, Inc.
Network Failure Preparation
This topic points out and asks a few questions pertaining to whether you are ready for a
network outage.
Network Failure Preparation
You can always recover more easily from
a network failure if you are prepared
ahead of time.
To determine if you are prepared, answer
a few questions.
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—1-4
Before discussing systematic troubleshooting methods, answer the questions that follow
regarding your preparedness to handle a VoIP network outage. Please answer these questions
on your own.
Do you have an accurate physical and logical map of your internetwork that outlines the
physical location of all of the VoIP devices on the network and how they are connected and
also a logical map of the network addresses, network numbers, and subnetworks?
Do you have a list of all network protocols that are implemented in your network and a list
of the network numbers, subnetworks, zones, and areas that are associated with those
network protocols?
Do you know which protocols are being routed and the correct, up-to-date configuration
information for each protocol?
Do know which protocols are being bridged? Are there any filters configured in any of
these bridges, and do you have a copy of these configurations? Are any of these protocols
applicable to Cisco CallManager?
Do you know all the points of contact to external networks, including any connections to
the Internet? For each external network connection, do you know which routing protocol is
being used?
Has your organization documented normal network behavior and performance so that you
can compare current problems with a baseline?
Note
How to conduct a baseline is taught in Cisco Internetwork Troubleshooting (CIT).
Copyright © 2004, Cisco Systems, Inc.
Applying Troubleshooting Methods
1-15
Systematic Troubleshooting Method
This topic describes systematic troubleshooting methods. It is critical that an IP telephony
environment is stable. Legacy PBX environments possess a low failure rate. Because Cisco
CallManager and IP telephony run on a data network and not on a voice network, you can
expect to encounter problems. Because businesses rely heavily on the voice network, you must
ensure that it continues to run with at least the same reliability as the legacy PBX environment.
The Question
Question: Why use a systematic
troubleshooting method?
Answer: A systematic troubleshooting
method will make it easier for you to
identify potential problems on a data
network and an IP telephony system.
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—1-5
A systematic troubleshooting method will make it easier for you to identify potential problems
on a data network and an IP telephony system. You can use a troubleshooting model to
methodically reduce a large set of possible causes of trouble to a smaller set of causes or a
single cause. Then you can fix the problem and restore the IP telephony network.
After you have resolved a problem, a systematic process of documenting the case helps you
capture, preserve, and communicate the experience gained while solving the problem. You can
also refer to this document if similar problems arise later. Such a method helps you increase the
expertise of the organization and reduces the time you will spend solving future problems.
1-16
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Problem-Solving Model
Start
Define Problem
Finished
Gather Facts
Document Facts
Consider Possibilities
Problem Resolved
Create Action Plan
Implement Action Plan
Observe Results
Utilize Process
Yes
Do
problem
symptoms
stop?
No
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—1-6
The following problem-solving model is one example of a systematic troubleshooting method:
Step 1
Define the problem clearly and understandably.
Step 2
Gather all relevant facts.
Step 3
Consider the likely possibilities.
Step 4
Create an action plan.
Step 5
Implement an action plan.
Step 6
Observe the results of the implemented action plan.
Step 7
If the problem symptoms do not stop, you can undo the changes made and
implement another action plan.
Step 8
If the problem stops, document the solution.
Note
If you have not previously approached problems systematically or have not considered using
a problem-solving model, you may find that it takes longer initially, but saves you time in the
end.
Copyright © 2004, Cisco Systems, Inc.
Applying Troubleshooting Methods
1-17
Sample Network Problem
Cluster A
Cluster B
IP Phone A
WAN (FR)
X
Cisco
CallManager A
H.323 or Intercluster Trunk
© 2004 Cisco Systems, Inc. All rights reserved.
IP Phone B
Y
Cisco
CallManager B
IPTT v4.0—1-7
You will learn to apply the problem-solving process to a sample voice network issue.
Example: Network Problem
The figure shows a network problem. You receive the following problem report:
A user in cluster A initiates a call to a user in cluster B. IP Phone B rings, but as soon as the
user answers the telephone, the user hears a fast busy signal.
Refer to the problem-solving model as you go through the problem-solving process.
1-18
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Define the Problem
This topic describes defining a problem.
Define Network Problem
Define Problem
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—1-8
A systematic approach to troubleshooting consists of a sequence of steps. First, you should
define the problem clearly and sufficiently. To make a clear problem statement, define the
problem in terms of a set of symptoms and the associated causes. Ideally, when defining the
problem, you will need to compare your current network configuration and performance against
a network baseline.
Note
A baseline is a set of data collected from targets after installation. Use the baseline data for
comparison with real-time data.
To define the problem, you first identify the general symptoms. Then you determine what
possible problems these symptoms may indicate. Your problem statements must refer to the
baselines that have already been established for the network. You should be able to identify the
characteristics of the network when the network is performing as expected. In addition, you
must have knowledge of the network aspects that have changed since the last record of baseline
performance.
Copyright © 2004, Cisco Systems, Inc.
Applying Troubleshooting Methods
1-19
Sample Network Problem:
Define the Problem
Cluster A
Cluster B
Audio Stream Fails
WAN (FR)
X
Cisco
CallManager A
H.323 or Intercluster Trunk
© 2004 Cisco Systems, Inc. All rights reserved.
Y
Cisco
CallManager B
IPTT v4.0—1-9
When referring to the sample network problem, your definition of the problem should be very
similar to the problem report itself; for example, the user in cluster A reported that when
attempting to call another user in cluster B, the telephone rings but returns a fast busy signal
when answered.
While you are defining the problem, you should already be formulating the possible causes of
the problem. For example, you may remember recently modifying the Cisco CallManager
region configurations. This can give you a head start in the right direction and may provide a
quick fix to the problem.
1-20
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Gather Facts
This topic describes the process of collecting information about your network problem.
Gather Facts
Define Problem
Gather Facts
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—1-10
The next step in the problem-solving model is to gather facts. You start by extracting pertinent
information about the problem from the end user by asking pointed questions. Examples of
good questions include the following:
Is this issue systemwide?
How often does this problem occur?
When did this problem start to happen?
Has there been any change to the systems?
Note
It is also important for you to know what works for the user. For example, if a user is having
problems calling one extension, find out what extensions the user can call. This will assist in
narrowing the scope of the problem.
In addition, you should question other key people involved with the network, such as network
administrators and managers.
To gather information, you can use internal or external tools to help collect data. Internal tools
are methods that you can use directly on Cisco equipment. Some examples of internal tools are
the show or debug commands. External tools are the methods you can use to gather
information from sources not directly related to Cisco equipment. Some examples of external
tools are protocol analyzers, Windows 2000 Performance Monitor, Cisco CallManager traces,
and network management systems.
Copyright © 2004, Cisco Systems, Inc.
Applying Troubleshooting Methods
1-21
Sample Network Problem: Gather Facts
Gathering the facts:
• Question the end users.
• Verify Cisco
CallManager
configuration.
Cluster A
Cluster B
X
WAN (FR)
Y
• Verify router
configuration.
• Use external tools, such
as network analyzers or
Cisco CallManager trace
files.
© 2004 Cisco Systems, Inc. All rights reserved.
H.323 or Intercluster
Cisco
Cisco
Trunk
CallManager A
CallManager B
IPTT v4.0—1-11
When referring to the sample network problem, the first step you should take is to question the
end users. The user can provide critical information that you may not find on an e-mail trouble
ticket or from a telephone call. The following provides you with a sample question and answer
session with the user of IP Phone A:
Question: Do you always receive a fast busy signal when calling IP Phone B?
—
Answer: This problem seems to happen intermittently. It started about two months
ago, but only occurred from time to time. Recently, it has been occurring quite
frequently.
Question: Have you changed the configuration of your telephone in any way?
—
Answer: No.
After you gather and consider the information given to you by the end user, you can begin to
verify your Cisco CallManager and router configurations. You realize that the IP Phones in
cluster B are all first generation Cisco IP Phones that support G.711 and G.723 codecs. The IP
Phones in Cluster A are second generation Cisco IP Phones, which support G.711 and G.729
coder-decoders (codecs). Upon verifying the Cisco CallManager region configuration, you see
that you have Cisco CallManager configured to require G.729 when communicating between
clusters. However, because the user informed you that the problem is intermittent, do you think
the problem directly relates to this configuration issue?
For these types of problems, tools that can gather information in real time can also be useful.
For example, you could have the IP Phone A user attempt to call the IP Phone B user while
performing a router debug, capturing data through a network analyzer, or logging the call setup
process through Cisco CallManager trace files.
1-22
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Consider Possibilities
This topic describes the process of determining the potential problems on your IP telephony
network.
Consider Possibilities
Define Problem
Gather Facts
Consider Possibilities Based on Facts
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—1-12
After you have gathered all of the available facts, you should consider the problem possibilities
based on those facts. You can set boundaries to help isolate the network problems. Remove
irrelevant network details from the set of items to check. You can also eliminate entire classes
of problems that are associated with system software and hardware.
Copyright © 2004, Cisco Systems, Inc.
Applying Troubleshooting Methods
1-23
Sample Network Problem:
Considering Possibilities
Possible problems:
Cluster A
Cluster B
• Incorrect region
definitions
• No transcoding resource
X
WAN (FR)
Y
• Access list on router
• RTP header compression
mismatch
H.323 or Intercluster
Cisco
Cisco
Trunk
CallManager A
CallManager B
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—1-13
By brainstorming the data gathered from the sample network problem, you may come up with a
few possible causes that could include the following:
Incorrect region definitions
No transcoding resource
Access list on router
Real-Time Transport Protocol (RTP) header compression mismatch
After considering these possibilities, you should create an action plan for the most likely
solution. For example, does it seem likely when you consider incorrect region definitions? You
know that the IP Phones in Cluster A only support the G.729 compressed codec and the IP
Phones in Cluster B only support the G.723 codec. Because sending G.711 across the WAN
would be unacceptable, what region definition should you use? Your solution should be to use
the G.729 codec, as you currently have it configured. This would invoke a transcoding resource
to convert between G.723 and G.729, which can be extremely resource intensive because there
is no direct conversion between these two codecs (Cisco CallManager must convert to G.711,
then to the alternative codec causing the transcoding resource requirement to double).
After you have considered this possibility, you can eliminate the “router access list” and “RTP
header compression mismatch” options because the calls do connect on occasion. The most
likely possibility relates to the transcoding resources.
1-24
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Create Action Plan
This topic describes the creation of an action plan.
Create Action Plan
Define Problem
Gather Facts
Consider Possibilities Based on Facts
Create Action Plan
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—1-14
When creating an action plan, you should employ a “divide and conquer” policy. You should
consider the most likely possibility from the list and determine the methods you can use to
correct the problem.
You must break the problem into small steps. You start from the troubled device and work
outward. At each step, determine if the network is functioning properly. This will help you
trace a path from the source of the problem to the destination.
Finally, collaborate with others to develop an action plan, especially if your expertise is in a
specific area. It will save time and give you experience in other areas.
Copyright © 2004, Cisco Systems, Inc.
Applying Troubleshooting Methods
1-25
Sample Network Problem:
Create Action Plan
Cluster A
Action Plan:
• Add additional
transcoding resources
Cluster B
X
Cisco
CallManager A
© 2004 Cisco Systems, Inc. All rights reserved.
WAN (FR)
H.323 or Intercluster
Trunk
Y
Cisco
CallManager B
IPTT v4.0—1-15
After you consider all possibilities and narrow your options down to the most likely cause, the
action plan should reflect a solution directly related to the problem. In the sample problem, it is
assumed that the transcoding resources were running short. The solution to this problem would
be to add transcoding resources.
Depending on the overall cost, you may decide to replace the first-generation IP Phones in the
cluster with second-generation IP Phones that support the G.729 codec instead. Of course,
before you make any purchases, you should be involved in extensive monitoring of the
transcoding resources to ensure that this is the true cause of the problem.
1-26
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Implement Action Plan
This topic describes how to properly implement your action plan.
Implement Action Plan
Define Problem
Gather Facts
Consider Possibilities Based on Facts
Create Action Plan
Implement Action Plan
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—1-16
When developing and executing the action plan, you should be as specific as possible. The plan
must identify a set of steps that you will execute, and you must carefully implement each step.
Be sure to keep track of exactly what you are testing. It is best that you list the action plan in a
step-by-step process on paper. You can use this documented, systematic action plan to take
notes while implementing the plan, allowing you to track successes and failures. You should
never change more than one variable at a time, because otherwise it will be difficult to
determine the ultimate solution to the problem.
The following are other items that you should consider during the implementation of the action
plan:
Make sure that the changes you made do not make the problem worse; if the changes do
make the problem worse, you should reverse the changes.
Limit the impact of the changes on other users.
You should minimize the extent or duration of potential security lapses, such as removing
an access list.
In addition to fully backing up Cisco CallManager or the Cisco Unity server, you should
maintain backup configurations of the routers and switches in your network.
Copyright © 2004, Cisco Systems, Inc.
Applying Troubleshooting Methods
1-27
Observe Results
This topic describes the method used to ensure that your action plan resolved the issue at hand.
Observe Results
Create Action Plan
Implement Action Plan
Observe Results
© 2004 Cisco Systems, Inc. All rights reserved.
Do
problem
symptoms
stop?
IPTT v4.0—1-17
After manipulating a variable to find a solution to a problem, be sure to gather results based on
the action plan, and determine whether you have resolved the problem. If you do resolve the
problem, make sure to document the solution in addition to the action plan process.
1-28
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Restart the Problem-Solving Process
This topic describes where to begin if your initial action plan fails.
Restart the Problem-Solving Process
Create Action Plan
Implement Action Plan
Observe Results
Restart Process
© 2004 Cisco Systems, Inc. All rights reserved.
Do
problem
symptoms
stop?
No
IPTT v4.0—1-18
After you observe the results and determine that the problem still exists, you should restart the
process from the possibilities based on the facts gathered. With the result of the last action plan,
you will be able to narrow the possibilities. Your narrowing of the possibilities should be an
ongoing process.
After you have created a new action plan, you should implement this plan and observe the
results. Again, it is important for you to undo any fixes that did not work in previous
implementations.
Copyright © 2004, Cisco Systems, Inc.
Applying Troubleshooting Methods
1-29
Document Results
This topic describes the documentation process after you find a solution.
Document Results
Finished
Document Facts
Problem Resolved
Implement Action Plan
Observe Results
Yes
Do
problem
symptoms
stop?
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—1-19
As soon as you resolve the problem, you must document your work.
Reasons for creating documentation include the following:
Documentation maintains the exact steps that you took to solve the problem.
Documentation provides you with a back-out plan in case the fixes applied worsen the
situation over time.
Documentation of the problem and resolution serve as an historical record for future
reference.
1-30
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Summary
This topic summarizes the key points discussed in this lesson.
Summary
• To troubleshoot an IP telephony system or data network, you
must use a problem-solving model. This model includes eight
steps.
• Define the symptoms of the network problem clearly and
understandably.
• Use internal and external tools to gather facts.
• Isolate the network problem and consider the possible causes.
• Develop an action plan.
• Implement an action plan.
• Evaluate the effectiveness of the action plan.
• If the problem still exists, restart the process from the
possibilities that are based on the facts previously gathered.
• Document the results of the action plan.
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—1-20
References
For additional information, refer to these resources:
Internetworking Troubleshooting Handbook, Second Edition:
http://www.cisco.com/univercd/cc/td/doc/cisintwk/itg_v1/tr1901.htm
Cisco CallManager (each CallManager version has its own troubleshooting link):
http://www.cisco.com/univercd/cc/td/doc/product/voice/c_callmg/
Copyright © 2004, Cisco Systems, Inc.
Applying Troubleshooting Methods
1-31
Quiz
Use the practice items here to review what you have learned in this lesson. The correct answers
are found in the Quiz Answer Key.
1-32
Q1)
A network user complains about problems calling outside of the building. The person
called is able to hear the caller, but the caller cannot hear the other person. You should
use the strategy presented in this lesson to formulate an action plan to solve this
problem. Rather than gathering information from a live network, you can use
Cisco.com to research this problem. (Hint: Search for “one-way audio issues.”)
Q2)
A network user is attempting to make an outside call. The user uses the access code
you have defined (9) and then dials the number. The user complains that halfway
through the dialing process, a second dial tone is heard. This occurs intermittently,
depending on whom the user is calling. You should use the strategy presented in this
lesson to formulate an action plan to solve this problem. Rather than gathering
information from a live network, you can use Cisco.com to research this problem.
Q3)
A network user is attempting to make a call across the IP WAN to an office located in
Michigan. When the user finishes dialing the number, there is nearly a 10-second delay
before the call goes through. This problem has only been occurring in the last few days.
You should use the strategy presented in this lesson to formulate an action plan to solve
this problem. Rather than gathering information from a live network, you can use
Cisco.com to research this problem.
Q4)
You are attempting to install the Cisco IP SoftPhone on a PC. When you are asked for
a username and password, you enter the account information that you have created for
the user, but the Cisco IP SoftPhone returns an error message. You verify that the PC
does have connectivity to Cisco CallManager and that you have properly created an
account for this user. You should use the strategy presented in this lesson to formulate
an action plan to solve this problem. Rather than gathering information from a live
network, you can use Cisco.com to research this problem.
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Quiz Answer Key
Q1)
Make sure that IP
routing is enabled on Cisco IOS gateway and routers.
Check basic IP routing.
Bind the H.323 signaling to a specific IP address on Cisco IOS gateway and routers.
Cut through two-way audio early using the voice rtp send-recv command on Cisco IOS gateway and
routers.
Check the cRTP settings on a link-by-link basis on Cisco IOS gateway and routers.
Verify the minimum software level for Network Address Translation (NAT) on Cisco IOS gateway
and routers.
Disable the voice-fastpath command on Cisco AS5350 Universal Gateway and Cisco AS5400
Universal Gateway.
Q2)
Learner answers will vary. However, learners should follow the lesson guidelines to formulate an action
plan that includes a verification of the dial plan to ensure that route patterns do not overlap.
Q3)
Learner answers will vary. However, learners should follow the lessons guidelines to formulate an action
plan that includes the verification of the dial plan and either the removal of specific route patterns or a
reduction in the T302 timer (interdigit timeout).
Q4)
Learner answers will vary. However, learners should follow the lesson guidelines to formulate an action
plan that includes the verification of the user account within Cisco CallManager to ensure that the
administrator has enabled the user for computer telephony integration (CTI) applications use and that the
administrator has associated the correct device to the user.
Copyright © 2004, Cisco Systems, Inc.
Applying Troubleshooting Methods
1-33
1-34
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Module 2
Troubleshooting Cisco
CallManager, Network
Signaling, and Dial Plan
Overview
After you have migrated your voice network to include Cisco CallManager functionality, the
entire structure of your voice network changes. Nearly all devices should now communicate
across the data network structure, keeping the number of analog devices used at a minimum.
Because you have replaced the PBX functionality with the Cisco CallManager cluster, a large
amount of your troubleshooting focus now changes to include the Cisco CallManager system
itself. This module presents the four most common areas of troubleshooting in a Cisco
CallManager-based network: the voice dial plan, network signaling, upgrades, and other Cisco
CallManager configurations.
Module Objectives
Upon completing this module, you will be able to troubleshoot common Cisco CallManager
configuration, integration, and operation issues.
Module Objectives
• Troubleshoot common Cisco CallManager
configurations, integration, and operation
problems
• Troubleshoot problems with Cisco CallManager
dial plan, call routing, and media resources
• Troubleshoot Cisco CallManager CTI and Cisco
Telephony Service Provider issues and potential
problems using Extended Services AutoAttendant
• Troubleshoot common Cisco CallManager upgrade
issues
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—2-2
Module Outline
The outline lists the components of this module.
Module Outline
• Troubleshooting Cisco CallManager Signaling
Architecture
• Troubleshooting Cisco CallManager Dial Plan, Call
Routing, and Media Resources
• Troubleshooting Cisco CallManager CTI Manger,
JTAPI, and TSP
• Troubleshooting Cisco CallManager Upgrades
© 2004 Cisco Systems, Inc. All rights reserved.
2-2
Cisco IP Telephony Troubleshooting (IPTT) v4.0
IPTT v4.0—2-3
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Cisco
CallManager Signaling
Architecture
Overview
Cisco CallManager establishes calls between devices via the following protocols: H.323,
Q.931, Skinny Client Control Protocol (SCCP), and Media Gateway Control Protocol (MGCP).
The H.323 interface communicates with devices and with various layers or components of
Cisco CallManager. This lesson will teach you about Cisco CallManager signaling, device
management, device definition, and call control architecture. You will learn how to use the
H.323 interface architecture and trace commands.
Relevance
To troubleshoot Cisco CallManager environment, you must understand the fundamental
principals underlying Cisco CallManager call control capabilities, including how
Cisco CallManager negotiates call processing to various protocol models and handles device
registration.
Objectives
Upon completing this lesson, you will be able to:
Describe Cisco CallManager architecture
Explain H.323 protocol functionality
Describe MGCP call setup
Explain session initiation protocol functionality
Explain ISDN Q.931 protocol functionality
Identify call flow traces within Cisco CallManager
Describe call preservation methods
Apply Cisco CallManager troubleshooting methods for problem solving using the webbased Trace tool
Learner Skills and Knowledge
To benefit fully from this lesson, you must have these prerequisite skills and knowledge:
Fundamental understanding of Cisco IOS commands at the Cisco CCNP® level
Basic understanding of Cisco voice networks
Outline
The outline lists the topics included in this lesson.
Outline
•
•
•
•
•
•
•
•
•
Overview
CallManager Signaling Overview
H.323 Protocol Overview
MGCP Protocol Functionality
Session Initiation Protocol Architecture
ISDN and Q.931
Station Definition and Call Flow
Call Preservation
Cisco CallManager Trace and Cisco Trace
Collection Tool
• Summary
• Quiz
© 2004 Cisco Systems, Inc. All rights reserved.
2-4
Cisco IP Telephony Troubleshooting (IPTT) v4.0
IPTT v4.0—2-3
Copyright © 2004, Cisco Systems, Inc.
Cisco CallManager Signaling Overview
This topic discusses how signaling occurs in a Cisco IP telephony network.
Cisco CallManager Signaling
CM1
CM2
SDL
SCCP
MGCP or H.323
RTP
IP Phone
© 2004 Cisco Systems, Inc. All rights reserved.
IP Phone
Gateway
IPTT v4.0—2-4
Cisco CallManager uses a variety of signaling protocols between the devices it controls. Cisco
CallManager uses SCCP to signal Cisco IP Phones, and uses H.323 or MGCP to signal
gateways. The Cisco CallManager servers in a cluster use signal distribution layer (SDL) links
to establish calls between devices registered to different Cisco CallManager servers. Media
connections between devices involved in a call are Real-Time Transport Protocol (RTP)
streaming sessions that occur between the devices directly.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Cisco CallManager, Network Signaling, and Dial Plan
2-5
H.323 Protocol Architecture
This topic discusses H.323, which is widely used for IP telephony because it transports any
combination of voice, video, and data.
H.323 Protocol
Application
Audio Signal
G.711
G.722
G.723.1
Data
T.127
G.728
G.729
Video Signal
H.261
H.263
T.126
Presentation
Session
Transport
RTCP
RAS
T.124
RTP
Supplementary Services
H.450.3
H.235
UDP
RTCP = RTP Control
Protocol
H.450.1
Control
H.245
H.225
T.125/T.122
H.450.2
X.224.0
TCP
Network
Data Link
Physical
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—2-5
International Telecommunication Union Telecommunication Standardization Sector (ITU-T)
H.323 specifies the protocol architecture for multimedia communications over unreliable
networks, such as IP. The specifications do not mandate the use of User Datagram Protocol
(UDP) TCP/IP at the transport and network layers of the Open Systems Interconnection (OSI)
reference model. However, the vast majority of implementations will probably include these
protocols.
The components that use the H.323 protocol are as follows:
Terminal: A terminal is an endpoint on the network that provides real-time, two-way
communications with another terminal or gateway. A terminal may provide voice only or
voice combined with video or data communications.
Gatekeeper: A gatekeeper is an entity on the network that provides address translation and
access control. As an option, the gatekeeper may provide other services, such as bandwidth
management.
Gateway: A gateway is an endpoint on the network that provides the necessary translation
services to allow H.323 terminals to communicate with terminals on other networks, such
as General Switched Telephone Network (GSTN), Narrowband-ISDN (N-ISDN), and
Broadband-ISDN (B-ISDN).
Multipoint control unit: A multipoint control unit is an endpoint on the network that
provides the capability for three or more terminals to participate in a multipoint conference.
2-6
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
H.323 Interface Architecture
H.323 Interface
H.225Init
H.323
Device
H.225D
(Per Station)
H.225Cdpc
Call
Control
(Per Call)
H.245
Interface
(Per
Connection)
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—2-6
The H.323 interface communicates with the devices on one side and with the various layers or
components of Cisco CallManager on the other side.
The H.323 interface communication consists of these major components:
H.225Init: The H.225Init process manages the dynamic creation and teardown of H.323
gateway devices.
H.225D: The H.225D process is responsible for handling all of the call-processing
messages for a particular device and retrieving data from the database.
H.225Cdpc: Cisco CallManager creates a single H.225Cdpc process for each active call.
The H.225Cdpc process is responsible for handling the call-related activities for the
duration of the call.
H.245 interface: The H.245 interface process manages the dynamic creation and teardown
of the H.245 interface per connection. The H.245 interface negotiates channel usage and
handles processes such as flow control and capabilities exchange of the IP Phone. For
example, the H.245 protocol helps the Cisco IP Phones negotiate a common audio coderdecoder (codec).
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Cisco CallManager, Network Signaling, and Dial Plan
2-7
Gatekeeper-Controlled H.323 Calls
Gatekeeper
ARQ
ACF
A
B
H.323 Version 2
Skinny
B
A
Skinny
RTP
H.323 RAS
Registered Cisco CallManager
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—2-7
When making an intercluster call using Call Admission Control (CAC), the Cisco IP telephony
network typically utilizes a gatekeeper to confirm or reject the call based on the allocated
bandwidth. A registered Cisco CallManager sends an admission request (ARQ) to its
appropriate gatekeeper. The ARQ packet notifies the gatekeeper of a call requesting to traverse
the WAN. The gatekeeper then compares this request to its configured parameters to determine
whether to disengage the call request. If a sufficient amount of bandwidth is available, the
gatekeeper will return an Admission Confirmation (ACF). Cisco CallManager exchanges these
ARQ and ACF messages with the gatekeeper using the H.225 registration, admission, and
status (RAS) protocol, which encompass all communication between Cisco CallManager and
the gatekeeper.
In this call flow example, a Cisco IP Phone located in cluster 1 is calling a Cisco IP Phone
located in cluster 2. To identify the components of the call flow, you must examine the call in
closer detail by examining the H.323 messages.
2-8
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Gatekeeper H.323 to H.323 Endpoints
Endpoint 1
Endpoint 2
RAS (ARQ)
GK
RAS (ACF)
Establish H.225 Signaling Channel (TCP Connection)
H.225 SETUP
H.225 Call Proceeding
RAS (ARQ)
RAS (ACF)
H.225 Alerting Connection (H.245 Address)
Terminal Capability Set + Acknowledgment Master-Slave Determination
Terminal Capability Set – Master-Slave Determination + Acknowledgment
Terminal Capability Set Acknowledgment – Master-Slave Determination + Acknowledgment
Open Logical Channel
Open Logical Channel (OLC) Acknowledgment with IP Address + Port from Side B
RTP Media Stream
RTP
Open Logical Channel
Open Logical Channel (OLC) Acknowledgment with IP Address + Port from Side A
H.245
RTP Media Stream
H.225
H.225 Release Complete
RAS (DRQ)
RAS (DRQ)
RAS (DCF)
RAS (DCF)
© 2004 Cisco Systems, Inc. All rights reserved.
RAS
IPTT v4.0—2-8
The figure shown here demonstrates the call flow of two H.323 devices using TCP for call
setup and teardown, and also media negotiation for RTP port numbers and codec values. The
process takes place as follows:
1. Endpoint 1 initiates ARQ to determine if it is able to make a call.
2. Gatekeeper responds with ACF.
3. Endpoint 1 sends call setup message to gatekeeper; gatekeeper routes message to
endpoint 2.
4. If endpoint 2 accepts, it must initiate an ARQ with the gatekeeper.
5. If the gatekeeper accepts the call, endpoint 2 sends a connect message to endpoint 1
specifying the H.245 call control channel for media capabilities exchange.
6. Clients exchange call capabilities with a terminal capability set message that describes the
ability of each client to send media streams or the audio and video codec capabilities of
each client.
7. After the capabilities exchange, clients have a compatible method for sending media
streams; the endpoints can now open multimedia communication channels.
8. To open a logical channel for sending media streams, the calling client sends an open
logical channel (OLC) message (H.245).
9. The receiving client responds with an OLC acknowledgement message (H.245).
—
Media streams send an unreliable channel, and control messages send a reliable
channel.
—
When the channel establishes, either client or gatekeeper can request call services;
that is, a client or gatekeeper can initiate increase or decrease of call bandwidth.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Cisco CallManager, Network Signaling, and Dial Plan
2-9
10. Either party can terminate the call.
11. Assume that endpoint 1 terminates the call.
12. Endpoint 1 completes transmission of media and closes logical channels that send media as
follows:
—
Endpoint 1 sends an end session command (H.245).
—
Endpoint 2 closes media logical channels and sends an end session command.
—
Endpoint 1 closes the H.245 control channel.
—
If call signaling channel is still open, the gatekeeper sends a release complete
message (Q.931) between clients to close this channel.
H.323 to Skinny Client Call Flow Diagram
Cluster A
RTP
IP Phone A
SCCP
X
WAN
(Frame Relay)
Y
H.323
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—2-9
Another type of call flow with Cisco CallManager includes its negotiation from an IP Phone to
an H.323 device, such as Microsoft NetMeeting. The call process itself is similar to the call
flow that exists between two H.323 devices. The primary difference is the reliance upon Cisco
CallManager by the IP Phone.
2-10
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
H.323 Client to Skinny Client Call Flow
H.323 Client
H.225 Setup
H.225 Setup Ack
Cisco CallManager
Cisco IP Phone
Station Call Information
Station Set Lamp (Blink)
Station Set Ringer (On)
Station Off Hook
H.225 Alerting
H.225 Connect
Station Set Lamp (Steady)
Station Set Ringer (Off)
H.245 Master-Slave Determination
H.245 Master-Slave Determination ACK
H.245 Terminal Capabilities Set
H.245 Terminal Capabilities Set ACK
H.245 Open Logical Channel
H.245 Open Logical Channel ACK
Station Start Media Reception
Station Start Media Transmission
Conversation
H.245 Request Channel Close
H.245 Request Channel Close ACK
H.225 Release Complete
Station On Hook
Station Stop Media Transmission
Station Stop Media Reception
Station Set Lamp (Off)
ACK = Acknowledgment
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—2-10
The figure shows a sample exchange of messages between an H.323-based client (such as
Microsoft NetMeeting) and a SCCP protocol client (such as a Cisco IP Phone).
Note
This is a sample only and does not exactly match the sequence or number of messages.
Notice the number of H.323 messages compared to the SCCP requirements. Because the
Skinny client is not fully H.323 compliant, Cisco CallManager acts as a proxy. Cisco
CallManager handles the larger part of the H.323 protocol signaling (H.225 and H.245
messages). The IP Phone supports the RTP protocol and is capable of streaming audio without
the assistance of Cisco CallManager.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Cisco CallManager, Network Signaling, and Dial Plan
2-11
MGCP Protocol Functionality
This topic discusses the MGCP protocol functionality and call setup process.
MGCP Protocol Functionality
• MGCP is a master-slave protocol, where the
gateways are expected to execute commands sent
by the call agents.
• MGCP contains “simple” endpoints; a simple
endpoint executes a small set of simple
transactions as instructed by the call agent.
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—2-11
The two basic MGCP constructs are endpoints and connections. An endpoint is a source for call
data (RTP/IP) that is flowing through the gateway. A common type of endpoint is found at the
physical interface between the Public Switched Telephone Network (PSTN)—also called the
plain old telephone service (POTS)— and the gateway. This type of endpoint might be an
analog voice port or a digital service level 0 (DS0) group. There are other types of endpoints as
well, and some are logical rather than physical. An endpoint is identified by a two-part endpoint
name that contains the name of the entity on which it exists (for example, an access server or
router) and the local name by which it is known (for example, a port identifier).
Call agents manage call flow using standard MGCP messages that are sent to the endpoints
under their control. The messages are delivered in standard ASCII text and can contain session
descriptions transmitted in Session Description Protocol (SDP), a text-based protocol. These
messages are sent over IP/UDP.
Call agents, such as a CallManager, keep track of endpoint and connection status through the
reporting by the gateway of standard events that are detected from endpoints and connections.
2-12
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
MGCP Messages
MGCP primitives
Endpoint Configuration
EPCF
(CA
EP)
Create Connection
CRCX
(CA
EP)
Modify Connection
MDCX
(CA
EP)
Delete Connection
DLCX
(CA <-> EP)
Notification Request
RQNT
(CA
EP)
Notify
NTFY
(CA
EP)
Audit Endpoint
AUEP
(CA
EP)
Audit Connection
AUCX
(CA
EP)
Restart In Progress
RSIP
(CA
EP)
CA = Call Agent
EP = Endpoint
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—2-12
MGCP uses a series of messages between Cisco CallManager and the gateway.
MGCP consists of the following nine messages:
EPCF ( endpoint configuration): Used to specify the encoding of the signals that the
endpoint receives. For example, in certain international telephony configurations, some
calls carry u-law encoding audio signals and others use A-law. The call agent uses the
EPCF message to pass this information to the gateway.
RQNT (notification request): Cisco CallManager can issue a notification request to a
gateway, instructing the gateway to watch for specific events such as hook actions or dual
tone multifrequency (DTMF) tones on a specified endpoint. Devices can also use the
RQNT message to request that a gateway apply a specific signal to an endpoint (such as
dial tone or ringback).
NTFY (notify): The gateway uses this message to notify Cisco CallManager when the
requested events occur.
CRCX (create connection): Cisco CallManager uses this message to create a connection
that terminates in an endpoint inside the gateway.
MDCX (modify connection): Cisco CallManager uses this message to change the
parameters associated with a previously established connection.
DLCX (delete connection): Cisco CallManager uses this message to delete an existing
connection. Gateways can also use the DLCX message to indicate that it can no longer
sustain a connection.
AUEP (audit endpoint): Cisco CallManager uses this message to audit the status of an
endpoint associated with it.
AUCX (audit connection): Cisco CallManager uses this message to audit the status of any
connection associated with it.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Cisco CallManager, Network Signaling, and Dial Plan
2-13
RSIP (restart in progress): The gateway uses this message to notify Cisco CallManager
that the service status has changed for the gateway or for a group of endpoints managed by
the gateway. There are three types of restart: restart (endpoint in service), graceful (wait
until call clearing), and forced (endpoint out of service).
2-14
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
MGCP Protocol Functionality
• Endpoint Identifiers: have two components that are
both case insensitive:
– Domain name of the gateway that is managing
the endpoint
– Local name within that gateway
• AALN/S1/SU0/0@AV-VG200-2.cisco.com
• AALN/S1/SU0/0 = Slot 1/SubUnit 0/Port 0
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—2-13
MGCP uses the concept of endpoints to identify a particular trunk to the PSTN or PBX. In a
Cisco IP telephony network, the endpoints can be either analog ports such as Foreign Exchange
Station (FXS) or Foreign Exchange Office (FXO) or a channel on a digital trunk (for example
T1 or E1). In this figure, you see the endpoint identifier, which is an analog port. The
components of the first endpoint identifier are as follows:
AALN indicates that the port is analog.
S1 means slot 1 of the gateway.
SU0/0 is a subunit on the gateway (port 0/0 on the module in slot 1)/
AV-VG200-2 is the host of the gateway.
“.cisco.com” is the IP domain that the gateway is in.
—
When troubleshooting MGCP gateways, you will see the endpoint identifier in the
Cisco CallManager traces.
—
What would be the definition of an endpoint with the following identifier? ?
S1/DS1-1/1@2600-gw1.cisco.com
The answer is as follows:
S1 indicates slot 1.
DS1-1/1 is a digital circuit in port 1/1 in channel 1 of the digital circuit.
2600-gw1 is the host name of the gateway.
“.cisco.com” is the IP domain that the gateway is in.
If you are looking for a call in a Cisco CallManager trace and the call had gone out a MGCP
gateway, you can tell which port the call went out by looking at the endpoint identifier for the
specific call.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Cisco CallManager, Network Signaling, and Dial Plan
2-15
Registration and Endpoint
Gateway
TCP Socket Open
RSIP (Restart)
Acknowledgment
with Endpoint Information
Simple Acknowledgment
“200 <n> OK”
Cisco CallManager
Open completes
Simple Acknowledgment
“200 <n> OK”
Audit Endpoint (AUEP)
(One per endpoint)
Request Notify (RQNT)
“RQNT R: L/hd”
(One per endpoint)
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—2-14
This figure shows an example of the registration process that a MGCP gateway goes though
when the MGCP gateway and Cisco CallManager have both been configured properly. Looking
at the return codes in Cisco CallManager traces can tell you a lot about a connection. Notice in
the figure the return code of 200, indicating that the message was executed successfully.
The following are the return code value range:
Values between 100 and 199 indicate a provisional response.
Values between 200 and 299 indicate a successful completion.
Values between 400 and 499 indicate a transient error.
Values between 500 and 599 indicate a permanent error.
2-16
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
MGCP FXS Call
NTFY O: L/hd
RQNT R: L/hu,D/[0-9*#] S:dl
Cisco
CallManger
NTFY O: 4
VG200
RQNT R: L/hu, D/[0-9*#] S:
NTFY O: 5
CRCX
200 OK (RTP Port Information)
MDCX
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—2-15
For any MGCP call, Cisco CallManager must create a connection between an endpoint and the
packet network or between two endpoints. When an MGCP connection is created, Cisco
CallManager can immediately make changes to the connection by sending messages to modify
the connection. When a call needs to be torn down, Cisco CallManager sends a message to
delete the connection. When a connection is made to an endpoint, the gateway assigns a
connection identifier for each connection. In the case of Cisco CallManager, there is never
more than one connection to an endpoint at any given time.
Cisco CallManager also applies certain signals, such as a dial tone, to an endpoint. An MGCP
FXS call performs the following steps:
Step 1
FXS telephone goes off-hook, and the gateway sends an NTFY message for an offhook event.
Step 2
Cisco CallManager sends an RQNT message with a digit map and a signal to turn on
the dial tone. Cisco CallManager requests that the device send each digit
individually rather than cumulatively.
Step 3
The user presses the first digit.
Step 4
The VG200 sends the NTFY message to Cisco CallManager. Cisco CallManager
sends an RQNT message to turn off the dial tone.
Step 5
The user sends digit.
Step 6
Cisco CallManager sends a CRCX message to create a call leg. The gateway sends a
response with the RTP SDP parameters—IP address and port number for audio
stream. Cisco CallManager sets up the connection with the remote RTP endpoint
and starts the ringing tone to the caller.
Step 7
Cisco CallManager sends an MDCX message to the caller, setting the IP address and
port of the other RTP endpoint.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Cisco CallManager, Network Signaling, and Dial Plan
2-17
Note
2-18
The MDCX message can also set the local send or receive state.
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Session Initiation Protocol Architecture
This topic discusses the characteristics of session initiation protocol (SIP).
SIP Basics
• SIP originally defined in RFC 2543; reintroduced in
RFC 3261
• Peer-to-peer protocol where end-devices initiate sessions
• Defines the signaling mechanism for multimedia
conferences
• SIP uses several existing IETF protocols:
– HTTP 1.1
– Session Description Protocol (SDP)—media negotiation
– Real-Time Transfer Protocol (RTP)—media
– Domain Name Service (DNS)—name resolution
– Others—DHCP, MIME, TFTP
• Text-based ASCII for easy implementation and debugging
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—2-16
SIP is an Internet Engineering Task Force (IETF) standard for multimedia conferencing over
IP. SIP is an ASCII-based, application layer control protocol that Voice over IP (VoIP)
equipment can use to establish, maintain, and terminate calls between two or more clients. The
IETF designed SIP as an alternative to the ITU-T H.323 protocol. Because the IETF released
SIP after H.323, SIP does not have as broad a base of user support as H.323. However, because
SIP is a simpler protocol that does not maintain the large overhead required by H.323, the user
support of SIP is steadily increasing. For example, MSN Messenger (Windows XP) now
supports SIP capabilities. In addition, Cisco IP Phones now have SIP firmware, allowing them
to communicate on native SIP networks.
SIP natively supports the following functionalities:
Determines the location of the target endpoint: SIP supports address resolution, name
mapping, and call redirection.
Determines the media capabilities of the target endpoint via SDP: SIP determines the
lowest level of common services between the endpoints. The SIP-compliant devices
establish conferences using only the media capabilities supported by all endpoints.
Determines the availability of the target endpoint: If a device cannot complete a call
because the target endpoint is unavailable, SIP determines whether the called party is
already on the telephone or did not answer in the allotted number of rings. The SIP device
then returns a message indicating why the target endpoint was unavailable.
Establishes a session between the originating and target endpoint: If a device
determines that it can complete the call, SIP establishes a session between the endpoints.
SIP also supports midcall changes, such as the addition of another endpoint to the
conference or the changing of a media characteristic or codec.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Cisco CallManager, Network Signaling, and Dial Plan
2-19
Handles the transfer and termination of calls: SIP supports the transfer of calls from one
endpoint to another. During a call transfer, SIP establishes a session between the transferee
and a new endpoint (specified by the transferring party) and terminates the session between
the transferee and the transferring party. At the end of a call, SIP terminates the sessions
between all parties.
2-20
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
SIP Architecture and Agents
Application
Services
E-Mail
LDAP
SQL
XML
CPL
3pcc
SIP Proxy, Registrar,
and Redirect Servers
SIP
SIP
SIP
SIP User
Agents (UAs)
PSTN
CAS or PRI
RTP
SQL = Structured Query Language
© 2004 Cisco Systems, Inc. All rights reserved.
Legacy PBX
IPTT v4.0—2-17
Similar to H.323, SIP is a peer-to-peer protocol. A peer-to-peer protocol indicates that a client
SIP device contains all the intelligence to make or receive calls without the assistance of an
intermediary device. The IETF calls the peers in a SIP network User Agents (UAs). A UA can
function in one or both of the following roles:
User Agent client (UAC): This is a client application that initiates a SIP request.
User Agent server (UAS): This is a server that contacts the user when the device receives
a SIP request (such as an incoming call).
Most SIP endpoints, such as Cisco IP Phones, are capable of functioning as both a UAC and a
UAS. For example, if you make a telephone call from a SIP Cisco IP Phone, the IP Phone
performs the role of the UAC. However, if you receive an incoming telephone call on a SIP
Cisco IP Phone, the IP Phone performs the role of the UAS by ringing. Whether the endpoint
functions as a UAC or a UAS depends on the UA that originates the request. Overall, you can
think of the calling party as the UAC and the called party as the UAS.
Gateways also act as clients in a SIP network. The SIP gateway provides many services. The
most common service is a translation between SIP endpoints and other terminal types (such as
H.323 or PSTN devices).
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Cisco CallManager, Network Signaling, and Dial Plan
2-21
While you can run smaller SIP networks on a peer-to-peer basis, managing large SIP-based
networks will usually require the help of SIP servers. SIP servers include the following:
Proxy server: The proxy server is an intermediate device that receives SIP requests from a
client and then forwards the requests on behalf of the client. Proxy servers receive SIP
messages and forward them to the next SIP server in the network. Proxy servers can
provide functions such as authentication, authorization, network access control, routing,
reliable request retransmission, and security.
Redirect server: The redirect server provides the client with information about the next
hop or hops that a message should take, and then the client contacts the next hop server or
UAS directly.
Registrar server: The registrar server processes requests from UACs for registration of
their current location. Registrar servers are often co-located with a redirect or proxy server.
The redirect and registrar servers add increased levels of manageability to your SIP network by
adding centralized control. Moving from a peer-to-peer SIP network to a managed SIP network
is analogous to moving from a peer-to-peer networking model to a server-networking or clientnetworking model. You now have centralized management, authentication, and control of your
voice network.
2-22
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Cisco CallManager SIP Trunk
SCCP
Conf
MGCP
CTI
SIP
XCODE
Voice mail
Apps
SCCP
Phones
Microsoft
Messenger
Cisco SIP
IP Phone
Cisco IOS
SIP Gateway
SoftPhones
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—2-18
In a call-processing environment that uses SIP, SIP trunks can be used to configure a signaling
interface with Cisco CallManager for SIP calls. SIP trunks (or signaling interfaces) connect
Cisco CallManager clusters with a SIP proxy server. A SIP signaling interface uses port-based
routing, and Cisco CallManager accepts calls from any gateway as long as the SIP messages
arrive on the port that is configured as a SIP signaling interface. The SIP signaling interface
uses requests and responses to establish, maintain, and terminate calls (or sessions) between
two or more endpoints.
Cisco CallManager requires an RFC 2833 DTMF-compliant media termination point (MTP)
device to make SIP calls. The current standard for SIP uses in-band RTP payload types to
indicate DTMF tones. Cisco Architecture for Voice, Video and Integrated Data (AVVID)
components, such as SCCP IP Phones, support only out-of-band DTMF payload types. Thus,
an RFC 2833-compliant MTP device acts as a translator between in-band and out-of-band
DTMFs.
You can initiate outgoing calls to a SIP device from any Cisco CallManager device. A
Cisco CallManager device includes SCCP IP Phones or analog devices that are connected to
FXS gateways. For example, an SCCP IP Phone can place a call to a SIP endpoint. The SIP
device answering the call triggers media establishment.
Any device on the SIP network, including SIP IP Phones or analog devices that are connected
to FXS gateways, can initiate incoming calls. For example, a SIP endpoint can initiate a call to
an SCCP IP Phone. The SCCP IP Phone answering the call triggers media establishment.
Supplementary services initiated by an SCCP endpoint during a SIP call are supported. SCCP
endpoints are internally managed by Cisco CallManager without affecting the connecting SIP
device.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Cisco CallManager, Network Signaling, and Dial Plan
2-23
SIP Network
SIP IP Phone
RTP
SIP IP Phone
3
2
2
1
1. SIP IP Phone contacts registrar server
to resolve destination SIP address.
2. Cisco SIP proxy server sends INVITE
messages to calling and called parties.
3. SIP IP Phones acknowledge the INVITE
message and contact each other
directly. The IP Phones stream audio
through RTP.
Cisco SIP Proxy Server
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—2-19
SIP is a simple, ASCII-based protocol that uses requests and responses to establish
communication among the various components in the network, ultimately establishing a
conference between two or more endpoints.
The SIP network identifies users by unique SIP addresses. A SIP address is similar to an e-mail
address and is in the format of sip:userID@gateway.com. The user ID can also be in the form
of an E.164 address.
Users register with a registrar server using their assigned SIP addresses. The registrar server
then provides the assigned SIP address to the location server upon request.
When a user initiates a call, a SIP device sends the request to a SIP server (either a proxy or a
redirect server). The request includes the address of the caller and the address of the called
party.
Over time, a SIP end user might move between end systems. The SIP device can dynamically
register the location of the end user the SIP server. The location server can use one or more
protocols, including Finger, Referral Whois (RWhois), and Lightweight Directory Access
Protocol (LDAP) to locate the end user. Because the end user can log in at more than one
station, the location server might return more than one address for the end user. If the request is
coming through a SIP proxy server, the proxy server will try each returned addresses until it
locates the end user.
Note
2-24
For more information, see RFC 2543, SIP: Session Initiation Protocol, which you can find at:
http://www.faqs.org/rfcs/.
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
ISDN and Q.931
This topic discusses the architecture of ISDN and Q.931, and how they relate to your VoIP
network.
ISDN Architecture
64-kbps
BISDN
Packet
Switching
ISDN
Switch
CPE
64-kbps
Switched and
Nonswitched
ISDN
Switch
CPE
Frame
Mode
Common
Channel SIG
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—2-21
ISDN is an international telecommunications standard for providing a digital service from the
customer premises to the dial-up telephone network. Q.931 is the connection control protocol
used for ISDN connections, roughly comparable to TCP in the Internet protocol stack.
The scenario shown in the figure is an environment where customer premises equipment (CPE)
exists with ISDN facilities end to end.
Note
Although not shown in this figure, ISDN will communicate with non-ISDN facilities; however,
the feature set reverts to plain voice service when it encounters such communication.
Shown on each side of the figure is an ISDN-enabled PBX or central office (CO) switch. The
CPE may be a standalone ISDN telephone set, an ISDN-enabled key system, an ISDN-enabled
PBX, or any other device (such as a router or remote access server) that has an ISDN interface.
Two standardized ISDN interfaces exist: BRI and PRI.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Cisco CallManager, Network Signaling, and Dial Plan
2-25
Q.931
Bits
8
•
•
•
Call reference:
establishes a unique
value between the user
and the network
Message type: can
be grouped into call
establishment, call
information phase, call
clearing and
miscellaneous
0
0
7
6
5
4
3
0
0
0
1
0
1
Length of CRV
2
Call Reference Value
(1 or 2 bytes)
0
Message Type
0/1
2
Protocol Discriminator (Q.931)
0
0
0
1
0
0
CR Flag
Information elements: are
self-contained entities
that further define the
message
Octet
3 or 4
4 or 5
Information Element Identifier
1
Length Information Element Identifier
2
Contents of Information Element
3
Next Information Element, and so on
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—2-22
Q.931 is the connection control protocol of ISDN that is roughly comparable to TCP in the
Internet protocol stack. Q.931 does not provide flow control or perform retransmission, because
the protocol assumes that the underlying layers are reliable and that the circuit-oriented nature
of ISDN allocates bandwidth in fixed increments of 64 kbps. Q.931 manages connection setup
and breakdown.
The general format of a Q.931 message includes a single byte protocol discriminator of 8 bits ,
a call reference value (CRV) to distinguish between different managed calls over the same data
channel (D channel), a message type, and various information elements (IEs) required by the
message type in question.
These components of a Q.931 message are defined as follows:
CRV: A unique value established between user and the network
Message type: Groups include call establishment, call information phase, call clearing, and
miscellaneous
IEs: Self-contained entities that further define the message
It is important to understand the functional elements of the Q.931 ISDN connection control
protocol, because the H.323 protocol adopted the Q.931 signaling standard. You can apply
many of your Q.931 troubleshooting skills to both ISDN and H.323 connectivity.
2-26
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Typical Q.931 Messages
Calling
Party
Network
Called
Party
Setup
Setup Acknowledgment
Information
Call Proceeding
Setup
Call Proceeding
Alerting
Alerting
Connect
Connect
Connection Acknowledgment
Connection Acknowledgment
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—2-23
A typical Q.931 setup process might have a structure similar to the figure shown here.
The most common IE contains these details:
Bearer capability: Specifies a requested service, such as packet or circuit mode, data rate,
and type of information content
Call reference: Specifics a device to associate a particular message with a specific call
Called party number: Identifies the telephone number dialed
Calling party number: Identifies the origin telephone number
Call state: Describes the current status of a call in terms of the standard Q.931 state
machine
Progress indicators: Provides information about the progress of the callprogress (This
indicator is used to determine whether the originating and destination addresses are nonISDN.)
Cause: Identifies the reason a call was rejected or disconnected
Channel identification: Identifies a bearer channel (B channel)
Date and time: Logs the date and time
Display: Displays human-readable text (Q.931 can specify this with almost any message to
provide text for a liquid crystal display (LCD), for example.)
Service profile identification: Contains a service profile identifier (SPID)
Signal: Provides call status tones, such as dial tone, ringing, intercept, network congestion,
busy, confirm, answer, call waiting, off-hook warning, and tone off
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Cisco CallManager, Network Signaling, and Dial Plan
2-27
Station Definition and Call Flow
This topic discusses the Skinny Cisco IP Phone connectivity to Cisco CallManager and call
flow architecture.
Station Connectivity and Call Flow
Cisco IP Phone
Cisco IP Phone
Cisco CallManager
Station Off-Hook
Station Play Tone (Dial Tone)
Station Set Lamp (Steady)
Station Digit Dialed
Station Stop Tone (Dial Tone)
Station Digit Dialed
Station Digit Dialed
Station Call Information
Station Play Tone (Ringback)
Station Set Lamp (Blink)
Station Set Ringer (On)
Station Off-Hook
Station Play Tone (Off)
Station Set Lamp (Steady)
Station Start Media Transmission
Station Start Media Transmission
Station Start Media Reception
Station Start Media Reception
Station Set Ringer (Off)
Conversation
Station Stop Media Transmission
Station Stop Media Reception
Station Set Lamp (Off)
Station On-Hook
Station On-Hook
Station Stop Media Transmission
Station Stop Media Reception
Station Set Lamp (Off)
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—2-24
This figure shows the Skinny client call-setup process. Notice the lack of H.323 protocol
messages in the setup and teardown of this call. Cisco CallManager utilizes SCCP, which is a
stimulus-response type of architecture to handle the call. The Skinny devices report any events
occurring or actions taken during a VoIP session to Cisco CallManager.
This example displays an exchange of messages between two stateless clients.
The devices exchange all messages over TCP. The Skinny device sends the voice conversation
using RTP without passing any RTP data through Cisco CallManager. The figure details an onnet call.
Note
This is only a sample and may not match the exact sequence of messages.
The Skinny client uses the following protocols:
TCP/IP to or from one or more Cisco CallManager servers to send and receive stimulus
RTP/UDP/IP to or from a similar Skinny client or H.323 terminal for audio
2-28
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Skinny Intracluster Calls
1000 calls 1001
Cisco CallManager Cluster
Cisco CallManager A
Cisco CallManager B
Skinny
1000
Skinny
RTP Stream
1001
Registered
Cisco CallManager
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—2-25
In this example, two IP Phones register to Cisco CallManager servers within the same cluster.
An IP Phone registered to one Cisco CallManager server places a call to an IP Phone registered
to another Cisco CallManager server. The IP Phones use SCCP to communicate to the Cisco
CallManager servers. Cisco CallManager servers use a call control process to handle the
internal call routing.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Cisco CallManager, Network Signaling, and Dial Plan
2-29
Registration Process
Cisco
CallManager
DHCP
YEILDTFTP
Cisco
5
3
1
2
4
1. Inline power-capable switch sends FLP
2. Switch provides VLAN information to IP
Phone
3. IP Phone sends DHCP request; receives
IP information and TFTP server address
4. IP Phone gets configuration from TFTP
server
5. IP Phone registers with CCM server
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—2-26
This figure shows the steps an IP Phone takes to register with a Cisco CallManager server.
Here are the steps that are taken by an IP Phone after it is powered up:
Step 1
The Inline power-capable switch sends a Flash Link Pulse (FLP).
Step 2
The IP Phone uses Cisco Discovery Protocol (CDP) to get its VLAN number. (The
Mute, Handset, and Speaker buttons illuminate, then the Configuring VLAN button
illuminates.)
Step 3
The IP Phone sends a DHCP request to the DHCP server via a broadcast. If a static
IP address is configured on the IP Phone, gateway address, no DHCP server request
is made. The DHCP server sends the IP address, subnetwork mask, Domain Name
System (DNS), default gateway, TFTP server address, static or DHCP (configuring
IP).
Step 4
The IP Phone receives a configuration from the Cisco CallManager TFTP
(configuring IP).
The TFTP server address can be obtained in the following ways:
Configure the static address on the IP Phone by choosing option 8, option 32 alternate
TFTP must be set to Yes.
Choose option 150 (single IP address) from the DHCP server.
Choose option 66 (first IP address or DNS name).
Looked up by CiscoCM1.your.domain.
List of up to three Call Managers, region information and keyboard template, version of
code to run, and get new code (one time only).
2-30
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Step 5
The IP Phone registers with the first Cisco CallManager server in the list of possible
three (in the configuring Cisco CallManager list), then registration takes place.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Cisco CallManager, Network Signaling, and Dial Plan
2-31
Call Preservation
This topic discusses the preservation of an active call if a device loses the connection to a
Cisco CallManager server.
Device Requirements
• Active connection
maintenance:
– RTP streams between
devices
Cisco
CallManager 1A
SCCP
• Disconnect supervision:
MGCP
PSTN
Switch
RTP
– End user
– Timed
Cisco
CallManager 1B
IP Phone
Gateway
– MSF
• Switchover algorithm:
– Graceful
– Immediate
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—2-27
Cisco CallManager supports call preservation. Understanding call preservation methods is
valuable for troubleshooting Cisco CallManager problems. Different elements in a VoIP
network communicate to establish calls and connections between devices.
If an error affects the signaling path controlled by a Cisco CallManager server, the server
cannot control the affected calls. Some common types of errors include Cisco CallManager
server failure and network failure. However, as long as there is no network error to affect
communication between the devices, the devices can preserve the RTP streaming sessions and
keep the call alive.
The three device requirements for supporting call preservation include the following:
Active connection maintenance
Disconnect supervision
Switchover algorithm
Active connection maintenance occurs when a device is in an active streaming mode. When a
device detects that the link to its active Cisco CallManager server is out of service, such as a
Cisco CallManager server failure or network failure, the device will maintain active media
connections (streams).
Note
2-32
In an active media connection, the send and receive RTP ports are active.
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Devices must provide at least one of the following disconnect supervision mechanisms for any
media connections preserved during system failure:
End-user release
Timed
Media Streaming Failure (MSF)
The switchover algorithms decide when a device will fail over to the secondary Cisco
CallManager server when the device detects that its link is out of service to the primary Cisco
CallManager server. The two switchover algorithms are as follows:
Graceful
Immediate
The two types of survival endpoints are IP Phones and MGCP gateways. If any of these devices
are streaming to each other, the stream is preserved even if the devices can not signal the Cisco
CallManager.
Calls made through an H.323 gateway can fail if the H.225 session between the Cisco
CallManager and the H.323 gateway is terminated. It does not matter what type of PSTN
interface is on the gateway, whether FXS, FXO, T1 PRI, T1 channel associated signaling
(CAS), because the signaling back to the Cisco CallManager is still H.323. The condition for a
dropped H.323 call is as follows: an H.323 call drops if the H.225 session is broken.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Cisco CallManager, Network Signaling, and Dial Plan
2-33
Cisco CallManager Trace and Trace Collection
Tool
This topic discusses the purpose, configuration, and examples of Cisco CallManager trace files.
Starting with Cisco CallManager version 4.0, the trace gathering process has changed. Cisco
CallManager versions 4.0 and above use a trace gathering client that is operated from a
desktop.
Cisco CallManager Trace
.
Trace performs three main functions:
• Configures trace parameters
• Collects trace parameters
• Analyzes trace data for troubleshooting problems
Major change in trace gathering process between Cisco CallManager 3.x
and 4.0(1)
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—2-28
To perform detailed troubleshooting of your IP telephony network, Cisco provides Cisco
CallManager trace capabilities. The Cisco CallManager server is capable of writing detailed
logging information to trace files. These trace files are usually in an ASCII text format that you
can read using an application such as Microsoft Windows Notepad.
In addition to the text-based output of the trace tool, Cisco CallManager serviceability provides
the web-based trace tool to assist the system administrator and support personnel in
troubleshooting Cisco CallManager problems. The trace tool performs the following three main
functions:
Configures trace parameters
Collects trace parameters
Analyzes trace data for troubleshooting problems
The trace and alarm tools work together. You configure trace and alarm settings for Cisco
CallManager services. You can direct alarms to the Microsoft Windows 2000 Event Viewer,
syslog, system diagnostic interface (SDI), SDL trace log files, or to all destinations. You can
base traces for Cisco CallManager services on debug levels, specific trace fields, and Cisco
CallManager devices such as telephones or gateways. You can perform a trace on the alarms
sent to the SDI or SDL trace log files.
2-34
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Cisco CallManager logs alarms in the SDI trace log file by checking the trace check box and
the enable trace file log check box in trace configuration, and the SDI alarm destination check
box in alarm configuration.
Trace Filter Settings
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—2-29
To configure Cisco CallManager for traces, click the Trace menu from the Cisco CallManager
Serviceability screen and choose Configuration. Then choose the Cisco CallManager server in
the cluster and the service you would like to monitor. You will usually gain the most detailed
troubleshooting information by monitoring the Cisco CallManager service. The figure displays
the available monitoring options for the Cisco CallManager service.
The trace configuration tool automatically provides the time and date of the trace. The trace
collection tool provides seven debug levels, ranging from error to detailed. Refer to the Debug
Levels table shown here when choosing an appropriate debug level.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Cisco CallManager, Network Signaling, and Dial Plan
2-35
Debug Levels
Level
Description
)VVSV
Traces alarm conditions and events. Used for all traces generated in abnormal
path. Uses minimum amount of CPU cycles.
7TIGMEP
Traces all error conditions plus process and device initialization messages.
7XEXI
8VERWMXMSR
Traces all special conditions plus subsystem state transitions that occur during
normal operation. Traces call-processing events.
7MKRMJMGERX
Traces all state transition conditions plus media layer events that occur during
normal operation.
)RXV])\MX
Traces all significant conditions plus entry and exit points of routines. Not all
services use this trace level (for example, Cisco CallManager does not).
%VFMXVEV]
Traces all entry/exit conditions plus low-level debugging information.
Note: Do not use this trace level with the Cisco CallManager service or the
Cisco IP Voice Media Streaming Application service during normal operation.
(IXEMPIH
Traces all arbitrary conditions plus detailed debugging information.
Note: Do not use this trace level with the Cisco CallManager service or the
Cisco IP Voice Media Streaming Application service during normal operation.
This table defines the trace fields of Cisco CallManager. During the laboratory exercises, you
will configure the Trace tool to analyze or baseline successful calls within your cluster.
2-36
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Trace Fields
Field Name
Description
)REFPI,1IWWEKI
8VEGI
Activates trace of H245 messages.
)REFPI(8()
8VEGI
Activates the logging of ISDN type of DT-24+/DE-30+ device
traces.
)REFPI46-8VEGI
Activates trace of PRI devices.
)REFPI-7(2
8VERWPEXMSR8VEGI
Activates ISDN message traces. Used for normal debugging.
)REFPI, +EXIOIITIV8VEGI
Activates trace of H.225 devices. Used for normal debugging.
)REFPI1MWGIPPERISYW
8VEGI
Activates trace of miscellaneous devices.
)REFPI'SRJIVIRGI
&VMHKI8VEGI
Activates trace of conference bridges (CFBs). Used for normal
debugging.
)REFPI1YWMGSR,SPH
8VEGI
Activates trace of music on hold (MOH) devices. Used to trace
MOH device status, such as “registered with Cisco
CallManager,” “unregistered with Cisco CallManager,” and
“resource allocation processed successfully or failed.”
)REFPI'16IEP8MQI
-RJSVQEXMSR7IVZIV
8VEGI
Activates Cisco CallManager real-time information traces that
the real-time information server uses.
)REFPI'(68VEGI
Activates traces for Call Detail Record (CDR).
)REFPI%REPSK8VYRO
8VEGI
Activates trace of all analog trunk gateways.
)REFPI%PP4LSRI
(IZMGI8VEGI
Activates trace of telephone devices. Trace information
includes Cisco IP SoftPhone devices. Used for normal
debugging.
)REFPI1848VEGI
Activates trace of media termination point (MTP) devices. Used
for normal debugging.
)REFPI%PP+EXI[E]
8VEGI
Activates trace of all analog and digital gateways.
)REFPI*SV[EVHERH
1MWGIPPERISYW8VEGI
Activates trace for call forwarding and all subsystems not
covered by another check box. Used for normal debugging.
)REFPI1+'48VEGI
Activates trace for MGCP devices. Used for normal debugging.
)REFPI1IHME6IWSYVGI
1EREKIV8VEGI
Activates trace for Media Resource Manager (MRM) activities.
)REFPI7-4'EPP
4VSGIWWMRK8VEGI
Activates trace for SIP call processing activities.
Note: Do not check this box during normal system operation.
)REFPI7-47XEGO8VEGI Activates trace for SIP stack trace.
Caution
Do not check the Enable Miscellaneous Trace box during normal system operation.
When performing a trace, you should only choose the values that you want to monitor.
Choosing the values to monitor ensures that Cisco CallManager does not record unnecessary
information, which saves time and makes it easier to analyze the information.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Cisco CallManager, Network Signaling, and Dial Plan
2-37
Gathering Trace Files
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—2-30
You can display trace files in formats other than text. If you choose to generate extensible
markup language (XML) traces, Cisco CallManager allows you to analyze the files directly
from the Cisco CallManager Serviceability tool. To reach the Trace Analysis window, choose
the Trace menu and choose Analysis. You can view both SDL and SDI trace files.
An SDL trace log file contains call-processing information from services such as Cisco
CallManager Computer Telephony Integration (CTI) Manager and Cisco TFTP. The system
traces the SDL of the call and logs state transitions into a log file.
An SDI trace log file contains information for all Cisco CallManager services. The system
traces SDI information from the services, logs run-time events, and traces them to a log file.
2-38
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Viewing XML Trace Files
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—2-31
After you choose the appropriate file, you can view the data in a web-page format. Cisco
CallManager displays the output in a readable format using tables. The information column
provides the data corresponding to the events traced.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Cisco CallManager, Network Signaling, and Dial Plan
2-39
Troubleshooting Trace Setting: Cisco
CallManager 4.0
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—2-32
This figure is the web page access when setting the Troubleshooting Trace Setting:
To use the Trace Collection Tool, you first have to set up the Troubleshooting Trace
Setting.
Go to the Cisco CallManager Serviceability web page and choose Troubleshooting Trace
Setting. This web page is where you choose the services that you want to run traces on.
You can choose the services per all servers in a cluster or per an individual server.
2-40
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Trace Setting: Cisco
CallManager 4.0 (Cont.)
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—2-33
If you were to access the Troubleshooting Trace Setting screen and saw what appears in the
figure, someone has already set the troubleshooting tracing.
The choice is to either reset the tracing or edit the current tracing and apply the change.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Cisco CallManager, Network Signaling, and Dial Plan
2-41
Troubleshooting Trace Setting: Cisco
CallManager 4.0 (Cont.)
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—2-34
If troubleshooting traces have been already set, when you enter the Trace Configuration
web page, there will be a message at the top of the page indicating that troubleshooting
traces have been set for the service details that you have accessed.
You can add check boxes to the trace filters settings without causing any problem to the
troubleshooting trace settings.
Once the troubleshooting trace settings have been set up and the details setting per service
have been set up, it is time to wait for the traces to capture the information needed to
troubleshoot.
2-42
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Trace Collection Tool: Cisco
CallManager 4.0
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—2-35
This figure shows where to download the Cisco CallManager Trace Collection tool. Download
the tool from the Plug-in Trace Collection tool icon.
Use the Trace Collection tool to collect trace information for any Cisco CallManager
service and the time and date of the trace for that service. Trace collection collects and zips
the chosen files.
Once the Cisco CallManager Trace Collection tool is downloaded to your desktop under a
folder in your program files named “Cisco CallManager Serviceability,” you can launch the
Trace Collection tool by clicking the Trace Collection tool icon that is created on your
desktop during installation.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Cisco CallManager, Network Signaling, and Dial Plan
2-43
Trace Collection Tool: Cisco
CallManager 4.0 (Cont.)
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—2-36
Once the trace collection tool has launched for the first time, you will need to set up the
access information. This figure shows the Cisco CallManager Trace Collection tool
authentication window.
This setup is critical and requires a Cisco CallManager DNS name or IP address. The
username and password entered in this window must match the username and password of
the administrator.
After you enter the appropriate access information, click the Next > button.
2-44
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Trace Collection Tool: Cisco
CallManager v4.0 (Cont.)
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—2-37
After you enter the appropriate access information as indicated on the previous slide and
click the Next > button, you will get a screen pop up looking like this figure.
This figure shows three tabs: Select CallManager Services, Select CallManager
Applications, and Select System Traces.
The Select CallManager Services tab allows you to choose which services you want. Make
sure that the troubleshooting trace settings that you have set up relative to services match
the trace collection that you indicate on this screen. In this figure, all services have been
chosen.
If you need to collect traces on applications or system traces or both, click on the tabs and
set up your trace collection criteria.
After you set up the services that you want to collect traces on, click the Next > button.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Cisco CallManager, Network Signaling, and Dial Plan
2-45
Trace Collection Tool: Cisco
CallManager 4.0 (Cont.)
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—2-38
This figure shows what can be chosen in the Select CallManager Applications tab.
Choose the appropriate application traces needed.
Trace Collection Tool: Cisco
CallManager 4.0 (Cont.)
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—2-39
This figure shows the Select System Traces tab, which provides the list of system logs that
you can collect. You can choose some or all of the logs.
Once you have set up all the trace criteria, click the Next > button.
2-46
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Trace Collection Tool: Cisco
CallManager 4.0 (Cont.)
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—2-40
This figure shows the Collect Traces window.
If you want to collect all available traces, choose Collect All Available Traces.
If you want to collect traces within a given date range, choose Traces for a Date Range,
choose a time zone from the Select Time Zone drop-down list box, and enter a date and
time in the From Time and To Time fields.
From the Zip File Location field, click the Browser button and enter the path name on the
client machine where you want to store the zip file.
Note
The default path is C:\ and the default file name is CiscoCallManagertraceCollection.zip. If
the file already exists, you get prompted to either replace the file or to provide a different
path name.
You can collect the output zip file as a single zip file or you can opt for the files to be split
into multiple files. To split the output zip files into multiple files, choose Create Multi
Volume Zip File when zipping the files option and enter a value for the Multiple Volume
File size, which would indicate the size of each file that gets spilt.
From the Compression Factor pull-down list, choose the compression factor value of the
zip file. The Trace Collection tool only allows you to enter values of 0 through 9 because
any higher values would only be useful if the files to be zipped were a binary type.
Click Collect Traces, when you are done.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Cisco CallManager, Network Signaling, and Dial Plan
2-47
Trace Collection Tool: Cisco
CallManager v4.0 (Cont.)
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—2-41
When you choose Collect traces, an alert message will appear letting you know the size of the
files and to confirm the trace collection. If the size of the file is OK to download, click Yes.
Note
2-48
At any time, only a single user can collect traces from a cluster. If more than a single user
tries to collect traces, a message indicates that another user is already collecting traces.
This check can only be performed when the publisher server is active in the network.
Otherwise, a warning is displayed, and you must cancel the trace collection. The limitation
on multiple trace collection stems from the high CPU utilization required in the trace
collection process.
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Trace Collection Tool: Cisco
CallManager 4.0 (Cont.)
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—2-42
After you confirm the size of the zip file to be downloaded, you will see a window display that
shows the progress of the zipping process as seen in this figure.
Trace Collection Tool: Cisco
CallManager 4.0 (Cont.)
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—2-43
During the zipping process, there may be traces or logs or both that do not exist. If the Trace
Collection tool goes out to collect on something that is not there, a window will appear as seen
in the figure. This indicates that files were not found. The zipping process will not stop, so just
click OK and wait until after the collection process has finished to research the missing files.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Cisco CallManager, Network Signaling, and Dial Plan
2-49
Example Trace: H.323 Client to
IP Phone
''1`7XEXMSR-RMX -RFSYRH7XMQz
3JJ,SSO1IWWEKI-(XGT,ERHPI!\H
''1`7XEXMSR( WXEXMSR3YXTYX(MWTPE]8I\X
XGT,ERHPI!\H(MWTPE]!
''1`(MKMXEREP]WMWQEXGL JUGR!GR!TWW!
HH!
''1`(MKMXEREP]WMWQEXGL JUGR!GR!TWW!
HH!
''1`(MKMXEREP]WMWQEXGL JUGR!GR!TWW!
HH!
''1`(MKMXEREP]WMWQEXGL JUGR!GR!TWW!
HH!
''1`(MKMXEREP]WMWQEXGL JUGR!GR!TWW!
HH!
''1`(MKMXEREP]WMWEREP]WMWVIWYPXW
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—2-44
In this example, a Cisco IP Phone (1001) is off-hook. The trace shows the unique messages;
shows the TCP handle; and shows the calling number, which is displayed on the Cisco IP
Phone. There is no called number at this point, because the user did not dial any digits. It is
very important that you first notice the TCP handle identifier of the IP Phone. This unique
identifier allows you to follow the individual IP Phone through the traced call flow. Each IP
Phone has its own unique TCP handle. In this example, you can see that Cisco CallManager
tracks the IP Phone assigned extension 1001 by its TCP handle of 0x5138d98.
In the trace, user 1001 dials 3333, one digit at a time—notice the dialed digits (dd) input. The
digit analysis process of Cisco CallManager is currently active and analyzing the digits to
discover where Cisco CallManager needs to route the call.
2-50
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Example Trace: H.323 Client to
IP Phone (Cont.)
`'EPPMRK4EVX]2YQFIV!
`(MEPMRK4EXXIVR!
`(MEPMRK6SYXI4EXXIVR6IKYPEV)\TVIWWMSR! `4VIXVERWJSVQ(MKMX7XVMRK!
`4VIXVERWJSVQ4SWMXMSREP1EXGL0MWX!
`'SPPIGXIH(MKMXW!
`4SWMXMSREP1EXGL0MWX!
''1`0SGEXMSRW3VMK!&;!(IWX!
&;! MQTPMIWMRJMRMXIF[EZEMPEFPI
''1`7XEXMSR(z WXEXMSR3YXTYX'EPP-RJS
'EPPMRK4EVX]2EQI!'EPPMRK4EVX]!
'EPPIH4EVX]2EQI!'EPPIH4EVX]!
XGT,ERHPI!\H
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—2-45
In the trace shown here, Cisco CallManager analyzed the digits, matched calling and called
parties, and parsed the information.
In the trace, the number 0 indicates the originating location, and the number 1 indicates the
destination location. The bandwidth of the originating location is determined by BW = –1. The
value –1 implies that the bandwidth is infinite. The bandwidth is infinite because the call
originated from a Cisco IP Phone located in a LAN environment. The bandwidth of the
destination location is determined by BW = 64. The call destination is a telephone located in
the PSTN, and the codec type used is G.711 (64 kbps).
The trace shows the calling party and called party information. In this example, the calling
party name and number are identical because Calling Party Name is blank.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Cisco CallManager, Network Signaling, and Dial Plan
2-51
Example Trace: H.323 Client to
IP Phone (Cont.)
''1`3YX1IWWEKI ,7IXYT1WK 4VSXSGSP!,4VSXSGSP
''1`11ERC-H! MIT!HWP!WETM!GIW!-T%HHV!IEG
-T4SVX!
''1`7XEXMSR( WXEXMSR3YXTYX'EPP-RJS'EPPMRK4EVX]2EQI!
'EPPMRK4EVX]!'EPPIH4EVX]2EQI!'EPPIH4EVX]!XGT,ERHPI!\H
''1`-R1IWWEKI ,%PIVX1WK 4VSXSGSP!,4VSXSGSP
''1`7XEXMSR( WXEXMSR3YXTYX3TIR6IGIMZI'LERRIPXGT,ERHPI!\H
Q]-4IEG ''1`7XEXMSR( 'SRJIVIRGI-(QWIG4EGOIX7M^I
GSQTVIWWMSR8]TI 1IHMEC4E]PSEHC+9PE[O
''1`,-RXIVJEGI TEXLWIWXEFPMWLIHMT!IEGTSVX!
''1`,-RXIVJEGI 30'SYXKSMRKGSRJMVQMT!IEGTSVX!
''1`1IHME1EREKIV [EMXC%Y'SRRIGX-RJS VIGIMZIHVIWTSRWI
JSV[EVHMRK
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—2-46
Before reviewing this trace, it is important to understand the terms associated with H.323. A
device uses several protocols when establishing an H.323 session. One protocol is H.225,
which the device uses for call signaling, and is a subset of Q.931. Another protocol is H.245,
which the device uses for capability exchange. One of the more important functions of H.245 is
the codec type negotiation, such as G.711 or G.729, between the calling and called side. When
the capability exchange is complete, the next important function of H.245 requires performing a
UDP port negotiation between the calling and called sides.
This trace shows that the device initializes the H.323 code and sends an H.225 setup message.
You can also see the traditional High-Level Data Link Control (HDLC) speech application
programming interface (SAPI) messages, the IP address of the called side in hexadecimal
format, and the port numbers.
This trace shows the calling and called party information and also the H.225 alerting message.
This trace also shows the compression type used for this call, G.711 mu-law.
After the device sends an H.225 alert message, the next portion of the H.323 negotiation is to
initialize H.245. This trace shows the calling party and called party information and the H.245
messages. Notice that the TCP handle value is the same as before, indicating the continuation
of the same call.
2-52
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Example Trace: H.323 Client to
IP Phone (Cont.)
''1`-R1IWWEKI ,'SRRIGX1WK 4VSXSGSP!
,4VSXSGSP
''1`7XEXMSR( WXEXMSR3YXTYX'EPP-RJS
'EPPMRK4EVX]2EQI!'EPPMRK4EVX]!'EPPIH4EVX]2EQI!
'EPPIH4EVX]!XGT,ERHPI!\H
''1`1IHME'SSVHMREXSV [EMXC%Y'SRRIGX-RJS-RH
''1`7XEXMSR( WXEXMSR3YXTYX7XEVX1IHME8VERWQMWWMSR
XGT,ERHPI!\HQ]-4IEG ''1`7XEXMSR( 6IQSXI-T%HHVIEG 6IQSXI6XT4SVX2YQFIVQWIG4EGOIX7M^I
GSQTVIWWMSR8]TI 1IHMEC4E]PSEHC+9PE[O
''1`0SGEXMSRW3VMK!&;!(IWX!&;! MQTPMIWMRJMRMXI
F[EZEMPEFPI
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—2-47
This trace displays the H.225 connect message. When the device receives the H.225 connect
message, the call connects. Notice how the devices prepare the RTP port numbers and identify
the proper codec for audio steaming. Cisco CallManager runs a second check on the available
bandwidth in the location. Because there are no bandwidth restrictions in this example, the call
processing continues.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Cisco CallManager, Network Signaling, and Dial Plan
2-53
Example Trace: H.323 Client to
IP Phone (Cont.)
''1`7XEXMSR-RMX -RFSYRH7XMQz 3R,SSO1IWWEKI-(
XGT,ERHPI!\H
''1`'SRRIGXMSR1EREKIV[EMXC%Y(MWGSRRIGX6IUYIWX
78347)77-32
''1`1IHME1EREKIV [EMXC%Y(MWGSRRIGX6IUYIWX 7XST7IWWMSR
WIRHMRKHMWGSRRIGXXS ERHVIQSZIGSRRIGXMSRJVSQPMWX
''1`(IZMGI7)4'9R6IKMWXIVW[MXL7(00MRO
XSQSRMXSV2SHI-(!
''1`7XEXMSR( WXEXMSR3YXTYX'PSWI6IGIMZI'LERRIP
XGT,ERHPI!\HQ]-4IEG ''1`7XEXMSR( WXEXMSR3YXTYX7XST1IHME8VERWQMWWMSR
XGT,ERHPI!\HQ]-4IEG ''1`-R1IWWEKI ,6IPIEWI'SQTPIXI1WK 4VSXSGSP!
,4VSXSGSP
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—2-48
This trace shows that Cisco CallManager has received an on-hook message from the Cisco IP
Phone (1001). As soon as Cisco CallManager receives an on-hook message, it sends the H.225
and Skinny disconnect messages to close the ports and disconnect the call. This final message
verifies that the H.225 release is complete. This indicates that the devices have terminated the
call.
2-54
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Summary
This topic summarizes the key points discussed in this lesson.
Summary
• The Cisco CallManager servers in a cluster use SDL links to establish calls
between devices registered to different Cisco CallManager servers, while media
connections between devices involved in a call are RTP streaming sessions that
occur directly between the devices.
• ITU-T H.323 specifies the protocol architecture for multimedia communications
over unreliable networks such as IP.
• MGCP consists of three main elements: endpoints, commands, and events or
signals.
• SIP is an ASCII-based, application-layer control protocol that VoIP equipment can
use to establish, maintain, and terminate calls between two or more clients.
• Q.931 is the connection control protocol for ISDN connections and is roughly
comparable to TCP in the Internet protocol stack.
• The H.323 recommendation for the stations provides mechanisms for
establishing, controlling, and clearing information flows.
• The three device requirements for supporting call preservation are active
connection maintenance, disconnect supervision, and switchboard algorithm.
• The Cisco CallManager Trace tool performs three main functions: configuring
trace parameters, collecting trace parameters, and analyzing trace data for
troubleshooting problems.
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—2-49
References
For additional information, refer to these resources:
Cisco CallManager trace file case studies:
http://www.cisco.com/warp/public/788/
Cisco SIP Proxy Server:
http://www.cisco.com/univercd/cc/td/doc/product/voice/sipproxy/
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Cisco CallManager, Network Signaling, and Dial Plan
2-55
Quiz
Use the practice items here to review what you learned in this lesson. The correct answers are
found in the Quiz Answer Key.
Q1)
Which of these protocols does Cisco CallManager NOT use for signaling?
A)
B)
C)
D)
Q2)
Which of the following H.323 components are used for codec negotiation?
A)
B)
C)
D)
Q3)
peer-to-peer
client/server
Layer 3
none of the above
Which of the following call preservation failover methods do Cisco IP Phones support?
A)
B)
C)
D)
2-56
H.225
H.245
RTP
H.450.1
What type of protocol is MGCP?
A)
B)
C)
D)
Q7)
H.323
MGCP
SCCP
none of the above
Which two of these H.323 functions does Cisco CallManager handle when connecting
a call between two Skinny-based Cisco IP Phones? (Choose two.)
A)
B)
C)
D)
Q6)
proxy server
redirect server
registrar server
registration server
ISDN uses the Q.931 protocol for connection control. Which other protocol uses Q.931
for this same purpose?
A)
B)
C)
D)
Q5)
H.225
H.225 RAS
H.245
H.450
Which of these servers is NOT a valid SIP server?
A)
B)
C)
D)
Q4)
MGCP
SCCP
RTP
H.323
graceful
immediate
initial
all of the above
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Lesson Review Answer Key
Q1)
C
Q2)
C
Q3)
D
Q4)
A
Q5)
A, B
Q6)
B
Q7)
A
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Cisco CallManager, Network Signaling, and Dial Plan
2-57
2-58
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Cisco
CallManager Dial Plan, Call
Routing, and Media Resources
Overview
Cisco CallManager provides the dial plan for most devices in an IP telephony system. This dial
plan is often the most complex troubleshooting area of your IP telephony network. Cisco IP
telephony functionality requires the use of media resources. Media resources provide services
such as transcoding, conferencing, music on hold (MOH), and media termination. Cisco
CallManager provides media resources that allow software and hardware resources to coexist.
This lesson will teach you the most common troubleshooting problems with the Cisco
CallManager dial plan and media resources. You will learn the direction that you should take
when encountering a new problem.
Relevance
The dial plan in the Cisco IP telephony arena is very flexible. You can accomplish almost any
task with the proper combination of route patterns, route plans, dial peers, translation patterns,
partitions, and calling search spaces.
The same flexibility that allows Cisco CallManager to adapt to almost any situation also makes
the Cisco CallManager dial plan complicated to troubleshoot. To develop better
troubleshooting skills in the Cisco CallManager dial plan, you need a sound understanding of
the way that the dial plan fits together.
Objectives
Upon completing this lesson, you will be able to:
List the primary functions of Cisco CallManager
Define partition and calling search space
Demonstrate problem-isolation techniques when troubleshooting route patterns in a dial
plan
Demonstrate Cisco CallManager problem-isolation methods for solving configuration
problems related to centralized call-processing models
Apply troubleshooting methods for solving media resource issues
Identify and troubleshoot media resource issues
Learner Skills and Knowledge
To benefit fully from this lesson, you must have these prerequisite skills and knowledge:
Basic understanding of Cisco CallManager dial plan
Outline
The outline lists the topics included in this lesson.
Outline
• Overview
• Cisco CallManager Overview
• Partitions and Calling Search Spaces
• Troubleshooting Route Patterns
• Distributed Call-Processing Model
• Centralized Call-Processing Model
• Troubleshooting Media Resources
• Summary
• Quiz
© 2004 Cisco Systems, Inc. All rights reserved.
2-60
Cisco IP Telephony Troubleshooting (IPTT) v4.0
IPTT v4.0—2-3
Copyright © 2004, Cisco Systems, Inc.
Cisco CallManager Overview
This topic provides an overview of Cisco CallManager dial plan and its functionality.
Cisco CallManager Dial Plan
WAN Down or Insufficient
Resources
User dials 9-408-526-1212
Route Pattern
9.@
User dials 9-408-526-1212
No Digit
Manipulation
Route List
“Remotes”
Digit Manipulation
1. Discard access code “9”
2. Send 408-526-1212 to GK
2nd
Choice
1st
Choice
Route Group
“IP WAN”
Digit Manipulation
1. Discard access code “9”
2. Insert 1
3. Send 1-408-526-1212 to PSTN
Route Group
“Local PSTN”
ARQ
408-526-1212
Send 408-526-1212 in
H.323 setup
IOS Gatekeeper
PSTN
IP WAN
Remote CM
Cisco CallManger strips leading digits
and uses internal dial length (1212).
© 2004 Cisco Systems, Inc. All rights reserved.
Local gateway receives DID (1212) and
sends internal dial length to Cisco CallManager.
IPTT v4.0—2-4
Cisco CallManager provides the dial plan for most devices as one of its primary functions.
Cisco CallManager accomplishes this in one of the following two ways:
Defining all calls or route patterns in the programming of Cisco CallManager. You can use
this method for MGCP gateways that may or may not use Cisco IOS software and, to an
extent, H.323 gateways.
Placing part of the dial plan in the gateway itself or in a Cisco IOS software-based
gatekeeper.
The dial plan in the Cisco IP telephony arena is very flexible. An administrator can accomplish
almost any task with the proper combination of route patterns, route plans, dial peers,
translation patterns, partitions, and calling search spaces.
This flexibility can both help and hinder troubleshooting. The same flexibility that allows Cisco
CallManager to adapt to almost any situation also makes the Cisco CallManager dial plan
complicated to troubleshoot. To develop better troubleshooting skills in the Cisco CallManager
dial plan, you need a sound understanding of the way that the dial plan fits together.
The Cisco CallManager dial plan includes the following areas: route lists and groups, route
patterns, translation patterns, dial peers, partitions and calling search spaces, and overlapping
dial plans. Having overlapping dial plans is something that can be accomplished; however, this
capability is not really included in the Cisco CallManager dial plan.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Cisco CallManager, Network Signaling, and Dial Plan
2-61
Route Plan Visual Objective
San Jose
Dallas
Secondary Voice Path
Prepend “1408” and send to PSTN
Gatekeeper(s)
Users required to dial
seven digits for intersite
calls “526-1111”
A
A
PSTN
(817) 283-xxxx
Five-Digit Internal Dialing
(408) 526-xxxx
Five-Digit Internal Dialing
Primary Voice Path
Strip “52” and deliver
IP WAN
© 2004 Cisco Systems, Inc. All rights reserved.
61111 to remote CCM
IPTT v4.0—2-5
In the figure, the caller in Dallas dials a seven-digit number to reach a telephone in San Jose.
The first route configured in Cisco CallManager sends the call over the IP WAN. The route can
send the call as seven digits or as a shorter number in route group programming. If a gateway
sends a seven-digit number, the Cisco CallManager at the destination needs to reduce the
incoming number to a length that matches its dial plan.
If the IP WAN connection is down or at its bandwidth limit, the Cisco CallManager selects the
next route group, which contains the PSTN trunks. Because the PSTN cannot access San Jose
from Dallas with seven digits, Cisco CallManager must insert the “1 + area code” to provide
the proper number of digits to the PSTN.
If all routes are in use or down, the Cisco CallManager returns a fast busy (or reorder) tone to
the calling telephone.
Tip
2-62
If users are getting a fast busy tone after taking the IP Phone off-hook or after pressing the
first digit of a number they are calling, this is most likely a CallManager configuration issue.
Conversely, if the users are getting a fast busy tone after dialing all digits, you can start
troubleshooting gateway issues. You can diagnose the issue to be a CallManager or
gateway issue by using traces; however, gathering traces is not always the fastest
approach.
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Understanding Route Plans
User Dials Number
Route Pattern
Route List
1st
Choice
2nd
Choice
Route Group
Route Group
1st
Choice
Device
(gateway)
2nd
Choice
Device
(gateway)
1st
Choice
Device
(gateway)
2nd
Choice
Device
(gateway)
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—2-6
The figure demonstrates the relationship among the major components of the Cisco
CallManager route plan. The major components are as follows:
Route list: The route list directs calls to one or more route groups that allow for call
routing based on preference. You can look at this group as a trunk group. The route list can
direct all calls to the primary route group or use the secondary route group if the primary is
unavailable.
Route pattern: The route pattern represents an E.164 address or address range that has
special route handling needs. The route pattern directs calls to a single route list to perform
this route handling. The route pattern performs the digit manipulation (such as subtracting
or adding digits) for the matched pattern. Only one digit manipulation table exists for any
given route pattern.
Note
E.164 is the numbering format that is recommended by the ITU-T.
Route group: The route group directs calls to one or more devices that allow for call
routing based on preference. You can look at this group as a trunk group. The route group
can direct all calls to the primary device or use the secondary route group if the primary is
unavailable.
Device: A device is typically a gateway or H.323 endpoint (such as Microsoft
NetMeeting).
When configuring the route plan, you must create it in reverse order. You must first create the
gateways so that they appear as options in the drop-down menus of the route group
configuration. Next, configure the route groups so that they will appear when you configure
route lists. Then, you will have the route lists as selectable options for route patterns.
Cisco CallManager attempts to match the number dialed to the route patterns. The route pattern
points to the route list, which contains an ordered list of route groups. The route groups contain
ordered lists of gateways.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Cisco CallManager, Network Signaling, and Dial Plan
2-63
Cisco CallManager attempts to use the first option in the route list. If all of the gateways in that
route group are busy or unavailable, Cisco CallManager attempts the next route group in the
route list. Cisco CallManager continues this course of action until it either finds an available
gateway or exhausts all route groups.
2-64
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Internal vs. External Numbers
DN
(Internal)
Route Pattern
(External)
Line #s
4005
9.1866xxxxxxxx
GRP Pick
3510
9.@
Park
200
9.1[2-9]xx[2-9]xxxxxx
MEET_ME_CONFERENCE
361x
9.1xxxxxxxxxx
9.11
9.91!
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—2-7
The following two types of numbers exist in the Cisco CallManager route plan:
Directory number (DN): A DN is a number that you can assign directly to an IP Phone or
to other features such as hunt or pickup groups. Some DNs, such as Meet-Me conference
numbers and call park numbers, may contain wildcards.
Route pattern: A route pattern is any number or range of numbers that you can use to
access a destination outside the Cisco CallManager cluster. This number or pattern may be
a specific number or may contain wildcards.
During digit analysis, the Cisco CallManager looks at all route patterns, including both internal
and external route patterns.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Cisco CallManager, Network Signaling, and Dial Plan
2-65
Most Commonly Used Wildcards
x
Single digit (0 through 9)
@
North American Numbering Plan
!
One or more digits (0 through 9)
[x-y]
Generic range notation
[^x-y]
Exclusion range notation
.
Terminates access code
#
Terminates interdigit timeout
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—2-8
The figure shows a list of the commonly used route pattern wildcards. Route pattern wildcards
and special characters allow a single route pattern to match a range of numbers (or addresses).
Use these wildcards and special characters to build instructions that enable Cisco CallManager
to manipulate a number before sending it to an adjacent system.
2-66
Note
When a routing pattern uses the “at” symbol (@) wildcard, Cisco CallManager automatically
recognizes the octothorp— or pound sign (#)—as an end-of-dialing character for
international calls. For routing patterns that do not use the @ wildcard, you must include the
# in the routing pattern to use the # to signal the end-of-dialing.
Note
Although the # is more universally recognized, using the term “octothorp” is important to help
the term gain broader acceptance, especially among international audiences.
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Wildcards and Special Characters
Character Description
@
Examples
The “at” symbol (@) wildcard matches all
North American Numbering Plan (NANP)
numbers.
Each route pattern can have only one @
wildcard.
The route pattern 9.@ routes or blocks all numbers recognized
by the NANP.
The following route pattern examples show NANP numbers
encompassed by the @ wildcard:
0
1411
19725551234
101028819725551234
01133123456789
x
The x wildcard matches any single digit in The route pattern 9xxx routes or blocks all numbers in the
the range 0 through 9.
range 9000 through 9999.
!
The route pattern 91! routes or blocks all numbers in the range
The exclamation point (!) wildcard
matches one or more digits in the range 0 910 through 91999999999999999999999.
through 9.
?
The question mark (?) wildcard matches
zero or more occurrences of the
preceding digit or wildcard value.
+
The plus sign (+) wildcard matches one or The route pattern 91x+ routes or blocks all numbers in the
more occurrences of the preceding digit or range 9100 through 91999999999999999999999.
wildcard value.
[]
The square bracket ([ ]) characters
enclose a range of values.
The route pattern 813510[012345] routes or blocks all numbers
in the range 8135100 through 8135105.
-
The hyphen (-) character, used with the
square brackets, denotes a range of
values.
The route pattern 813510[0-5] routes or blocks all numbers in
the range 8135100 through 8135105.
^
The circumflex (^) character, used with
the square brackets, negates a range of
values. It must be the first character
following the opening bracket ([).
The route pattern 813510[^1-5] routes or blocks all numbers in
the range 8135106 through 8135109.
The route pattern 91x? routes or blocks all numbers in the
range 91 through 91999999999999999999999.
Each route pattern can have only one “^”
character.
.
The dot (.) character, used as a delimiter, The route pattern 9.@ identifies the initial 9 as the Cisco
separates the Cisco CallManager access CallManager access code in an NANP call.
code from the DN.
Use this special character, with the
discard digits instructions, to take off the
Cisco CallManager access code before
sending the number to an adjacent
system.
Each route pattern can have only one dot
(.) character.
*
The asterisk (*) character can provide an
extra digit for special dialed numbers.
Copyright © 2004, Cisco Systems, Inc.
You can configure the route pattern *411 to provide access to
the internal operator for directory assistance.
Troubleshooting Cisco CallManager, Network Signaling, and Dial Plan
2-67
Character Description
#
Examples
The octothorp, or pound sign (#),
character generally identifies the end of
the dialing sequence.
The # character must be the last character
in the pattern.
The route pattern 901181910555# routes or blocks an
international number dialed from within the NANP. The #
character after the last 5 identifies that 5 as the last digit in the
sequence.
This table lists Cisco CallManager Administration fields that require route patterns and shows
the valid entries for each field.
Here are the typcial route patterns used in a Cisco CallManager deployment,avoiding the use of
“at” symbol (@):
911
9.911
[2-8]11 (services)
9.[2-8]11 (services)
9.[2-9]xxxxxx (local)
9.[2-9]xx[2-9]xxxxxx (ECS)
9.1[2-9]xx[2-9]xxxxxx (LD)
9.011! (International)
9.011!# (International)
Note
2-68
The asterisk (*) and octothorp, or pound sign (#), are not valid digits in all fields.
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Field Entries
Field
Valid Entries
Call Park Number/Range
[^0123456789-]x*#
Calling Party Transform Mask
0123456789x*#
Called Party Transform Mask
0123456789x*#
Caller ID DN (Gateways)
0123456789x*#
Directory Number
[^0123456789-]+?!x*#+
Directory Number (Call Pickup Group)
0123456789
External Phone Number Mask
0123456789x*#
Forward All
0123456789*#
Forward Busy
0123456789*#
Forward No Answer
0123456789*#
Meet-Me Conference Number
[^0123456789-]+?!x*#+
Prefix Digits
0123456789*#
Prefix DN (Gateways)
0123456789*#
Route Filter Tag Values
[^0123456789-]x*#
Route Pattern
[^0123456789-]+?!x*#+.@
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Cisco CallManager, Network Signaling, and Dial Plan
2-69
Digit Collection
Route Pattern
User dial string:
918665551212
9.1866xxxxxxx
Match!
9.@ <7-digit dialing>
9.1612[2-9]xxxxxx
CCM actions:
No other patterns
could match
extend call
9.[2-9]xxxxxx
911
9.911
Longest match – Wins !
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—2-9
In the figure, the first digit (9) matches all of the route patterns. Digit analysis waits for more
digits before routing the call. After the user dials a “1,” digit analysis eliminates patterns 2, 4,
and 6. The third dialed digit (8) eliminates patterns 3 and 5. Thus, only one pattern is a possible
match. However, digit analysis continues to wait for further digits before forwarding the call.
At this point, Cisco CallManager performs a look ahead to see if there is a device, such as a
telephone or gateway, available to handle the call. If the digits dialed potentially match a route
pattern, Cisco CallManager looks through the route list, route groups, and gateways in an
attempt to find an available device to handle the call. If the telephone is nonfunctional, or if no
valid gateways can handle the call, Cisco CallManager instructs the telephone to play a fast
busy (or reorder) tone.
After the user dials the last digit (2), Cisco CallManager routes the call to the appropriate
device.
2-70
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Interdigit Timeout
Route Pattern
User dial string:
911 <timeout>
Matches 1 billion+ digit
strings
Matches 10 million digit
strings
Matches 1 digit string
© 2004 Cisco Systems, Inc. All rights reserved.
9.1866xxxxxxx
9.@
9.1612[2-9]xxxxxx
9.xxxxxxx
911
Match!
9.911
IPTT v4.0—2-10
This figure demonstrates an interdigit timeout while a user places a call. As the Cisco
CallManager collects digits, it attempts to match route patterns in the database. Interdigit
timeout occurs when the closest-matched route pattern has a number of possible digit-string
matches. Cisco CallManager has the interdigit timer set to 15 seconds by default. You can
change this parameter under Service Parameters, using the TimerT302_msec field.
In certain cases, such as dialing 911, you do not want the user to wait for an interdigit time out.
Instead, you will want to make sure that the call goes out with urgent priority. The Urgent
Priority checkbox in the Route Pattern configuration page is often used to force the immediate
routing of certain numbers such as 911 as soon as a route pattern is matched without waiting for
an interdigit time out.
Note
Think carefully before reducing this timer below 15 seconds. If you set the timer too short,
and the user pauses between digits while dialing, the user will hear a reorder tone. The timer
must balance in between fast and slow dialers.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Cisco CallManager, Network Signaling, and Dial Plan
2-71
Number Transformations
Users
A - 5062
B - 5063
Users Dialed
Numbers
A - 91234
B - 91324
C - 91432
Wildcards allow all
dialed numbers to match
one route pattern.
Route Pattern
Discard Digit
Instruction
9.1xxx
Discard “9”
C - 5064
Users Dialed
Numbers
Caller ID
D - 1000
From 5000
Extension “1000”
Rings
A – 1234
B – 1324
C – 1432
A – 1000
B – 1000
C – 1000
Users Dialed
Numbers
“x000”
A – 5000
B – 5000
C – 5000
“x000”
Transform
Called Number
Users Directory
Numbers
Transform
Calling Number
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—2-11
This figure demonstrates the many ways that you can manipulate the caller ID and the dialed
number to meet the needs of the customer. For example, three IP Phones each dial a different
number. Each of the dialed numbers must match a route pattern before you can manipulate
them any further.
You can manipulate the caller ID and the dialed number by doing the following:
Discarding the access code or other parts of the dialed number. If you are not using the @
wildcard in the dial plan, you may only discard the access code (that portion of the dialed
number before the dot). All other discard instructions are only valid with the @ wildcard.
Completely or partially transforming the caller ID of the calling party. If you perform a
complete or partial transformation of the caller ID of the calling party, manipulate the caller
ID in this sequence:
Step 1
External Phone Number Mask (from line key programming)
Step 2
Calling Party Transform Mask (from route pattern, route group, or translation
pattern programming)
Completely or partially transforming the number dialed. If you perform a complete or
partial transformation of the number dialed, manipulate the called number in this sequence:
2-72
Step 1
Discard instructions (from route pattern, route group, or translation pattern
programming)
Step 2
Called Party Transform Mask (from route pattern, route group, or translation pattern
programming)
Step 3
Prefix Digits (from route pattern, route group, or translation pattern programming)
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Partitions and Calling Search Spaces
This topic discusses the use of partitions and calling search spaces in a voice network.
Class of Service
CoS 1
CoS 2
CoS 3
Lobby
Employee
Executive
Who can you call?
IP WAN
911
Local
PSTN
Long Distance
Employee
© 2004 Cisco Systems, Inc. All rights reserved.
Executive
International
IPTT v4.0—2-12
To provide a means for applying a class of service use partitions and call search space.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Cisco CallManager, Network Signaling, and Dial Plan
2-73
Partition Definition
Partition
Lobby
Partition
Employee
Partition
Executive
Partition
Gateway
Partition
911
LobbyPT
EmployeePT
ExecutivePT
Local and WAN
GatewayPT
E911PT
Directory Numbers
63500
63501
63502
63503
Directory Numbers
64050
64051
64052
6405x
Directory Numbers
64020
64021
64022
6402x
Route Pattern
9.@
9.8@
5.7xxxx
Route Pattern
911
9.911
• A partition is a logical grouping of patterns.
• All patterns in a partition are equally reachable.
• A partition is assigned to directory numbers and route
patterns.
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—2-13
A partition is a logical grouping of DNs and route patterns with similar characteristics. For
simplicity, you should name your partitions for their characteristics, such as
NYLongDistancePT, NY911PT, and so on. When you place a DN or route pattern into a
certain partition, you create a rule that specifies what devices can call that DN or route pattern.
Partitions do not significantly affect the performance of digit analysis; however, every partition
specified in the search space of a calling device requires that an additional analysis pass
through the analysis data structures. Digit analysis looks through every partition in a calling
search space for the best match. Cisco CallManager uses the order of the partitions listed in the
calling search space only to break ties when equally good matches exist in two different
partitions.
If you do not specify a partition for a pattern, Cisco CallManager lists the pattern in the null
partition to resolve dialed digits. Digit analysis always looks through the null partition last. If
you assign DNs or route patterns that do not reside in a named partition, these are placed in the
null partition.
2-74
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Calling Search Space Definition
Lobby Telephone
Calling Search Space
E911PT
EmployeePT
Employee Telephone Executive Telephone
Calling Search Space
E911PT
EmployeePT
LocalGWPT
WANGWPT
Calling Search Space
E911PT
EmployeePT
GWPT
ExecutivePT
Local and WAN
GWPT
Calling Search Space
E911
EmployeePT
• A calling search space is an ordered list of partitions.
• Digit analysis looks through the list of partitions when
searching for the closest match for the dialed number of
the caller.
• A calling search space is assigned to devices,
telephones, and gateways.
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—2-14
A calling search space is an ordered list of partitions that Cisco CallManager digit analysis
looks at before allowing a user to place a call. Calling search spaces govern the access of
calling devices, such as IP Phones and gateways, between partitions when attempting to
complete a call.
When you assign a calling search space to a device, the list of partitions in the calling search
space consists only of the partitions that you allow that device to reach. If users attempt to dial
a DN in a partition that you have not assigned to their calling search space, they hear a fast
busy signal (do a search on “reorder” in Cisco CallManager traces).
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Cisco CallManager, Network Signaling, and Dial Plan
2-75
Troubleshooting Route Patterns
Cisco CallManager installations have several different deployment types, such as campus and
multisite deployments, each with its own separate dial plan issues.
Deployment Models
Site A
A
Branch B
Campus
PSTN
PSTN
IP WAN
Headquarters
Branch B
Distributed
Branch C
PSTN
IP WAN
Headquarters
Centralized
© 2004 Cisco Systems, Inc. All rights reserved.
Branch C
IPTT v4.0—2-15
Dial plan problems that relate to the campus deployment model also apply to distributed and
centralized deployment models (such as overlapping dial plan and misplaced second dial tone).
The Route Plan Report in Cisco CallManager is a tool for troubleshooting route pattern issues.
At the Cisco CallManager Administration page, choose Route Plan > Route Plan Report and
find the route pattern you are having issues with. The Dialed Number Analyzer (DNA) service
can also be used to troubleshoot route pattern issues. With this tool, you can see how the route
pattern flows through Cisco CallManager. This tool can be used to troubleshoot route patterns
over gateways, phones, and trunks.
2-76
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Overlapping Dial Plan
Route Pattern
Wait 15 Seconds
911
9.1xxxxxxxxxx
What makes this overlapping ?
Route Call
911
9.1[2-9]xx[2-9]xxxxxx
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—2-16
One of the more common configuration problems is an overlapping dial plan. An overlapping
dial plan occurs when two or more patterns match a set of dialed digits.
In the figure, the Cisco CallManager digit-analysis process finds two possible matches to the
dialed digits 911. The first pattern is an exact match, but the second pattern could match if the
caller dials additional digits before the interdigit timeout occurs. If the caller waits for the
length of the interdigit timeout, digit analysis chooses the pattern of 911 and routes the call
according to the way that you have configured the rest of the route plan.
However, the customer may not want to wait 15 seconds before the call routes. You can avoid
this issue by not allowing the dial plan to overlap, if possible. If you configure the dial plan
with the rules of the North American Dial Plan (NADP) or E.164 in mind, you can avoid most
of these issues.
In this example, the NADP states that the digit dialed after a “1” cannot be a “1” or a “0.” In
addition, the first digit of the office code (the first digit of a seven-digit number) cannot be a
“1” or a “0.” The correct route pattern of 9.1[2-9]xx[2-9]xxxxxx eliminates the conflict with
911.
The dial plan in the Cisco CallManager is completely open and unblocked, which can work
either in your favor or against it. Careful attention to the dial plan eliminates most, if not all, of
these types of conflicts.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Cisco CallManager, Network Signaling, and Dial Plan
2-77
Shown here is a sample dial plan that covers most of the patterns encountered in an average
calling area.
Note
Check your local calling patterns against this sample to see if you must add or delete
additional patterns. The partition information is an example only. Adjust the partitions to
meet the needs of the specific customer site.
Routing Patterns
Route Pattern
Name
Partition
Discard Digits
9.11
Emergency 1
911
No Discard
9.911
Emergency 2
911
Pre-Dot
9.411
Information
Info
Pre-Dot
9.611
Telco Repair
Local
Pre-Dot
9.[2-9]xx xxxx
Local 7 Digit
Local
Pre-Dot
9.215 [2-9]xx xxxx
Local 10 Digit
Local
Pre-Dot
9.1 [2-9]xx [2-9]xx xxxx
11 Digit Long Dist
Long Dist
Pre-Dot
9.1 800 [2-9]xx xxxx
Toll Free
Local
Pre-Dot
9.1 866 [2-9]xx xxxx
Toll Free
Local
Pre-Dot
9.1 877 [2-9]xx xxxx
Toll Free
Local
Pre-Dot
9.1 888 [2-9]xx xxxx
Toll Free
Local
Pre-Dot
9.1 [2-9]xx 976 xxxx
976 Block
None
Pre-Dot
9.1 900 xxx xxxx
1 900 Block
None
Pre-Dot
9.1 976 xxxx
976 Block 2
None
Pre-Dot
9.976 xxxx
976 Block 3
None
Pre-Dot
9.0
Local Operator
Operator
Pre-Dot
9.00
Long Dist Operator
Operator
Pre-Dot
9.011!
International
International
Pre-Dot
9.011!#
International
International
Pre-Dot
Internal Numbers
Internal
Internal
Calling Search Space
2-78
Calling Search Space
Partitions
Execs (Unrestricted)
911, Info, Local, Long Dist, Operator, International, Internal
Office (No International)
911, Info, Local, Long Dist, Operator, Internal
Lobby (Internal Only)
911, Internal
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Second Dial Tone
Route Pattern
Outside Dial Tone
User dial string:
92106527977
92xx
9.[2-9]xx xxxx
9.[2-9]xx[2-9]xx xxxx
Yes
Cisco CallManager
actions:
• Route call
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—2-17
Another problem occurs when the second (or outside) dial tone plays at the wrong time in the
route pattern. If the caller dials several digits after the “9” before Cisco CallManager plays a
second dial tone, the dial tone plays at the wrong time.
You must understand how Cisco CallManager decides when to play the second dial tone. A
common misconception is that Cisco CallManager plays a second dial tone wherever the dot (.)
is located when the Provide Outside Dial Tone box is checked and a dot (.) is in the route
pattern. In fact, Cisco CallManager plays the second dial tone when all possible matches to the
dialed digits have the Provide Outside Dial Tone box checked.
In the figure, Cisco CallManager does not provide a second dial tone when the user dials “9”
because all three route patterns match 9210. Cisco CallManager does not play a second dial
tone until the user dials the “6,” which eliminates the first two route patterns from matching the
dialed digits. To avoid this problem, make sure that the Outside Dial Tone box is checked for
all external numbers and make sure that no internal numbers begin with your access code.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Cisco CallManager, Network Signaling, and Dial Plan
2-79
Second Dial Tone (Cont.)
Route Pattern
Outside Dial Tone
User dial string:
923125551212
92xx
9.[2-9]xx xxxx
9.[2-9]xx[2-9]xx xxxx
Yes
Cisco CallManager
actions:
• Route call
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—2-18
By default, Cisco CallManager applies the second dial tone as soon as all of the matching
patterns have the Provide Outside Dial Tone box checked.
If you need an access code longer than one digit, enter a pattern that contains one digit less than
the access code with the Provide Outside Dial Tone box unchecked. This causes Cisco
CallManager to wait until the end of the access code before it plays the dial tone.
In the figure, Cisco CallManager waits until the user dials a “2” before playing the second dial
tone. Here, all potential matches have the Provide Outside Dial Tone box checked.
2-80
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Improper Routing
Cisco Gateways
1st choice
Route
Group A 1st choice
Digital Gateway
8 cents per minute
Global Switch
or PBX
Route Pattern
Route
List
2nd choice
Route
Group B
WS-X6608-T1
12 cents per minute
Global Switch
or PBX
WS-X6624-FXS
15 cents per minute
Global Switch
or PBX
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—2-19
If calls are not routing to the proper gateway, the cause may be one of the following:
You may not have programmed Cisco CallManager to properly route calls to the gateway.
In this case, verify programming of route patterns, route lists, and route groups.
The gateway may not work. If all programming appears correct, verify that the gateway is
working. Cisco CallManager automatically advances to the next choice in either the route
group or list if the gateway is busy or down.
The telco trunks may be down. If this is the case, test the trunks using appropriate test
equipment (such as a transmission test set, PRI tester, or T-BERD).
Run DNA on the route pattern to see the call flow.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Cisco CallManager, Network Signaling, and Dial Plan
2-81
One-Way Calling
“Call me
back”
“OK”
1234
Partition A
1211
X
1211
CSS B, A
Partition B
1234
1211
1234
CSS B
Reorder
tone
What happens if both DNs were not assigned a partition?
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—2-20
In the figure, IP Phone 1 can call IP Phone 2, but IP Phone 2 cannot call IP
Phone 1. Typically, an issue with partitions and calling search spaces causes this problem.
For example, you will see this problem if the partition for IP Phone 2 is in the calling search
space for IP Phone 1, but not vice versa. To correct the problem, verify the partitions of both IP
Phones and ensure that you include each in the calling search space of the opposite IP Phone.
2-82
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Discard Digits Assignments
9.1010xxx1[2-9]xx[2-9]xx xxxx
Discard: PreDot 1010 dialing
9101022014085551212
9101022014085551212
PSTN
Dial Plan
Gateway
91010xxx.1[2-9]xx[2-9]xx xxxx
Discard: PreDot
9101022014085551212
14085551212
PSTN
Dial Plan
Gateway
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—2-21
The discard digits assignment on the route pattern may not work for the following two possible
reasons:
If you use a route pattern that does not contain the @ wildcard, the only discard instructions
that will work are PreDot, None, and PreDot/Trailing #. While you may use the other
discard instructions, they are only valid if you use the @ wildcard in the route pattern.
If you define a discard instruction in the route pattern and configure the discard instructions
of NoDigits instead of None in the route group configuration, the route group programming
overrides the route pattern. This override causes the route group to restore the number to
the one that the user originally dialed.
Use the DNA tool to assist in seeing the call flow. The DNA application with list how the
discard digits are seen. In other words, if you are discarding digits under the Route Pattern
field and have discards instructions under the Route List, the DNA application will show
the route list discard instructions. DNA is a very handy tool to spot discard instructions.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Cisco CallManager, Network Signaling, and Dial Plan
2-83
Reorder Tone
Reorder tone
Digit Dialed
Reorder tone
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—2-22
A user may hear an unexpected reorder tone because of a problem with translation patterns. A
second possibility requires a deeper analysis of the dial plan.
Translation pattern issues
If an administrator enters the Translation Pattern screen and presses the Insert button
mistakenly without entering any data, Cisco CallManager adds a translation pattern of None.
This addition into the null partition causes an immediate reorder tone when the IP Phone goes
off-hook.
Cisco CallManager always treats translation patterns as urgent and attempts to translate the
pattern of None (no digits dialed) into another pattern. The translation pattern changes to None
resulting in the Cisco CallManager cycling the digits through digit analysis again.
Cisco CallManager attempts to deal with the pattern ten times before it detects a loop. After
detecting a loop, Cisco CallManager instructs the IP Phone to play a reorder tone. To solve the
problem, remove the bad translation pattern.
Dial plan analysis
The second possible cause of an unexpected reorder tone is either a dial plan configured
incorrectly or problems with the gateways. If you attempt to call an IP Phone in another cluster
and Cisco CallManager plays a reorder tone after you have dialed only a few digits, you usually
have a configuration problem in Cisco CallManager with the route patterns, groups, or route
list.
Cisco CallManager looks ahead to find a valid and available gateway as soon as the number
dialed matches only one pattern. Cisco CallManager goes through the route list and groups to
find a valid or available gateway. If the Cisco CallManager server does not find a gateway, it
returns a reorder tone even though the user has not finished dialing the number.
2-84
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
In this situation, you must determine whether you are dealing with a route pattern problem or a
gateway problem. To make this determination, you should do the following:
Use the DNA tool; you should find the issue.
Verify that there is a valid route pattern containing the number that you are trying to dial
and make sure that the programming is correct for the route list and groups.
Perform a trace of the call to see where the problem lies.
Troubleshoot the gateways as follows:
—
Ping the remote Cisco CallManager server.
—
Verify that you have programmed a return gateway.
—
Dial a number that hits the PSTN gateway.
—
Test PSTN trunks with telco test equipment.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Cisco CallManager, Network Signaling, and Dial Plan
2-85
Distributed Call-Processing Model
This topic discusses the specific issues related to the distributed call-processing model.
Distributed Call-Processing Model
Branch B
VM or UM
CCM
VM or UM
CCM
PSTN
IP WAN
VM or UM
CCM
Headquarters
Primary path
Secondary path
VM or UM = voice mail or unified messaging
Branch C
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—2-23
Because the distributed call-processing model is two or more campus clusters interconnected
with WAN and PSTN links, all of the problems that occur in a campus deployment apply to
this model as well. In addition, some unique problems can occur in this call-processing model,
such as the following:
A call routes to the PSTN instead of using the WAN link first.
An off-net call sent to another cluster for a toll bypass gets a reorder tone.
A call to a telephone at the remote site immediately drops after the remote user picks up the
telephone.
2-86
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Call Routing to PSTN
Branch B
VM or UM
VM or UM
CCM
PSTN
IP WAN
CCM
VM or UM
CCM
Headquarters
Primary path
Secondary path
VM or UM = voice mail or unified messaging
© 2004 Cisco Systems, Inc. All rights reserved.
Branch C
IPTT v4.0—2-24
When a call routes to the PSTN instead of using the WAN link first, several possible causes
exist. The most common problem is that the H.323 intercluster trunk is not functioning
properly. Make a test call to the remote site and verify the route taken by the call. One way to
determine the gateway of the incoming call is to listen to the ring tone cadence. A PSTN (PRI)
call rings twice per ring cycle, while a WAN call rings once per ring cycle.
The problem may exist with the route list or group programming. To check this programming,
perform these steps:
Step 1
Check the Route Plan Report to make sure that your call is hitting the correct route
list and route group and that it is supposed to be going out the right gateway. Then
continue to Step 2 and so on.
Step 2
Verify that you have programmed the intercluster trunk, and ensure that the IP
address is correct.
Step 3
Verify that the intercluster trunk is in the proper WAN route group and that the
WAN route group is the first option in the WAN route list.
Step 4
Verify that you have directed the pattern that you are dialing to the correct route list.
Step 5
If all of the programming appears correct, try to ping the Cisco CallManager server
at the other end of the call.
Step 6
Perform all of the checks at the other site as well. The remote Cisco CallManager
server needs an intercluster trunk back to the originating site before the call can go
through.
Step 7
If these steps do not help to resolve the problem, run a trace on a call in each
direction to verify that Cisco CallManager is routing correctly.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Cisco CallManager, Network Signaling, and Dial Plan
2-87
Off-Net Call to Another Cluster
No Entry for
5551212 in
Dial Plan
VM or UM
CCM
9.8135551212
PSTN
5551212
Headquarters
IP WAN
Branch B
VM or UM
CCM
VM or UM
CCM
Primary path
Secondary path
VM or UM = voice mail or unified messaging
© 2004 Cisco Systems, Inc. All rights reserved.
Branch C
IPTT v4.0—2-25
Another problem occurs when an off-net call sent to another cluster for toll bypass gets a
reorder tone. Because this is an intercluster call, use the same troubleshooting steps that you
would use to check the route list or group programming.
However, the cause in this case could simply be a lack of proper digits sent to the far end.
Remember that the remote Cisco CallManager server sends the incoming digits through digit
analysis when they arrive. The incoming pattern must match a template that exists in the dial
plan of the remote Cisco CallManager server. Because most route patterns flag an off-net call
by adding a “9” to the number, the local Cisco CallManager server should forward this “9”
digit to the remote-end Cisco CallManager server. Then, the remote Cisco CallManager server
matches the incoming number with a route pattern and routes it to a PSTN gateway.
2-88
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Off-Net Call to Another Cluster (Cont.)
BRANCH_B_GWY_CSS
INTERNAL_PT
VM or UM
CCM
9.8135551212
PSTN
5551212
IP WAN
Branch B
VM or UM
CCM
VM or UM
CCM
Headquarters
Primary path
Secondary path
VM or UM = voice mail or unified messaging
© 2004 Cisco Systems, Inc. All rights reserved.
Branch C
IPTT v4.0—2-26
Another problem can exist with the calling search space of the gateway handling the incoming
call. As a security tactic, restrict access to other gateways to prevent misuse of the telephone
system. In this case, verify that the calling search space of the remote-end gateway has the
partition of the device (gateway or telephone) that you are trying to reach.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Cisco CallManager, Network Signaling, and Dial Plan
2-89
Remote Call Drops Immediately
G.729
G.723
IP Phone C
IP Phone A
30VIP
7960
SCCP
WAN(FR)
X
SCCP
Y
IP Phone B
IP Phone D
H.323 or Intercluster Trunk
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—2-27
If a call to a telephone at the remote site immediately drops after the remote user picks up the
handset, it is not a dial plan problem.
You can easily tell if the problem is in the dial plan or in another part of Cisco CallManager
configuration depending on whether the telephone rings. If the telephone at the far end rings,
the dial plan and call setup are working properly.
The most common problem with a call that sets up and rings but then drops right away is a
codec mismatch. This mismatch occurs when the Cisco CallManager server instructs one of the
telephones to send or receive at a codec that it cannot handle. If no transcoding (XCODE)
resources are available, the call will set up but then drop during the media setup phase.
You must provide hardware transcoding resources at the remote end or change the region
information to a codec that the remote telephone can handle.
The example presented here shows G.729 going to G.723. You can resolve this issue by
changing both telephones to G.711, because G.711 is the only common codec that these IP
Phones share.
Note
2-90
Transcoding resources cannot convert between G.729 and G.723. Transcoding resources
can only convert between G.711 and G.729 or G.723 and vice versa.
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Centralized Call-Processing Model
This topic discusses the specific issues related to the centralized call-processing model.
Centralized Call-Processing Model
Branch B
VM or UM
PSTN
IP WAN
Headquarters
Branch C
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—2-28
The multisite centralized model shares many of the same dial plan problems with the campus or
distributed models. However, there are other issues that have more of an impact on the
centralized model.
There are two issues that have a greater impact on the centralized model: calls routing to the
wrong gateway and calls not using the proper gateway for toll bypass. Both of these problems
relate directly to partitions and calling search spaces.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Cisco CallManager, Network Signaling, and Dial Plan
2-91
Geographic Boundaries
Centralized Cisco
CallManger Cluster
Site 1
Router or
Gateway
PSTN
IP WAN
Router
Hub
Intralocation calls
Interlocation calls
Local PSTN calls
© 2004 Cisco Systems, Inc. All rights reserved.
IP WAN
Site 2
IP WAN
Router
IPTT v4.0—2-29
To effectively use the centralized model across geographic boundaries, put gateways at each
location. This way, the telephones at the remote site use the local lines at their site for local
calls and long distance lines at the hub site. With this design, you must address items such as
call costs and available bandwidth. You can use partitions and calling search spaces to decide
which gateway a telephone will use.
2-92
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Media Resources
This topic discusses specific issues related to media resources.
Media Resources Overview
Media resources—
common issues:
• Software Conference
Bridge or softwarebased MTP
• Hardware Conference
Bridge or hardwarebased MTP
• Transcoder (XCODE)
• Media termination
points
• Music on hold (MOH)
Branch B
PSTN
IP WAN
Headquarters
© 2004 Cisco Systems, Inc. All rights reserved.
Centralized
Branch C
IPTT v4.0—2-30
An important part of any IP telephony network is a variety of media resources. A media
resource is a network device that terminates a media steam to provide some services to IP
telephony endpoints such as IP Phones and voice gateways. Five such media resources and
their common issues will be discussed in this topic.
Cisco IP telephony functionality requires the use of media resources. Media resources provide
services such as transcoding, conferencing, MOH, and media termination. Media resource
management provides access to media resources for all Cisco CallManager servers in a cluster.
Every Cisco CallManager server contains a software component, MRM. The MRM locates the
media resource necessary to connect media streams to complete a feature. Cisco CallManager
interfaces to these media resources using SCCP.
Here is a brief overview of media resources:
Software Conference Bridge/software-based media termination point (MTP): Unicast
bridge supporting G.711 and wideband audio stream conferences. A software MTP is
implemented by Cisco IP Voice Media Streaming Application. This feature transcodes Alaw to u-law and vise versa and adjusts packets sizes as required by two connections. MTPs
are also use to extend supplementary service to H.323 endpoints. When a MTP is used, the
voice streams flow through the MTP and the H.323 endpoint.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Cisco CallManager, Network Signaling, and Dial Plan
2-93
Hardware Conference Bridge/hardware-based MTP: All capabilities of a Software
Conference Bridge support low bit-rate streams types such as G.729a, global system for
mobile communication (GSM), or G.723. This allows some Hardware Conference Bridges
to handle mixed-mode conferences. In a mixed mode conference, the Hardware Conference
Bridge transcodes G.729a, GSM, and G.723 into G.711 streams, mixes them and then
encodes the resulting stream into the appropriate stream type for transmission back to the
user. Hardware MTPs are external to Cisco CallManager but controlled by Cisco
CallManager.
Transcoder: This is a device that takes the output of one voice codec and converts it in
real time into an input stream for a different codec type.
MTP: See the description of both software-based MTP and hardware-based MTP.
MOH: Provides unicast or multicast MOH streams (or both) from Cisco CallManager or a
standalone server, or both. Cisco CallManager controls the MOH streams.
Media resource allocation: This is through the MRM on Cisco CallManager.
2-94
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Common Media Resources Issues
• There are incorrect media resource groups and list
configuration.
• Cisco CallManager ignored the Meet-Me
Conference or Confrn soft key.
• Calls are dropped after they are connected.
• IP Phone cannot transfer or place on hold a
PSTN call.
• Unicast or multicast MOH is not heard internally
and or externally.
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—2-31
Most of the issues presented in this figure could be related; however, each will be considered
separately as if they are not related, as follows:
Incorrect media resource groups (MRGs) and list configuration: If MRGs are being
used to allocation media resources to users, look to make sure that the device with the issue
has proper access to the media resources. In most cases where MRGs are used, the issue is
incorrect configuration. In other words, the system administrator creating the media groups
forgot to add a resource.
Here are items to consider when troubleshooting MRGs:
—
Check to see if the media resource group list (MRGL) is configured correctly.
—
Check to see if the resources assigned to the MRGL are registered.
—
Check the Event Viewer for any media resource device errors relative to
unregistering devices.
—
If resources are not registered with Cisco CallManager, media resources will fail.
—
Review Cisco CallManager traces to see what event is causing the media resource
failure.
—
If you cannot isolate the issue and resolve it, call Cisco Technical Assistance Center
(TAC).
Cisco IP Phone user can not use confernce features: When you attempt to create a
conference and a CFB is not available, Cisco CallManager does not respond to the
Meet-Me Conference or Confrn soft key.
In other words, the buttons pressed are ignored. Some of the possible reasons of this
condition are as follows:
—
The CFB server is not registered with the Cisco CallManager server.
—
The CFB server failed for some reason (restart the server if needed).
—
All available CFBs in the null MRG are in use.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Cisco CallManager, Network Signaling, and Dial Plan
2-95
—
All available conference servers in the MRGL assigned to the conference controller
are full.
—
No conference servers in the MRGL are assigned to the conference controller.
Calls are dropped after connected: If a transcoder is required, this means that two
endpoints do not have matching codecs and cannot communicate with each other directly.
In this case, Cisco CallManager attempts to connect the call, which causes the call to
terminate without ever establishing a media connection. In this case, you should do the
following:
—
Make sure that the transcoder is registered with Cisco CallManager.
—
Review Cisco CallManager traces for the specific call setup and see if you can spot
“ERROR wait_AuConnectErrorInd” listed. This error is a common one for any
media setup problem. If you see this error, pay close attention to codec mismatches.
—
Another thing to look for in the Cisco CallManager traces is a response indicating
“wait_AuConnectRequest” followed by “no caps match, introduction trancoder.”
—
If you have hardware transcoders, make sure that the devices are registered with
Cisco CallManager. You may find it necessary to restart the gateways to reinitialize
the digital signal processor (DSP) used for transcoding.
—
Check with the Cisco TAC if you have not resolved the issues.
Cisco IP Phone cannot transfer or place on hold a PSTN call: If an MTP is required in a
given call and neither an MTP nor a transcoder (to act as a MTP) is available at that point
Cisco CallManager connects the call directly; however, supplementary services (such as
hold and transfer services) are not be available. The user is not notified but will find out
when trying to use a feature that requires an MTP. In these cases, you should do the
following:
—
Determine if you have any available software or hardware MTP resources registered
with Cisco CallManager.
—
Use Microsoft Performance Monitor to check the number of MTPs available.
—
In the Gateway Configuration screen in Cisco CallManager, make sure that the
Media Termination Point Required box is checked.
MOH is not heard internally and or externally: Someone places you on hold, and you
do not hear any music. Use the following steps to pinpoint the problem:
2-96
Step 1
Check the MOH server to see if it is registered with Cisco CallManager.
Step 2
Check the MRG and MRGL configuration.
Step 3
Verify the router configuration for multicast (if multicast is used).
Step 4
Verify the multicast capability of the terminating voice gateway.
Step 5
Verify the codec used by all devices involved.
Step 6
Verify the configuration of the location if you are using the centralized callprocessing model.
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
If needed, review Cisco CallManager traces for the specific call setup for MOH and look for a
“MrmAllocateMohResourcereq” trace. This trace tells you that the MRM is trying to locate the
MOH resource. If you see “device_lookup_DevicePidErr; Unable to locate
DeviceName=(name of Moh server)”, the MOH server has a problem and or it is not registered
with Cisco CallManager. You will then have to determine why the MOH server is not
registered. Restart the server if necessary.
When troubleshooting multicast problems with Cisco IOS software gateways, make sure the
Cisco IOS software load that you are using can accept multicast MOH. Multicast MOH is
supported for H.323 and MGCP endpoints is Cisco IOS Release 12.2(11)T and later.
When a user is not assigned to a MRGL with an MOH source, the user hears a tone on hold
instead of MOH, because the user does not have access to an MOH resource.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Cisco CallManager, Network Signaling, and Dial Plan
2-97
Transcoder (XCODE)
Voice Messaging
XCODE
G.729
G.711
MTP
Input Stream
Stream for
Supplementary
Services
H.323 v1
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—2-32
This figure shows a brief example of trancoding services and an MTP application.
One thing to note here is that an MTP application can act as a trancoder and also as MTP. Both
use DSP, and DSP provides the means for transcoding and setting up MTPs.
Media Resource Group Lists
Similar to Route Lists
and Route Groups
User Needs
Media Resource
Media
Resource
Manager
Assigned to Device
Media Resource
Group List
1st
Choice
2nd
Choice
Media
Resource
Group
1st
Choice
Media Resource
1
Media
Resource
Group
2nd
Choice
Media Resource
2
1st
Choice
Media Resource
3
2nd
Choice
Media Resource
4
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—2-33
MRGs and MRGLs are setup much like Route groups and route lists.
2-98
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Restrict Access to Conference Resources
Resource_List
1
2
3
4
MTP MRG
MTP1
MTP2
CONF MRG
SW-CONF1
SWCONF2
HW-CONF1
HW-CONF2
MOH MRG
MOH1
MOH2
NO_CONF_LIST
1
2
3
A
MTP MRG
MTP1
MTP2
MOH MRG
MOH1
MOH2
XCODE MRG
XCODE1
XCODE2
Result
Device cannot
use any
conference
resources.
XCODE MRG
XCODE1
XCODE2
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—2-34
This figure shows how to restrict conference resources from devices by the way that the MRGs
and MRGLs are configured.
Create an MRGL, called Resource_List, that has all the media resources, and then create an
MRGL called NO_CONF_LIST and assign MRGs in this order: MTP MRG, XCODE MRG,
and MOH MRG.
In the device configuration, assign NO_CONF_LIST as the MRGL of the device.
The result is that the device cannot use conference resources. Only the MTP, XCODE, and
MOH resources are available to the device.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Cisco CallManager, Network Signaling, and Dial Plan
2-99
Summary
This topic summarizes the key points discussed in this lesson.
Summary
• The Cisco CallManager dial plan design architecture includes route lists and
groups, route patterns, translation patterns, dial peers, partitions and calling
search spaces, and overlapping dial plans.
• You can define IP telephony CoS categories within partitions and calling
search spaces as internal to a PBX, on the public switched network, or on a
packet switched network.
• Different deployment types within a Cisco CallManager installation each
have their own separate dial-plan issues.
• Problems specific to the distributed call-processing model include calls
routing to the PSTN, off-net calls getting a reorder tone, and remote calls
dropping immediately.
• Problems specific to the centralized call-processing model include calls
routing to the wrong gateway and calls not using the proper gateway for a
toll bypass.
• Media resource groups all allow hardware and software resources to coexist
and allow users access to cluster and enable Cisco CallManager access
resources available in the cluster.
• Media resource groups help load balance resources within a group of
similar resources.
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—2-35
References
For additional information, refer to these resources:
Diagnosing Cisco CallManager Problems:
http://www.cisco.com/en/US/products/sw/voicesw/ps556/prod_troubleshooting_guide_cha
pter09186a00800c2f78.html.
Media resource management:
http://www.cisco.com/en/US/products/sw/voicesw/ps556/products_administration_guide_c
hapter09186a00800c4cb4.html.
Cisco CallManager 4.0 annunciator:
http://www.cisco.com/en/US/products/sw/voicesw/ps556/products_administration_guide_c
hapter09186a00801ec5b9.html#44093.
MTPs:
http://www.cisco.com/en/US/products/sw/voicesw/ps556/products_administration_guide_c
hapter09186a00801ec5be.html#1018268.
Cisco SIP Proxy Server: http://www.cisco.com/univercd/cc/td/doc/product/voice/sipproxy.
2-100
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Quiz
Use the practice items here to review what you learned in this lesson. The correct answers are
found in the Quiz Answer Key.
Q1)
Which of the following is NOT an external route plan element?
A)
B)
C)
D)
Q2)
Which route pattern wildcard represents a single digit (0 through 9)?
A)
B)
C)
D)
Q3)
a pattern of digits that are pointed to a gateway
a prioritized list of route patterns
a prioritized list of gateway devices
a prioritized list of route lists
Which of the following best describes a partition?
A)
B)
C)
D)
Q6)
after all of the digits have been dialed
when at least two digits are dialed
after each digit is dialed
after the user enters all the digits and presses the pound (#) sign
Which of the following describes a route group?
A)
B)
C)
D)
Q5)
?
!
x
@
When does Cisco CallManager perform digit analysis?
A)
B)
C)
D)
Q4)
directory number
route pattern
route group
route list
a logical grouping of route patterns
a logical grouping of Cisco CallManager servers
a logical grouping of gateway devices
a logical grouping of route lists
To ensure that a device has access to zero media resources, which of the following
things should you do?
A)
B)
C)
D)
Make sure that conferencing is not allowed.
Make sure that no media resources are assigned in the media group.
Unplug the IP Phone.
Make sure that there is a logical grouping of route lists.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Cisco CallManager, Network Signaling, and Dial Plan
2-101
Quiz Answer Key
2-102
Q1)
A
Q2)
D
Q3)
C
Q4)
D
Q5)
D
Q6)
B
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Cisco
CallManager CTI Manager,
JTAPI, and TSP
Overview
External applications that communicate with Cisco CallManager are managed through Cisco
CallManager CTI Manager. Whether your application uses Cisco Java Telephony Application
Programming Interface (JTAPI) or Cisco Telephony Service Provider (TSP), knowing how
these applications communicate with Cisco CallManager is critical when the need to
troubleshoot arises. In this lesson, the operational elements of Cisco CallManager JTAPI, TSP,
CTI route points, and CTI ports will be discussed. Toward the end of this lesson, how to
troubleshoot these elements is discussed.
Relevance
External applications are becoming very popular and knowing how Cisco CallManager CTI
Manager, CTI route points, CTI ports, and Cisco TSP operate will give you the knowledge to
troubleshoot issues.
Objectives
Upon completing this lesson, you will be able to:
Describe the functions of Cisco CallManager CTI Manager, JTAPI, and TSP
Identify the components of Cisco CallManager CTI Manager
Describe the steps in troubleshooting Cisco CallManager CTI, JTAPI, and TSP
Learner Skills and Knowledge
To benefit fully from this lesson, you must have these prerequisite skills and knowledge:
Fundamental understanding of how to configure CTI route points and CTI ports
Fundamental knowledge of how to install Cisco CallManager JTAPI and TSP
Outline
This topic discusses the lesson outline.
Outline
• Overview
• Computer Telephony Integration
• CTI Route Points and CTI Ports
• Cisco CallManager JTAPI and TSP
• Troubleshooting Cisco CallManager CTI Manager
• Troubleshooting Cisco CallManager JTAPI and
Cisco TSP
• Summary
• Quiz
© 2004 Cisco Systems, Inc. All rights reserved.
2-104
Cisco IP Telephony Troubleshooting (IPTT) v4.0
IPTT v4.0—2-3
Copyright © 2004, Cisco Systems, Inc.
Computer Telephony Integration
This topic discusses the specifics related to CTI route points and CTI ports in Cisco
CallManager.
Cisco CallManager CTI Manager
Cisco CallManager CTI Manager can be on Cisco
CallManager or a standalone server.
Cisco
CallManager
JTAPI or
TSP
CTI Manager
Primary
Cisco
CallManager
Cisco
CallManager
Cisco
CallManager
Applications communication with Cisco CallManager
through Cisco CallManager CTI Manager.
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—2-4
CTI enables you to leverage computer processing functions while making, receiving, and
managing telephone calls. CTI applications allow you to perform such tasks as retrieving
customer information from a database on the basis of information that caller ID provides. CTI
applications can also enable you to use information that an interactive voice response (IVR)
system captures, so that the call can be routed to the appropriate customer service
representative or so that the information is provided to the individual who is receiving the call.
Computer Telephony Integration Applications
The following list contains descriptions of some Cisco CTI applications that are available:
Cisco IP SoftPhone: Cisco IP SoftPhone, a desktop application, turns your computer into a
full-feature telephone with the added advantages of call tracking, desktop collaboration,
and one-click dialing from online directories. You can also use a Cisco IP SoftPhone in
tandem with a Cisco IP Phone to place, receive, and control calls from your desktop PC.
All features function in both modes of operation.
Cisco IP AutoAttendant: The Cisco IP AutoAttendant application works with
Cisco CallManager to receive calls on specific telephone extensions and to allow the caller
to choose an appropriate extension.
Cisco CallManager Attendant Console: This application provides a graphical user
interface (GUI) for controlling a Cisco IP Phone to perform attendant console functions.
Cisco Personal Assistant: Cisco Personal Assistant, a virtual secretary or personal
assistant, can selectively handle your incoming calls and help you make outgoing calls.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Cisco CallManager, Network Signaling, and Dial Plan
2-105
Cisco WebDialer: Cisco WebDialer is installed on a Cisco CallManager server and is used
in conjunction with Cisco CallManager to allow Cisco IP Phone users to make calls from
web and desktop applications.
Cisco CallManager CTI Manager
Cisco CallManager CTI Manager is a program that includes the CTI components that interface
with the applications separated out of Cisco CallManager. The CTI Manager service
communicates with Cisco CallManager by using the Cisco CallManager communication
framework, SDL. You can have one or more CTI Managers active in a cluster, but only one
CTI Manager can exist on an individual server. An application (JTAPI Telephony Application
Programming Interface [TAPI]) can have simultaneous connections to multiple Cisco
CallManager CTI Managers; however, an application can use only one connection at a time to
open a device with media termination.
Cisco CallManager does not distinguish between TAPI and JTAPI applications. Translations to
CTI Quick Buffer Encoding (CTIQBE) are done by the application and at the Cisco
CallManager CTI layer. Cisco CallManager CTI resource provisioning depends on the CTI
message transaction between the application and the Cisco CallManager.
CTI Controlled Devices
The following are CTI control device types:
Cisco IP Phones
CTI ports
CTI route points
CTI-controlled Cisco IP Phones comprise regular phones that a CTI application can control.
CTI ports as virtual devices can have one or more virtual lines, and software-based
Cisco CallManager applications such as Cisco SoftPhone, Cisco AutoAttendant, and
Cisco IP IVR use CTI ports. You configure CTI ports by using the same Cisco CallManager
Administration windows as you use to configure phones. For first-party call control, you must
add a CTI port for each active voice line.
A CTI route point virtual device can receive multiple, simultaneous calls for applicationcontrolled redirection. You can configure one or more lines on a CTI route point that users can
call to access the application. Applications can answer calls at a route point and can also be
redirected to a CTI port or IP Phone. Route points can receive multiple, simultaneous calls;
therefore, the application must specify the media and port for the call on a per-call basis.
CTI route points support the following features:
Answer a call
Make and receive multiple active calls
Redirect a call
Hold a call
Unhold a call
Drop a call
2-106
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
When a call arrives at a route point, the system must accept or answer it within a specified time.
Use the Directory Number Configuration window in Cisco CallManager Administration to
configure the number of simultaneous active calls on the route point that can terminate at
media. To configure the time allowed to answer a call, use the New Call Accept Timeout
service parameter.
Note
When a CTI device fails (during a Cisco CallManager failure, for example),
Cisco CallManager maintains media streams that are already connected between devices
(for devices that support this feature). Cisco CallManager drops calls that are in the process
of being set up or modified (transfer, conference, redirect, and so on).
CTI provides recovery of failure conditions that result from a failed Cisco CallManager node
within a cluster and failure of a Cisco CallManager CTI Manager. This section describes the
failover and fallback capabilities of the following components:
Cisco CallManager
Cisco CallManager CTI Manager
Applications (TAPI and JTAPI)
When a Cisco CallManager node in a cluster fails, the CTI Manager recovers the affected CTI
ports and route points by reopening these devices on another Cisco CallManager node. If an
application has a phone device open, the CTI Manager also reopens the phone when the phone
fails over to a different Cisco CallManager. If the Cisco IP Phone does not fail over to a
different Cisco CallManager, the CTI Manager cannot open the phone or a line on the phone.
The CTI Manager uses the Cisco CallManager group that is assigned to the device pool to
determine which Cisco CallManager to use to recover the CTI devices and phones that the
applications opened.
When the CTI Manager initially detects the Cisco CallManager failure, it notifies the
application (JTAPI and TAPI) that the devices on that Cisco CallManager server went out
of service. If no other Cisco CallManager server in the group is available, the devices
remain out of service. When those devices successfully rehome to another
Cisco CallManager server, the CTI Manager notifies the application that the devices are
back in service.
When a failed Cisco CallManager node comes back in service, the CTI Manager rehomes
the affected CTI ports and route points to their original Cisco CallManager server. The
rehoming process starts when calls are no longer being processed or active on the affected
device. Because devices cannot be rehomed while calls are being processed or active, the
rehoming process may not occur for a long time, especially for route points that can handle
many simultaneous calls.
If none of the Cisco CallManager servers in the Cisco CallManager group are available, the
CTI Manager waits until a Cisco CallManager server comes into service and tries to open
the CTI device again. If for some reason the Cisco CallManager server cannot open the
device or associated lines when it comes back into service, the CTI Manager closes the
device and lines.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Cisco CallManager, Network Signaling, and Dial Plan
2-107
Cisco CallManager CTI Manager
When a Cisco CallManager CTI Manager fails, the applications that are connected to the CTI
Manager can recover the affected resources by reopening these devices on another CTI
Manager. An application determines which CTI Manager to use on the basis of CTI Managers
that you defined as primary and backup when you set up the application (if supported by the
application). When the application connects to the new Cisco CallManager CTI Manager, it can
reopen the devices and lines that previously opened. An application can reopen a
Cisco IP Phone before the phone rehomes to the new Cisco CallManager server; however, it
cannot control the phone until the rehoming completes.
The Cisco CallManager CTI Manager service requires that the Cisco CallManager service
reside on one of the servers in the Cisco CallManager cluster (minimum of one).
The Cisco CallManager CTI Manager service requires the Cisco Database Layer Monitor
service.
Cisco CallManager CTI Manager must run on any server that has CTI applications running.
2-108
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Computer Telephony Integration—Cisco
CallManager
CallManager
Application
API (JTAPI)
CTIQBE
over
TCP/IP
Application Provider
Application
TAPISVR
API (TAPI)
TSP
C
T
I
Cisco CallManager
CTI Manager Service
Application Provider
Applications communicate with Cisco CallManager CTI Manager
using CTIQBE over TCP/IP.
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—2-5
Both Cisco CallManager JTAPI and TSP communicate with Cisco CallManager CTI Manager
over CTIQBE. Whether the application is coresident on Cisco CallManager, Cisco Customer
Response Applications (CRA), or a third-party application, the communications path to Cisco
CallManager CTI Manager is over CTIQBE.
CTIQBE is a protocol defined by Cisco. The JTAPI interface and Cisco WebAttendant also use
the CTIQBE interface. CTIQBE is a protocol that defines how messages are to be formatted.
CTIQBE sits on top of a TCP/IP interface. CTIQBE is a lot like TAPI. The Cisco CTIManager
is responsible for mapping CTIQBE messages into all of the SCCP Protocol messages to
control and monitor the IP Phones. CTIQBE is not a public interface.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Cisco CallManager, Network Signaling, and Dial Plan
2-109
Call Flow—Basic Application (AutoAttendant)
Cisco
CallManager
Extended
Services
1
Step 1: The user dials CTI route point DN.
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—2-6
The Cisco CallManager Extended Services application is being used to demonstrate call flow
from a user to the external application using Cisco CallManager CTI Manager. The next few
slides will show the call flow through Cisco CallManager to the AutoAttendant application.
The box with the headings System Status and Subsystem Status indicate that the configuration
in Cisco CallManager and in the CRA JTAPI application setup is correct. In other words, the
client information, such as user ID, password, and application version, matches that in Cisco
CallManager; the CTI route point and ports are associated with the JTAPI user in the DC
Directory; and the CTI route point and ports are registered with Cisco CallManager.
The AutoAttendant application co-residing in Cisco CallManager 4.0 is a scaled down version
of the actual CRA 3.5.1 service release 3 application.
The Extended Services application is a scaled down version of CRA 3.5.1 and is coresident on
Cisco CallManager.
2-110
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Call Flow—Basic Application (AutoAttendant)
(Cont.)
Extended
Services
Cisco
CallManager
2
1
Step 2: Cisco CallManager signals Extended Services AutoAttendant via
JTAPI that a call has arrived for CTI route point DN (call.received).
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—2-7
The call in this figure routed to the CTI route point. The next steps are for Cisco CallManager
Extended Services to see if the application has spare CTI ports.
Cisco CallManager CTI Manager is communicating to the Extended Services application via
CTIQBE; this occurs even if the Extended Services application is on the Cisco CallManager
server.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Cisco CallManager, Network Signaling, and Dial Plan
2-111
Call Flow—Basic Application (AutoAttendant)
(Cont.)
Extended
Services
Cisco
CallManager
2
1
Step 3: The Extended Services application checks to see how many
sessions and CTI ports are available.
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—2-8
At this point in the call, CTI ports are needed to route the call to the Cisco Extended Services
application. In a four-port application, only four calls can be used at any one time. Although
Step 3 is not presented, the application looks to see how many sessions and CTI ports are
available. There is not a call flow to show this process.
Call Flow—Basic Application (AutoAttendant)
(Cont.)
Extended
Services
Cisco
CallManager
2
4
1
Step 4: The Extended Services application associates the call with a
CTI port and instructs Cisco CallManager to send the call to the CTI
port (call.associated).
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—2-9
The Cisco CallManager JTAPI application in Extended Services is communicating with Cisco
CallManager via the CTI Manager. The application instructs Cisco CallManager via CTI
Manager to process the call in an idle CTI port.
2-112
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Call Flow—Basic Application (AutoAttendant)
(Cont.)
Extended
Services
Cisco
CallManager
2
4
5
1
Step 5: The call arrives at the CTI port, and Extended Services
accepts the call (call.accepted).
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—2-10
At this point in the call, the caller is connected to the Extended Services application over a CTI
port.
Call Flow—Basic Application (AutoAttendant)
(Cont.)
Extended
Services
Cisco
CallManager
2
4
5
6
1
Step 6: The Extended Services application creates a task to handle
processing of the call (script execution). The trace will show something like
“task = 3000000007”.
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—2-11
A task process is generated. This is for the scripts in Extended Services (AutoAttendant
scripts). The task process is like a TCP handle in Cisco CallManager. The task ID follows the
call in Extended Services only.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Cisco CallManager, Network Signaling, and Dial Plan
2-113
Call Flow—Basic Application (AutoAttendant)
(Cont.)
Cisco
CallManager
2
Extended
Services
4
5
7
1
Step 7: The AutoAttendant script answers the call using the “accept”
step (call.answered).
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—2-12
Call Flow—Basic Application (AutoAttendant)
(Cont.)
Cisco
CallManager
2
Extended
Services
4
5
7
1
8
Step 8: A media stream is set up between the IP Phone and AutoAttendant.
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—2-13
A media stream is set up between the user and AutoAttendant. The IP Phone (of the user) and
the application have an RTP stream setup via a CTI port.
2-114
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Call Flow—Basic Application (AutoAttendant)
(Cont.)
Cisco
CallManager
2
Extended
Services
4
5
7
1
8
9
User hears
“Welcome to the AutoAttendant” and so on
Step 9: The Extended Services server plays prompts., accepts user input
(digits), and directs the call.
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—2-14
The user now has full access to the AutoAttendant application. Although this application is
coresident on Cisco CallManager, the user has an RTP stream established with the application.
The stream is over the CTI port.
The main idea in the call flow was to show how the Cisco CallManager CTI Manager, the
applications, and JTAPI all interact. The call flows presented can all be seen in the traces on
Cisco CallManager and in the AutoAttendant trace files on the application engine.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Cisco CallManager, Network Signaling, and Dial Plan
2-115
CTI Route Points and CTI Ports
This topic discusses the specifics related to CTI route points and CTI ports in Cisco
CallManager.
CTI Route Points and CTI Ports
CTI Route
Points
x5000
CTI Ports
X5001
X5002
X5003
X5004
X5005
X5006
X5007
X5008
x5009
JTAPI-TSP
Applications
.
.
.
.
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—2-15
The control of the CTI route point is via the Cisco CallManager Directory User association.
CTI route points are considered pilot points for CTI ports.
CTI Route Points:
You can use Cisco CallManager JTAPI/TAPI to control CTI route points. CTI route points
allow Cisco JTAPI/TAPI applications to redirect incoming calls with an infinite queue
depth. This allows incoming calls to avoid busy signals.
When your application opens a line of this type, it can handle any incoming call by
disconnecting, accepting, or redirecting the call to some other DN.
CTI Ports:
For first-party call control, a CTI port device must exist in the Cisco CallManager. Because
each port can only have one active audio stream at a time, most configurations only need
one line per port.
Until the port is opened, anyone calling the DN associated with that CTI port device
receives a busy or reorder tone.
2-116
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Cisco CallManager JTAPI and TSP
This topic discusses the specifics related to Cisco CallManager JTAPI and TSP.
Cisco CallManager JTAPI and TSP Plug-In
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—2-16
Cisco CallManager JTAPI is used for Java-based applications. Cisco CallManager TSP is used
for TAPI-based applications, except when involving Cisco Unity; in those cases, Cisco UnityTSP is Skinny-based.
Cisco CallManager JTAPI and TSP applications require that users be administered in the
directory and be given privilege to control one or more devices.
When in doubt about the version you are running on any given machine, it is always best to
download the lastest version of Cisco CallManager.
In addition, because the versions of JTAPI and TSP change with Cisco CallManager releases, it
is always best to load the latest versions after an upgrade.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Cisco CallManager, Network Signaling, and Dial Plan
2-117
Troubleshooting Cisco CallManager CTI Manager
This topic discusses the specifics related to troubleshooting Cisco CallManager CTI Manager.
Troubleshooting Cisco CallManager CTI
Manager
Service
Activated
Started
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—2-17
Remember that if the client device JTAPI or the TSP versions (or both) do not match, the
application will probably fail to log into Cisco CallManager. This will result in CTI route
points and CTI ports not registering with Cisco CallManager.
Make sure that you have Layer 3 connectivity to the Cisco CallManager JTAPI or TSP
application (or both) residing in the client network. Ping the client device, and from the client
device, ping the Cisco CallManager CTI Manager.
The key is to make sure that the Cisco CallManager CTI Manager service is started and that
you do not see any Cisco CallManager CTI Manager information or errors in the Microsoft
Windows 2000 Event Viewer (application log).
Depending on the actual problem, the client application may not be authenticating with Cisco
CallManager, which would mean that the client applications cannot pass the first part of
activation. Check to see if DC Directory services are started and make sure the user is in the
DC Directory. If the Active Directory (AD) is being used for directory services, check the
system administrator of the primary domain controller (PDC) that the client application user
has permission to authenticate with Cisco CallManager.
2-118
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Cisco CallManager JTAPI and
TSP
This topic discusses the specifics related to troubleshooting Cisco CallManager JTAPI and
TSP.
Troubleshooting Cisco CallManager JTAPI
and TSP
CTI Manager Services
Application
Provider IP Address
IP Address to the Provider
User ID:
Password:
CTI Enable: checked
Device Association:
Network
User ID:
Password:
JTAPI and TSP Version
=
CTI Association:
JTAPI and TSP Version
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—2-18
When configured, JTAPI and TSP applications are intergrated with the Cisco CallManager
directory where the directroy account is associated with devices for third-party and first-party
call control. If the client side cannot authenticate with Cisco CallManager, the application will
fail. When the JTAPI or TSP application launches on the client side (which means CRA as
well), there is an LDAP authentication that takes place before the Cisco CallManager CTI
Manager will open a channel to allow communications with the Cisco CallManager server.
Here are the common issues with Cisco CallManager JTAPI:
The JTAPI user ID or password in Cisco CallManager was changed.
The JTAPI provider DNS is not working. (In this case, it is recommended that you use an
IP address.)
There was a network event; for example, there was a network outage of some kind, and the
client device could not communicate with Cisco CallManager CTI Manager.
An incorrect version of Cisco JTAPI is installed on the client machine. This usually
happens as a result of a Cisco CallManager upgrade.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Cisco CallManager, Network Signaling, and Dial Plan
2-119
If after the Cisco CallManager JTAPI has been installed and the applciation has been working
for some time, you now find that the application has failed for some reason, follow these steps
to resolve the issue:
Step 1
See if you can ping the Cisco CallManager CTI Manager from the device.
Step 2
See if there have been any changes to Cisco CallManager or to the servers or client
devices.
Step 3
Make sure that the server or the client device (or both) run the same version of
JTAPI as on Cisco CallManager.
Step 4
Make sure that the user in the Cisco CallManager Directory and the JTAPI user
haveidentical usernames and passwords in Cisco CallManager and that the Enable
CTI Application Use box is checked.
Step 5
Make sure that the JTAPI user is associated to control CTI route points and ports.
Step 6
Ensure that the Cisco CallManger CTI Manager service is started.
Step 7
If all else fails, set up Cisco CallManager to collect traces for the Cisco TAC. You
may want to set up traces for SDL traces.
Under System Parameters, set these parameters:
sdlTraceTypeFlags = 0x8000CB15
sdlTraceDataFlags = 0x00000110
sdlTraceDataFlags = True
Collect Cisco CallManager and SDL traces after re-creating the issue, if possible.
When troubleshooting Cisco CallManager JTAPI in CRA, follow these steps:
2-120
Step 1
Verify that the Engine is running and check the status of the subsystems.
Step 2
Verify in Cisco CallManager and on the Cisco Customer Response Applications
(CRA) server that CTI route points match.
Step 3
Verify in Cisco CallManager and on the CRA server that CTI ports match and are in
a consecutive range of numbers.
Step 4
Verify in Cisco CallManager and on the CRA server that CTI route points and CTI
ports are associated with the designated user.
Step 5
Verify that Telephony Applications are enabled.
Step 6
The JTAPI provider on the CRA Engine and the user ID and password must match
Cisco CallManager.
Step 7
It is always safe to download a fresh Cisco CallManager JTAPI plug-in to ensure
that you have the correct JTAPI version.
Step 8
Verify that the DC Directory configuration has not changed; it is recommended that
you use IP addressing as opposed to DNS.
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Step 9
CRA has tracing ability. To set up traces, choose System>Engine> Trace Files.
These files can provide valuable information regarding how the CRA server sees the
issue. Make sure that the CRA Engine is running and that the appropriate
subsystems are in the IN_SERVICE status.
Step 10
Remember that the JTAPI provider is Cisco CallManager CTI Manager. Use an IP
address rather than DNS name for the address in this field.
CTI route points and CTI ports will not register with Cisco CallManager in a JTAPI or TSP
applications unless the following occur:
The user ID and password are consistent in both the Cisco CallManager and client device.
The CTI route points and ports are associated, and the Enable CTI Application Use box is
checked.
The versions are consistent.
If the JTAPI subsystem in the CRA server shows PARTIAL_SERVICE, you probably have
misconfigured something or the CTI port range defined in the CRA does not match. In CRA
3.0 and later, CRA Administration will not let you configure a particular CTI route point or
ports until the CRA server has been authenticated with Cisco CallManager.
Setting up Traces for client side Cisco JTAPI:
Use the Cisco CallManager JTAPI tracing preferences application (JTPREFS.EXE) to
configure trace levels and trace destinations. Installation of the Cisco JTAPI Preferences utility
into the Program Files\JTAPITools directory takes place by default. To open the Cisco JTAPI
Preferences utility, choose Start > Programs > Cisco JTAPI > JTAPI Preferences.
Troubleshooting Cisco JTAPI in Cisco CallManager 4.0:
http://www.cisco.com/en/US/partner/products/sw/voicesw/ps556/products_administration_guid
e09186a00801ed6e7.html.
Some common Issues with Cisco CallManager TSP are listed. A client-side application has
failed for a variety of reasons, including the following:
Someone changed the user ID or password or both.
DNS is not working. (In this case, it is recommended that you use an IP address instead.)
An incorrect version of TSP is installed on the client machine, including CRA.
There was a network event; for example, there was a network outage of some kind, and the
client device could not communicate with Cisco CallManager CTI Manager.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Cisco CallManager, Network Signaling, and Dial Plan
2-121
When troubleshooting Cisco CallManager TSP, use the same steps as for Cisco CallManager
JTAPI, but also add these steps:
Step 1
See if you can ping the Cisco CallManager CTI Manager from the device.
Step 2
Validate the TSP version, as follows:
Choose Start>Settings> Control Panel > Phone and Modem
Options>Advanced tab> on the client device. Double-click
CiscoTSP(instance).tsp and note the TSP version. Look up the Cisco
CallManager version and validate that the TSP version is for that Cisco
CallManager release.
To be safe, download Cisco CallManager TSP off Cisco CallManager to the
client device. If you download the TSP and the application asks you if you want
to upgrade, you know that the TSP version on the client device is not the same
as on Cisco CallManager; at that point, follow the upgrade procedures. If, on the
other hand, the application asks if you want to reinstall, that is a sure indicator
that the version is the same.
Step 3
When setting up traces, you can set the traces up on the client machine. The log files
are found by doing a search on TraceFiles. When setting traces, always set them to
Detailed.
Step 4
You can set up Cisco CallManager CTI Manager traces and see what the Cisco
CallManager sees. In the CTI traces, you can see the LDAP access lookup and
whether the Cisco CallManager found the user in the directory.
When troubleshooting Cisco CallManager TSP used with third-party software, always make
sure that the TSP version being used has been tested and certified by Cisco. The Cisco TAC has
encountered many cases in which a third-party software application quit working after a Cisco
CallManager cluster upgrade because of changed TSP versions. This also applies to third-party
software using JTAPI.
Remember the following points:
JTAPI and TSP applications communicate with Cisco CallManager after authenticating.
CTI route points and CTI ports will not register with Cisco CallManager until after
successful authentication.
2-122
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Summary
This topic summarizes the key points discussed in this lesson.
Summary
• Cisco CallManager CTI Manager allows external
applications to communicate with Cisco CallManager.
• Cisco CallManager JTAPI and TSP use CTIQBE to
interface with Cisco CallManager- via CTI Manager.
• CTI route points and CTI ports are used to route
external application calls to and from Cisco
CallManager.
• Most issues related to JTAPI and TSP are due to
misconfiguration; this includes CRA server
Telephony Applications setup, JTAPI provider, and
DC Directory services.
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—2-19
References
For additional information, refer to this resource:
Troubleshooting Guide for Cisco CallManager, Release 4.0(1):
http://www.cisco.com/univercd/cc/td/doc/product/voice/c_callmg/4_0/trouble/4_0_1/index.
htm.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Cisco CallManager, Network Signaling, and Dial Plan
2-123
Quiz
Use the practice items here to review what you learned in this lesson. The correct answers are
found in the Quiz Answer Key.
Q1)
Cisco JTAPI and Cisco TSP communicate with Cisco CallManager over which
protocol.
A)
B)
C)
D)
Q2)
What is Cisco JTAPI used for?
A)
B)
C)
D)
Q3)
B)
C)
D)
Cisco JTAPI
Cisco TSP
Cisco CAR
Cisco TAPI
Which of the following is the most probable cause of a Cisco CallManager JTAPI
subsystem being in partial service?
A)
B)
C)
D)
2-124
to provide an audio stream termination point; also know as first-party call
control
to provide a virtual device that can receive multiple, simultaneous calls for
application-controlled redirection
both A and B
none of the above
A PC with a Cisco IP SoftPhone loaded on it uses which application to communicate
with Cisco CallManager CTI Manager?
A)
B)
C)
D)
Q5)
to provide call routing and digital analysis
for Cisco CallManager only
for AutoAttendant only
to communicate with Cisco CallManager CTI Manager over the Cisco
proprietary Quick Buffer Encoding interface
What is the main function of a CTI port?
A)
Q4)
HTTP
CTIQBE
UDP port 2000
TCP port 80
user ID and password mismatch
CTI route points and CTI ports unregistering
application engine stopping
Cisco CallManager CTI Manager service starting
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Quiz Answer Key
Q1)
B
Q2)
D
Q3)
D
Q4)
A
Q5)
B
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Cisco CallManager, Network Signaling, and Dial Plan
2-125
2-126
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Cisco
CallManager Upgrades
Overview
Upgrading a Cisco CallManager cluster is not a remedial task and there are preupgrade and
postupgrade checks that must be done. This lesson discusses the problems and issues related to
Cisco CallManager upgrades, providing insight into some problems that can occur and how to
troubleshoot upgrade issues. This lesson will also discuss upgrade planning and where to obtain
documentation to ensure that your upgrade goes smoothly.
Relevance
Planning and preparing a cluster for an upgrade is vital to a smooth upgrade. When upgrading
your Cisco CallManager cluster, there are preupgrade and postupgrade tasks that must be
followed. Not following the steps to upgrade using the documentation available at Cisco.com
increases the probability of experiencing an upgrade failure.
Objectives
Upon completing this lesson, you will be able to:
Explain the benefits of upgrading a Cisco CallManager cluster
Discuss the restrictions around upgrading from different Cisco CallManager versions
Explain the process of upgrading a Cisco CallManager cluster
Identify the issues and problems that may occur as a result of upgrading Cisco CallManager
Apply troubleshooting methods for solving Cisco CallManager upgrade-related problems
Learner Skills and Knowledge
To benefit fully from this lesson, you must have these prerequisite skills and knowledge:
Completion of Cisco IP Telephony (CIPT)
Basic knowledge of Windows 2000 system administration
Outline
The outline lists the topics included in this lesson.
Outline:
• Overview
• Cisco CallManager Upgrades
• Upgrade Restrictions and Procedures
• Cisco CallManager Upgrade Assistant Utility
• Issues That Can Arise from Upgrades
• Resolving Upgrade Issues
• Summary
• Quiz
© 2004 Cisco Systems, Inc. All rights reserved.
2-128
Cisco IP Telephony Troubleshooting (IPTT) v4.0
IPTT v4.0—2-3
Copyright © 2004, Cisco Systems, Inc.
Cisco CallManager Upgrades
This topic discusses the specifics related to upgrading a Cisco CallManager cluster.
Why Upgrade?
• New or improved telephony features
• New or improved system monitoring tools
• New or improved security features
• Defect resolution
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—2-4
There are many reasons to upgrade Cisco CallManager. Usually, customers upgrade for two
main reasons: to obtain feature enhancements and to resolve bugs found in the current Cisco
CallManager version. Upgrades can be done by ordering CD-ROMs at
http://www.cisco.com/upgrade, or the upgrade can be downloaded by accessing the Software
Center from Cisco.com. Cisco CallManager upgrades to version 4.0(1) will require that you
order CD-ROMs. No downloadable upgrades are offered at this time on Cisco.com.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Cisco CallManager, Network Signaling, and Dial Plan
2-129
Difference Between Upgrading and
Patching
• Upgrading
– Is a more complex, more permanent process to
fix bugs and introduce new features or
enhancements
• Patching
– Is a quick temporary fix—a reversible process to
fix a software bug
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—2-5
It is important to understand the difference between upgrading and patching. Patching provides
the means to apply a temporary fix to resolve software bug fixes; conversely, upgrading
provides the means to apply software fixes and introduces new features and enhancements.
There are two kinds of upgrades, major and minor. Major upgrades generally contain numerous
new features and functionality and increase the major number release number. Minor releases
generally contain a few small features and several bug fixes and enhancements. Minor releases
increment the minor release such as 4.0(1) to 4.1(0) or 4.0(1) to 4.0(2).
Here is an example of the Cisco CallManager release template:
'MWGS'EPP1EREKIV WV
4: = major release
0: = minor release
(1): = maintenance release
sr2: = service release
Major releases usually come out once per year. Minor releases may come out twice per year.
Maintenance releases may come out twice per year. Service releases may come out once per
month or once every two months, depending on the requirements.
2-130
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Upgrade Restrictions and Procedures
This topic discusses the specifics related to upgrade restrictions and procedures.
Upgrade Restrictions
Review Cisco CallManager compatibility matrix
To
Cisco
Cisco
CallManager CallManager
4.0(1)
3.3(3)
Cisco
CallManager
3.3(2)
Cisco
CallManager
3.2(3)
3.2(2c), 3.2(3),
3.3(2), 3.3(3).
3.1(4a), 3.1(4b), 3.1(4a), 3.1(4b),
3.2(2a), 3.2(2c), 3.2(2a), 3.2(2c).
3.2(3), 3.3(2).
Upgrade from
3.2(3) is not
supported due to
defect. Go to
3.3(3) directly.
3.0(12), 3.1(3a),
3.1(4a), 3.1(4b),
3.2(2a), 3.2(2c).
Minimum
SQL2k sp3 and
OS 2000.2.5sr2
required prior
to upgrade
Minimum
SQL2k sp3 and
OS 2000.2.4
required prior
to upgrade
Minimum SQL7 sp4
and OS 2000.2.3
required prior to
upgrade
From
© 2004 Cisco Systems, Inc. All rights reserved.
Minimum SQL2k
sp2 and OS
2000.2.3 required
prior to upgrade
IPTT v4.0—2-6
This figure shows the upgrade path from and to various releases. There are server operating
system requirements for certain Cisco CallManager upgrades; thus, you must review the Cisco
CallManager Compatibility Matrix before attempting any upgrade. Access the Cisco
CallManager Compatibility Matrix using this URL:
http://www.cisco.com/univercd/cc/td/doc/product/voice/c_callmg/ccmcomp.htm.
There are minimum requirements that must be met prior to a Cisco CallManager being
upgraded to the next release. The Cisco CallManager Compatibility Matrix will provide the
required information.
To upgrade a Cisco CallManager 3.1(4b) cluster to Cisco CallManager 4.0, you would first
need to upgrade the publisher to Cisco CallManager 3.3(3). To upgrade to Cisco CallManager
3.3(3), you would need to apply an OS and Structured Query Language (SQL) upgrade. After
the publisher is upgraded and operational, you would then upgrade the subscribers directly to
Cisco CallManager 4.0.
To upgrade a Cisco CallManager 3.2(2a) cluster to Cisco CallManager 4.0, you would first
need to upgrade the publisher to Cisco CallManager 3.3(3). To upgrade to Cisco CallManager
3.3(3), you would need to apply an OS and SQL upgrade. After the publisher is upgraded and
operational, you would then upgrade the subscribers directly to Cisco CallManager 4.0.
Note
Upgrades from Cisco CallManager 3.3(4) are not supported at this time. To upgrade to
Cisco CallManager 4.0 on a 3.3(4) version, the cluster will need to be rebuilt under the 4.0
version.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Cisco CallManager, Network Signaling, and Dial Plan
2-131
Upgrade Restrictions (Cont.)
To
From
Cisco
CallManager
3.2(a) & (c)
Cisco
CallManager
3.2(1)
Cisco
CallManager
3.1(4a) & (4b)
3.0(12), 3.1(2c),
3.1(3a), 3.1(4a),
3.1(4b), 3.2(1).
3.0(12), 3.1(2c),
3.1(3a).
3.0(12), 3.1(2c),
3.1(3a).
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—2-7
To upgrade a Cisco CallManager 3.1(3a) cluster to Cisco CallManager 4.0, you would first
need to upgrade the publisher to either Cisco CallManager 3.2(1), 3.2(2a), 3.2(2c) or 3.2(3). In
this scenario, you would want to select the latest version to upgrade to that would get you
closest to the target version. For example, you would want to take the 3.1(3a) publisher to
3.2(2c), then to Cisco CallManager 4.0. After the publisher is upgraded and operational, you
would then install Cisco CallManager 4.0 on the subscribers.
Note
2-132
In upgrading your Cisco CallManager cluster, you must follow the OS, SQL, and
Cisco CallManager upgrade path or the upgrade will fail. If you have questions
about the requirements in upgrading your Cisco CallManager cluster, visit the
Cisco CallManager Compatibility Matrix web page on Cisco.com and crossreference your current Cisco CallManager version with the target Cisco
CallManager. Make note of the OS and SQL upgrade requirements, then pull down
the Cisco CallManager upgrade guide on the target Cisco CallManager version and
follow the preupgrade and postupgrade procedures. If you have any questions
regarding the upgrade procedures, do not hesitate to open a priority 4 case with the
Cisco TAC.
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Upgrade Restrictions (Cont.)
• Cisco CallManager and 3.0, 3.1, and 3.2 require
Microsoft SQL 7.0.
• Cisco CallManager 3.3, 4.0, and above require
Microsoft SQL 2000.
• Upgrade from 3.0, 3.1, 3.2 to 3.3.3—bypass 3.3.2—
defect in DC Directory.
• Avoid creating passwords with special characters.
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—2-8
A majority of upgrade failures are due to missteps in the upgrade process. When preparing for
an upgrade, become familiar with what is required to get to the target version of Cisco
CallManager and always have a back-out plan. If you find that during the upgrade the progress
stops, you will need to reimage from the start. Always have a good backup of the Cisco
CallManager database and of the entire system.
Note
Cisco CallManager 3.0, 3.1, and 3.2 have their databases in the Microsoft SQL Server 7.0
structure. Cisco CallManager 3.3 and 4.0 have their databases in Microsoft SQL Server
2000. Microsoft SQL Server 7.0 will be converted to Microsoft SQL Server 2000 in the
upgrade process, but not in reverse.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Cisco CallManager, Network Signaling, and Dial Plan
2-133
Upgrade Procedures
• Review the Cisco CallManager Compatibility
Matrix.
• Draft an upgrade plan including a contingency plan
to back out of the upgrade.
• Run Cisco CallManager Upgrade Assistant Utility
on the publisher and subscribers.
• Follow the upgrade guide preupgrade and
postupgrade tasks.
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—2-9
Planning is the key to a successful Cisco CallManager cluster upgrade. That advice is stated
over and over again in this lesson. Best practice dictates that you know your preupgrade and
postupgrade procedures. Becoming familiar with upgrade procedures will make your upgrade
process smooth and will require less downtime.
Here is the link to upgrade to Cisco CallManager 4.0(1):
http://www.cisco.com/univercd/cc/td/doc/product/voice/c_callmg/4_0/install/upgrade/index.ht
m.
Best practices dictate that if you have a Cisco Media Convergence Server (MCS) with a
replaceable drive, purchase a spare drive and use this drive during the upgrade process. Follow
the drive removal process and steps to assist in recovering incase an upgrade fails. This link has
the steps for disk drive removal:
http://www.cisco.com/univercd/cc/td/doc/product/voice/c_callmg/4_0/install/upgrade/upgr401/
33upgr.htm#wp1042391.
This applies only for the MCS-7830, MCS-7835, or MCS-7845: Removing a drive with the
current system software allows you to recover back to the current system software incase of an
upgrade failure.
Tip
2-134
The Cisco TAC recommends that you download the upgrade guide on the version of Cisco
CallManager that you want to go to, review it, and follow every steps that pertain to your
release of Cisco CallManager. If you have any questions regarding the procedures, open a
severity 4 TAC service request and ask a Cisco TAC engineer review the upgrade guide
with you.
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Upgrade Procedures (Cont.)
• After you download the necessary upgrade files, it
is best to burn the files to a DVD or CD-ROM.
• Upgrade one server at a time.
• First upgrade the publisher, followed by the
subscribers.
• Subscribers should pull the latest database from
the publisher.
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—2-10
After downloading the upgrade files, it is best practice to burn those files to a DVD or CDROM, if possible.
The sequence in upgrading a Cisco CallManager cluster always starts with the publisher
followed by the subscribers. Upgrade one server at a time. For example, in a five-server cluster,
upgrade the publisher, then each subscriber, making sure that when a subscriber comes on line,
it synchronizes its database with the publisher.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Cisco CallManager, Network Signaling, and Dial Plan
2-135
Cisco CallManager Upgrade Assistant Utility
This topic covers the specific issues related to using the Cisco CallManager Upgrade Assistant
Utility before upgrading a Cisco CallManager server.
Cisco CallManager Upgrade Assistant
Overview
• Cisco created the Cisco CallManager Upgrade
Assistance Utility to help users determine whether
a given system is ready for an upgrade.
• You can obtain the Cisco CallManager Upgrade
Assistant Utility from the web.
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—2-11
The Cisco CallManager Upgrade Assistant Utility, a nonintrusive tool that does not change the
state of the system, detects the health of the servers in the Cisco CallManager cluster before
you perform an upgrade to Cisco CallManager 3.3(3).
Use Cisco CallManager Upgrade Assistant Utility Version 3.3(3a) if you plan to upgrade to
Cisco CallManager 3.3(3) from a compatible release of Cisco CallManager 3.1, 3.2, or 3.3.
The Cisco CallManager Upgrade Assistant Utility will not run if the server is not at the
minimum requirement for the Cisco CallManager 3.3(3) upgrade. This version-specific utility
identifies problems that could cause the Cisco CallManager upgrade to fail.
The Cisco CallManager Upgrade Assistant Utility does not correct the problem or problems;
you must perform the corrective action for any problem that the utility identifies. Cisco
strongly recommends that all servers in the cluster pass the validation before you upgrade any
servers to Cisco CallManager 3.3(3).
Your server login account must have administrative privileges to run the utility. You may log in
to the server by using the CCMAdministrator account username and password. Before you
begin the upgrade on the publisher database server, you must run the utility on all servers in the
cluster.
If any server fails the validation process, investigate and correct any problems before you begin
the upgrade on the publisher server. After you correct the problem or problems, run the utility
again before you upgrade. You can run this utility on all servers at the same time. Cisco
strongly recommends that you run this utility during a scheduled, maintenance window.
2-136
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
The Cisco CallManager Upgrade Assistant Utility checks the following on the servers:
Backup file integrity validation
Database setting validation
Cisco CallManager database validation and replication
Software version validation
DC Directory health check validation
Microsoft Windows domain validation
Security settings validation
Host name resolution validation
The validation results of the upgrade utility launch will display in the Cisco CallManager
Upgrade Assistant Summary window.
At the top of the window, a report summarizes the results for all modules and displays which
modules failed and which modules passed.
A link to the folder that contains all log files, including the Cisco CallManager Upgrade
Assistant Summary report, also displays. To identify a problem with the failed validation
module, review the following information that displays in the Summary window:
The first link points to the log file that specifies the error.
Click the first link and search for the error; for example, ERR: <message>.
The second link points to the corrective action file that describes the log file error message and
recommends the corrective action.
Click the second link to open the corrective action file. Search the corrective action file for the
error message that is noted in the log file. Review the description and corrective action.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Cisco CallManager, Network Signaling, and Dial Plan
2-137
Issues That Can Arise from Upgrades
This topic discusses the issues related to problems that can arise from upgrading a server.
Issues That Can Arise from Upgrades
• Unresolved upgrade utility issues
• Third-party vendor software may block services
from stopping (example NetIQ and unsupported
virus software)
• Possible DVD/CD-ROM drive failure (laser reader)
• Third-party CDR scripts (CDR failure)
• CRS/CRA, Cisco Conference Connection
(Cisco JTAPI, Cisco TSP, SQL CDR scripts)
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—2-12
A majority of upgrade issues come about after the upgrade because external systems to Cisco
CallManager were not updated after the upgrade. Servers that use Cisco CallManager JTAPI
and Cisco CallManager TSP all need to updated as the versions have changed once an upgrade
has occurred.
Always consider the impact an upgrade has on other severs that communicate with Cisco
CallManager. Additional downtime can be avoided. There can also be an impact on third-party
vendor scripts because the scripts prior to the upgrade pointed to a database version number
that no longer exists after an upgrade. Most third-party scripts do not update after an upgrade.
The vendor needs to know ahead of time what actions to take to avoid long downtime delays.
There are cases where the DVD/CD-ROM drive on an MCS server has failed, and thus the
upgrade cannot proceed. In most cases, a DVD/CD-ROM drive can be delivered on site.
2-138
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Issues That Can Arise from Upgrades
(Cont.)
• There may be inconsistent account passwords.
• Cisco requires that the following account
passwords be the same on every server in the
cluster:
– CCMAdministrator, BackAdmin, CCMEML,
CCMService, CCMServiceRW, CCMUser,
SQLSvc, SQL sa
• Cisco strongly recommends that special charters
not be used in account passwords.
• Many installations and upgrades complete
unsuccessfully because of password issues.
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—2-13
When upgrading a cluster, always be careful when entering passwords on the servers, because
incorrectly entered passwords can cause upgrades to fail.
Tip
Many keyboard, video, and mouse (KVM) switches inadvertently change the Caps Lock
status when switching between systems. Be sure to double-check that Caps Lock is off
before entering any passwords.
With respect to Cisco CallManager 4.0(1), the following account passwords need to be the
same in all the servers:
CCMAdministrator account: This account serves as the default Microsoft Windows NT
administration account. Cisco CallManager does not use this password.
BackAdmin account: The BackAdmin account supports the backup service on a Cisco
CallManager.
CCMCDR account: The CCMCDR account supports the Cisco CDR Insert service, the
Cisco Tomcat service, and the CDR Analysis and Reporting (CAR) tool.
CCMEML account: The CCMEML account supports the Cisco CallManager Extension
Mobility Logout service.
CCMService account: The CCMService account supports the Cisco Extended Functions
service and the Cisco Real-Time Information Server (RIS) Data Collector service.
CCMServiceRW account: The CCMServiceRW account supports Cisco CallManager and
the Cisco CallManager CTI Manager service.
CCMUser account: Use the CCMUser account for anonymous access to the Cisco
CallManager website. When you are accessing the Cisco CallManager web pages, this
account gives you access without logging into Microsoft Windows NT.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Cisco CallManager, Network Signaling, and Dial Plan
2-139
SQLSvc account: The SQLSvc account functions as the core account for server-to-server
interaction within a Cisco CallManager system. This account supports the Database Layer
Monitor service and must be the same on every machine in the cluster for the database
replication to work properly.
SQL Server Administration account: This account serves as the default SQL server
administration account. You only use the SQL Server administration password during
installation and migration.
2-140
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Resolving Upgrade Issues
This topic discusses the specifics related to how to resolve upgrade issues.
Resolving Upgrade Issues
• Install logs are found in C:\Program Files>
Common file> Cisco> Logs.
• Error messages can appear when upgrading from
Cisco CallManager 3.2 or 3.3 to 4.0.
• Do not forget to update or check LMHOST files. Not
updating or checking these files can cause major
issues with subscribers database replication.
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—2-14
In the event that an upgrade fails, use the logs to see during which process in the upgrade the
failure occurred. When upgrading from Cisco CallManager versions 3.2 or 3.3 to Cisco
CallManager 4.0, you may run into error messages that appear during the upgrade. If errors
appear, refer to the upgrade guide for resolution.
There are little nuances in upgrading from various Cisco CallManager versions. It is very
important that once you review the appropriate upgrade guide, you fully understand what is
required to successfully upgrade. The Cisco TAC receives many cases on failed upgrades, and
the culprit is missed steps. On occasion, you may run into a bug; however, there are fewer and
fewer bugs with the newest Cisco CallManager releases.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Cisco CallManager, Network Signaling, and Dial Plan
2-141
Summary
This topic summarizes the key points discussed in this lesson.
Summary
• Cisco CallManager cluster upgrades require
planning.
• Upgrade one server at a time.
• Account passwords must be consistent, with no
special characters in the password strings.
• Upon an upgrade failure, review the upgrade logs.
• If you have questions relative to upgrading your
cluster, open a priority 4 case with the Cisco TAC.
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—2-15
References
For additional information, refer to these resources:
Cisco CallManager Compatibility Matrix:
http://www.cisco.com/univercd/cc/td/doc/product/voice/c_callmg/ccmcomp.htm.
Cisco CallManager 4.0(1) error messages and solutions:
http://www.cisco.com/univercd/cc/td/doc/product/voice/c_callmg/4_0/install/upgrade/upgr
401/error.htm.
Cisco CallManager Upgrade Assistant Utility for Cisco CallManager 4.0(1):
http://www.cisco.com/cgi-bin/tablebuild.pl/callmgr-40. (You will need to log in to CCO.)
2-142
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Quiz
Use the practice items here to review what you learned in this lesson. The correct answers are
found in the Quiz Answer Key.
Q1)
Within a Cisco CallManager cluster, which accounts need to be the same in all servers?
A)
B)
C)
D)
Q2)
Before upgrading your Cisco CallManager cluster, which utility do you run on all
servers?
A)
B)
C)
D)
Q3)
C:\Program Files\Common file\Cisco\Logs
Event Viewer
RTMT
Cisco CallManager Traces
Which account supports the Database Layer Monitor service?
A)
B)
C)
D)
Q6)
TFTP server
Cisco CallManager CTI Manager sever
publisher
subscriber
If your publisher failed to upgrade, where would you look first to find the possible
cause?
A)
B)
C)
D)
Q5)
DCD scripts
Cisco CallManager Upgrade Assistant Utility
backup utility
none of the above
When upgrading a Cisco CallManager cluster, which server do you upgrade first?
A)
B)
C)
D)
Q4)
SQLSvc
CCMService
CCMCDR
all of the above
CCMUser
SQLSvc
SQL Server Administration
Windows NT administration
Best practice dictates that you know your _____ and _____ procedures before
upgrading your Cisco CallManager cluster.
A)
B)
C)
D)
account passwords, error messages
install logs, Event Viewer
preupgrade, postupgrade
both A and B
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Cisco CallManager, Network Signaling, and Dial Plan
2-143
Quiz Answer Key
2-144
Q1)
D
Q2)
B
Q3)
C
Q4)
A
Q5)
B
Q6)
C
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Module 3
Cisco AVVID Troubleshooting
Tools
Overview
Constantly changing pathways are inherent in an IP telephony network. To effectively manage
an IP telephony network, you must discover and monitor problems before they negatively
influence service. This module describes the Cisco CallManager troubleshooting tools and how
to use these tools to ensure the health and efficient operation of Cisco CallManager. You will
also learn how to use the basic tools that identify problems with the Microsoft SQL Server
2000 database and how database problems can affect Cisco CallManager functions.
Module Objectives
Upon completing this module, you will be able to troubleshoot Cisco CallManager and Cisco
Architecture for Voice, Video and Integrated Data (AVVID) components using the appropriate
utilities and management tools.
Module Objectives
• Apply appropriate Cisco CallManager and
Microsoft Windows 2000 troubleshooting and
monitoring tools
• Use various SQL and LDAP troubleshooting and
monitoring tools
• Use other tools for troubleshooting and monitoring
Cisco CallManager networks
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—3-2
Module Outline
The outline lists the components of this module.
Module Outline
• Applying Cisco CallManager and Operating System
Troubleshooting Tools
• Using Database Tools
• Troubleshooting Using Other Tools
© 2004 Cisco Systems, Inc. All rights reserved.
3-2
Cisco IP Telephony Troubleshooting (IPTT) v4.0
IPTT v4.0—3-3
Copyright © 2004, Cisco Systems, Inc.
Applying Cisco CallManager
and Operating System
Troubleshooting Tools
Overview
Cisco CallManager software and the underlying operating system and database software play a
significant role in the day-to-day operation of an IP telephony network. This lesson will teach
you the basic tools that you will use to identify problems with the IP telephony network. You
will learn about the Microsoft Windows 2000 Event Viewer, Microsoft Performance Monitor,
and the Cisco CallManager Administrative Serviceability Tool (AST). You will also learn
about the accounts and passwords that are inherent to Cisco CallManager.
Relevance
You can use Microsoft Windows 2000 tools to identify many problems with the configuration
or operation of a Cisco CallManager server. However, Cisco CallManager also has several
tools to help you identify problems before they occur and troubleshoot any problems that do
arise. This lesson will benefit those individuals who want to increase their understanding of the
process and the tools that are available to troubleshoot Cisco CallManager and the platform
upon which it operates.
Objectives
Upon completing this lesson, you will be able to:
Recognize Cisco CallManager component versions
Describe the functions of Cisco CallManager AST
Explain Cisco CallManager alarm functionality and targets
Describe the functions of the Real-Time Monitoring Tool
Describe the functions of the Microsoft Windows 200 Event Viewer
Describe Microsoft Performance Monitor functions
List the troubleshooting issues regarding accounts and passwords
Learner Skills and Knowledge
To benefit fully from this lesson, you must have these prerequisite skills and knowledge:
Basic working knowledge of a computer
Experience with troubleshooting network problems
Basic understanding of Cisco CallManager functionality
Outline
The outline lists the topics included in this lesson.
Outline
• Overview
• Cisco CallManager Component Versions
• Administrative Serviceability Tool
• Alarm Configuration
• Real-Time Monitoring Tool
• Microsoft Windows 2000 Event Viewer
• Microsoft Performance Monitor
• Account Passwords
• Summary
© 2004 Cisco Systems, Inc. All rights reserved.
3-4
Cisco IP Telephony Troubleshooting (IPTT) v4.0
IPTT v4.0—3-3
Copyright © 2004, Cisco Systems, Inc.
Cisco CallManager Component Versions
This topic describes how to access and view the Cisco CallManager Component Versions page.
Cisco CallManager Component Versions
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—3-4
Cisco CallManager Component Versions is a list of all the software pieces that make up an
operational Cisco CallManager. In managing performance and troubleshooting, you must check
the information listed in the Details section. Click the Details button, which is located on the
Cisco CallManager Administration page, to access information about component versions,
database layer (DBL) versions, and database connections.
If the information in Details does not appear, you may have a problem with the Cisco
CallManager configuration or the connection between Cisco CallManager and the Microsoft
Structured Query Language (SQL) Server 2000.
Copyright © 2004, Cisco Systems, Inc.
Cisco AVVID Troubleshooting Tools
3-5
Cisco CallManager Component Versions
(Cont.)
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—3-5
When you are troubleshooting problems or performing upgrades, it is helpful to know the
component version of the Cisco CallManager software that is running. Cisco Systems has
produced many Cisco CallManager point upgrades—for example, 3.2(1], 3.2(2a], 3.2(2c).
Therefore, individual Cisco CallManager file versions may differ between servers.
To find the file version of the Cisco CallManager software that you are troubleshooting, choose
Help > Component Versions, and then choose the specific server you wish to monitor. The
alphabetical list that appears is useful for determining if you have properly upgraded all servers
within the cluster. The example shown is only one of several pages that contain component
version information.
The component version should be identical on all servers in the cluster to ensure accurate Cisco
CallManager intracluster communication.
3-6
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Administrative Serviceability Tool
This topic describes how you can use Cisco CallManager AST to troubleshoot system
problems.
Cisco CallManager Serviceability
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—3-6
Cisco CallManager AST provides you with these services:
Alarm: Alarm saves information about Cisco CallManager service events for
troubleshooting and provides alarm message definitions. You can forward alarms to a trace
file for further analysis.
—
The AST alarms allow you to configure Cisco CallManager to write an event to a
trace file or the Microsoft Windows 2000 Event Viewer when an incident occurs,
such as the failure of a telephone to register. You can configure alarms for Cisco
CallManager servers in a cluster and for the services in each server.
—
The Alarm Definitions application stores alarm definitions and the recommended
actions in a Microsoft SQL Server 2000 database. The system administrator can
search the database for the definitions of all alarms. Definitions include the alarm
name, description, recommended action, severity, parameters, and monitors.
Copyright © 2004, Cisco Systems, Inc.
Cisco AVVID Troubleshooting Tools
3-7
Trace: Trace allows you to save a detailed log of Cisco CallManager events for
troubleshooting system problems. You can configure, collect, and analyze trace data.
—
Use the Trace Configuration application to specify the trace parameters; for
example, the Cisco CallManager server within the cluster, the Cisco CallManager
service on the server, the debug level, and the specific trace fields.
—
Use the Trace Analysis application to provide greater trace detail on a signal
distribution layer (SDL) trace, system diagnostic interface (SDI) trace, a Cisco
CallManager service type, or the time and date of the trace.
—
Use the Trace Collection application to collect the trace from a list of SDL or SDI
log files. You can choose a specific log file from the list and request information
from that log file, such as host address, IP address, trace type, and device name.
—
The Q.931 Translator application filters incoming data from Cisco CallManager SDI
logs and translates the data into Cisco IOS messages. The translator application
displays the message in the message translator interface.
—
New to Cisco CallManager 4.0 is the ability to selectively collect traces needed for
troubleshooting purposes or if required by the Cisco Technical Assistance Center
(TAC). Use this menu in combination with the Cisco CallManager Trace Collection
tool to quickly set up and retrieve Cisco CallManager traces.
Tools: The Tools service offers the following troubleshooting applications:
—
The Service Activation tool can activate and deactivate all of the Cisco CallManager
services for all Cisco CallManager servers.
—
The Control Center tool allows you to start, stop, and view the status of Cisco
CallManager services.
—
Cisco CallManager Serviceability provides a client side standalone plug-in, RealTime Monitoring Tool (RTMT), which monitors real-time behavior of the
components in a Cisco CallManager cluster. RTMT runs as an application and uses
HTTP and TCP to monitor device status, system performance, device discovery, and
computer telephony integration (CTI) applications. RTMT also connects directly to
devices by using HTTP for troubleshooting system problems.
—
View the Cisco IP Phone problem reports that are generated by the Quality Report
Tool (QRT), by using the QRT Viewer. QRT serves as a voice-quality and general
problem-reporting tool for Cisco CallManager IP Phones.
—
The Cisco Serviceability Reporter, a Microsoft Windows NT service, generates the
following five daily reports in Cisco CallManager Serviceability Administration:
Device Statistics Report, Server Statistics Report, Service Statistics Report, Call
Activities Report, and Alert Summary Report. Each report provides a summary
comprising different charts that display the statistics for that particular report. This
service gets installed on all the Cisco CallManager nodes in the cluster. The Cisco
Serviceability Reporter generates reports once a day on the basis of logged
information.
Application: The Application service in AST offers several troubleshooting applications,
including the following:
3-8
—
The Install Plug-Ins application can help you extend the functionality of Cisco
CallManager.
—
The Cisco CallManager Administration application provides a convenient, direct
link to the Cisco CallManager Administration page.
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Help: The Help service provides the following troubleshooting support:
—
The Contents and Index application provides you with online assistance.
—
For This Page is an application that provides online help corresponding to the page
that you are viewing.
—
The Component Versions application displays the latest installed component version
information for all Cisco CallManager servers in the cluster and lists servers in the
cluster with out-of-sync system components.
—
About Cisco CallManager Serviceability is an application that provides a link that
will take you directly to the main page of the Cisco CallManager AST.
Copyright © 2004, Cisco Systems, Inc.
Cisco AVVID Troubleshooting Tools
3-9
Alarm Configuration
This topic discusses the specific issues related to configuring the alarm setting in Cisco
CallManager.
Configuring Alarms
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—3-7
The Alarm Configuration application has two primary functions: configuring alarms and events
and creating alarm message definitions. You can use both functions to troubleshoot Cisco
CallManager problems.You can configure alarms for services running on each server in the
cluster, such as Cisco CallManager, TFTP, and Cisco CallManager CTI Manager.
You can also use alarms to provide run-time data, resolve problems, and access system status;
for example, you can use alarms to determine if telephones are registered and working. Alarm
information explains a specific problem and recommends a specific action. The alarm
information includes the application name, device name, and cluster name so that you can
troubleshoot problems that are not on your local Cisco CallManager server.
You can configure the alarm interface to send alarm information to multiple destinations, and
each destination can have its own alarm event level (from debug to emergency).
When a Cisco service issues an alarm, the alarm interface sends the alarm to the chosen
monitor (for example, an SDI trace). The monitor forwards the alarm or writes it to the final
destination, such as a log file.
You can also send alarm messages to the Microsoft Windows 2000 Event Viewer, the syslog,
an SDI trace log file, an SDL trace log file (for Cisco CallManager and Cisco CallManager
CTI Manager only), or to all destinations. You can use the trace file to collect and analyze the
alarms.
3-10
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Configuring Trace Files
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—3-8
The Cisco CallManager AST offers the web-based Trace tool to help you troubleshoot Cisco
CallManager problems. The Trace tool has these three functions:
Configures trace parameters
Collects trace files
Analyzes trace data for troubleshooting problems
Trace and alarm tools work together. You can configure trace and alarm settings for any of the
Cisco CallManager services. Configuring these settings is done primarily to assist the Cisco
TAC engineers. You can perform traces for Cisco CallManager services based on debug levels,
specific trace fields, and Cisco CallManager devices, such as telephones or gateways. You can
also perform a trace on the alarms that are sent to the SDI or SDL trace log files.
Note
When you enable a trace, system performance decreases. Traces should be enabled for
troubleshooting purposes only.
When troubleshooting Cisco CallManager problems, use the Trace Configuration tool to
specify trace parameters. The Trace Configuration window provides you with two types of
settings: trace filter and trace output.
You can specify these trace parameters:
Cisco CallManager server (within the cluster)
Cisco CallManager service on the server
Debug level
Specific trace fields
Output settings
Copyright © 2004, Cisco Systems, Inc.
Cisco AVVID Troubleshooting Tools
3-11
If the service is a call-processing application, such as Cisco CallManager CTI Manager, you
can configure a trace on devices, such as telephones and gateways. For example, you can
narrow the trace to all enabled telephones with a directory number (DN) beginning with 333.
Note
3-12
To log alarms in the SDI trace log file, you must do the following: check the Trace On box in
Trace Configuration, check the Enable Trace File Log box in Trace Configuration, and
check the SDI Alarm Destination check box in Alarm Configuration.
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Real-Time Monitoring Tool
This topic discusses the specifics relative to RTMT on the Cisco Media Convergence Server
(MCS).
Real-Time Monitoring Tool—Client Tool
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—3-9
Cisco CallManager RTMT is a downloadable client application on Cisco CallManager version
4.0(1) and above. To use RTMT, you must download RTMT from the primary Cisco
CallManager server and install it on a PC. To download RTMT, open a browser window and go
to the Cisco CallManager Administration page. Then go to the Install Plug-Ins page to
download the Cisco CallManager RTMT to your PC. Previous versions of Cisco CallManager
have RTMT running on the server.
You can use RTMT to view the real-time status of all components in a Cisco CallManager
cluster by monitoring and displaying performance and device status information. RTMT also
provides you with an alert-notification mechanism to make troubleshooting easier.
RTMT monitors various aspects of Cisco CallManager performance by periodically polling
Windows 2000 Performance Monitor counter values. RTMT searches by device name, IP
address, or domain name, and monitors the status of discovered devices. Device status
monitoring provides you with an alert-notification mechanism for gateway ports and channels.
You can view the properties of a device or counter by selecting the item and dragging the value
of that item to the monitor area.
Use component version information to determine if there are conflicts between clusters, or to
identify a software release that contains a bug. You can view component version information
by clicking the Information icon located in the bottom-right corner of the RTMT window.
Copyright © 2004, Cisco Systems, Inc.
Cisco AVVID Troubleshooting Tools
3-13
Real-Time Device Monitoring
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—3-10
You can monitor devices to determine conflicts in the overall operation of clusters. Device
monitoring displays information, such as telephone data, available media devices, CTI
connections, and configured voice-mail ports. Device monitoring displays information about
the devices, regardless of the device registration status.
Note
3-14
If you cannot determine which server in the cluster is unable to register a device, you can
view the status of the device in the monitoring section of RTMT.
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Device Monitor: Status, Name, and
Attributes
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—3-11
You can filter the devices that you want to monitor by choosing their shared characteristics,
such as registered or rejected. You can further refine the device monitor view of RTMT by
narrowing the search characteristics (for example, use DNs or subnetworks). Cisco
recommends that you choose only the attributes that you want to monitor, such as Node or
Status.
Note
If you have a reported problem—such as a specific DN that is not registering—you can
narrow the search to that DN only, instead of searching through an entire list of DNs.
Copyright © 2004, Cisco Systems, Inc.
Cisco AVVID Troubleshooting Tools
3-15
Cluster Information
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—3-12
You can view information about cluster operations by clicking the Summary button on the leftside of the Cisco CallManager Serviceability page. Cluster operation information allows you to
quickly determine the system status.
Note
3-16
If users are reporting problems with TFTP requests—such as nonfunctioning music on hold
(MOH) or a Cisco IP Phone ring that is reverting to a default ring—you can quickly check for
failed TFTP requests. This information helps you determine the problem source.
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Configuration Preferences
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—3-13
RTMT monitors the heartbeat of Cisco CallManager servers, Cisco TFTP, and
Cisco Telephony Call Dispatcher (TCD). The heartbeat acts as an indicator of the life of
whatever it is monitoring. When the heartbeat is lost, a blinking icon appears in the lower-right
corner of the RTMT window. To find when the heartbeat loss was detected, click the blinking
icon. An e-mail can notify you of the heartbeat loss.
Note
For information about RTMT, go to
http://www.cisco.com/en/US/partner/products/sw/voicesw/ps556/products_administration_gu
ide_chapter09186a00801ed19a.html#1182610.
Copyright © 2004, Cisco Systems, Inc.
Cisco AVVID Troubleshooting Tools
3-17
Microsoft Windows 2000 Event Viewer
This topic describes how the Microsoft Windows 2000 Event Viewer helps you identify
problems in the Cisco CallManager system.
Microsoft Windows 2000 Event Viewer
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—3-14
You can use the Microsoft Windows 2000 Event Viewer to look for events about specific
gateways, such as registration or reregistration, to pinpoint a problem. Because Cisco
CallManager acts as an application running within Microsoft Windows 2000, the server records
all errors related to the Cisco CallManager software itself to the application log in the event
viewer. The server records any errors related to the foundation Microsoft Windows 2000
system to the system log. You should check these logs on a regular basis to ensure that
application failures are identified.
There are the following three types of logs in the Microsoft Windows 2000 Event Viewer:
Application log: This log contains events that are logged by applications or programs, such
as Cisco CallManager.
System log: This log contains events that are logged by Microsoft Windows 2000 system
components, such as the failure of a system component or a device driver.
Security log: This log contains security events. Cisco CallManager does not report events
in this log.
3-18
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
There are the following three types of events in Microsoft Windows 2000 Event Viewer:
Error: This event indicates a current problem, such as the loss of data or functionality.
Warning: This event indicates a potential problem or future problem. This event does not
necessarily indicate an existing error.
Information: This event indicates the normal system information that Microsoft Windows
2000 may log. Information events appear as the result of normal Cisco CallManager
operations.
Copyright © 2004, Cisco Systems, Inc.
Cisco AVVID Troubleshooting Tools
3-19
Microsoft Performance Monitor
This topic describes how the Microsoft Performance Monitor enables you to display the status
of the Cisco CallManager system and operation.
Microsoft Performance Monitor
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—3-15
The Microsoft Performance Monitor is a Microsoft Windows 2000 system administration
application that can, by configuration, display a wide variety of system and task operational
parameters. This monitor reports information in real time.
You can configure the Microsoft Performance Monitor to monitor and log resource counters,
and more, from both system functions and applications. Cisco CallManager displays network
node information in real time. You can use a single performance monitor to simultaneously
collect data from multiple systems and store this data in a single log file. You can export this
log file into tab-separated values (TSVs) or comma separated values (CSVs), which you can
view in most spreadsheet applications.
You must have statistics enabled in Cisco CallManager to collect the data from the Microsoft
performance Monitor. In addition, you must select and enable Cisco CallManager parameters
for monitoring within the Microsoft Performance Monitor. Cisco CallManager parameters
include items such as object, counter, and instance.
3-20
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Accounts and Passwords
This topic examines the various administration accounts and how they affect Cisco
CallManager.
New Accounts and Passwords
Account Description
BackAdmin
Supports the Backup service on a CCM system
Cisco Call Manager
CDR
Supports the Cisco CDR Insert service, the Cisco Tomcat service, and the
CAR tool
Cisco Call Manager
EML
Supports the CCM Extension Mobility Logout service
Cisco Call Manager
Service
Supports the Cisco Extended Functions service and the Cisco RIS Data
Collector service
Cisco Call Manager
ServiceRW
Supports CCM and Cisco Call Manager CTI Manager services
CCMUser
Use the CCMUser account for anonymous access to the CCM website
SQLSvc
Functions as the core account used for server-to-server interaction within a
CCM system
SQL Server
Administration (sa)
Serves as the default SQL Server administration account
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—3-16
This list describes the administration accounts and how to use them during Cisco CallManager
operation:
CCMAdministrator account: The Administrator account serves as the default Microsoft
Windows 2000 administration account. The Cisco CallManager Administrator account
must have a password set to access Cisco CallManager Administration or Cisco
CallManager Serviceability.
BackAdmin account: The BackAdmin account is used to support the backup service on a
Cisco CallManager system.
CCMCDR account: The CCMCDR account supports the Cisco Call Detail Record (CDR)
Insert service, the Cisco Tomcat service, and the CDR Analysis and Reporting (CAR) tool.
CCMEML account: The CCMEML account supports the Cisco Call Manager Extension
Mobility Logout service.
CCMService account: The CCMService account supports the Cisco Extended Functions
service and the Cisco Real-Time Information Server (RIS) Data Collector service.
CCMServiceRW Account: The CCMServiceRW account supports Cisco CallManager
and Cisco CallManager CTI Manager services.
CCMUser account: You will use the CCMUser account for anonymous access to the
CCM website. When you are accessing CCM web pages, this account gives you access
without logging into Microsoft Windows 2000.
Copyright © 2004, Cisco Systems, Inc.
Cisco AVVID Troubleshooting Tools
3-21
SQLSvc Account: The SQLSvc account functions as the core account that is used for
server-to-server interaction within a CCM system. This account supports the Cisco
Database Layer Monitor service and must be the same on every machine in the cluster for
database replication to work properly. If the SQLSvc password has been changed on the
publisher from the installed default, replication of the publisher database will fail when you
add a new subscriber. If replication fails, change the new subscriber SQLSvc service
password to match the SQLSvc password on the publisher, and replication should succeed.
SQL Server Administration (sa) Account: This account serves as the default SQL server
administration account. You only use this password during installation and migration. Most
of the system does not use this account.
3-22
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Summary
This topic summarizes the key points discussed in this lesson.
Summary
• To troubleshoot problems, you must know the component
version of the Cisco Call Manager software that is running.
The component version should be identical on all servers in a
cluster to ensure accurate Cisco Call Manager intracluster
communication.
• You can configure RTMT to contain the information that you
need to troubleshoot Cisco Call Manager components.
• Microsoft Windows 2000 Event Viewer displays system,
security, and application error logs.
• Microsoft Performance Monitor displays the activities and status
of the Cisco Call Manager system. This monitor reports both
general and specific information in real time and collects Cisco
Call Manager device statistics.
• Each administration account has a specific function in Cisco
Call Manager operation. Although the CCMAdmin account
password may be different on all servers, the SQLSvc account
password must be identical for each device in the cluster.
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—3-17
References
For additional information, refer to this resource:
Administrative Tools Overview:
http://www.cisco.com/univercd/cc/td/doc/product/voice/c_callmg/3_1/sys_ad/3_1_2/ccmsy
s/a10tools.pdf.
Copyright © 2004, Cisco Systems, Inc.
Cisco AVVID Troubleshooting Tools
3-23
Quiz
Use the practice items here to review what you learned in this lesson. The correct answers are
found in the Quiz Answer Key.
Q1)
Which tool would you use to view system-level events on the Microsoft Windows 2000
operating system of Cisco CallManager?
A)
B)
C)
D)
Q2)
Which tool would you use to view charts and statistics about Cisco CallManager?
A)
B)
C)
D)
Q3)
Component Versions
Administrative Serviceability Tool
Microsoft Windows 2000 Event Viewer
Microsoft Performance Monitor
Which of the following tools will allow you to send e-mail alerts following a device
failure?
A)
B)
C)
D)
3-24
default administration account for Microsoft
web administration account for Cisco CallManager
server-to-server interaction by the Cisco CallManager system
Microsoft SQL Server 2000 administration account
Which utility should you use to view the current version of Cisco CallManager
components?
A)
B)
C)
D)
Q5)
Microsoft Windows 2000 Event Viewer
Microsoft Performance Monitor
Component Versions
Users and Computers
What is the primary function of the SQLSvc account?
A)
B)
C)
D)
Q4)
Microsoft Windows 2000 Event Viewer
Microsoft Performance Monitor
Component Versions
Users and Computers
Administrative Serviceability Tool
Microsoft Windows 2000 Event Viewer
Component Versions
Users and Computers
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Quiz Answer Key
Q1)
A
Q2)
B
Q3)
C
Q4)
A
Q5)
A
Copyright © 2004, Cisco Systems, Inc.
Cisco AVVID Troubleshooting Tools
3-25
3-26
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Using Database Tools
Overview
The proper functionality of Cisco CallManager relates directly to the health and operation of
the Microsoft SQL Server 2000 database. This lesson will teach you the basic tools to identify
problems with the Microsoft SQL Server 2000 database and how these problems can affect
Cisco CallManager operation.
Relevance
It is often necessary to work with the Microsoft SQL Server 2000 database. This lesson benefits
those learners who want to increase their understanding of Microsoft and Cisco CallManager
tools. The lesson provides information about how you can use these tools to identify problems,
dealing either with the configuration or with the operation of the Microsoft SQL Server 2000
database.
Objectives
Upon completing this lesson, you will be able to:
Recognize Microsoft SQL Server 2000 functions
Describe Microsoft SQL Enterprise Manager functions
Describe the process for reestablishing replication using Microsoft SQL Enterprise
Manager and the DBLHelper utility
Describe SQL command-line utilities
Interpret Microsoft SQL command-line utilities output
Describe Cisco AVVID directory service utilities
Learner Skills and Knowledge
To benefit fully from this lesson, you must have these prerequisite skills and knowledge:
Basic working knowledge of a computer
Experience troubleshooting network problems
Basic understanding of Cisco CallManager functionality
Outline
The outline lists the topics included in this lesson.
Outline
• Overview
• Introduction to Microsoft SQL Server 2000
• Microsoft SQL Enterprise Manager
• Re-Creating Subscriber Connections
• Microsoft SQL Query Analyzer
• Microsoft SQL Command-Line Utilities
• Cisco AVVID Directory Service Utilities
• Summary
• Quiz
© 2004 Cisco Systems, Inc. All rights reserved.
3-28
Cisco IP Telephony Troubleshooting (IPTT) v4.0
IPTT v4.0—3-3
Copyright © 2004, Cisco Systems, Inc.
Introduction to Microsoft SQL Server 2000
This topic explains the functions of the Microsoft SQL Server 2000 database.
Microsoft SQL Database Replication
Publisher
• Changes made to
the database
replicate via a
transactional
process.
• If the link is down,
SQL keeps a
transaction log
and replicates the
data when
possible.
© 2004 Cisco Systems, Inc. All rights reserved.
SQL
Client
Subscriber
Client
Subscriber
SQL
Client
SQL
Replication
Direction
Client
Computer
IPTT v4.0—3-4
The Microsoft SQL Server 2000 writes database information in the publisher database only.
The Microsoft SQL Server 2000 writes all of the entries made through the Cisco CallManager
web interface on a subscriber server in the publisher database. You will still be able to access
the Cisco CallManager Administration web interface if the publisher server is down because
the Cisco CallManager Administration utility continues to run using the Internet Information
Server (IIS) on all Cisco CallManager servers. However, any attempt to write to the database
when the publisher server is down returns an error message.
The replication of the database within a Cisco CallManager cluster is unidirectional and works
in the following way:
Changes made to the publisher database replicates via a transactional process. The changes
occur immediately, unless the link is down.
Microsoft SQL Server 2000 denies all data entry to the publisher database if the link to the
publisher, or the publisher itself, is down. CDRs are the one exception to this rule. The
Microsoft SQL Server 2000 database writes CDRs to the subscriber database where they
are temporarily stored. The server then replicates CDRs to the publisher when you restore
network connectivity.
The publisher database is writable; the subscriber database is read-only.
Copyright © 2004, Cisco Systems, Inc.
Cisco AVVID Troubleshooting Tools
3-29
Database Services and Microsoft SQL
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—3-5
It is important to understand how the Microsoft SQL Server 2000 database integrates with
Cisco CallManager. Understanding this relationship will help you troubleshoot problems with
the Microsoft SQL Server 2000 database.
To ensure accurate results, read from the publisher database only. The Microsoft SQL Server
2000 writes database information in the publisher database only. This database performs oneway replication (or unidirectional replication).
When building a publisher server, you can choose whether to install Cisco CallManager
Administration utilities or optional media resources, such as the Voice Media Streamer. If you
do not install Cisco CallManager components on the publisher server, it becomes a glass house
database. Glass house databases are common in larger clusters because the load on the
publisher server can be considerable. In smaller clusters, the publisher occasionally acts as a
backup for Cisco IP Phones.
During the creation of the publisher server, the Microsoft SQL Server 2000 tasks run using a
special user account. The publisher replicates data to the subscribers. These subscribers act as
backup databases. All of the tables replicate (except the CDR tables) and are maintained locally
by each Cisco CallManager server. The database name is CM03xx, where xx starts at 00 and
increases in increments with each migration.
3-30
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Microsoft SQL Enterprise Manager
This topic describes the functionality of Microsoft SQL 2000 and the Microsoft SQL Enterprise
Manager.
Publisher or Subscriber Database
Publisher
© 2004 Cisco Systems, Inc. All rights reserved.
Subscriber
IPTT v4.0—3-6
In the Microsoft SQL Enterprise Manager, expand the folders to the database level to determine
if you are working with the publisher or subscriber database. Choose Microsoft SQL
Servers\SQL Server Group\<server_name>\Databases\CCM0300. You will see one of the
following folders:
Publications folder: This folder will exist only on the publisher. To see all of the
subscribers, expand the Publications folder and click CCM0300.
Pull Subscriptions folder: This folder exists only on the subscribers.
Copyright © 2004, Cisco Systems, Inc.
Cisco AVVID Troubleshooting Tools
3-31
Publisher or Subscriber Database (Cont.)
Publisher database:
• Can also be recognized by
the presence of a
replication monitor
• Red circle with a white X in
it indicates an error
• Three important replication
agents:
– Snapshot agent
– Log reader agent
– Distribution agent
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—3-7
The publisher server is responsible for building transaction logs. The publisher server also
functions as a distribution agent when a connection link fails between servers. As a distribution
agent, the publisher server also has a replication monitor that is visible from the Microsoft SQL
Enterprise Manager.
3-32
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Database Replication
Glass House
SQL
Backup
Backup
SQL
Computer
© 2004 Cisco Systems, Inc. All rights reserved.
Application
SQL
Database
Access
Replication
Direction
IPTT v4.0—3-8
Servers read the local registry to find the location of all databases and try to contact the
publisher database first. Node servers read the database to check for additional servers and then
update the list of servers on the registry for the next boot.
The database layer tries to use the publisher database (read and write). If the publisher is not
available, the DBL uses a replicated database that exists on the local system before trying to use
any other options. During a failover, the DBL prevents data from being written to the local
database, except for CDRs.
Note
If you change the password of the SQLSvc replication account on the publisher, replication
of the publisher database will fail when you add a new subscriber.
Copyright © 2004, Cisco Systems, Inc.
Cisco AVVID Troubleshooting Tools
3-33
Recreating Subscriber Connections
This topic discusses the specifics relative to reestablishing SQL replications between a
publisher and subscriber.
Recreating Subscriber Connections
DBLHelper
Publisher
• Obtain DBLHelper from your service
provider before you need it.
• Publish then initialize.
On subscriber (manual process)
• Check status of subscriber database.
• Delete subscriptions.
• Recreate subscriptions.
Subscriber
Subscriber
On publisher (manual process)
• Reinitialize subscriptions.
• Start the replication snapshot agent.
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—3-9
The following three reasons are typically why replication breaks down:
1. Loss of network connectivity
2. SQLSvc passwords do not match
3. Broken subscription between the publisher and subscriber
You can test if replication is broken or failing by adding a dummy phone in the publisher and
looking in the subscriber Cisco CallManager Administration. The new device added should
appear in the Cisco CallManager database on the subscriber. You can either look in the Cisco
CallManager Administration or the SQL database in Enterprise Manager where you return all
rows under the Device database. If you do not see the device in the subscriber, you then have a
replication issue that needs immediate attention.
Loss of network connectivity will result in the publisher not being able to replicate to the
subscriber. Troubleshoot the network loss, and when the network comes back on line, your
servers should synchronize. If the servers do not synchronize, follow the instructions in using
DBLHelper or find the technical tip for reestablishing replication manually on Cisco.com.
3-34
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
To determine if the broken replication is because of a SQLSvc password issue, access the Event
Viewer on the publisher or subscriber and look in the system log. Look for a red circle marked
“Service Control Manager” with a white X through it; double-click the error. You should see
the following error message: “The SQLSERVERAGENT service failed to start due to the
following error: The service did not start due to logon failure.” This error tells you that the
SQLSvc password has changed. At this point, you need to consult the instructions per the
relevant Cisco CallManager release on how to change the SQLSvc password to be consistent
across the entire cluster.
If one or more subscriber connections break, launch the DBLHelper.exe application provided
by the Cisco from all the subscribers in the cluster. Pending no SQLSvc password or corruption
issues, the application should establish SQL replication.
Use the reference links provided here if you choose to establish SQL replication manually.
These links will guide you through a step-by-step process for checking and re-creating a broken
Cisco CallManager cluster SQL subscription For Cisco CallManager versions 3.0, 3.1, and 3.2,
use the following link:
http://www.cisco.com/warp/public/788/AVVID/recreate_subscribe_database.html.
For Cisco CallManager versions 3.3 and 4.0(1), use this link:
http://www.cisco.com/en/US/partner/products/sw/voicesw/ps556/products_tech_note09186a00
801d11a6.shtml.
The following link will guide you through a step-by-step process for checking and re-creating a
broken Cisco CallManager cluster SQL subscription using DBLHelper:
http://www.cisco.com/en/US/partner/products/sw/voicesw/ps556/products_tech_note09186a00
801e7ddf.shtml.
Copyright © 2004, Cisco Systems, Inc.
Cisco AVVID Troubleshooting Tools
3-35
Microsoft SQL Query Analyzer
This topic explains how to use the Microsoft SQL Query Analyzer to list all of the devices
associated with a given parameter.
Microsoft SQL Server Query Analyzer
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—3-11
The Microsoft SQL Query Analyzer (included with the Microsoft SQL Server 2000) runs
standard SQL queries. The results include listings of rows in the database by various matching
criteria, which is always the case with any SQL query. For further information, you may
consult any standard SQL text.
In the example shown here, a user has requested information on the devices (found in the
“Device” table in the columns “name” and “description”) assigned to the “test” Location. The
“test” Location is specified by linking the “Location” table to the “Device” table and then
selecting only those rows where the column “name” matches the string “test” (where
Location.name = “test”). The returned columns indicate that only one telephone,
SEP0006D79C246F, is in Location “test.”
Note
3-36
For more information on using the Microsoft SQL Query Analyzer, go to
http://www.microsoft.com/sql.
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Microsoft SQL Command-Line Utilities
This topic describes the various Microsoft SQL command-line utilities for troubleshooting
Cisco CallManager.
Command-Line Example
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—3-12
Cisco CallManager includes many command-line utilities that you can use to perform queries
and diagnostics, and to troubleshoot the Microsoft SQL Server 2000 database. The sample
output shown here is from the show db tables command. You can view this information
graphically through the Microsoft SQL Enterprise Manager, or you can use the command-line
access to view this information if the Microsoft SQL Enterprise Manager is not functioning
properly.
show Command Options
Command
Description
J JMPIREQI"
Returns the name of the file where the report is printed.
G GSP[MHXL"
Sets the width of each column in the database report (default 15).
[ GSR[MHXL"
Sets the width of the database report area (default 80).
Z
Enables the verbose mode.
#
Displays the help message.
(F
Returns the configuration database.
HFXEFPIW
Returns the database table names.
HFX XEFPIREQI"
Returns the contents of database table.
;MR
Returns the Microsoft Windows diagnostics report.
Copyright © 2004, Cisco Systems, Inc.
Cisco AVVID Troubleshooting Tools
3-37
Microsoft Command-Line Utilities
• nslookup <hostname>
• netstat - a
• ping <hostname>
• net start
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—3-13
Additional Microsoft command-line utilities used when troubleshooting are as follows:
nslookup <hostname>: This command checks for the host name for IP address resolution.
netstat – a: This command displays all connections and open port numbers.
ping <hostname>: This command verifies that the device is reachable via IP.
net start: This command displays all of the services that are currently running under
Microsoft Windows 2000.
You can use these command-line utilities to check network connectivity and to determine if
services are functioning on Cisco CallManager.
3-38
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Cisco AVVID Directory Service Utilities
This topic describes the functions of the Cisco AVVID directory service utilities.
Cisco AVVID Directory Service Utilities
Cisco CallManager versions 3.1 and 3.2
• avvid_cfg <publisher server name> <Cisco CallManager DB name>
• avvid_scfg <publisher server name> <subscriber server name>
• avvid_del
• avvid_imp
• avvid_recfg <primary SQL Server name> <Cisco CallManager DB
name>
• avvid_save
• avvid_restore
• cleandsa
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—3-14
Commands can be used to resolve DC Directory issues. You can delete broken DC Directory
replication and also backup the DC Directory to a text file. When experiencing DC Directory
issues, be very careful in using these commands because there is a risk of deleting user
information. Always consult the Cisco TAC before attempting these commands.
Keep in mind that various DC Directory commands are different per Cisco CallManager major
releases.
Before making any changes—such as adding, deleting, or editing the users in the directory—
make sure that DC Directory services is “started” on all servers in the cluster. Each server must
have the same DC Directory password for DC Directory to synchronize. This password is
established at installation or during an upgrade. If you believe that you have a DC Directory
issue and that the problem is not obvious, do not tackle the issue without opening a Cisco TAC
service request. Further troubleshooting using the commands listed below can cause a DC
Directory database corruption, resulting in the need to completely rebuild the DC Directory;
this would mean that you lose all user data.
If you cannot add a new user or view the global directory in Cisco CallManager, the DC
Directory may be the cause of the problem. The DC Directory service may have stopped, or the
DC Directory services that are running on the subscriber and the publisher may not be
synchronized. Users cannot access the databases if the subscriber and publisher are not
synchronized. In this case, you must clear the connections between the servers and reconnect
them. After you restore the server connections, replication can take place, and the databases can
resynchronize.
Copyright © 2004, Cisco Systems, Inc.
Cisco AVVID Troubleshooting Tools
3-39
You can run the commands shown from the command prompt of the Cisco CallManager server
after changing the directory to C:\dcdsrvr\bin.
Use the CCMPWDChanger application on all servers to set the DC Directory Manager
password. The password has to be consistent throughout the cluster. If this does not fix the
issues, call the Cisco TAC.
Tip
In the event that you have to open a Cisco TAC service request , be sure to zip up the
C:\dcdsrvr\logs and post the logs to the Cisco TAC case. The Cisco TAC needs these logs
to determine the issue. In nearly every DC Directory TAC case that is opened, the Cisco
TAC engineer will ask that you send the dcdsrvr logs.
The following commands allow you to restore a DC Directory database MetaLink connection
between server nodes:
avvid_cfg <publisher server name> <Cisco CallManager DB name>
—
Invoked by setup on the primary (glass house) server
—
Initializes and configures the DC Directory
—
Configures the MetaLink Open DataBase Connectivity (ODBC) import agreements
—
Allows you to substitute the primary server name with the IP address of the server
itself
avvid_scfg <publisher server name> <subscriber server name>
—
Invoked by the setup on the secondary server
—
Initializes and configures the DC Directory
—
Configures the DC Directory replication agreements
—
Provides the option of substituting IP addresses for names, where applicable
avvid_del
—
Invoked on the primary server
—
Deletes MetaLink ODBC import agreements
—
Deletes imported information in the DC Directory
avvid_imp
—
Invoked on the primary server
—
Creates MetaLink ODBC import agreements
avvid_recfg <publisher SQL Server name> <Cisco CallManager DB name>
—
Invoked on the primary server
—
Deletes MetaLink ODBC import agreements
—
Deletes imported information stored in the DC Directory
—
Reconfigures MetaLink ODBC import agreements
avvid_save
3-40
—
Invoked on the primary server
—
Saves the current user and user profile information in text files
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
avvid_restore
—
Invoked on the primary server
—
Restores the user and user profile information, and saves the information using the
avvid_save command
cleandsa
—
Caution
Clears out the existing Directory Information Database (DIB) by deleting it and then
replacing it with a clean DIB
You must be at the console of the actual server to execute these commands. Do not
execute these commands from a Terminal Services window because doing so may result in
data corruption.
Copyright © 2004, Cisco Systems, Inc.
Cisco AVVID Troubleshooting Tools
3-41
Cisco AVVID Directory Service Utilities
(Cont.)
Cisco CallManager versions 3.3 and 4.0
• reconfig_cluster <DC Directory Manager password>
• clean_publisher
• avvid_srepl <Primary Server name> <primary DC
Directory password>
• avvid_migrate_save <Primary Server name > <DC
Directory password>
• avvid_migrate_restore <Primary Server name> <DC
Directory password>
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—3-15
reconfig_cluster <DC Directory Manager password>
—
Reconfigures all replication agreements
—
Does not delete or modify existing directory data
—
Runs on the publisher
clean_publisher
—
Removes the replication agreements to all nonexistent subscribers from the
publisher DC Directory
—
Does not delete or modify existing directory data
—
Runs on the publisher and after removing any server from the cluster
vvid_srepl <Primary Server Name> <primary DC Directory password>
—
Configures a replication agreement between the subscriber and publisher DC
Directory
—
Runs on the subscribers
avvid_migrate_save <Primary Server Name> <DC Directory password>
3-42
—
Saves the current user and user profile information in text files
—
Saves directory data in Lightweight Directory Access Protocol (LDAF) Directory
Interchange Format (LDIF) format to and from
C:\dcdsrvr\run\dcx500\config\Migration-Backup
—
Runs on the publisher
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
avvid_migrate_restore <Primary Server Name> <DC Directory password>
—
Invoked only on the primary server
—
Restores directory data in LDIF format to and from
C:\dcdsrvr\run\dcx500\config\Migration-Backup
—
Runs on the publisher
Copyright © 2004, Cisco Systems, Inc.
Cisco AVVID Troubleshooting Tools
3-43
Cisco AVVID Directory Service Utilities
(Cont.)
DC Directory Tools in Cisco CallManager 3.3
and 4.0
• CCMRegBrowser: Checks DC Directory version,
install status, password
• CCMPWDChanger: Updates the password of CCM
Users such as Directory Manager, CCMSysUser
• Both tools located C:\DCDSrvr\Bin
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—3-16
Here are the DC Directory tools in Cisco CallManager 3.3 and 4.0:
CCMRegBrowser:
—
Allows you to view the DC Directory version
—
Provides a status of install (true or false)
—
The various configuration/status parameters are under
HKEY_LOCAL_MACHINE\SOFTWARE\Cisco Systems, Inc.\Directory
Configuration
—
Allows you to view configuration/status parameters for all servers within a cluster
CCMPWDChanger:
—
Updates the password of special CCM Users such as DC Directory Manager,
CCMSysUser
—
Works with all supported directory servers (Active Directory, Netscape, DC
Directory)
—
Updates both the directory and the registry on all Cisco CallManager servers
—
Must reboot all of the Cisco CallManager servers, because the password change will
be reflected in all the servers in the Cisco CallManager cluster
—
DC Directory, CCMAdministrator, CCMSysUser accounts changeable with this tool
(Use the pull-down menu to select CCMAdministrator and CCMSysUser
passwords).
Note
To use the CCMPWDChanger tool, DC Directory service has to be started. This applies to
changing CCMAdministrator and CCMSysUser passwords.
To access these tools go to the C:\ drive and look for the folder DCDSrvr. From the DCDSrvr
folder find and access the Bin folder. In the Bin folder, you will find these two tools.
3-44
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Restoring DC Directory Services
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—3-17
You must verify that the services of the DC Directory started properly. To view these services
choose Start > Programs > Administrative Tools > Services. After the services start
properly, use the ping command to verify the connectivity between the publisher and
subscriber servers.
Note
Cisco recommends that you schedule downtime to run the Cisco AVVID command-line
utilities; however, you may not have this option if the problems are severe and require
immediate attention.
Before running the avvid_cfg command, you should confirm which database your publisher is
using. To do this, choose Start > Programs > SQL Server > Enterprise Manager. Expand
the tree until you reach the publisher, and then open the appropriate Cisco CallManager
database folder. The last number in the CCM0300, CCM0301 series represents the Cisco
CallManager database.
Copyright © 2004, Cisco Systems, Inc.
Cisco AVVID Troubleshooting Tools
3-45
Summary
This topic summarizes the key points discussed in this lesson.
Summary
• Microsoft SQL Server 2000 integrates with Cisco Call
Manager components and writes information to the
publisher database.
• Use the Microsoft SQL Enterprise Manager to identify
the publisher and subscriber SQL servers.
• Microsoft SQL Query Analyzer is a tool that allows you
to search database tables for specific data.
• Microsoft SQL command-line utilities allow you to
initiate queries, perform diagnostics, and troubleshoot
the Microsoft SQL Server 2000 database.
• Cisco AVVID directory service command-line utlities
allow you to restore a DC Directory database MetaLink
connection between server nodes.
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—3-18
References
For additional information, refer to these resources:
Microsoft SQL Server 2000 information:
http://www.microsoft.com/sql.
Cisco AVVID tools: Use this link to access DC Directory information related to your
specific version of Cisco CallManager. Refer to the troubleshooting guides for DC
Directory issues or to do a search on Cisco.com.
http://www.cisco.com/univercd/cc/td/doc/product/voice/c_callmg/.
Removing a subscriber server from a cluster:
http://www.cisco.com/univercd/cc/td/doc/product/voice/c_callmg/4_0/sys_ad/4_0_1/ccmcf
g/index.htm.
3-46
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Quiz
Use the practice items here to review what you learned in this lesson. The correct answers are
found in the Quiz Answer Key.
Q1)
Which database contains the writable copy of the database?
A)
B)
C)
D)
Q2)
Which Microsoft SQL Server 2000 database component does the publisher create?
A)
B)
C)
D)
Q3)
Microsoft SQL Query Analyzer
DOS-enabled show commands
ping
nslookup
Which command-line utility would you use to determine open port numbers on your
Cisco CallManager server?
A)
B)
C)
D)
Q5)
transaction logs
CDRs
.dll files
ActiveX controls
Which two utilities allow you to view tables inside the Microsoft SQL Server 2000
database? (Choose two.)
A)
B)
C)
D)
Q4)
publisher
subscriber
primary
secondary
nslookup
ping
net start
netstat
Which Cisco AVVID command-line utility would you use to initialize and configure
the DC Directory on the publisher server using CallManager version 3.3?
A)
B)
C)
D)
avvid_cfg
avvid_scfg
avvid_restore
avvid_imp
Copyright © 2004, Cisco Systems, Inc.
Cisco AVVID Troubleshooting Tools
3-47
Quiz Answer Key
3-48
Q1)
A
Q2)
A
Q3)
A, B
Q4)
B
Q5)
A
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Using Other
Tools
Overview
You need to become familiar with the various external tools at your disposal when
troubleshooting IP telephony issues around Cisco CallManager. In this lesson, you will be
exposed to some of these tools, such as Q.931 Translator and its two versions—DBLHelper, an
SQL replication tool, and Dialed Number Analyzer (DNA) in Cisco CallManager 3.3.4 and 4.0.
This lesson also discusses the use of a protocol analyzer and when to use one.
Relevance
Quick resolution of IP telephony issues is essential to avoid extended downtime. The topics in
this lesson are very important to your success in troubleshooting IP telephony issues.
Objectives
Upon completing this lesson, you will be able to:
Describe Q.931 Translator functions
Describe the Enhanced/H.225 Translator functions
Describe DBLHelper functions
List the steps required in performing protocol analyzer functions in a VoIP environment
Describe the functions and usage of the Cisco DNA tool
Learner Skills and Knowledge
To benefit fully from this lesson, you must have these prerequisite skills and knowledge:
Understanding of Public Switched Telephone Network (PSTN) ISDN messaging
Understanding of how a protocol analyzer works and is attached to the network
Outline
The outline lists the topics included in this lesson.
Outline
• Overview
• Cisco CallManager Q.931 Translator
• Q.931/H.225 Translator X
• Cisco CallManager DBLHelper
• Network (Protocol) Analyzers
• Cisco Dialed Number Analyzer
• Summary
• Quiz
© 2004 Cisco Systems, Inc. All rights reserved.
3-50
Cisco IP Telephony Troubleshooting (IPTT) v4.0
IPTT v4.0—3-3
Copyright © 2004, Cisco Systems, Inc.
Cisco CallManager Q.931 Translator
This topic discusses the specifics related to the CallManager Q.931 Translator and how to use
it.
Q.931 Translator
• Cisco CallManager generates ISDN trace files,
which can be used to diagnose and troubleshoot
connectivity problems in Cisco CallManager
installations.
• The log files contain Q.931 type messages
(ISDN Layer 3 protocol).
• There are two types of Q.931 translators:
– Cisco CallManager and TranslatorX Tool or
Triple Combo
• Use the Q.931 Translator in CallManager to initially
diagnose ISDN PSTN and Cisco gateway issues.
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—3-4
The Q.931 Translator is an application that takes Cisco CallManager trace files and decodes the
hex format Q.931 messages into human readable format. The Q.931 Translator is bundled with
every Cisco CallManager installation. Depending on the version of Cisco CallManager that you
are using, there are two ways to access the application. This application can be found in the
C:\Program Files\Cisco\Bin directory. The application is called Q931 translator.exe. You can
also access this translator through Cisco CallManager Serviceability>Trace>Q931 Translator.
There are two versions of the Q.931 Translator. The first version is the one bundled with Cisco
CallManager. The second is a standalone application named Enhanced Q.931/H.225 Translator,
which is provided by the Cisco TAC.
Copyright © 2004, Cisco Systems, Inc.
Cisco AVVID Troubleshooting Tools
3-51
Q.931 Translator (Cont.)
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—3-5
When accessing the Cisco CallManager traces using the Q.931 Translator in Cisco
CallManager, you will need to select the traces that contain ISDN messages.
Follow these steps in accessing the Cisco CallManager Q.931 Translator:
Step 1
From the Cisco CallManager Administration window, choose
Application > Cisco CallManager Serviceability.
The Cisco CallManager Serviceability window displays.
Step 2
Choose Trace > Q931 Translator.
The Q931 Translator window displays.
Step 3
Choose the Cisco CallManager server for which you want to translate Q.931
messages.
Step 4
To choose an extensible markup language (XML) trace file format, click the XML
button or, to choose a Text trace file format, click the Text button.
Step 5
If you want to search for a particular trace file, enter the filename in the Search For
field.
Step 6
To begin the search, click the List Files button.
The trace files with the selected criteria display. The window displays a list of all files for the
server and the format you chose. The filename, size, and last modified date of each file
displays.
Step 7
3-52
Double-click the filename for which you want the Q.931 message translation. The
Q931 Translator window displays.
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
If the trace file that you chose does not have any ISDN messages in it, the error message, No
ISDN Messages in the File, displays.
If the trace file that you chose does have ISDN messages in it, the Q931 Translator window
contains the following fields: ISDN Message Text and IOS Translation. The ISDN Message
Text field displays all of the ISDN messages in the trace file. The IOS Translation field
displays the translated message of the selected ISDN message from the ISDN Message Text list
box.
Choose the ISDN message to be translated.
Step 8
The Cisco IOS translation text changes based on the chosen ISDN message.
To save the Cisco IOS translated messages for the ISDN messages, click the IOS
Format link.
Step 9
A file save dialog window displays; save the file.
Step 10
To return to the Q931 file search window, click the Back to List Trace Files link.
Q.931 Translator (Cont.)
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—3-6
This figure shows a sample of what the Q.931 Translator looks like from the Cisco
CallManager.
Copyright © 2004, Cisco Systems, Inc.
Cisco AVVID Troubleshooting Tools
3-53
Q.931/H.225 Translator X
This topic discusses the specifics related to the Cisco TAC Enhanced Q.931/H.225 Translator.
This enhanced version of the Q.931 translator can only be obtained from the Cisco TAC.
Q.931/H.225 Translator X
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—3-7
This figure shows a session initiation protocol (SIP) call that is in the Ringing state (180
Ringing). This message was sent from the Cisco CallManager server on the far end
(172.16.3.1) to the Cisco CallManager server 172.16.1.1). The Cisco CallManager server that
initiated the INVITE message was 172.16.1.1. As you can see, this tool can be very handy in
troubleshooting signals. If you could click on the second line from the top the application, it
would take you to the source of the line of the trace.
You can select which messages you want to view. If the gateway was a Media Gateway Control
Protocol (MGCP) gateway, the options to view MGCP messages would appear just as the SIP
messages and other check boxes.
Remember that the Q.931 Translator and the Q.931/H.225 Translator X are not replacements
for Cisco CallManager traces. These tools use the Cisco CallManager traces.
The Q.931/H.225 Translator X offers the following advantages over the standard Q.931
Translator:
Direction column: The method by which the standard Q.931 Translator decodes the
direction is flawed and therefore the direction column is sometimes incorrect. The
Enhanced Q.931 Translator properly decides the messages as “In” or “Out.”
Protocol column: This column tells you whether the message is a Q.931/H.225, Skinny
Client Control Protocol (SCCP), SIP or MGCP message.
Expanding IE decoding: Decodes more Q.931 information elements than the original
Q.931 Translator, including bearer capability, channel ID, numbering plan, and type of
calling and called number information elements.
3-54
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Find in message: This allows you to search for any text that appears in the decoded
message data. This means that you can search on calling or called part numbers, disconnect
codes, or any other value that you are looking for.
Filter messages: You can filter messages based on a specific call reference or protocol
type.
Copyright © 2004, Cisco Systems, Inc.
Cisco AVVID Troubleshooting Tools
3-55
Triple Combo Parser
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—3-8
This Triple Combo Parser Translator tool is an enhancement to the Q.931 Translator hosted on
Cisco CallManager and to the standalone Enhanced Q.931/H.225 Translator. This tool adds
SCCP translation and also combing Q.931/H.225 and MGCP messaging.
This tool is a handy tool for troubleshooting Q.931/H.225, MGCP and SCCP together or
separately. The Cisco TAC does not support this tool, although senior Cisco TAC engineers
wrote the application. You will need to open a Cisco TAC case to obtain this tool.
In this figure, a call is being made from DN 1002, Jane Doe, to a remote cluster DN, 4001, over
the PRI. You can also see the next signaling steps in the call, CALL_PROC and ALERTING
indicating that the IP Phone is ringing on the far end.
3-56
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Cisco CallManager DBLHelper
This topic defines Cisco DBLHelper and explains how it is used to reestablish SQL replication.
DBLHelper
• DBLHelper is a tool for reestablishing replication
between a publisher and subscriber.
• DBLHelper will republish or reinitialize replication.
• The NameResolution tab indicates whether there is
a Domain Name System entry or a LMHOST file
entry to match IP address with server names.
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—3-9
Replicating the SQL database is a core function of Cisco CallManager clusters. The server with
the master copy of the Cisco CallManager database is called the publisher, while the servers
replicating the database are called subscribers. The subscriber server consistently polls the
publisher server for any new changes to the publisher database. If any new changes have been
made, the subscriber performs a pull subscription to receive the most recent changes to the
database.
In the event that the subscriber stops replicating data from the publisher, you will need to
rebuild the relationships between the publisher and subscriber. The DBLHelper utility
republishes or reinitializes a broken subscription between the publisher and subscriber
databases.
For DBLHelper to work, the SQL sa account password needs to be the same on both the
publisher and subscriber, if running on Microsoft SQL Server 7.0. The same applies to
administrative rights if using Microsoft SQL Server 2000.
Copyright © 2004, Cisco Systems, Inc.
Cisco AVVID Troubleshooting Tools
3-57
DBLHelper
Use the “Help” button
to get a definition of
each column
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—3-10
In this figure, DBLHelper was run and found that replication was OK. Two green smiley faces
is what you want to see when launching this application. Specifically, look for Read/Write
listed as Yes/Yes, Protocol listed as tcp, and an SQL Version listed for Publisher and
Subscriber. You also want to see DBConnection indicated in a Good status. In this figure, the
user elected to republish, which deletes the subscription and recreates it. After the subscription
recreate, the reinitialize button needs to be selected to start the snapshot agent and attempt to
rebuild the subscription with the current database.
If you notice issues with Cisco CallManager failover or notice SQL replication errors in the
application event log, check for the DBLHelper.exe file first. This file is located in the
C:\program files\Cisco\Bin directory. If the file is not currently on the system, open a case with
the Cisco TAC detailing a broken SQL subscription or obtain the Cisco TAC Tech Tips on how
to reestablish broken replication.
For reestablishing broken replication on Cisco CallManager 3.0, 3.1, 3.2, go to:
http://www.cisco.com/en/US/partner/products/sw/voicesw/ps556/products_tech_note09186a00
80094aeb.shtml.
For reestablishing broken replication on Cisco CallManager 3.3, go to:
http://www.cisco.com/en/US/partner/products/sw/voicesw/ps556/products_tech_note09186a00
801d11a6.shtml.
Do a search on Cisco.com for further information on how to use DBHelper. There are various
versions that work with various Cisco CallManager releases.
3-58
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
DBLHelper (Cont.)
IP Addresses are
private address
from the Cisco lab
1433 = The during of time one server
took to ping the other server’s SQL
service.
80 = The duration of time
one server took to ping
the other server’s IIS
service.
© 2004 Cisco Systems, Inc. All rights reserved.
Copyright © 2004, Cisco Systems, Inc.
Use the “Help”
button to get a
definition of each
column
IPTT v4.0—3-11
Cisco AVVID Troubleshooting Tools
3-59
Network (Protocol) Analyzers
This topic discusses the specifics related to using a protocol analyzer to troubleshoot IP
telephony issues.
Protocol Analyzers
• Powerful tools for troubleshooting Callmanager
problems
• Let you see exactly what is happening on the wire
• Examining sniffer traces requires understanding
of OSI model
• Protocol analyzers used by Cisco TAC are
Ethereal, Sniffer Pro, Finisar Surveyor
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—3-12
Typically, you collect sniffer traces by connecting a laptop or other sniffer-equipped device on
a Catalyst port that is configured to span the VLAN or port or ports that contains the trouble
information. If no free port is available, connect the sniffer-equipped device on a hub that is
inserted between the switch and the device.
The Cisco TAC uses a few protocol analyzers such Ethereal, Network Associates Sniffer Pro
with Sniffer Voice add-on. Ethereal is freeware and will analyze Skinny packets as will Finisar
Surveyor.
Using a sniffer, you can see Cisco CallManager directory authentication occurring over the
wire and also see IP Phone activity. When XML applications are being utilized and issues arise,
The Cisco TAC will require sniffer traces to assist in troubleshooting.
Sniffer traces can be used to detect poor voice quality. A sniffer filter can be used to capture
Real-Time Transport Protocol (RTP) packets and its sequence numbers. Any large RTP stream
sequence numbers can indicate choppy voice. When troubleshooting a voice quality issue, use a
sniffer and trace the RTP stream packets in between every hop to see where the possible loss in
packets occur. A sniffer, if set up correctly, shows the source IP address, destination IP address,
the type of packet, the payload or codec information, sequence number of packets, and the titter
time.
3-60
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Cisco Dialed Number Analyzer Tool
This topic discusses the specifics related to the Cisco Dialed Number Analyzer Tool.
Cisco Dialed Number Analyzer
CCM 4.0(1):
As a downloadable client onto the server, DNA version 2.0(1)
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—3-13
Cisco CallManager architecture provides total flexibility in configuring dial plans by means of
translation patterns, route patterns, route lists, route groups and calling and called
transformations at different levels. This flexibility makes configuring a dial plan a very
complex process for system administration.
In such cases, the DNA tool can be used to diagnose the dial plans in deployed systems, for
doing predeployment dial plan tuning, for the tracing path for a given set of dialed digits, and
for identifying dial plan issues.
DNA provides end-to-end details of the call for dialed digits including the transformation
details of translation pattern, route pattern, and route list level transformations. DNA also
considers calling and called party transformations at the originating or terminating device if
configured on Cisco CallManager.
You can install DNA as a plug-in to Cisco CallManager. The tool allows you to test a Cisco
CallManager dial plan configuration before deploying it. You can also use the tool to analyze
dial plans after the dial plan is deployed.
DNA is a tool that can be used to diagnose dial plans deployed in Cisco CallManager, as
follows:
—
Tuning predeployment dial plans
—
Tracing the path for given dialed digits
—
Identifying any problems
DNA provides end-to-end details of the call for given dialed digits.
Copyright © 2004, Cisco Systems, Inc.
Cisco AVVID Troubleshooting Tools
3-61
This tool considers different types of calls, such as the following:
IP Phone to IP Phone
Gateway to IP Phone
IP Phone to gateway
Gateway-to-Gateway
Calls that feature specific patterns
To access DNA, follow these steps:
Step 1
Choose Start > Programs > Cisco Dialed Number Analyzer > Cisco Dialed
Number Analyzer.
The Enter Network Password dialog displays.
Step 2
In the Username field, enter a valid user ID.
Use the username that you use to access Cisco CallManager Administration.
Step 3
In the Password field, enter a valid password for the user ID.
Use the password that you use to access Cisco CallManager Administration.
Step 4
Click OK.
You are now logged into DNA.
The user interface has these three main features:
To search and select phones, gateways, or trunks to make calls
An interface to view dialing forest and digit discard instructions
Analysis output shown in separate browser window in tree format. (This window has these
three sections: Result Summary, Call Flow, and Alternate Matches.)
DNA allows you to save analysis results in an XML file, which can be opened with the View
Files screen.
DNA is installed as a Microsoft Windows NT service that can be started or stopped from the
Service Control page on DNA.
DNA allows the user to select phone devices, gateways, or trunks as the calling party to make
an analysis for dialed digits.
The calling search space of the selected device or line will be used by DNA for match finding.
On Cisco CallManager 3.3(4), DNA is installed as a plug-in on the server.
3-62
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
DNA Service Activation
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—3-14
Cisco CallManager Serviceability provides a web-based Service Activation tool that is used to
activate and deactivate Cisco CallManager services for servers. This section describes the
procedure to start the DNA service from Cisco CallManager Serviceability. You must have
installed DNA on the Cisco CallManager server.
Here is the process for starting the service:
Step 1
Access Cisco CallManager Administration on the server where you have installed
DNA.
Step 2
Choose Application > CiscoCallManager Serviceability.
The Cisco CallManager Serviceability window displays.
Step 3
Choose Tools > Control Center.
The Control Center window displays the list of configured Cisco CallManager servers in the
left pane.
Step 4
Choose the server where you have installed DNA.
The window displays the service names for the server that you choose, the current activation
status of the services, and the Cisco Tomcat web service information.
Step 5
Check the box next to the DNA service.
Step 6
Click Update. The window displays the services that you chose with an activation
status of “Activated.”
Copyright © 2004, Cisco Systems, Inc.
Cisco AVVID Troubleshooting Tools
3-63
3-64
Caution
Activate/deactivate services only from the Service Activation pages. If you
activate/deactivate services from the Service Control Manager window instead of from
Service Activation, entries do not get added to/removed from the database table; therefore,
services do not get properly configured or started and may be out of sync with the Cisco
CallManager database.
Note
Cisco CallManager services will not start until you activate them by using Service Activation.
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
DNA Control Center
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—3-15
Once you start the DNA service in Cisco CallManager, the services presented in figure should
also be in the “Enabled” state. If the services are not in the “Enabled” state, restart the DNA
service in Cisco CallManager, close the Cisco Dialed Number Analyzer Control Center web
page, and reopen the DNA web page.
Copyright © 2004, Cisco Systems, Inc.
Cisco AVVID Troubleshooting Tools
3-65
Phone Line Selection
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—3-16
The results of the analysis you perform contain information in the dialed digits call flow. Here
are the three areas of this call flow:
1. Results Summary
2. Call Flow
3. Alternate Matches
This figure displays information related to call routing for a selected device. Note the partition
and calling search space information for lines. This device has two lines. A user must select one
of the lines to make a call and enter dialed digits to go to the Dialed Number Analyzer Results
page.
3-66
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Output: Result Summary
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—3-17
This sample output screen shows the Results Summary expanded.
By default, Results Summary will be expanded and the other two nodes collapsed. If interested,
the user can expand the individual nodes by clicking on the plus sign (+) sign or can expand all
nodes by clicking the Expand All button.
Copyright © 2004, Cisco Systems, Inc.
Cisco AVVID Troubleshooting Tools
3-67
Output: Call Flow
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—3-18
In this example, for dialed digits 9-1-409-209-7872, this route pattern matched the route pattern
“9.@.”
Note the calling and called party transformations at the 9.@ route pattern. The pretransform
calling party number is 1350000, which after transformations has become 9728130000.
Similarly, for the caller party number, the pretransform called party number is 9-1-409-2097872 and, after transformations, it is 1-409-209-7872 with 9 stripped off because of the PreDot
digit discard instruction.
Finally, the call has landed on route list DNARL1 with two route groups in it. DNA shows
calling and called party transformations at route group level again. In each route group,
configured gateways are shown. Device nodes can be opened to see transformation details, if
any.
In the case of a match to a translation pattern, the translation pattern details will be shown first
in order followed by the final match details.
3-68
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Output: Alternate Matches
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—3-19
In this particular example, there was an overlapping pattern to the 9.@ route pattern configured
as 9.XXXXXXXXXX.
Output can be saved as an XML file.
Copyright © 2004, Cisco Systems, Inc.
Cisco AVVID Troubleshooting Tools
3-69
Hunt List / Line Group Example
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—3-20
From Cisco CallManager 4.0 onward, typically the Hunt List group would be used to configure
voice-mail pilot and ports. Here is an example with the voice-mail pilot as 9043 and 10 voice
mail ports 90431 to 90440. The Hunt List is configured as 9043, which is assigned to line group
VMLG with 10 ports.
For dialed digits 9043, the output is as shown in the figure. Note that the device type is shown
as a Cisco voice-mail port, and it does not have any forwarding information.
3-70
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Phantom DN Example
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—3-21
This figure shows an example of a phantom DN. For dialed digits 1350002, a match is found
because the DN exists in the database but there is no device associated with it.
Copyright © 2004, Cisco Systems, Inc.
Cisco AVVID Troubleshooting Tools
3-71
Summary
This topic summarizes the key points discussed in this lesson.
Summary
• Use the Q.931 translators available to troubleshoot ISDN
gateway and PSTN issues.
• Q.931 can be used to diagnose issues with
intercluster trunks.
• DBLHelper can be used to reestablish replication by
republishing and reinitializing SQL replication.
• Your protocol analyzer should include the ability to sniff
Skinny, MGCP, H.323, H.225, H.245, RTP, SQL, and LDAP
protocols.
• Dialed Number Analyzer can be installed as a plug-in to
Cisco CallManager where you can test and troubleshoot
dial plan configurations before and after a dial plan is
deployed.
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—3-22
References
For additional information, refer to these resources:
Reestablishing a Broken Cisco CallManager Cluster SQL Subscription with Cisco CallManager
3.0, 3.1, and 3.2:
http://www.cisco.com/en/US/partner/products/sw/voicesw/ps556/products_tech_note09186
a0080094aeb.shtml.
Reestablishing a Broken Cisco CallManager Cluster SQL Subscription with Cisco CallManager
3.3:
http://www.cisco.com/en/US/partner/products/sw/voicesw/ps556/products_tech_note09186
a00801d11a6.shtml.
Dialed Number Analyzer tool for Cisco CallManager 4.0 and Cisco CallManager 3.3:
http://www.cisco.com/univercd/cc/td/doc/product/voice/c_callmg/4_0/cdna/index.htm; and
http://www.cisco.com/univercd/cc/td/doc/product/voice/c_callmg/3_3/cdna/cdna334/index.
htm.
3-72
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Quiz
Use the practice items here to review what you learned in this lesson. The correct answers are
found in the Quiz Answer Key.
Q1)
What nonembedded tool can be used to translate ISDN messages into a Cisco IOS
format?
A)
B)
C)
D)
Q2)
You notice a device configuration on the publisher that does not show up on the
subscriber. Which tool would you use to see if replication is an issue?
A)
B)
C)
D)
Q3)
selecting the Republish button
selecting the Reinitialize button
restarting CCM.exe
restarting the Microsoft SQL server
What action would be needed to obtain an enhanced version of the Q.931 Translator?
A)
B)
C)
D)
Q6)
Cisco CallManager Trace
protocol analyzer
Q.931 Translator
all of the above
A snapshot agent can be initiated by doing which of the following things?
A)
B)
C)
D)
Q5)
DBMaster
DBUpdate
DBLHelper
DBHelper
A PSTN user is complaining about not seeing the name of the calling party displayed
when being called. Which tool would you use to diagnose this issue?
A)
B)
C)
D)
Q4)
protocol analyzer
Cisco CallManager Trace
Q.931 Translator
Enhanced Q.931 Translator / H.225
open a case with the Cisco TAC
order from Cisco.com
B only
all of the above
Which of the following statements about DNA database synchronization is correct?
A)
B)
C)
D)
DNA replicates and uses the Cisco CallManager database to analyze calls in a
dial plan.
DNA uses the Cisco CallManager database to analyze calls in a dial plan.
DNA does not replicate any database; it runs a script and analyzes the dial
plan.
None of the above are correct.
Copyright © 2004, Cisco Systems, Inc.
Cisco AVVID Troubleshooting Tools
3-73
Lesson Review Answer Key
3-74
Q1)
D
Q2)
C
Q3)
D
Q4)
B
Q5)
A
Q6)
A
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Module 4
Troubleshooting Network
Infrastructure
Overview
Network routers and switches make up the network infrastructure for Voice over IP (VoIP)
network traffic. When properly configured, this network infrastructure provides a seamless,
quality of service (QoS)-enabled transport for IP telephony traffic. If there is a problem in the
network infrastructure, it can affect the entire voice network. This module discusses the
troubleshooting methods that you can use to ensure that your network infrastructure continues
to operate successfully.
Module Objectives
Upon completing this module, you will be able to troubleshoot common switch, router,
gateway, and gatekeeper configuration, integration, and operation issues in Cisco Architecture
for Voice, Video and Integrated Data (AVVID) networks.
Module Objectives
• Troubleshoot common switching and IP routing
issues found within the Cisco AVVID environment
• Troubleshoot common gateway configuration,
integration, and operation issues
• Troubleshoot common Cisco gatekeeper and SIP
trunk configuration, integration, and operation
issues
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—4-2
Module Outline
The outline lists the components of this module.
Module Outline
• Troubleshooting Data Network Infrastructure
• Troubleshooting Gateways
• Troubleshooting Gatekeepers and Cisco SIP
Trunks
© 2004 Cisco Systems, Inc. All rights reserved.
4-2
Cisco IP Telephony Troubleshooting (IPTT) v4.0
IPTT v4.0—4-3
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Data Network
Infrastructure
Overview
A firm understanding of the underlying data network is important when troubleshooting IP
telephony problems. This lesson will not cover the basics of switching and routing but will
instead discuss common IP telephony issues related to switching and routing in a campus
environment. This lesson also discusses multicast in a VoIP network.
Relevance
Routers and switches, along with the communications links covered in other lessons, are the
technologies that provide the infrastructure to transport IP telephony traffic from its origin to its
destination. Any problems associated with the underlying data infrastructure can result in either
poor voice quality or the inability to complete calls.
Objectives
Upon completing this lesson, you will be able to:
List the common voice issues caused by data network infrastructure
Explain the common configuration issues in switches
Explain the routing configuration issues in routers
Evaluate how IP routing configuration can have an impact on device registration and call
setup
Describe how multicast configuration issues can affect MOH, and list the commands used
to troubleshoot MOH
Learner Skills and Knowledge
To benefit fully from this lesson, you must have these prerequisite skills and knowledge:
Fundamental understanding of Layer 2 and Layer 3 troubleshooting
Fundamental understanding of Cisco IOS commands at the Cisco CCNP® level
Prior Cisco VoIP exposure
Outline
The outline lists the topics included in this lesson.
Outline
• Overview
• Impact of Data Network
• Switching Configuration
• Auxiliary VLANs
• Router Configuration
• Multicast Routing
• Summary
• Quiz
© 2004 Cisco Systems, Inc. All rights reserved.
4-4
Cisco IP Telephony Troubleshooting (IPTT) v4.0
IPTT v4.0—4-3
Copyright © 2004, Cisco Systems, Inc.
Impact of Data Network
This topic discusses the specifics related to voice issues caused by the data network.
Impact of Data Network
Applications
(VMail, IVR, ICD, ...)
PSTN
SRST-Enabled
Router
CallManager
Cluster
IP WAN
Branch A
SRST-Enabled
Router
Headquarters
Gatekeeper
Branch B
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—4-4
Root
Symptom
Routing issue
Registration issues, no audio stream
NAT/access control lists (ACLs), and so on
One-way audio, no-way audio, registration issues
Domain Name System (DNS) issues
No access to URLs (Help, Services, and so on); Media
Gateway Control Protocol (MGCP) gateway registration
issues
Duplex/speed mismatch
Dynamic Host Configuration Protocol (DHCP) relay,
registration issues/quality
Cable issues
Catalyst @ 100 Mbps, distance, electromagnetic
interference (EMI); registration/quality/call disconnect
Bridge loops, spanning tree
Would typically affect everything
Power
Overloading switch so there are not redundant power
supplies
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Network Infrastructure
4-5
Switching Configuration
This topic describes how call setup or one-way voice problems can occur when traffic cannot
pass between VLANs or devices are connected to the wrong VLANs.
Switching
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—4-5
Most VoIP issues that arise from a switching environment come from incorrect configuration at
the switch level. Make sure that the configuration on your switch is correct. Configuration on
certain switching products runs Cisco IOS software or Catalyst software or both. Each has its
own way to be configured. The ease of troubleshooting depends on the topology and how it is
laid out.
Cisco IP Phones have a three-port switch and support IEEE 802.1Q trunking. Therefore, you
should use IEEE 802.1Q for the trunking protocol in a Cisco IP telephony network. IEEE
802.1Q trunks impose some limitations on the trunking strategy for a network. The following
restrictions apply when using IEEE 802.1Q trunks:
Both ends of the trunk line must have the same native VLAN for an IEEE 802.1Q trunk. If
the native VLAN on one end of the trunk is different from the native VLAN on the other
end, spanning-tree loops may occur.
Disabling Spanning Tree Protocol (STP) on the native VLAN of an IEEE 802.1Q trunk
without disabling it on every VLAN in the network can potentially cause STP loops. Cisco
recommends that you leave STP enabled on the native VLAN of an IEEE 802.1Q trunk or
disable STP on every VLAN in the network. Verify that your network is loop-free before
disabling STP.
If you have a link and the ports show that they are connected, and you cannot communicate
with another device, this usually indicates a problem above the physical layer (Layer 2 or
Layer 3).
4-6
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Depending on whether the switch uses Cisco IOS software or Catalyst software, on a Catalyst
switch, the show port status command displays a brief summary of all the ports on the switch.
The output of this command will display the status of each port (connected, not connected, or
disabled), the VLAN the port belongs to, the duplex setting (full or half), and the port speed
(10/100/1000).
Some switches, such as the Catalyst 4000, 5000, or 6000, may shut down the port if software
processes inside the switch detect an error. When you look at the port status, it will read
“errDisable”(for error disabled); you must fix the configuration problem and then manually
take the port out of the errDisable state. Some newer software versions—Catalyst software
5.4(1) and later—have the capability to automatically re-enable a port after a configurable
amount of time spent in the errDisable state. Some of the causes for this errDisable state are as
follows:
EtherChannel misconfiguration: If one side is configured for EtherChannel and the other
is not, it can cause the spanning-tree process to shut down the port on the side configured
for EtherChannel. If you try to configure EtherChannel and the ports involved do not have
the same settings (speed, duplex, trunking mode, and other settings) as their neighbor ports
across the link, it will cause the errDisable state.
Bridge protocol data unit (BPDU) port guard: Some newer versions of switch software
can monitor whether PortFast is enabled on a port. You should connect a port using
PortFast to an end station, not to the devices that generate spanning-tree packets or BPDUs.
When a BPDU moves into a port that has PortFast enabled, the switch changes to the
errDisable mode.
UniDirectional Link Detection (UDLD): UDLD is a protocol on some new versions of
Cisco IOS Software that discovers whether communication over a link is one-way only. A
broken fiber cable or other cabling or port issues or both can cause this one-way only
communication. These partially functional links can cause problems when the switches
involved do not detect that the link is partially broken. Spanning-tree loops can occur with
this problem. You can configure UDLD to change a port to the errDisable state when it
detects a unidirectional link.
Native VLAN mismatch: Before a port has trunking turned on, it belongs to a single
VLAN. When trunking is turned on, the port can carry traffic for many VLANs. The port
keeps a record of the VLAN that it was located in before trunking was turned on, which is
called the native VLAN. The native VLAN is central to IEEE 802.1Q trunking. If the
native VLAN on each end of the link does not match, a port will display the errDisable
state.
Duplex mismatches: IP Phones that are connected to a switch that have trunk connections
to another switch that has duplex mismatches can prevent the IP Phones from registering or
the IP Phones simply unregister with Cisco CallManager.
Port security violations: When the VLAN for the ports disappears, it causes inactive
ports. Each port in a switch belongs to a VLAN. Deleting a VLAN causes the port to
become inactive. To verify the configuration of VLANs on the switch, use the show vlan
command. This command will verify the VLANs that were created on the switch and
determine which ports belong to the VLANs.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Network Infrastructure
4-7
To verify that a certain port is configured as a trunk port, use the show trunk command. This
command works on set-based switches and displays the trunking mode, trunking encapsulation
(Inter-Switch Link [ISL] or IEEE 802.1Q), status, and native VLAN information for all trunk
ports on a switch. The show port capabilities command verifies that a port is capable of
trunking and determines the supported trunking encapsulations.
If you are installing a new switch, before creating VLANs on that switch, you must define a
VLAN Trunking Protocol (VTP) server and domain it belongs to. The show vtp domain
command verifies the creation of a VTP domain.
4-8
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Trunking
Catalyst 6500
Series Switch
ISL
Trunk
Catalyst 3550
XL Switch
VLAN 1
ISL
Trunk
Catalyst
3550 XL
Switch
Catalyst
3550 XL
Switch
VLAN 3
VLAN 2
© 2004 Cisco Systems, Inc. All rights reserved.
ISL
Trunk
ISL
Trunk
Catalyst 3550 XL
Switch
VLAN 2
VLAN 1
VLAN 3
IPTT v4.0—4-6
A routing process is required to pass information from one VLAN to another. This process is
known as interVLAN routing. An external router with a 10/100 Fast Ethernet interface that
supports trunking, such as a Cisco 2620 router, or a multilayer switch with an internal router
card, can perform interVLAN routing.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Network Infrastructure
4-9
VLAN Routing
Set-Based Switch
'EXEP]WX" IREFPI WIXXVYRORSRIKSXMEXIHSXU
Cisco IOS Switch
-377[MXGL
-377[MXGL
-377[MXGL
-377[MXGL
-377[MXGL
GSRJMK MRXIVJEGI*EWX)XLIVRIX
GSRJMKMJ W[MXGLTSVXXVYROIRGETWYPEXMSRHSXU
GSRJMKMJ W[MXGLTSVXXVYROREXMZIZPER
GSRJMKMJ W[MXGLTSVXQSHIXVYRO
GSRJMKMJ W[MXGLTSVXXVYROEPPS[IHZPEREPP
Cisco Router
6SYXIV GSRJMK MRXIVJEGI*EWX)XLIVRIX
6SYXIV GSRJMKWYFMJ IRGETWYPEXMSRHSX5REXMZI
6SYXIV GSRJMKWYFMJ MTEHHVIWW
6SYXIV GSRJMKWYFMJ MTLIPTIVEHHVIWW
6SYXIV GSRJMK MRXIVJEGI*EWX)XLIVRIX
6SYXIV GSRJMKWYFMJ IRGETWYPEXMSRHSX5
6SYXIV GSRJMKWYFMJ MTEHHVIWW
6SYXIV GSRJMKWYFMJ MTLIPTIVEHHVIWW
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—4-7
Here are some questions to consider:
1. Can you envision how the switch is communicating to the router and how the router sees
the VLANs and routes them?
2. With respect to the IP Phones, what is not shown in this figure?
3. Can you deduce what the voice VLAN is?
4. Can you deduce what the data VLAN is?
5. Where are the IP Phones getting their IP address and the address to the TFTP server?
6. If Layer 2 and Layer 3 were not configured correctly, what would be the impact?
In the figure shown here, Fast Ethernet interface 0/0 is configured to support IEEE 802.1Q
trunking. Each VLAN is configured on a separate subinterface with its specific VLAN
identifier and the IP address for that subinterface. In this configuration, each subinterface
includes an IP helper address.
The IP helper address is required because each VLAN is a separate broadcast domain, and IP
Phones use a broadcast address of 255.255.255.255 to find a DHCP server. Routers are
designed not to pass broadcast traffic from one broadcast domain (VLAN) to another. In this
example, the DHCP server resides on the 172.17.1.0 network. The IP helper address modifies
this behavior by changing the 255.255.255.255 broadcast into a unicast to the IP address
specified in the ip helper-address command.
For IEEE 802.1Q trunking, one VLAN is not tagged. This VLAN is called the native VLAN.
The native VLAN is used for untagged traffic when the port is in IEEE 802.1Q trunking mode.
While configuring IEEE 802.1Q trunking, you must identically configure the native VLAN on
each side of the trunk link. It is a common mistake not to match the native VLANs while
configuring IEEE 802.1Q trunking between the router and the switch.
4-10
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Verifying VLAN Routing
Set-Based Switch
'EXEP]WX" WLS[XVYRO
MRHMGEXIWZXTHSQEMRQMWQEXGL
4SVX1SHI)RGETWYPEXMSR7XEXYW2EXMZIZPER
SRHSXUXVYROMRK
Cisco IOS Switch
-377[MXGL" WLS[MRXJEWX)XLIVRIXW[MXGLTSVX
2EQI*E
7[MXGLTSVX)REFPIH
%HQMRMWXVEXMZIQSHIXVYRO
3TIVEXMSREP1SHIXVYRO
%HQMRMWXVEXMZI8VYROMRK)RGETWYPEXMSRHSXU
3TIVEXMSREP8VYROMRK)RGETWYPEXMSRHSXU
2IKSXMEXMSRSJ8VYROMRK(MWEFPIH
%GGIWW1SHI:0%2 -REGXMZI
8VYROMRK2EXMZI1SHI:0%2
Use the show vlans and show interface commands on the
Cisco router to verify interVLAN routing configuration.
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—4-8
You can use the following commands on the router to verify interVLAN routing:
GWLS[ZPER
:MVXYEP0%2-( -)))5)RGETWYPEXMSR Z0%28VYRO-RXIVJEGI*EWX)XLIVRIX
8LMWMWGSRJMKYVIHEWREXMZI:PERJSVXLIJSPPS[MRKMRXIVJEGI W *EWX)XLIVRIX
4VSXSGSPW'SRJMKYVIH%HHVIWW6IGIMZIH8VERWQMXXIH
-4
:MVXYEP0%2-( -)))5)RGETWYPEXMSR Z0%28VYRO-RXIVJEGI*EWX)XLIVRIX
4VSXSGSPW'SRJMKYVIH%HHVIWW6IGIMZIH8VERWQMXXIH
-4
GWLS[MRXIVJEGIWJEWX)XLIVRIX
*EWX)XLIVRIXMWYTPMRITVSXSGSPMWYT
,EVH[EVIMW%QH*)EHHVIWWMWIJI FMEIJI -RXIVRIXEHHVIWWMW
189F]XIW&;/FMX(0=YWIGVIPMEFMPMX]
X\PSEHV\PSEH
)RGETWYPEXMSR5:MVXYEP0%2:PER-(
%64X]TI%64%%648MQISYX
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Network Infrastructure
4-11
GWLS[MRXIVJEGIWJEWX)XLIVRIX
*EWX)XLIVRIXMWYTPMRITVSXSGSPMWYT
,EVH[EVIMW%QH*)EHHVIWWMWIJI FMEIJI -RXIVRIXEHHVIWWMW
189F]XIW&;/FMX(0=YWIGVIPMEFMPMX]
X\PSEHV\PSEH
)RGETWYPEXMSR5:MVXYEP0%2:PER-(
%64X]TI%64%%648MQISYX
This command can be used on a Cisco IOS software-based router to verify the status of a trunk
port:
WLS[MRXJEWX)XLIVRIXW[MXGLTSVX
2EQI*E
7[MXGLTSVX)REFPIH
%HQMRMWXVEXMZIQSHIXVYRO
3TIVEXMSREP1SHIXVYRO
%HQMRMWXVEXMZI8VYROMRK)RGETWYPEXMSRHSXU
3TIVEXMSREP8VYROMRK)RGETWYPEXMSRHSXU
2IKSXMEXMSRSJ8VYROMRK(MWEFPIH
%GGIWW1SHI:0%2
-REGXMZI
8VYROMRK2EXMZI1SHI:0%2 HIJEYPX 8VYROMRK:0%2W)REFPIH%00
8VYROMRK:0%2W%GXMZI
4VYRMRK:0%2W)REFPIH
4VMSVMX]JSVYRXEKKIHJVEQIW
3ZIVVMHIZPERXEKTVMSVMX]*%07)
:SMGI:0%2RSRI
4-12
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Auxiliary VLANs
This topic examines both standard and auxiliary VLANs.
Auxiliary VLANs
Native VLAN – Subnetwork 1
Data VLAN – Subnetwork 2
Auxiliary VLAN – Subnetwork 3
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—4-9
When the IP Phone powers up, it communicates with the switch using Cisco Discovery
Protocol (CDP). The switch then provides the telephone with its configured VLAN ID (voice
subnetwork), also known as the voice VLAN ID (VVID). Meanwhile, data devices continue to
reside in the native VLAN (or default VLAN) of the switch. A data device VLAN (data
subnetwork) is referred to as a port VLAN ID (PVID). Best practice dictates that a voice
VLAN, data VLAN, and native VLAN be configured. The native VLAN will handle all those
data packets that do not belong to the data VLAN. The native VLAN will ensure that the
packets are untagged and process the packets accordingly.
Auxiliary VLAN traffic is tagged with an IEEE 802.1Q tag; data VLAN traffic can be tagged
per configuration; the native VLAN traffic is not tagged. Because each VLAN must be in a
separate IP subnetwork, traffic between the auxiliary VLAN, data VLAN, and the native (or
any other) VLANs must be routed through a Layer 3 device.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Network Infrastructure
4-13
Configuring Auxiliary VLANs
Set-Based Switch
'EXEP]WX" IREFPI WIXTSVXEY\MPMEV]ZPER %Y\MPMEV]ZPERGSRJMKYVEXMSRWYGGIWWJYP
Cisco IOS Switch
-377[MXGL
-377[MXGL
-377[MXGL
-377[MXGL
-377[MXGL
-377[MXGL
-377[MXGL
GSRJMK MRXIVJEGI*EWX)XLIVRIX
GSRJMKMJ W[MXGLTSVXXVYROIRGETWYPEXMSRHSXU
GSRJMKMJ W[MXGLTSVXXVYROREXMZIZPER
GSRJMKMJ W[MXGLTSVXQSHIXVYRO
GSRJMKMJ W[MXGLTSVXZSMGIZPER
GSRJMKMJ WTERRMRKXVIITSVXJEWX
GSRJMKMJ W[MXGLTSVXQSHIXVYWX
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—4-10
To configure the VVID from the Catalyst software command-line interface (CLI), use the set
port auxiliaryvlan command. You can use this command to set the VVID on a single port, on
a range of ports, or for an entire module. In the following example, the VVID is set to 222 for
ports 2/1 through 2/3; when the telephone powers up, the switch instructs the IP Phone a voice
VLAN of 222.
'SRWSPI" IREFPI WIXTSVXEY\MPMEV]ZPER
%Y\MPMEV]ZPERGSRJMKYVEXMSRWYGGIWWJYP
These examples show how to display which ports are in which auxiliary VLAN:
'SRWSPI"WLS[TSVXEY\MPMEV]ZPER
%Y\MPMEV]:PEREY\:PER7XEXYW1SH4SVXW
'SRWSPI"WLS[TSVX
4SVX%Y\MPMEV]:PER%Y\:PER7XEXYW
EGXMZI
4-14
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
This is an example of VVID configuration on Catalyst switches running Cisco IOS software at
the interface level (for example, Catalyst 3524-PWR and 2900XL):
MRXIVJEGI*EWX)XLIVRIX
W[MXGLTSVXXVYROIRGETWYPEXMSRHSXU
W[MXGLTSVXXVYROREXMZIZPER 4:-("
W[MXGLTSVXQSHIXVYRO
W[MXGLTSVXZSMGIZPER ::-("
WTERRMRKXVIITSVXJEWX
W[MXGLTSVXQSHIXVYWX
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Network Infrastructure
4-15
Verifying Auxiliary VLANs
• Verify that CDP is enabled on switch ports
connected to IP Phones.
• Use the show port, show vlan, and show cdp
neighbors commands to troubleshoot problems
with auxiliary VLANs.
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—4-11
Because CDP carries the VLAN ID information for auxiliary VLANs, it is the best place to
start when troubleshooting. The show cdp neighbors and show cdp neighbors detail
commands verify CDP connections to directly connected neighbors. If the output of these
commands is empty, verify that CDP is not disabled on the switch. You can globally disable
CDP with the set cdp disable (set-based) or no cdp run (based on Cisco IOS software)
commands or on an interface-by-interface basis with the no cdp enable (Cisco IOS software)
command.
The commands set cdp enable (set-based) and cdp run (Cisco IOS software-based) enable
CDP globally, and the cdp enable (Cisco IOS software) command re-enables it on an interface.
Any other problems are most likely related to trunking, VLAN, or port issues; you can locate
these problems by using the commands discussed earlier in this lesson.
4-16
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Routing Configuration
This topic describes Layer 3 routing problems in a Cisco IP telephony network.
Routing and the Routing Table
Routing Table
Network A
.
.
Network B
.
.
.
.
Net B
.
.
.
Net B
.
.
.
Net B
.
.
.
Router 3
Router 1
Router 2
Router 4
Network A
© 2004 Cisco Systems, Inc. All rights reserved.
Router 5
Network B
IPTT v4.0—4-12
Layer 3 routing problems can cause problems in a Cisco IP telephony network, just as they
would in a data-only network. Layer 3 problems often play a role in IP telephony issues,
including VoIP calls not connecting, intercluster communication failing, and voice quality
issues. This discussion covers several tools for troubleshooting IP routing problems.
The ping command enables you to determine if there is a valid IP path between the source and
the destination device. This command is normally the first step for troubleshooting IP
connectivity problems. For example, you can use the ping command from the Cisco
CallManager server to verify the IP connectivity to devices, including voice gateways, IP
Phones, other Cisco CallManager servers, and other devices.
If the ping command is unsuccessful, you can use the show ip route command on a Cisco
router to view the routing table. By investigating the routing table, you can determine if the
router has the information to reach the destination network.
Determining the voice traffic path is also important. Too many hops, congested links, and lowspeed links can adversely affect voice quality. During the design and implementation phases of
your network, you may have specific paths and specific routers in each path optimized for
voice; for example, by using low latency queuing LLQ). However, because of a primary link
failure or poor design in other parts of the network, you may have problems with suboptimal
routing. The traceroute command can assist in identifying the path that voice packets travel.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Network Infrastructure
4-17
The ping Command
IP ping Test Characters
Character
Description
!
Each exclamation point indicates receipt of an ICMP echo reply
.
Each period indicates a timeout while waiting for an ICMP echo reply
U
A destination unreachable error PDU was received. This normally occurs when a router is
advertising a summary route to the requested destination, but the router(s) advertising the
summary route doesn’t have the complete route to the destination. This is also often returned
when an access list in the path is blocking ICMP traffic.
Q
Source quench (destination too busy)
M
Could not fragment
?
Unknown packet type
&
Packet lifetime exceeded
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—4-13
The ping command is one of the most useful commands available for troubleshooting network
problems. The ping command is an echo test that uses a series of Internet Control Message
Protocol (ICMP) echo messages to determine if it is possible to reach a specific host and to
provide certain network status information. A ping operation enables you to determine if the
destination host is reachable. If it is, the ping operation also provides the approximate roundtrip delay.
The ping command first sends an echo request packet to a destination address and then waits
for an echo reply. The ping is successful only if the echo request can reach the destination from
the source, and the echo reply can reach the source from the destination. The pinged device
must also have the ability to send the echo reply back to the source of the ping.
The ping command returns several different characters to display the status of an ICMP echo
reply. These characters and an explanation of each are listed in the figure shown here. This
output is an example from a ping command:
6SYXIVTMRK
8]TIIWGETIWIUYIRGIXSEFSVX
7IRHMRKF]XI-'14)GLSWXSXMQISYXMW
WIGSRHW
7YGGIWWVEXIMWTIVGIRX IP Phones will not respond to every ping, but they do respond every other ping.
4-18
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
This output indicates that the router does not have a valid route to this destination, the device is
not running IP, the device is incorrectly configured for IP, the device is not on the IP network,
or the device is powered off:
6SYXIVTMRK
8]TIIWGETIWIUYIRGIXSEFSVX
7IRHMRKF]XI-'14)GLSWXSXMQISYXMW
WIGSRHW
999
7YGGIWWVEXIMWTIVGIRX This output indicates that an incorrect summary route exists in the network or an access list is
blocking ICMP traffic from the destination device back to the source, which prevents echo
replies:
6SYXIVTMRK
8]TIIWGETIWIUYIRGIXSEFSVX
7IRHMRKF]XI-'14)GLSWXSXMQISYXMW
WIGSRHW
7YGGIWWVEXIMWTIVGIRX 8LMWMWTMRKFILEZMSVSJ-44LSRIWIZIV]SXLIVTMRKMWWYGGIWWJYP This output indicates that the router is process switching and trying to load balance across two
paths to the destination. This output also indicates that the router can communicate via IP with
the device at IP address 12.0.0.2. In the following example, only one path is valid:
6SYXIVTMRK
8]TIIWGETIWIUYIRGIXSEFSVX
7IRHMRKF]XI-'14)GLSWXSXMQISYXMW
WIGSRHW
7YGGIWWVEXIMWTIVGIRX VSYRHXVMTQMREZKQE\!
QW
Note
It is common for the first character of a successful ping to time out (for example, .!!!!). This
action indicates that the router had to perform an Address Resolution Protocol (ARP)
request to begin communication with the destination device.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Network Infrastructure
4-19
Extended Ping Mode
84%C,5TMRK
4VSXSGSP?MTA
8EVKIX-4EHHVIWW
6ITIEXGSYRX?A
(EXEKVEQWM^I?A
8MQISYXMRWIGSRHW?A
)\XIRHIHGSQQERHW?RA]
7SYVGIEHHVIWWSVMRXIVJEGI
8]TISJWIVZMGI?A
7IX(*FMXMR-4LIEHIV#?RSA
:EPMHEXIVITP]HEXE#?RSA
(EXETEXXIVR?\%&'(A
0SSWI7XVMGX6IGSVH8MQIWXEQT:IVFSWI?RSRIA
7[IITVERKISJWM^IW?RA
8]TIIWGETIWIUYIRGIXSEFSVX
7IRHMRKF]XI-'14)GLSWXSXMQISYXMWWIGSRHW
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—4-14
If you issue the ping command without specifying an IP address, the router goes into extended
ping mode. Extended ping mode allows you to specify the source address from which to send
the ICMP echo requests and modify the size and number of datagrams that you are sending.
You can use extended ping functions to investigate packet drops, variances in response times,
load balancing, packet-size transport problems, and similar types of conditions.
The figure shows how to modify the ping defaults to test the network. This command is useful
in situations where the ping command returns some or most (but not all) of the ICMP echo
replies.
An extended ping command has several functions, as follows:
Increase or decrease the ping timeout, which can help to identify congested networks
Increase, decrease, or vary the packet size, which can help to identify problems with large
packets or fragmentation issues
Use a specific source address instead of the default, which is useful with VoIP calls when
certain subnetworks are failing
Change the number of times to send the ping, which is useful in situations where you might
have a packet loss condition
Include the IP path (similar to the trace command) in the ping operation
4-20
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Verifying the IP Routing Table
84%C,5WLMTVSYXI
'SHIW'z GSRRIGXIH7z WXEXMG-z -+646z 6-41z QSFMPI&z &+4
(z )-+64)<z )-+64I\XIVREP3z 374*-%z 374*MRXIVEVIE
2z 374*277%I\XIVREPX]TI2z 374*277%I\XIVREPX]TI
)z 374*I\XIVREPX]TI)z 374*I\XIVREPX]TI)z )+4
-z -7-70z -7-7PIZIP0z -7-7PIZIPMEz -7-7MRXIVEVIE
'MWHMVIGXP]GSRRIGXIH7IVMEP
3?AZME7IVMEP
?AZME7IVMEP
3?AZME7IVMEP
MWWYFRIXXIHWYFRIXW
3?AZME7IVMEP
3?AZME7IVMEP
MWWYFRIXXIHWYFRIXW
3?AZME7IVMEP
3?AZME7IVMEP
3?AZME7IVMEP
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—4-15
To route packets to a destination device, a router must have information about the network for
the destination device. You can either enter this information statically with the use of the ip
route command, or you can allow the router to learn this information dynamically from another
router in the network through a routing protocol. Use the show ip route command to view the
routing table on a Cisco router. This command does not represent all known routes to a
destination subnetwork—only the best route or routes appear in the table. This table shows a
sample routing table entry.
IP Routing Table
Protocol
Subnetwork
Administrative
distance/cost
Next hop IP
address
Age of
route
Physical interface
to next hop
OSPF
10.11.1.0
[110/65]
via 192.168.12.1
03:29:37
Serial0/0.23
Descriptions of the entries in this IP routing table are as follows:
The Open Shortest Path First (OSPF) protocol is responsible for learning the route. The top
of the routing table has the legend for all routing protocols.
Each known network is indicated. If the routing protocol supports classless routing,
subnetworks are included.
When a router learns about a network from multiple routing protocols, a determination
must be made about which route to use. A concept known as “administrative distance” is
used. These are arbitrary values supplied to rank-order routes. The smaller the value, the
more reliable the routing protocol. As a result, the route with the lowest administrative
distance is the one placed in the routing table, regardless whether it is the best path.
In situations where two routes to the same network are presented from the same protocol,
routers learn the “cost” of each path. Cost is calculated differently for different protocols.
Cost might be hops or time—the metric varies by protocol. The router places the path with
the lowest cost into the table. If two paths are of equal cost, both routes are placed into the
table and distribute traffic across both.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Network Infrastructure
4-21
The next-hop IP address is the IP address of the next router in the path to the destination.
The age of the route indicates when the router learned the route. In some protocols, the
routes are not updated unless and until a network topology change occurs. In other
protocols, routes are continuously updated through periodic exchanges of routing tables.
The last entry is the physical interface (or subinterface) that is necessary to move to the
next-hop router.
When troubleshooting IP telephony problems with the show ip route command, also consider
the summary and default routes.
Summary routes can save space in the routing table by referring to multiple networks (for
example, 150.1.1.0, 150.1.2.0, and 150.1.3.0) with a broader entry of 150.1.0.0. In this
example, a router will route all packets designated for 150.1.x.x to the router advertising the
summary route. Other routers, farther down the line, verify that the packets arrive at the correct
destination. These routers should have routes to the more specific networks (for example,
150.1.1.0, 150.1.2.0, and 150.1.3.0).
Default routes are represented as 0.0.0.0 routes in the routing table and are marked with an
asterisk (*). Default routes take the summary route concept to the next level. Whenever a router
receives a packet, it checks the IP header for the destination address of the packet. After it
determines the destination address, it searches the routing table for a match. When a match is
located (either a specific or summary route), the router will route the packet out of the interface
connected to the next-hop router or to the destination device, if it is directly connected to the
router.
Default routes are necessary when the router cannot find a match in its routing table for a
destination. Normally, the router drops the packet and sends an unreachable error back to the
sender. However, default routes enable the router to send packets that are destined for a
network but not listed in the routing table to the next-hop router referenced in the default route
(0.0.0.0) entry. It is then the responsibility of the next-hop router to route the packet to its final
destination.
4-22
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Verifying a Specific Route
84%C,5WLMTVSYXI
6SYXMRKIRXV]JSV
/RS[RZMESWTJHMWXERGIQIXVMGX]TIMRXVEEVIE
0EWXYTHEXIJVSQSR7IVMEPEKS
6SYXMRK(IWGVMTXSV&PSGOW
JVSQEKSZME7IVMEP
6SYXIQIXVMGMWXVEJJMGWLEVIGSYRXMW
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—4-16
Use the show ip route <network> command in situations where the IP routing table is large
and where you know the destination network or subnetwork. You can use this command with a
specific network identifier (for example, 10.33.3.0) to display routes to that network only.
Using the show ip route command with a specific network identifier substantially reduces the
output of the show ip route command and allows you to focus on specific entries in the routing
table.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Network Infrastructure
4-23
The traceroute Command
84%C&6XVEGI
8VEGMRKXLIVSYXIXS
QWIGQWIGQWIG
QWIGQWIGQWIG
QWIGQWIGQWIG
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—4-17
The traceroute command discovers the routes that a packet takes when traveling to a
destination. The device (a router or a PC) sends out a sequence of User Datagram Protocol
(UDP) messages to an invalid port address at the remote host.
The device sends three datagrams, each with a Time to Live (TTL) field and a value of 1. The
TTL value of 1 causes the datagram to time out as soon as it reaches the first router in the path;
this router then responds with an “ICMP time exceeded” message indicating that the datagram
has expired.
The router sends another three UDP messages, each with the TTL value of 2, which causes the
second router to return ICMP time exceeded messages. This process continues until the packets
reach the other destination. Because these datagrams are trying to access an invalid port at the
destination host, the router returns “ICMP port unreachable messages,” indicating an
unreachable port; this event signals to the traceroute program that the device is responding.
The purpose of this process is to record the source of each ICMP time exceeded message to
provide a trace of the path that the packet took to reach the destination.
The traceroute command in the figure indicates that traffic going to 10.33.3.250 from the
source router (TPA2620_BR) went first to 192.168.22.2, then to 192.168.23.3, and finally to
192.168.33.30. The destination 10.33.3.250 is a local interface on the third router. The roundtrip delay is approximately 4 milliseconds (ms) for each hop.
4-24
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Verifying That Access Lists Are Not the
Problem
EGGIWWPMWXTIVQMXXGTER]ER]IUXIPRIX
EGGIWWPMWXTIVQMXXGTER]ER]IU
MRXIVJEGI7IVMEP
MTEHHVIWW
MTEGGIWWKVSYTSYX
IRGETWYPEXMSRJVEQIVIPE]
JVEQIVIPE]QETMTFVSEHGEWX
MRXIVJEGI*EWX)XLIVRIX
MTEHHVIWW
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—4-18
If a particular subnetwork appears in the routing table, it may indicate that all IP traffic can
reach the subnetwork. In many cases, customers build complex access lists that allow only
certain types of IP traffic to pass through an interface for security reasons. A Layer 4 protocol
(for example, TCP or UDP) or an application (for example, TCP or UDP port number) can
evaluate traffic.
In the figure shown here, TCP traffic destined only for ports 23 and 2000, can pass through the
S0/0 interface in the outbound direction. This access list allows only outbound Telnet and
outbound Skinny Client Control Protocol (SCCP) messages. Even though you do not see any
deny statements in the running-config, all access lists end with an implicit deny ip any any
command that blocks traffic not explicitly permitted in the access list statements.
If there are IP Phones on the LAN connected to this router, and the Cisco CallManager server is
across the Frame Relay network at a central site, the IP Phones can register with the Cisco
CallManager server but they cannot place or receive any calls. There are two reasons for this:
the 101 access blocks the TCP port 1720 (which the H.245 call setup uses), and the UDP ports
16384 through 32767 (which carry voice packets). You will need to remove or modify the
access list to allow VoIP across the Frame Relay network.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Network Infrastructure
4-25
Verifying That NAT Is Not the Problem
MRXIVJEGI7IVMEP
MTEHHVIWW
MTREXSYXWMHI
MRXIVJEGI*EWX)XLIVRIX
MTEHHVIWW
MTREXMRWMHI
JYPPHYTPI\
MTREXTSSP2%84330 TVIJM\PIRKXL
MTREXMRWMHIWSYVGIPMWX2%8C-27-()TSSP2%84330
MTEGGIWWPMWXI\XIRHIH2%8C-27-()
TIVQMXMTER]
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—4-19
You may encounter situations in which the local IP subnetworks are using private IP addresses
while the WAN links are in the public address space. Network Address Translation (NAT)
translates the addresses between the two address spaces. Port address translation (PAT) is not
supported and will not work in a VoIP environment.
In the figure shown here, telephones connected to the FA0/0 interface have a local IP address of
172.16.x.x, but they appear as having an IP address in the range of 216.58.148.135 through
216.58.148.195. For example, to ping a telephone from a remote network, you use the
216.58.148.135 through 216.58.148.195 addresses, not 172.16.x.x.
The approach to solving problems associated with NAT and PAT is similar to what you use for
basic IP failures. Ping first, verifying that the source IP address of the ping is in the same
subnetwork as the calling telephone. If the ping fails, check the routing tables to verify that the
network is reachable. If the routing table appears to be correct, but calls do not go through (or
have one-way voice issues), there may be an address or port translation problem. The debug ip
nat command can locate the problem.
NAT and PAT can cause problems on a VoIP network. Cisco IP Phones use the Skinny Station
Protocol to connect with and register to Cisco CallManager. Messages travel between the Cisco
CallManager and IP Phones include the IP address and port information that identifies other IP
Phone users for placing calls. To deploy NAT between the IP Phone and Cisco CallManager in
a scalable environment, NAT needs to detect the Skinny Station Protocol and process the
information passed within the messages. Cisco IOS Release 12.1(5)T added this support with
the ip nat service skinny command.
PAT causes problems because it overloads multiple inside private addresses to one outside
public address. Overloading causes port numbers to track the address translations. Tracking
address translations causes problems in VoIP networks, because H.323 is a multimedia protocol
that uses dynamic ports and reserved ports, such as TCP 1720 for H.245 call setup.
4-26
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Multicast Routing
This topic describes multicast routing in a Cisco IP telephony network.
Multicast Routing
Multicast and unicast audio sources:
• Multicast music on hold conserves system
resources.
• Multicast allows multiple users to use the same
audio source stream to provide music on hold.
• Multicast audio sources associate with an IP
address.
• Multicast entails managing devices, IP addresses,
and ports.
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—4-20
Is the music on hold (MOH) server registered with Cisco CallManager?
What about the MOH configuration (port or IP address multicast)?
Refer to the latest Solution Reference Network Design guide for Cisco CallManager 4.0 for
additional design and deployment considerations for MOH. Knowing how to configure MOH
for both unicast and multicast is required to troubleshoot and resolve issues.
The biggest cause of nonfunctional multicast MOH is incorrect configuration. If you are
applying MOH, start testing with unicast and then migrate to multicast. The Cisco Technical
Assistance Center (TAC) recommends the use of Protocol Independent Multicast (PIM) dense
mode when applying multicast MOH.
Here are the differences between how an MOH stream is set up in unicast versus multicast:
A unicast MOH server is told by Cisco CallManager to stream to the endpoint that should
receive the music stream.
A multicast MOH server is always playing the audio stream. When an endpoint is supposed
to listen to that stream, the endpoint is simply told to listen to the multicast address on a
certain port. When the endpoint goes off-hook, the endpoint is told to stop listening to the
multicast stream and is reconnected to the calling or called party. The routers between the
MOH server and the endpoint must be configured to forward multicast packets. If one of
the devices in the multicast stream is not configured to pass multicast traffic, the endpoint
will not hear MOH.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Network Infrastructure
4-27
When troubleshooting multicast problems with Cisco IOS software gateways, make sure that
the Cisco IOS software load you are using can accept multicast MOH.
Multicast MOH is supported for H.323 and MGCP endpoints in Cisco IOS Release 12.2(11)T
and above.
All routers in the multicast path must have “ip multicast-routing” in global configuration. The
same routers need to have “ip pim dense-mode” or “ip pim sparse-mode” on their interfaces.
Here is a sample configuration:
'IRXVEPVSYXIV
MTQYPXMGEWXVSYXMRK
MRXJE
MTTMQHIRWIQSHI
MTMKQTNSMRKVSYT
MTMKQTNSMRKVSYT
MRXW
MTTMQHIRWIQSHI
MTMKQTNSMRKVSYT
MTMKQTNSMRKVSYT
6IQSXIVSYXIV
MTQYPXMGEWXVSYXMRK
MRXJE
MTTMQHIRWIQSHI
MTMKQTNSMRKVSYT
MTMKQTNSMRKVSYT
MRXW
MTTMQHIRWIQSHI
MTMKQTNSMRKVSYT
MTMKQTNSMRKVSYT
To have the router join a multicast group, use the ip igmp join-group interface configuration
command. To cancel membership in a multicast group, use the no form of this command. The
two sample IP addresses come from Cisco CallManager servers with multicast MOH addresses
239.1.1.1 and 239.1.1.2.
The Max Hops field in the MOH server configuration indicates the maximum number of
routers that an audio source is allowed to cross. If max hops are set to zero, the audio source
must remain in its own subnetwork. If max hops are set to one, the audio source can cross up to
one router to the next subnetwork. Cisco recommends setting the max hops to the diameter of
the network.
4-28
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Addresses for IP multicast range from 224.0.1.0 to 239.255.255.255. Cisco strongly
discourages using public multicast addresses for MOH multicast. Instead, Cisco recommends
using an IP address in the range that is reserved for administratively controlled applications on
private networks (239.0.0.0 to 239.255.255.255).
Valid port numbers for multicast include even numbers that range from 16384 to 32767. (The
system reserves odd values.)
Tip
Multicast functions only if both media resource groups (MRGs) and media resource group
lists (MRGLs) are defined to include a multicast MOH server. For MRGs, you must include
an MOH server that is set up for multicast. Also, check the Use Multicast for MOH Audio box
when defining an MRG for multicast. For MRGLs that are associated with device pools and
devices, define the MRGL so that the MRG setup for multicast is the first group in the list.
This recommended practice facilitates the efforts by the device to find the multicast audio
source first.
In MOH processing, the device placed on hold determines the media resource to use; however,
the device that initiates the hold action determines the audio source to use.
Here are some deployment model MOH considerations:
Single Site:
Cisco recommends the use of G.711 (A-law or mu-law) coder-decoders (codecs) for all
MOH audio streams.
Some areas in the network might not support multicast; therefore, unicast MOH will suffice
until the network becomes too large to use unicast streams.
Centralized Multisite:
Cisco recommends the use multicast and the use of G.729 codecs for MOH traversing the
WAN.
Distributed Multisite:
Cisco recommends the use of unicast MOH—because multicast is not supported between
Cisco CallManager (over ICT])—and the use of G.729 codecs for all MOH servers.
Clustering Over the WAN:
Cisco recommends the use of G.729 codecs for all MOH servers.
Each site should have its own MOH server.
Common Troubleshooting Monitoring Multicast Commands:
show ip mroute
show ip igmp group detail
show ip igmp membership all
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Network Infrastructure
4-29
Summary
This topic summarizes the key points discussed in this lesson.
Summary
• Routers and switches and their interconnecting links
provide the physical infrastructure needed to transport
VoIP traffic.
• Tools for troubleshooting UP routing problems include
the following commands: ping, traceroute, show ip route,
and debug ip nat.
• You configure VLANs on switches on a port-by-port
basis.
• Until recently, a switch port was either dedicated to a
single VLAN or trunked (all VLANs). Recognizing the
need for a limited function trunk, Cisco developed
support for auxiliary VLANs.
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—4-21
References
For additional information, refer to these resources:
Cisco TAC voice troubleshooting guidelines:
http://www.cisco.com/en/US/products/sw/voicesw/ps556/products_tech_note09186a00800
a6a14.shtml.
Case Study: Troubleshooting Cisco IP Phone-to-Cisco IOS Gateway Calls:
http://www.cisco.com/univercd/cc/td/doc/product/voice/c_callmg/3_3/trouble/3_3_3/tbl33c
.htm.
4-30
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Quiz
Use the practice items here to review what you learned in this lesson. The correct answers are
found in the Quiz Answer Key.
Q1)
If you see the status message “FastEthernet1/0 is down, line protocol is down” on a
Catalyst 6500 series switch, what is the likely cause?
A)
B)
C)
D)
Q2)
To perform an extended ping, which command is necessary?
A)
B)
C)
D)
Q3)
an ISL trunk
extended VLAN support
an IEEE 802.1Q trunk
all of the above
Which two of these modes are valid switch trunk modes? (Choose two.)
A)
B)
C)
D)
Q5)
router(config)#ping
router#ping
router>ping
router#ping extended
To support auxiliary VLANs, which of the following should you configure on the
switch?
A)
B)
C)
D)
Q4)
There is a physical connectivity issue.
An attached device is using an incorrect encapsulation type.
There is an IP address conflict.
The line is configured as a trunk with an IP Phone attached.
ISL
IEEE 802.5Q
802.9
IEEE 802.1Q
Which multicast IP address range does Cisco recommend that you use?
A)
B)
C)
D)
239.0.0.0 to 239.255.255.255802.5
224.0.1.0 to 239.255.255.255
172.0.0.0 to 172.255.255.255
10.0.0.0 to 10.255.255.255
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Network Infrastructure
4-31
Quiz Answer Key
4-32
Q1)
A
Q2)
B
Q3)
C
Q4)
A, D
Q5)
A
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Gateways
Overview
This lesson will discuss the common issues related to gateway configuration, Public Switched
Telephone Network (PSTN) signaling, and various voice interfaces. This lesson will also cover
call flow through a gateway and dial peers operation and monitoring commands used in Cisco
IOS software. In addition, this lesson will discuss survival remote site telephony (SRST)—how
it operates and troubleshooting. You will learn troubleshooting techniques, including show and
debug commands, that you can use to diagnose and correct IP telephony problems related to
the underlying data infrastructure.
Relevance
Gateways and switches, along with the communications links covered in other lessons, are the
technologies that provide the infrastructure to transport IP telephony traffic from its origin to its
destination. Any problems associated with the underlying data infrastructure can result in either
poor voice quality or the inability to complete calls.
Objectives
Upon completing this lesson, you will be able to:
Identify common voice issues related to gateway configurations
Identify common issues with digital voice interfaces
Identify common Cisco IOS configuration issues for H.323, MGCP, and SIP
Describe the operation of call flow in a Cisco IOS gateway
Explain the operation and configuration of SRST in the branch office
Learner Skills and Knowledge
To benefit fully from this lesson, you must have these prerequisite skills and knowledge:
Fundamental understanding of Cisco IOS commands at the Cisco CCNP® level
Prior exposure to Cisco VoIP
Outline
The outline lists the topics included in this lesson.
Outline
• Overview
• Troubleshooting Common Gateway Issues
• Troubleshooting Common Digital Voice
Connections
• Signaling Protocol Configuration
• Dial Peers and Digit Analysis
• Router Call Flow and Call Control
• Survival Remote Site Telephony
• Summary
• Quiz
© 2004 Cisco Systems, Inc. All rights reserved.
4-34
Cisco IP Telephony Troubleshooting (IPTT) v4.0
IPTT v4.0—4-3
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Common Gateway Issues
The topic discusses the specifics related to common configuration on gateways.
Troubleshooting Common Gateway Issues
Applications
(VMail, IVR, ICD, ...)
PSTN
SRST-Enabled
Router
Cisco
CallManager
Cluster
IP WAN
Branch A
SRST-Enabled
Router
Headquarters
Gatekeeper
• The gateway is the interface between the data
network, IP WAN, and the PSTN.
© 2004 Cisco Systems, Inc. All rights reserved.
Branch B
IPTT v4.0—4-4
One of the main functions of a gateway is to translate functions for call setup and teardown
between the IP network and the PSTN. A gateway connects two dissimilar networks (for
example, an H.323 and MGCP network and a circuit-switched network). The gateway performs
functions such as protocol translation for call setup and release, conversation of media formats
from one network to another (for example, time-division multiplexing [TDM] to IP), and
transfer of information between the networks connected by the gateway.
For gateways to provide their functions as indicated previously, they must use a dial plan. A
dial plan defines the locations of phone numbers in a VoIP network so that calls can be
established by the gateway. A dial peer, an important concept in Cisco IOS software,
implements a dial plan in gateways. This dial plan driven by dial peers is mostly seen in H.323
gateways. An H.323 gateway uses its dial peer to direct where digits go and how the digits are
handled. MGCP gateways, on the other hand, have different behavior in terms of where the
digits hit the call agent and the call agent sets up the call.
Dial peers in an H.323 gateway determine the direction of the voice packet within the gateway.
There are two types of dial peers, VoIP and plain old telephone service (POTS). The POTS dial
peer determines the PSTN destination. POTS dial peers are seen in both H.323 and MGCP
gateways. VoIP dial peers, however, are not seen in MGCP gateways but are seen in H.323
gateways (except when MGCP gateways are configured for H.323 failover). The VoIP dial in
H.323 gateways is used to determine the IP destination of the terminating gateway or endpoint.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Network Infrastructure
4-35
In H.323 gateways, both dial peers, POTS and VoIP, are required to process calls; however, in
MGCP gateways, there are only POTS dial peers, and these peers point to the PSTN. When
H.323 and MGCP gateways operate with Cisco CallManager servers, H.323 does not send
keepalives to the Cisco CallManager; however the MGCP gateway does. If the Cisco
CallManager server goes down, so does the MGCP gateway relative to VoIP. However with
H.323, if the Cisco CallManager server goes down, the gateway can still handle VoIP traffic
with alternate dial peers. The MGCP gateway fully depends on the Cisco CallManager server.
Troubleshooting Common Gateway Issues
(Cont.)
PSTN
IP
H.323
• All PSTN signaling terminates on gateway.
• H.225 communication between gateway and Cisco
CallManager.
• H.323 is a “peer-to-peer” protocol.
T1/ISDN PRI
T1/CAS
Analog
H.225
Cisco CallManager
• Framing and Layer 2 signaling terminates at the
gateway.
MGCP
Backhauling
T1/ISDN PRI
T1/CAS
Analog
• Layer 3 signaling is backhauled to Cisco CallManager.
• MGCP is a “client-server” protocol.
PRI Layer
MGCP
Cisco CallManager
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—4-5
How the PSTN signals with the gateways can come in various forms such as ground start and
loop-start analog circuits, T1/ channel associated signaling (CAS) or T1/PRI ISDN circuits or
both. This lesson will focus on digital circuits starting with T1/CAS. CAS is the transmission of
signaling information within the information band, or in-band signaling. This means that voice
signals travel on the same circuits as line status, address, and alerting signals. Because there are
24 channels on a full T1 line, CAS interleaves signaling packets within voice packets, and
therefore, there are a full 24 channels to use for voice.
Various types of CAS are available in the T1 world. The most common forms of CAS signaling
are loop-start, ground start, and ear and mouth (E&M) signaling.
T1/PRI ISDN, on the other hand, uses common channel signaling (CCS). CCS is the
transmission of signaling information out of the information band. The most notable and widely
used form of this signaling type is ISDN. One disadvantage to using an ISDN PRI is the
removal of one digital service level 0 (DS0), or voice channel, in this case for signaling use.
One T1/PRI would have 23 DS0s, or bearer channels (B channels) for user data, and one DS0,
or data channel (D channel) for signaling. It is possible to control multiple PRIs with a single D
channel, each using Non-Facility Associated Signaling (NFAS). Therefore, you can configure
the other PRIs in the NFAS group to use all 24 DS0s as B channels. Using PRI signaling
ensures the maximum possible connection rates, especially with the advent of 56 K modems.
4-36
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
With H.323 gateways, PSTN signaling is handled by the gateway. Dialed digits coming into the
H.323 gateway are evaluated then processed per dial-peer matching to the Cisco CallManager
server. The Cisco CallManager server then receives the digits from the H.323 gateways and
sets up the call. With outbound calls from the Cisco CallManager server, digits are sent to the
H.323 gateway and again through dial-peer matching; the call is processed, handled by the
gateway to the PSTN.
However, with a MGCP gateway, signaling with the Cisco CallManager server is different.
With MGCP, calls come right into the Cisco CallManager server and are processed from there.
The gateway does no digit translating or digit analysis—unlike with H.323 gateways. The
reason that there are more PSTN signaling issues with H.323 gateways than with MGCP
gateways is due simply to how the Cisco CallManager server and gateways interact. H.323
gateways are considered peers to the Cisco CallManager server, and MGCP gateways are
controlled by the Cisco CallManager server.
Troubleshooting Common Gateway Issues
(Cont.)
• When a setup message arrives to the originating gateway with PI=3, this
message indicates to the gateway that in-band messages are available.
• Progress indicates of tones and announcements are available is signaled
by an alerting, call proceeding, progress, connect, setup
acknowledgment, or disconnect message contained in a PI of 1 or 8.
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—4-6
During an ISDN call establishment, the call may or may not leave the ISDN network because
of internetworking with another network, with a non-ISDN user, or with non-ISDN switching
equipment within the premises of the user. When such a situation occurs, a progress indication
is returned to the calling user either in a call control message when a state change is required
such as call proceeding, alerting, setup acknowledgment, and connect, or in the progress
message when no state change is appropriate.
Progress indicators (PIs) tell the gateway or Cisco CallManager or both that the call is or is not
end-end ISDN and thus signal whether setup messages are available in-band. The issue arises
when the messages are not present or are present and the gateway or Cisco CallManager or
both does not see them. One example is an IP Phone or PSTN phone that does not hear
ringback when making a call.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Network Infrastructure
4-37
In the next few slides, you will see issues related to progress indicators and the issues they can
cause when the gateway or Cisco CallManager (devices) or both look for out-band signaling
and the progress indicator indicates to the device to look in-band.
A lack of progress indicators in a message assumes that the originating device will provide the
appropriate tone signaling to the calling party. Analog and digital CAS PSTN circuits will
usually carry the information as in-band information. Conversely, with ISDN PRI, signaling is
out-band, on the D channel.
To set a specific PI in call setup, progress, or connect messages from an H.323 VoIP gateway,
use the progress_ind dial-peer configuration command. To restore the default condition, use
the no or disable forms of this command.
For setup messages from a VoIP dial peer, the values are 0, 1, or 3. For progress, connect, or
alert messages from a POTS dial peer, the values are 1, 2, or 8.
4-38
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Common Gateway Issues:
(H.323)
Call Direction
Cisco CallManager
Alerting PI=0
PSTN
H.323 Gateway In-band ringing
No ringback on an IP Phone when calling the PSTN
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—4-7
Symptom
A Cisco IP Phone user makes an outbound call to the PSTN/PBX through a Cisco IOS H.323
gateway and does not hear a ringback tone.
Problem Description
In this situation, the originating device expects in-band ringback tones but neither of the
following may be happening:
The PSTN/switch is not providing the ringback tone.
The Cisco IOS gateway is not cutting through the audio to the originating device.
The problem in this scenario is that Cisco CallManager automatically cuts through the audio to
the H.323 gateway as soon as the logical channel signal is complete. The IP Phone does not
hear a ringback tone because the alerting message sent by the PSTN did not contain a PI
indicating that in-band (the far end is being alerted) information is available. Typically, in an
end-to-end PSTN ISDN network, the end device is responsible for locally generating ringback
upon receiving an alert message. Cisco CallManager relies on in-band ringback when calling
out an H.323 gateway.
Solution
To fix this problem, use the progress_ind alert enable 8 command under the POTS dial peer
that matches for the outbound call leg of the specific call. As soon as this command is issued,
the Cisco IOS software treats an alerting message as if it came in with a PI of 8. A PI of 8
means that in-band information is available, and that the gateway is supposed to cut through the
audio upon receiving an alerting message.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Network Infrastructure
4-39
Note
The progress_ind alert and progress_ind setup commands are hidden in some versions
of Cisco IOS software and may not be visible within the help parser. However, if the
progress_ind progress command is available in the help parser, the progress_ind alert
and progress_ind setup commands will also be available and can be entered into the dial
peer in their entirety. These commands will subsequently appear in the running
configuration.
Troubleshooting Common Gateway Issues:
(H.323) (Cont.)
Call Direction
Cisco CallManager
H.225 Alerting
No ringback!
Setup PI=0
PSTN
H.323 Gateway In-band ringing
No ringback on a PSTN phone when calling an IP Phone
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—4-8
Symptom
A PSTN/PBX user places a call to a Cisco IP Phone through a Cisco IOS H.323 gateway and
does not hear a ringback tone before the call is answered.
Problem Description
As indicated in the previous ringback problem, a ringback issue can occur for calls coming into
the IP network from the PSTN. When the Cisco IOS software gateway receives a setup
message without a PI, the gateway does not play ringback toward the PSTN. This is because
the gateway assumes that the PSTN is an end-to-end ISDN and therefore relies on the far end to
play ringback to the originating device upon receiving the alerting message. If the network is
not an end-to-end ISDN, the setup message should arrive with a PI of 3, indicating that the
origination address is non-ISDN. In this scenario, the device on the PSTN does not send a PI.
To resolve this issue, you need to configure the gateway to play ringback toward the PSTN
regardless of the PI in the setup message. Note that even though you are trying to resolve the
issue with ringback on the POTS side, the listed parameters must be configured under the VoIP
dial-peers. The commands listed here force the Cisco gateway to treat every setup as if it has a
PI of 3. This forces the gateway to play in-band ringback toward the PSTN.
4-40
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Solutions
Use one of the following solutions:
Configure the Cisco IOS command progress_ind setup enable 3 under the voice dial-peer
# voip command in the Cisco gateway. This command forces the gateway to treat the
inbound ISDN setup message as if it came in with a PI of 3, and to generate an in-band
ringback tone toward the calling party if the H.225 alert message does not contain a PI of 1,
2, or 8.
An alternate to the progress_ind setup command is the dial-peer voice # voip
subcommand tone ringback alert-no-pi. This will cause the gateway to generate ringback
toward the calling party if an alert is received on the IP call leg with no PI present. It differs
from the progress_ind setup command in that the outbound H.225 setup message will not
contain a PI of 3 with the tone ringback command. Some devices may not accept setup
messages when a PI is included.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Network Infrastructure
4-41
Troubleshooting Common Gateway Issues:
(H.323) (Cont.)
Cisco CallManager
Progress Transfer
H.225 User Info
No ringback
Ringback!
PSTN
H.323 Gateway
No ringback to PSTN when IP Phones initiate a call transfer
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—4-9
Symptom
No ringback is heard by a PSTN/PBX user when a Cisco IP Phone initiates a call transfer
through a Cisco IOS H.323 gateway. A common problem is the lack of ringback toward the
PSTN user when an IP Phone user transfers a call to another phone.
Problem Description
The problem occurs from the lack of signaling passed through by the gateway when the call is
transferred. This issue arises when the logical channel for that call is torn down between the
gateway and the transferring device. With the progression signaling torn down, the PSTN user
does not hear a ringback tone.
Solutions
To get around this issue, the gateway needs run Cisco IOS Release 12.2(3) or later. In Cisco
CallManager, the version needs to be 3.0(8) or later. In Cisco CallManager, to get the signaling
to send ringback upon transfer, make sure that the ToSendH225UserInfoMsg service parameter
is set to “true.” In Cisco CallManager 3.2.3 and above, the service parameter accepts a
numerical value of 0, 1, or 2 instead of “true” or “false” as follows:
0 – Do not send progress tone information.
1 – Use the H225UserInfo message to send progress tone information.
2 – Use the H225info message to send progress tone information.
In Cisco CallManager 3.3 and above, make sure that under the Send H225UserInfo Message
selection, there is no check in the box for No RingBack and that there is a check in the box for
User Info for Ringback Tone.
4-42
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Common Gateway Issues:
(H.323) (Cont.)
Call Direction
Cisco CallManager
Disconnect pd=8
PSTN
No busy tone or
PSTN message is heard
H.323 Gateway
An IP Phone user does not hear a busy signal when calling a busy
number on the PSTN; instead, the call is disconnected.
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—4-10
Symptom
There is an IP Phone or analog phone (VoIP toll-bypass scenario) user who is disconnected
when calling a busy number or does not hear the announcement from the PSTN indicating a
nonworking number has been dialed (or both of these things happen).
Problem Description
When an IP Phone or analog phone places a call to a busy or nonworking number, the call is
disconnected as opposed to the caller receiving a busy tone or a message from the PSTN
indicating that the number being called in a nonworking number. The disconnect in this case is
occurring under normal call clearing conditions.
Solutions
To resolve this issue, configure the global configuration command voice call convert-discpito-prog (Cisco IOS Release 12.2[1) and later) in the Cisco gateway. By using this command,
you will convert a disconnect message with a PI to a progress message with a PI so that the
audio is cut through and the IP Phone or analog phone or both can hear the busy tones or
announcements or both from the PSTN.
In the VoIP toll-bypass scenario, most of these issues are resolved by upgrading the gateway or
gateways to a Cisco IOS Release of 12.1(3a)XI5 or 12.2(1) and later. However, if the
originating device or originating ISDN switch does not keep the call active when a H.225 ISDN
disconnect message is received, try using the voice call convert-discpi-to-prog command.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Network Infrastructure
4-43
Troubleshooting Common Gateway Issues:
(H.323) (Cont.)
Cisco CallManager
PSTN
H.323 Gateway
IP WAN
One-way or no-way audio
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—4-11
Symptoms
1. When establishing a phone call from an IP station through a Cisco IOS voice gateway, only
one of the parties receives audio (one-way communication).
2. When establishing a phone call from an IP station through the campus, only the calling
signaling is allowed; the Real-Time Transport Protocol (RTP) stream is not allowed.
3. When establishing a toll-bypass call between two Cisco gateways, only one of the parties
receives audio (one-way communication).
4. Some 411 and 911 calls can produce one-way audio
Problem Description
The causes for one-way audio in IP telephony can be varied; however the root of the problem is
usually related to IP routing issues.
Follow one or more the following solutions:
Check that the IP Phones are pointed to the correct default gateway.
—
Cisco IP Phone 7910: Press Settings, choose option 6, push volume down until the
Default Gateway field shows up
—
Cisco IP Phone 7940 and 7940: Press Settings, choose option 3, scroll down until
the Default Gateway field shows up
Check to see if IP routing enabled.
—
Some Cisco IOS gateways, such as the VG200, have IP routing disabled by default.
This will lead to one-way voice problems.
—
Enable IP routing in global configuration.
Check to ensure that access lists are not blocking the RTP traffic.
4-44
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
When the Cisco IOS gateway has multiple active IP interfaces, some of the H.323 signaling
may be sourced from one IP address and other parts of it may reference a different source
addresses. This can generate various kinds of problems, one of them being one-way audio. To
get around this problem, the H.323 signaling can be bound to a specific source address, which
can belong to a physical or virtual interface (loopback). The command syntax to use under the
interface configuration mode is h323-gateway voip bind srcaddr<ip address>. Configure this
command under the interface with the IP address that the Cisco CallManager points to.
Additionally, some 411 and 911 calls can produce one-way audio. In global configuration
mode, add the command voice rtp send-recv. Sometimes, the numbers 411 and 911 do not
send back answer supervision to the remote end calling. This causes issues with the gateway,
and thus the gateway does not cut through the audio path in both directions. Using this
command forces the gateway to cut through the audio in both directions.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Network Infrastructure
4-45
Troubleshooting Common Gateway Issues:
(H.323) (Cont.)
Call Direction
Cisco CallManager
PSTN/PBX
H.323 Gateway
No DTMF digits or audio passed on calls to PSTN/PBX
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—4-12
Symptom
A Cisco IP Phone user makes a PSTN call and hears an announcement message (for example,
“enter your account number....”) but cannot pass dual tone multifrequency (DTMF) digits. This
symptom also applies for VoIP toll bypass.
Problem Description
A Cisco IP Phone (in a Cisco CallManager scenario) or POTS phone (VoIP toll-bypass
scenario) call leaves through a Cisco IOS gateway, where the called number is usually an
interactive voice response (IVR) system that sends back an ISDN progress message, but does
not connect until some account information is entered. By default, the audio path is cut through
in the backward direction (toward the IP Phone or originating gateway), but not in the forward
direction, until the terminating gateway receives a connect message. Therefore, there is no
voice path to transmit DTMF tones or speech toward the terminating switch.
Solutions
Configure in Cisco IOS global configuration mode, the command voice rtp send-recv to
establish (cut through) the audio path in both directions prior to receiving an ISDN connect
message from the PSTN.
4-46
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Common Gateway Issues:
(MGCP)
Cisco CallManager
FXS/FXO
PSTN
ISDN-PRI/CAS
MGCP Gateway
IP WAN
The gateway or endpoints or both do not get registered
with CallManager.
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—4-13
Here are the top three common MGCP issues:
Fully qualified host name incorrect
Missing service type statement
Missing application mgcpapp command
Presented in this figure are the top issues reported to the Cisco AVVID TAC related to MGCP
gateways and Cisco CallManager. An explanation of each is provided here.
Symptom
The endpoints or gateways will not register with the call agent (Cisco
CallManager)Incorrect fully qualified host name:
—
If the IP domain name is configured on the gateway, the IP domain name is
concatenated with the host name to form a fully qualified domain name. This means
that in Cisco CallManager, the name of the gateway must have the IP domain name
concatenated to the host name or the gateway will not register. This is pending the
correct configuration for Cisco CallManager control, which will be discussed in this
lesson.
Missing ccm-manager commands:
—
Although the ccm-manager mgcp command is one of the top issues with MGCP
configuration, the other required command will be discussed as a reminder. Here are
the two commands that must be entered in global configuration mode on the
gateway so that Cisco CallManager can control the gateway and so that the gateway
knows where to get its extensible markup language (XML) file downloads:
ccm-manager mgcp: To enable the gateway to communicate with
Cisco CallManager through MGCP and to supply redundant control agent services,
use the ccm-manager mgcp command in global configuration mode.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Network Infrastructure
4-47
Note
The ccm-manager config command without the server is optional.
GGQQEREKIVW[MXGLFEGOMQQIHMEXI
GGQQEREKIVJEPPFEGOQKGT
GGQQEREKIVVIHYRHERXLSWX
GGQQEREKIVQKGT
Missing mgcp command statements in global configuration mode:
This command output is a generic sample of what needs to be added under global
configuration mode on a MGCP gateway:
QKGT
QKGTGEPPEKIRXWIVZMGIX]TI QKGTZIVWMSR
QKGTHXQJVIPE]ZSMTGSHIGEPPQSHISYXSJFERH
QKGTVXTYRVIEGLEFPIXMQISYX
QKGTTEGOEKIGETEFMPMX]VXTTEGOEKI
RSQKGTXMQIVVIGIMZIVXGT
There are various options in applying MGCP packages.
Missing application mgcp command statements:
HMEPTIIVZSMGITSXW
ETTPMGEXMSRQKGTETT
TSVX
HMEPTIIVZSMGITSXW
ETTPMGEXMSRQKGTETT
TSVX
HMEPTIIVZSMGITSXW
ETTPMGEXMSRQKGTETT
TSVX
Note
4-48
The application mgcp command statement must be attached to the POTS dial peers for
Cisco CallManager to see it and gain control of the endpoint. If the application mgcp
command statement is not present, Cisco CallManager will not register the endpoint.
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Common Gateway Issues:
(MGCP) (Cont.)
8SXEPRYQFIVSJLSWX
4VMSVMX]7XEXYW,SWX
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
4VMQEV]6IKMWXIVIH
*MVWXFEGOYT&EGOYTVIEH]
'YVVIRXEGXMZI'EPP1EREKIV
'YVVIRXFEGOYT'EPP1EREKIV
6IHYRHERXPMROTSVX
*EMPSZIV-RXIVZEPWIGSRHW
/IITEPMZI-RXIVZEPWIGSRHW
0EWXOIITEPMZIGLIGO
0EWX1+'4XVEJJMGXMQI
0EWXW[MXGLSZIVXMQI2SRI
7[MXGLFEGOQSHI-QQIHMEXI
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—4-14
Just because the gateway shows that it is registered does not mean that it is indeed registered
and that everything is configured correctly. What this really means is that the gateway is
connected to the Cisco CallManager server on TCP port 2428. Port 2428 is used to exchange
keepalives with Cisco CallManager. Cisco CallManager listens for MGCP messages on UDP
port 2427.
In Cisco CallManager, the gateway device should also show that it is registered. If in Cisco
CallManager the gateway does not show as registered, make sure that the gateway name is
entered correctly and consider whether the ip domain-name command is being used. If the
name seems OK, enter the gateway EXEC mode and enter no mgcp followed by mgcp, then
use a show ccm-manager command in EXEC mode and see if the gateway shows as
registered. If the gateway shows as registered, go to Cisco CallManager and reset the gateway
and see if Cisco CallManager then registers the gateway. In some cases, you may need to
power cycle the gateway. You should only power cycle after confirming the gateway
configuration, connectivity between the gateway and Cisco CallManager, and after doing the
steps that have been discussed.
Here are the three status types to look for:
Registered: The gateway is sending and receiving keepalives with Cisco CallManager on
TCP port 2428.
Registering with Cisco CallManager: The gateway is trying to connect to Cisco
CallManager over TCP port 2428.
Unregistered: Lost keepalives, cannot find Cisco CallManager.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Network Infrastructure
4-49
To ensure that your MGCP configuration is working properly, you can issue the commands
show ccm and show mgcp endpoint. The following is an example of the show mgcp endpoint
command:
%:WLS[QKGTIRHTSMRX
-RXIVJEGI8
)2(43-282%1):43687-+8=4)%(1-2
7HW$%:RSRIYT
7HW$%:RSRIYT
7HW$%:RSRIYT
7HW$%:RSRIYT
7HW$%:RSRIYT
7HW$%:RSRIYT
7HW$%:RSRIYT
7HW$%:RSRIYT
7HW$%:RSRIYT
7HW$%:RSRIYT
7HW$%:RSRIYT
7HW$%:RSRIYT
7HW$%:RSRIYT
7HW$%:RSRIYT
7HW$%:RSRIYT
7HW$%:RSRIYT
7HW$%:RSRIYT
7HW$%:RSRIYT
7HW$%:RSRIYT
7HW$%:RSRIYT
7HW$%:RSRIYT
7HW$%:RSRIYT
7HW$%:RSRIYT
4-50
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
The debug mgcp packet command will allow you to see the interaction between the gateway
and Cisco CallManager. Here is an example:
%:HIFYKQKGTTEGOIX
1IHME+EXI[E]'SRXVSP4VSXSGSPTEGOIXWHIFYKKMRKMWSR
%:XIVQMREPQSRMXSV
%:
1EVWIRHCQKGTCQWK1+'44EGOIXWIRXXS
"
1EV28*=$%:1+'4<
1EV1+'44EGOIXVIGIMZIHJVSQ
1EVWIRHCQKGTCQWK1+'44EGOIXWIRXXS
"
1EV28*=$%:1+'4<
1EV1+'44EGOIXVIGIMZIHJVSQ
Gateway name: AV-2620-4.
Cisco CallManager IP address 172.16.1.1.
X:0: The request identifier is set to 0. This means that any notify (NTFY)
commands sent back to Cisco CallManager use 0 as the request identifier so that
Cisco CallManager knows which request the notification corresponds to.
If you were to pull up the Cisco CallManager traces for this event, you would
see the same interaction logged.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Network Infrastructure
4-51
Troubleshooting Common Gateway Issues:
(Dropped Calls)
• Troubleshooting dropped calls when a PSTN call
has been established:
– Dropped calls can be the result of a flapping
circuit, phone, or gateway resetting.
• Use debug commands.
• Check the Microsoft Windows 2000 Event Viewer
in Cisco CallManager, look for cause codes, and
cross reference the cause codes to system
errors messages
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—4-15
The biggest challenge with isolating issues related to dropped calls is quickly isolating the
cause.
Determine whether the dropped calls problem is isolated to one phone or to a group of phones.
Perhaps you will find that the affected phones are all on a particular subnetwork or location.
Check the Microsoft Windows 2000 Event Viewer for phone or gateway resets.
You will see one warning and one error message for each phone that resets. This indicates that
the phone cannot keep its TCP connection to the Cisco CallManager alive, so the
Cisco CallManager resets the connection. This may occur because a phone was turned off or a
problem may exist in the network. If this is an intermittent problem, you may find it useful to
use Microsoft Performance Monitor to record phone registrations.
Here are links to assist in determining error messages seen in the Event Viewer on the server:
System error messages for Cisco CallManager: (Look for “Device type” and “Reason
Code”):
http://www.cisco.com/univercd/cc/td/doc/product/voice/c_callmg/3_3/alarms31.htm.
ISDN Switch Types, Codes, and Values:
http://www.cisco.com/univercd/cc/td/doc/product/software/ios113ed/dbook/disdn.htm.
If the call is going out of a gateway to the PSTN, you can use the Call Detail Record (CDR) to
determine which side is hanging up the call. You can obtain much of the same information by
enabling tracing on Cisco CallManager. Because the Trace tool can affect Cisco CallManager
performance, you will want to use this option only as a last resort or if your network is not yet
in production.
4-52
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Common Gateways
Issues: (Reorder Tone)
after
during
Cisco CallManager
H.323/MGCP/SIP
PSTN
Gateway
• Is reorder tone (fast busy) heard
during digit entry or after all digits
are entered?
© 2004 Cisco Systems, Inc. All rights reserved.
IP WAN
IPTT v4.0—4-16
To speed up your troubleshooting effort when dealing with reorder tones, it is important to pay
attention to when you are getting a reorder tone. Here are some rules of thumb to follow:
If a reorder tone is received after the first digit entry, look to the device configuration in
Cisco CallManager for the issue.
If you receive a reorder tone after entering all the digits, look to the target gateway. Review
the gateway configuration by running the dial plan wizard and seeing which gateway you
are hitting.
Another option to use is the Cisco Dialed Number Analyzer (DNA) tool offered in Cisco
CallManager 3.3.4 and 4.0(1).(For more information, go to
http://www.cisco.com/univercd/cc/td/doc/product/voice/c_callmg/4_0/cdna/index.htm.)
You can install DNA as a plug-in to Cisco CallManager. The tool allows you to test a
Cisco CallManager dial plan configuration before deploying it. You can also use the tool to
analyze a dial plan after the dial plan is deployed.
Because a dial plan can be complex, involving multiple devices, translation patterns, route
patterns, route lists, route groups, calling and called party transformations, and device-level
transformations, a dial plan may contain errors. You can use DNA to test a dial plan by
providing dialed digits as input. The tool analyzes the dialed digits and shows details of the
calls. You can use these results to diagnose the dial plan, identify problems if any, and tune the
dial plan before it is deployed.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Network Infrastructure
4-53
Troubleshooting Common WS-X6624 and
WS-X6608-T1/E1 Issues
• Layer 3 connectivity to Cisco CallManager
• Port registration problem represents one of the
most common issues
• MAC address of the port is incorrect in Cisco
CallManager
• DHCP is not enabled or disabled
• Missing or corrupt Device Default application load
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—4-17
In the event that you experience a registration problem with either an analog access WSX6624-FXS (Foreign Exchange Station)/X66234-FXO (Foreign Exchange Office) blade or a
digital access WS-6608-T1/E1 blade, follow these troubleshooting steps.
First, check that the gateway is up and running. All gateways have a heartbeat LED that blinks
1 second-on, 1 second-off when the gateway software is running normally.
If this LED is not blinking at all, or blinking very rapidly, the gateway software is not running.
Normally, this results in an automatic reset of the gateway. Also, it is normal for the gateway to
reset itself if it cannot complete the registration process after about 2 to 3 minutes. So, you may
happen to look at the heartbeat LED while the device is resetting; however, if the normal
blinking pattern does not appear in 10 to 15 seconds, the gateway has suffered a serious failure.
On Cisco access analog gateways, find the green heartbeat LED on the far right of the front
panel. On Cisco digital access gateways, find the red LED on the far left on the top edge of the
card. On the Cisco analog access WS-X6624 blade, a green LED appears inside the blade (not
visible from the front panel) on the far-right card edge near the front. Finally, on the digital
access WS-X6608, a separate heartbeat LED exists for each of the eight spans on the blade.
Eight red LEDs appear across the card (not visible from the front panel) about two-thirds of the
way toward the back.
Check that the gateway received its IP address. A standalone gateway must receive its IP
address via DHCP or BOOTP. A Catalyst gateway may receive its IP address by DHCP,
BOOTP, or by manual configuration.
If you have access to the DHCP server, the best way to check a standalone gateway is to verify
that the device has an outstanding lease on an IP address. If the gateway shows up on your
server, this provides a good indication but is not definitive. Delete the lease at the DHCP
server.
Reset the gateway.
4-54
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
If the gateway reappears on the server with a lease within a couple of minutes, everything is
working fine in this area; if not, either the gateway cannot contact the DHCP server (Is a
gateway improperly configured and not forwarding DHCP broadcasts? Is the server running?)
or cannot get a positive response (Is the IP address pool depleted?).
If performing these checks does not yield the answer, use a sniffer trace to determine the
specific problem.
Until the port on the gateway has an IP address, TFTP server address, and default gateway IP
address, the port will not register with Cisco CallManager.
Using the show port <mod> command will produce a status of whether the switch sees the
ports as active. Under the Status field, if you see “connected meaning,” this means that the
PSTN circuit is connected and that Layer 1 and Layer 2 are up. If you see “unknown” under
“Type,” this is sure sign that the switch does not see the port as active. You should also see
under “CallManagerState” registered or not registered; this status is self explanatory.
Using the show port <mod/port> command will produce a status specific to the port. In this
output, you should see the IP address assigned to the port, Cisco CallManager IP address, TFTP
IP address, and the Gateway IP address; if you do not see these, there is a configuration error.
The IP address of Cisco CallManager will appear once the port is registered with Cisco
CallManager. There are five key fields to look for: “status” needs to be connected; the Cisco
CallManager state needs to be registered; the TFTP server IP address needs to be listed; the
gateway IP address needs to be listed; and the “CallManagerState” must show as registered.
The reset <mod/port> command resets the port.
An IP route or default gateway must be set in the switch and sc0 must be set before any port
will register with Cisco CallManager. IP connectivity is required before registration.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Network Infrastructure
4-55
Troubleshooting Common Digital Voice
Connections
This topic discusses the specifics relative to troubleshooting digital voice connections on a
Cisco gateway.
Troubleshooting Common Digital Gateway
Issues
• Troubleshooting digital interfaces
• Start with Layer 1, then work up
– show controllers [t1 | e1] (shows error)
– show isdn status (Layers 2 and 3 status)
– debug q921/q931 (Layers 2 and 3 activity)
– debug vpm signal
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—4-18
The first things to examine when troubleshooting a T1 or E1 connection are the physical layer
statistics. Use the command show controllers [t1 | e1] to show all of the T1 or E1 interfaces on
the Cisco IOS gateway. You can specify a particular port at the end of the command to view the
statistics for a single port.
You can use the show controller t1 command to troubleshoot digital interface issues. The
show command has key information that can be used for troubleshooting, such as the
following:
Information to troubleshoot physical layer and data link layer problems
Local or remote alarm information, if any, on the T1 line
Use the show controller command to see if there are alarms or errors displayed by the
controller. To see if the framing, line coding, and slip seconds error counters are increasing,
execute the show controller t1 command repeatedly.
The alarms and procedures to correct them are addressed in this section. After each step, run
the show controller t1 command to see if any alarms occur.
Receive Alarm Indication Signal (Blue):
4-56
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
A receive (Rx) alarm indication signal (AIS) means that there is an alarm occurring on the line
upstream from the equipment that is connected to the port. The AIS failure is declared when an
AIS defect is detected at the input side of the circuit and still exists after the Loss of Frame
(LOF) failure is declared (caused by the unframed nature of the “all-ones” signal). The AIS
failure is cleared when the LOF failure is cleared.
To correct Rx AIS errors, complete the following steps:
Step 1
Check the show controller t1 [slot/port] command output to see if the framing
format configured on the port matches the framing format of the line. If not, change
the framing format on the controller to match the line.
To change the framing format, use the framing {SF | ESF} command in controller
configuration mode. For example:
%:GSRJMKYVIXIVQMREP
)RXIVGSRJMKYVEXMSRGSQQERHWSRITIVPMRI)RH[MXL'280>
%: GSRJMK GSRXVSPPIVX
%: GSRJMKGSRXVSPPI JVEQMRKIWJ
If you are seeing errors in the show controller command output, double check and
triple check the clocking, framing, and line coding configuration.
Step 2
Contact your service provider to check for an incorrect configuration within the
telco.
Receive Remote Alarm Indication (Yellow):
An Rx remote alarm indication (RAI) means that the far-end equipment has a
problem with the signal it is receiving from the upstream equipment.
For Super Frame (SF) links, the far-end alarm failure is declared when bit 6 of
all of the channels has been 0 for at least 335 ms. The failure is cleared when bit
6 of at least 1 channel is not 0 for a period usually less than 1 second and always
less than 5 seconds. The far-end alarm failure is not declared for SF links when a
Loss of Signal (LOS) is detected.
For Extended Superframe (ESF) links, the far-end alarm failure is declared if the
yellow alarm signal pattern occurs in at least 7 out of 10 contiguous 16-bit
pattern intervals. The failure is cleared if the yellow alarm signal pattern does
not occur in 10 contiguous 16-bit signal pattern intervals.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Network Infrastructure
4-57
To correct Rx RAI errors, complete the following steps:
Step 1
Insert an external loopback cable into the port. To create a loopback plug, do the
following things:
Use wire cutters to cut a working RJ-45/RJ-48 cable that is 5 inches long with a
connector attached.
Strip the wires.
Twist the wires from pin 1 and pin 4 together.
Twist the wires from pin 2 and pin 5 together.
The pins on an RJ-45/RJ-48 jack are numbered from 1 through 8. With the metal
pins facing toward you, pin 1 is the left-most pin.
Step 2
Use the show controller t1 command in EXEC mode to see if there are any alarms.
If you do not see any alarms, the local hardware is probably in good condition. In
that case, complete the following steps:
Step 3
Check the cabling. Ensure that the cable between the interface port and the T1
equipment of the service provider or T1 terminal equipment is connected correctly.
Ensure that the cable is connected to the correct ports. Correct the cable connections
if necessary.
Step 4
Check the cable integrity by looking for breaks or other physical abnormalities in the
cable. Ensure that the pinouts are set correctly. Replace the cable if necessary.
Step 5
Check the settings at the remote end and verify that they match your port settings.
If the problem persists, contact your service provider or attempt the following:
Remove the loopback plug and reconnect your T1 line.
Check the cabling.
Power cycle the gateway.
Connect the T1 line to a different port. Configure the port with the same settings as the line.
If the problem does not persist, the fault lies with the port. In this case, complete the
following steps:
—
Reconnect the T1 line to the original port.
—
Perform a hardware loop test.
If the hardware loopback does not bring the line up, the gateway module needs to be replaced.
Transmit Sending Remote Alarm (Red):
A red alarm is declared when the channel service unit (CSU) cannot synchronize with the
framing pattern on the T1 line.
4-58
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
To correct the transmitter from sending remote alarms, complete the following steps:
Step 1
Ensure that the framing format configured on the port matches the framing format of
the line. If not, change the framing format on the controller to match the format of
the line.
Step 2
Check the settings at the remote end and ensure that they match your port settings.
Step 3
Contact your service provider.
Transmit Alarm Indication Signal (Blue):
To correct transmit (Tx) AIS errors, complete the following steps:
Ensure that the framing format configured on the port matches the framing format of the
line. If not, change the framing format on the controller to match the format of the line.
Power cycle the gateway.
Connect the T1 line to a different port. Configure the port with the same settings as the line.
If the problem persists, complete the following steps:
Step 1
Perform a hardware loop test.
Step 2
Replace the T1 controller card.
Step 3
Contact the Cisco TAC about your problem to have the T1 controller card replaced.
Transmit Remote Alarm Indication (Yellow):
A Tx RAI at a digital service level 1 (DS1) interface means that the interface has a problem
with the signal that it is receiving from the far-end equipment.
To correct Tx RAI errors, complete the following steps:
Step 1
Check the settings at the remote end to ensure that they match your port settings.
Step 2
A Tx RAI is accompanied by another alarm. This alarm indicates the problem that
the T1 port/card is having with the signal from the far-end equipment. Troubleshoot
that condition to resolve the Tx RAI error.
As soon as you have verified physical layer connectivity, you can proceed to troubleshoot the
higher layers.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Network Infrastructure
4-59
Troubleshooting Common Digital Gateway
Issues (Cont.)
• T1/ISDN PRI
– show controller [t1 | e1] (Layer 1)
– show isdn status
– debug q921 (Layer 2)
– debug q931 (Layer 3)
• T1/CAS
– show controller (Layer 1)
– show voice all summary
– debug vpm signal
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—4-19
T1/ISDN PRI
One of the most useful commands for troubleshooting an ISDN PRI is the show isdn status
command. This command offers you a status of each layer: physical layer (Layer 1), data link
layer (Layer 2) or D-channel layer, followed by Layer 3. Here is an example of an output of the
show isdn status command:
PRI circuit up and processing calls:
%:WLS[MWHRWXEXYW
+PSFEP-7(27[MXGLX]TI!TVMQEV]IWW
-7(27IVMEPMRXIVJEGI
HWPMRXIVJEGI-7(27[MXGLX]TI!TVMQEV]IWW
0E]IV7XEXYW
%'8-:)
0E]IV7XEXYW
8)-!'IW!7%4-!7XEXI!
1908-40)C*6%1)C)78%&0-7,)(
0E]IV7XEXYW
%GXMZI0E]IV'EPP W %GXMZEXIHHWP''&W!
''&GEPPMH!(WETM!GIW!&GLER!GEPPX]TI!(%8%
''&GEPPMH!(WETM!GIW!&GLER!GEPPX]TI!(%8%
''&GEPPMH!(%WETM!GIW!&GLER!GEPPX]TI!(%8%
''&GEPPMH!()WETM!GIW!&GLER!GEPPX]TI!(%8%
''&GEPPMH!(*WETM!GIW!&GLER!GEPPX]TI!(%8%
Layer 1 Status: ACTIVE. This means that the physical layer is up, and that clocking
framing line coding are all working. If the Layer 1 status was listed as DEACTIVATED,
you would need to go back to the show controllers command and troubleshoot from there.
4-60
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Layer 2 Status: State = MULTIPLE_FRAME_ESTABLISHED. This indicates that the D
channel is up. If the D channel is not coming up, verify your ISDN switch type and
network/user configuration. Every PRI should have one side as the network side and the
other side as the user side. When you connect to the PSTN, almost always the PSTN is the
network side. When integrating with a PBX, you must select either network or user side.
Note
Cisco IOS software does not support the network side on all interfaces. Network side PRI is
currently only supported on primary-ni, primary-qsig, and primary net5 in Cisco IOS Release
12.2(11)T.
If after checking the switch type and network/user side configuration you still are not seeing
MULTIPLE_FRAME_ESTABLISHED, run the q921 debug commands The debug isdn
q921 command displays data link layer (Layer 2) access procedures that are occurring at the
gateway on the D channel.
Note
If the debug isdn q921 command is turned on and you do not receive any debug outputs,
first check and make sure you have enabled the terminal monitor command. Then try to
reset the controller or the D channel to get debug outputs. You can use the commands clear
controller t1 x or clear interface serial x:23 to reset the line.
Layer 3 Status: 5 Active Layer 3 Call(s). This indicates traffic on the bearer channels (B
channels) and traffic set up by the D channel.
The debug isdn q921 command can be used to troubleshoot D-channel issues. This debug
command is useful when troubleshooting ISDN Layer 2 signaling problems. The debug isdn
q921 command displays data link layer (Layer 2) access procedures that are occurring at the
gateway on the D channel.
Once you have enabled the debug isdn q921 command, and if Layer 2 is stable, the gateway
and PSTN should start synchronizing with each other. There will be Set Asynchronous
Balanced Mode Extended (SABME) messages that appear in the debug command output.
These messages indicate that Layer 2 is trying to initialize. Either side can send the message
and try to initialize with the other side. If the gateway receives the SABME message, it should
send back an Unnumbered Acknowledge frame (UAf). The gateway will then change the Layer
2 status to MULTIPLE_FRAME_ESTABLISHED. The following is an example of SAMBE
and UAf messages:
*Apr 12 04:14:43.967: ISDN Se0:23: RX <- SABMEp c/r=1 sapi=0 tei=0
*Apr 12 04:14:43.971: ISDN Se0:23: TX -> UAf c/r=1 sapi=0 tei=0
If the switch receives and recognizes the UAf frame, both devices will synchronize, and
periodic keepalives will be exchanged between the gateway and the PSTN ISDN switch. These
messages are in the form of Receiver Ready poll (RRp) and Receiver Ready final (RRf)
messages. These keepalives are seen 10 seconds apart and ensure that both sides are able to
communicate with each other. Here is an example:
*Apr 12 05:19:56.183: ISDN Se0:23: RX <- RRp sapi=0 tei=0 nr=18
*Apr 12 05:19:56.183: ISDN Se0:23: TX -> RRf sapi=0 tei=0 nr=18
*Apr 12 05:20:06.247: ISDN Se0:23: RX <- RRp sapi=0 tei=0 nr=18
*Apr 12 05:20:06.247: ISDN Se0:23: TX -> RRf sapi=0 tei=0 nr=18
*Apr 12 05:20:16.311: ISDN Se0:23: RX <- RRp sapi=0 tei=0 nr=18
*Apr 12 05:20:16.311: ISDN Se0:23: TX -> RRf sapi=0 tei=0 nr=18
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Network Infrastructure
4-61
PRI with D channel not coming up:
-7(27IVMEPMRXIVJEGI
HWPMRXIVJEGI-7(27[MXGLX]TI!TVMQEV]IWW
0E]IV7XEXYW
%'8-:)
0E]IV7XEXYW
8)-!'IW!7%4-!7XEXI!8)-C%77-+2)(
0E]IV7XEXYW
%GXMZI0E]IV'EPP W %GXMZEXIHHWP''&W!
8LI*VII'LERRIP1EWO\*****
Total Allocated ISDN CCBs = 5
There are times when the D channel does not come up correctly and stays in the
TEI_ASSIGNED state, or Layer 2 bounces up and down. This could be caused by one-way
transmission or missed keepalive packets. Once either side misses four consecutive keepalives,
it will try to reinitialize the Layer 2 link. This restart is done by resending the SABME
message. When this happens, you need to find out if those keepalives are actually placed on the
wire and if one side is not responding back to the keepalives once it receives them.
To isolate the problem, use the debug isdn q921 and show interface serial x:23 commands
and perform the following steps on the gateway and with the T1 service provider (telco):
Type the show interface serial x:23 command several times and ensure that the output
counter does increment and that there are no I/O drops or errors.
Create a T1 loopback plug and then plug it into the T1 port that you are troubleshooting.
The debug isdn q921 command output should show that you sent out the SABME message
and received back the following message:
6< &%(*6%1) \* 0MRIQE]FIPSSTIH
If no debugs appear, perform a shutdown and no shutdown command on the corresponding T1
controller.
The BAD FRAME message indicates that the gateway is performing correctly. The gateway
sends out the SABME packet and, because the message is looped back to the gateway, it
receives the same SABME message it just sent out it. The gateway will mark it as a BAD
FRAME and present the error message stating that the line may be looped. Because this is the
expected behavior for the looped circuit, the problem lies within the telco ISDN switch or
within the cabling from the demarcation point to the telco switch.
If the line is looped back and the gateway sends out SABME messages but does not get them
back, there might be a problem with the physical hardwire loopback plug or the gateway
interface itself.
If the gateway cannot ping itself, there might be a hardware problem with the T1 controller. If
so, you should call the Cisco TAC for assistance.
Once you have isolated and tested the gateway and the T1 ports and have verified that they are
working, you need to engage the telco for further troubleshooting.
Once the D channel is up, use the debug isdn q931 command and you will see Q.931 messages
along with all of the information elements. Most information elements are decoded so that you
can see the status of calls in plain text. You will see the called and calling numbers, numbering
plan, and type decoded along with the PIs information element.
4-62
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
T1/CAS
Troubleshooting CAS is rather straightforward. To debug T1 CAS in Cisco IOS software use
the debug vpm signal command on the gateway. You will be able to see the state of each DS0
as the DS0s are being used. For example, with a CAS FXS and FXO configuration, you should
be able to see the on-hook, off-hook, and connection states. With a CAS E&M configuration,
you should able to see em_onhook_offhook status. With E&M configured circuits, the message
htsp_process_event indicates a change in the T1 CAS DS0 status.
Here are commands used to troubleshoot T1 CAS:
show controllers
show voice call summary
Additional commands presented in this lesson
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Network Infrastructure
4-63
Signaling Protocol Configuration
This topic discusses the specifics related to identifying configuration issues on a gateway.
Signaling Protocol Configuration: H.323
A Common H.323 T1/ISDN PRI and CAS Configuration
controller T1 2/0
framing esf
linecode b8zs
pri-group timeslots 1-24
!
Controller T1 2/1
framing esf
linecode b8zs
ds0-group 0 timeslots 1-24 type e&m-wink-start
!
interface Serial2/0:23
no ip address
no logging event link-status
isdn switch-type primary-ni
isdn incoming-voice voice
no cdp enable
!
interface Serial2/1:23
no ip address
no logging event link-status
isdn switch-type primary-ni
isdn incoming-voice voice
no cdp enable
© 2004 Cisco Systems, Inc. All rights reserved.
voice-port 2/0:23
!
!
voice-port 2/1:0
!
!
dial-peer voice 3000 voip
destination-pattern 30..
session target ipv4:10.10.10.1
codec g711ulaw
!
!
dial-peer voice 12 pots
destination-pattern 1T
direct-inward-dial
prefix 1
!
!
dial-peer voice 22 pots
destination-pattern 011T
port 2/1:23
prefix 011
IPTT v4.0—4-20
This is a sample configuration of a High Density Voice Network Module (NM-HDV) with a
two-port voice WAN interface card (VWIC), serving as a H.323 voice gateway for a Cisco
CallManager. This gateway has two T1 connections to the PSTN/PBX, one through a PRI and
the second one through a T1/CAS circuit.
Common misconfigurations:
interface serial x/x:23isdn switch-type: This command overrides the ISDN switch type
command entered in global configuration mode.
ds0-group 0 timeslots 1-24 type e&m-wink-start: For PRI integration under the
controller interface, use the pri-group command.
4-64
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Signaling Protocol Configuration: MGCP
A Common MGCP T1/ISDN PRI Backhaul Configuration
mgcp
mgcp call-agent 10.10.10.1 2427 service-type mgcp version 0.1
mgcp dtmf-relay voip codec all mode out-of-band
mgcp rtp unreachable timeout 1000 action notify
mgcp modem passthrough voip mode cisco
mgcp sdp simple mgcp package-capability rtp-package
mgcp package-capability sst-package
no mgcp timer receive-rtcp
no mgcp explicit hookstate
!
ccm-manager mgcp
ccm-manager config server 10.10.10.1
ccm-manager config
!
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—4-21
This is a sample configuration of an NM-HDV with a two-port VWIC, serving as MGCP voice
gateway for Cisco CallManager. This gateway has one T1/ISDN PRI connection to the
PSTN/PBX.
Here are common misconfigurations:
ccm-manager mgcp: This command allows Cisco CallManager to obtain control over the
gateway using MGCP. Without this command, Cisco CallManager simply cannot send
MGCP commands to the gateway.
ccm-manager mgcp <ip address>: This command points the gateway to the TFTP server
where the MGCP process can get its XML files.
ccm-manager fallback-mgcp: This statement is needed for MGCP fallback to H.323 to
occur.
ccm-manager redundant-host: This statement is used for redundant backup for the
gateway. The address of the host is the backup Cisco CallManager server.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Network Infrastructure
4-65
Signaling Protocol Configuration: MGCP
(Cont.)
!
controller T1 2/0
framing esf
clock source internal
linecode b8zs
cablelength short 133
pri-group timeslots 1-24 service mgcp
!
interface Serial2/0:23
no ip address
no logging event link-status
isdn switch-type primary-ni
isdn protocol-emulate network
isdn incoming-voice voice
isdn T310 10000
isdn bind-l3 ccm-manager
no cdp enable
© 2004 Cisco Systems, Inc. All rights reserved.
!
voice-port 2/0:23
!
!
dial-peer cor custom
!
dial-peer voice 9991023 pots
application mgcpapp
port 2/0:23
IPTT v4.0—4-22
Here are common misconfigurations:
pri-group timeslots 1-24 service mgcp: MGCP control over the PRI is needed, and this
command allows that control to take place. Another common error is to have the DS0
group defined for PRI services. DS0 groups are for T1/CAS.
application mgcpapp: This command is needed so that Cisco CallManager can register
and manage this endpoint.
isdn bind-l3 ccm-manager: This command is required for backhauling Layer 3 (D
channel) signaling to Cisco CallManager.
4-66
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Dial Peers and Digit Analysis
This topic discusses the specifics relative to dial peers and digit analysis.
Voice over IP Components
Call Leg 1 for
POTS Dial-Peer 1
Call Leg 2 for
VoIP Dial-Peer 2
Source
Call Leg 4 for
POTS Dial-Peer 4
Destination
IP WAN
Dial-Peer 1
R4
Dial-Peer 3
Call Leg 3 for
VoIP Dial-Peer 3
R1
Dial-Peer 2
Dial-Peer 4
Four call legs are necessary for an end-to-end telephony
conversation, two configured in each router.
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—4-24
A voice call over a packet network is segmented into discrete call legs that are associated with
dial peers. (A dial peer is associated with each call leg.) A call leg is a logical connection
between two voice gateways or between a voice gateway and an IP telephony device, such as a
Cisco CallManager server.
Cisco IOS software uses two different types of dial peers. The two different types and their
characteristics are as follows:
POTS: POTS dial peers define the characteristics of a traditional telephony network
connection. POTS dial peers map a dial string to a specific voice port—FXS, FXO, or
E&M—on the local voice gateway. These voice ports normally connect to analog
telephones, fax machines, analog PSTN connections, or PBX connections.
Voice-network dial peers: Voice-network (Voice over X technology) dial peers define the
attributes of a packet voice-network connection. Voice-network dial peers map a dial string
to a remote network device. Some examples of these remote network devices include the
following:
—
Destination voice gateway
—
Cisco CallManager
—
H.323 gatekeeper
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Network Infrastructure
4-67
The specific type of voice-network dial peer depends on the packet network technology. Dial
peer technologies include the following:
VoIP: This dial peer is mapped to the IP address, DNS name, or server type of the
destination VoIP device that terminates the call. VoIP applies to all VoIP protocols, such as
H.323, session initiation protocol (SIP), and MGCP.
Voice over Frame Relay (VoFR): This dial peer is mapped to the data-link connection
identifier (DLCI) of the interface that the call uses to exit the router.
Voice over ATM (VoATM): This dial peer is mapped to the ATM virtual circuit for the
interface that the call uses to exit the router.
Note
4-68
You can also apply most of the troubleshooting techniques for VoIP dial peers to
troubleshooting VoFR and VoATM dial peers.
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Incoming and Outgoing Dial Peers
H.323 Gateway
Dial-Peer Matching
IP
VoIP
POTS
H.323 Gateway
PBX
1
POTS
2
3
IP
VoIP
Analog Phone
Cisco CallManager
The reverse is TRUE.
IP
POTS
POTS
PSTN
CME/SRST
IP Phone
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—4-25
For all calls going through a Cisco router, Cisco IOS software associates one dial peer to each
call leg. Dial peers are associated to main types, according to the type of call leg with which
they are associated, as follows:
POTS dial peers are associated with traditional TDM telephony call legs.
VoIP dial peers are associated with IP call legs.
The call flow examples for this figure are as follows:
Call 1 is for another H.323 gateway across the IP network to a traditional PBX connected
to the router. For this call, an incoming VoIP dial peer and outgoing POTS dial peer are
selected by the router.
Call 2 is for an analog phone connected to an FXS port on the router to a Cisco
CallManager cluster across the IP network. For this call, an incoming POTS dial peer and
an outgoing VoIP dial peer are selected by the router.
Call 3 is for an IP Phone controlled by Cisco CallManager Express or SRST to a PSTN
interface on the router. For this call, an automatically generated POTS dial peer (generated
by an ephone command) and outgoing POTS dial peer are selected.
Dial peers are used for both inbound and outbound call legs. An inbound call leg originates
when an incoming call comes to the router. An outbound call leg originates when an outgoing
call is placed from the router.
The router selects a dial peer for a call leg by matching the string that is defined by using the
answer-address, destination-pattern, or incoming called-number commands in the dial-peer
configuration.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Network Infrastructure
4-69
The following information elements appear in the call setup message:
Called number: This element (Dialed Number Identification Service [DNIS]) is the calldestination dial string that is derived from the ISDN setup message or CAS DNIS.
Calling number: This element (automatic number identification [ANI]) is a number string
that represents the origin of the call that is derived from the ISDN setup message or CAS.
Voice port: This element represents the POTS physical voice port.
When the voice gateway receives a call setup request, it performs a dial-peer match for the
incoming call to facilitate routing the call to different session applications. Although the voice
gateway does not match digit by digit, it uses the full digit string received in the setup request
for matching against configured dial peers.
The voice gateway selects an inbound dial peer by matching the information elements in the
setup message with the dial-peer attributes. It matches these items in the following order:
Called number (DNIS) with incoming called number: First, the voice gateway attempts
to match the called number of the call setup request with the configured incoming called
number of each dial peer. Because call setups always include DNIS information, Cisco
recommends using the incoming called-number command for inbound dial-peer
matching. This attribute has matching priority over the answer-address and destinationpattern commands. If no match is found, the voice gateway attempts to match the calling
number with the answer address.
Calling number (ANI) with answer address: The voice gateway attempts to match the
calling number of the call setup request with the answer address of each dial peer. This
attribute is useful in situations where you want to match calls based on the calling number.
If no match is found, the voice gateway attempts to match the destination pattern.
Calling number (ANI) with destination pattern: The voice gateway attempts to match
the calling number of the call setup request to the destination pattern of each dial peer. If no
match is found, the voice gateway attempts to match the configured dial-peer port to the
voice port.
Configured dial peer port to the voice port: The voice gateway attempts to match the
configured dial-peer port to the voice port associated with the incoming call. If multiple
dial peers have the same port configured, the voice gateway matches the first dial peer
added to the configuration.
Note
Configuring the dial-peer port to the voice port is not applicable to voice and/or dial platforms
including Cisco AS5300, AS5350, AS5400, AS5800, and AS5850. If the voice gateway does
not use all of the first three methods, it matches dial-peer 0 and treats the call like a dial
modem call. Customers may receive a modem tone instead of a dial tone for inbound calls.
If no match is found from using these methods, the voice gateway uses the default dial-peer 0
(pid:0).
Cisco IOS software matches only one of these conditions. It is not necessary to configure all of
the attributes in the dial peer or to match every attribute to the call setup information. The voice
gateway needs to meet only one condition to select a dial peer. Cisco IOS software stops
searching when it matches one dial peer.
4-70
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
The longest prefix match rule also applies while the voice gateway performs each step. If the
voice gateway locates multiple matches at each step, it uses the number with the longest
explicit match.
If Cisco IOS software does not match an incoming dial peer by the voice gateway, it
automatically routes the inbound call leg to a default dial peer (POTS or voice network). This
default dial peer is referred to as dial-peer 0.
Dial-peer 0 has a default configuration that you cannot change. The default dial peer will fail to
negotiate nondefault capabilities, services, and/or applications such as the following:
Nondefault voice-network capabilities: dtmf-relay, no vad, and other commands
Direct inward dialing (DID)
Toolkit Command Language (TCL) applications
Dial-peer 0 for inbound VoIP peers has the following configuration:
ER]GSHIG
MTTVIGIHIRGI
ZEHIREFPIH
RSVWZTWYTTSVX
JE\VEXIZSMGI
Dial-peer 0 for inbound POTS peers has the following configuration:
RSMZVETTPMGEXMSR
RSHMVIGXMR[EVHHMEPMRK
For matching outbound dial peers, the voice gateway uses the dial peer command destinationpattern called number. Based on the type of dial peer, you will require one of the following
commands to forward a call:
POTS dial peers: Use the port command to forward the call.
Voice-network dial peers: Use the session target command to forward the call.
When matching outbound peers, there are two cases to consider: DID and non-DID. An
example of an incoming dial peer configured with DID is as follows:
HMEPTIIVZSMGITSXW
MRGSQMRKGEPPIHRYQFIV
ZSMGITSVX(
HMVIGXMR[EVHHMEP
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Network Infrastructure
4-71
On DID calls (referred to as one-stage dialing), the setup message contains all the digits
necessary to route the call, and the voice gateway should not perform subsequent digit
collection. When the voice gateway searches for an outbound dial peer, it uses the entire
incoming dial string. This matching is variable in length by default. The voice gateway does not
use digit-by-digit matching because, by DID definition, it has already received all of the digits.
To clarify this concept, for example, assume that the DID dial string is 81690. In this case, the
router will match dial-peer 4 and forward the complete dial-string 81690:
HMEPTIIVZSMGIZSMT
HIWXMREXMSRTEXXIVR
WIWWMSRXEVKIXMTZ
HMEPTIIVZSMGIZSMT
HIWXMREXMSRTEXXIVR
WIWWMSRXEVKIXMTZ
A non-DID case is referred to as two-stage dialing. If the matched incoming dial peer does not
have DID configured, the voice gateway enters digit collection mode (digits are collected inband). The voice gateway completes outbound dial peer matching on a digit-by-digit basis.
Therefore, the voice gateway checks for dial-peer matches after receiving each digit and then
routes the call after making a complete match. To clarify this concept, assume that the dial
string is 81690; immediately after the router receives the digit, 6, it will match dial-peer 3 and
route the call (forwarding only the digits 816):
HMEPTIIVZSMGIZSMT
HIWXMREXMSRTEXXIVR
WIWWMSRXEVKIXMTZ
HMEPTIIVZSMGIZSMT
HIWXMREXMSRTEXXIVR
WIWWMSRXEVKIXMTZ
Now, assume that dial-peer 3 is configured for wildcard matching:
HMEPTIIVZSMGIZSMT
HIWXMREXMSRTEXXIVR
WIWWMSRXEVKIXMTZ
HMEPTIIVZSMGIZSMT
HIWXMREXMSRTEXXIVR
WIWWMSRXEVKIXMTZ
In this case, the longest-prefix rule will apply, and the voice gateway matches dial-peer 4 for
the outbound call leg.
Note
4-72
The dial-peer attribute destination-pattern functions differently when applied to inbound or
outbound call legs. For inbound dial peers, it matches the destination-pattern against the
calling number (ANI string). For outbound dial peers, it matches the destination-pattern
against the called number (DNIS string).
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Verifying Dial Peers
VSYXIVWLS[HMEPTIIVZSMGIWYQQEV]
HMEPTIIVLYRX
%(46)4%77
8%+8=4)1-234)646)*-<()784%88)62*)68,697)778%6+)84368
TSXWYTYT
ZSMTYTYTW]WXMTZ
VSYXIVWLS[HMEPTIIVZSMGIWYQQEV]
HMEPTIIVLYRX
%(46)4%77
8%+8=4)1-234)646)*-<()784%88)62*)68,697)778%6+)84368
TSXWYTYT
ZSMTYTYTW]WXMTZ
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—4-26
The operational status of a dial peer must be administratively up and valid for the voice
gateway to match it. To be considered operational, dial peers must meet one of the following
conditions:
The destination-pattern command is configured, and a voice port or session target is
configured (outbound dial peers).
The incoming called-number command is configured (inbound dial peers).
The answer-address command is configured (inbound dial peers).
To quickly verify the status of all configured dial peers, use the show dial-peer voice
summary command. To view detailed information about a dial peer, use the show dial-peer
voice <tag> command.
To validate the dial-peer configuration, use the show dialplan number <digit_string>
command. This command displays the dial peer, which the voice gateway matches by a string
of digits. If it is possible to match multiple dial peers, they will appear in the same order as they
are matched. The output of this command will look like this:
:+WLS[HMEPTPERRYQFIV
1EGVS)\T
:SMGI3ZIV-T4IIV
MRJSVQEXMSRX]TI!ZSMGI
XEK!HIWXMREXMSRTEXXIVR!D ERW[IVEHHVIWW!D TVIJIVIRGI!
KVSYT!%HQMRWXEXIMWYT3TIVEXMSRWXEXIMWYT
MRGSQMRKGEPPIHRYQFIV!D GSRRIGXMSRWQE\MQYQ!
YRPMQMXIH
ETTPMGEXMSREWWSGMEXIH
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Network Infrastructure
4-73
X]TI!ZSMTWIWWMSRXEVKIX!DMTZ XIGLRSPSK]TVIJM\
MTTVIGIHIRGI!9(4GLIGOWYQ!HMWEFPIH
WIWWMSRTVSXSGSP!GMWGSVIUUSW!FIWXIJJSVX
EGGUSW!FIWXIJJSVX
HXQJVIPE]!GMWGSVXT
JE\VEXI!ZSMGITE]PSEHWM^I!F]XIW
GSHIG!KVTE]PSEHWM^I!F]XIW
)\TIGXJEGXSV!-GTMJ!WMKREPMRKX]TI!GEW
:%(!IREFPIH4SSV53:8VET!HMWEFPIH
'SRRIGX8MQI!'LEVKIH9RMXW!
7YGGIWWJYP'EPPW!*EMPIH'EPPW!
%GGITXIH'EPPW!6IJYWIH'EPPW!
0EWX(MWGSRRIGX'EYWIMW
0EWX(MWGSRRIGX8I\XMWRSVQEPGEPPGPIEVMRK
0EWX7IXYT8MQI!
1EXGLIH(MKMXW
8EVKIXMTZ
If you are having trouble connecting a call, and you suspect the problem is associated with the
dial-peer configuration, you can try to resolve the problem by performing these tasks:
4-74
Step 1
Ping the associated IP address to confirm connectivity (as you learned in this
lesson).
Step 2
Use the show dial-peer voice command to verify that the operational status of the
dial peer is up.
Step 3
Use the show dialplan number command on the local and remote routers to verify
that the dialed digit string is correctly matched on both sides of the connection.
Step 4
If you have configured a number expansion, use the show num-exp command to
check that the partial number on the local router maps to the correct full E.164
telephone number on the remote router.
Step 5
If you have configured a codec value in the dial peer, a problem can occur if the
VoIP dial peers on either side of the connection have incompatible codec values.
Verify that both VoIP peers have the same codec value.
Step 6
If you are still unable to locate the problem, troubleshoot the VoIP subsystem of a
Cisco router, the call setup, and call control phases.
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Router Call Flow and Call Control
This topic discusses the specifics relative to router call flow and call control.
Router Call Flow
CLI
SNMP
Session
Application
Dial Plan
Mapper
Call Control API
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—4-27
This figure shows a high-level overview of how the gateway processes calls. It does not
provide detailed information about internal flows, but it provides a general idea of what is
going on inside the gateway for reference purposes. You will see acronyms and references to
internal logic in the output of some of the debug commands, such as call control application
programming interface (API).
These definitions explain the functions of the main components displayed in the gateway call
flow diagram:
Call control API (CCAPI): Three clients make use of the call control API: CLI, Simple
Network Management Protocol (SNMP) agent, and the session application. The main
functions of the CCAPI are as follows:
—
Identify the call legs
—
Decide which session application takes the call
—
Invoke the packet handler
—
Conference the call legs together
—
Start recording call statistics
Session application and Dial Plan Mapper: The session application uses the dial plan
mapper to map a number to a dial peer (local POTS or remote VoIP). The dial plan mapper
uses the dial-peer table to locate active dial peers.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Network Infrastructure
4-75
Telephony and VoIP Service Provider Interface (SPI): The telephony SPI
communicates with analog (FXS, FXO, and E&M) and digital (ISDN, Q Signaling [QSIG],
E&M) POTS dial peers. The VoIP SPI is the specific interface to the VoIP peers.
Telephony or domain-specific part drivers, or both, deliver services to the telephony SPI,
while the VoIP SPI relies on session protocols.
4-76
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Telephony Interface Architecture
CCAPI
Application
Request
Setup/Release Request
Voice Processor
Module
Setup/Release
Ind/Release
Application
Response
Voice
Telephony
Service Provider
DSP Resource Manager
DSP
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—4-28
This diagram shows the architecture of Cisco gateway telephony building blocks and how they
interact with each other. This list describes the functions and definitions of the main diagram
components:
CCAPI: This component is a software entity that establishes, terminates, and bridges call
legs.
Voice telephony service provider: This component is a Cisco IOS process that services
requests from the CCAPI and formulates the appropriate requests to the digital signal
processors (DSPs) or the voice port module (VPM).
VPM: This component is in charge of bridging and coordinating signaling processes
between the telephony port signaling state machine, the DSP Resource Manager (DSPRM),
and the voice telephony service provider.
DSPRM: This component provides interfaces that the voice telephony service provider
uses to send and receive messages to and from the DSPs.
Packet handler: This component forwards packets between the DSPs and the peer call
legs.
Call peer: This component is the opposite call leg, such as another telephony voice
connection (POTS), VoFR, VoATM, or a VoIP connection.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Network Infrastructure
4-77
Verifying On-Hook and Off-Hook Signaling
101110111001
• debug vpm signal
• debug vpm spi
• debug vpm dsp
• debug vpm all
• debug vpm port
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—4-29
These commands debug the VPM telephony interface and verify that on-hook and off-hook
signaling are functioning correctly:
debug vpm signal: This command collects debug information for signaling events and is
useful for resolving problems with signaling to a PBX.
debug vpm spi: This command traces the VPM SPI as it interfaces with CCAPI. It displays
information, the handling of each network indication, and application request.
debug vpm dsp: This command displays messages from the DSP on the VPM to the
gateway. You can use this command if you suspect that the VPM is not functional. It
determines if the VPM is responding to off-hook indications and evaluates the timing for
signaling messages from the interface.
debug vpm all: This EXEC command enables all of the debug vpm commands: debug
vpm spi, debug vpm signal, and debug vpm dsp.
debug vpm port: This command limits the debug output to a particular port.
This example shows debug vpm dsp command output only for port 1/0/0:
HIFYKZTQTSVX
HIFYKZTQHWT
4-78
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Shown here is sample output from the debug vpm signal command:
+EXI[E]HIFYKZTQWMKREP
*<7TSVXKSIWJVSQSRLSSOXSSJJLSSOWXEXI
LXWTCTVSGIWWCIZIRX?AJ\WPWCSRLSSOCSJJLSSO
LXWTCWIXYTCMRH
1EVLXWTCTVSGIWWCIZIRX?A
7IRHMRKVMRKMRKEPIVXXSGEPPIHTLSRI
1EVLXWTCTVSGIWWCIZIRX?A
LXWTCEPIVXCRSXMJ]
1EVLXWTCTVSGIWWCIZIRX?A
)RHSJTLSRIGEPPTSVXKSIWSRLSSO
1EVLXWTCTVSGIWWCIZIRX?A
1EVLXWTCTVSGIWWCIZIRX?A
J\WPWCSJJLSSOCSRLSSO
1EVLXWTCTVSGIWWCIZIRX?A
J\WPWCSJJLSSOCXMQIV
1EVLXWTCTVSGIWWCIZIRX?A
J\WPWCSRLSSOCVIPIEWI
Shown here is sample output from the debug vpm spi command:
+EXI[E]HIFYKZTQWTM
:SMGI4SVX1SHYPI7IWWMSRHIFYKKMRKMWIREFPIH
8LI(74MWTYXMRXSHMKMXGSPPIGXMSRQSHI
1EVHWTCHMKMXCGSPPIGXCSR?A
TEGOIXCPIR!GLERRIPCMH!
TEGOIXCMH!QMRCMRXIVCHIPE]!QE\CMRXIVCHIPE]!
QMQCQEOICXMQI!QE\CQEOI
CXMQI!QMRCFVEOICXMQI!QE\CFVEOICXMQI!
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Network Infrastructure
4-79
Verifying Digit Collection
101110111001
• show dialplan number
• debug vtsp session
• debug vtsp dsp
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—4-30
After you verify that the on-hook and off-hook signaling are working correctly, the next step in
troubleshooting and debugging a VoIP call is to verify that the voice port (digital or analog) is
receiving or sending the correct digits. If the voice port is sending or receiving incomplete or
incorrect digits, it will not match a dial peer, or the switch (central office [CO] or PBX) will not
ring the correct station. Some commands that you can use to verify the digits received and sent
are as follows:
show dialplan number <dial string>: This command shows the dial peers that the voice
port matches when an individual dials a particular telephone number.
debug vtsp session: This command displays information on how the voice port is
processing each network indication and application request, signaling indications, and DSP
control messages.
debug vtsp dsp: This command displays the digits as the voice port receives them.
debug vtsp all: This command enables the following debug voice telephony service
provider commands: debug vtsp session, debug vtsp error, and debug vtsp dsp.
Note
4-80
If you determine that the voice port is not sending or receiving the digits properly, it may be
necessary to use either a digit grabber (test tool) or T1 tester to verify that the voice port is
sending the digits at the correct frequency and timing interval. If the voice port is incorrectly
sending the digits for the switch (CO or PBX), you may need to adjust some values on the
gateway or switch (CO or PBX) so that they match and can interoperate. If you still
experience problems, you should also check the number translation tables in the switch (CO
or PBX) that may add or remove digits.
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Shown here is sample output from the debug vtsp dsp command:
+EXI[E]HIFYKZXWTHWT
:SMGIXIPITLSR]GEPPGSRXVSPHWTHIFYKKMRKMWSR
%'8-32'EPPIVTMGOIHYTLERHWIXERHHMEPIHHMKMXW
8LI(74HIXIGXW(81*HMKMXW(MKMX[EWHIXIGXIH[MXL32
XMQISJQWIG
1EVZXWTCTVSGIWWCHWTCQIWWEKI
17+C8<C(81*C(-+-8C&)+-2HMKMX!
1EVZXWTCTVSGIWWCHWTCQIWWEKI
17+C8<C(81*C(-+-8C3**HMKMX!HYVEXMSR!
1EVZXWTCTVSGIWWCHWTCQIWWEKI
17+C8<C(81*C(-+-8C&)+-2HMKMX!
1EVZXWTCTVSGIWWCHWTCQIWWEKI
17+C8<C(81*C(-+-8C3**HMKMX!HYVEXMSR!
1EVZXWTCTVSGIWWCHWTCQIWWEKI
17+C8<C(81*C(-+-8C&)+-2HMKMX!
1EVZXWTCTVSGIWWCHWTCQIWWEKI
17+C8<C(81*C(-+-8C3**HMKMX!HYVEXMSR!
1EVZXWTCTVSGIWWCHWTCQIWWEKI
17+C8<C(81*C(-+-8C&)+-2HMKMX!
1EVZXWTCTVSGIWWCHWTCQIWWEKI
17+C8<C(81*C(-+-8C3**HMKMX!HYVEXMSR!
Shown here is sample output from the debug vtsp session command:
+EXI[E]HIFYKZXWTWIWWMSR
:SMGIXIPITLSR]GEPPGSRXVSPWIWWMSRHIFYKKMRKMWSR
%'8-32'EPPIVTMGOIHYTLERHWIX
8LI(74MWEPPSGEXIHNMXXIVFYJJIVW:%(XLVIWLSPHWERH
WMKREPPIZIPWEVIWIX
1EVHWTCWIXCTPE]SYX? A
TEGOIXCPIR!GLERRIPCMH!TEGOIXCMH!QSHI!MRMXMEP!
QMR!QE\!JE\CRSQ!
1EVHWTCIGLSCGERGIPPIVCGSRXVSP? A
TEGOIXCPIR!GLERRIPCMH!TEGOIXCMH!JPEKW!\
1EVHWTCWIXCKEMRW? A
TEGOIXCPIR!GLERRIPCMH!TEGOIXCMH!MRCKEMR!
SYXCKEMR!
1EVHWTCZEHCIREFPI? A
TEGOIXCPIR!GLERRIPCMH!TEGOIXCMH!XLVIWL!
EGXCWIXYTCMRHCEGO
1EVHWTCZSMGICQSHI? A
TEGOIXCPIR!GLERRIPCMH!TEGOIXCMH!GSHMRKCX]TI!
ZSMGICJMIPHCWM^I!:%(CJPEK!IGLSCPIRKXL!GSQJSVXCR
SMWI!MRFERHCHIXIGX!HMKMXCVIPE]!
%+'CJPEK!EGXCWIXYTCMRHCEGO HWTCHXQJCQSHI EGXCWIXYTCMRHCEGOTEWWXLVYCQSHI!
RSCEYXSCW[MXGLSZIV!HWTCHXQJCQSHI :874C8
32)C(81*C13()
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Network Infrastructure
4-81
8LI(74MWTYXMRXSZSMGIQSHIERHHMEPXSRIMW
KIRIVEXIH
1EVHWTCGTCXSRICSR? A
TEGOIXCPIR!GLERRIPCMH!TEGOIXCMH!XSRICMH!RCJVIU!
JVIUCSJCJMVWX!JVIUCSJCWIGSRH!EQTCSJCJMVWX!
EQTCSJCWIGSRH!HMVIGXMSR!SRCXMQICJMVWX!
SJJCXMQICJMVWX!SRCXMQICWIGSRH!SJJCXMQICWIGSRH!
4-82
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Verifying Call Setup and Call Control
Phases
101110111001
• debug voice ccapi inout
• debug ip tcp transactions
• debug ip tcp packet
• debug h225 events
• debug h245 events
• debug cch323 h225
• debug cch323 h245
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—4-31
After verifying that voice port signaling is working properly and that the voice port is receiving
the correct digits, the next step is to troubleshoot the call setup and call control phases. These
factors explain the complexities of call control debugging:
Cisco VoIP gateways use H.323 signaling to complete calls. H.323 is made up of three
layers of call negotiation and call establishment: H.225, H.245, and H.323. These protocols
use a combination of TCP and UDP to set up and establish a call.
End-to-end VoIP debugging will show a number of Cisco devices, and problems with any
intermediary Cisco device can cause a call to fail.
End-to-end VoIP debugging is extensive and can create a large amount of debug output.
The primary command to debug end-to-end VoIP calls is debug voip ccapi inout.
If the call is failing and you believe that the cause is in the VoIP portion of the call setup, you
may need to look at the H.225 or H.245 TCP part of the call setup, as opposed to just the UDP
portion of the H.323 setup.
You can use these commands to debug the H.225 or H.245 call setup:
debug ip tcp transactions and debug ip tcp packet: These commands examine the TCP
portion of the H.225 and H.245 negotiation. They return the IP addresses, TCP ports, and
states of the TCP connections.
debug h225 events: This command examines the H.225 portion of the call negotiation.
Think of this as the Layer 1 part of the three-part H.323 call setup.
debug h245 events: This command examines the H.245 portion of the call negotiation.
Think of this as the Layer 2 part of the three-part H.323 call setup.
debug cch323 h225: This command shows actions involving the H.225 state machine.
debug cch323 h245: This command shows actions involving the H.245 state machine.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Network Infrastructure
4-83
Additional commands that provide useful information for specific situations include the
following:
debug mgcp: This command shows actions involving MGCP-related problems on MGCP
voice gateways.
debug ip rtp packets: This command shows actions involving RTP flows.
show call active voice: This command displays a list of all active voice calls.
show voice call summary: This command displays a brief summary of all active voice
calls.
show voice dsp: This command displays the status of the DSP chips, whether they are in
use or not. If they are in use, this command also determines their codec.
The majority of these debug commands are processor intensive and can cause major problems
in production networks. Cisco recommends that you test these commands in a lab environment
first to determine the implications on a production network. If testing is not possible, use the
debug commands with access lists during low utilization periods to limit their output.
You should run these commands with the time stamps enabled in the log. Enable time-stamping
by adding the service timestamps debug datetime msec and service timestamps log
datetime msec commands in enable mode. The time stamps help to determine the interval of
time between state changes.
4-84
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Survival Remote Site Telephony and Monitoring
This topic discusses the specifics related to SRST, including a definition of SRST, the features
of SRST, how SRST is configured, and how SRST operates.
What Is SRST?
Survivable Remote Site Telephony (SRST)
• Capability in branch office routers for IP telephony
redundancy
• Always available branch IP telephony
(including calls to and from PSTN)
• Ideal for centralized Cisco CallManager
deployment
• Licensed based on number of users at remote site
on Cisco IOS software plus
• Survivability scales up to 480 users dependent
upon platform
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—4-32
Survivable Remote Site Telephony:
Cisco SRST provides Cisco CallManager with fallback support for Cisco IP Phones
attached to a Cisco gateway on your local network. Cisco SRST enables gateways to
provide call-handling support for Cisco IP Phones when they lose connection to remote
primary, secondary, or tertiary Cisco CallManager installations or when the WAN
connection is down.
The SRST feature enables Cisco IOS gateways (3700 and 7200 series) to provide callhandling functions if the Cisco CallManager server at the main site goes down or becomes
unavailable.
Call handling is restored to the central Cisco CallManager server when the link becomes
available again.
SRST is a value-added feature that solves a clear requirement for customers—SRST
provides call control backup for remote sites and enables centralized call processing to give
customers operational and capital savings and to allow them to realize the benefits of
converged Cisco IP communications systems that have remote offices. SRST does not
support IP phones from other vendors because these vendors (Avaya, Nortel Networks,
Mitel Networks, Siemens AG, Alcatel, and so on) use proprietary call control protocols.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Network Infrastructure
4-85
What is SRST? (Cont.)
Hub Site A
Central Cisco
CallManager Cluster
CCM
Voice
Mail
CCM
CCM
primary
secondary
tertiary
IP WAN
PSTN
V
Spoke Site B
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—4-33
Cisco CallManager supports Cisco IP Phones at remote sites attached to Cisco multiservice
gateways across the WAN. Prior to Cisco SRST, when the WAN connection between a
gateway and the Cisco CallManager failed or connectivity with Cisco CallManager was lost for
some reason, the Cisco IP Phones on the network became unusable for the duration of the
failure. Cisco SRST overcomes this problem and ensures that the Cisco IP Phones offer
continuous (although minimal) service by providing call-handling support for Cisco IP Phones
directly from the Cisco SRST gateway. The system automatically detects a failure and uses
Simple Network-enabled Auto Provisioning (SNAP) technology to autoconfigure the branch
office gateway to provide call processing for the Cisco IP Phones that are registered with the
gateway. When the WAN link or connection to the primary Cisco CallManager server is
restored, call handling reverts back to the primary Cisco CallManager server.
When Cisco IP Phones lose contact with primary, secondary, and tertiary Cisco CallManager
servers, they must establish a connection to a local Cisco SRST gateway to sustain the callprocessing capability necessary to place and receive calls. The Cisco IP Phone retains the IP
address of the local Cisco SRST gateway as a default gateway in the Network Configuration
area of the Settings menu. This list supports a maximum of five default gateway entries;
however, Cisco CallManager accommodates a maximum of three entries. When a secondary
Cisco CallManager server is not available on the network, the IP address of the local Cisco
SRST gateway is retained as the standby connection for Cisco CallManager during normal
operation.
When the WAN link fails, calls in progress are sustained for the duration of the call. Calls in
transition and calls that have not yet connected are dropped and must be reinitiated once the
Cisco IP Phones reestablish connection to their local Cisco SRST gateway. Telephone service
remains unavailable from the time that connection to the remote Cisco CallManager server is
lost until the Cisco IP Phone establishes connection to the Cisco SRST gateway.
4-86
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Cisco SRST Compatibility
•SRST Version 3.1
Cisco IOS software 12.3(7)T, Cisco CallManager 3.1, 3.2, 3.3, and 4.0(1)
•SRST Version 3.0
Cisco IOS software 12.3(4)T, 12.2(15)ZJ, Cisco CallManager 3.1, 3.2, 3.3
•SRST Version 2.1
Cisco IOS software 12.2(15)T, 12.2(11)YT, Cisco CallManager 3.1, 3.2, 3.3
•SRST Version 2.02
Cisco IOS software 12.2(13)T, Cisco CallManager 3.05, 3.1, 3.2
•SRST Version 2.01
Cisco IOS software 12.2(11)T, Cisco CallManager 3.05, 3.1, 3.2
•SRST Version 2.0
Cisco IOS software 12.2(8)T, 12.2(2)XT, Cisco CallManager 3.05, 3.1, 3.2
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—4-34
Cisco CallManager 3.2 allows Cisco SRST to be switched on and off for each IP Phone. Cisco
CallManager 3.3 allows you to specify an IP address router by phone.
The Cisco CallManager fallback mode telephone service is available only to those Cisco IP
Phones that are supported by a Cisco SRST router. Other Cisco IP Phones on the network
remain out of service until they reestablish a connection with their primary, secondary, or
tertiary Cisco CallManager server.
Note
Pay particular attention to the version of Cisco CallManager that you are running, because
each version of Cisco CallManager is compatible with a specific set of Cisco SRST releases.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Network Infrastructure
4-87
How SRST Works: Normal Operation
Central Site Cisco
CallManager
PSTN
Real Time
Protocol
Branch Router
1750, 2600, 3600,
7200, Catalyst 4224
IAD2400
WAN
Headquarters
Keepalive: Ensures communication link with Cisco CallManager
Skinny: Protocol for call setup and teardown
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—4-35
The central Cisco CallManager server processes call requests from remote IP Phones .
The IP Phone has full Cisco CallManager functions.
The IP Phone sends keepalives to Cisco CallManager every 30 seconds (keepalive default
at 30 seconds, configurable)
The branch gateway has SRST functionality configured.
The branch gateway has no knowledge of the IP Phones at this point.
Note
4-88
Cisco CallManager fallback mode telephone service is available only to those
Cisco IP Phones that are supported by a Cisco SRST gateway. Other Cisco IP Phones on
the network remain out of service until they are able to reestablish a connection with their
primary, secondary, or tertiary Cisco CallManager.
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
How SRST Works: Network Failure
Central Site Cisco
CallManager
Keepalive to Cisco
CallManager
Branch Router
1750, 2600, 3600, 7200,
Catalyst 4224
IAD2400
PSTN
Real Time
Protocol
X
WAN
Keepalive to Cisco
CallManager
Headquarters
Keepalive (TCP)
Skinny
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—4-36
This figure shows an example of SRST operation in network failure mode.
Branch gateway automatically autoconfigures itself using (SNAP) technology.
IP Phones “rehome” to branch gateway but continue sending keepalive messages to the
central Cisco CallManager.
IP Phones use Skinny Station Protocol to the branch gateway for Enhanced 911 (E911) and
other calls.
The IP Phones display indicates “SRS Telephony Activated.”
Most configured features on IP Phones continue operating while in SRST mode.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Network Infrastructure
4-89
How SRST Works: Network Failure
Repaired
Central Site Cisco
CallManager
PSTN
Real Time
Protocol
Branch Router
1750, 2600, 3600,
7200, Catalyst 4224
IAD2400
WAN
Headquarters
Keepalive: Ensures communication link with Cisco CallManager
Skinny: Protocol for call setup and teardown
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—4-37
The Cisco SRST process on the gateway automatically detects a failure and uses SNAP)
technology to autoconfigure the branch office gateway to provide call processing for the
Cisco IP Phones that are registered with the gateway. When the WAN link or connection to the
primary Cisco CallManager server is restored, call handling reverts back to the primary Cisco
CallManager server.
4-90
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
SRST Configuration: Four Commands
7678 GSRJMK GEPPQEREKIVJEPPFEGO
• Enables Cisco CallManager fallback capability and puts you in a
submenu.
7678 GSRJMK MTWSYVGIEHHVIWW MTEHHVIWW"?TSVX TSVX"A
• Enables router to receive Skinny messages on this particular port. The
default port is 2000.
7678 GSRJMK QE\ITLSRIW
• Defaults to 0. Maximum phones that will be allowed to register.
7678 GSRJMK QE\HR
• Defaults to 0. Maximum number of directory numbers that can be
configured.
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—4-38
The most common application of SRST is to maintain basic IP telephony function to the IP
Phones at the remote branch offices in the event that the WAN link fails or that the Cisco
CallManager server at headquarters is no longer available. In this case, the remote branch
gateway will become active with SRST functions and take over the communication with the IP
Phones. IP Phones are able to call each other and make off-net calls to the PSTN. In addition,
basic phone functions such as hold and transfer are still available.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Network Infrastructure
4-91
There is one global command to configure for SRST: call-manager-fallback. This command
has a series of subcommands. Here are some examples of how to use some of the
subcommands of the call-manager-fallback command:
show call-manager-fallback
6SYXIVWLS[GEPPQEREKIVJEPPFEGO
EGGIWWGSHIJ\S
HIJEYPXHIWXMREXMSRTEXXIVR
HMEPTPERTEXXIVRi
MTWSYVGIEHHVIWWTSVX
OIITEPMZI
QE\ITLSRIW
QE\HR
XVERWJIVTEXXIVR
ZSMGIQEMP
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—4-39
access-code fxo 9
This subcommand associates the outgoing digit 9 with all FXO ports on the gateway. When the
digit 9 is dialed, one of the available FXO ports to the PSTN is seized. The user can then dial
the regular E.164 number to access a number in the PSTN. The advantage of this subcommand
is to select a type of ports rather a single port. Thus when a port is busy, the next available port
of the same type will be accessed.
default-destination 4312
This subcommand defines a default extension for all incoming calls that do not have an
appropriate called number to go to. In this example, when an incoming call from an FXO port
is not able to match to an appropriate called number, the call will be routed to extension 4312,
which in this case may be the receptionist.
transfer-pattern 5253
This subcommand allows the transfer of calls to non-IP Phone numbers. By default, all IP
Phones are allowed as transfer targets. In this example, calls can be transferred to all non-IP
Phone numbers with the prefix 525. The transfer of calls to an undefined number or prefix is
blocked. A maximum of 32 transfer patterns can be entered.
4-92
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
voicemail 35678
This subcommand defines a dial peer of an FXS port for the call to route to when the Messages
button on the IP Phone is pressed. In phase 1, this is supported by means of hanging an
answering machine off an FXS port. In this example, when the Message button on the IP Phone
is pressed, the call is directed to extension 35678, which is an answering machine. The user can
then listen to the messages.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Network Infrastructure
4-93
SRST Monitoring and Maintaining:
Commands
7678
WLS[GEPPQEREKIVJEPPFEGOEPP
• Displays the detailed configuration of all the Cisco IP phones, voice
ports, and dial peers of the Cisco SRST router.
7678
WLS[GEPPQEREKIVJEPPFEGOHMEPTIIV
• Displays the output of the dial peers of the Cisco SRST router.xxx.
7678
WLS[GEPPQEREKIVJEPPFEGOITLSRIHR
• Displays Cisco IP phone destination number.
7678
WLS[GEPPQEREKIVJEPPFEGOZSMGITSVX
• Displays output for the voice ports.
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—4-40
Here are more commands to be used to monitor and manage ephones in an SRST gateway:
show call-manager-fallback all: Displays the detailed configuration of all the
Cisco IP Phones, voice ports, and dial peers of the Cisco SRST gateway.
show call-manager-fallback dial-peer: Displays the output of the dial peers of the Cisco
SRST gateway.
show call-manager-fallback ephone-dn: Displays Cisco IP phone destination number.
show call-manager-fallback voice-port: Displays output for the voice ports.
show ephone dn dn-tag: Displays Cisco IP Phone destination number.
show ephone phone: Displays Cisco IP Phone status.
WLS[ITLSRISJJLSSO Displays Cisco IP Phone status for all phones that are off-hook.
show ephone registered: Displays Cisco IP Phone status for all phones that are currently
registered.
show ephone remote: Displays Cisco IP Phone status for all nonlocal phones (phones that
have no ARP entry).
show ephone ringing: Displays Cisco IP Phone status for all phones that are ringing.
show ephone summary: Displays a summary of all Cisco IP phones.
show ephone telephone-number phone-number: Displays Cisco IP Phone status for a
specific phone number.
show ephone unregistered: Displays Cisco IP Phone status for all unregistered phones.
show ephone-dn summary: Displays a summary of all Cisco IP Phone destination
numbers.
show voice port summary: Displays a summary of all voice ports.
show dial-peer voice summary: Displays a summary of all voice dial peers.
4-94
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting SRST:
SRST system administration guides for Cisco IOS Release 2.1 through Cisco IOS Release 3.1
can be found at the following URL:
http://www.cisco.com/en/US/products/sw/voicesw/ps2169/products_feature_guide09186a0080
18912f.html.
SRST Debugging Commands
Command
Description
HIFYKITLSRIEPEVQ
Sets SCCP Protocol alarm message debugging for the
Cisco IP phone.
HIFYKITLSRIHIXEMP
QEGEHHVIWW QEG
EHHVIWW
Sets detail debugging for the Cisco IP Phone.
HIFYKITLSRIOIITEPMZI
Sets keepalive debugging for the Cisco IP Phone.
HIFYKITLSRIPSSTFEGO
Sets Message Waiting Indicator (MWI) debugging for the
Cisco IP Phone.
HIFYKITLSRITEO
Provides voice packet level debugging and prints the contents of
one voice packet in every 1024 voice packets.
HIFYKITLSRIVE[
Provides raw low-level protocol debugging display for all SCCP
messages.
HIFYKITLSRIVIKMWXIV
Sets registration debugging for the Cisco IP Phone.
HIFYKITLSRIWXEXI
Sets state debugging for the Cisco IP Phone.
HIFYKITLSRI
WXEXMWXMGW
Sets statistics debugging for the Cisco IP Phone.
HIFYKITLSRIUSZQEG
EHHVIWW QEGEHHVIWW
Displays Quality of Voice (QoV) statistics for calls when preset
limits are exceeded.
HIFYKITLSRIIVVSV
QEGEHHVIWW QEG
EHHVIWW
Displays information about the types of debugging that are
enabled for your router. This debug closes out the debug
ephone detail command.
If the remote IP Phones lose communication with Cisco CallMananger, it is likely that the
DHCP server will also be out of reach. It is best practice to create a DHCP service on the
gateway so that the remote IP Phones will get their IP address from the gateway and
communicate with the TFTP server under option 150 as indicated in the DHCP pool defined on
the gateway; otherwise, you can simply use the gateway as a DHCP relay server, relaying
DHCP requests to another server. This best practice design will ensure smooth SRST
transitioning—in terms of transitioning into SRST mode and back in the event that SRST is
needed.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Network Infrastructure
4-95
Testing Procedures:
One way to test to see if SRST is working is to stop the Cisco CallManager service (ccm.exe)
on the cluster. This will force the IP Phones to register with a backup server, and because there
is no backup server, the IP Phones will be forced to register with the gateway. If your SRST
configuration is correct, you should see the IP Phones convert to ephones as they register with
the gateway. The next step after the phones register is to make a test call to another phone and
out to the PSTN and back.
Run the debug ephone register command on the gateway and stop ccm.exe on the cluster. You
should see output similar to the sample that follows, indicating that SRST is engaged. You will
see in this debug command output that IP Phones are considered ephones and are reregistered
with the gateway.
(*;+;HIFYKITLSRIVIKMWXIV
)4,32)VIKMWXVEXMSRHIFYKKMRKMWIREFPIH
(*;+;
1EV2I[7OMRR]WSGOIXEGGITXIH?A EGXMZI
1EVWMRCJEQMP]WMRCTSVXMRCEHHV
1EVWOMRR]CEHHCWSGOIX
1EV2I[7OMRR]WSGOIXEGGITXIH?A EGXMZI
1EVWMRCJEQMP]WMRCTSVXMRCEHHV
1EVWOMRR]CEHHCWSGOIX
1EV -44,32)6)+C%0%61
2EQI!7)4*0SEH! 0EWX!'1EFSVXIH8'4
1EV
7OMRR]7XEXMSR%PEVQ1IWWEKISRWSGOIX?A
7)4*
1EVWIZIVMX]-RJSVQEXMSREPT!?\A
T!?\
%'A
1EV2EQI!7)4*0SEH! 0EWX!'1EFSVXIH8'4
1EV -44,32)6)+C%0%61
2EQI!7)4*0SEH! 0EWX!'1EFSVXIH8'4
1EV
7OMRR]7XEXMSR%PEVQ1IWWEKISRWSGOIX?A
7)4*
1EVWIZIVMX]-RJSVQEXMSREPT!?\A
T!?\
4-96
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
%'A
1EV2EQI!7)4*0SEH! 0EWX!'1EFSVXIH8'4
1EVITLSRI ?A7XEXMSR6IKMWXIV1IWWEKI
JVSQ
1EVITLSRI ?A6IKMWXIV7XEXMSR-HIRXMJMIV
(IZMGI2EQI7)4
*
1EVITLSRI ?A7XEXMSR-HIRXMJMIV-RWXERGI
HIZMGI8]TI
1EVITLSRI?AWXEXMSR-T%HHV
1EVITLSRI?AQE\7XVIEQW
1EVITLSRI?ATVSXSGSP:IV
1EVITLSRI?ATLSRIWM^IHRWM^I
1EVITLSRI %PPS[ER]7OMRR]7IVZIV-4
EHHVIWW
1EVITLSRI?A*SYRHIRXV]JSV
*
1EVITLSRI?AWSGOIXGLERKIXS
1EVITLSRI?A*%-0)('037)(SPHWSGOIX
1EVITLSRI?A*SVGIHIZMGIWYFX]TIXS
1EVITLSRI?ATLSRI7)4*VI
EWWSGMEXI3/SRWSGOIX?A
1EV -44,32)6)+-78)6C2);ITLSRI
7)4*-47SGOIX(IZMGI8]TI4LSRILEW
VIKMWXIVIH
1EV4LSRIWSGOIX
1EV7OMRR]0SGEP-4EHHVIWW!SR
TSVX
1EV7OMRR]4LSRI-4EHHVIWW!
1EVITLSRI?A7MKREPTVSXSGSPZIVXS
TLSRI[MXLZIV
1EVITLSRI?A(EXI*SVQEX1(=
1EVITLSRI?A?7)4*A6IKMWXIV%GO
WIRXXSITLSRIOIITEPMZITIVMSH
1EVITLSRI?A'ETEFMPMXMIW6IUWIRX
1EVITLSRI ?A7XEXMSR6IKMWXIV1IWWEKI
JVSQ
1EVITLSRI ?A6IKMWXIV7XEXMSR-HIRXMJMIV
(IZMGI2EQI7)4
*
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Network Infrastructure
4-97
1EVITLSRI ?A7XEXMSR-HIRXMJMIV-RWXERGI
HIZMGI8]TI
1EVITLSRI?AWXEXMSR-T%HHV
1EVITLSRI?AQE\7XVIEQW
1EVITLSRI?ATVSXSGSP:IV
1EVITLSRI?ATLSRIWM^IHRWM^I
1EVITLSRI %PPS[ER]7OMRR]7IVZIV-4
EHHVIWW
1EVITLSRI?A*SYRHIRXV]JSV
*
1EVITLSRI?AWSGOIXGLERKIXS
1EVITLSRI?A*%-0)('037)(SPHWSGOIX
1EVITLSRI?A*SVGIHIZMGIWYFX]TIXS
1EVITLSRI?ATLSRI7)4*VI
EWWSGMEXI3/SRWSGOI
X?A
1EV -44,32)6)+-78)6C2);ITLSRI
7)4*-4
7SGOIX(IZMGI8]TI4LSRILEWVIKMWXIVIH
1EV4LSRIWSGOIX
1EV7OMRR]0SGEP-4EHHVIWW!SR
TSVX
1EV7OMRR]4LSRI-4EHHVIWW!
1EVITLSRI?A7MKREPTVSXSGSPZIVXS
TLSRI[MXLZIV
1EVITLSRI?A(EXI*SVQEX1(=
1EVITLSRI?A?7)4*A6IKMWXIV%GO
WIRXXSITLSRI
OIITEPMZITIVMSH
1EVITLSRI?A'ETEFMPMXMIW6IUWIRX
1EVITLSRI?A,IEHWIX7XEXYW1IWWEKI
1EVITLSRI?A,IEHWIX7XEXYW1IWWEKI
1EVITLSRI?A'ETEFMPMXMIW6IWVIGIMZIH
1EVITLSRI?A'ETWPMWX
;MHI&ERHC/QW
+9PE[OQW
+%PE[OQW
+%RRI\&QW
+%RRI\%[%RRI\&QW
+QW
+%RRI\%QW
4-98
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
1EVITLSRI?A'ETEFMPMXMIW6IWVIGIMZIH
1EVITLSRI?A'ETWPMWX
;MHI&ERHC/QW
+9PE[OQW
+%PE[OQW
+%RRI\&QW
+%RRI\%[%RRI\&QW
+QW
+%RRI\%QW
1EVITLSRI?A,IEHWIX7XEXYW1IWWEKI
1EVITLSRI?A,IEHWIX7XEXYW1IWWEKI
1EVITLSRI?A&YXXSR8IQTPEXI6IU1IWWEKI
1EVITLSRI?A&YXXSR8IQTPEXI6IUVIUYIWXMRK
JEPPFEGOMRJS
1EVITLSRI?A&YXXSR8IQTPEXI6IU1IWWEKI
1EVITLSRI?A&YXXSR8IQTPEXI6IUVIUYIWXMRK
JEPPFEGOMRJS
1EVITLSRI?A1EPPSG3/JSV
7IVZIV6IW1IWWEKI
1EVITLSRI?A1EPPSG3/JSV
7IVZIV6IW1IWWEKI
1EVITLSRI?A0MQMXMRKRYQFIVSJPMRI
FYXXSRWSRJVSQ
XS
1EVITLSRI?A0MQMXMRKRYQFIVSJPMRI
FYXXSRWSRJVSQ
XS
1EVITLSRI?A&YXXSR8IQTPEXI6IU1IWWEKI
1EVITLSRI?A?7)4*A7IXXMRK
PMRIWWTIIHHMEPW
SRTLSRI QE\CPMRI 1EVITLSRI?A*MVWX7TIIH(MEP&YXXSR
PSGEXMSRMW 1EVITLSRI?A'SRJMKYVIHWTIIHHMEP
FYXXSRW
1EVITLSRI?A&YXXSR8IQTPEXIPMRIW!
WTIIH!FYXXSRW!SJJWI
X!
1EVITLSRI?A&YXXSR8IQTPEXI6IU1IWWEKI
1EVITLSRI?A?7)4*A7IXXMRK
PMRIWWTIIHHMEPW
SRTLSRI QE\CPMRI Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Network Infrastructure
4-99
1EVITLSRI?A*MVWX7TIIH(MEP&YXXSR
PSGEXMSRMW 1EVITLSRI?A'SRJMKYVIHWTIIHHMEP
FYXXSRW
1EVITLSRI?A&YXXSR8IQTPEXIPMRIW!
WTIIH!FYXXSRW!SJJWI
X!
1EVITLSRI
?A7XEXMSR7SJX/I]8IQTPEXI6IU1IWWEKI
1EVITLSRI
?A7XEXMSR7SJX/I]8IQTPEXI6IW1IWWEKI
1EVITLSRI
?A7XEXMSR7SJX/I]8IQTPEXI6IU1IWWEKI
1EVITLSRI
?A7XEXMSR7SJX/I]8IQTPEXI6IW1IWWEKI
1EVITLSRI?A7XEXMSR7SJX/I]7IX6IU1IWWEKI
1EVITLSRI?A7XEXMSR7SJX/I]7IX6IW1IWWEKI
1EVITLSRI?A7XEXMSR7SJX/I]7IX6IU1IWWEKI
1EVITLSRI?A7XEXMSR7SJX/I]7IX6IW1IWWEKI
1EVITLSRI?A7XEXMSR0MRI7XEX6IU1IWWEKI
JVSQITLSRIPMRI
1EVITLSRI
?A?7)4*A7XEXMSR0MRI7XEX6IU1IWWEKIJVS
QITLSRIPMRI-RZEPMH(2
1EVITLSRI
?A?7)4*A7XEXMSR0MRI7XEX6IW1IWWEKIWIR
XXSITLSRI SJ 1EVITLSRI?A7XEXMSR0MRI7XEX6IU1IWWEKI
JVSQITLSRIPMRI
1EVITLSRI
?A?7)4*A7XEXMSR0MRI7XEX6IU1IWWEKIJVS
QITLSRIPMRI-RZEPMH(2
1EVITLSRI
?A?7)4*A7XEXMSR0MRI7XEX6IW1IWWEKIWIR
XXSITLSRI SJ 1EVITLSRI?A7XEXMSR0MRI7XEX6IU1IWWEKI
JVSQITLSRIPMRI
1EVITLSRI
?A?7)4*A7XEXMSR0MRI7XEX6IU1IWWEKIJVS
QITLSRIPMRI-RZEPMH(2
1EVITLSRI
?A?7)4*A7XEXMSR0MRI7XEX6IW1IWWEKIWIR
XXSITLSRI SJ 1EVITLSRI?A7XEXMSR0MRI7XEX6IU1IWWEKI
JVSQITLSRIPMRI
4-100
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
1EVITLSRI
?A?7)4*A7XEXMSR0MRI7XEX6IU1IWWEKIJVS
QITLSRIPMRI-RZEPMH(2
1EVITLSRI
?A?7)4*A7XEXMSR0MRI7XEX6IW1IWWEKIWIR
XXSITLSRI SJ 1EVITLSRI?A7XEXMSR0MRI7XEX6IU1IWWEKI
JVSQITLSRIPMRI
1EVITLSRI
?A?7)4*A7XEXMSR0MRI7XEX6IU1IWWEKIJVS
QITLSRIPMRI-RZEPMH(2
1EVITLSRI
?A?7)4*A7XEXMSR0MRI7XEX6IW1IWWEKIWIR
XXSITLSRI SJ 1EVITLSRI?A7XEXMSR0MRI7XEX6IU1IWWEKI
JVSQITLSRIPMRI
1EVITLSRI
?A?7)4*A7XEXMSR0MRI7XEX6IU1IWWEKIJVS
QITLSRIPMRI-RZEPMH(2
1EVITLSRI
?A?7)4*A7XEXMSR0MRI7XEX6IW1IWWEKIWIR
XXSITLSRI SJ 1EVITLSRI?A7XEXMSR0MRI7XEX6IU1IWWEKI
JVSQITLSRIPMRI
1EVITLSRI
?A?7)4*A7XEXMSR0MRI7XEX6IU1IWWEKIJVS
QITLSRIPMRI-RZEPMH(2
1EVITLSRI
?A?7)4*A7XEXMSR0MRI7XEX6IW1IWWEKIWIR
XXSITLSRI SJ 1EVITLSRI?A7XEXMSR0MRI7XEX6IU1IWWEKI
JVSQITLSRIPMRI
1EVITLSRI
?A?7)4*A7XEXMSR0MRI7XEX6IU1IWWEKIJVS
QITLSRIPMRI-RZEPMH(2
1EVITLSRI
?A?7)4*A7XEXMSR0MRI7XEX6IW1IWWEKIWIR
XXSITLSRI SJ 1EVITLSRI?A7XEXMSR0MRI7XEX6IU1IWWEKI
JVSQITLSRIPMRI
1EVITLSRI
?A?7)4*A7XEXMSR0MRI7XEX6IU1IWWEKIJVS
QITLSRIPMRI-RZEPMH(2
1EVITLSRI
?A?7)4*A7XEXMSR0MRI7XEX6IW1IWWEKIWIR
XXSITLSRI SJ Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Network Infrastructure
4-101
1EVITLSRI?A7XEXMSR0MRI7XEX6IU1IWWEKI
JVSQITLSRIPMRI
1EVITLSRI
?A?7)4*A7XEXMSR0MRI7XEX6IU1IWWEKIJVS
QITLSRIPMRI-RZEPMH(2
1EVITLSRI
?A?7)4*A7XEXMSR0MRI7XEX6IW1IWWEKIWIR
XXSITLSRI SJ 1EVITLSRI?A7XEXMSR0MRI7XEX6IU1IWWEKI
JVSQITLSRIPMRI
1EVITLSRI?A7XEXMSR0MRI7XEX6IU1IWWEKI
ITLSRIPMRI(2!
HIWG!PEFIP!
1EVITLSRI
?A?7)4*A7XEXMSR0MRI7XEX6IW1IWWEKIWIR
XXSITLSRI SJ 1EVITLSRI?A7OMRR]'SQTPIXI6IKMWXVEXMSR
1EVITLSRI?A7XEXMSR0MRI7XEX6IU1IWWEKI
JVSQITLSRIPMRI
1EVITLSRI?A7XEXMSR0MRI7XEX6IU1IWWEKI
ITLSRIPMRI(2!
HIWG!PEFIP!
1EVITLSRI
?A?7)4*A7XEXMSR0MRI7XEX6IW1IWWEKIWIR
XXSITLSRI SJ 1EVITLSRI?A7OMRR]'SQTPIXI6IKMWXVEXMSR
1EVITLSRI?A7XEXMSR7IVZIV6IU1IWWEKIJVSQ
ITLSRI
1EVITLSRI?A*VII3/JSV7IVZIV6IW1IWWEKI
1EV9WMRK7IVZIV0MWXJVSQTLSRI
1EV'1
1EV'1
1EV'1
1EV'1
1EV'1
1EVITLSRI?A7XEXMSR7IVZIV6IW1IWWEKIWIRX
XSITLSRI
1EVITLSRI?A7XEXMSR7IVZIV6IU1IWWEKIJVSQ
ITLSRI
1EVITLSRI?A*VII3/JSV7IVZIV6IW1IWWEKI
1EV9WMRK7IVZIV0MWXJVSQTLSRI
1EV'1
1EV'1
1EV'1
4-102
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
1EV'1
1EV'1
1EVITLSRI?A7XEXMSR7IVZIV6IW1IWWEKIWIRX
XSITLSRI
1EVITLSRI?A7OMRR]%ZEMPEFPI0MRIWWIX
JSVWSGOIX?A
1EVITLSRI?A%PVIEH]HSRI
7OMRR]'SQTPIXI6IKMWXVEXMSR
1EVITLSRI?A7OMRR]%ZEMPEFPI0MRIWWIX
JSVWSGOIX?A
1EVITLSRI?A%PVIEH]HSRI
7OMRR]'SQTPIXI6IKMWXVEXMSR
1EV7OMRR]EGXMZIWSGOIXPMWX 1EV
(*;+;WLS[ITLSRIWYQQEV]
ITLSRI1EG*8'4WSGOIX?AEGXMZI0MRI
6)+-78)6)(
QIHME%GXMZISJJLSSOVMRKMRKVIWIXVIWIXCWIRXHIFYK
-48IPIGEWXIVOIITEPMZI'1*EPPFEGO
ITLSRI1EG*8'4WSGOIX?AEGXMZI0MRI
6)+-78)6)(
QIHME%GXMZISJJLSSOVMRKMRKVIWIXVIWIXCWIRXHIFYK
-48IPIGEWXIVOIITEPMZI'1*EPPFEGO
1E\6IKMWXIVIH9RVIKMWXIVIH(IGIEWIH7SGOIXW
ITLSRICWIRHCTEGOIXTVSGIWWW[MXGLIH
1E\'SRJIVIRGIW[MXLEGXMZI EPPS[IH 7OMRR]1YWMG3R,SPH7XEXYW
%GXMZI13,GPMIRXW QE\ 1IHME'PMIRXW
2S13,JMPIPSEHIH
Once the Cisco CallManager servers are back on line, the ephones will unregister with the
gateway and reregister with the appropriate Cisco CallManager server.
When running ephone debug commands, you should see “Phone has unregistered normally”
output statements and “Phone has registered” statements.
Make sure that you test the process and watch the phones go into failover.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Network Infrastructure
4-103
Make sure that your SRST reference configuration is set up correctly in Cisco CallManager and
that the device pools reference the correct SRST reference such as “use default gateway” or use
the name that you created in SRST reference configuration.
If your phones do not register when expected, reset the phones to see if the phones find the
gateway. If all phones do not register, make sure that the IP address for the gateway is correct
and that you can ping the gateway.
4-104
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Summary
This topic summarizes the key points discussed in this lesson.
Summary
• A gateway is the interface between the data network, IP WAN,
and PSTN.
• When troubleshooting PSTN issues, start at Layer 1 and work up to
Layer 3.
• Before troubleshooting a PSTN issue, isolate whether the issue is
between the gateway and Cisco CallManager or between the gateway
and PSTN.
• Most common PSTN signaling issues are easily resolved.
• Become familiar with debug commands.
• Use the Cisco TAC website to review sample gateway configurations.
• In failure mode, the remote gateway automatically autoconfigures itself
using SNAP technology.
• In SRST mode, IP Phones rehome with their local gateway and keep
sending keepalives to the Cisco CallManager.
• The ccm-manager-fallback enables Cisco CallManager fallback command
capability and puts you in a submenu in Cisco IOS software.
© 2004 Cisco Systems, Inc. All rights reserved.
Copyright © 2004, Cisco Systems, Inc.
IPTT v4.0—4-41
Troubleshooting Network Infrastructure
4-105
References
For additional information, refer to these resources:
Cisco CallManager 4.0(1) and H.323 gateway configuration:
http://www.cisco.com/en/US/products/sw/voicesw/ps556/products_tech_note09186a00800
94636.shtml#cm4x.
Cisco TAC voice troubleshooting guidelines:
http://www.cisco.com/en/US/products/sw/voicesw/ps556/products_tech_note09186a00800
a6a14.shtml.
Cisco Survivable Remote Site Telephony (SRST): All Versions:
http://www.cisco.com/en/US/products/sw/voicesw/ps2169/products_feature_guide09186a0
08018912f.html.
Dial-peer configuration on a voice gateway:
http://www.cisco.com/univercd/cc/td/doc/product/software/ios123/123cgcr/vvfax_c/int_c/d
peer_c/index.htm.
How to Configure MGCP with Digital PRI and Cisco CallManager:
http://www.cisco.com/en/US/tech/tk652/tk701/technologies_configuration_example09186a
00801ad22f.shtml.
Configuring H.323 Gateways:
http://www.cisco.com/en/US/products/sw/iosswrel/ps1835/products_configuration_guide_c
hapter09186a0080080a66.html.
Understanding H323 Gatekeepers:
http://www.cisco.com/en/US/tech/tk652/tk701/technologies_tech_note09186a00800c5e0d.
shtml.
Basic two-zone gatekeeper configuration:
http://www.cisco.com/en/US/tech/tk652/tk701/technologies_configuration_example09186a
00800a9a56.shtml.
Call Routing/Dial Plan Including Cisco IOS Dial Peers:
http://www.cisco.com/pcgibin/Support/browse/psp_view.pl?p=Technologies:Voice_Call_Routing_Dial_Plans&s=Imp
lementation_and_Configuration#Samples_and_Tips.
Prerequisites for integrating Cisco SRST within a Cisco CallManager environment:
http://www.cisco.com/en/US/products/sw/iosswrel/ps5207/products_configuration_guide_c
hapter09186a00801f3ac8.html.
Cisco SRST—Setting up the Network:
http://www.cisco.com/en/US/products/sw/iosswrel/ps5207/products_configuration_guide_c
hapter09186a00801f3ab5.html#wp1331691.
4-106
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Quiz
Use the practice items here to review what you learned in this lesson. The correct answers are
found in the Quiz Answer Key.
Q1)
What is the role of a gateway?
A)
B)
C)
D)
Q2)
If the show isdn status command results in the Layer 1 status of DEACTIVATED,
what should you do?
A)
B)
C)
D)
Q3)
ccm-manager-fallback
show ephones
max-dn
max-ephones
When troubleshooting dropped calls on a gateway, you should do which of the
following things? (Choose two.)
A)
B)
C)
D)
Q6)
check the gateway ISDN switch type in global configuration mode
check the controller line coding, clocking, and framing
check the gateway ISDN switch type under the serial interface
none of the above
Which command is used to tell the remote gateway how many IP Phones to expect for
SRST?
A)
B)
C)
D)
Q5)
check the dial-peer configuration
check the switch type on the gateway
check the controller line coding, clocking, and framing
none of the above
If the show isdn status command results in the Layer 1 status of ACTIVE, but with
Layer 2 Status State=TEI_ASSIGNED, what should you do?
A)
B)
C)
D)
Q4)
to interface between the IP WAN, PSTN, and data network
to provide PSTN access to Cisco CallManager
to route data traffic only
none of the above
check Microsoft Windows 2000 Event Viewer for IP Phone resets
check Microsoft Windows 2000 Event Viewer for gateway resets
check Microsoft Windows 2000 Event Viewer for ccm.exe resets
contact the telco
To determine “Device type” and Reason Code” for device resets, which two Cisco
online documents would you use? (Choose two.)
A)
B)
C)
D)
system error messages for Cisco CallManager
check the Microsoft Windows 2000 Event Viewer for gateway resets
check the Microsoft Windows 2000 Event Viewer for ccm.exe resets
none of the above
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Network Infrastructure
4-107
Lesson Review Answer Key
4-108
Q1)
A
Q2)
C
Q3)
C
Q4)
D
Q5)
A, B
Q6)
A, B
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Gatekeepers
and Cisco SIP Trunks
Overview
Call Admission Control (CAC) is a major role that gatekeepers play in a VoIP environment.
This lesson will review the role of the gatekeeper and how the gatekeeper is integrated with
gateways and Cisco CallManager servers. This lesson will discuss signaling and configuration
issues related to gatekeepers integrating with gateways and Cisco CallManager. This lesson
discusses gatekeeper-controlled trunks integrated with Cisco CallManager, the various
different trunks, and why these trunks are used. This lesson also will discuss SIP trunks and
how to troubleshoot SIP trunks.
Relevance
To take the appropriate steps to resolve signaling and configuration issues, you need to
understand the signaling between devices.
Objectives
Upon completing this lesson, you will be able to:
Describe the role of a gatekeeper in a Cisco AVVID network
Identify common voice issues related to gatekeeper configurations
Discuss signaling between Cisco CallManager and a gatekeeper, and identify common
issues and troubleshooting tools
Describe Cisco SIP trunk operation with Cisco CallManager
Learner Skills and Knowledge
To benefit fully from this lesson, you must have these prerequisite skills and knowledge:
Exposure to Cisco CallManager, gateways, and gatekeeper configurations
Experience with registration, admission, and status (RAS) and H.323 signaling
Outline
The outline lists the topics included in this lesson.
Outline
• Overview
• Gatekeeper Overview
• Troubleshooting Gatekeepers with Gateways and
Cisco CallManager
• Gatekeeper and Cisco CallManager Call
Processing
• Troubleshooting Cisco SIP Trunks
• Summary
• Quiz
© 2004 Cisco Systems, Inc. All rights reserved.
4-110
Cisco IP Telephony Troubleshooting (IPTT) v4.0
IPTT v4.0—4-3
Copyright © 2004, Cisco Systems, Inc.
Gatekeeper Overview
This topic discusses the specifics related to gatekeeper signaling, troubleshooting gatekeepers,
and related configuration issues.
Gatekeeper Overview
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—4-4
The role of the gatekeeper is as follows:
A gatekeeper is an H.323 device on the network that provides services such as address
translation and network access control for H.323 terminals and gateways.
Gatekeepers are optional in an H.323 network, but if a gatekeeper is present, endpoints
must use its services.
Gatekeepers communicate to endpoints using RAS signaling, a derivative of Q.931.
The H.323 standard defines the mandatory and optional gatekeeper functions. The mandatory
gatekeeper functions are as follows:
Address translation: The gatekeeper translates H.323 IDs (such as gwy1@domain.com)
and E.164 numbers (standard telephone numbers) into endpoint IP addresses.
Admission control: The gatekeeper controls endpoint admission into the H.323 network.
To achieve this, the gatekeeper uses the following:
—
H.225 RAS messages
—
Admission Request (ARQ)
—
Admission Confirmation (ACF)
—
Admission Reject (ARJ)
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Network Infrastructure
4-111
Bandwidth control: Bandwidth control consists of managing endpoint bandwidth
requirements. To achieve this, the gatekeeper uses these H.225 RAS messages:
—
Bandwidth Request (BRQ)
—
Bandwidth Confirmation (BCF)
—
Bandwidth Reject (BRJ)
Zone management: The gatekeeper provides zone management for all registered
endpoints in the zone; for example, controlling the endpoint registration process.
The optional gatekeeper functions are as follows:
Call authorization: With this option, the gatekeeper can restrict access to certain terminals
or gateways and/or have time-of-day policies restrict access.
Call management: With this option, the gatekeeper maintains active call information and
uses it to indicate busy endpoints or redirect calls.
Bandwidth management: With this option, the gatekeeper can reject admission when the
required bandwidth is not available.
Call control signaling: With this option, the gatekeeper can route call-signaling messages
between H.323 endpoints using the Gatekeeper-Routed Call Signaling (GKRCS) model.
Alternatively, GKRCS gatekeepers allow endpoints to send H.225 call-signaling messages
directly to each other.
Note
4-112
Cisco IOS gatekeepers are direct endpoint signaling based.
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Gatekeeper Messages
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—4-5
This figure shows the RAS messages that are sent to and from the gatekeeper. Notice the
following various types of messages:
Gatekeeper discovery
Terminal/gateway registration
Terminal/gateway unregistration
Bandwidth change
Location request
Call admission
Disengage
Status queries
The gatekeeper can use the RAS channel to obtain information from endpoints. This can be
used to monitor whether the endpoints are online or offline. The following are the informational
messages:
IRQ (Information_Request): Sent from the gatekeeper to the endpoint requesting status.
IRR (Information_Request_Response): Sent from the endpoint to the gatekeeper in
response to an IRQ message. This message is also sent from the endpoint to the gatekeeper
if the gatekeeper requests periodic status updates. The IRR is used by gateways to inform
the gatekeeper about the active calls.
IACK (Info_Request_Acknowledge): Used by the gatekeeper to respond to IRR messages.
INACK (Info_Request_Neg_Acknowledge): Used by the gatekeeper to respond to IRR
messages.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Network Infrastructure
4-113
RAS Signaling
Directed Endpoint Signaling
H.225 (Q.931) Call Setup (TCP)
H.245 Call Control (TCP)
RTP (UDP)
The RAS channel is opened before any other channel and is
independent of the call setup and media transport channels.
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—4-6
Cisco IOS gatekeepers are directed endpoint signaling based; this means that the call setup
messages are handled by the originating and terminating gateway. All RAS messages sent to
endpoints use H.225 signaling over UDP. Call setup messages from the endpoints to other
endpoints use H.225 (Q.931) over TCP. Call control messages sent to and from endpoints use
H.245 signaling over TCP and media stream over UDP.
4-114
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Gatekeeper H.323 Call Flow
RAS (ARQ)
A
GK
RAS (ACF)
Establish H.225 Signaling Channel (TCP Connection)
B
H.225 SETUP
H.225 Call Proceeding
RAS (ARQ)
RAS (ACF)
H.225 Alerting Connection (H245 address)
Terminal Capability Set + Ack Master/Slave Determination
Terminal Capability Set – Master/Slave Determination + Ack
Terminal Capability Set Ack – Master/Slave Determination + Ack
Open Logical Channel (OLC)
Open Logical Channel (OLC) Ack with IP address + Port from side B
RTP
RTP media stream
Open Logical Channel (OLC)
Open Logical Channel (OLC) Ack with IP address + Port from side A
H.245
RTP media stream
H.225
H.225 Release Complete
RAS (DRQ)
RAS (DRQ)
RAS (DCF)
RAS (DCF)
© 2004 Cisco Systems, Inc. All rights reserved.
RAS
IPTT v4.0—4-7
This figure shows the signaling interaction between each gateway and its gatekeeper.
Signaling:
RAS: H.225 over UDP
Call setup: H.225 (Q.931 derivative) over TCP
Call control: H.245 over TCP
RTP stream: Over UDP
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Network Infrastructure
4-115
Single Zone Gatekeeper
Call Flow with Gatekeeper: Intrazone
GW1
GW2
x5000
x6000
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—4-8
This figure shows a single zone gatekeeper-to-gateway call flow. The signaling between the
gateway and gatekeeper is done using RAS messages. These messages are in the form of H.323
signaling. Admission messages between endpoints and gatekeepers provide the basis for call
admissions and bandwidth control. Gatekeepers (GK) authorize access to H.323 networks by
confirming or rejecting an admission request. (For the purpose of this series of slides and the
accompanying discussions, GK stands for gatekeeper and GW stands for gateway.)
Here is the call flow scenario:
1. Destination pattern x5000 picks up the phone and calls destination pattern x6000.
2. GW1 initiates a Registration Request (RRQ) and then receives the Registration
Confirmation (RCF) from the gatekeeper.
3. GW1 initiated an ARQ to the GK asking the GK if it can send a call to GW2 (destination
pattern 6000).
4. The GK does a lookup and finds extension 6000 on GW2 and returns an ACF with the IP
address of GW2.
5. GW1 sends a Q.931 call setup to GW2 with destination 6000.
6. GW2 initiates an RRQ to the GK and receives a RCF from the GK.
7. GW2 then initiates an ARQ to the GK asking permission to answer the call by GW1 and
receives an ACF from the GK along with the IP address of GW1.
8. GW2 setup a POTS call to destination pattern 6000.
9. When the destination pattern 6000 of GW2 answers the call, GW2 sends a Q.931 connect
message to GW1.
10. After call setup and connect, the GWs send an IRR to the GKs.
4-116
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
With respect to the signaling between the gateways and gatekeeper, what would happen if
network connectivity was disrupted during the initial RAS messaging between the originating
gateway and the gatekeeper?
The gatekeeper does not have to be on the same subnetwork, LAN, or in the same region. Is it
possible for routing to be an issue restricting CAC?
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Network Infrastructure
4-117
Multizone Gatekeeper
Call Flow with Multiple Gatekeepers: Interzone
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—4-9
This figure shows a multizone gatekeeper-to-gateway call flow. The RAS signaling in a
multizone environment is much like that of a single zone environment. Can you identify the
additional RAS messages in this call flow?
1. Destination pattern x5000 picks up the phone and enters x6000.
2. GW1 sends an RRQ to GK1 and receives an RCF from GK1.
3. GW1 sends an ARQ to GK1 asking for permission to call destination pattern x6000.
4. GK1 does a lookup and does not find a match for destination pattern x6000, GK1 does a
prefix lookup and finds the match with GK2; GK1 sends a Location Request (LRQ) to
GK2 and a Request in Progress (RIP) to GW1.
5. GK2 does a lookup and finds destination pattern x6000, and returns a Location Confirm
(LCF) to GK1 with the IP address of GW2.
6. GK1 returns an ACF to GW1 with the IP address of GW2.
7. GW1 sends a Q.931 call setup message to GW2 with destination pattern x6000.
8. GW2 sends to GK2 an ARQ asking permission to answer the call from GW1.
9. GK2 returns an ACF to GW2 with the IP address of GW1.
10. GW2 sets up the POTS call to destination pattern x6000.
11. When destination pattern x6000 answers the call, GW2 sends a Q.931 connect message to
GW1.
12. After the call setup and connect, the GWs send IRRs to the GKs.
4-118
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Note
A major functionality of gatekeepers is to keep track of other H.323 zones and to forward
calls appropriately. When many H.323 zones are present, gatekeeper configurations can
become very long. In such large VoIP installations, it is possible to configure a centralized
directory gatekeeper that contains a registry of all the different zones and coordinates LRQforwarding processes. With directory gatekeepers, no full mesh is needed between
interzone gatekeepers. Directory gatekeepers are beyond the scope of this lesson.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Network Infrastructure
4-119
Troubleshooting Gatekeepers with Gateways and
Cisco CallManager
This topic discusses the specifics relative to gatekeepers with gateways and Cisco CallManager.
Troubleshooting Gatekeepers with Gateways
and Cisco CallManager Check List
1. Verify GK configuration
• Does the gateway/Cisco CallManager belong to one zone?
• Verify that the dial plan is correct.
• Verify that the technology prefix is correct.
• Is bandwidth configured between zones?
2. Verify GK on gateway/Cisco CallManager
• Check device pool (codec usage)
• Verify check.
3. Verify gateway/Cisco CallManager registration with GK
• show gatekeeper endpoints
• debug ras
• debug h225 asn1
4. Verify call setup with GK
• debug ras
• show gatekeeper calls
• show gatekeeper zone status
• debug h225 asn1
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—4-10
Verify GK Configuration
The first thing to check when troubleshooting gatekeeper issues is to ensure that the
configuration on the gatekeeper is correct. The gateways or Cisco CallManagers should
only belong to one zone defined as a local zone. Configuring multiple clusters in the same
zone results in failed calls, because the gatekeeper will not know which calls should go to
which Cisco CallManager cluster.
Next, check to make sure that the dial plan is correct. If you are not prepending a
technology prefix to the dialed digits when you route calls to the gatekeeper, you need to
ensure that you have the default technology prefixes configured on the gatekeeper. If you
are not prepending technology prefixes, use the gw-type-prefix 1# default-technology
command in the configuration on the gatekeeper. Failure to use this command in cases
where you are not prepending technology prefixes with result in call route failure.
If you are using bandwidth control on the gatekeeper, make sure that the bandwidth from
one zone to another is configured correctly; bandwidth requirements vary by codec.
4-120
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Verify GK Configuration on the Gateway or Cisco CallManager
The major key here is to validate configuration on the gateway or Cisco CallManager and
make sure that the configuration on the gateway or in Cisco CallManager matches that of
the gatekeeper. If you are using an anonymous device, the Allow Anonymous Calls box
must be checked. Device protocol settings allow you to select what kinds of devices are
registered to the gatekeeper in addition to the Cisco CallManager cluster that you are
configuring. Cisco CallManager 3.2 and later automatically detects a gatekeeper-controlled
call between two Cisco CallManager servers and switches to the intercluster trunk variant
of H323.
In Cisco CallManager 3.3 and above, anonymous devices are replaced with trunk devices.
This allows you to route calls to multiple gatekeepers. When using a gatekeeper for CAC,
ensure that the zones and technology prefixes are configured on the trunk devices.
Verify Gateway or Cisco CallManager Registration with Gatekeeper
Use the show gatekeeper endpoints commands on the gatekeeper to verify gateway or
Cisco CallManager registration. If the gateway or Cisco CallManager is not registering
with the gatekeeper or registering in the wrong zone, check your configuration on the
gateway or Cisco CallManager and cross-reference the configuration with the gatekeeper.
Use the debug ras command to make sure that the gateway or Cisco CallManager is
attempting to register with the gatekeeper. The easiest way to force the Cisco CallManager
to register (use an RRQ) with the gatekeeper is to Reset the Gatekeeper from the Cisco
CallManager. To see the actual registration messages between the gatekeeper and Cisco
CallManager, use the debug h225 asn1 command. The debug command will show reject
registration error messages in the debug command output.
Note
Nearly all gatekeeper, gateway, and Cisco CallManager registration issues are due to
misconfiguration.
Verify Call Setup with GK
Once the gateway or Cisco CallManager registers with the gatekeeper, confirm call setup
by placing a call between the two Cisco CallManager clusters. Enable debug ras on the
gatekeeper. With the call active, you can run the command show gatekeeper calls to
bandwidth allocation. Additional information can be derived from running the show
gatekeeper zone status command. When using the show gatekeeper zone status
command, look for the maximum total bandwidth and current total bandwidth; these values
can you tell how much bandwidth remains for each zone.
Other than not enough bandwidth for a call to proceed, reasons for call rejection is are no
zone prefix is configured or configured incorrectly, and or the Cisco CallManager is not
configured or configured correctly.
Command Information
show gatekeeper endpoints: Use this command on the Cisco IOS device that is acting as
the gatekeeper. This command is useful to ensure that the devices have found the
gatekeeper and are registered.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Network Infrastructure
4-121
Troubleshooting Gatekeeper Endpoints
http://www.cisco.com/en/US/partner/tech/tk652/tk701/technologies_tech_note09186a00800e65
4c.shtml
show gatekeeper zone status: Use this command on the IOS device that is acting as the
gatekeeper. This command displays the bandwidth information for all zones.
show gatekeeper zone cluster: Use this command on the IOS device that is acting as the
gatekeeper. This command displays the bandwidth information. This command applies
when the gatekeeper is part of a gatekeeper cluster.
show gatekeeper gw: This command is used to verify the registration status of the
endpoints for the technology prefix.
show gatekeeper calls: This command, when used on the gatekeeper, displays the active
calls permitted by the gatekeeper and displays the bandwidth that each call is using.
Note
4-122
Troubleshooting Gatekeeper Bandwidth Management
http://www.cisco.com/en/US/partner/tech/tk652/tk701/technologies_white_paper09186a008
00c5f67.shtml
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Common Gatekeeper Issues
• Use: debug h225 asn1
• Four of the most common issues between
gateways and gatekeepers:
– RRJ: rejectReason duplicateAlias
– RRJ: rejectReason terminalExcluded
– RRJ: rejectReason securtyDenial
– RRJ: rejectReason invalidAlias
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—4-11
Registration is the process by which gateways, terminals, and/or MCUs join a zone and inform
the gatekeeper of their IP and alias addresses. Registration occurs after the discovery process.
Every gateway can register with only one active gatekeeper. There is only one active
gatekeeper per zone.
The common issues between gatekeepers and gateways are Registration Rejects (RRJs). The
most typical of rejections are as follows:
1. RRJ: rejectReason duplicateAlias
2. RRJ: rejectReason terminalExcluded
3. RRJ: rejectReason securityDenial
4. RRJ: rejectReason invalidAlias
Use the debug h225 asn1 command on the gateway or gatekeeper or both. Look for RRJ
rejectReason output statements.
5. RRJ: rejectReason duplicateAlias
6%7-2'31-2+4(9!
ZEPYI6EW1IWWEKI!VIKMWXVEXMSR6INIGX
_
VIUYIWX7IU2YQ
TVSXSGSP-HIRXMJMIV_a
VINIGX6IEWSRHYTPMGEXI%PMEW
_aKEXIOIITIV-HIRXMJMIV_KOaa
Cause: Duplicate E.164 ID or H.323 ID (Another gateway is registered to the
gatekeeper using the E.164 ID or H.323 ID that you are using.)
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Network Infrastructure
4-123
If the cause is a duplicate E.164 ID, change the destination pattern configured
under a POTS dial peer associated with an FXS port.
If the cause is a duplicate H.323 ID, change the H.323 ID of the gateway under
the H.323 VoIP interface.
6. RRJ: rejectReason terminalExcluded
1EV6%7398+3-2+4(9!
ZEPYI6EW1IWWEKI!KEXIOIITIV6INIGX
_
VIUYIWX7IU2YQ
TVSXSGSP-HIRXMJMIV_a
VINIGX6IEWSRXIVQMREP)\GPYHIH2900
a
Cause: This is the result of the subnetwork of the gateway being disabled in the
gatekeeper.
Check the gatekeeper configuration. You will most likely see the following
configuration:
KEXIOIITIV
^SRIPSGEPKOGMWGSGSQ
RS^SRIWYFRIXKOIREFPI
^SRITVIJM\KO
K[X]TITVIJM\HIJEYPXXIGLRSPSK]
RSWLYXHS[R
If you see this configuration, removing the no zone subnet gk 172.16.13.0/27
enable command will resolve the issue. To remove the command completely,
remove zone local gk cisco.com.
7. RRJ: rejectReason securityDenial
1EV6%7398+3-2+4(9!
ZEPYI6EW1IWWEKI!VIKMWXVEXMSR6INIGX
_
VIUYIWX7IU2YQ
TVSXSGSP-HIRXMJMIV_a
VINIGX6IEWSRWIGYVMX](IRMEP2900
KEXIOIITIV-HIRXMJMIV_KOa
a
4-124
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Cause: This is the result of the security commands being enabled on either the
gateway or gatekeeper and not matching. The configurations to look out for are
nonmatching H.323 ID, E. 164 ID, passwords, or the security token that the
gatekeeper requires.
To resolve the issue, check which security command has been configured in the
gatekeeper.
If the security h323-id command is enabled, make sure that the gatekeeper has
been configured as follows. (The IP addresses are samples.)
YWIVREQIK[TEWW[SVH[[
KEXIOIITIV
^SRIPSGEPKOGMWGSGSQ
RS^SRIWYFRIXKOIREFPI
^SRITVIJM\KO
WIGYVMX]LMH
WIGYVMX]TEWW[SVHWITEVEXSV
K[X]TITVIJM\HIJEYPXXIGLRSPSK]
RSWLYXHS[R
Also, make sure that the gateway has the following configuration:
MRXIVJEGI)XLIVRIX
MTEHHVIWW
LEPJHYTPI\
LKEXI[E]ZSMTMRXIVJEGI
LKEXI[E]ZSMTMHKOMTEHHV
LKEXI[E]ZSMTLMHK[[[
2SXI 1EOIWYVIXLIKEXI[E]HSIWRSXLEZIXLIJSPPS[MRK
GSQQERH
KEXI[E]
WIGYVMX]TEWW[SVHPIZIPIRHTSMRX
If the security E164 command is enabled, make sure that the gatekeeper has
been configured as follows:
YWIVREQIÃ)EHHVIWWXLIKEXI[E]XVMIWXS
VIKMWXIVIHXSKEXIOIITIV
KEXIOIITIV
^SRIPSGEPKOGMWGSGSQ
RS^SRIWYFRIXKOIREFPI
^SRITVIJM\KO
WIGYVMX])
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Network Infrastructure
4-125
K[X]TITVIJM\HIJEYPXXIGLRSPSK]
RSWLYXHS[R
If the security token command is enabled, make sure the gatekeeper has been
configured as follows:
KEXIOIITIV
^SRIPSGEPKOGMWGSGSQ
RS^SRIWYFRIXKOIREFPI
^SRITVIJM\KO
WIGYVMX]XSOIRVIUYMVIHJSVVIKMWXVEXMSR
K[X]TITVIJM\HIJEYPXXIGLRSPSK]
RSWLYXHS[R
Also, make sure that the gateway has the following configuration:
KEXI[E]
WIGYVMX]TEWW[SVHPIZIPIRHTSMRX
8. RRJ: rejectReason invalidAlias
1EV6%7398+3-2+4(9!
ZEPYI6EW1IWWEKI!VIKMWXVEXMSR6INIGX
_
VIUYIWX7IU2YQ
TVSXSGSP-HIRXMJMIV_a
VINIGX6IEWSRMRZEPMH%PMEW2900
KEXIOIITIV-HIRXMJMIV_KO%a
a
Cause: There is no zone prefix defined in the gatekeeper. Check the configuration on
the gatekeeper and add the zone prefix with the proper E.164 address. Configure the
gatekeeper as follows:
KEXIOIITIV
^SRIPSGEPKO%GMWGSGSQ
^SRITVIJM\KO%
^SRITVIJM\KO%
^SRITVIJM\KO%
RSWLYXHS[R
4-126
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Gatekeeper and Cisco CallManager Call
Processing
This topic discusses the specifics relative to gatekeeper and Cisco CallManager call processing.
Gatekeeper and Cisco CallManager Call
Processing
GK
IP WAN
Call Admission
1
Call Setup
2
E.164 Lookup
7
Call Setup
Call Admission
8
Ring
Off-Hook
4
Call Setup
9
Alerting
Connect
5
E.164 Lookup
Ringback
Connect RTP Stream
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—4-12
This figure depicts basic call processing using directed signaling between the gatekeeper (“GK”
in this figure and description) and the Cisco CallManager devices acting as gateways.
1. The call takes place between two Cisco CallManager devices over an IP WAN, as follows:
2. Call setup takes place between the IP Phone and the Cisco CallManager.
3. The Cisco CallManager conducts an E.164 lookup and determines that the call is offcluster.
4. Call admission occurs. The Cisco CallManager sends to the GK an ARQ asking for
admission to call the remote IP Phone. GK does a lookup, finds the remote phone is
registered to the remote Cisco CallManager acting as a gateway, and returns an ACF with
the IP address of the remote gateway.
5. Call setup occurs. The local gateway sends a Q.931 call setup message to the remote
gateway with the directory number (DN) of the remote IP Phone.
6. The remote gateway does an E.164 lookup.
7. Call admission occurs on the remote gateway.
8. Call setup occurs as the remote gateway sets up a call to the remote IP Phone.
9. Ringing occurs on the remote IP Phone.
10. Alerting is sent back to the origination gateway.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Network Infrastructure
4-127
11. Ringback is sent back to the originating gateway and passed to the originating IP Phone.
12. The remote phone goes off-hook.
13. A Q.931 connect indication is sent between the gateways.
14. Dual RTP streams are set up between IP Phones; conversation occurs. The gateways send
an IRR to the GK after the call is setup.
4-128
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Cisco SIP Trunks
This topic discusses the specifics related to SIP trunk troubleshooting.
Troubleshooting SIP Trunks
SIP Client
SKINNY Client
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—4-13
A call-processing environment that uses SIP uses SIP trunks to configure a signaling interface
with Cisco CallManager for SIP calls. SIP trunks (or signaling interfaces) connect Cisco
CallManager clusters with a SIP proxy server. A SIP signaling interface uses port-based
routing, and Cisco CallManager accepts calls from any gateway as long as the SIP messages
arrive on the port that is configured as a SIP signaling interface. The SIP signaling interface
uses requests and responses to establish, maintain, and terminate calls (or sessions) between
two or more endpoints.
Before jumping into the troubleshooting portion of Cisco CallManager and SIP trunks, the next
few slides provide a foundation on how the Cisco SIP Proxy Server interacts with Cisco
CallManager 4.0(1).
Cisco CallManager 4.0(1) can have SIP trunk intercluster trunks with other Cisco CallManager
4.0(1) clusters. Although SIP trunks are for the Cisco CallManager to integrate into a SIP
environment, this configuration is supportable and does work. The key in configuring the SIP
trunks between Cisco CallManager servers and a Cisco SIP Proxy Server is to make sure that
the outgoing port of one trunk is the same port number as the incoming port of the far end. Here
are some examples:
Cisco CallManager SIP trunk outgoing port (5060) to incoming port (5060) of Cisco
CallManager SIP trunk port
Cisco CallManager SIP trunk incoming port (5061) to outgoing port (5061) of Cisco
CallManager SIP trunk port
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Network Infrastructure
4-129
This also applies when integrating SIP Proxy Servers. With the Cisco SIP Proxy Server, make
sure that the outgoing transport type is UDP, not TCP. SIP IP Phones only use UDP for
transport. You can however use TCP as transport when integrating Cisco CallManager to Cisco
CallManager SIP trunks.
4-130
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Basic Outgoing Call: IP Phone to SIP
Gateway
MTP
SIP GW
OpenReceiveChannelAck(prerredCodec telephone-event)
OpenReceiveChannelAck(rtpAddress)
INVITE with SDP
100 Trying
180 Ring
plays ringback
200 OK with SDP
ACK
CCM establishes Media
RTP Stream 1
RTP Stream 2
hangs up
CCM releases Media resource
BYE
200 OK
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—4-14
This figure shows the signaling originating from an IP Phone. (In this figure and discussion,
“GW” stands for gateway.) What is not shown in the figure is the Cisco SIP Proxy Server and
how the server locates the SIP end device. The Cisco CallManager media termination point
(MTP) device is required for SCCP devices to communicate with SIP devices.
Here is the call flow scenario:
1. The IP Phone user initiates a call.
2. Cisco CallManager then allocates an MTP resource and opens its RTP port toward the
Cisco SIP Proxy Server.
3. Cisco CallManager sends an INVITE message to the Cisco SIP Proxy Server with the RTP
port information.
4. The Cisco SIP Proxy Server forwards the INVITE message to the SIP GW.
5. The Cisco SIP Proxy Server then receives the 183 progress message from the SIP GW to
inform Cisco CallManager that it has begun to initiate media streaming toward the MTP
RTP port in the initial INVITE message.
6. The Cisco SIP Proxy Server forwards the progress message to Cisco CallManager.
7. Upon receiving the 183 progress message, Cisco CallManager begins to establish a media
connection between the IP phone and MTP and the SIP GW. At this point, the IP Phone
user can hear in-band information (that is, a ringback tone or announcement) from the SIP
network.
8. The PSTN answers the call, which causes a 200 OK (connect) message to be sent to Cisco
CallManager.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Network Infrastructure
4-131
Here are the steps for DTMF generation:
1. The IP Phone user presses the keypad.
2. Cisco CallManager sends an SCCP message to request that the MTP device generate a
DTMF tone.
3. The MTP device constructs DTMF tones according to RFC 2833 and sends them on the
RTP payload to the SIP endpoint.
The important thing about the Cisco SIP trunk at this point is that the MTP device (RFC 2833
compliant) extracts DTMF tones from an IP Phone, which are out-of-band and converts the
tones to in-band and forwards the tones onto the gateway. The same process occurs if the
endpoint is a SIP IP Phone. If the phone was a SIP IP Phone and the gateway was an H.323 or
MGCP, the MTP device is in the middleware to allow DTMF generation. The configuration of
the SIP trunk is critical to the overall success of the Cisco CallManager-to-SIP interaction.
Media Termination Points:
An MTP software device allows Cisco CallManager to relay calls that are routed through SIP
or H.323 endpoints or gateways.
MTP, a Cisco software application, installs on a server during the software installation process.
You must activate and start the Cisco IP Voice Media Streaming Application service on the
server on which you configure the MTP device. For information on activating and starting
services, refer to the Cisco CallManager Serviceability Administration Guide.
Each MTP device that is defined in the database registers with the Media Resource Manager
(MRM). The MRM keeps track of the total available MTP devices in the system and of which
devices have available resources.
During resource reservation, the MRM determines the number of resources and identifies the
media resource type (in this case, the MTP) and the location of the registered MTP device. The
MRM updates its share resource table with the registration information and propagates the
registered information to the other Cisco CallManager servers within the cluster.
4-132
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Basic Incoming Call: SIP Gateway to IP
Phone
MTP
SIP GW
INVITE 12125551001 with SDP
100 Trying
Ringing
180 Ring
Answer
200 OK with SDP
CCM establishes Media
RTP Stream 1
RTP Stream 2
ACK
hangs up
CCM releases Media resource
BYE
200 OK
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—4-15
This figure shows an incoming call to an IP Phone and the related signaling. Again, what is not
shown is the Cisco SIP Proxy Server and the process it used to locate the Cisco CallManager.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Network Infrastructure
4-133
Performance Monitor SIP Counters
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—4-16
Use the Microsoft Windows Performance Monitor to assist in troubleshooting issues related to
Cisco CallManager-to-SIP calls. Cisco recommends that you become familiar with the
performance monitoring counters not only to troubleshoot Cisco CallManager-to-SIP
performance but also to monitor the status of other resources managed by Cisco CallManager.
Listed are the fields related to SIP devices as seen from Cisco CallManager.
The Cisco SIP object provides information about configured SIP devices.
Performance Monitor Counters:
CallsActive: This counter represents the number of calls that are currently active (in use)
on this SIP device.
CallsAttempted: This counter represents the number of calls that have been attempted on
this SIP device, including both successful and unsuccessful call attempts.
CallsCompleted: This counter represents the number of calls that were actually connected
(a voice path was established) from a SIP device. This number increases when the call
terminates.
CallsInProgress: This counter represents the number of calls that are currently in progress
on a SIP device, including all active calls. When all calls that are in progress are connected,
the number of “CallsInProgress” equals the number of “CallsActive.”
If the SIP calls are failing, pay close attention to whether the calls fail at digit entry on the
phone or after all digits have been dialed. If the reorder tone is received after entering the first
digit or two, there is a configuration issue in Cisco CallManager. However, if you receive a
reorder tone after all digits have been entered, start looking at the SIP trunk configuration,
making sure to look at both sides of the trunk. Make sure that the incoming port number
matches that of the far-end destination port number and vice versa. Additionally, make sure that
the MTP server is registered.
4-134
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Performance Monitor MTP Counters
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—4-17
Listed are the fields related to MTP devices as seen from Cisco CallManager:
The Cisco MTP device object provides information about registered Cisco MTP devices.
Performance Monitor Counters:
OutOfResources: This counter represents the total number of times that an attempt was
made to allocate an MTP resource from an MTP device and failed; for example, because all
resources were already in use.
ResourceActive: This counter represents the number of MTP resources that are currently
in use (active) for an MTP device. Each MTP resource uses two streams. An MTP in use
represents one MTP resource that has been allocated for use in a call.
ResourceAvailable: This counter represents the total number of MTP resources that are
not active and are still available to be used for an MTP device. Each MTP resource uses
two streams. An MTP in use represents one MTP resource that has been allocated for use in
a call.
ResourceTotal: This counter represents the total number of MTP resources that an MTP
device provides. This counter equals the sum of the counters ResourceAvailable and
ResourceActive.
Scenario: A SIP client receives a reorder tone when dialing an SCCP client extension. Check
the MTP device within Cisco CallManager and make sure that it is registered. Also check the
Cisco IP Voice Media Streaming Application service to ensure that it has started. In addition,
check the SIP and MTP performance monitor counters.
Use this link to look up the meaning of the counters found in Performance Monitor:
http://www.cisco.com/univercd/cc/td/doc/product/voice/c_callmg/4_0/service/serv401/ccmsrvs
/ssobjctr.htm#25468.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Network Infrastructure
4-135
CCM SIP Traces
SIP Settings
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—4-18
Use Cisco CallManager traces to troubleshoot issues.
The following two new fields have been added to SIP:
SIP Stack Trace
SIP Call Processing Trace
Trace selection will show SIP messages received and sent to the far end. Here is an example of
what you may see:
''1`-RGSQMRK7-4QIWWEKIJVSQ
SRTSVX[MXLF]XIW
7-48V]MRK
:ME7-48'4FVERGL!^L+F/FJGIG
*VSQ7MXIC%C7-4C8VYRO
WMT$"XEK!
8S WMT$"XEK!
(EXI1SR.YR+18
'EPP-(JJHIEEG$
'7IU-2:-8)
%PPS[)ZIRXWXIPITLSRIIZIRX
'SRXIRX0IRKXL
4-136
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting SIP Trunk
Cisco
CallManager
Incorrect
Cisco CallManager
or CSPS
Destination Port 5060
Incoming Port 5060
Incoming Port 5060
Destination Port 5060
Cisco
CallManager
Cisco CallManager
or CSPS
Correct
Destination Port 5060
Incoming Port 5061
Incoming Port 5060
Destination Port 5061
CSPS = Cisco SIP Proxy Server
Outgoing Transport Type:
Cisco CallManager to Cisco CallManager = TCP
Cisco CallManager to CSPS = UDP unless Record Route option is configured
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—4-19
The destination port to the far-end incoming port must match or the SIP calls will only work
one way. This figure shows the incorrect and correct way to configure the SIP trunk destination
and incoming ports.
Between Cisco CallManager servers, use the outgoing transport type TCP, and between the
Cisco SIP Proxy Servers use UDP, unless the Cisco SIP Proxy Server has the Record Route
option configured for TCP; otherwise, calls are set to fail.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Network Infrastructure
4-137
SIP Trunk “Started”
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—4-20
After configuring a SIP trunk in Cisco CallManager, and after the Cisco Proxy Server is
configured correctly, you can look in the Microsoft Windows Event Viewer under application
log and check for a “SIPStarted” alarm; this alarm is normal.
4-138
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Cisco CallManager Listens for SIP
Trunk Ports
IP Port 5060, 5061 and 5062 Listening
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—4-21
To see whether Cisco CallManager is listening to SIP calls on its incoming ports, use the
netstat -a command to look for the ports that you have configured.
To verify the Cisco SIP trunk incoming port numbers, go to the Trunk Configuration page.
What would happen to the calls if you saw that all three ports were the same?
Protocol Analyzer
Incoming Port
5060
Invite
Message
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—4-22
A protocol analyzer can be used to monitor LAN traffic (SIP messages to and from Cisco
CallManager).
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Network Infrastructure
4-139
SIP Service Parameters
Invite retries from
500 ms, 1 s, 2 s, 4 s, 8 s, 16s
Prack retries from 500 ms,
1 s, 2 s, 4 s, 4 s, 4 s, 4 s, 4s
Stay with defaults!
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—4-23
These timers are configurable in the Service Parameter page in the SIP section.
You should accept the defaults because these values have been tested with the Cisco SIP Proxy
Server.
4-140
Timer
Value (Default/Range)
Definition
trying
500 ms/100-1000 ms
The time to wait for a 100
response to an INVITE request
connect
500 ms/100-1000 ms
The time to wait for a 200
response to an acknowledgment
(ACK) request
disconnect
500 ms/100-1000 ms
The time to wait for a 200
response to a BYE request
expires
3 min/1-5 min
Limits the time duration for
which an INVITE is valid
rel1xx
500 ms/100-1000 ms
The amount of time that Cisco
CallManager should wait before
retransmitting the reliable 1xx
responses
prack
500 ms/100-1000 ms
The amount of time that Cisco
CallManager should wait before
retransmitting the PRACK
request
notify
500 ms/100-1000 ms
The amount of time that Cisco
CallManager should wait before
retransmitting the Notify
message
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
SIP Service Parameters (Cont.)
SIP Prack timer
SIP PayLoadType for DTMF
Enable Prack for
Reliable 1xx transmission
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—4-24
Cisco CallManager maintains the configuration data listed here for the SIP retries in the Service
Parameter page in the SIP Parameter section.
C ounter
Invite retry count
Response retry
count
D efault
V alue
5
6
Sug gest
Range
D efin ition
1 – 10
N umber of INV ITE
retries.
1 – 10
N umber of RESPO NSE
retries.
Bye retry co unt
10
1 – 10
Num ber of BYE retries.
Cancel retry count
10
1 – 10
Num ber of Cancel retries.
PRA CK retry
count
6
1 – 10
N umber of PRA CK
retries.
1 – 10
N umber of Reliable 1x x
respon se retries.
1 - 10
N umber of N OTIFY
retries.
Rel1x x retry cou nt
N otify retry count
Copyright © 2004, Cisco Systems, Inc.
10
6
Troubleshooting Network Infrastructure
4-141
Summary
This topic summarizes the key points discussed in this lesson.
Summary
• Gatekeepers provide CAC and bandwidth control.
• RAS signaling is done using the Directed Route Signaling.
• RAS signaling for gateways and Cisco CallManager servers are
the same.
• RAS messages are sent using H.225 messages over UDP.
• End devices establish call setup, not the gatekeeper.
• Cisco CallManager uses port 1720 to listen to H.323 devices.
• After Open Logical Channel and OLC ACK, the IP address and
port number are sent from the other side, then the audio stream
is established.
• To troubleshoot gatekeepers, gateways and Cisco CallManager,
check the zones and technology prefix matches on the
gatekeeper.
• Two types of Cisco CallManager gatekeeper-controlled trunks
are ICT and H.225.
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—4-25
Summary (Cont.)
• Active capabilities and waiting for the other site to initiate the
capabilities check box is new in Cisco CallManager 4.0(1).
• Gateway and Cisco CallManager call rejection issues can be seen
using various show and debug commands.
• An MTP device is used to allow a Cisco CallManager SCCP device
to talk with SIP Phones.
• The MTP check box is on by default and cannot be unchecked.
• SIP Phones send DTMF payload in-band; IP Phones send tone
out-of-band; MTP converts payload so Cisco CallManager can
understand the tones.
• Use Performance Monitor as a troubleshooting tool for all Cisco
CallManager issues, including MTP and SIP devices.
• Use Cisco CallManager traces when troubleshooting Cisco
CallManager-to-SIP call setup.
• Use the netstat -a command to see if Cisco CallManager is listening
to incoming SIP trunk ports.
© 2004 Cisco Systems, Inc. All rights reserved.
4-142
Cisco IP Telephony Troubleshooting (IPTT) v4.0
IPTT v4.0—4-26
Copyright © 2004, Cisco Systems, Inc.
References
For additional information, refer to these resources:
H.323 gatekeepers; gateways with Cisco CallManager:
http://www.cisco.com/en/US/tech/tk652/tk701/technologies_tech_note09186a00800a8928.shtml;
and
http://www.cisco.com/cgi-bin/Support/browse/psp_view.pl?p=Technologies:H323&viewall=true.
RFC 2833: http://www.faqs.org/rfcs/rfc2833.html.
MTP configuration in Cisco CallManager 4.0(1):
http://www.cisco.com/univercd/cc/td/doc/product/voice/c_callmg/4_0/sys_ad/4_0_1/ccmcfg/b04mtp
.htm.
Cisco SIP trunk configuration:
http://www.cisco.com/univercd/cc/td/doc/product/voice/c_callmg/4_0/sys_ad/4_0_1/ccmcfg/b06trun
k.htm#1118192.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Network Infrastructure
4-143
Quiz
Use the practice items here to review what you learned in this lesson. The correct answers are
found in the Quiz Answer Key.
Q1)
Is the exchange of terminal capabilities a function of the gatekeeper?
A)
B)
Q2)
Which RAS messages are sent by one gatekeeper to another gatekeeper when trying to
locate a destination endpoint?
A)
B)
C)
D)
Q3)
UDP
RTP
PHP
TCP
RAS H.225 uses which protocol for transport?
A)
B)
C)
D)
4-144
RTP
HTTP
UDP
TCP
H.245 uses which protocol for transport?
A)
B)
C)
D)
Q7)
ARJ over H.225 (UDP)
ARQ over H.245 (TCP)
ARQ over RTP (UDP
ARQ over H.225 (UDP)
H.225 call setup uses which protocol for transport?
A)
B)
C)
D)
Q6)
RRQ
DRQ
ARJ
RJR
After a gateways sent an RRQ message to the gatekeeper and received an RCF
message, which is the next RAS message expected to be sent by the gateway?
A)
B)
C)
D)
Q5)
RRQ/RCF
ARQ/ACF
LRQ/LCF
DRQ/DCF
If registration of a gateway to a gatekeeper was rejected, which RAS message is sent
by the gatekeeper?
A)
B)
C)
D)
Q4)
No; however, gateways would not be allowed to exchange capabilities unless
they were granted entry into the network by the gatekeeper.
Yes; however, gateways would not be allowed to exchange capabilities unless
they were granted into the network by the gatekeeper.
TCP
UDP/TCP
IP/TCP
UDP
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Q8)
After Open Logical Channel ACK is sent from the far end, what additional information
is sent?
A)
B)
C)
D)
Q9)
A gatekeeper is required in a VoIP network.
A)
B)
Q10)
ICT and H.245
ICT and H.225
ICT only
H.225 only
With gatekeeper-controlled ICT, active caps are off by default.
A)
B)
Q15)
Registration Request Time to Live value
Registration Retry Time Out
Enable Device, and Registration Time to Live value
none of the above
Which two kinds of trunks are used for gatekeeper control?
A)
B)
C)
D)
Q14)
issue a reset from Cisco CallManager
issue “reload” on the gatekeeper
neither A nor B
both A and B
How does a Cisco CallManager server register with its gatekeeper? (Hint: Gatekeeper
Configuration page)
A)
B)
C)
D)
Q13)
show gatekeeper endpoint
show gateway
debug h225 asn1
debug ras
both C and D
What is the easiest way to get Cisco CallManager to register with its registered
gatekeeper?
A)
B)
C)
D)
Q12)
true
false
If you are having calls rejected by the registered gatekeeper of Cisco CallManager,
which command would you use to show registration, admission and status messages in
real time?
A)
B)
C)
D)
E)
Q11)
IP address and port number
caller ID and port number
port number only
IP address only
true
false
Which Cisco CallManager device allows for SCCP-to-SIP communication?
A)
B)
C)
D)
MTP device
Cisco SIP Proxy Server
SIP trunk and MTP device registered with Cisco CallManager
none of the above
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Network Infrastructure
4-145
Q16)
Which monitoring tools can be used to view INVITE packets on the wire?
A)
B)
C)
D)
4-146
Cisco CallManager traces
protocol analyzer
Microsoft Performance Monitor
Microsoft Windows Event Viewer
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Quiz Answer Key
Q1)
A
Q2)
C
Q3)
D
Q4)
D
Q5)
D
Q6)
D
Q7)
D
Q8)
A
Q9)
A or B
Q10)
E
Q11)
A
Q12)
C
Q13)
B
Q14)
A or B
Q15)
C
Q16)
B
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Network Infrastructure
4-147
4-148
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Module 5
Troubleshooting Voice Quality
Issues
Overview
Isolating the source of voice quality issues is one of the most difficult problems that you can
face when troubleshooting an IP telephony network. Voice quality can be subjective, and data
networks are not sensitive to the same impairments or limitations that Voice over Data
networks are sensitive to. This module concentrates on maintaining the quality of the voice
traffic that crosses the data network and focuses on how to troubleshoot echo.
Module Objectives
Upon completing this module, you will be able to resolve quality of service (QoS) issues in a
complex IP telephony network using effective and appropriate troubleshooting and
implementation methods.
Module Objectives
• Describe common QoS issues with Cisco QoS
tools
• Troubleshoot common VoIP issues using the
methodologies recommended by Cisco
• Resolve echo problems in a VoIP network
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—5-2
Module Outline
The outline lists the components of this module.
Module Outline
• Resolving Voice QoS Issues with Cisco QoS Tools
• Troubleshooting Voice over IP Quality Problems
• Troubleshooting Echo
© 2004 Cisco Systems, Inc. All rights reserved.
5-2
Cisco IP Telephony Troubleshooting (IPTT) v4.0
IPTT v4.0—5-3
Copyright © 2004, Cisco Systems, Inc.
Resolving Voice QoS Issues
with Cisco QoS Tools
Overview
Before QoS can be configured in a network, it is important to understand just what QoS is and
why it is useful in solving the different problems that arise when different traffic types are
converged into a single network infrastructure. The basic concepts and key terminology of QoS
are explained in this lesson. Also included in this lesson are the three steps involved in
implementing a QoS policy and special QoS considerations for LANs.
Relevance
To understand the more technical aspects of network QoS, it is first important to understand the
basic concepts of QoS and to be able to define some key QoS terms.
Objectives
Upon completing this lesson, you will be able to:
Define the term “quality of service” with respect to traffic in a network
Identify the four key quality issues with converged networks
Explain the QoS requirements of common types of network applications
Define the term “QoS policy”
List and explain the key steps involved in implementing a QoS policy on a network
Identify QoS considerations of LAN switches
Learner Skills and Knowledge
To benefit fully from this lesson, you must have these prerequisite skills and knowledge:
Basic knowledge of internetworking with TCP/IP concepts
Outline
The outline lists the topics included in this lesson.
Outline
•
•
•
•
•
•
•
•
•
•
•
•
•
Overview
Quality of Service Defined
Converged Networks
Converged Networks: Quality Issues
Lack of Bandwidth
End-to-End Delay
Packet Loss
QoS Requirements
QoS Policy
QoS for Converged Networks
LAN QoS Considerations
Summary
Quiz
© 2004 Cisco Systems, Inc. All rights reserved.
5-4
Cisco IP Telephony Troubleshooting (IPTT) v4.0
IPTT v4.0—5-3
Copyright © 2004, Cisco Systems, Inc.
Quality of Service Defined
This topic defines the term “QoS.”
Quality of Service Defined
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—5-4
QoS is defined as “the ability of the network to provide better or ‘special’ service to selected
users or applications or both to the detriment of other users or applications or both.”
Cisco IOS QoS features enable network administrators to control and predictably service a
variety of networked applications and traffic types, thus allowing network managers to take
advantage of a new generation of media-rich and mission-critical applications.
The goal of QoS is to provide better and more predictable network service by providing
dedicated bandwidth, controlled jitter and latency, and improved loss characteristics. QoS
achieves these goals by providing tools for managing network congestion, shaping network
traffic, using expensive wide-area links more efficiently, and setting traffic policies across the
network. QoS offers intelligent network services that, when correctly applied, help to provide
consistent, predictable performance.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Voice Quality Issues
5-5
Converged Networks
This topic explains why QoS was not important in nonconverged networks.
Converged Networks:
Network Before Convergence
Traditional data traffic characteristics:
• Bursty data flow
• First-come, first-served access
• Mostly not time sensitive – delays OK
• Brief outages survivable
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—5-5
Historically, network engineering has been focused on connectivity. Different traffic types
(data, voice, video, and so on) have different network requirements and traffic characteristics.
Not too long ago, few tools existed to handle the differing needs of these traffic types, forcing
network engineers to build separate networks to handle these traffic requirements. Separate
networks lead to higher equipment, installation, and operating costs and require a larger support
staff.
For traditional data networks supporting applications such as file transfer or e-mail, the rates at
which data come onto the network resulted in bursty data flows. The data arrives in packets and
tries to grab as much bandwidth as it can at any given time. The access is very egalitarian;
access is on a first-come, first-served basis, so whoever gets there first, gets the bandwidth.
As a result of this somewhat anarchic way of attacking the network, the data rate is adaptive to
network conditions.
The protocols that have been developed for data networks adapt to the bursty nature of data
networks, and brief outages are survivable. Typically, if retrieving e-mail, a delay of a few
seconds is generally not noticeable. A delay of minutes is annoying, but not serious.
5-6
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Converged Networks:
Network After Convergence
Converged traffic characteristics:
• Constant small packet voice flow competes with
with bursty data flow
• Critical traffic must get priority
• Voice and video are time-sensitive
• Brief outages not acceptable
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—5-6
The graphic shows a converged network in which voice, video, and data traffic use the same
network facilities. Merging different traffic streams with dramatically differing requirements
can lead to a number of problems.
While packets carrying voice traffic are typically very small, they cannot tolerate delay and
delay variation as they traverse the network or voice quality will suffer. Voices will break up
and words will become incomprehensible.
On the other hand, packets carrying file transfer data are typically large and can survive delays
and drops. It is possible to retransmit part of a dropped file, but it is not feasible to retransmit a
part of a conversation.
The constant, but small packet voice flow competes with bursty data flows. Unless some
mechanism mediates the overall flow, voice quality will suffer terribly at times of network
congestion. The critical voice traffic must get priority.
Voice and video traffic is very time-sensitive. Voice and video traffic cannot be delayed, and
neither can be dropped without the resulting quality of voice and video suffering.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Voice Quality Issues
5-7
Converged Networks: Quality Issues
This topic describes the basic quality issues presented by converged networks.
Converged Networks:
Quality Issues
• Phone call: “I can’t understand you; your voice is
breaking up.”
• Teleconferencing: “The picture is very jerky.
Voice is not synchronized.”
• Brokerage house: “I needed that information two
hours ago. Where is it?”
• Call center: “Please hold while my screen
refreshes.”
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—5-7
With inadequate preparation of the network, voice transmission is choppy or unintelligible.
Gaps in speech are particularly troublesome where pieces of speech are interspersed with
silence, and speech literally disappears. In voice-mail systems, this silence is a problem. For
example, you dial 68614. In a situation where the gaps in speech are actually gaps in the tone,
68614 becomes 6688661144, because the gaps in speech are perceived as pauses in the touch
tones.
Poor caller interactivity is the consequence of delay. It causes two problems—echo and talker
overlap. Echo is caused by the signal reflections of the voice of the speaker from the far-end
telephone equipment back into the ear of the speaker. Talker overlap (or the problem of one
talker stepping on the speech of the other talker) becomes significant if the one-way delay
becomes greater than 250 milliseconds (ms). If bad, calls go to “walkie-talkie” mode.
Disconnected calls are the worst cases. If there are long gaps in speech, people hang up, or if
there are signaling problems, calls are disconnected. Such events are completely unacceptable
in the voice world yet are quite common for an inadequately prepared data network that is
attempting to carry voice.
Multimedia streams, such as those used in IP telephony or video conferencing, may be
extremely sensitive to delivery delays, creating unique QoS demands on the underlying
networks that carry them. When packets are delivered using the “best-effort” delivery model,
they may not arrive in order, in a timely manner, or at all. The result is unclear pictures, jerky
or slow movement or both, and sound not synchronized with the image.
5-8
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Converged Networks:
Quality Issues (Cont.)
Video Lack ing
Prope r QoS
• Lack of bandwidth: Multiple flows compete for a
limited amount of bandwidth.
• End-to-end delay (fixed and variable): Packets
have to traverse many network devices and links
that add up to the overall delay.
• Variation of delay (jitter): Sometimes there is a lot
of other traffic, which results in more delay.
• Packet loss: Packets may have to be dropped
when a link is congested.
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—5-8
The five big problems facing converged enterprise networks are as follows: bandwidth
capacity, delay issues, variable delay, variation of delay (also called jitter), and packet loss.
Large graphic files, multimedia uses, and increasing use for voice and video cause bandwidth
capacity problems over data networks.
Delay is the time it takes for a packet to reach the receiving endpoint after being transmitted
from the sending endpoint. This time is termed the “end-to-end delay,” and it consists of two
components: fixed network delay and variable network delay. Jitter is the delta, or difference,
in the total end-to-end delay values of two voice packets in the voice flow.
Two types of fixed delay are serialization and propagation delays. Serialization is the process of
placing bits on the circuit. The higher the circuit speed, the less time it takes to place the bits on
the circuit. Therefore, the higher the speed of the link, the less serialization delay is incurred.
Propagation delay is the time it takes for frames to transit the physical media.
Processing delay is a type of variable delay, and is the time required by a networking device to
look up the route, change the header, and complete other switching tasks. In some cases, the
packet also must be manipulated. For example, the encapsulation type or the hop count must be
changed. Each of these steps can contribute to the processing delay.
Queuing delay, another type of variable delay, is the time a packet spends in a queue or buffer
before being processed. Packets may be queued by routers or switches on an ingress interface
or egress interface or both. Queuing delay can be significant if a rate change occurs or if many
interfaces are aggregated into a single uplink.
Loss of packets is usually caused by congestion in the WAN, resulting in speech dropouts or a
stutter effect if the play-out side tries to accommodate by repeating previous packets.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Voice Quality Issues
5-9
Lack of Bandwidth
This topic explains how a lack of bandwidth can have an adverse impact on QoS in a network
and describes ways to effectively increase bandwidth on a link.
Lack of Bandwidth
Bad Voice Due to
Lack of BW
BWmax = min (10 MB, 256 k, 512 k, 100 M) = 256 kbps
BWavail = BWmax /Flows
• The maximum available bandwidth equals the bandwidth of the
weakest link
• Multiple flows are competing for the same bandwidth, resulting in
much less bandwidth being available to one single application
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—5-9
Bandwidth must be considered on the entire communication path between source and
destination. The example illustrates an empty network with four hops between a server and a
client. Each hop is using different media with a different bandwidth. The maximum available
bandwidth is equal to the bandwidth of the slowest link. So although the workstation has 10
Mbps of bandwidth, packets flowing between these devices must cross the slow-speed WAN
link at 256 kbps.
It is rare on a computer network that only a single communication flow will be present on the
network at a given time. In reality, multiple communication flows are competing for the same
bandwidth. Therefore, the calculation of the available bandwidth is much more complex in
cases where multiple flows are traversing the network. The calculation of the available
bandwidth in the illustration is a rough approximation.
5-10
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Ways to Increase Available
Bandwidth
• Upgrade the link. (Best solution, but also the most expensive)
• Forward the important packets first.
• Compress the payload of Layer 2 frames (it takes time).
• Compress the header of IP packets.
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—5-15
The best approach is to increase the link capacity to accommodate all applications and users
with some extra bandwidth to spare. This solution sounds simple enough, but in the real world,
it brings a high cost in terms of the money and time it takes to implement. Very often there are
also technological limitations to upgrading to a higher bandwidth.
Another option is to classify traffic into QoS classes and prioritize it according to importance
(business-critical traffic should get enough bandwidth, voice should get enough bandwidth, and
prioritized forwarding and the least important traffic should get the remaining bandwidth).
There are a wide variety of mechanisms available in the Cisco IOS Software that provide
bandwidth guarantees. Here are some examples:
Priority queuing (PQ) or custom queuing (CQ)
Class-based weighted fair queuing (CBWFQ)
Low latency queuing (LLQ)
LLQ is the preferred bandwidth guarantee mechanism in a Voice over IP (VoIP) network. LLQ
establishes a strict PQ for voice packets and CBWFQ for other traffic classes.
Optimizing link usage by compressing the payload of frames (virtually) increases the link
bandwidth. Compression, on the other hand, also increases delay because of the complexity of
compression algorithms. Using hardware compression can accelerate the compression of packet
payloads. Stacker and Predictor are two compression algorithms available in Cisco IOS
software.
Another link efficiency mechanism is header compression. This mechanism is especially
effective in networks where most packets carry small amounts of data (the payload-to-header
ratio is small). Typical examples of header compression are TCP header compression and RealTime Transport Protocol (RTP) header compression.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Voice Quality Issues
5-11
End-to-End Delay
This topic explains how end-to-end delay can have an adverse impact on QoS in a network and
describes ways to effectively reduce delay.
End-to-End Delay
Bad Voice Due to
Delay Variation
Delay = P1 + Q1 + P2 + Q2 + P3 + Q3 + P4 = X ms
• End-to-end delay equals a sum of all propagation, processing,
and queuing delays in the path.
• Propagation delay is fixed; processing and queuing delays are
unpredictable in best-effort networks.
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—5-16
Delay must be considered over the entire communication path, end to end. Therefore, the total
end-to-end delay is the sum total of all delay experienced over a communication path between a
sender and a receiver. The figure illustrates the impact that a network has on end-to-end delay.
Each hop in the network adds to the overall delay because of these factors:
Propagation delay is caused by the speed of light traveling in the media (for example, speed
of light traveling in fiber optics or copper media).
Serialization delay is the time it takes to clock all the bits in a packet onto the wire. This is
a fixed value that is a function of the link bandwidth.
Processing and queuing delays within a router are caused by a wide variety of conditions.
People generally ignore propagation delay, but it can be significant (about 40 ms coast to coast
over optical). Ping is one way to measure the round-trip time (RTT) of IP packets in a network.
5-12
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Example: Effects of Delay
Customer routers in New York and San Francisco are connected by a 128-kbps WAN link. The
customer sends a 66-B voice frame across the link. To transmit the frame (528 bits) takes 4.125
ms to clock out (serialization delay), but the last bit will not arrive until 40 ms after it clocks
out (propagation delay). The total delay is 44.125 ms. Change the circuit to a T1—528 bits
takes 0.344 ms to clock out (serialization delay), and the last bit arrives 40 ms after
transmission (propagation delay) for a total delay of 40.344 ms. In this case, the significant
factor is propagation delay. In the same situation but between Seattle and San Francisco,
serialization delay remains the same and propagation delay drops to around 6 ms, making 528
bits take 10.125 (128 K link) and 6.344 (T1 link). As you can see, both serialization and
propagation delays must be taken into account.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Voice Quality Issues
5-13
Ways to Reduce Delay
• Upgrade the link. (Best solution, but also the most expensive)
• Forward the important packets first.
• Compress the payload of Layer 2 frames (it takes time).
• Compress the header of IP packets.
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—5-25
Assuming that a router is powerful enough to make a forwarding decision in a negligible time,
it can be said that most of the processing, queuing, and serialization delay is influenced by the
following factors:
Average length of the queue
Average length of packets in the queue
Link bandwidth
Here are several approaches to accelerate packet dispatching of delay-sensitive flows:
Increase link capacity. Enough bandwidth causes queues to shrink, making sure packets do
not have to wait long before they can be transmitted. Additionally, more bandwidth reduces
serialization time. On the other hand, this might be an unrealistic approach because of the
costs associated with the upgrade.
A more cost-effective approach is to enable a queuing mechanism that can give priority to
delay-sensitive packets by forwarding them ahead of other packets. There are a wide
variety of queuing mechanisms available in Cisco IOS Software that have preemptive
queuing capabilities. Here are some examples:
—
PQ
—
CQ
—
Class-Based LLQ
Payload compression reduces the size of packets and, therefore, virtually increases link
bandwidth. Additionally, compressed packets are smaller and need less time to be
transmitted. On the other hand, compression uses complex algorithms that take time and
add to the delay. This approach is, therefore, not used to provide low-delay propagation of
packets.
5-14
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Header compression, on the other hand, is not as CPU-intensive and can be used in
combination with other mechanisms to reduce delay. It is especially useful for voice
packets that have a bad payload-to-header ratio, which is improved by reducing the header
of the packet (RTP header compression). By minimizing delay, jitter is also reduced (delay
is more predictable).
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Voice Quality Issues
5-15
Packet Loss
This topic explains how packet loss can have an adverse impact on QoS in a network and
describes ways to manage packet loss so that QoS is not affected.
Packet Loss
Bad Voice Due
to Packet Loss
• Tail drops occur when the output queue is full. These are common
drops that happen when a link is congested.
• Many other types of drops exist, usually the result of router
congestion, that are uncommon and may require a hardware upgrade
(input drop, ignore, overrun, frame errors).
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—5-26
The usual packet loss occurs when routers run out of buffer space for a particular interface
(output queue). The figure illustrates a full output queue of an interface, which causes newly
arriving packets to be dropped. The term used for such drops is simply “output drop” or “tail
drop” (packets are dropped at the tail of the queue).
Routers might also drop packets for other (less common) reasons. Here are some examples:
Input queue drop: The - main CPU is congested and cannot process packets (the input
queue is full).
Ignore: The router ran out of buffer space.
Overrun: The CPU is congested and cannot assign a free buffer to the new packet.
Frame errors (CRC, runt, giant): There is a hardware-detected error in a frame.
5-16
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Ways to Prevent Packet Loss
• Upgrade the link. (Best solution, but also the most expensive)
• Guarantee enough bandwidth to sensitive packets.
• Prevent congestion by randomly dropping less important packets before
congestion occurs.
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—5-31
Packet loss is usually a result of congestion on an interface. Most applications that use TCP
experience a slow down because of TCP adjusting to the network resources (dropped TCP
segments cause TCP sessions to reduce their window sizes). There are some other applications
that do not use TCP and cannot handle drops (fragile flows).
The following approaches can be taken to prevent drops of sensitive applications:
Increase link capacity to ease or prevent congestion.
Guarantee enough bandwidth and increase buffer space to accommodate bursts of fragile
applications. There are several mechanisms available in Cisco IOS Software that can
guarantee bandwidth or provide prioritized forwarding to drop-sensitive applications, or
both. Here are some examples:
—
PQ
—
CQ
—
IP RTP prioritization
—
CBWFQ
—
LLQ
Prevent congestion by dropping other packets before congestion occurs. Weighted random
early detection (WRED) can be used to start dropping other packets before congestion
occurs.
Here are some other mechanisms that can also be used to prevent congestion: traffic shaping
delays packets instead of dropping them (generic traffic shaping, Frame Relay traffic shaping,
and class-based shaping).
Traffic policing can limit the rate of less important packets to provide better service to dropsensitive packets (committed access rate and class-based policing).
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Voice Quality Issues
5-17
QoS Requirements
The following topic describes the QoS traffic requirements for voice, video, and data traffic.
QoS Traffic Requirements: Voice
• Latency –
< 150 ms*
• Jitter –
< 30 ms*
• Loss –
< 1 percent*
• 17-106 kbps guaranteed
priority bandwidth
per call
• 150 bps (+ Layer 2
overhead) guaranteed
bandwidth for voicecontrol traffic per call
*One-way requirements
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—5-32
Voice traffic has extremely stringent QoS requirements. Voice traffic generally generates a
smooth demand on bandwidth and has minimal impact on other traffic as long as it is managed.
While voice packets are typically small (60 B to 120 B), they cannot tolerate delay or drops.
The result of delays or drops is poor, and often unacceptable, voice quality. Because drops
cannot be tolerated, User Datagram Protocol (UDP) is used to package voice packets, because
TCP retransmit capabilities have no value.
Voice packets can tolerate no more than a 150-ms delay (one-way requirement) and less than 1
percent packet loss.
A typical voice call will require between 17 kbps to 106 kbps of guaranteed priority bandwidth
plus an additional 150 bps per call for voice-control traffic. Multiplying these bandwidth
requirements times the maximum number of calls expected during the busiest time period will
provide an indication of the overall bandwidth required for voice traffic.
5-18
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
QoS Traffic Requirements:
Videoconferencing
• Latency –
< 150 ms
• Jitter –
< 30 ms
• Loss –
< 1 percent
• Minimum priority
bandwidth guarantee
required is:
– Video stream + 20 percent
– For example: 384-kbps
stream would require 460
kbps of priority bandwidth
*One-way requirements
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—5-33
Much like voice, videoconferencing applications also have very stringent QoS requirements.
However, videoconferencing traffic is often bursty and greedy in nature and, as a result, can
have an impact on other traffic. As a result, it is important to understand the videoconferencing
requirements for a network and to provision carefully for videoconferencing.
The minimum bandwidth for a videoconferencing stream would require the actual bandwidth of
the stream—dependent upon the type of videoconferencing coder-decoder (codec) being
used—plus some overhead. For example, a 384-kbps video stream would actually require a
total of 460 kbps of priority bandwidth.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Voice Quality Issues
5-19
QoS Traffic Requirements: Data
• Different applications have
different traffic characteristics.
• Different versions of the same
application can have different
traffic characteristics.
• Classify data into a relativepriority model with no more than
four to five classes:
– Mission-critical applications:
Locally defined critical
applications
– Transactional: Interactive
traffic, preferred data service
– Best effort: Internet, e-mail,
unspecified traffic
– Less than best effort
(scavenger): Napster, Kazaa,
peer-to-peer applications
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—5-34
The QoS requirements for data traffic vary greatly.
Different applications (for example, a human resources application vs. an ATM application)
may make greatly different demands on the network. Even different versions of the same
application may have varying network traffic characteristics.
While data traffic can demonstrate either smooth or bursty characteristics depending upon the
application, data traffic differs from voice and video in terms of delay and drop sensitivity.
Almost all data applications can tolerate some delay and generally can tolerate high drop rates.
Because data traffic can tolerate drops, the retransmit capabilities of TCP become important
and, as a result, many data applications use TCP.
In enterprise networks, important (business-critical) applications are usually easy to identify.
Most applications can be identified based on TCP or UDP port numbers. Some applications use
dynamic port numbers, which makes classification somewhat more difficult. Cisco IOS
software supports network-based application recognition (NBAR), which can be used to
recognize dynamic port applications.
It is recommended that data traffic be classified into no more than four to five classes as
described in the figure. There will still remain additional classes for voice or video or both.
5-20
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
QoS Policy
This topic describes QoS policy.
QoS Policy
A network-wide
definition of the
specific levels of
quality of service
assigned to different
classes of network
traffic
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—5-35
A QoS policy is defined as “a network-wide definition of the specific levels of QoS assigned to
different classes of network traffic.”
Having a QoS policy is just as important in a converged network as is having a security policy.
A written and public QoS policy allows users to understand and negotiate for QoS in the
network.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Voice Quality Issues
5-21
QoS Policy (Cont.)
Align Network Resources with Business Priorities
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—5-36
The graphic shows how a QoS policy could be defined for a network for three different traffic
types as follows:
Enterprise resource planning (ERP): ERP applications have a high QoS priority and
must be available all the time to support replication between systems.
Video: Video applications are guaranteed 100 kbps of bandwidth, but can only operate
between 9:00 A.M. and 5:00 P.M. on weekdays.
Voice: Voice traffic is guaranteed less than 150 ms of delay in each direction; however,
that QoS guarantee is limited to the hours between 9:00 A.M. to 5:00 P.M. on weekdays
because there are no interoffice calls during nonbusiness hours. Toll calls are completely
restricted to avoid personal long-distance calls.
5-22
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
QoS for Converged Networks
This topic describes the steps to create a QoS policy.
Step 1: Identify Traffic and Its
Requirements
• Network audit
– Identify traffic on the
network.
• Business audit
– Determine how each
type of traffic is
important for
business.
• Service levels required
– Determine the
required response
time.
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—5-37
The first step in implementing QoS is to identify the traffic on the network and determine the
QoS requirements for the traffic.
Determine what users perceive the QoS problems to be. Measure the traffic on the network
during congested periods. Conduct CPU utilization assessment on the network devices of each
user during busy periods to determine when problems might be occurring.
Determine the business model and business goals, and obtain a list of business requirements.
This will help you to define the number of classes and to determine the business requirements
for each class of traffic.
Define the service levels required by different classes of traffic in terms of response time and
availability. What is the impact on business if a transaction is delayed by 2 or 3 seconds? Can
file transfers wait until the network is quiescent?
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Voice Quality Issues
5-23
Step 2: Divide the Traffic into Service
Classes
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—5-38
Once the majority of network traffic has been identified and measured, use the business
requirements to define classes of traffic.
Voice traffic, because of its stringent QoS requirements, will almost always exist in a class by
itself. Cisco Systems has developed specific QoS mechanisms, such as LLQ, that ensure that
voice always receives priority treatment over all other traffic.
Once the applications with the most critical requirements have been defined, the remaining
traffic classes are defined using the business requirements.
Example: Traffic Classification
For example, a typical enterprise might define five traffic classes as follows:
Voice: Absolute priority for VoIP traffic
Mission-critical applications: Small set of locally defined critical business applications
Transactional: Database access, transaction services, interactive traffic, preferred data
services
Best effort: Internet, e-mail
Less than best effort (scavenger): Napster, Kazaa, and other point-to-point applications
5-24
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Step 3: Define Policies for Each Service
Class
• Set minimum
bandwidth guarantee.
• Set maximum
bandwidth limits.
• Assign priorities to
each class.
• Manage congestion.
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—5-39
Finally, define a QoS policy for each class of service (CoS) defined. Defining a QoS policy
involves the following:
Setting a minimum bandwidth guarantee
Setting a maximum bandwidth limit
Assigning priorities to each class
Using QoS technologies, such as advanced queuing, to manage congestion
Example: Defining QoS Policies
For example, using the CoS defined thus far, QoS policies could be determined as:
Voice: Minimum bandwidth 1 Mbps; use QoS marking to mark voice packets as priority 5,
use LLQ to always give voice priority.
Mission critical: Minimum bandwidth 1 Mbps; use QoS marking to mark critical data
packets as priority 4, use CBWFQ to prioritize critical-class traffic flows.
Best effort: Maximum bandwidth 500 kbps, use QoS marking to mark these data packets
as priority 2, use CBWFQ to prioritize best-effort class traffic flows below mission-critical
and voice classes.
Less than best effort: Maximum bandwidth 100 kbps, use QoS marking to mark less-thanbest-effort data packets as priority 0, use WRED to drop these packets whenever the
network has a propensity for congestion.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Voice Quality Issues
5-25
LAN QoS Considerations
This topic describes LAN QoS considerations.
LAN QoS Considerations
• Bandwidth is typically not an issue.
• Buffer congestion is an issue.
• Buffer congestion occurs when there is a rate change or if
many interfaces are aggregated to a single uplink.
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—5-40
Until recently, the conventional wisdom has been that QoS was not an issue in an enterprise
campus network where bandwidth is plentiful. As applications such as IP telephony and
videoconferencing, and mission-critical data applications have been implemented in the
campus, it has become evident that buffer management, not just bandwidth, is an issue that
must be addressed. QoS functions are required to manage bandwidth and buffers to minimize
loss, delay, and delay variation.
In campus LANs, serialization delay is not a significant concern. The amount of time required
for LAN interfaces to serialize the bits of packets onto the physical media is negligible; it is not
significant enough to affect delay-sensitive applications. Additionally, in LANs, propagation
delay is of little concern because LANs by their very nature are not geographically dispersed.
The type of delay that is present in LANs is variation in delay, or jitter. This can adversely
affect voice and video quality by introducing packet loss through jitter buffer over-runs and
under-runs.
An additional contributor to packet loss in campus networks is transmit (Tx) buffer congestion.
Tx buffer congestion can happen if a rate change occurs or if many interfaces are aggregated
into a single uplink, resulting in an oversubscription of the capacity of the uplink to buffer
packets.
5-26
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
The bits of a traffic flow that run through a high-speed campus network serialize into and out of
switches at different rates, depending on the link speed of the physical interfaces that the bits of
traffic flow are traversing. When traffic serializes into a campus switch at gigabit speeds and is
switched to a 100-Mb interface, the switch must have buffering capabilities to hold, or queue,
the bits while it waits to transmit them. When a Tx buffer fills, ingress interfaces are not able to
place new traffic into the Tx buffer of the target interface. When the switch cannot place a
packet into the Tx queue because of Tx buffer congestion or exhaustion, packet drops will
occur.
Using multiple queues on the Tx interface minimizes the potential for dropped or delayed
traffic caused by Tx buffer congestion. By separating voice, video, and mission-critical data
(which are all sensitive to loss, delay, and delay variation) into their own queues, you can
prevent flows from being dropped at the ingress interface even when Tx buffer congestion is
experienced. You can also minimize delayed transmission due to non-QoS sensitive traffic
congestion by servicing the QoS sensitive queues in a priority fashion.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Voice Quality Issues
5-27
Summary
This topic summarizes the key points discussed in this lesson.
Summary
• Quality of service is the ability of the network to
provide better or “special” service to users or
applications or both.
• Converged networks create new requirements for
managing network traffic.
• Converged networks suffer from different quality
issues, including lack of adequate bandwidth,
end-to-end and variable delay, and lost packets.
• Many technologies exist today that can overcome
the problems presented by a lack of bandwidth,
delay, variable delay, and packet loss.
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—5-41
Summary (Cont.)
• Voice, video, and data have very different QoS
requirements to run effectively on a network.
• A QoS policy is a network-wide definition of the
specific levels of QoS assigned to classes of
network traffic.
• Building QoS requires three steps: identifying
requirements, classifying network traffic, and
defining network-wide policies for quality.
© 2004 Cisco Systems, Inc. All rights reserved.
5-28
Cisco IP Telephony Troubleshooting (IPTT) v4.0
IPTT v4.0—5-42
Copyright © 2004, Cisco Systems, Inc.
References
For additional information, refer to this resource:
For more information on implementing QoS, go to the following URL:
http://www.cisco.com/en/US/partner/tech/tk543/tk757/technologies_white_paper09186a00
8017f93b.shtml.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Voice Quality Issues
5-29
Quiz
Use the practice items here to review what you learned in this lesson. The correct answers are
found in the Quiz Answer Key.
Q1)
Which three types of traffic can typically tolerate packets being dropped? (Choose
three.)
A)
B)
C)
D)
Q2)
Which three of the following issues are key quality issues specifically for converged
networks? (Choose three.)
A)
B)
C)
D)
Q3)
assigning priorities to each class
setting a maximum bandwidth limit
setting a minimum bandwidth guarantee
combining traffic classes to go under one policy
Quality of service is defined as “the ability of the network to _____.”
A)
B)
C)
D)
5-30
compress headers
forward the most important packets first
upgrade bandwidth on the links
aggressively drop packets
Which three of the following tasks does QoS policy involve? (Choose three.)
A)
B)
C)
D)
Q6)
drop-sensitive
smooth, constant flow
benign, does not affect other traffic
relies on TCP to handle packet loss
What are three ways to reduce delay for time-sensitive packets in a network? (Choose
three.)
A)
B)
C)
D)
Q5)
lack of bandwidth
end-to-end delay
variable delay
propagation delay
Which three of the following represent characteristics of voice traffic? (Choose three.)
A)
B)
C)
D)
Q4)
file transfers
voice
e-mail
HTTP
improve the quality of voice transmission
offer end-to-end circuits with preferred priority
provide special services to users and applications
consistently move priority packets to the front of queues
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Quiz Answer Key
Q1)
A, C, D
Q2)
A, B, C
Q3)
A, B, C
Q4)
A, B, C
Q5)
A, B, C
Q6)
C
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Voice Quality Issues
5-31
5-32
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Voice over IP
Quality Problems
Overview
After devices establish VoIP calls, you must verify the voice quality of these calls. This lesson
will teach you about the considerations associated with achieving good voice quality.
Additionally, you will learn about the common causes of, and solutions for, voice quality
problems and useful commands to help identify the causes of these problems.
Relevance
Because voice quality trouble can be difficult to identify, understanding problem isolation
procedures is critical for voice quality troubleshooting.
Objectives
Upon completing this lesson, you will be able to:
Discuss potential voice quality problems and troubleshoot using the methodologies
recommended by Cisco
Describe how and when to use the Quality Report Tool
List the potential problems regarding Layer 2 voice quality using the troubleshooting
methodology recommended by Cisco
List the potential problems regarding Layer 3 voice quality using the troubleshooting
methodology recommended by Cisco
Demonstrate the effect of VAD on bandwidth and list the Cisco IOS commands to reliably
transmit DTMF tones
Learner Skills and Knowledge
To benefit fully from this lesson, you must have the following prerequisite skills and
knowledge:
Completion of Building Cisco Remote Access Networks (BCRAN), or
Completion of Deploying QoS for Enterprise Networks (DQOS)
Outline
The outline lists the topics included in this lesson.
Outline
• Overview
• Identifying and Isolating Voice Quality Problems
• Quality Report Tool for IP Phones
• Troubleshooting Layer 2 Voice Quality Problems
• Troubleshooting Layer 3 Voice Quality Problems
• Additional Considerations
• Summary
• Quiz
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—5-3
Refer to these links on Cisco.com when troubleshooting voice QoS issues.
http://www.cisco.com/cgiin/Support/browse/psp_view.pl?p=Technologies:Voice:QoS&s=Verification_and_Troubleshooting.
The topics available at this URL are as follows:
One-way voice:
—
http://www.cisco.com/en/US/partner/tech/tk652/tk698/technologies_tech_note09186a00
8009484b.shtml.
Choppy voice:
—
http://www.cisco.com/en/US/partner/tech/tk652/tk698/technologies_tech_note09186a00
800f6cf8.shtml.
CallManager H.323: One-way Voice Issue after Transfer or Hold:
—
http://www.cisco.com/en/US/partner/products/sw/voicesw/ps556/products_tech_note09
186a0080115a52.shtml.
Echo:
—
http://www.cisco.com/en/US/partner/tech/tk652/tk698/technologies_tech_note09186a00
80149a1f.shtml.
Troubleshooting Hissing and Static:
—
5-34
http://www.cisco.com/en/US/partner/tech/tk652/tk698/technologies_tech_note09186a00
800a9982.shtml.
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Identifying and Isolating Voice Quality Problems
Voice quality problems can be difficult to resolve. The problems may be intermittent, and
quality evaluation can be subjective. However, some tools and service aids, as addressed in this
topic, can help identify the potential causes and probable solutions of voice quality problems.
Troubleshooting VoIP Quality Problems
• Ensure call connects first
• Check voice quality next
– Adequate bandwidth for codec
– Do not exceed Frame Relay CIR
– Use fragmentation as needed
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—5-4
After devices establish a VoIP call, you must verify that the voice quality meets expectations.
You should keep background information and guidelines in mind when addressing voice
quality. For example, consider how much bandwidth a VoIP call consumes with each codec,
including Layer 2 and IP/ UDP/RTP headers.
You must understand the characteristics of the IP network used by the voice traffic.
Example
Keeping the bandwidth of a Frame Relay network under the committed information rate (CIR)
is better than exceeding the CIR (bursting), where the Frame Relay service provider could drop
or queue the voice packets. Take care to ensure that you control or eliminate delay and jitter as
much as possible. One-way delay should not exceed 150 ms (per the International
Telecommunication Union Telecommunication Standardization Sector [ITU-T] G.114
recommendation).
Use a queuing technique, such as LLQ, to allow your router to identify and prioritize VoIP
traffic.
Consider using fragmentation techniques when sending voice over slow-speed links (such as,
less than 768 kbps).
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Voice Quality Issues
5-35
Cisco supports Layer 2 fragmentation methods including the following:
Multilink PPP (MLP) for VoIP over PPP circuits
Frame Relay Forum (FRF).12 for VoIP over Frame Relay (VoIPovFR) circuits
FRF.11 Annex C for Voice over Frame Relay (VoFR) networks
Fragmentation of larger data packets reduces jitter and delay in sending VoIP traffic, because
the router can interleave the VoIP packets onto the link.
5-36
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
RTCP Call Information
Displays RTCP
call statistics
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—5-5
Start troubleshooting voice quality problems with the real-time data available during the call
from 79xx phones. The Cisco IP Phone 7940 and Cisco IP Phone 7960 display the RTP Control
Protocol (RTCP) call information by rapidly pressing the “i” button (or on some other models,
the question mark [?] button) twice. Displayed information includes statistics, such as the
following:
Sent and received packet counts, because the IP Phone opened the RTP stream
Send and receive stream type (for example, the codec being used)
Average and maximum jitter, because the IP Phone started the stream
Number of receive packets lost in transit
Number of receive packets discarded after being received
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Voice Quality Issues
5-37
Packet Loss
Normal
Frame
Relay
Packet Loss
© 2004 Cisco Systems, Inc. All rights reserved.
X
IPTT v4.0—5-6
Packet loss can have a major impact on voice quality. The most likely cause of packet loss is
congestion.
When analyzing packet loss, first determine the direction in which the devices are losing
packets. If there are multiple routers between the source and the destination, it may be
necessary to go to each router to discover where the problem is occurring. Issuing the show
interface command on the appropriate interface or interfaces can help you locate the source of
the problem. You may need to implement queuing to correct this problem. Consider
compressed RTP (cRTP) as another option for low-speed, high-traffic links.
Another possible cause of packet loss is a dirty line where interference corrupts the packets
during transmission, resulting in cyclic redundancy check (CRC) errors on the receiving
interface.
To check for a packet loss problem, you can use an extended ping operation. Check small
packet sizes and large packet sizes. It is possible that one router has a buffer problem, such as
occasionally running out of buffers, or perhaps the buffers need tuning.
5-38
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Jitter
Normal
Frame
Relay
Jitter
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—5-7
Jitter can also affect voice quality. Many factors can cause jitter. Congestion is the most likely
cause of jitter, in addition to being the most probable cause of packet loss. Congestion can
result in voice packets having to remain in the queue too long, causing the router to send them
without the proper interpacket gap. The solution is usually to configure a queuing method, such
as LLQ. For some companies, upgrading Cisco IOS software to support LLQ is not an
alternative. In this case, consider CBWFQ with IP RTP priority. If you already have queuing
configured, check the traffic classification statements.
CBWFQ was created to address the scaling issue of queuing based on microflow classification.
CBWFQ uses access lists to control how to classify traffic and uses bandwidth percentages to
control how classes are serviced, which gives a network administrator greater flexibility of both
IP and non-IP traffic. While CBWFQ does offer several advantages over weighted fair queuing
(WFQ), it still holds the same basic premise that all classes must be serviced in a relatively
“fair” manner in comparison to other classes. This can introduce jitter and latency for delaysensitive traffic, such as VoIP. Enhancements to CBWFQ were added to allow a PQ to be
defined within CBWFQ. These enhancements are called LLQ.
Routing problems can also cause jitter, such as two paths going to the same destination and the
router sending the voice packets across alternating paths. Ensure that your routing tables reflect
the best path to each destination.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Voice Quality Issues
5-39
Latency
Normal
Frame
Relay
Latency
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—5-8
Although you cannot see excessive delay or latency at the IP Phone or in the Call Management
Record (CMR) data, it can cause voice quality problems. If your network latency varies, it will
show up as jitter. If the delay is constant but too long, there may be either a Frame Relay issue
or a suboptimal routing issue.
Frame Relay networks have an inherent delay. However, when you correctly provision the
network, it is set up for optimal response. Over time, the path (through the various packet
switches) within the network can change, and the delay can increase. Benchmarking the
network and then periodically monitoring the response time can reduce the potential for this
condition.
5-40
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Quality Report Tool for IP Phones
The topic discusses the Quality Report Tool (QRT).
Quality Report Tool
• The Quality Report Tool (QRT) is a voice quality
and general problem-reporting tool for
Cisco CallManager IP Phones.
• QRT is a feature that extends to IP Phones as an
Microsoft Windows NT service.
• The Cisco Extended Functions service supports
the QRT feature.
• Make sure that the Cisco Extended Functions
service is activated.
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—5-9
QRT is installed as part of the Cisco CallManager installation. You can configure the
Cisco IP Phones of the users with QRT so that users can report problems with a phone call
during the call. Issues are reported using a Cisco IP Phone soft key labeled QRT. QRT is
supported for the Cisco IP Phone 7940 and Cisco IP Phone 7960. The QRT soft key is available
only when the IP Phone is in the Connected, Connected Conference, Connected Transfer,
and/or OnHook states.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Voice Quality Issues
5-41
Configuring QRT
Cisco CallManager 3.3, 4.0
• Copy softkey template from the standard user
template.
• Add QRT from “Connected” and “Connected
Conference.”
• Modify IP Phone softkey template.
• Ensure that Extended Services function is
activated.
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—5-10
Here are the steps to create QRT:
Step 1
Choose Device > Device Setting > Softkey Template
Step 2
Click the Add a New Softkey Template button
Step 3
Copy a softkey template. Use the standard user template and name the template
“Standard User with QRT”; choose Insert.
Step 4
Click the Configure Softkey Layout button on the right side of the web page.
On the left side of the web page, you will see a list of Call States. You are looking
for the “Connected” and “Connected Conference” call states.
5-42
Step 5
Choose the Connected call state. The Quality Report Tool (22) (QRT) will appear
in the Unselected Softkey” window. Click on the Quality Report Tool (22) (QRT)
button and move it from the left window to the right window using the arrows
between the windows. You should see Quality Report Tool (22) (QRT) in the
Selected Softkeys window.
Step 6
Use the same process for the Connected Conference call state. After you move the
Quality Report Tool (22) (QRT), you should see that tool in the Selected Softkey
window.
Step 7
To verify that you added a soft key, click on the Softkey Template Configuration
link. You should see the newly created “Standard User with QRT” listed as a softkey
template.
Step 8
You can now assign the softkey template to phones. The location to modify the
phones softkey template is under the heading Softkey Template Information in the
phone configuration screen. Your newly created template can be selected from the
pull-down menu. You will need to update and reset the phone for the soft key to
appear and start working on the phone.
Step 9
Make sure that the Cisco Extended Functions (CEF) service is activated.
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Configuring QRT
© 2004 Cisco Systems, Inc. All rights reserved.
Step 1
IPTT v4.0—5-11
From the Cisco CallManager Administration window, choose
Application > Cisco CallManager Serviceability.
The Cisco CallManager Serviceability window displays.
Step 2
Choose Tools > Phone Problem Reports Viewer.
The IP Phone Problem Reporting window displays.
Note
Because QRT service activation is configurable, the user may have changed the server. By
default, this drop-down menu highlights the current server where QRT is active.
Step 3
Choose the available Cisco CallManager server for which you would like to view a
problem report.
Step 4
Enter the “From” date in the “Date” box; for example, 2/2/2002.
Step 5
Enter the “To” date in the “Date” box; for example 2/3/2002.
Step 6
In the From Time selection box, click the Down arrow for the beginning hour,
minute, and second of the problem reporting information that you want to collect;
for example, 3 hour, 45 minute, 0 second.
Step 7
In the To Time selection box, click the Down arrow for the ending hour, minute, and
second of the problem reporting information that you want to collect; for example,
23 hour, 59 minute, 59 second.
Step 8
Click the Get Logs button.
Step 9
From the Extension Number drop-down menu, choose the extension number or
numbers that you want to be included in the report.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Voice Quality Issues
5-43
Step 10
From the Device drop-down menu, choose the device or devices that you want
included in the report.
Step 11
From the Category drop-down menu, choose the problem category that you want
included in the report.
Step 12
From the List of Fields box, choose the fields that you want included in the report
and click the Down arrow to place them in the Selected Fields box.
Step 13
Click the Display Records button to view the report in your browser.
Configuring QRT (Cont.)
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—5-12
This figure illustrates the result of choosing Display Records. The report content will depend on
the fields that you selected to report.
5-44
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Layer 2 Voice Quality Problems
With physical connectivity established, another logical troubleshooting target is Layer 2 of the
Open Systems Interconnection (OSI) reference model. This topic examines Layer 2 bottlenecks
and what can be done to overcome these network limitations.
Identifying Layer 2 Bottlenecks
• Bandwidth is typically not an issue.
• Buffer congestion is an issue.
• Buffer congestion occurs when there is a rate change or if
many interfaces are aggregated to a single uplink.
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—5-13
Until recently, conventional wisdom stated that QoS is not an issue in the enterprise campus
because of the bursty nature of data traffic and the capability to withstand buffer overflow and
packet loss. However, with the introduction of latency-sensitive applications, buffers, not
bandwidth, are the issue within the campus. Buffers can fill instantaneously. When this occurs,
the router can drop packets when they attempt to enter the interface buffer. For applications,
such as voice, this results in voice quality degradation.
Another troubleshooting target is speed mismatches. Where a Gigabit Ethernet segment flows
into an Ethernet segment, oversubscription is likely. Also, consider a situation in which you
have a server connected to a Fast Ethernet switch port, and multiple clients connected to Fast
Ethernet switch ports. With this configuration, clients could saturate the Fast Ethernet link to
the server if they all send simultaneously for an extended period.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Voice Quality Issues
5-45
Monitoring Performance of
Layer 2 Devices
Incoming Packet
Drop Packets with CoS
6 and 7
Threshold 4
Threshold 3
Threshold 2
Threshold 1
Data
Data
Data
Data
Data
Data
Data
CoS
4 and 5
Drop Packets with CoS
2 and 3
Drop Packets with CoS
0 and 1
Buffer
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—5-14
At times of traffic congestion, Layer 2 switch buffers start to fill up. When this happens, the
switch can either let the buffers overflow or drop packets to keep the buffers from overflowing.
The switch can use WRED to intelligently drop lower-priority traffic, keeping higher-priority
traffic in the queue. The “weighted” part of WRED looks at the CoS value of an Ethernet
frame. After the switch reaches a certain threshold, it drops frames with specified CoS values.
Thresholds are typically configurable by the administrator. Different switches implement a
different number of thresholds, and you must check the switch documentation. Cisco Catalyst
5x00 and 6x00 line cards that support WRED also allow the administrator to configure which
CoS values map to which thresholds. In the figure, threshold 4 has CoS values 6 and 7 mapped
to it. You could easily configure CoS 5 to map to threshold 4, if required.
As an example of the syntax used, on a Catalyst 6500 Series switch, running in hybrid mode,
you can view these WRED thresholds with the command show qos info. You can configure the
WRED thresholds with the set qos wred command. However, refer to current product
documentation for the Catalyst switch that you are troubleshooting.
5-46
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Selecting Layer 2 QoS Tools
Frame enters
switch
Apply
port
CoS
Apply
port
CoS
Yes
Port
untrusted?
ISL or
802.1Q?
No
Yes
Yes
No
Get CoS
from
ISL/dot1o
Yes
Port is set to
trust-cos
Port set
to trustipprec?
Port set to
trust-dscp?
No
No
To switching
engine
Input Scheduling
Port Trust States:
•
•
•
•
© 2004 Cisco Systems, Inc. All rights reserved.
trust-cos
trust-ipprec
trust-dscp
untrusted
IPTT v4.0—5-15
There are several marking approaches, such as CoS, IP precedence, and differentiated services
code point (DSCP). Based on these markings, you can configure some Cisco Layer 2 Catalyst
switches to make forwarding and dropping decisions. Specifically, you can configure Ethernet
ports with trust states that determine which of those markings the switch uses to make
forwarding and dropping decisions. You can also configure the switch port as “untrusted,”
meaning that regardless of the CoS, IP precedence, or DSCP marking, the switch treats the
frame with the CoS value that you have configured for the port. The figure details how the
switch evaluates an incoming frame based on the trust state of the port.
Some Catalyst switches use tools such as WRED and weighted round-robin (WRR), a queuing
method used by some high-end switches to make dropping and forwarding decisions.
Therefore, a Layer 2 troubleshooting step is to determine what marking, if any, the switch
considers. Remember that a Layer 2 CoS marking does not pass through a router.
You may have a Layer 2 switch configured for WRED. However, the frame entering the switch
may have passed through a router on its way to the switch. If this occurs, the router hop
stripped the CoS marking of the frame. Therefore, frames may not be entering the switch with
the anticipated Layer 2 CoS marking.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Voice Quality Issues
5-47
Troubleshooting Layer 3 Voice Quality Problems
Moving up the OSI reference model stack, this topic addresses the next voice quality
troubleshooting targets, existing at Layer 3.
Identifying Layer 3 Bottlenecks
Normal
Layer 3
Bottleneck
Frame
Relay
Latency
Jitter
X
Packet Loss
Insufficient WAN bandwidth causes:
• Buffer overflows
• Queuing delay
• Serialization delay
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—5-16
In an enterprise network, Layer 3 congestion troubleshooting targets typically exist at the WAN
edge. The bandwidth available to routers sitting at this WAN boundary may be insufficient to
transport packets in a timely fashion. As a result, packets may experience the following:
Buffer overflows: If insufficient bandwidth exists for a router to send traffic out on the
wire, the router attempts to buffer packets until sufficient bandwidth does exist. However,
if the buffer fills to capacity, the router discards newly arriving packets. Cisco calls this
occurrence “tail drop.”
Delay: If multiple traffic flows attempt to simultaneously exit a router interface when
bandwidth is not available, some packets temporarily remain in queue while the router
sends other packets. The packets waiting in the queue experience delay. A large packet
exiting a slow-speed (less than 768 kbps) link is another source of delay (serialization
delay).
Variable delay: The causes of delay, as described, also introduce variable interpacket
arrival times (jitter).
5-48
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Monitoring Performance of
Layer 3 Devices
6SYXIVWLS[ MRXIVJEGIWVERHSQHIXIGX
*EWX)XLIVRIXUYIYIWM^I
TEGOIXWSYXTYXHVSTW
;6)(UYIYIEZIVEKI
[IMKLX
4VIGIHIRGIQMRXLVIWLSPHQE\XLVIWLSPHQEVO[IMKLX
TEGOIXWSYXTYXHVSTWVERHSQXLVIWLSPH
4VIGIHIRGIQMRXLVIWLSPHQE\XLVIWLSPHQEVO[IMKLX
RSXVEJJMG
4VIGIHIRGIQMRXLVIWLSPHQE\XLVIWLSPHQEVO[IMKLX
TEGOIXWSYXTYXHVSTWVERHSQXLVIWLSPH
4VIGIHIRGIQMRXLVIWLSPHQE\XLVIWLSPHQEVO[IMKLX
RSXVEJJMG
4VIGIHIRGIQMRXLVIWLSPHQE\XLVIWLSPHQEVO[IMKLX
RSXVEJJMG
4VIGIHIRGIQMRXLVIWLSPHQE\XLVIWLSPHQEVO[IMKLX
RSXVEJJMG
4VIGIHIRGIQMRXLVIWLSPHQE\XLVIWLSPHQEVO[IMKLX
TEGOIXWSYXTYXHVSTWVERHSQXLVIWLSPH
4VIGIHIRGIQMRXLVIWLSPHQE\XLVIWLSPHQEVO[IMKLX
RSXVEJJMG
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—5-17
To prevent the previously described tail-drop scenario, you can enable the WRED feature.
WRED at Layer 3, similar to WRED at Layer 2, prevents a buffer from overflowing by
dropping packets based on their IP precedence or DSCP markings. The figure shows output
from the show interfaces random-detect command. This command can give you an idea of
how aggressively the router is discarding packets, which could indicate the need for more
bandwidth. You can use the show queuing command to display the random early detection
(RED) profile (drop thresholds) for each interface, which may need modification based on your
knowledge of network traffic patterns.
The queuing method for voice traffic recommended by Cisco is LLQ, which uses class maps
and policy maps. You can use the command show class-map to determine how the router is
classifying traffic. The show policy-map command displays how much bandwidth the router is
reserving for each class and which class the router is directing into the priority queue. The
output from these commands can give you insight into whether you are allocating network
bandwidth efficiently.
Link fragmentation and interleaving (LFI) tools (for example, MLP, FRF.12, and FRF.11
Annex C) minimize the impact of serialization delay. If you configure an interface for MLP,
use the show interfaces command to verify that interleaving is functioning. You can use the
show frame-relay fragment command to monitor FRF.12 or FRF.11 Annex C fragmentation.
The output from these commands might indicate that the router is not performing LFI when it
should be (such as on slower-speed links), or perhaps the router is performing LFI on higherspeed links and consuming extra bandwidth because of additional header information being
sent.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Voice Quality Issues
5-49
Selecting Layer 3 QoS Tools
Maximum
Threshold
Gold
Silver
Bronze
Queue
40%
Priority Delivery
25%
Guaranteed Bandwidth
10%
Best Effort
LLQ
PBX
Minimum
Threshold
WRED
1500-byte Packet Frag Frag Frag 1500-byte Packets
PBX
Cisco 3600
WAN
Cisco 3600
Headquarters
Larger Branch Office
LFI
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—5-18
Cisco offers the following QoS tools to address Layer 3 bottlenecks:
WRED: WRED prevents multiple TCP flows from going into TCP slow start, which leads
to global synchronization. Global synchronization wastes bandwidth for TCP flows.
Therefore, when troubleshooting a scenario in which you are monitoring multiple drops of
TCP packets, WRED might be an appropriate solution. Keep in mind that voice uses UDP,
and that you should not apply WRED to voice queues. For example, if you configure LLQ,
the policy map should enable WRED for data classes only.
LLQ: LLQ has the ability to distinguish between multiple traffic types (up to 64) and
reserve an amount of bandwidth for each of those traffic classifications. One or more of the
classes can have their traffic sent to a priority queue. Therefore, if you are troubleshooting
a situation in which the network is not giving voice traffic sufficient bandwidth, you can
use LLQ to classify voice traffic (perhaps based on an IP precedence value) and give a
specified amount of priority bandwidth to the voice traffic.
LFI: On circuits with speeds slower than 768 kbps, large data packets could cause
excessive latency for voice packets. Choose an appropriate LFI tool (for example, MLP,
FRF.12, or FRF.11 Annex C) for the media, and configure the fragment size to result in
approximately 10 ms to 20 ms of serialization delay. As a rule of thumb, you can take the
link speed and divide by 800 to obtain the fragment size, in bytes, that would yield a
serialization delay of 10 ms. For example, if the link speed were 64,000 bps, the fragment
size that would exit the interface in 10 ms is 80 bytes (64,000/800 = 80 bytes).
Consider the figure shown. The VoIP network has enough bandwidth to support two
simultaneous calls, but not three. If you do not prevent a third call, the voice quality of all calls
will suffer because of oversubscription of the bandwidth. Therefore, when troubleshooting
situations in which too many calls are causing voice quality issues, consider using one or more
Call Admission Control (CAC) tools. The goal of CAC tools is to permit a call only if there are
sufficient resources to support the call.
5-50
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Additional Considerations
Voice quality problems result from many different causes. Therefore, you should consider
multiple solutions, as addressed in this topic.
Additional Considerations
• Voice activity detection
• DTMF relay
• Frame Relay CIR
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—5-19
This figure shows other related feature that can cause voice quality issues.
Voice Activity Detection (VAD): Most customers who run voice over their WAN to
remote sites want to use as little bandwidth as possible without sacrificing voice quality.
This usually means using a compressed codec such as G.729; however, some clients choose
to gain additional bandwidth savings by using voice activity detection (VAD), also known
as silence suppression. You can achieve substantial bandwidth savings using VAD, but
VAD has its downside. For example, after a period of silence during a phone call, the IP
Phone or voice gateway must detect the beginning of normal speech and restart the
transmission of voice packets. If the IP Phone or gateway does not detect the beginning of a
talk pattern, the beginning of the talk pattern can be truncated or cut off sounding like
clipping. If you are experiencing the sound effects caused by VAD, disable VAD in the
Cisco CallManager server and under VoIP dial peers. Use the no vad command under VoIP
dial peers and set Silence Suppression and Silence Suppression of Gateways to false in the
System Parameters.
DTMF relay: The last activity that H.245 is used for is DTMF relay. VoIP networks
typically do not carry DTMF tones in-band, because tones can be distorted when
transported over compress codec technologies such as G.729. To resolve DTMF digit
recognition issues, DTMF tones are carried via the signaling channel instead of the voice
channel. Cisco CallManager sends DTMF tones using h245 signal format, but will only
accept DTMF using h245-alphanumeric. Use the h245-alphanumeric command under
VoIP dial peers.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Voice Quality Issues
5-51
Frame Relay CIR: When using Frame Relay to transport voice over the WAN, ensure the
following:
—
bc is 1/100 of CIR
—
be is set to zero
—
Fragment data packets to interleave with voice packets
—
Apply LLQ to the interfaces
Example configuration:
JVEQIVIPE]GMV
JVEQIVIPE]FG
JVEQIVIPE]FI
JVEQIVIPE]QMRGMV
JVEQIVIPE]JVEKQIRX
WIVZMGITSPMG]SYXTYX(*;XS7.'
GPEWWQETQEXGLEPP:3-')'860398
QEXGLMTHWGTEJ
GPEWWQETQEXGLEPP:3-')398
QEXGLMTHWGTIJ
GPEWWQETQEXGLEPP:3-')
QEXGLEGGIWWKVSYTREQI:3-')
GPEWWQETQEXGLEPP:3-')'860
QEXGLEGGIWWKVSYTREQI:3-')'860
TSPMG]QET(*;XS7.'
GPEWW:3-')398
TVMSVMX]
GPEWW:3-')'860
FERH[MHXLTIVGIRX
GPEWWGPEWWHIJEYPX
JEMVUYIYI
TSPMG]QET-2&392(
GPEWW:3-')
WIXHWGTIJ
GPEWW:3-')'860
WIXHWGTEJ
When troubleshooting voice quality problems, try using a different codec, and try the call with
VAD enabled and disabled. This could help isolate the problem down to the digital signal
processor (DSP), as opposed to the IP network.
If voice quality is acceptable but keypad tones do not work properly, add the dtmf-relay (dialpeer configuration mode) command to the routers. This command allows routers to transport
DTMF tones out of band.
5-52
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Commands
• show interface
• show access-lists
• show class-map
• show frame-relay fragment
• show policy-map
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—5-21
In addition to the dtmf-relay command, other commands to help isolate the cause of voice
quality problems include the following:
show interface: To look for bad packets, packets dropped, potential congestion (see
example)
show access-lists: To ensure that the router classifies the appropriate traffic
show class-map: To look at how traffic is being classified
show frame-relay fragment: To see Frame Relay fragmentation information
show policy-map: To examine policy details
As an example, consider the following output from the show interface serial x/y command:
84%C,5WLMRXIVW
7IVMEPMWYTPMRITVSXSGSPMWYT
,EVH[EVIMW4S[IV59-''7IVMEP
189F]XIW&;/FMX(0=YWIG
VIPMEFMPMX]X\PSEHV\PSEH
)RGETWYPEXMSR*6%1)6)0%=PSSTFEGORSXWIX
/IITEPMZIWIX WIG 01-IRUWIRX01-WXEXVIGZH01-YTHVIGZH(8)
01-YT
01-IRUVIGZH01-WXEXWIRX01-YTHWIRX
01-(0'-01-X]TIMW%27-%RRI\(JVEQIVIPE](8)
*67:'HMWEFPIH0%4*WXEXIHS[R
&VSEHGEWXUYIYIFVSEHGEWXWWIRXHVSTTIH
MRXIVJEGIFVSEHGEWXW
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Voice Quality Issues
5-53
0EWXMRTYXSYXTYXSYXTYXLERKRIZIV
0EWXGPIEVMRKSJWLS[MRXIVJEGIGSYRXIVW
-RTYXUYIYI WM^IQE\HVSTWJPYWLIW 8SXEPSYXTYX
HVSTW
5YIYMRKWXVEXIK][IMKLXIHJEMV
3YXTYXUYIYI WM^IQE\XSXEPXLVIWLSPHHVSTW 'SRZIVWEXMSRW EGXMZIQE\EGXMZIQE\XSXEP 6IWIVZIH'SRZIVWEXMSRW EPPSGEXIHQE\EPPSGEXIH QMRYXIMRTYXVEXIFMXWWIGTEGOIXWWIG
QMRYXISYXTYXVEXIFMXWWIGTEGOIXWWIG
TEGOIXWMRTYXF]XIWRSFYJJIV
6IGIMZIHFVSEHGEWXWVYRXWKMERXWXLVSXXPIW
MRTYXIVVSVW'6'JVEQISZIVVYRMKRSVIH
EFSVX
TEGOIXWSYXTYXF]XIWYRHIVVYRW
SYXTYXIVVSVWGSPPMWMSRWMRXIVJEGIVIWIXW
SYXTYXFYJJIVJEMPYVIWSYXTYXFYJJIVWW[ETTIHSYX
GEVVMIVXVERWMXMSRW
DCD=up DSR=up DTR=up RTS=up CTS=up
The bold fields are of interest when attempting to identify the cause of a voice quality problem.
As an example, consider the following output from the show policy-map interface
fastethernet x/y command:
(*;+;WLS[TSPMG]QETMRXJE
*EWX)XLIVRIX
7IVZMGITSPMG]MRTYX-2&392(
'PEWWQET:3-') QEXGLEPP TEGOIXWF]XIW
QMRYXISJJIVIHVEXIFTWHVSTVEXIFTW
1EXGLEGGIWWKVSYTREQI:3-')
5S77IX
HWGTIJ
4EGOIXWQEVOIH
'PEWWQET:3-')'860 QEXGLEPP TEGOIXWF]XIW
QMRYXISJJIVIHVEXIFTWHVSTVEXIFTW
1EXGLEGGIWWKVSYTREQI:3-')'860
5S77IX
HWGTEJ
4EGOIXWQEVOIH
5-54
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
'PEWWQETGPEWWHIJEYPX QEXGLER] TEGOIXWF]XIW
QMRYXISJJIVIHVEXIFTWHVSTVEXIFTW
1EXGLER]
Notice that there are no inbound drops on any of the call maps. Use these commands to monitor
QoS. It is a good idea to monitor the policy maps to see if you have any dropped packets. If you
have dropped packets, take the time to research the issue.
QoS Recommendations
Campus
Branch Office
PSTN
SRST
Router
IP WAN
Building Access
Building Distribution
•
Multiple queues
•
Multiple queues
•
LLQ
•
LLQ
•
Multiple queues
•
802.1Q/p
•
802.1Q/p
•
CBWFQ
•
CBWFQ
•
802.1Q/p
•
DSCP
•
DSCP
•
WRED
•
WRED
•
Fast link
convergence
•
LFI/FRF.12
•
LFI/FRF.12
•
cRTP
•
cRTP
•
FRTS, dTS
•
FRTS
•
DSCP
•
802.1Q/p
•
DSCP
•
NBAR
© 2004 Cisco Systems, Inc. All rights reserved.
Copyright © 2004, Cisco Systems, Inc.
WAN Module
Branch Router
Branch Switch
IPTT v4.0—5-20
Troubleshooting Voice Quality Issues
5-55
Summary
This topic summarizes the key points discussed in this lesson.
Summary
• Some models of Cisco IP Phones can display RTCP
information, which can be useful in troubleshooting voice
quality. The main voice quality problems that you face are
packet loss, jitter (variable delay), and latency.
• When troubleshooting voice quality at Layer 2, consider
queuing configurations on switches.
• To combat loss, delays, and jitter at Layer 3, consider
queuing, WRED, and LFI mechanisms.
• Besides protecting voice from data, voice quality also
requires that you protect voice from other voice traffic.
• Other voice quality troubleshooting targets include VAD,
DTMF relay, and Frame Relay traffic shaping.
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—5-22
References
For additional information, refer to these resources:
Frame Relay Fragmentation for Voice: http://www.cisco.com/public/support/tac/.
QoS Classification and Marking on Catalyst 6000 Family Switches Running in Hybrid
Mode: http://www.cisco.com/public/support/tac/.
Cisco CallManager softkey template configuration:
http://www.cisco.com/univercd/cc/td/doc/product/voice/c_callmg/4_0/sys_ad/4_0_1/ccmcf
g/index.htm.
Configuring PFC QoS:
http://www.cisco.com/univercd/cc/td/doc/product/lan/cat6000/121_8aex/swconfig/qos.htm.
VoIP Call Admission Control:
http://www.cisco.com/univercd/cc/td/doc/cisintwk/intsolns/voipsol/cac.htm.
CAC tools:
http://www.cisco.com/univercd/cc/td/doc/cisintwk/intsolns/voipsol/cac.htm.
5-56
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Quiz
Use the practice items here to review what you learned in this lesson. The correct answers are
found in the Quiz Answer Key.
Q1)
Which link fragmentation and interleaving tool is useful for VoIPovFR?
A)
B)
C)
D)
Q2)
Which two of the following are valid port trust states? (Choose two.)
A)
B)
C)
D)
Q3)
trust-ip
trust-cos
trust-dscp
trusted
Which command displays the RED profile (for example, drop thresholds) for each
interface on a router?
A)
B)
C)
D)
Q4)
MLP
FRF.11 Annex C
FRF.12
FRF 3.1
show queuing
show interface
show queue
show wred
What is the function of the dtmf-relay command?
A)
B)
C)
D)
The dtmf-relay command sends DTMF tones out of band.
The dtmf-relay command is required for tandem-switched VoFR connections.
The dtmf-relay command allows proprietary PBX protocols to be sent through
a Cisco router.
The dtmf-relay command is an electromechanical device inside voice-enabled
routers and is used to translate touch-tone digits into pulse digits.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Voice Quality Issues
5-57
Quiz Answer Key
5-58
Q1)
C
Q2)
B, C
Q3)
A
Q4)
A
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Echo
Overview
Echo is one of the more difficult voice quality problems to troubleshoot and it is one of the
most commonly encountered problems. Echo is present in almost any telephony network. IP
telephony, in particular, makes echo problems more apparent than they might be in a nonpacket
network.
Relevance
Before identifying echo in any IP telephony network, you first must know the possible causes
and the impact that echo has on an IP telephony network. Gathering key information regarding
the type and kind of echo is critical to isolate echo, reduce echo, or eliminate echo.
Objectives
In completing this module, you will be able to:
Identify what makes echo a QoS issue
Describe the sources and types of echo
List the characteristics of the echo canceller
List the techniques for measuring echo on different platforms
Describe how to adjust levels for solving echo issues
Learner Skills and Knowledge
To benefit fully from this lesson, you must have these prerequisite skills and knowledge:
Fundamental understanding of Cisco IP telephony networks
Be familiar with Catalyst software and Cisco IOS routers
Outline
The outline lists the topics included in this lesson.
Outline
• Overview
• Defining in an IP Telephony Network Echo
• Sources and Types of Echo
• Defining the Echo Canceller
• Measuring in an IP Telephony Network Echo
• Adjusting in an IP Telephony Network Echo
• Summary
• Quiz
© 2004 Cisco Systems, Inc. All rights reserved.
5-60
Cisco IP Telephony Troubleshooting (IPTT) v4.0
IPTT v4.0—5-3
Copyright © 2004, Cisco Systems, Inc.
Defining Echo in an IP Telephony Network
This topic discusses the specifics related to echo.
Where Can Echo Occur?
?
?
?
VM/UM
Cisco
CallManager
?
PSTN
Branch B
?
VM/UM
Cisco
CallManager
?
?
IP WAN
Headquarters
?
VM/UM
Cisco
CallManager
?
VM = voice messaging
UM = unified messaging
What is the probability of Echo?
Branch C
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—5-4
In a new deployment, the probability of experiencing echo is high, but if you know where to
look, you can reduce and eliminate echo quickly.
IP Phone to IP Phone echo is rare; however, when speakerphones are used and in close
proximity to one another, echo is usually heard. This occurs because of acoustic feedback
through the phone speaker and is heard when the speakerphone volume is set for 75 percent. IP
Phone loads can sometimes cause an issue with echo, but with the latest loads, the bugs,
relative to echo, have been eliminated.
The most common echo is talker echo, which is heard by IP Phone users. Most issues relative
to echo can be eliminated at the gateway. H.323 gateways have Cisco IOS software commands
that can assist in reducing or eliminating echo. Media Gateway Control Protocol (MGCP)
gateways have options within Cisco CallManager to assist in reducing or eliminating echo.
Catalyst 6500 Series switches and Catalyst 6608-T1/E1 ports, on the other hand, are a little
trickier and have few options for port tuning.
When troubleshooting echo, it is important to isolate where the echo is. If an IP Phone user
reports echo when talking with PSTN users, and other users on the same campus using the
same gateway do too, the telco circuits or lines on the gateway that they are on probably need
adjustment. You will learn how to adjust gateways in this lesson.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Voice Quality Issues
5-61
Here are the typical questions heard by the Cisco Technical Assistance Center (TAC) when
handling a service request on echo:
Is there echo occurring between IP Phones?
Are IP Phone speakerphones involved?
Is a Cisco SoftPhone or Cisco IP Communicator involved?
Who hears the echo? (This is critical for isolating the cause of echo; usually, it is the IP
Phone.)
If the IP Phone user hears echo, is the echo occurring all of the time?
If the IP Phone user hears echo, is it for calls made to the same phone number, various
numbers, or all numbers?
If the IP Phone user hears echo, does it only occur during certain times of the day?
If the IP Phone user hears echo, does it occur over the same gateway or over various
gateways?
Is the echo being heard systemwide?
Are the IP Phone users using a newly installed gateway?
What types of gateways are used?
What protocol is used?
Are remote office IP Phone users hearing echo?
With the answers to some or all of these questions, you can start to look for commonalities,
such as whether all reported echo involves call flow over a certain gateway. If this is the case,
look at the gateway configuration to see if any port tuning has been applied; if no port tuning
has been applied, and depending on the gateway type and protocol being used, set up port
tuning.
For H.323 gateways, use the following port tuning parameters:
input gain -1
output attenuation 2
echo cancel-coverage 32
For MGCP and other than Cisco IOS gateways, Cisco CallManager has port tuning for the
gateway.
The questions presented here are what Cisco TAC AVVID customer support engineers handle.
In most cases, the answers to these question point to where the issues reside, which usually
points to a gateway.
5-62
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Echo Is Always Present
Echo as a problem
is a function of the
echo delay and the
loudness of the
echo.
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—5-5
For echo to be a problem, all of the following conditions must exist:
An analog leakage path between analog Tx and Rx paths
Sufficient delay in echo return for echo to be perceived as annoying
Sufficient echo amplitude to be perceived as annoying
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Voice Quality Issues
5-63
Sources and Types of Echo
This topic discusses the specifics related to sources and types of echo that can be present in an
IP telephony network.
Source and Types of Echo
Talker Echo (most common)
IP Phone
Tx
Rx
PSTN
PSTN/PBX User
Rx
Tx
Hybrid
© 2004 Cisco Systems, Inc. All rights reserved.
Hybrid
IPTT v4.0—5-6
Talker echo occurs when the speech energy of a talker, transmitted down the primary signal
path, is coupled into the receiving path from the far end (or tail circuit). Talkers then hear their
own voice, delayed by the total echo path delay time. If the “echoed” signal has sufficient
amplitude and delay, the result can be annoying to the customer and can interfere with the
normal speech process. Talker echo is usually a direct result of the two-wire to four-wire
conversion that takes place through “hybrid” transformers.
5-64
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Source and Types of Echo (Cont.)
Listener Echo (less common)
IP Phone
Tx
Rx
PSTN
PSTN/PBX User
Rx
Tx
Hybrid
© 2004 Cisco Systems, Inc. All rights reserved.
Hybrid
IPTT v4.0—5-7
Listener echo occurs at the far end by circulating voice energy. Again, listener echo is generally
caused by the two-wire and four-wire hybrid transformers (caused by the “echo being echoed”).
The voice of the talker is echoed by the far-end hybrid, and when the echo comes back to the
listener, the hybrid on the side of the listener echoes the echo back toward the listener. The
effect is that the person listening hears both the talker and an echo of the talker.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Voice Quality Issues
5-65
Two- to Four-Wire Conversion and Echo
•
•
•
•
Due to a reflection
Impedance mismatch at the two-wire to four-wire hybrid
Most common reason for echo
Found mostly in tail-end circuits
Four-Wire
Trunk
Two-Wire
Subscriber
Loop
Hybrid
© 2004 Cisco Systems, Inc. All rights reserved.
Echo
Two-Wire
Subscriber
Loop
Hybrid
IPTT v4.0—5-8
This slide is an example of a two- to four-wire hybrid in which impedance is mismatched,
resulting in echo being heard on both ends.
Telco usually applies port tuning techniques to reduce echo or eliminate echo or both.
Echo is constant in a telco environment; however, low delay and low amplitude render echo not
an issue.
5-66
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Defining the Echo Canceller
This topic discusses the specifics related to echo cancellation.
Echo Canceller Characteristics
Echo Cancel Coverage
(Tail length)
Tail Circuit
IP Network
ACOM
Sout
^
H(t)
-
^y(t)
Output attenuation
Input gain
ERL
ERLE
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—5-9
This figure show the various characteristics of a Echo Canceller.
Input level: The input gain of a signal is performed before the echo canceller has seen the
echo.
Output level: The output attenuation of a signal is performed after the echo canceller has
seen the original output signal.
Echo canceller coverage: Echo canceller coverage involves the amount of time that the
echo canceller will remember a signal that has been output. This parameter must be set to a
value greater than the time the echo needs to return to the gateway.
—
Echo return loss (ERL): ERL is the level of echo returned from the tail circuit. For
example, if the speech signal enters the tail circuit from the network at a level of x
decibels (dB), the level of the echo coming back from the tail circuit into the serial
input (SIN) terminal of the echo canceller is x minus the ERL dB. ERL is important
because the echo canceller needs the echo volume level to be lower than the output
signal by a certain amount for the echo canceller to interpret the input signal as echo.
Most echo cancellers used in Cisco voice-enabled gateways require that the ERL be
a minimum of 6 dB for the echo canceller to work correctly. If the ERL is lower,
which equates to a louder signal, the echo canceller does not cancel the echo; the
echo canceller does not recognize the signal as echo.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Voice Quality Issues
5-67
—
Echo return loss enhancement (ERLE): ERLE refers to the additional echo loss
obtained through the operation of the echo canceller. An echo canceller is not a
perfect device; and the best an echo canceller can do is to attenuate the level of the
returning echo. ERLE is a measure of this echo attenuation through the echo
canceller. ERLE is the difference in level (in dB) between the signal arriving from
the tail circuit at the SIN terminal of the echo canceller and the level of the signal
leaving the echo canceller (and entering the network) at the serial output (SOUT)
terminal.
ACOM: ACOM is simply the total echo return loss seen across the SIN and SOUT
terminals of the echo canceller; ACOM is the sum of ERL + ERLE. ACOM is the echo
return loss seen by the network. ERL + ERLE equals the difference between the voice of
the originator and the echo that the originator hears. The goal is to make ACOM as large as
possible, because the larger the ACOM, the lower the level of echo the calling party hears.
5-68
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Measuring Echo in an IP Telephony Network
This topic discusses how to measure echo in an IP telephony network.
Measuring Echo Using IP Phone 7960
Cisco
CallManager
Off-Hook
PSTN
Router /
Gateway
1004-Hz Tone
Measure
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—5-10
There are various methods for measuring echo in an IP telephony network. One method is to
use an IP Phone.
Before measuring echo from an IP Phone, make sure that you remove all port characteristics
from the port used for the call (echo cancel converge, output attenuation, input gain).
If you have a phone number to which echo is reproducible, use a Cisco IP Phone 7960 or Cisco
IP Phone 7940 to perform the following steps to generate test tones and measure the ERL:
Step 1
To unlock the key pad, type **#.
Step 2
To enable the tone generator, type **3 on the Cisco IP Phone 7960 or Cisco IP
Phone 7940 keypad while the phone is not on a call. This will enable the Tone soft
key for as long as this phone is registered to a Cisco CallManager server.
Step 3
Next, place a call to another phone and hang up. You must do this because the
sequence **#** is the phone reset sequence.
Step 4
Place a call to the source of the echo.
Step 5
Once the call is established, press the i button on your IP Phone twice. This will
bring up the statistics for the call.
Step 6
If using **3 worked, you should have a Tone soft key available. Press the Tone key,
and the phone will begin to generate a 1004-Hz tone at -15 dB. The only way to stop
the tone is to end the call.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Voice Quality Issues
5-69
Step 7
Once the tone is being generated, keep the call up and then follow the procedure
here to measure the dB levels.
Step 8
Use the show call active voice Cisco IOS command to view the input and output
levels for your call, as follows:
Gateway#show call active voice
OutSignalLevel=-15
InSignalLevel=-15
ERLLevel=25
The test tone is being output as -15 and is being looped back with a 0 dB loss. Therefore, the
test tone is coming back at -15 dB. The ERL value has no significance at this point, because the
echo canceller does not consider the input signal to be echo.
If the ERL value is too low, the echo signal returning to the gateway might be too loud (within
6 dB of the talker signal), causing the echo canceller to consider it as voice (double-talk)
instead of echo. Therefore, the echo canceller will not cancel the echo. ERL should be between
6 dB and 20 dB for the echo canceller to engage.
5-70
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Measuring Echo in Cisco IOS (H.323
Gateways)
Use the Cisco IOS command show call active voice to
look at the input and output levels for your call. The
following is a call that echoes back the signal with no
attenuation.
+EXI[E]WL GEPPEGXMZIZSMGI
WRMT
3YX7MKREP0IZIP!
-R7MKREP0IZIP!
)600IZIP!
WRMT
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—5-11
In this figure, the test tone is being output as -15 and is being looped back to us with a 0 dB
loss, so it is coming back at -15 dB.
The ERL value here does not mean anything at this point, because the echo canceller does not
consider the input signal to be echo.
The OutSignalLevel shows the value of the level after the output attenuation has been applied
to the signal, and the InSignalLevel shows the level after the input gain has been applied.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Voice Quality Issues
5-71
Measuring Echo in Cisco IOS
(H.323 Gateways)(Cont.)
If you configure 1 dB of attenuation in each direction as
follows:
ZSMGITSVX
MRTYXKEMR
SYXTYXEXXIRYEXMSR
The resulting levels are as follows:
+EXI[E]WL GEPPEGXMZIZSMGI
WRMT
3YX7MKREP0IZIP!
-R7MKREP0IZIP!
)600IZIP!
WRMT
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—5-12
In this figure, notice that the OutSignalLevel is -16, because the -15 dB signal was attenuated
by 1 dB. The InSignalLevel is -17 dB because of the input gain of -1.
At this point, the real ERL is 2 dB; however, the echo canceller still does not acknowledge the
input signal as echo.
5-72
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Measuring Echo in Cisco IOS
(H.323 Gateways)(Cont.)
If you configure 2 dB of attenuation in each direction as
follows:
ZSMGITSVX
MRTYXKEMR
SYXTYXEXXIRYEXMSR
The resulting levels are as follows:
+EXI[E]WL GEPPEGXMZIZSMGI
WRMT
3YX7MKREP0IZIP!
-R7MKREP0IZIP!
)600IZIP!
WRMT
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—5-13
The OutSignalLevel is -17, because the -15 dB signal was attenuated by 2 dB. The
InSignalLevel is -19 dB because of the input gain of -2.
The expected ERL of 4 dB is now correct.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Voice Quality Issues
5-73
Measuring Echo on the 6608
• From the Catalyst software prompt, use the show
port voice active <mod>/<port> command to see the
active calls on that port.
• The 6608 does not show you input and output
levels; however, it does show the ERL and ACOM.
• Echo tuning is done from Cisco CallManager in the
Gateway Configuration page.
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—5-14
Use the show port voice active command to gather information about all active calls on the
system and detailed information on individual calls.
Use the show port voice <mod/port> command to view the system resources and specifics
associated with a particular voice call.
5-74
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Measuring Echo on the 6608 (Cont.)
ZHXP'EXO4&<" IREFPI WL TSVXZSMGIEGXMZI
4SVX
'LERRIP
6IQSXI-4EHHVIWW
6IQSXI9(44SVX
%'310IZIP'YVVIRX
'EPP7XEXIZSMGI
'SHIG8]TI+90%;4'1
'SHIV8]TI6EXI
)600IZIP
:SMGI%GXMZMX](IXIGXMSRHMWEFPIH
)GLS'ERGIPPEXMSRIREFPIH
*E\8VERWQMX(YVEXMSR QW ,M;EXIV4PE]SYX (IPE]
0S[;EXIV4PE]SYX (IPE]
6IGIMZI&]XIW
6IGIMZI(IPE]
6IGIMZI4EGOIXW
8VERWQMX&]XIW
8VERWQMX4EGOIXW
8\ (YVEXMSR QW :SMGI8\ (YVEXMSR QW © 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—5-15
In Cisco CallManager 3.1 and 3.2, an input gain of –3 and an output attenuation of 3 results in
the following measurement:
ACOM equals 37 dB
ERL level equals 6 dB
The idea is to get the ACOM up to about 330, 400 meaning 33, or 40 dB. This level should be
enough for the echo canceller to engage, thus canceling out the echo with the proper coverage
(use default of 32 ms).
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Voice Quality Issues
5-75
Important Considerations
• Be sure to check the levels going into and
out of Cisco Unity or other voice-mail system
when you are done tuning for echo.
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—5-16
Make changes one at a time.
For Cisco Unity, you may need to tweak the levels on the Cisco Unity server using the Wave
Gain utility.
When dealing with multiple trunks to the PSTN, try to isolate on which trunk the echo is
occurring.
5-76
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Adjusting Echo in an IP Telephony Network
This topic discusses the specifics used by the Cisco TAC in reducing or eliminating echo (or
both) in an IP telephony network.
Adjusting Levels on Various Gateways
ZSMGITSVX
MRTYXKEMR
SYXTYXEXXIRYEXMSR
IGLSGERGIPGSZIVEKI
IGLSGERGIPWYTTVIWWMSR
6608-T1/E1
(Cisco CallManager 4.0(1)
Cisco IOS H.323
MGCP
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—5-17
On Skinny/MGCP gateways (other than Cisco IOS gateways), adjust the input and output dB
levels from the Cisco CallManager Administration web page.
Adjust the output level using the Audio Signal Adjustment from IP Network menu. Positive
values increase volume, and negative values decrease volume.
Adjust the input level using the Audio Signal Adjustment into IP Network menu. Positive
values increase volume, and negative values decrease volume.
Reset the gateway after adjustment.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Voice Quality Issues
5-77
Eliminating Echo
So, how do you get rid of echo?
You need to ensure that you give the echo
canceller enough information to distinguish
between echo and normal conversation. The only
parameters you have control over are:
• Input level (Input gain)
• Output level (Output attenuation)
• Minimum ERL
• Echo canceller coverage
• Echo suppressor (Cisco IOS Release 12.2[11]T) for
convergence issues
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—5-18
Here are the starting values when attempting to reduce or eliminate echo:
Input gain: -1
Output attenuation: 2
Echo canceller coverage: 32
Echo suppression: 5
5-78
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Summary
This topic summarizes the key points discussed in this lesson.
Summary
• The most common type of echo is talker echo.
• Identify the source and type of echo.
• Reproduce the echo whenever possible.
• Isolate where echo is not coming from.
• Trace the call flow to the source of echo.
• Eliminate and or reduce the echo by applying basic
port tuning techniques.
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—5-19
References
For additional information, refer to this resource:
Troubleshooting Echo Problems between IP Phones and IOS Gateways:
http://www.cisco.com/en/US/tech/tk652/tk698/technologies_tech_note09186a0080149a1f.s
html#IOSecho Next Steps.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Voice Quality Issues
5-79
Quiz
Use the practice items here to review what you learned in this lesson. The correct answers are
found in the Quiz Answer Key.
Q1)
What is the most common type of echo in a VoIP network?
A)
B)
C)
D)
Q2)
When applying an input gain change to a voice port, you must do what to apply the
change?
A)
B)
C)
D)
Q3)
talker
hybrid
listener
NLP
The source of echo is generally located at _____.
A)
B)
C)
D)
5-80
hybrid
talker
listener
loud
_____ echo is generally caused by the two-wire and four-wire hybrid transformers.
A)
B)
C)
D)
Q5)
Shut the voice port.
Do not shut the port to apply the change.
Input gain is not supported on voice ports.
Changes are applied after exiting router configuration.
_____ echo occurs when the speech energy of a talker is, transmitted down the primary
signal path, is coupled into the receiving path from the far end.
A)
B)
C)
D)
Q4)
NLP echo
talker
listener
hybrid
the near gateway
the tail circuit
the IP cloud
the VoIP cloud
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Quiz Answer Key
Q1)
B
Q2)
B
Q3)
B
Q4)
B
Q5)
B
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Voice Quality Issues
5-81
5-82
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Module 6
Troubleshooting Cisco Unity
Overview
Today, the business world has many sources of communication. A connected professional
receives essential communication via telephone, e-mail, and faxes. Regardless of the source,
the professional must retrieve these important messages using a telephone, cellular telephone,
text pager, or e-mail interface. To notify individuals when urgent communications arrive,
business clients need vital information sent to external voice-mail systems or e-mail systems, or
need calls from their cellular telephones or home telephones. Cisco Unity puts all of these
pieces together.
Module Objectives
Upon completing this module, you will be able to troubleshoot common Cisco Unity
configuration, integration, and operation issues.
Module Objectives
Troubleshoot common Cisco Unity problems
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—6-2
Module Outline
The outline lists the components of this module.
Module Outline
Troubleshooting Common Configuration,
Integration, and Operation Issues
© 2004 Cisco Systems, Inc. All rights reserved.
6-2
Cisco IP Telephony Troubleshooting (IPTT) v4.0
IPTT v4.0—6-3
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Common
Cisco Unity Configuration,
Integration, and Operation
Issues
Overview
Losing voice mail can affect all aspects of business in a company. You should repair missioncritical infrastructure applications, such as Cisco Unity, quickly. This lesson will teach you
about using Cisco Unity 4.x. troubleshooting tools to quickly repair problems. You will also
learn about disaster recovery techniques.
Relevance
This lesson provides an in-depth explanation of Cisco Unity configuration issues and common
errors. With this information, you can save time by troubleshooting the correct system and
configuration issues.
Objectives
Upon completing this lesson, you will be able to:
Describe general troubleshooting steps from a Cisco Unity perspective
Describe Event Viewer troubleshooting functions
Explain Cisco Unity Administration tool functions
Describe miscellaneous Cisco Unity troubleshooting tools
Review disaster recovery techniques
Resolve call transfer problems
Resolve port problems
Resolve system programming problems
Resolve message problems
Resolve MWI problems
Learner Skills and Knowledge
To benefit fully from this lesson, you must have these prerequisite skills and knowledge:
Foundational understanding of Cisco voice networks
Fundamental understanding of Cisco IOS commands at the Cisco CCNP® level
Foundational understanding of Cisco CallManager dial plan
Basic knowledge of Microsoft SQL Server 2000 and Cisco CallManager
Basic understanding of operating systems
Prior exposure to IP telephony
Prior exposure to quality of service
6-4
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Outline
The outline lists the topics included in this lesson.
Outline
• Overview
• Troubleshooting Steps
• Event Notification Utility
• Cisco Unity Administration Tools
• Cisco Unity Troubleshooting Tools
• Call Transfer Problems
• Port Problems
• System Programming Problems
• Error Messages and Disaster Recovery
• Message Problems
• Message Waiting Indicators
• Summary
• Quiz
© 2004 Cisco Systems, Inc. All rights reserved.
Copyright © 2004, Cisco Systems, Inc.
IPTT v4.0—6-3
Troubleshooting Cisco Unity
6-5
Troubleshooting Steps
This topic explains the main steps in the troubleshooting process.
General Troubleshooting Steps
• Identify the problem.
• Duplicate the problem.
• Isolate the problem.
• Resolve the problem.
• Test the system.
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—6-4
Use the following troubleshooting steps to ensure a successful solution:
Step 1
Identify the problem: Obtain a description of the problem from the person or
persons who experienced it, making sure to define the problem in common terms.
Many times, you will arrive at a customer site, and the customer will have a list of
problems for you to solve. Several of the complaints are often related to one main
problem. The main problem is the one that you should address.
The customer may also complain about not receiving any messages. There are
several reasons for this problem including the following: Cisco Unity did not call to
deliver a message, a message light did not appear, or the person calling could not
hear the Cisco Unity recording over noise on the telephone line.
Step 2
6-6
Duplicate the problem: Determine if the person who reported the problem can
duplicate it. If not, attempt to duplicate the problem by making calls just as the
customer described. Try internal and external lines. Make multiple calls during the
same hours that the problem occurred previously. If you cannot duplicate the
problem, you may not be able to correct the problem on this service call. In that
case, have the customer keep a log of exactly what happens, including the date and
time. You can then resolve the problem by looking at log files in the voice-mail
system.
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Step 3
Isolate the problem: Does the problem occur only on a specific port or set of ports?
Does the problem occur only on internal or external calls? Does the problem occur
only with one Cisco Unity user (anyone who has an account on the Cisco Unity
system)? Does Event Viewer display an error?
Step 4
Resolve the problem: After you have isolated the problem, you can correct it.
Correcting the problem may include reconfiguring Cisco Unity software to eliminate
the problem, reprogramming the telephone system, replacing bad cables or
hardware, training the user, and other methods.
Step 5
Test the system: When you have resolved the problem, test the system under the
same conditions that you used to duplicate the problem.
Note
Cisco Unity documentation, available at Cisco.com, includes a portable data format (PDF)
copy of the Cisco Unity Troubleshooting Guide. This manual contains specific information
about running tests and offers ideas about problem resolution.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Cisco Unity
6-7
Event Notification Utility
This topic describes the Event Notification utility.
Event Notification Utility
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—6-5
Every Cisco Unity system has a default public distribution list named System Event Messages.
The only default member of this group is the Example Administrator, though you may add as
many members as necessary. You use the Event Notification utility to determine which
subscribers or distribution lists to notify for specific problems and system errors.
To start, you choose Start > Programs > Unity > Cisco Unity Tools Depot > Reporting
Tools> Event Notification Utility. You can also start by going to \Commserver\Gaen directory
and double-clicking GaenAdmin.exe. (The letters, GAEN, stand for General Audio Event
Notification.)
From the File menu, choose New NT Event Log and specify which system action should
notify you and what type of message (voice or e-mail) you want to send as notification. With
the event number, you can program the utility to notify a user of any system event, even those
generated by third-party software.
6-8
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Event Viewer Log Report
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—6-6
Use the Event Viewer log report to list events from the application log. You can generate a
report for all application events on the Cisco Unity server or for the events that apply only to
Cisco Unity. Cisco Unity writes events only to the application log. Cisco Unity does not write
events to the system or security logs.
Note
If you generate a report for all application events, you can identify the Cisco Unity events as
those events that end in _MC, such as AvLogMgrSvr_MC.
You can view application events with the Windows Event Viewer. Choose Start > Programs
> Administrative Tools > Event Viewer.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Cisco Unity
6-9
Cisco Unity Administration Tools
This topic describes the Cisco Unity Administration tools.
Subscriber Reports
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—6-7
Cisco Unity reports provide information about subscribers and system activity. These are the
types of Cisco Unity reports, and their functions:
Subscribers report: Use the Subscribers report to get a list of Cisco Unity subscribers.
You can generate the report for an individual subscriber or for all Cisco Unity subscribers
Subscriber Message Activity report: Use the Subscriber Message Activity report to
diagnose voice message problems reported by a subscriber; for example, voice messages
are not being sent or received, or the Message Waiting Indicator (MWI) is not being turned
on or off properly. You generate this report for an individual subscriber only.
Distribution Lists report: Use the Distribution Lists report to get a listing of all public
distribution lists and, optionally, the members in each list. You can generate the report for a
selected public distribution list or for all public distribution lists. If you want the report to
include the names of distribution list members, check the List All Members for Each
Distribution List box
Failed Login report: Use the Failed Login report to identify patterns of invalid logons,
which would indicate that an individual is trying to gain unauthorized access to
Cisco Unity. This report also identifies accounts that have been locked because the
maximum number of invalid logons has been exceeded. You can generate the Failed Login
report for all subscriber accounts. Additionally, you can indicate whether to show all failed
logon attempts (expand the report) or show only the last failure for each subscriber.
Note
6-10
Note that including the Failed Login report for the Cisco Unity Administrator logons requires
that auditing be enabled in system security policies. Cisco Unity setup enables auditing by
default.
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Storage Usage report: Use the Storage Usage report to obtain information about the
amount of storage space available and the disk space used for storing messages by each
subscriber. You can generate the Storage Usage report for a selected subscriber, for all
subscribers, or for a public distribution list.
Transfer Billing report: Use the Transfer Billing report to obtain information about calls
that are transferred from a subscriber account or from a call handler. You can use this
report for billing purposes or to keep track of transfers to long-distance phone numbers.
You can generate this report for all subscribers, or all billing IDs, or for a single
distribution list or single call handler. Note that the single subscriber, single billing ID,
distribution lists, and all call handler selection options are not supported in this release.
Note
Only those subscribers with call transfer data appear on the report.
Outcall Billing report: Use the Outcall Billing report to obtain information about
outbound calls made by Cisco Unity for message notifications. This report also provides
information about outbound calls that Cisco Unity makes to subscriber extensions when
subscribers use their phones as the recording and playback device for the Media Master.
You can use this report for billing purposes, or to keep track of message notifications sent
to long-distance phone numbers. You can generate the report for subscribers, billing IDs, or
for a distribution list.
Note
Note that the Dial Time option excludes subscribers with a billing ID of 0 (zero).
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Cisco Unity
6-11
Example Subscriber Report
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—6-8
This figure represents a completed Subscriber Report. Before running a report, you can choose
whether to view it in comma separated values (CSV) or HTML format. You can also choose to
automatically send e-mail reports to a subscriber, or have the subscriber view the reports in the
D:\Commserver\Reports directory.
6-12
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
System Reports
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—6-9
The Administrative Access activity report provides details of all the changes entered into the
Cisco Unity Administrator. This information provides an audit trail of the commands that the
system administrators enter.
Use the Port Usage report to determine if the voice-messaging system is running close to its
capacity. You can indicate the ports to include in the report. Enter port numbers, or ranges of
port numbers separated by commas (for example, 1,2,4,8), in the Ports to Show field. To
include all ports in the report, leave this field empty.
The Gather Unity System Info report provides information about the Cisco Unity server and
software. You must give this report to the Cisco Technical Assistance Center (TAC) when you
open a case.
Use the Unresolved References report to locate primary call handlers (call handlers that are
associated with a subscriber account), other call handlers, and interview handlers that are left
unresolved because of the improper deletion of a subscriber account. This problem occurs when
you delete a subscriber using Microsoft Exchange or Windows Administrator applications
without first deleting the subscriber using the Cisco Unity Administrator.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Cisco Unity
6-13
Example System Reports
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—6-10
You can have the Cisco Unity system e-mail the reports to you. The person who generates the
report automatically receives the e-mail with a link to the report file. To view the report, you
can click the link if the report was generated as a web page. If the report was generated as a
CSV file, you must open the report file in an application that supports CSV files, such as
Microsoft Excel.
If you do not want the report links e-mailed to you, the reports can be found in Windows
Explorer under the directory D:\CommServer\Reports.
6-14
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Cisco Unity Troubleshooting Tools
This topic describes Cisco Unity troubleshooting tools.
Cisco Unity Diagnostic Tool: Configuring
Traces
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—6-11
The Cisco Unity Diagnostic Tool allows the creation and viewing of diagnostic log files that
can be used to troubleshoot problems. This tool, found in the Diagnostic Tools folder, replaces
the diagnostic log functionality in Maestro Tools, and allows the system administrator or TAC
staff member to selectively run diagnostic traces at these two levels:
Macro traces: These are collections of component traces that help diagnose problems such
as Message Waiting Indicator (MWI) and system problems.
Micro traces: These are the component traces. Each component has up to 32 trace levels
that can be individually selected.
The Cisco Unity Diagnostic Tool also allows the system administrator or TAC staff member to
do the following tasks:
Create new log files on demand: This makes troubleshooting problems easier. When a
problem can be reproduced reliably, the system administrator can close all existing log files
and create new log files before reproducing the problem. This eliminates many unnecessary
and unrelated items from the logs.
Configure log settings: The system administrator can adjust the maximum disk space
allowed for all diagnostic log files. (The default setting is 400 MB.) The Logging
Properties screen also allows the system administrator to disable all diagnostic output by
clearing the Diagnostic Output check box.
Gather standard logs: This option provides the ability to quickly gather all or selected
Microsoft Windows and Cisco Unity logs.
Disable all traces: This is a quick way to return diagnostic logs to their default settings
after troubleshooting efforts are complete.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Cisco Unity
6-15
View Event Viewer log: The Event Viewer log files for either the local computer or
another computer can be viewed and exported.
Change display language for Microsoft Windows 2000 Event Viewer log messages
generated by Cisco Unity: This is a temporary change and is only in effect while the
Cisco Unity Diagnostic Tool is running.
6-16
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Skinny and TSP Traces
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—6-12
If, for example, you wanted to gather Skinny Telephony Service Provider (TSP) traces, simple
click Configure Macro Traces, choose Skinny TSP Trace, and click Next. Use the Gather
Logs File Wizard to gather the appropriate logs.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Cisco Unity
6-17
Call Transfer Problems
This topic describes call transfer problems.
Call Transfer Problems
Calls not transferring to the correct greeting
because of forward timer (non-IP Phone):
• Confirming that the forward timer in the phone
system is in synch with the “Rings to Wait For”
setting in Cisco Unity
• Confirming that the phone system programming
enables callers to hear the subscriber personal
greeting
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—6-13
For supervised transfers, the number of rings that Cisco Unity waits before routing a call to a
subscriber personal greeting (or to another extension) can be reconfigured. If the phone system
is programmed to forward calls, confirm that the phone system waits longer to forward a call
than Cisco Unity waits before taking a message.
If the phone system is forwarding the call to another extension before Cisco Unity can take a
message, the following may occur:
The caller does not hear the beginning of the subscriber personal greeting. (For example,
the subscriber greeting is “Hi, this is Terry. Please leave a message after the tone.” But the
caller hears only “...message after the tone.”)
The call is forwarded to another phone (for example, to the operator) rather than to the
subscriber personal greeting.
The call is forwarded to the opening greeting.
The caller hears only ringing.
Here are the steps for to synchronize the forward timer and the “Rings to Wait For” setting:
6-18
Step 1
In the phone system programming, find the value of the forward timer.
Step 2
In the Cisco Unity Administrator, choose Subscribers > Subscribers > Call
Transfer.
Step 3
Click Find, and find the subscriber whose calls are not being routed to the correct
greeting.
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Step 4
In the Transfer Incoming Calls to Subscriber’s Phone section, confirm that the “Yes,
Ring Subscriber’s Extension” box is checked.
Step 5
In the Transfer Type section, confirm that the Supervise Transfer box is checked.
Step 6
In the “Rings To Wait For” box, the value should be two rings fewer than the value
of the forward timer of the phone system, which you found in Step 1; this value is
typically not greater than four, and is never greater than eight. This value specifies
the number of rings that Cisco Unity waits before routing the call to the subscriber
personal greeting. If the values do not meet the parameters, either reprogram the
phone system so that it waits longer before forwarding unanswered calls, or change
the value in the Rings To Wait For box so that Cisco Unity routes the call before the
phone system forwards it.
Step 7
To change the default Rings To Wait For value for future subscribers, choose
Subscribers > Subscriber Template > Call Transfer.
Step 8
Determine whether the phone system changes the ringback cadence after a certain
number of rings. If it does so, in the Cisco Unity Administrator, set the Rings To
Wait For value to a number fewer than the number of rings at the initial cadence.
Step 9
If you have determined that the phone system is waiting longer to forward a call than
Cisco Unity is waiting to take a message, but Cisco Unity still is not routing calls to
the correct greeting, run the Learn Tones utility. For more information, see the Learn
Tones section.
Step 10
If you have run the Learn Tones utility, and Cisco Unity still is not routing calls to
the correct greeting, contact the Cisco TAC.
When callers hear the opening greeting instead of a subscriber personal greeting, confirm that
the integration is enabled and that the phone system settings are correct. If the settings are
incorrect, call forwarding to personal greeting and easy message access will not be enabled. Do
one of the following procedures, depending on your phone system integration:
To verify the phone system settings for the integration (All Integrations)
Step 1
On the Windows Start menu on the Cisco Unity server, choose Programs >
Cisco Unity > Manage Integrations. The Cisco Unity Telephony Integration
Manager (UTIM) appears.
Step 2
Confirm that the settings match those indicated in the integration guide for your
phone system.
Step 3
Correct any incorrect values for the phone system and click Save.
Step 4
If prompted, restart the Cisco Unity server.
Step 5
If you have confirmed that the phone system settings are correct, and callers still hear
the opening greeting after dialing the subscriber extension, contact the Cisco TAC.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Cisco Unity
6-19
Call Handler Call Transfer
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—6-14
If the transfer options of the call handler are not correctly configured, some call handlers will
not properly transfer calls. To solve this problem, choose Call Management > Call Handlers
> Call Transfer to verify that the call transfer parameters are set correctly. Ensure that the
configuration is set correctly.
6-20
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Port Problems
This topic describes port problems.
Verify Number of Ports
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—6-15
Call transfer problems can occur when the number of system ports is incorrect. If the Cisco
Unity server has more voice ports than the license file shows, Cisco Unity will not answer calls
on the extra ports.
To solve this problem, access the Cisco Unity Tools Depot, and under Administrative Tools
choose License Info Viewer. You will see the number of ports that the license is good for.
Look for Voice Ports and, to the right, you will see a value in the License column.
If the Cisco Unity system is licensed for 32 ports, make sure that Cisco CallManager is not
configured to support more than 32 ports.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Cisco Unity
6-21
Incorrect Number of System Key Ports
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—6-16
Verify that Cisco CallManager has been configured with the correct number of voice-mail
ports. If the Cisco CallManager voice-mail ports do not match the number of Cisco Unity ports,
the additional ports will not operate and may cause instability in the Cisco Unity system.
6-22
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Calls Sent to Wrong Cisco Unity Ports
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—6-18
In Cisco CallManager, verify that you are correctly hunting for ports that can accept inbound
calls. If the first port is busy, try the second port, and so on, until all of the ports that are set to
accept inbound calls have been tried.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Cisco Unity
6-23
System Programming Problems
This topic describes Cisco Unity system programming problems.
Telephone System Programming
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—6-18
If callers are hearing the opening greeting instead of a subscriber personal greeting, the
administrator may have incorrectly programmed the telephone system. Confirm that integration
is enabled and that the phone system settings are correct.
If the settings are incorrect, calls will forward to the personal greeting, and easy message access
will not enable. To solve this problem, verify the following:
Verify that no open Cisco Unity TSP warnings are logged in the Event Viewer.
Make sure that Cisco Unity is receiving digits from the phone system or Cisco
CallManager.
Verify that the IP switch configurations are correct .
6-24
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Error Messages and Disaster Recovery
This topic describes error messages and disaster recovery.
Dr. Watson Program
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—6-19
Dr. Watson is a program invoked by Microsoft Windows 2000 when a serious problem occurs
that is not handled by Cisco Unity. When Dr. Watson is invoked, a dialog box that contains an
error message appears (for example, “Dr. Watson encountering an error in the AvCsMgr.exe
process”). Dr. Watson errors may occur in other processes such as Tapisrv.exe and
Dlgc_srv.exe.
To obtain a Dr. Watson log, follow these steps:
Step 1
When a Dr. Watson error occurs, make a copy of the file Winnt\Drwtsn32.log.
Step 2
Before you attempt to reproduce the problem, from a command prompt, enter
drwtsn32 and press Enter.
Step 3
In the Number of Instructions field, enter 50.
Step 4
In the Number of Errors to Save field, enter the number of errors you want to record.
The default is 10.
Step 5
Under Options, confirm that the Dump All Thread Contexts, Append to Existing
Log File, Visual Notification, and Create Crash Dump File boxes are checked.
Step 6
Click OK to close the dialog box.
Step 7
Reproduce the problem.
Step 8
Make a copy of the file Winnt\Drwtsn32.log.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Cisco Unity
6-25
Disaster Recovery
• Verify Microsoft IIS permissions.
• Create a simple batch file for the Cisco Unity server.
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—6-20
You need a strategy that provides a schedule for performing full backups to restore Cisco Unity
quickly, if data is corrupted or lost. This topic discusses the issues relative to a system that has
been backed up.
If you cannot access the Cisco Unity server from Internet Explorer, or you cannot start Cisco
Unity after a recovery from backup, perform the following steps to verify Microsoft Internet
Information Server (IIS) permissions:
Step 1
Choose Start > Programs > Administrative Tools > Internet Service Manager.
Step 2
In the left pane, browse to the default website container for IIS.
Repeat Steps 3 through 10 for System Attendant, SAWeb, System Attendant Help
(SAHelp), Status, and Active Voice Extensible Markup Language (AvXml), and
then proceed to Step 11.
6-26
Step 3
Right-click the object, and choose Properties.
Step 4
Click the Virtual Directory tab.
Step 5
Under the Execute Permissions section, choose Script Only.
Step 6
Click the Directory Security tab.
Step 7
Under the Anonymous Access and Authentication Control section, click Edit.
Step 8
Confirm that the Anonymous Access box is unchecked.
Step 9
Click OK.
Step 10
Click OK to close the Properties dialog box.
Step 11
Confirm that SAWeb, SAHelp, Status, and AvXml objects have the appropriate
permissions, and close Internet Service Manager.
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
If you made any changes in Step 11, click Yes when prompted to save the console settings, and
restart the Cisco Unity server. If Cisco Unity does not start, or if you did not make any changes,
it is optional to reregister the components. To reregister components, you must create a batch
file for the Cisco Unity server to replace Regsvr32.exe. You can also open a command prompt,
and go to the C:\Commserver\Components directory to run Regsvr32.exe.
The following procedure explains how to reregister components:
Step 1
Start a text editor.
Step 2
Enter “for %%x in” (\commserver\components\*.dll) and “do regsvr32 %%x”.
Step 3
Save the file in the \CommServer\Components directory as reg.bat or another .bat
file.
Step 4
Open a command prompt window, choose the drive where Cisco Unity is installed
(the default is C:\CommServer), and type “cd \commserver\ components” to change
directories.
Step 5
Enter reg to run the batch file.
This procedure reregisters the following:
All the dynamic link libraries (DLLs) in the \CommServer\Components directory
All the DLLs that do not have the letter “P” in their filenames in the
\CommServer\Orderedcoms directory
The remaining DLLs in the \CommServer\Orderedcoms directory in this order:
AvPropertySetPsSvr.dll, AlCommonPsSvr.dll, MalCommonPsSvr.dll, and AvDohPsSvr.dll
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Cisco Unity
6-27
Message Problems
This topic describes the problems that can result when messages are delayed or disappear.
Delayed or Missing Messages
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—6-21
Message delays can occur when the Microsoft Exchange server is disconnected or down, when
a subscriber misunderstands the use of the Pound (#) key or when the system clock time is
incorrect.
Messages that are recorded while the primary Microsoft Exchange server is disconnected or off
line are stored until the server is brought back on line. The amount of time that the Microsoft
Exchange server is down affects when the message is stored and the time stamp that the user
hears on the message.
When a subscriber presses the Pound key while listening to a message, Cisco Unity saves the
message as a new message and skips to the next message. If a subscriber checks the messages
again, Cisco Unity plays the same message.
If the system clock is incorrect, a subscriber may believe that a message delay occurred. Verify
that the system time is correct in the operating system. Use Network Time Protocol (NTP), and
have all servers synchronize to an atomic clock.
6-28
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Mailbox Is Full
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—6-22
Messages can disappear when a mailbox is full. When a Microsoft Exchange mailbox has
exceeded the “Prohibit send and receive at (KB)” limit that is set in the Microsoft Exchange
Administrator, the user cannot send or receive any new messages. When a recipient mailbox is
full, the sender receives an undeliverable message. If the sender is the Cisco Unity Voice
Messaging system on behalf of an unknown caller (any outside caller leaving a message), the
nondelivery receipt (NDR) is sent to the Unaddressed Messages distribution list. Subscribers
should dispose of messages promptly so that the Microsoft Exchange mailbox does not exceed
its size limit.
Within Microsoft Exchange 2000, administrators can implement storage limits for a mailbox
store or, more granularly, for a specific mailbox.
When a Microsoft Exchange 2000 administrator implements limits on a mailbox store,
Microsoft Exchange 2000 writes those limits to the Configuration Domain Controller. The
Configuration Domain Controller is a domain controller (DC) specified by Microsoft Exchange
2000 for all reads and writes. Designating a single DC as the Configuration Domain Controller
prevents latency in configuration changes between Microsoft Exchange 2000 servers, because
all Microsoft Exchange 2000 servers are typically set to use the same Configuration Domain
Controller.
After writing to the Configuration Domain Controller, the Active Directory (AD) replicates the
data to the global catalog server (GC). The time is takes for replication will depend on the AD
topology and replication schedule.
When a Microsoft Exchange 2000 administrator implements limits on a specific mailbox, those
changes are written to the DC with which the instance of the Active Directory Users and
Computers snap-in is associated.
Every time data is written to a DC, AD increments the Update Sequence Number (USN). The
USN value of the last change to the object is stored in the USN Changed property, which is
present on all objects in the directory. It is important to note that the USN value for an object
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Cisco Unity
6-29
on one DC does not dictate the USN number for the same object on a different DC; the USN
value for an object is unique to each DC.
In a Microsoft Exchange 2000 integration, Cisco Unity discovers the mailbox store limits by
monitoring the GC. Cisco Unity monitors the GC through the AvDSGlobalCatalog service.
Mailbox store limits are discovered through monitoring the object or objects in the GC of the
ms-Exch-Private-MDB class. Each mailbox store in the forest is represented in the GC by an
object of the ms-Exch-Private-MDB class.
For more information, refer to the following document: Understanding How Exchange 2000
Storage Limits Work with Cisco Unity (Versions 4.0, 3.1, and 3.0 with Microsoft Exchange).
This document can be found at
http://www.cisco.com/en/US/products/sw/voicesw/ps2237/products_white_paper09186a00800f
af0f.shtml.
6-30
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Undeliverable Messages
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—6-23
This issue does not occur with Microsoft Exchange 2000 or Microsoft Exchange 2003, only
with Microsoft Exchange 5.5.
Messages returned to the Cisco Unity Voice Messaging system mailbox are forwarded
automatically to subscribers whose names appear on the Unaddressed Messages public
distribution list. The messages then must be forwarded to the intended recipients. Explain to
subscribers on the Unaddressed Messages public distribution list the importance of regularly
checking for and forwarding undeliverable messages.
Note
Note that if the mailboxes of the subscribers who are assigned to check the Unaddressed
Messages list exceed the Prohibit Send and Receive storage limit that is specified in
Microsoft Exchange, the messages sent to the Unaddressed Messages distribution list are
lost. To avoid this problem, specify a generous value for the Prohibit Send and Receive
storage limit for the mailbox of at least one subscriber who is a member of the Unaddressed
Messages list and encourage the subscriber to dispose of messages promptly so that the
Microsoft Exchange mailbox does not fill up.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Cisco Unity
6-31
Message Waiting Indicators
This topic describes how to set MWIs.
Cisco Telephony Integration Tool
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—6-24
You should set MWIs to on (red light on) whenever a new message has been left in a subscriber
voice-mail box and off (red light off) after the subscriber has listened to the new message.
MWI problems can occur when the telephone system settings and Cisco Unity settings do not
match with the telephony system or with Cisco CallManager. When the telephone system
settings in the Cisco Unity Telephony Integration tool do not match the type of telephone
system that Cisco Unity is connected to, the MWIs may not turn on and off. In addition, the
MWI directory numbers (DNs) used by Cisco Unity must match those defined in Cisco
CallManager. Verify that the Cisco Unity-CM TSP is correctly defining the MWI on and off
parameters.
6-32
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
MWIs
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—6-25
Verify that the MWI on and off DNs are unique within the cluster dial plan. You should check
each Cisco CallManager server in the cluster to ensure that the MWI on and off parameters are
defined, and that the MWI on and off DNs are unique.
The DNs that are configured in Cisco CallManager must match the configuration in Cisco
Unity. If this configuration is not consistent, users will experience MWI on and off issues.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Cisco Unity
6-33
Microsoft Exchange Server Down
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—6-26
When the Cisco Unity primary Microsoft Exchange server is off line or disconnected, Cisco
Unity stores recorded messages until the server is brought back on line. Because the MWI light
does not appear until after the home Microsoft Exchange server of the subscribers receive a
message, subscribers may experience a delay between the time the system recorded the
message and when they are notified that a new message has arrived. This delay is dependant on
the amount of time that the primary Microsoft Exchange server is down or disconnected. If
subscribers call Cisco Unity during a time when the Microsoft Exchange server is down and
they have messages, those messages will be presented to them. These subscribers can listen, but
not respond, to those messages. When the Microsoft Exchange server comes back on line, the
subscribers will be notified that they have a new message, even though they have listened to it.
6-34
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
A Single Port
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—6-27
A single port, designated only for MWI functions within an IP Phone system integration, can
change approximately 240 to 360 MWIs per hour, depending on the telephone system. An
analog integration can take up to 7 seconds per MWI change.
When the ports that turn MWIs on and off also perform other operations, they are often too
busy to turn MWIs on and off promptly. This problem indicates that there are not enough ports
designated for MWI functions.
To determine if MWI ports in Cisco Unity are overutilized, run a Port Usage report within the
Cisco Unity System Administrator. If the MWI ports are over utilized, dedicate one or more
ports for MWI on and off functions in the Cisco Unity Administrator.
Note
Systems that handle a large volume of calls may require additional ports to improve MWI
performance.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Cisco Unity
6-35
MWIs Lose Synchronization
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—6-28
MWIs may lose synchronization if the telephone system is off line when the MWI status
changes. To solve this problem, in Cisco Unity System Administrator, choose System >
Switch, and then click the Resynch Now button. This action forces Cisco Unity to examine its
entire subscriber database, checking to see if there are any new messages in the inbox and then
comparing that to what it knows about the MWI state of the phone of the subscriber; therefore,
it is best to perform this resynchronization after normal business hours.
6-36
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Ports Are Too Busy
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—6-29
Sometimes ports are too busy to make notification calls promptly. You can improve
notification performance by dedicating one or more ports to making notification calls. Choose
Unity SA > System > Ports to dedicate one or more ports for message notification.
Note
An organization that handles a large volume of calls may require additional ports to improve
notification performance.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Cisco Unity
6-37
Ports Hang
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—6-30
Calls sent to wrong Cisco Unity ports will cause the ports to hang. If the telephone system
sends calls to a port on Cisco Unity that is not configured to answer calls, Cisco Unity will not
answer the call. Thus, the Cisco Unity port cannot perform its primary goal, which is message
notification. In Cisco CallManager, verify that the ports in the Hunt Group match the ports that
are sending and receiving voice mail.
6-38
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Missed Notification Attempts
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—6-31
A subscriber who is frequently away from, or busy using, a notification device may repeatedly
miss notification attempts. To the subscriber, it appears that Cisco Unity has delayed the
message notification. To solve this problem, click the Restart a Notification Each Time a
New Message Arrives radio button, increase the number of times to try again, and decrease the
number of minutes to wait between tries in the Notification Options section.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Cisco Unity
6-39
Resolving Cisco CallManager Integration
Problems
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—6-32
Consider the following issues if Cisco CallManager is integrated with Cisco Unity:
You may need to enter the unique extensions for turning MWIs on and off in the Cisco
CallManager server, or you may need to restart the Cisco CallManager server to enable
these values.
A Cisco CallManager route plan may include unique extensions for turning MWIs on and
off. For example, a route plan could send all numbers starting with 9 to a gateway, while
the extension that turns MWIs on is 99991. Revise the route plan so that it does not include
the MWI extensions, or alter the extensions.
You may need to enter the unique extensions for turning MWIs on and off in the
MessageWaitingOnDn and MessageWaitingOffDn fields of the Cisco Unity-CM TSP, or
you may need to restart Cisco Unity to enable these values.
If the IP Phone is not in the same calling search space and partition as the Cisco Unity
voice-messaging ports, dial the extension that turns on the MWIs. If you hear the reorder
tone, the extensions for turning MWIs on are not assigned to the correct calling search
space and partition in Cisco CallManager. If you do not hear the reorder tone and the MWI
is not activated or deactivated, a route plan may be causing the problem.
If the unique extensions for turning MWIs on and off in Cisco CallManager are not
identical to the values entered in the MessageWaitingOnDn and MessageWaitingOffDn
fields of the Cisco Unity-CM TSP, change those extensions and restart the Cisco
CallManager and Cisco Unity servers.
If Cisco Unity integrates with multiple Cisco CallManager clusters, you must dedicate at
least one voice-messaging port to set MWIs for each cluster. For example, a two-cluster
environment requires at least two ports that are dedicated to setting MWIs, one port
sending MWI requests for the first cluster and another port sending MWI requests to the
second cluster. Confirm that each cluster has at least one voice-messaging port and that the
port is set to Dial Out MWI.
6-40
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Summary
This topic summarizes the key points discussed in this lesson.
Summary
• The main steps in the troubleshooting process include
identifying, duplicating, isolating, and resolving the
problem, and then testing the system.
• The Event Notification utility determines which
subscribers or distribution lists are notified of specific
problems and system errors.
• Cisco Unity Administration tools, such as the various
Cisco Unity reports, provide information about
subscribers and system activity.
• Cisco Unity records a large amount of information and
is capable of recording additional data about its
internal operations.
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—6-33
Summary (Cont.)
• Microsoft IIS permissions must be verified for disaster
recovery.
• Call transfer problems can occur when routing rules
are not working correctly.
• Port problems can occur when the number of system
key ports is incorrect.
• System programming problems can occur if the
telephone system is programmed incorrectly.
• Message delays can occur when the Microsoft
Exchange server is disconnected or down, when a
subscriber misunderstands the use of the Pound key,
or when the system clock time is incorrect.
• MWI problems can occur when the telephone system
settings are incorrect.
© 2004 Cisco Systems, Inc. All rights reserved.
Copyright © 2004, Cisco Systems, Inc.
IPTT v4.0—6-34
Troubleshooting Cisco Unity
6-41
References
For additional information, refer to this resource:
Cisco Unity: http://www.cisco.com/univercd/cc/td/doc/product/voice/c_unity/unity31/tsg/.
6-42
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Quiz
Use the practice items here to review what you learned in this lesson. The correct answers are
found in the Quiz Answer Key.
Q1)
Which step in the troubleshooting process ensures that you have successfully resolved
the Cisco Unity issue?
A)
B)
C)
D)
E)
Q2)
Which utility provides notification of system events?
A)
B)
C)
D)
Q3)
during day-to-day operations
during peak hours
during backup operations
when directed by the Cisco TAC
What occurs when the number of ports installed on the Cisco Unity server exceeds the
number of ports in the system key?
A)
B)
C)
D)
Q7)
Event Tracking Log
Administrative Access Activity report
System Configuration report
Administrative Security Log
When should an administrator enable low-level traces on the Cisco Unity server?
A)
B)
C)
D)
Q6)
system
security
application
active Directory
Which system report tracks all of the changes that the Cisco Unity Administrator
performs?
A)
B)
C)
D)
Q5)
Eventnotify.exe
GaenAdmin.exe
NotifyAdmin.exe
NotifyGaen.exe
Which Microsoft Windows 2000 Event Viewer log does Cisco Unity use to list errors?
A)
B)
C)
D)
Q4)
isolate
duplicate
identify
resolve
test
Cisco Unity operates with the additional ports.
Cisco Unity issues a temporary license that expires in 30 days.
Cisco Unity generates a warning and automatically removes the extra ports.
Cisco Unity does not enable the additional ports, and the Cisco Unity server
may become unstable.
What can cause a subscriber to hear a reorder tone when answering a call?
A)
B)
C)
D)
The Rings to Wait parameter is set too low.
The reorder tone generator is performing a supervised transfer.
The system needs to execute the Learn Tone utility.
The reorder tones are normal, and the subscriber should ignore them.
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Cisco Unity
6-43
Q8)
What occurs when a subscriber presses the Pound key while listening to a message?
A)
B)
C)
D)
E)
Q9)
What must you do when you move a mailbox to ensure that the Cisco Unity server
accepts the changes?
A)
B)
C)
D)
Q10)
C)
D)
The MWI light appears during the recording of the message.
The MWI light appears when the Microsoft Exchange server comes back on
line.
The MWI light appears during the recording of the message, but it turns off
when the Microsoft Exchange server comes back on line and resynchronizes
the MWIs.
The subscriber MWI light never appears.
What must you do when configuring Cisco CallManager to use unique extensions for
turning on and off MWIs?
A)
B)
C)
D)
6-44
Cisco Unity recognizes the change automatically.
You must edit the properties of the subscriber account to reflect the change.
You must restart the Cisco Unity server.
You must choose the Force SQL replication operation from the System
Administrator console.
What happens to a subscriber MWI if a voice-mail message is recorded when the
subscriber Microsoft Exchange server is down?
A)
B)
Q11)
The system deletes the message.
The system skips the message and saves it as an old message.
The system finishes playing the message and saves it as an old message.
The system skips the message and saves it as a new message.
The system finishes playing the message and saves it as a new message.
You must restart the Cisco CallManager server.
You must set the synchronization bit to 1.
The AVTSP.DLL must appear on the same partition as the server
AVVIDMWI.exe.
The MWI ports must appear in numerical order, and they must be the largest
number in the device pool.
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Quiz Answer Key
Q1)
E
Q2)
B
Q3)
C
Q4)
B
Q5)
D
Q6)
D
Q7)
A
Q8)
D
Q9)
C
Q10)
B
Q11)
A
Copyright © 2004, Cisco Systems, Inc.
Troubleshooting Cisco Unity
6-45
6-46
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Module 7
Escalating Cisco TAC Service
Requests
Overview
If you follow the common troubleshooting techniques and are still unable to resolve an IP
telephony problem, you should escalate the problem to either the Cisco Technical Assistance
Center (TAC) or the telephone service provider.
Module Objectives
Upon completing this module, you will be able to employ Cisco TAC as a troubleshooting and
escalation tool.
Module Objectives
• Open a Cisco TAC service request for escalation
and resolving VoIP issues using the appropriate
Cisco TAC tool
• Open a Cisco TAC service request using available
Cisco resources
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—7-2
Module Outline
The outline lists the components of this module.
Module Outline
• Using the Cisco TAC and Opening service
requests
• Cisco TAC Service Requests and Telephone
Service Providers
© 2004 Cisco Systems, Inc. All rights reserved.
7-2
Cisco IP Telephony Troubleshooting (IPTT) v4.0
IPTT v4.0—7-3
Copyright © 2004, Cisco Systems, Inc.
Using the Cisco TAC
Overview
The Cisco TAC provides a home page with a variety of tools to assist in troubleshooting
problems with Cisco equipment to all customers, partners, and resellers. This lesson will teach
you about the Cisco online technical knowledge bases, tools, and software that you can use to
solve IP telephony issues. You will learn how to use the TAC home page and Bug Toolkit to
research common network errors and bugs, and to answer questions about future upgrades.
Relevance
Before escalating the problem to a Cisco TAC engineer, reference the Cisco TAC home page
for useful information. Other individuals may have run into the same bug or specific problem
that you are facing and have posted their findings with the Cisco TAC, which updates the top
issues about common network errors regularly. The Cisco TAC home page also contains links
to reference tools, community groups, and field notices.
Objectives
Upon completing this lesson, you will be able to:
Describe the features and functions of the Cisco TAC
List the features and functions of the Cisco TAC home page
Describe and explain Bug Toolkit functions
List the methods of opening trouble tickets with the Cisco TAC
Describe the different TAC severity levels at which a case can be placed
List the methods of reporting and escalating trouble tickets to the telephone service
provider
Illustrate how to document the problem resolution
List the methods of escalating trouble tickets using severity levels
Learner Skills and Knowledge
To benefit fully from this lesson, you must have these prerequisite skills and knowledge:
Basic understanding of the Microsoft Windows 2000 operating system
Outline
The outline lists the topics included in this lesson.
Outline
•
•
•
•
•
Overview
Cisco TAC Overview
Cisco TAC Home Page
Bug Toolkit
Escalation Process Opening a Cisco TAC Service
Request
• Severity Levels
• Summary
• Quiz
© 2004 Cisco Systems, Inc. All rights reserved.
7-4
Cisco IP Telephony Troubleshooting (IPTT) v4.0
IPTT v4.0—7-3
Copyright © 2004, Cisco Systems, Inc.
Cisco TAC Overview
This topic describes the Cisco TAC, a technical support service available to all customers,
partners, and resellers. Technical assistance from Cisco is available via the web, e-mail, and
telephone to customers, partners, and resellers who hold a valid Cisco service contract.
Cisco TAC Overview
PSTN
ACD
• Available to all customers, partners, and resellers
• Available via web, e-mail, and telephone
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—7-4
Customers and partners who are registered with Cisco.com, and who hold a valid Cisco Service
Agreement and Cisco Partner Service Agreement, have full access to the Cisco TAC website
tools and content.
Note
If you hold a valid service agreement, but do not have a login ID or password, you can visit
the following website to become a registered Cisco.com user:
http://www.cisco.com/register/.
Note
Valid Cisco service agreement holders include Cisco SMARTnet, Cisco SMARTnet Onsite,
and Cisco Network Supported Accounts (NAC).
Copyright © 2004, Cisco Systems, Inc.
Escalating Cisco TAC Service Requests
7-5
Cisco TAC Home Page
This topic discusses the Cisco TAC home page, which you can access at
http://www.cisco.com/tac. The Cisco TAC home page is the launch point for accessing online
troubleshooting tools, upgrading software, and opening severity level 3 (P3) and severity level
4 (P4) service requests with a Cisco TAC engineer.
Cisco TAC Home Page
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—7-5
The Cisco TAC website provides more than 30,000 pages of technical content, including online
knowledge bases and tools. The Search tool can shorten your research session by helping you
locate relevant technical content.
Cisco has divided the TAC website into modules. The first three modules, Hardware Support,
Software Support, and Technology Support, contain all of the technical content on the Cisco
TAC website. These modules provide an intuitive and easy way to navigate through the
technical documents and tools on the site. Under the title bar of each module, you will find the
three most frequently accessed documents. To access the full set of documents contained in
each module, you can either click the title bar for each module, or click More, which is located
beneath the frequently accessed documents.
Additional modules provide fast access to specific types of information, such as technical
support tools, software downloads, the latest TAC news, and TAC contacts. The site also
features prominent links to key resources, such as the Case Open tool, the new TAC Advanced
Search, and Training Resources.
Inside each module, you will see the full list of products or technology topics displayed
alphabetically on the left navigation bar. The body of the page displays the most frequently
accessed hardware, software, or technology topics as a featured topic. The left navigation bar
tracks your movement on the Cisco TAC website, allowing you to navigate through the site
without losing your place.
7-6
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Use this review of home page links to begin your investigation of Cisco TAC website
resources:
The Tools and Utilities module gives you direct access to all of the Cisco TAC tools,
including the new TAC Advanced Search.
—
—
—
TAC Advanced Search is a powerful and easy-to-use tool that gives you the ability
to refine your search criteria to obtain superior results.
Search results include only technical support documents and resources, so that you
do not have to wade through unrelated content, such as news postings and presales
information.
Use the TAC Advanced Search tool to search for all types of TAC information,
including software bugs, TAC service requests, and error messages.
The What’s Hot in TAC module provides the latest technical support news and updates
from the Cisco TAC.
The Software Center Downloads module offers direct access to the Cisco Software Center
where you can research and download software.
The Contact TAC module includes all of the information that you must provide to open or
manage a TAC service request.
The Cisco TAC website also offers prominent links to frequently requested tools and
resources.
—
—
Note
On the left side of the page, you will find a link to the Case Open tool, along with
links to general TAC information, including the popular TAC Newsletter.
On the right side of the page, under Related Links, you will find links to training,
Networking Professionals Connection, and the Service Contract Center.
If you need help navigating the Cisco TAC website, you can work directly with a Cisco
representative via browser synchronization. For more information please visit
http://www.cisco.com/tac/ciscolive.
The Cisco TAC web seminars provide training on how to use Cisco TAC website resources
effectively. The seminars teach you how to find the technical information necessary to design
and support your networks, enhance your networking skills, implement and configure products
and networks, and troubleshoot network issues.
New seminars are available in languages other than English or that deal with specific
technologies. Recorded versions are also available for viewing at your convenience.
Copyright © 2004, Cisco Systems, Inc.
Escalating Cisco TAC Service Requests
7-7
Tools and Utilities
You must log into Cisco Connection Online – first.
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—7-6
For a complete list of online tools, choose Tools and Utilities from the Cisco TAC home page.
Many valuable, time-saving tools are available in the Tools and Utilities module, including the
following:
2600/3600 Memory Calculator: Computes the memory required for the Cisco 2600/3600
Series router
Voice Codec Bandwidth Calculator: Determines the amount of bandwidth needed for
different numbers of calls using various coder-decoder (codecs)
Cisco Feature Navigator: Matches Cisco IOS features to Cisco hardware platforms
Compatibility Advisor for the Catalyst 5000 and 6000: Identifies valid hardware
configurations for Catalyst software for supervisor engine software
Hardware/Software Compatibility Matrix: Determines the compatibility between
specific product numbers and software releases
Output Interpreter: Decodes Cisco IOS software commands and provides relevant errors
or warnings
Software Bug Toolkit: Identifies, evaluates, categorizes, and tracks defects that have a
real, or potential, impact on network operations or planning
Troubleshooting Assistant: Simulates the steps that a Cisco TAC engineer takes to
diagnose problems and provides a technical solution or recommendation
7-8
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Software Center
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—7-7
You can search, access, and download software and firmware upgrades from the Software
Center.
Note
To access the Software Center click Software Center on the Cisco TAC home page or go
to http://www.cisco.com/kobayashi/sw-center/.
The Software Center also contains links to other software-related technical support
applications, such as the Software Search Tool and Cisco IOS Upgrade Planner.
The Cisco TAC home page is a gateway to the many invaluable resources that are available
from the Cisco TAC. Familiarizing yourself with this home page will help you find exactly
what you need as quickly as possible.
Copyright © 2004, Cisco Systems, Inc.
Escalating Cisco TAC Service Requests
7-9
Bug Toolkit
This topic describes the Bug Toolkit, which allows you to determine if the problem you are
having is due to a known bug, and if so, what recourse to take.
Bug Toolkit
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—7-8
To access the Bug Toolkit, choose Site Map at the top of the main page at Cisco.com and then
choose Tools & Utilities. You will have to log in before the Bug Toolkit will be available.
The Bug Toolkit allows you to search for known bugs based on a known bug ID, software
version, feature set, product name, or keywords.
For example, the Bug Toolkit will complete the following actions:
1. Look for the defect that is causing the current symptoms
2. Determine the software release that is best for the situation
Note
The Bug Toolkit shows product defects, not enhancement bugs. Enhancement bugs should
not be visible.
For symptom diagnostics, perform these basic steps:
Step 1
Choose the major Cisco IOS release version, such as 11.3 or 12.1.
Step 2
Choose the maintenance revision of that release, if available, such as (12), (12a).
Note
7-10
In other words, this request is seeking to find the defects that are still present in this
particular Cisco IOS release.
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Step 3
In the Keyword field, enter any symptoms or other clues regarding the nature of the
problem, such as an error message or the action that caused the problem. (It is
generally not useful to specify a feature in this mode.)
Step 4
Click Search.
Note
Not all products have the version and feature selection lists described. However, regardless
of the available search criteria, the utility searches in the same manner. You can skip the
steps that do not apply.
For upgrade planning, perform these steps:
Step 1
Choose the major version to which you would like to upgrade.
Step 2
Choose the desired maintenance revision.
Step 3
Choose the features that are critical to your network functionality.
Step 4
Click Search.
By default, the Bug Toolkit uses a strict search method to retrieve defects that match the chosen
criteria, such as features and version, to assist with upgrade planning questions. The Keyword
field has a special attribute that allows the search more freedom. The search raises the score or
ranks the documents that contain the keywords or complex queries that are entered.
You may choose any of the available search criteria. If you do not choose criteria, the search
displays all defects for the targeted product. If you use this method, the Summary display can
further target the results by feature or severity.
You can disable the strict search. However, the defects truly relevant to the desired search may
not always be clear.
New features of the Bug Toolkit include the following:
Users can filter for a specific maintenance revision.
Feature summary provides per feature summaries of defects in each severity (1–4).
The summary displays the Fixed-in field, which shows which version of software fixed the
bug.
The summary displays a report on how many of the displayed defects are fixed in later
versions.
Severity is the value chosen and lower. For example, if a severity of 3 is chosen, severity 2
and severity 1 defects are also shown.
Copyright © 2004, Cisco Systems, Inc.
Escalating Cisco TAC Service Requests
7-11
Bug Toolkit Results
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—7-9
To obtain the diagnostic results of the reported symptoms, perform these actions:
On the Bug Toolkit results page, the detailed listing will provide the most likely causes of
the problem in rank order.
Click Bug ID to get the bug details and release note information (if any) for that defect.
For upgrade planning, note the following:
The Summary Report provides a comparison of the defect load for each feature chosen.
Use the hyperlinks that are available in the Summary Report to limit the results to more
detailed views, based on feature or severity.
The table explains the different bug statuses that will be listed as Bug Toolkit results.
7-12
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Bug Toolkit Statuses
Status
Description
Assigned
A bug report is assigned to an engineer, who is then responsible for either resolving
the bug or reassigning it. Normally, a development engineering manager assigns a
bug report to an engineer who is competent in the area of the problem.
Closed *
A bug report is valid, but a conscious decision has been made not to fix it at all or in
all releases. Normally, a development engineering manager moves a bug report to
this status. This status is not available in all projects.
Duplicate *
This bug report describes the same problem as another bug report.
Forwarded
A bug report is being forwarded to the appropriate project, because it was previously
submitted for the wrong project.
Held
A fix to the problem exists, but development engineering work is on hold pending
information from an outside source (for example, waiting for the customer to verify
the fix).
Information
required
This is a holding status for bug reports that are awaiting additional information
needed to determine the cause of the problem. This is easily retrieved information,
such as a trace or dump, or a more descriptive definition of the problem and
symptoms. This status is also used when a diagnostic or special image has been
provided to the customer to gather more information to continue the analysis. In
either service request, development engineering cannot make progress until the
required information is provided.
Junked *
A bug report is discarded because it does not describe a problem that requires
changes to hardware, software, or documentation.
More
A problem described in the bug report is fixed and tested in some, but not all,
versions in which it is to be fixed. Classifying a bug report with this status allows the
fixed code to be integrated into some releases. The engineer moves the bug report to
Resolved status when the problem is fixed in all versions.
New
This is a new bug report. DDTS classifies bug reports with the New status when they
are first entered into the database. A bug keeps this status until it is evaluated.
Open
The assigned engineer is actively working on the bug report. Normally, the assigned
engineer moves the bug report to this status to indicate that work is in progress.
Postponed
This is the holding status for bug reports that are not being actively addressed,
because the assigned engineer cannot handle them quickly or the engineering
manager has given them a lower severity.
Resolved *
A problem described in the bug report is fixed in all release versions that are targeted
to be fixed*, and all changes have been successfully tested. Normally, the assigned
engineer moves a bug report into this status.
NOTE: In certain rare circumstances, Cisco is unable to fix the bug in all versions in
which it is found. The defect will still have a Resolved status. Please contact the
Cisco TAC if a defect in this condition affects you.
Submitted
This is the status of bug report when it is initially submitted. This is a transitory status.
When the bug report is entered into the DDTS database, the status is changed to
New.
Irreproducible *
The evaluating or test engineer cannot reproduce a problem.
Verified *
This status means that a fix for the problem has been tested. This is the final resting
place for all fixed bug reports that are verified by a test engineer. Normally, the test
engineer moves a bug report to this status. Note that this status is rarely used.
Waiting
A bug represents an authentic hardware, software, or documentation problem that is
worthy of engineering attention, but no one is available to work on it at this time. A
development engineer, release engineer, or senior customer engineer, moves a bug
report to this status. The person who moves the bug to this status has reproduced
most bug reports with this status.
* The asterisk denotes major bug status.
Copyright © 2004, Cisco Systems, Inc.
Escalating Cisco TAC Service Requests
7-13
Bug Watcher
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—7-10
You can save searches and receive updates and alerts when new information is available about
a bug.
Saved Searches
The Bug Watcher tool allows you to create and manage saved searches. The ability to save a
search allows you to constantly monitor your specific network situation. After you save a
search, you can edit the search at any time to change the alert conditions, the defects watched,
or the network profile.
To create a saved search, follow these steps:
7-14
Step 1
Implement a search using one of the three functions listed under Search for Bugs.
Step 2
In the search result window, click This Search Criteria.
Step 3
Input a name for the saved search in the text box.
Step 4
Choose an existing group name or create a new group name for this saved search.
This group will contain the bugs identified using the search criteria that you
previously saved. Each time a new bug meets the search criteria, the search adds the
bug to the group that you have chosen.
Step 5
Click Save to save the changes.
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Bug Groups
The Bug Watcher tool allows you to create collections, or bins, of defects that you can use to
monitor the status of specific defects. When the status of a defect changes—for example, when
a software release integrates a fix—you can view the status of that defect in real time or choose
to receive e-mail update notifications of those changes. Bug Watcher can also continuously
update a bug group with new defects that match specific saved search criteria. When a new
defect that matches a saved search criteria is available, the defect will appear on your Bug
Group display under the list of watched defects. You then have the option of choosing any of
the new defects to watch.
E-Mail Notification Rules
To receive an e-mail notification when the status of a bug changes, you must include your email address in the profile information while creating or editing the bug group.
Note
You can specify that you want to receive e-mails for all status changes in a defect or for a
major status change only.
To create a bug group, complete the following steps:
Step 1
Implement a search using one of the three functions listed under Search for Bugs.
Step 2
In the search result window, check the boxes next to the bug or bugs that you want
to include in a bug group.
Step 3
Click My Selected Bugs.
Step 4
Choose an existing group name or create a new group name.
Step 5
Click Save to save the changes.
Note
You will have an opportunity to save your search criteria, if this is applicable.
My Stuff Tab
The My Stuff tab allows you to view, create, and/or modify existing bug groups or saved
searches. Click the My Stuff tab to see a list of all of your bug groups.
For example, to monitor defects in the flow-switching code for Cisco IOS software, use the
Bug Toolkit to choose the flow-switching feature set, and the correct software version. Then,
execute the search. Each defect in the results window has a checkbox that you use to choose
that defect for your flow-switching bug group. After choosing the defects from the initial
search, you can also subscribe to Saved Search, which advises you of the new defects that
effect flow switching in the versions that you have chosen.
Copyright © 2004, Cisco Systems, Inc.
Escalating Cisco TAC Service Requests
7-15
Escalation Process
This topic discusses the escalation process for referring problems to a Cisco TAC engineer.
Escalation Process
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—7-11
If you are unable to resolve a problem using the tools and resources that are available on the
Cisco TAC website, you should open a service request with a TAC engineer. You can open a
service request with TAC via the Cisco TAC website or by e-mail or telephone. Before opening
a TAC service request, you will need to be aware of the severity level of the problem.
7-16
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Severity Levels
This topic describes the four problem severity levels identified by the Cisco TAC team.
Severity Levels
• Severity 4: You need information concerning Cisco
product capabilities, installation advice, or basic
product configuration data.
• Severity 3: Network functionality is impaired, but
most business operations continue.
• Severity 2: The production network is severely
degraded and has an impact on significant aspects
of your business operations.
• Severity 1: The production network is down, with
the potential of causing critical impact to business
operations.
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—7-12
The four severity levels for a problem are as follows:
Severity 4 (S4): You need information concerning Cisco product capabilities, installation
advice, or basic product configuration data.
Severity 3 (S3): Your network performance is degraded. Network functionality is
noticeably impaired, but most business operations continue.
Severity 2 (S2): Your production network is severely degraded and affects significant
aspects of your business operations. You are willing to commit full-time resources during
business hours to resolve the situation.
Severity 1 (S1): Your production network is down, with the potential of causing critical
impact to business operations if service is not restored quickly. You are willing to commit
substantial resources around the clock to resolve the situation.
Copyright © 2004, Cisco Systems, Inc.
Escalating Cisco TAC Service Requests
7-17
Summary
This topic summarizes the key points discussed in this lesson.
Summary
• The Cisco TAC is a technical support service
available to customers and partners to resolve issues
related to Cisco products.
• The Cisco TAC home page provides troubleshooting
tools, software upgrades, and problem escalation
utilities.
• The “Bug Watcher” (Toolkit) allows you to determine
if a problem is a known issue and to monitor
existing bugs.
• You can escalate unresolved problems to a Cisco
TAC engineer through various methods.
• The Cisco TAC team has identified four priority levels
for a problem.
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—7-13
References
For additional information, refer to these resources:
Cisco TAC Quick Reference Guide:
http://www.cisco.com/public/news_training/TAC-leaflet.pdf.
Cisco TAC home page: http://www.cisco.com/tac.
Technical Support on Cisco.com: http://www.cisco.com/kobayashi/support/tac/index.shtml.
7-18
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Quiz
Use the practice items here to review what you learned in this lesson. The correct answers are
found in the Quiz Answer Key.
Q1)
Which of the following methods is NOT a way to reach Cisco TAC technical support?
A)
B)
C)
D)
Q2)
Which of the following is a valid method of accessing the Cisco TAC website?
A)
B)
C)
D)
Q3)
B)
C)
D)
a known problem with the TFTP server starting in Cisco CallManager version
3.3
an enhancement available for Cisco CallManager version 3.3 that increases
telephone capacity
an incompatibly issue between Cisco CallManager version 3.3 software and
Hewlett Packard servers
none of the above
You are having problems with your voice network. When not on a call, IP Phones
randomly reboot about once an hour. While this does not cause employees to miss any
calls, it is very annoying. Under which Cisco TAC severity level should you file your
complaint?
A)
B)
C)
D)
Q5)
http://www.cisco.com/tac
http://tac.cisco.com
http://www.tac.com
none of the above
Which of the following would be located through the Bug Toolkit utility?
A)
Q4)
e-mail
telephone
mail
website
priority level 1
priority level 2
priority level 3
priority level 4
You must download a firmware upgrade for your Cisco router. Which area of the Cisco
TAC website should you visit?
A)
B)
C)
D)
Download Resources
Bug Finder
Software Locator
Software Center
Copyright © 2004, Cisco Systems, Inc.
Escalating Cisco TAC Service Requests
7-19
Quiz Answer Key
7-20
Q1)
C
Q2)
A
Q3)
A
Q4)
C
Q5)
D
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Cisco TAC Service Requests
and Telephone Service
Providers
Overview
The Cisco TAC is a technical support service available to all customers, partners, and resellers.
This lesson will teach you how to open a Cisco TAC service request on line and via telephone.
You will also learn the procedure for contacting a telephone service provider if you are
troubleshooting a telco-related problem.
Relevance
P1 and P2 problems are handled by Cisco TAC engineers. When a P1 or P2 situation occurs,
you should contact the Cisco TAC via telephone and open a case immediately. The online
resources that are available at the Cisco TAC website address the majority of P3 and P4
problems.
Objectives
Upon completing this lesson, you will be able to:
Describe how to use the Case Open tool
List the telephone numbers to contact the Cisco TAC
Explain how to query various open Cisco TAC service requests
Describe the call routing process for Cisco TAC service requests
Explain how you would escalate a Cisco TAC service request
List why you would want to keep a Cisco TAC service request open
Illustrate documenting a Cisco TAC service request resolution
Describe the various processes for escalating a Cisco TAC service request
Learner Skills and Knowledge
To benefit fully from this lesson, you must have these prerequisite skills and knowledge:
General knowledge of the managerial structure of telco service providers
Knowledge of the test equipment and procedures used by telco service providers
Basic understanding of the Microsoft Windows 2000 operating system
Outline
The outline lists the topics included in this lesson.
Outline
•
•
•
•
•
•
•
•
•
•
•
Overview
Opening a Service Request On Line
Cisco TAC Telephone Numbers
Cisco TAC Service Request Querying Options
Contacting a Cisco TAC Center
Escalating a Case
Keeping a Case open
Documenting the Resolution
Escalation Summary
Summary
Quiz
© 2004 Cisco Systems, Inc. All rights reserved.
7-22
Cisco IP Telephony Troubleshooting (IPTT) v4.0
IPTT v4.0—7-3
Copyright © 2004, Cisco Systems, Inc.
Opening a Service Request On Line
This topic describes using the Cisco TAC Service Request Tool.
TAC Service Request Tool
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—7-4
P3 and P4 cases can be opened online via the Case Open tool. The online resources available
on the Cisco TAC website address the majority of P3 and P4 problems. However, if you are
unable to find the answer to a P3 or P4 problem using the tools and resources available on the
Cisco TAC website, you can open a TAC service request by clicking the Opening a Service
Request link on the left side of the Cisco TAC home page.
Note
You can also access the TAC Service Request Tool at
http://www.cisco.com/en/US/support/index.html. The Service Request Open tool
automatically suggests solutions during this process.
If the suggested solutions does not solve the service request, the service request is transferred to
a TAC engineer for further assistance.
Note
When submitting a case to the Cisco TAC, be clear and descriptive. Simply indicating that
something does not work is not descriptive enough and only delays the handling of your
case. Whenever possible, provide symptoms, attempted solutions, and unusual behavior in
the problem description. When contacting the service department, have all pertinent
information available. Double-check company information, contract numbers, and contact
information to ensure that your case is handled correctly and efficiently.
All P3 and P4 cases opened via the Cisco TAC website are given expedited handling over the
same level cases opened via telephone. Often, TAC engineers can solve a case quickly because
they have seen the problem before.
Copyright © 2004, Cisco Systems, Inc.
Escalating Cisco TAC Service Requests
7-23
Cisco TAC Telephone Numbers
This topic lists the telephone numbers for opening a Cisco TAC service request.
TAC Telephone Numbers
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—7-5
Cisco TAC engineers handle P1 and P2 problems directly. When a P1 or P2 situation occurs
(for example, when your production network is down or severely degraded), you should contact
the Cisco TAC via telephone and open a case immediately. Telephone access to Cisco TAC
engineers is available to all customers and partners who are holders of Cisco service contracts.
7-24
Note
The following URL contains a directory of Cisco TAC team toll-free numbers:
http://www.cisco.com/warp/public/687/Directory/DirTAC.shtml
Note
You can also call the Cisco TAC when you have a P3 or P4 problem and do not have
website access to use the Case Open tool.
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Cisco TAC Service Request Querying Options
This topic discusses the specific query options available when querying a Cisco TAC service
request.
Query Options
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—7-6
Use the form shown in the figure to add technical information and other notes to an existing
Cisco TAC service request. To update a case, select the contract or key number (in an
individual case) that you want to update. Fill in the optional information and click Submit
Query. A window appears with either a listing of cases that you may query again or an
individual case that may be updated below the Case History notes. You can also click Submit
Query to retrieve a list of cases recorded under the maintenance contract number for your
organization. The Status Codes shown in the Case Management tool are explained here:
CE-Pend: The Cisco TAC engineer assigned to your case is currently investigating or
troubleshooting the reported problem. No workaround has been identified at this time.
Close-Pend: The Cisco TAC engineer has provided you with a solution for your issue. If
you feel that the problem has not been solved, contact the assigned Cisco TAC engineer.
Cust-Pend: The Cisco TAC engineer assigned to your case has requested information from
you and is awaiting your response. No workaround has been identified at this time.
DE-Pend: The Cisco TAC engineer has submitted a development engineering request and
has forwarded the request to Cisco Development Engineering for investigation.
RMA-Out: The Cisco TAC engineer has sent you a hardware replacement.
Note
To check the status of a case and add updates, use the Cisco TAC service request
management tools at http://tools.cisco.com/ServiceRequestTool/create/launch.do.
Copyright © 2004, Cisco Systems, Inc.
Escalating Cisco TAC Service Requests
7-25
Contacting a Cisco TAC Center
This topic presents suggestions that can make escalating an issue to a telephone service
provider easier.
Call the Main Repair Number for
Businesses
PSTN
© 2004 Cisco Systems, Inc. All rights reserved.
ACD
IPTT v4.0—7-7
Consider the following process when escalating an issue to a telephone service provider:
1. Sufficiently test the problem, so that you are certain that the problem is with the telephone
service provider. Use a transmission test set to test analog trunks. Use a T1 tester to verify
that the T1 circuit is down. If you do not have a tester, swap out the T1 interface card to
make sure that there is not a problem in the hardware.
2. Be aware that your service ticket is not the only ticket that the service center is working on.
To ensure that you get the fastest service, call in the trouble ticket that has the most
complete information possible. The solution to the problem will be delayed if you give the
service provider incorrect or incomplete information.
3. Call the main repair number for businesses, which is located in your yellow pages. You
may have to wait in an automatic call distributor (ACD) queue before your call is routed to
the correct person.
Note
7-26
Be sure to call the correct number for repair. There are usually different numbers for
ordering, customer service, and other departments. Calling the wrong number increases the
amount of time spent waiting to talk to a service technician.
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Before calling the telephone service provider for help, gather the following information:
Circuit ID
Main telephone number, which is usually the first direct inward dial (DID) number in the
block of DIDs assigned to the circuit
An exact description of the problem. Here is an example:
—
Battery (side tone) on line but no dial tone
—
Red alarm on T1
—
No 911 information sent to 911 center on call
—
Improper DID digit coming in on channel associated T1
Contact person name and telephone number
Repair tickets are delayed when the service provider returns the call and the given contact
person is not available. The service provider will typically leave a message and put the ticket on
hold until the contact person calls back.
However, if the service provider is not busy, your ticket may be handled right away.
After a ticket is called in, give the telephone service provider time to look into the problem.
Within 24 hours, the person who initially took the call will give you some indication of how
long the repair should take. If this time frame is unacceptable, politely explain your time
restrictions to the person taking the call.
Note
If the repair is, in fact, an emergency, ask that the trouble ticket be escalated.
After the initial call, allow the service provider the estimated time frame to make the repair. If
the repair is not completed within the time frame, the provider should call the contact person to
explain why the repair is taking longer than expected.
If you do not hear from the service provider within the estimated time frame, call the repair
number again and ask for an update on the status of the ticket. You may be told that they need
to conduct further testing or that a technician has been dispatched to the site. This type of
troubleshooting will add more time to the ticket.
If the service provider response is unacceptable, decide whether you must escalate the problem
to the service center supervisor level. If the problem is a minor problem, consider waiting a
little while longer. However, if the problem is affecting customers, escalate to the next level of
management.
Copyright © 2004, Cisco Systems, Inc.
Escalating Cisco TAC Service Requests
7-27
Escalating a Case
This topic discusses the specifics on when you might want to escalate a case with the Cisco
TAC.
Escalation May Become Necessary
PSTN
© 2004 Cisco Systems, Inc. All rights reserved.
ACD
IPTT v4.0—7-8
To speak with a supervisor, ask the first level customer service representative to transfer you to
the next level supervisor or manager. If the service representative is unable or unwilling to
transfer you (it may be against an internal policy to transfer you), ask that the supervisor call
you back. Request the name of the representative who is handling your call and the name of the
supervisor who will return your call.
Allow the supervisor time to call you back. Plan on waiting at least 15 minutes. The customer
service representative needs time to brief the supervisor, and the supervisor may want to
research the problem before calling you. If you do not hear from the supervisor in a timely
manner, call the service provider back and ask for the supervisor directly.
Note
7-28
You or your customer may have a dedicated customer representative that works directly
with one of their contacts at the telephone service provider to resolve problems more
quickly.
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Keeping a Case Open
This topic discusses the specific on why you want to keep a Cisco TAC service request open.
Keep the Ticket Open
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—7-9
After the service provider has indicated that the problem is fixed, test the problem again. You
may be able to test the problem remotely, the customer may be able to test it for you, or you
may have to return to the site.
Do not take the word of the service provider that the problem is fixed. Your customer may not
be able to compensate for the loss of service between the time the service provider said that the
problem was fixed and the time you noticed that the problem was not fixed.
Note
Ask the service provider to keep the ticket open until you can verify that the problem has
been corrected. If you allow the ticket to be closed, and the problem still exists, the process
will have to start all over again.
Working with telephone service providers can be frustrating. The best course of action is to
thoroughly test the problem, provide a complete description to the service provider, and be
polite and courteous when talking with the repair personnel. Also, it is a good idea to give the
service provider the chance to correct the problem before you attempt to escalate the problem.
Copyright © 2004, Cisco Systems, Inc.
Escalating Cisco TAC Service Requests
7-29
Documenting the Resolution
This topic describes the importance of documenting the resolution of the problem.
Documentation is a critical aspect when troubleshooting networks.
Documenting the Resolution
Documentation should include:
• Symptoms of the problem
• Steps taken to correct the problem
• Effect of the problem on the overall network
• Cause of the problem
• Fix for the problem
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—7-10
When a networking problem is finally resolved, steps should be taken to document the problem
and the resolution of the problem. The documentation should include the symptoms of the
problem, steps that were taken to correct the problem, the effect of the problem on the overall
network, the cause of the problem, and, finally, the fix for the problem.
This documentation is helpful in the following ways:
If the problem returns, the fix that was used may not have been the actual fix. There could
be an underlying problem that needs to be addressed. This documentation can allow you to
start where the previous troubleshooting ended, saving time and effort.
If the problem occurs in a different part of the network, the documentation can be used to
repair the problem quickly.
7-30
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Escalation Summary
This topic summarizes the escalation processes that should be used with the Cisco TAC and
when contacting a telephone service provider.
Escalation Summary
PSTN
ACD
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—7-11
The following is a summary of the Cisco TAC escalation process :
To rule out a bug problem, use the Bug Watcher tool in the software Bug Toolkit.
If the problem is not bug-related, research the problem using the resources available on the
Cisco TAC home page.
If you are still unable to resolve the problem, escalate the problem to a Cisco TAC engineer
by opening a case with the TAC.
—
If the problem is a P3 or P4 case, use the Case Open tool available on the Cisco
TAC home page.
—
If the problem is a P1 or P2 case, use the appropriate telephone number to contact a
Cisco TAC engineer that handles your geographic region.
The following is a summary of the escalation process that should be used for telephone service
provider-related problems:
Test everything (CSU/DSU, WAN interface cards [WICs], router, and other equipment)
thoroughly to rule out a problem with the network equipment.
Collect all of the necessary information (circuit ID, contact telephone numbers, and other
information) before calling the telephone service provider.
Provide the service representative with all of the necessary information, including an exact
description of the problem, contact name and number, and other required information.
If you are not satisfied with the service or the turnaround time, you may want to escalate
the call to a supervisor.
Copyright © 2004, Cisco Systems, Inc.
Escalating Cisco TAC Service Requests
7-31
After the telephone service provider indicates that the problem has been resolved, instruct
the telephone service provider to leave the ticket open until you can thoroughly test and
verify that the problem has, in fact, been resolved.
In all troubleshooting situations, always document the problem and the resolution.
7-32
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Summary
This topic summarizes the key points discussed in this lesson.
Summary
• S3 and S4 cases can be opened on line via the Case Open tool.
• S1 and S2 cases can be handled by Cisco TAC engineers via
telephone.
• Escalate an issue to a telephone service provider if it is an
emergency or if the problem is affecting customers and you are
not satisfied with the time frame for addressing the problem.
• Appropriate problem documentation includes a description of
symptoms, corrective actions, effect of correction, cause of
problem, and the fix for the problem.
• The escalation process to the Cisco TAC includes using ruling
out bug, research on the Cisco TAC home page, and opening
the case with Cisco TAC—either on the home page or via
telephone.
• The escalation process to a telephone service provider includes
testing equipment, collecting information, and communicating
information. When the problem is resolved, leave the ticket open
until the problem has been tested.
© 2004 Cisco Systems, Inc. All rights reserved.
IPTT v4.0—7-12
References
For additional information, refer to these resources:
Cisco.com (requires CCO login and password):
http://www.cisco.com/kobayashi/support/tac/index.shtml.
Cisco Support (requires CCO login and password):
http://www.cisco.com/cgi-bin/Support/browse/index.pl?i=Technologies&f=1533.
Copyright © 2004, Cisco Systems, Inc.
Escalating Cisco TAC Service Requests
7-33
Quiz
Use the practice items here to review what you learned in this lesson. The correct answers are
found in the Quiz Answer Key.
Q1)
If a user opens a P3 TAC service request on the Cisco website, and then opens the
same case over the telephone, which case will Cisco respond to first?
A)
B)
C)
D)
Q2)
Which two priority levels will the Cisco TAC engineers answer directly? (Choose two.)
A)
B)
C)
D)
Q3)
check to see if this is a bug through the software Bug Toolkit
research the problem using the Cisco TAC home page
escalate the problem to a Cisco TAC engineer via the Cisco website
escalate the problem to a Cisco TAC engineer via the telephone
What is the proper method for opening an escalation case if you have a P3 or P4 level
problem?
A)
B)
C)
D)
7-34
circuit ID
IP address
an exact and thorough description of the problem
the main telephone number
If you are experiencing a problem related to your Cisco switch, what is the first thing
that you should do?
A)
B)
C)
D)
Q5)
priority level 1
priority level 2
priority level 3
priority level 4
Which of the following is NOT necessary when contacting a telephone service provider
for support?
A)
B)
C)
D)
Q4)
neither
website
telephone
both at the same time
call the appropriate Cisco TAC Bug Finder telephone number
use the Case Open tool on the Cisco TAC website
send the Cisco TAC an e-mail with a detailed problem description
none of the above
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Quiz Answer Key
Q1)
B
Q2)
A, B
Q3)
B
Q4)
A
Q5)
B
Copyright © 2004, Cisco Systems, Inc.
Escalating Cisco TAC Service Requests
7-35
7-36
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
IPTT
Lab Guide
Overview
This Lab Guide contains the lab exercises to complete for this course. The solutions
information is found in the Lab Exercise Answer Key.
When participating in these labs, please keep in mind that these labs are not cookbook lab
activities in which the documentation takes you step by step through finding the issues. You
must apply the lessons learned in this course in order to troubleshoot the issues to resolution or
identify the issues as per lab instructions.
Caution
Failure to repair the current problem before proceeding to the next trouble ticket can cause
unpredictable results in your Cisco CallManager environment.
Outline
This Lab Guide includes these exercises:
Lab Exercise 1-l: Applying Troubleshooting Methods
—
Lab applies to topics covered in Module 1, Lessons 1 and 2
Lab Exercises 2-1 through 2-8: Troubleshooting Cisco CallManager, Network Signaling,
and Dial Plan
—
Labs apply to topics covered in Module 2, Lessons 1, 2, 3, and 4
Lab Exercises 3-1 through 3-3: Cisco AVVID Troubleshooting Tools
—
Labs apply to topics covered in Module 3, Lessons 1, 2, and 3
Lab Exercises 4-1 through 4-6: Troubleshooting Network Infrastructure
—
Labs apply to topics covered in Module 4, Lessons 1, 2, 3, and 4
Lab Exercises 5-1 through 5-4: Troubleshooting Voice Quality Issues
—
Labs apply to topics covered in Module 5, Lessons 1, 2, and 3
Lab Exercises 6-1 through 6-3: Troubleshooting Cisco Unity
—
Labs apply to topics covered in Module 6, Lessons 1 and 2
Note to the Student
When conducting the labs in this course, use the troubleshooting methods learned in Module 1,
Lessons 1 and 2, for your baseline. The methodologies taught in Module 1 should provide a
solid foundation for finding lab issues.
2
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Lab Exercise 1-1: Trouble Ticket (Lab
Familiarization)
Lab Discovery and Call-Routing Failure
Case Number: 001
Objectives
After completing this exercise, you will be able to:
Discover the various components in your individual pods and become familiar with Cisco
CallManager, Unity, gateway configurations as well as the network configuration on your
IP Phones
Use proven problem isolation techniques to list the symptoms of common call-routing
problems
Apply diagnostic tools to solve classroom network voice-routing problems that simulate
real-life IPTT malfunctions
Lab Discovery
Log in to the Cisco CallManager, Unity, switch, and gateways and review the device
configurations. Make note of the IP Phone’s network configurations. Make note of the pods’ IP
Phones, active and standby Cisco CallManager, Unity services configuration, TFTP server IP
address, default gateway IP address, operational VLAN ID, and whether the phones have video
capabilities.
Problem Report
Users report that they can successfully complete local headquarters (HQ) calls. However, calls
to the remote branch extensions (y501 where y = pod number) fail or forward directly to voice
mail. Calls to other pods complete after some delay.
Restrictions
None; all tools and service aids can be used to identify the cause of the failure.
Additional Question
When the problem source is identified and resolved, explain why some calls were routed
successfully while others were not.
Copyright © 2004, Cisco Systems, Inc.
Lab Guide
3
Lab Exercise 2-1: Trouble Ticket (Paper Trace)
Call-Routing Failure
Case Number: 002
Objectives
After completing this exercise, you will be able to:
Use proven problem isolation techniques to list the symptoms of a phone that fails to
register with one or more Cisco CallManagers
Apply diagnostic tools to analyze a Cisco CallManager trace that simulates real-life IPTT
malfunctions
Problem Report
Users report that phones located at the central site are unable to call another local number;
however, all calls to remote phones work properly. Upon dialing a local, internal number, the
caller receives a fast-busy signal.
Restrictions
This is strictly a paper trace exercise.
Additional Question
When the problem source is identified and resolved, identify the steps needed for a phone to
properly place calls with its Cisco CallManager using trace entries to verify these steps.
4
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Lab Exercise 2-2: Trouble Ticket (Lab Equipment)
Call-Routing Failure
Case Number: 003
Objectives
After completing this exercise, you will be able to:
Use proven problem isolation techniques to list the symptoms of common call-routing
problems
Apply diagnostic tools to solve classroom network voice-routing problems that simulate
real-life IPTT malfunctions
Problem Report
Users report that calls to the branch office extensions fail. However, when users in other Cisco
CallManager clusters attempt to call your remote office, the call goes through successfully.
Restrictions
Only the Cisco CallManager Trace application may be used for this exercise. You will gather a
live trace, then analyze it to discover the source of the problem.
Additional Question
When the problem source is identified and resolved, explain why some calls were routed
successfully while others were not.
Copyright © 2004, Cisco Systems, Inc.
Lab Guide
5
Lab Exercise 2-3: Trouble Ticket (Paper Trace)
Call-Routing Failure
Case Number: 004
Objective
After completing this exercise, you will be able to:
Apply diagnostic tools to analyze a Cisco CallManager trace to determine if the trace
shows any abnormal or error conditions
Problem Report
An employee using extension y001 (where y = pod number) reports that all calls fail before the
employee can completely dial the number. The employee receives a fast-busy signal after
dialing a couple of digits when trying to place a call to a four-digit extension. Use the Trace file
to track down and troubleshoot this problem on Cisco CallManager.
Restrictions
You can use the paper trace and Cisco CallManager Administration utility to diagnose the
source of this problem.
6
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Lab Exercise 2-4: Trouble Ticket (Lab Equipment)
Call-Routing Failure
Case Number: 005
Objectives
After completing this exercise, you will be able to:
Use proven problem isolation techniques to list the symptoms of common call-routing
problems
Apply diagnostic tools to solve classroom network voice-routing problems that simulate
real-life IPTT malfunctions
Problem Report
Users are complaining that whenever they try to make a call to another pod from the analog
phone at the HQ (extension y401 where y = pod number), the call fails. The users state that they
receive a reorder tone when the fifth digit is dialed.
Restrictions
None; all tools and service aids can be used to identify the cause of the failure.
Additional Question
When the problem source is identified and resolved, explain why internal calls were routed
successfully while others were not.
Copyright © 2004, Cisco Systems, Inc.
Lab Guide
7
Lab Exercise 2-5: Problem Report (Lab
Equipment)
Call-Routing Failure to AutoAttendant
Case Number: 006
Objectives
After completing this exercise, you will be able to:
Use proven problem isolation techniques to list the symptoms of common call-routing
problems
Apply diagnostic tools to solve classroom network voice-routing problems that simulate
real-life IPTT malfunctions
Problem Report
Local calls to the AutoAttendant computer telephone integration (CTI) route point receive a
fast-busy signal when called.
Restrictions
None; all tools and service aids can be used to identify the cause of the failure.
8
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Lab Exercise 2-6: Problem Report (Lab
Equipment)
Call-Routing Failure to and from Cisco IP SoftPhone
Case Number: 007
Objectives
After completing this exercise, you will be able to:
Use proven problem isolation techniques to list the symptoms of common Cisco
CallManager problems
Apply diagnostic tools to solve classroom Cisco CallManager problems that simulate reallife IPTT malfunctions
Problem Report
When trying to use the Cisco IP SoftPhone to place a call, no line control is present. When a
local IP Phone user calls the local SoftPhone, the call goes directly to voice mail.
Restrictions
None; all tools and service aids can be used to identify the cause of the failure.
Copyright © 2004, Cisco Systems, Inc.
Lab Guide
9
Lab Exercise 2-7: Trouble Ticket (Lab Equipment)
Call-Routing Failure
Case Number: 008
Objectives
After completing this exercise, you will be able to:
Use proven problem isolation techniques to list the symptoms of common call-routing
problems
Apply diagnostic tools to solve classroom network voice-routing problems that simulate
real-life IPTT malfunctions
Problem Report
Local calls to headquarters and branches route successfully. However, no calls can be made
through the Public Switched Telephone Network (PSTN).
Restrictions
None; all tools and service aids can be used to identify the cause of the failure.
10
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Lab Exercise 2-8: Trouble Ticket (Lab Equipment)
Call-Routing Failure
Case Number: 009
Objectives
After completing this exercise, you will be able to:
Use proven problem isolation techniques to list the symptoms of common Cisco
CallManager problems
Apply diagnostic tools to solve classroom Cisco CallManager problems that simulate reallife IPTT malfunctions
Problem Report
When any phone is picked up, a reorder tone is immediately heard. No dialing can be done.
Restrictions
None; all tools and service aids can be used to identify the cause of the failure.
Copyright © 2004, Cisco Systems, Inc.
Lab Guide
11
Lab Exercise 3-1: Trouble Ticket (Lab Equipment)
Call-Routing Failure
Case Number: 010
Objectives
After completing this exercise, you will be able to:
Use proven problem isolation techniques to list the symptoms of device registration
problems
Apply diagnostic tools to solve classroom network voice device–related problems that
simulate real-life IPTT malfunctions
Problem Report
The Cisco CallManager redundancy is not working correctly. If the primary Cisco CallManager
fails, the phones in the branch and HQ offices are unable to connect to the secondary Cisco
CallManager server. Any new phones that are connected to the network fail to register at all.
Restrictions
None; all tools and service aids can be used to identify the cause of the failure.
Additional Question
When the problem source is identified and resolved, explain why some devices failed to
register.
12
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Lab Exercise 3-2: Trouble Ticket (Lab Equipment)
Call-Routing Failure
Case Number: 011
Objectives
After completing this exercise, you will be able to:
Use proven problem isolation techniques to list the symptoms of common Cisco
CallManager problems
Apply diagnostic tools to solve classroom network call setup problems that simulate reallife IPTT malfunctions
Problem Report
A new employee has been hired. You must add a new user object to the Cisco CallManager
Lightweight Directory Access Protocol (LDAP) directory to allow the new employee to control
his or her new IP Phone. While attempting to add a new user, you receive an error message
stating that a new user object cannot be created.
Restrictions
None; all tools and service aids can be used to identify the cause of the failure.
Copyright © 2004, Cisco Systems, Inc.
Lab Guide
13
Lab Exercise 3-3: Trouble Ticket (Lab Equipment)
Phone Registration Problem
Case Number: 012
Objectives
After completing this exercise, you will be able to:
Use proven problem isolation techniques to list the symptoms of common device
registration problems
Use a LAN monitor and Trace tool to solve a classroom network device registration
problem that simulates a real-life IPTT malfunction
Problem Report
The Cisco CallManager redundancy is not working correctly. If the primary Cisco CallManager
fails, the phones in the branch and HQ offices are unable to connect to the secondary Cisco
CallManager server.
Restrictions
None; all tools and service aids can be used to identify the cause of the failure.
14
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Lab Exercise 4-1: Trouble Ticket (Lab Equipment)
Call-Routing Failure
Case Number: 013
Objectives
After completing this exercise, you will be able to:
Use proven problem isolation techniques to list the symptoms of common Cisco
CallManager problems
Apply diagnostic tools to solve classroom network call-setup problems that simulate reallife IPTT malfunctions
Problem Report
Users report that calls fail or go directly to voice mail when they dial the pod remote branch
extensions (y501 where y = pod number). In addition, there is a delay before reaching some
extensions in other offices.
Restrictions
None; all tools and service aids can be used to identify the cause of the failure.
Copyright © 2004, Cisco Systems, Inc.
Lab Guide
15
Lab Exercise 4-2: Trouble Ticket (Lab Equipment)
One-Way Voice
Case Number: 014
Objectives
After completing this exercise, you will be able to:
Use proven problem isolation techniques to list the symptoms of common Cisco
CallManager problems
Apply diagnostic tools to solve classroom one-way voice problems that simulate real-life
IPTT malfunctions
Problem Report
Calls from the analog phone on the HQ router (Foreign Exchange Station [FXS] phone,
extension y401 where y = pod number) are failing.
Restrictions
None; all tools and service aids can be used to identify the cause of the failure.
Additional Question
When the problem source is identified and resolved, identify additional steps that you might
need to take if you encounter this particular problem.
16
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Lab Exercise 4-3: Trouble Ticket (Lab Equipment)
Call-Routing Failure
Case Number: 015
Objectives
After completing this exercise, you will be able to:
Use proven problem isolation techniques to list the symptoms of common call-routing
problems
Apply diagnostic tools to solve classroom network voice-routing problems that simulate
real-life IPTT malfunctions
Problem Report
Calls made from extension y401 (where y = pod number) can successfully route to other
intracluster extensions, but five-digit dialing cannot be used to reach extensions in other
clusters.
Restrictions
None; all tools and service aids can be used to identify the cause of the failure.
Additional Question
When the problem source is identified and resolved, explain why some calls were routed
successfully while others were not.
Copyright © 2004, Cisco Systems, Inc.
Lab Guide
17
Lab Exercise 4-4: Trouble Ticket (Lab Equipment)
Call-Completion Failure
Case Number: 016
Objectives
After completing this exercise, you will be able to:
Use proven problem isolation techniques to list the symptoms of common call-completion
problems
Apply diagnostic tools to solve classroom network voice-routing problems that simulate
real-life IPTT malfunctions
Problem Report
Calls made from extension y401 (where y = pod number) to other student pods get a ringback
tone, but when the extension is picked up, the ringback tone continues, then goes busy. The
called party hears dead silence.
Restrictions
None; all tools and service aids can be used to identify the cause of the failure.
Additional Question
When the problem source is identified and resolved, explain why this failure resulted in the
symptoms observed.
18
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Lab Exercise 4-5: Trouble Ticket (Gatekeeper)
Call Rejection
Case Number: 017
Objectives
After completing this exercise, you will be able to:
Use proven problem isolation techniques to list the symptoms of common call-routing
problems
Apply diagnostic tools to solve classroom network voice-routing problems that simulate
real-life IPTT malfunctions
Problem Report
Users report that when they call a remote site over the ICT, they get a reorder tone or a message
indicating the call cannot be completed as dialed. Users also report that local internal calls work
just fine. Try to make calls to other pods.
Restrictions
None; all tools and service aids can be used to identify the cause of the failure.
Additional Question
When the problem source is identified and resolved, explain why some calls were routed
successfully while others were not.
Copyright © 2004, Cisco Systems, Inc.
Lab Guide
19
Lab Exercise 4-6: Trouble Ticket (Gatekeeper)
Call Rejection
Case Number: 018
Objectives
After completing this exercise, you will be able to:
Use proven problem isolation techniques to list the symptoms of common call-routing
problems
Apply diagnostic tools to solve classroom network voice-routing problems that simulate
real-life IPTT malfunctions
Problem Report
Users report that when they call a remote site over the CTI, they get a reorder tone or a message
indicating the call cannot be completed as dialed. Users also report that local internal calls work
just fine. Try to make calls to other pods.
Restrictions
None; all tools and service aids can be used to identify the cause of the failure.
Additional Question
When the problem source is identified and resolved, explain why some calls were routed
successfully while others were not.
20
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Lab Exercise 5-1: Trouble Ticket (Lab Equipment)
Voice-Quality Problem
Case Number: 019
Objectives
After completing this exercise, you will be able to:
Use proven problem isolation techniques to list the symptoms of voice-quality problems
Apply diagnostic tools to solve classroom voice-quality problems that simulate real-life
IPTT situations
Problem Report
Users calling between clusters sometimes complain about poor voice quality.
Restrictions
None; all tools and service aids are available and can be used.
Copyright © 2004, Cisco Systems, Inc.
Lab Guide
21
Lab Exercise 5-2: Trouble Ticket (Lab Equipment)
Voice-Quality Issues
Case Number: 020
Objectives
After completing this exercise, you will be able to:
Use proven problem isolation techniques to list the symptoms of voice-quality problems
Apply diagnostic tools to solve classroom voice-quality problems that simulate real-life
IPTT situations
Problem Report
Users calling the branch office are complaining about poor voice quality.
Restrictions
None; all tools and service aids can be used.
22
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Lab Exercise 5-3: Trouble Ticket (Lab Equipment)
Voice-Quality Issues
Case Number: 021
Objectives
After completing this exercise, you will be able to:
Use proven problem isolation techniques to list the symptoms of voice-quality problems
Apply diagnostic tools to solve classroom voice-quality problems that simulate real-life
IPTT situations
Problem Report
Users calling from the HQ extensions to the branch extensions are experiencing one-way audio
issues. Users calling from the HQ extensions can hear the branch office users, but the branch
office users are unable to hear the HQ employees.
Restrictions
None; all tools and service aids are available and can be used.
Copyright © 2004, Cisco Systems, Inc.
Lab Guide
23
Lab Exercise 5-4: Trouble Ticket (Lab Equipment)
Voice-Quality Issues
Case Number: 022
Objectives
After completing this exercise, you will be able to:
Use proven problem isolation techniques to list the symptoms of voice-quality problems
Apply diagnostic tools to solve classroom voice-quality problems that simulate real-life
IPTT situations
Problem Report
Users calling other clusters sometimes complain about poor voice quality.
Restrictions
None; all tools and service aids can be used.
24
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Lab Exercise 6-1: Trouble Ticket (Lab Equipment)
Voice-Quality Issues
Case Number: 023
Objectives
After completing this exercise, you will be able to:
Use proven problem isolation techniques to list the symptoms of voice-mail problems
Apply diagnostic tools to solve classroom voice-mail problems that simulate real-life IPTT
situations
Problem Report
Users complain that they cannot check voice-mail messages. When subscribers dial the voicemail number, they receive a fast-busy signal.
Restrictions
None; all tools and service aids can be used.
Copyright © 2004, Cisco Systems, Inc.
Lab Guide
25
Lab Exercise 6-2: Trouble Ticket (Lab Equipment)
Voice-Quality Issues
Case Number: 024
Objectives
After completing this exercise, you will be able to:
Use proven problem isolation techniques to list the symptoms of voice-mail problems
Apply diagnostic tools to solve classroom voice-mail problems that simulate real-life IPTT
situations
Problem Report
Some users complain that their phone does not notify them when they have new voice-mail
messages.
Restrictions
None; all tools and service aids can be used.
26
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.
Lab Exercise 6-3: Trouble Ticket (Lab Equipment)
Voice-Quality Issues
Case Number: 025
Objectives
After completing this exercise, you will be able to:
Use proven problem isolation techniques to list the symptoms of voice-mail problems
Apply diagnostic tools to solve classroom voice-mail problems that simulate real-life IPTT
situations
Problem Report
A user at extension y001 (where y = pod number) reports the inability to receive any new voice
mails.
Restrictions
None; all tools and service aids can be used to identify the cause of the failure.
Copyright © 2004, Cisco Systems, Inc.
Lab Guide
27
28
Cisco IP Telephony Troubleshooting (IPTT) v4.0
Copyright © 2004, Cisco Systems, Inc.