InTracer Installation and Administration Guide,
Release 16
Last Updated September 30, 2015
Americas Headquarters
Cisco Systems, Inc.
170 West Tasman Drive
San Jose, CA 95134-1706
USA
http://www.cisco.com
Tel: 408 526-4000
800 553-NETS (6387)
Fax: 408 527-0883
THE SPECIFICATIONS AND INFORMATION REGARDING THE PRODUCTS IN THIS MANUAL ARE SUBJECT TO CHANGE WITHOUT NOTICE. ALL
STATEMENTS, INFORMATION, AND RECOMMENDATIONS IN THIS MANUAL ARE BELIEVED TO BE ACCURATE BUT ARE PRESENTED WITHOUT WARRANTY
OF ANY KIND, EXPRESS OR IMPLIED. USERS MUST TAKE FULL RESPONSIBILITY FOR THEIR APPLICATION OF ANY PRODUCTS.
THE SOFTWARE LICENSE AND LIMITED WARRANTY FOR THE ACCOMPANYING PRODUCT ARE SET FORTH IN THE INFORMATION PACKET THAT SHIPPED
WITH THE PRODUCT AND ARE INCORPORATED HEREIN BY THIS REFERENCE. IF YOU ARE UNABLE TO LOCATE THE SOFTWARE LICENSE OR LIMITED
WARRANTY, CONTACT YOUR CISCO REPRESENTATIVE FOR A COPY.
The Cisco implementation of TCP header compression is an adaptation of a program developed by the University of California, Berkeley (UCB) as part of UCB’s public domain
version of the UNIX operating system. All rights reserved. Copyright © 1981, Regents of the University of California.
NOTWITHSTANDING ANY OTHER WARRANTY HEREIN, ALL DOCUMENT FILES AND SOFTWARE OF THESE SUPPLIERS ARE PROVIDED “AS IS” WITH ALL
FAULTS. CISCO AND THE ABOVE-NAMED SUPPLIERS DISCLAIM ALL WARRANTIES, EXPRESSED OR IMPLIED, INCLUDING, WITHOUT LIMITATION, THOSE
OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT OR ARISING FROM A COURSE OF DEALING, USAGE, OR TRADE
PRACTICE.
IN NO EVENT SHALL CISCO OR ITS SUPPLIERS BE LIABLE FOR ANY INDIRECT, SPECIAL, CONSEQUENTIAL, OR INCIDENTAL DAMAGES, INCLUDING,
WITHOUT LIMITATION, LOST PROFITS OR LOSS OR DAMAGE TO DATA ARISING OUT OF THE USE OR INABILITY TO USE THIS MANUAL, EVEN IF CISCO OR
ITS SUPPLIERS HAVE BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.
Cisco and the Cisco Logo are trademarks of Cisco Systems, Inc. and/or its affiliates in the U.S. and other countries. A listing of Cisco's trademarks can be found at
www.cisco.com/go/trademarks. Third party trademarks mentioned are the property of their respective owners. The use of the word partner does not imply a partnership relationship
between Cisco and any other company.
Any Internet Protocol (IP) addresses and phone numbers used in this document are not intended to be actual addresses and phone numbers. Any examples, command display
output, network topology diagrams, and other figures included in the document are shown for illustrative purposes only. Any use of actual IP addresses or phone numbers in
illustrative content is unintentional and coincidental.
InTracer Installation and Administration Guide, Release 16
© 2015 Cisco Systems, Inc. All rights reserved.
CONTENTS
About this Guide ................................................................................................. v
Conventions Used ....................................................................................................................................vi
Contacting Customer Support ................................................................................................................. vii
InTracer Overview ............................................................................................... 9
Cisco InTracer - An Introduction ............................................................................................................. 10
Cisco InTracer Architecture .................................................................................................................... 11
Nodal Trace ........................................................................................................................................ 11
3GPP Trace........................................................................................................................................ 11
InTracer Architecture Scalability ........................................................................................................ 12
Nodal Trace ................................................................................................................................... 12
3GPP Trace ................................................................................................................................... 12
InTracer Components......................................................................................................................... 12
Component Table .......................................................................................................................... 12
InTracer Client Nodal Trace ........................................................................................................... 13
InTracer Client 3GPP Trace........................................................................................................... 13
R-InTracer / R-TCE ........................................................................................................................ 14
Event Processing ........................................................................................................................... 14
Event Storage ................................................................................................................................ 14
Event Filtering ................................................................................................................................ 15
Packet-loss detection ..................................................................................................................... 15
Health Monitoring ........................................................................................................................... 15
Resource Monitoring ...................................................................................................................... 15
Maintenance Activities ................................................................................................................... 16
Statistical Counters ........................................................................................................................ 16
C-InTracer Features and Functions ............................................................................................... 16
Query Processing .......................................................................................................................... 17
System Configuration ..................................................................................................................... 17
Audit Trail ....................................................................................................................................... 17
Health Monitoring ........................................................................................................................... 17
Resource Monitoring ...................................................................................................................... 17
Maintenance Activities ................................................................................................................... 18
Statistical Counters ........................................................................................................................ 18
InTracer Web User Interface.......................................................................................................... 18
InTracer System Requirements .............................................................................................................. 19
InTracer Client Pre-requisites ............................................................................................................ 19
R-InTracer / R-TCE Pre-requisites ..................................................................................................... 19
C-InTracer Pre-requisites ................................................................................................................... 19
InTracer User Work Station Pre-requisites ........................................................................................ 20
Supported Features ................................................................................................................................ 21
Nodal Trace ........................................................................................................................................ 21
3GPP Trace........................................................................................................................................ 21
InTracer Network Requirements ..................................................................... 23
Network Considerations ......................................................................................................................... 24
Interface between InTracer Client and R-InTracer ............................................................................. 24
InTracer Installation and Administration Guide, Release 16 ▄
iii
▀ Contents
Interface between InTracer Client and R-TCE ................................................................................... 24
Interface between R-InTracer and C-InTracer ................................................................................... 24
Network Default Port Utilization .......................................................................................................... 25
Time Synchronization ......................................................................................................................... 26
InTracer Provisioning Guidelines ................................................................... 27
R-InTracer Provisioning Guidelines ........................................................................................................ 28
R-TCE Provisioning Guidelines .............................................................................................................. 29
C-InTracer Provisioning Guidelines ........................................................................................................ 30
InTracer Component Installation ..................................................................... 31
Preparing for Installation ......................................................................................................................... 32
Unpacking Installation Files .................................................................................................................... 33
Pre Installation Checklist ........................................................................................................................ 34
Installation Methods ................................................................................................................................ 35
GUI based Installation ........................................................................................................................ 35
Console based Installation ................................................................................................................. 35
Installing the InTracer Software using the GUI-Based Installation Wizard ........................................ 35
Installing the InTracer Software using the Console-Based Installation Method ................................. 49
Post Installation Checklist ....................................................................................................................... 56
Uninstalling the InTracer .................................................................................. 59
Preparing for Uninstallation .................................................................................................................... 60
InTracer Application Uninstallation ......................................................................................................... 61
Using the GUI-based Uninstallation Method ...................................................................................... 61
Using the Console-based Uninstallation Method ............................................................................... 62
Post Uninstallation Verification Check list .............................................................................................. 64
Additional Data Cleanup ..................................................................................................................... 64
Upgrading the InTracer .................................................................................... 65
Pre-Upgradation Check .......................................................................................................................... 66
C-InTracer Upgradation .......................................................................................................................... 67
R-InTracer Upgradation .......................................................................................................................... 70
R-TCE Upgradation ................................................................................................................................ 72
Post Upgradation .................................................................................................................................... 74
InTracer Clustering Support for High Availability ......................................... 75
System and Hardware Recommendations for HA Deployment ............................................................. 76
Configuring the External Storage Disk on UCS for HA Deployment .................................................. 76
Setting up VERITAS Cluster Resources ................................................................................................ 80
Basic Flow of Resource Configuration ................................................................................................... 81
Installation Steps for Cluster Setup ........................................................................................................ 83
VCS Commands for Resource Configuration ......................................................................................... 87
Query Server main.cs ............................................................................................................................. 88
TCE Main.cs ........................................................................................................................................... 90
Nodal main.cs ......................................................................................................................................... 92
VCS Resource Configuration File ........................................................................................................... 97
Main.cf File Configuration ................................................................................................................. 100
Threshold Configuration ................................................................................ 105
System Thresholds ............................................................................................................................... 106
Process Thresholds .............................................................................................................................. 107
▄ InTracer Installation and Administration Guide, Release 16
iv
About this Guide
This document pertains to the features and functionality that are related to the Cisco® InTracer Installation and
Administration Guide.
InTracer Installation and Administration Guide, Release 16 ▄
v
About this Guide
▀ Conventions Used
Conventions Used
The following tables describe the conventions used throughout this documentation.
Icon
Notice Type
Description
Information Note
Provides information about important features or instructions.
Caution
Alerts you of potential damage to a program, device, or system.
Warning
Alerts you of potential personal injury or fatality. May also alert you of potential electrical hazards.
▄ InTracer Installation and Administration Guide, Release 16
vi
About this Guide
Contacting Customer Support ▀
Contacting Customer Support
Use the information in this section to contact customer support.
Refer to the support area of http://www.cisco.com for up-to-date product documentation or to submit a service request.
A valid username and password are required to access this site. Please contact your Cisco sales or service representative
for additional information.
InTracer Installation and Administration Guide, Release 16 ▄
vii
Chapter 1
InTracer Overview
This chapter provides an overview of the Cisco InTracer application and describes its architecture.
It includes the following sections:
 Introduction
 InTracer Architecture
 InTracer System Requirements
 Supported Features
InTracer Installation and Administration Guide, Release 16 ▄
9
InTracer Overview
▀ Cisco InTracer - An Introduction
Cisco InTracer - An Introduction
Cisco InTracer is a high-performance troubleshooting and monitoring solution for subscribers. It performs call tracing,
control data acquisition, processing and analysis of both active and historical subscriber sessions.
Cisco InTracer provides a framework for operators to analyze and investigate call flows and call events for subscriber
sessions in real time.
Cisco InTracer consists of two modules:
 The InTracer Client module:
Enables and configures on the Cisco gateway (to start sending subscriber traces (control plane information for
sessions) to InTracer Server module
 The InTracer Server module:
The InTracer Server module runs on an external box sitting close to the gateway to process and store subscriber
traces
Important: External applications are now supported by the Cisco MITG RHEL v5.9 OS on selected Cisco UCS
servers. The Cisco MITG RHEL v5.9 OS is a custom image that contains only those software packages required to
support compatible Cisco MITG external software applications, for example, in this case Cisco Intracer. Users must not
install any other applications on servers running the Cisco MITG RHEL v5.9 OS. For detailed software compatibility
information, refer to the “Cisco MITG RHEL v5.9 OS Application Note.”
▄ InTracer Installation and Administration Guide, Release 16
10
InTracer Overview
Cisco InTracer Architecture ▀
Cisco InTracer Architecture
This section describes InTracer Architecture.
Nodal Trace
The following figure shows a high level view of architecture of InTracer with other components in a deployment
scenario.
Figure 1.
Nodal Trace
For Nodal trace, the communication between the Gateway and the R-InTracer / R-TCE happens over a proprietary
protocol on top of UDP.
3GPP Trace
In 3GPP Trace communication happens through XML files being FTP pushed from the gateway to the R-TCE. R-TCE
receives the FTP pushed files from the gateway and parses them to store their data into the database.
InTracer Installation and Administration Guide, Release 16 ▄
11
InTracer Overview
▀ Cisco InTracer Architecture
Figure 2.
3GPP Trace
InTracer Architecture Scalability
Although the above architecture diagram shows a 1:1 mapping between the Gateway and the R-InTracer / R-TCE
server, the architecture is scalable.
Nodal Trace
In this trace, since a copy of all signaling packets for each subscriber session is sent from the Gateway to the RInTracer, deployments with high subscriber session rates might require more than one R-InTracer servers to be
configured per Gateway. This feature is supported.
3GPP Trace
In this trace, since signaling packets for only activated subscriber sessions are sent from the Gateway to the R-TCE,
multiple Gateways should be able to send XML trace files to a single R-TCE server. This feature is supported.
InTracer Components
This section provides the physical / logical components of InTracer solution.
Component Table
Component
Nodal Trace
3GPP Trace
InTracer client
When configured sends a copy of all
signaling packets for all sessions to the RInTracer over UDP.
When configured, sends a copy of all signaling packets for
activated trace sessions to the R-TCE by pushing the
generated XML trace files via FTP/SFTP.
R-InTracer
(Regional
InTracer)
Receives packets over UDP from configured
InTracer client, processes and stores them for
the configured storage period.
N/A
▄ InTracer Installation and Administration Guide, Release 16
12
InTracer Overview
Cisco InTracer Architecture ▀
Component
Nodal Trace
3GPP Trace
R-TCE(Regional
Trace Collection
Entity)
N/A
Receives XML trace files via FTP/SFTP from configured
InTracer client, parses the files and stores the information
for the configured storage period.
C-InTracer
(Central InTracer)
This component provides configuration and
management in the InTracer system. It is
capable of fetching data from all configured
R-InTracers and displaying coalesced
information to the end-user.
This component provides configuration and management in
the InTracer system. It is capable of fetching data from all
configured R-TCEs and displaying coalesced information to
the end-user. In addition to this, it also provides an interface
for activating/de-activating subscriber traces.
InTracer Web User
Interface
A unified web-interface of the product
providing configuration, management and
data-querying options.
A unified web-interface of the product providing
configuration, management and data-querying options,
along with Activation / De-activation of subscriber traces.
InTracer Client Nodal Trace
Generic Features
The following generic features are supported by the InTracer Client for Nodal Trace:
 To send a copy of all signaling packets for each subscriber session to configured R-InTracer
 To check the health of configured R-InTracer (up and running)
Following are the configuration items for the InTracer Client for Nodal Trace:
 Enabling InTracer
 R-InTracer configuration
 UDP heartbeat configuration parameters
Important:
Refer to the appropriate System and Product Administration Guides for detailed information.
Specific Features
Following are the additional configuration items to support the above features:
 License to export SA-keys (on the gateway)
 Enabling SA-keys in InTracer configuration
Important:
Refer to the appropriate System and Product Administration Guides for detailed information.
InTracer Client 3GPP Trace
The following generic features are supported by the InTracer Client for 3GPP Trace :
 To send a copy of all transmitted signaling packets for activated trace sessions to the configured R-TCE (3GPP
TCE - Trace Collection Entity)
 To send trace activation/deactivation failure notifications to the configured R-TCE
Following are the configuration items for the InTracer Client for 3GPP Trace:
InTracer Installation and Administration Guide, Release 16 ▄
13
InTracer Overview
▀ Cisco InTracer Architecture
 Configuration of of global trace parameters
 Configuration of TCE profiles (R-TCE configuration)
 Management-based Activation / Deactivation of Trace sessions
Important:
Refer to the appropriate System and Product Administration Guides for detailed information
R-InTracer / R-TCE
The basic functionality of the R-InTracer / R-TCE is to receive signaling packets from the gateways and process them.
It stores all the packets along with additional correlation information either received along with the packet or derived
after processing the packet.
Following are the features and functions supported by the R-InTracer / R-TCE:
Feature
Configurable
Event Processing
Yes
Event Storage
Yes
Packet-loss detection
Default
Nodal Trace
3GPP Trace
Yes
Yes
30 Days
Yes
Yes
Yes
Enabled
Yes
Yes
Health Monitoring
Yes
Enabled
Yes
Yes
Resource Monitoring
Yes
Enabled (Default threshold values)
Yes
Yes
Maintenance Activities
Yes
Enabled (Daily midnight)
Yes
Yes
Statistical Counters
No
Always
Yes
Yes
Event Processing
The basic task of the R-InTracer / R-TCE is to receive signaling packets from the gateways and process them. It then
persistently stores all these packets along with additional correlation information either received along with the packet
or derived after processing the packet.
Event Storage
The R-InTracer / R-TCE stores processed packets along with other correlation information to persistent storage. The
storage period is a configurable value between 2 to 365 days, default value is 30 days.
Important:
If event information exceeds the storage period, the event information is automatically deleted by the
system even if the system has sufficient additional storage available.
When the R-InTracer / R-TCE receives packets from the gateway (either through UDP sockets in case of Nodal trace or
through XML files in case of 3GPP trace), it requires some time before it persistently stores these packets and makes
them available for query. The possible reasons for this delay are:
 It needs to take care of out-of-order packet delivery over UDP in case of Nodal trace
 It needs to detect packet-losses and take appropriate actions in case of Nodal trace
▄ InTracer Installation and Administration Guide, Release 16
14
InTracer Overview
Cisco InTracer Architecture ▀
 It needs to perform event correlation
For better search, each query specified by the operator should be bounded by time intervals. The shorter the interval
specified, the lesser the amount of data to be searched which results in faster query response.
Important: InTracer is a best-effort solution and there can be possible losses of packet data even after
successfully processed by the R-InTracer / R-TCE. Such scenarios occur since some packet information is cached and
not persistently stored immediately. All efforts are made internally to minimize such scenarios.
Event Filtering
Event filtering allows operators to configure which packets to be processed/ignored by the R-InTracer / R-TCE.
Filtering can be done based on the event type and/or protocol type.The R-InTracer / R-TCE maintains statistical
counters about the filtered packets, which can later be used to verify this behaviour.
Important:
Events that are filtered out are not stored on persistent storage, hence cannot be queried later.
Packet-loss detection
Signaling packets may be received by the R-InTracer / R-TCE over UDP. Since UDP does not guarantee reliable packet
delivery, such network losses are detected by the R-InTracer / R-TCE at application level and reported by generating
alarms based on configuration.
Health Monitoring
Server heartbeat process is part of Health Monitoring of InTracer components. This is not only responsible for
starting/stopping of all the processes, but also has a watchdog function that ensures that:
 All the processes are up
 Every process is functioning normally
In an event where any of the process is not up or not functioning normally as expected, it can stop and restart the
process to restore the system to processing state.
It also has a function to guard the system and notify the operator using logs and alarm against persistent errors which
need operator intervention.
Resource Monitoring
Resource Monitoring has below two parts to monitor:
 CPU and Memory utilization of the system and other individual processes
 Disk utilization for Export, Data, Log, Crash Directories and for Disk itself
CPU and Memory utilization of the system and other individual processes: Helps monitor complete resource
utilization of InTracer.
Below table describes system resources of InTracer which can be monitored.
Process
Description
InTracer Installation and Administration Guide, Release 16 ▄
15
InTracer Overview
▀ Cisco InTracer Architecture
Process
Description
CPU Usage (%)
CPU utilization in percentage for specific InTracer process.
Memory Usage (%)
System memory utilization in percentage for specific InTracer process.
Disk Utilization Monitoring: Disk utilization monitoring helps monitor Disk Utilization of various components of
InTracer.
Disk Utilization monitoring provides for selected device name can be performed on the below components of InTracer:
 Mount Point: The name of the mount point of storage device.
 Filesystem ID: The identifier for Filesystem on storage device.
Maintenance Activities
The R-InTracer / R-TCE performs various routine activities for the system maintenance and for persistent storage.
These activities include the following:
 Call event information and SA Key information is stored for a configurable period configured at install time
 Running DB verification to ensure data is in proper and valid form
 Backup of InTracer system state information, if enabled
 Compression of processing log data, removal of older logs
These activities are performed in background by UNIX cron jobs that are invoked periodically by the system. Not all
these activities are done in a single run. These are shared across various invocations of the jobs at different times of the
day when the processing load on the R-InTracer / R-TCE is assumed to be less. The R-InTracer / R-TCE facilitates
operators to specify configurations for all these activities. These configurations can be performed through the InTracer
clients.
Statistical Counters
The R-InTracer / R-TCE process maintains current statistical information for each of its functions. The InTracer clients
provide an interface to view these statistics. These statistics can be used to:
 Perform analysis of call events
 Measure performance of the AGW and the R-InTracer / R-TCE and health of the link between them
C-InTracer Features and Functions
The following are the features and functions supported by the C-InTracer
Feature
Configurable
Default
Nodal Trace
3GPP Trace
Query Processing
Yes
-
Yes
Yes
User Management
Yes
Enabled
Yes
Yes
System Configuration
Yes
Admin only
Yes
Yes
Gateway Configuration
Yes
Admin only
No
Yes
▄ InTracer Installation and Administration Guide, Release 16
16
InTracer Overview
Cisco InTracer Architecture ▀
Feature
Configurable
Default
Nodal Trace
3GPP Trace
Query Management
Yes
All users
Yes
Yes
Audit Trail
Yes
30 days
Yes
Yes
Health Monitoring
Yes
Enabled
Yes
Yes
Resource Monitoring
Yes
Enabled (default threshold values)
Yes
Yes
Maintenance Activities
Yes
Enabled (daily midnight)
Yes
Yes
Statistical Counters
No
Always
Yes
Yes
Query Processing
For C-InTracer(Nodal installation), Session related queries can be based on IMSI, NAI, Call-Id, Disconnect reasons and
so on, limited by a time window.
For R-TCE (3gpp installation), Session related queries can be based on IMSI, IMEI/IMEISV, Trace Session reference,
and so on, limited by a time window.
System Configuration
Use the system setup to manage and configure In Tracer components such as the R-InTracer / R-TCE Server, the CInTracer and the Event Source.
Audit Trail
This keeps track of various system activities such as User session login/logout and configuration change activities. This
audit trail is helpful for troubleshooting and monitoring the system activities.
Health Monitoring
Server Hearth beat process is part of Health Monitoring of InTracer components. This is not only responsible for
starting/stopping of all the processes, but also has a watchdog function that ensures that:
 All the processes are up
 Every process is functioning normally
In an event where any of the process is not up or not functioning normally as expected, it can stop and restart the
process to restore the system to processing state.
It also has a function to guard the system and notify the operator using logs and alarm against persistent errors which
need operator intervention.
Resource Monitoring
Resource Monitoring has below two parts to monitor:
 CPU and Memory utilization of the system and other individual processes
 Disk utilization for Export, Data, Log, Crash Directories and for Disk itself
InTracer Installation and Administration Guide, Release 16 ▄
17
InTracer Overview
▀ Cisco InTracer Architecture
CPU and Memory utilization of the system and other individual processes: Helps monitor complete resource
utilization of InTracer. Below table describes system resources of InTracer which can be monitored.
Process
Description
CPU Usage (%)
CPU utilization in percentage for specific InTracer process.
Memory Usage (%)
System memory utilization in percentage for specific InTracer process.
Disk Utilization Monitoring Helps monitor Disk Utilization of various components of InTracer. Disk Utilization
monitoring Provides for selected device name can be performed on the below components of InTracer:
 Mount Point: The name of the mount point of storage device.
 Filesystem ID: The identifier for Filesystem on storage device.
Maintenance Activities
The C-InTracer performs various routine activities for the system maintenance and for persistent storage. These
activities include the following:
 Call event information and SA Key information is stored for a max period of 30 days and information exceeding
this storage period is automatically deleted by the server
 Running DB verification to ensure data is in proper and valid form
 Backup of InTracer system state information, if enabled
 Compression of processing log data, removal of older logs
These activities are performed in the background by UNIX cron jobs that are invoked periodically by the system. Not all
of these activities are done in a single run. These are shared across various invocations of the jobs at different times of
the day when the processing load on the R-InTracer / R-TCE is assumed to be less. The R-InTracer / R-TCE allows
operators to specify configurations for all of these activities. These configurations can be performed through the
InTracer clients.
Statistical Counters
The C-InTracer process maintains current statistical information for each of its functions. Cumulative statistics are
collected on a daily basis and stored in the server statistical database. The InTracer clients provide an interface to view
these statistics. These statistics can be used to:
 Perform analysis of call events
 Measure performance of the AGW and the R-InTracer / R-TCE and health of the link between them
InTracer Web User Interface
A unified Web-based user interface for performing configuration and querying in InTracer.
▄ InTracer Installation and Administration Guide, Release 16
18
InTracer Overview
InTracer System Requirements ▀
InTracer System Requirements
This section provides InTracer System Requirements.
InTracer Client Pre-requisites
The InTracer Nodal Trace solution is for Cisco ASR5xxx based gateway services. The InTracer Client in this case
requires an ASR5xxx based gateway.
R-InTracer / R-TCE Pre-requisites
Following are the hardware pre-requisites for the R-InTracer / R-TCE:
Hardware Parameters
Server option 1
Server option 2
Server
UCS 210 non NEBS compliant
UCS 240 M3 NEBS compliant
Memory
32 GB
32 GB
Internal Disk
2 * 300 GB SCSI
2 * 300 GB SCSI
HBA Card
2 * 4 Gbps FC ports
2 * 4 Gbps FC ports
Brocade Switch
Optional with UCS
Optional with UCS
External Disk
Sun StorageTek 2540
Sun StorageTek 2540
Network Interfaces
2 physical interfaces >= 100 Mbps
2 physical interfaces >= 100 Mbps
CPU
Important:
The HBA card, Brocade switch and External disk are required for 3GPP Trace only.
Following are the software pre-requisites for R-InTracer / R-TCE:
Software Parameters
Description
Operating System
Custom RHEL 64-bit image distributed by Cisco (with all required packages and patches)
File System
XFS
C-InTracer Pre-requisites
Following are the hardware pre-requisites for the C-InTracer:
Hardware Parameters
Server option 1
Server option 2
InTracer Installation and Administration Guide, Release 16 ▄
19
InTracer Overview
▀ InTracer System Requirements
Hardware Parameters
Server option 1
Server option 2
Server
UCS 210 non NEBS compliant
UCS 240 M3 NEBS compliant
Memory
32 GB
32 GB
Internal Disk
2 * 300 GB SCSI
2 * 300 GB SCSI
HBA Card
2 * 4 Gbps FC ports
2 * 4 Gbps FC ports
Brocade switch
Optional with UCS
Optional with UCS
External Disk
Sun StorageTek 2540
Sun StorageTek 2540
Network Interfaces
1 physical interfaces >= 100 Mbps
1 physical interfaces >= 100 Mbps
CPU
Important:
The HBA card, Brocade switch and External disk are required for 3GPP Trace only.
InTracer User Work Station Pre-requisites
Important:
There are no specific hardware pre-requisites for the InTracer User Work-station
Following are the software pre-requisites for User work station:
Software
Parameters
OS Option1
OS Option2
OS
Windows 7, Windows XP
Mac OS
Browsers
Firefox 3.6.28, Firefox 12, Internet Explorer 7 and 8.
Firefox 3.6
or above
Important: 'Browser not supported. See all supported browsers' message will
be displayed on the InTracer login/splash screen, even when supported browsers are
used.
▄ InTracer Installation and Administration Guide, Release 16
20
InTracer Overview
Supported Features ▀
Supported Features
The InTracer solution currently supports two kinds of tracing functionalities.
Nodal Trace
A tracing solution for Cisco ASR5xxx based gateway services. In this tracing solution, once tracing is activated on the
gateway, it sends a copy of all signaling packets (transmitted and received) for each subscriber session as part of normal
processing to an external server.
 Protocol packets are sent from the lowest layer where subscriber or session information is available.
 Additional call correlation information is also sent along with the protocol packets.
 Correlation of packets is limited to a single gateway.
 Additional Cisco ASR5xxx specific system events like Card / CPU / Task crash etc are sent for hardware /
software failures to associate call failures (if any) to these events.
3GPP Trace
A 3GPP standard based tracing solution. In this tracing solution, the gateway sends a copy of all signaling packets
(transmitted and received) for only the trace sessions that are explicitly activated (either through management-based
activation or signaling-based activation).
 All packets are GTPv2 packets.
 Additional correlation information like the Trace Session Reference is sent along with the protocol packets.
 Correlation of packets using Trace Session Reference is across all gateways on which the same trace is activated.
 Support available for PGW, SGW, GGSN and MME.
InTracer Installation and Administration Guide, Release 16 ▄
21
Chapter 2
InTracer Network Requirements
This chapter identifies the network requirements of various InTracer components. It includes the following sub-sections:
 Network Considerations
InTracer Installation and Administration Guide, Release 16 ▄
23
InTracer Network Requirements
▀ Network Considerations
Network Considerations
This section describes the network considerations.
Interface between InTracer Client and R-InTracer
For Nodal trace, the InTracer traffic does not use the management interface of the gateway, but requires a data interface.
This means that there has to be connectivity from one of the ASR5xxx Line Cards to the R-InTracer.
Following interface parameter values should be met to ensure proper operation of InTracer:
Interface Parameters
Values for Nodal Trace
Throughput
Minimum of 100 Mbps
NAT
Not allowed
Firewall
Not tested
Interface between InTracer Client and R-TCE
For 3GPP trace, the XML files are pushed from the 7600 / ASR5K gateway services to the R-TCE.
Following interface parameter values should be met to ensure proper operation of InTracer:
Interface Parameters
Values for 3GPP trace
Throughput
Minimum of 100 Mbps
NAT
May be used
Firewall
May be used
Interface between R-InTracer and C-InTracer
For performance reasons, it is recommended to use a separate physical interface on the R-InTracer to communicate with
the InTracer Client and the C-InTracer.
Following interface parameter values should be met to ensure proper operation of InTracer:
Interface Parameters
Values
Throughput
Minimum of 100 Mbps
NAT
May be used
Firewall
Not tested
▄ InTracer Installation and Administration Guide, Release 16
24
InTracer Network Requirements
Network Considerations ▀
Network Default Port Utilization
InTracer components use various TCP/UDP ports for communicating with each other. If firewalls are used on these
network interfaces, appropriate ports need to be opened. Following tables provide the default ports used by the
corresponding InTracer component.
InTracer Client
Port number
Port type
Description
UDP
For Nodal Trace only
On the interface with the R-InTracer
Opened by the InTracer Heart-beat Controller task to receive responses from configured R-InTracer(s)
R-InTracer
Port number
Port type
Description
45000
TCP
On the interface with the C-InTracer
For InTracer configuration and statistics management traffic
45000
UDP
On the interface with the C-InTracer
For internal use
45001
TCP
On the interface with the C-InTracer
For InTracer user query traffic
45001-45004
UDP
For Nodal Trace only
On the interface with InTracer Client
For receiving signaling packets from the Gateway
Port number
Port type
Description
45000
TCP
On the management interface
For InTracer configuration and statistics management traffic
45000
UDP
On the management interface
For internal use
45001
TCP
On the management interface
For InTracer user query traffic
45002
TCP
On the management interface
For InTracer user query background scheduling using RPC
For receiving signaling packets from the Gateway
8080
TCP
On the management interface
Apache HTTP server listen port
8443
TCP
Apache HTTP SSL Port
5432
TCP
On the management interface
PostgreSQL Database server listen port
C-InTracer
InTracer Installation and Administration Guide, Release 16 ▄
25
InTracer Network Requirements
▀ Network Considerations
Caution:
Same ports should be used during configuration which you have selected during InTracer installation.
Time Synchronization
To ensure proper operation and near real time functioning of InTracer search queries, all Intracer components including
the Client, R-InTracer(s) and the C-InTracer along with ASR5K should be time synchronized. It is recommended to use
NTP or similar supported protocol to perform this time synchronization.
Caution: If the InTracer components are not time synchronized it may result packet loss. In statistics it is
displayed as "Old dropped". This may impact the loss of call data and important events. It can also impact the in tracer
functionality.
▄ InTracer Installation and Administration Guide, Release 16
26
Chapter 3
InTracer Provisioning Guidelines
This chapter provides recommended hardware and its environment readiness for an InTracer component installation.
This chapter includes the following sub-sections:
 R-InTracer Provisioning Guidelines
 C-InTracer Provisioning Guidelines
Important:
The InTracer Client hardware and environment details are beyond the scope of this document.
InTracer Installation and Administration Guide, Release 16 ▄
27
InTracer Provisioning Guidelines
▀ R-InTracer Provisioning Guidelines
R-InTracer Provisioning Guidelines
This section describes the R-InTracer provisioning guidelines. Please go through the R-InTracer hardware and software
pre-requisites sections.
Once the hardware is chosen, please go through the MITG RHEL OS v5.9 installation guide. This helps to understand
the custom RHEL OS installation on the specified hardware. This guide describes the minimum hard-disk partitioning
requirements for the custom RHEL OS, which is done on the local disk.
R-InTracer Storage
The R-InTracer installation does not require a single huge logical volume. It can be installed on internal disk and the
application data can be stored on multiple sun storage tek or single huge emc logical volume with the XFS file system
▄ InTracer Installation and Administration Guide, Release 16
28
InTracer Provisioning Guidelines
R-TCE Provisioning Guidelines ▀
R-TCE Provisioning Guidelines
This section describes the R-TCE provisioning guidelines. Please go through the R-TCE hardware and software prerequisites sections.
Once the hardware is chosen, please go through the MITG RHEL OS v5.9 installation guide. This helps to understand
the custom RHEL OS installation on the specified hardware. This guide describes the minimum hard-disk partitioning
requirements for the custom RHEL OS, which is done on the local disk.
R-TCE Storage
The R-TCE requires an external storage disk array with appropriate storage capacity based on the InTracer sizing
requirements. The R-TCE requires a single huge logical volume with the XFS file system for its installation.
Important:
The R-TCE software installation and the InTracer data, both reside on this single logical volume.
InTracer Installation and Administration Guide, Release 16 ▄
29
InTracer Provisioning Guidelines
▀ C-InTracer Provisioning Guidelines
C-InTracer Provisioning Guidelines
This section describes the C-InTracer provisioning guidelines. Please go through the C-InTracer hardware and software
pre-requisites sections.
Once the hardware is chosen, please go through the MITG RHEL OS v5.9 installation guide. This helps to understand
the custom RHEL OS installation on the specified hardware. This guide describes the minimum hard-disk partitioning
requirements for the custom RHEL OS, which is done on the local disk.
▄ InTracer Installation and Administration Guide, Release 16
30
Chapter 4
InTracer Component Installation
This chapter provides information and procedure to install InTracer components.
This chapter includes following sections:
 Preparing for Installation
 Unpacking Installation Files
 Pre Installation Checklist
 Installation Methods
 Post Installation Checklist
InTracer Installation and Administration Guide, Release 16 ▄
31
InTracer Component Installation
▀ Preparing for Installation
Preparing for Installation
This section describes the steps that need to be performed prior to installation of an InTracer component.
InTracer dedicated user: It is recommended to have the InTracer installation and execution done by a
dedicated user other than the 'root'. The user may be named 'staradm'. Ensure that the home directory of this
user is set correctly like '/home/staradm' or '/export/home/staradm'. Also note that the user name should not
exceed 8 characters.
Important: InTracer must be installed by a non root user and it does not require 'root'
privileges for its functioning.
▄ InTracer Installation and Administration Guide, Release 16
32
InTracer Component Installation
Unpacking Installation Files ▀
Unpacking Installation Files
InTracer Application installation files are distributed as a single compressed zip file, ' intracer.X.Y.Z.zip', where
X,Y and Z together corresponds to the major and minor versions of InTracer.
Once the installation file is copied to the server machine in which the InTracer component is to be installed, use the
following procedure:
Step 1
Create a user folder for InTracer application.
mkdir <InTracer_APPS_directory>
Step 2
Copy or move the distributed InTracer application installation file to this directory.
cp intracer.X.Y.Z.zip to <InTracer_apps_directory>
OR
mv intracer.X.Y.Z.zip to <InTracer_apps_directory>
Step 3
Change directory to <InTracer_apps_directory> and unzip the installation file
unzip intracer.X.Y.Z.zip
De-compressing the above file results in the following:
 setup.bin - An executable binary file containing all the required application installation files
 README - A text file with basic installation, operation and troubleshooting instructions
InTracer Installation and Administration Guide, Release 16 ▄
33
InTracer Component Installation
▀ Pre Installation Checklist
Pre Installation Checklist
The user installing an InTracer component should verify the check-list items mentioned below prior to the installation:
 Ensure that the installation procedure is performed by the InTracer user created on this machine. Go through the
InTracer dedicated user section above for more details.
 It is strongly recommended to install only one InTracer component on a given server machine.
 The server machine on which the installation is to be done should have a defined 'hostname'.
 The network ports to be configured during the InTracer component installation should not be conflicting with
any other application on the given server machine. Refer to the Network Default Port Utilization section for
more details.
 Ensure sufficient disk space is available on the partition where the InTracer component is being installed. The
installation procedure might give an error if appropriate disk space is not available.
▄ InTracer Installation and Administration Guide, Release 16
34
InTracer Component Installation
Installation Methods ▀
Installation Methods
InTracer installation provides the following installation methods. Any one of them can be used to perform the InTracer
component installation.
GUI based Installation
This is the most often used installation method. Following are requirements that need to be met to using this method:
 It requires a logon account to the InTracer component installation machine with display terminal (monitor)
attached and some X-Windows client installed on the server
 It requires the network connectivity to this machine via Telnet or SSH using some X-Windows client on the
remote work-station.
Console based Installation
This method is available for users who do not have an X-Windows client available for remote connectivity to the
InTracer component installation machine via Telnet or SSH.
Installing the InTracer Software using the GUI-Based Installation Wizard
To install an InTracer component using the GUI based installation, follow the steps below:
Step 1
Change directory to <InTracer_apps_directory> and run the setup.bin file
cd <InTracer_apps_directory>./setup.bin
Caution: If X Server is not running or Display variable is not set properly system automatically starts
console based installation.
A GUI screen as shown in sample given below appears and starts the InTracer installation.
InTracer Installation and Administration Guide, Release 16 ▄
35
InTracer Component Installation
▀ Installation Methods
Step 2
Click Next to proceed.
Step 3
Choose the component to install from C-InTracer, R-InTracer and RTCE.
▄ InTracer Installation and Administration Guide, Release 16
36
InTracer Component Installation
Installation Methods ▀
Step 4
Choose using check box Nodal Trace or LTE Trace or both according to your requirement.
InTracer Installation and Administration Guide, Release 16 ▄
37
InTracer Component Installation
▀ Installation Methods
Step 5
Choose the InTracer installation mode from Standalone and Cluster failover.
▄ InTracer Installation and Administration Guide, Release 16
38
InTracer Component Installation
Installation Methods ▀
Step 6
Select Upgrade installation or New installation as per the requirement. Enter the directory path if the installation is new.
InTracer Installation and Administration Guide, Release 16 ▄
39
InTracer Component Installation
▀ Installation Methods
Step 7
Follow the on-screen prompts to progress through the various installation dialogs and configure the parameters as
required.
C-InTracer Installation and Parameter description
Parameter
Description
▄ InTracer Installation and Administration Guide, Release 16
40
Default Value
InTracer Component Installation
Installation Methods ▀
C-InTracer Component Configuration
IP Address
Select / Enter the IP Address
Not Selected
Mgmt Port
Enter the port number for management interface on InTracer Server
Component. This value must be between 1024 and 65530.
The management port should be unique and should not conflict with that of
any other InTracer component.
45000
Apache Server Configuration
SSL Port
Enter the port number for Apache server.This value must be between 1025
and 65530.
The port should be unique and should not conflict with that of any other
InTracer component port.
8443
Important: To use port number between 1025 through
65530 you need “root” privilege.
Port
Enter the port number for Apache server. This value must be between 1025
and 65530.
The port should be unique and should not conflict with that of any other
InTracer component port.
8080
InTracer Installation and Administration Guide, Release 16 ▄
41
InTracer Component Installation
▀ Installation Methods
RPC Server Configuration
Port
Enter the port number for RPC server. This value must be between 1024 and
65530.
The port should be unique and should not conflict with that of any other
InTracer component port.
45002
Postgres Configuration
Username
username is automatically populated where as port number should be
provided.
Port
Enter the port number for Postgres server. This value must be between 1024
and 65530.
The port should be unique and should not conflict with that of any other
InTracer component port.
▄ InTracer Installation and Administration Guide, Release 16
42
5432
InTracer Component Installation
Installation Methods ▀
Single Sign On (SSO) with Prime Central
Single Sign On
(SSO) with
Prime Central
Prime Central is a Cisco product used to provide an integrated user experience
of network analysis.
Alongside some other Cisco products, InTracer provides an option to the enduser to integrate its InTracer installation with Prime Central during
installation.
By integrating, it is possible to cross-launch InTracer through a Prime Central
UI without going through the authentication process, hence it is termed a
single-sign-on mechanism.
After such integration, the integrated InTracer becomes a part of the Prime
Suite of products and can no longer be invoked in a stand-alone fashion. User
functions like Logout, User management is there-after governed by Prime
Central and these options are not available on InTracer UI.
Enable SSO
with Prime
Central
Enables SSO with Prime Central details.
check/uncheck
Prime Central Details
InTracer Installation and Administration Guide, Release 16 ▄
43
InTracer Component Installation
▀ Installation Methods
DB Server host
name
Specify the host name local_host where you are installing the InTracer
Server.
NOTE: A host/server name must be specified for proper installation of
InTracer components
local_host
DB User
Specify the DB User db_user
primedba
DB Password
Specify the DB Password db_password
Admin123~
DB Port
Enter the port number for management interface on InTracer Server
Component. This value must be between 1025 and 65530.The management
port should be unique and should not conflict with that of any other InTracer
component.
1521
DB SID
Specify the DB Password db_sid
primedb
CAS URL
Specify the CAS URLcas_url
www.cisco.com:8443/SSO
Pre installation summary
Pre-Installation
summary
This section displays the pre installation summary.
This screen asks user to review the configured parameters before installation
starts and provides facility to modify the given parameters by clicking the
“Previous” button. The information included in this screen are:
 Installation location

Prime Central Details

Apache configuration SSL Port value

Prime Central Server Host name

Prime Central DB User

Prime Central DB Password

Prime Central DB Port

Prime Central DB SID

Prime Central CAS URL

RPC server port value

Post-gre Server Port
Review the given information carefully. Once you click on Install button the
system starts the installation procedure for selected component with
configured parameters.
Installation summary
This section displays the installation summary.
R-InTracer Installation and Parameter Descriptions
Parameter
Description
▄ InTracer Installation and Administration Guide, Release 16
44
Default Value
InTracer Component Installation
Installation Methods ▀
Parameter
Description
Default Value
Component Configuration
This section asks user to configure the InTracer components installation parameters. Client Parameters (IP address and
Management port) should be specified carefully so that it enabled communication between R-InTracer and C-InTracer.
C-InTracer Component Configuration
IP Address
Select / Enter the IP Address
Not Selected
Mgmt Port
Enter the port number for management interface on InTracer Server Component. This value
must be between 1024 and 65530.
The management port should be unique and should not conflict with that of any other
InTracer component.
45000
R-InTracer Component Configuration
Server Name
Specify the host name local_host where you are installing the InTracer Server.
NOTE: A host/server name must be specified for proper installation of InTracer
components.
local_host
IP Address
Select / Enter the IP Address
Not Selected
InTracer Installation and Administration Guide, Release 16 ▄
45
InTracer Component Installation
▀ Installation Methods
Parameter
Description
Default Value
Mgmt Port
Enter the port number for management interface on InTracer Server Component. This value
must be between 1024 and 65530
The management port should be unique and should not conflict with that of any other
InTracer component.
45000
Data Storage
Period
Enter the data storage period in days. This value must be between 2 and 365
Application Data Directories
Application Data
Directories
This screen appears only when the selected R-InTracer component Installation type is
Server Installation. This screen asks for application data directory1 through 4.
Important: This step appears only if you choose Nodal Trace while
Installing InTracer.
Pre Installation Summary
▄ InTracer Installation and Administration Guide, Release 16
46
InTracer Component Installation
Installation Methods ▀
Parameter
Description
Pre Installation
summary
This section displays the pre installation summary.
Default Value
This screen asks the user to review the configured parameters before installation starts and
provides facility to modify the given parameters by clicking the “Previous” button.
Review the given information carefully. Once you click on Install button, the system starts
the installation procedure for selected component with configured parameters.
Installation Summary
This section displays the installation summary.
R-TCE Installation and Parameter Descriptions
Parameter
Description
Default
Value
C-InTracer Component Configuration
IP Address
Select / Enter the IP Address
Not
Selected
Mgmt Port
Enter the port number for management interface on InTracer Server Component. This value
must be between 1024 and 65530.
The management port should be unique and should not conflict with that of any other InTracer
component.
45000
InTracer Installation and Administration Guide, Release 16 ▄
47
InTracer Component Installation
▀ Installation Methods
Parameter
Description
Default
Value
R-TCE Component Configuration
TCE Name
Specify the host name tce where you are installing the R-TCE.
IP Address
Select / Enter the IP Address
Not
Selected
Mgmt Port
Enter the port number for management interface on R-TCE. This value must be between 1024
and 65530
The management port should be unique and should not conflict with that of any other InTracer
component.
45000
Data Storage
Period
Enter the data storage period in days. This value must be between 2 and 365
Pre Installation Summary
Pre Installation
summary
This section displays the pre installation summary.
▄ InTracer Installation and Administration Guide, Release 16
48
InTracer Component Installation
Installation Methods ▀
Parameter
Description
Default
Value
This screen asks the user to review the configured parameters before installation starts and
provides facility to modify the given parameters by clicking the “Previous” button.
Review the given information carefully. Once you click on Install button, the system starts the
installation procedure for selected component with configured parameters.
Installation Summary
This section displays the installation summary.
Step 8
Click the Install button in Pre installation Summary screen to start the installation. Once you click on the Install button
the installation starts.
After successful completion of installation the system displays the Install Complete screen to indicate the successful
completion of installation with configured parameters.
Step 9
Click the Done button to finish the installation and proceed to the installation of any of the remaining component on
another machine as required.
Step 10
Once both the components installed on separate hosts, proceed to the section before starting the InTracer component(s).
Installing the InTracer Software using the Console-Based Installation Method
To install an InTracer component using the console based installation, follow the steps below:
Step 1
Change directory to <InTracer_apps_directory> and run the setup.bin file
cd <InTracer_apps_directory>./setup.bin -i console
Step 2
Choose the component to install from the choices available onscreen.
InTracer Installation and Administration Guide, Release 16 ▄
49
InTracer Component Installation
▀ Installation Methods
Step 3
Choose the type of trace.
Step 4
Select the installation mode.
▄ InTracer Installation and Administration Guide, Release 16
50
InTracer Component Installation
Installation Methods ▀
Step 5
C-InTracer Component Configuration: Enter the IP Address, Management port, Apache Port value, SSL port value and
RPC Server Management Port value.
Step 6
Configure Postgre by entering the Server name and Port.
InTracer Installation and Administration Guide, Release 16 ▄
51
InTracer Component Installation
▀ Installation Methods
Step 7
Configuration for SSO with Prime Central: Enter the Prime Central DB Server host name, User, Password, Port, SID
and CAS URL.
Step 8
R-InTracer Component Configuration: Enter the C-InTracer IP Address, Management port, R-InTracer name, IP
Address, Management Port value and Data Storage Period.
▄ InTracer Installation and Administration Guide, Release 16
52
InTracer Component Installation
Installation Methods ▀
Step 9
Enter the Application Data Directories path
Step 10
R-TCE Component Configuration: Enter the C-InTracer IP Address, Management port, R-TCE name, IP Address,
Management Port value and Data Storage Period.
InTracer Installation and Administration Guide, Release 16 ▄
53
InTracer Component Installation
▀ Installation Methods
Step 11
Follow the on-screen prompts to proceed through the installation script and configure the various parameters as
required.
Once you have completed the installation configuration and all processes have started, you receive a message indicating
that the required InTracer component was successfully installed.
=====================================================================================
=Introduction Welcome to InstallAnywhere wizard, It will guide you through the
installationof InTracer 12.2.171. InTracer Cisco Systems, Inc http://www.cisco.com
Copyright (C)2011 Cisco Systems, Inc. All Rights Reserved. The copyrights to certain
works contained in this software are owned by other third parties and used and
distributed under license. Certain components of this software are licensed under the GNU
General Public License (GPL) version 2.0 or the GNU Lesser General Public License (LGPL)
Version 2.1. A copy of each such license is available at
http://www.opensource.org/licenses/gpl-2.0.php and
http://www.opensource.org/licenses/lgpl-2.1.php PRESS <ENTER> TO CONTINUE:
Step 12
Press ENTER to continue the installation.
Step 13
Follow the on-screen prompts to proceed through the installation script and configure the various parameters as
required.
Step 14
The following Pre-Installation summary is displayed:
===============================================================================PreInstallation Summary------------------------ Please Review the Following Before
Continuing Product Name :InTracer Install Location :/home/kbollaba/InTracer Product
Components : C-InTracer Postgres Apache RPC Server Disk Space Information : Required :
760.61 MB Available : 138.25 GB Installation Mode : Standalone InTracer C-InTracer
Component Configuration IP Address of the C-InTracer : 10.105.87.195 Management Port
▄ InTracer Installation and Administration Guide, Release 16
54
InTracer Component Installation
Installation Methods ▀
value [1024-65530] : 45000 Postgres Configuration Username : kbollaba Port value [102465530] : 5432 RPC Server Configuration Port value : 45002 SSO with Prime Central :
Details Apache SSL Port : 8443 Prime Central DB Server Hostname : Prime Central DB User :
primedba Prime Central DB Port : 1521 Prime Central DB SID : primedb CAS URL : Apache
Configuration Port value : 8080 N. Next panel P. Previous panel E. Exit Installation
Enter your choice : (DEFAULT: N): N
Step 15
Once installation procedure is completed, following Post-Installation summary is displayed:
===============================================================================PostInstallation Summary-------------------------InTracer has been successfully installed to:
/home/kbollaba/InTracer Installation logfile located at:
/home/kbollaba/InTracer/InTracer_InstallLog.log Manage individual InTracer component as:
C-InTracer ./ipmssys qs start ./ipmssys qs stop Status of InTracer system can be checked
with: ./ipmssys status For Usage of ipmssys: ./ipmssys --help
Step 16
Verify that all InTracer processes were successfully started by looking at the on-screen messages in the console
window.
Important: Both the installation methods are completely same and all the operations possible through
one method are available in the other method.
InTracer Installation and Administration Guide, Release 16 ▄
55
InTracer Component Installation
▀ Post Installation Checklist
Post Installation Checklist
The user installing an InTracer component should verify the check-list items mentioned below once the installation
procedure is done.
 Check whether the installation successfully completed by looking at the post installation summary.
 To ensure proper functioning of R-InTracer, it is required to set the system UDP socket receive-buffer size to a
high value. For example, 512 MB, to do this, run either of the following commands as 'root':
 utils/ipmsserverconf
 To ensure proper functioning of R-InTracer, it is required to increase the open file limit for R-Intracer user. To
do so, add the following lines as a ‘root’ user in /etc/security/limits.conf file:
<R-Intracer username> soft nofile 65535
<R-Intracer username> hard nofile 65535
 To ensure InTracer component starting at server machine startup, appropriate system '/etc' directory changes are
automatically done by the Installation process. Check whether these changes are in place.
 To ensure proper maintenance of InTracer, appropriate 'cron' entries are added for the InTracer user by the
Installation process. Check whether these changes are in place.
 To ensure proper functioning of user queries, check whether all the InTracer components are time synchronized.
Go through the Time Synchronization section for more details.
 After C-Intracer Installation is done, the settings.py file under <Install_dir>/webgateway/query should contain
proper value of TIMEZONE. The timezone value in this file should match with the timezone present on server
box. Just make a note that this change in file always need a server restart.
 Once all InTracer components including C-InTracer and R-InTracer(s) are installed on various server machines,
check whether these can PING each other using the configured interface addressing.
 From the gateway, check whether PING to each of the R-InTracer IP address is successful.
 If the user wants to install their SSL certificates:
1. Copy SSLCertificateFile and SSLCertificateKeyFile files to C-Intracer installation directory. For
example, your_domain_name.crt and your_private.key.
Important: SSLCertificateFile should be your DigiCert certificate file (e.g.
your_domain_name.crt). SSLCertificateKeyFile should be the key file generated when you
created the CSR.
2. Change directory to C-Intracer installation directory.
bash-3.2$ cd <C-Intracer installation directory>
3. Edit httpd-ssl.conf located at 3rdparty/apache/conf/extra/httpd-ssl.conf using following command:
bash-3.2$vi 3rdparty/apache/conf/extra/httpd-ssl.conf
4. Replace SSLCertificateFile configuration parameter from httpd-ssl.conf with newly generated
certificate path. For example,
▄ InTracer Installation and Administration Guide, Release 16
56
InTracer Component Installation
Post Installation Checklist ▀
SSLCertificateFile "/path to C-Intracer installation
directory/your_domain_name.crt"
5. Replace SSLCertificateKeyFile configuration parameter from httpd-ssl.conf with newly generated key
path. For example,
SSLCertificateKeyFile " / path to C-Intracer installation directory
/your_private.key"
6. Save the file.
7. Restart apache server using following command:
bash-3.2$./ipmsapachectl stop
bash-3.2$./ipmsapachectl start
How to start the InTracer
Start the InTracer Query Server Web Client GUI.
https://<intracer_host>:8443/intracer/
Important: To launch InTracer Query Server Web Client GUI, system should have SSL certificates
for Apache Web server installed using SSL port 8443.
Important:
Use user ID ‘superuser’ and Password ‘superuser’ to access InTracer Query Server.
To start the Intracer Query Server Web Client GUI which is using other than default port.
https://<intracer_host>:XXX/intracer/
Important:
Use user ID ‘superuser’ and Password ‘superuser’ to access InTracer Query Server.
InTracer Installation and Administration Guide, Release 16 ▄
57
Chapter 5
Uninstalling the InTracer
This chapter provides information and procedure to Uninstall the InTracer components and management.
 Preparing for Uninstallation
 InTracer Application Uninstallation
 Post Uninstallation Verification Check list
InTracer Installation and Administration Guide, Release 16 ▄
59
Uninstalling the InTracer
▀ Preparing for Uninstallation
Preparing for Uninstallation
This sub-section describes the steps that need to be performed prior to uninstallation of an InTracer component.
Pre Uninstallation Check-List
The user uninstalling an InTracer component should verify the check-list items mentioned below prior to performing the
uninstallation.
 Ensure that the uninstallation is performed by the InTracer user created on this machine. Please go through the
InTracer dedicated user section above for more details.
 Ensure that the existing InTracer component(s) are stopped manually. Please go through the InTracer
management section for more details.
 If the uninstallation is performed as part of a InTracer passive upgrade, you may want to backup important
InTracer application data. Please follow appropriate data backup procedures before uninstalling the existing
InTracer version.
▄ InTracer Installation and Administration Guide, Release 16
60
Uninstalling the InTracer
InTracer Application Uninstallation ▀
InTracer Application Uninstallation
This section describes the procedure to uninstall the InTracer application software by GUI-based and console-based
installation wizard.
Using the GUI-based Uninstallation Method
This section provides instructions for uninstalling the InTracer application using the GUI-method.
Important: The GUI-based uninstall script provides a mechanism to protect all the historical data files
that are currently in use. However, it is still recommended that you manually perform a backup of these files
before proceeding with this procedure.
Step 1
Log into the sever on which the InTracer application is installed. Use the non root username and password.
Step 2
Go to the directory in which InTracer is to be installed by entering the following command
cd <InTracer_base_directory>
<InTracer_base_directory> is the installation folder where InTracer components are located.
Step 3
Execute the uninstall script by entering the following command
./Uninstall_InTracer/Uninstall_InTracer
InTracer uninstallation wizard dialog box appears.
InTracer Installation and Administration Guide, Release 16 ▄
61
Uninstalling the InTracer
▀ InTracer Application Uninstallation
Click Next, to proceed.
Step 4
Select ‘Yes’ to preserve the listed folders on the screen, click on Uninstall button to confirm and proceed.
Step 5
Wait for Uninstallation to complete.
Once Uninstallation procedure is completed, a screen appears to indicate the completion of uninstallation.
Using the Console-based Uninstallation Method
This section provides instructions for uninstalling the InTracer application using the Console-based method.
Important: The console-based uninstall script is designed to protect all the historical data files that are currently
in use. However, it is still recommended that you manually perform a backup of these files before proceeding with this
procedure.
Step 1
Log into the server on which the InTracer application is installed.
Step 2
Go to the directory in which the InTracer is installed by entering the following command.
cd <InTracer_install_directory>
Step 3
Execute the uninstall script by entering the following command.
./Uninstall_InTracer/Uninstall_InTracer -i console
▄ InTracer Installation and Administration Guide, Release 16
62
Uninstalling the InTracer
InTracer Application Uninstallation ▀
The console-based uninstallation starts and displays the following prompt.
===============================================================================In
Tracer (created with InstallAnywhere)-----------------------------------------------------------------------------===============================================================================Un
install InTracer------------------About to uninstall...InTracerThis will remove
features installed by InstallAnywhere. It will not removefiles and folders
created after the installation.PRESS <ENTER> TO CONTINUE:
Step 4
Follow the on-screen prompts to progress through the various uninstallation dialogs and configure the parameters as
required.
Step 5
Once uninstallation procedure is completed, following message appears to indicate the same.
Uninstall CompleteAll items were successfully uninstalled.
InTracer Installation and Administration Guide, Release 16 ▄
63
Uninstalling the InTracer
▀ Post Uninstallation Verification Check list
Post Uninstallation Verification Check list
This section describes the operations that need to be performed post InTracer component uninstallation.
The user should verify the below check-list items once the uninstallation procedure is done.
 Check whether the uninstallation successfully completed by looking at the post uninstallation summary.
 If the 'Preserve folders' option was selected during the uninstallation, all InTracer installation data except the
folders listed in that screen should be removed as part of the uninstallation process.
 If there is external (non-InTracer application) data in terms of files or folders present in the InTracer installation
directory, this data is not removed by the uninstallation process. This data can later be manually removed (if
required) as described in the Additional Data Cleanup section below.
 The InTracer Uninstallation procedure should ensure cleaning up of additional 'cron' entries made as part of
InTracer maintenance activities.
Additional Data Cleanup
The InTracer Uninstallation procedure removes only InTracer application specific data. Any external data in terms of
files or folders created by the InTracer user in the installation directory is not removed. To remove this data manually,
follow the steps below:
cd <InTracer_install_directory> rm -r *
▄ InTracer Installation and Administration Guide, Release 16
64
Chapter 6
Upgrading the InTracer
The information and instructions in this chapter should be used to upgrade an InTracer installation. These instructions
assume that you have already obtained the upgrade file(s) and have stored them on the server running the InTracer
application. InTracer supports upgrade to newer releases so that new functionalities are available to the end-user.
This chapter includes following sections:
 Pre-Upgradation Check
 C-InTracer Upgradation
 R-InTracer Upgradation
 R-TCE Upgradation
 Post Upgradation
InTracer Installation and Administration Guide, Release 16 ▄
65
Upgrading the InTracer
▀ Pre-Upgradation Check
Pre-Upgradation Check
Before upgrading InTracer components, perform the following check:
Step 1
Make sure that C-InTracer with data is installed at the location <C-InTracer-Home>.
Step 2
Make sure that R-InTracer with data is installed at the location <R-InTracer-Home>.
Step 3
Make sure that R-TCE with data is installed at the location <R-TCE-Home>.
▄ InTracer Installation and Administration Guide, Release 16
66
Upgrading the InTracer
C-InTracer Upgradation ▀
C-InTracer Upgradation
This section describes the steps required for manual upgradation of C-InTracer component.
Important: All files required to manually upgrade each component is available in the upgrade folder.
Stop individual component before proceeding with upgradation. To stop the component use the command:
./ipmssys stop
To upgrade C-InTracer, perform the following steps:
Step 1
Back up the configuration by performing the following steps:
Step a
Go to the directory in which the C-InTracer files are located by executing the command
cd <C-InTracer-Home>
Step b
From the upgrade folder, copy Backup.sh and C-InTracer-InstallValues.txt to <CInTracer-Home>
Step c
Backup the C-InTracer data by executing the command
./Backup.sh
Step d
To upgrade C-InTracer component, enter option 1.
Step e
To save the configuration information, enter <valid-location>. Your configuration information
is now available at <valid-location>/C-InTracer-InstallValues.txt
Step 2
Backup the C-InTracer data by executing the following commands:
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:`pwd`/3rdparty/postgres/lib
<C-InTracer-Home>/3rdparty/postgres/bin/pg_start
./3rdparty/postgres/bin/pg_dump -p <POSTGRES_PORT> ipmsdb >
<data_backup_path>/ipmsdb.backup ;
Important:
Step 3
The <data_backup_path> should be outside the folder created during installation.
Stop the C-InTracer component by executing the command
<C-InTracer-Home>/3rdparty/postgres/bin/pg_stop
Step 4
Copy the data under <Install_dir>/data/QS/scheduled_queries/ to a specific directory to be retrieved later.
Step 5
Uninstall the C-InTracer component by executing the command
./Uninstall_InTracer/Uninstall_InTracer
InTracer Installation and Administration Guide, Release 16 ▄
67
Upgrading the InTracer
▀ C-InTracer Upgradation
Step 6
Install the new binary at the same location, that is, < C-InTracer-Home>, using console mode, entering the same
configuration details as previous installation. Copy the data backed up in Step 4 under <C-IntracerHome>/data/QS/scheduled_queries/ directory.
Step 7
Go to the directory in which the C-InTracer files are located by executing the command
cd <C-InTracer-Home>
Step 8
Start the C-InTracer component by executing the command
./ipmssys start
Step 9
Stop the C-InTracer component by executing the command
./ipmssys stop
Step 10
Restore the data by executing the following commands:
<C-InTracer-Home>/3rdparty/postgres/bin/pg_start
./3rdparty/postgres/bin/psql -U <username> -p <POSTGRES_PORT> template1
drop database ipmsdb;
create database ipmsdb;
\q
./3rdparty/postgres/bin/psql -d ipmsdb -f <data_backup_path>/ipmsdb.backup -p
<POSTGRES_PORT>
Step 11
Copy the following files to <C-InTracer-Home> :
 update_postgres1.sql
 update_postgres2.sql
 sync_C.py
 sync_R.py
 sync_T.py
 Backup.sh
These files are available under <C-InTracer-Home>/upgrade folder.
Step 12
Apply the update_postgres1.sql file on the database, using the command
./3rdparty/postgres/bin/psql -U <username> -d ipmsdb -p <POSTGRES_PORT> -f
update_postgres1.sql
Step 13
Take backup by executing the command
./Backup.sh
Step 14
Choose option 4 - Restore.
▄ InTracer Installation and Administration Guide, Release 16
68
Upgrading the InTracer
C-InTracer Upgradation ▀
Step 15
Export relevant paths as mentioned in the set_paths.txt document in upgrade folder.
Step 16
Run python scripts as follows:
python sync_C.py; python sync_R.py; python sync_T.py
Step 17
Apply the update_postgres2.sql file on the database by executing the command
./3rdparty/postgres/bin/psql -U <username> -d ipmsdb -p <POSTGRES_PORT> -f
update_postgres2.sql
Step 18
Start the C-InTracer component by executing the command
./ipmssys start
Step 19
Stop the C-InTracer component by executing the command
./ipmssys stop
InTracer Installation and Administration Guide, Release 16 ▄
69
Upgrading the InTracer
▀ R-InTracer Upgradation
R-InTracer Upgradation
This section describes the steps required for manual upgradation of R-InTracer component.
Important: All files required to manually upgrade each component is available in the upgrade folder.
Stop individual component before proceeding with upgradation. To stop the component use the command:
./ipmssys stop
To upgrade R-InTracer, perform the following steps:
Step 1
Back up the configuration by performing the following steps:
Step a
Go to the directory in which the R-InTracer files are located by executing the command.
cd <R-InTracer-Home>
Step b
From the upgrade folder, copy Backup.sh and R-InTracer-InstallValues.txt to <RInTracer-Home>.
Step c
Backup the R-InTracer data by executing the following commands:
./Backup.sh
Step d
To upgrade R-InTracer component, enter option 2.
Step e
To save the configuration information, enter <valid-location>. Your configuration information
is now available at <valid-location>/R-InTracer-InstallValues.txt.
Step 2
Stop the R-InTracer component by executing the command.
./ipmssys stop
Step 3
Backup the R-InTracer data by executing the following commands:
tar -cvf nodal_data.tar intracer_data
Important: The intracer_data folder will be available in the DATA PATH given during installation.
This is as per default location.
cp nodal_data.tar <data_backup_path>
Important:
Step 4
This <data_backup_path> should be outside the folder created during installation.
Uninstall the R-InTracer component by executing the command.
./Uninstall_InTracer/Uninstall_InTracer
Step 5
Install the new binary at the same location, that is, <R-InTracer-Home>, using console mode, entering the same
configuration details as previous installation.
▄ InTracer Installation and Administration Guide, Release 16
70
Upgrading the InTracer
R-InTracer Upgradation ▀
Step 6
Go to the directory in which the R-InTracer files are located by executing the command.
cd <R-InTracer-Home>
Step 7
Restore the data by executing the following commands:
./ipmssys start
./ipmssys stop
rm -rf intracer_data
cp <data_backup_path>/nodal_data.tar .
tar -xvf nodal_data.tar
rm nodal_data.tar
Step 8
Start the R-InTracer component by executing the command.
./ipmssys start
InTracer Installation and Administration Guide, Release 16 ▄
71
Upgrading the InTracer
▀ R-TCE Upgradation
R-TCE Upgradation
This section describes the steps required for manual upgradation of R-TCE component.
Important: All files required to manually upgrade each component is available in the upgrade folder.
Stop individual component before proceeding with upgradation. To stop the component use the following
command: ./ipmssys stop
To upgrade R-TCE, perform the following steps:
Step 1
Back up the configuration by performing the following steps:
Step a
Go to the directory in which the R-TCE files are located by executing the command.
cd <R-TCE-Home>
Step b
From the upgrade folder, copy Backup.sh and R-TCE-Home-InstallValues.txt to <R-
TCE-Home>
Step c
Backup the R-TCE data by executing the following commands:
./Backup.sh
Step d
To upgrade R-TCE component, enter option 3.
Step e
To save the configuration information, enter <valid-location>. Your configuration information
is now available at <valid-location>/R-TCE-InstallValues.txt
Step 2
Stop the R-TCE component by executing the command.
./ipmssys stop
Step 3
Backup the R-TCE data by executing the following commands:
cd ./data/TCE/<TCE_NAME>
tar -cvf 3gpp_data.tar events
cp 3gpp_data.tar <data_backup_path>
Important:
Step 4
This <data_backup_path> should be outside the folder created during installation.
Uninstall the R-TCE component by executing the command.
./Uninstall_InTracer/Uninstall_InTracer
Step 5
Install the new binary at the same location, that is, <R-TCE-Home>, using console mode, entering the same
configuration details as previous installation.
Step 6
Go to the directory in which the R-TCE files are located by executing the command.
▄ InTracer Installation and Administration Guide, Release 16
72
Upgrading the InTracer
R-TCE Upgradation ▀
cd <R-TCE-Home>
Step 7
Restore the data by executing the following commands:
./ipmssys start
./ipmssys stop
cd ./data/TCE/<TCE_NAME>
rm -rf events
cp <data_backup_path>/3gpp_data.tar
tar -xvf 3gpp_data.tar
rm 3gpp_data.tar
Step 8
Start the R-TCE component by executing the command.
./ipmssys start
InTracer Installation and Administration Guide, Release 16 ▄
73
Upgrading the InTracer
▀ Post Upgradation
Post Upgradation
After upgrading InTracer components, perform the following steps:
Step 1
Restart the C-InTracer using the commands : ./ipmssys stop;./ipmssys start
Step 2
Clear your browser cache.
Step 3
Invoke InTracer UI.
Step 4
Check that all your data such as Active Traces, Search results, Saved or Scheduled Queries, Saved Users, User
Preferences and so on are all available in the newer upgraded version.
▄ InTracer Installation and Administration Guide, Release 16
74
Chapter 7
InTracer Clustering Support for High Availability
This chapter describes how to install and configure the Veritas cluster for high availability in InTracer. InTracer
application traces and analyzes near real time subscriber calls. InTracer heartbeat daemon monitors all the InTracer
processes and starts them when any process gets stopped. But if heart beat daemon itself goes down then there is no any
process which monitors the heart beat daemon. In clustering we monitor the heart beat daemon and whenever the heart
beat daemon gets stopped/crashed, clustering tries to restart it. Whenever machine gets crashed or if there is any loss of
communication between resources which clustering is going to monitor then failover happens to restore the loss of
communication.
This chapter contains the following sections:
 System and Hardware Recommendations for HA Deployment
 System and Hardware Recommendations for HA Deployment
 Setting up VERITAS Cluster Resources
 Basic Flow of Resource Configuration
 Installation Steps for Cluster Setup
 VCS Commands for Resource Configuration
 VCS Resource Configuration File
InTracer Installation and Administration Guide, Release 16 ▄
75
InTracer Clustering Support for High Availability
▀ System and Hardware Recommendations for HA Deployment
System and Hardware Recommendations for HA Deployment
The InTracer can be configured in HA mode with asymmetric or active/passive configuration. The following
requirements must be met in order to ensure the HA deployment in the InTracer system:
 2 UCS 460 M1 servers for cluster nodes
 Up to 4 Internal disks (depending upon the fault tolerance for boot disk/OS)
 Qlogic QLE2462 4Gb dual port FC HBA/Qlogic QLE2560 8 Gbps FC 1 port
 FC cables
 External Storage with multiple disks (EMC/Sun Storage)
 Veritas (VxFS) file system
The hardware setup for the Veritas Cluster Server (VCS) consists of two cluster nodes connected with an external
shared storage. UCS 460 supports multiple PCI slots for attaching the FC HBA cards. Both the cluster nodes must be
connected to the external storage. The cluster nodes must be installed with the Cisco MITG RHEL operating system,
Veritas Storage Foundation (Veritas Volume Manager and Veritas File System), and VCS (for High Availability).
The Veritas Volume Manager (VxVM) is used to create a single Disk Group (DG) containing multiple disks. A separate
group of disks is allocated for postgres data directory to achieve good performance.
The RAID recommendations are the same as that of standalone installation i.e. RAID-0 for InTracer application and
RAID-5 for database.
Separate disk/LUN from the shared storage is required for I/O fencing. I/O fencing is part of the VCS administration. It
is assumed that I/O fencing is already configured on the Veritas Cluster setup before the InTracer application is installed
for HA.
Configuring the External Storage Disk on UCS for HA Deployment
This section describes the procedure to configure an external storage disk on a UCS server for HA deployment.
It is recommended to create single disk group for the components C-InTracer / R-TCE:
 Configuration files, PostgreSQL data directory, Trace data files and actual data
It is recommended to create two separate disk groups for R-InTracer [Nodal]:
 Configuration files
 SGSN data
The number of disks in each group depends on the total throughput required and the size of disks in the external storage.
Please contact your system administrator for setting up the external storage to make the required number of disks
(Logical Disk Units (LUNs)) accessible from both the cluster nodes. When the external storage disks are made
accessible, you can see them connected using multipath command:
multipath -l
mpath6 (36006016069902d008892bc0dec14e111) dm-7 DGC,RAID 5
[size=300G][features=1 queue_if_no_path][hwhandler=1 emc][rw]
▄ InTracer Installation and Administration Guide, Release 16
76
InTracer Clustering Support for High Availability
System and Hardware Recommendations for HA Deployment ▀
\_ round-robin 0 [prio=0][active]
\_ 8:0:0:1 sdi 8:128 [active][undef]
\_ 8:0:1:1 sdk 8:160 [active][undef]
mpath5 (36006016069902d00563e8a20ec14e111) dm-6 DGC,RAID 5
[size=300G][features=1 queue_if_no_path][hwhandler=1 emc][rw]
\_ round-robin 0 [prio=0][active]
\_ 8:0:0:0 sdh 8:112 [active][undef]
\_ 8:0:1:0 sdj 8:144 [active][undef]
To configure an external storage disk into separate DGs:
Step 1
Rebuild the disk lists with the new disks detected by the kernel using the following command:
vxdctl initdmp
vxdctl enable
To see the status of the new disk, use the following command:
vxdisk -o alldgs list
DEVICE
Step 2
TYPE
DISK
GROUP
STATUS
disk_0
auto:none
-
-
online invalid
disk_1
auto:none
-
-
online invalid
disk_2
auto:none
-
-
online invalid
disk_3
auto:none
-
-
online invalid
emc_clariion0_28
auto
-
-
error
emc_clariion0_29
auto
-
-
error
DISK
GROUP
STATUS
To setup the disks, use the following commands:
/etc/vx/bin/vxdisksetup -i emc_clariion0_28
/etc/vx/bin/vxdisksetup -i emc_clariion0_29
To see the status of the new disk, use the command:
vxdisk -o alldgs list
DEVICE
TYPE
disk_0
auto:none
-
-
online invalid
disk_1
auto:none
-
-
online invalid
InTracer Installation and Administration Guide, Release 16 ▄
77
InTracer Clustering Support for High Availability
▀ System and Hardware Recommendations for HA Deployment
Step 3
disk_2
auto:none
-
-
online invalid
disk_3
auto:none
-
-
online invalid
emc_clariion0_28
auto:cdsdisk
-
-
online
emc_clariion0_29
auto:cdsdisk
-
-
online
With the newly initialized disks, create two separate DGs for one for storing the configuration files and one for SGSN
data storage only in case of R-InTracer [Nodal] installation:
vxdg init config_dg config_dg01=emc_clariion0_28
vxdg init db_dg db_dg01=emc_clariion0_29
Important:
You can specify more disks (using Step 2) and add the disk to disk groups.
After adding the disk into respective DGs, you can verify them using the following command:
vxdisk -o alldgs list
DEVICE
TYPE
DISK
GROUP
STATUS
disk_0
auto:none
-
-
online invalid
disk_1
auto:none
-
-
online invalid
disk_2
auto:none
-
-
online invalid
disk_3
auto:none
-
-
online invalid
emc_clariion0_28
auto:cdsdisk
config_dg01
emc_clariion0_29
auto:cdsdisk
db_dg0
config_dg01
db_dg
online
online
VxVM ensures that the newly created DGs are visible from both the cluster nodes. These disk groups can be used only
from one node at a time. You must import/deport a disk group from either node to use the disk groups and their
volumes.
Step 4
Create volumes in the disk groups using the following commands:
vxassist -g config_dg make config_vol 299g
For SGSN data storage, create four separate volumes of same size for db_db disk group
depending upon the disk space.
vxassist -g db_dg make db_vol0 70gvxassist -g db_dg make db_vol1 70gvxassist -g
db_dg make db_vol2 70gvxassist -g db_dg make db_vol3 70g
Step 5
Initialize the volumes with the VxFS file system using the following commands:
mkfs -t vxfs -o bsize=4096,largefiles /dev/vx/rdsk/db_dg/db_vol0mkfs -t vxfs -o
bsize=4096,largefiles /dev/vx/rdsk/db_dg/db_vol1mkfs -t vxfs -o
bsize=4096,largefiles /dev/vx/rdsk/db_dg/db_vol2mkfs -t vxfs -o
bsize=4096,largefiles /dev/vx/rdsk/db_dg/db_vol3
▄ InTracer Installation and Administration Guide, Release 16
78
InTracer Clustering Support for High Availability
System and Hardware Recommendations for HA Deployment ▀
For better performance, use a 4 KB block size and enable support for large files (more than 1 TB).
Step 6
Create the mount points and mount the volumes using the following commands:
mount -t vxfs -o largefiles /dev/vx/dsk/apps_dg/config_vol
/shared_intracer_config
mount -t vxfs -o largefiles /dev/vx/dsk/db_dg/db_vol0 /shared_intracer_db0mount t vxfs -o largefiles /dev/vx/dsk/db_dg/db_vol1 /shared_intracer_db1mount -t vxfs
-o largefiles /dev/vx/dsk/db_dg/db_vol2 /shared_intracer_db2mount -t vxfs -o
largefiles /dev/vx/dsk/db_dg/db_vol3 /shared_intracer_db3
InTracer Installation and Administration Guide, Release 16 ▄
79
InTracer Clustering Support for High Availability
▀ Setting up VERITAS Cluster Resources
Setting up VERITAS Cluster Resources
The following are the directories and files needed to be installed on shared disk:
1.
C-InTracer
2.
• /etc/ -> ipms_system.cfg and bootstrap.cfg file
• /data/-> Component specific configuration
• PostgreSQL data directory
R-InTracer - LTE
3.
• /etc/ -> ipms_system.cfg and bootstrap.cfg file
• /data/-> Component specific configuration,, TCE data and archived TCE files
R-InTracer - Nodal
• /etc/ -> ipms_system.cfg and bootstrap.cfg file
• /data/-> Component specific configuration
• All four volumes for storing SGSN data
The following are simple steps to configure the services and applications in Cluster:
1.
2.
3.
4.
Create a service group comprising of all resources required for the application
Add required resources to the service group and configure them
Assign dependencies between resources
Bring the service group online to make the resources available
After installation of InTracer application, the following resources need to be configured with the Veritas cluster:
 NIC — To monitor a Network Interface Card (NIC)
 IP — To monitor an IP address
 Disk Group, Volume, and Mount — for shared storage
 InTracer Application — comprising of all the InTracer related processes
The disk group, volume and mount points required on shared disk are as follows:
C-InTracer and R-TCE:
 Intracer-apps-dg -> One disk group required for storing the configuration files, PostgreSQL data directory in
case of C-InTracer and Subscriber Trace data (Berkeley DB) in case R-TCE and system and configuration
backup files.
 Intracer-apps-vol0 -> Only one volume is required for storing the above data
R-InTracer (NODAL): Currently SGSN data is stored on separate volumes. So to achieve this and for the performance,
two disk groups are required as below:

Intracer-apps-cfg-dg->' required for storing configuration files.
Intracer-apps-vol0-> only one volume is required for storing above data

Intracer-apps-db-db -> Required for storing SGSN data.
In this disk group, 4 separate volumes, as given below are required, which are mapped to respective eventproc process.

intercer-apps-vol0

intracer-apps-vol1

intracer-apps-vol2

intracer-apps-vol3
▄ InTracer Installation and Administration Guide, Release 16
80
InTracer Clustering Support for High Availability
Basic Flow of Resource Configuration ▀
Basic Flow of Resource Configuration
Overall design of application considering the shared disk and cluster nodes is as follows:
Figure 3.
C-InTracer and R-TCE
InTracer
Application
IP address
Shared Data
Mount
NIC
Shared Data
Volume
Shared Data
Disk Group
InTracer Installation and Administration Guide, Release 16 ▄
81
InTracer Clustering Support for High Availability
▀ Basic Flow of Resource Configuration
Figure 4.
R-InTracer (Nodal)
InTracer
Application
IP address
Shared Data
Mount
SGSN DB 4
Mounts
NIC
Shared Data
Volume
SGSN DB 4
Volumes
Shared Data
Disk Group
SGSN DB
Disk Group
▄ InTracer Installation and Administration Guide, Release 16
82
InTracer Clustering Support for High Availability
Installation Steps for Cluster Setup ▀
Installation Steps for Cluster Setup
This section describes installation steps for cluster setup.
Installing Cluster Failover for InTracer
Step 1
During installation process, there are two opt:
 Standalone installation
 Cluster failover
Choose “Cluster Failover” option to install Cluster Failover for InTracer.
Step 2
Selecting “Cluster Failover” option displays the following screen to the user to get the shared directory path. All of the
configuration files, PostgreSQL data dir and subscriber trace data is stored on this location.
After getting the shared disk path, installer installs all the following directories on shared disk (if not already present)
under the directory “intracer” which is the mount point created on the shared directory.
 /etc/
 /data/
 pgdata ' PostgreSQL data directory
InTracer Installation and Administration Guide, Release 16 ▄
83
InTracer Clustering Support for High Availability
▀ Installation Steps for Cluster Setup
Installer creates the symbolic link to the above directories in application code so that there are no code related changes
for changing the location.
In case of NODAL installation, all the data volumes required for storing SGSN data are on the shared disk, so in case of
cluster setup a note “Path should be on the shared disk” is shown on installer at the time of getting the SGSN data
directory path:
Step 3
Click Next to go to the Component Configuration screen to configure the selected component.
 The following Pre-Installation summary is displayed when Cluster Failover option is selected. Configured shared
disk path is shown along with the other configuration summary of the component.
▄ InTracer Installation and Administration Guide, Release 16
84
InTracer Clustering Support for High Availability
Installation Steps for Cluster Setup ▀
 The following Installation summary is displayed when Cluster Failover option is selected.
InTracer Installation and Administration Guide, Release 16 ▄
85
InTracer Clustering Support for High Availability
▀ Installation Steps for Cluster Setup
▄ InTracer Installation and Administration Guide, Release 16
86
InTracer Clustering Support for High Availability
VCS Commands for Resource Configuration ▀
VCS Commands for Resource Configuration
This section decribes how to configure VCS commands for Veritas Cluster.
Assume the configuration resources as follows:
Two disk-groups are - config_dg & db_dg.
The following is the resource configuration for R-InTracer [Nodal]:
Volumes are config_vol, db_vol0, db_vol1, db_vol2, db_vol3 mounted on /shared_intracer_config,
/shared_intracer_db0, /shared_intracer_db1, /shared_intracer_db2, /shared_intracer_db3 respectively.
InTracer installation directory - /<node1_path>/<component_name>
Component: R-InTracer [Nodal]
Path: /export/home/<username>/Nodal
Configuration data directories are as follows:
 /shared_intracer_config/data
 /shared_intracer_config/etc
 /shared_intracer_config/pgdata
SGSN data directories are as follows:
 /shared_intracer_db0
 /shared_intracer_db1
 /shared_intracer_db2
 /shared_intracer_db3
Shared ip-address - 10.4.83.151 (on NIC eth0)
InTracer Installation and Administration Guide, Release 16 ▄
87
InTracer Clustering Support for High Availability
▀ Query Server main.cs
Query Server main.cs
Step 1
The following commands are used to create Main Disk Group resource for Configuration data partition:
haconf -makerw
hagrp -add intracer-ha
hagrp -modify intracer-ha SystemList -add pnstextappsucs1 0 pnstextappsucs3 1
hagrp -modify intracer-ha AutoStartList pnstextappsucs1
hagrp -list
Step 2
The following commands are used to create Disk Group resource for Configuration data partition:
hares -add intracer-config-dg DiskGroup intracer-ha
hares -modify intracer-config-dg DiskGroup intracer_dg
hares -modify intracer-config-dg Enabled 1
Step 3
The following commands are used to create Volume resource for Configuration data partition:
hares -add intracer-config-vol Volume intracer-ha
hares -modify intracer-config-vol DiskGroup intracer_dg
hares -modify intracer-config-vol Volume intracer_data
hares -modify intracer-config-vol Enabled 1
Step 4
The following commands are used to create Mount resource for configuration partition:
hares -add intracer-config-mnt Mount intracer-ha
hares -modify intracer-config-mnt MountPoint /intracer_data
hares -modify intracer-config-mnt BlockDevice
/dev/vx/dsk/intracer_dg/intracer_data
hares -modify intracer-config-mnt FSType vxfs
hares -modify intracer-config-mnt FsckOpt %-y
hares -modify intracer-config-mnt MountOpt largefiles
hares -modify intracer-config-mnt Enabled 1
Step 5
The following commands are used to create Application resource for INTRACER processes:
hares -add intracer-app Application intracer-ha
hares -modify intracer-app User sdeshmuk
▄ InTracer Installation and Administration Guide, Release 16
88
InTracer Clustering Support for High Availability
Query Server main.cs ▀
hares -modify intracer-app StartProgram "/apps/intracer/182/QS/ipmssys start
with-monitor"
hares -modify intracer-app StopProgram "/apps/intracer/182/QS/ipmssys stop withmonitor"
hares -modify intracer-app CleanProgram "/apps/intracer/182/QS/ipmssys stop withmonitor"
hares -modify intracer-app PidFiles
"/apps/intracer/182/QS/data/QS/.ipmsqshbd.pid"
hares -modify intracer-app Enabled 1
Step 6
The following commands are used to create the NIC resource:
hares -add intracer-nic NIC intracer-ha
hares -modify intracer-nic Device eth0
hares -modify intracer-nic Enabled 1
Step 7
The following commands are used to create the IP resource:
hares -add intracer-ip IP intracer-ha
hares -modify intracer-ip Device eth0
hares -modify intracer-ip Address 10.105.85.128
hares -modify intracer-ip NetMask 255.255.255.0
hares -modify intracer-ip Enabled 1
Step 8
The following commands are used to set the resource dependencies:
hares -link intracer-app intracer-config-mnt
hares -link intracer-config-mnt intracer-config-vol
hares -link intracer-config-vol intracer-config-dg
hares -link intracer-app intracer-ip
hares -link intracer-ip intracer-nic
InTracer Installation and Administration Guide, Release 16 ▄
89
InTracer Clustering Support for High Availability
▀ TCE Main.cs
TCE Main.cs
Step 1
The following commands are used to create Main Disk Group resource for Configuration / TCE data partition:
haconf -makerw
hagrp -add intracer-ha
hagrp -modify intracer-ha SystemList -add pnstextappsucs1 0 pnstextappsucs3 1
hagrp -modify intracer-ha AutoStartList pnstextappsucs1
hagrp -list
Step 2
The following commands are used to create Disk Group resource for Configuration / TCE data partition:
hares -add intracer-config-dg DiskGroup intracer-ha
hares -modify intracer-config-dg DiskGroup intracer_dg
hares -modify intracer-config-dg Enabled 1
Step 3
The following commands are used to create Volume resource for Configuration / TCE data partition:
hares -add intracer-config-vol Volume intracer-ha
hares -modify intracer-config-vol DiskGroup intracer_dg
hares -modify intracer-config-vol Volume intracer_data
hares -modify intracer-config-vol Enabled 1
Step 4
The following commands are used to create Mount resource for configuration / TCE partition:
hares -add intracer-config-mnt Mount intracer-ha
hares -modify intracer-config-mnt MountPoint /intracer_data
hares -modify intracer-config-mnt BlockDevice
/dev/vx/dsk/intracer_dg/intracer_data
hares -modify intracer-config-mnt FSType vxfs
hares -modify intracer-config-mnt FsckOpt %-y
hares -modify intracer-config-mnt MountOpt largefiles
hares -modify intracer-config-mnt Enabled 1
Step 5
The following commands are used to create Application resource for INTRACER processes:
hares -add intracer-app Application intracer-ha
hares -modify intracer-app User sdeshmuk
▄ InTracer Installation and Administration Guide, Release 16
90
InTracer Clustering Support for High Availability
TCE Main.cs ▀
hares -modify intracer-app StartProgram "/apps/intracer/182/3GPP/ipmssys start"
hares -modify intracer-app StopProgram "/apps/intracer/182/3GPP/ipmssys stop"
hares -modify intracer-app CleanProgram "/apps/intracer/182/3GPP/ipmssys stop"
hares -modify intracer-app PidFiles
"/apps/intracer/182/3GPP/data/TCE/tce1/.ipmstcehbd.pid"
hares -modify intracer-app Enabled 1
Step 6
The following commands are used to create the NIC resource:
hares -add intracer-nic NIC intracer-ha
hares -modify intracer-nic Device eth0
hares -modify intracer-nic Enabled 1
Step 7
The following commands are used to create the IP resource:
hares -add intracer-ip IP intracer-ha
hares -modify intracer-ip Device eth0
hares -modify intracer-ip Address 10.105.85.128
hares -modify intracer-ip NetMask 255.255.255.0
hares -modify intracer-ip Enabled 1
Step 8
The following commands are used to set the resource dependencies:
hares -link intracer-app intracer-config-mnt
hares -link intracer-config-mnt intracer-config-vol
hares -link intracer-config-vol intracer-config-dg
hares -link intracer-app intracer-ip
hares -link intracer-ip intracer-nic
InTracer Installation and Administration Guide, Release 16 ▄
91
InTracer Clustering Support for High Availability
▀ Nodal main.cs
Nodal main.cs
Step 1
The following commands are used to create Main Disk Group resource for Configuration / TCE data partition:
haconf -makerw
hagrp -add intracer-sgsn
hagrp -modify intracer-sgsn SystemList -add pnstextappsucs1 0 pnstextappsucs3 1
hagrp -modify intracer-sgsn AutoStartList pnstextappsucs1
hagrp -list
Step 2
The following commands are used to create Disk Group resource for Configuration / TCE data partition:
hares -add intracer-config-dg DiskGroup intracer-sgsn
hares -modify intracer-config-dg DiskGroup intracer_dg
hares -modify intracer-config-dg Enabled 1
Step 3
The following commands are used to create Disk Group resource for SGSN data partition:
hares -add intracer-sgsn-dg DiskGroup intracer-sgsn
hares -modify intracer-sgsn-dg DiskGroup intracer_sgsn
hares -modify intracer-sgsn-dg Enabled 1
Step 4
The following commands are used to create Volume resource for Configuration / TCE data partition:
hares -add intracer-config-vol Volume intracer-sgsn
hares -modify intracer-config-vol DiskGroup intracer_dg
hares -modify intracer-config-vol Volume intracer_data
hares -modify intracer-config-vol Enabled 1
Step 5
The following commands are used to create Volume resource for SGSN data partition:
hares -add intracer-sgsn-vol0 Volume intracer-sgsn
hares -modify intracer-sgsn-vol0 DiskGroup intracer_sgsn
hares -modify intracer-sgsn-vol0 Volume intracer_vol0
hares -modify intracer-sgsn-vol0 Enabled 1
hares -add intracer-sgsn-vol1 Volume intracer-sgsn
hares -modify intracer-sgsn-vol1 DiskGroup intracer_sgsn
▄ InTracer Installation and Administration Guide, Release 16
92
InTracer Clustering Support for High Availability
Nodal main.cs ▀
hares -modify intracer-sgsn-vol1 Volume intracer_vol1
hares -modify intracer-sgsn-vol1 Enabled 1
hares -add intracer-sgsn-vol2 Volume intracer-sgsn
hares -modify intracer-sgsn-vol2 DiskGroup intracer_sgsn
hares -modify intracer-sgsn-vol2 Volume intracer_vol2
hares -modify intracer-sgsn-vol2 Enabled 1
hares -add intracer-sgsn-vol3 Volume intracer-sgsn
hares -modify intracer-sgsn-vol3 DiskGroup intracer_sgsn
hares -modify intracer-sgsn-vol3 Volume intracer_vol3
hares -modify intracer-sgsn-vol3 Enabled 1
Step 6
The following commands are used to create Mount resource for configuration partition:
hares -add intracer-config-mnt Mount intracer-sgsn
hares -modify intracer-config-mnt MountPoint /intracer_data
hares -modify intracer-config-mnt BlockDevice
/dev/vx/dsk/intracer_dg/intracer_data
hares -modify intracer-config-mnt FSType vxfs
hares -modify intracer-config-mnt FsckOpt %-y
hares -modify intracer-config-mnt MountOpt largefiles
hares -modify intracer-config-mnt Enabled 1
Step 7
The following commands are used to create Mount resource for SGSN partition:
hares -add intracer-sgsn-mnt0 Mount intracer-sgsn
hares -modify intracer-sgsn-mnt0 MountPoint /intracer_vol0
hares -modify intracer-sgsn-mnt0 BlockDevice
/dev/vx/dsk/intracer_sgsn/intracer_vol0
hares -modify intracer-sgsn-mnt0 FSType vxfs
hares -modify intracer-sgsn-mnt0 FsckOpt %-y
hares -modify intracer-sgsn-mnt0 MountOpt largefiles
hares -modify intracer-sgsn-mnt0 Enabled 1
hares -add intracer-sgsn-mnt1 Mount intracer-sgsn
hares -modify intracer-sgsn-mnt1 MountPoint /intracer_vol1
InTracer Installation and Administration Guide, Release 16 ▄
93
InTracer Clustering Support for High Availability
▀ Nodal main.cs
hares -modify intracer-sgsn-mnt1 BlockDevice
/dev/vx/dsk/intracer_sgsn/intracer_vol1
hares -modify intracer-sgsn-mnt1 FSType vxfs
hares -modify intracer-sgsn-mnt1 FsckOpt %-y
hares -modify intracer-sgsn-mnt1 MountOpt largefiles
hares -modify intracer-sgsn-mnt1 Enabled 1
hares -add intracer-sgsn-mnt2 Mount intracer-sgsn
hares -modify intracer-sgsn-mnt2 MountPoint /intracer_vol2
hares -modify intracer-sgsn-mnt2 BlockDevice
/dev/vx/dsk/intracer_sgsn/intracer_vol2
hares -modify intracer-sgsn-mnt2 FSType vxfs
hares -modify intracer-sgsn-mnt2 FsckOpt %-y
hares -modify intracer-sgsn-mnt2 MountOpt largefiles
hares -modify intracer-sgsn-mnt2 Enabled 1
hares -add intracer-sgsn-mnt3 Mount intracer-sgsn
hares -modify intracer-sgsn-mnt3 MountPoint /intracer_vol3
hares -modify intracer-sgsn-mnt3 BlockDevice
/dev/vx/dsk/intracer_sgsn/intracer_vol3
hares -modify intracer-sgsn-mnt3 FSType vxfs
hares -modify intracer-sgsn-mnt3 FsckOpt %-y
hares -modify intracer-sgsn-mnt3 MountOpt largefiles
hares -modify intracer-sgsn-mnt3 Enabled 1
Step 8
The following commands are used to create Application resource for INTRACER processes:
hares -add intracer-app Application intracer-sgsn
hares -modify intracer-app User sdeshmuk
hares -modify intracer-app StartProgram "/apps/intracer/182/NODAL/ipmssys start"
hares -modify intracer-app StopProgram "/apps/intracer/182/NODAL/ipmssys stop"
hares -modify intracer-app CleanProgram "/apps/intracer/182/NODAL/ipmssys stop"
hares -modify intracer-app PidFiles
"/apps/intracer/182/NODAL/data/TCE/tce1/.ipmstcehbd.pid"
▄ InTracer Installation and Administration Guide, Release 16
94
InTracer Clustering Support for High Availability
Nodal main.cs ▀
hares -modify intracer-app Enabled 1
Step 9
The following commands are used to create the NIC resource:
hares -add intracer-nic NIC intracer-sgsn
hares -modify intracer-nic Device eth0
hares -modify intracer-nic Enabled 1
Step 10
The following commands are used to create EVT-SOURCE NIC resource:
hares -add intracer-sgsn-nic NIC intracer-sgsn
hares -modify intracer-sgsn-nic Device eth1
hares -modify intracer-sgsn-nic Enabled 1
Step 11
The following commands are used to create the IP resource:
hares -add intracer-ip IP intracer-sgsn
hares -modify intracer-ip Device eth0
hares -modify intracer-ip Address 10.105.85.128
hares -modify intracer-ip NetMask 255.255.255.0
hares -modify intracer-ip Enabled 1
Step 12
The following commands are used to create EVT-SOURCE IP resource:
hares -add intracer-ip IP intracer-sgsn
hares -modify intracer-sgsn-ip Device eth1
hares -modify intracer-sgsn-ip Address 192.168.2.1
hares -modify intracer-sgsn-ip NetMask 255.255.255.0
hares -modify intracer-sgsn-ip Enabled 1
Step 13
The following commands are used to set the resource dependencies:
hares -link intracer-app intracer-config-mnt
hares -link intracer-config-mnt intracer-config-vol
hares -link intracer-config-vol intracer-config-dg
hares -link intracer-app intracer-sgsn-mnt0
hares -link intracer-sgsn-mnt0 intracer-sgsn-vol0
hares -link intracer-sgsn-vol0 intracer-sgsn-dg
hares -link intracer-app intracer-sgsn-mnt1
InTracer Installation and Administration Guide, Release 16 ▄
95
InTracer Clustering Support for High Availability
▀ Nodal main.cs
hares -link intracer-sgsn-mnt1 intracer-sgsn-vol1
hares -link intracer-sgsn-vol1 intracer-sgsn-dg
hares -link intracer-app intracer-sgsn-mnt2
hares -link intracer-sgsn-mnt2 intracer-sgsn-vol2
hares -link intracer-sgsn-vol2 intracer-sgsn-dg
hares -link intracer-app intracer-sgsn-mnt3
hares -link intracer-sgsn-mnt3 intracer-sgsn-vol3
hares -link intracer-sgsn-vol3 intracer-sgsn-dg
hares -link intracer-app intracer-ip
hares -link intracer-app intracer-sgsn-ip
hares -link intracer-ip intracer-nic
hares -link intracer-sgsn-ip intracer-nic
▄ InTracer Installation and Administration Guide, Release 16
96
InTracer Clustering Support for High Availability
VCS Resource Configuration File ▀
VCS Resource Configuration File
VCS resource configuration file (main.cf file) is created after executing resource configuration commands.
group intracer-ha (
SystemList = { pnstextappsucs3 = 0, pnstextappsucs1 = 1 }
)
Application intracer-app (
User = sdeshmuk
StartProgram = "/export/home/sdeshmuk/Nodal/ipmssys start"
StopProgram = "/export/home/sdeshmuk/Nodal/ipmssys stop"
PidFiles = {
"/export/home/sdeshmuk/Nodal/data/<component_name/.ipmsservhbd.pid"}
)
DiskGroup intracer-config-dg ( (
DiskGroup = config_dg
)
DiskGroup intracer-db-dg (
DiskGroup = db
)
IP intracer-ip (
Device = eth0
Address = "10.4.83.151"
NetMask = "255.255.255.0"
)
Mount intracer-config-mnt (
MountPoint = "/shared_apps"
BlockDevice = "/dev/vx/dsk/apps_dg/config_vol"
FSType = vxfs
MountOpt = largefiles
InTracer Installation and Administration Guide, Release 16 ▄
97
InTracer Clustering Support for High Availability
▀ VCS Resource Configuration File
FsckOpt = "-y"
)
Mount intracer-db0-mnt (
MountPoint = "/shared_db"
BlockDevice = "/dev/vx/dsk/db_dg/db_vol0"
FSType = vxfs
MountOpt = largefiles
FsckOpt = "-y"
)
Mount intracer-db1-mnt (
MountPoint = "/shared_db"
BlockDevice = "/dev/vx/dsk/db_dg/db_vol1"
FSType = vxfs
MountOpt = largefiles
FsckOpt = "-y"
)
Mount intracer-db2-mnt (
MountPoint = "/shared_db"
BlockDevice = "/dev/vx/dsk/db_dg/db_vol2"
FSType = vxfs
MountOpt = largefiles
FsckOpt = "-y"
)
Mount intracer-db3-mnt (
MountPoint = "/shared_db"
BlockDevice = "/dev/vx/dsk/db_dg/db_vol3"
FSType = vxfs
MountOpt = largefiles
FsckOpt = "-y"
▄ InTracer Installation and Administration Guide, Release 16
98
InTracer Clustering Support for High Availability
VCS Resource Configuration File ▀
)
NIC intracer-nic (
Device = eth0
)
Volume intracer-config-vol (
DiskGroup = config_dg
Volume = config_vol
)
Volume intracer-config-vol0 (
DiskGroup = config_dg
Volume = config_vol0
)
Volume intracer-config-vol1 (
DiskGroup = config_dg
Volume = config_vol1
)
Volume intracer-config-vol2 (
DiskGroup = config_dg
Volume = config_vol2
)
Volume intracer-config-vol3 (
DiskGroup = config_dg
Volume = config_vol3
)
intracer-app requires intracer-config-mnt
intracer-app requires intracer-config-mnt
intracer-app requires intracer-db-mnt
intracer-app requires intracer-ip
intracer-config-mnt requires intracer-config-vol
InTracer Installation and Administration Guide, Release 16 ▄
99
InTracer Clustering Support for High Availability
▀ VCS Resource Configuration File
intracer-config-vol requires intracer-config-dg
intracer-db0-mnt requires intracer-db-vol0
intracer-db-vol0 requires intracer-db-dg
intracer-db1-mnt requires intracer-db-vol1
intracer-db-vol1 requires intracer-db-dg
intracer-db2-mnt requires intracer-db-vol1
intracer-db-vol2 requires intracer-db-dg
intracer-db3-mnt requires intracer-db-vol3
intracer-db-vol3 requires intracer-db-dg
intracer-ip requires intracer-nic
Main.cf File Configuration
Below is the main.cf file for resource-groups and resources. Details are as below:Primary machine: pnqainpilot-x42701Secondary machine: pnqainpilot-x4270-2C-InTracer is installed at location: /xfs/InTracer/C-InTracer/R-TCE is
installed at location: /xfs/InTracer/R-TCEUsername: abcd
main.cf file contents:
include "OracleASMTypes.cf
include "types.cf"
include "Db2udbTypes.cf"
include "OracleTypes.cf"
include "SybaseTypes.cf"
cluster suncluster (
UserNames = { admin = dijBidIfjEjjHrjDig }
Administrators = { admin }
}
system pnqainpilot-x4270-1 (
}
system pnqainpilot-x4270-2 (
)
//C-InTracer configuration
▄ InTracer Installation and Administration Guide, Release 16
100
InTracer Clustering Support for High Availability
VCS Resource Configuration File ▀
group C-intracer-ha (
SystemList = { pnqainpilot-x4270-1 = 1, pnqainpilot-x4270-2 = 2 }
}
Application C-intracer-app (
User = abcd
StartProgram = "/xfs/InTracer/C-InTracer/ipmssys start with-monitor"
StopProgram = "/xfs/InTracer/C-InTracer/ipmssys stop with-monitor"
CleanProgram = "/xfs/InTracer/C-InTracer/ipmssys stop with-monitor"
PidFiles = { "/xfs/InTracer/C-InTracer/data/QS/.ipmsqshbd.pid" }
}
DiskGroup C-intracer-dg (
DiskGroup = config_dg
}
IP C-intracer-ip (
Device = eth0
Address = "10.105.87.247"
NetMask = "255.255.255.0"
}
Mount C-intracer-mnt (
MountPoint = "/shared_intracer_config"
BlockDevice = "/dev/vx/dsk/config_dg/config_vol"
FSType = vxfs
MountOpt = largefiles
FsckOpt = "-y"
)
NIC C-intracer-nic (
Device = eth0
)
Volume C-intracer-vol (
InTracer Installation and Administration Guide, Release 16 ▄
101
InTracer Clustering Support for High Availability
▀ VCS Resource Configuration File
DiskGroup = config_dg
Volume = config_vol
}
C-intracer-app requires C-intracer-ip
C-intracer-app requires C-intracer-mnt
C-intracer-ip requires C-intracer-nic
C-intracer-mnt requires C-intracer-vol
C-intracer-vol requires C-intracer-dg
//R-TCE Configuration
group R-TCE-ha (
SystemList = { pnqainpilot-x4270-1 = 1, pnqainpilot-x4270-2 = 2 }
)
Application R-tce-app (
User = hryadav
StartProgram = "/xfs/InTracer/R-TCE/ipmssys start"
StopProgram = "/xfs/InTracer/R-TCE/ipmssys stop"
CleanProgram = "/xfs/InTracer/R-TCE/ipmssys stop"
PidFiles = { "/xfs/InTracer/R-TCE/data/TCE/Ind/.ipmstcehbd.pid" }
)
DiskGroup R-tce-dg (
DiskGroup = db_dg
)
IP R-tce-ip (
Device = eth3
Address = "10.105.87.248"
NetMask = "255.255.255.0"
)
Mount R-tce-mnt (
MountPoint = "/shared_intracer_db"
▄ InTracer Installation and Administration Guide, Release 16
102
InTracer Clustering Support for High Availability
VCS Resource Configuration File ▀
BlockDevice = "/dev/vx/dsk/db_dg/db_vol"
FSType = vxfs
MountOpt = largefiles
FsckOpt = "-y"
)
NIC R-tce-nic (
Device = eth3
)
Volume R-tce-vol (
DiskGroup = db_dg
DiskGroup = db_dg
)
R-tce-app requires R-tce-ip
R-tce-app requires R-tce-mnt
R-tce-ip requires R-tce-nic
R-tce-mnt requires R-tce-vol
R-tce-vol requires R-tce-dg
InTracer Installation and Administration Guide, Release 16 ▄
103
Chapter 8
Threshold Configuration
Thresholding on the system is used to monitor the system for conditions that could potentially cause errors or outage.
Typically, these conditions are temporary (i.e. high CPU utilization, or packet collisions on a network) and are quickly
resolved. However, continuous or large numbers of these error conditions within a specific time interval may be
indicative of larger, more severe issues. The purpose of thresholding is to help identify potentially severe conditions so
that immediate action can be taken to minimize and/or avoid system downtime.
Intracer supports following two categories of thresholds for R-Intracer and R-TCE:
 System Thresholds
 Process Thresholds
InTracer Installation and Administration Guide, Release 16 ▄
105
Threshold Configuration
▀ System Thresholds
System Thresholds
The system thresholds are applicable to both R-Intracer and R-TCE. The following table describes the systems
thresholds.
Table 1. System Thresolds
Name
Description
Default
Value (%)
Total CPU usage
(%)
If total cpu usage by R-Intracer/R-TCE w.r.t total cpu present on the system exceeds the
configured threshold percentage via Intracer GUI then an alarm is raised.
25
Total memory
usage (%)
If total memory usage by R-Intracer/R-TCE w.r.t total memory present on the system exceeds
the configured threshold percentage via Intracer GUI then an alarm is raised.
25
Total disk usage
(%)
If total disk usage by R-Intracer/R-TCE w.r.t total disk present on system exceeds the
configured threshold percentage via Intracer GUI then an alarm is raised.
25
InTracer 'export'
dir size (%)
If export data directory size of R-Intracer/R-TCE w.r.t total disk present on the system
exceeds the configured threshold percentage via Intracer GUI then an alarm is raised.
25
InTracer 'log' dir
size (%)
If log data directory size of R-Intracer/R-TCE w.r.t total disk present on the system exceeds
the configured threshold percentage via Intracer GUI then an alarm is raised.
25
InTracer 'data' dir
size (%)
If data directory size of R-Intracer/R-TCE w.r.t total disk present on the system exceeds the
configured threshold percentage via Intracer GUI then an alarm is raised.
25
InTracer 'crash' dir
size (%)
If crash directory size of R-Intracer/R-TCE w.r.t total disk present on the system exceeds the
configured threshold percentage via Intracer GUI then an alarm is raised.
25
▄ InTracer Installation and Administration Guide, Release 16
106
Threshold Configuration
Process Thresholds ▀
Process Thresholds
The process thresholds are applicable to both R-Intracer and R-TCE. The following table describes the process
thresholds applicable to R-InTracer.
Table 2. R-InTracer Process Thresholds
Name
Description
Default
Value (%)
SERVHBD CPU
usage (%)
If server HBD process cpu usage of R-Intracer w.r.t total cpu present on the system
exceeds the configured threshold percentage via Intracer GUI then an alarm is raised.
25
SERVLOG CPU
usage (%)
If server log process cpu usage of R- Intracer w.r.t total cpu present on the system exceeds
the configured threshold percentage via Intracer GUI then an alarm is raised.
25
SERVMGR CPU
usage (%)
If server manager process cpu usage of R- Intracer w.r.t total cpu present on the system
exceeds the configured threshold percentage via Intracer GUI then an alarm is raised.
25
QUERYPROC CPU
usage (%)
If query process cpu usage of R- Intracer w.r.t total cpu present on the system exceeds the
configured threshold percentage via Intracer GUI then an alarm is raised.
25
EVTPROC-0 CPU
usage (%)
If event process 0 cpu usage of R- Intracer w.r.t total cpu present on the system exceeds
the configured threshold percentage via Intracer GUI then an alarm is raised.
25
EVTPROC-1 CPU
usage (%)
If event process 1 cpu usage of R- Intracer w.r.t total cpu present on the system exceeds
the configured threshold percentage via Intracer GUI then an alarm is raised.
25
EVTPROC-2 CPU
usage (%)
If event process 2 cpu usage of R- Intracer w.r.t total cpu present on the system exceeds
the configured threshold percentage via Intracer GUI then an alarm is raised.
25
EVTPROC-3 CPU
usage (%)
If event process 3 cpu usage of R- Intracer w.r.t total cpu present on the system exceeds
the configured threshold percentage via Intracer GUI then an alarm is raised.
25
EVTPROC-4 CPU
usage (%)
If event process 4 cpu usage of R- Intracer w.r.t total cpu present on the system exceeds
the configured threshold percentage via Intracer GUI then an alarm is raised.
25
EVTPROC-5 CPU
usage (%)
If event process 5 cpu usage of R- Intracer w.r.t total cpu present on the system exceeds
the configured threshold percentage via Intracer GUI then an alarm is raised.
25
EVTPROC-6 CPU
usage (%)
If event process 6 cpu usage of R- Intracer w.r.t total cpu present on the system exceeds
the configured threshold percentage via Intracer GUI then an alarm is raised.
25
EVTPROC-7 CPU
usage (%)
If event process 7 cpu usage of R- Intracer w.r.t total cpu present on the system exceeds
the configured threshold percentage via Intracer GUI then an alarm is raised.
25
The following table describes the process thresholds applicable to R-TCE.
Table 3. R-TCE Process Thresholds
Name
Description
Default
Value (%)
InTracer Installation and Administration Guide, Release 16 ▄
107
Threshold Configuration
▀ Process Thresholds
Name
Description
Default
Value (%)
TCEHBD CPU usage (%)
If TCE HDB process cpu usage of R-TCE w.r.t total cpu present on the system
exceeds the configured threshold percentage via Intracer GUI then an alarm is raised.
25
TCELOG CPU usage (%)
If TCE log process cpu usage of R-TCE w.r.t total cpu present on the system exceeds
the configured threshold percentage via Intracer GUI then an alarm is raised.
25
TCEMGR CPU usage (%)
If TCE manager process cpu usage of R-TCE w.r.t total cpu present on the system
exceeds the configured threshold percentage via Intracer GUI then an alarm is raised.
25
TCEQUERYPROC CPU
usage (%)
If TCE query process cpu usage of R-TCE w.r.t total cpu present on the system
exceeds the configured threshold percentage via Intracer GUI then an alarm is raised.
25
XMLPARSER CPU usage
(%)
If xml parser process cpu usage of R-TCE w.r.t total cpu present on the system
exceeds the configured threshold percentage via Intracer GUI then an alarm is raised.
25
▄ InTracer Installation and Administration Guide, Release 16
108