.
.
.
Have also a look at this link:
.
.
http://forums.itrc.hp.com/cm/Q
uestionAnswer/1,,0x51537bb0
4b5cd611abdb0090277a778c,
00.html
.
.
.
.
HP OpenView Remote NNM
Data Access & Warehouse
Consolidation
Gaining “live” access to NNM data and making
NNM data warehouse consolidation easy
OpenView Data Extraction and Reporting (version 1.02, 21 Nov 2000)
HP OpenView Integration Lab
Hewlett-Packard Company
Version 1.02
November 21, 2000
 HP OpenView Remote NNM Data Access & Warehouse Consolidation
Page 2
Notices
This publication is provided "as is" without warranty of any kind, either expressed or implied. Use of
this publication is at your own risk and Hewlett-Packard Company shall have no liability for damages of
any kind.
While reasonable precautions have been taken in the preparation of this document, Hewlett-Packard
Company assumes no responsibility for errors or omissions. This document may contain technical
inaccuracies or typographical errors. This document may be modified without notice.
The names of products and services included herein are trademarks
of their respective owners. The products described in this
publication may also be protected by one or more US patents,
foreign patents and/or pending applications, copyright and/or other
intellectual property rights.
Introduction
Objective and Purpose
This paper provides detailed instructions on implementing ODBC drivers to gain remote access to
NNM’s embedded database. It will also present three different ways to consolidate the database NNM
databases into one complete management database. Finally, one of the consolidation methods will be
chosen and detailed instructions for automating the process will be provided. With this information,
network and system managers can access NNM data across different platforms and systems. This
solves the issues of accessing data on a UNIX box using a Windows NT or Windows 2000 box. In
addition to remote data access, this information will provide network and system managers with
alternatives on how to combine the data from collection stations onto one NNM data warehouse.
Maintaining one management warehouse enables management and technical staff to have one source
for detailed reports and trending information.
This paper provides a review on setting up NNM data collection, remote SOLID access through ODBC,
and consolidation options using ODBC and FTP.
Finally, this paper describes a current case study that provides an automation option for consolidation.
Intended Audience
This paper has been created for several audiences:

OpenView Data Extraction and Reporting
Page 1
Network
administrators, system administrators, consultants and system integrators
who are responsible for using OpenView products to provide useful information at
all levels of an organization
IT
Managers who are interested in understanding consolidation capabilities of HP
OpenView Network Node Manager
Outline
This paper starts with a review of how to initiate trend data collection with NNM, which should be of
interest to all readers. The paper then covers three main topics:
Remote
SOLID Access describes the scenario in which the SOLID embedded
database can be accessed from a console separate from the one the database is
running on. This section describes the process of downloading and configuring
the SOLID ODBC driver.
Consolidation
Methods describes the process of gathering data from collections
stations onto one NNM data warehouse. This section will describe consolidation
methods using ODBC or FTP.
Consolidation
Automation describes the process of creating a script to make
database consolidation automatic. This section describes the command and
process involved in automating database consolidation.
This paper concludes with a discussion of strategic issues that can arise in NNM
when database consolidation is implemented (data loss and corruption).
Type Conventions
Text visible on the screen appears as a fixed-width
font.
Commands typed in on the command line appear as a bold
fixed-width font.
The author's comments appear as text in italics.
An example of these conventions follows:
root@crash> /opt/OV/bin/ovstatus -c
will yield a compressed status list of NNM services, based on the "ovsuf" file
Name
PID State
Last Message(s)
OVsPMD
ovwdb
ovtrapd
ovactiond
16644
16645
16647
16648
RUNNING
RUNNING
RUNNING
RUNNING
Initialization complete.
Initialization complete.
Initialization complete.
Most of the command line interfaces available to the user in HPUX are also available on Windows
NT/2000. The syntax, however, differs slightly. To maintain consistency, all examples are illustrated in
 HP OpenView Remote NNM Data Access & Warehouse Consolidation
Page 2
the HPUX environment. Readers should consult the appropriate product manual to generate the
correct syntax for Windows NT/2000.
This paper is intended to raise awareness and provide a jump-start for those interested in implementing
data reporting. This paper should be perceived as a complement to various OpenView product
manuals, not as a substitute for them. Where appropriate, pointers to specific documentation are
provided.
We hope you find this paper interesting and useful. If you have specific feedback regarding the content
or format of this paper, please send them to ovint@cnd.hp.com. We appreciate your interest in
OpenView data reporting.
Refresher on NNM Trend Data Collection
Description
snmpCollect is a service that is configured through the Options->Data Collection and
Thresholds:SNMP menu item of NNM. snmpCollect collects SNMP MIB values from network nodes at
regular intervals and, if so configured, stores them to a file. SnmpCollect can also check the collected
values against user-defined threshold values and generate events when the thresholds are exceeded.
The snmpCollect data files are compatible across computers with different architectures. For example,
an snmpColDump running on an HP-UX Series 700/800 computer can directly read snmpCollect data
files generated on a Windows NT/2000 computer, and vice-versa. The data is stored in network byte
order.
 HP OpenView Remote NNM Data Access & Warehouse Consolidation
Page 3
Storing Data
If data storing is enabled, collected data is stored in the directory \install_dir\databases\snmpCollect.
During online backup (See ovpause), data collection continues but data is temporarily stored in
\install_dir\tmp\snmpCollect along with the snmpCol.trace file. After online backup, all data collected
during the pause is appended to the regular \ install_dir \databases\snmpCollect data store and the \
install_dir \log\snmpCol.trace is brought up-to-date. In case of a system failure during online backup,
the data collector will automatically recover data collected during the pause. For each MIB variable,
snmpCollect creates two files:
A
binary data file containing the data collected for that variable. The binary file has a
name constructed from the MIB label specified in the MIB Data Collection menu, with
the instance number appended to it.
A
separate ID file containing the units, syntax, and object ID for the MIB variable. This ID
file has the same name as the binary data file, except there is a (!) appended to the
filename.
The author's comments appear as text in italics.
The content of the binary data file consists of quadruplets of startTime, endTime, IPaddress, and value.
The startTime, endTime, and IPaddress are 4-byte, unsigned long values in network byte order. The
collected value is an 8-byte, double value in network byte order. To convert the binary data to human
readable ASCII format, use snmpColDump, or write your own program using the following data
structure to parse the data file:
struct dcRecordStruct {
time_t begin; /* start time of interval */
time_t end;
/* time of SNMP response */
u_long ip;
u_long pad;
double val;
};
The way in which data is presented varies based on the type of data received. See mib.coerce for a
discussion on how data is presented, and how to control the way in which it is presented.
If the binary data file is removed at any time, snmpCollect creates a new file at the next collection. The
binary files grow unbounded, so it is advisable to remove them periodically. Be sure not to remove the
ID files; that is, file names with a (!) at the end of them. To delete earlier data, use the -r option to
snmpColDump.
The trace file also grows without bounds. It can also be removed, but tracing must be toggled twice to
stop and restart it.
 HP OpenView Remote NNM Data Access & Warehouse Consolidation
Page 4
Warnings
snmpCollect requires that pmd be running so it can listen for newly discovered nodes and send
threshold events.
When netmon discovers a new node, snmpCollect checks to see if the new node's IP address is
contained within any of the IP address wildcard conditions specified to determine if any collections
should be automatically started on the new node.
You can only collect data from managed nodes, unless the -u option is specified. Otherwise, if you
unmanage the node using ovw, collection ceases until you manage the node again.
If a node does not reply (because it or its agent is down), snmpCollect defers SNMP requests four
consecutive times within the first hour of the attempt. If there is still no response, an event described
previously can be sent to NNM. Deferring will continue on the node until a defer time has expired, or a
demand poll or object ID changed event occurs. At snmpCollect invocation, the -D deferminutes option
allows you to specify a different defer time.
Collections can also be done through a proxy agent node. The proxy target must be included in the
map, and the appropriate entries added through the Options->SNMP Configuration menu item of NNM.
Remote SOLID Access
This chapter of the paper describes how to remotely connect to the data in NNM’s SOLID embedded
database. The process of downloading and configuring the SOLID ODBC driver will be explained in
detail.
ODBC
ODBC (short for "Open Database Connectivity") can act as a bridge between the data source and a
query tool. ODBC is standards-based, and available on every major computing platform1. Using
1
ODBC is most prevalent in Windows NT/95/98 Operating Environments.
 HP OpenView Remote NNM Data Access & Warehouse Consolidation
Page 5
ODBC, a user connects to a data source, identified by a name. From there, an authenticated user will
be able to create, read, update, and delete data associated with that data source.
Conceptually, ODBC is a very powerful way of accessing data from many different sources. However,
users must keep several issues in mind when using ODBC:
ODBC
driver versions may often conflict, creating potential problems for several
applications that rely on ODBC. ODBC drivers, data source names, and other
ODBC-related information are managed via the "ODBC Administrator" facility,
accessible from the Control Panel on both Windows 95, Windows NT, and
Windows 2000.
The
complexity involved with ODBC quickly increases when the machine performing
the reporting is different than the machine containing the data. In order to make
ODBC work in this case, administrators must understand how to configure
network names into to ODBC data source, and manually configure ODBC drivers
on the remote machine.
Different
applications may work with ODBC in slightly different ways. As a result,
ODBC-based actions may behave differently among applications.
ODBC
acts as a common denominator among different database formats. Some
negative side effects of being a common denominator include slow performance,
and occasional failures that result from working with data types that are radically
different. Furthermore, ODBC is not a common connectivity standard with
respect to UNIX networking.
Obtaining the Driver
On your client machine, download the appropriate self-extracting ODBC package from
http://www.solidtech.com/developer/software.html. After the download is complete, run it and install the
driver.
Configuring the Driver
The following are steps to lead you through configuring the ODBC driver you just installed:
Open the Data Sources option on
In
your Control Panel.
the ODBC Data Source Administrator window, select the System DSN (Data
Source Name) tab. This tab let’s you add, remove or configure existing System
Data Sources.
Choose the add option to create a new data source.
Select
the Solid Server 2.3 driver from the list of drivers. This is the driver you just
downloaded and installed. Then hit the finish button.
 HP OpenView Remote NNM Data Access & Warehouse Consolidation
Page 6
In
the SOLID Server ODBC Setup window, enter a logical Data Source Name you
will remember and understand (this name will be what you supply your query tool
with to build a connection).
The DSN Description section is option is optional.
In
the NetworkName space add the string: TCPIP computername 2690. TCPIP is
the type of data connection you’ll have. Replace computername with the fully
qualified name of the computer that the NNM database is installed on. 2690 is
the default port the NNM embedded database is installed to.
Now you can use your favorite query tool to connect to the NNM data on the computer you configured
your driver for. If you want to connect to data on a different NNM station, go back into the System DSN
tab in the ODBC Data Source Administrator window, select the DSN you created and hit the configure
button. In the SOLID Server ODBC Setup window change the name of the computer in the
NetworkName section to the one you want to connect to.
You can also use the SQL-based query tool that is provided with NNM. ovdwquery takes a query
written in Structured Query Language (SQL), sends the query to the relational database configured as
the NNM data warehouse, and returns the results of the query to the standard output. When no options
are specified, ovdwquery reads the query from the standard input, connects to the database as the
default user (using the default password), and writes the output in TAB-separated format (each field of
data on each line of output separated by the TAB character).
The SQL query can contain one or more statements. The queries can be targeted at the NNM tables
defined in the data warehouse as described in the Reporting and Data Analysis with HP Openview
Network Node Manager book. Other tables defined in the warehouse can also be the subject of
queries.
 HP OpenView Remote NNM Data Access & Warehouse Consolidation
Page 7
The ovdwquery command has a –rdb command which gives you the ability to specify which
configured data source you want to connect to. The following is an example string you would enter in
command line:
root@crash> ovdwquery –rdb nnmodbc
Consolidation Methods
When NNM collects trending data, the entries are written in binary files under the
\install_dir\databases\snmpCollect directory. When an administrator wants the NNM data warehouse
to be updated with the entries collected in the binary files, they can choose one of two ways to do the
update. The administrator can either run an ovdwtrend –export from command line or he/she can do
an Export Trend Data command from the Data Warehouse option under the Tools menu in the NNM
interface. For our consolidation purposes, we are required to run our exports from the command line.
Note: A –v parameter can be added to all NNM commands mentioned in this paper to view the output
of the command.
ODBC “Push” Consolidation Method (See attached AVI file for example)
The ODBC consolidation uses a “push” philosophy in that it takes the binary files on NNM collection
stations and exports the data to the management station’s data warehouse. For ODBC consolidation,
the configuration is done exclusively from a command line on the collection stations. From the
command line the ovdwtrend command is issued with an -export and -rdb parameter to use the ODBC
connection to the data warehouse. Make sure you use the correct DSN to update the data warehouse.
The DSN’s NetworkName should be configured with the fully qualified machine name of the
management station. You can check this configuration in the SOLID Server ODBC Setup window
under the System DSN tab in the ODBC Data Source Administrator window. The ODBC Data Source
Administrator window can be found in the Control Panel. The following is an example string you would
enter in command line:
ovdwtrend -export -rdb odbcnnm -v
FTP “Put” Consolidation Method (See attached AVI file for example)
The FTP “put” consolidation method pushes the binary files from the NNM collection stations to the
management station’s data warehouse. Then an export is performed using the binary files received
from the collection stations. This export is done from command line on the management station. This
process is the most complicated and extensive of the three methods presented in this paper.
First, open a command line on each collection station and move to the
\install_dir\databases\snmpCollect directory. Then start an ftp session with the management station
and issue an mput * command to send all files in the current directory on the collection station to the
management station’s default FTP directory. Make sure that each set of binary files from each
collection station is in their own directory. You will do an export from each directory. Finally, in a
command line on the management station, issue an ovdwtrend command with an –exportfrom
parameter to point to where you saved the files on the management station. The following is an
example string you would enter in command line:
ovdwtrend -exportfrom c:\inetpub\ftproot –v
 HP OpenView Remote NNM Data Access & Warehouse Consolidation
Page 8
FTP “Get” Consolidation Method (See attached AVI file for example)
The FTP “get” consolidation method pulls the binary files from the collection stations to the
management station. Then an export is performed using the binary files received from each collection
station. This export is done from command line on the management station. This method is performed
exclusively from a command line on the management station.
First, open a command line on the management station. Then start an ftp session with the collection
station and move to the \install_dir \databases\snmpCollect directory. Issue an mget * command to
send all files in the current directory on the collection station to the management station’s current
working directory (C:\ is the default). Make sure that each set of binary files from each collection station
is in its own directory. You will do an export from each of these directories. Finally, in a command line
on the management station, issue an ovdwtrend command with an –exportfrom parameter to point to
where you saved the files on the management station. The following is an example string you would
enter in command line:
ovdwtrend -exportfrom c:\inetpub\ftproot –v
Case Study
This section discusses the process of automating the FTP “get” consolidation process. It will cover the
commands used and the data flow processes involved. Currently, we are in the process of creating this
solution for distribution. It should be available by late December 2000 or early January 2001.
Purpose and Process
The purpose of this script is to deliver an out-of-box solution to provide a simple way to automate NNM
data warehouse consolidation. The script will be a shell script if run on Unix or a batch script if run on
Windows. Or if you’re comfortable with Perl, you could modify it to Perl and it will be platform
independent. The script can be scheduled to run anytime with the default schedulers packaged with
Unix and Windows. Or you can use the scheduler provided with your VPO agent.
From the management station the batch/shell/Perl script will issue an NNM specific command
(xnmtopoconf) to pull every collection station name associated with the management station. After
parsing just the collection machine names into a list a data collection directory will be created for each
collection station. One at a time the script will “cd” to each data collection directory on the management
station and start and ftp session with the corresponding collection station. In the ftp session, the script
will cd the the directory where the binary data files are stored (\install_dir \databases\snmpCollect) and
issue the mget * command to pull the binary data files from the collection stations to the corresponding
data collection folders on the management station. Finally, after the management station has obtained
all the binary files from each collection station, an NNM command (ovdwtrend –exportfrom) will be
issued to export the contents of the binary file to the data warehouse.
 HP OpenView Remote NNM Data Access & Warehouse Consolidation
Page 9
Commands Needed
xnmtopoconf NNM command (to get all collection machine names)
mget * from management station on each collection station (to transfer binary files)
ovdwtrend -exportfrom command on each set of binary files retrieved (to export binary data to data
warehouse)
Warnings and Preventive Sources
Commands Needed
Overlapping domains collecting SNMP data will lead to data integrity and data loss issues!
Preventive Sources
To avoid these NNM issues and others like these, consult the following documents when configuring
and using NNM:
Designing Distributed NNM - concepts and theories for designing a distributed NNM implementation
for versions 4.1x, 5.x, and 6.0
Service Reporter Network Node Manager Integration (Packaged with Scaling the Heights zip file on
Channel Web) – Integration Package for the NNM/SQL Server or SOLID Database Scalability and Distribution – Ideas and features for scalability distribution in NNM, architecture of
NNM software, and specific configuration procedures for using the scaling and distribution features of
NNM
Performance and Configuration Guide – performance characterization and configuration information
for NNM assisting in sizing systems dedicated to running NNM in your environment
HP OpenView Data Extracting and Reporting – creating the reports you need using OpenView
products and PC-based reporting tools
 HP OpenView Remote NNM Data Access & Warehouse Consolidation
Page 10