Radia Usage Manager Best Practices Overview The purpose of this document is to provide a set of best practices with regard to the implementation and ongoing management of the Radia Usage Manager in a production environment. Configuring Inventory Inventory Scheduling Inventory should be run weekly at most. This is sufficient for reporting installed, but unused applications on each client. Inventory should be scheduled when the machine is not typically shutdown or in a high use situation. The UMINVENT class of the Radia database controls inventory processing in a Radia managed environment. Inventory Scan The inventory scan mode has an impact on how intensively the scan process runs, the scan accuracy, and scan times. The Scan Mode of S, which checks for file date and size, is typically sufficient and is the default if no scan mode is specified. An MD5 signature scan is always performed the first time a file is found. Configuring Collection Collection Scheduling Collection is a very low overhead request on the client and can be scheduled anytime. Usage collection should be run weekly at most. Daily collections are not necessary. This interval is sufficient for reporting application usage on each client. The UMCOLLCT class of the Radia database controls usage collection file processing in a Radia managed environment. If the client is Radia managed, then you may want to schedule the collection during the daily Radia connect to the device using a script or other means. This ensures that the network is available to the device when sending its usage data. Specify an interval of None to disable the Usage Manager’s internal scheduler. For non-Radia managed devices, you must use the Usage Manager’s internal scheduler. Tip: Stagger client collections for groups of devices for different days of the week and use the randomized collection scheduling feature. This ensures that network bandwidth and import bottlenecks are minimized. Collection Protocol It is strongly recommended that you use HTTP as the collection file network transfer protocol to send files to the Radia Integration Server. Radia Usage Manager Page 1 2/12/2016 Collection Failover The Version 2.1 client supports collection file failover to back-end collection destination locations. If multiple Integration Servers are available in your environment, when defining the destination locations, separate each location with a semi-colon. Should a failure occur when sending to the first location, then the second through nth locations are tried. For example: http://ip_addr1:3466/KB_Mgr1_Usage;http://ip_addr2:3466/KB_Mgr1_Usage The COLLDEST variable of the USAGE.UMDESTPT class defines the collection destination point location(s) when configuring Usage Manager collection through the Radia System Explorer. Note that the Knowledge Base Manager must be configured to check each of the destination locations for files to be imported into the usage database. A separate import task is needs to be configured for each destination location directory. Filtering – Exclude OS applications Important: Filter the collection to minimally exclude operating system files. Minimizing the amount of data collected in the backend SQL database is important for reporting performance and reporting on Notepad or Calculator usage is irrelevant. If device availability statistics are needed, collect the EXPLORER.EXE only. Sample filters are supplied as Radia objects to facilitate OS executable exclusions. Do not use concurrent usage unless it is absolutely needed. Concurrent usage has extremely high overhead during import and reporting. Usage Database Compression The absence (disabled) or presence (enabled) of the GZIP.EXE module determines whether the client sends compressed or uncompressed usage files to the collection destination point. Import processing time is greatly affected if the usage files are compressed, significantly reducing the import throughput. If you have sufficient network bandwidth, do not compress the usage database files. If you have insufficient network bandwidth, install the GZIP executable to the \IDMSYS directory. A default package is included in the USAGE domain of the RCS database. Radia Usage Manager Client Installation The status of the Usage Manager installation processes is logged in the Novadigm\Setup\Log\ directory. All installation processing is recorded to a Usage Manager specific log file and appended to any existing file content. Troubleshooting Usage collection and monitoring activity is logged to the \Novadigm\Usage Manager\Log\ directory. A new log is created daily and kept for 7 days by default. Use this log to troubleshoot Usage Manager client behavior. The HKLM/Software/Novadigm/Application Extensions/Usage Manager/Debug/ key can be used to enable additional logging needed for debugging. This key is a DWORD and for full debugging can be set to 0xffffffff. Note that this will cause extensive logging activity and should be used under the direction of HP Technical Support. This key should be deleted or set to 0x00000000 for normal logging operation. Radia Integration Server (RIS) Radia Usage Manager Page 2 2/12/2016 The Radia Integration Server has serves two distinct functions: it serves as the collection file destination point server that receives the usage data files from each client and; it serves as the web server that hosts the Usage Manager reporting. Collection File Destination Server Use a separate server for this function. This does not have to be a high-end server since its sole function is to receive files and store them into a directory. If compressed usage files are used, then ensure that this server has sufficient CPU processing power. Run the Knowledge Base Manager on this same device, or on a device located on the same LAN segment, to improve import speed and file transfers. Use fast disk drives if they are available as there is a great deal of I/O occurring against the usage data files. Locate this server on the network where collection files can be sent and stored. Usage Manager Reporting Server Usage Manager reporting must not be enabled on the same Integration Server as the collection file destination server. Current RIS technology single threads activity to the server which will cause severe throughput issues. Usage Manager reporting should not be enabled on a production use Integration Server or Management Portal used for managing the Radia infrastructure. Again, the single thread nature of the RIS server will cause excessive waits for all other users while usage reports are being run. Run a separate RIS server for each administrator requiring usage reports. This server can be installed on the administrator’s workstation since it uses very little resource. All database requests are sent to the database server where they are processed and the results returned. Radia Knowledge Base Manager Usage Manager SQL Database Purge Purging of old records from the database should be performed daily to ensure that report performance is acceptable. Usage data is automatically summarized from daily, to monthly and yearly records, to enable the administrator to choose which interval record types to use when generating reports. Typically, monthly records are chosen, which minimally speeds up the report generation process by 31 times over using daily data. Purging can be configured as follows: If reporting on daily usage is needed, you should keep a maximum of 32 days of data since daily data is automatically summarized in monthly records. Once deleted, daily data is no longer available for reporting. If reporting on daily data is not needed, you should keep zero (0) days of daily data. Keep a minimum of 12 months of monthly data. A good range is 12 – 18 months. Keep a minimum of 2 years of yearly data. A good range is 2 - 3 years. Location Server The Knowledge Base Manager can be installed on the same device as the RIS Collection File Destination Server or on another device that is on the same LAN segment. The LAN segment should have high bandwidth and a direct network route. Logging Logging is appended to a new log for each task on a monthly basis. Ensure that there is sufficient disk space to hold the monthly logs (minimum 40 MB). Radia Usage Manager Page 3 2/12/2016 Use the ‘Errors/Other’ log level to log errors and ignore logging of successful import messages. SQL Database Servers Co-location of Other Radia Databases Unless a small number of client devices ( <1000) are being monitored for usage, it is strongly recommended that the Usage Manager databases not share a common database with other Radia databases such as the Configuration Analyzer or Inventory Manager. The Usage Manager database reporting requires significant resources during report generation. Contention for these resources should be minimized. It is preferable that the database server device has a minimum of 2 processors and a minimum of 4 GB of memory. A server of this size should support approximately 5,000 – 10,000 devices given that sufficient collection filtering has been applied. Databases servicing large numbers of devices must have sufficient resources to support reporting service level expectations. Insufficient memory is a leading cause of poorly performing report query times. Disk Allocation Database servers are highly configurable and typically require tuning after initial data loads have been imported. However, it is important to accurately estimate the size and locations of disk storage prior to creating the database to minimize the cost of automatic extensions performed by the database as its size increases. SQL Server databases typically have larger average record sizes for the same data, requiring more disk space than Oracle databases servicing the same number of devices. Our experience has seen the following disk space needs for the base tables with no filtering of operating system files and no purging of daily usage records: o SQL Server - 6,000 devices required approximately 3.5GB disk storage, excluding log file needs which are dependent on database backup frequency. Expect to use approximately 5GB or more of log space. o Oracle – 7, 000 devices required approximately 3.5GB disk storage. Expect to use approximately 5GB or more of log space. If materialized views are used to improve report performance, significant increases in disk storage will be required. Our experience has seen the following disk space needs for base table plus materialized view table storage: o SQL Server – 6,000 devices required approximately 15GB disk storage o Oracle – 7,000 devices required approximately 15GB disk storage Temporary database space requirements may become quite large during the view materialization process if insufficient sort area sizes are defined. Our experience has seen an Oracle server require nearly 8GB of TEMPDB storage for 7,000 devices using a sort area size of 300K. To allow for growth, it is recommended to define your database as follows: o SQL Server For 5,000 – 10,000 devices define a data file (.MDF) file of 20GB and a log file (.LDF) of 5GB. This is defined using the SQL Server Enterprise Manager. o Oracle For 5,000 – 10,000 devices define your data tablespace file with 15GB and your index tablespace as 10GB. Edit the tablespace creation script shipped on the Radia Usage Manager CD/ROM. Place the data, log, index, etc… files on separate devices with separate disk controllers, if possible. Database Backups You must frequently back up your database. It is recommended that full weekly and daily differential backups are taken. Radia Usage Manager Page 4 2/12/2016 Apply all Maintenance Ensure that all Radia Usage Manager and database server service packs and maintenance have been applied. You must also ensure that the database tables and indexes are analyzed on a periodic basis to ensure that query plans are optimized by the cost based optimizers used by the database servers. Improving Reporting Performance - View Materialization View materialization adds de-normalized data tables and related indexes to your Radia Usage Manager database. These tables are created from the primary views associated with retrieving data from the normalized real-time tables. This process results in large, static data tables that contain de-normalized data. Creating materialized tables does not preclude reporting against real-time data in the Usage Manager database tables, but is an alternative reporting capability De-normalized data table access allows for significantly faster reporting using materialized tables versus using the real-time table data. Note that running this script may require large amounts of additional disk storage space for your database (i.e. 10GB or more for a 6000 device SQL Server database). Oracle Considerations Oracle database servers are highly configurable and typically require tuning after initial data loads have been imported. Oracle database configuration should be set for a large database. Here is a sample set of configuration parameters for Oracle 8i defined in the INIT.ORA file. db_file_multiblock_read_count = 32 db_block_buffers = 80000 log_buffer = 163840 : rollback_segments = (use Usage Manager schema defined roll back segments) db_block_size = 8192 sort_area_size = 52428800 sort_area_retained_size = 307200 For Oracle 9i or above, set a PGA_AGGREGATE_TOTAL value to a minimum of 1000MB. After a reasonable number of devices have been imported and before running reports, run the Oracle ANALYZE command against all of the Usage Manager tables and indexes to ensure that Oracle has current statistics and can make appropriate decisions during SQL execution plan binding. Radia Usage Manager Page 5 2/12/2016 Radia Usage Manager Page 6 2/12/2016