Technical white paper Best practice guide for HP Data Protector Protecting your most valuable asset: information HP Data Protector is the first data backup solution in the entire industry to provide a uniform deduplication solution from the source to the target, and protect mission-critical data in virtual and physical environments. More than 45,000 customers worldwide trust their HP Data Protector solution to protect their most valuable asset—information. Summary This white paper provides recommendations for the implementation of a backup and recovery strategy for HP Data Protector that allows you to optimize the way you use Data Protector in each IT environment. The guide concludes with references to other sources of information related to Data Protector and a glossary of important terms. Also included are detailed analyses of the configuration of the software, performance of the backup environment and the connected devices, settings in daily backup operations, as well as recommendations for practical applications of the solution. The first section of the guide summarizes the important steps of a successful HP Data Protector implementation. The second section explains the steps, where necessary, in more detail and illustrates them with the corresponding examples. This document is aimed at all users of HP Data Protector, system integrators and solution architects, as well as those who would like to optimize the way they use the software. Table of contents HP Data Protector Overview................................................................................................................................................... 3 Steps for Successful Implementation.................................................................................................................................. 3 What data should be backed up?........................................................................................................................................... 6 How, where and with the deployment of what technology should the backup be created?..................................... 7 Loss of Data / Downtimes....................................................................................................................................................... 7 Retention Periods..................................................................................................................................................................... 8 Licenses...................................................................................................................................................................................... 8 Data Volume, Data Growth, Sizing and Backup Times...................................................................................................... 9 Requirements and Recommendations for HP Data Protector Cell Server.................................................................. 10 Which DP components are required and what is their purpose?.................................................................................. 10 What is patch bundle and what order is required for the updates?.............................................................................. 12 How is the DP software installed on the server that requires a backup?.................................................................... 13 File ‘global’............................................................................................................................................................................... 13 File omnirc / .omnirc.............................................................................................................................................................. 14 Block Size................................................................................................................................................................................. 16 Segment Size........................................................................................................................................................................... 17 Disk Agent Buffers.................................................................................................................................................................. 18 Concurrency / Multiplexing................................................................................................................................................... 18 Multi-streaming...................................................................................................................................................................... 19 Null Device...............................................................................................................................................................................20 LAN vs. SAN Backups.............................................................................................................................................................21 Tools..........................................................................................................................................................................................22 Jukebox.................................................................................................................................................................................... 24 File Library............................................................................................................................................................................... 24 B2D – StoreOnce Software Deduplication.........................................................................................................................25 Evaluations in HP Data Protector........................................................................................................................................25 Notifications in HP Data Protector.......................................................................................................................................26 Connectivity to Monitoring Solutions.................................................................................................................................26 External Tools.........................................................................................................................................................................26 Purge.........................................................................................................................................................................................27 Recovery Tests and Disaster Recovery..............................................................................................................................28 Glossary....................................................................................................................................................................................29 Appendix A – Speeds.............................................................................................................................................................. 31 Additional Sources of Information...................................................................................................................................... 32 2 Technical white paper | Best practice guide for HP Data Protector HP Data Protector overview The diagram below describes the concept and functions of HP Data Protector, showing both the data streams and control paths of the software. Refer to the glossary at the end of the guide for explanations about the terms used in the diagram. GUI Cell Managers Manager of Managers Internal Database Files, Versions, Atributes Media, Devices, Libraries LAN Backup to Tape Application-Agent (Online-Agent) Backup-Agent (Disk-Agent) Media-Agent Application Control Data Backup to Virtual Tape / Catalyst Store User Zero Downtime Backup and Instant Recovery Backup to Disk HP Data Protector architecture Steps for a successful implementation A successful implementation of a backup and recovery strategy requires you to consider many factors. The points listed below are based on experience from a practical application, collected from many customers (SMBs and corporate groups), examining major aspects from planning to regular operation. The steps are divided into logical groups, so you can easily use this document as a reference for your own implementation of HP Data Protector (referred to in this guide as DP). Note: The details specified apply mostly to a Microsoft Windows environment, but are applicable to other settings and approaches, and can be used as a template for further improvement in your own environment. Backup and recovery strategy A stable and functioning backup and recovery environment is a critical part of a high-performing organization. With a solid data backup system in place, when loss of data occurs, it can be restored within a previously defined timeframe regardless of the cause: inadvertently or deliberately deleted, corrupted due to errors in applications, a hacker attack, data lost due to hardware errors, or other reasons. Another aspect of backup and recovery is the necessity to restore historical data, for example due to legally prescribed requirements. Regular backups are a simple form of disaster preparedness. Backups alone, however, are not a disaster recovery plan. The infrastructure of your organization also has to be examined. For instance, consider the following: • Do you have an alternative data center? • Are critical applications configured in a cluster? • Is the network connection configured as redundant? 3 Technical white paper | Best practice guide for HP Data Protector As part of a solid backup and recovery program, it is recommended that you integrate your company strategy with a disaster recovery plan that support the necessary backup and recovery requirements, availabilities, storage, and hardware redundancies that meets your unique needs. When considering your backup and recovery strategy and as you perform a detailed analysis of your infrastructure, consider the following points should therefore be examined intensively: • What data should be backed up? • How, where, and using what technology should the backup be created? • How much data loss is tolerable (RPO) and what are your acceptable downtimes (RTO)? • What are your data retention periods? Planning and preparation Ensuring the optimal implementation of a Data Protector environment requires detailed planning and preparation. The planning starts with the selection of the operating system for the installation of HP Data Protector Cell Manager. The Cell Manager is available for Linux, HP-UX, and Microsoft Windows. The 64-bit variant of each operating system is preferred, because it enables you to use the advanced options of HP Data Protector. In the case of Microsoft, MS Windows Server 2012 or 2012R2 is preferred. The DP Cell Servers can also be installed in a cluster environment, making it faster to entrust the protection of mission-critical data to a redundantly configured backup solution. Other aspects to consider in the planning phase: • Licenses • Data volume, data growth, scope of the infrastructure, backup times (backup window) • Requirements and recommendations for the Cell Server Installation and update Once the preparatory planning work for the backup and recovery environment has been completed, you can install the Cell Server. The process for installation is automated and can be completed in a few minutes. Sign in here to get support: https://softwaresupport.hp.com/group/softwaresupport/home A few questions that may arise during and after the installation: • Which DP components are required and what is their purpose? • What is a patch bundle and in what order do you load the updates? • How do you install the DP software on the server that requires a backup? Configuration Once the installation of HP Data Protector is completed, the solution is now ready, in principle, for deployment. Drives can be set up, pools can be defined, media can be formatted, backups can be set up and executed. But there are a few areas that still require configuration, as follows: HP Data Protector is installed with the so-called default settings, which are valid for virtually any environment without any further adjustment. Nonetheless, in practice, it is often advisable to change the Data Protector configuration files to your own environment, as these can, for example, contain limitations with the corresponding individual customization, or significantly influence the performance of the system. While descriptions of all the parameters are not included in this guide, you can refer to detailed descriptions in the configuration files and help files and/or the HP Data Protector manuals. The following paragraphs will provide a few recommendations from practical application. The customisable files are ‘globa’l and ‘omnirc’ (Linux / Unix: .omnirc): • global: The parameters in this file affect the entire environment managed by this Data Protector Cell Manager. For example, a change made to the timeout setting here will apply it to the entire environment managed under this Cell Manager. Important to remember, and in some cases necessary, parameters are described in the second section of the guide. • omnirc: The options in this file only affect the individual client for which backup is configured. In general, it is recommended only to change the settings in this file if required by the environment. The second section of the guide explains why customization is necessary for most environments and which parameters can be configured. 4 Technical white paper | Best practice guide for HP Data Protector Performance Performance tests should be considered part of the installation process for every Data Protector implementation. They document the status of the environment to be backed up and reveal possible bottlenecks in the infrastructure. The documentation of the conditions and results of the performance tests enables a comparison in the event of later repetition, allowing conclusions to be drawn regarding the performance of the infrastructure. Performance tests also enable optimal adjustment of the backup and recovery implementation to the circumstances of the environment, because theory and practice do not always match up 100%. The following points are relevant to the execution of the performance tests. The combination of all the functions described allow you to enhance the overall performance of your backups: • Block size: All drives (physical and virtual tape drives, as well as backup-to-disk drives) are set up in HP Data Protector with a default block size. For DP versions 7.X, the default value is 64 kB, and for version 8.X the default value is 256 kB. As a rule, increasing the size to 256 kB (DP versions < 8.X or for HP-UX) or 512 kB enhances the data throughput to the connected devices. • Segment size: When setting up new devices in HP Data Protector, a default segment size is specified depending on the media type used. As a rule, increasing the segment size also increases the data throughput and also the speed of the data backup. Depending on the media type that is used, the size should be increased gradually. • Disk agent buffers: Disk agent buffers have no direct influence on the overall performance of the backup, but in the case of network backups they can lead to a constant data flow on the media agent. The value 16 for this parameter has proven effective in practice. • Concurrency/multiplexing: On setting up the drives, a default concurrency is specified depending on the media type that is used; this value is often too low for the environment. Raising it to 10, for example, can considerably increase the backup speed and reduce the total duration of the backup. As a rule, a higher concurrency ensures better supply of the tape drive with data to be backed up. • Multi-streaming: Although the concurrency can be set when creating a backup specification, only one disk agent per mount point is started during the backup. In the case of very comprehensive mount points, this can lead to a bottleneck during the data backup. The individual configuration of multi-streaming can significantly enhance the performance of the backup for the individual mount points. • Null device: The null device describes a special drive type through which the maximum speed of a backup can be determined. Furthermore, the null device provides the possibility to test the settings for drives and identify possible bottlenecks in the environment. • LAN vs. SAN backups: The SAN backup is preferred, and allows you to make effective use of tape drive resources. High bandwidths are required for backups via network (key word: trunk or 10 GbE) to be able to take a tape device to the saturation limit. • Tools: Various tools are available for performance analysis, for example: HPReadData, HPCreateData and, as a complete package, HP LTT (Library and Tape Tools). Backup-to-disk technologies Disk-based backup targets (backup-to-disk) offer fast and reliable ways to recover data. HP Data Protector offers a continuum of backup target options that can be implemented as part of a recovery chain. Those options are: Data Protector Smart Cache, traditional disk-based backup appliances (example StoreOnce and Data Domain appliances), tape, and cloud. The following backup-to-disk technologies are available: • Jukebox • File Library • B2D – StoreOnce Software Deduplication • Data Protector Smart Cache Documentation and reporting Once the backup and recovery environment is fully installed and the backups are running in daily operation, you might think that the job is done. But it isn’t. An essential point for the implementation is still missing: documentation of the environment. Good documentation of the environment is a requirement for successful recovery in the event of a disaster. The documentation should contain at least the server names, IP addresses, mount points and directories to be backed up, priorities and order for the recovery (example: 1. MS Active Directory, 2. DNS, 3. MS Exchange), maximum recovery time, maximum loss of data, and the data volumes to be backed up. 5 Technical white paper | Best practice guide for HP Data Protector Optionally, reporting can be used to record and document the data growth of the backups. A schematic diagram of the environment helps other employees in the company to obtain quick insight into the environment. An emergency handbook might include, among other things, the description of processes and responsibilities, to help ensure efficient procedures in the event of a disaster. Other possibilities for documentation include: • Creating reports in HP Data Protector • Creating notifications in HP Data Protector • Connectivity to monitoring solutions, which helps to detect and document problems at an early stage • Documentation of the environment with external tools • The use of HP Backup Navigator, an interactive web-based reporting and analytics application for the entire Data Protector environment Maintenance With successful operation of the backup and recovery solution and with steady growth of the environment, the Data Protector installation can be faced with new challenges. However, in many cases the data center environment changes unpredictably as additional workloads are added or data grows faster than originally expected, which can result in insufficient backup windows to complete the backup task. In these cases, regular maintenance dates should be scheduled. Depending on the size of the environment, maintenance dates should set quarterly to twice a year. The objective of the maintenance is to proactively assess the state of the backup environment and fine-tune Data Protector parameters for optimal backup and recovery performance. The following activities should be included in a maintenance date: • Purge: A purge of the internal database helps to remove entries that are no longer required and ultimately ensures consistent good performance of the backup environment. This task is performed automatically with HP Data Protector version 8.X. For HP Data Protector versions 6 and 7, the purge is a necessary task to clean up and defragment the internal database. • Clean-up: The reality with any implementation is that the environment changes, new clients are installed, and servers are decommissioned. Often, clients that no longer exist are not removed from the Data Protector configuration. This is particularly noticeable, for example, when you want to create new SQL backups in the user interface of Data Protector, and results in the user interface running slowly. As a rule, the cause of this is a check of the name resolution for the backup clients being carried out in the background. This causes a timeout for systems that no longer exist and leads to significantly slower characteristics of the user interface. Regular detection and maintenance of the backup client is therefore advisable. • Patches: A maintenance date or times in which a no backup is carried out are good opportunities for the installation of new updates. This allows problems to be remedied and new functions to be enabled. A requirement for installation of the updates is a valid maintenance agreement. Furthermore, support packs for the server and updates of the firmware for the server and tape library can also be installed in this period. Also, the HP Library and Tape Tools are available to use to update the firmware of HP libraries. • Recovery tests and disaster recovery: Within the framework of maintenance, the recovery of files and servers should be verified at regular intervals to provide greater security and assurance in the event of serious incidents. • Documentation: Within the framework of maintenance, it is a good idea to verify and adapt the existing documentation of the backup environment. Moreover, if present, the emergency handbook can be revised and supplemented in the course of performing these other tasks. The BRICK tool can also be used to record the ACTUAL state of the Data Protector environment on the maintenance date and store it as documentation. What data should be backed up? The answer to this question depends on the requirements for the environment. In principle, everything should be backed up, including the system state. Only individual files or directories should be taken out of the backup by means of exclusion rules. In general, it applies that a backup must contain at least the data that are necessary for recovery of the application on the system. In the case of an MS SQL server, this involves at least a backup of the databases, and the transactions and the settings of the SQL server. The operating system could, for example, also be restored by rolling out a prepared operating system template with the deployment of virtual machines. 6 Making decisions about backing up data by an individual server is typically inadequate. In a company, there can be business processes that are mapped on a number of servers. For example, if the company operates a web service to be able to offer customers spare parts ordering, the web service does not consist only of the web server. As a rule, the Technical white paper | Best practice guide for HP Data Protector operation of the web service also requires connectivity to the internet, DNS, credit checks, possibly an Active Directory, possibly a file server, and a CRM solution. In case of a disaster, the entire service must therefore be examined and its data backed up and/or safeguarded against failures. Tip: In many cases, only the central location of the company is examined. A backup and recovery strategy for external offices should also be a constituent part of the definition. With HP Data Protector and the integrated deduplication technologies or the HP StoreOnce system, external offices can be easily included in the overall data protection strategy. How, where, and what technology should be used to create the backup? HP Data Protector provides a broad spectrum of options to safeguard mission-critical data against disaster. A few examples of the possibilities are provided below: • Integrations: The backup of applications and databases takes place by means of a special agent which ensures that applications are placed in a calm state (backup state) to enable application-consistent data backup. In the event of a disaster, the application can then be restored, including transactions. In the case of virtualized environments, the complete virtual machine, and with the Microsoft VSS framework also the application (for example: MS Exchange), can be backed up in the virtual machine by means of the integration. • Disk staging: This is a technology where the data are first of all cached on a hard disk area before they are copied onto a physical tape in a downstream process. With simultaneous integration of deduplication, a greater number of backups can be kept. Disk staging is suitable primarily for backups with a short retention period, as with intermediate storage and for rapid recovery in the event of a disaster. • Incremental forever: Incremental forever refers to a special process in which only the incremental changes to the data are backed up after an initial full backup. Here, the backup takes place partially or completely on a hard disk area, with regular consolidations of the data stocks, ensuring rapid recovery in the event of a disaster. • Deduplication: Deduplication refers to a method of detecting and removing redundancies on block level during the backup when the blocks to be backed up are compared with blocks that have already been backed up. This method is holistically integrated into the HP portfolio (HP StoreOnce systems and HP Data Protector) and permits, for example, complete integration of the external offices into the central backup and recovery strategy of a company. Tip: The combination of the different technologies in HP Data Protector provides comprehensive flexibility and optimal support for the company in the possible event of a disaster. Loss of data/downtimes The extent of loss of data and acceptable recovery times should be determined individually for each system or each service of the company. The type and frequency of backups in HP Data Protector can be derived from the data obtained. Tip: Please note that patching and upgrading Data Protector doesn’t create any downtime. Explanations of some key terms are provided below: • Recovery Point Objective (RPO) refers to the maximum loss of data in the event of a failure. As a rule, it is the time interval between two backups. This value specifies the maximum volume of transactions or data that can be lost between the last backup and a possible case of disaster. • Recovery Time Objective (RTO) refers to the agreed recovery time and, as a rule, is the period between occurrence of a disaster and recovery of the business processes. Possible additional activities can be: –– Hardware provision and recovery of the infrastructure –– Connection and configuration of the hardware on the network –– Recovery of the operating system –– Installation of the backup client software –– Identification of the data and information to be reloaded –– Identification of the required backup medium and its provision –– Data recovery from tape –– Recovery of databases –– Test and share of the reloaded data 7 Technical white paper | Best practice guide for HP Data Protector • The illustration shows an example of how a failure (case of disaster) can occur and what causes it: Disaster Physical Hardware (server) SAN switch Firewall Network Storage Data center (power/cooling) ... Logical Data deletion Data corruption Human error FW updates/patches Sabotage ... Tip: A list of all servers and services of the company with evaluation of the priorities, data volumes, backup frequencies and recovery times should be a constituent part of every disaster recovery plan. Retention periods The retention period refers to the period in which data must be available for recovery. With HP Data Protector, there are various possibilities for configuring the protection period of a backup, ranging from no protection period at all to days, weeks, months and years—all the way to permanent protection of the data. HP Data Protector is therefore also ideal for long-term data archival, though it should not be confused with an archiving solution. For example, the legal regulations in Germany based on the type of data (for example: documents with significance for tax, invoices, and annual accounts) and industry sector (for example: medical data) range from 6, 10, or 20 years up to permanent retention. Every company is under obligation to implement and comply with the regulations applicable to that company. Tip: To protect against inadvertent overwriting of the tapes (human error) or destruction of the data due to sabotage, it is recommended to copy the data onto physical tapes for long-term storage and store these at a suitable location outside of the company. Licenses Certain functions of HP Data Protector to be used in the planning phase are specified. On the basis of these considerations, it is easy to determine the required number of functionalities and licenses. The HP Storage Sizer tool is available to provide assistance. The Storage Sizer tool contains the HP Data Protector software Sizer, which can be called up separately. 8 Technical white paper | Best practice guide for HP Data Protector This application can be used to plan your backup and recovery project. The Data Protector environment can be licensed according to the traditional model (one license for each functionality) or the capacity-based model. The tool is available to partners of HP. Tip: If you do not know at the start of a project which functions of HP Data Protector are to be used, all the functions of HP Data Protector can be tested free of charge for a period of 60 days, within the framework of an evaluation. The software can be downloaded at http://www.hp.com/go/dataprotector. Data volume, data growth, sizing and backup times To determine which hardware (servers, SAN, network, tape libraries, HP StoreOnce systems, etc.) are required for the backup and recovery environment, it is important to know the size of the environment. The existing data volume, the data growth per year, and the backup window all influence selection of the hardware to be deployed. The following table helps determine the data volumes to be backed up. Data Types Backup To Disk (daily) Backup to Tape (weekly) Infrastructure 500 GB 2,500 GB Management 1,000 GB 5,000 GB B&R environment 500 GB 2,500 GB File Server 3,000 GB 15,000 GB Mail Databases 5,000 GB 25,000 GB Databases 2,000 GB 10,000 GB Total 12,000 GB 60,000 GB If you know the rate of data growth, the total backup volume can be calculated on the basis of the calculated data volume. The table below clarifies that when the project is viewed over a period of five years, you will want to double the data volume needs to be backed up at the end of the period. To determine the right values for the future, any examination or analysis should take account of this during planning of the backup and recovery environment. Growth Yearly 20% Backup to Tape (monthly) in TB 2014 240 2015 288 2016 346 2017 416 2018 500 Note: The tables are examples and must be viewed individually depending on the unique requirements of the organization. Backups are rarely carried out in the backup environment for 24 hours without interruption, because during the day there is often the need to give priority to production applications. Backups therefore usually take place outside of business hours, in a so-called backup time window. To determine the required hardware, it is essential to know how much time is available for the backup and the volume of data to be backed up within this time. For example, if the time is not sufficient for backup onto a tape device, the solution must be planned as larger or alternatives such as the snapshot-based backup on memory level (Zero Downtime Backup) must be considered. The HP Storage Sizer provides support in determining the required hardware. It plans the right libraries and appropriate StoreOnce system. The Sizer tool is available to partners of HP. Tip: For successful implementation of a backup and recovery environment, an examination of the data in external offices is just as important as information regarding the data in the headquarters. For example, if the data of the external office are to be replicated as deduplicated in the headquarters, the necessary requirements with regard to required capacities for StoreOnce systems and libraries must be met in the headquarters right at the start of the project. 9 Technical white paper | Best practice guide for HP Data Protector Requirements and recommendations for the HP Data Protector Cell Server The hardware requirements for HP Data Protector are described in the Quickspecs (link at the end of the document). In addition to the main memory that the operating system requires, HP Data Protector requires 256 MB main memory (512 MB recommended) for the Cell Manager. For backup jobs running in parallel, another 40 MB are required (assuming 10 backups running simultaneously, this means 400 MB), 64 MB for the installation server, 256 MB for the user interface and 64 MB (128 MB recommended) for the media agent. In total and rounded up, this totals a requirement of 1.5 GB main memory. Initially, there is no requirement in the Quickspecs regarding the number of processors. However, if the Cell Server is also to function simultaneously as media agent, the following general formula applies: 1 processor core per media agent/tape device to be addressed. If the StoreOnce software store in HP Data Protector is used, the recommendation is to deploy another 4 to 6 processor cores and 6 GB main memory (exact formula can be found in the Quickspecs). As a general minimum recommendation and as preparation for any future functions in the software, the backup server should have at least 8 processor cores and at least 8 GB main memory. Experience has shown that upgrading an existing server rarely makes good sense; instead, acquisition of the current HP ProLiant server model in each case is recommended. Tip: During planning of the backup server, attention should be paid to ensuring that the server is procured with more computing performance and main memory than required. This allows, within the framework of the lifecycle, new functions in HP Data Protector that have higher hardware requirements can be integrated (for example: integration of the StoreOnce software store). Which DP components are required and what is their purpose? On installation of the Cell Server, you are asked which components are to be installed on the system. As a rule, the proposed selection is the right one for the Cell Server, supplemented by the Disaster Recovery component. This applies to HP Data Protector versions 7.X and 8.X. Note: Before installation of the corresponding components, the support matrices must be checked with regard to compatibility of the versions. 10 Technical white paper | Best practice guide for HP Data Protector Available components and their significance (depending on deployed version, the selection can differ): • Disk Agent: Enables backup of the file systems of the backup clients. • (General) Media Agent: Enables access and data transfer to tape devices and tape libraries, as well as data transfer to backup-to-disk devices. • NDMP Media Agent: Supplements the general media agent to include functions for the backups of NDMP storage solutions. • User Interface: Contains the user interface of HP Data Protector as well as the commands to be run on the command line. • Manager-of-Manager User Interface: A special extension of the user interface required for management of a number of interconnected Cell Servers. • Java User Interface – see User Interface, Java-based: This special user interface is no longer available for DP version 8.X or higher. • HP P9000 XP Agent: Agent for snapshot backups of file systems of the HP P9000 (=Zero Downtime Backup). • HP P6000 EVA SMI-S Agent: Agent based on SMI-S standard for snapshot backups of file systems of the HP P6000 (=Zero Downtime Backup). • HP P4000 Agent: Agent for snapshot backups of file systems of the HP P4000 (HP StoreVirtual) (= Zero Downtime Backup). • HP P10000 Agent: Agent for snapshot backups of file systems of the HP P10000 (HP 3PAR StoreServ) (=Zero Downtime Backup). • MS Exchange Integration: Enables backup of MS Exchange 2003 Stores. • MS SQL Integration: Enables backup of MS SQL servers via the ‘Virtual Device Interface’. • MS SharePoint Integration: Enables backup of MS SharePoint servers (<version 2007). • MS SharePoint Server 2007/2010 Integration: Enables backup of the MS SharePoint farm. • MS Exchange Server 2010 Integration/2010: Enables backup of the databases of Microsoft Exchange 2010 and 2013 servers. • SAP R/3 Integration: Agent for backup of SAP (Oracle) databases with BRtools (backint and rman). • SAP DB Integration: Agent for backing up the MAXDB/SAPDB database. • SAP HANA Integration: Agent for backing up SAP HANA in-memory database • Oracle Integration: Agent for backing up Oracle databases. • Sybase Integration: Agent for backing up Sybase databases. • Informix Integration: Agent for backing up Informix databases. • Lotus Integration: Enables backup of Lotus Notes databases (NSF). • DB2 Integration: Agent for backing up DB2 databases. • HP NNM Backup Integration: Scripts for backup of an HP Network Node Manager implementation. • MS Volume Shadow Copy Integration: Enables backup of MS applications and file systems via the Microsoft VSS framework (for example: MS Exchange 2007). 11 Technical white paper | Best practice guide for HP Data Protector • VMware Integration (legacy): Integration for backing up virtual machines < ESX 5. • EMC Symetrix Agent: Agent for snapshot backups of file systems of an EMC Symetrix storage solution. • Automatic Disaster Recovery Module: Module for backup of the system states of a server as preparation for the recovery of a system with the Enhanced Automated Disaster Recovery method. • Documentation: Contains the complete documentation and help system in English, French, Japanese, or simple Chinese. The language integrations vary in the currently supported HP Data Protector versions. • VLS Automigration: In the case of a library connected directly to the HP VLS system, enables automatic migration (copy job) of the virtual tapes onto physical media. • Virtual Environment Integration: Enables backups of virtual machines for VMware and Hyper-V (XEN via scripts). • StoreOnce Software Deduplication: Enables the integration of a StoreOnce software store in HP Data Protector. • MS Exchange Granular Recovery Extension: Agent for the recovery of individual objects from a Microsoft Exchange backup (for example: mail and calendar objects – only for 2010/2013). • MS SharePoint Granular Recovery Extension: Agent for the recovery of individual objects from a SharePoint backup (for example: Sites – only for 2007/2010). • VMware Granular Recovery Extension Agent: Agent for the recovery of individual objects from a VMware backup (for example: a file in a virtual machine • VMware Granular Recovery Extension Web Plug-In: Plug-in for integration of the VMware Granular Recovery Extension into the VMware vCenter. Tip: Only the required components are installed on the Cell Server. These include the Media Agent, Disk Agent, User Interface, Documentation, Installation Server, and Automatic Disaster Recovery Module (Microsoft Windows). Attention should be paid to ensuring that the Cell Server is only used for backup and recovery tasks. The installation of other applications on the backup server endangers the calculated or specified backup and recovery times. What is a patch bundle and what order is required for updates? No software is free of errors, which is why updates for HP Data Protector are made available at regular intervals to rectify known problems or integrate new functions into the software, for example support for a new operating system. Note: A valid maintenance contract is required to download the updates from the HP website and their installation. A patch bundle is a collection of different related updates. The installation of the patch bundle is automated. Under certain circumstances, individual updates that are released later require the installation of a patch bundle. It is therefore recommended that you read the corresponding patch description before installation. Depending on availability, individual updates must always be installed in the following order: • Core – The most important constituent parts of HP Data Protector • Cell Server – All components relevant to the Cell Server • Cell Console – Constituent parts of the user interface • Disk Agent – Enables backup of the file systems of the backup clients • Media Agent – Enables access and data transfer to tape devices and tape libraries, as well as data transfer to backupto-disk devices • Documentation – Contains the complete documentation and help system in English, French, Japanese or Simplified Chinese. The language integrations vary in the currently supported HP Data Protector versions. • Integrations – All integrations (Microsoft Exchange, VE, Oracle, SQL, etc.) Tip: The updates should be installed directly after installation of the Cell Server and before the clients to be backed up are installed. When downloading and saving the updates, it helps to extend the file name with a description (for example: DA or MA for disk agent or media agent). 12 Technical white paper | Best practice guide for HP Data Protector How is HP Data Protector software installed on the server that requires a backup? Tip: A check of the name resolution for the server to be set up should be carried out before installation of the software. A successful so-called ‘forward’ and ‘reverse’ DNS lookup for the backup client and backup server must be possible. This prevents later errors in the backup and recovery environment. HP Data Protector software is installed on the backup clients via its user interface. For some systems, the installation is carried out per script (Novell NetWare) or manually and locally (for example: Microsoft Cluster and OpenVMS). Under certain circumstances, it can be necessary for distribution of the software on MS Windows 2008 Servers using the command omniinetpasswd to add a user with administrative rights on the Cell Server (see help with the error message ‘Cannot connect to SCM – Service Control Manager’). Further preparatory steps are necessary for the installation of Linux or Unix clients via the user interface of HP Data Protector such as: • Setting up a Linux or Unix installation server • Adaptation of the file .omnirc on the installation server • Creation of a public key and distribution of the key on the Linux or Unix servers to be backed up in the file authorized_ keys • Setting up an SSH connection to the server to be backed up to accept the host key Once these steps have been carried out, the Linux and Unix servers can also be installed via the Data Protector GUI. Tip: The backup clients should only be installed once all the necessary updates have been installed on the Cell Server and/or installation server. This accelerates the installation of the software on the clients. A client receives only the required components. As a rule, the disk agent and the automatic disaster recovery module are sufficient for backup of the file systems. The integration agents are only installed if backups of applications such as Microsoft Exchange or Oracle are desired. If an integration agent is installed but not used, Data Protector still requests a license. If no more licenses are available, a missing license is indicated in the list of licenses of Data Protector. File ‘global’ The file ‘global’ contains important configuration parameters for the backup environment and has effects on the characteristics of the entire cell. Storage location of the file on the Cell Server is the directory Data_Protector_program_data\Config\server\options or Data_Protector_home\Config\server\options for Windows systems and /etc/opt/omni/server/options/ for Linux and Unix systems. Following any changes to the file, the services have to be restarted on the Cell Server. Tip: With the introduction of version 8.X of HP Data Protector, adaptation of the file ‘global’ is possible via the HP Data Protector user interface. Note: Changes should only be made to the file ‘global’ after careful planning and after reading the parameter description in the file. The values for the individual parameters can deviate from the suggestions in each customer situation and must be regarded separately for each environment in all cases. The parameters that have often proved to be effective in practice are listed below in no particular order. They were regarded as expedient as a result of experience gained in different installations. • MaxBSessions=25 – Increases the default limitation of 5 possible backups running simultaneously to 25. This limitation was already raised to the value 100 with HP Data Protector 8.X. • SchedulerGranularity=1 – Enables starting of jobs every minute in HP Data Protector (instead of the default setting of every 15 minutes). • Ob2TapeStatistics=1 – This setting means that statistics regarding the use of the tapes are written in the file ‘media. log’. • DCDirAllocation=1 – If a number of DCBF directory are used (setting in the internal database), this parameter determines the even distribution according to site of new DCBF files. 13 Technical white paper | Best practice guide for HP Data Protector • FileMediumCapacity=100000 – Where file devices are used (null device media or jukebox media), the limitation for the medium is 100 MB. Adapting the value avoids, among other things, mount requests. • SmLogStartStop=1 – With this option, the Session Manager process logs the start and end times in the file sm.log. • AuditLogEnable=1 – This option is used to collect audit information. • AuditLogRetention=90 – This parameter specifies the number of months that the audit information is to be collected in the internal database. • SMTPSenderAddress=<e-mail sender> – This option specifies the sender address for sending email through HP Data Protector. • SMTPServer=<SMTP server> – Input of an SMTP server (mail gateway, mail relay, Microsoft Exchange server) enables HP Data Protector to send emails. • RecoveryIndexDir=FullPathToTheBackupDir – Specifies a backup directory for the file ‘obrindex.dat’ and contains information regarding the backup of the internal database, and the drive and medium used. It is useful as information in a disaster recovery scenario. • DailyMaintenanceTime=HH:MM – The default value is 12:00 midday. If backups are running at this point in time, the time for maintenance can be shifted accordingly. Specifying “None” or 99:99 disables this task. • DailyCheckTime=HH:MM – The default value is 12:30. If backups are running at this point in time, the time for the daily checks can be shifted accordingly. Specifying “None” disables this task. • SmDeviceErrorThreshold=5 – Specifies the number of unknown errors with a backup device before the device is automatically disable. The default value is 0. • SessionStatusWhenNoObjectToCopy=2 – Specifies in the case of a copy job where there were no objects to copy whether or not the session is shown as successful. The default value is 0 and terminates the backup with the status “Failed” and a critical error. • SessSuccessfulWhenNoObjectsBackedUp=1 – Specifies whether the backup is to be terminated as successful (1) or defective (0) if no objects could be backed up. This option is very helpful for transaction backups. • MountDelay=90 – Specifies how many minutes the backup with an active mount request waits for user interaction before the corresponding mount script is executed. The default value is 30 minutes. • SmWaitForDevice=180 – The parameter specifies how many minutes a backup is to wait for a free drive (‘Waiting for devices to get free’). When the time has elapsed, the backup is aborted. The default value is 60 minutes. • CopyAutomatedMaxObjects=2500 – Specifies how many objects are to be copied simultaneously in one copy job. This option is very helpful for transaction backups onto a hard disk area. The default value is 500 objects. This limitation was raised to the value 2500 with HP Data Protector 8.X. • FreePoolDeallocFreq=4 – The value specifies how many times a day the media pools are to be checked for free tapes in order to move them into a free pool. The default value 1 (=0:00); 4 means 00:00, 06:00, 12:00, 18:00. This value was raised to the value 4 with HP Data Protector 8.X. • MMFairLimit=95 – This value specifies the percentage the overwrite operations or number of valid months a medium is marked as ‘fair’. As default for LTO media, the number of overwrite operations is 250 and for the validity of the media 36 months. Tip: Changes to the file should be made directly after the installation, before the environment is deployed. When updates are loaded, it should be checked in the corresponding description of the patches whether variables have to be changed to achieve certain characteristics of the cell or whether they are no longer required. In the case of major versions of HP Data Protector, an adjustment and extension of the variables take place during the installation procedure. File omnirc/.omnirc The file ‘omnirc/.omnirc’ contains configuration parameters and has an effect on the characteristics of the backup client during backup or recovery. Depending on the operating system, changes in the file can have a positive effect on the performance of the backups and thus shorten the backup or recovery times. Storage location of the file on the client is the directory Data_Protector_program_data or Data_Protector_home for Microsoft Windows and /opt/omni/ for Linux and Unix systems (Mac, Novel,l and other Unix systems such as IBM-AIX differ). Changes to the file are used for the next backup or recovery operation; as a rule, it is not required to restart the services. 14 Technical white paper | Best practice guide for HP Data Protector As default, the file is not present on any of the systems in the cell; it must first be created from the template (omnirc. tmpl). The file should be created not only on the Cell Server, but also on each media agent and disk agent (client to be backed up). For Unix, the parameters recommend below can differ. Note: Changes should only be made to the file ‘omnirc’ after careful planning and after reading the parameter description in the file. The values for the individual parameters can deviate from the suggestions in each customer situation and must be regarded separately for each environment but do not have to be complete in all cases. The parameters that have often proved to be effective in practice are listed below in no particular order. They were regarded as faster, as a result of experience gained in different installations. Settings for the media agent: • OB2DEVMUTEXTIMEOUT=12000 – Deals with timeouts (2 minutes) for mutex locks (parallel access of several process, for example, on the robotics); as default, the parameter is not available. • OB2IPCNOSENDRECVLOOP=1 – The variable was introduced with DP 6.2X and it rectifies a problem in the case of slow backups on a Windows 2008 media agent. As default, the parameter is not available. • OB2COMPRESSIONRETRY=10 – Specifies how many attempts are to be made to determine the compression capability of the tape device. The default value is 1. • OB2AUTODDTIMEOUT=300 – The parameter specifies the maximum number of seconds are required to search backup devices in SAN environments. The default value is 600 seconds. • OB2FILELIBDISKFULL=1 – With use of a number of drives of a file library that write into the same directory, the specified memory space in MB is kept available in the event of a full hard disk. The default value is 0 and the hard disk runs full. • OB2NOREPACKAGEWARNING=1 – Deals with a problem in the case of copy jobs of devices with a small block size on devices with greater block size. It was introduced in DP 6.0 and as default is not available in version 7. • OB2MACHECKCAPACITY=1 – With this variable, HP Data Protector attempts to determine the possible capacity of LTO devices. The default value is 0. Settings for the disk agent or installation server: • OB2SHMIPC=0 – The value 0 means that the Media Agent, Disk Agent and Cell Manager establish the connections via the shared memory area. As default, the parameter is not available. • OB2IPCKEEPALIVE=1 – The value means that IPC connections remain open permanently instead of opening temporarily and closing. As default, the parameter is not available. • OB2SKIPDIRECTORY=1 – Permits the skipping of directories during the backup instead of only individual files. • OB2NOREMOTEWARNINGS=1 – This parameter is required under Linux and Unix; it prevents an error message if a file system that contains a mount point onto a local file system or a file system connected per NFS is to be backed up. • OB2_SSH_ENABLED=1 – The variable is required for a Linux or Unix installation server and specifies that the installation server is to run the remote installation via the SSH protocol. As default, ‘rsh’ and ‘rexec’ are used. • OB2FWPASSTHRU=1 – The variable is required for a Windows installation server so that the bootstrap service can register with the Windows firewall. Under certain circumstances, the default value 0 means that the required ports have be added manually on the client. • OB2NOTREEWALK=1 – Under Windows, this prevents the first treewalk before the backup to be made (browsing the directories for data to be backed up) and accelerates the backup of servers with a large number of small files. Tip: Changes to the file should be made directly after the installation, before the environment is deployed. When updates are loaded, it should be checked in the corres¬ponding description of the patches whether variables have to be changed to achieve certain characteristics of the client or whether they are no longer required. In the case of major versions of HP Data Protector, there is no adjustment or extension of the variables during the installation procedure. Adaptations must be carried out manually after the installation. 15 Technical white paper | Best practice guide for HP Data Protector Block size Increasing the block size to, for example, 256 kB or 512 kB, can considerably increase the throughput of a data backup. During data backup, the data are not written continuously onto the tape, rather they are divided into seg¬ments. The segments here are not written onto a device in one operation, rather in smaller units as blocks. The default size of the blocks for HP Data Protector on installation is 64 kB for all device types. This value applies to DP 7.X; for DP 8.X, the default is 256 kB. In a test with a backup of 40 GB onto a null device, the performance with a block size of 512 kB was virtually 25% higher than with default values. The block size depends on the deployed operating system, the device used for data backup, and the adapter used (SCSI, SAS, FC HBA). When an iSCSI tape device is used in the Windows environment (virtual or physical), for example, the registry must be modified first to enable a larger block size. On using the ‘Enhanced Automated Disaster Recovery’ function, it is important to bear in mind that Windows is only able to use larger blocks when the drivers for the tape drives have been loaded. As this is not possible during the disaster recovery operation, because the connected devices have to be addressed with a block size of 64 K during the backup itself. This applies to the Cell Server and possibly also to the media agent. Therefore, there must be a check beforehand as to what size is supported by the environment, because if the selected block size is too large, HP Data Protector is unable to restore data from a device configured in this way. Also bear in mind that the block size must be changed before formatting the tapes, as the block size is written in the medium header of the tape so that HP Data Protector recognizes the block size to be used on loading the tape. If the block size on the tape and the device are different, error messages can occur. A block size of 512 kB is recommended for data backups on an HP StoreOnce system or with use of the HP StoreOnce software store within HP Data Protector. How is the block size changed? In the GUI of HP Data Protector, in the context list, switch to ‘Devices & Media’. Extend the devices and, if necessary, the libraries, then select a drive and the properties of the drive. In the ‘Settings’ tab, open ‘Advanced’. In the displayed dialogue, switch to ‘Tab Size’ and set the desired block size. Then confirm the changes with ‘OK’ and in the result area save the changes with ‘Apply’. A block size of 64 kB is recommended for backups of the data of the media agents (including Cell Server). However, in order to achieve better performance for the backup of the other servers to be backed up, another logical drive with a high block size can be created in HP Data Protector. A number of logical drives can be created on a physical drive; the logical drives differ here with regard to the names and selected properties, for example the block size. The device lock name, however, should be identical for the logical drives. 16 Technical white paper | Best practice guide for HP Data Protector Tip: A block size of at least 256 kB and (depending on the operating system) of 512 kB is recommended for the best performance and has proven effective in practice. If the ‘Enhanced Automated Disaster Recovery’ function is used for media agents, an additional logical drive with the default block size of 64 kB is to be used. Segment Size Increasing the segment size can have a decisive influence on the performance of the data backup and data recovery. A tape or medium is divided into segments: data segments, catalogue segments, and a header segment. The header information of the tape is stored in the header segment and has the same size as the block size. Data are stored in blocks within the data segments. Information regarding the data segments is stored in the relevant catalogue segment. The information regarding the data segments is initially stored in the memory of the media agent and subsequently written into the catalogue segment as well as the internal database. The optimal size of the segment depends on the media type used and the type of data to be backed up. A greater segment size can improve the backup speed. The information regarding segments is written into the catalogue segment, whereby the catalogue size is limited to 12 MB (*). In the case of backups of a large number of small files, under certain circumstances the segment can be completed sooner. * Changeable in omnirc file - OB2OMNIMAXCATALOG Note: Although a larger segment size can improve the backup speed, it can slow down the recovery speed, as the data blocks are found more quickly if the restore is run from a tape with a smaller segment size. How is the segment size changed? In the GUI of HP Data Protector, in the context list, switch to ‘Devices & Media’. Extend the devices and, if necessary, the libraries, then select a drive and the properties of the drive. In the ‘Settings’ tab, open ‘Advanced’. In the displayed dialogue, switch to the ‘Sizes’ tab and set the desired segment size. Then confirm the changes with ‘OK’ and in the result area save the changes with ‘Apply’. The default segment size for LTO drives is 2,000 MB. It is recommended to test the performance of the backups for LTO drives in each environment and to increase the size depending on the LTO type used. Tip: A segment size of 8,000 for LTO drives is a good start value for other performance tests and will also lead to an increase in the backup speed with acceptable performance for recoveries, if current LTO-6 drive technology is used. 17 Technical white paper | Best practice guide for HP Data Protector Disk agent buffers HP Data Protector stores the data in memory before transferring it. Depending on the concurrency, the memory is divided into buffers. Per default, each buffer consists of 8 disk agent buffers that are the same size as the block size. Increasing the number of disk agent buffers does not significantly enhance performance. However, if the network bandwidth in the case of backups via the network fluctuates greatly, an increase in the number of disk agent buffers can ensure that the media agent always receives sufficient data to operate the drive that is used in the streaming mode. A reduction in the number of disk agent buffers is necessary if the media agent does not have sufficient shared memory. How are the disk agent buffers changed? In the GUI of HP Data Protector, in the context list, switch to ‘Devices & Media’. Extend the devices and, if necessary, the libraries, then select a drive and the properties of the drive. In the ‘Settings’ tab, open ‘Advanced’. In the displayed dia¬logue, switch to the ‘Sizes’ tab and set the desired number of disk agent buffers. Then confirm the changes with ‘OK’ and in the result area save the changes with ‘Apply’. Tip: The value 16 for the number of disk agent buffers has proven effective in practice when carrying out network backups. Concurrency/multiplexing The optimal setting of the disk agent concurrency, in combination with the number of drives to be used for the backup, enables a considerable enhancement of the backup speed. The number of disk agents started per media agent is referred to as disk agent concurrency. The number to be used can be set in the properties of the drive and in the backup specification. A value in the backup specification overwrites the value set on the drive. HP Data Protector sets a default concurrency when setting up the drives depending on the drive type used. The default value for LTO drives is 4; if file libraries are used, the default value is 3. The maximum value that can be set is 32. Depending on the backup that is run, the value can be even lower. For example, the value 1 is specified in the backup specification for the backup of a Microsoft Exchange server via the network. For the backup of a large number of ‘slow’ servers in the network, adjustment of the concurrency is an advantage, because the total runtime backup can be greatly reduced and better use can be made of the available network bandwidth. However, keep in mind that a value that is set too high can lead to fluctuations in the speed of network backups. It is recommended to set the value in such a way that at least the drive can be operated in the streaming mode and only set high enough so that the drive that is used reaches saturation. The table in section ‘LAN vs. SAN Backups’ is the basis for this. Note: For the backup onto an HP StoreOnce system or if the HP StoreOnce software store within HP Data Protector is used, the concurrency is to be set to 1 to enable effective deduplication of the data flow. To enhance the backup speed, it is recommended to increase the number of drives for the backup. If a high concurrency is used during the backup, the restoration can be slower, as the data to be restored are fragmented on the tape. How is the disk agent concurrency changed? • Possibility 1 – per drive: In the GUI of HP Data Protector, in the context list, switch to ‘Devices & Media’. Extend the devices and, if necessary, the libraries, then select a drive and the properties of the drive. In the ‘Settings’ tab, open ‘Advanced’. In the displayed dialogue, switch to the ‘Sizes’ tab and set the desired concurrency. Then confirm the changes with ‘OK’ and in the result area save the changes with ‘Apply’. 18 Technical white paper | Best practice guide for HP Data Protector • Possibility 2 – per drive in the backup specification: In the GUI of HP Data Protector, in the context list, switch to ‘Backup’ (the same description applies to the jobs in the context object operations). Depending on the display, extend the specifications, select a backup specification and select the properties of the job. In the ‘Target’ tab, extend the library if necessary and open the properties of the drive. In the displayed dialogue, set the desired concurrency in the ‘General’ tab. Then confirm the changes with ‘OK’ and in the result area save the changes with ‘Apply’. The optimal value for the disk agent concurrency depends heavily on the type of data to be backed up, on the drive and media types, and the amount of servers/mount points to be backed up. A value that is set too high can influence the restoration speed, as the data on the tape are fragmented, which causes more rewind time for the tape. This negative effect of concurrency can be avoided by storing the data initially on staging devices (for example, on a file library) and subse¬quently copying them to tape using object copy. Tip: The test of the concurrency with a null device has proven effective in practice as a suitable test for the optimal setting. In the case of backups via the network and backups of several servers / mount points in one backup specification, the value 10 has proven to be a reasonable starting value. Multi-streaming A configuration of multi-streaming can increase the total data throughput rate for the configured mount point and thus contribute to reducing the total duration of the backup. As default, only one disk agent per mount point to be backed up can be started in HP Data Protector. Multi-streaming (multiple data streams) enables backing up of a number of parts of a mount point at the same time in that a number of disk agents are started for this mount point. This method is very useful when very large mount points have to be backed up and saturation for the backup device cannot be reached, or when the drive to be used is slower than the disks can deliver their data. 19 Technical white paper | Best practice guide for HP Data Protector Note: For a backup with activated multi-streaming and a number of disk agents, it should be borne in mind that the data are stored in a number of objects in the internal database and in the case of recovery all parts of the mount point have to be selected for the recovery. One way around this is to adapt the description of the mount point. Possible scenarios for the application of multi-streaming can be: • File servers with very large mount points and a numerous small files – the backup with a single drive is too slow. In this case, configuration of the backup specification enables individual parts of the mount point to be backed up by a number of disk agents. • Large mount points where the data are on very fast hard disks and where the backup to a single drive is too slow. In this case, configuration of the backup specification enables individual parts of the mount point to be backed up by a number of disk agents on a number of media agents. How is multi-streaming configured? In the GUI of HP Data Protector, in the context list, switch to ‘Backup’. Depending on the display, extend the specifications, select a backup specification and select the properties of the job. In the ‘Source’ tab, select the desired server, right-click on the server object, and click ‘Copy’. This operation displays another object. Now, the directories to be backed up can be selected for each mount point. The optimal configuration of multi-streaming depends heavily on the environment involved and should be tested accordingly. The speed of the hard disks on which the data are located and whether saturation of the drive can be reached are two factors to be taken into account for the configuration. As long as saturation has not yet been reached and the disk agents are still able to deliver sufficient data, an allocation between 2 to 4 disk agents per mount point in combination with the value of the disk agent concurrency and number of deployed drives is a very good setting. Tip: The configuration is expedient for mount points with a size of more than 500 GB backup volume and if the tape device does not reach saturation. Depending on the environment, this setting can halve the backup time. Null device Performance tests on a null device are a very good possibility to determine the read speed of a system and the optimal drive technology for the backup. In the case of a backup of data onto a null device, the media agent (and thus the drive) is bypassed and the received data rejected. This allows the speed of a backup via the network to be determined and tests different settings of the block size, segment size, concurrency etc. Note: Before using the null device, it is recommended to change the parameter FileMediumCapacity in the file ‘global’, as the default value is 100 (MB) and subse¬quently causes a mount request. An adequately large medium can be created with a value of 100,000. 20 Technical white paper | Best practice guide for HP Data Protector How is a null device configured? In the GUI of HP Data Protector, in the context list, switch to ‘Devices & Media’ and ‘Add Device’. Use ‘Add Device’ to set up a new drive manually with device type: Standalone. Enter SCSI address in the next tab - for Windows ‘nul’ and for Linux and Unix ‘/dev/null’ is to be entered (the screenshot shows both variants). The remaining settings (Block Size, Concurrency, etc.) will be specified in the next steps. At the end, the drive configuration is saved and the null device is available for tests. Tip: Use of the null device is suitable not only for determining the optimal settings for the drives, but also for troubleshooting, as neither media nor drive space are required. LAN vs. SAN Backups For the decision as to whether the data backup is to be carried out via the network or via SAN, the possible data transfer rate to tape device has to be examined first. LTO-5 drives reach a data transfer rate of up to 140 MB/s; LTO-6 drives reach a data transfer rate of up to 160 MB/s. If the compression possibilities for the drives are also taken into account in this case, a data transfer rate of 280 MB/s for LTO-5 and 400 MB/s for LTO-6 drives can be achieved with an assumed compression ratio of 2:1 for LTO-5 and 2.5:1 for LTO-6. As file system backups usually achieve lower compression ratios in comparison with backups from databases, it is recommended to test the data transfer rates in each environment. The table below shows data transfer rates for LTO-4, LTO-5 and LTO-6 drives on the basis of the compression possibilities, to provide help in reaching a decision as to how data backups and possible recoveries can be carried out effectively. LTO 4 (MB/s) LTO 5 (MB/s) LTO 6 (MB/s) Native speed 120 140 160 Compression 2.0:1 (default) 240 280 — Compression 2.5:1 (default) — — 400 Compression 1.5:1 (i.e. file system backup) 180 210 240 Compression 3.5:1 (i.e. SAP R/3) 420 490 560 21 Technical white paper | Best practice guide for HP Data Protector The compression ratio is an average value for the drive and the degree of compression fluctuates during the backup. In order be able to exploit the maximum data transfer rate of the tape device, the client to be backed up must deliver a higher throughput. As a rule, the calculation is with the factor of 2; for example, in the case of a backup of an SAP database to an LTO-5 drive and an average compression ratio of 3.5:1, a data throughput rate of up to 980 MB/s must be reached. If you now compare the possible data transfer rates in the network, data transfer rates between 80 MB/s and 105 MB/s are usually reached in the 1-GbE network, depending on the topology. It can be seen that for a backup via the network at least one trunk consisting of several network adapters must be created. For LTO-6 as drive technology, at least one 10-GbE network is necessary. If the backup still has to be carried out via the network, the data transfer rate should be kept constant. An unfavorable data transfer rate shows the following screen: Fig. 1 With adaptation of the settings (files ‘omnirc’ and ‘global’, Concurrency, Block Size, etc.), a significantly higher data transfer rate (Fig. 1) can be achieved. Fig. 2 In both cases, the backup was carried out via one trunk (2 GbE – fault tolerance). Fig. 1 shows an effective utilization of capacity of 40% in the 1-GbE network and Fig. 2 a utilization of capacity of 50% in the 2-GbE network. There are alternatives for environments in which the implementation of trunks, 10-GbE, or SAN backups are not possible. In virtual environments, the bottleneck in the network can be bypassed using the ‘Virtual Environment Integration’ of HP Data Protector. In this case, the VMFS volumes of VMware can be presented to a media agent and the backups of the virtual machines take place via the SAN. For slow systems in the SAN, the backup can be carried out via a faster system by means of snapshot backup (Zero Downtime Backup). Another alternative is a backup per ‘incremental forever’ or via ‘Source Side Deduplication’ to minimize the network load. Tip: SAN backups should always be preferred over LAN backups in order to be able to reach the maximum data transfer rate of a tape device. If this is technically not possible or the fibre channel infrastructure does not deliver the transfer capacity for it, a higher bandwidth is to be made available for the backup via network (trunks, 10 GbE). Tools HP provides tools free of charge for performance measure¬ments on virtually any platform. The “HP Library und Tape Tools” as a complete package are ideal for this (HP LTT - http://h18006.www1.hp.com/products/storageworks/ltt/index. html). The tools enable measurement of the drive speeds with adjustable values for compression ratio and size of the files to be emulated. Moreover, the read and write speed for disk systems can be determined, which provides a good starting point for the expected performance in the case of backups with backup-to-disk technologies. HPReadData and HPCreateData (integrated in LTT) are available as standalone versions. The type of data flow to be simulated can also be configured here. For reading with HPReadData, productive data can be accessed. 22 Technical white paper | Best practice guide for HP Data Protector Alternatively, the test can be carried out within HP Data Protector with the enabled option ‘Display statistical infor¬mation’ on a null device; the average data transfer rate is displayed at the end of the backup. The option is disabled again after the test. The tool ‘omnispeed’ can also be deployed (see http://www.data-protector.org/ wordpress/?s=omnispeed). How is HP LTT used? For a performance test of the hard disks (read speed of the disks), switch to ‘Sys Perf’ in LTT and set the desired options at ‘Backup Performance Test’. The description corresponds to a concurrency of 3 in HP Data Protector where a block size of 512 kB is used. The result can be seen at ‘System Performance Results’ at the end of the test. The figure at ‘TOTAL’ corresponds to the total data throughput rate of the system. The speed of a restoration can be simulated at the ‘Restore Performance Test’ tab. This test is also suitable for determining the disk speed when backup-to-disk technologies are used in Data Protector. 23 Technical white paper | Best practice guide for HP Data Protector The result is shown at ‘System Performance Results’. In the specific example, the system has a data transfer rate of only 36 MB/s and is not suitable as a long-term backup-to-disk solution. Tip: The installation of LTT is recommended for every server with connected tape devices. Before starting up the media agent on a server, this tool should be used to carry out a measurement and document the results. The individually available tools can be used for the measurement on backup-to-disk targets. Jukebox Jukebox is a possibility to deploy backup-to-disk technologies in HP Data Protector free of charge. Licensing is according to the number of drives to be used and there is no automated media management. Tip: Experience has shown that Jukebox can only be used in productive environments to a limited extent, as its restrictions and licensing by drives make it inflexible. Depending on the environment and existing licenses, it is unable to achieve the desired data throughput. File Library The File Library is a device that writes the data onto internal or external disks (SAN or NAS shares). It consists of a number of directories to be configured and a number of definable drives (or writers). The File Library provides automatic media management and is currently the most frequently deployed backup-to-disk solution in Data Protector. The virtually created media are called ‘file depots’ and can, if required, also be read as a configured slot in a jukebox. The File Library is licensed with the advanced backup-to-disk license in TB and enables the creation of synthetic and virtual full backups (‘incremental forever’). On creation of the File Library, the size of the file depot files should be adapted, as a large number of media are created in a backup when the default value of 5 GB per medium is used. Tip: To test the File Library functions, also for established customers, HP offers 30-day test licenses for all functions in Data Protector. 24 Technical white paper | Best practice guide for HP Data Protector B2D – StoreOnce Software Deduplication Deduplication has been possible with HP Data Protector since version 6.21 and it represents the HP strategy for backupto-disk solutions. The ‘advanced backup-to-disk’ license is required for deduplication. Only the available disk space is licensed (maximum 20 TB per store root on one server; if several servers are used, the individual store roots need to be added up) and not the actual size of the backups created. Note: If StoreOnce software deduplication is used in HP Data Protector, there are additional requirements with regard to the number of CPUs and the main memory. Deduplication is a special compression method where the volume of the data to be backed up is reduced by not saving duplicates. The data flow is divided into blocks (hash-based chunking) and these blocks are compared with one another. If identical blocks are found, one of them is replaced with a pointer to a block that has already been saved. StoreOnce deduplication is holistically integrated in HP Data Protector; the deduplicated data can be transferred from the source to the target (source side, server side and target side). Tip: The software store is ideal for backups in external offices with a subsequent copying process to headquarters on another software store or an HP StoreOnce system using the Catalyst interface. The deduplicated data transferred via the WAN. Evaluations in HP Data Protector The software provides various reporting options. In larger environments, HP Backup Navigator is available that provides advanced analytics and reporting capabilities; in smaller environments (couple of servers backed up tape or disk), the integrated reports are often sufficient. For more information on Backup Navigator, visit: http://www.autonomy.com/adaptivebackup/backup-navigator Which evaluations are recommended? There are a large number of integrated reports in Data Protector. These are the most frequently used reports, along with a recommendation for frequency of execution: • Configured clients not used by Data Protector – 1x weekly • Look up schedule – 1x weekly • IDB size report – 1x weekly • File systems not configured for backup – 1x weekly • List of pools – 1x weekly, in larger environments daily • List of sessions – daily • Session errors – daily Tip: To automate the evaluations, a report group is set up in Data Protector and assigned a schedule. All of the reports created there are then executed automatically at the desired time. If e-mail notification is selected for an individual report, the created report is sent automatically depending on the set time. 25 Technical white paper | Best practice guide for HP Data Protector Notifications in HP Data Protector Various notifications are integrated into the software. If a certain event occurs in the backup and recovery environment, a corresponding notification is triggered. Which notifications are recommended? There are a number of configured notifications in Data Protector. The notifications used most frequently in practice are: • DeviceError • FileLibraryDiskUsage • HealthCheckFailed • IDBCorrupted • IDBPurgeNeeded • IDBSpaceLow • IDBTablespaceSpaceLow • MountRequest Tip: As default, all notifications are written into the event log by Data Protector. For important notifications, it is recommended to switch to e-mail notification (SMTP). Connectivity to Monitoring Solutions Any notification or any report can be sent across the Simple Network Monitoring Protocol (SNMP) interface per SNMP trap to a suitable application that can process SNMP traps. Well known for this are HP Operations, HP SIM or also Nagios (integrations and instructions are available for the HP products). All that needs to be done is to enter the SNMP target (Unix) in the evaluations or notifications. For Windows, the traps are sent to the target set in the Windows configuration. Tip: The description of the necessary steps to set up SNMP by means of ‘omnisnmp’ on the Cell Server can be found in Data Protector’s online help. The MiB tables can be down¬loaded from the HP website in the product area for HP SIM. External tools External tools are also suitable for documentation of the environment: HP Backup Navigator creates reports on the HP Data Protector environment (backups, media, etc.) for current and historical data for a single or multiple Cell Manager implementation. For more information on HP Backup Navigator, please visit: http://www.autonomy.com/adaptivebackup/backup-navigator. A large number of ready-made reports can be used. BRICK (‘Backup Related Information Collector and Keeper’) is also available. This tool collects information regarding the HP Data Protector configuration, installed updates, event logs, log files, etc. and presents the results in a clearly arranged report. For more information regarding this tool, refer to the website http://www.data-protector.org/wordpress/brick/. Tip: For regular documentation of the backup environment, BRICK can be run once a week on the Cell Server as a scheduled task. 26 Technical white paper | Best practice guide for HP Data Protector Purge Within the framework of daily maintenance on the Cell Server, a purge of the internal database is carried out unless daily maintenance is disabled. This purge, however, is often not sufficient for some environments or for environments with a large number clients to be backed up. Over time, the internal database can also be fragmented. This delays search access in the database. In this case, it is a good idea to carry out a thorough purge of the internal database within the framework of maintenance dates. The following points do not apply to HP Data Protector 8.X, as changes to the internal database mean that an extended purge is no longer necessary and is run automatically within the database. Note: The extended purge requires exclusive access to the internal database. During the entire operation, no backups may run and the HP Data Protector GUI is to be closed. If the purge is to be carried out on an MoM server, the constraints mentioned also apply to all other Cell Servers of the MoM environment. How is an extended purge carried out? The following steps are necessary for the extended purge of the internal database. Depending on the environment, the proposed value 31 for the retention period can be replaced by another value (for example: 60 days). The following procedure presupposes a basic understanding of the internal database, command and directories: • Omnidb - strip: Removes the catalogue information from all objects that are no longer protected. • Omnidbutil - clear: Marks all aborted backups with the status ‘In Progress’ as ‘Failed’ backups. • Omnidbutil - purge_failed_copies: Removes failed copy jobs. • omnidbutil - purge - sessions 31: Removes all backups without a protection period that are older than 31 days. • omnidbutil - purge - messages 31: Removes all messages of backups without a protection period that are older than 31 days. • omnidbutil - purge - dcbf -force: Removes all data that are no longer required from the DCBF directories. • omnidbutil - purge - mpos: Removes all object versions and media position that are no longer required from overwritten media. • omnidbutil - fixmpos: Synchronizes the MMDB after the purge of the DCBF directories. • omnidbutil - purge -filenames -force: Removes data that are no longer required from the ‘filenames’ – part of the internal database. The progress can now be observed in the Data Protector GUI. • Backup of the internal database: A backup of the internal database without IDB check is recommended directly after the purge, but this is not a requirement for the other steps. • Export of the catalogue database: A temporary directory is required for export of the database; it must be at least the same size as the db40 directory. The directory must not be in the network or on mapped drives. The command for the export is: ––omnidbutil -writedb -cdb <temporary directory> After the export, a copy of the MSG and DCBF directories is to be made. • Import of the catalogue database: The command for the import of the database is: ––omnidbutil -readdb -cdb <temporary directory> After the import of the database, the backup environment is available once again. The MSG and DCBF directories must be copied back if the complete database was removed via omnidbinit before the import. • Backup of the internal database: A backup of the internal database with IDB check is recommended directly after the import in order to verify the integrity of the database after the import. Note: An immediate purge of the internal database should be scheduled if the need for a purge is indicated in the event log by HP Data Protector. This occurs when 180 days have elapsed since the last purge, when a purge is likely to take longer than 120 minutes or the estimated number of file names to be deleted exceeds 6 million. An e-mail notification can also be configured for this. Tip: The report ‘Purge Preview’ can be used to calculate whether the overhead for a thorough purge is worthwhile. Alternatively, the command ‘omnirpt-report db_purge_preview’ can be used. 27 Technical white paper | Best practice guide for HP Data Protector Recovery tests and disaster recovery When was the recovery last tested? Depending on the size of the environment, the recovery of individual files can be a daily task or only a very rare task. What is the case, however, when an entire system or service is involved? When was the recovery last verified and logged for an entire system or service? Regular recovery tests not only train the adminis¬trators to proceed adeptly when a serious situation arises, they also validate the necessary processes within the environment. One example of this is the process of re-storing the tapes required for the recovery of a system. Tip: The following article can be used for recovery of the Cell Server by means of Enhanced Automated Disaster Recovery, a check of the necessary requirements and automatic creation of a boot medium: http://www.data-protector. org/wordpress/2011/03/recover-cell-server-with-enhanced-automated-disaster-recovery/. The article is valid up to HP Data Protector version 7.X under MS Windows 2008 R2. The recovery test of a system should take place regularly on a recurring, fixed date. The results of the test are to be logged and all additional preparatory and follow-up steps are to be documented. Note: Each of the following possibilities should be tested for the environment in the event of a disaster (see also: Data Protector online help for supported environments and restrictions). Once a decision has been made in favour of a method or a number of methods, the methods should be tested and logged accordingly. HP Data Protector provides various possibilities to recover a system (described below for Microsoft Windows). For recovery options using HP Data Protector Version 8.X, please refer to the online help included in the software. • One possibility is a new installation of the operating system on repaired hardware or rolling out a new virtual machine from a template, then installing the Data Protector client on this server and subsequently starting the recovery for the required data. • “Enhanced Automated Disaster Recovery” (EADR) is the best method for a recovery. It is also suitable for recovery of the Cell Server and of a Microsoft cluster. Here, all the drivers and a minimal operating system are included in an ISO image in advance. The ISO image can then be written onto a CD or integrated via HP iLO as a virtual drive. This method is supported for Windows and Linux operating systems. • ‘One Button Disaster Recovery’ – this method is similar to the EADR method, but here all the necessary steps of the EADR process are already carried out during the backup and written onto a bootable tape device. Tip: Installation of the component ‘Automatic Disaster Recovery Module’ on the clients and backing up the configuration of the server (system state / configuration object) enables recovery of an entire system when a serious situation arises and, if necessary, also on other hardware via the free function integrated in the starter pack ‘Enhanced Automated Disaster Recovery’. 28 Technical white paper | Best practice guide for HP Data Protector Glossary Term Definition Application online backup agent For the backup of databases, it is recommended that you deploy an application-specific online backup agent that allows you to back up the databases in line with the requirements of transactions during regular operation. Application Agents are available for: Oracle®, Informix, Sybase, MS SQL Server, MS SQL, MS Exchange, MS SharePoint Portal Server, SAP, SAP DB/MaxDB, SAP HANA, Baan IV, Lotus Notes, Solid Database, and DB2. Cell The cell typically represents a system or group of systems at a location or organizational unit connected to the same LAN, this forming a ‘backup cell’. The Cell Manager is the main system that controls and manages this cell. This central instance manages backups and recovery guidelines and tasks. Cell manager/cell server Each ‘backup cell’ requires a Cell Manager that specifies the backup strategy for the individual servers to be backed up and manages the index data. The index data contains the information about when which files from which directory have been backed up to where (bar code of the tape, etc.). The Cell Manager is therefore the main system in a cell on which the essential Data Protector software is installed and from which all backup and recovery tasks are administered. The graphical interface for administration can also be present on another system. Each cell has a separate Cell Manager. Client A client can be any system in the cell, such as a system that either delivers data for backup (disk agent or integration agent) or accepts data (media agent) and writes it onto a disk or tape device. Cluster A cluster is defined as a group of servers or other resources that behave jointly in the same way as an individual, logical system to enable high availability or also load distribution and parallel processing. A clustered computer can also be referred to as a clustered node. Data deduplication Data deduplication is a procedure for the compression of data in that duplicates are removed and replaced by a reference pointer. These procedures are complex, because a comparison of data blocks (so-called chunks) must always run during writing. Disaster case Occurrence of a catastrophe. Disaster recovery Recovery of the system after a complete failure. Disk agent Each server (client) from which data is to be backed up requires a disk agent that sends the files to be backed up from the file systems to a media agent. Disk staging Procedure in which the data backup is carried out on a hard disk area. Drive/tape A drive refers to a physical unit that receives data from a computer system and can write it onto a magnetic medium (often a tape). It can also read data from a medium and send them to a computer system. Drives can be tape devices (physical or virtual) and disk systems. E-LTU and E-Media “Electronic licenses to use” are products and licenses that are only delivered electronically. Encryption Encryption means that the backed-up data are stored in encrypted (coded) form. HP Data Protector contains automatic key management for both software and the encryption by LTO-4 to LTO-6 tapes. The user can then perform client-based encryption or use the encryption functionality of the LTO tape devices. HP Data Protector uses AES-256-bit encryption. Incremental forever Special form of data backup, in which only the incremental changes to the data are backed up after an initial full backup. 29 Technical white paper | Best practice guide for HP Data Protector 30 Term Definition Instance Defines each implementation of the application installed on a server. Instant recovery Snapshots located on a storage system can be made available to the system again very quickly (see also ‘Zero Downtime Backup’). Internal database /IDB The database of HP Data Protector, in which all information regarding backups, media, drives, etc. is stored. Library A pool of one tape device or a number of tape devices with storage bays for media in a housing. License Authorization to use the corresponding function. Limited License to Use (LTU)/ Test License A software license to use (LTU), which specifies in the period for which the license is valid its license description; for example one month, one year. LTU License To Use. Manager-of-manager (MoM) A number of cells can be grouped together and administered from a central cell. The MoM contains all the clients localized in a number of Data Protector cells. Media agent A media agent accepts data to be backed up, packages them in a save set, and stores them on a backup medium. A backup medium can be a disk or tape device. In the case of advanced backup-todisk, the save sets are first stored on a disk and later copied to a tape device. Medium Typically a cassette tape or a file on a hard disk that contains backup data. Online integration See ‘Application Online Backup Agent’ Pool Collection point for media in HP Data Protector. Quickspecs Special website at HP showing the specifications for a product in a clearly arranged form. Retention period Storage duration Slot A slot defines a tape insertion point in a tape library. Each slot can contain one medium, for example an LTO tape. HP Data Protector refers to each slot on the basis of an assigned digit. To read a medium, robotics move the medium from the slot into the drive. Source A source is the productive source from which data are drawn for the data backup, or reloaded for data recovery. System state The configuration of the server, consisting of settings for the network, information regarding the hard disks, users, etc. Virtual machine A virtual machine is a computer system that does not physically exist, rather is simulated by another computer. Examples of virtual machines in VMware, Hyper-V or XEN. Zero-downtime backup Snapshot-based backup of a storage solution. Technical white paper | Best practice guide for HP Data Protector Appendix A – Speeds LTO DRIVES LTO drives Speed – uncompressed Speed – compressed Colour HP LTO media Additional functions LTO-1 20 MB/s 40 MB/s Blue — LTO-2 40 MB/s 80 MB/s Dark Red — LTO-3 80 MB/s 160 MB/s Yellow WORM LTO-4 120 MB/s 240 MB/s Green WORM, encryption LTO-5 140 MB/s 280 MB/s Light blue WORM, LTFS, encryption LTO-6 160 MB/s 400 MB/s Purple WORM, LTFS, encryption Fibre channel HBAs FC Speed in Gbit/s Speed in MB/ss 1 GFC 1 100 2 GFC 2 200 4 GFC 4 400 8 GFC 8 800 16 GFC 16 1600 Network boards LAN Gross MBit/s Transfer Transfer theoretical MB/s practical MB/s 100BASE-X 100 12.5 8-10 1 GbE 1000 125 80-100 10 GbE 10 GbE 1000 1250 800-1000 Network boards Hard disks Bandwidth in MB/s SATA 150 SATA II 300 SATA III 600 SAS 300 SAS II 600 SAS III 1200 All specifications of speeds are gross values and will vary in practice. For LTO-1 drives to LTO-5 drives, a compression ratio of 2:1 is assumed; for LTO-6 drives, a compression ration of 2.5:1 is assumed. Sources of the values are articles from German-speaking Wikipedia (http://de.wikipedia.org). The specifications of each manufacturer provide statements regarding the data throughput rates of hard disks in storage systems. 31 Technical white paper | Best practice guide for HP Data Protector Additional sources of information HP Data Protector Website http://www.hp.com/go/dataprotector HP Data Protector Manuals (passport account required) http://support.openview.hp.com/selfsolve/manual HP Data Protector Quickspecs http:/h18000.www1.hp.com/products/quickspecs/11671_div/11671_div.pdf HP Deduplication Solutions http://www.hp.com/go/storeonce Enterprise Backup Solution Guide http://www.hp.com/go/ebs HP Library and Tape Tools http://www.hp.com/support/tapetools HP Storage Sizer (only available to partners) http://www.hp.com/go/storageworks/sizer Sign up for updates hp.com/go/getupdated Share with colleagues © Copyright 2015 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. The only warranties for HP products and services are set forth in the express warranty statements accompanying such products and services. Nothing herein should be construed as constituting an additional warranty. HP shall not be liable for technical or editorial errors or omissions contained herein. 20150521_LC_TWP_Best_Practice_Guide_for_HP_Data_Protector