Oracle Optimized Solution for
Secure Backup and Recovery
ORACLE WHITE PAPER
|
OCTOBER 2015
Table of Contents
Introduction
1
Solution Overview
1
Addressing Data Recovery Needs and Requirements
2
Creating More Secure Backup and Recovery Environments
3
Architecture Overview
Solution Software Components
4
4
Oracle Solaris 11
4
Oracle Linux
5
Oracle's StorageTek Automated Cartridge System Library Software
5
Oracle Key Manager
5
Oracle's StorageTek Tape Analytics Software
6
Oracle Enterprise Manager Cloud Control
7
Solution Hardware Components
7
Network Infrastructure
8
Oracle's SPARC Servers
8
Oracle's x86-Based Servers
9
Oracle ZFS Storage Appliance
9
Zero Data Loss Recovery Appliance
10
Oracle's StorageTek Tape Library Products
10
Oracle's StorageTek Tape Drives
12
Reference Architecture Components in Tape Environments
12
Oracle Secure Backup
13
ORACLE OPTIMIZED SOLUTION FOR SECURE BACKUP AND RECOVERY
Cloud-Based Database Backups
14
Backup Best Practices
14
Best Practices for Accomplishing Recovery Time and Recovery Point Objectives
15
Using a Combination of Software Tools
15
Best Practices for Backup and Recovery in Virtualized Environments or with Unstructured Data
16
Best Practices for Backup and Recovery in Virtualized Environments and
Unstructured Data with NDMP
Best Practices for a Secure Backup and Recovery Implementation
16
16
Security Technical Implementation Guides
16
Component-Level Security Recommendations
17
Best Practices for Backup and Recovery of Operating Systems
19
Oracle Solaris 11
19
Backing Up Domains
20
Backing Up Oracle Solaris Zones
20
Best Practices for Database Backups (Oracle Database 11g Release 2 or Higher)
21
Recommended Database Backup Schedule
21
Always Use a Fast Recovery Area
21
Additional Database Configuration Best Practices
21
Oracle RMAN Configuration Best Practices
22
Restore and Recovery Best Practices
22
Best Practices for Performing Local Disk-Based Backups with Oracle RAC
22
Best Practices for Performing Local Disk-Based Backups with Oracle RAC One
Node
ORACLE OPTIMIZED SOLUTION FOR SECURE BACKUP AND RECOVERY
23
Database Best Practices for Disk-Based Backups
23
Database Best Practices for Tape-Based Backups
23
Best Practices for Tape-Based Backups
Network Configuration Best Practices
24
24
InfiniBand Network to Media Server Configuration Best Practices
24
1 GbE or 10 GbE Network to Media Server Configuration Best Practices
24
Configure Persistent Bindings for Tape Devices
25
Back Up the Oracle Secure Backup Catalog
Best Practices for Oracle Engineered Systems Backups
Oracle SuperCluster Backup Best Practices
25
25
25
Backing Up the Oracle SuperCluster Infrastructure
25
For More Information
26
Oracle Exadata Backup Best Practices
26
Oracle Database Appliance Backup Best Practices
26
Configuring Oracle Database Appliance for Local and External Backups
26
For More Information
27
Best Practices for Oracle ZFS Storage Appliance
27
Use the Oracle Engineered Systems Backup Utility for Oracle ZFS Storage
Appliance
27
Network Configuration Best Practices
27
For More Information
28
Storage Configuration Best Practices
28
ORACLE OPTIMIZED SOLUTION FOR SECURE BACKUP AND RECOVERY
Configuring Oracle RMAN Projects and Shares on Oracle ZFS Storage
Appliance
29
Database Backup and Restore Best Practices with Oracle ZFS Storage Appliance and
Oracle Engineered Systems
29
Configure Network Connectivity
29
Follow Recommended Best Practices for Database Backup and Restore
30
Tune the Oracle Database Instance for Oracle RMAN
30
Configure Oracle RMAN
31
Best Practices Using Oracle ZFS Storage Appliance
Use ZFS Snapshots
31
31
Using Non-Oracle Exadata Storage for Oracle SuperCluster and Oracle Exadata
Database Machine Backups
32
Providing Backup and Recovery Savings and High Availability with Remote DR Sites 32
Conclusion
33
References
34
ORACLE OPTIMIZED SOLUTION FOR SECURE BACKUP AND RECOVERY
Introduction
Oracle engineered systems and Oracle Optimized Solutions offer massive scalability and performance
for today's larger and more complex workloads and increasingly diverse data environments.
Engineered, optimized, and integrated, these systems provide the radical processing speeds,
significantly faster deployments, and simplified data center operations needed to achieve unmatched
enterprise performance levels. Combining best-of-breed hardware and software components with
game-changing technical innovations, Oracle engineered systems and Oracle Optimized Solutions
deliver such massive capacity and extreme performance that they require a new way of thinking about
backup and recovery.
With ownership of both the hardware and software technology, Oracle is in the unique position of being
able to plan, design, and execute backup solutions for these powerful and scalable systems. This
paper describes Oracle Optimized Solution for Secure Backup and Recovery, with particular focus on
Oracle engineered systems, including Oracle Exadata Database Machine (Oracle Exadata), Oracle
Database Appliance, Oracle SuperCluster, and Oracle Optimized Solutions. This pretested, highperformance backup solution can be used to accelerate data protection processing and management
with breakthrough cost structures using Oracle software powered by Oracle's servers, operating
systems, and tape storage.
Delivering next-generation data protection with redundancy, high availability, and the security of
encryption, Oracle Optimized Solution for Secure Backup and Recovery offers a flexible, secure,
multitier architecture for large and small environments. The solution also provides virtually unlimited
scalability with centralized management and end-to-end data protection across heterogeneous
technologies.
Solution Overview
Oracle Optimized Solution for Secure Backup and Recovery is designed as a complete solution for performing
network backups of heterogeneous clients. These clients include both Oracle engineered systems and Oracle
Optimized Solutions. Leveraging built-in Oracle integration, Oracle Optimized Solution for Secure Backup and
Recovery provides pretested, recommended configurations for complete backup and recovery solutions. This
solution provides options for traditional disk and tape backups as well as cloud-based backups, and includes Oracle
=)66WRUDJH$SSOLDQFH2UDFOH¶V6WRUDJH7HNWDSHOLEUDULHVDQG2UDFOH¶V=HUR'DWD/RVV5HFRYHU\$SSOLDQFHas
shown in Figure 1. The architecture is designed to simplify solution implementation and facilitate future upgrades as
needed for data growth.
With complete end-to-end data protection for network backup clients, Oracle Optimized Solution for Secure Backup
and Recovery can also complement the Oracle Recovery Manager (Oracle RMAN) and Oracle Active Data Guard
data protection solutions using Oracle ZFS Storage Appliance for cost-effective, long-term storage on highperforming encrypted tape. Cloud-based database backups are also supported. Oracle Optimized Solution for
1 | ORACLE OPTIMIZED SOLUTION FOR SECURE BACKUP AND RECOVERY
Secure Backup and Recovery is a comprehensive solution for Oracle Database backups with support for Oracle
Database 9i or higher.
Figure 1. Oracle Optimized Solution for Secure Backup and Recovery supports different backup client types.
Oracle Optimized Solution for Secure Backup and Recovery is a unique offering in that it is both low cost and high
performance, making it a clear industry leader in terms of price/performance for backup and recovery. Indeed,
Oracle Secure Backup software licensing costs can be significantly less than the costs of comparable competitive
products, because the tape management software component carries a low-cost, one-time software licensing fee
per tape drive used. Using solutions that do not rely on tape backups can save on the costs of licensing tape
management software as well. For example, backups that utilize Oracle ZFS Storage Appliances or Oracle Exadata
storage systems are disk-only solutions and do not incur software licensing costs. In addition, the solution enables
the use of tape²the lowest-cost storage media, which is ideal for storing older backup copies. Capable of scaling
nearly linearly, the solution uses a building block approach that consists of pretested server and storage
configurations.
When this solution is deployed to back up Oracle engineered systems and Oracle Optimized Solutions, it
dramatically simplifies backup and recovery processing and management. Typically deployed in replicated
configurations, Oracle Optimized Solutions eliminate the need to deploy complex third party deduplication
technologies, because simple tape and disk backups can be performed at one or both sites for complete data
protection.
Addressing Data Recovery Needs and Requirements
When preparing data center backup and other data protection strategies, special attention must be paid to the
recovery point objectives (RPOs) and recovery time objectives (RTOs) for all data and applications. An RPO defines
the maximum acceptable level of data loss following an unplanned event²natural or man-made²that can cause
data to be lost. Measured backwards in time, and starting from the instant the event occurs, the RPO is specified in
seconds, minutes, hours, or days. It is used to help administrators select optimal disaster recovery technologies and
procedures, and it also helps to determine the minimum frequency for performing backups. For example, Wall Street
trading firms need an RPO of seconds or less, and must provide redundant, remote systems to ensure that
transactions are not lost. Organizations with more static data will not have such granular RPOs.
2 | ORACLE OPTIMIZED SOLUTION FOR SECURE BACKUP AND RECOVERY
An RTO is the period of time within which IT capabilities must be restored following an unplanned event. IT
departments determine RTOs by calculating the amount of revenue lost per unit of time as a result of the event, or
by determining the maximum amount of time that their organizations can afford to be without IT functionality.
Figure 2 illustrates various data center backup attributes, various Oracle backup and storage hardware, and which
hardware addresses the various attributes. These factors should be kept in mind when evaluating backup solutions
for Oracle engineered systems and Oracle Optimized Solutions.
Figure 2. Oracle Optimized Solution for Secure Backup and Recovery addresses a range of retention needs as well as requirements
for RTOs and RPOs.
Creating More Secure Backup and Recovery Environments
Protecting data is an essential business concern for all enterprises. Enterprise-level backup environments typically
use a centralized backup and recovery model, which simplifies administration as the size of data continues to grow.
However, this centralization also presents a potential security threat: If an intruder gains access to a centralized
backup or media server, data for many systems across the enterprise can be compromised. Physical security is, of
course, fundamental, and access to servers and media libraries must be protected and audited. Backup servers,
clients, networks, storage, and tape libraries should all be configured following recommended security guidelines to
provide end-to-end security. Data encryption is also recommended to protect data at rest and while in transit across
networks and to/from storage media.
The following steps can help create a more secure backup and recovery environment.
» Simplify the infrastructure. Most backup and recovery environments are based on a complex infrastructure,
making implementation and management complicated. This complexity increases the risk of security
vulnerabilities. A backup and recovery implementation as a whole is only as secure as its most vulnerable
component, and it can be challenging to securely configure the myriad interacting components and products in a
heterogeneous system. Oracle Optimized Solutions simplify backup and recovery implementations through the
use of consolidation and virtualization technologies. Oracle also offers security guidelines and recommendations,
and many Oracle components have security built-in by default.
» Reduce implementation flaws. Secure software is important but not sufficient by itself. Most security
vulnerabilities arise from flawed implementation and architecture, including improper configuration and access
control, lack of patch management, unencrypted communications, and inadequate security policies and
processes. Based on current security best practices, Oracle Optimized Solutions provide proven and tested
architecture recommendations for increased backup and recovery solution protection.
3 | ORACLE OPTIMIZED SOLUTION FOR SECURE BACKUP AND RECOVERY
» Eliminate performance and cost penalties. Many security processes, such as on-the-fly encryption/decryption,
can have a significant negative impact on the performance and cost of a backup and recovery solution. Oracle
2SWLPL]HG6ROXWLRQVOHYHUDJH2UDFOH¶V63$5&-based systems, which offer high-performance security using
cryptographic instruction accelerators that are directly integrated into the processor cores. By providing wirespeed security capabilities, Oracle systems eliminate the performance and cost penalties typically associated with
real-time, secure computing.
Architecture Overview
Oracle Optimized Solution for Secure Backup and Recovery features a scalable, multitier architecture that includes
backup clients, media servers, administrative servers, Oracle disk devices, and encrypted tape devices. Designed to
be software-agnostic, Oracle Optimized Solution for Secure Backup and Recovery can work with Oracle RMAN,
Oracle Secure Backup, Symantec NetBackup, or other third-party backup software. For illustration purposes, this
paper refers to use of Oracle Secure Backup software throughout.
Note that in disk-only environments²such as those using Oracle Exadata storage, Oracle¶V Exadata Storage
Expansion Racks, or Oracle ZFS Storage Appliances²no backup software is required. Disk backups can be
completed using the operating system and Oracle Database tools²such as Oracle RMAN²alone.
Solution Software Components
The recommended architectures in this solution feature common software components, as shown in Table 1. The
following sections describe these software components.
TABLE 1. SOLUTION SOFTWARE COMPONENTS Software Component
Description
Oracle Solaris 11, Oracle Linux
Operating system
2UDFOH¶VStorageTek Automated
Cartridge System Library Software
A centralized library management solution
Oracle Key Manager
A highly available encryption key management system for encrypted tape backups
2UDFOH¶VStorageTek Tape Analytics
Software that provides intelligent monitoring capabilities
Oracle Enterprise Manager Cloud
Control
Software that provides fully automated, unified data protection management
Oracle Solaris 11 Oracle Solaris 11 is the first operating system engineered for enterprise clouds. With built-in virtualization, Oracle
Solaris 11 offers fast, intelligent provisioning capabilities for rapid service setup and maintenance. The operating
system provides fully integrated security for users, applications, and devices with fine-grained delegated
management and the latest security standards. Scalable data management is achieved with Oracle Solaris ZFS, the
innovative default file system in Oracle Solaris 11. Oracle Solaris ZFS brings advanced storage features such as
built-in deduplication, encryption, and thin provisioning to enterprise servers, and Oracle Solaris ZFS snapshots help
to deliver boot environments that enable fail-safe updates and self-recovery.
Oracle Solaris 11 offers mission-critical infrastructure that simplifies software lifecycle management and cloud-scale
data management while providing advanced protection for public, private, and hybrid cloud environments. New
features increase performance, reliability, security, efficiency, and virtualization, as well as preserve full compatibility
4 | ORACLE OPTIMIZED SOLUTION FOR SECURE BACKUP AND RECOVERY
with a vast portfolio of existing third-party products and internally developed customer applications. Oracle Solaris 11
is engineered jointly with Oracle applications and middleware to deliver unique features that increase performance,
streamline management, and automate support for Oracle deployments.
Oracle Linux For those sites using Oracle¶V x86 servers, Oracle Linux brings the latest Linux innovations to market, delivering
extreme performance, advanced scalability, and reliability for enterprise applications and systems. Optimized for the
Oracle stack, Oracle Linux is built and tested to run Oracle hardware, databases, and middleware and is
recommended for all enterprise applications.
Oracle's StorageTek Automated Cartridge System Library Software Oracle's StorageTek Automated Cartridge System Library Software (StorageTek ACSLS) provides a strategic,
centralized library management solution. Figure 3 shows two environments²one without StorageTek ACSLS and
one after deploying StorageTek ACSLS.
Figure 3. StorageTek ACSLS provides centralized management and high availability for 2UDFOH¶VStorageTek tape libraries.
Without StorageTek ACSLS software, organizations must grapple with distributed management, multiple
administrators and libraries, excess environmental consumption, poor resource utilization, and higher total cost of
ownership. Deploying StorageTek ACSLS as part of a tape management solution enables centralized management
of a single library by a single administrator, with lower environmental consumption, standard policy management,
balanced workload and resource utilization, maximum storage efficiency, and lower cost of ownership. In addition,
StorageTek ACSLS can help tape environments meet high-availability objectives.
Oracle Key Manager Oracle Optimized Solution for Secure Backup and Recovery features Oracle Key Manager, which is a highly
available encryption key management system for encrypted tape backups. Oracle Key Manager 2.5 makes it easy to
implement and scale storage-based encryption²for operational and archive data²without unnecessary cost and
complexity. A sample configuration is shown in Figure 4. Oracle Key Manager 2.5 is designed with an emphasis on
simplicity, security, and scalability to help organizations realize the following advantages:
» High security. A hardened solution, Oracle Key Manager provides FIPS 140-2 Level 3 compliance (through
2UDFOH¶VSun Crypto Accelerator 6000 PCIe Card) and secure key protection throughout the key lifecycle, with a
dedicated key management and delivery network.
5 | ORACLE OPTIMIZED SOLUTION FOR SECURE BACKUP AND RECOVERY
» Interoperability. An open, standards-based architecture supports diverse storage devices²from mainframes to
open systems²under a single storage key management system.
» High availability. Active n-node clustering, dynamic load balancing, and automated failover ensure high
availability.
» Simplified management. A secure client GUI facilitates administration²whether it is at one site or worldwide²
through user-defined, policy-based automatic key management.
» Scalability. A single clustered Oracle Key Manager appliance pair can be used to manage thousands of storage
devices and millions of encryption keys, making the solution scale easily and nondisruptively.
Figure 4. An Oracle Key Manager sample configuration.
Oracle's StorageTek Tape Analytics Software 2UDFOH¶V6WRUDJH7HN7DSH$QDO\WLFVVRIWZDUHis an intelligent monitoring application that proactively captures library,
drive, and media health metrics and runs analytical calculations on these data elements. As a result, the software
can eliminate library, drive, and media errors and empower tape storage administrators to make proactive decisions
about tape environments prior to device failures. Figure 5 illustrates how the StorageTek Tape Analytics software
collects data from libraries, drives, and media.
6 | ORACLE OPTIMIZED SOLUTION FOR SECURE BACKUP AND RECOVERY
Figure 5. StorageTek Tape Analytics software simplifies tape monitoring.
Oracle Enterprise Manager Cloud Control Oracle Enterprise Manager Cloud Control is a centralized cloud management solution, with a single view of the
entire IT environment from applications to disk. It features a pluggable framework, providing an easy way to update
Oracle Enterprise Manager Cloud Control with support for the latest Oracle product releases. Several plugins, such
as one for Oracle Database, are installed by default. Some newer product releases, such as 2UDFOH¶VZero Data
Loss Recovery Appliance, require an additional plugin to be downloaded and installed.
Oracle Enterprise Manager Cloud Control is used to provide end-to-end data protection management in this
solution. This software provides a unified view from the time a backup is initiated to the time it is stored on disk or
tape, or replicated to another storage device in a remote data center. All backup locations are tracked, so that any
Oracle Recovery Manager (Oracle RMAN) restore and recovery operation can retrieve the most appropriate
backups, wherever they reside.
Advanced storage monitoring and reporting helps administrators effectively manage current and future throughput
and capacity. The amount of space needed for each database under management is predictively calculated based
on its historical backup space usage and recovery window goal. The space needed is displayed for each database
and the appliance aggregates the total space needed for all databases as a percentage of the total available storage, taking the guesswork out of accommodating data growth. Capacity reports provide summary and detailed
information on storage utilization as well as average and maximum throughput, plus detailed information on CPU,
memory, and IOPS. In addition, warnings can be generated if the space needed is within a user-configurable
threshold of total available space.
Solution Hardware Components
All recommended architectures in the solution feature common hardware components, as shown in Table 2.
TABLE 2. SOLUTION COMMON HARDWARE COMPONENTS Component
Description
Storage Area Network
Brocade switches
Administrative Servers
2UDFOH¶V[-based systems or SPARC servers
Media Servers
2UDFOH¶Vx86-based systems or SPARC servers
Tape Libraries
2UDFOH¶V StorageTek SL150, SL3000, and SL8500 tape libraries
7 | ORACLE OPTIMIZED SOLUTION FOR SECURE BACKUP AND RECOVERY
Tape Drives
Oracle¶s StorageTek T10000D or LTO 6 tape drives
StorageTek ACSLS
A single Oracle server running the StorageTek ACSLS software
Oracle Key Manager
Oracle Key Manager appliances
StorageTek Tape Analytics software
A single Oracle server running the StorageTek Tape Analytics software
Oracle ZFS Storage Appliance
Backup and recovery solution for Oracle engineered systems in customizable small,
medium, and large configurations
Exadata Storage Expansion Rack
Quarter-, half-, or full-rack configurations that add storage capacity to an Oracle Exadata
Database Machine or Oracle SuperCluster
Network Infrastructure Ethernet switches provide connections and link aggregation between the backup clients, media servers, and
administrative servers. Network speeds of both 1 gigabit per second and 10 gigabits per second are supported.
Connections from the media servers to the network clients are provided through a private 10 gigabit Ethernet (GbE)
network or connections into the solution's InfiniBand fabric.
Built-in Oracle InfiniBand switches are used to provide connectivity within Oracle engineered systems. InfiniBand
eliminates the physical complexity of multiple interconnects per system and provides immense bandwidth and
high-speed connectivity to support high-speed transfers of massive amounts of data. With high throughput, low
latency, and a scalable fabric suitable for fabric consolidation of interprocess communication, network, and storage,
InfiniBand delivers up to 63 percent higher transactions per second for Oracle Real Application Clusters (Oracle
RAC) than GbE networks.
SAN switches provide Fibre Channel connections between the Oracle Secure Backup media servers and the Fibre
Channel archive devices.
Oracle's SPARC Servers The solution features two members of Oracle's SPARC server family²SPARC T7-1 and 2UDFOH¶V)XMLWVX M10-1
servers (Figure 6). These servers are designed to provide breakthrough performance while maximizing reliability
and minimizing power consumption and complexity. The SPARC T7-1 server features a single SPARC M7
processor (32 cores) and up to 512 GB of system memory in a 2U rackmount enclosure. The SPARC M10-1 server,
a compact entry-level server, features DVLQJOH63$5&;RU63$5&;³WHQ´RU³WHQSOXV´processor (8 or 16
cores) and up to 512 GB of system memory in a space-saving 1U rackmount enclosure.
SPARC T7-1 and SPARC M10-1 servers are ideal for large-scale applications, enterprise-wide consolidation, and
database projects that require extreme reliability, availability, and security.
Figure 6. 2UDFOH¶VSPARC T7-1 and Fujitsu M10-1 servers provide high levels of performance and throughput.
8 | ORACLE OPTIMIZED SOLUTION FOR SECURE BACKUP AND RECOVERY
Oracle's x86-­Based Servers The solution also features members of Oracle's x86-based server family: Oracle Server X5-2 and Oracle Server
X5-2L servers (Figure 7). Oracle's comprehensive, open standards±based x86 systems provide the best platform to
run Oracle software when x86 architectures are required, offering enhanced reliability for data center environments.
Only Oracle provides an optimized hardware and software stack that comes complete with choice of OS,
virtualization software, and cloud management tools²all at no extra charge. Oracle's optimized hardware and
software stack has enabled enormous performance gains in its engineered systems and delivered world-record
benchmarks results.
Oracle Server X5-2 features up to two Intel Xeon processor E5-2600 v3 product family CPUs and up to 768 GB
memory in a compact 1U enclosure. Oracle Server X5-2L offers the same processors and memory capacity, and
adds expandability with support for up to 26 disk drives in a 2U enclosure.
Figure 7. Oracle¶V x86-based servers provide high performance and reliability.
Oracle ZFS Storage Appliance Oracle ZFS Storage Appliance (Figure 8) provides a flexible, high-performance, and low-cost backup and recovery
solution for Oracle engineered systems. Offering unmatched Oracle integration, high performance, efficiency,
simplified management, and low TCO, Oracle ZFS Storage Appliances are available in customizable small, medium,
and large configurations to suit varied backup requirements.
Figure 8. Oracle ZFS Storage Appliance provides a flexible, low-cost backup and recovery option for Oracle engineered systems.
9 | ORACLE OPTIMIZED SOLUTION FOR SECURE BACKUP AND RECOVERY
Zero Data Loss Recovery Appliance 2UDFOH¶V=HUR'DWD/RVV5HFRYHU\$SSOLDQFHVHHFigure 9) is an engineered system for database backup that
eliminates data loss exposure without impacting the performance of production environments. Compute, network,
and storage are integrated into a massively scalable appliance with a cloud-scale architecture that provides fully
automated database backup and recovery for multiple databases. The recovery appliance enables a centralized
backup strategy for large numbers of databases using cloud-scale, fault-tolerant hardware and storage.
Integrated with Oracle RMAN and featuring an incremental-forever backup strategy, the recovery appliance provides
minimal-impact backups. The databases send only changes, and all backup and tape processing is offloaded from
the production servers to the recovery appliance for improved system performance. Real-time database redo block
information is transmitted, eliminating potential data loss and providing instant protection for new transactions.
Database recoverability is improved with end-to-end reliability, visibility, and control of a database as a whole, rather
than as a disjoint set of files.
The recovery appliance features secure replication to help protect against disasters such as site outages or regional
GLVDVWHUV)OH[LEOHUHSOLFDWLRQWRSRORJLHVDUHVXSSRUWHGWRPDWFKDGDWDFHQWHU¶VUHTXLUHPHQWV)RUH[DPSOH
replication can be set up in a simple one-way topology, or two recovery appliances can be set up to replicate each
other, or a central recovery appliance can be used for replication from multiple satellite recovery appliances. In all
topologies, only changed blocks are replicated, minimizing WAN network usage.
Use of secure replication to a recovery appliance can help speed recovery times in the event of an outage. If a local
recovery appliance is not available, restore operations can run directly from a remote recovery appliance without first
staging the data locally.
Figure 92UDFOH¶V=HUR'ata Loss Recovery Appliance eliminates data loss and provides minimal-impact backups.
Oracle's StorageTek Tape Library Products
Oracle tape storage solutions maximize the bandwidth of Oracle Optimized Solution for Secure Backup and
Recovery. Providing fault isolation from production systems storage, the tape storage solutions enable easy
movement of backup media off-site and provide cost-effective retention of multiple backup copies. Tape media can
also be the most cost-effective method for the initial movement of large amounts of data. In the event that a large
database needs to be moved from site to site, such as to or from a secondary disaster recovery (DR) site, tapes can
10 | ORACLE OPTIMIZED SOLUTION FOR SECURE BACKUP AND RECOVERY
be shipped across the country quickly and cheaply, and subsequent incremental backups can be applied to make
the database current once again.
Oracle Optimized Solution for Secure Backup and Recovery features Oracle's StorageTek SL150, StorageTek
SL3000, and StorageTek SL8500 modular library systems (Figure 10). These reliable modular library systems scale
to suit a wide range of storage capacity requirements. The solution also features Oracle's StorageTek T10000D and
StorageTek LTO 6 tape drives (Figure 11), where applicable. All the recommended architectures include the
StorageTek SL3000 tape library, because this library offers the lowest-cost configuration with no single point of
failure. The tape libraries and tape drives are described below.
» StorageTek SL150 modular tape library. The StorageTek SL150 tape library scales from 30 to 300 LTO slots
with a maximum capacity of more than 750 terabytes of uncompressed data (using StorageTek LTO 6 drives) in a
standard 19-inch rack-mounted cabinet. The library also supports up to 20 tape drives with a native throughput
UDWHRIPRUHWKDQWHUDE\WHVSHUKRXU7KLVOLEUDU\¶VFDSDFLW\DQGVFDODELOLW\DUHdesigned for small and growing
or mid-sized business environments.
» StorageTek SL3000 modular library system. The StorageTek SL3000 modular library system offers an
innovative, eco-efficient approach to midrange storage, providing more choice and control for rapidly changing
environments. The StorageTek SL3000 modular library system scales from 200 to 5,9251 cartridge slots and from
1 to 56 tape drives in a footprint that provides linear growth in a rack environment. Its RealTime Growth capability
enables physical capacity to be installed in advance and tapped into incrementally with Capacity on Demand
license keys. The StorageTek SL3000 modular library system delivers a native throughput rate of 48.4 TB per
hour for maximum configurations.
» StorageTek SL8500 modular library system. The StorageTek SL8500 modular library system can be shared
across supercomputer, mainframe, Oracle Solaris, AS/400, Windows, and Linux environments, and across a
variety of other UNIX environments. Its Any Cartridge Any Slot technology enables any combination of supported
drives and media. The choices include 2UDFOH¶V StorageTek T10000 and StorageTek T9840 tape drives, as well
as LTO and SDLT tape drives. The StorageTek SL8500 modular library system has nearly limitless scalability,
and can currently scale to 100,880 slots (554.8 PB native storage) and up to 640 tape drives. Many libraries can
be consolidated onto a single StorageTek SL8500 modular library system, with modular and scalable capacity to
meet rapid growth requirements. The StorageTek SL8500 modular library system delivers a native throughput
rate of 580.6 TB per hour for maximum configurations.
1
Although the StorageTek SL3000 modular library system supports 5,925 cartridge slots, Oracle recommends moving to a StorageTek
SL8500 modular library system if capacity needs require over 3000 slots.
11 | ORACLE OPTIMIZED SOLUTION FOR SECURE BACKUP AND RECOVERY
Figure 10. Oracle's StorageTek modular library systems come in different sizes to accommodate data center needs.
Oracle's StorageTek Tape Drives Oracle Optimized Solution for Secure Backup and Recovery features Oracle's StorageTek T10000D and LTO6 tape
drives, shown in Figure 11.
» StorageTekT10000D tape drive. The StorageTek T10000D tape drive delivers a capacity of 8 TB of
uncompressed data and a throughput of 250 MB per second. These drive efficiencies help enable data centers to
store more data in the same footprint and shorten backup and recovery windows. StorageTek T10000D tape
drives support demanding, high-duty-cycle environments with dual-head, 32-channel technology, reducing the
number of tape passes and extending head and media life. These tape drives connect using dual-port Fibre
Channel options for compatibility with most storage environments.
» StorageTek LTO 6 tape drive. The StorageTek LTO 6 tape drive is Oracle's latest addition to its portfolio of tape
drives. With a capacity of 2.5 TB per second and a throughput rate of 160 MB per second, the StorageTek LTO 6
KDVPDQ\RIWKHIHDWXUHVRI2UDFOH¶V6WRUDJH7HN/72WDSHGULYHDQGRIIHUVVXSSRUWIRU6$6DQG)&LQWHUIDFHV
It comes in full- and half-height platforms, as single drives, or as part of an automation environment. IT
departments can choose the model that makes the most sense for the operating environment, application,
capacity, and performance. Oracle's scalable tape automation solutions are designed to accommodate whatever
drive and interface are selected.
Figure 11. StorageTek tape drives provide high-throughput capabilities.
Reference Architecture Components in Tape Environments
In environments using tape for backups, the primary logical components of Oracle Optimized Solution for Secure
Backup and Recovery include software that fulfills the following three roles. The software used can be Oracle
Secure Backup or other third-party backup software selected by IT staff.
12 | ORACLE OPTIMIZED SOLUTION FOR SECURE BACKUP AND RECOVERY
» Backup software administrative server. Each administrative domain must have exactly one administrative
server. The administrative server is configured with complete data for the other hosts in the domain, their roles,
and their attached tape devices. This data is maintained in a set of configuration files stored on the server.
The administrative server runs the scheduler, which starts and monitors each backup job. The scheduler keeps a
backup catalog with metadata for all backup and restore operations performed in the administrative domain.
OracOH¶Vx86-based servers or SPARC servers are recommended for use as administrative servers in this solution
because they deliver outstanding performance in the smallest possible configuration while providing considerable
room for growth. For optimal performance and availability, the administrative and media server roles should not
be mixed within the same server.
» Backup software media server. A media server is a host with at least one tape drive or library attached to it. The
media server transfers data to or from a volume loaded on one of the attached tape devices. Single media servers
can be attached to multiple tape libraries, and multiple media servers can share attachments to multiple tape
libraries.
The media server performs the backup operations under the direction of the administrative server, storing all
backup data on its connected storage devices. 2UDFOH¶V x86-based servers or SPARC servers are recommended
as media servers in this solution because they each offer the most I/O bandwidth for their platform type, a feature
that dramatically improves media server performance. Both server families also provide a large amount of
capacity for most environments. For optimal performance and availability, the administrative and media server
roles should not be mixed within the same server.
» Backup clients. While this solution focuses on backup and recovery for Oracle engineered systems and Oracle
Optimized Solutions, it can also be used for network backups of any client platform supported by Oracle Secure
Backup.
Although this solution includes multiple backup administrative and media servers, a single server can be deployed
to support both the administrative and media server workloads, if desired. Using two systems provides high
availability in a highly scalable solution, but the workloads can be combined on a single server. Additional media
servers and archive devices can be added as needed to address scalability requirements.
Oracle Secure Backup
Oracle Secure Backup provides centralized backup management for Oracle Database, heterogeneous file systems,
and Network Attached Storage (NAS) to disk and/or tape. Integrated with components of the Oracle technology
stack and optimized for Oracle Database backup, this scalable end-to-end backup solution eliminates multivendor
integration and support issues. With centralized, policy-based backup management and low-cost, single-component
licensing, this software reduces cost and complexity for reliable data protection across complex, heterogeneous
enterprise environments.
Oracle Secure Backup uses a central administrative server and one or more media servers to provide data
SURWHFWLRQIRUKHWHURJHQHRXVGLVWULEXWHGFOLHQWV5XQQLQJRQHLWKHU2UDFOH¶V[-based or SPARC servers, and
using Oracle tape archive devices and Oracle ZFS Storage Appliance, this software scales to support large
enterprise networks. The networks can contain a large number of clients (UNIX, Linux, or Windows), NAS devices,
or high-performance engineered systems such as Oracle Exadata, Oracle Database Appliance, and Oracle
SuperCluster. Data backup to both tapes and disk (new in Oracle Secure Backup 12.1) is supported. Oracle Secure
Backup uses standard tape formats and supports most popular tape drives and tape libraries across SAN, Ethernet
networks, Oracle InfiniBand networks, and SCSI environments. Dynamic tape-drive sharing ensures maximum
utilization of tape drives.
Built-in integration with Oracle products provides significant performance benefits for these products. Integration with
Oracle RMAN helps deliver optimized performance and achieve 25±40 percent faster backups for Oracle Database
than comparable media management utilities with up to 10 percent less CPU utilization. In Oracle Exadata
environments, optimized backup performance and support for Reliable Datagram Socket over Remote Direct
Memory Access (RDS/RDMA) can deliver approximately 50 percent more throughput per InfiniBand port. And tight
13 | ORACLE OPTIMIZED SOLUTION FOR SECURE BACKUP AND RECOVERY
integration with Oracle ZFS Storage Appliance and support for NDMP connectivity eliminates the need for an
additional media server to write backups to disk.
Encrypted backups secure data at rest, on tape, or on disk. Backups can be encrypted using host-based backup
HQFU\SWLRQRUSHUIRUPHGE\2UDFOH¶V6WRUDJH7HN/72RU7'WDSHGULYHV3URYHQHPEHGGHG66/WHFKQRORJ\
protects data while in transport and provides two-way server authentication for greater protection.
2UDFOH¶V x86-based or SPARC servers are used to create the optimal backup and recovery solution that best fits an
RUJDQL]DWLRQ¶VGDWDFHQWHUUHTXLUHPHQWVDQGVWDQGDUGV%RWKSODWIRUPVGHOLYHUEHVW-in-class network backup
performance and best-in-class reliability. SPARC servers from Oracle can deliver up to 20 percent better network
performance than the x86 server platforms. However, both types of platforms have similar bandwidth offerings in
terms of the number of expansion slots. The recommended components in the solution, including servers, SAN,
Ethernet and InfiniBand networks, tape library management, tape encryption management, and tape drives, are
configured for redundancy. To facilitate interoperability and deliver more stability, the same platform found in
2UDFOH¶Vhigh-performing appliance solutions, such as Oracle engineered systems and Oracle Optimized Solutions,
should be deployed wherever possible.
Cloud-Based Database Backups
Oracle Optimized Solution for Secure Backup and Recovery supports cloud-based database backups. Cloud-based
backups can serve as a complement to or a replacement for local backup and recovery solutions. By using a
cloud-based backup, an enterprise gains scalable capacity on demand and eliminates up-front capital investment in
storage hardware.
Oracle Optimized Solution for Secure Backup and Recovery can be deployed as part of a private cloud solution, for
enterprises requiring a dedicated internal private cloud implementation. Alternatively, the solution can be deployed
as part of a hybrid cloud solution, with production databases maintained on premises and cloud storage used for
database backups.
Two methods are supported when deploying Oracle Optimized Solution for Secure Backup and Recovery as part of
a hybrid cloud:
» Oracle Database Backup Service. Oracle Database Backup Service is a secure, scalable, reliable, and highperformance public cloud storage solution for storing Oracle Database backups. A client-side Oracle Database
Backup Module is used with Oracle RMAN to transparently handle the backup and restore operations.
» Oracle Secure Backup to Amazon Simple Storage Service (Amazon S3). The Oracle Secure Backup Cloud
Module, compatible with Oracle Database version 9i Release 2 or later, can be used to back up databases to
cloud-based storage services offered by Amazon S3. Oracle Secure Backup Cloud Module is implemented using
the Oracle RMAN System Backup to Tape (SBT) interface, which enables external backup libraries to be
seamlessly integrated with Oracle RMAN. With this cloud offering, local disk backups are sent directly to Amazon
S3 for offsite storage and are fully integrated with Oracle RMAN.
For more information on Oracle Database Backup Service, see cloud.oracle.com/database_backup.
Backup Best Practices
While in-depth coverage of backup best practices for each configuration of the solution is out of the scope of this
document, some recommended best practices are discussed in the following sections, along with information on
where to find thorough best practices for each Oracle solution.
14 | ORACLE OPTIMIZED SOLUTION FOR SECURE BACKUP AND RECOVERY
Best Practices for Accomplishing Recovery Time and Recovery Point Objectives
Oracle offers a complete technology stack composed of hardware and software. Tightly integrated, the multiple
layers of the stack contain tools for data protection and backup and recovery. Important tools can be found at the
application, database, operating system, virtualization, and storage levels, as shown in Figure 12. Following best
practices and combining these tools effectively can result in an optimal solution for backing up Oracle systems. In
fact, by not relying on any single tool, it is possible to create an ideal data protection environment that reduces
reliance on conventional backups.
Figure 12. Oracle offers data protection tools at various levels of the integrated stack.
Among the tools from each layer are the following:
» Application tools: Oracle Secure Backup and third-party backup software
» Database tools: Oracle Archive Log Mode, Data Guard, Oracle Flashback Database, Data Recovery Advisor,
and Oracle RMAN
» Operating system tools: Built-in backup and recovery tools, ZFS snapshots, and alternate boot environments (a
boot environment is an Oracle Solaris operating system tool).
» Storage tools: Exadata Storage Expansion Racks, Oracle ZFS Storage Appliances, and StorageTek Tape
storage
Using a Combination of Software Tools
The tools within the different layers of Oracle's stack can be used to help achieve diverse²and precise²recovery
time and recovery point objectives. By using various tools listed below, it is possible to protect valuable enterprise
data and production environments without relying on conventional²and lengthier²backup and restore operations.
» Oracle Flashback Database and Data Guard can help IT staff recover from logical errors and database outages
for recovery time objectives of 15 minutes to an hour.
» Oracle Archive Log Mode enables the database to be rolled back to any point in time within the recovery window
to meet specific recovery point objectives.
» Oracle Enterprise Manager Ops Center enables administrators to deploy Oracle Solaris and Oracle Linux
operating systems for fast operating system recovery.
» The default file system in Oracle Solaris 11 is ZFS, which supports snapshots, cloning, and replication. ZFS
provides the ability at the operating system level to utilize built-in commands for snapshots, cloning, and
replication.
15 | ORACLE OPTIMIZED SOLUTION FOR SECURE BACKUP AND RECOVERY
» Oracle ZFS Storage Appliance supports snapshots, cloning, and replication for specific recovery points.
Administrators can take a snapshot of an iSCSI LUN on Oracle ZFS Storage Appliance to create a backup.
Alternatively, deploying a remote Oracle ZFS Storage Appliance enables recovery in seconds by switching from
the primary to the remote appliance.
Best Practices for Backup and Recovery in Virtualized Environments or with Unstructured Data
When performing backup and recovery in virtualized environments or in environments with unstructured (nondatabase) data, traditional approaches consist of taking disks and applications offline and writing from disk to tape.
In the real world, however, organizations require 24x7 availability and cannot afford to take applications offline at all.
Oracle Solaris 11 uses Oracle Solaris ZFS (ZFS) as the default file system and ZFS provides features for capturing
a system snapshot that can be sent to an external Oracle ZFS Storage Appliance or stored on tape.
Oracle Optimized Solution for Secure Backup and Recovery is configured with an internal Oracle ZFS Storage
Appliance that can store the virtualized environments and file system snapshots. Oracle recommends the use of
these snapshots to maintain application availability while obtaining a copy of vital data.
Best Practices for Backup and Recovery in Virtualized Environments and Unstructured Data with NDMP
NDMP is an open standard protocol for network-based backup of network-attached storage (NAS) devices. It
facilitates interoperation of backup software and NAS hardware. Because NAS devices are not intended to host
applications such as backup software agents and clients, they cannot be set up as backup clients for software such
as Oracle Secure Backup. Instead, an NDMP service is created on the NAS device, using NDMP host types to
designate Oracle ZFS Storage Appliance as the backup target. These host types require the use of specific path
names. Please refer to My Oracle Support Note 1558851.1 for examples.
Best Practices for a Secure Backup and Recovery Implementation
Backup and recovery systems cannot rely solely on perimeter security. A combination of system-wide security
measures and best practices²including the rule of least privilege, strong authentication, access control, encryption,
auditing, disabling of unnecessary services, antimalware protections, and configuring system services for enhanced
security²should also be implemented for secure operations.
Oracle highly recommends leveraging existing recommendations and guidelines from product security guides,
Center for Internet Security (CIS) benchmarks, ISACA publications, and Department of Defense (DoD) Security
Technical Implementation Guides (STIGs) when designing a backup and recovery environment.
Security Technical Implementation Guides
STIGs are continually updated and currently available for many Oracle products. A list of STIGs relevant to this
solution is shown in Table 3.
TABLE 3. EXAMPLES OF RELEVANT STIGS STIG
Location
Oracle Solaris
iase.disa.mil/stigs/os/unix-linux/Pages/solaris.aspx Oracle Database 11g Release 2
iasecontent.disa.mil/stigs/zip/Apr2015/U_Oracle_Database_112g_V1R3_STIG.zip
Oracle Integrated Lights Out Manager
iase.disa.mil/stigs/app-security/database/Pages/exadata_lights.aspx
Oracle Exadata Storage Server
iase.disa.mil/stigs/app-security/database/Pages/exadata_storage.aspx
16 | ORACLE OPTIMIZED SOLUTION FOR SECURE BACKUP AND RECOVERY
Oracle¶V Sun Datacenter InfiniBand Switch 36
iase.disa.mil/stigs/app-security/database/Pages/exadata_infiniband.aspx
Oracle ZFS Storage Appliance
iase.disa.mil/stigs/app-security/database/Pages/exadata_zfs.aspx
Oracle WebLogic Server 12c
iase.disa.mil/stigs/Documents/u_oracle_weblogic_server_12c_v1r1_stig.zip
DoD Secure Telecommunications
iase.disa.mil/stigs/net_perimeter/telecommunications/Pages/index.aspx
Oracle Linux 6 Manual STIG
iasecontent.disa.mil/stigs/zip/Apr2015/U_Oracle_Linux_6_V1R2_STIG.zip
Storage Area Network (SAN)
iase.disa.mil/stigs/Documents/u_storage_area_network_v2r2_stig.zip
For more STIGs, please see the website iase.disa.mil/stigs/Pages/index.aspx.
Component-Level Security Recommendations
Oracle recommends the following component-level security guidelines.
» Change system default passwords. Using known vendor-provided default passwords is a common way cyber
criminals gain unauthorized access to infrastructure components. Changing all default passwords to stronger,
custom passwords is a mandatory step during infrastructure deployment.
» Keep component patching current. Ensure that all components are using the most recent firmware and
software versions to the extent possible. This tactic ensures that each component is protected by the latest
security patches and vulnerability fixes.
» Leverage isolated, purpose-based network interfaces. Network interfaces, virtual or physical, should be used
to separate architectural tiers, such as client access and management. In addition, consider using network
interfaces to separate tiers within a multitier architecture. This enables per-tier security policy monitoring and
enforcement mechanisms including network, application, and database firewalls as well as intrusion detection and
prevention systems.
» Enable encrypted network communications. Ensure all endpoints use encrypted network-based
communications, including secure protocols, algorithms, and key lengths. For Oracle WebLogic, use the UCrypto
provider to ensure that cryptography leverages the hardware assist capabilities of the SPARC platform.
» Enable encrypted data-at-rest protections.
» Use encrypted swap, /tmp, and ZFS datasets for any locations that could potentially house sensitive or
regulated data. This automatically takes advantage of cryptographic acceleration in Oracle Solaris.
» Use tape drive encryption to protect data that must leave the data center for off-site storage.
» For databases, use Transparent Data Encryption (TDE) to protect tablespaces that might store sensitive or
regulated data. TDE automatically takes advantage of cryptographic acceleration in Oracle Solaris on
SPARC systems.
» Secure the database. Refer to Oracle Optimized Solution for Secure Oracle Database security best practices
and recommendations.
» Deploy application services in Oracle Solaris non-global zones. Deploying applications within Oracle Solaris
non-global zones has several security advantages, such as kernel root kit prevention, prevention of direct memory
and device access, and improved control over security configuration (via zonecfg(1M)). This approach also
enables higher assurance auditing, because audit data is not stored in the Oracle Solaris non-global zone, but
rather in the Oracle Solaris global zone.
» Implement a baseline auditing policy. Use audit logs and reports to track user activity²including individual
transactions and changes to the system²and to flag events that fall out of normal parameters. These should be
implemented at both the Oracle Solaris and database levels. The baseline security audit policy should include
login/logout activity, administrative actions, and security actions, as well as specific command executions for
Oracle Solaris. This tactic enables auditing of a core set of security critical actions without overburdening the
system or database.
» Follow the rule of least privilege. Increase access control by granting only those privileges that a given
individual needs. This should be implemented at both the ERP system level and the infrastructure level.
17 | ORACLE OPTIMIZED SOLUTION FOR SECURE BACKUP AND RECOVERY
» Use strong authentication. Many intellectual property attacks use stolen credentials. Implementing strong
authentication methods, such as Kerberos, RADIUS, and SSL, can help prevent unauthorized access.
» Leverage role-based access control. As the number of applications and users increases, user-based identity
management can quickly become time consuming and labor intensive for IT staff. Consequentially, many users
are granted inappropriate authorities. Though it requires increased efforts during the design and implementation
phases, role-based access control (RBAC) is a popular option for low-maintenance, scalable access control, and
it can help alleviate the burden of identity management.
Table 4 lists Oracle SuperCluster security recommendations. A full list of relevant component security
recommendations is shown in Table 5.
TABLE 4. ORACLE SUPERCLUSTER SECURITY RECOMMENDATIONS Title
Location
³Best Practices for Securely Deploying the
SPARC SuperCluster T4-4´
oracle.com/technetwork/articles/servers-storage-admin/supercluster-security1723872.html
³SPARC SuperCluster T4-4 Platform Security
Principles and Capabilities´
oracle.com/us/products/servers-storage/servers/sparcenterprise/supercluster/supercluster-t4-4/ssc-security-pac-1716580.pdf
³Oracle SuperCluster T5-8 Security Technical
Implementation Guide (STIG) Validation and Best
Practices on the Database Servers´
oracle.com/technetwork/server-storage/hardware-solutions/stig-sparc-supercluster1841833.pdf
³Secure Database Consolidation Using the Oracle
SuperCluster T5-8 Platform´
oracle.com/technetwork/server-storage/sun-sparc-enterprise/documentation/o13-053securedb-osc-t5-8-1990064.pdf
TABLE 5. EXAMPLES OF COMPONENT SECURITY RECOMMENDATIONS Resource
Location
Oracle Solaris 11 Security Guidelines
docs.oracle.com/cd/E36784_01/html/E36837/index.html
Oracle Solaris 11.2 Security Compliance Guide
docs.oracle.com/cd/E36784_01/pdf/E39067.pdf
³(QFU\SW3URWHFWDQG6HFXUH<RXU%DFNXS'DWD
ZLWK2UDFOH6HFXUH%DFNXS´
oracle.com/technetwork/database/database-technologies/securebackup/learnmore/osb-103-encryption-335238.pdf
³Secure Deployment of Oracle VM Server for
SPARC´
oracle.com/technetwork/articles/systems-hardware-architecture/secure-ovm-sparcdeployment-294062.pdf
Oracle Solaris Cluster Security Guide
docs.oracle.com/cd/E39579_01/html/E39649/index.html
³User Authentication on the Solaris OS: Part 1´
oracle.com/technetwork/server-storage/solaris/user-auth-solaris1-138094.html
Oracle ILOM Security Guide
docs.oracle.com/cd/E37444_01/html/E37451/index.html
Database Advanced Security Administrator's
Guide
docs.oracle.com/cd/E11882_01/network.112/e40393/toc.htm
³Oracle Database 12c Security and Compliance´
oracle.com/technetwork/database/security/security-compliance-wp-12c-1896112.pdf
³Best Practices for Deploying Encryption and
Managing Its Keys on the Oracle ZFS Storage
Appliance´
oracle.com/technetwork/server-storage/sun-unified-storage/documentation/encryptionkeymgr-1126-2373254.pdf
Securing the Network in Oracle Solaris 11.2
docs.oracle.com/cd/E36784_01/html/E36838/index.html
Securing Users and Processes in Oracle Solaris
11.2
docs.oracle.com/cd/E36784_01/html/E37123/index.html
18 | ORACLE OPTIMIZED SOLUTION FOR SECURE BACKUP AND RECOVERY
Securing Systems and Attached Devices in
Oracle Solaris 11.2
docs.oracle.com/cd/E36784_01/html/E37121/index.html
Securing Files and Verifying File Integrity in
Oracle Solaris 11.2
docs.oracle.com/cd/E36784_01/html/E37122/index.html
Managing Encryption and Certificates in Oracle
Solaris 11.2
docs.oracle.com/cd/E36784_01/html/E37124/index.html
Developer's Guide to Oracle Solaris 11 Security
docs.oracle.com/cd/E36784_01/html/E36855/index.html
³Configuring Oracle GoldenGate Security´
docs.oracle.com/goldengate/1212/gg-winux/GWUAD/wu_security.htm#GWUAD354
´0DQDJLQJ6HFXULW\IRU%DFNXS1HWZRUNV´
docs.oracle.com/cd/E26569_01/doc.104/e21477/network_security.htm#OBINS277
Best Practices for Backup and Recovery of Operating Systems
Operating system backups are critical and should be made before and after every significant change to the Oracle
SuperCluster software. This includes performing backups before and after completing the following procedures:
» Installation or reconfiguration of significant Oracle or non-Oracle software
» Reconfiguration of significant operating parameters
» Before and after each quarterly full stack download patch (QFSDP) for Oracle SuperCluster
Oracle Solaris 11
The conventional method for backing up an operating system is to install backup and recovery software and perform backups of the operating system to disk or tape, potentially an inefficient approach in 24x7 environments. In Oracle
Solaris 10 and Oracle Solaris 11, the ZFS file system enables bare-metal recovery of the operating system using
ZFS snapshots, the ZFS commands zfs send and zfs receive, and ZFS stream files. A ZFS stream file is an
image of an entire ZFS file system placed into a file on an NFS share. The zfs send command creates a stream
representation of a ZFS snapshot that is written either to a file or to a different system using standard output. The
zfs receive command creates a snapshot whose contents are specified in the stream that is provided on
standard input. If a full stream is received, a new file system is created as well. This functionality enables administrators to send and receive ZFS snapshot data and file systems using these commands.
Here is a high-level outline of the steps used to perform a backup and bare-metal recovery of the operating system.
For details and commands, please see the Oracle SuperCluster Administrator's Guide, or My Oracle Support
Note 1558851.1.
To perform a backup, do the following:
» To start, create an NFS share on an Oracle ZFS Storage Appliance on which to store the snapshots.
» Take a ZFS snapshot of the operating system and send it to a stream file using the zfs send command.
To perform a restore of the snapshot, boot from the Oracle Solaris boot media, as follows:
» Create a new root pool, or rpool (a ZFS storage pool that contains a bootable ZFS root file system), on the
replacement disks.
» Mount the NFS share.
» To restore, issue a cat command on the stream file, piping it to the newly created pool name on the local disk
with the zfs receive command.
This procedure restores the zfs send file from the ZFS stream file back to the new ZFS pool. Using zfs send
and zfs receive is the preferred method for restoring a bare-metal operating system. The previously
19 | ORACLE OPTIMIZED SOLUTION FOR SECURE BACKUP AND RECOVERY
recommended method²using a deployment system²takes approximately eight hours contrasted with 45 minutes
when using the ZFS stream file method.
Backing Up Domains
The physical resources within Oracle servers can be virtualized for greater agility and utilization. Two different
methods can be used for virtualization. The first method creates domains, partitioning the hardware to use specified
physical resources and then physically or logically isolating one domain from another. The second virtualization
method uses Oracle Solaris Zones, where each zone is a virtualized operating system environment created within a
single instance of Oracle Solaris. Implementing zones enables the installation of multiple versions of Oracle Solaris
10 and Oracle Solaris 11 on a single system.
Various domain configurations and implementations are supported on Oracle servers. Domain configurations can
have different layouts due to the ability to install the domains on different disks. Disk resources can be allocated and
shared among domains, while some domains might not own any disks. Disks can be shared from other file systems
and appear to users as local hard drives. Single domain systems are typically fairly simple, as are two-domain
configurations.
When backing up domains, the backup methods are the same as for the operating system. The restoration
procedures vary, depending on what devices are installed with the domains, what disks are designated as targets,
and where the rpool is created to receive the restore.
Domains also can be backed up with block-level backups using NDMP and an Oracle ZFS Storage Appliance. Note,
however, that this method is limited to backing up entire data sets, and does not offer the ability to restore single
files. Single-file backup and recovery can be accomplished by installing Oracle Secure Backup in a zone to back up
the zone as a regular backup and recovery client.
Backing Up Oracle Solaris Zones
Backup and recovery of applications is supported in NFS, and backup and recovery procedures remain the same for
snapshots and replication of NFS shares, as well as for tape using NDMP. To back up NFS shares, use Oracle ZFS
Storage Appliance with NDMP. NDMP, sometimes referred to as dump mode, provides high-performance,
block-level backups. Note, however, that the ZFS NDMP functionality does not permit restoration of individual files,
so in order to restore a single file, the entire data set must be restored. For more information on using NDMP with
Oracle ZFS Storage ApplianceUHIHUWRWKH2UDFOHZKLWHSDSHU³NDMP Implementation Guide for the Sun ZFS
6WRUDJH$SSOLDQFH´
Oracle Solaris Zones typically contain applications and databases for different use cases. Oracle Solaris Zones can
be installed on internal server drives or on external iSCSI LUNs.
Zones stored on iSCSI LUNs cannot be backed up at the dump level with Oracle ZFS Storage Appliance²they
must be backed up in ZFS block mode. This is a high-performance method that works well for restoring the entire
zone. The ZFS block mode method does not support restoration of single files or directories; the entire data set must
be restored. Single-file backup and recovery can be accomplished by installing Oracle Secure Backup software in a
zone to back up the zone as a regular backup and recovery client.
For disk-only environments, zones can be backed up through the creation of snapshots, clones, or replication to
another Oracle ZFS Storage Appliance.
20 | ORACLE OPTIMIZED SOLUTION FOR SECURE BACKUP AND RECOVERY
Best Practices for Database Backups (Oracle Database 11g Release 2 or Higher)
The following sections outline database backup best practices designed for high availability, optimal performance,
and maximum data protection. These best practices should be followed for instances of Oracle Database running on
Oracle SuperCluster, Oracle Exadata, or Oracle Optimized Solutions.
Oracle Optimized Solution for Secure Backup and Recovery provides a choice of tape-based and disk-based
backup methods for protecting databases. This section includes general best practices for database backup as well
as recommendations for both disk-based and tape-based database backups.
For disk-based backups, this solution includes both Oracle ZFS Storage Appliance and Zero Data Loss Recovery
Appliance. The different configurations are designed to meet a variety of backup solution requirements. In general,
Zero Data Loss Recovery Appliance provides the highest performance and serviceability, while Oracle ZFS Storage
Appliance provides flexibility and a lower-cost option.
Recommended Database Backup Schedule
Oracle recommends the following schedule for backups of Oracle Database:
» Perform weekly Oracle RMAN level 0 (full) backups of the database.
» Perform daily cumulative Oracle RMAN incremental level 1 backups of the database and use block change
tracking.
» Roll incremental backups into a full backup and delay by 24 hours.
» Perform daily backups of the Oracle RMAN catalogs.
Always Use a Fast Recovery Area
One vital database backup and recovery best practice is to deploy the database with a Fast Recovery Area (FRA),
the location where the database stores all of its information for Oracle Flashback technologies, snapshots, database
backups, rollbacks, and more. The FRA enables the database to protect itself, by enabling the database to be rolled
back to a certain point in time and restored within a matter of seconds instead of minutes or hours using
conventional backup and recovery methods. Consequently, database recovery time is limited by the speed of the
device on which the FRA resides, and it should be located on the fastest performing storage possible. Oracle
engineered systems are designed with the FRA built in, because placing the FRA on anything but Oracle Exadata
internal storage or an external Exadata Storage Expansion Rack will lengthen the amount of time it takes for
recovery. Exadata Storage Expansion Racks can be used for extra space for the FRA, as needed.
Database backups can, if desired, be excluded from the FRA and stored instead on Oracle ZFS Storage Appliance.
In this case, the FRA on Oracle Exadata can be configured much smaller, because space for the full and
incremental backups is not required in the FRA. This configuration can provide cost savings, because lower-cost
storage can be used for database backup, without sacrificing system performance.
Additional Database Configuration Best Practices
There are other configuration parameters for Oracle Database that must be set in order to ensure optimal backup
performance:
» Set the initialization parameter _file_size_increase_increment=2143289344 to optimize the space
used when incremental (level 1) backups are taken on the FRA.
» Reset or remove the initialization parameters _backup_ksfq_bufsz and _backup_ksfq_bufcnt on
systems running Oracle Database release 11.2.0.2 or later releases.
21 | ORACLE OPTIMIZED SOLUTION FOR SECURE BACKUP AND RECOVERY
» Use a backup Oracle Net service to run against all database servers in the cluster for better performance and
high availability. The Oracle RMAN BACKUP command uses the service to automatically spread the backup load
evenly among target the instances offering the service.
» Set DB_RECOVERY_FILE_DEST_SIZE to bound space in the FRA. It is important that the value of this
parameter be set to less than the total free space in the disk group, which must take account of at least one disk
failure and preferably one Oracle Exadata Storage Server failure. If multiple databases are sharing the FRA,
ensure that the sum of the space allocated to the different databases is less than the free space in the disk group.
» Consider using Oracle Database Resource Manager to manage system resources, particularly I/O, on Oracle
SuperCluster.
Oracle RMAN Configuration Best Practices
» Parallelize backups across all database nodes, allowing all the disks, network connections, and system CPUs to
be leveraged for increased performance. Follow configuration instructions according to whether the backup
targets are disks or tapes.
» Use Oracle RMAN backup scripts to automate weekly and daily backups.
» Use two to eight Oracle RMAN channels per instance when performing backups to disks. When performing
backups to tape, configure one Oracle RMAN channel per tape drive and add tape drives to scale backup rates.
» Use Oracle RMAN incremental backups and block change tracking.
» To reduce backup time and resources, perform nightly incremental backups to the FRA and merge them into
the image copy backup on a regular basis. If recovery is needed, then the copies can be directly used as
normal data files and recovered to a consistent point, without the need for a restore operation, thus
significantly reducing overall recovery time.
» Enable Oracle RMAN block change tracking for fast incremental backups. This allows Oracle RMAN to avoid
scanning blocks that have not changed when creating incremental backups. Oracle Exadata Storage Server
also offloads block inspection from the database servers. Block change tracking is of greatest benefit for
databases where fewer than 20 percent of the blocks are changed daily. For block change rates greater than
20 percent, testing is recommended to ensure that backup times are reduced.
» Set Oracle RMAN configuration setting FILESPERSET=1 when performing incremental backups to specify the
maximum number of files in each backup set. A setting of 1 will allow for a faster single-file database restore
operation.
» Use an external Oracle RMAN recovery catalog repository as long as it is located on a system that is configured
to be highly redundant.
Restore and Recovery Best Practices
Oracle engineers performing testing in Oracle labs have discovered that optimum restore performance depends on
proper management of the backup chunk size. While the chunk size setting is not important during backups, the
restore operation must have Oracle RMAN backups broken into the best-performing chunk sizes for individual
environments or performance can suffer. Oracle recommends running tests in each individual environment²with the
actual data²to verify that the chunk size to be used for Oracle RMAN restores is going to provide optimal
performance.
The layout of each individual database determines performance. In addition, to increase recovery rate performance,
create a restore service that runs across all available database nodes. The service is used by the Oracle RMAN
restore command. Oracle RMAN automatically balances the restore load among the targeted instances.
Best Practices for Performing Local Disk-Based Backups with Oracle RAC
Scale backup rates written to local disk in the FRA on an Oracle Database Appliance by using an Oracle RAC
configuration.
» Use multiple instances and start with one Oracle RMAN channel per instance.
22 | ORACLE OPTIMIZED SOLUTION FOR SECURE BACKUP AND RECOVERY
» Continue to add Oracle RMAN channels for performance per instance. Optimal backup rates have been observed
by Oracle engineers with all Oracle RAC instances and one to four Oracle RMAN channels.
Best Practices for Performing Local Disk-Based Backups with Oracle RAC One Node
Scale backup rates written to local disk in the FRA on an Oracle Database Appliance by using a single instance and
Oracle RAC One Node configuration.
» Start with one Oracle RMAN channel.
» Continue to add Oracle RMAN channels to the single database instance to increase performance. Optimal
backup rates were observed with two to four Oracle RMAN channels.
Database Best Practices for Disk-Based Backups
Disk-only backups provide faster recovery times for data and logical corruptions and some tablespace point-in-time
recovery (TSPITR) scenarios. Backups to disk also provide the ability to use backups directly²with no restore
needed²by switching to a copy of the database, tablespace, or data file. Best practices for disk-based backup
configurations include the following:
» Scale backup rates for disk:
» Use all instances and start with two Oracle RMAN channels per instance.
» Continue to add another two more Oracle RMAN channels for performance.
» Utilize the automatic configuration of the Oracle Exadata Storage Server grid disk group layout during
deployment for better performance. This approach assigns the faster (outer) 40 percent of the disk to the
DATA area and the slower (inner) 60 percent of the disk to the fast recovery area (RECO) area.
» An alternative strategy is to purchase additional SATA Oracle Exadata storage specifically for storing the
FRA. This allows the application to leverage the full Oracle SuperCluster storage grid, allows the use of
lower-cost storage for backups, and provides better failure isolation by using separate backup hardware. To
reserve more space and bandwidth for the DATA disk group, Oracle recommends using a tape-based
backup solution, or at the very least, a hybrid approach where full database backups are written to tape and
incremental disk backups are written to the FRA.
» Configure a high-redundancy DATA disk group to contain the Oracle Cluster Registry (OCR) file, Oracle
Cluster voting disk, spfiles, data files, redo log groups, and control files for any disk-based backup solution.
Database Best Practices for Tape-Based Backups
In addition to the general best practices for Oracle Database backups mentioned above, best practices for
tape-based backups include the following:
» Perform daily backups of the Oracle Secure Backup or third-party backup and recovery software catalogs.
» Keep the number of archive logs to a minimum to avoid adversely impacting backup rates.
» Remember that heavy loads on an active database and fully consumed CPUs will affect backup rates.
» Use Oracle Secure Backup for fast, low-cost tape backups. Oracle Secure Backup provides the fastest database
backup to tape due to its tight integration with Oracle RMAN.
» Be aware that using Oracle Secure Backup enables the unused-block optimization capability. However, if the
backup is made directly to tape using a third-party media management product, then this does not have any effect
because the unused-block optimization is available only with Oracle Secure Backup.
» Configure the Network Time Protocol (NTP) daemon on Oracle Secure Backup servers. This step ensures that
the NTP daemon service on the Oracle Secure Backup administration and media servers are running and
configured to use the same time source as the Oracle engineered system.
» Configure the Preferred Network Interface (PNI) to direct the Oracle Secure Backup traffic over the InfiniBand
network interface. This parameter must be set when using a dedicated backup network.
23 | ORACLE OPTIMIZED SOLUTION FOR SECURE BACKUP AND RECOVERY
» Configure one Oracle RMAN channel per tape drive and add tape drives to scale backup rates. Backup
performance scales when more tape drives and Oracle RMAN channels are added, assuming there is available
throughput on the media server.
» Configure dedicated 1 GbE, 10 GbE, or InfiniBand interfaces. Using a dedicated interface for the transport
eliminates the impact on the client access network. Use InfiniBand for the best backup rates, especially for larger
databases that require fast backup rates and low CPU overhead. When not using InfiniBand, use 1 GbE for
smaller databases and 10 GbE for larger databases.
» Configure an Oracle RAC service for backup running on all database instances. This step reduces CPU utilization
on the database and media server nodes and load balances the backups.
» Use SQL*Net service load balancing to distribute Oracle RMAN channels evenly among the allocated instances.
» Tune the network communication when using a third-party media management vendor, and check with the
vendor²the vendor should also be able to provide configuration best practices.
Best Practices for Tape-Based Backups
In addition to the database best practices for tape-based backups mentioned above, there are some additional
general best practices to consider when performing tape-based backups.
Network Configuration Best Practices
Care must be taken when configuring the network connections for tape-based backups. The following two sections
detail configurations for connecting the media servers to the network.
InfiniBand Network to Media Server Configuration Best Practices When connecting media servers to the InfiniBand network for tape-based backups, do the following:
» Directly connect media servers to the InfiniBand fabric by adding an InfiniBand Quad Data Rate (QDR) host
channel adapter (QDR HCA) to the media server.
» Connect the HCA to two different Oracle SuperCluster InfiniBand switches to eliminate the switch as a single
point of failure and provide high availability. This configuration ensures transparent failover if connectivity is lost to
one of the ports.
» Configure bonding or IP network multipathing (IPMP) for the InfiniBand interfaces on the media server.
» Update OpenFabrics Enterprise Distribution on the media server.
» Configure IP over InfiniBand (IPoIB) connected mode for best performance.
» Configure MTU Size=65520 on InfiniBand for faster data transmission.
» Configure the media server to use the InfiniBand network.
1 GbE or 10 GbE Network to Media Server Configuration Best Practices The media servers for tape-based backups can be connected with one or more Ethernet ports that are either 1 GbE
or 10 GbE.
» Oracle Solaris enables link aggregation control protocol (LACP) bonding of interfaces for higher network
performance. Alternatively, administrators can use Oracle Solaris to configure two interfaces with IP network
multipathing (IPMP) for higher availability. LACP bonding requires a switch that can support LACP and enables
the connection of multiple Ethernet ports to different switches, but having the same Ethernet address.
» IPMP supports multiple interfaces on the same IP link, connected to a single switch. This is a highly redundant
configuration and increases availability, but does not provide high performance. This functionality is offered only
by Oracle Solaris.
» Configure the switch configuration, using 1 GbE or 10 GbE on the database server to create either a dual-port 1
GbE or 10 GbE configuration that supports higher backup rates.
» Configure 1 GbE or 10 GbE on the media server.
24 | ORACLE OPTIMIZED SOLUTION FOR SECURE BACKUP AND RECOVERY
Configure Persistent Bindings for Tape Devices It is very important that the environment maintains consistent device addresses. In SAN environments running
Oracle Linux, persistent bindings must be configured so that the device addresses do not change. If a device address changes, the media servers cannot access the device until the device configuration within Oracle Secure
Backup is updated. Please see My Oracle Support Note 971386.1 for an example of creating persistent bindings for
device attachments.
Back Up the Oracle Secure Backup Catalog
The Oracle Secure Backup catalog maintains backup metadata, scheduling, and configuration details for the backup
domain. Just as it is important to protect the Oracle RMAN catalog or control file, the Oracle Secure Backup catalog
should be backed up on a regular basis. In Oracle Secure Backup, the catalog backup has been preconfigured as
follows:
» Media family: OSB_Catalog_MF writes all catalog backups to the same tape or tapes.
» Job summary: OSB-­CATALOG-­SUM sends e-mail showing a daily report status of the catalog backup to users.
» Data set: OSB-­CATALOG-­DS defines all directories and files to back up for file system backups.
» Schedule: OSB-­CATALOG-­SCHED shows the schedule for the catalog backup.
The primary catalog backup configuration settings have been preconfigured, so only one step remains, which
requires user intervention.
» Edit the OSB-­CATALOG-­SCHED triggers to specify when to perform the backup.
For detailed and up-to-date information, please refer to My Oracle Support Note 1558851.1.
Best Practices for Oracle Engineered Systems Backups
The following sections detail best practices for backing up Oracle engineered systems along with pointers for more
detailed information.
Oracle SuperCluster Backup Best Practices
The recommended backup solution for some Oracle SuperCluster and Oracle Optimized Solutions environments
complements disk backups done on Oracle ZFS Storage Appliance by adding tape storage and tape storage
management. Adding tape storage and tape storage management extends onsite backup storage capacity and
enables the creation of off-site backup copies for long-term retention. Oracle recommends making tape backups of
primary and remote site data using Oracle RMAN backups and NDMP on an Oracle ZFS Storage Appliance through
Oracle Secure Backup media servers. For detailed procedures on performing a tape backup of Oracle SuperCluster,
refer to the Oracle white SDSHU³Protecting SPARC SuperCluster²Tape Backup with Symantec NetBackup´
Backing Up the Oracle SuperCluster Infrastructure The Oracle SuperCluster infrastructure information, including switch configurations for the InfiniBand fabric, is
created when the system is first deployed. This configuration information should be backed up to disk or tape to
enable quick recovery in case of human error or a disaster.
The Oracle SuperCluster backup tool, osc-­backup, is used to create a snapshot of the entire Oracle SuperCluster
infrastructure. This tool automates the process of backing up configuration information for all components in Oracle
SuperCluster, including networking configurations plus Oracle Solaris domains and Oracle Solaris Zones on the
FRPSXWHQRGHV,WLVUHFRPPHQGHGWRXVHWKLVWRROWRFUHDWHD³EDUH-PHWDOEDFNXS´at the following times:
» After initial deployment, to provide an initial backup snapshot of the Oracle SuperCluster infrastructure
25 | ORACLE OPTIMIZED SOLUTION FOR SECURE BACKUP AND RECOVERY
» Before and after reconfiguration of significant operating parameters
» Before and after installation of each quarterly full stack download patch (QFSDP) for Oracle SuperCluster
» As part of a regular maintenance schedule, at a frequency that is practical and feasible (monthly, for example)
The backups created by the osc-­backup tool are stored on the Oracle ZFS Storage Appliance that is internal to
Oracle SuperCluster. It is critical that this storage be backed up as part of an enterprise backup and recovery
strategy to safeguard against catastrophic failures and disasters. For more information on using the osc-­backup
tool, refer to My Oracle Support 'RFXPHQW³6XSHU&OXVWHU±How to back up a SuperCluster using the
osc-­backup tool.´
For More Information For more information, pOHDVHVHHWKH2UDFOHZKLWHSDSHU³Oracle Optimized Solution for Secure Backup and
Recovery of Oracle SuperCluster´DWoracle.com/us/solutions/oos/oracle-backup-and-recovery/overview/index.html
or refer to My Oracle Support Note 1558851.1.
Oracle Exadata Backup Best Practices
The Oracle Maximum Availability Architecture (MAA) group develops Oracle best-practices blueprints for Oracle
([DGDWDEDFNXSDQGUHFRYHU\7KHJURXSKDVSXEOLVKHGDZKLWHSDSHU³Backup and Recovery Performance and
Best Practices for Exadata Cell and Oracle Exadata Database Machine,´ZKLFKGHWDLOVEHVWSUDFWLFHV for backup and
recovery of Oracle Exadata systems. The paper contains performance results for various Oracle Exadata
configurations using both disk and tape backups. With detailed steps necessary for configuring the database, Oracle
RMAN commands, and network settings for optimal backups, the white paper also includes recommended
strategies for restore and recovery operations. In addition, readers can learn more about offloading backups using
Data Guard and gaining additional benefits through the use of Oracle Active Data Guard.
Oracle Exadata systems can be backed up to tape or disk. While both media types provide extremely fast backup
and restore rates, there are advantages to each. Tape-only solutions isolate faults from the Oracle Exadata Storage
Servers and maximize Oracle Exadata capacity and bandwidth. Disk-only backups provide faster recovery times for
data and logical corruptions and some tablespace point-in-time (TSPITR) scenarios. Disk backups also provide the
ability to use backups directly with no restore by switching to a copy of the database, tablespace, or data file.
Oracle Database Appliance Backup Best Practices
As with the other Oracle engineered systems, both disk and tape backups provide extremely fast backup and
restore rates, but there are advantages to each. Tape-only solutions isolate faults from the Oracle Database
Appliance and maximize appliance capacity and bandwidth. Disk-only backups provide faster recovery times for
data and logical corruptions and some tablespace point-in-time (TSPITR) scenarios. Backups to disk also provide
the ability to use backups directly with no restore by switching to a copy of the database, tablespace, or data file.
Configuring Oracle Database Appliance for Local and External Backups Choosing external versus local backup in the Oracle Database Appliance Manager during deployment will affect the
size of the disk groups +DATA and +RECO, which will determine the FRA size. When configuring the appliance,
selecting the external backup type option in the Oracle Database Appliance Manager Configurator utility assigns 80
percent of the disk to the DATA area and 20 percent of the disk to the RECO area. Selecting the local backup type
option in the utility assigns 40 percent of the disk to the DATA area and 60 percent of the disk to the RECO area.
26 | ORACLE OPTIMIZED SOLUTION FOR SECURE BACKUP AND RECOVERY
For More Information For a more detailed discussion of the best practices for backing up Oracle Database running on Oracle Database
Appliance, refer to VHFWLRQ³Best Practices for Database Backups (Oracle Database 11g Release 2 or Higher)´ in this
paper.
7KH2UDFOHZKLWHSDSHU³Backup and Recovery Best Practices for the Oracle Database Appliance´FRQWDLQVGHWDLOV
on recommended architectures, suggested configuration parameters, best practices for backup and recovery of
Oracle Database Appliances, and a detailed section on backing up appliances to network-attached storage (NAS)
devices. Also included are performance results for Oracle Database Appliance configurations using local or external
disks as backup targets, and the detailed steps necessary for configuring the various appliance components in order
to obtain optimal backup results. For more information, please see the white paper, or refer to My Oracle Support
Note 1558851.1)RUGHWDLOHGLQIRUPDWLRQRQWDSHEDFNXSVUHIHUWRWKH2UDFOHZKLWHSDSHU³Protecting Oracle
Database Appliance ² Tape Backup with Oracle Secure Backup´DWoracle.com/technetwork/serverstorage/engineered-systems/database-appliance/documentation/protecting-oda-with-osb-1674207.pdf.
Best Practices for Oracle ZFS Storage Appliance
Oracle recommends certain procedures for optimal configuration and performance of Oracle ZFS Storage Appliance
with Oracle SuperCluster or Oracle Exadata. Some of these procedures are outlined here.
Use the Oracle Engineered Systems Backup Utility for Oracle ZFS Storage Appliance Oracle recommends the use of the Oracle Engineered Systems Backup Utility for Oracle ZFS Storage Appliance.
This tool, which can be used to configure the Oracle ZFS Storage Appliance that is internal to the Oracle
SuperCluster and any additional external Oracle ZFS Storage Appliance used for backup, verifies the system
configuration and automates the necessary configuration steps, including the following:
» Configuring Oracle ZFS Storage Appliance. This includes creating a project and shares for Oracle RMAN.
» Configuring the Oracle SuperCluster database nodes. This includes enabling direct NFS (dNFS), creating mount
points, and adding backup services.
» Creating the final Oracle RMAN scripts. (Note: The Oracle RMAN run block scripts are intended for samples only,
and should be reviewed and modified as necessary to meet site-specific requirements.)
Although these configuration steps can be performed manually, use of the Oracle Engineered Systems Backup
Utility is recommended to reduce the risk of accidental misconfiguration.
The Oracle Engineered Systems Backup Utility for Oracle ZFS Storage Appliance can be downloaded from the
Oracle ZFS Storage Appliance Plug-in Downloads web page at oracle.com/technetwork/server-storage/sun-unifiedstorage/downloads/zfssa-plugins-1489830.html.
Network Configuration Best Practices Network configurations for Oracle ZFS Storage Appliance vary according to the environment where the appliance is
deployed. If Oracle ZFS Storage Appliance is to be used to back up one or two Oracle engineered systems and it is
located within 100 meters of the Oracle engineered systems, then InfiniBand networking can be used to connect all
the systems. With this recommended configuration, each Oracle engineered system resides on its own InfiniBand
fabric and different InfiniBand subnet.
When used with Oracle SuperCluster, Oracle ZFS Storage Appliance can be connected directly to the Oracle
SuperCluster InfiniBand leaf switches if Oracle ZFS Storage Appliance is the only other appliance or device (other
than Exadata Storage Expansion Rack) that will be connected to the infrastructure. Oracle ZFS Storage Appliance
can be connected to external InfiniBand leaf switches if more appliances or devices will be connected. If Oracle ZFS
Storage Appliance is to be used to back up more than two Oracle engineered systems, or Oracle ZFS Storage
27 | ORACLE OPTIMIZED SOLUTION FOR SECURE BACKUP AND RECOVERY
Appliance cannot be located within 100 meters of the Oracle engineered systems, then 10 GbE should be used to
connect the Oracle engineered systems to Oracle ZFS Storage Appliance.
Network Configuration for InfiniBand Connectivity
To provide the highest availability and performance, Oracle recommends that each Oracle ZFS Storage Appliance
controller be configured with two dual port QDR InfiniBand HCA cards, providing a total of four ports per Oracle ZFS
Storage Appliance controller. The controllers should also be connected to a pair of redundant 10 GbE switches to
provide continuous connectivity in the event of an outage. This configuration requires the use of an active/standby
IPMP group. For configuration details, see the Oracle white paper ³Configuring a Sun ZFS Backup Appliance with
Oracle SPARC SuperCluster´.
Network Configuration for 10 GbE Connectivity
To provide the highest availability and performance while not using the InfiniBand network, Oracle recommends that
each Oracle ZFS Storage Appliance controller be configured with two dual port 10 GbE optical cards, providing a
total of four ports per Oracle ZFS Storage Appliance controller. This configuration requires the use of an
active/standby IPMP group. Additionally, for performance reasons, the 10 GbE network should be configured with a
large MTU size, also known as jumbo frames. For configuration details, see the Oracle white paper ³Configuring a
Sun ZFS Backup Appliance with Oracle SPARC SuperCluster´
Configuring Network Cluster Resources
Once the network resources have been configured for either 10 GbE or InfiniBand, the interfaces need to be placed
under the control of the Oracle ZFS Storage Appliance cluster. Clustering the network resources provides
continuous access to the data on Oracle ZFS Storage Appliance in the event of a controller head failure in the
appliance.
Configuring Networking for Oracle ZFS Storage Appliance
The InfiniBand ports on Oracle ZFS Storage Appliance must be configured for IP multipathing. Four IP addresses
will be required for each Oracle ZFS Storage Appliance head²for a total of eight addresses²because the
interfaces will be running in an active-active configuration.
For More Information FRUPRUHGHWDLODQGDOLVWRISULQFLSOHVWRXVHIRUGDWDEDVHUHVWRUHRSHUDWLRQVVHHWKH2UDFOHZKLWHSDSHU³Backup
and Recovery Performance and Best Practices Using the Sun ZFS Storage Appliance with the Oracle Exadata
Database Machine´3OHDVHUHIHUWR0y Oracle Support Note 1354980.1 to get details on the recommended
software stack for Oracle ZFS Storage Appliance when used with Oracle Exadata. Refer to the Oracle white paper
³Configuring a Sun ZFS Backup Appliance with Oracle SPARC SuperCluster´ for more details on how to configure
Oracle ZFS Storage Appliance for use with Oracle SuperCluster to provide maximum throughput and availability.
Storage Configuration Best Practices Configuring Oracle ZFS Storage Appliance storage pools assigns physical disk drive resources to logical storage
pools for backup data storage. When backing up and recovering Oracle Database to Oracle ZFS Storage Appliance,
Oracle recommends that Oracle ZFS Storage Appliance disk resources be configured in two equally sized pools.
Configure the two pools by assigning half of the physical drives in each drive tray to each storage pool to maximize
system throughput.
Once the pools have been created, they should be placed under the control of the Oracle ZFS Storage Appliance
cluster to protect against the failure of a network component supporting the appliance or a component within the
28 | ORACLE OPTIMIZED SOLUTION FOR SECURE BACKUP AND RECOVERY
appliance controller itself. Doing so ensures that an Oracle RMAN database backup or restore operation can
continue to run. For more details, see the Oracle white pDSHU³Backup and Recovery Performance and Best
Practices Using the Sun ZFS Storage Appliance with the Oracle Exadata Database Machine´RUWKH Oracle white
SDSHU³Oracle Optimized Solution for Secure Backup and Recovery: Oracle SuperCluster Backup and Recovery´
Configuring Oracle RMAN Projects and Shares on Oracle ZFS Storage Appliance Share configuration is the process of setting up and running NFS mount points for client access. A project is an
entity that provides a higher-level management interface point for a collection of shares. Two projects should be
created for an Oracle Database 11g Release 2 Oracle SuperCluster configuration²one project per pool. To
optimize share management, update the default mount point for shares contained in the project to reference the
database name. To optimize a system for performance, create four shares for each project in each pool, for a total
of eight shares (four on each head).
Oracle provides the Engineered Systems Backup Configuration Utility²downloadable from Oracle Technology
Network at the Oracle ZFS Storage Appliance Plug-In Downloads page²that automates the process of creating
Oracle ZFS Storage Appliance projects and shares and creates the corresponding entries. The utility also creates
Oracle RMAN command files that can be used for backing up the database. Running the utility ensures that all of
the best practices of using Oracle ZFS Storage Appliance as an Oracle Exadata backup destination are followed.
» Create two projects, one per pool.
» &RQILJXUHWKHSURMHFWVWRXVHDZULWHELDVRI³WKURXJKSXW.´
» Configure the project¶s NFS Exceptions to allow access to the shares by the Oracle engineered system.
Database Backup and Restore Best Practices with Oracle ZFS Storage Appliance
and Oracle Engineered Systems
The following key practices should be implemented in order to obtain maximum availability and performance from
Oracle ZFS Storage Appliance. Note, however, that for any backup operation or changes to backup practices, IT
staff should perform testing to ensure that the backups complete within the available backup window and that the
database restore and recovery operations can meet recovery time objective requirements.
Configure Network Connectivity
Do the following to configure network connectivity:
» Enable Direct NFS. An Oracle Database instance running on an Oracle engineered system is backed up to
Oracle ZFS Storage Appliance using the Oracle Database 11g Release 2 Direct NFS option. Note, however, that
by default, the Direct NFS option is not enabled, and enabling Direct NFS requires a database shutdown in order
to relink the Oracle Database kernel. By enabling Direct NFS, the operating system kernel NFS processes are
bypassed and the Direct NFS utility opens up 1 MB network buffers solely for use by Oracle RMAN, bypassing
the system configured TCP network buffers, and speeding up the database backup and restore. For details, see
the Oracle white SDSHU³%DFNXSDQG5HFRYHU\3HUIRUPDQFHDQG%HVW3UDFWLFHV8VLQJWKHSun ZFS Storage
Appliance ZLWKWKH2UDFOH([DGDWD'DWDEDVH0DFKLQH´RUWKH2UDFOHZKLWHSDSHU³2UDFOH2SWLPL]HG6ROXWLRQIRU
Secure Backup and Recovery: Oracle SuperCluster Backup and Recovery´
» Configure oranfstab. It is not essential to create an oranfstab file for Oracle Database to utilize Direct
NFS. However, when using any of the following configurations, an oranfstab file must be created to fully utilize
the network resources between the database nodes in Oracle Exadata or Oracle SuperCluster and Oracle ZFS
Storage Appliance.
» An Oracle engineered system is to be connected to the Oracle ZFS Storage Appliance using InfiniBand
» An Oracle engineered system is running Oracle Solaris
» An active/active IPMP group is created on Oracle ZFS Storage Appliance
29 | ORACLE OPTIMIZED SOLUTION FOR SECURE BACKUP AND RECOVERY
Refer to My Oracle Support Note 1517107.1, RMAN Backup from SPARC SuperCluster to Oracle ZFS Storage
Appliance, for sample /etc/oranfstab files.
Follow Recommended Best Practices for Database Backup and Restore
Oracle recommends using a combination of Level 0 and Level 1 backup sets when backing up Oracle Database to
Oracle ZFS Storage Appliance. The frequency of backups is dictated by recovery time and recovery point objective
requirements. Note that when using Oracle ZFS Storage Appliance, Oracle does not recommend an incrementally
updated backup strategy using a combination of data file copies and incremental backup sets that are subsequently
merged into the data file copies. When an incrementally updated backup strategy is selected, the backup solution
becomes I/O bound during the merge process, and alleviating this bottleneck requires a significant number of
available disk spindles in order to achieve the required IOPS rate. For most IT departments, the following
recommendations should be sufficient:
» The Oracle Database FRA should remain on the Oracle Exadata Storage Servers that are part of the Oracle
Exadata or Oracle SuperCluster and should not be put on Oracle ZFS Storage Appliance.
» The weekly full level 0 backup should be written to Oracle ZFS Storage Appliance.
» A daily incremental level 1 backup should be written to Oracle ZFS Storage Appliance. A maximum of 16 Oracle
RMAN channels should be allocated and configured for both weekly level 0 and daily level 1 backups to write to
one of the 16 Oracle ZFS Storage Appliance shares created using the Oracle Engineered Systems Backup Utility
mentioned earlier (or created manually).
» Two separate backup jobs should be submitted if Oracle RMAN compression is used to preserve space on the
Oracle ZFS Storage Appliance but the database consists of a mixture of uncompressed data and compressed
data or data that cannot be compressed further. One of the backup jobs will create an uncompressed backup, and
the second will create a compressed backup. Otherwise, compressing data that does not typically yield good
compression lengthens the backup window significantly, consumes unnecessary CPU resources, and results in
very little space savings.
» Oracle RMAN offers an option to amortize the backup over a given time period and minimize the performance
impact to critical applications. For example, if there is an eight-hour backup window available for the weekly
level 0 backup of a 20 TB database, but critical database functions are still running during that backup window,
the Oracle RMAN duration minimize load option can spread the work over the entire backup window.
» The use of Oracle RMAN options such as sectionsize is recommended. These options are included in the
backup scripts generated by the Oracle Exadata Backup Configuration Utility. The sectionsize parameter
breaks up BIGFILE tablespaces into more manageable amounts. Oracle recommends that the largest
tablespaces be broken up into equal size sections for each Oracle RMAN channel. For instance, if there are two
tablespaces of 10 TB and 14 TB, and there are 16 Oracle RMAN channels, then the sectionsize parameter
should be set to approximately 320 GB.
» The Oracle Exadata Backup Configuration Utility does not enable Oracle RMAN or Oracle ZFS Storage Appliance
compression, because this is something that is dependent on the data to be backed up and the database
licensing options purchased. Enabling Oracle RMAN compression increases CPU utilization.
Tune the Oracle Database Instance for Oracle RMAN
Optimizing high-bandwidth backup and restore operations using Oracle RMAN and Oracle ZFS Storage Appliance
requires adjusting the instance parameters that control I/O buffering. For information about how to tune these
parameters, see My Oracle Support Note 1072545.1 ³RMAN Performance Tuning Using Buffer Memory
Parameters´ at http://support.oracle.com.
30 | ORACLE OPTIMIZED SOLUTION FOR SECURE BACKUP AND RECOVERY
Configure Oracle RMAN
Performance benefits can be realized by configuring 16 or 32 Oracle RMAN channels to be evenly distributed over
the Oracle Database instances and nodes in the Oracle RAC cluster. The channels also should be evenly
distributed over the shares exported from Oracle ZFS Storage Appliance.
Best Practices Using Oracle ZFS Storage Appliance
Traditional backups are performed with a base-level full backup and then backups at subsequent intervals to capture
only the files that have changed. This can be an inefficient method because restore operations must start with the
full backup and apply each incremental in the proper order. Also, any and all incremental backups must be applied
or the restore will not be complete.
When using external Oracle ZFS Storage Appliances for backups, it is possible to follow a particular strategy as
shown in Figure 13. Backup operations should start with a full level 0 backup on the first day. On the second day, IT
staff can perform an incremental (level 1) backup to record just the blocks that have changed from the previous day,
or they have the option to apply the level 1 incremental to the full level 0 backup, as shown in Figure 14 (using Day 3
as an example). The incremental applied backup merges any changes since the full backup and creates a single
restore point. This is a much more efficient method than the traditional combination of full and incremental backups
typically used to obtain an up-to-date restore.
Figure 13. Oracle¶V suggested backup strategy for databases.
Use ZFS Snapshots
The previous procedure can be taken a step further by utilizing the snapshot capabilities of Oracle ZFS Storage
Appliance, as illustrated in Figure 14. It starts again with a full level 0 backup on Day 1 and an incremental backup
on Day 2. On Day 3, the incremental backup is applied to the level 0 backup from the first day, and a snapshot is
taken of the applied incremental, resulting in an image that looks like a version of a full backup. This consumes a lot
less space because it is recording only changes.
31 | ORACLE OPTIMIZED SOLUTION FOR SECURE BACKUP AND RECOVERY
Figure 14. Oracle¶V suggested backup strategy with Oracle ZFS Storage Appliance.
Using Non-Oracle Exadata Storage for Oracle SuperCluster and Oracle Exadata
Database Machine Backups
While not recommended, non-Oracle Exadata storage solutions might be necessary in some cases. Differences in
network connectivity, storage configuration, and protocols used by non-Oracle Exadata storage will affect backup
performance and contribute complexity. The Oracle Maximum Availability Architecture white paper at
oracle.com/technetwork/database/features/availability/maa-tech-wp-sundbm-backup-final-129256.pdf contains
suggestions on how to avoid the pitfalls of non-Oracle Exadata storage in these environments.
For configuration details and up-to-date changes, please see My Oracle Support Note 1558851.1.
For more information on Oracle's Maximum Availability Architectures, please see the following resource:
oracle.com/au/products/database/maa-096107.html
Providing Backup and Recovery Savings and High Availability with Remote DR
Sites
Figure 15 shows the ultimate solution for providing the ability to restore IT operations while spending less for backup
and recovery. Traditional solutions require expensive backup and recovery equipment to restore the production
system as quickly as possible. An alternative solution is to have a second remote site with replicated applications
and data. Using primary and secondary remote sites eliminates the need for high-performance systems to recover
production operations. Instead of deploying Oracle SuperCluster and a high-performance disk array to back it up, a
secondary Oracle SuperCluster with Oracle's lower cost Oracle ZFS Storage Appliance or tape can be deployed. In
the event of a failure, production can be switched over immediately to the remote site. The primary site's recovery
time is irrelevant because production continues on the secondary systems. This approach provides better business
continuance and results in significant savings because backup and recovery system costs are lower.
When not running production operations as part of a business continuance effort, the remote replicated location can
be used for testing, development, and other functions. Oracle is developing the ability to generate reports from a
copy of the secondary site's production database, enabling IT staff to offload work from the production system and
keep the secondary database current. This solution provides savings for backup and recovery and maximizes
system availability. Many organizations distribute their recovery systems in this manner, with applications that can
32 | ORACLE OPTIMIZED SOLUTION FOR SECURE BACKUP AND RECOVERY
fail over from city to city. In fact, this design can be used almost universally, with the exception of firms that are
required to have large and elaborate backup and recovery systems for compliance reasons.
Figure 15. Oracle technologies enable maximum protection plus cost savings.
6LWHVWKDWHPSOR\2UDFOH¶V=HUR'DWD/RVV5HFRYHU\$SSOLDQFHIRUGDWDEDVHEDFNXSVFDQEHQHILWIURPWKHVHFXUH
replication features of the recovery appliance. Backups on a local recovery appliance can be easily and quickly
replicated via secure transport to a remote recovery appliance. Recovery operations can be run directly from the
remote recovery appliance, without first staging the data locally.
Conclusion
When deploying Oracle engineered systems, it is vital to update backup and recovery platforms at the same time,
because the resulting production environment will exhibit superior performance with far greater data throughput and
scalability. Backup and recovery solutions that are not designed to handle the unique performance characteristics of
Oracle engineered systems are unable to keep pace and likely will become the limiting factor in protecting against
data loss and maintaining suitable service levels.
Oracle Optimized Solution for Secure Backup and Recovery, powered by Oracle servers, provides an excellent
backup solution for heterogeneous networks. A comprehensive end-to-end data archive solution with centralized
management for simplified administration, Oracle Optimized Solution for Secure Backup and Recovery provides a
scalable architecture that easily adapts to growing storage demands. Indeed, the high performance and capacity of
this solution facilitates the consolidation and migration of existing SAN backups to more cost-efficient network
backups.
With a highly flexible architecture, Oracle Optimized Solution for Secure Backup and Recovery features virtually
unlimited scalability. The architecture supports multiple media servers working together under the administrative
control of an administrative server. Additional media servers can be deployed or multiple backup domains can be
configured to address evolving scalability requirements. Organizations using Oracle Optimized Solution for Secure
33 | ORACLE OPTIMIZED SOLUTION FOR SECURE BACKUP AND RECOVERY
Backup and Recovery can implement cost-effective and powerful backup solutions for the largest, most diverse data
center environments.
For configuration information, recommendations, sizing information, and all the latest information on Oracle
Optimized Solution for Secure Backup and Recovery, please refer to My Oracle Support Note 1558851.1.
References
For additional information, please visit the URLs listed in Table 6.
TABLE 6. REFERENCES FOR MORE INFORMATION Description
URL
Oracle Secure Backup software
oracle.com/technetwork/products/secure-backup/overview
Oracle servers
oracle.com/us/products/servers/overview
Oracle Solaris 11
oracle.com/technetwork/server-storage/solaris11/overview
Oracle Solaris Cluster
oracle.com/technetwork/server-storage/solaris-cluster/overview
Oracle Linux
oracle.com/us/technologies/linux/overview
Oracle Enterprise Manager
oracle.com/us/products/enterprise-manager/index
StorageTek Automated Cartridge System Library
Software
oracle.com/us/products/servers-storage/storage/tape-storage/acsls/overview
Oracle Key Manager
oracle.com/us/products/servers-storage/storage/storage-software/oracle-keymanager/overview
Oracle storage systems
oracle.com/us/products/storage/overview
Oracle Optimized Solutions
oracle.com/optimizedsolutions
Oracle Support Document 1558851.1 (Oracle
Optimized Solution for Secure Backup and Recovery)
https://support.oracle.com/epmos/faces/DocumentDisplay?id=1558851.1
34 | ORACLE OPTIMIZED SOLUTION FOR SECURE BACKUP AND RECOVERY
Oracle Corporation, World Headquarters
Worldwide Inquiries
500 Oracle Parkway
Phone: +1.650.506.7000
Redwood Shores, CA 94065, USA
Fax: +1.650.506.7200
CONNECT W ITH US
blogs.oracle.com/oracle
facebook.com/oracle
twitter.com/oracle
oracle.com
Copyright © 2015, Oracle and/or its affiliates. All rights reserved. This document is provided for information purposes only, and the
contents hereof are subject to change without notice. This document is not warranted to be error-free, nor subject to any other
warranties or conditions, whether expressed orally or implied in law, including implied warranties and conditions of merchantability or
fitness for a particular purpose. We specifically disclaim any liability with respect to this document, and no contractual obligations are
formed either directly or indirectly by this document. This document may not be reproduced or transmitted in any form or by any
means, electronic or mechanical, for any purpose, without our prior written permission.
Oracle and Java are registered trademarks of Oracle and/or its affiliates. Other names may be trademarks of their respective owners.
Intel and Intel Xeon are trademarks or registered trademarks of Intel Corporation. All SPARC trademarks are used under license and
are trademarks or registered trademarks of SPARC International, Inc. AMD, Opteron, the AMD logo, and the AMD Opteron logo are
trademarks or registered trademarks of Advanced Micro Devices. UNIX is a registered trademark of The Open Group. 0615
Oracle Optimized Solution for Secure Backup and Recovery
October 2015
Author: Dean Halbeisen