® Using Recovery Manager Connie Dialeris, Server Technologies cdialeri@us.oracle.com Last Updated: 8th July 1997 Agenda and Aims • • Agenda – Overview and Features – Recovery Manager Components and Architecture – Configuring Recovery Manager – Performing Backups – Performing Restoration – Recovery – Catalog Maintenance – Running Recovery Manager Commands – Reports on the Recovery Catalog – Miscellaneous Aims – To understand what Recovery Manager is – How to configure Recovery Manager – How to perform backups, restores and recoveries • Note – This presentation will not show all Recovery Manager features and syntax – For detailed information on Recovery Manager, please see the Oracle8 Backup and Recovery Guide Recovery Manager - Connie Dialeris 2 Overview and Features Recovery Manager - Connie Dialeris 3 What is Recovery Manager • Recovery Manager is a tool that: – manages the process of creating backups, and also manages the process of restoring and recovering from them – directs the Oracle8 server to back itself up, and restore and recover itself • The features of Recovery Manager are also available through the Oracle Enterprise Manager - Backup Manager GUI • Recovery Manager is a Command Line Interface (similar to SQL*DBA) – Provides a powerful, Operating System independent scripting language – Interactive or Batch Mode Recovery Manager - Connie Dialeris 4 Features • Incremental Backups (up to 4 levels. Level 0 and levels 1->4) • Corrupt block detection – during backup • v$backup_corruption, v$copy_corruption • also reported in the database’s alert log and trace files – restore • Easily Manages distributing database backups, restores and recoveries across clustered nodes in an Oracle Parallel Server • Performance – Automatic Parallelization of backup, restore and recovery – Multiplexing prevents flooding any one file with reads and writes while still keeping a tape drive streaming – Backups can be restricted to limit reads per file, per second to avoid interfering with OLTP work – No generation of extra redo during open database backups Recovery Manager - Connie Dialeris 5 Features • Easy backup of Archived redo logs • Able to limit: – Number of open files • Overcome O/S limits on concurrent files opened • Default 32 – Size of backup piece • This is important, as a backup piece size may easily be over the maximum file size allowed by the Operating System, or Media Manager Recovery Manager - Connie Dialeris 6 Components and Architecture Recovery Manager - Connie Dialeris 7 Overview of Oracle Database Components • • Instance (memory structures and processes) Control File (one or more multiplexed) server server server server Redo Log 116 Current Online Log SMON PMON DBWR LGWR ARCH Instance prod SGA Redo Log 117 Redo Log 118 Online Log • • • One or more Data Files Online Redo Log Archived Redo Log (only if in ARCHIVELOG mode) CF CF CF File 1 File 2 File 3 File 4 File 5 Database Recovery Manager - Connie Dialeris Redo Log 110 Redo Log 114 Redo Log 111 Redo Log 115 Redo Log 112 Redo Log 116 Redo Log 113 Archived Redo Log 8 Recovery Manager Components • Components of Recovery Manager – – – – Recovery Manager executable (rman) Target Database Recovery Catalog (or database’s control file) Channel • Recovery Manager – Translates Recovery Manager commands into a sequence of steps which operate on physical files – The Backup, Restore and Recovery Steps are sent to the target database for execution – If a Recovery Catalog is used, Recovery Manager generates steps which update the catalog – Recovery Manager co-ordinates and monitors the execution of these steps Recovery Manager - Connie Dialeris 9 Recovery Manager Components • Target Database – The database on which the specified backup, restore and recover actions will be executed • Recovery Catalog – Is a repository of information queried and updated by Recovery Manager – Is an Application, stored in an Oracle database – Contains information about: • • • • • Physical schema of the target database Datafile and Archivelog backup sets and pieces Datafile Copies Archived Redo Logs Stored scripts – Recovery Manager uses information in the Recovery Catalog to decide how to perform certain operations (if a Recovery Catalog is not used, Recovery Manager can query the target database’s control file to decide what actions to perform). Recovery Manager - Connie Dialeris 10 Recovery Manager Components • Channel – Allocating a channel tells Recovery Manager to initiate an Oracle server process for backup/restore/recovery on the target database – The number of channels allocated will be the maximum degree of parallelization – The type of channel allocated will determine the type of media used • type disk • type ‘SBT_TAPE’ – Channels can be used to impose limits: • read rate (number of buffers read per second, per file) • backup piece file size • maximum concurrently open files Recovery Manager - Connie Dialeris 11 Recovery Manager Architecture Recovery Manager server Recovery Catalog channel SBT_TAPE server (default) prod SGA Recovery Catalog CF SCF File 1 server (polling) File 2 File 3 server MML File 4 Target Media Manager Recovery Manager - Connie Dialeris 12 Configuring Recovery Manager Recovery Manager - Connie Dialeris 13 Configuring Recovery Manager • You must have Oracle8 installed. Recovery Manager is distributed with Oracle8. • Decisions required to configure Recovery Manager – Decide whether you will use Oracle password files You need to use password files if you: • Intend to connect to the target database remotely (for example, if you wish to use the Oracle Enterprise Manager Backup GUI) • Would like to specify a TNS alias on the connect string (for example, scott/tiger@tns_alias) – Decide how to backup supporting O/S files, such as init.ora files and password files – Decide whether to use a Recovery Catalog (discussed in more detail) Recovery Manager - Connie Dialeris 14 Deciding Whether to Use a Recovery Catalog • It is possible for Recovery Manager to use just the target databases’ control file. • Using RMAN without a catalog – There is no extra overhead of a Recovery Catalog – Does not allow easy Point-In-Time Recovery – If you lose your control file and need to restore and recover the database, restore and recovery are impossible without the assistance of Oracle World Wide Customer Support. WWCS can only help if the user maintains excellent records (I.e. the rman message log files) of which files were backed up, the date the backup occurred and the names of the output files – Optimally, the user should keep current backups of his controlfile in multiple locations that do not need to be restored using Recovery Manager – Stored scripts are not available • Oracle recommends the use of a Recovery Catalog for databases with more than 20 files. Recovery Manager - Connie Dialeris 15 Creating a Recovery Catalog • Choose whether to create a separate database for the Recovery Catalog, or install it in an existing database. – If you decide to create a separate database, then create it using the same procedures you would to create any Oracle database. – Cautions: • Do not install the recovery catalog in the same database as the target database • Do not put this database on the same physical disks as the target database • Decide whether to create a separate tablespace for the Recovery Catalog. Start Server Manager, and: SVRMGR> connect internal SVRMGR> create tablespace rcvcat datafile 2> ‘/oracle/rcvcat/prod/rcvcat_1.dbf’ size 20m; Recovery Manager - Connie Dialeris 16 Creating a Recovery Catalog • Create the schema owner (this example assumes there is a temp tablespace), and grant the required privileges. Start Server Manager, and: SVRMGR> connect internal SVRMGR> create user rman identified by rman 2> temporary tablespace temp 3> default tablespace rcvcat 4> quota unlimited on rcvcat; SVRMGR> grant recovery_catalog_owner to rman; • Create the Recovery Catalog Application. Start Server Manager, and: SVRMGR> connect rman/rman SVRMGR> @?/rdbms/admin/catrman Before continuing, check to ensure no errors were reported when running the script. Recovery Manager - Connie Dialeris 17 Creating a Recovery Catalog • If you use a Recovery Catalog, you must decide: – How, and how frequently, you will backup the Recovery Catalog – How to automate Catalog resyncs, and how often Use a utility such as cron (on Unix) to schedule regular resyncs • Irrespective of whether a Recovery Catalog is used, Recovery Manager sometimes needs to make a temporary backup of the controlfile to read. The location of this file is configurable: RMAN> set snapshot controlfile name to ‘/dev/vgd01/r300_1’; The default location of the snapshot controlfile on Unix systems is: $ORACLE_HOME/dbs/snapcf_<dbname>.f, where <dbname> is the database name. Recovery Manager - Connie Dialeris 18 Linking with a Media Manager • To be able to write to sequential devices such as Tape, you must link the Oracle server software with software which is written by a Media Manager. • This software (usually known as the Media Management Library, or Media Management Layer) is called by Oracle when it needs to write to, or read from devices supported by the Media Manager. • An example of linking Oracle on Solaris, with an MML (the MML is libobk.so, and is located in /usr/lib): > cd $ORACLE_HOME/rdbms/lib > make -f ins_rdbms.mk ioracle LLIBMM=/usr/lib/libobk.so You should shutdown all databases using that executable set before linking. Recovery Manager - Connie Dialeris 19 Starting Recovery Manager • There are a number of ways to start Recovery Manager, depending on how you have configured it. Below are a few examples: • Starting Recovery Manager locally (without a password file) – Without using a Recovery Catalog > setenv ORACLE_SID prod > rman nocatalog RMAN> connect target – Without a Recovery Catalog, if SCOTT has SYSDBA role > setenv ORACLE_SID prod > rman target scott/tiger nocatalog – Starting rman locally, with a Catalog > setenv ORACLE_SID prod > rman rcvcat rman/rman@rcat RMAN> connect target Where: • The target database instance is called ‘prod’, and we are connecting internal using a password file • The schema containing the Recovery Catalog is rman, with the same password, and it’s in a database called rcat Recovery Manager - Connie Dialeris 20 Starting Recovery Manager • Starting rman using TNS alias (e.g. remotely): – Starting rman, connecting to a SCOTT with SYSDBA priv, without a catalog > rman target scott/tiger@prod nocatalog – Starting rman when you use password files, without a catalog: > rman target internal/kernel@prod nocatalog – Starting rman if you use password files and a recovery catalog > rman target internal/kernel@prod rcvcat rman/rman@rcat Where: • The target database instance is called ‘prod’, and we are connecting internal using a password file • The schema containing the Recovery Catalog is rman, with the same password, and it’s in a database called rcat Recovery Manager - Connie Dialeris 21 Registering a Database in a Recovery Catalog • If using a Recovery Catalog, the first time you use Recovery Manager to perform any backups, you must register the Database with the Recovery Catalog. The target database you connect to will be the database registered in the catalog. RMAN> register database; • The register database command identifies the database to the Recovery Catalog. – All information about the target database’s structure is propagated to the Recovery Catalog from the target database’s control file. – The database must be mounted or open to perform this action. Recovery Manager - Connie Dialeris 22 Performing Backups Recovery Manager - Connie Dialeris 23 Backup Overview • The database can be Open or Closed during backup – If you are performing a closed backup, the database must be mounted, but not open SVRMGR> shutdown immediate SVRMGR> startup mount – If you are performing a backup while the database is open, the tablespaces should not be put in ‘hot backup’ mode (I.e. you should not issue an alter tablespace begin backup command). – Recovery Manager allows you to backup the whole database, or any logical part thereof (irrespective of whether the database is open or closed) • • • • database tablespace datafile control file • Archived logs may also be backed up Recovery Manager - Connie Dialeris 24 New & Changed Terminology • Whole backup A backup of all datafiles and the control file. • Full backup This replaces the term ‘full database backup’. A backup of one or more files, which is not an incremental. • Incremental backup A backup of datafiles which only includes the blocks which have changed since the last incremental. The base level 0 incremental will backup all blocks containing data. • Operating System Backup An Oracle7 style file backup, made by Operating System Utilities. • Open Backup A backup of any portion of the database while it is open (I.e. online). • Closed Backup A backup of any portion of the database while it is closed (I.e. offline). Recovery Manager - Connie Dialeris 25 Types of Backups • There are two types of Recovery Manager backups: – File Copies – Backup Sets • Recovery Manager automatically resynchronizes database structure information in the Recovery Catalog before executing certain commands (e.g. backup) • Recovery Manager propagates changes in the databases’ structure to the recovery catalog, before it executes certain commands (datafiles, resized files, or dropped tablespaces) Recovery Manager - Connie Dialeris 26 File Copy • A File Copy is created by the copy command and – – – – – Is an image copy of the file Can only be written to disk Contains only a single input file (datafile, archivelog or controlfile) Is most like an Operating System backup (contains all blocks) Can be a part of an incremental strategy; but a file copy always includes all blocks, and hence can only be a level 0 run { allocate channel d1 type disk; copy level 0 datafile 1 to ‘/oracle/prod/backup/file1.dbf’; } – Is usable immediately - does not need to be restored Datafile Copies can be used in-situ by issuing a switch command, or can be restored to the original location, if desired Recovery Manager - Connie Dialeris 27 Parallelization of File Copies • Parallelization of File Copies is achieved by copying multiple files within one copy command, as well as allocating multiple channels. run { allocate channel d1 type disk; allocate channel d2 type disk; allocate channel d3 type disk; allocate channel d4 type disk; copy #first <- RMAN comment datafile 1 to ‘/oracle/backup/prod1/d1/prd1.dbf’, datafile 2 to ‘/oracle/backup/prod1/d2/prd2.dbf’, datafile 3 to ‘/oracle/backup/prod1/d3/prd2.dbf’; copy #second datafile 4 to ‘/oracle/backup/prod1/d4/prd3.dbf’; sql ‘alter system archive log current’; } • Note that recovery manager executes commands serially. I.e. the previous command must finish before the next command is started. In this example, the first copy command will perform 3 copies in parallel. The second copy command is not executed until the command before it completes. Recovery Manager - Connie Dialeris 28 Backup Set • A Backup Set is created by the backup command and – Usually contains multiple source files (multiplexed in the output) – It is a logical entity, comprising one or more physical output files called backup pieces – Can Be Written to Disk or ‘SBT_TAPE’ – A restore operation is required to extract files from a backup set – There are two types of Backup Sets • Datafile backup sets (can include control files) • Archivelog backup sets (may only contain archivelogs) • Datafile backup sets – Can be Incremental or Full (are full, by default) – Do not include empty blocks Recovery Manager - Connie Dialeris 29 Backup Set - example • Whole database backup, of a database with 10 files (including the control file), writing out to two tape drives • The number of files per backup set will be restricted to 5, which means that two sets will be created, each with 5 files multiplexed. • If two channels are allocated, these sets will be created in parallel run { allocate channel t1 type ‘SBT_TAPE’; allocate channel t2 type ‘SBT_TAPE’; backup filesperset 5 format ‘df_%t_%s_%p’ (database); } • The format string determines the output file name (passed to sbtopen) • Recovery Manager will automatically perform the partitioning of files to channels, multiplex the files, and skip any unused blocks Recovery Manager - Connie Dialeris 30 Parallelization of Backup Sets • Parallelization of Backup Sets is achieved by allocating multiple channels, and specifying filesperset. – Recovery Manager establishes one database connection (server process) to the target database per channel – In this example, if the keyword filesperset is not included only one channel would be used, and that channel would create one piece which contains all five files. The other two channels would be idle. – Recovery Manager uses all 3 tape drives, each one to write a backup set, each set with a maximum of 1 file. run { allocate channel t1 type ‘SBT_TAPE’; allocate channel t2 type ‘SBT_TAPE’; allocate channel t3 type ‘SBT_TAPE’; backup format ‘df_%t_%s_%p’ filesperset 1 (datafile 10,11,12); sql ‘alter system archive log current’; } Recovery Manager - Connie Dialeris 31 Backup Pieces • A Backup Set is a logical entity, which is composed of one or more physical output files, called backup pieces • Typically, a backup set only has one output piece • When using ‘SBT_TAPE’ the only time a backup set is made of more than one piece, is when the piece size is limited by the user e.g. Tablespace backup to one tape drive, max. file size is 4Gb run { allocate channel t1 type ‘SBT_TAPE’; set limit channel t1 kbytes 4194304; backup format ‘df_%t_%s_%p’ filesperset 5 (tablespace user_data); } If the output file is < 4Gb, only one backup piece will be written. If the output size is > 4Gb, more than one backup piece will be written. Each backup piece will have blocks from five files interspersed together Recovery Manager - Connie Dialeris 32 Incremental Datafile Backup • Is a backup of a datafile which only copies out Oracle blocks which have been modified since a previous incremental – An incremental backup at level N (where N > 0) backs up all blocks modified since the previous incremental at a level <= N • • • • Are initially based on a Level 0 backup set, or a Level 0 File Copy Are faster than performing full (or level 0) backups Write out fewer blocks than when performing full (or level 0) backups Are, by default non-cumulative changed blocks all nonempty blocks Level Day 0 Sun 2 Mon 2 Tue 1 Wed 2 Thu Recovery Manager - Connie Dialeris 2 Fri 2 Sat 0 Sun 33 Cumulative Incremental Backups • Contain all blocks changed since the previous backup at a level less than N • Are provided for recovery speed • Cumulative Incrementals backup blocks previously backed up at the same level – May take longer and will write out more blocks than non-cumulative – However fewer backups at each level must be applied when recovering changed blocks all nonempty blocks Level Day 0 Sun 2 Mon 2C Tue 1 Wed 2 Thu Recovery Manager - Connie Dialeris 2C Fri 2C Sat 0 Sun 34 Archivelog Backup Sets • Only include archivelogs • Are always full backups • As Recovery Manager has access to control file information, it knows which logs have been archived This prevents a common problem experienced by users, of not knowing whether an archive log has been completely copied out to the archive log destination before attempting to back it up Example run { allocate channel t1 type ‘SBT_TAPE’; backup format ‘al_%t_%s_%p’ filesperset 10 (archivelog all delete input); } Recovery Manager - Connie Dialeris 35 Backup Sets, File Copies and O/S backups • Backup Sets and File Copies can be written to disk, then later backedup to ‘SBT_TAPE’ (one or many times) This allows the most recent backups to be made to disk, then migrated to sequential media before reusing the disk space for a more current backup • Oracle8 continues to support making O/S Backups – Valid O/S backups on disk can be cataloged in the Recovery Catalog for immediate use e.g. alter tablespace begin backup, break the mirror, then catalog the broken mirror half, alter tablespace end backup. Note in the case of breaking mirrors to make O/S backups, you must immediately notify recovery manager that the backup is no longer available when you resilver the mirror. This can be done using a recovery manager change...unavailable command. – Once the O/S backups have been cataloged, they can then be backed up by Recovery Manager to disk or to the MML Recovery Manager - Connie Dialeris 36 Restore Recovery Manager - Connie Dialeris 37 Restore Concepts • To reconstruct, or bring back an original file from backup • Recovery Manager allows you to restore any logical unit of a database: – – – – database tablespace datafile control file restore (database); restore (tablespace <name>,..,<name>); restore (datafile <name>,..,<name>); restore controlfile to ‘<location>’; • Archived logs may also be manually restored, however typically Recovery Manager automatically restores them if required during Recovery. restore (archivelog all|like <filename>|<archivelog range>| etc…); Recovery Manager - Connie Dialeris 38 Restore Concepts • What state the database should be in for various recoveries: – If you need to restore the control file, the instance must be started, but not mounted: SVRMGR> startup nomount – If you need to restore the whole database, or the system tablespace, the database can be mounted, but not open: SVRMGR> startup mount – If you need to restore a tablespace or datafile, the database can be open but the tablespace or datafile must be offline: SVRMGR> alter tablespace user_data offline temporary; or SVRMGR> alter database datafile 2> ‘/oracle/prod/file/user_data_1.dbf’ offline; Recovery Manager - Connie Dialeris 39 Restore • Recovery Manager: – Queries the Recovery Catalog to determine the configuration of the database, and to find out what backups are available for use. – Decides which backups are the optimal ones to use to restore the specified files • When compiling lists of files to restore, Recovery Manager takes into consideration: – Time Specifications (e.g. for a Point-In-Time Restore) – Any tags specified by users – The channel type (disk/’SBT_TAPE’) allocated. To ensure Recovery Manager will find the most recent backups, allocate the appropriate channel type. – Note that Datafile Copies do not need to be restored, they can be used insitu by executing a ‘switch’ command, which directs the database to use this file as the actual file in the database. Recovery Manager - Connie Dialeris 40 Restore Example • Restore the whole database, including control file (to each file’s original location). As the whole database and control file are being restored, the database should be started, but not mounted. Start Server Manager, then: SVRMGR> shutdown immediate SVRMGR> startup nomount Recovery Manager - Connie Dialeris 41 Restore Example Then start Recovery Manager: rman target internal/kernel@prod rcvcat rman/rman@rcat RMAN> run { 2> allocate channel t1 type ‘SBT_TAPE’; 3> allocate channel t2 type ‘SBT_TAPE’; 4> allocate channel d1 type disk; 5> restore controlfile to ‘/oracle/prod/files/ctrl_prod_1.ctl’; 6> replicate controlfile from 7> ‘/oracle/prod/files/ctrl_prod_1.ctl’; 8> sql ‘alter database mount’; 9> restore (database); 10> } The database has been restored (but not recovered). If the backup is consistent, the database could be opened without the application of redo (I.e. the database can be opened without recovery, however, a fake recovery is needed to open the database resetlogs) Recovery Manager - Connie Dialeris 42 Parallelization of Restore • Parallelization of Restore is controlled by the number of channels allocated and the number of backup sets which are required for restore. – This applies equally for the restore of level 0 or full backups, as well as the application of incremental backups of levels > 0 – In the previous example: – If there were 12 backup sets which needed to be read to restore the whole database, it would have been faster to allocate 12 channels, thereby increasing the degree of parallelization. – However if there had been only one backup set which needed to be read to restore the whole database, the second channel would not have been used at all. Recovery Manager - Connie Dialeris 43 Preparing for Point-In-Time Recovery • If a user error has necessitated the database be restored then recovered to a point-in-time immediately before the error, you will need to specify an until clause. The until clause can be specified for each restore and recovery command in the script, but it can also be defined first - in this way it will apply to all commands executed in that session. • With an until clause, Recovery Manager will restore a backup taken before that time. If this clause is omitted, the most recent backups of the database will be restored. This means a subsequent ‘recover.. until’ command may fail, if the copy of the database restored is already ahead of the desired recovery point-in-time. • A ‘time’ to restore until may specified by: – a time – a log sequence number or – an SCN (System Commit Number) Recovery Manager - Connie Dialeris 44 Preparing for Point-In-Time Recovery: Example • Restore the whole database, including control file (to the original locations) until time ‘1997/06/15 14:20:00’. > setenv NLS_DATE_FORMAT ‘YYYY/MM/DD HH24:MI:SS’ > setenv NLS_LANG american SVRMGR> shutdown abort SVRMGR> startup nomount SVRMGR> exit rman target internal/kernel@prod rcvcat rman/rman@rcat RMAN> run { 2> set until time ‘1997/06/15 14:20:00’; 3> allocate channel t1 type ‘SBT_TAPE’; 4> allocate channel t2 type ‘SBT_TAPE’; 5> allocate channel d1 type disk; 6> restore controlfile to ‘/oracle/prod/files/ctrl_prod_1.ctl’; 7> replicate controlfile from 8> ‘/oracle/prod/files/ctrl_prod_1.ctl’; 9> sql ‘alter database mount’; 10> restore database; 11> #recover database; #recover should be in same job, if possible 12> } Recovery Manager - Connie Dialeris 45 Recovery Recovery Manager - Connie Dialeris 46 Recovery Concepts • Recovery is the process whereby – A restored file is made current or – A restored file is made current to a specific point in time • If the database is in ARCHIVELOG mode, it is possible to either completely recover the database, or recover the database to some arbitrary point-in-time (incomplete recovery) • For NOARCHIVELOG mode databases, the only option is to restore from a consistent whole database backup • If a database is closed during recovery, all files in that database must be consistent with each other before the database can be opened. • If a database is open during a tablespace or datafile recovery, the tablespace or datafile must be consistent with the remaining datafiles before it can be brought online. Recovery Manager - Connie Dialeris 47 Recovery Concepts • There are two types of recovery: – Complete recovery A recovery where all changes made to the database since the restored backup have been applied (including changes recorded in the online redo logs). – Incomplete recovery A recovery where the recovery is stopped before all changes made to the database since this backup have been applied. Incomplete recoveries are performed: • To undo a user error (recover to the point-in-time before the error occurred) • When all redo information to complete the recovery is not available (this can happen for archivelogs as well as online logs) • When the online logs are lost (and are not mirrored) A ‘time’ to recover to may be identified by a time, a log sequence number or an SCN (System Commit Number) All files must be recovered to the same point-in-time. Recovery Manager - Connie Dialeris 48 Phases of Recovery • Recovery Manager performs recovery in 4 phases: – If a controlfile was restored, repair it – Apply any incremental backups which will make the files newer (Incremental backups are applied to a level 0 or full) – Apply redo which is currently on disk – If all redo required for recovery is not on disk, restore archivelogs, then apply them and any redo in the online redo logs needed for recovery Recovery Manager - Connie Dialeris 49 Tablespace Recovery: Example • This example shows the restoration and recovery of all files in the user_data tablespace. Assume the database is open. rman target internal/kernel@prod rcvcat rman/rman@rcat RMAN> run { 2> allocate channel t1 type ‘SBT_TAPE’; 3> sql ‘alter tablespace user_data offline immediate’; 4> restore 5> tablespace user_data; 6> recover 7> tablespace user_data; 8> sql ‘alter tablespace user_data online’; 9> } • The tablespace is now available for use. Recovery Manager - Connie Dialeris 50 Parallelization of Recovery • The parallelization of applying incremental backups is controlled by the number of channels allocated and the number of incremental backup sets which are needed for recovery (this is exactly the same as parallelizing restore) • Redo application – Can be parallelised by setting the init.ora parameter RECOVERY_PARALLELISM – Archived Redo Logs can be manually restored to disk in preparation for Recovery, or Recovery Manager will restore them automatically as they are required The restore of archived logs can be parallelized by allocating multiple channels Recovery Manager - Connie Dialeris 51 Example of Recovery With Incrementals • If a file had been backed up with the schedule shown below, and it required recovery on Sunday, the following actions would be initiated by Recovery Manager: • Restore the previous Sunday’s level 0 backup – Apply the level 1 backup from Wednesday – Apply the level 2’s from Thursday, Friday and Saturday – Apply any archived redo generated after the Saturday backup, and then apply the online redo logs Time of Failure Restored file Increm. applied Not required Redo Level Day 0 Sun 2 Mon 2 Tue 1 Wed 2 Thu Recovery Manager - Connie Dialeris 2 Fri 2 Sat 0 Sun 52 Point-In-Time Recovery • To complete the PIT Restore example: > setenv NLS_DATE_FORMAT ‘YYYY/MM/DD HH24:MI:SS’ > setenv NLS_LANG american SVRMGR> shutdown abort SVRMGR> startup nomount SVRMGR> exit rman target internal/kernel@prod rcvcat rman/rman@rcat RMAN> run { 2> set until time ‘1997/06/15 14:20:00’; 3> allocate channel t1 type ‘SBT_TAPE’; 4> allocate channel t2 type ‘SBT_TAPE’; 5> allocate channel d1 type disk; 6> restore controlfile to ‘/oracle/prod/files/ctrl_prod_1.ctl’; 7> replicate controlfile from 8> ‘/oracle/prod/files/ctrl_prod_1.ctl’; 9> sql ‘alter database mount’; 10> restore database; 11> recover database; 12> } Recovery Manager - Connie Dialeris 53 Catalog Maintenance Recovery Manager - Connie Dialeris 54 Catalog Maintenance Commands • There are a number of Recovery Manager commands provided which will allow you to maintain (modify) information in the Recovery Catalog. • Resyncing the Recovery Catalog RMAN> resync catalog; Causes Recovery Manager to compare the recovery catalog to the target database’ controlfile. The Catalog is then updated with new or changed information. Should be performed: – Each time the physical database structure changes e.g. • Addition (or resize) of datafile • Tablespace drop • Rollback segment create or drop – At least once a day (depending on how many archived logs are produced, and whether automatic file resize is turned on) Recovery Manager - Connie Dialeris 55 Catalog Maintenance Commands • The change command is used to update the availability information in the Recovery Catalog e.g. – To mark a backup piece, file copy or archivelog as unavailable or available for restore or recovery RMAN> change archivelog 2> ‘/disk1/archivelog/al_prd1_123.rdo’ unavailable; – To delete a backup piece, file copy or archivelog which is no longer required from the controlfile, recovery catalog and from the media it resides on – RMAN> allocate channel t1 for delete type ‘SBT_TAPE’; RMAN> change backuppiece <key> delete; – To remove references to a backup piece, file copy or archivelog from the controlfile and recovery catalog (e.g. if it was deleted by an O/S utility) – RMAN> change archivelog 2> ‘/disk1/archivelog/al_prd1_123.rdo’ uncatalog; Recovery Manager - Connie Dialeris 56 Catalog Maintenance Commands • If the DBA has deleted a large number of files on disk (e.g. archived logs), a more efficient method of reconciling the Recovery Catalog (and controlfile) with what is on disk, is to use the validate keyword RMAN> change archivelog all validate; This verifies that the files the Recovery Catalog believes are on disk, are there. If they are not there, these files are identified as deleted in the catalog. Recovery Manager - Connie Dialeris 57 Catalog Maintenance Commands • The catalog command: – If there are archivelogs, datafiles copies etc. on disk which were created before Recovery Manager was used, it is possible to identify the existence of these to the Recovery Catalog: RMAN> catalog archivelog ‘/disk1/archivelog/al_prd1_1234.rdo’; – This command is also used to identify O/S backups made to disk in the Recovery Catalog. The creation of O/S backups by breaking mirrors (with alter tablespace begin/end backup) Once the mirror has been resilvered, the backup cataloged must be uncataloged using a change...uncatalog command. Recovery Manager - Connie Dialeris 58 Running Recovery Manager Commands Recovery Manager - Connie Dialeris 59 Running Recovery Manager commands • Can be run by: – Typing them into the command line interface (as we have seen in all examples) RMAN> run { 2> allocate channel c1 type disk; 3> copy 4> datafile 10 to ‘/oracle/prod/backup/prod_10.dbf’; 5> } – Typed into an Operating System file, which is then invoked on the Recovery Manager command line The script above is typed in to an O/S file called copy_10, then is called when invoking rman: rman target internal/kernel@prod rcvcat rman/rman@rcat cmdfile copy_10 This is a good method if you plan to schedule backups using O/S utilities (e.g. cron) Recovery Manager - Connie Dialeris 60 Running Recovery Manager commands • The commands can also be stored in the Recovery Catalog, with a name. It is then called a Stored Script. A stored script is only associated with one target database. RMAN> create script copy_10 { 2> allocate channel c1 type disk; 3> copy 4> datafile 10 to ‘/oracle/prod/backup/prod_10.dbf’; 5> } To execute the stored script: RMAN> run { 2> execute script copy_10; 3> } Recovery Manager - Connie Dialeris 61 Lists and Reports on the Recovery Catalog Recovery Manager - Connie Dialeris 62 List Command • Queries the Recovery Catalog and produces a listing of it’s contents. You can list: – All databases known to the recovery catalog or incarnations of a database RMAN> list incarnation of database; – Backup Sets or File Copies containing a backup of a specified list of files RMAN> list backupset of datafile 2; RMAN> list backupset of datafile ‘/d3/dbf/user_prd1.dbf’; – Backup Sets or File Copies containing a backup of any datafile which is a member of a list of tablespaces RMAN> list copy of tablespace system; – Backup Sets or File Copies of the whole database, optionally restricted by time, datafile copy name, device name and tag – Backup Sets containing Archivelogs restricted by name, or by range. For example, list which backup sets contain archivelogs between the sequence 123 and 145 RMAN> list backupset of archivelog 2> from logseq 123 until logseq 145 thread 1; Recovery Manager - Connie Dialeris 63 Report Command • Analyzes the content of the recovery catalog in more detail than list • Produces reports which can answer questions such as: – What files need a backup? report need backup – What files are not recoverable due to unrecoverable operations? report unrecoverable – What backups can be deleted (.e. are obsolete)? report obsolete – What was the physical schema of the database at a previous point-in-time? report schema Recovery Manager - Connie Dialeris 64 Report Command • and report regular basis to ensure: report need backup unrecoverable should be used on a – Recovery is possible RMAN> report unrecoverable; – Recovery is possible within an acceptable time For Example, the following command reports all datafiles which would require the application of 3 or more incrementals to be recovered to their current state RMAN> report need backup incremental 3 database; The following command reports all datafiles belonging to the SYSTEM tablespace that haven’t had a backup (full or incremental) in 5 days or more. RMAN> report need backup days 5 tablespace system; Recovery Manager - Connie Dialeris 65 Miscellaneous Recovery Manager - Connie Dialeris 66 Troubleshooting • Loss of the Recovery Catalog – First method should be: to always have a valid backup of your Recovery Catalog, and use the normal Oracle recovery mechanisms – Second method: • create a new recovery catalog • catalog all objects (archivelogs, backup sets, file copies) which exist on disk • resync the catalog – From current controlfile (if available) – From backup controlfile (if not available), then change...uncatalog all obsolete records inserted into the recovery catalog from the resync from backup controlfile Recovery Manager - Connie Dialeris 67 Troubleshooting • “Failed to start RPC call” RMAN-08010: RMAN-10030: RMAN-10036: RMAN-08010: channel c8: including datafile number 47 in backupset RPC call appears to have failed to start on channel c9 RPC call ok on channel c9 channel c3: including datafile number 18 in backupset This is not a problem. It indicates one of two things has occurred: – The target database instance is slow – Temporary timing problem • Killing a Recovery Manager job – interactively: ^c – batch: kill the rman process Recovery Manager - Connie Dialeris Troubleshooting • Additional Information Messages RMAN-03007: exception occurred during execution, error is retryable RMAN-07004: unhandled exception during command execution on channel c4 RMAN-10032: unhandled exception during execution of job step 4: ORA-06512: at line 158 RMAN-10035: exception raised in RPC: ORA-19624: operation failed, retry possible ORA-19506: failed to create sequential file, name="df_98_1", parms="" ORA-27007: failed to open file HP-UX Error: 1003: Unknown system error Additional information: 7004 Additional information: 1 ORA-06512: at "SYS.DBMS_BACKUP_RESTORE", line 410 RMAN-10031: ORA-19624 occurred during call to DBMS_BACKUP_RESTORE.BACKUPPIECEC REATE The 7004 above translates to: no tape device found Decode them using skgfqsbt.h, or look in the README Recovery Manager - Connie Dialeris Message Stack: read bottom up Troubleshooting • If a backup/restore/recover does not seem to be progressing, check the progress of the job from within oracle by querying v$session_longops select sid, serial#, context, round(sofar/totalwork*100,2) "% Complete", substr(to_char(sysdate,'yymmdd hh24:mi:ss'),1,15) "Time Now" from v$session_longops where compnam = 'dbms_backup_restore'; This will produce an output such as: SID SERIAL# CONTEXT % Complete Time Now ---------- ---------- ---------- ---------- --------------12 56 970408 14:21:07 If the % Complete is not increasing between intervals of 2 minutes, there is a problem; it should actually increase faster. Check v$session_wait to see what events within Oracle are being waited for: select * from v$session_wait where wait_time = 0; • Also check sbtio.log Recovery Manager - Connie Dialeris 70 Troubleshooting • Default SBT Interface – It is possible to write to disk via the SBT interface which comes default with Oracle. – This is useful for simulating writing to tape when trying to identify whether a problem encountered is a Media Manager problem, or an Oracle problem. – To use the interface: • Ensure you do not have the Media Management library linked in • Allocate channels of type ‘SBT_TAPE’ • Fully qualify the location to write the piece out to (must be a file system) Recovery Manager - Connie Dialeris Media Manager Issues • • • • Maximum supported file sizes (size of backup piece) Uniqueness of backup piece name Restoring to another node Reconciling the Recovery Catalog and the Media Manager's catalog Ensure the Recovery Catalog is made aware of the unavailability of any backups invalidated by Tape (re)cycling. Basically, the Recovery Catalog must be kept in sync with the Media Manager’s catalog. You can use rman change…uncatalog commands to do this. Recovery Manager - Connie Dialeris OPS Issues • Restoring in parallel across nodes • Snapshot control file set snapshot controlfile to '/oracle/db_files/snapshot/snap_prod.cf'; • Archivelog Backup All files must be readable by the ‘default’ connection during command compilation time • Archivelog Restore Archivelogs may be restored to any node performing a Restore Operation • Archivelog Recovery The archivelogs must be readable by the ‘default’ channel, as it is this one that performs the recovery Recovery Manager - Connie Dialeris Performance • Using I/O Slaves (An option if Async I/O is not available) – ‘Slave’ server processes perform the reads and writes on behalf of the channel, and allow the server process to continue processing while reads are performed. – Configured by setting the following init.ora parameters: backup_tape_io_slaves = (true/false) backup_disk_io_slaves = n Where n is the number of slaves per channel • Avoiding saturating one file with too many read-requests is already achieved by Recovery Manager multiplexing files together, and switching files frequently • This has the benefit of keeping tape drive streaming, while also not flooding a particular file with too many read requests. If backup is still adversely affecting OLTP: – – set limit readrate X (limits buffers read per file, per second) each buffer is of size db_block_size*db_file_direct_io_count Recovery Manager - Connie Dialeris 74 After Resetlogs • Once a resetlogs has been performed on a database, you must notify Recovery Manager of this before you attempt any further backup operations RMAN> reset database; • Returning to an old Incarnation – If the user has decided that they need to restore and recover the database to a point-in-time before the resetlogs was performed, they need to indicate this intent to Recovery Manager, by performing a: RMAN> reset database to <incarnation>; – The <incarnation> to revert to is determined by: RMAN> list incarnation of database <identifier>; Recovery Manager - Connie Dialeris 75 Questions Recovery Manager - Connie Dialeris 76