LifeMarkets, LLC: CommerceAgency
LifeMarkets wants to make sure that your Commerce Agency database is properly backed up. Most of the backup software products cannot backup files that are open and in use. Although your users may not be logged into the Commerce Agency program, the database is open and "in use". To get a good backup of the database, you either need backup software designed to handle an active database (Oracle Agents) or the
Oracle instance must be stopped.
For additional background information concerning Oracle Database Backup techniques, most bookstores carry Oracle Backup and Recovery books, or Oracle DBA books.
On-Line “HOT” Backup;
Two backup utilities which allow you to continue using the database while it is being backed up are Veritas'
Backup Exec and Cheyenne's ArcServe Utility. These Backup Utilities provide an Optional software product called an Oracle Agent. It is this software, Oracle Agent, when installed as part of BackupExec or
ArcServe provides an On-Line “HOT” backup of the CommerceAgency Oracle database.
Off-Line “COLD” Backup:
Some backup software (including BackupExec and ArcServe) has the ability to run external procedures
(specifically DOS batch files) before and after it performing the normal backup of the files on a Server. For these programs, we have created batch files to stop, backup, and restart the Oracle instance. The base versions of these batch files are contained within the DBScripts directory of the CommerceAgency
Installation CD-Rom.
If the backup software cannot run an external procedure, you must shutdown the Oracle instance manually, perform the backup, then restart the instance. The LifeMarkets supplied batch files can perform this
Function.
Ensuring that a complete and restorable Oracle Database backup file is created and accessible for future recovery requires several setup steps and constant monitoring of the backup logs to ensure the success of a future restore.
To ensure that the backup of the Oracle Database occurs daily, the following items should be checked each day.
1) Using SVRMGR check to ensure that Archive Log Mode is enabled
(See instructions below to sign into SVRMGR for your version of Oracle)
SVRMGR> Archive Log List
Database log mode Archive Mode
Automatic archival Enabled (Indicates that Archiving is enabled)
Archive destination D:\OraHome1\oradata\ca\archive (location of Archive files)
Oldest online log sequence 2750 (Archive log sequence number)
Next log sequence to archive 2753 (Archive log sequence number)
Current log sequence 2753 (Archive log sequence number)
1

LifeMarkets, LLC: CommerceAgency
2) Check the number of files contained within the Archive Destination directory as indicated above. Delete the Archive files (ARC%.%) only after you are sure that they have been Backedup to tape. These files will be needed for a restore of the Oracle database.
3) If using the Oracle Backup Scripts, check to ensure that the database backup file has been created. The scripts are typically contained in a directory on the drive that contains the database, called BACKUPSCRIPTS or BACKUP. The file EXPPDQ.DMP, EXPCA.DMP, or
EXPddmm.DMP should be dated about the time that the BackupExec or ArcServe backup runs.
4) If the DMP file is named EXPddmm.DMP, the older dated versions of this file can be deleted, along with the log file named EXPddmm.LOG. Only delete files that have been backed-up to tape.
Oracle Archive Log Mode:
During the installation of the CommerceAgency system, and the creation of the Oracle Database, Archive
Log Mode is enabled. Enabling this option, provides for the automatic copy (backup) of the database transaction log files. During the course of a change transaction within the database (updating a case, agent, carrier, etc.), Oracle will write to an external file (REDO Log) the value of the data item (field on a tab), before and after the change. If an error were to occur during Oracle’s updating of the database, the system will have a means to “Rollback” the change to the database. Recovering the status of the database before error occurred.
Normally these copies of the transactions are deleted overtime, to make room for newer transactions. With
Archive Log Mode enabled, Oracle creates an external copy of the these transaction files, providing a means to “RollForward’ database transactions after a failure.
Example:
Oracle database receives a full/complete backup on Monday morning at 6:00am.
Users apply transactions to the database Monday morning. during this time, the Oracle Archive Log is making copies of each transaction within the REDO logs.
At about 11:00am the disk drive containing the Oracle database “Crashes”.
The hardware vendor is able to recover 95% of the data on the Crashed disk drive.
One of the files that was unrecoverable was an Oracle database file.
By using the backup of the database from Monday morning at 6:00am, and using the saved copies of the
REDO logs that was created by the Oracle Archive Log, the database is able to be restored up to the point in time of the disk crash, recovering most if not all of the transactions entered during the morning. After recovery is complete, a review of the restored data will be necessary to determine the amount of data (if any) that requires reentry.
Without the Oracle Archive Log enabled, the database could have been restored up to the completion of
Monday morning’s backup at 6:00am. Resulting in a loss of the morning data.
The maintenance of the Archive Log option requires the backup and deletion (after backup) the copied
REDO Log files. The process to remove these files, is a manual process, but is a small price, compared to the possible lost of data.
2

LifeMarkets, LLC: CommerceAgency
(Perform AFTER IMPORTING DATABASE!!!!)
When the oracle database is initially installed, it is in No Archive Log Mode . If you plan to use a Backup program that will backup the database while on-line you must place the database in Archive Log Mode . To place a system in Archive Log Mode follow the steps below:
1.
To verify the status of the system, whether in Archive or No Archive mode do the following: a) Sign into SVRMGR23 ( SVRMGRL for Oracle 8i) b) Connect as Internal with using the password for the Internal account
(svrmgr23> connect internal)
2.
Type the following command: Archive log list
The first line of this command will indicate if the system is in Archivelog or noarchivelog mode. If the system is in noarchivelog mode then continue to step #2. If the second line indicates that Automatic archival is Disabled then proceed to step #3. If the system is in Archive Log Mode and is enabled, then the system is saving the REDO logs.
3.
Setting the system to be in Archivelog Mode. To do this follow the items listed below: a.
Within the OraHome1 directory structure, create the directory x:\ORANT\RDMS73\ARCHIVE (Oracle 7.3.4) x:\ORAHOME1\ORADATA\CA\ARCHIVE (Oracle 8i) b.
Get all users off the system because the database must be shut down to do this step. c.
Log into SVRMGR23 ( SVRMGRL for Oracle 8i)
SVRMGR23> connect internal password: d.
Type in the command shutdown immediate; e.
Type in the command startup mount f.
Type in the command alter database archivelog; g.
If you receive an error message at step “f” indicating that backup recovery is needed continue with step #8. If you did not receive any message of this type continue with step
#3h. h.
Type in the command alter database open; i.
Type in the command archive log start; j.
Type in the command exit
4.
The system should now be in Archive log mode. To test this follow step #2 again.
5.
Alter the Oracle Initialization File INIT.ORA. a.
Locate the INIT.ORA file.
X:\ORANT\DATABASE (Oracle 7.3.4)
X:\ORAHOME1\ADMIN\CA\PFILE (Oracle 8i) b.
Using a text editor find the following Line: c.
Log_archive_start=true (if there is a # in front of this command remove it. The number d.
sign indicates that this line is commented out.)
The next two lines need to follow the Log_archive_start= true command.
log_archive_start = true log_archive_dest =%ORACLE_HOME%\RDMS73\ARCHIVE (Oracle 7.3.4) log_archive_dest =%ORACLE_HOME%\ORADATA\%ORACLE_SID%\ARCHIVE (Oracle 8i) log_archive_format = "ARC%S.%T" e.
Save the INIT.ORA file.
3

LifeMarkets, LLC: CommerceAgency
6.
Shutdown the Oracle instance by using the SVRMGR23 or SVRMGRL shutdown command.
(See steps above to sign into the Server Manager program)
7.
Re-start the Oracle instance by using the Startup command.
(See steps above to sign into the Server Manager program)
8.
If you received an error message about Database Recovery is needed you must do the following things: a) To be on the safe side shutdown the database and do a complete system Export. To do this you will want to have all users off the system. Export the Entire database. This will ensure that the data is safe. b) Within SVRMGR23 or SVRMGRL, issue the command c) SVRMGRL> RECOVER DATABASE; d) This puts the system in a state to allow you to do step #2 listed above. e) If problems occur with this command, Please call LifeMarkets technical support.
If the Oracle Database Server is not accessible or repeatedly shuts down, requiring a restart, Archive Log
Mode may not be Enabled or may configured incorrectly. Follow these steps to Enable Archive Log Mode to correct the problem.
1.
In most cases the Oracle Server is installed with Archive Mode DISABLED, and the customers are performing On-Line backups of the system. When an On-Line backup is run on an Oracle database,
Archive Mode needs to be Enabled, to allow the REDO Log (transaction log files) to be saved. The
Archive process makes copies of the REDO log files after transactions are completed. In the event of an
Oracle database failure, and a restore of the database, the archived copies of the REDO logs can be applied to the database, and provide recovery up to the point in time of the failure.
2.
Within the Oracle log files contained at
X:\ORANT\RDBMS73\TRACE (Oracle 7.3.4)
X:\ORAHOME1\DATABASE (Oracle 8i) the system will log errors indicating that another process has locked a portion of a file.
Example: *** 2000.04.05.07.29.51.708
KCF: write/open error dba=0xa0019b8 block=0x19b8 online=1 file=10 D:\ORAHOME1\DATABASE\NEWBUS.DAT error=9205 txt: 'OSD-04016: Error queuing an asynchronous I/O request.
O/S-Error: (OS 33) The process cannot access the file because another process has locked a portion of the file.
3.
Within Oracle Storage Manager , check the both the TableSpaces and DataFiles folder and ensure that all tablespaces and datafiles are On-line and available.
4.
If any datafiles are Off-line or in Recover mode then perform the following: a.
Sign into SVRMGR23 (SVRMGRL Oracle 8i) b.
SVRMGR> CONNECT INTERNAL (and password) c.
SVRMGR> SHUTDOWN IMMEDIATE; d.
SVRMGR> STARTUP MOUNT;
4

LifeMarkets, LLC: CommerceAgency
e.
SVRMGR> ALTER DATABASE DATAFILE ‘filename’ OFFLINE; f.
(Repeat for each datafile that is off-line) g.
SVRMGR> ALTER DATABASE OPEN; h.
SVRMGR> RECOVER DATAFILE ‘filename’; i.
(Repeat for each datafile that is off-line) j.
SVRMGR> ALTER DATABASE DATAFILE ‘filename’ ONLINE; k.
(Repeat for each datafile that is off-line) l.
SVRMGR> SHUTDOWN IMMEDIATE; m.
SVRMGR> EXIT n.
Sign back into SVRMGR23 o.
SVRMGR> CONNECT INTERNAL (and password) p.
SVRMGR> STARTUP
5.
If during the RECOVER Datafile steps, a datafile is unable to be recovered due to the Oracle system unable to find the REDO log files (Recovery is looking for a particular Thread and Sequence #) then
Archive Mode will need to be disabled, then the Recovery steps repeated. a.
(Disabling Oracle Archive Mode) b.
Sign into SVRMGRL c.
SVRMGR> CONNECT INTERNAL (and password) d.
SVRMGR> SHUTDOWN IMMEDIATE e.
SVRMGR> STARTUP MOUNT f.
SVRMGR> ALTER DATBASE NOARCHIVELOG ; g.
SVRMGR> SHUTDOWN IMMEDIATE h.
SVRMGR> EXIT i.
EDIT X:\ORANT\DATABASE\INITORCL.ORA (Oracle 7.3.4) or
X:\ORAHOME1\ADMIN\CA\PFILE\INIT.ORA (Oracle 8i) - Comment out the three (3) commands that begin with LOG_ARCHIVE j.
Sign back into SVRMGRL or the SQL Worksheet k.
SVRMGR> CONNECT INTERNAL (and password) l.
SVRMGR> STARTUP MOUNT m.
SVRMGR> ARCHIVE LOG LIST n.
SVRMGR> ALTER DATABASE OPEN; o.
SVRMGR> SHUTDOWN IMMEDIATE p.
SVRMGR> EXIT
6.
Before users access the database, turn Archive Log Mode back on. Follow the published instructions to turn Archive Log Mode on.
5

LifeMarkets, LLC: CommerceAgency
 Oracle 7.x
1.
From a Command prompt, type SET Oracle_SID=ORCL
2.
Start the Server Manager by typing SVRMGR23
3.
At the Server Manager prompt (SVRMGR>), type connect internal
4.
Enter password
5.
Stop the database by typing shutdown immediate and wait until you see
Database closed.
Database dismounted.
Oracle instance shut down.
 Oracle 8i
1.
From a Command prompt, type SET Oracle_SID=CA
2.
Start the Server Manager by typing SVRMGRL
3.
At the Server Manager prompt (SVRMGR>), type connect internal
4.
Enter password
5.
Stop the database by typing shutdown immediate and wait until you see
Database closed.
Database dismounted.
Oracle instance shut down.
If you must close the Command Prompt window, type exit to close the Server Manager and type exit to close the Command Prompt. Proceed with your backup.
If you exited the Server Manager, you must first perform steps 1 through 4 above to log back in.
Oracle 7.x and Oracle 8i
1.
Type startup
2.
The last two lines of startup messages should contain
Database mounted.
3.
Database opened.
4.
Type exit to exit the Server Manager
5.
Type exit to close the Command Prompt window
These are general instructions to perform an Oracle database recovery. Depending on what Oracle database files are corrupt or lost will determine the process used to restore the database. I would recommend one of the Oracle DBA Handbooks available from independent bookstores as a reference guide to administering an
Oracle database and performing a database backup and restore.
To Test the Oracle backup you can use the following steps. In the case of an actual disaster the state of the database at the time of the problem will need to be determined and these steps may need to be altered.
1) Restore the lost Oracle database files from the backup tape
2) Restore any Archive Log (Redo) files from the backup tape. You will only need the Archive log files that contain dated transactions after the Oracle backup started. (example: If the backup started at 2:00am, only the Archive Log files that contain transactions after 2:00am will be needed). This may require restoring
Archive Log files that have a creation date just before the Oracle backup starts.
3) Mount the database (do not perform a full database startup
(From a DOS prompt, issue the following commands)
6

LifeMarkets, LLC: CommerceAgency
C:\svrmgr23 (svrmgrl Oracle 8i)
SVRMGRL> connect internal password:
SVRMGRL> startup mount
SVRMGRL> recover database;
When prompted, enter the file names of each of the requested Archived Log (Redo) files that had been created since the previous Oracle database backup. These files will be located within the Oracle
Home directory structure, and should also have been backed up to tape. Enter the files names by date, starting with the oldest file first. When all the Archive Log files have been processed, start the database.
SVRMGRL> alter database open;
Shutdown the database, and start with a normal startup command.
SVRMGRL>
SVRMGRL>
shutdown immediate;
startup
4) Start the CommerceAgency application and look at the data, try to determine if the data contained within the database is correct, and reflects the condition of the database up to the failure.
Install, configure, and test the normal backup function of the selected backup software.
If using the Oracle Agent option of the software, follow the installation and setup instructions included by the software vendor. Be sure that the Oracle Archive Log Mode is enabled before using the Oracle Agent.
If using the LifeMarkets supplied Oracle backup scripts, configure the backup utilities to “Execute” the
Procedures during the beginning and end of the backup process. Each backup utility has a means to indicate
“Pre” and “Post” command procedures to be processed.
The procedures provided by LifeMarkets require the following setup.
Oracle 7.3.4
(1) Create new folder c:\Backup OR on the drive which has sufficient space.
(2) Create new folder c:\Backup\export
(3) Copy the following backup scripts to the folder c:\backup
(a) pdq2kcl.bat
(b) pdq2kop.bat
(c) stopdb.bat
(d) startdb.bat
(e) startres.sql
(f) stopdb.sql
(g) startdb.sql
(4) Edit the stopdb.bat & startdb.bat files and replace the service name, instance name
7

LifeMarkets, LLC: CommerceAgency
and listener name to fit your environment if necessary. The files assume the service
name as PDQ2000 and Instance name as ORCL.
(5) Edit each of the files, updating or inserting for each file name the directory path for the directory that contains these backup scripts.
Example:
Pdq2kcl.bat contains “@stopdb.bat >pdq2kcl.log” this should be “@ c:\backup\ stopdb.bat > c:\backup\ pdq2kcl.log”
(6) Edit file $ORACLE_HOME\network\admin\listener.ora and comment
#PASSWORDS_LISTENER= -- Because listener stop command doesn't recognized the password
(7) For database shutdown execute the pdq2kcl.bat file. This will perform following tasks:
(a) Shutdown the database
(b) Startup the database in exclusive mode
(c) Export the database
(d) Shutdown the database
(e) Stop the Oracle services
(f) Stop the Oracle listener
(8) For database startup execute the pdq2kop.bat file. This will perform following tasks:
(a) Startup the oracle listener
(b) Startup the oracle services
(c) Startup the database.
(9) Test the shutdown and startup procedures. Ensure that all users are out of the system, then at a DOS prompt execute
First: pdq2kcl.bat
Second: pdq2kop.bat
(10) If this test is successful, then Edit the Backup software Job attributes add pdq2kcl.bat as the “PRE” command procedure add pdq2kop.bat as the “POST” command procedure
(11) Test the Backup software. If need be, test the Backup software overnight, checking the log files the morning of the next day.
8

LifeMarkets, LLC: CommerceAgency
Oracle 8i
Option 1 – Backup scripts files CAStart.bat, CAStop.bat, Startdb.bat, Startdb.sql
Startxcl.sql, Stopdb.bat, Stopdb.sql
(1) Create the new folder D:\Backup OR on the drive that has sufficient space.
(2) Create the new folder D:\Backup\Export.
(3) Copy the following backup scripts to the folder D:\Backup
There are three versions of the backup scripts. One version for the possible disk
Drive name that could be used to hold the backup. DriveD is for disk drives named D.
DriveE is for disk drives named E. DriveF is for disk drives named F.
(a) CAStart.bat
(b) CAStop.bat
(c) Startdb.bat
(d) Startdb.sql
(e) Startxcl.sql
(f) Stopdb.bat
(g) Stopdb.sql
(4) Edit the stopdb.bat, startdb.bat, and startxcl.sql files and replace the service name, instance name
and listener name to fit your environment if necessary. The files assume the service
name as CA and Instance name as CA.
(5) Edit each of the files, updating or inserting for each file name the directory path for the directory that contains these backup scripts.
Example: castop.bat contains “@stopdb.bat >castop.log” this should be “@ d:\backup\ stopdb.bat > d:\backup\ castop.log”
(8) For database shutdown execute the CAStop.bat file. This will perform following tasks:
(a) Shutdown the database
(b) Startup the database in exclusive mode
(c) Export the database
(d) Shutdown the database
(e) Stop the Oracle services
(f) Stop the Oracle listener
(9) For database startup execute the CAStart.bat file. This will perform following tasks:
(a) Startup the oracle listener
(b) Startup the oracle services
(c) Startup the database.
9

LifeMarkets, LLC: CommerceAgency
(10) Test the shutdown and startup procedures. Ensure that all users are out of the system, then at a DOS prompt execute
First: castop.bat
Second: castart.bat
(11) If this test is successful, then Edit the Backup software Job attributes add castop.bat as the “PRE” command procedure add castart.bat as the “POST” command procedure
(12) Test the Backup software. If need be, test the Backup software overnight, checking the log files the morning of the next day.
Oracle 8i
Option 2 – Backup scripts files CABackup.bat, CAPerform.bat, Startxcl.sql
(1) Create the new folder D:\Backup OR on the drive that has sufficient space.
(2) Create the new folder D:\Backup\Export.
(3) Copy the following backup scripts to the folder D:\Backup (Contained in the compressed file
CABACKUP.EXE on the FTP Server)
(a) CABackup.bat
(b) CAPerform.bat
(c) Startxcl.sql
(4) Edit the CABackup.bat, CAPerform.bat, and Startxcl.sql files and replace the service name, and instance name for your environment if necessary. The files assume the service name as
CA and Instance name as CA.
(5) Edit each of the files, updating or inserting for each file name the directory path for the directory that contains these backup scripts.
Example:
CABackup.bat contains “@d:\backup\caperform.bat >d:\backup\caperform.log”
(6) To perform the database backup execute the CABackup.bat file. This will perform following tasks:
(a) Create a procedure (EXP84.bat) that will perform an export of the database
(b) Export the database creating a file with the day of the week as part of the file name example: EXPMON.DMP for Monday, EXPTHU.DMP for Thursday
(7) If this test is successful, then Edit the Backup software Job attributes add CABackup.bat as the “PRE” command procedure
(12) Test the Backup software. If need be, test the Backup software overnight, checking the log files the morning of the next day.
10