OCFS
Oracle Cluster FileSystem for Linux
Users Guide
Table
of
Contents
1 - Introduction. ................................................................................................................................... 4
2 - Obtaining OCFS............................................................................................................................. 4
3 - Installing OCFS .............................................................................................................................. 5
3.1 - Automatically mount OCFS during boot................................................................................ 5
3.2 - Configuring the OCFS. .............................................................................................................. 5
3.2.1 - /etc/ocfs.conf file. ................................................................................................................... 5
3.2.2 - Automatic Configuration using ocfstool. ............................................................................. 6
3.2.3 - Manual configuration ............................................................................................................... 6
3.3 - Loading ocfs. ............................................................................................................................... 7
3.4 - Formatting OCFS Partition....................................................................................................... 7
3.4.1 - Format using ocfstool. ............................................................................................................. 7
3.4.2 - Format using mkfs.ocfs. .......................................................................................................... 7
3.5 - Mount OCFS partitions. ............................................................................................................ 9
3.5.1 - Mounting manually. ................................................................................................................. 9
3.5.2 - Mounting automatically ........................................................................................................... 9
4 - Tools ................................................................................................................................................ 9
4.1 - ocfstool ......................................................................................................................................... 9
4.1.1 - Format...................................................................................................................................... 13
4.1.2 - Generate Config. .................................................................................................................... 14
4.2 - Extfinder .................................................................................................................................... 15
4.3 - Debugocfs .................................................................................................................................. 15
5 - Support Utilities ............................................................................................................................ 18
5.1 - /etc/init.d/ocfs ......................................................................................................................... 18
5.2 - /sbin/fsck.ocfs .......................................................................................................................... 18
5.3 - /sbin/load_ocfs ........................................................................................................................ 18
5.4 - /sbin/mkfs.ocfs ........................................................................................................................ 18
5.5 - /sbin/mounted.ocfs ................................................................................................................. 19
5.6 - /sbin/ocfs_uid_gen.................................................................................................................. 19
6 - Best Practices ................................................................................................................................ 19
7 - Frequently asked Questions & Answers ................................................................................... 20
Q. Can I install the Oracle Distribution in an OCFS partition? ................................................. 20
Q. My partitions don't mount automatically during boot. What's wrong? ............................... 20
Q. When running fsck.ocfs, it returns the error “WARNING: nonzero bytes after the
disk header structure”. What does it mean?................................................................................... 20
Q. What are the most appropriate (recommended) tool IO and capacity analysis of
OCFS? ................................................................................................................................................. 20
Q. Can I use ocfs on a NAS (Network Attached Storage) device like NetApp? ..................... 20
Q. Can I use LVM or MD to create my OCFS filesystem on top of it? ................................... 20
Q.I want to perform some tests using RAC/OCFS, but I don't want to spend too much
money on hardware. Do I have any other solution? .................................................................... 21
Q.Why do I have to include the option _netdev in the fstab on RedHat? ............................... 21
Q.What is the best way to archive the logs to an OCFS directory? ........................................... 21
Q.Is OCFS supported on 64 bit platform like Itanium? .............................................................. 21
Q. Can I run ocfs on non enterprise Linux distributions like RedHat 9? ................................. 21
Q.Do I need any specific version of United Linux to run OCFS? ............................................ 22
Q. Can I run the latest OCFS on a plain RedHat AS 2.1, without any errata applied? ........... 22
Q.What is the advantage of running ocfs against raw devices? .................................................. 22
Q.What happen if I have to change the IP Address of my systems? ......................................... 22
Q.My Network Interface Card had to be replaced. Do I need to do something? ................... 22
Q.I have a partition that is not mounted. How do I know if it is an ocfs partition or not?... 22
Q.Can I use my OCFS partition to store regular files? ................................................................ 23
Q.How much do I lose in terms of performance compared to raw devices? .......................... 23
Q.How do I enable async I/O on Oracle using OCFS? ............................................................. 23
Q.How do I backup my OCFS files? Can I use tar or other OS command? ........................... 23
Q.Is it possible to resize an existing OCFS partition?.................................................................. 23
Q.I'm having problems with OCFS. How can I debug OCFS? ................................................. 23
Q.Can I run OCFS in a stand-alone system? What are the advantages of running it? ............ 24
Q.I have a database running on OCFS in a stand-alone node. Why it is so slow compared
to other stand-alone systems running on ext3? ............................................................................. 24
Q.How can I obtain more information about OCFS? ................................................................. 24
Q.I have a customized RedHat AS kernel on my system. Does Oracle support OCFS and
the RDBMS on it? ............................................................................................................................. 24
Q.How do I know if my OCFS version is officially supported by Oracle? .............................. 24
8 - Appendix A ................................................................................................................................... 25
Introduction.
OCFS is a shared disk cluster filesystem. The current version (version 1)
released for Linux is specifically designed to alleviate the need for managing raw devices. It can contain all the oracle datafiles, archive log files and
controlfiles. It is however not designed as a general purpose filesystem.
This document describes the steps required to install OCFS on Linux and
will also give guidelines for optimizations and some more in depth understanding of how the filesystem works.
Downloading OCFS.
OCFS can be downloaded from “http://oss.oracle.com/projects/ocfs”
for the following distributions : RedHat Advanced Server 2.1 and United
Linux 1.0 (Conectiva, SuSe, TuboLinux and SCO). Oracle officially supports the Oracle database on OCFS if it is installed from the binary packages that are available for download.
If the user decides to download the source code and compile it, then there
will be no formal support provided by Oracle.
In addition to the OCFS binaries, we also provide a collection of utilities
(cp, dd, tar and textutils) that enable O_DIRECT. The updated tools are
recommended to be used as they make more efficient use of the operating
system in conjunction with OCFS.
Binary distributions for ia32 and ia64 can be found under each one of the
supported platforms.
See “Appendix A”
for a complete list
of
files
and
description of each
one of the OCFS
packages.
There are basically three rpm packages to download in order to install
OCFS. Those packages are:
OCFS-Support
OCFS-Tools
OCFS Module
Before downloading the OCFS Module, make sure it is compatible with
the kernel version in use (uname -a).
Installing OCFS
The
rpm
command in the
right will install
the
OCFS
packages if they
are not installed
yet
and
will
upgrade them if
they
were
previously
installed.
Installing OCFS is an easy process. After downloading the packages, issue
the following command on the directory where the packages were downloaded:
# rpm -Uhv ocfs*.rpm
This will install the support tools, the actual kernel module for the filesystem and a graphical configuration tool.
Automatically mount OCFS during boot.
After installing the OCFS packages, verify that the module will be properly
initialized on startup using the command:
# chkconfig –list |grep -i ocfs
If the output looks like :
ocfs
0:off 1:off 2:off 3:on
4:on
5:on
6:off
Then, no action is required, but if the output doesn't show “on” on 3,4
and 5 (rc levels), issue the following command to enable automatic startup
of the ocfs during boot:
# chkconfig ocfs on
Configuring OCFS.
OCFS depends on a node specific configuration file. This file is named
ocfs.conf and it is located in the /etc directory; it can be generated automatically or manually using the ocfstool. During the next sections, the
/etc/ocfs.conf file and both methods of configuring will be described in
detail. This file is needed on every node in the cluster and it is highly recommended to use ocfstool to configure each node.
1.1.1 - /etc/ocfs.conf file.
The /etc/ocfs.conf file can have the following parameters:
ip_address – Specify the IP Address to be used by the OCFS DLM. The
server must be able to reach all nodes participating on the cluster through
the interface related to the IP Address specified in this field.
ip_port – Specify the port to be used by the OCFS DLM to communicate
with the other nodes in the cluster. The port must be the same on all
nodes in the cluster.
Node_name – Specify the server hostname associated to the IP Address
specified in the ip_address parameter.
comm_voting – Specify which method for voting is going to be used by
OCFS. If set to 0 (default), it means that OCFS will be voting using the
disk, if set to 1, it will be using the network to vote. If the OCFS is set to
use network to vote and it becomes unavailable for some reason, it will automatically (and transparently) fall back to disk. Enabling comm_voting
will drastically increase performance for regular filesystem operations such
as rm, mv, mkdir etc.
guid – This parameter is automatically filled by the ocfs_uid_gen utility
and should never be manually changed.
1.1.2 - Automatic Configuration using ocfstool.
In order to perform the automatic configuration with ocfstool, it is necessary that the GUI environment is properly set and enable.
Open a GUI session as root and execute the command “ocfstool”. Make
sure the DISPLAY variable is set before starting the tool.
When the ocfstool window(See Illustration 1) open, invoke the Generate
Config(See Illustration 7) task by either pressing the key sequence
<CTRL-G> or using the menu, clicking on Tasks>Generate Config . For
more information on the parameters, check the item Generate Config in
the ocfstool section.
1.1.3 - Manual configuration
Although the manual configuration is provided, Oracle strongly recom-
mends the usage of the ocfstool since it does provide a reliable, consistent
and easy way to properly configure OCFS.
To manually configure the OCFS, create the file /etc/ocfs.conf based on
the sample below on each one of the nodes participant of the cluster.
Make sure the parameters are consistent among nodes.
if using a specific
network,
make
sure all nodes will
have
hostname
and IP addresses
related to the
same subnet.
# ocfs config
# Ensure this file exists in /etc directory #
node_name = ca-test2.us.oracle.com
ip_address = 10.0.0.1
ip_port = 7000
comm_voting = 1
After the file is created, execute the utility ocfs_uid_gen with the -c argument as root in order to generate the unique identification key necessary
for the OCFS to identify itself in the cluster. After the generation of the uid
key, the /etc/ocfs.conf file should looks like:
#
# ocfs config
# Ensure this file exists in /etc#
node_name = ca-testt2.us.oracle.com
ip_address = 10.0.0.1
ip_port = 7000
comm_voting = 1
guid = 9B2996991BCB25DF4CBB0003470CFE75
Loading ocfs.
The /etc/init.d/ocfs startup script is provided in the package and automatically loads the OCFS module if there is an entry for it in /etc/fstab.
Using this startup script is the preferred method to load OCFS as it does
all the verification before loading the module and mounting the partitions.
If for some reason there still is a need of manually load the OCFS module,
just issue the command load_ocfs as root. If the process is successfully
executed, it should show a message like:
# load_ocfs
/sbin/insmod ocfs node_name=ca-test2.us.oracle.com ip_address=10.0.0.1
ip_port=7000 cs=1859 guid=9B2996991BCB25DF4CBB0003470CFE75
Using /lib/modules/2.4.9-e-enterprise-ABI/ocfs/ocfs.o
Formatting an OCFS Partition.
Similar to configuring the OCFS, there are two ways to format an OCFS
partition. One is using the “ocfstool” command and the GUI environment, and the other is using the “mkfs.ocfs” command from the shell
prompt. Both commands needs to be executed as root.
None of the ocfs utilities will partition the disk at any time. So, before
formatting, choose the utility of your preference and partition the disk according to the needs of your implementation, and make sure the
disk/partition is not being used by anything else to avoid data loss.
With OCFS you have to format a partition only once on one node, after
that every node will be able to mount this filesystem. (of course the device
needs to be visible on every node in the cluster)
1.1.4 - Format using ocfstool.
Start by invoking the ocfstool command like described in the section
“3.2.2 Automatic configuration using ocfstool”. After getting the ocfstool
window(See illustration 6), press either the sequence key <CTRL-F> or
from the menu, choose Tasks>Format. Fill in all fields and click the “OK”
button. For more information on the parameters, check the format item under
ocfstool section.
1.1.5 - Format using mkfs.ocfs.
For people who do not have a GUI available, use mkfs.ocfs. If a GUI is
available, the preferred method to format an OCFS partition is to use the
ocfstool. The mkfs.ocfs command has the following syntax:
# mkfs.ocfs
usage: mkfs.ocfs -b block-size [-C] [-F] [-g gid] [-h] -L volume-label
-m mount-path [-n] [-p permissions] [-q] [-u uid] [-V] device
-b Block size in kilo bytes
-C Clear all data blocks
-F Force format existing OCFS volume
-g GID for the root directory
-h Help
-L Volume label
-m Path where this device will be mounted
-n Query only
-p Permissions for the root directory
-q Quiet execution
-u UID for the root directory
-V Print version and exit
When using the mkfs.ocfs command, the user has to provide all the information that is prompt by the ocfstool utility.
The usage of the “-C” argument will force the mkfs.ocfs to clear all blocks.
Depending on the size of the partition, it may be a long process.
The “-F” argument should be used only if the partition was previously
formatted as an OCFS.
The “-b” argument specify the blocksize that the partition will be formatted. The blocksize specifies the maximum size of the partition that can be
mounted. It goes from 4k to 1M and allows volumes from 32Gb up to 8Tb.
Format with 128kb blocksize is optimal size. Sizes between 4kb and 1mb
are supported. The smaller blocksizes will have a performance penalty,
but will be useful for the future when we will support regular files. 128kb
blocksize means that every file created with content uses up a minimum
of 128kb space on disk. Even if there is only 1 byte of data in the file. The
filesystem will allocate chunks of space in <blocksize> chunks.
The example below show an ordinary partition being formatted and its
output.
# mkfs.ocfs -F -b 128 -g dba -u oracle -L /u01 -m /u01 - p 775 /dev/sdb1
Checking heart beat on volume ..........
Clearing volume header sectors...Cleared volume header sectors
Clearing node config sectors...Cleared node config sectors
Clearing publish sectors...Cleared publish sectors
Clearing vote sectors...Cleared vote sectors
Clearing bitmap sectors...Cleared bitmap sectors
Clearing data block...Cleared data block
Writing volume header...Wrote volume header
#
Mount OCFS partitions.
1.1.6 - Mounting manually.
At least for the first time, it is good to have the partitions mounted manually instead of automatically. That's because the user has control over all
the process and can check if everything works fine.
To mount the new OCFS partition, use the mount command with the “-t”
argument specifying “ocfs” in front of it. The example below shows how
an OCFS partition is mounted:
# mount -t ocfs /dev/sdb1 /u01
1.1.7 - Mounting automatically
To mount the OCFS partitions automatically, just add the partition information to the /etc/fstab file. The example below show an entry for
RedHat AS 2.1.
The
_netdev
option
is
supported
on
RedHat, but may
not be for UL
distributions.
Check the mount
man pages for
details.
/dev/sdb1
/u01
ocfs
_netdev
/dev/sdd1
/u02
ocfs
_netdev
Tools
ocfstool
ocfstool is a GUI frontend for managing and debugging OCFS volumes on
the system; and also the preferred method for managing OCFS. One can
mount and unmount volumes, format partitions, view information and
individual files, see the current node map, and block bitmap.
After starting the tool you are presented with a window consisting of 2 sections
or segments.
The top portion maps all known partitions that are OCFS formatted and allow
users to mount and unmount these partitions. The mount operation will try to
mount the filesystem to the mountpoint specified during the format operation.
Like any other filesystem the umount operation will only succeed if there is no
process using it.
The bottom portion has a series of folders that are divided by areas of management and browsing. It will show the information related to the device selected in
the top portion of the screen.
The preferences
menu has effect
only
to
the
general
folder,
allowing the user,
group
and
protection fields
to be updated
when in advanced
mode.
The information that can be obtained is divided in :
General – Contain general information about the filesystem (See Illustration 1).
Like ocfs version that formatted the device, mountpoint, size of the filesystem,
number of extents, userid and groupid with privileges on the filesystem and the
appropriate permission.
File Listing – Will show file information about the filesystem (See Illustration 2).
Selecting a file or directory, one can see its information, like size, allocation unit,
ownership and protection. One can also see which nodes have the partition
mounted at that specific point in time.
Configured Nodes – On this folder, one can see which nodes have the selected
partition (See Illustration 3).
Bitmap View – On this folder, one can see the bitmap allocation for the selected partition (See Illustration 4).
Free Space – On this folder, one can see a list of free space for the selected partition (See Illustration 5). This list will show the size and the bit# of the free
space.
Illustration 1
Illustration 2
Illustration 3
Illustration 4
Illustration 5
In addition to the two portions showed, there are the tasks that can be
reached by the menu. The available tasks on the menu are:
1.1.8 - Format.
There are two ways to invoke the format window, one is pressing the
<CTRL+F> key sequence and the other is invoking by selecting the menu
Tasks>Format.
When the format window is invoked, the following options will be available:
Device – From the pull-down menu, select the device that is going to be OCFS
formatted. Make sure the device is not in use by any other application or filesystem before proceeding.
Blocksize – Select the blocksize on which the partition is going to be formatted.
Valid values are 4k, 8k, 16k 32k, 64k, 128k, 256k, 512k and 1024k. The optimal
size suggested by Oracle is 128k. Smaller blocksize can be selected but they will
carry some performance penalty. This is also going to limit the size of the partition that can be used when formatting with OCFS (32Gb to 8Tb).
Volume Label – This will specify the volume label. This is useful if the user
wants to use the volume label to mount the filesystem.
Mountpoint – Specify the location where the partition is going to be mounted.
The mount point must exist on all nodes in the cluster that will share the device.
User – Specify the user that will own the filesystem. When the filesystem is
mounted, it is automatically owned by the user specified in this field.
Group – Specify the group that the filesystem will belong to. Similar to the User
field.
Protection – Sets the default permission of the filesystem when mounted. Usually set to 0755.
Clear all Data Blocks – When checked, this option will make ocfs format block
by block, zeroing all the filesystem. This option will increase the time necessary
to format the partition considerably. (SLOW)
Force – This options needs to be checked if the partition to be formatted was
previously formatted by OCFS.
Illustration 6
1.1.9 - Generate Config.
Invoking this task will promptly generate the /etc/ocfs.conf file according
to the server configuration. Mandatory fields will be automatically filled
with the server information. There is no need to do any further steps as the
process will automatically generate the guid in the configuration file..
When the Generate Config window is invoked, the following options will be
available:
Interface – Select the interface that is going to be used by the OCFS to generate
unique identification. This interface will also be used by the OCFS Distributed
Lock Manager (DLM) in future implementation. Make sure that all nodes that
will be sharing the OCFS can be reached through the selected interface.
Port – Select the port number that is going to be used to communicate to the
other nodes. Any non-used port can be assigned to OCFS. Make sure all nodes
have the same port configured.
Node-Name – Select the node name associated to the interface selected.
Preferred Node Number – This is an optional field. If there is a need to have
an specific node assigned to a number, fill this field, otherwise, leave it to the
OCFS.
Illustration 7
Extfinder
Extfinder is an utility to report a list of contiguous free extents in the ocfs
filesystem. It will show how fragmented the filesystem is after usage.
Debugocfs
The debugocfs utility is used to extract metadata information from the
OCFS partition. This information is useful to help diagnose eventual
problems that may arise. The debugocfs syntax is :
debugocfs: Usage: debugocfs [-?] [-h] [-g] [-l] [-v range] [-p range]
[-d /dir/name] [-f /file/name [-s /path/to/file]] [-a range] [-A range]
[-b range] [-B range] [-r range] [-c range] [-L range] [-M range]
[-n nodenum] /dev/name
-h: volume header
-g: global bitmap
-l: full listing of all file entries
-v: vote sector
-2: print 8-byte number as 2 4-byte numbers
-p: publish sector
-d: ocfs_dir_node structure for a given path
-f: ocfs_file_entry structure for a given file
-F: ocfs_file_entry and ocfs_extent_group structures for a given file
-s: suck file out to a given location
-a: file allocation system file
-A: dir allocation system file
-b: file allocation bitmap system file
-B: dir allocation bitmap system file
-r: recover log file system file
-c: cleanup log system file
-L: vol metadata log system file
-M: vol metadata system file
-n: perform action as node number given
/dev/name: readable device
range: node numbers to inspect (0-31), commas and dashes ok
ex. 0-3,5,14-17
Below,
some
Volume Header.
examples
of
[root@ca-test2 root]# debugocfs -h /dev/sdb12
diskheader:
the
debugocfs
output
version = 1.2
signature = OracleCFS
mount_point = /u01
serial_num = 0
device_size = 4194860544
start_off = 0
bitmap_off = 56320
publ_off = 23552
vote_off = 39936
root_bitmap_off = 0
data_start_off = 1368064
root_bitmap_size = 0
root_off = 2416640
root_size = 0
cluster_size = 131072
num_nodes = 32
num_clusters = 31989
dir_node_size = 0
file_node_size = 0
internal_off = 1368064
prot_bits = 493
uid = 1011
gid = 1011
excl_mount = -1
volumelabel:
curr_master = 0
file_lock = OCFS_DLM_NO_LOCK
oin_node_map = 00000000000000000000000000000000
seq_num = 0
label = /u01
label_len = 4
[root@ca-test2 root]#
Vote Sector.
[root@ca-test2 root]# debugocfs -v 0-2 /dev/sdb12
vote0:
seq_num = 0
dir_ent = 0
open_handle = No
Vote0 = (0x00000000)
Vote1 = (0x00000000)
Vote2 = (0x00000000)
vote1:
seq_num = 0
dir_ent = 0
open_handle = No
Vote0 = (0x00000000)
Vote1 = (0x00000000)
Vote2 = (0x00000000)
vote2:
seq_num = 0
dir_ent = 0
open_handle = No
Vote0 = (0x00000000)
Vote1 = (0x00000000)
Vote2 = (0x00000000)
[root@ca-test2 root]#
NOTE: The path
specified for the
file and directory
structure
are
relative to the
mountpoint, not
to
the
root
filesystem. In the
example in the
right, to obtain
file entry structure
for
/u01/oradata/smd
b/control01.ctl,
the
mountpoint
/u01 is not used.
ocfs_file_entry structure
[root@ca-test2 root]# ls /u01/oradata/smdb/dbfiles/control01.ctl
/u01/oradata/smdb/dbfiles/control01.ctl
[root@ca-test2 root]# debugocfs -f /oradata/smdb/dbfiles/control01.ctl /dev/sdb12
fileinfo:
Name = /oradata/smdb/dbfiles/control01.ctl
curr_master = 0
file_lock = OCFS_DLM_ENABLE_CACHE_LOCK
oin_node_map = 10000000000000000000000000000000
seq_num = 0
local_ext = true
granularity = -1
filename = control01.ctl
filename_len = 13
file_size = 12509184
alloc_size = 12582912
attribs = OCFS_ATTRIB_REG
prot_bits = S_IRUSR S_IWUSR S_IXUSR S_IRGRP S_IWGRP S_IXGRP S_IROTH
S_IXOTH
uid = 1011
gid = 1011
create_time = Fri Aug 29 16:49:10 2003
modify_time = Fri Aug 29 16:49:10 2003
dir_node_ptr = 2940928
this_sector = 2941440
last_ext_ptr = 0
sync_flags = OCFS_SYNC_FLAG_VALID
link_cnt = 0
next_del = 0
next_free_ext = 1
extent[0].file_off = 0
extent[0].num_bytes = 12582912
extent[0].disk_off = 4644864
extent[1].file_off = 0
extent[1].num_bytes = 0
extent[1].disk_off = 0
extent[2].file_off = 0
extent[2].num_bytes = 0
extent[2].disk_off = 0
[root@ca-test2 root]#
Support Utilities
In this section, a description of each one of the utilities belonging to the
ocfs-support package. These packages are the minimum necessary to
make the OCFS work properly.
/etc/init.d/ocfs
This script is responsible to automatically load the ocfs module, perform a
sanity check on the environment and mount all the ocfs partitions listed in
/etc/fstab.
/sbin/fsck.ocfs
Perform a filesystem check on the partition. All nodes of a cluster that
share a specific device need to have them unmounted before running
fsck.ocfs, if one of the nodes has the partition mounted, the fsck.ocfs will
fail. The command syntax is:
usage: fsck.ocfs [-N] [-v] device
-N No write
-V Version
-v Verbose
-q Quiet
/sbin/load_ocfs
The load_ocfs script will call the insmod command and load the ocfs
module with all the correct parameters.
/sbin/mkfs.ocfs
This command is used to build a the filesystem structure in a partition.
The syntax of the mkfs.ocfs command is :
usage: mkfs.ocfs -b block-size [-C] [-F] [-g gid] [-h] -L volume-label
-m mount-path [-n] [-p permissions] [-q] [-u uid] [-V] device
-b Block size in kilo bytes
-C Clear all data blocks
-F Force format existing OCFS volume
-g GID for the root directory
-h Help
-L Volume label
-m Path where this device will be mounted
-n Query only
-p Permissions for the root directory
-q Quiet execution
-u UID for the root directory
-V Print version and exit
/sbin/mounted.ocfs
This command mounted.ocfs is used basically to check which nodes have
a specific device mounted. The command syntax is :
usage: mounted.ocfs <device>
/sbin/ocfs_uid_gen
The ocfs_uid_gen command is usually used once during the configuration
of the OCFS after it is installed. Its function is to generate the unique key,
in the /etc/ocfs.conf file, that identify the node in the OCFS node manager. The command syntax is :
Usage: ocfs_uid_gen -c
ocfs_uid_gen -r
-c
Create a new GUID for a new node before it enters the
cluster.
-r
Recover the GUID, inserting the new MAC address. This
operation is only for existing nodes whose MAC address
has changed.
Best Practices
For
RedHat
AS
2.1
go
to
http://oss.oracle.com/projects/ocfs/dist/documentation/RHAS_best_p
ractices.html.
ForUnited
Linux,
go
to
http://oss.oracle.com/projects/ocfs/dist/documentation/UL_best_pract
ices.txt.
Frequently asked Questions & Answers
Q. Can I install the Oracle Distribution in an
OCFS partition?
A. No. OCFS version 1 supports only database files
(datafiles, logfiles, controlfiles, archivelog files, srvm
configuration file and Oracle Cluster Manager quorum file).
Q. My partitions don't
during boot. What's wrong?
mount
automatically
A. The ocfs script on /etc/init.d directory is not enable.
Use the chkconfig command to verify that and enable it.
Instructions on how to enable it are listed under item 3.1
- Automatically mount OCFS during boot.
Q. When running fsck.ocfs, it returns the error
“WARNING: nonzero bytes after the disk header structure”. What does it
mean?
A. Earlier releases of mkfs.ocfs did not clear up 1st block
entirely. We've since fixed the issue in the format
utility. As far as functioning goes, one can safely ignore
this message.
Q. What are the most appropriate (recommended)
tool IO and capacity analysis of OCFS?
A. Your usual tools (sar, vmstat, iostat) will
work, it handles io just like any other
filesystem.
Q. Can I use ocfs on a NAS (Network Attached
Storage) device like NetApp?
A. No. NAS works through network and the physical volume
and filesystem is under the fileserver control. OCFS needs
to use a direct attached storage device or SAN (Storage
Area Network) where the local server has total control over
the physical volumes.
Q. Can I use LVM or
filesystem on top of it?
MD
to
create
my
OCFS
A. LVM and MD are not cluster aware and it makes it very
difficult to manage in a cluster environment. No tests have
been performed using LVM or MD. It may work, but it also
may not. Corruptions are likely to happen since they are
not cluster aware software.
Q.I want to perform some tests using RAC/OCFS,
but I don't want to spend too much money on
hardware. Do I have any other solution?
A.Yes. Users can run on firewire. Oracle does provide a
special kernel and ocfs modules to run on firewire. More
information
can
be
found
at
http://oss.oracle.com/projects/firewire/.
Q.Why do I have to include the option _netdev in
the fstab on RedHat?
A.The
_netdev
parameter
is
used
when
the
filesystem
resides
on
a
device
that
requires network access, it is used to prevent
the system from attempting to mount these
filesystems
until
the
network
has
been
enabled on the system.
Q.What is the best way to archive the logs to an
OCFS directory?
A.The ideal is to create a directory for each one
of the nodes that will be archiving on OCFS. This
way, the logs will be kept organized and there
will be no concurrency when two or more nodes
archive at the same time. This doesn't mean there
will be only archivelogs of the node on its
directory, because Oracle does allow a node to
perform an archive operation if a specific node
is down for long time to reduce recovery time.
Q.Is OCFS
Itanium?
supported
on
64
bit
platform
like
A.Yes, you can find specific packages for ia32
and ia64.
Q. Can I run ocfs on non
distributions like RedHat 9?
A.OCFS
is
primarily
developed
enterprise
for
Linux
Enterprise
systems and therefore not officially supported on
platforms other than RedHat Advanced Server 2.1
or UL. The binaries can be compiled from source
at your own risk.
Q.Do I need any specific version of United Linux
to run OCFS?
A. Yes. The latest package of OCFS will work with
UL SP1, but Oracle strongly recommends to run on
SP2 or SP2a.
Q. Can I run the latest OCFS on a plain RedHat AS
2.1, without any errata applied?
A.No. Before the errata e.12, there was a
specific module for each one of the errata
releases. After e.12, the module is generic,
working with any of the erratas released. Oracle
strongly recommends the usage of the latest
errata available.
Q.What is the advantage of running ocfs against
raw devices?
A.There are many advantages on running OCFS. The
biggest and more important is that it looks and
feels like a regular filesystem, so users don't
have
to
worry
with
all
the
raw
devices
administration difficulties. Also, users don't
have to create a partition for each datafile, a
few partitions or even a single partition,
depending on the needs and storage configuration,
may be able to handle big databases. It also
eliminate the restriction of 255 raw partitions
limit on Linux, giving almost endless number of
datafiles that can be created.
Q.What happen if I have to change the IP Address
of my systems?
A.Changing IP address on the same interface used
by the ocfs will require you to change the IP
Address specified in the /etc/ocfs.conf file
only.
Q.My Network Interface Card had to be replaced.
Do I need to do something?
A.Yes. Run the utility “ocfs_uid_gen -r” to update
the /etc/ocfs.conf file with the new uid.
Q.I have a partition that is not mounted. How do
I know if it is an ocfs partition or not?
A. Run the command “debugocfs -h /dev/xxxx” as root. If it
is an ocfs partition, the signature will show “OracleCFS”.
For all other type of filesystem, it will just return
something else, not necessarily the fstype.
Q.Can I use my OCFS partition to store regular
files?
A.Like Oracle Distribution, regular files are not supported
on OCFS at the moment. This is planned for OCFS 2.
Q.How much do I lose in
compared to raw devices?
terms
of
performance
A.Not too much. Without async I/O, performance have been
fluctuating from 0 to 5% of raw devices. Considering all
the benefits of a filesystem against raw, the loss is not
that big. Using async I/O, ocfs has shown to be about 5%
faster than raw devices.
Q.How do I enable async I/O on Oracle using OCFS?
A.First of all, it is necessary to relink the Oracle RDBMS
to enable asynchronous I/O (cd $ORACLE_HOME/rdbms/lib; make
-f ins_rdbms.mk async_on). Then add to the init.ora file,
the parameters :
filesystemio_options=setall
_dbwr_async_io=TRUE
tape_asynch_io=true
disk_asynch_io=true
Q.How do I backup my OCFS files? Can I use tar or
other OS command?
A.The best way to perform an Oracle backup is to use RMAN.
Since a lot of users have their own scripts that usually
uses tar or other OS commands, we have made available a set
of commands that support direct_io, so that users can take
hot
backup
using
tar,
cp,
dd.
Go
to
http://oss.oracle.com/projects/coreutils/files/ to get the
fileutils/coreutils appropriate to your environment. Note
that for RHAS 2.1 and SLES8, the package is called
fileutils and for RHEL3, it is called coreutils.
Q.Is it possible
partition?
to
resize
an
existing
OCFS
A.Yes, tuneocfs command will do that. Please refer to the
tuneocfs manpages for additional information on syntax.
Q.I'm having problems with OCFS. How can I debug
OCFS?
A.One needs to be careful enabling debug on ocfs because it
usually fills the /var/log/messages file very quickly. To
enable OCFS tracing:
echo -1 > /proc/sys/kernel/ocfs/debug_level
echo -1 > /proc/sys/kernel/ocfs/debug_context
To disable OCFS tracing:
echo 0 > /proc/sys/kernel/ocfs/debug_level
echo 0 > /proc/sys/kernel/ocfs/debug_context
Q.Can I run OCFS in a stand-alone system? What
are the advantages of running it?
A.Yes. There is no problems on running OCFS in a standalone system since you use it for database files only
(Datafile, logfiles, controlfiles, srvm configuration file
and Oracle Cluster Manager quorum file). The advantage of
using it on stand-alone is that if one needs to upgrade to
a RAC environment, it will be a smooth transition,
basically adding a new node, recompiling the Oracle RDBMS
kernel to enable RAC mode and add a new online log thread.
Q.I have a database running on OCFS in a standalone node. Why it is so slow compared to other
stand-alone systems running on ext3?
A.OCFS does direct_io, which bypass the Linux cache. Ext3
in other hands makes full usage of the Linux cache.
Depending on the memory available, it is possible that one
will mostly find the data in cache, not having to fetch
from the disk. But this condition happens only if there are
enough free memory for cache. As you use more and more
memory, performance drops significantly.
Q.How can I obtain more information about OCFS?
A.Go to http://oss.oracle.com/ocfs. You will be able to
find documents, sources, binaries
and all sort of
information.
Q.I have a customized RedHat AS kernel on my
system. Does Oracle support OCFS and the RDBMS on
it?
A.No. Oracle only supports standard kernels for OCFS and
Oracle RDBMS. It would be impossible to track and reproduce
any problem without knowing all the customizations that
were performed in a specific installation.
Q.How do I know if my OCFS version is officially
supported by Oracle?
A.Check the file /var/log/messages for a message that looks
like “Oracle Cluster FileSystem 1.0.9-PROD Wed Jul 30
16:48:00 PDT 2003 (build 686e542792ebee44eaa1f)”
Q.Is
OCFS is mountable on Linux and Windows 2000
simultaneously?
A.No. The on-disk format is not the same. Is similar... but
not the same.
Appendix A
Ocfs-support package Information:
Name
: ocfs-support
Relocations: (not relocatable)
Version
: 1.0.9
Vendor: Oracle Corporation
Release
:5
Build Date: Wed 30 Jul 2003 07:48:40 PM EDT
Install date: (not installed)
Build Host: ca-build1.us.oracle.com
Group
: System Environment/Kernel
Source RPM: ocfs-2.4.9-e-1.0.9-5.src.rpm
Size
: 698099
License: GPL
Packager : nobody <nobody@oracle.com>
URL
: http://ocfs.otncast.otnxchange.oracle.com/servlets/ProjectHome
Summary
: Support programs for the Oracle Cluster Filesystem
Description :
Support programs for using the Oracle Cluster Filesystem.
/etc/init.d/ocfs
/sbin/fsck.ocfs
/sbin/load_ocfs
/sbin/mkfs.ocfs
/sbin/mounted.ocfs
/sbin/ocfs_uid_gen
Ocfs-tools package information:
Name
: ocfs-tools
Relocations: (not relocatable)
Version
: 1.0.9
Vendor: Oracle Corporation
Release
:5
Build Date: Wed 30 Jul 2003 07:48:40 PM EDT
Install date: (not installed)
Build Host: ca-build1.us.oracle.com
Group
: System Environment/Kernel
Source RPM: ocfs-2.4.9-e-1.0.9-5.src.rpm
Size
: 169381
License: GPL
Packager : nobody <nobody@oracle.com>
URL
: http://ocfs.otncast.otnxchange.oracle.com/servlets/ProjectHome
Summary
: Tools for managing the Oracle Cluster Filesystem
Description :
Tools to manage the Oracle Cluster Filesystem
/usr/bin
/usr/bin/cdslctl
/usr/bin/debugocfs
/usr/bin/extfinder
/usr/bin/ocfstool
/usr/share
/usr/share/man
/usr/share/man/man1
/usr/share/man/man1/cdslctl.1.gz
/usr/share/man/man1/ocfstool.1.gz
Ocfs Module Package Information:
This
example
show
the
information about
the
enterprise
module.
The
same files will be
found in the other
supported
versions (summit,
smp, u)
Name
: ocfs-2.4.9-e-enterprise
Relocations: (not relocateable)
Version
: 1.0.9
Vendor: Oracle Corporation
Release
:5
Build Date: Wed 30 Jul 2003 07:48:40 PM EDT
Install date: (not installed)
Build Host: ca-build1.us.oracle.com
Group
: System Environment/Kernel
Source RPM: ocfs-2.4.9-e-1.0.9-5.src.rpm
Size
: 1104069
License: GPL
Packager : nobody <nobody@oracle.com>
URL
: http://ocfs.otncast.otnxchange.oracle.com/servlets/ProjectHome
Summary
: The Oracle Cluster Filesystem for enterprise systems.
Description :
OCFS is the Oracle Cluster Filesystem. This package is compiled for
symmetric processor kernels on machines with more than 4GB of RAM.
/lib/modules/2.4.9-e-enterprise-ABI/ocfs
/lib/modules/2.4.9-e-enterprise-ABI/ocfs-noaio
/lib/modules/2.4.9-e-enterprise-ABI/ocfs-noaio/ocfs.o
/lib/modules/2.4.9-e-enterprise-ABI/ocfs/ocfs.o
OCFS - Oracle Clustered Filesystem for Linux
Users Guide.
August 2003
Author: Marcos E. Matsunaga
Contributing Authors: Wim Coekaerts, Kurt Hackel, Sunil Mushran, Manish Singh and
Mark Fasheh.
Oracle Corporation
World Headquarters
500 Oracle Parkway
Redwood Shores, CA 94065
U.S.A.
Worldwide Inquiries:
Phone: +1.650.506.7000
Fax: +1.650.506.7200
www.oracle.com
For information about the Oracle Open Source, visit us at http://oss.oracle.com
Oracle is a registered trademark of Oracle Corporation. Various
product and service names referenced herein may be trademarks
of Oracle Corporation. All other product and service names
mentioned may be trademarks of their respective owners.
Copyright © 2001 Oracle Corporation
All
rights
reserved.