F How to Complete Preinstallation Tasks Manually

F.4.1.2.1 Installing and Configuring the Oracle ASM Library Driver Software

ASMLIB is already included with Oracle Linux packages, and with SUSE Linux Enterprise Server. If you are a member of the Unbreakable Linux Network, then you can install the ASMLIB RPMs by subscribing to the Oracle Linux channel, and using yum to retrieve the most current package for your system and kernel. For additional information, see the following URL:

http://www.oracle.com/technetwork/topics/linux/asmlib/index-101839.html

To install and configure the ASMLIB driver software manually, follow these steps:

1. Enter the following command to determine the kernel version and architecture of the system:

   # uname -rm
   

2. Download the required ASMLIB packages from the Oracle Technology Network website:

   http://www.oracle.com/technetwork/server-storage/linux/downloads/index-088143.html
   

   Note:

   You must install oracleasm-support package version 2.0.1 or later to use ASMLIB on Red Hat Enterprise Linux 5 Advanced Server. ASMLIB is already included with SUSE Linux Enterprise Server distributions.

   See Also:

   My Oracle Support Note 1089399.1 for information about ASMLIB support with Red Hat distributions:

   https://support.oracle.com/CSP/main/article?cmd=show&type=NOT&id=1089399.1

3. Switch user to the root user:

   $ su -
   

4. Install the following packages in sequence, where version is the version of the ASMLIB driver, arch is the system architecture, and kernel is the version of the kernel that you are using:

   oracleasm-support-version.arch.rpm
   oracleasm-kernel-version.arch.rpm
   oracleasmlib-version.arch.rpm
   

   Enter a command similar to the following to install the packages:

   # rpm -ivh oracleasm-support-version.arch.rpm \
              oracleasm-kernel-version.arch.rpm \
              oracleasmlib-version.arch.rpm
   

   For example, if you are using the Red Hat Enterprise Linux 5 AS kernel on an AMD64 system, then enter a command similar to the following:

   # rpm -ivh oracleasm-support-2.1.3-1.el5.x86_64.rpm \
        oracleasm-2.6.18-194.26.1.el5xen-2.0.5-1.el5.x86_64.rpm \
        oracleasmlib-2.0.4-1.el5.x86_64.rpm
   

5. Enter the following command to run the oracleasm initialization script with the configure option:

   # /usr/sbin/oracleasm configure -i
   

   Note:

   The oracleasm command in /usr/sbin is the command you should use. The /etc/init.d path is not deprecated, but the oracleasm binary in that path is now used typically for internal commands.

6. Enter the following information in response to the prompts that the script displays:

   Table F-2 ORACLEASM Configure Prompts and Responses

   Prompt	Suggested Response
   Default user to own the driver interface:	Standard groups and users configuration: Specify the Oracle software owner user (for example, oracle). Job role separation groups and users configuration: Specify the Oracle Grid Infrastructure software owner user (for example, grid).
   Default group to own the driver interface:	Standard groups and users configuration: Specify the OSDBA group for the database (for example, dba). Job role separation groups and users configuration: Specify the OSASM group for storage administration (for example, asmadmin).
   Start Oracle ASM Library driver on boot (y/n):	Enter y to start the Oracle Automatic Storage Management library driver when the system starts.
   Scan for Oracle ASM disks on boot (y/n)	Enter y to scan for Oracle ASM disks when the system starts.

   
   

   The script completes the following tasks:

   • Creates the /etc/sysconfig/oracleasm configuration file

   • Creates the /dev/oracleasm mount point

   • Mounts the ASMLIB driver file system

     Note:

     The ASMLIB driver file system is not a regular file system. It is used only by the Oracle ASM library to communicate with the Oracle ASM driver.

7. Enter the following command to load the oracleasm kernel module:

   # /usr/sbin/oracleasm init
   

8. Repeat this procedure on all nodes in the cluster where you want to install Oracle RAC.

F.4.1.2.2 Configuring Disk Devices to Use Oracle ASM Library Driver on x86 Systems

To configure the disk devices to use in an Oracle ASM disk group, follow these steps:

1. If you intend to use IDE, SCSI, or RAID devices in the Oracle ASM disk group, then follow these steps:

   1. If necessary, install or configure the shared disk devices that you intend to use for the disk group and restart the system.

   2. Enter the following command to identify the device name for the disks to use, enter the following command:

      # /sbin/fdisk -l
      

      Depending on the type of disk, the device name can vary. Table F-3 describes some types of disk paths:

      Table F-3 Types of Linux Storage Disk Paths

      Disk Type	Device Name Format	Description
      IDE disk	/dev/hdxn	In this example, x is a letter that identifies the IDE disk and n is the partition number. For example, /dev/hda is the first disk on the first IDE bus.
      SCSI disk	/dev/sdxn	In this example, x is a letter that identifies the SCSI disk and n is the partition number. For example, /dev/sda is the first disk on the first SCSI bus.
      RAID disk	/dev/rd/cxdypz /dev/ida/cxdypz	Depending on the RAID controller, RAID devices can have different device names. In the examples shown, x is a number that identifies the controller, y is a number that identifies the disk, and z is a number that identifies the partition. For example, /dev/ida/c0d1 is the second logical drive on the first controller.

      
      

      To include devices in a disk group, you can specify either whole-drive device names or partition device names.

      Note:

      Oracle recommends that you create a single whole-disk partition on each disk.

   3. Use either fdisk or parted to create a single whole-disk partition on the disk devices.

2. Enter a command similar to the following to mark a disk as an Oracle ASM disk:

   # /usr/sbin/oracleasm createdisk DISK1 /dev/sdb1
   

   In this example, DISK1 is the name you assign to the disk.

   Note:

   The disk names that you specify can contain uppercase letters, numbers, and the underscore character. They must start with an uppercase letter.

   If you are using a multi-pathing disk driver with Oracle ASM, then make sure that you specify the correct logical device name for the disk.

3. To make the disk available on the other nodes in the cluster, enter the following command as root on each node:

   # /usr/sbin/oracleasm scandisks
   

   This command identifies shared disks attached to the node that are marked as Oracle ASM disks.

F.4.1.2.3 Configuring Disk Devices to Use ASM Library Driver on IBM zSeries Systems

1. If you formatted the DASD with the compatible disk layout, then enter a command similar to the following to create a single whole-disk partition on the device:

   # /sbin/fdasd -a /dev/dasdxxxx
   

2. Enter a command similar to the following to mark a disk as an ASM disk:

   # /etc/init.d/oracleasm createdisk DISK1 /dev/dasdxxxx
   

   In this example, DISK1 is a name that you want to assign to the disk.

   Note:

   The disk names that you specify can contain uppercase letters, numbers, and the underscore character. They must start with an uppercase letter.

   If you are using a multi-pathing disk driver with ASM, then make sure that you specify the correct logical device name for the disk.

3. To make the disk available on the other cluster nodes, enter the following command as root on each node:

   # /etc/init.d/oracleasm scandisks
   

   This command identifies shared disks attached to the node that are marked as ASM disks.

F.4.1.2.4 Administering the Oracle ASM Library Driver and Disks

To administer the Oracle Automatic Storage Management library driver (ASMLIB) and disks, use the /usr/sbin/oracleasm initialization script with different options, as described in Table F-4:

oracleasm configure -i

oracleasm enable

oracleasm restart

oracleasm createdisk VOL1 /dev/sda1

oracleasm deletedisk VOL1

oracleasm querydisk

oracleasm listdisks

oracleasm scandisks

oracleasm renamedisk VOL1 VOL2

F.4.1.3.1 About Using Oracle ASM with Multipath Disks

Oracle ASM requires that each disk is uniquely identified. If the same disk appears under multiple paths, then it causes errors. In a multipath disk configuration, the same disk can appear three times:

1. The initial path to the disk

2. The second path to the disk

3. The multipath disk access point

For example: If you have one local disk, /dev/sda, and one disk attached with external storage, then your server shows two connections, or paths, to that external storage. The Linux SCSI driver shows both paths. They appear as /dev/sdb and /dev/sdc. The system may access either /dev/sdb or /dev/sdc, but the access is to the same disk.

If you enable multipathing, then you have a multipath disk (for example, /dev/multipatha), which can access both /dev/sdb and /dev sdc; any I/O to multipatha can use either the sdb or sdc path. If a system is using the /dev/sdb path, and that cable is unplugged, then the system shows an error. But the multipath disk will switch from the /dev/sdb path to the /dev/sdc path.

Most system software is unaware of multipath configurations. They can use any paths (sdb, sdc or multipatha). ASMLIB also is unaware of multipath configurations.

By default, ASMLIB recognizes the first disk path that Linux reports to it, but because it imprints an identity on that disk, it recognizes that disk only under one path. Depending on your storage driver, it may recognize the multipath disk, or it may recognize one of the single disk paths.

Instead of relying on the default, you should configure Oracle ASM to recognize the multipath disk.

F.4.1.3.2 Disk Scan Ordering

The ASMLIB configuration file is located in the path /etc/sysconfig/oracleasm. It contains all the startup configuration you specified with the command /etc/init.d/oracleasm configure. That command cannot configure scan ordering.

The configuration file contains many configuration variables. The ORACLEASM_SCANORDER variable specifies disks to be scanned first. The ORACLEASM_SCANEXCLUDE variable specifies the disks that are to be ignored.

Configure values for ORACLEASM_SCANORDER using space-delimited prefix strings. A prefix string is the common string associated with a type of disk. For example, if you use the prefix string sd, then this string matches all SCSI devices, including /dev/sda, /dev/sdb, /dev/sdc and so on. Note that these are not globs. They do not use wild cards. They are simple prefixes. Also note that the path is not a part of the prefix. For example, the /dev/ path is not part of the prefix for SCSI disks that are in the path /dev/sd*.

For Oracle Linux and Red Hat Enterprise Linux version 5, when scanning, the kernel sees the devices as /dev/mapper/XXX entries. By default, the device file naming scheme udev creates the /dev/mapper/XXX names for human readability. Any configuration using ORACLEASM_SCANORDER should use the /dev/mapper/XXX entries.

F.4.1.3.3 Configuring Disk Scan Ordering to Select Multipath Disks

To configure ASMLIB to select multipath disks first, complete the following procedure:

1. Using a text editor, open the ASMLIB configuration file /etc/sysconfig/oracleasm.

2. Edit the ORACLEASM_SCANORDER variable to provide the prefix path of the multipath disks. For example, if the multipath disks use the prefix multipath (/dev/mapper/multipatha, /dev/mapper/multipathb and so on), and the multipath disks mount SCSI disks, then provide a prefix path similar to the following:

   ORACLEASM_SCANORDER="multipath sd"
   

3. Save the file.

When you have completed this procedure, then when ASMLIB scans disks, it first scans all disks with the prefix string multipath, and labels these disks as Oracle ASM disks using the /dev/mapper/multipathX value. It then scans all disks with the prefix string sd. However, because ASMLIB recognizes that these disks have already been labeled with the /dev/mapper/multipath string values, it ignores these disks. After scanning for the prefix strings multipath and sd, Oracle ASM then scans for any other disks that do not match the scan order.

In the example in step 2, the key word multipath is actually the alias for multipath devices configured in /etc/multipath.conf under the multipaths section. For example:

multipaths {
       multipath {
               wwid                    3600508b4000156d700012000000b0000
               alias                   multipath
               ...
       }
       multipath {
               ...
               alias                   mympath
               ...
       }
       ...
}

The default device name is in the format /dev/mapper/mpath* (or a similar path).

F.4.1.3.4 Configuring Disk Order Scan to Exclude Single Path Disks

To configure ASMLIB to exclude particular single path disks, complete the following procedure:

1. Using a text editor, open the ASMLIB configuration file /etc/sysconfig/oracleasm.

2. Edit the ORACLEASM_SCANEXCLUDE variable to provide the prefix path of the single path disks. For example, if you want to exclude the single path disks /dev sdb and /dev/sdc, then provide a prefix path similar to the following:

   ORACLEASM_SCANEXCLUDE="sdb sdc"
   

3. Save the file.

When you have completed this procedure, then when ASMLIB scans disks, it scans all disks except for the disks with the sdb and sdc prefixes, so that it ignores /dev/sdb and /dev/sdc. It does not ignore other SCSI disks, nor multipath disks. If you have a multipath disk (for example, /dev/multipatha), which accesses both /dev/sdb and /dev/sdc, but you have configured ASMLIB to ignore sdb and sdc, then ASMLIB ignores these disks and instead marks only the multipath disk as an Oracle ASM disk.