Anyone who has used Oracle ASM on Linux in the last 5 – 10 years is most likely familiar with ASMLib. Oracle provides this software and associated kernel module in order to easily manage disks that are intended to be used as ASM disks for database storage. When it was first introduced there was no reliable way to persistently name disks and assign permissions easily on Linux. An external disk might be /dev/sdc one time, and /dev/sdg the next reboot. What’s an admin to do? ASMLib allowed us to label the disks so that no matter what device name they might have, we can still refer to them within ASM and know which disk we’re talking to, as well as handling the permissions needed. However, as the years have moved on the udev system has appeared and it seems like it does pretty much everything ASMLib can do.
Is it time to consider a change?
Why even consider changing? Well I first found myself in a bind when I was setting up a three node Oracle 11gR2 RAC within a VMWare vSphere hypervisor. Since this is just a POC setup I chose to go with the common strategy of using CentOS Linux to stand in for Red Hat Enterprise Linux 6.4. Setup was going swimmingly until I went to get the asmlib kernel module for this setup. Turns out that Oracle stopped providing this module after RHEL/Centos 5. It makes perfect sense, but it leaves me without a kernel module.
I remembered the udev system is capable of all sorts of manipulations to devices and quickly found this great documentation on fundamental rule writing. With a little help from random sources on the Internet (linked below) I quickly had a set of disks in place that I could go ahead and install Oracle with.
Hard drives can be sliced and diced and presented to the host several different ways. Let’s look at how udev can help us in different situations.
The Easy Way
The simplest setup is you have a single hard drive attached to your host that you want to use for ASM disks. Say you have an internal SCSI drive for the OS/software install, and an external drive that always appears as /dev/sdb. Using fdisk or gparted you can create several partitions on this drive. In my example sdb1, sdb2 and sdb3 were created as 2 GB partitions meant for the OCR and Vote diskgroup required when Grid Infrastructure is installed. sdb5 and sdb6 are both 50GB and destined for a diskgroup named +DATA.
In order for Oracle to make use of these disks they need to be owned by the oracle user and group with read/write permissions for owner and group. To make this happen change directories to /etc/udev/rules.d on your host. If you list the directory you will probably have a few files named things like 70-persistent-net.rules and the like. Any filename that ends in .rules will be evaluated, lower numbered rule files will be evaluated before higher numbered ones. Let’s create one called 99-oracle.rules to put our rules in so that they run near the end of processing.
Let’s look at some rules we can put into that file to do some disk magic:
KERNEL==”sdb1″, NAME+=”oracleasm/ocrvote1″, OWNER=”oracle”, GROUP=”dba”, MODE=”0660″
KERNEL==”sdb2″, NAME+=”oracleasm/ocrvote2″, OWNER=”oracle”, GROUP=”dba”, MODE=”0660″
KERNEL==”sdb3″, NAME+=”oracleasm/ocrvote3″, OWNER=”oracle”, GROUP=”dba”, MODE=”0660″
KERNEL==”sdb5″, NAME+=”oracleasm/data1″, OWNER=”oracle”, GROUP=”dba”, MODE=”0660″
KERNEL==”sdb6″, NAME+=”oracleasm/data2″, OWNER=”oracle”, GROUP=”dba”, MODE=”0660″
- there is a kernel device named ‘/dev/sdb1’
- Create a device called /dev/oracleasm/ocrvote1
- AND Change its owner to ORACLE user
- AND Change its group to DBA
- AND Change the file permissions to 0660 (owner and group read/write).
total 0brw-rw—-. 1 oracle dba 8, 21 Dec 18 13:57 data1brw-rw—-. 1 oracle dba 8, 22 Dec 18 13:57 data2brw-rw—-. 1 oracle dba 8, 17 Dec 18 12:29 ocrvote1brw-rw—-. 1 oracle dba 8, 18 Dec 18 13:57 ocrvote2brw-rw—-. 1 oracle dba 8, 19 Dec 18 13:57 ocrvote3
A Slightly Less Trivial Case
- the kernel device matches /dev/sd* (any scsi disk device)
- AND put it in the dba group
- AND the device is a BLOCK device
- AND the block device is a DISK
- AND the disk has serial number 36000c290e2feb0a79b4ebe855688ae7c
- Name it /dev/oracleasm/ocrvote1
- AND make it owned by user oracle
- AND change permissions to 0660 (owner,group read/write)