UDEV Rules for OEL6 on multipathed device

Questions relating to Oracle Real Application Clusters (RAC) and Clusterware.

Moderator: Tim...

UDEV Rules for OEL6 on multipathed device

Postby jcnars » Tue Sep 25, 2012 1:51 am

Thanks much Tim for the article on configuring udev rules for OEL6

I've been hurting myself really hard on how do you make it to work for a PARTITION ON A MULTIPATH-ED disk.
For example:
a LUN 360a98000653354596b4a6d584d4b5055 with WWID is configured in /etc/multipath.conf as:
Code: Select all
        multipath {
                wwid                    360a98000653354596b4a6d584d4b5055
                alias                   index
        }

It then shows up as /dev/mapper/index.
Then if a partition is created on /dev/mapper/index, it shows up as /dev/mapper/index1 (after kpatx -a /dev/mapper/index).

The problem is I am not able to create the ASM Disk ON THE PARTITION.
Because scsi_id doesn't return anything for the partition.
Code: Select all
[root@oralina udev]# scsi_id --page=0x83 --whitelisted --replace-whitespace --device=/dev/mapper/index
360a98000653354596b4a6d584d4b5055
[root@oralina udev]#
[root@oralina udev]# scsi_id --page=0x83 --whitelisted --replace-whitespace --device=/dev/mapper/index1
[root@oralina udev]#


So, the following works on the /dev/mapper/index...as expected

Code: Select all
KERNEL=="dm-*", PROGRAM=="scsi_id --page=0x83 --whitelisted --replace-whitespace --device=/dev/%k", RESULT=="360a98000653354596b4a6d584d4b5055", NAME+="oracleasm/disks-INDEX", OWNER="oracle", GROUP="dba", MODE="0660"


But, I have no idea on how to make it work on top of partitioned disk.

Thanks much.
jcnars
Member
 
Posts: 2
Joined: Tue Sep 25, 2012 1:37 am

Re: UDEV Rules for OEL6 on multipathed device

Postby jcnars » Tue Sep 25, 2012 3:34 pm

Alright, I got it to work based on this life-saver:
https://murtazahabib.wordpress.com/2012/08/20/4/comment-page-1/

Discovered in the evening that all the partitions I had created on the LUNs (based on the NetApp knowledge base best practice) had disappeared.
After some research found that the udev rule were operating on the parent device rather than the partitioned device as scsi_id draws a blank for partitions.
Once I created & mounted a diskgroup on that parent device, ASM wipes out the partition (of course!).

After some study of the udevadm tool, finalized this for udev rules:
Code: Select all
ENV{DM_UUID}==”part1-mpath-360a98000653354596b4a6d584d4b5055", NAME+="oracleasm/disk-indexpart", OWNER="oracle", GROUP="dba", MODE="0660"
ENV{DM_UUID}==”part1-mpath-360a980006533546156346d584c4a544f", NAME+="oracleasm/disk-redobpart", OWNER="oracle", GROUP="dba", MODE="0660"
ENV{DM_UUID}==”part1-mpath-360a98000653354596b4a6d584d4e6f63", NAME+="oracleasm/disk-datapart", OWNER="oracle", GROUP="dba", MODE="0660"
ENV{DM_UUID}==”part1-mpath-360a980006533546156346d584c462d37", NAME+="oracleasm/disk-redoapart", OWNER="oracle", GROUP="dba", MODE="0660"

What the above does is:
While evaluating device rules (i.e while scanning devices), if it comes across a device by the attribute
Code: Select all
DM_UUID==part1-mpath-360a98000653354596b4a6d584d4b5055
, it will create the
Code: Select all
/dev/oracleasm/disk-indexpart
device with the ownerships
Code: Select all
oracle:dba
.

To see more about the attributes of each device,
udevadm info -p /devices/virtual/block/dm-8 -q all
, where
Code: Select all
 /devices/virtual/block/dm-8
is the multipathed device under
Code: Select all
/sys/block/
.

Code: Select all
[root@oralina ~]# udevadm info -p /devices/virtual/block/dm-8  -q all
P: /devices/virtual/block/dm-8
N: oracleasm/disk-indexpart
L: 10
S: mapper/indexp1
S: disk/by-id/dm-name-indexp1
S: disk/by-id/dm-uuid-part1-mpath-360a98000653354596b4a6d584d4b5055
S: disk/by-label/\x28
S: block/253:8
E: UDEV_LOG=3
E: DEVPATH=/devices/virtual/block/dm-8
E: MAJOR=253
E: MINOR=8
E: DEVNAME=/dev/oracleasm/disk-indexpart
E: DEVTYPE=disk
E: SUBSYSTEM=block
E: DM_SBIN_PATH=/sbin
E: DM_UDEV_PRIMARY_SOURCE_FLAG=1
E: DM_NAME=indexp1
E: DM_UUID=part1-mpath-360a98000653354596b4a6d584d4b5055
E: DM_SUSPENDED=0
E: DM_UDEV_RULES_VSN=2
E: ID_FS_LABEL=_
E: ID_FS_LABEL_ENC=\x28
E: ID_FS_TYPE=oracleasm
E: ID_FS_USAGE=filesystem
E: MPATH_SBIN_PATH=/sbin
E: LVM_SBIN_PATH=/sbin
E: DEVLINKS=/dev/mapper/indexp1 /dev/disk/by-id/dm-name-indexp1 /dev/disk/by-id/dm-uuid-part1-mpath-360a98000653354596b4a6d584d4b5055 /dev/disk/by-label/\x28 /dev/block/253:8


Since, the above DM_UUID is tied to both the WWID and also the partition number, it’s not subject to change & guaranteed to be unique. In other will always give persistent naming for disks, even if the LUN is discovered arbitrarily as sd*.

Oracle has a heck of a lot of metalink docs on how to configure udev for RHEL5/OEL5, RHEL4/OEL4.
But not one on OEL6/RHEL6. No blog entries, apart from the life-saver above.
Many blogs out there talk about how to use udev on Virtual devices in their laptops. Some even go to the multipathing area, but not one except the one talk about udev for partitions on multipath-ed disks.

So, to speak:
LUN -> multipath-ed pseudo-device -> partition on this device -> udev rules on this partitioned device -> to create ASM disks -> to create ASM diskgroups
jcnars
Member
 
Posts: 2
Joined: Tue Sep 25, 2012 1:37 am

Re: UDEV Rules for OEL6 on multipathed device

Postby Tim... » Tue Sep 25, 2012 8:46 pm

Wow. Very complete answer to your own question. :)

Cheers

Tim...
Tim...
Oracle ACE Director
Oracle ACE of the Year 2006 - Oracle Magazine Editors Choice Awards
OakTable Member
OCP DBA 7.3, 8, 8i, 9i, 10g, 11g
OCP Advanced PL/SQL Developer
Oracle Database: SQL Certified Expert
My website: http://www.oracle-base.com
My blog: http://www.oracle-base.com/blog
Tim...
Site Admin
 
Posts: 17966
Joined: Mon Nov 01, 2004 5:56 pm
Location: England, UK


Return to Oracle RAC and Clusterware

Who is online

Users browsing this forum: No registered users and 1 guest