Oracle RAC - udev rules for the ASM disks
Oracle RAC Solution · Config document · referenced from Storage and ASM disks
noteThe SCSI identifiers in this file are made-up values of the right shape. On another system they come from
/lib/udev/scsi_id -g -u /dev/sdX.The udev rules that give the first partition of each of the five shared iSCSI LUNs two stable names in /dev and the owner, group and mode Oracle ASM needs, whatever sdX name the kernel assigns.
| Item | Value |
|---|---|
| Path | /etc/udev/rules.d/59-persistent-disk.rules |
| On | oradb01 and oradb02, identical, because both nodes read the same identifiers from the shared LUNs |
| Shown here | the whole file |
| Owner, group and mode set by the rules | grid, asmadmin, 0660 |
| Activated with | not recorded in the notes |
| Operating system | SUSE Linux Enterprise Server 11 SP3 |
The file
# The first partition of the block device with ID "14f504e46494c45526469736b30312d303030302d30303031" will be called "ORADB-CRS" and "ASMDISK1". KERNEL=="sd?1", SUBSYSTEM=="block", PROGRAM=="/lib/udev/scsi_id -g -u -d /dev/$parent", RESULT=="14f504e46494c45526469736b30312d303030302d30303031", SYMLINK+="ASMDISK1", OWNER="grid", GROUP="asmadmin", MODE="0660" KERNEL=="sd?1", SUBSYSTEM=="block", PROGRAM=="/lib/udev/scsi_id -g -u -d /dev/$parent", RESULT=="14f504e46494c45526469736b30312d303030302d30303031", SYMLINK+="ORADB-CRS", OWNER="grid", GROUP="asmadmin", MODE="0660" # The first partition of the block device with ID "14f504e46494c45526469736b30322d303030302d30303032" will be called "ORADB-DATA01" and "ASMDISK2". KERNEL=="sd?1", SUBSYSTEM=="block", PROGRAM=="/lib/udev/scsi_id -g -u -d /dev/$parent", RESULT=="14f504e46494c45526469736b30322d303030302d30303032", SYMLINK+="ASMDISK2", OWNER="grid", GROUP="asmadmin", MODE="0660" KERNEL=="sd?1", SUBSYSTEM=="block", PROGRAM=="/lib/udev/scsi_id -g -u -d /dev/$parent", RESULT=="14f504e46494c45526469736b30322d303030302d30303032", SYMLINK+="ORADB-DATA01", OWNER="grid", GROUP="asmadmin", MODE="0660" # The first partition of the block device with ID "14f504e46494c45526469736b30332d303030302d30303033" will be called "ORADB-DATA02" and "ASMDISK3". KERNEL=="sd?1", SUBSYSTEM=="block", PROGRAM=="/lib/udev/scsi_id -g -u -d /dev/$parent", RESULT=="14f504e46494c45526469736b30332d303030302d30303033", SYMLINK+="ASMDISK3", OWNER="grid", GROUP="asmadmin", MODE="0660" KERNEL=="sd?1", SUBSYSTEM=="block", PROGRAM=="/lib/udev/scsi_id -g -u -d /dev/$parent", RESULT=="14f504e46494c45526469736b30332d303030302d30303033", SYMLINK+="ORADB-DATA02", OWNER="grid", GROUP="asmadmin", MODE="0660" # The first partition of the block device with ID "14f504e46494c45526469736b30342d303030302d30303034" will be called "ORADB-FRA" and "ASMDISK4". KERNEL=="sd?1", SUBSYSTEM=="block", PROGRAM=="/lib/udev/scsi_id -g -u -d /dev/$parent", RESULT=="14f504e46494c45526469736b30342d303030302d30303034", SYMLINK+="ASMDISK4", OWNER="grid", GROUP="asmadmin", MODE="0660" KERNEL=="sd?1", SUBSYSTEM=="block", PROGRAM=="/lib/udev/scsi_id -g -u -d /dev/$parent", RESULT=="14f504e46494c45526469736b30342d303030302d30303034", SYMLINK+="ORADB-FRA", OWNER="grid", GROUP="asmadmin", MODE="0660" # The first partition of the block device with ID "14f504e46494c45526469736b30352d303030302d30303035" will be called "ORADB-ONTRLG" and "ASMDISK5". KERNEL=="sd?1", SUBSYSTEM=="block", PROGRAM=="/lib/udev/scsi_id -g -u -d /dev/$parent", RESULT=="14f504e46494c45526469736b30352d303030302d30303035", SYMLINK+="ASMDISK5", OWNER="grid", GROUP="asmadmin", MODE="0660" KERNEL=="sd?1", SUBSYSTEM=="block", PROGRAM=="/lib/udev/scsi_id -g -u -d /dev/$parent", RESULT=="14f504e46494c45526469736b30352d303030302d30303035", SYMLINK+="ORADB-ONTRLG", OWNER="grid", GROUP="asmadmin", MODE="0660"
| Identifier ends in | Kernel device when the notes were written | Symlinks |
|---|---|---|
…30312d303030302d30303031 | sdc1 | /dev/ORADB-CRS, /dev/ASMDISK1 |
…30322d303030302d30303032 | sdd1 | /dev/ORADB-DATA01, /dev/ASMDISK2 |
…30332d303030302d30303033 | sde1 | /dev/ORADB-DATA02, /dev/ASMDISK3 |
…30342d303030302d30303034 | sdf1 | /dev/ORADB-FRA, /dev/ASMDISK4 |
…30352d303030302d30303035 | sdg1 | /dev/ORADB-ONTRLG, /dev/ASMDISK5 |
Every rule has the same parts.
| Part | Meaning |
|---|---|
KERNEL=="sd?1" | matches the first partition of a disk with a one-letter name, sda1 to sdz1 |
SUBSYSTEM=="block" | block devices only |
PROGRAM=="/lib/udev/scsi_id -g -u -d /dev/$parent" | runs scsi_id against the parent disk of the partition; -g treats the device as whitelisted, -u replaces spaces in the result, -d names the device node |
RESULT=="…" | compares the output of that program with the identifier of one LUN |
SYMLINK+="…" | adds one name in /dev that points to the partition |
OWNER, GROUP, MODE | set on the device node of the partition itself, not on the symlink |
Each disk has two rules that differ only in the symlink. One rule with SYMLINK+="ASMDISK1 ORADB-CRS" would do the same; the file is shown as it was. The program runs once per rule and per partition, so scsi_id is called ten times for every sd?1 that appears, including the system disk.
The ORADB-* names are what oracleasm createdisk is given in ASMLib commands. The ASMDISKn symlinks have the same names as the ASMLib labels but are something else: ASM finds its disks as ORCL:ASMDISKn through ASMLib, and nothing in the notes uses /dev/ASMDISKn.
All five identifiers begin with 14f504e46494c4552, which after the leading 1 is the ASCII string OPNFILER in hexadecimal. The notes say nothing else about the storage system behind the LUNs.
Checked against udev in SLES 15 SP7 and SLES 16.0
| As built | Today |
|---|---|
/lib/udev/scsi_id | The path is /usr/lib/udev/scsi_id. The short options are unchanged; -g is now spelled --allowlisted in its long form. The program is not deprecated |
KERNEL, SUBSYSTEM, PROGRAM, RESULT, SYMLINK+=, OWNER, GROUP, MODE, $parent | Still valid syntax |
PROGRAM calling scsi_id in every rule | Not needed: 60-persistent-storage.rules already runs scsi_id and partitions inherit ID_SERIAL, so a rule can match ENV{DEVTYPE}=="partition", ENV{ID_SERIAL}=="<id>" |
| two rules per disk | One rule with two SYMLINK+= values does the same |
file name 59-persistent-disk.rules | Rules files are processed in lexicographic order. A 59- file runs before 60-persistent-storage.rules: right for the PROGRAM form, too early for a rule on ID_SERIAL, which needs a number above 60. Oracle's example name is 99-oracle-asmdevices.rules |
| activation not recorded | udevadm control --reload, then udevadm trigger; reloading alone does not touch devices that already exist |
| udev ownership and ASMLib together | Oracle describes device persistence with ASMLib and with udev rules as alternatives. With ASMLib the access comes from the driver interface, so OWNER, GROUP and MODE here were redundant |
With the path changed the file still works. I would write it today as five rules on ID_SERIAL in a file numbered above 60, or leave udev out and let ASMLib find the disks by label.