Linux Storage - mppUtil commands
Linux Storage Solution · Config document · referenced from RDAC installation and operation
noteThe array and LUN WWNs (
600a0b80…) are made-up values of the right shape. mppUtil -g 0 reports AVTEnabled: Y although the vendor documentation quoted in the notes says AVT must be disabled for RDAC; the notes do not comment on it.mppUtil is the RDAC (MPP) driver's control tool: it lists the arrays the driver knows, shows its internal state per array, reports the controller and path states, and reads and sets the driver variables at run time. This document holds the usage list from my notes and the examples that were run with output on mppsrv01 on 7 September 2010.
| Item | Value |
|---|---|
| Command | /usr/sbin/mppUtil |
| Run as | root on mppsrv01 |
| Host | Red Hat Enterprise Linux AS 4, kernel 2.6.9-89.0.9.ELsmp, two QLogic HBAs host0 and host1, virtual MPP adapter host2 |
| Arrays | DS5300-SITEA (ID 0, main), DS5300-SITEB (ID 1, standby), two LUNs each, controller firmware 7.60.28.xx |
| Driver version | RDAC 09.03.0B05.0331 |
| Related | mppUpdate after a SaveSettings change, see mppUpdate and lsvdev |
The commands
The usage list as the notes keep it, one line per task; the table after it gives the task of each line. The notes keep no output for them:
$ mppUtil -d 0x80200008 $ mppUtil -d 0x0 $ mppUtil -e 4 $ mppUtil -g 1 $ mppUtil -s failback $ mppUtil -a $ mppUtil -a "array_name" $ mppUtil -o $ mppUtil -o "ErrorLevel=0x2" $ mppUtil -S
| Command | Task, as the notes word it |
|---|---|
mppUtil -d 0x80200008 | show every debug message for synchronous I/O activity, logging to the console |
mppUtil -d 0x0 | turn all debug messages off |
mppUtil -e 4 | show controller failover and fatal errors |
mppUtil -g 1 | show the RDAC driver's internal information for the target with ID 1 |
mppUtil -s failback | find the failed controller that has come back online |
mppUtil -a | list every disk array this server can see |
mppUtil -a "array_name" | show the driver's internal information for the array with that name |
mppUtil -o | show every available special option (feature options) |
mppUtil -o "ErrorLevel=0x2" | set the special option ErrorLevel to 0x2 |
mppUtil -S | show the run-time information of the controllers and of the paths to the server |
The examples with output
Every disk array this machine can see:
$ mppUtil -aoutput 12 lines
Hostname = mppsrv01 Domainname = (none) Time = GMT 09/07/2010 08:54:48 --------------------------------------------------------------- Info of Array Module's seen by this Host. --------------------------------------------------------------- ID WWN Type Name --------------------------------------------------------------- 0 600a0b800012345600000000aabbcc01 FC DS5300-SITEA 1 600a0b800012345700000000aabbcc02 FC DS5300-SITEB ---------------------------------------------------------------
Every special option, with its current, default, minimum and maximum value:
$ mppUtil -ooutput 13 lines
Current Default Minimum Maximum Variable Options Value Value Value Value ---------------------------------------- ---------- ---------- ---------- ---------- DebugLevel 0x0 0x0 0x0 0xffffffff ErrorLevel 0x3 0x3 0x0 0x5 DisableLUNRebalance 0x0 0x0 0x0 0x4 LoadBalancePolicy 0x1 0x1 0x0 0x2 ClassicModeFailover 0x0 0x0 0x0 0x1 BusResetTimeout 0x96 0x0 0x0 0x12c Action Options --------------------------------------- No actions defined
Setting the variable ErrorLevel to 0x2 in memory only: the change is not lasting, and after a restart the value is back to what it was. The notes keep no output:
$ mppUtil -o ErrorLevel=0x2
Setting the same variable and saving it (SaveSettings), which makes the change lasting; mppUpdate then rebuilds the initrd with the new value. Again no output is kept:
$ mppUtil -o ErrorLevel=0x2,SaveSettings $ mppUpdate
The driver's internal information for one array, named by its ID from mppUtil -a. The usage list says -g 1 shows "the target with ID 1"; the example runs -g 0, the main array:
$ mppUtil -g 0
output 123 lines
Hostname = mppsrv01
Domainname = (none)
Time = GMT 09/07/2010 08:58:08
MPP Information:
----------------
ModuleName: DS5300-SITEA SingleController: N
VirtualTargetID: 0x000 ScanTriggered: N
ObjectCount: 0x000 AVTEnabled: Y
WWN: 600a0b800012345600000000aabbcc01 RestoreCfg: N
ModuleHandle: none Page2CSubPage: Y
FirmwareVersion: 7.60.28.xx FailoverMethod: C
ScanTaskState: 0x00000000
LBPolicy: LeastQueueDepth
Controller 'A' Status:
-----------------------
ControllerHandle: none ControllerPresent: Y
UTMLunExists: N Failed: N
NumberOfPaths: 2 FailoverInProg: N
ServiceMode: N
Path #1
---------
DirectoryVertex: present Present: Y
PathState: OPTIMAL
PathId: 77000000 (hostId: 0, channelId: 0, targetId: 0)
Path #2
---------
DirectoryVertex: present Present: Y
PathState: OPTIMAL
PathId: 77010000 (hostId: 1, channelId: 0, targetId: 0)
Controller 'B' Status:
-----------------------
ControllerHandle: none ControllerPresent: Y
UTMLunExists: N Failed: N
NumberOfPaths: 2 FailoverInProg: N
ServiceMode: N
Path #1
---------
DirectoryVertex: present Present: Y
PathState: OPTIMAL
PathId: 77000001 (hostId: 0, channelId: 0, targetId: 1)
Path #2
---------
DirectoryVertex: present Present: Y
PathState: OPTIMAL
PathId: 77010001 (hostId: 1, channelId: 0, targetId: 1)
Lun Information
---------------
Lun #1 - WWN: 600a0b80001234560000ab00ddeeff01
----------------
LunObject: present CurrentOwningPath: B
RemoveEligible: N BootOwningPath: B
NotConfigured: N PreferredPath: B
DevState: OPTIMAL ReportedPresent: Y
ReportedMissing: N
NeedsReservationCheck: N
TASBitSet: N
NotReady: N
Busy: N
Quiescent: N
VD_Ownership_Transfer_Attempt_Count: 0
Controller 'A' Path
--------------------
NumLunObjects: 2 RoundRobinIndex: 0
Path #1: LunPathDevice: present
DevState: OPTIMAL
RemoveState: 0x0 StartState: 0x1 PowerState: 0x0
Path #2: LunPathDevice: present
DevState: OPTIMAL
RemoveState: 0x0 StartState: 0x1 PowerState: 0x0
Controller 'B' Path
--------------------
NumLunObjects: 2 RoundRobinIndex: 0
Path #1: LunPathDevice: present
DevState: OPTIMAL
RemoveState: 0x0 StartState: 0x1 PowerState: 0x0
Path #2: LunPathDevice: present
DevState: OPTIMAL
RemoveState: 0x0 StartState: 0x1 PowerState: 0x0
Lun #2 - WWN: 600a0b80001234570000ae00ddeeff02
----------------
LunObject: present CurrentOwningPath: A
RemoveEligible: N BootOwningPath: A
NotConfigured: N PreferredPath: A
DevState: OPTIMAL ReportedPresent: Y
ReportedMissing: N
NeedsReservationCheck: N
TASBitSet: N
NotReady: N
Busy: N
Quiescent: N
VD_Ownership_Transfer_Attempt_Count: 0
Controller 'A' Path
--------------------
NumLunObjects: 2 RoundRobinIndex: 0
Path #1: LunPathDevice: present
DevState: OPTIMAL
RemoveState: 0x0 StartState: 0x1 PowerState: 0x0
Path #2: LunPathDevice: present
DevState: OPTIMAL
RemoveState: 0x0 StartState: 0x1 PowerState: 0x0
Controller 'B' Path
--------------------
NumLunObjects: 2 RoundRobinIndex: 0
Path #1: LunPathDevice: present
DevState: OPTIMAL
RemoveState: 0x0 StartState: 0x1 PowerState: 0x0
Path #2: LunPathDevice: present
DevState: OPTIMAL
RemoveState: 0x0 StartState: 0x1 PowerState: 0x0The live report of the controller and path states for each LUN. The notes place this report before the mppBusRescan that found the standby array, and it shows the main array alone; the mppUtil -a above, time-stamped GMT 09/07/2010 08:54:48, already lists both arrays, so the examples were not kept in the order they were run:
$ mppUtil -Soutput 8 lines
H2C0T0 Active Active DS5300-SITEA
H0C0T0L001 Up H0C0T1L001 Up
H1C0T0L001 Up H1C0T1L001 Up
H0C0T0L002 Up H0C0T1L002 Up
H1C0T0L002 Up H1C0T1L002 Up
Missing Arrays
There are no missing arraysThe notes then show the mppBusRescan that found the standby array; its output is in mppBusRescan commands. The same report with both arrays known:
$ mppUtil -Soutput 13 lines
H2C0T0 Active Active DS5300-SITEA
H0C0T0L001 Up H0C0T1L001 Up
H1C0T0L001 Up H1C0T1L001 Up
H0C0T0L002 Up H0C0T1L002 Up
H1C0T0L002 Up H1C0T1L002 Up
H2C0T1 Active Active DS5300-SITEB
H1C0T2L001 Up H1C0T3L001 Up
H0C0T3L001 Up H0C0T2L001 Up
H1C0T2L002 Up H1C0T3L002 Up
H0C0T3L002 Up H0C0T2L002 Up
Missing Arrays
There are no missing arraysReading the outputs
| Output | Line | Reading |
|---|---|---|
-a | ID, WWN, Type, Name | the driver's number for the array (the argument of -g, the first field of /var/mpp/devicemapping), the array WWN (the name of its .wwn file), the transport, the array name |
-o | the six variables | the subset of /etc/mpp.conf that can be changed at run time; BusResetTimeout is the only one whose current value (0x96, 150) is not its default (0x0), and it matches BusResetTimeout=150 in the file |
-g 0 | MPP Information | the array as a module: name, virtual target ID 0x000 (target 0 on host2), WWN, controller firmware 7.60.28.xx, FailoverMethod: C, AVTEnabled: Y, load-balancing policy LeastQueueDepth (LoadBalancePolicy=1 in the file) |
-g 0 | Controller 'A' Status, Controller 'B' Status | per controller: present, not failed, two paths each, no failover in progress; each path OPTIMAL with a PathId whose decoding the output prints itself, 77000000 (hostId: 0, channelId: 0, targetId: 0). The last byte is the target, the second byte the host; what the leading 77 stands for the notes do not say |
-g 0 | Lun Information | per LUN: its WWN, CurrentOwningPath, BootOwningPath and PreferredPath (controller B owns LUN 1, controller A owns LUN 2), DevState: OPTIMAL, and under each controller the two LUN path objects with their states |
-S | H2C0T0 Active Active DS5300-SITEA | the array's address on the virtual adapter, the two controller states, the name: the same first line as in the array's .wwn file |
-S | H0C0T0L001 Up | one physical path to one LUN and its state: host 0, channel 0, target 0, LUN 1. The left column lists the paths to one controller, the right column those to the other |
The second -S report shows the standby array with the crossed target numbers that its .wwn file also has (H1C0T2 with H0C0T3, H1C0T3 with H0C0T2); see /var/mpp files. The LUN ownership in -g 0 (LUN 1 on controller B, LUN 2 on controller A) is, as I understand it, the array's own preferred-controller setting, which the driver follows; with LBPolicy: LeastQueueDepth the two paths to the owning controller share the I/O. Both readings are mine from the names; the notes do not describe the policy.
Checked against NetApp SANtricity 12.10 and multipath-tools 0.15.1
| As built | Today |
|---|---|
mppUtil as the driver's control and reporting tool | RDAC and its tools were discontinued with SANtricity OS 11.25; the readme of this build lists man pages for RDAC, mppUtil, mppBusRescan and mppUpdate, which the notes did not keep. In the replacement layer, DM-Multipath, the path state comes from multipath -ll, which the Red Hat and SUSE storage guides use |
AVTEnabled: Y in mppUtil -g 0, against the quoted requirement to disable AVT | The readme of the previous build (09.03.0B05.0233) explains the requirement: AVT is "automatically enabled in the Linux storage partitioning host type" and "causes contention when an RDAC is installed", to be disabled with a script or by using the LNXCL host type; IBM's Storage Manager readme says LNXCL "must be used". The notes do not show the array side, so whether this host's arrays really had AVT on cannot be settled here. In 2015 the Linux host type with AVT on was renamed LNXAVT and a new Linux type with AVT off replaced it; today's type is "Linux DM-MP (Kernel 3.10 or later)" |
LBPolicy: LeastQueueDepth, failover by the driver | multipath-tools' built-in table for vendor IBM, product ^1818: group_by_prio, prio rdac, path_checker rdac, features "2 pg_init_retries 50", failback immediate, no_path_retry 30; the kernel attaches the handler scsi_dh_rdac by itself. NetApp: leave /etc/multipath.conf blank |
controller firmware 7.60.28.xx on a DS5300 | DS5300 end of support 31 January 2018 |