LINUXOR.SK ... open source notes ...

Linux Storage 07 - LUNs seen and SCSI bus rescan

category: solutionz · date: 2013-12-31 · updated: 2026-10-03 · author: LALA

Linux Storage Solution · Previous: Linux HBA and SCSI inventory · Next: RDAC multipath driver basics

With the HBA ports online and the array port visible as a remote port behind each of them, the last question on the host was the one the whole SAN was built for: which LUNs does HV01 see, over which path, and how does the kernel get told to look again when the array exports a new one. This part is the SCSI device list of HV01 as the kernel printed it in /proc/scsi/scsi, what it says and what it does not, and the three ways the notes rescan the bus. It is also where the 2013 notes end: everything above the SCSI device, the multipath layer and the XenServer storage repository, was not written down. Everything here ran as root on HV01 and, apart from the rescans, only reads.

What the host sees

/proc/scsi/scsi is the kernel's flat list of attached SCSI devices, one block of three lines per device: the address Host: Channel: Id: Lun:, the vendor, model and revision strings from the device's INQUIRY data, and its type. On HV01 it has sixteen entries. The whole listing is a Config document, /proc/scsi/scsi; the table folds it.

HostChannelIdLUNVendor and modelTypeCount
scsi0030000HP P220i, Rev 4.68RAID1
scsi0000000HP LOGICAL VOLUME, Rev 4.68Direct-Access1
scsi1000000 to 053PARdata VV, Rev 3122Direct-Access6
scsi100002543PARdata SES, Rev 3122Enclosure1
scsi2000000 to 053PARdata VV, Rev 3122Direct-Access6
scsi200002543PARdata SES, Rev 3122Enclosure1

scsi0 is the HP P220i Smart Array with the blade's local disks: the hpsa driver shows the controller itself on channel 3 as a RAID device and the one logical volume on channel 0 as a disk. scsi1 and scsi2 are the two HBA ports from the previous part. Each sees one target, Id 00, which is the 3PAR port it is zoned to, and behind that target six virtual volumes (VV) at LUN 0 to 5 and one enclosure services device (SES) at LUN 254. Two entries of the listing, LUN 0 and LUN 254 of scsi1, as an excerpt (the entries for LUN 1 to 5 lie between them in the notes):

output 6 lines
Host: scsi1 Channel: 00 Id: 00 Lun: 00
  Vendor: 3PARdata Model: VV               Rev: 3122
  Type:   Direct-Access                    ANSI  SCSI revision: 06
Host: scsi1 Channel: 00 Id: 00 Lun: 254
  Vendor: 3PARdata Model: SES              Rev: 3122
  Type:   Enclosure                        ANSI  SCSI revision: 06

The kernel prints scsi2 before scsi1. That is the order in which the devices were registered, not a statement about the paths, and I would not read anything into it. The ANSI SCSI revision is 05 for the HP devices and 06 for the 3PAR ones; as I understand the INQUIRY version field, that is SPC-3 against SPC-4. The notes do not say which 3PAR OS version Rev: 3122 stands for.

The same six volumes, twice

The two lists of six are presumably not twelve disks. host1 reaches the array through SW1 and, by the cabling schema, 3PAR port 0:1:1, host2 through SW2 and, by the same schema, port 1:1:1 (the port names themselves decode the other way round, see SCSI addressing and FC names), and the array presumably exports the same six volumes on both ports, so that every volume arrives twice, once per fabric; the notes cannot prove it, as the next paragraph says. That is what two fabrics are for: a volume stays reachable when one switch, one HBA port or one array port is gone.

mermaid
flowchart LR
  hv["HV01"]
  h1["host1, WWPN 50:01:43:80:12:34:56:c0"]
  h2["host2, WWPN 50:01:43:80:12:34:56:c2"]
  r1["rport-1:0-0, 3PAR 21:12:00:02:ac:00:ab:cd via SW1"]
  r2["rport-2:0-0, 3PAR 20:11:00:02:ac:00:ab:cd via SW2"]
  subgraph p1["LUNs through host1"]
    a0["1:0:0:0 VV"]
    a1["1:0:0:1 VV"]
    a2["1:0:0:2 VV"]
    a3["1:0:0:3 VV"]
    a4["1:0:0:4 VV"]
    a5["1:0:0:5 VV"]
    a254["1:0:0:254 SES"]
  end
  subgraph p2["LUNs through host2"]
    b0["2:0:0:0 VV"]
    b1["2:0:0:1 VV"]
    b2["2:0:0:2 VV"]
    b3["2:0:0:3 VV"]
    b4["2:0:0:4 VV"]
    b5["2:0:0:5 VV"]
    b254["2:0:0:254 SES"]
  end
  hv --> h1 --> r1
  hv --> h2 --> r2
  r1 --> a0
  r1 --> a1
  r1 --> a2
  r1 --> a3
  r1 --> a4
  r1 --> a5
  r1 --> a254
  r2 --> b0
  r2 --> b1
  r2 --> b2
  r2 --> b3
  r2 --> b4
  r2 --> b5
  r2 --> b254

The kernel does not know whether 1:0:0:3 and 2:0:0:3 are the same volume, and neither do the notes; it makes two block devices, and the layer that folds them into one is multipathing, which on this host would have been XenServer's. The notes stop before it. They do not hold the XenServer multipath configuration or any multipath.conf, they do not say which 3PAR virtual volume carries which LUN number, they give no sizes, and they show no /dev/sd* names or a scsi_id run that would pair the two sides. There was no failover test either. For a reader who wants to see what a multipath driver does with two such lists, the 2010 part of this Solution shows one, the RDAC driver with its four paths per LUN (RDAC multipath driver basics), on a different system.

Two details of the list are worth knowing. The SES device at LUN 254 is not a disk: HPE documents it as the enclosure services device that the host persona enables (SESLun, part of persona 1 "Generic" and persona 2 "Generic-ALUA"), used by HPE's Host Explorer to report the host to the array, and the HP 3PAR Citrix guide of 2014 warns that a volume exported at LUN 254 needs a host reboot before XenServer can use it. Nothing in the notes uses it. And the single Id: 00 on both FC hosts follows from tgtid_bind_type wwpn of the previous part: one remote port, one target, and it keeps its number as long as the array port keeps its WWPN.

Three ways to rescan

A LUN exported after the host booted does not appear by itself; the kernel has to be told to scan the bus again. The notes keep three ways to do it, and they are the same three I still reach for.

WayCommandScope
the scriptscsi-rescanevery SCSI host in the system, known devices and free addresses
by handwrite - - - into /sys/class/scsi_host/<hostN>/scanone SCSI host
the loopthe same write for every name in /sys/class/fc_hostevery FC host, nothing else

The script is rescan-scsi-bus.sh, which this system installed under the name scsi-rescan. The notes point to its home, http://www.garloff.de/kurt/linux/#rescan-scsi, under a heading about "the tools and commands of the various vendors", and give no version. The manual way is the kernel interface the script itself uses:

bash
$ echo "- - -" > /sys/class/scsi_host/<hostN>/scan;

The notes had a reminder in place of the host number, "fill in, usually the last two", which here is <hostN>; on HV01 the last two are host1 and host2, so the line is run twice. The three dashes are, as I understand the kernel's scan attribute, the channel, the target and the LUN to scan, and a dash means all of them. The trailing semicolon is in the notes and does nothing. The loop makes the "usually the last two" exact by taking the host names from the FC host class, so it never touches the local controller:

bash
$ for host in `ls /sys/class/fc_host`; do echo "- - -" > /sys/class/scsi_host/${host}/scan; done
mermaid
flowchart TB
  s["start"]
  ls["list /sys/class/fc_host"]
  n["next name, host1 then host2"]
  w["write - - - into scsi_host/name/scan"]
  q{"names left?"}
  e["done"]
  s --> ls --> n --> w --> q
  q -- "yes" --> n
  q -- "no" --> e

The three forms with the full scsi-rescan output are a Config document, SCSI bus rescan.

What scsi-rescan printed

The script was run on the bus as it stood, with every LUN already present, so its output is a second inventory rather than a discovery. It found the three host adapters with their drivers, hpsa for host0 and qla2xxx for host1 and host2, scanned host0 for target IDs 0 to 7 and the FC hosts for all target IDs, and printed every device it already knew with the prefix OLD: and the same three lines /proc/scsi/scsi shows. The first nine lines of the output:

output 9 lines
Host adapter 0 (hpsa) found.
Host adapter 1 (qla2xxx) found.
Host adapter 2 (qla2xxx) found.
Scanning SCSI subsystem for new devices
Scanning host 0 for  SCSI target IDs  0 1 2 3 4 5 6 7, all LUNs
Scanning for device 0 0 0 0 ...
OLD: Host: scsi0 Channel: 00 Id: 00 Lun: 00
      Vendor: HP       Model: LOGICAL VOLUME   Rev: 4.68
      Type:   Direct-Access                    ANSI SCSI revision: 05

The rest of the output is in the Config document; its last two lines are the result: 0 new device(s) found. and 0 device(s) removed. A new LUN would have printed NEW: instead of OLD:, as I understand the script; the notes have no such run. One thing in the output I report rather than explain: the LUNs of host1 and of host2 are scanned in the order 0, 1, 2, 254, 3, 4, 5, while /proc/scsi/scsi lists them 0 to 5 and then 254. The notes do not remark on it. My guess is that the script lists the existing devices and sorts the names as strings, so 254 sorts between 2 and 3; I did not confirm it in the script's source.

What is not in the notes

The host side of the 2013 notes ends here, with the LUNs visible and the bus rescannable. Missing, and to be looked for elsewhere: the XenServer multipath configuration and storage repository, the pairing of 3PAR virtual volumes to LUN numbers and to /dev/sd* devices, the sizes, the 3PAR host definition and exports, and any test of what happens when one of the two paths is cut. Anyone building this today would also check the per-LUN queue depth and the path timeouts of the previous part against the array vendor's host guide before putting load on it; the notes did not.

What I would do differently

← solutionz