Linux Storage 04 - FC aliases, zones and zonesets

Linux Storage Solution · Previous: Cisco MDS base configuration · Next: Port security

Zoning is the access control of a Fibre Channel fabric: it decides which ports may talk to which. This Article is the zoning of FCSwitch1 as it was built: first the two databases that show who is logged in, then one alias per port, one zone per server, one zoneset per VSAN, and the activation that replaced the interface-based zoning that was there before. The command set with all outputs is a Config document, MDS FC alias and zoning commands; the two login databases are MDS flogi and fcns databases; and the running configuration of the zoning after the activation is one document per VSAN, VSAN 11 and VSAN 12.

Who is logged in

Before anything is zoned, the switch is asked who is there. Two commands at the exec prompt of FCSwitch1, as admin:

$ sh flogi database
$ sh fcns database

The fabric login (flogi) database has one row per interface that completed a fabric login: the VSAN, the FCID the switch handed out, and the port and node WWN the device presented. The name server (fcns) database lists the same ports per VSAN with the vendor the switch reads from the WWN and the FC-4 feature, which is what tells an initiator (a host HBA, scsi-fcp:init) from a target (an array port, scsi-fcp:target). The two joined, with the host names from the schema and from the aliases below:

Seven logins, four in VSAN 11 and three in VSAN 12, exactly the seven interfaces assigned in the base configuration. The HP the name server prints against the 50:01:43:80 addresses is the OUI of the HBAs; the 3PAR addresses get no vendor name. The two array ports share one node WWN, 2f:f7:00:02:ac:00:ab:cd, as two ports of one array should; each HBA port has a node WWN of its own, one above its port WWN. The row for bay1 is the one that can be checked from the host: on HV01, host1 reports port_name 0x50014380123456c0 and port_id 0x2a0000, the same values with the colons taken out, as SCSI addressing and FC names shows.

This listing is the source of every WWN typed in the rest of this Article and in Port security. Nothing was copied from a label or a vendor sheet; what the switch saw log in is what was zoned.

Aliases

An FC alias gives a port WWN a name, so that the zones below can be read without a table. On NX-OS an alias belongs to one VSAN, which is why each was created with vsan 11 or vsan 12. In configuration mode on FCSwitch1, the first alias as typed; fcalias name opens a sub-mode for the member pwwn line:

$ fcalias name HV01 vsan 11
$ member pwwn 50:01:43:80:12:34:56:c0

The server aliases carry the host name, the storage aliases the generic controller-and-port name of the schema (C0P0, C0P1) rather than HP's node:slot:port. The 3PAR port numbers in both tables are the cabling schema's; the port names themselves decode to node 1, see SCSI addressing and FC names. The author's comment above the fourth alias says "the FC alias for server HV03, which sits in BAY4" while the command creates HV04; the command is right, the comment was copied from the line above and not corrected.

Single-initiator zones

A zone is a set of members that may talk to each other; everything outside the zone is invisible to them. The zones here each hold exactly two aliases, one server and the storage port of its VSAN. This is single-initiator zoning: one initiator per zone, never two. The reason, as I understand it, is that two HBAs in one zone can see each other, and an HBA that sees another HBA logs in to it, probes it, and reacts to its state changes, which is noise at best and a hang at worst; the storage port is the only thing a server HBA needs to reach. One zone per server also means that a server can be added or removed by touching one zone. In configuration mode on FCSwitch1, the DC zone for HV01:

$ zone name HV01-Storage1 vsan 11
$ member fcalias Storage1-C0P0
$ member fcalias HV01
$ exit

The members are aliases, so the zones read as what they mean, and the switch resolves them to WWNs when the zoneset is activated. The notes do the DMZ zones first and the DC zones second; the order does not matter, because nothing is in force until a zoneset is activated.

Zonesets and activation

A zoneset collects the zones of one VSAN, and exactly one zoneset per VSAN can be active. Two were created, PROD-DMZ for VSAN 12 with the two DMZ zones and PROD-DC for VSAN 11 with the three DC zones. The whole chain for VSAN 11:

flowchart LR
  subgraph al["fcalias in VSAN 11"]
    a1["HV01, 50:01:43:80:12:34:56:c0"]
    a2["HV02, 50:01:43:80:12:34:56:d4"]
    a3["MGMT01, 50:01:43:80:12:34:40:2c"]
    a4["Storage1-C0P0, 21:12:00:02:ac:00:ab:cd"]
  end
  subgraph zn["zone"]
    z1["HV01-Storage1"]
    z2["HV02-Storage1"]
    z3["MGMT01-Storage1"]
  end
  zs["zoneset PROD-DC vsan 11"]
  act["active zoneset of VSAN 11"]
  a1 --> z1
  a4 --> z1
  a2 --> z2
  a4 --> z2
  a3 --> z3
  a4 --> z3
  z1 --> zs
  z2 --> zs
  z3 --> zs
  zs -- "zoneset activate name PROD-DC vsan 11" --> act

Before each activation the notes look at what is active. For VSAN 12 it was a zoneset named ZONESET_V12 with one zone, Z_FC1_b3_FC1_e2_V12, whose members were not WWNs but switch interfaces: interface bay3, interface bay4 and interface ext2, each with the switch's own WWN 20:00:54:7f:ee:ab:cd:18. For VSAN 11 it was ZONESET_V11 with three such zones, Z_FC1_b1_FC1_e1_V11, Z_FC1_b2_FC1_e1_V11 and Z_FC1_b5_FC1_e1_V11, each pairing one bay with ext1. The notes do not say where these came from; the names read to me as "FC switch 1, bay N, to FC switch 1, ext N" and look like a first zoning done by port before the WWNs were known. The activation, in configuration mode on FCSwitch1:

$ zoneset activate name PROD-DC vsan 11

WARNING: You are trying to activate zoneset PROD-DC, which is different from current active zoneset ZONESET_V11. Do you want to continue? (y/n) [n] y
Zoneset activation initiated. check zone status

The warning is the switch pointing out that this is a replacement, not a change to the active set, and that every device not in the new set loses its paths the moment it is answered. The answer was y both times, and the show zoneset active afterwards showed PROD-DC and PROD-DMZ with every member marked *, logged in, and resolved to pwwn. The author marked both results as OK.

flowchart TB
  subgraph before["Before: ZONESET_V11 active, members by interface"]
    o1["Z_FC1_b1_FC1_e1_V11, bay1 and ext1"]
    o2["Z_FC1_b2_FC1_e1_V11, bay2 and ext1"]
    o3["Z_FC1_b5_FC1_e1_V11, bay5 and ext1"]
  end
  subgraph after["After: PROD-DC active, members by pwwn"]
    n1["HV01-Storage1, HV01 and Storage1-C0P0"]
    n2["HV02-Storage1, HV02 and Storage1-C0P0"]
    n3["MGMT01-Storage1, MGMT01 and Storage1-C0P0"]
  end
  before -- "zoneset activate name PROD-DC vsan 11" --> after

In VSAN 11 the old and the new zoning pair the same things: bay1 with ext1 is HV01 with Storage1-C0P0. What changed is what a member is. An interface member says "whatever logs in on this port"; a pwwn member says "this device, on whatever port". With port members a blade moved to another bay would lose its storage and a stranger put into the bay would gain it; with WWN members the zoning follows the HBA. The price is that a replaced HBA has a new WWN and needs the alias updated, which is what the aliases are for.

What the running configuration shows

After each activation the notes run sh run zone vsan N. The listing has two sections. The active zone database holds the zoneset in force, with its zones written as member pwwn lines because the aliases were resolved at activation. The full zone database holds everything defined in the VSAN: the aliases, the new zones with member fcalias, and also the old interface-based zones and the old zonesets ZONESET_V11 and ZONESET_V12, which nobody deleted. Between the two sections the switch prints zoneset activate name PROD-DC vsan 11 and then do clear zone database vsan 11. That line is in the output; the notes do not explain it, and as I understand it it is how NX-OS marks the boundary so that a replay rebuilds the full database from scratch, but I have not verified that. The listings are the two running-configuration documents linked above.

One thing in them is worth knowing. In VSAN 12 the active copy of the old zone Z_FC1_b3_FC1_e2_V12, shown before the activation, had three members, bay3, bay4 and ext2, so both DMZ servers and the storage sat in one zone; the full-database copy of the same zone in the running configuration has two, bay3 and ext2. The active and the full database are separate copies, so the full one was presumably edited after that zoneset had been activated and never re-activated. That is my inference; the notes say nothing about it. In VSAN 11 the two copies agree.

What is missing

The notes do not hold the zoning of SW2, and so not the zones that give each blade its second path. They do not delete the old zonesets, do not run copy running-config startup-config, and do not show the array side: which 3PAR virtual volumes are exported to which host and how the hosts are defined there is not in them. That HV01 saw six LUNs through each HBA after this is in LUNs seen and SCSI bus rescan.

Today

Checked against the MDS NX-OS 9.4(5a) guides: every command of this Article, the zoneset activate warning and the shape of show running-config zone with its do clear zone database line are unchanged, and single-initiator zoning is still what Cisco calls "the most efficient approach to zoning". The advice around the commands moved. On names, Cisco's guide says "Device aliases should be used to simplify the management of world wide names (WWNs) whenever possible" and "Operate device aliases in Enhanced mode whenever possible", which I read as device aliases instead of the FC aliases used here; smart zoning, off by default, "eliminates the need to create a single initiator to single target zones" (how it does that I take from my general understanding of the feature, not from the guide's sentence); and zoneset overwrite-control vsan N can refuse exactly the kind of replacement done here, a zoneset of another name activated over the current one, unless force is given. The defaults are what they were: basic zoning, smart zoning off, default zone denied, full zone set not distributed. Interface members are still valid, and I found no sentence in the guides that prefers pwwn members over them.

What I would do differently

Delete ZONESET_V11, ZONESET_V12 and the Z_FC1_… zones from the full zone database once the new zonesets were active, so that the running configuration shows one zoning per VSAN and the next person does not have to work out which one counts. Save the configuration, and write down that it was saved. And on a current switch, use device aliases in enhanced mode instead of FC aliases, as I read Cisco's current advice.