2010 - Základy IBM/LSI MPIO ovládača RDAC (Redundant Disk Array Controller) pre Linux
Anglická verzia tohto dokumentu: 2010 - The basics of the IBM/LSI MPIO driver RDAC (Redundant Disk Array Controller) for Linux.
|=-------------------------------------------------=[ FC SAN ]=-----------------------------------------------=| +--------------------------------------------------------------------------------------------------------------+ | | | IBM / LSI | | | | Redundant Disk Array Controller | | MPP (Multi-Path Proxy Driver) | | | | | +--------------------------------------------------------------------------------------------------------------+ |=-------------------------------------------------=[ LH 2010 ]=----------------------------------------------=|
POZNAMKY k RDAC (Redundant Disk Array Controller) a.k.a. Multi-Path Proxy Driver alebo MPP
- Zakladne INFO
- Instalacia
- Pouzitie (priklady)
- Pouzitie (priklady s vystupmi)
- Problemy
- LINKzZakladne INFO
Ovladac RDAC (Redundant Disk Array Controller a.k.a. Multi-Path Proxy alebo MPP) pridava novu vrstvu do
ovladacoveho zasobnika (stacku) OS Linux. Tento ovladac je potrebny na zabezpecenie viacerych ciest (multi-path)
smerom k diskovemu polu. Je to kombinacia dvoch ovladacov:
[1] vysoko urovnovy ovladac
[2] virtualny HBA ovladac
--------------------------------------------------------------------------------------------------------
[1] Vysoko urovnovy ovladac ma rovnaku rolu ako "sd" (SCSI Disk Driver) alebo "sg" (SCSI Generic Driver)
ovladac. Sluzi pre potreby diskovych poli a poskytuje ioctl rozhranie pre uzivatelsky priestor. Je
zodpovedny za manazment viacerych ciest k diskovemu polu.
[2] Virtualny HBA ovladac je Linuxovy nizkourovnovy ovladac. Je umiestneny na rovnakej urovni ovladacoveho
zasobniku (stacku) ako kazdy iny ovladac pre QLogic alebo LSI fibre channel HBA.
When the host has more than one HBA and all those HBAs have access to one or more IBM TotalStorage DS4xxx,
a multipath driver is required. For the DS4xxx family this is the redundant disk array controller/multipath
proxy (RDAC/MPP) driver. On AIX, the base OS already contains the RDAC driver, also known as MPIO. For Linux,
you must download and install the RDAC/MPP package manually.
The RDAC multipath driver consists of two modules:
* The mppUpper: module is loaded before any HBA and prevents the HBA to publicize LUNs to the system.
* The mppVhba: module is a virtual HBA that sits on top of the real HBA. The virtual HBA is responsible
for bundling the different paths to the same LUN, organizing the traffic from and to the LUN, and
presenting it to the OS as single entity.
Note that if RDAC/MPP is used, a non-failover HBA device driver is required, and auto volume transfer (AVT)
must be disabled at the DS4xxx level. You achieve this by changing the host type from Linux to LNXCL. Check
the latest version of the Readme.txt delivered with the RDAC/MPP driver and follow the instructions.
Dolezite prikazy
--------------------------------------------------------------------------------------------------------
# mppUtil - prikaz na vykonavanie roznych funkcii RDAC ovladaca
# mppUpdate - prikaz aktualizuje zaznamy v konfiguracnom subore /var/mpp/devicemapping
# mppBusRescan - prikaz na vyhladanie noveho disk. pola, odstranenie alebo odpojenie zariadeni a aktualizaciu velkosti zariadeni
# hot_add - linka na mppBusRescan (/usr/sbin/hot_add -> /usr/sbin/mppBusRescan)
# /opt/mpp/lsvdev - zobrazi zakladne mapovanie LUNov
# /etc/init.d/mpp - dynamicke zistenie zariadeni pocas bootovania (ak mame viac poli je vhodne to vypnut aby po restarte nevznikali nezelane problemy)
# /opt/mpp/mppSupport - prikaz vytvori log subor s RDAC informaciami -> 'mppSupportdata_hostname_RDACversion_datetime.tar.gz' v adresari /tmp
Dolezite subory
--------------------------------------------------------------------------------------------------------
[1] /etc/mpp.conf - konfiguracny subor MPP ovladaca
[2] /var/mpp/* - konfiguracne subory MPP ovladaca (stale zmeny disk. poli, LUNov)
[3] /proc/mpp/* - behove (runtime) informacie MPP ovladaca
[1] Konfiguracny subor MPP ovladaca (man RDAC -> popis k jednotlivym konfiguracnym direktivam)
--------------------------------------------------------------------------------------------------------
# cat /etc/mpp.conf
--------------------------------------------------------------------------------------------------------
virtualDiskProductId=VirtualDisk
DebugLevel=0x0
NotReadyWaitTime=270
BusyWaitTime=270
QuiescenceWaitTime=270
InquiryWaitTime=60
MaxLunsPerArray=256
MaxPathsPerController=4
ScanInterval=60
InquiryInterval=1
MaxArrayModules=30
ErrorLevel=3
SelectionTimeoutRetryCount=0
UaRetryCount=10
RetryCount=10
SynchTimeout=170
FailOverQuiescenceTime=20
FailoverTimeout=120
FailBackToCurrentAllowed=1
ControllerIoWaitTime=300
ArrayIoWaitTime=600
DisableLUNRebalance=0
SelectiveTransferMaxTransferAttempts=5
SelectiveTransferMinIOWaitTime=3
IdlePathCheckingInterval=60
RecheckFailedPathWaitTime=30
FailedPathCheckingInterval=60
ArrayFailoverWaitTime=300
PrintSenseBuffer=0
ClassicModeFailover=0
AVTModeFailover=0
LunFailoverDelay=3
LoadBalancePolicy=1
ImmediateVirtLunCreate=0
BusResetTimeout=150
AllowHBAsgDevs=0
S2ToS3Key=1b1fa2921abbbc44
--------------------------------------------------------------------------------------------------------
[2] Konfiguracne subory MPP ovladaca (stala konfiguracia disk. poli, LUNov)
--------------------------------------------------------------------------------------------------------
# ls -ltr /var/mpp
--------------------------------------------------------------------------------------------------------
-rw------- 1 root root 361 Sep 7 11:57 600a0b800047dfb8000000004b591ee7.wwn
-rw------- 1 root root 361 Sep 7 11:57 600a0b800047dd34000000004b612dbb.wwn
-rw------- 1 root root 15 Sep 7 12:17 devicemapping
--------------------------------------------------------------------------------------------------------
[2] # cat ./600a0b800047dfb8000000004b591ee7.wwn
--------------------------------------------------------------------------------------------------------
H2C0T0 :Active :Active :DS5300-POLUS
maxLun256:CAP 0 :CAP 1 :CAP 2 :CAP 3 :CBP 0 :CBP 1 :CBP 2 :CBP 3 :
L001 :H0C0T0:01H1C0T0:01---------:---------:H0C0T1:01H1C0T1:01---------:---------:
L002 :H0C0T0:02H1C0T0:02---------:---------:H0C0T1:02H1C0T1:02---------:---------:
--------------------------------------------------------------------------------------------------------
[2] # cat ./600a0b800047dd34000000004b612dbb.wwn
--------------------------------------------------------------------------------------------------------
H2C0T1 :Active :Active :DS5300-SITEL
maxLun256:CAP 0 :CAP 1 :CAP 2 :CAP 3 :CBP 0 :CBP 1 :CBP 2 :CBP 3 :
L001 :H1C0T2:01H0C0T3:01---------:---------:H1C0T3:01H0C0T2:01---------:---------:
L002 :H1C0T2:02H0C0T3:02---------:---------:H1C0T3:02H0C0T2:02---------:---------:
--------------------------------------------------------------------------------------------------------
[2] # cat /var/mpp/devicemapping
--------------------------------------------------------------------------------------------------------
0:DS5300-POLUS
1:DS5300-SITEL
[3] # ls -lR /proc/mpp
--------------------------------------------------------------------------------------------------------
...
[3] Vsetky runtime informacie RDAC ovladaca sa nachadzaju v suborovom systeme '/proc'
--------------------------------------------------------------------------------------------------------
/proc/mpp/*
[3] Runtime informacie virtualneho MPP adaptera
Cislo adaptera moze byt ine na roznych systemoch (cislo je pridelovane SCSI medzivrstvou [middle layer])
--------------------------------------------------------------------------------------------------------
/proc/scsi/mpp/<cislo_adaptera>
[3] Zaznamy pre vsetky diskove polia, ktore su viditelne pre server
--------------------------------------------------------------------------------------------------------
/proc/mpp/<nazov_disk_pola>
[3] Zaznamy pre vsetky diskove radice daneho diskoveho pola
--------------------------------------------------------------------------------------------------------
/proc/mpp/<nazov_disk_pola>/controllerA
/proc/mpp/<nazov_disk_pola>/controllerB
[3] Nizkourovnovy ovladac moze byt (qlogic, emulex)
HCT znamena:
Host# - cislo low-level ovladaca, ktory je prideleny medzivrstvou (mid-layer) SCSI
Channel# -
Target# -
--------------------------------------------------------------------------------------------------------
/proc/mpp/<nazov_disk_pola>/controller<A/B>/<low-level-driver><HCT#>Instalacia
Rozbalime potrebny ovladac
--------------------------------------------------------------------------------------------------------
# tar -xvzf ./rdac-LINUX-09.03.0B05.0331-source.tar.gz
Prekompilujeme ovladac
--------------------------------------------------------------------------------------------------------
# cd linuxrdac-09.03.0B05.0331/
# make clean
# make
# make install
Upravime GRUB a pridame zaznam pre "nove" jadro + initrd
--------------------------------------------------------------------------------------------------------
# chattr -i /boot/grub/menu.lst
# nano /boot/grub/menu.lst
--------------------------------------------------------------------------------------------------------
title Red Hat Enterprise Linux AS (2.6.9-89.0.9.ELsmp) with MPP support
root (hd0,0)
kernel /vmlinuz-2.6.9-89.0.9.ELsmp ro root=/dev/VG_ROOT/LV_ROOT rhgb quiet
initrd /mpp-2.6.9-89.0.9.ELsmp.img
--------------------------------------------------------------------------------------------------------
# chattr +i /boot/grub/menu.lst
Rebootneme server
--------------------------------------------------------------------------------------------------------
# reboot
Skontrolujeme ci je nacitany modul jadra MPP
--------------------------------------------------------------------------------------------------------
# lsmod | grep mpp
--------------------------------------------------------------------------------------------------------
mppVhba 138656 0
mppUpper 139132 1 mppVhba
scsi_mod 145297 8 mppVhba,usb_storage,cciss,qla2xxx,scsi_transport_fc,mppUpper,sg,sd_mod
Skontrolujeme ci je namapovany LUN
--------------------------------------------------------------------------------------------------------
# ls -lR /proc/mpp
--------------------------------------------------------------------------------------------------------
/proc/mpp:
total 0
dr-xr-xr-x 4 root root 0 Aug 13 15:56 DS5300-POLUS
/proc/mpp/DS5300-POLUS:
total 0
dr-xr-xr-x 4 root root 0 Aug 13 15:56 controllerA
dr-xr-xr-x 4 root root 0 Aug 13 15:56 controllerB
-rw-r--r-- 1 root root 0 Aug 13 15:56 virtualLun1
-rw-r--r-- 1 root root 0 Aug 13 15:56 virtualLun2
-rw-r--r-- 1 root root 0 Aug 13 15:56 virtualLun3
/proc/mpp/DS5300-POLUS/controllerA:
total 0
dr-xr-xr-x 2 root root 0 Aug 13 15:56 qla2xxx_h0c0t0
dr-xr-xr-x 2 root root 0 Aug 13 15:56 qla2xxx_h1c0t0
/proc/mpp/DS5300-POLUS/controllerA/qla2xxx_h0c0t0:
total 0
-rw-r--r-- 1 root root 0 Aug 13 15:56 LUN1
-rw-r--r-- 1 root root 0 Aug 13 15:56 LUN2
-rw-r--r-- 1 root root 0 Aug 13 15:56 LUN3
/proc/mpp/DS5300-POLUS/controllerA/qla2xxx_h1c0t0:
total 0
-rw-r--r-- 1 root root 0 Aug 13 15:56 LUN1
-rw-r--r-- 1 root root 0 Aug 13 15:56 LUN2
-rw-r--r-- 1 root root 0 Aug 13 15:56 LUN3
/proc/mpp/DS5300-POLUS/controllerB:
total 0
dr-xr-xr-x 2 root root 0 Aug 13 15:56 qla2xxx_h0c0t1
dr-xr-xr-x 2 root root 0 Aug 13 15:56 qla2xxx_h1c0t1
/proc/mpp/DS5300-POLUS/controllerB/qla2xxx_h0c0t1:
total 0
-rw-r--r-- 1 root root 0 Aug 13 15:56 LUN1
-rw-r--r-- 1 root root 0 Aug 13 15:56 LUN2
-rw-r--r-- 1 root root 0 Aug 13 15:56 LUN3
/proc/mpp/DS5300-POLUS/controllerB/qla2xxx_h1c0t1:
total 0
-rw-r--r-- 1 root root 0 Aug 13 15:56 LUN1
-rw-r--r-- 1 root root 0 Aug 13 15:56 LUN2
-rw-r--r-- 1 root root 0 Aug 13 15:56 LUN3
FINITO
--------------------------------------------------------------------------------------------------------
;-)Pouzitie (priklady)
mppUtil
================================================================================================================
Zobrazenie vsetkych debug sprav pre synchronne I/O aktivity a loguj do konzoly
--------------------------------------------------------------------------------------------------------
# mppUtil -d 0x80200008
Vypnutie vsetkych debug sprav
--------------------------------------------------------------------------------------------------------
# mppUtil -d 0x0
Zobrazenie failoveru kontrolera a fatalne chyby
--------------------------------------------------------------------------------------------------------
# mppUtil -e 4
Zobrazenie internych informacii RDAC ovladaca pre target s ID 1
--------------------------------------------------------------------------------------------------------
# mppUtil -g 1
Najdenie chybneho kontrolera, ktory je znova online
--------------------------------------------------------------------------------------------------------
# mppUtil -s failback
Vypis vsetkych disk. poli, ktore su detekovane tymto serverom
--------------------------------------------------------------------------------------------------------
# mppUtil -a
Zobrazenie internych informacii RDAC ovladaca pre disk. pole s nazvom 'pole'
--------------------------------------------------------------------------------------------------------
# mppUtil -a "array_name"
Zobrazenie vsetkych dostupnych specialnych volieb (feature_options)
--------------------------------------------------------------------------------------------------------
# mppUtil -o
Nastavenie specialnej volby 'ErrorLevel' na hodnotu '0x2'
--------------------------------------------------------------------------------------------------------
# mppUtil -o "ErrorLevel=0x2"
Zobrazenie behovych (run-time) informacii kontrolera a ciest k serveru
--------------------------------------------------------------------------------------------------------
# mppUtil -S
mppUpdate
================================================================================================================
Odstranenie disk. pola s nazvom 'DS5300-SITEL' zo suboru '/var/mpp/devicemapping'
--------------------------------------------------------------------------------------------------------
# mppUpdate -d DS5300-SITEL
Odstranenie (clear) vsetkych zaznamov disk. poli zo suboru '/var/mpp/devicemapping'
--------------------------------------------------------------------------------------------------------
# mppUpdate -c
mppBusRescan (hot_add)
================================================================================================================
Najdenie (scan) noveho diskoveho pola na fyzickej zbernici (SAN)
--------------------------------------------------------------------------------------------------------
# mppBusRescan
# mppBusRescan -s
# mppBusRescan --scan
alebo
# hot_add
# hot_add -s
# hot_add --scan
Odstranenie (delete) odmapovaneho (nenamapovaneho) alebo odpojeneho zariadenia
--------------------------------------------------------------------------------------------------------
# hot_add -d
# hot_add --delete
alebo
# mppBusRescan -d
# mppBusRescan --delete
Kontrola ci sa zmenilo zariadenie/LUN. Ak nastala zmena aktualizuje sa RDAC s novym remapovanym
zariadenim alebo LUNom
--------------------------------------------------------------------------------------------------------
# mppBusRescan -u
# mppBusRescan --update
Kontrola ci sa zmenila kapacita (capacity) zariadenia/LUNu. Ak nastala zmena aktualizuje sa kapacita zariadenia/LUNu
--------------------------------------------------------------------------------------------------------
# mppBusRescan -c
# mppBusRescan --capacity
lsvdev
================================================================================================================
Zobrazime zakladne mapovanie LUNov
--------------------------------------------------------------------------------------------------------
# /opt/mpp/lsvdev
--------------------------------------------------------------------------------------------------------
Array Name Lun sd device
-------------------------------------
DS5300-POLUS 1 -> /dev/sda
DS5300-POLUS 2 -> /dev/sdb
DS5300-SITEL 1 -> /dev/sdc
DS5300-SITEL 2 -> /dev/sddPouzitie (priklady s vystupmi)
mppUtil
================================================================================================================
Zobrazenie vsetkych diskovych poli, kore vidi tento stroj
--------------------------------------------------------------------------------------------------------
# mppUtil -a
--------------------------------------------------------------------------------------------------------
Hostname = simspcea
Domainname = (none)
Time = GMT 09/07/2010 08:54:48
---------------------------------------------------------------
Info of Array Module's seen by this Host.
---------------------------------------------------------------
ID WWN Type Name
---------------------------------------------------------------
0 600a0b800047dfb8000000004b591ee7 FC DS5300-POLUS
1 600a0b800047dd34000000004b612dbb FC DS5300-SITEL
---------------------------------------------------------------
Zobrazenie vsetkych specialnych volieb
--------------------------------------------------------------------------------------------------------
# mppUtil -o
--------------------------------------------------------------------------------------------------------
Current Default Minimum Maximum
Variable Options Value Value Value Value
---------------------------------------- ---------- ---------- ---------- ----------
DebugLevel 0x0 0x0 0x0 0xffffffff
ErrorLevel 0x3 0x3 0x0 0x5
DisableLUNRebalance 0x0 0x0 0x0 0x4
LoadBalancePolicy 0x1 0x1 0x0 0x2
ClassicModeFailover 0x0 0x0 0x0 0x1
BusResetTimeout 0x96 0x0 0x0 0x12c
Action Options
---------------------------------------
No actions defined
--------------------------------------------------------------------------------------------------------
Nastavenie premennej 'ErrorLevel' na hodnotu 0x2 (nestala zmena [iba v pamati], po restarte sa hodnota
nastavi na povodnu hodnotu)
--------------------------------------------------------------------------------------------------------
# mppUtil -o ErrorLevel=0x2
Nastavenie premennej 'ErrorLevel' na hodnotu 0x2 a ulozenie hodnoty (stala zmena [SaveSettings])
--------------------------------------------------------------------------------------------------------
# mppUtil -o ErrorLevel=0x2,SaveSettings
# mppUpdate
Zobrazenie internych informacii RDAC ovladaca pre specifikovane diskoveho pole (specifikacie pomocou ID [mppUtil -a])
--------------------------------------------------------------------------------------------------------
# mppUtil -g 0
--------------------------------------------------------------------------------------------------------
Hostname = simspcea
Domainname = (none)
Time = GMT 09/07/2010 08:58:08
MPP Information:
----------------
ModuleName: DS5300-POLUS SingleController: N
VirtualTargetID: 0x000 ScanTriggered: N
ObjectCount: 0x000 AVTEnabled: Y
WWN: 600a0b800047dfb8000000004b591ee7 RestoreCfg: N
ModuleHandle: none Page2CSubPage: Y
FirmwareVersion: 7.60.28.xx FailoverMethod: C
ScanTaskState: 0x00000000
LBPolicy: LeastQueueDepth
Controller 'A' Status:
-----------------------
ControllerHandle: none ControllerPresent: Y
UTMLunExists: N Failed: N
NumberOfPaths: 2 FailoverInProg: N
ServiceMode: N
Path #1
---------
DirectoryVertex: present Present: Y
PathState: OPTIMAL
PathId: 77000000 (hostId: 0, channelId: 0, targetId: 0)
Path #2
---------
DirectoryVertex: present Present: Y
PathState: OPTIMAL
PathId: 77010000 (hostId: 1, channelId: 0, targetId: 0)
Controller 'B' Status:
-----------------------
ControllerHandle: none ControllerPresent: Y
UTMLunExists: N Failed: N
NumberOfPaths: 2 FailoverInProg: N
ServiceMode: N
Path #1
---------
DirectoryVertex: present Present: Y
PathState: OPTIMAL
PathId: 77000001 (hostId: 0, channelId: 0, targetId: 1)
Path #2
---------
DirectoryVertex: present Present: Y
PathState: OPTIMAL
PathId: 77010001 (hostId: 1, channelId: 0, targetId: 1)
Lun Information
---------------
Lun #1 - WWN: 600a0b800047dfb80000abba4c2c3c1c
----------------
LunObject: present CurrentOwningPath: B
RemoveEligible: N BootOwningPath: B
NotConfigured: N PreferredPath: B
DevState: OPTIMAL ReportedPresent: Y
ReportedMissing: N
NeedsReservationCheck: N
TASBitSet: N
NotReady: N
Busy: N
Quiescent: N
VD_Ownership_Transfer_Attempt_Count: 0
Controller 'A' Path
--------------------
NumLunObjects: 2 RoundRobinIndex: 0
Path #1: LunPathDevice: present
DevState: OPTIMAL
RemoveState: 0x0 StartState: 0x1 PowerState: 0x0
Path #2: LunPathDevice: present
DevState: OPTIMAL
RemoveState: 0x0 StartState: 0x1 PowerState: 0x0
Controller 'B' Path
--------------------
NumLunObjects: 2 RoundRobinIndex: 0
Path #1: LunPathDevice: present
DevState: OPTIMAL
RemoveState: 0x0 StartState: 0x1 PowerState: 0x0
Path #2: LunPathDevice: present
DevState: OPTIMAL
RemoveState: 0x0 StartState: 0x1 PowerState: 0x0
Lun #2 - WWN: 600a0b800047dd6c0000ae344c6b9649
----------------
LunObject: present CurrentOwningPath: A
RemoveEligible: N BootOwningPath: A
NotConfigured: N PreferredPath: A
DevState: OPTIMAL ReportedPresent: Y
ReportedMissing: N
NeedsReservationCheck: N
TASBitSet: N
NotReady: N
Busy: N
Quiescent: N
VD_Ownership_Transfer_Attempt_Count: 0
Controller 'A' Path
--------------------
NumLunObjects: 2 RoundRobinIndex: 0
Path #1: LunPathDevice: present
DevState: OPTIMAL
RemoveState: 0x0 StartState: 0x1 PowerState: 0x0
Path #2: LunPathDevice: present
DevState: OPTIMAL
RemoveState: 0x0 StartState: 0x1 PowerState: 0x0
Controller 'B' Path
--------------------
NumLunObjects: 2 RoundRobinIndex: 0
Path #1: LunPathDevice: present
DevState: OPTIMAL
RemoveState: 0x0 StartState: 0x1 PowerState: 0x0
Path #2: LunPathDevice: present
DevState: OPTIMAL
RemoveState: 0x0 StartState: 0x1 PowerState: 0x0
--------------------------------------------------------------------------------------------------------
Report o stave kontrolerov a ciest pre kazdy LUN v realnom case
--------------------------------------------------------------------------------------------------------
# mppUtil -S
--------------------------------------------------------------------------------------------------------
H2C0T0 Active Active DS5300-POLUS
H0C0T0L001 Up H0C0T1L001 Up
H1C0T0L001 Up H1C0T1L001 Up
H0C0T0L002 Up H0C0T1L002 Up
H1C0T0L002 Up H1C0T1L002 Up
Missing Arrays
There are no missing arrays
--------------------------------------------------------------------------------------------------------
Najdenie noveho diskoveho pola
--------------------------------------------------------------------------------------------------------
# mppBusRescan
--------------------------------------------------------------------------------------------------------
scan qla2 HBA host /sys/class/scsi_host/host1...
found 1:0:2:1
found 1:0:2:2
found 1:0:3:1
found 1:0:3:2
scan qla2 HBA host /sys/class/scsi_host/host0...
found 0:0:2:1
found 0:0:2:2
found 0:0:3:1
found 0:0:3:2
run /usr/sbin/mppUtil -s busscan...
scan mpp virtual host /sys/class/scsi_host/host2...
found 2:0:1:1->/dev/sdc
found 2:0:1:2->/dev/sdd
--------------------------------------------------------------------------------------------------------
Report o stave kontrolerov a ciest pre kazdy LUN v realnom case (po najdeni noveho disk. pola)
--------------------------------------------------------------------------------------------------------
# mppUtil -S
--------------------------------------------------------------------------------------------------------
H2C0T0 Active Active DS5300-POLUS
H0C0T0L001 Up H0C0T1L001 Up
H1C0T0L001 Up H1C0T1L001 Up
H0C0T0L002 Up H0C0T1L002 Up
H1C0T0L002 Up H1C0T1L002 Up
H2C0T1 Active Active DS5300-SITEL
H1C0T2L001 Up H1C0T3L001 Up
H0C0T3L001 Up H0C0T2L001 Up
H1C0T2L002 Up H1C0T3L002 Up
H0C0T3L002 Up H0C0T2L002 Up
Missing Arrays
There are no missing arrays
--------------------------------------------------------------------------------------------------------
mppUpdate
================================================================================================================
Vymazanie disk. pola s nazvom 'DS5300-SITEL' zo suboru /var/mpp/devicemapping
--------------------------------------------------------------------------------------------------------
# mppUpdate -d DS5300-SITEL
--------------------------------------------------------------------------------------------------------
The module DS5300-SITEL was removed from the MPP persistence mapping list (/var/mpp/devicemapping).
Detected 2 QLogic Host Adapter Port(s) on the system
Creating new MPP initrd image...
--------------------------------------------------------------------------------------------------------
mppBusRescan (hot_add)
================================================================================================================
Najdenie (scan) noveho diskoveho pola na fyzickej zbernici (SAN)
--------------------------------------------------------------------------------------------------------
# mppBusRescan -s
--------------------------------------------------------------------------------------------------------
Starting new devices re-scan...
scan mptscsih HBA host /sys/class/scsi_host/host5...
no new device found
scan mptscsih HBA host /sys/class/scsi_host/host4...
no new device found
scan mptscsih HBA host /sys/class/scsi_host/host3...
no new device found
scan mptscsih HBA host /sys/class/scsi_host/host2...
found 2:0:0:7
found 2:0:0:9
found 2:0:1:7
found 2:0:1:9
found 2:0:2:7
found 2:0:2:9
found 2:0:3:7
found 2:0:3:9
scan mptscsih HBA host /sys/class/scsi_host/host1...
no new device found
scan mptscsih HBA host /sys/class/scsi_host/host0...
no new device found
run /usr/sbin/mppUtil -s busscan...
scan mpp virtual host /sys/class/scsi_host/host6...
found 6:0:0:7->/dev/sdo
found 6:0:0:9->/dev/sdp
--------------------------------------------------------------------------------------------------------
Odstranenie (delete) odmapovaneho (nenamapovaneho) alebo odpojeneho zariadenia
--------------------------------------------------------------------------------------------------------
# mppBusRescan -d
--------------------------------------------------------------------------------------------------------
Searching for unmapped or disconnected MPP devices.
Found 6:0:0:7/sdo as unmapped/disconnected device. Device removed.
Found 6:0:0:9/sdp as unmapped/disconnected device. Device removed.
--------------------------------------------------------------------------------------------------------
Kontrola ci sa zmenilo zariadenie/LUN. Ak nastala zmena aktualizuje sa RDAC z novym remapovanym
zariadenim alebo LUNom
--------------------------------------------------------------------------------------------------------
# mppBusRescan -u
--------------------------------------------------------------------------------------------------------
Searching for remapped MPP devices.
Found 6:0:0:7/sdo as remapped device. Device updated.
Found 6:0:0:9/sdp as remapped device. Device updated.
--------------------------------------------------------------------------------------------------------
Kontrola ci sa zmenila kapacita (capacity) zariadenia/LUNu. Ak nastala zmena aktualizuje sa kapacita
zariadenia/LUNu
--------------------------------------------------------------------------------------------------------
# mppBusRescan -c
--------------------------------------------------------------------------------------------------------
Searching for MPP device with size modified
Found 6:0:0:7/sdo with size modified. Device size updated.
Found 6:0:0:9/sdp with size modified. Device size updated.
--------------------------------------------------------------------------------------------------------PROBLEMY
[1] Problem
Server vidi 2 diskove polia pricom 'Hlavne' pole sa mirroruje na 'Zalozne' (problem hlavne ak sa pouziva LVM,
kedy nie je mozne pouzit devlabel na jednoznacnu identifikaciu pripojujuceho zariadenia)
================================================================================================================
Chybovy stav je nasledovny
--------------------------------------------------------------------------------------------------------
# mppUtil -a
--------------------------------------------------------------------------------------------------------
Hostname = simspcea
Domainname = (none)
Time = GMT 09/08/2010 10:50:35
---------------------------------------------------------------
Info of Array Module's seen by this Host.
---------------------------------------------------------------
ID WWN Type Name
---------------------------------------------------------------
0 600a0b800047dfb8000000004b591ee7 FC DS5300-POLUS
1 600a0b800047dd34000000004b612dbb FC DS5300-SITEL
---------------------------------------------------------------
Chybovy stav je nasledovny
--------------------------------------------------------------------------------------------------------
# cat ./devicemapping
--------------------------------------------------------------------------------------------------------
0:DS5300-POLUS
1:DS5300-SITEL
Chybovy stav je nasledovny
--------------------------------------------------------------------------------------------------------
# ls -l
--------------------------------------------------------------------------------------------------------
-rw------- 1 root root 361 Sep 7 11:57 600a0b800047dd34000000004b612dbb.wwn
-rw------- 1 root root 361 Sep 7 11:57 600a0b800047dfb8000000004b591ee7.wwn
-rw------- 1 root root 15 Sep 8 13:26 devicemapping
Pozadovany stav je nasledovny
--------------------------------------------------------------------------------------------------------
# mppUtil -a
--------------------------------------------------------------------------------------------------------
Hostname = simspcea
Domainname = (none)
Time = GMT 09/08/2010 10:50:35
---------------------------------------------------------------
Info of Array Module's seen by this Host.
---------------------------------------------------------------
ID WWN Type Name
---------------------------------------------------------------
0 600a0b800047dfb8000000004b591ee7 FC DS5300-POLUS
---------------------------------------------------------------
Pozadovany stav je nasledovny
--------------------------------------------------------------------------------------------------------
# cat ./devicemapping
--------------------------------------------------------------------------------------------------------
0:DS5300-POLUS
Pozadovany stav je nasledovny
--------------------------------------------------------------------------------------------------------
# ls -l
--------------------------------------------------------------------------------------------------------
-rw------- 1 root root 361 Sep 7 11:57 600a0b800047dfb8000000004b591ee7.wwn
-rw------- 1 root root 15 Sep 8 13:26 devicemapping
[1] Riesenie
================================================================================================================
[1] Vypneme pokusy o objavenie novych poli pri boote
[2] Skontrolujeme ci je mpp vypnute
--------------------------------------------------------------------------------------------------------
[1]# chkconfig mpp off
[2]# chkconfig --list | grep mpp
Odstranime diskove pole s nazvom 'DS5300-SITEL' z persistentneho mapovania
--------------------------------------------------------------------------------------------------------
# mppUpdate -d DS5300-SITEL
--------------------------------------------------------------------------------------------------------
The module DS5300-SITEL was removed from the MPP persistence mapping list (/var/mpp/devicemapping).
Detected 2 QLogic Host Adapter Port(s) on the system
Creating new MPP initrd image...
Odstranime WWN zaznam pre disk. pole 'DS5300-SITEL'
--------------------------------------------------------------------------------------------------------
# rm ./600a0b800047dd34000000004b612dbb.wwn
Aktualizujeme initrd obraz
--------------------------------------------------------------------------------------------------------
# mppUpdate
Restartneme stroj aby sa uplatnili nastavenia
--------------------------------------------------------------------------------------------------------
# init 6
Overime ze server vidi iba jedno (pozadovane) disk. pole
--------------------------------------------------------------------------------------------------------
# mppUtil -a
# cat ./devicemapping
[2] Problem
Potreba odstranenia LUNu (hot remove LUN)
================================================================================================================
[2] Riesenie
================================================================================================================
Odstranienie LUNu (hot remove LUN)
--------------------------------------------------------------------------------------------------------
# echo "1" > /sys/block/sdc/device/delete
# echo "1" > /sys/block/sdd/device/delete
Pricom sd* je diskove zariadenie, ktore chceme zo systemu odstranit.
Odstranienie LUNu (hot remove LUN)
--------------------------------------------------------------------------------------------------------
# echo 1 > /sys/class/scsi_device/h:c:t:l/device/delete
h is the HBA number
c is the channel on the HBA
t is the SCSI target ID
l is the LUN
Informacie o nazve zariadenia, cislo HBA, HBA channel, SCSI target a LUN mozeme ziskat nasledovne
--------------------------------------------------------------------------------------------------------
# ls -l /dev/disk/by-*
ENGLISH POPIS
--------------------------------------------------------------------------------------------------------
Take the path offline using echo offline > /sys/block/sda/device/state.
This will cause any subsequent IO sent to the device on this path to be failed immediately.
Device-mapper-multipath will continue to use the remaining paths to the device.
Kto pouziva co? (miniscript)
--------------------------------------------------------------------------------------------------------
for DSK in `ls /sys/block/`
do
if [ -e "/sys/block/${DSK}/device/vendor" ]
then
VENDOR=`cat /sys/block/${DSK}/device/vendor`
MODEL=`cat /sys/block/${DSK}/device/model`
echo “${DSK} ${VENDOR} ${MODEL}
fi
done
Zoznam ciest veducich k disk. polu s ID 1 (DS5300-SITEL)
--------------------------------------------------------------------------------------------------------
# mppUtil -g 1 | grep PathId
--------------------------------------------------------------------------------------------------------
PathId: 77000002 (hostId: 0, channelId: 0, targetId: 2)
PathId: 77010002 (hostId: 1, channelId: 0, targetId: 2)
PathId: 77000003 (hostId: 0, channelId: 0, targetId: 3)
PathId: 77010003 (hostId: 1, channelId: 0, targetId: 3)
Zmazeme vsetky cesty k disk. polu s ID 1 (DS5300-SITEL)
--------------------------------------------------------------------------------------------------------
# echo 1>/sys/bus/scsi/devices/0\:0\:2\:1/delete
# echo 1>/sys/bus/scsi/devices/0\:0\:2\:2/delete
# echo 1>/sys/bus/scsi/devices/1\:0\:2\:1/delete
# echo 1>/sys/bus/scsi/devices/1\:0\:2\:2/delete
# echo 1>/sys/bus/scsi/devices/0\:0\:3\:1/delete
# echo 1>/sys/bus/scsi/devices/0\:0\:3\:2/delete
# echo 1>/sys/bus/scsi/devices/1\:0\:3\:1/delete
# echo 1>/sys/bus/scsi/devices/1\:0\:3\:2/deleteLINKz *****
****** Sun StorageTek RDAC Multipath Failover Driver for Linux OS
--------------------------------------------------------------------------------------------------------
http://docs.sun.com/source/820-4738-13/chapsing.html
****** SCSI - Hot add, remove, rescan of SCSI devices
--------------------------------------------------------------------------------------------------------
http://www.ibm.com/developerworks/wikis/display/linuxp/scsi+-+hot+add,+remove,+rescan+of+scsi+devices
****** Online Storage Guide
--------------------------------------------------------------------------------------------------------
http://www.redhat.com/docs/en-US/Red_Hat_Enterprise_Linux/html/Online_Storage_Reconfiguration_Guide/index.html
****** LVM, Sysfs and Me
--------------------------------------------------------------------------------------------------------
http://ukuug.org/events/spring2009/programme/lvm-sysfs-me.pdf
XenServer und storage
--------------------------------------------------------------------------------------------------------
http://www.slideshare.net/ADannbacher/xenserver-und-storage
IBM System Storage DS4000/DS5000 Linux RDAC multipath driver package version 09.03.0B05.0331 FOR USE IN
LINUX 2.6 KERNEL
--------------------------------------------------------------------------------------------------------
http://www.lsi.com/rdac/ibm_sw_ds4krdac_09.03.0B05.0331_linux_anycpu.txt
Precitat Readme.txt v 'rdac-LINUX-09.03.0B05.0331-source.tar.gz'
--------------------------------------------------------------------------------------------------------
Readme.txt
DS4000 Implementation Cookbook
--------------------------------------------------------------------------------------------------------
http://utcc.utoronto.ca/ftp/docs/IBM/Storage%20Servers/DS4000%20Implementation%20Cookbook%20v1.0.pdf
Post-Installation Tasks
Dell™ PowerVault™ Modular Disk 3000 Systems Installation Guide
--------------------------------------------------------------------------------------------------------
http://support.dell.com/support/edocs/systems/md3000/en/2ndgen/ig/html/postinst.htm
Engenio Linux RDAC Driver Package for IBM DS4000/DS5000 Storage Subsystems
--------------------------------------------------------------------------
http://www.lsi.com/rdac/ds4000.html
Engenio Linux RDAC Driver Package for IBM DS3000 Storage Subsystems
--------------------------------------------------------------------------------------------------------
http://www.lsi.com/rdac/ds3000.html
multipath-tools
--------------------------------------------------------------------------------------------------------
http://christophe.varoqui.free.fr/
The Linux SCSI Generic (sg) Driver
--------------------------------------------------------------------------------------------------------
http://sg.danny.cz/sg/
Guide to the implementation of SCSI in Linux 2.6.X kernel
--------------------------------------------------------------------------------------------------------
http://devresources.linuxfoundation.org/dev/opendrivers/documentation/scsi_design_analysis_2_6_v2.pdfLINKz ***
Blades and external storage: Set up a fault-tolerant environment
How to set up an IBM BladeCenter HS21/LS21 server to boot from external DS3200 SAS storage
--------------------------------------------------------------------------------------------------------
http://www.ibm.com/developerworks/linux/library/l-sasboot/
Multipath Implementations in XenServer
--------------------------------------------------------------------------------------------------------
http://support.citrix.com/article/CTX121364
RDAC capable of hot removing unmapped LUNs?
--------------------------------------------------------------------------------------------------------
http://arstechnica.com/civis/viewtopic.php?f=21&t=34163
Using RDAC multipath driver
--------------------------------------------------------------------------------------------------------
http://www.redbooks.ibm.com/redbooks/SG246700/wwhelp/wwhimpl/js/html/wwhelp.htmLINKz - NEW
Linux on System z - Device Drivers, Features, and Commands - Linux Kernel 2.6 - Development stream
--------------------------------------------------------------------------------------------------------
http://public.dhe.ibm.com/software/dw/linux390/docu/l26ddd02.pdf