Warning Livedoc is no longer being updated and will be deprecated shortly. Please refer to https://documentation.tjhsst.edu.

SAN/iSCSI Administration

From Livedoc - The Documentation Repository
Jump to: navigation, search

This page contains administration guidelines and commands for iSCSI exports on the CSL SAN.

Adding an iSCSI LUN

This is used to add a new iSCSI LUN to the SAN.

ZVOL creation

All of the iSCSI exports on the SAN are currently backed by ZVOL block devices. ZVOLs provide much better performance than using raw datafiles over top of a normal filesystem and are easily expanded and managed using ZFS.

For ease of management, related ZVOLs are typically grouped underneath a ZFS filesystem. First we create the filesystem, then we create each of the ZVOLs beneath it. For most VMs, you will need two ZVOLs; one for the root partition and one for swap.

1  zfs create apocalypse/vms/<vm>
2  zfs create -V 15G apocalypse/vms/<vm>/root
3  zfs create -V 1G apocalypse/vms/<vm>/swap

iSCSI Exports

iSCSI exports using targetcli

Assuming a structure like:

/dev/zvol/sonic/vms/openafs3/vicepb
/dev/zvol/sonic/vms/openafs3/root
/dev/zvol/sonic/vms/openafs3/vicepa
/dev/zvol/sonic/vms/openafs3/swap

Then you can generate targetcli commands for 2 AFS servers running on 3 different VM servers with 4 different disks on sonic:

 1 ip="172.16.4.1"
 2 for i in openafs1 openafs3; do
 3   for h in root swap vicepa vicepb; do
 4     echo "/backstores/iblock create name=$i-$h dev=/dev/zvol/sonic/vms/$i/$h"
 5     echo "/iscsi create iqn.1992-03.edu.tjhsst:target:sonic.0:sn.$i-$h"
 6     echo "/iscsi/iqn.1992-03.edu.tjhsst:target:sonic.0:sn.$i-$h/tpg1/luns create /backstores/iblock/$i-$h"
 7     echo "/iscsi/iqn.1992-03.edu.tjhsst:target:sonic.0:sn.$i-$h/tpg1/portals create ip_address=$ip ip_port=3260"
 8     echo "/iscsi/iqn.1992-03.edu.tjhsst:target:sonic.0:sn.$i-$h/tpg1 set attribute authentication=0"
 9     for j in galapagos antipodes cocos valdes; do
10       echo "/iscsi/iqn.1992-03.edu.tjhsst:target:sonic.0:sn.$i-$h/tpg1/acls create iqn.1992-03.edu.tjhsst:initiator:$j.csl.tjhsst.edu.0"
11     done
12   done
13 done
14 echo "/ saveconfig"
15 echo "/ saveconfig"

If you want to get the disk WWNs you can do something like:

root@barrel:/var/target# cat backup-2016-12-22_00:10:36.lio | egrep '(disk|wwn)' | tr -s ' ' ' ' | cut -d" " -f3
openafs1-root
754d8a3c-6fad-40f5-9fbe-37da93ed4edb
openafs1-swap
4afe8f78-3d12-4134-9fad-71439cd8f5cb
openafs1-vicepa
8e3ce331-d0fe-4a21-bc2a-64a75d4fce7b
openafs1-vicepb
b0411608-4eed-4681-ac4b-94775f4e44ac
openafs3-root
36f78b40-b306-4305-9106-3624af31af11
openafs3-swap
4d0b59b1-67dd-491e-bf82-d276c333184a
openafs3-vicepa
54a325d2-5cd9-4fb7-9026-0c678fdda1d1
openafs3-vicepb
a001501f-f470-4cc1-b411-628fd5427d46


Client Config with open-iscsi

See [1] for iscsiadm commands.

Find the proper_target_name of the LUNs you want to add on the storage server in /var/target/. It should look something like

 iqn.1992-03.edu.tjhsst:target:apocalypse.0:sn.jday-root

For apocalypse, the IP address is 172.16.3.1 and for sonic, it is 172.16.4.1. Find the SID by running

 iscsiadm -m discovery -t sendtargets -p 172.16.3.1:3260

Find the line with the proper_target_name you found earlier and the SID should be in brackets. To manually login, run

 iscsiadm -m node -T iqn.1992-03.edu.tjhsst:target:apocalypse.0:sn.jday-root --login

Finally, each VM Server needs to be told to rescan for iSCSI LUN changes. IMPORTANT - DO NOT restart iscsid or you will crash all of the VMs on the server; rather use the following command to instruct iscsid to scan for changes.

 iscsiadm -m session --rescan


With the WWN you found earlier, take the first 8 characters and grep for it in /dev/mapper on the VM server. You should have your WWID now.


Multipath Configuration

Multipath now needs to be configured on the client servers. This is most easily done on one server and then copied to the others. On one of the client servers, run:

Note: this will probably be deprecated soon as we'll just have a LACP trunk for the server instead of multiple IPs

Currently barrel does not use multipath!

 multipath

Then, for each LUN you are adding, run:

 ls /dev/mapper/ | grep 090427202e6db84c #(replace this with the SCSI_SN of the LUN)
 36001405090427202e6db84c000000000

For each LUN, add an entry in /etc/multipath.conf similar to the following; replacing the WWID with the value returned above, and the alias with the name of the LUN.

 multipath {
     wwid 36001405090427202e6db84c000000000
     alias apocalypse-ion-root
     features                "1 queue_if_no_path"
 }

Next for each LUN, you need to remove the temporary multipath alias (which defaults to the WWID) by running:

 multipath -f 36001405090427202e6db84c000000000 #(replace this with the WWID)

Finally, re-run the multipath command to regenerate the multipath devices with the new aliases.

You should now have an entry in /dev/mapper/ for each LUN you added to the SAN. Once you have verified that everything is working, you need to replicate this configuration across the other systems connected to the SAN. Copy the updated /etc/multipath.conf to them, then run:

 iscsiadm -m session --rescan
 multipath

See the main SAN article for a list of systems currently connected to the SAN via iSCSI.