Oracle Linux and iSCSI
What is iSCSI ?
The Internet Small Computer System Interface (iSCSI) is an IP-based standard for connecting storage devices. iSCSI encapsulates SCSI commands in IP network packets.
A client system (iSCSI initiator) accesses the storage server (iSCSI target) over an IP network. To an iSCSI initiator, the storage appears to be locally attached.
An iSCSI target is typically a dedicated, network-connected storage device but it can also be a general-purpose computer.
Configuring an iSCSI target
As described above, an ISCSI target is typically a network-connected storage device; however with Oracle Linux 6 and 7 iSCSI targets can be created and consumed by other Oracle 6 and 7 instances using iSCSI initiators.
Oracle Linux 7 iSCSI target creation and configuration
If the iSCSI target server firewall is running then we need to enable the firewall to pass iSCSI traffic for the chosen port. If the default port of 3260 has been used then the following firewall command can be used:
[root@myol7server ~]# firewall-cmd –permanent –add-service iscsi-target
If a non default port has been chosen then run this command using the desired port:
[root@myol7server ~]# firewall-cmd –permanent --add-port=<port>/tcp
Following any firewall changes we need to reload the firewall:
[root@myol7server ~]# firewall-cmd –reload
Oracle Linux 7 provides the targetcli utility which launches its own shell:
[root@myol7server ~]# targetcli
targetcli shell version 2.1.fb41
Copyright 2011-2013 by Datera, Inc and others.
For help on commands, type 'help'.
/>
From here we can list any created objects in a hierarchical manner:
/> ls
o- / ..................................................................... [...]
o- backstores .......................................................... [...]
| o- block .............................................. [Storage Objects: 0]
| o- fileio ............................................. [Storage Objects: 0]
| o- pscsi .............................................. [Storage Objects: 0]
| o- ramdisk ............................................ [Storage Objects: 0]
o- iscsi ........................................................ [Targets: 0]
o- loopback ..................................................... [Targets: 0]
/>
For an initial configuration this will be empty or display 0.
We move to the /backstores/block directory to create a block storage object in this example based upon a 10GB storage partition (/dev/sdb). The name is arbitrary and our example uses LUN_0:
/> cd /backstores/block
/backstores/block>
/backstores/block> create name=LUN_0 dev=/dev/sdb
Created block storage object LUN_0 using /dev/sdb.
If we run the ls command in this directory we can see the LUN we created:
/backstores/block> ls
o- block .................................................. [Storage Objects: 1]
o- LUN_0 ........................ [/dev/sdb (10.0GiB) write-thru deactivated]
We change to the /iscsi directory and create an iSCSI target and IQN (iSCSI Qualified Name). It is possible to use your own name for the target. If in our example you use the create command with no target name a default target name and IQN is assigned:
/backstores/block> cd /iscsi
/iscsi> create
Created target iqn.2003-01.org.linux-iscsi.myol7server.x8664:sn.87b2a5cf5bcd.
Created TPG 1.
Global pref auto_add_default_portal=true
Created default portal listening on all IPs (0.0.0.0), port 3260.
We can now list the Target Portal Group (TPG) hierarchy using the ls command. TPGs allow iSCSI to support multiple complete configurations within one target. This is useful for complex quality-of-service configurations. targetcli will automatically create one TPG when the target is created:
/iscsi> ls
o- iscsi .......................................................... [Targets: 1]
o- iqn.2003-01.org.linux-iscsi.myol7server.x8664:sn.87b2a5cf5bcd ..... [TPGs: 1]
o- tpg1 ............................................. [no-gen-acls, no-auth]
o- acls ........................................................ [ACLs: 0]
o- luns ........................................................ [LUNs: 0]
o- portals .................................................. [Portals: 1]
o- 0.0.0.0:3260 ................................................... [OK]
We change directory to the luns subdirectory of the TPG hierarchy where we will add our LUN_0 to the TPG.
Note: the /tpg1/luns directory is needed after iqn.2003-01.org.linux-iscsi.myol7server.x8664:sn.87b2a5cf5bcd
/iscsi> cd iqn.2003-01.org.linux-iscsi.myol7server.x8664:sn.87b2a5cf5bcd/tpg1/luns
/iscsi/iqn.20...bcd/tpg1/luns>
/iscsi/iqn.20...bcd/tpg1/luns> create /backstores/block/LUN_0
Created LUN 0.
We change to the portals directory of the TPG hierarchy to specify the IP address and port of the iSCSI endpoint. The create target / IQN process creates a default portal which listens on 0.0.0.0 port 3260. To configure an IP address and if needed a different port this default portal needs to be removed before creating a new one:
cd ../
cd portals
/iscsi/iqn.20.../tpg1/portals> delete 0.0.0.0 3260
The IP address in our example is the IP address of the myol7server. If you omit the port (as in my example) the default port of 3260 is used:
/iscsi/iqn.20.../tpg1/portals> create 10.80.154.38
Using default IP port 3260
Created network portal 10.80.154.38:3260.
The syntax for the command is create <IP Address> <port>
We can now list object hierarchy using the ls / command which will show the configured block storage objects and TPG:
/iscsi/iqn.20.../tpg1/portals> ls /
o- / ..................................................................... [...]
o- backstores .......................................................... [...]
| o- block .............................................. [Storage Objects: 1]
| | o- LUN_0 ...................... [/dev/sdb (10.0GiB) write-thru activated]
| o- fileio ............................................. [Storage Objects: 0]
| o- pscsi .............................................. [Storage Objects: 0]
| o- ramdisk ............................................ [Storage Objects: 0]
o- iscsi ........................................................ [Targets: 1]
| o- iqn.2003-01.org.linux-iscsi.myol7server.x8664:sn.87b2a5cf5bcd ... [TPGs: 1]
| o- tpg1 ........................................... [no-gen-acls, no-auth]
| o- acls ...................................................... [ACLs: 0]
| o- luns ...................................................... [LUNs: 1]
| | o- lun0 .................................... [block/LUN_0 (/dev/sdb)]
| o- portals ................................................ [Portals: 1]
| o- 10.80.154.38:3260 ............................................ [OK]
o- loopback ..................................................... [Targets: 0]
Best practice is to configure authentication between the target and initiators; we will configure CHAP and ACL (Access Control List) authentication. The ACL will define the initiators that are allowed to connect to the iSCSI target. This can be any string in the same format for example iqn.2016-02.local.simon:ol71. This string needs to be present on the iSCSI initiator in order to connect to the iSCSI target. It is possible to use the contents of the /etc/initiatorname.iscsi file. For example on an Oracle Linux 7 server:
[root@myol7server ~]# cat /etc/iscsi/initiatorname.iscsi
InitiatorName=iqn.1988-12.com.oracle:1a36686194fc
Back on the iSCSI target server we create an ACL and then define a username and password for CHAP:
/iscsi/iqn.20.../tpg1/portals> cd ../
/iscsi/iqn.20...a5cf5bcd/tpg1> cd acls
/iscsi/iqn.20...bcd/tpg1/acls>
/iscsi/iqn.20...bcd/tpg1/acls> create iqn.2016-02.local.simon:ol71
Created Node ACL for iqn.2016-02.local.simon:ol71
Created mapped LUN 0.
We change to the new ACL directory and add a user and password:
/iscsi/iqn.20...bcd/tpg1/acls> cd iqn.2016-02.local.simon:ol71
/iscsi/iqn.20...al.simon:ol71> set auth userid=mychapuser
Parameter userid is now 'mychapuser'.
/iscsi/iqn.20...al.simon:ol71> set auth password=mychappassword
Parameter password is now 'mychappassword'.
We now verify the configuration:
/iscsi/iqn.20...al.simon:ol71> cd /
/> ls
o- / ..................................................................... [...]
o- backstores .......................................................... [...]
| o- block .............................................. [Storage Objects: 1]
| | o- LUN_0 ...................... [/dev/sdb (10.0GiB) write-thru activated]
| o- fileio ............................................. [Storage Objects: 0]
| o- pscsi .............................................. [Storage Objects: 0]
| o- ramdisk ............................................ [Storage Objects: 0]
o- iscsi ........................................................ [Targets: 1]
| o- iqn.2003-01.org.linux-iscsi.myol7server.x8664:sn.87b2a5cf5bcd ... [TPGs: 1]
| o- tpg1 ........................................... [no-gen-acls, no-auth]
| o- acls ...................................................... [ACLs: 1]
| | o- iqn.2016-02.local.simon:ol71 ..................... [Mapped LUNs: 1]
| | o- mapped_lun0 ............................. [lun0 block/LUN_0 (rw)]
| o- luns ...................................................... [LUNs: 1]
| | o- lun0 .................................... [block/LUN_0 (/dev/sdb)]
| o- portals ................................................ [Portals: 1]
| o- 10.80.154.38:3260 ............................................ [OK]
o- loopback ..................................................... [Targets: 0]
/>
We have to traverse to the root directory and save the configuration so it persists across reboots of the iSCSI target server:
/iscsi/iqn.20...bcd/tpg1/acls> cd /
/> saveconfig
Last 10 configs saved in /etc/target/backup.
Configuration saved to /etc/target/saveconfig.json
Finally we restart the iSCSI target service and enable it to start at subsequent reboots. The enable command is necessary and if not enabled to start at boot time the configuration will not be loaded:
[root@myol6server ~]# systemctl restart target.service
[root@myol6server ~]# systemctl enable target.service
[r
Oracle Linux 6 iSCSI target creation and configuration
There are two approaches to this configuration:
- Configuration files which are persistent over reboots but changes are not instantly available
- Online configuration using the tgtadm command; changes are available instantly, but not consistent over reboots
We will use the tgtadm command and then show how to output the configuration into a file and then populate the relevant configuration file to ensure our configuration persists over reboots. Therefore our approach will be the first approach [1].
Firstly we need to check the the scsi-target-utils package is installed:
[root@myol6server ~]# rpm -qa scsi-target-utils
scsi-target-utils-1.0.24-18.0.1.el6.x86_64
If the package is not installed then install it:
[root@myol6server ~]# yum install scsi-target-utils
By default iSCSI runs on port 3260. If a firewall is being used the the following configuration is needed:
[root@myol6server ~]# iptables -I INPUT -p tcp -m tcp --dport 3260 -j ACCEPT
[root@myol6server ~]# service iptables save
[root@myol6server ~]# service iptables restart
Edit the /etc/tgt/targets.conf file to enable the initial iSCSI target configuration. Examples are contained within the file to customize the desired presentation. This example is based upon a 10GB storage partition (/dev/sdb).
For our example we will use the following:
<target iqn.2017-10.com.mydomain.myol6server:target1>
direct-store /dev/sdb # LUN 1
</target>
The syntax for the target name is as follows:
iqn.YYYY-MM.reverse_FQDN[:target_name]
where:
- YYYY
-MM - Specifies the year and month that the naming authority took ownership of the domain.
reverse_FQDN- Specifies the reverse fully qualified domain name of the naming authority.
- target_name
- Specifies an optional target name, which identifies the target at a site.
We need to start the iSCSI target service and also set the service to automatically start upon any restart:
[root@ myol6server ~]# service tgtd start
Starting SCSI target daemon: [ OK ]
[root@ myol6server ~]# chkconfig tgtd on
We can check that the iSCSI target has been correctly configured:
[root@myol6server ~]# tgtadm -o show -m target
Target 1: iqn.2017-10.com.mydomain.myol6server:target1
System information:
Driver: iscsi
State: ready
I_T nexus information:
LUN information:
LUN: 0
Type: controller
SCSI ID: IET 00010000
SCSI SN: beaf10
Size: 0 MB, Block size: 1
Online: Yes
Removable media: No
Prevent removal: No
Readonly: No
Backing store type: null
Backing store path: None
Backing store flags:
LUN: 1
Type: disk
SCSI ID: IET 00010001
SCSI SN: beaf11
Size: 10737 MB, Block size: 512
Online: Yes
Removable media: No
Prevent removal: No
Readonly: No
Backing store type: rdwr
Backing store path: /dev/sdb
Backing store flags:
Account information:
ACL information:
ALL
For further configuration we will use the tgtadm utility which also has some useful external scripts to manipulate configuration. For example the tgt-admin script has some useful features:
[root@myol6server ~]# tgt-admin
Usage:
tgt-admin [OPTION]...
This tool configures tgt targets.
-e, --execute read /etc/tgt/targets.conf and execute tgtadm commands
--delete <value> delete all or selected targets
(see "--delete help" for more info)
--offline <value> put all or selected targets in offline state
(see "--offline help" for more info)
--ready <value> put all or selected targets in ready state
(see "--ready help" for more info)
--update <value> update configuration for all or selected targets
(see "--update help" for more info)
-s, --show show all the targets
-C, --control-port <NNNN> specify the control port to connect to
-c, --conf <conf file> specify an alternative configuration file
--ignore-errors continue even if tgtadm exits with non-zero code
-f, --force force some operations even if the target is in use
-p, --pretend only print tgtadm options
--dump dump current tgtd configuration (note: does not
include detailed parameters, like write caching)
-v, --verbose increase verbosity (show tgtadm commands)
-h, --help show this help
The tgt-setup-lun utility is useful for creating targets, adding disks to targets and also can specify which iSCSI initiators are allowed to connect to the iSCSI target:
[root@myol6server ~]# tgt-setup-lun
usage:
tgt-setup-lun -n tgt_name -d dev -b bs_name -t transport [initiator_IP1 initiator_IP2 ...]
defaults:
backing store: rdwr
transport: iscsi
initiator: ALL
examples:
tgt-setup-lun -n tgt-1 -d /dev/sdb1 192.168.1.2
tgt-setup-lun -n tgt-2 -d /tmp/null -b null -t iser
tgt-setup-lun -n tgt-3 -d ~/disk3.bin -b rdwr 192.168.1.2 192.168.1.3
The common switches for the command are as follows:
-n = target name
-d = target device
The IP addresses at the end of the command which are space separated are the IP addresses of the iSCSI initiators which are allowed to access the iSCSI target. As stated above these utilities configure changes instantly but are not persistent across reboots.
We can use the tgtadm utility on its own to create or manipulate configuration of an iSCSI target. For example, an ACL which defines the iSCSI initiators that can connect to the iSCSI target. We can either configure by the iSCSI initiator id (gathered from the iSCSI initiator /etc/initiatorname.iscsi file)or via IP address. The two example commands show firstly configuration by iSCSI initiator, then IP. To obtain the target id run the tgtadm -o show -m target command and look for the target ID number on the first line (in our example 1):
Target 1: iqn.2017-10.com.mydomain.myol6server:target1
[root@myol6server ~]# tgtadm --lld iscsi --op bind --mode target --tid 1 --initiator-name iqn.1988-12.com.oracle:1a36686194fc
[root@myol6server ~]# tgtadm --lld iscsi --op bind --mode target --tid 1 --initiator-address 10.80.121.118
[root@myol6server ~]# tgtadm --lld iscsi --op show --mode target
Target 1: iqn.2017-10.com.mydomain.myol6server:target1
System information:
Driver: iscsi
State: ready
I_T nexus information:
LUN information:
LUN: 0
Type: controller
SCSI ID: IET 00010000
SCSI SN: beaf10
Size: 0 MB, Block size: 1
Online: Yes
Removable media: No
Prevent removal: No
Readonly: No
Backing store type: null
Backing store path: None
Backing store flags:
LUN: 1
Type: disk
SCSI ID: IET 00010001
SCSI SN: beaf11
Size: 10737 MB, Block size: 512
Online: Yes
Removable media: No
Prevent removal: No
Readonly: No
Backing store type: rdwr
Backing store path: /dev/sdb
Backing store flags:
Account information:
ACL information:
ALL
10.80.121.118
iqn.1988-12.com.oracle:1a36686194fc
We can see in the last two lines of the command output the IP and iSCSI initiator in the Access Control List (ACL). We can also configure CHAP using the tgtadm command. The first command using –op new creates the CHAP user and password. The second command using –op bind attaches the CHAP user to the target id:
[root@myol6server ~]# tgtadm --lld iscsi --op new --mode account --user mychapuser --password mychappassword
[root@myol6server ~]# tgtadm --lld iscsi --op bind --mode account --tid 1 --user mychapuser
[root@myol6server ~]# tgtadm -o show -m target
Target 1: iqn.2017-10.com.mydomain.myol6server:target1
System information:
Driver: iscsi
State: ready
I_T nexus information:
LUN information:
LUN: 0
Type: controller
SCSI ID: IET 00010000