Using iscsi with multiple targets
Sven Willenberger
sven at dmv.com
Tue Jul 15 13:45:33 UTC 2008
On Mon, 2008-07-14 at 11:29 +0300, Danny Braniss wrote:
> > FreeBSD 7.0
> >
> > I have 2 machines with identical configurations/hardware, let's call them A (master)
> > and B (slave). I have installed iscsi-target from ports and have set up 3 targets
> > representing the 3 drives I wish to be connected to from A.
> >
> > The Targets file:
> > # extents file start length
> > extent0 /dev/da1 0 465GB
> > extent1 /dev/da2 0 465GB
> > extent2 /dev/da3 0 465GB
> >
> > # target flags storage netmask
> > target0 rw extent0 192.168.0.1/24
> > target1 rw extent1 192.168.0.1/24
> > target2 rw extent2 192.168.0.1/24
> >
> > I then start up iscsi_target and all is good.
> >
> > Now on A I have set up my /etc/iscsi.conf file as follows:
> >
> > # cat /etc/iscsi.conf
> > data1 {
> > targetaddress=192.168.0.252
> > targetname=iqn.1994-04.org.netbsd.iscsi-target:target0
> > initiatorname=iqn.2005-01.il.ac.huji.cs::BSD-2-1.sven.local
> > }
> > data2 {
> > targetaddress=192.168.0.252
> > targetname=iqn.1994-04.org.netbsd.iscsi-target:target1
> > initiatorname=iqn.2005-01.il.ac.huji.cs::BSD-2-1.sven.local
> > }
> > data3 {
> > targetaddress=192.168.0.252
> > targetname=iqn.1994-04.org.netbsd.iscsi-target:target2
> > initiatorname=iqn.2005-01.il.ac.huji.cs::BSD-2-1.sven.local
> > }
> >
> > So far so good, now come the issues. First of all, it would appear that with
> > iscontrol one can only start one "named" session at a time; for example
> > /sbin/iscontrol -n data1
> > /sbin/iscontrol -n data2
> > /sbin/isconrtol -n data3
> >
> > I guess that is ok, except that each invocation of iscontrol resets the other
> > sessions. Here is the camcontrol and dmesg output from running the above 3 commands.
> >
> > # camcontrol devlist
> > <AMCC 9550SXU-8L DISK 3.08> at scbus0 target 0 lun 0 (pass0,da0)
> > <AMCC 9550SXU-8L DISK 3.08> at scbus0 target 1 lun 0 (pass1,da1)
> > <AMCC 9550SXU-8L DISK 3.08> at scbus0 target 2 lun 0 (pass2,da2)
> > <AMCC 9550SXU-8L DISK 3.08> at scbus0 target 3 lun 0 (pass3,da3)
> > <NetBSD NetBSD iSCSI 0> at scbus1 target 0 lun 0 (da5,pass5)
> > <NetBSD NetBSD iSCSI 0> at scbus1 target 1 lun 0 (da6,pass6)
> > <NetBSD NetBSD iSCSI 0> at scbus1 target 2 lun 0 (da4,pass4)
> >
> >
> > [ /sbin/iscontrol -n data1 ]
> > da4 at iscsi0 bus 0 target 0 lun 0
> > da4: <NetBSD NetBSD iSCSI 0> Fixed Direct Access SCSI-3 device
> >
> > [ /sbin/iscontrol -n data2 ]
> > (da4:iscsi0:0:0:0): lost device
> > (da4:iscsi0:0:0:0): removing device entry
> > da4 at iscsi0 bus 0 target 0 lun 0
> > da4: <NetBSD NetBSD iSCSI 0> Fixed Direct Access SCSI-3 device
> > da5 at iscsi0 bus 0 target 1 lun 0
> > da5: <NetBSD NetBSD iSCSI 0> Fixed Direct Access SCSI-3 device
> >
> > [ /sbin/iscontrol -n data3 ]
> > (da4:iscsi0:0:0:0): lost device
> > (da4:iscsi0:0:0:0): removing device entry
> > (da5:iscsi0:0:1:0): lost device
> > (da5:iscsi0:0:1:0): removing device entry
> > da4 at iscsi0 bus 0 target 2 lun 0
> > da4: <NetBSD NetBSD iSCSI 0> Fixed Direct Access SCSI-3 device
> > da5 at iscsi0 bus 0 target 0 lun 0
> > da5: <NetBSD NetBSD iSCSI 0> Fixed Direct Access SCSI-3 device
> > da6 at iscsi0 bus 0 target 1 lun 0
> > da6: <NetBSD NetBSD iSCSI 0> Fixed Direct Access SCSI-3 device
> >
> >
> > It would appear that rather than appending the new device to the end of the "da"
> > devices, it starts to do some type of naming queue after the second device. If I am
> > to use these devices in any type of automated setup, how can make sure that after
> > these commands, "da6" will always be target 1 (i.e. /dev/da2 on the slave machine).
> >
> > Next, there is no "startup" script for iscontrol - would that simply have to be
> > added the system or is there a way with sysctl that it could be done. The plan here
> > is use gmirror such that /dev/da1 on A is mirrored with the /dev/da1 on B using iscsi.
>
> Hi Sven,
> I just tried it here, and it seems that at the end all is ok :-)
> I think the lost/removing/found has something to do to iscontrol calling
> camcontrol rescan - I will check this later, but the end result is that
> you should have all /dev/da's.
> I don't see any reasonable safe way to tie a scsi# (/dev/dan),
> except to label (see glabel) the disk.
> The startup script is, at the moment, not trivial, but I'm attaching
> it, so someone can suggest improvements :-)
> #!/bin/sh
>
> # PROVIDE: iscsi
> # REQUIRE: NETWORKING
> # BEFORE: DAEMON
> # KEYWORD: nojail shutdown
>
> #
> # Add the following lines to /etc/rc.conf to enable iscsi:
> #
> # iscsi_enable="YES"
> # iscsi_fstab="/etc/fstab.iscsi"
>
> . /etc/rc.subr
> . /cs/share/etc/rc.subr
>
> name=iscsi
> rcvar=`set_rcvar`
>
> command=/sbin/iscontrol
>
> iscsi_enable=${iscsi_enable:-"NO"}
> iscsi_fstab=${iscsi_fstab:-"/etc/fstab.iscsi"}
> iscsi_exports=${iscsi_exports:-"/etc/exports.iscsi"}
> iscsi_debug=${iscsi_debug:-0}
> start_cmd="iscsi_start"
> faststop_cmp="iscsi_stop"
> stop_cmd="iscsi_stop"
>
> start_precmd="iscontrol_precmd"
> iscontrol_prog=${iscontrol_prog:-"iscontrol"}
> iscontrol_log=${iscontrol_log:-"/var/log/$iscontrol_prog"}
> iscontrol_syslog=${iscontrol_syslog:-"644 3 100 * JC"}
>
> iscontrol_precmd()
> {
> setup_syslog "$iscontrol_prog" "$iscontrol_log" "$iscontrol_syslog"
> }
>
> iscsi_wait()
> {
> dev=$1
> trap "echo 'wait loop cancelled'; exit 1" 2
> count=0
> while true; do
> if [ -c $dev ]; then
> break;
> fi
> if [ $count -eq 0 ]; then
> echo -n Waiting for ${dev}': '
> fi
> count=$((${count} + 1))
> if [ $count -eq 6 ]; then
> echo " Failed for dev=$dev"
> return 0
> break
> fi
> echo -n '.'
> sleep 5;
> done
> echo "$dev ok."
> return 1
> }
>
> iscsi_start()
> {
> #
> # load needed modules
> for m in iscsi_initiator geom_label; do
> kldstat -qm $m || kldload $m
> done
>
> sysctl debug.iscsi_initiator=$iscsi_debug
> #
> # start iscontrol for each target
> if [ -n "${iscsi_targets}" ]; then
> for target in ${iscsi_targets}; do
> ${command} ${rc_flags} -n ${target}
> done
> fi
>
> if [ -f "${iscsi_fstab}" ]; then
> while read spec file type opt t1 t2
> do
> case ${spec} in
> \#*|'')
> ;;
> *)
> if iscsi_wait ${spec}; then
> break;
> fi
> echo type=$type spec=$spec file=$file
> fsck -p ${spec} && mkdir -p ${file} && mount ${spec} ${file}
> chmod 755 ${file}
> ;;
> esac
> done < ${iscsi_fstab}
> fi
>
> if [ -f "${iscsi_exports}" ]; then
> cat ${iscsi_exports} >> /etc/exports
> #/etc/rc.d/mountd reload
> kill -1 `cat /var/run/mountd.pid`
> fi
> }
>
> iscsi_stop()
> {
> echo 'iscsi stopping'
> while read spec file type opt t1 t2
> do
> case ${spec} in
> \#*|'')
> ;;
> *)
> echo iscsi: umount $spec
> umount -fv $spec
> ;;
> esac
> done < ${iscsi_fstab}
> }
>
> load_rc_config $name
> run_rc_command "$1"
Thanks for the script and information. I have found that so long as I
add the targets in the same order each time (i.e. iscontrol -n data1,
then data2, then data3) the actual scsi target will be the same even
though the device number will change. So if I do something like:
camcontrol devlist | grep "scbus1 target 0" | sed 's/^.*da\(.\).*/da\1/'
I can parse out the device number it was assigned.
After some other reports it would appear as though iscsi will not work
for my needs (which basically involves creating a mirrored
pool/filesystem spanning 2 machines) as iscsi will lock when the target
machine goes down. I will continue this in a new thread as it strays
from the question originally asked here.
Sven
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
Url : http://lists.freebsd.org/pipermail/freebsd-stable/attachments/20080715/f647aae6/attachment.pgp
More information about the freebsd-stable
mailing list