iSCSI/ZFS strangeness

[Ben RUBSON]

> Le 29 oct. 2015 à 03:26, Philip Murray <pmurray at nevada.net.nz> a écrit :
> 
> 
>> On 29/10/2015, at 2:57 PM, Michael W. Lucas <mwlucas at michaelwlucas.com> wrote:
>> 
>> Hi,
>> 
>> I'm experimenting with iSCSI HA with FreeBSD 10.2 amd64. I know people
>> do this sort of thing, but I can't figure out what I'm missing. (Most
>> of the tutorials cover HAST instead). I suspect the real problem is
>> "Lucas doesn't know the right search terms."
>> 
>> The goal is to make an iSCSI-based ZFS pool that's available to two
>> separate hosts, and remains available even if one of the iSCSI servers
>> fails. Instead, the pool hangs when either of the iSCSI servers goes
>> down.
> 
> I’m no expert and have never used iSCSI and FreeBSD before, but I think you might 
> want to look at the kern.iscsi.fail_on_disconnection sysctl.
> 
> man 4 iscsi
> 
> That means the devices will fail instead of hang, and ZFS might decide to mark those 
> devices as faulted instead of waiting for them to respond.

Hi,

Yes, kern.iscsi.fail_on_disconnection=1 in sysctl.conf will do the trick.
In addition, you can tune the following to minimize the impact / freeze when iSCSI fails :
kern.iscsi.ping_timeout=10
kern.iscsi.iscsid_timeout=10
kern.iscsi.login_timeout=10

In addition, in ctl.conf, perhaps you should use one target instruction per disk.
I began with all my disks/luns into a unique target configuration, but when a disk fails and you replace it, you need to refresh the iSCSI configuration using iscsictl -M. At this moment you will loose the connection to all disks, leading into many ZFS devices failed...
So I think a target per disks is better, then you only refresh the dead target : iscsictl -M -i <the_target_session>

And last thing, thank you very much for your quality books, a must-have :)

Best regards,

Ben