iSCSI SAN

[joris dedieu]

Hi,

Le lun. 26 avr. 2021 à 14:24, Julien Cigar <julien at perdition.city> a écrit :

> Hello,
>
> I'm wondering if something has already been written to implement a
> fully redundant and highly available FreeBSD ZFS based iSCSI SAN?
>

> I've setup some FreeBSD iSCSI SAN-like in the past (for small
> structures) and it has always worked well. However upgrades have always
> been painfull and, although there is ZFS, redundant power supplies,
> redundant switches with multipath, it's still a SPOF if a non-redundant
> component dies, like the motherboard. It's not like an HPE MSA-like
> system where everything is redundant out of the box.
>

>From my experience HA mechanisms are an infinite source of pain. Don't
forget the power of simplicity. What uptime do you get with your simple
setup ? When was the last time you see a decent mainboard crash ?
Also don't forget that FreeBSD has glusterfs and ceph.

If you want to do something similar to proprietary chassis, you should have
to look at SAS HBA and JBOD chassis, ATAoE chassis (if it still exists) or
stuff like that to attach your disks to your two mainboard. Still (OMG)
dealing with zfs (import -f) on failover, cluster STONITH and other voodoo.
You will have fully redundant design (see
https://i.stack.imgur.com/ijjpk.png )

Cheers
Joris


> So the idea came to me for the iSCSI target to setup 2 physical servers
> with a bunch of disks, create some raidzx on them and export one ZFS
> volume per initiator on each target, a bit like on (1)
>

> I've tried to setup that in a small "lab", with some jails, gmultipath,
> two switches, and several VLANs. Unfortunately no 10 gbits to test, but
> 3x1Gbits LAGG with LACP.
>
> The downside of this setup is that "half" of the storage is (temporarily)
> lost when a target reboots (freebsd-update, upgrades, etc), which
> de-facto disqualifies gmirror + UFS on the initiator side as a full
> resync of required and takes ages. With ZFS you don't have this problem
> as only the delta is resync. For the few tests I did it seems to work,
> but I'm wondering: zfs over zvol .. is it sane? does it makes sense?
> could I disable checksum on the initiator side to speed up things? do
> you see any race condition or ... with this setup (sync, etc)?
>
> What do you think?
>
> Thanks!
> Julien
>
> (1) https://gist.github.com/silenius/c6d1020aca54c47f71aa9f2a19a55ffe
>
>
> --
> Julien Cigar
> Belgian Biodiversity Platform (http://www.biodiversity.be)
> PGP fingerprint: EEF9 F697 4B68 D275 7B11  6A25 B2BB 3710 A204 23C0
> No trees were killed in the creation of this message.
> However, many electrons were terribly inconvenienced.
> _______________________________________________
> freebsd-fs at freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-fs
> To unsubscribe, send any mail to "freebsd-fs-unsubscribe at freebsd.org"
>