Julien Cigar julien at
Mon Apr 26 12:24:52 UTC 2021


I'm wondering if something has already been written to implement a 
fully redundant and highly available FreeBSD ZFS based iSCSI SAN?

I've setup some FreeBSD iSCSI SAN-like in the past (for small 
structures) and it has always worked well. However upgrades have always
been painfull and, although there is ZFS, redundant power supplies,
redundant switches with multipath, it's still a SPOF if a non-redundant 
component dies, like the motherboard. It's not like an HPE MSA-like
system where everything is redundant out of the box.

So the idea came to me for the iSCSI target to setup 2 physical servers 
with a bunch of disks, create some raidzx on them and export one ZFS 
volume per initiator on each target, a bit like on (1)

I've tried to setup that in a small "lab", with some jails, gmultipath,
two switches, and several VLANs. Unfortunately no 10 gbits to test, but 
3x1Gbits LAGG with LACP.

The downside of this setup is that "half" of the storage is (temporarily) 
lost when a target reboots (freebsd-update, upgrades, etc), which
de-facto disqualifies gmirror + UFS on the initiator side as a full
resync of required and takes ages. With ZFS you don't have this problem
as only the delta is resync. For the few tests I did it seems to work, 
but I'm wondering: zfs over zvol .. is it sane? does it makes sense?
could I disable checksum on the initiator side to speed up things? do
you see any race condition or ... with this setup (sync, etc)?

What do you think?



Julien Cigar
Belgian Biodiversity Platform (
PGP fingerprint: EEF9 F697 4B68 D275 7B11  6A25 B2BB 3710 A204 23C0
No trees were killed in the creation of this message.
However, many electrons were terribly inconvenienced.

More information about the freebsd-fs mailing list