Re: ZFS on a shared iSCSI

07.02.24 16:24, Julien Cigar:
> On Wed, Feb 07, 2024 at 04:12:21PM +0200, Oleksandr Kryvulia wrote:
>> 07.02.24 14:24, Julien Cigar:
>>> On Wed, Feb 07, 2024 at 11:55:54AM +0100, Andrea Brancatelli wrote:
>>>> Hello guys, I'm not 100% this is the correct list to ask this question, if
>>>> not please feel free to point me in the right direction.
>>>>
>>>> I was wondering what could be the best recipe to have an HA cluster sharing
>>>> an external ZFS storage.
>>>>
>>>> Let's say I have two servers running a bunch of Jails and, thus, I'd like
>>>> to use ZFS as the underlying storage layer and I have an external (iSCSI)
>>>> storage connected.
>>>>
>>>> Would it be "easily possible" to have some (2?) iSCSI LUN exposed to both
>>>> servers and then activate the pool on one or the other server?
>>>>
>>>> The idea would be to reactivate the filesystem from server A on server B if
>>>> the server A fails.
>>>>
>>>> Would it be "easier" to replicate everything and zfs send datas back and
>>>> forth? Clearly that would mean doubling datas and havin a scheduled replica
>>>> with a possible delay in data replication, so I'd like to avoid this.
>>>>
>>>> Any thoughts?
>>> I asked something similar a few years ago, the whole thread may be an interesting
>>> read: https://lists.freebsd.org/pipermail/freebsd-fs/2016-June/023456.html
>>>
>>> Just to say that I tried a lot of different things, and came up to the
>>> conclusion that for critical production usage all those things are
>>> fragile and with hidden dragons everywhere.
>>>
>>> I love FreeBSD (I use it exclusively everywhere, also at work), but
>>> there aren't any supported open-source solution for something like an
>>> "highly available ZFS cluster". On the commercial side there is RSF-1
>>> and beasts like Pure storage ($$$) but ...
>>>
>>> I would really like to see CEPH ported to FreeBSD, that would be
>>> extremely useful
>> What about www/minio? Does anyone have any experience of production use? We
>> have a lot of bhyve vm's backened by local zfs storage, but I am interesting
>> in some kind of hyperconverged solution.
> I'm not sure Minio offers block storage..?
No, but maybe md(4) can be used if needed. For bhyve vm's in my case we 
can move from zvol to img files.