Fun with HAST and inter-host connections
julien at perdition.city
Thu Aug 31 11:49:38 UTC 2017
On Wed, Aug 30, 2017 at 10:40:03PM +0100, Frank Leonhardt wrote:
> Please note - I'm pushing what can be done with commodity hardware for
> amusement here, and trying to do interesting things with FreeBSD. I *do*
> want to do it this way.
> Okay, so the game is this: I've got some identical hosts with disks and
> I've been working on ways of clustering them for a long time. It's what
> I do for fun, right?
> I'm currently playing with failover storage. Not necessarily with HAST;
> iSCSI is fun and I'm messing with geom stuff in general. But let's stick
> with HAST as it illustrates the dilemma.
> When I started this game, 1Gb Ethernet was blistering. Now it's not so
> hot. How do I "network" the hosts with as much throughput as possible
> (with IP sockets, preferably)? Options:
> 1) 10Gb Ethernet is expensive. 10Gb switches even more so, but I can do
> 2) LAG is more about failover than speed. And anyway, you end up needing
> a lot of Ethernet ports on each host and it soon gets crazy.
> 3) Fibrechannel - if I bought a few old (cheap) fiberchannel cards, I'm
> not sure how I could use them point-to-point. Is this possible and has
> anyone done it?
> 4) USB 3.1. 10Gb. PCIe cards cost about $30. Now this sounds fun. IP
> over USB anyone?
> Now please don't advise me to get a pair of fabric switches and do the
> job properly. That's not what this is about. I want to see if it's
> possible to make a fast(er) storage solution using cheap components. A
> sort of Redundant Array of Inexpensive NAS.
> Thoughts anyone? In particular, is the USB 3.1 idea crazy? And is anyone
> else crazy enough to be trying the same thing?
I've set up a low-cost redundant storage here at work with ZFS + iSCSI,
basically a zpool mirror over 2 local disks and 2 iSCSI disks. Failover
is done through CARP and devd. It works well (except when Jumbo Frames
are turned on) for almost a year now. Just for extra security, and also
because some people (smarter than me in this area) advised me against
this approach, I've setup a third node with ZFS replication (zrep).
Basically if you can afford to lose "some" data, I would suggest to go
with ZFS replication.
> Thanks, Frank.
> freebsd-questions at freebsd.org mailing list
> To unsubscribe, send any mail to "freebsd-questions-unsubscribe at freebsd.org"
Belgian Biodiversity Platform (http://www.biodiversity.be)
PGP fingerprint: EEF9 F697 4B68 D275 7B11 6A25 B2BB 3710 A204 23C0
No trees were killed in the creation of this message.
However, many electrons were terribly inconvenienced.
-------------- next part --------------
A non-text attachment was scrubbed...
Size: 833 bytes
Desc: not available
More information about the freebsd-questions