Fun with HAST and inter-host connections
julien at perdition.city
Thu Aug 31 15:34:17 UTC 2017
On Thu, Aug 31, 2017 at 01:33:30PM +0200, Julien Cigar wrote:
> On Wed, Aug 30, 2017 at 10:40:03PM +0100, Frank Leonhardt wrote:
> > Please note - I'm pushing what can be done with commodity hardware for
> > amusement here, and trying to do interesting things with FreeBSD. I *do*
> > want to do it this way.
> > Okay, so the game is this: I've got some identical hosts with disks and
> > I've been working on ways of clustering them for a long time. It's what
> > I do for fun, right?
> > I'm currently playing with failover storage. Not necessarily with HAST;
> > iSCSI is fun and I'm messing with geom stuff in general. But let's stick
> > with HAST as it illustrates the dilemma.
> > When I started this game, 1Gb Ethernet was blistering. Now it's not so
> > hot. How do I "network" the hosts with as much throughput as possible
> > (with IP sockets, preferably)? Options:
> > 1) 10Gb Ethernet is expensive. 10Gb switches even more so, but I can do
> > point-to-point.
> > 2) LAG is more about failover than speed. And anyway, you end up needing
> > a lot of Ethernet ports on each host and it soon gets crazy.
> > 3) Fibrechannel - if I bought a few old (cheap) fiberchannel cards, I'm
> > not sure how I could use them point-to-point. Is this possible and has
> > anyone done it?
> > 4) USB 3.1. 10Gb. PCIe cards cost about $30. Now this sounds fun. IP
> > over USB anyone?
> > Now please don't advise me to get a pair of fabric switches and do the
> > job properly. That's not what this is about. I want to see if it's
> > possible to make a fast(er) storage solution using cheap components. A
> > sort of Redundant Array of Inexpensive NAS.
> > Thoughts anyone? In particular, is the USB 3.1 idea crazy? And is anyone
> > else crazy enough to be trying the same thing?
> I've set up a low-cost redundant storage here at work with ZFS + iSCSI,
> basically a zpool mirror over 2 local disks and 2 iSCSI disks. Failover
> is done through CARP and devd. It works well (except when Jumbo Frames
> are turned on) for almost a year now. Just for extra security, and also
> because some people (smarter than me in this area) advised me against
> this approach, I've setup a third node with ZFS replication (zrep).
> Basically if you can afford to lose "some" data, I would suggest to go
> with ZFS replication.
(of course by "ZFS replication" I mean snapshot and send/receive)
> > Thanks, Frank.
> > _______________________________________________
> > freebsd-questions at freebsd.org mailing list
> > https://lists.freebsd.org/mailman/listinfo/freebsd-questions
> > To unsubscribe, send any mail to "freebsd-questions-unsubscribe at freebsd.org"
> Julien Cigar
> Belgian Biodiversity Platform (http://www.biodiversity.be)
> PGP fingerprint: EEF9 F697 4B68 D275 7B11 6A25 B2BB 3710 A204 23C0
> No trees were killed in the creation of this message.
> However, many electrons were terribly inconvenienced.
Belgian Biodiversity Platform (http://www.biodiversity.be)
PGP fingerprint: EEF9 F697 4B68 D275 7B11 6A25 B2BB 3710 A204 23C0
No trees were killed in the creation of this message.
However, many electrons were terribly inconvenienced.
-------------- next part --------------
A non-text attachment was scrubbed...
Size: 833 bytes
Desc: not available
More information about the freebsd-questions