CEPH + FreeBSD

[Mark Saad]

All
 What about leofs. It's in ports has  and s3 obj store and NFS support out of the box


http://www.freshports.org/databases/leofs/
http://leo-project.net


---
Mark Saad | nonesuch at longcount.org

> On Sep 6, 2015, at 6:18 PM, Rick Macklem <rmacklem at uoguelph.ca> wrote:
> 
> Jordan Hubbard wrote:
>> 
>>> On Sep 3, 2015, at 4:17 AM, Rick Macklem <rmacklem at uoguelph.ca> wrote:
>>> 
>>> Slightly off topic but, btw, there is a port of GLusterFS and those folks
>>> do seem
>>> interested in seeing it brought "up to speed". I am not sure how mature it
>>> is at
>>> this point, but it has been known to build on amd64. (I don't have an amd64
>>> machine,
>>> so I haven't gotten around to building/testing it, but I do plan to try and
>>> use
>>> it as a basis for a pNFS server, if I can figure out how to get the FH info
>>> out of it.
>>> I'm working on that;-)
>> 
>> There are at least two distributed (multi-node) object stores for FreeBSD
>> that I know of.
>> 
>> One is glusterfs, for which I’m not even really clear on the status of the
>> ports for.  I don’t see any glusterfs port in the master branch of
>> https://github.com/freebsd/freebsd-ports (or
>> https://github.com/freebsd/freebsd-ports/tree/branches/2015Q3 for that
>> matter).
>> 
>> Our FreeNAS ports tree (https://github.com/freenas/ports), in which we have a
>> bit more latitude to add and curate our own ports, has both a net/glusterfs
>> and sysutils/glusterfs, from separate sources (looks like we need to clean
>> things up) - net/glusterfs lists craig001 at lerwick.hopto.org as the
>> MAINTAINER and is at version 3.6.2.  The sysutils/glusterfs port lists
>> bapt at FreeBSD.org as the MAINTAINER and is at version 20140811.
>> 
>> I’m not really sure about the provenance since we were simply evaluating
>> glusterfs for awhile and may have pulled in interim versions from those
>> sources, but obviously it would be best to have an official maintainer and
>> someone in the FreeBSD project actually curating a glusterfs port so that
>> all users of FreeBSD can use it.  It would also be fairly key to your own
>> efforts, assuming you decide to pursue glusterfs as a foundation technology
>> for pNFS.
>> 
>> The other object store, which is pretty mature and is currently leading the
>> pack (of two :) ) for inclusion into FreeNAS is RiakCS from Basho.  There is
>> a port currently in databases/riak but it’s pretty out of date at version
>> 1.4.12 (the current version is 2.0.1, with 2.0 being a major upgrade of
>> RiakCS).
>> 
>> We are very interested in investigating various ways of shimming RiakCS to
>> NFS, using RiakCS a back-end store.   Is that something you’d be amenable to
>> discussing?   I’d be happy to send you an amd64 architecture machine to
>> develop on. :)
> Hmm. From a quick look at their web page (I looked once before as well), I don't
> think RiakCS has what I need to do pNFS in a reasonable (for me) amount of effort.
> Two things that glusterFS has that I am hoping to use (and I don't think RiakCS has
> either of these) are:
> - A Fuse file system interface which allows the kernel nfsd to access the store as
>  a file system, so that it can provide the metadata services (NFS without the reads/writes).
> - A userland NFSv3 server in each node which will allow the node to act as a data server.
> 
> If I am wrong and RiakCS does support a VFS file system interface (via Fuse or ???), then
> please correct me. With that, it might be a reasonable alternative.
> I'll admit I've spent a little time looking at the glusterFS sources and haven't yet
> solved the problem of how to generate the file handles I need, but that sounds trivial
> compared with an entire Fuse and/or VFS file system interface, I think?
> 
> In general, using a cloud object store to implement a pNFS server is a *mis*use of
> the technology, imho. I think it may be possible with glusterFS, since that technology
> seems to be based on a cluster file system, which is what a pNFS server can also use.
> 
> I think there would be a lot of work involved in mapping a POSIX file system onto the
> Riak database and then exporting that via NFS, etc. It might also be more practical to
> do this via a userland NFS service than the kernel based one currently in FreeBSD.
> (glusterFS is starting to use the NFS-ganesha server, but I believe it is pretty Linux specific,
> so I doubt it would be useful for Riak running on FreeBSD?)
> 
> rick
> 
>> - Jordan
> _______________________________________________
> freebsd-fs at freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-fs
> To unsubscribe, send any mail to "freebsd-fs-unsubscribe at freebsd.org"