CEPH + FreeBSD

[Outback Dingo]

On Wed, Sep 9, 2015 at 11:31 AM, Mark Saad <nonesuch at longcount.org> wrote:

> All
>  What about leofs. It's in ports has  and s3 obj store and NFS support out
> of the box
>
>
> http://www.freshports.org/databases/leofs/
> http://leo-project.net


LeoFS supperts NFSv3 and does not have a lock manager....



>
>
>
> ---
> Mark Saad | nonesuch at longcount.org
>
> > On Sep 6, 2015, at 6:18 PM, Rick Macklem <rmacklem at uoguelph.ca> wrote:
> >
> > Jordan Hubbard wrote:
> >>
> >>> On Sep 3, 2015, at 4:17 AM, Rick Macklem <rmacklem at uoguelph.ca> wrote:
> >>>
> >>> Slightly off topic but, btw, there is a port of GLusterFS and those
> folks
> >>> do seem
> >>> interested in seeing it brought "up to speed". I am not sure how
> mature it
> >>> is at
> >>> this point, but it has been known to build on amd64. (I don't have an
> amd64
> >>> machine,
> >>> so I haven't gotten around to building/testing it, but I do plan to
> try and
> >>> use
> >>> it as a basis for a pNFS server, if I can figure out how to get the FH
> info
> >>> out of it.
> >>> I'm working on that;-)
> >>
> >> There are at least two distributed (multi-node) object stores for
> FreeBSD
> >> that I know of.
> >>
> >> One is glusterfs, for which I’m not even really clear on the status of
> the
> >> ports for.  I don’t see any glusterfs port in the master branch of
> >> https://github.com/freebsd/freebsd-ports (or
> >> https://github.com/freebsd/freebsd-ports/tree/branches/2015Q3 for that
> >> matter).
> >>
> >> Our FreeNAS ports tree (https://github.com/freenas/ports), in which we
> have a
> >> bit more latitude to add and curate our own ports, has both a
> net/glusterfs
> >> and sysutils/glusterfs, from separate sources (looks like we need to
> clean
> >> things up) - net/glusterfs lists craig001 at lerwick.hopto.org as the
> >> MAINTAINER and is at version 3.6.2.  The sysutils/glusterfs port lists
> >> bapt at FreeBSD.org as the MAINTAINER and is at version 20140811.
> >>
> >> I’m not really sure about the provenance since we were simply evaluating
> >> glusterfs for awhile and may have pulled in interim versions from those
> >> sources, but obviously it would be best to have an official maintainer
> and
> >> someone in the FreeBSD project actually curating a glusterfs port so
> that
> >> all users of FreeBSD can use it.  It would also be fairly key to your
> own
> >> efforts, assuming you decide to pursue glusterfs as a foundation
> technology
> >> for pNFS.
> >>
> >> The other object store, which is pretty mature and is currently leading
> the
> >> pack (of two :) ) for inclusion into FreeNAS is RiakCS from Basho.
> There is
> >> a port currently in databases/riak but it’s pretty out of date at
> version
> >> 1.4.12 (the current version is 2.0.1, with 2.0 being a major upgrade of
> >> RiakCS).
> >>
> >> We are very interested in investigating various ways of shimming RiakCS
> to
> >> NFS, using RiakCS a back-end store.   Is that something you’d be
> amenable to
> >> discussing?   I’d be happy to send you an amd64 architecture machine to
> >> develop on. :)
> > Hmm. From a quick look at their web page (I looked once before as well),
> I don't
> > think RiakCS has what I need to do pNFS in a reasonable (for me) amount
> of effort.
> > Two things that glusterFS has that I am hoping to use (and I don't think
> RiakCS has
> > either of these) are:
> > - A Fuse file system interface which allows the kernel nfsd to access
> the store as
> >  a file system, so that it can provide the metadata services (NFS
> without the reads/writes).
> > - A userland NFSv3 server in each node which will allow the node to act
> as a data server.
> >
> > If I am wrong and RiakCS does support a VFS file system interface (via
> Fuse or ???), then
> > please correct me. With that, it might be a reasonable alternative.
> > I'll admit I've spent a little time looking at the glusterFS sources and
> haven't yet
> > solved the problem of how to generate the file handles I need, but that
> sounds trivial
> > compared with an entire Fuse and/or VFS file system interface, I think?
> >
> > In general, using a cloud object store to implement a pNFS server is a
> *mis*use of
> > the technology, imho. I think it may be possible with glusterFS, since
> that technology
> > seems to be based on a cluster file system, which is what a pNFS server
> can also use.
> >
> > I think there would be a lot of work involved in mapping a POSIX file
> system onto the
> > Riak database and then exporting that via NFS, etc. It might also be
> more practical to
> > do this via a userland NFS service than the kernel based one currently
> in FreeBSD.
> > (glusterFS is starting to use the NFS-ganesha server, but I believe it
> is pretty Linux specific,
> > so I doubt it would be useful for Riak running on FreeBSD?)
> >
> > rick
> >
> >> - Jordan
> > _______________________________________________
> > freebsd-fs at freebsd.org mailing list
> > https://lists.freebsd.org/mailman/listinfo/freebsd-fs
> > To unsubscribe, send any mail to "freebsd-fs-unsubscribe at freebsd.org"
> _______________________________________________
> freebsd-fs at freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-fs
> To unsubscribe, send any mail to "freebsd-fs-unsubscribe at freebsd.org"
>