CEPH + FreeBSD

Wed Sep 9 12:28:44 UTC 2015

Rakshith Venkatesh wrote:
> If there is a need to support cifs along with NFS then locking surely is
> needed to maintain data sanity and avoid problems due to concurrent access
> to files.
For my case, this doesn't affect choice of cluster file system, since the locking
is taken care of by the NFSv4.1 server. You are, however, correct that it would be really
nice to have the NFSv4.1 server working more closely with Samba (or other open source CIFS
implementations) for the Windows op locks, etc.
Right now, the NFSv4.1 server can optionally grab POSIX style advisory locks on the underlying
file system so that Samba (and local syscalls) can see them.
It does nothing w.r.t. the Opens (which are basically Windows op locks, or whatever Samba calls them?

I'll admit I've avoided this, since I know nothing about Samba. (I'm only one retired guy doing this
in his spare time.) Soneday, it would be nice if someone who knows the Samba side could work on this?

rick

> On 09-Sep-2015 7:55 am, "Mark Saad" <nonesuch at longcount.org> wrote:
> 
> >
> > On Sep 8, 2015, at 9:59 PM, Outback Dingo <outbackdingo at gmail.com> wrote:
> >
> >
> >
> > On Wed, Sep 9, 2015 at 11:31 AM, Mark Saad <nonesuch at longcount.org> wrote:
> >
> >> All
> >>  What about leofs. It's in ports has  and s3 obj store and NFS support
> >> out of the box
> >>
> >>
> >> http://www.freshports.org/databases/leofs/
> >> http://leo-project.net
> >
> >
> > LeoFS supperts NFSv3 and does not have a lock manager....
> >
> >
> > Well, nlm it's not a hard requirement for most NFS things . NFS is there
> > for direct access of the data . So in its current form it's not a pnfs
> > setup , but there is no reason it could not be extended .
> >
> >  Well it could be worse , manta, joyent's compute and storage platform has
> > a NFS server written in node.js . Yes nfs in JavaScript , what else would
> > you expect from them .  :)
> >
> >
> > ---
> > Mark Saad | nonesuch at longcount.org
> >
> >
> >
> >>
> >>
> >>
> >> ---
> >> Mark Saad | nonesuch at longcount.org
> >>
> >> > On Sep 6, 2015, at 6:18 PM, Rick Macklem <rmacklem at uoguelph.ca> wrote:
> >> >
> >> > Jordan Hubbard wrote:
> >> >>
> >> >>> On Sep 3, 2015, at 4:17 AM, Rick Macklem <rmacklem at uoguelph.ca>
> >> wrote:
> >> >>>
> >> >>> Slightly off topic but, btw, there is a port of GLusterFS and those
> >> folks
> >> >>> do seem
> >> >>> interested in seeing it brought "up to speed". I am not sure how
> >> mature it
> >> >>> is at
> >> >>> this point, but it has been known to build on amd64. (I don't have an
> >> amd64
> >> >>> machine,
> >> >>> so I haven't gotten around to building/testing it, but I do plan to
> >> try and
> >> >>> use
> >> >>> it as a basis for a pNFS server, if I can figure out how to get the
> >> FH info
> >> >>> out of it.
> >> >>> I'm working on that;-)
> >> >>
> >> >> There are at least two distributed (multi-node) object stores for
> >> FreeBSD
> >> >> that I know of.
> >> >>
> >> >> One is glusterfs, for which I’m not even really clear on the status of
> >> the
> >> >> ports for.  I don’t see any glusterfs port in the master branch of
> >> >> https://github.com/freebsd/freebsd-ports (or
> >> >> https://github.com/freebsd/freebsd-ports/tree/branches/2015Q3 for that
> >> >> matter).
> >> >>
> >> >> Our FreeNAS ports tree (https://github.com/freenas/ports), in which
> >> we have a
> >> >> bit more latitude to add and curate our own ports, has both a
> >> net/glusterfs
> >> >> and sysutils/glusterfs, from separate sources (looks like we need to
> >> clean
> >> >> things up) - net/glusterfs lists craig001 at lerwick.hopto.org as the
> >> >> MAINTAINER and is at version 3.6.2.  The sysutils/glusterfs port lists
> >> >> bapt at FreeBSD.org as the MAINTAINER and is at version 20140811.
> >> >>
> >> >> I’m not really sure about the provenance since we were simply
> >> evaluating
> >> >> glusterfs for awhile and may have pulled in interim versions from those
> >> >> sources, but obviously it would be best to have an official maintainer
> >> and
> >> >> someone in the FreeBSD project actually curating a glusterfs port so
> >> that
> >> >> all users of FreeBSD can use it.  It would also be fairly key to your
> >> own
> >> >> efforts, assuming you decide to pursue glusterfs as a foundation
> >> technology
> >> >> for pNFS.
> >> >>
> >> >> The other object store, which is pretty mature and is currently
> >> leading the
> >> >> pack (of two :) ) for inclusion into FreeNAS is RiakCS from Basho.
> >> There is
> >> >> a port currently in databases/riak but it’s pretty out of date at
> >> version
> >> >> 1.4.12 (the current version is 2.0.1, with 2.0 being a major upgrade of
> >> >> RiakCS).
> >> >>
> >> >> We are very interested in investigating various ways of shimming
> >> RiakCS to
> >> >> NFS, using RiakCS a back-end store.   Is that something you’d be
> >> amenable to
> >> >> discussing?   I’d be happy to send you an amd64 architecture machine to
> >> >> develop on. :)
> >> > Hmm. From a quick look at their web page (I looked once before as
> >> well), I don't
> >> > think RiakCS has what I need to do pNFS in a reasonable (for me) amount
> >> of effort.
> >> > Two things that glusterFS has that I am hoping to use (and I don't
> >> think RiakCS has
> >> > either of these) are:
> >> > - A Fuse file system interface which allows the kernel nfsd to access
> >> the store as
> >> >  a file system, so that it can provide the metadata services (NFS
> >> without the reads/writes).
> >> > - A userland NFSv3 server in each node which will allow the node to act
> >> as a data server.
> >> >
> >> > If I am wrong and RiakCS does support a VFS file system interface (via
> >> Fuse or ???), then
> >> > please correct me. With that, it might be a reasonable alternative.
> >> > I'll admit I've spent a little time looking at the glusterFS sources
> >> and haven't yet
> >> > solved the problem of how to generate the file handles I need, but that
> >> sounds trivial
> >> > compared with an entire Fuse and/or VFS file system interface, I think?
> >> >
> >> > In general, using a cloud object store to implement a pNFS server is a
> >> *mis*use of
> >> > the technology, imho. I think it may be possible with glusterFS, since
> >> that technology
> >> > seems to be based on a cluster file system, which is what a pNFS server
> >> can also use.
> >> >
> >> > I think there would be a lot of work involved in mapping a POSIX file
> >> system onto the
> >> > Riak database and then exporting that via NFS, etc. It might also be
> >> more practical to
> >> > do this via a userland NFS service than the kernel based one currently
> >> in FreeBSD.
> >> > (glusterFS is starting to use the NFS-ganesha server, but I believe it
> >> is pretty Linux specific,
> >> > so I doubt it would be useful for Riak running on FreeBSD?)
> >> >
> >> > rick
> >> >
> >> >> - Jordan
> >> > _______________________________________________
> >> > freebsd-fs at freebsd.org mailing list
> >> > https://lists.freebsd.org/mailman/listinfo/freebsd-fs
> >> > To unsubscribe, send any mail to "freebsd-fs-unsubscribe at freebsd.org"
> >> _______________________________________________
> >> freebsd-fs at freebsd.org mailing list
> >> https://lists.freebsd.org/mailman/listinfo/freebsd-fs
> >> To unsubscribe, send any mail to "freebsd-fs-unsubscribe at freebsd.org"
> >>
> >
> >
> 