pNFS server Plan B

[Rick Macklem]

Jordan Hubbard wrote:
> OK, wow.  This appears to have turned into something of a referendum on NFS
> and, just based on Rick and Doug’s defense of pNFS, I also think my
> commentary on that may have been misconstrued somewhat.
> 
Actually, I thought it had become a referendum on LDAP;-)

As for defending pNFS, all I was trying to say was that "although it is hard
to believe, it has taken 10years for pNFS to hit the streets". As such, it
is anyone's guess w.r.t. whether or not it will become widely adopted?
If it came across as more than that, I am the one that should be apologizing
and am in no way discouraged by any of the comments.

> So, let me just set the record straight by saying that I’m all in favor of
> pNFS.  It addresses a very definite need in the Enterprise marketplace and
> gives FreeBSD yet another arrow in its quiver when it comes to being “a
> player” in that (ever-growing) arena.  The only point I was trying to make
> before was that if we could ALSO address clustering in a more general way as
> part of providing a pNFS solution, that would be great.
When I did a fairly superficial evaluation of the open source clustering systems
out there (looking at online doc and not actually their code), it seemed that
GlusterFS was the best bet for "one size fits all".
It had:
- a distributed file system (replication, etc) with a POSIX/FUSE interface.
- SwiftOnFile that put the Swift/Openstack on top of this.
- It had decentralized metadata handling.
For pNFS:
- It had a NFSv3 server built into it.
- Was ported to FreeBSD.

The others were:
- Object store only with no POSIX file system support
or
- Single centralized metadata store (MooseFS, for example)
- No FreeBSD port and rumoured to be hard to port (Ceph, Lustre are two examples).

Now that I've worked with GlusterFS a little bit, I am skeptical that it can
deliver adequate performance for pNFS using the nfsd. I am still hoping I will
be proven wrong on this, but???

A GlusterFS/Ganesha-NFS user space solution may be feasible. This is what the
GlusterFS folk are planning. However, for FreeBSD...
- Ganesha-NFS apparently was ported to FreeBSD, but the port was removed from
  their source tree and it is said it now uses Linux-specific thread primitives.
  --> As such, I have no idea what effort is involved in getting this ported and
      working well on FreeBSD is.
- I would also wait until this is working in Linux and would want to do an
  evaluation of that, to make sure it actually works/performs well, before
  considering this.
*** For me personally, I am probably not interested in working on this. I
    know the FreeBSD nfsd kernel code well and can easily work with that,
    but Ganesha-NFS would be an entirely different beast.

Bottom line, at this point I am skeptical that a generic clustering system
will work for pNFS.

>   I am not, however,
> the one writing the code and if my comments were in any way discouraging to
> the folks that are, I apologize and want to express my enthusiasm for it.
> If iXsystems engineering resources can contribute in any way to moving this
> ball forward, let me know and we’ll start doing so.
> 
Well, although they may not be useful for building a pNFS server, but some sort
of evaluation of the open source clustering systems might be useful.
Sooner or later, the Enterprise marketplace may want one or more of these and
it seems to me that having one of these layered on top of ZFS may be an attractive
solution.
- Some will never be ported to FreeBSD, but the ones that are could probably be
  evaluated fairly easily, if you have the resources.

Since almost all the code I've written gets reused if I do a PlanB, I will
probably pursue that, leaving the GlusterFS interface bits in place in case
they are useful.

Thanks for all the interesting comments, rick

> On the more general point of “NFS is hard, let’s go shopping” let me also say
> that it’s kind of important not to conflate end-user targeted solutions with
> enterprise solutions.  Setting up a Kerberized NFSv4, for example, is not
> really designed to be trivial to set up and if anyone is waiting for that to
> happen, they may be waiting a very long time (like, forever).  NFS and SMB
> are both fairly simple technologies to use if you restrict yourself to
> using, say, just 20% of their overall feature-sets.  Once you add ACLs,
> Directory Services, user/group and permissions mappings, and any of the
> other more enterprise-centric features of these filesharing technologies,
> however, things rapidly get more complicated and the DevOps people who
> routinely play in these kinds of environments are quite happy to have all
> those options available because they’re not consumers operating in consumer
> environments.
> 
> Sun didn’t design NFS to be particularly consumer-centric, for that matter,
> and if you think SMB is “simple” because you clicked Network on Windows
> Explorer one day and stuff just automagically appeared, you should try
> operating it in a serious Windows Enterprise environment (just flip through
> some of the SMB bugs in the FreeNAS bug tracker -
> https://bugs.freenas.org/projects/freenas/issues?utf8=✓&set_filter=1&f%5B%5D=status_id&op%5Bstatus_id%5D=*&f%5B%5D=category_id&op%5Bcategory_id%5D=%3D&v%5Bcategory_id%5D%5B%5D=57&f%5B%5D=&c%5B%5D=tracker&c%5B%5D=status&c%5B%5D=priority&c%5B%5D=subject&c%5B%5D=assigned_to&c%5B%5D=updated_on&c%5B%5D=fixed_version&group_by=
> - if you want to see the kinds of problems users wrestle with all the time).
> 
> Anyway, I’ll get off the soapbox now, I just wanted to dispute the premise
> that “simple file sharing” that is also “secure file sharing” and “flexible
> file sharing” doesn’t really exist.  The simplest end-user oriented file
> sharing system I’ve used to date is probably AFP, and Apple has been trying
> to kill it for years, probably because it doesn’t have all those extra knobs
> and Kerberos / Directory Services integration business users have been
> asking for (it’s also not particularly industry standard).
> 
> - Jordan
> 
> 
> 
> _______________________________________________
> freebsd-fs at freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-fs
> To unsubscribe, send any mail to "freebsd-fs-unsubscribe at freebsd.org"