pNFS server Plan B

Willem Jan Withagen wjw at digiware.nl
Fri Jun 24 08:31:54 UTC 2016


On 24-6-2016 09:35, Jordan Hubbard wrote:
> 
>> On Jun 22, 2016, at 1:56 AM, Willem Jan Withagen <wjw at digiware.nl>
>> wrote:
>> 
>> In the spare time I have left, I'm trying to get a lot of small
>> fixes into the ceph tree to get it actually compiling, testing, and
>> running on FreeBSD. But Ceph is a lot of code, and since a lot of
>> people are working on it, the number of code changes are big.
> 
> Hi Willem,
> 
> Yes, I read your paper on the porting effort!
> 
> I also took a look at porting ceph myself, about 2 years ago, and
> rapidly concluded that it wasn’t a small / trivial effort by any
> means and would require a strong justification in terms of ceph’s
> feature set over glusterfs / moose / OpenAFS / RiakCS / etc.   Since
> that time, there’s been customer interest but nothing truly “strong”
> per-se.  

I've been going at it since last November... And all I go in are about 3
batches of FreeBSD specific commits. Lots has to do with release windows
and code slush, like we know on FreeBSD. But then still reviews tend to
slow and I need people to push to look at them. Whilst in the mean time
all kinds of thing get pulled and inserted in the tree, that seriously
are not FreeBSD. Sometimes I see them during commit, and "negotiate"
better comparability with the author. At other times I missed the whole
thing, and I need to rebase to get ride of merge conflicts. To find out
the hard way that somebody has made the whole
peer communication async. And has thrown kqueue for the BSDs at it. But
they don't work (yet). So to get my other patches in, if First need to
fix this. Takes a lot of time .....

That all said I was in Geneva and a lot of the Ceph people were there
including Sage Weil. And I go the feeling they appreciated a larger
community. I think they see what ZFS has done with OpenZFS and see that
communities get somewhere.

Now on of the things to do to continue, now that I sort of can compile
and run the first testset, is set up sort of my own Jenkins stuff. So
that I can at least test drive some of the tree automagically to get
some testcoverage of the code on FreeBSD. In my mind (and Sage warned me
that that will be more or less required) it is the only way to actually
get a serious foot in the door with the Ceph guys.

> My attraction to ceph remains centered around at least these
> 4 things:
> 
> 1. Distributed Object store with S3-compatible ReST API 
> 2. Interoperates with Openstack via Swift compatibility 
> 3. Block storage > (RADOS) - possibly useful for iSCSI and other block storage
> requirements 
> 4. Filesystem interface
> 
> Is there anything we can do to help?  

I'll get back on that in a separate Email.

> Do the CEPH folks seem
> receptive to actually having a “Tier 1” FreeBSD port?  I know that
> stas@ did an early almost-port awhile back, but it never reached
> fruition and my feeling was that they (ceph) might be a little
> gun-shy about seeing another port that might wind up in the same
> place, crufting up their code base to no purpose.  

Well, as you know, I for the era before there was automake....
So then porting was still very much an art. So I've been balancing
between crufting up the code, and hiding things nice and cleanly in C++
classes and place. And as an go inbetween stuff get stuck in compat.h.

One of my slides was actually about the impact of foreign code in the
tree. And uptill now that is relatively minimal. Which seemed to please
a lot of the folks. But they also like the idee that getting FreeBSD
stuff in actually showed code weakness (and fixes) in the odd corners.

> Do you have any
> initial impressions about that?  I’ve never talked to any of the 3
> principle guys working on the project and this is pure guesswork on
> my part.

I think they are going their own path, like writting their own datastore
so they can do things they require that posix can't deliver.
And as such are also diverging from what is default on Linux.

The systemarchitect in me also sees things in pain happen, because of
the "reinvention" of things. But then again, that happens with projects
this big. Things like checksums, compression, encryption, ....
Lots of stuff I've seen happen to ZFS over its time.
But so be it, everybody gets to chose their own axes to grind.

The community person to talk to is perhaps Patrick McGarry, but even
Sage would be good to talk to.

--WjW


More information about the freebsd-fs mailing list