CEPH + FreeBSD

[Jordan Hubbard]

> On Sep 8, 2015, at 7:29 PM, Rakshith Venkatesh <vrock28 at gmail.com> wrote:
> 
> If there is a need to support cifs along with NFS then locking surely is
> needed to maintain data sanity and avoid problems due to concurrent access
> to files.

Well, sort of.  The file sharing semantics between NFS and CIFS are so different in not just in how they lock files, but in the Windows vs Unix permission models, that it’s essentially lunatic to attempt to share the same ZFS dataset to both NFS and Windows clients.

FreeNAS does not even allow you to do that - you pick a dataset for CIFS sharing and it applies a completely different set of default ACLs and even goes so far as to disable Unix permission changes (chown/chmod) completely so that clueless system admins don’t do local Unix permission operations on the files and end up blowing certainly critical (to Windows) ACLs off, rendering the share unusable to Windows clients.  It happened so often that we ultimately had to lay down the law and explicitly disallow that scenario.

In the past, Samba3 also used to essentially lie about a lot of the Unix permissions to Windows clients in order to provide some semblance of interoperability, but that ended up causing more problems than it solved (it’s better to be predictable than magical) and in Samba4, it’s far more inclined to simply tell the truth, which also tripped up a lot of sysadmins who were used to it lying before.  Once they managed to unscramble their permissions and actually leverage ACLs to the full extent they were meant to be used, they learned what the Samba team already knew:  NFS ACLs != Windows ACLs.

Or, as they said it best in Ghostbusters:

Egon Spengler: There's something very important I forgot to tell you.
Peter Venkman: What?
Spengler: Don't cross the streams.
Venkman: Why?
Spengler: It would be bad.
Venkman: I'm fuzzy on the whole good/bad thing. What do you mean, "bad"?
Spengler: Try to imagine all life as you know it stopping instantaneously and every molecule in your body exploding at the speed of light.
Ray Stantz: Total protonic reversal!
Venkman: Right. That's bad. Okay. All right. Important safety tip. Thanks, Egon.

So yeah, the dream of sharing CIFS and NFS from the same dataset is probably never going to be a reality.  I mean, it’s *technically possible* just like all things are technically possible, but you’d have to come up with some sort of underlying “Universal permissions and locking model” that could then be mapped and translated upwards to both Unix and Windows clients such that each saw exactly what they wanted and needed to see, and then you’d need to commit to maintaining that model across multiple versions of Windows (in which the semantics have changed in subtle ways over time) and Unix.  The Samba folks really tried to fake that up in userland and as near as I can figure out, they eventually realized it was just a terrible place to be and stopped trying to force incompatible species to live together, which is what we also decided.  Yeah, it kind of sucks not being able to share exactly the same files for R/W access to both Windows and Unix clients simultaneously.  Deal with it.  Life is cruel.  :-)

JFYI,

- Jordan