Possible bug in NFSv4 with krb5p security?

[Rick Macklem]

Elias Martenson wrote:
> On 12 February 2013 23:20, Rick Macklem < rmacklem at uoguelph.ca >
> wrote:
> 
> 
> 
> 
> 
> 
> There is (in case you missed it on google):
> http://code.google.com/p/macnfsv4/wiki/FreeBSD8KerberizedNFSSetup
> (Nothing much has changed since FreeBSD8, except the name of the
> client
> side patch for host based initiator credentials in the keytab file.)
> I was hoping others would add/update the wiki and it would eventually
> become FreeBSD doc, but that hasn't happened.
> 
> 
> 
> 
> Thank you for the link. I have indeed found that, and I have followed
> it to the letter.
> 
> 
> I have up the exact same thing from Ubuntu machines as well as from
> Solaris, and I do have a fairly good understanding of Kerberos.
> FreeBSD however, is pretty new to me.
> 
> 
> Other than that, the various discussions in the archive on this list
> may help. Unfortunately, I don't know of an easy way to figure out
> what is busted. I always suggest looking at the packets in wireshark,
> but for some reason, I get the impression that folk don't like
> doing this? It is what I do first when I run into NFS issues.
> 
> 
> 
> I've been looking at the dumps using Wireshark. Well, I had to drop
> down the security since everything is encrypted when using keb5p. I do
> get the same errors using sec=krb5.
> 
> 
> When looking at this trace, I see a normal OPEN request followed by a
> NFS4ERR_ACCESS as a reply. The Kerberos credentials are of course
> encrypted, so I can't really say anything about that part.
> 
Well, it sounds like you are doing all the right stuff, so I don't know
why it is returning EACCES?

I'm not a ZFS person, so I never test that. If you have a UFS file system
you could export for testing, that might be worth a try. ZFS likes to do
things differently;-)

You can look at the authentication stuff in the RPC header:
Actually, the credentials in the RPC header aren't encrypted, although
they are binary data. It's been a while since I looked at the RFC, but
the authenticator is basically:
- an RPCSEC_GSS version (must be 1 for FreeBSD to be able to use it)
- a type that will be DATA in this case
- a credential handle (just a blob of bits the server uses as a shorthand
    for the principal)
- a sequence# used to subvert replay attempts

Then the authentication verifier is an encrypted checksum of the above,
that the server uses to verify it.

All the Kerberos stuff happens via NFS Null RPCs, where the GSSAPI tokens
are passed as data (a Null RPC has no arguments) and the credential handle
and a session key get established. (The Kerberos ticket is inside the GSSAPI
token for the first of these Null RPCs.)

> 
> Note that NFS4 with Kerberos security never uses the user ID numbers.
> They purely use the Kerberos principals for authorisation. This is
> different from the default "sys" security model that blindly uses user
> ID's.
> 
Yep, of course. The Kerberos user principal "name at REALM" is translated
to a uid + gid list by the gssd via a lookup of "name" as the username.
The uid + gid list is then associated with that credential handle I mentioned
above.

> 
> 
> 
> > nfscbd_enable="YES"
> Needed for client side only, and only if delegations are
> enabled at the server and you want the client to acquire
> delegations. (Delegations are not enabled by default on the
> FreeBSD NFSv4 server.)
> 
> 
> 
> Noted. Thank you. I will change this.
> 
> 
> 
> > rpc_lockd_enable="YES"
> > rpc_statd_enable="YES"
> You shouldn't need rpc_lockd or rpc_statd for NFSv4,
> since they are only used for NFSv3.
> 
> 
> 
> Good point. I'll disable those too.
> 
> 
> 
> I don't know why a Linux client would mix NFSv3 RPCs with NFSv4 ones,
> 
> 
> I was suggested to set vfs.nfsd.server_min_nfsvers to 4 in order to
> completely disable NFS version below 4. I did this and got rid of the
> stray NFS3 requests. It didn't solve the original problem though.
> 
> 
> 
> > If anyone is able to confirm whether or not this actually has been
> > tested
> > in 9.1-CURRENT, I'd appreciate it. Also, if not, then I'd love to
> > know
> > where I should start looking for a solution. I'm experienced in
> > system
> > level programming (having worked on Solaris at Sun in a previous
> > life), but
> > a pointer where to start would be helpful.
> >
> Usually, when everything is being done by "nobody", it indicates that
> the mapping between <uid, gids> <-> name at domain isn't working
> correctly.
> (Looking at the packets in wireshark, you need to look at the
> attributes
> called Owner and Owner_Group to see what they are being set to.)
> 
> 
> 
> I actually doubt this. First of all, I have the correct idmapd setup
> working from FreeBSD to Ubuntu (I can see that since I can see the
> correct user names in "ls" even though the user ID's differ). On OSX I
> haven't got it to work yet.
> 
> 
> But, the behaviour is the same on both systems.
> 
> 
> This is actually expected, as the permission checks are orthogonal to
> the ID mapping.
> 
> 
> The most common problem (since you do have nfsuserd running on the
> server)
> is for the "domain" spec to be different between client and server.
> FreeBSD's nfsuserd defaults to the domain part of the machine's
> hostname.
> Linux's rpc.idmapd sets the domain from /etc/idmapd.conf (at least I
> think
> that's what it is called) and many distros ship with it set to
> "my.domain"
> or something like that.
> 
> 
> 
> Correct. I have set this correctly. I know this, since once I did,
> "ls" started giving me the correct user names.
> 
> 
> As such, I'd start by checking the Linux client and seeing what it has
> for
> the domain spec. in /etc/idmapd.conf.
> 
> If you want to override the default for FreeBSD, there is a command
> line
> option for nfsuserd to do this. I can't remember what it is, but "man
> nfsuserd"
> will give you the answer and it can be set in /etc/rc.conf using
> nfsuserd_flags.
> 
> 
> 
> Thank you. I'm definitely willing to double-check this and I am not
> going to claim to know exactly what's going on in the realms of NFS
> security. :-)
> 
> 
> If this is configured correctly, then looking at the packets in
> wireshark is
> the best starting point w.r.t. figuring out what is broken. I do
> limited
> testing of this and it works for me. I don't know how many others use
> it,
> although some definitely have fun getting it working. (Usually it is
> the
> Kerberos part on the client side that causes the most grief.)
> 
> 
> 
> It certainly is fun. But it gets frustrating when one fights a single
> problem for weeks on end.
> 
> 
> Far too few shops use Kerberos though.
> 
Or maybe too many;-) A public key system might not be as secure, but it
would be far easier to manage imho.

> 
> 
> Regards,
> Elias