NFSv4 Kerberos mount from Linux

[Rick Macklem]

Felix Winterhalter wrote:
On 10/4/18 5:21 PM, Rick Macklem wrote:
[stuff snipped]
>>> I am now trying to mount this directory as root first without having to
>>> deal with user keytabs or tickets.
>>>
>>> This works fine with -sec=sys and nfsv4.1 and nfsv3 and -sec=krb5p.
>>> This does not however work with nfsv4 and krb5p or any other krb5 flavor.
>> Sorry, I'm not sure what you are saying here. Is it
>> 1 - no version of NFS works for krb5p or
>> 2 - NFSv4.1 works for krb5p, but NFSv4.0 does not or
>> 3 - only nfsv3 works for krb5p
[snipped lots of text]
>
>#3 is indeed what was happening. I could mount with krb5p for nfsv3
>(which I was not aware was even doable) however nfsv4 would stubbornly
>refuse to do any mounting.
Yes, RPCSEC_GSS was done by Sun for NFSv3 and it was a good fit, since NFSv3
does not have any server state to maintain. As such, all RPCs are atomic operations
done by users (which for Kerberized mounts must have a TGT in a credential cache).

NFSv4 wasn't really a good fit for the model, because the NFSv4 server maintains
lock state (NFSv4 Opens are a form of lock used by Windows at file open time).
There are "state maintenance" operations that must be done by the user doing
the mount (usually root), where they typically don't have a TGT in a credential
cache.
--> The ugly solution for this is typically a host-based client credential in a keytab
      on the client. (Usually a principal like "root/client-host.domain at REALM" or
      "host/client-host.domain at REALM" or "nfs/client-host.domain at REALM"
       in the default keytab on the client.)

>I have now after a lot of try and error figured out what I need to do in
>order to make it work.
>
>To start with I have kerberos credentials with both host/ and nfs/ on
>both client and server. Mounting nfsv4 shares with krb5p from a linux
>server has also worked in this context.
Yes, I'm assuming that satisfied the host-based client credential as I described
above.

>I leave you to judge whether what I found out is intended behaviour or
>if something weird is going on.
Yes, sounds like intended behaviour, since the client must have a Kerberos
credential to use for the "state maintenance" operations that are not done on
behalf of a user.

>My exports file originally looked something like this:
>
>/nfsTests/ /nfsTests/testexport /nfsTests/otherexport -maproot=root
>-sec=krb5p clients
>
>V4: /nfsTests -sec=krb5p clients
>
>Which allowed me to do nfsv3 krb5p mounts but not nfsv4 krb5p mounts.
>
>Changing the exports file to this:
>
>/nfsTests/ /nfsTests/testexport /nfsTests/otherexport -maproot=root
>-sec=krb5p clients
>
>V4: /nfsTests -sec=krb5p,krb5i clients
This suggests that there is a bug in the client, where it uses krb5i instead of krb5p
at some point in the mounting process. (I have also seen cases where the client
erroneously falls back on using sys at some point in the mounting process.)
(You did mention before you were using the Linux client. If you are using a FreeBSD
 client, I would be interested in looking at this.)

>Allows nfsv4 krb5p mounts to work for some reason I do not understand.
>Not setting the -sec option on the V4 line apparently defaults to
>-sec=sys and doesn't allow any krb5 mounts. I'm not sure that this is a
>good default as I wasn't even aware that the -sec option needed to be
>set on this line.
In FreeBSD, defaults are meant to maintain backwards compatibility. This means that
AUTH_SYS should work by default. Also, AUTH_SYS is what 99.9999% of FreeBSD
NFS mounts still use, from what I've seen.)

>I've got packet traces of the nfsv3 krb5 and krb5i mounts and I'll make
>traces of the two nfsv4 mount attempts and send them to you if you're
>interested. I'm still not sure what exactly is happening here.
The successful one for NFSv4 might be interesting. If you look at it in
wireshark, I suspect you'd find somewhere during the mount that it
did RPCs which were not krb5p and that would show why the addition
of krb5i made it work.

I did suggest you start with -sec=sys:krb5:krb5i:krb5p and, once that works,
remove the security flavours one at a time until the mount doesn't work.
(Then you capture packets for the minimal case that does work and look at
 what security flavours the client is using for all RPCs done during the mount.)

You now know why almost no one uses Kerberized NFSv4 mounts.
Unfortunately, the NFSv4 working group has never gotten around to
a better solution. Discussion of a host based encryption technique using
something like SSL has happened, but no one has gone beyond that.

rick