NFSv4 Kerberos mount from Linux

[Rick Macklem]

Felix Winterhalter wrote:
>Hello everyone,
>
>I've been trying to get a kerberized nfsv4 mount to work from a Debian
>Stretch client to a FreeBSD 11.2 server.
>
>My export file looks like:
>
>V4: / -sec=krb5p clients
>
>/testexport -maproot=root -sec=krb5p clients
>
Btw, if you only mounting "/testexport", you can specify the "V4:" as
V4: /testexport -sec=krb5p clients
and then the mount on the client uses "/" as the server mountpoint, like
# mount -t nfs -o nfsvers=4 <server>:/ /mnt
(This avoids the server having to search for "testexport" in the "/" directory
 during mounting and might avoid some problems when "/" isn't an exported
 file system. There are "hooks" in the FreeBSD server to make the search work,
 but I've never been 100% certain they will work for Kerberos and/or ZFS.)

Btw, in case the Linux client is falling back on using AUTH_SYS at some point
during the mount, you could try allowing both krb5 and auth_sys by setting
"-sec=sys,krb5,krb5i,krb5p" for both of the above lines. (I'd also suggest you
try krb5 or krb5i until you get it working, since any packet traces are
easier to decode, although once one krb5 variant works, they all should.)

>I am now trying to mount this directory as root first without having to
>deal with user keytabs or tickets.
>
>This works fine with -sec=sys and nfsv4.1 and nfsv3 and -sec=krb5p.
> This does not however work with nfsv4 and krb5p or any other krb5 flavor.
Sorry, I'm not sure what you are saying here. Is it
1 - no version of NFS works for krb5p or
2 - NFSv4.1 works for krb5p, but NFSv4.0 does not or
3 - only nfsv3 works for krb5p

If it #3, that is what I would expect. For NFSv4 (and NFSv4.1, I believe) to work
a host based initiator credential is needed for the client host. The only ways I
know of that you can get this is by
- creating such an entry in your KDC and then putting a keytab entry for it in the client
or
- on the FreeBSD client, doing the mount as a user after that user has done a kinit.
  (There are mount options for this on FreeBSD and it also requires setting the sysctl
    vfs.usermount to 1 so non-root can do mounts.) Since you are using a Linux
    client I have no idea how this might be done on Linux.

I have no idea how the Linux server might allow an NFSv4 mount to work without
Kerberos credentials for the "state maintaining operations" done by root (or the
user doing the mount for the FreeBSD client)?
- Maybe they allow these operations to be done via AUTH_SYS. To me, this would
  sound like a security hole, but I'm not a security guy...

If you have #3, I know the FreeBSD server won't allow what you are trying to do.
If you want to find out what the Linux server does to make it work, you could
capture packets via tcpdump or similar and look at them in wireshark.
(I'd suggest krb5 or krb5i for this, so that the packet data isn't encrypted, since
 it makes the wireshark decoding a lot more useful.)
- If you get such a packet trace, you could email it to me and I can take a look.
  (I am curious how a Linux server might make this work.)

Most of the above only applies if you are experiencing #3, where NFSv3 works
for krb5, but NFSv4 (and 4.1) does not.

>The only errors we have been able to get is an error by gssd:
>
>gssd_pname_to_uid: failed major=0xd0000 minor=-1765328227
I can't remember what this means, but I think it is saying that the principal
name didn't exist in the password database. (Maybe Linux has some "special"
reserved principal name it uses for "state maintaining operations"?)

>Searching for this error has lead us to an old entry in the mailing list:
>
>https://lists.freebsd.org/pipermail/freebsd-fs/2016-May/023132.html
>
>Which apparently has this problem unresolved with extremely similar
>symptoms.
>
>Mounting from the Linux client to a similar Linux server under the same
>KDC with nfsv4 krb5p works without any problem.
>
>Also access to the FreeBSD server with sshd and GSSAPI works fine. So
>the keytab for the FreeBSD host seems to work fine.
>
>This is extremely frustrating as I have been at this problem for days
>now without any real way to even debug the issue.
Here's what I would do:
1 - Try an NFSv3 mount with krb5. (It wasn't obvious if you already have that
     working or not.) If that works, then...
2 - Try a mount from a FreeBSD client as a user, by doing
     # sysctl vfs.usermount=1
     - login as non-root user and kinit
     - as this user, try a mount like
     % mount -t nfs -o nfsv4,sec=krb5 <server>:/ /mnt
3 - If this works, then you probably need a hostbased client credential in the
      keytab on the client.

rick