Several NFS errors on diskless clients

[Rick Macklem]

Beeblebrox wrote:
> Hi Rick, thanks for answering, but it seems there's a slight
> mis-understanding.
> 
> * I have 3 NFS exports, mount version on client confirmed with
> "nfsstat -m":
> V2: /data/amd64,  mounts to root as ro
> V4: /usr/local,  mounts to same as ro
> V4: /home,  mounts to same as rw
> 
> * Root is being mounted as V2, not a problem for me. However, when I
> "ls -la" /home or /usr/local (the V4 mounts), I see file ownership
> problem (numbers instead of actual user name). This is true for
> folders owned ANY user (root, me, etc):
> drwxr-xr-x  106  32767  32767  date-time   folder-name  (always the
> same number 32767)
> 
> * I turned on some debugging with "nfsuserd -verbose" and I can see:
> added uid=0 name=root  \  added gid=0 name=root
> added uid=1001 name=me  \  added gid=1001 name=me
> added uid=350 name=_sabnzbd  \  added gid=350 name=_sabnzbd
> 
> * Most Likely reason: The NFS-V2 export (client root /data/amd64
> folder) is at the same time a Jail. This probably causes "some
> process number in the jail" not matching up with process number on
> Host, thus the failure to assign proper name to file ownership. My
> /etc/jail.conf:
> pxe {   name = pxe;
>         path = /data/amd64;
>         ip4.addr = 192.168.2.1;  #subnet as /32 breaks tftp
>         host.hostname = pxe.me;
>         devfs_ruleset = 12;
>         allow.raw_sockets;
>         mount.devfs;
>         mount.fstab = /etc/fstab.pxe;	}
> 
> * I start the diskless client system with below script. Any
> additional pxe service (dhcp, tftp, etc) is defined in
> /data/amd64/etc/rc.conf.
> jail -c pxe
> rpcbind -s -h 192.168.2.1
> mountd -rnl -h 192.168.2.1
> nfsd -ut -n 4 -h 192.168.2.1
> nfsuserd -verbose
> 
Well, I have no idea what effect a jail would have (never used one),
but since it appears that nfsuserd is resolving numbers to names...
The owner and owner_group name in NFSv4 looks like:
  <user-or-group>@<dns-domain>
Since the "user-or-group" part seems to be working, then maybe the
  <dns-domain> part isn't the same?
(It doesn't really matter what it is, so long as it is the same
 for the server and client.)
The "-domain" command line option on nfsuserd allows you to set it
to whatever you want. (Without that option, it should be the dns-domain
part of the client/server's fully qualified hostname.)

However, what you write below suggests that the mapping is working,
so I don't understand what is going on.

> * Starting dbus/hald on client while nfsuserd is running results in:
> a) added gid=556 name=messagebus  \  added gid=562 name=polkit
> b) No message about "nfs_getpages: error 13"
> 
Well, the AUTH_SYS authenticator has the uid/gid-list for the process
that is doing the nfs_getpages read. This uid/gid-list is used by
the server to determine if EACCES is returned.
It has nothing to do with nfsuserd, which only affects what the
owner and owner_group strings are. (These strings are only is getattr
and setattr operations.) Maybe this application does a Setattr
(chown, chgrp...) on the file and that would work for NFSv4 if
the mapping stuff is working. (But it doesn't seem to be, given the
above.)

> * If I don't run nfsuserd, or if I use NFS V2/3 for the /usr/local
> export:
> a) File permissions on the non-root exports get displayed by name.
Without nfsuserd, the server will put the uid/gid# in the string and
a fairly recent client will understand that for NFSv4. For NFSv2/3,
the uid/gid#s are always used.

> b) I see the "nfs_getpages: error 13" message.
> 
I can think of a couple of explanations for this:
1 - The user/group database isn't consistent between client and
    server (ie. missing and/or different #s).
2 - Some other thread is doing the nfs_getpages() and using a
    different uid than the process. (I can't think of how this
    would happen for pageins, although I think it can happen for
    pageouts.)
--> Looking at the packets in wireshark will tell you what is
    actually going on.

Just in case you didn't already know, if you are going to use NFSv4
over AUTH_SYS, the clients and server need consistent user and group
databases. Same names and same numbers.

rick

> * I did not try the "-maproot=root" solution, because it does not
> address the problem of regular users (ie me) having incorrect file
> ownership under /home/me.
> 
> * File ownership under /var/run is correct because it's not NFS (it's
> MD)
> 
> I hope I was better able to explain the issue this time.
> Regards.
> --
> FreeBSD_amd64_11-Current_RadeonKMS
>