Re: nfs4: some shares have invisible files & directories

Hi,

Do you have some local filesytem mounted over the NFS4 mount? Your remark about 'It was "resolved" by `zfs destroy zroot/usr/{ports,src}`' made me think in this direction. Did you do the zfs destroy on the NFS client or on the NFS server?

Regards,
Ronald.

 
Van: Dave Cottlehuber <dch@FreeBSD.org>
Datum: dinsdag, 1 april 2025 12:17
Aan: current@freebsd.org
Onderwerp: nfs4: some shares have invisible files & directories
> 
> TLDR I have working nfs4 mounts but in some mounts, none of the
> files are visible to ls etc. But if you know the filename, you can
> still access it directly by name. Why?
> 
> I should have a very simple setup for nfsv4. What I need is to
> have ro exports /usr/{ports,obj,src} available to clients, all
> FreeBSD.
> 
> I can mount all exported filesystems, but only /usr/obj has visible
> files on the client, and I don't understand why. This behaviour is
> consistent across clients.
> 
> during beinstall.sh from clients, they are unable to access files
> that definitely exist in /usr/obj/* such as /usr/bin/cc & /bin/rm.
> 
> It was "resolved" by `zfs destroy zroot/usr/{ports,src}` & recreating
> them. I don't recall ever using getfacl/setfacl, and I can't think
> of anything else that might make the mount work but the files
> be invisible. Sadly I did not think to just rename the afflicted
> datasets.
> 
> Does anybody know what might cause this? I did find a mention of
> this from SUSE linux, but in 2011, and I don't see any interesting
> commits related to readdir recently.
> 
> https://forums.opensuse.org/t/11-4-strange-nfs-v4-problem-invisible-files/63107
> 
> ## server
> 
> - fast build box running nfsv4 only
> - 15-CURRENT, up to date
> - providing /usr/{src,obj,ports} to 3 clients
> - this is a zfs system, and each are on separate datasets
> 
> ```
> # egrep -hr 'nfs|mount' /etc/rc.conf*
> 
> mountd_enable=YES
> nfs_server_enable=YES
> nfsv4_server_enable=YES
> nfsv4_server_only=YES
> 
> # cat /etc/exports
> V4: /usr -sec=sys
> /usr/src -ro
> /usr/obj -ro
> /usr/ports -ro
> 
> # ls -AFGhld /usr/{src,obj,ports}
> drwxr-xr-x   3 dch wheel    3B Feb 28 23:27 /usr/obj/
> drwxr-xr-x  70 dch wheel   86B Apr  1 06:55 /usr/ports/
> drwxr-xr-x  27 dch wheel   47B Mar 31 13:26 /usr/src/
> 
> # getfacl /usr/src
> # file: /usr/src
> # owner: dch
> # group: wheel
>             owner@:rwxp--aARWcCos:-------:allow
>             group@:rwxp--a-R-c--s:-------:allow
>          everyone@:r-x---a-R-c--s:-------:allow
> # getfacl /usr/src/COPYRIGHT
> # file: /usr/src/COPYRIGHT
> # owner: dch
> # group: wheel
>             owner@:rw-p--aARWcCos:-------:allow
>             group@:rw-p--a-R-c--s:-------:allow
>          everyone@:r-----a-R-c--s:-------:allow
> ```
> 
> # clients
> 
> - either 14.2-RELEASE or also 15-CURRENT
> - no daemons
> 
> ```
> # mount_nfs -o vers=4,nolockd,retrycnt=0,noatime,ro 172.16.1.4:/obj /usr/obj
> # ls /usr/obj/usr/src/amd64.amd64/
> bin/  ... lots more files, very good
> 
> # mount_nfs -o vers=4,nolockd,retrycnt=0,noatime,ro 172.16.1.4:/src /usr/src
> # mkdir /usr/src/foo
> mkdir: /usr/src/foo: Read-only file system # good its clearly mounted
> 
> # ls -AFGhl /usr/src
> total 0
> 
> # head /usr/src/COPYRIGHT
> The compilation of software known as FreeBSD is distributed under the
> following terms:
> .... what how is this file even here? ls shows nothing, tar also fails
> ```
> 
> A+
> Dave
> ———
> O for a muse of fire, that would ascend the brightest heaven of invention!
>  
> 
> 
>