readdir() -> d_type always zero on NFS v3 mounted filesystems

[Ronald Klop]

On 4/29/21 10:24 AM, N.J. Mann wrote:
> Hi,
> 
> 
> I recently changed over from using svn to gitup to update /usr/ports
> on my local system and have been experiencing problems since.  At first
> I thought it was an issue with gitup itself, but now I believe it is
> either a kernel issue or a configuration issue.  I originally posted
> about the problem to the freebsd-ports mailing list:
> https://lists.freebsd.org/pipermail/freebsd-ports/2021-April/120929.html
> 
> Since then I have dug deeper and come to the conclusion that it is not a
> problem with gitup.
> 
> The issue I am seeing is that gitup is unable to delete files and
> directories, even complete ports, which have been removed from the
> repository.  gitup basically does the following:
> 
> prune_tree(base_path)
> {
> if ((directory = opendir(base_path)) != NULL) {
>      while ((entry = readdir(directory)) != NULL) {
>          snprintf(full_path, sizeof(full_path), "%s/%s", base_path, entry->d_name);
>          if (entry->d_type == DT_DIR) {
>              prune_tree(full_path);
>          } else {
>              if ((remove(full_path) != 0) && (errno != ENOENT))
>                   err(EXIT_FAILURE, "prune_tree: cannot remove %s", full_path);
>              }
>          }
>          closedir(directory);
>          if (rmdir(base_path) != 0)
>              err(EXIT_FAILURE, "prune_tree: cannot remove %s", base_path);
>      }
> }
> 
> When gitup is run on either a UFS or ZFS file system this works.  However,
> when I run it on a NFS v3 mounted filesystem it fails.  Liberal addition
> of printf's shows that for non-NFS mounted filesystems d_type contains the
> correct value for each file/directory, but for NFS mounted file systems it
> is zero - other fields such as d_name and d_namlen are correct.  While
> debugging this I added a call to stat() and that always returns the correct
> values.
> 
> At this point I started digging in libc and quickly found that readdir()
> is basically a wrapper around a system call.  Digging in the kernel I
> quickly got out of my depth and hence my posting here.  I then wrote a
> simple test programme which also shows the issue.  I have attached the
> source for the test programme and the output from two runs, the first on
> an NFS mounted file system and the second on a local UFS file system.
> 
> Before gitup started failing I had not seen any failures like this and that
> was why I initially assumed this must be a gitup issue.  I now believe this
> is either a kernel issue or a confuration issue.
> 
> My configuration is as follows:
> 
> file server:
>    /exports/ports - ZFS file system for FreeBSD ports repo exported via NFS
>    /remote/ports  - FreeBSD ports repo NFS (v3 rw,tcp) mounted from /export/ports
>                     on file server
>    /usr/ports     - symbolic link to /remote/ports
> client machines:
>    /remote/ports  - FreeBSD ports repo NFS (v3 ro,tcp) mounted from /export/ports
>                     on file server
>    /usr/ports     - symbolic link to /remote/ports
> 
> I run gitup in /remote/ports on the file server and then pkg, portmaster, &.c,
> in /usr/ports on the file server and then the client machines.  All systems are
> running 11-STABLE from about 12 days ago - I usually update every couple of weeks.
> 
> Any assistance, suggestions, patches, clue bats, will be gratefully accepted.
> 
> 
> Regards,
>          Nick.
> 


Nice analysis. The manual page dirent(5) says:
"BUGS
      The usage of the member d_type of struct dirent is unportable as it is
      FreeBSD-specific.  It also may fail on certain file systems, for example
      the cd9660 file system.
"

I think gitup would be more standards compliant if it used stat(2) for determining the type of a file/dir. An issue about this can best be reported at the upstream project: https://github.com/johnmehr/gitup .

If the NFS server/client can implement d_type is outside of my knowledge.

Regards,
Ronald.