readdir() -> d_type always zero on NFS v3 mounted filesystems

[Ronald Klop]

On 4/29/21 1:14 PM, Ronald Klop wrote:
> On 4/29/21 10:24 AM, N.J. Mann wrote:
>> Hi,
>>
>>
>> I recently changed over from using svn to gitup to update /usr/ports
>> on my local system and have been experiencing problems since.  At first
>> I thought it was an issue with gitup itself, but now I believe it is
>> either a kernel issue or a configuration issue.  I originally posted
>> about the problem to the freebsd-ports mailing list:
>> https://lists.freebsd.org/pipermail/freebsd-ports/2021-April/120929.html
>>
>> Since then I have dug deeper and come to the conclusion that it is not a
>> problem with gitup.
>>
>> The issue I am seeing is that gitup is unable to delete files and
>> directories, even complete ports, which have been removed from the
>> repository.  gitup basically does the following:
>>
>> prune_tree(base_path)
>> {
>> if ((directory = opendir(base_path)) != NULL) {
>>      while ((entry = readdir(directory)) != NULL) {
>>          snprintf(full_path, sizeof(full_path), "%s/%s", base_path, entry->d_name);
>>          if (entry->d_type == DT_DIR) {
>>              prune_tree(full_path);
>>          } else {
>>              if ((remove(full_path) != 0) && (errno != ENOENT))
>>                   err(EXIT_FAILURE, "prune_tree: cannot remove %s", full_path);
>>              }
>>          }
>>          closedir(directory);
>>          if (rmdir(base_path) != 0)
>>              err(EXIT_FAILURE, "prune_tree: cannot remove %s", base_path);
>>      }
>> }
>>
>> When gitup is run on either a UFS or ZFS file system this works.  However,
>> when I run it on a NFS v3 mounted filesystem it fails.  Liberal addition
>> of printf's shows that for non-NFS mounted filesystems d_type contains the
>> correct value for each file/directory, but for NFS mounted file systems it
>> is zero - other fields such as d_name and d_namlen are correct.  While
>> debugging this I added a call to stat() and that always returns the correct
>> values.
>>
>> At this point I started digging in libc and quickly found that readdir()
>> is basically a wrapper around a system call.  Digging in the kernel I
>> quickly got out of my depth and hence my posting here.  I then wrote a
>> simple test programme which also shows the issue.  I have attached the
>> source for the test programme and the output from two runs, the first on
>> an NFS mounted file system and the second on a local UFS file system.
>>
>> Before gitup started failing I had not seen any failures like this and that
>> was why I initially assumed this must be a gitup issue.  I now believe this
>> is either a kernel issue or a confuration issue.
>>
>> My configuration is as follows:
>>
>> file server:
>>    /exports/ports - ZFS file system for FreeBSD ports repo exported via NFS
>>    /remote/ports  - FreeBSD ports repo NFS (v3 rw,tcp) mounted from /export/ports
>>                     on file server
>>    /usr/ports     - symbolic link to /remote/ports
>> client machines:
>>    /remote/ports  - FreeBSD ports repo NFS (v3 ro,tcp) mounted from /export/ports
>>                     on file server
>>    /usr/ports     - symbolic link to /remote/ports
>>
>> I run gitup in /remote/ports on the file server and then pkg, portmaster, &.c,
>> in /usr/ports on the file server and then the client machines.  All systems are
>> running 11-STABLE from about 12 days ago - I usually update every couple of weeks.
>>
>> Any assistance, suggestions, patches, clue bats, will be gratefully accepted.
>>
>>
>> Regards,
>>          Nick.
>>
> 
> 
> Nice analysis. The manual page dirent(5) says:
> "BUGS
>      The usage of the member d_type of struct dirent is unportable as it is
>      FreeBSD-specific.  It also may fail on certain file systems, for example
>      the cd9660 file system.
> "
> 
> I think gitup would be more standards compliant if it used stat(2) for determining the type of a file/dir. An issue about this can best be reported at the upstream project: https://github.com/johnmehr/gitup .


Ah, somebody already found it: https://github.com/johnmehr/gitup/issues/67
You could add the information from the readdir(5) and stat(2) man pages.

Regards,
Ronald.

  
> If the NFS server/client can implement d_type is outside of my knowledge.
> 
> Regards,
> Ronald.
> 
> _______________________________________________
> freebsd-fs at freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-fs
> To unsubscribe, send any mail to "freebsd-fs-unsubscribe at freebsd.org"