readdir/telldir/seekdir problem (i think)

Julian Elischer julian at freebsd.org
Fri Apr 24 10:42:13 UTC 2015


On 4/24/15 6:12 AM, Rick Macklem wrote:
> John Baldwin wrote:
>> On Thursday, April 23, 2015 05:02:08 PM Julian Elischer wrote:
>>> On 4/23/15 11:20 AM, Julian Elischer wrote:
>>>> I'm debugging a problem being seen with samba 3.6.
>>>>
>>>> basically  telldir/seekdir/readdir don't seem to work as
>>>> advertised..
>>> ok so it looks like readdir() (and friends) is totally broken in
>>> the face
>>> of deletes unless you read the entire directory at once or reset to
>>> the
>>> the first file before the deletes, or earlier.
>> I'm not sure that Samba isn't assuming non-portable behavior.  For
>> example:
>>
>> From
>> http://pubs.opengroup.org/onlinepubs/009695399/functions/readdir_r.html
>>
>> If a file is removed from or added to the directory after the most
>> recent call
>> to opendir() or rewinddir(), whether a subsequent call to readdir()
>> returns an
>> entry for that file is unspecified.
>>
>> While this doesn't speak directly to your case, it does note that you
>> will
>> get inconsistencies if you scan a directory concurrent with add and
>> remove.
>>
>> UFS might kind of work actually since deletes do not compact the
>> backing
>> directory, but I suspect NFS and ZFS would not work.  In addition,
>> our
>> current NFS support for seekdir is pretty flaky and can't be fixed
>> without
>> changes to return the seek offset for each directory entry (I believe
>> that
>> the projects/ino64 patches include this since they are breaking the
>> ABI of
>> the relevant structures already).  The ABI breakage makes this a very
>> non-trivial task.  However, even if you have that per-item cookie, it
>> is
>> likely meaningless in the face of filesystems that use any sort of
>> more
>> advanced structure than an array (such as trees, etc.) to store
>> directory
>> entries.  POSIX specifically mentions this in the rationale for
>> seekdir:
>>
>> http://pubs.opengroup.org/onlinepubs/009695399/functions/seekdir.html
>>
>> One of the perceived problems of implementation is that returning to
>> a given point in a directory is quite difficult to describe
>> formally, in spite of its intuitive appeal, when systems that use
>> B-trees, hashing functions, or other similar mechanisms to order
>> their directories are considered. The definition of seekdir() and
>> telldir() does not specify whether, when using these interfaces, a
>> given directory entry will be seen at all, or more than once.
>>
>> In fact, given that quote, I would argue that what Samba is doing is
>> non-portable.  This would seem to indicate that a conforming seekdir
>> could
>> just change readdir to immediately return EOF until you call
>> rewinddir.
>>
> Btw, Linux somehow makes readdir()/unlink() work for NFS. I haven't looked,
> but I strongly suspect that it reads the entire directory upon either opendir()
> or the first readdir().
>
> Oh, and I hate to say it, but I suspect Linux defines the "standard" on
> this and not POSIX. (In other words, if it works on Linux, it isn't broken;-)
>
> rick

here's an interesting datapoint.  If the test program is run on 
kFreeBSD using glibc, it runs without flaw.

OS-X  (bsd derived libc) HFS+ fails
FreeBSD libc (UFS) fails
FreeBSD libc (ZFS) fails
FreeBSD glibc  succceeds
Centos 6.5 glibc succeeds

some NFS tests would be nice to do too I guess...
glibc authors seem to have done something right.. it even copes with 
FreeBSD kernel..



>
>> --
>> John Baldwin
>> _______________________________________________
>> freebsd-current at freebsd.org mailing list
>> http://lists.freebsd.org/mailman/listinfo/freebsd-current
>> To unsubscribe, send any mail to
>> "freebsd-current-unsubscribe at freebsd.org"
>>
> _______________________________________________
> freebsd-current at freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-current
> To unsubscribe, send any mail to "freebsd-current-unsubscribe at freebsd.org"
>
>



More information about the freebsd-current mailing list