9.3 NFS client bug?

[Rick Macklem]

Garrett Wollman wrote:
> <<On Tue, 7 Oct 2014 17:06:02 -0400 (EDT), Rick Macklem
> <rmacklem at uoguelph.ca> said:
> 
> >> By ay chance is your ZFS server allocating a ZFS inode (whatever
> >> they
> >> call it) with a fileno/inode# that doesn't fit in 32 bits?
> 
> Nope.  (And I want to emphasize that Ubuntu clients don't exhibit the
> problem, so whatever it is, it doesn't appear to be related to the
> backing filesystem on the server.)
> 
> > Btw, I tried your bonnie++ command on my test machines, but they
> > failed
> > part way through because they couldn't allocate boundary tags. (At
> > least
> > now I can reproduce that easily, although it is the M_NOWAIT case
> > where
> > the thread just loops in the kernel.)
> 
> You really need to get some server-class hardware.  Perhaps the
> Foundation can help?
> 
> I'm currently using the following command, which seems to reproduce
> the problem reliably on my client and server:
> 
> bonnie++ -s 0 -n 144:0:0:1:16384 -u 4294967294 -D
> 
> This minimizes the amount of temporary storage required.
> 
> -GAWollman
> 
I took a look at the bonnie++ sources and it does the removal of the
files in a loop like
                   while ((dp = readdir(dir)) != NULL)
                           unlink(dp->d_name);
As far as I know, this has never worked correctly for FreeBSD. The
unlink() invalidates the directory offset cookies and then it has
trouble finding the next entry.
To make the above loop work correctly for FreeBSD, it needs to be
re-written to start at the beginning of the directory after each
unlink().

I can't remember why it was coded to invalidate the offset cookies,
since they should still be ok after an unlink() for UFS, but maybe
other file systems break.

Maybe jhb@ can remember more about this?

OpenBSD doesn't use the buffer cache for NFS Readdir, but I can't
remember if it also suffers from this issue.

Sorry, but I hadn't run into this for a while, so I didn't spot it, rick