amd64/74811: [nfs] df, nfs mount, negative Avail -> 32/64-bit confusion

[Bruce Evans]

[Redirected a bit]

On Tue, 18 Sep 2007, Astrodog wrote:
> On 9/18/07, Astrodog <astrodog at gmail.com> wrote:
>> On 9/18/07, Bruce Evans <brde at optusnet.com.au> wrote:

>>> -current still breaks negative avail counts on the server by clamping them
>>> them to 0, so the bug is less obvious on buggy clients.
>>
>> It appears that RFC 1094 calls for blocks free to be unsigned (2.2.8).
>> I don't know how this could be handled, besides clamping, though.
>
> Rather, it calls for blocks free, and blocks availible to be unsigned. D'oh.

RFC 1094 only covers nfsv2.  That is so crufty that its RFC even
specifies precisely unsigned for almost everything.  IIRC, nfsv3 also
specifies an unsigned type for the avail count, but that type is
uint64_t, and nfsv4 is similar.

This is clearly a bug in the spec, or rather, the spec doesn't support
BSD's primary file system or what BSD's nfs has always done.

nfs in FreeBSD-[1-4] ignores the spec and passes negative values as
large unsigned ones, mostly by blindly copying bits.  The server was
broken on 2004/04/12 (between 5.2R and 5.3R).  FreeBSD clients still
try to support negative values being passed as large unsigned ones,
but the clients with a 32-bit statfs have a lot of sign extension and
overflow bugs that are most serious for such values.

The design bug also affects statvfs(3).  POSIX standardized this but
not BSD's statfs(2).  Most things in struct statvfs are typedefed
almost to a fault (fsblkcnt_t for block counts, and fsfilcnt_t for
file counts), but fsblkcnt_t is specified to be an unsigned type, so
negative avail counts cannot work without hacks.

I don't know how to work around the design bug for all clients.
Clamping on the server seems to be best if the client doesn't support
negative avail counts.  NetBSD has large changes in this area, but
they seem to reduce to clamping.  In at least nfs_vfsops.c 1.144
(2005/01/02):

- On the server, there is no clamping, but I think negative values
   can't happen anyway because the avail counts are obtained from the
   statvfs interface and statvfs is broken (but see below about f_bresvd;
   f_bresvd is not used here so something like clamping happens
   automatically).

   NetBSD has also fixed bogus truncation of file counts to 32 bits in
   the v3 case.  Truncation is still blind, but only has to be to 32
   bits for the v2 case.

- on the client, the avail count is converted into statvfs's avail
   count (f_bavail) plus a NetBSD (?) extension of statvfs (f_bresvd).
   I think f_bresvd is NetBSD's solution to the design bug for statvfs
   and NetBSD needs this more than FreeBSD because NetBSD has converted
   many (?) utilities from statfs to statvfs.  For nfs_statvfs(),
   f_bresvd is initialized to f_bfree - f_bavail (where the free and
   avail counts are whatever is passed by then server>.  Then under a
   COMPAT_20 ifdef, avail counts which are so large that they can only
   be from an "old" server that is trying to pass a negative count,
   because b_avail to be set to 0.  In applications like df, the final
   avail count is f_bfree - f_bresvd.  This can easily be negative,
   and should be negative when f_bfree is 0 (no space for non-root) and
   f_bresvd is nonzero (some space for root).  However, nfs can only
   initialize things correctly if the server is "old" (= not broken to
   spec).  If the server is not "old" then the initializations are just:

 	f_bfree =  server f_bfree
 	f_bavail = server f_bavail
 	f_bresvd = f_bfree - f_bavail   # XXX no way to know server f_bresvd

and these are used like the following in df:

 	# f_bavail is not used in df!
 	avail = f_bfree - f_bresvd
 	      = f_bfree - (f_bfree - f_bavail)
 	      = f_bavail
 	      = server f_bavail (cast to int64_t)

The resulting `int64_t avail' can only be negative if f_bavail is
"negative" on the server, but we are using a difference and never
using f_bavail in df to avoid abusing f_bavail for holding negative
values, and in the case where the server actually passes us a "negative"
f_bavail and COMPAT_20 is configured, then we clobber f_bavail to 0
in nfs_statvfs() and end up getting avail = f_bfree in df -- completely
wrong.

So the NetBSD code only seems to give the correct result accidentally,
if the correct result is to print a negative avail count in df.  It
takes an "old" server and a client that thinks it doesn't support old
servers (no COMPAT_20 configured).

I cannot see how to get the correct result non-accidentally without
using the hack of passing negative values as large unsigned ones.
Passing negative values as the difference of two unsigned values works
with NetBSD's extension to statvfs (f_bresvd), but it doesn't work for
nfs because it requires an extra value which the protocol doesn't
support AFAIK (not far).

Bruce