newnfs client and statfs

[Bruce Evans]

On Tue, 3 May 2011, Rick Macklem wrote:

[attributions lost]
>> ...
>> As I said before, NetBSD's nfs tries to make this work for nfs, but I
>> couldn't this worked in NetBSD or anything I could think of, since the
>> extension is not in the nfs protocol. Now I think it does work, but
>> still can't see how. Details: NetBSD puts f_bavail on the wire without

Nah, it cannot work.

>> clamping it (it just scales it). Now I think f_bavail is never
>> negative
>> in NetBSD, so this scaling doesn't involves the usual sign extension
>> and overflow bugs, or abuse of the top bit. The client zaps negative
>> values for v3 f_bavail but not for other things, and initializes
>> f_bresvd:
>> from a 2005 version ofs nfs_vfsops.c:
>>
>> % if (v3) {
>> % sbp->f_frsize = sbp->f_bsize = NFS_FABLKSIZE;
>> % tquad = fxdr_hyper(&sfp->sf_tbytes);
>> % sbp->f_blocks = ((quad_t)tquad / (quad_t)NFS_FABLKSIZE);
>> % tquad = fxdr_hyper(&sfp->sf_fbytes);
>> % sbp->f_bfree = ((quad_t)tquad / (quad_t)NFS_FABLKSIZE);
>> % tquad = fxdr_hyper(&sfp->sf_abytes);
>> % tquad = ((quad_t)tquad / (quad_t)NFS_FABLKSIZE);
>> % sbp->f_bresvd = sbp->f_bfree - tquad;

Hmm, the tabs are more mangled than usual.

>> I still can't see how this initialization works. f_bresvd has to end
>> up as nonzero if root has a reserve, and drop to zero as the reserve
>> is used up. sf_fbytes - sf_abytes must give this reserve.
>>
>> % sbp->f_bavail = tquad;
>> % #ifdef COMPAT_20
>> % /* Handle older NFS servers returning negative values */
>> % if ((quad_t)sbp->f_bavail < 0)
>> % sbp->f_bavail = 0;
>> % #endif
>>
>> NetBSD's own server puts f_bavail on the wire unchanged except for
>> scaling,
>> so it is now clear that f_bavail is never negative in NetBSD.
>>
>> % tquad = fxdr_hyper(&sfp->sf_tfiles);
>> % sbp->f_files = tquad;
>> % tquad = fxdr_hyper(&sfp->sf_ffiles);
>> % sbp->f_ffree = tquad;
>> % sbp->f_favail = tquad;
>>
>> "Negative" values for this are not zapped.
>>
>> % sbp->f_fresvd = 0;
>>
>> This reserv is not really supported. Supporting it is impossible since
>> there is not as much redundancy in the wire values for the file counts
>> as for the block counts.
>>
>> % sbp->f_namemax = MAXNAMLEN;
>> % } else {
>> % sbp->f_bsize = NFS_FABLKSIZE;
>> % sbp->f_frsize = fxdr_unsigned(int32_t, sfp->sf_bsize);
>> % sbp->f_blocks = fxdr_unsigned(int32_t, sfp->sf_blocks);
>> % sbp->f_bfree = fxdr_unsigned(int32_t, sfp->sf_bfree);
>> % sbp->f_bavail = fxdr_unsigned(int32_t, sfp->sf_bavail);
>>
>> Still has old bugs.
>>
>> % sbp->f_fresvd = 0;
>> % sbp->f_files = 0;
>> % sbp->f_ffree = 0;
>> % sbp->f_favail = 0;
>> % sbp->f_fresvd = 0;
>> % sbp->f_namemax = MAXNAMLEN;
>> % }
>>
>> Next steps: someone should look at why there are 3 nfsv3 protocol
>> fields for the block counts when only 2 are strictly needed.

> Here is the RFCs definition of the 3 fields:
>      tbytes
>         The total size, in bytes, of the file system.
>
>      fbytes
>         The amount of free space, in bytes, in the file
>         system.
>
>      abytes
>         The amount of free space, in bytes, available to the
>         user identified by the authentication information in
>         the RPC.  (This reflects space that is reserved by the
>         file system; it does not reflect any quota system
>         implemented by the server.)

So nfs does support a specially restricted amount of free space available
to a mere user, but it doesn't support this amount being negative.  BSD
uses a negative amount for this to indicate how far away fom having any
space to use the user is.

> I suspect that most systems running FFS (mis)use abytes to represent
> the non-root value, even when "root" does the RPC. If they didn't
> do that, then abytes would be different when root did statfs and that
> would be confusing to a typical client.
>
> Since you don't know if the server's file system is one like FFS that
> has a "minfree" (and you don't know what "minfree" is), you can't
> reliably calculate a negative f_bavail from the above, from what I
> can see.

Yes, it's very annoying that only 3 numbers are available, and 3 numbers
are supplied, and the number corresponding to minfree can be recovered
from the 3 numbers supplied, but only when abytes > 0.  (fbytes - abytes)
gives the amount of free space _not_ available to the user and therefore
the amount if free space reserved.  Under the condition abytes > 0,
for file systems like ffs, none of the original reservation (according
to minfree) is used, so (fbytes - abytes) also gives the size of the
original reservation.  But when the reservation starts being used,
abytes is clamped to 0 on broken systems, so the linear relations
between the 3 numbers and the alternative more useful 3 numbers (tbytes,
fbytes, origreservedbytes) are broken, so there is no way to recover
the original reservation, or equivalently, the amount of the reservation
that is used.  NetBSD uses:

 	tquad = fxdr_hyper(&sfp->sf_abytes);
 	tquad = ((quad_t)tquad / (quad_t)NFS_FABLKSIZE);
 	sbp->f_bresvd = sbp->f_bfree - tquad;

This is (fbytes - abytes) in blocks, so it only works when abytes > 0.

Repeating part of the above:
> I suspect that most systems running FFS (mis)use abytes to represent
> the non-root value, even when "root" does the RPC. If they didn't
> do that, then abytes would be different when root did statfs and that
> would be confusing to a typical client.

Negative abytes is even more useful for root (or for any user privileged
enough to use the reserve).  It tells how much of the reserve is used.
Users that can eat the reserve should try not to, and when they have
they should try to release space to get back to the full reserve.
Without negative abytes, there is no API in statfs(2) to tell how much
has been eaten.  NetBSD's f_bresvd in stavfs(2) might be able to tell,
but it is unclear if it is supposed to give the original reserve or
the current reserve, and it is already hard enough to decode the 3
numbers into 3 useful ones.

Bruce