lost dotdot caching pessimizes nfs especially

[Bruce Evans]

On Sun, 15 Oct 2006, Mohan Srinivasan wrote:

> The clearing of the attrcache on nfs_open() is a requirement for close-to-open
> consistency, and this change fixed bugs that we saw internally relating to
> close-to-open consistency.
>
>> and associated changes give silly behaviour that almost doubles the
>> number of Access RPCs.  One of the associated changes clears n_attrstamp
>> on close().  Then on open(), since lookup() is called before the above
>> is reached, nfs_access_otw() has always just been called, and the above
>                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>> forces another call.
>
> That is not true with NFSv2 which doesn't have an access call, in which case
> nfsspec_access() calls VOP_GETATTR, which may or may not go over the wire.
>
> Also, what would happen with NFSv3 if we get an access cache hit ?

I didn't think about NFSv2 or check the details for NFSv3 until now.
It is nfs_lookup() that always calls VOP_GETATTR(), and VOP_GETATTR()
must go on the wire in the case being described (lookup after close)
since we flushed the attribute cache entry in nfs_close().  The
difference for v2 is that nfs_getattr() normally uses a Getattr request
for v2 and an Access request for v3.

For NFSv3, nfs_lookup()'s behaviour is correct for the attribute cache
is not as good as it could easily be for the attribute cache.  In
nfs_lookup() after a recent close(), in the usual cases all caches are
hit except we just cleared the attribute cache, so nfs_lookup() does
the following:

     VOP_ACCESS();        # Cache hit.  Access granted.
     cache_lookup();      # Positive cache hit.
     VOP_GETATTR();       # Cache miss.  Succeeds.
     # Now we have fresh attributes in the v3 case, but we granted access
     # based on the old attributes, so we unnecessarily lost full
     # open/close consistency.

In unusual cases, there is an acccess cache miss.  Then for v3,
VOP_ACCESS() refreshes the attribute cache too, VOP_GETTATR() is a
cache hit, and there is full open/close consistency.

> If lookup() can be made to pass a flag into nfs_open() that an otw getattr was
> done, then we can eliminate the clearing of the attrcache in nfs_open(). But
> absent that flag, I don't see how you can eliminate the fetch of fresh attributes
> in nfs_open().

Of course something like such a flag is needed.  See my previous mail for
more details (there should be another flag for nfs_lookup() so that the
entire open() is consistent).  For nfs_open(), I was thinking more of
a generation count.  Now I wonder about exclusive locking and blockages.
VOP_OPEN() is now exclusively locked, but I don't now if the same lock
covers the lookup.  With exclusive locking, not even a flag is needed.
Without exclusive locking, blocking might be a problem.

Bruce