lost dotdot caching pessimizes nfs especially
bde at zeta.org.au
Sun Oct 15 23:30:02 PDT 2006
On Sun, 15 Oct 2006, Mohan Srinivasan wrote:
> The clearing of the attrcache on nfs_open() is a requirement for close-to-open
> consistency, and this change fixed bugs that we saw internally relating to
> close-to-open consistency.
>> and associated changes give silly behaviour that almost doubles the
>> number of Access RPCs. One of the associated changes clears n_attrstamp
>> on close(). Then on open(), since lookup() is called before the above
>> is reached, nfs_access_otw() has always just been called, and the above
>> forces another call.
> That is not true with NFSv2 which doesn't have an access call, in which case
> nfsspec_access() calls VOP_GETATTR, which may or may not go over the wire.
> Also, what would happen with NFSv3 if we get an access cache hit ?
I didn't think about NFSv2 or check the details for NFSv3 until now.
It is nfs_lookup() that always calls VOP_GETATTR(), and VOP_GETATTR()
must go on the wire in the case being described (lookup after close)
since we flushed the attribute cache entry in nfs_close(). The
difference for v2 is that nfs_getattr() normally uses a Getattr request
for v2 and an Access request for v3.
For NFSv3, nfs_lookup()'s behaviour is correct for the attribute cache
is not as good as it could easily be for the attribute cache. In
nfs_lookup() after a recent close(), in the usual cases all caches are
hit except we just cleared the attribute cache, so nfs_lookup() does
VOP_ACCESS(); # Cache hit. Access granted.
cache_lookup(); # Positive cache hit.
VOP_GETATTR(); # Cache miss. Succeeds.
# Now we have fresh attributes in the v3 case, but we granted access
# based on the old attributes, so we unnecessarily lost full
# open/close consistency.
In unusual cases, there is an acccess cache miss. Then for v3,
VOP_ACCESS() refreshes the attribute cache too, VOP_GETTATR() is a
cache hit, and there is full open/close consistency.
> If lookup() can be made to pass a flag into nfs_open() that an otw getattr was
> done, then we can eliminate the clearing of the attrcache in nfs_open(). But
> absent that flag, I don't see how you can eliminate the fetch of fresh attributes
> in nfs_open().
Of course something like such a flag is needed. See my previous mail for
more details (there should be another flag for nfs_lookup() so that the
entire open() is consistent). For nfs_open(), I was thinking more of
a generation count. Now I wonder about exclusive locking and blockages.
VOP_OPEN() is now exclusively locked, but I don't now if the same lock
covers the lookup. With exclusive locking, not even a flag is needed.
Without exclusive locking, blocking might be a problem.
More information about the freebsd-fs