How to invalidate NFS read cache?

[Robert Watson]

On Tue, 12 May 2009, Robert Watson wrote:

> Normally, NFS clients implement open-to-close consistency, which dictates 
> that when a close() occurs on client A, all pending writes on the file 
> should be issued to the server before close() returns, so that a signal to 
> client B to open() the file can validate its cache before open() returns.

This should, of course, read "close-to-open consistency" -- I plead jetlag 
after an overnight flight back form Boston to the UK :-)

Robert N M Watson
Computer Laboratory
University of Cambridge

>
> This raises the following question: is client A closing the file, and is 
> client B then opening it?
>
> If not: relying on writes being visible on the client B before the close() on 
> A and a fresh open() on B is not guaranteed to work, although we can discuss 
> ways to improve behavior with respect to expectation.  Try modifying your 
> application and see if it gets the desired behavior, and then we can discuss 
> ways to improve what you're seeing.
>
> If you are: this is probably a bug in our caching and or issuing of NFS RPCs. 
> We cache both attribute and access data -- perhaps there is an open() path 
> where we issue neither RPC?  In the case of open, we likely should test for a 
> valid access cache entry, and if there is one, issue an attribute read, and 
> otherwise just issue an access check which will piggyback fresh attribute 
> data on the reply.  Perhaps there is a bug here somewhere.
>
> A few other misc questions:
>
> - Could you confirm you're using NFSv3 on all clients.  Are there any special
>  mount options in use?
> - What version of FreeBSD are you running with?
>
> In FreeBSD 8.x, we now have DTrace probes for all of the above events -- 
> VOPs, attribute cache hit/miss/load/flush, access cache hit/miss/load/flush, 
> RPCs, etc, which we can use to debug the problem.  I haven't yet MFC'd these 
> to 7.x, but if you're able to run a very fresh 7-STABLE, I can probably 
> produce a patch to add it for you in a few days.
>
> Robert N M Watson
> Computer Laboratory
> University of Cambridge
>