lost dotdot caching pessimizes nfs especially

[Mohan Srinivasan]

Hi Rick

--- rick at snowhite.cis.uoguelph.ca wrote:

> > SunOS and Solaris have provided close-to-open consistency for a very long time (for
> > at least 15 years now). 
> 
> I believe you (and I understand the principal "if Solaris does X, that's the
> way it needs to be"). Until very recently, Solaris sources weren't open and
> I didn't have access to them. That was important in the "bad old days", since
> Sun's Legal Beagles did once send a threatening letter w.r.t. my NFS
> violating their proprietary...
> 
> I'd argue that these days it's "if Linux does X, that's the way it needs
> to be".:-)

Probably so. But to my mind, Solaris still has the most robust NFS client implementation
out there (I have no affiliation with Sun whatsoever), which is why I mentioned Solaris.
I have not looked at Linux wrt cto consistency.

> > Not having the NFS client enforce close-to-open consistency will break a heck of a lot
> > of applications. Since the other NFS clients (that matter) Solaris and Linux support it, 
> > I would argue that not supporting cto consistency is not really an option. 
> 
> I am a bit surprised that a lot of applications break. To do so, they must
> be running on multiple clients, read/write sharing the same NFS mounted file(s)
> and use some back-end protocol that says something like "I've just closed it,
> so you can now open it" so inconsistencies < 1 minute, causes problems.

Such applications are very common. 

1) An application where multiple clients can do something like this :

   Acquire a file lock;
   open();
   Do I/O
   close();
   Drop the file lock;

   Without cto consistency, there's no way this is going to work.

   That is what a large application where I am employed does. And I would 
   think that would be very common case elsewhere too.

(You can replace the file lock with a byte range lock post-open(), but that
won't change the result).

2) Without cto consistency, something as simple as editing a file on one
   client and compiling it on another won't work anymore. Breaking this is
   sure to send people with pitchforks running after the perpetrator :)

> NFSv4 simply
> says that clients that care about cache consistency should use byte
> range locking. Here's what RFC3530 (the NFSv4 RFC) says:

I don't know anything about NFSv4 (or Delegations). I figured that by the time
NFSv4 gains wide acceptance, I'd be happily retired :), so I haven't bothered.
I am mostly interested in a robust NFSv3 implementation :)

At least in the context of NFSv2/3, byte range locking is a necessary but 
not sufficient condition for correctness. For correctness, you require
Byte Range Locking + Direct IO (or some equivalent that bypasses client 
caching). 

> For NFSv2/3, maintaining close-to-open consistency may be appropriate
> (and necessary), but it can result in a big performance hit. 

Cool. So we agree on that :)

I agree there's a performance hit. Which is unavoidable. Best we can do is
mitigate it with things like a nocto mount option, adding namei flags that
NFS can set if it did an otw getattr from the lookup etc.

mohan