lost dotdot caching pessimizes nfs especially

[rick at snowhite.cis.uoguelph.ca]

> SunOS and Solaris have provided close-to-open consistency for a very long time (for
> at least 15 years now). 

I believe you (and I understand the principal "if Solaris does X, that's the
way it needs to be"). Until very recently, Solaris sources weren't open and
I didn't have access to them. That was important in the "bad old days", since
Sun's Legal Beagles did once send a threatening letter w.r.t. my NFS
violating their proprietary...

I'd argue that these days it's "if Linux does X, that's the way it needs
to be".:-)

> Not having the NFS client enforce close-to-open consistency will break a heck of a lot
> of applications. Since the other NFS clients (that matter) Solaris and Linux support it, 
> I would argue that not supporting cto consistency is not really an option. 

I am a bit surprised that a lot of applications break. To do so, they must
be running on multiple clients, read/write sharing the same NFS mounted file(s)
and use some back-end protocol that says something like "I've just closed it,
so you can now open it" so inconsistencies < 1 minute, causes problems.
I wish those applications had been common a decade ago becuase the NFS
community might have cared about cache consistency and something along
the lines of my experimental NQNFS might have happenned. NFSv4 simply
says that clients that care about cache consistency should use byte
range locking. Here's what RFC3530 (the NFSv4 RFC) says:

at top of Page 14:
   If an application wants to serialize access to file data, file
   locking of the file data ranges in question should be used.

Admittedly the paragraph that preceeds this almost says what Solaris is doing
and seems to contradict Sec 9:
9.  Client-Side Caching

   Client-side caching of data, of file attributes, and of file names is
   essential to providing good performance with the NFS protocol.
   Providing distributed cache coherence is a difficult problem and
   previous versions of the NFS protocol have not attempted it.
   Instead, several NFS client implementation techniques have been used
   to reduce the problems that a lack of coherence poses for users.
   These techniques have not been clearly defined by earlier protocol
   specifications and it is often unclear what is valid or invalid
   client behavior.

   The NFS version 4 protocol uses many techniques similar to those that
   have been used in previous protocol versions.  The NFS version 4
   protocol does not provide distributed cache coherence.  However, it
   defines a more limited set of caching guarantees to allow locks and
   share reservations to be used without destructive interference from
   client side caching.

   In addition, the NFS version 4 protocol introduces a delegation
   mechanism which allows many decisions normally made by the server to
   be made locally by clients.  This mechanism provides efficient
   support of the common cases where sharing is infrequent or where
   sharing is read-only.

Once clients figure out how to effectively use Delegations, I think there
will be significant performance improvements. Unfortunately, we haven't
gotten there yet.

For NFSv2/3, maintaining close-to-open consistency may be appropriate
(and necessary), but it can result in a big performance hit. For example,
try an experiment where you turn off "push writes on close" in the code
and see what effect that has on performance for the common, non-read/write
shared files case. (nb: I don't think you can get away with doing this
without some other cache consistency guarantees, but it would sure be
nice from a performance point of view. That was what nqnfs was all about.
If you are really bored, you can go to http://www.cis.uoguelph.ca/~nfsv4 and
read my ancient nqnfs paper.)

rick