open() and ESTALE error

[Andrey Alekseyev]

Don,

> One case where there is a difference between timing out old file handles
> and just invalidating them on ESTALE:

Frankly, I just didn't find any mechanism in the STABLE kernel that
does "timing out" for file handles. Do you mean, it would be nice to have
it or are you trying to point it out to me? ;-P

> client%	cmd1 > file1; cmd2 > file2
> server% mv file1 tmpfile; mv file2 file1; mv tmpfile file1
> 
> wait an hour
> 
> client% cat /dev/null > file1
> 
> If file handles are cached indefinitely, and the client didn't recycle
> the vnode for file1, which file on the server got truncated?  Since
> neither file was deleted on the server, you can't rely on ESTALE to
> detect this situation.

Eh, but the generation number for file1 should have been changed! This will
result in a definite ESTALE error for file1 from the server. That is, I
believe that if you attempt to open("file1", O_CREAT) after an hour, you'll
get ESTALE from the server (on which nfs_request() will invalidate "file1"
namecache entry and vnode+nfsnode+old-file-handle) and the second vn_open()
will re-lookup file1 and get a valid new file handle.

Actually, this is what indeed happens if the second open() comes from the
userland application :)  I'm just trying to eliminate the need of modifying
a generic application.

For my example with moves, the next "cat" will always(!) succeed.

> Question: does the timeout of the directory attributes cause open() do
> do an NFS lookup on the file, or does open() just find the vnode in the
> cache and use its cached handle?

Well, for open() without O_CREAT the sequence is this:
open() -> vn_open() -> namei() -> lookup() -> VOP_LOOKUP() -> nfs_lookup()
          |
          VOP_ACCESS() -> nfs_access() [ -> nfs3_access_otw() <possibly>]
          |
          VOP_OPEN() -> nfs_open()

Lookup is always done first (obviously). It may return cached name which
contains a pointer to a cached vnode/nfsnode. Cached vnode/nfsnode is used
further in VOP_ACCESS() and VOP_OPEN(). Either function may or may not
update file attributes cached inside nfsnode. Neither VOP_ACCESS() or
VOP_OPEN() ever updates the *file handle*. File handle comes from
VOP_LOOKUP().  And VOP_LOOKUP() only places it there if vnode/nfsnode isn't
cached.  Which I believe happens only if there is no cached filename in
the namecache. I really tried to do my best to describe everything in:
http://www.blackflag.ru/patches/nfs_attr.txt
Please take a look.

Whether ESTALE came from VOP_ACCESS() or VOP_OPEN() depends on several
factors. Namely, the value of nfsaccess_cache_timeout sysctl, acmin/acmax
and the age of the file in question.

Generally speaking, if nfsaccess_cache_timeout is less than acmin,
VOP_ACCESS() that comes right before VOP_OPEN() in vn_open() will try to do
an "access" RPC request and it'll fail if the file handle is stale. If
nfsaccess_cache_timeout is greater than acmin, than it's possible that
VOP_ACCESS() will answer "yes" basing on the cached attributes, but
VOP_GETATTR(), which is called from nfs_open() (which is VOP_OPEN() for
NFS) will in turn "go to the wire" and still nfs_request() will fail with
ESTALE.

Hope, I'm making it clear :)