integrating nfsv4 locking with nlm and local locking

[Rick Macklem]

My nfsv4 server currently does VOP_ADVLOCK() with the non-blocking F_SETLK
type and I had thought that was sufficient, but I now realize (thanks to
a recent post by Zachary Loafman) that this breaks when a delegation for
the file is issued to a client. (When a delegation for a file is issued
to a client, it can do byte range locking locally, and the server doesn't
know about these to do VOP_ADVLOCK() on the server machine.)

I believe that Zachary would like to discuss a more general solution, 
including how to handle Open/Share locks, but in the meantime I'd like to 
solve this specific case in as simple a way as possible.

Basically, I need a way to make sure delegations for a file don't exist
when local byte range locking or locking via the NLM is being done on
the file.

The simplest thing I can think of is the following:
When VOP_ADVLOCK() is called for a file (outside of the nfsv4 server),
do two things:
 	1 - Make sure any outstanding delegations are recalled.
             I already have a function that does this, so it is a matter
             of figuring out where to put the call(s).
 	2 - Set a flag on the vnode, so that my nfsv4 server knows not to
 	    issue another delegation for that file.
             (I could test for locks via VOP_ADVLOCK() before issuing a
 	     delegation, but that has two problems.)
 	    1 - Since the vnode is unlocked for VOP_ADVLOCK(), there could
 	        be a race where the nfsv4 server issues a delegation
                 between the time outstanding delegations are recalled at
                 #1 above and the VOP_ADVLOCK() sets the lock that I would
                 see during the test.
             2 - It would have to keep checking for a lock and might issue
 	        a delegation at a point where no lock is held, but one
                 will be acquired soon, forcing the delegation recall.
                 (It's much easier to not issue a delegation than recall
                  one.)
             Once this flag is set, I think it would be ok if the flag
             remains set until the vnode is recycled, since it seems
             fairly likely that, once byte range locking is done on a
             file, more will happen.
             (If people were agreeable to the vnode flag, it looks like
              a VV_xxx flag would make more sense than a VI_xxx one. I
              think an atomic_set_int() would be sufficient to set it,
              even though the vnode lock isn't held?)

So, how does this sound? rick