integrating nfsv4 locking with nlm and local locking

[John Baldwin]

On Thursday 09 April 2009 3:04:37 pm Rick Macklem wrote:
> My nfsv4 server currently does VOP_ADVLOCK() with the non-blocking F_SETLK
> type and I had thought that was sufficient, but I now realize (thanks to
> a recent post by Zachary Loafman) that this breaks when a delegation for
> the file is issued to a client. (When a delegation for a file is issued
> to a client, it can do byte range locking locally, and the server doesn't
> know about these to do VOP_ADVLOCK() on the server machine.)
> 
> I believe that Zachary would like to discuss a more general solution, 
> including how to handle Open/Share locks, but in the meantime I'd like to 
> solve this specific case in as simple a way as possible.
> 
> Basically, I need a way to make sure delegations for a file don't exist
> when local byte range locking or locking via the NLM is being done on
> the file.
> 
> The simplest thing I can think of is the following:
> When VOP_ADVLOCK() is called for a file (outside of the nfsv4 server),
> do two things:
>  	1 - Make sure any outstanding delegations are recalled.
>              I already have a function that does this, so it is a matter
>              of figuring out where to put the call(s).
>  	2 - Set a flag on the vnode, so that my nfsv4 server knows not to
>  	    issue another delegation for that file.
>              (I could test for locks via VOP_ADVLOCK() before issuing a
>  	     delegation, but that has two problems.)
>  	    1 - Since the vnode is unlocked for VOP_ADVLOCK(), there could
>  	        be a race where the nfsv4 server issues a delegation
>                  between the time outstanding delegations are recalled at
>                  #1 above and the VOP_ADVLOCK() sets the lock that I would
>                  see during the test.
>              2 - It would have to keep checking for a lock and might issue
>  	        a delegation at a point where no lock is held, but one
>                  will be acquired soon, forcing the delegation recall.
>                  (It's much easier to not issue a delegation than recall
>                   one.)
>              Once this flag is set, I think it would be ok if the flag
>              remains set until the vnode is recycled, since it seems
>              fairly likely that, once byte range locking is done on a
>              file, more will happen.
>              (If people were agreeable to the vnode flag, it looks like
>               a VV_xxx flag would make more sense than a VI_xxx one. I
>               think an atomic_set_int() would be sufficient to set it,
>               even though the vnode lock isn't held?)

You have to hold the vnode lock to set a VV flag always.  Even if you do an 
atomic operation to set your flag, another thread might be setting a flag at 
the same time using non-atomic ops and could clobber your change (if it does 
a read-modify-write and reads a value that pre-dates your atomic_set_int() 
but its write posts after your write).

-- 
John Baldwin