svn commit: r220761 - head/sys/fs/nfsclient

[Rick Macklem]

> On Sun, Apr 17, 2011 at 11:04:04PM +0000, Rick Macklem wrote:
> > Author: rmacklem
> > Date: Sun Apr 17 23:04:03 2011
> > New Revision: 220761
> > URL: http://svn.freebsd.org/changeset/base/220761
> >
> > Log:
> >   Add checks for MNTK_UNMOUNTF at the beginning of three
> >   functions, so that threads don't get stuck in them during
> >   a forced dismount. nfs_sync/VFS_SYNC() needs this, since it is
> >   called by dounmount() before VFS_UNMOUNT(). The nfscl_nget()
> >   case makes sure that a thread doing an VOP_OPEN() or
> >   VOP_ADVLOCK() call doesn't get blocked before attempting
> >   the RPC. Attempting RPCs don't block, since they all
> >   fail once a forced dismount is in progress.
> >   The third one at the beginning of nfsrpc_close()
> >   is done so threads don't get blocked while doing VOP_INACTIVE()
> >   as the vnodes are cleared out.
> >   With these three changes plus a change to the umount(1)
> >   command so that it doesn't do "sync()" for the forced case
> >   seem to make forced dismounts work for the experimental NFS
> >   client.
> >
> >   MFC after: 2 weeks
> >
> > Modified:
> >   head/sys/fs/nfsclient/nfs_clrpcops.c
> >   head/sys/fs/nfsclient/nfs_clstate.c
> >   head/sys/fs/nfsclient/nfs_clvfsops.c
> >
> > Modified: head/sys/fs/nfsclient/nfs_clrpcops.c
> > ==============================================================================
> > --- head/sys/fs/nfsclient/nfs_clrpcops.c Sun Apr 17 22:31:36 2011
> > (r220760)
> > +++ head/sys/fs/nfsclient/nfs_clrpcops.c Sun Apr 17 23:04:03 2011
> > (r220761)
> > @@ -567,6 +567,11 @@ nfsrpc_close(vnode_t vp, int doclose, NF
> >
> >  	if (vnode_vtype(vp) != VREG)
> >  		return (0);
> > +
> > + /* For forced unmounts, just return. */
> > + if ((vp->v_mount->mnt_kern_flag & MNTK_UNMOUNTF) != 0)
> > + return (0);
> > +
> Is there anything that would prevent the MNTK_UNMOUNTF flag from being
> set immediately after the test returned success ?
> 
> Usually, the tests for MNTK_UNMOUNTF are bugs. I coould see how
> terminating
> the RPCs would be useful for forced unmounts, but do not think that
> preventing entry into close would help.
> 
I think you are correct, in that the one in nfsrpc_close() is harmless,
but not needed. (I suspect I added it to fix a crash and then didn't
realize that it's the one in nfscl_getcl() that matters.)

Basically, ncl_inactive() would call nfsrpc_close() which calls
nfscl_doclose() which calls nfscl_getcl(), so when nfscl_getcl()
returns failure, that should be sufficient.

As for the nfscl_getcl() one, I don't think it is currently quite
right either. The problem is a race with nfscl_umount(), which gets
called from nfs_unmount().

It goes like this:
- for a forced dismount, nfs_unmount():
  - cancels all in-progress RPCs plus all new RPC attempts will fail
    {newnfs_nmcancelreqs() does this }
  - calls nfscl_umount()
    --> nfscl_umount() gets the exclusive lock for open/lock state
        modifications.
        - igotlock = nfsv4_lock(&clp->nfsc_lock,...);
        This will delay while any refcnt (shared lock) on nfsc_lock
        is held.
        - once this exclusive lock is acquired, it zaps the state
          information (any RPC attempts simply fail, due to the
          above)

- in nfscl_getcl(), it gets a refcnt (shared lock) on nfsc_lock, so
  once that happens, the above will block until it is released.

--> My mistake was putting the test for MNTK_UNMOUNTF above that point
    in nfscl_getcl(), so there is a race if it is set after the test,
    but before acquiring the refcnt on nfsc_lock.

Thanks for pointing this out so I took another look. I'll either revert
the patch or add one that fixes the above. (ie. move the test in nfscl_getcl()
down and probably get rid of the one in nfsrpc_close(), since the test in
nfscl_getcl() should be sufficient.)

rick