NFS inuse vnode sillyrename with > 1 links

[Bryan Drewery]

The context for this is in https://reviews.freebsd.org/D6254 with some
discussion already.

First my use-case:

cd some_nfs_dir/
mkdir src dst
touch src/foo
ln src/foo dst/foo
tail -F dst/foo &
rm -rf src/

This fails in rm/FTS (Directory not empty) in rmdir(1) since the removal
of src/foo renames it to src/.nfs*.  Then FTS tries to rmdir not
realizing there is a new file.

My real use-case is building Python in a temporary directory,
hardlinking it into a staging directory (for running from without
polluting /).  I then run the staging version of python for the rest of
the build.  If I try to rebuild Python while something else is running
the hard-linked Python from the staging directory then the Python build
fails since it is trying to rm -Rf a directory that just got a silly
rename, even though I was executing a different named link.  I
understand that a vnode inuse count is on the vnode and not the name,
but in this case I expect rm -Rf to just work since I wasn't running the
link being deleted.  Implementation complexities and history aside, this
should just work, even if it means a hack to FTS/rm rather than
nfs_remove.  It makes no sense from a user-perspective and has no sane
workaround except to not hardlink or use userland r/w locking to avoid
removing a binary in-use.

So to NFS.  I know nothing about the protocol or history.  I just know
that in this case it seems fine to not do a silly rename on a file if it
has multiple links, since the vnode will still be around to be used by
the kernel for in-use consumers.

-- 
Regards,
Bryan Drewery