geom_vfs: disallow multiple readonly mounts of a device

[Andriy Gapon]

Currently FreeBSD allows multiple readonly mounts of the same device.
This feature seems to be introduced along with GEOM.  Before that, only one mount
of a device was allowed, whether readonly or not.  Other (major) BSDs still work
that way.
Unfortunately, the feature has never really worked correctly because of some
architectural/design issues in our buffer cache layer/interface.
Multiple shared mounts are allowed to happen but later on, during filesystem
access or at unmount time, a system would crash.

Because of this, I propose to disable shared mounting feature for time being.
In my opinion it is very unlikely that anybody depends on this feature now, but
nullfs can be used as its replacement.  In fact, it could be even more efficient
if the same files are access via different mount points.

The proposed patch is a greatly reduced version of a patch by Pawel Jakub Dawidek:
http://people.freebsd.org/~pjd/patches/mro_mount.patch
Pawel's patch, in fact, fixes the shared mounting feature.
Unfortunately, at least one filesystem (FFS) is quite intrusive at what it does
with a device vnode (or rather its bufobj) when it attempts mounting.
So, in some edge cases (synthetic or accidental) a system may still crash even
with the Pawel's patch.  So, I think that it should not be committed until after
all filesystems in the tree are made to play nice.  Otherwise, I like it.

As an afterthought, perhaps it is the Pawel's patch that we should actually use
with one change - add a sysctl that enables/disables shared mounts and make them
disabled by default.

Thank you very for the feedback.

--- a/sys/geom/geom_vfs.c
+++ b/sys/geom/geom_vfs.c
@@ -161,6 +161,10 @@ g_vfs_open
 	g_topology_assert();

 	*cpp = NULL;
+	bo = &vp->v_bufobj;
+	if (bo->bo_private != vp)
+		return (EBUSY);
+
 	pp = g_dev_getprovider(vp->v_rdev);
 	if (pp == NULL)
 		return (ENOENT);
@@ -176,6 +180,6 @@ g_vfs_open
 	vnode_create_vobject(vp, pp->mediasize, curthread);
 	VFS_UNLOCK_GIANT(vfslocked);
 	*cpp = cp;
-	bo = &vp->v_bufobj;
+	cp->private = vp;
 	bo->bo_ops = g_vfs_bufops;
 	bo->bo_private = cp;
@@ -196,5 +200,6 @@ g_vfs_close
 	gp = cp->geom;
 	bo = gp->softc;
 	bufobj_invalbuf(bo, V_SAVE, 0, 0);
+	bo->bo_private = cp->private;
 	g_wither_geom_close(gp, ENXIO);
 }

-- 
Andriy Gapon