VFS: C99 sparse format for struct vfsops

[M. Warner Losh]

In message: <20030603155157.GH35187 at survey.codeburst.net>
            Paul Richards <paul at freebsd-services.com> writes:
: I'm not sure that kobj actually needs to be MP safe if the kobj
: struct is always embedded in a structure at a higher level i.e.
: a use of kobj in say the device driver code will not interact with
: it's use by say the linker code so the locking can be done on device
: driver or linker level structs.

Lock lock data, not code.

The kobj dispatch tables are shared between all instances of the
object.  So if you have two instances of the driver, the driver locks
protect softc for that driver.  However, both softc's reference the
kobj structures, directly or indirectly, that are not MP safe.  If
instance 1 takes out a lock, that doesn't prevent instance 2 from
getting screwed by the following code:

#define KOBJOPLOOKUP(OPS,OP) do {					\
	kobjop_desc_t _desc = &OP##_##desc;				\
	kobj_method_t *_ce =						\
	    &OPS->cache[_desc->id & (KOBJ_CACHE_SIZE-1)];		\
	if (_ce->desc != _desc) {					\
		KOBJOPMISS;						\
[1]		kobj_lookup_method(OPS->cls->methods, _ce, _desc);	\
	} else {							\
		KOBJOPHIT;						\
	}								\
[2]	_m = _ce->func;							\
} while(0)

Notice how OPS has a cache array.  If instance 1 is calling the same
ID as instance 2, modulo KOBJ_CACHE_SIZE, then a collision occurs and
a race happens betwen [1] and [2]:

thread 1		  thread 2

_ce = ...
			_ce = ...
HIT
			MISS
			set's *_ce via kobj_lookup_method
_m gets set
_m called
			_m set
			_m called


Notice how thread 1's _m gets set based on the results of the kobj
lookup, and we have a race, even if thread1 and thread2 took out their
driver instance locks.

Warner