stable kqueue locking up and running on SMP

Brian Fundakowski Feldman green at
Tue Apr 20 21:39:54 PDT 2004

Please test out and provide feedback for the latest iteration:
Things are now working in exactly the way I described I was going to 
implement them -- there is one kqueue lock, and if, as I suspect, it will 
_not_ be a point of contention, there is no reason at all to complicate 
things further.

To do finer-grained locking, klists will have to be overhauled to become an 
actual, protected, object instead of hanging along with the object they 
represent.  More complication in the form of read-locking (or holds) will 
also be necessary for that.  For now, though, they belong to _kqueue_, and 
not struct selinfo or the object that contains them normally.  A more 
complicated scheme for kqueues, knotes, klist(-alike)s would benefit greatly 
from divorce of kqueue from struct filedesc, if anyone is ever to tackle 
that (if there is ever a return on investment; MUTEX_PROFILING should help 
us figure that out.)

The only known issue on my SMP box is this warning at boot time; I am unable 
to track down why witness(4) believes there to be a lock order reversal, but 
I welcome someone else to help identify this one:
lock order reversal
 1st 0xc0661500 global kqueue lock (global kqueue lock) @ kern/kern_event.c:413
 2nd 0xc45bc438 filedesc structure (filedesc structure) @ kern/kern_event.c:422
Stack backtrace:
backtrace(c0616b80,c45bc438,c060fd6e,c060fd6e,c0610941) at backtrace+0x17
witness_checkorder(c45bc438,9,c0610941,1a6,c455d594) at witness_checkorder+0x6f6
_mtx_lock_flags(c45bc438,0,c0610938,1a6,c455d594) at _mtx_lock_flags+0x9a
kqueue(c45b73f0,db138d14,c19970ec,8133000,0) at kqueue+0x120
syscall(2f,2f,2f,8133000,2d) at syscall+0x272
Xint0x80_syscall() at Xint0x80_syscall+0x1f
--- syscall (362), eip = 0x282a7b8f, esp = 0xbfbfbcac, ebp = 0xbfbfc368 ---

The major stress testers I know about are using make -j with -DUSE_KQUEUE 
compilation flags on make(1) and src/tools/regression/gaithstress.  I notice 
no problems, but I haven't also tested to make sure nested kqueues are doing 
what is expected.  The only missing feature is the ill-conceived NOTE_TRACK. 
I do not think that returning for that one note type EINVAL is a deal-breaker
when trying to make kqueue not suck for 5.3; it is far more useful as a 
novelty than utility, especially considering I've never seen mention of it 
in actual software that uses kqueue.

Brian Fundakowski Feldman                           \'[ FreeBSD ]''''''''''\
  <> green at                               \  The Power to Serve! \
 Opinions expressed are my own.                       \,,,,,,,,,,,,,,,,,,,,,,\

More information about the freebsd-arch mailing list