proc lock might become an sx lock?

[John Baldwin]

On Thursday 08 March 2007 10:18:13 pm Julian Elischer wrote:
> currently the thread list in the process is protected by the sched lock.
> for a process with a lot of threads this is probably not a good idea.
> I experimented with making it protected by the proc loc, but the following 
> sort of thing happens a lot:
> 
>         sx_slock(&allproc_lock);
>         FOREACH_PROC_IN_SYSTEM(p) {
>                 mtx_lock_spin(&sched_lock);
>                 FOREACH_THREAD_IN_PROC(p, td) {
>                         ...
>                 }
>                 mtx_unlock_spin(&sched_lock);
> 
> Changing the protection of the thread list to use the proc lock would
> replace the sched_lock with the proc lock, but.....
> this has a problem.. the proc lock is a mutex and can therefore not be 
inside the
> allproc_lock.
> 
> and in fact you get:
> 
>  Trying to mount root from ufs:/dev/aacd0s1d
>  panic: blockable sleep lock (sleep mutex) process lock @ 
kern/sched_4bsd.c:383
>  cpuid = 2
>  KDB: enter: panic
>  [thread pid 48 tid 100054 ]
>  Stopped at      kdb_enter+0x2b: nop
>  db> bt
>  Tracing pid 48 tid 100054 td 0xc5ff4a20
>  kdb_enter(c06ce300) at kdb_enter+0x2b
>  panic(c06d3506,c06e7061,c06cd73b,c06cff47,17f,...) at panic+0x11c
>  witness_checkorder(c60a12c8,9,c06cff47,17f) at witness_checkorder+0xb8
>  _mtx_lock_flags(c60a12c8,0,c06cff3e,17f,85,...) at _mtx_lock_flags+0x87
>  schedcpu(e65a9d24,c0516f50,0,e65a9d38,c6259000,...) at schedcpu+0x80
>  schedcpu_thread(0,e65a9d38) at schedcpu_thread+0x9
> 
> My reading of the man page is that making it an sx lock and locking it in
> shared mode would be sufficient for this sort of thing (assuming we are not 
changing
> the thread list) and would be just fine.
> 
> I'm not very familiar with the implementation of sx locks in freeBSD so I'm 
just learning
> about them. 
> 
> am I reading this right? and does anyone else have any thoughts on this?

Use rwlocks to make a mutex have reader/writer semantics but still fit into 
the current lock order.  However, you likely should coordinate sched_lock 
changes like this with Attilio and Jeff R first as they are removing 
sched_lock and already have substantial diffs.

-- 
John Baldwin