LOR on kldunload nve/radix node head (v6?)

[John Baldwin]

On Sunday 30 October 2005 04:37 am, Bjoern A. Zeeb wrote:
> Hi,
>
> upon kldunload if_nve I got this. I added it with LOR ID # 166:
> 	http://sources.zabbadoz.net/freebsd/lor.html#166
>
> lock order reversal:
>  1st 0xffffff00009a7180 nve0 (network driver) @
> sys/modules/nve/../../dev/nve/if_nve.c:674 2nd 0xffffff0000b234f0 radix
> node head (radix node head) @ sys/net/route.c:148 KDB: stack backtrace:
> kdb_backtrace() at kdb_backtrace+0x37
> witness_checkorder() at witness_checkorder+0x6f2
> _mtx_lock_flags() at _mtx_lock_flags+0x7a
> rtalloc1() at rtalloc1+0x94
> in6_ifremloop() at in6_ifremloop+0x6d
> in6_purgeaddr() at in6_purgeaddr+0x8b
> if_purgeaddrs() at if_purgeaddrs+0xba
> if_detach() at if_detach+0x31
> ether_ifdetach() at ether_ifdetach+0x79
> nve_detach() at nve_detach+0x240
> device_detach() at device_detach+0x85
> devclass_delete_driver() at devclass_delete_driver+0xb7
> driver_module_handler() at driver_module_handler+0xec
> module_unload() at module_unload+0x56
> linker_file_unload() at linker_file_unload+0x93
> kern_kldunload() at kern_kldunload+0xa2
> kldunloadf() at kldunloadf+0x19
> syscall() at syscall+0x350
> Xfast_syscall() at Xfast_syscall+0xa8
> --- syscall (444, FreeBSD ELF64, kldunloadf),
>     rip = 0x800678c9c, rsp = 0x7fffffffe4d8, rbp = 0x7fffffffe980 ---

The locking in this driver is broken.  This LOR is just one symptom.  Let me 
work up a patch.  BTW, this driver probably should not be using a spin lock 
since it doesn't have an INTR_FAST handler.  Actually, this driver has _lots_ 
of locking issues.  It wasn't even locking the driver when transmitting 
packets handed to it by the stack!  Try this patch:

http://www.FreeBSD.org/~jhb/patches/nve_locking.patch

Also, I have some debug printf()s in there because I think we can stub out the 
osmtx entirely.  Let me know if you get any printf's on the console that 
include "normal mutex not held".  Thanks!

-- 
John Baldwin <jhb at FreeBSD.org>  <><  http://www.FreeBSD.org/~jhb/
"Power Users Use the Power to Serve"  =  http://www.FreeBSD.org