kern/76432: recursive locking in the network stack

[Hans Petter Selasky]

>Number:         76432
>Category:       kern
>Synopsis:       recursive locking in the network stack
>Confidential:   no
>Severity:       non-critical
>Priority:       medium
>Responsible:    freebsd-bugs
>State:          open
>Quarter:        
>Keywords:       
>Date-Required:
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Tue Jan 18 21:00:46 GMT 2005
>Closed-Date:
>Last-Modified:
>Originator:     hselasky at c2i.net
>Release:        FreeBSD 5.3-RC1 i386
>Organization:
>Environment:
System: FreeBSD 5.3-RC1 FreeBSD 5.3-RC1 #182: Fri Jan 14 13:45:31 CET 2005 
root@ :/usr/obj/usr/src/sys/custom i386

>Description:
 1) lock with name "rtentry" can recurse at line 197 in the 
file /usr/src/sys/net/route.c, which causes a panic

Backtrace:

panic()
_mtx_lock_sleep()
_mtx_lock_flags()
rtalloc1()
ifa_ifwithroute()
rt_getifa()
route_output()
raw_usend()
rts_send()
sosend()
soo_write()
dofilewrite()
write()
syscall()

 2) Adding flag MTX_RECURSE to mtx_init(), in the file "src/sys/net/route.h" 
leads to another bug:

lock order reversal:

1st rtentry @ /usr/src/sys/net/rtsock.c:429
2nd radix node head @ /usr/src/sys/net/route.c:148

>How-To-Repeat:
 run "ppp" after "dhclient"

>Fix:
 1) run "route delete 0.0.0.0" before running ppp

 2) patch for route.h
*** /usr/src/sys/net/route.h.ref        Tue Jan 18 21:16:05 2005
--- /usr/src/sys/net/route.h    Tue Jan 18 21:17:32 2005
***************
*** 280,286 ****
  #ifdef _KERNEL
  
  #define       RT_LOCK_INIT(_rt) \
!       mtx_init(&(_rt)->rt_mtx, "rtentry", NULL, MTX_DEF | MTX_DUPOK)
  #define       RT_LOCK(_rt)            mtx_lock(&(_rt)->rt_mtx)
  #define       RT_UNLOCK(_rt)          mtx_unlock(&(_rt)->rt_mtx)
  #define       RT_LOCK_DESTROY(_rt)    mtx_destroy(&(_rt)->rt_mtx)
--- 280,286 ----
  #ifdef _KERNEL
  
  #define       RT_LOCK_INIT(_rt) \
!       mtx_init(&(_rt)->rt_mtx, "rtentry", NULL, MTX_DEF | MTX_DUPOK | 
MTX_RECURSE)
  #define       RT_LOCK(_rt)            mtx_lock(&(_rt)->rt_mtx)
  #define       RT_UNLOCK(_rt)          mtx_unlock(&(_rt)->rt_mtx)
  #define       RT_LOCK_DESTROY(_rt)    mtx_destroy(&(_rt)->rt_mtx)
>Release-Note:
>Audit-Trail:
>Unformatted: