kern/55175: LOR in select and poll

Kris Kennaway kris at obsecurity.org
Fri Aug 1 20:20:15 PDT 2003 

>Number:         55175
>Category:       kern
>Synopsis:       LOR in select and poll
>Confidential:   no
>Severity:       serious
>Priority:       low
>Responsible:    freebsd-bugs
>State:          open
>Quarter:        
>Keywords:       
>Date-Required:
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Fri Aug 01 20:20:13 PDT 2003
>Closed-Date:
>Last-Modified:
>Originator:     Kris Kennaway
>Release:        FreeBSD 5.1-CURRENT
>Organization:
FreeBSD
>Environment:
System: FreeBSD enigma.obsecurity.org 5.1-CURRENT FreeBSD 5.1-CURRENT #0: Sat Jul 19 18:37:43 PDT 2003 kkenn at enigma.obsecurity.org:/usr/src/sys/sparc64/compile/ENIGMA sparc64


>Description:
	Since upgrading the bento i386 package build machines to
5.1-CURRENT, the following two lock order reversals have been seen.

lock order reversal
 1st 0xc6c1c334 filedesc structure (filedesc structure) @
+/a/asami/portbuild/i386/src-client/sys/kern/sys_generic.c:902
 2nd 0xc04aa120 Giant (Giant) @ /a/asami/portbuild/i386/src-client/sys/fs/specfs/spec_vnops.c:372
Stack backtrace:
backtrace(c043d4af,c04aa120,c0439aa4,c0439aa4,c0434e3d) at backtrace+0x17
witness_lock(c04aa120,8,c0434e3d,174,1bc) at witness_lock+0x672
_mtx_lock_flags(c04aa120,0,c0434e3d,174,c043daba) at _mtx_lock_flags+0xba
spec_poll(d8dddaf8,d8dddb18,c02d119c,d8dddaf8,c04939a0) at spec_poll+0x134
spec_vnoperate(d8dddaf8,c04939a0,c520b124,40,c675e300) at spec_vnoperate+0x18
vn_poll(c44c5e14,40,c675e300,c6222d10,c675e300) at vn_poll+0x3c
selscan(c6222d10,d8dddb98,d8dddb88,6,4) at selscan+0x13e
kern_select(c6222d10,6,bfbff5c0,0,0) at kern_select+0x36f
select(c6222d10,d8dddd10,c0455899,3ee,5) at select+0x66
syscall(2f,2f,2f,8055050,bfbff5b8) at syscall+0x273
Xint0x80_syscall() at Xint0x80_syscall+0x1d
--- syscall (93), eip = 0x280ccacc, esp = 0x2832eb68, ebp = 0x2832ebc0 ---
Debugger("witness_lock")
Stopped at      Debugger+0x54:  xchgl   %ebx,in_Debugger.0

#8  0xc0290ed7 in witness_lock (lock=0xc04aa120, flags=8,
    file=0xc0434e3d "/a/asami/portbuild/i386/src-client/sys/fs/specfs/spec_vnops.c", line=372)
    at /a/asami/portbuild/i386/src-client/sys/kern/subr_witness.c:838
#9  0xc0261f4a in _mtx_lock_flags (m=0x0, opts=0, file=0xc04d17a8 "", line=-1068850912)
    at /a/asami/portbuild/i386/src-client/sys/kern/kern_mutex.c:334
#10 0xc0231154 in spec_poll (ap=0xd8dddaf8)
    at /a/asami/portbuild/i386/src-client/sys/fs/specfs/spec_vnops.c:372
#11 0xc0230648 in spec_vnoperate (ap=0x0)
    at /a/asami/portbuild/i386/src-client/sys/fs/specfs/spec_vnops.c:122
#12 0xc02d119c in vn_poll (fp=0x0, events=0, active_cred=0xc675e300, td=0x0) at vnode_if.h:537
#13 0xc02945ae in selscan (td=0xc6222d10, ibits=0xd8dddb98, obits=0xd8dddb88, nfd=6)
    at /a/asami/portbuild/i386/src-client/sys/sys/file.h:272
#14 0xc029412f in kern_select (td=0xc6222d10, nd=6, fd_in=0xbfbff5c0, fd_ou=0x0, fd_ex=0x0, tvp=0xd8dddcd4)
    at /a/asami/portbuild/i386/src-client/sys/kern/sys_generic.c:822
#15 0xc0293da6 in select (td=0x0, uap=0xd8dddd10)
    at /a/asami/portbuild/i386/src-client/sys/kern/sys_generic.c:726
#16 0xc03ef9b3 in syscall (frame=
      {tf_fs = 47, tf_es = 47, tf_ds = 47, tf_edi = 134565968, tf_esi = -1077938760, tf_ebp = 674425792,
+tf_isp = -656548492, tf_ebx = 0, tf_edx = -1077938752, tf_ecx = 0, tf_eax = 93, tf_trapno = 12, tf_err = 2,
+tf_eip = 671926988, tf_cs = 31, tf_eflags = 534, tf_esp = 674425704, tf_ss = 47})
    at /a/asami/portbuild/i386/src-client/sys/i386/i386/trap.c:1008
#17 0xc03dfbed in Xint0x80_syscall () at {standard input}:144
---Can't read userspace from dump, or kernel process---

lock order reversal
 1st 0xc6a69634 filedesc structure (filedesc structure) @
+/a/asami/portbuild/i386/src-client/sys/kern/sys_generic.c:1071
 2nd 0xc04aa120 Giant (Giant) @ /a/asami/portbuild/i386/src-client/sys/fs/specfs/spec_vnops.c:372
Stack backtrace:
backtrace(c043d4af,c04aa120,c0439aa4,c0439aa4,c0434e3d) at backtrace+0x17
witness_lock(c04aa120,8,c0434e3d,174,246) at witness_lock+0x672
_mtx_lock_flags(c04aa120,0,c0434e3d,174,c043daba) at _mtx_lock_flags+0xba
spec_poll(d8dfcb44,d8dfcb64,c02d119c,d8dfcb44,c04939a0) at spec_poll+0x134
spec_vnoperate(d8dfcb44,c04939a0,c52cfa44,41,c6cfd280) at spec_vnoperate+0x18
vn_poll(c45dc880,41,c6cfd280,c5f7a4c0,c6cfd280) at vn_poll+0x3c
pollscan(c5f7a4c0,d8dfcbd4,2,3e7,10) at pollscan+0xb0
poll(c5f7a4c0,d8dfcd10,c0455899,3ee,3) at poll+0x252
syscall(2f,2f,2f,0,2) at syscall+0x273
Xint0x80_syscall() at Xint0x80_syscall+0x1d

#8  0xc0290ed7 in witness_lock (lock=0xc04aa120, flags=8,
    file=0xc0434e3d "/a/asami/portbuild/i386/src-client/sys/fs/specfs/spec_vnops.c", line=372)
    at /a/asami/portbuild/i386/src-client/sys/kern/subr_witness.c:838
#9  0xc0261f4a in _mtx_lock_flags (m=0x0, opts=0, file=0xc04d1818 "", line=-1068850912)
    at /a/asami/portbuild/i386/src-client/sys/kern/kern_mutex.c:334
#10 0xc0231154 in spec_poll (ap=0xd8dfcb44)
    at /a/asami/portbuild/i386/src-client/sys/fs/specfs/spec_vnops.c:372
#11 0xc0230648 in spec_vnoperate (ap=0x0)
    at /a/asami/portbuild/i386/src-client/sys/fs/specfs/spec_vnops.c:122
#12 0xc02d119c in vn_poll (fp=0x0, events=0, active_cred=0xc6cfd280, td=0x0) at vnode_if.h:537
#13 0xc0294c50 in pollscan (td=0xc5f7a4c0, fds=0xd8dfcbdc, nfd=2)
    at /a/asami/portbuild/i386/src-client/sys/sys/file.h:272
#14 0xc02948a2 in poll (td=0xc5f7a4c0, uap=0xd8dfcd10)
    at /a/asami/portbuild/i386/src-client/sys/kern/sys_generic.c:1001
#15 0xc03ef9b3 in syscall (frame=
      {tf_fs = 47, tf_es = 47, tf_ds = 47, tf_edi = 0, tf_esi = 2, tf_ebp = -1077940448, tf_isp = -656421516,
+tf_ebx = 673224876, tf_edx = 139153408, tf_ecx = 137314336, tf_eax = 209, tf_trapno = 0, tf_err = 2, tf_eip =
+672942388, tf_cs = 31, tf_eflags = 659, tf_esp = -1077940508, tf_ss = 47})
    at /a/asami/portbuild/i386/src-client/sys/i386/i386/trap.c:1008
#16 0xc03dfbed in Xint0x80_syscall () at {standard input}:144
---Can't read userspace from dump, or kernel process---

The original mail may be found here:

http://docs.freebsd.org/cgi/getmsg.cgi?fetch=73697+0+current/freebsd-current
Message-ID: <20030727233351.GB80934 at rot13.obsecurity.org>

rwatson replied with:

> I've bumped into some similar problems -- it's a property of how we
> current lock select().  We hold the file descriptor lock for the duration
> of polling each object being "selected", and if any of those objects has
> to grab a lock for any reason, it has to implicitly fall after the file
> descriptor lock.  I actually run into this in some of our MAC code,
> because I need to grab a vnode lock to authorize polling the vnode using
> VOP_POLL(), and since the vnode lock is a sleep lock, this generates a
> WITNESS warning.  Unfortunately, it's not immediately clear what a better
> locking scheme would look like without going overboard on the fine-grained
> side.  We probably need to grab Giant before entering the select code
> since it's highly likely something in there will require Giant -- it
> reaches down into VFS, the device stuff, socket code, tc.

>How-To-Repeat:
	Unknown.

>Fix:


>Release-Note:
>Audit-Trail:
>Unformatted:

More information about the freebsd-bugs mailing list