kern/136945: [ufs] [lor] filedesc structure/ufs (poll)

[Rene Ladan]

2009/7/29 John Baldwin <jhb at freebsd.org>:
> On Wednesday 29 July 2009 5:52:24 am Rene Ladan wrote:
>> 2009/7/28 John Baldwin <jhb at freebsd.org>:
>> > On Tuesday 28 July 2009 10:03:40 am Rene Ladan wrote:
>> >> 2009/7/28 John Baldwin <jhb at freebsd.org>:
>> >> > On Monday 27 July 2009 10:00:05 am Rene Ladan wrote:
>> >> >> The following reply was made to PR kern/136945; it has been noted by
>> > GNATS.
>> >> >>
>> >> >> From: Rene Ladan <rene at freebsd.org>
>> >> >> To: John Baldwin <jhb at freebsd.org>
>> >> >> Cc: bug-followup at freebsd.org
>> >> >> Subject: Re: kern/136945: [ufs] [lor] filedesc structure/ufs (poll)
>> >> >> Date: Mon, 27 Jul 2009 15:51:15 +0200
>> >> >>
>> >> >>  2009/7/27 John Baldwin <jhb at freebsd.org>:
>> >> >>  > I would actually expect this to be the correct order for these two
>> >> > locks.=
>> >> >>   =A0Can
>> >> >>  > you capture the output of the 'debug.witness.fullgraph' sysctl to a
>> > file?
>> >> >>  >
>> >> >>  Yes, see attachment.  I'm still running the same 8.0-BETA2.
>> >> >
>> >> > Hmm, the attachment was eaten by a grue, can you post the file
> somewhere?
>> >> >
>> >> Yes, see ftp://rene-ladan.nl/pub/freebsd/kern_136945.txt
>> >
>> > Ok, it looks like it did encounter a UFS -> filedesc order at some
> point.  Can
>> > you patch sys/kern/subr_witness.c to add a section to the order_lists[]
> array
>> > after the 'ZFS locking list' and before the spin locks list that looks
> like
>> > this:
>> >
>> >        { "filedesc structure", &lock_class_sx },
>> >        { "ufs", &lock_class_lockmgr},
>> >        { NULL, NULL },
>> >
>> The LOR seems to be gone, previously it showed up only once right
>> after booting the system.
>>
>> But now a new LOR (according to the LOR page) seems pop up:
>> Trying to mount root from ufs:/dev/ad0s1a
>> lock order reversal:
>>  1st 0xffffff0002a4ad80 ufs (ufs) @ /usr/src/sys/ufs/ffs/ffs_vfsops.c:1465
>>  2nd 0xffffff0002b29a48 filedesc structure (filedesc structure) @
>> /usr/src/sys/kern/kern_descrip.c:2478
>> KDB: stack backtrace:
>> db_trace_self_wrapper() at db_trace_self_wrapper+0x2a
>> _witness_debugger() at _witness_debugger+0x49
>> witness_checkorder() at witness_checkorder+0x7ea
>> _sx_xlock() at _sx_xlock+0x44
>> mountcheckdirs() at mountcheckdirs+0x80
>> vfs_donmount() at vfs_donmount+0xfbf
>> kernel_mount() at kernel_mount+0xa1
>> vfs_mountroot_try() at vfs_mountroot_try+0x177
>> vfs_mountroot() at vfs_mountroot+0x47d
>> start_init() at start_init+0x62
>> fork_exit() at fork_exit+0x12a
>> fork_trampoline() at fork_trampoline+0xe
>> --- trap 0, rip = 0, rsp = 0xffffff800001ad30, rbp = 0 ---
>>
>> The output of `df' and `mount' looks ok.
>
> Yes, this is the "real" LOR as "filedesc" -> "ufs" in the poll() case should
> be the normal order.  I believe this should fix it.  mountcheckdirs() doesn't
> need the vnodes locked, it just needs the caller to hold references on them
> so they aren't recycled:
>
> --- //depot/projects/smpng/sys/kern/vfs_mount.c#96
> +++ /home/jhb/work/p4/smpng/sys/kern/vfs_mount.c
> @@ -1069,9 +1069,10 @@
>                vfs_event_signal(NULL, VQ_MOUNT, 0);
>                if (VFS_ROOT(mp, LK_EXCLUSIVE, &newdp))
>                        panic("mount: lost mount");
> +               VOP_UNLOCK(newdp, 0);
> +               VOP_UNLOCK(vp, 0);
>                mountcheckdirs(vp, newdp);
> -               vput(newdp);
> -               VOP_UNLOCK(vp, 0);
> +               vrele(newdp);
>                if ((mp->mnt_flag & MNT_RDONLY) == 0)
>                        error = vfs_allocate_syncvnode(mp);
>                vfs_unbusy(mp);
>
The LOR is still present, but at a different place without the
mountcheckdirs() call (not on the LOR page either) :

Trying to mount root from ufs:/dev/ad0s1a
lock order reversal:
 1st 0xffffff0002a4ad80 ufs (ufs) @ /usr/src/sys/kern/vfs_subr.c:2083
 2nd 0xffffff000233f048 filedesc structure (filedesc structure) @
/usr/src/sys/kern/vfs_mount.c:1485
KDB: stack backtrace:
db_trace_self_wrapper() at db_trace_self_wrapper+0x2a
_witness_debugger() at _witness_debugger+0x49
witness_checkorder() at witness_checkorder+0x7ea
_sx_xlock() at _sx_xlock+0x44
set_rootvnode() at set_rootvnode+0x57
vfs_mountroot_try() at vfs_mountroot_try+0x371
vfs_mountroot() at vfs_mountroot+0x47d
start_init() at start_init+0x62
fork_exit() at fork_exit+0x12a
fork_trampoline() at fork_trampoline+0xe
--- trap 0, rip = 0, rsp = 0xffffff800001ad30, rbp = 0 ---

René