[Bug 290550] LOR: lock order reversal - zfs and devfs/nullfs

From: <bugzilla-noreply_at_freebsd.org>
Date: Sun, 26 Oct 2025 22:00:22 UTC
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=290550

            Bug ID: 290550
           Summary: LOR: lock order reversal - zfs and devfs/nullfs
           Product: Base System
           Version: 14.3-STABLE
          Hardware: Any
                OS: Any
            Status: New
          Severity: Affects Only Me
          Priority: ---
         Component: kern
          Assignee: bugs@FreeBSD.org
          Reporter: vova@fbsd.ru

On shutdown of the system with jails on kernel with std.debug:

jails-astart: auto-stop jail: www
lock order reversal:
 1st 0xfffff803c8203af0 zfs (zfs, lockmgr) @
/usr/src/sys/fs/nullfs/null_vnops.c:786
 2nd 0xfffff80035c383f0 devfs (devfs, lockmgr) @
/usr/src/sys/kern/vfs_subr.c:3320
lock order zfs -> devfs attempted at:
#0 0xffffffff80b830b1 at witness_checkorder+0xbe1
#1 0xffffffff80ada431 at lockmgr_xlock+0x51
#2 0xffffffff8110a4ca at VOP_LOCK1_APV+0x3a
#3 0xffffffff80c25c33 at _vn_lock+0x53
#4 0xffffffff80c0b85f at vget_finish+0xaf
#5 0xffffffff8098df9e at devfs_allocv+0xbe
#6 0xffffffff8098d693 at devfs_root+0x43
#7 0xffffffff80c1146a at vfs_cache_root_fallback+0xea
#8 0xffffffff80c0ccbb at vflush+0x4b
#9 0xffffffff8098d58e at devfs_unmount+0x3e
#10 0xffffffff80c00e08 at dounmount+0x658
#11 0xffffffff80c0074c at kern_unmount+0x2ec
#12 0xffffffff81041f0a at amd64_syscall+0x15a
#13 0xffffffff81013b2b at fast_syscall_common+0xf8
lock order reversal:
 1st 0xfffff800358cd3f0 nullfs (nullfs, lockmgr) @
/usr/src/sys/kern/vfs_mount.c:2264
 2nd 0xfffff800358cd770 zfs (zfs, lockmgr) @
/usr/src/sys/fs/nullfs/null_vnops.c:786
lock order zfs -> nullfs established at:
#0 0xffffffff80b82834 at witness_checkorder+0x364
#1 0xffffffff80ad913b at lockmgr_xlock_hard+0x6b
#2 0xffffffff80ad99ff at __lockmgr_args+0x1ff
#3 0xffffffff83589c20 at null_reclaim+0x60
#4 0xffffffff8110a32d at VOP_RECLAIM_APV+0x5d
#5 0xffffffff80c0d9f4 at vgonel+0x474
#6 0xffffffff80c0e379 at vrecyclel+0x69
#7 0xffffffff80c0e2ef at vrecycle+0x2f
#8 0xffffffff83589b60 at null_inactive+0x40
#9 0xffffffff8110a00d at VOP_INACTIVE_APV+0x5d
#10 0xffffffff80c0cb12 at vinactivef+0xc2
#11 0xffffffff80c0c93b at vinactive+0x8b
#12 0xffffffff80c0c0fc at vput_final+0x29c
#13 0xffffffff80c21c31 at kern_frmdirat+0x271
#14 0xffffffff81041f0a at amd64_syscall+0x15a
#15 0xffffffff81013b2b at fast_syscall_common+0xf8
lock order nullfs -> zfs attempted at:
#0 0xffffffff80b830b1 at witness_checkorder+0xbe1
#1 0xffffffff80ada431 at lockmgr_xlock+0x51
#2 0xffffffff8110a4ca at VOP_LOCK1_APV+0x3a
#3 0xffffffff83589db8 at null_lock+0xb8
#4 0xffffffff8110a4ca at VOP_LOCK1_APV+0x3a
#5 0xffffffff80c25c33 at _vn_lock+0x53
#6 0xffffffff80c0cd59 at vflush+0xe9
#7 0xffffffff83588ca0 at nullfs_unmount+0x50
#8 0xffffffff80c00e08 at dounmount+0x658
#9 0xffffffff80c0074c at kern_unmount+0x2ec
#10 0xffffffff81041f0a at amd64_syscall+0x15a
#11 0xffffffff81013b2b at fast_syscall_common+0xf8
jails-astart: auto-stop jail: nnvfilm
jails-astart: auto-stop jail: px
44373jails-astart: auto-stop jail: zabbix
jails-astart: auto-stop jail: hat
jails-astart: auto-stop jail: tg
jails-astart: auto-stop jail: smb
lock order reversal:
 1st 0xfffff8036de1e770 zfs (zfs, lockmgr) @
/usr/src/sys/fs/nullfs/null_vnops.c:786
 2nd 0xfffff804602b5770 fdescfs (fdescfs, lockmgr) @
/usr/src/sys/kern/vfs_subr.c:3320
lock order zfs -> fdescfs attempted at:
#0 0xffffffff80b830b1 at witness_checkorder+0xbe1
#1 0xffffffff80ad8a24 at lockmgr_lock_flags+0x184
#2 0xffffffff8110a4ca at VOP_LOCK1_APV+0x3a
#3 0xffffffff80c25c33 at _vn_lock+0x53
#4 0xffffffff80c0b85f at vget_finish+0xaf
#5 0xffffffff83420262 at fdesc_root+0x22
#6 0xffffffff80c0ccbb at vflush+0x4b
#7 0xffffffff8342020e at fdesc_unmount+0x6e
#8 0xffffffff80c00e08 at dounmount+0x658
#9 0xffffffff80c0074c at kern_unmount+0x2ec
#10 0xffffffff81041f0a at amd64_syscall+0x15a
#11 0xffffffff81013b2b at fast_syscall_common+0xf8
Oct 27 00:54:17 srv syslogd: exiting on signal 15
epair7a: link state changed to DOWN
eth0: link state changed to DOWN
2025-10-27T00:54:18.428268+03:00 srv.fbsd.ru apcupsd 2172 - - apcupsd shutdown
succeeded
Waiting (max 60 seconds) for system process `vnlru' to stop... done
Waiting (max 60 seconds) for system process `syncer' to stop...
Syncing disks, vnodes remaining... 0 0 0 0 0 0 0 done

14-stable/087fc6ae0a8c
all jails are zfs-based, and also have devfs and number of nullfs mounts

-- 
You are receiving this mail because:
You are the assignee for the bug.