[Bug 224292] processes are hanging in state ufs / possible deadlock in file system

[bugzilla-noreply at freebsd.org]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=224292

Ali Abdallah <ali.abdallah at suse.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |ali.abdallah at suse.com

--- Comment #13 from Ali Abdallah <ali.abdallah at suse.com> ---
While upgrading my FreeBSD 13-CURRENT box yesterday, I ran into this issue,
when compiling the kernel and then world with -j8, I was ending up always with
all the cc processes stuck in biowr and ufs. At the end I've upgraded my box
through NFS from my 13-CURRENT server (zfs based).

But when upgrading ports from the same server, using pkg upgrade -f, it was
also getting stuck with the same symptoms. I've managed to take a kernel dump,
then booted into single user mode, disabled  soft update journaling: (-j), then
booted the system and upgraded my packages, all went ok.

Looking at the dump, I see lots of dd processes (launched by pkg I believe),
that are in the following state:

read 345 (Thread 100752 (PID=32594: dd)):
#0  sched_switch (td=0xfffffe00fdd9be00, flags=<optimized out>) at
/usr/src/sys/kern/sched_ule.c:2147
#1  0xffffffff803db4be in mi_switch (flags=flags at entry=260) at
/usr/src/sys/kern/kern_synch.c:542
#2  0xffffffff8042a43d in sleepq_switch (wchan=wchan at entry=0x0,
pri=pri at entry=2122752) at /usr/src/sys/kern/subr_sleepqueue.c:608
#3  0xffffffff8042a327 in sleepq_wait (wchan=<unavailable>, pri=<unavailable>)
at /usr/src/sys/kern/subr_sleepqueue.c:659
#4  0xffffffff803a9f7f in sleeplk (lk=lk at entry=0xfffff8000535f810,
flags=flags at entry=2122752, ilk=<optimized out>, ilk at entry=0xfffff8000535f840,
wmesg=<optimized out>, wmesg at entry=0xffffffff806c8ccb "ufs", pri=<optimized
out>, pri at entry=96, timo=timo at entry=51, queue=1) at
/usr/src/sys/kern/kern_lock.c:301
#5  0xffffffff803a88e2 in lockmgr_slock_hard (lk=0xfffff8000535f810,
flags=flags at entry=2122752, ilk=0xfffff8000535f840,
ilk at entry=0xfffffe00fdaff658, file=<optimized out>, line=<optimized out>,
lwa=lwa at entry=0x0) at /usr/src/sys/kern/kern_lock.c:696
#6  0xffffffff803a84e8 in lockmgr_lock_flags (lk=<optimized out>,
lk at entry=0xfffff8000535f810, flags=flags at entry=2122752,
ilk=ilk at entry=0xfffff8000535f840, file=<unavailable>, line=<unavailable>) at
/usr/src/sys/kern/kern_lock.c:1056
#7  0xffffffff805c90e5 in ffs_lock (ap=0xfffffe00fdaff658, ap at entry=<error
reading variable: value is not available>) at
/usr/src/sys/ufs/ffs/ffs_vnops.c:494
#8  0xffffffff804be098 in VOP_LOCK1 (vp=0xfffff8000535f7a0, flags=2106368,
file=0xffffffff806fac3e "/usr/src/sys/kern/vfs_lookup.c", line=877) at
./vnode_if.h:1096
#9  _vn_lock (vp=vp at entry=0xfffff8000535f7a0, flags=2106368,
file=0xffffffff806fac3e "/usr/src/sys/kern/vfs_lookup.c", line=line at entry=877)
at /usr/src/sys/kern/vfs_vnops.c:1741
#10 0xffffffff80497a18 in lookup (ndp=ndp at entry=0xfffffe00fdaff970) at
/usr/src/sys/kern/vfs_lookup.c:875
#11 0xffffffff804973dd in namei (ndp=ndp at entry=0xfffffe00fdaff970) at
/usr/src/sys/kern/vfs_lookup.c:631
--Type <RET> for more, q to quit, c to continue without paging--
#12 0xffffffff804bd590 in vn_open_cred (ndp=ndp at entry=0xfffffe00fdaff970,
flagp=<optimized out>, flagp at entry=0xfffffe00fdaffa94, cmode=<optimized out>,
vn_open_flags=vn_open_flags at entry=0, cred=<optimized out>,
fp=0xfffff80003d49140) at /usr/src/sys/kern/vfs_vnops.c:252
#13 0xffffffff804bd44d in vn_open (ndp=<unavailable>,
ndp at entry=0xfffffe00fdaff970, flagp=<unavailable>,
flagp at entry=0xfffffe00fdaffa94, cmode=<unavailable>, fp=<unavailable>) at
/usr/src/sys/kern/vfs_vnops.c:193
#14 0xffffffff804b3723 in kern_openat (td=0xfffffe00fdd9be00, fd=<optimized
out>, path=<optimized out>, pathseg=<optimized out>, flags=1539,
mode=<optimized out>) at /usr/src/sys/kern/vfs_syscalls.c:1143
#15 0xffffffff8068d5cc in syscallenter (td=0xfffffe00fdd9be00) at
/usr/src/sys/amd64/amd64/../../kern/subr_syscall.c:189
#16 amd64_syscall (td=0xfffffe00fdd9be00, traced=0) at
/usr/src/sys/amd64/amd64/trap.c:1156


I will enable back -j, and kernel deadlocks debugging features, to hopefully
get more debugging information.

-- 
You are receiving this mail because:
You are the assignee for the bug.