[Bug 273418] [panic] Repeating kernel panic on open(/dev/console)

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=273418

            Bug ID: 273418
           Summary: [panic] Repeating kernel panic on open(/dev/console)
           Product: Base System
           Version: 13.2-STABLE
          Hardware: Any
                OS: Any
            Status: New
          Severity: Affects Only Me
          Priority: ---
         Component: kern
          Assignee: bugs@FreeBSD.org
          Reporter: eugen@freebsd.org

A 13.2-STABLE/amd64 server sometimes runs flawlessly for several weeks, but
sometimes panices with same backtrace at midnight after newsyslog rotates and
compresses logs then sends SIGHUP to the syslogd that closes and reopens all
channels including /dev/console. The kernel panices on open(/dev/console)
system call sometimes. The system was source-updated from 12.4-STABLE/amd64 to
the commit https://cgit.freebsd.org/src/commit/?h=stable/13&id=8711fd210

This is regression since 12.4-STABLE.

# conscontrol
Configured: ttyv0
 Available: uart,ttyv0,gdb
    Muting: off
# sysctl kern.vty
kern.vty: vt
# last | grep boot | head -5
boot time                                  Mon Aug 28 00:08
boot time                                  Sun Aug 27 00:07
boot time                                  Sat Aug 26 00:09
boot time                                  Tue Aug 22 00:08
boot time                                  Mon Aug 21 00:09

And I have 5 crashdumps with same backtrace. Custom kernel has debugging
options that point to use-after-free (0xdeadc0dedeadc0de, see below).

options         KDB                     # Enable kernel debugger support.
options         KDB_UNATTENDED
options         KDB_TRACE
options         DDB                     # Support DDB.
options         GDB                     # Support remote GDB.
options        INVARIANTS              # Enable calls of extra sanity checkin
options        INVARIANT_SUPPORT       # Extra sanity checks of internal
structures, required by IN
options        WITNESS                 # Enable checks to detect deadlocks and
cycles
options        WITNESS_SKIPSPIN        # Don't run witness on spinlocks for
speedoptions

The backtrace:

#0  __curthread () at /data/src/sys/amd64/include/pcpu_aux.h:55
#1  doadump (textdump=textdump@entry=1) at
/data/src/sys/kern/kern_shutdown.c:396
#2  0xffffffff80c0dd43 in kern_reboot (howto=260) at
/data/src/sys/kern/kern_shutdown.c:484
#3  0xffffffff80c0e1af in vpanic (fmt=<optimized out>,
ap=ap@entry=0xfffffe0152e8d600)
    at /data/src/sys/kern/kern_shutdown.c:923
#4  0xffffffff80c0df33 in panic (fmt=<unavailable>) at
/data/src/sys/kern/kern_shutdown.c:847
#5  0xffffffff811178b7 in trap_fatal (frame=0xfffffe0152e8d690, eva=0)
    at /data/src/sys/amd64/amd64/trap.c:942
#6  <signal handler called>
#7  devfs_populate_loop (dm=dm@entry=0xfffff8044020a000,
cleanup=cleanup@entry=0)
    at /data/src/sys/fs/devfs/devfs_devs.c:533
#8  0xffffffff80a9a0fa in devfs_populate (dm=dm@entry=0xfffff8044020a000)
    at /data/src/sys/fs/devfs/devfs_devs.c:677
#9  0xffffffff80a9f318 in devfs_populate_vp (vp=0xfffff804401d9988)
    at /data/src/sys/fs/devfs/devfs_vnops.c:359
#10 0xffffffff80a9d61b in devfs_lookup (ap=0xfffffe0152e8da30)
    at /data/src/sys/fs/devfs/devfs_vnops.c:1187
#11 0xffffffff80cecbb1 in VOP_LOOKUP (dvp=0xfffff804401d9988,
vpp=0xfffffe0152e8dd10,
    cnp=0xfffffe0152e8dd38) at ./vnode_if.h:69
#12 lookup (ndp=ndp@entry=0xfffffe0152e8dcb8) at
/data/src/sys/kern/vfs_lookup.c:1092
#13 0xffffffff80cebba2 in namei (ndp=ndp@entry=0xfffffe0152e8dcb8)
    at /data/src/sys/kern/vfs_lookup.c:617
#14 0xffffffff80d11f90 in vn_open_cred (ndp=ndp@entry=0xfffffe0152e8dcb8,
    flagp=flagp@entry=0xfffffe0152e8ddd4, cmode=cmode@entry=0,
    vn_open_flags=vn_open_flags@entry=16, cred=0xfffff80440282500,
fp=0xfffff8005a296af0)
    at /data/src/sys/kern/vfs_vnops.c:328
#15 0xffffffff80d08c58 in kern_openat (td=0xfffffe003753b000, fd=-100,
    path=0x1fc80b443e60 <error: Cannot access memory at address
0x1fc80b443e60>,
    pathseg=UIO_USERSPACE, flags=6, mode=<optimized out>)
    at /data/src/sys/kern/vfs_syscalls.c:1158
#16 0xffffffff81118283 in syscallenter (td=<optimized out>)
    at /data/src/sys/amd64/amd64/../../kern/subr_syscall.c:190
#17 amd64_syscall (td=0xfffffe003753b000, traced=0) at
/data/src/sys/amd64/amd64/trap.c:1183
#18 <signal handler called>
#19 0x00001fc80cc4504a in ?? ()
Backtrace stopped: Cannot access memory at address 0x1fc80b443d58

(kgdb) frame 15
#15 0xffffffff80d08c58 in kern_openat (td=0xfffffe003753b000, fd=-100,
    path=0x1fc80b443e60 <error: Cannot access memory at address
0x1fc80b443e60>,
    pathseg=UIO_USERSPACE, flags=6, mode=<optimized out>)
    at /data/src/sys/kern/vfs_syscalls.c:1158
1158            error = vn_open_cred(&nd, &flags, cmode, VN_OPEN_WANTIOCTLCAPS,

(kgdb) p nd
$4 = {ni_dirp = 0x1fc80b443e60 <error: Cannot access memory at address
0x1fc80b443e60>,
  ni_segflg = UIO_USERSPACE, ni_rightsneeded = 0xfffffe0152e8ddb0, ni_startdir
= 0x0,
  ni_rootdir = 0xfffff80003f36000, ni_topdir = 0x0, ni_dirfd = -100, ni_lcf =
0, ni_filecaps = {
    fc_rights = {cr_rights = {0, 0}}, fc_ioctls = 0x0, fc_nioctls = -1,
fc_fcntls = 0},
  ni_vp = 0x0, ni_dvp = 0xfffff804401d9988, ni_resflags = 1, ni_debugflags = 3,
ni_loopcnt = 0,
  ni_pathlen = 1, ni_next = 0xfffff8039aa8200c "", ni_cnd = {cn_origflags =
8683588,
    cn_flags = 344227908, cn_thread = 0xfffffe003753b000, cn_cred =
0xfffff80440282500,
    cn_nameiop = LOOKUP, cn_lkflags = 532480, cn_pnbuf = 0xfffff8039aa82000
"/dev/console",
    cn_nameptr = 0xfffff8039aa82005 "console", cn_namelen = 7}, ni_cap_tracker
= {
    tqh_first = 0x0, tqh_last = 0xfffffe0152e8dd78}, ni_dvp_seqc = 928231424,
  ni_vp_seqc = 4294966784}

(kgdb) frame 7
#7  devfs_populate_loop (dm=dm@entry=0xfffff8044020a000,
cleanup=cleanup@entry=0)
    at /data/src/sys/fs/devfs/devfs_devs.c:533
533                         cdp->cdp_dirents[dm->dm_idx] != NULL) {
(kgdb) p cdp->cdp_dirents[dm->dm_idx]
Cannot access memory at address 0xdeadc0dedeadc0de

-- 
You are receiving this mail because:
You are the assignee for the bug.