Re: db> reset -> panic: lock (sleep mutex) eventhandler not locked @ /usr/src/sys/kern/subr_eventhandler.c:272
Date: Tue, 04 Nov 2025 15:41:06 UTC
Hi both, On 10/28/25 06:59, Ryan Libby wrote: > On Tue, Oct 28, 2025 at 2:46 AM Ryan Libby <rlibby@freebsd.org> wrote: >> >> On Mon, Oct 27, 2025 at 7:31 AM Bjoern A. Zeeb >> <bzeeb-lists@lists.zabbadoz.net> wrote: >>> >>> Hi, >>> >>> on main-ish I get the following. I am a bit concerned as over the last >>> year or two our reset paths had issues like this more and more. >>> >>> >>> db> reset >>> panic: lock (sleep mutex) eventhandler not locked @ /usr/src/sys/kern/subr_eventhandler.c:272 >>> cpuid = 1 >>> time = 1025 >>> KDB: stack backtrace: >>> db_trace_self() at db_trace_self >>> db_trace_self_wrapper() at db_trace_self_wrapper+0x38 >>> vpanic() at vpanic+0x1a0 >>> panic() at panic+0x48 >>> witness_unlock() at witness_unlock+0x140 >>> __mtx_unlock_flags() at __mtx_unlock_flags+0x54 >>> eventhandler_find_list() at eventhandler_find_list+0xbc >>> kern_reboot() at kern_reboot+0x244 >>> db_reset() at db_reset+0xec >>> db_command() at db_command+0x2f4 >>> db_command_loop() at db_command_loop+0x58 >>> db_trap() at db_trap+0x100 >>> kdb_trap() at kdb_trap+0x350 >>> handle_el1h_sync() at handle_el1h_sync+0x18 >>> --- exception, esr 0xf2000000 >>> kdb_alt_break_internal() at kdb_alt_break_internal+0x1a8 >>> kdb_alt_break() at kdb_alt_break+0x10 >>> uart_intr_rxready() at uart_intr_rxready+0x88 >>> uart_intr() at uart_intr+0x124 >>> intr_event_handle() at intr_event_handle+0xf4 >>> intr_isrc_dispatch() at intr_isrc_dispatch+0x60 >>> arm_gic_intr() at arm_gic_intr+0x118 >>> intr_irq_handler() at intr_irq_handler+0x98 >>> handle_el1h_irq() at handle_el1h_irq+0x18 >>> --- interrupt >>> cpu_idle() at cpu_idle+0x78 >>> sched_idletd() at sched_idletd+0x494 >>> fork_exit() at fork_exit+0x78 >>> fork_trampoline() at fork_trampoline+0x18 >>> Uptime: 17m5s >>> Automatic reboot in 15 seconds - press a key on the console to abort >>> >>> -- >>> Bjoern A. Zeeb r15:7 >>> Bjoern, I apologize that this change once again is hindering the reset path. I share your concern, as do others, please see the renewed discussion for a possible path forward: https://reviews.freebsd.org/D37981 >> >> Would your kernel config match this condition in sys/sys/mutex.h? >> #if LOCK_DEBUG > 0 || defined(MUTEX_NOINLINE) >> >> It looks like we might be missing some SCHEDULER_STOPPED() checks in >> those code paths. It's a twisty maze of macros that I haven't totally >> followed though. td_locks might be getting broken too. >> >> Ryan > > Reading sys/sys/_lock.h, LOCK_DEBUG will be on with WITNESS, which this > kernel clearly has (and also with INVARIANTS, both of which are defaults > for GENERIC). > Ryan, thanks for your analysis. This one in particular was known to me, and I've just put the change up in phabricator. As best I can tell this is all that is missing... https://reviews.freebsd.org/D53582 Best, Mitchell