From nobody Mon Jan 19 17:32:49 2026 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4dvyg25mYNz6P7QN for ; Mon, 19 Jan 2026 17:50:54 +0000 (UTC) (envelope-from freebsd-listen@fabiankeil.de) Received: from mail.fabiankeil.de (fabiankeil.de [95.211.138.7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4dvyg22JwSz3Kx5 for ; Mon, 19 Jan 2026 17:50:54 +0000 (UTC) (envelope-from freebsd-listen@fabiankeil.de) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=fabiankeil.de header.s=blafasel header.b=QDHSlbX9; dmarc=none; spf=pass (mx1.freebsd.org: domain of freebsd-listen@fabiankeil.de designates 95.211.138.7 as permitted sender) smtp.mailfrom=freebsd-listen@fabiankeil.de Received: by mail.fabiankeil.de (Postfix) id ED9D525E9 for ; Mon, 19 Jan 2026 18:42:38 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=fabiankeil.de; s=blafasel; t=1768844559; bh=9QyISQ/qt3dsXIvPo5kCEhC8iHnR4dF8XxFNFo5WRt8=; h=Date:From:To:Subject:In-Reply-To:References:Reply-To; b=QDHSlbX9gp8N3YybDwDZFp7hR47V53F1CbIW/VGSVTOfEBBIkFidx2K5TsPI9yPU+ ItP/cgHEXa6FMBUDthyu26VYnxL5DzBlKfFcWIr5Jf5NZ++GQtUCEA82mnoOzuCUoh e7JoxJjQ789RnaXQXbqjWz7bfwF5UDVwNzCcicmEMalbJ57zwFRVHSOV3+uJN4A/zQ SCqb3gz47POXqggYd7OgN+8AKkqggpyUaESzSeAuntV+Ag+Gp3b8CLS9h/2L1dK/ar Qt4aN60guXY7F2UcboCGXOvqRfgQWlbH9YGe1/3ottj7T8+v1lYO1ebHhXrAwv7Lea EJzqA5aqxxb9Q== Date: Mon, 19 Jan 2026 18:32:49 +0100 From: Fabian Keil To: freebsd-hackers@freebsd.org Subject: Re: "(ada0:ahcich0:0:0:0): Periph destroyed", followed by panic Message-ID: <20260119183249.34199c88.freebsd-listen@fabiankeil.de> In-Reply-To: <20260118122005.06288476.freebsd-listen@fabiankeil.de> References: <20260117175331.4b5eb147.freebsd-listen@fabiankeil.de> <20260118122005.06288476.freebsd-listen@fabiankeil.de> Reply-To: freebsd-hackers@freebsd.org List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Spamd-Bar: ++ X-Spamd-Result: default: False [2.71 / 15.00]; REPLYTO_EQ_TO_ADDR(5.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; MID_CONTAINS_FROM(1.00)[]; NEURAL_HAM_SHORT(-0.99)[-0.990]; ONCE_RECEIVED(0.20)[]; R_DKIM_ALLOW(-0.20)[fabiankeil.de:s=blafasel]; R_SPF_ALLOW(-0.20)[+ip4:95.211.138.7:c]; MIME_GOOD(-0.10)[text/plain]; ARC_NA(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; RCVD_TLS_LAST(0.00)[]; GREYLIST(0.00)[pass,body]; DMARC_NA(0.00)[fabiankeil.de: no valid DMARC record]; REPLYTO_DOM_NEQ_FROM_DOM(0.00)[]; FROM_HAS_DN(0.00)[]; MISSING_XM_UA(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; TO_DN_NONE(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; DKIM_TRACE(0.00)[fabiankeil.de:+]; MLMMJ_DEST(0.00)[freebsd-hackers@freebsd.org]; RCVD_COUNT_ONE(0.00)[1]; ASN(0.00)[asn:60781, ipnet:95.211.0.0/16, country:NL]; MIME_TRACE(0.00)[0:+]; HAS_REPLYTO(0.00)[freebsd-hackers@freebsd.org] X-Rspamd-Queue-Id: 4dvyg22JwSz3Kx5 Fabian Keil wrote on 2026-01-18 at 12:20:05: > Warner Losh wrote on 2026-01-17 at 10:29:44: > > You should never pass NULL as the path to xpt_async(). It's a required > > element of the call: the path you are sending the async event to. And the > > path must have either a device or a bus, which that traceback didn't have. > > Now, how did it come to pass that the bus and/or device was NULL here? > > That's where the bug is... xpt_create_path always fills in the bus (via > > xpt_compile_path). One common way for it to become NULL like this is > > xpt_release_path is called on it. Maybe you can use dtrace to find where > > that's coming to pass? I'm not seeing anything super quick in the code, but > > I'm just doing a quick analysis to answer this email + past experience... > While I have not yet been able to reproduce the exact same panic on the > same system, I ran into a related issue while trying to reproduce the > problem in a byhve vm. > > Running ahcich0-detacher.sh: [...] > in two terminals at the same time within seconds reliably results in: [...] > (kgdb) f 8 > #8 0xffffffff80369a66 in xpt_async (async_code=256, path=0xfffff8000606eb60, async_arg=0x0) at /usr/src/sys/cam/cam_xpt.c:4321 > 4321 xpt_freeze_simq(path->bus->sim, 1); [...] > (kgdb) p *path > $1 = {periph = 0x0, bus = 0x0, target = 0x0, device = 0x0} I've come up with a patch that seems to prevent the problem [0]. > I also once managed to get devctl sleeping "forever" by running only > one ahcich0-detacher.sh instance, but haven't been able to reproduce > this yet. Now that the panic is prevented this issue can be reliably triggered by running ahcich0-detacher.sh in parallel for a while. (kgdb) where #0 sched_switch (td=td@entry=0xfffff8010a749740, flags=flags@entry=259) at /usr/src/sys/kern/sched_ule.c:2437 #1 0xffffffff80a34da3 in mi_switch (flags=flags@entry=259) at /usr/src/sys/kern/kern_synch.c:548 #2 0xffffffff80a88233 in sleepq_switch (wchan=0xfffff80006256380, pri=) at /usr/src/sys/kern/subr_sleepqueue.c:608 #3 0xffffffff80a880fc in sleepq_wait (wchan=, pri=) at /usr/src/sys/kern/subr_sleepqueue.c:659 #4 0xffffffff80a34311 in _sleep (ident=ident@entry=0xfffff80006256380, lock=lock@entry=0xfffffe058a9f9400, priority=priority@entry=43, wmesg=0xffffffff8105a969 "simfree", sbt=sbt@entry=0, pr=pr@entry=0, flags=256) at /usr/src/sys/kern/kern_synch.c:223 #5 0xffffffff803676c0 in cam_sim_free (sim=0xfffff80006256380, free_devq=free_devq@entry=1) at /usr/src/sys/cam/cam_sim.c:157 #6 0xffffffff804bc6cd in ahci_ch_detach (dev=0xfffff800060a2800, dev@entry=) at /usr/src/sys/dev/ahci/ahci.c:969 #7 0xffffffff80a63404 in DEVICE_DETACH (dev=0xfffff800060a2800) at ./device_if.h:234 #8 device_detach (dev=dev@entry=0xfffff800060a2800) at /usr/src/sys/kern/subr_bus.c:2712 #9 0xffffffff80a6b307 in devctl2_ioctl (cdev=, cmd=, cmd@entry=, data=0xfffff8010a7e1400 "ahcich0", data@entry=, fflag=, td=0xfffff8010a749740, td@entry=) at /usr/src/sys/kern/subr_bus.c:5843 #10 0xffffffff808d3da1 in devfs_ioctl (ap=0xfffffe058835bc58) at /usr/src/sys/fs/devfs/devfs_vnops.c:963 #11 0xffffffff80f4a9ed in VOP_IOCTL_APV (vop=0xffffffff8149bd28 , a=a@entry=0xfffffe058835bc58) at vnode_if.c:1229 #12 0xffffffff80b3b030 in VOP_IOCTL (vp=, command=, data=, fflag=, cred=, td=) at ./vnode_if.h:637 #13 0xffffffff80b3b030 in vn_ioctl (fp=, com=, data=, active_cred=, td=) #14 0xffffffff808d446e in devfs_ioctl_f (fp=, fp@entry=, com=, com@entry=, data=, data@entry=, cred=, cred@entry=, td=, td@entry=) at /usr/src/sys/fs/devfs/devfs_vnops.c:894 #15 0xffffffff80aa1ef1 in fo_ioctl (fp=0xfffff8000fccadc0, com=2157462530, data=0xfffff8010a7e1400, active_cred=, td=0xfffff8010a749740) at /usr/src/sys/sys/file.h:379 #16 kern_ioctl (td=td@entry=0xfffff8010a749740, fd=, com=com@entry=2157462530, data=data@entry=0xfffff8010a7e1400 "ahcich0") at /usr/src/sys/kern/sys_generic.c:810 #17 0xffffffff80aa1be3 in sys_ioctl (td=0xfffff8010a749740, uap=0xfffff8010a749b40) at /usr/src/sys/kern/sys_generic.c:715 #18 0xffffffff80ef6d9a in syscallenter (td=0xfffff8010a749740) at /usr/src/sys/amd64/amd64/../../kern/subr_syscall.c:193 #19 amd64_syscall (td=0xfffff8010a749740, traced=0) at /usr/src/sys/amd64/amd64/trap.c:1241 #20 #21 0x0000225fa814557a in ?? () Backtrace stopped: Cannot access memory at address 0x225fa68c9038 (kgdb) f 5 #5 0xffffffff803676c0 in cam_sim_free (sim=0xfffff80006256380, free_devq=free_devq@entry=1) at /usr/src/sys/cam/cam_sim.c:157 157 error = msleep(sim, mtx, PRIBIO, "simfree", 0); (kgdb) p *sim $1 = {sim_action = 0xffffffff804bcc80 , sim_poll = 0xffffffff804bd360 , sim_name = 0xffffffff8104741f "ahcich", softc = 0xfffffe058a9f8000, mtx = 0xfffffe058a9f9400, links = {tqe_next = 0x0, tqe_prev = 0x0}, path_id = 0, unit_number = 0, bus_id = 0, max_tagged_dev_openings = 32, max_dev_openings = 2, flags = 0, devq = 0xfffff80005d94880, refcount = 1} (kgdb) p *mtx $2 = {lock_object = {lo_name = 0xffffffff81070545 "AHCI channel lock", lo_flags = 16973824, lo_data = 0, lo_witness = 0xfffff802bfd7f700}, mtx_lock = 0} (kgdb) l 152 mtx_assert(mtx, MA_OWNED); 153 } 154 KASSERT(sim->refcount >= 1, ("sim->refcount >= 1")); 155 sim->refcount--; 156 if (sim->refcount > 0) { 157 error = msleep(sim, mtx, PRIBIO, "simfree", 0); 158 KASSERT(error == 0, ("invalid error value for msleep(9)")); 159 } 160 KASSERT(sim->refcount == 0, ("sim->refcount == 0")); 161 if (mtx == &cam_sim_free_mtx) /* sim->mtx == NULL */ I'm not sure what to do about this. Fabian [0]