From nobody Sat Jan 24 04:59:25 2026 X-Original-To: freebsd-hackers@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4dyjgh28RRz6P9vx for ; Sat, 24 Jan 2026 05:16:00 +0000 (UTC) (envelope-from freebsd-listen@fabiankeil.de) Received: from mail.fabiankeil.de (fabiankeil.de [95.211.138.7]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4dyjgg2WVgz3jpR for ; Sat, 24 Jan 2026 05:15:59 +0000 (UTC) (envelope-from freebsd-listen@fabiankeil.de) Authentication-Results: mx1.freebsd.org; dkim=pass header.d=fabiankeil.de header.s=blafasel header.b="dccWD/ol"; dmarc=none; spf=pass (mx1.freebsd.org: domain of freebsd-listen@fabiankeil.de designates 95.211.138.7 as permitted sender) smtp.mailfrom=freebsd-listen@fabiankeil.de Received: by mail.fabiankeil.de (Postfix) id 316422D06 for ; Sat, 24 Jan 2026 06:09:20 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=fabiankeil.de; s=blafasel; t=1769231361; bh=o0lfODB3rC5tHndT+gsSPJcxQR/3MEWRBePSWCzVOdQ=; h=Date:From:To:Subject:In-Reply-To:References:Reply-To; b=dccWD/olTg5CMwaJ97Z6Px2C4uLr1O2rZoNYNoD5KHSkX+TA3wPhsPSqQmRbXFFAS H6dDyfBKNBcL1bVBoWEMf83X5pmohX4cYYrcAJmwysp66Fmdcw2bANSWrIxfjdBDDI HTK9xh9Z8QJQK5hvvQ7bszMZqiszIwYLrtEMA4bpUZMbpdouuocbnPIVtTvliBe9/h XuS27o/RcUDWZ6LejStgB+lxhQuWoW53O6cfhYM5ZD6rZGVCM9iMR3qZ4bRT2FrbBO NSXVMeQ0tzvQZ0zz3Vd+yLOl6qOnh2MRtPseyem33NzbO50esnj58wVZJYh1+dhbxx xwJ2zyzx1mVtw== Date: Sat, 24 Jan 2026 05:59:25 +0100 From: Fabian Keil To: freebsd-hackers@freebsd.org Subject: Re: "(ada0:ahcich0:0:0:0): Periph destroyed", followed by panic Message-ID: <20260124055925.554160e5.freebsd-listen@fabiankeil.de> In-Reply-To: <20260122160503.329043ed.freebsd-listen@fabiankeil.de> References: <20260117175331.4b5eb147.freebsd-listen@fabiankeil.de> <20260118122005.06288476.freebsd-listen@fabiankeil.de> <20260119183249.34199c88.freebsd-listen@fabiankeil.de> <20260121150230.16e3be9d.freebsd-listen@fabiankeil.de> <20260122160503.329043ed.freebsd-listen@fabiankeil.de> Reply-To: freebsd-hackers@freebsd.org List-Id: Technical discussions relating to FreeBSD List-Archive: https://lists.freebsd.org/archives/freebsd-hackers List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-hackers@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-Spamd-Bar: ++ X-Spamd-Result: default: False [2.72 / 15.00]; REPLYTO_EQ_TO_ADDR(5.00)[]; MID_CONTAINS_FROM(1.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.98)[-0.978]; R_DKIM_ALLOW(-0.20)[fabiankeil.de:s=blafasel]; ONCE_RECEIVED(0.20)[]; R_SPF_ALLOW(-0.20)[+ip4:95.211.138.7:c]; MIME_GOOD(-0.10)[text/plain]; RCVD_TLS_LAST(0.00)[]; ARC_NA(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCPT_COUNT_ONE(0.00)[1]; GREYLIST(0.00)[pass,body]; DMARC_NA(0.00)[fabiankeil.de: no valid DMARC record]; FROM_HAS_DN(0.00)[]; RCVD_COUNT_ONE(0.00)[1]; DKIM_TRACE(0.00)[fabiankeil.de:+]; TO_DN_NONE(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; REPLYTO_DOM_NEQ_FROM_DOM(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-hackers@freebsd.org]; MISSING_XM_UA(0.00)[]; MLMMJ_DEST(0.00)[freebsd-hackers@freebsd.org]; MIME_TRACE(0.00)[0:+]; HAS_REPLYTO(0.00)[freebsd-hackers@freebsd.org] X-Rspamd-Queue-Id: 4dyjgg2WVgz3jpR Fabian Keil wrote on 2026-01-22 at 16:05:03: > Fabian Keil wrote on 2026-01-21 at 15:02:30: > > > In the meantime the system with the real flaky disk (elektrobier4) > > panicked again in xpt_done_process() and I noticed that a similar > > problem has been reported in PR/277499 "panic in doneq0 xpt_done_td > > xpt_done_process after HDD falling off the bus (Periph destroyed)" [0] > > which references PR 263704 "Panic because device went away with > > XPT_ATA_IO pending" [1]. Both are "Closed FIXED" at the moment. > > > > For now I've set kern.cam.iosched.dynamic=0 on elektrobier4 as well > > to see how the behaviour differs when ada0 acts up again which > > I expect to happen within days. > > With kern.cam.iosched.dynamic=0 the system tolerated the device loss: With this patch [0] and kern.cam.iosched.dynamic=1 the system tolerated the disk errors as well and the disk even remained online: 2026-01-24T03:02:56.326737+01:00 elektrobier4 kernel <2>1 - - - (ada0:ahcich0:0:0:0): READ_FPDMA_QUEUED. ACB: 60 20 88 10 47 40 02 00 00 00 00 00 2026-01-24T03:02:56.327434+01:00 elektrobier4 kernel <2>1 - - - (ada0:ahcich0:0:0:0): CAM status: Auto-Sense Retrieval Failed 2026-01-24T03:02:56.327457+01:00 elektrobier4 kernel <2>1 - - - (ada0:ahcich0:0:0:0): Error 5, Unretryable error 2026-01-24T03:02:56.327477+01:00 elektrobier4 kernel <2>1 - - - GEOM_ELI: g_eli_read_done() failed (error=5) gpt/rpool-ada2.eli[READ(offset=19353636864, length=16384)] 2026-01-24T03:02:56.327491+01:00 elektrobier4 kernel <2>1 - - - ahcich0: Timeout on slot 14 port 0 2026-01-24T03:02:56.327506+01:00 elektrobier4 kernel <2>1 - - - ahcich0: is 04000000 cs 00004000 ss 00000000 rs 00004000 tfd 451 serr 00400000 cmd 0000ee17 2026-01-24T03:02:56.327522+01:00 elektrobier4 kernel <2>1 - - - ahcich0: AHCI reset: periph already gone. 2026-01-24T03:02:56.327535+01:00 elektrobier4 kernel <2>1 - - - (ada0:ahcich0:0:0:0): WRITE_FPDMA_QUEUED. ACB: 61 08 80 e8 f1 40 15 00 00 00 00 00 2026-01-24T03:02:56.327550+01:00 elektrobier4 kernel <2>1 - - - (ada0:ahcich0:0:0:0): CAM status: Auto-Sense Retrieval Failed 2026-01-24T03:02:56.327566+01:00 elektrobier4 kernel <2>1 - - - (ada0:ahcich0:0:0:0): Error 5, Unretryable error 2026-01-24T03:02:56.432163+01:00 elektrobier4 kernel <2>1 - - - GEOM_ELI: g_eli_write_done() failed (error=5) gpt/dpool-ada2.eli[WRITE(offset=128165412864, length=4096)] 2026-01-24T03:02:56.432266+01:00 elektrobier4 kernel <2>1 - - - (ada0:ahcich0:0:0:0): WRITE_FPDMA_QUEUED. ACB: 61 08 f8 08 16 40 1d 00 00 00 00 00 2026-01-24T03:02:56.432298+01:00 elektrobier4 kernel <2>1 - - - (ada0:ahcich0:0:0:0): CAM status: ATA Status Error 2026-01-24T03:02:56.432321+01:00 elektrobier4 kernel <2>1 - - - (ada0:ahcich0:0:0:0): ATA status: 00 () 2026-01-24T03:02:56.432388+01:00 elektrobier4 kernel <2>1 - - - (ada0:ahcich0:0:0:0): RES: 00 00 00 00 00 00 00 00 00 00 00 2026-01-24T03:02:56.432424+01:00 elektrobier4 kernel <2>1 - - - (ada0:ahcich0:0:0:0): Retrying command, 3 more tries remain fk@elektrobier4 ~ $zpool status pool: dpool state: ONLINE status: One or more devices has experienced an unrecoverable error. An attempt was made to correct the error. Applications are unaffected. action: Determine if the device needs to be replaced, and clear the errors using 'zpool clear' or replace the device with 'zpool replace'. see: https://openzfs.github.io/openzfs-docs/msg/ZFS-8000-9P scan: resilvered 8.59G in 00:13:38 with 0 errors on 2026-01-23 14:23:58 config: NAME STATE READ WRITE CKSUM dpool ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 gpt/dpool-ada1.eli ONLINE 0 0 0 gpt/dpool-ada2.eli ONLINE 0 1 0 errors: No known data errors pool: rpool state: ONLINE status: One or more devices has experienced an unrecoverable error. An attempt was made to correct the error. Applications are unaffected. action: Determine if the device needs to be replaced, and clear the errors using 'zpool clear' or replace the device with 'zpool replace'. see: https://openzfs.github.io/openzfs-docs/msg/ZFS-8000-9P scan: resilvered 11.9M in 00:00:12 with 0 errors on 2026-01-23 14:13:14 config: NAME STATE READ WRITE CKSUM rpool ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 gpt/rpool-ada2.eli ONLINE 1 0 0 gpt/rpool-ada1.eli ONLINE 0 0 0 errors: No known data errors This is before importing 6abefcdd2585019. Fabian [0]: