Re: "(ada0:ahcich0:0:0:0): Periph destroyed", followed by panic

Fabian Keil <freebsd-listen@fabiankeil.de> wrote on 2026-01-22 at 16:05:03:

> Fabian Keil <freebsd-listen@fabiankeil.de> wrote on 2026-01-21 at 15:02:30:
> 
> > In the meantime the system with the real flaky disk (elektrobier4)
> > panicked again in xpt_done_process() and I noticed that a similar
> > problem has been reported in PR/277499 "panic in doneq0 xpt_done_td
> > xpt_done_process after HDD falling off the bus (Periph destroyed)" [0]
> > which references PR 263704 "Panic because device went away with
> > XPT_ATA_IO pending" [1]. Both are "Closed FIXED" at the moment.
> > 
> > For now I've set kern.cam.iosched.dynamic=0 on elektrobier4 as well
> > to see how the behaviour differs when ada0 acts up again which
> > I expect to happen within days.
> 
> With kern.cam.iosched.dynamic=0 the system tolerated the device loss:

With this patch [0] and kern.cam.iosched.dynamic=1 the system tolerated
the disk errors as well and the disk even remained online:

2026-01-24T03:02:56.326737+01:00 elektrobier4 kernel <2>1 - - - (ada0:ahcich0:0:0:0): READ_FPDMA_QUEUED. ACB: 60 20 88 10 47 40 02 00 00 00 00 00
2026-01-24T03:02:56.327434+01:00 elektrobier4 kernel <2>1 - - - (ada0:ahcich0:0:0:0): CAM status: Auto-Sense Retrieval Failed
2026-01-24T03:02:56.327457+01:00 elektrobier4 kernel <2>1 - - - (ada0:ahcich0:0:0:0): Error 5, Unretryable error
2026-01-24T03:02:56.327477+01:00 elektrobier4 kernel <2>1 - - - GEOM_ELI: g_eli_read_done() failed (error=5) gpt/rpool-ada2.eli[READ(offset=19353636864, length=16384)]
2026-01-24T03:02:56.327491+01:00 elektrobier4 kernel <2>1 - - - ahcich0: Timeout on slot 14 port 0
2026-01-24T03:02:56.327506+01:00 elektrobier4 kernel <2>1 - - - ahcich0: is 04000000 cs 00004000 ss 00000000 rs 00004000 tfd 451 serr 00400000 cmd 0000ee17
2026-01-24T03:02:56.327522+01:00 elektrobier4 kernel <2>1 - - - ahcich0: AHCI reset: periph already gone.
2026-01-24T03:02:56.327535+01:00 elektrobier4 kernel <2>1 - - - (ada0:ahcich0:0:0:0): WRITE_FPDMA_QUEUED. ACB: 61 08 80 e8 f1 40 15 00 00 00 00 00
2026-01-24T03:02:56.327550+01:00 elektrobier4 kernel <2>1 - - - (ada0:ahcich0:0:0:0): CAM status: Auto-Sense Retrieval Failed
2026-01-24T03:02:56.327566+01:00 elektrobier4 kernel <2>1 - - - (ada0:ahcich0:0:0:0): Error 5, Unretryable error
2026-01-24T03:02:56.432163+01:00 elektrobier4 kernel <2>1 - - - GEOM_ELI: g_eli_write_done() failed (error=5) gpt/dpool-ada2.eli[WRITE(offset=128165412864, length=4096)]
2026-01-24T03:02:56.432266+01:00 elektrobier4 kernel <2>1 - - - (ada0:ahcich0:0:0:0): WRITE_FPDMA_QUEUED. ACB: 61 08 f8 08 16 40 1d 00 00 00 00 00
2026-01-24T03:02:56.432298+01:00 elektrobier4 kernel <2>1 - - - (ada0:ahcich0:0:0:0): CAM status: ATA Status Error
2026-01-24T03:02:56.432321+01:00 elektrobier4 kernel <2>1 - - - (ada0:ahcich0:0:0:0): ATA status: 00 ()
2026-01-24T03:02:56.432388+01:00 elektrobier4 kernel <2>1 - - - (ada0:ahcich0:0:0:0): RES: 00 00 00 00 00 00 00 00 00 00 00
2026-01-24T03:02:56.432424+01:00 elektrobier4 kernel <2>1 - - - (ada0:ahcich0:0:0:0): Retrying command, 3 more tries remain

fk@elektrobier4 ~ $zpool status
  pool: dpool
 state: ONLINE
status: One or more devices has experienced an unrecoverable error.  An
        attempt was made to correct the error.  Applications are unaffected.
action: Determine if the device needs to be replaced, and clear the errors
        using 'zpool clear' or replace the device with 'zpool replace'.
   see: https://openzfs.github.io/openzfs-docs/msg/ZFS-8000-9P
  scan: resilvered 8.59G in 00:13:38 with 0 errors on 2026-01-23 14:23:58
config:

        NAME                    STATE     READ WRITE CKSUM
        dpool                   ONLINE       0     0     0
          mirror-0              ONLINE       0     0     0
            gpt/dpool-ada1.eli  ONLINE       0     0     0
            gpt/dpool-ada2.eli  ONLINE       0     1     0

errors: No known data errors

  pool: rpool
 state: ONLINE
status: One or more devices has experienced an unrecoverable error.  An
        attempt was made to correct the error.  Applications are unaffected.
action: Determine if the device needs to be replaced, and clear the errors
        using 'zpool clear' or replace the device with 'zpool replace'.
   see: https://openzfs.github.io/openzfs-docs/msg/ZFS-8000-9P
  scan: resilvered 11.9M in 00:00:12 with 0 errors on 2026-01-23 14:13:14
config:

        NAME                    STATE     READ WRITE CKSUM
        rpool                   ONLINE       0     0     0
          mirror-0              ONLINE       0     0     0
            gpt/rpool-ada2.eli  ONLINE       1     0     0
            gpt/rpool-ada1.eli  ONLINE       0     0     0

errors: No known data errors

This is before importing 6abefcdd2585019.

Fabian

[0]: <https://www.fabiankeil.de/sourcecode/electrobsd/ahci-Don-t-try-to-tell-the-XPT-about-resets-if-the-periph-is-already-gone-20260123.diff>