Re: 14.3 panics: zfs corruption (was: supervisor write data, page not present)

On Fri, 5 Sep 2025 12:54:27 -0700 Mark Millard <marklmi@yahoo.com> wrote:
>
> > On Thu, 4 Sep 2025 09:11:53 -0400 "J.R. Oldroyd" <fbsd@opal.com> wrote:  
> > >
> > > Well, even with locally compiled kernel AND now also with the suggested
> > > microcode update in place, this system continues to panic after 1-3 days
> > > of uptime.  As noted before, this never happened on 14.2 or many earlier
> > > kernels.
> > > 
> > > 	-jr
> > > 

Following up on this...

This panic appears to have been due to corruption that occurred to several
files on a ZFS file system.

A day or so prior to upgrading this system from 14.2 to 14.3 there had been
a power interruption to the system.  Apparently this resulted in a set of
eight files ending up in a corrupted state:

find: archive/20421: Invalid argument
find: archive/20438: File exists
find: archive/20439: File exists
find: archive/20446: File exists
find: archive/20447: File exists
find: archive/20448: File exists
find: archive/20449: File exists
find: archive/20450: File exists

Running ls -l on these showed them all with mode and size 0 and the
date showing as the Epoch.  So, corrupted inode info?

Following a copy of all other files to a new ZFS fs, restoring these
eight from backup and destroying the fs with the corruption, there have
been no further panics.  The system now has a couple of weeks of uptime.
Previously it was panicing every few days.

I'd note that there was no indication of any error in "zpool status".
Nor did attempting to access any of these files using ls or cat or other
cause a panic.  The panics were seemingly at random times, usually
overnight.

The panics were when running 14.3-RELEASE-p2 both from freebsd-update
and when compiled locally.

	-jr