Errors on a file on a zpool: How to remove?

[Bernd Walter]

On Sun, Jan 31, 2010 at 11:49:30PM -0500, Rich wrote:
> Well, that's frustrating.
> 
> I moved all the data off of that FS on the pool, onto a newly-created
> FS on the pool. It took a few days.
> 
> I just attempted to zfs destroy the old filesystem.
> 
> The command hung for 30 minutes, at which point I decided that this
> was likely not doing anything useful, especially since operations to
> all but the latest-created filesystem on the pool now hung, as well as
> zfs list.
> 
> I got this fun thing out of /var/log/messages, too:
> Jan 31 23:12:05 manticore root: ZFS: checksum mismatch, zpool=rigatoni
> path=/dev/da5 offset=446136819712 size=8192
> Jan 31 23:12:05 manticore root: ZFS: checksum mismatch, zpool=rigatoni
> path=/dev/da2 offset=320393101312 size=8192
> Jan 31 23:12:05 manticore root: ZFS: checksum mismatch, zpool=rigatoni
> path=/dev/da4 offset=1231402180608 size=8192
> Jan 31 23:12:05 manticore root: ZFS: checksum mismatch, zpool=rigatoni
> path=/dev/da5 offset=446136819712 size=8192
> Jan 31 23:12:05 manticore root: ZFS: checksum mismatch, zpool=rigatoni
> path=/dev/da2 offset=320393101312 size=8192
> Jan 31 23:12:05 manticore root: ZFS: checksum mismatch, zpool=rigatoni
> path=/dev/da4 offset=1231402180608 size=8192
> Jan 31 23:12:05 manticore root: ZFS: zpool I/O failure, zpool=rigatoni error=86
> Jan 31 23:12:05 manticore root: ZFS: vdev I/O failure, zpool=rigatoni
> path= offset= size= error=
> 
> The last one, of course, being fascinating.
> 
> Okay, how annoying. I try sysctl debug.kdb.panic=1, and it hangs. Uh-oh.
> 
> I reboot physically, and the system hangs on "Mounting local
> filesystems..." for a long time.
> 
> I power the machine off, unplug all 5 devices in that pool, and boot
> the machine again. It boots rapidly and fine.
> 
> I export the pool (now reporting that all 5 devices were missing,
> unsurprisingly), power the machine off, plug the 5 devices back in,
> and power on.
> 
> Machine boots. zpool import rigatoni succeeds. zfs list now hangs. df
> output shows no filesystems from that pool, and /var/log/messages got
> another copy of what I just pasted above.

I have had a strange corruption with an older FreeBSD version after a
panic one day.
On boot it excercised the harddisks for more than 20 hours before I
finally gave up.
The "solution" was to import the pool and manually mount the unbroken
volumes.
Disk excercise startet only when mounting the broken FS.
It was possible to clone the most recent snapshot of the broken filesystem
and use this, which was good because it was a few hours more recent than
the last backup.
I never deleted the broken FS, because the machine was already in
transition to be replaced.
At least you might be able to read the correct filesystems by hand mounting
and have the pool available, which might allow trying a delete again.
But I'm not sure if zfs import allows importing without mounting.

-- 
B.Walter <bernd at bwct.de> http://www.bwct.de
Modbus/TCP Ethernet I/O Baugruppen, ARM basierte FreeBSD Rechner uvm.