Deleting files/dirs from partially damaged ZFS filesystem

[Gleb Kurtsou]

Hello,

I have ZFS filesystem with some inconsistencies I'd like to fix. It's
root filesystem on my laptop, so backup/restore everything is rather
troublesome. ZFS scrub doesn't show/fix any errors. These inconsistencies
occurred several months ago and I had no other issues with filesystem
since then.

These issues can be interesting for those working on ZFS stability. I
wasn't able to find any useful information on ZFS debugging tools, so
posting it here.

1. ~/.mozilla-bug2/firefox/5iyxnqmf.default-1
The name used to be different but I was able to rename it. Process
accessing this directory stalls and can't be killed. System used to
panic on accessing it before (NULL pointer dereference, have to stack
trace any longer), but that was fixed just before 9-CURRENT.  It seems
that kernel buffer that contained structures related to this directory
was changed with random data, and then written to disk (So that
checksums are fine but data is incorrect)

2. /test
Directory looks empty, but can't be deleted.
/ # ls -Al /test
total 0
/ # rm -rf /test
rm: /test: Directory not empty

Most likely because I've managed to create file/directory with name
longer then MAXNAMLEN (255 bytes) in this directory. I was running
tools/regression/fstest suite to test stacked filesystem that performed
manipulations on file names.

Files can be added/deleted in this directory.

It seems both these issues can be fixed by marking appropriate device
blocks bad, running scrub, and marking blocks good, or simply trashing
appropriate checksums, so that scrub can fix them. But I can't find
block offsets nor the way to change checksums.

/ # uname -a
FreeBSD tops 9.0-CURRENT FreeBSD 9.0-CURRENT #16 r198029+762c399-dirty: Wed Oct 28 16:11:49 EET 2009     root at tops:/usr/obj/usr/freebsd-src/local/sys/TOPS  amd64


/ # zpool status
  pool: tank
 state: ONLINE
 scrub: none requested
config:

        NAME        STATE     READ WRITE CKSUM
        tank        ONLINE       0     0     0
          ada0s3d   ONLINE       0     0     0

errors: No known data errors

/ # zfs list
NAME         USED  AVAIL  REFER  MOUNTPOINT
tank         116G  8,39G  18,2G  legacy
tank/home   87,5G  8,39G  62,7G  /home
tank/local  4,45G  8,39G  4,45G  /usr/local
tank/ports  5,48G  8,39G  5,48G  /usr/ports

/ # zdb -C
tank
    version=13
    name='tank'
    state=0
    txg=298580
    pool_guid=12986731317200074631
    hostid=1869410071
    hostname='tops'
    vdev_tree
        type='root'
        id=0
        guid=12986731317200074631
        children[0]
                type='disk'
                id=0
                guid=11828906155092156003
                path='/dev/ad0s3d'
                whole_disk=0
                metaslab_array=23
                metaslab_shift=30
                ashift=9
                asize=135652442112
                is_log=0
                DTL=153