geom_mirror/UFS weirdness with 7.2-STABLE

[Boris Kochergin]

Ahoy. I noticed some very odd things in my file server's kernel buffer 
this morning (there were actually a ton of these--this is a snippet):

Jul 20 05:54:10 exodus smartd[763]: Device: /dev/ad1, FAILED SMART 
self-check. BACK UP DATA NOW!
Jul 20 05:57:57 exodus kernel: 
g_vfs_done():mirror/boots1[READ(offset=-4569735194538825728, 
length=16384)]error = 5
Jul 20 05:57:57 exodus kernel: bad block 8806809555123731765, ino 4430620
Jul 20 05:57:57 exodus kernel: pid 35 (softdepflush), uid 0 inumber 
4430620 on /: bad block
Jul 20 05:57:57 exodus kernel: 
g_vfs_done():mirror/boots1[READ(offset=-7379534036451358720, 
length=16384)]error = 5
Jul 20 05:57:57 exodus kernel: bad block 5076457091570183473, ino 4430620
Jul 20 05:57:57 exodus kernel: pid 35 (softdepflush), uid 0 inumber 
4430620 on /: bad block
Jul 20 05:57:57 exodus kernel: 
g_vfs_done():mirror/boots1[READ(offset=8010486245121781760, 
length=16384)]error = 5
Jul 20 05:57:57 exodus kernel: bad block 1859394416713520222, ino 4430620
Jul 20 05:57:57 exodus kernel: pid 35 (softdepflush), uid 0 inumber 
4430620 on /: bad block
Jul 20 05:57:57 exodus kernel: bad block 8993345361309361786, ino 4430620
Jul 20 05:57:57 exodus kernel: pid 35 (softdepflush), uid 0 inumber 
4430620 on /: bad block
Jul 20 05:57:57 exodus kernel: bad block 5442079378433802039, ino 4430620
Jul 20 05:57:57 exodus kernel: pid 35 (softdepflush), uid 0 inumber 
4430620 on /: bad block
Jul 20 05:57:57 exodus kernel: bad block -386473693423142096, ino 4430620
Jul 20 05:57:57 exodus kernel: pid 35 (softdepflush), uid 0 inumber 
4430620 on /: bad block
Jul 20 05:57:57 exodus kernel: bad block -6538821972600747, ino 4430620
Jul 20 05:57:57 exodus kernel: pid 35 (softdepflush), uid 0 inumber 
4430620 on /: bad block
Jul 20 05:57:57 exodus kernel: bad block 7280706165078778427, ino 4430620
Jul 20 05:57:57 exodus kernel: pid 35 (softdepflush), uid 0 inumber 
4430620 on /: bad block
Jul 20 05:57:57 exodus kernel: bad block 2788813269397304649, ino 4430620
Jul 20 05:57:57 exodus kernel: pid 35 (softdepflush), uid 0 inumber 
4430620 on /: bad block
Jul 20 05:57:57 exodus kernel: bad block 8324768912580846323, ino 4430620
Jul 20 05:57:57 exodus kernel: pid 35 (softdepflush), uid 0 inumber 
4430620 on /: bad block
Jul 20 05:57:57 exodus kernel: bad block 8358350883763232034, ino 4430620
Jul 20 05:57:57 exodus kernel: pid 35 (softdepflush), uid 0 inumber 
4430620 on /: bad block

The filesystem lives atop a two-disk geom_mirror:

# gmirror status
       Name    Status  Components
mirror/boot  COMPLETE  ad0
                       ad1

...and the output of "df" on it also shows something odd:

# df /
Filesystem         1K-blocks                 Used               Avail 
Capacity  Mounted on
/dev/mirror/boots1  37846636 -4058799239201906816 4058799239236725722 
-11656883301279%    /

The system is a:

# uname -a
FreeBSD exodus.poly.edu 7.2-STABLE FreeBSD 7.2-STABLE #3: Sat Jul 11 
16:22:02 EDT 2009     root at exodus.poly.edu:/usr/obj/usr/src/sys/EXODUS  
amd64

Regarding smartd yelling at me about /dev/ad1, it's been doing that for 
long while before this. There is one sector on the drive that cannot be 
read, but the disk has otherwise been fine for months. My experience 
with geom_mirror has been that it disconnects members from an array if 
they experience I/O errors, so this seems to be something different. Any 
clues?

-Boris