g_vfs_done():da3s1a[READ(offset=81064794762854400, length=8192)]error = 5

[Scott Long]

d_elbracht wrote:
> we are trying to diagnose errors seen on 6.2, SMP, amd64, cvsup'ed of
> 2007-10-09
> 
> Mainboard is a Tyan Thunder h2000M (S3992-E) with 16 GB RAM and 2 x Opteron
> 2216, da3 is on a 3ware 9550-12
> 
> we are seeing this error:
> g_vfs_done():da3s1a[READ(offset=81064794762854400, length=8192)]error = 5
> on a 12 GB Hyperdrive
> 
> the offset changes sometimes, but it is always 81064794xxxxxxxxx and well
> out the 12GB range.
> 
> We did have the Hyperdrive connected directly to the mainboards SATA0 (ad4)
> with similar errors.
> We used to have a md instead of the hyperdrive before, coming up with
> similar errors.
> 
> Blocksize on the partition is 8192 (newsfs -b 8192 ..). 
> We did have a blocksize of 65536 before, but after some hours (sometimes
> days), the machine will be unresponsible with "newbuf" as a waitmessage in
> top and has to be hard-reset. 
> Regarding "newbuf", as well as nbufkv and nbufbs, I will write a seperate
> message to the list.
> 
> According to systat -vm, da3 does tps > 500 (yes, that's a lot)
> 
> This leads to an assumption, the error has to do with very high IOs per
> second on a SMP machine.
> The system-disk is a RAID1 on an ICP 5805. All other disks (51) are 20
> gstripe'd partitions.
> 
> Any hint to diagnose / fix the problem is well appreciated.
> 
> Cheers,
> 
> Dieter
> 

I can geneate 30,000 I/O's per second for hours on end on several types
of storage hardware on FreeBSD SMP, and have no problems.  Since you're
seeing this problem both when connected to a 3ware controller and when
connected to a simple ATA/SATA controller (both of which have also been
observed to do high amounts of I/O with no problems), I suspect that the
problem is with your disk device, not with FreeBSD.  I don't know
anything about a "hyperdrive" though, so more information might help.

Scott