Can't read a full block, only got 8193 bytes.
Ask Bjørn Hansen
ask at develooper.com
Sun Feb 19 06:37:14 UTC 2012
We're recycling an old database server with room for 16 disks as a backup server (our old database servers had 12-20 15k disks; the new ones one or two SSDs and they're faster).
We have a box running FreeBSD 8.2 with 7 disks in a ZFS raidz2 (and a spare). It's using an older 3ware card with all the disks (2TB WD green "ears" ones) setup as a "single" unit on the 3ware controller and though slow is basically working great. We have a small program to smartly purge old snapshots that I wrote after a year and tens of thousands of snapshots: https://github.com/abh/zfs-snapshot-cleaner
The new box is running 9.0 with a 3ware 9690SA-4I4E card with the latest firmware (4.10.00.024). We're using Seagate 3TB barracuda disks (big and cheap; good for backups).
Now for the problem: When running bonnie++ we get a few ZFS checksum errors and (weirder) we get this error from bonnie:
"Can't read a full block, only got 8193 bytes."
This seems to only be when testing a single ZFS disk or a UFS partition. Testing a raidz1 we just get checksum errors noted in zpool status, but no errors reading (though read speeds are ~10MB/second across four disks -- writing sequentially was ~230MB/second).
Any ideas where to start look?
Our best guess is that the 3ware controller can't play nicely with the disks; we're planning to try some older/smaller disks on Monday and then trying the same system and disks with Linux to see if the 3ware driver there works differently.
More information about the freebsd-stable