FreeBSD 10, ServeRAID M5210e, syspd corruption
Borja Marcos
borjam at sarenet.es
Fri Feb 14 12:44:35 UTC 2014
(crossposting to -Stable just in case)
Hello,
I am configuring an IBM server with FreeBSD 10-RELEASE, a ServeRAID M5210e and 23 SSD disks.
uname -a
FreeBSD hostname 10.0-RELEASE FreeBSD 10.0-RELEASE #1: Fri Feb 14 09:35:12 CET 2014 toor@
hostname:/usr/obj/usr/src/sys/GENERIC amd64
The server has a SAS backplane and a controller recognized by the mfi driver.
mfi0 Adapter:
Product Name: ServeRAID M5210e
Serial Number: 3CJ0SG
Firmware: 24.0.2-0013
RAID Levels: JBOD, RAID0, RAID1, RAID10
Battery Backup: not present
NVRAM: 32K
Onboard Memory: 0M
Minimum Stripe: 64K
Maximum Stripe: 64K
As I am intending to use ZFS, I need direct access to the disks, no need for fancy RAID features.
I have seen that the newest cards support a so-called "syspd" mode that gives direct acess to the disks.
However, in this configuration, syspd consistently corrupts data on the disks.
I have done tests with three models of disks:
- Samsung SSD 840 BB0Q (1 TB)
- OCZ-VERTEX4 1.5 (512 GB)
- SEAGATE ST9146803SS FS03 (136 GB)
In the three cases there is data corruption. Using FFS on the disks results in a panic if I run a benchmark, for example, bonnie++.
Using ZFS (I've been creating one disk pools to test) I don't get panics but the data is consistently corrupted. The writes work, but whenever
there is read activity (either bonnie++ reaching the "rewrite" phase, or a ZFS scrub), ZFS detects data corruption.
Trying the über neat hw.mfi.allow_cam_disk_passthrough (which is great, because ZFS can detect the SSDs and issue TRIM commands)
I get the same result: data corruption.
However, I have tried to create a one-disk raid0 volume, and in that case it works like a charm, no corruption at all, so I can safely
assume that this is not a defective backplane, expander or cabling.
So:
mfisyspd -> CORRUPT
da -> CORRUPT
mfid -> NOT CORRUPT
Any ideas? Could be a driver error or a firmware problem, I am clueless for now.
Anything I can test? The machine is not in production, I can try patches or whatever.
Thanks!!
Borja.
More information about the freebsd-stable
mailing list