RAID - hardware vs. ZFS

[rainer at ultra-secure.de]

Am 2020-05-12 18:07, schrieb Graham Allan via freebsd-fs:
> I have implemented ZFS on HPE SmartArray controllers, but not really
> out of choice - it was the hardware available. I'd prefer just to use
> a JBOD controller. You lose all the benefits of ZFS knowledge of
> hardware and drive state, but of course there are useful benefits to
> ZFS outside of that (volume management, snapshots etc) so it's not a
> total loss.
> 
> When I did create a ZFS pool on the hardware RAID, I just used a
> single large hardware RAID volume (I didn't try to expose the
> individual drives either as single-drive RAID-0 volumes, or drive
> pass-through). I found the drive pass-through on Gen9 SmartArray
> controllers to be very flaky, though the Gen10 ones look like they
> might be better.



What Gen9 controllers do is "authenticate" the drive first (or try to).

This means that when you first insert a replacement-drive, the 
controller checks if it's a genuine HPE drive.
The driver (at least in 11.3) interprets this as some kind of rejection 
and subsequently, the drive is ignored.
You have to reboot to make the drive known to FreeBSD.

Gen10 use the SmartPQI driver and I think it works better.
(P208/P408 HBAs).

HOWEVER: make sure you download the latest firmware update DVD 
(https://spp.hpe.com) and apply it. These things were unstable as hell 
otherwise.

Also, because there is no way to update the firmware from the OS, you 
need to boot from the ISO/USB.

In a server with three SmartPQI HBAs, a SAS expander and 32 disks 
(partly in an external shelf), updating all firmware (which was a couple 
of revisions behind for HBA, NICs, ILO etc.) took over an hour for a 
DL380Gen10.