ZFS on Hardware RAID

[Steven Hartland]

On 21/01/2019 14:17, Borja Marcos wrote:
>
>> On 20 Jan 2019, at 16:33, jdelisle <jdelisle at gmail.com> wrote:
>> I can say for certain that with Dell PERC 6/e RAID (SAS 8087ELP based)
>> cards, when the virtual devices are entire disks each in a RAID0
>> configuration, the way the adapter writes data to them is effectively the
>> same as JBOD.  You can take a RAID0 drive from a PERC 6/e controller and
>> use in with an HBA/ "IT Mode" JBOD controller without issue.  I personally
>> migrated 14x 1TB drives off PERC 6/e (all RAID0's) to a SAS 6gbps HBA, and
>> my ZFS VDEV's and pools were totally fine.  You can move back and forth no
>> problem.  I can't say this is the case for ALL RAID controllers, I've only
>> done this with an old PERC 6/e.
> However a single disk RAID0 volume (what manufacturers call a JBOD) is not the same
> as a disk. With the “virtual JBOD” you lose direct CAM access, which means that some
> advanced options won’t be supported.
>
> For example, if you are using SSDs ZFS knows when to issue a TRIM (or, in general,
> BIO_DELETE) command. Does the “virtual JBOD” offer that option?
Typically yes it does.
> The same could be said of other functionalities available when you talk directly
> to the disks.
>
>
>> That's why I did as I describe above.  I didn't have the money to buy an
>> HBA until a year or two later, and I had good backups, so I used the PERC
>> 6/e.
> Depending on the LSI model used in that card you can make it behave like an HBA.
>
Indeed you can for most, but its not an easy or error free option, 
requires *force *flashing from IR to IT firmware, and if it goes wrong 
you can end up with a bricked controller.

We've used LSI's in IR mode (RAID) but as individual disks JBOD with ZFS 
raid2z or mirroring on top for a long time and on lots of machines, not 
had any issues to report. So while I would never use hardware RAID from 
those controllers with ZFS, JBOD on them is an easy and reliable option 
in our experience.

     Regards
     Steve