ZFS on Hardware RAID

[Xin Li]

On 1/20/19 01:24, Maciej Jan Broniarz wrote:
> Hi,
> 
> I am thinking about the scenario with ZFS on single disks configured to RAID0 by hw raid.
> Please correct me, if i'm wrong, but HW Raid uses a dedicated unit to process all RAID related work (eg. parity checks).
> With ZFS the job is done by CPU. How significant is the performance loss in that particular case?

But you don't really skip that cost when layering ZFS on top of a
hardware RAID.

ZFS still does checksumming, etc., and you really want it to do its job
because a) it would tell you which files were corrupted in the worst
case scenario, and b) when doing scrubs, ZFS only scans areas with data,
while in hardware RAID case, the filesystem is opaque and it has to scan
everything.

> 
> mjb
> 
> 
> ----- Oryginalna wiadomość -----
> Od: "andy thomas" <andy at time-domain.co.uk>
> Do: "Rich" <rincebrain at gmail.com>
> DW: "Maciej Jan Broniarz" <gausus at gausus.net>, "freebsd-fs" <freebsd-fs at freebsd.org>
> Wysłane: niedziela, 20 styczeń 2019 9:45:21
> Temat: Re: ZFS on Hardware RAID
> 
> I have to agree with your comment that hardware RAID controllers add 
> another layer of opaque complexity but for what it's worth, I have to 
> admit ZFS on h/w RAID does work and can work well in practice.
> 
> I run a number of very busy webservers (Dell PowerEdge 2950 with LSI 
> MegaRAID SAS 1078 controllers) with the first two disks in RAID 1 as the 
> FreeBSD system disk and the remaining 4 disks configured as RAID 0 virtual 
> disks making up a ZFS RAIDz1 pool with 3 disks plus one hot spare. 
> With 6-10 jails running on each server, these have been running for 
> years with no problems at all.
> 
> Andy
> 
> On Sat, 19 Jan 2019, Rich wrote:
> 
>> The two caveats I'd offer are:
>> - RAID controllers add an opaque complexity layer if you have problems
>> - e.g. if you're using single-disk RAID0s to make a RAID controller
>> pretend to be an HBA, if the disk starts misbehaving, you have an
>> additional layer of behavior (how the RAID controller interprets
>> drives misbehaving and shows that to the OS) to figure out whether the
>> drive is bad, the connection is loose, the controller is bad, ...
>> - abstracting the redundancy away from ZFS means that ZFS can't
>> recover if it knows there's a problem but the underlying RAID
>> controller doesn't - that is, say you made a RAID-6, and ZFS sees some
>> block fail checksum. There's not a way to say "hey that block was
>> wrong, try that read again with different disks" to the controller, so
>> you're just sad at data loss on your nominally "redundant" array.
>>
>> - Rich
>>
>> On Sat, Jan 19, 2019 at 11:44 AM Maciej Jan Broniarz <gausus at gausus.net> wrote:
>>>
>>> Hi,
>>>
>>> I want to use ZFS on a hardware-raid array. I have no option of making it JBOD. I know it is best to use ZFS on JBOD, but
>>> that possible in that particular case. My question is - how bad of an idea is it. I have read very different opinions on that subject, but none of them seems conclusive.
>>>
>>> Any comments and especially case studies are most welcome.
>>> All best,
>>> mjb
>>> _______________________________________________
>>> freebsd-fs at freebsd.org mailing list
>>> https://lists.freebsd.org/mailman/listinfo/freebsd-fs
>>> To unsubscribe, send any mail to "freebsd-fs-unsubscribe at freebsd.org"
>> _______________________________________________
>> freebsd-fs at freebsd.org mailing list
>> https://lists.freebsd.org/mailman/listinfo/freebsd-fs
>> To unsubscribe, send any mail to "freebsd-fs-unsubscribe at freebsd.org"
>>
> 
> 
> ----------------------------
> Andy Thomas,
> Time Domain Systems
> 
> Tel: +44 (0)7866 556626
> Fax: +44 (0)20 8372 2582
> http://www.time-domain.co.uk
> _______________________________________________
> freebsd-fs at freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-fs
> To unsubscribe, send any mail to "freebsd-fs-unsubscribe at freebsd.org"
> 


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: OpenPGP digital signature
URL: <http://lists.freebsd.org/pipermail/freebsd-fs/attachments/20190120/687a513c/attachment.sig>