Recommended HBA for ZFS, contemporary

Mon Aug 22 19:01:40 UTC 2016

On 8/22/16 12:55 PM, Ben RUBSON wrote:
>> On 22 Aug 2016, at 19:13, Linda Kateley <lkateley at kateley.com> wrote:
>>
>> Not just IT firmware, but you need to make sure the driver and firmware match.
>>
>> I try to use 16 if possible, 20 works but I have only found the 20.00.04.00 version of firmware to be reliable. Would love to hear if people have different experience. I am about to try the rocket raid in IT mode because I am tired of fighting with lsi firmware. Very very stable when you get the right combo, 1000's of phantom checksum errors if not.
> FreeBSD 10.3 :
> # egrep ^mps0 /var/run/dmesg.boot
> mps0: <Avago Technologies (LSI) SAS2008>
> mps0: Firmware: 20.00.04.00, Driver: 20.00.00.00-fbsd
>
> FreeBSD 11-RC1 :
> # egrep ^mps0 /var/run/dmesg.boot
> mps0: <Avago Technologies (LSI) SAS2008>
> mps0: Firmware: 20.00.04.00, Driver: 21.01.00.00-fbsd
>
> Strangely enough there is no P21 driver/firmware @Avago :
> http://www.avagotech.com/products/server-storage/host-bus-adapters/sas-9211-8i#downloads
> So I assume firmware P20 is the right one to use with FreeBSD 11.
> Well it is : https://forums.freenas.org/index.php?threads/firmware-version-p21.45130/#post-304399
I have worked with those guys, and they consistently say there is no 
difference in the firmware revs, but... I have seen major differences.. 
The 21 sounds like an oem release. Not sure how you can putback and 
require oem driver on an opensource release. I have a bunch of systems 
that I just downrev'ed the freenas, or didn't upgrade. If there is a 21 
oem driver and if you are running supermicro hardware you should be able 
to get the driver/firmware from them.

Avago has the drivers and firmware for each release. Just want to make 
sure they are the same. Would love to leave everything on 16 for 6gb and 
9 for 12gb(actually I really like the version 10 on 12g sas but avago 
pulled it). Both those are tried and true..

If you have a 9211-8e or i that's working, life is good. The other thing 
I have tested is running the firmware for i(internal) card on an 
e(external) card and seems to have no impact. Also have taken the 
firmware for the 9300 card and put in on a roc and it worked just fine, 
then the next system I work on it doesn't even go in..

>
> I did not test firmwares 20.00.05.00 / 20.00.06.00 / 20.00.07.00.
> Did you ?
Yes I tested .07. I did well with it, but I have had others tell me they 
got the phantom checksum errors so I will rerun my tests.. I am actually 
working on a system right now that is showing phantom checksum errors 
with the 07 release, so hoping to downrev..

I really can't wait to start testing something else besides lsi, after 
20 years hoping to find something new that works, actually almost 
everything else I have worked with works, but there is always some 
caveat ....
I have a quanta coming next week with a rocket raid. I have friends in 
eu that swear by the quanta stuff and I work with a company that says 
they have 100's of them deployed, so it's worth a try..I am also going 
back to retesting some perc's.
> Unfortunately changelog for 20.00.05.00 is no more available online, so no idea what it does.
> The 2 others are, 3 + 1 defects corrected, should be worth it at first glance.
It is really frustrating when trying to deploy more than one system. I 
keep copies of most of the older firmware just in case. The 
documentation for avago firmware is horrific. The stuff that works well 
is the old stuff the 9211's and 9300's. The newer stuff like roc's are 
very hard to get firmware for unless you find the handful of people who 
know at supermicro or lsi. When you get into oem, then you have to trust 
your vendor. There is a about 30 or so storage vendors that i know of(so 
there is probably a bunch more) who have just taken freebsd or freenas 
or omni and put it onto some supermicro hardware and are selling as a 
storage product. There are only a few I have run into who have created 
value-add.
>
>> I am starting to like 12g 3008 cards better as the firmware seems to be a little more stable at version 9. Been testing 12 and so far so good, but haven't run my larger stress tests yet..
>>
>> I would definitely go to the avago site and make sure there is driver and firmware for the card you are purchasing before you buy.
>>
>> Linda
>>
>> On 8/22/16 11:18 AM, Ben RUBSON wrote:
>>>> On 22 Aug 2016, at 18:14, InterNetX - Juergen Gotteswinter <juergen.gotteswinter at internetx.com> wrote:
>>>>
>>>> LSI SAS 2008 Based HBA, there are dozens of OEM available for Budget Prices
>>> With the IT firmware.
>>>
>>>> Am 22.08.2016 um 18:08 schrieb Dmitry Morozovsky:
>>>>> Dear colleagues,
>>>>>
>>>>> what is the list of preferred disk controllers to build storage server, say
>>>>> 12-24 amd more 3.5" disks?  Not too much IOPS are expected, and some could be
>>>>> mitigated with L2ARC with only metadata cached, I suppose.
>>>>>
>>>>> Two usage schemes are file server for big files (mostly cold storage) and
>>>>> backup/surveillance server (constant stream writes, occasional random reads)
>>>>>
>>>>> Any hints? Thanks!
>>>>>
>>>> _______________________________________________
>>>> freebsd-fs at freebsd.org mailing list
>>>> https://lists.freebsd.org/mailman/listinfo/freebsd-fs
>>>> To unsubscribe, send any mail to "freebsd-fs-unsubscribe at freebsd.org"
>>> _______________________________________________
>>> freebsd-fs at freebsd.org mailing list
>>> https://lists.freebsd.org/mailman/listinfo/freebsd-fs
>>> To unsubscribe, send any mail to "freebsd-fs-unsubscribe at freebsd.org"
>> _______________________________________________
>> freebsd-fs at freebsd.org mailing list
>> https://lists.freebsd.org/mailman/listinfo/freebsd-fs
>> To unsubscribe, send any mail to "freebsd-fs-unsubscribe at freebsd.org"
> _______________________________________________
> freebsd-fs at freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-fs
> To unsubscribe, send any mail to "freebsd-fs-unsubscribe at freebsd.org"