Supermicro HBA

Valeri Galtsev galtsev at
Thu Jan 17 16:01:28 UTC 2019

On 1/17/19 8:28 AM, Tenzin Lhakhang wrote:
> If cost is negligible between Supermicro HBA using 3008 controller versus
> Broadcom/Avago/LSI card.  I'd say opt for the Broadcom card, their
> support + drivers + firmware downloads is much better than Supermicro's FTP
> site.  Broadcom keeps whitepapers, archive of all their past released
> firmwares.
> Must dos:
> - Make sure to do hot-add hot-remove disks (with workload).
> - Make sure to do disk led blink light tests.  You will have to use the
> sas3ircu utility, unless you have OS specific utilities to find them.
> - Perform an fio disk benchmark to get baseline system performance.
> Issues I've seen:
> - SuperMicro's FTP site for their firmware doesn't include archive copies,
> only latest.  Sometimes latest FW doesn't work nicely and hopefully you
> have an old FW downloaded prior.
> - SuperMicro FTP site:
> - Sometimes SM HBA's FW and LSI HBA's FW can be used interchangeable and
> sometimes not.  If you flashed a LSI HBA with SMC firmware, then you have
> to add -nossid when flashing using the sas3flash when going back to LSI
> firmware.  Sometimes it flashes fine and everything looks ok, you see all
> the disks, but when you blink disk 4, it blinks disk 1.  When you
> physically remove disk 2, ZFS says disk 3 is gone.  It's best to thoroughly
> qualify the system after FW flash.

Thanks, Tenzin, for nice write up!

I only would udd one general thing about Sipermicro. If you get 
Supermicro system board ("motherboard" is common jargon name for system 
board for over couple of decades), then make sure you go with Intel 
CPUs. If you need/prefer AMD CPUs, stay away from Supermicro. They are 
notoriously poorly designing system boards for AMD CPUs. I've seen 
several bad/flaky due to poor design myself (that is for AMD CPUs, not 
Intel); you also can search reports of that on the web. In my book one 
bad is one too many if it is you who got that. I tend to get Tyan ones 
whenever I can, and definitely when I want AMD CPUs. Tyan is in server 
board business since forever, and they never had flops I would hear about.

Sorry about small rant; I hope this may help someone.


> Broadcom page for 9300-8i
> -
> - It's a bit hard to understand all the different downloads, but they're
> helpful when you understand the workflow to flash FW and use the sas3ircu
> utility.  In the latest release, they've bundled all the required downloads
> into the Firmware download.  In the past you would have to individually
> find the different ROM and BIN files.
> All said, Supermicro HBA when it's working, it works pretty well.  I've
> seen HBA failures, but the systems were usually 5+ years old.
> - If you have multiple HBAs in a system, record which physical controller
> is on PCIe slot is the logical controller reported by OS.  Otherwise when a
> single HBA fails, it's a bit hard to trace which software HBA is which
> physical HBA.
> Thanks,
> Tenzin
> On Thu, Jan 17, 2019 at 7:51 AM Paul Pathiakis via freebsd-fs <
> freebsd-fs at> wrote:
>>   I've used the LSI 3008.  I haven't had any issues with it.  It is
>> supported by that driver.  (I haven't built a server with one lately)
>> Yes, avoid all the hardware RAID cards they are unnecessary and a JBOD
>> controller with ZFS is a good choice.  Make sure it supports the SAS3 spec
>> of 12 Gb/s.  That's where the speed is.  I found the following as an FYI.
>> Supermicro LSI SAS3008 HBA Review | - Storage Reviews
>> |
>> |
>> |
>> |  |  |
>>   |
>>   |
>> |
>> |  |
>> Supermicro LSI SAS3008 HBA Review | - Storage Reviews
>> The Supermicro LSI SAS3008 HBAs (which share the same controller as the
>> LSI 9300-8i HBAs) are engineered to deli...
>>   |
>>   |
>>   |
>> I looked at it as more of a specs education.  It looks solid.  I've used
>> Supermicro without issue in the past.  (I just decommissioned my home
>> server which was Supermicro and has that very card in it.)
>> There is a lot to consider when using ZFS beyond just the hardware.  Don't
>> get me wrong.... I want to have ZFS' baby.  :D  Just be sure of all the
>> nuances of HDD, SSD, Hybrids, how much memory you have to dedicate to ZFS
>> and CPU cores you have.  You have to take into consideration all the ZFS
>> features you're planning to make use of now and in the future.
>> Also, are you ever planning on expanding the storage to have an additional
>> JBOD shelf?  If so, you may want a card with some external connectors.
>> Some of the people at have done some serious
>> research on application specific throughput of ZFS and I believe they also
>> spec out SuperMicro servers too.  It comes down to IOPS, raw throughput,
>> etc.  (I'm actually talking to them right now about some very large backup
>> servers that can handle 0.75 PB.... The consideration I have is space and
>> using RAIDZ2 and multiple streams from 10Gb interfaces and serious
>> compression and deduplication.  SO,  IOPS not so much, but heavy raw I/O
>> and RAID checksum computation and dedup.
>> There's also things like dedicated SSDs as ZIL and cache to be thought
>> about.
>> So:  go up the theoretical OSI layer model and optimize each layer right
>> through the application layer. :D  (I actually find it fun)
>> I hope this all helps.
>> P
>>      On Thursday, January 17, 2019, 4:33:38 AM CST, Julien Cigar
>> <julien at> wrote:
>>   Hello,
>> We are planning to replace some (web) applications servers (currently
>> running HPE) with Supermicro and the vendor offers the following
>> choices for the Hardware Raid Controller/HBA 4P part:
>> 1) Supermicro AOC-S3008L-L8E, LSI 3008 8 x SATA/SAS III JBOD controller, up
>> to 122 hard drives via expander backplane, PCI-E, ideal for Nexenta/ZFS
>> + € 205,6
>> 2) LSI 9300-4I, SATA/SAS III JBOD controller, tot 256 harde schijven via
>> expander backplanes, PCI-E, ideal for Nexenta/ZFS
>> + € 214,5
>> 3) LSI MegaRAID 9341-4i bulk, 4 x SATA/SAS 12Gbs internal entry level
>> hardware RAID, no cache/BBU possible, PCI-e
>> + € 177,97
>> 4) LSI MegaRAID 9361-4i 1GB cache, 4 x SATA/SAS 12Gbs internal
>> hardware RAID, max. 240 hdd using expander backplanes
>> + € 401,7
>> 5) LSI 9300-4i4e, SATA/SAS III JBOD controller, 4 x internal, 4 x external,
>> up to 256 hard drives via expander backplane, Nexenta Certified, ideal
>> for ZFS, PCI-E
>> + € 273,95
>> 6) LSI MegaRAID 9380-4i4e bulk, 8 x SATA/SAS 12Gbs, 4 x external and 4 x
>> internal hardware RAID, 1024MB cache, up to 128 hard drives via expander
>> backplane, support for SSD CacheCade 2.0 write and read caching,
>> CacheVault support (advised), ideal for high en
>> + € 676
>> As the plan is to use ZFS, I was planning to choose the
>> "Supermicro AOC-S3008L-L8E, LSI 3008" (it looks like it is supported by
>> mpr) and I was wondering if anyone has any feedback on it ? Would
>> another option be a better choice ?
>> Thank you!
>> Julien
>> --
>> Julien Cigar
>> Belgian Biodiversity Platform (
>> PGP fingerprint: EEF9 F697 4B68 D275 7B11  6A25 B2BB 3710 A204 23C0
>> No trees were killed in the creation of this message.
>> However, many electrons were terribly inconvenienced.
>> _______________________________________________
>> freebsd-fs at mailing list
>> To unsubscribe, send any mail to "freebsd-fs-unsubscribe at"
> _______________________________________________
> freebsd-questions at mailing list
> To unsubscribe, send any mail to "freebsd-questions-unsubscribe at"

Valeri Galtsev
Sr System Administrator
Department of Astronomy and Astrophysics
Kavli Institute for Cosmological Physics
University of Chicago
Phone: 773-702-4247

More information about the freebsd-questions mailing list