hardware for home use large storage

[Karl Denninger]

Jeremy Chadwick wrote:
> On Tue, Feb 09, 2010 at 06:53:26AM -0600, Karl Denninger wrote:
>   
>> Jeremy Chadwick wrote:
>>     
>>> On Tue, Feb 09, 2010 at 05:21:32PM +1100, Andrew Snow wrote:
>>>   
>>>       
>>>> http://www.supermicro.com/products/motherboard/ATOM/ICH9/X7SPA.cfm?typ=H
>>>>
>>>> Supermicro just released a new Mini-ITX fanless Atom server board
>>>> with 6xSATA ports (based on Intel ICH9) and a PCIe 16x slot.  It
>>>> takes up to 4GB of RAM, and there's even a version with KVM-over-LAN
>>>> for headless operation and remote management.
>>>>     
>>>>         
>>> Neat hardware.  But with regards to the KVM-over-LAN stuff: it's IPMI,
>>> and Supermicro has a very, *very* long history of having shoddy IPMI
>>> support.  I've been told the latter by too many different individuals in
>>> the industry (some co-workers, some work at Yahoo, some at Rackable,
>>> etc.) for me to rely on it.  If you *have* to go this route, make sure
>>> you get the IPMI module which has its own dedicated LAN port on the
>>> module and ***does not*** piggyback on top of an existing LAN port on
>>> the mainboard.
>>>   
>>>       
>> What's wrong with the Supermicro IPMI implementations?  I have several -
>> all have a SEPARATE LAN port on the main board for the IPMI KVM
>> (separate and distinct from the board's primary LAN ports), and I've not
>> had any trouble with any of them.
>>     
>
> http://unix.derkeiler.com/Mailing-Lists/FreeBSD/current/2008-01/msg01206.html
> http://forums.freebsd.org/showthread.php?t=7750
> http://www.beowulf.org/archive/2007-November/019925.html
> http://bivald.com/lessons-learned/2009/06/supermicro_ipmi_problems_web_i.html
> http://lists.freebsd.org/pipermail/freebsd-stable/2008-August/044248.html
> http://lists.freebsd.org/pipermail/freebsd-stable/2008-August/044237.html
>
> (Last thread piece does mention that the user was able to get keyboard
> working by disabling umass(4) of all things)
>
> It gets worse when you use one of the IPMI modules that piggybacks on an
> existing Ethernet port -- the NIC driver for the OS, from the ground up,
> has to be fully aware of ASF and any quirks/oddities involved.  For
> example, on bge(4) and bce(4), you'll find this (bge mentioned below):
>
>   hw.bge.allow_asf
>         Allow the ASF feature for cooperating with IPMI.  Can cause sys-
>         tem lockup problems on a small number of systems.  Disabled by
>         default.
>
> So unless the administrator intentionally sets the loader tunable prior
> to booting the OS installation, they'll find all kinds of MAC problems
> as a result of the IPMI piggybacking.  "Why isn't this enabled by
> default?"  I believe because there were reports of failures/problems on
> people's systems who *did not* have IPMI cards.  Lose-lose situation.
>
> If you really want me to dig up people at Yahoo who have dealt with IPMI
> on thousands of Supermicro servers and the insanity involved (due to
> bugs, quirks, or implementation differences between the IPMI firmwares
> and which revision/model of module used), I can do so.  Most of the
> complaints I've heard of stem from serial-over-IPMI.  I don't think
> it'd be a very positive/"supportive" thread, however.  :-)
>
> One similar product that does seem to work well is iLO, available on
> HP/Compaq hardware.
>   
I load these things over the IPKVM all the time.  I leave a DVD-ROM in
the drive when I install them and my initial load is done over the IPKVM
on the board.  It "just works."

Maybe they have had trouble in the past (most of those complaints look
to be 2007/2008 issues), but the current stuff I use from them (their
dual XEON boards) haven't given me a lick of trouble.  And you can't
argue with the price of the boards I use, considering that they have
dual gigabit networking ports plus a separate IPMI LAN interface,
support ECC memory and dual Xeons.

I don't use the IPMI protocol itself but I **DO** use the remote console
and management over HTTPS.  No problems at all and FreeBSD has yet to
throw up on it in any way.

-- Karl