hardware for home use large storage

Tue Feb 9 22:32:03 UTC 2010

On Tue, 9 Feb 2010, Jeremy Chadwick wrote:

> On Tue, Feb 09, 2010 at 06:53:26AM -0600, Karl Denninger wrote:
>> Jeremy Chadwick wrote:
>>> On Tue, Feb 09, 2010 at 05:21:32PM +1100, Andrew Snow wrote:
>>>
>>>> http://www.supermicro.com/products/motherboard/ATOM/ICH9/X7SPA.cfm?typ=H
>>>>
>>>> Supermicro just released a new Mini-ITX fanless Atom server board
>>>> with 6xSATA ports (based on Intel ICH9) and a PCIe 16x slot.  It
>>>> takes up to 4GB of RAM, and there's even a version with KVM-over-LAN
>>>> for headless operation and remote management.
>>>>
>>>
>>> Neat hardware.  But with regards to the KVM-over-LAN stuff: it's IPMI,
>>> and Supermicro has a very, *very* long history of having shoddy IPMI
>>> support.  I've been told the latter by too many different individuals in
>>> the industry (some co-workers, some work at Yahoo, some at Rackable,
>>> etc.) for me to rely on it.  If you *have* to go this route, make sure
>>> you get the IPMI module which has its own dedicated LAN port on the
>>> module and ***does not*** piggyback on top of an existing LAN port on
>>> the mainboard.
>>>
>> What's wrong with the Supermicro IPMI implementations?  I have several -
>> all have a SEPARATE LAN port on the main board for the IPMI KVM
>> (separate and distinct from the board's primary LAN ports), and I've not
>> had any trouble with any of them.
>
> http://unix.derkeiler.com/Mailing-Lists/FreeBSD/current/2008-01/msg01206.html
> http://forums.freebsd.org/showthread.php?t=7750
> http://www.beowulf.org/archive/2007-November/019925.html
> http://bivald.com/lessons-learned/2009/06/supermicro_ipmi_problems_web_i.html
> http://lists.freebsd.org/pipermail/freebsd-stable/2008-August/044248.html
> http://lists.freebsd.org/pipermail/freebsd-stable/2008-August/044237.html
>
> (Last thread piece does mention that the user was able to get keyboard
> working by disabling umass(4) of all things)

I have a box down at Softlayer (one of the few major server rental outfits 
that officially supports FreeBSD), and one of the reasons I went with them 
is that they advertised "IP-KVM support".  Turns out they run Supermicro 
boxes with the IPMI card.  It mostly works, but it is very quirky and you 
have to use a very wonky Java client app to get the remote console.  You 
have to build a kernel that omits certain USB devices to make the keyboard 
work over the KVM connection (and their stock FBSD install has it 
disabled).

I can usually get in, but sometimes I have to open a ticket with them and 
a tech does some kind of reset on the card.  I don't know if they a 
hitting a button on the card/chassis or if they have some way to do this 
remotely.  After they do that, I'll see something like this in dmesg:

umass0: <Peppercon AG Multidevice, class 0/0, rev 2.00/0.01, addr 2> on 
uhub4
ums0: <Peppercon AG Multidevice, class 0/0, rev 2.00/0.01, addr 2> on 
uhub4
ums0: 3 buttons and Z dir.
ukbd0: <Peppercon AG Multidevice, class 0/0, rev 2.00/0.01, addr 2> on 
uhub4
kbd2 at ukbd0

The umass device is to support the "virtual media" feature that simply 
does not work.  It's supposed to allow you to point the ipmi card at an 
iso or disk image on an SMB server and boot your server off of it.  I had 
no luck with this.

All the IPMI power on/off, reset, and hw monitoring functions do work well 
though.

> It gets worse when you use one of the IPMI modules that piggybacks on an
> existing Ethernet port -- the NIC driver for the OS, from the ground up,
> has to be fully aware of ASF and any quirks/oddities involved.  For
> example, on bge(4) and bce(4), you'll find this (bge mentioned below):
>
>  hw.bge.allow_asf
>        Allow the ASF feature for cooperating with IPMI.  Can cause sys-
>        tem lockup problems on a small number of systems.  Disabled by
>        default.
>
> So unless the administrator intentionally sets the loader tunable prior
> to booting the OS installation, they'll find all kinds of MAC problems
> as a result of the IPMI piggybacking.  "Why isn't this enabled by
> default?"  I believe because there were reports of failures/problems on
> people's systems who *did not* have IPMI cards.  Lose-lose situation.

I don't think they have this setup, or if they do, they are using it on 
the internal LAN, so I don't notice any weirdness.

> If you really want me to dig up people at Yahoo who have dealt with IPMI
> on thousands of Supermicro servers and the insanity involved (due to
> bugs, quirks, or implementation differences between the IPMI firmwares
> and which revision/model of module used), I can do so.  Most of the
> complaints I've heard of stem from serial-over-IPMI.  I don't think
> it'd be a very positive/"supportive" thread, however.  :-)
>
> One similar product that does seem to work well is iLO, available on
> HP/Compaq hardware.

I've heard great things about that.  It seems like a much better design - 
it's essentially a small server that is independent from the main host. 
Has it's own LAN and serial ports as well.

Charles

> -- 
> | Jeremy Chadwick                                   jdc at parodius.com |
> | Parodius Networking                       http://www.parodius.com/ |
> | UNIX Systems Administrator                  Mountain View, CA, USA |
> | Making life hard for others since 1977.              PGP: 4BD6C0CB |
>
> _______________________________________________
> freebsd-stable at freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-stable
> To unsubscribe, send any mail to "freebsd-stable-unsubscribe at freebsd.org"
>