Cruel and unusual problems with Proliant ML350
koitsu at FreeBSD.org
Mon Nov 13 20:45:40 UTC 2006
On Mon, Nov 13, 2006 at 08:22:39PM +0100, Greg Byshenk wrote:
> Don't you really need to have a monitor, as well? I _have_ worked
> "blind" before, but I didn't enjoy it. I can imagine having a
> keyboard with me when wandering around, but wouldn't normally have
> a monitor. I had always thought that the preferred solution for
> this sort of case was to use a serial console.
Sure do (re: monitor). Most co-location facilities provide a "cart"
that contains both keyboards and monitors on it, which you can wheel
around and use for VGA console configuration.
If you notice, though, most keyboard vendors (Logitech, Microsoft,
Kensington, Dell, Apple, and I'm sure many others) are now selling
USB-only keyboards. They do not include a USB-to-PS/2 adapter (and
I'll remind people: the USB-to-PS/2 adapters you get for your mice
*will not* work with PS/2 keyboards! Different protocol, same socket).
So it's becoming hard to get PS/2 keyboards too!
> And what seems to be becoming common on servers is a BIOS that allows
> you to fully redirect to serial, including BIOS configuration. The
> servers that I have recently purchased have had a keyboard and monitor
> plugged into them _once_ -- for the first BIOS setup -- and then never
And I can speak about this a bit too, because I've spent the past 5
years fighting with serial console on x86 architecture in general.
BIOS-level serial console, generally speaking, is utter garbage on
most x86 servers I've dealt with. Sparcs have OpenBoot (or whatever
it's called), which is apparently fantastic.
Take SuperMicro, for example. These BIOSes provide only 3 options:
what COM port you want serial console to use, what speed, and "leave
Agent enabled after BIOS config". Some (hardly all) provide a 4th
option: what terminal emulation you want (ANSI, VT100+, etc.).
I won't talk about the emulation aspect, because anyone familiar with
BIOS-level serial console knows what you get: crap. Full screen
redraw a hundred times a second across a 9600bps connection? Yeah,
sounds good! Screen clears for no particular reason? Awesome.
Hey, did you want to read that BIOS output about "Bank 2 memory
bad?" Nope, too bad, it's gone, replaced with random escape
characters and missing data. Did you want hardware flow control
with that BIOS? No sorry, we don't offer that, you'll have to
accept missing characters. (Okay, so maybe I did talk about it.)
The "Agent enabled" option is what you want to use to get serial
console redirection to work with things like boot0, yadda yadda:
that is, redirect the literal 80x25 text VGA console to a serial
The problem is this: the instant any software on the host machine
touches the serial port interrupt in any way shape or form, the
BIOS Agent locks up, and you stop getting serial output altogether.
You get nothing. Dead in the water. Nada. Zilch.
I've spent a lot of time trying to get FreeBSD to **not** touch
the serial port in any way. I can get boot0 to behave, and even
boot2/loader to comply. But the instant the kernel loads (if I
remember right, you never even see the Copyright message), even
with device.hints saying sio.0.disabled="1" __or__ building the
kernel w/out any sio device at all, you lose the Agent.
Now I'll mention this part: has anyone here tried talking to
a motherboard vendor before about BIOS changes, fixes, or any-
thing of that nature? Good luck. You'll be stonewalled by a
technical support group who seems to think whatever it is you
want is a result of you not knowing what you're doing. "No,
your ACPI DSDT is incorrect, I need to talk to an engineer"
"OK we have forward mail to engineer, thanks you". *blink*
All of this is where two pieces of present-day technology win:
KVM-over-IP devices, and iLO/LOM cards. Now, the problem with
iLO/LOM are only available with specific vendors (HP/Compaq and
Sun), and many are known to be implemented in a bizarre way
(that is, "sharing" an Ethernet port with the actual host
mainboard. HP/Compaq doesn't do this, their iLO cards have
their own Ethernet port/NIC). The problem with the 'shared'
method is that it causes all sort-of ARP madness on local
networks. It reminds me of the problem with IPMI modules right
now, where the board essentially answers with two separate MAC
addresses, confusing ARP tables all over the place.
As for KVM-over-IP devices: fantastic in every way... except
for price. They're overpriced by 4x or so. What you get is
a generic 1U box which usually runs Linux, has some D-sub and
PS/2 break-out boxes (you have to buy each one for US$50 each
or so), uses open-source software, and has some ADCs. The cost?
Oh, a miniscule US$4500 lets you handle 8 devices.
There is a third solution: one of those PC Weasel cards. Awesome
idea, but overpriced. I'm not going to pay US$350 per PCI card,
I'm sorry. But in defence of the vendor, it's apparently one guy
building them and running the whole outfit, so I guess I can
understand the need for a higher profit margin and what not... but
still, overpriced by a ton when you think about the cost of
ICs that are used (they're in the cents range). Don't forget that
these might not fit into a low-profile 1U box (the height of the
card might not work with a riser/extender).
For administrators who run small non-profit (that is, literally no
profit *and* no cash-flow of any kind) organisations like myself,
these products are big bucks. I can afford to shell out US$4500
for a KVM-over-IP box, but the vendor had better be licking my boots
if I need any sort-of support, and had better be giving me free
| Jeremy Chadwick jdc at parodius.com |
| Parodius Networking http://www.parodius.com/ |
| UNIX Systems Administrator Mountain View, CA, USA |
| Making life hard for others since 1977. PGP: 4BD6C0CB |
More information about the freebsd-stable