SuperMicro IPMI/SOL and ipmitool troubles

[John Baldwin]

On Wednesday, November 19, 2014 05:02:49 PM Andreas Nilsson wrote:
> On Wed, Nov 19, 2014 at 3:28 PM, Andreas Nilsson <andrnils at gmail.com> wrote:
> > On Fri, Nov 14, 2014 at 7:30 PM, Dmitry Morozovsky <marck at rinet.ru> wrote:
> >> Daniel,
> >> 
> >> nice to see you here too ;)
> >> 
> >> On Fri, 14 Nov 2014, Daniel O'Connor wrote:
> >> > On 12 Nov 2014, at 19:43, Andreas Nilsson <andrnils at gmail.com> wrote:
> >> > > unclear is the word for it :) And thanks for looking into this.
> >> 
> >> ipmi/ilo is
> >> 
> >> > > important on a server os.
> >> 
> >> > > I found a reference to it in a ML post:
> >> http://lists.freebsd.org/pipermail/freebsd-stable/2013-February/072464.ht
> >> ml
> >> 
> >> > I started that thread :)
> >> > I did get it working on the hardware I was using (Supermicro X9SCL-F
> >> 
> >> and X8SIL-F)
> >> 
> >> > I used the following BIOS settings
> >> > 
> >> >       ? Remote Access - Enabled
> >> >       ? Serial Port Number - COM3
> >> >       ? Serial Port Mode - 115200, 8, n, 1
> >> >       ? Flow Control - Hardware
> >> >       ? Redirection After BIOS POST - Always
> >> >       ? Terminal Type - VT100
> >> >       ? VT-UTF8 Combo Key Support - Disabled
> >> >       ? Sredir Memory Display Delay - No Delay
> >> > 
> >> > And the following in loader.conf
> >> > # Give preference to VGA console
> >> > console="vidconsole,comconsole"
> >> > # Uncomment below and comment above to give serial console preference
> >> > #console="comconsole,vidconsole"
> >> > comconsole_speed="115200"
> >> > boot_multicons="YES"
> >> > hint.uart.0.flags="0x0"
> >> > hint.uart.2.at="isa"
> >> > hint.uart.2.port="0x3E8"
> >> > hint.uart.2.flags="0x30"
> >> > 
> >> > And this in /etc/ttys
> >> > # IPMI console
> >> > # Note: The Java console viewer doesn't seem to be very smart as it
> >> 
> >> doesn't
> >> 
> >> > # properly support VT100
> >> > cuau2   "/usr/libexec/getty 3wire.115200"       vt100   on secure
> >> > 
> >> > I could then access it using ipmitool like so
> >> > ipmitool -H remoteip -U ADMIN -I lanplus sol activate
> >> > [login]
> >> > export TERM=xterm
> >> > 
> >> > Note that I wanted vidconsole by default because mostly the systems
> >> 
> >> were used by people local to them, however we could break into the loader
> >> and type 'set console=comconsole,vidconsole? and then get everything over
> >> the serial console for remote trouble shooting.
> >> 
> >> > You may also wish to check the IPMI configuration via the web interface
> >> 
> >> - by default it will failover to port 0 and it has terrible default
> >> passwords. I changed the passwords and forced it to use the dedicated
> >> IPMI
> >> port even if nothing was connected to it.
> >> 
> >> Well, I'm almost done with most of our SM server, even concentrated
> >> console on
> >> our console server with such a simple config:
> >> 
> >> ---- 8< ----
> >> # ipmi/sol console template
> >> default ipmi {
> >> 
> >>         master  localhost;
> >>         type    exec;
> >>         exec    /usr/local/bin/ipmitool -f /usr/local/etc/ipmi-pass -U
> >> 
> >> root -I lanplus -H %.int sol activate;
> >> 
> >>         execsubst       %=cs;
> >>         #idletimeout    6h;
> >>         
> >>         break 0 { string "~B"; }
> >> 
> >> }
> >> 
> >> console gwn1    { include ipmi; }
> >> console gwn2    { include ipmi; }
> >> console gwn3    { include ipmi; }
> >> console gwn4    { include ipmi; }
> >> console gwn5    { include ipmi; }
> >> console gwn6    { include ipmi; }
> >> console gwn7    { include ipmi; }
> >> console gwn8    { include ipmi; }
> >> 
> >> console gwc2    { include ipmi; }
> >> ---- 8< ----
> >> 
> >> This has console logging (including possible panics) as a surplus
> >> 
> >> --
> >> Sincerely,
> >> D.Marck                                     [DM5020, MCK-RIPE, DM3-RIPN]
> >> [ FreeBSD committer:                                 marck at FreeBSD.org ]
> >> ------------------------------------------------------------------------
> >> *** Dmitry Morozovsky --- D.Marck --- Wild Woozle --- marck at rinet.ru ***
> >> ------------------------------------------------------------------------
> > 
> > Hello again,
> > 
> > Searching on hw.uart.console, I found:
> > http://lists.freebsd.org/pipermail/svn-src-head/2013-February/044641.html
> > , a very enlightening thread.
> > 
> > Basically: "ohh, you want to use something other than COM1 and tried to
> > get away with just changing hint.uart stuff, which has worked for a while,
> > ha, no way..." No heads up, nothing.
> > 
> > Sorry to say jhb@ but is not a rare case. It is if not the default, a
> > very common setup on every HP server with iLO, and it holds for most all
> > OOB style serial emulation I have ever had the (dis)pleasure of working
> > with.

This was done _specifically_ so you could use non-COM1 for both loader and
kernel with one thing to change.  That is, you don't use hint.uart.X.flags
after this.  I have used this with many SuperMicro servers that use COM2 and
COM3 because I wanted the entire path (boot loader and kernel) to work, not
the kernel only.  Having only the kernel means I can't break into the loader
prompt to boot a different kernel, single user, etc.

To clarify, are you using _different_ serial ports for the loader vs the
kernel?  That is the use case I considered to be rare.  Every single server
I have ever worked with (though not iLO, mostly Dell and SuperMicro) uses the
same COM port for serial redirection rather for SOL or via actual cables.
I've yet to use a system that, for example, used COM1 for the loader and COM2 
for the kernel.  You are saying that every HP server uses COM1 for the loader 
and COM2 for the kernel?

-- 
John Baldwin