SuperMicro IPMI/SOL and ipmitool troubles

Thu Nov 20 18:09:09 UTC 2014

On Thu, Nov 20, 2014 at 5:45 PM, John Baldwin <jhb at freebsd.org> wrote:

> On Wednesday, November 19, 2014 5:15:05 pm Andreas Nilsson wrote:
> > On Wed, Nov 19, 2014 at 8:05 PM, John Baldwin <jhb at freebsd.org> wrote:
> >
> > > On Wednesday, November 19, 2014 05:02:49 PM Andreas Nilsson wrote:
> > > > On Wed, Nov 19, 2014 at 3:28 PM, Andreas Nilsson <andrnils at gmail.com
> >
> > > wrote:
> > > > > On Fri, Nov 14, 2014 at 7:30 PM, Dmitry Morozovsky <marck at rinet.ru
> >
> > > wrote:
> > > > >> Daniel,
> > > > >>
> > > > >> nice to see you here too ;)
> > > > >>
> > > > >> On Fri, 14 Nov 2014, Daniel O'Connor wrote:
> > > > >> > On 12 Nov 2014, at 19:43, Andreas Nilsson <andrnils at gmail.com>
> > > wrote:
> > > > >> > > unclear is the word for it :) And thanks for looking into
> this.
> > > > >>
> > > > >> ipmi/ilo is
> > > > >>
> > > > >> > > important on a server os.
> > > > >>
> > > > >> > > I found a reference to it in a ML post:
> > > > >>
> > >
> http://lists.freebsd.org/pipermail/freebsd-stable/2013-February/072464.ht
> > > > >> ml
> > > > >>
> > > > >> > I started that thread :)
> > > > >> > I did get it working on the hardware I was using (Supermicro
> X9SCL-
> F
> > > > >>
> > > > >> and X8SIL-F)
> > > > >>
> > > > >> > I used the following BIOS settings
> > > > >> >
> > > > >> >       ? Remote Access - Enabled
> > > > >> >       ? Serial Port Number - COM3
> > > > >> >       ? Serial Port Mode - 115200, 8, n, 1
> > > > >> >       ? Flow Control - Hardware
> > > > >> >       ? Redirection After BIOS POST - Always
> > > > >> >       ? Terminal Type - VT100
> > > > >> >       ? VT-UTF8 Combo Key Support - Disabled
> > > > >> >       ? Sredir Memory Display Delay - No Delay
> > > > >> >
> > > > >> > And the following in loader.conf
> > > > >> > # Give preference to VGA console
> > > > >> > console="vidconsole,comconsole"
> > > > >> > # Uncomment below and comment above to give serial console
> > > preference
> > > > >> > #console="comconsole,vidconsole"
> > > > >> > comconsole_speed="115200"
> > > > >> > boot_multicons="YES"
> > > > >> > hint.uart.0.flags="0x0"
> > > > >> > hint.uart.2.at="isa"
> > > > >> > hint.uart.2.port="0x3E8"
> > > > >> > hint.uart.2.flags="0x30"
> > > > >> >
> > > > >> > And this in /etc/ttys
> > > > >> > # IPMI console
> > > > >> > # Note: The Java console viewer doesn't seem to be very smart
> as it
> > > > >>
> > > > >> doesn't
> > > > >>
> > > > >> > # properly support VT100
> > > > >> > cuau2   "/usr/libexec/getty 3wire.115200"       vt100   on
> secure
> > > > >> >
> > > > >> > I could then access it using ipmitool like so
> > > > >> > ipmitool -H remoteip -U ADMIN -I lanplus sol activate
> > > > >> > [login]
> > > > >> > export TERM=xterm
> > > > >> >
> > > > >> > Note that I wanted vidconsole by default because mostly the
> systems
> > > > >>
> > > > >> were used by people local to them, however we could break into the
> > > loader
> > > > >> and type 'set console=comconsole,vidconsole? and then get
> everything
> > > over
> > > > >> the serial console for remote trouble shooting.
> > > > >>
> > > > >> > You may also wish to check the IPMI configuration via the web
> > > interface
> > > > >>
> > > > >> - by default it will failover to port 0 and it has terrible
> default
> > > > >> passwords. I changed the passwords and forced it to use the
> dedicated
> > > > >> IPMI
> > > > >> port even if nothing was connected to it.
> > > > >>
> > > > >> Well, I'm almost done with most of our SM server, even
> concentrated
> > > > >> console on
> > > > >> our console server with such a simple config:
> > > > >>
> > > > >> ---- 8< ----
> > > > >> # ipmi/sol console template
> > > > >> default ipmi {
> > > > >>
> > > > >>         master  localhost;
> > > > >>         type    exec;
> > > > >>         exec    /usr/local/bin/ipmitool -f
> /usr/local/etc/ipmi-pass -
> U
> > > > >>
> > > > >> root -I lanplus -H %.int sol activate;
> > > > >>
> > > > >>         execsubst       %=cs;
> > > > >>         #idletimeout    6h;
> > > > >>
> > > > >>         break 0 { string "~B"; }
> > > > >>
> > > > >> }
> > > > >>
> > > > >> console gwn1    { include ipmi; }
> > > > >> console gwn2    { include ipmi; }
> > > > >> console gwn3    { include ipmi; }
> > > > >> console gwn4    { include ipmi; }
> > > > >> console gwn5    { include ipmi; }
> > > > >> console gwn6    { include ipmi; }
> > > > >> console gwn7    { include ipmi; }
> > > > >> console gwn8    { include ipmi; }
> > > > >>
> > > > >> console gwc2    { include ipmi; }
> > > > >> ---- 8< ----
> > > > >>
> > > > >> This has console logging (including possible panics) as a surplus
> > > > >>
> > > > >> --
> > > > >> Sincerely,
> > > > >> D.Marck                                     [DM5020, MCK-RIPE,
> > > DM3-RIPN]
> > > > >> [ FreeBSD committer:
> marck at FreeBSD.org
> > > ]
> > > > >>
> > >
> ------------------------------------------------------------------------
> > > > >> *** Dmitry Morozovsky --- D.Marck --- Wild Woozle ---
> marck at rinet.ru
> > > ***
> > > > >>
> > >
> ------------------------------------------------------------------------
> > > > >
> > > > > Hello again,
> > > > >
> > > > > Searching on hw.uart.console, I found:
> > > > >
> > >
> http://lists.freebsd.org/pipermail/svn-src-head/2013-February/044641.html
> > > > > , a very enlightening thread.
> > > > >
> > > > > Basically: "ohh, you want to use something other than COM1 and
> tried
> to
> > > > > get away with just changing hint.uart stuff, which has worked for a
> > > while,
> > > > > ha, no way..." No heads up, nothing.
> > > > >
> > > > > Sorry to say jhb@ but is not a rare case. It is if not the
> default, a
> > > > > very common setup on every HP server with iLO, and it holds for
> most
> > > all
> > > > > OOB style serial emulation I have ever had the (dis)pleasure of
> working
> > > > > with.
> > >
> > > This was done _specifically_ so you could use non-COM1 for both loader
> and
> > > kernel with one thing to change.  That is, you don't use
> hint.uart.X.flags
> > > after this.  I have used this with many SuperMicro servers that use
> COM2
> > > and
> > > COM3 because I wanted the entire path (boot loader and kernel) to work,
> not
> > > the kernel only.  Having only the kernel means I can't break into the
> > > loader
> > > prompt to boot a different kernel, single user, etc.
> > >
> > > To clarify, are you using _different_ serial ports for the loader vs
> the
> > > kernel?  That is the use case I considered to be rare.  Every single
> server
> > > I have ever worked with (though not iLO, mostly Dell and SuperMicro)
> uses
> > > the
> > > same COM port for serial redirection rather for SOL or via actual
> cables.
> > > I've yet to use a system that, for example, used COM1 for the loader
> and
> > > COM2
> > > for the kernel.  You are saying that every HP server uses COM1 for the
> > > loader
> > > and COM2 for the kernel?
> > >
> > > --
> > > John Baldwin
> > >
> >
> > Hello,
> >
> > For Supermicro:
> > The box was in a rack when I started this, so I don't really know how the
> > bios is setup, but as far as I can gather, getting
> > "BTX loader yadda yadda" seems to always happen, on any COM-port.
> > The actual output from loading modules and so on only happens with
> specific
> > settings. Here ( as in an answer to Dmitry I accidentally sent in
> private )
> > things seems to be on different COMs: Setting: comconsole_port=0x2F8
> (com2)
> > show loader and goes dark on handoff to kernel
> > comconsole_port=0x3E8 (com3) show no loader and stays dark
> >
> > Neither of them gives me console even after successful boot. The only way
> > to get serial is setting hw.uart.console="br:9600" explicitly.
>
> On the SuperMicro systems I have played with (X8DTU and the X9DRW
> equivalent),
> comconsole_port=0x2f8 worked for both BIOS, loader, and kernel, but for
> BIOS
> to work I did have to enable Console Redirection in the BIOS.  For 2U
> machines
> (X8DAH at least, I think also for X9 equivalent), SOL was on COM3 instead
> of
> COM2, but using comconsole_port alone worked for BIOS, loader, and kernel.
>
> I also set the speed to 115200 (conspeed) which IIRC might have been the
> default setting in the BIOS when Console Redirection was enabled via SOL.
> You can check that remotely for SOL via ipmitool by seeing what baud rate
> it is set to.  I think if IPMI expects 115200 and you use 9600, that might
> explain black output.
>
> --
> John Baldwin
>

Hello,

I spent the day with new ( as in directly out of the box ) supermicro
x9sc(a/i) machine, which as you say has SOL on com3. I don't have logs
available now, but short summary was that only way to get it work was to
set the hints.uart and then hw.uart.console="br:9600" ( or another speed )
or some bogus value.

Best regards
Andreas