SuperMicro IPMI/SOL and ipmitool troubles
John Baldwin
jhb at freebsd.org
Thu Nov 20 20:53:24 UTC 2014
On Thursday, November 20, 2014 1:09:06 pm Andreas Nilsson wrote:
> On Thu, Nov 20, 2014 at 5:45 PM, John Baldwin <jhb at freebsd.org> wrote:
>
> > On Wednesday, November 19, 2014 5:15:05 pm Andreas Nilsson wrote:
> > > On Wed, Nov 19, 2014 at 8:05 PM, John Baldwin <jhb at freebsd.org> wrote:
> > >
> > > > On Wednesday, November 19, 2014 05:02:49 PM Andreas Nilsson wrote:
> > > > > On Wed, Nov 19, 2014 at 3:28 PM, Andreas Nilsson <andrnils at gmail.com
> > >
> > > > wrote:
> > > > > > On Fri, Nov 14, 2014 at 7:30 PM, Dmitry Morozovsky <marck at rinet.ru
> > >
> > > > wrote:
> > > > > >> Daniel,
> > > > > >>
> > > > > >> nice to see you here too ;)
> > > > > >>
> > > > > >> On Fri, 14 Nov 2014, Daniel O'Connor wrote:
> > > > > >> > On 12 Nov 2014, at 19:43, Andreas Nilsson <andrnils at gmail.com>
> > > > wrote:
> > > > > >> > > unclear is the word for it :) And thanks for looking into
> > this.
> > > > > >>
> > > > > >> ipmi/ilo is
> > > > > >>
> > > > > >> > > important on a server os.
> > > > > >>
> > > > > >> > > I found a reference to it in a ML post:
> > > > > >>
> > > >
> > http://lists.freebsd.org/pipermail/freebsd-stable/2013-February/072464.ht
> > > > > >> ml
> > > > > >>
> > > > > >> > I started that thread :)
> > > > > >> > I did get it working on the hardware I was using (Supermicro
> > X9SCL-
> > F
> > > > > >>
> > > > > >> and X8SIL-F)
> > > > > >>
> > > > > >> > I used the following BIOS settings
> > > > > >> >
> > > > > >> > ? Remote Access - Enabled
> > > > > >> > ? Serial Port Number - COM3
> > > > > >> > ? Serial Port Mode - 115200, 8, n, 1
> > > > > >> > ? Flow Control - Hardware
> > > > > >> > ? Redirection After BIOS POST - Always
> > > > > >> > ? Terminal Type - VT100
> > > > > >> > ? VT-UTF8 Combo Key Support - Disabled
> > > > > >> > ? Sredir Memory Display Delay - No Delay
> > > > > >> >
> > > > > >> > And the following in loader.conf
> > > > > >> > # Give preference to VGA console
> > > > > >> > console="vidconsole,comconsole"
> > > > > >> > # Uncomment below and comment above to give serial console
> > > > preference
> > > > > >> > #console="comconsole,vidconsole"
> > > > > >> > comconsole_speed="115200"
> > > > > >> > boot_multicons="YES"
> > > > > >> > hint.uart.0.flags="0x0"
> > > > > >> > hint.uart.2.at="isa"
> > > > > >> > hint.uart.2.port="0x3E8"
> > > > > >> > hint.uart.2.flags="0x30"
> > > > > >> >
> > > > > >> > And this in /etc/ttys
> > > > > >> > # IPMI console
> > > > > >> > # Note: The Java console viewer doesn't seem to be very smart
> > as it
> > > > > >>
> > > > > >> doesn't
> > > > > >>
> > > > > >> > # properly support VT100
> > > > > >> > cuau2 "/usr/libexec/getty 3wire.115200" vt100 on
> > secure
> > > > > >> >
> > > > > >> > I could then access it using ipmitool like so
> > > > > >> > ipmitool -H remoteip -U ADMIN -I lanplus sol activate
> > > > > >> > [login]
> > > > > >> > export TERM=xterm
> > > > > >> >
> > > > > >> > Note that I wanted vidconsole by default because mostly the
> > systems
> > > > > >>
> > > > > >> were used by people local to them, however we could break into the
> > > > loader
> > > > > >> and type 'set console=comconsole,vidconsole? and then get
> > everything
> > > > over
> > > > > >> the serial console for remote trouble shooting.
> > > > > >>
> > > > > >> > You may also wish to check the IPMI configuration via the web
> > > > interface
> > > > > >>
> > > > > >> - by default it will failover to port 0 and it has terrible
> > default
> > > > > >> passwords. I changed the passwords and forced it to use the
> > dedicated
> > > > > >> IPMI
> > > > > >> port even if nothing was connected to it.
> > > > > >>
> > > > > >> Well, I'm almost done with most of our SM server, even
> > concentrated
> > > > > >> console on
> > > > > >> our console server with such a simple config:
> > > > > >>
> > > > > >> ---- 8< ----
> > > > > >> # ipmi/sol console template
> > > > > >> default ipmi {
> > > > > >>
> > > > > >> master localhost;
> > > > > >> type exec;
> > > > > >> exec /usr/local/bin/ipmitool -f
> > /usr/local/etc/ipmi-pass -
> > U
> > > > > >>
> > > > > >> root -I lanplus -H %.int sol activate;
> > > > > >>
> > > > > >> execsubst %=cs;
> > > > > >> #idletimeout 6h;
> > > > > >>
> > > > > >> break 0 { string "~B"; }
> > > > > >>
> > > > > >> }
> > > > > >>
> > > > > >> console gwn1 { include ipmi; }
> > > > > >> console gwn2 { include ipmi; }
> > > > > >> console gwn3 { include ipmi; }
> > > > > >> console gwn4 { include ipmi; }
> > > > > >> console gwn5 { include ipmi; }
> > > > > >> console gwn6 { include ipmi; }
> > > > > >> console gwn7 { include ipmi; }
> > > > > >> console gwn8 { include ipmi; }
> > > > > >>
> > > > > >> console gwc2 { include ipmi; }
> > > > > >> ---- 8< ----
> > > > > >>
> > > > > >> This has console logging (including possible panics) as a surplus
> > > > > >>
> > > > > >> --
> > > > > >> Sincerely,
> > > > > >> D.Marck [DM5020, MCK-RIPE,
> > > > DM3-RIPN]
> > > > > >> [ FreeBSD committer:
> > marck at FreeBSD.org
> > > > ]
> > > > > >>
> > > >
> > ------------------------------------------------------------------------
> > > > > >> *** Dmitry Morozovsky --- D.Marck --- Wild Woozle ---
> > marck at rinet.ru
> > > > ***
> > > > > >>
> > > >
> > ------------------------------------------------------------------------
> > > > > >
> > > > > > Hello again,
> > > > > >
> > > > > > Searching on hw.uart.console, I found:
> > > > > >
> > > >
> > http://lists.freebsd.org/pipermail/svn-src-head/2013-February/044641.html
> > > > > > , a very enlightening thread.
> > > > > >
> > > > > > Basically: "ohh, you want to use something other than COM1 and
> > tried
> > to
> > > > > > get away with just changing hint.uart stuff, which has worked for a
> > > > while,
> > > > > > ha, no way..." No heads up, nothing.
> > > > > >
> > > > > > Sorry to say jhb@ but is not a rare case. It is if not the
> > default, a
> > > > > > very common setup on every HP server with iLO, and it holds for
> > most
> > > > all
> > > > > > OOB style serial emulation I have ever had the (dis)pleasure of
> > working
> > > > > > with.
> > > >
> > > > This was done _specifically_ so you could use non-COM1 for both loader
> > and
> > > > kernel with one thing to change. That is, you don't use
> > hint.uart.X.flags
> > > > after this. I have used this with many SuperMicro servers that use
> > COM2
> > > > and
> > > > COM3 because I wanted the entire path (boot loader and kernel) to work,
> > not
> > > > the kernel only. Having only the kernel means I can't break into the
> > > > loader
> > > > prompt to boot a different kernel, single user, etc.
> > > >
> > > > To clarify, are you using _different_ serial ports for the loader vs
> > the
> > > > kernel? That is the use case I considered to be rare. Every single
> > server
> > > > I have ever worked with (though not iLO, mostly Dell and SuperMicro)
> > uses
> > > > the
> > > > same COM port for serial redirection rather for SOL or via actual
> > cables.
> > > > I've yet to use a system that, for example, used COM1 for the loader
> > and
> > > > COM2
> > > > for the kernel. You are saying that every HP server uses COM1 for the
> > > > loader
> > > > and COM2 for the kernel?
> > > >
> > > > --
> > > > John Baldwin
> > > >
> > >
> > > Hello,
> > >
> > > For Supermicro:
> > > The box was in a rack when I started this, so I don't really know how the
> > > bios is setup, but as far as I can gather, getting
> > > "BTX loader yadda yadda" seems to always happen, on any COM-port.
> > > The actual output from loading modules and so on only happens with
> > specific
> > > settings. Here ( as in an answer to Dmitry I accidentally sent in
> > private )
> > > things seems to be on different COMs: Setting: comconsole_port=0x2F8
> > (com2)
> > > show loader and goes dark on handoff to kernel
> > > comconsole_port=0x3E8 (com3) show no loader and stays dark
> > >
> > > Neither of them gives me console even after successful boot. The only way
> > > to get serial is setting hw.uart.console="br:9600" explicitly.
> >
> > On the SuperMicro systems I have played with (X8DTU and the X9DRW
> > equivalent),
> > comconsole_port=0x2f8 worked for both BIOS, loader, and kernel, but for
> > BIOS
> > to work I did have to enable Console Redirection in the BIOS. For 2U
> > machines
> > (X8DAH at least, I think also for X9 equivalent), SOL was on COM3 instead
> > of
> > COM2, but using comconsole_port alone worked for BIOS, loader, and kernel.
> >
> > I also set the speed to 115200 (conspeed) which IIRC might have been the
> > default setting in the BIOS when Console Redirection was enabled via SOL.
> > You can check that remotely for SOL via ipmitool by seeing what baud rate
> > it is set to. I think if IPMI expects 115200 and you use 9600, that might
> > explain black output.
> >
> > --
> > John Baldwin
> >
>
> Hello,
>
> I spent the day with new ( as in directly out of the box ) supermicro
> x9sc(a/i) machine, which as you say has SOL on com3. I don't have logs
> available now, but short summary was that only way to get it work was to
> set the hints.uart and then hw.uart.console="br:9600" ( or another speed )
> or some bogus value.
>
> I'll get back to you tomorrow with the specifics ( the files contains
> kenv|grep uart , dmesg | grep uart , tail -4 /boot/loader.conf and results
> with loader (shown|not shown), kernel output (shown|not shown) and login
> prompt (shown|not shown).
ipmitool sol info would also be good to have, e.g.:
Set in progress : set-complete
Enabled : true
Force Encryption : true
Force Authentication : true
Privilege Level : USER
Character Accumulate Level (ms) : 60
Character Send Threshold : 96
Retry Count : 7
Retry Interval (ms) : 1000
Volatile Bit Rate (kbps) : 115.2
Non-Volatile Bit Rate (kbps) : 115.2
Payload Channel : 1 (0x01)
Payload Port : 623
is from an X8DTU system.
--
John Baldwin
More information about the freebsd-stable
mailing list