[SUSPECTED SPAM]Re: HEADS UP: FreeBSD/Xen dom0 UEFI support

[Roger Pau Monné]

On Sat, May 08, 2021 at 07:22:37PM -0600, Stephen Walker-Weinshenker wrote:
> On Mon, Apr 26, 2021 at 3:30 AM Roger Pau Monné <roger.pau at citrix.com>
> wrote:
> 
> > On Sun, Apr 25, 2021 at 05:10:44PM -0600, Stephen Walker-Weinshenker wrote:
> > > On Thu, Apr 22, 2021 at 12:58 AM Roger Pau Monné <roger.pau at citrix.com>
> > > wrote:
> > >
> > > > On Wed, Apr 21, 2021 at 08:32:18PM -0600, Stephen Walker-Weinshenker
> > wrote:
> > > > > Hello Roger:
> > > > >
> > > > > Thank you for your work on this. I am new to Xen and FreeBSD but am
> > > > willing
> > > > > to help test this and get it in a release version.
> > > > >
> > > > > I have 3 Intel NUC 8v5PNK that I am planning on using for Xen hosts,
> > but
> > > > > they can only boot in UEFI mode (intel disabled BIOS booting per
> > their
> > > > > official documentation even though it is still an option in the BIOS
> > > > > /shrug). Until this patch came along I could not get any Xen
> > distribution
> > > > > to boot since they all require BIOS mode.
> > > > >
> > > > > I have 2 of the 3 NUCS running 14.0-CURRENT at git commit
> > > > > 55fc118be8f0cd20e2123cfd2728c5f32317a9fa (after your patch was
> > merged)
> > > > and
> > > > > I have attempted to follow the instructions at
> > > > >
> > > >
> > https://docs.freebsd.org/en_US.ISO8859-1/books/handbook/virtualization-host-xen.html
> > > > > in order to get the Xen kernel booting. Unfortunately, when I
> > rebooted
> > > > > after making the changes, I get an error saying:
> > > > >
> > > > > >Loading Xen kernel...
> > > > > >Failed to load Xen kernel '/boot/xen'
> > > > > >can't load 'kernel'
> > > > > >
> > > > > >can't load 'kernel'
> > > >
> > > > Did you update the loader on the EFI partition?
> > > >
> > > > After an installworld on my box I would usually do:
> > > >
> > > > # mount -t msdosfs /dev/ada0p1 /mnt/
> > > > # cp /boot/loader.efi /mnt/EFI/freebsd/loader.efi
> > > > # umount /mnt
> > > >
> > > > To update the EFI loader. You might have to check the output of
> > > > `efibootmgr -v`to make sure you are replacing the correct loader. I
> > > > guess there must also be a way to do this with efibootmgr.
> > > >
> > > > For whatever reason my EFI partition nvd0p1 was already mounted at
> > > /boot/efi,
> > > so my copy command was
> > > # cp -f /boot/loader.efi /boot/efi/efi/freebsd/loader.efi
> > >
> > > I did this prior to the last reboot after changing the boot config as
> > > recommended by
> > >
> > > This fixed the issue of the xen kernel not booting on one of the boxes.
> > > (did the copy prior to the final reboot) Looks like I will have to
> > > reinstall the OS on the other one (not a big deal)
> > >
> > > Now, I am just getting a blank screen on boot. I am unable to ssh into
> > the
> > > Dom0 either.
> >
> > What's your Xen command line? (ie: xen_cmdline string in
> > loader.conf)
> >
> > Which version of Xen are you using?
> >
> > Do you have serial output available on those boxes? That might help
> > get some messages if everything else fails.
> >
> > Roger.
> >
> 
> Sorry for the delay in responding. Real life took precedence over this
> testing.
> 
> xen_cmdline="dom0_mem=2048M dom0_max_vcpus=1 dom0=pvh com1=115200,8n1
> guest_loglvl=all loglvl=all console=vga,com1"

Can you try adding 'watchdog' to xen_cmdline and see if that forces
the box to reboot after it goes black?

If Xen is able to boot, and the problem is with FreeBSD dom0 you might
try also adding vga=keep to xen_cmdline, that way we might get some
output from dom0 (if that's indeed where the issue is happening).

> xen-kernel version = 4.14.1_1
> xen-tools version = 4.14.1_1
> 
> Both installed from packages, rather than ports.
> 
> These boxes are Intel NUCs that I normally manage over intel AMT using mesh
> commander. I tried enabling the virtual serial port through AMT but no
> dice.

I never tried using the AMT serial port, first thing would be to get
it to work with plain FreeBSD, then we can see about getting it
working with Xen. FWIW, a lot of the test boxes I work with use SoL,
and I would expect the AMT serial to work as well.

> I know the nucs have a hardware serial port, but I don't have the
> cable to attach to the header at the moment. I have ordered some headers
> and crimp pins to attach to the serial port for more debugging, but they
> will not get here for about a week.

I did the original EFI work on a NUC7i7DNHE using the serial port.
Last time I tried HDMI worked with Xen.

> I also tried connecting directly to the NUC's HDMI port but I just got a
> blank screen again, after all the boot messages appeared.

Are those messages that you see Xen boot messages or FreeBSD boot
messages?

Does the box reboot itself?

I'm afraid we will require some more information to help diagnose
what's going on.

Thanks, Roger.