Re: BETA5: Xen 4.19.2-pre domU crash/don't start

On Fri, Nov 21, 2025 at 07:37:27AM +0100, Manuel Kuklinski wrote:
> Hi!
> 
> Am Donnerstag 20 November 2025 um 18:19:55 +0100, schrieb Roger Pau Monné 20K:
> > Don't use nestedhvm, it doesn't work properly.  Does the VM work if
> > you disable the nestedhvm option?
> 
> I need nestedhvm, since I run GNS3/containerlab and vrnetlab in a Linux
> domU. I choose Xen, especially because of this capability. Before that I
> used bhyve, but the nested result was painfully slow :-(
> 
> Disabling nestedhvm didn't change the outcome, but please see below:

But those are containers, not VMs, and hence you provably don't need
nested HVM to run them?

Have you tried disabling the nestedhvm option and see if those still
work?

> > This seems concerning.  Don't you get more output here that contains
> > the instruction stream that failed to emulate?
> > 
> > Are you using a custom build of SeaBIOS?
> > 
> > Maybe the newer build of SeaBIOS is broken, possibly a result of the
> > update of the default GCC version to use.
> > 
> > You could try to resort to fetching a build of SeaBIOS from 14.3 ports
> > tree, if that makes a difference?  SeaBIOS is just a binary blob, so
> > doesn't have any dependency on where it's built.  You could even
> > pickup a SeaBIOS blob built on Linux and place it in
> > /usr/local/share/seabios/bios.bin.
> > 
> > > (XEN) d4v0 VMRESUME error: 0x7
> > 
> > According to the SDM this is:
> > 
> > "VM entry with invalid control field(s)"
> > 
> > Which could be caused by the guest attempting to enable the
> > virtualization extensions inside the VM, and thus corrupting the VM
> > state.
> > 
> > I haven't checked the control fields dumped below for sanity, because
> > it's a pain to do it manually.  We should add an automatic checker in
> > Xen that points out the incompatibilities, but alas.
> > 
> > Let me know if removing `nestedhvm=1` from the xl.cfg file solves the
> > issue.  Or attempting to boot with a good know version of SeaBIOS,
> > I've uploaded my current SeaBIOS build to:
> > 
> > https://people.freebsd.org/~royger/bios.bin
> > 
> > If you want to give it a try.  You should just copy it to
> > /usr/local/share/seabios/bios.bin replacing your existing build.
> 
> The SeaBIOS binary you provided did the trick:
> 
> - - - - - - - - - - 8< - - - - - - - - - -
> 
> # xl list
> Name                                        ID   Mem VCPUs      State   Time(s)
> Domain-0                                     0  8191     8     r-----   42736.7
> alpine-containerlab                          8  2039     2     -b----      29.7
> 
> - - - - - - - - - - >8 - - - - - - - - - -
> 
> Thank you :-)

Oh, so we have a broken SeaBIOS build in 15.0.  I need to look into
that, it might get solved by updating SeaBIOS to the latest version.

Thanks, Roger.