After 12.1->12.2 upgrade FreeBSD bhyve guest does not start anymore

Wed Jan 13 12:45:45 UTC 2021

Hello.

Due to a 12.x specific bug, I run a 11.3 guest using vm-bhyve.
This is its config:
> loader="bhyveload"
> cpu=1
> memory=512M
> network0_type="virtio-net"
> network0_switch="public"
> disk0_type="virtio-blk"
> disk0_name="disk0"
> disk0_dev="sparse-zvol"
> uuid="..."
> network0_mac="..."

On monday I upgraded the host from 12.1 to 12.2 and, since then, it 
won't boot automatically, but will stop at the loader prompt.

Dmesg:
> /boot/kernel/kernel text=0x1564b08 data=0x145330+0x4cdf30 syms=[0x8+0x16daf0+0x8+0x186a43]
> /boot/entropy size=0x1000
> Booting...
> Copyright (c) 1992-2019 The FreeBSD Project.
> Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
>         The Regents of the University of California. All rights reserved.
> FreeBSD is a registered trademark of The FreeBSD Foundation.
> FreeBSD 11.3-RELEASE #0 r349754: Fri Jul  5 04:45:24 UTC 2019
>     root at releng2.nyi.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64
> FreeBSD clang version 8.0.0 (tags/RELEASE_800/final 356365) (based on LLVM 8.0.0)
> VT: init without driver.
> CPU: Intel(R) Core(TM) i5-7400 CPU @ 3.00GHz (3000.12-MHz K8-class CPU)
>   Origin="GenuineIntel"  Id=0x906e9  Family=0x6  Model=0x9e  Stepping=9
>   Features=0x9f83fbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,MMX,FXSR,SSE,SSE2,SS,HTT,PBE>
>   Features2=0xfeda7a17<SSE3,PCLMULQDQ,DTES64,DS_CPL,SSSE3,SDBG,FMA,CX16,xTPR,PCID,SSE4.1,SSE4.2,MOVBE,POPCNT,AESNI,XSAVE,OSXSAVE,AVX,F16C,RDRAND,HV>
>   AMD Features=0x24100800<SYSCALL,NX,Page1GB,LM>
>   AMD Features2=0x121<LAHF,ABM,Prefetch>
>   Structured Extended Features=0x40729<FSGSBASE,BMI1,AVX2,BMI2,ERMS,INVPCID,RDSEED>
>   Structured Extended Features3=0x400<MD_CLEAR>
>   XSAVE Features=0x1<XSAVEOPT>
>   TSC: P-state invariant
> Hypervisor: Origin = "bhyve bhyve "
> real memory  = 536870912 (512 MB)
> avail memory = 474779648 (452 MB)
> Event timer "LAPIC" quality 600
> ACPI APIC Table: <BHYVE  BVMADT  >
> ioapic0 <Version 1.1> irqs 0-31 on motherboard
> Timecounter "TSC-low" frequency 1500060031 Hz quality 1000
> random: entropy device external interface
> kbd1 at kbdmux0
> 000.000029 [4213] netmap_init               netmap: loaded module
> module_register_init: MOD_LOAD (vesa, 0xffffffff81005eb0, 0) error 19
> random: registering fast source Intel Secure Key RNG
> random: fast provider: "Intel Secure Key RNG"
> nexus0
> cryptosoft0: <software crypto> on motherboard
> acpi0: <BHYVE BVXSDT> on motherboard
> acpi0: Power Button (fixed)
> atrtc0: <AT realtime clock> port 0x70-0x71 irq 8 on acpi0
> atrtc0: registered as a time-of-day clock, resolution 1.000000s
> Event timer "RTC" frequency 32768 Hz quality 0
> attimer0: <AT timer> port 0x40-0x43 irq 0 on acpi0
> Timecounter "i8254" frequency 1193182 Hz quality 0
> Event timer "i8254" frequency 1193182 Hz quality 100
> hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff on acpi0
> Timecounter "HPET" frequency 16777216 Hz quality 950
> Event timer "HPET" frequency 16777216 Hz quality 550
> Event timer "HPET1" frequency 16777216 Hz quality 450
> Event timer "HPET2" frequency 16777216 Hz quality 450
> Event timer "HPET3" frequency 16777216 Hz quality 450
> Event timer "HPET4" frequency 16777216 Hz quality 450
> Event timer "HPET5" frequency 16777216 Hz quality 450
> Event timer "HPET6" frequency 16777216 Hz quality 450
> Event timer "HPET7" frequency 16777216 Hz quality 450
> Timecounter "ACPI-fast" frequency 3579545 Hz quality 900
> acpi_timer0: <32-bit timer at 3.579545MHz> port 0x408-0x40b on acpi0
> pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
> pcib0: could not evaluate _ADR - AE_NOT_FOUND
> pci0: <ACPI PCI bus> on pcib0
> virtio_pci0: <VirtIO PCI Block adapter> port 0x2000-0x207f mem 0xc0000000-0xc0001fff irq 16 at device 4.0 on pci0
> vtblk0: <VirtIO Block Adapter> on virtio_pci0
> vtblk0: 10240MB (20971520 512 byte sectors)
> virtio_pci1: <VirtIO PCI Network adapter> port 0x2080-0x209f mem 0xc0002000-0xc0003fff irq 17 at device 5.0 on pci0
> vtnet0: <VirtIO Networking Adapter> on virtio_pci1
> vtnet0: Ethernet address: 58:9c:fc:09:e1:26
> vtnet0: netmap queues/slots: TX 1/1024, RX 1/512
> 000.000566 [ 503] vtnet_netmap_attach       vtnet attached txq=1, txd=1024 rxq=1, rxd=512
> isab0: <PCI-ISA bridge> at device 31.0 on pci0
> isa0: <ISA bus> on isab0
> atkbdc0: <Keyboard controller (i8042)> port 0x60,0x64 irq 1 on acpi0
> atkbd0: <AT Keyboard> irq 1 on atkbdc0
> kbd0 at atkbd0
> atkbd0: [GIANT-LOCKED]
> driver bug: Unable to set devclass (class: atkbdc devname: (unknown))
> psm0: <PS/2 Mouse> irq 12 on atkbdc0
> psm0: [GIANT-LOCKED]
> psm0: model Generic PS/2 mouse, device ID 0
> uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0
> uart0: console (9600,n,8,1)
> uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0
> vga0: <Generic ISA VGA> at port 0x3b0-0x3bb iomem 0xb0000-0xb7fff on isa0
> ppc0: cannot reserve I/O port range
> Timecounters tick every 10.000 msec
> usb_needs_explore_all: no devclass
> 
> Loader variables:
> 
> Manual root filesystem specification:
>   <fstype>:<device> [options]
>       Mount <device> using filesystem <fstype>
>       and with the specified (optional) option list.
> 
>     eg. ufs:/dev/da0s1a
>         zfs:tank
>         cd9660:/dev/cd0 ro
>           (which is equivalent to: mount -t cd9660 -o ro /dev/cd0 /)
> 
>   ?               List valid disk boot devices
>   .               Yield 1 second (for background tasks)
>   <empty line>    Abort manual input
> 
> mountroot>

Then, if I just type "ufs:/dev/vtbd0a" it will start correctly.

Can anyone explain why this happens and how to avoid it?

  bye & Thanks
	av.