[Bug 278535] bhyve: occasional long delays during boot

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=278535

            Bug ID: 278535
           Summary: bhyve: occasional long delays during boot
           Product: Base System
           Version: 15.0-CURRENT
          Hardware: Any
                OS: Any
            Status: New
          Severity: Affects Only Me
          Priority: ---
         Component: bhyve
          Assignee: virtualization@FreeBSD.org
          Reporter: lexi.freebsd@le-fay.org

when testing on bhyve, i've noticed that the kernel will sometimes hang during
boot.

console output:

?cLoading kernel...
/boot/kernel/kernel text=0x1852c8 text=0xec48a8 text=0x4852bc data=0x180+0xe80
data=0x199700+0x466900 0x8+0x19db08+0x8+0x1c80cb|
Loading configured modules...
/etc/hostid size=0x25
/boot/entropy size=0x1000

Hit [Enter] to boot immediately, or any other key for command prompt.
Booting [/boot/kernel/kernel]...
staging 0xb4000000 (not copying) tramp 0xbe278000 PT4 0xbe26f000
Start @ 0xffffffff80386000 ...
GDB: no debug ports present
KDB: debugger backends: ddb
KDB: current backend: ddb
---<<BOOT>>---
Copyright (c) 1992-2024 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
        The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 15.0-CURRENT #3 lf/bridge-inet-n269607-75357a4ae2c4: Mon Apr 22
20:24:11 BST 2024
   
lexi@ilythia.eden.le-fay.org:/src/obj/src/freebsd/src/bridge-inet/amd64.amd64/sys/GENERIC
amd64
FreeBSD clang version 18.1.4 (https://github.com/llvm/llvm-project.git
llvmorg-18.1.4-0-ge6c3289804a6)
WARNING: WITNESS option enabled, expect reduced performance.
VT: init without driver.
CPU: AMD Ryzen 7 5800X3D 8-Core Processor            (3541.95-MHz K8-class CPU)
  Origin="AuthenticAMD"  Id=0xa20f12  Family=0x19  Model=0x21  Stepping=2
 
Features=0x1783fbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,MMX,FXSR,SSE,SSE2,HTT>
 
Features2=0xfed83203<SSE3,PCLMULQDQ,SSSE3,FMA,CX16,SSE4.1,SSE4.2,MOVBE,POPCNT,AESNI,XSAVE,OSXSAVE,AVX,F16C,RDRAND,HV>
  AMD Features=0x26500800<SYSCALL,NX,MMX+,FFXSR,Page1GB,LM>
  AMD
Features2=0x444231fb<LAHF,CMP,ExtAPIC,CR8,ABM,SSE4A,MAS,Prefetch,SKINIT,WDT,TCE,Topology,DBE,ADMSKX>
  Structured Extended
Features=0x201403a9<FSGSBASE,BMI1,AVX2,SMEP,BMI2,ERMS,RDSEED,SMAP,SHA>
  Structured Extended Features2=0x600<VAES,VPCLMULQDQ>
  XSAVE Features=0x1<XSAVEOPT>
  AMD Extended Feature Extensions ID EBX=0x7<CLZERO,IRPerf,XSaveErPtr>
  TSC: P-state invariant
Hypervisor: Origin = "bhyve bhyve "
real memory  = 8589934592 (8192 MB)
avail memory = 8256217088 (7873 MB)
Event timer "LAPIC" quality 600
ACPI APIC Table: <BHYVE  BVAPIC >
FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs
FreeBSD/SMP: 4 package(s) x 1 core(s)
random: registering fast source Intel Secure Key RNG
random: fast provider: "Intel Secure Key RNG"
random: unblocking device.
ioapic0 <Version 1.1> irqs 0-31
Launching APs: 3 2 1
[hang here...]

the VM does eventually recover, after several minutes, and finishes booting. 
however when this happens, the boot is very slow -- in the case i just noticed,
it took about 15 minutes to boot to a multiuser login prompt.

this may be related to high CPU load on the host, since i normally use bhyve
when working on src, so i'm testing and building at the same time.  i have not
noticed this issue on production bhyve instances where the host is mostly idle.

host:  FreeBSD 15.0-CURRENT #29 lf/main-n269030-a8c2ee4fe736: Thu Apr 18
19:57:54 BST 2024    
srcmastr@daphne.eden.le-fay.org:/src/obj/src/freebsd/lf/main/amd64.amd64/sys/LF

guest: FreeBSD 15.0-CURRENT #3 lf/bridge-inet-n269607-75357a4ae2c4: Mon Apr 22
20:24:11 BST 2024    
lexi@ilythia.eden.le-fay.org:/src/obj/src/freebsd/src/bridge-inet/amd64.amd64/sys/GENERIC

-- 
You are receiving this mail because:
You are the assignee for the bug.