bhyve: boot custom kernel without workaround
- Go to: [ bottom of page ] [ top of archives ] [ this month ]
Date: Thu, 28 May 2026 03:57:39 UTC
Seeking guidance on what I might be doing wrong, and how to avoid the manual workaround (reload-conf) specified below: I've built a custom kernel based off 15.0-STABLE then installed it to a raw disk (kernfs.raw), which I intend to boot from in a bhyve container per: https://freebsdfoundation.org/wp-content/uploads/2021/11/Kernel_Development_Recipes.pdf Command: bhyve -c 1 -m 512M -H -A -P -G :1234 \ -s 0:0,hostbridge \ -s 1:0,lpc \ -s 2:0,virtio-blk,FreeBSD-15.0-STABLE-amd64.raw \ -s 3:0,virtio-blk,kernfs.raw \ -l com1,stdio \ -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd \ myvm However, the custom kernel image on the additional partition is not loaded by default in the bhyve container even though I've applied the rc.conf, sysctl.conf and fstab modifications as specified in the pdf. I can boot into the custom kernel stored on the additional virtio-blk device with the following workaround: Launch bhyve vm and modify /boot/loader.conf, point `kernel` at the additional partition with the custom kernel: kernel="disk1p1:/boot/kernel" Now restart vm and: - drop into boot loader - OK reload-conf - OK boot uname -a now shows the custom kernel’s label. Troubleshooting: I've dumped the relevant boot loader variables before and after `reload-conf` Before: OK show ... kernel=kernel kernel_options= kernels_autodetect=YES loaddev=disk0p4: ... After: OK reload-conf: OK show ... kernel=disk1p1:/boot/kernel <==== updated kernel_options= kernels_autodetect=YES loaddev=disk0p4: ... My impression from Mark Johnston's article is that the modified sysctl.conf and fstab files should've caused the vm to boot off of the custom kernel partition on the 2nd virtio-blk device (by ensuring it’s mounted at /boot/kernel/kernel). I’ve looked through the forums which helped identify the workaround, but hoping to avoid it every time I spin up the container. Thanks, Mark