bhyve + USB disks as slots + virtio-blk driver + passt-thru will not allow any OS to boot

From: Mario Marietto <>
Date: Sun, 13 Mar 2022 17:02:06 UTC

What I want to achieve is to pass thru two of my NTFS "formatted" disks to
a Windows 11 VM,but without passing them thru using the USB controller in
FreeBSD with a bhyve virtual machine (in the example below I tried to boot
Windows 11 from the nvme disk nvd0) AT at the same time I want to pass thru
my graphic card to a Windows 11 and / or Linux VM.

I'm using this FreeBSD version :

*FreeBSD marietto 13.0-RELEASE FreeBSD 13.0-RELEASE #5
n244809-dff3dead3734: Wed Feb 23 13:16:32 CET
2022     marietto@marietto:/usr/obj/usr/src/amd64.amd64/sys/GENERIC  amd64*

I've configured the bhyve VM like this :

*bhyve -S -c sockets=1,cores=2,threads=2 -m 4G -w -H -A \
-s 0,hostbridge \
-s 1,ahci-hd,/dev/nvd0 \
-s 2,virtio-blk,/dev/da4 \
-s 3,virtio-blk,/dev/da2 \
-s 4:0,passthru,2/0/0 \
-s 4:1,passthru,2/0/1 \
-s 4:2,passthru,2/0/2 \
-s 4:3,passthru,2/0/3 \
-s 8,virtio-net,tap2 \
-s 9,virtio-9p,sharename=/ \
-s 10,hda,play=/dev/dsp,rec=/dev/dsp \
-s 29,fbuf,tcp= <>,w=1440,h=900 \
-s 30,xhci,tablet \
-s 31,lpc \
-l bootrom,/usr/local/share/uefi-firmware/BHYVE_BHF_CODE.fd \
vm2 < /dev/null & sleep 2 && vncviewer 0:2*

These are the NTFS disks that I would like to see inside the Windows 11
guest os :

-s 2,virtio-blk,/dev/da4 \
-s 3,virtio-blk,/dev/da2 \
=>         34  19532873661  da4  GPT  (9.1T)
           34        32734    1  ms-reserved  (16M)
32768  19532838912    2  ms-basic-data  (9.1T)
19532871680         2015       - free -  (1.0M)

=>         34  23437705149  da2  GPT  (11T)
           34         2014       - free -  (1.0M)
2048  23437701120    1  ms-basic-data  (11T)
23437703168         2015       - free -  (1.0M)*

Using the whole disks in bhyve they are recognized by fdisk. But as I
said,what matters a lot for me,is to boot the nvme physical disk passing
through at the same time my graphic card but it does not work. I care so
much about this feature,because Ubuntu boots much much faster if I use the
physical disk instead of the img / raw file. And this is again more valid
if I want to boot Windows 11. It gets frozen e if I use a raw / img disk. I
can't use it. But if I use the nvme disk where I have installed it
physically it is much,much faster and I can use it.

This is what happens : (linux and windows 11 installed on the disk nvme
don't boot)

I will keep the discussion updated also here :