Several bhyve quirks
grehan at freebsd.org
Wed Mar 25 15:44:48 UTC 2015
I'll let Neel take care of the time questions.
> 3. Even moderate guest disk I/O completely kills guest network
> performance. For example, whenever security(8) (security(7) in FreeBSD)
> runs, guest network throughput drops from 150+ Mbps to ~20 Mbps, and
> jitter from ping jumps from <0.01 ms to 100+ ms. If I try to build
> something in the guest, then network becomes almost unusable.
> The network performance degradation only affects the guest that's
> generating the I/O; high I/O on guest B doesn't affect guest A, nor
> would high I/O on the host.
> I'm using both virtio-blk and virio-net drivers, and the guests' disk
> images are backed by zvol+geli. Removing geli has no effect.
> There are some commits in CURRENT that suggests improved virtio
> performance, but I'm not comfortable running CURRENT. Is there a
> workaround I could use for 10.1?
In 10.1, virtio-blk i/o is done sychronously in the context of the
guest vCPU exit. If it's a single vCPU guest, or the virtio-net
interrupt happens to be delivered to that vCPU, performance will suffer.
A workaround is to use ahci-hd for the disk emulation and not
virtio-blk. The AHCI emulation does i/o in a dedicated thread and
doesn't block the vCPU thread.
> 4. virtio-blk always reports the virtual disk as having 512-byte
> sectors, and so I get I/O errors on OpenBSD guests when the disk image
> is backed by zvol+geli with 4K sector size. Curiously, this only seems
> to affect zvol+geli; with just zvol it seems to work. Also, it works
> either way on Linux guests.
> ATM I changed the zvol / geli sector size to 512 bytes, which probably
> made #2 worse. I think this bug / feature is addressed by:
> but again is there a workaround to force a specific sector size for 10.1?
The only workaround for 10.1 would be to use ahci-hd instead of
virtio-blk. The correct sector size will be reported there.
> 5. This may be better directed at OpenBSD but I'll ask here anyway: if I
> enable virtio-rnd then OpenBSD would not boot with "couldn't map
> interrupt" error. The kernel in bsd.rd will boot, but not the installed
> kernel (or the one built from STABLE; I forgot). Again, Linux seems
> unaffected, but I couldn't tell if it's actually working.
Try using the -W option to bhyve. This will force the bhyve virtio
code to advertize (non-standard) MSI interrupt capability which OpenBSD
will then use to allocate vectors.
More information about the freebsd-virtualization