Is the BHyVe guest as suitable for high-performance disk IO as the host?

Neel Natu neelnatu at
Mon May 11 04:30:45 UTC 2015

Hi Tinker,

On Sun, May 10, 2015 at 6:16 AM, Tinker <tinkr at> wrote:
> Hi Neel,
> Thank you very much for your response -
> That's great!
> I guess this should deliver the full capacity for basically any IO system
> around, be it a fast SSD or ramdisk. (Since the guest may not need to flush
> data immediately to the host, I guess read performance is the more sensitive
> point.)
> Which disk emulation mode is best for a BSD guest, "virtio-blk" or
> "ahci-hd"?

You'll need to experiment because there are so many moving pieces: the
guest driver implementation, VM exit overhead, capabilities of a
emulated disk controller. From the narrow perspective of VM-exit
overhead 'virtio-blk' is better because the CPU decodes all the
information needed to handle the I/O exit.

> In general, should any other consideration be made for performance (using
> the "direct" or "nocache" BHyVe configuration options, particular
> consideration for proper sector alignment when using a disk image stored on
> the host's SSD-based ZFS, mounting the host ZFS filesystem with "noatime")?

Using "nocache" will prevent double-buffering of the virtual disk file
in the host. Using "direct" will guarantee consistency of the virtual
disk file on the underlying storage device. If you are purely focused
on performance then don't use either option but be aware that you'll
be using buffer cache on the host and also that the guest filesystem
could be corrupted on an unplanned host shutdown.

With respect to alignment I am not sure if you can request a specific
alignment when creating a flat file for the virtual disk. However, if
you use a ZVOL as a backing device then it can be created on top of a
partition which has the correct alignment.


> Kind regards,
> Tinker
> On 2015-05-10 04:03, Neel Natu wrote:
>> Hi Tinker,
>> On Sat, May 9, 2015 at 9:49 AM, Tinker <tinkr at> wrote:
>>> Hi!
>>> For an environment with very heavy parallell IO, should the performance
>>> be
>>> just as good in a BHyVe guest as in the FreeBSD host environment?
>>> What I thought of is that I guess within the host environment, the
>>> storage
>>> subsystem should have all kinds of optimizations like an internal work
>>> queue
>>> that pushes lots of work alinearly/asynchronously to the disk controller
>>> and
>>> this way allows it, in turn, to give all its performance.
>>> Does the virtualized disk interface carry over all that goodness to the
>>> guest?
>> bhyve creates 8 worker threads for each virtual disk controller (both
>> ahci and virtio-blk).
>> All guest I/O is handled asynchronously by these worker threads which
>> provide parallelism.
>>> ( seems to say yes, presuming you configure
>>> BHyVe to run the virtual disk in AHCI mode?)
>> The wiki is out of date.
>> Since r280037 the virtio-blk emulation also gets the benefits of using
>> the block_if worker threads.
>> best
>> Neel
>>> Thanks!
>>> Tinker
>>> _______________________________________________
>>> freebsd-virtualization at mailing list
>>> To unsubscribe, send any mail to
>>> "freebsd-virtualization-unsubscribe at"

More information about the freebsd-virtualization mailing list