Bhyve storage improvements (was: Several bhyve quirks)

Alexander Motin mav at FreeBSD.org
Mon Apr 6 20:39:02 UTC 2015


Hi, Julian.

> I had some time to try it out today, but I'm still having issues:

I've just made experiment alike to your with making bhyve to work on top
of GEOM device instead of preferable "dev" mode of ZVOL. And I indeed
reproduced the problem. But the problem that I see is not related to the
block size. The block size is reported to the guest correctly as 4K, and
as I can see it works as such at least in FreeBSD guest.

The problem is in the way how bhyve inter-operates with block/GEOM
devices. bhyve sends requests to the kernel with preadv()/pwritev()
calls, specifying scatter/gather lists of buffer addresses provided by
the guest. But GEOM code can not handle scatter/gather lists, only
sequential buffer, and so single request is split into several. The
problem is that splitting happens according to scatter/gather elements,
and those elements in general case may not be multiple to the block
size, that is fatal for GEOM and any block device.

I am not yet sure how to fix this problem. The most straightforward way
is to copy the data at some point to collect elements of scatter/gather
list into something sequential to pass to GEOM, but that requires
additional memory allocation, and the copying is not free.  May be some
cases could be optimized to work without copying but with some clever
page mapping, but that seems absolutely not trivial.

-- 
Alexander Motin


More information about the freebsd-virtualization mailing list