bhyve - map user-space buffer in the VM

[Neel Natu]

Hi Stefano,

Sorry for the late reply.

On Tue, Jun 30, 2015 at 2:57 AM, Stefano Garzarella
<stefanogarzarella at gmail.com> wrote:
> Hi all,
> I'm working on bhyve to support ptnetmap (netmap passthrough) for my GSoC.
> http://wiki.freebsd.org/SummerOfCode2015/ptnetmapOnBhyve
>
> I've already implemented it on QEMU/KVM and, to expose the netmap memory in
> the VM, I used a PCI BAR and I mapped the netmap memory (the buffer returned
> by mmap() on netmap fd in the user space) on the BAR in the VM through KVM
> API.
>
> In bhyve I able to create a PCI BAR but I only found vm_map_pptdev_mmio()
> API to map physical host page into the VM. I saw that this IOCTL is
> implemented in the kernel module with sglist_append_phys() to populate a
> scatter/gather list, than it is mapped in VM.
>
> I would like to add a new function in the API to map a user buffer in the
> VM:
> int vm_map_user_buf(struct vmctx *ctx, vm_paddr_t gpa, size_t len, void
> *host_buf)
>
> I would implement it adding a new IOCTL and I would create, in the kernel
> module, a new function like vmm_mmio_alloc() (sys/amd64/vmm/vmm_mem.c) but
> using sglist_append_user() instead of sglist_append_phys().
>
> What do you think about it?
> There is another way to map an user-space buffer in the VM?
>

bhyve represents the guest address space as a vmspace that is
independent from the bhyve process's vmspace.

If a mapping is shared between the two address spaces then both
mappings should point to the same vm_object. Also, the virtual machine
lifetime can span the bhyve process lifetime. Therefore a shared
mapping that points to the bhyve process's heap or data segment needs
special handling.

So, the general case of mapping an arbitrary bhyve user-space mapping
into the guest address space is not trivial.

Regarding 'vmm_mmio_alloc()' - this API maps PCI passthru device BARs
into the guest address space.

Note that it deals with both issues pointed above:
1. vmm.ko owns the passthru device so it can guarantee that the BAR
addresses cannot change underneath the guest.

2. vm_cleanup() guarantees that the passthru mappings are removed from
the guest address space before the next bhyve process starts running.

The 'vmm_mmio_alloc()/vmm_mmio_free()' APIs are very narrow so should
not be used a model for the general case.

Having said that I think that for the specific case (netmap) your
proposal might work assuming wired mappings of the netmap buffers. If
the underlying netmap buffers are not physically contiguous then there
will be a bit more work to do.

I hope that allows you to make progress with the GSoC while we figure
out a long-term viable solution.

best
Neel

> Thanks,
> Stefano
>