bhyve(8) passthru affects ahci-hd with device backend [Was: Re: Unexpected ahci-hd bytes when running in bhyve(8)]

[Harry Schmalzbauer]

 Bezüglich Harry Schmalzbauer's Nachricht vom 29.10.2016 19:35 (localtime):
>  Bezüglich Harry Schmalzbauer's Nachricht vom 29.10.2016 17:32 (localtime):
>
> …
>> Like mentioned, while reading the first 448 bytes on the host, I get
>> identical results from /usr/local/guest.img and /dev/ada4, but when
>> attaching /dev/ada4 to ahci-hd (-s 7,ahci-hd,/dev/ada4) and inspecting
>> inside vmm, all I see is 0x0, while ahci-hd attached
>> /usr/local/guest.img shows the same pmbr as on the host!?
>>
>> Do I have to exclude /dev/ada4 on the host from geom? As soon as bhyve
>> opens /dev/ada4, all partitions vanish from the host – probably ada4
>> itself gets blocked somehow?
…
> Just another symptom I can only describe, not debug:
> Opening /dev/adaX on the host works by 'hd /dev/ada4 | less',
> but not inside the guest, where it just leads to endless IO when trying
> the same on the ahci-hd attached /dev/ada4

The described defects only happen when passthru is used with bhyve(8)!!!

If I simply don't attach the passthru-device (keeping memory wired),
everything works the way it's supposed to do.
Opening the guest-ada1-device with hexdump works, geom tastes GPT and
the first 448 bytes show exactly the pMBR like on the host.
As soon as I start the guest with the passthru device (doesn't matter
which slot, tested with a 82574L), the ahci-hd can't be read nymore,
just returning 0x0 when dumped.

Shall I file a bug report? Anybody aware of that or any idea where to
start fixing?

To summarize:
FreeBSD-11-RELEASE hosting a bhyve(8) guest with a physical device as
storage backend (regardless if accessed through virtio-blk or ahci-hd)
corrupts guest-disk access if there's also a passthrough device attached.
If you use a file-backed ahci-hd (or virtio-blk) device, the problem
doesn't show up, regardelss if there's passthru involved or not.


Thanks,

-Harry