bhyve PCIe passthrough on AMD Epyc

[Kevin Day]

I'm trying to get PCIe device (video card) passthrough working to a Windows bhyve VM, should this work now?

I have the device blocked off in loader.conf, and added hw.vmm.amdvi.enable="1".  After doing so this is how it appears in pciconf:

ppt0 at pci0:97:0:0:	class=0x030000 card=0x27173842 chip=0x128b10de rev=0xa1 hdr=0x00
    bar   [10] = type Memory, range 32, base 0xe0000000, size 16777216, disabled
    bar   [14] = type Prefetchable Memory, range 64, base 0xfcf0000000, size 134217728, disabled
    bar   [1c] = type Prefetchable Memory, range 64, base 0xfcf8000000, size 33554432, disabled
    bar   [24] = type I/O Port, range 32, base 0x3000, size 128, disabled
    cap 01[60] = powerspec 3  supports D0 D3  current D0
    cap 05[68] = MSI supports 1 message, 64 bit
    cap 10[78] = PCI-Express 2 legacy endpoint max data 256(256) RO NS
                 link x8(x8) speed 5.0(5.0)
    ecap 0002[100] = VC 1 max VC0
    ecap 0004[128] = Power Budgeting 1
    ecap 000b[600] = Vendor 1 ID 1

When trying to attach it to a VM with "-s 9:0,passthru,97/0/0" I get this almost immediately after it starts up:

Assertion failed: (error == 0), function modify_bar_registration, file /usr/src/usr.sbin/bhyve/pci_emul.c, line 504.

Also on the console I'm seeing a lot of messages like:

ivhd0: Error: completion failed tail:0x1c0, head:0x0.
ivhd0: Dump all the commands:
  [CMD0, off:0xff0] opcode= 0x1 0x304f1629 0x30 0xa5a5
  [CMD1, off:0x0] opcode= 0x1 0x304f1629 0x30 0xa5a5
  [CMD2, off:0x10] opcode= 0x1 0x304f1629 0x30 0xa5a5
  [CMD3, off:0x20] opcode= 0x1 0x304f1629 0x30 0xa5a5
  [CMD4, off:0x30] opcode= 0x1 0x304f1629 0x30 0xa5a5
  [CMD5, off:0x40] opcode= 0x1 0x304f1629 0x30 0xa5a5
  [CMD6, off:0x50] opcode= 0x1 0x304f1629 0x30 0xa5a5
  [CMD7, off:0x60] opcode= 0x2 0x6101 0x0 0x0
  [CMD8, off:0x70] opcode= 0x1 0x304f1629 0x30 0xa5a5
  [CMD9, off:0x80] opcode= 0x1 0x304f1629 0x30 0xa5a5
  [CMD10, off:0x90] opcode= 0x1 0x304f1629 0x30 0xa5a5
  [CMD11, off:0xa0] opcode= 0x1 0x304f1629 0x30 0xa5a5
  [CMD12, off:0xb0] opcode= 0x1 0x304f1629 0x30 0xa5a5
  [CMD13, off:0xc0] opcode= 0x1 0x304f1629 0x30 0xa5a5
  [CMD14, off:0xd0] opcode= 0x1 0x304f1629 0x30 0xa5a5
  [CMD15, off:0xe0] opcode= 0x1 0x304f1629 0x30 0xa5a5
  [CMD16, off:0xf0] opcode= 0x1 0x304f1629 0x30 0xa5a5
  [CMD17, off:0x100] opcode= 0x1 0x304f1629 0x30 0xa5a5
  [CMD18, off:0x110] opcode= 0x2 0x6100 0x0 0x0
  [CMD19, off:0x120] opcode= 0x1 0x304f1629 0x30 0xa5a5
  [CMD20, off:0x130] opcode= 0x1 0x304f1629 0x30 0xa5a5
  [CMD21, off:0x140] opcode= 0x1 0x304f1629 0x30 0xa5a5
  [CMD22, off:0x150] opcode= 0x1 0x304f1629 0x30 0xa5a5
  [CMD23, off:0x160] opcode= 0x1 0x304f1629 0x30 0xa5a5
  [CMD24, off:0x170] opcode= 0x1 0x304f1629 0x30 0xa5a5
  [CMD25, off:0x180] opcode= 0x1 0x304f1629 0x30 0xa5a5
  [CMD26, off:0x190] opcode= 0x1 0x304f1629 0x30 0xa5a5
  [CMD27, off:0x1a0] opcode= 0x1 0x304f1629 0x30 0xa5a5
  [CMD28, off:0x1b0] opcode= 0x1 0x304f1629 0x30 0xa5a5