bhyve PCIe passthrough on AMD Epyc

[Rodney W. Grimes]

> I'm trying to get PCIe device (video card) passthrough working to a Windows bhyve VM, should this work now?

No, this should not work now.

> I have the device blocked off in loader.conf, and added hw.vmm.amdvi.enable="1".  After doing so this is how it appears in pciconf:
> 
> ppt0 at pci0:97:0:0:	class=0x030000 card=0x27173842 chip=0x128b10de rev=0xa1 hdr=0x00
>     bar   [10] = type Memory, range 32, base 0xe0000000, size 16777216, disabled
>     bar   [14] = type Prefetchable Memory, range 64, base 0xfcf0000000, size 134217728, disabled
>     bar   [1c] = type Prefetchable Memory, range 64, base 0xfcf8000000, size 33554432, disabled
>     bar   [24] = type I/O Port, range 32, base 0x3000, size 128, disabled
>     cap 01[60] = powerspec 3  supports D0 D3  current D0
>     cap 05[68] = MSI supports 1 message, 64 bit
>     cap 10[78] = PCI-Express 2 legacy endpoint max data 256(256) RO NS
>                  link x8(x8) speed 5.0(5.0)
>     ecap 0002[100] = VC 1 max VC0
>     ecap 0004[128] = Power Budgeting 1
>     ecap 000b[600] = Vendor 1 ID 1
> 
> When trying to attach it to a VM with "-s 9:0,passthru,97/0/0" I get this almost immediately after it starts up:
> 
> Assertion failed: (error == 0), function modify_bar_registration, file /usr/src/usr.sbin/bhyve/pci_emul.c, line 504.
> 
> Also on the console I'm seeing a lot of messages like:
> 
> ivhd0: Error: completion failed tail:0x1c0, head:0x0.
> ivhd0: Dump all the commands:
>   [CMD0, off:0xff0] opcode= 0x1 0x304f1629 0x30 0xa5a5
>   [CMD1, off:0x0] opcode= 0x1 0x304f1629 0x30 0xa5a5
>   [CMD2, off:0x10] opcode= 0x1 0x304f1629 0x30 0xa5a5
>   [CMD3, off:0x20] opcode= 0x1 0x304f1629 0x30 0xa5a5
>   [CMD4, off:0x30] opcode= 0x1 0x304f1629 0x30 0xa5a5
>   [CMD5, off:0x40] opcode= 0x1 0x304f1629 0x30 0xa5a5
>   [CMD6, off:0x50] opcode= 0x1 0x304f1629 0x30 0xa5a5
>   [CMD7, off:0x60] opcode= 0x2 0x6101 0x0 0x0
>   [CMD8, off:0x70] opcode= 0x1 0x304f1629 0x30 0xa5a5
>   [CMD9, off:0x80] opcode= 0x1 0x304f1629 0x30 0xa5a5
>   [CMD10, off:0x90] opcode= 0x1 0x304f1629 0x30 0xa5a5
>   [CMD11, off:0xa0] opcode= 0x1 0x304f1629 0x30 0xa5a5
>   [CMD12, off:0xb0] opcode= 0x1 0x304f1629 0x30 0xa5a5
>   [CMD13, off:0xc0] opcode= 0x1 0x304f1629 0x30 0xa5a5
>   [CMD14, off:0xd0] opcode= 0x1 0x304f1629 0x30 0xa5a5
>   [CMD15, off:0xe0] opcode= 0x1 0x304f1629 0x30 0xa5a5
>   [CMD16, off:0xf0] opcode= 0x1 0x304f1629 0x30 0xa5a5
>   [CMD17, off:0x100] opcode= 0x1 0x304f1629 0x30 0xa5a5
>   [CMD18, off:0x110] opcode= 0x2 0x6100 0x0 0x0
>   [CMD19, off:0x120] opcode= 0x1 0x304f1629 0x30 0xa5a5
>   [CMD20, off:0x130] opcode= 0x1 0x304f1629 0x30 0xa5a5
>   [CMD21, off:0x140] opcode= 0x1 0x304f1629 0x30 0xa5a5
>   [CMD22, off:0x150] opcode= 0x1 0x304f1629 0x30 0xa5a5
>   [CMD23, off:0x160] opcode= 0x1 0x304f1629 0x30 0xa5a5
>   [CMD24, off:0x170] opcode= 0x1 0x304f1629 0x30 0xa5a5
>   [CMD25, off:0x180] opcode= 0x1 0x304f1629 0x30 0xa5a5
>   [CMD26, off:0x190] opcode= 0x1 0x304f1629 0x30 0xa5a5
>   [CMD27, off:0x1a0] opcode= 0x1 0x304f1629 0x30 0xa5a5
>   [CMD28, off:0x1b0] opcode= 0x1 0x304f1629 0x30 0xa5a5
> 
> 
> _______________________________________________
> freebsd-virtualization at freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
> To unsubscribe, send any mail to "freebsd-virtualization-unsubscribe at freebsd.org"
> 

-- 
Rod Grimes                                                 rgrimes at freebsd.org