bhyve PCIe passthrough on AMD Epyc

[Kevin Day]

> On Feb 28, 2019, at 12:28 PM, Anish <akgupt3 at gmail.com> wrote:
> 
> Hi Kevin,
> >I'm just pointing out that it's an AMD Epyc because I know the AMD IOMMU/AMD-Vi code was added somewhat more recently than Intel's and is maybe less tested.
> I tested on Ryzen-I, not sure if anyone tried it on newer Ryzen/Epyc systems.
> 
> >CPU0: local APIC error 0x40
> This is concerning, as per AMD docs, bit6 is for illegal vector which indicate CPU is getting invalid interrupts.
> 
> Can you share hw.vmm sysctl output?
> 
> -Anish

Sure:

hw.vmm.amdvi.domain_id: 4
hw.vmm.amdvi.disable_io_fault: 0
hw.vmm.amdvi.ptp_level: 4
hw.vmm.amdvi.host_ptp: 1
hw.vmm.amdvi.enable: 1
hw.vmm.amdvi.count: 8
hw.vmm.npt.pmap_flags: 507
hw.vmm.svm.num_asids: 32768
hw.vmm.svm.disable_npf_assist: 0
hw.vmm.svm.features: 113919
hw.vmm.svm.vmcb_clean: 1023
hw.vmm.vmx.l1d_flush_sw: 0
hw.vmm.vmx.l1d_flush: 0
hw.vmm.vmx.vpid_alloc_failed: 0
hw.vmm.vmx.posted_interrupt_vector: -1
hw.vmm.vmx.cap.posted_interrupts: 0
hw.vmm.vmx.cap.virtual_interrupt_delivery: 0
hw.vmm.vmx.cap.invpcid: 0
hw.vmm.vmx.cap.monitor_trap: 0
hw.vmm.vmx.cap.unrestricted_guest: 0
hw.vmm.vmx.cap.pause_exit: 0
hw.vmm.vmx.cap.halt_exit: 0
hw.vmm.vmx.initialized: 0
hw.vmm.vmx.cr4_zeros_mask: 0
hw.vmm.vmx.cr4_ones_mask: 0
hw.vmm.vmx.cr0_zeros_mask: 0
hw.vmm.vmx.cr0_ones_mask: 0
hw.vmm.vmx.no_flush_rsb: 0
hw.vmm.ept.pmap_flags: 0
hw.vmm.vrtc.flag_broken_time: 1
hw.vmm.ppt.devices: 1
hw.vmm.iommu.enable: 1
hw.vmm.iommu.initialized: 1
hw.vmm.bhyve_xcpuids: 312
hw.vmm.topology.cpuid_leaf_b: 1
hw.vmm.create: beavis
hw.vmm.destroy: beavis
hw.vmm.trace_guest_exceptions: 0
hw.vmm.ipinum: 251
hw.vmm.halt_detection: 1