[Bug 222996] FreeBSD 11.1-12 on Hyper-V with PCI Express Pass Through

[bugzilla-noreply at freebsd.org]

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=222996

--- Comment #23 from Michael <michael.adm at gmail.com> ---
Passthrough Intel X710 in Windows 2019 Gen2 Hyper-V guest Freebsd-CURRENT
works, but not as we would like.

Action roadmap:
1. disable the necessary adapter in the Device Manager
2. Add adapter to R03 VM
$MyEth = Get-PnpDevice  -PresentOnly | Where-Object {$_.Class -eq "Net"} |
Where-Object {$_.Status -eq "Error"}
$DataOfNetToDDismount = Get-PnpDeviceProperty DEVPKEY_Device_LocationPaths
-InstanceId $MyEth[0].InstanceId
$locationpath = ($DataOfNetToDDismount).data[0]
Dismount-VmHostAssignableDevice -locationpath $locationpath -force
Get-VMHostAssignableDevice
Add-VMAssignableDevice -LocationPath $locationpath -VMName R03

R03 dmesg:
pcib0: <Hyper-V PCI Express Pass Through> on vmbus0
pci0: <PCI bus> on pcib0
ixl0: <Intel(R) Ethernet Controller X710 for 10GbE SFP+ - 2.1.0-k> at device
0.0 on pci0
ixl0: fw 5.40.47690 api 1.5 nvm 5.40 etid 80002d35 oem 1.264.0
ixl0: PF-ID[1]: VFs 64, MSI-X 129, VF MSI-X 5, QPs 768, I2C
ixl0: Using 1024 TX descriptors and 1024 RX descriptors
ixl0: queue equality override not set, capping rx_queues at 4 and tx_queues at
4
ixl0: Using 4 RX queues 4 TX queues
ixl0: Using MSI-X interrupts with 5 vectors
ixl0: Ethernet address: 3c:fd:fe:21:02:e2
ixl0: Allocating 4 queues for PF LAN VSI; 4 queues active
ixl0: PCI Express Bus: Speed 8.0GT/s Width x8
ixl0: Failed to initialize SR-IOV (error=2)
ixl0: netmap queues/slots: TX 4/1024, RX 4/1024
ixl0: Link is up, 10 Gbps Full Duplex, Requested FEC: None, Negotiated FEC:
None, Autoneg: False, Flow Control: Full
ixl0: link state changed to UP

3. Add vlan to ixl0
root at r03:~ # ifconfig vlan48 create
root at r03:~ # ifconfig vlan48 vlan 48 vlandev ixl0
root at r03:~ # ifconfig vlan48 inet 10.10.221.51/24
root at r03:~ # ping 10.10.221.1
PING 10.10.221.1 (10.10.221.1): 56 data bytes
ping: sendto: Network is down
ping: sendto: Network is down
^C
--- 10.10.221.1 ping statistics ---
2 packets transmitted, 0 packets received, 100.0% packet loss
root at r03:~ # ifconfig ixl0 up
root at r03:~ # ping 10.10.221.1
PING 10.10.221.1 (10.10.221.1): 56 data bytes
64 bytes from 10.10.221.1: icmp_seq=0 ttl=64 time=0.965 ms
64 bytes from 10.10.221.1: icmp_seq=1 ttl=64 time=0.381 ms
64 bytes from 10.10.221.1: icmp_seq=2 ttl=64 time=0.526 ms
^C
--- 10.10.221.1 ping statistics ---
3 packets transmitted, 3 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 0.381/0.624/0.965/0.248 ms

4. Remove X710 adapter from R03 VM for "neutered" live migration
$locationpath = (Get-VmHostAssignableDevice).LocationPath
Remove-VMAssignableDevice -LocationPath $locationpath -VMName R03
Mount-VmHostAssignableDevice -locationpath $locationpath

R03 dmesg:
ixl0: Vlan in use, detach first
ixl0: Vlan in use, detach first
pcib0: detached
ixl0: Vlan in use, detach first

5. Adding again the adapter to the migrated R03 VM (#1 and #2)
root at r03:~ # ifconfig
...
ixl0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
       
options=e507bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6>
        ether 3c:fd:fe:21:02:e2
        media: Ethernet autoselect (10GBase-AOC <full-duplex,rxpause,txpause>)
        status: active
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
vlan48: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=600703<RXCSUM,TXCSUM,TSO4,TSO6,LRO,RXCSUM_IPV6,TXCSUM_IPV6>
        ether 3c:fd:fe:21:02:e2
        inet 10.10.221.51 netmask 0xffffff00 broadcast 10.10.221.255
        groups: vlan
        vlan: 48 vlanpcp: 0 parent interface: ixl0
        media: Ethernet autoselect (10GBase-AOC <full-duplex,rxpause,txpause>)
        status: active
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
ixl1: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500
       
options=e507bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6>
        ether 3c:fd:fe:21:02:e2
        media: Ethernet autoselect (10GBase-AOC <full-duplex,rxpause,txpause>)
        status: active
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
root at r03:~ # ping 10.10.221.1
PING 10.10.221.1 (10.10.221.1): 56 data bytes
^C
--- 10.10.221.1 ping statistics ---
5 packets transmitted, 0 packets received, 100.0% packet loss

How to make the adapter return to its place, rather than "double" in ixl1. Live
migration is one of the cornerstones of virtualization.

And what threatens, for normal functioning, an error message:
ixl0: Failed to initialize SR-IOV (error=2)

-- 
You are receiving this mail because:
You are the assignee for the bug.