[Bug 290026] bhyve igb passthru - device unusable until reset

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=290026

            Bug ID: 290026
           Summary: bhyve igb passthru - device unusable until reset
           Product: Base System
           Version: 14.3-RELEASE
          Hardware: Any
                OS: Any
            Status: New
          Severity: Affects Only Me
          Priority: ---
         Component: bhyve
          Assignee: virtualization@FreeBSD.org
          Reporter: pprocacci@gmail.com

FreeBSD 14.3.

FreeBSD BYVE VM
------------------------------------
# ping google.com
ping: cannot resolve google.com: Name does not resolve

# ifconfig igb0 down; ifconfig igb0 up
# ping google.com
PING google.com (142.250.65.206): 56 data bytes
64 bytes from 142.250.65.206: icmp_seq=0 ttl=113 time=19.794 ms

igb0@pci0:0:6:0:        class=0x020000 rev=0x03 hdr=0x00 vendor=0x8086
device=0x1533 subvendor=0x1849 subdevice=0x1533
    vendor     = 'Intel Corporation'
    device     = 'I210 Gigabit Network Connection'
    class      = network
    subclass   = ethernet


FreeBSD Host
------------------------------------
ppt0@pci0:35:0:0:       class=0x020000 rev=0x03 hdr=0x00 vendor=0x8086
device=0x1533 subvendor=0x1849 subdevice=0x1533
    vendor     = 'Intel Corporation'
    device     = 'I210 Gigabit Network Connection'
    class      = network
    subclass   = ethernet


VM-BHYVE Config:
------------------------------------
loader="bhyveload"
cpu=4
memory=2G
passthru0="35/0/0"
disk0_type="nvme"
disk0_name="disk0"
disk0_dev="sparse-zvol"
network0_type="virtio-net"
network0_switch="public"


Pertinent VM Bootup Info:
-------------------------
Starting Network: lo0 vtnet0 igb0.
lo0: flags=1008049<UP,LOOPBACK,RUNNING,MULTICAST,LOWER_UP> metric 0 mtu 16384
        options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
        inet 127.0.0.1 netmask 0xff000000
        inet6 ::1 prefixlen 128
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3
        groups: lo
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
vtnet0: flags=1008802<BROADCAST,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
        options=80028<VLAN_MTU,JUMBO_MTU,LINKSTATE>
        ether 58:9c:fc:0f:1f:6c
        media: Ethernet autoselect (10Gbase-T <full-duplex>)
        status: active
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
igb0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500    
options=4e527bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6,HWSTATS,MEXTPG>
        ether d0:50:99:d4:b9:fd
        media: Ethernet autoselect
        status: no carrier
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
igb0: link state changed to UP
Starting devd.
Starting Network: vtnet0.
vtnet0: flags=1008802<BROADCAST,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
        options=80028<VLAN_MTU,JUMBO_MTU,LINKSTATE>
        ether 58:9c:fc:0f:1f:6c
        media: Ethernet autoselect (10Gbase-T <full-duplex>)
        status: active
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
Waiting 30s for the default route interface: .............................



Once multi user (Note: It's not seeing dhcp responses though the requests are
leaving)
--------------------
igb0: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0
mtu 1500     
options=4e527bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6,HWSTATS,MEXTPG>
        ether d0:50:99:d4:b9:fd
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>



After down/up (Note: DHCP is working just as it should)
--------------------
igb0: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0
mtu 1500       
options=4e527bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6,HWSTATS,MEXTPG>
        ether d0:50:99:d4:b9:fd
        inet 192.168.35.152 netmask 0xffffff00 broadcast 192.168.35.255
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>


It's like the igb device has only been partially initialized by bhyve as a
down/up fixes this.
Any thoughts here?

-- 
You are receiving this mail because:
You are the assignee for the bug.