sr-iov issues, reset_hw() failed with error -100
Ultima
ultima1252 at gmail.com
Mon Feb 22 21:47:59 UTC 2016
Yeah, dmesg does show 48. 12 Cores and 24 threads each. 48 total
On Mon, Feb 22, 2016 at 4:35 PM, Steven Hartland <killing at multiplay.co.uk>
wrote:
> isn't that 48 cores (12 real 12 virtual) per CPU?
>
>
> On 22/02/2016 21:26, Ultima wrote:
>
>> This system has 24 cores (e5-2670v3)x2
>>
>> Ultima
>>
>> On Mon, Feb 22, 2016 at 3:53 PM, Pieper, Jeffrey E <
>> jeffrey.e.pieper at intel.com> wrote:
>>
>> Just out of curiosity, how many cores does your system have?
>>>
>>> Jeff
>>>
>>> -----Original Message-----
>>> From: owner-freebsd-current at freebsd.org [mailto:
>>> owner-freebsd-current at freebsd.org] On Behalf Of Ultima
>>> Sent: Monday, February 22, 2016 12:02 PM
>>> To: Eric Joyner <erj at freebsd.org>
>>> Cc: freebsd-current at freebsd.org; freebsd-virtualization at freebsd.org
>>> Subject: Re: sr-iov issues, reset_hw() failed with error -100
>>>
>>> After reboot...
>>>
>>> ifconfig ix1 up
>>>
>>> dhclient ix1
>>> DHCPDISCOVER on ix1 to 255.255.255.255 port 67 interval 4
>>> DHCPOFFER from 192.168.1.1
>>> DHCPREQUEST on ix1 to 255.255.255.255 port 67
>>> DHCPACK from 192.168.1.1
>>> bound to 192.168.1.145 -- renewal in 21600 seconds.
>>>
>>> ix0 down
>>> ping 192.168.1.1
>>> PING 192.168.1.1 (192.168.1.1): 56 data bytes
>>> 64 bytes from 192.168.1.1: icmp_seq=0 ttl=64 time=0.149 ms
>>> 64 bytes from 192.168.1.1: icmp_seq=1 ttl=64 time=0.171 ms
>>> 64 bytes from 192.168.1.1: icmp_seq=2 ttl=64 time=0.167 ms
>>>
>>> iovctl -Cf /etc/iovctl.conf
>>>
>>> ping 192.168.1.1
>>> PING 192.168.1.1 (192.168.1.1): 56 data bytes
>>> ^C
>>> --- 192.168.1.1 ping statistics ---
>>> 29 packets transmitted, 0 packets received, 100.0% packet loss
>>> ifconfig ix1 up
>>> ping 192.168.1.1
>>> PING 192.168.1.1 (192.168.1.1): 56 data bytes
>>> ^C
>>> --- 192.168.1.1 ping statistics ---
>>> 12 packets transmitted, 0 packets received, 100.0% packet loss
>>>
>>> ix1 is no longer usable until a restart...
>>>
>>> iovctl -Dd ix1
>>> ifconfig ix1 up
>>> ping 192.168.1.1
>>> PING 192.168.1.1 (192.168.1.1): 56 data bytes
>>> ^C
>>> --- 192.168.1.1 ping statistics ---
>>> 9 packets transmitted, 0 packets received, 100.0% packet loss
>>>
>>>
>>>
>>> Is there anything else that maybe useful?
>>>
>>> here is my ifconfig at the end (after ifconfig ix0 up)
>>>
>>>
>>> ix0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0
>>> mtu 1500
>>>
>>>
>>> options=e400b9<RXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6>
>>> ether -Hidden-
>>> inet 192.168.1.8 netmask 0xffffff00 broadcast 192.168.1.255
>>> inet 192.168.1.9 netmask 0xffffff00 broadcast 192.168.1.255
>>> nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
>>> media: Ethernet autoselect (10Gbase-T <full-duplex,rxpause,txpause>)
>>> status: active
>>> ix1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
>>>
>>>
>>> options=e407bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6>
>>> ether -Hidden-
>>> inet 192.168.1.145 netmask 0xffffff00 broadcast 192.168.1.255
>>> nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
>>> media: Ethernet autoselect (10Gbase-T <full-duplex,rxpause,txpause>)
>>> status: active
>>> lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
>>> options=600003<RXCSUM,TXCSUM,RXCSUM_IPV6,TXCSUM_IPV6>
>>> inet6 ::1 prefixlen 128
>>> inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3
>>> inet 127.0.0.1 netmask 0xff000000
>>> nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
>>> groups: lo
>>> bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu
>>> 1500
>>> ether -Hidden-
>>> nd6 options=9<PERFORMNUD,IFDISABLED>
>>> groups: bridge
>>> id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
>>> maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
>>> root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
>>> member: ix0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
>>> ifmaxaddr 0 port 1 priority 128 path cost 2000
>>> member: epair0a flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
>>> ifmaxaddr 0 port 5 priority 128 path cost 2000
>>> epair0a: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST>
>>> metric
>>> 0 mtu 1500
>>> options=8<VLAN_MTU>
>>> ether -Hidden-
>>> inet6 fe80::ff:70ff:fe00:50a%epair0a prefixlen 64 scopeid 0x5
>>> nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
>>> media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>)
>>> status: active
>>> groups: epair
>>>
>>> On Mon, Feb 22, 2016 at 1:51 PM, Eric Joyner <erj at freebsd.org> wrote:
>>>
>>> Did you do an ifconfig up on ix1 before loading the VF driver?
>>>>
>>>> On Sat, Feb 20, 2016 at 11:57 AM Ultima <ultima1252 at gmail.com> wrote:
>>>>
>>>> Decided to do some testing with iovctl to see how sr-iov is coming
>>>>>
>>>> along.
>>>
>>>> Currently when adding the vf's there are a couple errors, and the
>>>>>
>>>> network
>>>
>>>> no longer function after iovctl is started. My guess is the reset_hw()
>>>>> call
>>>>> that is failing. Any ideas why this call would fail? I tested this on
>>>>>
>>>> both
>>>
>>>> ports, ix1 is detached and unused for this test, however inserting a
>>>>>
>>>> cable
>>>
>>>> results in an unusable port. iovctl -Dd ix1 removes the vf's, however
>>>>> functionality is still not restored without a system restart.
>>>>>
>>>>> FreeBSD S1 11.0-CURRENT FreeBSD 11.0-CURRENT #0 r295736: Wed Feb 17
>>>>> 21:17:28 EST 2016 root at S1:/usr/obj/usr/src/sys/MYKERNEL amd64
>>>>>
>>>>> /boot/loader.conf
>>>>> hw.ix.num_queues="4"
>>>>>
>>>>> /etc/iovctl.conf
>>>>> PF {
>>>>> device : ix1;
>>>>> num_vfs : 31;
>>>>> }
>>>>>
>>>>> DEFAULT {
>>>>> passthrough : true;
>>>>> }
>>>>> VF-0 {
>>>>> passthrough : false;
>>>>> }
>>>>> VF-1 {
>>>>> passthrough : false;
>>>>> }
>>>>>
>>>>> # iovctl -C -f /etc/iovctl.conf
>>>>>
>>>>> dmesg
>>>>> ixv0: <Intel(R) PRO/10GbE Virtual Function Network Driver, Version -
>>>>> 1.4.6-k> at device 0.129 on pci12
>>>>> ixv0: Using MSIX interrupts with 2 vectors
>>>>> ixv0: ixgbe_reset_hw() failed with error -100
>>>>> device_attach: ixv0 attach returned 5
>>>>> ixv0: <Intel(R) PRO/10GbE Virtual Function Network Driver, Version -
>>>>> 1.4.6-k> at device 0.131 on pci12
>>>>> ixv0: Using MSIX interrupts with 2 vectors
>>>>> ixv0: ixgbe_reset_hw() failed with error -100
>>>>> device_attach: ixv0 attach returned 5
>>>>> pci12: <network, ethernet> at device 0.133 (no driver attached)
>>>>> pci12: <network, ethernet> at device 0.135 (no driver attached)
>>>>> pci12: <network, ethernet> at device 0.137 (no driver attached)
>>>>> pci12: <network, ethernet> at device 0.139 (no driver attached)
>>>>> pci12: <network, ethernet> at device 0.141 (no driver attached)
>>>>> pci12: <network, ethernet> at device 0.143 (no driver attached)
>>>>> pci12: <network, ethernet> at device 0.145 (no driver attached)
>>>>> pci12: <network, ethernet> at device 0.147 (no driver attached)
>>>>> pci12: <network, ethernet> at device 0.149 (no driver attached)
>>>>> pci12: <network, ethernet> at device 0.151 (no driver attached)
>>>>> pci12: <network, ethernet> at device 0.153 (no driver attached)
>>>>> pci12: <network, ethernet> at device 0.155 (no driver attached)
>>>>> pci12: <network, ethernet> at device 0.157 (no driver attached)
>>>>> pci12: <network, ethernet> at device 0.159 (no driver attached)
>>>>> pci12: <network, ethernet> at device 0.161 (no driver attached)
>>>>> pci12: <network, ethernet> at device 0.163 (no driver attached)
>>>>> pci12: <network, ethernet> at device 0.165 (no driver attached)
>>>>> pci12: <network, ethernet> at device 0.167 (no driver attached)
>>>>> pci12: <network, ethernet> at device 0.169 (no driver attached)
>>>>> pci12: <network, ethernet> at device 0.171 (no driver attached)
>>>>> pci12: <network, ethernet> at device 0.173 (no driver attached)
>>>>> pci12: <network, ethernet> at device 0.175 (no driver attached)
>>>>> pci12: <network, ethernet> at device 0.177 (no driver attached)
>>>>> pci12: <network, ethernet> at device 0.179 (no driver attached)
>>>>> pci12: <network, ethernet> at device 0.181 (no driver attached)
>>>>> pci12: <network, ethernet> at device 0.183 (no driver attached)
>>>>> pci12: <network, ethernet> at device 0.185 (no driver attached)
>>>>> pci12: <network, ethernet> at device 0.187 (no driver attached)
>>>>> pci12: <network, ethernet> at device 0.189 (no driver attached)
>>>>>
>>>>> pciconf -lv
>>>>> ix1 at pci0:129:0:1: class=0x020000 card=0x00001458 chip=0x15288086
>>>>> rev=0x01 hdr=0x00
>>>>> vendor = 'Intel Corporation'
>>>>> device = 'Ethernet Controller 10-Gigabit X540-AT2'
>>>>> class = network
>>>>> subclass = ethernet
>>>>> none155 at pci0:129:0:129: class=0x020000 card=0x00001458 chip=0x15158086
>>>>> rev=0x01 hdr=0x00
>>>>> vendor = 'Intel Corporation'
>>>>> device = 'X540 Ethernet Controller Virtual Function'
>>>>> class = network
>>>>> subclass = ethernet
>>>>> none156 at pci0:129:0:131: class=0x020000 card=0x00001458 chip=0x15158086
>>>>> rev=0x01 hdr=0x00
>>>>> vendor = 'Intel Corporation'
>>>>> device = 'X540 Ethernet Controller Virtual Function'
>>>>> class = network
>>>>> subclass = ethernet
>>>>> ppt0 at pci0:129:0:133: class=0x020000 card=0x00001458 chip=0x15158086
>>>>> rev=0x01 hdr=0x00
>>>>> vendor = 'Intel Corporation'
>>>>> device = 'X540 Ethernet Controller Virtual Function'
>>>>> class = network
>>>>> subclass = ethernet
>>>>> ppt1 at pci0:129:0:135: class=0x020000 card=0x00001458 chip=0x15158086
>>>>> rev=0x01 hdr=0x00
>>>>> vendor = 'Intel Corporation'
>>>>> device = 'X540 Ethernet Controller Virtual Function'
>>>>> class = network
>>>>> subclass = ethernet
>>>>> ppt2 at pci0:129:0:137: class=0x020000 card=0x00001458 chip=0x15158086
>>>>> rev=0x01 hdr=0x00
>>>>> vendor = 'Intel Corporation'
>>>>> device = 'X540 Ethernet Controller Virtual Function'
>>>>> class = network
>>>>> subclass = ethernet
>>>>> ppt3 at pci0:129:0:139: class=0x020000 card=0x00001458 chip=0x15158086
>>>>> rev=0x01 hdr=0x00
>>>>> vendor = 'Intel Corporation'
>>>>> device = 'X540 Ethernet Controller Virtual Function'
>>>>> class = network
>>>>> subclass = ethernet
>>>>> ppt4 at pci0:129:0:141: class=0x020000 card=0x00001458 chip=0x15158086
>>>>> rev=0x01 hdr=0x00
>>>>> vendor = 'Intel Corporation'
>>>>> device = 'X540 Ethernet Controller Virtual Function'
>>>>> class = network
>>>>> subclass = ethernet
>>>>> ppt5 at pci0:129:0:143: class=0x020000 card=0x00001458 chip=0x15158086
>>>>> rev=0x01 hdr=0x00
>>>>> vendor = 'Intel Corporation'
>>>>> device = 'X540 Ethernet Controller Virtual Function'
>>>>> class = network
>>>>> subclass = ethernet
>>>>> ppt6 at pci0:129:0:145: class=0x020000 card=0x00001458 chip=0x15158086
>>>>> rev=0x01 hdr=0x00
>>>>> vendor = 'Intel Corporation'
>>>>> device = 'X540 Ethernet Controller Virtual Function'
>>>>> class = network
>>>>> subclass = ethernet
>>>>> ppt7 at pci0:129:0:147: class=0x020000 card=0x00001458 chip=0x15158086
>>>>> rev=0x01 hdr=0x00
>>>>> vendor = 'Intel Corporation'
>>>>> device = 'X540 Ethernet Controller Virtual Function'
>>>>> class = network
>>>>> subclass = ethernet
>>>>> ppt8 at pci0:129:0:149: class=0x020000 card=0x00001458 chip=0x15158086
>>>>> rev=0x01 hdr=0x00
>>>>> vendor = 'Intel Corporation'
>>>>> device = 'X540 Ethernet Controller Virtual Function'
>>>>> class = network
>>>>> subclass = ethernet
>>>>> ppt9 at pci0:129:0:151: class=0x020000 card=0x00001458 chip=0x15158086
>>>>> rev=0x01 hdr=0x00
>>>>> vendor = 'Intel Corporation'
>>>>> device = 'X540 Ethernet Controller Virtual Function'
>>>>> class = network
>>>>> subclass = ethernet
>>>>> ppt10 at pci0:129:0:153: class=0x020000 card=0x00001458 chip=0x15158086
>>>>> rev=0x01 hdr=0x00
>>>>> vendor = 'Intel Corporation'
>>>>> device = 'X540 Ethernet Controller Virtual Function'
>>>>> class = network
>>>>> subclass = ethernet
>>>>> ppt11 at pci0:129:0:155: class=0x020000 card=0x00001458 chip=0x15158086
>>>>> rev=0x01 hdr=0x00
>>>>> vendor = 'Intel Corporation'
>>>>> device = 'X540 Ethernet Controller Virtual Function'
>>>>> class = network
>>>>> subclass = ethernet
>>>>> ppt12 at pci0:129:0:157: class=0x020000 card=0x00001458 chip=0x15158086
>>>>> rev=0x01 hdr=0x00
>>>>> vendor = 'Intel Corporation'
>>>>> device = 'X540 Ethernet Controller Virtual Function'
>>>>> class = network
>>>>> subclass = ethernet
>>>>> ppt13 at pci0:129:0:159: class=0x020000 card=0x00001458 chip=0x15158086
>>>>> rev=0x01 hdr=0x00
>>>>> vendor = 'Intel Corporation'
>>>>> device = 'X540 Ethernet Controller Virtual Function'
>>>>> class = network
>>>>> subclass = ethernet
>>>>> ppt14 at pci0:129:0:161: class=0x020000 card=0x00001458 chip=0x15158086
>>>>> rev=0x01 hdr=0x00
>>>>> vendor = 'Intel Corporation'
>>>>> device = 'X540 Ethernet Controller Virtual Function'
>>>>> class = network
>>>>> subclass = ethernet
>>>>> ppt15 at pci0:129:0:163: class=0x020000 card=0x00001458 chip=0x15158086
>>>>> rev=0x01 hdr=0x00
>>>>> vendor = 'Intel Corporation'
>>>>> device = 'X540 Ethernet Controller Virtual Function'
>>>>> class = network
>>>>> subclass = ethernet
>>>>> ppt16 at pci0:129:0:165: class=0x020000 card=0x00001458 chip=0x15158086
>>>>> rev=0x01 hdr=0x00
>>>>> vendor = 'Intel Corporation'
>>>>> device = 'X540 Ethernet Controller Virtual Function'
>>>>> class = network
>>>>> subclass = ethernet
>>>>> ppt17 at pci0:129:0:167: class=0x020000 card=0x00001458 chip=0x15158086
>>>>> rev=0x01 hdr=0x00
>>>>> vendor = 'Intel Corporation'
>>>>> device = 'X540 Ethernet Controller Virtual Function'
>>>>> class = network
>>>>> subclass = ethernet
>>>>> ppt18 at pci0:129:0:169: class=0x020000 card=0x00001458 chip=0x15158086
>>>>> rev=0x01 hdr=0x00
>>>>> vendor = 'Intel Corporation'
>>>>> device = 'X540 Ethernet Controller Virtual Function'
>>>>> class = network
>>>>> subclass = ethernet
>>>>> ppt19 at pci0:129:0:171: class=0x020000 card=0x00001458 chip=0x15158086
>>>>> rev=0x01 hdr=0x00
>>>>> vendor = 'Intel Corporation'
>>>>> device = 'X540 Ethernet Controller Virtual Function'
>>>>> class = network
>>>>> subclass = ethernet
>>>>> ppt20 at pci0:129:0:173: class=0x020000 card=0x00001458 chip=0x15158086
>>>>> rev=0x01 hdr=0x00
>>>>> vendor = 'Intel Corporation'
>>>>> device = 'X540 Ethernet Controller Virtual Function'
>>>>> class = network
>>>>> subclass = ethernet
>>>>> ppt21 at pci0:129:0:175: class=0x020000 card=0x00001458 chip=0x15158086
>>>>> rev=0x01 hdr=0x00
>>>>> vendor = 'Intel Corporation'
>>>>> device = 'X540 Ethernet Controller Virtual Function'
>>>>> class = network
>>>>> subclass = ethernet
>>>>> ppt22 at pci0:129:0:177: class=0x020000 card=0x00001458 chip=0x15158086
>>>>> rev=0x01 hdr=0x00
>>>>> vendor = 'Intel Corporation'
>>>>> device = 'X540 Ethernet Controller Virtual Function'
>>>>> class = network
>>>>> subclass = ethernet
>>>>> ppt23 at pci0:129:0:179: class=0x020000 card=0x00001458 chip=0x15158086
>>>>> rev=0x01 hdr=0x00
>>>>> vendor = 'Intel Corporation'
>>>>> device = 'X540 Ethernet Controller Virtual Function'
>>>>> class = network
>>>>> subclass = ethernet
>>>>> ppt24 at pci0:129:0:181: class=0x020000 card=0x00001458 chip=0x15158086
>>>>> rev=0x01 hdr=0x00
>>>>> vendor = 'Intel Corporation'
>>>>> device = 'X540 Ethernet Controller Virtual Function'
>>>>> class = network
>>>>> subclass = ethernet
>>>>> ppt25 at pci0:129:0:183: class=0x020000 card=0x00001458 chip=0x15158086
>>>>> rev=0x01 hdr=0x00
>>>>> vendor = 'Intel Corporation'
>>>>> device = 'X540 Ethernet Controller Virtual Function'
>>>>> class = network
>>>>> subclass = ethernet
>>>>> ppt26 at pci0:129:0:185: class=0x020000 card=0x00001458 chip=0x15158086
>>>>> rev=0x01 hdr=0x00
>>>>> vendor = 'Intel Corporation'
>>>>> device = 'X540 Ethernet Controller Virtual Function'
>>>>> class = network
>>>>> subclass = ethernet
>>>>> ppt27 at pci0:129:0:187: class=0x020000 card=0x00001458 chip=0x15158086
>>>>> rev=0x01 hdr=0x00
>>>>> vendor = 'Intel Corporation'
>>>>> device = 'X540 Ethernet Controller Virtual Function'
>>>>> class = network
>>>>> subclass = ethernet
>>>>> ppt28 at pci0:129:0:189: class=0x020000 card=0x00001458 chip=0x15158086
>>>>> rev=0x01 hdr=0x00
>>>>> vendor = 'Intel Corporation'
>>>>> device = 'X540 Ethernet Controller Virtual Function'
>>>>> class = network
>>>>> subclass = ethernet
>>>>>
>>>>> Ultima
>>>>> _______________________________________________
>>>>> freebsd-current at freebsd.org mailing list
>>>>> https://lists.freebsd.org/mailman/listinfo/freebsd-current
>>>>> To unsubscribe, send any mail to "
>>>>>
>>>> freebsd-current-unsubscribe at freebsd.org
>>>
>>>> "
>>>>>
>>>>> _______________________________________________
>>> freebsd-current at freebsd.org mailing list
>>> https://lists.freebsd.org/mailman/listinfo/freebsd-current
>>> To unsubscribe, send any mail to "
>>> freebsd-current-unsubscribe at freebsd.org"
>>>
>>> _______________________________________________
>> freebsd-current at freebsd.org mailing list
>> https://lists.freebsd.org/mailman/listinfo/freebsd-current
>> To unsubscribe, send any mail to "freebsd-current-unsubscribe at freebsd.org
>> "
>>
>
> _______________________________________________
> freebsd-current at freebsd.org mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-current
> To unsubscribe, send any mail to "freebsd-current-unsubscribe at freebsd.org"
>
More information about the freebsd-virtualization
mailing list