Bogus "igb1: Could not setup receive structures" in 8-STABLE
Terry Kennedy
TERRY at tmk.com
Thu Oct 14 23:24:18 UTC 2010
I've run across a strange problem with the igb driver in 8-STABLE - when
I try to do anything with the second igb interface, I get one or more "igb1:
Could not setup receive structures" error messages.
This can be reproduced as simply as booting in single-user mode with an
empty /boot/loader.conf and doing:
ifconfig igb0 up
ifconfig igb1 up
I've tried to track this down, and as far as I can see, this is from some
change introduced between 8.1-RELEASE (igb 1.9.5) and the current 8-STABLE
(igb 2.0.1). When I try it when booting from the 8.1-RELEASE amd64 DVD, I
can bring up both interfaces. When I try it with an 8-STABLE kernel, I get
the error and igb1 is missing the "RUNNING" flag:
igb0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 9000
options=1bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4>
ether 00:25:90:xx:xx:bc
inet6 fe80::225:90ff:fe02:xxbc%igb0 prefixlen 64 scopeid 0x1
nd6 options=3<PERFORMNUD,ACCEPT_RTADV>
media: Ethernet 1000baseT <full-duplex>
status: active
igb1: flags=8803<UP,BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 9000
options=1bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4>
ether 00:25:90:xx:xx:bd
inet6 fe80::225:90ff:fe02:xxbd%igb1 prefixlen 64 tentative scopeid 0x2
nd6 options=3<PERFORMNUD,ACCEPT_RTADV>
media: Ethernet 1000baseT <full-duplex>
status: active
I see 3 mbuf_jumbo_page allocation failures from:
(1:20) rz1m:/sys/dev/e1000# vmstat -z | grep -v 0\$
ITEM SIZE LIMIT USED FREE REQUESTS FAILURES
64 Bucket: 536, 0, 263, 3, 263, 106
128 Bucket: 1048, 0, 523, 2, 525, 139
mbuf_jumbo_page: 4096, 12800, 12307, 493, 30343, 3
which correspond to the 3 "igb1: Could not setup receive structures" mess-
ages. If I try another "ifconfig igb1 up", I get another console message and
the counter goes to 4. If I bump the kern.ipc.nmbjumbop sysctl to a larger
value, like 15000, I get the same error message when trying to work with the
igb1 device, so I don't think it is a "real" error but indicates a problem
in the driver.
This is on a Supermicro X8DTH-iF, BIOS 2.0a (latest) with a dual on-board
82576. The dev.igb sysctl's for the two ports (excluding 0 values) are at-
tached. Note that most of the igb1 values are zero:
(0:34) rz1m:/sys/dev/e1000# sysctl -a | grep igb.1 | grep -v ": 0"
dev.igb.1.%desc: Intel(R) PRO/1000 Network Connection version - 2.0.1
dev.igb.1.%driver: igb
dev.igb.1.%location: slot=0 function=1
dev.igb.1.%pnpinfo: vendor=0x8086 device=0x10c9 subvendor=0x15d9 subdevice=0x0400 class=0x020000
dev.igb.1.%parent: pci1
dev.igb.1.nvm: -1
dev.igb.1.flow_control: 3
dev.igb.1.enable_aim: 1
dev.igb.1.rx_processing_limit: 100
dev.igb.1.link_irq: 1
dev.igb.1.device_control: 13632065
dev.igb.1.extended_int_mask: 2147483648
dev.igb.1.fc_high_water: 47488
dev.igb.1.fc_low_water: 47472
(0:35) rz1m:/sys/dev/e1000# sysctl -a | grep igb.0 | grep -v ": 0"
dev.igb.0.%desc: Intel(R) PRO/1000 Network Connection version - 2.0.1
dev.igb.0.%driver: igb
dev.igb.0.%location: slot=0 function=0
dev.igb.0.%pnpinfo: vendor=0x8086 device=0x10c9 subvendor=0x15d9 subdevice=0x0400 class=0x020000
dev.igb.0.%parent: pci1
dev.igb.0.nvm: -1
dev.igb.0.flow_control: 3
dev.igb.0.enable_aim: 1
dev.igb.0.rx_processing_limit: 100
dev.igb.0.link_irq: 4
dev.igb.0.device_control: 1087373889
dev.igb.0.rx_control: 67338274
dev.igb.0.interrupt_mask: 4
dev.igb.0.extended_int_mask: 2147484671
dev.igb.0.fc_high_water: 47488
dev.igb.0.fc_low_water: 47472
dev.igb.0.queue0.txd_head: 823
dev.igb.0.queue0.txd_tail: 823
dev.igb.0.queue0.tx_packets: 1402
dev.igb.0.queue0.rxd_head: 319
dev.igb.0.queue0.rxd_tail: 318
dev.igb.0.queue0.rx_packets: 319
dev.igb.0.queue0.rx_bytes: 69075
dev.igb.0.queue1.txd_head: 2
dev.igb.0.queue1.txd_tail: 2
dev.igb.0.queue1.tx_packets: 1
dev.igb.0.queue1.rxd_head: 52
dev.igb.0.queue1.rxd_tail: 51
dev.igb.0.queue1.rx_packets: 52
dev.igb.0.queue1.rx_bytes: 6369
dev.igb.0.queue2.txd_head: 27
dev.igb.0.queue2.txd_tail: 27
dev.igb.0.queue2.tx_packets: 13
dev.igb.0.queue2.rxd_head: 72
dev.igb.0.queue2.rxd_tail: 71
dev.igb.0.queue2.rx_packets: 72
dev.igb.0.queue2.rx_bytes: 8789
dev.igb.0.queue3.txd_head: 177
dev.igb.0.queue3.txd_tail: 177
dev.igb.0.queue3.tx_packets: 64
dev.igb.0.queue3.rxd_head: 88
dev.igb.0.queue3.rxd_tail: 87
dev.igb.0.queue3.rx_packets: 88
dev.igb.0.queue3.rx_bytes: 16454
dev.igb.0.queue4.rxd_head: 21
dev.igb.0.queue4.rxd_tail: 20
dev.igb.0.queue4.rx_packets: 21
dev.igb.0.queue4.rx_bytes: 3547
dev.igb.0.queue5.txd_head: 19
dev.igb.0.queue5.txd_tail: 19
dev.igb.0.queue5.tx_packets: 9
dev.igb.0.queue5.rxd_head: 120
dev.igb.0.queue5.rxd_tail: 119
dev.igb.0.queue5.rx_packets: 120
dev.igb.0.queue5.rx_bytes: 35518
dev.igb.0.queue6.txd_head: 21
dev.igb.0.queue6.txd_tail: 21
dev.igb.0.queue6.tx_packets: 10
dev.igb.0.queue6.rxd_head: 21
dev.igb.0.queue6.rxd_tail: 20
dev.igb.0.queue6.rx_packets: 21
dev.igb.0.queue6.rx_bytes: 2876
dev.igb.0.queue7.txd_head: 100
dev.igb.0.queue7.txd_tail: 100
dev.igb.0.queue7.tx_packets: 50
dev.igb.0.queue7.rxd_head: 74
dev.igb.0.queue7.rxd_tail: 73
dev.igb.0.queue7.rx_packets: 1098
dev.igb.0.queue7.rx_bytes: 116918
dev.igb.0.queue8.txd_head: 11
dev.igb.0.queue8.txd_tail: 11
dev.igb.0.queue8.tx_packets: 6
dev.igb.0.queue8.rxd_head: 25
dev.igb.0.queue8.rxd_tail: 24
dev.igb.0.queue8.rx_packets: 25
dev.igb.0.queue8.rx_bytes: 3698
dev.igb.0.mac_stats.total_pkts_recvd: 3138
dev.igb.0.mac_stats.good_pkts_recvd: 1815
dev.igb.0.mac_stats.bcast_pkts_recvd: 383
dev.igb.0.mac_stats.mcast_pkts_recvd: 114
dev.igb.0.mac_stats.rx_frames_64: 217
dev.igb.0.mac_stats.rx_frames_65_127: 673
dev.igb.0.mac_stats.rx_frames_128_255: 779
dev.igb.0.mac_stats.rx_frames_256_511: 61
dev.igb.0.mac_stats.rx_frames_512_1023: 46
dev.igb.0.mac_stats.rx_frames_1024_1522: 39
dev.igb.0.mac_stats.good_octets_recvd: 270378
dev.igb.0.mac_stats.good_octets_txd: 252216
dev.igb.0.mac_stats.total_pkts_txd: 1554
dev.igb.0.mac_stats.good_pkts_txd: 1554
dev.igb.0.mac_stats.bcast_pkts_txd: 36
dev.igb.0.mac_stats.mcast_pkts_txd: 65
dev.igb.0.mac_stats.tx_frames_64: 32
dev.igb.0.mac_stats.tx_frames_65_127: 219
dev.igb.0.mac_stats.tx_frames_128_255: 1222
dev.igb.0.mac_stats.tx_frames_256_511: 55
dev.igb.0.mac_stats.tx_frames_512_1023: 19
dev.igb.0.mac_stats.tx_frames_1024_1522: 7
dev.igb.0.interrupts.asserts: 3350
dev.igb.0.interrupts.rx_pkt_timer: 1815
dev.igb.0.interrupts.tx_abs_timer: 1815
dev.igb.0.interrupts.tx_queue_empty: 1554
dev.igb.0.host.rx_good_bytes: 270378
dev.igb.0.host.tx_good_bytes: 252216
Both ports are cabled to the same Cisco switch. If I swap the cables at
the FreeBSD end between igb0 and igb1, the problem stays with igb1 so I
don't think it is the switch.
I have 3 identical systems, all of which exhibit this same issue. Un-
fortunately, I don't have any other hardware with dual igb's.
I can give a developer root access as well as a web-based remote console
if needed to track this down.
Terry Kennedy http://www.tmk.com
terry at tmk.com New York, NY USA
More information about the freebsd-stable
mailing list